Most AI projects don’t fail because the model is bad.
They fail because the infrastructure around the model was picked backwards. A vector database when prompt caching would do. Three agent frameworks when no evals exist. A gateway added after the bill explodes.
I spent the last few months building Lattice — a curated index of 112 tools behind working AI systems. Not 50,000 tools. 112 that actually run in production, organized as a production stack from substrate to surface:
1. Inference Serving
2. Routing & Gateways
3. Retrieval & Vector Stores
4. Fine-tuning
5. Agent Frameworks
6. Workflow Orchestration
7. Guardrails
8. Prompt Engineering
9. Evaluation & Observability
The order is the argument. You can’t tune weights before you can serve them. You can’t evaluate what you can’t observe.
Here is what that work taught me.
check it out at : https://lattice.kkshah2005.workers.dev/
1. Start from the symptom, not the category
Nobody wakes up thinking “I need a vector database.”
They think:
- Why is my app so slow?
- Why is this costing $4,000/month?
- Why is RAG giving wrong answers?
- Why does my agent work in demo and fail in prod?
- How do I know if my prompt change helped?
Every one of those maps to a different layer. Slow = inference + routing. Expensive = routing + caching + model choice. Wrong answers = retrieval + data quality. Unreliable agents = workflows + tracing + guardrails. Can’t measure = evals.
If you start from the category, you shop. If you start from the symptom, you diagnose.
2. Every tool needs a “when NOT to use it”
This is the sentence most directories skip.
“Best for X” is marketing. “Skip this when Y” is engineering.
Examples from the index:
- Don’t self-host vLLM if you don’t control GPUs and on-call. Rent the API.
- Don’t add pgvector if you have <50k chunks and no metadata filters. You don’t need a dedicated vector store yet.
- Don’t adopt LangChain for a single linear chain. You are buying an abstraction you won’t use.
- Don’t build fine-tuning before you’ve tried retrieval + 20 eval cases. 70% of “model is dumb” is retrieval, not weights.
I wrote a use-when and skip-when for all 112 entries. The second half is the one engineers actually thank you for.
3. The 9 layers collapse to 3 bands
Nine colors is decoration. Nobody holds nine things in working memory.
In practice:
Band I — Compute (Layers 1–2): too slow, too expensive. Owned by platform/infra.
Band II — State (Layers 3–4): wrong answers. Owned by data/ML.
Band III — Control (Layers 5–9): unreliable, unmeasured. Owned by whoever ships the feature.
Band III is 5 of 9 layers. That’s not an accident — that’s where the market concentrated in the last 18 months. Gateways, agents, workflows, guardrails, prompts, evals. Differentiation lives there now.
If you’re a lead deciding roadmap: fix Band I first (it pages you), Band II second (it lies to users), Band III always (it’s never done).
4. Freshness matters more than coverage
A directory with stale prices and licences is worse than no directory.
Licence changed from Apache-2.0 to proprietary? Cost went from free to $0.60/M tokens? Deployment went from self-host to API-only? Those three facts decide architecture. If they’re six months old, they’re misleading.
My rule: if licence/cost/deployment hasn’t been re-checked in 6 months, the build should fail. Not warn. Fail. The site shouldn’t deploy with stale figures in it.
You don’t need to implement it that strictly, but date-stamp your decisions. “We chose X in Sept 2026 because Y cost Z” beats “we chose X” every time you revisit it in 6 months.
5. Cross-layer decisions beat within-layer shootouts
vLLM vs SGLang vs TGI is useful. But the decisions that save months are cross-layer:
- Gateway vs evals: what to build first?
- Retrieval vs fine-tuning vs prompt optimization?
- Gateway vs guardrails vs evals?
Vendors will never publish those. Langfuse won’t tell you to buy evals before a gateway. A gateway vendor won’t tell you prompt caching beats migrating models. Only someone who sells neither can be the tiebreaker.
When you’re choosing, ask the cross-layer question first. Within-layer is optimization. Cross-layer is strategy.
6. What I’d tell a solo builder vs a platform team
Same index, different door:
Solo builder: pick the boring managed thing. One model behind an API, one managed vector store, hosted evals. You need sleep more than control.
Platform engineer: own routing, orchestration, and evals. Those three decide everyone else’s velocity.
Tech lead: write your comparison as a decision record. Tools considered, verdict, what would change it. Future-you will thank you.
EM: ask build/buy/wait in people-cost, not tool-cost. Self-hosting is never free — it’s an on-call rotation.
Where this is wrong
No index of this kind is complete. Coverage is uneven by construction. Layer boundaries are simplifications. Some role assignments are judgement calls. Licence and cost are vendor-reported and change fast.
That’s why methodology and limitations should be published alongside recommendations — not in a footnote.
If you maintain one of these 112 tools and something is wrong, corrections are the feature, not the embarrassment.
What I’m building next: symptom checklists (slow, expensive, wrong, unreliable, unmeasured), head-to-head comparisons that end in a verdict, and alternatives pages for every tool.
If you’ve shipped RAG or agents to prod, I’d love to hear: which layer burned you the most?
Follow for the next essay: Why observability is not logging.
Safety note: This draft contains zero internal details — no competitor names/analysis, no traffic splits, no launch/HN/Reddit/newsletter tactics, no citation queries, no monetisation/revenue logic, no infra/bot/logging internals, no file paths or repo structure. Safe to paste directly to Medium.
I Looked At 112 AI Tools So You Don’t Have To. Here’s What Actually Breaks in Prod.
I didn’t set out to build an index. I was just tired of picking the wrong tool.
Last year I watched a team — good team — spend three weeks tuning a vector database. Turned out they had 12,000 chunks. They didn’t need a vector database. They needed 20 eval cases and better chunking. That one stung, because I’ve done the same thing myself.
So I started keeping notes. That notes file turned into Lattice, my little curated index of 112 tools behind AI systems that actually work. I organized them the way they break in real life, bottom to top:
Inference, routing, retrieval, fine-tuning, agents, workflows, guardrails, prompts, evals.
Not because it’s pretty. Because that’s the dependency order. You can’t fix what you can’t serve, and you can’t evaluate what you can’t see.
Here’s what I learned doing it.
Nobody arrives with “I need a vector DB”
Real life sounds more like this:
“Why is this so damn slow?”
“Why did we just get a $3,800 bill?”
“Why is it confidently making stuff up?”
“Worked in the demo. Falls over with 10 users. Why?”
I stopped organizing by category after a while and started organizing by symptom. Slow is usually inference + routing. Expensive is almost always routing + no caching + wrong model choice. Wrong answers? That’s retrieval 7 times out of 10, not the model.
Shop by category if you want to browse. Diagnose by symptom if you want to ship.
The most useful sentence is “don’t use this when…”
Every directory will tell you “best for X.” Almost none will tell you when to walk away.
That second half is the whole game though. A few I wrote down and still stand by:
I love vLLM. I would not self-host it if I didn’t have GPUs and someone on call. Just pay for the API.
pgvector is great. If you have under ~50k chunks and no weird filters? You don’t need a dedicated vector store yet. You’re buying ops work.
LangChain? Fine for messy agents. For one linear chain? It’s overhead. I’ve ripped it out twice.
And fine-tuning — look, I get the urge. But I’ve seen so many teams jump to fine-tuning when their retrieval was just bad. Try fixing retrieval and writing 20 golden evals first. It’s boring. It works.
Every one of the 112 entries in my index has both sides now. Use when, skip when. Engineers only ever message me about the second one.
I had 9 layers. I use 3.
Nine colors looks nice in a diagram. You can’t hold nine things in your head at 2am during an incident.
It really collapses to:
1. Compute — is it slow or expensive? That’s inference + routing. Platform folks live here.
2. State — is it wrong? Retrieval + fine-tuning. Data folks live here.
3. Control — is it flaky or unmeasurable? That’s everything else — agents, workflows, guardrails, prompts, evals. That’s… 5 out of 9 layers. Tells you where all the tooling money went in the last 18 months.
If you’re leading a team, that order is your roadmap. Fix compute because it pages you. Fix state because it lies to users. Then live in control forever, because that part is never really “done.”
Stale data is worse than no data
This one bit me.
I had a cost figure that was four months old. Off by 3x. Someone made a build-vs-buy call off it. My fault.
Now I’m obsessive about it: licence, price, deployment — if I haven’t re-checked it recently, I don’t trust it, and I put a date on it. If it’s old, my build literally refuses to ship. Extreme? Maybe. But I’d rather have no page than a confident wrong page.
You should do the lightweight version: write “decided Sept 2026 because X cost Y” in your decision doc. Future-you in March will be grateful.
The cross-layer question saves months
vLLM vs SGLang matters. But you know what matters more?
“Should we build a gateway first, or evals first?”
“Should we do RAG, fine-tune, or just fix the prompt?”
No vendor will answer that honestly for you. Of course Langfuse thinks you need observability first. Of course a gateway vendor thinks you need a gateway first. They’re not lying, they’re just… selling one of the options.
You need someone — or some doc — with no skin in the game to say “do evals first, gateway second.” That’s what I try to be with cross-layer comparisons.
Ask the cross-layer question before the within-layer shootout. Within-layer is optimization. Cross-layer is strategy.
Quick, opinionated advice
If you’re solo: pick boring and managed. One model API, one hosted store, hosted evals. Sleep > control. I mean it.
If you’re platform: own routing, durable execution, and evals. Those three decide everyone else’s speed.
If you’re a tech lead: write down what you considered and what would change your mind. One paragraph. You’ll forget in 3 months otherwise — I always do.
If you’re managing: self-hosting is never free. It’s an on-call rotation wearing a cost-saving costume.
Where I’m probably wrong
This index is uneven. Some layers I know cold, some I’m still learning. Layer lines are fuzzy in real systems. Licence/pricing changes faster than anyone can track.
I publish what I don’t know alongside what I think I know. If you maintain one of these tools and I got it wrong — tell me. Corrections have fixed like a third of my early calls. That’s a feature.
I’m doing symptom checklists and head-to-heads next — slow, expensive, wrong, flaky, unmeasurable. With actual verdicts, not “it depends” hand-waving.
Curious: what layer burned you worst? For me it was retrieval. Twice.