Search for a command to run...
No comments yet. Be the first to comment.
Privacy isn't dead. It's just being rebuilt. The old internet promised us "move fast and break things." We moved fast. We broke things. Now we're realizing what we broke: privacy. Every LLM you use wa
Coding is changing faster than you think. Not just "AI helps with autocomplete." AI is becoming the primary developer. You become the reviewer. The Skill Inversion Old model (2020): Human writes code.
Single AI agents are limited. They make mistakes. They miss context. They take wrong paths. Multi-agent systems don't have these problems. They have different problems. But better ones. The Single Age
You've learned basic RAG. You embed documents. You retrieve them. You feed them to an LLM. It generates an answer. Then you deploy it. And it breaks. Because the real world isn't simple retrieval. Adv
The biggest AI models aren't winning anymore. GPT-4: 1.8 trillion parameters Claude 3 Opus: 200+ billion parameters Claude 3.5 Sonnet: Unknown but smaller Llama 3.1: 405 billion parameters PhiMoe (ope
Muhammad Zulqarnain | Full Stack AI Engineer & Geospatial Developer
16 posts
A blog by Muhammad Zulqarnain — Full Stack AI Engineer & Geospatial Developer based in Turku, Finland. I write about RAG systems, LLMs, Prompt Engineering, Next.js, TypeScript, and geospatial development. Practical insights, deep dives, and real-world AI solutions.
Most RAG tutorials are lies. Not intentionally. But they show you a demo that works in a Jupyter notebook, and reality is very different.
I've built 7 production RAG systems. Here's what actually matters:
1. Precision vs Recall Trade-off
Precision: % of retrieved documents that are relevant Recall: % of relevant documents you actually retrieve
Theory says you want both high. Reality: Pick one.
In my lending underwriting system:
High precision setup: 91% precision, 42% recall
Users trust results
Miss important edge cases
High recall setup: 64% precision, 88% recall
Catch edge cases
Users distrust noisy results
We went with 76% precision / 72% recall. Balanced. Imperfect. Works.
2. Latency Under Load
Your 200ms response time in testing becomes 2000ms in production with 100 concurrent users.
Vector DB latency compounds:
Query embedding: 50ms
Vector search: 120ms
Retrieval: 30ms
LLM reranking: 80ms
LLM generation: 800ms
Total: 1080ms. Your 200ms target assumed no users.
3. The Staleness Problem (Nobody Talks About This)
Your RAG system uses documents from yesterday. A client updates their policy today. Your system gives outdated information.
I've seen companies lose users because of this.
Solution: Update vectors when source documents change. But this gets complicated with 500K+ documents.
We built an async vector update system that updates changed documents within 5 minutes. Cost: \(2,000/month in infrastructure. Saves maybe \)50K/month in support burden.
Building RAG:
Embedding model (fine-tuning): $5-20K
Vector DB (initial): $0-5K
Integration/testing: 80-120 hours
Total: $15-30K
Running RAG (monthly):
Embeddings API: $500-2K
Vector DB: $500-5K
LLM calls: $2K-10K
Monitoring/updates: $1K-3K
Total: $4K-20K/month
Most companies underestimate operational costs by 5x.
Reranking > Raw retrieval quality
Retrieving 50 documents and reranking them with an LLM beats retrieving 5 perfect documents
Reranking costs $0.001-0.01 per query
Massive ROI
Query expansion matters more than I thought
User asks: "Can I return items?"
Expanded: "Can I return items? What's the return policy? Return procedures? Refunds?"
Gets 40% better retrieval
Costs: 20ms per query
Small custom embedding beats large generic embedding
768-dim domain-specific > 1536-dim OpenAI generic
90% of the quality at 30% of the cost
Monitor embedding staleness
Track: "How old are the embeddings for this document?"
Alert: If embeddings > 7 days old
Regenerate weekly
Set up A/B testing from day one
Test precision vs recall
Test reranking on/off
Test different embedding models
You'll change your strategy
Plan for failure modes
Vector DB down? Fall back to keyword search
Embedding service down? Cache embeddings
LLM down? Return raw documents
Have a graceful degradation strategy
User feedback is gold
Track: "Was this retrieval helpful?"
Build dataset from "No" answers
Fine-tune on that data every month
Your system gets better over time
Success metric: User satisfaction with retrieved documents
First deployment: 58% satisfied After embedding tuning: 71% satisfied
After reranking: 82% satisfied After user feedback loop (3 months): 87% satisfied
Took 6 months to get good. Still improving.
RAG is not "load documents, embed them, done."
It's: monitoring, updating, A/B testing, reranking, feedback loops, and constant iteration.
The systems that work treat RAG as a living, evolving product. Not a static pipeline.
Do that, and you'll build RAG that actually serves users well.cs From Production
