RAG in Production for Platform Teams

This is for you if

• You've been asked to put an AI assistant on your team's runbooks or wiki and make it production-ready
• A RAG demo works on ten questions and you need it reliable, secure and measurable
• You're a platform engineer or SRE moving toward AI engineering or ML platform roles
• You want senior-level answers to the RAG questions interviewers now ask

What's inside

• A complete, tested Python RAG service: FastAPI, pgvector, Ollama and the OpenAI-compatible API
• Heading-aware chunking for runbooks, with overlap and code blocks kept intact
• Hybrid search with Reciprocal Rank Fusion in SQL and in Qdrant, plus cross-encoder reranking
• Why permission filters silently eat results in HNSW, and the pgvector iterative-scan fix
• Document-level permissions enforced inside retrieval, with an automated leak test
• Incremental ingestion that handles edits, deletions, ACL changes and the empty-mount trap
• An evaluation harness for recall@k, MRR, faithfulness and refusals, gated in GitHub Actions
• OpenTelemetry GenAI tracing and a full kind deployment with CloudNativePG, Ollama and Jaeger
• 14 hands-on labs, a troubleshooting playbook, 24 interview Q&As and a 30-day plan

After this guide you can

• Design and build a RAG service that cites sources and refuses when it doesn't know
• Choose between pgvector and a dedicated vector database, and tune HNSW for filtered search
• Prove that restricted documents never reach the wrong user, on every pull request
• Measure retrieval and answer quality, and block regressions in CI
• Trace, deploy and troubleshoot the whole stack on Kubernetes

Contents

1. The mental modelp. 4
2. Chunking and embeddingsp. 9
3. Vector stores: pgvector and Qdrantp. 22
4. Hybrid search and rerankingp. 31
5. Building the RAG servicep. 40
6. Permissions and multi-tenancyp. 47
7. Ingestion and freshnessp. 52
8. Evaluating RAGp. 55
9. Observability and deploymentp. 62
10. Troubleshooting playbookp. 74
11. Production checklist and design decisionsp. 77
12. Interview questionsp. 79
13. Cheat sheet and 30-day planp. 88