N.V.J.K KartikKartik

Technical writing · 28 posts on DEV and Medium

Most of them start with the bug.

September 2026

  1. My CI evals were green. A regression still paged me at 3 AM.The eval that finally caught a bad regression for me was not the one in CI. CI had run twelve hours earlier, against a frozen test set, and passed. The regression was sitting in…Medium · 3 min
  2. LlamaIndex makes RAG easy to build and hard to debug. Here is how I evaluate it.I built a RAG app on LlamaIndex in about four lines. Wire an index to a query engine, point it at my...DEV · 6 min
  3. Why Your Agent Passes Every Eval and Still Fails in ProductionA curated eval set is a snapshot. Production is a river. Six ways the snapshot ages the day it ships, and the loop that stops the offline-pass-production-fail gap from being…Medium · 8 min

August 2026

  1. My DSPy pipeline compiled beautifully and got worse in productionI love the DSPy idea. You stop hand-editing prompts and let the compiler optimize them for you...DEV · 3 min
  2. When Your SWE-bench Run Dies, It Is Almost Never Your PatchesThe SWE-bench harness builds a tall stack of Docker images and runs each task in its own container. That is what makes the scores reproducible, and it is where every first-run…Medium · 4 min
  3. AWS Bedrock's built-in eval graded my agent green. It only looks at a quarter of it.I built an agent on AWS Bedrock the low-code way. Pick a foundation model, wire an action group to a...DEV · 5 min
  4. I uninstalled three AI coding CLIs in a week. The model was never the problem.I tried a bunch of terminal coding agents last month. Some I reopened every morning. Some I...DEV · 4 min
  5. Before You Pay for a Bigger Model, Audit the Harness Around ItTwo teams ran the same model on the same coding benchmark. One finished 63 percent of tasks, the other 76. The only difference was the software wrapped around the model, and that…Medium · 8 min
  6. Choosing the Right Voice Agent Testing Platform: 14 Questions to AskEvery voice agent I have shipped demoed perfectly. Then a real caller talked over it, changed their...DEV · 5 min
  7. I Built an Agent Harness by Hand. The Loop Was Tiny. The Five Bugs Were Not.An agent harness is a while-loop, a dispatcher, and a growing message list. The loop you can read in one sitting. The bugs in how you hand tool results back are what actually cost…Medium · 7 min
  8. Every agent on my team passed its own tests. The team still shipped wrong answers.I had a three-agent setup: a planner, a researcher, and a critic passing work between them. I graded...DEV · 4 min
  9. The Prototype Took an Afternoon. The Production Version Took a Quarter.Shipping a working demo on a frontier model is easy. Shipping the same workflow as a reliable, observable, audited system is a different job, and in 2026 that job is mostly…Medium · 6 min
  10. My PDF QA Bot Cited Page 47 of a 30-Page Contract. The Retriever Was the Bug.A PDF QA chatbot is a retrieval problem, not a generation one. If the retriever surfaces the wrong chunk, a perfectly grounded answer is still wrong, and your generation scores…Medium · 8 min
  11. How I vet MCP servers before trusting them in my agentI plugged an MCP server into our agent to give it a new lookup tool. It worked fine in testing. What...DEV · 3 min
  12. Every dashboard was green while my agent made things up. Here is how I debugged it.A user asked our support agent how to reset two-factor auth, and it confidently walked them through...DEV · 4 min
  13. How I Rebuilt My LLM Eval Stack From the Floor Up, Part 2: Taking It DistributedPart 1 built the deterministic floor. The deterministic floor works until your dataset outgrows one machine. Here are the four runners I chose between, and how to pick by…Medium · 8 min
  14. My LLM app was fully traced. During an incident the trace was still useless.A regression came in for our German enterprise users on the support agent. Quality had dropped for...DEV · 5 min
  15. How I Rebuilt My LLM Eval Stack From the Floor Up, Part 1: The Deterministic ChecksThis is the first post in a series on how I rebuilt my eval stack after a judge-only setup cost me a five-figure bill and still missed the obvious bug. Part 1 is the floor: the…Medium · 8 min

July 2026

  1. My agent passed every check and still broke production in an hour. Here's the CI/CD I run now.A while back my team merged what looked like a harmless change: a small refactor to our agent's...DEV · 4 min
  2. Stop Optimizing Only the Prompt: An Agent Has Five AxesAn agent has five optimizable surfaces, not one. Optimizing only the prompt captures roughly a fifth of the headroom, and the four axes most teams skip are usually the ones…Medium · 10 min
  3. How I generate LLM test cases that actually catch bugsI got tired of hand-writing test cases for our agent, so I did the obvious 2026 thing: I pointed an...DEV · 6 min
  4. Evaluating an intent classifier: what I check beyond accuracyI built the intent router that sits at the front of our support agent. It reads what a user typed and...DEV · 5 min
  5. Evaluating LangGraph agents: the four things I score per stepI had a LangGraph agent running in production and a regression set of fifty queries it passed...DEV · 5 min
  6. I picked a coding agent off a leaderboard. It flopped on our codebase.Last year my team had to pick a coding agent, and I volunteered to run the evaluation. I felt good...DEV · 5 min
  7. I built an LLM eval framework from scratch. Here is what I wish I had bought instead.One weekend I wrote an LLM eval framework in about two hundred lines of Python. It demoed...DEV · 6 min

June 2026

  1. How I caught the voice-agent failures my eval dashboard kept missingI was working on a retail support voice agent, and on paper it looked great. Transcription quality,...DEV · 5 min
  2. How I A/B test LLM prompts without fooling myselfA while back I was building a support assistant and hit a simple-sounding question: is this new...DEV · 5 min
  3. How I set up RAG evals in CI/CD so they actually catch regressionsI have hit this a few times.A PR lands late in the day, the RAG eval runs in under a minute, green...DEV · 6 min

A snapshot of every DEV post and Medium’s latest ten, taken 2026-09-25. Older Medium posts are on Medium.