N.V.J.K KartikKartik

Platform engineering · Future AGI · 2026 · Go · Python · TypeScript

Agent Command Center

An open-source LLM gateway, the Portkey or Bifrost kind: one OpenAI-compatible API in front of every model provider, and the client SDKs people call it with.

Plate: one request fanning out to several model providers, with a server tool kept byte for byte.

What I did

I published v1.0.0 of the client SDKs and opened their public repository: an OpenAI-compatible Python client, a TypeScript client (ESM and CJS), and packages for LangChain.js, LlamaIndex.TS, React chat UIs and the Vercel AI SDK.

I wrote the gateway’s cross-format translation, so a request in OpenAI, Anthropic or Gemini format can reach any configured provider.

In the gateway, I fixed Claude’s server tools, such as web search, being silently dropped when callers used the OpenAI-format endpoint. Tools that aren’t plain functions now keep the caller’s original bytes and replay them, so they round-trip on every provider path.

I gave the gateway an OTLP/HTTP exporter, so a self-hosted gateway can send its traces to any collector, with prompts and completions on the spans, redacted before they’re truncated.

I found that the Anthropic-format and Gemini-format endpoints never ran the gateway’s plugins, so that traffic had no trace, no cost and no budget check. Every billable endpoint runs them now, and a test fails if one stops.

SDK 1.0, PR #1 ↗Server tools fix, PR #2302 ↗OTLP export, PR #2186 ↗Plugins on every endpoint, PR #2231 ↗

Who I made it with

nik13 wrote the first version of the gateway and of these SDKs, and azain-commits built its Django integration. The gateway is Future AGI team work.

The call I made

Keep the bytes you can’t translate

A translator shouldn’t drop what it doesn’t understand. Tools that aren’t plain functions keep the caller’s original bytes and are replayed as they came.

Before
  1. Caller sends toolsa function and a web search
  2. Translator keeps type “function”no error, no drop header
  3. Web search is gone
  4. The model answers from memoryit looks like a worse model
After
  1. Caller sends toolsa function and a web search
  2. Anything that isn’t a functionkeeps its original bytes
  3. Replayed to the provideras the caller sent it
  4. Round-trips on every provider path

What it’s for

Agent Command Center sits between an application and its model providers, the way Portkey, Bifrost or LiteLLM do: a single Go binary with one OpenAI-compatible API, and the Anthropic and Gemini formats too.

Around the routing is what a team needs in production: 15 routing strategies, failover, circuit breakers and cost-aware routing among them; exact and semantic caching; budgets, quotas and virtual keys; 18 built-in guardrail scanners; and tracing.

Why it was hard

A gateway that translates between provider formats has to be lossless. A field it silently drops looks like the model getting worse, not like a gateway bug.

The bug that looked like a model problem

The Anthropic translator kept only tools of type “function”. A web search tool was discarded with no error and no drop header, so the model answered from its training data. To the caller it looked like a worse answer, not a failure.

Bugs a demo never shows

Turning tracing on could kill the gateway. Three plugins that run after the parallel stage read and wrote one request context at once, and Go ends the process on that, with every request in flight. I hit it while testing the OTLP exporter; it was older than that work.

The native Anthropic path built its cache key from an empty message list, so every prompt to a model hashed to the same key. The second request got an empty 200 back, and the provider was never called.

Expired API keys synced from the control plane kept authenticating, because the expiry never left Django. And Gemini’s thinking tokens were missing from the cost, so every run with thinking on was under-billed.

Receipts

  1. Agent Command Center README (future-agi/future-agi, agentcc-gateway)The product’s own description: one OpenAI-compatible API, 15 routing strategies, exact and semantic caching, budgets, quotas and virtual keys, 18 built-in guardrail scanners, Prometheus and OpenTelemetry. It compares itself there with Portkey, Bifrost, LiteLLM and Helicone. Future AGI team product.
  2. future-agi/agent-command-center-sdk PR #1 — AgentCC SDK v1.0.0 and the public repository (merged 22 Apr 2026)v1.0.0 of agentcc (Python), @agentcc/client, @agentcc/langchain, @agentcc/llamaindex, @agentcc/react, @agentcc/vercel, as one commit by Kartik. The SDKs were first written by nik13 in an internal repository and were already on npm and PyPI as 0.1.0 (9 Apr 2026).
  3. npm — @agentcc/client versions0.1.0 published 9 Apr 2026 by azainfi; 1.0.0 published 22 Apr 2026 by nvjkkartik. PyPI agentcc: 0.1.0 on 9 Apr, 1.0.0 on 22 Apr 2026.
  4. Agent Command Center: cross-format translation (agentcc-gateway/internal/translation)The public code. Authorship is from internal PR future-agi-temp#57 (merged 21 Apr 2026, 20 of 20 commits by Kartik), which a visitor can’t open: requests to /v1/chat/completions, /v1/messages and /v1beta/models/* can route to any configured provider.
  5. future-agi/future-agi PR #2302 — Anthropic server tools on the OpenAI-format endpoint (merged 25 Aug 2026)Non-function tools were silently dropped by the Anthropic translator; the fix keeps and replays the caller’s original bytes. Authored by Kartik.
  6. future-agi/future-agi PR #2186 — gateway traces over OTLP/HTTP with prompts and completions (merged 18 Aug 2026, v1.28.0)An OTLP/HTTP exporter (the otel plugin could only print spans to stdout); bodies behind otel.include_bodies, redacted before truncation; byte-bounded batches; streamed completions assembled; image, embedding, rerank, search and OCR bodies. The stacked #2183, #2187 and #2188 were collapsed into it. Authored by Kartik (11 of 11 commits). It also fixed a runtime fatal (concurrent map read and write) when several post-parallel plugins shared one request context, which took the gateway down with every in-flight request.
  7. future-agi/future-agi PR #2231 — run the plugin pipeline on /v1/messages and Gemini generateContent (merged 21 Aug 2026)Those two handlers never called the plugin engine, so Anthropic-SDK and Google-GenAI-SDK traffic had no span, request log, cost, budget, credits or rate limits. Fixed on all eight paths, with a regression test over all 11 billable handlers. Its review fallout fixed the native Anthropic path hashing every prompt to one exact-cache key (a second request was served a content-less 200 without calling the provider) and the streaming usage tee leaking its goroutines on client disconnect. Authored by Kartik.
  8. future-agi/future-agi PR #936: enforce control-plane key expiry on synced keys (merged 18 Jun 2026)Expired keys synced from the Django control plane authenticated indefinitely: expires_at never left Django, so the gateway never saw it. Authored by Kartik (3 of 3 commits).
  9. future-agi/future-agi PR #853 — Error Feed cluster RCA agent, perf sweep, billing (merged 27 Jun 2026)Root-cause agent over ClickHouse traces (at launch: per-trace summaries on a lite model, the investigation on the main model, ~$0.03–0.08 a run), streamed Fix tab, cached synthesis, billing wiring. Feed endpoints before → after: overview ~1.5 s → 49 ms, trends ~800 ms → 280 ms, sidebar ~1.2 s → 100 ms, list ~1.7 s → 47 ms. Authored by Kartik: 83 of 113 commits. velalagan-pixel (19) built the redesigned Overview and Fix tab UI; the KarthikAvinashFI, commitPirate and cdileep23 commits are unrelated dev-branch fixes carried in by a merge. The agent itself is in ee (internal ee#121, all 54 commits by Kartik). It also folds Gemini thinking tokens into completion tokens, which the cost header had left out.

Every source on the site

Next, a little further back in timeSynthetic data generation →