Topic
observability
7 posts
- Agents5 min
My LLM tracing store was 138 GB. The traces were 3.2 MB of it
ClickHouse disk usage under self-hosted Langfuse: 117 GB of it profiling its own queries. The fix sat unapplied for 18 days, and the table was too big to drop.
- Evals7 min
44 error alerts, 19 automatic fixes, and no gate on how big a fix could be
AI code review on auto-fixed alerts: the alert-to-deploy path had gates for relevance, kind, review, QA and cost, and none for size. What 19 fixes looked like, and why file count lies.
- Agents4 min
The numbers halved and the fix never ran
Deployment verification the hard way: wall clock down 53 percent, tool calls down 47, and the line the fix exists to produce absent in both runs. The config was a symlink shared by every worktree.
- Inference7 min
The flag was set and the feature was not there
GPU not being used, silently: the variable that turns on GPU embedding was set, but the binary was built without GPU support, so it ran on CPU. 306 seconds, not 51.
- Evals5 min
The span that answered a question it never measured
OpenTelemetry LLM spans on every tool call told me the agent never ran anything in parallel, and I withdrew it. The numbers were not wrong. I had written them myself.
- Agents8 min
Building the guardrails I needed after a session died
Claude Code hooks and a status line for context usage and compaction. Both read the transcript rather than an API field, and that choice is the post.
- Agents8 min
I instrumented everything except where the decisions happen
LLM agent observability with a connected, green tracing dashboard, and zero traces from the three components that actually make choices.