
How to Evaluate an AI Agent That Books Your Meetings
Task-completing agents can't be graded like chatbots. Here's an eval framework built around side effects, reversibility, and the cost of being wrong.
Blog
Practical writing on productivity, AI, and building software.

Task-completing agents can't be graded like chatbots. Here's an eval framework built around side effects, reversibility, and the cost of being wrong.

The bottleneck for multi-agent setups isn't a smarter model — it's shared state, clean handoffs, and knowing who owns the calendar. Coordination is the real work.

Physics and brain analogies make LLMs feel intuitive — and quietly lead us astray. Here's what to reason about instead.

Coding agents only see the present, so they keep re-solving problems your team already solved. The fix isn't a smarter model — it's durable, searchable context.

Using one model to fact-check another can turn errors into a mutual admiration society. Here's how the recursion happens and how to break it.

Bigger context windows won't fix agents that forget. Persistent, structured project memory is the real unlock for sustained knowledge work.

A practical rule for AI-native software: use an agent only when the steps are unknown and the goal is clear. Use a deterministic workflow for everything else.

Email and calendar data breaks most RAG setups because relevance is temporal and social, not just semantic. A bigger embedding model won't save you.

Authentication tells us an agent is real. Delegation tells us who it's acting for — and that's the primitive we're missing. Here's how to build it.

AI agents are non-human identities that deserve just-in-time, scoped access — not the permanent keys most teams still hand out. Here's how to do it.

Model choice is now a routing decision, not a beauty contest. Here's how to match task, latency, and cost across the big providers.

AI coding agents love to invent package names. Attackers register those names and wait. Here's how the hallusquatting attack works and how to defend against it.
Get started
Claim your address before someone else does — free to start, with an AI-native inbox built in.