· 4 min read
Evaluating AI Agents: Why Task Success Beats Benchmarks
Public leaderboards reward the wrong things for agentic work. Here's a practical eval harness built around real task completion and side-effect safety.
Blog
From the Tamaton team
Practical writing on productivity, AI, and building software.
All#ai#ai-agents#ai-productivity#applied-ai#automation#calendar#context-switching#deep-work#documents#email#engineering#engineering-leadership#invisible-ai#knowledge-work#llm-evaluation#product-updates#productivity#prompt-engineering#remote-work#search#security#spreadsheets#startups#storage#tamaton#unified-workspace#workflow#workflows
· 4 min read
Spreadsheets as Tool Use: Letting LLMs Compute, Not Guess
Why LLMs should generate and execute formulas instead of hallucinating math — plus concrete patterns for verifiable spreadsheet automation.
· 4 min read
The Inbox Is the Hardest Agent Problem in Productivity
Email looks routine, but it's the toughest environment for autonomy: ambiguous intent, irreversible sends, and tangled threading state. Here's what reliable inbox automation really takes.
· 4 min read
Context Windows Are Cheap, Attention Is Not
Stuffing a million tokens into a prompt degrades reasoning more than it helps. The real skill is curating what the model actually attends to.
· 5 min read
RAG vs. Long Context: When to Retrieve and When to Stuff
A practical decision framework for choosing retrieval, full-context, or hybrid approaches based on data volatility, cost, and accuracy.
· 4 min read
Inbox Zero Is Obsolete: Triage for an Agent-Run Inbox
When agents read and draft your email, success isn't an empty inbox — it's well-designed triage rules, escalation thresholds, and approval boundaries.
· 4 min read
Why Your Inbox Is the Best RAG Corpus You Already Own
Email threads encode decisions, commitments, and relationships better than any wiki — making your inbox the highest-signal grounding source for an AI assistant.
· 4 min read
Structured Output: Getting Reliable Tables From LLMs
Practical patterns for coaxing consistent, spreadsheet-ready data out of LLMs — schema enforcement, validation loops, and the pitfalls that quietly corrupt your tables.
· 4 min read
Search Across Your Files Is Broken — Semantic Won't Save It
Pure vector search over personal files stumbles on recency, permissions, and exact terms. Hybrid retrieval plus metadata is the fix.
· 4 min read
Evaluating AI Agents: Beyond Pass/Fail Benchmarks
Final-answer accuracy hides how agents actually work. Here's why trajectory, tool-use, and recovery metrics matter — plus a practical scoring rubric.
· 5 min read
How to Evaluate an AI Agent That Touches Your Inbox
Benchmark scores won't tell you if an email agent is safe to trust. Here's a practical eval harness built on task completion and harm metrics.
· 4 min read
Evaluating AI Agents on Multi-Step Office Tasks
Single-turn benchmarks miss what matters. Here's a practical eval harness for agents that move data across email, docs, sheets, and calendar.
Get started
Reserve your @tamaton.ai email
Claim your address before someone else does — free to start, with an AI-native inbox built in.