· 4 min read
When Your AI Change 'Works' But Success Drops 20 Points
Prompt and model tweaks that pass every smoke test can quietly wreck real outcomes. Here's how to evaluate agents on results instead of vibes.
Blog
From the Tamaton team
Practical writing on productivity, AI, and building software.
All#ai#ai-agents#ai-productivity#applied-ai#automation#calendar#context-switching#deep-work#documents#email#engineering#engineering-leadership#invisible-ai#knowledge-work#llm-evaluation#product-updates#productivity#prompt-engineering#remote-work#search#security#spreadsheets#startups#storage#tamaton#unified-workspace#workflow#workflows
· 4 min read
Stop Vibe-Checking Your AI: Build a 50-Case Eval Set
Public benchmarks won't tell you if a model can triage your inbox or reconcile your spreadsheet. Here's how to build a private, brutally specific eval set in an afternoon.
· 5 min read
Chatbot vs. Agent: The Line That Actually Matters
A chatbot returns a response. An agent decides and executes a sequence. Confusing the two is why so many 'agentic' rollouts quietly stall out.
· 5 min read
One Model to Rule Them All Is a Bad Default
Picking a single LLM for every task leaves capability and money on the table. Route by task instead: cheap triage, strong drafting, dedicated verification.
· 5 min read
Your Inbox Is a Database. Query It Like One.
Email isn't a list of messages — it's an unindexed database you never designed. Here's how to query it in natural language without hallucinating your way into a bad reply.
· 5 min read
When to Trust an LLM: Lessons from Radiologists Who Don't Get Fooled
Medical research on why some experts resist bad AI advice — and how to turn those findings into a practical framework for judging LLM reliability in your own work.
· 5 min read
Your Inbox Is the Best RAG Corpus You Already Own
Email, calendar, and file storage are already chunked, timestamped, and permission-aware. Treating them as your retrieval corpus beats dumping documents into a vector database.
· 6 min read
Nobody Evaluates Their Prompts. Here's a 30-Minute Fix.
A practical walkthrough for building a 40-example eval set in a spreadsheet — plus the LLM-as-judge traps that make most accuracy numbers meaningless.
· 5 min read
Your AI Agent Should Never Know Your Password
Agents that log in as you aren't a feature — they're a security architecture failure. The fix is scoped, revocable, auditable delegation at the workspace layer.
· 5 min read
How Tamaton Agents Finish Multi-Step Tasks Without Losing the Thread
A concrete walkthrough of building a Tamaton agent that triages inbox, drafts replies with real context, and books follow-ups — with a human holding the approval button.
· 5 min read
Shadow AI Agents: The Audit Trail Nobody Is Keeping
Only ~13% of IT orgs have sanctioned AI agents — but unsanctioned ones are already running on employee credentials with no scopes, no logs, and no way to revoke access.
· 5 min read
Context Rot: Why Long Prompts Quietly Break Agents
Long, harmless context isn't neutral. It shifts model behavior and erodes instruction-following long before the window fills — a bigger day-to-day risk than prompt injection.
Page 1 of 21Older →
Get started
Reserve your @tamaton.ai email
Claim your address before someone else does — free to start, with an AI-native inbox built in.