The rules I never break
One night in November, our Slack agent was handed a batch of broad questions. For each set of questions, it started four or five sub-agents in parallel.
In about two hours it ran 21 sub-sessions and more than a thousand model calls. Across the day it made roughly eight times its normal number of calls, and the organisation's API credit ran out, which stopped all three of our agents at once.
The frustrating part: a rule for exactly this already existed. Its instructions said that broad, multi-repository investigations go to the investigator agent as cards, one at a time. The model had the rule and delegated anyway. It was the first time it had used sub-agents at all.
So I didn't add another sentence to the prompt. I took the delegation tool away from it. The investigator kept bounded parallel delegation. Since then I've treated a small set of rules differently from everything else: they don't live where the model can talk itself out of them.
This is part 3 of a series on what I learned putting AI agents to work in my own job. Part 2 was about letting code, not the model, make the decisions.
Where a rule lives
I rank every control by how hard it is for the agent to get around. If the tool doesn't exist, the model can't use it. If the server refuses, the model can't argue with it. A hook that runs before every action is strong, once you've verified it's actually active. A sentence in the prompt is the weakest lock there is: it only holds when the model decides to follow it.
The principle that falls out of this: anything irreversible or visible outside the system needs one of the strong locks. Prompt rules are for intent, tone and things you can undo.
I wrote about the same ladder, applied to keeping source code out of Slack, in the first post on this blog. It's the same ordering as that post's five layers, with human approval shown as its own step; drift detection stays what it was there, a safety net after the fact rather than a lock.
The six rules below are the ones I don't bend. Not all of them can be enforced in code; where they can, they are.
1. A person owns everything that leaves
An agent drafts; a person is responsible. A customer who got the wrong answer doesn't care that an agent wrote it.
In practice that means agents propose and people decide:
- The idea reviewer turns a feature idea into a written scope and acceptance criteria. Nothing becomes a ticket until a person adopts it.
- Among my three agents, only the Slack agent, kenny, writes to the issue tracker. The investigator and the idea reviewer leave drafts, which keeps every agent write in one place I can check. The search monitor's issues are filed by a scheduled kenny job, so they go through the same gate.
- When a monitor files a ticket for a problem, the agent can move it as far as "in review". Closing it is mine.
- The Slack agent doesn't promise refunds, exceptions or fixes. That's a decision, and decisions have owners.
2. Minimise data: approved identifiers, no names or contact details
Everything an agent reads goes to the model provider. So the rule for what goes in is about what you'd be comfortable sending there, and the answer is: as little as the task needs.
- Approved identifiers instead of people. An order number or a member ID is enough to investigate almost anything. Names, contact details and payment data stay out of Slack, tickets and agent notes. Those IDs still need controlled access; they aren't automatically anonymous.
- Key names, never key values. An agent can say which secret a service needs and where it comes from. It never repeats the secret.
- Mask what spreads. Screenshots attached to tickets are masked, because more people read the tracker than the place the agent works.
- Remember the fallback. When our main model provider refused calls during that credit outage, traffic went to a second provider: first as an automatic fallback, then by my choice until the credit was topped up. For about sixteen hours, the agents' model requests went to the second provider. I logged the exact window, because "where did our data go?" deserves a precise answer.
3. Check what your data really is before writing rules
While writing the rules for the investigator agent, I described its local database as test data, safe to change. Then I looked closer: the row counts and dates said it was a copy of production.
That changed the rules completely: anything that could change state was blocked until we'd checked every path out of the system.
Then I swung too far the other way and assumed every outbound channel would reach real customers. I checked the outbound integrations one by one. Most were isolated; one could still send real messages. We disabled it before reopening state-changing work.
Two lessons I still use. Establish what data you're dealing with from evidence (counts, dates), not from what it's called. And assess risk item by item; a blanket "it's dangerous" is as wrong as a blanket "it's fine".
4. Never approve what I can't explain
When I wrote a guide for my team on working with agents, I put this in as a rule, not advice: if an agent asks for approval and you can't explain, in your own words, what it will do, refuse and ask. I hold myself to it.
My local coding agent asks before it runs commands or makes changes I haven't allowed; if I can't explain a command, I ask the agent to explain it, and if I still can't, I don't run it.
The mirror image matters just as much. Several times a safety check stopped my agent mid-task: once over granting write access to a database, once over moving a credential, once over auto-approving a hook. Each time the agent stopped, and I did the step myself or changed the setting deliberately.
A block sends the decision back to me. I review why it fired and explicitly decide whether the action or the permission should change; I don't ask the agent to evade the control. When a guard says no, nobody looks for a way around it. That includes me, and it includes the agent.
5. Scope is cost
The night of the credit outage is the expensive version of a simple point: an agent's cost and blast radius scale with how much you let it do per request.
So every tool has a budget per question:
- Requests to the live site are capped per question and counted as actual requests, so a script that loops still counts every call.
- Database queries are capped per question, and only aggregates and identifiers leave the database.
- Cloud access is a read-only account. A write attempt fails on the provider's side, whatever the model decides.
And the way you ask matters as much as the limits. "Investigate everything about refunds" invites a fan-out. "These three cases, from last month" gets a faster, cheaper and more accurate answer.
6. Count before you claim
Early on, I told myself the idea reviewer had "hammered production". When I counted, it had made about five requests. The heavier traffic had come from the investigator's cards, whose instructions were missing the request limit the Slack agent already had. The fix was the same sentence in all three agents' instructions, not a new restriction on the wrong agent.
I hold the agents to the same standard. Our Slack agent says how sure it is: checked in the code, read in the docs, or inferred. Its worst failure is stating something unchecked as if it were checked. If I expect that from an agent, I owe it to the team to count before I make a claim about the agent.
What I took from it
- Put the rules that must hold in a lock the model can't argue with. Prompts are for intent.
- Agents propose; a person decides, and a person closes the loop.
- Minimise data: approved identifiers, not names or contact details.
- Know where your data goes, including the fallback.
- Find out what your data is before you write rules for it, then check risks one at a time.
- Don't approve what you can't explain, and don't route around a guard.
- Give every tool a budget per request, and ask narrow questions.
- Count before you claim, especially about your own systems.
The series
- From doing the work to delegating it
- Let code decide, let the LLM explain
- The rules I never break (this post)
- Context is the job
- Agents forget, so I write it down
- kenny: an agent that does my job
- Hermes: reminders that carry the check
- How to start in two weeks
Which of your agent's rules live only in a prompt today, and what would it take to move one of them to a lock the model can't argue with?
Comments
No comments yet. Start the discussion.