When I build with an AI agent, it produces interface quickly and forgets my design decisions overnight. It reaches for a colour the design has no place for, undoes a layout choice because it looked like an accident, and asks what a main button should look like when that was settled months ago. Close the conversation and yesterday's correction is gone. So I stopped correcting and started writing things down.
An agent, if you have not used one, is an AI assistant you talk to in plain language, which then changes the real files of a website or an app for you. It is fast and it is capable, and every conversation starts from nothing. Think of a very quick new colleague who forgets everything they learned each evening. You end up re-explaining the same context every morning, context you already paid for once.
Three things that kept going wrong
All three are design problems before they are technical ones.
It asked about things the design already answered. "Should this button be blue?" on a site that has exactly one colour for things you can press. Each question is a small interruption, and together they cost more than doing the work myself.
It undid decisions it did not know were decisions. A choice gets made for a reason, the reason lives only in my head or in a conversation that has since closed, and six weeks later a new conversation sees something that looks arbitrary and tidies it up. An arbitrary-looking choice is the first thing a newcomer fixes.
And my feedback evaporated. A correction, a preference, a "no, not like that" applied to the conversation I gave it in and nowhere else. The next one made the same mistake, because nothing recorded that I had already ruled it out, or why.
Three rules, all about when to ask
Default to the design, ask about taste
If the design system already answers the question, use the answer and keep going. A design system is the shared set of colours, text sizes, spacing and building blocks a product is made from, so most small questions have an answer in it already. Stop only for a real design choice with no obvious winner, or for anything that cannot be undone.
Save the questions for the end
Open questions go into one list at the end, not into the middle of the work. If five things were unclear, I want to settle five things once, not be stopped five separate times.
Assume a strange choice was a decision
Before changing something that looks odd, find out why it is like that. Look for a note that explains it before assuming it was a mistake. The reasoning behind a design disappears the moment someone who did not know it was there replaces it.
Put together, the goal is an agent that behaves like a colleague who has been around long enough to know why things are the way they are.
Four kinds of notes
Decisions
One short note for every choice that is not obvious: which options there were, which one won, and why. A later change gets a new note rather than rewriting the old one, so the argument stays readable. Fittingly, the first note in this system is the one about adopting it.
Feedback
My reaction goes in word for word, before anyone interprets it. Interpretation is where the nuance gets lost, and with the original words on record a later conversation can work out the right lesson again, even when the first reading of my own feedback was slightly off.
What changed
One line for every change a visitor would notice, written as it happens rather than reconstructed afterwards.
Where we left off
For work that runs over more than one conversation: what is done, what is next, and enough context that picking it up again does not mean replaying everything that led here.
Generic taste versus house taste
Much of what an agent knows about good interface work comes from instructions other people have written and shared: how to review a screen, how to make a page accessible, how to lay one out. Most of it is good, and using it is a shortcut worth taking. But it carries someone else's habits, and someone else's idea of how a product should look.
Every product makes some different choices on purpose. Follow the generic advice literally and you slowly redesign your own product into the average of everyone else's, one reasonable-sounding suggestion at a time.
So the order is written down: my own rules outrank the borrowed ones whenever the two disagree. The borrowed instructions are kept as they are and only updated deliberately, and my corrections sit beside them and win. Without a stated order the house style loses by attrition, because "this is how it is usually done" is a persuasive argument even when it is about somebody else's product.
What it looks like when it works
The clearest evidence I have came from an afternoon that went badly on purpose.
I was making diagrams for these articles with an agent. The approach changed three times: drawings of an interface, then the real interface shrunk to fit the page, then the real interface rearranging itself for whatever space it had. Each change got a note, and each new note replaced the last instead of editing it. By the end there were three notes about the same feature, two of them wrong.
That looks like waste and it is the opposite. The third note could say exactly why the second approach failed, because the second one had written down what it was betting on. Without that, the third attempt would have been a fresh guess, and the most likely fresh guess is the first idea again.
The other half is recording feedback word for word. I once typed, in my usual mix of Dutch and English, "building mini pc, en asked ai to render my house mogen weg", roughly "and those two can go". The agent read it as "remove the images from those two articles". I meant "take those two articles offline". Because my exact sentence was on record next to its reading of it, the mistake was visible the moment it was questioned, and fixing it took a minute. Had only the interpretation been kept, the note would have confidently preserved the wrong one.
The same problem as the design side
I have written about the judgment layer: the part of the work that decides whether something is right rather than just plausible, which a model cannot supply on its own. This is the same idea applied to the way of working instead of to a single result. Instead of checking every output by hand, I wrote the judgment down once, so it applies by default. The agent still needs a person who decides. It just needs fewer reminders about what that person already decided.