Notes from building an AI design collaborator · part 8
How do you teach an AI your taste?
· 5 min read
You can write down a design system. You can write down your process. You cannot write down your taste, which is annoying, because taste is most of what makes a designer a designer.
Taste is the thing that goes "no, not like that" without always being able to say why in the moment. It's the reason two designers with the same brief and the same components hand back different work. So when I started leaning on an AI for real design work, this was the part I was sure it would never get. And mostly it didn't. It would do something slightly off, I'd correct it, and next week it would do the exact same slightly-off thing again, cheerfully, like we'd never met.
The correction that evaporates
That's the real problem. Not that the AI has bad taste. It's that your corrections evaporate. You fix the same thing forever because nothing you say sticks past the session.
So I built a feedback loop. Two ways in.
One is passive. It watches what I accept and what I throw out, and at the end of a session it writes down what it noticed. Not a transcript, and not a log of what I did. Judgement calls only, which means an override, a correction, a place where I took the recommendation and immediately changed one thing about it. It tells me in one line what it wrote, every time. No silent observation, because a thing quietly building a file about how you work is a different product to the one I wanted.
The other is me saying it out loud. I say "no, we always do it this way," and it does two things at once: changes its behaviour right now, in this session, and writes the durable version down.
What an entry actually looks like
This is the part that took a few tries to get right, and it's the bit worth stealing.
Every entry is a pair. An observation, which is what concretely happened, and a principle, which is written in a fixed shape: in this situation, do this, because that. Plus the dates it's been seen, whether I named it or the AI noticed it, which part of the workflow it applies to, and how strongly it's held.
The "because" is not decoration. An entry without one is a description, and a description can't be applied to a situation it didn't come from. "Prateek prefers denser wireframes" tells the AI nothing about a screen it hasn't seen. "In lo-fi, fill the content region rather than capping the width, because the sparse version makes reviewers argue about whitespace instead of structure" is usable on Monday, on a screen nobody has drawn yet. We made the because mandatory. No because, no promotion, however obvious the preference seems.
The part I almost got wrong
Here's the trap I walked into. The first version turned every offhand comment into a hard rule instantly. I muttered one thing about spacing on a bad day and the AI treated it as law forever after. That's worse than no memory, because now it's confidently wrong in my name.
So nothing the AI merely notices gets acted on. Passive observations land as candidates, on a watch list, and stay there. A candidate gets promoted one of two ways: it shows up three or more times across at least two different features, or I name it explicitly, in which case it promotes immediately.
That asymmetry is deliberate and it's my favourite part of the design. Deliberate beats frequent. Me saying "we always do it this way" once is a stronger signal than the AI watching me do something similar three times, because I know which of my behaviours are principles and which are Tuesday.
There's a way back down, too. If an established rule gets contradicted in practice, the AI raises it once, openly, and asks. Rules that can only accumulate eventually strangle you.
Whose taste, exactly
The profiles are per designer, not global, because my taste and my teammate's genuinely differ and flattening them helps nobody. They're plain markdown, in the repo, and anyone can open their own and delete a line they disagree with. That transparency isn't a nicety, it's the thing that makes the whole arrangement acceptable.
Above that there's a collation step. When two or more designers independently hold the same principle, it's a candidate to become a team principle rather than a personal one, and a human approves each promotion. What I got wrong at first was where those team principles go. If a principle simply adds something the process was silent about, a shared file is fine. But if it contradicts a step in the process, filing it as a preference is useless, because the process outranks it and the process just wins. Those have to become an edit to the step itself.
The half that didn't work
I'd rather tell you this than let the post end on a win it hasn't earned.
The capture works. The consumption doesn't. Nothing reads the profiles back as a to-do list, so principles get recorded faithfully and then sit there, waiting to be rediscovered by me saying the same thing out loud again.
I know exactly how badly, because I caught myself doing it. I gave what I thought was fresh feedback on two things, quite forcefully, and both had already been recorded a week earlier, one of them almost word for word. They only got acted on because I repeated myself. Worse, four different designers had independently written down the same complaint about onboarding, which is double the bar for it becoming a team principle, and it sat there uncollated for a fortnight because nobody had run the step that looks.
There's a routing mechanism for this now. As I write, it has never run a full pass, so I'm not going to tell you it's solved.
Taste can't be specified up front, but it can be accumulated. The move is to capture preferences as rules you can promote, not corrections you repeat, and to make the AI earn a rule before it trusts it. Then, and this is the bit I underrated, make sure something actually reads them. A perfect memory nobody consults is just a tidier way of forgetting.
Part of a series on building an AI collaborator for our design team at Xflow. Each post stands on its own.
- ai
- design-craft
The work behind the series
Designing the Instructions
This post is one thread out of a three-month project: an AI design collaborator for a payments team. The case study is the whole of it: what worked, what broke, and what is still unproven.
Read the case study →