Notes from building an AI design collaborator · part 13
I sat down to turn my workflow into a swarm of agents. Most of it refused.
· 5 min read
Everyone's excited about agent swarms right now. One orchestrator, a dozen sub-agents, the whole job happening at once. It looks powerful, and for some work it genuinely is, so one afternoon I sat down fully intending to turn my design workflow into one. I spent most of that afternoon working out that I couldn't, and the reason turned out to be more useful than the swarm would have been.
The spine won't fan out
My workflow is a chain. You can't research before you have requirements. You can't wireframe before you have the information architecture. Each step eats the one before it. That's a dependency chain, and a dependency chain is the exact shape parallelism can't help, because there's nothing to run at the same time. Everything is standing in line, waiting for the thing in front of it.
It's worse than just "no speedup," because the chain is also where the judgment lives, and judgment does not survive being chopped up and handed to five agents who can't see each other. I tried a small version and got exactly what you'd expect, work that felt designed by a committee that never actually met. So the spine stays a single line, with me signing off between steps. That's not me being precious about it. It's the shape of the work.
Where the swarm actually earns its keep
The parallelism does have a home, just not where I first pointed it. It belongs inside the steps that are naturally wide. When I went through my nine steps and asked "is this one job or many independent ones," the answer sorted them cleanly:
| Step | Fan out? | Shape |
|---|---|---|
| Research | Strongly | Independent lookups. Competitor scans, analytics pulls, source gathering. Cheap model gathers, strong model synthesises |
| Validate and critique | Strongly | One agent per lens: accessibility, hierarchy, consistency, copy |
| Information architecture | Sometimes | Mostly synthesis. Optionally generate two or three structures in parallel, then judge |
| Lo-fi wireframes | Sometimes | A judge panel of alternatives works, with a catch, below |
| Everything else | No | Dependency chain |
Research is the obvious one. "Go find out about this" isn't a chain, it's a stack of independent lookups, and you can happily run those at once and pull the results together afterwards.
The critique pass is the one I'd push hardest. You want the accessibility read and the hierarchy read and the copy read happening at once, by agents that aren't watching each other, precisely so they don't quietly converge on one polite opinion. Independence isn't a side effect there. It's the feature.
The catch, and the thing I had backwards, is the lo-fi row. Generating four wireframe alternatives in parallel is a genuinely good use of fan-out, but every one of those agents needs the full accumulated context, the requirements and the research and the architecture and the taste profile, or you get four confident wireframes for a product nobody works at. So that's the one place where the parallel work all has to run on the expensive model. Fanning out doesn't make the work cheaper there. It makes it wider.
The prerequisite nobody mentions
A sub-agent is exactly as good as the context packet you hand it, and this is where most fan-out plans quietly fail. You can't parallelise a step whose inputs live in your head, or in the scroll of the current conversation.
Which means the boring infrastructure I built for a completely different reason turned out to be the thing that makes fan-out possible at all: the per-feature state file and the contract header on every step. Inputs, outputs, hard rules, what this step is not for. That is the packet. Without it, handing work to a sub-agent is just hoping.
If you're planning a fan-out and you don't have those, build those first. The parallelism is the easy half.
The model bit I had backwards
One more thing I got wrong. I was assigning models by importance, big model for the important steps, small model for the cheap ones. Wrong axis. The right one is judgment versus fetching. Anything that needs taste, or has to hold context across steps, gets the strong model, however small the task looks. Anything that's just go-get-this gets the cheap one, however important it feels. The trap is handing a judgment call to a cheap model because the task looked minor on the surface. That's how you save a few tokens and quietly lose the plot.
Worth checking whether your runtime can even express that. In a terminal session you pick the model per sub-agent with a flag, and you can script it. In some desktop apps the model is a session-level setting, so the elegant tiering you designed collapses into whatever you picked when you opened the window.
The one place it did work, unglamorously
I'll give the swarm its due, because it earned it somewhere I wasn't looking.
When we rebuilt our design system out of the code, the re-audit was six agents in parallel, each taking a slice of the component source and comparing it against the spec. That's fan-out in its natural habitat: dozens of small independent comparisons, no shared judgment, an obvious way to merge the results. It found real divergences that a single pass would have skimmed past, and I wrote about what it turned up.
Note what that job is, though. It's an audit. Checking is wide, deciding is narrow, and most of a designer's day is the narrow kind.
I still haven't built the big swarm, and I've stopped feeling behind for it. Most of what I do is a little bit of gathering wrapped around a lot of deciding, and you really don't want a swarm doing the deciding.
Part of a series on building an AI collaborator for our design team at Xflow. Each post stands on its own.
- ai
- agents
The work behind the series
Designing the Instructions
This post is one thread out of a three-month project: an AI design collaborator for a payments team. The case study is the whole of it: what worked, what broke, and what is still unproven.
Read the case study →