Skip to main content

Notes from building an AI design collaborator · part 1

I stopped describing my product to AI. I just gave it the HTML.

· 10 min read

A couple of years ago I wrote about coding my own portfolio site, back when I was a designer who could barely tell a flex div from a margin. I'm still mostly that designer. I've just picked up a few opinions since.

Here's one. The reason your AI keeps handing you generic design help isn't the model. It's that the model has never actually seen your product.

The one everyone's reading

There's a great piece going around by Thariq, an engineer at Anthropic, called "the unreasonable effectiveness of HTML." If you build anything with AI, go read it. His argument is that HTML beats Markdown as the thing an AI writes for you. Specs, prototypes, a whole living design system in one file, all of it in HTML because HTML is dense, it's visual, and it keeps you in the loop instead of handing everything off to the machine.

I agree with all of it. I want to talk about the other direction, the one I don't see anyone talking about. Not the HTML your AI writes for you. The HTML you give it.

The average of the internet

Ask an AI to redesign one of your screens and you get something clean, sensible, textbook, and not at all yours. It isn't wrong. It's just average.

I hit this every single day at Xflow. I'd ask for a small change to a page in our dashboard and get back a gorgeous redesign of a page we'd never built. It moved a toggle that didn't exist. It tidied up a section that was never there. Very confident. Completely made up.

Of course it was. The model has seen thousands of settings pages, so it hands you the average of all of them. And in a cross-border payments dashboard full of compliance states and payout flows, an average isn't a starting point. It's a well-dressed guess.

That's the word I keep landing on. Honest. A design is honest to your product when it comes from what the product actually is, not from the AI's idea of what a product like yours probably looks like.

I tried describing it. I tried screenshots.

My first fix was to just describe the thing better. "Sidebar with these five items, header with the account switcher," and so on. This turns you into a slow, lossy, human screenshotter. It doesn't scale past two screens, it goes stale the second the product changes, and the AI still builds its own version of your sentence anyway.

Screenshots were the next idea. Better at looks, useless for work, because the AI can't edit a picture. It looks at it, forms an impression, and redraws the whole thing from memory. A screenshot is something to point at, not something to change.

Then the obvious finally landed, and it's really just Thariq's point flipped around. If HTML is the best thing an AI can write, it's also the best thing it can read. My product was already HTML sitting in a browser. I'd been reading it out to the AI like it was over the phone, when I could have just sent the file.

So I gave it the HTML

At Xflow we built a small pipeline that grabs our real pages as self-contained HTML. The real DOM, the styles baked in, the page able to open on its own. I'm a designer and not an engineer, so version one was held together with code that would've made our developers wince. It worked, and we cleaned it up later.

Now the AI opens the actual page and edits the actual page. I mention a toggle and it touches the real toggle, in the section it really lives in, with our real styles. The same one-line request that used to come back as fiction now comes back as an edit.

One thing I'll say up front, because we work in payments and somebody always asks. We capture against a test account, in test mode, so live customer financials never get baked into a file sitting on someone's laptop. The capture then strips the test-mode banner out, so the baseline isn't visibly a test artifact and can stand in for the real screen.

None of this is an Xflow thing, which is the point. Whatever you work on is already HTML in a browser, and that HTML is a more honest brief than anything else you can hand an AI. More honest than a Figma file, which is a drawing of the product and not the product. More honest than your own description, which is really just your memory of it on a good day.

But won't that cost a fortune in tokens?

Fair question, and yes, if you're careless. Raw product HTML is massive. Framework wrappers, the same inline styles a thousand times over, tracking scripts, images stuffed straight into the markup. Send all of that and you've burned a small fortune before the AI has done a thing.

So you don't send the raw page. You clean it first, and it's worth knowing what the cleaning actually does, because "clean it" hides the two decisions that matter.

The first is where the CSS goes. Bake the styles into every page and each file stands alone, which is lovely for sharing and terrible for a library, because you're paying for the same stylesheet once per screen. Pull the CSS out into one shared file that every captured page links to and you pay for it once. On our dashboard that's the difference between roughly 580 KB a page and roughly 110 KB a page. About five times smaller, for a change that costs you nothing except that the file no longer opens in isolation.

So we keep both. A self-contained copy, everything inlined, for sending someone a single file. And an editable copy, one shared stylesheet, pretty-printed markup, images left as remote URLs instead of giant base64 blobs. The editable one is the substrate the AI actually works on, and the fact that a human can read its diff turns out to matter more than I expected.

The second decision is what you delete. Tracking scripts, hydration scripts, dead analytics, the detached tooltip that happened to be open when you hit save. What's left is mostly structure and words, which is cheaper to send and easier for the model to read.

The half of the product that has no address

This is the caveat I wish someone had given me on day one.

"Open a screen and save it" assumes your screens have URLs. Ours mostly don't. When we crawled our own dashboard we found about fifteen addressable pages and around thirty overlays: modals, wizard steps, tab panels, the drawer that slides in over the table. Two thirds of the product, and none of it reachable by typing an address.

You can't capture a wizard's third step by navigating to it, because there's no "it" to navigate to. And a script that clicks its way through does fine until it hits a step where Next stays disabled until you've picked a partner, which is roughly step two of anything real.

What actually works is boring. A person walks the flow in a browser, saving each step as they go, and a cleaner turns those raw saves into the same kind of editable snapshot the automated capture produces. Slow, manual, and the only thing that reaches the two thirds of the product where the interesting work happens.

Four ways this bites you

Every one of these cost me an afternoon, so here they are for free.

Your primary buttons go hollow. Save a page whose stylesheet got duplicated and the framework's own button reset gets re-emitted after the utility layer. Same specificity, later in the file, so it wins. Your filled buttons render as outlines and you spend an hour blaming your own CSS.

A browser extension can poison the whole page. If a devtools or analytics toolbar is running when you save, its shadow DOM gets serialized into your file. Shadow DOM is scoped in the live page. Once it's flattened into your page's stylesheet it isn't, so its defensive little .hidden { display: none !important } starts clobbering your app's real class of the same name. That one hid our desktop nav site-wide and took ages to trace.

Open a modal, capture a frozen page. Apps lock body scroll while a dialog is up. Capture with that class still applied and the saved page can't scroll at all.

Check it at the real breakpoint. I verified one capture in a narrow preview pane and it looked perfect, because the nav it had broken is desktop-only and was correctly hidden at that width. The bug survived the check that existed to catch it. Look at the baseline at the width your product is actually used at.

Try this in ten minutes

You don't need any of our tooling to feel the difference. Open a screen in your product, hit "Save Page As" in your browser and pick "Webpage, Complete", and there's your HTML. Give that file to your AI and ask for your next real change. Then ask for the exact same change with nothing attached, and look at the two.

One of them edits your product. The other invents a cousin of it.

That's the demo. For anything beyond it the browser's own save is the weak link, and the tool worth knowing about is the SingleFile extension, which saves a far more faithful single-file copy. Pair it with a script that strips the junk and shares one stylesheet and you've got the beginnings of a real library instead of a folder of one-offs.

These days I run one test on anything an AI designs for me. Could this ship tomorrow without inventing a component we don't have? If yes, it's honest. If it needs three controls that don't exist, the model drew me the internet, not our product.

One honest catch, though. A brand new screen has no HTML to hand over, so for genuinely new work you're back to describing, or better, giving the AI your design system's real components to build from. You're still grounding it in something real. Just the system this time instead of a page.

What I built

Doing that capture by hand got old fast. Save the page, clean it up, do it again next week when the product moved. So I built a tool that does it for any product. You point it at a URL and it turns the real thing into a context library your AI can read: the pages, the structure, even the product's own colours and type, all pulled in as facts, cleaned down so it's cheap to feed in.

It's called the Design Context Kit, it's free and open source, and it's up now. If this honest-to-your-product idea clicks for you, that's the fastest way to try it on your own product. The rest of the series gets into how the pieces actually work.


This is the first post in a series about building an AI collaborator for our design team at Xflow, one that works from the real product, follows a real design process, and actually remembers why we made the calls we made. Each post stands on its own, and the rest of the series is linked below.

Next: why I gave the AI my design process, and none of my design decisions.

  • ai
  • design-process
  • html

The work behind the series

Designing the Instructions

This post is one thread out of a three-month project: an AI design collaborator for a payments team. The case study is the whole of it: what worked, what broke, and what is still unproven.

Read the case study →