Designing with agentic coding tools is in and Figma is out—at least that’s the sentiment. After spending some time with these new tools, I do feel like they have a very different rhythm of making things.
One common pattern of designing with agentic coding tools is parallelization: having multiple agents running at the same time working on different aspects of the UI. The benefit of parallelization is that I don’t have to wait for each agent to finish running, which is often the bottleneck (latency) and what breaks flow. The downside is it’s cognitively demanding to manage multiple threads of chat and many mini-context-switching (What’s this chat doing? Have it finished the task? Did it succeed?). There’s also a lack of visual signal for what each agent is working on, so I end up having to take a few seconds to reorient myself to what each chat’s work is every time I switch among them.
There’s a certain gap between intention and result that is different from working in a canvas-based design tool. In canvas tools, the hand-eye feedback loop (execution-evaluation) is almost instant. You make a change and see the change happen in real time. For agentic tools, you express intent for a batch of changes and evaluate them together. While I find this significantly better than a single-thread iteration, it’s definitely a new way of working that takes time to get used to. A “queueing” feature also helps when you want to keep adding to the same thread while the agent is still working on the previous task.
This reminds me of Bret Victor’s principle of “creators should have immediate connection to what they create”. There are two sides of argument to this: the widening gap between intent specification and output evaluation certainly makes it feel like you are further from the thing you create because the result feels less immediate than direct manipulation. Before, changing a color, font, radius, shadow, or just moving things around took milliseconds, and it felt more spatial and tactile. But on the other hand, now you are directly working with the material of software, so you have direct access to all the capability the material has to offer (think all the things you get for free designing in the browser) and you don’t have to go through another step of translating visual to code, which arguably makes the creator closer to the final output.
One way to think about it is how visual the output is and whether you are solving problems in existing conventions or trying to invent new patterns. The App-Site Spectrum is relevant here. When working on utilitarian interactive software, it’s beneficial to work in code because you are mostly assembling UI patterns with states, edge cases, latency that are traditionally hard to design statically and best to explore and confront in code. On the other hand, if you are designing for something visually expressive and the goal is novelty, the iteration speed bump and constraints imposed by code are frustrating.
Invention by definition requires breaking or pushing existing patterns, and manipulating things spatially is a much more natural, precise, and efficient way to come up with things you can’t even describe yet. You probably want a free canvas to make a hundred changes in a variety of resolutions to evaluate the outcome in a much tighter feedback loop—then push what you arrive at back to code. In these instances, creating bespoke, disposable design tools to explore specific visual output to be used in a larger composition is much more helpful. Designing gradient with shader is an obvious one that comes to mind, but there are a lot more examples such as generating dynamic patterns at scale, creating custom parameters for image effects, or illustrating data visualizations.
Another problem with designing in chat is you need to have a clear vision of what you want. When you set out to design or make something, that’s not always the case. Designing is a process of understanding, to quote Victor again: “You come to understand the system by pointing to things, adjusting things, moving yourself around the space of possibilities.” You develop your vision by going through numerous ideas and their combinations, comparing and reasoning across different options. Canvas as an interface is great for seeing options side by side, recognizing your assumptions, and identifying latent variables. However, the current agentic coding tools operates in a temporal dimension by default—showing the latest thread, the latest message, and the latest version of your design. As a result, you go through much less variations or keep doubling down on the one and often the first option. Perfect for polish, but there’s a lot of friction for exploration.
Of course you can always ask the tool to create that space for you, like Josh does. But instead of putting the onus on the user, what if we make the tools naturally encourage exploration through its own primitives and laws of physics? Tools—or technology for that matter—tend to reveal a way of thinking while at the same time concealing another. If “the medium is the message” has indeed become trite, we ought to remember that the affordances we design shape how people work. Ultimately, we should make technologies work for human, not the other way around.
We are in a “diverging” phase of tooling and everyone is turning into toolkenstein by stitching together their own toolkit. Agentic tools and the “everyone is builder” ethos are exciting but there are important lessons to learn from the canvas. Maybe agentic tools will bring in a spatial dimension that offers direct manipulation and freeform exploration. Maybe we will use a combination of new canvas tools in conjunction with agentic coding tools that make the roundtrip seamless. Who knows!