What does it actually take to design human-AI interaction? Insights from Agata Jałosińska

The Two Axes Every AI Designer Needs to Understand

Before you can design a human-AI interaction, you need to understand what you’re designing on two fundamental scales.

The first is system capacity – ranging from a tightly defined set of automated actions to a system that learns continuously from its users and develops capabilities you didn’t explicitly program. The more open-ended the system, the harder it becomes to prototype, anticipate edge cases, or apply existing design toolkits.

The second is output complexity – from a simple right/wrong answer to a full autonomous decision-making loop, like navigating a car in real traffic.

So with the term Autopilot we, we are already trying to aim for something which is very complex, which has huge capabilities and the human is not in control. And this is like a top tier of, top tier, top quarter of the spectrum to which we can dream of, to which we can be ambitious to gain with AI. – Agata Jałosińska

Most products live somewhere in the middle of these two axes. The design problem is that teams rarely stop to map where they actually are – and so they build interactions they haven’t fully described.

Names Are Design Decisions (Whether You Treat Them That Way or Not)

Autopilot implies you can disengage. Assistant implies it supports you, but you lead. Copilot implies shared responsibility – but with ambiguity about who controls what and when.

And names carry cognitive contracts. Some contracts with people and Autopilot implies you can disengage as a person. I can be not involved. Assistant implies it like it supports me but I lead. Copilot implies shared responsibility. – Agata Jałosińska

Agata is careful to distinguish between naming as a marketing decision and naming as a design decision. A well-designed autonomous car experience can still communicate “you need to stay engaged” – through the seat, the wheel, the dashboard, the entire physical interaction model. The design carries meaning even when the brand name oversells.

The problem is that both designers and users are still learning what “interacting with AI” actually means. Without a clear vocabulary, we reach for metaphors – magic, dreams, the machine coming alive. Those metaphors carry assumptions. And assumptions left unexamined become design failures waiting to happen.

The Control Spectrum: Where Does Your Product Actually Sit?

One of the most practical frameworks Agata brings to her work is around defining control. The ideal isn’t always maximum autonomy – it’s matching the level of human oversight to the actual stakes and complexity of the task.

What we would ideally aim for is high output complexity, high system capacity and high human control. Not leaving it to the autonomous case like the autopilot, but we can deal with it somehow. – Agata Jałosińska

This matters especially as the industry shifts toward agentic AI systems – where AI agents communicate with each other through protocols, and neither party in the interaction is human. In those architectures, the feedback loop to the user becomes critical. If the system doesn’t explain what it’s doing, why it’s doing it, and what it can’t do, the human has no real control — regardless of what the product page says.

When we talk about agentic interaction, it’s impossible to understand what is happening if the system doesn’t talk to you. – Agata Jałosińska

The 19 Guidelines That Aren’t Just for Chatbots

Agata points to a research-backed framework that product teams often overlook: the 19 Guidelines for Human-AI Interaction developed by Saleema Amershi and colleagues at Microsoft.

The guidelines are deceptively practical. They include things like:

• Make clear what the system can and cannot do
• Explain how well the system does what it’s designed to do
• Tell users why the system behaved in a certain way
• Help users understand the consequences of continued use

These aren’t abstract principles. In an agentic context, they translate directly to interface requirements: an agent that surfaces its own limitations, flags when it’s operating outside its intended scope, and gives the user meaningful visibility into its reasoning.

So we want the agent to translate to us, okay, I’m, I don’t know, limiting the number of the activities that I can conduct because the situation is this and this and that and I will explain this to you. – Agata Jałosińska

The word “transparency” gets thrown around a lot in AI design. Agata deliberately avoids it – not because it’s wrong, but because it’s been stretched to mean so many things that it stops being actionable. The guidelines give you something concrete to build against instead.

Why AI Is Harder to Design for Than Anything That Came Before

Every technology creates design challenges. But AI is different in a specific way: the speed of its proliferation has massively outrun our ability to build frameworks for it.

We know like we had first, first ChatGPT was like four years ago, three years ago and the technology is everywhere. This didn’t happen with, with any kind of car, with any kind of, I don’t know, lamp, cell phone, mobile, nothing. – Agata Jałosińska

The result is that designers are trying to prototype for systems they can’t fully predict, with toolkits built for more bounded problems.

So we can have one technology but then giving it to 10 users and the technology can learn. It’s very difficult to prototype for these kind of situations. It’s very difficult to come up with scenarios that can, that can happen. So designers also face a problem with like maybe not lack of toolkits, but current toolkits being quite difficult to apply to this situation. – Agata Jałosińska

That’s not an argument for slowing down. It’s an argument for investing in frameworks early – and for being honest about what you don’t yet know.

What a Researcher Brings to a Product Team

Agata is open to collaborations with product teams, and she’s direct about what that looks like at its best.

It’s not about importing academic knowledge. It’s about having someone who can organize the chaos – name what’s happening in spaces that don’t have names yet, identify which questions are most important, and map what connects to what.

I think the problem with collaborations is that we very often focus on only what I would like to have from it, but not how my work could actually support this specific project. – Agata Jałosińska

What me as an academic researcher can bring is absolutely the competence of organizing chaos and to being able to name what is happening in these unknown spaces. – Agata Jałosińska

The most useful thing an outside researcher can do, she says, isn’t fill a knowledge gap. It’s slow the team down long enough to ask the right questions – and find the holes before users do.

Sometimes I say that even our teams, our designers are people not designing as drawing, but designing as asking questions. – Agata Jałosińska

Key Takeaways

AI is not like other things we’ve designed for. It’s not just the complexity of the system — it’s the speed of adoption, the openness of the capability space, and the fact that the system keeps changing based on who uses it.

Designing well for it means:

• Mapping your product on two axes – system capacity and output complexity – before you name it, brand it, or prototype it.
• Treating names as design decisions, not just marketing. The word you choose creates expectations users will act on.
• Being explicit about control – where it sits, what it looks like, and what happens when the system operates outside its intended range.
• Building toward transparency in practical terms: what can the system do, how well does it do it, why did it behave that way, what happens next.
• Bringing outside perspective in early – whether that’s a researcher, a collaborator, or someone who hasn’t spent three months inside your product assumptions.

We’re in an early period of figuring out how to design human-AI interaction well. The frameworks exist. The research is there. The challenge is applying it in real products, at real speed – without borrowing metaphors that carry more promise than the product can deliver.