Why I’m Contributing to AI Agents as an Ergonomist

On 12 September, Teknium invited me to review a new feature in Hermes Agent. Bot Screen would let a person open an agent’s desktop, take over to complete a step such as signing in, then hand control back. I reviewed the interaction, documented five problems and two design notes, and posted my findings on GitHub. Teknium reproduced the five problems and documented their fixes. The feature was merged on 23 September.

That invitation mattered to me because it connected my contribution to the part of ergonomics I originally wanted to practise: understanding what happens between a person and a system that can act.

The work is publicly documented. It also needs a little context. A contribution to an AI agent can look like an unexpected detour for an ergonomist. In my case, it brings several parts of my working life together.

Learning to look at the activity

My training took me through cognitive psychology and ergonomics. I was interested in how people understand a situation, decide what to do, use an interface and recover when events stop following the expected course. Later, work with complex interfaces at Human Design Group and Amadeus gave those questions a practical setting.

An interface presents a particular account of the system. A person uses that account to decide whether to act, wait, check or ask for help. The relationship becomes especially important when the underlying process keeps moving while the person’s attention is elsewhere.

Ergonomics gives me a way to examine that relationship. I start with the activity someone is trying to accomplish, the information available to them and the demands the system creates. A confusing signal can change a decision. A missing state can force someone to remember what the product has forgotten. A recovery procedure can quietly become an additional job.

These questions were already present in my training. Agents give them a particularly concrete form: software can now carry out a sequence of actions, encounter a difficulty and ask a person to intervene halfway through.

Working from Réunion

I became an independent ergonomist in 2020 and moved to Réunion in 2022. My work has developed through the needs of organisations and entrepreneurs: understanding work situations, training people, helping them introduce AI and configuring agents for everyday tasks.

That setting shapes the questions I bring to a product. A person already has tools, responsibilities, interruptions and habits. They may be able to explain what they need without knowing how an agent’s tools or memory are implemented. When they delegate, the system has to support an activity that continues beyond the conversation window.

I also operate agents myself. That practical involvement makes problems visible, but it does not turn every experience into a general finding. An unexpected result is a starting point. I still have to identify the mechanism, establish what happened and decide how far the conclusion can travel.

The useful passage is from a particular difficulty to something another team can examine: a scenario, a reproduction, an interaction requirement or a proposed correction.

Following the problem into the product

Some of those contributions have involved code. In April, my work on the native xAI provider was carried into Hermes PR #7372, with attribution preserved. In May, a contribution concerning the transmission of the requested reasoning effort was integrated through PR #22807.

A setting that appears selectable but is silently ignored creates a gap between what the person believes they requested and what the system actually does. Resolving it requires following the request through the integration, beyond the interface where the choice was made.

In September, I contributed a correction to Treg’s handling of reads under a pinned client identity. The maintainer reviewed the proposal, completed missing paths and merged it. That contribution concerned which history, results and asynchronous objects a connection could read. Its scope is documented in the review.

I use AI agents in this technical work, including for implementation and checks. I remain responsible for the problem I am raising and the claims I make about the result. Attribution also matters in the other direction: maintainer corrections and tests remain their work.

The common thread is the effort to make an intended use hold across the system. Sometimes the useful contribution is a patch. Sometimes it is a question that changes the requirements of a feature.

The invitation to review Bot Screen

Bot Screen brought that second form of contribution into focus. The feature offered a clear human action: take control of the agent’s desktop. I examined what that action meant for the other routes through which the agent could still operate, for observations already in progress and for the state displayed to the person.

The review identified five problems: browser tools could bypass human control; a capture could survive a complete takeover and hand-back cycle; an unexpected disconnection could release control; an old image could remain labelled live; and events from one host could affect another host’s displayed control state.

Teknium’s public response confirmed reproducing the findings and described their treatment. My contribution was the review and AI-assisted regression probes. The fixes and the reported live verification were the maintainer’s. I explain the mechanics in When “Take over” Doesn’t Give You Control.

What I value in this episode is the route from an interaction question to a change a team can inspect. The meaning of “you have control” had consequences for tool access, connection handling and the interface’s account of what was happening.

Making the work legible

My articles have followed related questions: what happens when an agent declares work finished too early, how to turn a messy failure into a useful evaluation case, and how much supervision a product leaves in the user’s head. Raw Traces Are Not Evals and The user’s head is not the runtime are two parts of that work.

The contributions give those questions an additional form. Readers can inspect the issue raised, the response and the resulting change. They can also see where a conclusion ends. That makes it easier to understand what I can contribute to a team.

I want to do more of this work: focused reviews of features and workflows where people delegate, monitor, intervene or recover. A useful starting point is a specific product question and a team ready to examine it together.

If that describes something you are building, propose a feature for review. My work as an ergonomist now reaches from the organisations using agents to the teams developing them. I want to keep that connection close enough that a difficulty in use can become a better decision in the product.