Designing Agents forReal-World Spaces:Tools, Execution, andMemory at seenzus 为真实空间设计 Agent：seenzus 的工具、执行与记忆

A user asked seenzus to turn a light blue. Our tool didn’t yet support color parameters, so the model tried changing the brightness instead. Dimming a light won’t turn it blue.

Some lights only switch on and off; others support color. The agent needs to know which kind it’s dealing with. After it sends a command, the interface may accept it without confirming that the light has turned blue. Even if the operation works, should the color chosen for tonight’s party still apply tomorrow?

We changed the tools to take some of that guesswork out of the model’s job. They tell it what the device can do and which parameters to use, then report what the execution evidence confirms. The model still has to understand the request and choose an action. Code can handle device rules and parameter checks. Memory needs to keep conditions such as “only tonight” with the request.

A personal agent connected to HA can already do a lot. We read the implementations of REST skills for OpenClaw and Hermes’s built-in tools, along with ha-mcp’s search and control confirmation. ha-mcp has already done considerable work on the problems specific to home control.¹²³

Tools should explain device capabilities and execution outcomes

Take “turn that light in the study blue.” Once device connections and space assignments are configured, the agent can locate the target by space, read its capabilities, and decide how to act. When the result comes back, it needs to decide what it can confirm to the user.

Locate the device before reading its details

A home may contain dozens or hundreds of device entities. Finding a study light doesn’t require reading all their attributes. Search returns candidate locations and the identifiers needed for subsequent calls. Once the target is clear, the agent reads its state and action specifications.

If there are two lights in the study and nothing in the conversation identifies “that light,” the agent needs to ask. If the user explicitly requests all the study lights, it should use that scope. We’ve made this distinction in the tool descriptions and prompt so that multiple matches don’t automatically trigger a clarification question.

Let tools prepare the parameters each device supports

To control a light, the model needs to know what this particular one can do. Reading state in seenzus also returns the actions the device can currently execute, with parameter formats and valid ranges. If color control is available, the result includes color parameters. If the current integration only exposes on/off control, the result says so.

The capability description shown to the agent and the executor’s parameter checks use the same definitions. Before dispatch, the executor reads the device’s capabilities again and rejects unsupported operations with an explanation. We don’t want the tool to describe one set of rules while the executor checks another.

Hermes’s service lookup provides general field descriptions for a device category, and ha-mcp can expand full field specifications. The model still needs to check these against the individual device’s attributes to work out which parameters apply.²³ seenzus does that comparison inside the tool and returns the device’s available actions, saving the model from assembling them again on each request.

A brightness value can be perfectly valid and still have nothing to do with the blue the user requested. The prompt has to address that too. We tell the agent to explain a capability limitation and obtain agreement before substituting a different action. Parameter validation checks whether a call is valid; it can’t decide whether the agent has departed from the user’s intent.

If an interface doesn’t report color control, the integration may simply not expose it. The agent shouldn’t tell the user the light’s hardware cannot change color on that evidence alone.

Sending the command doesn’t settle the result

After a control request, seenzus returns a result based on the execution evidence it has received.

• Confirmed means state evidence confirms the target was reached, within the evidence the integration can provide.
• Accepted means the integration accepted the request; whether the device reached the target is still unknown.
• Timed out without confirmation means the command may have been sent, but the result is unknown.
• Already satisfied means the device was already at the target state, so no duplicate command was sent.
• Not sent means the device or connection was unavailable and the request wasn’t dispatched.

Confirming that a light is on isn’t enough to tell the user it’s blue. A timeout without confirmation calls for checking state before retrying. With a toggle operation, an extra call could switch the light straight back off. Whether the model should keep looking or finish the request depends on the evidence in that result.

ha-mcp already provides state confirmation, distinguishing dispatched, confirmed, and partially completed operations. That’s useful work to learn from.³ We also found a Hermes user report about a TV script that took 20 to 40 seconds. The tool timed out at 15 seconds, while HA subsequently completed execution.⁴ In that situation, the tool needs to say the result is still unknown. Otherwise, the model might assume nothing happened and try again.

Which tokens can we save?

We used to insert a device list into the context automatically. We thought knowing what was in the home would make later requests easier. But a request to turn on one air conditioner could give the model information about dozens of unrelated devices first.

We’ve removed that automatic injection. If the target is known, the agent can read its state directly; otherwise, it searches first. Tools for browsing a space or the full directory remain available when needed. Compared with our own earlier version, switching on one air conditioner no longer means sending the model unrelated device details through this path.

We went back over what each tool returns too. Location helps the model recognize a search candidate; the identifier needs to stay so it can make the next call. At the state-reading stage, it needs the action parameters. When deciding whether to keep a field, we ask whether the model will need it next.

We nearly cut too much from those results. A light can retain RGB values while operating in color-temperature mode. When compacting the state response, we omitted the current mode. Review caught the risk that the remaining values could mislead the model about the current color, so we put the field back. That low-level detail tells the model which values are actually in effect.

Action parameter formats and ranges are worth keeping for the same reason. The model needs them for its next step. Cutting them only to require another query may cost more tokens. We now consider the whole task, including additional queries and retries after mistakes.

Some decisions don’t need repeating at all. For “turn the light off in ten minutes,” the device and parameters can be settled in the conversation and saved as a fixed, authorized command. An executor runs it when it’s due. The model doesn’t need to decide again which light to switch off or which parameters to use. Tasks that still require judgment at execution time continue to use a model. Hermes also provides cron scheduling.⁵ In seenzus, we separate these two kinds of task. Running a settled command directly saves that model invocation.

We think this preparation matters more as the number and variety of devices grow. The model has less unrelated information to read and fewer device rules to assemble. A command that’s already settled doesn’t need another model decision. Those are specific places to save tokens and waiting time, though measuring the saving against another well-tuned setup still requires matched conditions.

Memory must preserve conditions and respond to corrections

Suppose the user adds, “Only for tonight’s party. I normally want warm light.” Memory needs to keep “tonight’s party” with the request. Retrieve “the user likes blue” next time without that condition, and you’ve remembered it incorrectly.

OpenClaw and Hermes also provide long-term memory and update mechanisms.⁶ In seenzus, we need to decide where personal preferences and room information belong, and whether a temporary request should apply later. Our memory read/write rules distinguish personal and environmental information and require the conditions to be kept with it.

“I get cold easily” can belong in personal memory. A resident’s statement that the study gets afternoon sun can belong in environmental memory, available to members with the appropriate access. Mentioning a room doesn’t make private information shareable. Using a personal preference to guide an authorized action in the living room doesn’t make that preference a rule for everyone either.

Who can read a memory is a question of privacy. Whether a bedroom temperature preference applies in the living room depends on its conditions. They’re different questions. Personal memory follows the user, but must retain those conditions; “a little cooler tonight” shouldn’t overwrite a standing preference.

When a user says, “I got that wrong earlier; I actually prefer warm light,” the old memory needs to change. Appending another sentence leaves the next conversation to reconcile two contradictory records. seenzus’s learning process can use execution records and user feedback to check and revise its understanding. A light turning blue establishes an execution result. Whether the user liked the outcome requires their feedback.

The learning process reviews those execution records and feedback, revises memory where needed, and leaves it available for later service. We want a correction to give the agent a basis for doing things differently, without making the user explain the history again. It can still miss information or get the interpretation wrong. Successful execution proves the device did something; it doesn’t prove the user prefers it that way.

Remembering a habit doesn’t give the agent permission to act on it. Repeatedly seeing someone turn the lights off at ten can support a suggestion. Turning them off every night at ten requires separate authorization; confirming the habit isn’t an execution permit.

Don’t ask the model to redo work code can handle

The model still makes mistakes after this work. In tests, it sometimes chooses a poor search term or stops before it has finished looking. Giving it a way to keep searching doesn’t guarantee it will use it. We still need to work on that.

We want to spend our time on these details. The model can interpret what someone asks and choose an action without having to reconstruct device rules on every request. seenzus’s tools handle the rules we can establish in code. Its memory keeps the conditions attached to a request, and execution records and feedback help correct what it has learned. That division of work is built into the Space Agent users get, rather than left for each conversation to work out again.

Notes and sources

1. An OpenClaw community member's REST skill description, published on the Home Assistant forum on February 26, 2026. It instructs the agent to use credentials with HA GET/POST endpoints, rather than providing a separate typed tool implementation. The complete public example is an implementation sample, not a performance baseline. ↩
2. Hermes (NousResearch/hermes-agent), homeassistant_tool.py, inspected September 17, 2026. Tools include ha_list_entities (domain/area filtering and projection to identifier, state, and name), ha_get_state (full attributes for one entity), ha_list_services (domain-level service and field descriptions, with selector metadata omitted), and ha_call_service (format validation, a blocklist covering six domains including shell_command, and success/service/affected_entities output). Domain matching uses the entity-ID prefix; area matching uses name/attribute substrings rather than an area-registry lookup. The POST timeout constant is 15 seconds. ↩
3. ha-mcp (homeassistant-ai/ha-mcp), README.md, master snapshot 026647e, with pyproject.toml declaring 8.5.0; inspected September 17, 2026. Search supports area/floor/alias filtering and fuzzy matching in tools_search.py, plus limit/offset and result_fields projection. Service lookup can return full field schemas with detail_level=full in tools_services.py. Control results distinguish dispatched/confirmed/partial/transitions in tools_service.py. Approval policies include a Read Only mode and per-tool switches in middleware.py. ha-mcp is a community project, not HA's official MCP or an official OpenClaw plugin. ↩
4. Hermes issue #96121, reported August 27, 2026. The user described a TV/media-player script taking roughly 20 to 40 seconds, exceeding Hermes's 15-second timeout while HA subsequently completed execution. The issue was still open when inspected. This is one user's account, not a general benchmark; the 15-second POST timeout was verified in source. ↩
5. Hermes's official cron.md describes native scheduling for one-off and recurring tasks, with skill bindings. We did not audit its scheduled executor and make no claim that it must invoke a model at execution time. Inspected September 17, 2026. ↩
6. OpenClaw's official memory.md covers USER.md, MEMORY.md, dated notes, and background organization; skills.md describes skill creation and tool-use guidance. Hermes's official memory.md describes profile memory, session search, memory/skill updates after corrections, and write approval. These capabilities are documented by the projects; we did not audit their complete learning implementations. Inspected September 17, 2026. ↩