The next layer of agent context lies beyond the screen

Getting an agent to turn on an air conditioner is no longer much of a problem. A model interprets the request, a tool finds the device, and the command goes out within seconds.

We got stuck on a different request: “Make it more comfortable in here.” Which room is “here”? What is the temperature now? Is a window open? Who is in the room, and what does comfortable usually mean to that person? Did somebody change the device a minute ago? Knowing how to call an air-conditioning tool does not bring any of that information along for free.

Today’s models are already good at reading conversations, documents, code, and web pages. seenzus is trying to add another layer of context beyond the screen: places, spaces, devices, what is changing now, and what a space has taught the agent over time. We want more of the physical world to become readable and understandable to an agent, and to remain available when it wakes again. Device control is only the first application built on that context.

Today’s agents mostly read the world brought into their window

Agent context usually means the material made available for one inference: the system prompt, this conversation, a few documents, relevant files from a codebase, search results, or tool output. The model has a finite window, so the product must decide what to preload, what to fetch later, and what should not compete for attention. Anthropic describes context engineering in similar terms: selecting a limited set of high-signal material from a changing universe of potentially useful information.¹

This context is indispensable. It lets a general model enter a specific job and understand what the user has already said. It can also carry the local truth of a company or a codebase that the model could never know from training alone.

It still has a common entrance. Someone must write, upload, retrieve, or return a piece of information before the model can use it. Even when an agent searches on its own, it mostly searches material that already exists in digital systems. A document does not open a window between turns. A repository does not become colder after sunset.

The physical world is not another file format. It keeps happening.

The physical world does not wait for the next message

After a user closes a conversation, the room continues to change. The sun moves west. The temperature rises. Someone opens a window, a device falls offline, a café starts closing, or a child falls asleep in the next room. When the agent wakes again, it is not looking at the next page of the previous conversation. It is returning to a place that has already moved on by half an hour, a day, or a month.

Physical context is therefore situated in space and time. The same device state means something different in a bedroom and a café. “Turn on the light” means something different when somebody arrives than it does after everyone has gone to bed. A space is also shared. Actions, devices, and habits from several people meet there, unlike a private document owned by one person.

Anind Dey and Gregory Abowd’s 1999 definition of context remains useful here. They described context as information that characterizes the situation of an entity, and a context-aware system as one that uses relevant context to provide information or services for the user’s task.² In a physical space, the relevant material rarely comes from the current sentence alone. It may also come from live device state, recent operations, the purpose of the room, or a preference a resident stated earlier.

For that reason, we do not treat access to real devices as proof that an agent understands the physical world. Control gives it a way to touch the place. Context tells it what it is touching.

More physical context gives a space continuity

We do want seenzus to collect more physical-world context. “More” does not mean piling sensor readings into a data lake nobody reads. It means filling in the continuity of a place.

The basic skeleton says which Sites and Spaces exist and which Devices belong in them. Live state and recent operations tell the agent what is happening now. A resident might say that a bedroom gets strong western sun, or the system might gradually notice that its lights are usually dimmed at night. Those softer facts keep the agent from meeting the same room as a stranger every time. Weather, time, outside events, and new device sources can expand what it understands next.

These materials do not need to become one universal table, but they do need to refer to the same environment. seenzus currently uses a spatial skeleton to answer “what is here,” live state and Operation History to answer “what just happened,” and environment or personal Memory to keep experience that may matter later. The same Smart Space Agent uses all of them.

The point of accumulating context is not to make the next prompt longer. It is to avoid reconstructing the whole world at every interaction. The agent already knows which Space “here” refers to. It can distinguish experience that may persist from state that must be read again now. Familiarity comes from that continuity.

A signal is not the same as an understanding. A device may be offline because it lost power or because its network failed. Raising the temperature for several days may not establish a permanent preference. Physical information always has gaps. Long-lived context gives the agent more evidence and more chances to correct itself, but uncertainty remains.

Device control is the first application on this context

Control is the most visible use of physical context. The result is immediate: a light turns on, the air conditioner starts, or a curtain closes. It is also a reasonable place to begin. Understanding has limited value if the agent cannot turn it into service.

If the product stops there, however, the LLM is just a more natural remote control. The user still notices the problem, inspects the current state, decides on the response, and speaks the command. A button has become a sentence, but the product has not taken on much more responsibility.

The long-term goal for seenzus is a living world model of a physical space. Control is the first application on that model. Comfort, energy use, presence, security, and the daily operation of a small business can draw on the same spatial context. Each needs to know what is here, what is happening now, what happened recently, and how the people who use the place want it cared for.

As that context grows, an agent can move from hearing a command toward understanding why it makes sense now. “Make it more comfortable in here” can draw on the current room, live temperature, available equipment, and personal preference. An open window before confirmed rain can first become something worth mentioning instead of another device the user must remember to inspect.

We have not finished all of those capabilities. The choice we have made is about the foundation beneath them. We are strengthening one agent’s understanding of one physical environment so that new services can use the same context, rather than building an unrelated automation stack for every scenario.

Physical context determines the shape of seenzus

Putting physical context at the center changes how the rest of the product fits together.

An agent without physical data can only guess, so seenzus ingests state from different device sources. Raw identifiers cannot tell a model where “here” is, which is why Sites, Spaces, and Devices need an editable spatial skeleton. Operation History catches actions that happen outside conversation. Memory and continued learning cover a different gap: no device manual can fully describe how a room is used or what its residents prefer.

Inside seenzus, conversation, spatial cleaning, scheduled tasks, and learning are handled by different agents. They read from and write back to the same physical-world context. The user still meets one “I”: it watches and acts during the day, reviews experience when appropriate, and wakes up knowing the same place. The internal split exists to maintain continuity, not to create four robots that fail to recognize one another.

This changes how we think about product progress. Adding a tool is useful, but we also ask what new part of reality the agent can read. Does a new device source enter the same spatial language? Can a signal be connected to place, time, and prior events? When a user corrects the agent once, does its next judgment change?

Models will keep improving. What we want to accumulate is the world they can rely on each time they wake.

Richer physical context requires boundaries from the start

“Collecting more physical context” can easily sound like knowing as much as possible. That is a dangerous direction for an agent entering homes, shops, and offices.

Physical context often sits closer to daily life than a document does. It may reveal who uses a Space, when a device is active, or how a person’s schedule and preferences change. Over time, it can tempt a system to infer things the user never said. Collection and boundaries therefore have to grow together. A personal preference cannot silently become a shared fact about the whole space. A Site outside a member’s view cannot enter context merely because the model finds it relevant. Memory cannot stand in for current state. Learning a habit does not authorize an action.

Boundaries also require the product to admit what it cannot see. A sensor may be missing, a device may be offline, and the person behind an operation may be unclear. seenzus needs to preserve “we do not know” rather than filling a gap with old data to make its world model look complete.

We want the context to become richer, but each piece of information still needs an origin, a visibility boundary, and a point at which it must be checked again. Without those limits, “understands you better over time” would quickly feel like surveillance.

We are building toward an agent that becomes familiar with a place

There is no moment when physical-world context is finished. New devices, sensors, external signals, and ordinary actions will keep expanding what the agent can understand. A larger schema or a stronger model cannot accomplish this on its own. Ingestion, organization, Memory, learning, and action all have to keep pointing at the same place.

We still do not know which physical signals deserve priority. We also lack enough long-running use to claim that more context will always improve judgment or reduce interruptions. Collection has no value by itself. It counts only when the next interaction changes, when the user has less to explain or one less thing to inspect.

This is the direction behind seenzus. Many agents live inside windows that open for one task at a time, waiting for someone to turn the world into text. We want an agent to enter a space that keeps changing outside the conversation, to see and remember that place over time, and to revise its understanding when the people there correct it.

In the end, the evidence behind its next judgment should come from more than the model. It should also come from the reality the agent has continued to read in this particular place.

---

Notes and sources

1. Anthropic Applied AI team, Effective context engineering for AI agents, September 29, 2025. The article treats agent context as a finite resource and discusses tradeoffs among prompts, tools, retrieved information, and runtime discovery. We use it to describe the context engineering familiar to current digital agents, not to set the limits of a physical-space agent. Accessed August 12, 2026. ↩
2. Anind K. Dey and Gregory D. Abowd, Towards a Better Understanding of Context and Context-Awareness, Georgia Tech, 1999. The paper defines context as information that characterizes an entity's situation and connects context-aware behavior to the user's task. We draw on that definition when describing the spatial, task-specific, and changing nature of physical context. Accessed August 12, 2026. ↩