Shuai Liu

Always-on personal AI · systems survey · 2026

A Survey towards a Continuing and Self-Evolving Loop

How should an always-on assistant close the loop without turning persistent access into uncontrolled authority?

Proposal

Problem

Assistants are leaving the chat window for continuous interfaces: glasses, earbuds, browsers, operating systems. None of the adjacent literatures asks the question this creates: how is a task noticed in a continuous stream, and whether, when, and how should the system act at all? Each community ships one strong module and inherits silent assumptions about the others; the assumptions collide on a shared always-on interface.

Current work

Why significant

Four stakes, in increasing order of ambition:

  1. It names the actual bottleneck. The field's implicit roadmap is "scale each module"; the survey's thesis is that the binding constraint is integration under privacy, budget, and interruption limits — coupling policies, not module capability. If that thesis is right, the next unit of progress is a policy that moves a payload across a channel, and the survey is the first map drawn at that granularity.
  2. It converts "closed loop" from marketing into an auditable claim. LoopAudit gives the claim teeth: evidence levels E1–E4, five paired channel interventions, adapter and replay contracts, and a driving-automation-style L0–L5 closure ladder where a system's level is its longest closed prefix. Every surveyed system becomes a checkable position instead of a prose summary — and future system cards have a yardstick their authors did not choose themselves.
  3. It separates capability racing from closure lag. Execution skill has raced ahead to delegated action while personal-state closure stalls at conditioned recall; the ladder makes this asymmetry visible and names the unoccupied tier (outcome-adaptive assistance, L5) as a concrete research target rather than a vibe.
  4. The timing is a forcing function. Glasses, earbuds, and OS-level agents are shipping now; the always-on surface will exist before the field agrees on how to evaluate it. A framework that arrives before standardization gets to set the terms — and the loop-vs-ledger split against the scooping survey means the two works partition the space rather than collide.

Core hypotheses

  1. The binding constraint is integration, not capability. Perception, memory, and tool use are each well served by foundation models; what fails is the coupling between them under privacy, budget, and interruption limits.
  2. Coupling must be evidenced, not asserted. A camera, a vector store, and a planner can coexist while no payload demonstrably crosses between them. Channel-local intervention is the only attribution that separates a genuinely closed loop from better zero-shot perception.
  3. Closure lags capability. Execution skill has raced to delegated action while personal-state closure typically stops at conditioned recall, and outcome-adaptive assistance is unoccupied.

Contributions (define → survey → operationalize)

  1. A unifying systems framework — every surveyed work receives a checkable position: module, space-or-policy contribution, channels evidenced, binding regime.
  2. A comprehensive framework-organized review — five communities, module spaces and policies, integrated systems on four deployment surfaces, and the benchmark landscape.
  3. LoopAudit, an executable audit protocol — canonical trace, evidence levels E1–E4, five paired channel interventions, adapter and replay contracts, plus L0–L5 closure levels and a system card.

Idea Trajectory

How the framing evolved

2026-04 · ProEgoMem shelved → survey pivot — The systems paper was parked; its taxonomy and paper list seeded the survey corpus.

2026-04 · Landscape scan: the narrow gate — Deng/Liao's proactive-dialogue trilogy blocked a broad "proactive agents" survey. Interim conclusion: retreat to the Memory × Proactive × Egocentric intersection. superseded

2026-07 · Positioning reversal: the big umbrella — Confirmed the opposite bet: a system-and-applications survey entered through the wearable interface as forcing function, not a niche intersection.

2026-08-05 · Scoop crisis → three-axis differentiation — arXiv 2606.30306 "Always-On Agents" (136 pp) landed on the same phrase. Resolution after full-text autopsy: they audit the ledger (persistent-state lifecycle, legitimacy), we audit the loop (signal around sensing–memory–action, efficacy); their return arc closes through state governance, ours closes through the world; their typed event stream cannot express our two sensing-side channels.

2026-08-08 · Framing freeze — Rejected for v1: ten-policy top-level factorization, fast/slow dual-loop reserved for RSI position paper, action-as-world-model world models demoted to one anticipation mechanism, past/present/future as chapter structure demoted to interpretive layer.

2026-08-14 · The policy-coupled restructure — Channels renamed and given named coupling policies (Gating → M→A, Memory Prior → M→S, each with a κ-policy); modules became policy-bearing functions with a shared space/policy lens; LoopAudit became its own chapter; the old headline counts were withdrawn pending recoding — the former Gate column had mixed act-or-silence gating with memory-to-action conditioning.

2026-08-19 · L0–L5 closure levels — A driving-automation-style ladder anchored in the channel chain: a system's level is the longest closed prefix, judged by intervention evidence, with capability and proof burden rising together. Operating regimes were simultaneously narrowed from a global organizing axis to a diagnostic: same loop, different binding question. regimes-as-structure dropped

2026-08-20 · Claims vs. verdicts — The ingestion stage's channel annotations were demoted from audit findings to claims: unclaimed channels report "not claimed (absence not verified)", manipulation checks void mis-targeted interventions, and the deterministic engine signs only what it ran.