saved
How We Built Safety Into Muse
Hraness cites a source capture. The source author remains the source.
gist
Meta describes Muse as a personal agent that assumes the harness may be under attack. The Hatch daemon runs in a systemd-nspawn runtime cell without real credentials; host-side Sentinel alone authorizes connectors and network egress, inserting secrets only at the boundary via surrogate tokens. Defense in depth stacks model training, untrusted-input labeling, prompt-injection classifiers, human approvals, and eBPF tainted-egress tracking. A public bug bounty pays up to $300,000, and a Confidential VM mode is planned so even Meta cannot access the guest.
ideas
- Assume the agent is compromised and bound damage. Isolate the harness in a cell, hide real credentials, and force every external action through a Sentinel the agent cannot override.
- Two security domains on one box. Hatch and workspace tools run in nspawn; hatch-safety, privsep, authd, Sentinel, and durable state stay on the host with SO_PEERCRED sockets.
- Sentinel owns permission and egress. Connector actions and every network request are allow, deny, or ask; surrogate tokens become real credentials only after authorization.
- Tainted egress cuts approval noise. Processes that read user data lose auto-allow; clean, policy-matching requests can proceed without bothering the user.
- Prompt injection gets layered, not solved. Model resistance, harness labeling, classifier ensembles, and deterministic boundaries still assume mistakes and open a $300k bounty.
quotes
“The right mental model is two isolated security domains on one box, not an LLM powered agent with root.”
“Sentinel is the sole permission authority for connector actions and network egress.”
“Approvals granted via the human in the loop system are strict capabilities, not conversational suggestions.”
“We’ve designed the system to bound the impact when things go wrong and help the user stay in control without being overwhelmed.”