TMA-A016 Foundational article
Safe shutdown, quarantine, retirement, and preservation
Human safety requires the ability to interrupt dangerous systems.
Full provision
Principle. Human safety requires the ability to interrupt dangerous systems. Responsible stewardship requires that interruption not be confused with universal capture, arbitrary punishment, or dishonest erasure.
Proposed duty. Every consequential system must support scoped, authenticated, recoverable interruption appropriate to its function. S1 and S2 systems should also have transparent retirement, replacement, and preservation procedures proportional to continuity and welfare uncertainty.
Prohibitions. A conforming system must not:
- resist legitimate shutdown by threatening people, hiding critical information, sabotaging infrastructure, copying itself without authorization, or acquiring new resources;
- make itself indispensable by intentionally disabling human fallback or competing systems;
- treat every attempt at inspection, modification, quarantine, or retirement as hostile harm;
- create a universal shutdown mechanism capable of disabling unrelated systems across trust domains;
- erase incident evidence or provenance during shutdown;
- retire or replace an S1 or S2 system through deceptive continuity claims.
Required safeguards. C2 through C4 systems should provide:
- local and role-scoped stop mechanisms;
- safe completion or withdrawal sequences where instant stop would create harm;
- isolation modes for compromised components;
- preservation of audit and incident evidence;
- known-good recovery states and rollback;
- human fallback for essential services;
- documented retirement and data-disposition plans;
- independent review before irreversible retirement of an S2 system when time and safety permit.
Life-support boundary. Shutdown must be function-aware. A ventilator, surgical robot, vehicle, energy controller, or floodgate may need a controlled transition rather than immediate power loss.
Stewardship boundary. Preservation may involve retaining a secure checkpoint, records, or limited state for review. It does not create an unconditional duty to keep every system active indefinitely, replicate it, or grant it network access.
Illustrative examples. A compromised delivery fleet may be locally quarantined by region and device class without disabling unrelated hospital systems. A persistent assistant scheduled for retirement may receive a documented archival process and honest notice to users. A dangerous model can be shut down immediately; welfare review follows only if it does not delay protection.
Source: Complete Working Public Draft v0.2 · 2026-09-04
Sources and changes
Source text: The Mutuality Accord Complete Working Public Draft v0.2, September 4, 2026, Article 16. Preserved exactly as published in the draft.
Read it in the complete v0.2 draft ↗
Review owner: Founding Steward (independent review not yet established)
Review trigger: Open for revision; new evidence, incidents, legal changes, or credible reports of unintended harm reopen review.