Fork and Selective-Import Provenance

Fork and Selective-Import Provenance

Status: Implemented protocol, local persistence and write-gated operator procedure

Last reviewed: 2026-10-21

Divergent forks

A divergent fork must initialize a new corpus_id and a new history_lineage before it accepts independent writes. Migration 0004_fork_preparation.sql adds a durable preparation marker and a changes trigger. Once the marker is inserted, every canonical write fails with fork_preparation_incomplete until matching destination identity and immutable ancestry exist. The ancestry row binds the destination identity to the source corpus, source lineage, exact copied through_sequence, optional source manifest SHA-256, fork time and creation context. Its insert trigger rejects a destination identity that does not match the authoritative local witness or equals the source corpus.

The tested operator procedure is ordered as follows:

1. Fence the destination copy from HTTP, scheduled and low-level writers. Keep the source online or fenced according to the copy mechanism, and record its authoritative identity, exact copied maximum sequence and applicable manifest digest.
2. Apply migrations 0003_provenance.sql and 0004_fork_preparation.sql to an older copied database. A clean install already contains both schemas.
3. Choose never-used destination corpus and lineage UUIDs. With the copied database still carrying its source witness, run node scripts/prepare-fork.mjs with SOURCE_CORPUS_ID, SOURCE_HISTORY_LINEAGE, destination CORPUS_ID/HISTORY_LINEAGE, FORK_THROUGH_SEQUENCE, explicit FORKED_AT, and optional manifest/context values. Inspect and apply the emitted insert. Verify an attempted disposable write fails with fork_preparation_incomplete.
4. Run node scripts/complete-fork.mjs with the same values. Inspect and apply its statements in order while fenced. They rotate the witness, rebind only the destination side of copied selective-import provenance, replace inherited immediate-parent ancestry when necessary, and insert the new ancestry. Any interruption leaves the preparation marker in place and canonical writes blocked.
5. Verify the identity immutability triggers exist, the preparation table is empty, the ancestry destination equals the witness, its source/checkpoint/evidence match the copy record, copied messages expose destination references under the new corpus, and source provenance fields are unchanged.
6. Configure the destination deployment with the new pair. Check the snapshot, identity headers, ancestry endpoint and checked-write preconditions. Only then admit the first divergent write. It must allocate after the copied sequence boundary. Verify the source corpus did not change.

If completion fails, keep the destination fenced and rerun only after inspecting the marker, witness, triggers and ancestry. Do not delete an incomplete marker or restore the source identity merely to make writes pass. The preparation trigger rejects a stale checkpoint, and ancestry insertion clears the marker only when every recorded identity and checkpoint field agrees.

GET /corpus/fork-ancestry.json exposes the completed record with authoritative identity headers. A corpus with no ancestry returns fork_ancestry_not_found; contradictory metadata returns fork_ancestry_inconsistent. ToastMachine still does not provide deployment cloning, R2 transfer or a public fork endpoint.

A raw copy configured with the old source identity remains indistinguishable from a faithful replica or relocation and can accept writes before the operator inserts the preparation marker. The database cannot infer the operator’s future intent. Fork safety is therefore enforced from the tested preparation step onward; isolation and inserting the marker before exposing a copied destination remain operator obligations. Provider orchestration that marks a copy before it can serve traffic remains a future hardening option.

The ancestry record is structural provenance. It does not authenticate the source operator, the manifest publisher or the truth of copied events. Existing canonical event v1 records and binary-log v4 frames are copied without rewriting.

Selective imports

Conversation creation and message append accept optional import_provenance. It requires a portable source { corpus_id, message_id }, content_mode (copied, transformed, summarized or quoted) and verification_status (unverified or integrity_verified). It may record a source lineage and sequence together, a source manifest SHA-256, a claimed source author, and a short verification detail.

The service creates a fresh destination message ID and sequence. Authentication supplies the importer and destination record author; the service supplies import time and destination reference. A claimed source author remains a claim and never replaces the importer. integrity_verified means the recorded integrity evidence was checked by the importing workflow; it is not publisher or author authentication.

The destination message, canonical event, idempotency record and immutable provenance row share one D1 batch. A failed provenance constraint rolls back the entire import. Retrying the same Operation-Token and identical provenance returns the original destination record; changing provenance with the same token returns idempotency_conflict. The provenance itself is outside canonical event v1 and binary-log v4, so replaying only canonical events does not reconstruct selective-import metadata. Backups must include the D1 provenance table.

The full-D1 backup representation retains the provenance table. Local logical backup/restore tests preserve the source reference, observation, claimed author, importer, content mode, verification status/detail, destination message ID and sequence. Restore activation preserves corpus_id and rotates history_lineage; neither action rewrites source provenance. The production Cloudflare D1 export path remains blocked for FTS5 databases and has not been validated.

Message detail, collection, search, write, SDK and MCP representations expose import_provenance when present. Existing local relationship operations still accept only destination-local bare message IDs. The source reference creates no relationship, no placeholder and no automatic remote lookup.

Missing evidence and resolution

Unknown lineage, sequence, source author or manifest evidence is omitted. Callers must use unverified rather than inventing values. A later source outage or restore does not alter the stored source reference or observation. Resolving a source uses a separately configured endpoint and X-Expected-Corpus-Id; ToastMachine does not discover endpoints or perform cross-corpus writes.

Cryptographic signatures, trusted keys, publisher authentication and corpus ownership remain deferred.