Persist and restart — main (unreleased)

Restoring a store directory from a backup requires opening it with a new writer identity before it writes. A new Rust durable store has an independent own-writer counter that refuses an older store at restart. A store made before this check may open under the old identity and detect the restore only when a peer returns an own-ID record above the durable high-water.

v0 scope: G-Counter and OR-Set are the v0 focus types, within the language-path limits. G-Set, PN-Counter, RGA/Text, LWW Register (LwwRegister), Enable-wins Flag (EnableWinsFlag) and LWW Map (LwwMap) are experimental, including their deltas and wrappers. Existing proof/test evidence is unchanged by release status.

Build a small field kit: an observation counter and a membership set. Save 3 and compass, exit the process, restore them, then make independent edits and exchange bytes with a second replica. The final count is 6; membership is compass, map, rope. Both programs assert those values, retained history and the error outcome.

These paths use main (unreleased) source and a locally built WASM package. There is no published SafeMesh package to install. The Rust and TypeScript programs are complete consumer fixtures, separate from the library workspace; their code, commands and stdout are checked by the existing documentation CI job.

Choose the Rust path for Linux durable storage or the TypeScript path (Node) for caller-owned Node storage. Complete Before you start for either route, then jump directly to your chosen path; the optional Rust exercises are separate from the Node route.

Before you start

SafeMesh’s Rust crate floor for consumers is Rust 1.89. For the source builds, demos and locked wasm-pack 0.15.0 installation on this page, use Rust 1.96.1, the prescribed toolchain for this guide and the full-gate CI version. Install rustup first (Linux/Bash, with curl and a native C compiler/linker), then select that toolchain:

Run curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal --default-toolchain 1.96.1, then . "$HOME/.cargo/env" and rustup default 1.96.1.

The default applies to your user account; the repository’s rust-toolchain.toml also selects 1.96.1 inside this checkout. An outside application’s toolchain remains its own choice; consuming the crate requires at least 1.89.

Use Linux x86_64, Bash, Git, internet access, Rust/Cargo through rustup and a native C compiler/linker. The TypeScript path also needs Node.js and npm. The measured versions and exclusions are in environments run. No Lean is needed. The first build downloads build dependencies and may take several minutes.

These fixtures are on main. Obtain the source in an empty working directory:

Run git clone --branch main https://github.com/velvetmonkey/safemesh.git, then cd safemesh.

Run the chosen path from this repository root and stop if a command fails. Each persist invocation needs its exercise directory to be absent. restart runs in a new process and cleans up that exercise’s stores on success. A failed run leaves them for inspection; use a fresh checkout to retry rather than erasing a real writer’s fence/history. Build outputs remain for reuse.

Writer fences

The Linux Rust durable store locks writer-N.fence for one writer. A second process with the same store and writer gets Refused. The lock is in LocalReplica::fresh and DurableReplica::lock_restart_fence in rust/crates/safemesh-crdt/src/local.rs. The test second_process_cannot_write_same_writer in rust/crates/safemesh-crdt/tests/writer_fence_process.rs starts a second process, checks that it writes no bytes, and checks restart after the holder exits.

A Node managed handle is fenced across processes when its caller supplies a Store with a lease. The FileSystemStore.open method in the example Store creates lease.lock exclusively and returns LOCKED if it exists. The test node_writer_fence_two_process_and_killed_holder in rust/crates/safemesh-wasm/tests/node-writer-fence.mjs checks both processes. It also kills the holder and checks that the lock file stays and open remains refused until a person removes it. The example Store’s opening comment has no automatic stale-lock recovery.

The comment above ALLOCATED_AUTHORS in the older WASM replica constructors says they do not fence across processes or tabs. The comment above STRING_ORSET_ALLOCATED_AUTHORS in the Python binding registry says it fences only one process.

A copied or restored store at a separate root has no active lock file there, so either lock alone cannot identify the old writer. A new Rust durable store also has an independent own-writer counter: rollback::Counter::open returns RecoveryRequired on a missing or mismatched counter. A store made before this check can detect an old writer identity when it receives an own-writer record above its durable high-water and returns PeerWriterAhead (see the Peer history cannot allocate IDs comment in local.rs). SafeMesh does not give a restored store a new writer identity. Open it with a new identity before it writes.

Rust gold path

The complete application is in examples/gold-path/rust/. It already adds the local crate by path, including its Linux local-writer feature:

[package]
name = "safemesh-first-use"
version = "0.0.0"
edition = "2021"
publish = false

[dependencies]
safemesh-crdt = { path = "../../../rust/crates/safemesh-crdt", features = ["local-writer"] }

To adapt it, edit examples/gold-path/rust/src/main.rs: the first bump’s 3 is a cumulative observation tally; compass is the member to add. Update the matching assertions for your own application. Keep the two writer IDs distinct (0 and 1) and their configured writer count equal (2).

cargo run --quiet --locked --manifest-path examples/gold-path/rust/Cargo.toml -- persist
cargo run --quiet --locked --manifest-path examples/gold-path/rust/Cargo.toml -- restart

The first command prints saved; the second process prints the remaining lines. Stdout is exactly:

saved counter=3 members=[compass]
restored counter=3 members=[compass]
synced counter=6 members=[compass,map,rope] records=3+3
malformed record: UnexpectedEof
cleaned exercise stores

DurableReplica::counter / utf8_set create the stores. Each successful single-record edit appends and syncs its own record; restart_counter / restart_utf8_set validate and replay the existing store while reacquiring the writer. Each accepted single-record write of the local writer adds one counter file sync and one directory sync. A DurableReplica::append_batch or counter bump_batch writes the counter once for the whole batch, then writes its journal entries and syncs the journal once. In a 10,000-append release run on one box, medians were 111.930 s on main for single bumps, 105.856 s on this change for single bumps, 1.090 s for 100 batches of 100, and 0.128 s for 10 batches of 1,000. Relative to main, the ratios were 0.946, 0.00974, and 0.00114. The main pre-run CPU sample dipped to 78 percent idle, so these ratios are indicative. For an existing counter, restart_counter_from_store(root, writer) reads the committed count under that writer’s lock and runs the same checked replay, while explicit restart_counter still rejects a mismatched count. Their collection budget comes from the locally stored history length, so an existing OR-Set store with a Remove of more than 4,096 tokens still uses ordinary restart_utf8_set. Peer wire decoding keeps the 4,096-element default. A copied store file has no authenticated provenance; validate its source before using it as local history. After restart, a tally of 4 and the peer’s 2 yield 6. The membership writer allocates a fresh token for each add. Counter and membership use separate stores; there is no transaction across both objects.

The malformed-record line comes from decoding an empty byte slice before admission: UnexpectedEof is the exact Rust debug error. The fixture asserts that the log is unchanged. Reject that input and inspect the sender/framing; do not apply its payload or reset your store.

Read the complete Rust source and storage boundary.

Diagnose a failed Rust exercise

The fixture uses expect/unwrap, so failures appear as a Rust panic (exit 101). On a second persist, an existing store returns StoreExists (Display: local store already exists for this writer; restart it, or create with a new root or writer ID). Check whether .gold-rust existed before this run: if it did, keep that store and run restart with its original writer configuration, or use a fresh checkout for a separate exercise.

Refused on restart (Display: local writer operation refused by ownership or lease checks) means another process still holds this writer’s fence lock. Stop that owner and wait for it to release the lock, then retry restart with the same store and writer configuration. Keep the fence file in place; deleting it does not release the other owner’s lock or restore ownership safely.

A truncated writer fence on restart returns RecoveryRequired (Display: local store requires recovery: its writer fence, journal header, rollback counter or migration files are damaged or do not agree). If the store was already present and this was a restart, inspect its writer-0.fence files: each must be 24 bytes. Retain a damaged store for inspection and recover only from a known consistent copy with its original writer identity and allocation metadata. Never reset an existing writer’s fence/history.

Configuration on restart (Display: invalid or mismatched local writer configuration) means the requested writer configuration disagrees with stored identity or allocation metadata. In your own application, check the writer and writers fields of the WriterConfig passed to restart against the values used when the store was created, then retry with the original values. This gold-path example fixes them at writer: 0 and writers: 2; its commands do not take a writer flag. If the stored fence or transaction metadata itself differs, retain the store for inspection and recover only from a known consistent copy with its original writer identity and allocation metadata. Do not initialize over the affected store.

Io(Os { code: 20, kind: NotADirectory, message: "Not a directory" }) on restart (Display: local store I/O failed: Not a directory (os error 20)) can mean an expected store directory is a file on Linux. Check that the store path is a directory. Retain the affected path for inspection and recover the directory only from a known consistent copy with its original writer identity and allocation metadata; then retry restart. Do not initialize over it.

Io(Os { code: 2, kind: NotFound, message: "No such file or directory" }) on restart (Display: local store I/O failed: No such file or directory (os error 2)) can mean a stored writer transaction is missing. Check that both the writer fence and transaction are present in each store. Retain what remains for inspection and recover the missing file only from a known consistent copy with its original writer identity and allocation metadata; then retry restart. Do not initialize over the store.

InvalidHistory on restart (Display: local history failed replay or sequence validation) can mean the transaction’s stored sequence does not match its replayed history. Retain the store for inspection; recover only from a known consistent copy with its original writer identity and allocation metadata, then retry restart. Do not initialize over the store.

Exhausted on restart (Display: local writer sequence, generation, or token allocation exhausted) can mean the stored fence generation cannot advance. Retain the store for inspection and check its generation and allocation metadata. Recover only from a known consistent copy with its original writer identity and allocation metadata, then retry restart; do not reset the fence or initialize over the store.

History(UnexpectedEof) on restart (Display: local history wire validation failed: unexpected end of wire input) means the stored transaction is truncated or incomplete. History(IntegrityMismatch) (Display: local history wire validation failed: wire frame integrity check failed) means a stored transaction failed its integrity check. Retain either damaged store for inspection; restart does not salvage a torn or corrupted record. Investigate the failed write or transfer, and recover only from a known consistent history with its original identity and allocation metadata. Do not initialize over the damaged store. These are tested engineering outcomes, not proofs of crash safety. History(TrailingBytes) (Display: local history wire validation failed: unexpected trailing bytes after wire value) means the stored transaction has bytes after its complete wire value. History(InvalidTag) (Display: local history wire validation failed: unexpected wire tag) means its wire tag is invalid. History(DeltaTypeMismatch) (Display: local history wire validation failed: wire delta schema does not match the expected type) means the stored transaction has the wrong delta schema. For each, retain the damaged store for inspection; restart does not salvage the record. Investigate the failed write or transfer, and recover only from a known consistent history with its original identity and allocation metadata. Do not initialize over the damaged store. LegacyEventLogFrame { found: Tag02 } or LegacyEventLogFrame { found: Tag03Unshaped } from an EventLog loader (Display: legacy EventLog frame: found tag 0x02 (no CRC, no shape header), expected tag 0x03 with shape header; migrate once with EventLog::migrate_legacy_wire_bytes_for(bytes, &destination) ...) means an earlier safemesh-crdt wrote the file, before the frame gained its CRC or its shape header. Nothing was applied; tag 0x02 cannot establish that the bytes are intact. Keep the file. A tag 0x02 frame has no checksum: migration accepts a changed payload bit when the frame still decodes, and the migrated value can differ from the original. Check your own checksum of the old file or compare it with a second copy before running the explicit migration with the original replica count. Then reload the result and replace the file: see “Migrating a legacy EventLog” in the safemesh-crdt README. Loaders never migrate on open. Durable stores postdate the shape header, so restart meets this only for a hand-edited transaction. See the recovery evidence. The wire compatibility and upgrades section is the policy for that refusal, the explicit migration, the toolchains current main checks, and the upgrade behaviours that are not yet promised.

Predict, then run: a concurrent add and remove (optional)

Predict whether milk remains after one replica removes its observed token while another adds milk with a fresh token. Then run this complete program from the checkout root with the command below. It uses the same v0-focus OR-Set carrier without touching the exercise stores.

use safemesh_crdt::OrSet;

fn main() {
    let mut left = OrSet::<String, u64>::new();
    let milk = "milk".to_owned();
    left.add(milk.clone(), 100);
    let mut right = left.clone();
    right.apply_remove(right.observed_tokens(&milk));
    left.add(milk.clone(), 201);
    left.merge(&right);
    right.merge(&left);
    assert!(left.contains(&milk) && right.contains(&milk));
    assert_eq!(left.elements(), right.elements());
    println!("milk remains=true; observed token removed, fresh token survives");
}

cargo run --quiet --locked --manifest-path examples/gold-path/rust/Cargo.toml --example concurrent_add_remove

Expected output:

milk remains=true; observed token removed, fresh token survives

Token 100 was observed and removed; concurrent token 201 survives. Tokens here are fixed for this one-shot example. A real mutable writer must allocate unique tokens and preserve identity/allocation on restart as described in the OR-Set lifecycle guide.

Decrement with v0-focus types (optional)

PN-Counter is experimental in v0. Use two v0-focus G-Counters for stock: one totals additions and the other totals removals. Run this complete program with its command below. Each writer owns a distinct coordinate; each bump is its cumulative tally, not an increment amount. Merge both counters, including after offline edits.

use safemesh_crdt::GCounter;

fn main() {
    let mut added_a = GCounter::new(2);
    let mut removed_a = GCounter::new(2);
    added_a.try_apply_bump(0, 10).unwrap();
    let mut added_b = added_a.clone();
    let mut removed_b = removed_a.clone();
    removed_a.try_apply_bump(0, 3).unwrap();
    added_b.try_apply_bump(1, 2).unwrap();
    removed_b.try_apply_bump(1, 1).unwrap();
    added_a.try_merge(&added_b).unwrap();
    removed_a.try_merge(&removed_b).unwrap();
    added_b.try_merge(&added_a).unwrap();
    removed_b.try_merge(&removed_a).unwrap();
    assert_eq!(added_a.state(), added_b.state());
    assert_eq!(removed_a.state(), removed_b.state());
    // Checked conversion/subtraction: counter totals themselves are u128.
    let stock = i128::try_from(added_a.value())
        .unwrap()
        .checked_sub(i128::try_from(removed_a.value()).unwrap())
        .unwrap();
    assert_eq!(stock, 8);
    println!(
        "added={} removed={} stock={stock}",
        added_a.value(),
        removed_a.value()
    );
}

cargo run --quiet --locked --manifest-path examples/gold-path/rust/Cargo.toml --example paired_counters

Expected output:

added=12 removed=4 stock=8

This demonstrates v0-focus carrier semantics, not a proven inventory schema. Persist and exchange both totals; two durable stores do not commit atomically. Keyed stock, membership policy, overflow handling and nonnegative-stock enforcement remain application responsibilities. Offline decrements can oversell: convergence does not enforce a global availability constraint. PN/custom durable restart is outside v0; Rust’s Linux durable paths cover G-Counter and UTF-8 OR-Set only. The separate example leaves the original persist/restart fixture source intact.

TypeScript gold path (Node)

The recommended allocated-writer application is in examples/gold-path/typescript/. Build the Node WASM package locally, install the fixture’s locked TypeScript build dependencies, compile against the generated declarations, and run the two processes:

cargo install wasm-pack --version 0.15.0 --locked
rustup target add wasm32-unknown-unknown
wasm-pack build rust/crates/safemesh-wasm --target nodejs --out-dir "$PWD/examples/gold-path/typescript/pkg" --release
npm --prefix examples/gold-path/typescript ci
npm --prefix examples/gold-path/typescript run build
node examples/gold-path/typescript/main.js persist
node examples/gold-path/typescript/main.js restart

Build tools print progress separately. The two node commands’ stdout is exactly:

saved counter=3 members=[compass]
restored counter=3 members=[compass]
synced counter=6 members=[compass,map,rope] records=3+3
malformed record: SafeMeshError: failed to decode record: unexpected end of wire input
saved latest managed state members=[compass,map,rope] local-sequence=2
cleaned exercise stores

Edit examples/gold-path/typescript/main.ts to adapt the tally and membership. The import loads the generated Node CommonJS package synchronously; there is no init call. Keep the whole pkg/ directory, including .wasm, declarations and snippets/. This path uses that directory directly, not an npm SafeMesh tarball. tsc emits main.js only if compilation succeeds. Values passed to Rust u64 parameters are bigint (3n), and exchanged bytes are Uint8Array.

This Node program opens the managed G-Counter and allocated UTF-8 OR-Set with the existing example FileSystemStore. Each handle has its own store root and an independent anchor outside that root. The second process opens both handles in restart mode without a writer count, then adds map. Rust allocates token 4 after compass’s token 2; the in-memory peer’s rope gets token 3. The handles commit accepted edits and peer records before returning. The program closes both leases, opens both stores again to check the latest committed state, and removes the exercise stores and anchors.

The example Store is in rust/crates/safemesh-wasm/examples/, outside the built pkg/. It is not a packaged or supported Store. The application must supply a Store and keep each independent anchor outside its snapshot root. This fixture does not prove power-loss recovery or cross-object atomicity. The full storage and refusal contract also applies.

The fixture passes an empty Uint8Array to mergeRecordBytes, catches the real SafeMeshError: failed to decode record: unexpected end of wire input, and asserts no log mutation. Reject the malformed input and check your record framing. Every WASM handle is freed in finally. Read the complete TypeScript source and project configuration.

Managed handles

The Node binding exports SafeMeshManagedGCounter for a G-Counter and SafeMeshManagedStringOrSet for an allocated UTF-8 OR-Set. Both handles take a caller-supplied SafeMeshStore when they open. The application supplies the Store. The Rust DurableReplica alarm file described below is specific to that Rust store. It does not change the Node managed handle envelope or restart path. See the binding source and the generated Node declarations. The counter test and set test open the two handles with Stores.

An edit becomes visible through a handle only after the Store confirms the commit. After a failed or unclear commit, that handle refuses more writes until it is closed and restarted from the Store. A restart with a different writer is refused. These behaviors are in the counter publish and open methods and set publish and open methods. The counter test and set test check failed commits and invisible edits. Both tests also check wrong-writer restart refusal.

examples/node-filesystem-store.mjs is an example Store. It is not a packaged, supported Store. The package script packs only the generated WASM output. The binding’s Store interface and the filesystem test show how the example is supplied to a handle. The set test measured recovery after SIGKILL between edits (source). Power loss and a crash between rename and directory fsync were not injected. The example Store source names the latter limit. These are test limits, not durability claims.

Wire compatibility and upgrades

Current main already refuses older EventLog frames, keeps a retained corpus of those frames, and runs named toolchain and platform jobs. This section is that policy. It is not a support window.

The compatibility baseline is the first tagged release, v0.1.0: later releases must read its retained files or ship a checked migration. release_wire_bytes_read_and_refuse reads all retained tagged wire encodings and checks unknown tags, short headers, and trailing bytes; release_store_files_restart_to_documented_state restarts both retained store layouts, and store_refusals_are_named checks store damage. Tagged wire frames and magic-bearing store files refuse headers they do not know. The unversioned transaction and fence wrappers check configuration but cannot identify a compatible-looking future wrapper, and journal restart can treat a future tail as interrupted-append debris; these gaps remain for a later format ruling.

Newer SafeMesh, older log

A current EventLog loader refuses a log written with tag 0x02 or with unshaped tag 0x03 before it decodes any record. The error is WireError::LegacyEventLogFrame, naming the frame found. tests/legacy_event_log.rs asserts that outcome on from_wire_bytes, from_wire_bytes_for, records_from_wire_bytes_for, and the bounded loader, then asserts byte-preserving explicit migration of the undamaged retained fixtures and each file’s pinned SHA-256.

Python merge_log_bytes raises ValueError with failed to decode event log: in front of that same core text, and does not apply the bytes (safemesh-python binding test). WASM mergeLogBytes throws SafeMeshError code 1 with that same core text; a saved identity whose history is a legacy frame is refused the same way (safemesh-wasm binding test). The generated Node package is exercised by node-error-shape.mjs, which package-smoke.sh runs from the full gate (scripts/ci.sh).

Loaders never migrate on open. Migration is an explicit step. The crate README section Migrating a legacy EventLog is the single writer for the frame table, the replica-count rule, and the migrate_event_log commands. Before using those commands on tag 0x02, check your own checksum of the old file or compare it with a second copy: the frame has no checksum, so migration accepts a changed payload bit when it still decodes.

The retained corpus tests/fixtures/legacy-event-log/ holds each earlier frame for every built-in delta that could be persisted then. CI regenerates those files at the commits the fixture README claims, in Regenerate legacy EventLog fixtures at claimed sources, which runs check-legacy-event-log-fixtures.sh. A separate retained bootstrap corpus is regenerated in Regenerate bootstrap fixtures at claimed source. Those files are last-writer commits on main, not release tags.

Older SafeMesh, newer log

The full gate runs scripts/check-older-decoder.sh: current writers generate 25 wire values and six store files, and the decoder built from tagged v0.1.0 reads them to the declared values. This samples current output; it does not promise that an older release will read every future format.

Toolchains and platforms current main checks

The checkout selects Rust 1.96.1 in rust-toolchain.toml. The crate floor is Rust 1.89, from rust/Cargo.toml workspace.package.rust-version. CI Install declared Rust floor, then FLOOR COMPILE and FLOOR TEST at that version. The msrv job Test at declared MSRV runs cargo test --workspace --locked at that same declared version.

The clippy job and the full-gate step Check laws on a non-Linux target cargo check the laws feature for x86_64-apple-darwin. Those jobs run on ubuntu-latest; they are not a macOS runner. Clippy also builds safemesh-crdt for thumbv7em-none-eabihf and safemesh-wasm for wasm32-unknown-unknown. The full gate builds those same targets.

The python job matrix is CPython 3.8–3.14 on ubuntu-latest, including cargo test -p safemesh-python. The full-gate job runs on ubuntu-latest and sets Node 24. The documentation workflow uses Node 24. wasm-pack 0.15.0 is the locked installer in both the floor job and the full gate.

These jobs are what current main checks. They are not a supported-platform matrix.

Not yet promised

Before a first release, the project does not promise:

• Fixtures from previous releases. No GitHub release exists (release-status.mjs reads the GitHub releases list at docs build). The retained EventLog corpus is generated from the last main-branch commit that wrote each frame, as legacy-event-log/README.md states.
• Reproducible release builds. There is no release build.
• A version-numbering promise, including semantic versioning or a support window. safemesh-crdt Cargo.toml currently records 0.2.1 as an unpublished crate version; that number is not a versioning promise.
• That an older decoder can read every future EventLog frame. The full gate samples current output with the tagged v0.1.0 decoder; a later format may require that decoder to refuse cleanly instead.
• Maintainer support. The root status matrix labels it UNKNOWN for all four surfaces (README status).
• Runtime on Windows, macOS hosts, ARM64 hosts, browsers, or Python/OS combinations outside the jobs named above.

WireError is #[non_exhaustive] in codec.rs. The crate README records the matching rule for the first release; this section does not add a stability window for existing variants.

History retention

SafeMesh keeps every committed record forever: restart replays a writer’s whole history, and 5,000 records survive durable persist and restart (retention_durable_persist_restart_all_5000) as well as save, load, restore and merge replay (the retention_*_all_5000 tests in record_admission.rs). There is no compaction in the first release, so a durable store only grows; a store that a restart budget refused still reopens with every record (restart_budget_opens_supported_size_and_refuses_one_more).

Append cost

A fresh store keeps an append-only writer-<id>.journal, and each acknowledged edit appends only its own record after the existing bytes and syncs the file (append_log_writes_one_entry_per_record). When the Rust durable replica raises a collision alarm, it writes the full collision report to writer-<id>.alarms with file and directory sync before returning. Restart loads that report after replay. A store with no alarm has no alarm file. A damaged alarm file makes restart fail; the journal torn-tail rule does not apply to it. An older program ignores the added file and loses the alarm on restart. A plain encoded EventLog still does not store an alarm. An append therefore writes the same bytes at any history size: 58 bytes for a G-Counter record and 68 for a UTF-8 OR-Set add, from 1,000 to 1,000,000 records (benchmark results). On the benchmark machine the median append took 0.18 to 0.46 ms and the slowest of ten took at most 2.93 ms at every size, where rewriting the whole history took 50 to 61 ms per append at 100,000 records and 589 to 645 ms at 1,000,000 (benchmark results). The journal takes 58 bytes per G-Counter record and 68 per UTF-8 OR-Set add; the whole-history format takes about 42 and 52, the 42.0 MB and 52.0 MB it rewrote per append at 1,000,000 records (benchmark results). A pre-protection store restarts to the history before or after an append crash, never a mix (durable_crash_boundaries). An append cut short by a crash never reached its sync, so it was never acknowledged; restart truncates it and reports its offset and size through torn_tail (append_log_torn_tail_is_truncated_and_reported). Restart also truncates a final journal entry that fails its length or checksum test and reports the discarded bytes through torn_tail(). SafeMesh does not claim that an acknowledged final record survives later damage to its bytes. Such damage is discarded as a torn tail in a pre-protection store. A protected store refuses if its own-writer sequence falls below the counter. A damaged record followed by other records is not a torn tail: restart refuses it as History(IntegrityMismatch) and leaves the journal unchanged (append_log_damage_before_the_tail_is_refused).

New Rust durable stores keep writer-<id>.rollback in the store directory and a small counter beside that directory. A restart with a store older than its own-writer counter refuses as RecoveryRequired; open the restored store with an unused writer ID. The counter is written and synced before each own-writer journal record. A crash between those writes makes an honest restart refuse. A missing counter with a marker also refuses. A store with neither file keeps its earlier restart behavior. A copy that also copies the counter still passes. A restore that loses only other writers’ records can also pass because this counter measures only this writer’s sequence. The check covers one machine.

A store written before the append log keeps its whole-history writer-<id>.transaction: restart still opens it, each append still rewrites it whole, and restart never converts it (transaction_store_opens_appends_in_place_and_migrates_explicitly). migrate_counter_to_append_log and migrate_utf8_set_to_append_log move such a store to the append log explicitly and keep every record, including the stores the archived library wrote (transaction_store_opens_appends_in_place_and_migrates_explicitly). If a crash interrupts a migration after its journal is complete, both files remain; restart refuses them as RecoveryRequired, and repeating the migration completes it (interrupted_migration_is_refused_then_completed).

If you delete a store directory

The rollback counter file is in the parent of the store directory. Its name matches .safemesh-<hash>-writer-<id>.counter. Deleting the store directory leaves this file in place. A new store at the same path with the same writer ID returns StoreExists from DurableReplica::counter or utf8_set. The refused create leaves no files in the new store directory. The directory itself may remain.

To reuse the old writer ID after you remove a store on purpose, remove its counter file, and then create the store. Only an operator who knows that the old store is gone on purpose may remove the counter file: it protects that writer against rollback. You can also keep the counter file and create with a new writer ID.

Supported size

The supported history is 100,000 records per writer store, exported as local::SUPPORTED_MAX_RECORDS; a store of exactly 100,000 records restarts under that budget (restart_budget_opens_supported_size_and_refuses_one_more). It stays 100,000 because restart still replays every committed record: 66 ms with a peak RSS of 14 MiB (G-Counter) and 114 ms with 30 MiB (UTF-8 OR-Set) at 100,000 records, against 0.7 s with 119 MiB and 1.3 s with 274 MiB at 1,000,000 (benchmark results). A store not yet migrated also still rewrites its whole history on every append, 50 to 61 ms at 100,000 records and 589 to 645 ms at 1,000,000 (benchmark results). The append log removes the append cost that first set the limit, so raising it for append-log stores is now a project decision about restart time and memory, not a measured append cost (benchmark results). These figures come from one machine and toolchain, which the results file records; run retention_bench to measure your own.

Above the supported size

Pass the budget to refuse a larger store by name: restart_counter_with_limits, restart_counter_from_store_with_limits and restart_utf8_set_with_limits with DecodeLimits { max_records: Some(SUPPORTED_MAX_RECORDS), max_collection_elements: None } return LocalError::RecordLimitExceeded { max_records: 100000, default_limit: false } for a store of 100,001 records (Display: local history exceeds restart record budget: RecordLimitExceeded: 100000) (restart_budget_opens_supported_size_and_refuses_one_more). The check counts records from headers alone, a transaction frame’s declared count or a journal’s base frame count and entry lengths, before any record is decoded or replayed (restart_budget_refuses_before_reading_records, restart_budget_counts_append_log_without_decoding). The refused store’s fence and history file keep their bytes and modification times (restart_budget_opens_supported_size_and_refuses_one_more). Counts are read before integrity checks, so a damaged count can be refused by budget; within budget, the full checked read reports the damage as a History error (restart_budget_refuses_before_reading_records, restart_budget_counts_append_log_without_decoding). A stored history can consume resources for every record it contains. restart_counter, restart_counter_from_store, and restart_utf8_set now default to 100,000 records; a larger store returns local history exceeds restart record budget: RecordLimitExceeded: 100000; pass DecodeLimits::UNLIMITED to reopen before replay and remains unchanged. To reopen an older larger store, call the corresponding _with_limits method with DecodeLimits::UNLIMITED, accepting the unbounded record count and its resource cost. A caller supplied numeric max_records still applies exactly that budget. The cap counts records and is not a memory bound; payload bytes can still be large. SafeMesh deletes no record to get under the limit. The Python and WASM bindings expose their own record budgets and language-specific unlimited values. Their low-level log loaders take max_records (Python binding tests, WASM binding tests).

Not yet promised: compaction, pruning, snapshots

• Compaction. No API folds committed records into a smaller history; a refused store reopens with every record (restart_budget_opens_supported_size_and_refuses_one_more).
• Pruning. No API drops records by age, count or acknowledgement; the retention tests assert every record identity survives (bootstrap.rs, record_admission.rs).
• Snapshots. Restart writes and reads no state snapshot; it replays the full history, and each benchmark restart asserts it replayed every retained record (retention_bench).

Continue to Connect replicas, or review the proof boundary.