EXPERIMENT 08 / FILESYSTEM AUTOMATION

Every file.
Every consequence.

A project tree under inspection.
Five runtimes. One verified filesystem.

Nift completes the work.
Bulk checksum and mirror throughput still lag.

Inspect the findings ↓

◉ project/READ · VERIFY · RECONCILE

▾ project/

▾ group00/ … group19/

▾ depth0/ … depth2/

▾ bucket000/

f000000.mdGENERATE

f000001.jsonINSPECT

f000004.datCOPY

↳ output/manifest.tsv ✓ VERIFIED

1,000→10,000→100,000 files

SCAN→INSPECT→SELECT→READ→HASH→TRANSFORM→WRITE

Nift 4.12.0 final candidate · Python 3.15.0 · Node 24.21.0 · Bun 1.4.3 · Go 1.27.2

01project/walk/

Follow every path.

A directory walk becomes a runtime workload long before it becomes a bandwidth test.

Traversal scaling — Five runtimes enumerate the same three-level fixture and return a verified path inventory Logarithmic axis; equal vertical steps represent multiplicative changes. SVG ↗

Traversal scaling · booted isolated request, median · ms
Workload	Nift	Python	Node	Bun	Go
1k files	22.877	8.966	12.035	11.726	5.619
10k files	196.061	73.194	47.143	31.965	45.458
100k files	2,029.268	703.614	356.774	210.880	451.046

Exact values above. On small screens, scroll the table independently →

02project/inspect/

Metadata has a cost.

Type-only checks acquire size metadata too. The explicit stat workload adds two queries per file.

Metadata scaling — Exists/type/size inspection, with checked metadata inventories Logarithmic axis; equal vertical steps represent multiplicative changes. SVG ↗

Metadata scaling · booted isolated request, median · ms
Workload	Nift	Python	Node	Bun	Go
1k files	34.056	13.063	19.813	16.265	9.538
10k files	309.684	113.802	104.888	77.531	90.535
100k files	3,210.583	1,177.859	886.655	670.540	845.759

Exact values above. On small screens, scroll the table independently →

Historical 20261011 metadata trace

Logical files10,000

Walk: newfstatat21,373

Stat: newfstatat41,373

Historical20261011 traced requests, not final-candidate counts; linear zero-origin counts. Counts include directory/runtime work. Only four readlinks per trace. Syscall counts do not explain all wall time.

03project/select/

Inspect first. Select second.

Every filter stats every file before selection. Selection percentages describe results, not avoided metadata work.

Filtering / 10k files — Extension .md selects 20%; group00 prefix 5%; size ≥768 bytes 40%. Extension/group/size are correlated Logarithmic axis; equal vertical steps represent multiplicative changes. Bars start at the displayed positive minimum, not zero. SVG ↗

Filtering / 10k files · booted isolated request, median · ms
Workload	Nift	Python	Node	Bun	Go
Name 1%	550.021	132.821	101.850	77.676	67.646
Name 10%	543.735	134.178	99.377	76.986	71.627
Name 50%	607.154	138.870	109.221	80.265	76.184
Extension	378.766	105.971	92.159	70.592	70.459
Prefix	365.017	103.940	92.260	72.501	67.436
Size	324.383	110.011	96.160	73.193	78.562

Exact values above. On small screens, scroll the table independently →

04project/read/

Small files expose scalar work.

The checksum visits every byte in the language. Large-file reads load the full payload but aggregate endpoints only.

Small-file read + checksum — Portable h=(31h+byte)%1000000007 checksum; noncryptographic; output inventories included Logarithmic axis; equal vertical steps represent multiplicative changes. SVG ↗

Small-file read + checksum · booted isolated request, median · ms
Workload	Nift	Python	Node	Bun	Go
1k files	1,019.024	113.848	57.197	56.240	26.617
10k files	10,530.112	1,127.583	408.164	421.197	249.394
100k files	102,441.259	11,525.007	3,627.604	4,062.430	2,472.142

Exact values above. On small screens, scroll the table independently →

100 MiB bulk-read contrast — These different contracts isolate a bulk-read contrast, not a subtraction-based estimate of checksum cost Logarithmic axis; equal vertical steps represent multiplicative changes. Bars start at the displayed positive minimum, not zero. SVG ↗

100 MiB bulk-read contrast · booted isolated request, median · ms
Workload	Nift	Python	Node	Bun	Go
10 × 10 MiB	293.858	54.389	99.725	71.541	46.997
100 × 1 MiB	220.743	23.525	106.170	54.369	34.573

Exact values above. On small screens, scroll the table independently →

05project/fingerprint/

A fingerprint is not a security digest.

Native SHA-256 is unavailable in this Nift snapshot. The portable checksum cannot substitute for a cryptographic release manifest.

Native SHA-256 / 10k files — — means unavailable; it is not zero or a failed observation Logarithmic axis; equal vertical steps represent multiplicative changes. Bars start at the displayed positive minimum, not zero. SVG ↗

Native SHA-256 / 10k files · booted isolated request, median · ms
Workload	Nift	Python	Node	Bun	Go
SHA-256	—	295.954	290.092	191.766	237.422

Exact values above. On small screens, scroll the table independently →

06project/write/

Create. Overwrite. Copy.

Stream writes include source reads; copies verify bytes and paths, not permissions or metadata.

Write and copy — Small stream writes, existing-file overwrite, 100 MiB streams, and a 100k-file copy are distinct contracts Logarithmic axis; equal vertical steps represent multiplicative changes. Bars start at the displayed positive minimum, not zero. SVG ↗

Write and copy · booted isolated request, median · ms
Workload	Nift	Python	Node	Bun	Go
Create 10k	1,961.167	927.347	701.237	747.142	675.893
Overwrite 10k	2,567.117	1,459.327	1,207.568	464.282	1,095.094
10 × 10 MiB	407.801	152.489	218.158	169.130	155.823
100 × 1 MiB	343.660	142.333	223.981	170.532	145.462
Copy 100k	16,647.781	11,739.109	12,887.422	7,814.195	6,175.213

Exact values above. On small screens, scroll the table independently →

07project/mirror/

Unchanged still means inspected.

Both trees are fingerprinted. Identical destination files retain their timestamps.

source/

  keep.md unchanged
~ edit.md new bytes
+ add.dat new path

destination/ → reconciled

  keep.md preserve mtime
~ edit.md replace bytes
+ add.dat copy
- old.dat remove

Mirror / 10k files — Mixed =10% content +1% add/delete; add/delete-only =1% replacement paths. Diagram illustrates semantics, not a measured individual filename Logarithmic axis; equal vertical steps represent multiplicative changes. Bars start at the displayed positive minimum, not zero. SVG ↗

Mirror / 10k files · booted isolated request, median · ms
Workload	Nift	Python	Node	Bun	Go
Clean	21,479.644	2,640.587	923.544	936.652	543.993
1% content	21,048.003	2,609.835	876.791	932.018	572.536
10% content	21,153.051	2,662.650	987.168	940.122	648.510
Mixed	20,744.535	2,756.055	987.482	967.749	639.076
Add/delete	20,726.105	2,446.563	839.236	942.746	560.993

Exact values above. On small screens, scroll the table independently →

08project/changes/

What changed depends on the contract.

Modification-time access is unavailable in this Nift snapshot. Content and timestamp detection intentionally answer different questions.

Change detection / 10k files — Same 1% selected set: changed bytes/new time; same bytes/new time; changed bytes/restored time. Time-only detection misses restored-time edits; content ignores time-only edits Logarithmic axis; equal vertical steps represent multiplicative changes. Bars start at the displayed positive minimum, not zero. SVG ↗

Change detection / 10k files · booted isolated request, median · ms
Workload	Nift	Python	Node	Bun	Go
Time: bytes	—	213.947	200.263	169.598	134.602
Time: time	—	210.445	203.443	167.721	132.169
Time: restored	—	213.778	197.703	170.574	135.238
Hash: bytes	20,165.358	2,151.856	718.198	791.047	416.777
Hash: time	21,031.620	2,198.582	691.207	772.116	421.593
Hash: restored	20,065.571	2,154.094	713.792	847.003	417.963

Exact values above. On small screens, scroll the table independently →

09project/pipeline/

The project, end to end.

Generate documents, copy selected assets and produce a sorted manifest. This is controlled project automation, not a website build.

Project workflows / 10k files — Manifest =path/size/portable fingerprint; generate prefixes inputs; mixed ignores group19, generates .md and copies .dat. Traversal still visits ignored paths Logarithmic axis; equal vertical steps represent multiplicative changes. Bars start at the displayed positive minimum, not zero. SVG ↗

Project workflows / 10k files · booted isolated request, median · ms
Workload	Nift	Python	Node	Bun	Go
Manifest	11,105.603	1,144.951	437.788	428.656	283.115
Generate	13,139.376	2,095.683	1,150.373	1,154.427	791.976
Mixed	5,921.117	940.791	578.602	513.347	348.163

Exact values above. On small screens, scroll the table independently →

10project/memory/

Measure the runtime after exec.

Fifteen final-candidate post-exec observations use the original verified walk protocol on a separate four-vCPU / 8 GiB Ubuntu 24.04 Linode of the same plan. Each runtime/size has one observation; these are checkpoints, not a distribution or significance claim.

Post-exec walk high-water mark — Linux /proc VmHWM after exec, exact full walk response and before exit; one verified request per runtime/size, zero warmups; separate final-candidate diagnostic Linear zero-origin axis. SVG ↗

Post-exec walk high-water mark · separate post-exec diagnostic, one verified request per cell · MiB
Workload	Nift	Python	Node	Bun	Go
1k files	8.875	20.039	53.055	27.648	4.582
10k files	25.727	18.816	67.129	32.750	10.180
100k files	193.848	45.871	128.609	79.875	73.910

Exact values above. On small screens, scroll the table independently →

The separate final noop-child control rose from16.75 to80.75 MiB when its parent allocated64MiB; that confirms why launch-inclusive wait4 accounting is not substituted for these post-exec checkpoints. The fifteen new values use the frozen accepted protocol and pinned peers; no constant has been subtracted.

New final post-exec raw values and identities ↗

11project/diagnostics/

Where Nift spends the work.

Historical20261011 profiles and metadata traces below are retained analytical leads, not new final profiles; final exactly-once gates pass.

P0 · CORRECTNESS

One call must mean one call.

The historical reduction printed CALLED three times versus once for a bound result. Finalc14 inline and bound reductions both execute exactly once. The original diagnostic is retained; it is not a current defect.

P1 · METADATA

Reuse what the tree already knows.

The inspector queries status and file size even for type-only checks. Extra 20,000 stat calls accompany the 10k explicit-stat workload. Preserve races, symlinks and error contracts when consolidating.

P1 · ALLOCATION

5,578,191 allocations.

The separate 10k stat heaptrack diagnostic records 1,111,623 temporary allocations. String construction and condition trimming are concrete leads; instrumentation changes timing.

P1/P2 · SCALAR

Bytes become evaluator work.

Read profiles include AST evaluation, RuntimeValue lifetime, loop dispatch and variable lookup. Bulk reads are far less extreme. This motivates decomposition, not a claim that one function explains the gap.

Template Engines also profiled expression/string/allocation work; Concurrency profiled AST/value handling while demonstrating real multicore scaling. A shared value/execution mechanism is plausible, but their differing paths do not establish a single root cause.

12project/practitioner/

Would I choose it?

Usable for bounded Nift-owned automation; large-tree throughput is weak in this snapshot.

Human + agent: Python for new standalone general automation, Go for bulk work, Nift for bounded existing-project operations. Agent-primary: the same conditional choices; explicit ownership/inputs/lifecycle help inspection but autonomous edit/recovery productivity was not measured. For new filesystem or release tooling today, I would choose Python for compact library-driven automation or Go for a maintained standalone utility. Traversal overhead, repeated metadata queries, allocation volume, slow per-byte interpreter work and copy/mirror costs are operational problems. Missing timestamps, native cryptographic hashing and clear recursive cleanup add real contract gaps. Final exactly-once gates pass; side-effect reliability remains essential.

What would change my preference?

Reusable metadata-bearing directory entries; fewer status/size queries; substantially less path/string/value allocation; faster scalar processing; native SHA-256 and timestamps; safe recursive helpers; continued exactly-once reliability; and competitive large-tree scaling. I would retain explicit roots, artifact ownership, lifecycle and independently checkable outputs.

Qualitative preference · present day versus a reliable, capable, competitive Nift
Workflow	Today	Mature-Nift counterfactual
Nift-owned project	Nift for bounded covered operations	Nift: shared policy and lifecycle
Agent-authored automation	Python/Go for a new filesystem tool	Nift: programmable explicit contracts, without another executor underneath
New release tooling	Python/Go: hashes, timestamps, cleanup	Nift for project-attached release policy; Go for standalone delivery
Large tree scanner	Go: throughput and typed metadata/errors	Nift becomes viable if scaling and memory are genuinely competitive
Portable standalone utility	Go: compiled binary, no installed script runtime	Still often Go: build/distribute per-platform binaries with embedded dependencies

Existing Python or JS module requirements remain specific integration reasons to choose those runtimes. This Linux campaign does not certify Windows paths, symlinks or portability. Explicitness is useful only when the runtime reliably enforces it.

13project/method/

Frozen evidence. Open interpretation.

Official timing remains frozen. A separate fifteen-cell final-candidate diagnostic refreshes the original post-exec memory view without changing its protocol.

Fresh four-vCPU / 8 GiB Ubuntu 24.04 Linode; AMD EPYC 7542; affinity 0–3. Four-CPU affinity. 752 cells; 14,264 measured observations + 752 retained warmups; independent exact/reset/distribution audits passed.

Final candidate: c14d06835bca515a5281e41da0e40219c2d60ef0.

Fresh timing includes launch and exit; booted timing begins after readiness and ends at the complete response. Every observation uses a new runtime. Fixtures are restored, fully read and statted outside timing. Runtime order rotates; no outliers are removed. Full result sets/output SHA-256 checks and independent distribution/reset audits pass. Low-count stress tails are descriptive.

Warm Linux filesystem state, ASCII paths, fixed three-level corpus, no symlink/permission/glob or physical-bandwidth claim. Five file sizes (96–2,048 bytes) and extensions correlate with groups. No fsync or global cache dropping. Output serialization is inside timing; canonical oracle sorting is outside (Nift ls itself sorts).

Frozen identities / timing tails / implementation boundaries

Primary 75e77bb4bb6fb0b8c49920f641c77b394aec5985; supplement 1ab31488eb949eb8f8cd6d50b0eb9fd59e1ad78e; core c14d06835bca515a5281e41da0e40219c2d60ef0. Evidence commit 19e390183b9bcb039abe6aa553ad88a2305b49ee.

All 752 distributions, p95/min/max, commands, source, correctness and implementation boundaries remain in the evidence and full appendix. The launch/lifetime memory accounting limitation is preserved alongside the corrected diagnostic.

Canonical frozen evidence ↗ · Complete distributions ↗ · Core handover ↗