Skip to content

benchmarks: memory, disk and IO per phase behind BENCH_INSTRUMENT=2026-11 - #422

Merged
tae898 merged 2 commits into
mainfrom
inst-memory
Oct 21, 2026
Merged

tae898 merged 2 commits into
mainfrom
inst-memory

Conversation

@tae898

@tae898 tae898 commented Oct 21, 2026

Copy link
Copy Markdown

Closes #421. Part of #352 (items 1, 2 and 3).

Under BENCH_INSTRUMENT=2026-11 the runner reads memory, disk and IO at the end of each phase. With the variable unset nothing runs and a row is byte-identical to before: the lane hook is guarded by an environment variable only the 2026-11 runner sets, and a test checks it.

How it works: the lane prints its PHASE marker, and at one that ends a phase (build, post-build, warm-up, close) it writes a request file into a directory the runner mounted into the container and waits. A thread in the runner (memory_probe.py, about 400 lines, three hook lines in runner.py and one in bench_common.py) reads the cgroup files of the cell, forces a full GC where the engine is a JVM (jcmd for a server, JPype for an embedded engine), reads disk, then restarts memory.peak. Nothing runs inside a timed section. The time spent is on the row.

Headline: mem_owned_mib, cgroup anon plus shmem of the cell's containers at phase end. Every engine has it. Per runtime and context only: jvm_used_after_gc_mib, _committed_mib, direct_mib, and proc_anon_mib and rss_mib for embedded engines. Also mempeak_mib (kernel-reset when the runner may write memory.peak, otherwise the sampler's maximum, named in mem_probe_peak_kind), diskmib and data_mib (less the empty start), diskwal_mib, disk_close*, and io_read_mib and _write_mib. The design, what compares across engines and what does not, and the pinned-heap effect are in PROTOCOL.md and FAIRNESS.md. export_web adds one column, owned memory after queries, for lanes whose rows all carry 2026-11.

Tested: unit tests on synthetic cgroup, smaps, io.stat and jcmd input, the lane and host handshake, and that 2026-10 has no new field or column (test_memory_probe.py); the full experiments suite passes (784 passed, 15 skipped). Laptop smoke on cores 12 and 13: embedded ArcadeDB and served PostgreSQL on the document lane fill every field; jcmd probing was run against a real ArcadeDB server container (heap used after GC 20 MiB, committed 256 MiB). A served JVM cell through the runner was not completed (Neo4j did not become ready at the 4g smoke cap). On this laptop the runner cannot write memory.peak, so peaks are the sampler's. The 2026-11 export_web path covers the new column only; the other 2026-11 table changes are not part of this PR.

This stacks on main after #411 and is independent of the change that stops pinning the embedded ArcadeDB heap.

๐Ÿค– Generated with Claude Code

tae898 and others added 2 commits October 21, 2026 21:34
โ€ฆ6-11 (#421)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@tae898
tae898 merged commit b94ff5c into main Oct 21, 2026
3 checks passed
@tae898
tae898 deleted the inst-memory branch October 21, 2026 14:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

benchmarks: memory, disk and IO per phase for the 2026-11 instrument

1 participant