Hi hackers, This message continues the Umbra v3 P1-P7 review unit in the original thread.
I mistakenly sent the August 8 v3 update as a separate thread. This message continues the earlier Umbra submission; the material and attachments are otherwise unchanged. Since the prior submission, I have refactored Umbra into a PostgreSQL-oriented three-slot physical-placement model and optimized the implementation around this model. With full_page_writes=on, Umbra's performance is now close to md with full_page_writes=off. I have completed a separate audit of P1-P7, so I am submitting that review unit now. Later parts of the implementation already exist, but their audit is still in progress and they are deliberately excluded from this review request. The attached umbra-p1-p7-20260808.tar.gz contains P1-P7 only. Each patch has its own commit message describing its capability, ownership boundary, invariant, dependency, and non-goals. SHA256 (umbra-p1-p7-20260808.tar.gz): cc1ca7345b3d2503626643a825f57247414d6e12a5e84a15330c907c0b52931f The review scope is: 1. P1-P7: smgr selection, the md-compatible physical umfile layer, private MAP metadata bootstrap and residency, compatibility limits, and the first working mapped layout. Later Umbra work includes: 2. Selector-shift WAL, redo, and checkpoint publication and correctness. 3. Performance and operational optimization. The first stage introduces the infrastructure and layout state required by the later selector-shift work: smgr selection, umfile, private MAP metadata, the initial mapped layout, and selector read/cache support. Selector-shift WAL, checkpoint-overlap handling, compaction/restacking, and other later mechanisms are not included in this review unit. Umbra places logical-to-physical block layout below smgr. MAP is Umbra-owned, relation-bound physical-layout metadata. Table and index AMs continue to operate only on logical block numbers and do not interpret MAP pages, physical slots, or selectors. The model uses three physical slots for each logical page. The slots retain the older page versions required for redo and rotate in order. This increases physical page space, but allows Umbra to avoid the WAL amplification caused by traditional full-page writes while full_page_writes remains enabled. Recent TPC-C results are below. The workload uses 800 warehouses and runs for 20 minutes, with checkpoint_timeout=4min and checkpoint_completion_target=0.9. ``` terminals md FPW=on md FPW=off Umbra FPW=on ------------------------------------------------------- 200 385616 614035 604807 400 279927 540942 568584 800 192126 420854 426757 ``` WAL volume per New-Order is: ``` terminals md FPW=on md FPW=off Umbra FPW=on ------------------------------------------------------- 200 26.05 KiB 7.53 KiB 7.54 KiB 400 36.25 KiB 7.83 KiB 7.81 KiB 800 37.71 KiB 8.40 KiB 8.24 KiB ``` The complete benchmark summary and hardware report are attached. md with full_page_writes=off is included only as an upper-bound and sensitivity reference; it is not a correctness-equivalent configuration. The relatively stable branch for testing or inspecting the complete design remains: https://github.com/nayishan/postgre_umbra/tree/chunksplit-restack Please reply in this thread so that the discussion remains in one place. Regards, Mingwei Jia
hardware_disk_report.txt
Description: Binary data
umbra-p1-p7-20260808.tar.gz
Description: Binary data
tpcc_checkpoint_4min_concurrency_200_400_800_en.md
Description: Binary data
