Hi all,
Gentle ping on this RFC from Aug 18.
In short: overlaybd is an openly specified image format (CNCF-hosted;
read-only blob stack + writable top, O(1) lookup regardless of chain
depth) deployed at scale for container/VM image streaming. The RFC
proposes an in-tree C block driver, starting with read-only support
(index merge at open, raw + ZFile layers), writes handled by a
qcow2/raw file on top.
I'd especially appreciate thoughts on the four questions in the RFC:
1. Is a new in-tree format driver acceptable in principle?
2. Is read-only-first a reasonable initial scope (with iotests and a
docs/interop spec page)?
3. Remote blobs: compose over block/curl, or fetch in-driver?
4. CLI surface: point -drive at the OCI-style JSON config, or
per-layer options?
Question 1 is the main gate -- if in-principle feedback is easier to
give than a full design review, that alone would help me decide
whether to proceed with the first series.
Thanks,
Huiba
Huiba Li <[email protected]> 于2026年8月18日周二 17:38写道：
>
> Hi,
>
> I would like to propose adding a QEMU block driver for the
> overlaybd image format, and I am looking for early feedback on
> scope and design before sending patches.
>
> What overlaybd is
> -----------------
> Overlaybd is an open-source image format
> (https://containerd.github.io/overlaybd/)
> hosted as a sub-project of containerd (CNCF graduated). A disk
> image is a stack of read-only snapshot blobs plus a writable top
> file, in a layout compatible with OCI image registries: a base
> image is stored once and shared by every derivative image, and
> data is fetched on demand rather than pulled in full before boot.
>
> Two properties distinguish it from qcow2/VHDX/VMDK-style chains:
>
> * Lookup cost is O(1) in chain depth. At open time the per-file
> indices are merged into a single in-memory index (an
> extent-based log-structured merge tree), so a read never walks
> backing files, and per-file index caches do not multiply with
> snapshots.
>
> * The merged index is small (about 300 KB average for 50+ GB
> images in our production data), and because extents are
> variable-length, writes land at 512-byte granularity with no
> cluster copy-on-write.
>
> The format is openly specified:
> https://github.com/containerd/overlaybd/tree/main/docs/specs/lsmt.md
> https://github.com/containerd/overlaybd/tree/main/docs/specs/zfile.md
>
> Reference implementation:
> https://github.com/containerd/overlaybd
>
> Overlaybd has been in production at Alibaba for years (Taobao,
> Tmall, Alibaba Cloud, Function Compute) and is used by Azure AKS
> Artifact Streaming, Databricks serverless compute, and several
> microVM-based sandbox platforms. The design is documented in two
> USENIX ATC papers (DADI, ATC'20; FaaSNet, ATC'21). I am one of the
> overlaybd/DADI authors.
>
> Why a QEMU driver
> -----------------
> Today overlaybd images are served to VMs through a kernel blk dev
> and a userspace daemon. That fits container workloads well, but
> a native QEMU driver would let any QEMU-based stack (libvirt,
> KubeVirt, OpenStack, plain qemu-system-*) boot overlaybd images
> directly, with QEMU's usual amenities (backing files, block jobs,
> throttling, migration) and no extra daemon on the host. This is
> beneficial to agent sandboxes.
>
> Proposed implementation
> -----------------------
> A new block format driver, block/overlaybd.c, written in C from the
> open spec (like the existing vmdk/vhdx drivers) rather than binding
> the C++ reference implementation:
>
> * v1: read-only support. Local layer files, index merge at open,
> raw and ZFile blobs (lz4/zstd are already optional QEMU
> dependencies). Writes can be served by a qcow2/raw file on top,
> exactly like booting a read-only backing image today.
> * v2: native writable layer, and lazy fetching of remote blobs —
> either composed over the existing curl driver or with a small
> built-in HTTP fetcher; guidance welcome.
>
> Questions
> ---------
> 1. Is a new in-tree format driver acceptable in principle, given
> the format is stable, openly specified, and deployed at scale?
> 2. Is read-only-first a reasonable merge scope for the initial
> series (with iotests and a docs/interop/ spec page)?
> 3. For remote blobs: compose over block/curl, or fetch in-driver
> with a local cache?
> 4. CLI surface: point -drive at the OCI-style JSON image config
> (which lists the layer blobs), or expose per-layer options?
>
> I will write the iotests and the docs/interop format page, and add
> myself to MAINTAINERS in the first series.
>
> Thanks for any comments,
> Huiba Li
