Hi all, Blubber v1.5.0 <https://doc.wikimedia.org/releng/blubber/CHANGELOG.html#v1-5-0> has been released! Prepend your blubber.yaml with this line to use it.
# syntax=docker-registry.wikimedia.org/repos/releng/blubber/buildkit:v1.5.0 ...and so has v1.2.0 <https://doc.wikimedia.org/releng/blubber/CHANGELOG.html#v1-2-0>, and v1.3.0 <https://doc.wikimedia.org/releng/blubber/CHANGELOG.html#v1-3-0>, and v1.4.0 <https://doc.wikimedia.org/releng/blubber/CHANGELOG.html#v1-4-0>... A while back Blubber was refactored to use the BuildKit LLB <https://docs.docker.com/build/buildkit/#llb> API directly instead of relying on intermediate dockerfile transpilation. In addition to the runtime benefits, this made it much easier to implement new features. Over the past few months, this refactor has borne fruit and there are a number of new features to share with you all. Exclude files from copies and requirements (since v1.2.0 <https://doc.wikimedia.org/releng/blubber/CHANGELOG.html#v1-2-0>) A new exclude field allows you to... exclude files that match any of the provided glob patterns from the copy operation. The exclude field can be used with both copies and builder.requirements. See the reference <https://doc.wikimedia.org/releng/blubber/configuration.html#exclude-27> docs and examples <https://doc.wikimedia.org/releng/blubber/examples/01-basic-usage.html#files-are-excluded-using-explicit-glob-patterns> for details. Cache mounts for custom builders (since v1.2.0 <https://doc.wikimedia.org/releng/blubber/CHANGELOG.html#v1-2-0>) Builders can now specify a number of directories to be cached between builds. Caches can greatly speed up builds that depend on intermediate on-disk state (e.g. compilers, package managers, etc.), and they also help to reduce the size of resulting images since cached directories are not included in exported images. See the reference <https://doc.wikimedia.org/releng/blubber/configuration.html#caches> docs and examples <https://doc.wikimedia.org/releng/blubber/examples/04-builders.html#using-caches-to-speed-up-builds> for details. Source mounts for custom builders (since v1.4.0 <https://doc.wikimedia.org/releng/blubber/CHANGELOG.html#v1-4-0>) Builders can now mount the filesystems of other variants, source images, or even the local build context during their execution. Source mounts are useful when you need files from another source as inputs for a build step but you don't want those files in the resulting variant image. See the reference <https://doc.wikimedia.org/releng/blubber/configuration.html#mounts-3> docs and examples <https://doc.wikimedia.org/releng/blubber/examples/04-builders.html#using-a-mount-to-read-files-from-another-variant> for details. Signed additional APT sources (since v1.2.0 <https://doc.wikimedia.org/releng/blubber/CHANGELOG.html#v1-2-0>) You can now provide an inline public key for any additional APT source using the new signed-by field, ensuring packages installed from that source are verified prior to installation. This primarily serves build step use cases where folks would otherwise have to download and install the package (or install from source) using a custom builder. See the reference <https://doc.wikimedia.org/releng/blubber/configuration.html#signed-by> docs and examples <https://doc.wikimedia.org/releng/blubber/examples/03-installing-packages.html#provide-a-public-key-for-an-additional-apt-source> for details. Inline scripts for custom builders and a simplified command syntax (since v1.3.0 <https://doc.wikimedia.org/releng/blubber/CHANGELOG.html#v1-3-0>) Inline script can be provided for custom builders for more verbose build/complex steps. Scripts are not included in the image filesystem but are mounted temporarily during the build step. The custom builder command can now be a simple string instead of the goofy JSON list syntax. See the reference <https://doc.wikimedia.org/releng/blubber/configuration.html#script-1> docs and examples <https://doc.wikimedia.org/releng/blubber/examples/04-builders.html#defining-inline-builder-scripts> for details. Build arguments (since v1.5.0 <https://doc.wikimedia.org/releng/blubber/CHANGELOG.html#v1-2-0>) Blubber now supports build arguments that can be set/overridden at build time via client tooling (e.g. docker buildx build --build-arg foo=bar). Build arguments can be used to reduce redundancy and make configurations more flexible. Note that only some fields support build argument expansion. See the reference <https://doc.wikimedia.org/releng/blubber/configuration.html#arguments> docs and examples <https://doc.wikimedia.org/releng/blubber/examples/08-build-arguments.html#using-an-argument-to-generalize-a-builder-variant> for details. README rewrite (since v1.5.0 <https://doc.wikimedia.org/releng/blubber/CHANGELOG.html#v1-2-0>) Blubber has changed so much over the years. It started as a simple Dockerfile transpiler that helped us build container images for MediaWiki services (that were all implemented in Node.JS at the time). It has since evolved into a general purpose BuildKit frontend (a.k.a. Dockerfile syntax) that is compatible with all Docker/BuildKit based tooling. The documentation <https://doc.wikimedia.org/releng/blubber/> has been rewritten to reflect its evolved state, recommended usage, and compatibility with tooling like Docker Buildx, Docker Bake, Docker Compose. Have a read through it! See the CHANGELOG <https://doc.wikimedia.org/releng/blubber/CHANGELOG.html#v1-5-0> for other changes and fixes in recent releases. As always, feel free to ask for help using Blubber in #wikimedia-releng on IRC. Cheers, Dan -- Dan Duvall Staff Software Engineer, Release Engineering Wikimedia Foundation
_______________________________________________ Wikitech-l mailing list -- [email protected] To unsubscribe send an email to [email protected] https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
