Hi all, Since this is a considerable change to the .blend file format, here's a mail to make everyone aware of this change: As of yesterday [1], Blender supports loading .blend files that are compressed using Zstandard and will save files in this format if "Compressed" is enabled. Of course, loading gzipped files (the format used before) is still supported.
This doesn't affect you if you only interact with .blend files through the latest Blender version, but it means that compressed files from latest builds (and from Blender 3.0+, once released) will not load in older versions. In order to open them there, you can re-save them without compression in a recent version or decompress them using external tools (e.g. "unzstd" on Linux). Additionally, this means that external tools that open .blend files (e.g. third-party software importers or pipeline tools) will need to be updated to support Zstandard compression as well. The benefit of the new format is that it produces files that have a similar size to gzipped files, but loading and especially saving are much faster (roughly -60% and -90%, respectively, for my test files). For examples, see the release notes [2]. Additionally, Zstandard is quite flexible - compression levels range from extremely fast (LZ4-like) to extremely small (LZMA-like). Currently, Blender uses level 3, but it can load any standard-compliant file - therefore, this level could be changed (or exposed as a setting) in the future. Additionally, loading times are mostly independent of compression level, so you can do things like re-compressing files as part of your pipeline for e.g. network syncing without affecting usability. Internally, the format is a Zstandard-wrapped "raw" .blend file, as before with gzip. However, there is one additional detail: For more efficient partial reading of linked .blend files, Blender tries to skip parts of the file. This was not supported by gzip, and it would usually not be supported by Zstandard. However, since the file contents are divided into independent sections, we can skip entire sections if we know the mapping from uncompressed to compressed position. Therefore, Blender now encodes this mapping into a "skippable frame" - a section of the file that does not produce any data when decompressed, but can be used for storing additional info. The format here follows the upstream seekable Zstandard format [3]. If a file is missing this section (e.g. because it was compressed manually), it can still be loaded without issues, but more memory might be required. - Lukas [1] https://developer.blender.org/rB2ea66af7 [2] https://wiki.blender.org/wiki/Reference/Release_Notes/3.0/Core [3] https://github.com/facebook/zstd/blob/dev/contrib/seekable_format/zstd_seekable_compression_format.md _______________________________________________ Bf-committers mailing list Bf-committers@blender.org List details, subscription details or unsubscribe: https://lists.blender.org/mailman/listinfo/bf-committers