Hi everyone! you may all have heard and read about CVE-2024-3094. If not, please do so now [1], [2].
This incident has exposed many long standing issues and should not be seen as a singular event, but rather as the result of several unhealthy patterns. And while OpenWrt was not affected by the resulting vulnerability (because we don't ship OpenSSH by default; and our build of OpenSSH isn't downstream patched for integration with systemd...), this and the fact that the added backdoor was discovered rather fast are both just lucky coincidents. All of that comes in a moment that the NVD [3] has practically stopped working with the public [4], at least temporarily, and that has severely increased the pressure on open source projects to handle such incidents responsibly **by themselves**. (how?) Of course there are many conclusions to draw and discussions to be had when it comes to the libxz bootdoor incidence, many on the purely technical level (why did Debian and RedHat downstream-patch OpenSSH? Why did a dependency-monster like systemd become the de-facto standard on Linux installations? ...), and all of them need to be re-evaluated in the light of this attack imho. In many ways, we are already better off than most Linux distros out there -- not because of deliberate decisions with security in mind, but because of our tendency to minimalism and avoiding bloat due to resource limitations on the target devices, and having a reduced attack surface is just an indirect consequence of that. However, after reading up about the details of this backdoored release tarball, I believe that the current tendency to use tarballs rather than (reproducible!) git checkouts is also problematic to begin with. Stuff like 'make dist' seems like a weird relic nowadays, creates more problems than it could potentially solve, bandwidth is ubiquitous, and we already got our own tarball mirror of git checkouts done by the buildbots (see PKG_MIRROR_HASH). So why not **always** use that instead of potentially shady and hard to verify tarballs? Why do we need to rely one proprietary hacks such as Gibhub codeload just to safe a few megabytes of traffic and a few seconds of build time? There are even too many problems to reproduce even those supposedly automated Github-generated tarballs. Nobody actually checks that. 9bd7d8b, c7c2257, 77368ec, 86994e1, 954142f, 4c5d910, 21f713d, ... Probably all of those have trivial causes and there isn't anything malicious going on there. But it shows the general problem: That while we are quite good with (much more complicated) reproducible builds, nobody seems to care about (actually trivial) reproducible **sources**. It's too trivial and boring to care, I suppose. And then there is the not exactly glorious role of Github in that whole mess, blocking the affected repo (making analysis harder or impossible), blocking the account of the semi-retired old maintainer who is not to blame, ... Hence I believe we should use git checkouts and reference to (ideally signed) git tags when ever we can. Package maintainers should always have a local checkout of the repository of the software they are packaging for OpenWrt, and use 'git fetch origin' or 'git pull' to keep it up-to-date, as to make sure that the repo history remains unchanged. Git has a lot of security built-in, and by using tarballs as a base for our package builds we are basically throwing all that away, for the sake of saving a negligible amount of resources on the build infrastructure. Always using git checkouts instead of tarballs would also makes it much easier for maintainers to at least have a quick look at the changes made in an upstream project between versions (a quick scroll over 'git diff oldtag..newtag' or even just 'git log --stat oldtag..newtag' doesn't take much more time than manually validating a release tarball GPG signature in most cases, if there even is any...). Hiding a malicious change in a commit is infinitely harder than hiding it in a tarball. Those are just my 2 cents, I think this cries for a wider debate and I encourage everyone to participate in it. Of course, the real problem (which is much deeper) is the lack of maintainer resources for critical infrastructure. This message [5] says it all. Cheers Daniel [1]: https://www.openwall.com/lists/oss-security/2024/03/29/4 [2]: https://boehs.org/node/everything-i-know-about-the-xz-backdoor [3]: https://nvd.nist.gov/ [4]: https://www.helpnetsecurity.com/2024/03/19/nvd-vulnerability-management/ [5]: https://www.mail-archive.com/xz-devel@tukaani.org/msg00567.html _______________________________________________ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel