Hello world, When we had our last tech-ctte meeting, 2020.10.21¹, I volunteered to write up a summary of our positions regarding this bug. Then... Well, life happened, and I have not had the time to sit down and write until today -- A couple of hours before our next meeting. Several other discussions have happened as well, most notably, the article by Jonathan Corbet on this issue in LWN².
¹ http://meetbot.debian.net/debian-ctte/2020/debian-ctte.2020-10-21-17.58.html http://meetbot.debian.net/debian-ctte/2020/debian-ctte.2020-10-21-17.58.log.html ² https://lwn.net/Articles/835599/ # Vendoring: Impedance mismatch with our long-established # practices/traditions ###### I believe the issue of vendoring to be central to the discussion of what Debian is, and what its role should be. We are very lucky to have proponents of both sides of this issue in the Committee, and I'll try to keep my ideas as centered as possible (of course, disclaimer: I do hold a position, I really do not like vendoring; we have the luck of having Elana in the ctte, as she is a developer of the affected package, and Sean, who is a Debian Policy editor). Elana started off with a very important and true point: Elana> I think this bug was sort of inevitable. Debian policy cruft is clashing with how people actually build and distribute software these days. we have an appeal to override a maintainer on the basis of "policy" and the "Debian way being superior" without any real technical examination of the merits of "the Debian way" We are quite far from 1996, yes, and many languages have for a long time delivered their own packaging systems, "freeing" the users from the need of installing a myriad of little packages, and "freeing" the distribution caregivers (and I don't only mean the developers, but also the ftp-masters) and infrastructure from carrying this myriad of little packages. Elana and Sean seem to share the thought that each language ecosystem could work with somewhat different rules: Sean> It might be reasonable to vendor like mad for something written in Go but not for something written in Haskell, say Elana> Debian policy is specifically built around the distribution and execution of dynamically linked C/C++ applications and libraries. distros in general were. but modern software above that base does not necessarily operate under the same assumptions and it is silly to apply policy that was designed for applications that are dynamically linked against programs that are statically linked and are impossible to dynamically link Simon mentioned that "our identity should be about shipping high-quality packages, whatever that means", and mentioning that "our packaging policy is designed for medium-sized packages", refereed back to the discussions had long time ago over tiny Javascript snippets packaged as packages, back in the starting days of the nodejs growth. # DFSG-freeness checking ###### Sean and I shared the concern that excessive vendoring (say, having tens to hundreds of libraries shipped as part of a single package) place a very heavy burden on our ftp-masters when checking DFSG-freeness; coupling this with the rapid change rate that "new-wave" software development usually has, if adding / dropping dependencies from already packaged software is usual practice, DFSG-checks would not just have to be made in NEW, but as a continuous process. Far from sustainable, and impossible to do by our ftp-master team practices. # Security issues ####### The issue of security support was also mentioned: If many packages start shipping tens or hundreds of vendored libraries, how can we ensure security support? This has long been a pride point for Debian and, in general, for the Linux distributions model. We package independent libraries, dynamically link them, and security supports "just happens" for the users. Now, what happens in languages as Go or Rust, where linking is done statically? They would have to be at least recompiled when any of their constitutive libraries is updated. But what if the libraries are vendored-in? Security issues are prone to be hidden from our sight. I'm pasting here a bit of the discussion that happened later during the meeting: having this discussion K8S-specific, Elana mentioned that "that is a big part of the tension. sometimes, you have an upstream where the authors are less resourced and responsive than the downstream. this case is almost certainly the opposite". At this point, we found some friction as to _what_ we are discussing on: Is this bug specifically on Kubernetes, which should be taken as a special case? Or would we try to rule as the Technical Committee on vendoring in general in the Debian ecosystem? Elana repeated her assurance that the Kubernetes team is thorough in their freeness-checking and security practices; I insisted on "not discussing about K8S, but about a bundling practice to which K8S subscribes". I do fear that, if K8S is special-cased as an example of well-written software, we will be standing on the verge of a slippery slope; Sean stated we could keep the discussion specific to this package: Sean> can't we qualify our response to this bug by saying we're expressing a view on what's appropriate for this package and do not intend to settle bigger vendoring questions for Debian? To what I answered: Gunnar> Yes, but no matter what we do, we will be setting some precedents. So, we have to keep the bigger picture in mind. Elana mentioned that this issue is typical for Golang packages, but "does not necessarily translate to e.g. JS packages". Simon mentions that several properties of Golang's packaging are shared with Rust, and given the amount of Rust in GNOME, Firefox or Chromium, "if we set policies that rule out shipping Firefox, or chromium, or GNOME, then we no longer have a useful distro". Elana mentions several large Golang projects which share this amount of vendoring with K8S, and said, Elana> disappointingly to me, most of these applications have basically given up on OS packaging, they either provide their own debs via a PPA or generally say "do not install using a distro package manager ever" # Test-passing assurance ###### Phil brings up an interesting point: Many users want the specific library versions that were packaged with a given K8S release because, given the scope of K8S (as a container orchestrator), test coverage can be fundamental; Elana agrees "unless Debian wants to start running its own suite of Kubernetes compliance tests". And, if K8S were to be effectively recompiled every time one of its ~200 dependencies were to be upgraded, said tests would have to happen on each of the users' machines - Clearly not ideal, and a long deviation from standard practice! (and... Well, quite unlikely to be able to perform installs without, say, network activity - that would break other bits of Policy for sure!) There is a course of action that's unlikely to leave everybody happy, but is worth considering: Phil said: Phil> that makes it seem like what we actually need is a decent way of letting our users install the upstream binaries, rather than trying to force those packages into Debian as a special case Sean agreed with him; probably something as K8S due to its nature is not suited for Debian inclusion (but we'd still have to ensure it's easy to build and install, not from packages). But the quesion is where to draw the line: Elana> is the future of Debian "here's a glibc platform base" ala manylinux scope and then everything else is someone else's problem? do we become some sort of nix-ish thing? (...) which is why I think the scope of applications included in Debian needs to be wider than just a common base with flatpaks or similar So... It's not like we reached a conclusion, but I do feel the meeting was interesting and the discussion very much worthy. Yes, this will surely end up in reinforcing the notion that the Technical Committee is a slow and bureaucratic beast :-) But... well, I'll stop apologizing. Only some minutes to go before the meeting, and I want to give the rest of the Committee the opportunity to check if I didn't misrepresent them or skip too much of their opinion.
signature.asc
Description: PGP signature