Helmut Grohne <hel...@subdivi.de> writes: > To me, this leaves more question marks than earlier. What applications > of aliasing do you envision that would benefit here? Anything concrete?
So, I do want to stay focused on Sam's point, which is that we should avoid this specific argument by noting that we're not going to get dpkg aliasing support within a workable length of time anyway, given where we're at today. I'm therefore not willing to put in the effort required to deeply flesh out use cases, and I'm sure my initial thoughts are both incomplete and inaccurate. This is more of a high-level design intuition that stems from some basic architectural principles, such as "dpkg should be the authority for what Debian installs on the file system so that it can ensure global consistency." But to give you a concrete answer, here are the sorts of things that come to mind even after this transition is complete in Debian: * Old packages that still use aliased paths installed on current systems. * Locally-built packages with /bin paths in data.tar.gz. * Weird local symlinks to move files around in the file system. * Some unforseen future transition where we want to merge two directories. More generally, the way I am thinking about it from a high level is that this transition has uncovered problems with dpkg's understanding of symlinks. (Well, uncovered for some of us who weren't paying close atention; I'm sure Guillem has been aware of them for eons.) Its model of the world is too simple; there are POSIX file system semantics that it does not understand and therefore mishandles. Ideally we would fix this. > Why did you see waiting for a dpkg patch as a reasonable approach > earlier? Because I thought it would be easier than it turns out to be. > I think we have roughly three categories of dpkg changes on the table: > * Minimal hacks that would paper over some of the effects without > causing a generally applicable aliasing support > * A mechanism where dpkg is being told about aliasing symlinks > explicitly and thus would work with them > * An extension of the dpkg filesystem database wherein it would be able > to determine the filetype for every filesystem object. In particular, > this would allow it to tell directories from symlinks apart and > handle the overlap in a meaningful way. > Which of these do you consider "aliasing support"? The second and the third. > The third category is quite involved. Since it changes the internal > layout of dpkg's database, a prerequisite for doing this is removing > direct accesses to the database by other tools. Guillem has been doing > that work and you can find more information about it at > https://wiki.debian.org/Teams/Dpkg/Spec/MetadataTracking. Note that this > work goes beyond what is needed for aliasing as it also considers > extending the metadata format in binary packages to be able to represent > e.g. extended attributes. Indeed, this is the point. This is exactly what Sam and I are both saying: a good architectural solution is going to take long enough that we can't block this transition on it, and therefore debating whether or not that solution is desirable is beside the point and is just creating more things to argue about without resolving the problem in front of us. > Did you expect Guillem to implement this faster? I think we have an entire project full of people with expert skills, including some truly impressive C programmers, and I find our inability to muster those resources to improve *the* foundational program on which the work of the entire distribution rests on, in a way that meets a very high quality bar and is sustainable for the project, to be disappointing. I'm not assigning blame; I'm simply saying that it makes me sad that the assumption is that only Guillem alone is able to work on this project and therefore its success is constrained by his available time. You will note that I am not ramping up on dpkg development despite being an expert C programmer. I really am not assigning blame. I'm sad about a lot of things that I could be doing and currently am not. > The second category seems fairly narrow and tailored to the /usr-merge > application. Do you consider it to support aliasing in the generic way > that you had in mind? I didn't have a specific generic way in mind. I think it would be a cleaner architecture than what we're going to do instead, but not as clean as what Guillem is working on. I have some concerns that Guillem is trying to boil the ocean and he's missing some viable incremental approach, but this is only a vague concern and I absolutely do not have the expertise in dpkg to argue that case. Clearly Guillem has convinced himself that it is the correct approach and he is an expert in this area, so he is probably correct and I am probably wrong. > Since Guillem is working on the third category, whom did you imagine to > be working on the second one? The answer to that question that I have been pushing since the beginning of this debacle is "the people who wanted to do the /usr-merge transition." But clearly that was never going to happen, so this is just another thing for me to be sad about. > Please bear in mind that a general form of aliasing support entails all > sorts of crazy corner cases. Yes. The problem that dpkg is trying to solve is inherently full of crazy corner cases. I don't think aliasing support is at all unusual in that sense; just about every problem in this space has lots of crazy corner cases. This is unavoidable. In my ideal world, we would meet this head-on by making dpkg robust against crazy corner cases via an excellent architecture and a thoroughly comprehensive test suite. It is quite likely that in the process we will discover that some of the existing dpkg concepts are insufficiently robust and need to be improved so that they're less fragile. In other words, the fact that packaging problems give rise to crazy corner caases is, to me, a reason to put *more* effort into dpkg, not less, because solving those crazy corner cases is what makes our package management something that we can be deeply proud of. This is something on which I wholeheartedly agree with Guillem (and, I think, you): the problem is complex and requires careful design and thorough testing. -- Russ Allbery (r...@debian.org) <https://www.eyrie.org/~eagle/>