On Sat, Aug 13, 2016 at 9:53 AM Olaf Flebbe <[email protected]> wrote: > Hi Christopher, > > thanks for reaching out to us! > > To be honest, we are getting into packaging problems recently: > > We are downloading dependencies from public repositories. Some artifacts > are not suitable for the POWER8 and AARCH64 platforms we like to support. > since they do not contain the proper shared libraries for these > architectures. Right now I do not have a clue how to provide a suitable > solution to this, since we like to > built packges as unmodified as possible. > > This is one area where downstream packagers can provide feedback to upstream communities to extend native code support to more platforms.
> Fedora (Centos/Debian/Ubuntu) approach not to download anything, compiling > all the artifacts for themselfs and reuse these artifacts when compiling > upper software layers should solve this problem. > I think we may agree on this point. I very much like to pickup jar's built > by fedora if these fix our problems. > > But, as far as I read the documents Fedora still sticks to the "one > version" mantra: There should be only one version of a library (jar) on the > system. (The same on Debian, btw) > > Upstream devs often use ancient library versions and we cannot easily > upgrade to newer versions, since they are not api compatible. > However, other Fedora projects require new version and these are already > packaged. Do we have to upgrade all the sources ??? > One of the biggest advantages of a distro is dependency convergence. This is important for stability, security, and interoperability. Individual upstream projects can do their own thing with dependencies, and that's fine when used in isolation, but that's where the power of having downstream packaging comes in. Yes, it means sometimes the upstream project has to be slightly patched to converge. Fedora does permit multiple versions to be packaged as "compat" packages, but they have an altered naming convention. This is suitable for some things (log4j1 and log4j2, for example), but not necessarily others (commons-math vs. commons-math3... easier to patch to use math3 rather than package math2). So, yes, sometimes it means upgrading sources... and sometimes it means encouraging upstream communities to move to newer versions. > For instance nobody likes to upgrade protobuf 2.5.1 in Hadoop to a current > version since it may break the networking protocol and the compatiblity to > other hadoop products. > > Frankly, nobody is in the position the clean all the mess up. > > I'm optimistic that such things can be addressed reasonably. Encouraging upstream to stick to relatively modern versions can help, and downstream packagers/vendors can give upstream the confidence that those changes won't create a bad user experience. My personal philosophy is that upstream should use relatively modern versions, and downstream can patch to provide support for older environments, as necessary. This is better than upstream being stuck on old software, and packagers having to patch to make things work in modern environments. Everything is on a case-by-case scenario, though, and some dependencies can't be easily updated without breaking things, but those older dependencies can be packaged, too. Multiple versions is an option. > Without fedora/centos/RHEL allowing different versions of a library to > coexist on the system, even only to sidestep dependency problems, I see no > chance of having substancial part of our effort into fedora. > > They do allow multiple versions: https://fedoraproject.org/wiki/Packaging:Naming#Multiple_packages_with_the_same_base_name, but these shouldn't be used to circumvent the goal of dependency convergence. For example, Fedora still ships jline1 and jline2, as well as log4j 1.12, and log4j 2. I think one of the biggest advantages that the dependency convergence of downstream vendors/packagers can provide is the ability for upstream to be confident enough to modernize their dependencies. Without strong downstream packaging support, upstream is too afraid to make any changes in their dependencies. Knowing that downstream is doing proper dependency convergence and integration means that they don't have to worry about breaking users so much, when they modernize their upstream applications. There will always be special cases like protobuf (any serializer, actually), but even that serves as a good example: If Hadoop were confident that there were downstream packaging available which converge on a common version of protobuf, to provide interoperability, would they be so afraid to update to protobuf 2.6? I think probably not, because they're not thinking that people are going to get their tools in a dependency-convergent environment. They're worried that their decisions are going to diverge from other upstream decisions.
