Hi Christopher, thanks for reaching out to us!
To be honest, we are getting into packaging problems recently: We are downloading dependencies from public repositories. Some artifacts are not suitable for the POWER8 and AARCH64 platforms we like to support. since they do not contain the proper shared libraries for these architectures. Right now I do not have a clue how to provide a suitable solution to this, since we like to built packges as unmodified as possible. Fedora (Centos/Debian/Ubuntu) approach not to download anything, compiling all the artifacts for themselfs and reuse these artifacts when compiling upper software layers should solve this problem. I think we may agree on this point. I very much like to pickup jar's built by fedora if these fix our problems. But, as far as I read the documents Fedora still sticks to the "one version" mantra: There should be only one version of a library (jar) on the system. (The same on Debian, btw) Upstream devs often use ancient library versions and we cannot easily upgrade to newer versions, since they are not api compatible. However, other Fedora projects require new version and these are already packaged. Do we have to upgrade all the sources ??? For instance nobody likes to upgrade protobuf 2.5.1 in Hadoop to a current version since it may break the networking protocol and the compatiblity to other hadoop products. Frankly, nobody is in the position the clean all the mess up. Without fedora/centos/RHEL allowing different versions of a library to coexist on the system, even only to sidestep dependency problems, I see no chance of having substancial part of our effort into fedora. I was working on silencing build sanitizer tools and to produce src rpm and src deb packages which can be recompiled by itself. But even that was a tough job: See for instance BIGTOP-2151 : I failed to add an empty dir into the source rpm. If you still like to contribute, please file JIRA's! Olaf > Am 13.08.2016 um 05:13 schrieb Christopher <[email protected]>: > > On Fri, Aug 12, 2016 at 9:41 PM Roman Shaposhnik <[email protected]> > wrote: > >> Hi Christopher! >> >> On Fri, Aug 12, 2016 at 4:47 PM, Christopher <[email protected]> wrote: >>> Hi Bigtop Developers, >>> >>> I see that Bigtop packaging for RHEL/CentOS 6 and 7 and Fedora 20. >>> >>> I was just wondering if there were any motivation within this community >> to >>> put some of that packaging effort into those upstream communities, either >>> to replace the packaging that BigTop provides, or to compliment it. >>> >>> I'm currently a package maintainer for Fedora and EPEL (which is >> maintained >>> by the Fedora community), for Hadoop, ZooKeeper, and Accumulo, and I >> think >>> these Big Data packages could really use some additional support from >> more >>> packagers. I also think that perhaps we can deduplicate some effort. >>> >>> Is anybody interested in helping with the packaging in the Fedora/EPEL >>> communities to support Fedora and EL distros? >> >> I'm definitely very much interested in this. Especially since you have the >> kind >> of deep expertise of being an official maintainer. In fact, that's >> actually the >> biggest issue that we faced last time we were thinking about upstreaming >> Bigtop packaging into Linux distros -- packaging guidelines. >> >> > > My expertise is limited. I don't want to overstate it. Fedora is a > community, just like Apache, and is open source. There's nothing official > about it. :) I took on some of these as a novice, and for the most part, I > still am. > > (see my response to Konstantin). > > >> For better or for worth, Bigtop ended up defining the layout and policies >> for >> all major commercial Hadoop offerings and changing that may not be much >> of an option. If we keep Bigtop packaging the way it is I'm not sure how >> much >> good will we will get on the Linux distro side. >> >> Examples off the top of my head here include: >> 1. the way Bigtop packaging deals with jars >> 2. the way Bigtop packaging deals with /etc >> >> > > Fedora and Red Hat have gotten a *lot* better about packaging Java lately. > I know that's one area that Linux distros have typically lagged behind in, > but the state of things now are much much better... consistent locations > for config files, env scripts, launch scripts, jar locations, rpm macros > for creating launch scripts, tools to construct classpath, maven/ant-ivy > stuffs to resolve dependencies from distro-packaged RPMs, rpm metadata to > resolve RPMs by maven coordinates, etc. > > So perhaps, a good first step for you would be to pick a simple package >> (lets say Zookeeper) and tell us how much of Fedora packaging guidelines >> we're still violating and how much of that is non-negotiable vs. could be >> fixed. >> >> Thanks, >> Roman. >> > > I'd have to look at how Bigtop is packaging things today. Understanding > these differences is part of what I'm requesting here from this community. > That and I figure some of Bigtop's community already acts as a liaisons to > the upstream projects, for packaging-related issues/improvements/fixes. > That's the kind of expertise I think Fedora/EPEL needs. In exchange, I can > also do my best to try to help improve Bigtop packaging based on my > experience in Fedora/EPEL, especially as I understand the differences in > the packaging philosophies at Bigtop, and the separate goals of the project. > > For now, I can offer a link to some of the packaging guidelines for > Java/Maven in Fedora (https://fedoraproject.org/wiki/Packaging:Java) and > some git repos for the SPEC/distro-specific patches: > http://pkgs.fedoraproject.org/cgit/rpms/accumulo.git/ > http://pkgs.fedoraproject.org/cgit/rpms/hadoop.git/ > http://pkgs.fedoraproject.org/cgit/rpms/zookeeper.git/ > > If anybody wants to help out with any of these, I can offer my assistance > getting started as a co-maintainer.
signature.asc
Description: Message signed with OpenPGP using GPGMail
