Hi, Kengo, I just noticed you updated the BOM of Bigtop v1.5 on BIGTOP-3123.
One thing I'd like to know your and other folks' thought on Elasticsearch as Bigtop component. There is a ticket (https://issues.apache.org/jira/browse/BIGTOP-3219) for this. And I've finished most of the sub-works (build/packaging/deployement). Patch could be found at: https://github.com/apache/bigtop/pull/566 For the smoke test part I think it could be done in this weekend. So if that Jira (BIGTOP-3219) can be done in next week, do you think it is OK to include Elasticsearch-5.6.14 in v1.5? Regards, Jun Kengo Seki <sek...@apache.org> 于2019年11月22日周五 上午12:09写道： > Thanks for the comments, everyone! > If there's still no objection in a few days, I'm going to update > BIGTOP-3123's description. > > > 'contrib' module will be easier for us to maintain traditional distros > and cnb. > > Agreed. I think that merging the cnb branch later will be a hard work too. > > > What do you think about the components? > > Is there a list of components you'd like to upgrade? > > I'd like to upgrade the following components: > > - Zookeeper: 3.4.13 > - Hadoop: 3.2.1 (or 2.10.0 if packaging Hadoop 3 is too hard, as Olaf > mentioned before) > - HBase: 2.2.2 > - Hive: 3.1.2 > - Tez: 0.9.2 > - Spark: 2.4.4 (or 3.0.0, if GA is released before long) > - Phoenix: 5.0.0 > - Kafka: 2.3.1 > - Ignite: 2.7.6 > - Zeppelin: 0.8.2 > > My Teammates and I are trying to package them, and all of them > are successfully built anyway. But we have not tested them yet, > and I'm sure many problems will be found from now, just as Olaf > already came across on Hadoop 3... :) > > > the community was lean to the direction of having important component > better supported > > instead of spending resources for 20~30 components. > > Totally agreed. If we succeed to package Hadoop 3, > I'd like to drop inactive components which can't be built with it, > at least for now. > > > Just one concern about the puppet recipes compatibility across multiple > puppet versions > > Exactly. I think it's difficult to support Puppet 3, 4 and 5 with a > single manifest or config file, > so I'm thinking to create a new "puppet5" directory beside the > existing "puppet" directory > and put the manifests and config files for Puppet 5 into it (and when > we drop the distros > using Puppet 3 and 4 completely in the future, we can drop the > existing "puppet" directory > and promote "puppet5" to "puppet"). > If it doesn't work as expected, I'll ask you the possibility to drop > old versions in the next release again. > > > one source of problems was using puppet-forge for instance for > puppet-stdlib and puppet-apt, > > Yeah, puppet modules seem to be installed into /usr/share/puppet/modules > via apt on Debian 9 and Ubuntu 16.04, while /etc/puppet/modules via > `puppet module install` on CentOS 7, as you said. > Maybe we have to consolidate them somehow, or specify both of them > for `--modulepath` (I'm not sure if it works though), or choose either of > them > in accordance with the distro. > > Kengo Seki <sek...@apache.org> > > On Thu, Nov 21, 2019 at 3:17 PM Olaf Flebbe <o...@oflebbe.de> wrote: > > > > hi > > > > one source of problems was using puppet-forge for instance for > puppet-stdlib and puppet-apt, since they require rather new versions. look > out for 'puppet module install ....'. While all distros using apt do have > matching prepackaged versions in their repository. > > > > other was different search paths of all these versions. we never fixed > that consistently. > > > > olaf > > > > Von meinem iPad gesendet > > > > > Am 21.11.2019 um 03:43 schrieb Jun HE <ju...@apache.org>: > > > > > > I'm fine with the new distros list. Just one concern about the puppet > > > recipes compatibility across multiple puppet versions (3.8.5 for > > > Ubuntu-16.04, 4.8.2 for Debian-9, and 5.x for other new distros). I > didn't > > > do any investigation yet. If such issues arise, I'll vote for drop > distros > > > with older puppet. > > > > > > Evans Ye <evan...@apache.org> 于2019年11月21日周四 上午1:51写道： > > > > > >> Fine by me for the OS side. > > >> What do you think about the components? Is there a list of components > you'd > > >> like to upgrade? > > >> We can target a subset of current supported matrix as we previously > > >> discussed about this and the community was lean to the direction of > having > > >> important component better supported instead of spending resources for > > >> 20~30 components. > > >> > > >> Youngwoo Kim (김영우) <yw...@apache.org> 於 2019年11月20日 週三 上午9:41寫道： > > >> > > >>> Kengo, > > >>> > > >>> Looks good to me. I think puppet on CentOS 8 would be fine. > > >>> > > >>> On Cloud Native Bigtop, I believe we should consider that components > as a > > >>> 'contrib' at this point. > > >>> I'm considering about Jay's idea, making 'CNB' on master as a contrib > > >>> module. A development branch is good but on our "two-tracks" > development, > > >>> 'contrib' module will be easier for us to maintain traditional > distros > > >> and > > >>> cnb. > > >>> > > >>> Thanks, > > >>> Youngwoo > > >>> > > >>>> On Wed, Nov 20, 2019 at 9:28 AM Kengo Seki <sek...@apache.org> > wrote: > > >>> > > >>>> Hi folks, > > >>>> > > >>>> I'd like to discuss the target distros for the next 1.5.0 release > [1], > > >>>> because over 1.5 years have passed since Ubuntu 18.04 was released > > >>>> and the next LTS will be released within half a year. In addition, > > >>>> Fedora 26 and openSUSE 42.3 have already been EOL'd. > > >>>> > > >>>> (I understand the "Cloud Native Bigtop" project is going on > > >>>> and am really looking forward to it, but my customers still requires > > >>>> the traditional software stack :) > > >>>> > > >>>> Based on the past discussion [2], here's my proposal: > > >>>> > > >>>> - Add Debian 10, Fedora 31 and Ubuntu 18.04 as the target distros > > >>>> and use the puppet package provided by each distro, so that > > >>>> we can support all CPU architectures (x86_64, aarch64, and > ppc64le). > > >>>> Their puppet versions are 5.4.0 (ubuntu) and 5.5.10 (debian and > > >>> fedora). > > >>>> > > >>>> Keep Debian 9 and Ubuntu 16.04 since they are still in the support > > >>>> period. > > >>>> > > >>>> Drop Fedora 26 since it has reached to the EOL on 2018-05-29. > > >>>> > > >>>> - Add CentOS 8. Unfortunately, that version doesn't seem to > > >>>> provide the distro's puppet package, even including EPEL. > > >>>> Even though, I'd like to support it since that distro > > >>>> (and RHEL8) are widely used especially in enterprise systems. > > >>>> So, as the next best option, how about using Puppet 5.5 provided by > > >>>> Puppetlabs and only supporting the x86_64 architecture on this > > >> version? > > >>>> > > >>>> Keep CentOS 7 since it's still in the support period. > > >>>> > > >>>> - Drop openSUSE 42.3 since it has reached to the EOL on 2019-07-01 > > >>>> and don't add a new version of that distro, as discussed in [2]. > > >>>> > > >>>> To summarize the above, the supported distros and their versions > > >>>> in the 1.5.0 release are as follows: > > >>>> > > >>>> - CentOS 7, 8 (8 is only supported on x86_64) > > >>>> - Debian 9, 10 > > >>>> - Fedora 31 > > >>>> - Ubuntu 16.04, 18.04 > > >>>> > > >>>> Does this sound reasonable? I'd appreciate any comments or > suggestions. > > >>>> > > >>>> (Honestly, I'd actually like to drop CentOS 7, Debian 9, and Ubuntu > > >>> 16.04, > > >>>> so that we can consolidate the Puppet version to 5.x. > > >>>> But it may be too aggressive for users.) > > >>>> > > >>>> [1]: https://issues.apache.org/jira/browse/BIGTOP-3123 > > >>>> [2]: > > >>>> > > >>> > > >> > https://lists.apache.org/thread.html/26e14cf36e9cfd61e0de581ed83bf305565c2e65234f1ce3bfb97628@%3Cdev.bigtop.apache.org%3E > > >>>> > > >>>> Kengo Seki <sek...@apache.org> > > >>>> > > >>> > > >> >
