Hi Jeff, Regarding your comment on https://issues.apache.org/jira/browse/BIGTOP-2754
Well, pretty ugly story. The thread leading to reverting zookeeper is here: https://lists.apache.org/thread.html/c2e9705ee7bdc8afb7b9e79683dda18b0079aec215079b608862c99b@%3Cdev.bigtop.apache.org%3E <https://lists.apache.org/thread.html/c2e9705ee7bdc8afb7b9e79683dda18b0079aec215079b608862c99b@%3Cdev.bigtop.apache.org%3E> Included you will find the unresolved Hadoop JIRA tickets https://issues.apache.org/jira/browse/HADOOP-12928 <https://issues.apache.org/jira/browse/HADOOP-12928> https://issues.apache.org/jira/browse/HADOOP-13413 <https://issues.apache.org/jira/browse/HADOOP-13413> However, Jay flashed a proposal several times that we should "go container". I am now ready to see this necessary. I think the hadoop ecosystem should interoperate on stable network protocols, not by sharing Java artifacts. This would put the final nail in the coffin of the attempt harmonising protocols by symlinking client and server packages in order to enforce same versions of client and server. We would give up most of our inter-package dependencies, every package get its own dependency packaged, pulled from whatever the upstream project decided. This may solve issues like the zookeeper / hadoop interdependency . Supplying a zookeeper-3.11 server package would be not an issue any more. Packages needing zookeeper client would use their zookeeper client to access the server. And than think "Hadoop-3" as a further example. For dependencies with native libraries this may lead to issues, which can be solved IMHO. I am now out of the position to control a reasonable large hadoop bigtop infra, so I am not sure this path will lead to disaster area, since I cannot test the long time stability any more. Other Bigtoppers, please comment, since this will have pretty big impact . I like to see traditional Bigtop bare metal -- no containers -- supported as well. At least as a corner case. Cheers Olaf > Am 16.12.2017 um 02:54 schrieb Jeff Widman (JIRA) <[email protected]>: > > > [ > https://issues.apache.org/jira/browse/BIGTOP-2754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16293553#comment-16293553 > ] > > Jeff Widman commented on BIGTOP-2754: > ------------------------------------- > > Is there a ticket in Hadoop tracking this? > > Zookeeper 3.4.6-->3.4.9 have a couple of bugs affecting snapshot garbage > collection (ZOOKEEPER-1797 and ZOOKEEPER-2574), so we want to upgrade to > 3.4.11. > > However, when I tried pulling that from Zookeeper repo, I noticed that it > pins Java 6 which breaks on Ubuntu 16.04: > https://github.com/apache/zookeeper/blob/release-3.4.6/src/packages/update-zookeeper-env.sh#L152 > > I looked at the Zookeeper master branch, and looks like Zookeeper decided in > ZOOKEEPER-1604 to delegate packaging to the Bigtop project. However, when I > came here, I was disappointed to see that the newest Zookeeper version that > Bigtop packages is 3.4.6.... > > For those of us who use Zookeeper apart from Hadoop but don't want to deal > with the headaches of packaging it'd be nice to figure out a way to get a > newer zookeeper package released. I'm happy to take a look at the Hadoop > project to see if they can upgrade to 3.4.11, but not sure where to find > their rationale for pinning to 3.4.6... > >> Revert BIGTOP-2730: Upgrade Zookeeper to version 3.4.10 >> ------------------------------------------------------- >> >> Key: BIGTOP-2754 >> URL: https://issues.apache.org/jira/browse/BIGTOP-2754 >> Project: Bigtop >> Issue Type: Bug >> Affects Versions: 1.2.0 >> Reporter: Olaf Flebbe >> Assignee: Olaf Flebbe >> Fix For: 1.2.1, 1.3.0 >> >> Attachments: BIGTOP-2754.patch >> >> >> Since Hadoop enforces zookeeper-3.6 we will revert BIGTOP-2730 in order to >> wait for upstream to deal with this issue. > > > > -- > This message was sent by Atlassian JIRA > (v6.4.14#64029)
signature.asc
Description: Message signed with OpenPGP
