Hi Evans,
> Am 20.12.2017 um 13:01 schrieb Evans Ye <[email protected]>: > > I am 100% up to go container. We need to embrace the changes and right now > container seems to be a dominating solution. > > Breaking down into a concrete plan, we got some choices: > > 1. Leverage docker provisiner to provision cluster on swarm using compose Docker provisioner AFAIK is like having virtual machines in docker: Installing more than one service/all services in one container. Why not splitting different services into seperate containers in order to remove interdependencies between packages? > 2. Develop docker based packaging solution > 2a. Reuse our RPM/DEB packages to create docker images 2a: Yes, please. > 2b. Develop a brand new packaging solution for docker and drop puppet deploy > > I would favor 2a. > > Any other thoughts? > > > > Olaf Flebbe <[email protected]>於 2017年12月17日 週日，上午1:57寫道： > >> Hi Jeff, >> >> Regarding your comment on *https://issues.apache.org/jira/browse/BIGTOP-2754 >> <https://issues.apache.org/jira/browse/BIGTOP-2754>* >> >> Well, pretty ugly story. The thread leading to reverting zookeeper is >> here: >> >> >> https://lists.apache.org/thread.html/c2e9705ee7bdc8afb7b9e79683dda18b0079aec215079b608862c99b@%3Cdev.bigtop.apache.org%3E >> >> Included you will find the unresolved Hadoop JIRA tickets >> >> https://issues.apache.org/jira/browse/HADOOP-12928 >> >> https://issues.apache.org/jira/browse/HADOOP-13413 >> >> >> However, Jay flashed a proposal several times that we should "go >> container". I am now ready to see this necessary. >> >> I think the hadoop ecosystem should interoperate on stable network >> protocols, not by sharing Java artifacts. >> >> This would put the final nail in the coffin of the attempt harmonising >> protocols by symlinking client and server packages in order to enforce same >> versions of client and server. We would give up most of our inter-package >> dependencies, every package get its own dependency packaged, pulled from >> whatever the upstream project decided. >> >> This may solve issues like the zookeeper / hadoop interdependency . >> Supplying a zookeeper-3.11 server package would be not an issue any more. >> Packages needing zookeeper client would use their zookeeper client to >> access the server. And than think "Hadoop-3" as a further example. >> >> For dependencies with native libraries this may lead to issues, which can >> be solved IMHO. >> >> I am now out of the position to control a reasonable large hadoop bigtop >> infra, so I am not sure this path will lead to disaster area, since I >> cannot test the long time stability any more. >> >> Other Bigtoppers, please comment, since this will have pretty big impact . >> >> I like to see traditional Bigtop bare metal -- no containers -- supported >> as well. At least as a corner case. >> >> Cheers >> Olaf >> >> >> >> >> Am 16.12.2017 um 02:54 schrieb Jeff Widman (JIRA) <[email protected]>: >> >> >> [ >> https://issues.apache.org/jira/browse/BIGTOP-2754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16293553#comment-16293553 >> ] >> >> Jeff Widman commented on BIGTOP-2754: >> ------------------------------------- >> >> Is there a ticket in Hadoop tracking this? >> >> Zookeeper 3.4.6-->3.4.9 have a couple of bugs affecting snapshot garbage >> collection (ZOOKEEPER-1797 and ZOOKEEPER-2574), so we want to upgrade to >> 3.4.11. >> >> However, when I tried pulling that from Zookeeper repo, I noticed that it >> pins Java 6 which breaks on Ubuntu 16.04: >> https://github.com/apache/zookeeper/blob/release-3.4.6/src/packages/update-zookeeper-env.sh#L152 >> >> I looked at the Zookeeper master branch, and looks like Zookeeper decided >> in ZOOKEEPER-1604 to delegate packaging to the Bigtop project. However, >> when I came here, I was disappointed to see that the newest Zookeeper >> version that Bigtop packages is 3.4.6.... >> >> For those of us who use Zookeeper apart from Hadoop but don't want to deal >> with the headaches of packaging it'd be nice to figure out a way to get a >> newer zookeeper package released. I'm happy to take a look at the Hadoop >> project to see if they can upgrade to 3.4.11, but not sure where to find >> their rationale for pinning to 3.4.6... >> >> Revert BIGTOP-2730: Upgrade Zookeeper to version 3.4.10 >> ------------------------------------------------------- >> >> Key: BIGTOP-2754 >> URL: https://issues.apache.org/jira/browse/BIGTOP-2754 >> Project: Bigtop >> Issue Type: Bug >> Affects Versions: 1.2.0 >> Reporter: Olaf Flebbe >> Assignee: Olaf Flebbe >> Fix For: 1.2.1, 1.3.0 >> >> Attachments: BIGTOP-2754.patch >> >> >> Since Hadoop enforces zookeeper-3.6 we will revert BIGTOP-2730 in order >> to wait for upstream to deal with this issue. >> >> >> >> >> -- >> This message was sent by Atlassian JIRA >> (v6.4.14#64029) >> >>
