That's right, I'm using CentOS 7. --David
On Thursday, 17 March 2016, Konstantin Boudnik <[email protected] <javascript:_e(%7B%7D,'cvml','[email protected]');>> wrote: > Hmm... that's plain nuts... Looks like latest ignite is pulling in some > crap > like osgi... OMG, WTF? > > Thanks for discovering this David! Somehow it never got stepped upon in my > tests. Are you on centos7? > > Cos > > On Wed, Mar 16, 2016 at 11:39PM, David Starina wrote: > > Hi, > > > > Roman, Evans, thank you for the tips - that helped a lot. Still have some > > problems though, an error when running puppet apply ... > > > > This is a sample of errors I got: > > > > > > *Debug: Executing '/bin/yum -d 0 -e 0 -y install > > ignite-hadoop-service'Error: Could not update: Execution of '/bin/yum -d > 0 > > -e 0 -y install ignite-hadoop-service' returned 1: Error: Package: > > ignite-hadoop-1.5.0.final-1.el7.centos.noarch (Bigtop) > Requires: > > osgi(org.apache.servicemix.bundles.spring-beans)Error: Package: > > ignite-hadoop-1.5.0.final-1.el7.centos.noarch (Bigtop) > Requires: > > osgi(org.apache.servicemix.bundles.spring-context-support)Error: Package: > > ignite-hadoop-1.5.0.final-1.el7.centos.noarch (Bigtop) > Requires: > > osgi(org.apache.servicemix.bundles.spring-jdbc)Error: Package: > > ignite-hadoop-1.5.0.final-1.el7.centos.noarch (Bigtop) > Requires: > > osgi(org.apache.servicemix.bundles.spring-core)* > > ..... > > > > Also, after that: > > > > > > > > > > > > > > *Notice: > > > /Stage[main]/Ignite_hadoop::Deploy/Ignite_hadoop::Server[ignite-hadoop-node]/File[/etc/default/ignite-hadoop]: > > Dependency Package[ignite-hadoop] has failures: trueWarning: > > > /Stage[main]/Ignite_hadoop::Deploy/Ignite_hadoop::Server[ignite-hadoop-node]/File[/etc/default/ignite-hadoop]: > > Skipping because of failed dependenciesNotice: > > /Stage[main]/Hadoop::Datanode/Package[hadoop-hdfs-datanode]: Dependency > > Package[openjdk-7-jre-headless] has failures: trueWarning: > > /Stage[main]/Hadoop::Datanode/Package[hadoop-hdfs-datanode]: Skipping > > because of failed dependenciesNotice: > > /Stage[main]/Hadoop::Nodemanager/Package[hadoop-yarn-nodemanager]: > > Dependency Package[openjdk-7-jre-headless] has failures: trueWarning: > > /Stage[main]/Hadoop::Nodemanager/Package[hadoop-yarn-nodemanager]: > Skipping > > because of failed dependenciesNotice: > > > /Stage[main]/Ignite_hadoop::Deploy/Ignite_hadoop::Server[ignite-hadoop-node]/File[/etc/ignite-hadoop/conf/default-config.xml]: > > Dependency Package[ignite-hadoop] has failures: true* > > ..... > > > > > > This is the "site.yaml" I used ("hadoop" wasn't listed in components, > but I > > think it should be there?) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > *bigtop::hadoop_head_node: "NameNode"hadoop::hadoop_storage_dirs: - > > /data/1hadoop_cluster_node::cluster_components: - hadoop - > ignite_hadoop > > - hue - mahout - yarn - zookeeperbigtop::jdk_package_name: > > "openjdk-7-jre-headless"bigtop::bigtop_repo_uri: > > "http://bigtop-repos.s3.amazonaws.com/releases/1.1.0/centos/7/x86_64 > > <http://bigtop-repos.s3.amazonaws.com/releases/1.1.0/centos/7/x86_64>"* > > > > > > > > > > Any idea what I am doing wrong? > > > > --David > > > > On Tue, Mar 15, 2016 at 6:57 PM, Evans Ye <[email protected]> wrote: > > > > > David > > > > > > Most of the time you need to run same command on all nodes. "pdsh" > would > > > be a good tool to use. > > > > > > 1. You need to have puppet agent installed on every node. yum -y > install > > > puppet will do the trick, or you need to download repo, or use > > > bigtop/bigtop_toolchain/bin/puppetize.sh directly. There's no deploy > effort > > > other than just install the package. > > > > > > 2. Masterless means each puppet agent takes commands and runs > > > independently. We DONT use puppet master, which is a centralized > master > > > daemon to dispatch works to all the registered puppet agent. > > > > > > 3. Next, you need to copy bigtop puppet recipes and bigtop hiera > > > configuration to all nodes, or at least make them available for all > nodes > > > via nfs. > > > > > > 4. Finally, execute the puppet apply command on all nodes and the > puppet > > > recipes will automatically figure out it's role. There no need to > deploy > > > head node first and then slaves. > > > > > > Ask if you still have questions. > > > > > > > > > Thanks > > > > > > Evans > > > 2016年3月15日 下午4:10於 "David Starina" <[email protected]>寫道： > > > > > >> Hi guys, > > >> > > >> me again. I haven't set up Puppet before, can you just point me in the > > >> right direction? I know this is not a Puppet mailing list, but I bet > you > > >> have some tips for me :-) > > >> > > >> So what do I need to do? Install Puppet on every node? What does it > mean > > >> that the mode of Puppet is "*masterless"* - do I just have to do a > basic > > >> installation of Puppet and not care much about setting it up, and then > > >> deploy bigtop on every node separately? And when setting the Hadoop > head > > >> node - does that mean that I need to deploy Bigtop on this node > first, and > > >> Bigtop will know this is the master node by the url or IP I provide > to it? > > >> And then deploy Bigtop via Puppet on all the other nodes and they will > > >> connect to the head node (which is the NameNode, I suppose?). > > >> > > >> I can do this via trial and error, but if you guys can help me reduce > the > > >> "error" part, that will be great :) Sorry for such beginner > questions, I > > >> haven't used Puppet before ... The reason I'm asking is, I got access > to a > > >> "physical" (actually virtual, but it's the same) environment for a > limited > > >> amount of time (two weeks), and I need to start testing on it asap - > and > > >> since my day job takes a lot of my time, if I wanted to do this > without > > >> help it could take me two weeks just to set up the environment - but > I need > > >> to install it, which is not the central part of my project, and then > run > > >> some ML algorithms on a text corpus, which will be the main part of > the > > >> project ... > > >> > > >> --David > > >> > > >> On Thu, Mar 10, 2016 at 11:28 AM, David Starina < > [email protected]> > > >> wrote: > > >> > > >>> Thanks for the info! > > >>> > > >>> --David > > >>> > > >>> On Tue, Mar 8, 2016 at 7:15 PM, Jay Vyas < > [email protected]> > > >>> wrote: > > >>> > > >>>> I think there is enough info on youtube that you can cobble > together a > > >>>> syllabus if you need structured learning. > > >>>> > > >>>> See cos/Romans Bigtop talks which are on YouTube and then my Bigtop > > >>>> internals hackathon video. That gives you both the high and low > level > > >>>> details to do whatever you want. > > >>>> > > >>>> > On Mar 8, 2016, at 1:11 PM, Konstantin Boudnik <[email protected]> > > >>>> wrote: > > >>>> > > > >>>> > We also had this discussion to start producing "The Big Book of > Apache > > >>>> > Bigtop" like HBase does, but never got into the materialization > part > > >>>> of that > > >>>> > project. > > >>>> > > > >>>> > Cos > > >>>> > > > >>>> >> On Tue, Mar 08, 2016 at 11:44PM, Evans Ye wrote: > > >>>> >> Hi David, > > >>>> >> > > >>>> >> If you just want to take Bigtop binaries to deploy a cluster, you > > >>>> can refer > > >>>> >> to our puppet management recipes: > > >>>> >> > https://github.com/apache/bigtop/tree/master/bigtop-deploy/puppet > > >>>> >> > > >>>> >> BTW bigtop provisioner is the best source to reference for how > bigtop > > >>>> >> provision a cluster: > > >>>> >> > > >>>> > https://cwiki.apache.org/confluence/display/BIGTOP/Bigtop+Provisioner+User+Guide > > >>>> >> > > >>>> >> Please ask us if you find anything confusing. > > >>>> >> > > >>>> >> > > >>>> >> Evans > > >>>> >> > > >>>> >> > > >>>> >> > > >>>> >> 2016-03-08 0:48 GMT+08:00 David Starina <[email protected] > >: > > >>>> >> > > >>>> >>> Hi, > > >>>> >>> > > >>>> >>> Is there some version of Bigtop User Guide available? I know it > > >>>> used to be > > >>>> >>> on the be on http://bigtop.apache.org/ under the "User Guide" > > >>>> link, but > > >>>> >>> there's nothing there anymore ... > > >>>> >>> > > >>>> >>> I need some guide on how to install Bigtop on a physical cluster > > >>>> ... It > > >>>> >>> was easy to create a virtual cluster on my machine using > Vagrant, > > >>>> but I > > >>>> >>> cannot find a manual to install on a physical cluster ... > > >>>> >>> > > >>>> >>> Cheers, > > >>>> >>> David > > >>>> >>> > > >>>> > > >>> > > >>> > > >> >
