Sorry, forgot to add bigtop dev mailing list -------- Original Message -------- Subject: Apache Flume NG 1.0.0 with Apache Hadoop 0.20.205 and Apache Hadoop 0.23 Date: Fri, 06 Jan 2012 18:49:32 -0800 From: Bruno Mahé <[email protected]> To: [email protected]
Hi, I am pleased to announce that Apache Flume 1.0.0 (incubating) was packaged and pushed to Apache Bigtop trunk (working with Apache hadoop 0.20.205) as well as the branch dedicated to Hadoop 0.23. Flume 1.0.0 has been pushed as a simple update to Bigtop trunk. Regarding Flume packages in the hadoop 0.23 branch, they are named flume-ng and flume-ng-node. flume-ng provides most of the files, while flume-ng-node provides an init script named flume-ng-node. flume-ng-node will start a node by pointing its configuration directory to /etc/flume-ng/conf and the configuration file /etc/flume-ng/conf.empty/flume.conf (empty by default). All the output is redirected to /var/log/flume-ng/flume-ng-node.log. Note also that flume-ng packages can be installed and run in parallel with flume 0.9.3 already provided by Apache Bigtop. Repositories and virtual machines will be available shortly as soon as the builds complete. To install repositories on your machine for flume 1.0.0 and hadoop 0.20.205 on your favourite os: * CentOS/RHEL 5: Download http://bigtop01.cloudera.org:8080/view/Trunk/job/Bigtop-trunk-matrix/label=centos5/lastSuccessfulBuild/artifact/output/bigtop.repo in /etc/yum.repos.d/ * CentOS/RHEL 6: Download http://bigtop01.cloudera.org:8080/view/Trunk/job/Bigtop-trunk-matrix/label=centos6/lastSuccessfulBuild/artifact/output/bigtop.repo in /etc/yum.repos.d/ * Fedora 15: Download http://bigtop01.cloudera.org:8080/view/Trunk/job/Bigtop-trunk-matrix/label=fedora15/lastSuccessfulBuild/artifact/output/bigtop.repo in /etc/yum.repos.d/ * Fedora 16: Downloadhttp://bigtop01.cloudera.org:8080/view/Trunk/job/Bigtop-trunk-matrix/label=fedora16/lastSuccessfulBuild/artifact/output/ bigtop.repo in /etc/yum.repos.d/ * Ubuntu Lucid: Downloadhttp://bigtop01.cloudera.org:8080/view/Trunk/job/Bigtop-trunk-matrix/label=lucid/lastSuccessfulBuild/artifact/output/bigtop.list in /etc/apt/sources.list.d/ * SLES 11: Type as the root user: zypper ar -f http://bigtop01.cloudera.org:8080/view/Trunk/job/Bigtop-trunk-matrix/label=sles11/lastSuccessfulBuild/artifact/output/bigtop.repo To install repositories on your machine for flume 1.0.0 and hadoop 0.23 branch on your favourite os: * CentOS/RHEL 5: Download http://bigtop01.cloudera.org:8080/view/Hadoop%200.23/job/Bigtop-23-matrix/label=centos5/lastSuccessfulBuild/artifact/output/bigtop.repo in /etc/yum.repos.d/ * CentOS/RHEL 6: Download http://bigtop01.cloudera.org:8080/view/Hadoop%200.23/job/Bigtop-23-matrix/label=centos6/lastSuccessfulBuild/artifact/output/bigtop.repo in /etc/yum.repos.d/ * Fedora 15: Download http://bigtop01.cloudera.org:8080/view/Hadoop%200.23/job/Bigtop-23-matrix/label=fedora15/lastSuccessfulBuild/artifact/output/bigtop.repo in /etc/yum.repos.d/ * Fedora 16: Download http://bigtop01.cloudera.org:8080/view/Hadoop%200.23/job/Bigtop-23-matrix/label=fedora16/lastSuccessfulBuild/artifact/output/bigtop.repo in /etc/yum.repos.d/ * Ubuntu Lucid: Download http://bigtop01.cloudera.org:8080/view/Hadoop%200.23/job/Bigtop-23-matrix/label=lucid/lastSuccessfulBuild/artifact/output/bigtop.list in /etc/apt/sources.list.d/ * SLES 11: Type as the root user: zypper ar -f http://bigtop01.cloudera.org:8080/view/Hadoop%200.23/job/Bigtop-23-matrix/label=sles11/lastSuccessfulBuild/artifact/output/bigtop.repo If you wish to download the VMs built from Apache Bigtop (incubating) jenkins instance, here are the links: * flume 1.0.0 and hadoop 0.20.205 in Apache Bigtop trunk: - KVM: http://bit.ly/tlPZCz - VMware: http://bit.ly/s1V42p - VirtualBox: http://bit.ly/s5vuj8 * flume 1.0.0 and hadoop 0.23 branch in Apache Bigtop (branch hadoop-0.23): - KVM: http://bit.ly/sBdEVX - VMware: http://bit.ly/vj22mo - VirtualBox: http://bit.ly/ttwm5k * The jenkins job creating these images is located there: http://bit.ly/vM3tLP Once the chosen artefact is downloaded and expanded, you just need to tell your virtualization tool to import an existing disk. Be also careful regarding the RAM allocated to your VM. Apache Hadoop 0.23 has become quite memory hungry and will not be able to run the pi example on a VM with 1024MB of RAM. Please, don't hesitate to share your feedback, ideas or issues on Apache Bigtop (incubating) mailing list or the ticket tracker. Thanks, Bruno
