Hi, For sles we need 1 more command line option -Dhadoop.tar="http://public-repo-1.hortonworks.com/HDP/sles11sp1/2.x/GA/2.2.0.0/tars/hadoop-2.6.0.2.2.0.0-2041.tar.gz";
For CentOS5 -Dhadoop.tar="http://public-repo-1.hortonworks.com/HDP/centos5/2.x/GA/2.2.0.0/tars/hadoop-2.6.0.2.2.0.0-2041.tar.gz"; BR, Dmytro Sen ________________________________________ From: Siddharth Wagle Sent: Friday, March 06, 2015 7:11 PM To: Dmitry Sen Subject: Fw: AMS issue with SNAPPY ________________________________________ From: Harp, Michael <michael.h...@teradata.com> Sent: Thursday, March 05, 2015 12:59 PM To: Siddharth Wagle Cc: Myroslav Papyrkovskyy Subject: Re: AMS issue with SNAPPY I think our problem is how we are building AMS, did a install with http://s3.amazonaws.com/dev.hortonworks.com/ambari/suse11/2.x/latest/2.0.0. dev/ambari.repo and I am not seeing the problem. Do we have an official way to build AMS? This is my current command; mvn package -Dstorm.version=0.9.3.2.2.1.0-2341 -Dbuild-rpm -DskipTests Thanks, Mike On 3/3/15, 7:56 PM, "Siddharth Wagle" <swa...@hortonworks.com> wrote: >Hi Mike, > >What I have seen while developing AMS is that If native hadoop libs are >not in the classpath or if they are older that latest version, HMaster >refuses to start, however not sure why it would be any different on SLES. > >I will try to repro this tomorrow in-house and share the results. > >BR, >Sid >________________________________________ >From: Harp, Michael <michael.h...@teradata.com> >Sent: Tuesday, March 03, 2015 3:46 PM >To: Siddharth Wagle >Cc: Myroslav Papyrkovskyy >Subject: Re: AMS issue with SNAPPY > >Right saw the reference to Hmaster and thought normal hbase master >service; > >ams 32270 32257 2 17:55 ? 00:00:35 >/opt/teradata/jvm64/jdk7/bin/java -Dproc_master >-XX:OnOutOfMemoryError=kill -9 %p -Xmx1024m -XX:+UseConcMarkSweepGC >-XX:ErrorFile=/var/opt/teradata/log/ambari-metrics-collector/hs_err_pid%p. >l >og >-Djava.io.tmpdir=/var/opt/teradata/lib/ambari-metrics-collector/hbase-tmp >-Djava.library.path=/usr/lib/ams-hbase/lib/hadoop-native/ -verbose:gc >-XX:+PrintGCDetails -XX:+PrintGCDateStamps >-Xloggc:/var/opt/teradata/log/ambari-metrics-collector/gc.log-201503031755 >-Xmx1024m -Dhbase.log.dir=/var/opt/teradata/log/ambari-metrics-collector >-Dhbase.log.file=hbase-ams-master-golem1.log >-Dhbase.home.dir=/usr/lib/ams-hbase/bin/.. -Dhbase.id.str=ams >-Dhbase.root.logger=INFO,RFA -Dhbase.security.logger=INFO,RFAS >org.apache.hadoop.hbase.master.HMaster start > >golem1:~ # ll /usr/lib/ams-hbase/lib/hadoop-native >total 4700 >-rw-r--r-- 1 root root 1315650 2015-03-03 05:17 libhadoop.a >-rw-r--r-- 1 root root 1487716 2015-03-03 05:17 libhadooppipes.a >-rw-r--r-- 1 root root 773720 2015-03-03 05:17 libhadoop.so >-rw-r--r-- 1 root root 582760 2015-03-03 05:17 libhadooputils.a >-rw-r--r-- 1 root root 361210 2015-03-03 05:17 libhdfs.a >-rw-r--r-- 1 root root 229811 2015-03-03 05:17 libhdfs.so >-rw-r--r-- 1 root root 19848 2015-03-03 05:17 libsnappy.so.1 > > > > > > > >On 3/3/15, 3:15 PM, "Siddharth Wagle" <swa...@hortonworks.com> wrote: > >>Hi Mike, >> >>This is hbase process, AMS starts it own self-contained HMaster in >>default settings. >> >>> ps aux | grep ams >> >>This should give you the AMS Hbase process details. >> >>BR, >>Sid >>________________________________________ >>From: Harp, Michael <michael.h...@teradata.com> >>Sent: Tuesday, March 03, 2015 2:01 PM >>To: Siddharth Wagle >>Subject: Re: AMS issue with SNAPPY >> >>I am still seeing the error with a build this morning off >>trunk(f78854ec151e7217f49de8d31943784452b324c2) using HDP-2.2.0.0-2041. >>Not seeing /usr/lib/ams-hbase/lib/hadoop-native/ in the hbase classpath? >> >>hbase 14569 14556 0 07:13 ? 00:01:36 >>/opt/teradata/jvm64/jdk7/bin/java -Dproc_master >>-XX:OnOutOfMemoryError=kill -9 %p -Xmx1000m -Dhdp.version=2.2.0.0-2041 >>-XX:+UseConcMarkSweepGC >>-XX:ErrorFile=/var/opt/teradata/log/hbase/hs_err_pid%p.log -verbose:gc >>-XX:+PrintGCDetails -XX:+PrintGCDateStamps >>-Xloggc:/var/opt/teradata/log/hbase/gc.log-201503030713 -Xmx4096m >>-Dhbase.log.dir=/var/opt/teradata/log/hbase >>-Dhbase.log.file=hbase-hbase-master-golem1.log >>-Dhbase.home.dir=/usr/hdp/current/hbase-master/bin/.. >>-Dhbase.id.str=hbase >>-Dhbase.root.logger=INFO,RFA >>-Djava.library.path=:/usr/hdp/2.2.0.0-2041/hadoop/lib/native/Linux-amd64- >>6 >>4 >>:/usr/hdp/2.2.0.0-2041/hadoop/lib/native >>-Dhbase.security.logger=INFO,RFAS >>org.apache.hadoop.hbase.master.HMaster start >> >>Here is the blueprint I am using just in case its relevant, >>https://gist.github.com/d5fb8c01082c27552bca >>Thanks, >>Mike >> >> >> >> >>On 3/3/15, 1:00 PM, "Siddharth Wagle" <swa...@hortonworks.com> wrote: >> >>>Hi Wally, >>> >>>I see that you do not have the changes made to fix this issue (commit >>>fdd80ec81d93b80899bef593420abb726884e3fc), since it went in on Feb 26 >>>2015. >>>You should be able to port these changes over and re-test. >>> >>>BR, >>>Sid >>> >>>________________________________________ >>>From: Benbenek, Waldyn J <waldyn.benbe...@unisys.com> >>>Sent: Tuesday, March 03, 2015 12:13 PM >>>To: dev@ambari.apache.org >>>Subject: RE: AMS issue with SNAPPY >>> >>>Sid, >>> >>>The Hadoop level is 2.6. Sles is 11 sp3. >>> >>>The libraries are there. I don't what AMS is using for its HMaster >>>process. I will cluster setup the setup and see what I can find out. >>>About the call when it fails. >>> >>>Wally >>> >>> >>> >>>-----Original Message----- >>>From: Siddharth Wagle [mailto:swa...@hortonworks.com] >>>Sent: Tuesday, March 03, 2015 10:33 AM >>>To: dev@ambari.apache.org >>>Subject: Re: AMS issue with SNAPPY >>> >>>Hi, >>> >>>The native hadoop libs have been packaged with AMS at, >>>/usr/lib/ams-hbase/lib/hadoop-native >>>including SNAPPY. This should be set as >>>-Djava.library.path=/usr/lib/ams-hbase/lib/hadoop-native/, for the >>>HMaster process. >>> >>>1. What is the version of the Hadoop stack that is deployed? >>>2. What is the result of this operation on SLES: "which hadoop"? >>> >>>Would one of you like to open a Jira for this? >>> >>>Best Regards, >>>Sid >>>________________________________________ >>>From: Benbenek, Waldyn J <waldyn.benbe...@unisys.com> >>>Sent: Tuesday, March 03, 2015 7:51 AM >>>To: dev@ambari.apache.org >>>Subject: RE: AMS issue with SNAPPY >>> >>>I am having this issue as well. Is this a build or an environment >>>problem? >>> >>>-----Original Message----- >>>From: Harp, Michael [mailto:michael.h...@teradata.com] >>>Sent: Tuesday, February 24, 2015 4:05 PM >>>To: dev@ambari.apache.org >>>Subject: AMS issue with SNAPPY >>> >>>I am only able to get AMS up and running by changing >>>timeline.metrics.hbase.compression.scheme=SNAPPY to >>>timeline.metrics.hbase.compression.scheme=None. >>>Any additional steps needed for snappy beyond installing snappy and >>>snappy-devel? >>>Environment is sles11sp1 using trunk with AMS embedded mode. >>>Thanks, >>>Mike >>> >>>06:43:39,634 ERROR [main] PhoenixHBaseAccessor:306 - Error creating >>>Metrics Schema in HBase using Phoenix. >>>org.apache.phoenix.exception.PhoenixIOException: >>>org.apache.hadoop.hbase.DoNotRetryIOException: Compression algorithm >>>'snappy' previously failed test. >>> >>> >>>
