Re: build hbase2.1 without htrace-core-3.1.0-incubating.jar
The previous snippet was generated on hbase-2.0 branch. Regenerating the dependency tree out with hbase-2.1 branch, I don't see 3.2.0 of htrace at all. FYI On Mon, Aug 20, 2018 at 6:32 PM Sean Busbey wrote: > That does not look like an Apache HBase version number on > hbase-server. 陈叶超 even said "HBase 2.1" in the original email. > > > > > On Mon, Aug 20, 2018 at 2:17 PM, Ted Yu wrote: > > Looking at the dependency tree output, I see the following: > > > > [INFO] org.apache.hbase:hbase-server:jar:2.0.0.3.0.0.0-SNAPSHOT > > ... > > [INFO] +- org.apache.htrace:htrace-core:jar:3.2.0-incubating:compile > > > > FYI > > > > On Mon, Aug 20, 2018 at 8:10 AM Sean Busbey wrote: > > > >> neither Hadoop 3.1 nor HBase 2.1 use that version of HTrace. what are > >> you trying to do? > >> > >> On Sun, Aug 19, 2018 at 11:24 PM, 陈叶超 > >> wrote: > >> > hi: > >> > > >> > > >> > I build hbase 2.1 with hadoop 3.1,and will lost > >> htrace-core-3.1.0-incubating.jar in \lib\client-facing-thirdparty\ > >> > > >> > but i found in the apache site download tar.gz , > >> > > >> > my build command is : "mvn -P build-with-jdk8,hadoop-3.0 > >> -DskipTests=true clean package install assembly:single" > >> > > >> > what am i missing ? > >> > > >> > > >> > > >> > > >> > 陈叶超 Yechao Chen > >> > 中移（苏州）软件技术有限公司|中国移动苏州研发中心 | 大数据产品部 > >> > China Mobile (Suzhou) Software Technology Co., Ltd. > >> > Mobile: (+86) 18896724791 > >> > Email: chenyec...@cmss.chinamobile.com > >> >
Re: build hbase2.1 without htrace-core-3.1.0-incubating.jar
That does not look like an Apache HBase version number on hbase-server. 陈叶超 even said "HBase 2.1" in the original email. On Mon, Aug 20, 2018 at 2:17 PM, Ted Yu wrote: > Looking at the dependency tree output, I see the following: > > [INFO] org.apache.hbase:hbase-server:jar:2.0.0.3.0.0.0-SNAPSHOT > ... > [INFO] +- org.apache.htrace:htrace-core:jar:3.2.0-incubating:compile > > FYI > > On Mon, Aug 20, 2018 at 8:10 AM Sean Busbey wrote: > >> neither Hadoop 3.1 nor HBase 2.1 use that version of HTrace. what are >> you trying to do? >> >> On Sun, Aug 19, 2018 at 11:24 PM, 陈叶超 >> wrote: >> > hi: >> > >> > >> > I build hbase 2.1 with hadoop 3.1,and will lost >> htrace-core-3.1.0-incubating.jar in \lib\client-facing-thirdparty\ >> > >> > but i found in the apache site download tar.gz , >> > >> > my build command is : "mvn -P build-with-jdk8,hadoop-3.0 >> -DskipTests=true clean package install assembly:single" >> > >> > what am i missing ? >> > >> > >> > >> > >> > 陈叶超 Yechao Chen >> > 中移（苏州）软件技术有限公司|中国移动苏州研发中心 | 大数据产品部 >> > China Mobile (Suzhou) Software Technology Co., Ltd. >> > Mobile: (+86) 18896724791 >> > Email: chenyec...@cmss.chinamobile.com >>
Re: Phoenix CsvBulkLoadTool fails with java.sql.SQLException: ERROR 103 (08004): Unable to establish connection
(-cc user@hbase, +bcc user@hbase)
How about the rest of the stacktrace? You didn't share the cause.
On 8/20/18 1:35 PM, Mich Talebzadeh wrote:
This was working fine before my Hbase upgrade to 1.2.6
I have Hbase version 1.2.6 and Phoenix
version apache-phoenix-4.8.1-HBase-1.2-bin
This command bulkloading into Hbase through phoenix failsnow fails
HADOOP_CLASSPATH=${HOME}/jars/hbase-protocol-1.2.6.jar:${HBASE_HOME}/conf hadoop
jar ${HBASE_HOME}/lib/phoenix-4.8.1-HBase-1.2-client.jar
org.apache.phoenix.mapreduce.CsvBulkLoadTool --table ${TABLE_NAME}
--input hdfs://rhes75:9000/${REFINED_HBASE_SUB_DIR}/${FILE_NAME}_${dir}.txt
hadoop jar
/data6/hduser/hbase-1.2.6/lib/phoenix-4.8.1-HBase-1.2-client.jar
org.apache.phoenix.mapreduce.CsvBulkLoadTool --table
MARKETDATAHBASEBATCH --input
hdfs://rhes75:9000//data/prices/2018-08-20_refined/populate_Phoenix_table_MARKETDATAHBASEBATCH_2018-08-20.txt
+
HADOOP_CLASSPATH=/home/hduser/jars/hbase-protocol-1.2.6.jar:/data6/hduser/hbase-1.2.6/conf
With the following error
2018-08-20 18:29:47,248 INFO [main] zookeeper.ZooKeeper: Client
environment:java.library.path=/home/hduser/hadoop-3.1.0/lib
2018-08-20 18:29:47,248 INFO [main] zookeeper.ZooKeeper: Client
environment:java.io.tmpdir=/tmp
2018-08-20 18:29:47,248 INFO [main] zookeeper.ZooKeeper: Client
environment:java.compiler=
2018-08-20 18:29:47,248 INFO [main] zookeeper.ZooKeeper: Client
environment:os.name =Linux
2018-08-20 18:29:47,248 INFO [main] zookeeper.ZooKeeper: Client
environment:os.arch=amd64
2018-08-20 18:29:47,248 INFO [main] zookeeper.ZooKeeper: Client
environment:os.version=3.10.0-862.3.2.el7.x86_64
2018-08-20 18:29:47,248 INFO [main] zookeeper.ZooKeeper: Client
environment:user.name =hduser
2018-08-20 18:29:47,248 INFO [main] zookeeper.ZooKeeper: Client
environment:user.home=/home/hduser
2018-08-20 18:29:47,248 INFO [main] zookeeper.ZooKeeper: Client
environment:user.dir=/data6/hduser/streaming_data/2018-08-20
2018-08-20 18:29:47,249 INFO [main] zookeeper.ZooKeeper: Initiating
client connection, connectString=rhes75:2181 sessionTimeout=9
watcher=hconnection-0x493d44230x0, quorum=rhes75:2181, baseZNode=/hbase
2018-08-20 18:29:47,261 INFO [main-SendThread(rhes75:2181)]
zookeeper.ClientCnxn: Opening socket connection to server
rhes75/50.140.197.220:2181 . Will not
attempt to authenticate using SASL (unknown error)
2018-08-20 18:29:47,264 INFO [main-SendThread(rhes75:2181)]
zookeeper.ClientCnxn: Socket connection established to
rhes75/50.140.197.220:2181 , initiating session
2018-08-20 18:29:47,281 INFO [main-SendThread(rhes75:2181)]
zookeeper.ClientCnxn: Session establishment complete on server
rhes75/50.140.197.220:2181 , sessionid =
0x1002ea99eed0077, negotiated timeout = 4
Exception in thread "main" java.sql.SQLException: ERROR 103 (08004):
Unable to establish connection.
at
org.apache.phoenix.exception.SQLExceptionCode$Factory$1.newException(SQLExceptionCode.java:455)
Any thoughts?
Thanks
Dr Mich Talebzadeh
LinkedIn
/https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw/
http://talebzadehmich.wordpress.com
*Disclaimer:* Use it at your own risk.Any and all responsibility for any
loss, damage or destruction of data or any other property which may
arise from relying on this email's technical content is explicitly
disclaimed. The author will in no case be liable for any monetary
damages arising from such loss, damage or destruction.
Re: build hbase2.1 without htrace-core-3.1.0-incubating.jar
Looking at the dependency tree output, I see the following: [INFO] org.apache.hbase:hbase-server:jar:2.0.0.3.0.0.0-SNAPSHOT ... [INFO] +- org.apache.htrace:htrace-core:jar:3.2.0-incubating:compile FYI On Mon, Aug 20, 2018 at 8:10 AM Sean Busbey wrote: > neither Hadoop 3.1 nor HBase 2.1 use that version of HTrace. what are > you trying to do? > > On Sun, Aug 19, 2018 at 11:24 PM, 陈叶超 > wrote: > > hi: > > > > > > I build hbase 2.1 with hadoop 3.1,and will lost > htrace-core-3.1.0-incubating.jar in \lib\client-facing-thirdparty\ > > > > but i found in the apache site download tar.gz , > > > > my build command is : "mvn -P build-with-jdk8,hadoop-3.0 > -DskipTests=true clean package install assembly:single" > > > > what am i missing ? > > > > > > > > > > 陈叶超 Yechao Chen > > 中移（苏州）软件技术有限公司|中国移动苏州研发中心 | 大数据产品部 > > China Mobile (Suzhou) Software Technology Co., Ltd. > > Mobile: (+86) 18896724791 > > Email: chenyec...@cmss.chinamobile.com >
Phoenix CsvBulkLoadTool fails with java.sql.SQLException: ERROR 103 (08004): Unable to establish connection
This was working fine before my Hbase upgrade to 1.2.6
I have Hbase version 1.2.6 and Phoenix
version apache-phoenix-4.8.1-HBase-1.2-bin
This command bulkloading into Hbase through phoenix failsnow fails
HADOOP_CLASSPATH=${HOME}/jars/hbase-protocol-1.2.6.jar:${HBASE_HOME}/conf
hadoop jar ${HBASE_HOME}/lib/phoenix-4.8.1-HBase-1.2-client.jar
org.apache.phoenix.mapreduce.CsvBulkLoadTool --table ${TABLE_NAME} --input
hdfs://rhes75:9000/${REFINED_HBASE_SUB_DIR}/${FILE_NAME}_${dir}.txt
hadoop jar /data6/hduser/hbase-1.2.6/lib/phoenix-4.8.1-HBase-1.2-client.jar
org.apache.phoenix.mapreduce.CsvBulkLoadTool --table MARKETDATAHBASEBATCH
--input
hdfs://rhes75:9000//data/prices/2018-08-20_refined/populate_Phoenix_table_MARKETDATAHBASEBATCH_2018-08-20.txt
+
HADOOP_CLASSPATH=/home/hduser/jars/hbase-protocol-1.2.6.jar:/data6/hduser/hbase-1.2.6/conf
With the following error
2018-08-20 18:29:47,248 INFO [main] zookeeper.ZooKeeper: Client
environment:java.library.path=/home/hduser/hadoop-3.1.0/lib
2018-08-20 18:29:47,248 INFO [main] zookeeper.ZooKeeper: Client
environment:java.io.tmpdir=/tmp
2018-08-20 18:29:47,248 INFO [main] zookeeper.ZooKeeper: Client
environment:java.compiler=
2018-08-20 18:29:47,248 INFO [main] zookeeper.ZooKeeper: Client
environment:os.name=Linux
2018-08-20 18:29:47,248 INFO [main] zookeeper.ZooKeeper: Client
environment:os.arch=amd64
2018-08-20 18:29:47,248 INFO [main] zookeeper.ZooKeeper: Client
environment:os.version=3.10.0-862.3.2.el7.x86_64
2018-08-20 18:29:47,248 INFO [main] zookeeper.ZooKeeper: Client
environment:user.name=hduser
2018-08-20 18:29:47,248 INFO [main] zookeeper.ZooKeeper: Client
environment:user.home=/home/hduser
2018-08-20 18:29:47,248 INFO [main] zookeeper.ZooKeeper: Client
environment:user.dir=/data6/hduser/streaming_data/2018-08-20
2018-08-20 18:29:47,249 INFO [main] zookeeper.ZooKeeper: Initiating client
connection, connectString=rhes75:2181 sessionTimeout=9
watcher=hconnection-0x493d44230x0, quorum=rhes75:2181, baseZNode=/hbase
2018-08-20 18:29:47,261 INFO [main-SendThread(rhes75:2181)]
zookeeper.ClientCnxn: Opening socket connection to server rhes75/
50.140.197.220:2181. Will not attempt to authenticate using SASL (unknown
error)
2018-08-20 18:29:47,264 INFO [main-SendThread(rhes75:2181)]
zookeeper.ClientCnxn: Socket connection established to rhes75/
50.140.197.220:2181, initiating session
2018-08-20 18:29:47,281 INFO [main-SendThread(rhes75:2181)]
zookeeper.ClientCnxn:
Session establishment complete on server rhes75/50.140.197.220:2181,
sessionid = 0x1002ea99eed0077, negotiated timeout = 4
Exception in thread "main" java.sql.SQLException: ERROR 103 (08004): Unable
to establish connection.
at
org.apache.phoenix.exception.SQLExceptionCode$Factory$1.newException(SQLExceptionCode.java:455)
Any thoughts?
Thanks
Dr Mich Talebzadeh
LinkedIn *
https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
*
http://talebzadehmich.wordpress.com
*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.
Re: build hbase2.1 without htrace-core-3.1.0-incubating.jar
neither Hadoop 3.1 nor HBase 2.1 use that version of HTrace. what are you trying to do? On Sun, Aug 19, 2018 at 11:24 PM, 陈叶超 wrote: > hi: > > > I build hbase 2.1 with hadoop 3.1,and will lost > htrace-core-3.1.0-incubating.jar in \lib\client-facing-thirdparty\ > > but i found in the apache site download tar.gz , > > my build command is : "mvn -P build-with-jdk8,hadoop-3.0 -DskipTests=true > clean package install assembly:single" > > what am i missing ? > > > > > 陈叶超 Yechao Chen > 中移（苏州）软件技术有限公司|中国移动苏州研发中心 | 大数据产品部 > China Mobile (Suzhou) Software Technology Co., Ltd. > Mobile: (+86) 18896724791 > Email: chenyec...@cmss.chinamobile.com
build hbase2.1 without htrace-core-3.1.0-incubating.jar
hi: I build hbase 2.1 with hadoop 3.1,and will lost htrace-core-3.1.0-incubating.jar in \lib\client-facing-thirdparty\ but i found in the apache site download tar.gz , my build command is : "mvn -P build-with-jdk8,hadoop-3.0 -DskipTests=true clean package install assembly:single" what am i missing ? 陈叶超 Yechao Chen 中移（苏州）软件技术有限公司|中国移动苏州研发中心 | 大数据产品部 China Mobile (Suzhou) Software Technology Co., Ltd. Mobile: (+86) 18896724791 Email: chenyec...@cmss.chinamobile.com
hbase logs show the performance of compact isnt good. any suggstion to find out the reason?
Hi All, I'm a newbie against Hbase. When I use YCSB to load data into Hbase, the compaction in two cluster looks different. DEV: 6 nodes. A and B are master and standby for HDFS and Hbase. C/D/E/F are the HDFS datanode and regionservers. PROD: 8 nodes. A and B are master and standby for HDFS and Hbase. C/D/E/F/G/H are the HDFS datanode and regionservers. Network is 1000mbs. Hadoop: 2.7.3 Hbase: 1.2.6 JDK:1.8 Zookeep: install on C/D/E In DEV, I notice the compaction 4G files are about 30 seconds; however, it tooks about 30 seconds to process 400MB files in PROD. For example, this is logs from one node of DEV: 2018-08-10 16:30:47,535 INFO [regionserver/node103/172.28.200.103:16020-shortCompactions-1533803656579] regionserver.HStore: Completed compaction of 6 file(s) in family of usertable_nosplit,,1533878791857.be420543a609c32475386855dee93a82. into abe3a5c0dbbd4708ae2523f9b4a53ad8(size=1.4 G), total size for store is 4.0 G. This selection was in queue for 0sec, and took 18sec to execute. This is from PROD: 2018-08-17 23:29:28,751 INFO [regionserver/YJB-HADOOP-74-23/192.168.74.23:16020-shortCompactions-1534519629607] regionserver.HStore: Completed compaction of 3 (all) file(s) in family of usertable_nosplit,,1534518587365.95595277af1c56260b85de113c9687f1. into 934669bf6a8a407aa6ee385439c4a7b5(size=430.8 M), total size for store is 430.8 M. This selection was in queue for 0sec, and took 30sec to execute. I thought this might be a problem of GC. So I change to use CG1 with this article "https://product.hubspot.com/blog/g1gc-tuning-your-hbase-cluster; . The performance improve a little, not to good. Is there any suggestion to find out what causes the problem and improve the performance? BTW: I'm not sure what kind information can help so I attach my hbase-site.xml and hbase-env.sh first. BTW2: to use GC1 against regionserver, I create a speraeted hbase-env.sh for regionserver. Thanks, Dillon hbase-env(regionserver).sh Description: hbase-env(regionserver).sh hbase-env(master).sh Description: hbase-env(master).sh hbase.rootdir hdfs://HaHdfs/apps/hbase/data hbase.cluster.distributed true hbase.master 16000 hbase.tmp.dir /opt/app/hbase/tmp hbase.zookeeper.quorum YJB-HADOOP-74-23,YJB-HADOOP-74-24,YJB-HADOOP-74-25 hbase.zookeeper.property.dataDir /opt/app/zookeeper-3.4.9/data hbase.zookeeper.property.clientPort 2181 zookeeper.session.timeout 12 hbase.regionserver.restart.on.zk.expire true hbase.ipc.server.callqueue.handler.factor 0.5 hbase.regionserver.region.split.policy org.apache.hadoop.hbase.regionserver.DisabledRegionSplitPolicy hbase.regionserver.handler.count 30 hbase.hregion.memstore.flush.size 268435456 hbase.hstore.compaction.max 20 hbase.hstore.compaction.min 3 hbase.hstore.compaction.min.size 268435456
