I understand what you're saying, but that's the problem, and that's where I'm
confused.Only the exception log I sent before, there are no other exceptions,
and there is no exception information in other log files.Here is the log
contents of the new node (slave03:/hbase-hadoop-regionserver-slave03.log).The
horizontal lines separate the contents of start-hbase.sh and hbase-daemon.sh
start regionserver(slave03).This show the fact that by the time the cluster was
started using start-hbase.sh, slave03 had responded but failed to launch a
regionserver.
**********************************************************************Fri Jul
26 09:38:40 CST 2019 Starting regionserver on slave03
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 30733
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 4096
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
2019-07-26 09:38:46,348 WARN [main] util.NativeCodeLoader: Unable to load
native-hadoop library for your platform... using builtin-java classes where
applicable
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Fri Jul 26 09:49:08 CST 2019 Starting regionserver on slave03
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 30733
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 4096
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
2019-07-26 09:49:08,956 INFO [main] regionserver.HRegionServer: STARTING
executorService HRegionServer
2019-07-26 09:49:08,957 INFO [main] util.VersionInfo: HBase 2.1.2
2019-07-26 09:49:08,958 INFO [main] util.VersionInfo: Source code repository
git://kalashnikov/Users/stack/checkouts/hbase.git
revision=1dfc418f77801fbfb59a125756891b9100c1fc6d
2019-07-26 09:49:08,958 INFO [main] util.VersionInfo: Compiled by stack on Sun
Dec 30 21:45:09 PST 2018
2019-07-26 09:49:08,958 INFO [main] util.VersionInfo: From source with
checksum 5701bcce2fc4832a4b2990adc735b516
2019-07-26 09:49:09,471 INFO [main] util.ServerCommandLine: hbase.tmp.dir:
/tmp/hbase-hadoop
2019-07-26 09:49:09,471 INFO [main] util.ServerCommandLine: hbase.rootdir:
hdfs://sicluster/hbase
2019-07-26 09:49:09,471 INFO [main] util.ServerCommandLine:
hbase.cluster.distributed: true
2019-07-26 09:49:09,471 INFO [main] util.ServerCommandLine:
hbase.zookeeper.quorum: slave01,slave02,slave03
2019-07-26 09:49:09,471 INFO [main] util.ServerCommandLine:
env:HBASE_LOGFILE=hbase-hadoop-regionserver-slave03.log
2019-07-26 09:49:09,472 INFO [main] util.ServerCommandLine:
env:PATH=/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/java/jdk1.8.0_191/bin:/usr/java/jdk1.8.0_191/jre/bin:/usr/hadoop/hadoop-2.9.1/bin:/usr/hadoop/hbase-2.1.2/bin:/usr/hadoop/zookeeper-3.4.6/bin:/home/hadoop/.local/bin:/home/hadoop/bin
2019-07-26 09:49:09,472 INFO [main] util.ServerCommandLine:
env:HISTCONTROL=ignoredups
2019-07-26 09:49:09,472 INFO [main] util.ServerCommandLine:
env:XDG_DATA_DIRS=/home/hadoop/.local/share/flatpak/exports/share/:/var/lib/flatpak/exports/share/:/usr/local/share/:/usr/share/
2019-07-26 09:49:09,472 INFO [main] util.ServerCommandLine: env:HISTSIZE=1000
2019-07-26 09:49:09,472 INFO [main] util.ServerCommandLine:
env:JAVA_HOME=/usr/java/jdk1.8.0_191
2019-07-26 09:49:09,472 INFO [main] util.ServerCommandLine: env:TERM=xterm
2019-07-26 09:49:09,472 INFO [main] util.ServerCommandLine:
env:LANG=en_US.UTF-8
2019-07-26 09:49:09,472 INFO [main] util.ServerCommandLine:
env:XDG_SESSION_ID=15
2019-07-26 09:49:09,472 INFO [main] util.ServerCommandLine:
env:SELINUX_LEVEL_REQUESTED=
2019-07-26 09:49:09,472 INFO [main] util.ServerCommandLine:
env:SELINUX_ROLE_REQUESTED=
2019-07-26 09:49:09,472 INFO [main] util.ServerCommandLine:
env:MAIL=/var/spool/mail/hadoop
2019-07-26 09:49:09,472 INFO [main] util.ServerCommandLine:
env:LD_LIBRARY_PATH=:/usr/hadoop/hadoop-2.9.1/lib/native
2019-07-26 09:49:09,472 INFO [main] util.ServerCommandLine: env:LOGNAME=hadoop
2019-07-26 09:49:09,472 INFO [main] util.ServerCommandLine:
env:HBASE_REST_OPTS=
2019-07-26 09:49:09,472 INFO [main] util.ServerCommandLine:
env:PWD=/home/hadoop
2019-07-26 09:49:09,472 INFO [main] util.ServerCommandLine:
env:HBASE_ROOT_LOGGER=INFO,RFA
2019-07-26 09:49:09,472 INFO [main] util.ServerCommandLine:
env:LESSOPEN=||/usr/bin/lesspipe.sh %s
2019-07-26 09:49:09,472 INFO [main] util.ServerCommandLine: env:SHELL=/bin/bash
2019-07-26 09:49:09,472 INFO [main] util.ServerCommandLine:
env:HBASE_HEAPSIZE=2048
2019-07-26 09:49:09,473 INFO [main] util.ServerCommandLine:
env:SELINUX_USE_CURRENT_RANGE=
2019-07-26 09:49:09,473 INFO [main] util.ServerCommandLine:
env:HBASE_ENV_INIT=true
2019-07-26 09:49:09,473 INFO [main] util.ServerCommandLine:
env:HBASE_IDENT_STRING=hadoop
2019-07-26 09:49:09,473 INFO [main] util.ServerCommandLine:
env:HBASE_ZNODE_FILE=/tmp/hbase-hadoop-regionserver.znode
2019-07-26 09:49:09,473 INFO [main] util.ServerCommandLine:
env:SSH_TTY=/dev/pts/0
2019-07-26 09:49:09,473 INFO [main] util.ServerCommandLine:
env:SSH_CLIENT=192.168.174.37 42241 22
2019-07-26 09:49:09,473 INFO [main] util.ServerCommandLine:
env:HBASE_LOG_PREFIX=hbase-hadoop-regionserver-slave03
2019-07-26 09:49:09,473 INFO [main] util.ServerCommandLine:
env:HBASE_LOG_DIR=/usr/hadoop/hbase-2.1.2/logs
2019-07-26 09:49:09,473 INFO [main] util.ServerCommandLine: env:USER=hadoop
2019-07-26 09:49:09,473 INFO [main] util.ServerCommandLine:
party/audience-annotations-0.5.0.jar:/usr/hadoop/hbase-2.1.2/lib/client-facing-thirdparty/commons-logging-1.2.jar:/usr/hadoop/hbase-2.1.2/lib/client-facing-thirdparty/findbugs-annotations-1.3.9-1.jar:/usr/hadoop/hbase-2.1.2/lib/client-facing-thirdparty/htrace-core4-4.2.0-incubating.jar:/usr/hadoop/hbase-2.1.2/lib/client-facing-thirdparty/log4j-1.2.17.jar:/usr/hadoop/hbase-2.1.2/lib/client-facing-thirdparty/slf4j-api-1.7.25.jar:/usr/hadoop/hadoop-2.9.1/etc/hadoop:/usr/hadoop/hadoop-2.9.1/share/hadoop/common/lib/*:/usr/hadoop/hadoop-2.9.1/share/hadoop/common/*:/usr/hadoop/hadoop-2.9.1/share/hadoop/hdfs:/usr/hadoop/hadoop-2.9.1/share/hadoop/hdfs/lib/*:/usr/hadoop/hadoop-2.9.1/share/hadoop/hdfs/*:/usr/hadoop/hadoop-2.9.1/share/hadoop/yarn:/usr/hadoop/hadoop-2.9.1/share/hadoop/yarn/lib/*:/usr/hadoop/hadoop-2.9.1/share/hadoop/yarn/*:/usr/hadoop/hadoop-2.9.1/share/hadoop/mapreduce/lib/*:/usr/hadoop/hadoop-2.9.1/share/hadoop/mapreduce/*:/usr/hadoop/hadoop-2.9.1/contrib/capacity-scheduler/*.jar
2019-07-26 09:49:09,473 INFO [main] util.ServerCommandLine:
env:HBASE_MANAGES_ZK=false
2019-07-26 09:49:09,473 INFO [main] util.ServerCommandLine:
env:SSH_CONNECTION=192.168.174.37 42241 192.168.129.243 22
2019-07-26 09:49:09,473 INFO [main] util.ServerCommandLine:
env:HBASE_AUTOSTART_FILE=/tmp/hbase-hadoop-regionserver.autostart
2019-07-26 09:49:09,473 INFO [main] util.ServerCommandLine:
env:HOSTNAME=slave03
2019-07-26 09:49:09,473 INFO [main] util.ServerCommandLine:
env:HADOOP_HOME=/usr/hadoop/hadoop-2.9.1
2019-07-26 09:49:09,473 INFO [main] util.ServerCommandLine:
env:HBASE_NICENESS=0
2019-07-26 09:49:09,474 INFO [main] util.ServerCommandLine: env:HBASE_OPTS=
-XX:+UseConcMarkSweepGC -Dhbase.log.dir=/usr/hadoop/hbase-2.1.2/logs
-Dhbase.log.file=hbase-hadoop-regionserver-slave03.log
-Dhbase.home.dir=/usr/hadoop/hbase-2.1.2 -Dhbase.id.str=hadoop
-Dhbase.root.logger=INFO,RFA
-Djava.library.path=/usr/hadoop/hadoop-2.9.1/lib/native
-Dhbase.security.logger=INFO,RFAS
2019-07-26 09:49:09,474 INFO [main] util.ServerCommandLine:
env:HBASE_SECURITY_LOGGER=INFO,RFAS
2019-07-26 09:49:09,474 INFO [main] util.ServerCommandLine:
env:XDG_RUNTIME_DIR=/run/user/1000
2019-07-26 09:49:09,474 INFO [main] util.ServerCommandLine:
env:HBASE_THRIFT_OPTS=
2019-07-26 09:49:09,474 INFO [main] util.ServerCommandLine:
env:HBASE_HOME=/usr/hadoop/hbase-2.1.2
2019-07-26 09:49:09,474 INFO [main] util.ServerCommandLine:
1:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.axv=01;35:*.anx=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=01;36:*.au=01;36:*.flac=01;36:*.mid=01;36:*.midi=01;36:*.mka=01;36:*.mp3=01;36:*.mpc=01;36:*.ogg=01;36:*.ra=01;36:*.wav=01;36:*.axa=01;36:*.oga=01;36:*.spx=01;36:*.xspf=01;36:
2019-07-26 09:49:09,474 INFO [main] util.ServerCommandLine: env:SHLVL=3
2019-07-26 09:49:09,474 INFO [main] util.ServerCommandLine:
env:HOME=/home/hadoop
2019-07-26 09:49:09,474 INFO [main] util.ServerCommandLine:
env:ZOOKEEPER_HOME=/usr/hadoop/zookeeper-3.4.6
2019-07-26 09:49:09,474 INFO [main] util.ServerCommandLine:
env:MALLOC_ARENA_MAX=4
2019-07-26 09:49:09,475 INFO [main] util.ServerCommandLine: vmName=Java
HotSpot(TM) 64-Bit Server VM, vmVendor=Oracle Corporation, vmVersion=25.191-b12
2019-07-26 09:49:09,475 INFO [main] util.ServerCommandLine:
vmInputArguments=[-Dproc_regionserver, -XX:OnOutOfMemoryError=kill -9 %p,
-Xmx2048m, -XX:+UseConcMarkSweepGC,
-Dhbase.log.dir=/usr/hadoop/hbase-2.1.2/logs,
-Dhbase.log.file=hbase-hadoop-regionserver-slave03.log,
-Dhbase.home.dir=/usr/hadoop/hbase-2.1.2, -Dhbase.id.str=hadoop,
-Dhbase.root.logger=INFO,RFA,
-Djava.library.path=/usr/hadoop/hadoop-2.9.1/lib/native,
-Dhbase.security.logger=INFO,RFAS]
2019-07-26 09:49:09,939 INFO [main] metrics.MetricRegistries: Loaded
MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2019-07-26 09:49:10,194 INFO [main] regionserver.RSRpcServices:
regionserver/slave03:16020 server-side Connection retries=45
2019-07-26 09:49:10,205 INFO [main] ipc.RpcExecutor: Instantiated
default.FPBQ.Fifo with queueClass=class
java.util.concurrent.LinkedBlockingQueue; numCallQueues=3, maxQueueLength=300,
handlerCount=30
2019-07-26 09:49:10,207 INFO [main] ipc.RpcExecutor: Instantiated
priority.FPBQ.Fifo with queueClass=class
java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=300,
handlerCount=20
2019-07-26 09:49:10,207 INFO [main] ipc.RpcExecutor: Instantiated
replication.FPBQ.Fifo with queueClass=class
java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=300,
handlerCount=3
2019-07-26 09:49:10,316 INFO [main] ipc.RpcServerFactory: Creating
org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService,
hbase.pb.AdminService
2019-07-26 09:49:10,467 INFO [main] io.ByteBufferPool: Created with
bufferSize=64 KB and maxPoolSize=1.88 KB
2019-07-26 09:49:10,564 INFO [main] ipc.NettyRpcServer: Bind to
/192.168.129.243:16020
2019-07-26 09:49:10,651 INFO [main] hfile.CacheConfig: Allocating onheap
LruBlockCache size=805.90 MB, blockSize=64 KB
2019-07-26 09:49:10,658 INFO [main] hfile.CacheConfig: Created cacheConfig:
blockCache=LruBlockCache{blockCount=0, currentSize=605.43 KB, freeSize=805.31
MB, maxSize=805.90 MB, heapSize=605.43 KB, minSize=765.61 MB, minFactor=0.95,
multiSize=382.80 MB, multiFactor=0.5, singleSize=191.40 MB, singleFactor=0.25},
cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
prefetchOnOpen=false
2019-07-26 09:49:10,659 INFO [main] hfile.CacheConfig: Created cacheConfig:
blockCache=LruBlockCache{blockCount=0, currentSize=605.43 KB, freeSize=805.31
MB, maxSize=805.90 MB, heapSize=605.43 KB, minSize=765.61 MB, minFactor=0.95,
multiSize=382.80 MB, multiFactor=0.5, singleSize=191.40 MB, singleFactor=0.25},
cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false,
cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false,
prefetchOnOpen=false
2019-07-26 09:49:11,707 INFO [main] fs.HFileSystem: Added intercepting call to
namenode#getBlockLocations so can do block reordering using class
org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks
2019-07-26 09:49:11,719 INFO [main] fs.HFileSystem: Added intercepting call to
namenode#getBlockLocations so can do block reordering using class
org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks
2019-07-26 09:49:11,963 INFO [main] zookeeper.RecoverableZooKeeper: Process
identifier=regionserver:16020 connecting to ZooKeeper
ensemble=slave01:2181,slave02:2181,slave03:2181
2019-07-26 09:49:12,005 INFO [main] zookeeper.ZooKeeper: Client
environment:zookeeper.version=3.4.10-39d3a4f269333c922ed3db283be479f9deacaa0f,
built on 03/23/2017 10:13 GMT
2019-07-26 09:49:12,005 INFO [main] zookeeper.ZooKeeper: Client
environment:host.name=slave03
2019-07-26 09:49:12,005 INFO [main] zookeeper.ZooKeeper: Client
environment:java.version=1.8.0_191
2019-07-26 09:49:12,005 INFO [main] zookeeper.ZooKeeper: Client
environment:java.vendor=Oracle Corporation
2019-07-26 09:49:12,005 INFO [main] zookeeper.ZooKeeper: Client
environment:java.home=/usr/java/jdk1.8.0_191/jre
2019-07-26 09:49:12,005 INFO [main] zookeeper.ZooKeeper:
op-2.9.1/share/hadoop/mapreduce/lib/guice-servlet-3.0.jar:/usr/hadoop/hadoop-2.9.1/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.9.1.jar:/usr/hadoop/hadoop-2.9.1/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-2.9.1.jar:/usr/hadoop/hadoop-2.9.1/share/hadoop/mapreduce/hadoop-mapreduce-client-app-2.9.1.jar:/usr/hadoop/hadoop-2.9.1/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.9.1-tests.jar:/usr/hadoop/hadoop-2.9.1/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-2.9.1.jar:/usr/hadoop/hadoop-2.9.1/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-2.9.1.jar:/usr/hadoop/hadoop-2.9.1/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.9.1.jar:/usr/hadoop/hadoop-2.9.1/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.9.1.jar:/usr/hadoop/hadoop-2.9.1/share/hadoop/mapreduce/hadoop-mapreduce-client-common-2.9.1.jar:/usr/hadoop/hadoop-2.9.1/share/hadoop/mapreduce/hbase-shaded-mapreduce-2.1.2.jar:/usr/hadoop/hadoop-2.9.1/contrib/capacity-scheduler/*.jar
2019-07-26 09:49:12,006 INFO [main] zookeeper.ZooKeeper: Client
environment:java.library.path=/usr/hadoop/hadoop-2.9.1/lib/native
2019-07-26 09:49:12,006 INFO [main] zookeeper.ZooKeeper: Client
environment:java.io.tmpdir=/tmp
2019-07-26 09:49:12,006 INFO [main] zookeeper.ZooKeeper: Client
environment:java.compiler=<NA>
2019-07-26 09:49:12,006 INFO [main] zookeeper.ZooKeeper: Client
environment:os.name=Linux
2019-07-26 09:49:12,006 INFO [main] zookeeper.ZooKeeper: Client
environment:os.arch=amd64
2019-07-26 09:49:12,006 INFO [main] zookeeper.ZooKeeper: Client
environment:os.version=3.10.0-693.el7.x86_64
2019-07-26 09:49:12,006 INFO [main] zookeeper.ZooKeeper: Client
environment:user.name=hadoop
2019-07-26 09:49:12,006 INFO [main] zookeeper.ZooKeeper: Client
environment:user.home=/home/hadoop
2019-07-26 09:49:12,006 INFO [main] zookeeper.ZooKeeper: Client
environment:user.dir=/home/hadoop
2019-07-26 09:49:12,008 INFO [main] zookeeper.ZooKeeper: Initiating client
connection, connectString=slave01:2181,slave02:2181,slave03:2181
sessionTimeout=90000
watcher=org.apache.hadoop.hbase.zookeeper.PendingWatcher@4e92466a
2019-07-26 09:49:12,038 INFO [main-SendThread(slave02:2181)]
zookeeper.ClientCnxn: Opening socket connection to server
slave02/192.168.129.245:2181. Will not attempt to authenticate using SASL
(unknown error)
2019-07-26 09:49:12,043 INFO [main-SendThread(slave02:2181)]
zookeeper.ClientCnxn: Socket connection established to
slave02/192.168.129.245:2181, initiating session
2019-07-26 09:49:12,121 INFO [main-SendThread(slave02:2181)]
zookeeper.ClientCnxn: Session establishment complete on server
slave02/192.168.129.245:2181, sessionid = 0x26c2bc912ae0004, negotiated timeout
= 40000
2019-07-26 09:49:12,274 INFO [main] util.log: Logging initialized @3707ms
2019-07-26 09:49:12,369 INFO [main] http.HttpRequestLog: Http request log for
http.requests.regionserver is not defined
2019-07-26 09:49:12,383 INFO [main] http.HttpServer: Added global filter
'safety' (class=org.apache.hadoop.hbase.http.HttpServer$QuotingInputFilter)
2019-07-26 09:49:12,383 INFO [main] http.HttpServer: Added global filter
'clickjackingprevention'
(class=org.apache.hadoop.hbase.http.ClickjackingPreventionFilter)
2019-07-26 09:49:12,385 INFO [main] http.HttpServer: Added filter
static_user_filter
(class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter)
to context regionserver
2019-07-26 09:49:12,385 INFO [main] http.HttpServer: Added filter
static_user_filter
(class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter)
to context logs
2019-07-26 09:49:12,385 INFO [main] http.HttpServer: Added filter
static_user_filter
(class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter)
to context static
2019-07-26 09:49:12,413 INFO [main] http.HttpServer: Jetty bound to port 16030
2019-07-26 09:49:12,414 INFO [main] server.Server: jetty-9.3.19.v20170502
2019-07-26 09:49:12,465 INFO [main] handler.ContextHandler: Started
o.e.j.s.ServletContextHandler@299270eb{/logs,file:///usr/hadoop/hbase-2.1.2/logs/,AVAILABLE}
2019-07-26 09:49:12,465 INFO [main] handler.ContextHandler: Started
o.e.j.s.ServletContextHandler@69fa8e76{/static,file:///usr/hadoop/hbase-2.1.2/hbase-webapps/static/,AVAILABLE}
2019-07-26 09:49:12,641 INFO [main] handler.ContextHandler: Started
o.e.j.w.WebAppContext@35e8316e{/,file:///usr/hadoop/hbase-2.1.2/hbase-webapps/regionserver/,AVAILABLE}{file:/usr/hadoop/hbase-2.1.2/hbase-webapps/regionserver}
2019-07-26 09:49:12,646 INFO [main] server.AbstractConnector: Started
ServerConnector@6349035c{HTTP/1.1,[http/1.1]}{0.0.0.0:16030}
2019-07-26 09:49:12,646 INFO [main] server.Server: Started @4079ms
2019-07-26 09:49:12,697 INFO [regionserver/slave03:16020]
regionserver.HRegionServer: ClusterId : c8baea8c-8e21-44f1-a98d-02629e68cbaf
2019-07-26 09:49:12,951 INFO
[ReadOnlyZKClient-slave01:2181,slave02:2181,slave03:2181@0x55686585]
zookeeper.ZooKeeper: Initiating client connection,
connectString=slave01:2181,slave02:2181,slave03:2181 sessionTimeout=90000
watcher=org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$36/1312072885@b363317
2019-07-26 09:49:12,955 INFO
[ReadOnlyZKClient-slave01:2181,slave02:2181,slave03:2181@0x55686585-SendThread(slave01:2181)]
zookeeper.ClientCnxn: Opening socket connection to server
slave01/192.168.129.244:2181. Will not attempt to authenticate using SASL
(unknown error)
2019-07-26 09:49:12,956 INFO
[ReadOnlyZKClient-slave01:2181,slave02:2181,slave03:2181@0x55686585-SendThread(slave01:2181)]
zookeeper.ClientCnxn: Socket connection established to
slave01/192.168.129.244:2181, initiating session
2019-07-26 09:49:12,972 INFO
[ReadOnlyZKClient-slave01:2181,slave02:2181,slave03:2181@0x55686585-SendThread(slave01:2181)]
zookeeper.ClientCnxn: Session establishment complete on server
slave01/192.168.129.244:2181, sessionid = 0x16c2bc8d3660007, negotiated timeout
= 40000
2019-07-26 09:49:13,070 INFO [regionserver/slave03:16020]
regionserver.RegionServerCoprocessorHost: System coprocessor loading is enabled
2019-07-26 09:49:13,070 INFO [regionserver/slave03:16020]
regionserver.RegionServerCoprocessorHost: Table coprocessor loading is enabled
2019-07-26 09:49:13,083 INFO [regionserver/slave03:16020]
regionserver.HRegionServer: reportForDuty to master=master,16000,1564105120175
with port=16020, startcode=1564105749497
2019-07-26 09:49:13,367 INFO [regionserver/slave03:16020] wal.WALFactory:
Instantiating WALProvider of type class
org.apache.hadoop.hbase.wal.FSHLogProvider
2019-07-26 09:49:13,707 INFO [regionserver/slave03:16020]
regionserver.MetricsRegionServerWrapperImpl: Computing regionserver metrics
every 5000 milliseconds
2019-07-26 09:49:13,753 INFO [regionserver/slave03:16020]
regionserver.MemStoreFlusher: globalMemStoreLimit=805.9 M,
globalMemStoreLimitLowMark=765.6 M, Offheap=false
2019-07-26 09:49:13,762 INFO [regionserver/slave03:16020]
throttle.PressureAwareCompactionThroughputController: Compaction throughput
configurations, higher bound: 20.00 MB/second, lower bound 10.00 MB/second, off
peak: unlimited, tuning period: 60000 ms
2019-07-26 09:49:13,799 INFO [regionserver/slave03:16020]
regionserver.HRegionServer: CompactionChecker runs every PT10S
2019-07-26 09:49:13,876 INFO [regionserver/slave03:16020]
regionserver.HeapMemoryManager: Starting, tuneOn=false
2019-07-26 09:49:13,876 INFO [SplitLogWorker-slave03:16020]
regionserver.SplitLogWorker: SplitLogWorker slave03,16020,1564105749497 starting
2019-07-26 09:49:13,883 INFO [regionserver/slave03:16020]
regionserver.ChunkCreator: Allocating data MemStoreChunkPool with chunk size 2
MB, max count 362, initial count 0
2019-07-26 09:49:13,885 INFO [regionserver/slave03:16020]
regionserver.ChunkCreator: Allocating index MemStoreChunkPool with chunk size
204.80 KB, max count 402, initial count 0
2019-07-26 09:49:13,934 INFO [regionserver/slave03:16020]
regionserver.HRegionServer: Serving as slave03,16020,1564105749497, RpcServer
on slave03/192.168.129.243:16020, sessionid=0x26c2bc912ae0004
2019-07-26 09:49:13,939 INFO [regionserver/slave03:16020]
quotas.RegionServerRpcQuotaManager: Quota support disabled
2019-07-26 09:49:13,939 INFO [regionserver/slave03:16020]
quotas.RegionServerSpaceQuotaManager: Quota support disabled, not starting
space quota manager.
2019-07-26 09:49:17,309 INFO [regionserver/slave03:16020] wal.AbstractFSWAL:
WAL configuration: blocksize=256 MB, rollsize=128 MB,
prefix=slave03%2C16020%2C1564105749497, suffix=,
logDir=hdfs://sicluster/hbase/WALs/slave03,16020,1564105749497,
archiveDir=hdfs://sicluster/hbase/oldWALs
2019-07-26 09:49:17,668 INFO [regionserver/slave03:16020] wal.AbstractFSWAL:
Slow sync cost: 211 ms, current pipeline: []
2019-07-26 09:49:17,668 INFO [regionserver/slave03:16020] wal.AbstractFSWAL:
New WAL
/hbase/WALs/slave03,16020,1564105749497/slave03%2C16020%2C1564105749497.1564105757309
----- 原始邮件 -----
发件人：OpenInx <[email protected]>
收件人：Hbase-User <[email protected]>, [email protected]
主题：Re: Add node(regionserver) exception
日期：2019年07月25日 12点34分
You mean you can start the RS by hbase-daemons.sh, but failed by
start-hbase.sh ? That makes no sense, because the start-hbase.sh is just
executing the hbase-daemons.sh scripts.
Could you provide more information about the failure ?
On Thu, Jul 25, 2019 at 10:26 AM <[email protected]> wrote:
> Hello User.
>
> (HBase version: 2.1.2；Hadoop version:2.9.1)。
> I added a new hbase regionserver node, which failed to launch the
> regionserver service on it when I started the hbase cluster using
> start-hbase.sh.However, when I used the hbase-daemone.sh start regionserver
> on the new node, it started correctly and the hbase cluster state was
> normal. I could also see the new node on the UI(master:60010).When
> stop-hbase.sh, the services of all nodes can be shut down normally. Here
> is the printed information in the log(hbase-hadoop-master-slave03.log):Thu
> Jul 25 09:19:17 CST 2019 Starting master on slave03
> core file size (blocks, -c) 0
> data seg size (kbytes, -d) unlimited
> scheduling priority (-e) 0
> file size (blocks, -f) unlimited
> pending signals (-i) 30733
> max locked memory (kbytes, -l) 64
> max memory size (kbytes, -m) unlimited
> open files (-n) 1024
> pipe size (512 bytes, -p) 8
> POSIX message queues (bytes, -q) 819200
> real-time priority (-r) 0
> stack size (kbytes, -s) 8192
> cpu time (seconds, -t) unlimited
> max user processes (-u) 4096
> virtual memory (kbytes, -v) unlimited
> file locks (-x) unlimited
> 2019-07-25 09:19:19,389 WARN [main] util.NativeCodeLoader: Unable to load
> native-hadoop library for your platform... using builtin-java classes where
> applicablePlease help to solve this problem。Best regards, Li Wei
