Re: Enabling Tracing makes HMaster service fail to start

2017-11-08 Thread Josh Elser 
Please note that there is a difference between Phoenix Tracing and the 
TRACE log4j level.


It appears that you're using a version of Phoenix which is incompatible 
with the version of HBase/Hadoop that you're running. The implementation 
of PhoenixMetricsSink is incompatible with the interface/abstract-class 
that HBase/Hadoop is expecting.


This may be a classpath or Phoenix version issue, or you may have 
stumbled onto a bug.


On 11/8/17 6:33 AM, Mallieswari Dineshbabu wrote:

Hi All,

I am working with HBase-Phoenix, /everything works fine/. In addition 
trying to enable Tracing  in 
Phoenix with the following steps,


 1. Copy ‘hadoop-metrics2-hbase.PROPERTIES’ from Phoenix package to
HBase conf folder.
 2. ‘hadoop-metrics2-phoenix.PROPERTIES’ file will be in ‘Phoenix/bin’
location by default. So I left it as it is.
 3. Add the following property to phoenix configuration,



    phoenix.trace.frequency

   always



After doing the above, HBase’s HMaster fails to start with the following 
exception; Please tell if you have any suggestion on this,


2017-11-08 16:46:56,118 INFO  [main] regionserver.RSRpcServices: 
master/Selfuser-VirtualBox/172.16.203.117:6 
 server-side HConnection retries=140


2017-11-08 16:46:56,520 INFO  [main] ipc.SimpleRpcScheduler: Using 
deadline as user call queue, count=3


2017-11-08 16:46:56,554 INFO  [main] ipc.RpcServer: 
master/Selfuser-VirtualBox/192.16.203.117:6 
: started 10 reader(s) listening on port=6


*2017-11-08 16:46:56,839 INFO  [main] impl.MetricsConfig: loaded 
properties from hadoop-metrics2-hbase.properties*


*2017-11-08 16:46:56,926 INFO  [main] trace.PhoenixMetricsSink: Writing 
tracing metrics to phoenix table*


*2017-11-08 16:46:56,933 ERROR [main] master.HMasterCommandLine: Master 
exiting*


*java.lang.RuntimeException: Failed construction of Master: class 
org.apache.hadoop.hbase.master.HMaster. *


*at 
org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:2512)*


at 
org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:231)


at 
org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:137)


at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)

at 
org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:126)


at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:2522)

Caused by: java.lang.AbstractMethodError: 
org.apache.phoenix.trace.PhoenixMetricsSink.init(Lorg/apache/commons/configuration/SubsetConfiguration;)V


at 
org.apache.hadoop.metrics2.impl.MetricsConfig.getPlugin(MetricsConfig.java:199)


at 
org.apache.hadoop.metrics2.impl.MetricsSystemImpl.newSink(MetricsSystemImpl.java:530)


at 
org.apache.hadoop.metrics2.impl.MetricsSdrddystemImpl.configureSinks(MetricsSystemImpl.java:502)


at 
org.apache.hadoop.metrics2.impl.MetricsSystemImpl.configure(MetricsSystemImpl.java:481)


at 
org.apache.hadoop.metrics2.impl.MetricsSystemImpl.start(MetricsSystemImpl.java:189)


at 
org.apache.hadoop.metrics2.impl.MetricsSystemImpl.init(MetricsSystemImpl.java:164)


at 
org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.init(DefaultMetricsSystem.java:54)


at 
org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.initialize(DefaultMetricsSystem.java:50)


at 
org.apache.hadoop.hbase.metrics.BaseSourceImpl$DefaultMetricsSystemInitializer.init(BaseSourceImpl.java:49)


at 
org.apache.hadoop.hbase.metrics.BaseSourceImpl.(BaseSourceImpl.java:72)


at 
org.apache.hadoop.hbase.ipc.MetricsHBaseServerSourceImpl.(MetricsHBaseServerSourceImpl.java:66)


at 
org.apache.hadoop.hbase.ipc.MetricsHBaseServerSourceFactoryImpl.getSource(MetricsHBaseServerSourceFactoryImpl.java:48)


at 
org.apache.hadoop.hbase.ipc.MetricsHBaseServerSourceFactoryImpl.create(MetricsHBaseServerSourceFactoryImpl.java:38)


at 
org.apache.hadoop.hbase.ipc.MetricsHBaseServer.(MetricsHBaseServer.java:40)


at org.apache.hadoop.hbase.ipc.RpcServer.(RpcServer.java:2040)

at 
org.apache.hadoop.hbase.regionserver.RSRpcServices.(RSRpcServices.java:930)


at 
org.apache.hadoop.hbase.master.MasterRpcServices.(MasterRpcServices.java:231)


at 
org.apache.hadoop.hbase.master.HMaster.createRpcServices(HMaster.java:572)


at 
org.apache.hadoop.hbase.regionserver.HRegionServer.(HRegionServer.java:531)


at org.apache.hadoop.hbase.master.HMaster.(HMaster.java:412)

at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)

at 
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)


at 
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)


at java.lang.reflect.Constructor.newInstance(Constructor.java:526)

at org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:2505)

... 5 more

Note: Additionally based on this Hortonworks reference

Enabling Tracing makes HMaster service fail to start

2017-11-08 Thread Mallieswari Dineshbabu 
Hi All,



I am working with HBase-Phoenix, *everything works fine*. In addition
trying to enable Tracing  in
Phoenix with the following steps,

   1. Copy ‘hadoop-metrics2-hbase.PROPERTIES’ from Phoenix package to HBase
   conf folder.
   2. ‘hadoop-metrics2-phoenix.PROPERTIES’ file will be in ‘Phoenix/bin’
   location by default. So I left it as it is.
   3. Add the following property to phoenix configuration,

  

  phoenix.trace.frequency

  always





After doing the above, HBase’s HMaster fails to start with the following
exception; Please tell if you have any suggestion on this,



2017-11-08 16:46:56,118 INFO  [main] regionserver.RSRpcServices:
master/Selfuser-VirtualBox/172.16.203.117:6 server-side HConnection
retries=140

2017-11-08 16:46:56,520 INFO  [main] ipc.SimpleRpcScheduler: Using deadline
as user call queue, count=3

2017-11-08 16:46:56,554 INFO  [main] ipc.RpcServer:
master/Selfuser-VirtualBox/192.16.203.117:6: started 10 reader(s)
listening on port=6

*2017-11-08 16:46:56,839 INFO  [main] impl.MetricsConfig: loaded properties
from hadoop-metrics2-hbase.properties*

*2017-11-08 16:46:56,926 INFO  [main] trace.PhoenixMetricsSink: Writing
tracing metrics to phoenix table*

*2017-11-08 16:46:56,933 ERROR [main] master.HMasterCommandLine: Master
exiting*

*java.lang.RuntimeException: Failed construction of Master: class
org.apache.hadoop.hbase.master.HMaster. *

*   at
org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:2512)*

   at
org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:231)

   at
org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:137)

   at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)

   at
org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:126)

   at
org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:2522)

Caused by: java.lang.AbstractMethodError:
org.apache.phoenix.trace.PhoenixMetricsSink.init(Lorg/apache/commons/configuration/SubsetConfiguration;)V

   at
org.apache.hadoop.metrics2.impl.MetricsConfig.getPlugin(MetricsConfig.java:199)

   at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl.newSink(MetricsSystemImpl.java:530)

   at
org.apache.hadoop.metrics2.impl.MetricsSdrddystemImpl.configureSinks(MetricsSystemImpl.java:502)

   at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl.configure(MetricsSystemImpl.java:481)

   at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl.start(MetricsSystemImpl.java:189)

   at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl.init(MetricsSystemImpl.java:164)

   at
org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.init(DefaultMetricsSystem.java:54)

   at
org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.initialize(DefaultMetricsSystem.java:50)

   at
org.apache.hadoop.hbase.metrics.BaseSourceImpl$DefaultMetricsSystemInitializer.init(BaseSourceImpl.java:49)

   at
org.apache.hadoop.hbase.metrics.BaseSourceImpl.(BaseSourceImpl.java:72)

   at
org.apache.hadoop.hbase.ipc.MetricsHBaseServerSourceImpl.(MetricsHBaseServerSourceImpl.java:66)

   at
org.apache.hadoop.hbase.ipc.MetricsHBaseServerSourceFactoryImpl.getSource(MetricsHBaseServerSourceFactoryImpl.java:48)

   at
org.apache.hadoop.hbase.ipc.MetricsHBaseServerSourceFactoryImpl.create(MetricsHBaseServerSourceFactoryImpl.java:38)

   at
org.apache.hadoop.hbase.ipc.MetricsHBaseServer.(MetricsHBaseServer.java:40)

   at
org.apache.hadoop.hbase.ipc.RpcServer.(RpcServer.java:2040)

   at
org.apache.hadoop.hbase.regionserver.RSRpcServices.(RSRpcServices.java:930)

   at
org.apache.hadoop.hbase.master.MasterRpcServices.(MasterRpcServices.java:231)

   at
org.apache.hadoop.hbase.master.HMaster.createRpcServices(HMaster.java:572)

   at
org.apache.hadoop.hbase.regionserver.HRegionServer.(HRegionServer.java:531)

   at
org.apache.hadoop.hbase.master.HMaster.(HMaster.java:412)

   at
sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)

   at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)

   at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)

   at
java.lang.reflect.Constructor.newInstance(Constructor.java:526)

   at
org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:2505)

   ... 5 more





Note: Additionally based on this Hortonworks reference
,
did the following changes in log4j.PROPERTIES