We were trying to enable Phoenix query tracing and ran into problem. Traces are not being written to SYSTEM.TRACING_STATS table.
We're using HBase 1.3 and Phoenix 4.13.1. We were trying to do this according to this official manual: https://phoenix.apache.org/tracing.html . We've set up hadoop-metrics2-phoenix.properties and hadoop-metrics2-hbase.properties as is from the manual. These files are located at $PHOENIX_HOME/bin/config/hadoop-metrics2-phoenix.properties and $HBASE_HOME/conf/hadoop-metrics2-hbase.properties . hbase-site.xml at $PHOENIX_HOME/bin contains phoenix.trace.frequency property which is set to "always". Despite that when we perform some queries from sqlline.py, traces are not being written to SYSTEM.TRACING_STATS. This table is not even created. During investigation of this problem we dived into Phoenix sources. What confuses us is that we can't find org.apache.phoenix.trace.PhoenixTableMetricsWriter class in sources of Phoenix 4.13.1. Is that correct name of metrics writer class? And what confuses us even more is that turned out that we're specifying unexistent class to HBase, but despite that HBase Master starts and there are no errors in logs. I would expect NoClassDefFoundError or ClassNotFoundException or something like that. Looks like HBase is not even trying to load or instantiate this class. In 4.13.1 org.apache.phoenix.trace.TableWriter class looks like metrics writer class, but after specifying it in phoenix.sink.tracing.writer-class and hbase.sink.tracing.writer-class properties in hadoop-metrics2-phoenix.properties and hadoop-metrics2-hbase.properties there is still no luck: SYSTEM.TRACING_STATS table is not created after we perform some queries in sqlline.py . We've also tried to trace queries performed by our Spark application that uses spark phoenix plugin, but ran into same problem: SYSTEM.TRACING_STATS table is not even initialized. phoenix.trace.frequency = always is specified in HBase config object in our Spark application. Also we've tried to trace queries performed by IntelliJ IDEA via JDBC connection, phoenix.trace.frequency property was set as custom JDBC connection property. Can you please advice what we're doing wrong? Thanks, Stepan.
