[
https://issues.apache.org/jira/browse/PHOENIX-4701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16475115#comment-16475115
]
Hudson commented on PHOENIX-4701:
---------------------------------
ABORTED: Integrated in Jenkins build Phoenix-4.x-HBase-1.3 #132 (See
[https://builds.apache.org/job/Phoenix-4.x-HBase-1.3/132/])
PHOENIX-4701 Write client-side metrics asynchronously to SYSTEM.LOG
(ankitsinghal59: rev f73f98c7a77fbe7e41930fc23b9fa3850fc0d260)
* (edit)
phoenix-core/src/main/java/org/apache/phoenix/monitoring/MutationMetricQueue.java
* (edit)
phoenix-core/src/main/java/org/apache/phoenix/schema/MetaDataClient.java
* (edit) bin/hbase-site.xml
* (edit) phoenix-core/src/main/java/org/apache/phoenix/log/TableLogWriter.java
* (edit) phoenix-core/src/main/java/org/apache/phoenix/query/QueryServices.java
* (edit)
phoenix-core/src/main/java/org/apache/phoenix/jdbc/PhoenixConnection.java
* (edit)
phoenix-core/src/main/java/org/apache/phoenix/jdbc/PhoenixResultSet.java
* (edit)
phoenix-core/src/main/java/org/apache/phoenix/compile/StatementContext.java
* (edit)
phoenix-core/src/main/java/org/apache/phoenix/iterate/SerialIterators.java
* (edit)
phoenix-core/src/main/java/org/apache/phoenix/query/ConnectionlessQueryServicesImpl.java
* (edit)
phoenix-core/src/main/java/org/apache/phoenix/iterate/ParallelIterators.java
* (edit) phoenix-core/src/main/java/org/apache/phoenix/log/QueryLoggerUtil.java
* (edit)
phoenix-core/src/main/java/org/apache/phoenix/monitoring/TaskExecutionMetricsHolder.java
* (edit)
phoenix-core/src/main/java/org/apache/phoenix/query/ConnectionQueryServicesImpl.java
* (edit) phoenix-core/src/main/java/org/apache/phoenix/log/QueryLogger.java
* (edit) phoenix-core/src/main/java/org/apache/phoenix/log/LogLevel.java
* (edit) phoenix-core/src/main/java/org/apache/phoenix/util/QueryUtil.java
* (add) phoenix-core/src/main/java/org/apache/phoenix/log/QueryStatus.java
* (edit)
phoenix-core/src/main/java/org/apache/phoenix/monitoring/SpoolingMetricsHolder.java
* (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/QueryLoggerIT.java
* (edit)
phoenix-core/src/main/java/org/apache/phoenix/monitoring/OverAllQueryMetrics.java
* (edit)
phoenix-core/src/test/java/org/apache/phoenix/iterate/SpoolingResultIteratorTest.java
* (edit)
phoenix-core/src/main/java/org/apache/phoenix/monitoring/MetricType.java
* (edit)
phoenix-core/src/main/java/org/apache/phoenix/monitoring/ReadMetricQueue.java
* (edit)
phoenix-core/src/main/java/org/apache/phoenix/iterate/ChunkedResultIterator.java
* (edit) phoenix-core/src/main/java/org/apache/phoenix/query/QueryConstants.java
* (edit)
phoenix-core/src/main/java/org/apache/phoenix/jdbc/PhoenixStatement.java
* (edit) phoenix-core/src/main/java/org/apache/phoenix/log/RingBufferEvent.java
* (edit)
phoenix-hive/src/main/java/org/apache/phoenix/hive/mapreduce/PhoenixRecordReader.java
* (edit)
phoenix-core/src/main/java/org/apache/phoenix/monitoring/MemoryMetricsHolder.java
* (delete) phoenix-core/src/main/java/org/apache/phoenix/log/QueryLogState.java
* (edit) phoenix-core/src/main/java/org/apache/phoenix/log/QueryLogInfo.java
* (edit)
phoenix-core/src/it/java/org/apache/phoenix/monitoring/PhoenixMetricsIT.java
* (edit)
phoenix-core/src/main/java/org/apache/phoenix/log/RingBufferEventTranslator.java
* (edit)
phoenix-core/src/main/java/org/apache/phoenix/mapreduce/PhoenixRecordReader.java
* (edit) phoenix-core/src/main/java/org/apache/phoenix/util/SchemaUtil.java
* (edit)
phoenix-core/src/main/java/org/apache/phoenix/execute/MutationState.java
* (edit) phoenix-core/src/main/java/org/apache/phoenix/log/LogWriter.java
* (edit)
phoenix-core/src/main/java/org/apache/phoenix/monitoring/ScanMetricsHolder.java
* (edit)
phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesOptions.java
* (add) phoenix-core/src/main/java/org/apache/phoenix/monitoring/MetricUtil.java
> Write client-side metrics asynchronously to SYSTEM.LOG
> ------------------------------------------------------
>
> Key: PHOENIX-4701
> URL: https://issues.apache.org/jira/browse/PHOENIX-4701
> Project: Phoenix
> Issue Type: Improvement
> Reporter: James Taylor
> Assignee: Ankit Singhal
> Priority: Major
> Fix For: 4.14.0, 5.0.0
>
> Attachments: PHOENIX-4701.patch, PHOENIX-4701_master.patch,
> PHOENIX-4701_v2.patch, PHOENIX-4701_wip1.patch, PHOENIX-4701_wip2.patch,
> PHOENIX-4701_wip3.patch
>
>
> Rather than inventing a new, different set of client-side metrics to persist,
> we should just persist our [client
> metrics|http://phoenix.apache.org/metrics.html] in the SYSTEM.LOG. The
> metrics captures all the same information as your QueryLogInfo (and much
> more), rolls all the information up to a single set of metrics for each
> Phoenix statement (aggregating/merging parallel scans, etc), and can emits a
> single log line (which could be written in a single upsert statement). At
> SFDC, we emit this information into a file system log in a layer above (and
> use Splunk to produce nifty dashboard for monitoring), but this could easily
> be emitted directly in Phoenix and go through your asynchronous write path
> (and then use Phoenix queries to produce the same kind of dashboards). The
> only piece would be to add the concept of a log level to each metric to
> enable statically controlling which metrics are output.
> With this approach, the SYSTEM.LOG table could be declared immutable and use
> our dense storage format with a single byte for column encoding and get a
> 3-5x perf gain. This would also open the door for users to potentially add
> secondary indexes on the table. See schema identified in the wip2 patch.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
