[ https://issues.apache.org/jira/browse/PHOENIX-4701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16452569#comment-16452569 ]
James Taylor edited comment on PHOENIX-4701 at 4/25/18 4:13 PM: ---------------------------------------------------------------- Thanks for the response, [~an...@apache.org] and [~samarthjain]. I think the best approach would be to persist our [client metrics|http://phoenix.apache.org/metrics.html] in the SYSTEM.LOG instead of inventing a new mechanism. The metrics captures all the same information as your QueryLogInfo (and much more), rolls all the information up to a single set of metrics for each Phoenix statement (aggregating/merging parallel scans, etc), and can emits a single log line (which could be written in a single upsert statement). At SFDC, we emit this information in a layer above (and use Splunk to produce nifty dashboard for monitoring), but this could easily be emitted directly in Phoenix and go through your asynchronous write path (and then use Phoenix queries to produce the same kind of dashboards). The only piece would be to add the concept of a log level to each metric to enable statically controlling which metrics are output. was (Author: jamestaylor): Thanks for the response, [~an...@apache.org] and [~samarthjain]. I think the best approach would be to persist our [client metrics|http://phoenix.apache.org/metrics.html] in the SYSTEM.LOG instead of inventing a new mechanism. The metrics captures all the same information as your QueryLogInfo (and much more), rolls all the information up to a single set of metrics for each Phoenix statement (aggregating/merging parallel scans, etc), and can emits a single log line (which could be written in a single upsert statement). At SFDC, we emit this information in a layer above, but this could easily be emitted directly in Phoenix and go through your asynchronous write path. The only piece would be to add the concept of a log level to each metric to enable statically controlling which metrics are output. > Improve schema of SYSTEM.LOG table > ---------------------------------- > > Key: PHOENIX-4701 > URL: https://issues.apache.org/jira/browse/PHOENIX-4701 > Project: Phoenix > Issue Type: Bug > Reporter: James Taylor > Assignee: James Taylor > Priority: Major > Fix For: 4.14.0, 5.0.0 > > Attachments: PHOENIX-4701_wip1.patch, PHOENIX-4701_wip2.patch > > > If possible, the SYSTEM.LOG table would benefit greatly (3-5x perf gain) > from being declared as immutable with a column encoding of 1 byte and a > storage format of SINGLE_CELL_ARRAY_WITH_OFFSETS. -- This message was sent by Atlassian JIRA (v7.6.3#76005)