[
https://issues.apache.org/jira/browse/PHOENIX-1452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14223594#comment-14223594
]
Andrew Purtell commented on PHOENIX-1452:
-----------------------------------------
The description and first comment on this issue have a much wider scope than
the issue header. It might be better to change this to an umbrella and propose
pieces of the work as subtasks.
Typically, HBase time series metrics (and I'd imagine this will be the same for
Phoenix, because Phoenix is hosted within HBase) are exported through the
metrics system for collection by operational systems like otsdb or ganglia. Ops
teams will resort to log scraping but that's not preferred. Things like # of
spool file, memory management usage, histograms of wait times would fall into
this category, the metrics subsystem has good support for them.
For per-query logging, a single log line per query as proposed that provides
statistics and related information about the physical plan makes sense too.
> Add logging to allow easier monitoring of client-side Thread Pool and Queue
> usage
> ----------------------------------------------------------------------------------
>
> Key: PHOENIX-1452
> URL: https://issues.apache.org/jira/browse/PHOENIX-1452
> Project: Phoenix
> Issue Type: Improvement
> Affects Versions: 5.0.0, 4.2
> Reporter: Jan Fernando
>
> For performance testing and tuning of features that use Phoenix and for
> production monitoring it would be really helpful to easily be able to extract
> statistics about Phoenix's client-side Thread Pool and Queue Depth usage to
> help with tuning and being able to correlate the impact of tuning these 2
> parameters to query performance.
> For global per JVM logging one of the following would meet my needs, with a
> preference for #2:
> 1. A simple log line that that logs the data in ThreadPoolExecutor.toString()
> at a configurable interval
> 2. Exposing the ThreadPoolExecutor metrics in PhoenixRuntime or other global
> client exposed class and allow client to do their own logging.
> In addition to this it would also be really valuable to have a single log
> line per query that provides statistics about the level of parallelism i.e.
> number of parallel scans being executed. I don't full explain plan level of
> data but a good heuristic to be able to track over time how queries are
> utilizing the thread pool as data size grows etc.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
