[
https://issues.apache.org/jira/browse/IMPALA-9865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17235864#comment-17235864
]
guojingfeng edited comment on IMPALA-9865 at 11/20/20, 3:19 AM:
----------------------------------------------------------------
Hi, [~tarmstrong] denser format of runtime profile may be a big work, and
Cloudera Manager should do some capability work to deal with the new format.
Is it possible to abandon some detailed counters of runtime-profile to reduce
memory consumption in executor serialize phase. aka: in-memory presentation
level like glog level 、log4j levels.
In some critical sql cases (slow queries) we set a high value of presentation
levels to do debug analysis as well as most other sql cases we set a low value.
was (Author: guojingfeng):
Hi, [~tarmstrong] denser format of runtime profile may be a big work, and
Cloudera Manager should do some capability work to deal with the new format.
Is it possible to abandon some detailed counters of runtime-profile to reduce
memory consumption in executor serialize phase. aka: in-memory presentation
level like log level for log4j levels.
In some critical sql cases (slow queries) we set a high value of presentation
levels to do debug analysis as well as most other sql cases we set a low value.
> Utility to pretty-print thrift profiles at various levels
> ---------------------------------------------------------
>
> Key: IMPALA-9865
> URL: https://issues.apache.org/jira/browse/IMPALA-9865
> Project: IMPALA
> Issue Type: Sub-task
> Components: Backend
> Reporter: Tim Armstrong
> Assignee: Tim Armstrong
> Priority: Major
> Attachments: image-2020-11-20-09-25-08-082.png
>
>
> The prototyping work in IMPALA-9382 revealed some hard trade-offs between
> having a full-fidelity text profile and readability.
> We want to have a text profile with less information by default so that it is
> more readable, and rely on the thrift profile for more detailed debugging.
> This would be easier if we provided a utility that can pretty-print a thrift
> profile at different levels of detail.
> This JIRA is to reduce the default level of pretty-printing for aggregated
> profiles, but provide a utility that can dump both the basic and full
> versions. My thought is to start off with the same 4 levels as explain, but
> maybe only implement 2 levels to start off with - basic and extended.
> The utility should be able to handle the same cases as
> bin/parse-thrift-profile.py (profile log and single profile in a file) and
> maybe print only a specified query from a profile log. We can use the
> DeserializeFromArchiveString() method that was removed in IMPALA-9381, then
> pretty-print the deserialised profile.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
