Hi Todd, I am making a simple fix for IMPALA-6568 <https://issues.apache.org/jira/browse/IMPALA-6568> add missing Query Compilation section to profiles. which would add the Timeline to all responses to createExecRequest.
It sounds like your change is more deep. If you go ahead with your change it sounds like my change might be redundant. Thanks -Andrew On Tue, Sep 4, 2018 at 4:02 PM Todd Lipcon <[email protected]> wrote: > Hey folks, > > I'm working on a patch to add some more diagnostics from the planning > process into query profiles. > > Currently, only the planning "Timeline" is reported back as part of the > response to createExecRequest. As part of the fetch-on-demand catalog work > I'd like to start exposing various counters such as cache hit/miss counts, > time spent on remote calls to the catalog, etc. Even in the existing code > paths, I can see some similar metrics being useful. > > My current thinking is to remove the 'timeline' (TEventSequence) field > from TExecRequest and replace it with a full TRuntimeProfileNode. I'd then > add some capability in the Java side to fill in counters, etc, in this > structure. > > Any concerns with this approach before I go down this path? Are there any > compatibility guarantees I need to uphold with the profile output of > queries? > > -Todd > -- > Todd Lipcon > Software Engineer, Cloudera >
