On Tue, Sep 4, 2018 at 4:28 PM, Andrew Sherman <asher...@cloudera.com> wrote:
> Hi Todd, > > I am making a simple fix for > IMPALA-6568 <https://issues.apache.org/jira/browse/IMPALA-6568> add > missing > Query Compilation section to profiles. > which would add the Timeline to all responses to createExecRequest. > > It sounds like your change is more deep. If you go ahead with your change > it sounds like my change might be redundant. > I'm not sure if it's totally redundant. Do you have a WIP patch already? I'm not sure why the existing timeline doesn't show up in all statement types, so maybe some changes are needed related to that, and then those changes will still be necessary when exposing a full profile node? Agreed we're likely to conflict, at the least, though. Do you have an estimate of when your patch will be up for review so we can coordinate which one goes in first? -Todd > > > On Tue, Sep 4, 2018 at 4:02 PM Todd Lipcon <t...@cloudera.com> wrote: > > > Hey folks, > > > > I'm working on a patch to add some more diagnostics from the planning > > process into query profiles. > > > > Currently, only the planning "Timeline" is reported back as part of the > > response to createExecRequest. As part of the fetch-on-demand catalog > work > > I'd like to start exposing various counters such as cache hit/miss > counts, > > time spent on remote calls to the catalog, etc. Even in the existing code > > paths, I can see some similar metrics being useful. > > > > My current thinking is to remove the 'timeline' (TEventSequence) field > > from TExecRequest and replace it with a full TRuntimeProfileNode. I'd > then > > add some capability in the Java side to fill in counters, etc, in this > > structure. > > > > Any concerns with this approach before I go down this path? Are there any > > compatibility guarantees I need to uphold with the profile output of > > queries? > > > > -Todd > > -- > > Todd Lipcon > > Software Engineer, Cloudera > > > -- Todd Lipcon Software Engineer, Cloudera
