Hi Todd, I'm going to put the patch up for review any minute. After I finish first time setup on https://gerrit.cloudera.org
-Andrew On Tue, Sep 4, 2018 at 4:43 PM Todd Lipcon <t...@cloudera.com> wrote: > On Tue, Sep 4, 2018 at 4:28 PM, Andrew Sherman <asher...@cloudera.com> > wrote: > > > Hi Todd, > > > > I am making a simple fix for > > IMPALA-6568 <https://issues.apache.org/jira/browse/IMPALA-6568> add > > missing > > Query Compilation section to profiles. > > which would add the Timeline to all responses to createExecRequest. > > > > It sounds like your change is more deep. If you go ahead with your change > > it sounds like my change might be redundant. > > > > I'm not sure if it's totally redundant. Do you have a WIP patch already? > I'm not sure why the existing timeline doesn't show up in all statement > types, so maybe some changes are needed related to that, and then those > changes will still be necessary when exposing a full profile node? > > Agreed we're likely to conflict, at the least, though. Do you have an > estimate of when your patch will be up for review so we can coordinate > which one goes in first? > > -Todd > > > > > > > On Tue, Sep 4, 2018 at 4:02 PM Todd Lipcon <t...@cloudera.com> wrote: > > > > > Hey folks, > > > > > > I'm working on a patch to add some more diagnostics from the planning > > > process into query profiles. > > > > > > Currently, only the planning "Timeline" is reported back as part of the > > > response to createExecRequest. As part of the fetch-on-demand catalog > > work > > > I'd like to start exposing various counters such as cache hit/miss > > counts, > > > time spent on remote calls to the catalog, etc. Even in the existing > code > > > paths, I can see some similar metrics being useful. > > > > > > My current thinking is to remove the 'timeline' (TEventSequence) field > > > from TExecRequest and replace it with a full TRuntimeProfileNode. I'd > > then > > > add some capability in the Java side to fill in counters, etc, in this > > > structure. > > > > > > Any concerns with this approach before I go down this path? Are there > any > > > compatibility guarantees I need to uphold with the profile output of > > > queries? > > > > > > -Todd > > > -- > > > Todd Lipcon > > > Software Engineer, Cloudera > > > > > > > > > -- > Todd Lipcon > Software Engineer, Cloudera >
