On Tue, Sep 4, 2018 at 4:46 PM, Andrew Sherman <asher...@cloudera.com> wrote:
> Hi Todd, > > I'm going to put the patch up for review any minute. After I finish first > time setup on https://gerrit.cloudera.org OK, I guess that settles whose patch goes in first, then, because I haven't written a line of code yet :-D -Todd > > > -Andrew > > On Tue, Sep 4, 2018 at 4:43 PM Todd Lipcon <t...@cloudera.com> wrote: > > > On Tue, Sep 4, 2018 at 4:28 PM, Andrew Sherman <asher...@cloudera.com> > > wrote: > > > > > Hi Todd, > > > > > > I am making a simple fix for > > > IMPALA-6568 <https://issues.apache.org/jira/browse/IMPALA-6568> add > > > missing > > > Query Compilation section to profiles. > > > which would add the Timeline to all responses to createExecRequest. > > > > > > It sounds like your change is more deep. If you go ahead with your > change > > > it sounds like my change might be redundant. > > > > > > > I'm not sure if it's totally redundant. Do you have a WIP patch already? > > I'm not sure why the existing timeline doesn't show up in all statement > > types, so maybe some changes are needed related to that, and then those > > changes will still be necessary when exposing a full profile node? > > > > Agreed we're likely to conflict, at the least, though. Do you have an > > estimate of when your patch will be up for review so we can coordinate > > which one goes in first? > > > > -Todd > > > > > > > > > > > On Tue, Sep 4, 2018 at 4:02 PM Todd Lipcon <t...@cloudera.com> wrote: > > > > > > > Hey folks, > > > > > > > > I'm working on a patch to add some more diagnostics from the planning > > > > process into query profiles. > > > > > > > > Currently, only the planning "Timeline" is reported back as part of > the > > > > response to createExecRequest. As part of the fetch-on-demand catalog > > > work > > > > I'd like to start exposing various counters such as cache hit/miss > > > counts, > > > > time spent on remote calls to the catalog, etc. Even in the existing > > code > > > > paths, I can see some similar metrics being useful. > > > > > > > > My current thinking is to remove the 'timeline' (TEventSequence) > field > > > > from TExecRequest and replace it with a full TRuntimeProfileNode. I'd > > > then > > > > add some capability in the Java side to fill in counters, etc, in > this > > > > structure. > > > > > > > > Any concerns with this approach before I go down this path? Are there > > any > > > > compatibility guarantees I need to uphold with the profile output of > > > > queries? > > > > > > > > -Todd > > > > -- > > > > Todd Lipcon > > > > Software Engineer, Cloudera > > > > > > > > > > > > > > > -- > > Todd Lipcon > > Software Engineer, Cloudera > > > -- Todd Lipcon Software Engineer, Cloudera
