Thanks Bill.
Sent from my iPhone
On Jan 22, 2013, at 10:10 PM, Bill Graham wrote:
> Sure, here you go:
>
> https://github.com/twitter/ambrose/blob/master/pig/src/main/java/com/twitter/ambrose/pig/AmbrosePigProgressNotificationListener.java
>
>
>
> On Tue, Jan 22, 2013 at 10:04 PM, Prashant Komm
Sure, here you go:
https://github.com/twitter/ambrose/blob/master/pig/src/main/java/com/twitter/ambrose/pig/AmbrosePigProgressNotificationListener.java
On Tue, Jan 22, 2013 at 10:04 PM, Prashant Kommireddi
wrote:
> Thanks Bill. Can you please point me to the Ambrose code that uses PPNL?
>
> I
Thanks Bill. Can you please point me to the Ambrose code that uses PPNL?
I will open a JIRA for getting hooks with explain in.
Sent from my iPhone
On Jan 22, 2013, at 9:03 PM, Bill Graham wrote:
> Yeah, getting at the info here is tricky. For Ambrose we're getting info
> about submitted jobs,
Yeah, getting at the info here is tricky. For Ambrose we're getting info
about submitted jobs, so we can just hook into the lifecycle of
PigProgressNotificationListener. The PPNL notifiers are pretty coupled to
PigStatsUtil and ScriptState, which aren't invoked during explain.
The bulk of the acti
I think that this is all available, it's just not the easiest thing to get
at. If you look at the explain plan, it has a lot of this info, and you can
definitely get at that info. I'm not sure if it has the reducers or if
that's post MR setup, but you should be able to.
That said, I do not think i
Jon/others - any pointers on this? I would like to patch in hooks if this
is not possible at the moment.
-Prashant
On Mon, Jan 21, 2013 at 5:47 PM, Prashant Kommireddi wrote:
> At the moment, basically info on I/O paths, operators used (group by,
> foreach ..), job level info such as number of r
At the moment, basically info on I/O paths, operators used (group by,
foreach ..), job level info such as number of reducers etc.
On Mon, Jan 21, 2013 at 5:30 PM, Jonathan Coveney wrote:
> What level of information would you like? IE if you do "explain relation,"
> which of the three do you want
What level of information would you like? IE if you do "explain relation,"
which of the three do you want to hook into?
2013/1/21 Prashant Kommireddi
> Been coding with the APIs and wondering if there is anything that allows
> you to only retrieve the operators, I/O paths etc without actually i
