The important bit would be the app server support.i think given that, Tez would be pretty easy. Ambrose ui is not actually super mr specific. It's a dag display with some extras.
There would certainly be work to be done there. Just suggesting it as an option -- work needs to be done somewhere, regardless. On Jan 24, 2014, at 1:16 PM, Rohini Palaniswamy <[email protected]> wrote: > bq. From a priority point of view, which is more important for Pig - a > history UI for completed applications or an AM UI to monitor a running > application? > Both are important. But I guess if there is a choice to be made, > probably will go with history UI as you can at least look at it and debug > later looking at the logs and config. It would be difficult to always catch > the AM in action and difficult to debug before the run finishes. > > bq. Ambrose could provide both if someone hooked it up to app history > server. > I am not sure how that would work, unless Tez support is added to > Ambrose. > > Regards, > Rohini > > > > On Thu, Jan 23, 2014 at 6:00 PM, Dmitriy Ryaboy <[email protected]> wrote: > >> Ambrose could provide both if someone hooked it up to app history server. >> >> >> On Thu, Jan 23, 2014 at 5:11 PM, Hitesh Shah <[email protected]> wrote: >> >>> At the moment, we are looking at integrating Tez with YARN's application >>> history server mostly for data-related integration. YARN will be >> providing >>> webservices that provides access to this data and therefore allow anyone >> to >>> build a UI. However, there are no concrete plans/timelines around the UI >>> implementation. >>> >>> From a priority point of view, which is more important for Pig - a >> history >>> UI for completed applications or an AM UI to monitor a running >> application? >>> >>> thanks >>> -- Hitesh >>> >>> On Jan 23, 2014, at 4:53 PM, Rohini Palaniswamy wrote: >>> >>>> Bikas came to the Pig Sprint Planning and we had some good discussions >> on >>>> the approaches we are taking. >>>> >>>> Prioritized set of requirements from the Pig Team: >>>> >>>> Q1: >>>> - Partitioned unsorted output >>>> - API to start Input fetch (TEZ-668) >>>> - Fix for any issues with 1-1 Edge found by us as we plan to use them >>>> for one stage in order-by. >>>> - UI for Tez jobs >>>> >>>> Q2: >>>> - Advanced Memory management >>>> - Memory manager for inputs based on map output sizes. >>>> - Memory manager for outputs >>>> As a workaround till then we will set io.sort.mb on the input and >>>> output descriptors ourselves manually based on number of edges. >>>> >>>> Q3/Q4: >>>> - Partial aggregator to determine number of reduces >>>> - Split edge support - same output to multiple vertices >>>> >>>> >>>> Hitesh, >>>> Bikas said you would have information on the UI for Tez as you are >>>> driving it with YARN team. Can you brief us on what we can expect and >>>> possibly when? >>>> >>>> >>>> Regards, >>>> Rohini >>
