Add progress notification listener to PigRunner API
---------------------------------------------------
Key: PIG-1478
URL: https://issues.apache.org/jira/browse/PIG-1478
Project: Pig
Issue Type: Improvement
Reporter: Richard Ding
Assignee: Richard Ding
Fix For: 0.8.0
PIG-1333 added PigRunner API to allow Pig users and tools to get a status/stats
object back after executing a Pig script. The new API, however, is synchronous
(blocking). It's known that a Pig script can spawn tens (even hundreds) MR jobs
and take hours to complete. Therefore it'll be nice to give progress feedback
to the callers during the execution.
The proposal is to add an optional parameter to the API:
{code}
public abstract class PigRunner {
public static PigStats run(String[] args, PigProgressNotificationListener
listener) {...}
}
{code}
The new listener is defined as following:
{code}
package org.apache.pig.tools.pigstats;
public interface PigProgressNotificationListener extends
java.util.EventListener {
// just before the launch of MR jobs for the script
public void LaunchStartedNotification(int numJobsToLaunch);
// number of jobs submitted in a batch
public void jobsSubmittedNotification(int numJobsSubmitted);
// a job is started
public void jobStartedNotification(String assignedJobId);
// a job is completed successfully
public void jobFinishedNotification(JobStats jobStats);
// a job is failed
public void jobFailedNotification(JobStats jobStats);
// a user output is completed successfully
public void outputCompletedNotification(OutputStats outputStats);
// updates the progress as percentage
public void progressUpdatedNotification(int progress);
// the script execution is done
public void launchCompletedNotification(int numJobsSucceeded);
}
{code}
Any thoughts?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
