You should look at the job counters and start and end time to get that information. PigStats and PigProgressNotificaitonListener ( https://pig.apache.org/docs/r0.17.0/test.html#pig-statistics) are other ways to get that information if you are invoking pig programmatically.
On Mon, Nov 18, 2019 at 12:56 PM Alex Nastetsky <alex.nastet...@verve.com> wrote: > I would like to track metrics of the Pig job across runs. For example: > - overall runtime > - records read > - records written > > This is available in the Pig client log and needs to be parsed out: > > > Successfully read 14171 records (401 bytes) from: "..." > > Successfully stored 8728906 records (642415596 bytes) in: "..." > > 2019-11-16 20:51:43,448 [MainThread] INFO org.apache.pig.Main - Pig > > script completed in 6 minutes, 46 seconds and 854 milliseconds (406854 > ms) > > > Is there a way to get this data in more machine-friendly format, like > CSV/JSON/etc? >
