[
https://issues.apache.org/jira/browse/PIG-2207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13082638#comment-13082638
]
Dmitriy V. Ryaboy commented on PIG-2207:
----------------------------------------
Right, the client communication is done via counters, and if they want to
debug, they can go look in the logs. Other than syntax errors, all of my Pig
job debugging starts with looking at task logs anyway.
I see what you meant with the counter names now -- so the counter would be
"PigCounters.UDF_WARNING_1-dereferencing_null_in_LOWER" for example. This
accomplishes the goal of communicating the error to the client, but still
produces many counters, which you just said goes against the recommendation in
MAPREDUCE-1943, and also makes the counter name too long -- 1943 restricts
counter names to 64 chars.
What is the value of keeping all udf counters in the same group?
> Support custom counters for aggregating warnings from different udfs
> --------------------------------------------------------------------
>
> Key: PIG-2207
> URL: https://issues.apache.org/jira/browse/PIG-2207
> Project: Pig
> Issue Type: Improvement
> Reporter: Thejas M Nair
> Labels: newbie
> Fix For: 0.10
>
>
> Pig allows udfs to aggregate warning messages instead of writing out a
> separate warning message each time. Udfs can do this by logging the warning
> using EvalFunc.warn(String msg, Enum) call. But the udfs are forced to use
> PigWarning class if the warning needs to be printed at the end of the pig
> script .
> For example, with the changes in PIG-2191, some of the builtin udfs are using
> PigWarning.UDF_WARNING_1 as argument in calls to EvalFunc.warn. This will
> result in the warning count being printed on STDERR -
> {code}
> 2011-08-05 22:10:29,285 [main] WARN
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - Encountered Warning UDF_WARNING_1 2 time(s).
> 2011-08-05 22:10:29,285 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - Success!
> {code}
> But it would be better if a udf such as the LOWER udf could use a custom
> warning counter, and the STDERR is like -
> {code}
> 2011-08-05 22:10:29,285 [main] WARN
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - Encountered Warning LOWER_FUNC_INPUT_WARNING 2 time(s).
> 2011-08-05 22:10:29,285 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - Success!
> {code}
> A new function could be added to support this - (something like)
> EvalFunc.warn(String warnName, String warnMsg); A specific counter group
> could be used for udf warnings (see org.apache.hadoop.mapred.Counters), and
> counters for that group could be done during final warning aggregation in
> done in MapReduceLauncher.computeWarningAggregate().
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
