[
https://issues.apache.org/jira/browse/PIG-1389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12884834#action_12884834
]
Richard Ding commented on PIG-1389:
-----------------------------------
We use Hadoop Path as a parser to parse the input/output locations for two use
cases:
* Determine the scheme of the location (e.g., hdfs, hbase, file, har, ...), and
* Get the short file name of the location
Ashutosh is right that this approach doesn't work with location string as in
PIG-1229:
{code}
jdbc:hsqldb:file:/tmp/batchtest;hsqldb.default_table_type=cached;hsqldb.cache_rows=100
{code}
A RuntimeException is thrown when trying to parse it.
The proposal is to use Java URI as parser instead for these use cases (Java URI
throws a checked exception for invalid syntax).
> Implement Pig counter to track number of rows for each input files
> -------------------------------------------------------------------
>
> Key: PIG-1389
> URL: https://issues.apache.org/jira/browse/PIG-1389
> Project: Pig
> Issue Type: Improvement
> Affects Versions: 0.7.0
> Reporter: Richard Ding
> Assignee: Richard Ding
> Fix For: 0.8.0
>
> Attachments: PIG-1389.patch, PIG-1389.patch, PIG-1389_1.patch
>
>
> A MR job generated by Pig not only can have multiple outputs (in the case of
> multiquery) but also can have multiple inputs (in the case of join or
> cogroup). In both cases, the existing Hadoop counters (e.g.
> MAP_INPUT_RECORDS, REDUCE_OUTPUT_RECORDS) can not be used to count the number
> of records in the given input or output. PIG-1299 addressed the case of
> multiple outputs. We need to add new counters for jobs with multiple inputs.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
