[ https://issues.apache.org/jira/browse/PIG-2924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13493019#comment-13493019 ]
Bill Graham commented on PIG-2924: ---------------------------------- Sorry for the delay on the review Chelsoo. Looking good. A few more comments. Let me know what you think. - I think we need to pass the {{POStore}} instead of the location. PigStorage impls provided by random parties might not all abide by a unique namespacing convention in their location syntax. For example, {{VerticaStorer}} uses a syntax like "{[db_schema].[table_name]}" (curly brackets included). Another implementor could use the same syntax. - JobStats.getOuputSize could be simplified by doing this, which is more commonly done: {noformat} String reporterNames = conf.get( PigStatsOutputSizeReader.OUTPUT_SIZE_READER_KEY, FileBasedOutputSizeReader.class.getCanonicalName()); {noformat} - Does {{PigContext.instantiateFuncFromSpec(className)}} (without appending "()") not work? - It seems like it would be reasonable for {{PigStatsOutputSizeReader.getOutputSize}} to throw IOException all the way up to {{JobStats}}. - Let's make {{DummyOutputSizeReader}} an inner class of {{TestJobStats}} since that package is already totally bloated. - In {{pig.properties}} reducers' should not have an apostrophe (no possessive for inanimate objects). > PigStats should not be assuming all Storage classes to be file-based storage > ---------------------------------------------------------------------------- > > Key: PIG-2924 > URL: https://issues.apache.org/jira/browse/PIG-2924 > Project: Pig > Issue Type: Bug > Components: tools > Affects Versions: 0.9.2, 0.10.0 > Reporter: Harsh J > Assignee: Cheolsoo Park > Attachments: PIG-2924-2.patch, PIG-2924-3.patch, PIG-2924-4.patch, > PIG-2924.patch > > > Using PigStatsUtil (like Oozie does) to collect JobStats for jobs that use a > HBaseStorage blows up when the stats are asked to be accumulated. > This is because JobStats (which adds stuff up) is assuming all storages are > file based and that it can do listStatus/etc. operations on their > filespec-provided filename. For HBaseStorage, this is set to the tablename > and there's no such file, leading to an exception (FileNotFound or Invalid > URI - depending on using 'tablename' or 'hbase://tablename'). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira