> On Sept. 12, 2012, 8:58 p.m., Travis Crawford wrote: > > Overall this looks good Feng, thanks for diving into external table write > > support. Most of the comments below are documentation/polish rather than > > major changes. > > > > The only part of this change that's a little weird is requiring a wrapper. > > However, the issue there is changing the HCatStorer interface which is a > > separate issue, and if we choose to change that interface in the future > > this code is not affected. So I think its appropriate to treat them as > > separate issues.
Yes, this is a simpler than an interface change. If the property is not set in the UDFContext, no code path is changed. A similar support can be added to HCatStorer later. > On Sept. 12, 2012, 8:58 p.m., Travis Crawford wrote: > > hcatalog-pig-adapter/src/test/java/org/apache/hcatalog/pig/HCatStorerWrapper.java, > > line 28 > > <https://reviews.apache.org/r/7043/diff/2/?file=152684#file152684line28> > > > > Please include a javadoc about this class as it could be quite > > confusing to someone seeing this the first time. > > > > I believe the reason this class exists is because we don't want to > > change the HCatStorer interface to add a new parameter, so this class is > > added for testing. > > > > If we do change the HCatStorer interface I really think we should > > consider using a single string and using an option parser, which gives us a > > lot of flexibility in the future, and would be the last time the interface > > needs to change. Good point. Javadoc added. > On Sept. 12, 2012, 8:58 p.m., Travis Crawford wrote: > > hcatalog-pig-adapter/src/test/java/org/apache/hcatalog/pig/TestHCatStorerWrapper.java, > > line 42 > > <https://reviews.apache.org/r/7043/diff/2/?file=152685#file152685line42> > > > > How about using guava Files: > > > > File tmpExternalDir = Files.createTempDir(); > > > > > > http://docs.guava-libraries.googlecode.com/git/javadoc/com/google/common/io/Files.html#createTempDir() Good point. Fixed. > On Sept. 12, 2012, 8:58 p.m., Travis Crawford wrote: > > hcatalog-pig-adapter/src/test/java/org/apache/hcatalog/pig/TestHCatStorerWrapper.java, > > line 49 > > <https://reviews.apache.org/r/7043/diff/2/?file=152685#file152685line49> > > > > How about using an assert on a single line: > > > > Assert.assertEquals(0, driver.run(cmd).getResponseCode()) - Feng ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/7043/#review11428 ----------------------------------------------------------- On Sept. 11, 2012, 9:59 p.m., Feng Peng wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/7043/ > ----------------------------------------------------------- > > (Updated Sept. 11, 2012, 9:59 p.m.) > > > Review request for hcatalog, Dmitriy Ryaboy and Travis Crawford. > > > Description > ------- > > We added a property called HCAT_PIG_STORER_EXTERNAL_LOCATION to the > UDFContext of HCatStorer: when the property is set AND the table type is > EXTERNAL, HCatStorer writes the data to the external path the property > specifies. > > > This addresses bug hcatalog-500. > https://issues.apache.org/jira/browse/hcatalog-500 > > > Diffs > ----- > > hcatalog-pig-adapter/src/main/java/org/apache/hcatalog/pig/HCatStorer.java > c46db33 > > hcatalog-pig-adapter/src/test/java/org/apache/hcatalog/pig/HCatStorerWrapper.java > PRE-CREATION > > hcatalog-pig-adapter/src/test/java/org/apache/hcatalog/pig/TestHCatStorerWrapper.java > PRE-CREATION > src/java/org/apache/hcatalog/common/HCatConstants.java 626d91b > src/java/org/apache/hcatalog/mapreduce/FileOutputCommitterContainer.java > 8e692dc > src/java/org/apache/hcatalog/mapreduce/FosterStorageHandler.java 3f368d8 > > Diff: https://reviews.apache.org/r/7043/diff/ > > > Testing > ------- > > Added a corresponding unit test. Also tested on our cluster. > > > Thanks, > > Feng Peng > >
