> On May 22, 2017, 11:50 a.m., Peter Vary wrote: > > Thanks for the patch Zsolt! > > > > It is important to be able to check the data even if does not fit into > > memory! Thanks for the catch! > > > > As for the patch, few comments from YETUS: > > - Checkstyle: I will not put all of them here, but there is a problem with > > the indentation. Could you please check the checkstyle output of > > > > mvn checkstyle:checkstyle -Dcheckstyle.consoleOutput=true -DskipTests > > -Pitests -Ptest-patch > > > > - ASF headers are missing: > > !????? > > /Users/petervary/dev/upstream/hive/metastore/src/java/org/apache/hadoop/hive/metastore/metatool/BlockRetrieverIterable.java > > !????? > > /Users/petervary/dev/upstream/hive/metastore/src/java/org/apache/hadoop/hive/metastore/metatool/BlockRetrieverIterator.java > > !????? > > /Users/petervary/dev/upstream/hive/metastore/src/java/org/apache/hadoop/hive/metastore/metatool/EntityUpdater.java > > !????? > > /Users/petervary/dev/upstream/hive/metastore/src/java/org/apache/hadoop/hive/metastore/metatool/IDataProvider.java > > !????? > > /Users/petervary/dev/upstream/hive/metastore/src/java/org/apache/hadoop/hive/metastore/metatool/LocationEntity.java > > !????? > > /Users/petervary/dev/upstream/hive/metastore/src/java/org/apache/hadoop/hive/metastore/metatool/LocationEntityImplementations.java > > !????? > > /Users/petervary/dev/upstream/hive/metastore/src/java/org/apache/hadoop/hive/metastore/metatool/LocationUpdater.java > > !????? > > /Users/petervary/dev/upstream/hive/metastore/src/java/org/apache/hadoop/hive/metastore/metatool/RetrieverIterable.java > > !????? > > /Users/petervary/dev/upstream/hive/metastore/src/java/org/apache/hadoop/hive/metastore/metatool/ReturnValue.java > > !????? > > /Users/petervary/dev/upstream/hive/metastore/src/java/org/apache/hadoop/hive/metastore/metatool/UpdateParams.java > > !????? > > /Users/petervary/dev/upstream/hive/metastore/src/java/org/apache/hadoop/hive/metastore/metatool/UriUpdateChecker.java > > !????? > > /Users/petervary/dev/upstream/hive/metastore/src/test/org/apache/hadoop/hive/metastore/metatool/BlockRetrieverIterableTest.java > > !????? > > /Users/petervary/dev/upstream/hive/metastore/src/test/org/apache/hadoop/hive/metastore/metatool/DataProviderStub.java > > !????? > > /Users/petervary/dev/upstream/hive/metastore/src/test/org/apache/hadoop/hive/metastore/metatool/EntityUpdaterTest.java > > !????? > > /Users/petervary/dev/upstream/hive/metastore/src/test/org/apache/hadoop/hive/metastore/metatool/LocationEntityImplementationsTest.java > > !????? > > /Users/petervary/dev/upstream/hive/metastore/src/test/org/apache/hadoop/hive/metastore/metatool/LocationUpdaterTest.java > > !????? > > /Users/petervary/dev/upstream/hive/metastore/src/test/org/apache/hadoop/hive/metastore/metatool/ReturnValueTest.java > > !????? > > /Users/petervary/dev/upstream/hive/metastore/src/test/org/apache/hadoop/hive/metastore/metatool/UriUpdateCheckerTest.java > > !????? > > /Users/petervary/dev/upstream/hive/metastore/src/test/org/apache/hadoop/hive/metastore/tools/HiveMetaToolTest.java > > > > Thanks, > > Peter > > Zsolt Fekete wrote: > Thanks Peter, I will fix the checkstyle errors. > I have 2 questions: can you please give some pointers about how can I run > Yetus for hive? And what is "-Ptest-patch"?
Wwe are working on Yetus it with Zoltan. You can start from HIVE-15051. You are right, the -Ptest-patch is not needed, but will not cause any harm either :D. Just copied the command from the Yetus output without any thinking :D - Peter ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59408/#review175622 ----------------------------------------------------------- On May 19, 2017, 5:05 p.m., Zsolt Fekete wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/59408/ > ----------------------------------------------------------- > > (Updated May 19, 2017, 5:05 p.m.) > > > Review request for hive. > > > Repository: hive-git > > > Description > ------- > > Currently HiveMetaTool reads full tables (as DataNucleus entities) into > memory by calling PersistenceManager's retrieveAll(). > > See these methods of ObjectStore: updateMDatabaseURI, updateTblPropURI, > updateMStorageDescriptorTblPropURI, updateMStorageDescriptorTblURI, > updateSerdeURI. > > This might cause failure when the affected tables (SDS, DBS, TABLE_PARAMS, > SD_PARAMS, SERDES, SERDE_PARAMS) are too big. > > > Diffs > ----- > > metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java > b28983f > > metastore/src/java/org/apache/hadoop/hive/metastore/metatool/BlockRetrieverIterable.java > PRE-CREATION > > metastore/src/java/org/apache/hadoop/hive/metastore/metatool/BlockRetrieverIterator.java > PRE-CREATION > > metastore/src/java/org/apache/hadoop/hive/metastore/metatool/EntityUpdater.java > PRE-CREATION > > metastore/src/java/org/apache/hadoop/hive/metastore/metatool/IDataProvider.java > PRE-CREATION > > metastore/src/java/org/apache/hadoop/hive/metastore/metatool/LocationEntity.java > PRE-CREATION > > metastore/src/java/org/apache/hadoop/hive/metastore/metatool/LocationEntityImplementations.java > PRE-CREATION > > metastore/src/java/org/apache/hadoop/hive/metastore/metatool/LocationUpdater.java > PRE-CREATION > > metastore/src/java/org/apache/hadoop/hive/metastore/metatool/RetrieverIterable.java > PRE-CREATION > > metastore/src/java/org/apache/hadoop/hive/metastore/metatool/ReturnValue.java > PRE-CREATION > > metastore/src/java/org/apache/hadoop/hive/metastore/metatool/UpdateParams.java > PRE-CREATION > > metastore/src/java/org/apache/hadoop/hive/metastore/metatool/UriUpdateChecker.java > PRE-CREATION > metastore/src/java/org/apache/hadoop/hive/metastore/tools/HiveMetaTool.java > 22e246f > > metastore/src/test/org/apache/hadoop/hive/metastore/metatool/BlockRetrieverIterableTest.java > PRE-CREATION > > metastore/src/test/org/apache/hadoop/hive/metastore/metatool/DataProviderStub.java > PRE-CREATION > > metastore/src/test/org/apache/hadoop/hive/metastore/metatool/EntityUpdaterTest.java > PRE-CREATION > > metastore/src/test/org/apache/hadoop/hive/metastore/metatool/LocationEntityImplementationsTest.java > PRE-CREATION > > metastore/src/test/org/apache/hadoop/hive/metastore/metatool/LocationUpdaterTest.java > PRE-CREATION > > metastore/src/test/org/apache/hadoop/hive/metastore/metatool/ReturnValueTest.java > PRE-CREATION > > metastore/src/test/org/apache/hadoop/hive/metastore/metatool/UriUpdateCheckerTest.java > PRE-CREATION > > metastore/src/test/org/apache/hadoop/hive/metastore/tools/HiveMetaToolTest.java > PRE-CREATION > > > Diff: https://reviews.apache.org/r/59408/diff/2/ > > > Testing > ------- > > The new tests passed: > mvn test > -Dtest=BlockRetrieverIterableTest,EntityUpdaterTest,UriUpdateCheckerTest,LocationUpdaterTest,ReturnValueTest,LocationEntityImplementationsTest,IntegrationTest > > > Thanks, > > Zsolt Fekete > >
