-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68687/
-----------------------------------------------------------
(Updated Oct. 15, 2018, 2:13 p.m.)
Review request for Sqoop, Boglarka Egyed, daniel voros, and Szabolcs Vasas.
Bugs: SQOOP-3381
https://issues.apache.org/jira/browse/SQOOP-3381
Repository: sqoop-trunk
Description (updated)
-------
This change upgrades our parquet library to the newest version and a whole lot
of libraries to newer versions with it.
As we will need to register a data supplier in the fix for parquet decimal
support (SQOOP-3382), we will need a version that contains PARQUET-243. We need
to upgrade the Parquet library to a version that contains this fix and is
compatible with Hadoop 3.0.
A few things to note:
- hadoop's version is still 2.8.0
- hive is upgraded to 2.1.1
- the rest of the dependency changes are required for the hive version bump.
There is are a few changes in the codebase, but of course no new functionality
at all:
- in the TestParquetImport class, the new implementation returns a Utf8 object
for Strings written out.
- Added the security policy and related code changes from the patch for
SQOOP-3305 (upgrade hadoop) written by Daniel Voros.
- modified HiveMiniCluster config so it won't try to start a web ui (it's
unnecessary during tests anyway)
Diffs (updated)
-----
build.gradle fc7fc0c4
gradle.properties 0d30378d
gradle/sqoop-package.gradle 1a8d994d
ivy.xml 670cb32d
ivy/libraries.properties 8f3dab2b
src/java/org/apache/sqoop/avro/AvroUtil.java 1663b1d1
src/java/org/apache/sqoop/hive/HiveImport.java 48800366
src/java/org/apache/sqoop/mapreduce/hcat/DerbyPolicy.java PRE-CREATION
src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatUtilities.java 784b5f2a
src/java/org/apache/sqoop/mapreduce/parquet/hadoop/HadoopParquetExportJobConfigurator.java
2180cc20
src/java/org/apache/sqoop/mapreduce/parquet/hadoop/HadoopParquetImportJobConfigurator.java
90b910a3
src/java/org/apache/sqoop/mapreduce/parquet/hadoop/HadoopParquetMergeJobConfigurator.java
66ebc5b8
src/test/org/apache/sqoop/TestParquetExport.java be1d8164
src/test/org/apache/sqoop/TestParquetImport.java 2810e318
src/test/org/apache/sqoop/TestParquetIncrementalImportMerge.java adad0cc1
src/test/org/apache/sqoop/hive/TestHiveServer2ParquetImport.java b55179a4
src/test/org/apache/sqoop/hive/minicluster/HiveMiniCluster.java 9dd54486
src/test/org/apache/sqoop/util/ParquetReader.java f1c2fe10
testdata/hcatalog/conf/hive-site.xml 8a84a5d3
Diff: https://reviews.apache.org/r/68687/diff/4/
Changes: https://reviews.apache.org/r/68687/diff/3-4/
Testing (updated)
-------
Ant unit and 3rd party tests were successful.
gradlew test and thirdpartytest were succesful as well.
Thanks,
Fero Szabo
