Re: Review Request 67873: Add Hive support to the new Parquet writing implementation

2018-07-16 Thread Szabolcs Vasas 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67873/
---

(Updated July 16, 2018, 10:16 a.m.)


Review request for Sqoop.


Changes
---

Unused import is removed.


Bugs: SQOOP-3335
https://issues.apache.org/jira/browse/SQOOP-3335


Repository: sqoop-trunk


Description
---

SQOOP-3328 adds a new Parquet reading and writing implementation to Sqoop it 
does not add support to Hive Parquet imports. The task of this Jira is to add 
this missing functionality.


Diffs (updated)
-

  src/java/org/apache/sqoop/hive/HiveTypes.java ad00535e5 
  src/java/org/apache/sqoop/hive/TableDefWriter.java 27d988c53 
  src/java/org/apache/sqoop/mapreduce/parquet/ParquetImportJobConfigurator.java 
eb6d08f8a 
  
src/java/org/apache/sqoop/mapreduce/parquet/hadoop/HadoopParquetImportJobConfigurator.java
 3f35faf86 
  
src/java/org/apache/sqoop/mapreduce/parquet/kite/KiteParquetImportJobConfigurator.java
 feb3bf19b 
  src/java/org/apache/sqoop/tool/BaseSqoopTool.java e505c2671 
  src/java/org/apache/sqoop/tool/ImportTool.java 25c3f7031 
  src/test/org/apache/sqoop/TestParquetIncrementalImportMerge.java d8d3af40f 
  src/test/org/apache/sqoop/hive/TestHiveServer2ParquetImport.java PRE-CREATION 
  src/test/org/apache/sqoop/hive/TestHiveServer2TextImport.java 3d115ab3e 
  src/test/org/apache/sqoop/hive/TestHiveTypesForAvroTypeMapping.java 
PRE-CREATION 
  src/test/org/apache/sqoop/hive/TestTableDefWriter.java 3ea61f646 
  src/test/org/apache/sqoop/testutil/BaseSqoopTestCase.java ac6db0b14 
  src/test/org/apache/sqoop/tool/TestHiveServer2OptionValidations.java 
4d3f93898 


Diff: https://reviews.apache.org/r/67873/diff/2/

Changes: https://reviews.apache.org/r/67873/diff/1-2/


Testing
---

Executed unit and third party test cases.


Thanks,

Szabolcs Vasas

Re: Review Request 67873: Add Hive support to the new Parquet writing implementation

2018-07-16 Thread Boglarka Egyed 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67873/#review206096
---


Fix it, then Ship it!




Hi Szabolcs,

This is an awesome improvement in Sqoop, thank you so much for adding it!

Your change LGTM, I had a very minor finding regarding an unusued import. 
Please correct it and then your patch can be committed I think.

Unit and 3rd party tests ran successfully with your patch.

Thanks,
Bogi


src/java/org/apache/sqoop/hive/TableDefWriter.java
Line 33 (original), 38 (patched)


Became an unused import.


- Boglarka Egyed


On July 10, 2018, 11:26 a.m., Szabolcs Vasas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/67873/
> ---
> 
> (Updated July 10, 2018, 11:26 a.m.)
> 
> 
> Review request for Sqoop.
> 
> 
> Bugs: SQOOP-3335
> https://issues.apache.org/jira/browse/SQOOP-3335
> 
> 
> Repository: sqoop-trunk
> 
> 
> Description
> ---
> 
> SQOOP-3328 adds a new Parquet reading and writing implementation to Sqoop it 
> does not add support to Hive Parquet imports. The task of this Jira is to add 
> this missing functionality.
> 
> 
> Diffs
> -
> 
>   src/java/org/apache/sqoop/hive/HiveTypes.java ad00535e5 
>   src/java/org/apache/sqoop/hive/TableDefWriter.java 27d988c53 
>   
> src/java/org/apache/sqoop/mapreduce/parquet/ParquetImportJobConfigurator.java 
> eb6d08f8a 
>   
> src/java/org/apache/sqoop/mapreduce/parquet/hadoop/HadoopParquetImportJobConfigurator.java
>  3f35faf86 
>   
> src/java/org/apache/sqoop/mapreduce/parquet/kite/KiteParquetImportJobConfigurator.java
>  feb3bf19b 
>   src/java/org/apache/sqoop/tool/BaseSqoopTool.java 8d318327a 
>   src/java/org/apache/sqoop/tool/ImportTool.java 25c3f7031 
>   src/test/org/apache/sqoop/TestParquetIncrementalImportMerge.java d8d3af40f 
>   src/test/org/apache/sqoop/hive/TestHiveServer2ParquetImport.java 
> PRE-CREATION 
>   src/test/org/apache/sqoop/hive/TestHiveServer2TextImport.java 3d115ab3e 
>   src/test/org/apache/sqoop/hive/TestHiveTypesForAvroTypeMapping.java 
> PRE-CREATION 
>   src/test/org/apache/sqoop/hive/TestTableDefWriter.java 3ea61f646 
>   src/test/org/apache/sqoop/testutil/BaseSqoopTestCase.java ac6db0b14 
>   src/test/org/apache/sqoop/tool/TestHiveServer2OptionValidations.java 
> 4d3f93898 
> 
> 
> Diff: https://reviews.apache.org/r/67873/diff/1/
> 
> 
> Testing
> ---
> 
> Executed unit and third party test cases.
> 
> 
> Thanks,
> 
> Szabolcs Vasas
> 
>

Re: Review Request 67873: Add Hive support to the new Parquet writing implementation

2018-07-14 Thread daniel voros 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67873/#review206088
---


Ship it!




Looks good, thank you! Ship it!

- daniel voros


On July 10, 2018, 11:26 a.m., Szabolcs Vasas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/67873/
> ---
> 
> (Updated July 10, 2018, 11:26 a.m.)
> 
> 
> Review request for Sqoop.
> 
> 
> Bugs: SQOOP-3335
> https://issues.apache.org/jira/browse/SQOOP-3335
> 
> 
> Repository: sqoop-trunk
> 
> 
> Description
> ---
> 
> SQOOP-3328 adds a new Parquet reading and writing implementation to Sqoop it 
> does not add support to Hive Parquet imports. The task of this Jira is to add 
> this missing functionality.
> 
> 
> Diffs
> -
> 
>   src/java/org/apache/sqoop/hive/HiveTypes.java ad00535e5 
>   src/java/org/apache/sqoop/hive/TableDefWriter.java 27d988c53 
>   
> src/java/org/apache/sqoop/mapreduce/parquet/ParquetImportJobConfigurator.java 
> eb6d08f8a 
>   
> src/java/org/apache/sqoop/mapreduce/parquet/hadoop/HadoopParquetImportJobConfigurator.java
>  3f35faf86 
>   
> src/java/org/apache/sqoop/mapreduce/parquet/kite/KiteParquetImportJobConfigurator.java
>  feb3bf19b 
>   src/java/org/apache/sqoop/tool/BaseSqoopTool.java 8d318327a 
>   src/java/org/apache/sqoop/tool/ImportTool.java 25c3f7031 
>   src/test/org/apache/sqoop/TestParquetIncrementalImportMerge.java d8d3af40f 
>   src/test/org/apache/sqoop/hive/TestHiveServer2ParquetImport.java 
> PRE-CREATION 
>   src/test/org/apache/sqoop/hive/TestHiveServer2TextImport.java 3d115ab3e 
>   src/test/org/apache/sqoop/hive/TestHiveTypesForAvroTypeMapping.java 
> PRE-CREATION 
>   src/test/org/apache/sqoop/hive/TestTableDefWriter.java 3ea61f646 
>   src/test/org/apache/sqoop/testutil/BaseSqoopTestCase.java ac6db0b14 
>   src/test/org/apache/sqoop/tool/TestHiveServer2OptionValidations.java 
> 4d3f93898 
> 
> 
> Diff: https://reviews.apache.org/r/67873/diff/1/
> 
> 
> Testing
> ---
> 
> Executed unit and third party test cases.
> 
> 
> Thanks,
> 
> Szabolcs Vasas
> 
>

Review Request 67873: Add Hive support to the new Parquet writing implementation

2018-07-10 Thread Szabolcs Vasas 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67873/
---

Review request for Sqoop.


Bugs: SQOOP-3335
https://issues.apache.org/jira/browse/SQOOP-3335


Repository: sqoop-trunk


Description
---

SQOOP-3328 adds a new Parquet reading and writing implementation to Sqoop it 
does not add support to Hive Parquet imports. The task of this Jira is to add 
this missing functionality.


Diffs
-

  src/java/org/apache/sqoop/hive/HiveTypes.java ad00535e5 
  src/java/org/apache/sqoop/hive/TableDefWriter.java 27d988c53 
  src/java/org/apache/sqoop/mapreduce/parquet/ParquetImportJobConfigurator.java 
eb6d08f8a 
  
src/java/org/apache/sqoop/mapreduce/parquet/hadoop/HadoopParquetImportJobConfigurator.java
 3f35faf86 
  
src/java/org/apache/sqoop/mapreduce/parquet/kite/KiteParquetImportJobConfigurator.java
 feb3bf19b 
  src/java/org/apache/sqoop/tool/BaseSqoopTool.java 8d318327a 
  src/java/org/apache/sqoop/tool/ImportTool.java 25c3f7031 
  src/test/org/apache/sqoop/TestParquetIncrementalImportMerge.java d8d3af40f 
  src/test/org/apache/sqoop/hive/TestHiveServer2ParquetImport.java PRE-CREATION 
  src/test/org/apache/sqoop/hive/TestHiveServer2TextImport.java 3d115ab3e 
  src/test/org/apache/sqoop/hive/TestHiveTypesForAvroTypeMapping.java 
PRE-CREATION 
  src/test/org/apache/sqoop/hive/TestTableDefWriter.java 3ea61f646 
  src/test/org/apache/sqoop/testutil/BaseSqoopTestCase.java ac6db0b14 
  src/test/org/apache/sqoop/tool/TestHiveServer2OptionValidations.java 
4d3f93898 


Diff: https://reviews.apache.org/r/67873/diff/1/


Testing
---

Executed unit and third party test cases.


Thanks,

Szabolcs Vasas