[
https://issues.apache.org/jira/browse/PHOENIX-129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13937393#comment-13937393
]
Hudson commented on PHOENIX-129:
--------------------------------
SUCCESS: Integrated in Apache Phoenix - Branch:3.0 #37 (See
[https://builds.apache.org/job/Phoenix-3.0/37/])
PHOENIX-129 Improve MapReduce-based import (GabrielReid) (jamestaylor: rev
ed9f58bae048ccc1e5653808e59d6c3231c91529)
* phoenix-core/src/test/java/org/apache/phoenix/util/ColumnInfoTest.java
* phoenix-core/src/main/java/org/apache/phoenix/util/CSVCommonsLoader.java
* phoenix-assembly/src/build/all.xml
* phoenix-core/src/main/java/org/apache/phoenix/mapreduce/CsvBulkLoadTool.java
* phoenix-core/src/it/java/org/apache/phoenix/mapreduce/CsvBulkLoadToolIT.java
* phoenix-core/src/main/java/org/apache/phoenix/schema/PDataType.java
*
phoenix-core/src/test/java/org/apache/phoenix/mapreduce/CsvToKeyValueMapperTest.java
* bin/csv-bulk-loader.py
*
phoenix-core/src/main/java/org/apache/phoenix/map/reduce/util/ConfigReader.java
* bin/readme.txt
* phoenix-core/src/main/java/org/apache/phoenix/map/reduce/CSVBulkLoader.java
* phoenix-assembly/src/build/mapreduce.xml
*
phoenix-core/src/main/java/org/apache/phoenix/mapreduce/ImportPreUpsertKeyValueProcessor.java
*
phoenix-core/src/main/java/org/apache/phoenix/mapreduce/CsvToKeyValueMapper.java
* phoenix-assembly/pom.xml
*
phoenix-core/src/test/java/org/apache/phoenix/mapreduce/CsvBulkLoadToolTest.java
* phoenix-core/src/main/java/org/apache/phoenix/util/ColumnInfo.java
* phoenix-core/src/main/java/org/apache/phoenix/map/reduce/MapReduceJob.java
* phoenix-core/src/main/java/org/apache/phoenix/mapreduce/CsvBulkImportUtil.java
*
phoenix-core/src/test/java/org/apache/phoenix/mapreduce/CsvBulkImportUtilTest.java
> Improve MapReduce-based import
> ------------------------------
>
> Key: PHOENIX-129
> URL: https://issues.apache.org/jira/browse/PHOENIX-129
> Project: Phoenix
> Issue Type: Improvement
> Reporter: Gabriel Reid
> Assignee: Gabriel Reid
> Attachments: PHOENIX-129-3.0.patch, PHOENIX-129-3.0_2.patch,
> PHOENIX-129-master.patch, PHOENIX-129-master_2.patch
>
>
> In implementing PHOENIX-66, it was noted that the current MapReduce-based
> importer implementation has a number issues, including the following:
> * CSV handling is largely replicated from the non-MR code, with no ability to
> specify custom separators
> * No automated tests, and code is written in a way that makes it difficult to
> test
> * Unusual custom config loading and handling instead of using
> GenericOptionParser and ToolRunner and friends
> The initial work towards PHOENIX-66 included refactoring the MR importer
> enough to use common code, up until the development of automated testing
> exposed the fact that the MR importer could use some major refactoring.
> This ticket is a proposal to do a relatively major rework of the MR import,
> fixing the above issues. The biggest improvements that will result from this
> are a common codebase for handling CSV input, and the addition of automated
> testing for the MR import.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
