[
https://issues.apache.org/jira/browse/SQOOP-428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13192574#comment-13192574
]
[email protected] commented on SQOOP-428:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3600/#review4566
-----------------------------------------------------------
Looks good. Have you run manual tests with it too?
src/test/com/cloudera/sqoop/TestAvroImport.java
<https://reviews.apache.org/r/3600/#comment10194>
You should check that the files that are written are compressed (by looking
at DataFileReader's metadata).
We also need a test for --compress.
- Tom
On 2012-01-24 14:07:58, Lars Francke wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/3600/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-01-24 14:07:58)
bq.
bq.
bq. Review request for Sqoop.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. This basically only ports all the code from Avro's (1.5.4)
AvroOutputFormat to the new MR API.
bq.
bq. I've changed the test to extract the common functionality into a helper
method because they are the same apart from the two command line arguments.
bq.
bq. I could have deleted AvroJob completely but as I was told last time that
binary compatibility needs to be maintained I left it in. It's not needed
anymore as all necessary functionality can be gotten from Avro's own version of
that file as far as I can tell. So if it's okay to delete that redundant file
(two actually, cloudera and apache package) let me know and I'll provide a new
patch.
bq.
bq.
bq. This addresses bug SQOOP-428.
bq. https://issues.apache.org/jira/browse/SQOOP-428
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. src/java/org/apache/sqoop/mapreduce/AvroJob.java a57aaf1
bq. src/java/org/apache/sqoop/mapreduce/AvroOutputFormat.java 96befd7
bq. src/test/com/cloudera/sqoop/TestAvroImport.java 1b8b046
bq.
bq. Diff: https://reviews.apache.org/r/3600/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. All tests pass for hadoopversion=20 but TestColumnTypes fails for me on
23. I can't see how that's related though.
bq.
bq.
bq. Thanks,
bq.
bq. Lars
bq.
bq.
> AvroOutputFormat doesn't support compression even though documentation claims
> it does
> -------------------------------------------------------------------------------------
>
> Key: SQOOP-428
> URL: https://issues.apache.org/jira/browse/SQOOP-428
> Project: Sqoop
> Issue Type: Bug
> Components: docs
> Affects Versions: 1.4.0-incubating
> Reporter: Lars Francke
> Priority: Minor
> Labels: avro, document
> Attachments: SQOOP-428.1.patch
>
>
> The documentation claims that Avro files can be compressed as well:
> {quote}
> By default, data is not compressed. You can compress your data by using the
> deflate (gzip) algorithm with the -z or --compress argument, or specify any
> Hadoop compression codec using the --compression-codec argument. This applies
> to SequenceFile, text, and Avro files.
> {quote}
> This is not true as the AvroOutputFormat currently doesn't support
> compression.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
