[
https://issues.apache.org/jira/browse/HIVE-5102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Phabricator updated HIVE-5102:
------------------------------
Attachment: HIVE-5102.D12849.1.patch
omalley requested code review of "HIVE-5102 [jira] ORC getSplits should create
splits based the stripes".
Reviewers: JIRA
HIVE-5102
Currently ORC inherits getSplits from FileFormat, which basically makes a split
per an HDFS block. This can create too little parallelism and would be better
done by having getSplits look at the file footer and create splits based on the
stripes.
TEST PLAN
EMPTY
REVISION DETAIL
https://reviews.facebook.net/D12849
AFFECTED FILES
ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java
ql/src/java/org/apache/hadoop/hive/ql/io/orc/ReaderImpl.java
ql/src/java/org/apache/hadoop/hive/ql/io/orc/StripeInformation.java
ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestInputOutputFormat.java
ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestOrcFile.java
shims/src/0.20/java/org/apache/hadoop/hive/shims/Hadoop20Shims.java
shims/src/0.20S/java/org/apache/hadoop/hive/shims/Hadoop20SShims.java
shims/src/0.23/java/org/apache/hadoop/hive/shims/Hadoop23Shims.java
shims/src/common/java/org/apache/hadoop/hive/shims/HadoopShims.java
MANAGE HERALD RULES
https://reviews.facebook.net/herald/view/differential/
WHY DID I GET THIS EMAIL?
https://reviews.facebook.net/herald/transcript/30807/
To: JIRA, omalley
> ORC getSplits should create splits based the stripes
> -----------------------------------------------------
>
> Key: HIVE-5102
> URL: https://issues.apache.org/jira/browse/HIVE-5102
> Project: Hive
> Issue Type: Bug
> Components: File Formats
> Reporter: Owen O'Malley
> Assignee: Owen O'Malley
> Attachments: HIVE-5102.D12579.1.patch, HIVE-5102.D12579.2.patch,
> HIVE-5102.D12849.1.patch
>
>
> Currently ORC inherits getSplits from FileFormat, which basically makes a
> split per an HDFS block. This can create too little parallelism and would be
> better done by having getSplits look at the file footer and create splits
> based on the stripes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
