[jira] [Updated] (HIVE-2711) Make the header of RCFile unique
[ https://issues.apache.org/jira/browse/HIVE-2711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-2711: --- Resolution: Fixed Fix Version/s: 0.10 Status: Resolved (was: Patch Available) Committed to trunk. Thanks, Owen! > Make the header of RCFile unique > > > Key: HIVE-2711 > URL: https://issues.apache.org/jira/browse/HIVE-2711 > Project: Hive > Issue Type: Bug > Components: Serializers/Deserializers >Reporter: Owen O'Malley >Assignee: Owen O'Malley > Fix For: 0.10 > > Attachments: HIVE-2711.D2115.1.patch, HIVE-2711.D2115.2.patch, > HIVE-2711.D2115.3.patch, HIVE-2711.D2571.1.patch, rc-file-v0.rc > > > The RCFile implementation was copied from Hadoop's SequenceFile and copied > the 'magic' string in the header. This means that you can't use the header to > distinguish between RCFiles and SequenceFiles. > I'd propose that we create a new header for RCFiles (RCF?) to replace the > current SEQ. To maintain compatibility, we'll need to continue to accept the > current 'SEQ\06' and just make new files contain the new header. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2711) Make the header of RCFile unique
[ https://issues.apache.org/jira/browse/HIVE-2711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley updated HIVE-2711: Attachment: rc-file-v0.rc Here's the binary file that arc didn't include in the patch. > Make the header of RCFile unique > > > Key: HIVE-2711 > URL: https://issues.apache.org/jira/browse/HIVE-2711 > Project: Hive > Issue Type: Bug > Components: Serializers/Deserializers >Reporter: Owen O'Malley >Assignee: Owen O'Malley > Attachments: HIVE-2711.D2115.1.patch, HIVE-2711.D2115.2.patch, > HIVE-2711.D2115.3.patch, HIVE-2711.D2571.1.patch, rc-file-v0.rc > > > The RCFile implementation was copied from Hadoop's SequenceFile and copied > the 'magic' string in the header. This means that you can't use the header to > distinguish between RCFiles and SequenceFiles. > I'd propose that we create a new header for RCFiles (RCF?) to replace the > current SEQ. To maintain compatibility, we'll need to continue to accept the > current 'SEQ\06' and just make new files contain the new header. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2711) Make the header of RCFile unique
[ https://issues.apache.org/jira/browse/HIVE-2711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley updated HIVE-2711: Status: Patch Available (was: Open) Sorry, I didn't re-run the test cases after removing the unused fields from the header. All of the unit tests pass now. > Make the header of RCFile unique > > > Key: HIVE-2711 > URL: https://issues.apache.org/jira/browse/HIVE-2711 > Project: Hive > Issue Type: Bug > Components: Serializers/Deserializers >Reporter: Owen O'Malley >Assignee: Owen O'Malley > Attachments: HIVE-2711.D2115.1.patch, HIVE-2711.D2115.2.patch, > HIVE-2711.D2115.3.patch, HIVE-2711.D2571.1.patch > > > The RCFile implementation was copied from Hadoop's SequenceFile and copied > the 'magic' string in the header. This means that you can't use the header to > distinguish between RCFiles and SequenceFiles. > I'd propose that we create a new header for RCFiles (RCF?) to replace the > current SEQ. To maintain compatibility, we'll need to continue to accept the > current 'SEQ\06' and just make new files contain the new header. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2711) Make the header of RCFile unique
[ https://issues.apache.org/jira/browse/HIVE-2711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-2711: -- Attachment: HIVE-2711.D2115.3.patch omalley updated the revision "HIVE-2711 [jira] Make the header of RCFile unique". Reviewers: JIRA Fixed unit tests based on the new file sizes. REVISION DETAIL https://reviews.facebook.net/D2115 AFFECTED FILES ql/src/java/org/apache/hadoop/hive/ql/io/RCFile.java ql/src/test/data/rc-file-v0.rc ql/src/test/org/apache/hadoop/hive/ql/io/TestRCFile.java ql/src/test/results/clientpositive/alter_concatenate_indexed_table.q.out ql/src/test/results/clientpositive/alter_merge.q.out ql/src/test/results/clientpositive/alter_merge_stats.q.out ql/src/test/results/clientpositive/create_merge_compressed.q.out ql/src/test/results/clientpositive/ctas.q.out ql/src/test/results/clientpositive/partition_wise_fileformat.q.out ql/src/test/results/clientpositive/partition_wise_fileformat3.q.out ql/src/test/results/clientpositive/sample10.q.out > Make the header of RCFile unique > > > Key: HIVE-2711 > URL: https://issues.apache.org/jira/browse/HIVE-2711 > Project: Hive > Issue Type: Bug > Components: Serializers/Deserializers >Reporter: Owen O'Malley >Assignee: Owen O'Malley > Attachments: HIVE-2711.D2115.1.patch, HIVE-2711.D2115.2.patch, > HIVE-2711.D2115.3.patch, HIVE-2711.D2571.1.patch > > > The RCFile implementation was copied from Hadoop's SequenceFile and copied > the 'magic' string in the header. This means that you can't use the header to > distinguish between RCFiles and SequenceFiles. > I'd propose that we create a new header for RCFiles (RCF?) to replace the > current SEQ. To maintain compatibility, we'll need to continue to accept the > current 'SEQ\06' and just make new files contain the new header. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2711) Make the header of RCFile unique
[ https://issues.apache.org/jira/browse/HIVE-2711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-2711: -- Attachment: HIVE-2711.D2115.2.patch omalley updated the revision "HIVE-2711 [jira] Make the header of RCFile unique". Reviewers: JIRA updated to current trunk REVISION DETAIL https://reviews.facebook.net/D2115 AFFECTED FILES ql/src/java/org/apache/hadoop/hive/ql/io/RCFile.java ql/src/test/data/rc-file-v0.rc ql/src/test/org/apache/hadoop/hive/ql/io/TestRCFile.java > Make the header of RCFile unique > > > Key: HIVE-2711 > URL: https://issues.apache.org/jira/browse/HIVE-2711 > Project: Hive > Issue Type: Bug > Components: Serializers/Deserializers >Reporter: Owen O'Malley >Assignee: Owen O'Malley > Attachments: HIVE-2711.D2115.1.patch, HIVE-2711.D2115.2.patch, > HIVE-2711.D2571.1.patch > > > The RCFile implementation was copied from Hadoop's SequenceFile and copied > the 'magic' string in the header. This means that you can't use the header to > distinguish between RCFiles and SequenceFiles. > I'd propose that we create a new header for RCFiles (RCF?) to replace the > current SEQ. To maintain compatibility, we'll need to continue to accept the > current 'SEQ\06' and just make new files contain the new header. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2711) Make the header of RCFile unique
[ https://issues.apache.org/jira/browse/HIVE-2711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-2711: -- Attachment: HIVE-2711.D2571.1.patch omalley requested code review of "HIVE-2711 [jira] Make the header of RCFile unique". Reviewers: JIRA HIVE-2711 Make the header of RCFile unique wrt SequenceFile The RCFile implementation was copied from Hadoop's SequenceFile and copied the 'magic' string in the header. This means that you can't use the header to distinguish between RCFiles and SequenceFiles. I'd propose that we create a new header for RCFiles (RCF?) to replace the current SEQ. To maintain compatibility, we'll need to continue to accept the current 'SEQ\06' and just make new files contain the new header. TEST PLAN EMPTY REVISION DETAIL https://reviews.facebook.net/D2571 AFFECTED FILES ql/src/java/org/apache/hadoop/hive/ql/io/RCFile.java ql/src/test/data/rc-file-v0.rc ql/src/test/org/apache/hadoop/hive/ql/io/TestRCFile.java MANAGE HERALD DIFFERENTIAL RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/5835/ Tip: use the X-Herald-Rules header to filter Herald messages in your client. > Make the header of RCFile unique > > > Key: HIVE-2711 > URL: https://issues.apache.org/jira/browse/HIVE-2711 > Project: Hive > Issue Type: Bug > Components: Serializers/Deserializers >Reporter: Owen O'Malley >Assignee: Owen O'Malley > Attachments: HIVE-2711.D2115.1.patch, HIVE-2711.D2571.1.patch > > > The RCFile implementation was copied from Hadoop's SequenceFile and copied > the 'magic' string in the header. This means that you can't use the header to > distinguish between RCFiles and SequenceFiles. > I'd propose that we create a new header for RCFiles (RCF?) to replace the > current SEQ. To maintain compatibility, we'll need to continue to accept the > current 'SEQ\06' and just make new files contain the new header. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2711) Make the header of RCFile unique
[ https://issues.apache.org/jira/browse/HIVE-2711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-2711: --- Status: Open (was: Patch Available) > Make the header of RCFile unique > > > Key: HIVE-2711 > URL: https://issues.apache.org/jira/browse/HIVE-2711 > Project: Hive > Issue Type: Bug > Components: Serializers/Deserializers >Reporter: Owen O'Malley >Assignee: Owen O'Malley > Attachments: HIVE-2711.D2115.1.patch > > > The RCFile implementation was copied from Hadoop's SequenceFile and copied > the 'magic' string in the header. This means that you can't use the header to > distinguish between RCFiles and SequenceFiles. > I'd propose that we create a new header for RCFiles (RCF?) to replace the > current SEQ. To maintain compatibility, we'll need to continue to accept the > current 'SEQ\06' and just make new files contain the new header. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2711) Make the header of RCFile unique
[ https://issues.apache.org/jira/browse/HIVE-2711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley updated HIVE-2711: Status: Patch Available (was: Open) > Make the header of RCFile unique > > > Key: HIVE-2711 > URL: https://issues.apache.org/jira/browse/HIVE-2711 > Project: Hive > Issue Type: Bug > Components: Serializers/Deserializers >Reporter: Owen O'Malley >Assignee: Owen O'Malley > Attachments: HIVE-2711.D2115.1.patch > > > The RCFile implementation was copied from Hadoop's SequenceFile and copied > the 'magic' string in the header. This means that you can't use the header to > distinguish between RCFiles and SequenceFiles. > I'd propose that we create a new header for RCFiles (RCF?) to replace the > current SEQ. To maintain compatibility, we'll need to continue to accept the > current 'SEQ\06' and just make new files contain the new header. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2711) Make the header of RCFile unique
[ https://issues.apache.org/jira/browse/HIVE-2711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-2711: -- Attachment: HIVE-2711.D2115.1.patch omalley requested code review of "HIVE-2711 [jira] Make the header of RCFile unique". Reviewers: JIRA HIVE-2711 Make the header of RCFile unique wrt SequenceFile The RCFile implementation was copied from Hadoop's SequenceFile and copied the 'magic' string in the header. This means that you can't use the header to distinguish between RCFiles and SequenceFiles. I'd propose that we create a new header for RCFiles (RCF?) to replace the current SEQ. To maintain compatibility, we'll need to continue to accept the current 'SEQ\06' and just make new files contain the new header. TEST PLAN EMPTY REVISION DETAIL https://reviews.facebook.net/D2115 AFFECTED FILES ql/src/java/org/apache/hadoop/hive/ql/io/RCFile.java ql/src/test/data/rc-file-v0.rc ql/src/test/org/apache/hadoop/hive/ql/io/TestRCFile.java MANAGE HERALD DIFFERENTIAL RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/4587/ Tip: use the X-Herald-Rules header to filter Herald messages in your client. > Make the header of RCFile unique > > > Key: HIVE-2711 > URL: https://issues.apache.org/jira/browse/HIVE-2711 > Project: Hive > Issue Type: Bug > Components: Serializers/Deserializers >Reporter: Owen O'Malley >Assignee: Owen O'Malley > Attachments: HIVE-2711.D2115.1.patch > > > The RCFile implementation was copied from Hadoop's SequenceFile and copied > the 'magic' string in the header. This means that you can't use the header to > distinguish between RCFiles and SequenceFiles. > I'd propose that we create a new header for RCFiles (RCF?) to replace the > current SEQ. To maintain compatibility, we'll need to continue to accept the > current 'SEQ\06' and just make new files contain the new header. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
