[jira] [Commented] (HIVE-3529) Incorrect partition bucket/sort metadata when overwriting partition with different metadata from table
[ https://issues.apache.org/jira/browse/HIVE-3529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13548004#comment-13548004 ] Hudson commented on HIVE-3529: -- Integrated in Hive-trunk-hadoop2 #54 (See [https://builds.apache.org/job/Hive-trunk-hadoop2/54/]) HIVE-3529 Incorrect partition bucket/sort metadata when overwriting partition with different metadata from table (Kevin Wilfong via namit) (Revision 1403363) Result = ABORTED namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1403363 Files : * /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java * /hive/trunk/ql/src/test/queries/clientpositive/alter_numbuckets_partitioned_table2.q * /hive/trunk/ql/src/test/queries/clientpositive/alter_table_serde2.q * /hive/trunk/ql/src/test/results/clientpositive/alter_numbuckets_partitioned_table.q.out * /hive/trunk/ql/src/test/results/clientpositive/alter_numbuckets_partitioned_table2.q.out * /hive/trunk/ql/src/test/results/clientpositive/alter_table_serde2.q.out Incorrect partition bucket/sort metadata when overwriting partition with different metadata from table -- Key: HIVE-3529 URL: https://issues.apache.org/jira/browse/HIVE-3529 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.10.0 Reporter: Kevin Wilfong Assignee: Kevin Wilfong Fix For: 0.10.0 Attachments: HIVE-3529.1.patch.txt If you have a partition with bucket/sort metadata set, then you alter the table to have different bucket/sort metadata, and insert overwrite the partition with hive.enforce.bucketing=true and/or hive.enforce.sorting=true, the partition data will be bucketed/sorted by the table's metadata, but the partition will have the same metadata. This could result in wrong results. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3529) Incorrect partition bucket/sort metadata when overwriting partition with different metadata from table
[ https://issues.apache.org/jira/browse/HIVE-3529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13485942#comment-13485942 ] Namit Jain commented on HIVE-3529: -- +1 Incorrect partition bucket/sort metadata when overwriting partition with different metadata from table -- Key: HIVE-3529 URL: https://issues.apache.org/jira/browse/HIVE-3529 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.10.0 Reporter: Kevin Wilfong Assignee: Kevin Wilfong Attachments: HIVE-3529.1.patch.txt If you have a partition with bucket/sort metadata set, then you alter the table to have different bucket/sort metadata, and insert overwrite the partition with hive.enforce.bucketing=true and/or hive.enforce.sorting=true, the partition data will be bucketed/sorted by the table's metadata, but the partition will have the same metadata. This could result in wrong results. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3529) Incorrect partition bucket/sort metadata when overwriting partition with different metadata from table
[ https://issues.apache.org/jira/browse/HIVE-3529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13486502#comment-13486502 ] Hudson commented on HIVE-3529: -- Integrated in Hive-trunk-h0.21 #1763 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1763/]) HIVE-3529 Incorrect partition bucket/sort metadata when overwriting partition with different metadata from table (Kevin Wilfong via namit) (Revision 1403363) Result = SUCCESS namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1403363 Files : * /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java * /hive/trunk/ql/src/test/queries/clientpositive/alter_numbuckets_partitioned_table2.q * /hive/trunk/ql/src/test/queries/clientpositive/alter_table_serde2.q * /hive/trunk/ql/src/test/results/clientpositive/alter_numbuckets_partitioned_table.q.out * /hive/trunk/ql/src/test/results/clientpositive/alter_numbuckets_partitioned_table2.q.out * /hive/trunk/ql/src/test/results/clientpositive/alter_table_serde2.q.out Incorrect partition bucket/sort metadata when overwriting partition with different metadata from table -- Key: HIVE-3529 URL: https://issues.apache.org/jira/browse/HIVE-3529 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.10.0 Reporter: Kevin Wilfong Assignee: Kevin Wilfong Fix For: 0.10.0 Attachments: HIVE-3529.1.patch.txt If you have a partition with bucket/sort metadata set, then you alter the table to have different bucket/sort metadata, and insert overwrite the partition with hive.enforce.bucketing=true and/or hive.enforce.sorting=true, the partition data will be bucketed/sorted by the table's metadata, but the partition will have the same metadata. This could result in wrong results. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3529) Incorrect partition bucket/sort metadata when overwriting partition with different metadata from table
[ https://issues.apache.org/jira/browse/HIVE-3529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13479239#comment-13479239 ] Kevin Wilfong commented on HIVE-3529: - Also, the serde parameters are wrong as well. Incorrect partition bucket/sort metadata when overwriting partition with different metadata from table -- Key: HIVE-3529 URL: https://issues.apache.org/jira/browse/HIVE-3529 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.10.0 Reporter: Kevin Wilfong Assignee: Kevin Wilfong If you have a partition with bucket/sort metadata set, then you alter the table to have different bucket/sort metadata, and insert overwrite the partition with hive.enforce.bucketing=true and/or hive.enforce.sorting=true, the partition data will be bucketed/sorted by the table's metadata, but the partition will have the same metadata. This could result in wrong results. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3529) Incorrect partition bucket/sort metadata when overwriting partition with different metadata from table
[ https://issues.apache.org/jira/browse/HIVE-3529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13479254#comment-13479254 ] Kevin Wilfong commented on HIVE-3529: - The following information is now copied from the table to the partition when writing to an existing partition: bucket columns bucket number sort columns/orders serde parameters Incorrect partition bucket/sort metadata when overwriting partition with different metadata from table -- Key: HIVE-3529 URL: https://issues.apache.org/jira/browse/HIVE-3529 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.10.0 Reporter: Kevin Wilfong Assignee: Kevin Wilfong Attachments: HIVE-3529.1.patch.txt If you have a partition with bucket/sort metadata set, then you alter the table to have different bucket/sort metadata, and insert overwrite the partition with hive.enforce.bucketing=true and/or hive.enforce.sorting=true, the partition data will be bucketed/sorted by the table's metadata, but the partition will have the same metadata. This could result in wrong results. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3529) Incorrect partition bucket/sort metadata when overwriting partition with different metadata from table
[ https://issues.apache.org/jira/browse/HIVE-3529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13479253#comment-13479253 ] Kevin Wilfong commented on HIVE-3529: - https://reviews.facebook.net/D6087 Incorrect partition bucket/sort metadata when overwriting partition with different metadata from table -- Key: HIVE-3529 URL: https://issues.apache.org/jira/browse/HIVE-3529 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.10.0 Reporter: Kevin Wilfong Assignee: Kevin Wilfong Attachments: HIVE-3529.1.patch.txt If you have a partition with bucket/sort metadata set, then you alter the table to have different bucket/sort metadata, and insert overwrite the partition with hive.enforce.bucketing=true and/or hive.enforce.sorting=true, the partition data will be bucketed/sorted by the table's metadata, but the partition will have the same metadata. This could result in wrong results. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3529) Incorrect partition bucket/sort metadata when overwriting partition with different metadata from table
[ https://issues.apache.org/jira/browse/HIVE-3529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13469067#comment-13469067 ] Kevin Wilfong commented on HIVE-3529: - My proposed fix is to by default always overwrite the partition's bucket/sorting metadata with that of the table when overwriting a table. My main motivation for doing this vs. using the partition's metadata is dynamic partitions. The potential for having to manage maintaining all the different bucket/sorting schemes across several partitions which are overwritten dynamically sounds like a new feature rather than a bug fix, and could be done in a separate JIRA. Incorrect partition bucket/sort metadata when overwriting partition with different metadata from table -- Key: HIVE-3529 URL: https://issues.apache.org/jira/browse/HIVE-3529 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.10.0 Reporter: Kevin Wilfong Assignee: Kevin Wilfong If you have a partition with bucket/sort metadata set, then you alter the table to have different bucket/sort metadata, and insert overwrite the partition with hive.enforce.bucketing=true and/or hive.enforce.sorting=true, the partition data will be bucketed/sorted by the table's metadata, but the partition will have the same metadata. This could result in wrong results. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
