[jira] [Commented] (HIVE-12505) Insert overwrite in same encrypted zone silently fails to remove some existing files
[ https://issues.apache.org/jira/browse/HIVE-12505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15043575#comment-15043575 ] Lefty Leverenz commented on HIVE-12505: --- Will this also be committed to branch-2.0 for the 2.0.0 release? > Insert overwrite in same encrypted zone silently fails to remove some > existing files > > > Key: HIVE-12505 > URL: https://issues.apache.org/jira/browse/HIVE-12505 > Project: Hive > Issue Type: Bug > Components: Encryption >Affects Versions: 1.2.1 >Reporter: Chaoyu Tang >Assignee: Chaoyu Tang > Fix For: 1.3.0, 2.1.0 > > Attachments: HIVE-12505.1.patch, HIVE-12505.2.patch, > HIVE-12505.3.patch, HIVE-12505.patch > > > With HDFS Trash enabled but its encryption zone lower than Hive data > directory, insert overwrite command silently fails to trash the existing > files during overwrite, which could lead to unexpected incorrect results > (more rows returned than expected) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12505) Insert overwrite in same encrypted zone silently fails to remove some existing files
[ https://issues.apache.org/jira/browse/HIVE-12505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15043602#comment-15043602 ] Chaoyu Tang commented on HIVE-12505: Thanks, [~leftylev] I backported it branch-2.0 > Insert overwrite in same encrypted zone silently fails to remove some > existing files > > > Key: HIVE-12505 > URL: https://issues.apache.org/jira/browse/HIVE-12505 > Project: Hive > Issue Type: Bug > Components: Encryption >Affects Versions: 1.2.1 >Reporter: Chaoyu Tang >Assignee: Chaoyu Tang > Fix For: 1.3.0, 2.1.0 > > Attachments: HIVE-12505.1.patch, HIVE-12505.2.patch, > HIVE-12505.3.patch, HIVE-12505.patch > > > With HDFS Trash enabled but its encryption zone lower than Hive data > directory, insert overwrite command silently fails to trash the existing > files during overwrite, which could lead to unexpected incorrect results > (more rows returned than expected) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12505) Insert overwrite in same encrypted zone silently fails to remove some existing files
[ https://issues.apache.org/jira/browse/HIVE-12505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15038480#comment-15038480 ] Chaoyu Tang commented on HIVE-12505: Failed tests are not related to this patch. > Insert overwrite in same encrypted zone silently fails to remove some > existing files > > > Key: HIVE-12505 > URL: https://issues.apache.org/jira/browse/HIVE-12505 > Project: Hive > Issue Type: Bug > Components: Encryption >Affects Versions: 1.2.1 >Reporter: Chaoyu Tang >Assignee: Chaoyu Tang > Attachments: HIVE-12505.1.patch, HIVE-12505.2.patch, > HIVE-12505.3.patch, HIVE-12505.patch > > > With HDFS Trash enabled but its encryption zone lower than Hive data > directory, insert overwrite command silently fails to trash the existing > files during overwrite, which could lead to unexpected incorrect results > (more rows returned than expected) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12505) Insert overwrite in same encrypted zone silently fails to remove some existing files
[
https://issues.apache.org/jira/browse/HIVE-12505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15038253#comment-15038253
]
Hive QA commented on HIVE-12505:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12775370/HIVE-12505.3.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 13 failed/errored test(s), 9886 tests
executed
*Failed tests:*
{noformat}
TestHWISessionManager - did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_acid_globallimit
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cbo_udf_max
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_dynamic_partition_pruning
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_vectorized_dynamic_partition_pruning
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_acid_globallimit
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_mergejoin
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import
org.apache.hadoop.hive.metastore.TestHiveMetaStorePartitionSpecs.testFetchingPartitionsWithDifferentSchemas
org.apache.hadoop.hive.metastore.TestHiveMetaStorePartitionSpecs.testGetPartitionSpecs_WithAndWithoutPartitionGrouping
org.apache.hive.jdbc.TestJdbcWithMiniHS2.testAddJarDataNucleusUnCaching
org.apache.hive.jdbc.TestSSL.testSSLVersion
org.apache.hive.jdbc.miniHS2.TestHs2Metrics.testMetrics
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6216/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6216/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-6216/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 13 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12775370 - PreCommit-HIVE-TRUNK-Build
> Insert overwrite in same encrypted zone silently fails to remove some
> existing files
>
>
> Key: HIVE-12505
> URL: https://issues.apache.org/jira/browse/HIVE-12505
> Project: Hive
> Issue Type: Bug
> Components: Encryption
>Affects Versions: 1.2.1
>Reporter: Chaoyu Tang
>Assignee: Chaoyu Tang
> Attachments: HIVE-12505.1.patch, HIVE-12505.2.patch,
> HIVE-12505.3.patch, HIVE-12505.patch
>
>
> With HDFS Trash enabled but its encryption zone lower than Hive data
> directory, insert overwrite command silently fails to trash the existing
> files during overwrite, which could lead to unexpected incorrect results
> (more rows returned than expected)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-12505) Insert overwrite in same encrypted zone silently fails to remove some existing files
[ https://issues.apache.org/jira/browse/HIVE-12505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15036709#comment-15036709 ] Aihua Xu commented on HIVE-12505: - +1 on the new patch. I feel we should trash the oldPath when oldPath is not subdir of destf since currently we lose the capability to bring the old data back for such case, but this is not related to this jira. We can have a followup if we see this as an issue. > Insert overwrite in same encrypted zone silently fails to remove some > existing files > > > Key: HIVE-12505 > URL: https://issues.apache.org/jira/browse/HIVE-12505 > Project: Hive > Issue Type: Bug > Components: Encryption >Affects Versions: 1.2.1 >Reporter: Chaoyu Tang >Assignee: Chaoyu Tang > Attachments: HIVE-12505.1.patch, HIVE-12505.2.patch, > HIVE-12505.3.patch, HIVE-12505.patch > > > With HDFS Trash enabled but its encryption zone lower than Hive data > directory, insert overwrite command silently fails to trash the existing > files during overwrite, which could lead to unexpected incorrect results > (more rows returned than expected) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12505) Insert overwrite in same encrypted zone silently fails to remove some existing files
[ https://issues.apache.org/jira/browse/HIVE-12505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15031956#comment-15031956 ] Chaoyu Tang commented on HIVE-12505: The six test failures are not related to this patch. > Insert overwrite in same encrypted zone silently fails to remove some > existing files > > > Key: HIVE-12505 > URL: https://issues.apache.org/jira/browse/HIVE-12505 > Project: Hive > Issue Type: Bug > Components: Encryption >Affects Versions: 1.2.1 >Reporter: Chaoyu Tang >Assignee: Chaoyu Tang > Attachments: HIVE-12505.1.patch, HIVE-12505.2.patch, HIVE-12505.patch > > > With HDFS Trash enabled but its encryption zone lower than Hive data > directory, insert overwrite command silently fails to trash the existing > files during overwrite, which could lead to unexpected incorrect results > (more rows returned than expected) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12505) Insert overwrite in same encrypted zone silently fails to remove some existing files
[
https://issues.apache.org/jira/browse/HIVE-12505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15031304#comment-15031304
]
Hive QA commented on HIVE-12505:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12774658/HIVE-12505.2.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 6 failed/errored test(s), 9866 tests executed
*Failed tests:*
{noformat}
TestHWISessionManager - did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import
org.apache.hadoop.hive.metastore.TestHiveMetaStorePartitionSpecs.testFetchingPartitionsWithDifferentSchemas
org.apache.hadoop.hive.metastore.TestHiveMetaStorePartitionSpecs.testGetPartitionSpecs_WithAndWithoutPartitionGrouping
org.apache.hive.jdbc.TestSSL.testSSLVersion
org.apache.hive.jdbc.miniHS2.TestHs2Metrics.testMetrics
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6167/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6167/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-6167/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 6 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12774658 - PreCommit-HIVE-TRUNK-Build
> Insert overwrite in same encrypted zone silently fails to remove some
> existing files
>
>
> Key: HIVE-12505
> URL: https://issues.apache.org/jira/browse/HIVE-12505
> Project: Hive
> Issue Type: Bug
> Components: Encryption
>Affects Versions: 1.2.1
>Reporter: Chaoyu Tang
>Assignee: Chaoyu Tang
> Attachments: HIVE-12505.1.patch, HIVE-12505.2.patch, HIVE-12505.patch
>
>
> With HDFS Trash enabled but its encryption zone lower than Hive data
> directory, insert overwrite command silently fails to trash the existing
> files during overwrite, which could lead to unexpected incorrect results
> (more rows returned than expected)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-12505) Insert overwrite in same encrypted zone silently fails to remove some existing files
[
https://issues.apache.org/jira/browse/HIVE-12505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15028660#comment-15028660
]
Hive QA commented on HIVE-12505:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12774060/HIVE-12505.1.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 965 failed/errored test(s), 9865 tests
executed
*Failed tests:*
{noformat}
TestHWISessionManager - did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_acid_vectorization
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_acid_vectorization_partition
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_acid_vectorization_project
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_add_part_exist
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter3
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter4
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter5
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_char2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_index
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_numbuckets_partitioned_table2_h23
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_numbuckets_partitioned_table_h23
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_partition_change_col
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_partition_coltype
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_partition_update_status
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_rename_partition
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_rename_partition_authorization
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_rename_table
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_table_cascade
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_table_serde2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_varchar2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_analyze_table_null_partition
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_analyze_tbl_part
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_part
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_archive_excludeHadoop20
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_archive_multi
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_authorization_2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_authorization_6
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_authorization_explain
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join14
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join19
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join25
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join9
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_avro_date
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_avro_partitioned
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_avro_partitioned_native
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_avro_schema_evolution_native
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_avro_timestamp
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucket3
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucket_groupby
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin13
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketsortoptimize_insert_1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketsortoptimize_insert_2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketsortoptimize_insert_3
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketsortoptimize_insert_4
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketsortoptimize_insert_5
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketsortoptimize_insert_6
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketsortoptimize_insert_7
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketsortoptimize_insert_8
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cbo_gby
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cbo_gby_empty
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cbo_join
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cbo_limit
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cbo_rp_auto_join0
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cbo_rp_gby
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cbo_rp_gby_empty
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cbo_rp_join
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cbo_rp_join0
