[jira] [Commented] (HIVE-13911) load inpath fails throwing org.apache.hadoop.security.AccessControlException
[
https://issues.apache.org/jira/browse/HIVE-13911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15716593#comment-15716593
]
Sahil Takiar commented on HIVE-13911:
-
[~hsubramaniyan] this patch states:
{code}
// If we do a rename for a non-local file, we will be transfering the original
// file permissions from source to the destination. Else, in case of mvFile()
where we
// copy from source to destination, we will inherit the destination's parent
group ownership.
{code}
Could you explain this in more detail? When renaming a file on the same
filesystem, say from a staging directory to the final table location, why would
we want to preserve the permissions of the src file? Wouldn't we want to just
set the permissions to be the same as the parent directory (e.g. inherit parent
permissions)?
> load inpath fails throwing org.apache.hadoop.security.AccessControlException
>
>
> Key: HIVE-13911
> URL: https://issues.apache.org/jira/browse/HIVE-13911
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
>Affects Versions: 2.1.0
>Reporter: Hari Sankar Sivarama Subramaniyan
>Assignee: Hari Sankar Sivarama Subramaniyan
> Fix For: 2.1.0
>
> Attachments: HIVE-13911.1.patch, HIVE-13911.2.patch,
> HIVE-13911.3.patch, HIVE-13911.4.patch, HIVE-13911.5.patch
>
>
> Similar to HIVE-13857
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-13911) load inpath fails throwing org.apache.hadoop.security.AccessControlException
[
https://issues.apache.org/jira/browse/HIVE-13911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15318495#comment-15318495
]
Hive QA commented on HIVE-13911:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12808518/HIVE-13911.5.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 12 failed/errored test(s), 10220 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_acid_globallimit
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_acid_table_stats
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_create_func1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_12
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_13
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_list_bucket
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subquery_multiinsert
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_constprog_partitioner
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_index_bitmap3
org.apache.hive.hcatalog.hbase.TestPigHBaseStorageHandler.org.apache.hive.hcatalog.hbase.TestPigHBaseStorageHandler
org.apache.hive.jdbc.TestJdbcWithLocalClusterSpark.testPermFunc
org.apache.hive.jdbc.TestJdbcWithMiniMr.testPermFunc
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/29/testReport
Console output:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/29/console
Test logs:
http://ec2-50-18-27-0.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-29/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 12 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12808518 - PreCommit-HIVE-MASTER-Build
> load inpath fails throwing org.apache.hadoop.security.AccessControlException
>
>
> Key: HIVE-13911
> URL: https://issues.apache.org/jira/browse/HIVE-13911
> Project: Hive
> Issue Type: Bug
>Reporter: Hari Sankar Sivarama Subramaniyan
>Assignee: Hari Sankar Sivarama Subramaniyan
> Attachments: HIVE-13911.1.patch, HIVE-13911.2.patch,
> HIVE-13911.3.patch, HIVE-13911.4.patch, HIVE-13911.5.patch
>
>
> Similar to HIVE-13857
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-13911) load inpath fails throwing org.apache.hadoop.security.AccessControlException
[ https://issues.apache.org/jira/browse/HIVE-13911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15317776#comment-15317776 ] Ashutosh Chauhan commented on HIVE-13911: - +1 pending tests > load inpath fails throwing org.apache.hadoop.security.AccessControlException > > > Key: HIVE-13911 > URL: https://issues.apache.org/jira/browse/HIVE-13911 > Project: Hive > Issue Type: Bug >Reporter: Hari Sankar Sivarama Subramaniyan >Assignee: Hari Sankar Sivarama Subramaniyan > Attachments: HIVE-13911.1.patch, HIVE-13911.2.patch, > HIVE-13911.3.patch, HIVE-13911.4.patch, HIVE-13911.5.patch > > > Similar to HIVE-13857 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13911) load inpath fails throwing org.apache.hadoop.security.AccessControlException
[ https://issues.apache.org/jira/browse/HIVE-13911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15317570#comment-15317570 ] Ashutosh Chauhan commented on HIVE-13911: - seems like srcGroup assignment is reversed. > load inpath fails throwing org.apache.hadoop.security.AccessControlException > > > Key: HIVE-13911 > URL: https://issues.apache.org/jira/browse/HIVE-13911 > Project: Hive > Issue Type: Bug >Reporter: Hari Sankar Sivarama Subramaniyan >Assignee: Hari Sankar Sivarama Subramaniyan > Attachments: HIVE-13911.1.patch, HIVE-13911.2.patch, > HIVE-13911.3.patch, HIVE-13911.4.patch, HIVE-13911.5.patch > > > Similar to HIVE-13857 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13911) load inpath fails throwing org.apache.hadoop.security.AccessControlException
[
https://issues.apache.org/jira/browse/HIVE-13911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15317535#comment-15317535
]
Hive QA commented on HIVE-13911:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12808431/HIVE-13911.4.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 8 failed/errored test(s), 10220 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_acid_globallimit
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_acid_table_stats
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_12
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_13
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subquery_multiinsert
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_constprog_partitioner
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_index_bitmap3
org.apache.hadoop.hive.llap.tezplugins.TestLlapTaskSchedulerService.testDelayedLocalityNodeCommErrorImmediateAllocation
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/20/testReport
Console output:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/20/console
Test logs:
http://ec2-50-18-27-0.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-20/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 8 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12808431 - PreCommit-HIVE-MASTER-Build
> load inpath fails throwing org.apache.hadoop.security.AccessControlException
>
>
> Key: HIVE-13911
> URL: https://issues.apache.org/jira/browse/HIVE-13911
> Project: Hive
> Issue Type: Bug
>Reporter: Hari Sankar Sivarama Subramaniyan
>Assignee: Hari Sankar Sivarama Subramaniyan
> Attachments: HIVE-13911.1.patch, HIVE-13911.2.patch,
> HIVE-13911.3.patch, HIVE-13911.4.patch
>
>
> Similar to HIVE-13857
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-13911) load inpath fails throwing org.apache.hadoop.security.AccessControlException
[ https://issues.apache.org/jira/browse/HIVE-13911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15315268#comment-15315268 ] Hari Sankar Sivarama Subramaniyan commented on HIVE-13911: -- Looking at it. Looks like skipping setFullFileStatus() altogether when mvFile is called is incorrect in some scenario although it worked for the test cases I tried out. Thanks. > load inpath fails throwing org.apache.hadoop.security.AccessControlException > > > Key: HIVE-13911 > URL: https://issues.apache.org/jira/browse/HIVE-13911 > Project: Hive > Issue Type: Bug >Reporter: Hari Sankar Sivarama Subramaniyan >Assignee: Hari Sankar Sivarama Subramaniyan > Attachments: HIVE-13911.1.patch, HIVE-13911.2.patch, > HIVE-13911.3.patch > > > Similar to HIVE-13857 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13911) load inpath fails throwing org.apache.hadoop.security.AccessControlException
[
https://issues.apache.org/jira/browse/HIVE-13911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15315205#comment-15315205
]
Hive QA commented on HIVE-13911:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12807844/HIVE-13911.3.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 11 failed/errored test(s), 10204 tests
executed
*Failed tests:*
{noformat}
TestJdbcWithMiniHA - did not produce a TEST-*.xml file
TestJdbcWithMiniMr - did not produce a TEST-*.xml file
TestOperationLoggingAPIWithTez - did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_12
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_13
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_rand_partitionpruner3
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subquery_multiinsert
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_constprog_partitioner
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_index_bitmap3
org.apache.hadoop.hive.ql.security.TestExtendedAcls.testLoadLocal
org.apache.hadoop.hive.ql.security.TestFolderPermissions.testLoadLocal
{noformat}
Test results:
http://ec2-54-177-240-2.us-west-1.compute.amazonaws.com/job/PreCommit-HIVE-MASTER-Build/513/testReport
Console output:
http://ec2-54-177-240-2.us-west-1.compute.amazonaws.com/job/PreCommit-HIVE-MASTER-Build/513/console
Test logs:
http://ec2-50-18-27-0.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-513/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 11 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12807844 - PreCommit-HIVE-MASTER-Build
> load inpath fails throwing org.apache.hadoop.security.AccessControlException
>
>
> Key: HIVE-13911
> URL: https://issues.apache.org/jira/browse/HIVE-13911
> Project: Hive
> Issue Type: Bug
>Reporter: Hari Sankar Sivarama Subramaniyan
>Assignee: Hari Sankar Sivarama Subramaniyan
> Attachments: HIVE-13911.1.patch, HIVE-13911.2.patch,
> HIVE-13911.3.patch
>
>
> Similar to HIVE-13857
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-13911) load inpath fails throwing org.apache.hadoop.security.AccessControlException
[ https://issues.apache.org/jira/browse/HIVE-13911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15313294#comment-15313294 ] Ashutosh Chauhan commented on HIVE-13911: - +1 pending tests > load inpath fails throwing org.apache.hadoop.security.AccessControlException > > > Key: HIVE-13911 > URL: https://issues.apache.org/jira/browse/HIVE-13911 > Project: Hive > Issue Type: Bug >Reporter: Hari Sankar Sivarama Subramaniyan >Assignee: Hari Sankar Sivarama Subramaniyan > Attachments: HIVE-13911.1.patch, HIVE-13911.2.patch, > HIVE-13911.3.patch > > > Similar to HIVE-13857 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13911) load inpath fails throwing org.apache.hadoop.security.AccessControlException
[ https://issues.apache.org/jira/browse/HIVE-13911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15312906#comment-15312906 ] Hari Sankar Sivarama Subramaniyan commented on HIVE-13911: -- When the source file is local, the patch#1 doesnt work as both destFs and srcFs point to org.apache.hadoop.hive.ql.io.ProxyLocalFileSystem when tested locally. I am adding the condition in patch#2. Tested it locally. Thanks Hari > load inpath fails throwing org.apache.hadoop.security.AccessControlException > > > Key: HIVE-13911 > URL: https://issues.apache.org/jira/browse/HIVE-13911 > Project: Hive > Issue Type: Bug >Reporter: Hari Sankar Sivarama Subramaniyan >Assignee: Hari Sankar Sivarama Subramaniyan > Attachments: HIVE-13911.1.patch > > > Similar to HIVE-13857 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13911) load inpath fails throwing org.apache.hadoop.security.AccessControlException
[ https://issues.apache.org/jira/browse/HIVE-13911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15311411#comment-15311411 ] Hari Sankar Sivarama Subramaniyan commented on HIVE-13911: -- Draft patch pending verification. cc [~ashutoshc] > load inpath fails throwing org.apache.hadoop.security.AccessControlException > > > Key: HIVE-13911 > URL: https://issues.apache.org/jira/browse/HIVE-13911 > Project: Hive > Issue Type: Bug >Reporter: Hari Sankar Sivarama Subramaniyan >Assignee: Hari Sankar Sivarama Subramaniyan > Attachments: HIVE-13911.1.patch > > > Similar to HIVE-13857 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
