[jira] [Commented] (HIVE-5133) webhcat jobs that need to access metastore fails in secure mode
[ https://issues.apache.org/jira/browse/HIVE-5133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13799923#comment-13799923 ] Hudson commented on HIVE-5133: -- FAILURE: Integrated in Hive-trunk-h0.21 #2408 (See [https://builds.apache.org/job/Hive-trunk-h0.21/2408/]) HIVE-5133: webhcat jobs that need to access metastore fails in secure mode (Eugene Koifman via Daniel Dai) (daijy: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1533658) * /hive/trunk/hcatalog/src/test/e2e/templeton/inpdir/hcatloadstore.pig * /hive/trunk/hcatalog/src/test/e2e/templeton/tests/jobsubmission.conf * /hive/trunk/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/CompleteDelegator.java * /hive/trunk/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/HiveDelegator.java * /hive/trunk/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/JarDelegator.java * /hive/trunk/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/LauncherDelegator.java * /hive/trunk/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/PigDelegator.java * /hive/trunk/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/SecureProxySupport.java * /hive/trunk/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/Server.java * /hive/trunk/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/StreamingDelegator.java * /hive/trunk/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/tool/DelegationTokenCache.java * /hive/trunk/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/tool/TempletonControllerJob.java webhcat jobs that need to access metastore fails in secure mode --- Key: HIVE-5133 URL: https://issues.apache.org/jira/browse/HIVE-5133 Project: Hive Issue Type: Bug Components: WebHCat Affects Versions: 0.11.0 Reporter: Thejas M Nair Assignee: Eugene Koifman Fix For: 0.13.0 Attachments: HIVE-5133.1.patch, HIVE-5133.1.test.patch, HIVE-5133.2.patch, HIVE-5133.3.patch, HIVE-5133.5.patch, HIVE-5133.6.patch Webhcat job submission requests result in the pig/hive/mr job being run from a map task that it launches. In secure mode, for the pig/hive/mr job that is run to be authorized to perform actions on metastore, it has to have the delegation tokens from the hive metastore. In case of pig/MR job this is needed if hcatalog is being used in the script/job. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HIVE-5133) webhcat jobs that need to access metastore fails in secure mode
[ https://issues.apache.org/jira/browse/HIVE-5133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=1386#comment-1386 ] Hudson commented on HIVE-5133: -- ABORTED: Integrated in Hive-trunk-hadoop2 #511 (See [https://builds.apache.org/job/Hive-trunk-hadoop2/511/]) HIVE-5133: webhcat jobs that need to access metastore fails in secure mode (Eugene Koifman via Daniel Dai) (daijy: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1533658) * /hive/trunk/hcatalog/src/test/e2e/templeton/inpdir/hcatloadstore.pig * /hive/trunk/hcatalog/src/test/e2e/templeton/tests/jobsubmission.conf * /hive/trunk/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/CompleteDelegator.java * /hive/trunk/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/HiveDelegator.java * /hive/trunk/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/JarDelegator.java * /hive/trunk/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/LauncherDelegator.java * /hive/trunk/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/PigDelegator.java * /hive/trunk/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/SecureProxySupport.java * /hive/trunk/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/Server.java * /hive/trunk/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/StreamingDelegator.java * /hive/trunk/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/tool/DelegationTokenCache.java * /hive/trunk/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/tool/TempletonControllerJob.java webhcat jobs that need to access metastore fails in secure mode --- Key: HIVE-5133 URL: https://issues.apache.org/jira/browse/HIVE-5133 Project: Hive Issue Type: Bug Components: WebHCat Affects Versions: 0.11.0 Reporter: Thejas M Nair Assignee: Eugene Koifman Fix For: 0.13.0 Attachments: HIVE-5133.1.patch, HIVE-5133.1.test.patch, HIVE-5133.2.patch, HIVE-5133.3.patch, HIVE-5133.5.patch, HIVE-5133.6.patch Webhcat job submission requests result in the pig/hive/mr job being run from a map task that it launches. In secure mode, for the pig/hive/mr job that is run to be authorized to perform actions on metastore, it has to have the delegation tokens from the hive metastore. In case of pig/MR job this is needed if hcatalog is being used in the script/job. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HIVE-5133) webhcat jobs that need to access metastore fails in secure mode
[ https://issues.apache.org/jira/browse/HIVE-5133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13800073#comment-13800073 ] Hudson commented on HIVE-5133: -- FAILURE: Integrated in Hive-trunk-hadoop2-ptest #145 (See [https://builds.apache.org/job/Hive-trunk-hadoop2-ptest/145/]) HIVE-5133: webhcat jobs that need to access metastore fails in secure mode (Eugene Koifman via Daniel Dai) (daijy: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1533658) * /hive/trunk/hcatalog/src/test/e2e/templeton/inpdir/hcatloadstore.pig * /hive/trunk/hcatalog/src/test/e2e/templeton/tests/jobsubmission.conf * /hive/trunk/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/CompleteDelegator.java * /hive/trunk/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/HiveDelegator.java * /hive/trunk/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/JarDelegator.java * /hive/trunk/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/LauncherDelegator.java * /hive/trunk/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/PigDelegator.java * /hive/trunk/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/SecureProxySupport.java * /hive/trunk/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/Server.java * /hive/trunk/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/StreamingDelegator.java * /hive/trunk/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/tool/DelegationTokenCache.java * /hive/trunk/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/tool/TempletonControllerJob.java webhcat jobs that need to access metastore fails in secure mode --- Key: HIVE-5133 URL: https://issues.apache.org/jira/browse/HIVE-5133 Project: Hive Issue Type: Bug Components: WebHCat Affects Versions: 0.11.0 Reporter: Thejas M Nair Assignee: Eugene Koifman Fix For: 0.13.0 Attachments: HIVE-5133.1.patch, HIVE-5133.1.test.patch, HIVE-5133.2.patch, HIVE-5133.3.patch, HIVE-5133.5.patch, HIVE-5133.6.patch Webhcat job submission requests result in the pig/hive/mr job being run from a map task that it launches. In secure mode, for the pig/hive/mr job that is run to be authorized to perform actions on metastore, it has to have the delegation tokens from the hive metastore. In case of pig/MR job this is needed if hcatalog is being used in the script/job. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HIVE-5133) webhcat jobs that need to access metastore fails in secure mode
[ https://issues.apache.org/jira/browse/HIVE-5133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13800081#comment-13800081 ] Hudson commented on HIVE-5133: -- SUCCESS: Integrated in Hive-trunk-hadoop1-ptest #208 (See [https://builds.apache.org/job/Hive-trunk-hadoop1-ptest/208/]) HIVE-5133: webhcat jobs that need to access metastore fails in secure mode (Eugene Koifman via Daniel Dai) (daijy: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1533658) * /hive/trunk/hcatalog/src/test/e2e/templeton/inpdir/hcatloadstore.pig * /hive/trunk/hcatalog/src/test/e2e/templeton/tests/jobsubmission.conf * /hive/trunk/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/CompleteDelegator.java * /hive/trunk/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/HiveDelegator.java * /hive/trunk/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/JarDelegator.java * /hive/trunk/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/LauncherDelegator.java * /hive/trunk/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/PigDelegator.java * /hive/trunk/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/SecureProxySupport.java * /hive/trunk/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/Server.java * /hive/trunk/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/StreamingDelegator.java * /hive/trunk/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/tool/DelegationTokenCache.java * /hive/trunk/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/tool/TempletonControllerJob.java webhcat jobs that need to access metastore fails in secure mode --- Key: HIVE-5133 URL: https://issues.apache.org/jira/browse/HIVE-5133 Project: Hive Issue Type: Bug Components: WebHCat Affects Versions: 0.11.0 Reporter: Thejas M Nair Assignee: Eugene Koifman Fix For: 0.13.0 Attachments: HIVE-5133.1.patch, HIVE-5133.1.test.patch, HIVE-5133.2.patch, HIVE-5133.3.patch, HIVE-5133.5.patch, HIVE-5133.6.patch Webhcat job submission requests result in the pig/hive/mr job being run from a map task that it launches. In secure mode, for the pig/hive/mr job that is run to be authorized to perform actions on metastore, it has to have the delegation tokens from the hive metastore. In case of pig/MR job this is needed if hcatalog is being used in the script/job. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HIVE-5133) webhcat jobs that need to access metastore fails in secure mode
[
https://issues.apache.org/jira/browse/HIVE-5133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13799673#comment-13799673
]
Hive QA commented on HIVE-5133:
---
{color:red}Overall{color}: -1 no tests executed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12609078/HIVE-5133.6.patch
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1162/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1162/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Tests failed with: NonZeroExitCodeException: Command 'bash
/data/hive-ptest/working/scratch/source-prep.sh' failed with exit status 1 and
output '+ [[ -n '' ]]
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ export 'M2_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ M2_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ cd /data/hive-ptest/working/
+ tee /data/hive-ptest/logs/PreCommit-HIVE-Build-1162/source-prep.txt
+ [[ true == \t\r\u\e ]]
+ rm -rf ivy maven
+ mkdir -p maven ivy
+ [[ svn = \s\v\n ]]
+ [[ -n '' ]]
+ [[ -d apache-svn-trunk-source ]]
+ [[ ! -d apache-svn-trunk-source/.svn ]]
+ [[ ! -d apache-svn-trunk-source ]]
+ cd apache-svn-trunk-source
+ svn revert -R .
Reverted 'common/src/java/org/apache/hadoop/hive/conf/HiveConf.java'
Reverted 'common/src/java/org/apache/hadoop/hive/common/ObjectPair.java'
Reverted 'ql/src/test/results/clientpositive/auto_sortmerge_join_9.q.out'
Reverted
'ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/PhysicalOptimizer.java'
Reverted 'ql/src/java/org/apache/hadoop/hive/ql/plan/ExplainWork.java'
Reverted
'ql/src/java/org/apache/hadoop/hive/ql/parse/ExplainSemanticAnalyzer.java'
Reverted 'ql/src/java/org/apache/hadoop/hive/ql/exec/Task.java'
Reverted 'ql/src/java/org/apache/hadoop/hive/ql/exec/ExplainTask.java'
++ awk '{print $2}'
++ egrep -v '^X|^Performing status on external'
++ svn status --no-ignore
+ rm -rf build hcatalog/build hcatalog/core/build
hcatalog/storage-handlers/hbase/build hcatalog/server-extensions/build
hcatalog/webhcat/svr/build hcatalog/webhcat/java-client/build
hcatalog/hcatalog-pig-adapter/build common/src/gen
ql/src/test/results/clientpositive/explain_rearrange.q.out
ql/src/test/queries/clientpositive/explain_rearrange.q
ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/StageIDsRearranger.java
+ svn update
Uhcatalog/src/test/e2e/templeton/tests/jobsubmission.conf
Ahcatalog/src/test/e2e/templeton/inpdir/hcatloadstore.pig
U
hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/CompleteDelegator.java
U
hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/JarDelegator.java
U
hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/PigDelegator.java
U
hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/HiveDelegator.java
U
hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/LauncherDelegator.java
A
hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/tool/DelegationTokenCache.java
U
hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/tool/TempletonControllerJob.java
U
hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/Server.java
U
hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/StreamingDelegator.java
U
hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/SecureProxySupport.java
Fetching external item into 'hcatalog/src/test/e2e/harness'
Updated external to revision 1533667.
Updated to revision 1533667.
+ patchCommandPath=/data/hive-ptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hive-ptest/working/scratch/build.patch
+ [[ -f /data/hive-ptest/working/scratch/build.patch ]]
+ chmod +x /data/hive-ptest/working/scratch/smart-apply-patch.sh
+ /data/hive-ptest/working/scratch/smart-apply-patch.sh
/data/hive-ptest/working/scratch/build.patch
The patch does not appear to apply with p0 to p2
+ exit 1
'
{noformat}
This message is automatically generated.
webhcat jobs that need to access metastore fails in secure mode
---
Key: HIVE-5133
URL: https://issues.apache.org/jira/browse/HIVE-5133
Project: Hive
Issue Type: Bug
Components: WebHCat
Affects Versions: 0.11.0
Reporter: Thejas M Nair
Assignee: Eugene Koifman
Fix For: 0.13.0
Attachments: HIVE-5133.1.patch, HIVE-5133.1.test.patch,
HIVE-5133.2.patch, HIVE-5133.3.patch, HIVE-5133.5.patch, HIVE-5133.6.patch
Webhcat job submission requests result in the pig/hive/mr job being run from
a map task that it
[jira] [Commented] (HIVE-5133) webhcat jobs that need to access metastore fails in secure mode
[ https://issues.apache.org/jira/browse/HIVE-5133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13799683#comment-13799683 ] Eugene Koifman commented on HIVE-5133: -- not sure what the issue is here. I just checkout https://github.com/apache/hive/commit/f7f32f2d9e5e29227ad800ed830d8f711b5dbd7a and applied this patch - it applies cleanly webhcat jobs that need to access metastore fails in secure mode --- Key: HIVE-5133 URL: https://issues.apache.org/jira/browse/HIVE-5133 Project: Hive Issue Type: Bug Components: WebHCat Affects Versions: 0.11.0 Reporter: Thejas M Nair Assignee: Eugene Koifman Fix For: 0.13.0 Attachments: HIVE-5133.1.patch, HIVE-5133.1.test.patch, HIVE-5133.2.patch, HIVE-5133.3.patch, HIVE-5133.5.patch, HIVE-5133.6.patch Webhcat job submission requests result in the pig/hive/mr job being run from a map task that it launches. In secure mode, for the pig/hive/mr job that is run to be authorized to perform actions on metastore, it has to have the delegation tokens from the hive metastore. In case of pig/MR job this is needed if hcatalog is being used in the script/job. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HIVE-5133) webhcat jobs that need to access metastore fails in secure mode
[ https://issues.apache.org/jira/browse/HIVE-5133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13799691#comment-13799691 ] Thejas M Nair commented on HIVE-5133: - Thats because the test applies patch to latest trunk. webhcat jobs that need to access metastore fails in secure mode --- Key: HIVE-5133 URL: https://issues.apache.org/jira/browse/HIVE-5133 Project: Hive Issue Type: Bug Components: WebHCat Affects Versions: 0.11.0 Reporter: Thejas M Nair Assignee: Eugene Koifman Fix For: 0.13.0 Attachments: HIVE-5133.1.patch, HIVE-5133.1.test.patch, HIVE-5133.2.patch, HIVE-5133.3.patch, HIVE-5133.5.patch, HIVE-5133.6.patch Webhcat job submission requests result in the pig/hive/mr job being run from a map task that it launches. In secure mode, for the pig/hive/mr job that is run to be authorized to perform actions on metastore, it has to have the delegation tokens from the hive metastore. In case of pig/MR job this is needed if hcatalog is being used in the script/job. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HIVE-5133) webhcat jobs that need to access metastore fails in secure mode
[ https://issues.apache.org/jira/browse/HIVE-5133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13799701#comment-13799701 ] Eugene Koifman commented on HIVE-5133: -- this link is the latest trunk (before Danie's checkin of this patch) webhcat jobs that need to access metastore fails in secure mode --- Key: HIVE-5133 URL: https://issues.apache.org/jira/browse/HIVE-5133 Project: Hive Issue Type: Bug Components: WebHCat Affects Versions: 0.11.0 Reporter: Thejas M Nair Assignee: Eugene Koifman Fix For: 0.13.0 Attachments: HIVE-5133.1.patch, HIVE-5133.1.test.patch, HIVE-5133.2.patch, HIVE-5133.3.patch, HIVE-5133.5.patch, HIVE-5133.6.patch Webhcat job submission requests result in the pig/hive/mr job being run from a map task that it launches. In secure mode, for the pig/hive/mr job that is run to be authorized to perform actions on metastore, it has to have the delegation tokens from the hive metastore. In case of pig/MR job this is needed if hcatalog is being used in the script/job. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HIVE-5133) webhcat jobs that need to access metastore fails in secure mode
[ https://issues.apache.org/jira/browse/HIVE-5133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13798646#comment-13798646 ] Thejas M Nair commented on HIVE-5133: - +1 webhcat jobs that need to access metastore fails in secure mode --- Key: HIVE-5133 URL: https://issues.apache.org/jira/browse/HIVE-5133 Project: Hive Issue Type: Bug Components: WebHCat Affects Versions: 0.11.0 Reporter: Thejas M Nair Assignee: Eugene Koifman Attachments: HIVE-5133.1.patch, HIVE-5133.1.test.patch, HIVE-5133.2.patch, HIVE-5133.3.patch, HIVE-5133.5.patch Webhcat job submission requests result in the pig/hive/mr job being run from a map task that it launches. In secure mode, for the pig/hive/mr job that is run to be authorized to perform actions on metastore, it has to have the delegation tokens from the hive metastore. In case of pig/MR job this is needed if hcatalog is being used in the script/job. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HIVE-5133) webhcat jobs that need to access metastore fails in secure mode
[
https://issues.apache.org/jira/browse/HIVE-5133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13796652#comment-13796652
]
Hive QA commented on HIVE-5133:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12608625/HIVE-5133.5.patch
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 4411 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_script_broken_pipe1
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1138/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1138/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests failed with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
webhcat jobs that need to access metastore fails in secure mode
---
Key: HIVE-5133
URL: https://issues.apache.org/jira/browse/HIVE-5133
Project: Hive
Issue Type: Bug
Components: WebHCat
Affects Versions: 0.11.0
Reporter: Thejas M Nair
Assignee: Eugene Koifman
Attachments: HIVE-5133.1.patch, HIVE-5133.1.test.patch,
HIVE-5133.2.patch, HIVE-5133.3.patch, HIVE-5133.5.patch
Webhcat job submission requests result in the pig/hive/mr job being run from
a map task that it launches. In secure mode, for the pig/hive/mr job that is
run to be authorized to perform actions on metastore, it has to have the
delegation tokens from the hive metastore.
In case of pig/MR job this is needed if hcatalog is being used in the
script/job.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HIVE-5133) webhcat jobs that need to access metastore fails in secure mode
[ https://issues.apache.org/jira/browse/HIVE-5133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13774727#comment-13774727 ] Eugene Koifman commented on HIVE-5133: -- HIVE-5133.3.patch has additional issue: org.apache.hive.hcatalog.templeton.tool.TempletonControllerJob and org.apache.hive.hcatalog.templeton.tool.MSTokenCleanOutputFormat are using org.apache.hcatalog.common.HCatUtil; they should be using org.apache.hive.hcatalog.common.HCatUtil webhcat jobs that need to access metastore fails in secure mode --- Key: HIVE-5133 URL: https://issues.apache.org/jira/browse/HIVE-5133 Project: Hive Issue Type: Bug Components: WebHCat Affects Versions: 0.11.0 Reporter: Thejas M Nair Assignee: Thejas M Nair Attachments: HIVE-5133.1.patch, HIVE-5133.1.test.patch, HIVE-5133.2.patch, HIVE-5133.3.patch Webhcat job submission requests result in the pig/hive/mr job being run from a map task that it launches. In secure mode, for the pig/hive/mr job that is run to be authorized to perform actions on metastore, it has to have the delegation tokens from the hive metastore. In case of pig/MR job this is needed if hcatalog is being used in the script/job. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5133) webhcat jobs that need to access metastore fails in secure mode
[ https://issues.apache.org/jira/browse/HIVE-5133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13765697#comment-13765697 ] Deepesh Khandelwal commented on HIVE-5133: -- [~thejas], your latest patch is missing the test artifact hcatalog/src/test/e2e/templeton/inpdir/hcatloadstore.pig. Can you please include that in your patch? webhcat jobs that need to access metastore fails in secure mode --- Key: HIVE-5133 URL: https://issues.apache.org/jira/browse/HIVE-5133 Project: Hive Issue Type: Bug Components: WebHCat Affects Versions: 0.11.0 Reporter: Thejas M Nair Assignee: Thejas M Nair Attachments: HIVE-5133.1.patch, HIVE-5133.1.test.patch, HIVE-5133.2.patch Webhcat job submission requests result in the pig/hive/mr job being run from a map task that it launches. In secure mode, for the pig/hive/mr job that is run to be authorized to perform actions on metastore, it has to have the delegation tokens from the hive metastore. In case of pig/MR job this is needed if hcatalog is being used in the script/job. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5133) webhcat jobs that need to access metastore fails in secure mode
[ https://issues.apache.org/jira/browse/HIVE-5133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13755052#comment-13755052 ] Deepesh Khandelwal commented on HIVE-5133: -- The current patch does fix the Pig but the MR job would fail to find missing HCat classes in classpath, see HIVE-5188. webhcat jobs that need to access metastore fails in secure mode --- Key: HIVE-5133 URL: https://issues.apache.org/jira/browse/HIVE-5133 Project: Hive Issue Type: Bug Components: WebHCat Affects Versions: 0.11.0 Reporter: Thejas M Nair Assignee: Thejas M Nair Attachments: HIVE-5133.1.patch Webhcat job submission requests result in the pig/hive/mr job being run from a map task that it launches. In secure mode, for the pig/hive/mr job that is run to be authorized to perform actions on metastore, it has to have the delegation tokens from the hive metastore. In case of pig/MR job this is needed if hcatalog is being used in the script/job. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5133) webhcat jobs that need to access metastore fails in secure mode
[ https://issues.apache.org/jira/browse/HIVE-5133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13753882#comment-13753882 ] Thejas M Nair commented on HIVE-5133: - Thanks for the feedback, I will create a new patch addressing these comments. I also need to add e2e tests. Note about the patch - With this change for submitting pig or MR jobs you need to specify usehcatalog=true as a POST param. (in curl command -d usehcatalog=true ). In case of pig this argument is option, it is sufficient that you have a arg='-useHCatalog' POST param. webhcat jobs that need to access metastore fails in secure mode --- Key: HIVE-5133 URL: https://issues.apache.org/jira/browse/HIVE-5133 Project: Hive Issue Type: Bug Components: WebHCat Affects Versions: 0.11.0 Reporter: Thejas M Nair Assignee: Thejas M Nair Attachments: HIVE-5133.1.patch Webhcat job submission requests result in the pig/hive/mr job being run from a map task that it launches. In secure mode, for the pig/hive/mr job that is run to be authorized to perform actions on metastore, it has to have the delegation tokens from the hive metastore. In case of pig/MR job this is needed if hcatalog is being used in the script/job. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-5133) webhcat jobs that need to access metastore fails in secure mode
[ https://issues.apache.org/jira/browse/HIVE-5133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13748882#comment-13748882 ] Eugene Koifman commented on HIVE-5133: -- 1. Server#pig() Server#mapReduceJar() - the JavaDoc lists all the parameters but no explanation of what each means. At the very least new params should be documented. Saying that it matches REST API isn’t really useful since http://hive.apache.org/docs/hcat_r0.5.0/pig.html doesn’t describe all of them, especially the parameter just added. 2. PigDelegator#hasPigArgUseHcat() - would be useful to add a comment about what -useHCatalog does (or a URL) 3. I think MSTokenCleanOutputFormat is missing from the patch 4. In TempletonControllerJob - wouldn’t be cleaner to pass a variable in TCJ c’tor() that says delegation token is needed rather than passing via args and having to remove it? 5. Why does TempletonControllerJob#run() use UserGroupInformation.getCurrentUser(), but TempletonControllerJob#buildHCatDeletgationToken() uses UgiFactory. Is there specific reason for this? 6. I think it would be really useful to add 10-20 lines of JavaDoc that explains why this whole thing is implemented/how it’s designed. It would be a big help for maintainers (and doc writes) webhcat jobs that need to access metastore fails in secure mode --- Key: HIVE-5133 URL: https://issues.apache.org/jira/browse/HIVE-5133 Project: Hive Issue Type: Bug Components: WebHCat Affects Versions: 0.11.0 Reporter: Thejas M Nair Assignee: Thejas M Nair Attachments: HIVE-5133.1.patch Webhcat job submission requests result in the pig/hive/mr job being run from a map task that it launches. In secure mode, for the pig/hive/mr job that is run to be authorized to perform actions on metastore, it has to have the delegation tokens from the hive metastore. In case of pig/MR job this is needed if hcatalog is being used in the script/job. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
