[jira] [Commented] (HADOOP-16163) NPE in setup/teardown of ITestAbfsDelegationTokens
[
https://issues.apache.org/jira/browse/HADOOP-16163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16784500#comment-16784500
]
Hudson commented on HADOOP-16163:
-
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #16127 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/16127/])
HADOOP-16163. NPE in setup/teardown of ITestAbfsDelegationTokens. (stevel: rev
05df151d09b8e02e08cfacb3970355fa69a50699)
* (edit) hadoop-tools/hadoop-azure/pom.xml
> NPE in setup/teardown of ITestAbfsDelegationTokens
> --
>
> Key: HADOOP-16163
> URL: https://issues.apache.org/jira/browse/HADOOP-16163
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure, test
>Affects Versions: 3.3.0
>Reporter: Steve Loughran
>Assignee: Steve Loughran
>Priority: Major
> Fix For: 3.3.0
>
> Attachments: HADOOP-16163-001.patch
>
>
> Managed to create this in a local (variant) branch:
> {Code}
> [ERROR] Errors:
> [ERROR]
> org.apache.hadoop.fs.azurebfs.extensions.ITestAbfsDelegationTokens.testCanonicalization(org.apache.hadoop.fs.azurebfs.extensions.ITestAbfsDelegationTokens)
> [ERROR] Run 1: ITestAbfsDelegationTokens.setup:107 » NullPointer
> [ERROR] Run 2: ITestAbfsDelegationTokens.teardown:130 » NullPointer
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16163) NPE in setup/teardown of ITestAbfsDelegationTokens
[
https://issues.apache.org/jira/browse/HADOOP-16163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16784468#comment-16784468
]
Steve Loughran commented on HADOOP-16163:
-
tested locally, all good: +1
I still think removing the need for security to be on for the DT binding to go
live makes sense long term
Even if the DT issuing mechanism requires kerberos to be enabled, at the far
end, in the cluster, the nodes will be authenticating on DTs themselves, and
yet should still renable the DT support so that if asked for a token they can
relay the existing one. Of course, that does require the DT plugin to know how
to do that
> NPE in setup/teardown of ITestAbfsDelegationTokens
> --
>
> Key: HADOOP-16163
> URL: https://issues.apache.org/jira/browse/HADOOP-16163
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure, test
>Affects Versions: 3.3.0
>Reporter: Steve Loughran
>Assignee: Steve Loughran
>Priority: Major
> Attachments: HADOOP-16163-001.patch
>
>
> Managed to create this in a local (variant) branch:
> {Code}
> [ERROR] Errors:
> [ERROR]
> org.apache.hadoop.fs.azurebfs.extensions.ITestAbfsDelegationTokens.testCanonicalization(org.apache.hadoop.fs.azurebfs.extensions.ITestAbfsDelegationTokens)
> [ERROR] Run 1: ITestAbfsDelegationTokens.setup:107 » NullPointer
> [ERROR] Run 2: ITestAbfsDelegationTokens.teardown:130 » NullPointer
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16163) NPE in setup/teardown of ITestAbfsDelegationTokens
[
https://issues.apache.org/jira/browse/HADOOP-16163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16784132#comment-16784132
]
Da Zhou commented on HADOOP-16163:
--
This issue can be fixed by moving the test to non-parallel execution task.
> NPE in setup/teardown of ITestAbfsDelegationTokens
> --
>
> Key: HADOOP-16163
> URL: https://issues.apache.org/jira/browse/HADOOP-16163
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure, test
>Affects Versions: 3.3.0
>Reporter: Steve Loughran
>Assignee: Steve Loughran
>Priority: Major
> Attachments: HADOOP-16163-001.patch
>
>
> Managed to create this in a local (variant) branch:
> {Code}
> [ERROR] Errors:
> [ERROR]
> org.apache.hadoop.fs.azurebfs.extensions.ITestAbfsDelegationTokens.testCanonicalization(org.apache.hadoop.fs.azurebfs.extensions.ITestAbfsDelegationTokens)
> [ERROR] Run 1: ITestAbfsDelegationTokens.setup:107 » NullPointer
> [ERROR] Run 2: ITestAbfsDelegationTokens.teardown:130 » NullPointer
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16163) NPE in setup/teardown of ITestAbfsDelegationTokens
[
https://issues.apache.org/jira/browse/HADOOP-16163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16783508#comment-16783508
]
Steve Loughran commented on HADOOP-16163:
-
Looking at this, I'd point the blame clearly at whatever the test is doing
regarding kerberos minicluster setup and reset.
Why do we do that? Mainly because the ABFS client will only serve up DTs if
kerberos is enabled.
It doesn't have to do that. HDFS does it because it is relying on Kerberos for
auth. If ABFS has a DT provider which can serve up OAuth tokens, then it can do
this even while Kerberos is off, and it radically simplifies testing, as
there's no need to do things with kerberos miniclusters.
Note: MR and Spark will still only collect DTs if kerberos is enabled, but they
are free to change that behaviour.
Is there a risk of problems if we disable DT issuing? Only in the case of: A DT
provider was declared that is only expected to run if kerberos is enabled. Any
provider like that would have to check for kerberos running and downgrade
-though really you'd want the cluster setup to be consistent here.
> NPE in setup/teardown of ITestAbfsDelegationTokens
> --
>
> Key: HADOOP-16163
> URL: https://issues.apache.org/jira/browse/HADOOP-16163
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure, test
>Affects Versions: 3.3.0
>Reporter: Steve Loughran
>Assignee: Steve Loughran
>Priority: Major
>
> Managed to create this in a local (variant) branch:
> {Code}
> [ERROR] Errors:
> [ERROR]
> org.apache.hadoop.fs.azurebfs.extensions.ITestAbfsDelegationTokens.testCanonicalization(org.apache.hadoop.fs.azurebfs.extensions.ITestAbfsDelegationTokens)
> [ERROR] Run 1: ITestAbfsDelegationTokens.setup:107 » NullPointer
> [ERROR] Run 2: ITestAbfsDelegationTokens.teardown:130 » NullPointer
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16163) NPE in setup/teardown of ITestAbfsDelegationTokens
[
https://issues.apache.org/jira/browse/HADOOP-16163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16783496#comment-16783496
]
Steve Loughran commented on HADOOP-16163:
-
{code}
[ERROR] Tests run: 9, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 1.086 s
<<< FAILURE! - in
org.apache.hadoop.fs.azurebfs.extensions.ITestAbfsDelegationTokens
[ERROR]
testCanonicalization(org.apache.hadoop.fs.azurebfs.extensions.ITestAbfsDelegationTokens)
Time elapsed: 0.075 s <<< ERROR!
java.lang.NullPointerException
at
org.apache.hadoop.security.UserGroupInformation.doSubjectLogin(UserGroupInformation.java:1836)
at
org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytabAndReturnUGI(UserGroupInformation.java:1215)
at
org.apache.hadoop.fs.azurebfs.extensions.KerberizedAbfsCluster.createAliceUser(KerberizedAbfsCluster.java:131)
at
org.apache.hadoop.fs.azurebfs.extensions.ITestAbfsDelegationTokens.setup(ITestAbfsDelegationTokens.java:107)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
at
org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
[ERROR]
testCanonicalization(org.apache.hadoop.fs.azurebfs.extensions.ITestAbfsDelegationTokens)
Time elapsed: 0 s <<< ERROR!
java.lang.NullPointerException
at
org.apache.hadoop.security.UserGroupInformation.doSubjectLogin(UserGroupInformation.java:1836)
at
org.apache.hadoop.security.UserGroupInformation.createLoginUser(UserGroupInformation.java:710)
at
org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:660)
at
org.apache.hadoop.fs.azurebfs.extensions.ITestAbfsDelegationTokens.teardown(ITestAbfsDelegationTokens.java:130)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
at
org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
{code}
> NPE in setup/teardown of ITestAbfsDelegationTokens
> --
>
> Key: HADOOP-16163
> URL: https://issues.apache.org/jira/browse/HADOOP-16163
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure, test
>Affects Versions: 3.3.0
>Reporter: Steve Loughran
>Assignee: Steve Loughran
>Priority: Major
>
> Managed to create this in a local (variant) branch:
> {Code}
> [ERROR] Errors:
> [ERROR]
> org.apache.hadoop.fs.azurebfs.extensions.ITestAbfsDelegationTokens.testCanonicalization(org.apache.hadoop.fs.azurebfs.extensions.ITestAbfsDelegationTokens)
> [ERROR] Run 1: ITestAbfsDelegationTokens.setup:107 » NullPointer
> [ERROR] Run 2: ITestAbfsDelegationTokens.teardown:130 » NullPointer
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
