[jira] [Work logged] (HADOOP-17675) LdapGroupsMapping$LdapSslSocketFactory ClassNotFoundException

2021-05-04 Thread ASF GitHub Bot (Jira) 


 [ 
https://issues.apache.org/jira/browse/HADOOP-17675?focusedWorklogId=592658=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-592658
 ]

ASF GitHub Bot logged work on HADOOP-17675:
---

Author: ASF GitHub Bot
Created on: 04/May/21 10:34
Start Date: 04/May/21 10:34
Worklog Time Spent: 10m 
  Work Description: sodonnel merged pull request #2965:
URL: https://github.com/apache/hadoop/pull/2965


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 592658)
Time Spent: 40m  (was: 0.5h)

> LdapGroupsMapping$LdapSslSocketFactory ClassNotFoundException
> -
>
> Key: HADOOP-17675
> URL: https://issues.apache.org/jira/browse/HADOOP-17675
> Project: Hadoop Common
>  Issue Type: Improvement
>  Components: common
>Affects Versions: 3.2.2
>Reporter: Tamas Mate
>Assignee: István Fajth
>Priority: Major
>  Labels: pull-request-available
> Attachments: stacktrace.txt
>
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> Using LdapGroupsMapping with SSL enabled causes ClassNotFoundException when 
> it is called through native threads, such as Apache Impala does.
> When a thread is attached to the VM, the currentThread's context classloader 
> is null, so when jndi internally tries to use the current thread's context 
> classloader to load the socket factory implementation, the 
> Class.forname(String, boolean, ClassLoader) method gets a null as the loader  
> uses the bootstrap classloader.
>  Meanwhile the LdapGroupsMapping class and the SslSocketFactory defined in it 
> is loaded by the application classloader from its classpath.
> As the bootstrap classloader does not have hadoop-common in its classpath, 
> when a native thread tries to use/load the LdapGroupsMapping class it can't 
> because the bootstrap loader can't load anything from hadoop-common. The 
> correct solution seems to be to set the currentThread's context classloader 
> to the classloader of LdapGroupsMapping class before initializing the jndi 
> internals, and then reset to the original value after, with that we can 
> ensure that the behaviour of other things does not change, but this failure 
> can be avoided as well.
> Attached the complete stacktrace to this Jira.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Work logged] (HADOOP-17675) LdapGroupsMapping$LdapSslSocketFactory ClassNotFoundException

2021-04-30 Thread ASF GitHub Bot (Jira) 


 [ 
https://issues.apache.org/jira/browse/HADOOP-17675?focusedWorklogId=591596=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-591596
 ]

ASF GitHub Bot logged work on HADOOP-17675:
---

Author: ASF GitHub Bot
Created on: 30/Apr/21 15:47
Start Date: 30/Apr/21 15:47
Worklog Time Spent: 10m 
  Work Description: hadoop-yetus commented on pull request #2965:
URL: https://github.com/apache/hadoop/pull/2965#issuecomment-830187086


   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |  22m 18s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  0s |  |  codespell was not available.  |
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain 
any @author tags.  |
   | -1 :x: |  test4tests  |   0m  0s |  |  The patch doesn't appear to include 
any new or modified tests. Please justify why no new tests are needed for this 
patch. Also please list what manual steps were performed to verify this patch.  
|
    _ trunk Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |  32m 26s |  |  trunk passed  |
   | +1 :green_heart: |  compile  |  22m 35s |  |  trunk passed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04  |
   | +1 :green_heart: |  compile  |  19m  5s |  |  trunk passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10  |
   | +1 :green_heart: |  checkstyle  |   0m 59s |  |  trunk passed  |
   | +1 :green_heart: |  mvnsite  |   1m 27s |  |  trunk passed  |
   | +1 :green_heart: |  javadoc  |   0m 57s |  |  trunk passed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04  |
   | +1 :green_heart: |  javadoc  |   1m 31s |  |  trunk passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10  |
   | +1 :green_heart: |  spotbugs  |   2m 21s |  |  trunk passed  |
   | +1 :green_heart: |  shadedclient  |  18m 18s |  |  branch has no errors 
when building and testing our client artifacts.  |
    _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   0m 55s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |  24m 39s |  |  the patch passed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04  |
   | +1 :green_heart: |  javac  |  24m 39s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |  20m 55s |  |  the patch passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10  |
   | +1 :green_heart: |  javac  |  20m 55s |  |  the patch passed  |
   | +1 :green_heart: |  blanks  |   0m  0s |  |  The patch has no blanks 
issues.  |
   | +1 :green_heart: |  checkstyle  |   0m 58s |  |  the patch passed  |
   | +1 :green_heart: |  mvnsite  |   1m 24s |  |  the patch passed  |
   | +1 :green_heart: |  javadoc  |   0m 56s |  |  the patch passed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04  |
   | +1 :green_heart: |  javadoc  |   1m 27s |  |  the patch passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10  |
   | +1 :green_heart: |  spotbugs  |   2m 29s |  |  the patch passed  |
   | +1 :green_heart: |  shadedclient  |  18m 18s |  |  patch has no errors 
when building and testing our client artifacts.  |
    _ Other Tests _ |
   | +1 :green_heart: |  unit  |  17m 21s |  |  hadoop-common in the patch 
passed.  |
   | +1 :green_heart: |  asflicense  |   0m 46s |  |  The patch does not 
generate ASF License warnings.  |
   |  |   | 211m 55s |  |  |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2965/2/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hadoop/pull/2965 |
   | Optional Tests | dupname asflicense compile javac javadoc mvninstall 
mvnsite unit shadedclient spotbugs checkstyle codespell |
   | uname | Linux 39b51fbd677c 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 
05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/bin/hadoop.sh |
   | git revision | trunk / b94a4e0c908c8904aecde6021a1622cd96350e9a |
   | Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
   | Multi-JDK versions | 
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 
/usr/lib/jvm/java-8-openjdk-amd64:Private 
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
   |  Test Results | 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2965/2/testReport/ |
   | Max. process+thread count | 3135 (vs. ulimit of 5500) |
   | modules | C: hadoop-common-project/hadoop-common U: 
hadoop-common-project/hadoop-common |
   | Console output | 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2965/2/console |
   |

[jira] [Work logged] (HADOOP-17675) LdapGroupsMapping$LdapSslSocketFactory ClassNotFoundException

2021-04-29 Thread ASF GitHub Bot (Jira) 


 [ 
https://issues.apache.org/jira/browse/HADOOP-17675?focusedWorklogId=591350=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-591350
 ]

ASF GitHub Bot logged work on HADOOP-17675:
---

Author: ASF GitHub Bot
Created on: 30/Apr/21 03:28
Start Date: 30/Apr/21 03:28
Worklog Time Spent: 10m 
  Work Description: hadoop-yetus commented on pull request #2965:
URL: https://github.com/apache/hadoop/pull/2965#issuecomment-829780056


   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |  21m 39s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  0s |  |  codespell was not available.  |
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain 
any @author tags.  |
   | -1 :x: |  test4tests  |   0m  0s |  |  The patch doesn't appear to include 
any new or modified tests. Please justify why no new tests are needed for this 
patch. Also please list what manual steps were performed to verify this patch.  
|
    _ trunk Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |  33m 13s |  |  trunk passed  |
   | +1 :green_heart: |  compile  |  22m 26s |  |  trunk passed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04  |
   | +1 :green_heart: |  compile  |  19m  5s |  |  trunk passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10  |
   | +1 :green_heart: |  checkstyle  |   0m 58s |  |  trunk passed  |
   | +1 :green_heart: |  mvnsite  |   1m 30s |  |  trunk passed  |
   | +1 :green_heart: |  javadoc  |   0m 57s |  |  trunk passed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04  |
   | +1 :green_heart: |  javadoc  |   1m 29s |  |  trunk passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10  |
   | +1 :green_heart: |  spotbugs  |   2m 21s |  |  trunk passed  |
   | +1 :green_heart: |  shadedclient  |  18m 17s |  |  branch has no errors 
when building and testing our client artifacts.  |
    _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   0m 53s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |  21m 42s |  |  the patch passed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04  |
   | +1 :green_heart: |  javac  |  21m 42s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |  18m 56s |  |  the patch passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10  |
   | +1 :green_heart: |  javac  |  18m 56s |  |  the patch passed  |
   | +1 :green_heart: |  blanks  |   0m  0s |  |  The patch has no blanks 
issues.  |
   | +1 :green_heart: |  checkstyle  |   0m 57s |  |  the patch passed  |
   | +1 :green_heart: |  mvnsite  |   1m 26s |  |  the patch passed  |
   | +1 :green_heart: |  javadoc  |   0m 55s |  |  the patch passed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04  |
   | +1 :green_heart: |  javadoc  |   1m 32s |  |  the patch passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10  |
   | -1 :x: |  spotbugs  |   2m 31s | 
[/new-spotbugs-hadoop-common-project_hadoop-common.html](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2965/1/artifact/out/new-spotbugs-hadoop-common-project_hadoop-common.html)
 |  hadoop-common-project/hadoop-common generated 1 new + 0 unchanged - 0 fixed 
= 1 total (was 0)  |
   | +1 :green_heart: |  shadedclient  |  18m 24s |  |  patch has no errors 
when building and testing our client artifacts.  |
    _ Other Tests _ |
   | -1 :x: |  unit  |  17m 44s | 
[/patch-unit-hadoop-common-project_hadoop-common.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2965/1/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt)
 |  hadoop-common in the patch passed.  |
   | +1 :green_heart: |  asflicense  |   0m 48s |  |  The patch does not 
generate ASF License warnings.  |
   |  |   | 207m 26s |  |  |
   
   
   | Reason | Tests |
   |---:|:--|
   | SpotBugs | module:hadoop-common-project/hadoop-common |
   |  |  Load of known null value in 
org.apache.hadoop.security.LdapGroupsMapping.getDirContext()  At 
LdapGroupsMapping.java:in 
org.apache.hadoop.security.LdapGroupsMapping.getDirContext()  At 
LdapGroupsMapping.java:[line 664] |
   | Failed junit tests | hadoop.security.ssl.TestReloadingX509TrustManager |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2965/1/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hadoop/pull/2965 |
   | Optional Tests | dupname asflicense compile javac javadoc mvninstall 
mvnsite unit shadedclient spotbugs checkstyle codespell |
   | uname | Linux 4042e2ffc2aa

[jira] [Work logged] (HADOOP-17675) LdapGroupsMapping$LdapSslSocketFactory ClassNotFoundException

2021-04-29 Thread ASF GitHub Bot (Jira) 


 [ 
https://issues.apache.org/jira/browse/HADOOP-17675?focusedWorklogId=591314=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-591314
 ]

ASF GitHub Bot logged work on HADOOP-17675:
---

Author: ASF GitHub Bot
Created on: 29/Apr/21 23:59
Start Date: 29/Apr/21 23:59
Worklog Time Spent: 10m 
  Work Description: fapifta opened a new pull request #2965:
URL: https://github.com/apache/hadoop/pull/2965


   As stated in this article:
   
https://www.infoworld.com/article/2077344/find-a-way-out-of-the-classloader-maze.html
   
   A native thread has its context classloader set to null by default.
   If the context classloader which is used internally by JNDI to load a class 
is
   null, then the bootstrap classloader is used, according to the apidoc here:
   
https://docs.oracle.com/javase/8/docs/api/java/lang/Class.html#forName-java.lang.String-boolean-java.lang.ClassLoader-
   
   JNDI uses this form with the context classloader as can be seen here:
   
https://github.com/openjdk/jdk11u/blob/master/src/java.naming/share/classes/com/sun/jndi/ldap/VersionHelper.java#L107
   or here:
   
https://github.com/openjdk/jdk8u/blob/master/jdk/src/share/classes/com/sun/jndi/ldap/VersionHelper12.java#L72
   
   In Impala this call happens from a Thread created in native space, so in that
   case, the System/Application classloader loads LdapSslSocketFactory fine in
   LdapGroupsMapping.getDirContext() while creating the environment, but then
   InitialDirContext constructor gets to instantiation the LdapSslSocketFactory
   inside JNDI with the help of the linked VersionHelper impelmentations,
   and fails to load the class with the bootstrap classloader as context
   classloader is null.
   
   In order to solve this problem, we can safely use the classloader of the
   LdapGroupsMapping class, as it had to load the LdapSslSocketFactory class
   before.
   
   ## NOTICE
   
   Please create an issue in ASF JIRA before opening a pull request,
   and you need to set the title of the pull request which starts with
   the corresponding JIRA issue number. (e.g. HADOOP-X. Fix a typo in YYY.)
   For more details, please see 
https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 591314)
Remaining Estimate: 0h
Time Spent: 10m

> LdapGroupsMapping$LdapSslSocketFactory ClassNotFoundException
> -
>
> Key: HADOOP-17675
> URL: https://issues.apache.org/jira/browse/HADOOP-17675
> Project: Hadoop Common
>  Issue Type: Improvement
>  Components: common
>Affects Versions: 3.2.2
>Reporter: Tamas Mate
>Assignee: István Fajth
>Priority: Major
> Attachments: stacktrace.txt
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Using LdapGroupsMapping with SSL enabled causes ClassNotFoundException when 
> it is called through native threads, such as Apache Impala does.
> When a thread is attached to the VM, the currentThread's context classloader 
> is null, so when jndi internally tries to use the current thread's context 
> classloader to load the socket factory implementation, the 
> Class.forname(String, boolean, ClassLoader) method gets a null as the loader  
> uses the bootstrap classloader.
>  Meanwhile the LdapGroupsMapping class and the SslSocketFactory defined in it 
> is loaded by the application classloader from its classpath.
> As the bootstrap classloader does not have hadoop-common in its classpath, 
> when a native thread tries to use/load the LdapGroupsMapping class it can't 
> because the bootstrap loader can't load anything from hadoop-common. The 
> correct solution seems to be to set the currentThread's context classloader 
> to the classloader of LdapGroupsMapping class before initializing the jndi 
> internals, and then reset to the original value after, with that we can 
> ensure that the behaviour of other things does not change, but this failure 
> can be avoided as well.
> Attached the complete stacktrace to this Jira.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org