[ 
https://issues.apache.org/jira/browse/HADOOP-19848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18069799#comment-18069799
 ] 

ASF GitHub Bot commented on HADOOP-19848:
-----------------------------------------

surendralilhore commented on PR #8365:
URL: https://github.com/apache/hadoop/pull/8365#issuecomment-4160323967

   @anujmodi2021, @ayushtkn 
   
   I think we should still commit this change, even if we are not planning a 
release for 3.4, since this code is no longer part of the active codebase in 
3.5 and trunk.
   
   Spark master is still using 3.4.3, so some users may encounter this issue. 
Keeping this change will serve as a useful reference for others.
   
   
https://github.com/apache/spark/blob/0419e68b7e7cdda443a44efa3abe251595af3e0a/pom.xml#L130




> AzureNativeFileSystemStore fails with NoSuchMethodError on Jetty 9.4.57+
> ------------------------------------------------------------------------
>
>                 Key: HADOOP-19848
>                 URL: https://issues.apache.org/jira/browse/HADOOP-19848
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs/azure
>    Affects Versions: 3.4.0
>            Reporter: Surendra Singh Lilhore
>            Assignee: Surendra Singh Lilhore
>            Priority: Major
>              Labels: pull-request-available
>
> When using hadoop-azure with Apache Spark 4.0, {{AzureNativeFileSystemStore}} 
> fails to load with:
>  
> {noformat}
> Caused by: java.lang.ExceptionInInitializerError: Exception 
> java.lang.NoSuchMethodError: 'java.util.Properties 
> org.eclipse.jetty.util.log.Log.getProperties()' [in thread "Thread-31"]
>       at 
> org.apache.hadoop.fs.azure.AzureNativeFileSystemStore.createPermissionJsonSerializer(AzureNativeFileSystemStore.java:429)
>       at 
> org.apache.hadoop.fs.azure.AzureNativeFileSystemStore.<clinit>(AzureNativeFileSystemStore.java:331)
>       at 
> org.apache.hadoop.fs.azure.NativeAzureFileSystem.createDefaultStore(NativeAzureFileSystem.java:1485)
>       at 
> org.apache.hadoop.fs.azure.NativeAzureFileSystem.initialize(NativeAzureFileSystem.java:1410)
>       at 
> org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3615){noformat}
>   *Root cause:*
>   {{AzureNativeFileSystemStore.createPermissionJsonSerializer()}} calls 
> {{org.eclipse.jetty.util.log.Log.getProperties()}} to suppress Jetty's log  
> announcement. While this method exists in {{jetty-util}} 9.4.57 (as shipped 
> with
>   Hadoop 3.4.x), Spark 4.0 bundles a different version of Jetty on the 
> runtime classpath where this method has been removed. This causes a 
> compile-vs-runtime classpath mismatch — the code compiles fine against 
> Hadoop's Jetty version but fails at runtime under Spark's classloader.
>   This is a fragile dependency on a Jetty internal API 
> ({{{}org.eclipse.jetty.util.log.Log{}}}) that is unnecessary. Jetty reads the 
> {{org.eclipse.jetty.util.log.announce}} property from system properties, so 
> {{System.setProperty() }}achieves the same effect without coupling to Jetty 
> internals.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to