Hangxiang Yu created FLINK-31366:
------------------------------------
Summary: Exception is thrown when s3a and s3p are used together
Key: FLINK-31366
URL: https://issues.apache.org/jira/browse/FLINK-31366
Project: Flink
Issue Type: Bug
Components: FileSystems
Reporter: Hangxiang Yu
Assignee: Hangxiang Yu
h3. Exception
When s3a and s3p plugins exist at the same time, an exception will be thrown as
below:
{code:java}
Caused by:java.lang.NoSuchMethodError:
org.apache.flink.fs.s3.common.AbstractS3FileSystemFactory.<init>
(Ljava/lang/String;Lorg/apache/flink/fs/s3presto/common/HadoopConfigLoader;)
{code}
h3. Why
In the construction method of AbstractS3FileSystemFactory, s3a shades
HadoopConfigLoader into org.apache.flink.fs.s3hadoop.common.HadoopConfigLoader,
and s3p shades HadoopConfigLoader into org.
apache.flink.fs.s3presto.common.HadoopConfigLoader when package.
(see shade-plugin)
But the AbstractS3FileSystemFactory class will only be loaded once when loading
even if there are two plugins.
So it may first uses s3a plugin to load AbstractS3FileSystemFactory, and the
construction method is loaded into AbstractS3FileSystemFactory(String name,
org.apache. flink.fs.s3hadoop.common.HadoopConfigLoader hadoopConfigLoader), at
this time s3p will be abnormal when it is constructed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
