[ https://issues.apache.org/jira/browse/FLINK-17989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chesnay Schepler closed FLINK-17989. ------------------------------------ Fix Version/s: (was: 1.11.1) (was: 1.10.1) (was: 1.9.3) (was: 1.11.0) Resolution: Duplicate > java.lang.NoClassDefFoundError: > org/apache/flink/fs/azure/common/hadoop/HadoopRecoverableWriter > ----------------------------------------------------------------------------------------------- > > Key: FLINK-17989 > URL: https://issues.apache.org/jira/browse/FLINK-17989 > Project: Flink > Issue Type: Bug > Components: Connectors / FileSystem, FileSystems > Affects Versions: 1.9.3, 1.10.1, 1.11.0, 1.11.1 > Environment: Ubuntu 18 > Java 1.8 > Flink 1.9.x and 1.10.x > Reporter: Israel Ekpo > Priority: Critical > > In the POM.xml classes from certain packages are relocated and filtered out > of the final jar In the POM.xml classes from certain packages are relocated > and filtered out of the final jar > This is causing errors for customers and users that are using the > StreamingFileSink with Azure Blob Storage in Flink versions 1.9.x, 1.10.x and > possibly 1.11.x > [https://github.com/apache/flink/blob/release-1.9/flink-filesystems/flink-azure-fs-hadoop/pom.xml#L170|https://github.com/apache/flink/blob/release-1.9/flink-filesystems/flink-azure-fs-hadoop/pom.xml#L170https://github.com/apache/flink/blob/release-1.9/flink-filesystems/flink-fs-hadoop-shaded/pom.xml#L233] > [https://github.com/apache/flink/blob/release-1.9/flink-filesystems/flink-fs-hadoop-shaded/pom.xml#L233|https://github.com/apache/flink/blob/release-1.9/flink-filesystems/flink-azure-fs-hadoop/pom.xml#L170https://github.com/apache/flink/blob/release-1.9/flink-filesystems/flink-fs-hadoop-shaded/pom.xml#L233] > > I would like to know why the relocation is happening and the reasoning behind > the exclusion and filtering of the classes > It seems to affect just the Azure file systems in my sample implementations > > {code:java} > String outputPath = > "wasbs://contai...@account.blob.core.windows.net/streaming-output/"; > final StreamingFileSink<Integer> sink = StreamingFileSink .forRowFormat(new > Path(outputPath), new SimpleStringEncoder<Integer>("UTF-8")) .build(); > stream.addSink(sink); > // execute programenv.execute(StreamingJob.class.getCanonicalName()); > {code} > {code:java} > //Exception Details Below > 2020-05-27 17:23:16java.lang.NoClassDefFoundError: > org/apache/flink/fs/azure/common/hadoop/HadoopRecoverableWriter at > org.apache.flink.fs.azure.common.hadoop.HadoopFileSystem.createRecoverableWriter(HadoopFileSystem.java:202) > at > org.apache.flink.core.fs.SafetyNetWrapperFileSystem.createRecoverableWriter(SafetyNetWrapperFileSystem.java:69) > at > org.apache.flink.streaming.api.functions.sink.filesystem.Buckets.<init>(Buckets.java:112) > at > org.apache.flink.streaming.api.functions.sink.filesystem.StreamingFileSink$RowFormatBuilder.createBuckets(StreamingFileSink.java:242) > at > org.apache.flink.streaming.api.functions.sink.filesystem.StreamingFileSink.initializeState(StreamingFileSink.java:327) > at > org.apache.flink.streaming.util.functions.StreamingFunctionUtils.tryRestoreFunction(StreamingFunctionUtils.java:178) > at > org.apache.flink.streaming.util.functions.StreamingFunctionUtils.restoreFunctionState(StreamingFunctionUtils.java:160) > at > org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.initializeState(AbstractUdfStreamOperator.java:96) > at > org.apache.flink.streaming.api.operators.AbstractStreamOperator.initializeState(AbstractStreamOperator.java:281) > at > org.apache.flink.streaming.runtime.tasks.StreamTask.initializeState(StreamTask.java:881) > at > org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:395) > at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:705) at > org.apache.flink.runtime.taskmanager.Task.run(Task.java:530) at > java.lang.Thread.run(Thread.java:748)Caused by: > java.lang.ClassNotFoundException: > org.apache.flink.fs.azure.common.hadoop.HadoopRecoverableWriter at > java.net.URLClassLoader.findClass(URLClassLoader.java:382) at > java.lang.ClassLoader.loadClass(ClassLoader.java:418) at > org.apache.flink.util.ChildFirstClassLoader.loadClass(ChildFirstClassLoader.java:60) > at java.lang.ClassLoader.loadClass(ClassLoader.java:351){code} -- This message was sent by Atlassian Jira (v8.3.4#803005)