Hi Timo, Thanks for the reply, I filed https://issues.apache.org/jira/browse/FLINK-24687
The file connector dependency is via flink-table-common and there are main source references (see below). Thomas [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.8.0:compile (default-compile) on project flink-table-common: Compilation failure: Compilation failure: [ERROR] /Users/thomas/src/flink/flink-table/flink-table-common/src/main/java/org/apache/flink/table/descriptors/FileSystemValidator.java:[22,44] package org.apache.flink.connector.file.sink does not exist [ERROR] /Users/thomas/src/flink/flink-table/flink-table-common/src/main/java/org/apache/flink/table/descriptors/FileSystemValidator.java:[23,43] package org.apache.flink.connector.file.src does not exist [ERROR] /Users/thomas/src/flink/flink-table/flink-table-common/src/main/java/org/apache/flink/table/factories/BulkReaderFormatFactory.java:[23,43] package org.apache.flink.connector.file.src does not exist [ERROR] /Users/thomas/src/flink/flink-table/flink-table-common/src/main/java/org/apache/flink/table/factories/BulkReaderFormatFactory.java:[24,50] package org.apache.flink.connector.file.src.reader does not exist [ERROR] /Users/thomas/src/flink/flink-table/flink-table-common/src/main/java/org/apache/flink/table/factories/BulkReaderFormatFactory.java:[35,39] cannot find symbol [ERROR] symbol: class BulkFormat [ERROR] /Users/thomas/src/flink/flink-table/flink-table-common/src/main/java/org/apache/flink/table/factories/BulkReaderFormatFactory.java:[35,59] cannot find symbol [ERROR] symbol: class FileSourceSplit [ERROR] /Users/thomas/src/flink/flink-table/flink-table-common/src/main/java/org/apache/flink/table/connector/format/BulkDecodingFormat.java:[22,43] package org.apache.flink.connector.file.src does not exist [ERROR] /Users/thomas/src/flink/flink-table/flink-table-common/src/main/java/org/apache/flink/table/connector/format/BulkDecodingFormat.java:[23,50] package org.apache.flink.connector.file.src.reader does not exist [ERROR] /Users/thomas/src/flink/flink-table/flink-table-common/src/main/java/org/apache/flink/table/connector/format/BulkDecodingFormat.java:[31,63] cannot find symbol [ERROR] symbol: class BulkFormat [ERROR] /Users/thomas/src/flink/flink-table/flink-table-common/src/main/java/org/apache/flink/table/connector/format/BulkDecodingFormat.java:[31,77] cannot find symbol [ERROR] symbol: class FileSourceSplit On Tue, Oct 26, 2021 at 5:13 AM Timo Walther <twal...@apache.org> wrote: > > Hi Thomas, > > thanks for your feedback. > > The error that you are experiencing is definitely a bug in 1.13.3 and > the missing method should be reintroduced in the next patch version to > make code compiled against older patch versions run again. > > Regarding the discussion points: > > I agree that flink-table-blink uber jar should not contain a dependency > to flink-connector-base. Even filesystem connectors should be optional > and put in a dedicated module that is not in /lib by default. > > With having the flink-table-blink uber jar in /lib we would like to > improve the SQL experience as this API is as important as the DataStream > API nowadays. But the depenencies should be minimal nevertheless. > > Regards, > Timo > > > On 23.10.21 00:59, Thomas Weise wrote: > > Hi, > > > > As part of upgrading to Flink 1.13.3 from 1.13.2 we run into the > > following problem with KafkaSource (Flink distribution is 1.13.2 and > > the application was built with 1.13.3): > > > > java.lang.NoSuchMethodError: 'void > > org.apache.flink.connector.base.source.reader.fetcher.SingleThreadFetcherManager.<init>(org.apache.flink.connector.base.source.reader.synchronization.FutureCompletingBlockingQueue, > > java.util.function.Supplier, java.util.function.Consumer)' > > at > > org.apache.flink.connector.kafka.source.reader.fetcher.KafkaSourceFetcherManager.<init>(KafkaSourceFetcherManager.java:67) > > at > > org.apache.flink.connector.kafka.source.KafkaSource.createReader(KafkaSource.java:160) > > at > > org.apache.flink.connector.kafka.source.KafkaSource.createReader(KafkaSource.java:127) > > > > It turns out that flink-table-blink_2.12-1.13.2.jar contains > > flink-connector-base and because that jar is under lib the 1.13.2 > > connector base gets picked up instead of the one bundled in the > > application jar. > > > > (The constructor in > > org.apache.flink.connector.base.source.reader.fetcher.SingleThreadFetcherManager > > was added in 1.13.3.) > > > > There are a few points I would like to discuss: > > > > 1) Version compatibility: A *patch* version should ideally not > > introduce such a change, it should be forward and backward compatible. > > Hopefully this will be the case after 1.14 with stable source API. > > 2) flink-table-blink - if it is meant to be self contained and usable > > as a library - should not leak its shaded dependencies. It contains > > FileSource and other deps from Flink, can those be relocated? > > 3) Do we need flink-table-blink under lib? Can it be bundled with the > > application instead? It would be great if the dependencies under lib > > are strictly Flink core. > > > > Thanks, > > Thomas > > >