Re: Flink write ADLS show error: No FileSystem for scheme "file"

2024-07-03 Thread Xiao Xu
Hi, Gabor, I'm curious about why this happened in Azure file and not in other file format(I tried use s3 and it works OK) Gabor Somogyi 于2024年7月2日周二 16:59写道: > I see, thanks for sharing. > > The change what you've made makes sense. Let me explain the details. > Each and every plugin has it's

Re: Flink write ADLS show error: No FileSystem for scheme "file"

2024-07-02 Thread Gabor Somogyi
I see, thanks for sharing. The change what you've made makes sense. Let me explain the details. Each and every plugin has it's own class loader. The reason behind that is to avoid dependency collision with Flink's main class loader. I think if the mentioned change works when it's added as normal

Re: Flink write ADLS show error: No FileSystem for scheme "file"

2024-06-26 Thread Xiao Xu
Hi, Gabar, Thanks to reply, I make sure that not conflict in maven, all the hadoop dependency is in provided scope, and checked my result jar it not contains (src/main/resources/META-INF/services). This is my pom: http://maven.apache.org/POM/4.0.0;

Re: Flink write ADLS show error: No FileSystem for scheme "file"

2024-06-26 Thread Gabor Somogyi
Hi Xiao, I'm not quite convinced that the azure plugin ruined your workload, I would take a look at the dependency graph you've in the pom. Adding multiple deps can conflict in terms of class loader services (src/main/resources/META-INF/services). As an example you've 2 such dependencies where