Re: Flink Python API and HADOO_CLASSPATH

2021-05-18 Thread Eduard Tudenhoefner
Hi Dian, thanks a lot for the explanation and help. Option 2) is what I needed and it works. Regards Eduard On Tue, May 18, 2021 at 6:21 AM Dian Fu wrote: > Hi, > > 1) The cause of the exception: > The dependencies added via pipeline.jars / pipeline.classpaths will be > used to construct user

Re: Flink Python API and HADOO_CLASSPATH

2021-05-17 Thread Dian Fu
Hi, 1) The cause of the exception: The dependencies added via pipeline.jars / pipeline.classpaths will be used to construct user class loader. For your job, the exception happens when HadoopUtils.getHadoopConfiguration is called. The reason is that HadoopUtils is provided by Flink which is

Fwd: Flink Python API and HADOO_CLASSPATH

2021-05-17 Thread Eduard Tudenhoefner
Hello, I was wondering whether anyone has tried and/or had any luck creating a custom catalog with Iceberg + Flink via the Python API ( https://iceberg.apache.org/flink/#custom-catalog)? When doing so, the docs mention that dependencies need to be specified via *pipeline.jars* /