Hello,

I was wondering whether anyone has tried and/or had any luck creating a
custom catalog with Iceberg + Flink via the Python API (
https://iceberg.apache.org/flink/#custom-catalog)?

When doing so, the docs mention that dependencies need to be specified via
*pipeline.jars* / *pipeline.classpaths* (
https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/dev/python/dependency_management/).
I'm providing the iceberg flink runtime + the hadoop libs via those and I
can confirm that I'm landing in the *FlinkCatalogFactory* but then things
fail because it doesn't see the *hadoop dependencies* for some reason.

What would be the right way to provide the *HADOOP_CLASSPATH* when using
the Python API? I have a minimal code example that shows the issue here:
https://gist.github.com/nastra/92bc3bc7b7037d956aa5807988078b8d#file-flink-py-L38


Thanks

Reply via email to