Hi,
currently our documentation for running quickstart examples uses maven
and `exec:java` mojo. Unfortunately, there is a (not yet confirmed, but
likely, see [1], [2]) Flink bug which causes Flink runner 1.18 to fail.
The bug causes MiniCluster (which is used in local testing) use wrong
classloader to load Flink's classes, resulting in
ClassNotFoundException. The only known workaround possible is using
`exec:exec` to fork new process having correct classpath right from the
start of the Java process (exec:java uses context classloader before
running user-core main() method). This can be done quite easily for
running post-commit tests (runQuickstartJavaFlinkLocal), but any user
who would try to run examples using the documented way (running mvn
exec:java) would run into the bug.
The question is - can we change the way we run examples, so that a new
JVM process with correctly set-up classpath can be spawn to run the user
code? What would be the best way of doing this? Otherwise we must either
fix the (probable) bug in Flink or release version 1.18 of Flink Runner
which will fail to run quickstart examples. More info in [3].
Thanks for any suggestions.
Jan
[1] https://www.mail-archive.com/user@flink.apache.org/msg52035.html
[2] https://lists.apache.org/thread/wdo4rz4q9qh0j11o5b76d45zlqkdx55n
[3] https://github.com/apache/beam/pull/31062
