Hi!
I'm trying to just get my bearings with dataproc and flink/beam. When
trying to run the wordcount example
<https://cloud.google.com/dataproc/docs/concepts/components/flink#portable_beam_jobs>
with
a long-running YARN sessions
<https://cloud.google.com/dataproc/docs/concepts/components/flink>. The
error looks like:
Traceback (most recent call last):
File "wordcount.py", line 99, in <module>
run()
File "wordcount.py", line 94, in run
output | 'Write' >> WriteToText(known_args.output)
File
"/home/jtran/.local/lib/python3.8/site-packages/apache_beam/pipeline.py",
line 586, in __exit__
self.result.wait_until_finish()
File
"/home/jtran/.local/lib/python3.8/site-packages/apache_beam/runners/portability/portable_runner.py",
line 599, in wait_until_finish
raise self._runtime_exception
RuntimeError: Pipeline
BeamApp-jtran-0708141941-5fcff870_effe0238-2afb-4707-a981-133bc46618fd
failed in state FAILED: java.util.ServiceConfigurationError:
com.fasterxml.jackson.databind.Module: Provider
com.fasterxml.jackson.module.jaxb.JaxbAnnotationModule not a subtype
flink version 1.12, dataproc image 2.0, beam version 2.30
I found some mention of this when trying to run the example with EMR but no
mention with dataproc (https://issues.apache.org/jira/browse/BEAM-10430).
The workaround is unclear to me, is there something wrong I'm doing with
setting this up? Any advice would be greatly appreciated. Thanks in advance!
