Hello,
I'm having issues editing the default Flink memory settings. I'm attempting to
run a Flink task on a cluster at scale. The log shows my edited config settings
having been read into the program, but they're having no effect. Here's the
trace:
17/06/05 23:45:41 INFO flink.FlinkRunner: Start
Hi Will,
How are you starting your cluster/executing your program? From what I can
gather you are using Beam, is that right? The line about the FlinkMiniCluster
seems strange because this would hint at the fact that the Runner is trying to
execute in local mode.
Best,
Aljoscha
> On 6. Jun 201
Aljoscha,
You're correct that I'm using Beam. Here's the shell script I'm using to start
the job:
hadoop jar **path to jar file** org.apache.beam.examples.complete.TfIdf
\--runner=FlinkRunner \--input= **path to directory** \--output=tfIdf
There are two lines before that to set the Flink config p
Hi Will,
I’m afraid that will simply run your program in Flink local mode, where the
configuration settings are ignored because an in-process Flink Cluster is being
started.
For running a Beam pipeline on YARN you have two options right now:
1. Start a Flink YARN session as described here
htt
