Hi Pankaj,
When you start a session cluster with the bin/yarn-session.sh script,
Flink will create the cluster and then write a "Yarn Properties file"
named ".yarn-properties-YOUR_USER_NAME" in the directory:
either the one specified by the option "yarn.properties-file.location"
in the flink-conf.yaml or in your local
System.getProperty("java.io.tmpdir"). This file will contain the
applicationId of the cluster and
it will be picked up by any future calls to `flink run`. Could you
check if this file exists and if it is updated every time you create a
cluster?
Thanks,
Kostas
On Thu, Dec 12, 2019 at 2:22 PM vino yang <yanghua1...@gmail.com> wrote:
>
> Hi Pankaj,
>
> Can you tell us what's Flink version do you use? And can you share the Flink
> client and job manager log with us?
>
> This information would help us to locate your problem.
>
> Best,
> Vino
>
> Pankaj Chand <pankajchanda...@gmail.com> 于2019年12月12日周四 下午7:08写道：
>>
>> Hello,
>>
>> When using Flink on YARN in session mode, each Flink job client would
>> automatically know the YARN cluster to connect to. It says this somewhere in
>> the documentation.
>>
>> So, I killed the Flink session cluster by simply killing the YARN
>> application using the "yarn kill" command. However, when starting a new
>> Flink session cluster and trying to submit Flink jobs to yarn-session, Flink
>> complains that the old cluster (it gave the port number and YARN application
>> ID) is not available.
>>
>> It seems like the details of the old cluster were still stored somewhere in
>> Flink. So, I had to completely replace the Flink folder with a new one.
>>
>> Does anyone know the proper way to kill a Flink+YARN session cluster to
>> completely remove it so that jobs will get submitted to a new Flink session
>> cluster?
>>
>> Thanks,
>>
>> Pankaj
