Hi,
I am looking into the cause YARN starts new application attempt on Flink
1.5.2. The challenge is getting the logs for the first attempt. After
checking YARN I discovered that in the first attempt and the second one
application manager (job manager) gets assigned the same container id (is
this expected ?) In this case logs from the first attempt are overwritten?
I found that *setKeepContainersAcrossApplicationAttempts* is enabled here
here
<https://github.com/apache/flink/blob/2ec72123e347e684ac40a1e1111a79a11211aadb/flink-yarn/src/main/java/org/apache/flink/yarn/AbstractYarnClusterDescriptor.java#L1340>
The second challenge is understanding if the job will be restored into new
application attempts or new application attempt will just have flink
running without any job?
Regards,
Pawel
*First attempt:*
pawel_bartoszek@ip-10-4-X-X ~]$ yarn container -list
appattempt_1538570922803_0020_000001
18/10/08 10:16:16 INFO client.RMProxy: Connecting to ResourceManager at
ip-10-4-X-X.eu-west-1.compute.internal/10.4.108.26:8032
Total number of containers :1
Container-Id Start Time Finish Time
State Host Node Http Address
LOG-URL
container_1538570922803_0020_02_000001 Mon Oct 08 09:47:17 +0000 2018
N/A RUNNING
ip-10-4-X-X.eu-west-1.compute.internal:8041
http://ip-10-4-X-X.eu-west-1.compute.internal:8042
http://ip-10-4-X-X.eu-west-1.compute.internal:8042/node/containerlogs/container_1538570922803_0020_02_000001/pawel_bartoszek
*Second attempt:*
[pawel_bartoszek@ip-10-4-X-X ~]$ yarn container -list
appattempt_1538570922803_0020_000002
18/10/08 10:16:37 INFO client.RMProxy: Connecting to ResourceManager at
ip-10-4-X-X.eu-west-1.compute.internal/10.4.X.X:8032
Total number of containers :1
Container-Id Start Time Finish Time
State Host Node Http Address
LOG-URL
container_1538570922803_0020_02_000001 Mon Oct 08 09:47:17 +0000 2018
N/A RUNNING
ip-10-4-X-X.eu-west-1.compute.internal:8041
http://ip-10-4-X-X.eu-west-1.compute.internal:8042
http://ip-10-4-X-X.eu-west-1.compute.internal:8042/node/containerlogs/container_1538570922803_0020_02_000001/pawel_bartoszek
