[jira] [Commented] (FLINK-20143) use `yarn.provided.lib.dirs` config deploy job failed in yarn per job mode
[ https://issues.apache.org/jira/browse/FLINK-20143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17233592#comment-17233592 ] Kostas Kloudas commented on FLINK-20143: Yes, I will try to merge it today and hopefully it will make it in 1.12 [~zhisheng]. > use `yarn.provided.lib.dirs` config deploy job failed in yarn per job mode > -- > > Key: FLINK-20143 > URL: https://issues.apache.org/jira/browse/FLINK-20143 > Project: Flink > Issue Type: Bug > Components: Client / Job Submission, Deployment / YARN >Affects Versions: 1.12.0, 1.11.2 >Reporter: zhisheng >Assignee: Yang Wang >Priority: Major > Labels: pull-request-available > Attachments: image-2020-11-13-17-21-47-751.png, > image-2020-11-13-17-22-06-111.png, image-2020-11-13-18-43-55-188.png > > > use follow command deploy flink job to yarn failed > {code:java} > ./bin/flink run -m yarn-cluster -d -ynm flink-1.12-test -ytm 3g -yjm 3g -yD > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib > ./examples/streaming/StateMachineExample.jar > {code} > log: > {code:java} > $ ./bin/flink run -m yarn-cluster -d -ynm flink-1.12-test -ytm 3g -yjm 3g -yD > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib > ./examples/streaming/StateMachineExample.jar$ ./bin/flink run -m yarn-cluster > -d -ynm flink-1.12-test -ytm 3g -yjm 3g -yD > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib > ./examples/streaming/StateMachineExample.jarSLF4J: Class path contains > multiple SLF4J bindings.SLF4J: Found binding in > [jar:file:/data1/app/flink-1.12-SNAPSHOT/lib/log4j-slf4j-impl-2.12.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: > Found binding in > [jar:file:/data1/app/hadoop-2.7.3-snappy-32core12disk/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: > Found binding in > [jar:file:/data1/app/hadoop-2.7.3-snappy-32core12disk/share/hadoop/tools/lib/hadoop-aliyun-2.9.2-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: > See http://www.slf4j.org/codes.html#multiple_bindings for an > explanation.SLF4J: Actual binding is of type > [org.apache.logging.slf4j.Log4jLoggerFactory]2020-11-13 16:14:30,347 INFO > org.apache.flink.yarn.cli.FlinkYarnSessionCli [] - Dynamic > Property set: > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib2020-11-13 > 16:14:30,347 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli > [] - Dynamic Property set: > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/libUsage with > built-in data generator: StateMachineExample [--error-rate > ] [--sleep ]Usage > with Kafka: StateMachineExample --kafka-topic [--brokers > ]Options for both the above setups: [--backend ] > [--checkpoint-dir ] [--async-checkpoints ] > [--incremental-checkpoints ] [--output OR null for > stdout] > Using standalone source with error rate 0.00 and sleep delay 1 millis > 2020-11-13 16:14:30,706 WARN > org.apache.flink.yarn.configuration.YarnLogConfigUtil [] - The > configuration directory ('/data1/app/flink-1.12-SNAPSHOT/conf') already > contains a LOG4J config file.If you want to use logback, then please delete > or rename the log configuration file.2020-11-13 16:14:30,947 INFO > org.apache.hadoop.yarn.client.AHSProxy [] - Connecting > to Application History server at > FAT-hadoopuat-69117.vm.dc01.tech/10.69.1.17:102002020-11-13 16:14:30,958 INFO > org.apache.flink.yarn.YarnClusterDescriptor [] - No path > for the flink jar passed. Using the location of class > org.apache.flink.yarn.YarnClusterDescriptor to locate the jar2020-11-13 > 16:14:31,065 INFO > org.apache.hadoop.yarn.client.ConfiguredRMFailoverProxyProvider [] - Failing > over to rm22020-11-13 16:14:31,130 INFO > org.apache.flink.yarn.YarnClusterDescriptor [] - The > configured JobManager memory is 3072 MB. YARN will allocate 4096 MB to make > up an integer multiple of its minimum allocation memory (2048 MB, configured > via 'yarn.scheduler.minimum-allocation-mb'). The extra 1024 MB may not be > used by Flink.2020-11-13 16:14:31,130 INFO > org.apache.flink.yarn.YarnClusterDescriptor [] - The > configured TaskManager memory is 3072 MB. YARN will allocate 4096 MB to make > up an integer multiple of its minimum allocation memory (2048 MB, configured > via 'yarn.scheduler.minimum-allocation-mb'). The extra 1024 MB may not be > used by Flink.2020-11-13 16:14:31,130 INFO > org.apache.flink.yarn.YarnClusterDescriptor [] - Cluster > specification: ClusterSpecification{masterMemoryMB=3072, > taskManagerMemoryMB=3072,
[jira] [Commented] (FLINK-20143) use `yarn.provided.lib.dirs` config deploy job failed in yarn per job mode
[ https://issues.apache.org/jira/browse/FLINK-20143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17233539#comment-17233539 ] zhisheng commented on FLINK-20143: -- thanks [~fly_in_gis] , it works well now, thanks [~kkl0u] too, it will push to 1.12.0 ? > use `yarn.provided.lib.dirs` config deploy job failed in yarn per job mode > -- > > Key: FLINK-20143 > URL: https://issues.apache.org/jira/browse/FLINK-20143 > Project: Flink > Issue Type: Bug > Components: Client / Job Submission, Deployment / YARN >Affects Versions: 1.12.0, 1.11.2 >Reporter: zhisheng >Assignee: Yang Wang >Priority: Major > Labels: pull-request-available > Attachments: image-2020-11-13-17-21-47-751.png, > image-2020-11-13-17-22-06-111.png, image-2020-11-13-18-43-55-188.png > > > use follow command deploy flink job to yarn failed > {code:java} > ./bin/flink run -m yarn-cluster -d -ynm flink-1.12-test -ytm 3g -yjm 3g -yD > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib > ./examples/streaming/StateMachineExample.jar > {code} > log: > {code:java} > $ ./bin/flink run -m yarn-cluster -d -ynm flink-1.12-test -ytm 3g -yjm 3g -yD > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib > ./examples/streaming/StateMachineExample.jar$ ./bin/flink run -m yarn-cluster > -d -ynm flink-1.12-test -ytm 3g -yjm 3g -yD > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib > ./examples/streaming/StateMachineExample.jarSLF4J: Class path contains > multiple SLF4J bindings.SLF4J: Found binding in > [jar:file:/data1/app/flink-1.12-SNAPSHOT/lib/log4j-slf4j-impl-2.12.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: > Found binding in > [jar:file:/data1/app/hadoop-2.7.3-snappy-32core12disk/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: > Found binding in > [jar:file:/data1/app/hadoop-2.7.3-snappy-32core12disk/share/hadoop/tools/lib/hadoop-aliyun-2.9.2-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: > See http://www.slf4j.org/codes.html#multiple_bindings for an > explanation.SLF4J: Actual binding is of type > [org.apache.logging.slf4j.Log4jLoggerFactory]2020-11-13 16:14:30,347 INFO > org.apache.flink.yarn.cli.FlinkYarnSessionCli [] - Dynamic > Property set: > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib2020-11-13 > 16:14:30,347 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli > [] - Dynamic Property set: > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/libUsage with > built-in data generator: StateMachineExample [--error-rate > ] [--sleep ]Usage > with Kafka: StateMachineExample --kafka-topic [--brokers > ]Options for both the above setups: [--backend ] > [--checkpoint-dir ] [--async-checkpoints ] > [--incremental-checkpoints ] [--output OR null for > stdout] > Using standalone source with error rate 0.00 and sleep delay 1 millis > 2020-11-13 16:14:30,706 WARN > org.apache.flink.yarn.configuration.YarnLogConfigUtil [] - The > configuration directory ('/data1/app/flink-1.12-SNAPSHOT/conf') already > contains a LOG4J config file.If you want to use logback, then please delete > or rename the log configuration file.2020-11-13 16:14:30,947 INFO > org.apache.hadoop.yarn.client.AHSProxy [] - Connecting > to Application History server at > FAT-hadoopuat-69117.vm.dc01.tech/10.69.1.17:102002020-11-13 16:14:30,958 INFO > org.apache.flink.yarn.YarnClusterDescriptor [] - No path > for the flink jar passed. Using the location of class > org.apache.flink.yarn.YarnClusterDescriptor to locate the jar2020-11-13 > 16:14:31,065 INFO > org.apache.hadoop.yarn.client.ConfiguredRMFailoverProxyProvider [] - Failing > over to rm22020-11-13 16:14:31,130 INFO > org.apache.flink.yarn.YarnClusterDescriptor [] - The > configured JobManager memory is 3072 MB. YARN will allocate 4096 MB to make > up an integer multiple of its minimum allocation memory (2048 MB, configured > via 'yarn.scheduler.minimum-allocation-mb'). The extra 1024 MB may not be > used by Flink.2020-11-13 16:14:31,130 INFO > org.apache.flink.yarn.YarnClusterDescriptor [] - The > configured TaskManager memory is 3072 MB. YARN will allocate 4096 MB to make > up an integer multiple of its minimum allocation memory (2048 MB, configured > via 'yarn.scheduler.minimum-allocation-mb'). The extra 1024 MB may not be > used by Flink.2020-11-13 16:14:31,130 INFO > org.apache.flink.yarn.YarnClusterDescriptor [] - Cluster > specification: ClusterSpecification{masterMemoryMB=3072, > taskManagerMemoryMB=3072, slotsPerTaskManager=2}2020-11-13
[jira] [Commented] (FLINK-20143) use `yarn.provided.lib.dirs` config deploy job failed in yarn per job mode
[ https://issues.apache.org/jira/browse/FLINK-20143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17232689#comment-17232689 ] Yang Wang commented on FLINK-20143: --- [~zhisheng] I have attached a PR to fix this issue. Also I verified your command in a Yarn cluster after this change, it works well. Please have a try. {code:java} ./bin/flink run -m yarn-cluster -d -ynm flink-1.12-test -ytm 3g -yjm 3g -yD yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib ./examples/streaming/StateMachineExample.jar {code} > use `yarn.provided.lib.dirs` config deploy job failed in yarn per job mode > -- > > Key: FLINK-20143 > URL: https://issues.apache.org/jira/browse/FLINK-20143 > Project: Flink > Issue Type: Bug > Components: Client / Job Submission, Deployment / YARN >Affects Versions: 1.12.0 >Reporter: zhisheng >Priority: Major > Labels: pull-request-available > Attachments: image-2020-11-13-17-21-47-751.png, > image-2020-11-13-17-22-06-111.png, image-2020-11-13-18-43-55-188.png > > > use follow command deploy flink job to yarn failed > {code:java} > ./bin/flink run -m yarn-cluster -d -ynm flink-1.12-test -ytm 3g -yjm 3g -yD > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib > ./examples/streaming/StateMachineExample.jar > {code} > log: > {code:java} > $ ./bin/flink run -m yarn-cluster -d -ynm flink-1.12-test -ytm 3g -yjm 3g -yD > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib > ./examples/streaming/StateMachineExample.jar$ ./bin/flink run -m yarn-cluster > -d -ynm flink-1.12-test -ytm 3g -yjm 3g -yD > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib > ./examples/streaming/StateMachineExample.jarSLF4J: Class path contains > multiple SLF4J bindings.SLF4J: Found binding in > [jar:file:/data1/app/flink-1.12-SNAPSHOT/lib/log4j-slf4j-impl-2.12.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: > Found binding in > [jar:file:/data1/app/hadoop-2.7.3-snappy-32core12disk/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: > Found binding in > [jar:file:/data1/app/hadoop-2.7.3-snappy-32core12disk/share/hadoop/tools/lib/hadoop-aliyun-2.9.2-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: > See http://www.slf4j.org/codes.html#multiple_bindings for an > explanation.SLF4J: Actual binding is of type > [org.apache.logging.slf4j.Log4jLoggerFactory]2020-11-13 16:14:30,347 INFO > org.apache.flink.yarn.cli.FlinkYarnSessionCli [] - Dynamic > Property set: > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib2020-11-13 > 16:14:30,347 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli > [] - Dynamic Property set: > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/libUsage with > built-in data generator: StateMachineExample [--error-rate > ] [--sleep ]Usage > with Kafka: StateMachineExample --kafka-topic [--brokers > ]Options for both the above setups: [--backend ] > [--checkpoint-dir ] [--async-checkpoints ] > [--incremental-checkpoints ] [--output OR null for > stdout] > Using standalone source with error rate 0.00 and sleep delay 1 millis > 2020-11-13 16:14:30,706 WARN > org.apache.flink.yarn.configuration.YarnLogConfigUtil [] - The > configuration directory ('/data1/app/flink-1.12-SNAPSHOT/conf') already > contains a LOG4J config file.If you want to use logback, then please delete > or rename the log configuration file.2020-11-13 16:14:30,947 INFO > org.apache.hadoop.yarn.client.AHSProxy [] - Connecting > to Application History server at > FAT-hadoopuat-69117.vm.dc01.tech/10.69.1.17:102002020-11-13 16:14:30,958 INFO > org.apache.flink.yarn.YarnClusterDescriptor [] - No path > for the flink jar passed. Using the location of class > org.apache.flink.yarn.YarnClusterDescriptor to locate the jar2020-11-13 > 16:14:31,065 INFO > org.apache.hadoop.yarn.client.ConfiguredRMFailoverProxyProvider [] - Failing > over to rm22020-11-13 16:14:31,130 INFO > org.apache.flink.yarn.YarnClusterDescriptor [] - The > configured JobManager memory is 3072 MB. YARN will allocate 4096 MB to make > up an integer multiple of its minimum allocation memory (2048 MB, configured > via 'yarn.scheduler.minimum-allocation-mb'). The extra 1024 MB may not be > used by Flink.2020-11-13 16:14:31,130 INFO > org.apache.flink.yarn.YarnClusterDescriptor [] - The > configured TaskManager memory is 3072 MB. YARN will allocate 4096 MB to make > up an integer multiple of its minimum allocation memory (2048 MB, configured > via 'yarn.scheduler.minimum-allocation-mb'). The extra 1024 MB may not be > used by
[jira] [Commented] (FLINK-20143) use `yarn.provided.lib.dirs` config deploy job failed in yarn per job mode
[ https://issues.apache.org/jira/browse/FLINK-20143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17232590#comment-17232590 ] Kostas Kloudas commented on FLINK-20143: Thanks [~fly_in_gis], feel free to ping me for a review. > use `yarn.provided.lib.dirs` config deploy job failed in yarn per job mode > -- > > Key: FLINK-20143 > URL: https://issues.apache.org/jira/browse/FLINK-20143 > Project: Flink > Issue Type: Bug > Components: Client / Job Submission, Deployment / YARN >Affects Versions: 1.12.0 >Reporter: zhisheng >Priority: Major > Attachments: image-2020-11-13-17-21-47-751.png, > image-2020-11-13-17-22-06-111.png, image-2020-11-13-18-43-55-188.png > > > use follow command deploy flink job to yarn failed > {code:java} > ./bin/flink run -m yarn-cluster -d -ynm flink-1.12-test -ytm 3g -yjm 3g -yD > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib > ./examples/streaming/StateMachineExample.jar > {code} > log: > {code:java} > $ ./bin/flink run -m yarn-cluster -d -ynm flink-1.12-test -ytm 3g -yjm 3g -yD > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib > ./examples/streaming/StateMachineExample.jar$ ./bin/flink run -m yarn-cluster > -d -ynm flink-1.12-test -ytm 3g -yjm 3g -yD > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib > ./examples/streaming/StateMachineExample.jarSLF4J: Class path contains > multiple SLF4J bindings.SLF4J: Found binding in > [jar:file:/data1/app/flink-1.12-SNAPSHOT/lib/log4j-slf4j-impl-2.12.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: > Found binding in > [jar:file:/data1/app/hadoop-2.7.3-snappy-32core12disk/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: > Found binding in > [jar:file:/data1/app/hadoop-2.7.3-snappy-32core12disk/share/hadoop/tools/lib/hadoop-aliyun-2.9.2-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: > See http://www.slf4j.org/codes.html#multiple_bindings for an > explanation.SLF4J: Actual binding is of type > [org.apache.logging.slf4j.Log4jLoggerFactory]2020-11-13 16:14:30,347 INFO > org.apache.flink.yarn.cli.FlinkYarnSessionCli [] - Dynamic > Property set: > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib2020-11-13 > 16:14:30,347 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli > [] - Dynamic Property set: > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/libUsage with > built-in data generator: StateMachineExample [--error-rate > ] [--sleep ]Usage > with Kafka: StateMachineExample --kafka-topic [--brokers > ]Options for both the above setups: [--backend ] > [--checkpoint-dir ] [--async-checkpoints ] > [--incremental-checkpoints ] [--output OR null for > stdout] > Using standalone source with error rate 0.00 and sleep delay 1 millis > 2020-11-13 16:14:30,706 WARN > org.apache.flink.yarn.configuration.YarnLogConfigUtil [] - The > configuration directory ('/data1/app/flink-1.12-SNAPSHOT/conf') already > contains a LOG4J config file.If you want to use logback, then please delete > or rename the log configuration file.2020-11-13 16:14:30,947 INFO > org.apache.hadoop.yarn.client.AHSProxy [] - Connecting > to Application History server at > FAT-hadoopuat-69117.vm.dc01.tech/10.69.1.17:102002020-11-13 16:14:30,958 INFO > org.apache.flink.yarn.YarnClusterDescriptor [] - No path > for the flink jar passed. Using the location of class > org.apache.flink.yarn.YarnClusterDescriptor to locate the jar2020-11-13 > 16:14:31,065 INFO > org.apache.hadoop.yarn.client.ConfiguredRMFailoverProxyProvider [] - Failing > over to rm22020-11-13 16:14:31,130 INFO > org.apache.flink.yarn.YarnClusterDescriptor [] - The > configured JobManager memory is 3072 MB. YARN will allocate 4096 MB to make > up an integer multiple of its minimum allocation memory (2048 MB, configured > via 'yarn.scheduler.minimum-allocation-mb'). The extra 1024 MB may not be > used by Flink.2020-11-13 16:14:31,130 INFO > org.apache.flink.yarn.YarnClusterDescriptor [] - The > configured TaskManager memory is 3072 MB. YARN will allocate 4096 MB to make > up an integer multiple of its minimum allocation memory (2048 MB, configured > via 'yarn.scheduler.minimum-allocation-mb'). The extra 1024 MB may not be > used by Flink.2020-11-13 16:14:31,130 INFO > org.apache.flink.yarn.YarnClusterDescriptor [] - Cluster > specification: ClusterSpecification{masterMemoryMB=3072, > taskManagerMemoryMB=3072, slotsPerTaskManager=2}2020-11-13 16:14:31,681 WARN > org.apache.hadoop.hdfs.shortcircuit.DomainSocketFactory [] - The > short-circuit
[jira] [Commented] (FLINK-20143) use `yarn.provided.lib.dirs` config deploy job failed in yarn per job mode
[ https://issues.apache.org/jira/browse/FLINK-20143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17232558#comment-17232558 ] Yang Wang commented on FLINK-20143: --- Hmm. I think maybe I find the root cause. When the {{yarn.provided.lib.dirs}} is set to the non-qualified path(e.g. hdfs:///path/of/sharedLib), the {{URI#relativize}} in {{YarnApplicationFileUploader#getAllFilesInProvidedLibDirs}} could not work as expected. [~zhisheng] So for your situation, I guess all the deployment(e.g. yarn-per-job, yarn-application, yarn-session) could not work effectively if you are using non-qualified path. [~kkl0u] Even though we have a work around, specify a qualified path(e.g. hdfs://hdpdev/path/of/sharedLib), I think it is better to fix this issue. I will attach a PR for this ticket. > use `yarn.provided.lib.dirs` config deploy job failed in yarn per job mode > -- > > Key: FLINK-20143 > URL: https://issues.apache.org/jira/browse/FLINK-20143 > Project: Flink > Issue Type: Bug > Components: Client / Job Submission, Deployment / YARN >Affects Versions: 1.12.0 >Reporter: zhisheng >Priority: Major > Attachments: image-2020-11-13-17-21-47-751.png, > image-2020-11-13-17-22-06-111.png, image-2020-11-13-18-43-55-188.png > > > use follow command deploy flink job to yarn failed > {code:java} > ./bin/flink run -m yarn-cluster -d -ynm flink-1.12-test -ytm 3g -yjm 3g -yD > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib > ./examples/streaming/StateMachineExample.jar > {code} > log: > {code:java} > $ ./bin/flink run -m yarn-cluster -d -ynm flink-1.12-test -ytm 3g -yjm 3g -yD > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib > ./examples/streaming/StateMachineExample.jar$ ./bin/flink run -m yarn-cluster > -d -ynm flink-1.12-test -ytm 3g -yjm 3g -yD > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib > ./examples/streaming/StateMachineExample.jarSLF4J: Class path contains > multiple SLF4J bindings.SLF4J: Found binding in > [jar:file:/data1/app/flink-1.12-SNAPSHOT/lib/log4j-slf4j-impl-2.12.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: > Found binding in > [jar:file:/data1/app/hadoop-2.7.3-snappy-32core12disk/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: > Found binding in > [jar:file:/data1/app/hadoop-2.7.3-snappy-32core12disk/share/hadoop/tools/lib/hadoop-aliyun-2.9.2-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: > See http://www.slf4j.org/codes.html#multiple_bindings for an > explanation.SLF4J: Actual binding is of type > [org.apache.logging.slf4j.Log4jLoggerFactory]2020-11-13 16:14:30,347 INFO > org.apache.flink.yarn.cli.FlinkYarnSessionCli [] - Dynamic > Property set: > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib2020-11-13 > 16:14:30,347 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli > [] - Dynamic Property set: > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/libUsage with > built-in data generator: StateMachineExample [--error-rate > ] [--sleep ]Usage > with Kafka: StateMachineExample --kafka-topic [--brokers > ]Options for both the above setups: [--backend ] > [--checkpoint-dir ] [--async-checkpoints ] > [--incremental-checkpoints ] [--output OR null for > stdout] > Using standalone source with error rate 0.00 and sleep delay 1 millis > 2020-11-13 16:14:30,706 WARN > org.apache.flink.yarn.configuration.YarnLogConfigUtil [] - The > configuration directory ('/data1/app/flink-1.12-SNAPSHOT/conf') already > contains a LOG4J config file.If you want to use logback, then please delete > or rename the log configuration file.2020-11-13 16:14:30,947 INFO > org.apache.hadoop.yarn.client.AHSProxy [] - Connecting > to Application History server at > FAT-hadoopuat-69117.vm.dc01.tech/10.69.1.17:102002020-11-13 16:14:30,958 INFO > org.apache.flink.yarn.YarnClusterDescriptor [] - No path > for the flink jar passed. Using the location of class > org.apache.flink.yarn.YarnClusterDescriptor to locate the jar2020-11-13 > 16:14:31,065 INFO > org.apache.hadoop.yarn.client.ConfiguredRMFailoverProxyProvider [] - Failing > over to rm22020-11-13 16:14:31,130 INFO > org.apache.flink.yarn.YarnClusterDescriptor [] - The > configured JobManager memory is 3072 MB. YARN will allocate 4096 MB to make > up an integer multiple of its minimum allocation memory (2048 MB, configured > via 'yarn.scheduler.minimum-allocation-mb'). The extra 1024 MB may not be > used by Flink.2020-11-13 16:14:31,130 INFO > org.apache.flink.yarn.YarnClusterDescriptor [] - The > configured TaskManager
[jira] [Commented] (FLINK-20143) use `yarn.provided.lib.dirs` config deploy job failed in yarn per job mode
[ https://issues.apache.org/jira/browse/FLINK-20143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17232488#comment-17232488 ] Yang Wang commented on FLINK-20143: --- [~zhisheng] Could you add the hdfs schema in the {{yarn.provided.lib.dirs}} and have a try again. For example, {{-yD yarn.provided.lib.dirs=hdfs://hdpdev/flink/flink-1.12-SNAPSHOT/lib}} > use `yarn.provided.lib.dirs` config deploy job failed in yarn per job mode > -- > > Key: FLINK-20143 > URL: https://issues.apache.org/jira/browse/FLINK-20143 > Project: Flink > Issue Type: Bug > Components: Client / Job Submission, Deployment / YARN >Affects Versions: 1.12.0 >Reporter: zhisheng >Priority: Major > Attachments: image-2020-11-13-17-21-47-751.png, > image-2020-11-13-17-22-06-111.png, image-2020-11-13-18-43-55-188.png > > > use follow command deploy flink job to yarn failed > {code:java} > ./bin/flink run -m yarn-cluster -d -ynm flink-1.12-test -ytm 3g -yjm 3g -yD > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib > ./examples/streaming/StateMachineExample.jar > {code} > log: > {code:java} > $ ./bin/flink run -m yarn-cluster -d -ynm flink-1.12-test -ytm 3g -yjm 3g -yD > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib > ./examples/streaming/StateMachineExample.jar$ ./bin/flink run -m yarn-cluster > -d -ynm flink-1.12-test -ytm 3g -yjm 3g -yD > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib > ./examples/streaming/StateMachineExample.jarSLF4J: Class path contains > multiple SLF4J bindings.SLF4J: Found binding in > [jar:file:/data1/app/flink-1.12-SNAPSHOT/lib/log4j-slf4j-impl-2.12.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: > Found binding in > [jar:file:/data1/app/hadoop-2.7.3-snappy-32core12disk/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: > Found binding in > [jar:file:/data1/app/hadoop-2.7.3-snappy-32core12disk/share/hadoop/tools/lib/hadoop-aliyun-2.9.2-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: > See http://www.slf4j.org/codes.html#multiple_bindings for an > explanation.SLF4J: Actual binding is of type > [org.apache.logging.slf4j.Log4jLoggerFactory]2020-11-13 16:14:30,347 INFO > org.apache.flink.yarn.cli.FlinkYarnSessionCli [] - Dynamic > Property set: > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib2020-11-13 > 16:14:30,347 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli > [] - Dynamic Property set: > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/libUsage with > built-in data generator: StateMachineExample [--error-rate > ] [--sleep ]Usage > with Kafka: StateMachineExample --kafka-topic [--brokers > ]Options for both the above setups: [--backend ] > [--checkpoint-dir ] [--async-checkpoints ] > [--incremental-checkpoints ] [--output OR null for > stdout] > Using standalone source with error rate 0.00 and sleep delay 1 millis > 2020-11-13 16:14:30,706 WARN > org.apache.flink.yarn.configuration.YarnLogConfigUtil [] - The > configuration directory ('/data1/app/flink-1.12-SNAPSHOT/conf') already > contains a LOG4J config file.If you want to use logback, then please delete > or rename the log configuration file.2020-11-13 16:14:30,947 INFO > org.apache.hadoop.yarn.client.AHSProxy [] - Connecting > to Application History server at > FAT-hadoopuat-69117.vm.dc01.tech/10.69.1.17:102002020-11-13 16:14:30,958 INFO > org.apache.flink.yarn.YarnClusterDescriptor [] - No path > for the flink jar passed. Using the location of class > org.apache.flink.yarn.YarnClusterDescriptor to locate the jar2020-11-13 > 16:14:31,065 INFO > org.apache.hadoop.yarn.client.ConfiguredRMFailoverProxyProvider [] - Failing > over to rm22020-11-13 16:14:31,130 INFO > org.apache.flink.yarn.YarnClusterDescriptor [] - The > configured JobManager memory is 3072 MB. YARN will allocate 4096 MB to make > up an integer multiple of its minimum allocation memory (2048 MB, configured > via 'yarn.scheduler.minimum-allocation-mb'). The extra 1024 MB may not be > used by Flink.2020-11-13 16:14:31,130 INFO > org.apache.flink.yarn.YarnClusterDescriptor [] - The > configured TaskManager memory is 3072 MB. YARN will allocate 4096 MB to make > up an integer multiple of its minimum allocation memory (2048 MB, configured > via 'yarn.scheduler.minimum-allocation-mb'). The extra 1024 MB may not be > used by Flink.2020-11-13 16:14:31,130 INFO > org.apache.flink.yarn.YarnClusterDescriptor [] - Cluster > specification: ClusterSpecification{masterMemoryMB=3072, > taskManagerMemoryMB=3072,
[jira] [Commented] (FLINK-20143) use `yarn.provided.lib.dirs` config deploy job failed in yarn per job mode
[ https://issues.apache.org/jira/browse/FLINK-20143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17231522#comment-17231522 ] Kostas Kloudas commented on FLINK-20143: [~fly_in_gis] do you have an idea about this issue? > use `yarn.provided.lib.dirs` config deploy job failed in yarn per job mode > -- > > Key: FLINK-20143 > URL: https://issues.apache.org/jira/browse/FLINK-20143 > Project: Flink > Issue Type: Bug > Components: Client / Job Submission, Deployment / YARN >Affects Versions: 1.12.0 >Reporter: zhisheng >Priority: Major > Attachments: image-2020-11-13-17-21-47-751.png, > image-2020-11-13-17-22-06-111.png, image-2020-11-13-18-43-55-188.png > > > use follow command deploy flink job to yarn failed > {code:java} > ./bin/flink run -m yarn-cluster -d -ynm flink-1.12-test -ytm 3g -yjm 3g -yD > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib > ./examples/streaming/StateMachineExample.jar > {code} > log: > {code:java} > $ ./bin/flink run -m yarn-cluster -d -ynm flink-1.12-test -ytm 3g -yjm 3g -yD > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib > ./examples/streaming/StateMachineExample.jar$ ./bin/flink run -m yarn-cluster > -d -ynm flink-1.12-test -ytm 3g -yjm 3g -yD > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib > ./examples/streaming/StateMachineExample.jarSLF4J: Class path contains > multiple SLF4J bindings.SLF4J: Found binding in > [jar:file:/data1/app/flink-1.12-SNAPSHOT/lib/log4j-slf4j-impl-2.12.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: > Found binding in > [jar:file:/data1/app/hadoop-2.7.3-snappy-32core12disk/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: > Found binding in > [jar:file:/data1/app/hadoop-2.7.3-snappy-32core12disk/share/hadoop/tools/lib/hadoop-aliyun-2.9.2-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: > See http://www.slf4j.org/codes.html#multiple_bindings for an > explanation.SLF4J: Actual binding is of type > [org.apache.logging.slf4j.Log4jLoggerFactory]2020-11-13 16:14:30,347 INFO > org.apache.flink.yarn.cli.FlinkYarnSessionCli [] - Dynamic > Property set: > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib2020-11-13 > 16:14:30,347 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli > [] - Dynamic Property set: > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/libUsage with > built-in data generator: StateMachineExample [--error-rate > ] [--sleep ]Usage > with Kafka: StateMachineExample --kafka-topic [--brokers > ]Options for both the above setups: [--backend ] > [--checkpoint-dir ] [--async-checkpoints ] > [--incremental-checkpoints ] [--output OR null for > stdout] > Using standalone source with error rate 0.00 and sleep delay 1 millis > 2020-11-13 16:14:30,706 WARN > org.apache.flink.yarn.configuration.YarnLogConfigUtil [] - The > configuration directory ('/data1/app/flink-1.12-SNAPSHOT/conf') already > contains a LOG4J config file.If you want to use logback, then please delete > or rename the log configuration file.2020-11-13 16:14:30,947 INFO > org.apache.hadoop.yarn.client.AHSProxy [] - Connecting > to Application History server at > FAT-hadoopuat-69117.vm.dc01.tech/10.69.1.17:102002020-11-13 16:14:30,958 INFO > org.apache.flink.yarn.YarnClusterDescriptor [] - No path > for the flink jar passed. Using the location of class > org.apache.flink.yarn.YarnClusterDescriptor to locate the jar2020-11-13 > 16:14:31,065 INFO > org.apache.hadoop.yarn.client.ConfiguredRMFailoverProxyProvider [] - Failing > over to rm22020-11-13 16:14:31,130 INFO > org.apache.flink.yarn.YarnClusterDescriptor [] - The > configured JobManager memory is 3072 MB. YARN will allocate 4096 MB to make > up an integer multiple of its minimum allocation memory (2048 MB, configured > via 'yarn.scheduler.minimum-allocation-mb'). The extra 1024 MB may not be > used by Flink.2020-11-13 16:14:31,130 INFO > org.apache.flink.yarn.YarnClusterDescriptor [] - The > configured TaskManager memory is 3072 MB. YARN will allocate 4096 MB to make > up an integer multiple of its minimum allocation memory (2048 MB, configured > via 'yarn.scheduler.minimum-allocation-mb'). The extra 1024 MB may not be > used by Flink.2020-11-13 16:14:31,130 INFO > org.apache.flink.yarn.YarnClusterDescriptor [] - Cluster > specification: ClusterSpecification{masterMemoryMB=3072, > taskManagerMemoryMB=3072, slotsPerTaskManager=2}2020-11-13 16:14:31,681 WARN > org.apache.hadoop.hdfs.shortcircuit.DomainSocketFactory [] - The > short-circuit local
[jira] [Commented] (FLINK-20143) use `yarn.provided.lib.dirs` config deploy job failed in yarn per job mode
[ https://issues.apache.org/jira/browse/FLINK-20143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17231415#comment-17231415 ] Kostas Kloudas commented on FLINK-20143: Hi [~zhisheng], I run the command that I wrote in my previous comment on a local yarn cluster, and it seems to be working. I do not have your config.yaml though, as I start with the default one. > use `yarn.provided.lib.dirs` config deploy job failed in yarn per job mode > -- > > Key: FLINK-20143 > URL: https://issues.apache.org/jira/browse/FLINK-20143 > Project: Flink > Issue Type: Bug > Components: Client / Job Submission, Deployment / YARN >Affects Versions: 1.12.0 >Reporter: zhisheng >Priority: Major > Attachments: image-2020-11-13-17-21-47-751.png, > image-2020-11-13-17-22-06-111.png, image-2020-11-13-18-43-55-188.png > > > use follow command deploy flink job to yarn failed > {code:java} > ./bin/flink run -m yarn-cluster -d -ynm flink-1.12-test -ytm 3g -yjm 3g -yD > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib > ./examples/streaming/StateMachineExample.jar > {code} > log: > {code:java} > $ ./bin/flink run -m yarn-cluster -d -ynm flink-1.12-test -ytm 3g -yjm 3g -yD > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib > ./examples/streaming/StateMachineExample.jar$ ./bin/flink run -m yarn-cluster > -d -ynm flink-1.12-test -ytm 3g -yjm 3g -yD > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib > ./examples/streaming/StateMachineExample.jarSLF4J: Class path contains > multiple SLF4J bindings.SLF4J: Found binding in > [jar:file:/data1/app/flink-1.12-SNAPSHOT/lib/log4j-slf4j-impl-2.12.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: > Found binding in > [jar:file:/data1/app/hadoop-2.7.3-snappy-32core12disk/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: > Found binding in > [jar:file:/data1/app/hadoop-2.7.3-snappy-32core12disk/share/hadoop/tools/lib/hadoop-aliyun-2.9.2-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: > See http://www.slf4j.org/codes.html#multiple_bindings for an > explanation.SLF4J: Actual binding is of type > [org.apache.logging.slf4j.Log4jLoggerFactory]2020-11-13 16:14:30,347 INFO > org.apache.flink.yarn.cli.FlinkYarnSessionCli [] - Dynamic > Property set: > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib2020-11-13 > 16:14:30,347 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli > [] - Dynamic Property set: > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/libUsage with > built-in data generator: StateMachineExample [--error-rate > ] [--sleep ]Usage > with Kafka: StateMachineExample --kafka-topic [--brokers > ]Options for both the above setups: [--backend ] > [--checkpoint-dir ] [--async-checkpoints ] > [--incremental-checkpoints ] [--output OR null for > stdout] > Using standalone source with error rate 0.00 and sleep delay 1 millis > 2020-11-13 16:14:30,706 WARN > org.apache.flink.yarn.configuration.YarnLogConfigUtil [] - The > configuration directory ('/data1/app/flink-1.12-SNAPSHOT/conf') already > contains a LOG4J config file.If you want to use logback, then please delete > or rename the log configuration file.2020-11-13 16:14:30,947 INFO > org.apache.hadoop.yarn.client.AHSProxy [] - Connecting > to Application History server at > FAT-hadoopuat-69117.vm.dc01.tech/10.69.1.17:102002020-11-13 16:14:30,958 INFO > org.apache.flink.yarn.YarnClusterDescriptor [] - No path > for the flink jar passed. Using the location of class > org.apache.flink.yarn.YarnClusterDescriptor to locate the jar2020-11-13 > 16:14:31,065 INFO > org.apache.hadoop.yarn.client.ConfiguredRMFailoverProxyProvider [] - Failing > over to rm22020-11-13 16:14:31,130 INFO > org.apache.flink.yarn.YarnClusterDescriptor [] - The > configured JobManager memory is 3072 MB. YARN will allocate 4096 MB to make > up an integer multiple of its minimum allocation memory (2048 MB, configured > via 'yarn.scheduler.minimum-allocation-mb'). The extra 1024 MB may not be > used by Flink.2020-11-13 16:14:31,130 INFO > org.apache.flink.yarn.YarnClusterDescriptor [] - The > configured TaskManager memory is 3072 MB. YARN will allocate 4096 MB to make > up an integer multiple of its minimum allocation memory (2048 MB, configured > via 'yarn.scheduler.minimum-allocation-mb'). The extra 1024 MB may not be > used by Flink.2020-11-13 16:14:31,130 INFO > org.apache.flink.yarn.YarnClusterDescriptor [] - Cluster > specification: ClusterSpecification{masterMemoryMB=3072, > taskManagerMemoryMB=3072,
[jira] [Commented] (FLINK-20143) use `yarn.provided.lib.dirs` config deploy job failed in yarn per job mode
[ https://issues.apache.org/jira/browse/FLINK-20143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17231369#comment-17231369 ] zhisheng commented on FLINK-20143: -- {code:java} 22020-11-13 18:46:43,014 INFO org.apache.flink.client.cli.CliFrontend [] - 2020-11-13 18:46:43,014 INFO org.apache.flink.client.cli.CliFrontend [] - 2020-11-13 18:46:43,019 INFO org.apache.flink.client.cli.CliFrontend [] - Starting Command Line Client (Version: 1.12-SNAPSHOT, Scala: 2.11, Rev:c55420b, Date:2020-11-05T05:29:49+01:00)2020-11-13 18:46:43,019 INFO org.apache.flink.client.cli.CliFrontend [] - OS current user: deploy2020-11-13 18:46:43,415 INFO org.apache.flink.client.cli.CliFrontend [] - Current Hadoop/Kerberos user: deploy2020-11-13 18:46:43,416 INFO org.apache.flink.client.cli.CliFrontend [] - JVM: Java HotSpot(TM) 64-Bit Server VM - Oracle Corporation - 1.8/25.92-b142020-11-13 18:46:43,416 INFO org.apache.flink.client.cli.CliFrontend [] - Maximum heap size: 7136 MiBytes2020-11-13 18:46:43,416 INFO org.apache.flink.client.cli.CliFrontend [] - JAVA_HOME: /app/jdk/2020-11-13 18:46:43,418 INFO org.apache.flink.client.cli.CliFrontend [] - Hadoop version: 2.7.32020-11-13 18:46:43,418 INFO org.apache.flink.client.cli.CliFrontend [] - JVM Options:2020-11-13 18:46:43,418 INFO org.apache.flink.client.cli.CliFrontend [] - -Dlog.file=/data1/app/flink-1.12-SNAPSHOT/log/flink-deploy-client-FAT-hadoopuat-69120.vm.dc01. .tech.log2020-11-13 18:46:43,418 INFO org.apache.flink.client.cli.CliFrontend [] - -Dlog4j.configuration=file:/data1/app/flink-1.12-SNAPSHOT/conf/log4j-cli.properties2020-11-13 18:46:43,418 INFO org.apache.flink.client.cli.CliFrontend [] - -Dlog4j.configurationFile=file:/data1/app/flink-1.12-SNAPSHOT/conf/log4j-cli.properties2020-11-13 18:46:43,418 INFO org.apache.flink.client.cli.CliFrontend [] - -Dlogback.configurationFile=file:/data1/app/flink-1.12-SNAPSHOT/conf/logback.xml2020-11-13 18:46:43,419 INFO org.apache.flink.client.cli.CliFrontend [] - Program Arguments:2020-11-13 18:46:43,420 INFO org.apache.flink.client.cli.CliFrontend [] - run2020-11-13 18:46:43,420 INFO org.apache.flink.client.cli.CliFrontend [] - -t2020-11-13 18:46:43,421 INFO org.apache.flink.client.cli.CliFrontend [] - yarn-per-job2020-11-13 18:46:43,421 INFO org.apache.flink.client.cli.CliFrontend [] - -Dexecution.attached=false2020-11-13 18:46:43,421 INFO org.apache.flink.client.cli.CliFrontend [] - -Dyarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib2020-11-13 18:46:43,421 INFO org.apache.flink.client.cli.CliFrontend [] - ./examples/streaming/StateMachineExample.jar2020-11-13 18:46:43,421 INFO org.apache.flink.client.cli.CliFrontend [] - Classpath:
[jira] [Commented] (FLINK-20143) use `yarn.provided.lib.dirs` config deploy job failed in yarn per job mode
[ https://issues.apache.org/jira/browse/FLINK-20143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17231365#comment-17231365 ] zhisheng commented on FLINK-20143: -- !image-2020-11-13-18-43-55-188.png! does not has any log, i had say just now;) > use `yarn.provided.lib.dirs` config deploy job failed in yarn per job mode > -- > > Key: FLINK-20143 > URL: https://issues.apache.org/jira/browse/FLINK-20143 > Project: Flink > Issue Type: Bug > Components: Client / Job Submission, Deployment / YARN >Affects Versions: 1.12.0 >Reporter: zhisheng >Priority: Major > Attachments: image-2020-11-13-17-21-47-751.png, > image-2020-11-13-17-22-06-111.png, image-2020-11-13-18-43-55-188.png > > > use follow command deploy flink job to yarn failed > {code:java} > ./bin/flink run -m yarn-cluster -d -ynm flink-1.12-test -ytm 3g -yjm 3g -yD > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib > ./examples/streaming/StateMachineExample.jar > {code} > log: > {code:java} > $ ./bin/flink run -m yarn-cluster -d -ynm flink-1.12-test -ytm 3g -yjm 3g -yD > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib > ./examples/streaming/StateMachineExample.jar$ ./bin/flink run -m yarn-cluster > -d -ynm flink-1.12-test -ytm 3g -yjm 3g -yD > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib > ./examples/streaming/StateMachineExample.jarSLF4J: Class path contains > multiple SLF4J bindings.SLF4J: Found binding in > [jar:file:/data1/app/flink-1.12-SNAPSHOT/lib/log4j-slf4j-impl-2.12.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: > Found binding in > [jar:file:/data1/app/hadoop-2.7.3-snappy-32core12disk/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: > Found binding in > [jar:file:/data1/app/hadoop-2.7.3-snappy-32core12disk/share/hadoop/tools/lib/hadoop-aliyun-2.9.2-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: > See http://www.slf4j.org/codes.html#multiple_bindings for an > explanation.SLF4J: Actual binding is of type > [org.apache.logging.slf4j.Log4jLoggerFactory]2020-11-13 16:14:30,347 INFO > org.apache.flink.yarn.cli.FlinkYarnSessionCli [] - Dynamic > Property set: > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib2020-11-13 > 16:14:30,347 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli > [] - Dynamic Property set: > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/libUsage with > built-in data generator: StateMachineExample [--error-rate > ] [--sleep ]Usage > with Kafka: StateMachineExample --kafka-topic [--brokers > ]Options for both the above setups: [--backend ] > [--checkpoint-dir ] [--async-checkpoints ] > [--incremental-checkpoints ] [--output OR null for > stdout] > Using standalone source with error rate 0.00 and sleep delay 1 millis > 2020-11-13 16:14:30,706 WARN > org.apache.flink.yarn.configuration.YarnLogConfigUtil [] - The > configuration directory ('/data1/app/flink-1.12-SNAPSHOT/conf') already > contains a LOG4J config file.If you want to use logback, then please delete > or rename the log configuration file.2020-11-13 16:14:30,947 INFO > org.apache.hadoop.yarn.client.AHSProxy [] - Connecting > to Application History server at > FAT-hadoopuat-69117.vm.dc01.tech/10.69.1.17:102002020-11-13 16:14:30,958 INFO > org.apache.flink.yarn.YarnClusterDescriptor [] - No path > for the flink jar passed. Using the location of class > org.apache.flink.yarn.YarnClusterDescriptor to locate the jar2020-11-13 > 16:14:31,065 INFO > org.apache.hadoop.yarn.client.ConfiguredRMFailoverProxyProvider [] - Failing > over to rm22020-11-13 16:14:31,130 INFO > org.apache.flink.yarn.YarnClusterDescriptor [] - The > configured JobManager memory is 3072 MB. YARN will allocate 4096 MB to make > up an integer multiple of its minimum allocation memory (2048 MB, configured > via 'yarn.scheduler.minimum-allocation-mb'). The extra 1024 MB may not be > used by Flink.2020-11-13 16:14:31,130 INFO > org.apache.flink.yarn.YarnClusterDescriptor [] - The > configured TaskManager memory is 3072 MB. YARN will allocate 4096 MB to make > up an integer multiple of its minimum allocation memory (2048 MB, configured > via 'yarn.scheduler.minimum-allocation-mb'). The extra 1024 MB may not be > used by Flink.2020-11-13 16:14:31,130 INFO > org.apache.flink.yarn.YarnClusterDescriptor [] - Cluster > specification: ClusterSpecification{masterMemoryMB=3072, > taskManagerMemoryMB=3072, slotsPerTaskManager=2}2020-11-13 16:14:31,681 WARN > org.apache.hadoop.hdfs.shortcircuit.DomainSocketFactory [] - The >
[jira] [Commented] (FLINK-20143) use `yarn.provided.lib.dirs` config deploy job failed in yarn per job mode
[ https://issues.apache.org/jira/browse/FLINK-20143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17231361#comment-17231361 ] Kostas Kloudas commented on FLINK-20143: Also could you run the same command with DEBUG logging enabled? > use `yarn.provided.lib.dirs` config deploy job failed in yarn per job mode > -- > > Key: FLINK-20143 > URL: https://issues.apache.org/jira/browse/FLINK-20143 > Project: Flink > Issue Type: Bug > Components: Client / Job Submission, Deployment / YARN >Affects Versions: 1.12.0 >Reporter: zhisheng >Priority: Major > Attachments: image-2020-11-13-17-21-47-751.png, > image-2020-11-13-17-22-06-111.png > > > use follow command deploy flink job to yarn failed > {code:java} > ./bin/flink run -m yarn-cluster -d -ynm flink-1.12-test -ytm 3g -yjm 3g -yD > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib > ./examples/streaming/StateMachineExample.jar > {code} > log: > {code:java} > $ ./bin/flink run -m yarn-cluster -d -ynm flink-1.12-test -ytm 3g -yjm 3g -yD > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib > ./examples/streaming/StateMachineExample.jar$ ./bin/flink run -m yarn-cluster > -d -ynm flink-1.12-test -ytm 3g -yjm 3g -yD > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib > ./examples/streaming/StateMachineExample.jarSLF4J: Class path contains > multiple SLF4J bindings.SLF4J: Found binding in > [jar:file:/data1/app/flink-1.12-SNAPSHOT/lib/log4j-slf4j-impl-2.12.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: > Found binding in > [jar:file:/data1/app/hadoop-2.7.3-snappy-32core12disk/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: > Found binding in > [jar:file:/data1/app/hadoop-2.7.3-snappy-32core12disk/share/hadoop/tools/lib/hadoop-aliyun-2.9.2-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: > See http://www.slf4j.org/codes.html#multiple_bindings for an > explanation.SLF4J: Actual binding is of type > [org.apache.logging.slf4j.Log4jLoggerFactory]2020-11-13 16:14:30,347 INFO > org.apache.flink.yarn.cli.FlinkYarnSessionCli [] - Dynamic > Property set: > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib2020-11-13 > 16:14:30,347 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli > [] - Dynamic Property set: > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/libUsage with > built-in data generator: StateMachineExample [--error-rate > ] [--sleep ]Usage > with Kafka: StateMachineExample --kafka-topic [--brokers > ]Options for both the above setups: [--backend ] > [--checkpoint-dir ] [--async-checkpoints ] > [--incremental-checkpoints ] [--output OR null for > stdout] > Using standalone source with error rate 0.00 and sleep delay 1 millis > 2020-11-13 16:14:30,706 WARN > org.apache.flink.yarn.configuration.YarnLogConfigUtil [] - The > configuration directory ('/data1/app/flink-1.12-SNAPSHOT/conf') already > contains a LOG4J config file.If you want to use logback, then please delete > or rename the log configuration file.2020-11-13 16:14:30,947 INFO > org.apache.hadoop.yarn.client.AHSProxy [] - Connecting > to Application History server at > FAT-hadoopuat-69117.vm.dc01.tech/10.69.1.17:102002020-11-13 16:14:30,958 INFO > org.apache.flink.yarn.YarnClusterDescriptor [] - No path > for the flink jar passed. Using the location of class > org.apache.flink.yarn.YarnClusterDescriptor to locate the jar2020-11-13 > 16:14:31,065 INFO > org.apache.hadoop.yarn.client.ConfiguredRMFailoverProxyProvider [] - Failing > over to rm22020-11-13 16:14:31,130 INFO > org.apache.flink.yarn.YarnClusterDescriptor [] - The > configured JobManager memory is 3072 MB. YARN will allocate 4096 MB to make > up an integer multiple of its minimum allocation memory (2048 MB, configured > via 'yarn.scheduler.minimum-allocation-mb'). The extra 1024 MB may not be > used by Flink.2020-11-13 16:14:31,130 INFO > org.apache.flink.yarn.YarnClusterDescriptor [] - The > configured TaskManager memory is 3072 MB. YARN will allocate 4096 MB to make > up an integer multiple of its minimum allocation memory (2048 MB, configured > via 'yarn.scheduler.minimum-allocation-mb'). The extra 1024 MB may not be > used by Flink.2020-11-13 16:14:31,130 INFO > org.apache.flink.yarn.YarnClusterDescriptor [] - Cluster > specification: ClusterSpecification{masterMemoryMB=3072, > taskManagerMemoryMB=3072, slotsPerTaskManager=2}2020-11-13 16:14:31,681 WARN > org.apache.hadoop.hdfs.shortcircuit.DomainSocketFactory [] - The > short-circuit local reads feature cannot be
[jira] [Commented] (FLINK-20143) use `yarn.provided.lib.dirs` config deploy job failed in yarn per job mode
[ https://issues.apache.org/jira/browse/FLINK-20143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17231360#comment-17231360 ] Kostas Kloudas commented on FLINK-20143: Can't you use {{yarn logs -applicationId application_1599741232083_22011}} to get the logs as the message says? > use `yarn.provided.lib.dirs` config deploy job failed in yarn per job mode > -- > > Key: FLINK-20143 > URL: https://issues.apache.org/jira/browse/FLINK-20143 > Project: Flink > Issue Type: Bug > Components: Client / Job Submission, Deployment / YARN >Affects Versions: 1.12.0 >Reporter: zhisheng >Priority: Major > Attachments: image-2020-11-13-17-21-47-751.png, > image-2020-11-13-17-22-06-111.png > > > use follow command deploy flink job to yarn failed > {code:java} > ./bin/flink run -m yarn-cluster -d -ynm flink-1.12-test -ytm 3g -yjm 3g -yD > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib > ./examples/streaming/StateMachineExample.jar > {code} > log: > {code:java} > $ ./bin/flink run -m yarn-cluster -d -ynm flink-1.12-test -ytm 3g -yjm 3g -yD > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib > ./examples/streaming/StateMachineExample.jar$ ./bin/flink run -m yarn-cluster > -d -ynm flink-1.12-test -ytm 3g -yjm 3g -yD > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib > ./examples/streaming/StateMachineExample.jarSLF4J: Class path contains > multiple SLF4J bindings.SLF4J: Found binding in > [jar:file:/data1/app/flink-1.12-SNAPSHOT/lib/log4j-slf4j-impl-2.12.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: > Found binding in > [jar:file:/data1/app/hadoop-2.7.3-snappy-32core12disk/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: > Found binding in > [jar:file:/data1/app/hadoop-2.7.3-snappy-32core12disk/share/hadoop/tools/lib/hadoop-aliyun-2.9.2-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: > See http://www.slf4j.org/codes.html#multiple_bindings for an > explanation.SLF4J: Actual binding is of type > [org.apache.logging.slf4j.Log4jLoggerFactory]2020-11-13 16:14:30,347 INFO > org.apache.flink.yarn.cli.FlinkYarnSessionCli [] - Dynamic > Property set: > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib2020-11-13 > 16:14:30,347 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli > [] - Dynamic Property set: > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/libUsage with > built-in data generator: StateMachineExample [--error-rate > ] [--sleep ]Usage > with Kafka: StateMachineExample --kafka-topic [--brokers > ]Options for both the above setups: [--backend ] > [--checkpoint-dir ] [--async-checkpoints ] > [--incremental-checkpoints ] [--output OR null for > stdout] > Using standalone source with error rate 0.00 and sleep delay 1 millis > 2020-11-13 16:14:30,706 WARN > org.apache.flink.yarn.configuration.YarnLogConfigUtil [] - The > configuration directory ('/data1/app/flink-1.12-SNAPSHOT/conf') already > contains a LOG4J config file.If you want to use logback, then please delete > or rename the log configuration file.2020-11-13 16:14:30,947 INFO > org.apache.hadoop.yarn.client.AHSProxy [] - Connecting > to Application History server at > FAT-hadoopuat-69117.vm.dc01.tech/10.69.1.17:102002020-11-13 16:14:30,958 INFO > org.apache.flink.yarn.YarnClusterDescriptor [] - No path > for the flink jar passed. Using the location of class > org.apache.flink.yarn.YarnClusterDescriptor to locate the jar2020-11-13 > 16:14:31,065 INFO > org.apache.hadoop.yarn.client.ConfiguredRMFailoverProxyProvider [] - Failing > over to rm22020-11-13 16:14:31,130 INFO > org.apache.flink.yarn.YarnClusterDescriptor [] - The > configured JobManager memory is 3072 MB. YARN will allocate 4096 MB to make > up an integer multiple of its minimum allocation memory (2048 MB, configured > via 'yarn.scheduler.minimum-allocation-mb'). The extra 1024 MB may not be > used by Flink.2020-11-13 16:14:31,130 INFO > org.apache.flink.yarn.YarnClusterDescriptor [] - The > configured TaskManager memory is 3072 MB. YARN will allocate 4096 MB to make > up an integer multiple of its minimum allocation memory (2048 MB, configured > via 'yarn.scheduler.minimum-allocation-mb'). The extra 1024 MB may not be > used by Flink.2020-11-13 16:14:31,130 INFO > org.apache.flink.yarn.YarnClusterDescriptor [] - Cluster > specification: ClusterSpecification{masterMemoryMB=3072, > taskManagerMemoryMB=3072, slotsPerTaskManager=2}2020-11-13 16:14:31,681 WARN > org.apache.hadoop.hdfs.shortcircuit.DomainSocketFactory [] - The
[jira] [Commented] (FLINK-20143) use `yarn.provided.lib.dirs` config deploy job failed in yarn per job mode
[ https://issues.apache.org/jira/browse/FLINK-20143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17231346#comment-17231346 ] zhisheng commented on FLINK-20143: -- {code:java} $ ./bin/flink run -t yarn-per-job -Dexecution.attached=false -Dyarn.provided.lib.dirs="hdfs:///flink/flink-1.12-SNAPSHOT/lib" ./examples/streaming/StateMachineExample.jar$ ./bin/flink run -t yarn-per-job -Dexecution.attached=false -Dyarn.provided.lib.dirs="hdfs:///flink/flink-1.12-SNAPSHOT/lib" ./examples/streaming/StateMachineExample.jarSLF4J: Class path contains multiple SLF4J bindings.SLF4J: Found binding in [jar:file:/data1/app/flink-1.12-SNAPSHOT/lib/log4j-slf4j-impl-2.12.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: Found binding in [jar:file:/data1/app/hadoop-2.7.3-snappy-32core12disk/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: Found binding in [jar:file:/data1/app/hadoop-2.7.3-snappy-32core12disk/share/hadoop/tools/lib/hadoop-aliyun-2.9.2-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]Usage with built-in data generator: StateMachineExample [--error-rate ] [--sleep ]Usage with Kafka: StateMachineExample --kafka-topic [--brokers ]Options for both the above setups: [--backend ] [--checkpoint-dir ] [--async-checkpoints ] [--incremental-checkpoints ] [--output OR null for stdout] Using standalone source with error rate 0.00 and sleep delay 1 millis 2020-11-13 18:05:51,974 WARN org.apache.flink.yarn.configuration.YarnLogConfigUtil [] - The configuration directory ('/data1/app/flink-1.12-SNAPSHOT/conf') already contains a LOG4J config file.If you want to use logback, then please delete or rename the log configuration file.2020-11-13 18:05:52,202 INFO org.apache.hadoop.yarn.client.AHSProxy [] - Connecting to Application History server at FAT-hadoopuat-69117.vm.dc01.hellocloud.tech/10.69.1.17:102002020-11-13 18:05:52,213 INFO org.apache.flink.yarn.YarnClusterDescriptor [] - No path for the flink jar passed. Using the location of class org.apache.flink.yarn.YarnClusterDescriptor to locate the jar2020-11-13 18:05:52,324 INFO org.apache.hadoop.yarn.client.ConfiguredRMFailoverProxyProvider [] - Failing over to rm22020-11-13 18:05:52,387 INFO org.apache.flink.yarn.YarnClusterDescriptor [] - The configured JobManager memory is 1600 MB. YARN will allocate 2048 MB to make up an integer multiple of its minimum allocation memory (2048 MB, configured via 'yarn.scheduler.minimum-allocation-mb'). The extra 448 MB may not be used by Flink.2020-11-13 18:05:52,388 INFO org.apache.flink.yarn.YarnClusterDescriptor [] - The configured TaskManager memory is 1728 MB. YARN will allocate 2048 MB to make up an integer multiple of its minimum allocation memory (2048 MB, configured via 'yarn.scheduler.minimum-allocation-mb'). The extra 320 MB may not be used by Flink.2020-11-13 18:05:52,388 INFO org.apache.flink.yarn.YarnClusterDescriptor [] - Cluster specification: ClusterSpecification{masterMemoryMB=2048, taskManagerMemoryMB=1728, slotsPerTaskManager=2}2020-11-13 18:05:52,932 WARN org.apache.hadoop.hdfs.shortcircuit.DomainSocketFactory [] - The short-circuit local reads feature cannot be used because libhadoop cannot be loaded.2020-11-13 18:05:55,076 INFO org.apache.flink.yarn.YarnClusterDescriptor [] - Submitting application master application_1599741232083_220112020-11-13 18:05:55,307 INFO org.apache.hadoop.yarn.client.api.impl.YarnClientImpl [] - Submitted application application_1599741232083_220112020-11-13 18:05:55,308 INFO org.apache.flink.yarn.YarnClusterDescriptor [] - Waiting for the cluster to be allocated2020-11-13 18:05:55,310 INFO org.apache.flink.yarn.YarnClusterDescriptor [] - Deploying cluster, current state ACCEPTED The program finished with the following exception: org.apache.flink.client.program.ProgramInvocationException: The main method caused an error: Could not deploy Yarn job cluster. at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:330) at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:198) at org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:114) at org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:743) at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:242) at org.apache.flink.client.cli.CliFrontend.parseAndRun(CliFrontend.java:971) at
[jira] [Commented] (FLINK-20143) use `yarn.provided.lib.dirs` config deploy job failed in yarn per job mode
[ https://issues.apache.org/jira/browse/FLINK-20143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17231326#comment-17231326 ] Kostas Kloudas commented on FLINK-20143: Also the above command seems to be problematic. {code:java} ./bin/flink run -m yarn-cluster -d -Dyarn.provided.lib.dirs="hdfs:///flink/flink-1.12-SNAPSHOT/lib" ./examples/streaming/StateMachineExample.jar {code} What if you use? {code:java} ./bin/flink run -t yarn-per-job -Dexecution.attached=false -Dyarn.provided.lib.dirs="hdfs:///flink/flink-1.12-SNAPSHOT/lib" ./examples/streaming/StateMachineExample.jar {code} and please check the logs to see if the shared dir is picked up or you are shipping everything from the client. > use `yarn.provided.lib.dirs` config deploy job failed in yarn per job mode > -- > > Key: FLINK-20143 > URL: https://issues.apache.org/jira/browse/FLINK-20143 > Project: Flink > Issue Type: Bug > Components: Client / Job Submission, Deployment / YARN >Affects Versions: 1.12.0 >Reporter: zhisheng >Priority: Major > Attachments: image-2020-11-13-17-21-47-751.png, > image-2020-11-13-17-22-06-111.png > > > use follow command deploy flink job to yarn failed > {code:java} > ./bin/flink run -m yarn-cluster -d -ynm flink-1.12-test -ytm 3g -yjm 3g -yD > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib > ./examples/streaming/StateMachineExample.jar > {code} > log: > {code:java} > $ ./bin/flink run -m yarn-cluster -d -ynm flink-1.12-test -ytm 3g -yjm 3g -yD > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib > ./examples/streaming/StateMachineExample.jar$ ./bin/flink run -m yarn-cluster > -d -ynm flink-1.12-test -ytm 3g -yjm 3g -yD > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib > ./examples/streaming/StateMachineExample.jarSLF4J: Class path contains > multiple SLF4J bindings.SLF4J: Found binding in > [jar:file:/data1/app/flink-1.12-SNAPSHOT/lib/log4j-slf4j-impl-2.12.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: > Found binding in > [jar:file:/data1/app/hadoop-2.7.3-snappy-32core12disk/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: > Found binding in > [jar:file:/data1/app/hadoop-2.7.3-snappy-32core12disk/share/hadoop/tools/lib/hadoop-aliyun-2.9.2-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: > See http://www.slf4j.org/codes.html#multiple_bindings for an > explanation.SLF4J: Actual binding is of type > [org.apache.logging.slf4j.Log4jLoggerFactory]2020-11-13 16:14:30,347 INFO > org.apache.flink.yarn.cli.FlinkYarnSessionCli [] - Dynamic > Property set: > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib2020-11-13 > 16:14:30,347 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli > [] - Dynamic Property set: > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/libUsage with > built-in data generator: StateMachineExample [--error-rate > ] [--sleep ]Usage > with Kafka: StateMachineExample --kafka-topic [--brokers > ]Options for both the above setups: [--backend ] > [--checkpoint-dir ] [--async-checkpoints ] > [--incremental-checkpoints ] [--output OR null for > stdout] > Using standalone source with error rate 0.00 and sleep delay 1 millis > 2020-11-13 16:14:30,706 WARN > org.apache.flink.yarn.configuration.YarnLogConfigUtil [] - The > configuration directory ('/data1/app/flink-1.12-SNAPSHOT/conf') already > contains a LOG4J config file.If you want to use logback, then please delete > or rename the log configuration file.2020-11-13 16:14:30,947 INFO > org.apache.hadoop.yarn.client.AHSProxy [] - Connecting > to Application History server at > FAT-hadoopuat-69117.vm.dc01.hellocloud.tech/10.69.1.17:102002020-11-13 > 16:14:30,958 INFO org.apache.flink.yarn.YarnClusterDescriptor > [] - No path for the flink jar passed. Using the location of class > org.apache.flink.yarn.YarnClusterDescriptor to locate the jar2020-11-13 > 16:14:31,065 INFO > org.apache.hadoop.yarn.client.ConfiguredRMFailoverProxyProvider [] - Failing > over to rm22020-11-13 16:14:31,130 INFO > org.apache.flink.yarn.YarnClusterDescriptor [] - The > configured JobManager memory is 3072 MB. YARN will allocate 4096 MB to make > up an integer multiple of its minimum allocation memory (2048 MB, configured > via 'yarn.scheduler.minimum-allocation-mb'). The extra 1024 MB may not be > used by Flink.2020-11-13 16:14:31,130 INFO > org.apache.flink.yarn.YarnClusterDescriptor [] - The > configured TaskManager memory is 3072 MB. YARN will allocate 4096 MB to make > up an integer multiple of its minimum allocation memory
[jira] [Commented] (FLINK-20143) use `yarn.provided.lib.dirs` config deploy job failed in yarn per job mode
[ https://issues.apache.org/jira/browse/FLINK-20143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17231321#comment-17231321 ] zhisheng commented on FLINK-20143: -- Are there any other methods to make job config compatibility?[~kkl0u] > use `yarn.provided.lib.dirs` config deploy job failed in yarn per job mode > -- > > Key: FLINK-20143 > URL: https://issues.apache.org/jira/browse/FLINK-20143 > Project: Flink > Issue Type: Bug > Components: Client / Job Submission, Deployment / YARN >Affects Versions: 1.12.0 >Reporter: zhisheng >Priority: Major > Attachments: image-2020-11-13-17-21-47-751.png, > image-2020-11-13-17-22-06-111.png > > > use follow command deploy flink job to yarn failed > {code:java} > ./bin/flink run -m yarn-cluster -d -ynm flink-1.12-test -ytm 3g -yjm 3g -yD > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib > ./examples/streaming/StateMachineExample.jar > {code} > log: > {code:java} > $ ./bin/flink run -m yarn-cluster -d -ynm flink-1.12-test -ytm 3g -yjm 3g -yD > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib > ./examples/streaming/StateMachineExample.jar$ ./bin/flink run -m yarn-cluster > -d -ynm flink-1.12-test -ytm 3g -yjm 3g -yD > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib > ./examples/streaming/StateMachineExample.jarSLF4J: Class path contains > multiple SLF4J bindings.SLF4J: Found binding in > [jar:file:/data1/app/flink-1.12-SNAPSHOT/lib/log4j-slf4j-impl-2.12.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: > Found binding in > [jar:file:/data1/app/hadoop-2.7.3-snappy-32core12disk/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: > Found binding in > [jar:file:/data1/app/hadoop-2.7.3-snappy-32core12disk/share/hadoop/tools/lib/hadoop-aliyun-2.9.2-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: > See http://www.slf4j.org/codes.html#multiple_bindings for an > explanation.SLF4J: Actual binding is of type > [org.apache.logging.slf4j.Log4jLoggerFactory]2020-11-13 16:14:30,347 INFO > org.apache.flink.yarn.cli.FlinkYarnSessionCli [] - Dynamic > Property set: > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib2020-11-13 > 16:14:30,347 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli > [] - Dynamic Property set: > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/libUsage with > built-in data generator: StateMachineExample [--error-rate > ] [--sleep ]Usage > with Kafka: StateMachineExample --kafka-topic [--brokers > ]Options for both the above setups: [--backend ] > [--checkpoint-dir ] [--async-checkpoints ] > [--incremental-checkpoints ] [--output OR null for > stdout] > Using standalone source with error rate 0.00 and sleep delay 1 millis > 2020-11-13 16:14:30,706 WARN > org.apache.flink.yarn.configuration.YarnLogConfigUtil [] - The > configuration directory ('/data1/app/flink-1.12-SNAPSHOT/conf') already > contains a LOG4J config file.If you want to use logback, then please delete > or rename the log configuration file.2020-11-13 16:14:30,947 INFO > org.apache.hadoop.yarn.client.AHSProxy [] - Connecting > to Application History server at > FAT-hadoopuat-69117.vm.dc01.hellocloud.tech/10.69.1.17:102002020-11-13 > 16:14:30,958 INFO org.apache.flink.yarn.YarnClusterDescriptor > [] - No path for the flink jar passed. Using the location of class > org.apache.flink.yarn.YarnClusterDescriptor to locate the jar2020-11-13 > 16:14:31,065 INFO > org.apache.hadoop.yarn.client.ConfiguredRMFailoverProxyProvider [] - Failing > over to rm22020-11-13 16:14:31,130 INFO > org.apache.flink.yarn.YarnClusterDescriptor [] - The > configured JobManager memory is 3072 MB. YARN will allocate 4096 MB to make > up an integer multiple of its minimum allocation memory (2048 MB, configured > via 'yarn.scheduler.minimum-allocation-mb'). The extra 1024 MB may not be > used by Flink.2020-11-13 16:14:31,130 INFO > org.apache.flink.yarn.YarnClusterDescriptor [] - The > configured TaskManager memory is 3072 MB. YARN will allocate 4096 MB to make > up an integer multiple of its minimum allocation memory (2048 MB, configured > via 'yarn.scheduler.minimum-allocation-mb'). The extra 1024 MB may not be > used by Flink.2020-11-13 16:14:31,130 INFO > org.apache.flink.yarn.YarnClusterDescriptor [] - Cluster > specification: ClusterSpecification{masterMemoryMB=3072, > taskManagerMemoryMB=3072, slotsPerTaskManager=2}2020-11-13 16:14:31,681 WARN > org.apache.hadoop.hdfs.shortcircuit.DomainSocketFactory [] - The > short-circuit local reads feature cannot
[jira] [Commented] (FLINK-20143) use `yarn.provided.lib.dirs` config deploy job failed in yarn per job mode
[ https://issues.apache.org/jira/browse/FLINK-20143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17231319#comment-17231319 ] zhisheng commented on FLINK-20143: -- in our production environment,has many flink job,every job have the -ytm and -yjm -ynm config,if we upgrade to 1.12,It could change a lot [~kkl0u] > use `yarn.provided.lib.dirs` config deploy job failed in yarn per job mode > -- > > Key: FLINK-20143 > URL: https://issues.apache.org/jira/browse/FLINK-20143 > Project: Flink > Issue Type: Bug > Components: Client / Job Submission, Deployment / YARN >Affects Versions: 1.12.0 >Reporter: zhisheng >Priority: Major > Attachments: image-2020-11-13-17-21-47-751.png, > image-2020-11-13-17-22-06-111.png > > > use follow command deploy flink job to yarn failed > {code:java} > ./bin/flink run -m yarn-cluster -d -ynm flink-1.12-test -ytm 3g -yjm 3g -yD > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib > ./examples/streaming/StateMachineExample.jar > {code} > log: > {code:java} > $ ./bin/flink run -m yarn-cluster -d -ynm flink-1.12-test -ytm 3g -yjm 3g -yD > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib > ./examples/streaming/StateMachineExample.jar$ ./bin/flink run -m yarn-cluster > -d -ynm flink-1.12-test -ytm 3g -yjm 3g -yD > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib > ./examples/streaming/StateMachineExample.jarSLF4J: Class path contains > multiple SLF4J bindings.SLF4J: Found binding in > [jar:file:/data1/app/flink-1.12-SNAPSHOT/lib/log4j-slf4j-impl-2.12.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: > Found binding in > [jar:file:/data1/app/hadoop-2.7.3-snappy-32core12disk/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: > Found binding in > [jar:file:/data1/app/hadoop-2.7.3-snappy-32core12disk/share/hadoop/tools/lib/hadoop-aliyun-2.9.2-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: > See http://www.slf4j.org/codes.html#multiple_bindings for an > explanation.SLF4J: Actual binding is of type > [org.apache.logging.slf4j.Log4jLoggerFactory]2020-11-13 16:14:30,347 INFO > org.apache.flink.yarn.cli.FlinkYarnSessionCli [] - Dynamic > Property set: > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib2020-11-13 > 16:14:30,347 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli > [] - Dynamic Property set: > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/libUsage with > built-in data generator: StateMachineExample [--error-rate > ] [--sleep ]Usage > with Kafka: StateMachineExample --kafka-topic [--brokers > ]Options for both the above setups: [--backend ] > [--checkpoint-dir ] [--async-checkpoints ] > [--incremental-checkpoints ] [--output OR null for > stdout] > Using standalone source with error rate 0.00 and sleep delay 1 millis > 2020-11-13 16:14:30,706 WARN > org.apache.flink.yarn.configuration.YarnLogConfigUtil [] - The > configuration directory ('/data1/app/flink-1.12-SNAPSHOT/conf') already > contains a LOG4J config file.If you want to use logback, then please delete > or rename the log configuration file.2020-11-13 16:14:30,947 INFO > org.apache.hadoop.yarn.client.AHSProxy [] - Connecting > to Application History server at > FAT-hadoopuat-69117.vm.dc01.hellocloud.tech/10.69.1.17:102002020-11-13 > 16:14:30,958 INFO org.apache.flink.yarn.YarnClusterDescriptor > [] - No path for the flink jar passed. Using the location of class > org.apache.flink.yarn.YarnClusterDescriptor to locate the jar2020-11-13 > 16:14:31,065 INFO > org.apache.hadoop.yarn.client.ConfiguredRMFailoverProxyProvider [] - Failing > over to rm22020-11-13 16:14:31,130 INFO > org.apache.flink.yarn.YarnClusterDescriptor [] - The > configured JobManager memory is 3072 MB. YARN will allocate 4096 MB to make > up an integer multiple of its minimum allocation memory (2048 MB, configured > via 'yarn.scheduler.minimum-allocation-mb'). The extra 1024 MB may not be > used by Flink.2020-11-13 16:14:31,130 INFO > org.apache.flink.yarn.YarnClusterDescriptor [] - The > configured TaskManager memory is 3072 MB. YARN will allocate 4096 MB to make > up an integer multiple of its minimum allocation memory (2048 MB, configured > via 'yarn.scheduler.minimum-allocation-mb'). The extra 1024 MB may not be > used by Flink.2020-11-13 16:14:31,130 INFO > org.apache.flink.yarn.YarnClusterDescriptor [] - Cluster > specification: ClusterSpecification{masterMemoryMB=3072, > taskManagerMemoryMB=3072, slotsPerTaskManager=2}2020-11-13 16:14:31,681 WARN >
[jira] [Commented] (FLINK-20143) use `yarn.provided.lib.dirs` config deploy job failed in yarn per job mode
[ https://issues.apache.org/jira/browse/FLINK-20143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17231317#comment-17231317 ] zhisheng commented on FLINK-20143: -- {code:java} ./bin/flink run -m yarn-cluster -d -Dyarn.provided.lib.dirs="hdfs:///flink/flink-1.12-SNAPSHOT/lib" ./examples/streaming/StateMachineExample.jar {code} i use this command(remove the -ynm flink-1.12-test -ytm 3g -yjm ), it runs ok > use `yarn.provided.lib.dirs` config deploy job failed in yarn per job mode > -- > > Key: FLINK-20143 > URL: https://issues.apache.org/jira/browse/FLINK-20143 > Project: Flink > Issue Type: Bug > Components: Client / Job Submission, Deployment / YARN >Affects Versions: 1.12.0 >Reporter: zhisheng >Priority: Major > Attachments: image-2020-11-13-17-21-47-751.png, > image-2020-11-13-17-22-06-111.png > > > use follow command deploy flink job to yarn failed > {code:java} > ./bin/flink run -m yarn-cluster -d -ynm flink-1.12-test -ytm 3g -yjm 3g -yD > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib > ./examples/streaming/StateMachineExample.jar > {code} > log: > {code:java} > $ ./bin/flink run -m yarn-cluster -d -ynm flink-1.12-test -ytm 3g -yjm 3g -yD > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib > ./examples/streaming/StateMachineExample.jar$ ./bin/flink run -m yarn-cluster > -d -ynm flink-1.12-test -ytm 3g -yjm 3g -yD > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib > ./examples/streaming/StateMachineExample.jarSLF4J: Class path contains > multiple SLF4J bindings.SLF4J: Found binding in > [jar:file:/data1/app/flink-1.12-SNAPSHOT/lib/log4j-slf4j-impl-2.12.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: > Found binding in > [jar:file:/data1/app/hadoop-2.7.3-snappy-32core12disk/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: > Found binding in > [jar:file:/data1/app/hadoop-2.7.3-snappy-32core12disk/share/hadoop/tools/lib/hadoop-aliyun-2.9.2-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: > See http://www.slf4j.org/codes.html#multiple_bindings for an > explanation.SLF4J: Actual binding is of type > [org.apache.logging.slf4j.Log4jLoggerFactory]2020-11-13 16:14:30,347 INFO > org.apache.flink.yarn.cli.FlinkYarnSessionCli [] - Dynamic > Property set: > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib2020-11-13 > 16:14:30,347 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli > [] - Dynamic Property set: > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/libUsage with > built-in data generator: StateMachineExample [--error-rate > ] [--sleep ]Usage > with Kafka: StateMachineExample --kafka-topic [--brokers > ]Options for both the above setups: [--backend ] > [--checkpoint-dir ] [--async-checkpoints ] > [--incremental-checkpoints ] [--output OR null for > stdout] > Using standalone source with error rate 0.00 and sleep delay 1 millis > 2020-11-13 16:14:30,706 WARN > org.apache.flink.yarn.configuration.YarnLogConfigUtil [] - The > configuration directory ('/data1/app/flink-1.12-SNAPSHOT/conf') already > contains a LOG4J config file.If you want to use logback, then please delete > or rename the log configuration file.2020-11-13 16:14:30,947 INFO > org.apache.hadoop.yarn.client.AHSProxy [] - Connecting > to Application History server at > FAT-hadoopuat-69117.vm.dc01.hellocloud.tech/10.69.1.17:102002020-11-13 > 16:14:30,958 INFO org.apache.flink.yarn.YarnClusterDescriptor > [] - No path for the flink jar passed. Using the location of class > org.apache.flink.yarn.YarnClusterDescriptor to locate the jar2020-11-13 > 16:14:31,065 INFO > org.apache.hadoop.yarn.client.ConfiguredRMFailoverProxyProvider [] - Failing > over to rm22020-11-13 16:14:31,130 INFO > org.apache.flink.yarn.YarnClusterDescriptor [] - The > configured JobManager memory is 3072 MB. YARN will allocate 4096 MB to make > up an integer multiple of its minimum allocation memory (2048 MB, configured > via 'yarn.scheduler.minimum-allocation-mb'). The extra 1024 MB may not be > used by Flink.2020-11-13 16:14:31,130 INFO > org.apache.flink.yarn.YarnClusterDescriptor [] - The > configured TaskManager memory is 3072 MB. YARN will allocate 4096 MB to make > up an integer multiple of its minimum allocation memory (2048 MB, configured > via 'yarn.scheduler.minimum-allocation-mb'). The extra 1024 MB may not be > used by Flink.2020-11-13 16:14:31,130 INFO > org.apache.flink.yarn.YarnClusterDescriptor [] - Cluster > specification: ClusterSpecification{masterMemoryMB=3072, >
[jira] [Commented] (FLINK-20143) use `yarn.provided.lib.dirs` config deploy job failed in yarn per job mode
[ https://issues.apache.org/jira/browse/FLINK-20143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17231306#comment-17231306 ] Kostas Kloudas commented on FLINK-20143: As discussed in the issue, you have to specify the full config option name prefixed by {{-D}} when using the {{GenericCLI}}. This means for example {{-Dtaskmanager.memory.process.size=...}}. > use `yarn.provided.lib.dirs` config deploy job failed in yarn per job mode > -- > > Key: FLINK-20143 > URL: https://issues.apache.org/jira/browse/FLINK-20143 > Project: Flink > Issue Type: Bug > Components: Client / Job Submission, Deployment / YARN >Affects Versions: 1.12.0 >Reporter: zhisheng >Priority: Major > Attachments: image-2020-11-13-17-21-47-751.png, > image-2020-11-13-17-22-06-111.png > > > use follow command deploy flink job to yarn failed > {code:java} > ./bin/flink run -m yarn-cluster -d -ynm flink-1.12-test -ytm 3g -yjm 3g -yD > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib > ./examples/streaming/StateMachineExample.jar > {code} > log: > {code:java} > $ ./bin/flink run -m yarn-cluster -d -ynm flink-1.12-test -ytm 3g -yjm 3g -yD > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib > ./examples/streaming/StateMachineExample.jar$ ./bin/flink run -m yarn-cluster > -d -ynm flink-1.12-test -ytm 3g -yjm 3g -yD > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib > ./examples/streaming/StateMachineExample.jarSLF4J: Class path contains > multiple SLF4J bindings.SLF4J: Found binding in > [jar:file:/data1/app/flink-1.12-SNAPSHOT/lib/log4j-slf4j-impl-2.12.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: > Found binding in > [jar:file:/data1/app/hadoop-2.7.3-snappy-32core12disk/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: > Found binding in > [jar:file:/data1/app/hadoop-2.7.3-snappy-32core12disk/share/hadoop/tools/lib/hadoop-aliyun-2.9.2-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: > See http://www.slf4j.org/codes.html#multiple_bindings for an > explanation.SLF4J: Actual binding is of type > [org.apache.logging.slf4j.Log4jLoggerFactory]2020-11-13 16:14:30,347 INFO > org.apache.flink.yarn.cli.FlinkYarnSessionCli [] - Dynamic > Property set: > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib2020-11-13 > 16:14:30,347 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli > [] - Dynamic Property set: > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/libUsage with > built-in data generator: StateMachineExample [--error-rate > ] [--sleep ]Usage > with Kafka: StateMachineExample --kafka-topic [--brokers > ]Options for both the above setups: [--backend ] > [--checkpoint-dir ] [--async-checkpoints ] > [--incremental-checkpoints ] [--output OR null for > stdout] > Using standalone source with error rate 0.00 and sleep delay 1 millis > 2020-11-13 16:14:30,706 WARN > org.apache.flink.yarn.configuration.YarnLogConfigUtil [] - The > configuration directory ('/data1/app/flink-1.12-SNAPSHOT/conf') already > contains a LOG4J config file.If you want to use logback, then please delete > or rename the log configuration file.2020-11-13 16:14:30,947 INFO > org.apache.hadoop.yarn.client.AHSProxy [] - Connecting > to Application History server at > FAT-hadoopuat-69117.vm.dc01.hellocloud.tech/10.69.1.17:102002020-11-13 > 16:14:30,958 INFO org.apache.flink.yarn.YarnClusterDescriptor > [] - No path for the flink jar passed. Using the location of class > org.apache.flink.yarn.YarnClusterDescriptor to locate the jar2020-11-13 > 16:14:31,065 INFO > org.apache.hadoop.yarn.client.ConfiguredRMFailoverProxyProvider [] - Failing > over to rm22020-11-13 16:14:31,130 INFO > org.apache.flink.yarn.YarnClusterDescriptor [] - The > configured JobManager memory is 3072 MB. YARN will allocate 4096 MB to make > up an integer multiple of its minimum allocation memory (2048 MB, configured > via 'yarn.scheduler.minimum-allocation-mb'). The extra 1024 MB may not be > used by Flink.2020-11-13 16:14:31,130 INFO > org.apache.flink.yarn.YarnClusterDescriptor [] - The > configured TaskManager memory is 3072 MB. YARN will allocate 4096 MB to make > up an integer multiple of its minimum allocation memory (2048 MB, configured > via 'yarn.scheduler.minimum-allocation-mb'). The extra 1024 MB may not be > used by Flink.2020-11-13 16:14:31,130 INFO > org.apache.flink.yarn.YarnClusterDescriptor [] - Cluster > specification: ClusterSpecification{masterMemoryMB=3072, > taskManagerMemoryMB=3072, slotsPerTaskManager=2}2020-11-13
[jira] [Commented] (FLINK-20143) use `yarn.provided.lib.dirs` config deploy job failed in yarn per job mode
[ https://issues.apache.org/jira/browse/FLINK-20143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17231301#comment-17231301 ] Kostas Kloudas commented on FLINK-20143: I am not sure if I can figure out what is happening from what is here. > use `yarn.provided.lib.dirs` config deploy job failed in yarn per job mode > -- > > Key: FLINK-20143 > URL: https://issues.apache.org/jira/browse/FLINK-20143 > Project: Flink > Issue Type: Bug > Components: Client / Job Submission, Deployment / YARN >Affects Versions: 1.12.0 >Reporter: zhisheng >Priority: Major > Attachments: image-2020-11-13-17-21-47-751.png, > image-2020-11-13-17-22-06-111.png > > > use follow command deploy flink job to yarn failed > {code:java} > ./bin/flink run -m yarn-cluster -d -ynm flink-1.12-test -ytm 3g -yjm 3g -yD > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib > ./examples/streaming/StateMachineExample.jar > {code} > log: > {code:java} > $ ./bin/flink run -m yarn-cluster -d -ynm flink-1.12-test -ytm 3g -yjm 3g -yD > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib > ./examples/streaming/StateMachineExample.jar$ ./bin/flink run -m yarn-cluster > -d -ynm flink-1.12-test -ytm 3g -yjm 3g -yD > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib > ./examples/streaming/StateMachineExample.jarSLF4J: Class path contains > multiple SLF4J bindings.SLF4J: Found binding in > [jar:file:/data1/app/flink-1.12-SNAPSHOT/lib/log4j-slf4j-impl-2.12.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: > Found binding in > [jar:file:/data1/app/hadoop-2.7.3-snappy-32core12disk/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: > Found binding in > [jar:file:/data1/app/hadoop-2.7.3-snappy-32core12disk/share/hadoop/tools/lib/hadoop-aliyun-2.9.2-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: > See http://www.slf4j.org/codes.html#multiple_bindings for an > explanation.SLF4J: Actual binding is of type > [org.apache.logging.slf4j.Log4jLoggerFactory]2020-11-13 16:14:30,347 INFO > org.apache.flink.yarn.cli.FlinkYarnSessionCli [] - Dynamic > Property set: > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib2020-11-13 > 16:14:30,347 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli > [] - Dynamic Property set: > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/libUsage with > built-in data generator: StateMachineExample [--error-rate > ] [--sleep ]Usage > with Kafka: StateMachineExample --kafka-topic [--brokers > ]Options for both the above setups: [--backend ] > [--checkpoint-dir ] [--async-checkpoints ] > [--incremental-checkpoints ] [--output OR null for > stdout] > Using standalone source with error rate 0.00 and sleep delay 1 millis > 2020-11-13 16:14:30,706 WARN > org.apache.flink.yarn.configuration.YarnLogConfigUtil [] - The > configuration directory ('/data1/app/flink-1.12-SNAPSHOT/conf') already > contains a LOG4J config file.If you want to use logback, then please delete > or rename the log configuration file.2020-11-13 16:14:30,947 INFO > org.apache.hadoop.yarn.client.AHSProxy [] - Connecting > to Application History server at > FAT-hadoopuat-69117.vm.dc01.hellocloud.tech/10.69.1.17:102002020-11-13 > 16:14:30,958 INFO org.apache.flink.yarn.YarnClusterDescriptor > [] - No path for the flink jar passed. Using the location of class > org.apache.flink.yarn.YarnClusterDescriptor to locate the jar2020-11-13 > 16:14:31,065 INFO > org.apache.hadoop.yarn.client.ConfiguredRMFailoverProxyProvider [] - Failing > over to rm22020-11-13 16:14:31,130 INFO > org.apache.flink.yarn.YarnClusterDescriptor [] - The > configured JobManager memory is 3072 MB. YARN will allocate 4096 MB to make > up an integer multiple of its minimum allocation memory (2048 MB, configured > via 'yarn.scheduler.minimum-allocation-mb'). The extra 1024 MB may not be > used by Flink.2020-11-13 16:14:31,130 INFO > org.apache.flink.yarn.YarnClusterDescriptor [] - The > configured TaskManager memory is 3072 MB. YARN will allocate 4096 MB to make > up an integer multiple of its minimum allocation memory (2048 MB, configured > via 'yarn.scheduler.minimum-allocation-mb'). The extra 1024 MB may not be > used by Flink.2020-11-13 16:14:31,130 INFO > org.apache.flink.yarn.YarnClusterDescriptor [] - Cluster > specification: ClusterSpecification{masterMemoryMB=3072, > taskManagerMemoryMB=3072, slotsPerTaskManager=2}2020-11-13 16:14:31,681 WARN > org.apache.hadoop.hdfs.shortcircuit.DomainSocketFactory [] - The > short-circuit local reads
[jira] [Commented] (FLINK-20143) use `yarn.provided.lib.dirs` config deploy job failed in yarn per job mode
[ https://issues.apache.org/jira/browse/FLINK-20143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17231300#comment-17231300 ] zhisheng commented on FLINK-20143: -- [~kkl0u] yes, -ytm and -yjm does not take effect,i create a issue some days ago https://issues.apache.org/jira/browse/FLINK-19973 > use `yarn.provided.lib.dirs` config deploy job failed in yarn per job mode > -- > > Key: FLINK-20143 > URL: https://issues.apache.org/jira/browse/FLINK-20143 > Project: Flink > Issue Type: Bug > Components: Client / Job Submission, Deployment / YARN >Affects Versions: 1.12.0 >Reporter: zhisheng >Priority: Major > Attachments: image-2020-11-13-17-21-47-751.png, > image-2020-11-13-17-22-06-111.png > > > use follow command deploy flink job to yarn failed > {code:java} > ./bin/flink run -m yarn-cluster -d -ynm flink-1.12-test -ytm 3g -yjm 3g -yD > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib > ./examples/streaming/StateMachineExample.jar > {code} > log: > {code:java} > $ ./bin/flink run -m yarn-cluster -d -ynm flink-1.12-test -ytm 3g -yjm 3g -yD > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib > ./examples/streaming/StateMachineExample.jar$ ./bin/flink run -m yarn-cluster > -d -ynm flink-1.12-test -ytm 3g -yjm 3g -yD > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib > ./examples/streaming/StateMachineExample.jarSLF4J: Class path contains > multiple SLF4J bindings.SLF4J: Found binding in > [jar:file:/data1/app/flink-1.12-SNAPSHOT/lib/log4j-slf4j-impl-2.12.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: > Found binding in > [jar:file:/data1/app/hadoop-2.7.3-snappy-32core12disk/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: > Found binding in > [jar:file:/data1/app/hadoop-2.7.3-snappy-32core12disk/share/hadoop/tools/lib/hadoop-aliyun-2.9.2-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: > See http://www.slf4j.org/codes.html#multiple_bindings for an > explanation.SLF4J: Actual binding is of type > [org.apache.logging.slf4j.Log4jLoggerFactory]2020-11-13 16:14:30,347 INFO > org.apache.flink.yarn.cli.FlinkYarnSessionCli [] - Dynamic > Property set: > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib2020-11-13 > 16:14:30,347 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli > [] - Dynamic Property set: > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/libUsage with > built-in data generator: StateMachineExample [--error-rate > ] [--sleep ]Usage > with Kafka: StateMachineExample --kafka-topic [--brokers > ]Options for both the above setups: [--backend ] > [--checkpoint-dir ] [--async-checkpoints ] > [--incremental-checkpoints ] [--output OR null for > stdout] > Using standalone source with error rate 0.00 and sleep delay 1 millis > 2020-11-13 16:14:30,706 WARN > org.apache.flink.yarn.configuration.YarnLogConfigUtil [] - The > configuration directory ('/data1/app/flink-1.12-SNAPSHOT/conf') already > contains a LOG4J config file.If you want to use logback, then please delete > or rename the log configuration file.2020-11-13 16:14:30,947 INFO > org.apache.hadoop.yarn.client.AHSProxy [] - Connecting > to Application History server at > FAT-hadoopuat-69117.vm.dc01.hellocloud.tech/10.69.1.17:102002020-11-13 > 16:14:30,958 INFO org.apache.flink.yarn.YarnClusterDescriptor > [] - No path for the flink jar passed. Using the location of class > org.apache.flink.yarn.YarnClusterDescriptor to locate the jar2020-11-13 > 16:14:31,065 INFO > org.apache.hadoop.yarn.client.ConfiguredRMFailoverProxyProvider [] - Failing > over to rm22020-11-13 16:14:31,130 INFO > org.apache.flink.yarn.YarnClusterDescriptor [] - The > configured JobManager memory is 3072 MB. YARN will allocate 4096 MB to make > up an integer multiple of its minimum allocation memory (2048 MB, configured > via 'yarn.scheduler.minimum-allocation-mb'). The extra 1024 MB may not be > used by Flink.2020-11-13 16:14:31,130 INFO > org.apache.flink.yarn.YarnClusterDescriptor [] - The > configured TaskManager memory is 3072 MB. YARN will allocate 4096 MB to make > up an integer multiple of its minimum allocation memory (2048 MB, configured > via 'yarn.scheduler.minimum-allocation-mb'). The extra 1024 MB may not be > used by Flink.2020-11-13 16:14:31,130 INFO > org.apache.flink.yarn.YarnClusterDescriptor [] - Cluster > specification: ClusterSpecification{masterMemoryMB=3072, > taskManagerMemoryMB=3072, slotsPerTaskManager=2}2020-11-13 16:14:31,681 WARN >
[jira] [Commented] (FLINK-20143) use `yarn.provided.lib.dirs` config deploy job failed in yarn per job mode
[ https://issues.apache.org/jira/browse/FLINK-20143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17231287#comment-17231287 ] zhisheng commented on FLINK-20143: -- !image-2020-11-13-17-21-47-751.png! !image-2020-11-13-17-22-06-111.png! > use `yarn.provided.lib.dirs` config deploy job failed in yarn per job mode > -- > > Key: FLINK-20143 > URL: https://issues.apache.org/jira/browse/FLINK-20143 > Project: Flink > Issue Type: Bug > Components: Client / Job Submission, Deployment / YARN >Affects Versions: 1.12.0 >Reporter: zhisheng >Priority: Major > Attachments: image-2020-11-13-17-21-47-751.png, > image-2020-11-13-17-22-06-111.png > > > use follow command deploy flink job to yarn failed > {code:java} > ./bin/flink run -m yarn-cluster -d -ynm flink-1.12-test -ytm 3g -yjm 3g -yD > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib > ./examples/streaming/StateMachineExample.jar > {code} > log: > {code:java} > $ ./bin/flink run -m yarn-cluster -d -ynm flink-1.12-test -ytm 3g -yjm 3g -yD > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib > ./examples/streaming/StateMachineExample.jar$ ./bin/flink run -m yarn-cluster > -d -ynm flink-1.12-test -ytm 3g -yjm 3g -yD > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib > ./examples/streaming/StateMachineExample.jarSLF4J: Class path contains > multiple SLF4J bindings.SLF4J: Found binding in > [jar:file:/data1/app/flink-1.12-SNAPSHOT/lib/log4j-slf4j-impl-2.12.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: > Found binding in > [jar:file:/data1/app/hadoop-2.7.3-snappy-32core12disk/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: > Found binding in > [jar:file:/data1/app/hadoop-2.7.3-snappy-32core12disk/share/hadoop/tools/lib/hadoop-aliyun-2.9.2-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: > See http://www.slf4j.org/codes.html#multiple_bindings for an > explanation.SLF4J: Actual binding is of type > [org.apache.logging.slf4j.Log4jLoggerFactory]2020-11-13 16:14:30,347 INFO > org.apache.flink.yarn.cli.FlinkYarnSessionCli [] - Dynamic > Property set: > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib2020-11-13 > 16:14:30,347 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli > [] - Dynamic Property set: > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/libUsage with > built-in data generator: StateMachineExample [--error-rate > ] [--sleep ]Usage > with Kafka: StateMachineExample --kafka-topic [--brokers > ]Options for both the above setups: [--backend ] > [--checkpoint-dir ] [--async-checkpoints ] > [--incremental-checkpoints ] [--output OR null for > stdout] > Using standalone source with error rate 0.00 and sleep delay 1 millis > 2020-11-13 16:14:30,706 WARN > org.apache.flink.yarn.configuration.YarnLogConfigUtil [] - The > configuration directory ('/data1/app/flink-1.12-SNAPSHOT/conf') already > contains a LOG4J config file.If you want to use logback, then please delete > or rename the log configuration file.2020-11-13 16:14:30,947 INFO > org.apache.hadoop.yarn.client.AHSProxy [] - Connecting > to Application History server at > FAT-hadoopuat-69117.vm.dc01.hellocloud.tech/10.69.1.17:102002020-11-13 > 16:14:30,958 INFO org.apache.flink.yarn.YarnClusterDescriptor > [] - No path for the flink jar passed. Using the location of class > org.apache.flink.yarn.YarnClusterDescriptor to locate the jar2020-11-13 > 16:14:31,065 INFO > org.apache.hadoop.yarn.client.ConfiguredRMFailoverProxyProvider [] - Failing > over to rm22020-11-13 16:14:31,130 INFO > org.apache.flink.yarn.YarnClusterDescriptor [] - The > configured JobManager memory is 3072 MB. YARN will allocate 4096 MB to make > up an integer multiple of its minimum allocation memory (2048 MB, configured > via 'yarn.scheduler.minimum-allocation-mb'). The extra 1024 MB may not be > used by Flink.2020-11-13 16:14:31,130 INFO > org.apache.flink.yarn.YarnClusterDescriptor [] - The > configured TaskManager memory is 3072 MB. YARN will allocate 4096 MB to make > up an integer multiple of its minimum allocation memory (2048 MB, configured > via 'yarn.scheduler.minimum-allocation-mb'). The extra 1024 MB may not be > used by Flink.2020-11-13 16:14:31,130 INFO > org.apache.flink.yarn.YarnClusterDescriptor [] - Cluster > specification: ClusterSpecification{masterMemoryMB=3072, > taskManagerMemoryMB=3072, slotsPerTaskManager=2}2020-11-13 16:14:31,681 WARN > org.apache.hadoop.hdfs.shortcircuit.DomainSocketFactory [] - The > short-circuit local reads feature
[jira] [Commented] (FLINK-20143) use `yarn.provided.lib.dirs` config deploy job failed in yarn per job mode
[ https://issues.apache.org/jira/browse/FLINK-20143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17231284#comment-17231284 ] zhisheng commented on FLINK-20143: -- [~kkl0u] it does not have jobmanager log and taskmanager log > use `yarn.provided.lib.dirs` config deploy job failed in yarn per job mode > -- > > Key: FLINK-20143 > URL: https://issues.apache.org/jira/browse/FLINK-20143 > Project: Flink > Issue Type: Bug > Components: Client / Job Submission, Deployment / YARN >Affects Versions: 1.12.0 >Reporter: zhisheng >Priority: Major > > use follow command deploy flink job to yarn failed > {code:java} > ./bin/flink run -m yarn-cluster -d -ynm flink-1.12-test -ytm 3g -yjm 3g -yD > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib > ./examples/streaming/StateMachineExample.jar > {code} > log: > {code:java} > $ ./bin/flink run -m yarn-cluster -d -ynm flink-1.12-test -ytm 3g -yjm 3g -yD > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib > ./examples/streaming/StateMachineExample.jar$ ./bin/flink run -m yarn-cluster > -d -ynm flink-1.12-test -ytm 3g -yjm 3g -yD > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib > ./examples/streaming/StateMachineExample.jarSLF4J: Class path contains > multiple SLF4J bindings.SLF4J: Found binding in > [jar:file:/data1/app/flink-1.12-SNAPSHOT/lib/log4j-slf4j-impl-2.12.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: > Found binding in > [jar:file:/data1/app/hadoop-2.7.3-snappy-32core12disk/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: > Found binding in > [jar:file:/data1/app/hadoop-2.7.3-snappy-32core12disk/share/hadoop/tools/lib/hadoop-aliyun-2.9.2-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: > See http://www.slf4j.org/codes.html#multiple_bindings for an > explanation.SLF4J: Actual binding is of type > [org.apache.logging.slf4j.Log4jLoggerFactory]2020-11-13 16:14:30,347 INFO > org.apache.flink.yarn.cli.FlinkYarnSessionCli [] - Dynamic > Property set: > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib2020-11-13 > 16:14:30,347 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli > [] - Dynamic Property set: > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/libUsage with > built-in data generator: StateMachineExample [--error-rate > ] [--sleep ]Usage > with Kafka: StateMachineExample --kafka-topic [--brokers > ]Options for both the above setups: [--backend ] > [--checkpoint-dir ] [--async-checkpoints ] > [--incremental-checkpoints ] [--output OR null for > stdout] > Using standalone source with error rate 0.00 and sleep delay 1 millis > 2020-11-13 16:14:30,706 WARN > org.apache.flink.yarn.configuration.YarnLogConfigUtil [] - The > configuration directory ('/data1/app/flink-1.12-SNAPSHOT/conf') already > contains a LOG4J config file.If you want to use logback, then please delete > or rename the log configuration file.2020-11-13 16:14:30,947 INFO > org.apache.hadoop.yarn.client.AHSProxy [] - Connecting > to Application History server at > FAT-hadoopuat-69117.vm.dc01.hellocloud.tech/10.69.1.17:102002020-11-13 > 16:14:30,958 INFO org.apache.flink.yarn.YarnClusterDescriptor > [] - No path for the flink jar passed. Using the location of class > org.apache.flink.yarn.YarnClusterDescriptor to locate the jar2020-11-13 > 16:14:31,065 INFO > org.apache.hadoop.yarn.client.ConfiguredRMFailoverProxyProvider [] - Failing > over to rm22020-11-13 16:14:31,130 INFO > org.apache.flink.yarn.YarnClusterDescriptor [] - The > configured JobManager memory is 3072 MB. YARN will allocate 4096 MB to make > up an integer multiple of its minimum allocation memory (2048 MB, configured > via 'yarn.scheduler.minimum-allocation-mb'). The extra 1024 MB may not be > used by Flink.2020-11-13 16:14:31,130 INFO > org.apache.flink.yarn.YarnClusterDescriptor [] - The > configured TaskManager memory is 3072 MB. YARN will allocate 4096 MB to make > up an integer multiple of its minimum allocation memory (2048 MB, configured > via 'yarn.scheduler.minimum-allocation-mb'). The extra 1024 MB may not be > used by Flink.2020-11-13 16:14:31,130 INFO > org.apache.flink.yarn.YarnClusterDescriptor [] - Cluster > specification: ClusterSpecification{masterMemoryMB=3072, > taskManagerMemoryMB=3072, slotsPerTaskManager=2}2020-11-13 16:14:31,681 WARN > org.apache.hadoop.hdfs.shortcircuit.DomainSocketFactory [] - The > short-circuit local reads feature cannot be used because libhadoop cannot be > loaded.2020-11-13 16:14:33,417 INFO >
[jira] [Commented] (FLINK-20143) use `yarn.provided.lib.dirs` config deploy job failed in yarn per job mode
[ https://issues.apache.org/jira/browse/FLINK-20143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17231283#comment-17231283 ] Kostas Kloudas commented on FLINK-20143: Also I think that your second command is not correct. You are using {{-t}} which activates the {{GenericCLI}} but then you specify parameters using the {{YarnSessionCLI}} convention of putting a {{-y}} as a prefix. Can you verify if the memory specifications you put are picked up? > use `yarn.provided.lib.dirs` config deploy job failed in yarn per job mode > -- > > Key: FLINK-20143 > URL: https://issues.apache.org/jira/browse/FLINK-20143 > Project: Flink > Issue Type: Bug > Components: Client / Job Submission, Deployment / YARN >Affects Versions: 1.12.0 >Reporter: zhisheng >Priority: Major > > use follow command deploy flink job to yarn failed > {code:java} > ./bin/flink run -m yarn-cluster -d -ynm flink-1.12-test -ytm 3g -yjm 3g -yD > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib > ./examples/streaming/StateMachineExample.jar > {code} > log: > {code:java} > $ ./bin/flink run -m yarn-cluster -d -ynm flink-1.12-test -ytm 3g -yjm 3g -yD > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib > ./examples/streaming/StateMachineExample.jar$ ./bin/flink run -m yarn-cluster > -d -ynm flink-1.12-test -ytm 3g -yjm 3g -yD > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib > ./examples/streaming/StateMachineExample.jarSLF4J: Class path contains > multiple SLF4J bindings.SLF4J: Found binding in > [jar:file:/data1/app/flink-1.12-SNAPSHOT/lib/log4j-slf4j-impl-2.12.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: > Found binding in > [jar:file:/data1/app/hadoop-2.7.3-snappy-32core12disk/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: > Found binding in > [jar:file:/data1/app/hadoop-2.7.3-snappy-32core12disk/share/hadoop/tools/lib/hadoop-aliyun-2.9.2-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: > See http://www.slf4j.org/codes.html#multiple_bindings for an > explanation.SLF4J: Actual binding is of type > [org.apache.logging.slf4j.Log4jLoggerFactory]2020-11-13 16:14:30,347 INFO > org.apache.flink.yarn.cli.FlinkYarnSessionCli [] - Dynamic > Property set: > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib2020-11-13 > 16:14:30,347 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli > [] - Dynamic Property set: > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/libUsage with > built-in data generator: StateMachineExample [--error-rate > ] [--sleep ]Usage > with Kafka: StateMachineExample --kafka-topic [--brokers > ]Options for both the above setups: [--backend ] > [--checkpoint-dir ] [--async-checkpoints ] > [--incremental-checkpoints ] [--output OR null for > stdout] > Using standalone source with error rate 0.00 and sleep delay 1 millis > 2020-11-13 16:14:30,706 WARN > org.apache.flink.yarn.configuration.YarnLogConfigUtil [] - The > configuration directory ('/data1/app/flink-1.12-SNAPSHOT/conf') already > contains a LOG4J config file.If you want to use logback, then please delete > or rename the log configuration file.2020-11-13 16:14:30,947 INFO > org.apache.hadoop.yarn.client.AHSProxy [] - Connecting > to Application History server at > FAT-hadoopuat-69117.vm.dc01.hellocloud.tech/10.69.1.17:102002020-11-13 > 16:14:30,958 INFO org.apache.flink.yarn.YarnClusterDescriptor > [] - No path for the flink jar passed. Using the location of class > org.apache.flink.yarn.YarnClusterDescriptor to locate the jar2020-11-13 > 16:14:31,065 INFO > org.apache.hadoop.yarn.client.ConfiguredRMFailoverProxyProvider [] - Failing > over to rm22020-11-13 16:14:31,130 INFO > org.apache.flink.yarn.YarnClusterDescriptor [] - The > configured JobManager memory is 3072 MB. YARN will allocate 4096 MB to make > up an integer multiple of its minimum allocation memory (2048 MB, configured > via 'yarn.scheduler.minimum-allocation-mb'). The extra 1024 MB may not be > used by Flink.2020-11-13 16:14:31,130 INFO > org.apache.flink.yarn.YarnClusterDescriptor [] - The > configured TaskManager memory is 3072 MB. YARN will allocate 4096 MB to make > up an integer multiple of its minimum allocation memory (2048 MB, configured > via 'yarn.scheduler.minimum-allocation-mb'). The extra 1024 MB may not be > used by Flink.2020-11-13 16:14:31,130 INFO > org.apache.flink.yarn.YarnClusterDescriptor [] - Cluster > specification: ClusterSpecification{masterMemoryMB=3072, > taskManagerMemoryMB=3072, slotsPerTaskManager=2}2020-11-13
[jira] [Commented] (FLINK-20143) use `yarn.provided.lib.dirs` config deploy job failed in yarn per job mode
[ https://issues.apache.org/jira/browse/FLINK-20143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17231276#comment-17231276 ] Kostas Kloudas commented on FLINK-20143: Can you share with us the job manager and task manager logs [~zhisheng]? This may help figuring out what is happening. > use `yarn.provided.lib.dirs` config deploy job failed in yarn per job mode > -- > > Key: FLINK-20143 > URL: https://issues.apache.org/jira/browse/FLINK-20143 > Project: Flink > Issue Type: Bug > Components: Client / Job Submission, Deployment / YARN >Affects Versions: 1.12.0 >Reporter: zhisheng >Priority: Major > > use follow command deploy flink job to yarn failed > {code:java} > ./bin/flink run -m yarn-cluster -d -ynm flink-1.12-test -ytm 3g -yjm 3g -yD > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib > ./examples/streaming/StateMachineExample.jar > {code} > log: > {code:java} > $ ./bin/flink run -m yarn-cluster -d -ynm flink-1.12-test -ytm 3g -yjm 3g -yD > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib > ./examples/streaming/StateMachineExample.jar$ ./bin/flink run -m yarn-cluster > -d -ynm flink-1.12-test -ytm 3g -yjm 3g -yD > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib > ./examples/streaming/StateMachineExample.jarSLF4J: Class path contains > multiple SLF4J bindings.SLF4J: Found binding in > [jar:file:/data1/app/flink-1.12-SNAPSHOT/lib/log4j-slf4j-impl-2.12.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: > Found binding in > [jar:file:/data1/app/hadoop-2.7.3-snappy-32core12disk/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: > Found binding in > [jar:file:/data1/app/hadoop-2.7.3-snappy-32core12disk/share/hadoop/tools/lib/hadoop-aliyun-2.9.2-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J: > See http://www.slf4j.org/codes.html#multiple_bindings for an > explanation.SLF4J: Actual binding is of type > [org.apache.logging.slf4j.Log4jLoggerFactory]2020-11-13 16:14:30,347 INFO > org.apache.flink.yarn.cli.FlinkYarnSessionCli [] - Dynamic > Property set: > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/lib2020-11-13 > 16:14:30,347 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli > [] - Dynamic Property set: > yarn.provided.lib.dirs=hdfs:///flink/flink-1.12-SNAPSHOT/libUsage with > built-in data generator: StateMachineExample [--error-rate > ] [--sleep ]Usage > with Kafka: StateMachineExample --kafka-topic [--brokers > ]Options for both the above setups: [--backend ] > [--checkpoint-dir ] [--async-checkpoints ] > [--incremental-checkpoints ] [--output OR null for > stdout] > Using standalone source with error rate 0.00 and sleep delay 1 millis > 2020-11-13 16:14:30,706 WARN > org.apache.flink.yarn.configuration.YarnLogConfigUtil [] - The > configuration directory ('/data1/app/flink-1.12-SNAPSHOT/conf') already > contains a LOG4J config file.If you want to use logback, then please delete > or rename the log configuration file.2020-11-13 16:14:30,947 INFO > org.apache.hadoop.yarn.client.AHSProxy [] - Connecting > to Application History server at > FAT-hadoopuat-69117.vm.dc01.hellocloud.tech/10.69.1.17:102002020-11-13 > 16:14:30,958 INFO org.apache.flink.yarn.YarnClusterDescriptor > [] - No path for the flink jar passed. Using the location of class > org.apache.flink.yarn.YarnClusterDescriptor to locate the jar2020-11-13 > 16:14:31,065 INFO > org.apache.hadoop.yarn.client.ConfiguredRMFailoverProxyProvider [] - Failing > over to rm22020-11-13 16:14:31,130 INFO > org.apache.flink.yarn.YarnClusterDescriptor [] - The > configured JobManager memory is 3072 MB. YARN will allocate 4096 MB to make > up an integer multiple of its minimum allocation memory (2048 MB, configured > via 'yarn.scheduler.minimum-allocation-mb'). The extra 1024 MB may not be > used by Flink.2020-11-13 16:14:31,130 INFO > org.apache.flink.yarn.YarnClusterDescriptor [] - The > configured TaskManager memory is 3072 MB. YARN will allocate 4096 MB to make > up an integer multiple of its minimum allocation memory (2048 MB, configured > via 'yarn.scheduler.minimum-allocation-mb'). The extra 1024 MB may not be > used by Flink.2020-11-13 16:14:31,130 INFO > org.apache.flink.yarn.YarnClusterDescriptor [] - Cluster > specification: ClusterSpecification{masterMemoryMB=3072, > taskManagerMemoryMB=3072, slotsPerTaskManager=2}2020-11-13 16:14:31,681 WARN > org.apache.hadoop.hdfs.shortcircuit.DomainSocketFactory [] - The > short-circuit local reads feature cannot be used because libhadoop cannot