使用的是perjob模式提交作业,没有使用yarn-seesion。为什么perjob模式提交有这个-yd参数会有问题,还是没太懂。
在 2020-04-15 08:52:11,"tison" <wander4...@gmail.com> 写道:
>-yd 参数影响的是你是否使用 perjob 模式提交作业,简单地说
>
>with -yd 以 perjob 模式提交作业,即启动一个新集群
>without -yd 提交到一个现有的 Flink on YARN 集群
>
>哪个是你的需求呢?有没有实现用 yarn-session 启动 Flink on YARN 集群呢?
>
>Best,
>tison.
>
>
>guanyq <dlgua...@163.com> 于2020年4月15日周三 上午8:46写道:
>
>> 提交失败,本人测试与-yd参数有关系,<br/>这个参数去掉就可以提交了。<br/>但是不知道 -yd这个参数影响了什么?
>> At 2020-04-14 15:31:00, "guanyq" <dlgua...@163.com> wrote:
>> >提交失败,yarn资源也还有很多,为什么会提交失败呢?
>> >
>> >提交脚本
>> >./bin/flink run -m yarn-cluster \
>> >-ynm TestDataProcess \
>> >-yd \
>> >-yn 2 \
>> >-ytm 1024 \
>> >-yjm 1024 \
>> >-c com.data.processing.unconditionalacceptance.TestDataProcess \
>> >./tasks/UnconditionalAcceptanceDataProcess.jar \
>> >
>> >
>> >yarn资源
>> >Apps Submitted Apps Pending    Apps Running    Apps Completed  Containers
>> Running      Memory Used     Memory Total    Memory Reserved VCores Used
>>  VCores Total    VCores Reserved Active Nodes    Decommissioned Nodes
>> Lost Nodes      Unhealthy Nodes Rebooted Nodes
>> >239    0       12      227     173     334 GB  1.42 TB 0 B     173
>>  288     0       9       0       0       0       0
>> >
>> >
>> >
>> >2020-04-14 15:14:19,002 INFO
>> org.apache.flink.yarn.AbstractYarnClusterDescriptor           - Deployment
>> took more than 60 seconds. Please check if the requested resources are
>> available in the YARN cluster
>> >2020-04-14 15:14:19,253 INFO
>> org.apache.flink.yarn.AbstractYarnClusterDescriptor           - Deployment
>> took more than 60 seconds. Please check if the requested resources are
>> available in the YARN cluster
>> >2020-04-14 15:14:19,504 INFO
>> org.apache.flink.yarn.AbstractYarnClusterDescriptor           - Deployment
>> took more than 60 seconds. Please check if the requested resources are
>> available in the YARN cluster
>> >2020-04-14 15:14:19,755 INFO
>> org.apache.flink.yarn.AbstractYarnClusterDescriptor           - Deployment
>> took more than 60 seconds. Please check if the requested resources are
>> available in the YARN cluster
>> >2020-04-14 15:14:20,006 INFO
>> org.apache.flink.yarn.AbstractYarnClusterDescriptor           - Deployment
>> took more than 60 seconds. Please check if the requested resources are
>> available in the YARN cluster
>> >2020-04-14 15:14:20,257 INFO
>> org.apache.flink.yarn.AbstractYarnClusterDescriptor           - Deployment
>> took more than 60 seconds. Please check if the requested resources are
>> available in the YARN cluster
>> >2020-04-14 15:14:20,508 INFO
>> org.apache.flink.yarn.AbstractYarnClusterDescriptor           - Deployment
>> took more than 60 seconds. Please check if the requested resources are
>> available in the YARN cluster
>> >2020-04-14 15:14:20,759 INFO
>> org.apache.flink.yarn.AbstractYarnClusterDescriptor           - Deployment
>> took more than 60 seconds. Please check if the requested resources are
>> available in the YARN cluster
>> >2020-04-14 15:14:21,011 INFO
>> org.apache.flink.yarn.AbstractYarnClusterDescriptor           - Deployment
>> took more than 60 seconds. Please check if the requested resources are
>> available in the YARN cluster
>> >2020-04-14 15:14:21,262 INFO
>> org.apache.flink.yarn.AbstractYarnClusterDescriptor           - Deployment
>> took more than 60 seconds. Please check if the requested resources are
>> available in the YARN cluster
>> >2020-04-14 15:14:21,513 INFO
>> org.apache.flink.yarn.AbstractYarnClusterDescriptor           - Deployment
>> took more than 60 seconds. Please check if the requested resources are
>> available in the YARN cluster
>> >2020-04-14 15:14:21,764 INFO
>> org.apache.flink.yarn.AbstractYarnClusterDescriptor           - Deployment
>> took more than 60 seconds. Please check if the requested resources are
>> available in the YARN cluster
>> >2020-04-14 15:14:22,015 INFO
>> org.apache.flink.yarn.AbstractYarnClusterDescriptor           - Deployment
>> took more than 60 seconds. Please check if the requested resources are
>> available in the YARN cluster
>> >2020-04-14 15:14:22,265 INFO
>> org.apache.flink.yarn.AbstractYarnClusterDescriptor           - Deployment
>> took more than 60 seconds. Please check if the requested resources are
>> available in the YARN cluster
>> >2020-04-14 15:14:22,517 INFO
>> org.apache.flink.yarn.AbstractYarnClusterDescriptor           - Deployment
>> took more than 60 seconds. Please check if the requested resources are
>> available in the YARN cluster
>> >2020-04-14 15:14:22,768 INFO
>> org.apache.flink.yarn.AbstractYarnClusterDescriptor           - Deployment
>> took more than 60 seconds. Please check if the requested resources are
>> available in the YARN cluster
>> >2020-04-14 15:14:23,019 INFO
>> org.apache.flink.yarn.AbstractYarnClusterDescriptor           - Deployment
>> took more than 60 seconds. Please check if the requested resources are
>> available in the YARN cluster
>> >2020-04-14 15:14:23,270 INFO
>> org.apache.flink.yarn.AbstractYarnClusterDescriptor           - Deployment
>> took more than 60 seconds. Please check if the requested resources are
>> available in the YARN cluster
>> >2020-04-14 15:14:23,521 INFO
>> org.apache.flink.yarn.AbstractYarnClusterDescriptor           - Deployment
>> took more than 60 seconds. Please check if the requested resources are
>> available in the YARN cluster
>> >2020-04-14 15:14:23,772 INFO
>> org.apache.flink.yarn.AbstractYarnClusterDescriptor           - Deployment
>> took more than 60 seconds. Please check if the requested resources are
>> available in the YARN cluster
>> >2020-04-14 15:14:24,025 INFO
>> org.apache.flink.yarn.AbstractYarnClusterDescriptor           - Deployment
>> took more than 60 seconds. Please check if the requested resources are
>> available in the YARN cluster
>> >2020-04-14 15:14:24,276 INFO
>> org.apache.flink.yarn.AbstractYarnClusterDescriptor           - Deployment
>> took more than 60 seconds. Please check if the requested resources are
>> available in the YARN cluster
>>

回复