[
https://issues.apache.org/jira/browse/FLINK-18681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17165397#comment-17165397
]
Xintong Song commented on FLINK-18681:
--------------------------------------
Hi [~apach...@163.com],
It seems I've misunderstood what you meant. I thought you were complaining
about the dependency conflict, but actually you were complaining about the
resource increasing caused by the dependency conflict. I apologize for that
misunderstanding.
I think you are right. This is indeed a problem that one problematic Flink
application impacts the entire cluster by taking lots of resources. Could you
describe a bit more information to help us understand the problem better? I'm
particularly interested in how exactly does the resources increase.
- Does the application keeps requesting new containers but never release them?
- What are the status of requested containers? Are the actually allocated or
reserved?
- Are there AM failovers? (In other words, do you see many application attempts
on Yarn?)
It would also be helpful if you can share the complete JM logs, if there's no
sensitive information.
> The jar package version conflict causes the task to continue to increase and
> grab resources
> -------------------------------------------------------------------------------------------
>
> Key: FLINK-18681
> URL: https://issues.apache.org/jira/browse/FLINK-18681
> Project: Flink
> Issue Type: Bug
> Affects Versions: 1.11.0
> Reporter: wangtaiyang
> Priority: Major
>
> When I submit a flink task to yarn, the default resource configuration is
> 1G&1core, but in fact this task will always increase resources 2core, 3core,
> and so on. . . 200core. . . Then I went to look at the JM log and found the
> following error:
> {code:java}
> //代码占位符
> java.lang.NoSuchMethodError:
> org.apache.commons.cli.Option.builder(Ljava/lang/String;)Lorg/apache/commons/cli/Option$Builder;java.lang.NoSuchMethodError:
>
> org.apache.commons.cli.Option.builder(Ljava/lang/String;)Lorg/apache/commons/cli/Option$Builder;
> at
> org.apache.flink.runtime.entrypoint.parser.CommandLineOptions.<clinit>(CommandLineOptions.java:28)
> ~[flink-dist_2.11-1.11.1.jar:1.11.1] at
> org.apache.flink.runtime.clusterframework.BootstrapTools.lambda$getDynamicPropertiesAsString$0(BootstrapTools.java:648)
> ~[flink-dist_2.11-1.11.1.jar:1.11.1] at
> java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:267)
> ~[?:1.8.0_191]
> .......
> java.lang.NoClassDefFoundError: Could not initialize class
> org.apache.flink.runtime.entrypoint.parser.CommandLineOptionsjava.lang.NoClassDefFoundError:
> Could not initialize class
> org.apache.flink.runtime.entrypoint.parser.CommandLineOptions at
> org.apache.flink.runtime.clusterframework.BootstrapTools.lambda$getDynamicPropertiesAsString$0(BootstrapTools.java:648)
> ~[flink-dist_2.11-1.11.1.jar:1.11.1] at
> java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:267)
> ~[?:1.8.0_191] at
> java.util.HashMap$KeySpliterator.forEachRemaining(HashMap.java:1553)
> ~[?:1.8.0_191] at
> java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
> ~[?:1.8.0_191]{code}
> Finally, it is confirmed that it is caused by the commands-cli version
> conflict, but the task reporting error has not stopped and will continue to
> grab resources and increase. Is this a bug?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
