Flink application using kafka topics as source and destination. Using
javaVersion = '1.11'
flinkVersion = '1.11.1'
scalaBinaryVersion ='2.11'
the application is primarily using Flink SQL apis. We have a StatementSet and
add sql inserts to that set using addInsertSql.
when there are more insert statements (say 10+) running job outside of flink
cluster using ./gradlew shadowJar run fails with following error
[GroupWindowAggregate(groupBy………) ERROR
org.apache.flink.streaming.runtime.tasks.StreamTask - Error during disposal of
stream operator.
java.lang.NullPointerException: null
at
org.apache.flink.table.runtime.operators.window.WindowOperator.dispose(WindowOperator.java:318)
~[flink-table-runtime-blink_2.11-1.11.1.jar:1.11.1]
at
org.apache.flink.streaming.runtime.tasks.StreamTask.disposeAllOperators(StreamTask.java:703)
[flink-streaming-java_2.11-1.11.1.jar:1.11.1]
at
org.apache.flink.streaming.runtime.tasks.StreamTask.cleanUpInvoke(StreamTask.java:635)
[flink-streaming-java_2.11-1.11.1.jar:1.11.1]
at
org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:542)
[flink-streaming-java_2.11-1.11.1.jar:1.11.1]
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:721)
[flink-runtime_2.11-1.11.1.jar:1.11.1]
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:546)
[flink-runtime_2.11-1.11.1.jar:1.11.1]
at java.lang.Thread.run(Thread.java:834) [?:?]
some approaches that did not work:
Having multiple statement set. This works locally, but not with flink cluster
with error saying cannot have multiple execute statements.
Changing partition count of kafka topics. No impact.
This e-mail is intended only for the use of the addressees. Any copying,
forwarding, printing or other use of this e-mail by persons other than the
addressees is not authorized. This e-mail may contain information that is
privileged, confidential and exempt from disclosure. If you are not the
intended recipient, please notify us immediately by return e-mail (including
the original message in your reply) and then delete and discard all copies of
the e-mail. Thank you.
HB75
