rmetzger commented on pull request #14499:
URL: https://github.com/apache/flink/pull/14499#issuecomment-761008849
During my manual test, it seems that I can intercept a System.exit() in user
code of a batch job:
```
2021-01-15 16:16:01,950 WARN
org.apache.flink.runtime.security.FlinkSecurityManager [] - Exiting JVM
with status 0 is monitored: The system will exit due to this call.
org.apache.flink.runtime.UserSystemExitException: Flink user code attempted
to exit JVM.
at
org.apache.flink.runtime.security.FlinkSecurityManager.checkExit(FlinkSecurityManager.java:180)
[flink-dist_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
at java.lang.Runtime.exit(Runtime.java:107) [?:1.8.0_252]
at java.lang.System.exit(System.java:973) [?:1.8.0_252]
at de.robertmetzger.BatchJob$1.map(BatchJob.java:51)
[blob_p-f236836ff40f8dac63909ec8c69fb02a0823541c-6253f79b4f1453ea0b6abe7b75c2857e:?]
at de.robertmetzger.BatchJob$1.map(BatchJob.java:46)
[blob_p-f236836ff40f8dac63909ec8c69fb02a0823541c-6253f79b4f1453ea0b6abe7b75c2857e:?]
at
org.apache.flink.runtime.operators.chaining.ChainedMapDriver.collect(ChainedMapDriver.java:78)
[flink-dist_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
at
org.apache.flink.runtime.operators.util.metrics.CountingCollector.collect(CountingCollector.java:35)
[flink-dist_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
at
org.apache.flink.runtime.operators.DataSourceTask.invoke(DataSourceTask.java:204)
[flink-dist_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:762)
[flink-dist_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:571)
[flink-dist_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_252]
```
For streaming jobs, intercepting regular operators (such as map), works fine
as well.
However for sources, it won't work.
The StackTrace to a source is the following:
```
2021-01-15 16:21:45,859 INFO de.robertmetzger.StreamingJob
[] - I'M here on the Taskmanager
java.lang.RuntimeException: null
at de.robertmetzger.StreamingJob$2.map(StreamingJob.java:70)
[blob_p-eb6613e604b074ad283e611328dcbf036a1f86cb-4d496ef1ec5656acaa10612f41dc4839:?]
at de.robertmetzger.StreamingJob$2.map(StreamingJob.java:67)
[blob_p-eb6613e604b074ad283e611328dcbf036a1f86cb-4d496ef1ec5656acaa10612f41dc4839:?]
at
org.apache.flink.streaming.api.operators.StreamMap.processElement(StreamMap.java:38)
[flink-dist_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
at
org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.pushToOperator(CopyingChainingOutput.java:71)
[flink-dist_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
at
org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.collect(CopyingChainingOutput.java:46)
[flink-dist_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
at
org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.collect(CopyingChainingOutput.java:26)
[flink-dist_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
at
org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:50)
[flink-dist_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
at
org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:28)
[flink-dist_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
at
org.apache.flink.streaming.api.operators.StreamSourceContexts$ManualWatermarkContext.processAndCollect(StreamSourceContexts.java:317)
[flink-dist_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
at
org.apache.flink.streaming.api.operators.StreamSourceContexts$WatermarkContext.collect(StreamSourceContexts.java:411)
[flink-dist_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
at de.robertmetzger.StreamingJob$1.run(StreamingJob.java:57)
[blob_p-eb6613e604b074ad283e611328dcbf036a1f86cb-4d496ef1ec5656acaa10612f41dc4839:?]
at
org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:110)
[flink-dist_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
at
org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:66)
[flink-dist_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
at
org.apache.flink.streaming.runtime.tasks.SourceStreamTask$LegacySourceFunctionThread.run(SourceStreamTask.java:245)
[flink-dist_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT]
```
I believe sources are relatively important, in particular because some users
have their own implementations there.
@hwanju do you want to address this in this PR, or in a follow up?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
