sys.exist(1) led to standalonesession daemon closed

[Tony Wei]

Hi,

I used a scala library called scallop[1] to parse my job’s arguments. When
the argument didn’t
exist in the config setting, the default behavior of scallop would call
sys.exit(1).

It is not a problem when I’m using flink cli to submit job. However, when I
used rest api to submit
job, it seems that sys.exit(1) will leads to standalonesession daemon
closed. Maybe the reason is
that rest server is also in the same process as standalonesession daemon.
Am I correct?

If this is the root cause, is this an expected behavior and users should be
aware of not using
sys.exit(1) in their jobs?

I tested this on 1.6.0 standalone session cluster with flip-6 mode. And
here are my testing job
and logs before and after the submission.

package com.appier.rt.rt_match
> import org.apache.flink.api.scala.createTypeInformation
> import org.apache.flink.streaming.api.scala.StreamExecutionEnvironment
> import org.rogach.scallop.{ScallopConf, ScallopOption}
> object TestMain {
>   def main(args: Array[String]): Unit = {
>     object Args extends ScallopConf(args) {
>       val mode: ScallopOption[String] = opt[String](default =
> Some("development"))
>       verify
>     }
>     val env = StreamExecutionEnvironment.getExecutionEnvironment
>     env.fromElements(Args.mode()).map(a => a)
>     env.execute()
>   }
> }


Submit by flink-cli

> $ ./bin/flink run -c com.appier.rt.rt_match.TestMain -p 2 -d
> rt-match-assembly-4.5.1-SNAPSHOT.jar --mo xyz
> Starting execution of program
> [scallop] Error: Unknown option 'mo'


Submit by rest-api

> 2018-11-05 13:27:58,800 TRACE
> org.apache.flink.runtime.webmonitor.handlers.JarListHandler   - Received
> request /jars/.
> 2018-11-05 13:27:59,679 TRACE
> org.apache.flink.runtime.rest.FileUploadHandler               - Received
> request. URL:/jobs/overview Method:GET
> 2018-11-05 13:27:59,680 TRACE
> org.apache.flink.runtime.rest.handler.job.JobsOverviewHandler  - Received
> request /jobs/overview.
> 2018-11-05 13:28:01,752 TRACE
> org.apache.flink.runtime.rest.FileUploadHandler               - Received
> request. URL:/jars/ Method:GET
> 2018-11-05 13:28:01,753 TRACE
> org.apache.flink.runtime.webmonitor.handlers.JarListHandler   - Received
> request /jars/.
> 2018-11-05 13:28:02,682 TRACE
> org.apache.flink.runtime.rest.FileUploadHandler               - Received
> request. URL:/jobs/overview Method:GET
> 2018-11-05 13:28:02,683 TRACE
> org.apache.flink.runtime.rest.handler.job.JobsOverviewHandler  - Received
> request /jobs/overview.
> 2018-11-05 13:28:03,899 TRACE
> org.apache.flink.runtime.rest.FileUploadHandler               - Received
> request.
> URL:/jars/7413f82a-d650-4729-873e-a94150ffe9d0_rt-match-assembly-4.5.1-SNAPSHOT.jar/run?entry
> class=com.appier.rt.rt_match.TestMain&parallelism=2&program-args=--mo+xyz
> Method:POST
> 2018-11-05 13:28:03,902 TRACE
> org.apache.flink.runtime.webmonitor.handlers.JarRunHandler    - Received
> request
> /jars/7413f82a-d650-4729-873e-a94150ffe9d0_rt-match-assembly-4.5.1-SNAPSHOT.jar/run?entry
> class=com.appier.rt.rt_match.TestMain&parallelism=2&program-args=--mo+xyz.
> 2018-11-05 13:28:04,751 TRACE
> org.apache.flink.runtime.rest.FileUploadHandler               - Received
> request. URL:/jars/ Method:GET
> 2018-11-05 13:28:04,752 TRACE
> org.apache.flink.runtime.webmonitor.handlers.JarListHandler   - Received
> request /jars/.
> 2018-11-05 13:28:04,760 INFO
> org.apache.flink.runtime.blob.TransientBlobCache              - Shutting
> down BLOB cache
> 2018-11-05 13:28:04,761 INFO  org.apache.flink.runtime.blob.BlobServer
>                   - Stopped BLOB server at 0.0.0.0:42075


Best,
Tony Wei.

[1] https://github.com/scallop/scallop