I have managed to locate the issue with timeout, changing `web.timeout` was
the solution. However, now I am getting the following error :
019-11-12 16:58:00,741 INFO
org.apache.parquet.hadoop.ParquetInputFormat - Total
input paths to process : 671
2019-11-12 16:58:04,878 INFO
org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total
input files to process : 519
2019-11-12 16:58:04,878 INFO
org.apache.parquet.hadoop.ParquetInputFormat - Total
input paths to process : 519
2019-11-12 16:58:08,017 INFO
org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total
input files to process : 382
2019-11-12 16:58:08,017 INFO
org.apache.parquet.hadoop.ParquetInputFormat - Total
input paths to process : 382
2019-11-12 16:58:12,277 INFO
org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total
input files to process : 551
2019-11-12 16:58:12,277 INFO
org.apache.parquet.hadoop.ParquetInputFormat - Total
input paths to process : 551
2019-11-12 16:58:16,530 INFO
org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total
input files to process : 507
2019-11-12 16:58:16,530 INFO
org.apache.parquet.hadoop.ParquetInputFormat - Total
input paths to process : 507
2019-11-12 16:58:20,080 INFO
org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total
input files to process : 478
2019-11-12 16:58:20,080 INFO
org.apache.parquet.hadoop.ParquetInputFormat - Total
input paths to process : 478
2019-11-12 16:58:23,736 INFO
org.apache.flink.runtime.dispatcher.StandaloneDispatcher -
Received JobGraph submission f0cd12c8e5e9d7e95cfc2f685c451089 (Flink
Java Job at Tue Nov 12 16:40:19 UTC 2019).
2019-11-12 16:58:23,738 ERROR
org.apache.flink.runtime.rest.handler.job.JobSubmitHandler -
Unhandled exception.
org.apache.flink.runtime.client.JobSubmissionException: Job has
already been submitted.
at
org.apache.flink.runtime.dispatcher.Dispatcher.submitJob(Dispatcher.java:268)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcInvocation(AkkaRpcActor.java:274)
at
org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:189)
at
org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.handleRpcMessage(FencedAkkaRpcActor.java:74)
at
org.apache.flink.runtime.rpc.akka.AkkaRpcActor.onReceive(AkkaRpcActor.java:147)
at
org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.onReceive(FencedAkkaRpcActor.java:40)
at
akka.actor.UntypedActor$$anonfun$receive$1.applyOrElse(UntypedActor.scala:165)
at akka.actor.Actor$class.aroundReceive(Actor.scala:502)
at akka.actor.UntypedActor.aroundReceive(UntypedActor.scala:95)
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526)
at akka.actor.ActorCell.invoke(ActorCell.scala:495)
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)
at akka.dispatch.Mailbox.run(Mailbox.scala:224)
at akka.dispatch.Mailbox.exec(Mailbox.scala:234)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at
scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at
scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at
scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
Which also seems weird, since I only submit job once.
