When you submit a program via the REST API, the main method executes inside the JobManager process. Unfortunately a static variable is used to establish the execution environment that the program obtains from `ExecutionEnvironment.getExecutionEnvironment()`. From the stack trace it appears that two main methods are executing simultaneously and one is corrupting the other.
On Mon, Aug 7, 2017 at 8:21 AM, Francisco Gonzalez Barea < francisco.gonza...@piksel.com> wrote: > Hi there! > > We are doing some POCs submitting jobs remotely to Flink. We tried with > Flink CLI and now we´re testing the Rest API. > > So the point is that when we try to execute a set of requests in an async > way (using CompletableFutures) only a couple of them run successfully. For > the rest we get the exception copied at the end of the email. > > Do you know the reason for this? > > Thanks in advance!! > Regards, > > org.apache.flink.client.program.ProgramInvocationException: The main > method caused an error. > at org.apache.flink.client.program.PackagedProgram.callMainMethod( > PackagedProgram.java:545) > at org.apache.flink.client.program.PackagedProgram. > invokeInteractiveModeForExecution(PackagedProgram.java:419) > at org.apache.flink.client.program.OptimizerPlanEnvironment. > getOptimizedPlan(OptimizerPlanEnvironment.java:80) > at org.apache.flink.client.program.ClusterClient.getOptimizedPlan( > ClusterClient.java:318) > at org.apache.flink.runtime.webmonitor.handlers.JarActionHandler. > getJobGraphAndClassLoader(JarActionHandler.java:72) > at org.apache.flink.runtime.webmonitor.handlers.JarRunHandler. > handleJsonRequest(JarRunHandler.java:61) > at org.apache.flink.runtime.webmonitor.handlers. > AbstractJsonRequestHandler.handleRequest(AbstractJsonRequestHandler. > java:41) > at org.apache.flink.runtime.webmonitor.RuntimeMonitorHandler. > respondAsLeader(RuntimeMonitorHandler.java:109) > at org.apache.flink.runtime.webmonitor.RuntimeMonitorHandlerBase. > channelRead0(RuntimeMonitorHandlerBase.java:97) > at org.apache.flink.runtime.webmonitor.RuntimeMonitorHandlerBase. > channelRead0(RuntimeMonitorHandlerBase.java:44) > at io.netty.channel.SimpleChannelInboundHandler.channelRead( > SimpleChannelInboundHandler.java:105) > at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead( > AbstractChannelHandlerContext.java:339) > at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead( > AbstractChannelHandlerContext.java:324) > at io.netty.handler.codec.http.router.Handler.routed(Handler.java:62) > at io.netty.handler.codec.http.router.DualAbstractHandler.channelRead0( > DualAbstractHandler.java:57) > at io.netty.handler.codec.http.router.DualAbstractHandler.channelRead0( > DualAbstractHandler.java:20) > at io.netty.channel.SimpleChannelInboundHandler.channelRead( > SimpleChannelInboundHandler.java:105) > at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead( > AbstractChannelHandlerContext.java:339) > at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead( > AbstractChannelHandlerContext.java:324) > at org.apache.flink.runtime.webmonitor.HttpRequestHandler.channelRead0( > HttpRequestHandler.java:159) > at org.apache.flink.runtime.webmonitor.HttpRequestHandler.channelRead0( > HttpRequestHandler.java:65) > at io.netty.channel.SimpleChannelInboundHandler.channelRead( > SimpleChannelInboundHandler.java:105) > at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead( > AbstractChannelHandlerContext.java:339) > at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead( > AbstractChannelHandlerContext.java:324) > at io.netty.channel.ChannelInboundHandlerAdapter.channelRead( > ChannelInboundHandlerAdapter.java:86) > at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead( > AbstractChannelHandlerContext.java:339) > at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead( > AbstractChannelHandlerContext.java:324) > at io.netty.handler.codec.ByteToMessageDecoder.channelRead( > ByteToMessageDecoder.java:242) > at io.netty.channel.CombinedChannelDuplexHandler.channelRead( > CombinedChannelDuplexHandler.java:147) > at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead( > AbstractChannelHandlerContext.java:339) > at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead( > AbstractChannelHandlerContext.java:324) > at io.netty.channel.DefaultChannelPipeline.fireChannelRead( > DefaultChannelPipeline.java:847) > at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read( > AbstractNioByteChannel.java:131) > at io.netty.channel.nio.NioEventLoop.processSelectedKey( > NioEventLoop.java:511) > at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized( > NioEventLoop.java:468) > at io.netty.channel.nio.NioEventLoop.processSelectedKeys( > NioEventLoop.java:382) > at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354) > at io.netty.util.concurrent.SingleThreadEventExecutor$2. > run(SingleThreadEventExecutor.java:111) > at io.netty.util.concurrent.DefaultThreadFactory$ > DefaultRunnableDecorator.run(DefaultThreadFactory.java:137) > at java.lang.Thread.run(Thread.java:748) > \nCaused by: java.util.ConcurrentModificationException > at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:901) > at java.util.ArrayList$Itr.next(ArrayList.java:851) > at org.apache.flink.api.java.operators.OperatorTranslation. > translateToPlan(OperatorTranslation.java:49) > at org.apache.flink.api.java.ExecutionEnvironment.createProgramPlan( > ExecutionEnvironment.java:1065) > at org.apache.flink.api.java.ExecutionEnvironment.createProgramPlan( > ExecutionEnvironment.java:1032) > at org.apache.flink.client.program.OptimizerPlanEnvironment.execute( > OptimizerPlanEnvironment.java:47) > at com.piksel.sequoia.media.common.launcher.Launcher.main( > Launcher.java:56) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at sun.reflect.NativeMethodAccessorImpl.invoke( > NativeMethodAccessorImpl.java:62) > at sun.reflect.DelegatingMethodAccessorImpl.invoke( > DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at org.apache.flink.client.program.PackagedProgram.callMainMethod( > PackagedProgram.java:528) > ... 39 more\n”} > > > This message is private and confidential. If you have received this > message in error, please notify the sender or serviced...@piksel.com and > remove it from your system. > > Piksel Inc is a company registered in the United States, 2100 Powers Ferry > Road SE, Suite 400, Atlanta, GA 30339 >
