Have you tried running without setting the environmentType flag? On Fri, May 24, 2019 at 8:24 AM 青雉（祁明良） <m...@xiaohongshu.com> wrote:
> Hi all, > > This is Mingliang, I followed the document to setup beam pipeline on local > Flink cluster, but when I switch to remote Flink cluster, it doesn’t work > directly and I can’t find many materials talking about this. > > Firstly, I don’t know if I have to install anything on the Flink > jobmanager / taskmanager machine, actually I just setup the Flink cluster > itself, let’s say Jobmanager on machine A and two taskmanager on machine > B,C. > > Next I installed apache_beam python package on machine D, and started the > beam jobservice end point on machine D with: ./gradlew > beam-runners-flink_2.11-job-server:runShadow -PflinkMasterUrl=A:8081 > > Then I submit the word_count example downloaded from beam to the > jobservice end point on machine D with environment_type = LOOPBACK > > The error message I received was attached below. > > Any helps will be appreciated, > Thanks, > Mingliang > > - Versions > Beam: release-2.12.0 > Flink: 1.5.6 > > - Log > ---------------- > Caused by: java.lang.Exception: The user defined 'open()' method caused an > exception: > org.apache.beam.vendor.grpc.v1p13p1.io.grpc.StatusRuntimeException: > UNAVAILABLE: io exception > at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:498) > at org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:368) > at org.apache.flink.runtime.taskmanager.Task.run(Task.java:703) > ... 1 more > Caused by: > org.apache.beam.vendor.guava.v20_0.com.google.common.util.concurrent.UncheckedExecutionException: > org.apache.beam.vendor.grpc.v1p13p1.io.grpc.StatusRuntimeException: > UNAVAILABLE: io exception > at > org.apache.beam.vendor.guava.v20_0.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2214) > at > org.apache.beam.vendor.guava.v20_0.com.google.common.cache.LocalCache.get(LocalCache.java:4053) > at > org.apache.beam.vendor.guava.v20_0.com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:4057) > at > org.apache.beam.vendor.guava.v20_0.com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4986) > at > org.apache.beam.vendor.guava.v20_0.com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:4992) > at > org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory.forStage(DefaultJobBundleFactory.java:186) > at > org.apache.beam.runners.flink.translation.functions.FlinkDefaultExecutableStageContext.getStageBundleFactory(FlinkDefaultExecutableStageContext.java:49) > at > org.apache.beam.runners.flink.translation.functions.ReferenceCountingFlinkExecutableStageContextFactory$WrappedContext.getStageBundleFactory(ReferenceCountingFlinkExecutableStageContextFactory.java:203) > at > org.apache.beam.runners.flink.translation.functions.FlinkExecutableStageFunction.open(FlinkExecutableStageFunction.java:143) > at > org.apache.flink.api.common.functions.util.FunctionUtils.openFunction(FunctionUtils.java:36) > at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:494) > ... 3 more > Caused by: > org.apache.beam.vendor.grpc.v1p13p1.io.grpc.StatusRuntimeException: > UNAVAILABLE: io exception > at > org.apache.beam.vendor.grpc.v1p13p1.io.grpc.stub.ClientCalls.toStatusRuntimeException(ClientCalls.java:222) > at > org.apache.beam.vendor.grpc.v1p13p1.io.grpc.stub.ClientCalls.getUnchecked(ClientCalls.java:203) > at > org.apache.beam.vendor.grpc.v1p13p1.io.grpc.stub.ClientCalls.blockingUnaryCall(ClientCalls.java:132) > at > org.apache.beam.model.fnexecution.v1.BeamFnExternalWorkerPoolGrpc$BeamFnExternalWorkerPoolBlockingStub.notifyRunnerAvailable(BeamFnExternalWorkerPoolGrpc.java:152) > at > org.apache.beam.runners.fnexecution.environment.ExternalEnvironmentFactory.createEnvironment(ExternalEnvironmentFactory.java:109) > at > org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$1.load(DefaultJobBundleFactory.java:178) > at > org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$1.load(DefaultJobBundleFactory.java:161) > at > org.apache.beam.vendor.guava.v20_0.com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3628) > at > org.apache.beam.vendor.guava.v20_0.com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2336) > at > org.apache.beam.vendor.guava.v20_0.com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2295) > at > org.apache.beam.vendor.guava.v20_0.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2208) > ... 13 more > Caused by: > org.apache.beam.vendor.grpc.v1p13p1.io.netty.channel.AbstractChannel$AnnotatedConnectException: > Connection refused: localhost/127.0.0.1:45955 > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) > at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717) > at > org.apache.beam.vendor.grpc.v1p13p1.io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:325) > at > org.apache.beam.vendor.grpc.v1p13p1.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:340) > at > org.apache.beam.vendor.grpc.v1p13p1.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:635) > at > org.apache.beam.vendor.grpc.v1p13p1.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:582) > at > org.apache.beam.vendor.grpc.v1p13p1.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:499) > at > org.apache.beam.vendor.grpc.v1p13p1.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:461) > at > org.apache.beam.vendor.grpc.v1p13p1.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:884) > at > org.apache.beam.vendor.grpc.v1p13p1.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) > ... 1 more > Caused by: java.net.ConnectException: Connection refused > ... 11 more > > 本邮件及其附件含有小红书公司的保密信息，仅限于发送给以上收件人或群组。禁止任何其他人以任何形式使用（包括但不限于全部或部分地泄露、复制、或散发）本邮件中的信息。如果您错收了本邮件，请您立即电话或邮件通知发件人并删除本邮件！ > > This communication may contain privileged or other confidential > information of Red. If you have received it in error, please advise the > sender by reply e-mail and immediately delete the message and any > attachments without copying or disclosing the contents. Thank you. > > -- Kyle Weaver | Software Engineer | github.com/ibzib | kcwea...@google.com | +16502035555
