Could you try adding "--experiments=beam_fn_api" to the runner options? Kyle did mention that it should be fixed in 2.16.0.
On 2019/09/17 16:40:41, Matthew Patterson <[email protected]> wrote: > Folks, > > Am sure I am doing it wrong, but have been beating head against wall long > enough I need to ask for help. > > My goal is to use the python SDK to run jobs on AWS flink cluster, but very > little is working, so I am trying to back down to the very simplest thing > that works, and build back up: not going so well. > > Now I am running, or trying to, on a minikube link cluster. Now running from > just java/mvn, so I go through the following: > > 1. bring-up-cluster (see below) > 2. grab-the-code (see below) > 3. maven-submit (see below) > > Eventual failure is > > “[ERROR] Failed to execute goal > org.codehaus.mojo:exec-maven-plugin:1.6.0:java (default-cli) on project > word-count-beam: An exception occured while executing the Java class. > Pipeline execution failed: Could not retrieve the execution result. (JobID: > 6edad04faa43fbf22bdea24171ba9d59): Failed to submit JobGraph. [Failed to > deserialize JobGraph.]” > > and log is attached (I hope) > > Other details: > > on osx > > > > ➜ minikube version > > minikube version: v1.3.1 > > commit: ca60a424ce69a4d79f502650199ca2b52f29e631 > > > ➜ kubectl version > > Client Version: version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.6", > GitCommit:"96fac5cd13a5dc064f7d9f4f23030a6aeface6cc", GitTreeState:"clean", > BuildDate:"2019-08-19T11:13:49Z", GoVersion:"go1.12.9", Compiler:"gc", > Platform:"darwin/amd64"} > > Server Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.2", > GitCommit:"f6278300bebbb750328ac16ee6dd3aa7d3549568", GitTreeState:"clean", > BuildDate:"2019-08-05T09:15:22Z", GoVersion:"go1.12.5", Compiler:"gc", > Platform:"linux/amd64"} > > > ➜ flink --version > > Version: 1.8.1, Commit ID: 7297bac > > ➜ python -c"import apache_beam; print(apache_beam.__version__)" > 2.15.0 > > ➜ curl localhost:8081 > <!-- > ~ Licensed to the Apache Software Foundation (ASF) under one > ~ or more contributor license agreements. See the NOTICE file > ~ distributed with this work for additional information > ~ regarding copyright ownership. The ASF licenses this file > ~ to you under the Apache License, Version 2.0 (the > ~ "License"); you may not use this file except in compliance > ~ with the License. You may obtain a copy of the License at > ~ > ~ http://www.apache.org/licenses/LICENSE-2.0 > ~ > ~ Unless required by applicable law or agreed to in writing, software > ~ distributed under the License is distributed on an "AS IS" BASIS, > ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. > ~ See the License for the specific language governing permissions and > ~ limitations under the License. > --> > > <!doctype html> > <html lang="en"> > <head> > <meta charset="utf-8"> > <meta http-equiv="X-UA-Compatible" content="IE=edge"> > <meta name="viewport" content="width=device-width, initial-scale=1"> > <title>Apache Flink Web Dashboard</title> > <link rel="apple-touch-icon" sizes="180x180" > href="assets/favicon/apple-touch-icon.png"> > <link rel="icon" type="image/png" href="assets/favicon/favicon-32x32.png" > sizes="32x32"> > <link rel="icon" type="image/png" href="assets/favicon/favicon-16x16.png" > sizes="16x16"> > <link rel="manifest" href="assets/favicon/manifest.json"> > <link rel="mask-icon" href="assets/favicon/safari-pinned-tab.svg" > color="#aa1919"> > <link rel="shortcut icon" href="assets/favicon/favicon.ico"> > <meta name="msapplication-config" > content="assets/favicon/browserconfig.xml"> > <meta name="theme-color" content="#ffffff"> > <base href="./"><link rel="stylesheet" > href="styles.30d0912c1ece284d8d9a.css"></head> > <body> > <flink-root></flink-root> > <script type="text/javascript" > src="runtime.ebbfed06bb8e06aaa889.js"></script><script type="text/javascript" > src="es2015-polyfills.5e343224e81eefb7658e.js" nomodule></script><script > type="text/javascript" > src="polyfills.b37850e8279bc3caafc9.js"></script><script > type="text/javascript" src="main.dc6e0997d092df051db0.js"></script></body> > </html> > > Thanks for any ideas, > Matt > > >>> bring-up-cluster > > # [yamls from > here](https://ci.apache.org/projects/flink/flink-docs-stable/ops/deployment/kubernetes.html) > minikube start > minikube ssh 'sudo ip link set docker0 promisc on' > > kubectl create -f flink-configuration-configmap.yaml > kubectl create -f jobmanager-service.yaml > kubectl create -f jobmanager-deployment.yaml > kubectl create -f taskmanager-deployment.yaml > kubectl apply -f jobmanager-rest-service.yaml > > >>> grab-the-code > mvn archetype:generate \ > -DarchetypeGroupId=org.apache.beam \ > -DarchetypeArtifactId=beam-sdks-java-maven-archetypes-examples \ > -DarchetypeVersion=2.15.0 \ > -DgroupId=org.example \ > -DartifactId=word-count-beam \ > -Dversion="0.1" \ > -Dpackage=org.apache.beam.examples \ > -DinteractiveMode=false > > >>> maven-submit > > # kubectl port-forward svc/flink-jobmanager-rest 8081:8081 # in separate > shell, as blocking > mvn package exec:java -X > -Dexec.mainClass=org.apache.beam.examples.DebuggingWordCount \ > -Dexec.args="--runner=FlinkRunner\ > --flinkMaster="localhost:8081"\ > --filesToStage=target/word-count-beam-bundled-0.1.jar\ > --output=<directory-that-exists>" -Pflink-runner > > >
