Hi Matthew,
Beam 2.16.0 is not yet released hence you are getting the error. Can you
try using 2.15.0 version?
Thanks,
Ankur
On Wed, Sep 18, 2019 at 6:59 AM Matthew Patterson <mpatter...@aligntech.com>
wrote:
> Tried
>
> "
> mvn archetype:generate \
> -DarchetypeGroupId=org.apache.beam \
> -DarchetypeArtifactId=beam-sdks-java-maven-archetypes-examples \
> -DarchetypeVersion=2.16.0 \
> -DgroupId=org.example \
> -DartifactId=word-count-beam \
> -Dversion="0.1" \
> -Dpackage=org.apache.beam.examples \
> -DinteractiveMode=false
> "
> got
>
> -> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-archetype-plugin:3.1.1:generate
> (default-cli) on project standalone-pom: The desired archetype does not
> exist (org.apache.beam:beam-sdks-java-maven-archetypes-examples:2.16.0) ->
> [Help 1]
>
>
> On 9/17/19, 7:51 PM, "Benjamin Tan" <benjamintanwei...@gmail.com> wrote:
>
> CAUTION: This email originated from outside of the organization. Do
> not click links or open attachments unless you recognize the sender and
> know the content is safe.
>
>
> Ah. So maybe try 2.16.0?
>
> On 2019/09/17 23:47:16, Kyle Weaver <kcwea...@google.com> wrote:
> > --experiments=beam_fn_api doesn't apply here, as this is a Java
> pipeline
> > using the non-portable version of the Flink runner.
> >
> > Kyle Weaver | Software Engineer | github.com/ibzib |
> kcwea...@google.com
> >
> >
> > On Tue, Sep 17, 2019 at 4:41 PM Benjamin Tan <
> benjamintanwei...@gmail.com>
> > wrote:
> >
> > > Could you try adding "--experiments=beam_fn_api" to the runner
> options?
> > > Kyle did mention that it should be fixed in 2.16.0.
> > >
> > > On 2019/09/17 16:40:41, Matthew Patterson <
> mpatter...@aligntech.com>
> > > wrote:
> > > > Folks,
> > > >
> > > > Am sure I am doing it wrong, but have been beating head against
> wall
> > > long enough I need to ask for help.
> > > >
> > > > My goal is to use the python SDK to run jobs on AWS flink
> cluster, but
> > > very little is working, so I am trying to back down to the very
> simplest
> > > thing that works, and build back up: not going so well.
> > > >
> > > > Now I am running, or trying to, on a minikube link cluster. Now
> running
> > > from just java/mvn, so I go through the following:
> > > >
> > > > 1. bring-up-cluster (see below)
> > > > 2. grab-the-code (see below)
> > > > 3. maven-submit (see below)
> > > >
> > > > Eventual failure is
> > > >
> > > > “[ERROR] Failed to execute goal
> > > org.codehaus.mojo:exec-maven-plugin:1.6.0:java (default-cli) on
> project
> > > word-count-beam: An exception occured while executing the Java
> class.
> > > Pipeline execution failed: Could not retrieve the execution
> result. (JobID:
> > > 6edad04faa43fbf22bdea24171ba9d59): Failed to submit JobGraph.
> [Failed to
> > > deserialize JobGraph.]”
> > > >
> > > > and log is attached (I hope)
> > > >
> > > > Other details:
> > > >
> > > > on osx
> > > >
> > > >
> > > >
> > > > ➜ minikube version
> > > >
> > > > minikube version: v1.3.1
> > > >
> > > > commit: ca60a424ce69a4d79f502650199ca2b52f29e631
> > > >
> > > >
> > > > ➜ kubectl version
> > > >
> > > > Client Version: version.Info{Major:"1", Minor:"14",
> > > GitVersion:"v1.14.6",
> GitCommit:"96fac5cd13a5dc064f7d9f4f23030a6aeface6cc",
> > > GitTreeState:"clean", BuildDate:"2019-08-19T11:13:49Z",
> > > GoVersion:"go1.12.9", Compiler:"gc", Platform:"darwin/amd64"}
> > > >
> > > > Server Version: version.Info{Major:"1", Minor:"15",
> > > GitVersion:"v1.15.2",
> GitCommit:"f6278300bebbb750328ac16ee6dd3aa7d3549568",
> > > GitTreeState:"clean", BuildDate:"2019-08-05T09:15:22Z",
> > > GoVersion:"go1.12.5", Compiler:"gc", Platform:"linux/amd64"}
> > > >
> > > >
> > > > ➜ flink --version
> > > >
> > > > Version: 1.8.1, Commit ID: 7297bac
> > > >
> > > > ➜ python -c"import apache_beam; print(apache_beam.__version__)"
> > > > 2.15.0
> > > >
> > > > ➜ curl localhost:8081
> > > > <!--
> > > > ~ Licensed to the Apache Software Foundation (ASF) under one
> > > > ~ or more contributor license agreements. See the NOTICE file
> > > > ~ distributed with this work for additional information
> > > > ~ regarding copyright ownership. The ASF licenses this file
> > > > ~ to you under the Apache License, Version 2.0 (the
> > > > ~ "License"); you may not use this file except in compliance
> > > > ~ with the License. You may obtain a copy of the License at
> > > > ~
> > > > ~
> https://nam01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.apache.org%2Flicenses%2FLICENSE-2.0&data=02%7C01%7Cmpatterson%40aligntech.com%7C118675de36e745219ee508d73bc9fb6f%7C9ac44c96980a481bae23d8f56b82c605%7C0%7C0%7C637043611035577083&sdata=4WV%2Bk3Wao0qrbwKJEJTYxG0KixNNGEN%2FW%2FQLz2MrbDk%3D&reserved=0
> > > > ~
> > > > ~ Unless required by applicable law or agreed to in writing,
> software
> > > > ~ distributed under the License is distributed on an "AS IS"
> BASIS,
> > > > ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
> or
> > > implied.
> > > > ~ See the License for the specific language governing
> permissions and
> > > > ~ limitations under the License.
> > > > -->
> > > >
> > > > <!doctype html>
> > > > <html lang="en">
> > > > <head>
> > > > <meta charset="utf-8">
> > > > <meta http-equiv="X-UA-Compatible" content="IE=edge">
> > > > <meta name="viewport" content="width=device-width,
> initial-scale=1">
> > > > <title>Apache Flink Web Dashboard</title>
> > > > <link rel="apple-touch-icon" sizes="180x180"
> > > href="assets/favicon/apple-touch-icon.png">
> > > > <link rel="icon" type="image/png"
> > > href="assets/favicon/favicon-32x32.png" sizes="32x32">
> > > > <link rel="icon" type="image/png"
> > > href="assets/favicon/favicon-16x16.png" sizes="16x16">
> > > > <link rel="manifest" href="assets/favicon/manifest.json">
> > > > <link rel="mask-icon"
> href="assets/favicon/safari-pinned-tab.svg"
> > > color="#aa1919">
> > > > <link rel="shortcut icon" href="assets/favicon/favicon.ico">
> > > > <meta name="msapplication-config"
> > > content="assets/favicon/browserconfig.xml">
> > > > <meta name="theme-color" content="#ffffff">
> > > > <base href="./"><link rel="stylesheet"
> > > href="styles.30d0912c1ece284d8d9a.css"></head>
> > > > <body>
> > > > <flink-root></flink-root>
> > > > <script type="text/javascript"
> > > src="runtime.ebbfed06bb8e06aaa889.js"></script><script
> > > type="text/javascript"
> src="es2015-polyfills.5e343224e81eefb7658e.js"
> > > nomodule></script><script type="text/javascript"
> > > src="polyfills.b37850e8279bc3caafc9.js"></script><script
> > > type="text/javascript"
> src="main.dc6e0997d092df051db0.js"></script></body>
> > > > </html>
> > > >
> > > > Thanks for any ideas,
> > > > Matt
> > > >
> > > > >>> bring-up-cluster
> > > >
> > > > # [yamls from here](
> > >
> https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fci.apache.org%2Fprojects%2Fflink%2Fflink-docs-stable%2Fops%2Fdeployment%2Fkubernetes.html&data=02%7C01%7Cmpatterson%40aligntech.com%7C118675de36e745219ee508d73bc9fb6f%7C9ac44c96980a481bae23d8f56b82c605%7C0%7C0%7C637043611035577083&sdata=CISNOH4Yw5oYzkQnq9fc%2B64svTpuvMHVXl9j5Qxv2FU%3D&reserved=0
> > > )
> > > > minikube start
> > > > minikube ssh 'sudo ip link set docker0 promisc on'
> > > >
> > > > kubectl create -f flink-configuration-configmap.yaml
> > > > kubectl create -f jobmanager-service.yaml
> > > > kubectl create -f jobmanager-deployment.yaml
> > > > kubectl create -f taskmanager-deployment.yaml
> > > > kubectl apply -f jobmanager-rest-service.yaml
> > > >
> > > > >>> grab-the-code
> > > > mvn archetype:generate \
> > > > -DarchetypeGroupId=org.apache.beam \
> > > >
> -DarchetypeArtifactId=beam-sdks-java-maven-archetypes-examples \
> > > > -DarchetypeVersion=2.15.0 \
> > > > -DgroupId=org.example \
> > > > -DartifactId=word-count-beam \
> > > > -Dversion="0.1" \
> > > > -Dpackage=org.apache.beam.examples \
> > > > -DinteractiveMode=false
> > > >
> > > > >>> maven-submit
> > > >
> > > > # kubectl port-forward svc/flink-jobmanager-rest 8081:8081 # in
> separate
> > > shell, as blocking
> > > > mvn package exec:java -X
> > > -Dexec.mainClass=org.apache.beam.examples.DebuggingWordCount \
> > > > -Dexec.args="--runner=FlinkRunner\
> > > > --flinkMaster="localhost:8081"\
> > > > --filesToStage=target/word-count-beam-bundled-0.1.jar\
> > > > --output=<directory-that-exists>" -Pflink-runner
> > > >
> > > >
> > > >
> > >
> >
>
>
>
