"WARNING:root:No unique name set for transform ..." should not affect the pipeline's ability to complete successfully. Is the pipeline failing? If so, could you share more logs?
Kyle Weaver | Software Engineer | github.com/ibzib | [email protected] On Mon, Sep 23, 2019 at 11:43 AM Matthew Patterson <[email protected]> wrote: > After setting runner, adding *'--environment_type=LOOPBACK'*, I now get > > > > “ > > … > > WARNING:root:No unique name set for transform > fn/read/ref_PCollection_PCollection_17:0 > > WARNING:root:No unique name set for transform > fn/read/ref_PCollection_PCollection_17:0 > > WARNING:root:No unique name set for transform > fn/write/ref_PCollection_PCollection_24:0 > > … > > ” > > when running wordcount example: > > > > > > *# We use the save_main_session option because one or more DoFn's in this > # workflow rely on global context (e.g., a module imported at module > level). *pipeline_options = runner_options > pipeline_options.view_as(SetupOptions).save_main_session = > > *True *p = beam.Pipeline(options=pipeline_options) > > > > *# Read the text file[pattern] into a PCollection. *lines = p | *'read' *>> > ReadFromText(_input) > > > *# Count the occurrences of each word. **def *count_ones(word_ones): > (word, ones) = word_ones > *return *(word, sum(ones)) > > counts = (lines > | *'split' *>> > (beam.ParDo(WordExtractingDoFn()).with_output_types(unicode)) > | *'pair_with_one' *>> beam.Map(*lambda *x: (x, 1)) > | *'group' *>> beam.GroupByKey() > | *'count' *>> beam.Map(count_ones)) > > > *# Format the counts into a PCollection of strings. **def * > format_result(word_count): > (word, count) = word_count > *return **'%s: %d' *% (word, count) > > output = counts | *'format' *>> beam.Map(format_result) > > > > *# Write the output using a "Write" transform that has side effects. # > pylint: disable=expression-not-assigned *output | *'write' *>> > WriteToText(_output) > > result = p.run() > result.wait_until_finish() > > > > > > > > *From: *Matthew Patterson <[email protected]> > *Reply-To: *"[email protected]" <[email protected]> > *Date: *Monday, September 23, 2019 at 2:22 PM > *To: *"[email protected]" <[email protected]> > *Subject: *Re: Word-count example > > > > Sorry: needed to add LOOPBACK > > > > *From: *Matthew Patterson <[email protected]> > *Reply-To: *"[email protected]" <[email protected]> > *Date: *Monday, September 23, 2019 at 2:18 PM > *To: *"[email protected]" <[email protected]> > *Subject: *Re: Word-count example > > > > OK, now using release-2.15.0, calling word-count example with runner, > after building & calling > > > > ./gradlew :runners:flink:1.5:job-server:runShadow (tried 1.8 as well) > > > > , as > > > > “*f'--runner=**PortableRunner**'*, *f'--job_endpoint=localhost:8099'* > > ” > > …and getting > > > > File > "/Users/mpatterson/miniconda/envs/aws/lib/python3.7/site-packages/apache_beam/runners/portability/portable_runner.py", > line 446, in wait_until_finish > > self._job_id, self._state, self._last_error_message())) > > RuntimeError: Pipeline > BeamApp-mpatterson-0923181154-d9c8a662_737ea859-53e2-4d7e-ba94-2955a1642f94 > failed in state FAILED: > org.apache.beam.vendor.grpc.v1p21p0.io.grpc.StatusRuntimeException: > CANCELLED: cancelled before receiving half close > > > > Ideas? Thanks! > > > > *From: *Matthew Patterson <[email protected]> > *Reply-To: *"[email protected]" <[email protected]> > *Date: *Friday, September 20, 2019 at 9:27 AM > *To: *"[email protected]" <[email protected]> > *Subject: *Re: Word-count example > > > > 411: > > > > Although it does not seem correct, adding a symbolic link to `virtualenv` > in ` <path-to-beam>/sdks/python` solves the issue of not finding > `virtualenv`. > > > > *From: *Matthew Patterson <[email protected]> > *Reply-To: *"[email protected]" <[email protected]> > *Date: *Thursday, September 19, 2019 at 2:56 PM > *To: *"[email protected]" <[email protected]> > *Subject: *Re: Word-count example > > > > Re Kyle: > > > > Just so you may believe me, see below(virtualenv installed, error occurs > nonetheless): currently I am trying to figure out if it is a difference > between .bash_profile and .bashrc. > > > > Thanks, > > Matt > > > > “”” > > Caused by: net.rubygrapefruit.platform.NativeException: Could not start > 'virtualenv' > > at > net.rubygrapefruit.platform.internal.DefaultProcessLauncher.start(DefaultProcessLauncher.java:27) > > at > net.rubygrapefruit.platform.internal.WrapperProcessLauncher.start(WrapperProcessLauncher.java:36) > > at > org.gradle.process.internal.ExecHandleRunner.startProcess(ExecHandleRunner.java:97) > > at > org.gradle.process.internal.ExecHandleRunner.run(ExecHandleRunner.java:70) > > ... 4 more > > Caused by: java.io.IOException: Cannot run program "virtualenv" (in > directory "/Users/mpatterson/dev/beam/sdks/python"): error=2, No such file > or directory > > at > net.rubygrapefruit.platform.internal.DefaultProcessLauncher.start(DefaultProcessLauncher.java:25) > > ... 7 more > > Caused by: java.io.IOException: error=2, No such file or directory > > ... 8 more > > > > > > * Get more help at *https://help.gradle.org <https://help.gradle.org>* > > > > Deprecated Gradle features were used in this build, making it incompatible > with Gradle 6.0. > > Use '--warning-mode all' to show the individual deprecation warnings. > > See > https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings > > > > *BUILD FAILED* in 0s > > 5 actionable tasks: 3 executed, 2 up-to-date > > bash-3.2$ ls /Users/mpatterson/dev/beam/sdks/python > > MANIFEST.in build.gradle gen_protos.py > setup.py tox.ini > > OWNERS container scripts > test-suites > > apache_beam findSupportedPython.groovy setup.cfg > test_config.py > > bash-3.2$ pip install virtualenv > > Requirement already satisfied: virtualenv in > /Users/mpatterson/miniconda/lib/python3.7/site-packages (16.7.5) > > bash-3.2$ ls /Users/mpatterson/dev/beam/sdks/python > > MANIFEST.in build.gradle gen_protos.py > setup.py tox.ini > > OWNERS container scripts > test-suites > > apache_beam findSupportedPython.groovy setup.cfg > test_config.py > > bash-3.2$ > > > > ””” > > > > *From: *Matthew Patterson <[email protected]> > *Reply-To: *"[email protected]" <[email protected]> > *Date: *Thursday, September 19, 2019 at 2:31 PM > *To: *"[email protected]" <[email protected]> > *Subject: *Re: Word-count example > > > > Have done: hard to get gradle to respect env (conda? Bash? Fish? Zsh?): > > > > *From: *Kyle Weaver <[email protected]> > *Reply-To: *"[email protected]" <[email protected]> > *Date: *Thursday, September 19, 2019 at 2:29 PM > *To: *"[email protected]" <[email protected]> > *Subject: *Re: Word-count example > > > > *CAUTION:* This email originated from outside of the organization. Do not > click links or open attachments unless you recognize the sender and know > the content is safe. > > > > I'm guessing you need to install virtualenv: `pip install virtualenv` > > > Kyle Weaver | Software Engineer | github.com/ibzib > <https://nam01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgithub.com%2Fibzib&data=02%7C01%7Cmpatterson%40aligntech.com%7Cb6d317f8a70940d2ced408d74052f2bf%7C9ac44c96980a481bae23d8f56b82c605%7C0%7C0%7C637048597336750345&sdata=YfCAcNKIdXqeFQ7RyDKv1NZO4vgjTmc0d31xK9wJSX0%3D&reserved=0> > | [email protected] > > > > > > On Thu, Sep 19, 2019 at 11:27 AM Matthew Patterson < > [email protected]> wrote: > > Kyle, > > > > Excellent, will do: unfortunately switch to 2.16 was only thing that fixed > > > > FAILURE: Build failed with an exception. > > > > * What went wrong: > > Execution failed for task ':sdks:python:setupVirtualenv'. > > > A problem occurred starting process 'command 'virtualenv'' > > > > > > …if you have suggestions there, I would be most appreciative. > > Matt > > > > > > *From: *Kyle Weaver <[email protected]> > *Reply-To: *"[email protected]" <[email protected]> > *Date: *Thursday, September 19, 2019 at 2:25 PM > *To: *"[email protected]" <[email protected]> > *Subject: *Re: Word-count example > > > > *CAUTION:* This email originated from outside of the organization. Do not > click links or open attachments unless you recognize the sender and know > the content is safe. > > > > You should probably use 2.15, since 2.16 release artifacts have not been > published yet. Just follow the instructions that > say --runner=PortableRunner, not --runner=FlinkRunner, otherwise you'll hit > that other deserialization bug that was mentioned.. > > > Kyle Weaver | Software Engineer | github.com/ibzib > <https://nam01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgithub.com%2Fibzib&data=02%7C01%7Cmpatterson%40aligntech.com%7Cb6d317f8a70940d2ced408d74052f2bf%7C9ac44c96980a481bae23d8f56b82c605%7C0%7C0%7C637048597336760341&sdata=2AnoIOi2rlZnX1TkUNL5BcN8gPGTN7t0bbEtMLE4eCM%3D&reserved=0> > | [email protected] > > > > > > On Thu, Sep 19, 2019 at 11:19 AM Matthew Patterson < > [email protected]> wrote: > > Kyle, > > > > Happily: should I be working from (source) release-2.15.0 or > release-2.16.0: presumably still specifying flink-1.8. > > Matt > > > > *From: *Kyle Weaver <[email protected]> > *Reply-To: *"[email protected]" <[email protected]> > *Date: *Thursday, September 19, 2019 at 2:16 PM > *To: *"[email protected]" <[email protected]> > *Subject: *Re: Word-count example > > > > *CAUTION:* This email originated from outside of the organization. Do not > click links or open attachments unless you recognize the sender and know > the content is safe. > > > > Re Benjamin & Ankur: I don't think `--experiments=beam_fn_api` applies > here. If I understand correctly, Matthew is just trying to run an > old-fashioned Beam Java jar, nothing to do with portability/Python, and > judging by the stack trace provided, > https://issues.apache.org/jira/browse/BEAM-8037 > <https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FBEAM-8037&data=02%7C01%7Cmpatterson%40aligntech.com%7Cb6d317f8a70940d2ced408d74052f2bf%7C9ac44c96980a481bae23d8f56b82c605%7C0%7C0%7C637048597336770339&sdata=mUyefW8aVc%2B6ycry0afaA%2BcRBXctn4aQ1tJJ7IICokA%3D&reserved=0> > is > not the issue here. > > > > Re Matthew: I suspect a Flink version mismatch between the Flink cluster > and the Beam runner. However, I am not familiar with the Maven commands > you're using. Since you're planning on running Python eventually anyway, > could you try following the "portable"/Python instructions on > https://beam.apache.org/documentation/runners/flink/ > <https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbeam.apache.org%2Fdocumentation%2Frunners%2Fflink%2F&data=02%7C01%7Cmpatterson%40aligntech.com%7Cb6d317f8a70940d2ced408d74052f2bf%7C9ac44c96980a481bae23d8f56b82c605%7C0%7C0%7C637048597336770339&sdata=wcmNi%2FMN0ryq787vQ54%2FKn37GI97D8GcgWxAXOqVvD4%3D&reserved=0> > and > let us know how that goes? > > > Kyle Weaver | Software Engineer | github.com/ibzib > <https://nam01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgithub.com%2Fibzib&data=02%7C01%7Cmpatterson%40aligntech.com%7Cb6d317f8a70940d2ced408d74052f2bf%7C9ac44c96980a481bae23d8f56b82c605%7C0%7C0%7C637048597336780334&sdata=XCXdKOj8DiDZu9qNPRl8rl3xqZaczzoLgb%2FH%2FDi8zTc%3D&reserved=0> > | [email protected] > > > > > > On Thu, Sep 19, 2019 at 10:52 AM Matthew Patterson < > [email protected]> wrote: > > Thanks Ankur, > > > > As one who speaks almost no gradle, is there a preferred way to get gradle > to respect the conda configured python on `gradlew build` ? > > > > Matt > > > > *From: *Ankur Goenka <[email protected]> > *Reply-To: *"[email protected]" <[email protected]> > *Date: *Thursday, September 19, 2019 at 1:50 PM > *To: *"[email protected]" <[email protected]> > *Subject: *Re: Word-count example > > > > *CAUTION:* This email originated from outside of the organization. Do not > click links or open attachments unless you recognize the sender and know > the content is safe. > > > > We have this bug in 2.15 which is discussed here > https://lists.apache.org/thread.html/76150a1ffca859bae7af0c6fb91724dc405dc55efc51c3b515f0520b@%3Cuser.beam.apache.org%3E > <https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread.html%2F76150a1ffca859bae7af0c6fb91724dc405dc55efc51c3b515f0520b%40%253Cuser.beam.apache.org%253E&data=02%7C01%7Cmpatterson%40aligntech.com%7Cb6d317f8a70940d2ced408d74052f2bf%7C9ac44c96980a481bae23d8f56b82c605%7C0%7C0%7C637048597336790329&sdata=TkmwQ4GqRBycmn4s%2BgCkMyG8SO6dsYNKL9NRYIPlfX0%3D&reserved=0> > > > > For now, please add "--experiments=beam_fn_api" to your pipeline to make > it work. > > > > On Thu, Sep 19, 2019 at 6:03 AM Matthew Patterson < > [email protected]> wrote: > > Hi Ankur, > > > > Yes, I was using 2.15, but was getting failure to deserialize. > > > > Thanks, > > Matt > > > > *From: *Ankur Goenka <[email protected]> > *Reply-To: *"[email protected]" <[email protected]> > *Date: *Wednesday, September 18, 2019 at 9:34 PM > *To: *"[email protected]" <[email protected]> > *Subject: *Re: Word-count example > > > > *CAUTION:* This email originated from outside of the organization. Do not > click links or open attachments unless you recognize the sender and know > the content is safe. > > > > Hi Matthew, > > > > Beam 2.16.0 is not yet released hence you are getting the error. Can you > try using 2.15.0 version? > > > > Thanks, > > Ankur > > > > On Wed, Sep 18, 2019 at 6:59 AM Matthew Patterson < > [email protected]> wrote: > > Tried > > " > mvn archetype:generate \ > -DarchetypeGroupId=org.apache.beam \ > -DarchetypeArtifactId=beam-sdks-java-maven-archetypes-examples \ > -DarchetypeVersion=2.16.0 \ > -DgroupId=org.example \ > -DartifactId=word-count-beam \ > -Dversion="0.1" \ > -Dpackage=org.apache.beam.examples \ > -DinteractiveMode=false > " > got > > -> [ERROR] Failed to execute goal > org.apache.maven.plugins:maven-archetype-plugin:3.1.1:generate > (default-cli) on project standalone-pom: The desired archetype does not > exist (org.apache.beam:beam-sdks-java-maven-archetypes-examples:2.16.0) -> > [Help 1] > > > On 9/17/19, 7:51 PM, "Benjamin Tan" <[email protected]> wrote: > > CAUTION: This email originated from outside of the organization. Do > not click links or open attachments unless you recognize the sender and > know the content is safe. > > > Ah. So maybe try 2.16.0? > > On 2019/09/17 23:47:16, Kyle Weaver <[email protected]> wrote: > > --experiments=beam_fn_api doesn't apply here, as this is a Java > pipeline > > using the non-portable version of the Flink runner. > > > > Kyle Weaver | Software Engineer | github.com/ibzib > <https://nam01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgithub.com%2Fibzib&data=02%7C01%7Cmpatterson%40aligntech.com%7Cb6d317f8a70940d2ced408d74052f2bf%7C9ac44c96980a481bae23d8f56b82c605%7C0%7C0%7C637048597336800326&sdata=8mLZgtzoEg9T%2BZmtEMLGUDjIofbeDuaRkBb5kY0HF9U%3D&reserved=0> > | [email protected] > > > > > > On Tue, Sep 17, 2019 at 4:41 PM Benjamin Tan < > [email protected]> > > wrote: > > > > > Could you try adding "--experiments=beam_fn_api" to the runner > options? > > > Kyle did mention that it should be fixed in 2.16.0. > > > > > > On 2019/09/17 16:40:41, Matthew Patterson < > [email protected]> > > > wrote: > > > > Folks, > > > > > > > > Am sure I am doing it wrong, but have been beating head against > wall > > > long enough I need to ask for help. > > > > > > > > My goal is to use the python SDK to run jobs on AWS flink > cluster, but > > > very little is working, so I am trying to back down to the very > simplest > > > thing that works, and build back up: not going so well. > > > > > > > > Now I am running, or trying to, on a minikube link cluster. Now > running > > > from just java/mvn, so I go through the following: > > > > > > > > 1. bring-up-cluster (see below) > > > > 2. grab-the-code (see below) > > > > 3. maven-submit (see below) > > > > > > > > Eventual failure is > > > > > > > > “[ERROR] Failed to execute goal > > > org.codehaus.mojo:exec-maven-plugin:1.6.0:java (default-cli) on > project > > > word-count-beam: An exception occured while executing the Java > class. > > > Pipeline execution failed: Could not retrieve the execution > result. (JobID: > > > 6edad04faa43fbf22bdea24171ba9d59): Failed to submit JobGraph. > [Failed to > > > deserialize JobGraph.]” > > > > > > > > and log is attached (I hope) > > > > > > > > Other details: > > > > > > > > on osx > > > > > > > > > > > > > > > > ➜ minikube version > > > > > > > > minikube version: v1.3.1 > > > > > > > > commit: ca60a424ce69a4d79f502650199ca2b52f29e631 > > > > > > > > > > > > ➜ kubectl version > > > > > > > > Client Version: version.Info{Major:"1", Minor:"14", > > > GitVersion:"v1.14.6", > GitCommit:"96fac5cd13a5dc064f7d9f4f23030a6aeface6cc", > > > GitTreeState:"clean", BuildDate:"2019-08-19T11:13:49Z", > > > GoVersion:"go1.12.9", Compiler:"gc", Platform:"darwin/amd64"} > > > > > > > > Server Version: version.Info{Major:"1", Minor:"15", > > > GitVersion:"v1.15.2", > GitCommit:"f6278300bebbb750328ac16ee6dd3aa7d3549568", > > > GitTreeState:"clean", BuildDate:"2019-08-05T09:15:22Z", > > > GoVersion:"go1.12.5", Compiler:"gc", Platform:"linux/amd64"} > > > > > > > > > > > > ➜ flink --version > > > > > > > > Version: 1.8.1, Commit ID: 7297bac > > > > > > > > ➜ python -c"import apache_beam; print(apache_beam.__version__)" > > > > 2.15.0 > > > > > > > > ➜ curl localhost:8081 > > > > <!-- > > > > ~ Licensed to the Apache Software Foundation (ASF) under one > > > > ~ or more contributor license agreements. See the NOTICE file > > > > ~ distributed with this work for additional information > > > > ~ regarding copyright ownership. The ASF licenses this file > > > > ~ to you under the Apache License, Version 2.0 (the > > > > ~ "License"); you may not use this file except in compliance > > > > ~ with the License. You may obtain a copy of the License at > > > > ~ > > > > ~ > https://nam01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.apache.org%2Flicenses%2FLICENSE-2.0&data=02%7C01%7Cmpatterson%40aligntech.com%7C118675de36e745219ee508d73bc9fb6f%7C9ac44c96980a481bae23d8f56b82c605%7C0%7C0%7C637043611035577083&sdata=4WV%2Bk3Wao0qrbwKJEJTYxG0KixNNGEN%2FW%2FQLz2MrbDk%3D&reserved=0 > <https://nam01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.apache.org%2Flicenses%2FLICENSE-2.0&data=02%7C01%7Cmpatterson%40aligntech.com%7Cb6d317f8a70940d2ced408d74052f2bf%7C9ac44c96980a481bae23d8f56b82c605%7C0%7C0%7C637048597336800326&sdata=pU44hhkq715BHRd2UaPCdVjmXHrg8%2FVr9LmjiIj8Q24%3D&reserved=0> > > > > ~ > > > > ~ Unless required by applicable law or agreed to in writing, > software > > > > ~ distributed under the License is distributed on an "AS IS" > BASIS, > > > > ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express > or > > > implied. > > > > ~ See the License for the specific language governing > permissions and > > > > ~ limitations under the License. > > > > --> > > > > > > > > <!doctype html> > > > > <html lang="en"> > > > > <head> > > > > <meta charset="utf-8"> > > > > <meta http-equiv="X-UA-Compatible" content="IE=edge"> > > > > <meta name="viewport" content="width=device-width, > initial-scale=1"> > > > > <title>Apache Flink Web Dashboard</title> > > > > <link rel="apple-touch-icon" sizes="180x180" > > > href="assets/favicon/apple-touch-icon.png"> > > > > <link rel="icon" type="image/png" > > > href="assets/favicon/favicon-32x32.png" sizes="32x32"> > > > > <link rel="icon" type="image/png" > > > href="assets/favicon/favicon-16x16.png" sizes="16x16"> > > > > <link rel="manifest" href="assets/favicon/manifest.json"> > > > > <link rel="mask-icon" > href="assets/favicon/safari-pinned-tab.svg" > > > color="#aa1919"> > > > > <link rel="shortcut icon" href="assets/favicon/favicon.ico"> > > > > <meta name="msapplication-config" > > > content="assets/favicon/browserconfig.xml"> > > > > <meta name="theme-color" content="#ffffff"> > > > > <base href="./"><link rel="stylesheet" > > > href="styles.30d0912c1ece284d8d9a.css"></head> > > > > <body> > > > > <flink-root></flink-root> > > > > <script type="text/javascript" > > > src="runtime.ebbfed06bb8e06aaa889.js"></script><script > > > type="text/javascript" > src="es2015-polyfills.5e343224e81eefb7658e.js" > > > nomodule></script><script type="text/javascript" > > > src="polyfills.b37850e8279bc3caafc9.js"></script><script > > > type="text/javascript" > src="main.dc6e0997d092df051db0.js"></script></body> > > > > </html> > > > > > > > > Thanks for any ideas, > > > > Matt > > > > > > > > >>> bring-up-cluster > > > > > > > > # [yamls from here]( > > > > https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fci.apache.org%2Fprojects%2Fflink%2Fflink-docs-stable%2Fops%2Fdeployment%2Fkubernetes.html&data=02%7C01%7Cmpatterson%40aligntech.com%7C118675de36e745219ee508d73bc9fb6f%7C9ac44c96980a481bae23d8f56b82c605%7C0%7C0%7C637043611035577083&sdata=CISNOH4Yw5oYzkQnq9fc%2B64svTpuvMHVXl9j5Qxv2FU%3D&reserved=0 > <https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fci.apache.org%2Fprojects%2Fflink%2Fflink-docs-stable%2Fops%2Fdeployment%2Fkubernetes.html&data=02%7C01%7Cmpatterson%40aligntech.com%7Cb6d317f8a70940d2ced408d74052f2bf%7C9ac44c96980a481bae23d8f56b82c605%7C0%7C0%7C637048597336810322&sdata=Xj2HftLL5JSL5IKKoFpEix5wuo15ctDbE9k64ti9%2B%2FM%3D&reserved=0> > > > ) > > > > minikube start > > > > minikube ssh 'sudo ip link set docker0 promisc on' > > > > > > > > kubectl create -f flink-configuration-configmap.yaml > > > > kubectl create -f jobmanager-service.yaml > > > > kubectl create -f jobmanager-deployment.yaml > > > > kubectl create -f taskmanager-deployment.yaml > > > > kubectl apply -f jobmanager-rest-service.yaml > > > > > > > > >>> grab-the-code > > > > mvn archetype:generate \ > > > > -DarchetypeGroupId=org.apache.beam \ > > > > > -DarchetypeArtifactId=beam-sdks-java-maven-archetypes-examples \ > > > > -DarchetypeVersion=2.15.0 \ > > > > -DgroupId=org.example \ > > > > -DartifactId=word-count-beam \ > > > > -Dversion="0.1" \ > > > > -Dpackage=org.apache.beam.examples \ > > > > -DinteractiveMode=false > > > > > > > > >>> maven-submit > > > > > > > > # kubectl port-forward svc/flink-jobmanager-rest 8081:8081 # in > separate > > > shell, as blocking > > > > mvn package exec:java -X > > > -Dexec.mainClass=org.apache.beam.examples.DebuggingWordCount \ > > > > -Dexec.args="--runner=FlinkRunner\ > > > > --flinkMaster="localhost:8081"\ > > > > --filesToStage=target/word-count-beam-bundled-0.1.jar\ > > > > --output=<directory-that-exists>" -Pflink-runner > > > > > > > > > > > > > > > > > > >
