[jira] [Commented] (BEAM-5663) Add tox suites for various Python 3 versions
[ https://issues.apache.org/jira/browse/BEAM-5663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16645989#comment-16645989 ] Manu Zhang commented on BEAM-5663: -- Do you (or anyone) know how to set up Python version on Jenkins ? > Add tox suites for various Python 3 versions > > > Key: BEAM-5663 > URL: https://issues.apache.org/jira/browse/BEAM-5663 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-core >Reporter: Manu Zhang >Priority: Major > > Currently, Python 3.5.2 is set up for Jenkins tests but we've seen test > failings across various Python 3 versions. It will be valuable to add tox > suites for Python 3.4, 3.5, 3.6 and 3.7 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (BEAM-5663) Add tox suites for various Python 3 versions
[ https://issues.apache.org/jira/browse/BEAM-5663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manu Zhang reassigned BEAM-5663: Assignee: (was: Ahmet Altay) > Add tox suites for various Python 3 versions > > > Key: BEAM-5663 > URL: https://issues.apache.org/jira/browse/BEAM-5663 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-core >Reporter: Manu Zhang >Priority: Major > > Currently, Python 3.5.2 is set up for Jenkins tests but we've seen test > failings across various Python 3 versions. It will be valuable to add tox > suites for Python 3.4, 3.5, 3.6 and 3.7 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (BEAM-5663) Add tox suites for various Python 3 versions
Manu Zhang created BEAM-5663: Summary: Add tox suites for various Python 3 versions Key: BEAM-5663 URL: https://issues.apache.org/jira/browse/BEAM-5663 Project: Beam Issue Type: Sub-task Components: sdk-py-core Reporter: Manu Zhang Assignee: Ahmet Altay Currently, Python 3.5.2 is set up for Jenkins tests but we've seen test failings across various Python 3 versions. It will be valuable to add tox suites for Python 3.4, 3.5, 3.6 and 3.7 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (BEAM-5620) Some tests use assertItemsEqual method, not available in Python 3
[ https://issues.apache.org/jira/browse/BEAM-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16637744#comment-16637744 ] Manu Zhang commented on BEAM-5620: -- [~tvalentyn], I've seen fixes like [https://github.com/apache/beam/blob/c34c367f5da6f9bef8a46471195470923a201af9/sdks/python/apache_beam/coders/coders_test_common.py#L61] where `assertItemsEqual` is assigned to `assertCountEqual` on Python 2. Do you think it's a good idea to put it in a subclass of `TestCase` ? > Some tests use assertItemsEqual method, not available in Python 3 > - > > Key: BEAM-5620 > URL: https://issues.apache.org/jira/browse/BEAM-5620 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-core >Reporter: Valentyn Tymofieiev >Priority: Major > > See: > https://github.com/apache/beam/search?q=assertItemsEqual_q=assertItemsEqual -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (BEAM-5607) Typing library no longer provisional in Python 3.7
[
https://issues.apache.org/jira/browse/BEAM-5607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Manu Zhang updated BEAM-5607:
-
Issue Type: Sub-task (was: Improvement)
Parent: BEAM-1251
> Typing library no longer provisional in Python 3.7
> --
>
> Key: BEAM-5607
> URL: https://issues.apache.org/jira/browse/BEAM-5607
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-py-core
>Reporter: Manu Zhang
>Assignee: Manu Zhang
>Priority: Minor
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> This is an issue coming from future since the target version in the [initial
> proposal|https://lists.apache.org/thread.html/5371469de567357b1431606f766217ef73a9098dc45046f51a6ecceb@%3Cdev.beam.apache.org%3E]
> is up to 3.6. Running python tests in 3.7 would throw the following error
> because the [typing library is no longer provisional in Python
> 3.7|https://github.com/python/typing#important-dates]
> {code}
> sdks/python/.eggs/typing-3.6.6-py3.7.egg/typing.py", line 1004, in __new__
> self._abc_registry = extra._abc_registry
> AttributeError: type object 'Callable' has no attribute '_abc_registry'
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (BEAM-5607) Typing library no longer provisional in Python 3.7
[
https://issues.apache.org/jira/browse/BEAM-5607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Manu Zhang reassigned BEAM-5607:
Assignee: Manu Zhang (was: Ahmet Altay)
> Typing library no longer provisional in Python 3.7
> --
>
> Key: BEAM-5607
> URL: https://issues.apache.org/jira/browse/BEAM-5607
> Project: Beam
> Issue Type: Improvement
> Components: sdk-py-core
>Reporter: Manu Zhang
>Assignee: Manu Zhang
>Priority: Minor
>
> This is an issue coming from future since the target version in the [initial
> proposal|https://lists.apache.org/thread.html/5371469de567357b1431606f766217ef73a9098dc45046f51a6ecceb@%3Cdev.beam.apache.org%3E]
> is up to 3.6. Running python tests in 3.7 would throw the following error
> because the [typing library is no longer provisional in Python
> 3.7|https://github.com/python/typing#important-dates]
> {code}
> sdks/python/.eggs/typing-3.6.6-py3.7.egg/typing.py", line 1004, in __new__
> self._abc_registry = extra._abc_registry
> AttributeError: type object 'Callable' has no attribute '_abc_registry'
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (BEAM-5607) Typing library no longer provisional in Python 3.7
Manu Zhang created BEAM-5607:
Summary: Typing library no longer provisional in Python 3.7
Key: BEAM-5607
URL: https://issues.apache.org/jira/browse/BEAM-5607
Project: Beam
Issue Type: Improvement
Components: sdk-py-core
Reporter: Manu Zhang
Assignee: Ahmet Altay
This is an issue coming from future since the target version in the [initial
proposal|https://lists.apache.org/thread.html/5371469de567357b1431606f766217ef73a9098dc45046f51a6ecceb@%3Cdev.beam.apache.org%3E]
is up to 3.6. Running python tests in 3.7 would throw the following error
because the [typing library is no longer provisional in Python
3.7|https://github.com/python/typing#important-dates]
{code}
sdks/python/.eggs/typing-3.6.6-py3.7.egg/typing.py", line 1004, in __new__
self._abc_registry = extra._abc_registry
AttributeError: type object 'Callable' has no attribute '_abc_registry'
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (BEAM-5317) Finish Python 3 porting for options module
[ https://issues.apache.org/jira/browse/BEAM-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manu Zhang reassigned BEAM-5317: Assignee: Manu Zhang > Finish Python 3 porting for options module > -- > > Key: BEAM-5317 > URL: https://issues.apache.org/jira/browse/BEAM-5317 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-core >Reporter: Robbe >Assignee: Manu Zhang >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (BEAM-4129) Update Gearpump runner instructions for running WordCount example
[ https://issues.apache.org/jira/browse/BEAM-4129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16447127#comment-16447127 ] Manu Zhang commented on BEAM-4129: -- Submitted https://github.com/apache/beam/pull/5200 > Update Gearpump runner instructions for running WordCount example > - > > Key: BEAM-4129 > URL: https://issues.apache.org/jira/browse/BEAM-4129 > Project: Beam > Issue Type: Sub-task > Components: runner-gearpump >Reporter: Scott Wegner >Assignee: Manu Zhang >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > With the move to Gradle, the Gearpump README.md needs to be updated with > instructions on how to execute WordCount with the new build system. > [https://github.com/apache/beam/blob/master/runners/gearpump/README.md] > > It may be easiest to generate a quickstart task similar to other runners: > https://github.com/apache/beam/blob/3cdecd0e3e28dd39990d7d1dea8931e871a6ba62/runners/flink/build.gradle#L135 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (BEAM-4129) Update Gearpump runner instructions for running WordCount example
[ https://issues.apache.org/jira/browse/BEAM-4129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16446745#comment-16446745 ] Manu Zhang commented on BEAM-4129: -- Sure, it seems most runners don't keep a README along with source codes. Shall we remove it ? > Update Gearpump runner instructions for running WordCount example > - > > Key: BEAM-4129 > URL: https://issues.apache.org/jira/browse/BEAM-4129 > Project: Beam > Issue Type: Sub-task > Components: runner-gearpump >Reporter: Scott Wegner >Assignee: Manu Zhang >Priority: Minor > > With the move to Gradle, the Gearpump README.md needs to be updated with > instructions on how to execute WordCount with the new build system. > [https://github.com/apache/beam/blob/master/runners/gearpump/README.md] > > It may be easiest to generate a quickstart task similar to other runners: > https://github.com/apache/beam/blob/3cdecd0e3e28dd39990d7d1dea8931e871a6ba62/runners/flink/build.gradle#L135 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (BEAM-3242) Gearpump runner does not respect ParDo's lifecycle on case of exceptions
[ https://issues.apache.org/jira/browse/BEAM-3242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16265606#comment-16265606 ] Manu Zhang commented on BEAM-3242: -- Hi Ismael, thanks for reporting. Could you elaborate on what should be respected in case of exceptions ? > Gearpump runner does not respect ParDo's lifecycle on case of exceptions > > > Key: BEAM-3242 > URL: https://issues.apache.org/jira/browse/BEAM-3242 > Project: Beam > Issue Type: Bug > Components: runner-gearpump >Affects Versions: 2.2.0 >Reporter: Ismaël Mejía >Assignee: Manu Zhang >Priority: Minor > > The lifecycle of the DoFn is not respected in case of exception in any of the > lifecycle methods after setup. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (BEAM-2788) Use SerializablePipelineOptions to serde PipelineOptions
Manu Zhang created BEAM-2788: Summary: Use SerializablePipelineOptions to serde PipelineOptions Key: BEAM-2788 URL: https://issues.apache.org/jira/browse/BEAM-2788 Project: Beam Issue Type: Improvement Components: runner-gearpump Affects Versions: 2.0.0 Reporter: Manu Zhang Assignee: Huafeng Wang Priority: Minor -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (BEAM-2759) Support Custom Windows in Gearpump runner
[
https://issues.apache.org/jira/browse/BEAM-2759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16121601#comment-16121601
]
Manu Zhang commented on BEAM-2759:
--
Thanks, we will look into it.
> Support Custom Windows in Gearpump runner
> -
>
> Key: BEAM-2759
> URL: https://issues.apache.org/jira/browse/BEAM-2759
> Project: Beam
> Issue Type: New Feature
> Components: runner-gearpump
>Reporter: Etienne Chauchot
>Assignee: Manu Zhang
>
> Running {{WindowTest#testMergingCustomWindows()}} and
> {{WindowTest#testMergingCustomWindowsKeyedCollection()}} in Gearpump runner
> show that these kind of windows do not get merged.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (BEAM-2756) Update Beam site to reflect Gearpump Runner on master
[ https://issues.apache.org/jira/browse/BEAM-2756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16120931#comment-16120931 ] Manu Zhang commented on BEAM-2756: -- duplicate of https://issues.apache.org/jira/browse/BEAM-2692 ? > Update Beam site to reflect Gearpump Runner on master > - > > Key: BEAM-2756 > URL: https://issues.apache.org/jira/browse/BEAM-2756 > Project: Beam > Issue Type: Task > Components: website >Reporter: Kenneth Knowles >Assignee: Manu Zhang > Fix For: Not applicable > > > These places should be changed, and maybe others: > - it is no longer an "ongoing project" in the Contribute section > - it should be listed with the other runners on the Documentation section -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (BEAM-79) Gearpump runner
[ https://issues.apache.org/jira/browse/BEAM-79?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117833#comment-16117833 ] Manu Zhang commented on BEAM-79: Hi [~takidau], We've reached a milestone after merging gearpump-runner into master. Future development will be under new JIRAs. Can we close this issue now ? Thanks for asking on Twitter and initiating this work a year ago. Manu > Gearpump runner > --- > > Key: BEAM-79 > URL: https://issues.apache.org/jira/browse/BEAM-79 > Project: Beam > Issue Type: New Feature > Components: runner-gearpump >Reporter: Tyler Akidau >Assignee: Manu Zhang > > Intel is submitting Gearpump (http://www.gearpump.io) to ASF > (https://wiki.apache.org/incubator/GearpumpProposal). Appears to be a mix of > low-level primitives a la MillWheel, with some higher level primitives like > non-merging windowing mixed in. Seems like it would make a nice Beam runner. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (BEAM-2693) GearpumpRunner Fn API based ParDo operator
[ https://issues.apache.org/jira/browse/BEAM-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manu Zhang updated BEAM-2693: - Issue Type: New Feature (was: Bug) > GearpumpRunner Fn API based ParDo operator > -- > > Key: BEAM-2693 > URL: https://issues.apache.org/jira/browse/BEAM-2693 > Project: Beam > Issue Type: New Feature > Components: runner-gearpump >Reporter: Manu Zhang >Assignee: Manu Zhang > > This is copied from Kenn's description in other jiras. > To run non-Java SDK code is to put together an operator that manages a Fn API > client DoFnRunner and an SDK harness Fn API server. > (filing to organize steps, details of this may evolve as it is implemented). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (BEAM-2694) Python shim for submitting to GearpumpRunner
[ https://issues.apache.org/jira/browse/BEAM-2694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manu Zhang updated BEAM-2694: - Issue Type: New Feature (was: Bug) > Python shim for submitting to GearpumpRunner > > > Key: BEAM-2694 > URL: https://issues.apache.org/jira/browse/BEAM-2694 > Project: Beam > Issue Type: New Feature > Components: runner-gearpump >Reporter: Manu Zhang >Assignee: Manu Zhang > Labels: beam-python-everywhere > > Shamelessly copied from Kenn's descriptions in other jiras, > Whatever the result of https://s.apache.org/beam-job-api, Python users will > need to be able to pass --runner=GearpumpRunner and have it work. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (BEAM-2695) GearpumpRunner shim for Job API
[ https://issues.apache.org/jira/browse/BEAM-2695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manu Zhang updated BEAM-2695: - Issue Type: New Feature (was: Bug) > GearpumpRunner shim for Job API > > > Key: BEAM-2695 > URL: https://issues.apache.org/jira/browse/BEAM-2695 > Project: Beam > Issue Type: New Feature > Components: runner-gearpump >Reporter: Manu Zhang >Assignee: Manu Zhang > > From Kenn's descriptions in other jiras, > Whatever the result of https://s.apache.org/beam-job-api we will need a way > for the JVM-based GearpumpRunner to receive and run pipelines authors in > Python. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (BEAM-2697) Support for new Timer API in GearpumpRunner
Manu Zhang created BEAM-2697: Summary: Support for new Timer API in GearpumpRunner Key: BEAM-2697 URL: https://issues.apache.org/jira/browse/BEAM-2697 Project: Beam Issue Type: New Feature Components: runner-gearpump Reporter: Manu Zhang Assignee: Manu Zhang -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (BEAM-2698) Support Splittable DoFn in GearpumpRunner
Manu Zhang created BEAM-2698: Summary: Support Splittable DoFn in GearpumpRunner Key: BEAM-2698 URL: https://issues.apache.org/jira/browse/BEAM-2698 Project: Beam Issue Type: New Feature Components: runner-gearpump Reporter: Manu Zhang Assignee: Manu Zhang -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (BEAM-2696) Support new State API in GearpumpRunner
Manu Zhang created BEAM-2696: Summary: Support new State API in GearpumpRunner Key: BEAM-2696 URL: https://issues.apache.org/jira/browse/BEAM-2696 Project: Beam Issue Type: New Feature Components: runner-gearpump Reporter: Manu Zhang Assignee: Manu Zhang -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (BEAM-2695) GearpumpRunner shim for Job API
Manu Zhang created BEAM-2695: Summary: GearpumpRunner shim for Job API Key: BEAM-2695 URL: https://issues.apache.org/jira/browse/BEAM-2695 Project: Beam Issue Type: Bug Components: runner-gearpump Reporter: Manu Zhang Assignee: Manu Zhang >From Kenn's descriptions in other jiras, Whatever the result of https://s.apache.org/beam-job-api we will need a way for the JVM-based GearpumpRunner to receive and run pipelines authors in Python. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (BEAM-2694) Python shim for submitting to GearpumpRunner
[ https://issues.apache.org/jira/browse/BEAM-2694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manu Zhang updated BEAM-2694: - Labels: beam-python-everywhere (was: ) > Python shim for submitting to GearpumpRunner > > > Key: BEAM-2694 > URL: https://issues.apache.org/jira/browse/BEAM-2694 > Project: Beam > Issue Type: Bug > Components: runner-gearpump >Reporter: Manu Zhang >Assignee: Manu Zhang > Labels: beam-python-everywhere > > Shamelessly copied from Kenn's descriptions in other jiras, > Whatever the result of https://s.apache.org/beam-job-api, Python users will > need to be able to pass --runner=GearpumpRunner and have it work. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (BEAM-2694) Python shim for submitting to GearpumpRunner
Manu Zhang created BEAM-2694: Summary: Python shim for submitting to GearpumpRunner Key: BEAM-2694 URL: https://issues.apache.org/jira/browse/BEAM-2694 Project: Beam Issue Type: Bug Components: runner-gearpump Reporter: Manu Zhang Assignee: Manu Zhang Shamelessly copied from Kenn's descriptions in other jiras, Whatever the result of https://s.apache.org/beam-job-api, Python users will need to be able to pass --runner=GearpumpRunner and have it work. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (BEAM-2691) Add gearpump-runner to jenkins-precommit tests
[
https://issues.apache.org/jira/browse/BEAM-2691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Manu Zhang updated BEAM-2691:
-
Labels: beam-python-everywhere (was: )
> Add gearpump-runner to jenkins-precommit tests
> --
>
> Key: BEAM-2691
> URL: https://issues.apache.org/jira/browse/BEAM-2691
> Project: Beam
> Issue Type: Bug
> Components: runner-gearpump
>Reporter: Manu Zhang
>Assignee: Manu Zhang
> Labels: beam-python-everywhere
>
> We are not able to do so now because {{java.lang.NoClassDefFoundError: Could
> not initialize class org.apache.beam.sdk.options.PipelineOptionsFactory} will
> be thrown if flink-runner and gearpump-runner are enabled simultaneously as
> shown in https://github.com/apache/beam/pull/3637#issuecomment-318256758.
> Either we find the root cause for the error and fix it or we revisit this
> after the new pipelined Jenkins config is in.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (BEAM-2693) GearpumpRunner Fn API based ParDo operator
Manu Zhang created BEAM-2693: Summary: GearpumpRunner Fn API based ParDo operator Key: BEAM-2693 URL: https://issues.apache.org/jira/browse/BEAM-2693 Project: Beam Issue Type: Bug Components: runner-gearpump Reporter: Manu Zhang Assignee: Manu Zhang This is copied from Kenn's description in other jiras. To run non-Java SDK code is to put together an operator that manages a Fn API client DoFnRunner and an SDK harness Fn API server. (filing to organize steps, details of this may evolve as it is implemented). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (BEAM-2692) Post-merge change to Beam website for gearpump-runner
Manu Zhang created BEAM-2692: Summary: Post-merge change to Beam website for gearpump-runner Key: BEAM-2692 URL: https://issues.apache.org/jira/browse/BEAM-2692 Project: Beam Issue Type: Bug Components: runner-gearpump Reporter: Manu Zhang Assignee: Manu Zhang # Add gearpump-runner to ["QuickStart-Java" page|https://beam.apache.org/get-started/quickstart-java/] # Add gearpump-runner to ["WordCount" page|https://beam.apache.org/get-started/wordcount-example/] # Remove "Under development" on ["Using the Documentation" page|https://beam.apache.org/documentation/] # Remove gearpump-runner from [feature-branches list|https://beam.apache.org/contribute/work-in-progress/#feature-branches] # Add a "Gearpump Runner" page This jira can further be decomposed into sub-tasks. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (BEAM-2691) Add gearpump-runner to jenkins-precommit tests
Manu Zhang created BEAM-2691:
Summary: Add gearpump-runner to jenkins-precommit tests
Key: BEAM-2691
URL: https://issues.apache.org/jira/browse/BEAM-2691
Project: Beam
Issue Type: Bug
Components: runner-gearpump
Reporter: Manu Zhang
Assignee: Manu Zhang
We are not able to do so now because {{java.lang.NoClassDefFoundError: Could
not initialize class org.apache.beam.sdk.options.PipelineOptionsFactory} will
be thrown if flink-runner and gearpump-runner are enabled simultaneously as
shown in https://github.com/apache/beam/pull/3637#issuecomment-318256758.
Either we find the root cause for the error and fix it or we revisit this after
the new pipelined Jenkins config is in.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (BEAM-2664) Make latest stable version more visible on Beam website
Manu Zhang created BEAM-2664: Summary: Make latest stable version more visible on Beam website Key: BEAM-2664 URL: https://issues.apache.org/jira/browse/BEAM-2664 Project: Beam Issue Type: Improvement Components: website Reporter: Manu Zhang Assignee: Davor Bonaci Priority: Minor Currently, one can find the latest stable version from the [Downloads Page |https://beam.apache.org/get-started/downloads/] in the "GET STARTED" pull down. It's better to make it more visible for users to find out. One improvement is to *add a "Download Beam" button* linking to the "Downloads Page" to the home page, like [Spark|http://spark.apache.org/] and [Flink|http://flink.apache.org/]. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (BEAM-2614) Harness doesn't build with Java7
[
https://issues.apache.org/jira/browse/BEAM-2614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16086800#comment-16086800
]
Manu Zhang commented on BEAM-2614:
--
The real problem is {{beam-sdks-java-javadoc}} has a dependency on
{{beam-sdks-java-harness}} regardless of Java version and fail to build as
following
{code}
[ERROR] Failed to execute goal on project beam-sdks-java-javadoc: Could not
resolve dependencies for project
org.apache.beam:beam-sdks-java-javadoc:pom:2.1.0: Failure to find
org.apache.beam:beam-sdks-java-harness:jar:2.1.0 in
https://repo.maven.apache.org/maven2 was cached in the local repository,
resolution will not be reattempted until the update interval of central has
elapsed or updates are forced
{code}
> Harness doesn't build with Java7
>
>
> Key: BEAM-2614
> URL: https://issues.apache.org/jira/browse/BEAM-2614
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-extensions
>Affects Versions: 2.0.0, 2.1.0
>Reporter: Jean-Baptiste Onofré
>Assignee: Jean-Baptiste Onofré
>
> Beam is supposed to fully build with Java7. However, the {{harness}} module
> doesn't:
> {code}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:3.6.1:compile
> (default-compile) on project beam-sdks-java-harness: Fatal error compiling:
> invalid target release: 1.8 -> [Help 1]
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Resolved] (BEAM-2526) Add direct-runner as runtime dependency and default profile
[ https://issues.apache.org/jira/browse/BEAM-2526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manu Zhang resolved BEAM-2526. -- Resolution: Duplicate Fix Version/s: Not applicable > Add direct-runner as runtime dependency and default profile > --- > > Key: BEAM-2526 > URL: https://issues.apache.org/jira/browse/BEAM-2526 > Project: Beam > Issue Type: Bug > Components: dsl-sql >Affects Versions: Not applicable >Reporter: Manu Zhang >Assignee: Manu Zhang >Priority: Minor > Fix For: Not applicable > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (BEAM-2526) Add direct-runner as runtime dependency and default profile
Manu Zhang created BEAM-2526: Summary: Add direct-runner as runtime dependency and default profile Key: BEAM-2526 URL: https://issues.apache.org/jira/browse/BEAM-2526 Project: Beam Issue Type: Bug Components: dsl-sql Affects Versions: Not applicable Reporter: Manu Zhang Assignee: Manu Zhang Priority: Minor -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (BEAM-688) Failure of beam-sdks-java-maven-archetypes-starter with undeclared dependency error
[
https://issues.apache.org/jira/browse/BEAM-688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16064859#comment-16064859
]
Manu Zhang commented on BEAM-688:
-
Related to https://issues.apache.org/jira/browse/MSHARED-47 ?
> Failure of beam-sdks-java-maven-archetypes-starter with undeclared dependency
> error
> ---
>
> Key: BEAM-688
> URL: https://issues.apache.org/jira/browse/BEAM-688
> Project: Beam
> Issue Type: Bug
> Components: build-system
>Reporter: Scott Wegner
> Labels: flake
>
> The starter archetype has flaky dependencies. It is reported to fail reliably
> on repeatedly install.
> {noformat}
> [INFO] --- maven-dependency-plugin:2.10:analyze-only (default) @
> beam-sdks-java-maven-archetypes-starter ---
> [WARNING] Used undeclared dependencies found:
> [WARNING]org.slf4j:slf4j-api:jar:1.7.14:runtime
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (BEAM-688) Failure of beam-sdks-java-maven-archetypes-starter with undeclared dependency error
[
https://issues.apache.org/jira/browse/BEAM-688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16062985#comment-16062985
]
Manu Zhang commented on BEAM-688:
-
I'm seeing this error again on latest master.
> Failure of beam-sdks-java-maven-archetypes-starter with undeclared dependency
> error
> ---
>
> Key: BEAM-688
> URL: https://issues.apache.org/jira/browse/BEAM-688
> Project: Beam
> Issue Type: Bug
> Components: build-system
>Reporter: Scott Wegner
> Labels: flake
>
> The starter archetype has flaky dependencies. It is reported to fail reliably
> on repeatedly install.
> {noformat}
> [INFO] --- maven-dependency-plugin:2.10:analyze-only (default) @
> beam-sdks-java-maven-archetypes-starter ---
> [WARNING] Used undeclared dependencies found:
> [WARNING]org.slf4j:slf4j-api:jar:1.7.14:runtime
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (BEAM-2514) Improve error message for missing required options of Beam pipeline
Manu Zhang created BEAM-2514: Summary: Improve error message for missing required options of Beam pipeline Key: BEAM-2514 URL: https://issues.apache.org/jira/browse/BEAM-2514 Project: Beam Issue Type: Improvement Components: sdk-java-core Affects Versions: 2.0.0 Reporter: Manu Zhang Assignee: Manu Zhang Priority: Minor -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (BEAM-968) Update capability matrix to include gearpump-runner
[ https://issues.apache.org/jira/browse/BEAM-968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manu Zhang resolved BEAM-968. - Resolution: Fixed Fix Version/s: Not applicable > Update capability matrix to include gearpump-runner > --- > > Key: BEAM-968 > URL: https://issues.apache.org/jira/browse/BEAM-968 > Project: Beam > Issue Type: Sub-task > Components: runner-gearpump, website >Reporter: Manu Zhang >Assignee: Huafeng Wang > Fix For: Not applicable > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (BEAM-969) Add a gearpump runner web page under "learn/runners"
[ https://issues.apache.org/jira/browse/BEAM-969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manu Zhang resolved BEAM-969. - Resolution: Fixed Fix Version/s: Not applicable > Add a gearpump runner web page under "learn/runners" > > > Key: BEAM-969 > URL: https://issues.apache.org/jira/browse/BEAM-969 > Project: Beam > Issue Type: Sub-task > Components: runner-gearpump, website >Reporter: Manu Zhang >Assignee: Huafeng Wang > Fix For: Not applicable > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (BEAM-2416) Add AkkaIO
[ https://issues.apache.org/jira/browse/BEAM-2416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16039955#comment-16039955 ] Manu Zhang commented on BEAM-2416: -- [~jbonofre], do you mean an akka-stream connector ? Could you give more descriptions ? > Add AkkaIO > -- > > Key: BEAM-2416 > URL: https://issues.apache.org/jira/browse/BEAM-2416 > Project: Beam > Issue Type: New Feature > Components: sdk-java-extensions >Reporter: Jean-Baptiste Onofré >Assignee: Jean-Baptiste Onofré > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (BEAM-2317) Incorrect Javadoc for GroupAlsoByWindowEvaluatorFactory
Manu Zhang created BEAM-2317: Summary: Incorrect Javadoc for GroupAlsoByWindowEvaluatorFactory Key: BEAM-2317 URL: https://issues.apache.org/jira/browse/BEAM-2317 Project: Beam Issue Type: Bug Components: runner-direct Affects Versions: 2.0.0 Reporter: Manu Zhang Assignee: Thomas Groh Priority: Minor It seems Javadoc for GroupAlsoByWindowEvaluatorFactory is forked from GroupByKeyOnlyEvaluatorFactory -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (BEAM-973) Add end user and developer documentation to gearpump-runner
[ https://issues.apache.org/jira/browse/BEAM-973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manu Zhang resolved BEAM-973. - Resolution: Fixed Fix Version/s: Not applicable > Add end user and developer documentation to gearpump-runner > --- > > Key: BEAM-973 > URL: https://issues.apache.org/jira/browse/BEAM-973 > Project: Beam > Issue Type: Sub-task > Components: runner-gearpump >Reporter: Manu Zhang >Assignee: Manu Zhang > Fix For: Not applicable > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (BEAM-972) Add basic level of unit testing to gearpump runner
[ https://issues.apache.org/jira/browse/BEAM-972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manu Zhang resolved BEAM-972. - Resolution: Fixed Fix Version/s: Not applicable > Add basic level of unit testing to gearpump runner > -- > > Key: BEAM-972 > URL: https://issues.apache.org/jira/browse/BEAM-972 > Project: Beam > Issue Type: Sub-task > Components: runner-gearpump, testing >Reporter: Manu Zhang >Assignee: Huafeng Wang > Fix For: Not applicable > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Assigned] (BEAM-972) Add basic level of unit testing to gearpump runner
[ https://issues.apache.org/jira/browse/BEAM-972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manu Zhang reassigned BEAM-972: --- Assignee: Huafeng Wang (was: Manu Zhang) > Add basic level of unit testing to gearpump runner > -- > > Key: BEAM-972 > URL: https://issues.apache.org/jira/browse/BEAM-972 > Project: Beam > Issue Type: Sub-task > Components: runner-gearpump, testing >Reporter: Manu Zhang >Assignee: Huafeng Wang > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Assigned] (BEAM-968) Update capability matrix to include gearpump-runner
[ https://issues.apache.org/jira/browse/BEAM-968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manu Zhang reassigned BEAM-968: --- Assignee: Huafeng Wang (was: Manu Zhang) > Update capability matrix to include gearpump-runner > --- > > Key: BEAM-968 > URL: https://issues.apache.org/jira/browse/BEAM-968 > Project: Beam > Issue Type: Sub-task > Components: runner-gearpump, website >Reporter: Manu Zhang >Assignee: Huafeng Wang > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (BEAM-971) Add support for merging windows in gearpump runner
[ https://issues.apache.org/jira/browse/BEAM-971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manu Zhang resolved BEAM-971. - Resolution: Fixed Fix Version/s: Not applicable > Add support for merging windows in gearpump runner > -- > > Key: BEAM-971 > URL: https://issues.apache.org/jira/browse/BEAM-971 > Project: Beam > Issue Type: Sub-task > Components: runner-gearpump >Reporter: Manu Zhang >Assignee: Manu Zhang > Fix For: Not applicable > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (BEAM-970) Add side-input/side-output support to gearpump-runner
[ https://issues.apache.org/jira/browse/BEAM-970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manu Zhang resolved BEAM-970. - Resolution: Fixed Fix Version/s: Not applicable > Add side-input/side-output support to gearpump-runner > - > > Key: BEAM-970 > URL: https://issues.apache.org/jira/browse/BEAM-970 > Project: Beam > Issue Type: Sub-task > Components: runner-gearpump >Reporter: Manu Zhang >Assignee: Manu Zhang > Fix For: Not applicable > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (BEAM-1086) Upgrade to latest Gearpump snapshot
[ https://issues.apache.org/jira/browse/BEAM-1086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manu Zhang resolved BEAM-1086. -- Resolution: Fixed Fix Version/s: Not applicable > Upgrade to latest Gearpump snapshot > --- > > Key: BEAM-1086 > URL: https://issues.apache.org/jira/browse/BEAM-1086 > Project: Beam > Issue Type: Improvement > Components: runner-gearpump >Reporter: Manu Zhang >Assignee: Manu Zhang > Fix For: Not applicable > > > The latest Gearpump snapshot version is available under [apache repo | > https://repository.apache.org/content/repositories/snapshots/org/apache/gearpump/gearpump-core_2.11/0.8.3-SNAPSHOT/]. > To support the Gearpump runner, we need to continuously evolve Gearpump and > bring in Beam capabilities. Depending on a snapshot version will greatly > accelerate the integration. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
