[jira] [Assigned] (BEAM-419) Non-transient non-serializable instance field in CombineFnUtil$NonSerializableBoundedKeyedCombineFn

[Neelesh Srinivas Salian (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Neelesh Srinivas Salian reassigned BEAM-419:


Assignee: Neelesh Srinivas Salian

> Non-transient non-serializable instance field in 
> CombineFnUtil$NonSerializableBoundedKeyedCombineFn
> ---
>
> Key: BEAM-419
> URL: https://issues.apache.org/jira/browse/BEAM-419
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-java-core
>Reporter: Scott Wegner
>Assignee: Neelesh Srinivas Salian
>Priority: Minor
>  Labels: findbugs, newbie, starter
>
> [FindBugs 
> SE_BAD_FIELD|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L363]:
>  Non-transient non-serializable instance field in serializable class
> Applies to: 
> [CombineFnUtil$NonSerializableBoundedKeyedCombineFn.context|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/util/CombineFnUtil.java#L170].
> This is a good starter bug. When fixing, please remove the corresponding 
> entries from 
> [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml]
>  and verify the build passes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (BEAM-421) StateTags$CombiningValueStateTag inherits equals from super class but without incorporating its state

[Neelesh Srinivas Salian (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Neelesh Srinivas Salian reassigned BEAM-421:


Assignee: Neelesh Srinivas Salian

> StateTags$CombiningValueStateTag inherits equals from super class but without 
> incorporating its state
> -
>
> Key: BEAM-421
> URL: https://issues.apache.org/jira/browse/BEAM-421
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-java-core
>Reporter: Scott Wegner
>Assignee: Neelesh Srinivas Salian
>Priority: Minor
>  Labels: findbugs, newbie, starter
>
> [FindBugs 
> EQ_DOESNT_OVERRIDE_EQUALS|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L555]:
>  Class doesn't override equals in superclass
> Applies to: 
> [StateTags$CombiningValueStateTag|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/util/state/StateTags.java#L344].
>  CombiningValueStateTag inherits its equals implementation from its parent 
> class, KeyedCombiningValueStateTag. CombiningValueStateTag has additional 
> fields which are not included in the equality implementation, which may be 
> incorrect.
> This is a good starter bug. When fixing, please remove the corresponding 
> entries from 
> [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml]
>  and verify the build passes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BEAM-410) ApproximateQuantiles$QuantileBuffer defines compareTo but not equals

[Neelesh Srinivas Salian (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15604098#comment-15604098
 ] 

Neelesh Srinivas Salian commented on BEAM-410:
--

[~swegner], can we close this in favor of BEAM-409?
Or I can take it up if [~jason.white] is alright with it?

> ApproximateQuantiles$QuantileBuffer defines compareTo but not equals
> 
>
> Key: BEAM-410
> URL: https://issues.apache.org/jira/browse/BEAM-410
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-java-core
>Reporter: Scott Wegner
>Assignee: Neelesh Srinivas Salian
>Priority: Minor
>  Labels: findbugs, newbie, starter
>
> [FindBugs 
> EQ_COMPARETO_USE_OBJECT_EQUALS|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml#L280]:
>  Class defines compareTo(...) and uses Object.equals()
> Applies to: 
> [ApproximateQuanitles$QuantileBuffer|https://github.com/apache/incubator-beam/blob/58a029a06aea1030279e5da8f9fa3114f456c1db/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/ApproximateQuantiles.java#L627].
> This is a good starter bug. When fixing, please remove the corresponding 
> entries from 
> [findbugs-filter.xml|https://github.com/apache/incubator-beam/blob/master/sdks/java/build-tools/src/main/resources/beam/findbugs-filter.xml]
>  and verify the build passes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BEAM-814) Improve performance when staging files

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15604035#comment-15604035
 ] 

ASF GitHub Bot commented on BEAM-814:
-

GitHub user dhalperi opened a pull request:

https://github.com/apache/incubator-beam/pull/1184

[BEAM-814] Dataflow: parallelize hashing/compressing of files to stage

R: @lukecwik 

This is a WIP for the linked issue. Early feedback appreciated.

Decided to parallelize just hashing/compressing separately from uploading, 
because this should use a very tiny buffer per file and so can be parallelized 
nearly arbitrarily. In contrast, uploading uses a very large buffer and cannot 
be parallelized simply.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/dhalperi/incubator-beam 
dataflow-runner-speedup

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-beam/pull/1184.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1184


commit 04c7e1cc8596a7be8496e8f7e1e186e727366409
Author: Dan Halperin 
Date:   2016-10-25T00:27:23Z

PackageUtil: remove deprecated, unused function

commit b625e840b78a8dd5686be66bf949b8e42dba42db
Author: Dan Halperin 
Date:   2016-10-25T02:20:53Z

PackageUtil: parallelize hashing of files




> Improve performance when staging files
> --
>
> Key: BEAM-814
> URL: https://issues.apache.org/jira/browse/BEAM-814
> Project: Beam
>  Issue Type: Improvement
>  Components: runner-dataflow
>Reporter: Daniel Halperin
>Assignee: Daniel Halperin
>Priority: Minor
>
> Right now, {{PackageUtil}} hashes/compresses/sizes files, checks if they have 
> already been staged, and uploads them if not, completely serially.
> There is some low-hanging fruit to parallelize this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-beam pull request #1184: [BEAM-814] Dataflow: parallelize hashing/...

[dhalperi]

GitHub user dhalperi opened a pull request:

https://github.com/apache/incubator-beam/pull/1184

[BEAM-814] Dataflow: parallelize hashing/compressing of files to stage

R: @lukecwik 

This is a WIP for the linked issue. Early feedback appreciated.

Decided to parallelize just hashing/compressing separately from uploading, 
because this should use a very tiny buffer per file and so can be parallelized 
nearly arbitrarily. In contrast, uploading uses a very large buffer and cannot 
be parallelized simply.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/dhalperi/incubator-beam 
dataflow-runner-speedup

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-beam/pull/1184.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1184


commit 04c7e1cc8596a7be8496e8f7e1e186e727366409
Author: Dan Halperin 
Date:   2016-10-25T00:27:23Z

PackageUtil: remove deprecated, unused function

commit b625e840b78a8dd5686be66bf949b8e42dba42db
Author: Dan Halperin 
Date:   2016-10-25T02:20:53Z

PackageUtil: parallelize hashing of files




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (BEAM-814) Improve performance when staging files

[Daniel Halperin (JIRA)]

Daniel Halperin created BEAM-814:


 Summary: Improve performance when staging files
 Key: BEAM-814
 URL: https://issues.apache.org/jira/browse/BEAM-814
 Project: Beam
  Issue Type: Improvement
  Components: runner-dataflow
Reporter: Daniel Halperin
Assignee: Daniel Halperin
Priority: Minor


Right now, {{PackageUtil}} hashes/compresses/sizes files, checks if they have 
already been staged, and uploads them if not, completely serially.

There is some low-hanging fruit to parallelize this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BEAM-810) Beam's uses of archetypes are problematic

[Kenneth Knowles (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15604022#comment-15604022
 ] 

Kenneth Knowles commented on BEAM-810:
--

Can you link to an example failure of the archetypes? Since I have this paged 
in, maybe something will come to me.

> Beam's uses of archetypes are problematic
> -
>
> Key: BEAM-810
> URL: https://issues.apache.org/jira/browse/BEAM-810
> Project: Beam
>  Issue Type: Bug
>  Components: build-system
>Affects Versions: Not applicable
>Reporter: Daniel Halperin
>
> * Using a version range in the archetype pom.xml is inaccurate, because we 
> don't have a stable API. We can't actually expect an older version to work as 
> we make backwards-incompatible changes.
> * In PR #936, a decision was made to pin the version of the archetype. This 
> requires manual changes to update many places (archetype-resources), which 
> breaks the release process in a few different ways. See [BEAM-806].
> * Travis uses `mvn verify`, not `mvn install` -- so archetypes are not 
> properly tested and instead we're actually picking up the SNAPSHOT archetypes 
> deployed the prior night (IIRC).
> * There was some other issue with parallelizing maven where archetypes are 
> not properly marked as depending on "you can't test the archetype until 
> you've installed the examples packages".
> * We're not actually testing that the archetype-generated projects actually 
> compile. [BEAM-641].
> This is all pretty ugly. Suggestions wanted.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (BEAM-810) Beam's uses of archetypes are problematic

[Kenneth Knowles (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15604018#comment-15604018
 ] 

Kenneth Knowles edited comment on BEAM-810 at 10/25/16 3:14 AM:


PR [#1181|https://github.com/apache/incubator-beam/pull/1181] should fix one 
issue created by the use of {{mvn verify}}. Perhaps a similar approach will 
work for the archetypes. It is unfortunate to do so much useless work.


was (Author: kenn):
PR [#1181|https://github.com/apache/incubator-beam/pull/1181] should fix one 
issue created by the use of `mvn verify`. Perhaps a similar approach will work 
for the archetypes. It is unfortunate to do so much useless work.

> Beam's uses of archetypes are problematic
> -
>
> Key: BEAM-810
> URL: https://issues.apache.org/jira/browse/BEAM-810
> Project: Beam
>  Issue Type: Bug
>  Components: build-system
>Affects Versions: Not applicable
>Reporter: Daniel Halperin
>
> * Using a version range in the archetype pom.xml is inaccurate, because we 
> don't have a stable API. We can't actually expect an older version to work as 
> we make backwards-incompatible changes.
> * In PR #936, a decision was made to pin the version of the archetype. This 
> requires manual changes to update many places (archetype-resources), which 
> breaks the release process in a few different ways. See [BEAM-806].
> * Travis uses `mvn verify`, not `mvn install` -- so archetypes are not 
> properly tested and instead we're actually picking up the SNAPSHOT archetypes 
> deployed the prior night (IIRC).
> * There was some other issue with parallelizing maven where archetypes are 
> not properly marked as depending on "you can't test the archetype until 
> you've installed the examples packages".
> * We're not actually testing that the archetype-generated projects actually 
> compile. [BEAM-641].
> This is all pretty ugly. Suggestions wanted.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BEAM-810) Beam's uses of archetypes are problematic

[Kenneth Knowles (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15604018#comment-15604018
 ] 

Kenneth Knowles commented on BEAM-810:
--

PR [#1181|https://github.com/apache/incubator-beam/pull/1181] should fix one 
issue created by the use of `mvn verify`. Perhaps a similar approach will work 
for the archetypes. It is unfortunate to do so much useless work.

> Beam's uses of archetypes are problematic
> -
>
> Key: BEAM-810
> URL: https://issues.apache.org/jira/browse/BEAM-810
> Project: Beam
>  Issue Type: Bug
>  Components: build-system
>Affects Versions: Not applicable
>Reporter: Daniel Halperin
>
> * Using a version range in the archetype pom.xml is inaccurate, because we 
> don't have a stable API. We can't actually expect an older version to work as 
> we make backwards-incompatible changes.
> * In PR #936, a decision was made to pin the version of the archetype. This 
> requires manual changes to update many places (archetype-resources), which 
> breaks the release process in a few different ways. See [BEAM-806].
> * Travis uses `mvn verify`, not `mvn install` -- so archetypes are not 
> properly tested and instead we're actually picking up the SNAPSHOT archetypes 
> deployed the prior night (IIRC).
> * There was some other issue with parallelizing maven where archetypes are 
> not properly marked as depending on "you can't test the archetype until 
> you've installed the examples packages".
> * We're not actually testing that the archetype-generated projects actually 
> compile. [BEAM-641].
> This is all pretty ugly. Suggestions wanted.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BEAM-812) Shade guava in beam-sdks-java-io-google-cloud-platform

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15604015#comment-15604015
 ] 

ASF GitHub Bot commented on BEAM-812:
-

GitHub user dhalperi opened a pull request:

https://github.com/apache/incubator-beam/pull/1183

[BEAM-812] gcp-io: add ApiSurfaceTest

Copied from SDK, but I manually recreated/verified the whitelist.

Surely there are refactoring improvements to be had so that we can more 
easily
reuse ApiSurface across modules.

R: @kennknowles 
R: @davorbonaci 

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/dhalperi/incubator-beam dataflow-gcp-shade

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-beam/pull/1183.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1183


commit 0cf01507324216771c21587c205fbce1a94e14a0
Author: Dan Halperin 
Date:   2016-10-25T03:08:20Z

gcp-io: add ApiSurfaceTest

Copied from SDK, but I manually recreated the whitelist.

Surely there are refactoring improvements to be had so that we can more 
easily
reuse ApiSurface across modules.




> Shade guava in beam-sdks-java-io-google-cloud-platform
> --
>
> Key: BEAM-812
> URL: https://issues.apache.org/jira/browse/BEAM-812
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-java-gcp
>Reporter: Daniel Halperin
>Assignee: Daniel Halperin
> Fix For: 0.4.0-incubating
>
>
> Looking at 0.3.0-incubating RC1, we are not properly shading Guava.
> https://repository.apache.org/content/repositories/staging/org/apache/beam/beam-sdks-java-io-google-cloud-platform/0.3.0-incubating/beam-sdks-java-io-google-cloud-platform-0.3.0-incubating.pom
> has 
> {code}
> 
>   com.google.guava
>   guava
>  {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-beam pull request #1183: [BEAM-812] gcp-io: add ApiSurfaceTest

[dhalperi]

GitHub user dhalperi opened a pull request:

https://github.com/apache/incubator-beam/pull/1183

[BEAM-812] gcp-io: add ApiSurfaceTest

Copied from SDK, but I manually recreated/verified the whitelist.

Surely there are refactoring improvements to be had so that we can more 
easily
reuse ApiSurface across modules.

R: @kennknowles 
R: @davorbonaci 

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/dhalperi/incubator-beam dataflow-gcp-shade

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-beam/pull/1183.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1183


commit 0cf01507324216771c21587c205fbce1a94e14a0
Author: Dan Halperin 
Date:   2016-10-25T03:08:20Z

gcp-io: add ApiSurfaceTest

Copied from SDK, but I manually recreated the whitelist.

Surely there are refactoring improvements to be had so that we can more 
easily
reuse ApiSurface across modules.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-beam pull request #1182: Display data: minor cleanups

[dhalperi]

GitHub user dhalperi opened a pull request:

https://github.com/apache/incubator-beam/pull/1182

Display data: minor cleanups

* wrong @Nullable
* Remove unused function

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/dhalperi/incubator-beam display-data-fixup

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-beam/pull/1182.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1182


commit 19d9daffef6fa38fe25873b20526db6bcf90d84a
Author: Dan Halperin 
Date:   2016-10-25T02:39:27Z

Display data: minor cleanups

* wrong @Nullable
* Remove unused function




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-beam pull request #1181: Fix .travis/test_wordcount.sh to not reac...

[kennknowles]

GitHub user kennknowles opened a pull request:

https://github.com/apache/incubator-beam/pull/1181

Fix .travis/test_wordcount.sh to not reach out to snapshot repository

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

 - [x] Make sure the PR title is formatted like:
   `[BEAM-] Description of pull request`
 - [x] Make sure tests pass via `mvn clean verify`. (Even better, enable
   Travis-CI on your fork and ensure the whole test matrix passes).
 - [x] Replace `` in the title with the actual Jira issue
   number, if there is one.
 - [x] If this contribution is large, please file an Apache
   [Individual Contributor License 
Agreement](https://www.apache.org/licenses/icla.txt).

---


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/kennknowles/incubator-beam travis-install

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-beam/pull/1181.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1181


commit 91fff7399fc2cbf702421717911f624e6fcc8755
Author: Kenneth Knowles 
Date:   2016-10-25T03:08:23Z

Show bash trace of .travis/test_wordcount.sh

commit ab1c995b104435bcc9814d16d80ffe4d0bb5b7e1
Author: Kenneth Knowles 
Date:   2016-10-25T03:09:10Z

Add `package` to the mvn command in .travis/test_wordcount.sh

This should make the snapshot artifacts available in the
maven reactor, according to http://stackoverflow.com/a/18514450/4820657,
fixing issues with our build depending on the snapshot repository.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-beam pull request #1152: Touch ~/.m2/settings.xml for Travis CI on...

[kennknowles]

Github user kennknowles closed the pull request at:

https://github.com/apache/incubator-beam/pull/1152


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-beam pull request #1165: Stop running WordCount on Travis CI

[kennknowles]

Github user kennknowles closed the pull request at:

https://github.com/apache/incubator-beam/pull/1165


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-beam pull request #1180: Use our Travis settings everywhere

[kennknowles]

GitHub user kennknowles opened a pull request:

https://github.com/apache/incubator-beam/pull/1180

Use our Travis settings everywhere

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

 - [x] Make sure the PR title is formatted like:
   `[BEAM-] Description of pull request`
 - [x] Make sure tests pass via `mvn clean verify`. (Even better, enable
   Travis-CI on your fork and ensure the whole test matrix passes).
 - [x] Replace `` in the title with the actual Jira issue
   number, if there is one.
 - [x] If this contribution is large, please file an Apache
   [Individual Contributor License 
Agreement](https://www.apache.org/licenses/icla.txt).

---

I believe this is a better fix for #1137

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/kennknowles/incubator-beam travis-settings

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-beam/pull/1180.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1180


commit 0df8bbcb28fefd1e4cc658b4db7696346ed283cd
Author: Kenneth Knowles 
Date:   2016-10-25T02:31:26Z

Revert "Fix invalid snapshot repo in Travis CI's maven settings.xml"

This reverts commit fbc23ed9234088c9d88bf88b19a3a4d6d5e52446.

commit 7ed8355522f34b97f5ec7ea9785df996fc0b8e1d
Author: Kenneth Knowles 
Date:   2016-10-25T02:34:04Z

Set up Maven settings globally for Travis CI builds




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-beam pull request #1179: Remove timestamp that stymies smart rebui...

[kennknowles]

GitHub user kennknowles opened a pull request:

https://github.com/apache/incubator-beam/pull/1179

Remove timestamp that stymies smart rebuilds

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

 - [x] Make sure the PR title is formatted like:
   `[BEAM-] Description of pull request`
 - [x] Make sure tests pass via `mvn clean verify`. (Even better, enable
   Travis-CI on your fork and ensure the whole test matrix passes).
 - [x] Replace `` in the title with the actual Jira issue
   number, if there is one.
 - [x] If this contribution is large, please file an Apache
   [Individual Contributor License 
Agreement](https://www.apache.org/licenses/icla.txt).

---

This moves our timestamp from a `properties` file to the manifest, where it 
belongs. Thus it is no longer a change in the `src` tree so rebuilds can be 
skipped properly. There are some other commits to also speed up the build.

I believe the `-Xpkginfo:always` commit is known to be broken on openJDK 7 
but I have left it here to confirm my recollection.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/kennknowles/incubator-beam sdk-timestamp

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-beam/pull/1179.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1179


commit 16790810c668ecc96b8530347539c7de3367c4c5
Author: Kenneth Knowles 
Date:   2016-05-04T04:27:44Z

Remove timestamp in source that caused spurious builds

commit c1ca59148d03db5340e8335de3bb2ba112f4168f
Author: Kenneth Knowles 
Date:   2016-05-04T04:30:51Z

Exclude package-info.java from compiled sources list

This is a workaround for [MCOMPILER-205], which causes
maven-compiler-plugin to always detect stale sources due to sources
that do not create a class file, such as package-info.java.

An alternative workaround is to add dummy annotations to all
package-info.java files, which will induce package-info.class
files.

[MCOMPILER-205](https://issues.apache.org/jira/browse/MCOMPILER-205)

commit 9a133145ecff7c5c4b3a1831bbad93143e701515
Author: Kenneth Knowles 
Date:   2016-05-04T21:07:22Z

Add ReleaseInfo to bigtable package and use for User-agent

commit 812ebcf14e622a22d2b7766705a1f935cfd752a8
Author: Kenneth Knowles 
Date:   2016-10-25T02:19:16Z

Use -Xpkginfo:always flag




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Jenkins build is still unstable: beam_PostCommit_RunnableOnService_FlinkLocal #660

[Apache Jenkins Server]

See 

[GitHub] incubator-beam pull request #1148: [BEAM-753] Pin the version of dependencie...

[aaltay]

Github user aaltay closed the pull request at:

https://github.com/apache/incubator-beam/pull/1148


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (BEAM-753) Pin versions of all dependencies

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15603806#comment-15603806
 ] 

ASF GitHub Bot commented on BEAM-753:
-

Github user aaltay closed the pull request at:

https://github.com/apache/incubator-beam/pull/1148


> Pin versions of all dependencies
> 
>
> Key: BEAM-753
> URL: https://issues.apache.org/jira/browse/BEAM-753
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py
>Reporter: Ahmet Altay
>Assignee: Ahmet Altay
> Fix For: Not applicable
>
>
> ERROR: Failure: ImportError (cannot import name locked_file)
> --
> Traceback (most recent call last):
>   File 
> "/usr/local/google/home/altay/Desktop/beam/test/incubator-beam/sdks/python/nose-1.3.7-py2.7.egg/nose/loader.py",
>  line 418, in loadTestsFromName
> addr.filename, addr.module)
>   File 
> "/usr/local/google/home/altay/Desktop/beam/test/incubator-beam/sdks/python/nose-1.3.7-py2.7.egg/nose/importer.py",
>  line 47, in importFromPath
> return self.importFromDir(dir_path, fqname)
>   File 
> "/usr/local/google/home/altay/Desktop/beam/test/incubator-beam/sdks/python/nose-1.3.7-py2.7.egg/nose/importer.py",
>  line 94, in importFromDir
> mod = load_module(part_fqname, fh, filename, desc)
>   File 
> "/usr/local/google/home/altay/Desktop/beam/test/incubator-beam/sdks/python/apache_beam/__init__.py",
>  line 78, in 
> from apache_beam import io
>   File 
> "/usr/local/google/home/altay/Desktop/beam/test/incubator-beam/sdks/python/apache_beam/io/__init__.py",
>  line 21, in 
> from apache_beam.io.avroio import *
>   File 
> "/usr/local/google/home/altay/Desktop/beam/test/incubator-beam/sdks/python/apache_beam/io/avroio.py",
>  line 29, in 
> from apache_beam.io import filebasedsource
>   File 
> "/usr/local/google/home/altay/Desktop/beam/test/incubator-beam/sdks/python/apache_beam/io/filebasedsource.py",
>  line 31, in 
> from apache_beam.io import concat_source
>   File 
> "/usr/local/google/home/altay/Desktop/beam/test/incubator-beam/sdks/python/apache_beam/io/concat_source.py",
>  line 24, in 
> from apache_beam.io import iobase
>   File 
> "/usr/local/google/home/altay/Desktop/beam/test/incubator-beam/sdks/python/apache_beam/io/iobase.py",
>  line 818, in 
> from apache_beam.runners.dataflow.native_io.iobase import *
>   File 
> "/usr/local/google/home/altay/Desktop/beam/test/incubator-beam/sdks/python/apache_beam/runners/__init__.py",
>  line 23, in 
> from apache_beam.runners.dataflow_runner import DataflowPipelineRunner
>   File 
> "/usr/local/google/home/altay/Desktop/beam/test/incubator-beam/sdks/python/apache_beam/runners/dataflow_runner.py",
>  line 43, in 
> from apache_beam.internal.clients import dataflow as dataflow_api
>   File 
> "/usr/local/google/home/altay/Desktop/beam/test/incubator-beam/sdks/python/apache_beam/internal/clients/dataflow/__init__.py",
>  line 23, in 
> from apitools.base.py import *
>   File 
> "/usr/local/google/home/altay/Desktop/beam/test/incubator-beam/sdks/python/.tox/py27/local/lib/python2.7/site-packages/apitools/base/py/__init__.py",
>  line 22, in 
> from apitools.base.py.credentials_lib import *
>   File 
> "/usr/local/google/home/altay/Desktop/beam/test/incubator-beam/sdks/python/.tox/py27/local/lib/python2.7/site-packages/apitools/base/py/credentials_lib.py",
>  line 50, in 
> from oauth2client import locked_file
> ImportError: cannot import name locked_file



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[2/2] incubator-beam git commit: Closes #1175

[dhalperi]

Closes #1175


Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/7de8d571
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/7de8d571
Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/7de8d571

Branch: refs/heads/master
Commit: 7de8d571d6a86de0dfdbb1756e101c72319f5db4
Parents: d285e85 6291a6f
Author: Dan Halperin 
Authored: Mon Oct 24 18:03:29 2016 -0700
Committer: Dan Halperin 
Committed: Mon Oct 24 18:03:29 2016 -0700

--
 pom.xml | 3 ---
 runners/pom.xml | 1 +
 2 files changed, 1 insertion(+), 3 deletions(-)
--

[jira] [Commented] (BEAM-810) Beam's uses of archetypes are problematic

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15603775#comment-15603775
 ] 

ASF GitHub Bot commented on BEAM-810:
-

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-beam/pull/1175


> Beam's uses of archetypes are problematic
> -
>
> Key: BEAM-810
> URL: https://issues.apache.org/jira/browse/BEAM-810
> Project: Beam
>  Issue Type: Bug
>  Components: build-system
>Affects Versions: Not applicable
>Reporter: Daniel Halperin
>
> * Using a version range in the archetype pom.xml is inaccurate, because we 
> don't have a stable API. We can't actually expect an older version to work as 
> we make backwards-incompatible changes.
> * In PR #936, a decision was made to pin the version of the archetype. This 
> requires manual changes to update many places (archetype-resources), which 
> breaks the release process in a few different ways. See [BEAM-806].
> * Travis uses `mvn verify`, not `mvn install` -- so archetypes are not 
> properly tested and instead we're actually picking up the SNAPSHOT archetypes 
> deployed the prior night (IIRC).
> * There was some other issue with parallelizing maven where archetypes are 
> not properly marked as depending on "you can't test the archetype until 
> you've installed the examples packages".
> * We're not actually testing that the archetype-generated projects actually 
> compile. [BEAM-641].
> This is all pretty ugly. Suggestions wanted.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-beam pull request #1175: [BEAM-810] pom.xml: make Dataflow runner ...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-beam/pull/1175


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[1/2] incubator-beam git commit: pom.xml: make Dataflow runner like every other runner

[dhalperi]

Repository: incubator-beam
Updated Branches:
  refs/heads/master d285e85da -> 7de8d571d


pom.xml: make Dataflow runner like every other runner

Pom dependencies no longer require it to be treated specially.


Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/6291a6f6
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/6291a6f6
Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/6291a6f6

Branch: refs/heads/master
Commit: 6291a6f674a98b8898d807e72ebf2ad5cc5d3ce7
Parents: d285e85
Author: Dan Halperin 
Authored: Mon Oct 24 13:16:01 2016 -0700
Committer: Dan Halperin 
Committed: Mon Oct 24 18:03:28 2016 -0700

--
 pom.xml | 3 ---
 runners/pom.xml | 1 +
 2 files changed, 1 insertion(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/6291a6f6/pom.xml
--
diff --git a/pom.xml b/pom.xml
index 2338f55..df388d4 100644
--- a/pom.xml
+++ b/pom.xml
@@ -139,9 +139,6 @@
 
   
 sdks
-
-runners/google-cloud-dataflow-java
 runners
 

Jenkins build is back to stable : beam_PostCommit_MavenVerify » Apache Beam :: Examples :: Java #1630

[Apache Jenkins Server]

See 

Jenkins build is back to stable : beam_PostCommit_MavenVerify #1630

[Apache Jenkins Server]

See 

Jenkins build is back to stable : beam_PostCommit_MavenVerify » Apache Beam :: Runners :: Direct Java #1630

[Apache Jenkins Server]

See 

Jenkins build is still unstable: beam_PostCommit_RunnableOnService_FlinkLocal #659

[Apache Jenkins Server]

See 

[jira] [Commented] (BEAM-498) Make DoFnWithContext the new DoFn

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15603514#comment-15603514
 ] 

ASF GitHub Bot commented on BEAM-498:
-

GitHub user kennknowles opened a pull request:

https://github.com/apache/incubator-beam/pull/1178

[BEAM-498] Remove some more uses of OldDoFn

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

 - [x] Make sure the PR title is formatted like:
   `[BEAM-] Description of pull request`
 - [x] Make sure tests pass via `mvn clean verify`. (Even better, enable
   Travis-CI on your fork and ensure the whole test matrix passes).
 - [x] Replace `` in the title with the actual Jira issue
   number, if there is one.
 - [x] If this contribution is large, please file an Apache
   [Individual Contributor License 
Agreement](https://www.apache.org/licenses/icla.txt).

---

R: @tgroh 

In the DirectRunner tests, I ported the syntax, though not the logic since 
it doesn't actually run yet. I'm not totally certain these tests are needed 
beyond future ROS tests.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/kennknowles/incubator-beam ValueWithRecordId

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-beam/pull/1178.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1178


commit 3a2f5d1c720590eabee7667b686533f762698297
Author: Kenneth Knowles 
Date:   2016-10-24T22:57:18Z

Remove use of OldDoFn in ValueWithRecordId

commit 4219bb5493c3b9f70ccdf78d8820649453cf0a6d
Author: Kenneth Knowles 
Date:   2016-10-24T23:09:13Z

Remove use of OldDoFn from some DirectRunner tests




> Make DoFnWithContext the new DoFn
> -
>
> Key: BEAM-498
> URL: https://issues.apache.org/jira/browse/BEAM-498
> Project: Beam
>  Issue Type: New Feature
>  Components: sdk-java-core
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>  Labels: backward-incompatible
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-beam pull request #1178: [BEAM-498] Remove some more uses of OldDo...

[kennknowles]

GitHub user kennknowles opened a pull request:

https://github.com/apache/incubator-beam/pull/1178

[BEAM-498] Remove some more uses of OldDoFn

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

 - [x] Make sure the PR title is formatted like:
   `[BEAM-] Description of pull request`
 - [x] Make sure tests pass via `mvn clean verify`. (Even better, enable
   Travis-CI on your fork and ensure the whole test matrix passes).
 - [x] Replace `` in the title with the actual Jira issue
   number, if there is one.
 - [x] If this contribution is large, please file an Apache
   [Individual Contributor License 
Agreement](https://www.apache.org/licenses/icla.txt).

---

R: @tgroh 

In the DirectRunner tests, I ported the syntax, though not the logic since 
it doesn't actually run yet. I'm not totally certain these tests are needed 
beyond future ROS tests.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/kennknowles/incubator-beam ValueWithRecordId

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-beam/pull/1178.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1178


commit 3a2f5d1c720590eabee7667b686533f762698297
Author: Kenneth Knowles 
Date:   2016-10-24T22:57:18Z

Remove use of OldDoFn in ValueWithRecordId

commit 4219bb5493c3b9f70ccdf78d8820649453cf0a6d
Author: Kenneth Knowles 
Date:   2016-10-24T23:09:13Z

Remove use of OldDoFn from some DirectRunner tests




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (BEAM-813) Support metadata in Avro sink

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15603504#comment-15603504
 ] 

ASF GitHub Bot commented on BEAM-813:
-

GitHub user nevillelyh opened a pull request:

https://github.com/apache/incubator-beam/pull/1177

[BEAM-813] support metadata in Avro sink



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/nevillelyh/incubator-beam 
neville/avro-metadata

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-beam/pull/1177.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1177


commit f1590f91bd6291f92f4532cf40b73aa3a0d923bd
Author: Neville Li 
Date:   2016-10-24T22:56:36Z

[BEAM-813] support metadata in Avro sink




> Support metadata in Avro sink
> -
>
> Key: BEAM-813
> URL: https://issues.apache.org/jira/browse/BEAM-813
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-java-core
>Reporter: Neville Li
>Assignee: Neville Li
>Priority: Minor
>
> It'd be nice to support custom metadata in Avro files. This change is similar 
> to [BEAM-701].



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-beam pull request #1177: [BEAM-813] support metadata in Avro sink

[nevillelyh]

GitHub user nevillelyh opened a pull request:

https://github.com/apache/incubator-beam/pull/1177

[BEAM-813] support metadata in Avro sink



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/nevillelyh/incubator-beam 
neville/avro-metadata

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-beam/pull/1177.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1177


commit f1590f91bd6291f92f4532cf40b73aa3a0d923bd
Author: Neville Li 
Date:   2016-10-24T22:56:36Z

[BEAM-813] support metadata in Avro sink




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (BEAM-813) Support metadata in Avro sink

[Neville Li (JIRA)]

Neville Li created BEAM-813:
---

 Summary: Support metadata in Avro sink
 Key: BEAM-813
 URL: https://issues.apache.org/jira/browse/BEAM-813
 Project: Beam
  Issue Type: Improvement
  Components: sdk-java-core
Reporter: Neville Li
Assignee: Neville Li
Priority: Minor


It'd be nice to support custom metadata in Avro files. This change is similar 
to [BEAM-701]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (BEAM-813) Support metadata in Avro sink

[Neville Li (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Neville Li updated BEAM-813:

Description: It'd be nice to support custom metadata in Avro files. This 
change is similar to [BEAM-701].  (was: It'd be nice to support custom metadata 
in Avro files. This change is similar to [BEAM-701])

> Support metadata in Avro sink
> -
>
> Key: BEAM-813
> URL: https://issues.apache.org/jira/browse/BEAM-813
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-java-core
>Reporter: Neville Li
>Assignee: Neville Li
>Priority: Minor
>
> It'd be nice to support custom metadata in Avro files. This change is similar 
> to [BEAM-701].



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Jenkins build became unstable: beam_PostCommit_MavenVerify #1629

[Apache Jenkins Server]

See 

Jenkins build became unstable: beam_PostCommit_MavenVerify » Apache Beam :: Examples :: Java #1629

[Apache Jenkins Server]

See 

Jenkins build became unstable: beam_PostCommit_MavenVerify » Apache Beam :: Runners :: Direct Java #1629

[Apache Jenkins Server]

See 

Jenkins build is still unstable: beam_PostCommit_RunnableOnService_FlinkLocal #658

[Apache Jenkins Server]

See 

[jira] [Commented] (BEAM-498) Make DoFnWithContext the new DoFn

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15603407#comment-15603407
 ] 

ASF GitHub Bot commented on BEAM-498:
-

GitHub user kennknowles opened a pull request:

https://github.com/apache/incubator-beam/pull/1176

[BEAM-498] Make DoFnInfo carry OldDoFn or DoFn

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

 - [x] Make sure the PR title is formatted like:
   `[BEAM-] Description of pull request`
 - [x] Make sure tests pass via `mvn clean verify`. (Even better, enable
   Travis-CI on your fork and ensure the whole test matrix passes).
 - [x] Replace `` in the title with the actual Jira issue
   number, if there is one.
 - [x] If this contribution is large, please file an Apache
   [Individual Contributor License 
Agreement](https://www.apache.org/licenses/icla.txt).

---

This will allow consumers to prepare to accept `DoFn` while still accepting 
existing jobs that use `OldDoFn`.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/kennknowles/incubator-beam DoFnInfo

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-beam/pull/1176.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1176


commit 37e1dcb5baeabd00b177f6919107efabbeafa8a3
Author: Kenneth Knowles 
Date:   2016-10-24T22:11:12Z

Make DoFnInfo carry OldDoFn or DoFn

This will allow consumers to prepare to accept
DoFn while still accepting existing jobs that
use OldDoFn.




> Make DoFnWithContext the new DoFn
> -
>
> Key: BEAM-498
> URL: https://issues.apache.org/jira/browse/BEAM-498
> Project: Beam
>  Issue Type: New Feature
>  Components: sdk-java-core
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>  Labels: backward-incompatible
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-beam pull request #1176: [BEAM-498] Make DoFnInfo carry OldDoFn or...

[kennknowles]

GitHub user kennknowles opened a pull request:

https://github.com/apache/incubator-beam/pull/1176

[BEAM-498] Make DoFnInfo carry OldDoFn or DoFn

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

 - [x] Make sure the PR title is formatted like:
   `[BEAM-] Description of pull request`
 - [x] Make sure tests pass via `mvn clean verify`. (Even better, enable
   Travis-CI on your fork and ensure the whole test matrix passes).
 - [x] Replace `` in the title with the actual Jira issue
   number, if there is one.
 - [x] If this contribution is large, please file an Apache
   [Individual Contributor License 
Agreement](https://www.apache.org/licenses/icla.txt).

---

This will allow consumers to prepare to accept `DoFn` while still accepting 
existing jobs that use `OldDoFn`.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/kennknowles/incubator-beam DoFnInfo

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-beam/pull/1176.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1176


commit 37e1dcb5baeabd00b177f6919107efabbeafa8a3
Author: Kenneth Knowles 
Date:   2016-10-24T22:11:12Z

Make DoFnInfo carry OldDoFn or DoFn

This will allow consumers to prepare to accept
DoFn while still accepting existing jobs that
use OldDoFn.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (BEAM-812) Shade guava in beam-sdks-java-io-google-cloud-platform

[Daniel Halperin (JIRA)]

Daniel Halperin created BEAM-812:


 Summary: Shade guava in beam-sdks-java-io-google-cloud-platform
 Key: BEAM-812
 URL: https://issues.apache.org/jira/browse/BEAM-812
 Project: Beam
  Issue Type: Improvement
  Components: sdk-java-gcp
Reporter: Daniel Halperin
Assignee: Daniel Halperin
 Fix For: 0.4.0-incubating


Looking at 0.3.0-incubating RC1, we are not properly shading Guava.

https://repository.apache.org/content/repositories/staging/org/apache/beam/beam-sdks-java-io-google-cloud-platform/0.3.0-incubating/beam-sdks-java-io-google-cloud-platform-0.3.0-incubating.pom

has 

{code}

  com.google.guava
  guava

[GitHub] incubator-beam pull request #1135: Add timestamps to maven output on Travis ...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-beam/pull/1135


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[2/2] incubator-beam git commit: Closes #1135

[dhalperi]

Closes #1135


Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/d285e85d
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/d285e85d
Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/d285e85d

Branch: refs/heads/master
Commit: d285e85da7d24e7cfc3ede951f00248a11f21b76
Parents: 44b9f4e 7f82a57
Author: Dan Halperin 
Authored: Mon Oct 24 15:04:15 2016 -0700
Committer: Dan Halperin 
Committed: Mon Oct 24 15:04:15 2016 -0700

--
 .travis.yml | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
--

[1/2] incubator-beam git commit: Add timestamps to maven output on Travis CI

[dhalperi]

Repository: incubator-beam
Updated Branches:
  refs/heads/master 44b9f4ebb -> d285e85da


Add timestamps to maven output on Travis CI


Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/7f82a573
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/7f82a573
Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/7f82a573

Branch: refs/heads/master
Commit: 7f82a573d00a5a30331b7bbb8757e55f4a2d93ae
Parents: 44b9f4e
Author: Kenneth Knowles 
Authored: Wed Oct 19 14:34:17 2016 -0700
Committer: Dan Halperin 
Committed: Mon Oct 24 15:04:14 2016 -0700

--
 .travis.yml | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/7f82a573/.travis.yml
--
diff --git a/.travis.yml b/.travis.yml
index a9ba6d1..277f883 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -48,7 +48,9 @@ before_install:
   - cat ~/.m2/settings.xml
   - sed -i.bak -e 
's|https://nexus.codehaus.org/snapshots/|https://oss.sonatype.org/content/repositories/codehaus-snapshots/|g'
 ~/.m2/settings.xml
   - cat ~/.m2/settings.xml
-  - echo "MAVEN_OPTS='-Xmx1024m -XX:MaxPermSize=512m 
-XX:+BytecodeVerificationLocal'" > ~/.mavenrc
+  - echo 'MAVEN_OPTS="$MAVEN_OPTS -Xmx1024m -XX:MaxPermSize=512m 
-XX:+BytecodeVerificationLocal"' >> ~/.mavenrc
+  - echo $'MAVEN_OPTS="$MAVEN_OPTS -Dorg.slf4j.simpleLogger.showDateTime=true 
-Dorg.slf4j.simpleLogger.dateTimeFormat=-MM-dd\'T\'HH:mm:ss.SSS"' >> 
~/.mavenrc
+  - cat ~/.mavenrc
   - if [ "$TRAVIS_OS_NAME" == "osx" ]; then export 
JAVA_HOME=$(/usr/libexec/java_home); fi
   - if [ "$TRAVIS_OS_NAME" == "linux" ]; then jdk_switcher use "$CUSTOM_JDK"; 
fi
   - export BEAM_SUREFIRE_ARGLINE="-Xmx512m"

[GitHub] incubator-beam pull request #1175: pom.xml: make Dataflow runner like every ...

[dhalperi]

GitHub user dhalperi opened a pull request:

https://github.com/apache/incubator-beam/pull/1175

pom.xml: make Dataflow runner like every other runner

Pom dependencies no longer require it to be treated specially.

R: @davorbonaci @jbonofre 

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/dhalperi/incubator-beam 
archetype-build-order-tweaks

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-beam/pull/1175.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1175


commit a6d34f48d10a222d6e51dc4f0008efde071651db
Author: Dan Halperin 
Date:   2016-10-24T20:16:01Z

pom.xml: make Dataflow runner like every other runner

Pom dependencies no longer require it to be treated specially.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (BEAM-811) DoFnLifecycleManager may call teardown on a DoFn multiple times

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15603293#comment-15603293
 ] 

ASF GitHub Bot commented on BEAM-811:
-

GitHub user tgroh opened a pull request:

https://github.com/apache/incubator-beam/pull/1174

[BEAM-811] Improve teardown behavior in DoFnLifecycleManager

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

 - [ ] Make sure the PR title is formatted like:
   `[BEAM-] Description of pull request`
 - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable
   Travis-CI on your fork and ensure the whole test matrix passes).
 - [ ] Replace `` in the title with the actual Jira issue
   number, if there is one.
 - [ ] If this contribution is large, please file an Apache
   [Individual Contributor License 
Agreement](https://www.apache.org/licenses/icla.txt).

---

Use Cache invalidation hooks to teardown DoFns that are no longer in the
cache. Ensure that remove() and removeAll() report thrown exceptions
even though the exceptions are not thrown by the LoadingCache.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/tgroh/incubator-beam multi-teardown

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-beam/pull/1174.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1174


commit 7558e1461c44cf63fa5c89976a41c2feab0a92a3
Author: Thomas Groh 
Date:   2016-10-24T20:43:43Z

Improve teardown behavior in DoFnLifecycleManager

Use Cache invalidation hooks to teardown DoFns that are no longer in the
cache. Ensure that remove() and removeAll() report thrown exceptions
even though the exceptions are not thrown by the LoadingCache.




> DoFnLifecycleManager may call teardown on a DoFn multiple times
> ---
>
> Key: BEAM-811
> URL: https://issues.apache.org/jira/browse/BEAM-811
> Project: Beam
>  Issue Type: Bug
>  Components: runner-direct
>Reporter: Thomas Groh
>Assignee: Thomas Groh
> Fix For: 0.4.0-incubating
>
>
> Because the iterator returned by ConcurrentMap.iterator() is weakly 
> consistent, we may obtain a DoFn that is simultaneously removed from the map 
> while iterating in DoFnLifecycleManager#removeAll and removed via the 
> explicit remove call in DoFnLifecycleManger#remove
> Instead, utilize the Cache#invalidate() method and an appropriate 
> RemovalListener to ensure that DoFn instances that are removed are torn down, 
> and exceptions are appropriately reported.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-beam pull request #1174: [BEAM-811] Improve teardown behavior in D...

[tgroh]

GitHub user tgroh opened a pull request:

https://github.com/apache/incubator-beam/pull/1174

[BEAM-811] Improve teardown behavior in DoFnLifecycleManager

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

 - [ ] Make sure the PR title is formatted like:
   `[BEAM-] Description of pull request`
 - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable
   Travis-CI on your fork and ensure the whole test matrix passes).
 - [ ] Replace `` in the title with the actual Jira issue
   number, if there is one.
 - [ ] If this contribution is large, please file an Apache
   [Individual Contributor License 
Agreement](https://www.apache.org/licenses/icla.txt).

---

Use Cache invalidation hooks to teardown DoFns that are no longer in the
cache. Ensure that remove() and removeAll() report thrown exceptions
even though the exceptions are not thrown by the LoadingCache.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/tgroh/incubator-beam multi-teardown

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-beam/pull/1174.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1174


commit 7558e1461c44cf63fa5c89976a41c2feab0a92a3
Author: Thomas Groh 
Date:   2016-10-24T20:43:43Z

Improve teardown behavior in DoFnLifecycleManager

Use Cache invalidation hooks to teardown DoFns that are no longer in the
cache. Ensure that remove() and removeAll() report thrown exceptions
even though the exceptions are not thrown by the LoadingCache.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Jenkins build is still unstable: beam_PostCommit_RunnableOnService_FlinkLocal #657

[Apache Jenkins Server]

See 

[GitHub] incubator-beam-site pull request #52: Add pointers to 0.3.0-incubating relea...

[aljoscha]

GitHub user aljoscha opened a pull request:

https://github.com/apache/incubator-beam-site/pull/52

Add pointers to 0.3.0-incubating release artifacts



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/aljoscha/incubator-beam-site release-0.3.0

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-beam-site/pull/52.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #52


commit dece23095fbbfe0f26b708151eba4d141240ed45
Author: Aljoscha Krettek 
Date:   2016-10-24T21:07:42Z

Add pointers to 0.3.0-incubating release artifacts




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Jenkins build is still unstable: beam_PostCommit_RunnableOnService_FlinkLocal #656

[Apache Jenkins Server]

See 

Jenkins build is back to stable : beam_PostCommit_RunnableOnService_GoogleCloudDataflow #1410

[Apache Jenkins Server]

See 

[jira] [Created] (BEAM-811) DoFnLifecycleManager may call teardown on a DoFn multiple times

[Thomas Groh (JIRA)]

Thomas Groh created BEAM-811:


 Summary: DoFnLifecycleManager may call teardown on a DoFn multiple 
times
 Key: BEAM-811
 URL: https://issues.apache.org/jira/browse/BEAM-811
 Project: Beam
  Issue Type: Bug
  Components: runner-direct
Reporter: Thomas Groh
Assignee: Thomas Groh
 Fix For: 0.4.0-incubating


Because the iterator returned by ConcurrentMap.iterator() is weakly consistent, 
we may obtain a DoFn that is simultaneously removed from the map while 
iterating in DoFnLifecycleManager#removeAll and removed via the explicit remove 
call in DoFnLifecycleManger#remove

Instead, utilize the Cache#invalidate() method and an appropriate 
RemovalListener to ensure that DoFn instances that are removed are torn down, 
and exceptions are appropriately reported.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

svn commit: r16664 - /dev/incubator/beam/0.3.0-incubating/

[aljoscha]

Author: aljoscha
Date: Mon Oct 24 20:48:49 2016
New Revision: 16664

Log:
Add Apache Beam 0.3.0-incubating RC1 Files

Added:
dev/incubator/beam/0.3.0-incubating/

dev/incubator/beam/0.3.0-incubating/apache-beam-0.3.0-incubating-source-release.zip
   (with props)

dev/incubator/beam/0.3.0-incubating/apache-beam-0.3.0-incubating-source-release.zip.asc

dev/incubator/beam/0.3.0-incubating/apache-beam-0.3.0-incubating-source-release.zip.md5

dev/incubator/beam/0.3.0-incubating/apache-beam-0.3.0-incubating-source-release.zip.sha1

Added: 
dev/incubator/beam/0.3.0-incubating/apache-beam-0.3.0-incubating-source-release.zip
==
Binary file - no diff available.

Propchange: 
dev/incubator/beam/0.3.0-incubating/apache-beam-0.3.0-incubating-source-release.zip
--
svn:mime-type = application/octet-stream

Added: 
dev/incubator/beam/0.3.0-incubating/apache-beam-0.3.0-incubating-source-release.zip.asc
==
--- 
dev/incubator/beam/0.3.0-incubating/apache-beam-0.3.0-incubating-source-release.zip.asc
 (added)
+++ 
dev/incubator/beam/0.3.0-incubating/apache-beam-0.3.0-incubating-source-release.zip.asc
 Mon Oct 24 20:48:49 2016
@@ -0,0 +1,17 @@
+-BEGIN PGP SIGNATURE-
+Version: GnuPG v2
+
+iQIcBAABCgAGBQJYDmMEAAoJEKj0/ZcSHXKT0AsQALEuUYjX4ieWfvrE4PoTBcvH
+MXv/AmpO8p11BxWmFIjdGPNw/HqWOJbrYR1PgWOZBUvrKW3XjiIE7rkezzP26sfg
+zIKsKO1ij7nw7lbHXPxnmH79n/hUjLZ+BWsurikSyT+nrRORrc51JSlc0IZel9p0
+n8DkYAvBFV6kRMe3cZRtY6DSkn3owV3EqmM7QdOhENyi+6whBGLx9W0+h3nbULdV
+OZd9Mnjqck3JB5m78WDZ8XjzXYYBQ8ewAlAdLullRzmyPx0vuNi2+qi9bZe2UL+C
+0433ZJVEmVhIun9CsWg9sK0KZuh/YVSEFBo3ZUy17GMUOedvn9ypATXHymYnm3IP
+DaSAOoHQfjkAbGZ3DuqCvYiC0+w9fQXy2rUy7z7x+wbQ1aqmjWJUWbk/YJZrAVgI
+IQgvHrRKSGyFNvPTbNQIaAvV9WGN0qCpfhxRNHdBqR0SYVTVrYmG+jlxqlIlwdlT
+mycr1dfdPVhLCP2zIgadMMYz5au+cAG7qcqKreQZj33RLB0xN27fpMpuBrDDTE5p
+ilRbHlqWnjrmW0LMC+4PA3zEMydOuyhBrwO327F1QhDOxypBp7fzOUtj0snBbAn9
+/6ygai8E5nIUlsjqD40NV/SehuZyIJUeALTqkNjd6SYKMQhMsH4MPzpNPyiVdM+Q
+xfkxXm4/tltPEweG5eZ/
+=9GwF
+-END PGP SIGNATURE-

Added: 
dev/incubator/beam/0.3.0-incubating/apache-beam-0.3.0-incubating-source-release.zip.md5
==
--- 
dev/incubator/beam/0.3.0-incubating/apache-beam-0.3.0-incubating-source-release.zip.md5
 (added)
+++ 
dev/incubator/beam/0.3.0-incubating/apache-beam-0.3.0-incubating-source-release.zip.md5
 Mon Oct 24 20:48:49 2016
@@ -0,0 +1,2 @@
+apache-beam-0.3.0-incubating-source-release.zip: 
+09 7B 6A 0A C9 3E 71 C1  05 0C 71 65 E9 0C 4F AE

Added: 
dev/incubator/beam/0.3.0-incubating/apache-beam-0.3.0-incubating-source-release.zip.sha1
==
--- 
dev/incubator/beam/0.3.0-incubating/apache-beam-0.3.0-incubating-source-release.zip.sha1
 (added)
+++ 
dev/incubator/beam/0.3.0-incubating/apache-beam-0.3.0-incubating-source-release.zip.sha1
 Mon Oct 24 20:48:49 2016
@@ -0,0 +1,2 @@
+apache-beam-0.3.0-incubating-source-release.zip: 
+0A61 770D 236C 3B08 0C65  6A62 EBD7 29F0 3AAF 28FB

[jira] [Commented] (BEAM-498) Make DoFnWithContext the new DoFn

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15603111#comment-15603111
 ] 

ASF GitHub Bot commented on BEAM-498:
-

GitHub user kennknowles opened a pull request:

https://github.com/apache/incubator-beam/pull/1173

[BEAM-498] Port some example utils from OldDoFn to DoFn

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

 - [x] Make sure the PR title is formatted like:
   `[BEAM-] Description of pull request`
 - [x] Make sure tests pass via `mvn clean verify`. (Even better, enable
   Travis-CI on your fork and ensure the whole test matrix passes).
 - [x] Replace `` in the title with the actual Jira issue
   number, if there is one.
 - [x] If this contribution is large, please file an Apache
   [Individual Contributor License 
Agreement](https://www.apache.org/licenses/icla.txt).

---

R: @dhalperi 

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/kennknowles/incubator-beam OldDoFn-examples

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-beam/pull/1173.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1173


commit 0f18bf3f2e01f2f6ce7eb2b710e95b9dc7646a4e
Author: Kenneth Knowles 
Date:   2016-10-24T20:28:06Z

Port some example utils from OldDoFn to DoFn




> Make DoFnWithContext the new DoFn
> -
>
> Key: BEAM-498
> URL: https://issues.apache.org/jira/browse/BEAM-498
> Project: Beam
>  Issue Type: New Feature
>  Components: sdk-java-core
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>  Labels: backward-incompatible
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-beam pull request #1173: [BEAM-498] Port some example utils from O...

[kennknowles]

GitHub user kennknowles opened a pull request:

https://github.com/apache/incubator-beam/pull/1173

[BEAM-498] Port some example utils from OldDoFn to DoFn

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

 - [x] Make sure the PR title is formatted like:
   `[BEAM-] Description of pull request`
 - [x] Make sure tests pass via `mvn clean verify`. (Even better, enable
   Travis-CI on your fork and ensure the whole test matrix passes).
 - [x] Replace `` in the title with the actual Jira issue
   number, if there is one.
 - [x] If this contribution is large, please file an Apache
   [Individual Contributor License 
Agreement](https://www.apache.org/licenses/icla.txt).

---

R: @dhalperi 

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/kennknowles/incubator-beam OldDoFn-examples

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-beam/pull/1173.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1173


commit 0f18bf3f2e01f2f6ce7eb2b710e95b9dc7646a4e
Author: Kenneth Knowles 
Date:   2016-10-24T20:28:06Z

Port some example utils from OldDoFn to DoFn




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Updated] (BEAM-745) Display Data fails on transforms with nested components of the same type

[Scott Wegner (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Wegner updated BEAM-745:
--
Labels: backward-incompatible  (was: )

> Display Data fails on transforms with nested components of the same type
> 
>
> Key: BEAM-745
> URL: https://issues.apache.org/jira/browse/BEAM-745
> Project: Beam
>  Issue Type: Bug
>Reporter: Scott Wegner
>Assignee: Scott Wegner
>  Labels: backward-incompatible
> Fix For: 0.3.0-incubating
>
>
> Display Data supports the notion of "sub components", components within a 
> transform class which can contribute their own display data. We add a 
> namespace to display data items based on the originating component, which 
> keeps the display data items unique within the step.
> There are instances where a component is included multiple times within a 
> step. We handle the case of the same instance being shared by simply ignoring 
> it the second time. However, we don't handle the case of a separate instance 
> being added of the same class. Currently the separate instances will add 
> display data with the same namespace and key, causing a failure.
> This can come up for example when infrastructure at different levels wrap and 
> re-wrap a component. We saw this with a bounded source being adapted multiple 
> times, Bounded -> Unbounded -> Bounded -> Unbounded. The 
> BoundedToUnboundedSourceAdapter was included multiple times with separate 
> instances and caused a failure while populating display data.
> See: https://github.com/apache/incubator-beam/pull/1088 for a failing test 
> case.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[1/2] incubator-beam git commit: OffsetBasedSource: allow empty sources

[dhalperi]

Repository: incubator-beam
Updated Branches:
  refs/heads/master dfaf2a896 -> 44b9f4ebb


OffsetBasedSource: allow empty sources

As one example, empty files exist.


Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/7e9ad995
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/7e9ad995
Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/7e9ad995

Branch: refs/heads/master
Commit: 7e9ad9954e50bd01fba4cda84c182af895b2c23f
Parents: dfaf2a8
Author: Dan Halperin 
Authored: Fri Oct 21 12:21:45 2016 -0700
Committer: Dan Halperin 
Committed: Mon Oct 24 13:16:50 2016 -0700

--
 .../apache/beam/sdk/io/OffsetBasedSource.java   |  4 +--
 .../beam/sdk/io/OffsetBasedSourceTest.java  | 30 +++-
 2 files changed, 25 insertions(+), 9 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/7e9ad995/sdks/java/core/src/main/java/org/apache/beam/sdk/io/OffsetBasedSource.java
--
diff --git 
a/sdks/java/core/src/main/java/org/apache/beam/sdk/io/OffsetBasedSource.java 
b/sdks/java/core/src/main/java/org/apache/beam/sdk/io/OffsetBasedSource.java
index 6e49cc3..e9a398d 100644
--- a/sdks/java/core/src/main/java/org/apache/beam/sdk/io/OffsetBasedSource.java
+++ b/sdks/java/core/src/main/java/org/apache/beam/sdk/io/OffsetBasedSource.java
@@ -146,8 +146,8 @@ public abstract class OffsetBasedSource extends 
BoundedSource {
 this.endOffset >= 0,
 "End offset has value %s, must be non-negative", this.endOffset);
 checkArgument(
-this.startOffset < this.endOffset,
-"Start offset %s must be before end offset %s",
+this.startOffset <= this.endOffset,
+"Start offset %s may not be larger than end offset %s",
 this.startOffset, this.endOffset);
 checkArgument(
 this.minBundleSize >= 0,

http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/7e9ad995/sdks/java/core/src/test/java/org/apache/beam/sdk/io/OffsetBasedSourceTest.java
--
diff --git 
a/sdks/java/core/src/test/java/org/apache/beam/sdk/io/OffsetBasedSourceTest.java
 
b/sdks/java/core/src/test/java/org/apache/beam/sdk/io/OffsetBasedSourceTest.java
index 923b4b4..6584e5d 100644
--- 
a/sdks/java/core/src/test/java/org/apache/beam/sdk/io/OffsetBasedSourceTest.java
+++ 
b/sdks/java/core/src/test/java/org/apache/beam/sdk/io/OffsetBasedSourceTest.java
@@ -62,7 +62,7 @@ public class OffsetBasedSourceTest {
 }
 
 @Override
-public boolean producesSortedKeys(PipelineOptions options) throws 
Exception {
+public boolean producesSortedKeys(PipelineOptions options) {
   return false;
 }
 
@@ -85,7 +85,7 @@ public class OffsetBasedSourceTest {
 }
 
 @Override
-public OffsetBasedReader createReader(PipelineOptions options) 
throws IOException {
+public CoarseRangeReader createReader(PipelineOptions options) {
   return new CoarseRangeReader(this);
 }
   }
@@ -105,7 +105,7 @@ public class OffsetBasedSourceTest {
 }
 
 @Override
-public boolean startImpl() throws IOException {
+public boolean startImpl() {
   current = getCurrentSource().getStartOffset();
   while (current % granularity != 0) {
 ++current;
@@ -114,7 +114,7 @@ public class OffsetBasedSourceTest {
 }
 
 @Override
-public boolean advanceImpl() throws IOException {
+public boolean advanceImpl() {
   ++current;
   return true;
 }
@@ -130,7 +130,7 @@ public class OffsetBasedSourceTest {
 }
 
 @Override
-public void close() throws IOException { }
+public void close() { }
   }
 
   public static void assertSplitsAre(List> 
splits,
@@ -211,7 +211,7 @@ public class OffsetBasedSourceTest {
 // in the face of that.
 PipelineOptions options = PipelineOptionsFactory.create();
 CoarseRangeSource source = new CoarseRangeSource(13, 35, 1, 10);
-try (BoundedSource.BoundedReader reader = 
source.createReader(options)) {
+try (CoarseRangeReader reader = source.createReader(options)) {
   List items = new ArrayList<>();
 
   assertEquals(0.0, reader.getFractionConsumed(), 1e-6);
@@ -304,7 +304,7 @@ public class OffsetBasedSourceTest {
   public void testSplitAtFraction() throws IOException {
 PipelineOptions options = PipelineOptionsFactory.create();
 CoarseRangeSource source = new CoarseRangeSource(13, 35, 1, 10);
-try (CoarseRangeReader reader = (CoarseRangeReader) 
source.createReader(options)) {
+try (CoarseRangeReader reader = source.createReader(options)) {
   List originalItems = new ArrayList<>();
   assertTrue(reader.start());
   originalItems.add(reader.getCurrent());
@@ -341,4 +341

[GitHub] incubator-beam pull request #1168: OffsetBasedSource: allow empty sources

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-beam/pull/1168


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[2/2] incubator-beam git commit: Closes #1168

[dhalperi]

Closes #1168


Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/44b9f4eb
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/44b9f4eb
Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/44b9f4eb

Branch: refs/heads/master
Commit: 44b9f4ebbc3229d8a9ec46b0c377e57ad0176dbe
Parents: dfaf2a8 7e9ad99
Author: Dan Halperin 
Authored: Mon Oct 24 13:16:51 2016 -0700
Committer: Dan Halperin 
Committed: Mon Oct 24 13:16:51 2016 -0700

--
 .../apache/beam/sdk/io/OffsetBasedSource.java   |  4 +--
 .../beam/sdk/io/OffsetBasedSourceTest.java  | 30 +++-
 2 files changed, 25 insertions(+), 9 deletions(-)
--

[jira] [Commented] (BEAM-742) Move trigger state machines to runners-core, convert triggers to AST

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15603036#comment-15603036
 ] 

ASF GitHub Bot commented on BEAM-742:
-

GitHub user kennknowles opened a pull request:

https://github.com/apache/incubator-beam/pull/1172

[BEAM-742] Remove pieces of Trigger now owned by TriggerStateMachine

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

 - [x] Make sure the PR title is formatted like:
   `[BEAM-] Description of pull request`
 - [x] Make sure tests pass via `mvn clean verify`. (Even better, enable
   Travis-CI on your fork and ensure the whole test matrix passes).
 - [x] Replace `` in the title with the actual Jira issue
   number, if there is one.
 - [x] If this contribution is large, please file an Apache
   [Individual Contributor License 
Agreement](https://www.apache.org/licenses/icla.txt).

---

No reviewer yet; currently posting for self-review.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/kennknowles/incubator-beam Trigger-delete

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-beam/pull/1172.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1172


commit 2612ccfea450f9a99589fd8e549c8148a3faad36
Author: Kenneth Knowles 
Date:   2016-10-24T19:57:37Z

Remove pieces of Trigger now owned by TriggerStateMachine




> Move trigger state machines to runners-core, convert triggers to AST
> 
>
> Key: BEAM-742
> URL: https://issues.apache.org/jira/browse/BEAM-742
> Project: Beam
>  Issue Type: Sub-task
>  Components: runner-core, sdk-java-core
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-beam pull request #1172: [BEAM-742] Remove pieces of Trigger now o...

[kennknowles]

GitHub user kennknowles opened a pull request:

https://github.com/apache/incubator-beam/pull/1172

[BEAM-742] Remove pieces of Trigger now owned by TriggerStateMachine

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

 - [x] Make sure the PR title is formatted like:
   `[BEAM-] Description of pull request`
 - [x] Make sure tests pass via `mvn clean verify`. (Even better, enable
   Travis-CI on your fork and ensure the whole test matrix passes).
 - [x] Replace `` in the title with the actual Jira issue
   number, if there is one.
 - [x] If this contribution is large, please file an Apache
   [Individual Contributor License 
Agreement](https://www.apache.org/licenses/icla.txt).

---

No reviewer yet; currently posting for self-review.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/kennknowles/incubator-beam Trigger-delete

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-beam/pull/1172.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1172


commit 2612ccfea450f9a99589fd8e549c8148a3faad36
Author: Kenneth Knowles 
Date:   2016-10-24T19:57:37Z

Remove pieces of Trigger now owned by TriggerStateMachine




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Jenkins build became unstable: beam_PostCommit_RunnableOnService_GoogleCloudDataflow #1409

[Apache Jenkins Server]

See 

incubator-beam git commit: [maven-release-plugin] rollback changes from release preparation of v0.3.0-incubating-RC1

[aljoscha]

Repository: incubator-beam
Updated Branches:
  refs/heads/release-0.3.0-incubating a4985be4a -> 7f374604c


[maven-release-plugin] rollback changes from release preparation of 
v0.3.0-incubating-RC1


Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/7f374604
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/7f374604
Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/7f374604

Branch: refs/heads/release-0.3.0-incubating
Commit: 7f374604c6c55b609652ad9c24ab591ad1352511
Parents: a4985be
Author: Aljoscha Krettek 
Authored: Mon Oct 24 21:45:25 2016 +0200
Committer: Aljoscha Krettek 
Committed: Mon Oct 24 21:45:25 2016 +0200

--
 examples/java/pom.xml   | 2 +-
 examples/java8/pom.xml  | 2 +-
 examples/pom.xml| 2 +-
 pom.xml | 4 ++--
 runners/core-java/pom.xml   | 2 +-
 runners/direct-java/pom.xml | 2 +-
 runners/flink/examples/pom.xml  | 2 +-
 runners/flink/pom.xml   | 2 +-
 runners/flink/runner/pom.xml| 2 +-
 runners/google-cloud-dataflow-java/pom.xml  | 2 +-
 runners/pom.xml | 2 +-
 runners/spark/pom.xml   | 2 +-
 sdks/java/build-tools/pom.xml   | 2 +-
 sdks/java/core/pom.xml  | 2 +-
 sdks/java/extensions/join-library/pom.xml   | 2 +-
 sdks/java/extensions/pom.xml| 2 +-
 sdks/java/io/google-cloud-platform/pom.xml  | 2 +-
 sdks/java/io/hdfs/pom.xml   | 2 +-
 sdks/java/io/jdbc/pom.xml   | 2 +-
 sdks/java/io/jms/pom.xml| 2 +-
 sdks/java/io/kafka/pom.xml  | 2 +-
 sdks/java/io/kinesis/pom.xml| 2 +-
 sdks/java/io/mongodb/pom.xml| 2 +-
 sdks/java/io/pom.xml| 2 +-
 sdks/java/java8tests/pom.xml| 2 +-
 sdks/java/maven-archetypes/examples/pom.xml | 2 +-
 sdks/java/maven-archetypes/pom.xml  | 2 +-
 sdks/java/maven-archetypes/starter/pom.xml  | 2 +-
 sdks/java/microbenchmarks/pom.xml   | 2 +-
 sdks/java/pom.xml   | 2 +-
 sdks/pom.xml| 2 +-
 31 files changed, 32 insertions(+), 32 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/7f374604/examples/java/pom.xml
--
diff --git a/examples/java/pom.xml b/examples/java/pom.xml
index 8b2f371..37cb15a 100644
--- a/examples/java/pom.xml
+++ b/examples/java/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.beam
 beam-examples-parent
-0.3.0-incubating
+0.3.0-incubating-SNAPSHOT
 ../pom.xml
   
 

http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/7f374604/examples/java8/pom.xml
--
diff --git a/examples/java8/pom.xml b/examples/java8/pom.xml
index 70073fa..44cf918 100644
--- a/examples/java8/pom.xml
+++ b/examples/java8/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.beam
 beam-examples-parent
-0.3.0-incubating
+0.3.0-incubating-SNAPSHOT
 ../pom.xml
   
 

http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/7f374604/examples/pom.xml
--
diff --git a/examples/pom.xml b/examples/pom.xml
index c10b743..2efb146 100644
--- a/examples/pom.xml
+++ b/examples/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.beam
 beam-parent
-0.3.0-incubating
+0.3.0-incubating-SNAPSHOT
 ../pom.xml
   
 

http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/7f374604/pom.xml
--
diff --git a/pom.xml b/pom.xml
index 3836205..6a24d98 100644
--- a/pom.xml
+++ b/pom.xml
@@ -34,7 +34,7 @@
   http://beam.incubator.apache.org
   2016
 
-  0.3.0-incubating
+  0.3.0-incubating-SNAPSHOT
 
   
 
@@ -48,7 +48,7 @@
 
scm:git:https://git-wip-us.apache.org/repos/asf/incubator-beam.git
 
scm:git:https://git-wip-us.apache.org/repos/asf/incubator-beam.git
 
https://git-wip-us.apache.org/repos/asf?p=incubator-beam.git;a=summary
-v0.3.0-incubating-RC1
+release-0.3.0-incubating
   
 
   

http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/7f374604/runners/core-java/pom.xml
--
diff --git a/runners/core-java/pom.xml b/runners/core-java/pom.xml
index b4799eb..d84c420 100644
--- a/runners/core-java/pom.xml
+++ b/runners/core-java/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.beam
 beam-runners-parent
-0.3.0-incubating
+0.3.0-incubating-SNAPSHOT
  

[incubator-beam] Git Push Summary

[aljoscha]

Repository: incubator-beam
Updated Tags:  refs/tags/v0.3.0-incubating-RC1 [created] 5d86ff7f0

incubator-beam git commit: [maven-release-plugin] prepare release v0.3.0-incubating-RC1

[aljoscha]

Repository: incubator-beam
Updated Branches:
  refs/heads/release-0.3.0-incubating e2d8a62e3 -> a4985be4a


[maven-release-plugin] prepare release v0.3.0-incubating-RC1


Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/a4985be4
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/a4985be4
Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/a4985be4

Branch: refs/heads/release-0.3.0-incubating
Commit: a4985be4a2ba7a6780e8f637d313518add8b2c11
Parents: e2d8a62
Author: Aljoscha Krettek 
Authored: Mon Oct 24 21:45:04 2016 +0200
Committer: Aljoscha Krettek 
Committed: Mon Oct 24 21:45:04 2016 +0200

--
 examples/java/pom.xml   | 2 +-
 examples/java8/pom.xml  | 2 +-
 examples/pom.xml| 2 +-
 pom.xml | 4 ++--
 runners/core-java/pom.xml   | 2 +-
 runners/direct-java/pom.xml | 2 +-
 runners/flink/examples/pom.xml  | 2 +-
 runners/flink/pom.xml   | 2 +-
 runners/flink/runner/pom.xml| 2 +-
 runners/google-cloud-dataflow-java/pom.xml  | 2 +-
 runners/pom.xml | 2 +-
 runners/spark/pom.xml   | 2 +-
 sdks/java/build-tools/pom.xml   | 2 +-
 sdks/java/core/pom.xml  | 2 +-
 sdks/java/extensions/join-library/pom.xml   | 2 +-
 sdks/java/extensions/pom.xml| 2 +-
 sdks/java/io/google-cloud-platform/pom.xml  | 2 +-
 sdks/java/io/hdfs/pom.xml   | 2 +-
 sdks/java/io/jdbc/pom.xml   | 2 +-
 sdks/java/io/jms/pom.xml| 2 +-
 sdks/java/io/kafka/pom.xml  | 2 +-
 sdks/java/io/kinesis/pom.xml| 2 +-
 sdks/java/io/mongodb/pom.xml| 2 +-
 sdks/java/io/pom.xml| 2 +-
 sdks/java/java8tests/pom.xml| 2 +-
 sdks/java/maven-archetypes/examples/pom.xml | 2 +-
 sdks/java/maven-archetypes/pom.xml  | 2 +-
 sdks/java/maven-archetypes/starter/pom.xml  | 2 +-
 sdks/java/microbenchmarks/pom.xml   | 2 +-
 sdks/java/pom.xml   | 2 +-
 sdks/pom.xml| 2 +-
 31 files changed, 32 insertions(+), 32 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/a4985be4/examples/java/pom.xml
--
diff --git a/examples/java/pom.xml b/examples/java/pom.xml
index 37cb15a..8b2f371 100644
--- a/examples/java/pom.xml
+++ b/examples/java/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.beam
 beam-examples-parent
-0.3.0-incubating-SNAPSHOT
+0.3.0-incubating
 ../pom.xml
   
 

http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/a4985be4/examples/java8/pom.xml
--
diff --git a/examples/java8/pom.xml b/examples/java8/pom.xml
index 44cf918..70073fa 100644
--- a/examples/java8/pom.xml
+++ b/examples/java8/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.beam
 beam-examples-parent
-0.3.0-incubating-SNAPSHOT
+0.3.0-incubating
 ../pom.xml
   
 

http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/a4985be4/examples/pom.xml
--
diff --git a/examples/pom.xml b/examples/pom.xml
index 2efb146..c10b743 100644
--- a/examples/pom.xml
+++ b/examples/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.beam
 beam-parent
-0.3.0-incubating-SNAPSHOT
+0.3.0-incubating
 ../pom.xml
   
 

http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/a4985be4/pom.xml
--
diff --git a/pom.xml b/pom.xml
index 6a24d98..3836205 100644
--- a/pom.xml
+++ b/pom.xml
@@ -34,7 +34,7 @@
   http://beam.incubator.apache.org
   2016
 
-  0.3.0-incubating-SNAPSHOT
+  0.3.0-incubating
 
   
 
@@ -48,7 +48,7 @@
 
scm:git:https://git-wip-us.apache.org/repos/asf/incubator-beam.git
 
scm:git:https://git-wip-us.apache.org/repos/asf/incubator-beam.git
 
https://git-wip-us.apache.org/repos/asf?p=incubator-beam.git;a=summary
-release-0.3.0-incubating
+v0.3.0-incubating-RC1
   
 
   

http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/a4985be4/runners/core-java/pom.xml
--
diff --git a/runners/core-java/pom.xml b/runners/core-java/pom.xml
index d84c420..b4799eb 100644
--- a/runners/core-java/pom.xml
+++ b/runners/core-java/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.beam
 beam-runners-parent
-0.3.0-incubating-SNAPSHOT
+0.3.0-incubating
 ../pom.xml
   
 

http://gi

[jira] [Commented] (BEAM-809) Create a KryoRegistrator for the SparkRunner.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15602978#comment-15602978
 ] 

ASF GitHub Bot commented on BEAM-809:
-

GitHub user amitsela opened a pull request:

https://github.com/apache/incubator-beam/pull/1171

[BEAM-809] Create a KryoRegistrator for the SparkRunner.

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

 - [ ] Make sure the PR title is formatted like:
   `[BEAM-] Description of pull request`
 - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable
   Travis-CI on your fork and ensure the whole test matrix passes).
 - [ ] Replace `` in the title with the actual Jira issue
   number, if there is one.
 - [ ] If this contribution is large, please file an Apache
   [Individual Contributor License 
Agreement](https://www.apache.org/licenses/icla.txt).

---



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/amitsela/incubator-beam BEAM-809

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-beam/pull/1171.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1171


commit 4e00575a4a8e25ed3618f2f7672d3c24c662a149
Author: Sela 
Date:   2016-10-24T19:35:39Z

[BEAM-809] Create a KryoRegistrator for the SparkRunner.




> Create a KryoRegistrator for the SparkRunner. 
> --
>
> Key: BEAM-809
> URL: https://issues.apache.org/jira/browse/BEAM-809
> Project: Beam
>  Issue Type: Bug
>  Components: runner-spark
>Reporter: Amit Sela
>Assignee: Amit Sela
>
> Create a custom KryoRegistrator for the Spark runner's needs - mostly 
> immutable collections.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Jenkins build is still unstable: beam_PostCommit_RunnableOnService_FlinkLocal #655

[Apache Jenkins Server]

See 

[GitHub] incubator-beam pull request #1171: [BEAM-809] Create a KryoRegistrator for t...

[amitsela]

GitHub user amitsela opened a pull request:

https://github.com/apache/incubator-beam/pull/1171

[BEAM-809] Create a KryoRegistrator for the SparkRunner.

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

 - [ ] Make sure the PR title is formatted like:
   `[BEAM-] Description of pull request`
 - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable
   Travis-CI on your fork and ensure the whole test matrix passes).
 - [ ] Replace `` in the title with the actual Jira issue
   number, if there is one.
 - [ ] If this contribution is large, please file an Apache
   [Individual Contributor License 
Agreement](https://www.apache.org/licenses/icla.txt).

---



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/amitsela/incubator-beam BEAM-809

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-beam/pull/1171.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1171


commit 4e00575a4a8e25ed3618f2f7672d3c24c662a149
Author: Sela 
Date:   2016-10-24T19:35:39Z

[BEAM-809] Create a KryoRegistrator for the SparkRunner.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[incubator-beam] Git Push Summary [forced push!] [Forced Update!]

[aljoscha]

Repository: incubator-beam
Updated Branches:
  refs/heads/release-0.3.0-incubating 564bbcc4b -> e2d8a62e3 (forced update)

[jira] [Commented] (BEAM-806) Maven Release Plugin Does Not Set Archetype Versions

[Daniel Halperin (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15602965#comment-15602965
 ] 

Daniel Halperin commented on BEAM-806:
--

I filed [BEAM-810] which is relevant and includes more archetype-related pain.

> Maven Release Plugin Does Not Set Archetype Versions
> 
>
> Key: BEAM-806
> URL: https://issues.apache.org/jira/browse/BEAM-806
> Project: Beam
>  Issue Type: Improvement
>  Components: build-system
>Affects Versions: 0.1.0-incubating, 0.2.0-incubating, 0.3.0-incubating
>Reporter: Aljoscha Krettek
>Priority: Blocker
> Fix For: 0.3.0-incubating
>
>
> When running {{mvn release:prepare}} as described in the new release guide 
> this does not update the version of the poms in the archetypes. To be clear, 
> the version of the archetype pom is updated, the pom in 
> {{archetype-resources}} (the pom of the project generated by the archetype) 
> is not updated.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BEAM-696) Side-Inputs non-deterministic with merging main-input windows

[Amit Sela (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15602964#comment-15602964
 ] 

Amit Sela commented on BEAM-696:


After further discussion in dev list this seems like a documentation issue.
As [~robertwb] mentioned, Dataflow documentation mentions this here: 
https://cloud.google.com/dataflow/model/combine#using-combine-perkey and Beam 
documentation might want to be a bit more clear on this and point that this 
could affect runners differently.

[~pei...@gmail.com] maybe change the name of this ticket to be about the 
non-deterministic triggering mentioned ?

> Side-Inputs non-deterministic with merging main-input windows
> -
>
> Key: BEAM-696
> URL: https://issues.apache.org/jira/browse/BEAM-696
> Project: Beam
>  Issue Type: Bug
>  Components: beam-model
>Reporter: Ben Chambers
>Assignee: Pei He
>
> Side-Inputs are non-deterministic for several reasons:
> 1. Because they depend on triggering of the side-input (this is acceptable 
> because triggers are by their nature non-deterministic).
> 2. They depend on the current state of the main-input window in order to 
> lookup the side-input. This means that with merging
> 3. Any runner optimizations that affect when the side-input is looked up may 
> cause problems with either or both of these.
> This issue focuses on #2 -- the non-determinism of side-inputs that execute 
> within a Merging WindowFn.
> Possible solution would be to defer running anything that looks up the 
> side-input until we need to extract an output, and using the main-window at 
> that point. Specifically, if the main-window is a MergingWindowFn, don't 
> execute any kind of pre-combine, instead buffer all the inputs and combine 
> later.
> This could still run into some non-determinism if there are triggers 
> controlling when we extract output.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (BEAM-810) Beam's uses of archetypes are problematic

[Daniel Halperin (JIRA)]

Daniel Halperin created BEAM-810:


 Summary: Beam's uses of archetypes are problematic
 Key: BEAM-810
 URL: https://issues.apache.org/jira/browse/BEAM-810
 Project: Beam
  Issue Type: Bug
  Components: build-system
Affects Versions: Not applicable
Reporter: Daniel Halperin


1. Using a version range in the archetype pom.xml is inaccurate, because we 
don't have a stable API. We can't actually expect an older version to work as 
we make backwards-incompatible changes.

1. In PR #936, a decision was made to pin the version of the archetype. This 
requires manual changes to update many places (archetype-resources), which 
breaks the release process in a few different ways. [See BEAM-806].

1. Travis uses `mvn verify`, not `mvn install` -- so archetypes are not 
properly tested and instead we're actually picking up the SNAPSHOT archetypes 
deployed the prior night (IIRC).

1. There was some other issue with parallelizing maven where archetypes are not 
properly marked as depending on "you can't test the archetype until you've 
installed the examples packages".

1. We're not actually testing that the archetype-generated projects actually 
compile. [BEAM-641].

This is all pretty ugly. Suggestions wanted.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (BEAM-810) Beam's uses of archetypes are problematic

[Daniel Halperin (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Daniel Halperin updated BEAM-810:
-
Description: 
* Using a version range in the archetype pom.xml is inaccurate, because we 
don't have a stable API. We can't actually expect an older version to work as 
we make backwards-incompatible changes.

* In PR #936, a decision was made to pin the version of the archetype. This 
requires manual changes to update many places (archetype-resources), which 
breaks the release process in a few different ways. See [BEAM-806].

* Travis uses `mvn verify`, not `mvn install` -- so archetypes are not properly 
tested and instead we're actually picking up the SNAPSHOT archetypes deployed 
the prior night (IIRC).

* There was some other issue with parallelizing maven where archetypes are not 
properly marked as depending on "you can't test the archetype until you've 
installed the examples packages".

* We're not actually testing that the archetype-generated projects actually 
compile. [BEAM-641].

This is all pretty ugly. Suggestions wanted.

  was:
1. Using a version range in the archetype pom.xml is inaccurate, because we 
don't have a stable API. We can't actually expect an older version to work as 
we make backwards-incompatible changes.

1. In PR #936, a decision was made to pin the version of the archetype. This 
requires manual changes to update many places (archetype-resources), which 
breaks the release process in a few different ways. See [BEAM-806].

1. Travis uses `mvn verify`, not `mvn install` -- so archetypes are not 
properly tested and instead we're actually picking up the SNAPSHOT archetypes 
deployed the prior night (IIRC).

1. There was some other issue with parallelizing maven where archetypes are not 
properly marked as depending on "you can't test the archetype until you've 
installed the examples packages".

1. We're not actually testing that the archetype-generated projects actually 
compile. [BEAM-641].

This is all pretty ugly. Suggestions wanted.


> Beam's uses of archetypes are problematic
> -
>
> Key: BEAM-810
> URL: https://issues.apache.org/jira/browse/BEAM-810
> Project: Beam
>  Issue Type: Bug
>  Components: build-system
>Affects Versions: Not applicable
>Reporter: Daniel Halperin
>
> * Using a version range in the archetype pom.xml is inaccurate, because we 
> don't have a stable API. We can't actually expect an older version to work as 
> we make backwards-incompatible changes.
> * In PR #936, a decision was made to pin the version of the archetype. This 
> requires manual changes to update many places (archetype-resources), which 
> breaks the release process in a few different ways. See [BEAM-806].
> * Travis uses `mvn verify`, not `mvn install` -- so archetypes are not 
> properly tested and instead we're actually picking up the SNAPSHOT archetypes 
> deployed the prior night (IIRC).
> * There was some other issue with parallelizing maven where archetypes are 
> not properly marked as depending on "you can't test the archetype until 
> you've installed the examples packages".
> * We're not actually testing that the archetype-generated projects actually 
> compile. [BEAM-641].
> This is all pretty ugly. Suggestions wanted.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (BEAM-810) Beam's uses of archetypes are problematic

[Daniel Halperin (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Daniel Halperin updated BEAM-810:
-
Description: 
1. Using a version range in the archetype pom.xml is inaccurate, because we 
don't have a stable API. We can't actually expect an older version to work as 
we make backwards-incompatible changes.

1. In PR #936, a decision was made to pin the version of the archetype. This 
requires manual changes to update many places (archetype-resources), which 
breaks the release process in a few different ways. See [BEAM-806].

1. Travis uses `mvn verify`, not `mvn install` -- so archetypes are not 
properly tested and instead we're actually picking up the SNAPSHOT archetypes 
deployed the prior night (IIRC).

1. There was some other issue with parallelizing maven where archetypes are not 
properly marked as depending on "you can't test the archetype until you've 
installed the examples packages".

1. We're not actually testing that the archetype-generated projects actually 
compile. [BEAM-641].

This is all pretty ugly. Suggestions wanted.

  was:
1. Using a version range in the archetype pom.xml is inaccurate, because we 
don't have a stable API. We can't actually expect an older version to work as 
we make backwards-incompatible changes.

1. In PR #936, a decision was made to pin the version of the archetype. This 
requires manual changes to update many places (archetype-resources), which 
breaks the release process in a few different ways. [See BEAM-806].

1. Travis uses `mvn verify`, not `mvn install` -- so archetypes are not 
properly tested and instead we're actually picking up the SNAPSHOT archetypes 
deployed the prior night (IIRC).

1. There was some other issue with parallelizing maven where archetypes are not 
properly marked as depending on "you can't test the archetype until you've 
installed the examples packages".

1. We're not actually testing that the archetype-generated projects actually 
compile. [BEAM-641].

This is all pretty ugly. Suggestions wanted.


> Beam's uses of archetypes are problematic
> -
>
> Key: BEAM-810
> URL: https://issues.apache.org/jira/browse/BEAM-810
> Project: Beam
>  Issue Type: Bug
>  Components: build-system
>Affects Versions: Not applicable
>Reporter: Daniel Halperin
>
> 1. Using a version range in the archetype pom.xml is inaccurate, because we 
> don't have a stable API. We can't actually expect an older version to work as 
> we make backwards-incompatible changes.
> 1. In PR #936, a decision was made to pin the version of the archetype. This 
> requires manual changes to update many places (archetype-resources), which 
> breaks the release process in a few different ways. See [BEAM-806].
> 1. Travis uses `mvn verify`, not `mvn install` -- so archetypes are not 
> properly tested and instead we're actually picking up the SNAPSHOT archetypes 
> deployed the prior night (IIRC).
> 1. There was some other issue with parallelizing maven where archetypes are 
> not properly marked as depending on "you can't test the archetype until 
> you've installed the examples packages".
> 1. We're not actually testing that the archetype-generated projects actually 
> compile. [BEAM-641].
> This is all pretty ugly. Suggestions wanted.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BEAM-696) Side-Inputs non-deterministic with merging main-input windows

[Amit Sela (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15602952#comment-15602952
 ] 

Amit Sela commented on BEAM-696:


Yep, found that piece of code ;) thanks.

> Side-Inputs non-deterministic with merging main-input windows
> -
>
> Key: BEAM-696
> URL: https://issues.apache.org/jira/browse/BEAM-696
> Project: Beam
>  Issue Type: Bug
>  Components: beam-model
>Reporter: Ben Chambers
>Assignee: Pei He
>
> Side-Inputs are non-deterministic for several reasons:
> 1. Because they depend on triggering of the side-input (this is acceptable 
> because triggers are by their nature non-deterministic).
> 2. They depend on the current state of the main-input window in order to 
> lookup the side-input. This means that with merging
> 3. Any runner optimizations that affect when the side-input is looked up may 
> cause problems with either or both of these.
> This issue focuses on #2 -- the non-determinism of side-inputs that execute 
> within a Merging WindowFn.
> Possible solution would be to defer running anything that looks up the 
> side-input until we need to extract an output, and using the main-window at 
> that point. Specifically, if the main-window is a MergingWindowFn, don't 
> execute any kind of pre-combine, instead buffer all the inputs and combine 
> later.
> This could still run into some non-determinism if there are triggers 
> controlling when we extract output.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[incubator-beam] Git Push Summary

[aljoscha]

Repository: incubator-beam
Updated Tags:  refs/tags/v0.3.0-incubating-RC1 [deleted] 47447b7e0

[jira] [Updated] (BEAM-641) Need to test the generated archetypes projects

[Daniel Halperin (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Daniel Halperin updated BEAM-641:
-
Affects Version/s: 0.4.0-incubating

> Need to test the generated archetypes projects
> --
>
> Key: BEAM-641
> URL: https://issues.apache.org/jira/browse/BEAM-641
> Project: Beam
>  Issue Type: Test
>  Components: testing
>Reporter: Pei He
>Assignee: Pei He
>Priority: Critical
> Fix For: 0.4.0-incubating
>
>
> Travis and Jenkins pre-submits don't test building the generated archetypes 
> projects.
> Currently, changes to archetypes have to be manually verified by:
> mvn archetype:generate \
> -DarchetypeArtifactId=beam-sdks-java-maven-archetypes-examples \
> -DarchetypeGroupId=org.apache.beam \
> -DarchetypeVersion=0.3.0-incubating-SNAPSHOT \
> -DgroupId=com.example \
> -DartifactId=first-beam \
> -Dversion="0.3.0-incubating-SNAPSHOT" \
> -DinteractiveMode=false \
> -Dpackage=org.apache.beam.examples
> and did "mvn clean install" in first-beam project.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (BEAM-641) Need to test the generated archetypes projects

[Daniel Halperin (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Daniel Halperin updated BEAM-641:
-
Priority: Critical  (was: Minor)

> Need to test the generated archetypes projects
> --
>
> Key: BEAM-641
> URL: https://issues.apache.org/jira/browse/BEAM-641
> Project: Beam
>  Issue Type: Test
>  Components: testing
>Reporter: Pei He
>Assignee: Pei He
>Priority: Critical
> Fix For: 0.4.0-incubating
>
>
> Travis and Jenkins pre-submits don't test building the generated archetypes 
> projects.
> Currently, changes to archetypes have to be manually verified by:
> mvn archetype:generate \
> -DarchetypeArtifactId=beam-sdks-java-maven-archetypes-examples \
> -DarchetypeGroupId=org.apache.beam \
> -DarchetypeVersion=0.3.0-incubating-SNAPSHOT \
> -DgroupId=com.example \
> -DartifactId=first-beam \
> -Dversion="0.3.0-incubating-SNAPSHOT" \
> -DinteractiveMode=false \
> -Dpackage=org.apache.beam.examples
> and did "mvn clean install" in first-beam project.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (BEAM-641) Need to test the generated archetypes projects

[Daniel Halperin (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Daniel Halperin updated BEAM-641:
-
Affects Version/s: (was: 0.4.0-incubating)

> Need to test the generated archetypes projects
> --
>
> Key: BEAM-641
> URL: https://issues.apache.org/jira/browse/BEAM-641
> Project: Beam
>  Issue Type: Test
>  Components: testing
>Reporter: Pei He
>Assignee: Pei He
>Priority: Critical
> Fix For: 0.4.0-incubating
>
>
> Travis and Jenkins pre-submits don't test building the generated archetypes 
> projects.
> Currently, changes to archetypes have to be manually verified by:
> mvn archetype:generate \
> -DarchetypeArtifactId=beam-sdks-java-maven-archetypes-examples \
> -DarchetypeGroupId=org.apache.beam \
> -DarchetypeVersion=0.3.0-incubating-SNAPSHOT \
> -DgroupId=com.example \
> -DartifactId=first-beam \
> -Dversion="0.3.0-incubating-SNAPSHOT" \
> -DinteractiveMode=false \
> -Dpackage=org.apache.beam.examples
> and did "mvn clean install" in first-beam project.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (BEAM-641) Need to test the generated archetypes projects

[Daniel Halperin (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Daniel Halperin updated BEAM-641:
-
Fix Version/s: 0.4.0-incubating

> Need to test the generated archetypes projects
> --
>
> Key: BEAM-641
> URL: https://issues.apache.org/jira/browse/BEAM-641
> Project: Beam
>  Issue Type: Test
>  Components: testing
>Reporter: Pei He
>Assignee: Pei He
>Priority: Critical
> Fix For: 0.4.0-incubating
>
>
> Travis and Jenkins pre-submits don't test building the generated archetypes 
> projects.
> Currently, changes to archetypes have to be manually verified by:
> mvn archetype:generate \
> -DarchetypeArtifactId=beam-sdks-java-maven-archetypes-examples \
> -DarchetypeGroupId=org.apache.beam \
> -DarchetypeVersion=0.3.0-incubating-SNAPSHOT \
> -DgroupId=com.example \
> -DartifactId=first-beam \
> -Dversion="0.3.0-incubating-SNAPSHOT" \
> -DinteractiveMode=false \
> -Dpackage=org.apache.beam.examples
> and did "mvn clean install" in first-beam project.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BEAM-693) pydoc is not working

[Ahmet Altay (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15602948#comment-15602948
 ] 

Ahmet Altay commented on BEAM-693:
--

Hi Marco, I do not know the answer to your question. Would you be interested in 
investigating this?

If this is really a matter of running setup.py in the way you mentioned before 
running pydoc then it should be noted in the README. Another question I have 
is, what is the standard for other python projects using pydoc?

Finally we need to have a test, maybe a post commit shell script that verifies 
that pydoc continues to work. (This could be a follow up issue and does not 
need to be part of this.)

> pydoc is not working
> 
>
> Key: BEAM-693
> URL: https://issues.apache.org/jira/browse/BEAM-693
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py
>Reporter: Ahmet Altay
>Priority: Minor
>
> Repro:
> Start the pydoc server (pydoc -p ) and navigate to the apache_beam root:
> http://localhost:/apache_beam.html
> Following errors are shown instead of the actual documentation:
> problem in apache_beam - : No module named avro
> problem in apache_beam - : cannot import name 
> coders



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BEAM-806) Maven Release Plugin Does Not Set Archetype Versions

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15602943#comment-15602943
 ] 

ASF GitHub Bot commented on BEAM-806:
-

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-beam/pull/1169


> Maven Release Plugin Does Not Set Archetype Versions
> 
>
> Key: BEAM-806
> URL: https://issues.apache.org/jira/browse/BEAM-806
> Project: Beam
>  Issue Type: Improvement
>  Components: build-system
>Affects Versions: 0.1.0-incubating, 0.2.0-incubating, 0.3.0-incubating
>Reporter: Aljoscha Krettek
>Priority: Blocker
> Fix For: 0.3.0-incubating
>
>
> When running {{mvn release:prepare}} as described in the new release guide 
> this does not update the version of the poms in the archetypes. To be clear, 
> the version of the archetype pom is updated, the pom in 
> {{archetype-resources}} (the pom of the project generated by the archetype) 
> is not updated.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[2/2] incubator-beam git commit: Closes #1169

[dhalperi]

Closes #1169


Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/dfaf2a89
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/dfaf2a89
Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/dfaf2a89

Branch: refs/heads/master
Commit: dfaf2a896029aac62d9db1f141206d42209d
Parents: b437c6b 5d01184
Author: Dan Halperin 
Authored: Mon Oct 24 12:22:29 2016 -0700
Committer: Dan Halperin 
Committed: Mon Oct 24 12:22:29 2016 -0700

--
 .../examples/src/main/resources/archetype-resources/pom.xml  | 8 
 .../starter/src/main/resources/archetype-resources/pom.xml   | 2 +-
 .../src/test/resources/projects/basic/reference/pom.xml  | 2 +-
 3 files changed, 6 insertions(+), 6 deletions(-)
--

[GitHub] incubator-beam pull request #1169: [BEAM-806] update archetype poms in maste...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-beam/pull/1169


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[1/2] incubator-beam git commit: [BEAM-806] update archetype poms in master

[dhalperi]

Repository: incubator-beam
Updated Branches:
  refs/heads/master b437c6b9e -> dfaf2a896


[BEAM-806] update archetype poms in master

Per the linked JIRA, maven release:branch did not do the right thing.


Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/5d011847
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/5d011847
Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/5d011847

Branch: refs/heads/master
Commit: 5d011847a658ff1cbd1e4eb0e0c581ceecbfc2c0
Parents: b437c6b
Author: Dan Halperin 
Authored: Mon Oct 24 11:12:41 2016 -0700
Committer: Dan Halperin 
Committed: Mon Oct 24 11:12:41 2016 -0700

--
 .../examples/src/main/resources/archetype-resources/pom.xml  | 8 
 .../starter/src/main/resources/archetype-resources/pom.xml   | 2 +-
 .../src/test/resources/projects/basic/reference/pom.xml  | 2 +-
 3 files changed, 6 insertions(+), 6 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/5d011847/sdks/java/maven-archetypes/examples/src/main/resources/archetype-resources/pom.xml
--
diff --git 
a/sdks/java/maven-archetypes/examples/src/main/resources/archetype-resources/pom.xml
 
b/sdks/java/maven-archetypes/examples/src/main/resources/archetype-resources/pom.xml
index 147c2dc..7926578 100644
--- 
a/sdks/java/maven-archetypes/examples/src/main/resources/archetype-resources/pom.xml
+++ 
b/sdks/java/maven-archetypes/examples/src/main/resources/archetype-resources/pom.xml
@@ -108,21 +108,21 @@
 
   org.apache.beam
   beam-sdks-java-core
-  0.3.0-incubating-SNAPSHOT
+  0.4.0-incubating-SNAPSHOT
 
 
 
 
   org.apache.beam
   beam-runners-direct-java
-  0.3.0-incubating-SNAPSHOT
+  0.4.0-incubating-SNAPSHOT
   runtime
 
 
 
   org.apache.beam
   beam-runners-google-cloud-dataflow-java
-  0.3.0-incubating-SNAPSHOT
+  0.4.0-incubating-SNAPSHOT
   runtime
 
 
@@ -130,7 +130,7 @@
 
   org.apache.beam
   beam-sdks-java-io-google-cloud-platform
-  0.3.0-incubating-SNAPSHOT
+  0.4.0-incubating-SNAPSHOT
 
 
 

http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/5d011847/sdks/java/maven-archetypes/starter/src/main/resources/archetype-resources/pom.xml
--
diff --git 
a/sdks/java/maven-archetypes/starter/src/main/resources/archetype-resources/pom.xml
 
b/sdks/java/maven-archetypes/starter/src/main/resources/archetype-resources/pom.xml
index 0f7d3de..5287d1f 100644
--- 
a/sdks/java/maven-archetypes/starter/src/main/resources/archetype-resources/pom.xml
+++ 
b/sdks/java/maven-archetypes/starter/src/main/resources/archetype-resources/pom.xml
@@ -55,7 +55,7 @@
 
   org.apache.beam
   beam-sdks-java-core
-  0.3.0-incubating-SNAPSHOT
+  0.4.0-incubating-SNAPSHOT
 
 
 

http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/5d011847/sdks/java/maven-archetypes/starter/src/test/resources/projects/basic/reference/pom.xml
--
diff --git 
a/sdks/java/maven-archetypes/starter/src/test/resources/projects/basic/reference/pom.xml
 
b/sdks/java/maven-archetypes/starter/src/test/resources/projects/basic/reference/pom.xml
index 66091fe..cce2f99 100644
--- 
a/sdks/java/maven-archetypes/starter/src/test/resources/projects/basic/reference/pom.xml
+++ 
b/sdks/java/maven-archetypes/starter/src/test/resources/projects/basic/reference/pom.xml
@@ -55,7 +55,7 @@
 
   org.apache.beam
   beam-sdks-java-core
-  0.3.0-incubating-SNAPSHOT
+  0.4.0-incubating-SNAPSHOT
 
 
 

incubator-beam git commit: [maven-release-plugin] prepare release v0.3.0-incubating-RC1

[aljoscha]

Repository: incubator-beam
Updated Branches:
  refs/heads/release-0.3.0-incubating e2d8a62e3 -> 564bbcc4b


[maven-release-plugin] prepare release v0.3.0-incubating-RC1


Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/564bbcc4
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/564bbcc4
Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/564bbcc4

Branch: refs/heads/release-0.3.0-incubating
Commit: 564bbcc4bc0ef12933302fa3e63c6942d4ee94dc
Parents: e2d8a62
Author: Aljoscha Krettek 
Authored: Mon Oct 24 21:22:02 2016 +0200
Committer: Aljoscha Krettek 
Committed: Mon Oct 24 21:22:02 2016 +0200

--
 examples/java/pom.xml   | 2 +-
 examples/java8/pom.xml  | 2 +-
 examples/pom.xml| 2 +-
 pom.xml | 4 ++--
 runners/core-java/pom.xml   | 2 +-
 runners/direct-java/pom.xml | 2 +-
 runners/flink/examples/pom.xml  | 2 +-
 runners/flink/pom.xml   | 2 +-
 runners/flink/runner/pom.xml| 2 +-
 runners/google-cloud-dataflow-java/pom.xml  | 2 +-
 runners/pom.xml | 2 +-
 runners/spark/pom.xml   | 2 +-
 sdks/java/build-tools/pom.xml   | 2 +-
 sdks/java/core/pom.xml  | 2 +-
 sdks/java/extensions/join-library/pom.xml   | 2 +-
 sdks/java/extensions/pom.xml| 2 +-
 sdks/java/io/google-cloud-platform/pom.xml  | 2 +-
 sdks/java/io/hdfs/pom.xml   | 2 +-
 sdks/java/io/jdbc/pom.xml   | 2 +-
 sdks/java/io/jms/pom.xml| 2 +-
 sdks/java/io/kafka/pom.xml  | 2 +-
 sdks/java/io/kinesis/pom.xml| 2 +-
 sdks/java/io/mongodb/pom.xml| 2 +-
 sdks/java/io/pom.xml| 2 +-
 sdks/java/java8tests/pom.xml| 2 +-
 sdks/java/maven-archetypes/examples/pom.xml | 2 +-
 sdks/java/maven-archetypes/pom.xml  | 2 +-
 sdks/java/maven-archetypes/starter/pom.xml  | 2 +-
 sdks/java/microbenchmarks/pom.xml   | 2 +-
 sdks/java/pom.xml   | 2 +-
 sdks/pom.xml| 2 +-
 31 files changed, 32 insertions(+), 32 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/564bbcc4/examples/java/pom.xml
--
diff --git a/examples/java/pom.xml b/examples/java/pom.xml
index 37cb15a..8b2f371 100644
--- a/examples/java/pom.xml
+++ b/examples/java/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.beam
 beam-examples-parent
-0.3.0-incubating-SNAPSHOT
+0.3.0-incubating
 ../pom.xml
   
 

http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/564bbcc4/examples/java8/pom.xml
--
diff --git a/examples/java8/pom.xml b/examples/java8/pom.xml
index 44cf918..70073fa 100644
--- a/examples/java8/pom.xml
+++ b/examples/java8/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.beam
 beam-examples-parent
-0.3.0-incubating-SNAPSHOT
+0.3.0-incubating
 ../pom.xml
   
 

http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/564bbcc4/examples/pom.xml
--
diff --git a/examples/pom.xml b/examples/pom.xml
index 2efb146..c10b743 100644
--- a/examples/pom.xml
+++ b/examples/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.beam
 beam-parent
-0.3.0-incubating-SNAPSHOT
+0.3.0-incubating
 ../pom.xml
   
 

http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/564bbcc4/pom.xml
--
diff --git a/pom.xml b/pom.xml
index 6a24d98..3836205 100644
--- a/pom.xml
+++ b/pom.xml
@@ -34,7 +34,7 @@
   http://beam.incubator.apache.org
   2016
 
-  0.3.0-incubating-SNAPSHOT
+  0.3.0-incubating
 
   
 
@@ -48,7 +48,7 @@
 
scm:git:https://git-wip-us.apache.org/repos/asf/incubator-beam.git
 
scm:git:https://git-wip-us.apache.org/repos/asf/incubator-beam.git
 
https://git-wip-us.apache.org/repos/asf?p=incubator-beam.git;a=summary
-release-0.3.0-incubating
+v0.3.0-incubating-RC1
   
 
   

http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/564bbcc4/runners/core-java/pom.xml
--
diff --git a/runners/core-java/pom.xml b/runners/core-java/pom.xml
index d84c420..b4799eb 100644
--- a/runners/core-java/pom.xml
+++ b/runners/core-java/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.beam
 beam-runners-parent
-0.3.0-incubating-SNAPSHOT
+0.3.0-incubating
 ../pom.xml
   
 

http://gi

[jira] [Commented] (BEAM-696) Side-Inputs non-deterministic with merging main-input windows

[Aljoscha Krettek (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15602937#comment-15602937
 ] 

Aljoscha Krettek commented on BEAM-696:
---

Just a clarification, the Flink runner does not pre combine when we have 
merging windows AND side inputs. If you just have merging windows then these 
are incrementally combined (you said pre-combined). If you have side inputs and 
a non-merging window function then we also incrementally combine (pre-combine).

> Side-Inputs non-deterministic with merging main-input windows
> -
>
> Key: BEAM-696
> URL: https://issues.apache.org/jira/browse/BEAM-696
> Project: Beam
>  Issue Type: Bug
>  Components: beam-model
>Reporter: Ben Chambers
>Assignee: Pei He
>
> Side-Inputs are non-deterministic for several reasons:
> 1. Because they depend on triggering of the side-input (this is acceptable 
> because triggers are by their nature non-deterministic).
> 2. They depend on the current state of the main-input window in order to 
> lookup the side-input. This means that with merging
> 3. Any runner optimizations that affect when the side-input is looked up may 
> cause problems with either or both of these.
> This issue focuses on #2 -- the non-determinism of side-inputs that execute 
> within a Merging WindowFn.
> Possible solution would be to defer running anything that looks up the 
> side-input until we need to extract an output, and using the main-window at 
> that point. Specifically, if the main-window is a MergingWindowFn, don't 
> execute any kind of pre-combine, instead buffer all the inputs and combine 
> later.
> This could still run into some non-determinism if there are triggers 
> controlling when we extract output.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (BEAM-809) Create a KryoRegistrator for the SparkRunner.

[Amit Sela (JIRA)]

Amit Sela created BEAM-809:
--

 Summary: Create a KryoRegistrator for the SparkRunner. 
 Key: BEAM-809
 URL: https://issues.apache.org/jira/browse/BEAM-809
 Project: Beam
  Issue Type: Bug
  Components: runner-spark
Reporter: Amit Sela
Assignee: Amit Sela


Create a custom KryoRegistrator for the Spark runner's needs - mostly immutable 
collections.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

incubator-beam git commit: Manually change archetype versions to 0.3.0-incubating [Forced Update!]

[aljoscha]

Repository: incubator-beam
Updated Branches:
  refs/heads/release-0.3.0-incubating 08c276b55 -> e2d8a62e3 (forced update)


Manually change archetype versions to 0.3.0-incubating


Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/e2d8a62e
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/e2d8a62e
Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/e2d8a62e

Branch: refs/heads/release-0.3.0-incubating
Commit: e2d8a62e38e8f24b97b6b9b53bcbb635d8c34d33
Parents: c26b63b
Author: Aljoscha Krettek 
Authored: Mon Oct 24 21:04:37 2016 +0200
Committer: Aljoscha Krettek 
Committed: Mon Oct 24 21:04:37 2016 +0200

--
 .../examples/src/main/resources/archetype-resources/pom.xml  | 8 
 .../starter/src/main/resources/archetype-resources/pom.xml   | 2 +-
 .../src/test/resources/projects/basic/reference/pom.xml  | 2 +-
 3 files changed, 6 insertions(+), 6 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/e2d8a62e/sdks/java/maven-archetypes/examples/src/main/resources/archetype-resources/pom.xml
--
diff --git 
a/sdks/java/maven-archetypes/examples/src/main/resources/archetype-resources/pom.xml
 
b/sdks/java/maven-archetypes/examples/src/main/resources/archetype-resources/pom.xml
index 147c2dc..c79585c 100644
--- 
a/sdks/java/maven-archetypes/examples/src/main/resources/archetype-resources/pom.xml
+++ 
b/sdks/java/maven-archetypes/examples/src/main/resources/archetype-resources/pom.xml
@@ -108,21 +108,21 @@
 
   org.apache.beam
   beam-sdks-java-core
-  0.3.0-incubating-SNAPSHOT
+  0.3.0-incubating
 
 
 
 
   org.apache.beam
   beam-runners-direct-java
-  0.3.0-incubating-SNAPSHOT
+  0.3.0-incubating
   runtime
 
 
 
   org.apache.beam
   beam-runners-google-cloud-dataflow-java
-  0.3.0-incubating-SNAPSHOT
+  0.3.0-incubating
   runtime
 
 
@@ -130,7 +130,7 @@
 
   org.apache.beam
   beam-sdks-java-io-google-cloud-platform
-  0.3.0-incubating-SNAPSHOT
+  0.3.0-incubating
 
 
 

http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/e2d8a62e/sdks/java/maven-archetypes/starter/src/main/resources/archetype-resources/pom.xml
--
diff --git 
a/sdks/java/maven-archetypes/starter/src/main/resources/archetype-resources/pom.xml
 
b/sdks/java/maven-archetypes/starter/src/main/resources/archetype-resources/pom.xml
index 0f7d3de..c0244ff 100644
--- 
a/sdks/java/maven-archetypes/starter/src/main/resources/archetype-resources/pom.xml
+++ 
b/sdks/java/maven-archetypes/starter/src/main/resources/archetype-resources/pom.xml
@@ -55,7 +55,7 @@
 
   org.apache.beam
   beam-sdks-java-core
-  0.3.0-incubating-SNAPSHOT
+  0.3.0-incubating
 
 
 

http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/e2d8a62e/sdks/java/maven-archetypes/starter/src/test/resources/projects/basic/reference/pom.xml
--
diff --git 
a/sdks/java/maven-archetypes/starter/src/test/resources/projects/basic/reference/pom.xml
 
b/sdks/java/maven-archetypes/starter/src/test/resources/projects/basic/reference/pom.xml
index 66091fe..69d5c11 100644
--- 
a/sdks/java/maven-archetypes/starter/src/test/resources/projects/basic/reference/pom.xml
+++ 
b/sdks/java/maven-archetypes/starter/src/test/resources/projects/basic/reference/pom.xml
@@ -55,7 +55,7 @@
 
   org.apache.beam
   beam-sdks-java-core
-  0.3.0-incubating-SNAPSHOT
+  0.3.0-incubating
 
 
 

[jira] [Commented] (BEAM-808) Increase "spark.port.maxRetries" to avoid BindException in ROS.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15602881#comment-15602881
 ] 

ASF GitHub Bot commented on BEAM-808:
-

GitHub user amitsela opened a pull request:

https://github.com/apache/incubator-beam/pull/1170

[BEAM-808] Increase "spark.port.maxRetries" to avoid BindException in…

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

 - [ ] Make sure the PR title is formatted like:
   `[BEAM-] Description of pull request`
 - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable
   Travis-CI on your fork and ensure the whole test matrix passes).
 - [ ] Replace `` in the title with the actual Jira issue
   number, if there is one.
 - [ ] If this contribution is large, please file an Apache
   [Individual Contributor License 
Agreement](https://www.apache.org/licenses/icla.txt).

---

… ROS.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/amitsela/incubator-beam BEAM-808

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-beam/pull/1170.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1170


commit 8f930b2fb97a377c6c168a33ca415a3eec7d41f4
Author: Sela 
Date:   2016-10-24T18:52:03Z

[BEAM-808] Increase "spark.port.maxRetries" to avoid BindException in ROS.




> Increase "spark.port.maxRetries" to avoid BindException in ROS.
> ---
>
> Key: BEAM-808
> URL: https://issues.apache.org/jira/browse/BEAM-808
> Project: Beam
>  Issue Type: Improvement
>  Components: runner-spark
>Reporter: Amit Sela
>Assignee: Amit Sela
>
> The default is 16, and it's hard to know which port failed to bind since 
> there's no logging.
> Let's start with 64.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-beam pull request #1170: [BEAM-808] Increase "spark.port.maxRetrie...

[amitsela]

GitHub user amitsela opened a pull request:

https://github.com/apache/incubator-beam/pull/1170

[BEAM-808] Increase "spark.port.maxRetries" to avoid BindException in…

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

 - [ ] Make sure the PR title is formatted like:
   `[BEAM-] Description of pull request`
 - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable
   Travis-CI on your fork and ensure the whole test matrix passes).
 - [ ] Replace `` in the title with the actual Jira issue
   number, if there is one.
 - [ ] If this contribution is large, please file an Apache
   [Individual Contributor License 
Agreement](https://www.apache.org/licenses/icla.txt).

---

… ROS.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/amitsela/incubator-beam BEAM-808

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-beam/pull/1170.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1170


commit 8f930b2fb97a377c6c168a33ca415a3eec7d41f4
Author: Sela 
Date:   2016-10-24T18:52:03Z

[BEAM-808] Increase "spark.port.maxRetries" to avoid BindException in ROS.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (BEAM-808) Increase "spark.port.maxRetries" to avoid BindException in ROS.

[Amit Sela (JIRA)]

Amit Sela created BEAM-808:
--

 Summary: Increase "spark.port.maxRetries" to avoid BindException 
in ROS.
 Key: BEAM-808
 URL: https://issues.apache.org/jira/browse/BEAM-808
 Project: Beam
  Issue Type: Improvement
  Components: runner-spark
Reporter: Amit Sela
Assignee: Amit Sela


The default is 16, and it's hard to know which port failed to bind since 
there's no logging.
Let's start with 64.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (BEAM-807) [SparkRunner] Replace OldDoFn with DoFn

[Amit Sela (JIRA)]

Amit Sela created BEAM-807:
--

 Summary: [SparkRunner] Replace OldDoFn with DoFn
 Key: BEAM-807
 URL: https://issues.apache.org/jira/browse/BEAM-807
 Project: Beam
  Issue Type: Improvement
  Components: runner-spark
Reporter: Amit Sela
Assignee: Amit Sela






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BEAM-800) DoFnLifeCycleManager should hold DoFnInvoker, not DoFn/OldDoFn

[Kenneth Knowles (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15602848#comment-15602848
 ] 

Kenneth Knowles commented on BEAM-800:
--

The porting of the DirectRunner occured on 
[#1157|https://github.com/apache/incubator-beam/pull/1157], with lots of little 
fixes to make it all ready. The commit that actually ported the runner is 
[1919d8b|https://github.com/apache/incubator-beam/pull/1157/commits/1919d8b3a850bd146137652546da851ee461cd28].
 If you take a look, the {{DoFnLifecycleManager}} caches each {{DoFn}} 
instance, and tears it down, but the repeated instantiation of a 
{{DoFnInvoker}} for it is redundant. It is possible that even the 
{{DoFnRunner}} could be cached here (a {{DoFnRunner}} is essentially a 
{{DoFnInvoker}} fixed to one particular context).

> DoFnLifeCycleManager should hold DoFnInvoker, not DoFn/OldDoFn
> --
>
> Key: BEAM-800
> URL: https://issues.apache.org/jira/browse/BEAM-800
> Project: Beam
>  Issue Type: Bug
>  Components: runner-direct
>Reporter: Kenneth Knowles
>Priority: Minor
>
> The {{DirectRunner}}'s support class {{DoFnLifecycleManager}} holds a cache 
> of deserialized {{OldDoFn}} s, now being ported to {{DoFn}} s. But to execute 
> a {{DoFn}} there is another layer of indirection through a {{DoFnInvoker}}, 
> which is the best object to cache here.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Jenkins build is still unstable: beam_PostCommit_RunnableOnService_FlinkLocal #654

[Apache Jenkins Server]

See 

[jira] [Commented] (BEAM-806) Maven Release Plugin Does Not Set Archetype Versions

[Daniel Halperin (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15602815#comment-15602815
 ] 

Daniel Halperin commented on BEAM-806:
--

Hmm, we used to use a range but no longer do. 

https://github.com/apache/incubator-beam/commit/1f30255edcdd9c1e445b69248191c8552724f086#diff-4795b1d27449c01332aad192348eL111


> Maven Release Plugin Does Not Set Archetype Versions
> 
>
> Key: BEAM-806
> URL: https://issues.apache.org/jira/browse/BEAM-806
> Project: Beam
>  Issue Type: Improvement
>  Components: build-system
>Affects Versions: 0.1.0-incubating, 0.2.0-incubating, 0.3.0-incubating
>Reporter: Aljoscha Krettek
>Priority: Blocker
> Fix For: 0.3.0-incubating
>
>
> When running {{mvn release:prepare}} as described in the new release guide 
> this does not update the version of the poms in the archetypes. To be clear, 
> the version of the archetype pom is updated, the pom in 
> {{archetype-resources}} (the pom of the project generated by the archetype) 
> is not updated.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BEAM-757) The SparkRunner should utilize the SDK's DoFnRunner instead of writing it's own.

[Kenneth Knowles (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15602812#comment-15602812
 ] 

Kenneth Knowles commented on BEAM-757:
--

For the {{DoFn}} switch, you could open a new issue for SparkRunner or just 
reference BEAM-498. I suspect it is one hopefully-easy PR.

> The SparkRunner should utilize the SDK's DoFnRunner instead of writing it's 
> own.
> 
>
> Key: BEAM-757
> URL: https://issues.apache.org/jira/browse/BEAM-757
> Project: Beam
>  Issue Type: Improvement
>  Components: runner-spark
>Reporter: Amit Sela
>Assignee: Amit Sela
>
> The SDK now provides DoFnRunner implementations, and so to avoid maintaining 
> against the SDK, the runner should leverage the runner API instead.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BEAM-806) Maven Release Plugin Does Not Set Archetype Versions

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15602763#comment-15602763
 ] 

ASF GitHub Bot commented on BEAM-806:
-

GitHub user dhalperi opened a pull request:

https://github.com/apache/incubator-beam/pull/1169

[BEAM-806] update archetype poms in master

Per the linked JIRA, maven release:branch did not do the right thing.

R: @aljoscha 
CC: @davorbonaci 

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/dhalperi/incubator-beam beam-806

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-beam/pull/1169.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1169


commit 5d011847a658ff1cbd1e4eb0e0c581ceecbfc2c0
Author: Dan Halperin 
Date:   2016-10-24T18:12:41Z

[BEAM-806] update archetype poms in master

Per the linked JIRA, maven release:branch did not do the right thing.




> Maven Release Plugin Does Not Set Archetype Versions
> 
>
> Key: BEAM-806
> URL: https://issues.apache.org/jira/browse/BEAM-806
> Project: Beam
>  Issue Type: Improvement
>  Components: build-system
>Affects Versions: 0.1.0-incubating, 0.2.0-incubating, 0.3.0-incubating
>Reporter: Aljoscha Krettek
>Priority: Blocker
> Fix For: 0.3.0-incubating
>
>
> When running {{mvn release:prepare}} as described in the new release guide 
> this does not update the version of the poms in the archetypes. To be clear, 
> the version of the archetype pom is updated, the pom in 
> {{archetype-resources}} (the pom of the project generated by the archetype) 
> is not updated.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BEAM-806) Maven Release Plugin Does Not Set Archetype Versions

[Daniel Halperin (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15602768#comment-15602768
 ] 

Daniel Halperin commented on BEAM-806:
--

The PR I sent just fixes master.

Some crappy workarounds:

* Make the version be {{LATEST}} instead.
* Make the version be a bigger range like {{[0.0.0, 1.0.0)}}. That assumes we 
have a stable API, which we do not. :)

> Maven Release Plugin Does Not Set Archetype Versions
> 
>
> Key: BEAM-806
> URL: https://issues.apache.org/jira/browse/BEAM-806
> Project: Beam
>  Issue Type: Improvement
>  Components: build-system
>Affects Versions: 0.1.0-incubating, 0.2.0-incubating, 0.3.0-incubating
>Reporter: Aljoscha Krettek
>Priority: Blocker
> Fix For: 0.3.0-incubating
>
>
> When running {{mvn release:prepare}} as described in the new release guide 
> this does not update the version of the poms in the archetypes. To be clear, 
> the version of the archetype pom is updated, the pom in 
> {{archetype-resources}} (the pom of the project generated by the archetype) 
> is not updated.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-beam pull request #1169: [BEAM-806] update archetype poms in maste...

[dhalperi]

GitHub user dhalperi opened a pull request:

https://github.com/apache/incubator-beam/pull/1169

[BEAM-806] update archetype poms in master

Per the linked JIRA, maven release:branch did not do the right thing.

R: @aljoscha 
CC: @davorbonaci 

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/dhalperi/incubator-beam beam-806

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-beam/pull/1169.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1169


commit 5d011847a658ff1cbd1e4eb0e0c581ceecbfc2c0
Author: Dan Halperin 
Date:   2016-10-24T18:12:41Z

[BEAM-806] update archetype poms in master

Per the linked JIRA, maven release:branch did not do the right thing.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---