[jira] [Commented] (BEAM-1316) DoFn#startBundle should not be able to output
[
https://issues.apache.org/jira/browse/BEAM-1316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15997560#comment-15997560
]
ASF GitHub Bot commented on BEAM-1316:
--
Github user asfgit closed the pull request at:
https://github.com/apache/beam/pull/2695
> DoFn#startBundle should not be able to output
> -
>
> Key: BEAM-1316
> URL: https://issues.apache.org/jira/browse/BEAM-1316
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Thomas Groh
>Assignee: Thomas Groh
> Labels: backward-incompatible
> Fix For: First stable release
>
>
> While within startBundle and finishBundle, the window in which elements are
> output is not generally defined. Elements must always be output from within a
> windowed context, or the {{WindowFn}} used by the {{PCollection}} may not
> operate appropriately.
> startBundle and finishBundle are suitable for operational duties, similarly
> to {{setup}} and {{teardown}}, but within the scope of some collection of
> input elements. This includes actions such as clearing field state within a
> DoFn and ensuring all live RPCs complete successfully before committing
> inputs.
> Sometimes it might be reasonable to output from {{@FinishBundle}} but it is
> hard to imagine a situation where output from {{@StartBundle}} is useful in a
> way that doesn't seriously abuse things.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (BEAM-1316) DoFn#startBundle should not be able to output
[
https://issues.apache.org/jira/browse/BEAM-1316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15993732#comment-15993732
]
ASF GitHub Bot commented on BEAM-1316:
--
Github user asfgit closed the pull request at:
https://github.com/apache/beam/pull/2836
> DoFn#startBundle should not be able to output
> -
>
> Key: BEAM-1316
> URL: https://issues.apache.org/jira/browse/BEAM-1316
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Thomas Groh
>Assignee: Thomas Groh
> Fix For: First stable release
>
>
> While within startBundle and finishBundle, the window in which elements are
> output is not generally defined. Elements must always be output from within a
> windowed context, or the {{WindowFn}} used by the {{PCollection}} may not
> operate appropriately.
> startBundle and finishBundle are suitable for operational duties, similarly
> to {{setup}} and {{teardown}}, but within the scope of some collection of
> input elements. This includes actions such as clearing field state within a
> DoFn and ensuring all live RPCs complete successfully before committing
> inputs.
> Sometimes it might be reasonable to output from {{@FinishBundle}} but it is
> hard to imagine a situation where output from {{@StartBundle}} is useful in a
> way that doesn't seriously abuse things.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (BEAM-1316) DoFn#startBundle should not be able to output
[
https://issues.apache.org/jira/browse/BEAM-1316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15993498#comment-15993498
]
ASF GitHub Bot commented on BEAM-1316:
--
GitHub user sb2nov opened a pull request:
https://github.com/apache/beam/pull/2836
[BEAM-1316] Remove the usage of mock from ptransform tests
Be sure to do all of the following to help us incorporate your contribution
quickly and easily:
- [ ] Make sure the PR title is formatted like:
`[BEAM-] Description of pull request`
- [ ] Make sure tests pass via `mvn clean verify`.
- [ ] Replace `` in the title with the actual Jira issue
number, if there is one.
- [ ] If this contribution is large, please file an Apache
[Individual Contributor License
Agreement](https://www.apache.org/licenses/icla.pdf).
---
Mock patch creates problems if we try to pickle the DoFn so remove it and
clean up the tests
R: @aaltay PTAL
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/sb2nov/beam BEAM-1316-improve-ptransform-test
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/beam/pull/2836.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2836
commit 0eeb33bd1c41d1fe19ca71ce2b54485a9d7b717a
Author: Sourabh Bajaj
Date: 2017-05-02T18:48:19Z
[BEAM-1316] Remove the usage of mock from ptransform tests
> DoFn#startBundle should not be able to output
> -
>
> Key: BEAM-1316
> URL: https://issues.apache.org/jira/browse/BEAM-1316
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Thomas Groh
>Assignee: Thomas Groh
> Fix For: First stable release
>
>
> While within startBundle and finishBundle, the window in which elements are
> output is not generally defined. Elements must always be output from within a
> windowed context, or the {{WindowFn}} used by the {{PCollection}} may not
> operate appropriately.
> startBundle and finishBundle are suitable for operational duties, similarly
> to {{setup}} and {{teardown}}, but within the scope of some collection of
> input elements. This includes actions such as clearing field state within a
> DoFn and ensuring all live RPCs complete successfully before committing
> inputs.
> Sometimes it might be reasonable to output from {{@FinishBundle}} but it is
> hard to imagine a situation where output from {{@StartBundle}} is useful in a
> way that doesn't seriously abuse things.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (BEAM-1316) DoFn#startBundle should not be able to output
[
https://issues.apache.org/jira/browse/BEAM-1316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15993437#comment-15993437
]
Sourabh Bajaj commented on BEAM-1316:
-
[~altay] done.
> DoFn#startBundle should not be able to output
> -
>
> Key: BEAM-1316
> URL: https://issues.apache.org/jira/browse/BEAM-1316
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Thomas Groh
>Assignee: Thomas Groh
> Fix For: First stable release
>
>
> While within startBundle and finishBundle, the window in which elements are
> output is not generally defined. Elements must always be output from within a
> windowed context, or the {{WindowFn}} used by the {{PCollection}} may not
> operate appropriately.
> startBundle and finishBundle are suitable for operational duties, similarly
> to {{setup}} and {{teardown}}, but within the scope of some collection of
> input elements. This includes actions such as clearing field state within a
> DoFn and ensuring all live RPCs complete successfully before committing
> inputs.
> Sometimes it might be reasonable to output from {{@FinishBundle}} but it is
> hard to imagine a situation where output from {{@StartBundle}} is useful in a
> way that doesn't seriously abuse things.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (BEAM-1316) DoFn#startBundle should not be able to output
[
https://issues.apache.org/jira/browse/BEAM-1316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15993294#comment-15993294
]
Ahmet Altay commented on BEAM-1316:
---
[~sb2nov] could we remove the sdk-py component from this issues if it is fixed
for python? (for tracking purposes.)
> DoFn#startBundle should not be able to output
> -
>
> Key: BEAM-1316
> URL: https://issues.apache.org/jira/browse/BEAM-1316
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core, sdk-py
>Reporter: Thomas Groh
>Assignee: Thomas Groh
> Fix For: First stable release
>
>
> While within startBundle and finishBundle, the window in which elements are
> output is not generally defined. Elements must always be output from within a
> windowed context, or the {{WindowFn}} used by the {{PCollection}} may not
> operate appropriately.
> startBundle and finishBundle are suitable for operational duties, similarly
> to {{setup}} and {{teardown}}, but within the scope of some collection of
> input elements. This includes actions such as clearing field state within a
> DoFn and ensuring all live RPCs complete successfully before committing
> inputs.
> Sometimes it might be reasonable to output from {{@FinishBundle}} but it is
> hard to imagine a situation where output from {{@StartBundle}} is useful in a
> way that doesn't seriously abuse things.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (BEAM-1316) DoFn#startBundle should not be able to output
[
https://issues.apache.org/jira/browse/BEAM-1316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15985830#comment-15985830
]
ASF GitHub Bot commented on BEAM-1316:
--
Github user asfgit closed the pull request at:
https://github.com/apache/beam/pull/2715
> DoFn#startBundle should not be able to output
> -
>
> Key: BEAM-1316
> URL: https://issues.apache.org/jira/browse/BEAM-1316
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core, sdk-py
>Reporter: Thomas Groh
>Assignee: Thomas Groh
> Fix For: First stable release
>
>
> While within startBundle and finishBundle, the window in which elements are
> output is not generally defined. Elements must always be output from within a
> windowed context, or the {{WindowFn}} used by the {{PCollection}} may not
> operate appropriately.
> startBundle and finishBundle are suitable for operational duties, similarly
> to {{setup}} and {{teardown}}, but within the scope of some collection of
> input elements. This includes actions such as clearing field state within a
> DoFn and ensuring all live RPCs complete successfully before committing
> inputs.
> Sometimes it might be reasonable to output from {{@FinishBundle}} but it is
> hard to imagine a situation where output from {{@StartBundle}} is useful in a
> way that doesn't seriously abuse things.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (BEAM-1316) DoFn#startBundle should not be able to output
[
https://issues.apache.org/jira/browse/BEAM-1316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15985648#comment-15985648
]
ASF GitHub Bot commented on BEAM-1316:
--
Github user sb2nov closed the pull request at:
https://github.com/apache/beam/pull/2715
> DoFn#startBundle should not be able to output
> -
>
> Key: BEAM-1316
> URL: https://issues.apache.org/jira/browse/BEAM-1316
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core, sdk-py
>Reporter: Thomas Groh
>Assignee: Thomas Groh
> Fix For: First stable release
>
>
> While within startBundle and finishBundle, the window in which elements are
> output is not generally defined. Elements must always be output from within a
> windowed context, or the {{WindowFn}} used by the {{PCollection}} may not
> operate appropriately.
> startBundle and finishBundle are suitable for operational duties, similarly
> to {{setup}} and {{teardown}}, but within the scope of some collection of
> input elements. This includes actions such as clearing field state within a
> DoFn and ensuring all live RPCs complete successfully before committing
> inputs.
> Sometimes it might be reasonable to output from {{@FinishBundle}} but it is
> hard to imagine a situation where output from {{@StartBundle}} is useful in a
> way that doesn't seriously abuse things.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (BEAM-1316) DoFn#startBundle should not be able to output
[
https://issues.apache.org/jira/browse/BEAM-1316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15985649#comment-15985649
]
ASF GitHub Bot commented on BEAM-1316:
--
GitHub user sb2nov reopened a pull request:
https://github.com/apache/beam/pull/2715
[BEAM-1316] start bundle should not output any elements
Be sure to do all of the following to help us incorporate your contribution
quickly and easily:
- [ ] Make sure the PR title is formatted like:
`[BEAM-] Description of pull request`
- [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable
Travis-CI on your fork and ensure the whole test matrix passes).
- [ ] Replace `` in the title with the actual Jira issue
number, if there is one.
- [ ] If this contribution is large, please file an Apache
[Individual Contributor License
Agreement](https://www.apache.org/licenses/icla.pdf).
---
R: @chamikaramj @robertwb PTAL
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/sb2nov/beam BEAM-1316-start-bundle-no-output
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/beam/pull/2715.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2715
commit 007945fb2ff9e3cd4332b1d93bd375005aba5f7d
Author: Sourabh Bajaj
Date: 2017-04-26T22:09:03Z
[BEAM-1316] start bundle should not output any elements
> DoFn#startBundle should not be able to output
> -
>
> Key: BEAM-1316
> URL: https://issues.apache.org/jira/browse/BEAM-1316
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core, sdk-py
>Reporter: Thomas Groh
>Assignee: Thomas Groh
> Fix For: First stable release
>
>
> While within startBundle and finishBundle, the window in which elements are
> output is not generally defined. Elements must always be output from within a
> windowed context, or the {{WindowFn}} used by the {{PCollection}} may not
> operate appropriately.
> startBundle and finishBundle are suitable for operational duties, similarly
> to {{setup}} and {{teardown}}, but within the scope of some collection of
> input elements. This includes actions such as clearing field state within a
> DoFn and ensuring all live RPCs complete successfully before committing
> inputs.
> Sometimes it might be reasonable to output from {{@FinishBundle}} but it is
> hard to imagine a situation where output from {{@StartBundle}} is useful in a
> way that doesn't seriously abuse things.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (BEAM-1316) DoFn#startBundle should not be able to output
[
https://issues.apache.org/jira/browse/BEAM-1316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15985631#comment-15985631
]
ASF GitHub Bot commented on BEAM-1316:
--
GitHub user sb2nov opened a pull request:
https://github.com/apache/beam/pull/2715
[BEAM-1316] start bundle should not output any elements
Be sure to do all of the following to help us incorporate your contribution
quickly and easily:
- [ ] Make sure the PR title is formatted like:
`[BEAM-] Description of pull request`
- [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable
Travis-CI on your fork and ensure the whole test matrix passes).
- [ ] Replace `` in the title with the actual Jira issue
number, if there is one.
- [ ] If this contribution is large, please file an Apache
[Individual Contributor License
Agreement](https://www.apache.org/licenses/icla.pdf).
---
R: @chamikaramj @robertwb PTAL
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/sb2nov/beam BEAM-1316-start-bundle-no-output
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/beam/pull/2715.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2715
> DoFn#startBundle should not be able to output
> -
>
> Key: BEAM-1316
> URL: https://issues.apache.org/jira/browse/BEAM-1316
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core, sdk-py
>Reporter: Thomas Groh
>Assignee: Thomas Groh
> Fix For: First stable release
>
>
> While within startBundle and finishBundle, the window in which elements are
> output is not generally defined. Elements must always be output from within a
> windowed context, or the {{WindowFn}} used by the {{PCollection}} may not
> operate appropriately.
> startBundle and finishBundle are suitable for operational duties, similarly
> to {{setup}} and {{teardown}}, but within the scope of some collection of
> input elements. This includes actions such as clearing field state within a
> DoFn and ensuring all live RPCs complete successfully before committing
> inputs.
> Sometimes it might be reasonable to output from {{@FinishBundle}} but it is
> hard to imagine a situation where output from {{@StartBundle}} is useful in a
> way that doesn't seriously abuse things.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (BEAM-1316) DoFn#startBundle should not be able to output
[
https://issues.apache.org/jira/browse/BEAM-1316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15983961#comment-15983961
]
ASF GitHub Bot commented on BEAM-1316:
--
GitHub user tgroh opened a pull request:
https://github.com/apache/beam/pull/2695
[BEAM-1316] Forbid output in StartBundle
Be sure to do all of the following to help us incorporate your contribution
quickly and easily:
- [ ] Make sure the PR title is formatted like:
`[BEAM-] Description of pull request`
- [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable
Travis-CI on your fork and ensure the whole test matrix passes).
- [ ] Replace `` in the title with the actual Jira issue
number, if there is one.
- [ ] If this contribution is large, please file an Apache
[Individual Contributor License
Agreement](https://www.apache.org/licenses/icla.pdf).
---
TODO: Add comments, audit any other context creation.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/tgroh/beam no_start_bundle_output
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/beam/pull/2695.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2695
commit 5d369da267405911e534e49ad74fdb64bc2c3150
Author: Thomas Groh
Date: 2017-04-26T01:20:21Z
Forbid output in StartBundle
> DoFn#startBundle should not be able to output
> -
>
> Key: BEAM-1316
> URL: https://issues.apache.org/jira/browse/BEAM-1316
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Thomas Groh
>Assignee: Thomas Groh
> Fix For: First stable release
>
>
> While within startBundle and finishBundle, the window in which elements are
> output is not generally defined. Elements must always be output from within a
> windowed context, or the {{WindowFn}} used by the {{PCollection}} may not
> operate appropriately.
> startBundle and finishBundle are suitable for operational duties, similarly
> to {{setup}} and {{teardown}}, but within the scope of some collection of
> input elements. This includes actions such as clearing field state within a
> DoFn and ensuring all live RPCs complete successfully before committing
> inputs.
> Sometimes it might be reasonable to output from {{@FinishBundle}} but it is
> hard to imagine a situation where output from {{@StartBundle}} is useful in a
> way that doesn't seriously abuse things.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (BEAM-1316) DoFn#startBundle should not be able to output
[
https://issues.apache.org/jira/browse/BEAM-1316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15937767#comment-15937767
]
Kenneth Knowles commented on BEAM-1316:
---
I'm going to abuse existing issues to describe this:
1. {{@StartBundle}} should not be able to output
2. New {{DoFn}} should have a way to output to a specific window
3. {{@FinishBundle}} should be required to specify the window
Otherwise we have a bunch of quasi-dupes.
> DoFn#startBundle should not be able to output
> -
>
> Key: BEAM-1316
> URL: https://issues.apache.org/jira/browse/BEAM-1316
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Thomas Groh
>
> While within startBundle and finishBundle, the window in which elements are
> output is not generally defined. Elements must always be output from within a
> windowed context, or the {{WindowFn}} used by the {{PCollection}} may not
> operate appropriately.
> startBundle and finishBundle are suitable for operational duties, similarly
> to {{setup}} and {{teardown}}, but within the scope of some collection of
> input elements. This includes actions such as clearing field state within a
> DoFn and ensuring all live RPCs complete successfully before committing
> inputs.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
