[jira] [Commented] (BEAM-1124) Python ValidateRunner Test test_multi_valued_singleton_side_input Break Postcommit

2018-07-25 Thread Mark Liu (JIRA) 


[ 
https://issues.apache.org/jira/browse/BEAM-1124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16556453#comment-16556453
 ] 

Mark Liu commented on BEAM-1124:


The fix should work. This test is passing for a while.

Latest green Jenkins run for VR test:
https://builds.apache.org/view/A-D/view/Beam/job/beam_PostCommit_Py_VR_Dataflow/642/

> Python ValidateRunner Test test_multi_valued_singleton_side_input Break 
> Postcommit
> --
>
> Key: BEAM-1124
> URL: https://issues.apache.org/jira/browse/BEAM-1124
> Project: Beam
>  Issue Type: Bug
>  Components: runner-dataflow, sdk-py-core, testing
>Reporter: Mark Liu
>Assignee: Ahmet Altay
>Priority: Major
> Fix For: Not applicable
>
>
> Python test_multi_valued_singleton_side_input test, a ValidatesRunner test 
> that running on dataflow service, failed and broke 
> postcommit(https://builds.apache.org/view/Beam/job/beam_PostCommit_Python_Verify/853/).
> Here is the stack trace:
> {code}
> Traceback (most recent call last):
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/sdks/python/apache_beam/dataflow_test.py",
>  line 186, in test_multi_valued_singleton_side_input
> pipeline.run()
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/sdks/python/apache_beam/pipeline.py",
>  line 159, in run
> return self.runner.run(self)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/sdks/python/apache_beam/runners/dataflow_runner.py",
>  line 195, in run
> % getattr(self, 'last_error_msg', None), self.result)
> DataflowRuntimeException: Dataflow pipeline failed:
> (99aeafa7a8dffcc7): Traceback (most recent call last):
>   File 
> "/usr/local/lib/python2.7/dist-packages/dataflow_worker/batchworker.py", line 
> 514, in do_work
> work_executor.execute()
>   File "dataflow_worker/executor.py", line 892, in 
> dataflow_worker.executor.MapTaskExecutor.execute 
> (dataflow_worker/executor.c:24008)
> op.start()
>   File "dataflow_worker/executor.py", line 456, in 
> dataflow_worker.executor.DoOperation.start (dataflow_worker/executor.c:13870)
> def start(self):
>   File "dataflow_worker/executor.py", line 483, in 
> dataflow_worker.executor.DoOperation.start (dataflow_worker/executor.c:13685)
> self.dofn_runner = common.DoFnRunner(
>   File "apache_beam/runners/common.py", line 89, in 
> apache_beam.runners.common.DoFnRunner.__init__ 
> (apache_beam/runners/common.c:3469)
> args, kwargs, [side_input[global_window]
>   File 
> "/usr/local/lib/python2.7/dist-packages/apache_beam/transforms/sideinputs.py",
>  line 192, in __getitem__
> _FilteringIterable(self._iterable, target_window), self._view_options)
>   File "/usr/local/lib/python2.7/dist-packages/apache_beam/pvalue.py", line 
> 279, in _from_runtime_iterable
> 'PCollection with more than one element accessed as '
> ValueError: PCollection with more than one element accessed as a singleton 
> view.
> {code}
> Worker logs in here:
> https://builds.apache.org/view/Beam/job/beam_PostCommit_Python_Verify/853/console
> In order to temporarily ignore this test in postcommit, we can comment out 
> annotation "@attr('ValidatesRunner')" of this test. Then it will only run as 
> a unit test (execute by DirectRunner), but not run as a ValidatesRunner test.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (BEAM-1124) Python ValidateRunner Test test_multi_valued_singleton_side_input Break Postcommit

2018-06-20 Thread Luke Cwik (JIRA) 


[ 
https://issues.apache.org/jira/browse/BEAM-1124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16518434#comment-16518434
 ] 

Luke Cwik commented on BEAM-1124:
-

I don't think this test should ever pass because we should error out if we ever 
see more then one value in a singleton side input.

> Python ValidateRunner Test test_multi_valued_singleton_side_input Break 
> Postcommit
> --
>
> Key: BEAM-1124
> URL: https://issues.apache.org/jira/browse/BEAM-1124
> Project: Beam
>  Issue Type: Bug
>  Components: runner-dataflow, sdk-py-core, testing
>Reporter: Mark Liu
>Assignee: Ahmet Altay
>Priority: Major
> Fix For: Not applicable
>
>
> Python test_multi_valued_singleton_side_input test, a ValidatesRunner test 
> that running on dataflow service, failed and broke 
> postcommit(https://builds.apache.org/view/Beam/job/beam_PostCommit_Python_Verify/853/).
> Here is the stack trace:
> {code}
> Traceback (most recent call last):
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/sdks/python/apache_beam/dataflow_test.py",
>  line 186, in test_multi_valued_singleton_side_input
> pipeline.run()
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/sdks/python/apache_beam/pipeline.py",
>  line 159, in run
> return self.runner.run(self)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/sdks/python/apache_beam/runners/dataflow_runner.py",
>  line 195, in run
> % getattr(self, 'last_error_msg', None), self.result)
> DataflowRuntimeException: Dataflow pipeline failed:
> (99aeafa7a8dffcc7): Traceback (most recent call last):
>   File 
> "/usr/local/lib/python2.7/dist-packages/dataflow_worker/batchworker.py", line 
> 514, in do_work
> work_executor.execute()
>   File "dataflow_worker/executor.py", line 892, in 
> dataflow_worker.executor.MapTaskExecutor.execute 
> (dataflow_worker/executor.c:24008)
> op.start()
>   File "dataflow_worker/executor.py", line 456, in 
> dataflow_worker.executor.DoOperation.start (dataflow_worker/executor.c:13870)
> def start(self):
>   File "dataflow_worker/executor.py", line 483, in 
> dataflow_worker.executor.DoOperation.start (dataflow_worker/executor.c:13685)
> self.dofn_runner = common.DoFnRunner(
>   File "apache_beam/runners/common.py", line 89, in 
> apache_beam.runners.common.DoFnRunner.__init__ 
> (apache_beam/runners/common.c:3469)
> args, kwargs, [side_input[global_window]
>   File 
> "/usr/local/lib/python2.7/dist-packages/apache_beam/transforms/sideinputs.py",
>  line 192, in __getitem__
> _FilteringIterable(self._iterable, target_window), self._view_options)
>   File "/usr/local/lib/python2.7/dist-packages/apache_beam/pvalue.py", line 
> 279, in _from_runtime_iterable
> 'PCollection with more than one element accessed as '
> ValueError: PCollection with more than one element accessed as a singleton 
> view.
> {code}
> Worker logs in here:
> https://builds.apache.org/view/Beam/job/beam_PostCommit_Python_Verify/853/console
> In order to temporarily ignore this test in postcommit, we can comment out 
> annotation "@attr('ValidatesRunner')" of this test. Then it will only run as 
> a unit test (execute by DirectRunner), but not run as a ValidatesRunner test.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (BEAM-1124) Python ValidateRunner Test test_multi_valued_singleton_side_input Break Postcommit

2017-04-10 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/BEAM-1124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963251#comment-15963251
 ] 

ASF GitHub Bot commented on BEAM-1124:
--

Github user asfgit closed the pull request at:

https://github.com/apache/beam/pull/2467


> Python ValidateRunner Test test_multi_valued_singleton_side_input Break 
> Postcommit
> --
>
> Key: BEAM-1124
> URL: https://issues.apache.org/jira/browse/BEAM-1124
> Project: Beam
>  Issue Type: Bug
>  Components: runner-dataflow, sdk-py, testing
>Reporter: Mark Liu
>Assignee: Ahmet Altay
>
> Python test_multi_valued_singleton_side_input test, a ValidatesRunner test 
> that running on dataflow service, failed and broke 
> postcommit(https://builds.apache.org/view/Beam/job/beam_PostCommit_Python_Verify/853/).
> Here is the stack trace:
> {code}
> Traceback (most recent call last):
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/sdks/python/apache_beam/dataflow_test.py",
>  line 186, in test_multi_valued_singleton_side_input
> pipeline.run()
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/sdks/python/apache_beam/pipeline.py",
>  line 159, in run
> return self.runner.run(self)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/sdks/python/apache_beam/runners/dataflow_runner.py",
>  line 195, in run
> % getattr(self, 'last_error_msg', None), self.result)
> DataflowRuntimeException: Dataflow pipeline failed:
> (99aeafa7a8dffcc7): Traceback (most recent call last):
>   File 
> "/usr/local/lib/python2.7/dist-packages/dataflow_worker/batchworker.py", line 
> 514, in do_work
> work_executor.execute()
>   File "dataflow_worker/executor.py", line 892, in 
> dataflow_worker.executor.MapTaskExecutor.execute 
> (dataflow_worker/executor.c:24008)
> op.start()
>   File "dataflow_worker/executor.py", line 456, in 
> dataflow_worker.executor.DoOperation.start (dataflow_worker/executor.c:13870)
> def start(self):
>   File "dataflow_worker/executor.py", line 483, in 
> dataflow_worker.executor.DoOperation.start (dataflow_worker/executor.c:13685)
> self.dofn_runner = common.DoFnRunner(
>   File "apache_beam/runners/common.py", line 89, in 
> apache_beam.runners.common.DoFnRunner.__init__ 
> (apache_beam/runners/common.c:3469)
> args, kwargs, [side_input[global_window]
>   File 
> "/usr/local/lib/python2.7/dist-packages/apache_beam/transforms/sideinputs.py",
>  line 192, in __getitem__
> _FilteringIterable(self._iterable, target_window), self._view_options)
>   File "/usr/local/lib/python2.7/dist-packages/apache_beam/pvalue.py", line 
> 279, in _from_runtime_iterable
> 'PCollection with more than one element accessed as '
> ValueError: PCollection with more than one element accessed as a singleton 
> view.
> {code}
> Worker logs in here:
> https://builds.apache.org/view/Beam/job/beam_PostCommit_Python_Verify/853/console
> In order to temporarily ignore this test in postcommit, we can comment out 
> annotation "@attr('ValidatesRunner')" of this test. Then it will only run as 
> a unit test (execute by DirectRunner), but not run as a ValidatesRunner test.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BEAM-1124) Python ValidateRunner Test test_multi_valued_singleton_side_input Break Postcommit

2017-04-07 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/BEAM-1124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15961610#comment-15961610
 ] 

ASF GitHub Bot commented on BEAM-1124:
--

GitHub user aaltay opened a pull request:

https://github.com/apache/beam/pull/2467

[BEAM-1124] enable test_multi_valued_singleton_side_input test

R: @markflyhigh 

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/aaltay/beam side

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/2467.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2467


commit e6c7c04e7631bc995dc57447268b896c517b3b05
Author: Ahmet Altay 
Date:   2017-04-08T00:54:14Z

enable test_multi_valued_singleton_side_input test




> Python ValidateRunner Test test_multi_valued_singleton_side_input Break 
> Postcommit
> --
>
> Key: BEAM-1124
> URL: https://issues.apache.org/jira/browse/BEAM-1124
> Project: Beam
>  Issue Type: Bug
>  Components: runner-dataflow, sdk-py, testing
>Reporter: Mark Liu
>Assignee: Ahmet Altay
>
> Python test_multi_valued_singleton_side_input test, a ValidatesRunner test 
> that running on dataflow service, failed and broke 
> postcommit(https://builds.apache.org/view/Beam/job/beam_PostCommit_Python_Verify/853/).
> Here is the stack trace:
> {code}
> Traceback (most recent call last):
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/sdks/python/apache_beam/dataflow_test.py",
>  line 186, in test_multi_valued_singleton_side_input
> pipeline.run()
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/sdks/python/apache_beam/pipeline.py",
>  line 159, in run
> return self.runner.run(self)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/sdks/python/apache_beam/runners/dataflow_runner.py",
>  line 195, in run
> % getattr(self, 'last_error_msg', None), self.result)
> DataflowRuntimeException: Dataflow pipeline failed:
> (99aeafa7a8dffcc7): Traceback (most recent call last):
>   File 
> "/usr/local/lib/python2.7/dist-packages/dataflow_worker/batchworker.py", line 
> 514, in do_work
> work_executor.execute()
>   File "dataflow_worker/executor.py", line 892, in 
> dataflow_worker.executor.MapTaskExecutor.execute 
> (dataflow_worker/executor.c:24008)
> op.start()
>   File "dataflow_worker/executor.py", line 456, in 
> dataflow_worker.executor.DoOperation.start (dataflow_worker/executor.c:13870)
> def start(self):
>   File "dataflow_worker/executor.py", line 483, in 
> dataflow_worker.executor.DoOperation.start (dataflow_worker/executor.c:13685)
> self.dofn_runner = common.DoFnRunner(
>   File "apache_beam/runners/common.py", line 89, in 
> apache_beam.runners.common.DoFnRunner.__init__ 
> (apache_beam/runners/common.c:3469)
> args, kwargs, [side_input[global_window]
>   File 
> "/usr/local/lib/python2.7/dist-packages/apache_beam/transforms/sideinputs.py",
>  line 192, in __getitem__
> _FilteringIterable(self._iterable, target_window), self._view_options)
>   File "/usr/local/lib/python2.7/dist-packages/apache_beam/pvalue.py", line 
> 279, in _from_runtime_iterable
> 'PCollection with more than one element accessed as '
> ValueError: PCollection with more than one element accessed as a singleton 
> view.
> {code}
> Worker logs in here:
> https://builds.apache.org/view/Beam/job/beam_PostCommit_Python_Verify/853/console
> In order to temporarily ignore this test in postcommit, we can comment out 
> annotation "@attr('ValidatesRunner')" of this test. Then it will only run as 
> a unit test (execute by DirectRunner), but not run as a ValidatesRunner test.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BEAM-1124) Python ValidateRunner Test test_multi_valued_singleton_side_input Break Postcommit

2017-03-30 Thread Daniel Halperin (JIRA) 

[ 
https://issues.apache.org/jira/browse/BEAM-1124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15949386#comment-15949386
 ] 

Daniel Halperin commented on BEAM-1124:
---

[~altay] any progress here?

> Python ValidateRunner Test test_multi_valued_singleton_side_input Break 
> Postcommit
> --
>
> Key: BEAM-1124
> URL: https://issues.apache.org/jira/browse/BEAM-1124
> Project: Beam
>  Issue Type: Bug
>  Components: runner-dataflow, sdk-py, testing
>Reporter: Mark Liu
>Assignee: Ahmet Altay
>
> Python test_multi_valued_singleton_side_input test, a ValidatesRunner test 
> that running on dataflow service, failed and broke 
> postcommit(https://builds.apache.org/view/Beam/job/beam_PostCommit_Python_Verify/853/).
> Here is the stack trace:
> {code}
> Traceback (most recent call last):
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/sdks/python/apache_beam/dataflow_test.py",
>  line 186, in test_multi_valued_singleton_side_input
> pipeline.run()
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/sdks/python/apache_beam/pipeline.py",
>  line 159, in run
> return self.runner.run(self)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/sdks/python/apache_beam/runners/dataflow_runner.py",
>  line 195, in run
> % getattr(self, 'last_error_msg', None), self.result)
> DataflowRuntimeException: Dataflow pipeline failed:
> (99aeafa7a8dffcc7): Traceback (most recent call last):
>   File 
> "/usr/local/lib/python2.7/dist-packages/dataflow_worker/batchworker.py", line 
> 514, in do_work
> work_executor.execute()
>   File "dataflow_worker/executor.py", line 892, in 
> dataflow_worker.executor.MapTaskExecutor.execute 
> (dataflow_worker/executor.c:24008)
> op.start()
>   File "dataflow_worker/executor.py", line 456, in 
> dataflow_worker.executor.DoOperation.start (dataflow_worker/executor.c:13870)
> def start(self):
>   File "dataflow_worker/executor.py", line 483, in 
> dataflow_worker.executor.DoOperation.start (dataflow_worker/executor.c:13685)
> self.dofn_runner = common.DoFnRunner(
>   File "apache_beam/runners/common.py", line 89, in 
> apache_beam.runners.common.DoFnRunner.__init__ 
> (apache_beam/runners/common.c:3469)
> args, kwargs, [side_input[global_window]
>   File 
> "/usr/local/lib/python2.7/dist-packages/apache_beam/transforms/sideinputs.py",
>  line 192, in __getitem__
> _FilteringIterable(self._iterable, target_window), self._view_options)
>   File "/usr/local/lib/python2.7/dist-packages/apache_beam/pvalue.py", line 
> 279, in _from_runtime_iterable
> 'PCollection with more than one element accessed as '
> ValueError: PCollection with more than one element accessed as a singleton 
> view.
> {code}
> Worker logs in here:
> https://builds.apache.org/view/Beam/job/beam_PostCommit_Python_Verify/853/console
> In order to temporarily ignore this test in postcommit, we can comment out 
> annotation "@attr('ValidatesRunner')" of this test. Then it will only run as 
> a unit test (execute by DirectRunner), but not run as a ValidatesRunner test.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)