[jira] [Commented] (BEAM-8324) Pre/Postcommit Dataflow IT tests fail: _create_function() takes from 2 to 6 positional arguments but 7 were given
[
https://issues.apache.org/jira/browse/BEAM-8324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16951276#comment-16951276
]
Valentyn Tymofieiev commented on BEAM-8324:
---
The infinite recursion error that appears with dill upgrade also appears
without dill upgrade if we launch the tests through nose:
https://issues.apache.org/jira/browse/BEAM-8397.
> Pre/Postcommit Dataflow IT tests fail: _create_function() takes from 2 to 6
> positional arguments but 7 were given
> -
>
> Key: BEAM-8324
> URL: https://issues.apache.org/jira/browse/BEAM-8324
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core
>Reporter: Valentyn Tymofieiev
>Assignee: Valentyn Tymofieiev
>Priority: Blocker
> Fix For: 2.16.0
>
> Time Spent: 2h
> Remaining Estimate: 0h
>
> {noformat}
> Dataflow pipeline failed. State: FAILED, Error:
> Traceback (most recent call last):
> File
> "/usr/local/lib/python3.7/site-packages/dataflow_worker/batchworker.py", line
> 773, in run
> self._load_main_session(self.local_staging_directory)
> File
> "/usr/local/lib/python3.7/site-packages/dataflow_worker/batchworker.py", line
> 489, in _load_main_session
> pickler.load_session(session_file)
> File
> "/usr/local/lib/python3.7/site-packages/apache_beam/internal/pickler.py",
> line 287, in load_session
> return dill.load_session(file_path)
> File "/usr/local/lib/python3.7/site-packages/dill/_dill.py", line 410, in
> load_session
> module = unpickler.load()
> TypeError: _create_function() takes from 2 to 6 positional arguments but 7
> were given
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (BEAM-8324) Pre/Postcommit Dataflow IT tests fail: _create_function() takes from 2 to 6 positional arguments but 7 were given
[
https://issues.apache.org/jira/browse/BEAM-8324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16941287#comment-16941287
]
Valentyn Tymofieiev commented on BEAM-8324:
---
Sent https://github.com/apache/beam/pull/9695. See also:
https://github.com/uqfoundation/dill/issues/341.
> Pre/Postcommit Dataflow IT tests fail: _create_function() takes from 2 to 6
> positional arguments but 7 were given
> -
>
> Key: BEAM-8324
> URL: https://issues.apache.org/jira/browse/BEAM-8324
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core
>Reporter: Valentyn Tymofieiev
>Assignee: Valentyn Tymofieiev
>Priority: Blocker
> Fix For: 2.16.0
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> {noformat}
> Dataflow pipeline failed. State: FAILED, Error:
> Traceback (most recent call last):
> File
> "/usr/local/lib/python3.7/site-packages/dataflow_worker/batchworker.py", line
> 773, in run
> self._load_main_session(self.local_staging_directory)
> File
> "/usr/local/lib/python3.7/site-packages/dataflow_worker/batchworker.py", line
> 489, in _load_main_session
> pickler.load_session(session_file)
> File
> "/usr/local/lib/python3.7/site-packages/apache_beam/internal/pickler.py",
> line 287, in load_session
> return dill.load_session(file_path)
> File "/usr/local/lib/python3.7/site-packages/dill/_dill.py", line 410, in
> load_session
> module = unpickler.load()
> TypeError: _create_function() takes from 2 to 6 positional arguments but 7
> were given
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (BEAM-8324) Pre/Postcommit Dataflow IT tests fail: _create_function() takes from 2 to 6 positional arguments but 7 were given
[
https://issues.apache.org/jira/browse/BEAM-8324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16941284#comment-16941284
]
Valentyn Tymofieiev commented on BEAM-8324:
---
I think 0.3.0. will be a safer option for now. We can continue investigate
0.3.1.1. changes in https://issues.apache.org/jira/browse/BEAM-5878.
> Pre/Postcommit Dataflow IT tests fail: _create_function() takes from 2 to 6
> positional arguments but 7 were given
> -
>
> Key: BEAM-8324
> URL: https://issues.apache.org/jira/browse/BEAM-8324
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core
>Reporter: Valentyn Tymofieiev
>Assignee: Valentyn Tymofieiev
>Priority: Blocker
> Fix For: 2.16.0
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> {noformat}
> Dataflow pipeline failed. State: FAILED, Error:
> Traceback (most recent call last):
> File
> "/usr/local/lib/python3.7/site-packages/dataflow_worker/batchworker.py", line
> 773, in run
> self._load_main_session(self.local_staging_directory)
> File
> "/usr/local/lib/python3.7/site-packages/dataflow_worker/batchworker.py", line
> 489, in _load_main_session
> pickler.load_session(session_file)
> File
> "/usr/local/lib/python3.7/site-packages/apache_beam/internal/pickler.py",
> line 287, in load_session
> return dill.load_session(file_path)
> File "/usr/local/lib/python3.7/site-packages/dill/_dill.py", line 410, in
> load_session
> module = unpickler.load()
> TypeError: _create_function() takes from 2 to 6 positional arguments but 7
> were given
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (BEAM-8324) Pre/Postcommit Dataflow IT tests fail: _create_function() takes from 2 to 6 positional arguments but 7 were given
[
https://issues.apache.org/jira/browse/BEAM-8324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16941261#comment-16941261
]
Valentyn Tymofieiev commented on BEAM-8324:
---
I am investigating whether this should be <0.3.1 or <0.3.2. It appears that
0.3.1.1. causes stackoverflow errors on master in one of the tests on Python
3.7, when running through tox.
{noformat}
test_remote_runner_display_data
(apache_beam.runners.dataflow.dataflow_runner_test.DataflowRunnerTest) ...
Fatal Python error: Cannot recover from stack overflow.
Current thread 0x7fc19e177740 (most recent call first):
File
"/usr/local/google/home/valentyn/projects/beam/beam/beam/sdks/python/target/.tox/py37-gcp/lib/python3.7/site-packages/dill/_dill.py",
line 382 in get
File "/usr/lib/python3.7/pickle.py", line 502 in save
File "/usr/lib/python3.7/pickle.py", line 771 in save_tuple
File "/usr/lib/python3.7/pickle.py", line 504 in save
File "/usr/lib/python3.7/pickle.py", line 786 in save_tuple
File "/usr/lib/python3.7/pickle.py", line 504 in save
File "/usr/lib/python3.7/pickle.py", line 638 in save_reduce
...
{noformat}
However there is no error if we run the test by itself:
{noformat}
python ./setup.py test -s
apache_beam.runners.dataflow.dataflow_runner_test.DataflowRunnerTest.test_remote_runner_display_data
test_remote_runner_display_data
(apache_beam.runners.dataflow.dataflow_runner_test.DataflowRunnerTest) ... ok
--
Ran 1 test in 0.063s
OK
{noformat}
> Pre/Postcommit Dataflow IT tests fail: _create_function() takes from 2 to 6
> positional arguments but 7 were given
> -
>
> Key: BEAM-8324
> URL: https://issues.apache.org/jira/browse/BEAM-8324
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core
>Reporter: Valentyn Tymofieiev
>Assignee: Valentyn Tymofieiev
>Priority: Blocker
> Fix For: 2.16.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> {noformat}
> Dataflow pipeline failed. State: FAILED, Error:
> Traceback (most recent call last):
> File
> "/usr/local/lib/python3.7/site-packages/dataflow_worker/batchworker.py", line
> 773, in run
> self._load_main_session(self.local_staging_directory)
> File
> "/usr/local/lib/python3.7/site-packages/dataflow_worker/batchworker.py", line
> 489, in _load_main_session
> pickler.load_session(session_file)
> File
> "/usr/local/lib/python3.7/site-packages/apache_beam/internal/pickler.py",
> line 287, in load_session
> return dill.load_session(file_path)
> File "/usr/local/lib/python3.7/site-packages/dill/_dill.py", line 410, in
> load_session
> module = unpickler.load()
> TypeError: _create_function() takes from 2 to 6 positional arguments but 7
> were given
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (BEAM-8324) Pre/Postcommit Dataflow IT tests fail: _create_function() takes from 2 to 6 positional arguments but 7 were given
[
https://issues.apache.org/jira/browse/BEAM-8324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16941248#comment-16941248
]
Ahmet Altay commented on BEAM-8324:
---
Let's put an upperbound. How about
dill>=0.3.1.1,<0.3.2.
I believe this impacts 2.16.0 release. See:
https://github.com/apache/beam/blob/release-2.16.0/sdks/python/setup.py
> Pre/Postcommit Dataflow IT tests fail: _create_function() takes from 2 to 6
> positional arguments but 7 were given
> -
>
> Key: BEAM-8324
> URL: https://issues.apache.org/jira/browse/BEAM-8324
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core
>Reporter: Valentyn Tymofieiev
>Assignee: Valentyn Tymofieiev
>Priority: Major
> Fix For: 2.16.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> {noformat}
> Dataflow pipeline failed. State: FAILED, Error:
> Traceback (most recent call last):
> File
> "/usr/local/lib/python3.7/site-packages/dataflow_worker/batchworker.py", line
> 773, in run
> self._load_main_session(self.local_staging_directory)
> File
> "/usr/local/lib/python3.7/site-packages/dataflow_worker/batchworker.py", line
> 489, in _load_main_session
> pickler.load_session(session_file)
> File
> "/usr/local/lib/python3.7/site-packages/apache_beam/internal/pickler.py",
> line 287, in load_session
> return dill.load_session(file_path)
> File "/usr/local/lib/python3.7/site-packages/dill/_dill.py", line 410, in
> load_session
> module = unpickler.load()
> TypeError: _create_function() takes from 2 to 6 positional arguments but 7
> were given
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (BEAM-8324) Pre/Postcommit Dataflow IT tests fail: _create_function() takes from 2 to 6 positional arguments but 7 were given
[
https://issues.apache.org/jira/browse/BEAM-8324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16941147#comment-16941147
]
Valentyn Tymofieiev commented on BEAM-8324:
---
I also checked that 2.15.0 release was not affected by this.
> Pre/Postcommit Dataflow IT tests fail: _create_function() takes from 2 to 6
> positional arguments but 7 were given
> -
>
> Key: BEAM-8324
> URL: https://issues.apache.org/jira/browse/BEAM-8324
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core
>Reporter: Valentyn Tymofieiev
>Assignee: Valentyn Tymofieiev
>Priority: Major
> Fix For: 2.16.0
>
>
> {noformat}
> Dataflow pipeline failed. State: FAILED, Error:
> Traceback (most recent call last):
> File
> "/usr/local/lib/python3.7/site-packages/dataflow_worker/batchworker.py", line
> 773, in run
> self._load_main_session(self.local_staging_directory)
> File
> "/usr/local/lib/python3.7/site-packages/dataflow_worker/batchworker.py", line
> 489, in _load_main_session
> pickler.load_session(session_file)
> File
> "/usr/local/lib/python3.7/site-packages/apache_beam/internal/pickler.py",
> line 287, in load_session
> return dill.load_session(file_path)
> File "/usr/local/lib/python3.7/site-packages/dill/_dill.py", line 410, in
> load_session
> module = unpickler.load()
> TypeError: _create_function() takes from 2 to 6 positional arguments but 7
> were given
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (BEAM-8324) Pre/Postcommit Dataflow IT tests fail: _create_function() takes from 2 to 6 positional arguments but 7 were given
[
https://issues.apache.org/jira/browse/BEAM-8324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16941146#comment-16941146
]
Valentyn Tymofieiev commented on BEAM-8324:
---
I think we should restore the upper bound for dill, since it may be difficult
to avoid this breakages unless dill has compatibility guarantees between
versions.
> Pre/Postcommit Dataflow IT tests fail: _create_function() takes from 2 to 6
> positional arguments but 7 were given
> -
>
> Key: BEAM-8324
> URL: https://issues.apache.org/jira/browse/BEAM-8324
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core
>Reporter: Valentyn Tymofieiev
>Assignee: Valentyn Tymofieiev
>Priority: Major
> Fix For: 2.16.0
>
>
> {noformat}
> Dataflow pipeline failed. State: FAILED, Error:
> Traceback (most recent call last):
> File
> "/usr/local/lib/python3.7/site-packages/dataflow_worker/batchworker.py", line
> 773, in run
> self._load_main_session(self.local_staging_directory)
> File
> "/usr/local/lib/python3.7/site-packages/dataflow_worker/batchworker.py", line
> 489, in _load_main_session
> pickler.load_session(session_file)
> File
> "/usr/local/lib/python3.7/site-packages/apache_beam/internal/pickler.py",
> line 287, in load_session
> return dill.load_session(file_path)
> File "/usr/local/lib/python3.7/site-packages/dill/_dill.py", line 410, in
> load_session
> module = unpickler.load()
> TypeError: _create_function() takes from 2 to 6 positional arguments but 7
> were given
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (BEAM-8324) Pre/Postcommit Dataflow IT tests fail: _create_function() takes from 2 to 6 positional arguments but 7 were given
[
https://issues.apache.org/jira/browse/BEAM-8324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16941140#comment-16941140
]
Valentyn Tymofieiev commented on BEAM-8324:
---
I think the cause of the error is that we pickle main session using
dill==0.3.1.1, and unpickle it using dill==0.2.9 (which is installed on DF
workers).
We can update the version of Dill in DF workers, or restrict the upperbound
for dill, which may be advisable given [~yoshiki.obata]'s report in
https://issues.apache.org/jira/browse/BEAM-5878?focusedCommentId=16941114=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16941114.
> Pre/Postcommit Dataflow IT tests fail: _create_function() takes from 2 to 6
> positional arguments but 7 were given
> -
>
> Key: BEAM-8324
> URL: https://issues.apache.org/jira/browse/BEAM-8324
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core
>Reporter: Valentyn Tymofieiev
>Assignee: Valentyn Tymofieiev
>Priority: Major
> Fix For: 2.16.0
>
>
> {noformat}
> Dataflow pipeline failed. State: FAILED, Error:
> Traceback (most recent call last):
> File
> "/usr/local/lib/python3.7/site-packages/dataflow_worker/batchworker.py", line
> 773, in run
> self._load_main_session(self.local_staging_directory)
> File
> "/usr/local/lib/python3.7/site-packages/dataflow_worker/batchworker.py", line
> 489, in _load_main_session
> pickler.load_session(session_file)
> File
> "/usr/local/lib/python3.7/site-packages/apache_beam/internal/pickler.py",
> line 287, in load_session
> return dill.load_session(file_path)
> File "/usr/local/lib/python3.7/site-packages/dill/_dill.py", line 410, in
> load_session
> module = unpickler.load()
> TypeError: _create_function() takes from 2 to 6 positional arguments but 7
> were given
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (BEAM-8324) Pre/Postcommit Dataflow IT tests fail: _create_function() takes from 2 to 6 positional arguments but 7 were given
[
https://issues.apache.org/jira/browse/BEAM-8324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16940566#comment-16940566
]
Valentyn Tymofieiev commented on BEAM-8324:
---
This may be triggered by a dill release that happened this weekend and may
potentially affect Beam release.
cc: [~markliu] [~altay] [~lcwik]
> Pre/Postcommit Dataflow IT tests fail: _create_function() takes from 2 to 6
> positional arguments but 7 were given
> -
>
> Key: BEAM-8324
> URL: https://issues.apache.org/jira/browse/BEAM-8324
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core
>Reporter: Valentyn Tymofieiev
>Assignee: Valentyn Tymofieiev
>Priority: Major
>
> {noformat}
> Dataflow pipeline failed. State: FAILED, Error:
> Traceback (most recent call last):
> File
> "/usr/local/lib/python3.7/site-packages/dataflow_worker/batchworker.py", line
> 773, in run
> self._load_main_session(self.local_staging_directory)
> File
> "/usr/local/lib/python3.7/site-packages/dataflow_worker/batchworker.py", line
> 489, in _load_main_session
> pickler.load_session(session_file)
> File
> "/usr/local/lib/python3.7/site-packages/apache_beam/internal/pickler.py",
> line 287, in load_session
> return dill.load_session(file_path)
> File "/usr/local/lib/python3.7/site-packages/dill/_dill.py", line 410, in
> load_session
> module = unpickler.load()
> TypeError: _create_function() takes from 2 to 6 positional arguments but 7
> were given
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
