[jira] [Work logged] (BEAM-5745) Util test on annotations fails
[
https://issues.apache.org/jira/browse/BEAM-5745?focusedWorklogId=161400=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-161400
]
ASF GitHub Bot logged work on BEAM-5745:
Author: ASF GitHub Bot
Created on: 01/Nov/18 00:59
Start Date: 01/Nov/18 00:59
Worklog Time Spent: 10m
Work Description: udim commented on issue #6687: [BEAM-5745] Fix
annotation test for py3
URL: https://github.com/apache/beam/pull/6687#issuecomment-434896307
@HuangLED @ihji @markflyhigh
Please revert the change that removes `warnings.simplefilter("once")`, as it
disables all deprecation notices for Beam SDK.
I've opened https://issues.apache.org/jira/browse/BEAM-5932 for this.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 161400)
Time Spent: 5h 10m (was: 5h)
> Util test on annotations fails
> ---
>
> Key: BEAM-5745
> URL: https://issues.apache.org/jira/browse/BEAM-5745
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-py-core
>Reporter: Ruoyun Huang
>Assignee: Ruoyun Huang
>Priority: Minor
> Fix For: Not applicable
>
> Time Spent: 5h 10m
> Remaining Estimate: 0h
>
> Traceback (most recent call last):
> File
> "/usr/local/google/home/ruoyun/projects/beam/sdks/python/apache_beam/utils/annotations_test.py",
> line 142, in test_frequency
> label_check_list=[])
> File
> "/usr/local/google/home/ruoyun/projects/beam/sdks/python/apache_beam/utils/annotations_test.py",
> line 149, in check_annotation
> self.assertIn(fnc_name + ' is ' + annotation_type,
> str(warning[-1].message))
> AssertionError: 'fnc2_test_annotate_frequency is experimental' not found in
> 'fnc_test_annotate_frequency is experimental.'
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-5745) Util test on annotations fails
[ https://issues.apache.org/jira/browse/BEAM-5745?focusedWorklogId=158909=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-158909 ] ASF GitHub Bot logged work on BEAM-5745: Author: ASF GitHub Bot Created on: 25/Oct/18 21:52 Start Date: 25/Oct/18 21:52 Worklog Time Spent: 10m Work Description: HuangLED commented on issue #6687: [BEAM-5745] Fix annotation test for py3 URL: https://github.com/apache/beam/pull/6687#issuecomment-433218405 > Can you please rename the commit message? Looks like after squashing the commits we picked up the last commit's message, usually the first commit messages are more informative since they describe the purpose of the change. Thank you. The change LGTM as long as the test pass and you checked that they pass under Python 3 as well. Done renaming the squashing commit. + @swegner for merge This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 158909) Time Spent: 4h 50m (was: 4h 40m) > Util test on annotations fails > --- > > Key: BEAM-5745 > URL: https://issues.apache.org/jira/browse/BEAM-5745 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-core >Reporter: Ruoyun Huang >Assignee: Ruoyun Huang >Priority: Minor > Time Spent: 4h 50m > Remaining Estimate: 0h > > Traceback (most recent call last): > File > "/usr/local/google/home/ruoyun/projects/beam/sdks/python/apache_beam/utils/annotations_test.py", > line 142, in test_frequency > label_check_list=[]) > File > "/usr/local/google/home/ruoyun/projects/beam/sdks/python/apache_beam/utils/annotations_test.py", > line 149, in check_annotation > self.assertIn(fnc_name + ' is ' + annotation_type, > str(warning[-1].message)) > AssertionError: 'fnc2_test_annotate_frequency is experimental' not found in > 'fnc_test_annotate_frequency is experimental.' -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-5745) Util test on annotations fails
[ https://issues.apache.org/jira/browse/BEAM-5745?focusedWorklogId=158383=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-158383 ] ASF GitHub Bot logged work on BEAM-5745: Author: ASF GitHub Bot Created on: 24/Oct/18 23:15 Start Date: 24/Oct/18 23:15 Worklog Time Spent: 10m Work Description: HuangLED commented on issue #6687: [BEAM-5745] Fix annotation test for py3 URL: https://github.com/apache/beam/pull/6687#issuecomment-432859666 Run Python PostCommit This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 158383) Time Spent: 4h 20m (was: 4h 10m) > Util test on annotations fails > --- > > Key: BEAM-5745 > URL: https://issues.apache.org/jira/browse/BEAM-5745 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-core >Reporter: Ruoyun Huang >Assignee: Ruoyun Huang >Priority: Minor > Time Spent: 4h 20m > Remaining Estimate: 0h > > Traceback (most recent call last): > File > "/usr/local/google/home/ruoyun/projects/beam/sdks/python/apache_beam/utils/annotations_test.py", > line 142, in test_frequency > label_check_list=[]) > File > "/usr/local/google/home/ruoyun/projects/beam/sdks/python/apache_beam/utils/annotations_test.py", > line 149, in check_annotation > self.assertIn(fnc_name + ' is ' + annotation_type, > str(warning[-1].message)) > AssertionError: 'fnc2_test_annotate_frequency is experimental' not found in > 'fnc_test_annotate_frequency is experimental.' -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-5745) Util test on annotations fails
[ https://issues.apache.org/jira/browse/BEAM-5745?focusedWorklogId=158384=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-158384 ] ASF GitHub Bot logged work on BEAM-5745: Author: ASF GitHub Bot Created on: 24/Oct/18 23:15 Start Date: 24/Oct/18 23:15 Worklog Time Spent: 10m Work Description: HuangLED removed a comment on issue #6687: [BEAM-5745] Fix annotation test for py3 URL: https://github.com/apache/beam/pull/6687#issuecomment-432854523 Run Python PostCommit This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 158384) Time Spent: 4.5h (was: 4h 20m) > Util test on annotations fails > --- > > Key: BEAM-5745 > URL: https://issues.apache.org/jira/browse/BEAM-5745 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-core >Reporter: Ruoyun Huang >Assignee: Ruoyun Huang >Priority: Minor > Time Spent: 4.5h > Remaining Estimate: 0h > > Traceback (most recent call last): > File > "/usr/local/google/home/ruoyun/projects/beam/sdks/python/apache_beam/utils/annotations_test.py", > line 142, in test_frequency > label_check_list=[]) > File > "/usr/local/google/home/ruoyun/projects/beam/sdks/python/apache_beam/utils/annotations_test.py", > line 149, in check_annotation > self.assertIn(fnc_name + ' is ' + annotation_type, > str(warning[-1].message)) > AssertionError: 'fnc2_test_annotate_frequency is experimental' not found in > 'fnc_test_annotate_frequency is experimental.' -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-5745) Util test on annotations fails
[ https://issues.apache.org/jira/browse/BEAM-5745?focusedWorklogId=158371=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-158371 ] ASF GitHub Bot logged work on BEAM-5745: Author: ASF GitHub Bot Created on: 24/Oct/18 22:59 Start Date: 24/Oct/18 22:59 Worklog Time Spent: 10m Work Description: tvalentyn commented on issue #6687: [BEAM-5745] Fix annotation test for py3 URL: https://github.com/apache/beam/pull/6687#issuecomment-432856565 Can you please rename the commit message? Looks like after squashing the commits we picked up the last commit's message, usually the first commit messages are more informative since they describe the purpose of the change. Thank you. The change LGTM as long as the test pass and you checked that they pass under Python 3 as well. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 158371) Time Spent: 4h 10m (was: 4h) > Util test on annotations fails > --- > > Key: BEAM-5745 > URL: https://issues.apache.org/jira/browse/BEAM-5745 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-core >Reporter: Ruoyun Huang >Assignee: Ruoyun Huang >Priority: Minor > Time Spent: 4h 10m > Remaining Estimate: 0h > > Traceback (most recent call last): > File > "/usr/local/google/home/ruoyun/projects/beam/sdks/python/apache_beam/utils/annotations_test.py", > line 142, in test_frequency > label_check_list=[]) > File > "/usr/local/google/home/ruoyun/projects/beam/sdks/python/apache_beam/utils/annotations_test.py", > line 149, in check_annotation > self.assertIn(fnc_name + ' is ' + annotation_type, > str(warning[-1].message)) > AssertionError: 'fnc2_test_annotate_frequency is experimental' not found in > 'fnc_test_annotate_frequency is experimental.' -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-5745) Util test on annotations fails
[ https://issues.apache.org/jira/browse/BEAM-5745?focusedWorklogId=158366=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-158366 ] ASF GitHub Bot logged work on BEAM-5745: Author: ASF GitHub Bot Created on: 24/Oct/18 22:49 Start Date: 24/Oct/18 22:49 Worklog Time Spent: 10m Work Description: HuangLED commented on issue #6687: [BEAM-5745] Fix annotation test for py3 URL: https://github.com/apache/beam/pull/6687#issuecomment-432854523 Run Python PostCommit This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 158366) Time Spent: 4h (was: 3h 50m) > Util test on annotations fails > --- > > Key: BEAM-5745 > URL: https://issues.apache.org/jira/browse/BEAM-5745 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-core >Reporter: Ruoyun Huang >Assignee: Ruoyun Huang >Priority: Minor > Time Spent: 4h > Remaining Estimate: 0h > > Traceback (most recent call last): > File > "/usr/local/google/home/ruoyun/projects/beam/sdks/python/apache_beam/utils/annotations_test.py", > line 142, in test_frequency > label_check_list=[]) > File > "/usr/local/google/home/ruoyun/projects/beam/sdks/python/apache_beam/utils/annotations_test.py", > line 149, in check_annotation > self.assertIn(fnc_name + ' is ' + annotation_type, > str(warning[-1].message)) > AssertionError: 'fnc2_test_annotate_frequency is experimental' not found in > 'fnc_test_annotate_frequency is experimental.' -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-5745) Util test on annotations fails
[ https://issues.apache.org/jira/browse/BEAM-5745?focusedWorklogId=158315=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-158315 ] ASF GitHub Bot logged work on BEAM-5745: Author: ASF GitHub Bot Created on: 24/Oct/18 21:06 Start Date: 24/Oct/18 21:06 Worklog Time Spent: 10m Work Description: tvalentyn commented on issue #6687: [BEAM-5745] Fix annotation test for py3 URL: https://github.com/apache/beam/pull/6687#issuecomment-432828218 One way to rebase your change on top of master, manually resolve merge conflict, and squash/fixup commits into 1 is via `git pull --rebase=interactive origin master`. Looking at the conflict, you'd have to resolve it manually or start over with the PR using a new branch, that has most recent code from master. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 158315) Time Spent: 3h 50m (was: 3h 40m) > Util test on annotations fails > --- > > Key: BEAM-5745 > URL: https://issues.apache.org/jira/browse/BEAM-5745 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-core >Reporter: Ruoyun Huang >Assignee: Ruoyun Huang >Priority: Minor > Time Spent: 3h 50m > Remaining Estimate: 0h > > Traceback (most recent call last): > File > "/usr/local/google/home/ruoyun/projects/beam/sdks/python/apache_beam/utils/annotations_test.py", > line 142, in test_frequency > label_check_list=[]) > File > "/usr/local/google/home/ruoyun/projects/beam/sdks/python/apache_beam/utils/annotations_test.py", > line 149, in check_annotation > self.assertIn(fnc_name + ' is ' + annotation_type, > str(warning[-1].message)) > AssertionError: 'fnc2_test_annotate_frequency is experimental' not found in > 'fnc_test_annotate_frequency is experimental.' -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-5745) Util test on annotations fails
[ https://issues.apache.org/jira/browse/BEAM-5745?focusedWorklogId=158283=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-158283 ] ASF GitHub Bot logged work on BEAM-5745: Author: ASF GitHub Bot Created on: 24/Oct/18 19:47 Start Date: 24/Oct/18 19:47 Worklog Time Spent: 10m Work Description: HuangLED commented on a change in pull request #6687: [BEAM-5745] Fix annotation test for py3 URL: https://github.com/apache/beam/pull/6687#discussion_r227934912 ## File path: sdks/python/apache_beam/utils/annotations_test.py ## @@ -32,9 +32,12 @@ def test_deprecated_with_since_current_message(self): @deprecated(since='v.1', current='multiply', extra_message='Do this') def fnc_test_deprecated_with_since_current_message(): return 'lol' + # In python2, the test framework by default adds an ignore filtering rule. Review comment: Acked. Thanks. Though this PR is now in conflict with another non-trivial merge. Does not seem trivial to get all resolved. Please let me know if there is any way better than copy over the master version and apply all my changes again. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 158283) Time Spent: 3h 40m (was: 3.5h) > Util test on annotations fails > --- > > Key: BEAM-5745 > URL: https://issues.apache.org/jira/browse/BEAM-5745 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-core >Reporter: Ruoyun Huang >Assignee: Ruoyun Huang >Priority: Minor > Time Spent: 3h 40m > Remaining Estimate: 0h > > Traceback (most recent call last): > File > "/usr/local/google/home/ruoyun/projects/beam/sdks/python/apache_beam/utils/annotations_test.py", > line 142, in test_frequency > label_check_list=[]) > File > "/usr/local/google/home/ruoyun/projects/beam/sdks/python/apache_beam/utils/annotations_test.py", > line 149, in check_annotation > self.assertIn(fnc_name + ' is ' + annotation_type, > str(warning[-1].message)) > AssertionError: 'fnc2_test_annotate_frequency is experimental' not found in > 'fnc_test_annotate_frequency is experimental.' -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-5745) Util test on annotations fails
[
https://issues.apache.org/jira/browse/BEAM-5745?focusedWorklogId=158264=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-158264
]
ASF GitHub Bot logged work on BEAM-5745:
Author: ASF GitHub Bot
Created on: 24/Oct/18 18:43
Start Date: 24/Oct/18 18:43
Worklog Time Spent: 10m
Work Description: tvalentyn commented on a change in pull request #6687:
[BEAM-5745] Fix annotation test for py3
URL: https://github.com/apache/beam/pull/6687#discussion_r227913982
##
File path: sdks/python/apache_beam/utils/annotations_test.py
##
@@ -32,9 +32,12 @@ def test_deprecated_with_since_current_message(self):
@deprecated(since='v.1', current='multiply', extra_message='Do this')
def fnc_test_deprecated_with_since_current_message():
return 'lol'
+ # In python2, the test framework by default adds an ignore filtering
rule.
Review comment:
Thanks. Turns out DeprecationWarnings are ignored by default, not just on
Python 2, see: https://docs.python.org/3/library/warnings.html. So I suggest
the following wording (let's add a clarifying comment for all test methods that
need this).
```
def test_deprecated_with_since_current_message(self):
with warnings.catch_warnings(record=True) as w:
# Deprecation warnings are ignored by default, turn them on to enable
testing.
warnings.simplefilter("once", DeprecationWarning)
...
```
Adding this to each individual test is the right thing, since we add a
filter override inside `catch_warnings` context manager, so it will not affect
other tests in the suite.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 158264)
Time Spent: 3.5h (was: 3h 20m)
> Util test on annotations fails
> ---
>
> Key: BEAM-5745
> URL: https://issues.apache.org/jira/browse/BEAM-5745
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-py-core
>Reporter: Ruoyun Huang
>Assignee: Ruoyun Huang
>Priority: Minor
> Time Spent: 3.5h
> Remaining Estimate: 0h
>
> Traceback (most recent call last):
> File
> "/usr/local/google/home/ruoyun/projects/beam/sdks/python/apache_beam/utils/annotations_test.py",
> line 142, in test_frequency
> label_check_list=[])
> File
> "/usr/local/google/home/ruoyun/projects/beam/sdks/python/apache_beam/utils/annotations_test.py",
> line 149, in check_annotation
> self.assertIn(fnc_name + ' is ' + annotation_type,
> str(warning[-1].message))
> AssertionError: 'fnc2_test_annotate_frequency is experimental' not found in
> 'fnc_test_annotate_frequency is experimental.'
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-5745) Util test on annotations fails
[
https://issues.apache.org/jira/browse/BEAM-5745?focusedWorklogId=158263=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-158263
]
ASF GitHub Bot logged work on BEAM-5745:
Author: ASF GitHub Bot
Created on: 24/Oct/18 18:42
Start Date: 24/Oct/18 18:42
Worklog Time Spent: 10m
Work Description: tvalentyn commented on a change in pull request #6687:
[BEAM-5745] Fix annotation test for py3
URL: https://github.com/apache/beam/pull/6687#discussion_r227913982
##
File path: sdks/python/apache_beam/utils/annotations_test.py
##
@@ -32,9 +32,12 @@ def test_deprecated_with_since_current_message(self):
@deprecated(since='v.1', current='multiply', extra_message='Do this')
def fnc_test_deprecated_with_since_current_message():
return 'lol'
+ # In python2, the test framework by default adds an ignore filtering
rule.
Review comment:
Thanks. Turns out DeprecationWarnings are ignored by default, not just on
Python 2, see: https://docs.python.org/3/library/warnings.html. So I suggest
the following wording (let's add a clarifying comment for all test methods that
need this).
```
def test_deprecated_with_since_current_message(self):
with warnings.catch_warnings(record=True) as w:
# Deprecation warnings are ignored by default, turn them on to enable
testing.
warnings.simplefilter("once", DeprecationWarning)
```
Adding this to each individual test is the right thing, since we add a
filter override inside `catch_warnings` context manager, so it will not affect
other tests in the suite.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 158263)
Time Spent: 3h 20m (was: 3h 10m)
> Util test on annotations fails
> ---
>
> Key: BEAM-5745
> URL: https://issues.apache.org/jira/browse/BEAM-5745
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-py-core
>Reporter: Ruoyun Huang
>Assignee: Ruoyun Huang
>Priority: Minor
> Time Spent: 3h 20m
> Remaining Estimate: 0h
>
> Traceback (most recent call last):
> File
> "/usr/local/google/home/ruoyun/projects/beam/sdks/python/apache_beam/utils/annotations_test.py",
> line 142, in test_frequency
> label_check_list=[])
> File
> "/usr/local/google/home/ruoyun/projects/beam/sdks/python/apache_beam/utils/annotations_test.py",
> line 149, in check_annotation
> self.assertIn(fnc_name + ' is ' + annotation_type,
> str(warning[-1].message))
> AssertionError: 'fnc2_test_annotate_frequency is experimental' not found in
> 'fnc_test_annotate_frequency is experimental.'
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-5745) Util test on annotations fails
[
https://issues.apache.org/jira/browse/BEAM-5745?focusedWorklogId=156584=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-156584
]
ASF GitHub Bot logged work on BEAM-5745:
Author: ASF GitHub Bot
Created on: 20/Oct/18 21:40
Start Date: 20/Oct/18 21:40
Worklog Time Spent: 10m
Work Description: HuangLED commented on a change in pull request #6687:
[BEAM-5745] Fix annotation test for py3
URL: https://github.com/apache/beam/pull/6687#discussion_r226840785
##
File path: sdks/python/apache_beam/utils/annotations_test.py
##
@@ -32,9 +32,10 @@ def test_deprecated_with_since_current_message(self):
@deprecated(since='v.1', current='multiply', extra_message='Do this')
def fnc_test_deprecated_with_since_current_message():
return 'lol'
+ warnings.simplefilter("once")
Review comment:
This is only needed in python2.
in python2, the whole list beings with a 'ignore' filtering rule. see a
print out if we don't turn it on here: ``` BEGING [('ignore', <_sre.SRE_Pattern
object at 0x7f3581e22b70>, , <_sre.SRE_Pattern
object at 0x7f358192a510>, 0), ('ignore', <_sre.SRE_Pattern object at
0x7f3581154290>, , <_sre.SRE_Pattern object at
0x7f358192a510>, 0), ('ignore', <_sre.SRE_Pattern object at 0x7f3581154160>,
, <_sre.SRE_Pattern object at 0x7f358192a510>, 0),
('always', None, , None, 0), ... ...
```
this is also shown by the printout in the research you shared last time.
Since our decision is to remove frequency control in our module, it no
longer matters which lib in the testing framework adds filtering rule. And
agree that is a good idea to leave the filtering strategy decision to the end
user whoever using the library.
This line just basically just turns the warning ON so that we can have
something to validate in the test case. And alternatively we may use "always",
or anything but 'ignore'. This line is not about frequency. In other words,
the test case by default it has `ignore`, so we are making a fix here by
turning the warning on, just to have some data to test with.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 156584)
Time Spent: 3h 10m (was: 3h)
> Util test on annotations fails
> ---
>
> Key: BEAM-5745
> URL: https://issues.apache.org/jira/browse/BEAM-5745
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-py-core
>Reporter: Ruoyun Huang
>Assignee: Ruoyun Huang
>Priority: Minor
> Time Spent: 3h 10m
> Remaining Estimate: 0h
>
> Traceback (most recent call last):
> File
> "/usr/local/google/home/ruoyun/projects/beam/sdks/python/apache_beam/utils/annotations_test.py",
> line 142, in test_frequency
> label_check_list=[])
> File
> "/usr/local/google/home/ruoyun/projects/beam/sdks/python/apache_beam/utils/annotations_test.py",
> line 149, in check_annotation
> self.assertIn(fnc_name + ' is ' + annotation_type,
> str(warning[-1].message))
> AssertionError: 'fnc2_test_annotate_frequency is experimental' not found in
> 'fnc_test_annotate_frequency is experimental.'
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-5745) Util test on annotations fails
[
https://issues.apache.org/jira/browse/BEAM-5745?focusedWorklogId=156582=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-156582
]
ASF GitHub Bot logged work on BEAM-5745:
Author: ASF GitHub Bot
Created on: 20/Oct/18 21:38
Start Date: 20/Oct/18 21:38
Worklog Time Spent: 10m
Work Description: HuangLED commented on a change in pull request #6687:
[BEAM-5745] Fix annotation test for py3
URL: https://github.com/apache/beam/pull/6687#discussion_r226840785
##
File path: sdks/python/apache_beam/utils/annotations_test.py
##
@@ -32,9 +32,10 @@ def test_deprecated_with_since_current_message(self):
@deprecated(since='v.1', current='multiply', extra_message='Do this')
def fnc_test_deprecated_with_since_current_message():
return 'lol'
+ warnings.simplefilter("once")
Review comment:
This is only needed in python2.
in python2, the whole list beings with a 'ignore' filtering rule. see a
print out if we don't turn it on here: ``` ... [('ignore', <_sre.SRE_Pattern
object at 0x7f3581e22b70>, , <_sre.SRE_Pattern
object at 0x7f358192a510>, 0), ('ignore', <_sre.SRE_Pattern object at
0x7f3581154290>, , <_sre.SRE_Pattern object at
0x7f358192a510>, 0), ('ignore', <_sre.SRE_Pattern object at 0x7f3581154160>,
, <_sre.SRE_Pattern object at 0x7f358192a510>, 0),
('always', None, , None, 0), ... ```
this is also shown by the printout in the research you shared last time.
Since our decision is to remove the once filtering rule in our module, it no
longer matter which lib in the testing framework adds filtering rule. And I
agree that is a good idea to leave the filtering strategy decision to the end
user whoever using the library.
This line just basically just turns the warning ON so that we can have
something to validate in the test case. And alternatively we may use "always",
or anything but 'ignore'. Essentially the same here. This line has nothing
to do with frequency. In other words, the test case by default it has
`ignore`, so we are making a fix here by turning the warning on, just to have
some data to test with.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 156582)
Time Spent: 2h 50m (was: 2h 40m)
> Util test on annotations fails
> ---
>
> Key: BEAM-5745
> URL: https://issues.apache.org/jira/browse/BEAM-5745
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-py-core
>Reporter: Ruoyun Huang
>Assignee: Ruoyun Huang
>Priority: Minor
> Time Spent: 2h 50m
> Remaining Estimate: 0h
>
> Traceback (most recent call last):
> File
> "/usr/local/google/home/ruoyun/projects/beam/sdks/python/apache_beam/utils/annotations_test.py",
> line 142, in test_frequency
> label_check_list=[])
> File
> "/usr/local/google/home/ruoyun/projects/beam/sdks/python/apache_beam/utils/annotations_test.py",
> line 149, in check_annotation
> self.assertIn(fnc_name + ' is ' + annotation_type,
> str(warning[-1].message))
> AssertionError: 'fnc2_test_annotate_frequency is experimental' not found in
> 'fnc_test_annotate_frequency is experimental.'
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-5745) Util test on annotations fails
[
https://issues.apache.org/jira/browse/BEAM-5745?focusedWorklogId=156581=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-156581
]
ASF GitHub Bot logged work on BEAM-5745:
Author: ASF GitHub Bot
Created on: 20/Oct/18 21:25
Start Date: 20/Oct/18 21:25
Worklog Time Spent: 10m
Work Description: HuangLED commented on a change in pull request #6687:
[BEAM-5745] Fix annotation test for py3
URL: https://github.com/apache/beam/pull/6687#discussion_r226840785
##
File path: sdks/python/apache_beam/utils/annotations_test.py
##
@@ -32,9 +32,10 @@ def test_deprecated_with_since_current_message(self):
@deprecated(since='v.1', current='multiply', extra_message='Do this')
def fnc_test_deprecated_with_since_current_message():
return 'lol'
+ warnings.simplefilter("once")
Review comment:
This is only needed in python2.
in python2, the whole list beings with a 'ignore' filtering rule. I didn't
do detailed tracing my self, but this is also shown by the printout in the
research you shared last time.
Since our decision is to remove the once filtering rule in our module, it no
longer matter which lib in the testing framework adds filtering rule. And I
agree that is a good idea to leave the filtering strategy decision to the end
user whoever using the library.
This line just basically just turns the warning ON so that we can have
something to validate in the test case. And alternatively we may use "always",
or anything but 'ignore'. Essentially the same here. This line has nothing
to do with frequency. In other words, the test case by default it has
`ignore`, so we are making a fix here by turning the warning on, just to have
some data to test with.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 156581)
Time Spent: 2h 40m (was: 2.5h)
> Util test on annotations fails
> ---
>
> Key: BEAM-5745
> URL: https://issues.apache.org/jira/browse/BEAM-5745
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-py-core
>Reporter: Ruoyun Huang
>Assignee: Ruoyun Huang
>Priority: Minor
> Time Spent: 2h 40m
> Remaining Estimate: 0h
>
> Traceback (most recent call last):
> File
> "/usr/local/google/home/ruoyun/projects/beam/sdks/python/apache_beam/utils/annotations_test.py",
> line 142, in test_frequency
> label_check_list=[])
> File
> "/usr/local/google/home/ruoyun/projects/beam/sdks/python/apache_beam/utils/annotations_test.py",
> line 149, in check_annotation
> self.assertIn(fnc_name + ' is ' + annotation_type,
> str(warning[-1].message))
> AssertionError: 'fnc2_test_annotate_frequency is experimental' not found in
> 'fnc_test_annotate_frequency is experimental.'
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-5745) Util test on annotations fails
[ https://issues.apache.org/jira/browse/BEAM-5745?focusedWorklogId=156504=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-156504 ] ASF GitHub Bot logged work on BEAM-5745: Author: ASF GitHub Bot Created on: 20/Oct/18 00:49 Start Date: 20/Oct/18 00:49 Worklog Time Spent: 10m Work Description: HuangLED commented on issue #6687: [BEAM-5745] Fix annotation test for py3 URL: https://github.com/apache/beam/pull/6687#issuecomment-431534833 Updated based on discussions: 1) filter rule in annotation module is removed. 2) Test case on the frequency limit is removed. 3) In a few existing test case , added simplefilter to counter act whatever is added by testing framework. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 156504) Time Spent: 2h 20m (was: 2h 10m) > Util test on annotations fails > --- > > Key: BEAM-5745 > URL: https://issues.apache.org/jira/browse/BEAM-5745 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-core >Reporter: Ruoyun Huang >Assignee: Ruoyun Huang >Priority: Minor > Time Spent: 2h 20m > Remaining Estimate: 0h > > Traceback (most recent call last): > File > "/usr/local/google/home/ruoyun/projects/beam/sdks/python/apache_beam/utils/annotations_test.py", > line 142, in test_frequency > label_check_list=[]) > File > "/usr/local/google/home/ruoyun/projects/beam/sdks/python/apache_beam/utils/annotations_test.py", > line 149, in check_annotation > self.assertIn(fnc_name + ' is ' + annotation_type, > str(warning[-1].message)) > AssertionError: 'fnc2_test_annotate_frequency is experimental' not found in > 'fnc_test_annotate_frequency is experimental.' -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-5745) Util test on annotations fails
[ https://issues.apache.org/jira/browse/BEAM-5745?focusedWorklogId=156069=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-156069 ] ASF GitHub Bot logged work on BEAM-5745: Author: ASF GitHub Bot Created on: 18/Oct/18 18:17 Start Date: 18/Oct/18 18:17 Worklog Time Spent: 10m Work Description: HuangLED commented on issue #6687: [BEAM-5745] Fix annotation test for py3 URL: https://github.com/apache/beam/pull/6687#issuecomment-431110744 Thanks Valentyn for sharing the research. Agree that we'd better rethink the annotation machanism, which will require some dedicated efforts and beyond the scope of this PR. +1 to remove the line of simplefilter in py code. Regarding the unit test. I vote for keeping the unit test and adding simplefilter() in the test to counter act what is done by testing framework (a.k.a. the options#2 I mentioned earlier) Thoughts? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 156069) Time Spent: 1h 50m (was: 1h 40m) > Util test on annotations fails > --- > > Key: BEAM-5745 > URL: https://issues.apache.org/jira/browse/BEAM-5745 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-core >Reporter: Ruoyun Huang >Assignee: Ruoyun Huang >Priority: Minor > Time Spent: 1h 50m > Remaining Estimate: 0h > > Traceback (most recent call last): > File > "/usr/local/google/home/ruoyun/projects/beam/sdks/python/apache_beam/utils/annotations_test.py", > line 142, in test_frequency > label_check_list=[]) > File > "/usr/local/google/home/ruoyun/projects/beam/sdks/python/apache_beam/utils/annotations_test.py", > line 149, in check_annotation > self.assertIn(fnc_name + ' is ' + annotation_type, > str(warning[-1].message)) > AssertionError: 'fnc2_test_annotate_frequency is experimental' not found in > 'fnc_test_annotate_frequency is experimental.' -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-5745) Util test on annotations fails
[
https://issues.apache.org/jira/browse/BEAM-5745?focusedWorklogId=156055=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-156055
]
ASF GitHub Bot logged work on BEAM-5745:
Author: ASF GitHub Bot
Created on: 18/Oct/18 17:33
Start Date: 18/Oct/18 17:33
Worklog Time Spent: 10m
Work Description: tvalentyn commented on issue #6687: [BEAM-5745] Fix
annotation test for py3
URL: https://github.com/apache/beam/pull/6687#issuecomment-431096243
For completeness, the overriding filter `('default', None, , None, 0)` is added by unit test framework here:
https://github.com/python/cpython/blob/1a2b52bbd2c6513aaf5bb915890ea7647713d5df/Lib/unittest/runner.py#L161.
Full stacktrace:
```
File "./setup.py", line 230, in
'test': generate_protos_first(test),
File "/tmp/py3env/lib/python3.5/site-packages/setuptools/__init__.py",
line 140, in setup
return distutils.core.setup(**attrs)
File "/usr/lib/python3.5/distutils/core.py", line 148, in setup
dist.run_commands()
File "/usr/lib/python3.5/distutils/dist.py", line 955, in run_commands
self.run_command(cmd)
File "/usr/lib/python3.5/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File "./setup.py", line 165, in run
super(cmd, self).run()
File "/tmp/py3env/lib/python3.5/site-packages/setuptools/command/test.py",
line 228, in run
self.run_tests()
File "/tmp/py3env/lib/python3.5/site-packages/setuptools/command/test.py",
line 250, in run_tests
exit=False,
File "/usr/lib/python3.5/unittest/main.py", line 94, in __init__
self.runTests()
File "/usr/lib/python3.5/unittest/main.py", line 255, in runTests
self.result = testRunner.run(self.test)
File "/usr/lib/python3.5/unittest/runner.py", line 161, in run
warnings.simplefilter(self.warnings)
File ".../beam/sdks/python/apache_beam/utils/annotations.py", line 105, in
my_simplefilter_patch
traceback.print_stack(file=sys.stdout)# <--- My monkey-patch.
```
So the behavior of annotations doesn't change on Python 3, but some
internals of unit test framework did. I lean towards removing
`warnings.simplefilter("once")` from our code, since it's not done right, and
removing unit tests that verify that behavior.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 156055)
Time Spent: 1h 40m (was: 1.5h)
> Util test on annotations fails
> ---
>
> Key: BEAM-5745
> URL: https://issues.apache.org/jira/browse/BEAM-5745
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-py-core
>Reporter: Ruoyun Huang
>Assignee: Ruoyun Huang
>Priority: Minor
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> Traceback (most recent call last):
> File
> "/usr/local/google/home/ruoyun/projects/beam/sdks/python/apache_beam/utils/annotations_test.py",
> line 142, in test_frequency
> label_check_list=[])
> File
> "/usr/local/google/home/ruoyun/projects/beam/sdks/python/apache_beam/utils/annotations_test.py",
> line 149, in check_annotation
> self.assertIn(fnc_name + ' is ' + annotation_type,
> str(warning[-1].message))
> AssertionError: 'fnc2_test_annotate_frequency is experimental' not found in
> 'fnc_test_annotate_frequency is experimental.'
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-5745) Util test on annotations fails
[ https://issues.apache.org/jira/browse/BEAM-5745?focusedWorklogId=155786=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-155786 ] ASF GitHub Bot logged work on BEAM-5745: Author: ASF GitHub Bot Created on: 18/Oct/18 09:13 Start Date: 18/Oct/18 09:13 Worklog Time Spent: 10m Work Description: robertwb commented on issue #6687: [BEAM-5745] Fix annotation test for py3 URL: https://github.com/apache/beam/pull/6687#issuecomment-430935698 Thanks for the detailed analysis, @tvalentyn . I agree both with your assessment that we're just covering things up rather than finding the root cause, and that we should reconsider the whole annotations system. For the purpose of shipping a Python 3 compatible SDK, I propose that resolving this double-printing is low priority and we simply relax this test for Python 3 until all other kinks have been worked out. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 155786) Time Spent: 1.5h (was: 1h 20m) > Util test on annotations fails > --- > > Key: BEAM-5745 > URL: https://issues.apache.org/jira/browse/BEAM-5745 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-core >Reporter: Ruoyun Huang >Assignee: Ruoyun Huang >Priority: Minor > Time Spent: 1.5h > Remaining Estimate: 0h > > Traceback (most recent call last): > File > "/usr/local/google/home/ruoyun/projects/beam/sdks/python/apache_beam/utils/annotations_test.py", > line 142, in test_frequency > label_check_list=[]) > File > "/usr/local/google/home/ruoyun/projects/beam/sdks/python/apache_beam/utils/annotations_test.py", > line 149, in check_annotation > self.assertIn(fnc_name + ' is ' + annotation_type, > str(warning[-1].message)) > AssertionError: 'fnc2_test_annotate_frequency is experimental' not found in > 'fnc_test_annotate_frequency is experimental.' -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-5745) Util test on annotations fails
[
https://issues.apache.org/jira/browse/BEAM-5745?focusedWorklogId=155734=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-155734
]
ASF GitHub Bot logged work on BEAM-5745:
Author: ASF GitHub Bot
Created on: 18/Oct/18 07:42
Start Date: 18/Oct/18 07:42
Worklog Time Spent: 10m
Work Description: tvalentyn commented on issue #6687: [BEAM-5745] Fix
annotation test for py3
URL: https://github.com/apache/beam/pull/6687#issuecomment-430907854
1. The goal of `annotations.py` is to create a mechanism that will allow to
print a warning message when some Beam APIs are used. It is well documented in
the docstring
https://github.com/apache/beam/blob/0acb027cd7d909732e5a07ed0d2cbbedf5fbc907/sdks/python/apache_beam/utils/annotations.py#L18
My intuition says we may be reinventing the wheel here, but without further
research I don't have a recommendation at the moment.
2. One aspect of existing implementation is that we want such warnings be
seen only once, if possible, to avoid noise. This is achieved by adding a
warning filter, see:
https://docs.python.org/3/library/warnings.html#warning-filter. Unfortunately,
the restriction we impose in
https://github.com/apache/beam/blob/0acb027cd7d909732e5a07ed0d2cbbedf5fbc907/sdks/python/apache_beam/utils/annotations.py#L94
is very broad, and will apply to **all** warnings which will be generated, not
just warnings produced by annotated functions. This may have unintended side
effects beyond the scope of the code the restriction is meant for. This is what
@robertwb called out.
3. To understand why tests started to fail, we can add look into the filter
list. Everytime we add a new warning filter, it is added to the beginning list,
the order of filter application is defined in the documentation (see above). At
any point, we can print current filter list with:
```
import pprint
pprint(warnings.filter)
```
By the time we run the annotated functions in `test_frequency`, the filter
list on Python 2 looks like this:
```
...
('always', None, , None, 0), #
<--- Filter added by numpy library after our filter.
('once', None, , None, 0), # <--- Filter
added by us. Notice that it applies to all warnings.
('ignore', None, , None, 0),
# <--- Filters added before our filter.
...
```
On Python 3, the filter list looks like this:
```
...
('default', None, , None, 0), # <--- !!! Something
adds this rule.
...
('always', None, , None, 0),
('once', None, , None, 0) # < Rule we add
...
```
``` ('default', None, , None, 0)``` overrides our behavior
to print warnings only once. Therefore, the deprecation warnings are printed
twice and the test fails. Might be interesting to track down what adds this
rule. By setting `warnings.simplefilter("once")` inside `annotate` method, as
this PR proposes, we basically start a fight with somebody over whose rule will
be at the top.
I think we should reconsider whether we want to keep our annotation
mechanism, if so, we need a better way to manage the warnings. Either not add
any restrictions, or otherwise find a proper way to localize them. cc:
@robertwb @aaltay
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 155734)
Time Spent: 1h 20m (was: 1h 10m)
> Util test on annotations fails
> ---
>
> Key: BEAM-5745
> URL: https://issues.apache.org/jira/browse/BEAM-5745
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-py-core
>Reporter: Ruoyun Huang
>Assignee: Ruoyun Huang
>Priority: Minor
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> Traceback (most recent call last):
> File
> "/usr/local/google/home/ruoyun/projects/beam/sdks/python/apache_beam/utils/annotations_test.py",
> line 142, in test_frequency
> label_check_list=[])
> File
> "/usr/local/google/home/ruoyun/projects/beam/sdks/python/apache_beam/utils/annotations_test.py",
> line 149, in check_annotation
>
[jira] [Work logged] (BEAM-5745) Util test on annotations fails
[ https://issues.apache.org/jira/browse/BEAM-5745?focusedWorklogId=155703=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-155703 ] ASF GitHub Bot logged work on BEAM-5745: Author: ASF GitHub Bot Created on: 18/Oct/18 05:44 Start Date: 18/Oct/18 05:44 Worklog Time Spent: 10m Work Description: tvalentyn commented on issue #6687: [BEAM-5745] Fix annotation test for py3 URL: https://github.com/apache/beam/pull/6687#issuecomment-430882473 The original Python 2 implementation has issues to begin with... will add an extended comment in a bit. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 155703) Time Spent: 1h 10m (was: 1h) > Util test on annotations fails > --- > > Key: BEAM-5745 > URL: https://issues.apache.org/jira/browse/BEAM-5745 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-core >Reporter: Ruoyun Huang >Assignee: Ruoyun Huang >Priority: Minor > Time Spent: 1h 10m > Remaining Estimate: 0h > > Traceback (most recent call last): > File > "/usr/local/google/home/ruoyun/projects/beam/sdks/python/apache_beam/utils/annotations_test.py", > line 142, in test_frequency > label_check_list=[]) > File > "/usr/local/google/home/ruoyun/projects/beam/sdks/python/apache_beam/utils/annotations_test.py", > line 149, in check_annotation > self.assertIn(fnc_name + ' is ' + annotation_type, > str(warning[-1].message)) > AssertionError: 'fnc2_test_annotate_frequency is experimental' not found in > 'fnc_test_annotate_frequency is experimental.' -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-5745) Util test on annotations fails
[ https://issues.apache.org/jira/browse/BEAM-5745?focusedWorklogId=10=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-10 ] ASF GitHub Bot logged work on BEAM-5745: Author: ASF GitHub Bot Created on: 17/Oct/18 18:38 Start Date: 17/Oct/18 18:38 Worklog Time Spent: 10m Work Description: HuangLED edited a comment on issue #6687: [BEAM-5745] Fix annotation test for py3 URL: https://github.com/apache/beam/pull/6687#issuecomment-430734886 > It seems odd to set this globally once a function is called (and not in a context). An alternative way, for the sake of fixing this failing test, is to explicitly call warnings.simplefilter() in each test case. This Option#2 of fix works as well. The downside though, is that the discrepency between py2 and py3 would still exist. My hypothesis on the root cause, is that python2 and python3 treat unscoped global function call differently. (funny thing is, this warning.simplefilter is called in py3, there is something else under the hood for which I don't know how) Didn't find a definitive documentation on this though, please fill in if you are aware of good discussions on this. (+python experts @pabloem @tvalentyn ) Weighing these two options, I slightly prefer option#1 in this PR because with this fix, this module by itself behave exactly the same behavior regardless in an py2 or py3 environment. And I don't think there is performance concern. The downside here is the code does not look elegant enough. to wrap it up: Option#1: Pro: this annotations.py module behaves the same in py2 and py3 ; Cons: Code does not look elegant. Option#2: Pro: only need to change test file. Cons: We will need to handle the py2/py3 discrepency in all related test cases, and will need to do that in the future non-testing module if anything depend on this annotations.py. Other alternative, feel free to recommend. I'd be happy to try out. Or, if our consensus is to go with Option#2, I shall update this PR accordingly. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 10) Time Spent: 1h (was: 50m) > Util test on annotations fails > --- > > Key: BEAM-5745 > URL: https://issues.apache.org/jira/browse/BEAM-5745 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-core >Reporter: Ruoyun Huang >Assignee: Ruoyun Huang >Priority: Minor > Time Spent: 1h > Remaining Estimate: 0h > > Traceback (most recent call last): > File > "/usr/local/google/home/ruoyun/projects/beam/sdks/python/apache_beam/utils/annotations_test.py", > line 142, in test_frequency > label_check_list=[]) > File > "/usr/local/google/home/ruoyun/projects/beam/sdks/python/apache_beam/utils/annotations_test.py", > line 149, in check_annotation > self.assertIn(fnc_name + ' is ' + annotation_type, > str(warning[-1].message)) > AssertionError: 'fnc2_test_annotate_frequency is experimental' not found in > 'fnc_test_annotate_frequency is experimental.' -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-5745) Util test on annotations fails
[ https://issues.apache.org/jira/browse/BEAM-5745?focusedWorklogId=155542=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-155542 ] ASF GitHub Bot logged work on BEAM-5745: Author: ASF GitHub Bot Created on: 17/Oct/18 18:20 Start Date: 17/Oct/18 18:20 Worklog Time Spent: 10m Work Description: HuangLED commented on issue #6687: [BEAM-5745] Fix annotation test for py3 URL: https://github.com/apache/beam/pull/6687#issuecomment-430734886 > It seems odd to set this globally once a function is called (and not in a context). An alternative way, for the sake of fixing this failing test, is to explicitly call warnings.simplefilter() in each test case. This Option#2 of fix works as well. The downside though, is that the discrepency between py2 and py3 would still exist. My hypothesis on the root cause, is that python2 and python3 treat unscoped global function call differently. (funny thing is, this warning.simplefilter is called in py3, there is something else under the hood for which I don't know how) Didn't find a definitive documentation on this though, please fill in if you are aware of good discussions on this. (+python experts @pabloem @tvalentyn ) Weighing these two options, I prefer option#1 in this PR because with this fix, this module by itself behave exactly the same behavior regardless in an py2 or py3 environment. And I don't think there is performance concern. The downside here is the code does not look elegant enough. to wrap it up: Option#1: Pro: this annotations.py module behaves the same in py2 and py3 ; Cons: Code does not look elegant. Option#2: Pro: only need to change test file. Cons: We will need to handle the py2/py3 discrepency in all related test cases, and will need to do that in the future non-testing module if anything depend on this annotations.py. Please make recommendations if you know an Option#3. Thanks. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 155542) Time Spent: 50m (was: 40m) > Util test on annotations fails > --- > > Key: BEAM-5745 > URL: https://issues.apache.org/jira/browse/BEAM-5745 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-core >Reporter: Ruoyun Huang >Assignee: Ruoyun Huang >Priority: Minor > Time Spent: 50m > Remaining Estimate: 0h > > Traceback (most recent call last): > File > "/usr/local/google/home/ruoyun/projects/beam/sdks/python/apache_beam/utils/annotations_test.py", > line 142, in test_frequency > label_check_list=[]) > File > "/usr/local/google/home/ruoyun/projects/beam/sdks/python/apache_beam/utils/annotations_test.py", > line 149, in check_annotation > self.assertIn(fnc_name + ' is ' + annotation_type, > str(warning[-1].message)) > AssertionError: 'fnc2_test_annotate_frequency is experimental' not found in > 'fnc_test_annotate_frequency is experimental.' -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-5745) Util test on annotations fails
[ https://issues.apache.org/jira/browse/BEAM-5745?focusedWorklogId=154625=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-154625 ] ASF GitHub Bot logged work on BEAM-5745: Author: ASF GitHub Bot Created on: 16/Oct/18 07:27 Start Date: 16/Oct/18 07:27 Worklog Time Spent: 10m Work Description: robertwb commented on issue #6687: [BEAM-5745] Fix annotation test for py3 URL: https://github.com/apache/beam/pull/6687#issuecomment-430130441 It seems odd to set this globally once a function is called (and not in a context). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 154625) Time Spent: 40m (was: 0.5h) > Util test on annotations fails > --- > > Key: BEAM-5745 > URL: https://issues.apache.org/jira/browse/BEAM-5745 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-core >Reporter: Ruoyun Huang >Assignee: Ruoyun Huang >Priority: Minor > Time Spent: 40m > Remaining Estimate: 0h > > Traceback (most recent call last): > File > "/usr/local/google/home/ruoyun/projects/beam/sdks/python/apache_beam/utils/annotations_test.py", > line 142, in test_frequency > label_check_list=[]) > File > "/usr/local/google/home/ruoyun/projects/beam/sdks/python/apache_beam/utils/annotations_test.py", > line 149, in check_annotation > self.assertIn(fnc_name + ' is ' + annotation_type, > str(warning[-1].message)) > AssertionError: 'fnc2_test_annotate_frequency is experimental' not found in > 'fnc_test_annotate_frequency is experimental.' -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-5745) Util test on annotations fails
[ https://issues.apache.org/jira/browse/BEAM-5745?focusedWorklogId=154537=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-154537 ] ASF GitHub Bot logged work on BEAM-5745: Author: ASF GitHub Bot Created on: 15/Oct/18 23:55 Start Date: 15/Oct/18 23:55 Worklog Time Spent: 10m Work Description: HuangLED commented on issue #6687: [BEAM-5745] Fix annotation test for py3 URL: https://github.com/apache/beam/pull/6687#issuecomment-430053843 R: @tvalentyn beamimport tap test passes for this PR. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 154537) Time Spent: 0.5h (was: 20m) > Util test on annotations fails > --- > > Key: BEAM-5745 > URL: https://issues.apache.org/jira/browse/BEAM-5745 > Project: Beam > Issue Type: Improvement > Components: sdk-py-core >Reporter: Ruoyun Huang >Assignee: Ruoyun Huang >Priority: Minor > Time Spent: 0.5h > Remaining Estimate: 0h > > Traceback (most recent call last): > File > "/usr/local/google/home/ruoyun/projects/beam/sdks/python/apache_beam/utils/annotations_test.py", > line 142, in test_frequency > label_check_list=[]) > File > "/usr/local/google/home/ruoyun/projects/beam/sdks/python/apache_beam/utils/annotations_test.py", > line 149, in check_annotation > self.assertIn(fnc_name + ' is ' + annotation_type, > str(warning[-1].message)) > AssertionError: 'fnc2_test_annotate_frequency is experimental' not found in > 'fnc_test_annotate_frequency is experimental.' -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-5745) Util test on annotations fails
[ https://issues.apache.org/jira/browse/BEAM-5745?focusedWorklogId=154174=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-154174 ] ASF GitHub Bot logged work on BEAM-5745: Author: ASF GitHub Bot Created on: 14/Oct/18 22:30 Start Date: 14/Oct/18 22:30 Worklog Time Spent: 10m Work Description: HuangLED commented on issue #6687: [BEAM-5745] Fix annotation test for py3 URL: https://github.com/apache/beam/pull/6687#issuecomment-429668942 Run Python PostCommit This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 154174) Time Spent: 20m (was: 10m) > Util test on annotations fails > --- > > Key: BEAM-5745 > URL: https://issues.apache.org/jira/browse/BEAM-5745 > Project: Beam > Issue Type: Improvement > Components: sdk-py-core >Reporter: Ruoyun Huang >Assignee: Ruoyun Huang >Priority: Minor > Time Spent: 20m > Remaining Estimate: 0h > > Traceback (most recent call last): > File > "/usr/local/google/home/ruoyun/projects/beam/sdks/python/apache_beam/utils/annotations_test.py", > line 142, in test_frequency > label_check_list=[]) > File > "/usr/local/google/home/ruoyun/projects/beam/sdks/python/apache_beam/utils/annotations_test.py", > line 149, in check_annotation > self.assertIn(fnc_name + ' is ' + annotation_type, > str(warning[-1].message)) > AssertionError: 'fnc2_test_annotate_frequency is experimental' not found in > 'fnc_test_annotate_frequency is experimental.' -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-5745) Util test on annotations fails
[ https://issues.apache.org/jira/browse/BEAM-5745?focusedWorklogId=154173=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-154173 ] ASF GitHub Bot logged work on BEAM-5745: Author: ASF GitHub Bot Created on: 14/Oct/18 22:28 Start Date: 14/Oct/18 22:28 Worklog Time Spent: 10m Work Description: HuangLED opened a new pull request #6687: [BEAM-5745] Fix annotation test for py3 URL: https://github.com/apache/beam/pull/6687 Root cause: warnings.simplefilter method have limited scope in py3, which makes the "once" filter fail. Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). It will help us expedite review of your Pull Request if you tag someone (e.g. `@username`) to look at it. Post-Commit Tests Status (on master branch) Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark --- | --- | --- | --- | --- | --- | --- | --- Go | [](https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/lastCompletedBuild/) | --- | --- | --- | --- | --- | --- Java | [](https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex_Gradle/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump_Gradle/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza_Gradle/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Gradle/lastCompletedBuild/) Python | [](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/) | --- | [](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/) [](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/lastCompletedBuild/) | --- | --- | --- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 154173) Time Spent: 10m Remaining Estimate: 0h > Util test on annotations fails > --- > > Key: BEAM-5745 > URL: https://issues.apache.org/jira/browse/BEAM-5745 > Project: Beam > Issue Type: Improvement > Components: sdk-py-core >
