[jira] [Work logged] (BEAM-9186) Allow passing a comparison function for assert_that(..., equal_to()) tests
[ https://issues.apache.org/jira/browse/BEAM-9186?focusedWorklogId=378043=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-378043 ] ASF GitHub Bot logged work on BEAM-9186: Author: ASF GitHub Bot Created on: 28/Jan/20 05:24 Start Date: 28/Jan/20 05:24 Worklog Time Spent: 10m Work Description: tvalentyn commented on issue #10637: [BEAM-9186] Allow injection of custom equality function. URL: https://github.com/apache/beam/pull/10637#issuecomment-579085878 Thanks, @sorensenjs ! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 378043) Time Spent: 1h 10m (was: 1h) > Allow passing a comparison function for assert_that(..., equal_to()) tests > -- > > Key: BEAM-9186 > URL: https://issues.apache.org/jira/browse/BEAM-9186 > Project: Beam > Issue Type: New Feature > Components: testing >Reporter: Jeffrey Sorensen >Priority: Minor > Time Spent: 1h 10m > Remaining Estimate: 0h > > Currently testing beam pipelines that produce objects that cannot be sorted > or compared is currently problematical, and developers are introducing > Map(str) and similar transforms in tests to transform PCollections into a > testable form. Adding by injection a comparison function will allow the > creation of more comprehensible tests. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9186) Allow passing a comparison function for assert_that(..., equal_to()) tests
[ https://issues.apache.org/jira/browse/BEAM-9186?focusedWorklogId=378042=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-378042 ] ASF GitHub Bot logged work on BEAM-9186: Author: ASF GitHub Bot Created on: 28/Jan/20 05:24 Start Date: 28/Jan/20 05:24 Worklog Time Spent: 10m Work Description: tvalentyn commented on pull request #10637: [BEAM-9186] Allow injection of custom equality function. URL: https://github.com/apache/beam/pull/10637 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 378042) Time Spent: 1h (was: 50m) > Allow passing a comparison function for assert_that(..., equal_to()) tests > -- > > Key: BEAM-9186 > URL: https://issues.apache.org/jira/browse/BEAM-9186 > Project: Beam > Issue Type: New Feature > Components: testing >Reporter: Jeffrey Sorensen >Priority: Minor > Time Spent: 1h > Remaining Estimate: 0h > > Currently testing beam pipelines that produce objects that cannot be sorted > or compared is currently problematical, and developers are introducing > Map(str) and similar transforms in tests to transform PCollections into a > testable form. Adding by injection a comparison function will allow the > creation of more comprehensible tests. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9186) Allow passing a comparison function for assert_that(..., equal_to()) tests
[ https://issues.apache.org/jira/browse/BEAM-9186?focusedWorklogId=378021=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-378021 ] ASF GitHub Bot logged work on BEAM-9186: Author: ASF GitHub Bot Created on: 28/Jan/20 02:59 Start Date: 28/Jan/20 02:59 Worklog Time Spent: 10m Work Description: tvalentyn commented on issue #10637: [BEAM-9186] Allow injection of custom equality function. URL: https://github.com/apache/beam/pull/10637#issuecomment-579056939 Run Python PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 378021) Time Spent: 40m (was: 0.5h) > Allow passing a comparison function for assert_that(..., equal_to()) tests > -- > > Key: BEAM-9186 > URL: https://issues.apache.org/jira/browse/BEAM-9186 > Project: Beam > Issue Type: New Feature > Components: testing >Reporter: Jeffrey Sorensen >Priority: Minor > Time Spent: 40m > Remaining Estimate: 0h > > Currently testing beam pipelines that produce objects that cannot be sorted > or compared is currently problematical, and developers are introducing > Map(str) and similar transforms in tests to transform PCollections into a > testable form. Adding by injection a comparison function will allow the > creation of more comprehensible tests. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9186) Allow passing a comparison function for assert_that(..., equal_to()) tests
[ https://issues.apache.org/jira/browse/BEAM-9186?focusedWorklogId=378022=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-378022 ] ASF GitHub Bot logged work on BEAM-9186: Author: ASF GitHub Bot Created on: 28/Jan/20 02:59 Start Date: 28/Jan/20 02:59 Worklog Time Spent: 10m Work Description: tvalentyn commented on issue #10637: [BEAM-9186] Allow injection of custom equality function. URL: https://github.com/apache/beam/pull/10637#issuecomment-579056983 Run PythonLint Precommit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 378022) Time Spent: 50m (was: 40m) > Allow passing a comparison function for assert_that(..., equal_to()) tests > -- > > Key: BEAM-9186 > URL: https://issues.apache.org/jira/browse/BEAM-9186 > Project: Beam > Issue Type: New Feature > Components: testing >Reporter: Jeffrey Sorensen >Priority: Minor > Time Spent: 50m > Remaining Estimate: 0h > > Currently testing beam pipelines that produce objects that cannot be sorted > or compared is currently problematical, and developers are introducing > Map(str) and similar transforms in tests to transform PCollections into a > testable form. Adding by injection a comparison function will allow the > creation of more comprehensible tests. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9186) Allow passing a comparison function for assert_that(..., equal_to()) tests
[ https://issues.apache.org/jira/browse/BEAM-9186?focusedWorklogId=378020=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-378020 ] ASF GitHub Bot logged work on BEAM-9186: Author: ASF GitHub Bot Created on: 28/Jan/20 02:51 Start Date: 28/Jan/20 02:51 Worklog Time Spent: 10m Work Description: tvalentyn commented on issue #10637: [BEAM-9186] Allow injection of custom equality function. URL: https://github.com/apache/beam/pull/10637#issuecomment-579055320 Run PythonLint Precommit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 378020) Time Spent: 0.5h (was: 20m) > Allow passing a comparison function for assert_that(..., equal_to()) tests > -- > > Key: BEAM-9186 > URL: https://issues.apache.org/jira/browse/BEAM-9186 > Project: Beam > Issue Type: New Feature > Components: testing >Reporter: Jeffrey Sorensen >Priority: Minor > Time Spent: 0.5h > Remaining Estimate: 0h > > Currently testing beam pipelines that produce objects that cannot be sorted > or compared is currently problematical, and developers are introducing > Map(str) and similar transforms in tests to transform PCollections into a > testable form. Adding by injection a comparison function will allow the > creation of more comprehensible tests. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9186) Allow passing a comparison function for assert_that(..., equal_to()) tests
[ https://issues.apache.org/jira/browse/BEAM-9186?focusedWorklogId=378019=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-378019 ] ASF GitHub Bot logged work on BEAM-9186: Author: ASF GitHub Bot Created on: 28/Jan/20 02:51 Start Date: 28/Jan/20 02:51 Worklog Time Spent: 10m Work Description: tvalentyn commented on issue #10637: [BEAM-9186] Allow injection of custom equality function. URL: https://github.com/apache/beam/pull/10637#issuecomment-579055282 Run Python PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 378019) Time Spent: 20m (was: 10m) > Allow passing a comparison function for assert_that(..., equal_to()) tests > -- > > Key: BEAM-9186 > URL: https://issues.apache.org/jira/browse/BEAM-9186 > Project: Beam > Issue Type: New Feature > Components: testing >Reporter: Jeffrey Sorensen >Priority: Minor > Time Spent: 20m > Remaining Estimate: 0h > > Currently testing beam pipelines that produce objects that cannot be sorted > or compared is currently problematical, and developers are introducing > Map(str) and similar transforms in tests to transform PCollections into a > testable form. Adding by injection a comparison function will allow the > creation of more comprehensible tests. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9186) Allow passing a comparison function for assert_that(..., equal_to()) tests
[ https://issues.apache.org/jira/browse/BEAM-9186?focusedWorklogId=377008=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-377008 ] ASF GitHub Bot logged work on BEAM-9186: Author: ASF GitHub Bot Created on: 24/Jan/20 19:36 Start Date: 24/Jan/20 19:36 Worklog Time Spent: 10m Work Description: aaltay commented on pull request #10637: [BEAM-9186] Allow injection of custom equality function. URL: https://github.com/apache/beam/pull/10637#discussion_r370807791 ## File path: sdks/python/apache_beam/testing/util.py ## @@ -154,41 +154,48 @@ def equal_to_per_window(expected_window_to_elements): # Note that equal_to checks if expected and actual are permutations of each # other. However, only permutations of the top level are checked. Therefore # [1,2] and [2,1] are considered equal and [[1,2]] and [[2,1]] are not. -def equal_to(expected): +def equal_to(expected, equals_fn=None): - def _equal(actual): + def _equal(actual, equals_fn=equals_fn): expected_list = list(expected) # Try to compare actual and expected by sorting. This fails with a # TypeError in Python 3 if different types are present in the same # collection. It can also raise false negatives for types that don't have # a deterministic sort order, like pyarrow Tables as of 0.14.1 -try: - sorted_expected = sorted(expected) - sorted_actual = sorted(actual) - if sorted_expected != sorted_actual: -raise BeamAssertException( -'Failed assert: %r == %r' % (sorted_expected, sorted_actual)) +if not equals_fn: + try: +sorted_expected = sorted(expected) +sorted_actual = sorted(actual) +if sorted_expected == sorted_actual: + return + except (BeamAssertException, TypeError): Review comment: We can remove `BeamAssertException` from here. The remaining code in the block would not raise that exception. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 377008) Remaining Estimate: 0h Time Spent: 10m > Allow passing a comparison function for assert_that(..., equal_to()) tests > -- > > Key: BEAM-9186 > URL: https://issues.apache.org/jira/browse/BEAM-9186 > Project: Beam > Issue Type: New Feature > Components: testing >Reporter: Jeffrey >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > Currently testing beam pipelines that produce objects that cannot be sorted > or compared is currently problematical, and developers are introducing > Map(str) and similar transforms in tests to transform PCollections into a > testable form. Adding by injection a comparison function will allow the > creation of more comprehensible tests. -- This message was sent by Atlassian Jira (v8.3.4#803005)
