[
https://issues.apache.org/jira/browse/BEAM-8837?focusedWorklogId=357446&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-357446
]
ASF GitHub Bot logged work on BEAM-8837:
----------------------------------------
Author: ASF GitHub Bot
Created on: 10/Dec/19 21:42
Start Date: 10/Dec/19 21:42
Worklog Time Spent: 10m
Work Description: KevinGG commented on pull request #10321: [BEAM-8837]
Fix pcoll_visualization tests
URL: https://github.com/apache/beam/pull/10321#discussion_r356293018
##########
File path:
sdks/python/apache_beam/runners/interactive/display/pcoll_visualization_test.py
##########
@@ -47,16 +49,19 @@
class PCollectionVisualizationTest(unittest.TestCase):
def setUp(self):
+ ie.new_env()
# Allow unit test to run outside of ipython kernel since we don't test the
# frontend rendering in unit tests.
pv._pcoll_visualization_ready = True
# Generally test the logic where notebook is connected to the assumed
# ipython kernel by forcefully setting notebook check to True.
ie.current_env()._is_in_notebook = True
- self._p = beam.Pipeline()
+ self._p = beam.Pipeline(ir.InteractiveRunner())
+ ib.watch(self)
# pylint: disable=range-builtin-not-iterating
self._pcoll = self._p | 'Create' >> beam.Create(range(1000))
+ ib.watch(self)
Review comment:
The first time is to watch the locals including the pipeline `self._p`.
Then during the pipeline construction time, Interactive Beam will implicitly
modify the labels such as `Create` if the code is executed in ipython to
something like `Cell 1: Create`.
The second watch is to notify Interactive Beam about the PCollections that
have been defined so far.
Then when pipeline gets executed, Interactive Beam will cache for
PCollections watched in the local scope. The cache will be available even if
the `pipeline.run()` or data visualization/introspection is not in the local
scope later on.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 357446)
Time Spent: 1h 10m (was: 1h)
> PCollectionVisualizationTest: possible bug
> ------------------------------------------
>
> Key: BEAM-8837
> URL: https://issues.apache.org/jira/browse/BEAM-8837
> Project: Beam
> Issue Type: Improvement
> Components: sdk-py-core
> Reporter: Udi Meiri
> Assignee: Ning Kang
> Priority: Major
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> This seems like a bug, even though the test passes:
> {code}
> test_display_plain_text_when_kernel_has_no_frontend
> (apache_beam.runners.interactive.display.pcoll_visualization_test.PCollectionVisualizationTest)
> ... Exception in thread Thread-4405:
> Traceback (most recent call last):
> File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
> self.run()
> File
> "/usr/local/google/home/ehudm/src/beam/sdks/python/test-suites/tox/py37/build/srcs/sdks/python/.eggs/timeloop-1.0.2-py3.7.egg/timeloop/job.py",
> line 19, in run
> self.execute(*self.args, **self.kwargs)
> File
> "/usr/local/google/home/ehudm/src/beam/sdks/python/test-suites/tox/py37/build/srcs/sdks/python/apache_beam/runners/interactive/display/pcoll_visualization.py",
> line 132, in continuous_update_display
> updated_pv.display_facets(updating_pv=pv)
> File
> "/usr/local/google/home/ehudm/src/beam/sdks/python/test-suites/tox/py37/build/srcs/sdks/python/apache_beam/runners/interactive/display/pcoll_visualization.py",
> line 209, in display_facets
> data = self._to_dataframe()
> File
> "/usr/local/google/home/ehudm/src/beam/sdks/python/test-suites/tox/py37/build/srcs/sdks/python/apache_beam/runners/interactive/display/pcoll_visualization.py",
> line 278, in _to_dataframe
> for el in self._to_element_list():
> File
> "/usr/local/google/home/ehudm/src/beam/sdks/python/test-suites/tox/py37/build/srcs/sdks/python/apache_beam/runners/interactive/display/pcoll_visualization.py",
> line 266, in _to_element_list
> if ie.current_env().cache_manager().exists('full', self._cache_key):
> AttributeError: 'NoneType' object has no attribute 'exists'
> ok
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
