[beam] branch master updated: Fix Python 3 compatibility bug in pickler.py

[ccy]

This is an automated email from the ASF dual-hosted git repository.

ccy pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
     new 46a70a8  Fix Python 3 compatibility bug in pickler.py
     new 7ddf735  Merge pull request #7953 from udim/pickler
46a70a8 is described below

commit 46a70a8b4691a169c9e018299765e1bdb88239f4
Author: Udi Meiri <eh...@google.com>
AuthorDate: Tue Feb 26 18:03:32 2019 -0800

    Fix Python 3 compatibility bug in pickler.py
    
    sys.modules changes during iteration. It seems that the intention was to
    make a copy using sys.modules.values(), which returns a copy in Python
    2, but a view in Python 3.
    
    The issue comes up when using pytest, which implements lazy module
    loading.
---
 sdks/python/apache_beam/internal/pickler.py | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/sdks/python/apache_beam/internal/pickler.py 
b/sdks/python/apache_beam/internal/pickler.py
index 234914a..29e4f4b 100644
--- a/sdks/python/apache_beam/internal/pickler.py
+++ b/sdks/python/apache_beam/internal/pickler.py
@@ -164,6 +164,17 @@ if 'save_module' in dir(dill.dill):
     obj_id = id(obj)
     if not known_module_dicts or '__file__' in obj or '__package__' in obj:
       if obj_id not in known_module_dicts:
+        # Trigger loading of lazily loaded modules (such as pytest vendored
+        # modules).
+        # This first pass over sys.modules needs to iterate on a copy of
+        # sys.modules since lazy loading modifies the dictionary, hence the use
+        # of list().
+        for m in list(sys.modules.values()):
+          try:
+            _ = m.__dict__
+          except AttributeError:
+            pass
+
         for m in sys.modules.values():
           try:
             if (m