[
https://issues.apache.org/jira/browse/BEAM-6877?focusedWorklogId=283400&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-283400
]
ASF GitHub Bot logged work on BEAM-6877:
----------------------------------------
Author: ASF GitHub Bot
Created on: 26/Jul/19 14:47
Start Date: 26/Jul/19 14:47
Worklog Time Spent: 10m
Work Description: tvalentyn commented on pull request #8893: [BEAM-6877]
trivial_inference: make remaining tests pass
URL: https://github.com/apache/beam/pull/8893#discussion_r307759872
##########
File path: sdks/python/apache_beam/typehints/trivial_inference.py
##########
@@ -458,10 +458,18 @@ def infer_return_type_func(f, input_types, debug=False,
depth=0):
pop_count = arg + 2
return_type = Any
elif opname == 'CALL_FUNCTION_EX':
- # TODO(udim): Handle variable argument lists. Requires handling
kwargs
- # first.
- pop_count = (arg & 1) + 3
- return_type = Any
+ has_kwargs = arg & 1 # type: int
+ pop_count = has_kwargs + 1
+ if has_kwargs:
+ # TODO(udim): Unimplemented. Requires same functionality as a
+ # CALL_FUNCTION_KW implementation.
+ return_type = Any
+ else:
+ args_and_callable = state.stack[-1]
Review comment:
I was reading https://docs.python.org/3/library/dis.html, which says:
> CALL_FUNCTION_EX(flags)
> Calls a callable object with variable set of positional and keyword
arguments. If the lowest bit of flags is set, the top of the stack contains a
mapping object containing additional keyword arguments. **Below that is an
iterable object containing positional arguments and a callable object to
call.** BUILD_MAP_UNPACK_WITH_CALL and BUILD_TUPLE_UNPACK_WITH_CALL can be used
for merging multiple mapping objects and iterables containing arguments. Before
the callable is called, the mapping object and iterable object are each
“unpacked” and their contents passed in as keyword and positional arguments
respectively. CALL_FUNCTION_EX pops all arguments and the callable object off
the stack, calls the callable object with those arguments, and pushes the
return value returned by the callable object.
>
I was not sure whether the phrase in bold means that callable and arguments
are stored in the one element of the stack `state.stack[-1]`, or arg list and
callable are stored in two consecutive elements of the stack below, i.e. arg
in state.stack[-1], and callable in state.stack[-2].
I wrote this snippet to experiment and try to find the answer:
```
from apache_beam.typehints import trivial_inference
def f(*kwargs):
return kwargs[0] + kwargs[1]
def g():
return f(*[5, 6])
def run(argv=None):
print(g())
trivial_inference.infer_return_type_func(g, [], debug=True)
if __name__ == '__main__':
run()
```
When I run this code on this PR, I get an error, that does not happen on
master. I used Python 3.6.
(py3) :python$ python -m test
```
/home/valentyn/projects/beam/beam/beam/sdks/python/apache_beam/__init__.py:84:
UserWarning: Some syntactic constructs of Python 3 are not yet fully supported
by Apache Beam.
'Some syntactic constructs of Python 3 are not yet fully supported by '
11
#####
<function g at 0x7f8a2d7ac9d8> 140231445236184 []
6 0 LOAD_GLOBAL 0 (f)
2 LOAD_CONST 1 (5)
4 LOAD_CONST 2 (6)
6 BUILD_LIST 2
8 CALL_FUNCTION_EX 0
10 RETURN_VALUE
0 LOAD_GLOBAL 0 (f) Executing simple op LOAD_GLOBAL
Stack: [Const[<function f at 0x7f8a2d7ac950>]] Vars: []
{}
2 LOAD_CONST 1 (5) Executing simple op LOAD_CONST
Stack: [Const[<function f at 0x7f8a2d7ac950>], Const[5]] Vars: []
{}
4 LOAD_CONST 2 (6) Executing simple op LOAD_CONST
Stack: [Const[<function f at 0x7f8a2d7ac950>], Const[5], Const[6]] Vars: []
{}
6 BUILD_LIST 2 Executing simple op BUILD_LIST
Stack: [Const[<function f at 0x7f8a2d7ac950>], List[int]] Vars: []
{}
8 CALL_FUNCTION_EX 0 has kwargs: 0
Traceback (most recent call last):
File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/home/valentyn/projects/beam/beam/beam/sdks/python/test.py", line
14, in <module>
run()
File "/home/valentyn/projects/beam/beam/beam/sdks/python/test.py", line
10, in run
trivial_inference.infer_return_type_func(g, [], debug=True)
File
"/home/valentyn/projects/beam/beam/beam/sdks/python/apache_beam/typehints/trivial_inference.py",
line 471, in infer_return_type_func
return_type = infer_return_type(args_and_callable[-1].value,
TypeError: 'ListConstraint' object does not support indexing
```
What do you think, @udim?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 283400)
Time Spent: 11h 40m (was: 11.5h)
> TypeHints Py3 Error: Type inference tests fail on Python 3.6 due to bytecode
> changes
> ------------------------------------------------------------------------------------
>
> Key: BEAM-6877
> URL: https://issues.apache.org/jira/browse/BEAM-6877
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-py-core
> Reporter: Robbe
> Assignee: Udi Meiri
> Priority: Major
> Time Spent: 11h 40m
> Remaining Estimate: 0h
>
> Type inference doesn't work on Python 3.6 due to [bytecode to wordcode
> changes|https://docs.python.org/3/whatsnew/3.6.html#cpython-bytecode-changes].
> Type inference always returns Any on Python 3.6, so this is not critical.
> Affected tests are:
> *transforms.ptransform_test*:
> - test_combine_properly_pipeline_type_checks_using_decorator
> - test_mean_globally_pipeline_checking_satisfied
> - test_mean_globally_runtime_checking_satisfied
> - test_count_globally_pipeline_type_checking_satisfied
> - test_count_globally_runtime_type_checking_satisfied
> - test_pardo_type_inference
> - test_pipeline_inference
> - test_inferred_bad_kv_type
> *typehints.trivial_inference_test*:
> - all tests in TrivialInferenceTest
> *io.gcp.pubsub_test.TestReadFromPubSubOverride*:
> * test_expand_with_other_options
> * test_expand_with_subscription
> * test_expand_with_topic
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
