[
https://issues.apache.org/jira/browse/BEAM-9639?focusedWorklogId=421580&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-421580
]
ASF GitHub Bot logged work on BEAM-9639:
----------------------------------------
Author: ASF GitHub Bot
Created on: 13/Apr/20 19:38
Start Date: 13/Apr/20 19:38
Worklog Time Spent: 10m
Work Description: robertwb commented on pull request #11270:
[BEAM-9639][BEAM-9608] Improvements for FnApiRunner
URL: https://github.com/apache/beam/pull/11270#discussion_r407663602
##########
File path:
sdks/python/apache_beam/runners/portability/fn_api_runner/execution.py
##########
@@ -367,6 +413,73 @@ def _build_process_bundle_descriptor(self):
state_api_service_descriptor=self.state_api_service_descriptor(),
timer_api_service_descriptor=self.data_api_service_descriptor())
+ def commit_output_views_to_state(self):
+ """Commit bundle outputs to state to be consumed as side inputs later.
+
+ Only the outputs that should be side inputs are committed to state.
+ """
+ data_side_input = {} # type: DataSideInput
+ for pcoll, si_ids in self.stage.downstream_side_inputs.items():
+ for (consumer_transform_name, tag), access_pattern in si_ids.items():
+ data_side_input[consumer_transform_name, tag] = (
+ translations.create_buffer_id(pcoll), access_pattern)
+ self.execution_context.commit_side_inputs_to_state(data_side_input)
+
+ def extract_bundle_inputs(self):
Review comment:
extract_bundle_inputs_and_outputs?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 421580)
Time Spent: 1.5h (was: 1h 20m)
> Abstract bundle execution logic from stage execution logic
> ----------------------------------------------------------
>
> Key: BEAM-9639
> URL: https://issues.apache.org/jira/browse/BEAM-9639
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-py-core
> Reporter: Pablo Estrada
> Assignee: Pablo Estrada
> Priority: Major
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> The FnApiRunner currently works on a per-stage manner, and does not abstract
> single-bundle execution much. This work item is to clearly define the code to
> execute a single bundle.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
