[
https://issues.apache.org/jira/browse/BEAM-10409?focusedWorklogId=474966&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-474966
]
ASF GitHub Bot logged work on BEAM-10409:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 26/Aug/20 19:11
Start Date: 26/Aug/20 19:11
Worklog Time Spent: 10m
Work Description: codecov[bot] edited a comment on pull request #12185:
URL: https://github.com/apache/beam/pull/12185#issuecomment-681052485
# [Codecov](https://codecov.io/gh/apache/beam/pull/12185?src=pr&el=h1) Report
> :exclamation: No coverage uploaded for pull request base
(`master@1e969b4`). [Click here to learn what that
means](https://docs.codecov.io/docs/error-reference#section-missing-base-commit).
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/beam/pull/12185?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #12185 +/- ##
=========================================
Coverage ? 34.30%
=========================================
Files ? 695
Lines ? 82362
Branches ? 9323
=========================================
Hits ? 28251
Misses ? 53688
Partials ? 423
```
------
[Continue to review full report at
Codecov](https://codecov.io/gh/apache/beam/pull/12185?src=pr&el=continue).
> **Legend** - [Click here to learn
more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by
[Codecov](https://codecov.io/gh/apache/beam/pull/12185?src=pr&el=footer). Last
update
[1e969b4...a2ab49c](https://codecov.io/gh/apache/beam/pull/12185?src=pr&el=lastupdated).
Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 474966)
Time Spent: 5h 20m (was: 5h 10m)
> Add combiner packing to graph optimizer phases
> ----------------------------------------------
>
> Key: BEAM-10409
> URL: https://issues.apache.org/jira/browse/BEAM-10409
> Project: Beam
> Issue Type: Improvement
> Components: runner-core
> Reporter: Yifan Mai
> Assignee: Yifan Mai
> Priority: P2
> Time Spent: 5h 20m
> Remaining Estimate: 0h
>
> Some use cases of Beam (e.g. [TensorFlow
> Transform|https://github.com/tensorflow/transform]) create thousands of
> Combine stages with a common parent. The large number of stages can cause
> performance issues on some runners. To alleviate, a graph optimization phase
> could be added to the translations module that packs compatible Combine
> stages into a single stage.
> The graph optimization for CombinePerKey would work as follows: If
> CombinePerKey stages have a common input, one input each, and one output
> each, pack the stages into a single stage that runs all CombinePerKeys and
> outputs resulting tuples to a new PCollection. A subsequent stage unpacks
> tuples from this PCollection and sends them to the original output
> PCollections.
> There is an additional issue with supporting this for CombineGlobally:
> because of the intermediate KeyWithVoid stage between the CombinePerKey
> stages and the input stage, the CombinePerKey stages do not have a common
> input stage, and cannot be packed. To support CombineGlobally, a common
> sibling elimination graph optimization phase can be used to combine the
> KeyWithVoid stages. After this, the CombinePerKey stages would have a common
> input and can be packed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
