[jira] [Created] (BEAM-8381) Typehints on DoFn subclass are incorrect set on superclass

Yifan Mai created BEAM-8381: --- Summary: Typehints on DoFn subclass are incorrect set on superclass Key: BEAM-8381 URL: https://issues.apache.org/jira/browse/BEAM-8381 Project: Beam Issue Type: Bug

[jira] [Created] (BEAM-8519) Typehint for Python CombineFn Accumulator

Yifan Mai created BEAM-8519: --- Summary: Typehint for Python CombineFn Accumulator Key: BEAM-8519 URL: https://issues.apache.org/jira/browse/BEAM-8519 Project: Beam Issue Type: New Feature

[jira] [Updated] (BEAM-8519) Typehint for Python CombineFn Accumulator

[ https://issues.apache.org/jira/browse/BEAM-8519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifan Mai updated BEAM-8519: Description: Feature request: Allow users to specify a typehint for the accumulator type of CombineFn, eithe

[jira] [Updated] (BEAM-8381) Typehints on DoFn subclass are incorrectly set on superclass

[ https://issues.apache.org/jira/browse/BEAM-8381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifan Mai updated BEAM-8381: Summary: Typehints on DoFn subclass are incorrectly set on superclass (was: Typehints on DoFn subclass are

[jira] [Created] (BEAM-8520) Expansion of _CombinePerKeyWithHotKeyFanout has no typehints

Yifan Mai created BEAM-8520: --- Summary: Expansion of _CombinePerKeyWithHotKeyFanout has no typehints Key: BEAM-8520 URL: https://issues.apache.org/jira/browse/BEAM-8520 Project: Beam Issue Type: Bu

[jira] [Commented] (BEAM-3645) Support multi-process execution on the FnApiRunner

[ https://issues.apache.org/jira/browse/BEAM-3645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924585#comment-16924585 ] Yifan Mai commented on BEAM-3645: - While testing this I noticed that the multi-process run

[jira] [Updated] (BEAM-8149) [FnApiRunner]multi-process runner does not terminate cleanly upon receiving SIGINT

[ https://issues.apache.org/jira/browse/BEAM-8149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifan Mai updated BEAM-8149: Description: The multi-process runner does not handle SIGINT gracefully. To reproduce, run wordcount.py usi

[jira] [Updated] (BEAM-8149) [FnApiRunner]multi-process runner does not terminate cleanly upon receiving SIGINT

[ https://issues.apache.org/jira/browse/BEAM-8149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifan Mai updated BEAM-8149: Description: The multi-process runner does not handle SIGINT gracefully. To reproduce, run wordcount.py usi

[jira] [Comment Edited] (BEAM-3645) Support multi-process execution on the FnApiRunner

[ https://issues.apache.org/jira/browse/BEAM-3645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924585#comment-16924585 ] Yifan Mai edited comment on BEAM-3645 at 9/6/19 8:54 PM: - While te

[jira] [Closed] (BEAM-7121) Provide deterministic version of Python's ProtoCoder

[ https://issues.apache.org/jira/browse/BEAM-7121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifan Mai closed BEAM-7121. --- Resolution: Done Fix Version/s: Not applicable > Provide deterministic version of Python's ProtoCoder >

[jira] [Commented] (BEAM-562) DoFn Reuse: Add new methods to DoFn

[ https://issues.apache.org/jira/browse/BEAM-562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16841552#comment-16841552 ] Yifan Mai commented on BEAM-562: This is implemented in #7994. > DoFn Reuse: Add new metho

[jira] [Resolved] (BEAM-562) DoFn Reuse: Add new methods to DoFn

[ https://issues.apache.org/jira/browse/BEAM-562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifan Mai resolved BEAM-562. Resolution: Done Fix Version/s: Not applicable > DoFn Reuse: Add new methods to DoFn > ---

[jira] [Closed] (BEAM-563) DoFn Reuse: Update DirectRunner

[ https://issues.apache.org/jira/browse/BEAM-563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifan Mai closed BEAM-563. -- Resolution: Done Fix Version/s: Not applicable This was also done in https://github.com/apache/beam/pull/7

[jira] [Created] (BEAM-7340) DoFn.teardown metrics are lost in Python SDK

Yifan Mai created BEAM-7340: --- Summary: DoFn.teardown metrics are lost in Python SDK Key: BEAM-7340 URL: https://issues.apache.org/jira/browse/BEAM-7340 Project: Beam Issue Type: Bug Compo

[jira] [Commented] (BEAM-529) Check immutability violations in DirectPipelineRunner

[ https://issues.apache.org/jira/browse/BEAM-529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16856227#comment-16856227 ] Yifan Mai commented on BEAM-529: No, not planning to work on this in the near future. As di

[jira] [Commented] (BEAM-529) Check immutability violations in DirectPipelineRunner

[ https://issues.apache.org/jira/browse/BEAM-529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16856777#comment-16856777 ] Yifan Mai commented on BEAM-529: Sorry, I haven't captured the proposal on JIRA yet. The g

[jira] [Comment Edited] (BEAM-529) Check immutability violations in DirectPipelineRunner

[ https://issues.apache.org/jira/browse/BEAM-529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16856777#comment-16856777 ] Yifan Mai edited comment on BEAM-529 at 6/5/19 3:06 PM: Sorry, I ha

[jira] [Created] (BEAM-6746) Support DoFn.setup and DoFn.teardown in Python

Yifan Mai created BEAM-6746: --- Summary: Support DoFn.setup and DoFn.teardown in Python Key: BEAM-6746 URL: https://issues.apache.org/jira/browse/BEAM-6746 Project: Beam Issue Type: Bug Com

[jira] [Updated] (BEAM-6746) Support DoFn.setup and DoFn.teardown in Python

[ https://issues.apache.org/jira/browse/BEAM-6746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifan Mai updated BEAM-6746: Issue Type: Improvement (was: Bug) > Support DoFn.setup and DoFn.teardown in Python > -

[jira] [Closed] (BEAM-6746) Support DoFn.setup and DoFn.teardown in Python

[ https://issues.apache.org/jira/browse/BEAM-6746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifan Mai closed BEAM-6746. --- Resolution: Duplicate Fix Version/s: Not applicable Looks like this was already filed as BEAM-562 > Su

[jira] [Created] (BEAM-7121) Provide deterministic version of Python's ProtoCoder

Yifan Mai created BEAM-7121: --- Summary: Provide deterministic version of Python's ProtoCoder Key: BEAM-7121 URL: https://issues.apache.org/jira/browse/BEAM-7121 Project: Beam Issue Type: Improvement

[jira] [Updated] (BEAM-7121) Provide deterministic version of Python's ProtoCoder

[ https://issues.apache.org/jira/browse/BEAM-7121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifan Mai updated BEAM-7121: Description: Passing deterministic=true to proto's [SerializeToString|[https://github.com/protocolbuffers/pr

[jira] [Updated] (BEAM-7121) Provide deterministic version of Python's ProtoCoder

[ https://issues.apache.org/jira/browse/BEAM-7121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifan Mai updated BEAM-7121: Description: Passing deterministic=true to proto's [SerializeToString|https://github.com/protocolbuffers/pro

[jira] [Updated] (BEAM-7121) Provide deterministic version of Python's ProtoCoder

[ https://issues.apache.org/jira/browse/BEAM-7121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifan Mai updated BEAM-7121: Description: Passing deterministic=true to proto's [SerializeToString|https://github.com/protocolbuffers/pr

[jira] [Created] (BEAM-10409) Combiner Packing Graph Optimization

Yifan Mai created BEAM-10409: Summary: Combiner Packing Graph Optimization Key: BEAM-10409 URL: https://issues.apache.org/jira/browse/BEAM-10409 Project: Beam Issue Type: Improvement Co

[jira] [Updated] (BEAM-10409) Add combiner packing to graph optimizer phases

[ https://issues.apache.org/jira/browse/BEAM-10409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifan Mai updated BEAM-10409: - Summary: Add combiner packing to graph optimizer phases (was: Add Combiner Packing to Graph Optimizer Ph

[jira] [Updated] (BEAM-10409) Add Combiner Packing to Graph Optimizer Phases

[ https://issues.apache.org/jira/browse/BEAM-10409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifan Mai updated BEAM-10409: - Summary: Add Combiner Packing to Graph Optimizer Phases (was: Combiner Packing Graph Optimization) > Ad

[jira] [Assigned] (BEAM-10409) Add combiner packing to graph optimizer phases

[ https://issues.apache.org/jira/browse/BEAM-10409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifan Mai reassigned BEAM-10409: Assignee: Yifan Mai > Add combiner packing to graph optimizer phases > ---

[jira] [Assigned] (BEAM-11196) Composite transform structure not preserved with fusion.

[ https://issues.apache.org/jira/browse/BEAM-11196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifan Mai reassigned BEAM-11196: Assignee: Yifan Mai > Composite transform structure not preserved with fusion. > -

[jira] [Commented] (BEAM-11196) Composite transform structure not preserved with fusion.

[ https://issues.apache.org/jira/browse/BEAM-11196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17226502#comment-17226502 ] Yifan Mai commented on BEAM-11196: -- Summarizing an an offline conversation with [~robert

[jira] [Commented] (BEAM-11196) Composite transform structure not preserved with fusion.

[ https://issues.apache.org/jira/browse/BEAM-11196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17230185#comment-17230185 ] Yifan Mai commented on BEAM-11196: -- [https://github.com/apache/beam/pull/13202] introduc

[jira] [Created] (BEAM-11024) Visit composite transform nodes in Pipeline._check_replacement()

Yifan Mai created BEAM-11024: Summary: Visit composite transform nodes in Pipeline._check_replacement() Key: BEAM-11024 URL: https://issues.apache.org/jira/browse/BEAM-11024 Project: Beam Issue

[jira] [Created] (BEAM-10641) Add Combiner Packing to Dataflow

Yifan Mai created BEAM-10641: Summary: Add Combiner Packing to Dataflow Key: BEAM-10641 URL: https://issues.apache.org/jira/browse/BEAM-10641 Project: Beam Issue Type: Improvement Compo

[jira] [Assigned] (BEAM-10641) Add Combiner Packing to Dataflow

[ https://issues.apache.org/jira/browse/BEAM-10641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifan Mai reassigned BEAM-10641: Assignee: Yifan Mai > Add Combiner Packing to Dataflow > > >

[jira] [Created] (BEAM-10779) Support multiple main inputs and side inputs for PTransformOverride

Yifan Mai created BEAM-10779: Summary: Support multiple main inputs and side inputs for PTransformOverride Key: BEAM-10779 URL: https://issues.apache.org/jira/browse/BEAM-10779 Project: Beam Iss

[jira] [Updated] (BEAM-10779) Support multiple main inputs and side inputs for PTransformOverride

[ https://issues.apache.org/jira/browse/BEAM-10779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifan Mai updated BEAM-10779: - Description: Currently, {{PTransformOverride}} only supports overriding {{PTransform}} s that have a [si

[jira] [Updated] (BEAM-10779) Support multiple main inputs and side inputs for PTransformOverride

[ https://issues.apache.org/jira/browse/BEAM-10779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifan Mai updated BEAM-10779: - Description: Currently, {{PTransformOverride}} only supports overriding {{PTransform}}s that have a [sin

[jira] [Created] (BEAM-10781) Allow PTransformOverride to use AppliedPTransform when constructing replacement transform

Yifan Mai created BEAM-10781: Summary: Allow PTransformOverride to use AppliedPTransform when constructing replacement transform Key: BEAM-10781 URL: https://issues.apache.org/jira/browse/BEAM-10781 Proje

[jira] [Assigned] (BEAM-10781) Allow PTransformOverride to use AppliedPTransform when constructing replacement transform

[ https://issues.apache.org/jira/browse/BEAM-10781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifan Mai reassigned BEAM-10781: Assignee: Yifan Mai > Allow PTransformOverride to use AppliedPTransform when constructing > repla

[jira] [Updated] (BEAM-12305) Add pack_combiners to FlinkRunner and SparkRunner

[ https://issues.apache.org/jira/browse/BEAM-12305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifan Mai updated BEAM-12305: - Summary: Add pack_combiners to FlinkRunner and SparkRunner (was: Add pack_combiners to FlinkRunner) > A

[jira] [Created] (BEAM-12798) Configurable combiner packing limit

Yifan Mai created BEAM-12798: Summary: Configurable combiner packing limit Key: BEAM-12798 URL: https://issues.apache.org/jira/browse/BEAM-12798 Project: Beam Issue Type: Bug Components

[jira] [Created] (BEAM-12199) Add pack_combiners to defaults

Yifan Mai created BEAM-12199: Summary: Add pack_combiners to defaults Key: BEAM-12199 URL: https://issues.apache.org/jira/browse/BEAM-12199 Project: Beam Issue Type: Bug Components: sdk

[jira] [Updated] (BEAM-12199) Add pack_combiners to defaults

[ https://issues.apache.org/jira/browse/BEAM-12199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifan Mai updated BEAM-12199: - Issue Type: New Feature (was: Bug) > Add pack_combiners to defaults > -- > >

[jira] [Updated] (BEAM-10781) Allow PTransformOverride to use AppliedPTransform when constructing replacement transform

[ https://issues.apache.org/jira/browse/BEAM-10781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifan Mai updated BEAM-10781: - Resolution: Fixed Status: Resolved (was: Open) > Allow PTransformOverride to use AppliedPTransfo

[jira] [Updated] (BEAM-11700) Optimizations skipped in PortableRunner when pre_optimize is 'default' or 'all'

[ https://issues.apache.org/jira/browse/BEAM-11700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifan Mai updated BEAM-11700: - Resolution: Fixed Status: Resolved (was: Open) > Optimizations skipped in PortableRunner when pr

[jira] [Updated] (BEAM-10779) Support multiple main inputs and side inputs for PTransformOverride

[ https://issues.apache.org/jira/browse/BEAM-10779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifan Mai updated BEAM-10779: - Resolution: Fixed Status: Resolved (was: Open) > Support multiple main inputs and side inputs fo

[jira] [Updated] (BEAM-11196) Composite transform structure not preserved with fusion.

[ https://issues.apache.org/jira/browse/BEAM-11196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifan Mai updated BEAM-11196: - Resolution: Fixed Status: Resolved (was: Open) > Composite transform structure not preserved wit

[jira] [Commented] (BEAM-11196) Composite transform structure not preserved with fusion.

[ https://issues.apache.org/jira/browse/BEAM-11196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17326116#comment-17326116 ] Yifan Mai commented on BEAM-11196: -- I think the current solution is reasonable: when we

[jira] [Updated] (BEAM-12199) Add pack_combiners to defaults

[ https://issues.apache.org/jira/browse/BEAM-12199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifan Mai updated BEAM-12199: - Resolution: Fixed Status: Resolved (was: Triage Needed) > Add pack_combiners to defaults > -

[jira] [Updated] (BEAM-12199) Add pack_combiners to Dataflow default phases

[ https://issues.apache.org/jira/browse/BEAM-12199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifan Mai updated BEAM-12199: - Description: Run this optimizer by default in DataflowRunner. pack_combiners only packs combiners with th

[jira] [Updated] (BEAM-11787) test_run_packable_combine_globally ValidatesRunner test fails for Dataflow

[ https://issues.apache.org/jira/browse/BEAM-11787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifan Mai updated BEAM-11787: - Resolution: Fixed Status: Resolved (was: Open) > test_run_packable_combine_globally ValidatesRun

[jira] [Updated] (BEAM-11694) Re-enable combiner packing for DataflowRunner, FnApiRunner and PortableRunner

[ https://issues.apache.org/jira/browse/BEAM-11694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifan Mai updated BEAM-11694: - Summary: Re-enable combiner packing for DataflowRunner, FnApiRunner and PortableRunner (was: Re-enable c

[jira] [Commented] (BEAM-11700) Optimizations skipped in PortableRunner when pre_optimize is 'default' or 'all'

[ https://issues.apache.org/jira/browse/BEAM-11700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17278232#comment-17278232 ] Yifan Mai commented on BEAM-11700: -- Cherry-pick is ready: #13882 > Optimizations skippe

[jira] [Commented] (BEAM-11715) Combiner packing creates an incorrect proto

[ https://issues.apache.org/jira/browse/BEAM-11715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17278236#comment-17278236 ] Yifan Mai commented on BEAM-11715: -- Fixing translations.pack_combiners turns out to be m

[jira] [Commented] (BEAM-11700) Optimizations skipped in PortableRunner when pre_optimize is 'default' or 'all'

[ https://issues.apache.org/jira/browse/BEAM-11700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17279263#comment-17279263 ] Yifan Mai commented on BEAM-11700: -- What are the symptoms? Does this refer to PortableRu

[jira] [Commented] (BEAM-11715) Combiner packing creates an incorrect proto

[ https://issues.apache.org/jira/browse/BEAM-11715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17279322#comment-17279322 ] Yifan Mai commented on BEAM-11715: -- PR #13884 is ready for review again. I have tested i

[jira] [Commented] (BEAM-11715) Combiner packing creates an incorrect proto

[ https://issues.apache.org/jira/browse/BEAM-11715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17279797#comment-17279797 ] Yifan Mai commented on BEAM-11715: -- Sent cherry-pick in [#13901|https://github.com/apach

[jira] [Commented] (BEAM-11700) Optimizations skipped in PortableRunner when pre_optimize is 'default' or 'all'

[ https://issues.apache.org/jira/browse/BEAM-11700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17279805#comment-17279805 ] Yifan Mai commented on BEAM-11700: -- I'm happy to send a new PR to do this, but do we hav

[jira] [Commented] (BEAM-11787) beam_PostCommit_Py_VR_Dataflow is perma red

[ https://issues.apache.org/jira/browse/BEAM-11787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17282669#comment-17282669 ] Yifan Mai commented on BEAM-11787: -- Looks like there was a behavior change for Dataflow

[jira] [Updated] (BEAM-10641) Add Combiner Packing to Dataflow

[ https://issues.apache.org/jira/browse/BEAM-10641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifan Mai updated BEAM-10641: - Fix Version/s: (was: 2.28.0) > Add Combiner Packing to Dataflow > >

[jira] [Commented] (BEAM-10641) Add Combiner Packing to Dataflow

[ https://issues.apache.org/jira/browse/BEAM-10641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17286756#comment-17286756 ] Yifan Mai commented on BEAM-10641: -- Unfortunately combiner packing for Dataflow was roll

[jira] [Updated] (BEAM-12199) Add pack_combiners to Dataflow default phases

[ https://issues.apache.org/jira/browse/BEAM-12199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifan Mai updated BEAM-12199: - Description: Run this optimizer by default in DataflowRunner, FnApiRunner and PortableRunner. pack_combin

[jira] [Updated] (BEAM-12199) Add pack_combiners to default phases

[ https://issues.apache.org/jira/browse/BEAM-12199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifan Mai updated BEAM-12199: - Summary: Add pack_combiners to default phases (was: Add pack_combiners to Dataflow default phases) > Ad

[jira] [Created] (BEAM-12305) Add pack_combiners to FlinkRunner

Yifan Mai created BEAM-12305: Summary: Add pack_combiners to FlinkRunner Key: BEAM-12305 URL: https://issues.apache.org/jira/browse/BEAM-12305 Project: Beam Issue Type: Bug Components:

[jira] [Commented] (BEAM-12324) TranslationsTest.test_run_packable_combine_* failing on PostCommit_Py_VR_Dataflow

[ https://issues.apache.org/jira/browse/BEAM-12324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17342726#comment-17342726 ] Yifan Mai commented on BEAM-12324: -- Taking a look. > TranslationsTest.test_run_packable

[jira] [Commented] (BEAM-12324) TranslationsTest.test_run_packable_combine_* failing on PostCommit_Py_VR_Dataflow

[ https://issues.apache.org/jira/browse/BEAM-12324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17342923#comment-17342923 ] Yifan Mai commented on BEAM-12324: -- Relevant logs: https://gist.github.com/yifanmai/f1c7

[jira] [Commented] (BEAM-12324) TranslationsTest.test_run_packable_combine_* failing on PostCommit_Py_VR_Dataflow

[ https://issues.apache.org/jira/browse/BEAM-12324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17342955#comment-17342955 ] Yifan Mai commented on BEAM-12324: -- Actually we already don't pack combiners when in str

[jira] [Commented] (BEAM-12324) TranslationsTest.test_run_packable_combine_* failing on PostCommit_Py_VR_Dataflow

[ https://issues.apache.org/jira/browse/BEAM-12324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17343623#comment-17343623 ] Yifan Mai commented on BEAM-12324: -- For the record, the resolution was that core.Combin

[jira] [Created] (BEAM-11644) translations.pack_combiners optimizer causes breaking change to metrics API

Yifan Mai created BEAM-11644: Summary: translations.pack_combiners optimizer causes breaking change to metrics API Key: BEAM-11644 URL: https://issues.apache.org/jira/browse/BEAM-11644 Project: Beam

[jira] [Commented] (BEAM-11644) translations.pack_combiners optimizer causes breaking change to metrics API

[ https://issues.apache.org/jira/browse/BEAM-11644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17271770#comment-17271770 ] Yifan Mai commented on BEAM-11644: -- The fix to make pack_combiners optional rather than

[jira] [Created] (BEAM-11694) Re-enable combiner packing for FnApiRunner and PortableRunner

Yifan Mai created BEAM-11694: Summary: Re-enable combiner packing for FnApiRunner and PortableRunner Key: BEAM-11694 URL: https://issues.apache.org/jira/browse/BEAM-11694 Project: Beam Issue Typ

[jira] [Assigned] (BEAM-11694) Re-enable combiner packing for FnApiRunner and PortableRunner

[ https://issues.apache.org/jira/browse/BEAM-11694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifan Mai reassigned BEAM-11694: Assignee: Yifan Mai > Re-enable combiner packing for FnApiRunner and PortableRunner >

[jira] [Created] (BEAM-11695) Remove translations.pack_combiners optimizer from defaults

Yifan Mai created BEAM-11695: Summary: Remove translations.pack_combiners optimizer from defaults Key: BEAM-11695 URL: https://issues.apache.org/jira/browse/BEAM-11695 Project: Beam Issue Type: B

[jira] [Updated] (BEAM-11644) translations.pack_combiners optimizer causes breaking change to metrics API

[ https://issues.apache.org/jira/browse/BEAM-11644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifan Mai updated BEAM-11644: - Fix Version/s: (was: 2.28.0) > translations.pack_combiners optimizer causes breaking change to metric

[jira] [Commented] (BEAM-11644) translations.pack_combiners optimizer causes breaking change to metrics API

[ https://issues.apache.org/jira/browse/BEAM-11644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17272502#comment-17272502 ] Yifan Mai commented on BEAM-11644: -- I replaced this bug with BEAM-11695 (short term remo

[jira] [Created] (BEAM-11700) Optimizations skipped in PortableRunner when pre_optimize is 'default' or 'all'

Yifan Mai created BEAM-11700: Summary: Optimizations skipped in PortableRunner when pre_optimize is 'default' or 'all' Key: BEAM-11700 URL: https://issues.apache.org/jira/browse/BEAM-11700 Project: Beam

[jira] [Assigned] (BEAM-11700) Optimizations skipped in PortableRunner when pre_optimize is 'default' or 'all'

[ https://issues.apache.org/jira/browse/BEAM-11700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifan Mai reassigned BEAM-11700: Assignee: Yifan Mai > Optimizations skipped in PortableRunner when pre_optimize is 'default' or >

[jira] [Updated] (BEAM-11700) Optimizations skipped in PortableRunner when pre_optimize is 'default' or 'all'

[ https://issues.apache.org/jira/browse/BEAM-11700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifan Mai updated BEAM-11700: - Description: An indentation error in portable_runner.py is causing optimizations to not run when pre_opti

[jira] [Updated] (BEAM-10641) Add Combiner Packing to Dataflow

[ https://issues.apache.org/jira/browse/BEAM-10641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifan Mai updated BEAM-10641: - Fix Version/s: 2.28.0 Resolution: Fixed Status: Resolved (was: Triage Needed) > Add Co

[jira] [Updated] (BEAM-11695) Remove translations.pack_combiners optimizer from defaults

[ https://issues.apache.org/jira/browse/BEAM-11695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifan Mai updated BEAM-11695: - Resolution: Fixed Status: Resolved (was: Triage Needed) > Remove translations.pack_combiners opt

[jira] [Commented] (BEAM-11715) Combiner packing creates an incorrect proto

[ https://issues.apache.org/jira/browse/BEAM-11715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17275442#comment-17275442 ] Yifan Mai commented on BEAM-11715: -- Work items: # Add a ValidatesRunner that triggers t

[jira] [Commented] (BEAM-11715) Combiner packing creates an incorrect proto

[ https://issues.apache.org/jira/browse/BEAM-11715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17275449#comment-17275449 ] Yifan Mai commented on BEAM-11715: -- Dataflow with portable job submission _should_ work

[jira] [Commented] (BEAM-11715) Combiner packing creates an incorrect proto

[ https://issues.apache.org/jira/browse/BEAM-11715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17275451#comment-17275451 ] Yifan Mai commented on BEAM-11715: -- I'm not too familiar with how to test x-lang stuff,

[jira] [Commented] (BEAM-11715) Combiner packing creates an incorrect proto

[ https://issues.apache.org/jira/browse/BEAM-11715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17275460#comment-17275460 ] Yifan Mai commented on BEAM-11715: -- >From offline sync: * I will implement "rewrite tra

[jira] [Commented] (BEAM-11700) Optimizations skipped in PortableRunner when pre_optimize is 'default' or 'all'

[ https://issues.apache.org/jira/browse/BEAM-11700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17276791#comment-17276791 ] Yifan Mai commented on BEAM-11700: -- No progress today. The PR seems to generate actual t

[jira] [Commented] (BEAM-11715) Combiner packing creates an incorrect proto

[ https://issues.apache.org/jira/browse/BEAM-11715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17276795#comment-17276795 ] Yifan Mai commented on BEAM-11715: -- The changes from my previous comment are in draft #1

[jira] [Created] (BEAM-11355) pipeline_from_stages after sort_stages can return non-topologically ordered pipelines

Yifan Mai created BEAM-11355: Summary: pipeline_from_stages after sort_stages can return non-topologically ordered pipelines Key: BEAM-11355 URL: https://issues.apache.org/jira/browse/BEAM-11355 Project:

[jira] [Commented] (BEAM-11196) Composite transform structure not preserved with fusion.

[ https://issues.apache.org/jira/browse/BEAM-11196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17245635#comment-17245635 ] Yifan Mai commented on BEAM-11196: -- FYI we ended up doing a cherry pick to 2.26.0 [http

[jira] [Updated] (BEAM-10409) Add combiner packing to graph optimizer phases

[ https://issues.apache.org/jira/browse/BEAM-10409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifan Mai updated BEAM-10409: - Resolution: Fixed Status: Resolved (was: Open) > Add combiner packing to graph optimizer phases

[jira] [Created] (BEAM-11429) User-friendly stage name for packed combiners

Yifan Mai created BEAM-11429: Summary: User-friendly stage name for packed combiners Key: BEAM-11429 URL: https://issues.apache.org/jira/browse/BEAM-11429 Project: Beam Issue Type: Bug