[jira] [Commented] (FLINK-19671) Update EditorConfig file to be useful

2020-10-19 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-19671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17216516#comment-17216516 ] Zhu Zhu commented on FLINK-19671: - Hi [~aljoscha], I found the continuation indentation

[jira] [Commented] (FLINK-19671) Update EditorConfig file to be useful

2020-10-19 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-19671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17216583#comment-17216583 ] Zhu Zhu commented on FLINK-19671: - I agree that we should have a rule which is easy to a

[jira] [Created] (FLINK-19712) Pipelined region failover should skip restarting tasks in CREATED

2020-10-19 Thread Zhu Zhu (Jira)
Zhu Zhu created FLINK-19712: --- Summary: Pipelined region failover should skip restarting tasks in CREATED Key: FLINK-19712 URL: https://issues.apache.org/jira/browse/FLINK-19712 Project: Flink Issu

[jira] [Created] (FLINK-19714) Assign different logical pipelined regions of a DataSet job to different slot sharing group

2020-10-19 Thread Zhu Zhu (Jira)
Zhu Zhu created FLINK-19714: --- Summary: Assign different logical pipelined regions of a DataSet job to different slot sharing group Key: FLINK-19714 URL: https://issues.apache.org/jira/browse/FLINK-19714 Pro

[jira] [Updated] (FLINK-19714) Assign different logical pipelined regions of a DataSet job to different slot sharing group

2020-10-19 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-19714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu updated FLINK-19714: Description: Currently, job vertices of a DataSet job will be put into the same SlotSharingGroup. It does

[jira] [Updated] (FLINK-19714) Assign different logical pipelined regions of a DataSet job to different slot sharing group

2020-10-19 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-19714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu updated FLINK-19714: Description: Currently, job vertices of a DataSet job will be put into the same SlotSharingGroup. It does

[jira] [Commented] (FLINK-19712) Pipelined region failover should skip restarting tasks in CREATED

2020-10-19 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-19712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17216683#comment-17216683 ] Zhu Zhu commented on FLINK-19712: - cc [~trohrmann] [~azagrebin] > Pipelined region fail

[jira] [Commented] (FLINK-19714) Assign different logical pipelined regions of a DataSet job to different slot sharing group

2020-10-19 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-19714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17216682#comment-17216682 ] Zhu Zhu commented on FLINK-19714: - cc [~trohrmann] [~azagrebin] > Assign different logi

[jira] [Closed] (FLINK-20285) LazyFromSourcesSchedulingStrategy is possible to schedule non-CREATED vertices

2020-11-23 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-20285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu closed FLINK-20285. --- Resolution: Fixed fixed via: 1.12&master: d4a54673fe73431bbb009185335d6f393ed8f69d 1.11: 3854c303e7910c639c8

[jira] [Commented] (FLINK-20284) Error happens in TaskExecutor when closing JobMaster connection if there was a python UDF

2020-11-25 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-20284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17239030#comment-17239030 ] Zhu Zhu commented on FLINK-20284: - Thanks for the investigation and fixing this problem!

[jira] [Created] (FLINK-20439) Consider simplifying or removing mechanism to scheduleOrUpdateConsumers

2020-12-01 Thread Zhu Zhu (Jira)
Zhu Zhu created FLINK-20439: --- Summary: Consider simplifying or removing mechanism to scheduleOrUpdateConsumers Key: FLINK-20439 URL: https://issues.apache.org/jira/browse/FLINK-20439 Project: Flink

[jira] [Commented] (FLINK-20439) Consider simplifying or removing mechanism to scheduleOrUpdateConsumers

2020-12-01 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-20439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17241388#comment-17241388 ] Zhu Zhu commented on FLINK-20439: - `JobMasterGateway#scheduleOrUpdateConsumers()` makes

[jira] [Updated] (FLINK-20439) Consider simplifying or removing mechanism to scheduleOrUpdateConsumers

2020-12-14 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-20439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu updated FLINK-20439: Parent: (was: FLINK-15626) Issue Type: Task (was: Sub-task) > Consider simplifying or removin

[jira] [Updated] (FLINK-20439) Consider simplifying or removing mechanism to scheduleOrUpdateConsumers

2020-12-14 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-20439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu updated FLINK-20439: Parent: FLINK-20589 Issue Type: Sub-task (was: Task) > Consider simplifying or removing mechanism

[jira] [Created] (FLINK-20619) Remove InputDependencyConstraint and InputDependencyConstraintChecker

2020-12-15 Thread Zhu Zhu (Jira)
Zhu Zhu created FLINK-20619: --- Summary: Remove InputDependencyConstraint and InputDependencyConstraintChecker Key: FLINK-20619 URL: https://issues.apache.org/jira/browse/FLINK-20619 Project: Flink

[jira] [Commented] (FLINK-21030) Broken job restart for job with disjoint graph

2021-01-19 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-21030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17268409#comment-17268409 ] Zhu Zhu commented on FLINK-21030: - Agreed to trigger a global failover to bring FINISHED

[jira] [Comment Edited] (FLINK-21030) Broken job restart for job with disjoint graph

2021-01-19 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-21030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17268409#comment-17268409 ] Zhu Zhu edited comment on FLINK-21030 at 1/20/21, 7:21 AM: --- Ag

[jira] [Closed] (FLINK-20931) Remove globalModVersion from ExecutionGraph

2021-01-20 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-20931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu closed FLINK-20931. --- Resolution: Done Done via 759d7347ddccc576b4f039e99ac27af1df9ab260 > Remove globalModVersion from Execution

[jira] [Commented] (FLINK-15626) Remove legacy scheduler

2021-01-24 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-15626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17271074#comment-17271074 ] Zhu Zhu commented on FLINK-15626: - [~trohrmann] in the future we may want to factor all

[jira] [Closed] (FLINK-15626) Remove legacy scheduler

2021-01-24 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-15626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu closed FLINK-15626. --- Resolution: Done > Remove legacy scheduler > --- > > Key: FLINK-15626 >

[jira] [Closed] (FLINK-10429) Redesign Flink Scheduling, introducing dedicated Scheduler component

2021-01-24 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-10429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu closed FLINK-10429. --- Resolution: Done > Redesign Flink Scheduling, introducing dedicated Scheduler component > --

[jira] [Updated] (FLINK-21110) Optimize Scheduler Performance for Large-Scale Jobs

2021-01-24 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-21110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu updated FLINK-21110: Description: According to the result of scheduler benchmarks we implemented in FLINK-20612, the bottlenec

[jira] [Commented] (FLINK-16017) Improve attachJobGraph Performance

2021-01-30 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17275554#comment-17275554 ] Zhu Zhu commented on FLINK-16017: - [~rmetzger] yes it is already fixed via FLINK-17180.

[jira] [Closed] (FLINK-16017) Improve attachJobGraph Performance

2021-01-30 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu closed FLINK-16017. --- Resolution: Duplicate > Improve attachJobGraph Performance > -- > >

[jira] [Commented] (FLINK-21030) Broken job restart for job with disjoint graph

2021-02-01 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-21030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17276846#comment-17276846 ] Zhu Zhu commented on FLINK-21030: - Thanks for the investigation! [~mapohl] Does this mea

[jira] [Commented] (FLINK-21030) Broken job restart for job with disjoint graph

2021-02-01 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-21030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17276920#comment-17276920 ] Zhu Zhu commented on FLINK-21030: - Sounds good to me. [~yunta] would you also take a loo

[jira] [Commented] (FLINK-21030) Broken job restart for job with disjoint graph

2021-02-02 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-21030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17277769#comment-17277769 ] Zhu Zhu commented on FLINK-21030: - I guess I had misunderstood the protocol of {{Checkp

[jira] [Assigned] (FLINK-21273) Remove unused ExecutionVertexSchedulingRequirements

2021-02-03 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-21273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu reassigned FLINK-21273: --- Assignee: Yangze Guo > Remove unused ExecutionVertexSchedulingRequirements > --

[jira] [Commented] (FLINK-21273) Remove unused ExecutionVertexSchedulingRequirements

2021-02-03 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-21273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17278592#comment-17278592 ] Zhu Zhu commented on FLINK-21273: - Thanks for firing this ticket! [~karmagyz] I have ass

[jira] [Commented] (FLINK-21110) Optimize Scheduler Performance for Large-Scale Jobs

2021-02-04 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-21110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17278666#comment-17278666 ] Zhu Zhu commented on FLINK-21110: - [~Thesharing] It's better to open different JIRA tick

[jira] [Updated] (FLINK-21273) Remove unused ExecutionVertexSchedulingRequirements

2021-02-06 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-21273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu updated FLINK-21273: Issue Type: Improvement (was: Bug) > Remove unused ExecutionVertexSchedulingRequirements > --

[jira] [Closed] (FLINK-21273) Remove unused ExecutionVertexSchedulingRequirements

2021-02-06 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-21273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu closed FLINK-21273. --- Fix Version/s: 1.13.0 Resolution: Done Done via 04a45ab3c24fa80a987f1409c521c8857cc35155 > Remove un

[jira] [Updated] (FLINK-21273) Remove unused ExecutionVertexSchedulingRequirements

2021-02-06 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-21273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu updated FLINK-21273: Affects Version/s: 1.12.0 > Remove unused ExecutionVertexSchedulingRequirements >

[jira] [Commented] (FLINK-17726) Scheduler should take care of tasks directly canceled by TaskManager

2021-02-07 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-17726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17280409#comment-17280409 ] Zhu Zhu commented on FLINK-17726: - Thanks for relaunching this discussion and proposing

[jira] [Comment Edited] (FLINK-17726) Scheduler should take care of tasks directly canceled by TaskManager

2021-02-07 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-17726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17280409#comment-17280409 ] Zhu Zhu edited comment on FLINK-17726 at 2/7/21, 8:23 AM: -- Than

[jira] [Comment Edited] (FLINK-17726) Scheduler should take care of tasks directly canceled by TaskManager

2021-02-07 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-17726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17280409#comment-17280409 ] Zhu Zhu edited comment on FLINK-17726 at 2/7/21, 8:23 AM: -- Than

[jira] [Commented] (FLINK-20612) Add benchmarks for scheduler

2021-02-08 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-20612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17281166#comment-17281166 ] Zhu Zhu commented on FLINK-20612: - The PR is merged: fa61585879aacc3b5f7650a229613b3d944

[jira] [Commented] (FLINK-22677) Scheduler should invoke ShuffleMaster#registerPartitionWithProducer by a real asynchronous fashion

2021-07-05 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-22677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17375217#comment-17375217 ] Zhu Zhu commented on FLINK-22677: - Problems below could happen if enabling partition reg

[jira] [Commented] (FLINK-22677) Scheduler should invoke ShuffleMaster#registerPartitionWithProducer by a real asynchronous fashion

2021-07-05 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-22677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17375218#comment-17375218 ] Zhu Zhu commented on FLINK-22677: - One thing need to mention is that I did not change {{

[jira] [Commented] (FLINK-23262) FileReadingWatermarkITCase.testWatermarkEmissionWithChaining fails on azure

2021-07-06 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-23262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17375376#comment-17375376 ] Zhu Zhu commented on FLINK-23262: - another instance: https://dev.azure.com/apache-flink/

[jira] [Closed] (FLINK-15031) Automatically calculate required network memory for fine-grained jobs

2021-07-06 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-15031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu closed FLINK-15031. --- Fix Version/s: (was: 1.12.0) 1.14.0 Resolution: Fixed Done via 2c52816540e0d5

[jira] [Commented] (FLINK-23218) Distribute the ShuffleDescriptors via blob server

2021-07-08 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-23218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17377811#comment-17377811 ] Zhu Zhu commented on FLINK-23218: - 1. To not affect existing users, I prefer limit to no

[jira] [Commented] (FLINK-23218) Distribute the ShuffleDescriptors via blob server

2021-07-09 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-23218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17377961#comment-17377961 ] Zhu Zhu commented on FLINK-23218: - 10GB looks a bit too large to limit the blob size by

[jira] [Commented] (FLINK-23218) Distribute the ShuffleDescriptors via blob server

2021-07-09 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-23218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17377963#comment-17377963 ] Zhu Zhu commented on FLINK-23218: - I took another think and 10GB sounds good to me now.

[jira] [Closed] (FLINK-22017) Regions may never be scheduled when there are cross-region blocking edges

2021-07-15 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-22017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu closed FLINK-22017. --- Assignee: Zhilong Hong Resolution: Fixed Fixed via d2005268b1eeb0fe928b69c5e56ca54862fbf508 eb8100f7a

[jira] [Issue Comment Deleted] (FLINK-22017) Regions may never be scheduled when there are cross-region blocking edges

2021-07-15 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-22017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu updated FLINK-22017: Comment: was deleted (was: This issue was labeled "stale-critical" 7 ago and has not received any updates

[jira] [Issue Comment Deleted] (FLINK-22017) Regions may never be scheduled when there are cross-region blocking edges

2021-07-15 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-22017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu updated FLINK-22017: Comment: was deleted (was: I am the [Flink Jira Bot|https://github.com/apache/flink-jira-bot/] and I help

[jira] [Issue Comment Deleted] (FLINK-22017) Regions may never be scheduled when there are cross-region blocking edges

2021-07-15 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-22017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu updated FLINK-22017: Comment: was deleted (was: This critical issue is unassigned and itself and all of its Sub-Tasks have not

[jira] [Issue Comment Deleted] (FLINK-22017) Regions may never be scheduled when there are cross-region blocking edges

2021-07-15 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-22017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu updated FLINK-22017: Comment: was deleted (was: I am the [Flink Jira Bot|https://github.com/apache/flink-jira-bot/] and I help

[jira] [Updated] (FLINK-22676) The partition tracker should support remote shuffle properly

2021-07-18 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-22676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu updated FLINK-22676: Affects Version/s: 1.4 > The partition tracker should support remote shuffle properly > --

[jira] [Assigned] (FLINK-22676) The partition tracker should support remote shuffle properly

2021-07-18 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-22676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu reassigned FLINK-22676: --- Assignee: Jin Xing > The partition tracker should support remote shuffle properly > ---

[jira] [Updated] (FLINK-22676) The partition tracker should support remote shuffle properly

2021-07-18 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-22676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu updated FLINK-22676: Fix Version/s: 1.14.0 > The partition tracker should support remote shuffle properly > ---

[jira] [Updated] (FLINK-22676) The partition tracker should support remote shuffle properly

2021-07-18 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-22676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu updated FLINK-22676: Affects Version/s: (was: 1.4) 1.14.0 > The partition tracker should support rem

[jira] [Closed] (FLINK-22676) The partition tracker should support remote shuffle properly

2021-07-18 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-22676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu closed FLINK-22676. --- Resolution: Done Done via 62a342b647fc1eac7f87769be92fda798649d6d4 > The partition tracker should support r

[jira] [Assigned] (FLINK-22672) Some enhancements for pluggable shuffle service framework

2021-07-18 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-22672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu reassigned FLINK-22672: --- Assignee: Jin Xing > Some enhancements for pluggable shuffle service framework > --

[jira] [Closed] (FLINK-22677) Scheduler should invoke ShuffleMaster#registerPartitionWithProducer by a real asynchronous fashion

2021-07-19 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-22677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu closed FLINK-22677. --- Resolution: Done Done via 0d099b79fddc5e254884e44f2167c625744079a4 0b28fadccfb6b0d2a85592ced9e98b03a0c2d3bf

[jira] [Commented] (FLINK-23218) Distribute the ShuffleDescriptors via blob server

2021-07-21 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-23218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17384868#comment-17384868 ] Zhu Zhu commented on FLINK-23218: - I think the PR mentioned above should be https://git

[jira] [Commented] (FLINK-23218) Distribute the ShuffleDescriptors via blob server

2021-07-21 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-23218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17384876#comment-17384876 ] Zhu Zhu commented on FLINK-23218: - I took a look at the code and looks to me that transi

[jira] [Comment Edited] (FLINK-23218) Distribute the ShuffleDescriptors via blob server

2021-07-21 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-23218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17384876#comment-17384876 ] Zhu Zhu edited comment on FLINK-23218 at 7/21/21, 1:02 PM: --- I

[jira] [Commented] (FLINK-23218) Distribute the ShuffleDescriptors via blob server

2021-07-21 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-23218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17385234#comment-17385234 ] Zhu Zhu commented on FLINK-23218: - Just summarize the investigation and discussion, a co

[jira] [Commented] (FLINK-23218) Distribute the ShuffleDescriptors via blob server

2021-07-22 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-23218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17385644#comment-17385644 ] Zhu Zhu commented on FLINK-23218: - Thanks for confirming! [~trohrmann] And thanks for th

[jira] [Commented] (FLINK-23470) Use blocking shuffles but pipeline within a slot for batch mode

2021-07-23 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-23470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17386107#comment-17386107 ] Zhu Zhu commented on FLINK-23470: - Sorry I did not see the discussion in FLINK-23402 unt

[jira] [Commented] (FLINK-23479) IncrementalAggregateJsonPlanTest.testIncrementalAggregateWithSumCountDistinctAndRetraction fail

2021-07-26 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-23479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17387460#comment-17387460 ] Zhu Zhu commented on FLINK-23479: - another instance: https://dev.azure.com/apache-flink

[jira] [Closed] (FLINK-23218) Distribute the ShuffleDescriptors via blob server

2021-07-26 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-23218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu closed FLINK-23218. --- Resolution: Done Done via ee7e9c3b87f6533d6f54361fddc71585d6b8ad61 > Distribute the ShuffleDescriptors via

[jira] [Assigned] (FLINK-23218) Distribute the ShuffleDescriptors via blob server

2021-07-26 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-23218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu reassigned FLINK-23218: --- Assignee: Zhilong Hong > Distribute the ShuffleDescriptors via blob server > --

[jira] [Closed] (FLINK-23005) Cache the compressed serialized value of ShuffleDescriptors

2021-07-26 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-23005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu closed FLINK-23005. --- Resolution: Done Done via a3f72f20acd4df1dbdc61e145d1d932f61ca63f8 6812d18c358ce007a5cbcd685f32f59c70b03a49

[jira] [Commented] (FLINK-23402) Expose a consistent GlobalDataExchangeMode

2021-07-28 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-23402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1730#comment-1730 ] Zhu Zhu commented on FLINK-23402: - +1 for option #2 to rename {{ShuffleMode}} as well as

[jira] [Assigned] (FLINK-23354) Limit the size of ShuffleDescriptors in PermanentBlobCache on TaskExecutor

2021-07-28 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-23354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu reassigned FLINK-23354: --- Assignee: Zhilong Hong > Limit the size of ShuffleDescriptors in PermanentBlobCache on TaskExecutor

[jira] [Updated] (FLINK-22773) Optimize the construction of pipelined regions

2021-07-28 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-22773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu updated FLINK-22773: Fix Version/s: 1.14.0 > Optimize the construction of pipelined regions > -

[jira] [Assigned] (FLINK-22773) Optimize the construction of pipelined regions

2021-07-28 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-22773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu reassigned FLINK-22773: --- Assignee: Zhilong Hong > Optimize the construction of pipelined regions > -

[jira] [Assigned] (FLINK-23172) Links of restart strategy in configuration page is broken

2021-07-28 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-23172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu reassigned FLINK-23172: --- Assignee: Zhilong Hong > Links of restart strategy in configuration page is broken > --

[jira] [Commented] (FLINK-23172) Links of restart strategy in configuration page is broken

2021-07-28 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-23172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17389206#comment-17389206 ] Zhu Zhu commented on FLINK-23172: - Thanks for reporting this problem! [~Thesharing] I ha

[jira] [Closed] (FLINK-23354) Limit the size of ShuffleDescriptors in PermanentBlobCache on TaskExecutor

2021-07-29 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-23354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu closed FLINK-23354. --- Resolution: Done Done via 5c475d41fea3c81557e0d463bed1c94024dd0da5 > Limit the size of ShuffleDescriptors i

[jira] [Commented] (FLINK-23593) Performance regression on 15.07.2021

2021-08-03 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-23593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17392091#comment-17392091 ] Zhu Zhu commented on FLINK-23593: - I'd like to understand why the regression happens due

[jira] [Assigned] (FLINK-23599) Remove JobVertex#connectIdInput

2021-08-03 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-23599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu reassigned FLINK-23599: --- Assignee: Zhilong Hong > Remove JobVertex#connectIdInput > --- > >

[jira] [Closed] (FLINK-23599) Remove JobVertex#connectIdInput

2021-08-03 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-23599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu closed FLINK-23599. --- Resolution: Done Done via ec9ff1ee5e33529260d6a3adfad4b0b34efde55e > Remove JobVertex#connectIdInput >

[jira] [Assigned] (FLINK-22767) Optimize the initialization of LocalInputPreferredSlotSharingStrategy

2021-08-03 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-22767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu reassigned FLINK-22767: --- Assignee: Zhilong Hong > Optimize the initialization of LocalInputPreferredSlotSharingStrategy > --

[jira] [Closed] (FLINK-23172) Links to Task Failure Recovery page on Configuration page are broken

2021-08-03 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-23172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu closed FLINK-23172. --- Resolution: Fixed Fixed via 5183b2af9d467708725bd1454a671bc7689159a5 46bf6d68ee97684949ba3ad38dc18ff7c80009

[jira] [Commented] (FLINK-23590) StreamTaskTest#testProcessWithUnAvailableInput is flaky

2021-08-03 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-23590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17392737#comment-17392737 ] Zhu Zhu commented on FLINK-23590: - https://dev.azure.com/apache-flink/apache-flink/_buil

[jira] [Assigned] (FLINK-22910) Refine ShuffleMaster lifecycle management for pluggable shuffle service framework

2021-08-04 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-22910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu reassigned FLINK-22910: --- Assignee: Yingjie Cao > Refine ShuffleMaster lifecycle management for pluggable shuffle service >

[jira] [Assigned] (FLINK-23249) Introduce ShuffleMasterContext to ShuffleMaster

2021-08-04 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-23249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu reassigned FLINK-23249: --- Assignee: Yingjie Cao > Introduce ShuffleMasterContext to ShuffleMaster > -

[jira] [Assigned] (FLINK-23214) Make ShuffleMaster a cluster level shared service

2021-08-04 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-23214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu reassigned FLINK-23214: --- Assignee: Yingjie Cao > Make ShuffleMaster a cluster level shared service > ---

[jira] [Assigned] (FLINK-22675) Add lifecycle methods to ShuffleMaster

2021-08-04 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-22675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu reassigned FLINK-22675: --- Assignee: Zhu Zhu > Add lifecycle methods to ShuffleMaster > --

[jira] [Assigned] (FLINK-22674) Provide JobID when apply shuffle resource by ShuffleMaster

2021-08-04 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-22674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu reassigned FLINK-22674: --- Assignee: Yingjie Cao > Provide JobID when apply shuffle resource by ShuffleMaster > --

[jira] [Closed] (FLINK-22675) Add lifecycle methods to ShuffleMaster

2021-08-04 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-22675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu closed FLINK-22675. --- Resolution: Done Done via 80df36b51af791f67126e07e182015ea6ea73fd2 > Add lifecycle methods to ShuffleMaster

[jira] [Assigned] (FLINK-22675) Add lifecycle methods to ShuffleMaster

2021-08-04 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-22675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu reassigned FLINK-22675: --- Assignee: Yingjie Cao (was: Zhu Zhu) > Add lifecycle methods to ShuffleMaster > --

[jira] [Closed] (FLINK-22674) Provide JobID when apply shuffle resource by ShuffleMaster

2021-08-04 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-22674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu closed FLINK-22674. --- Resolution: Done Done via 6bc8399e7f1738ec22cb1082c096269b5106cee5 > Provide JobID when apply shuffle resou

[jira] [Closed] (FLINK-23214) Make ShuffleMaster a cluster level shared service

2021-08-04 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-23214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu closed FLINK-23214. --- Resolution: Done Done via 81e1db3c439c1758dccd1a20f2f6b70120f48ef7 > Make ShuffleMaster a cluster level sha

[jira] [Updated] (FLINK-22675) Add lifecycle methods to ShuffleMaster

2021-08-04 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-22675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu updated FLINK-22675: Fix Version/s: 1.14.0 > Add lifecycle methods to ShuffleMaster > -- >

[jira] [Updated] (FLINK-23214) Make ShuffleMaster a cluster level shared service

2021-08-04 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-23214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu updated FLINK-23214: Fix Version/s: 1.14.0 > Make ShuffleMaster a cluster level shared service > --

[jira] [Closed] (FLINK-23249) Introduce ShuffleMasterContext to ShuffleMaster

2021-08-04 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-23249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu closed FLINK-23249. --- Resolution: Done Done via 0ee4038ef596b22630bc814f677fa489d3796241 > Introduce ShuffleMasterContext to Shuf

[jira] [Updated] (FLINK-22674) Provide JobID when apply shuffle resource by ShuffleMaster

2021-08-04 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-22674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu updated FLINK-22674: Fix Version/s: 1.14.0 > Provide JobID when apply shuffle resource by ShuffleMaster > -

[jira] [Commented] (FLINK-23593) Performance regression on 15.07.2021

2021-08-05 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-23593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17393836#comment-17393836 ] Zhu Zhu commented on FLINK-23593: - Thanks for the updates! [~sewen] I think your guess

[jira] [Updated] (FLINK-23172) Links to Task Failure Recovery page on Configuration page are broken

2021-08-05 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-23172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu updated FLINK-23172: Affects Version/s: 1.13.2 > Links to Task Failure Recovery page on Configuration page are broken > ---

[jira] [Commented] (FLINK-23593) Performance regression on 15.07.2021

2021-08-05 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-23593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17393950#comment-17393950 ] Zhu Zhu commented on FLINK-23593: - >> Could the larger difference between local benchmar

[jira] [Comment Edited] (FLINK-23593) Performance regression on 15.07.2021

2021-08-05 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-23593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17393950#comment-17393950 ] Zhu Zhu edited comment on FLINK-23593 at 8/5/21, 12:19 PM: --- >>

[jira] [Commented] (FLINK-16069) Creation of TaskDeploymentDescriptor can block main thread for long time

2021-08-08 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17395728#comment-17395728 ] Zhu Zhu commented on FLINK-16069: - Thanks for making the improvements and sharing the re

[jira] [Closed] (FLINK-16069) Creation of TaskDeploymentDescriptor can block main thread for long time

2021-08-08 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-16069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu Zhu closed FLINK-16069. --- Resolution: Duplicate > Creation of TaskDeploymentDescriptor can block main thread for long time > -

[jira] [Commented] (FLINK-23593) Performance regression on 15.07.2021

2021-08-09 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-23593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17395983#comment-17395983 ] Zhu Zhu commented on FLINK-23593: - I tried the benchmarks locally before/after applying

[jira] [Comment Edited] (FLINK-23593) Performance regression on 15.07.2021

2021-08-09 Thread Zhu Zhu (Jira)
[ https://issues.apache.org/jira/browse/FLINK-23593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17395983#comment-17395983 ] Zhu Zhu edited comment on FLINK-23593 at 8/9/21, 11:20 AM: --- I

<    10   11   12   13   14   15   16   17   18   19   >