[jira] [Commented] (BEAM-8944) Python SDK harness performance degradation with UnboundedThreadPoolExecutor

2020-05-01 Thread Luke Cwik (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17097665#comment-17097665 ] Luke Cwik commented on BEAM-8944: - I was able to come up with a compromise that I believe should work well

[jira] [Commented] (BEAM-8944) Python SDK harness performance degradation with UnboundedThreadPoolExecutor

2020-05-01 Thread Maximilian Michels (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17097302#comment-17097302 ] Maximilian Michels commented on BEAM-8944: -- [~lcwik] For the time being, do you think we could

[jira] [Commented] (BEAM-8944) Python SDK harness performance degradation with UnboundedThreadPoolExecutor

2020-04-03 Thread Maximilian Michels (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17074641#comment-17074641 ] Maximilian Michels commented on BEAM-8944: -- Thank you for your thoughts Luke. I agree that the

[jira] [Commented] (BEAM-8944) Python SDK harness performance degradation with UnboundedThreadPoolExecutor

2020-04-02 Thread Luke Cwik (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17073871#comment-17073871 ] Luke Cwik commented on BEAM-8944: - I was having difficulty in producing a solution that both created

[jira] [Commented] (BEAM-8944) Python SDK harness performance degradation with UnboundedThreadPoolExecutor

2020-04-02 Thread Maximilian Michels (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17073657#comment-17073657 ] Maximilian Michels commented on BEAM-8944: -- My first though was to change the implementation to

[jira] [Commented] (BEAM-8944) Python SDK harness performance degradation with UnboundedThreadPoolExecutor

2020-04-01 Thread Luke Cwik (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17073142#comment-17073142 ] Luke Cwik commented on BEAM-8944: - I took a look at the implementation of the UnboundedThreadPoolExecutor

[jira] [Commented] (BEAM-8944) Python SDK harness performance degradation with UnboundedThreadPoolExecutor

2020-04-01 Thread Maximilian Michels (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17072860#comment-17072860 ] Maximilian Michels commented on BEAM-8944: -- I see that the usage pattern in the description is a

[jira] [Commented] (BEAM-8944) Python SDK harness performance degradation with UnboundedThreadPoolExecutor

2020-04-01 Thread Luke Cwik (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17072842#comment-17072842 ] Luke Cwik commented on BEAM-8944: - The Python 3.5 implementation of the ThreadPoolExecutor:

[jira] [Commented] (BEAM-8944) Python SDK harness performance degradation with UnboundedThreadPoolExecutor

2020-04-01 Thread Luke Cwik (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17072840#comment-17072840 ] Luke Cwik commented on BEAM-8944: - So the original analysis done in the description of the bug isn't

[jira] [Commented] (BEAM-8944) Python SDK harness performance degradation with UnboundedThreadPoolExecutor

2020-04-01 Thread Maximilian Michels (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17072693#comment-17072693 ] Maximilian Michels commented on BEAM-8944: -- !checkpoint-duration.png! yellow: 2.18.0 as

[jira] [Commented] (BEAM-8944) Python SDK harness performance degradation with UnboundedThreadPoolExecutor

2020-03-31 Thread Maximilian Michels (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17072103#comment-17072103 ] Maximilian Michels commented on BEAM-8944: -- Essentially, yes. Especially, this is a concern for

[jira] [Commented] (BEAM-8944) Python SDK harness performance degradation with UnboundedThreadPoolExecutor

2020-03-31 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17071998#comment-17071998 ] Yichi Zhang commented on BEAM-8944: --- [~mxm] the checkpoint duration is lower means that without

[jira] [Commented] (BEAM-8944) Python SDK harness performance degradation with UnboundedThreadPoolExecutor

2019-12-21 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17001802#comment-17001802 ] Yichi Zhang commented on BEAM-8944: --- I think so > Python SDK harness performance degradation with

[jira] [Commented] (BEAM-8944) Python SDK harness performance degradation with UnboundedThreadPoolExecutor

2019-12-20 Thread Ahmet Altay (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17001589#comment-17001589 ] Ahmet Altay commented on BEAM-8944: --- Could this be closed after the cherry pick PR

[jira] [Commented] (BEAM-8944) Python SDK harness performance degradation with UnboundedThreadPoolExecutor

2019-12-19 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17000448#comment-17000448 ] Yichi Zhang commented on BEAM-8944: --- then yeah, it'll affect python streaming jobs (which is only on

[jira] [Commented] (BEAM-8944) Python SDK harness performance degradation with UnboundedThreadPoolExecutor

2019-12-19 Thread Udi Meiri (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17000445#comment-17000445 ] Udi Meiri commented on BEAM-8944: - The point of my question is to figure whether this issue is a 2.18

[jira] [Commented] (BEAM-8944) Python SDK harness performance degradation with UnboundedThreadPoolExecutor

2019-12-19 Thread Udi Meiri (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17000443#comment-17000443 ] Udi Meiri commented on BEAM-8944: - By current I don't mean the current Beam release but the current

[jira] [Commented] (BEAM-8944) Python SDK harness performance degradation with UnboundedThreadPoolExecutor

2019-12-19 Thread Udi Meiri (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17000313#comment-17000313 ] Udi Meiri commented on BEAM-8944: - Does this bug affect current production runners (such as Dataflow

[jira] [Commented] (BEAM-8944) Python SDK harness performance degradation with UnboundedThreadPoolExecutor

2019-12-11 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16994104#comment-16994104 ] Yichi Zhang commented on BEAM-8944: --- Seems like the original change was to solve deadlock and stuckness

[jira] [Commented] (BEAM-8944) Python SDK harness performance degradation with UnboundedThreadPoolExecutor

2019-12-10 Thread Yichi Zhang (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16993141#comment-16993141 ] Yichi Zhang commented on BEAM-8944: --- CC: [~angoenka] > Python SDK harness performance degradation with