[jira] [Updated] (SPARK-8835) Provide pluggable Congestion Strategies to deal with Streaming load
[ https://issues.apache.org/jira/browse/SPARK-8835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] François Garillot updated SPARK-8835: - Description: Second part of [SPARK-7398|https://issues.apache.org/jira/browse/SPARK-7398] (which has an over-arching, high-level design doc). An (outdated) [PoC implementation|https://github.com/huitseeker/spark/tree/ReactiveStreamingBackPressureControl/] exists. Note that throttling in the current (Spark 1.5.0) way pushes down the (fast-overflowing) buffer of received elements down to the TCP stack, which then has to deal with the break. This can be witnessed here : https://github.com/skyluc/spark-streaming-testbed/tree/test-runs-006/test-runs-006 was: Second part of [SPARK-7398|https://issues.apache.org/jira/browse/SPARK-7398] (which has an over-arching, high-level design doc). An (outdated) [PoC implementation|https://github.com/huitseeker/spark/tree/ReactiveStreamingBackPressureControl/] exists. > Provide pluggable Congestion Strategies to deal with Streaming load > --- > > Key: SPARK-8835 > URL: https://issues.apache.org/jira/browse/SPARK-8835 > Project: Spark > Issue Type: Sub-task > Components: Streaming >Reporter: François Garillot > > Second part of [SPARK-7398|https://issues.apache.org/jira/browse/SPARK-7398] > (which has an over-arching, high-level design doc). > An (outdated) [PoC > implementation|https://github.com/huitseeker/spark/tree/ReactiveStreamingBackPressureControl/] > exists. > Note that throttling in the current (Spark 1.5.0) way pushes down the > (fast-overflowing) buffer of received elements down to the TCP stack, which > then has to deal with the break. This can be witnessed here : > https://github.com/skyluc/spark-streaming-testbed/tree/test-runs-006/test-runs-006 -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-8835) Provide pluggable Congestion Strategies to deal with Streaming load
[ https://issues.apache.org/jira/browse/SPARK-8835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] François Garillot updated SPARK-8835: - Description: Second part of [SPARK-7398|https://issues.apache.org/jira/browse/SPARK-7398] (which has an over-arching, high-level design doc). An (outdated) [PoC implementation|https://github.com/huitseeker/spark/tree/ReactiveStreamingBackPressureControl/] exists. was: Second part of [SPARK-7398|https://issues.apache.org/jira/browse/SPARK-7398] (which has an over-arching, high-level design doc). An (outdated) [PoC implementation|https://github.com/typesafehub/spark/pull/13] exists. > Provide pluggable Congestion Strategies to deal with Streaming load > --- > > Key: SPARK-8835 > URL: https://issues.apache.org/jira/browse/SPARK-8835 > Project: Spark > Issue Type: Sub-task > Components: Streaming >Reporter: François Garillot > > Second part of [SPARK-7398|https://issues.apache.org/jira/browse/SPARK-7398] > (which has an over-arching, high-level design doc). > An (outdated) [PoC > implementation|https://github.com/huitseeker/spark/tree/ReactiveStreamingBackPressureControl/] > exists. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-8835) Provide pluggable Congestion Strategies to deal with Streaming load
[ https://issues.apache.org/jira/browse/SPARK-8835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] François Garillot updated SPARK-8835: - Shepherd: Tathagata Das > Provide pluggable Congestion Strategies to deal with Streaming load > --- > > Key: SPARK-8835 > URL: https://issues.apache.org/jira/browse/SPARK-8835 > Project: Spark > Issue Type: Sub-task > Components: Streaming >Reporter: François Garillot > > Second part of [SPARK-7398|https://issues.apache.org/jira/browse/SPARK-7398] > (which has an over-arching, high-level design doc). > An (outdated) [PoC > implementation|https://github.com/typesafehub/spark/pull/13] exists. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-8835) Provide pluggable Congestion Strategies to deal with Streaming load
[ https://issues.apache.org/jira/browse/SPARK-8835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] François Garillot updated SPARK-8835: - Description: Second part of [SPARK-7398|https://issues.apache.org/jira/browse/SPARK-7398] (which has an over-arching, high-level design doc). An (outdated) [PoC implementation|https://github.com/typesafehub/spark/pull/13] exists. > Provide pluggable Congestion Strategies to deal with Streaming load > --- > > Key: SPARK-8835 > URL: https://issues.apache.org/jira/browse/SPARK-8835 > Project: Spark > Issue Type: Sub-task > Components: Streaming >Reporter: François Garillot > > Second part of [SPARK-7398|https://issues.apache.org/jira/browse/SPARK-7398] > (which has an over-arching, high-level design doc). > An (outdated) [PoC > implementation|https://github.com/typesafehub/spark/pull/13] exists. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
