[GitHub] spark issue #21188: [SPARK-24046][SS] Fix rate source rowsPerSecond <= rampU...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21188 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/4227/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21188: [SPARK-24046][SS] Fix rate source rowsPerSecond <= rampU...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21188 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21188: [SPARK-24046][SS] Fix rate source rowsPerSecond <= rampU...
Github user xuanyuanking commented on the issue: https://github.com/apache/spark/pull/21188 @maasg as comment in #21194, I just consider we should not change the behavior while `seconds > rampUpTimeSeconds`. Maybe it more important than smooth. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21188: [SPARK-24046][SS] Fix rate source rowsPerSecond <= rampU...
Github user maasg commented on the issue: https://github.com/apache/spark/pull/21188 Hi Jerry, There's an improvement from the original situation, but the initial ramp-up phase starts only when the time gets very close to `rampUpTime`. Here you have another example that shows the behavior: Above: The original implementation Below: This PR. (Notebook src:https://gist.github.com/maasg/95eab0e49eeef9c7457bca2900bf412e)  This behavior is not consistent with the working, case when `rowsPerSecond > rampUpTime` In this case, the ramp-up is quite smooth:  --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21188: [SPARK-24046][SS] Fix rate source rowsPerSecond <= rampU...
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/21188 Isn't this a flat ramp-up smoothly increasing the rows per second? Your proposal is another solution, but just two options... --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21188: [SPARK-24046][SS] Fix rate source rowsPerSecond <= rampU...
Github user maasg commented on the issue: https://github.com/apache/spark/pull/21188 Hi Jerry, I don't think the issue is solved with this patch. I plugged the new function in my notebook and it still shows a rather flat ramp-up:  I created a PR with an alternative rate computation: https://github.com/apache/spark/pull/21194 Could you have a look? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21188: [SPARK-24046][SS] Fix rate source rowsPerSecond <= rampU...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21188 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89952/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21188: [SPARK-24046][SS] Fix rate source rowsPerSecond <= rampU...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21188 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21188: [SPARK-24046][SS] Fix rate source rowsPerSecond <= rampU...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21188 **[Test build #89952 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89952/testReport)** for PR 21188 at commit [`bf62aed`](https://github.com/apache/spark/commit/bf62aed080c9a2b6b46e8ee656c70b5ae76c0d45). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21188: [SPARK-24046][SS] Fix rate source rowsPerSecond <= rampU...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21188 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2736/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21188: [SPARK-24046][SS] Fix rate source rowsPerSecond <= rampU...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21188 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21188: [SPARK-24046][SS] Fix rate source rowsPerSecond <= rampU...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21188 **[Test build #89952 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89952/testReport)** for PR 21188 at commit [`bf62aed`](https://github.com/apache/spark/commit/bf62aed080c9a2b6b46e8ee656c70b5ae76c0d45). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
