Github user yuanboliu commented on the issue:
https://github.com/apache/spark/pull/21690
Thanks very much
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user yuanboliu commented on the issue:
https://github.com/apache/spark/pull/21690
After applying this patch, my application can be running successfully. This
issue could happen in the case of many topics(hundreds of ) consumed
Github user yuanboliu commented on the issue:
https://github.com/apache/spark/pull/21690
@koeninger Sorry to interrupt, could you take a look at my patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr
Github user yuanboliu commented on the issue:
https://github.com/apache/spark/pull/21690
@koeninger Thanks for your reply. Agree with you. there is no need to to
use pause repeatedly.
This is my test without any pause, and the app master stuck for a long time
without any process
Github user yuanboliu commented on the issue:
https://github.com/apache/spark/pull/21690
@koeninger Thanks for your details. Sorry quite busy this week. I will
delete the last pause, test the patch on my own cluster this weekend and give
feedback asap
Github user yuanboliu commented on the issue:
https://github.com/apache/spark/pull/21690
The first pause is used to stop poll() in the method paranoidPoll
The second one is attached because of p.partition().
I'm not sure whether the state of pause will be rewritten after
Github user yuanboliu commented on the issue:
https://github.com/apache/spark/pull/21690
@koeninger Sorry to interrupt, would you please review my patch? Thanks in
advance.
---
-
To unsubscribe, e-mail: reviews
Github user yuanboliu commented on the issue:
https://github.com/apache/spark/pull/21690
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
GitHub user yuanboliu opened a pull request:
https://github.com/apache/spark/pull/21690
[SPARK-24713]AppMatser of spark streaming kafka OOM if there are hundâ¦
We have hundreds of kafka topics need to be consumed in one application.
The application master will throw OOM exception
