[GitHub] [spark] SaurabhChawla100 edited a comment on pull request #29413: [SPARK-32597][CORE] Tune Event Drop in Async Event Queue

2020-08-16 Thread GitBox
SaurabhChawla100 edited a comment on pull request #29413: URL: https://github.com/apache/spark/pull/29413#issuecomment-674664896 > Yeah so you have described the affects of it dropping the events, which I know. The thing I want to know is why it dropped the events in your cases.

[GitHub] [spark] SaurabhChawla100 edited a comment on pull request #29413: [SPARK-32597][CORE] Tune Event Drop in Async Event Queue

2020-08-13 Thread GitBox
SaurabhChawla100 edited a comment on pull request #29413: URL: https://github.com/apache/spark/pull/29413#issuecomment-673593703 >I don't get your point here. it can already be configured on a per queue basis (set "spark.scheduler.listenerbus.eventqueue.$name.capacity") and if user sets

[GitHub] [spark] SaurabhChawla100 edited a comment on pull request #29413: [SPARK-32597][CORE] Tune Event Drop in Async Event Queue

2020-08-13 Thread GitBox
SaurabhChawla100 edited a comment on pull request #29413: URL: https://github.com/apache/spark/pull/29413#issuecomment-673398423 > sorry I'm still not seeing any difference here then increasing the size of the current queue? If both are not really allocating memory for the entire amount

[GitHub] [spark] SaurabhChawla100 edited a comment on pull request #29413: [SPARK-32597][CORE] Tune Event Drop in Async Event Queue

2020-08-12 Thread GitBox
SaurabhChawla100 edited a comment on pull request #29413: URL: https://github.com/apache/spark/pull/29413#issuecomment-672918107 > so one question, here, you are seeing events dropped that cause hangs in the application code? What queue were they in? So we have seen the event drop

[GitHub] [spark] SaurabhChawla100 edited a comment on pull request #29413: [SPARK-32597][CORE] Tune Event Drop in Async Event Queue

2020-08-12 Thread GitBox
SaurabhChawla100 edited a comment on pull request #29413: URL: https://github.com/apache/spark/pull/29413#issuecomment-672918107 > so one question, here, you are seeing events dropped that cause hangs in the application code? What queue were they in? So we have seen the event drop