[GitHub] [spark] otterc edited a comment on pull request #30139: [SPARK-31069][CORE] high cpu caused by chunksBeingTransferred in external shuffle service

[GitBox]

otterc edited a comment on pull request #30139:
URL: https://github.com/apache/spark/pull/30139#issuecomment-717001727


   > IMO, we can add a lock to keep strong consistence of value 
`totalChunksBeingTransferred`, such as
   > 
![image](https://user-images.githubusercontent.com/46485123/97126839-f2abfb80-1772-11eb-9271-54775defde35.png)
   > 
   > And I have run the test in desc, this change has little effect on 
performance. WDYT?
   
   This should have a considerable impact on the performance when there are 
multiple open streams because updates of different streams would lock on  a 
single object `totalChunksBeingTransferred`. Isn't that the case?
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org