[GitHub] [incubator-uniffle] zuston opened a new issue, #90: [Performance Optimization] Improve the speed of writing index file in shuffle server

Wed, 27 Jul 2022 03:26:24 -0700 


zuston opened a new issue, #90:
URL: https://github.com/apache/incubator-uniffle/issues/90

   ### Motivation
   When I test uniffle performance, i found a huge performance drop due to the 
low speed of writing index file. Flame graph attached:
   
   
![reliao_img_1658917352873](https://user-images.githubusercontent.com/8609142/181224611-faf20e2b-a65a-44ad-ad15-560addf43c85.png)
   
   ### Solution
   Use the `dataOutputStream = new DataOutputStream(new 
BufferedOutputStream(fileOutputStream));` in `LocalFileWriter`. Please refer 
to: 
https://stackoverflow.com/questions/38439410/java-dataoutputstream-writelong-byte-double-int-speed
   
   ### Result
   
   After apply this optimization:
   
![reliao_img_1658917526994](https://user-images.githubusercontent.com/8609142/181225186-b883658b-ac40-40a2-8cb7-22e154bc3d7f.png)
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to