[ 
https://issues.apache.org/jira/browse/SPARK-17231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Allman updated SPARK-17231:
-----------------------------------
    Attachment: master 2.jpg
                logging_perf_improvements 2.jpg

> Avoid building debug or trace log messages unless the respective log level is 
> enabled
> -------------------------------------------------------------------------------------
>
>                 Key: SPARK-17231
>                 URL: https://issues.apache.org/jira/browse/SPARK-17231
>             Project: Spark
>          Issue Type: Improvement
>          Components: Spark Core
>    Affects Versions: 2.0.0
>         Environment: Spark cluster with 8 r3.8xl EC2 worker instances
>            Reporter: Michael Allman
>            Priority: Minor
>         Attachments: logging_perf_improvements 2.jpg, 
> logging_perf_improvements.jpg, master 2.jpg, master.jpg
>
>
> While debugging the performance of a large GraphX connected components 
> computation, I found several places in the {{network-common}} and 
> {{network-shuffle}} code bases where trace or debug log messages are 
> constructed even if the respective log level is disabled. Refactoring the 
> respective code to avoid these constructions except where necessary led to a 
> modest but measurable reduction in task time, GC time and the ratio thereof.
> (PR to come.)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

Reply via email to