[
https://issues.apache.org/jira/browse/SPARK-17231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Allman updated SPARK-17231:
-----------------------------------
Attachment: master 2.jpg
logging_perf_improvements 2.jpg
> Avoid building debug or trace log messages unless the respective log level is
> enabled
> -------------------------------------------------------------------------------------
>
> Key: SPARK-17231
> URL: https://issues.apache.org/jira/browse/SPARK-17231
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Affects Versions: 2.0.0
> Environment: Spark cluster with 8 r3.8xl EC2 worker instances
> Reporter: Michael Allman
> Priority: Minor
> Attachments: logging_perf_improvements 2.jpg,
> logging_perf_improvements.jpg, master 2.jpg, master.jpg
>
>
> While debugging the performance of a large GraphX connected components
> computation, I found several places in the {{network-common}} and
> {{network-shuffle}} code bases where trace or debug log messages are
> constructed even if the respective log level is disabled. Refactoring the
> respective code to avoid these constructions except where necessary led to a
> modest but measurable reduction in task time, GC time and the ratio thereof.
> (PR to come.)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
