[ https://issues.apache.org/jira/browse/STORM-1215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15009200#comment-15009200 ]
ASF GitHub Bot commented on STORM-1215: --------------------------------------- GitHub user kishorvpatil opened a pull request: https://github.com/apache/storm/pull/888 [STORM-1215] Use Asynchronous logger for improving performance. The changes to loggers for all daemons and workers. - Turn off immediateFlush. - Enable async logging. - Turn off Immediate Fail for log events on syslog logger to avoid making blocking call without timeout. You can merge this pull request into a Git repository by running: $ git pull https://github.com/kishorvpatil/incubator-storm STORM-1215 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/storm/pull/888.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #888 ---- commit 601efa5cad6604bf922f7f1ef843e3f8ae9bf1f2 Author: Kishor Patil <kpa...@yahoo-inc.com> Date: 2015-11-17T16:14:56Z All daemons should Async Logging to avoid locking issues commit 1660da0e36b6f5647bc21f9813f3eca1c3c53afa Author: Kishor Patil <kpa...@yahoo-inc.com> Date: 2015-11-17T17:23:38Z Make worker log appenders async for performance ---- > Use Async Loggers to avoid locking and logging overhead > -------------------------------------------------------- > > Key: STORM-1215 > URL: https://issues.apache.org/jira/browse/STORM-1215 > Project: Apache Storm > Issue Type: Improvement > Components: storm-core > Reporter: Kishor Patil > Assignee: Kishor Patil > > The loggers are synchronous with immediateFlush to disk, making some of the > daemons slow down. In some other cases, nimbus is slow too with submit-lock. > Making loggers asynchronous with no necessity to write to disk on every > logger event would improve cpu resource usage for logging. > {code} > "pool-7-thread-986" #1025 prio=5 os_prio=0 tid=0x00007f0f9628c800 nid=0x1b84 > runnable [0x00007f0f0fa2a000] > java.lang.Thread.State: RUNNABLE > at java.io.FileOutputStream.writeBytes(Native Method) > at java.io.FileOutputStream.write(FileOutputStream.java:326) > at > java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82) > at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140) > - locked <0x00000003c00ae520> (a java.io.BufferedOutputStream) > at java.io.PrintStream.write(PrintStream.java:482) > - locked <0x00000003c00ae500> (a java.io.PrintStream) > at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221) > at sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:291) > at sun.nio.cs.StreamEncoder.flushBuffer(StreamEncoder.java:104) > - locked <0x00000003c00ae640> (a java.io.OutputStreamWriter) > at java.io.OutputStreamWriter.flushBuffer(OutputStreamWriter.java:185) > at java.io.PrintStream.write(PrintStream.java:527) > - locked <0x00000003c00ae500> (a java.io.PrintStream) > at java.io.PrintStream.print(PrintStream.java:669) > at java.io.PrintStream.println(PrintStream.java:806) > - locked <0x00000003c00ae500> (a java.io.PrintStream) > at > org.apache.logging.log4j.status.StatusConsoleListener.log(StatusConsoleListener.java:81) > at > org.apache.logging.log4j.status.StatusLogger.logMessage(StatusLogger.java:218) > at > org.apache.logging.log4j.spi.AbstractLogger.logMessage(AbstractLogger.java:727) > at > org.apache.logging.log4j.spi.AbstractLogger.logIfEnabled(AbstractLogger.java:716) > at > org.apache.logging.log4j.spi.AbstractLogger.error(AbstractLogger.java:344) > at > org.apache.logging.log4j.core.appender.DefaultErrorHandler.error(DefaultErrorHandler.java:59) > at > org.apache.logging.log4j.core.appender.AbstractAppender.error(AbstractAppender.java:86) > at > org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.append(AbstractOutputStreamAppender.java:116) > at > org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:99) > at > org.apache.logging.log4j.core.config.LoggerConfig.callAppenders(LoggerConfig.java:430) > at > org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:409) > at > org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:367) > at org.apache.logging.log4j.core.Logger.logMessage(Logger.java:112) > at > org.apache.logging.log4j.spi.AbstractLogger.logMessage(AbstractLogger.java:727) > at > org.apache.logging.log4j.spi.AbstractLogger.logIfEnabled(AbstractLogger.java:716) > at org.apache.logging.slf4j.Log4jLogger.info(Log4jLogger.java:198) > at clojure.tools.logging$eval1$fn__7.invoke(NO_SOURCE_FILE:0) > at clojure.tools.logging.impl$fn__28$G__8__39.invoke(impl.clj:16) > at clojure.tools.logging$log_STAR_.invoke(logging.clj:59) > at backtype.storm.daemon.nimbus$mk_assignments.doInvoke(nimbus.clj:781) > at clojure.lang.RestFn.invoke(RestFn.java:410) > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)