[jira] [Updated] (LOG4J2-163) Create asynchronous Logger for low-latency logging
[ https://issues.apache.org/jira/browse/LOG4J2-163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Remko Popma updated LOG4J2-163: --- Attachment: (was: documentation-update-20130402.patch) Create asynchronous Logger for low-latency logging -- Key: LOG4J2-163 URL: https://issues.apache.org/jira/browse/LOG4J2-163 Project: Log4j 2 Issue Type: Improvement Affects Versions: 2.0-beta4 Reporter: Remko Popma Assignee: Ralph Goers Fix For: 2.0-beta5 Attachments: async-perf-comparison.png, FastLog4j-v2-for-beta4.zip, FastLog4j-v3-for-beta4.zip, FastLog4j-v4-for-beta4.zip, LOG4J2-163-log4j-async-20130320.patch, LOG4J2-163-log4j-async-20130331a.patch, LOG4J2-163-log4j-async-20130331-images.zip, LOG4J2-163-log4j-async-20130331.patch, LOG4J2-163-log4j-async.patch One of the main considerations for selecting a logging library is performance, specifically, how long it takes for a call to Logger.log to return. (See the comments of LOG4J-151 for a discussion of latency versus application throughput and logging throughput.) I believe it is possible to improve this performance by an order of magnitude by having an asynchronous Logger implementation that hands off the work to a separate thread as early as possible. The disk I/O would be done in this separate thread. AsynchAppender is not a good match for these requirements, as with that approach (a) the logging call still needs to flow down the hierarchy to the appender, doing synchronization and creating objects at various points on the way, and (b) when serializing the LogEvent, the getSource() method is always called, which is expensive. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org For additional commands, e-mail: log4j-dev-h...@logging.apache.org
[jira] [Updated] (LOG4J2-163) Create asynchronous Logger for low-latency logging
[ https://issues.apache.org/jira/browse/LOG4J2-163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Remko Popma updated LOG4J2-163: --- Attachment: (was: documentation-update-images-20130402.patch) Create asynchronous Logger for low-latency logging -- Key: LOG4J2-163 URL: https://issues.apache.org/jira/browse/LOG4J2-163 Project: Log4j 2 Issue Type: Improvement Affects Versions: 2.0-beta4 Reporter: Remko Popma Assignee: Ralph Goers Fix For: 2.0-beta5 Attachments: async-perf-comparison.png, FastLog4j-v2-for-beta4.zip, FastLog4j-v3-for-beta4.zip, FastLog4j-v4-for-beta4.zip, LOG4J2-163-log4j-async-20130320.patch, LOG4J2-163-log4j-async-20130331a.patch, LOG4J2-163-log4j-async-20130331-images.zip, LOG4J2-163-log4j-async-20130331.patch, LOG4J2-163-log4j-async.patch One of the main considerations for selecting a logging library is performance, specifically, how long it takes for a call to Logger.log to return. (See the comments of LOG4J-151 for a discussion of latency versus application throughput and logging throughput.) I believe it is possible to improve this performance by an order of magnitude by having an asynchronous Logger implementation that hands off the work to a separate thread as early as possible. The disk I/O would be done in this separate thread. AsynchAppender is not a good match for these requirements, as with that approach (a) the logging call still needs to flow down the hierarchy to the appender, doing synchronization and creating objects at various points on the way, and (b) when serializing the LogEvent, the getSource() method is always called, which is expensive. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org For additional commands, e-mail: log4j-dev-h...@logging.apache.org
[jira] [Updated] (LOG4J2-163) Create asynchronous Logger for low-latency logging
[ https://issues.apache.org/jira/browse/LOG4J2-163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Remko Popma updated LOG4J2-163: --- Attachment: documentation-update-images-20130402.patch documentation-update-20130402.patch Please find attached files documentation-update-images-20130402.patch and documentation-update-20130402.patch. They contain updates for the async manual page, the overall performance page and the team page (I added myself as a contributor, I hope that is ok.) Please unzip the images into the src/site/resources/images/ directory. Create asynchronous Logger for low-latency logging -- Key: LOG4J2-163 URL: https://issues.apache.org/jira/browse/LOG4J2-163 Project: Log4j 2 Issue Type: Improvement Affects Versions: 2.0-beta4 Reporter: Remko Popma Assignee: Ralph Goers Fix For: 2.0-beta5 Attachments: async-perf-comparison.png, documentation-update-20130402.patch, documentation-update-images-20130402.patch, FastLog4j-v2-for-beta4.zip, FastLog4j-v3-for-beta4.zip, FastLog4j-v4-for-beta4.zip, LOG4J2-163-log4j-async-20130320.patch, LOG4J2-163-log4j-async-20130331a.patch, LOG4J2-163-log4j-async-20130331-images.zip, LOG4J2-163-log4j-async-20130331.patch, LOG4J2-163-log4j-async.patch One of the main considerations for selecting a logging library is performance, specifically, how long it takes for a call to Logger.log to return. (See the comments of LOG4J-151 for a discussion of latency versus application throughput and logging throughput.) I believe it is possible to improve this performance by an order of magnitude by having an asynchronous Logger implementation that hands off the work to a separate thread as early as possible. The disk I/O would be done in this separate thread. AsynchAppender is not a good match for these requirements, as with that approach (a) the logging call still needs to flow down the hierarchy to the appender, doing synchronization and creating objects at various points on the way, and (b) when serializing the LogEvent, the getSource() method is always called, which is expensive. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org For additional commands, e-mail: log4j-dev-h...@logging.apache.org
[jira] [Updated] (LOG4J2-163) Create asynchronous Logger for low-latency logging
[ https://issues.apache.org/jira/browse/LOG4J2-163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Remko Popma updated LOG4J2-163: --- Attachment: LOG4J2-163-log4j-async-20130331a.patch Ralph, thanks for your feedback! Please find attached file LOG4J2-163-log4j-async-20130331a.patch This fixes the issues you mentioned: * removed @author tags from source code * removed com.lmax classes, declared dependency in pom, fixed imports and updated documentation The disruptor version I used is 3.0.0.beta3. (the images referenced in the manual have not changed, so I did not upload them again. Please unzip them into the src/site/resources/images/ directory) Create asynchronous Logger for low-latency logging -- Key: LOG4J2-163 URL: https://issues.apache.org/jira/browse/LOG4J2-163 Project: Log4j 2 Issue Type: Improvement Affects Versions: 2.0-beta4 Reporter: Remko Popma Attachments: async-perf-comparison.png, FastLog4j-v2-for-beta4.zip, FastLog4j-v3-for-beta4.zip, FastLog4j-v4-for-beta4.zip, LOG4J2-163-log4j-async-20130320.patch, LOG4J2-163-log4j-async-20130331a.patch, LOG4J2-163-log4j-async-20130331-images.zip, LOG4J2-163-log4j-async-20130331.patch, LOG4J2-163-log4j-async.patch One of the main considerations for selecting a logging library is performance, specifically, how long it takes for a call to Logger.log to return. (See the comments of LOG4J-151 for a discussion of latency versus application throughput and logging throughput.) I believe it is possible to improve this performance by an order of magnitude by having an asynchronous Logger implementation that hands off the work to a separate thread as early as possible. The disk I/O would be done in this separate thread. AsynchAppender is not a good match for these requirements, as with that approach (a) the logging call still needs to flow down the hierarchy to the appender, doing synchronization and creating objects at various points on the way, and (b) when serializing the LogEvent, the getSource() method is always called, which is expensive. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org For additional commands, e-mail: log4j-dev-h...@logging.apache.org
[jira] [Updated] (LOG4J2-163) Create asynchronous Logger for low-latency logging
[ https://issues.apache.org/jira/browse/LOG4J2-163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Remko Popma updated LOG4J2-163: --- Attachment: LOG4J2-163-log4j-async-20130331.patch Please find attached file LOG4J2-163-log4j-async-20130331.patch Changes: * LOG4J2-153 includeLocation attribute on LoggerConfig and AsyncAppender with proper defaulting * Updated async manual page Create asynchronous Logger for low-latency logging -- Key: LOG4J2-163 URL: https://issues.apache.org/jira/browse/LOG4J2-163 Project: Log4j 2 Issue Type: Improvement Affects Versions: 2.0-beta4 Reporter: Remko Popma Attachments: async-perf-comparison.png, FastLog4j-v2-for-beta4.zip, FastLog4j-v3-for-beta4.zip, FastLog4j-v4-for-beta4.zip, LOG4J2-163-log4j-async-20130320.patch, LOG4J2-163-log4j-async-20130331.patch, LOG4J2-163-log4j-async.patch One of the main considerations for selecting a logging library is performance, specifically, how long it takes for a call to Logger.log to return. (See the comments of LOG4J-151 for a discussion of latency versus application throughput and logging throughput.) I believe it is possible to improve this performance by an order of magnitude by having an asynchronous Logger implementation that hands off the work to a separate thread as early as possible. The disk I/O would be done in this separate thread. AsynchAppender is not a good match for these requirements, as with that approach (a) the logging call still needs to flow down the hierarchy to the appender, doing synchronization and creating objects at various points on the way, and (b) when serializing the LogEvent, the getSource() method is always called, which is expensive. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org For additional commands, e-mail: log4j-dev-h...@logging.apache.org
[jira] [Updated] (LOG4J2-163) Create asynchronous Logger for low-latency logging
[ https://issues.apache.org/jira/browse/LOG4J2-163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Remko Popma updated LOG4J2-163: --- Attachment: LOG4J2-163-log4j-async-20130331-images.zip Create asynchronous Logger for low-latency logging -- Key: LOG4J2-163 URL: https://issues.apache.org/jira/browse/LOG4J2-163 Project: Log4j 2 Issue Type: Improvement Affects Versions: 2.0-beta4 Reporter: Remko Popma Attachments: async-perf-comparison.png, FastLog4j-v2-for-beta4.zip, FastLog4j-v3-for-beta4.zip, FastLog4j-v4-for-beta4.zip, LOG4J2-163-log4j-async-20130320.patch, LOG4J2-163-log4j-async-20130331-images.zip, LOG4J2-163-log4j-async-20130331.patch, LOG4J2-163-log4j-async.patch One of the main considerations for selecting a logging library is performance, specifically, how long it takes for a call to Logger.log to return. (See the comments of LOG4J-151 for a discussion of latency versus application throughput and logging throughput.) I believe it is possible to improve this performance by an order of magnitude by having an asynchronous Logger implementation that hands off the work to a separate thread as early as possible. The disk I/O would be done in this separate thread. AsynchAppender is not a good match for these requirements, as with that approach (a) the logging call still needs to flow down the hierarchy to the appender, doing synchronization and creating objects at various points on the way, and (b) when serializing the LogEvent, the getSource() method is always called, which is expensive. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org For additional commands, e-mail: log4j-dev-h...@logging.apache.org
[jira] [Updated] (LOG4J2-163) Create asynchronous Logger for low-latency logging
[ https://issues.apache.org/jira/browse/LOG4J2-163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Remko Popma updated LOG4J2-163: --- Attachment: async-perf-comparison.png Just for fun I attached a graph comparing the throughput performance of async loggers to async appenders (incl. log4j-1.x and logback). Ralph, I hope to submit an updated patch that includes the LOG4J2-153 changes this weekend. Create asynchronous Logger for low-latency logging -- Key: LOG4J2-163 URL: https://issues.apache.org/jira/browse/LOG4J2-163 Project: Log4j 2 Issue Type: Improvement Affects Versions: 2.0-beta4 Reporter: Remko Popma Attachments: async-perf-comparison.png, FastLog4j-v2-for-beta4.zip, FastLog4j-v3-for-beta4.zip, FastLog4j-v4-for-beta4.zip, LOG4J2-163-log4j-async-20130320.patch, LOG4J2-163-log4j-async.patch One of the main considerations for selecting a logging library is performance, specifically, how long it takes for a call to Logger.log to return. (See the comments of LOG4J-151 for a discussion of latency versus application throughput and logging throughput.) I believe it is possible to improve this performance by an order of magnitude by having an asynchronous Logger implementation that hands off the work to a separate thread as early as possible. The disk I/O would be done in this separate thread. AsynchAppender is not a good match for these requirements, as with that approach (a) the logging call still needs to flow down the hierarchy to the appender, doing synchronization and creating objects at various points on the way, and (b) when serializing the LogEvent, the getSource() method is always called, which is expensive. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org For additional commands, e-mail: log4j-dev-h...@logging.apache.org
[jira] [Updated] (LOG4J2-163) Create asynchronous Logger for low-latency logging
[ https://issues.apache.org/jira/browse/LOG4J2-163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Remko Popma updated LOG4J2-163: --- Attachment: async-throughput-comparison.jpg Create asynchronous Logger for low-latency logging -- Key: LOG4J2-163 URL: https://issues.apache.org/jira/browse/LOG4J2-163 Project: Log4j 2 Issue Type: Improvement Affects Versions: 2.0-beta4 Reporter: Remko Popma Attachments: async-perf-comparison.png, async-throughput-comparison.jpg, FastLog4j-v2-for-beta4.zip, FastLog4j-v3-for-beta4.zip, FastLog4j-v4-for-beta4.zip, LOG4J2-163-log4j-async-20130320.patch, LOG4J2-163-log4j-async.patch One of the main considerations for selecting a logging library is performance, specifically, how long it takes for a call to Logger.log to return. (See the comments of LOG4J-151 for a discussion of latency versus application throughput and logging throughput.) I believe it is possible to improve this performance by an order of magnitude by having an asynchronous Logger implementation that hands off the work to a separate thread as early as possible. The disk I/O would be done in this separate thread. AsynchAppender is not a good match for these requirements, as with that approach (a) the logging call still needs to flow down the hierarchy to the appender, doing synchronization and creating objects at various points on the way, and (b) when serializing the LogEvent, the getSource() method is always called, which is expensive. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org For additional commands, e-mail: log4j-dev-h...@logging.apache.org
[jira] [Updated] (LOG4J2-163) Create asynchronous Logger for low-latency logging
[ https://issues.apache.org/jira/browse/LOG4J2-163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Remko Popma updated LOG4J2-163: --- Attachment: (was: async-throughput-comparison.jpg) Create asynchronous Logger for low-latency logging -- Key: LOG4J2-163 URL: https://issues.apache.org/jira/browse/LOG4J2-163 Project: Log4j 2 Issue Type: Improvement Affects Versions: 2.0-beta4 Reporter: Remko Popma Attachments: async-perf-comparison.png, FastLog4j-v2-for-beta4.zip, FastLog4j-v3-for-beta4.zip, FastLog4j-v4-for-beta4.zip, LOG4J2-163-log4j-async-20130320.patch, LOG4J2-163-log4j-async.patch One of the main considerations for selecting a logging library is performance, specifically, how long it takes for a call to Logger.log to return. (See the comments of LOG4J-151 for a discussion of latency versus application throughput and logging throughput.) I believe it is possible to improve this performance by an order of magnitude by having an asynchronous Logger implementation that hands off the work to a separate thread as early as possible. The disk I/O would be done in this separate thread. AsynchAppender is not a good match for these requirements, as with that approach (a) the logging call still needs to flow down the hierarchy to the appender, doing synchronization and creating objects at various points on the way, and (b) when serializing the LogEvent, the getSource() method is always called, which is expensive. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org For additional commands, e-mail: log4j-dev-h...@logging.apache.org
[jira] [Updated] (LOG4J2-163) Create asynchronous Logger for low-latency logging
[ https://issues.apache.org/jira/browse/LOG4J2-163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Remko Popma updated LOG4J2-163: --- Attachment: LOG4J2-163-log4j-async-20130320.patch Please find attached file LOG4J2-163-log4j-async-20130320.patch. Changes: * Updated for changes in trunk * Upgraded to latest version of LMAX Disruptor (3.0.0-beta3) for better performance in multi-threaded scenarios * Added manual page (src/site/xdoc/manual/async.xml) * Added performance and JUnit tests * Various small improvements As before the patch file contains the new module log4j-async, and also contains patches for the following JIRA tickets that the log4j-async module depends on: LOG4J2-151 LOG4J2-153 LOG4J2-157 LOG4J2-164 I also updated pom.xml to add the module and change the compilation version to Java 6. Create asynchronous Logger for low-latency logging -- Key: LOG4J2-163 URL: https://issues.apache.org/jira/browse/LOG4J2-163 Project: Log4j 2 Issue Type: Improvement Affects Versions: 2.0-beta4 Reporter: Remko Popma Attachments: FastLog4j-v2-for-beta4.zip, FastLog4j-v3-for-beta4.zip, FastLog4j-v4-for-beta4.zip, LOG4J2-163-log4j-async-20130320.patch, LOG4J2-163-log4j-async.patch One of the main considerations for selecting a logging library is performance, specifically, how long it takes for a call to Logger.log to return. (See the comments of LOG4J-151 for a discussion of latency versus application throughput and logging throughput.) I believe it is possible to improve this performance by an order of magnitude by having an asynchronous Logger implementation that hands off the work to a separate thread as early as possible. The disk I/O would be done in this separate thread. AsynchAppender is not a good match for these requirements, as with that approach (a) the logging call still needs to flow down the hierarchy to the appender, doing synchronization and creating objects at various points on the way, and (b) when serializing the LogEvent, the getSource() method is always called, which is expensive. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org For additional commands, e-mail: log4j-dev-h...@logging.apache.org
[jira] [Updated] (LOG4J2-163) Create asynchronous Logger for low-latency logging
[ https://issues.apache.org/jira/browse/LOG4J2-163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Remko Popma updated LOG4J2-163: --- Attachment: LOG4J2-163-log4j-async.patch Sorry for the long wait. Please find attached file LOG4J2-163-log4j-async.patch. This file contains the async loggers and FastFileAppenders in a new module log4j-async, and also contains patches for the following JIRA tickets that the log4j-async module depends on: LOG4J2-151 LOG4J2-153 LOG4J2-157 LOG4J2-164 I also updated pom.xml to add the module and change the compilation version to Java 6. Create asynchronous Logger for low-latency logging -- Key: LOG4J2-163 URL: https://issues.apache.org/jira/browse/LOG4J2-163 Project: Log4j 2 Issue Type: Improvement Affects Versions: 2.0-beta4 Reporter: Remko Popma Attachments: FastLog4j-v2-for-beta4.zip, FastLog4j-v3-for-beta4.zip, FastLog4j-v4-for-beta4.zip, LOG4J2-163-log4j-async.patch One of the main considerations for selecting a logging library is performance, specifically, how long it takes for a call to Logger.log to return. (See the comments of LOG4J-151 for a discussion of latency versus application throughput and logging throughput.) I believe it is possible to improve this performance by an order of magnitude by having an asynchronous Logger implementation that hands off the work to a separate thread as early as possible. The disk I/O would be done in this separate thread. AsynchAppender is not a good match for these requirements, as with that approach (a) the logging call still needs to flow down the hierarchy to the appender, doing synchronization and creating objects at various points on the way, and (b) when serializing the LogEvent, the getSource() method is always called, which is expensive. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org For additional commands, e-mail: log4j-dev-h...@logging.apache.org
[jira] [Updated] (LOG4J2-163) Create asynchronous Logger for low-latency logging
[ https://issues.apache.org/jira/browse/LOG4J2-163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Remko Popma updated LOG4J2-163: --- Attachment: FastLog4j-v3-for-beta4.zip FastLog4j-v2-for-beta4.zip Create asynchronous Logger for low-latency logging -- Key: LOG4J2-163 URL: https://issues.apache.org/jira/browse/LOG4J2-163 Project: Log4j 2 Issue Type: Improvement Affects Versions: 2.0-beta4 Reporter: Remko Popma Attachments: FastLog4j-v2-for-beta4.zip, FastLog4j-v3-for-beta4.zip One of the main considerations for selecting a logging library is performance, specifically, how long it takes for a call to Logger.log to return. (See the comments of LOG4J-151 for a discussion of latency versus application throughput and logging throughput.) I believe it is possible to improve this performance by an order of magnitude by having an asynchronous Logger implementation that hands off the work to a separate thread as early as possible. The disk I/O would be done in this separate thread. AsynchAppender is not a good match for these requirements, as with that approach (a) the logging call still needs to flow down the hierarchy to the appender, doing synchronization and creating objects at various points on the way, and (b) when serializing the LogEvent, the getSource() method is always called, which is expensive. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org For additional commands, e-mail: log4j-dev-h...@logging.apache.org
