[
https://issues.apache.org/jira/browse/LOG4J2-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15148374#comment-15148374
]
Remko Popma commented on LOG4J2-1274:
-------------------------------------
It is currently used in the JMS Appender, so looks like we need it.
> Layout improvements to enable avoiding temporary object allocation
> ------------------------------------------------------------------
>
> Key: LOG4J2-1274
> URL: https://issues.apache.org/jira/browse/LOG4J2-1274
> Project: Log4j 2
> Issue Type: Improvement
> Components: Layouts
> Affects Versions: 2.5
> Reporter: Remko Popma
>
> *Problem*
> The current Layout API does not make it easy for implementors to avoid
> creating temporary objects. Especially these methods:
> {code}
> byte[] toByteArray(LogEvent);
> T toSerializable(LogEvent);
> {code}
> The byte array returned from {{toByteArray(LogEvent)}} cannot be re-used
> between log events, since the caller cannot know how many bytes a partially
> filled array contains.
> In practice, all Layout implementations in Log4j 2 except SerializedLayout
> implement the {{StringLayout}} subinterface. This means that the
> {{toSerializable()}} method needs to return a new String object for every log
> event.
> *Forces*
> I am interested in reducing or even eliminating the allocation of temporary
> objects for text-based layouts. Many of these use (and re-use) a
> StringBuilder to build a text representation of the current log event. Once
> this text representation is built, it needs to be converted to bytes that the
> Appender can consume. I am aware of two ways in the JDK to convert text to
> bytes:
> * the various {{String#getBytes}} methods - these all allocate a new byte
> array for each invocation
> * the underlying {{java.nio.charset.CharsetEncoder}} used internally by
> String - especially method {{CoderResult encode(CharBuffer in, ByteBuffer
> out, boolean endOfInput)}} which converts characters to bytes without object
> allocation.
> The last method is interesting because this gives us an opportunity to also
> reduce the amount of copying by directly supplying the ByteBuffer buffer used
> by RandomAccessFileAppender, or the MappedByteBuffer of the
> MemoryMappedFileAppender.
> The resulting API needs to support the fact that implementations may need to
> call {{CharsetEncoder#encode}} multiple times:
> * The ByteBuffer may not have enough remaining space to hold all the data;
> {{CharsetEncoder#encode}} returns {{CoderResult.OVERFLOW}} to signal this so
> the caller can consume the contents and reset/clear the buffer before
> continuing.
> * The CharBuffer may not be large enough to hold the full text representation
> of the log event. Again, {{CharsetEncoder#encode}} may need to be invoked
> multiple times.
> *Proposal*
> (Thinking out loud here, I'm open to suggestions.)
> It may be sufficient for the layout interface to have a single additional new
> method:
> {code}
> /**
> * Formats the event suitable for display and writes the result to the
> specified destination.
> *
> * @param event The Logging Event.
> * @param destination Holds the ByteBuffer to write into.
> */
> void writeTo(LogEvent e, ByteBufferDestination destination);
> {code}
> Appenders that want to be allocation-free need to implement the
> {{ByteBufferDestination}} interface:
> {code}
> public interface ByteBufferDestination {
> ByteBuffer getByteBuffer();
> /**
> * Consumes the buffer content and returns a buffer with more available()
> space
> * (which may or may not be the same instance).
> * <p>
> * Called by the producer when the buffer becomes too full
> * to write more data into it.
> */
> ByteBuffer drain(ByteBuffer buf);
> }
> {code}
> Usage: for example RandomAccessFileAppender code can look like this:
> {code}
> // RandomAccessFileAppender
> public void append(final LogEvent event) {
> getLayout().writeTo(event, (ByteBufferDestination) manager);
> }
> {code}
> Layout implementation of the {{writeTo}} method: layouts need to know how to
> convert LogEvents to text, but writing this text into the ByteBuffer can be
> delegated to a helper:
> {code}
> // some layout
> public void writeTo(LogEvent event, ByteBufferDestination destination) {
> StringBuilder text = toText(event, getCachedStringBuilder());
>
> TextEncoderHelper helper = getCachedHelper();
> helper.encodeWithoutAllocation(text, destination);
> }
> /**
> * Creates a text representation of the specified log event
> * and writes it into the specified StringBuilder.
> * <p>
> * Implementations are free to return a new StringBuilder if they can
> * detect in advance that the specified StringBuilder is too small.
> */
> StringBuilder toText(LogEvent e, StringBuilder destination) {} // existing
> logic goes here
> public String toSerializable(LogEvent event) { // factored out logic to
> toText()
> return toText(event, getCachedStringBuilder()).toString();
> }
> {code}
> Helper contains utility code for moving the text into a CharBuffer, and for
> repeatedly calling {{CharsetEncoder#encode}}.
> {code}
> public class TextEncoderHelper {
> TextEncoderHelper(Charset charset) {} // create CharsetEncoder
> void encodeWithoutAllocation(StringBuilder text, BinaryDestination
> destination) {
> ByteBuffer byteBuf = destination.getByteBuffer();
> CharBuffer charBuf = getCachedCharBuffer();
> charBuf.reset();
> int start = 0;
> int todoChars = text.length();
> do {
> int copied = copy(text, start, charBuf);
> start += copied;
> todoChars -= copied;
> boolean endOfInput = todoChars <= 0;
>
> charBuf.flip();
> CodeResult result;
> do {
> result = charsetEncoder.encode(charBuf, byteBuf, endOfInput);
> if (result == CodeResult.OVERFLOW) { // byteBuf full
> // destination consumes contents
> // and returns byte buffer with more capacity
> byteBuf = destination.drain(byteBuf);
> }
> } while (result == CodeResult.OVERFLOW);
> } while (!endOfInput);
> }
>
> /**
> * Copies characters from the StringBuilder into the CharBuffer,
> * starting at the specified offset and ending when either all
> * characters have been copied or when the CharBuffer is full.
> *
> * @return the number of characters that were copied
> */
> int copy(StringBuilder source, int offset, CharBuffer destination) {}
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org
For additional commands, e-mail: log4j-dev-h...@logging.apache.org
