[
https://issues.apache.org/jira/browse/LOG4J2-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15146850#comment-15146850
]
Remko Popma commented on LOG4J2-1274:
-------------------------------------
I like the name ByteBufferDestination.
About my question: Is there any concern that adding a method would break user
code if users created a custom Layout implementation? (The Layout interface is
in log4j-core, not log4j-api, which gives us more freedom, but I'd like to
confirm we're all okay with this.)
> Layout improvements to enable avoiding temporary object allocation
> ------------------------------------------------------------------
>
> Key: LOG4J2-1274
> URL: https://issues.apache.org/jira/browse/LOG4J2-1274
> Project: Log4j 2
> Issue Type: Improvement
> Components: Layouts
> Affects Versions: 2.5
> Reporter: Remko Popma
>
> *Problem*
> The current Layout API does not make it easy for implementors to avoid
> creating temporary objects. Especially these methods:
> {code}
> byte[] toByteArray(LogEvent);
> T toSerializable(LogEvent);
> {code}
> The byte array returned from {{toByteArray(LogEvent)}} cannot be re-used
> between log events, since the caller cannot know how many bytes a partially
> filled array contains.
> In practice, all Layout implementations in Log4j 2 except SerializedLayout
> implement the {{StringLayout}} subinterface. This means that the
> {{toSerializable()}} method needs to return a new String object for every log
> event.
> *Forces*
> I am interested in reducing or even eliminating the allocation of temporary
> objects for text-based layouts. Many of these use (and re-use) a
> StringBuilder to build a text representation of the current log event. Once
> this text representation is built, it needs to be converted to bytes that the
> Appender can consume. I am aware of two ways in the JDK to convert text to
> bytes:
> * the various {{String#getBytes}} methods - these all allocate a new byte
> array for each invocation
> * the underlying {{java.nio.charset.CharsetEncoder}} used internally by
> String - especially method {{CoderResult encode(CharBuffer in, ByteBuffer
> out, boolean endOfInput)}} which converts characters to bytes without object
> allocation.
> The last method is interesting because this gives us an opportunity to also
> reduce the amount of copying by directly supplying the ByteBuffer buffer used
> by RandomAccessFileAppender, or the MappedByteBuffer of the
> MemoryMappedFileAppender.
> The resulting API needs to support the fact that implementations may need to
> call {{CharsetEncoder#encode}} multiple times:
> * The ByteBuffer may not have enough remaining space to hold all the data;
> {{CharsetEncoder#encode}} returns {{CoderResult.OVERFLOW}} to signal this so
> the caller can consume the contents and reset/clear the buffer before
> continuing.
> * The CharBuffer may not be large enough to hold the full text representation
> of the log event. Again, {{CharsetEncoder#encode}} may need to be invoked
> multiple times.
> *Proposal*
> (Thinking out loud here, I'm open to suggestions.)
> It may be sufficient for the layout interface to have a single method where
> callers can specify the destination StringBuilder.
> {code}
> public interface TextLayout {
> /**
> * Creates a text representation of the specified log event
> * and writes it into the specified StringBuilder.
> * <p>
> * Implementations are free to return a new StringBuilder if they can
> * detect in advance that the specified StringBuilder is too small.
> */
> StringBuilder toText(LogEvent e, StringBuilder destination);
> }
> {code}
> Appenders that use ByteBuffers can implement this interface:
> {code}
> public interface BinaryDestination {
> ByteBuffer getByteBuffer();
> /**
> * Consumes the buffer content and clear()s it.
> */
> void drain(ByteBuffer buf);
> }
> {code}
> Then Appender code can look like this, delegating all the work to a helper:
> {code}
> // appender
> public void append(final LogEvent event) {
> if (getLayout() instanceof TextLayout) {
> TextEncoderHelper helper = ... // get cached helper
> helper.appendWithoutAllocation(event, (TextLayout) getLayout(),
> (BinaryDestination) this);
> } else {
> // current append logic
> ...
> }
> }
> {code}
> Helper contains re-usable code for moving the text into a CharBuffer, and for
> repeatedly calling {{CharsetEncoder#encode}}.
> {code}
> public class TextEncoderHelper {
> TextEncoderHelper(Charset charset) {}
> void appendWithoutAllocation(LogEvent event, TextLayout textLayout,
> BinaryDestination destination) {
> StringBuilder sb = textLayout.toText(event, getCachedStringBuilder());
>
> ByteBuffer byteBuf = destination.getByteBuffer();
> CharBuffer charBuf = getCachedCharBuffer();
> charBuf.reset();
> int start = 0;
> int todoChars = sb.length();
> do {
> int copied = copy(sb, start, charBuf);
> start += copied;
> todoChars -= copied;
> boolean endOfInput = todoChars <= 0;
>
> charBuf.flip();
> CodeResult result;
> do {
> result = charsetEncoder.encode(charBuf, byteBuf, endOfInput);
> if (result == CodeResult.OVERFLOW) { // byteBuf full
> destination.drain(byteBuf); // consumes contents and
> clears the byte buffer
> }
> } while (result == CodeResult.OVERFLOW);
> } while (!endOfInput);
> }
>
> /**
> * Copies characters from the StringBuilder into the CharBuffer,
> * starting at the specified offset and ending when either all
> * characters have been copied or when the CharBuffer is full.
> *
> * @return the number of characters that were copied
> */
> int copy(StringBuilder source, int offset, CharBuffer destination) {}
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
