[ 
https://issues.apache.org/jira/browse/LOG4J2-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15148374#comment-15148374
 ] 

Remko Popma commented on LOG4J2-1274:
-------------------------------------

It is currently used in the JMS Appender, so looks like we need it.

> Layout improvements to enable avoiding temporary object allocation
> ------------------------------------------------------------------
>
>                 Key: LOG4J2-1274
>                 URL: https://issues.apache.org/jira/browse/LOG4J2-1274
>             Project: Log4j 2
>          Issue Type: Improvement
>          Components: Layouts
>    Affects Versions: 2.5
>            Reporter: Remko Popma
>
> *Problem*
> The current Layout API does not make it easy for implementors to avoid 
> creating temporary objects. Especially these methods:
> {code}
> byte[] toByteArray(LogEvent);
> T toSerializable(LogEvent);
> {code}
> The byte array returned from {{toByteArray(LogEvent)}} cannot be re-used 
> between log events, since the caller cannot know how many bytes a partially 
> filled array contains.
> In practice, all Layout implementations in Log4j 2 except SerializedLayout 
> implement the {{StringLayout}} subinterface. This means that the 
> {{toSerializable()}} method needs to return a new String object for every log 
> event.
> *Forces*
> I am interested in reducing or even eliminating the allocation of temporary 
> objects for text-based layouts. Many of these use (and re-use) a 
> StringBuilder to build a text representation of the current log event. Once 
> this text representation is built, it needs to be converted to bytes that the 
> Appender can consume. I am aware of two ways in the JDK to convert text to 
> bytes:
> * the various {{String#getBytes}} methods - these all allocate a new byte 
> array for each invocation
> * the underlying {{java.nio.charset.CharsetEncoder}} used internally by 
> String - especially method {{CoderResult encode(CharBuffer in, ByteBuffer 
> out, boolean endOfInput)}} which converts characters to bytes without object 
> allocation.
> The last method is interesting because this gives us an opportunity to also 
> reduce the amount of copying by directly supplying the ByteBuffer buffer used 
> by RandomAccessFileAppender, or the MappedByteBuffer of the 
> MemoryMappedFileAppender.
> The resulting API needs to support the fact that implementations may need to 
> call {{CharsetEncoder#encode}} multiple times:
> * The ByteBuffer may not have enough remaining space to hold all the data; 
> {{CharsetEncoder#encode}} returns {{CoderResult.OVERFLOW}} to signal this so 
> the caller can consume the contents and reset/clear the buffer before 
> continuing.
> * The CharBuffer may not be large enough to hold the full text representation 
> of the log event. Again, {{CharsetEncoder#encode}} may need to be invoked 
> multiple times.
> *Proposal*
> (Thinking out loud here, I'm open to suggestions.)
> It may be sufficient for the layout interface to have a single additional new 
> method:
> {code}
> /**
>  * Formats the event suitable for display and writes the result to the 
> specified destination.
>  *
>  * @param event The Logging Event.
>  * @param destination Holds the ByteBuffer to write into.
>  */
> void writeTo(LogEvent e, ByteBufferDestination destination);
> {code}
> Appenders that want to be allocation-free need to implement the 
> {{ByteBufferDestination}} interface:
> {code}
> public interface ByteBufferDestination {
>     ByteBuffer getByteBuffer();
>     /**
>      * Consumes the buffer content and returns a buffer with more available() 
> space
>      * (which may or may not be the same instance).
>      * <p>
>      * Called by the producer when the buffer becomes too full
>      * to write more data into it.
>      */
>     ByteBuffer drain(ByteBuffer buf);
> }
> {code}
> Usage: for example RandomAccessFileAppender code can look like this:
> {code}
> // RandomAccessFileAppender
> public void append(final LogEvent event) {
>     getLayout().writeTo(event, (ByteBufferDestination) manager);
> }
> {code}
> Layout implementation of the {{writeTo}} method: layouts need to know how to 
> convert LogEvents to text, but writing this text into the ByteBuffer can be 
> delegated to a helper:
> {code}
> // some layout 
> public void writeTo(LogEvent event, ByteBufferDestination destination) {
>     StringBuilder text = toText(event, getCachedStringBuilder());
>       
>     TextEncoderHelper helper = getCachedHelper();
>     helper.encodeWithoutAllocation(text, destination);
> }
> /**
>  * Creates a text representation of the specified log event
>  * and writes it into the specified StringBuilder.
>  * <p>
>  * Implementations are free to return a new StringBuilder if they can
>  * detect in advance that the specified StringBuilder is too small.
>  */
> StringBuilder toText(LogEvent e, StringBuilder destination) {} // existing 
> logic goes here
> public String toSerializable(LogEvent event) { // factored out logic to 
> toText()
>     return toText(event, getCachedStringBuilder()).toString();
> }
> {code}
> Helper contains utility code for moving the text into a CharBuffer, and for 
> repeatedly calling {{CharsetEncoder#encode}}. 
> {code}
> public class TextEncoderHelper {
>     TextEncoderHelper(Charset charset) {} // create CharsetEncoder
>     void encodeWithoutAllocation(StringBuilder text, BinaryDestination 
> destination) {
>         ByteBuffer byteBuf = destination.getByteBuffer();
>         CharBuffer charBuf = getCachedCharBuffer();
>         charBuf.reset();
>         int start = 0;
>         int todoChars = text.length();
>         do {
>             int copied = copy(text, start, charBuf);
>             start += copied;
>             todoChars -= copied;
>             boolean endOfInput = todoChars <= 0;
>     
>             charBuf.flip();
>             CodeResult result;
>             do {
>                 result = charsetEncoder.encode(charBuf, byteBuf, endOfInput);
>                 if (result == CodeResult.OVERFLOW) { // byteBuf full
>                     // destination consumes contents
>                     // and returns byte buffer with more capacity
>                     byteBuf = destination.drain(byteBuf);
>                 }
>             } while (result == CodeResult.OVERFLOW);
>         } while (!endOfInput);
>     }
>     
>     /**
>      * Copies characters from the StringBuilder into the CharBuffer,
>      * starting at the specified offset and ending when either all
>      * characters have been copied or when the CharBuffer is full.
>      *
>      * @return the number of characters that were copied
>      */
>     int copy(StringBuilder source, int offset, CharBuffer destination) {}
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org
For additional commands, e-mail: log4j-dev-h...@logging.apache.org

Reply via email to