[
https://issues.apache.org/jira/browse/LOG4J2-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15320175#comment-15320175
]
Anthony Maire commented on LOG4J2-1397:
---------------------------------------
Here is my usecase:
- The application is receiving FIX messages (Ascii-based protocol used in
financial services) at a rate from a few thousands to a hundred thousands
messages per second
- The application need to log every message received (mostly for legal and
auditing reasons)
- Latency is critical, so logging need to be asynchronous since the jitter
induced by kernel write back can be very high at this rate
Our (simplified) architecture is something like this :
{code}
public class Connection {
private SocketChannel channel;
private ByteBuffer buffer;
private MessageListener listener;
...
//called when the main loop involving a NIO Selector told us that there
is data to process for this socket
public void onDataReady(){
channel.read(buffer);
if(containsCompleteMessage(buffer)){
listener.onIncomingMessage(this, buffer);
}
}
public String toString(){
return "Connection [remoteAddress=" +
channel.getRemoteAddress() + "]";
}
}
public class MessageListener(){
private static final Logger messageLog =
LoggerFactory.getLogger("message.logger");
private static final Logger log =
LoggerFactory.getLogger(MessageListener.class);
private FIXMessageCodec codec;
...
public void onIncomingMessage(ByteBuffer buffer, Connection connection){
messageLog.info("Message on {}: IN {}", connection, buffer);
FIXMessage businessMessage = codec.decode(buffer);
buffer.clear();
log.debug("start business processing for {}", businessMessage);
doBusinessProcessing(businessMessage);
}
}
{code}
So we have a dedicated logger for these raw messages that we need to store on
disk.
Currently we are using logback through SLF4J, and the first line of
onIncomingMessage is replaced by something like this:
{code}
if(messageLog.isInfoEnabled()){
ByteBuffer copy = ByteBuffer.allocate(buffer.remaining());
copy.put(buffer);
buffer.rewind();
copy.flip();
messageLog.info("Message on {}: IN {}", connection, copy);
}
{code}
And we have some custom layout that will process the content of the ByteBuffer
in the background thread of the AsyncAppender.
Making this copy is not currently an issue since logback is already allocating
roughly 20 times the buffer's size per event, but since we are now looking to
use log4j for it's low allocation rate, we want to avoid that garbage.
> Support ByteBufferLayout
> ------------------------
>
> Key: LOG4J2-1397
> URL: https://issues.apache.org/jira/browse/LOG4J2-1397
> Project: Log4j 2
> Issue Type: New Feature
> Components: Layouts
> Affects Versions: 2.6
> Reporter: Remko Popma
>
> As requested by Kirk Pepperdine on the Mechanical Sympathy [mailing
> list|https://groups.google.com/d/msg/mechanical-sympathy/klefjRqlpQE/-1WF59IGAwAJ]:
> Support a new Layout that takes ObjectMessages containing a ByteBuffer
> payload and write them to the appender without modification.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-dev-unsubscr...@logging.apache.org
For additional commands, e-mail: log4j-dev-h...@logging.apache.org
