[
https://issues.apache.org/jira/browse/SAMZA-479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14271585#comment-14271585
]
Chris Riccomini commented on SAMZA-479:
---------------------------------------
Ahh, no I think Martin and I might not have been quite clear. We don't want to
change the log.info interface. That should remain string. The thing part of the
code that we are talking about is StreamAppender.append:
{code}
protected void append(LoggingEvent event) {
if (!recursiveCall.get()) {
try {
recursiveCall.set(true);
OutgoingMessageEnvelope outgoingMessageEnvelope =
new OutgoingMessageEnvelope(systemStream, key.getBytes("UTF-8"),
subAppend(event).getBytes("UTF-8"));
systemProducer.send(SOURCE, outgoingMessageEnvelope);
} catch (UnsupportedEncodingException e) {
throw new SamzaException("can not send the log messages", e);
} finally {
recursiveCall.set(false);
}
}
}
{code}
This method takes in a full LoggingEvent, and sends the
OutgoingMessageEnvelope's value as a string. If we were to send the value as a
LoggingEvent instead, we'd make it easier for developers to provide other
Serdes for LoggingEvent (e.g. one that uses Avro, Protobuf, JSON, etc), rather
than just forcing a string serde.
One complexity in doing this is that the subAppend method currently refers to
this.layout:
{code}
private String subAppend(LoggingEvent event) {
if (this.layout == null) {
return event.getRenderedMessage();
} else {
return this.layout.format(event).trim();
}
}
{code}
If the OutgoingMessageEnvelope were to send the LoggingEvent, then the Serde
that encodes the LoggingEvent wouldn't be able to easily get access to
this.layout.format(). I'm not quite sure if there's a good way to fix that.
This part is probably the most important piece of the logic, though, as it's
what allows us to set custom log-line formats (to include things like container
name, timestamp, class, etc) in the log4j.xml.
> Make StreamAppender pluggable for different log formats
> -------------------------------------------------------
>
> Key: SAMZA-479
> URL: https://issues.apache.org/jira/browse/SAMZA-479
> Project: Samza
> Issue Type: Improvement
> Affects Versions: 0.9.0
> Reporter: Yan Fang
> Assignee: Yan Fang
> Fix For: 0.9.0
>
>
> Currently the StreamAppender only accepts String and sends String as the
> format for all the logs. It will be useful to have StreamAppender to accept
> and send other formats, such as Avro, JSON, etc. So the idea is to move the
> encoding of the LoggingEvent to a serde.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
