[
https://issues.apache.org/jira/browse/SAMZA-479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14271937#comment-14271937
]
Yan Fang commented on SAMZA-479:
--------------------------------
I see.
1. So basically what you mean is that, we want to see the logs like (in json
format)
{code}
{
"logger":"org.apache.samza.system",
"timestamp":"1439976540689",
"level":"WARN",
"containerName":"samza-container-1"
"message":"foo messages here",
"throwable":"java.Exception"
},
{code}
right?
2. If 1 is right,
{quote}
If the OutgoingMessageEnvelope were to send the LoggingEvent, then the Serde
that encodes the LoggingEvent wouldn't be able to easily get access to
this.layout.format().
{quote}
I think in order to get this work, we need to either write new "Serde" for
LoggingEvent or write new layout class.
1) write new "Serde": then we let OutgoingMessageEnvelop accepts the
LoggingEvent and do all the processing in the "Serde" class. But this Serde
class will be totally different from what we have now, because it needs to
process the LoggingEvent and transform that into JSON and then byte[]. It is
true that we will not use any layout class.
2) write new layout class: the new layout.format returns a JSON object (or
other objects). This is not the original layout class anymore. Because original
layout.format() always returns String, while what we want actually is a
different kind of object. So actually the new layout class will not be
compatible with other appenders. We can not use any existing layout classes to
StreamAppender either. So then the subAppend will return a JSON object. Our
current jsonSerde will work in this case.
I am leaning to the approach 2. In fact both approaches get rid of the layout
class. Is this acceptable? When the users are using the String, we still can
use the layout class. There are some exiting [json
layout|https://github.com/michaeltandy/log4j-json/blob/master/src/main/java/uk/me/mjt/log4jjson/SimpleJsonLayout.java],
[avro
layout|https://github.com/devhary/avro-log4j/blob/master/src/main/com/avrolog/log4j/layout/AvroLogLayout.java],
but I think none of them works in our situation.
Any thoughts about this?
> Make StreamAppender pluggable for different log formats
> -------------------------------------------------------
>
> Key: SAMZA-479
> URL: https://issues.apache.org/jira/browse/SAMZA-479
> Project: Samza
> Issue Type: Improvement
> Affects Versions: 0.9.0
> Reporter: Yan Fang
> Assignee: Yan Fang
> Fix For: 0.9.0
>
>
> Currently the StreamAppender only accepts String and sends String as the
> format for all the logs. It will be useful to have StreamAppender to accept
> and send other formats, such as Avro, JSON, etc. So the idea is to move the
> encoding of the LoggingEvent to a serde.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
