Re: How to configure ObjectMapper of Log4j2 JSON Layout

[Dominik Sandjaja]

Argh, good point:
https://issues.apache.org/jira/browse/LOG4J2-2670

On 30.07.19, 20:30, "Ralph Goers"  wrote:

Did you create a Jira issue for this? One of us might get around to it but 
we never will without the Jira issue reminding us.

Re: How to configure ObjectMapper of Log4j2 JSON Layout

[Ralph Goers]

Did you create a Jira issue for this? One of us might get around to it but we 
never will without the Jira issue reminding us.

Ralph

> On Jul 16, 2019, at 8:12 AM, Dominik Sandjaja  
> wrote:
> 
> Hello everybody,
> 
> thank you all for the discussion about this topic.
> 
> My initial thought was that I was maybe missing some configuration option for 
> setting the classname of one of the initialization classes to override the 
> object creation.
> But after looking a bit into the Log4J Plugin system I agree, that this is 
> the cleanest way forward from an overall architectural point of view.
> 
> I don't think that I will be able to provide a Pull Request within the next 
> few weeks (hooray for holidays!), but if time permits, I could have a shot at 
> it in September.
> 
> Until then, we will most likely remove this one exception from the Exception 
> chain.
> 
> Thank you all!
> 
> On 13.07.19, 22:34, "Matt Sicker"  wrote:
> 
>Oh, that's a part of the plugin system I often forget about since most
>plugins are core plugins. In any case, a plugin of some sort would be
>the most flexible here I think.
> 
>On Sat, 13 Jul 2019 at 15:21, Ralph Goers  
> wrote:
>> 
>> Just because it is a plugin doesn’t mean it can be used anywhere. The 
>> JacksonFactory could collect plugins for its own category, which would mean 
>> they couldn’t be specified in the configuration. Only “core” plugins can 
>> directly be used as parameters in the configuration. But yes, once it is a 
>> core plugin it could be passed into the newWriter method as I describe.
>> 
>> Ralph
>> 
>>> On Jul 13, 2019, at 10:12 AM, Matt Sicker  wrote:
>>> 
>>> Being a plugin means you can inject it as a @PluginElement wherever. Plus,
>>> that makes it easier for users to write a custom plugin class to configure
>>> any other options they want. Comparable to the DataSourceFactory or
>>> whatever it’s called for JDBC drivers in that plugin.
>>> 
>>> Keeping it as an element should make it more flexible with future
>>> programmatic configuration ideas. The more component based everything is,
>>> the simpler it’ll be to make customizations without users digging into the
>>> internals of LoggerConfig et al.
>>> 
>>> On Fri, Jul 12, 2019 at 18:36, Ralph Goers 
>>> wrote:
>>> 
 Actually, I was thinking that we can keep the existing ObjectMapper but
 supply a Function that can “enhance” the ObjectMapper either at the end of
 the Log4jJsonObjectMapper constructor, at the end of
 JacksonFactory.newObjectMapper, or after the ObjectMapper is constructed in
 JacksonFactory.newWriter. Whether that is done via a Plugin or some other
 way I hadn’t looked at too much. But I suppose it could either be a Plugin
 that is a parameter to the Layout and then passed into the newWriter method
 or it could be looked up by the JSON method in JacksonFactory. Passing it
 on the Layout probably makes it more flexible.
 
 Ralph
 
> On Jul 12, 2019, at 4:13 PM, Carter Kozak  wrote:
> 
> I like the idea of an ObjectMapperFactory plugin point. That gives us a
 cleaner builder API on JsonLayout where we can potentially pass in a
 supplier for an ObjectMapper instance from programmatic configuration.
> 
> On Fri, Jul 12, 2019, at 19:10, Matt Sicker wrote:
>> And by plugin, see for example the various BlockingQueueFactory plugins.
>> 
>> On Fri, Jul 12, 2019 at 18:09, Matt Sicker  wrote:
>> 
>>> Plugin maybe? I think we could potentially extend the plugin system as
 a
>>> general dependency and configuration injection system. We can make less
>>> special case methods of pseudo components and options outside that.
>>> 
>>> On Fri, Jul 12, 2019 at 17:34, Gary Gregory 
>>> wrote:
>>> 
 Hi Dominik
 
 I see two ways of addressing this:
 1) Add a setObjectMapper(ObjectMapper)
 to org.apache.logging.log4j.core.layout.JsonLayout.Builder
 This would work well for programmatic configurations
 and/or:
 2) Add a setObjectMapperFactory(String)
 to org.apache.logging.log4j.core.layout.JsonLayout.Builder
 This would add a class name for implementors of a new interface
 ObjectMapperFactory.
 This would work nicely for configuration files.
 
 To keep all users in play, 2) seems best.
 
 Thoughts?
 
 Gary
 
 
 On Fri, Jul 12, 2019 at 8:38 AM Dominik Sandjaja <
 dominik.sandj...@trivago.com> wrote:
 
> Hello,
> 
> we are using the JSON layout in our Spring Boot application to easily
> ingest the logs in Elasticsearch.
> 
> One message that we log is about failed conversions, when an incoming
> request parameter cannot be converted correctly. As we use Spring’s
> Converter functionality for this, these 

Re: How to configure ObjectMapper of Log4j2 JSON Layout

[Dominik Sandjaja]

Hello everybody,

thank you all for the discussion about this topic.

My initial thought was that I was maybe missing some configuration option for 
setting the classname of one of the initialization classes to override the 
object creation.
But after looking a bit into the Log4J Plugin system I agree, that this is the 
cleanest way forward from an overall architectural point of view.

I don't think that I will be able to provide a Pull Request within the next few 
weeks (hooray for holidays!), but if time permits, I could have a shot at it in 
September.

Until then, we will most likely remove this one exception from the Exception 
chain.

Thank you all!

On 13.07.19, 22:34, "Matt Sicker"  wrote:

Oh, that's a part of the plugin system I often forget about since most
plugins are core plugins. In any case, a plugin of some sort would be
the most flexible here I think.

On Sat, 13 Jul 2019 at 15:21, Ralph Goers  
wrote:
>
> Just because it is a plugin doesn’t mean it can be used anywhere. The 
JacksonFactory could collect plugins for its own category, which would mean 
they couldn’t be specified in the configuration. Only “core” plugins can 
directly be used as parameters in the configuration. But yes, once it is a core 
plugin it could be passed into the newWriter method as I describe.
>
> Ralph
>
> > On Jul 13, 2019, at 10:12 AM, Matt Sicker  wrote:
> >
> > Being a plugin means you can inject it as a @PluginElement wherever. 
Plus,
> > that makes it easier for users to write a custom plugin class to 
configure
> > any other options they want. Comparable to the DataSourceFactory or
> > whatever it’s called for JDBC drivers in that plugin.
> >
> > Keeping it as an element should make it more flexible with future
> > programmatic configuration ideas. The more component based everything 
is,
> > the simpler it’ll be to make customizations without users digging into 
the
> > internals of LoggerConfig et al.
> >
> > On Fri, Jul 12, 2019 at 18:36, Ralph Goers 
> > wrote:
> >
> >> Actually, I was thinking that we can keep the existing ObjectMapper but
> >> supply a Function that can “enhance” the ObjectMapper either at the 
end of
> >> the Log4jJsonObjectMapper constructor, at the end of
> >> JacksonFactory.newObjectMapper, or after the ObjectMapper is 
constructed in
> >> JacksonFactory.newWriter. Whether that is done via a Plugin or some 
other
> >> way I hadn’t looked at too much. But I suppose it could either be a 
Plugin
> >> that is a parameter to the Layout and then passed into the newWriter 
method
> >> or it could be looked up by the JSON method in JacksonFactory. Passing 
it
> >> on the Layout probably makes it more flexible.
> >>
> >> Ralph
> >>
> >>> On Jul 12, 2019, at 4:13 PM, Carter Kozak  wrote:
> >>>
> >>> I like the idea of an ObjectMapperFactory plugin point. That gives us 
a
> >> cleaner builder API on JsonLayout where we can potentially pass in a
> >> supplier for an ObjectMapper instance from programmatic configuration.
> >>>
> >>> On Fri, Jul 12, 2019, at 19:10, Matt Sicker wrote:
>  And by plugin, see for example the various BlockingQueueFactory 
plugins.
> 
>  On Fri, Jul 12, 2019 at 18:09, Matt Sicker  wrote:
> 
> > Plugin maybe? I think we could potentially extend the plugin system 
as
> >> a
> > general dependency and configuration injection system. We can make 
less
> > special case methods of pseudo components and options outside that.
> >
> > On Fri, Jul 12, 2019 at 17:34, Gary Gregory 
> > wrote:
> >
> >> Hi Dominik
> >>
> >> I see two ways of addressing this:
> >> 1) Add a setObjectMapper(ObjectMapper)
> >> to org.apache.logging.log4j.core.layout.JsonLayout.Builder
> >> This would work well for programmatic configurations
> >> and/or:
> >> 2) Add a setObjectMapperFactory(String)
> >> to org.apache.logging.log4j.core.layout.JsonLayout.Builder
> >> This would add a class name for implementors of a new interface
> >> ObjectMapperFactory.
> >> This would work nicely for configuration files.
> >>
> >> To keep all users in play, 2) seems best.
> >>
> >> Thoughts?
> >>
> >> Gary
> >>
> >>
> >> On Fri, Jul 12, 2019 at 8:38 AM Dominik Sandjaja <
> >> dominik.sandj...@trivago.com> wrote:
> >>
> >>> Hello,
> >>>
> >>> we are using the JSON layout in our Spring Boot application to 
easily
> >>> ingest the logs in Elasticsearch.
> >>>
> >>> One message that we log is about failed conversions, when an 
incoming
> >>> request parameter cannot be converted correctly. As we use 
Spring’s
> 

Re: How to configure ObjectMapper of Log4j2 JSON Layout

[Matt Sicker]

Oh, that's a part of the plugin system I often forget about since most
plugins are core plugins. In any case, a plugin of some sort would be
the most flexible here I think.

On Sat, 13 Jul 2019 at 15:21, Ralph Goers  wrote:
>
> Just because it is a plugin doesn’t mean it can be used anywhere. The 
> JacksonFactory could collect plugins for its own category, which would mean 
> they couldn’t be specified in the configuration. Only “core” plugins can 
> directly be used as parameters in the configuration. But yes, once it is a 
> core plugin it could be passed into the newWriter method as I describe.
>
> Ralph
>
> > On Jul 13, 2019, at 10:12 AM, Matt Sicker  wrote:
> >
> > Being a plugin means you can inject it as a @PluginElement wherever. Plus,
> > that makes it easier for users to write a custom plugin class to configure
> > any other options they want. Comparable to the DataSourceFactory or
> > whatever it’s called for JDBC drivers in that plugin.
> >
> > Keeping it as an element should make it more flexible with future
> > programmatic configuration ideas. The more component based everything is,
> > the simpler it’ll be to make customizations without users digging into the
> > internals of LoggerConfig et al.
> >
> > On Fri, Jul 12, 2019 at 18:36, Ralph Goers 
> > wrote:
> >
> >> Actually, I was thinking that we can keep the existing ObjectMapper but
> >> supply a Function that can “enhance” the ObjectMapper either at the end of
> >> the Log4jJsonObjectMapper constructor, at the end of
> >> JacksonFactory.newObjectMapper, or after the ObjectMapper is constructed in
> >> JacksonFactory.newWriter. Whether that is done via a Plugin or some other
> >> way I hadn’t looked at too much. But I suppose it could either be a Plugin
> >> that is a parameter to the Layout and then passed into the newWriter method
> >> or it could be looked up by the JSON method in JacksonFactory. Passing it
> >> on the Layout probably makes it more flexible.
> >>
> >> Ralph
> >>
> >>> On Jul 12, 2019, at 4:13 PM, Carter Kozak  wrote:
> >>>
> >>> I like the idea of an ObjectMapperFactory plugin point. That gives us a
> >> cleaner builder API on JsonLayout where we can potentially pass in a
> >> supplier for an ObjectMapper instance from programmatic configuration.
> >>>
> >>> On Fri, Jul 12, 2019, at 19:10, Matt Sicker wrote:
>  And by plugin, see for example the various BlockingQueueFactory plugins.
> 
>  On Fri, Jul 12, 2019 at 18:09, Matt Sicker  wrote:
> 
> > Plugin maybe? I think we could potentially extend the plugin system as
> >> a
> > general dependency and configuration injection system. We can make less
> > special case methods of pseudo components and options outside that.
> >
> > On Fri, Jul 12, 2019 at 17:34, Gary Gregory 
> > wrote:
> >
> >> Hi Dominik
> >>
> >> I see two ways of addressing this:
> >> 1) Add a setObjectMapper(ObjectMapper)
> >> to org.apache.logging.log4j.core.layout.JsonLayout.Builder
> >> This would work well for programmatic configurations
> >> and/or:
> >> 2) Add a setObjectMapperFactory(String)
> >> to org.apache.logging.log4j.core.layout.JsonLayout.Builder
> >> This would add a class name for implementors of a new interface
> >> ObjectMapperFactory.
> >> This would work nicely for configuration files.
> >>
> >> To keep all users in play, 2) seems best.
> >>
> >> Thoughts?
> >>
> >> Gary
> >>
> >>
> >> On Fri, Jul 12, 2019 at 8:38 AM Dominik Sandjaja <
> >> dominik.sandj...@trivago.com> wrote:
> >>
> >>> Hello,
> >>>
> >>> we are using the JSON layout in our Spring Boot application to easily
> >>> ingest the logs in Elasticsearch.
> >>>
> >>> One message that we log is about failed conversions, when an incoming
> >>> request parameter cannot be converted correctly. As we use Spring’s
> >>> Converter functionality for this, these `IllegalArgumentException`s
> >> are
> >>> wrapped in a `ConversionFailedException`. This exception has fields
> >> of
> >> type
> >>> `TypeDescriptor` which hold information about the source and the
> >> target
> >>> type of the conversion.
> >>>
> >>> The problem is, that these `TypeDescriptor` objects cannot be
> >> serialized
> >>> to JSON. The simple call
> >>>
> >>>
> >>>
> >>
> >> com.fasterxml.jackson.databind.ObjectMapper().writeValueAsString(org.springframework.core.convert.TypeDescriptor.valueOf(String.javaClass))
> >>>
> >>> throws an exception like
> >>>
> >>> ERROR StatusLogger
> >>> com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Direct
> >>> self-reference leading to cycle (through reference chain:
> >>>
> >>
> >> org.springframework.core.ResolvableType["componentType"]->org.springframework.core.ResolvableType["componentType"])
> >>> com.fasterxml.jackson.databind.exc.InvalidDefinitionException: 

Re: How to configure ObjectMapper of Log4j2 JSON Layout

[Ralph Goers]

Just because it is a plugin doesn’t mean it can be used anywhere. The 
JacksonFactory could collect plugins for its own category, which would mean 
they couldn’t be specified in the configuration. Only “core” plugins can 
directly be used as parameters in the configuration. But yes, once it is a core 
plugin it could be passed into the newWriter method as I describe.

Ralph

> On Jul 13, 2019, at 10:12 AM, Matt Sicker  wrote:
> 
> Being a plugin means you can inject it as a @PluginElement wherever. Plus,
> that makes it easier for users to write a custom plugin class to configure
> any other options they want. Comparable to the DataSourceFactory or
> whatever it’s called for JDBC drivers in that plugin.
> 
> Keeping it as an element should make it more flexible with future
> programmatic configuration ideas. The more component based everything is,
> the simpler it’ll be to make customizations without users digging into the
> internals of LoggerConfig et al.
> 
> On Fri, Jul 12, 2019 at 18:36, Ralph Goers 
> wrote:
> 
>> Actually, I was thinking that we can keep the existing ObjectMapper but
>> supply a Function that can “enhance” the ObjectMapper either at the end of
>> the Log4jJsonObjectMapper constructor, at the end of
>> JacksonFactory.newObjectMapper, or after the ObjectMapper is constructed in
>> JacksonFactory.newWriter. Whether that is done via a Plugin or some other
>> way I hadn’t looked at too much. But I suppose it could either be a Plugin
>> that is a parameter to the Layout and then passed into the newWriter method
>> or it could be looked up by the JSON method in JacksonFactory. Passing it
>> on the Layout probably makes it more flexible.
>> 
>> Ralph
>> 
>>> On Jul 12, 2019, at 4:13 PM, Carter Kozak  wrote:
>>> 
>>> I like the idea of an ObjectMapperFactory plugin point. That gives us a
>> cleaner builder API on JsonLayout where we can potentially pass in a
>> supplier for an ObjectMapper instance from programmatic configuration.
>>> 
>>> On Fri, Jul 12, 2019, at 19:10, Matt Sicker wrote:
 And by plugin, see for example the various BlockingQueueFactory plugins.
 
 On Fri, Jul 12, 2019 at 18:09, Matt Sicker  wrote:
 
> Plugin maybe? I think we could potentially extend the plugin system as
>> a
> general dependency and configuration injection system. We can make less
> special case methods of pseudo components and options outside that.
> 
> On Fri, Jul 12, 2019 at 17:34, Gary Gregory 
> wrote:
> 
>> Hi Dominik
>> 
>> I see two ways of addressing this:
>> 1) Add a setObjectMapper(ObjectMapper)
>> to org.apache.logging.log4j.core.layout.JsonLayout.Builder
>> This would work well for programmatic configurations
>> and/or:
>> 2) Add a setObjectMapperFactory(String)
>> to org.apache.logging.log4j.core.layout.JsonLayout.Builder
>> This would add a class name for implementors of a new interface
>> ObjectMapperFactory.
>> This would work nicely for configuration files.
>> 
>> To keep all users in play, 2) seems best.
>> 
>> Thoughts?
>> 
>> Gary
>> 
>> 
>> On Fri, Jul 12, 2019 at 8:38 AM Dominik Sandjaja <
>> dominik.sandj...@trivago.com> wrote:
>> 
>>> Hello,
>>> 
>>> we are using the JSON layout in our Spring Boot application to easily
>>> ingest the logs in Elasticsearch.
>>> 
>>> One message that we log is about failed conversions, when an incoming
>>> request parameter cannot be converted correctly. As we use Spring’s
>>> Converter functionality for this, these `IllegalArgumentException`s
>> are
>>> wrapped in a `ConversionFailedException`. This exception has fields
>> of
>> type
>>> `TypeDescriptor` which hold information about the source and the
>> target
>>> type of the conversion.
>>> 
>>> The problem is, that these `TypeDescriptor` objects cannot be
>> serialized
>>> to JSON. The simple call
>>> 
>>> 
>>> 
>> 
>> com.fasterxml.jackson.databind.ObjectMapper().writeValueAsString(org.springframework.core.convert.TypeDescriptor.valueOf(String.javaClass))
>>> 
>>> throws an exception like
>>> 
>>> ERROR StatusLogger
>>> com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Direct
>>> self-reference leading to cycle (through reference chain:
>>> 
>> 
>> org.springframework.core.ResolvableType["componentType"]->org.springframework.core.ResolvableType["componentType"])
>>> com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Direct
>>> self-reference leading to cycle (through reference chain:
>>> 
>> 
>> org.springframework.core.ResolvableType["componentType"]->org.springframework.core.ResolvableType["componentType"])
>>> at
>>> 
>> 
>> com.fasterxml.jackson.databind.exc.InvalidDefinitionException.from(InvalidDefinitionException.java:77)
>>> at
>>> 
>> 
>> 

Re: How to configure ObjectMapper of Log4j2 JSON Layout

[Ralph Goers]

Actually, I was thinking that we can keep the existing ObjectMapper but supply 
a Function that can “enhance” the ObjectMapper either at the end of the 
Log4jJsonObjectMapper constructor, at the end of 
JacksonFactory.newObjectMapper, or after the ObjectMapper is constructed in 
JacksonFactory.newWriter. Whether that is done via a Plugin or some other way I 
hadn’t looked at too much. But I suppose it could either be a Plugin that is a 
parameter to the Layout and then passed into the newWriter method or it could 
be looked up by the JSON method in JacksonFactory. Passing it on the Layout 
probably makes it more flexible.

Ralph

> On Jul 12, 2019, at 4:13 PM, Carter Kozak  wrote:
> 
> I like the idea of an ObjectMapperFactory plugin point. That gives us a 
> cleaner builder API on JsonLayout where we can potentially pass in a supplier 
> for an ObjectMapper instance from programmatic configuration.
> 
> On Fri, Jul 12, 2019, at 19:10, Matt Sicker wrote:
>> And by plugin, see for example the various BlockingQueueFactory plugins.
>> 
>> On Fri, Jul 12, 2019 at 18:09, Matt Sicker  wrote:
>> 
>>> Plugin maybe? I think we could potentially extend the plugin system as a
>>> general dependency and configuration injection system. We can make less
>>> special case methods of pseudo components and options outside that.
>>> 
>>> On Fri, Jul 12, 2019 at 17:34, Gary Gregory 
>>> wrote:
>>> 
 Hi Dominik
 
 I see two ways of addressing this:
 1) Add a setObjectMapper(ObjectMapper)
 to org.apache.logging.log4j.core.layout.JsonLayout.Builder
 This would work well for programmatic configurations
 and/or:
 2) Add a setObjectMapperFactory(String)
 to org.apache.logging.log4j.core.layout.JsonLayout.Builder
 This would add a class name for implementors of a new interface
 ObjectMapperFactory.
 This would work nicely for configuration files.
 
 To keep all users in play, 2) seems best.
 
 Thoughts?
 
 Gary
 
 
 On Fri, Jul 12, 2019 at 8:38 AM Dominik Sandjaja <
 dominik.sandj...@trivago.com> wrote:
 
> Hello,
> 
> we are using the JSON layout in our Spring Boot application to easily
> ingest the logs in Elasticsearch.
> 
> One message that we log is about failed conversions, when an incoming
> request parameter cannot be converted correctly. As we use Spring’s
> Converter functionality for this, these `IllegalArgumentException`s are
> wrapped in a `ConversionFailedException`. This exception has fields of
 type
> `TypeDescriptor` which hold information about the source and the target
> type of the conversion.
> 
> The problem is, that these `TypeDescriptor` objects cannot be serialized
> to JSON. The simple call
> 
> 
> 
 com.fasterxml.jackson.databind.ObjectMapper().writeValueAsString(org.springframework.core.convert.TypeDescriptor.valueOf(String.javaClass))
> 
> throws an exception like
> 
> ERROR StatusLogger
> com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Direct
> self-reference leading to cycle (through reference chain:
> 
 org.springframework.core.ResolvableType["componentType"]->org.springframework.core.ResolvableType["componentType"])
> com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Direct
> self-reference leading to cycle (through reference chain:
> 
 org.springframework.core.ResolvableType["componentType"]->org.springframework.core.ResolvableType["componentType"])
> at
> 
 com.fasterxml.jackson.databind.exc.InvalidDefinitionException.from(InvalidDefinitionException.java:77)
> at
> 
 com.fasterxml.jackson.databind.SerializerProvider.reportBadDefinition(SerializerProvider.java:1191)
> at
> 
 com.fasterxml.jackson.databind.ser.BeanPropertyWriter._handleSelfReference(BeanPropertyWriter.java:944)
> at
> 
 com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:721)
> at
> 
 com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
> at
> 
 com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
> 
> The error can be prevented if `Mixin`s are added to the `ObjectMapper`
> which ignore the corresponding class.
> 
> The `Log4jJsonObjectMapper` is initialized in the `JacksonFactory.JSON`
> class and I currently do not see any way to somehow configure this
> ObjectMapper with additional MixIns.
> 
> Hence, my question finally is:
> How can I configure Log4J’s JSON mapping to not break on such
> non-serializable log content?
> 
> Thank you!
> Dominik
> 
 
>>> --
>>> Matt Sicker 
>>> 
>> -- 
>> Matt Sicker 
>> 



-
To unsubscribe, e-mail: 

Re: How to configure ObjectMapper of Log4j2 JSON Layout

[Carter Kozak]

I like the idea of an ObjectMapperFactory plugin point. That gives us a cleaner 
builder API on JsonLayout where we can potentially pass in a supplier for an 
ObjectMapper instance from programmatic configuration.

On Fri, Jul 12, 2019, at 19:10, Matt Sicker wrote:
> And by plugin, see for example the various BlockingQueueFactory plugins.
> 
> On Fri, Jul 12, 2019 at 18:09, Matt Sicker  wrote:
> 
> > Plugin maybe? I think we could potentially extend the plugin system as a
> > general dependency and configuration injection system. We can make less
> > special case methods of pseudo components and options outside that.
> >
> > On Fri, Jul 12, 2019 at 17:34, Gary Gregory 
> > wrote:
> >
> >> Hi Dominik
> >>
> >> I see two ways of addressing this:
> >> 1) Add a setObjectMapper(ObjectMapper)
> >> to org.apache.logging.log4j.core.layout.JsonLayout.Builder
> >> This would work well for programmatic configurations
> >> and/or:
> >> 2) Add a setObjectMapperFactory(String)
> >> to org.apache.logging.log4j.core.layout.JsonLayout.Builder
> >> This would add a class name for implementors of a new interface
> >> ObjectMapperFactory.
> >> This would work nicely for configuration files.
> >>
> >> To keep all users in play, 2) seems best.
> >>
> >> Thoughts?
> >>
> >> Gary
> >>
> >>
> >> On Fri, Jul 12, 2019 at 8:38 AM Dominik Sandjaja <
> >> dominik.sandj...@trivago.com> wrote:
> >>
> >> > Hello,
> >> >
> >> > we are using the JSON layout in our Spring Boot application to easily
> >> > ingest the logs in Elasticsearch.
> >> >
> >> > One message that we log is about failed conversions, when an incoming
> >> > request parameter cannot be converted correctly. As we use Spring’s
> >> > Converter functionality for this, these `IllegalArgumentException`s are
> >> > wrapped in a `ConversionFailedException`. This exception has fields of
> >> type
> >> > `TypeDescriptor` which hold information about the source and the target
> >> > type of the conversion.
> >> >
> >> > The problem is, that these `TypeDescriptor` objects cannot be serialized
> >> > to JSON. The simple call
> >> >
> >> >
> >> >
> >> com.fasterxml.jackson.databind.ObjectMapper().writeValueAsString(org.springframework.core.convert.TypeDescriptor.valueOf(String.javaClass))
> >> >
> >> > throws an exception like
> >> >
> >> > ERROR StatusLogger
> >> > com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Direct
> >> > self-reference leading to cycle (through reference chain:
> >> >
> >> org.springframework.core.ResolvableType["componentType"]->org.springframework.core.ResolvableType["componentType"])
> >> > com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Direct
> >> > self-reference leading to cycle (through reference chain:
> >> >
> >> org.springframework.core.ResolvableType["componentType"]->org.springframework.core.ResolvableType["componentType"])
> >> > at
> >> >
> >> com.fasterxml.jackson.databind.exc.InvalidDefinitionException.from(InvalidDefinitionException.java:77)
> >> > at
> >> >
> >> com.fasterxml.jackson.databind.SerializerProvider.reportBadDefinition(SerializerProvider.java:1191)
> >> > at
> >> >
> >> com.fasterxml.jackson.databind.ser.BeanPropertyWriter._handleSelfReference(BeanPropertyWriter.java:944)
> >> > at
> >> >
> >> com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:721)
> >> > at
> >> >
> >> com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
> >> > at
> >> >
> >> com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
> >> >
> >> > The error can be prevented if `Mixin`s are added to the `ObjectMapper`
> >> > which ignore the corresponding class.
> >> >
> >> > The `Log4jJsonObjectMapper` is initialized in the `JacksonFactory.JSON`
> >> > class and I currently do not see any way to somehow configure this
> >> > ObjectMapper with additional MixIns.
> >> >
> >> > Hence, my question finally is:
> >> > How can I configure Log4J’s JSON mapping to not break on such
> >> > non-serializable log content?
> >> >
> >> > Thank you!
> >> > Dominik
> >> >
> >>
> > --
> > Matt Sicker 
> >
> -- 
> Matt Sicker 
> 

Re: How to configure ObjectMapper of Log4j2 JSON Layout

[Matt Sicker]

And by plugin, see for example the various BlockingQueueFactory plugins.

On Fri, Jul 12, 2019 at 18:09, Matt Sicker  wrote:

> Plugin maybe? I think we could potentially extend the plugin system as a
> general dependency and configuration injection system. We can make less
> special case methods of pseudo components and options outside that.
>
> On Fri, Jul 12, 2019 at 17:34, Gary Gregory 
> wrote:
>
>> Hi Dominik
>>
>> I see two ways of addressing this:
>> 1) Add a setObjectMapper(ObjectMapper)
>> to org.apache.logging.log4j.core.layout.JsonLayout.Builder
>>   This would work well for programmatic configurations
>> and/or:
>> 2) Add a setObjectMapperFactory(String)
>> to org.apache.logging.log4j.core.layout.JsonLayout.Builder
>>   This would add a class name for implementors of a new interface
>> ObjectMapperFactory.
>>   This would work nicely for configuration files.
>>
>> To keep all users in play, 2) seems best.
>>
>> Thoughts?
>>
>> Gary
>>
>>
>> On Fri, Jul 12, 2019 at 8:38 AM Dominik Sandjaja <
>> dominik.sandj...@trivago.com> wrote:
>>
>> > Hello,
>> >
>> > we are using the JSON layout in our Spring Boot application to easily
>> > ingest the logs in Elasticsearch.
>> >
>> > One message that we log is about failed conversions, when an incoming
>> > request parameter cannot be converted correctly. As we use Spring’s
>> > Converter functionality for this, these `IllegalArgumentException`s are
>> > wrapped in a `ConversionFailedException`. This exception has fields of
>> type
>> > `TypeDescriptor` which hold information about the source and the target
>> > type of the conversion.
>> >
>> > The problem is, that these `TypeDescriptor` objects cannot be serialized
>> > to JSON. The simple call
>> >
>> >
>> >
>> com.fasterxml.jackson.databind.ObjectMapper().writeValueAsString(org.springframework.core.convert.TypeDescriptor.valueOf(String.javaClass))
>> >
>> > throws an exception like
>> >
>> > ERROR StatusLogger
>> > com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Direct
>> > self-reference leading to cycle (through reference chain:
>> >
>> org.springframework.core.ResolvableType["componentType"]->org.springframework.core.ResolvableType["componentType"])
>> > com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Direct
>> > self-reference leading to cycle (through reference chain:
>> >
>> org.springframework.core.ResolvableType["componentType"]->org.springframework.core.ResolvableType["componentType"])
>> > at
>> >
>> com.fasterxml.jackson.databind.exc.InvalidDefinitionException.from(InvalidDefinitionException.java:77)
>> > at
>> >
>> com.fasterxml.jackson.databind.SerializerProvider.reportBadDefinition(SerializerProvider.java:1191)
>> > at
>> >
>> com.fasterxml.jackson.databind.ser.BeanPropertyWriter._handleSelfReference(BeanPropertyWriter.java:944)
>> > at
>> >
>> com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:721)
>> > at
>> >
>> com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
>> > at
>> >
>> com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
>> >
>> > The error can be prevented if `Mixin`s are added to the `ObjectMapper`
>> > which ignore the corresponding class.
>> >
>> > The `Log4jJsonObjectMapper` is initialized in the `JacksonFactory.JSON`
>> > class and I currently do not see any way to somehow configure this
>> > ObjectMapper with additional MixIns.
>> >
>> > Hence, my question finally is:
>> > How can I configure Log4J’s JSON mapping to not break on such
>> > non-serializable log content?
>> >
>> > Thank you!
>> > Dominik
>> >
>>
> --
> Matt Sicker 
>
-- 
Matt Sicker 

Re: How to configure ObjectMapper of Log4j2 JSON Layout

[Matt Sicker]

Plugin maybe? I think we could potentially extend the plugin system as a
general dependency and configuration injection system. We can make less
special case methods of pseudo components and options outside that.

On Fri, Jul 12, 2019 at 17:34, Gary Gregory  wrote:

> Hi Dominik
>
> I see two ways of addressing this:
> 1) Add a setObjectMapper(ObjectMapper)
> to org.apache.logging.log4j.core.layout.JsonLayout.Builder
>   This would work well for programmatic configurations
> and/or:
> 2) Add a setObjectMapperFactory(String)
> to org.apache.logging.log4j.core.layout.JsonLayout.Builder
>   This would add a class name for implementors of a new interface
> ObjectMapperFactory.
>   This would work nicely for configuration files.
>
> To keep all users in play, 2) seems best.
>
> Thoughts?
>
> Gary
>
>
> On Fri, Jul 12, 2019 at 8:38 AM Dominik Sandjaja <
> dominik.sandj...@trivago.com> wrote:
>
> > Hello,
> >
> > we are using the JSON layout in our Spring Boot application to easily
> > ingest the logs in Elasticsearch.
> >
> > One message that we log is about failed conversions, when an incoming
> > request parameter cannot be converted correctly. As we use Spring’s
> > Converter functionality for this, these `IllegalArgumentException`s are
> > wrapped in a `ConversionFailedException`. This exception has fields of
> type
> > `TypeDescriptor` which hold information about the source and the target
> > type of the conversion.
> >
> > The problem is, that these `TypeDescriptor` objects cannot be serialized
> > to JSON. The simple call
> >
> >
> >
> com.fasterxml.jackson.databind.ObjectMapper().writeValueAsString(org.springframework.core.convert.TypeDescriptor.valueOf(String.javaClass))
> >
> > throws an exception like
> >
> > ERROR StatusLogger
> > com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Direct
> > self-reference leading to cycle (through reference chain:
> >
> org.springframework.core.ResolvableType["componentType"]->org.springframework.core.ResolvableType["componentType"])
> > com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Direct
> > self-reference leading to cycle (through reference chain:
> >
> org.springframework.core.ResolvableType["componentType"]->org.springframework.core.ResolvableType["componentType"])
> > at
> >
> com.fasterxml.jackson.databind.exc.InvalidDefinitionException.from(InvalidDefinitionException.java:77)
> > at
> >
> com.fasterxml.jackson.databind.SerializerProvider.reportBadDefinition(SerializerProvider.java:1191)
> > at
> >
> com.fasterxml.jackson.databind.ser.BeanPropertyWriter._handleSelfReference(BeanPropertyWriter.java:944)
> > at
> >
> com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:721)
> > at
> >
> com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
> > at
> >
> com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
> >
> > The error can be prevented if `Mixin`s are added to the `ObjectMapper`
> > which ignore the corresponding class.
> >
> > The `Log4jJsonObjectMapper` is initialized in the `JacksonFactory.JSON`
> > class and I currently do not see any way to somehow configure this
> > ObjectMapper with additional MixIns.
> >
> > Hence, my question finally is:
> > How can I configure Log4J’s JSON mapping to not break on such
> > non-serializable log content?
> >
> > Thank you!
> > Dominik
> >
>
-- 
Matt Sicker 

Re: How to configure ObjectMapper of Log4j2 JSON Layout

[Gary Gregory]

Hi Dominik

I see two ways of addressing this:
1) Add a setObjectMapper(ObjectMapper)
to org.apache.logging.log4j.core.layout.JsonLayout.Builder
  This would work well for programmatic configurations
and/or:
2) Add a setObjectMapperFactory(String)
to org.apache.logging.log4j.core.layout.JsonLayout.Builder
  This would add a class name for implementors of a new interface
ObjectMapperFactory.
  This would work nicely for configuration files.

To keep all users in play, 2) seems best.

Thoughts?

Gary


On Fri, Jul 12, 2019 at 8:38 AM Dominik Sandjaja <
dominik.sandj...@trivago.com> wrote:

> Hello,
>
> we are using the JSON layout in our Spring Boot application to easily
> ingest the logs in Elasticsearch.
>
> One message that we log is about failed conversions, when an incoming
> request parameter cannot be converted correctly. As we use Spring’s
> Converter functionality for this, these `IllegalArgumentException`s are
> wrapped in a `ConversionFailedException`. This exception has fields of type
> `TypeDescriptor` which hold information about the source and the target
> type of the conversion.
>
> The problem is, that these `TypeDescriptor` objects cannot be serialized
> to JSON. The simple call
>
>
> com.fasterxml.jackson.databind.ObjectMapper().writeValueAsString(org.springframework.core.convert.TypeDescriptor.valueOf(String.javaClass))
>
> throws an exception like
>
> ERROR StatusLogger
> com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Direct
> self-reference leading to cycle (through reference chain:
> org.springframework.core.ResolvableType["componentType"]->org.springframework.core.ResolvableType["componentType"])
> com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Direct
> self-reference leading to cycle (through reference chain:
> org.springframework.core.ResolvableType["componentType"]->org.springframework.core.ResolvableType["componentType"])
> at
> com.fasterxml.jackson.databind.exc.InvalidDefinitionException.from(InvalidDefinitionException.java:77)
> at
> com.fasterxml.jackson.databind.SerializerProvider.reportBadDefinition(SerializerProvider.java:1191)
> at
> com.fasterxml.jackson.databind.ser.BeanPropertyWriter._handleSelfReference(BeanPropertyWriter.java:944)
> at
> com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:721)
> at
> com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
> at
> com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
>
> The error can be prevented if `Mixin`s are added to the `ObjectMapper`
> which ignore the corresponding class.
>
> The `Log4jJsonObjectMapper` is initialized in the `JacksonFactory.JSON`
> class and I currently do not see any way to somehow configure this
> ObjectMapper with additional MixIns.
>
> Hence, my question finally is:
> How can I configure Log4J’s JSON mapping to not break on such
> non-serializable log content?
>
> Thank you!
> Dominik
>

Re: How to configure ObjectMapper of Log4j2 JSON Layout

[Volkan Yazıcı]

Hello Dominik,

I don't know how you can achieve that using JsonLayout. That said, it
sounds like a really valid use case and I have already created a
log4j2-logstash-layout ticket for it
. I plan to
implement it in a couple of weeks and cut a new release. I will post a
follow-up to this thread when that happens.

Cheers.

On Fri, Jul 12, 2019 at 2:38 PM Dominik Sandjaja <
dominik.sandj...@trivago.com> wrote:

> Hello,
>
> we are using the JSON layout in our Spring Boot application to easily
> ingest the logs in Elasticsearch.
>
> One message that we log is about failed conversions, when an incoming
> request parameter cannot be converted correctly. As we use Spring’s
> Converter functionality for this, these `IllegalArgumentException`s are
> wrapped in a `ConversionFailedException`. This exception has fields of type
> `TypeDescriptor` which hold information about the source and the target
> type of the conversion.
>
> The problem is, that these `TypeDescriptor` objects cannot be serialized
> to JSON. The simple call
>
>
> com.fasterxml.jackson.databind.ObjectMapper().writeValueAsString(org.springframework.core.convert.TypeDescriptor.valueOf(String.javaClass))
>
> throws an exception like
>
> ERROR StatusLogger
> com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Direct
> self-reference leading to cycle (through reference chain:
> org.springframework.core.ResolvableType["componentType"]->org.springframework.core.ResolvableType["componentType"])
> com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Direct
> self-reference leading to cycle (through reference chain:
> org.springframework.core.ResolvableType["componentType"]->org.springframework.core.ResolvableType["componentType"])
> at
> com.fasterxml.jackson.databind.exc.InvalidDefinitionException.from(InvalidDefinitionException.java:77)
> at
> com.fasterxml.jackson.databind.SerializerProvider.reportBadDefinition(SerializerProvider.java:1191)
> at
> com.fasterxml.jackson.databind.ser.BeanPropertyWriter._handleSelfReference(BeanPropertyWriter.java:944)
> at
> com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:721)
> at
> com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
> at
> com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
>
> The error can be prevented if `Mixin`s are added to the `ObjectMapper`
> which ignore the corresponding class.
>
> The `Log4jJsonObjectMapper` is initialized in the `JacksonFactory.JSON`
> class and I currently do not see any way to somehow configure this
> ObjectMapper with additional MixIns.
>
> Hence, my question finally is:
> How can I configure Log4J’s JSON mapping to not break on such
> non-serializable log content?
>
> Thank you!
> Dominik
>