FWIW, if we want to do dependency- and garbage-free JSON serialization (not parsing), we have org.apache.logging.log4j.core.util.JsonUtils in log4j-core (used by GelfLayout).
On Wed, May 3, 2017 at 10:43 PM, Matt Sicker <[email protected]> wrote: > Remko has a ticket with a proposed binary logging format. Does that include > all the relevant data to reconstruct LogEvents by reading the file(s)? > > On 3 May 2017 at 15:36, Ralph Goers <[email protected]> wrote: > > > While I agree with Gary, my recommendation would be to either come up > with > > a custom serialization/deserialization format, use one where we can just > > copy a class or two, or customize the SerializedLayout to not include > > arbitrary classes. > > > > Ralph > > > > > On May 3, 2017, at 1:29 PM, Matt Sicker <[email protected]> wrote: > > > > > > I suppose serialization would be more important for dependency-free. I > > > don't believe there's any way to make Jackson GC-free, for example, as > a > > > use case here. > > > > > > On 3 May 2017 at 15:14, Gary Gregory <[email protected]> wrote: > > > > > >> This really feels silly to me. Doing JSON and XML IO is not 100% > trivial > > >> and removing the dep is not a driver for me and not a "feature". Well, > > the > > >> parsing is not trivial, generation is simple even with escaping. But > > why? I > > >> really don't see the point for the apps I write at least. I mean, I > > have my > > >> jar, a pile of third party jars including at least 2 log4j jars, api > and > > >> core. > > >> > > >> The point of this feature is that for the smallest app, I have my > jar, 2 > > >> log4j jars instead of the those plus jackson or gson (if we wanted to > > redo > > >> JSON)? > > >> > > >> Seems like a small gain at a high increase in complexity. IMO. > > >> > > >> Gary > > >> > > >> On May 3, 2017 12:49 PM, "Matt Sicker" <[email protected]> wrote: > > >> > > >> I'll make a ticket for adding dependency-free JSON serialization and > > >> deserialization (probably separate tickets). > > >> > > >> On 3 May 2017 at 12:57, Remko Popma <[email protected]> wrote: > > >> > > >>> Thank you for the clarification, Mikael! > > >>> We may have to live with an external dependency initially until the > > JSON > > >>> serializer that Matt mentioned is ready. > > >>> > > >>> > > >>> > > >>> On Wed, May 3, 2017 at 5:22 PM, Mikael Ståldal < > > >> [email protected]> > > >>> wrote: > > >>> > > >>>> GelfLayout currently lacks markers, context stack and detailed > > >> structured > > >>>> stacktrace (it has a string formatted stacktrace), thread id, > > location, > > >>>> etc. > > >>>> > > >>>> The GELF standard i extensible, but only with key-value pairs with > > >> string > > >>>> or numeric values, it cannot be extended with an arbitrary JSON > > >>> structure. > > >>>> > > >>>> GelfLayout currently have two Log4j specific extensions for > loggerName > > >>> and > > >>>> threadName. We could add some more, but it is not feasible to store > a > > >>>> detailed structured stacktrace there. > > >>>> > > >>>> If we aim for complete reconstruction of a LogEvent, with all > details, > > >>> then > > >>>> GELF is not a suitable format (and neither is Syslog/RFC5424). > > >>>> > > >>>> On Wed, May 3, 2017 at 3:44 AM, Matt Sicker <[email protected]> > wrote: > > >>>> > > >>>>> GelfLayout follows the standard from Graylog, similar in idea to > the > > >>>> syslog > > >>>>> standard. > > >>>>> > > >>>>> On 2 May 2017 at 19:34, Remko Popma <[email protected]> wrote: > > >>>>> > > >>>>>> That sounds good! > > >>>>>> Essentially we want a layout that allows the receiver to > > >> reconstruct > > >>>> the > > >>>>>> LogEvent (assuming the receiver has all required classes for > custom > > >>>>>> messages etc). > > >>>>>> > > >>>>>> Isn't GelfLayout quite close? What info does it leave out? > > >>>>>> > > >>>>>> > > >>>>>> > > >>>>>> (Shameless plug) Every java main() method deserves > > >>> http://picocli.info > > >>>>>> > > >>>>>>> On May 3, 2017, at 0:44, Matt Sicker <[email protected]> wrote: > > >>>>>>> > > >>>>>>> I added some minimal code in the escape pattern converter for > > >>>> handling > > >>>>>> JSON > > >>>>>>> string encoding. We can probably include a minimal JSON > > >>> serialization > > >>>>>>> "library" inside log4j-core which could also be included in the > > >>>> general > > >>>>>>> GC-free ecosystem we have going on. > > >>>>>>> > > >>>>>>>> On 2 May 2017 at 10:14, Mikael Ståldal < > > >> [email protected] > > >>>> > > >>>>>> wrote: > > >>>>>>>> > > >>>>>>>> Oh, I did not think about that aspect. Both JsonLayout and a > > >>>> potential > > >>>>>> new > > >>>>>>>> AvroLayout (will) have external dependency. > > >>>>>>>> > > >>>>>>>> Without external dependency, we currently have GelfLayout, > > >>>>> PatternLayout > > >>>>>>>> and RFC5424Layout. > > >>>>>>>> > > >>>>>>>> GelfLayout and RFC5424Layout would be useful in some cases, but > > >>> they > > >>>>> do > > >>>>>> not > > >>>>>>>> have all information present in SerializedLayout and JsonLayout. > > >>>>>>>> > > >>>>>>>> RFC5424Layout and PatternLayout can be configured to include all > > >>>>>>>> information, but that's quite involved. > > >>>>>>>> > > >>>>>>>>> On Tue, May 2, 2017 at 4:11 PM, Remko Popma < > > >>> [email protected] > > >>>>> > > >>>>>> wrote: > > >>>>>>>>> > > >>>>>>>>> What layout do we have available that does not require an > > >>> external > > >>>>>>>>> dependency? > > >>>>>>>>> > > >>>>>>>>> On Tue, May 2, 2017 at 8:38 PM, Mikael Ståldal < > > >>>>>>>> [email protected]> > > >>>>>>>>> wrote: > > >>>>>>>>> > > >>>>>>>>>> Given the inherent security problems with Java object > > >>>> serialization > > >>>>>>>>>> (highlighted by CVE-2017-5645), I do suggest that we deprecate > > >>>>>>>>>> SerializedLayout and remove it as default for SocketAppender, > > >>> and > > >>>>> all > > >>>>>>>>> other > > >>>>>>>>>> appenders which currently have it as default. (We can still > > >> keep > > >>>>>>>>>> SerializedLayout, with a warning about security issues in > > >>>>>>>> documentation, > > >>>>>>>>>> but users will have to enable it explicitly.) > > >>>>>>>>>> > > >>>>>>>>>> Some people have missed the fact that you can configure > > >>>>> SocketAppender > > >>>>>>>>> with > > >>>>>>>>>> another layout. > > >>>>>>>>>> > > >>>>>>>>>> I suggest we do this in the 2.9 release. > > >>>>>>>>>> > > >>>>>>>>>> I know this will break some existing configurations, but given > > >>> the > > >>>>>>>>> security > > >>>>>>>>>> problems, I think that is a price we have to pay in this case. > > >>>>>>>>>> > > >>>>>>>>>> We have a JIRA ticket for a new Avro based binary layout: > > >>>>>>>>>> https://issues.apache.org/jira/browse/LOG4J2-1871 > > >>>>>>>>>> > > >>>>>>>>>> If we implement that in time for 2.9, we can recommend it as a > > >>>>>>>>> replacement > > >>>>>>>>>> for SerializedLayout. If not, we could recommend JsonLayout > > >>> which > > >>>>>>>> should > > >>>>>>>>>> contain all necessary information. > > >>>>>>>>>> > > >>>>>>>>>> -- > > >>>>>>>>>> [image: MagineTV] > > >>>>>>>>>> > > >>>>>>>>>> *Mikael Ståldal* > > >>>>>>>>>> Senior software developer > > >>>>>>>>>> > > >>>>>>>>>> *Magine TV* > > >>>>>>>>>> [email protected] > > >>>>>>>>>> Grev Turegatan 3 | 114 46 Stockholm, Sweden | > > >>> www.magine.com > > >>>>>>>>>> > > >>>>>>>>>> Privileged and/or Confidential Information may be contained in > > >>>> this > > >>>>>>>>>> message. If you are not the addressee indicated in this > > >> message > > >>>>>>>>>> (or responsible for delivery of the message to such a person), > > >>> you > > >>>>> may > > >>>>>>>>> not > > >>>>>>>>>> copy or deliver this message to anyone. In such case, > > >>>>>>>>>> you should destroy this message and kindly notify the sender > > >> by > > >>>>> reply > > >>>>>>>>>> email. > > >>>>>>>>>> > > >>>>>>>>> > > >>>>>>>> > > >>>>>>>> > > >>>>>>>> > > >>>>>>>> -- > > >>>>>>>> [image: MagineTV] > > >>>>>>>> > > >>>>>>>> *Mikael Ståldal* > > >>>>>>>> Senior software developer > > >>>>>>>> > > >>>>>>>> *Magine TV* > > >>>>>>>> [email protected] > > >>>>>>>> Grev Turegatan 3 | 114 46 Stockholm, Sweden | > > >> www.magine.com > > >>>>>>>> > > >>>>>>>> Privileged and/or Confidential Information may be contained in > > >>> this > > >>>>>>>> message. If you are not the addressee indicated in this message > > >>>>>>>> (or responsible for delivery of the message to such a person), > > >> you > > >>>> may > > >>>>>> not > > >>>>>>>> copy or deliver this message to anyone. In such case, > > >>>>>>>> you should destroy this message and kindly notify the sender by > > >>>> reply > > >>>>>>>> email. > > >>>>>>>> > > >>>>>>> > > >>>>>>> > > >>>>>>> > > >>>>>>> -- > > >>>>>>> Matt Sicker <[email protected]> > > >>>>>> > > >>>>> > > >>>>> > > >>>>> > > >>>>> -- > > >>>>> Matt Sicker <[email protected]> > > >>>>> > > >>>> > > >>>> > > >>>> > > >>>> -- > > >>>> [image: MagineTV] > > >>>> > > >>>> *Mikael Ståldal* > > >>>> Senior software developer > > >>>> > > >>>> *Magine TV* > > >>>> [email protected] > > >>>> Grev Turegatan 3 | 114 46 Stockholm, Sweden | www.magine.com > > >>>> > > >>>> Privileged and/or Confidential Information may be contained in this > > >>>> message. If you are not the addressee indicated in this message > > >>>> (or responsible for delivery of the message to such a person), you > may > > >>> not > > >>>> copy or deliver this message to anyone. In such case, > > >>>> you should destroy this message and kindly notify the sender by > reply > > >>>> email. > > >>>> > > >>> > > >> > > >> > > >> > > >> -- > > >> Matt Sicker <[email protected]> > > >> > > > > > > > > > > > > -- > > > Matt Sicker <[email protected]> > > > > > > > > > -- > Matt Sicker <[email protected]> > -- [image: MagineTV] *Mikael Ståldal* Senior software developer *Magine TV* [email protected] Grev Turegatan 3 | 114 46 Stockholm, Sweden | www.magine.com Privileged and/or Confidential Information may be contained in this message. If you are not the addressee indicated in this message (or responsible for delivery of the message to such a person), you may not copy or deliver this message to anyone. In such case, you should destroy this message and kindly notify the sender by reply email.
