Remko has a ticket with a proposed binary logging format. Does that include all the relevant data to reconstruct LogEvents by reading the file(s)?
On 3 May 2017 at 15:36, Ralph Goers <[email protected]> wrote: > While I agree with Gary, my recommendation would be to either come up with > a custom serialization/deserialization format, use one where we can just > copy a class or two, or customize the SerializedLayout to not include > arbitrary classes. > > Ralph > > > On May 3, 2017, at 1:29 PM, Matt Sicker <[email protected]> wrote: > > > > I suppose serialization would be more important for dependency-free. I > > don't believe there's any way to make Jackson GC-free, for example, as a > > use case here. > > > > On 3 May 2017 at 15:14, Gary Gregory <[email protected]> wrote: > > > >> This really feels silly to me. Doing JSON and XML IO is not 100% trivial > >> and removing the dep is not a driver for me and not a "feature". Well, > the > >> parsing is not trivial, generation is simple even with escaping. But > why? I > >> really don't see the point for the apps I write at least. I mean, I > have my > >> jar, a pile of third party jars including at least 2 log4j jars, api and > >> core. > >> > >> The point of this feature is that for the smallest app, I have my jar, 2 > >> log4j jars instead of the those plus jackson or gson (if we wanted to > redo > >> JSON)? > >> > >> Seems like a small gain at a high increase in complexity. IMO. > >> > >> Gary > >> > >> On May 3, 2017 12:49 PM, "Matt Sicker" <[email protected]> wrote: > >> > >> I'll make a ticket for adding dependency-free JSON serialization and > >> deserialization (probably separate tickets). > >> > >> On 3 May 2017 at 12:57, Remko Popma <[email protected]> wrote: > >> > >>> Thank you for the clarification, Mikael! > >>> We may have to live with an external dependency initially until the > JSON > >>> serializer that Matt mentioned is ready. > >>> > >>> > >>> > >>> On Wed, May 3, 2017 at 5:22 PM, Mikael Ståldal < > >> [email protected]> > >>> wrote: > >>> > >>>> GelfLayout currently lacks markers, context stack and detailed > >> structured > >>>> stacktrace (it has a string formatted stacktrace), thread id, > location, > >>>> etc. > >>>> > >>>> The GELF standard i extensible, but only with key-value pairs with > >> string > >>>> or numeric values, it cannot be extended with an arbitrary JSON > >>> structure. > >>>> > >>>> GelfLayout currently have two Log4j specific extensions for loggerName > >>> and > >>>> threadName. We could add some more, but it is not feasible to store a > >>>> detailed structured stacktrace there. > >>>> > >>>> If we aim for complete reconstruction of a LogEvent, with all details, > >>> then > >>>> GELF is not a suitable format (and neither is Syslog/RFC5424). > >>>> > >>>> On Wed, May 3, 2017 at 3:44 AM, Matt Sicker <[email protected]> wrote: > >>>> > >>>>> GelfLayout follows the standard from Graylog, similar in idea to the > >>>> syslog > >>>>> standard. > >>>>> > >>>>> On 2 May 2017 at 19:34, Remko Popma <[email protected]> wrote: > >>>>> > >>>>>> That sounds good! > >>>>>> Essentially we want a layout that allows the receiver to > >> reconstruct > >>>> the > >>>>>> LogEvent (assuming the receiver has all required classes for custom > >>>>>> messages etc). > >>>>>> > >>>>>> Isn't GelfLayout quite close? What info does it leave out? > >>>>>> > >>>>>> > >>>>>> > >>>>>> (Shameless plug) Every java main() method deserves > >>> http://picocli.info > >>>>>> > >>>>>>> On May 3, 2017, at 0:44, Matt Sicker <[email protected]> wrote: > >>>>>>> > >>>>>>> I added some minimal code in the escape pattern converter for > >>>> handling > >>>>>> JSON > >>>>>>> string encoding. We can probably include a minimal JSON > >>> serialization > >>>>>>> "library" inside log4j-core which could also be included in the > >>>> general > >>>>>>> GC-free ecosystem we have going on. > >>>>>>> > >>>>>>>> On 2 May 2017 at 10:14, Mikael Ståldal < > >> [email protected] > >>>> > >>>>>> wrote: > >>>>>>>> > >>>>>>>> Oh, I did not think about that aspect. Both JsonLayout and a > >>>> potential > >>>>>> new > >>>>>>>> AvroLayout (will) have external dependency. > >>>>>>>> > >>>>>>>> Without external dependency, we currently have GelfLayout, > >>>>> PatternLayout > >>>>>>>> and RFC5424Layout. > >>>>>>>> > >>>>>>>> GelfLayout and RFC5424Layout would be useful in some cases, but > >>> they > >>>>> do > >>>>>> not > >>>>>>>> have all information present in SerializedLayout and JsonLayout. > >>>>>>>> > >>>>>>>> RFC5424Layout and PatternLayout can be configured to include all > >>>>>>>> information, but that's quite involved. > >>>>>>>> > >>>>>>>>> On Tue, May 2, 2017 at 4:11 PM, Remko Popma < > >>> [email protected] > >>>>> > >>>>>> wrote: > >>>>>>>>> > >>>>>>>>> What layout do we have available that does not require an > >>> external > >>>>>>>>> dependency? > >>>>>>>>> > >>>>>>>>> On Tue, May 2, 2017 at 8:38 PM, Mikael Ståldal < > >>>>>>>> [email protected]> > >>>>>>>>> wrote: > >>>>>>>>> > >>>>>>>>>> Given the inherent security problems with Java object > >>>> serialization > >>>>>>>>>> (highlighted by CVE-2017-5645), I do suggest that we deprecate > >>>>>>>>>> SerializedLayout and remove it as default for SocketAppender, > >>> and > >>>>> all > >>>>>>>>> other > >>>>>>>>>> appenders which currently have it as default. (We can still > >> keep > >>>>>>>>>> SerializedLayout, with a warning about security issues in > >>>>>>>> documentation, > >>>>>>>>>> but users will have to enable it explicitly.) > >>>>>>>>>> > >>>>>>>>>> Some people have missed the fact that you can configure > >>>>> SocketAppender > >>>>>>>>> with > >>>>>>>>>> another layout. > >>>>>>>>>> > >>>>>>>>>> I suggest we do this in the 2.9 release. > >>>>>>>>>> > >>>>>>>>>> I know this will break some existing configurations, but given > >>> the > >>>>>>>>> security > >>>>>>>>>> problems, I think that is a price we have to pay in this case. > >>>>>>>>>> > >>>>>>>>>> We have a JIRA ticket for a new Avro based binary layout: > >>>>>>>>>> https://issues.apache.org/jira/browse/LOG4J2-1871 > >>>>>>>>>> > >>>>>>>>>> If we implement that in time for 2.9, we can recommend it as a > >>>>>>>>> replacement > >>>>>>>>>> for SerializedLayout. If not, we could recommend JsonLayout > >>> which > >>>>>>>> should > >>>>>>>>>> contain all necessary information. > >>>>>>>>>> > >>>>>>>>>> -- > >>>>>>>>>> [image: MagineTV] > >>>>>>>>>> > >>>>>>>>>> *Mikael Ståldal* > >>>>>>>>>> Senior software developer > >>>>>>>>>> > >>>>>>>>>> *Magine TV* > >>>>>>>>>> [email protected] > >>>>>>>>>> Grev Turegatan 3 | 114 46 Stockholm, Sweden | > >>> www.magine.com > >>>>>>>>>> > >>>>>>>>>> Privileged and/or Confidential Information may be contained in > >>>> this > >>>>>>>>>> message. If you are not the addressee indicated in this > >> message > >>>>>>>>>> (or responsible for delivery of the message to such a person), > >>> you > >>>>> may > >>>>>>>>> not > >>>>>>>>>> copy or deliver this message to anyone. In such case, > >>>>>>>>>> you should destroy this message and kindly notify the sender > >> by > >>>>> reply > >>>>>>>>>> email. > >>>>>>>>>> > >>>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> -- > >>>>>>>> [image: MagineTV] > >>>>>>>> > >>>>>>>> *Mikael Ståldal* > >>>>>>>> Senior software developer > >>>>>>>> > >>>>>>>> *Magine TV* > >>>>>>>> [email protected] > >>>>>>>> Grev Turegatan 3 | 114 46 Stockholm, Sweden | > >> www.magine.com > >>>>>>>> > >>>>>>>> Privileged and/or Confidential Information may be contained in > >>> this > >>>>>>>> message. If you are not the addressee indicated in this message > >>>>>>>> (or responsible for delivery of the message to such a person), > >> you > >>>> may > >>>>>> not > >>>>>>>> copy or deliver this message to anyone. In such case, > >>>>>>>> you should destroy this message and kindly notify the sender by > >>>> reply > >>>>>>>> email. > >>>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> -- > >>>>>>> Matt Sicker <[email protected]> > >>>>>> > >>>>> > >>>>> > >>>>> > >>>>> -- > >>>>> Matt Sicker <[email protected]> > >>>>> > >>>> > >>>> > >>>> > >>>> -- > >>>> [image: MagineTV] > >>>> > >>>> *Mikael Ståldal* > >>>> Senior software developer > >>>> > >>>> *Magine TV* > >>>> [email protected] > >>>> Grev Turegatan 3 | 114 46 Stockholm, Sweden | www.magine.com > >>>> > >>>> Privileged and/or Confidential Information may be contained in this > >>>> message. If you are not the addressee indicated in this message > >>>> (or responsible for delivery of the message to such a person), you may > >>> not > >>>> copy or deliver this message to anyone. In such case, > >>>> you should destroy this message and kindly notify the sender by reply > >>>> email. > >>>> > >>> > >> > >> > >> > >> -- > >> Matt Sicker <[email protected]> > >> > > > > > > > > -- > > Matt Sicker <[email protected]> > > > -- Matt Sicker <[email protected]>
