Oh, I don't want to suggest this is a Lo4j issue at all. I am wondering if
there is a pattern we should document aside from "know your third party
objects" which is not helpful for us to say. It sounds like there is
nothing for us to do.
Gary
On Fri, Jan 26, 2024, 7:16 AM Apache <ralph.go...@dslextreme.com> wrote:
> That is kind of my point. Anyone who logs an object that behaves this way
> is asking for trouble. It really sounds like an esoteric edge case to me. I
> don’t view this as a Log4J problem as the error wouldn’t even have Log4J in
> the stack trace.
>
> Ralph
>
> > On Jan 25, 2024, at 9:52 PM, Gary Gregory <garydgreg...@gmail.com>
> wrote:
> >
> > I have no control over this. I think this is a general problem with
> third
> > party objects and toString(). There is no toString(fromHereToThere) or
> > toStringReader() so I can't see a general way to deal with it. Even if I
> > created a wrapper for the Object and called toString(), truncated and
> > cached the result and have the wrapper return the shorter string in its
> > toString(), I still would have toString()'d the original...
> >
> > Gary
> >
> >> On Thu, Jan 25, 2024, 11:33 PM Ralph Goers <ralph.go...@dslextreme.com>
> >> wrote:
> >>
> >> OK. The only good way to handle that is to parse the YAML/JSON file
> while
> >> streaming it and extract just the fields you want to include in the
> logs.
> >>
> >> Ralph
> >>
> >>> On Jan 25, 2024, at 6:40 PM, Gary Gregory <garydgreg...@gmail.com>
> >> wrote:
> >>>
> >>> Well, it's worse than that because the object is an object created by
> >>> parsing a YAML (or JSON) file, then the toString() of that object
> >>> renders a String in some other format.
> >>>
> >>> Gary
> >>>
> >>> On Thu, Jan 25, 2024 at 7:45 PM Ralph Goers <
> ralph.go...@dslextreme.com>
> >> wrote:
> >>>>
> >>>> Volkan & Matt,
> >>>>
> >>>> Neither of those is going to help. The issue is that when the toString
> >> method is called it reads a whole file in and stores it as a String.
> This
> >> could cause the OOM error. Truncating it in a layout simply limits how
> much
> >> of the String is printed. Even Gary’s proposal of calling substring() is
> >> still going to operate on the whole String. He would really need a
> method
> >> that accepts the max number of characters to read from the file.
> >>>>
> >>>> Ralph
> >>>>
> >>>>> On Jan 25, 2024, at 2:49 PM, Volkan Yazıcı <vol...@yazi.ci> wrote:
> >>>>>
> >>>>> *[Just responding to Matt. I don't have an answer for Gary.]*
> >>>>>
> >>>>> `JsonTemplateLayout` has `maxStringLength`, and related with it,
> >>>>> `truncatedStringSuffix`.
> >>>>>
> >>>>> On Thu, Jan 25, 2024 at 9:45 PM Matt Sicker <m...@musigma.org>
> wrote:
> >>>>>
> >>>>>> You can use the %maxLength{…}{N} pattern converter with
> PatternLayout
> >> at
> >>>>>> least. Unfortunately, I don’t think any other layouts support a
> >> similar
> >>>>>> option.
> >>>>>>
> >>>>>>> On Jan 25, 2024, at 10:55, Gary D. Gregory <ggreg...@apache.org>
> >> wrote:
> >>>>>>>
> >>>>>>> Hi All,
> >>>>>>>
> >>>>>>> I'd like to ask how to if we can devise advice around an issue I
> ran
> >>>>>> into this week.
> >>>>>>>
> >>>>>>> One of our test suites processes about 40K files of test fixtures.
> >> These
> >>>>>> inputs are parsed, processed, and asserted. This randomly fails on a
> >> call
> >>>>>> to Logger#debug(), randomly in that it happens usually once per
> build,
> >>>>>> somewhere in a logging call. But it usually fails with a call that
> >> looks
> >>>>>> like this:
> >>>>>>>
> >>>>>>> logger.debug("This is fun" + myFunObject);
> >>>>>>>
> >>>>>>> To simplify things, let's say that it turns out that after an
> >> underlying
> >>>>>> third party jar file version upgrade the call to
> >> myFunObject#toString() no
> >>>>>> longer returns Object#toString() but rather (again to simplify) the
> >>>>>> contents of the file that was parsed to create myFunObject. This
> >> toString()
> >>>>>> can be megabytes. The solution is obvious:
> >>>>>>>
> >>>>>>> logger.debug("This is fun", myFunObject::toString)
> >>>>>>>
> >>>>>>> And our CI builds no longer randomly fail since our default logging
> >> does
> >>>>>> not log at the debug level.
> >>>>>>>
> >>>>>>> A better solution could be:
> >>>>>>>
> >>>>>>> logger.debug("This is fun", () ->
> myFunObject.toString().substring(0,
> >>>>>> 100))
> >>>>>>>
> >>>>>>> where I still want _some_ information better than making my own
> >>>>>> toString() with System#identityHashCode(Object) or somesuch. Sure,
> >>>>>> .toString() is still called but it does not make it down into
> >> logging. In
> >>>>>> my case the OOME happened in myFunObject::toString so the
> substring()
> >>>>>> example would not have worked.
> >>>>>>>
> >>>>>>> My question is: Should we document some general advice on this
> >> pattern
> >>>>>> and what should the advice be? Would it make sense to have some
> >> features
> >>>>>> where we truncate/reject Strings above a threshold. And yes, calling
> >>>>>> myFunObject.toString() can still still get me in trouble.
> >>>>>>>
> >>>>>>> Gary
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>
> >>
> >>
>
>
