I have no control over this. I think this is a general problem with third party objects and toString(). There is no toString(fromHereToThere) or toStringReader() so I can't see a general way to deal with it. Even if I created a wrapper for the Object and called toString(), truncated and cached the result and have the wrapper return the shorter string in its toString(), I still would have toString()'d the original...
Gary On Thu, Jan 25, 2024, 11:33 PM Ralph Goers <[email protected]> wrote: > OK. The only good way to handle that is to parse the YAML/JSON file while > streaming it and extract just the fields you want to include in the logs. > > Ralph > > > On Jan 25, 2024, at 6:40 PM, Gary Gregory <[email protected]> > wrote: > > > > Well, it's worse than that because the object is an object created by > > parsing a YAML (or JSON) file, then the toString() of that object > > renders a String in some other format. > > > > Gary > > > > On Thu, Jan 25, 2024 at 7:45 PM Ralph Goers <[email protected]> > wrote: > >> > >> Volkan & Matt, > >> > >> Neither of those is going to help. The issue is that when the toString > method is called it reads a whole file in and stores it as a String. This > could cause the OOM error. Truncating it in a layout simply limits how much > of the String is printed. Even Gary’s proposal of calling substring() is > still going to operate on the whole String. He would really need a method > that accepts the max number of characters to read from the file. > >> > >> Ralph > >> > >>> On Jan 25, 2024, at 2:49 PM, Volkan Yazıcı <[email protected]> wrote: > >>> > >>> *[Just responding to Matt. I don't have an answer for Gary.]* > >>> > >>> `JsonTemplateLayout` has `maxStringLength`, and related with it, > >>> `truncatedStringSuffix`. > >>> > >>> On Thu, Jan 25, 2024 at 9:45 PM Matt Sicker <[email protected]> wrote: > >>> > >>>> You can use the %maxLength{…}{N} pattern converter with PatternLayout > at > >>>> least. Unfortunately, I don’t think any other layouts support a > similar > >>>> option. > >>>> > >>>>> On Jan 25, 2024, at 10:55, Gary D. Gregory <[email protected]> > wrote: > >>>>> > >>>>> Hi All, > >>>>> > >>>>> I'd like to ask how to if we can devise advice around an issue I ran > >>>> into this week. > >>>>> > >>>>> One of our test suites processes about 40K files of test fixtures. > These > >>>> inputs are parsed, processed, and asserted. This randomly fails on a > call > >>>> to Logger#debug(), randomly in that it happens usually once per build, > >>>> somewhere in a logging call. But it usually fails with a call that > looks > >>>> like this: > >>>>> > >>>>> logger.debug("This is fun" + myFunObject); > >>>>> > >>>>> To simplify things, let's say that it turns out that after an > underlying > >>>> third party jar file version upgrade the call to > myFunObject#toString() no > >>>> longer returns Object#toString() but rather (again to simplify) the > >>>> contents of the file that was parsed to create myFunObject. This > toString() > >>>> can be megabytes. The solution is obvious: > >>>>> > >>>>> logger.debug("This is fun", myFunObject::toString) > >>>>> > >>>>> And our CI builds no longer randomly fail since our default logging > does > >>>> not log at the debug level. > >>>>> > >>>>> A better solution could be: > >>>>> > >>>>> logger.debug("This is fun", () -> myFunObject.toString().substring(0, > >>>> 100)) > >>>>> > >>>>> where I still want _some_ information better than making my own > >>>> toString() with System#identityHashCode(Object) or somesuch. Sure, > >>>> .toString() is still called but it does not make it down into > logging. In > >>>> my case the OOME happened in myFunObject::toString so the substring() > >>>> example would not have worked. > >>>>> > >>>>> My question is: Should we document some general advice on this > pattern > >>>> and what should the advice be? Would it make sense to have some > features > >>>> where we truncate/reject Strings above a threshold. And yes, calling > >>>> myFunObject.toString() can still still get me in trouble. > >>>>> > >>>>> Gary > >>>>> > >>>> > >>>> > >> > >
