Perhaps mention of StringBuilderFormattable? It helps work around the “load the
string all at once” problem.
> On Jan 26, 2024, at 06:36, Gary Gregory <garydgreg...@gmail.com> wrote:
>
> Oh, I don't want to suggest this is a Lo4j issue at all. I am wondering if
> there is a pattern we should document aside from "know your third party
> objects" which is not helpful for us to say. It sounds like there is
> nothing for us to do.
>
> Gary
>
> On Fri, Jan 26, 2024, 7:16 AM Apache <ralph.go...@dslextreme.com> wrote:
>
>> That is kind of my point. Anyone who logs an object that behaves this way
>> is asking for trouble. It really sounds like an esoteric edge case to me. I
>> don’t view this as a Log4J problem as the error wouldn’t even have Log4J in
>> the stack trace.
>>
>> Ralph
>>
>>> On Jan 25, 2024, at 9:52 PM, Gary Gregory <garydgreg...@gmail.com>
>> wrote:
>>>
>>> I have no control over this. I think this is a general problem with
>> third
>>> party objects and toString(). There is no toString(fromHereToThere) or
>>> toStringReader() so I can't see a general way to deal with it. Even if I
>>> created a wrapper for the Object and called toString(), truncated and
>>> cached the result and have the wrapper return the shorter string in its
>>> toString(), I still would have toString()'d the original...
>>>
>>> Gary
>>>
>>>> On Thu, Jan 25, 2024, 11:33 PM Ralph Goers <ralph.go...@dslextreme.com>
>>>> wrote:
>>>>
>>>> OK. The only good way to handle that is to parse the YAML/JSON file
>> while
>>>> streaming it and extract just the fields you want to include in the
>> logs.
>>>>
>>>> Ralph
>>>>
>>>>> On Jan 25, 2024, at 6:40 PM, Gary Gregory <garydgreg...@gmail.com>
>>>> wrote:
>>>>>
>>>>> Well, it's worse than that because the object is an object created by
>>>>> parsing a YAML (or JSON) file, then the toString() of that object
>>>>> renders a String in some other format.
>>>>>
>>>>> Gary
>>>>>
>>>>> On Thu, Jan 25, 2024 at 7:45 PM Ralph Goers <
>> ralph.go...@dslextreme.com>
>>>> wrote:
>>>>>>
>>>>>> Volkan & Matt,
>>>>>>
>>>>>> Neither of those is going to help. The issue is that when the toString
>>>> method is called it reads a whole file in and stores it as a String.
>> This
>>>> could cause the OOM error. Truncating it in a layout simply limits how
>> much
>>>> of the String is printed. Even Gary’s proposal of calling substring() is
>>>> still going to operate on the whole String. He would really need a
>> method
>>>> that accepts the max number of characters to read from the file.
>>>>>>
>>>>>> Ralph
>>>>>>
>>>>>>> On Jan 25, 2024, at 2:49 PM, Volkan Yazıcı <vol...@yazi.ci> wrote:
>>>>>>>
>>>>>>> *[Just responding to Matt. I don't have an answer for Gary.]*
>>>>>>>
>>>>>>> `JsonTemplateLayout` has `maxStringLength`, and related with it,
>>>>>>> `truncatedStringSuffix`.
>>>>>>>
>>>>>>> On Thu, Jan 25, 2024 at 9:45 PM Matt Sicker <m...@musigma.org>
>> wrote:
>>>>>>>
>>>>>>>> You can use the %maxLength{…}{N} pattern converter with
>> PatternLayout
>>>> at
>>>>>>>> least. Unfortunately, I don’t think any other layouts support a
>>>> similar
>>>>>>>> option.
>>>>>>>>
>>>>>>>>> On Jan 25, 2024, at 10:55, Gary D. Gregory <ggreg...@apache.org>
>>>> wrote:
>>>>>>>>>
>>>>>>>>> Hi All,
>>>>>>>>>
>>>>>>>>> I'd like to ask how to if we can devise advice around an issue I
>> ran
>>>>>>>> into this week.
>>>>>>>>>
>>>>>>>>> One of our test suites processes about 40K files of test fixtures.
>>>> These
>>>>>>>> inputs are parsed, processed, and asserted. This randomly fails on a
>>>> call
>>>>>>>> to Logger#debug(), randomly in that it happens usually once per
>> build,
>>>>>>>> somewhere in a logging call. But it usually fails with a call that
>>>> looks
>>>>>>>> like this:
>>>>>>>>>
>>>>>>>>> logger.debug("This is fun" + myFunObject);
>>>>>>>>>
>>>>>>>>> To simplify things, let's say that it turns out that after an
>>>> underlying
>>>>>>>> third party jar file version upgrade the call to
>>>> myFunObject#toString() no
>>>>>>>> longer returns Object#toString() but rather (again to simplify) the
>>>>>>>> contents of the file that was parsed to create myFunObject. This
>>>> toString()
>>>>>>>> can be megabytes. The solution is obvious:
>>>>>>>>>
>>>>>>>>> logger.debug("This is fun", myFunObject::toString)
>>>>>>>>>
>>>>>>>>> And our CI builds no longer randomly fail since our default logging
>>>> does
>>>>>>>> not log at the debug level.
>>>>>>>>>
>>>>>>>>> A better solution could be:
>>>>>>>>>
>>>>>>>>> logger.debug("This is fun", () ->
>> myFunObject.toString().substring(0,
>>>>>>>> 100))
>>>>>>>>>
>>>>>>>>> where I still want _some_ information better than making my own
>>>>>>>> toString() with System#identityHashCode(Object) or somesuch. Sure,
>>>>>>>> .toString() is still called but it does not make it down into
>>>> logging. In
>>>>>>>> my case the OOME happened in myFunObject::toString so the
>> substring()
>>>>>>>> example would not have worked.
>>>>>>>>>
>>>>>>>>> My question is: Should we document some general advice on this
>>>> pattern
>>>>>>>> and what should the advice be? Would it make sense to have some
>>>> features
>>>>>>>> where we truncate/reject Strings above a threshold. And yes, calling
>>>>>>>> myFunObject.toString() can still still get me in trouble.
>>>>>>>>>
>>>>>>>>> Gary
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>
>>>>
>>
>>
