OK. The only good way to handle that is to parse the YAML/JSON file while
streaming it and extract just the fields you want to include in the logs.
Ralph
> On Jan 25, 2024, at 6:40 PM, Gary Gregory <garydgreg...@gmail.com> wrote:
>
> Well, it's worse than that because the object is an object created by
> parsing a YAML (or JSON) file, then the toString() of that object
> renders a String in some other format.
>
> Gary
>
> On Thu, Jan 25, 2024 at 7:45 PM Ralph Goers <ralph.go...@dslextreme.com>
> wrote:
>>
>> Volkan & Matt,
>>
>> Neither of those is going to help. The issue is that when the toString
>> method is called it reads a whole file in and stores it as a String. This
>> could cause the OOM error. Truncating it in a layout simply limits how much
>> of the String is printed. Even Gary’s proposal of calling substring() is
>> still going to operate on the whole String. He would really need a method
>> that accepts the max number of characters to read from the file.
>>
>> Ralph
>>
>>> On Jan 25, 2024, at 2:49 PM, Volkan Yazıcı <vol...@yazi.ci> wrote:
>>>
>>> *[Just responding to Matt. I don't have an answer for Gary.]*
>>>
>>> `JsonTemplateLayout` has `maxStringLength`, and related with it,
>>> `truncatedStringSuffix`.
>>>
>>> On Thu, Jan 25, 2024 at 9:45 PM Matt Sicker <m...@musigma.org> wrote:
>>>
>>>> You can use the %maxLength{…}{N} pattern converter with PatternLayout at
>>>> least. Unfortunately, I don’t think any other layouts support a similar
>>>> option.
>>>>
>>>>> On Jan 25, 2024, at 10:55, Gary D. Gregory <ggreg...@apache.org> wrote:
>>>>>
>>>>> Hi All,
>>>>>
>>>>> I'd like to ask how to if we can devise advice around an issue I ran
>>>> into this week.
>>>>>
>>>>> One of our test suites processes about 40K files of test fixtures. These
>>>> inputs are parsed, processed, and asserted. This randomly fails on a call
>>>> to Logger#debug(), randomly in that it happens usually once per build,
>>>> somewhere in a logging call. But it usually fails with a call that looks
>>>> like this:
>>>>>
>>>>> logger.debug("This is fun" + myFunObject);
>>>>>
>>>>> To simplify things, let's say that it turns out that after an underlying
>>>> third party jar file version upgrade the call to myFunObject#toString() no
>>>> longer returns Object#toString() but rather (again to simplify) the
>>>> contents of the file that was parsed to create myFunObject. This toString()
>>>> can be megabytes. The solution is obvious:
>>>>>
>>>>> logger.debug("This is fun", myFunObject::toString)
>>>>>
>>>>> And our CI builds no longer randomly fail since our default logging does
>>>> not log at the debug level.
>>>>>
>>>>> A better solution could be:
>>>>>
>>>>> logger.debug("This is fun", () -> myFunObject.toString().substring(0,
>>>> 100))
>>>>>
>>>>> where I still want _some_ information better than making my own
>>>> toString() with System#identityHashCode(Object) or somesuch. Sure,
>>>> .toString() is still called but it does not make it down into logging. In
>>>> my case the OOME happened in myFunObject::toString so the substring()
>>>> example would not have worked.
>>>>>
>>>>> My question is: Should we document some general advice on this pattern
>>>> and what should the advice be? Would it make sense to have some features
>>>> where we truncate/reject Strings above a threshold. And yes, calling
>>>> myFunObject.toString() can still still get me in trouble.
>>>>>
>>>>> Gary
>>>>>
>>>>
>>>>
>>
