Re: Use RTC in Log4j and Log4Net

[Ralph Goers]

I sort of agree and sort of not. It would be nice to have automation that can 
enforce some rules. You can’t do that if there are none.

We could agree in principle to a small number of files and lines but set the 
enforcement to a larger number to allow for some exceptions. For example, if we 
agreed on 10 files max with no more than 5 lines each (50 lines total) we could 
set the enforcement at 20 files with a max of 200 lines (notice that is not 10 
lines per file).

Ralph

> On Sep 18, 2024, at 2:17 AM, Gary Gregory  wrote:
> 
> This proposal is two pages (on my phone)! I can't imagine counting lines
> and walking through these steps, it's crazy making IMO. How would you count
> lines anyway? From diff output?
> 
> I'd rather skip the bikeshedding and let devs create PRs when they think it
> makes sense. IOW, when they want a review. Otherwise, I trust devs to check
> their work.
> 
> Gary
> 
> On Wed, Sep 18, 2024, 3:46 AM Volkan Yazıcı  wrote:
> 
>> I agree in principle, but...
>> 
>> *Exemption criteria*
>> 
>> I agree with Ralph on that we should specify criteria on what shall be
>> exempt from this PR-driven workflow policy. We should of course materialize
>> these criteria collaboratively. Below is my attempt for a starting point:
>> 
>> Changes can be exempt from this policy if and only if they suffice *only
>> one* of the following conditions:
>> 
>>   1. Grammatical corrections to the documentation (incl. Javadoc and
>>   release notes)
>>   2. Code typo fixes¹ less than 10 LoC
>>   3. Infrastructure fixes¹ touching to `pom.xml` or CI scripts, and less
>>   than 10 LoC
>> 
>> ¹ A "fix" is assumed to not introduce a change in the expected behaviour of
>> the associated component. Changing the expected behaviour does not qualify
>> a fix.
>> 
>> *Scoped repositories*
>> 
>> I suggest extending the scope of this policy to cover the following
>> repositories too:
>> 
>>   1. `logging-parent`
>>   2. `logging-log4j-jakarta`
>>   3. `logging-jmx-gui`
>>   4. `logging-samples`
>>   5. `logging-site`
>>   6. `logging-log4net-site`
>> 
>> *Maintainer availability*
>> 
>> The PR-driven workflow can fly because we have full-time maintainers. But
>> that will not be the case anymore in 2-3 months due to funds drying out. I
>> suspect introducing such a policy with strict deadlines (e.g., 24 hours is
>> a pretty tight time budget for a project with no full-time maintainers) and
>> no exemptions might jeopardize the streamlining of development in the long
>> run. That said, as Christian noted, we can adapt/drop the policy later on
>> if needed.
>> 
>> I disagree with Matt's *"losing momentum from waiting for an unnecessary
>> code review"* statement. In the last three years or so – even before STF
>> funding and Piotr! – well-scoped PRs, where the author onboards reviewers
>> with sufficient navigation tips, have been processed very swiftly, AFAICT.
>> 
>> On Wed, Sep 18, 2024 at 12:16 AM Ralph Goers 
>> wrote:
>> 
>>> This isn’t a vote so I am not going to. If I had to vote I wouldn’t vote
>>> for a policy that requires RTC always. However, I would vote for a policy
>>> that requires RTC when specified criteria are met.
>>> 
>>> Ralph
>>> 
>>>> On Sep 17, 2024, at 10:28 AM, Ralph Goers 
>>> wrote:
>>>> 
>>>> First, the obvious. I haven’t committed much in a while. The last
>>> several I did I used PRs primarily because it makes it easier for people
>> to
>>> review the changes but I didn’t necessarily wait for a review.  For
>> really
>>> simple stuff I've never use a PR. However, with the switch from Jira to
>>> GitHub issues it might make more sense to use a PR since you have to have
>>> something to track the problem. But if there is already an issue then I
>>> would probably just use that. Again, depending on what is being done.
>>>> 
>>>> I wouldn’t classify any of the work I’ve seen you doing recently as
>>> trivial or minor, so you only doing PRs makes sense.
>>>> 
>>>> Ralph
>>>> 
>>>>> On Sep 17, 2024, at 7:52 AM, Piotr P. Karwasz <
>> piotr.karw...@gmail.com>
>>> wrote:
>>>>> 
>>>>> Hi Ralph,
>>>>> 
>>>>> On Tue, 17 Sept 2024 at 15:47, Ralph Goers <
>> ralph.go...@dslextreme.com>
>>> wrote:
>>>>>> 
>>>>>> Why? i.e. - what currently isn’t working?
>>>>> 
>>>>> I merely wish to formalize what is already happening and set up a
>>>>> branch protection rule to enforce it.
>>>>> 
>>>>> Note that I have never seen a PR in Log4Net being merged without a
>>> review.
>>>>> 
>>>>> On the other hand you can probably find a couple of my recent PRs that
>>>>> don't have a formal review. Sure, I must have certainly consulted with
>>>>> someone on Slack before I merged them, but there is no sign of it.
>>>>> Let's make it formal, so users also see it.
>>>>> 
>>>>> Piotr
>>>> 
>>> 
>>> 
>> 

Re: Use RTC in Log4j and Log4Net

[Ralph Goers]

This isn’t a vote so I am not going to. If I had to vote I wouldn’t vote for a 
policy that requires RTC always. However, I would vote for a policy that 
requires RTC when specified criteria are met.

Ralph

> On Sep 17, 2024, at 10:28 AM, Ralph Goers  wrote:
> 
> First, the obvious. I haven’t committed much in a while. The last several I 
> did I used PRs primarily because it makes it easier for people to review the 
> changes but I didn’t necessarily wait for a review.  For really simple stuff 
> I've never use a PR. However, with the switch from Jira to GitHub issues it 
> might make more sense to use a PR since you have to have something to track 
> the problem. But if there is already an issue then I would probably just use 
> that. Again, depending on what is being done.
> 
> I wouldn’t classify any of the work I’ve seen you doing recently as trivial 
> or minor, so you only doing PRs makes sense. 
> 
> Ralph
> 
>> On Sep 17, 2024, at 7:52 AM, Piotr P. Karwasz  
>> wrote:
>> 
>> Hi Ralph,
>> 
>> On Tue, 17 Sept 2024 at 15:47, Ralph Goers  
>> wrote:
>>> 
>>> Why? i.e. - what currently isn’t working?
>> 
>> I merely wish to formalize what is already happening and set up a
>> branch protection rule to enforce it.
>> 
>> Note that I have never seen a PR in Log4Net being merged without a review.
>> 
>> On the other hand you can probably find a couple of my recent PRs that
>> don't have a formal review. Sure, I must have certainly consulted with
>> someone on Slack before I merged them, but there is no sign of it.
>> Let's make it formal, so users also see it.
>> 
>> Piotr
> 

Re: Use RTC in Log4j and Log4Net

[Ralph Goers]

First, the obvious. I haven’t committed much in a while. The last several I did 
I used PRs primarily because it makes it easier for people to review the 
changes but I didn’t necessarily wait for a review.  For really simple stuff 
I've never use a PR. However, with the switch from Jira to GitHub issues it 
might make more sense to use a PR since you have to have something to track the 
problem. But if there is already an issue then I would probably just use that. 
Again, depending on what is being done.

I wouldn’t classify any of the work I’ve seen you doing recently as trivial or 
minor, so you only doing PRs makes sense. 

Ralph

> On Sep 17, 2024, at 7:52 AM, Piotr P. Karwasz  wrote:
> 
> Hi Ralph,
> 
> On Tue, 17 Sept 2024 at 15:47, Ralph Goers  wrote:
>> 
>> Why? i.e. - what currently isn’t working?
> 
> I merely wish to formalize what is already happening and set up a
> branch protection rule to enforce it.
> 
> Note that I have never seen a PR in Log4Net being merged without a review.
> 
> On the other hand you can probably find a couple of my recent PRs that
> don't have a formal review. Sure, I must have certainly consulted with
> someone on Slack before I merged them, but there is no sign of it.
> Let's make it formal, so users also see it.
> 
> Piotr

Re: Use RTC in Log4j and Log4Net

[Ralph Goers]

Why? i.e. - what currently isn’t working?

Ralph

> On Sep 17, 2024, at 1:30 AM, Piotr P. Karwasz  wrote:
> 
> Hi all,
> 
> Inspired by the way the Log4Net team works, I would like to propose a
> switch from our CTR policy to RTC:
> 
> * every change to the main branches should go through a pull request,
> * pull requests can be merged if they have 1 positive review and no
> requested changes. To allow everyone to look at the PR, let's not
> merge them before at least 24 hours have passed.
> * if a pull request does not receive any review in 72 hours, as a last
> resort we merge them without a review.
> 
> I would like to apply this policy to our most active repos:
> 
> * l-log4j2
> * l-log4j-kotlin
> * l-log4j-scala
> * l-log4j-transform
> * l-log4j-tools
> * l-log4net
> 
> I am NOT proposing this change to the equally active l-log4cxx
> repository, since the Log4cxx team has an established workflow that
> does not use formal reviews.
> 
> I would prefer to explicitly add a branch protection rule that
> requires reviews. If a PR is not reviewed within 72 hours, please ping
> the Slack channel so someone can review it.
> 
> What do you think?
> 
> Piotr

Re: [log4j] Spring Boot 3.4.0 will introduce structured logging

[Ralph Goers]

While this is nice I don’t think it will result in killing off logging 
frameworks. While it supports the MDC it does not support structured messages.

At the moment I don’t think it will support custom ContextDataProviders, but 
that should come for free when I can get log4j-context-data completed.

It also presumes you want all your ThreadContext keys included. We pass the 
OAuth token in the ThreadContextMap and would not what that included as it is 
huge and provides no value.

At the moment to perform any customization you have to write your own 
StructuredLogFormatter. The example they provide isn’t going to perform 
particularly well which makes me wonder what the default implementation does.

Ralph

> On Sep 13, 2024, at 2:07 AM, Volkan Yazıcı  wrote:
> 
> See the related release note
> .
> With this change, Spring Boot will effectively be providing an
> implementation-agnostic logging system abstraction featuring:
> 
>   - Level support
>   - Pattern layout support
>   - Structured (i.e., JSON) layout support (New!)
> 
> Note that many modern SOA deployment solutions
>  expect application logs to
> be written to the console, which renders the need for specialized appenders
> obsolete. Given this, if I may say, Spring Boot's logging abstraction
> pretty much makes the need to employ/configure a logging system obsolete –
> Spring Boot aims to cover 90% of logging-related use cases with its
> in-house abstractions. I can imagine a majority of its users will be able
> to develop and deploy to production without any `log4j2.xml`,
> `logback.xml`, or `logging.properties` configurations.

Re: Preparation of 3.0.0-beta3

[Ralph Goers]

Even with Gary volunteering I don’t see a reason why beta3 would need to wait 
for that.

Ralph

> On Sep 10, 2024, at 8:08 AM, Piotr P. Karwasz  wrote:
> 
> Hi Gary,
> 
> On Tue, 10 Sept 2024 at 14:40, Gary Gregory  wrote:
>> 
>> I can do the File to Path work if everyone is OK with the idea.
> 
> That would be fantastic. I already opened a related issue some time
> ago[1] and dropped some notes there.
> 
> Regarding the advantages of `Path` vs `File`, you probably know them by heart:
> 
> * `Path` has a consistent behavior. If something fails, it throws. No
> need to check the return value of a `File` method.
> * `Path` supports additional filesystems. I have successfully written
> tests that use JimFS instead of the default file system.
> * `Files.newOutputStream()` uses the same options on each platform, so
> no more locked files on Windows that can not be deleted.
> 
> Piotr
> 
> [1] https://github.com/apache/logging-log4j2/issues/2117

Re: [ANNOUNCE] Apache Log4j `2.24.0` released

[Ralph Goers]

Thanks Piotr. Then you just need to change the announcement template to point 
directly to the Downloads page. Of course, it should also continue to have the 
link to the full web site.

Ralph

> On Sep 8, 2024, at 12:38 AM, Piotr P. Karwasz  wrote:
> 
> Hi Ralph,
> 
> On Sun, 8 Sept 2024 at 00:29, Ralph Goers  wrote:
>> However, I either didn’t look carefully enough or the page changed (more 
>> likely the first than the second) as I did not notice that the downloads 
>> page actually does have the specific links for the artifacts. In my first 
>> look I only saw the link to the Logging Services download page.
> 
> The page changed. As I mentioned in the discussion thread[1], I
> incorporated all your suggestions and added them to RC2.
> 
> Piotr
> 
> [1] https://lists.apache.org/thread/l3p4dd5kxjx676glplxfhn9849xg9vkv

Re: [ANNOUNCE] Apache Log4j `2.24.0` released

[Ralph Goers]

Literally the only requirement for the announcement is that it contains a 
direct link to where the source archive can be downloaded.

Personally, I like to contain a synopsis of the release (bug fix, minor 
enhancements, etc) with 2 or 3 significant items if there are any and then 
provide a link to the release notes.

Ralph

> On Sep 6, 2024, at 11:49 PM, Piotr P. Karwasz  wrote:
> 
> Hi Ralph,
> 
> On Sat, 7 Sept 2024 at 07:00, Ralph Goers  wrote:
>> 
>> As I said previously, you should expect this will NOT get moderated through 
>> the ASF announce list.
> 
> The e-mail has the same format as the one I sent for 2.23.0[1] and is
> very similar to the one for 3.0.0-alpha1[2].
> 
> Looking at other announcement emails, we could probably modify ours by:
> 
> * Stripping them of the release notes, except the short intro we write
> manually. The entire list of changes can probably be replaced with a
> link to the release notes.
> * Adding the download and release notes links explicitly.
> 
> What do you think?
> 
> Piotr
> 
> [1] https://lists.apache.org/thread/8fycgpb2k274bc4y8gp1zswy5jpzc2jh
> [2] https://lists.apache.org/thread/z3otp9jr9bpo0r732k02g1mdlfkjb3gm

Re: [VOTE] Release Apache Log4j `2.24.0`

[Ralph Goers]

Personally, I would appreciate a document I can read that documents the build 
process. Even though I am sure it is all Maven plugins of some kind it isn’t 
unusual for the order and configuration of the plugins to be important.

I’d really like to know what I need to do to get it to work for some existing 
project or a brand new one.

Ralph

> On Sep 4, 2024, at 3:56 AM, Christian Grobmeier  wrote:
> 
> 
> On Tue, Sep 3, 2024, at 21:51, Gary Gregory wrote:
>> From my point of view, building has gotten worse and less reliable over
>> time :-(
> 
> To be fair, the new build system was, at least for me, welcoming and 
> straightforward to use. Although complex under the hood, it’s a major 
> improvement over the build system we had in log4j1 (like Stone Age to 
> enterprise) and also lots better than the various documents with different 
> content we had with early log4j2
> 
> I am not sure why it did not build on your machine but on the others, but it 
> is certainly worth to improve the system we have today. 
> 
> I see a trend within this team that only few people push forward changes like 
> maintaining the build. I think it would be beneficial if others would learn 
> more about how it works and the details so everyone can provide patches.
> 
> Volkan already offered to help. If there is more interest we maybe could have 
> video training session on the details, assuming Volkan would be willing to do 
> this as well. I would be certainly interested in this.
> 
> Cheers 
> 
>> My -1 vote reflects this.
>> 
>> Gary
>> 
>> On Tue, Sep 3, 2024, 2:55 PM Volkan Yazıcı > > wrote:
>> 
>>> Gary, do you know what is the difference between RC1 and RC2? Nothing.
>>> Piotr only kindly added a one-liner condition check to the contending
>>> (FQDN-related) test to make it up to you. That is the only difference –
>>> plus, he updated the review kit (shared in the email) to avoid the
>>> reproducibility check on Windows. Put another way, RC1 is effectively
>>> identical to RC2, bit by bit.
>>> 
>>> My point is, 3 people verified the release and CI runs passed on all 3
>>> platforms – there is definitely something unexpected in your setup. As you
>>> know better, issuing an RC is a time and energy consuming task. Besides RM,
>>> other voters put effort into it too. Would you mind asking for further help
>>> instead of downvoting a release due to local failures, please? I would have
>>> been more than happy to assist you in a video call, instead of re-issuing
>>> the whole release.
>>> 
>>> On Tue, Sep 3, 2024 at 4:04 PM Gary D. Gregory 
>>> wrote:
>>> 
 -1
 
 On Windows, I deleting my entire .m2/repository folder and then ran
 
 mvnw -Prelease clean verify artifact:compare
>>> -Dreference.repo=%NEXUS_REPO%
 
 and got:
 
 [INFO] Minimal buildinfo generated from downloaded artifacts:
 
>>> C:\Users\ggregory\rc\2.24.0\src\target\reference\log4j-bom-2.24.0.buildinfo
 [ERROR] size mismatch log4j-bom-2.24.0.pom: investigate with diffoscope
 target\reference\org.apache.logging.log4j\log4j-bom-2.24.0.pom
 .flattened-pom.xml
 [ERROR] size mismatch log4j-bom-2.24.0-cyclonedx.xml: investigate with
 diffoscope
 target\reference\org.apache.logging.log4j\log4j-bom-2.24.0-cyclonedx.xml
 target\bom.xml
 [ERROR] Reproducible Build output summary: 0 files ok, 2 different
 [ERROR] see diff target\reference\log4j-bom-2.24.0.buildinfo
 target\log4j-bom-2.24.0.buildinfo
 [ERROR] see also
 https://maven.apache.org/guides/mini/guide-reproducible-builds.html
 [INFO] Reproducible Build output comparison saved to
 C:\Users\ggregory\rc\2.24.0\src\target\log4j-bom-2.24.0.buildcompare
 [INFO] Aggregate buildcompare copied to
 C:\Users\ggregory\rc\2.24.0\src\target\log4j-bom-2.24.0.buildcompare
 [INFO]
 
 [INFO] Reactor Summary for Apache Log4j BOM 2.24.0:
 [INFO]
 [INFO] Apache Log4j BOM ... FAILURE
>>> [02:58
 min]
 [
 
 So I give up after trying macOS, Linux, and Windows.
 
 Gary
 
 On 2024/09/03 13:23:59 "Gary D. Gregory" wrote:
> Note that I add "clean" *(why does the kit not use "clean"?)
> 
> mvnw -Prelease clean verify artifact:compare
>>> -Dreference.repo=$NEXUS_REPO
> 
> Gary
> 
> On 2024/09/03 13:21:32 "Gary D. Gregory" wrote:
>> It's fails differently on Ubuntu:
>> 
>> ...
>> [INFO] --- artifact:3.5.1:compare (default-cli) @ log4j-api ---
>> [WARNING]  property is inherited from
 outside the reactor, it should be defined in parent POM from reactor
 /mnt/c/Users/ggregory/rc/2.24.0/src/.flattened-pom.xml
>> [INFO] Reference buildinfo file not found: it will be generated from
 downloaded reference artifacts
>> [INFO] Reference build java.version: 17 (from MANIFEST.MF
 Build-Jdk-Spec)
>>

Re: [DISCUSS][VOTE] Release Apache Log4j `2.24.0`

[Ralph Goers]

Thanks Piotr. It is good to know this as I often have lots of source code and 
build from various things on my computer.

Ralph

> On Sep 3, 2024, at 12:45 PM, Piotr P. Karwasz  wrote:
> 
> Hi Volkan,
> 
> On Tue, 3 Sept 2024 at 20:54, Volkan Yazıcı  wrote:
>> My point is, 3 people verified the release and CI runs passed on all 3
>> platforms – there is definitely something unexpected in your setup. As you
>> know better, issuing an RC is a time and energy consuming task. Besides RM,
>> other voters put effort into it too. Would you mind asking for further help
>> instead of downvoting a release due to local failures, please? I would have
>> been more than happy to assist you in a video call, instead of re-issuing
>> the whole release.
> 
> The call to cancel the vote was mine, since the -1 is not a veto.
> 
> That said, please consider that:
> 
> * Around 10% of all builds fail due to a test. You can find a list of
> broken tests on Develocity[1] and try to fix them.
> * The SBOM is generated based on the dependencies in your local Maven
> repo. If you happen to be the Release Manager of some of Log4j
> dependencies, you might have some pre-releases and RCs in the repo
> instead of the official versions.
> 
> Piotr
> 
> [1] 
> https://ge.apache.org/scans/tests?search.relativeStartTime=P28D&search.rootProjectNames=Apache%20Log4j%20BOM&search.timeZoneId=Europe%2FWarsaw#

Re: (logging-parent) branch main updated: Modify review kit

[Ralph Goers]

? Clean takes a no time. It ensures no junk is lying around.

Ralph

> On Sep 3, 2024, at 10:58 AM, Volkan Yazıcı  wrote:
> 
> I will remove `clean`, since there is nothing to clean.
> We shouldn't add unnecessary overhead.
> 
> On Tue, Sep 3, 2024 at 7:01 PM Piotr P. Karwasz 
> wrote:
> 
>> Hi Gary,
>> 
>> On Tue, 3 Sept 2024 at 18:25, Gary Gregory  wrote:
>>> 
>>> Why doesn't the maven build in the review kit (below) invoke clean goal?
>>> 
 +sh mvnw -Prelease clean verify artifact:compare
 -Dreference.repo=$NEXUS_REPO
 +
 +# Run tests only (Windows)
 +sh mvnw -Prelease clean verify artifact:compare
 \ No newline at end of file
>> 
>> Normally there is nothing to clean, but as you can see I added `clean`
>> to both UNIX and Windows.
>> 
>> Piotr
>> 

Re: [VOTE] Release Apache Log4j `2.24.0`

[Ralph Goers]

Volkan,

I think Gary told you.  He has the right to complain and vote on a release as 
he feels. Although I believe it is wrong to do so, I could have voted -1 on the 
release do to the release notes and web site issues. With 3 +1 votes Piotr 
always had the option to go ahead with the release anyway if he deemed Gary’s 
complaints to be minor. Please remember a -1 on a release is NOT a veto.

Ralph

> On Sep 3, 2024, at 1:07 PM, Volkan Yazıcı  wrote:
> 
>> You're not helping the cause.
> 
> I am offering a video call to assist you. What else do you want me to do?
> 
> 
> On Tue, Sep 3, 2024 at 9:51 PM Gary Gregory  wrote:
> 
>> Volcan,
>> 
>> Please stop complaining about the hours I've already sunk into validation
>> on 3 different operating system on two different machines. You're not
>> helping the cause.
>> 
>> I can't help but notice the irony that one of the failures is in the
>> "reproducible" part of the build.
>> 
>> From my point of view, building has gotten worse and less reliable over
>> time :-(
>> 
>> My -1 vote reflects this.
>> 
>> Gary
>> 
>> On Tue, Sep 3, 2024, 2:55 PM Volkan Yazıcı  wrote:
>> 
>>> Gary, do you know what is the difference between RC1 and RC2? Nothing.
>>> Piotr only kindly added a one-liner condition check to the contending
>>> (FQDN-related) test to make it up to you. That is the only difference –
>>> plus, he updated the review kit (shared in the email) to avoid the
>>> reproducibility check on Windows. Put another way, RC1 is effectively
>>> identical to RC2, bit by bit.
>>> 
>>> My point is, 3 people verified the release and CI runs passed on all 3
>>> platforms – there is definitely something unexpected in your setup. As
>> you
>>> know better, issuing an RC is a time and energy consuming task. Besides
>> RM,
>>> other voters put effort into it too. Would you mind asking for further
>> help
>>> instead of downvoting a release due to local failures, please? I would
>> have
>>> been more than happy to assist you in a video call, instead of re-issuing
>>> the whole release.
>>> 
>>> On Tue, Sep 3, 2024 at 4:04 PM Gary D. Gregory 
>>> wrote:
>>> 
 -1
 
 On Windows, I deleting my entire .m2/repository folder and then ran
 
 mvnw -Prelease clean verify artifact:compare
>>> -Dreference.repo=%NEXUS_REPO%
 
 and got:
 
 [INFO] Minimal buildinfo generated from downloaded artifacts:
 
>>> 
>> C:\Users\ggregory\rc\2.24.0\src\target\reference\log4j-bom-2.24.0.buildinfo
 [ERROR] size mismatch log4j-bom-2.24.0.pom: investigate with diffoscope
 target\reference\org.apache.logging.log4j\log4j-bom-2.24.0.pom
 .flattened-pom.xml
 [ERROR] size mismatch log4j-bom-2.24.0-cyclonedx.xml: investigate with
 diffoscope
 
>> target\reference\org.apache.logging.log4j\log4j-bom-2.24.0-cyclonedx.xml
 target\bom.xml
 [ERROR] Reproducible Build output summary: 0 files ok, 2 different
 [ERROR] see diff target\reference\log4j-bom-2.24.0.buildinfo
 target\log4j-bom-2.24.0.buildinfo
 [ERROR] see also
 https://maven.apache.org/guides/mini/guide-reproducible-builds.html
 [INFO] Reproducible Build output comparison saved to
 C:\Users\ggregory\rc\2.24.0\src\target\log4j-bom-2.24.0.buildcompare
 [INFO] Aggregate buildcompare copied to
 C:\Users\ggregory\rc\2.24.0\src\target\log4j-bom-2.24.0.buildcompare
 [INFO]
 
>> 
 [INFO] Reactor Summary for Apache Log4j BOM 2.24.0:
 [INFO]
 [INFO] Apache Log4j BOM ... FAILURE
>>> [02:58
 min]
 [
 
 So I give up after trying macOS, Linux, and Windows.
 
 Gary
 
 On 2024/09/03 13:23:59 "Gary D. Gregory" wrote:
> Note that I add "clean" *(why does the kit not use "clean"?)
> 
> mvnw -Prelease clean verify artifact:compare
>>> -Dreference.repo=$NEXUS_REPO
> 
> Gary
> 
> On 2024/09/03 13:21:32 "Gary D. Gregory" wrote:
>> It's fails differently on Ubuntu:
>> 
>> ...
>> [INFO] --- artifact:3.5.1:compare (default-cli) @ log4j-api ---
>> [WARNING]  property is inherited
>> from
 outside the reactor, it should be defined in parent POM from reactor
 /mnt/c/Users/ggregory/rc/2.24.0/src/.flattened-pom.xml
>> [INFO] Reference buildinfo file not found: it will be generated
>> from
 downloaded reference artifacts
>> [INFO] Reference build java.version: 17 (from MANIFEST.MF
 Build-Jdk-Spec)
>> [INFO] Reference build os.name: Unix (from pom.properties newline)
>> [INFO] Minimal buildinfo generated from downloaded artifacts:
 
>>> 
>> /mnt/c/Users/ggregory/rc/2.24.0/src/log4j-api/target/reference/log4j-api-2.24.0.buildinfo
>> [ERROR] sha512 mismatch log4j-api-2.24.0-sources.jar: investigate
>>> with
 diffoscope
 
>>> 
>> log4j-api/target/reference/org.apache.logging.log4j/log4j-api-2.24.0-sources.jar
 log4j-api/target/log4j-api-2.24.

Re: [VOTE] Release Apache Log4j `2.24.0`

[Ralph Goers]

+1

Verified the signatures and checksums

I rann the build twice with both failing but each failing differently.

My machine info

Apache Maven 3.8.6 (84538c9988a25aec085021c365c560670ad80f63)
Maven home: /opt/maven/maven
Java version: 17.0.6, vendor: Eclipse Adoptium, runtime: 
/Library/Java/JavaVirtualMachines/temurin-17.jdk/Contents/Home
Default locale: en_US, platform encoding: UTF-8
OS name: "mac os x", version: "14.6.1", arch: "x86_64", family: “mac"


However, this machine is actually an Apple M2 Max with 96GB memory. 

Run 1

[INFO] Results:
[INFO] 
[ERROR] Failures: 
[ERROR]   HttpAppenderTest.testAppendCustomHeader:254 Expected at least one 
request matching: {
  "url" : "/test/log4j/",
  "method" : "POST",
  "headers" : {
"Host" : {
  "contains" : "localhost"
},
"X-Test" : {
  "equalTo" : "header value"
},
"X-Runtime" : {
  "equalTo" : "OpenJDK Runtime Environment (build 17.0.6+10) from Eclipse 
Adoptium"
},
"Content-Type" : {
  "contains" : "application/json"
}
  },
  "bodyPatterns" : [ {
"contains" : "\"message\" : \"Hello, world!\""
  } ]
}
Requests received: [ ]
[ERROR] Errors: 
[ERROR]   JeroMqAppenderTest.testMultiThreadedServer:155 » ConditionTimeout 
Condition with Lambda expression in 
org.apache.logging.log4j.core.appender.mom.jeromq.JeroMqAppenderTest was not 
fulfilled within 5 seconds.
[INFO] 
[ERROR] Tests run: 2253, Failures: 1, Errors: 1, Skipped: 31


Run 2

[INFO] Results:
[INFO] 
[ERROR] Failures: 
[ERROR]   RollingAppenderDirectCronTest.testAppender(LoggerContext, 
RollingFileAppender) 
Expecting actual not to be empty
[INFO] 
[ERROR] Tests run: 2253, Failures: 1, Errors: 0, Skipped: 31

I installed the latest Oracle Java 17 for Apple architecture and got.

Apache Maven 3.8.6 (84538c9988a25aec085021c365c560670ad80f63)
Maven home: /opt/maven/maven
Java version: 17.0.11, vendor: Oracle Corporation, runtime: 
/Library/Java/JavaVirtualMachines/jdk-17.0.11.jdk/Contents/Home
Default locale: en_US, platform encoding: UTF-8
OS name: "mac os x", version: "14.6.1", arch: "aarch64", family: "mac"


Succeeded twice.  Go figure.

Ralph


> On Aug 31, 2024, at 12:30 PM, Piotr P. Karwasz  
> wrote:
> 
> This is a vote to release the Apache Log4j `2.24.0`.
> 
> Website: https://logging.staged.apache.org/log4j/2.24.0/index.html
> GitHub: https://github.com/apache/logging-log4j2
> Commit: 08053687456f6be61ee8206da782a3d051928a57
> Distribution: https://dist.apache.org/repos/dist/dev/logging/log4j
> Nexus: 
> https://repository.apache.org/content/repositories/orgapachelogging-1293
> Signing key: 0x077e8893a6dcc33dd4a4d5b256e73ba9a0b592d0
> 
> Please download, test, and cast your votes on this mailing list.
> 
> [ ] +1, release the artifacts
> [ ] -1, don't release, because...
> 
> This vote is open for 72 hours and will pass unless getting a
> net negative vote count. All votes are welcome and we encourage
> everyone to test the release, but only the Logging Services PMC
> votes are officially counted. At least 3 +1 votes and more
> positive than negative votes are required.
> 
> == Review kit
> 
> The minimum set of steps needed to review the uploaded distribution
> files in the Subversion repository can be summarized as follows:
> 
># Check out the distribution
>svn co https://dist.apache.org/repos/dist/dev/logging/log4j/2.24.0 && cd $_
> 
># Verify checksums
>shasum --check *.sha512
> 
># Verify signatures
>wget -O - https://downloads.apache.org/logging/KEYS | gpg --import
>for sigFile in *.asc; do gpg --verify $sigFile; done
> 
># Verify reproduciblity
>umask 0022
>unzip *-src.zip -d src
>cd src
>export 
> NEXUS_REPO=https://repository.apache.org/content/repositories/orgapachelogging-1293
>sh mvnw -Prelease verify artifact:compare -Dreference.repo=$NEXUS_REPO
># If preferred, augment `mvnw` with `-DskipTests` to speed things up
> 
> == Release Notes
> 
> This release contains improvements and changes in several areas of Apache 
> Log4j:
> 
> === Log4j API
> 
> The `2.24.0` version of Log4j API has been enhanced with changes from
> the 3.x branch and will be used by both Log4j 2 Core and Log4j 3 Core
> releases.
> The changes include:
> 
> * A faster default `ThreadContextMap`.
> * Enhanced GraalVM support: native binaries that use Log4j API will no
> longer require additional GraalVM configuration.
> * The configuration properties subsystem now only accepts the official
> pre-2.10 property names and the normalized post-2.10 names.
> Check your configuration for typos.
> 
> === Documentation
> 
> The Apache Log4j 2 website has been almost entirely rewritten to
> provide improved documentation and faster access to the information
> you need.
> 
> [1] https://logging.staged.apache.org/log4j/2.24.0/index.html
> 
> === Bridges
> 
> The JUL-to-Log4j API and Log4j 1-to-Log4j API will no longer be able
> to modify the configuration of Log4j Core by default.
> If such a functionality is requi

Re: [Discuss][VOTE] Release Apache Log4j `2.24.0`

[Ralph Goers]

Just looked at apache-log4j-2.24.0-email-announce.txt in the dist directory.  
(I am not actually sure what that is there to be honest). The email does NOT 
include a download link so will definitely be rejected.

Ralph

> On Aug 31, 2024, at 1:55 PM, Ralph Goers  wrote:
> 
> I looked at the download page and it does NOT meet the requirements for a 
> release announcement:
> "Announcements must contain a link to the relevant download page for the 
> source.”  The download page contains a link to a generic directory listing of 
> all Logging Services releases.
> 
> You can quibble here about whether what is being done meets the requirements, 
> but it doesn’t matter. All announcements get moderated and Sebb will check 
> this and if he doesn’t like it he will reject the announcement email.
> 
> Note that the web site is NOT officially part of the release so I am NOT 
> going to vote -1 due to any web site issues. Just be aware you are likely to 
> have problems announcing the release.
> 
> Ralph
> 
> 
>> On Aug 31, 2024, at 12:30 PM, Piotr P. Karwasz  
>> wrote:
>> 
>> This is a vote to release the Apache Log4j `2.24.0`.
>> 
>> Website: https://logging.staged.apache.org/log4j/2.24.0/index.html
>> GitHub: https://github.com/apache/logging-log4j2
>> Commit: 08053687456f6be61ee8206da782a3d051928a57
>> Distribution: https://dist.apache.org/repos/dist/dev/logging/log4j
>> Nexus: 
>> https://repository.apache.org/content/repositories/orgapachelogging-1293
>> Signing key: 0x077e8893a6dcc33dd4a4d5b256e73ba9a0b592d0
>> 
>> Please download, test, and cast your votes on this mailing list.
>> 
>> [ ] +1, release the artifacts
>> [ ] -1, don't release, because...
>> 
>> This vote is open for 72 hours and will pass unless getting a
>> net negative vote count. All votes are welcome and we encourage
>> everyone to test the release, but only the Logging Services PMC
>> votes are officially counted. At least 3 +1 votes and more
>> positive than negative votes are required.
>> 
>> == Review kit
>> 
>> The minimum set of steps needed to review the uploaded distribution
>> files in the Subversion repository can be summarized as follows:
>> 
>>   # Check out the distribution
>>   svn co https://dist.apache.org/repos/dist/dev/logging/log4j/2.24.0 && cd $_
>> 
>>   # Verify checksums
>>   shasum --check *.sha512
>> 
>>   # Verify signatures
>>   wget -O - https://downloads.apache.org/logging/KEYS | gpg --import
>>   for sigFile in *.asc; do gpg --verify $sigFile; done
>> 
>>   # Verify reproduciblity
>>   umask 0022
>>   unzip *-src.zip -d src
>>   cd src
>>   export 
>> NEXUS_REPO=https://repository.apache.org/content/repositories/orgapachelogging-1293
>>   sh mvnw -Prelease verify artifact:compare -Dreference.repo=$NEXUS_REPO
>>   # If preferred, augment `mvnw` with `-DskipTests` to speed things up
>> 
>> == Release Notes
>> 
>> This release contains improvements and changes in several areas of Apache 
>> Log4j:
>> 
>> === Log4j API
>> 
>> The `2.24.0` version of Log4j API has been enhanced with changes from
>> the 3.x branch and will be used by both Log4j 2 Core and Log4j 3 Core
>> releases.
>> The changes include:
>> 
>> * A faster default `ThreadContextMap`.
>> * Enhanced GraalVM support: native binaries that use Log4j API will no
>> longer require additional GraalVM configuration.
>> * The configuration properties subsystem now only accepts the official
>> pre-2.10 property names and the normalized post-2.10 names.
>> Check your configuration for typos.
>> 
>> === Documentation
>> 
>> The Apache Log4j 2 website has been almost entirely rewritten to
>> provide improved documentation and faster access to the information
>> you need.
>> 
>> [1] https://logging.staged.apache.org/log4j/2.24.0/index.html
>> 
>> === Bridges
>> 
>> The JUL-to-Log4j API and Log4j 1-to-Log4j API will no longer be able
>> to modify the configuration of Log4j Core by default.
>> If such a functionality is required, it must be explicitly enabled.
>> 
>> === Modules
>> 
>> The following Log4j Core additional modules have been removed:
>> 
>> `log4j-flume-ng`::
>> The module has been moved to the Flume project and follows the Apache
>> Flume release lifecycle.
>> 
>> `log4j-kubernetes`::
>> The module has been moved to the
>> https://github.com/fabric8io/kubernetes-client/blob/main/doc/KubernetesLog4j.md[Fabric8.io
>> Kubernetes p

Re: [Discuss][VOTE] Release Apache Log4j `2.24.0`

[Ralph Goers]

No. ScopedContext would have been new in 2.24.0.  Piotr didn’t wanted it 
included in 2.24.0 as we agreed that all the private crap for the ThreadContext 
and ScopedContext and other Context related stuff should be moved to a new 
module. In the meantime he was to move the private ThreadContext stuff into 
Log4j-core. I haven’t actually checked the release yet but I am assuming that 
happened.

Ralph

> On Aug 31, 2024, at 2:21 PM, Gary Gregory  wrote:
> 
> I've not looked at details but Ralph's comment hints that we need to
> explain to users how to migrate if it's anything more than an import
> change. WDYT?
> 
> Gary
> 
> On Sat, Aug 31, 2024, 5:06 PM Ralph Goers 
> wrote:
> 
>> The API section has
>> 
>> Thread Context is mostly superseded by Scoped Context, which, unlike
>> Thread Context,
>>• is safe to use in servlet applications
>>• can store any Object-typed value
>> 
>> 
>> While I totally agree with this I was under the impression that
>> ScopedContext was removed from Log4j 2.24.0, so this will lead to user
>> questions.
>> 
>> Ralph
>> 
>>> On Aug 31, 2024, at 2:00 PM, Ralph Goers 
>> wrote:
>>> 
>>> The release notes page says:
>>> 
>>> "log4j-flume-ngT
>>> The module has been moved to the Flume project and follows the Apache
>> Flume release lifecycle.
>>> 
>>> We NEVER discussed this. We simply discussed moving it to another repo.
>> As in 3.0.0, where more modules are split out, I believe it would be more
>> appropriate to say that they will have their own release cycles.
>>> 
>>> Ralph
>>> 
>>>> On Aug 31, 2024, at 1:55 PM, Ralph Goers 
>> wrote:
>>>> 
>>>> I looked at the download page and it does NOT meet the requirements for
>> a release announcement:
>>>> "Announcements must contain a link to the relevant download page for
>> the source.”  The download page contains a link to a generic directory
>> listing of all Logging Services releases.
>>>> 
>>>> You can quibble here about whether what is being done meets the
>> requirements, but it doesn’t matter. All announcements get moderated and
>> Sebb will check this and if he doesn’t like it he will reject the
>> announcement email.
>>>> 
>>>> Note that the web site is NOT officially part of the release so I am
>> NOT going to vote -1 due to any web site issues. Just be aware you are
>> likely to have problems announcing the release.
>>>> 
>>>> Ralph
>>>> 
>>>> 
>>>>> On Aug 31, 2024, at 12:30 PM, Piotr P. Karwasz <
>> piotr.karw...@gmail.com> wrote:
>>>>> 
>>>>> This is a vote to release the Apache Log4j `2.24.0`.
>>>>> 
>>>>> Website: https://logging.staged.apache.org/log4j/2.24.0/index.html
>>>>> GitHub: https://github.com/apache/logging-log4j2
>>>>> Commit: 08053687456f6be61ee8206da782a3d051928a57
>>>>> Distribution: https://dist.apache.org/repos/dist/dev/logging/log4j
>>>>> Nexus:
>> https://repository.apache.org/content/repositories/orgapachelogging-1293
>>>>> Signing key: 0x077e8893a6dcc33dd4a4d5b256e73ba9a0b592d0
>>>>> 
>>>>> Please download, test, and cast your votes on this mailing list.
>>>>> 
>>>>> [ ] +1, release the artifacts
>>>>> [ ] -1, don't release, because...
>>>>> 
>>>>> This vote is open for 72 hours and will pass unless getting a
>>>>> net negative vote count. All votes are welcome and we encourage
>>>>> everyone to test the release, but only the Logging Services PMC
>>>>> votes are officially counted. At least 3 +1 votes and more
>>>>> positive than negative votes are required.
>>>>> 
>>>>> == Review kit
>>>>> 
>>>>> The minimum set of steps needed to review the uploaded distribution
>>>>> files in the Subversion repository can be summarized as follows:
>>>>> 
>>>>> # Check out the distribution
>>>>> svn co https://dist.apache.org/repos/dist/dev/logging/log4j/2.24.0
>> && cd $_
>>>>> 
>>>>> # Verify checksums
>>>>> shasum --check *.sha512
>>>>> 
>>>>> # Verify signatures
>>>>> wget -O - https://downloads.apache.org/logging/KEYS | gpg --import
>>>>> for sigFile in *.asc; do gpg --verify

Re: [Discuss][VOTE] Release Apache Log4j `2.24.0`

[Ralph Goers]

The API section has 

Thread Context is mostly superseded by Scoped Context, which, unlike Thread 
Context,
• is safe to use in servlet applications
• can store any Object-typed value


While I totally agree with this I was under the impression that ScopedContext 
was removed from Log4j 2.24.0, so this will lead to user questions.

Ralph

> On Aug 31, 2024, at 2:00 PM, Ralph Goers  wrote:
> 
> The release notes page says:
> 
> "log4j-flume-ngT
> The module has been moved to the Flume project and follows the Apache Flume 
> release lifecycle.
> 
> We NEVER discussed this. We simply discussed moving it to another repo. As in 
> 3.0.0, where more modules are split out, I believe it would be more 
> appropriate to say that they will have their own release cycles.
> 
> Ralph
> 
>> On Aug 31, 2024, at 1:55 PM, Ralph Goers  wrote:
>> 
>> I looked at the download page and it does NOT meet the requirements for a 
>> release announcement:
>> "Announcements must contain a link to the relevant download page for the 
>> source.”  The download page contains a link to a generic directory listing 
>> of all Logging Services releases.
>> 
>> You can quibble here about whether what is being done meets the 
>> requirements, but it doesn’t matter. All announcements get moderated and 
>> Sebb will check this and if he doesn’t like it he will reject the 
>> announcement email.
>> 
>> Note that the web site is NOT officially part of the release so I am NOT 
>> going to vote -1 due to any web site issues. Just be aware you are likely to 
>> have problems announcing the release.
>> 
>> Ralph
>> 
>> 
>>> On Aug 31, 2024, at 12:30 PM, Piotr P. Karwasz  
>>> wrote:
>>> 
>>> This is a vote to release the Apache Log4j `2.24.0`.
>>> 
>>> Website: https://logging.staged.apache.org/log4j/2.24.0/index.html
>>> GitHub: https://github.com/apache/logging-log4j2
>>> Commit: 08053687456f6be61ee8206da782a3d051928a57
>>> Distribution: https://dist.apache.org/repos/dist/dev/logging/log4j
>>> Nexus: 
>>> https://repository.apache.org/content/repositories/orgapachelogging-1293
>>> Signing key: 0x077e8893a6dcc33dd4a4d5b256e73ba9a0b592d0
>>> 
>>> Please download, test, and cast your votes on this mailing list.
>>> 
>>> [ ] +1, release the artifacts
>>> [ ] -1, don't release, because...
>>> 
>>> This vote is open for 72 hours and will pass unless getting a
>>> net negative vote count. All votes are welcome and we encourage
>>> everyone to test the release, but only the Logging Services PMC
>>> votes are officially counted. At least 3 +1 votes and more
>>> positive than negative votes are required.
>>> 
>>> == Review kit
>>> 
>>> The minimum set of steps needed to review the uploaded distribution
>>> files in the Subversion repository can be summarized as follows:
>>> 
>>>  # Check out the distribution
>>>  svn co https://dist.apache.org/repos/dist/dev/logging/log4j/2.24.0 && cd $_
>>> 
>>>  # Verify checksums
>>>  shasum --check *.sha512
>>> 
>>>  # Verify signatures
>>>  wget -O - https://downloads.apache.org/logging/KEYS | gpg --import
>>>  for sigFile in *.asc; do gpg --verify $sigFile; done
>>> 
>>>  # Verify reproduciblity
>>>  umask 0022
>>>  unzip *-src.zip -d src
>>>  cd src
>>>  export 
>>> NEXUS_REPO=https://repository.apache.org/content/repositories/orgapachelogging-1293
>>>  sh mvnw -Prelease verify artifact:compare -Dreference.repo=$NEXUS_REPO
>>>  # If preferred, augment `mvnw` with `-DskipTests` to speed things up
>>> 
>>> == Release Notes
>>> 
>>> This release contains improvements and changes in several areas of Apache 
>>> Log4j:
>>> 
>>> === Log4j API
>>> 
>>> The `2.24.0` version of Log4j API has been enhanced with changes from
>>> the 3.x branch and will be used by both Log4j 2 Core and Log4j 3 Core
>>> releases.
>>> The changes include:
>>> 
>>> * A faster default `ThreadContextMap`.
>>> * Enhanced GraalVM support: native binaries that use Log4j API will no
>>> longer require additional GraalVM configuration.
>>> * The configuration properties subsystem now only accepts the official
>>> pre-2.10 property names and the normalized post-2.10 names.
>>> Check your configuration for typos.
>>> 
>>> === Documentation
>>> 
>

Re: [Discuss][VOTE] Release Apache Log4j `2.24.0`

[Ralph Goers]

The release notes page says:

"log4j-flume-ngT
The module has been moved to the Flume project and follows the Apache 
Flume release lifecycle.

We NEVER discussed this. We simply discussed moving it to another repo. As in 
3.0.0, where more modules are split out, I believe it would be more appropriate 
to say that they will have their own release cycles.

Ralph

> On Aug 31, 2024, at 1:55 PM, Ralph Goers  wrote:
> 
> I looked at the download page and it does NOT meet the requirements for a 
> release announcement:
> "Announcements must contain a link to the relevant download page for the 
> source.”  The download page contains a link to a generic directory listing of 
> all Logging Services releases.
> 
> You can quibble here about whether what is being done meets the requirements, 
> but it doesn’t matter. All announcements get moderated and Sebb will check 
> this and if he doesn’t like it he will reject the announcement email.
> 
> Note that the web site is NOT officially part of the release so I am NOT 
> going to vote -1 due to any web site issues. Just be aware you are likely to 
> have problems announcing the release.
> 
> Ralph
> 
> 
>> On Aug 31, 2024, at 12:30 PM, Piotr P. Karwasz  
>> wrote:
>> 
>> This is a vote to release the Apache Log4j `2.24.0`.
>> 
>> Website: https://logging.staged.apache.org/log4j/2.24.0/index.html
>> GitHub: https://github.com/apache/logging-log4j2
>> Commit: 08053687456f6be61ee8206da782a3d051928a57
>> Distribution: https://dist.apache.org/repos/dist/dev/logging/log4j
>> Nexus: 
>> https://repository.apache.org/content/repositories/orgapachelogging-1293
>> Signing key: 0x077e8893a6dcc33dd4a4d5b256e73ba9a0b592d0
>> 
>> Please download, test, and cast your votes on this mailing list.
>> 
>> [ ] +1, release the artifacts
>> [ ] -1, don't release, because...
>> 
>> This vote is open for 72 hours and will pass unless getting a
>> net negative vote count. All votes are welcome and we encourage
>> everyone to test the release, but only the Logging Services PMC
>> votes are officially counted. At least 3 +1 votes and more
>> positive than negative votes are required.
>> 
>> == Review kit
>> 
>> The minimum set of steps needed to review the uploaded distribution
>> files in the Subversion repository can be summarized as follows:
>> 
>>   # Check out the distribution
>>   svn co https://dist.apache.org/repos/dist/dev/logging/log4j/2.24.0 && cd $_
>> 
>>   # Verify checksums
>>   shasum --check *.sha512
>> 
>>   # Verify signatures
>>   wget -O - https://downloads.apache.org/logging/KEYS | gpg --import
>>   for sigFile in *.asc; do gpg --verify $sigFile; done
>> 
>>   # Verify reproduciblity
>>   umask 0022
>>   unzip *-src.zip -d src
>>   cd src
>>   export 
>> NEXUS_REPO=https://repository.apache.org/content/repositories/orgapachelogging-1293
>>   sh mvnw -Prelease verify artifact:compare -Dreference.repo=$NEXUS_REPO
>>   # If preferred, augment `mvnw` with `-DskipTests` to speed things up
>> 
>> == Release Notes
>> 
>> This release contains improvements and changes in several areas of Apache 
>> Log4j:
>> 
>> === Log4j API
>> 
>> The `2.24.0` version of Log4j API has been enhanced with changes from
>> the 3.x branch and will be used by both Log4j 2 Core and Log4j 3 Core
>> releases.
>> The changes include:
>> 
>> * A faster default `ThreadContextMap`.
>> * Enhanced GraalVM support: native binaries that use Log4j API will no
>> longer require additional GraalVM configuration.
>> * The configuration properties subsystem now only accepts the official
>> pre-2.10 property names and the normalized post-2.10 names.
>> Check your configuration for typos.
>> 
>> === Documentation
>> 
>> The Apache Log4j 2 website has been almost entirely rewritten to
>> provide improved documentation and faster access to the information
>> you need.
>> 
>> [1] https://logging.staged.apache.org/log4j/2.24.0/index.html
>> 
>> === Bridges
>> 
>> The JUL-to-Log4j API and Log4j 1-to-Log4j API will no longer be able
>> to modify the configuration of Log4j Core by default.
>> If such a functionality is required, it must be explicitly enabled.
>> 
>> === Modules
>> 
>> The following Log4j Core additional modules have been removed:
>> 
>> `log4j-flume-ng`::
>> The module has been moved to the Flume project and follows the Apache
>> Flume release lifecycle.
>> 
>> `log4j-ku

[Discuss][VOTE] Release Apache Log4j `2.24.0`

[Ralph Goers]

I looked at the download page and it does NOT meet the requirements for a 
release announcement:
"Announcements must contain a link to the relevant download page for the 
source.”  The download page contains a link to a generic directory listing of 
all Logging Services releases.

You can quibble here about whether what is being done meets the requirements, 
but it doesn’t matter. All announcements get moderated and Sebb will check this 
and if he doesn’t like it he will reject the announcement email.

Note that the web site is NOT officially part of the release so I am NOT going 
to vote -1 due to any web site issues. Just be aware you are likely to have 
problems announcing the release.

Ralph


> On Aug 31, 2024, at 12:30 PM, Piotr P. Karwasz  
> wrote:
> 
> This is a vote to release the Apache Log4j `2.24.0`.
> 
> Website: https://logging.staged.apache.org/log4j/2.24.0/index.html
> GitHub: https://github.com/apache/logging-log4j2
> Commit: 08053687456f6be61ee8206da782a3d051928a57
> Distribution: https://dist.apache.org/repos/dist/dev/logging/log4j
> Nexus: 
> https://repository.apache.org/content/repositories/orgapachelogging-1293
> Signing key: 0x077e8893a6dcc33dd4a4d5b256e73ba9a0b592d0
> 
> Please download, test, and cast your votes on this mailing list.
> 
> [ ] +1, release the artifacts
> [ ] -1, don't release, because...
> 
> This vote is open for 72 hours and will pass unless getting a
> net negative vote count. All votes are welcome and we encourage
> everyone to test the release, but only the Logging Services PMC
> votes are officially counted. At least 3 +1 votes and more
> positive than negative votes are required.
> 
> == Review kit
> 
> The minimum set of steps needed to review the uploaded distribution
> files in the Subversion repository can be summarized as follows:
> 
># Check out the distribution
>svn co https://dist.apache.org/repos/dist/dev/logging/log4j/2.24.0 && cd $_
> 
># Verify checksums
>shasum --check *.sha512
> 
># Verify signatures
>wget -O - https://downloads.apache.org/logging/KEYS | gpg --import
>for sigFile in *.asc; do gpg --verify $sigFile; done
> 
># Verify reproduciblity
>umask 0022
>unzip *-src.zip -d src
>cd src
>export 
> NEXUS_REPO=https://repository.apache.org/content/repositories/orgapachelogging-1293
>sh mvnw -Prelease verify artifact:compare -Dreference.repo=$NEXUS_REPO
># If preferred, augment `mvnw` with `-DskipTests` to speed things up
> 
> == Release Notes
> 
> This release contains improvements and changes in several areas of Apache 
> Log4j:
> 
> === Log4j API
> 
> The `2.24.0` version of Log4j API has been enhanced with changes from
> the 3.x branch and will be used by both Log4j 2 Core and Log4j 3 Core
> releases.
> The changes include:
> 
> * A faster default `ThreadContextMap`.
> * Enhanced GraalVM support: native binaries that use Log4j API will no
> longer require additional GraalVM configuration.
> * The configuration properties subsystem now only accepts the official
> pre-2.10 property names and the normalized post-2.10 names.
> Check your configuration for typos.
> 
> === Documentation
> 
> The Apache Log4j 2 website has been almost entirely rewritten to
> provide improved documentation and faster access to the information
> you need.
> 
> [1] https://logging.staged.apache.org/log4j/2.24.0/index.html
> 
> === Bridges
> 
> The JUL-to-Log4j API and Log4j 1-to-Log4j API will no longer be able
> to modify the configuration of Log4j Core by default.
> If such a functionality is required, it must be explicitly enabled.
> 
> === Modules
> 
> The following Log4j Core additional modules have been removed:
> 
> `log4j-flume-ng`::
> The module has been moved to the Flume project and follows the Apache
> Flume release lifecycle.
> 
> `log4j-kubernetes`::
> The module has been moved to the
> https://github.com/fabric8io/kubernetes-client/blob/main/doc/KubernetesLog4j.md[Fabric8.io
> Kubernetes project] and follows the Fabric8.io release lifecycle.
> 
> `log4j-mongodb3`::
> The module based on MongoDB Java client version 3.x has been removed.
> Please migrate to `log4j-mongodb` (client version 5.x) or
> `log4j-mongodb4` (client version 4.x).
> 
> === JMX changes
> 
> Starting in version 2.24.0, JMX support is disabled by default and can
> be re-enabled via the `log4j2.disableJmx=false` system property.
> 
> === Added
> 
> * Add a faster `DefaultThreadContextMap` implementation. (#2330)
> * Add Logback throwable-consuming semantics as an option in
> `log4j-slf4j-impl` and `log4j-slf4j2-impl`. Users can enable it by
> setting the property `log4j2.messageFactory` to
> `org.apache.logging.slf4j.message.ThrowableConsumingMessageFactory`.
> (#2363)
> * Add trace context fields to `GcpLayout.json` (#2498)
> * Add _"Plugin Reference"_ to the website. It is a Javadoc-on-steroids
> focusing on Log4j plugins. (#1954)
> * Automate website deployment using the new CI infrastructure shipped
> with `org.apache.loggin

Re: Delaying publication of `log4j-flume-ng` in 3.x

[Ralph Goers]

My only concern for splitting it out is the same one I have for all of our 
modules, whether they reside in the same repo as Log4j or not. That is, they 
all need to be tested against the latest versions of Log4j. 

In the case of the Flume appender unless something specifically is being done 
to it there are only 2 reasons it needs a new release: 
1. It needs to be updated for a new version of Flume
2. It needs to be updated for something Log4j requires.

We know that for 3.0 we changed how Plugins bind with Log4j Core. While 2.x 
bindings should still work I would prefer that ALL Log4j provided plugins 
natively support 3.x.

All that is to say, all of this could be done with the Flume Appender residing 
in its own repo. It could either provide its own 3.x branch or provide a 
log4j-flume-3x jar. 

I kind of doubt it but did we make it so a plugin could be built for both 2.x 
and 3.x at the same time? I.e. Still provide a log4jplugins.dat file and have 
the ServiceLoader bindings?

So I will say I am fine with moving it to its own module.  Can it be moved 
while keeping the 2.x and main branches? 

I looked at the Nexus repo stats and was actually surprised that log4j-flume-ng 
falls smack in the middle of the pack. That said, 2/3 of our artifacts show up 
as 0% - they are dwarfed by these:

log4j-bom41,694,665.00  25.80%
log4j-api  38,197,952.00  23.64%
log4j-core21,092,891.00  13.05%
log4j-to-slf4j17,363,471.00   10.75%
log4j13,990,810.00.8.66%
log4j-slf4j-impl   9,797,055.00 6.06%
log4j-1.2-api  4,452,489.00 2.76%
log4j-layout-template-json 4,393,231.00 2.72%
log4j-jul  2,799,082.00.1.73%
log4j-slf4j2-impl 2,695,794.00.1.67%
log4j-web   2,367,956.00.1.47%
log4j-appserver745,912.00. 0.46%
log4j-jcl 606,258.00  0.38%
log4j-iostreams185,659.00   0.11%
log4j-slf4j18-impl  173,961.00   0.11%
log4j-spring-boot   158,438.00  0.10%

However, usage of the Flume Appender appears to be greater than some others 
where you might not think it would be

log4j-flume-ng 10,367.00 0.01%
log4j-taglib   10,236.00 0.01%
log4j-spring-cloud-config  9,769.00  0.01%
log4j-couchdb9,331.000.01%
log4j-jmx-gui8,345.000.01%
log4j-liquibase  5,903.000.00%
log4j-jakarta-smtp   4,201.000.00%
log4j-mongodb4   3,741.00  0.00%
log4j-jpa2,507.000.00%
log4j-mongodb3   1,047.000.00%

Ralph


> On Aug 29, 2024, at 5:12 AM, Piotr P. Karwasz  wrote:
> 
> Hi all,
> 
> The `log4j-flume-ng` module _de facto_ contains 3 different appenders:
> 
> * an Avro appender, that only depends on Avro and Avro IPC. Since it
> only communicates with Flume via network, it doesn't need to even
> depend on Flume, it just needs to use the same protocol.
> * a Persistent appender, which works the same way as Avro appender,
> but stores the messages to a local database first. It has an
> additional dependency on `je` (a mostly unmaintained implementation of
> BerkeleyDB).
> * an Embedded appender, which is the only one that actually depends on Flume.
> 
> It also contains an undocumented (and untested) `Log4jEventSource`
> plugin for Flume.
> 
> The use of optional dependencies is not very JPMS friendly and we
> probably would need to split it into 3 modules. To prevent delays in
> releasing 3.0.0 I would propose to:
> 
> * either move the `log4j-flume-ng` module to a separate repo and
> release version `3.0.0` together with the next Flume release.
> * or move it from the `main` branch to a different branch that will be
> merged, when the module is ready.
> 
> What do you think?
> 
> Piotr

Re: Release timeline for log4j 2.24.0?

[Ralph Goers]

Note that this uses a custom appender that saves only the last log event 
written. At the end of the test that record is written via System.println. As I 
recall, for the StringArrayThreadContextMap it prints “null”.

Ralph

> On Aug 14, 2024, at 4:45 PM, Ralph Goers  wrote:
> 
> Try this one 
> https://github.com/apache/logging-log4j2/blob/2.25.x/log4j-perf-test/src/main/java/org/apache/logging/log4j/perf/jmh/ThreadContextVsScopedContextBenchmark.java
> 
> Ralph
> 
>> On Aug 14, 2024, at 7:23 AM, John Engebretson  wrote:
>> 
>> Ralph, I get a 404 on that link to your code, could you please verify the
>> link is good?
>> 
>> FWIW:
>> - the unit test on StringArrayThreadContextMap thoroughly tests puts
>> (verified by gets).  I won't rule out a missed case, of course.
>> - the JMH test on the underlying datastructure (UnmodifiableArrayBackedMap)
>> is attached at https://github.com/apache/logging-log4j2/issues/2319, as
>> well as a description of the performance characteristics.
>> 
>> I'd love to hear more about any conflicting findings, and will dive in
>> once I get your code.
>> Thanks!  :)
>>   John
>> 
>> 
>> 
>> On Tue, Aug 13, 2024 at 5:07 PM Ralph Goers 
>> wrote:
>> 
>>> The benchmark code I used was
>>> https://github.com/apache/logging-log4j2/blob/feature/move-thread-context/log4j-perf-test/src/main/java/org/apache/logging/log4j/perf/jmh/ThreadContextVsScopedContextBenchmark.java
>>> 
>>> Ralph
>>> 
>>>> On Aug 13, 2024, at 3:00 PM, Ralph Goers 
>>> wrote:
>>>> 
>>>> Piotr, I am still concerned that you never looked into the issues I had
>>> with StringArrayThreadContextMap. My performance tests were showing that it
>>> was fast because it wasn’t actually setting values.
>>>> 
>>>> Ralph
>>>> 
>>>>> On Aug 13, 2024, at 12:43 PM, Piotr P. Karwasz 
>>> wrote:
>>>>> 
>>>>> Hi John,
>>>>> 
>>>>> On Tue, 13 Aug 2024 at 21:10, John Engebretson 
>>> wrote:
>>>>>> HI - just curious when the updated ThreadContextMap will be available?
>>>>>> The latest release is 2.23.1, over five months ago.
>>>>> 
>>>>> We just finished a long series of documentation tasks that kept us
>>>>> occupied for the past since May[1].
>>>>> 
>>>>> I will be able to concentrate on cutting this almost mythical 2.24.0
>>>>> release in the next few weeks.
>>>>> 
>>>>> Piotr
>>>>> 
>>>>> [1] https://github.com/apache/logging-log4j2/issues/2542
>>>> 
>>> 
>>> 
> 

Re: Release timeline for log4j 2.24.0?

[Ralph Goers]

Try this one 
https://github.com/apache/logging-log4j2/blob/2.25.x/log4j-perf-test/src/main/java/org/apache/logging/log4j/perf/jmh/ThreadContextVsScopedContextBenchmark.java

Ralph

> On Aug 14, 2024, at 7:23 AM, John Engebretson  wrote:
> 
>  Ralph, I get a 404 on that link to your code, could you please verify the
> link is good?
> 
> FWIW:
> - the unit test on StringArrayThreadContextMap thoroughly tests puts
> (verified by gets).  I won't rule out a missed case, of course.
> - the JMH test on the underlying datastructure (UnmodifiableArrayBackedMap)
> is attached at https://github.com/apache/logging-log4j2/issues/2319, as
> well as a description of the performance characteristics.
> 
>  I'd love to hear more about any conflicting findings, and will dive in
> once I get your code.
>  Thanks!  :)
>John
> 
> 
> 
> On Tue, Aug 13, 2024 at 5:07 PM Ralph Goers 
> wrote:
> 
>> The benchmark code I used was
>> https://github.com/apache/logging-log4j2/blob/feature/move-thread-context/log4j-perf-test/src/main/java/org/apache/logging/log4j/perf/jmh/ThreadContextVsScopedContextBenchmark.java
>> 
>> Ralph
>> 
>>> On Aug 13, 2024, at 3:00 PM, Ralph Goers 
>> wrote:
>>> 
>>> Piotr, I am still concerned that you never looked into the issues I had
>> with StringArrayThreadContextMap. My performance tests were showing that it
>> was fast because it wasn’t actually setting values.
>>> 
>>> Ralph
>>> 
>>>> On Aug 13, 2024, at 12:43 PM, Piotr P. Karwasz 
>> wrote:
>>>> 
>>>> Hi John,
>>>> 
>>>> On Tue, 13 Aug 2024 at 21:10, John Engebretson 
>> wrote:
>>>>> HI - just curious when the updated ThreadContextMap will be available?
>>>>> The latest release is 2.23.1, over five months ago.
>>>> 
>>>> We just finished a long series of documentation tasks that kept us
>>>> occupied for the past since May[1].
>>>> 
>>>> I will be able to concentrate on cutting this almost mythical 2.24.0
>>>> release in the next few weeks.
>>>> 
>>>> Piotr
>>>> 
>>>> [1] https://github.com/apache/logging-log4j2/issues/2542
>>> 
>> 
>> 

Re: Open GitHub discussions for Log4j Scala and Kotlin

[Ralph Goers]

I thought gitbox was still an alternative to having to use GitHub to submit 
PRs. That said, it wouldn’t be for discussions, etc.

Ralph

> On Aug 14, 2024, at 3:31 AM, Gary Gregory  wrote:
> 
> I think that to "deactivate" the mailing lists you refer to would be very
> bad and might not even be allowed by Apache.
> 
> You cannot/shouldn't force people to create a GitHub (Microsoft) account
> just to talk to us.
> 
> Odd as it sounds, I've had someone told me they would not submit a PR
> because they did not want to create a GH account. It feels
> counterproductive from my POV not to submit PRs but it shouldn't lock
> people out.
> 
> There might also be parts of the world where GH is not available but Apache
> servers are (more of a geopolitical issue).
> 
> Gary
> 
> On Wed, Aug 14, 2024, 3:43 AM Piotr P. Karwasz 
> wrote:
> 
>> Hi Gary,
>> 
>> On Fri, 9 Aug 2024 at 12:15, Gary Gregory  wrote:
>>> While it is a different community, I have received negative feedback in
>>> Commons against the trend to spread information all over the place, and
>> we
>>> don't use GitHub.
>>> 
>>> The TLDR is that in the past it was easier to find information because
>> you
>>> only had the mailing list and later Jira. Now you have multiple mailing
>>> lists, Jira, emails from GitHub, plus discussions in GitHub PRs, plus
>>> Slack. Adding to the list is GitHub issues...
>> 
>> Might I propose a split of concerns between GH Discussions and the
>> mailing lists:
>> 
>> * We use GH Discussions as a replacement for `log4j-user`,
>> `log4net-user`, `log4cxx-user`. Two of these mailing lists are dead
>> anyway, since subscribing to a ML for a one-off question is a hustle.
>> * We use `dev@logging` as always for reaching consensus and decisions
>> concerning the directions we choose for development.
>> 
>> BTW: Can we deactivate all the mailing lists except `dev`, `private`
>> and `security`?
>> 
>> Piotr
>> 

Re: Release timeline for log4j 2.24.0?

[Ralph Goers]

The benchmark code I used was 
https://github.com/apache/logging-log4j2/blob/feature/move-thread-context/log4j-perf-test/src/main/java/org/apache/logging/log4j/perf/jmh/ThreadContextVsScopedContextBenchmark.java

Ralph

> On Aug 13, 2024, at 3:00 PM, Ralph Goers  wrote:
> 
> Piotr, I am still concerned that you never looked into the issues I had with 
> StringArrayThreadContextMap. My performance tests were showing that it was 
> fast because it wasn’t actually setting values.
> 
> Ralph
> 
>> On Aug 13, 2024, at 12:43 PM, Piotr P. Karwasz  
>> wrote:
>> 
>> Hi John,
>> 
>> On Tue, 13 Aug 2024 at 21:10, John Engebretson  wrote:
>>> HI - just curious when the updated ThreadContextMap will be available?
>>> The latest release is 2.23.1, over five months ago.
>> 
>> We just finished a long series of documentation tasks that kept us
>> occupied for the past since May[1].
>> 
>> I will be able to concentrate on cutting this almost mythical 2.24.0
>> release in the next few weeks.
>> 
>> Piotr
>> 
>> [1] https://github.com/apache/logging-log4j2/issues/2542
> 

Re: Release timeline for log4j 2.24.0?

[Ralph Goers]

Piotr, I am still concerned that you never looked into the issues I had with 
StringArrayThreadContextMap. My performance tests were showing that it was fast 
because it wasn’t actually setting values.

Ralph

> On Aug 13, 2024, at 12:43 PM, Piotr P. Karwasz  
> wrote:
> 
> Hi John,
> 
> On Tue, 13 Aug 2024 at 21:10, John Engebretson  wrote:
>>  HI - just curious when the updated ThreadContextMap will be available?
>> The latest release is 2.23.1, over five months ago.
> 
> We just finished a long series of documentation tasks that kept us
> occupied for the past since May[1].
> 
> I will be able to concentrate on cutting this almost mythical 2.24.0
> release in the next few weeks.
> 
> Piotr
> 
> [1] https://github.com/apache/logging-log4j2/issues/2542

Re: Should Log4j API bridges have a 3.x release?

[Ralph Goers]

I am fine with this plan.

Ralph

> On Aug 9, 2024, at 1:35 PM, Piotr P. Karwasz  wrote:
> 
> Hi Matt,
> 
> On Fri, 9 Aug 2024 at 20:29, Matt Sicker  wrote:
>> 
>> Well, one thing that changed in JUL is that it requires the java.logging 
>> module. Otherwise, the Java 8+ stuff is for the System.Logger API.
> 
> Right, maybe a 3.x version of `log4j-jul` would be useful. Besides the
> artifact has an optional dep on `log4j-core`. We might want to split
> it into 2 artifacts.
> Regarding `log4j-to-jul`, it is so rarely used, I don't see the point
> to maintain two identical versions.
> 
> Summarizing:
> 
> * `log4j-iostreams`, `log4j-slf4j-impl`, `log4j-slfj42-impl`,
> `log4j-to-jul` and `log4j-to-slf4j` could be removed from `main`.
> * `log4j-jpl` and `log4j-jul` should stay.
> 
> Do you agree?
> 
> Piotr

Re: Integration tests

[Ralph Goers]

I believe I have asked for a log4j-its module for a while now since that is the 
only way you can validate modules actually work together. I am fine with it 
being in a separate repo.

Ralph

> On Aug 8, 2024, at 11:35 PM, Piotr P. Karwasz  wrote:
> 
> Hi all,
> 
> Unless I am mistaken, adding tests that run under JPMS is problematic
> in the `apache/logging-log4j2` repository. Even if I create a new
> Maven module for these tests:
> 
> * IntelliJ IDEA still complains that it doesn't know a
> `org.apache.logging.log4j.core` module.
> * These tests might be flaky, since the `module-info.class` file is
> destroyed before each compilation step and created right after it.
> 
> Should I create a new `apache/logging-log4j-its` repository for these
> tests? I can configure the CI to run them daily or after a snapshot
> has been generated.
> 
> Piotr
> 
> PS: Naming of the repo is as usual a problem.

Re: Updating the Logging Services Logo

[Ralph Goers]

What am I missing here?  In looking at all the project sites none of our logos 
have a feather in them. Once the ASF provides a replacement logo we can simply 
replace it.

Why are we talking about new logos? FWIW none of our projects existed in the 
1980s so I don’t know where that comment came from. I believe it was Gary who 
won the contest for that 10 or so years ago. I still like it.

Ralph

> On Jul 22, 2024, at 12:58 PM, Gary Gregory  wrote:
> 
> I find it peculiar that ALL feathers are now illegal. What about the
> feather of a Phoenix?
> 
> On Mon, Jul 22, 2024, 3:37 PM Christian Grobmeier 
> wrote:
> 
>> I am only asking to remove native american imagery as the feathers.
>> As I was involved in the branding process, I am confident we can remove
>> them.
>> 
>> We can update the 1980-style logo after the ASF logo was voted on.
>> Until then, we will be compliant almost immediately by dropping the
>> feathers.
>> 
>> I didn't expect any negative feedback for removing the feathers, but
>> please let me know if anyone is really opposed to it or if it is just a
>> general comment
>> 
>> On Sat, Jul 20, 2024, at 12:01, Gary Gregory wrote:
>>> That sounds good.
>>> 
>>> I'm not doing the work here but I would imagine that if I did I'd only
>> want
>>> to do it once ;-) For my time, I'd either strip the images and call it
>>> done, or wait for new images.
>>> 
>>> Gary
>>> 
>>> On Sat, Jul 20, 2024, 1:45 AM Dominik Psenner 
>> wrote:
>>> 
 The blog post mentions the 7 oct 2024 as the deadline to be compliant.
>> This
 coincides with the day when the new marketing guidelines are unveiled.
>> It
 is probably impossible to implement this within hours. Therefore I
>> propose
 to ask Marketing for assistance/guiding/new marketing material right
>> now in
 the light of the upcoming deadline. It will take some time anyway until
>> we
 receive a feedback, maybe even several feedback loops. With this action
>> we
 can be well ahead of any upcoming deadline and do not have to rush for
>> any
 changes. And we may even be able to evaluate whether we want to remove
>> OR
 replace existing images. Primarily to avoid breaking links, but
>> ultimately
 also updating marketing material in a retrospective.
 
 On Fri, 19 Jul 2024, 21:41 Gary Gregory, 
>> wrote:
 
> I think this is a bad idea because it is putting the cart before the
 horse
> as we say in English. Let Apache come up with our new branding and
>> then
 use
> that. It seems all feathers of any kind are not allowed?
> 
> Gary
> 
> On Fri, Jul 19, 2024, 2:15 PM Christian Grobmeier <
>> grobme...@apache.org>
> wrote:
> 
>> Hello,
>> 
>> following this blog post:
>> https://news.apache.org/foundation/entry/evolving-the-asf-brand
>> 
>> I would like to propose to remove all feathers as here:
>> https://logging.apache.org/
>> https://logging.apache.org/log4j/2.x/images/ls-logo.jpg
>> 
>> And all sub-websites with simlar images.
>> 
>> I would like to move on with this now, since it will take quite a
>> while
> to
>> complete it anyway.
>> 
>> Thoughts?
>> 
>> Kind regards,
>> Christian
>> 
> 
 
>> 

Re: Redundant abstractions: property sources, lookups and pattern converters

[Ralph Goers]

> On Jul 17, 2024, at 2:11 PM, Piotr P. Karwasz  wrote:
> 
> Hi Matt,
> 
> On Wed, 17 Jul 2024 at 19:03, Matt Sicker  wrote:
>> I’ll note one design constraint behind property sources and lookups. 
>> Property sources are defined in the API while lookups are defined in Core.
> 
> Since Log4j Core 3 has a different properties system, property sources
> are technically defined in the API, but are almost unused in the API.
> They are only used:
> 
> 1. For the `log4j.provider` property[1],
> 2. For the `SimpleLoggerContext` and related classes[2].
> 
> Status logger since 2.23.0 does not depend on `PropertiesUtil`[3].
> The remaining properties (including the configuration of
> `ThreadContextMap`) are used in Log4j Core. In this way Log4j Core 3
> can have its own properties system.
> 
>> Besides that, property sources are intended for use with configuration 
>> (either for components outside the config file or for reuse in a config 
>> file) while lookups are intended for use with log events, though the ability 
>> to use properties in log events makes this confusing.
> 
> Most lookups don't really profit from the `LogEvent` parameter[4] and
> those that do, have a corresponding `PatternConverter`.

Conversely, most PatternConverters are only useful in a log event. Allowing 
them to be used outside of one would be awful. The only exception that comes to 
mind is the DateConverter.

Ralph

Re: Redundant abstractions: property sources, lookups and pattern converters

[Ralph Goers]

> On Jul 17, 2024, at 3:09 AM, Piotr P. Karwasz  wrote:
> 
> Hi Ralph,
> 
> On Tue, 16 Jul 2024 at 15:17, Ralph Goers  wrote:
>>> * we have a `SpringLookup` and a `SpringPropertySource`.
>> 
>> SpringLookup returns the value of a key. A property source locates a set of 
>> keys. So comparing them is apples to oranges. Or rather - an apple to a 
>> bushel of apples. PropertySources are NOT available to the logging 
>> configuration. However, a Lookup can certainly use a PropertySource to 
>> locate the keys it is to resolve.
> 
> Yes, property sources can not be addressed directly, since they are
> not namespaced, as you remark below.
> Other than that lookups and property sources are pretty similar: the
> `forEach()` methods of property sources are only useful for caching
> and I am not sure caching dozens of not Log4j related properties is
> useful.
> 
>>> * we have a `${date:...}` lookup and a `%d{...}`. Note that
>> 
>>> `$${date:...}` and `%d{...}` give the same result in a
>>> `PatternLayout`!
>> 
>> While they do in the Layout they do not have the same behavior when used in 
>> the FilePattern on a Rolling Appender.
> 
> Good point, I have a note about that in the revamped
> RollingFileAppender description.
> 
> Although `$${date:...}` and `%d{...}` are not equivalent for
> `filePattern`, I don't see a reason for users to use the former.
> A file pattern like `$${date:-MM}/%d{-MM-dd}.log` that I often
> see in user questions, simply doesn't do what the user wants. The
> `2024-06-30.log` file ends up in the `2024-07` folder!
> The user should use `%d{-MM}/%d{-MM-dd}.log`, which works
> correctly since 2012 according to Git.

Really? It only has a 50% chance of being correct as only one of those can be 
used to determine the rollover interval. As I said, this should be changed so 
that the rollover interval is in a separate attribute.


> 
>> IMO, it was a poor decision to use the data in the file pattern to determine 
>> the rollover interval.
> 
> I share the same opinion. Autodetection is very useful for simple
> cases, but we might want to add a `frequency` config attribute to
> allow users to override autodetection.
> 
> Same thing applies to the autodetection of compress actions: users
> might want to configure those explicitly, e.g., if they want to
> configure the compression level and other parameters.
> 
>>> ## Equivalent concepts in other frameworks
>>> 
>>> Spring Boot's equivalent of both `PropertiesUtil` and `Interpolator`
>>> is the single `Environment`[1] concept:
>> 
>> However, Spring’s variables are primarily used while configuring the 
>> application. The same is true with PropertySources. OTOH Lookups are meant 
>> to be used while the application is running.
> 
> If we exclude RoutingAppender, most lookups are used at configuration
> time. Less than 10 attributes accept lookups at runtime. I have
> documented them all recently:
> 
> https://logging.staged.apache.org/log4j/2.x/manual/configuration.html#lazy-property-substitution

Any sentence that starts with “If we exclude…” means you want to break those 
things being excluded.

> 
>>> What do you think?
>> 
>> I am not convinced.
>> 
>> The advantage with Lookups is that they can be namespaced. i.e. 
>> ${ctx:myKey}. This is a nice short way of expressing which Map you want to 
>> look in. Specifying ${myKey} would require merging all the Maps together.
> 
> Namespacing can be both an advantage and a disadvantage:
> 
> * you don't want ephemeral context data to pollute the global
> namespace, so `${ctx:myKey}` is necessary. However in many places
> (like the `pattern` of a `Routes`) component, it could be replaced by
> `%X{myKey}`.
> * often users are startled by the necessity of adding a namespace,
> e.g. `${sys:myKey}`, especially if the migrate from Log4j 1.
> The `sys` and `env` lookups should often be used together, e.g.
> `${sys:myKey:-${env:myKey}}`. This would be much simpler if we remove
> the namespace.

There are better ways to deal with this than dumping namespaces entirely. For 
example, creating a lookup that is an alias for the system property lookup 
followed by the environment lookup. Or a dynamic way to create a new lookup by 
listing the lookups you want to include and their order.

> 
>> I am not a fan of making either of these pluggable. Making StrSubstitutor 
>> pluggable is a security risk. PropertiesUtil is pluggable by way of custom 
>> PropertySources.
> 
> It is a security risk, but it is not **our** security risk.
> If we make `StrSubstitutor` pluggable

Re: Redundant abstractions: property sources, lookups and pattern converters

[Ralph Goers]

> On Jul 15, 2024, at 6:30 AM, Piotr P. Karwasz  wrote:
> 
> Since the creation of Log4j 2, the concepts of `PropertySource` and
> `StrLookup` and the concepts of `StrLookup` and
> `LogEventPatternConverter` have strongly converged to each other.
> We might consider removing some of them in Log4j Core 4.
> 
> ## Problem
> 
> Log4j Core uses multiple map-like abstractions that evaluate strings:
> 
> * `PropertySource`s are used to provide additional sources of
> **externalized configuration** to
> `PropertiesUtil/PropertyEnvironment`, which is used to modify default
> values of Log4j components.
> * `StrLookup`s are used by `Interpolator` to interpolate the
> attributes of the configuration file, which is something a
> `PropertiesUtil/PropertyEnvironment` could also do.
> The `Interpolator` is also used at runtime by `PatternLayout` to
> format log events.
> * `LogEventPatternConverter`s, which are used by `PatternLayout` to
> format log events.
> 
> Since the domains of these abstractions overlap, we end up
> implementing these in pairs. For example:
> 
> * we have a `SpringLookup` and a `SpringPropertySource`.

SpringLookup returns the value of a key. A property source locates a set of 
keys. So comparing them is apples to oranges. Or rather - an apple to a bushel 
of apples. PropertySources are NOT available to the logging configuration. 
However, a Lookup can certainly use a ProeprtySource to locate the keys it is 
to resolve.


> * we have a `${date:...}` lookup and a `%d{...}`. Note that

> `$${date:...}` and `%d{...}` give the same result in a
> `PatternLayout`!

While they do in the Layout they do not have the same behavior when used in the 
FilePattern on a Rolling Appender. IMO, it was a poor decision to use the data 
in the file pattern to determine the rollover interval.

> 
> ## Equivalent concepts in other frameworks
> 
> Spring Boot's equivalent of both `PropertiesUtil` and `Interpolator`
> is the single `Environment`[1] concept:

However, Spring’s variables are primarily used while configuring the 
application. The same is true with PropertySources. OTOH Lookups are meant to 
be used while the application is running.

> 
> * The `Environment` provides default values of Spring Boot beans, like
> `PropertiesUtil` does.
> * `${name}` expressions can be used in XML bean definitions[2], which
> is the concept that resembles our configuration file.
> 
> If `${name}` expressions are too restrictive (e.g. you can use them to
> get the current date), SpEL expressions can be used. Therefore SpEL
> expressions ressemble the way we use pattern converters.
> 
> The Jakarta world also released a `Config`[3] concept, which works
> pretty much like Spring's `Environment`.
> 
> ## Possible solutions
> 
> For Log4j 4 we could consider:
> 
> * removing the `StrLookup` concept and provide a simple `Interpolator`
> that uses `PropertiesUtil`.
> * make both `PropertiesUtil` and `StrSubstitutor` pluggable: external
> frameworks might want to replace our implementations of these
> components entirely with their own implementations.
> * enhance the few configuration attributes that accept lazy lookups to
> accept `PatternLayout` patterns instead.
> 
> What do you think?

I am not convinced. 

The advantage with Lookups is that they can be namespaced. i.e. ${ctx:myKey}. 
This is a nice short way of expressing which Map you want to look in. 
Specifying ${myKey} would require merging all the Maps together.
I am not a fan of making either of these pluggable. Making StrSubstitutor 
pluggable is a security risk. PropertiesUtil is pluggable by way of custom 
PropertySources.

Compared to a Lookup a PatternConverter is fairly expensive. A Lookup simply 
returns the value of a key from a Map. OTOH a PatternConverter is really a 
formatter. Granted we do have things like the upper Lookup that acts as a 
formatter but its capabilities are purposefully limited.

Ralph

Re: Release timeline for 2.24.0

[Ralph Goers]

I have been working on creating log4j-context-data. My plan was to remove stuff 
from Log4j API after I have something workable. Even though 2.24.0 won’t 
include log4j-context-data I want to make sure that what we are leaving will be 
compatible. 

Ralph

> On Jun 20, 2024, at 4:15 AM, Gary Gregory  wrote:
> 
> I lost track of where we are after our video discussion regarding all the
> scoped classes and 2.24.0. Where are we on those?
> 
> Also, can we get information on what happened with the conference call
> regarding the (IMO bad) idea on conditional testing? I sure hope we're not
> using it.
> 
> Gary
> 
> On Tue, May 28, 2024, 12:27 PM Matt Sicker  wrote:
> 
>> Probably discussions about the ScopedContext API in general.
>> 
>>> On May 26, 2024, at 14:49, Ralph Goers 
>> wrote:
>>> 
>>> 
>>> 
>>>> On May 26, 2024, at 11:06 AM, Christian Grobmeier 
>> wrote:
>>>> 
>>>> 
>>>> 
>>>> On Sat, May 25, 2024, at 04:22, Gary Gregory wrote:
>>>>> I'd like to have the PMC meet to review and discuss ScopedContext,
>>>>> which I am not caught up on, and whatever else we should chat about.
>>>> 
>>>> Me too.
>>>> 
>>>> I feel that all the discussion on the ScopedContext should be in an
>> experimental branch or an additional jar file rather than the stable code
>> base.
>>>> 
>>>> I see it popping up so often, I start to believe it's "not there yet.”
>>> 
>>> See what popping up so often?
>>> 
>>> Ralph
>> 
>> 

Video Call on June 2, 2024

[Ralph Goers]

Several of the PMC members met yesterday to primarily discuss the ScopedContext 
and getting release 2.24.0 ready. The notes from that meeting are at 
https://docs.google.com/document/d/1U7IXrgNYKX3aFE1IEVDmYradt5nxBn2NJM8P9_f1f0M/edit?usp=sharing.

Ralph

Re: Should Log4j API bridges have a 3.x release?

[Ralph Goers]

Log4j 2.x will run fine on Java 8.  However, the logging Parent POM requires 
Java 17 to run the build.

Ralph

> On Jun 3, 2024, at 5:47 AM, John Engebretson  wrote:
> 
>  I'm sorry for being unclear!  What I mean is that the current code built
> on 8 will behave and perform identically to the current code built on 17.
> Is that correct?
> John
> 
> On Thu, May 30, 2024 at 10:04 AM Ralph Goers 
> wrote:
> 
>> 
>> 
>>> On May 30, 2024, at 7:43 AM, John Engebretson 
>> wrote:
>>> 
>>> I agree on the complexity argument - and FWIW we still have many apps on
>>> JDK 8 and are intimidated by the Spring upgrade, so I understand that.
>> :)
>>> Sounds like we agree there's no runtime impact?
>>>John
>>> 
>> 
>> I am not sure what you mean there. We are now building 2.x with JDK 17 but
>> targeting JDK 8. With 3.x we can take advantage of anything added in JDK 11
>> or 17 so that could have a runtime impact.
>> 
>> Ralph
>> 
>> 

Re: Should Log4j API bridges have a 3.x release?

[Ralph Goers]

> On May 30, 2024, at 7:43 AM, John Engebretson  wrote:
> 
> I agree on the complexity argument - and FWIW we still have many apps on
> JDK 8 and are intimidated by the Spring upgrade, so I understand that.  :)
> Sounds like we agree there's no runtime impact?
> John
> 

I am not sure what you mean there. We are now building 2.x with JDK 17 but 
targeting JDK 8. With 3.x we can take advantage of anything added in JDK 11 or 
17 so that could have a runtime impact.

Ralph

Re: Should Log4j API bridges have a 3.x release?

[Ralph Goers]

> On May 30, 2024, at 6:37 AM, Engebretson, John  
> wrote:
> 
> On 2024/04/10 01:45:19 Ralph Goers wrote:
>> 
>> 
>>> On Apr 9, 2024, at 3:52 PM, Piotr P. Karwasz  wrote:
>>> 
>>> 
>>> BTW: Maybe SLF4J still uses Java 8, but the latest Logback uses Java 11.
>> 
>> Ask me if I care about Logback. ;-)
>> 
>> Ralph
> 
> 
> 
>  Following up on this chain re: JDK 8 vs. 17 – code compiled against JDK 8 
> should run the same as that code compiled against JDK 17.  Language features 
> are clearly limited to JDK 8 but that doesn’t appear to be an immediate issue.
> 
>  So, other than enabling new features, what is the driver for targeting JDK 
> 17?
> 
>  John

For 3.x?  Maintenance is simpler for one. The more JDKs we support the more I 
have to have installed on my laptop. It also does make it simpler to take 
advantage of and support new language features.

Remember that most of us have $dayjobs. Over 90% of the apps I support at work 
have moved off of JDK 8. Most are on JDK 11 but some have moved up to JDK 17. 
To be honest, all are apps would be there if moving to Spring 3.x wasn’t so 
hard.

Ralph

Re: Release timeline for 2.24.0

[Ralph Goers]

> On May 26, 2024, at 11:06 AM, Christian Grobmeier  
> wrote:
> 
> 
> 
> On Sat, May 25, 2024, at 04:22, Gary Gregory wrote:
>> I'd like to have the PMC meet to review and discuss ScopedContext,
>> which I am not caught up on, and whatever else we should chat about.
> 
> Me too.
> 
> I feel that all the discussion on the ScopedContext should be in an 
> experimental branch or an additional jar file rather than the stable code 
> base.
> 
> I see it popping up so often, I start to believe it's "not there yet.”

See what popping up so often?

Ralph

Re: Release timeline for 2.24.0

[Ralph Goers]

Thanks!  Is there any chance you could provide some sample code we could use to 
perform performance testing?  Based on your description I have a suspicion that 
ScopedContext would perform even better but at this point that is just 
speculation without tests. Of course, that may be minimal and might not be 
worth the effort of having to change code.

You say that it is mostly for a logging/metrics library. Does that mean it is 
storing counters that it has to periodically increment? I’m curious there since 
the MDC/ThreadContextMap only support strings.

Ralph

> On May 24, 2024, at 12:00 PM, John Engebretson  wrote:
> 
>  Certainly!  Our usage is widespread in our applications, and mostly
> wrapped inside an internal logging/metrics library.  Focusing on our most
> performance-critical applications, our usage falls into two categories:
> 
> 1. MDC sets/gets, ranging from 1 to 8 values at any given time.
> 2. CloseableThreadContext, with attribute counts varying from 0 to 15
> values at any given time.
> 
>  Our performance profiling/investigation/tooling highlighted the cost of
> HashMap operations when modifying the context; that cost involved cpu time,
> critical-path clock time, and memory allocation/GC.  The largest amount of
> work comes from copying the old HashMap into the new one - iterating across
> empty buckets, creating new Nodes, recalculating hashcodes, etc.
>  2.24.0 will reduce the runtime cost with no application changes.  We like
> that outcome.  :)
>    John
> 
> On Fri, May 24, 2024 at 1:47 PM Ralph Goers 
> wrote:
> 
>> 
>> 
>>> On May 24, 2024, at 11:10 AM, Piotr P. Karwasz 
>> wrote:
>>> 
>>> Hi John,
>>> 
>>> On Fri, 24 May 2024 at 17:17, John Engebretson 
>> wrote:
>>>> As
>>> I stated several times, 'ThreadContext.get()` is a mistake, because it
>>> allows users to abuse the API and transport logging-unrelated data
>>> through `ThreadContext`. If we could correct the past, I would remove
>>> `ThreadContext.get()` and let `ThreadContext.put()` return the old
>>> value.
>> 
>> John,
>> 
>> ThreadContextMap is obviously very important to your use case. Can you
>> describe how you are using it? What about the way you are using it made you
>> notice it was having an impact on performance?
>> 
>> Ralph

Re: Release timeline for 2.24.0

[Ralph Goers]

> On May 24, 2024, at 11:10 AM, Piotr P. Karwasz  
> wrote:
> 
> Hi John,
> 
> On Fri, 24 May 2024 at 17:17, John Engebretson  wrote:
>>  As
> I stated several times, 'ThreadContext.get()` is a mistake, because it
> allows users to abuse the API and transport logging-unrelated data
> through `ThreadContext`. If we could correct the past, I would remove
> `ThreadContext.get()` and let `ThreadContext.put()` return the old
> value.

John,

ThreadContextMap is obviously very important to your use case. Can you describe 
how you are using it? What about the way you are using it made you notice it 
was having an impact on performance? 

Ralph

Re: Author lines in documentation

[Ralph Goers]

IMO, yes. I had them due to something needed them in the tooling we used to 
use. But like code, tags in the docs aren’t very valuable.

Ralph

> On May 14, 2024, at 8:00 AM, Piotr P. Karwasz  wrote:
> 
> Can we remove author lines in documentation pages?
> 
> These lines are becoming crowded and somehow IntelliJ IDEA formats
> author lines badly (i.e. inserts a blank line before them and they end
> up in the text.
> 
> Of course this doesn't need to be a PMC decision: when I reformat a
> page I'll remove the names of the authors that are OK with it (and I
> won't add mine ;-) ).
> 
> Piotr

Re: Canonicalization of URLs on our website

[Ralph Goers]

The IDE lets you view the page you are editing but generally it doesn’t show 
you the nav or the rest of the site. I am also not confident that everything 
will be styled exactly the same as it will appear in the browser.
So, in short, the IDE is great for editing but I like to do my reviews on a 
local site.

Ralph

> On May 8, 2024, at 9:50 AM, Piotr P. Karwasz  wrote:
> 
> Hi Volkan,
> 
> On Wed, 8 May 2024 at 14:57, Volkan Yazıcı  wrote:
>> In my opinion,
>> 
>>   - Being able to view `target/site` with just using my browser and
>>   nothing else is super convenient. The development experience is much
>>   smoother.
> 
> If we use a non-default `html_extension_style`, you can still preview
> the website locally. You just need to modify the Antora Playbook
> (AFAIK Antora does not support variable substitution in the playbook).
> We can also set `html_extension_style: default` in our Git repo and
> configure the CI to modify it when it builds the staging site.
> 
> I usually use my IDE to preview the documentation pages, so for me it
> is not a problem.
> 
> Anyway I value consistency more than one particular solution. If we
> prefer to stick with the `default` style, for SEO purposes we should
> ensure that all our internal links end up in `.html`. This mainly
> applies to index files.
> 
> Piotr

Re: Canonicalization of URLs on our website

[Ralph Goers]

Volkan,

I completely agree with you. I prefer to review my site changes either in 
target/site or by doing a deploy to a local directly. In both cases I use the 
file protocol to view it.

Ralph

> On May 8, 2024, at 5:55 AM, Volkan Yazıcı  wrote:
> 
> All non-default `html_extension_style` options require to run a web server.
> 
> In my opinion,
> 
>   - Being able to view `target/site` with just using my browser and
>   nothing else is super convenient. The development experience is much
>   smoother.
>   - None of the advantages you cited for switching from `/foo.html` to
>   `/foo`, `/foo/index.html`, etc. is worth the trouble/complexity it will
>   introduce.
> 
> In short, I am not inclined to change the current path naming scheme. That
> said, I don't want to sound bossy. I would appreciate it if others can join
> the discussion with their arguments.
> 
> On Wed, May 8, 2024 at 10:22 AM Piotr P. Karwasz 
> wrote:
> 
>> Hi all,
>> 
>> On Sun, 21 Apr 2024 at 20:19, Volkan Yazıcı  wrote:
>>>   1. Could you show us the Antora configuration option you mentioned
>>>   and how we can use it to achieve what you propose?
>> 
>> I found the perfect Antora setting: `html_extension_style`[1].
>> 
>> The option I am proposing corresponds to the `drop` style:
>> 
>> * a `/foo/bar.html` file will be referenced as `foo/bar`,
>> * a `/foo/index.html` file will be referenced as `foo/`.
>> 
>> The `indexify` style is very similar, but it always uses a trailing
>> `/` for the file names.
>> 
>> I see both pros and cons for the two styles:
>> 
>> ## `indexify` style
>> 
>> Pros:
>> * Doesn't make a difference between "normal" HTML files and folders.
>> If we transform `foo.html` into `foo/index.html` and add subpages, the
>> URL remains always `foo/`.
>> * We restore the old URLs like `/log4j/2.x/log4j-api/` that became
>> `/log4j/2.x/log4j-api.html`.
>> * Works on every HTTP server (even Python's).
>> 
>> Cons:
>> * We need a lot of HTTP redirects like
>> `/log4j/2.x/manual/configuration.html` ->
>> `/log4j/2.x/manual/configuration/`
>> 
>> ## `drop` style
>> 
>> Pros:
>> * We don't need redirects for the current pages, only a global rewrite
>> rule that states that we prefer to omit the `.html` suffix.
>> * It is shorter than the `indexify` style.
>> * It is easier to implement on already compiled pages: no need to
>> move/rename files.
>> 
>> Cons:
>> * If `foo.html` becomes `foo/index.html` the canonical URL changes
>> from `foo` to `foo/`. However the redirect from the old to the new URL
>> is done automatically by most servers.
>> * It doesn't work with all web servers, but it works with Apache HTTP
>> Server.
>> 
>> What do you think about adopting the `drop` style?
>> 
>> Piotr
>> 
>> PS: Javadoc also can use the `drop` style. See e.g. Jakarta drops the
>> `.html` (and apparently capital letters) from its Javadoc.
>> 
>> [1]
>> https://docs.antora.org/antora/latest/playbook/urls-html-extension-style/
>> [2]
>> https://jakarta.ee/specifications/servlet/6.0/apidocs/jakarta.servlet/jakarta/servlet/filter
>> 

Re: Closing inactive issues

[Ralph Goers]

Periodically I have tried to go through old issues and close those that I know 
will get no activity. I’d prefer to do that than just close them all. Now that 
Jira is locked down at least the list of issues there will never grow.

In doing that I would actually recommend creating an issue at GitHub for any 
issues that should remain active (even if the contents are just a link to Jira) 
and still close it in Jira. So in the end all the Jira issues should be closed.

Ralph

> On Apr 24, 2024, at 6:18 AM, Piotr P. Karwasz  wrote:
> 
> Hi all,
> 
> We have currently almost 1000 issues for the Log4j project alone.
> 
> Let us be honest, most of these issues will **never** be solved, we
> barely have time for the issues reported against the current Log4j
> version.
> 
> What do you think about automatically closing as "Won't Fix" or
> "Abandoned" the issues without any activity in the last 2 years[1]?
> The issues will not disappear from JIRA, but we will no longer spend
> time on them (we don't look at them anyway).
> 
> Piotr
> 
> [1] 
> https://issues.apache.org/jira/browse/LOG4J2-3079?filter=-4&jql=project%20%3D%20LOG4J2%20AND%20resolution%20%3D%20Unresolved%20AND%20updated%20%3C%3D%20-104w

Re: Canonicalization of URLs on our website

[Ralph Goers]

I have always viewed index.html as a special case. When navigating to the root 
of a site - l.a.o/log4j/2.x/ - should be sufficient as it should default to 
index.html. However, the “real” url includes index.html. 

Other pages should always be whatever.html IMO. 

Ralph

> On Apr 20, 2024, at 1:17 PM, Piotr P. Karwasz  wrote:
> 
> Hi,
> 
> I scanned our https://logging.apache.org/ website and found out that
> the internal hyperlinks between our pages are not consistent. For
> example links to:
> 
> https://logging.apache.org/log4j/2.x/
> 
> might appear in hyperlinks with an URI path of:
> 
> * `/log4j/2.x` (which causes a 301 HTTP redirect),
> * `/log4j/2.x/`,
> * `/log4j/2.x/index.html`.
> 
> This lack of uniformity can cause several problems:
> 
> * search engines might treat those 3 links as equivalent, but not necessarily.
> * if an `index.html` file is moved, we need to provide a redirect for
> all 3 alternatives: a recent example is
> `/log4j/2.x/log4j-1.2-api/index.html` that was moved to
> `/log4j2/2.x/log4j-1.2-api.html`.
> * for the rare people that actually look at the URL of a page, it
> doesn't seem coherent.
> 
> So I would propose to adopt only one of the 3 alternatives and stick
> to it as much as possible? Which one should we choose?
> 
> The simplest one (`/log4j/2.x/index.html`) does not require a Web
> server and can be viewed locally and can be viewed using the `file:`
> scheme in a browser. However I find it less elegant than
> `/log4j/2.x/`.
> Antora is probably able to generate both versions through some
> configuration option, so choosing `/log4j/2.x/` does not preclude the
> possibility to generate a different version to check the web site
> locally.
> 
> Another canonicalization we might apply regards trailing `.html`
> extensions in the URL. The current website supports both:
> 
> * `/log4j2/log4j-api`,
> * `/log4j2/log4j-api.html`.
> 
> through `mod_negotiation`. Should we use the version with a trailing
> `.html` or without it? The `https://apache.org/` website hides the
> `.html` extension in most the links.
> 
> Piotr

Re: End-of-life date for `log4j-1.2-api`

[Ralph Goers]

Did we make a decision on this?

Ralph

> On Apr 8, 2024, at 4:02 PM, Ralph Goers  wrote:
> 
> 
> 
>> On Apr 8, 2024, at 2:40 PM, Jochen Wiedmann  
>> wrote:
>> 
>> On Mon, Apr 8, 2024 at 1:11 PM Apache  wrote:
>> 
>>> My opinion is to drop it from 3.0.0. 2.x is going to live a long time 
>>> still. By the time it dies Log4J 1.x will have been dead well over 15 
>>> years, maybe even 20. That would give users plenty of time to be aware that 
>>> they need to plan to upgrade.
>> 
>> How long ago was it, that all these JNDI, and JMS related issues where
>> found? Yes, three years. And I remember very well, how customers
>> basically stormed my employers house, because some ancient code (which
>> should have been updated years ago) is using these "dead" libraries.
>> 
>> And, do you remember also, how long it took at the time, to push out
>> 1.2.18? Wait, that was never published? Instead, we have
>> https://github.com/qos-ch/reload4j.
>> 
>> Please, just because you think, that you can master these things:
>> Don't assume, that others can.
> 
> I think you missed the point. Log4j 3.x will ONLY support Java 17 and above. 
> That would mean that someone would have had to port & verify that code 
> written for pre-Java 6 now runs on Java 17+ in order to use Log4j 3.x to 
> support their Log4j 1.x application.
> 
> As I said, Log4j 2.x isn’t going away any time soon. Even then, I suspect 
> that log4j-1.2-api from Log4j 2.x will probably work with the rest of Log4j 
> 3.x especially now that Log4j API is staying at 2.x (although at some point 
> its minimum JDK will be upgraded). It is possible that we may decide to fork 
> the 1.2 API module into its own repo after the rest of 2.x is retired. Who 
> knows? That will be years from now.  
> 
> All we are deciding here is to NOT include it in 3.x.
> 
> Ralph

Re: Context data propagation and logger-bound context data

[Ralph Goers]

In theory that library could use the ContextData class I just added to get 
context data no matter how it got there. However, the Micrometer library’s 
setValue and getValue set and retrieve the full map, not values. That won’t 
play nice with a ScopedContext since once you set it you would immediately be 
leaving the context so what you set would be instantly forgotten. Under the 
covers of course it could push a new map on the queue but that would likely 
behave very differently than the way ScopedContext does - i.e. you are 
completely replacing maps not incrementally modifying them.

Ralph

> On Apr 15, 2024, at 10:07 AM, Piotr P. Karwasz  
> wrote:
> 
> Hi,
> 
> On Mon, 15 Apr 2024 at 18:44, Matt Sicker  > wrote:
>> Context Propagation support :: Micrometer Context Propagation
>> docs.micrometer.io
>> 
>>  Context 
>> Propagation support :: Micrometer Context Propagation 
>> 
>> docs.micrometer.io 
>> 
>> 
>> 
>> This is more in line with what I was thinking we should support. I’m not 
>> suggesting that we add Micrometer as a dependency to the API or Core, but we 
>> can replicate the minimal API here or figure out a smaller API for 
>> integration purposes.
> 
> I agree. To support `context-propagation` we don't need to add anything to 
> the API or SPI. To support it efficiently, we need the two methods from PR 
> [1].
> 
> Note that SLF4J is already supported by `context-propagation` (cf. [2]).
> 
> Piotr
> 
> [1] https://github.com/apache/logging-log4j2/pull/2442
> [2] https://github.com/micrometer-metrics/context-propagation/pull/194

Re: Should Log4j API bridges have a 3.x release?

[Ralph Goers]

> On Apr 9, 2024, at 3:52 PM, Piotr P. Karwasz  wrote:
> 
> 
> BTW: Maybe SLF4J still uses Java 8, but the latest Logback uses Java 11.

Ask me if I care about Logback. ;-)

Ralph

Re: Should Log4j API bridges have a 3.x release?

[Ralph Goers]

You are forgetting one small thing.  Plugins in 3.x use ServiceLoader while 2.x 
uses the Log4j2Plugins.dat file. While 3.x supports the old format it will be 
much better for users to not have to use the transformer when building uber 
jars as most tooling supports ServiceLoader out of the box.

Not only is 3.x compiled with Java 17 that is also the target version. If the 
component they are targeting also has a minimum version of Java 17 then it 
definitely makes sense IMO to have then in 3.x. However, SLF4J still targets 
Java 8 and I haven’t seen any indication that will change anytime soon.

I do agree with your assumption about Spring Boot though. 

Ralph

> On Apr 9, 2024, at 12:34 PM, Piotr P. Karwasz  wrote:
> 
> Hi all,
> 
> Since Log4j Core 3.x moved to `log4j-api` 2.24.0, many artifacts that
> only depend on `log4j-api` became redundant in the 3.x branch:
> 
> * `log4j-iostreams`,
> * `log4j-jpl`,
> * `log4j-jul`,
> * `log4j-slf4j-impl`,
> * `log4j-slf4j2-impl`,
> * `log4j-to-jul`,
> * `log4j-to-slf4j`.
> 
> Right now the `3.x` version of these artifacts is basically the same
> as the `2.x` version, except it is compiled with Java 17 instead of
> Java 8. They could have some regression though.
> 
> What should we do with those artifacts? They are already JPMS modules,
> they don't need dependency injection, they barely use 2 Log4j system
> properties (in `log4j-jul`, but JUL by definition has a single global
> logger context, so these props must be global).
> 
> I would propose to remove them from the `main` branch and just
> reference the `2.24.0` version in the `3.0.0` version of `log4j-bom`.
> 
> It will be easier to convince Spring Boot to bump the version of
> `log4j-bom` to `3.0.0` if the bump will not change **anything** in
> `spring-boot-starter-logging` (the default Logback-based stack) and
> will only change the version of `log4j-core` in
> `spring-boot-starter-log4j2`.
> 
> Piotr

Re: End-of-life date for `log4j-1.2-api`

[Ralph Goers]

> On Apr 8, 2024, at 2:40 PM, Jochen Wiedmann  wrote:
> 
> On Mon, Apr 8, 2024 at 1:11 PM Apache  wrote:
> 
>> My opinion is to drop it from 3.0.0. 2.x is going to live a long time still. 
>> By the time it dies Log4J 1.x will have been dead well over 15 years, maybe 
>> even 20. That would give users plenty of time to be aware that they need to 
>> plan to upgrade.
> 
> How long ago was it, that all these JNDI, and JMS related issues where
> found? Yes, three years. And I remember very well, how customers
> basically stormed my employers house, because some ancient code (which
> should have been updated years ago) is using these "dead" libraries.
> 
> And, do you remember also, how long it took at the time, to push out
> 1.2.18? Wait, that was never published? Instead, we have
> https://github.com/qos-ch/reload4j.
> 
> Please, just because you think, that you can master these things:
> Don't assume, that others can.

I think you missed the point. Log4j 3.x will ONLY support Java 17 and above. 
That would mean that someone would have had to port & verify that code written 
for pre-Java 6 now runs on Java 17+ in order to use Log4j 3.x to support their 
Log4j 1.x application.

As I said, Log4j 2.x isn’t going away any time soon. Even then, I suspect that 
log4j-1.2-api from Log4j 2.x will probably work with the rest of Log4j 3.x 
especially now that Log4j API is staying at 2.x (although at some point its 
minimum JDK will be upgraded). It is possible that we may decide to fork the 
1.2 API module into its own repo after the rest of 2.x is retired. Who knows? 
That will be years from now.  

All we are deciding here is to NOT include it in 3.x.

Ralph

Re: Context data propagation and logger-bound context data

[Ralph Goers]

Volkan,

No more hand waving. Please see 
https://github.com/apache/logging-log4j2/pull/2419.

I should note that while implementing the classes I added to support this makes 
it easier I did not have to make any changes to the logging internals to make 
this work.

Ralph

> On Mar 22, 2024, at 1:45 AM, Volkan Yazıcı  wrote:
> 
> No, it is not the same thing Matt. Let me be as explicit as I can:
> 
> var logger0 = getLogger();  // MDC: {}
> var logger1 = logger0.withContextData("x", 1);  // MDC: {x: 1}
> var logger2 = logger1.withContextData("y", 2);  // MDC: {x: 1, y: 2}
> 
> This is the functionality being requested. Whoever claims this can be done 
> using a `MessageFactory`, they need to share a working [pseudo] code, instead 
> of hand waving. So far, nobody responded to this. Piotr, speculated on a 
> non-existing `Logger#withMessageFactory(MessageFactory)`, but there is not 
> one single working example shared. Hence, unless you can prove me wrong with 
> a working practical[1] example, the requested feature is currently known to 
> be not practically possible in Log4j.
> 
> [1] Implementing `logger.withContextData("x", 1)` with 50 LoC Java code using 
> the existing Log4j feature set is not a "practical example".
> 
> P.S. For Log4j 3 API Javadocs, you can browse to 
> https://logging.apache.org/log4j/3.x and search for "Javadoc" in the menu. 
> (Obviously, same works for Log4j 2 too.)
> 
> On Thu, Mar 21, 2024 at 6:10 PM Matt Sicker  wrote:
> LogManager - log4j-api 3.0.0-alpha1 javadoc
> javadoc.io
> 
> Pass your custom MessageFactory here. It’s an optional argument when creating 
> the Logger.
> 
> Also, I’m not sure where to even find the current javadocs for the API. 
> javadoc.io only seems to have this alpha release.
> 
> 
>> On Mar 21, 2024, at 04:34, Volkan Yazıcı  wrote:
>> 
>> Ralph, could you show how those two users can use a `MessageFactory` to
>> create `Logger`s with predefined additional context data?
>> 
>> On Thu, Mar 21, 2024 at 7:25 AM Ralph Goers  wrote:
>> 
>>> Unfortunately this is another message I somehow didn't get in my inbox.
>>> Replying to it via lists.a.o is not a great experience but is the best I
>>> can do.
>>> 
>>> On 2024/03/20 13:51:56 Volkan Yazıcı wrote:
>>>> I agree with the way Piotr dissects the problem. I think `ScopedContext`,
>>>> even though it has its own merits, doesn't address the problem reported
>>> by
>>>> users. They simply want a new logger associated with some additional
>>>> context data.
>>> 
>>> Two users do.  I have personally been asked for something like
>>> ScopedContext several times.
>>> As I replied to Piotr, we already solved the problem of adding data to
>>> Loggers. That is what MessageFactories are intended for.
>>> 
>>>> 
>>>> *[See my comments below.]*
>>>> 
>>>> On Mon, Mar 18, 2024 at 10:40 AM Piotr P. Karwasz <
>>> piotr.karw...@gmail.com>
>>>> wrote:
>>>> 
>>>>> * we can create a `Logger` wrapper "bound" to context data as Mikko
>>>>> does. This wrapper will take care of setting the `ThreadContext`
>>>>> before the logger call and restore it after it.
>>>> 
>>>> Creating a wrapper `Logger` can work without needing to deal with
>>>> `ThreadContext`. I can think of two different ways to carry this out:
>>>> 
>>>>   1. Currently, `AbstractLogger` only creates `Message`s. We can rework
>>> it
>>>>   to create `LogEvent`s too. Once `AbstractLogger` gets its hand on a
>>>>   `LogEvent`, it can enrich its context data as it wishes.
>>>>   2. We can extend `ContextDataInjector` with a new `void
>>>>   injectContextData(Logger logger, StringMap target)` method, provide a
>>>>   `ContextDataInjector` implementation that branches on `logger
>>> instanceof
>>>>   ContextDataProvider`, and call `ContextDataInjector` with the
>>> associated
>>>>   `Logger` in `LogEventFactory`.
>>> 
>>> We can do lots of things, most of which I wouldn't recommend. As to yours:
>>> 1. Logger/AbstractLogger got very complex with Async, Garbage Free,
>>> Reliablity Strategies, etc. Trying to move creating the LogEvent sooner is
>>> likely to be a major PITA and could seriously impact performance. While we
>>> could add a context map to AbstractLogger we would have to pass that on the
>>> logging calls to LoggerConfig and deal wi

Re: Context data propagation and logger-bound context data

[Ralph Goers]

> On Mar 22, 2024, at 1:45 AM, Volkan Yazıcı  wrote:
> 
> No, it is not the same thing Matt. Let me be as explicit as I can:
> 
> var logger0 = getLogger();  // MDC: {}
> var logger1 = logger0.withContextData("x", 1);  // MDC: {x: 1}
> var logger2 = logger1.withContextData("y", 2);  // MDC: {x: 1, y: 2}
> 
> This is the functionality being requested. Whoever claims this can be done 
> using a `MessageFactory`, they need to share a working [pseudo] code, instead 
> of hand waving. So far, nobody responded to this. Piotr, speculated on a 
> non-existing `Logger#withMessageFactory(MessageFactory)`, but there is not 
> one single working example shared. Hence, unless you can prove me wrong with 
> a working practical[1] example, the requested feature is currently known to 
> be not practically possible in Log4j.

I am working on this Volkan and with real code. It will be included in my PR 
branch shortly. You simply have to be patient.

Ralph

Re: Context data propagation and logger-bound context data

[Ralph Goers]

> On Mar 21, 2024, at 3:19 PM, Piotr P. Karwasz  wrote:
> 
> 
> PS: In the `main` branch I plan to replace the default
> ReusableMessageFactory with ReusableLogEventFactory:
> 
> * this change should have a minimal impact on memory (events are not
> much larger than messages),
> * it can skip a data transfer between ReusableMessage and MutableLogEvent,
> * it allows the message factory to collect data that is usually
> reserved for events.

I have to question this. I don’t understand the concept of replacing a 
MessageFactory with a LogEventFactory. They are completely different things. 
What does this even mean?

Ralph

Re: Context data propagation and logger-bound context data

[Ralph Goers]

> On Mar 21, 2024, at 2:48 PM, Raman Gupta  wrote:
> 
> On Thu, Mar 21, 2024 at 5:30 AM Piotr P. Karwasz 
> wrote:
> 
>> Hi Ralph,
>> 
>> On Thu, 21 Mar 2024 at 07:03, Ralph Goers 
>> wrote:
>>>> 1. Raman and Mikko would like to bind context data to an object
>>>> implementing the `Logger` interface or more generally to a service
>>>> object (e.g. resource manager, DAO and variants),
>>> 
>>> Yes, I’ve seen that. Personally, I am not much of a fan of this use case
>> as it is pretty easy to add the data you want to a single class. That said,
>> we already offer a solution. Allowing a MessageFactory to be provided on a
>> Logger was done for exactly this reason.
>>> 
>>> For example, a User could configure a custom MessageFactory that
>> provides an extension of MapMessage that causes the message to include data
>> from the class or resource. Going to the extreme of trying to shoehorn in
>> Context data as well simply isn’t necessary.
>> 
>> Great point! This effectively solves Raman's and Mikko's problem. We
>> just need to introduce a new sub-interface of `Message` and require
>> all implementations to support it.
>> 
>> 
> I'm not sure it does. I differentiate between context and message. When I
> want the data from the resource in the context, it's because the data is
> contextual, not part of the message.
> 
> This approach seems to be not much better than making sure all log messages
> in a class have a consistent string format so it can be grepped.

No, it is nothing like that. LogEvents effectively support 2 kinds of Maps; 
those associated with the ThreadContext (or injected into the LogEvents 
contextMap, or key/value pairs associated with MapMessages. In the 
PatternLayout these are distinguished by %X and %K. If you know the keys you 
want you can use a pattern like:

%d %p %C{1.} [%t] {requestId=%X{requestId} customerNumber=%X{customerNumber} 
userName=%K{userName} %{userType} - %m%n

Thus making the ThreadContext key/values and Message key/values virtually 
indistinguishable. 

The only issue at the moment is the result of %m on a MapMessage isn’t really 
what you would like. I am looking into addressing that as I have run across 
this before in things I have tried to do.

Ralph

Re: Context data propagation and logger-bound context data

[Ralph Goers]

Unfortunately this is another message I somehow didn't get in my inbox. 
Replying to it via lists.a.o is not a great experience but is the best I can do.

On 2024/03/20 13:51:56 Volkan Yazıcı wrote:
> I agree with the way Piotr dissects the problem. I think `ScopedContext`,
> even though it has its own merits, doesn't address the problem reported by
> users. They simply want a new logger associated with some additional
> context data.

Two users do.  I have personally been asked for something like ScopedContext 
several times.
As I replied to Piotr, we already solved the problem of adding data to Loggers. 
That is what MessageFactories are intended for.

> 
> *[See my comments below.]*
> 
> On Mon, Mar 18, 2024 at 10:40 AM Piotr P. Karwasz 
> wrote:
> 
> > * we can create a `Logger` wrapper "bound" to context data as Mikko
> > does. This wrapper will take care of setting the `ThreadContext`
> > before the logger call and restore it after it.
> 
> Creating a wrapper `Logger` can work without needing to deal with
> `ThreadContext`. I can think of two different ways to carry this out:
> 
>1. Currently, `AbstractLogger` only creates `Message`s. We can rework it
>to create `LogEvent`s too. Once `AbstractLogger` gets its hand on a
>`LogEvent`, it can enrich its context data as it wishes.
>2. We can extend `ContextDataInjector` with a new `void
>injectContextData(Logger logger, StringMap target)` method, provide a
>`ContextDataInjector` implementation that branches on `logger instanceof
>ContextDataProvider`, and call `ContextDataInjector` with the associated
>`Logger` in `LogEventFactory`.

We can do lots of things, most of which I wouldn't recommend. As to yours:
1. Logger/AbstractLogger got very complex with Async, Garbage Free, Reliablity 
Strategies, etc. Trying to move creating the LogEvent sooner is likely to be a 
major PITA and could seriously impact performance. While we could add a context 
map to AbstractLogger we would have to pass that on the logging calls to 
LoggerConfig and deal with all that that means - remember, a LoggerConfig can 
be handling multiple Loggers.
2. I don't recommend extending ContextDataInjector. That proved difficult to 
work with which is why we now recommend using ContextDataProviders. You really 
can only have one ContextDataInjector. Also, please note that 
ContextDataInjector is called while constructing the LogEvent. The LogEvent 
isn't passed the Logger, only the LoggerName. Looking up the Logger to do this 
is yet another way to slow down logging.

> 
> On Tue, Mar 19, 2024 at 7:45 AM Ralph Goers 
> wrote:
> > In the meantime, I provided
> https://github.com/apache/logging-log4j2/pull/2385 which I very loosely
> modeled after ScopedValues.
> 
> The fact that `ScopedContext` tries to imitate `ScopedValue` using
> `ThreadLocal`s is extremely confusing (from a user's pov) and risky
> liability (from a maintainer's pov). I guess you wanted to implement *a*
> `ScopedValue` without using *the* `ScopedValue` to be compatible with Java
> 8. If so, that really sounds like the `o.a.l.log4j.util.Supplier` downward
> spiral. We can rather have an *internal* `Log4jScopedValue` interface and
> provide Java 8 (using `InheritableThreadLocal`) and Java 21+ (using
> `ScopedValue`) compatible solutions in an MRJ (Multi-Release JAR).

I am NOT trying to make ScopedContext compatible with ScopedValue. I am trying 
to make it conceptually close enough to ScopedValue that users will understand 
what it is doing.
We can argue about naming if you want. Gary has already expressed his opinion.
> 
> We can integrate `ScopedContext` to the `LogEventFactory` by providing a
> specialized `ContextDataInjector` plugin – assuming `LogEventFactory`
> employs all available `ContextDataInjector` plugins.

ScopedContext is integrated with a ContextDataProvider, which is the supported 
way to do this. Again, you cannot have more than one ContextDataInjector so 
providing "specialized versions" is a pipe dream. You will simply have to 
enhance the one we already have. ContextDataInjector is NOT a plugin.

> 
> I find the current ceremony also too long:
> `ScopedContext.getCurrent().where("key1", "value1").run(...)`. I would
> rather aim for `ScopedContext.run(key, value, runnable)` and similar
> `ScopedContext.op(..., runnable)` interaction.

Those are going to be provided as well.

Ralph

Re: Context data propagation and logger-bound context data

[Ralph Goers]

> On Mar 18, 2024, at 2:39 AM, Piotr P. Karwasz  wrote:
> 
> Hi,
> 
> Given the ongoing context data propagation in three Github
> issues/PRs/discussions (cf. [1], [2] and [3]). I think we should move
> the debate to the mailing list, which guarantees the maximal reachout.
> 
> == Problem summary
> 
> We have two different problems regarding context data in log files:
> 
> 1. Raman and Mikko would like to bind context data to an object
> implementing the `Logger` interface or more generally to a service
> object (e.g. resource manager, DAO and variants),

Yes, I’ve seen that. Personally, I am not much of a fan of this use case as it 
is pretty easy to add the data you want to a single class. That said, we 
already offer a solution. Allowing a MessageFactory to be provided on a Logger 
was done for exactly this reason.

For example, a User could configure a custom MessageFactory that provides an 
extension of MapMessage that causes the message to include data from the class 
or resource. Going to the extreme of trying to shoehorn in Context data as well 
simply isn’t necessary.


> 2. Other users would like to bind context data to a processing
> pipeline, regardless of which threads it uses.
> 

ScopedContext covers this use case and is the one I find far more compelling. 
That is why I am adding support for passing data to Threads to my PR. I will 
let you know when it is there.

Ralph

Re: Context data propagation and logger-bound context data

[Ralph Goers]

1. I am experimenting with adding Thread support to ScopedContext as we speak. 
It should be straightforward. 

I am not sure we ever want to deal with ScopedValue. ScopedContext provides the 
same functionality plus integration with logging.

Ralph

> On Mar 20, 2024, at 4:22 AM, Piotr P. Karwasz  wrote:
> 
> Hi Ralph,
> 
> On Tue, 19 Mar 2024 at 07:45, Ralph Goers  wrote:
>> 2. The links you provide describe the problem of passing contexts to threads 
>> magnificently. Unfortunately, they solve it by requiring you NOT to use 
>> standard JDK tooling for managing threads. I am not prepared to be dependent 
>> on any specific framework other than the JDK.
>> ...
>> 3. As I stated, I am not prepared to provide a solution that is dependent on 
>> another framework. However that framework is welcome to integrate with us.
> 
> I would propose to meet those libraries half-way:
> 
> * we extend our SPI to provide a fast method to copy the contents of
> the ThreadLocal used by ThreadContextMap, restore it on another thread
> or clear it,
> * the `context-propagation` library integrates our newly added SPI.
> 
>> 2. As I noted, adding a wrapper around a Logger doesn’t accomplish anything, 
>> at least in terms of getting data into LogEvents.
> 
> The wrapper could do something like:
> 
> public void info(String message) {
>if (logger.isEnabled(null, Level.INFO, message)) {
>try (CloseableThreadContext.Instance ignored =
> CloseableThreadContext.putAll(context)) {
>logger.info(message);
>}
>}
> }
> 
> using only Log4j API. Log4j Core could do better of course.
> 
>> SLF4J has recently introduced 
>> https://www.slf4j.org/apidocs-2.1.0-alpha0/org.slf4j/org/slf4j/MDCAmbit.html 
>> which to me looks like a horrible way to add a “scope-based” context as it 
>> relies on the user to perform cleanup.
> 
> This is a horrible workaround of the way "try-with-resources" work,
> when "catch" clauses are also present. Since the resource is closed
> **before** the "catch" code is executed, the new context keys are no
> longer available in the "catch" clauses.
> 
>> Java 21 introduces ScopedValues - 
>> https://docs.oracle.com/en/java/javase/21/docs/api/java.base/java/lang/ScopedValue.html.
>>  It does allow ScopedValues to be passed to child threads but I haven’t 
>> experimented with it enough yet to know how it really works.
> 
> In the future we will certainly switch to this, but for now I would
> keep all usages of `ScopedValue` in an `o.a.l.l.experimental` package,
> so that users know that we can remove it any time we want.
> 
> Piotr

Re: Please update Flume DOAP

[Ralph Goers]

Interesting. I hadn’t noticed that the GitHub repos had changed urls.

Ralph

> On Mar 19, 2024, at 1:40 PM, Jan Friedrich  wrote:
> 
> Hi Sebb,
> 
> is this what you want?
> 
> https://github.com/apache/logging-flume/pull/421
> 
> Regards.
> 
> Jan
> 
> Tuesday, March 19, 2024, 6:44:10 PM, you wrote:
> 
>> PING - still needs to be fixed.
> 
>> On Fri, 15 Mar 2024 at 00:42, sebb  wrote:
>>> 
>>> Note: this is the line that needs to be fixed:
>>> 
>>> https://github.com/apache/logging-flume/blob/trunk/doap_Flume.rdf#L29
>>> 
>>> On Thu, 14 Mar 2024 at 08:57, sebb  wrote:
 
 Ping?
 
 On Thu, 7 Mar 2024 at 11:42, sebb  wrote:
> 
> https://github.com/apache/logging-flume/blob/trunk/doap_Flume.rdf
> still shows the PMC as Flume; please update to show PMC Logging.
> 
> Thanks,
> Sebb
> 

Re: Context data propagation and logger-bound context data

[Ralph Goers]

First, lets consider the two usages you highlighted

1. In theory it is possible to add properties to a Logger. In practice, we only 
allow this from the LoggerConfig. While the values of those properties can 
include Lookup references this isn’t going to help much since there is no way 
for the Lookup to reference any specific object instance. While the user could 
override Logger they would also have to override AsyncLogger for a complete 
solution, but even doing that isn’t going to help much since nothing in 
Logger/AbstractLogger creates a LogEvent so does not provide any place to add 
properties associated with the instance. Creating a context at the class level 
is certainly level but the only way to locate it is via a ThreadLocal. This 
introduces problems of having objects tied to classes and ClassLoaders and 
means Thread termination may hang if everything isn’t recycled correctly.  In 
short, the solution for this is complex.

2. The links you provide describe the problem of passing contexts to threads 
magnificently. Unfortunately, they solve it by requiring you NOT to use 
standard JDK tooling for managing threads. I am not prepared to be dependent on 
any specific framework other than the JDK. 

Specifics about your solutions:

1. Again, adding Property elements to a Logger doesn’t really help for the use 
cases requested. We support this now and if it was sufficient we wouldn’t have 
these requests.
2. As I noted, adding a wrapper around a Logger doesn’t accomplish anything, at 
least in terms of getting data into LogEvents.
3. As I stated, I am not prepared to provide a solution that is dependent on 
another framework. However that framework is welcome to integrate with us.

Other items of note:
SLF4J has recently introduced 
https://www.slf4j.org/apidocs-2.1.0-alpha0/org.slf4j/org/slf4j/MDCAmbit.html 
which to me looks like a horrible way to add a “scope-based” context as it 
relies on the user to perform cleanup.
Java 21 introduces ScopedValues - 
https://docs.oracle.com/en/java/javase/21/docs/api/java.base/java/lang/ScopedValue.html.
 It does allow ScopedValues to be passed to child threads but I haven’t 
experimented with it enough yet to know how it really works.

In the meantime, I provided https://github.com/apache/logging-log4j2/pull/2385 
which I very loosely modeled after ScopedValues. ScopedContext allows you to do:

public class SomeClass {

private static final Logger LOGGER = 
LogManager.getLogger(SomeClass.class);

public void someMethod(String arg1, Request request) {
ScopedContext.INITIAL_CONTEXT
.where(“param”, “arg1”)
.where(“loginId”, request.getLoginid())
.run(() -> {
LOGGER.info (“This will 
include param and loginId in the LogEvent’s ThreadContext Map”);
});
} 

Nesting of ScopedContext’s is allowed. They can be used in place of 
ScoedVallues but also allow them to included in the LogEvent. A ScopedContext 
can also be saved and passed to another thread. Because ScopedContexts are 
immutable there is no risk in them being shared across threads. It simply 
requires the ScopedContext’s run method be called on the child thread to 
register it and ensure it is cleaned up when no longer needed.

Ralph

> On Mar 18, 2024, at 2:39 AM, Piotr P. Karwasz  wrote:
> 
> Hi,
> 
> Given the ongoing context data propagation in three Github
> issues/PRs/discussions (cf. [1], [2] and [3]). I think we should move
> the debate to the mailing list, which guarantees the maximal reachout.
> 
> == Problem summary
> 
> We have two different problems regarding context data in log files:
> 
> 1. Raman and Mikko would like to bind context data to an object
> implementing the `Logger` interface or more generally to a service
> object (e.g. resource manager, DAO and variants),
> 2. Other users would like to bind context data to a processing
> pipeline, regardless of which threads it uses.
> 
> == Possible solutions
> 
> As far as I know there are currently the following approaches we can
> take for problem 1:
> 
> * we can add a list of `` elements in the configuration of a
> ``, which will add the given keys to all loggers using that
> configuration. This is something we can do for very static data, e.g.
> we can add to each log statement the name of the library that issued
> it.
> * we can create a `Logger` wrapper "bound" to context data as Mikko
> does. This wrapper will take care of setting the `ThreadContext`
> before the logger call and restore it after it.
> 
> For problem 2, the traditional solution was provided by the
> inheritability of `ThreadLocal`s, but this approach only works if the
> current thread creates new ones, it does not work with thread pools.
> 
> Now there seems to be another emerging solution provided by the
> `context-propagation` [9] library and explained by Dariusz Jędrzejczyk
> in his blog series

Re: [VOTE] Release Apache Log4net 2.0.16

[Ralph Goers]

+1 

Verified signatures
Verified hashes
Verified License and Notice files.

Note - the copyright year should be the first year the code was created. You 
can update it to include “-(currentYear}” but that is not strictly necessary.

Ralph

> On Mar 4, 2024, at 10:48 AM, Jan Friedrich  wrote:
> 
> +1
> 
> Unit tests on my machine were successful.
> We integrated the new version into our test environment and all manual tests 
> were successful.
> 
> Jan
> 
>> +1
> 
>> Verified signatures.
>> Verified hashes.
> 
>> `NOTICE` contains 2022 as the copyright year, but I don't find it a
>> blocker. (I have fixed it in `master`.)
> 
>> On Fri, Mar 1, 2024 at 2:19 PM Davyd McColl  wrote:
> 
>>> Hi all
>>> 
>>> 
>>> This is the vote to release Apache log4net version 2.0.16
>>> 
>>> 
>>> Website:
>>> https://logging.staged.apache.org/log4net/release/release-notes.html
>>> 
>>> GitHub: https://github.com/apache/logging-log4net
>>> 
>>> GitHub release (pre-release):
>>> https://github.com/apache/logging-log4net/releases/tag/2.0.16-rc1
>>> 
>>> Distribution: I'm not sure -
>>> https://dist.apache.org/repos/dist/dev/logging/log4net should have
>>> 2.0.16 binaries and source (I've added via SVN), but I'm not seeing
>>> them. Any help appreciated.
>>> 
>>> 
>>> 
>>> Please have a look at the staging site & artifacts and test (if you can
>>> - clone, `npm i`, `npm test`)
>>> 
>>> [ ] +1, release the artifacts
>>> 
>>> [ ] -1, don't release, because
>>> 
>>> 
>>> (thanks Piotr: I copied most of your last VOTE mail!)
>>> 
>>> 
>>> -d
>>> 
>>> 
> 

Re: Removal of 2.3.x and 2.12.x websites (was: Clean site repo)

[Ralph Goers]

> On Mar 4, 2024, at 6:24 AM, Piotr P. Karwasz  wrote:
> 
> There is a further simplification possible: since Oracle's extended
> support for Java 7 has expired in July 2022, shouldn't we also
> redirect the `2.3.x` and `2.12.x` websites to `2.x`?

Why would we do that? I mean we don’t officially support Java 6 or 7 yet we did 
create patches for them for Log4Shell. While I hope they never need to be 
updated again I don’t think it hurts anything to leave the doc alone.

Ralph

Re: Checksum of release 2.23.0 does not seem to be correct

[Ralph Goers]

> On Mar 1, 2024, at 6:55 AM, Piotr P. Karwasz  wrote:
> 
> Hi Piers,
> 
> On Fri, 1 Mar 2024 at 14:14, Piers Uso Walter  wrote:
>> Thanks for the quick response.
>> And yes, everything is OK on your side.
>> 
>> I did indeed somehow manage to download the HTML file as the zip archive.
>> That explains why the checksum was wrong.
>> 
>> How embarrassing:-(
> 
> I did the exact same thing, that is why I remarked it in the answer.
> 
> Maybe we should replace the links on the web page? There are actually
> people (like me and probably you) that don't download everything
> through the browser.
> What do you think?

Replace them with what? We are required to use the chooser app so that the user 
downloads from an archive, not the main ASF repo.

Ralph

Re: Distribution location was: [VOTE] Release Apache Log4j 3.0.0-beta2 RC1

[Ralph Goers]

Just navigate to https://archive.apache.org/dist/logging/ and take a look. Note 
that under log4j you will find version directories. Under log4j-audit you will 
not find directories. Basically it is however they get added to 
https://dist.apache.org/repos/dist/dev/logging/. It is just a mirror of what 
gets committed there.

I would suggest it makes sense to use version directories on a project that has 
lots of releases.  For consistency it would make sense to do that everywhere 
but that ship has already sailed for existing projects.

Ralph

> On Feb 18, 2024, at 10:09 AM, Piotr P. Karwasz  
> wrote:
> 
> Hi Gary,
> 
> On Sun, 18 Feb 2024 at 15:12, Gary Gregory  wrote:
>> Note that the verification instructions below or the distro process or
>> both need changes because the wget and all commands get EVERYTHING and
>> work on EVERYTHING, which in this case means that BOTH release
>> candidates for 2.23.0 and 3.0.0-beta2 are downloaded and instructions
>> work on both at the same time, obviously not the intent.
>> 
>> Over at Commons, we account for this use case by using the version in
>> the distro folder, so, for example, here you'd RC in
>> https://dist.apache.org/repos/dist/dev/logging/log4j/3.0.0-beta2, not
>> https://dist.apache.org/repos/dist/dev/logging/log4j
> 
> +1 for version specific folders.
> 
> You didn't notice it but the CI removed the 2.23.0 distribution files,
> when it committed 3.0.0-beta2.
> 
> How exactly does `dist.apache.org` work with `archive.apache.org`?
> Will it still work with directories like `/log4j/3.0.0-beta2/`?
> 
> Piotr

Re: [log4j] Remove `` in `main`

[Ralph Goers]

There could be if there was a StatusLogger per LoggerContext. But you would 
still need a global StatusLogger. But doing that seems like overkill. 

On the one hand it is very convenient to configure the level in the config 
file, but on the other the fact that it only takes place halfway through 
configuration is a bit of a problem as that is largely when it is needed.

There is also the issue of compatibility. If support for it is removed we would 
still need to tolerate its presence. In the beginning we could log an Info 
message and then after a while convert it to a warn message before support is 
removed.

To be honest, this isn’t a top priority on the list of things that need to be 
addressed for me which is partly why I guess I am hesitant to do anything. It 
feels like we keep talk about making changes but aren’t really focused on 
getting 3.0 done. Of course, deciding to stick with the 2.x API threw a wrinkle 
into the works of getting 3.x out the door but I’d still like that to be our 
top focus.

Ralph

> On Feb 13, 2024, at 10:21 AM, Matt Sicker  wrote:
> 
> If there were a way to allow for multiple StatusLogger configurations 
> concurrently, then I’d support keeping it in the config file. Otherwise, I’m 
> somewhat ambivalent about this.
> 
>> On Feb 12, 2024, at 13:29, Volkan Yazıcı  wrote:
>> 
>> *Abstract:* I want to remove from `main` the feature that a `Configuration`
>> (e.g., `` in a `log4j2.xml`)
>> configuring the `StatusLogger`, and instead, only allow configuration using
>> properties.
>> 
>> `StatusLogger` can be configured in following ways:
>> 
>>  1. Passing system properties to the Java process (e.g.,
>>  `-Dlog4j2.StatusLogger.level=INFO`)
>>  2. Providing properties in a `log4j2.StatusLogger.properties` file in
>>  the classpath
>>  3. Using Log4j configuration (i.e., `>  dest="out">`) in a `log4j2.xml` in the classpath. This is facilitated by
>>  `StatusConfiguration`.
>>  4. Programmatically (i.e., `StatusLogger.getLogger().setLevel(...)`)
>> 
>> `StatusConfiguration`-based configuration has certain caveats:
>> 
>>  - There is a time between the first `StatusLogger` access and a
>>  configuration file (e.g., `log4j2.xml`) read. Hence, there is a time window
>>  that only the defaults will be effective.
>>  - `StatusConfiguration` is created per `LoggerContext` and each LC
>>  mutates the (globally shared!) `StatusLogger`. Hence, the last loaded
>>  configuration always becomes the effective one.
>> 
>> Given the purpose of `StatusLogger` (i.e., the logger of the logging
>> system) and above shortcomings related to `StatusConfiguration`, I want to
>> remove the `StatusConfiguration`-based configuration in `main`. Thoughts?
>> Objections?
> 

Re: [VOTE] Move Chainsaw to dormant

[Ralph Goers]

+1

Ralph

> On Feb 12, 2024, at 2:31 PM, Christian Grobmeier  wrote:
> 
> Hello,
> 
> This vote is to put Chainsaw to the "Dormant" components. There is much work 
> to be done on this component, but not enough hours can be committed to do 
> that work. To reflect this situation to the user, it is better to move 
> Chainsaw to the dormant section and communicate it as "no longer maintained." 
>  The component can be moved back to the "active" state whenever people are 
> actively working on it. The main branch in the repository will be marked with 
> the new state, but the repository will not be archived for now.
> 
> Please note:
> 
> [ ] +1, move to dormant
> [ ]  -1, don't because...
> 
> I will leave this vote open for the usual 72 hours.
> 
> Thank you!
> 
> Kind regards,
> Christian

Re: [chainsaw] Status change?

[Ralph Goers]

If Scott is +1 then let’s start a vote thread for this.

Ralph

> On Feb 7, 2024, at 8:56 AM, Robert Middleton  wrote:
> 
> +1
> 
> While I agree that it can be useful, it was never really in a state
> where it is.  I think it has a lot of good ideas, but to make it more
> modern and practical it needs to have a much better workflow.
> 
> I may mess around with it more at some point, but it would take a lot
> to be practical.
> 
> If there is a concerted effort in the future to improve it by people
> who do find it useful, I would definitely be open to looking at it
> again.
> 
> -Robert Middleton
> 
> On Wed, Feb 7, 2024 at 2:56 AM Volkan Yazıcı  wrote:
>> 
>> +1
>> 
>> Allow me to make some corrections:
>> 
>> `XmlLayout` is dropped in Log4j 3, not in Log4j 2.
>> 
>> Logstash (the L in the ELK, Elasticsearch-Logstash-Kibana, stack) supports
>> reading logs from a file formatted using a particular pattern. You combine
>> Filebeat with grok filter
>> 
>> to achieve that.
>> 
>> 
>> On Wed, Feb 7, 2024 at 5:19 AM Scott Deboy  wrote:
>> 
>>> Thank you for spending time working on it Christian.
>>> 
>>> I started contributing to Chainsaw in 2003. I agree. It's time.
>>> 
>>> The primary benefit of Chainsaw was its built-in support for real-time
>>> tailing of ssh-accessible pattern-layout based logs  - something
>>> Kibana doesn't support well, and something no-one ever really
>>> understood about it.
>>> 
>>> It was always a dev-focused tool - it works great for local dev, and
>>> works in some prod envs, if you spend enough time to get the setup to
>>> work.
>>> 
>>> There was no great reason to move off of the log41 deps really - they
>>> aren't used for anything other than parsing the patternlayout, but
>>> log4j1 is dead, so I get it.
>>> 
>>> I use it for my work, and will continue to do so, but the
>>> chainsaw-with-log4j1-dep branch. I may revert master back to that,
>>> because why not.
>>> 
>>> +1
>>> 
>>> Thanks again for putting up with my persistence to try and make it
>>> useful to folks - I appreciate it  :)
>>> 
>>> Scott
>>> 
>>> 
>>> On 2/6/24, Christian Grobmeier  wrote:
 Hello
 
 we have had Chainsaw for a long time in our product list, and I can
>>> totally
 see that some - myself included - are emotionally attached to it.
>>> However,
 due to my work on it, I have given it some additional thought.
 
 After working with the Chainsaw code base for a while, I saw that many
 features were commented out and removed when migrating to Log4j 2.
 
 Some basic features, such as "Open Logfile to view it directly." were
 removed. It is already hard to recover the functionality since
>>> log4j-extras
 no longer exists. In addition, as I learned recently, Log4j 2 has removed
 the XML Formatter. The old implementation of Chainsaw could only open
 XML-formatted log files.
 
 Honestly, there is much work to make Chainsaw a working product again. I
 mostly did refactorings and clean-ups, but I am not even through. I could
 continue like this for two more months.
 
 Restoring the old functionality and making it functional again requires
>>> even
 more months.
 If we had completed it, we would have restored a Swing application,
>>> mostly
 replaced by Kibana stacks.
 
 At this point, I don't see how we can write the tons of code necessary,
>>> and
 also not how useful it would be. Either all our users are using Log4j 1,
>>> or
 we don't have any users at all for Chainsaw, since it didn't work.
 
 For that reason, I would like to propose to move Chainsaw to dormant. If
>>> we
 feel for it, we can work and fix it - we should not archive the repo.
>>> But I
 would like to make clear that Chainsaw is not in good shape, and people
 should only use it only "at their own risk."
 
 I would like to make clear that this proposal is not something I say
>>> easily,
 but I feel it is in the best interest of our users to communicate how we
 currently see the status of this project.
 
 Please note, that I don't have much time to continue to work on it in the
 next months.
 
 Remembering the last discussion about this: Scott, are you OK with that
 move? I know it's your baby, but as long as we don't have a working
>>> product,
 we should move it. I am open to moving it back when we somehow get rid of
 all the problems.
 
 Please let me know if one of you has an alternate proposal - we can also
 discuss it in the next call.
 
 Kind regards,
 Christian
 
 --
 The Apache Software Foundation
 V.P., Data Privacy
 
>>> 

Re: Clean site repo

[Ralph Goers]

When you say “hard to work with it” what does that mean? All that should ever 
be required is to do 

git rebase asf-staging

I have never had that take more than a few seconds.

Are you really saying that the staging site is hard to work with?  My 
understanding is that “we” are working on reworking the web site. Volkan has 
mentioned some ideas to me which would allow us to keep the relevant info from 
previous releases.

I don’t like the idea of having multiple efforts going on.

Ralph

> On Feb 6, 2024, at 9:05 AM, Piotr P. Karwasz  wrote:
> 
> Hi all,
> 
> The current `asf-site` branch of the `logging-log4j-site` repo has
> about 450k files, which makes it very hard to work on it.
> 
> Most of those are websites of old releases that I doubt anybody
> (except search engines) visits. Those websites also pollute search
> engine results: several times I found the page of an old release
> better ranked that the current version.
> 
> Therefore I created a new branch `clean-log4j`[1] (which is published
> as [2]) that contains only these directories:
> 
> * 1.x: last 1.x release.
> * 2.3: last 2.3 release. Although not strictly necessary, I thought
> that people stuck with Java 6 will appreciate a website without all
> the features from newer releases.
> * 2.12: last 2.12 release.
> * 2.x: latest 2.x release.
> * 3.x: latest 3.x release.
> * extras: last Apache Log4j Extras release.
> 
> and the XML schemata for our changelog format (although they should be
> moved to `/xml/ns/changelog`).
> 
> All the `log4j-` and similar directories are redirected
> (permanent 301 redirect) to one of the remaining one: e.g.
> `log4j-2.0.1` is redirected to `2.3`, `log4j-2.4` is redirected to
> `2.12`, while `2.13.0` (sic!) is redirected to `2.x`.
> 
> For those that like to go down memory lane, the branch contains 65
> commits for each one of our releases.
> 
> What do you think about replacing `asf-site` with this branch?
> 
> Piotr
> 
> [1] https://github.com/apache/logging-log4j-site/tree/clean-staging
> [2] https://logging-clean.staged.apache.org/log4j

Re: [log4j] Nested logging and async. message formatting

[Ralph Goers]

On 2024/02/05 15:17:29 Volkan Yazıcı wrote:
> 
> AFAIK, nested logging is not a documented feature. Yet one can find several
> tickets people has filed issues that were fixed by us. Hence, it is safe to
> conclude that it is used.

You are correct that it is not documented. Nor should it be. As you are seeing 
you cannot count on how it will behave. I believe any unit tests that expect a 
particular behavior should be removed.

> 
> I also don't know why recursion is not allowed in `AppenderControl`.
> (Related history is too old and could not get in while migrating from SVN
> to Git.)

Recursion is not allowed because it creates a stack overflow exception. For 
example, The user logs a message that routes to an Appender that uses a 
framework that logs a message. That message gets routed back to the same 
Appender which again calls the same framework which logs a message. This will 
endlessly repeat. The recursion check prevents this from happening and the 
nested log message is discarded.  This is actually the ONLY behavior of nested 
logging that should be documented and guaranteed. Passing objects to logging 
that themselves perform logging as they are formatted has to result in the 
unpredictable category since the formatting cannot be guaranteed to occur the 
same way under all conditions.

Ralph

Re: [log4j] performance issue 2.22.1 vs. 2.22.0

[Ralph Goers]

> On Jan 29, 2024, at 12:35 PM, Piotr P. Karwasz  
> wrote:
> 
> Hi Andreas,
> 
> On Mon, 29 Jan 2024 at 18:45,  wrote:
>> Note, this happens even though effectively just two lines are really logged.
>> If DEBUG or TRACE were enabled in the loggers, then tens if not hundreds of 
>> thousands of lines would be logged - so definitely,
>> there is a lot of logging going on.
>> 
>> This has to do with the org.apache.logging.log4j.ThreadContext.
>> If I remove just the following one line from my code, then the performance 
>> is back at normal.
>> 
>> ThreadContext.put(key, value);
>> 
>> So I think this could have been caused by issue 2238
>> (Avoid a slow exception catch in JdkMapAdapterStringMap constructor #2238).
> 
> Thank you for reporting the issue. Can you add an excerpt of your
> Log4j Core configuration and the OS you are using?
> 
> As far as I can see from the code issue #2238 affects configurations
> that use a DynamicThresholdFilter or ThreadContextMapFilter as global
> filter or use a ContextMapLookup.
> 
> Other parts could be affected, but only if log events are generated.
> This does not occur for disabled log events, so only the two logged
> events might be slower.
> 
> Piotr
> 
> PS: I created PR #2256 to fix issue #2238:
> https://github.com/apache/logging-log4j2/pull/2256


Piotr,

It is pretty obvious that the change you made has dire consequences.

Every call to injectContextData for 
ThreadContextDataInjector.ForDefaultThreadContextMap is going to construct a 
new JdkMapAdapterStringMap and cause this exception to be thrown. This should 
happen every time a LogEvent is created. Depending on how filtering is set up 
this is going to kill performance. So if they have the Logger set to Trace but 
then filter out every record in the AppenderRef filter performance would go to 
hell even though nothing is being logged.

You are going to have to find another way to resolve #2238

Ralph

Re: [Log4j] If and how to document potential OOME

[Ralph Goers]

OK. The only good way to handle that is to parse the YAML/JSON file while 
streaming it and extract just the fields you want to include in the logs.

Ralph

> On Jan 25, 2024, at 6:40 PM, Gary Gregory  wrote:
> 
> Well, it's worse than that because the object is an object created by
> parsing a YAML (or JSON) file, then the toString() of that object
> renders a String in some other format.
> 
> Gary
> 
> On Thu, Jan 25, 2024 at 7:45 PM Ralph Goers  
> wrote:
>> 
>> Volkan & Matt,
>> 
>> Neither of those is going to help. The issue is that when the toString 
>> method is called it reads a whole file in and stores it as a String. This 
>> could cause the OOM error. Truncating it in a layout simply limits how much 
>> of the String is printed. Even Gary’s proposal of calling substring() is 
>> still going to operate on the whole String. He would really need a method 
>> that accepts the max number of characters to read from the file.
>> 
>> Ralph
>> 
>>> On Jan 25, 2024, at 2:49 PM, Volkan Yazıcı  wrote:
>>> 
>>> *[Just responding to Matt. I don't have an answer for Gary.]*
>>> 
>>> `JsonTemplateLayout` has `maxStringLength`, and related with it,
>>> `truncatedStringSuffix`.
>>> 
>>> On Thu, Jan 25, 2024 at 9:45 PM Matt Sicker  wrote:
>>> 
>>>> You can use the %maxLength{…}{N} pattern converter with PatternLayout at
>>>> least. Unfortunately, I don’t think any other layouts support a similar
>>>> option.
>>>> 
>>>>> On Jan 25, 2024, at 10:55, Gary D. Gregory  wrote:
>>>>> 
>>>>> Hi All,
>>>>> 
>>>>> I'd like to ask how to if we can devise advice around an issue I ran
>>>> into this week.
>>>>> 
>>>>> One of our test suites processes about 40K files of test fixtures. These
>>>> inputs are parsed, processed, and asserted. This randomly fails on a call
>>>> to Logger#debug(), randomly in that it happens usually once per build,
>>>> somewhere in a logging call. But it usually fails with a call that looks
>>>> like this:
>>>>> 
>>>>> logger.debug("This is fun" + myFunObject);
>>>>> 
>>>>> To simplify things, let's say that it turns out that after an underlying
>>>> third party jar file version upgrade the call to myFunObject#toString() no
>>>> longer returns Object#toString() but rather (again to simplify) the
>>>> contents of the file that was parsed to create myFunObject. This toString()
>>>> can be megabytes. The solution is obvious:
>>>>> 
>>>>> logger.debug("This is fun", myFunObject::toString)
>>>>> 
>>>>> And our CI builds no longer randomly fail since our default logging does
>>>> not log at the debug level.
>>>>> 
>>>>> A better solution could be:
>>>>> 
>>>>> logger.debug("This is fun", () -> myFunObject.toString().substring(0,
>>>> 100))
>>>>> 
>>>>> where I still want _some_ information better than making my own
>>>> toString() with System#identityHashCode(Object) or somesuch. Sure,
>>>> .toString() is still called but it does not make it down into logging. In
>>>> my case the OOME happened in myFunObject::toString so the substring()
>>>> example would not have worked.
>>>>> 
>>>>> My question is: Should we document some general advice on this pattern
>>>> and what should the advice be? Would it make sense to have some features
>>>> where we truncate/reject Strings above a threshold. And yes, calling
>>>> myFunObject.toString() can still still get me in trouble.
>>>>> 
>>>>> Gary
>>>>> 
>>>> 
>>>> 
>> 

Re: [Log4j] If and how to document potential OOME

[Ralph Goers]

Volkan & Matt,

Neither of those is going to help. The issue is that when the toString method 
is called it reads a whole file in and stores it as a String. This could cause 
the OOM error. Truncating it in a layout simply limits how much of the String 
is printed. Even Gary’s proposal of calling substring() is still going to 
operate on the whole String. He would really need a method that accepts the max 
number of characters to read from the file.

Ralph

> On Jan 25, 2024, at 2:49 PM, Volkan Yazıcı  wrote:
> 
> *[Just responding to Matt. I don't have an answer for Gary.]*
> 
> `JsonTemplateLayout` has `maxStringLength`, and related with it,
> `truncatedStringSuffix`.
> 
> On Thu, Jan 25, 2024 at 9:45 PM Matt Sicker  wrote:
> 
>> You can use the %maxLength{…}{N} pattern converter with PatternLayout at
>> least. Unfortunately, I don’t think any other layouts support a similar
>> option.
>> 
>>> On Jan 25, 2024, at 10:55, Gary D. Gregory  wrote:
>>> 
>>> Hi All,
>>> 
>>> I'd like to ask how to if we can devise advice around an issue I ran
>> into this week.
>>> 
>>> One of our test suites processes about 40K files of test fixtures. These
>> inputs are parsed, processed, and asserted. This randomly fails on a call
>> to Logger#debug(), randomly in that it happens usually once per build,
>> somewhere in a logging call. But it usually fails with a call that looks
>> like this:
>>> 
>>> logger.debug("This is fun" + myFunObject);
>>> 
>>> To simplify things, let's say that it turns out that after an underlying
>> third party jar file version upgrade the call to myFunObject#toString() no
>> longer returns Object#toString() but rather (again to simplify) the
>> contents of the file that was parsed to create myFunObject. This toString()
>> can be megabytes. The solution is obvious:
>>> 
>>> logger.debug("This is fun", myFunObject::toString)
>>> 
>>> And our CI builds no longer randomly fail since our default logging does
>> not log at the debug level.
>>> 
>>> A better solution could be:
>>> 
>>> logger.debug("This is fun", () -> myFunObject.toString().substring(0,
>> 100))
>>> 
>>> where I still want _some_ information better than making my own
>> toString() with System#identityHashCode(Object) or somesuch. Sure,
>> .toString() is still called but it does not make it down into logging. In
>> my case the OOME happened in myFunObject::toString so the substring()
>> example would not have worked.
>>> 
>>> My question is: Should we document some general advice on this pattern
>> and what should the advice be? Would it make sense to have some features
>> where we truncate/reject Strings above a threshold. And yes, calling
>> myFunObject.toString() can still still get me in trouble.
>>> 
>>> Gary
>>> 
>> 
>> 

Re: [log4j] Making Log4j 2 API "the Log4j API"

[Ralph Goers]

> On Jan 18, 2024, at 12:59 PM, Piotr P. Karwasz  
> wrote:
> 
> Hi Ralph,
> 
> On Thu, 18 Jan 2024 at 18:18, Ralph Goers  wrote:
>>> Besides we already broke their code, when 2.7 introduced breaking
>>> changes to ConfigurationFactory.
>> 
>> ? The contribution I made was done last year - long past 2.7.
> 
> Yes, what I meant is that Spring Boot uses some Log4j Core specific
> APIs. They are certainly aware that each major version might require a
> rewrite to their code.

That is what I am saying. Since I wrote the code they most certainly expect it 
won’t change.

> 
>>> I am more afraid about users calling `LogManager#getContext` and other
>>> magical snippets of code that I don't consider being part of the
>>> public API (that method should have probably been protected from the
>>> start).
>> 
>> Except tons of users call it for some reason.
> 
> Yes, unfortunately. They will cry, when their code breaks and I don't
> have any problems in breaking such a code during a major release.

I do. Primarily becase it could be happening in some random library they have 
no control over.

> 
> I think we do not stress out enough in our documentation that a
> programmatic solution of a configuration problem is in 99% wrong!
> Unfortunately we even provide bad examples of it in our FAQ:
> 
> https://logging.apache.org/log4j/3.x/faq.html#reconfig_from_code

None of us have ever been happy about programmatic configuration and we have 
gone out of our way to make it unnecessary. But users still do it. Some even 
have valid reasons. I am sure Spring does it to some degree,
> 
> As far as the shutdown example on that page is elegant and stable, the
> programmatic configuration example doesn't even consider the
> possibility of using other logging backends.
> 
>>> In 2022 we saw a lot of questions about users looking to replace
>>> `PropertyConfigurator` from 1.x. Unless I am mistaken, calling
>>> `PropertyConfigurator` was not required since Log4j 1.0.
>> 
>> I am not sure what that has to do with the rest of your points.
> 
> I fear our users will face the exact same problems, when migrating to
> Log4j Core 3.x, that they faced migrating from Log4j 1.x.
> Already in Log4j 1.x they had code that should never been there (Log4j
> 1.x looked up a `log4j.properties` file in the classpath and provided
> a `log4j.configuration` property from its **first** stable release).
> They migrated from Log4j 1.x to 2.x by just translating the same
> faulty code into another major version.

Yes, I am aware that users frequently just want to make something they have 
work even though we provided better ways.
> 
> Maybe we should have a blog post about that.

I doubt most of the users who are doing this would read it.

Ralph

Re: [log4j] Making Log4j 2 API "the Log4j API"

[Ralph Goers]

I have a much more liberal view of the API. That is, if the class isn’t 
documented as private AND it can be reasonably used outside of Log4j then it is 
public.  However, I would actually have to look at every class to see how much 
different it is from your list.

Ralph


> On Jan 18, 2024, at 9:25 AM, Piotr P. Karwasz  wrote:
> 
> Hi Ralph,
> 
> On Thu, 18 Jan 2024 at 16:15, Ralph Goers  wrote:
>> Note that LogManager has
>> 
>> import org.apache.logging.log4j.simple.SimpleLoggerContextFactory;
>> import org.apache.logging.log4j.spi.LoggerContext;
>> import org.apache.logging.log4j.spi.LoggerContextFactory;
>> import org.apache.logging.log4j.spi.LoggingSystem;
>> import org.apache.logging.log4j.spi.Terminable;
>> import org.apache.logging.log4j.status.StatusLogger;
>> import org.apache.logging.log4j.util.StackLocatorUtil;
>> import org.apache.logging.log4j.util.Strings;
>> 
>> That would require users to have the SPI would it not?
> 
> On Thu, 18 Jan 2024 at 16:20, Ralph Goers  wrote:
>> And ThreadContext contains
>> 
>> import org.apache.logging.log4j.message.ParameterizedMessage;
>> import org.apache.logging.log4j.spi.LoggingSystem;
>> import org.apache.logging.log4j.spi.LoggingSystemProperty;
>> import org.apache.logging.log4j.spi.ReadOnlyThreadContextMap;
>> import org.apache.logging.log4j.spi.ThreadContextMap;
>> import org.apache.logging.log4j.spi.ThreadContextStack;
>> import org.apache.logging.log4j.util.InternalApi;
>> import org.apache.logging.log4j.util.Strings
> 
> IMHO, the public API contains:
> 
> 1. all the types in the `o.a.l.l` package,
> 2. all the superclasses and superinterfaces of a type in the public
> API (recursively),
> 3. all the types that appear in the signature of public methods of a
> type in the public API (recursively).
> 
> Which mean the the Log4j API contains `o.a.l.l` and the following
> types from other packages:
> 
> org.apache.logging.log4j.message.EntryMessage
> org.apache.logging.log4j.message.ExitMessage
> org.apache.logging.log4j.message.FlowMessage
> org.apache.logging.log4j.message.FlowMessageFactory
> org.apache.logging.log4j.message.Message
> org.apache.logging.log4j.message.MessageFactory
> org.apache.logging.log4j.spi.LoggerContext
> org.apache.logging.log4j.spi.ExtendedLogger
> org.apache.logging.log4j.spi.LoggerContextFactory
> org.apache.logging.log4j.spi.LoggerRegistry
> org.apache.logging.log4j.spi.StandardLevel
> org.apache.logging.log4j.spi.ReadOnlyThreadContextMap
> org.apache.logging.log4j.util.BiConsumer
> org.apache.logging.log4j.util.ReadOnlyStringMap
> org.apache.logging.log4j.util.StringMap
> org.apache.logging.log4j.util.StringBuilderFormattable
> org.apache.logging.log4j.util.Supplier
> org.apache.logging.log4j.util.TriConsumer
> 
> Removing a method from any one of these can break user's code. Some of
> these (certainly `Supplier`, but to a smaller extent `Message`,
> `BiConsumer` and `TriConsumer`) are implemented by users, so even
> adding an abstract method to those, will break user code.
> 
> Piotr

Re: [log4j] Making Log4j 2 API "the Log4j API"

[Ralph Goers]

> On Jan 18, 2024, at 9:01 AM, Piotr P. Karwasz  wrote:
> 
> Hi Ralph,
> 
> On Thu, 18 Jan 2024 at 15:56, Ralph Goers  wrote:
>> Spring uses PropertiesUtil. I suspect it isn’t alone. That would mean 
>> anything impacted by the property changes would have to be in the spi or 
>> abstracted to reference something in the spi. I am assuming a log4j-spi-2.x 
>> would also be needed. There are many other utility classes that have never 
>> been off limits for users to use.
>> 
>> While your idea could work it definitely would impact some users.
> 
> Spring Boot contributors were probably aware that `PropertiesUtil` is
> not part of the public API. ;-)

Well, I contributed the PR and have always assumed PropertiesUtil was public. 
They modified my PR to limit the scope of the PropertySource but not much more 
than that.


> Besides we already broke their code, when 2.7 introduced breaking
> changes to ConfigurationFactory.

? The contribution I made was done last year - long past 2.7.

> I am more afraid about users calling `LogManager#getContext` and other
> magical snippets of code that I don't consider being part of the
> public API (that method should have probably been protected from the
> start).

Except tons of users call it for some reason.

> 
> In 2022 we saw a lot of questions about users looking to replace
> `PropertyConfigurator` from 1.x. Unless I am mistaken, calling
> `PropertyConfigurator` was not required since Log4j 1.0.

I am not sure what that has to do with the rest of your points.

Ralph

Re: [log4j] Making Log4j 2 API "the Log4j API"

[Ralph Goers]

> On Jan 18, 2024, at 8:14 AM, Ralph Goers  wrote:
> 
> 
> 
>> On Jan 18, 2024, at 7:55 AM, Ralph Goers  wrote:
>> 
>> 
>> 
>>> On Jan 18, 2024, at 5:37 AM, Volkan Yazıcı  wrote:
>>> 
>>> If we
>>> 
>>> 1. move all non-API (`AbstractLogger`, `PropertiesUtil`, etc.) classes
>>> in `log4j-api-3.x` to a new `log4j-spi-3.x` module, and
>>> 2. Only implement `log4j-api-2.x` *interfaces* (not abstract classes!)
>>> in `log4j-core`
>>> 
>>> we can have a Log4j 3 without a `log4j-api-3.x` module, right?
>> 
>> Spring uses PropertiesUtil. I suspect it isn’t alone. That would mean 
>> anything impacted by the property changes would have to be in the spi or 
>> abstracted to reference something in the spi. I am assuming a log4j-spi-2.x 
>> would also be needed. There are many other utility classes that have never 
>> been off limits for users to use.
>> 
>> While your idea could work it definitely would impact some users.
> 
> 
> Note that LogManager has
> 
> import org.apache.logging.log4j.simple.SimpleLoggerContextFactory;
> import org.apache.logging.log4j.spi.LoggerContext;
> import org.apache.logging.log4j.spi.LoggerContextFactory;
> import org.apache.logging.log4j.spi.LoggingSystem;
> import org.apache.logging.log4j.spi.Terminable;
> import org.apache.logging.log4j.status.StatusLogger;
> import org.apache.logging.log4j.util.StackLocatorUtil;
> import org.apache.logging.log4j.util.Strings;
> 
> That would require users to have the SPI would it not?

And ThreadContext contains

import org.apache.logging.log4j.message.ParameterizedMessage;
import org.apache.logging.log4j.spi.LoggingSystem;
import org.apache.logging.log4j.spi.LoggingSystemProperty;
import org.apache.logging.log4j.spi.ReadOnlyThreadContextMap;
import org.apache.logging.log4j.spi.ThreadContextMap;
import org.apache.logging.log4j.spi.ThreadContextStack;
import org.apache.logging.log4j.util.InternalApi;
import org.apache.logging.log4j.util.Strings;


I actually think your idea of splitting the API into API and SPI makes sense 
but it doesn’t seem like it is going to be all that simple to do.  The API will 
have to only reference interfaces that are in the api and the spi would 
implement them. I don’t know if that can be done without impacting users.

Ralph

Re: [log4j] Making Log4j 2 API "the Log4j API"

[Ralph Goers]

> On Jan 18, 2024, at 7:55 AM, Ralph Goers  wrote:
> 
> 
> 
>> On Jan 18, 2024, at 5:37 AM, Volkan Yazıcı  wrote:
>> 
>> If we
>> 
>>  1. move all non-API (`AbstractLogger`, `PropertiesUtil`, etc.) classes
>>  in `log4j-api-3.x` to a new `log4j-spi-3.x` module, and
>>  2. Only implement `log4j-api-2.x` *interfaces* (not abstract classes!)
>>  in `log4j-core`
>> 
>> we can have a Log4j 3 without a `log4j-api-3.x` module, right?
> 
> Spring uses PropertiesUtil. I suspect it isn’t alone. That would mean 
> anything impacted by the property changes would have to be in the spi or 
> abstracted to reference something in the spi. I am assuming a log4j-spi-2.x 
> would also be needed. There are many other utility classes that have never 
> been off limits for users to use.
> 
> While your idea could work it definitely would impact some users.


Note that LogManager has

import org.apache.logging.log4j.simple.SimpleLoggerContextFactory;
import org.apache.logging.log4j.spi.LoggerContext;
import org.apache.logging.log4j.spi.LoggerContextFactory;
import org.apache.logging.log4j.spi.LoggingSystem;
import org.apache.logging.log4j.spi.Terminable;
import org.apache.logging.log4j.status.StatusLogger;
import org.apache.logging.log4j.util.StackLocatorUtil;
import org.apache.logging.log4j.util.Strings;

That would require users to have the SPI would it not?

Ralph

Re: [log4j] Making Log4j 2 API "the Log4j API"

[Ralph Goers]

> On Jan 18, 2024, at 5:37 AM, Volkan Yazıcı  wrote:
> 
> If we
> 
>   1. move all non-API (`AbstractLogger`, `PropertiesUtil`, etc.) classes
>   in `log4j-api-3.x` to a new `log4j-spi-3.x` module, and
>   2. Only implement `log4j-api-2.x` *interfaces* (not abstract classes!)
>   in `log4j-core`
> 
> we can have a Log4j 3 without a `log4j-api-3.x` module, right?

Spring uses PropertiesUtil. I suspect it isn’t alone. That would mean anything 
impacted by the property changes would have to be in the spi or abstracted to 
reference something in the spi. I am assuming a log4j-spi-2.x would also be 
needed. There are many other utility classes that have never been off limits 
for users to use.

While your idea could work it definitely would impact some users.

Ralph

Re: [log4j] Making Log4j 2 API "the Log4j API"

[Ralph Goers]

> On Jan 18, 2024, at 6:15 AM, Gary Gregory  wrote:
> 
> Using the same API jar for 3.x core is intriguing. I like the idea of
> a cleaned-up API jar (no custom Supplier) that can front 2.x and 3.x.
> 

I don’t think that is what Volkan is proposing as it definitely would break 
compatibility between 2.x versions of the API.

Ralph

Re: [log4j] Making Log4j 2 API "the Log4j API"

[Ralph Goers]

Now that I have had 10 seconds to think about it.  The change to the property 
syntax and how PropertiesUtil works will create serious problems for what you 
are proposing.

Ralph

> On Jan 17, 2024, at 10:02 PM, Ralph Goers  wrote:
> 
> The quick answer to this question is “I don’t know”. When we first started on 
> the 3.x adventure I can assure you that log4j-api was very different in the 
> 3.x branch because of the changes we needed to make for JPMS and to the 
> build. However, since we have since carried those changes back to 2.x to a 
> large degree it may be that you are correct and we don’t need to create a 3.x 
> version of the API.
> 
> We would need to compare the two branches of log4j-api and see what the 
> differences are.
> 
> Ralph
> 
>> On Jan 17, 2024, at 9:11 AM, Volkan Yazıcı  wrote:
>> 
>> Given Ralph and Piotr are strongly opinionated about keeping
>> `log4j-api-3.x` binary compatible to `log4j-api-2.x`, can we not release
>> `log4j-api-3.x` in `main` and make `main` only depend on `log4j-api-2.x`
>> instead? (We can move the contents of the `spi` package in `log4j-api-3.x`
>> to a separate `log4j-spi` module in `main`.) This will make everything
>> crystal clear:
>> 
>>  - Log4j 3 is just a major improvement over the backend
>>  - Log4j 3 still supports Log4j 2 API
>>  - We can move the Log4j 2 API to a separate repository with its own
>>  release life cycle (ala SLF4J)
>>  - When time comes to make a new Log4j API where PMC agrees to make
>>  breaking changes, we can call that one Log4j 3 API
>> 
>> I would appreciate it if you can help me to understand if I am
>> missing something. Otherwise, I would like to know why we need to make a
>> major release for a project that is identical to its previous version.
> 

Re: [log4j] Making Log4j 2 API "the Log4j API"

[Ralph Goers]

I am afraid I don’t really understand that. How does moving the spine content 
to another module help?  Doesn’t that mean users would now need 
log4j-api-2.x.jar and log4j-spi-3,x,jar? What is the benefit of that?

Ralph

> On Jan 17, 2024, at 12:09 PM, Matt Sicker  wrote:
> 
> That might work, yeah.
> 
>> On Jan 17, 2024, at 12:46 PM, Volkan Yazıcı  wrote:
>> 
>> We can move the spi package content in main to a separate module in main.
>> SPI problem is solved?
>> 
>> On Wed, 17 Jan 2024 at 18:33, Matt Sicker  wrote:
>> 
>>> I suspect this won’t work that well once I’ve implemented
>>> https://github.com/apache/logging-log4j2/issues/1977 as the current
>>> provider SPI is fairly lacking. It might make more sense to release the
>>> main API as 3.0.0 and have 2.x depend on the updated API.
>>> 
 On Jan 17, 2024, at 10:11 AM, Volkan Yazıcı  wrote:
 
 Given Ralph and Piotr are strongly opinionated about keeping
 `log4j-api-3.x` binary compatible to `log4j-api-2.x`, can we not release
 `log4j-api-3.x` in `main` and make `main` only depend on `log4j-api-2.x`
 instead? (We can move the contents of the `spi` package in
>>> `log4j-api-3.x`
 to a separate `log4j-spi` module in `main`.) This will make everything
 crystal clear:
 
 - Log4j 3 is just a major improvement over the backend
 - Log4j 3 still supports Log4j 2 API
 - We can move the Log4j 2 API to a separate repository with its own
 release life cycle (ala SLF4J)
 - When time comes to make a new Log4j API where PMC agrees to make
 breaking changes, we can call that one Log4j 3 API
 
 I would appreciate it if you can help me to understand if I am
 missing something. Otherwise, I would like to know why we need to make a
 major release for a project that is identical to its previous version.
>>> 
>>> 
> 

Re: [log4j] Making Log4j 2 API "the Log4j API"

[Ralph Goers]

The quick answer to this question is “I don’t know”. When we first started on 
the 3.x adventure I can assure you that log4j-api was very different in the 3.x 
branch because of the changes we needed to make for JPMS and to the build. 
However, since we have since carried those changes back to 2.x to a large 
degree it may be that you are correct and we don’t need to create a 3.x version 
of the API.

We would need to compare the two branches of log4j-api and see what the 
differences are.

Ralph

> On Jan 17, 2024, at 9:11 AM, Volkan Yazıcı  wrote:
> 
> Given Ralph and Piotr are strongly opinionated about keeping
> `log4j-api-3.x` binary compatible to `log4j-api-2.x`, can we not release
> `log4j-api-3.x` in `main` and make `main` only depend on `log4j-api-2.x`
> instead? (We can move the contents of the `spi` package in `log4j-api-3.x`
> to a separate `log4j-spi` module in `main`.) This will make everything
> crystal clear:
> 
>   - Log4j 3 is just a major improvement over the backend
>   - Log4j 3 still supports Log4j 2 API
>   - We can move the Log4j 2 API to a separate repository with its own
>   release life cycle (ala SLF4J)
>   - When time comes to make a new Log4j API where PMC agrees to make
>   breaking changes, we can call that one Log4j 3 API
> 
> I would appreciate it if you can help me to understand if I am
> missing something. Otherwise, I would like to know why we need to make a
> major release for a project that is identical to its previous version.

Re: [log4j] `.asf.yaml` between branches

[Ralph Goers]

> On Jan 16, 2024, at 9:15 AM, Ralph Goers  wrote:
> 
> 
> 
>> On Jan 15, 2024, at 11:18 PM, Piotr P. Karwasz  
>> wrote:
>> 
>> Hi Ralph,
>> 
>> On Tue, 16 Jan 2024 at 01:56, Ralph Goers  wrote:
>>> 
>>> I don’t understand what it means to keep both staging and publish in 
>>> “asf-site”. By definition, the asf-site branch is the live web-site and 
>>> asf-staging is the staging web site.  Are you talking about the build 
>>> scripts or something?
>> 
>> We are talking about this `.asf.yaml` content:
>> 
>> publish:
>> profile: ~
>> whoami: asf-site
>> subdir: content/logging-parent
>> 
>> staging:
>> profile: ~
>> whoami: asf-staging
>> subdir: content/logging-parent
>> 
>> Due to the `whoami` attribute, the `.asf.yaml` file on both the
>> `asf-site` and `asf-staging` branch can be the same. INFRA will ignore
>> the `staging` instruction on `asf-site` and the `publish` instruction
>> on `asf-staging`.
> 
> Thanks. That makes sense then.

Actually, it almost HAS to be that way if you are going to be able to simply do 
a merge from staging to the publish branch to go live.

Ralph

Re: [log4j] `.asf.yaml` between branches

[Ralph Goers]

> On Jan 15, 2024, at 11:18 PM, Piotr P. Karwasz  
> wrote:
> 
> Hi Ralph,
> 
> On Tue, 16 Jan 2024 at 01:56, Ralph Goers  wrote:
>> 
>> I don’t understand what it means to keep both staging and publish in 
>> “asf-site”. By definition, the asf-site branch is the live web-site and 
>> asf-staging is the staging web site.  Are you talking about the build 
>> scripts or something?
> 
> We are talking about this `.asf.yaml` content:
> 
> publish:
>  profile: ~
>  whoami: asf-site
>  subdir: content/logging-parent
> 
> staging:
>  profile: ~
>  whoami: asf-staging
>  subdir: content/logging-parent
> 
> Due to the `whoami` attribute, the `.asf.yaml` file on both the
> `asf-site` and `asf-staging` branch can be the same. INFRA will ignore
> the `staging` instruction on `asf-site` and the `publish` instruction
> on `asf-staging`.

Thanks. That makes sense then.

Ralph

Re: Should we keep `log4j-appserver` and `log4j-jcl` in 3.x?

[Ralph Goers]

I am OK with this. It would be best if we could get Tomcat to publish a 
tomcat-log4j artifact.

Ralph

> On Jan 15, 2024, at 1:50 PM, Piotr P. Karwasz  wrote:
> 
> Hi all,
> 
> While 3.x is approaching its first stable release as a meteor burning
> through the sky, its core is getting smaller as it approaches earth.
> 
> That is why I would propose to remove two further modules from 3.x:
> 
> * The functionality of `log4j-jcl` is included in `commons-logging`
> version 1.3.0 (and previously it was also provided by `spring-jcl`). I
> assume that users migrating to Log4j 3.x can also migrate to
> `commons-logging` 1.3.0.
> 
> * The Jetty 9.x logger, which is half of the functionality of
> `log4j-appserver`, can be replaced by `log4j-slf4j-impl` if the users
> are willing to migrate to Jetty 10.x. Since community support for
> Jetty 9.x ended two years ago, I guess user will migrate to a
> supported Jetty version first and only then think about Log4j 3.x.
> 
> * The Tomcat logger (the other half of `log4j-appserver`) is only
> part of the puzzle of artifacts required to have a fully functional
> logging system in Tomcat. I usually also needed my own artifacts from
> copernik-eu/log4j-plugins[1] to have something comparable (but still
> worse) than what WildFly has. I can adopt this part of
> `log4j-appserver` into copernik-eu/log4j-plugins and make a release
> before Log4j 3.0.0 is out. If there is any user interest in it, I can
> "contribute back" the Tomcat logger and other components in a future
> release.
> 
> What do you think?
> 
> [1] https://github.com/copernik-eu/log4j-plugins

Re: [log4j] `.asf.yaml` between branches

[Ralph Goers]

I don’t understand what it means to keep both staging and publish in 
“asf-site”. By definition, the asf-site branch is the live web-site and 
asf-staging is the staging web site.  Are you talking about the build scripts 
or something? 

I started to reply to this earlier today but got sidetracked with my day job. 

To be honest, I still prefer the repo per web site model but I am OK with any 
solution that:
* is 100% consistent no matter which web site it is. 
* doesn’t require a Confluence page or Readme to locate the web site source for 
a project

By consistent I mean that if it is using branches then the branch name follows 
a pattern such as asf-site-2.x and asf-staged-2.x even if we are only 
publishing a single version of the web site for the project.

Ralph

> On Jan 15, 2024, at 11:52 AM, Volkan Yazıcı  wrote:
> 
> Even though it wouldn't be of my preference, keeping both `staging` and
> `publish` configuration in `asf-site` sounds like a middle ground I can
> live with. I will appreciate it if you can adapt your existing changes (in
> `logging-log4j-kotlin`, `logging-parent`, etc.) to comply with this, unless
> you have already done so.
> 
> On Mon, Jan 15, 2024 at 12:27 PM Piotr P. Karwasz 
> wrote:
> 
>> Hi Volkan,
>> 
>> On Mon, 15 Jan 2024 at 11:07, Volkan Yazıcı  wrote:
 * you can stage the website for a release with a simple:
 
 git checkout asf-staging
 git reset --hard asf-site
 unzip ...
 git push -f
 
>>> 
>>> You can do the same in the existing setup too. You just need a `sed`
>>> one-liner to adapt the `.asf.yaml` content:
>>> 
>>> $ sed -i -e 's/^publish:/staging:/g' -e 's/^  whoami:.+/  :whoami:
>>> asf-staging/g' asf.yaml
>>> 
>>> Not to mention this is a detail that will be a part of the CI-based
>> release
>>> process. That is, no PMC member will need to recall or type any of these
>>> `git`, `sed`, `unzip`, etc. commands to cut a release.
>>> 
>>> Given I addressed your "quickly stage a website" concern, are we good?
>> 
>> I agree with your objections on not **requiring** all `.asf.yaml` to
>> be in sync. However I also don't see a reason not to have a `staging`
>> configuration on the `asf-site` branch and a `publish` configuration
>> on the `asf-staging` branch.
>> 
>> I would prefer for the CI not to meddle with critical files such as
>> `.asf.yaml`, especially if changes to these files are almost never
>> required.
>> 
>> Piotr
>> 
>> PS: I removed the `publish` and `staging` sections on the main branch
>> of `logging-parent` and the `github` section on the site branches.
>> 

Re: Spring 3 and Log4j 3

[Ralph Goers]

On 2024/01/10 09:27:22 Volkan Yazıcı wrote:
> Thanks for chasing this Ralph, it is indeed a nice step forwards.
> 
> The aforementioned Log4j issue is fixed in #2062
> :
> 
>1. You submitted the PR on Dec 5 at 04:15 (GMT+1) and merged it at
>05:58. Next time, would you mind giving us a couple of days for the review,
>please?

AFAIK we are still doing CTR. Usually I would have just committed this directly 
but since we need something to track in the release notes I decided to create a 
PR.  If I had thought this to be anything more than a straightforward bug fix I 
would have asked for a review and waited.

>2. Piotr and I both reviewed the changes and we had remarks. None of
>them have been addressed so far. Would you mind responding to them, please?
>(See the PR for the comments.)

Sure. I apologize that I didn't notice the remarks due to the sheer volume of 
email I get from GitHub.

>3. Could you backport the fix to `2.x` branch too, please?

I could. I am still debating whether we should. It still isn't clear to me how 
much more has to be done to release 3.0.0. Once that is out we should be 
encouraging Spring 3.x users to upgrade to Log4j 3.

Ralph

Re: [log4j] ProGuard support in `log4j-api`

[Ralph Goers]

Yeah - if we add it to the code base that implies that we are testing it. I 
really don’t want to be in the position where we start adding customizations 
for every tool a user might be using. This is very much in line with our not 
wanting to keep adding more and more integrations .

Ralph

> On Jan 11, 2024, at 11:36 AM, Volkan Yazıcı  wrote:
> 
> ProGuard  – a GPL2-licensed,
> widely used JAR optimizer and obfuscator in the Android world – doesn't
> work with `log4j-api` out of the box. In summary, since Log4j uses
> reflection, ProGuard needs to be configured to avoid removing certain Log4j
> classes. A user has submitted (#2182)
>  the ProGuard
> configuration addressing the issue. We can either distribute it in
> `META-INF/proguard/log4j.pro` and make `log4j-api` work out-of-the-box with
> ProGoard or simply document this. Given how certain PMC members are
> strongly allergic to breaking backward compatibility even in major
> releases, I am afraid distributing this as a part of `log4j-api` will
> become a liability we cannot get rid of in our lifetime. I will share this
> with the user and ask him to convert his PR to a documentation update. If
> you disagree, please let me know.

Re: Removal of `log4j-jcl` in `3.x`

[Ralph Goers]

Fine by me.

Ralph

> On Jan 10, 2024, at 3:11 PM, Piotr P. Karwasz  wrote:
> 
> Hi all,
> 
> Since `commons-logging` 1.3.0 was published last month and it supports
> Log4j API out-of-the-box, does it still make sense to keep `log4j-jcl`
> around in 3.x?
> 
> I would keep the 2.x version, so that users don't need to modify their
> stacks. However we might assume that 3.x adopters are already running
> the latest versions of the remaining dependencies, specifically they
> upgraded `commons-logging`. This is why I would propose to remove
> `log4j-jcl` from 3.x.
> 
> Remark that Spring users have been using `spring-jcl` for a very long
> time (which also supports the Log4j API), so they didn't need
> `log4j-jcl` either.
> 
> Piotr

Re: Spring 3 and Log4j 3

[Ralph Goers]

> On Jan 10, 2024, at 11:29 AM, Piotr P. Karwasz  
> wrote:
> 
> Hi Matt,
> 
> On Wed, 10 Jan 2024 at 18:45, Matt Sicker  wrote:
>> 
>> This might affect 2.x, though it’s largely in the Spring environment 
>> property source. Given the application lifecycle there, Spring doesn’t seem 
>> to remove its own closed property sources at shutdown, hence the exception. 
>> The issue was only reported against Spring 3, though.
> 
> A lot of 2.x and 3.x here. ;-)
> What I meant is the patch should be ported to Log4j 2.x, so it can
> work with Spring 3.x. After all Spring is currently using the 2.x
> branch of Log4j.
> 

I fixed two bugs. One was reported against 2.x ages ago. The second was first 
reported in the Spring issue. 

Yes, they can be ported to 2.x but if we release 3.0.0 in the next couple of 
months I would prefer to tell users just to upgrade since 3.x specifically 
targets the environment Spring 3.x uses.

Ralph

Spring 3 and Log4j 3

[Ralph Goers]

FYI - in 
https://github.com/spring-projects/spring-boot/issues/33450#issuecomment-1883014368
 has confirmed that Log4j 3.0.0-beta1 now works correctly with Spring 3.x.

Ralph

Re: PropertyKey abstraction

[Ralph Goers]

PropertyKey was created so that all “components” could be specified as an enum 
value, thus ensuring consistency.  

The split between component and key is used in every declaration of a property. 

It is unfortunate that the keys also had to be specified as static constants so 
they could be used in @SetSystemProperty. I would have preferred to be able to 
use the getSystemKey method.

I do have objections. While the interface and enum usage isn’t perfect moving 
back to static constants is much worse in my opinion. If I hadn’t thought that 
I would have done that in the first place.

If you can figure out a better approach just propose that but please leave it 
the way it is until then.

I should note that some of the issues I encountered when I did this were:
1. Because PropertyCompoment is an enum all components have to be defined 
globally even if they only apply to a specific module.
2. Each module needs to be able to define the properties that are specific to 
that module.
3. The PropertyKey is actually the trailing portion of the key as it is defined 
in the system. There they are always prefixed by the Log4j identifier and their 
scope.

Ralph

> On Jan 4, 2024, at 2:13 AM, Piotr P. Karwasz  wrote:
> 
> Hi all,
> 
> Looking at the `PropertyKey` abstraction in `3.x`, I was wondering
> what advantages it brings over simple string constants. From my
> perspective it has more cons than pros.
> 
> Pros:
> 
> * it is typesafe; a committer must create an instance of it to use it;
> 
> Cons:
> 
> * it is hard to list for documentation purposes. It would be easier
> to document all available properties if they were declared as:
> 
> /**
> * Makes Foo into bars.
> */
> @Log4jProperty
> private static final String FOO_BAR = "Foo.bar";
> 
>  Such a definition can be trivially detected by an annotation
> processor, whereas currently I would need to call `PropertyKey#getKey`
> to even find the name of the property.
> 
> * the split into "component" and "name" isn't really used in real
> code, so `PropertyKey` is basically a wrapper for a single String,
> * IMHO the `getSystemKey` method does not belong in `PropertyKey`. It
> is something that is used only by those property sources that are
> global in the JVM (like env variables or system properties). Other
> property sources like `TestPropertySource` have no use of it.
> * since a `PropertyKey` is not a compile-time constant, we have
> several `Constant` classes that contain string constants, so that we
> can use them in tests (e.g. in the `@SetSystemProperty` annotation).
> 
> If you don't have any objections, I would prefer to refactor all these
> `PropertyKey` implementations to simple classes that contain only
> constant string fields.
> 
> In a **longer** term it would be nice to reintroduce type safety, but
> for the **values** of properties instead of their keys. We could copy
> the way Spring does it, create classes like:
> 
> /**
> * Configures a https://logging.apache.org/log4j/3.x/recycler.html
> */
> @Log4jPropertySource(prefix = "Recycler")
> public class RecyclerConfig {
> 
> /**
> * Type of recycler.
> */
> public String type;
> 
> /**
> * Size of the recyclers queue.
> */
> public int size;
> }
> 
> and we could retrieve all the properties at once with
> `PropertyEnvironment#getProperties(Recycler.class)`.
> 
> What do you think?

Re: `a.o.l.l.core.parser` package removal

[Ralph Goers]

Yeah - Given that ticket it is obvious to me that this class should be moved to 
another module. I don’t think moving it to Chainsaw or Flume is the correct 
thing to do.  For now, I would put it into log4j-samples, either under 
log4j-server or in its own module. 

Note that a Flume event consists of a Map of “headers” and a body. The body is 
the result of calling layout.getFormattedMessage() and the headers generally 
would contain values from the ThreadContextMap as well as fields from the Map 
if the LogEvent contains a MapMessage. So in many cases parsing the message 
isn’t necessary since the fields are usually already available as headers.

Ralph

> On Jan 2, 2024, at 3:30 PM, Piotr P. Karwasz  wrote:
> 
> Hi Ralph,
> 
> On Tue, 2 Jan 2024 at 13:21, Ralph Goers  wrote:
>> 
>> Why is it included if it isn’t used?
> 
> Apparently this was added by Mikael in:
> https://issues.apache.org/jira/browse/LOG4J2-1986
> 
> There were no significant changes/bug fixes after the initial submission.
> 
> Piotr

Re: `a.o.l.l.core.parser` package removal

[Ralph Goers]

Why is it included if it isn’t used? 

Ralph

> On Jan 2, 2024, at 4:09 AM, Piotr P. Karwasz  wrote:
> 
> Hi,
> 
> While working on PR#2142[1] I noticed that we have an
> `a.o.l.l.core.parser` package that depends on Jackson.
> 
> Since Log4j itself never parses log events, I would propose to remove
> it from `log4j-core` and optionally move it somewhere else (Chainsaw
> or Flume?).
> 
> My main concern is vulnerability exposure:
> 
> * I would like to prevent CVEs like CVE-2019-17571[2] from being
> published against `log4j-core` in the future. Dealing with CVEs that
> say "code that we never use is vulnerable to..." bring a lot of
> useless PR/documentation work: we'll need to explain to users how to
> mitigate a vulnerability that is almost never exploitable and our
> users will also have to evaluate the exploitability of the CVE in
> their own applications,
> * in some not so far future we'll need to publish VEX records to
> comply with regulation. Every time Jackson will publish a
> deserialization vulnerability, we'll need to state that we are
> vulnerable.
> 
> What do you think?
> 
> Piotr
> 
> [1] https://github.com/apache/logging-log4j2/pull/2142
> [2] https://www.cvedetails.com/cve/CVE-2019-17571/

Re: [VOTE] Release Apache Log4j 3.0.0-beta1 (RC2)

[Ralph Goers]

+1 from me. Everything seems ok.

Ralph

> On Dec 19, 2023, at 2:00 PM, Volkan Yazıcı  wrote:
> 
> This is a vote to release the Apache Log4j 3.0.0-beta1 RC2.
> 
> Website: https://logging.staged.apache.org/log4j/3.x
> GitHub: https://github.com/apache/logging-log4j2
> Commit: 416cd4dcf419b59c88054d2001d34c7fec010560
> Distribution: https://dist.apache.org/repos/dist/dev/logging/log4j
> Nexus: 
> https://repository.apache.org/content/repositories/orgapachelogging-1252
> Signing key: 0x077e8893a6dcc33dd4a4d5b256e73ba9a0b592d0
> 
> Please download, test, and cast your votes on this mailing list.
> 
> [ ] +1, release the artifacts
> [ ] -1, don't release, because...
> 
> This vote is open for 72 hours and will pass unless getting a
> net negative vote count. All votes are welcome and we encourage
> everyone to test the release, but only the Logging Services PMC
> votes are officially counted.
> 
> PLEASE USE THIS THREAD ONLY FOR VOTING +1 OR -1. IF YOU HAVE THOUGHTS,
> CONCERNS, QUESTIONS, ETC. SHARE THEM ELSEWHERE. THIS IS A BETA
> RELEASE. WE INTEND TO HAVE SEVERAL OTHER BETA RELEASES. THIS IS NOT
> THE CONCLUSIVE `3.0.0` RELEASE.
> 
> == Review Kit
> 
> The minimum set of steps needed to review the uploaded distribution
> files in the Subversion repository can be summarized as follows:
> 
># Check out the distribution
>svn co https://dist.apache.org/repos/... && cd $_
> 
># Verify checksums
>shasum --check *.sha512
> 
># Verify signatures
>wget -O - https://downloads.apache.org/logging/KEYS | gpg --import
>for sigFile in *.asc; do gpg --verify $sigFile; done
> 
># Verify reproduciblity
>umask 0022
>unzip *-src.zip -d src
>cd src
>export NEXUS_REPO=https://repository.apache.org/content/...
>sh mvnw -Prelease verify artifact:compare -Dreference.repo=$NEXUS_REPO
> 
> == Release Notes
> 
> This is the first beta release of the upcoming major release, i.e., `3.0.0`.
> 
> === Added
> 
> * Add annotations for nullability. (LOG4J2-1477)
> * Remove deprecated code. (LOG4J2-2493)
> * Add a more generalized dependency injection system to plugins
> inspired by JSR 330. (LOG4J2-2803)
> * Add and enhance structured properties for per-context settings
> outside configuration files. (LOG4J2-3299[LOG4J2-3299], #1473)
> * Automate artifact publishing and release preparation. (LOG4J2-3466)
> * Add support for dependency injection of plugins into container types
> such as `Optional`, `Collection`, `Set`, `Stream`,
> `List`, and `Map`. (LOG4J2-3496)
> * Add support for `ConstraintValidator` in plugin classes. (LOG4J2-3497)
> 
> === Changed
> 
> * Remove liquibase-log4j2 maven module (#1193)
> * Make the output of annotation processing reproducible. (#1520)
> * Replace `synchronized` blocks with locks for improved performance
> with virtual threads. (#1532)
> * Removes additional `isFiltered` checks in `AsyncLoggerConfig`. (#1550)
> * Ignore exceptions thrown by PropertySources. Eliminate
> ClassCastException when SimpleLoggerContext is used.
> (spring-projects/spring-boot#33450, #1799)
> * Update `com.lmax:disruptor` to version `4.0.0` (#1829)
> * Migrate most tests to JUnit 5. This includes a more powerful set of
> test extensions. (LOG4J2-2653)
> * Make Log4j use its own BOM. (LOG4J2-3511)
> * Change encoding of HTTP Basic Authentication to UTF-8. (#1970)
> * Upgraded the required compiler version to Java 17
> * Upgraded the required runtime version to Java 17
> * Update `actions/checkout` to version `4.1.1` (#1869)
> * Update `actions/setup-java` to version `3.13.0` (#1809)
> * Update `actions/setup-python` to version `4.7.1` (#1831)
> * Update `ch.qos.logback:logback-classic` to version `1.4.14` (#2028)
> * Update `com.datastax.cassandra:cassandra-driver-core` to version
> `3.11.5` (#1889)
> * Update `com.fasterxml.jackson:jackson-bom` to version `2.16.0` (#1974)
> * Update `com.github.luben:zstd-jni` to version `1.5.5-11` (#2032)
> * Update `com.github.spotbugs:spotbugs-maven-plugin` to version
> `4.7.3.6` (#1879)
> * Update `com.github.tomakehurst:wiremock-jre8` to version `2.35.1` (#1765)
> * Update 
> `com.google.code.java-allocation-instrumenter:java-allocation-instrumenter`
> to version `3.3.4` (#2102)
> * Update `com.google.errorprone:error_prone_core` to version `2.23.0` (#1871)
> * Update `com.google.guava:guava-testlib` to version `32.1.3-jre` (#1934)
> * Update `com.h2database:h2` to version `2.2.224` (#1917)
> * Update `commons-codec:commons-codec` to version `1.16.0` (#2054)
> * Update `commons-io:commons-io` to version `2.15.1` (#2035)
> * Update `commons-logging:commons-logging` to version `1.3.0` (#2046)
> * Update `de.flapdoodle.reverse:de.flapdoodle.reverse` to version
> `1.7.2` (#2000)
> * Update `io.netty:netty-bom` to version `4.1.104.Final` (#2097)
> * Update `net.java.dev.jna:jna` to version `5.14.0` (#2082)
> * Update `org.apache.aries.spifly:org.apache.aries.spifly.dynamic.bundle`
> to version `1.3.7` (#2053)
> * Update `org.apache.commons:commons

Re: [VOTE] Release Apache Log4j 3.0.0-beta1

[Ralph Goers]

Christian,

The vote has been open for 6 days because we were under the impression the vote 
was going be cancelled based on Piotr’s feedback. I can commit to having the 
review done in 72 hrs if the release is cut today or tomorrow. This slow down 
for me at work this time of the year so between now and New Years Day is a 
great time to get stuff done.

Ralph

> On Dec 19, 2023, at 6:12 AM, Christian Grobmeier  wrote:
> 
> Hi Volkan
> 
> On Tue, Dec 19, 2023, at 13:43, Volkan Yazıcı wrote:
>> I am cancelling this vote. I may try to issue an RC2 this week if time
>> allows. If you think that is inconvenient due to upcoming xmas, and/or you
>> want to issue the RC2 yourself, please let me know.
> 
> please don't cut an RC2 this week. This vote took 6 days and some nitpicks. I 
> am afraid it might be open over christmas. Apart from that, I know how tight 
> your schedule is, so you may take it as a relief to not cut another one :)
> 
> Thanks for your hard work!
> 
> 
>> 
>> On Wed, Dec 13, 2023 at 4:26 PM Volkan Yazıcı  wrote:
>> 
>>> This is a vote to release the Apache Log4j 3.0.0-beta1.
>>> 
>>> Website: https://logging.staged.apache.org/log4j
>>> GitHub: https://github.com/apache/logging-log4j2
>>> Commit: c5dbdcfeb0216e1e3e333436e9b4d04cc3b8e6fd
>>> Distribution: https://dist.apache.org/repos/dist/dev/logging/log4j
>>> Nexus:
>>> https://repository.apache.org/content/repositories/orgapachelogging-1246
>>> Signing key: 0x077e8893a6dcc33dd4a4d5b256e73ba9a0b592d0
>>> 
>>> Please download, test, and cast your votes on this mailing list.
>>> 
>>> [ ] +1, release the artifacts
>>> [ ] -1, don't release, because...
>>> 
>>> This vote is open for 72 hours and will pass unless getting a
>>> net negative vote count. All votes are welcome and we encourage
>>> everyone to test the release, but only the Logging Services PMC
>>> votes are officially counted.
>>> 
>>> == Review Kit
>>> 
>>> The minimum set of steps needed to review the uploaded distribution
>>> files in the Subversion repository can be summarized as follows:
>>> 
>>># Check out the distribution
>>>svn co https://dist.apache.org/repos/... && cd $_
>>> 
>>># Verify checksums
>>>shasum --check *.sha512
>>> 
>>># Verify signatures
>>>wget -O - https://downloads.apache.org/logging/KEYS | gpg --import
>>>for sigFile in *.asc; do gpg --verify $sigFile; done
>>> 
>>># Verify reproduciblity
>>>umask 0022
>>>unzip *-src.zip -d src
>>>cd src
>>>export NEXUS_REPO=https://repository.apache.org/content/...
>>>sh mvnw -Prelease \
>>>verify artifact:compare \
>>>-Dreference.repo=$NEXUS_REPO \
>>>-Dcyclonedx.skip
>>> 
>>> Some SBOM discrepancy is causing reproducibility mismatch, hence the
>>> `-Dcyclonedx.skip`. Since `2.x` and `main` are greatly diverged, I couldn't
>>> figure out the missing piece yet.
>>> 
>>> == Release Notes
>>> 
>>> This is the first beta release of the upcoming major release, i.e.,
>>> `3.0.0`.
>>> 
>>> === Added
>>> 
>>> * Add annotations for nullability. (LOG4J2-1477)
>>> * Remove deprecated code. (LOG4J2-2493)
>>> * Add a more generalized dependency injection system to plugins inspired
>>> by JSR 330. (LOG4J2-2803)
>>> * Add and enhance structured properties for per-context settings outside
>>> configuration files. (1473)
>>> * Automate artifact publishing and release preparation. (LOG4J2-3466)
>>> * Add support for dependency injection of plugins into container types
>>> such as `Optional`, `Collection`, `Set`, `Stream`, `List`,
>>> and `Map`. (LOG4J2-3496)
>>> * Add support for `ConstraintValidator` in plugin classes. (LOG4J2-3497)
>>> 
>>> === Changed
>>> 
>>> * Remove liquibase-log4j2 maven module (#1193)
>>> * Make the output of annotation processing reproducible. (#1520)
>>> * Replace `synchronized` blocks with locks for improved performance with
>>> virtual threads. (#1532)
>>> * Removes additional `isFiltered` checks in `AsyncLoggerConfig`. (#1550)
>>> * Ignore exceptions thrown by PropertySources. Eliminate
>>> ClassCastException when SimpleLoggerContext is used.
>>> (spring-projects/spring-boot#33450, #1799)
>>> * Update `com.lmax:disruptor` to version `4.0.0` (#1829)
>>> * Migrate most tests to JUnit 5. This includes a more powerful set of test
>>> extensions. (LOG4J2-2653)
>>> * Make Log4j use its own BOM. (LOG4J2-3511)
>>> * Change encoding of HTTP Basic Authentication to UTF-8. (#1970)
>>> * Upgraded the required compiler version to Java 17
>>> * Upgraded the required runtime version to Java 17
>>> * Update `actions/checkout` to version `4.1.1` (#1869)
>>> * Update `actions/setup-java` to version `3.13.0` (#1809)
>>> * Update `actions/setup-python` to version `4.7.1` (#1831)
>>> * Update `ch.qos.logback:logback-classic` to version `1.4.14` (#2028)
>>> * Update `com.datastax.cassandra:cassandra-driver-core` to version
>>> `3.11.5` (#1889)
>>> * Update `com.fasterxml.jackson:jackson-bom` to version `2.16.0` (#1974)
>>> * Update `com.github.luben:zstd-j

Re: Versioning scheme

[Ralph Goers]

Maybe you could add comments to the schema to define what the enumerations are 
intended for and what effect they have on the release?

Ralph

> On Dec 12, 2023, at 10:13 AM, Volkan Yazıcı  wrote:
> 
> Got it. I will land a log4j-changelog change accordingly, implement the CI
> magic, and update the release instructions.
> 
> On Tue, 12 Dec 2023 at 17:41, Piotr P. Karwasz 
> wrote:
> 
>> Hi Volkan,
>> 
>> On Tue, 12 Dec 2023 at 17:00, Volkan Yazıcı  wrote:
>>> Piotr, could you give me a well-defined formula of your desired
>> versioning
>>> scheme concerning minor/patch bumps?
>> 
>> I agree with what Ralph proposed.
>> 
>> What I don't agree with is the current practice to set up the **next**
>> release version always to the next minor release (with an empty set of
>> changes).
>> I would set it to the next patch release and only transform it into a
>> minor release if we add an `added` or `changed` changelog entry.
>> As Ralph said, the CI could do it.
>> 
>> Piotr
>> 

Re: Versioning scheme

[Ralph Goers]

Yes.

Ralph

> On Dec 12, 2023, at 9:03 AM, Volkan Yazıcı  wrote:
> 
> Ralph, do you mean if all changes are a subset of bug fixes, deprecations,
> or updates, then it is a patch release?
> 
> On Tue, 12 Dec 2023 at 16:23, Ralph Goers 
> wrote:
> 
>> I have to agree with Piotr’s example. Simply upgrading a dependency
>> doesn’t, on its own, change any code in Log4j.
>> 
>> I see three possible solutions for this:
>> 
>> 1. Do not allow any dependency updates until a “scheduled” mentor release
>> (once every 2 or 3 months). Frankly I’d be fine with this except when a
>> dependency has a major security vulnerability.
>> 2. Change all dependency versions to be ranges such that they don’t
>> require a release for users to include new dependency releases.  (I cannot
>> really recommend doing this).
>> 3. Add a new type to represent dependency updates. This would also not
>> require a minor release. This is really the most appropriate fix as
>> updating a dependency is not a change to anything in Log4j.
>> 
>> I would suggest adding another enumeration named “update”.
>> 
>> Ralph
>> 
>>> On Dec 12, 2023, at 7:19 AM, Piotr P. Karwasz 
>> wrote:
>>> 
>>> Hi Volkan,
>>> 
>>> On Tue, 12 Dec 2023 at 13:02, Volkan Yazıcı > vol...@yazi.ci>> wrote:
>>>> 
>>>> On Tue, Dec 12, 2023 at 11:47 AM Piotr P. Karwasz <
>> piotr.karw...@gmail.com>
>>>> wrote:
>>>> 
>>>>> * we lack a way to classify dependency updates. A concrete example:
>>>>> did the Commons DBCP2 bump to 2.11.0 change anything in
>>>>> `log4j-jdbc-dbcp2`? I don't think so, the artifact compiles with
>>>>> version 2.2.0 of the artifact. We are only stating that
>>>>> `log4j-jdbc-dbcp2` will **also** work with version 2.11.0.
>>>>> 
>>>> 
>>>> Exactly! Hence, it is clear that this is neither a bug fix, nor a
>>>> deprecation. That is, there is no ambiguity that this goes into a minor
>>>> release.
>>> 
>>> And here I don't agree. Dependabot upgrades provide **no** benefit to
>>> the generated JAR files. Even the bytecode is exactly the same as
>>> before the upgrade.
>>> 
>>> What these changes provide is part of our "Secure by default" or
>>> "Up-to-date by default" feature: Log4j artifacts will never freeze our
>>> clients dependencies and will work with the newest versions of the
>>> libraries we depend upon.
>>> 
>>> If you need to **manually** fix code for the upgrade to work (as you
>>> did for Jackson 2.16.0), then you can change the automatically
>>> generated entry from "fixed/upgraded" to "changed".
>>> 
>>>>> I don't think this warrants a minor version bump.
>>>>> 
>>>> 
>>>> This is what I am trying to eliminate Piotr: opinions. When a person
>> starts
>>>> thinking *"Shall this be a patch or minor release?"*, the outcome is
>>>> inherently subjective. We cannot completely get rid of subjective
>>>> assessment, but assist it with guardrails.
>>> 
>>> These guardrails seem to follow the easy path: let's just do minor
>>> releases, so nobody will tell us we are wrong. If you add ".0.0" to
>>> all Google Chrome versions, Chrome will also follow semver to the
>>> letter. It just loses the spirit.
>>> 
>>>>> The bump to JDK 17 was necessary, very useful for us, but users don't
>>>>> really care what JDK was used for compilation.
>>>> 
>>>> 
>>>> What? Users, that is, developers using `logging-parent` as their parent
>>>> project, do certainly care about this change. Why wouldn't they? This is
>>>> *not* a simple change. It took us months to bump the compiler in Log4j.
>> I
>>>> think your statement has an incorrect assumption on who the users of
>>>> `logging-parent` are.
>>> 
>>> Sorry, I was still talking about Log4j. For `logging-parent` users the
>>> requirement to use JDK 17 is a minor change, but `log4j-core` users do
>>> not care what JDK we are using for compilation. Therefore the switch
>>> to JDK 17 for compilation is not reason enough to bump Log4j to
>>> 2.23.0.
>>> 
>>>> I have the impression that you want to classify library updates that
>> don't
>>>> disrupt the user experience as a patch release. If there is nothing
>> urgent
>>>> about them, why do a patch release at all? Isn't the point of a patch
>>>> release is to fix something, urgently? Piggybacking library updates into
>>>> patch releases defeats the purpose of patch releases and makes the line
>>>> between minor and patch releases blur, and that is the crux of our
>>>> disagreement.
>>> 
>>> I would do a release at all if it only contains changes in the
>>> dependency versions. The only exception I would make is vulnerable
>>> dependencies, **if** we are affected by the vulnerability. If this is
>>> the case feel free to replace "fixed/upgraded" with "changed" in the
>>> changelog.
>>> 
>>> In case a dependency upgrade does not influence the bytecode (i.e. our
>>> artifacts still work with the old version), I would simply disregard
>>> the upgrade when computing the required version dump.
>>> 
>>> Piotr
>> 
>> 

Re: Versioning scheme

[Ralph Goers]

> On Dec 11, 2023, at 2:32 PM, Piotr P. Karwasz  wrote:
> 
> I propose to keep the old strategy: after a release we set the version
> number to the next patch release.
> Only if we merge a change that requires a minor bump (we can tag
> those), we bump the release to the next minor version.

Actually, the strategy was to always make the version number a patch release. 
It really didn’t matter what the version number was though. When I ran the 
release plugin I always specified the release version so all the pom versions 
got updated regardless of what they were.

I would prefer that the CI system determine the release number based on what is 
in the changelog. That simply means we need to have enough enumerations to 
handle all the cases and be able to classify each enumeration as either patch 
or minor.

Ralph

Re: Versioning scheme

[Ralph Goers]

I have to agree with Piotr’s example. Simply upgrading a dependency doesn’t, on 
its own, change any code in Log4j.

I see three possible solutions for this:

1. Do not allow any dependency updates until a “scheduled” mentor release (once 
every 2 or 3 months). Frankly I’d be fine with this except when a dependency 
has a major security vulnerability.
2. Change all dependency versions to be ranges such that they don’t require a 
release for users to include new dependency releases.  (I cannot really 
recommend doing this).
3. Add a new type to represent dependency updates. This would also not require 
a minor release. This is really the most appropriate fix as updating a 
dependency is not a change to anything in Log4j. 

I would suggest adding another enumeration named “update”.

Ralph

> On Dec 12, 2023, at 7:19 AM, Piotr P. Karwasz  wrote:
> 
> Hi Volkan,
> 
> On Tue, 12 Dec 2023 at 13:02, Volkan Yazıcı  > wrote:
>> 
>> On Tue, Dec 12, 2023 at 11:47 AM Piotr P. Karwasz 
>> wrote:
>> 
>>> * we lack a way to classify dependency updates. A concrete example:
>>> did the Commons DBCP2 bump to 2.11.0 change anything in
>>> `log4j-jdbc-dbcp2`? I don't think so, the artifact compiles with
>>> version 2.2.0 of the artifact. We are only stating that
>>> `log4j-jdbc-dbcp2` will **also** work with version 2.11.0.
>>> 
>> 
>> Exactly! Hence, it is clear that this is neither a bug fix, nor a
>> deprecation. That is, there is no ambiguity that this goes into a minor
>> release.
> 
> And here I don't agree. Dependabot upgrades provide **no** benefit to
> the generated JAR files. Even the bytecode is exactly the same as
> before the upgrade.
> 
> What these changes provide is part of our "Secure by default" or
> "Up-to-date by default" feature: Log4j artifacts will never freeze our
> clients dependencies and will work with the newest versions of the
> libraries we depend upon.
> 
> If you need to **manually** fix code for the upgrade to work (as you
> did for Jackson 2.16.0), then you can change the automatically
> generated entry from "fixed/upgraded" to "changed".
> 
>>> I don't think this warrants a minor version bump.
>>> 
>> 
>> This is what I am trying to eliminate Piotr: opinions. When a person starts
>> thinking *"Shall this be a patch or minor release?"*, the outcome is
>> inherently subjective. We cannot completely get rid of subjective
>> assessment, but assist it with guardrails.
> 
> These guardrails seem to follow the easy path: let's just do minor
> releases, so nobody will tell us we are wrong. If you add ".0.0" to
> all Google Chrome versions, Chrome will also follow semver to the
> letter. It just loses the spirit.
> 
>>> The bump to JDK 17 was necessary, very useful for us, but users don't
>>> really care what JDK was used for compilation.
>> 
>> 
>> What? Users, that is, developers using `logging-parent` as their parent
>> project, do certainly care about this change. Why wouldn't they? This is
>> *not* a simple change. It took us months to bump the compiler in Log4j. I
>> think your statement has an incorrect assumption on who the users of
>> `logging-parent` are.
> 
> Sorry, I was still talking about Log4j. For `logging-parent` users the
> requirement to use JDK 17 is a minor change, but `log4j-core` users do
> not care what JDK we are using for compilation. Therefore the switch
> to JDK 17 for compilation is not reason enough to bump Log4j to
> 2.23.0.
> 
>> I have the impression that you want to classify library updates that don't
>> disrupt the user experience as a patch release. If there is nothing urgent
>> about them, why do a patch release at all? Isn't the point of a patch
>> release is to fix something, urgently? Piggybacking library updates into
>> patch releases defeats the purpose of patch releases and makes the line
>> between minor and patch releases blur, and that is the crux of our
>> disagreement.
> 
> I would do a release at all if it only contains changes in the
> dependency versions. The only exception I would make is vulnerable
> dependencies, **if** we are affected by the vulnerability. If this is
> the case feel free to replace "fixed/upgraded" with "changed" in the
> changelog.
> 
> In case a dependency upgrade does not influence the bytecode (i.e. our
> artifacts still work with the old version), I would simply disregard
> the upgrade when computing the required version dump.
> 
> Piotr

Re: Versioning scheme

[Ralph Goers]

The rule for this seems extremely simple to me.  The changelog uses

Re: [flume] Website upgrade? (building the current site fails)

[Ralph Goers]

> On Dec 9, 2023, at 3:27 PM, Christian Grobmeier  wrote:
> 
> 
> On Sat, Dec 9, 2023, at 17:30, Ralph Goers wrote:
>> Did you follow the instructions at 
>> https://cwiki.apache.org/confluence/display/FLUME/Updating+the+Web+Site? 
>> I don’t know why it would be using Sphinx 1.1.2. 
> 
> Yes, I was failing at step 4 (mvn package)
> I think, it is the underlying Python libraries, they don't support 
> non-Windows OS, but I have no proof

I have always used MacOS so that cannot be it.


> 
>> For me it has always  used 1.1.3.
> 
> It is running Sphinx v1.1.3 after I upgraded to Sphinx-Maven plugin 1.0.3 - 
> but no success.

Why did you need to upgrade?  1.0.3 is specified in the pom.xml. 

> 
> 
>> To be honest I am not a fan of Sphinx or restructured text markup 
>> language. I would love to redo the whole site. I also have always hated 
>> the logo. It is supposed to be a log coming down a chute (i.e. a flume) 
>> but to me it looks like a giant snail. I would love to see that redone 
>> with something that really looks like a log coming down a chute.
> 
> Oh no, now I can see the snail too :)
> 
> I am happy with making it different and even integrating it into 
> logging-site. It should be trivial, and Flume can keep its template. I cannot 
> easily update the logo, but we can quickly improve the general look and feel.
> 
> Let me know if you agree with using logging-site for this content; then I 
> start migrating everything from RST.

Yes, it should reside at logging.apache.org/flume 
<http://logging.apache.org/flume>. flume.apache.org <http://flume.apache.org/> 
should redirect to that.

> 
>> FYI - here is the output of mvn dependency:resolve-plugins for the sphinx 
>> plugin
> 
> Yes, it is the same for me. I guess my Mac is the problem :-)

I don’t see how. As I said, I have always used a Mac. Are you using JDK 11? 

The python libraries I have installed on my machine are 

python
Python 2.7.18 (v2.7.18:8d21aa21f2, Apr 19 2020, 20:48:48) 
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.

That said, I am not sure these even get used. The maven plugin has Jython 
embedded in it.

I have a theory. In looking at my Java installs they all appear to be x86 
(which runs fine on an m2 Mac). I will try tomorrow on my Mac Studio which has 
the arm JDKs installed.

Ralph

Re: [flume] Website upgrade? (building the current site fails)

[Ralph Goers]

Obviously, I have never had a problem building the Flume web site, although I 
haven’t done it since the last release. I just ran the build with no problems.

Did you follow the instructions at 
https://cwiki.apache.org/confluence/display/FLUME/Updating+the+Web+Site? I 
don’t know why it would be using Sphinx 1.1.2. For me it has always used 1.1.3.

To be honest I am not a fan of Sphinx or restructured text markup language. I 
would love to redo the whole site. I also have always hated the logo. It is 
supposed to be a log coming down a chute (i.e. a flume) but to me it looks like 
a giant snail. I would love to see that redone with something that really looks 
like a log coming down a chute.

FYI - here is the output of mvn dependency:resolve-plugins for the sphinx plugin

[INFO]org.tomdz.maven:sphinx-maven-plugin:maven-plugin:1.0.3:runtime
[INFO]   org.tomdz.maven:sphinx-maven-plugin:jar:1.0.3
[INFO]   org.apache.maven.reporting:maven-reporting-api:jar:3.0
[INFO]   org.apache.maven.doxia:doxia-sink-api:jar:1.0
[INFO]   org.apache.maven.reporting:maven-reporting-impl:jar:2.0.5
[INFO]   org.apache.maven:maven-project:jar:2.0.6
[INFO]   org.apache.maven:maven-settings:jar:2.0.6
[INFO]   org.apache.maven:maven-profile:jar:2.0.6
[INFO]   org.apache.maven:maven-model:jar:2.0.6
[INFO]   org.apache.maven:maven-artifact-manager:jar:2.0.6
[INFO]   org.apache.maven:maven-repository-metadata:jar:2.0.6
[INFO]   org.apache.maven.wagon:wagon-provider-api:jar:1.0-beta-2
[INFO]   org.apache.maven:maven-plugin-registry:jar:2.0.6
[INFO]   org.apache.maven:maven-plugin-api:jar:2.0.6
[INFO]   org.apache.maven.doxia:doxia-core:jar:1.0
[INFO]   org.apache.maven.doxia:doxia-site-renderer:jar:1.0
[INFO]   org.codehaus.plexus:plexus-i18n:jar:1.0-beta-7
[INFO]   org.codehaus.plexus:plexus-velocity:jar:1.1.7
[INFO]   org.apache.velocity:velocity:jar:1.5
[INFO]   commons-lang:commons-lang:jar:2.1
[INFO]   org.apache.maven.doxia:doxia-decoration-model:jar:1.0
[INFO]   commons-collections:commons-collections:jar:3.2
[INFO]   org.apache.maven.doxia:doxia-module-apt:jar:1.0
[INFO]   org.apache.maven.doxia:doxia-module-fml:jar:1.0
[INFO]   org.apache.maven.doxia:doxia-module-xdoc:jar:1.0
[INFO]   org.apache.maven.doxia:doxia-module-xhtml:jar:1.0
[INFO]   org.apache.maven.shared:maven-doxia-tools:jar:1.0.2
[INFO]   commons-validator:commons-validator:jar:1.2.0
[INFO]   commons-beanutils:commons-beanutils:jar:1.7.0
[INFO]   commons-digester:commons-digester:jar:1.6
[INFO]   commons-logging:commons-logging:jar:1.0.4
[INFO]   oro:oro:jar:2.0.8
[INFO]   xml-apis:xml-apis:jar:1.0.b2
[INFO]   org.codehaus.plexus:plexus-utils:jar:1.5.8
[INFO]   org.apache.maven:maven-plugin-descriptor:jar:2.2.1
[INFO]   org.apache.maven:maven-artifact:jar:2.2.1
[INFO]   
org.codehaus.plexus:plexus-container-default:jar:1.0-alpha-9-stable-1
[INFO]   junit:junit:jar:3.8.1
[INFO]   classworlds:classworlds:jar:1.1-alpha-2
[INFO]   org.apache.commons:commons-compress:jar:1.1
[INFO]   commons-io:commons-io:jar:1.3.2
[INFO]   org.python:jython-standalone:jar:2.5.2



> On Dec 9, 2023, at 9:00 AM, Christian Grobmeier  wrote:
> 
> Hi,
> 
> I work with flume-site, which uses Sphinx.
> 
> Unfortunately, I cannot build the site because of the message below.
> A quick search gave me this issue:
> https://github.com/prestodb/presto/issues/3849
> 
> I am assuming now that Sphinx does not support mkdirs on my system (Mac).
> 
> Digging deeper, I saw this error might be fixed in more recent versions since 
> the current maven plugin did not see an update in 10 years:
> https://mvnrepository.com/artifact/org.tomdz.maven/sphinx-maven-plugin
> 
> It seems there is a successor, that was not active for a few years too:
> 
> https://trustin.github.io/sphinx-maven-plugin/basic-usage.html
> https://mvnrepository.com/artifact/kr.motd.maven/sphinx-maven-plugin
> 
> I could do the following:
> 
> 1) let somebody else run mvn package, in the hope Sphinx works there
> 2) try to upgrade to the other Sphinx plugin, that also seems not maintained, 
> but maybe works
> 3) create a Docker container, so it is independent of my operating system
> 4) replace it with something else (I would propose what I use with logging 
> site)
> 5) make this repo redirect to logging.apache.org/flume and transfer all 
> content the logging-site repository
> 
> What do you think? Ilike 5) most, to be honest (less things to deal with)
> 
> Kind regards,
> Christian
> 
> 
> # Sphinx version: 1.1.2
> # Python version: 2.5.2
> # Docutils version: 0.8.1 release
> # Jinja2 version: 2.6
> Traceback (most recent call last):
> ...
>  File "/ASF/flume-site/target/sphinx/sphinx/util/osutil.py", line 55, in 
> ensuredir
>os.makedirs(path)
>  File 
> "~/.m2/repository/org/python/jython-standalone/2.5.2/jython-standalone-2.5.2.jar/Lib/os.py",
>  line 158, in m