Answers below: > On Jan 8, 2022, at 17:34, Andrew Purtell <apurt...@apache.org> wrote: > > Log4J 1 has known concurrency issues but many projects live with them or > work around them. For example, several "Big Data" Apache projects have been > fine with it, themselves internally highly concurrent and performance > sensitive. Log4J 1 might not be a Platonic ideal, but certainly good > enough, as evidenced by the popularity of those projects, operational in > numerous high scale and/or performance sensitive environments.
It’s also a fairly common source of application performance problems that leads to reduction in logs being stored and the added difficulty of debugging problems with less context. Here are just some of the Bugzilla issues you’ve been conveniently ignoring for years: 50213 Category callAppenders synchronization causes java.lang.Thread.State: BLOCKED 46878 Deadlock in 1.2.15 caused by AsyncAppender and ThrowableInformation classes 41214 Deadlock with RollingFileAppender 44700 Log4J locks rolled log files (files can’t be deleted) 49481 Log4j stops writing to file, and then causes server to lockup 50323 Vulnerability in NTEventLogAppender 50463 AsyncAppender causing deadlock when dispatcher thread dies 50858 Classloader leak when using Log4j in a webapp container such as Tomcat, WebLogic 52141 [STUCK] ExecuteThread...Blocked trying to get lock: org/apache/log4j/Logger@0xc501e0a8[fat lock] 54009 Thread is getting Blocked 54325 Concurrency issues in AppenderAttachableImpl > In a perfect world Log4J 2 would have been API compatible and configuration > format compatible with 1, such that everyone could have migrated years ago, Can you point out any specific incompatibilities? Particularly in the current version. > but the Logging PMC/devs decided otherwise. Whether or not we assess that > as desirable, we arrive at the present, with hundreds, perhaps thousands, > of open source and internal projects stuck on Log4J 1 due to API or > operational compatibility concerns that would carry downstream to their > users. No matter what direction one might look (Log4J2, logback, etc.) > there are compatibility blockers. Staying with Log4J 1 can be very much > acceptable for this reason if the security concerns are addressed. Are you using the JMS appender? Are you using the socket receiver? If the answer is no to those questions, you don’t have security concerns besides the more glaring fact that you’re depending on end of life software that has been marked as such for going on 7 years now. > Sure, it > won't evolve much, and there will remain gotchas, but that is true with > anything... Some gotchas exceed others in concern. Compare lack of > asynchronous logging with the risks of a turing complete substitution > engine operating on untrusted user input. This is bullshit. JNDI is what leads to your Turing completeness. Not to mention that this substitution feature is meant for the configuration file, not for log messages, and the library has been updated as such regardless of downstream users possibly relying on the default behavior before it was apparent there was a security flaw. Based on how tepid you are about upgrading for even the slightest inconvenience, there’s a reason why version 2 maintains as much backward compatibility as possible by default to be least surprising. Plenty of work and security review has since been put into the library to disable and remove everything concerning related to string substitution of configuration files using JNDI, but this is also a wider problem for any library or application that uses JNDI without restricting it to only java URIs (e.g., users that use LDAP via JNDI). If the only reason you want to resurrect version 1 is due to a bug in version 2, then I suggest you leave the software industry entirely as nearly 100% of software in existence has flaws, many of them critical. > The Logging PMC is the hostile party here as far as I can tell, operating > in defiance of the community of users that have made the points I have just > written here abundantly clear for years. So far, we had one contributor show up with a complete hostile attitude who refused to engage with the PMC and explain what he wanted to do besides go wild with version 1. Later, Ceki came back and made his first commit in probably a decade attempting to do the same less than a day after we had completed voting on what to do with version 1, again, without ANY comments about what his intentions were or any engagement with the PMC. So far, these two people have spent most of their time trolling in other mailing lists on Apache or elsewhere about the situation while continuing to ignore the PMC’s arguments. If you have a desire to update version 1 that doesn’t boil down to “log4j2 bad bcuz log4shell lol get rekt n00bz”, please explain. > > On Sat, Jan 8, 2022 at 1:58 PM Matt Sicker <boa...@gmail.com> wrote: > >> The problem with v1 is that it doesn’t “just work”. There are numerous >> dead locks and other concurrency problems that were unable to be fixed >> without breaking various points of compatibility which is why Logback and >> Log4j2 even exist rather than continuing v1. There would also be difficulty >> in making a new release without volunteers contributing to the existing PMC >> to eventually join the PMC to be able to more directly create releases. >> Apache isn’t going to allow a hostile fork at the Incubator, so it’d still >> need to work with us or create an external fork that can’t be confused with >> the Apache version (which would still require a bunch of source code >> changes to point to the new package name if not already using shading or >> similar repackaging). >> >> I think it’s been said before, but contributing to the v2 backward >> compatibility system is a great way to join the project in such a way as to >> eventually form new releases of v1 with a greater understanding of the full >> impact of changing anything about v1 after such a long time. A poorly made >> release still wouldn’t be a drop in upgrade for projects that haven’t >> bothered upgrading in several years, and one that starts to introduce >> incompatible API changes would further complicate backward compatibility in >> v2 as well as complicating external projects that rely on a stable legacy >> API to convert from (like Logback’s config file converter thing or the >> numerous custom plugins for v1). So far, we haven’t seen any reasonably >> serious proposals of how anyone expects to do this without causing further >> mayhem in the future. >> >> — >> Matt Sicker >> >>> On Jan 8, 2022, at 14:32, Rohit Yadav <ro...@apache.org> wrote: >>> >>> Hi Matt, >>> >>> Thanks for replying. I think the main issues I found following the guide >> [1] for Apache CloudStack (ACS) are: >>> >>> - APIs are not backward compatible fully, certainly everywhere the >> imports have to be fixed >>> - The config xml files are not fully compatible requiring some changes >>> - Our codebase is pretty large with several maven modules/projects >> enabled by certain flags, testing changes and ensuring CloudStack works >> post-migration adds effort and risks >>> - Lack of an automatic tooling that can do this reliably and efficiently >> (for example, the Go lang ecosystem has gofmt and other tooling to migrate >> codebase across std library/lang changes). >>> - ACS's own tech debt and dependency issues: for example, we're using >> log4j-extras (https://logging.apache.org/log4j/extras/) which is >> completely discontinued, and have code tied around gson library (for one or >> more reason, we've hesitated to upgrade both log4j and gson dependencies) >>> >>> Appreciate all the hard work the Log4j team is doing and we'll report >> issues as/when we hit them. As a consumer of the dependency, 1.x gave us >> enough mileage and end of the day we want logging to be boring that just >> works; and therefore I'm simply exploring all possible options (both >> migration to 2.x or support/maintain 1.x fork) while understanding their >> tradeoffs. >>> >>> [1] https://logging.apache.org/log4j/2.x/manual/migration.html >>> >>>> On 2022/01/08 19:51:07 Matt Sicker wrote: >>>> It would be nice if you filed any issues with Log4j2 about problems >> with migration. It would have been nice to hear about these issues back >> when v1 stopped development, but this is the next best time to do so. The >> Log4j team are actively working to fill in any remaining gaps on backward >> compatibility; making new releases of EOL software with numerous >> implementation errors inherent to its architecture seems like going >> backwards. >>>> >>>> — >>>> Matt Sicker >>>> >>>>>> On Jan 8, 2022, at 13:45, Rohit Yadav <ro...@apache.org> wrote: >>>>> >>>>> Hi all, >>>>> >>>>> I agree and extend support for Andrew's remarks. Apache CloudStack too >> uses log4j 1.x and our use case is simply a logging library that 1.x just >> satisfies. The effort to migrate to 2.x is not quick, at least in our >> initial investigation and a migration may likely require huge effort in >> testing. >>>>> >>>>> Assuming this quick upgrade path doesn't exist, and I already read >> struggles by other projects trying to migrate to 2.x - maintaining 1.x and >> doing a 1.2.x release makes more sense than investing weeks in migration >> and testing to 2.x, while maintaining the same artifact IDs. I'm up for >> volunteering and supporting 1.x maintenance. >>>>> >>>>> Regards, >>>>> Rohit Yadav >>>>> PMC, Apache CloudStack >>>>> >>>>> On 2021/12/21 21:54:34 Andrew Purtell wrote: >>>>>>> as for the v1 :: COBOL analogy, that’s not a bad comparison. >> Basically, >>>>>> users who haven’t bothered to upgrade in 10 years will have to end up >>>>>> paying astronomical costs for consultants who can still work on >> ancient >>>>>> software effectively to help modify their systems. >>>>>> >>>>>> I have to take some exception to this. If the log4j 2.x configuration >> files >>>>>> were compatible _enough_ with 1.x then taking this position would be >>>>>> understandable. However, because they are not compatible in the way >> that >>>>>> users require -- and the backwards compatibility is still marked as >>>>>> 'experimental', even -- it is not great to blame users "who haven't >>>>>> bothered to upgrade in 10 years". Turning this around, why is >> backwards >>>>>> compatibility still experimental after 10 years? I am involved with >> several >>>>>> Apache projects where we would love to upgrade from log4j 1 to log4j >> 2, and >>>>>> have been talking about it for _years_. However, we have not been >> able to >>>>>> easily do so because we actually care about operational cross-version >>>>>> compatibility for our users. On some of these projects we are still >> stuck >>>>>> on log4j 1. >>>>>> >>>>>> I also support continuing releasing for log4j 1.x, and would >> volunteer some >>>>>> of my time to assist in the effort. >>>>>> >>>>>> >>>>>> On Tue, Dec 21, 2021 at 12:34 PM Matt Sicker <boa...@gmail.com> >> wrote: >>>>>> >>>>>>> Nobody in the Logging PMC is blocking a release here. What we don’t >> want >>>>>>> is to falsely advertise that v1 is still under development. We >> already have >>>>>>> a huge increase in mailing list, PR, and other traffic ever since >>>>>>> Log4Shell, and if we resurrect v1, then it’ll quickly become >> impossible to >>>>>>> keep up with all the activity given the size of the PMC. If any >> non-trivial >>>>>>> work is to be done in v1, we’d prefer to see more than one person >> working >>>>>>> on that, especially if we want to add more PMC members to oversee v1 >> in the >>>>>>> first place. >>>>>>> >>>>>>> And as for the v1 :: COBOL analogy, that’s not a bad comparison. >>>>>>> Basically, users who haven’t bothered to upgrade in 10 years will >> have to >>>>>>> end up paying astronomical costs for consultants who can still work >> on >>>>>>> ancient software effectively to help modify their systems. Was Maven >> even >>>>>>> widely used back when v1 was popular? Or were people still using a >> mix of >>>>>>> make or Ant? >>>>>>> -- >>>>>>> Matt Sicker >>>>>>> >>>>>>>> On Dec 21, 2021, at 07:13, Romain Manni-Bucau < >> rmannibu...@gmail.com> >>>>>>> wrote: >>>>>>>> >>>>>>>> Le mar. 21 déc. 2021 à 12:33, Enrico Olivelli <eolive...@gmail.com> >> a >>>>>>>> écrit : >>>>>>>> >>>>>>>>> Vladimir, >>>>>>>>> I totally support this proposal. >>>>>>>>> >>>>>>>>> Which are actually the steps we need to cut a release of log4j 1.x >> ? >>>>>>>>> - establish an Apache project ? >>>>>>>>> >>>>>>>> >>>>>>>> 1. Send a patch to apply on >>>>>>>> http://svn.apache.org/repos/asf/logging/log4j/trunk >>>>>>>> >>>>>>>> >>>>>>>>> - do the fix >>>>>>>>> >>>>>>>> >>>>>>>> 2. Get it applied >>>>>>>> >>>>>>>> >>>>>>>>> - cut a release >>>>>>>>> >>>>>>>>> Can this be done inside another Apache Project who "adopts" the >> log4j >>>>>>>>> sources if the Logging Project doesn't want to do it ? >>>>>>>>> >>>>>>>> >>>>>>>> The PMC of log4j2 is logging project so it should be done there, if >> not >>>>>>> the >>>>>>>> project can be forked inside Apache but should change of package >> until we >>>>>>>> get the perms to reuse the same one which means likely as much work >> as >>>>>>> just >>>>>>>> getting it done at logging project so hope it is not needed ;). >>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> Enrico >>>>>>>>> >>>>>>>>> >>>>>>>>> Il giorno mar 21 dic 2021 alle ore 08:36 Vladimir Sitnikov < >>>>>>>>> sitnikov.vladi...@gmail.com> ha scritto: >>>>>>>>> >>>>>>>>>>> Just wondering, is it even fulfilling the criteria of incubation? >>>>>>>>>> >>>>>>>>>> I believe, the world does not need "active development in log4j >> 1.x" >>>>>>>>>> nowadays. >>>>>>>>>> What everybody needs from log4j 1.x is to fix security issues, fix >>>>>>>>>> outstanding issues (if any), >>>>>>>>>> keep the project buildable (e.g. avoid using outdated build >> systems), >>>>>>>>> etc. >>>>>>>>>> >>>>>>>>>>> it doesn't seem that sustainability is proven. >>>>>>>>>> >>>>>>>>>> The problem is log4j 1.x is like COBOL of logging. There are apps >> that >>>>>>>>> are >>>>>>>>>> just stuck with log4j 1.x. >>>>>>>>>> The proof of sustainability is that lots of existing apps will >> never >>>>>>>>>> upgrade to 2.x because 2.x is incompatible. >>>>>>>>>> If the compatibility layer of 2.x would be improved to handle >> 99.999% >>>>>>> of >>>>>>>>>> apps, >>>>>>>>>> then we could indeed move 1.x to the attic. >>>>>>>>>> >>>>>>>>>> The Incubator Cookbook says: >>>>>>>>>>> The ASF provides software for the public good, >>>>>>>>>> >>>>>>>>>> As I described, log4j 2.x is not a direct replacement for log4j >> 1.x, >>>>>>> and >>>>>>>>>> there are **lots** of applications >>>>>>>>>> that can't easily be upgraded to 2.x due to testing, >> configuration, and >>>>>>>>>> implementation issues. >>>>>>>>>> >>>>>>>>>> The current Logging PMC is focused on log4j 2.x only, and they >> have no >>>>>>>>>> desire to release 1.x >>>>>>>>>> >>>>>>>>>>> active development but focus only on CVE fixes >>>>>>>>>> >>>>>>>>>> I would say, the primary goal of resurrecting 1.x is to focus on >> CVEs, >>>>>>>>> and >>>>>>>>>> keep the project buildable and testable. >>>>>>>>>> However, it might be the case, that certain fixes or features >> would >>>>>>>>> appear. >>>>>>>>>> >>>>>>>>>> The sad story is that the industry is using 1.x A LOT, and what >> Logging >>>>>>>>> PMC >>>>>>>>>> did was >>>>>>>>>> they ignored the community, and they just stopped maintaining 1.x >> and >>>>>>>>>> focused on an incompatible 2.x >>>>>>>>>> >>>>>>>>>> Not only do they stop maintaining 1.x, but they also deny others >> to >>>>>>> pick >>>>>>>>> up >>>>>>>>>> the maintenance task. >>>>>>>>>> >>>>>>>>>> What I am trying to do now is to pick up that maintenance >> activity. >>>>>>>>>> >>>>>>>>>> Vladimir >>>>>>>>>> >>>>>>>>> >>>>>>> >>>>>>> >>>>>>> --------------------------------------------------------------------- >>>>>>> To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org >>>>>>> For additional commands, e-mail: general-h...@incubator.apache.org >>>>>>> >>>>>>> >>>>>> >>>>>> -- >>>>>> Best regards, >>>>>> Andrew >>>>>> >>>>>> Words like orphans lost among the crosstalk, meaning torn from truth's >>>>>> decrepit hands >>>>>> - A23, Crosstalk >>>>>> >>>>> >>>>> --------------------------------------------------------------------- >>>>> To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org >>>>> For additional commands, e-mail: general-h...@incubator.apache.org >>>>> >>>> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org >>> For additional commands, e-mail: general-h...@incubator.apache.org >>> >> > > > -- > Best regards, > Andrew > > Words like orphans lost among the crosstalk, meaning torn from truth's > decrepit hands > - A23, Crosstalk
