When I was looking at the PR I also noticed a potentially annoying and compatibility breaking problem with the properties file format. Is it really true that with log4j1 the properties begin with 'log4j.' and with log4j2 the 'log4j.' prefix is removed? If so user configured files won't be compatible unless we can intercept the parse of the file and map the one to the other, somehow. Pardon my ignorance if that would not be necessary and there is actually a path to compatibility. Anyway I mention it because if it is an issue, for both of these issues, perhaps we can plug something in rather than rely on the log4j community? It is just a thought...
On Mon, Feb 7, 2022 at 4:26 PM 张铎(Duo Zhang) <palomino...@gmail.com> wrote: > Yes, the current solution does not work, they do not want to do variable > lookup before entering the format free configuration processing step. > > Will try to push the log4j community to give another solution. > > Thanks. > > Andrew Purtell <apurt...@apache.org> 于2022年2月8日周二 03:54写道： > > > So it looks like LOG4J2-3341 is going to be reverted? Backport seems > > premature until this is resolved with the new solution. > > > > On Sat, Feb 5, 2022 at 12:04 PM Andrew Purtell <apurt...@apache.org> > > wrote: > > > > > Thank you for providing a PR, will review on Monday. > > > > > > On Sat, Feb 5, 2022 at 5:16 AM 张铎(Duo Zhang) <palomino...@gmail.com> > > > wrote: > > > > > >> https://github.com/apache/hbase/pull/4096 > > >> > > >> The PR based on log4j 2.17.2-SNAPSHOT is ready. > > >> > > >> PTAL. > > >> > > >> Next I will build the tarball and test whether the logging works as > > >> expected. > > >> > > >> 张铎(Duo Zhang) <palomino...@gmail.com> 于2022年2月5日周六 16:05写道： > > >> > > >> > On the config file format, on master branch I switched to xml > because > > >> most > > >> > log4j2 related resources(for example, answers on stackoverflow...) > are > > >> xml > > >> > based. But LOG4J2-3341 can only work with properties based config > file > > >> > format so the plan is to change back to properties file. > > >> > Of course you still need to tweak the config files, as the config > > names > > >> > are not exactly the same, but if we have LOG4J2-3341 then I do not > > think > > >> > our users need to tweak the scripts, this is good news at least. > > >> > > > >> > We have already tried our best to not introduce log4j dependencies > to > > >> our > > >> > downstream users so I think the current branch-2.5 is already > > >> > 'incompatible' enough for our users as in the old time we were > likely > > to > > >> > pull in log4j and slf4j-log4j if they depend on hbase-testing-util. > > But > > >> for > > >> > me, I do not think introducing log4j as a dependency is the correct > > way > > >> so > > >> > I would like to include this and add a section in the release > > >> announcement > > >> > to say this. > > >> > > > >> > In short, I'm +1 on switching to log4j2 on branch-2.5 and I will > offer > > >> my > > >> > help to land the changes. > > >> > > > >> > Thanks. > > >> > > > >> > Andrew Purtell <apurt...@apache.org> 于2022年2月1日周二 02:49写道： > > >> > > > >> >> That is good news. > > >> >> WDYT about integrating this work into branch-2.5 / 2.5.0, and > > >> branch-2, of > > >> >> course. Is it compatible enough? Needing to update a properties > file > > >> and > > >> >> tweaks to launch scripts, if any, would be fine and can be handled > > in a > > >> >> release note. Switching away from a properties based format to an > XML > > >> >> format would not (just as example). > > >> >> > > >> >> On Sun, Jan 30, 2022 at 4:57 AM 张铎(Duo Zhang) < > palomino...@gmail.com > > > > > >> >> wrote: > > >> >> > > >> >> > Good news, LOG4J2-3341 has been landed. It will be released in > > log4j > > >> >> > 2.17.2. > > >> >> > > > >> >> > Will start to test it soon. > > >> >> > > > >> >> > Thanks. > > >> >> > > > >> >> > 张铎(Duo Zhang) <palomino...@gmail.com> 于2022年1月25日周二 17:15写道： > > >> >> > > > >> >> > > I would prefer we have LOG4J2-3341 first before releasing any > 2.x > > >> >> > releases > > >> >> > > with log4j2. I pinged the log4j community once but no response > > yet. > > >> >> Will > > >> >> > > try to ping again soon. > > >> >> > > > > >> >> > > Andrew Purtell <andrew.purt...@gmail.com> 于2022年1月25日周二 > 10:09写道： > > >> >> > > > > >> >> > >> Reload4J is a great option when we really should maintain > fairly > > >> >> strict > > >> >> > >> compatibility (patch releases) and less so when not (minors, > > >> majors). > > >> >> > >> > > >> >> > >> We haven’t released 2.5.0 yet. My fault, mainly... It’s been > > >> lagging. > > >> >> > >> Perhaps we could backport Duo’s work from 3.0.0-alpha into > > >> >> branch-2.5 / > > >> >> > >> 2.5.0? This would be a minor release, so, while some > operational > > >> >> > >> compatibility breaks would be somewhat surprising, it could be > > >> more > > >> >> > >> understandable. > > >> >> > >> > > >> >> > >> > On Jan 24, 2022, at 5:00 PM, Zach York <zy...@apache.org> > > >> wrote: > > >> >> > >> > > > >> >> > >> > +1 on the original discussion > > >> >> > >> > > > >> >> > >> > I think that all makes sense, we can't know whether one > > >> dependency > > >> >> is > > >> >> > >> more > > >> >> > >> > vulnerable/going to be more work to maintain or not. > However, > > I > > >> >> think > > >> >> > >> > perhaps the more interesting question is whether we should > be > > >> okay > > >> >> > with > > >> >> > >> > using EOL dependencies in any active release line. CVEs > > >> happen... I > > >> >> > >> think > > >> >> > >> > it's important to be able to react quickly. By depending on > > any > > >> EOL > > >> >> > >> code in > > >> >> > >> > our active release lines, we severely limit our ability to > > >> quickly > > >> >> > deal > > >> >> > >> > with CVEs. I understand it's not always easy (the backwards > > >> >> > >> incompatibility > > >> >> > >> > of log4j2 and the properties files are good examples), but > > it's > > >> >> also > > >> >> > >> been > > >> >> > >> > 6+ years since log4j 1.x has become EOL. > > >> >> > >> > > > >> >> > >> >> On Fri, Jan 21, 2022 at 11:47 AM Andrew Purtell < > > >> >> apurt...@apache.org > > >> >> > > > > >> >> > >> wrote: > > >> >> > >> >> > > >> >> > >> >> I will offer a guess as answer to the question originally > > >> >> proposed, > > >> >> > >> which > > >> >> > >> >> is, no the Logging PMC is not going to behave in a > > cooperative > > >> >> manner > > >> >> > >> to > > >> >> > >> >> Reload4J. The best we can hope for, and I personally doubt > > it, > > >> is > > >> >> > they > > >> >> > >> will > > >> >> > >> >> not be actively hostile. Decisions made by Apache PMC can > > >> >> > >> unfortunately be > > >> >> > >> >> made on the basis of years-long running arguments and > > >> personality > > >> >> > >> >> conflicts, and Logging is unfortunately one of them. Just > my > > >> >> opinion. > > >> >> > >> You > > >> >> > >> >> won't change it. Fortunately that is neither here nor > there. > > We > > >> >> > aren't > > >> >> > >> here > > >> >> > >> >> to judge up front if Reload4J can respond adequately to > > >> >> hypothetical > > >> >> > >> future > > >> >> > >> >> security issues. That is not knowable and remains to be > seen. > > >> It > > >> >> also > > >> >> > >> >> remains to be seen if Log4J 2 is going to respond > adequately > > to > > >> >> > future > > >> >> > >> >> security issues. Or Netty. Or Jersey. Or Jackson. Or any of > > our > > >> >> other > > >> >> > >> risky > > >> >> > >> >> third party dependencies (as measured by having > "interesting" > > >> >> CVEs). > > >> >> > >> What > > >> >> > >> >> we can do is look at the current track record, which has > been > > >> >> > adequate > > >> >> > >> so > > >> >> > >> >> far. > > >> >> > >> >> > > >> >> > >> >> > > >> >> > >> >> On Fri, Jan 21, 2022 at 11:35 AM Wei-Chiu Chuang > > >> >> > >> >> <weic...@cloudera.com.invalid> wrote: > > >> >> > >> >> > > >> >> > >> >>> The reload4j is maintained by one of the Apache Logging > PMC. > > >> And > > >> >> a > > >> >> > >> >> release > > >> >> > >> >>> was made to address the latest log4j 1.x CVEs announced > > only a > > >> >> day > > >> >> > >> after. > > >> >> > >> >>> > > >> >> > >> >>> There is a thread in the private@logging that mentioned > the > > >> >> “new” > > >> >> > >> >> log4j1.x > > >> >> > >> >>> CVEs were actually not new. They were announced years > before > > >> for > > >> >> 2.x > > >> >> > >> >> which > > >> >> > >> >>> happens to also impact 1.x. But because 1.x was considered > > >> EOL, > > >> >> they > > >> >> > >> >> didn’t > > >> >> > >> >>> bother to mention 1.x were affected. It is only now that > > 1.x’s > > >> >> > getting > > >> >> > >> >>> interest that they did this. > > >> >> > >> >>> > > >> >> > >> >>> Sean Busbey <bus...@apache.org>於 2022年1月22日 週六，上午2:47寫道： > > >> >> > >> >>> > > >> >> > >> >>>> It's relevant to what kind of mitigation this is. The > > >> >> effectiveness > > >> >> > >> of > > >> >> > >> >>>> reload4j to deal with "the critical CVEs of log4j 1" is > > >> limited > > >> >> by > > >> >> > >> how > > >> >> > >> >>>> likely it is that they know about them. > > >> >> > >> >>>> > > >> >> > >> >>>> Otherwise at the next CVE we're back in the same place > > where > > >> >> > >> downstream > > >> >> > >> >>>> users aren't meaningfully more protected. And in that > case > > >> >> perhaps > > >> >> > we > > >> >> > >> >>> would > > >> >> > >> >>>> do better for our users by e.g. putting more emphasis > > >> upgrading > > >> >> > >> across > > >> >> > >> >>> our > > >> >> > >> >>>> releases or providing breaking changes to get the pain > over > > >> >> with. > > >> >> > >> >>>> > > >> >> > >> >>>> On Fri, Jan 21, 2022 at 11:03 AM Andrew Purtell < > > >> >> > >> >>> andrew.purt...@gmail.com> > > >> >> > >> >>>> wrote: > > >> >> > >> >>>> > > >> >> > >> >>>>> That does not seem germane to this discussion. We don’t > > >> >> > investigate > > >> >> > >> >> and > > >> >> > >> >>>>> attempt to manage the security reporting arrangement of > > any > > >> of > > >> >> our > > >> >> > >> >>> other > > >> >> > >> >>>>> third party dependencies. > > >> >> > >> >>>>> > > >> >> > >> >>>>>> On Jan 21, 2022, at 7:59 AM, Sean Busbey < > > >> bus...@apache.org> > > >> >> > >> >> wrote: > > >> >> > >> >>>>>> > > >> >> > >> >>>>>> Has anyone asked the ASF Logging PMC if they'll > forward > > >> >> security > > >> >> > >> >>>> reports > > >> >> > >> >>>>>> against log4j 1 to the reload4j project? > > >> >> > >> >>>>>> > > >> >> > >> >>>>>>> On Fri, Jan 21, 2022 at 3:33 AM Pankaj Kumar < > > >> >> > >> >>> pankajku...@apache.org> > > >> >> > >> >>>>> wrote: > > >> >> > >> >>>>>>> > > >> >> > >> >>>>>>> +1 for reload4j. > > >> >> > >> >>>>>>> > > >> >> > >> >>>>>>> Regards, > > >> >> > >> >>>>>>> Pankaj > > >> >> > >> >>>>>>> > > >> >> > >> >>>>>>>> On Fri, Jan 21, 2022, 2:39 PM 张铎(Duo Zhang) < > > >> >> > >> >> palomino...@gmail.com > > >> >> > >> >>>> > > >> >> > >> >>>>> wrote: > > >> >> > >> >>>>>>>> > > >> >> > >> >>>>>>>> Already filed HBASE-26691. > > >> >> > >> >>>>>>>> > > >> >> > >> >>>>>>>> Wei-Chiu Chuang <weic...@apache.org> 于2022年1月21日周五 > > >> 16:53写道： > > >> >> > >> >>>>>>>> > > >> >> > >> >>>>>>>>> +1 I am doing the same in Hadoop. > > >> >> > >> >>>>>>>>> > > >> >> > >> >>>>>>>>> On Fri, Jan 21, 2022 at 4:51 PM Viraj Jasani < > > >> >> > >> >> vjas...@apache.org> > > >> >> > >> >>>>>>> wrote: > > >> >> > >> >>>>>>>>> > > >> >> > >> >>>>>>>>>> +1 for Reload4J migration in active release > branches. > > >> >> > >> >>>>>>>>>> > > >> >> > >> >>>>>>>>>> > > >> >> > >> >>>>>>>>>> On Fri, 21 Jan 2022 at 12:52 PM, Andrew Purtell < > > >> >> > >> >>>>>>>>> andrew.purt...@gmail.com> > > >> >> > >> >>>>>>>>>> wrote: > > >> >> > >> >>>>>>>>>> > > >> >> > >> >>>>>>>>>>> +1 for migrating to Reload4J. It is binary and > > >> >> configuration > > >> >> > >> >>>>>>>> compatible > > >> >> > >> >>>>>>>>>>> with log4j 1 so meets our compatibility > guidelines. > > >> >> > >> >>>>>>>>>>> > > >> >> > >> >>>>>>>>>>> If this is an agreeable plan I can make the > changes > > >> in a > > >> >> PR > > >> >> > >> >> and > > >> >> > >> >>> we > > >> >> > >> >>>>>>>> can > > >> >> > >> >>>>>>>>> do > > >> >> > >> >>>>>>>>>>> a round of new releases. > > >> >> > >> >>>>>>>>>>> > > >> >> > >> >>>>>>>>>>>> On Jan 20, 2022, at 10:16 PM, Duo Zhang < > > >> >> > zhang...@apache.org > > >> >> > >> >>> > > >> >> > >> >>>>>>>> wrote: > > >> >> > >> >>>>>>>>>>>> > > >> >> > >> >>>>>>>>>>>> On master we have already migrated to log4j2, > but > > >> for > > >> >> all > > >> >> > >> >>> other > > >> >> > >> >>>>>>>>>> release > > >> >> > >> >>>>>>>>>>>> lines we are still on log4j1. > > >> >> > >> >>>>>>>>>>>> > > >> >> > >> >>>>>>>>>>>> Recently there are several new CVEs for log4j1, > so > > I > > >> >> think > > >> >> > we > > >> >> > >> >>>>>>>> should > > >> >> > >> >>>>>>>>>> also > > >> >> > >> >>>>>>>>>>>> address them for release lines other than master. > > >> >> > >> >>>>>>>>>>>> > > >> >> > >> >>>>>>>>>>>> One possible solution is to also migrate log4j2 > but > > >> use > > >> >> > >> >> log4j12 > > >> >> > >> >>>>>>>>> bridge > > >> >> > >> >>>>>>>>>> to > > >> >> > >> >>>>>>>>>>>> maintain the compatibility, but we have already > > known > > >> >> that > > >> >> > >> >>>>>>> log4j12 > > >> >> > >> >>>>>>>>>> bridge > > >> >> > >> >>>>>>>>>>>> can not work perfectly with hadoop, as hadoop has > > >> some > > >> >> > >> >>> customized > > >> >> > >> >>>>>>>>>> log4j1 > > >> >> > >> >>>>>>>>>>>> appender implementations, which inherit some > log4j1 > > >> >> > appenders > > >> >> > >> >>>>>>> which > > >> >> > >> >>>>>>>>> are > > >> >> > >> >>>>>>>>>>> not > > >> >> > >> >>>>>>>>>>>> part of the log4j12 bridge. > > >> >> > >> >>>>>>>>>>>> > > >> >> > >> >>>>>>>>>>>> Reload4j is a fork of the log4j1 and has fixed > the > > >> >> critical > > >> >> > >> >>> CVEs, > > >> >> > >> >>>>>>>> so > > >> >> > >> >>>>>>>>> it > > >> >> > >> >>>>>>>>>>> is > > >> >> > >> >>>>>>>>>>>> less hurt to replace log4j with reload4j. > > >> >> > >> >>>>>>>>>>>> > > >> >> > >> >>>>>>>>>>>> Suggestions are welcomed. > > >> >> > >> >>>>>>>>>>>> > > >> >> > >> >>>>>>>>>>>> Thanks. Regards > > >> >> > >> >>>>>>>>>>> > > >> >> > >> >>>>>>>>>> > > >> >> > >> >>>>>>>>> > > >> >> > >> >>>>>>>> > > >> >> > >> >>>>>>> > > >> >> > >> >>>>> > > >> >> > >> >>>> > > >> >> > >> >>> > > >> >> > >> >> > > >> >> > >> >> > > >> >> > >> >> -- > > >> >> > >> >> Best regards, > > >> >> > >> >> Andrew > > >> >> > >> >> > > >> >> > >> >> Unrest, ignorance distilled, nihilistic imbeciles - > > >> >> > >> >> It's what we’ve earned > > >> >> > >> >> Welcome, apocalypse, what’s taken you so long? > > >> >> > >> >> Bring us the fitting end that we’ve been counting on > > >> >> > >> >> - A23, Welcome, Apocalypse > > >> >> > >> >> > > >> >> > >> > > >> >> > > > > >> >> > > > >> >> > > >> >> > > >> >> -- > > >> >> Best regards, > > >> >> Andrew > > >> >> > > >> >> Unrest, ignorance distilled, nihilistic imbeciles - > > >> >> It's what we’ve earned > > >> >> Welcome, apocalypse, what’s taken you so long? > > >> >> Bring us the fitting end that we’ve been counting on > > >> >> - A23, Welcome, Apocalypse > > >> >> > > >> > > > >> > > > > > > > > > -- > > > Best regards, > > > Andrew > > > > > > Unrest, ignorance distilled, nihilistic imbeciles - > > > It's what we’ve earned > > > Welcome, apocalypse, what’s taken you so long? > > > Bring us the fitting end that we’ve been counting on > > > - A23, Welcome, Apocalypse > > > > > > > > > -- > > Best regards, > > Andrew > > > > Unrest, ignorance distilled, nihilistic imbeciles - > > It's what we’ve earned > > Welcome, apocalypse, what’s taken you so long? > > Bring us the fitting end that we’ve been counting on > > - A23, Welcome, Apocalypse > > > -- Best regards, Andrew Unrest, ignorance distilled, nihilistic imbeciles - It's what we’ve earned Welcome, apocalypse, what’s taken you so long? Bring us the fitting end that we’ve been counting on - A23, Welcome, Apocalypse
