Yep that's what I tried, roughly - there is an old jira about it. The issue is that Spark does need to configure some concrete logging framework in a few cases, as do other libs, and that isn't what the shims cover. Could be possible now or with more cleverness but the simple thing didn't work out IIRC.
On Tue, Nov 9, 2021, 4:32 PM Stephen Coy <s...@infomedia.com.au> wrote: > Hi there, > > It’s true that the preponderance of log4j 1.2.x in many existing live > projects is kind of a pain in the butt. > > But there is a solution. > > 1. Migrate all Spark code to use slf4j APIs; > > 2. Exclude log4j 1.2.x from any dependencies sucking it in; > > 3. Include the log4j-over-slf4j bridge jar and slf4j-api jars; > > 4. Choose your favourite modern logging implementation and add it as a > “runtime" dependency together with it’s slf4j binding jar (if needed). > > In fact in the short term you can replace steps 1 and 2 with "remove the > log4j 1.2.17 jar from the distribution" and it should still work. > > The slf4j project also includes a commons-logging shim for capturing its > output too. > > FWIW, the slf4j project is run by one of the original log4j developers. > > Cheers, > > Steve C > > > On 9 Nov 2021, at 11:11 pm, Sean Owen <sro...@gmail.com> wrote: > > No plans that I know of. It's not that Spark uses it so much as its > dependencies. I tried and failed to upgrade it a couple years ago. you are > welcome to try, and open a PR if successful. > > On Tue, Nov 9, 2021 at 6:09 AM Ajay Kumar <ajay.praja...@gmail.com> wrote: > >> Hi Team, >> We wanted to send Spark executor logs to a centralized logging server >> using TCP Socket. I see that the spark log4j version is very old(1.2.17) >> and it does not support JSON logs over tcp sockets on containers. >> I wanted to konw what is the plan for upgrading the log4j version to >> log4j2. >> Thanks in advance. >> Regards, >> Ajay >> > > This email contains confidential information of and is the copyright of > Infomedia. It must not be forwarded, amended or disclosed without consent > of the sender. If you received this message by mistake, please advise the > sender and delete all copies. Security of transmission on the internet > cannot be guaranteed, could be infected, intercepted, or corrupted and you > should ensure you have suitable antivirus protection in place. By sending > us your or any third party personal details, you consent to (or confirm you > have obtained consent from such third parties) to Infomedia’s privacy > policy. http://www.infomedia.com.au/privacy-policy/ >
