Hi Rion
I guess you’re building your own docker image for the deployment right?
For switching to Logback I’m doing the following command (sbt-docker) when
building the image.
val eclasspath = (Compile / externalDependencyClasspath).value
val logbackClassicJar = eclasspath.files.find(file =>
file.getName.contains("logback-classic"))
logbackClassicJar.foreach(logback => add(logback, «opt/flink/lib»))
Given the error message that you provided I think the dependency is missing in
the lib folder (don’t confuse this with the usrlib folder).
Kind Regards
Dominik
From: Rion Williams <[email protected]>
Date: Thursday, 22 February 2024 at 13:09
To: Bünzli Dominik, INI-DNA-INF <[email protected]>
Cc: [email protected] <[email protected]>
Subject: Re: Using Custom JSON Formatting with Flink Operator
Be aware: This is an external email.
Hi Dominick,
In this case the jobs are running using application-mode. All of these were
previously working as expected for the legacy jobs using the same configuration
(however those were running via Ververica Platform and targeting Flink 1.15.2).
I had somewhat expected similar behaviors but it seems there’s something that
is missing.
Thanks,
Rion
On Feb 22, 2024, at 1:15 AM, [email protected] wrote:
Good morning Rion,
Are you in session job mode or application mode? I’ve had some similar issues
(logback) lately and it turned out that I also needed to add the additional
dependencies (I guess JsonTemplateLayout is one of them) to the lib folder of
the deployment.
Kind regards
Dominik
From: Rion Williams <[email protected]>
Date: Thursday, 22 February 2024 at 00:46
To: Flink User List <[email protected]>
Subject: Using Custom JSON Formatting with Flink Operator
Be aware: This is an external email.
Hey Flinkers,
Recently I’ve been in the process of migrating a series of older Flink jobs to
use the official operator and have run into a snag on the logging front.
I’ve attempted to use the following configuration for the job:
```
logConfiguration:
log4j-console.properties: |+
rootLogger.level = INFO
rootLogger.appenderRef.console.ref = ConsoleAppender
rootLogger.appenderRef.rolling.ref = RollingFileAppender
...
appender.console.name = ConsoleAppender
appender.console.type = Console
appender.console.layout.type = JsonTemplateLayout
appender.console.layout.eventTemplateUri = classpath:GcpLayout.json
```
However once the job begins running, I’m met with the following errors in the
logs:
```
ERROR Unable to locate plugin type for JsonTemplateLayout
ERROR Unable to locate plugin for JsonTemplateLayout
ERROR Could not create plugin of type class
org.apache.logging.log4j.core.appender.ConsoleAppender for element Console:
java.lang.NullPointerException java.lang.NullPointerException
```
I believe that all of the appropriate references are correct in the actual
shaded jar itself as I can see things like the JsonTemplateLayout inside of it
(under org.apache.logging.log4j.template.json.JsonTemplateLayout ) as well as
the GcpLayout that I’m targeting in the root of the shaded jar as well
(including trying several adjustments to shade exceptions, adding a log4j
specific shade transformer, etc.)
I’ve tried adjusting several different knobs/configurations but I’m still
continually getting this same error. I’d be happy to share any additional
configuration for the job any/or the FlinkDeployment where applicable.
Just a bit stumped here on something that feels like it should just work.
