Thanks, Robert!
I tried the classloader.resolve.order: parent-first option but ran into
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder" errors
(because I use logback so I followed
https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/advanced/logging/#configuring-logback
and removed log4j-slf4j-impl from the classpath. But putting all my classes
in lib/ instead of usrlib/ fixed that problem, and everything now runs
fine. Thanks!
El vie, 28 ene 2022 a las 6:11, Robert Metzger ()
escribió:
> Hi Javier,
>
> I suspect that TwitterServer is using some classloading / dependency
> injection / service loading "magic" that is causing this.
> I would try to find out, either by attaching a remote debugger (should be
> possible when executing in cluster mode locally) or by adding log
> statements in the code, what the URI it's trying to load looks like.
>
> On the cluster, Flink is using separate classloaders for the base flink
> system, and the user code (as opposed to executing in the IDE, where
> everything is loaded from the same loader). Check out this page and try out
> the config arguments:
> https://nightlies.apache.org/flink/flink-docs-master/docs/ops/debugging/debugging_classloading/
>
>
>
> On Wed, Jan 26, 2022 at 4:13 AM Javier Vegas wrote:
>
>> I am porting a Scala service to Flink in order to make it more scalable
>> via running it in a cluster. All my Scala services extends a base Service
>> class that extends TwitterServer (
>> https://github.com/twitter/twitter-server/blob/develop/server/src/main/scala/com/twitter/server/TwitterServer.scala)
>> and that base class contains a lot of logic about resource initialization,
>> logging, stats and error handling, monitoring, etc that I want to keep
>> using in my class. I ported my logic to Flink sources and sinks, and
>> everything worked fine when I ran my class in single JVM mode either from
>> sbt or my IDE, Flink's jobmanager and taskmanagers start and run my app.
>> But when I try to run my application in cluster mode, when launching my
>> class with "./bin/standalone-job.sh start --job-classname" the
>> jobmanager runs into a "IllegalArgumentException: URI is not hierarchical"
>> error on initialization, apparently because TwitterServer is trying to load
>> something from the class path (see attached full log).
>>
>> Is there anything I can do to run a class that extends TwitterServer in a
>> Flink cluster? I have tried making my class not extend it and it worked
>> fine, but I really want to keep using all the common infraestructure logic
>> that I have in my base class that extends TwitterServer.
>>
>> Thanks!
>>
>
