Hi Javier, I suspect that TwitterServer is using some classloading / dependency injection / service loading "magic" that is causing this. I would try to find out, either by attaching a remote debugger (should be possible when executing in cluster mode locally) or by adding log statements in the code, what the URI it's trying to load looks like.
On the cluster, Flink is using separate classloaders for the base flink system, and the user code (as opposed to executing in the IDE, where everything is loaded from the same loader). Check out this page and try out the config arguments: https://nightlies.apache.org/flink/flink-docs-master/docs/ops/debugging/debugging_classloading/ On Wed, Jan 26, 2022 at 4:13 AM Javier Vegas <jve...@strava.com> wrote: > I am porting a Scala service to Flink in order to make it more scalable > via running it in a cluster. All my Scala services extends a base Service > class that extends TwitterServer ( > https://github.com/twitter/twitter-server/blob/develop/server/src/main/scala/com/twitter/server/TwitterServer.scala) > and that base class contains a lot of logic about resource initialization, > logging, stats and error handling, monitoring, etc that I want to keep > using in my class. I ported my logic to Flink sources and sinks, and > everything worked fine when I ran my class in single JVM mode either from > sbt or my IDE, Flink's jobmanager and taskmanagers start and run my app. > But when I try to run my application in cluster mode, when launching my > class with "./bin/standalone-job.sh start --job-classname" the jobmanager > runs into a "IllegalArgumentException: URI is not hierarchical" error on > initialization, apparently because TwitterServer is trying to load > something from the class path (see attached full log). > > Is there anything I can do to run a class that extends TwitterServer in a > Flink cluster? I have tried making my class not extend it and it worked > fine, but I really want to keep using all the common infraestructure logic > that I have in my base class that extends TwitterServer. > > Thanks! >
