Hi Arvid, Thanks, will set the scope to Provided and try.
Are there public examples in GitHub that demonstrate a sample app in Minikube? Sandeep > On 23-Mar-2021, at 3:17 PM, Arvid Heise <ar...@apache.org> wrote: > > Hi Sandeep, > > please have a look at [1], you should add most Flink dependencies as provided > - exceptions are connectors (or in general stuff that is not in flink/lib/ or > flink/plugins). > > [1] > https://ci.apache.org/projects/flink/flink-docs-stable/dev/project-configuration.html#setting-up-a-project-basic-dependencies > > <https://ci.apache.org/projects/flink/flink-docs-stable/dev/project-configuration.html#setting-up-a-project-basic-dependencies> > On Tue, Mar 23, 2021 at 5:28 AM Sandeep khanzode <sand...@shiftright.ai > <mailto:sand...@shiftright.ai>> wrote: > Hi Arvid, > > I copy the JAR to the usrlib folder. This works in the Cloud EKS cluster. I > wanted to set this up for my testing purposes. > > Below is the Dockerfile: > FROM apache/flink:1.12.1-java11 > RUN mv /opt/flink/opt/flink-queryable-state-runtime_2.12-1.12.1.jar > /opt/flink/lib/flink-queryable-state-runtime_2.12-1.12.1.jar > ADD myJar.jar /opt/flink/usrlib/myJar.jar > > … But, in my process, this is a Fat JAR created by the Maven Shade Plugin. > Are you saying that all Flink classes should not be part of the user JAR? How > does that work? Do we set the scope of the dependencies to compile (or, not > runtime) for Flink Jars? Do we have any samples/examples that shows this? > Would be really helpful. > > >> On 22-Mar-2021, at 8:00 PM, Arvid Heise <ar...@apache.org >> <mailto:ar...@apache.org>> wrote: >> >> Hi Sandeep, >> >> The first error definitively indicates a classloading issue, which may also >> be the cause for the second error. >> >> Can you describe where you put your jar inside the docker image and which >> execution mode you are using? As a general rule, the jar is not supposed to >> go into flink/lib. >> Also make sure to never shade non-connector classes of Flink into your jar. >> A typical user jar should be ~1MB. >> >> On Fri, Mar 19, 2021 at 8:58 PM Sandeep khanzode <sand...@shiftright.ai >> <mailto:sand...@shiftright.ai>> wrote: >> Hello, >> >> I have a fat JAR compiled using the Man Shade plugin and everything works >> correctly when I deploy it on a standalone local cluster i.e. one job and >> one task manager node. >> >> But I installed Minikube and the same JAR file packaged into a docker image >> fails with weird serialization errors: >> >> Caused by: java.lang.ClassCastException: cannot assign instance of >> java.lang.invoke.SerializedLambda to field >> org.apache.flink.streaming.runtime.partitioner.KeyGroupStreamPartitioner.keySelector >> of type org.apache.flink.api.java.functions.KeySelector in instance of >> org.apache.flink.streaming.runtime.partitioner.KeyGroupStreamPartitioner >> >> >> … or in certain cases, if I comment out everything except the Kafka Source, >> then ... >> Caused by: java.lang.NoClassDefFoundError: Could not initialize class >> org.apache.kafka.common.requests.MetadataRequest$Builder >> >> >> Is there anything I am missing with the Minikube setup? I initially tried >> with the steps for the Job Application cluster on the website, but I was >> unable to get the /usrlib mounted from the hostpath. >> >> >> So, I created a simple docker image from ... >> apache/flink:1.12.1-java11 >> >> But I have not had any success getting the same job to run here. Please let >> me know if there are well-known steps or issues that I can check. >> >> Thanks, >> Sandeep >
