Re: How to config the flink to load libs in myself path
For example, now I had my custom table source or sink which were builed in a independent jar , and my main code will depend on it. But I don't want to package custom connector jar with main code in a jar flie. In other words, I want to get a thin jar not a fat jar. So I think I can put the custom connector jar in *flink/lib*. before I run my job. In fact, it really work.My jobmanager yaml like as below:-containers:... volumeMounts:- mountPath: /opt/flink/conf name: flink-config-volume- mountPath: /opt/flink/lib name: volume-1618910657181- mountPath: /opt/flink/flink-uploadjar name: volume-1618911748381 - mountPath: /opt/flink/plugins/oss-fs-hadoop/flink-oss-fs-hadoop-1.12.2.jar name: volume-1618916463815 volumes:- configMap: defaultMode: 420items: - key: flink-conf.yaml path: flink-conf.yaml - key: log4j-console.properties path: log4j-console.propertiesname: flink-config name: flink-config-volume- hostPath:path: /data/volumes/flink/volume-for-session/cxylib-common-jartype: '' name: volume-1618910657181- hostPath:path: /home/uploadjartype: '' name: volume-1618911748381 - hostPath:path: /data/volumes/flink/volume-for-session/plugins/oss-fs-hadoop/flink-oss-fs-hadoop-1.12.2.jar type: '' name: volume-1618916463815-As the yaml, I have to mount Host Machine path to container path.Now I deploy flink in k8s cluster which has three nodes, so I have to put my all jar in three nodes.And then If I change some codes, I also have to package and put then in three nodes.So if flink support to config the flink to load lib in myself path, I can use aliyun oss pv and pvc to mount oss path directly. Like my other yaml as below:-containers: ... volumeMounts: - mountPath: /data name: volume-trino-volume ... volumes:- name: volume-trino-volume persistentVolumeClaim: claimName: trino-volume ... -So if flink support to config like "flink.lib.path : /data/myself/lib", it will very convenient.I don't know if you know what I mean. -- Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
Re: How to config the flink to load libs in myself path
Hi, chenxuying There is currently no official support for this. What I am curious about is why you have this requirement. In theory, you can always build your own image. Best, Guowei On Mon, Apr 19, 2021 at 9:58 PM chenxuying wrote: > Hi all, I deployed the flink in K8S by session cluster [1] > the default plugin path is /opt/flink/plugins, > the default lib path is /opt/flink/lib, > the default usrlib path is /opt/flink/usrlib, > I wonder if it is possible for change the default path. > For example, I wish flink don't load libs from /opt/flink/lib , and my want > it to load libs files from /data/flink/lib. and I can't move > /data/flink/lib to /opt/flink/lib > So how to config the flink to load lib in myself path > > [1]: > https://ci.apache.org/projects/flink/flink-docs-release-1.12/deployment/resource-providers/standalone/kubernetes.html#session-cluster-resource-definitions > > > > >
How to config the flink to load libs in myself path
Hi all, I deployed the flink in K8S by session cluster [1] the default plugin path is /opt/flink/plugins, the default lib path is /opt/flink/lib, the default usrlib path is /opt/flink/usrlib, I wonder if it is possible for change the default path. For example, I wish flink don't load libs from /opt/flink/lib , and my want it to load libs files from /data/flink/lib. and I can't move /data/flink/lib to /opt/flink/lib So how to config the flink to load lib in myself path [1]: https://ci.apache.org/projects/flink/flink-docs-release-1.12/deployment/resource-providers/standalone/kubernetes.html#session-cluster-resource-definitions
