hi Martijn: Some platform users may not package all the jars to the fat jars, spark also has - jars for dependencies https://stackoverflow.com/questions/29099115/spark-submit-add-multiple-jars-in-classpath
On 2022/10/27 06:48:52 Martijn Visser wrote: > Hi Jacky Lau, > > Since you've sent the email to multiple mailing lists, I've decided to > reply to the one that you've sent to both the Dev and User ML. > > > but it is not possible for platform users to create fat jars to package > all their dependencies into the final jar package > > Can you elaborate on why that's not possible? > > Best regards, > > Martijn > > On Thu, Oct 27, 2022 at 6:59 AM Jacky Lau <li...@gmail.com> wrote: > > > Hi guys: > > > > I'd like to initiate a discussion about adding command-line arguments to > > support user-dependent jar packages. > > > > Currently flink supports user's main jars through -jarfile or without > > setting this , the flink client will treat the first argument after that as > > the user master jar package when it encounters the first command line > > argument that cannot be parsed. but it is not possible for platform users > > to create fat jars to package all their dependencies into the final jar > > package. In the meantime, the configuration pipeline.jars is currently > > exposed, and this value is overridden by command-line arguments such as > > -jarfile. > > > > And If the user is using both the command-line argument and the > > pipeline.jars argument, which can make the user werild and confused. In > > addition, we should specify the priority "command line parameter > -D > > dynamic parameter > flink-conf.yml configuration file parameter" in docs > > >
