Re: Compiling Spark UDF at runtime

Thanks! yes, this would be an option of course. HDFS or Alluxio. Sincerely, Michael Shtelma On Fri, Jan 12, 2018 at 3:26 PM, Georg Heiler wrote: > You could store the jar in hdfs. Then even in yarn cluster mode your give > workaround should work. > Michael Shtelma

Re: Compiling Spark UDF at runtime

You could store the jar in hdfs. Then even in yarn cluster mode your give workaround should work. Michael Shtelma schrieb am Fr. 12. Jan. 2018 um 12:58: > Hi all, > > I would like to be able to compile Spark UDF at runtime. Right now I > am using Janino for that. > My problem

Compiling Spark UDF at runtime

Hi all, I would like to be able to compile Spark UDF at runtime. Right now I am using Janino for that. My problem is, that in order to make my compiled functions visible to spark, I have to set janino classloader (janino gives me classloader with compiled UDF classes) as context class loader