Hi, Is it possible to submit Scala code without uploading a JAR? I have tried the following:
object LivySubmission { def main(args: Array[String]): Unit = { val client = new LivyClientBuilder(false) .setURI(new URI("http://localhost:8998/")) .build() .asScalaClient val job = client.run { context => context.createStreamingContext(15000) 1 + 1 } job.onSuccess { case r => Console.println("1 + 1 = " + r) } } } Result is: "log": [ "Caused by: java.lang.ClassNotFoundException: org.apache.livy.scalaapi.LivyScalaClient$$anon$2", "\tat java.net.URLClassLoader.findClass(URLClassLoader.java:382)", "\tat java.lang.ClassLoader.loadClass(ClassLoader.java:418)", "\tat java.lang.ClassLoader.loadClass(ClassLoader.java:351)", "\tat java.lang.Class.forName0(Native Method)", "\tat java.lang.Class.forName(Class.java:348)", "\tat org.apache.livy.shaded.kryo.kryo.util.DefaultClassResolver.readName(DefaultClassResolver.java:136)", "\t... 40 more", "21/03/22 23:21:30 INFO ExecutorAllocationManager: Request to remove executorIds: 1", "\nYARN Diagnostics: " ] It looks like it's trying to load the anonymous function submitted to client.run and can't find it, is that correct? Regards, Patrick