Re: Spark interpreter Repl injection

Thanks I created the issue Regards Carlos On Tue 9. Mar 2021 at 19:02, moon soo Lee wrote: > Pyspark interpreter have 'intp' variable exposed in its repl environment > (for internal use). And we can resolve reference to Spark interpreter from > the 'intp' variable. However, scala repl environmen

Re: Spark interpreter Repl injection

Pyspark interpreter have 'intp' variable exposed in its repl environment (for internal use). And we can resolve reference to Spark interpreter from the 'intp' variable. However, scala repl environment in Spark Interpreter doesn't expose any variables that is useful for finding Spark Interpreter its

Re: Spark interpreter Repl injection

Looks good Moon Is there a specific reason why you needed the pyspark interpreter to access the spark interpreter? Could not the spark interpreter programmatically access itself (and the same for the pyspark interpreter) Would the issue be to expose the z.interpret() method? Best regards Carlos

Re: Spark interpreter Repl injection

I see. If you want to specify a file, precode might not the best option. I found a hacky way to do it. Accessing SparkInterpreter instance object from PysparkInterpreter. %pyspark sparkIntpField = intp.getClass().getDeclaredField("sparkInterpreter") sparkIntpField.setAccessible(True) sparkIntp = s

Re: Spark interpreter Repl injection

Are you able to specify a file on the precode? For now my work around is from within the note and with the rest api , to add a paragraph with the code I want to inject ( which can come from a file ) It works ok , but with run all or schedule the code gets updated in the note , but the old Code stil

Re: Spark interpreter Repl injection

Hi, How about precode ? "zeppelin.SparkInterpreter.precode" can run scala code. Thanks, moon On Sat, Mar 6, 2021 at 4:51 AM Carlos Diogo wrote: > That does not work if you want to have Scala code in a file ( common

Re: Spark interpreter Repl injection

That does not work if you want to have Scala code in a file ( common functions) which you want to invoke in the note The alternative is to compile the code and then add the jar which would be normal for an application. But zeppelin is about scripting so this is a request I get very often from the u

Re: Spark interpreter Repl injection

Why not copying scala code in zeppelin and run the notebook directly ? Carlos Diogo 于2021年3月6日周六 下午3:51写道： > Dear all > I have been trying to find a was to inject scala Code ( from String) > into the spark interpreter > In pyspark is easy with the exec function > It should not be very difficul