Hi, I have set up Zeppelin to use docker to run interpreters using <property> <name>zeppelin.run.mode</name> <value>docker</value> <description>'auto|local|k8s|docker'</description> </property>
<property> <name>zeppelin.docker.container.image</name> <value> apache/zeppelin:0.10.0 </value> <description>Docker image for interpreters</description> </property> This works so far, as interpreter images are being launched and used, however I run into an issue with the Spark Interpreter. In the 0.10.0 docker image, the DockerInterpreter tries to delete all interpreters, that are not relevant for Spark. There I do get a permission denied message. The Dockerfile uses USER 1000 and the Zeppelin files in the image are owned by root, thus a delete fails. This happens in org/apache/zeppelin/interpreter/launcher/DockerInterpreterProcess.java private void rmInContainer(String containerId, String path) Please advice. Thanks. Cheers Brian Log snippet: [...] rm: cannot remove '/opt/zeppelin/interpreter/angular/zeppelin-angular-0.10.0.jar' INFO [2021-11-02 09:29:29,030] ({SchedulerFactory3} DockerInterpreterProcess.java[execInContainer]:591) - : Permission denied INFO [2021-11-02 09:29:29,030] ({SchedulerFactory3} DockerInterpreterProcess.java[execInContainer]:591) - rm: cannot remove '/opt/zeppelin/interpreter/angular/META-INF/LICENSE': Permission denied rm: cannot remove '/opt/zeppelin/interpreter/angular/META-INF/DEPENDENCIES': Permission denied INFO [2021-11-02 09:29:29,030] ({SchedulerFactory3} DockerInterpreterProcess.java[execInContainer]:591) - rm: cannot remove '/opt/zeppelin/interpreter/angular/META-INF/NOTICE': Permission denied INFO [2021-11-02 09:29:29,030] ({SchedulerFactory3} DockerInterpreterProcess.java[execInContainer]:591) - rm: cannot remove '/opt/zeppelin/interpreter/flink-cmd/interpreter-setting.json': Permissi on denied rm: cannot remove '/opt/zeppelin/interpreter/flink-cmd/zeppelin-flink-cmd-0.10.0.jar' INFO [2021-11-02 09:29:29,030] ({SchedulerFactory3} DockerInterpreterProcess.java[execInContainer]:591) - : Permission denied INFO [2021-11-02 09:29:29,030] ({SchedulerFactory3} DockerInterpreterProcess.java[execInContainer]:591) - rm: cannot remove '/opt/zeppelin/interpreter/flink-cmd/META-INF/LICENSE': Permission denied rm: cannot remove '/opt/zeppelin/interpreter/flink-cmd/META-INF/DEPENDENCIES': Permission denied INFO [2021-11-02 09:29:29,030] ({SchedulerFactory3} DockerInterpreterProcess.java[execInContainer]:591) - rm: cannot remove '/opt/zeppelin/interpreter/flink-cmd/META-INF/NOTICE': Permission denied INFO [2021-11-02 09:29:29,031] ({SchedulerFactory3} DockerInterpreterProcess.java[execInContainer]:581) - exec container commmand: mkdir /opt/zeppelin -p INFO [2021-11-02 09:29:29,082] ({SchedulerFactory3} DockerInterpreterProcess.java[execInContainer]:581) - exec container commmand: mkdir /opt/zeppelin/conf -p WARN [2021-11-02 09:29:29,114] ({SchedulerFactory3} DockerInterpreterProcess.java[copyRunFileToContainer]:452) - /etc/krb5.conf file not found, Did not upload the krb5.conf to the container! INFO [2021-11-02 09:29:29,114] ({SchedulerFactory3} DockerInterpreterProcess.java[execInContainer]:581) - exec container commmand: mkdir /opt/zeppelin/bin -p INFO [2021-11-02 09:29:29,177] ({SchedulerFactory3} DockerInterpreterProcess.java[execInContainer]:581) - exec container commmand: mkdir /opt/zeppelin/interpreter/spark -p INFO [2021-11-02 09:29:40,185] ({SchedulerFactory3} DockerInterpreterProcess.java[execInContainer]:581) - exec container commmand: mkdir /tmp/zeppelin-tar -p WARN [2021-11-02 09:29:40,404] ({SchedulerFactory3} NotebookServer.java[onStatusChange]:1986) - Job 20180530-101750_1491737301 is finished, status: ERROR, exception: null, result: %text java.lang.IllegalStateException: Container 8d64e1b1c58327771aa08678f555f95b3d0bbca37cb7b7662220594a4f39f2da is not running. at com.spotify.docker.client.DefaultDockerClient.execCreate(DefaultDockerClient .java:1650) at org.apache.zeppelin.interpreter.launcher.DockerInterpreterProcess.execInCont ainer(DockerInterpreterProcess.java:584) at org.apache.zeppelin.interpreter.launcher.DockerInterpreterProcess.mkdirInCon tainer(DockerInterpreterProcess.java:563) at org.apache.zeppelin.interpreter.launcher.DockerInterpreterProcess.deployToCo ntainer(DockerInterpreterProcess.java:539) at org.apache.zeppelin.interpreter.launcher.DockerInterpreterProcess.copyRunFil eToContainer(DockerInterpreterProcess.java:533) at org.apache.zeppelin.interpreter.launcher.DockerInterpreterProcess.start(Dock erInterpreterProcess.java:237) at org.apache.zeppelin.interpreter.ManagedInterpreterGroup.getOrCreateInterpret erProcess(ManagedInterpreterGroup.java:68) at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getOrCreateInterpre terProcess(RemoteInterpreter.java:104) at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.internal_create(Rem oteInterpreter.java:154) at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.open(RemoteInterpre ter.java:126) at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getFormType(RemoteI nterpreter.java:271) at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:440) at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:71) at org.apache.zeppelin.scheduler.Job.run(Job.java:172) at org.apache.zeppelin.scheduler.AbstractScheduler.runJob(AbstractScheduler.jav a:132) at org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(RemoteScheduler. java:182) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java :515) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTa sk.run(ScheduledThreadPoolExecutor.java:304) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecut or.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecu tor.java:628) at java.base/java.lang.Thread.run(Thread.java:829)