Re: How to disable creating a new notebook for few users
Am Fr., 2. Juli 2021 um 01:23 Uhr schrieb Jeff Zhang : > I don't think zeppelin has such a feature yet. > > Great Info 于2021年7月2日周五 上午2:42写道： > >> I am using Zeppelin 0.8.2, different kinds of users use our zeppelin. >> There is one category of users wherein they just need to run the notebook. >> I know how to set the authorization to disable 'edit or running' the >> notebook but I did not find the option to disable creating a new notebook >> for these users. >> > Fyi, i already created a ticket as feature request for this https://issues.apache.org/jira/browse/ZEPPELIN-4716 > >> Thanks, >> Cod >> > > > -- > Best Regards > > Jeff Zhang >
Re: Question: Adding Dependencies in with the Spark Interpreter with Kubernetes
Am Mi., 13. Mai 2020 um 21:59 Uhr schrieb Hetul Patel : > > Are dependency downloads supported with zeppelin and spark over > kubernetes? Or am I required to add the dependency jars directly to my > spark docker image and add them to the classpath? > > Hi Hetu, i don't use docker but to connect to my cassandra from the spark cluster i have to set SPARK_SUBMIT_OPTIONS='--packages com.datastax.spark:spark-cassandra-connector_2.11:2.4.3' HTH, Sebastian. > Thanks, > Hetu >
Notebook permissions
Hi, i want logged in users allow to only view and use existing notebooks, but not to create new ones. Is that possible somehow? Thx+BR, Sebastian.
Re: Spark performance after update to 0.8.2
Hi Jeff,
Am Di., 24. März 2020 um 09:39 Uhr schrieb Jeff Zhang :
> Do you use the same version of spark for zeppelin 0.7 & 0.8 ? What kind of
> job do you run ? Is it easy to reproduce it ?
>
Zeppelin comes with spark 2.1.0 in 0.7.3 and spark 2.2.1 in 0.8.2, so
different version.
In this case it is a 'cache table ...' statement in the %spark.sql
interpreter and i see the larger delay in task executions everytime i
execute it with the different zeppelin versions.
BR,
Sebastian.
>
> Sebastian Albrecht 于2020年3月24日周二
> 下午4:24写道：
>
>> Hi,
>> i just upgraded from zeppelin 0.7.3 to 0.8.2. I am using zeppelin for
>> spark in local mode and 0.8.2 came with spark updated to 2.2.1. I now
>> experience slower spark job executions. I can show it in these log
>> snippets: task executions with 0.7.3: milliseconds, 0.8.2: seconds
>>
>> 0.7.3
>> INFO [2020-03-12 00:01:22,105] ({Executor task launch worker-2}
>> Logging.scala[logInfo]:54) - Code generated in 24.742647 ms
>> INFO [2020-03-12 00:01:22,139] ({Executor task launch worker-2}
>> Logging.scala[logInfo]:54) - Code generated in 23.640976 ms
>> INFO [2020-03-12 00:01:22,174] ({Executor task launch worker-2}
>> Logging.scala[logInfo]:54) - Code generated in 11.667064 ms
>> INFO [2020-03-12 00:01:22,193] ({Executor task launch worker-0}
>> Logging.scala[logInfo]:54) - Code generated in 14.959374 ms
>> INFO [2020-03-12 00:01:22,872] ({Executor task launch worker-6}
>> Logging.scala[logInfo]:54) - Finished task 6.0 in stage 8.0 (TID 14). 2593
>> bytes result sent to driver
>> INFO [2020-03-12 00:01:22,873] ({dispatcher-event-loop-2}
>> Logging.scala[logInfo]:54) - Starting task 8.0 in stage 8.0 (TID 16,
>> localhost, executor driver, partition 8, NODE_LOCAL, 9086 bytes)
>> INFO [2020-03-12 00:01:22,875] ({Executor task launch worker-6}
>> Logging.scala[logInfo]:54) - Running task 8.0 in stage 8.0 (TID 16)
>> INFO [2020-03-12 00:01:22,876] ({task-result-getter-0}
>> Logging.scala[logInfo]:54) - Finished task 6.0 in stage 8.0 (TID 14) in
>> 2061 ms on localhost (executor driver) (1/97)
>> INFO [2020-03-12 00:01:22,938] ({Executor task launch worker-2}
>> Logging.scala[logInfo]:54) - Finished task 2.0 in stage 8.0 (TID 10). 2593
>> bytes result sent to driver
>> INFO [2020-03-12 00:01:22,939] ({dispatcher-event-loop-7}
>> Logging.scala[logInfo]:54) - Starting task 9.0 in stage 8.0 (TID 17,
>> localhost, executor driver, partition 9, NODE_LOCAL, 8838 bytes)
>> INFO [2020-03-12 00:01:22,946] ({task-result-getter-1}
>> Logging.scala[logInfo]:54) - Finished task 2.0 in stage 8.0 (TID 10) in
>> 2135 ms on localhost (executor driver) (2/97)
>> INFO [2020-03-12 00:01:22,962] ({Executor task launch worker-8}
>> Logging.scala[logInfo]:54) - Running task 9.0 in stage 8.0 (TID 17)
>> INFO [2020-03-12 00:01:22,983] ({Executor task launch worker-3}
>> Logging.scala[logInfo]:54) - Finished task 3.0 in stage 8.0 (TID 11). 2506
>> bytes result sent to driver
>> INFO [2020-03-12 00:01:22,984] ({dispatcher-event-loop-3}
>> Logging.scala[logInfo]:54) - Starting task 10.0 in stage 8.0 (TID 18,
>> localhost, executor driver, partition 10, NODE_LOCAL, 8839 bytes)
>> INFO [2020-03-12 00:01:22,985] ({task-result-getter-2}
>> Logging.scala[logInfo]:54) - Finished task 3.0 in stage 8.0 (TID 11) in
>> 2173 ms on localhost (executor driver) (3/97)
>> INFO [2020-03-12 00:01:22,986] ({Executor task launch worker-2}
>> Logging.scala[logInfo]:54) - Running task 10.0 in stage 8.0 (TID 18)
>> INFO [2020-03-12 00:01:23,068] ({Executor task launch worker-0}
>> Logging.scala[logInfo]:54) - Finished task 0.0 in stage 8.0 (TID 8). 2506
>> bytes result sent to driver
>> INFO [2020-03-12 00:01:23,069] ({dispatcher-event-loop-6}
>> Logging.scala[logInfo]:54) - Starting task 11.0 in stage 8.0 (TID 19,
>> localhost, executor driver, partition 11, NODE_LOCAL, 9333 bytes)
>> INFO [2020-03-12 00:01:23,070] ({Executor task launch worker-3}
>> Logging.scala[logInfo]:54) - Running task 11.0 in stage 8.0 (TID 19)
>> INFO [2020-03-12 00:01:23,070] ({task-result-getter-3}
>> Logging.scala[logInfo]:54) - Finished task 0.0 in stage 8.0 (TID 8) in 2262
>> ms on localhost (executor driver) (4/97)
>> INFO [2020-03-12 00:01:23,088] ({Executor task launch worker-7}
>> Logging.scala[logInfo]:54) - Finished task 7.0 in stage 8.0 (TID 15). 2506
>> bytes result sent to driver
>> INFO [2020-03-12 00:01:23,089] ({dispatcher-event-loop-2}
>> Logging.scala[logInfo]:54) - Starting task 12.0 in stage 8.0 (TID 20,
>> localhost, executor driver, partition 12, NODE_LOCAL, 8591 bytes)
>> INFO [2020-03-1
Spark performance after update to 0.8.2
Hi,
i just upgraded from zeppelin 0.7.3 to 0.8.2. I am using zeppelin for spark
in local mode and 0.8.2 came with spark updated to 2.2.1. I now experience
slower spark job executions. I can show it in these log snippets: task
executions with 0.7.3: milliseconds, 0.8.2: seconds
0.7.3
INFO [2020-03-12 00:01:22,105] ({Executor task launch worker-2}
Logging.scala[logInfo]:54) - Code generated in 24.742647 ms
INFO [2020-03-12 00:01:22,139] ({Executor task launch worker-2}
Logging.scala[logInfo]:54) - Code generated in 23.640976 ms
INFO [2020-03-12 00:01:22,174] ({Executor task launch worker-2}
Logging.scala[logInfo]:54) - Code generated in 11.667064 ms
INFO [2020-03-12 00:01:22,193] ({Executor task launch worker-0}
Logging.scala[logInfo]:54) - Code generated in 14.959374 ms
INFO [2020-03-12 00:01:22,872] ({Executor task launch worker-6}
Logging.scala[logInfo]:54) - Finished task 6.0 in stage 8.0 (TID 14). 2593
bytes result sent to driver
INFO [2020-03-12 00:01:22,873] ({dispatcher-event-loop-2}
Logging.scala[logInfo]:54) - Starting task 8.0 in stage 8.0 (TID 16,
localhost, executor driver, partition 8, NODE_LOCAL, 9086 bytes)
INFO [2020-03-12 00:01:22,875] ({Executor task launch worker-6}
Logging.scala[logInfo]:54) - Running task 8.0 in stage 8.0 (TID 16)
INFO [2020-03-12 00:01:22,876] ({task-result-getter-0}
Logging.scala[logInfo]:54) - Finished task 6.0 in stage 8.0 (TID 14) in
2061 ms on localhost (executor driver) (1/97)
INFO [2020-03-12 00:01:22,938] ({Executor task launch worker-2}
Logging.scala[logInfo]:54) - Finished task 2.0 in stage 8.0 (TID 10). 2593
bytes result sent to driver
INFO [2020-03-12 00:01:22,939] ({dispatcher-event-loop-7}
Logging.scala[logInfo]:54) - Starting task 9.0 in stage 8.0 (TID 17,
localhost, executor driver, partition 9, NODE_LOCAL, 8838 bytes)
INFO [2020-03-12 00:01:22,946] ({task-result-getter-1}
Logging.scala[logInfo]:54) - Finished task 2.0 in stage 8.0 (TID 10) in
2135 ms on localhost (executor driver) (2/97)
INFO [2020-03-12 00:01:22,962] ({Executor task launch worker-8}
Logging.scala[logInfo]:54) - Running task 9.0 in stage 8.0 (TID 17)
INFO [2020-03-12 00:01:22,983] ({Executor task launch worker-3}
Logging.scala[logInfo]:54) - Finished task 3.0 in stage 8.0 (TID 11). 2506
bytes result sent to driver
INFO [2020-03-12 00:01:22,984] ({dispatcher-event-loop-3}
Logging.scala[logInfo]:54) - Starting task 10.0 in stage 8.0 (TID 18,
localhost, executor driver, partition 10, NODE_LOCAL, 8839 bytes)
INFO [2020-03-12 00:01:22,985] ({task-result-getter-2}
Logging.scala[logInfo]:54) - Finished task 3.0 in stage 8.0 (TID 11) in
2173 ms on localhost (executor driver) (3/97)
INFO [2020-03-12 00:01:22,986] ({Executor task launch worker-2}
Logging.scala[logInfo]:54) - Running task 10.0 in stage 8.0 (TID 18)
INFO [2020-03-12 00:01:23,068] ({Executor task launch worker-0}
Logging.scala[logInfo]:54) - Finished task 0.0 in stage 8.0 (TID 8). 2506
bytes result sent to driver
INFO [2020-03-12 00:01:23,069] ({dispatcher-event-loop-6}
Logging.scala[logInfo]:54) - Starting task 11.0 in stage 8.0 (TID 19,
localhost, executor driver, partition 11, NODE_LOCAL, 9333 bytes)
INFO [2020-03-12 00:01:23,070] ({Executor task launch worker-3}
Logging.scala[logInfo]:54) - Running task 11.0 in stage 8.0 (TID 19)
INFO [2020-03-12 00:01:23,070] ({task-result-getter-3}
Logging.scala[logInfo]:54) - Finished task 0.0 in stage 8.0 (TID 8) in 2262
ms on localhost (executor driver) (4/97)
INFO [2020-03-12 00:01:23,088] ({Executor task launch worker-7}
Logging.scala[logInfo]:54) - Finished task 7.0 in stage 8.0 (TID 15). 2506
bytes result sent to driver
INFO [2020-03-12 00:01:23,089] ({dispatcher-event-loop-2}
Logging.scala[logInfo]:54) - Starting task 12.0 in stage 8.0 (TID 20,
localhost, executor driver, partition 12, NODE_LOCAL, 8591 bytes)
INFO [2020-03-12 00:01:23,090] ({task-result-getter-0}
Logging.scala[logInfo]:54) - Finished task 7.0 in stage 8.0 (TID 15) in
2275 ms on localhost (executor driver) (5/97)
INFO [2020-03-12 00:01:23,090] ({Executor task launch worker-7}
Logging.scala[logInfo]:54) - Running task 12.0 in stage 8.0 (TID 20)
INFO [2020-03-12 00:01:23,266] ({Executor task launch worker-1}
Logging.scala[logInfo]:54) - Finished task 1.0 in stage 8.0 (TID 9). 2506
bytes result sent to driver
INFO [2020-03-12 00:01:23,267] ({dispatcher-event-loop-7}
Logging.scala[logInfo]:54) - Starting task 13.0 in stage 8.0 (TID 21,
localhost, executor driver, partition 13, NODE_LOCAL, 9458 bytes)
INFO [2020-03-12 00:01:23,268] ({task-result-getter-1}
Logging.scala[logInfo]:54) - Finished task 1.0 in stage 8.0 (TID 9) in 2457
ms on localhost (executor driver) (6/97)
INFO [2020-03-12 00:01:23,269] ({Executor task launch worker-1}
Logging.scala[logInfo]:54) - Running task 13.0 in stage 8.0 (TID 21)
INFO [2020-03-12 00:01:23,278] ({Executor task launch worker-4}
Logging.scala[logInfo]:54) - Finished task 4.0 in stage 8.0 (TID 12). 2506
bytes result sent to driver
INFO [2020-03-12
Re: use more than one host with spark cassandra connector
Hi Jeff, thank you. Is it possible to change the spark property in a %spark paragraph which was set interpreter UI or has the interpreter to be restarted for that? BR, Sebastian. Am Sa., 7. Sept. 2019 um 04:36 Uhr schrieb Jeff Zhang : > Sebastian, > > I don't think spark interpreter of zeppelin does anything for cassandra. > IIUC spark.cassandra.connection.host is spark-cassandra connector > property, as long as spark support it, it should work in zeppelin. So I > would suggest to you to figure out how to set more than one host in spark > cassandra connector first. > > Sebastian Albrecht 于2019年9月7日周六 > 上午12:25写道： > >> >> Am Fr., 6. Sept. 2019 um 17:04 Uhr schrieb Jeff Zhang : >> >>> It is better to ask this kind of question in spark community. As long as >>> spark support it, you can do it in zeppelin. >>> >>> >> Hi Jeff, >> thank you for your answer. Zeppelin already does is in the spark >> interpreter: set the property spark.cassandra.connection.host , add the >> cassandra-connector to the dependencies and go. But where do i configure an >> additional host? In the same interpreter. Do i have to add a separate spark >> interpreter? >> >> Thank you, >> Sebastian. >> >> >>> Sebastian Albrecht 于2019年9月6日周五 >>> 下午9:27写道： >>> >>>> Hi, >>>> is there a way to connect a zeppelin instance to more than one >>>> cassandra hosts via the spark cassandra connector to use the data from them >>>> in spark? >>>> >>>> BR, >>>> Sebastian. >>>> >>> >>> >>> -- >>> Best Regards >>> >>> Jeff Zhang >>> >> > > -- > Best Regards > > Jeff Zhang >
Re: use more than one host with spark cassandra connector
Am Fr., 6. Sept. 2019 um 17:04 Uhr schrieb Jeff Zhang : > It is better to ask this kind of question in spark community. As long as > spark support it, you can do it in zeppelin. > > Hi Jeff, thank you for your answer. Zeppelin already does is in the spark interpreter: set the property spark.cassandra.connection.host , add the cassandra-connector to the dependencies and go. But where do i configure an additional host? In the same interpreter. Do i have to add a separate spark interpreter? Thank you, Sebastian. > Sebastian Albrecht 于2019年9月6日周五 > 下午9:27写道： > >> Hi, >> is there a way to connect a zeppelin instance to more than one cassandra >> hosts via the spark cassandra connector to use the data from them in spark? >> >> BR, >> Sebastian. >> > > > -- > Best Regards > > Jeff Zhang >
use more than one host with spark cassandra connector
Hi, is there a way to connect a zeppelin instance to more than one cassandra hosts via the spark cassandra connector to use the data from them in spark? BR, Sebastian.
