Hello guys..i know its irrelevant to this topic but i've been looking desperately for the solution. I am facing en exception http://apache-spark-user-list.1001560.n3.nabble.com/how-to-resolve-you-must-build-spark-with-hive-exception-td27390.html
plz help me.. I couldn't find any solution.. On Fri, Jul 22, 2016 at 10:07 PM, Andy Davidson < a...@santacruzintegration.com> wrote: > Hi Pseudo > > I do not know much about zeppelin . What languages are you using? > > I have been doing my data exploration and graphing using python mostly > because early on spark had good support for python. Its easy to collect() > data as a local PANDAS object. I think at this point R should work well. > You should be able to easily collect() your data as a R dataframe. I have > not tried to Rstudio. > > I typically run the Jupiter notebook server in my data center. I find the > notebooks really nice. I typically use matplotlib to generates my graph. > There are a lot of graphing packages. > > *Attached is the script I use to start the notebook server*. This script > and process works but is a little hacky You call it as follows > > > # > # on a machine in your cluster > # > $ cd dirWithNotebooks > > # all the logs will be in startIPythonNotebook.sh.out > # nohup allows you to log in start your notebook server and log out. > $ nohup startIPythonNotebook.sh > startIPythonNotebook.sh.out & > > # > # on you local machine > # > > # because of firewalls I need to open an ssh tunnel > $ ssh -o ServerAliveInterval=120 -N -f -L localhost:8889:localhost:7000 > myCluster > > # connect to the notebook server using the browser of you choice > > http://localhost:8889 > > > > # > # If you need to stop your notebooks server you may need to kill the server > # there is probably a cleaner way to do this > # $ ps -el | head -1; ps -efl | grep python > # > > <http://jupyter.org/>http://jupyter.org/ > > > P.S. Jupiter is in the process of being released. The new Juypter lab > alpha was just announced it looks really sweet. > > > > From: pseudo oduesp <pseudo20...@gmail.com> > Date: Friday, July 22, 2016 at 2:08 AM > To: Andrew Davidson <a...@santacruzintegration.com> > Subject: Re: spark and plot data > > HI andy , > thanks for reply , > i tell it just hard to each time switch between local concept and > destributed concept , for example zepplin give easy way to interact with > data ok , but it's hard to configure on huge cluster with lot of node in my > case i have cluster with 69 nodes and i process huge volume of data with > pyspark and it cool but when i want to plot some chart i get hard job to > make it . > > i sampling my result or aggregate , take for example if i user > randomforest algorithme in machine learning i want to retrive most > importante features with my version alerady installed in our cluster > (1.5.0) i can't get this. > > do you have any solution. > > Thanks > > 2016-07-21 18:44 GMT+02:00 Andy Davidson <a...@santacruzintegration.com>: > >> Hi Pseudo >> >> Plotting, graphing, data visualization, report generation are common >> needs in scientific and enterprise computing. >> >> Can you tell me more about your use case? What is it about the current >> process / workflow do you think could be improved by pushing plotting (I >> assume you mean plotting and graphing) into spark. >> >> >> In my personal work all the graphing is done in the driver on summary >> stats calculated using spark. So for me using standard python libs has not >> been a problem. >> >> Andy >> >> From: pseudo oduesp <pseudo20...@gmail.com> >> Date: Thursday, July 21, 2016 at 8:30 AM >> To: "user @spark" <user@spark.apache.org> >> Subject: spark and plot data >> >> Hi , >> i know spark it s engine to compute large data set but for me i work >> with pyspark and it s very wonderful machine >> >> my question we don't have tools for ploting data each time we have to >> switch and go back to python for using plot. >> but when you have large result scatter plot or roc curve you cant use >> collect to take data . >> >> somone have propostion for plot . >> >> thanks >> >> > > > --------------------------------------------------------------------- > To unsubscribe e-mail: user-unsubscr...@spark.apache.org >