Re: Table not found (python to sql)‏

Hi, Can you try without creating sqlContext manually? # sqlContext = SQLContext(sc) Thanks, moon On Tue, Aug 4, 2015 at 8:20 AM Braden Callahan1 wrote: > I have an issue where I create a table in Python (pyspark) and the sql > interpreter > does not see the table. If I create it using scala

Re: Yarn + Spark + Zepplin ?

That worked. Why ? Can you share a comprehensive iist of examples. On Mon, Aug 3, 2015 at 4:59 PM, Alex wrote: > Hi, > > inside %spark you do not need to create SqlContext manually: > as with "sc" for SparkContext, Interpreter already have injected "sqlc" > val. > > Also AFAIK println statemen

Re: Unsubscribe

YOU SHALL NOT PASS! On Aug 3, 2015 1:23 PM, "Aries Kay" wrote: > > > On Mon, Aug 3, 2015 at 1:00 PM, ÐΞ€ρ@Ҝ (๏̯͡๏) wrote: > >> Thanks a lot for all these documents. Appreciate your effort & time. >> >> On Mon, Aug 3, 2015 at 10:15 AM, Christian Tzolov >> wrote: >> >>> ÐΞ€ρ@Ҝ (๏̯͡๏), >>> >>> I'v

Re: Yarn + Spark + Zepplin ?

Hi, inside %spark you do not need to create SqlContext manually: as with "sc" for SparkContext, Interpreter already have injected "sqlc" val. Also AFAIK println statement should be in the separate paragraph. Can you try using that and see if it helps? -- Kind regards, Alexander > On 04 Aug 20

Re: Table not found (python to sql)‏

I think you would need to create a dataframe from this RDD with schema and then register as temp table to use with %sql. "Sent from my iPhone, plaese excuse any typos :)" > On Aug 3, 2015, at 7:19 PM, Braden Callahan1 wrote: > > I have an issue where I create a table in Python (pyspark) and

Re: Add jar file for spark interpreter use

Hi, first approach supposed to work, right now there is an open issue to track the progress of fixing it https://issues.apache.org/jira/browse/ZEPPELIN-194 -- Kind regards, Alexander > On 04 Aug 2015, at 06:38, Ming Han Teh wrote: > > Hi, > > I am trying to add a jar file which contains a

Table not found (python to sql)‏

I have an issue where I create a table in Python (pyspark) and the sql interpreter does not see the table. If I create it using scala both python and sql pick it up without an issue. Example: %pyspark sqlContext = SQLContext(sc) tweet = sqlContext.read.json("/temp/twitter_data_small/") tweet

Zeppelin examples

Could some one share list of comprehensive examples to test out various charts & features of Zeppelin with sample datasets that i can download along with queries. (I have spark + yarn + zeppelin, hence those examples are preferred) -- Deepak

Re: Add jar file for spark interpreter use

I am using Spark in standalone mode. I am not using YARN. Thanks. On Mon, Aug 3, 2015 at 2:43 PM, ÐΞ€ρ@Ҝ (๏̯͡๏) wrote: > How about setting spark.yarn.jar from Spark interpreter from UI > > spark.yarn.jar a.jar:b.jar:c.jar > > On Mon, Aug 3, 2015 at 2:38 PM, Ming Han Teh wrote: > >> Hi, >> >> I

Re: Add jar file for spark interpreter use

How about setting spark.yarn.jar from Spark interpreter from UI spark.yarn.jar a.jar:b.jar:c.jar On Mon, Aug 3, 2015 at 2:38 PM, Ming Han Teh wrote: > Hi, > > I am trying to add a jar file which contains a UDF to be used in the > zeppelin spark interpreter. > > Is there a difference between the

Add jar file for spark interpreter use

Hi, I am trying to add a jar file which contains a UDF to be used in the zeppelin spark interpreter. Is there a difference between the below two approaches: *Approach 1)* %dep z.load("somefile") *Approach 2)* sc.addJar("somejar") The first approach appears to load the jar file, but %sql fa

Re: Yarn + Spark + Zepplin ?

I am unable to see the visualization with Zeppelin from blog : http://hortonworks.com/blog/introduction-to-data-science-with-apache-spark/ Notebook %spark val sqlContext = new org.apache.spark.sql.SQLContext(sc) import sqlContext.implicits._ import java.sql.Date import org.apache.spark.sql.Row c

Re: Yarn + Spark + Zepplin ?

Fixed it mvn clean package -Pspark-1.3 -Dspark.version=1.3.1 -Dhadoop.version=2.7.0 -Phadoop-2.6 -Pyarn -DskipTests Earlier i had mvn clean install -DskipTests -Pspark-1.3 -Dspark.version=1.3.1 -Phadoop-2.7 -Pyarn On Mon, Aug 3, 2015 at 1:31 PM, ÐΞ€ρ@Ҝ (๏̯͡๏) wrote: > I have hadoop cluster u

Re: Yarn + Spark + Zepplin ?

I have hadoop cluster up using Ambari. It also allowed me to install Spark 1.3.1 and i can run sample spark application & Yarn application. So cluster is up and running fine. I got Zeppelin setup on a new box and was able to launch UI. I modified spark interpreter to set masteryarn-clientspark.a

Unsubscribe

On Mon, Aug 3, 2015 at 1:00 PM, ÐΞ€ρ@Ҝ (๏̯͡๏) wrote: > Thanks a lot for all these documents. Appreciate your effort & time. > > On Mon, Aug 3, 2015 at 10:15 AM, Christian Tzolov > wrote: > >> ÐΞ€ρ@Ҝ (๏̯͡๏), >> >> I've successfully run Zeppelin with Spark on YARN. I'm using Ambari and >> PivotalH

Re: Yarn + Spark + Zepplin ?

Thanks a lot for all these documents. Appreciate your effort & time. On Mon, Aug 3, 2015 at 10:15 AM, Christian Tzolov wrote: > ÐΞ€ρ@Ҝ (๏̯͡๏), > > I've successfully run Zeppelin with Spark on YARN. I'm using Ambari and > PivotalHD30. PHD30 is ODP compliant so you should be able to repeat the > c

Re: Yarn + Spark + Zepplin ?

ÐΞ€ρ@Ҝ (๏̯͡๏), I've successfully run Zeppelin with Spark on YARN. I'm using Ambari and PivotalHD30. PHD30 is ODP compliant so you should be able to repeat the configuration for HDP (e.g. hortonworks). 1. Before you start with Zeppelin, make sure that your Spark/YARN env. works from the command li

Why SparkEnv reference is maintained in SparkInterpreter

I saw that SparkInterpreter.java class maintains a ref to SparkEnv objects and sets it before interpreting any command in interpretInput function. Why is this required? Regards Rajat

Documented and Published- Install zeppelin on CDH

Hi All, Recently we have published a document on how to build zeppelin binaries with different profiles(pyspark , yarn) and setup zeppelin on CDH. Please check the document. https://software.intel.com/sites/default/files/managed/bb/bf/Apache-Zeppelin.pdf or Same our document re-published by clo

RE: Yarn + Spark + Zepplin ?

Hi Deepak, I have documented everything here. Please check published document. https://software.intel.com/sites/default/files/managed/bb/bf/Apache-Zeppelin.pdf Thanks Karthik Vadla From: ÐΞ€ρ@Ҝ (๏̯͡๏) [mailto:deepuj...@gmail.com] Sent: Sunday, August 2, 2015 9:25 PM To: users@zeppelin.incubator.

Re: Yarn + Spark + Zepplin ?

Hi Deepak, Hortonworks has its own howto post here: http://hortonworks.com/blog/introduction-to-data-science-with-apache-spark/ Hope it helps. Thanks, -Randy From: "ÐΞ€ρ@Ҝ (๏̯͡๏)" Reply-To: "users@zeppelin.incubator.apache.org" Date: Monday, August

Re: completion() method

Yes, please file an issue. Thanks, moon On Mon, Aug 3, 2015 at 7:19 PM Christian Tzolov wrote: > Hi Moon, > > Thanks for confirming the issue. It is broken and doesn't work with any > language interpreter Spark. > > ba30b09 >

Re: Yarn + Spark + Zepplin ?

I used Ambari to setup Hadoop cluster and they claim that their is pure open source version of Hadoop (2.7.x) Am not using CDH On Mon, 3 Aug 2015 at 6:01 AM Todd Nist wrote: > Not sure which Hadoop package (vendor) you are using, but there is a good > write up here on how to configure Zeppelin

Re: Yarn + Spark + Zepplin ?

Not sure which Hadoop package (vendor) you are using, but there is a good write up here on how to configure Zeppelin with Yarn for Cloudera, I would imagine most of this will carry over to any Hadoop package. http://blog.cloudera.com/blog/2015/07/how-to-install-apache-zeppelin-on-cdh/ HTH -Todd

Re: [ANNOUNCE] Apache Zeppelin 0.5.0-incubating released

Congratulations on achieving this milestone and good luck for many more to achieve in near future! Best, Dhaval "Sent from my iPhone, plaese excuse any typos :)" > On Aug 3, 2015, at 1:29 AM, Rajat Gupta wrote: > > congrats guys > >> On Mon, Aug 3, 2015 at 9:41 AM, Luke Han wrote: >> Co

Re: completion() method

Hi Moon, Thanks for confirming the issue. It is broken and doesn't work with any language interpreter Spark. ba30b09 is quite of a change and it doesn't make it easy to spot where it might have went