Re: Loading already existing tables in spark shell

2015-08-25 Thread Ishwardeep Singh
Hi Jeetendra, Please try the following in spark shell. it is like executing an sql command. sqlContext.sql(use database name) Regards, Ishwardeep From: Jeetendra Gangele gangele...@gmail.com Sent: Tuesday, August 25, 2015 12:57 PM To: Ishwardeep Singh Cc

Re: Loading already existing tables in spark shell

2015-08-24 Thread Ishwardeep Singh
Hi Jeetendra, I faced this issue. I did not specify the database where this table exists. Please set the database by using use database command before executing the query. Regards, Ishwardeep From: Jeetendra Gangele gangele...@gmail.com Sent: Monday,

RE: Spark SQL support for Hive 0.14

2015-08-05 Thread Ishwardeep Singh
Thanks Steve and Michael for your response. Is there a tentative release date for Spark 1.5? From: Michael Armbrust [mailto:mich...@databricks.com] Sent: Tuesday, August 4, 2015 11:53 PM To: Steve Loughran ste...@hortonworks.com Cc: Ishwardeep Singh ishwardeep.si...@impetus.co.in; user

Spark SQL support for Hive 0.14

2015-08-04 Thread Ishwardeep Singh
Hi, Does spark SQL support Hive 0.14? The documentation refers to Hive 0.13. Is there a way to compile spark with Hive 0.14? Currently we are using Spark 1.3.1. Thanks -- View this message in context:

Re: Unable to query existing hive table from spark sql 1.3.0

2015-08-03 Thread Ishwardeep Singh
Your table is in which database - default or result. By default spark will try to look for table in default database. If the table exists in the result database try to prefix the table name with database name like select * from result.salarytest or set the database by executing use database name

RE: [Spark SQL 1.3.1] data frame saveAsTable returns exception

2015-05-14 Thread Ishwardeep Singh
Hi Michael Ayan, Thank you for your response to my problem. Michael do we have a tentative release date for Spark version 1.4? Regards, Ishwardeep From: Michael Armbrust [mailto:mich...@databricks.com] Sent: Wednesday, May 13, 2015 10:54 PM To: ayan guha Cc: Ishwardeep Singh; user Subject

[Spark SQL 1.3.1] data frame saveAsTable returns exception

2015-05-13 Thread Ishwardeep Singh
Hi , I am using Spark SQL 1.3.1. I have created a dataFrame using jdbc data source and am using saveAsTable() method but got the following 2 exceptions: java.lang.RuntimeException: Unsupported datatype DecimalType() at scala.sys.package$.error(package.scala:27) at

Re: [Spark SQL 1.3.1] data frame saveAsTable returns exception

2015-05-13 Thread Ishwardeep Singh
Hi, I am using spark-shell and the steps using which I can reproduce the issue are as follows: scala val dateDimDF= sqlContext.load(jdbc,Map(url-jdbc:teradata://192.168.145.58/DBS_PORT=1025,DATABASE=BENCHQADS,LOB_SUPPORT=OFF,USER= BENCHQADS,PASSWORD=abc,dbtable - date_dim)) scala

Re: Unable to join table across data sources using sparkSQL

2015-05-08 Thread Ishwardeep Singh
Finally got it working. I was trying to access hive using the jdbc driver like I was trying to access the terradata. It took me some time to figure out that default sqlContext created by Spark supported hive and it uses the hive-site.xml in spark conf folder to access hive. I had to use my

RE: Unable to join table across data sources using sparkSQL

2015-05-05 Thread Ishwardeep Singh
, Ishwardeep From: ankitjindal [via Apache Spark User List] [mailto:ml-node+s1001560n22766...@n3.nabble.com] Sent: Tuesday, May 5, 2015 5:00 PM To: Ishwardeep Singh Subject: RE: Unable to join table across data sources using sparkSQL Just check the Schema of both the tables using frame.printSchema

RE: Unable to join table across data sources using sparkSQL

2015-05-05 Thread Ishwardeep Singh
User List] [mailto:ml-node+s1001560n22762...@n3.nabble.com] Sent: Tuesday, May 5, 2015 1:26 PM To: Ishwardeep Singh Subject: Re: Unable to join table across data sources using sparkSQL Hi I was doing the same but with a file in hadoop as a temp table and one table in sql server but i succeeded

RE: Unable to compile spark 1.1.0 on windows 8.1

2014-12-01 Thread Ishwardeep Singh
Hi Judy, Thank you for your response. When I try to compile using maven mvn -Dhadoop.version=1.2.1 -DskipTests clean package I get an error Error: Could not find or load main class . I have maven 3.0.4. And when I run command sbt package I get the same exception as earlier. I have done the

Unable to compile spark 1.1.0 on windows 8.1

2014-11-27 Thread Ishwardeep Singh
Hi, I am trying to compile spark 1.1.0 on windows 8.1 but I get the following exception. [info] Compiling 3 Scala sources to D:\myworkplace\software\spark-1.1.0\project\target\scala-2.10\sbt0.13\classes... [error] D:\myworkplace\software\spark-1.1.0\project\SparkBuild.scala:26: object sbt is