Hello Sankar,
"Add JAR" in SQL is not supported at the moment. We are working on it (
https://issues.apache.org/jira/browse/SPARK-2219). For now, can you try
SparkContext.addJar or using "--jars <your-jar>" to launch spark shell?
Thanks,
Yin
On Fri, Aug 22, 2014 at 2:01 PM, S Malligarjunan <smalligarju...@yahoo.com>
wrote:
> Hello Yin/All.
>
> @Yin - Thanks for helping. I solved the sql parser error. I am getting the
> following exception now
>
> scala> hiveContext.hql("ADD JAR s3n://hadoop.anonymous.com/lib/myudf.jar
> ");
> warning: there were 1 deprecation warning(s); re-run with -deprecation for
> details
> 14/08/22 17:58:55 INFO SessionState: converting to local s3n://
> hadoop.anonymous.com/lib/myudf.jar
> 14/08/22 17:58:56 ERROR SessionState: Unable to register
> /tmp/3d273a4c-0494-4bec-80fe-86aa56f11684_resources/myudf.jar
> Exception: org.apache.spark.repl.SparkIMain$TranslatingClassLoader cannot
> be cast to java.net.URLClassLoader
> java.lang.ClassCastException:
> org.apache.spark.repl.SparkIMain$TranslatingClassLoader cannot be cast to
> java.net.URLClassLoader
> at
> org.apache.hadoop.hive.ql.exec.Utilities.addToClassPath(Utilities.java:1680)
>
>
> Thanks and Regards,
> Sankar S.
>
>
>
> On Friday, 22 August 2014, 22:53, S Malligarjunan
> <smalligarju...@yahoo.com.INVALID> wrote:
>
>
> Hello Yin,
>
> Forgot to mention one thing, the same query works fine in Hive and Shark..
>
> Thanks and Regards,
> Sankar S.
>
>
>
> On , S Malligarjunan <smalligarju...@yahoo.com> wrote:
>
>
> Hello Yin,
>
> I have tried the create external table command as well. I get the same
> error.
> Please help me to find the root cause.
>
> Thanks and Regards,
> Sankar S.
>
>
>
> On Friday, 22 August 2014, 22:43, Yin Huai <huaiyin....@gmail.com>
> wrote:
>
>
> Hi Sankar,
>
> You need to create an external table in order to specify the location of
> data (i.e. using CREATE EXTERNAL TABLE user1 .... LOCATION). You can
> take a look at this page
> <https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-Create/Drop/TruncateTable>
> for
> reference.
>
> Thanks,
>
> Yin
>
>
> On Thu, Aug 21, 2014 at 11:12 PM, S Malligarjunan <
> smalligarju...@yahoo.com.invalid> wrote:
>
> Hello All,
>
> When i execute the following query
>
>
> val hiveContext = new org.apache.spark.sql.hive.HiveContext(sc)
>
> CREATE TABLE user1 (time string, id string, u_id string, c_ip string,
> user_agent string) ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t' LINES
> TERMINATED BY '\n' STORED AS TEXTFILE LOCATION 's3n://
> hadoop.anonymous.com/output/qa/cnv_px_ip_gnc/ds=2014-06-14/')
>
> I am getting the following error
> org.apache.spark.sql.hive.HiveQl$ParseException: Failed to parse: CREATE
> TABLE user1 (time string, id string, u_id string, c_ip string, user_agent
> string) ROW FORMAT DELIMITED FIELDS TERMINATED BY ' ' LINES TERMINATED BY
> '
> ' STORED AS TEXTFILE LOCATION 's3n://
> hadoop.anonymous.com/output/qa/cnv_px_ip_gnc/ds=2014-06-14/')
> at org.apache.spark.sql.hive.HiveQl$.parseSql(HiveQl.scala:215)
> at org.apache.spark.sql.hive.HiveContext.hiveql(HiveContext.scala:98)
> at org.apache.spark.sql.hive.HiveContext.hql(HiveContext.scala:102)
> at $iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:22)
> at $iwC$$iwC$$iwC$$iwC.<init>(<console>:27)
> at $iwC$$iwC$$iwC.<init>(<console>:29)
> at $iwC$$iwC.<init>(<console>:31)
> at $iwC.<init>(<console>:33)
> at <init>(<console>:35)
>
> Kindly let me know what could be the issue here.
>
> I have cloned spark from github. Using Hadoop 1.0.3
>
> Thanks and Regards,
> Sankar S.
>
>
>
>
>
>
>
>
>
