Could you check the log of executor to find the full stack trace ?
On Tue, Apr 26, 2016 at 12:30 AM, Mich Talebzadeh <mich.talebza...@gmail.com
> wrote:
> Hi,
>
> This JDBC connection was working fine in Spark 1.5,2
>
> val HiveContext = new org.apache.spark.sql.hive.HiveContext(sc)
> val sqlContext = new HiveContext(sc)
> println ("\nStarted at"); sqlContext.sql("SELECT
> FROM_unixtime(unix_timestamp(), 'dd/MM/yyyy HH:mm:ss.ss')
> ").collect.foreach(println)
> //
> var _ORACLEserver : String = "jdbc:oracle:thin:@rhes564:1521:mydb"
> var _username : String = "scratchpad"
> var _password : String = "xxxxxxx"
> //
> val s = HiveContext.load("jdbc",
> Map("url" -> _ORACLEserver,
> "dbtable" -> "(SELECT to_char(ID) AS ID, to_char(CLUSTERED) AS CLUSTERED,
> to_char(SCATTERED) AS SCATTERED, to_char(RANDOMISED) AS RANDOMISED,
> RANDOM_STRING, SMALL_VC, PADDING FROM scratchpad.dummy)",
> "user" -> _username,
> "password" -> _password))
>
> s.toDF.registerTempTable("tmp")
>
>
> // Need to create and populate target ORC table sales in database test in
> Hive
> //
> HiveContext.sql("use test")
> //
> // Drop and create table
> //
> HiveContext.sql("DROP TABLE IF EXISTS test.dummy2")
> var sqltext : String = ""
> sqltext = """
> CREATE TABLE test.dummy2
> (
> ID INT
> , CLUSTERED INT
> , SCATTERED INT
> , RANDOMISED INT
> , RANDOM_STRING VARCHAR(50)
> , SMALL_VC VARCHAR(10)
> , PADDING VARCHAR(10)
> )
> CLUSTERED BY (ID) INTO 256 BUCKETS
> STORED AS ORC
> TBLPROPERTIES ( "orc.compress"="SNAPPY",
> "orc.create.index"="true",
> "orc.bloom.filter.columns"="ID",
> "orc.bloom.filter.fpp"="0.05",
> "orc.stripe.size"="268435456",
> "orc.row.index.stride"="10000" )
> """
> HiveContext.sql(sqltext)
> //
> sqltext = """
> INSERT INTO TABLE test.dummy2
> SELECT
> *
> FROM tmp
> """
> HiveContext.sql(sqltext)
>
> In Spark 1.6.1, it is throwing error as below
>
>
> org.apache.spark.SparkException: Job aborted due to stage failure: Task 0
> in stage 1.0 failed 4 times, most recent failure: Lost task 0.3 in stage
> 1.0 (TID 4, rhes564): java.lang.IllegalStateException: Did not find
> registered driver with class oracle.jdbc.OracleDriver
>
> Is this a new bug introduced in Spark 1.6.1?
>
>
> Thanks
>
--
Best Regards
Jeff Zhang
