Hi,
I am trying to create custom schema but its throwing below error
scala> import org.apache.spark.sql.hive.HiveContext
> import org.apache.spark.sql.hive.HiveContext
>
> scala> import org.apache.spark.sql.hive.orc._
> import org.apache.spark.sql.hive.orc._
>
> scala> val hiveContext = new org.apache.spark.sql.hive.HiveContext(sc)
> 15/12/23 04:42:09 WARN SparkConf: The configuration key
> 'spark.yarn.applicationMaster.waitTries' has been deprecated as of Spark
> 1.3 and and may be removed in the future. Please use the new key
> 'spark.yarn.am.waitTime' instead.
> 15/12/23 04:42:09 INFO HiveContext: Initializing execution hive, version
> 0.13.1
> hiveContext: org.apache.spark.sql.hive.HiveContext =
> org.apache.spark.sql.hive.HiveContext@3ca50ddf
>
> scala> import org.apache.spark.sql.types.{StructType, StructField,
> StringType, IntegerType,FloatType ,LongType ,TimestampType };
> import org.apache.spark.sql.types.{StructType, StructField, StringType,
> IntegerType, FloatType, LongType, TimestampType}
>
> scala> val loandepoSchema = StructType(Seq(
> | StructField("C1" StringType, true),
> | StructField("COLUMN2", StringType , true),
> | StructField("COLUMN3", StringType, true),
> | StructField("COLUMN4", StringType, true),
> | StructField("COLUMN5", StringType , true),
> | StructField("COLUMN6", StringType, true),
> | StructField("COLUMN7", StringType, true),
> | StructField("COLUMN8", StringType, true),
> | StructField("COLUMN9", StringType, true),
> | StructField("COLUMN10", StringType, true),
> | StructField("COLUMN11", StringType, true),
> | StructField("COLUMN12", StringType, true),
> | StructField("COLUMN13", StringType, true),
> | StructField("COLUMN14", StringType, true),
> | StructField("COLUMN15", StringType, true),
> | StructField("COLUMN16", StringType, true),
> | StructField("COLUMN17", StringType, true)
> | StructField("COLUMN18", StringType, true),
> | StructField("COLUMN19", StringType, true),
> | StructField("COLUMN20", StringType, true),
> | StructField("COLUMN21", StringType, true),
> | StructField("COLUMN22", StringType, true)))
> <console>:25: error: value StringType is not a member of String
> StructField("C1" StringType, true),
> ^
>
Would really appreciate the guidance/pointers.
Thanks,
Divya
