Hi,
I am able to successfully create shark table with 3 columns and 2 rows.
val recList = List((" value A1", "value B1","value C1"),
("value A2", "value B2","value c2"));
val dbFields =List ("Col A", "Col B","Col C")
val rdd = sc.parallelize(recList)
RDDTable(rdd).saveAsTable("table_1", dbFields)
I have a scenario where table will have 60 columns. How to achieve it using
RDDTable.
I tried creating a List[(Seq[String],Seq[String])] , but it throws below
exception.Any help /pointer will help.
Exception in thread "main" shark.api.DataTypes$UnknownDataTypeException:
scala.collection.Seq
at shark.api.DataTypes.fromClassTag(DataTypes.java:133)
at shark.util.HiveUtils$$anonfun$1.apply(HiveUtils.scala:106)
at shark.util.HiveUtils$$anonfun$1.apply(HiveUtils.scala:105)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
at scala.collection.immutable.List.foreach(List.scala:318)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:244)
at scala.collection.AbstractTraversable.map(Traversable.scala:105)
at shark.util.HiveUtils$.createTableInHive(HiveUtils.scala:105)
at shark.api.RDDTableFunctions.saveAsTable(RDDTableFunctions.scala:63)
Thanks
Subacini