It might be a problem when inserting into a partitioned table. It worked fine to when the target table was unpartitioned.
Can you confirm this? Thanks, Du On 9/26/14, 4:48 PM, "Du Li" <l...@yahoo-inc.com.INVALID> wrote: >Hi, > >I was loading data into a partitioned table on Spark 1.1.0 >beeline-thriftserver. The table has complex data types such as map<string, >string> and array<map<string,string>>. The query is like ³insert overwrite >table a partition (Š) select Š² and the select clause worked if run >separately. However, when running the insert query, there was an error as >follows. > >The source code of Cast.scala seems to only handle the primitive data >types, which is perhaps why the MatchError was thrown. > >I just wonder if this is still work in progress, or I should do it >differently. > >Thanks, >Du > > >---- >scala.MatchError: MapType(StringType,StringType,true) (of class >org.apache.spark.sql.catalyst.types.MapType) > >org.apache.spark.sql.catalyst.expressions.Cast.cast$lzycompute(Cast.scala: >2 >47) > >org.apache.spark.sql.catalyst.expressions.Cast.cast(Cast.scala:247) > >org.apache.spark.sql.catalyst.expressions.Cast.eval(Cast.scala:263) > >org.apache.spark.sql.catalyst.expressions.Alias.eval(namedExpressions.scal >a >:84) > >org.apache.spark.sql.catalyst.expressions.InterpretedMutableProjection.app >l >y(Projection.scala:66) > >org.apache.spark.sql.catalyst.expressions.InterpretedMutableProjection.app >l >y(Projection.scala:50) > scala.collection.Iterator$$anon$11.next(Iterator.scala:328) > scala.collection.Iterator$$anon$11.next(Iterator.scala:328) > >org.apache.spark.sql.hive.execution.InsertIntoHiveTable.org$apache$spark$s >q >l$hive$execution$InsertIntoHiveTable$$writeToFile$1(InsertIntoHiveTable.sc >a >la:149) > >org.apache.spark.sql.hive.execution.InsertIntoHiveTable$$anonfun$saveAsHiv >e >File$1.apply(InsertIntoHiveTable.scala:158) > >org.apache.spark.sql.hive.execution.InsertIntoHiveTable$$anonfun$saveAsHiv >e >File$1.apply(InsertIntoHiveTable.scala:158) > org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:62) > org.apache.spark.scheduler.Task.run(Task.scala:54) > >org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:177) > >java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java: >1 >145) > >java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java >: >615) > java.lang.Thread.run(Thread.java:722) > > > > > > >--------------------------------------------------------------------- >To unsubscribe, e-mail: user-unsubscr...@spark.apache.org >For additional commands, e-mail: user-h...@spark.apache.org >
