Try use: org.apache.hadoop.hive.serde2.RegexSerDe GP
On 27 Jul 2015, at 09:35, ZhuGe <t...@outlook.com<mailto:t...@outlook.com>> wrote: Hi all: I am testing the performance of hive on spark sql. The existing table is created with ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.RegexSerDe' WITH SERDEPROPERTIES ( 'input.regex' = '(.*?)\\|\\^\\|(.*?)\\|\\^\\|(.*?)\\|\\^\\|(.*?)\\|\\^\\|(.*?)\\|\\^\\|(.*?)\\|\\^\\|(.*?)\\|\\^\\|(.*?)\\|\\^\\|(.*?)\\|\\^\\|(.*?)\\|\\^\\|(.*?)\\|\\^\\|(.*?)\\|\\^\\|(.*?)\\|\\^\\|(.*?)\\|\\^\\|(.*?)\\|\\^\\|(.*?)\\|\\^\\|(.*?)' ,'output.format.string' = '%1$s %2$s %3$s %4$s %5$s %16$s %7$s %8$s %9$s %10$s %11$s %12$s %13$s %14$s %15$s %16$s %17$s ') STORED AS TEXTFILE location '/data/BaseData/wx/xx/xx/xx/xx'; When i use spark sql(spark-shell) to query the existing table, got exception like this: Caused by: MetaException(message:java.lang.ClassNotFoundException Class org.apache.hadoop.hive.contrib.serde2.RegexSerDe not found) at org.apache.hadoop.hive.metastore.MetaStoreUtils.getDeserializer(MetaStoreUtils.java:382) at org.apache.hadoop.hive.ql.metadata.Partition.getDeserializer(Partition.java:249) I add the jar dependency in the spark-shell command, still do not work. SPARK_SUBMIT_OPTS="-XX:MaxPermSize=256m" ./bin/spark-shell --jars /data/dbcenter/cdh5/spark-1.4.0-bin-hadoop2.4/hive-contrib-0.13.1-cdh5.2.0.jar,postgresql-9.2-1004-jdbc41.jar How should i fix the problem? Cheers
