I am trying to insert into temporary table created on a Spark (v 1.6) DataFrame
loaded using Phoenix-Spark (v 4.4) plugin. Below is the code:
val sc = new SparkContext("local", "phoenix-test")
val configuration = new Configuration()
configuration.set("zookeeper.znode.parent", "/hbase-unsecure")
val df = sqlContext.phoenixTableAsDataFrame(
"EMAIL_ENRON", Array("MAIL_FROM", "MAIL_TO"), conf = configuration)
df.registerTempTable("TEMP_TABLE");
Table which is defined in HBase using Phoenix is :
TABLE EMAIL_ENRON(MAIL_FROM BIGINT NOT NULL, MAIL_TO BIGINT NOT NULL CONSTRAINT
pk PRIMARY KEY(MAIL_FROM, MAIL_TO));
While trying to insert into temporary table, spark-shell gives the below error
Insert statement:
sqlContext.sql("insert into table TEMP_TABLE select t.* from (select 55,66) t");
Exception:
scala> sqlContext.sql("insert into table TEMP_TABLE select t.* from (select
55,66) t");
16/11/16 11:15:46 INFO ParseDriver: Parsing command: insert into table
TEMP_TABLE select t.* from (select 55,66) t
16/11/16 11:15:46 INFO ParseDriver: Parse Completed
org.apache.spark.sql.AnalysisException: unresolved operator 'InsertIntoTable
LogicalRDD [MAIL_FROM#0L,MAIL_TO#1L], MapPartitionsRDD[3] at createDataFrame at
PhoenixRDD.scala:117, Map(), false, false;
at
org.apache.spark.sql.catalyst.analysis.CheckAnalysis$class.failAnalysis(CheckAnalysis.scala:38)
at
org.apache.spark.sql.catalyst.analysis.Analyzer.failAnalysis(Analyzer.scala:44)
at
org.apache.spark.sql.catalyst.analysis.CheckAnalysis$$anonfun$checkAnalysis$1.apply(CheckAnalysis.scala:203)
What is the correct way to insert into a temporary table ? From the exception
it looks like that Phoenix-Spark does allow inserting into temporary table, may
be the syntax that i am using is incorrect.
________________________________
NOTE: This message may contain information that is confidential, proprietary,
privileged or otherwise protected by law. The message is intended solely for
the named addressee. If received in error, please destroy and notify the
sender. Any use of this email is prohibited when received in error. Impetus
does not represent, warrant and/or guarantee, that the integrity of this
communication has been maintained nor that the communication is free of errors,
virus, interception or interference.
