Praveen Tallapudi created SPARK-19687:
-----------------------------------------
Summary: SPARK supports for Postgres JSONB to store JSON data, if
yes, kindly please help us with any examples.
Key: SPARK-19687
URL: https://issues.apache.org/jira/browse/SPARK-19687
Project: Spark
Issue Type: Question
Components: Spark Core
Affects Versions: 2.1.1
Reporter: Praveen Tallapudi
Dear Team,
I am little new to Scala development and trying to find the solution for the
below. Please forgive me if this is not the correct place to post this question.
I am trying to insert data from a data frame into postgres table.
Dataframe Schema:
root
|-- ID: string (nullable = true)
|-- evtInfo: struct (nullable = true)
| |-- @date: string (nullable = true)
| |-- @time: string (nullable = true)
| |-- @timeID: string (nullable = true)
| |-- TranCode: string (nullable = true)
| |-- custName: string (nullable = true)
| |-- evtInfo: array (nullable = true)
| | |-- element: string (containsNull = true)
| |-- Type: string (nullable = true)
| |-- opID: string (nullable = true)
| |-- tracNbr: string (nullable = true)
DataBase Table Schema:
CREATE TABLE public.test
(
id bigint NOT NULL,
evtInfo jsonb NOT NULL,
evt_val bigint NOT NULL
)
When I use dataFrame_toSave.write.mode(SaveMode.Append).jdbc(dbUrl,
"public.test", dbPropForDFtoSave) to save the data, I am seeing the below error.
Exception in thread "main" java.lang.IllegalArgumentException: Can't get JDBC
type for
struct<@dateEvt:string,@timeEvt:string,@timeID:string,CICSTranCode:string,custName:string,evtInfo:array<string>,evtType:string,operID:string,trackingNbr:string>
Can you please suggest the best approach to save the data frame into the
posgres JSONB table.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
