Hi team,
I am loading an CSV. One column contains a json value. I am unable to parse
that column properly. Below is the details. Can you please check once?
val df1=spark.read.option("inferSchema","true").
option("header","true").option("quote", "\"")
.option("escape", "\"").csv("/FileStore/tables/sample_file_structure.csv")
sample data:
----------------
column1,column2,column3
123456789,"{ "moveId" : "123456789", "dob" : null, "username" :
"abcdef", "language" : "en" }",11
123456789,"{ "moveId" : "123456789", "dob" : null, "username" : "ghi,
jkl", "language" : "en" }",12 123456789,"{ "moveId" : "123456789",
"dob" : null, "username" : "mno, pqr", "language" : "en" }",13
output:
-----------
+---------+--------------------+---------------+
| column1| column2| column3 |
+---------+--------------------+---------------+
|123456789|"{ "moveId" : "...| "dob" : null|
|123456789|"{ "moveId" : "...| "dob" : null|
+---------+--------------------+---------------+
Thanks,
Elango
