gt; <http://www.amazon.com/Big-Data-Analytics-Spark-Practitioners/dp/1484209656/>
>
>
>
> *From:* Andrés Ivaldi [mailto:iaiva...@gmail.com]
> *Sent:* Wednesday, January 27, 2016 7:17 PM
> *To:* Cheng, Hao
> *Cc:* Sahil Sareen; Al Pivonka; user
>
> *Subject:* Re: JS
day, January 27, 2016 7:17 PM
To: Cheng, Hao
Cc: Sahil Sareen; Al Pivonka; user
Subject: Re: JSON to SQL
I'm using DataFrames reading the JSON exactly as you say, and I can get the
scheme from there. Reading the documentation, I realized that is possible to
create Dynamically a Stru
“a.b[0].c” format of expression.
From: Andrés Ivaldi [mailto:iaiva...@gmail.com]
Sent: Thursday, January 28, 2016 3:39 AM
To: Sahil Sareen
Cc: Al Pivonka; user
Subject: Re: JSON to SQL
I'm really brand new with Scala, but if I'm defining a case class then is
becouse I know how is the json's
/Hive/LanguageManual+LateralView,
> as well as the “a.b[0].c” format of expression.
>
>
>
>
>
> *From:* Andrés Ivaldi [mailto:iaiva...@gmail.com]
> *Sent:* Thursday, January 28, 2016 3:39 AM
> *To:* Sahil Sareen
> *Cc:* Al Pivonka; user
> *Subject:* Re: JSON to S
Are you using an Relational Database?
If so why not use a nojs DB ? then pull from it to your relational?
Or utilize a library that understands Json structure like Jackson to obtain
the data from the Json structure the persist the Domain Objects ?
On Wed, Jan 27, 2016 at 9:45 AM, Andrés Ivaldi
We dont have Domain Objects, its a service like a pipeline, data is read
from source and they are saved it in relational Database
I can read the structure from DataFrames, and do some transformations, I
would prefer to do it with Spark to be consistent with the process
On Wed, Jan 27, 2016 at
Isn't this just about defining a case class and using
parse(json).extract[CaseClassName] using Jackson?
-Sahil
On Wed, Jan 27, 2016 at 11:08 PM, Andrés Ivaldi wrote:
> We dont have Domain Objects, its a service like a pipeline, data is read
> from source and they are saved
I'm really brand new with Scala, but if I'm defining a case class then is
becouse I know how is the json's structure is previously?
If I'm able to define dinamicaly a case class from the JSON structure then
even with spark I will be able to extract the data
On Wed, Jan 27, 2016 at 4:01 PM,
More detail is needed.
Can you provide some context to the use-case ?
On Wed, Jan 27, 2016 at 8:33 AM, Andrés Ivaldi wrote:
> Hello, I'm trying to Save a JSON filo into SQL table.
>
> If i try to do this directly the IlligalArgumentException is raised, I
> suppose this is
Sure,
The Job is like an etl, but without interface, so I decide the rules of how
the JSON will be saved into a SQL Table.
I need to Flatten the hierarchies where is possible in case of list flatten
also, nested objects Won't be processed by now
{"a":1,"b":[2,3],"c"="Field", "d":[4,5,6,7,8] }
10 matches
Mail list logo