Eventually it would be nice for us to have some sort of function to do the
conversion you are talking about on a single column, but for now I usually
hack it as you suggested:
val withId = origRDD.map { case (id, str) => s"""{"id":$id,
${str.trim.drop(1)}""" }
val table = sqlContext.jsonRDD(withId)
On Thu, Jan 29, 2015 at 1:29 AM, Tobias Pfeiffer <t...@preferred.jp> wrote:
> Hi Ayoub,
>
> thanks for your mail!
>
> On Thu, Jan 29, 2015 at 6:23 PM, Ayoub <benali.ayoub.i...@gmail.com>
> wrote:
>>
>> SQLContext and hiveContext have a "jsonRDD" method which accept an
>> RDD[String] where the string is a JSON String a returns a SchemaRDD, it
>> extends RDD[Row] which the type you want.
>>
>> After words you should be able to do a join to keep your tuple.
>>
>
> I'm afraid that's not so easy, because you can only join on a certain key,
> and the key is exactly what I have to drop in order to infer the schema.
>
> Thanks
> Tobias
>
>
