Hello Jianshi, You meant you want to convert a Map to a Struct, right? We can extract some useful functions from JsonRDD.scala, so others can access them.
Thanks, Yin On Mon, Dec 8, 2014 at 1:29 AM, Jianshi Huang <jianshi.hu...@gmail.com> wrote: > I checked the source code for inferSchema. Looks like this is exactly what > I want: > > val allKeys = rdd.map(allKeysWithValueTypes).reduce(_ ++ _) > > Then I can do createSchema(allKeys). > > Jianshi > > On Sun, Dec 7, 2014 at 2:50 PM, Jianshi Huang <jianshi.hu...@gmail.com> > wrote: > >> Hmm.. >> >> I've created a JIRA: https://issues.apache.org/jira/browse/SPARK-4782 >> >> Jianshi >> >> On Sun, Dec 7, 2014 at 2:32 PM, Jianshi Huang <jianshi.hu...@gmail.com> >> wrote: >> >>> Hi, >>> >>> What's the best way to convert RDD[Map[String, Any]] to a SchemaRDD? >>> >>> I'm currently converting each Map to a JSON String and do >>> JsonRDD.inferSchema. >>> >>> How about adding inferSchema support to Map[String, Any] directly? It >>> would be very useful. >>> >>> Thanks, >>> -- >>> Jianshi Huang >>> >>> LinkedIn: jianshi >>> Twitter: @jshuang >>> Github & Blog: http://huangjs.github.com/ >>> >> >> >> >> -- >> Jianshi Huang >> >> LinkedIn: jianshi >> Twitter: @jshuang >> Github & Blog: http://huangjs.github.com/ >> > > > > -- > Jianshi Huang > > LinkedIn: jianshi > Twitter: @jshuang > Github & Blog: http://huangjs.github.com/ >
