Ah, thanks for reporting that. I've filed: https://issues.apache.org/jira/browse/SPARK-3238
On Tue, Aug 26, 2014 at 2:16 PM, Jim <[email protected]> wrote: > > The problem was that the json field names were not proper scala/java field > names (they contained spaces, dashes, plus signs, and various other > symbols). Works now. Thanks. > > > On 08/26/2014 04:06 PM, Nathan Howell wrote: > >> I've used it successfully for schemas containing a mix of nested structs >> and arrays.. somewhere in the 50-100 column range. >> >> -n >> >> On 8/26/14, 1:01 PM, "Jim" <[email protected]> wrote: >> >> Funny you should mention that. I tried that first. It failed on the >>> saveAsParquetFile with a cryptic: >>> >>> java.lang.RuntimeException: Unsupported dataType: >>> StructType(ArrayBuffer(StructField( ... 500 columns worth of the >>> same...) [1.7784] failure: `,' expected but `A' found" >>> >>> I assumed this had to do with not including a schema. >>> >>> On 08/26/2014 03:31 PM, Dmitriy Ryaboy wrote: >>> >>>> Nice -- using Spark to infer the json schema. Also a good way to do >>>> that. >>>> Does it handle nesting and everything? >>>> >>> >
