Re: strange behavior when I chain data frame transformations

2016-05-13 Thread Andy Davidson
:38 PM To: Andrew Davidson <a...@santacruzintegration.com> Cc: "user @spark" <user@spark.apache.org> Subject: Re: strange behavior when I chain data frame transformations > In the structure shown, tag is under element. > > I wonder if that was a factor. > > On F

Re: strange behavior when I chain data frame transformations

2016-05-13 Thread Ted Yu
In the structure shown, tag is under element. I wonder if that was a factor. On Fri, May 13, 2016 at 11:49 AM, Andy Davidson < a...@santacruzintegration.com> wrote: > I am using spark-1.6.1. > > I create a data frame from a very complicated JSON file. I would assume > that query planer would

strange behavior when I chain data frame transformations

2016-05-13 Thread Andy Davidson
I am using spark-1.6.1. I create a data frame from a very complicated JSON file. I would assume that query planer would treat both version of my transformation chains the same way. // org.apache.spark.sql.AnalysisException: Cannot resolve column name "tag" among (actor, body, generator, pip,