Re: Spark 2.0 - Parquet data with fields containing periods "."

Yes, I just tested it against the nightly build from 8/31. Looking at the PR, I'm happy the test added verifies my issue. Thanks. -Don On Wed, Aug 31, 2016 at 6:49 PM, Hyukjin Kwon wrote: > Hi Don, I guess this should be fixed from 2.0.1. > > Please refer this PR.

Re: Spark 2.0 - Parquet data with fields containing periods "."

Hi Don, I guess this should be fixed from 2.0.1. Please refer this PR. https://github.com/apache/spark/pull/14339 On 1 Sep 2016 2:48 a.m., "Don Drake" wrote: > I am in the process of migrating a set of Spark 1.6.2 ETL jobs to Spark > 2.0 and have encountered some

Spark 2.0 - Parquet data with fields containing periods "."

I am in the process of migrating a set of Spark 1.6.2 ETL jobs to Spark 2.0 and have encountered some interesting issues. First, it seems the SQL parsing is different, and I had to rewrite some SQL that was doing a mix of inner joins (using where syntax, not inner) and outer joins to get the SQL