Re: Will nested field performance improve?

2016-04-15 Thread Michael Armbrust
> > If we expect fields nested in structs to always be much slower than flat > fields, then I would be keen to address that in our ETL pipeline with a > flattening step. If it's a known issue that we expect will be fixed in > upcoming releases, I'll hold off. > The difference might be even larger

Will nested field performance improve?

2016-04-15 Thread James Aley
Hello, I'm trying to make a call on whether my team should invest time added a step to "flatten" our schema as part of our ETL pipeline to improve performance of interactive queries. Our data start out life as Avro before being converted to Parquet, and so we follow the Avro idioms of creating