Re: How does extending an existing parquet with columns affect impala/spark performance?

>From spark point of view it shouldn’t effect. it’s possible to extend columns of new parquet files and it won’t affect Performance and not required to change spark application code. On Tue, Apr 3, 2018 at 9:14 AM Vitaliy Pisarev wrote: > This is not strictly a

How does extending an existing parquet with columns affect impala/spark performance?

This is not strictly a spark question but I'll give it a shot: have an existing setup of parquet files that are being queried from impala and from spark. I intend to add some 30 relatively 'heavy' columns to the parquet. Each column would store an array of structs. Each struct can have from 5 to