I noticed when querying struct data in spark sql, we are requesting the whole column from parquet files. Is this intended or is there some kind of config to control this behaviour? Wouldn't it be better to request just the struct field?
- SPARK SQL- Parquet projection pushdown for nested data Sadhan Sood
- Re: SPARK SQL- Parquet projection pushdown for neste... Michael Armbrust