Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21320 Try this when `spark.sql.nestedSchemaPruning.enabled` is on? ```SQL withTable("t1") { spark.sql( """ |Create table t1 (`id` INT,`CoL1` STRING, |`coL2` STRUCT<`a`: TIMESTAMP, `b`: INT, `c`: ARRAY<DATE>>, |`col3` INT, |`col4` INT, |`Col5` STRUCT<`a`: STRING, `b`: STRUCT<`a`: INT, `b`: ARRAY<INT>, `c`: INT>, `c`: INT>) |USING parquet """.stripMargin) spark.sql("SELECT id from t1 where col2.b < 7").count() } ```
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org