Yes, Text file is schema less. Spark does not know what to skip so it will read everything. Parquet as you have stated is capable of taking advantage of predicate push down.
..Manas -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Does-Spark-SQL-have-to-scan-all-the-columns-of-a-table-in-text-format-tp25505p25684.html Sent from the Apache Spark User List mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
