Re: Spark SQL "SELECT ... LIMIT" scans the entire Hive table?

2015-11-05 Thread Jon Gregg
>> of >> doing a .take(5) on it and returning results immediately. >> >> Is there a way to get Spark SQL to use .take(5) instead of the Hive logic >> of >> scanning the full table when running a SELECT? >> >> >> >> -- >> View this me

Spark SQL "SELECT ... LIMIT" scans the entire Hive table?

2015-10-05 Thread YaoPau
f scanning the full table when running a SELECT? -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-SQL-SELECT-LIMIT-scans-the-entire-Hive-table-tp24938.html Sent from the Apache Spark User List mailing list archive at

Re: Spark SQL "SELECT ... LIMIT" scans the entire Hive table?

2015-10-05 Thread Michael Armbrust
this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Spark-SQL-SELECT-LIMIT-scans-the-entire-Hive-table-tp24938.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > -