Github user wangyum commented on a diff in the pull request:
https://github.com/apache/spark/pull/23027#discussion_r233686986
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/FilterPushdownBenchmark.scala
---
@@ -131,6 +134,15 @@ object FilterPushdownBenchmark extends BenchmarkBase
with SQLHelper {
}
}
+ Seq(false, true).foreach { pushDownEnabled =>
+ val name = s"InMemoryTable Vectorized ${if (pushDownEnabled)
s"(Pushdown)" else ""}"
+ benchmark.addCase(name) { _ =>
+ withSQLConf(SQLConf.IN_MEMORY_PARTITION_PRUNING.key ->
s"$pushDownEnabled") {
--- End diff --
I think the InMemoryTable's partition same to Parquet RowGroup(@kiszk
please correct if I'm wrong). We put them together and it's easy to compare
performance.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
