iemejia commented on code in PR #12400: URL: https://github.com/apache/gluten/pull/12400#discussion_r3616195527
########## backends-velox/src/test/scala/org/apache/spark/sql/execution/benchmark/FileHandleCacheBenchmark.scala: ########## @@ -0,0 +1,127 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.spark.sql.execution.benchmark + +import org.apache.gluten.config.VeloxConfig + +import org.apache.spark.benchmark.Benchmark + +/** + * Benchmark to measure the effect of Velox file handle caching on repeated scans of many small + * Parquet files. File handle caching avoids repeated open/close overhead per file, which is + * especially beneficial for remote filesystems (S3, HDFS, ABFS) where handle creation involves + * network round-trips (DNS, TCP, auth). + * + * Even on local filesystems the overhead is measurable when scanning hundreds of small files + * multiple times (e.g., repeated queries on a heavily-partitioned table). + * + * NOTE: `fileHandleCacheEnabled` is a static config (read at backend init). To compare on vs off, + * run this benchmark twice with different Spark configurations: + * {{{ + * # With file handle cache enabled (default): + * bin/spark-submit --class <this class> \ + * --conf spark.gluten.sql.columnar.backend.velox.fileHandleCacheEnabled=true \ + * --jars <spark core test jar>,<sql core test jar> \ + * <application jar> + * + * # With file handle cache disabled: + * bin/spark-submit --class <this class> \ + * --conf spark.gluten.sql.columnar.backend.velox.fileHandleCacheEnabled=false \ + * --jars <spark core test jar>,<sql core test jar> \ + * <application jar> + * }}} + * + * Expected result: with caching enabled, repeated scans should show lower wall-clock time due to + * avoiding per-file open() syscalls (or remote filesystem connection establishment) on subsequent + * scans of the same files. + */ +object FileHandleCacheBenchmark extends SqlBasedBenchmark { Review Comment: Fair point on the missing numbers — I added them, and want to be upfront about what this benchmark does and doesn't show. Local run (200 Parquet files × 5000 rows, 10 scans/iter, best of 5), cache ON vs OFF: | Case | Cache OFF (best/avg ms) | Cache ON (best/avg ms) | |---|---|---| | full scan | 2625 / 3521 | 2549 / 2794 | | filtered scan | 2734 / 2899 | 2710 / 2861 | | column pruning | 2304 / 2434 | 2262 / 2312 | The effect on local FS is within noise, which is expected — a local `open()` is cheap. I also traced the remote path in the Velox source to check whether the cache saves the open-time round-trip it's designed for, and on the Gluten path it mostly doesn't: Gluten passes each file's size into the split, so `S3ReadFile`/`AbfsReadFile` skip their `HeadObject`/`GetProperties` on open (they return early when size is known). The main exception is ABFS+OAuth, where the adapter rebuilds the token credential per open, so a cached handle avoids repeated AAD token fetches. So this benchmark is best understood as a local-FS reproducibility harness, not proof of a remote win (it only exercises a local `withTempPath`). I'm inclined to keep it on that basis — it's not wired into CI and follows the existing `SqlBasedBenchmark` pattern — but since it doesn't demonstrate the remote case, I'm equally happy to remove it if you'd prefer. Your call as reviewer. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
