This is an automated email from the ASF dual-hosted git repository.
mingliang pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/gluten.git
The following commit(s) were added to refs/heads/main by this push:
new 93558344c0 [VL] Remove dead config
spark.gluten.velox.fs.s3a.connect.timeout (#11810)
93558344c0 is described below
commit 93558344c079ba745fd005731d47b40dc55d91d4
Author: James Xu <[email protected]>
AuthorDate: Tue Mar 24 09:52:20 2026 +0800
[VL] Remove dead config spark.gluten.velox.fs.s3a.connect.timeout (#11810)
The config `spark.gluten.velox.fs.s3a.connect.timeout` was originally
introduced in GLUTEN-5659 and explicitly read by ConfigExtractor.cc to
set the S3 TCP connect timeout (connectTimeoutMs).
In GLUTEN-9182, ConfigExtractor.cc was refactored to use a suffix-map
approach reading `spark.hadoop.fs.s3a.connection.establish.timeout`
for connectTimeoutMs and `spark.hadoop.fs.s3a.connection.timeout` for
requestTimeoutMs. The old key was never cleaned up, leaving
`AWS_S3_CONNECT_TIMEOUT` defined in VeloxConfig.scala but never
referenced, and the key forwarded into the native conf map but never
consumed by any C++ code.
Remove the config definition, its entries in nativeKeys and
getNativeBackendConf defaults, and the documentation row.
---
.../src/main/scala/org/apache/gluten/config/VeloxConfig.scala | 6 ------
docs/velox-configuration.md | 1 -
.../src/main/scala/org/apache/gluten/config/GlutenConfig.scala | 2 --
3 files changed, 9 deletions(-)
diff --git
a/backends-velox/src/main/scala/org/apache/gluten/config/VeloxConfig.scala
b/backends-velox/src/main/scala/org/apache/gluten/config/VeloxConfig.scala
index 2619003201..633979613d 100644
--- a/backends-velox/src/main/scala/org/apache/gluten/config/VeloxConfig.scala
+++ b/backends-velox/src/main/scala/org/apache/gluten/config/VeloxConfig.scala
@@ -551,12 +551,6 @@ object VeloxConfig extends ConfigRegistry {
.stringConf
.createWithDefault("legacy")
- val AWS_S3_CONNECT_TIMEOUT =
- buildConf("spark.gluten.velox.fs.s3a.connect.timeout")
- .doc("Timeout for AWS s3 connection.")
- .stringConf
- .createWithDefault("200s")
-
val VELOX_ORC_SCAN_ENABLED =
buildConf("spark.gluten.sql.columnar.backend.velox.orc.scan.enabled")
.doc("Enable velox orc scan. If disabled, vanilla spark orc scan will be
used.")
diff --git a/docs/velox-configuration.md b/docs/velox-configuration.md
index 29ad925d52..859c6356c1 100644
--- a/docs/velox-configuration.md
+++ b/docs/velox-configuration.md
@@ -78,7 +78,6 @@ nav_order: 16
| spark.gluten.sql.enable.enhancedFeatures
| true | Enable some features including iceberg native write
and other features.
[...]
| spark.gluten.sql.rewrite.castArrayToString
| true | When true, rewrite `cast(array as String)` to
`concat('[', array_join(array, ', ', null), ']')` to allow offloading to Velox.
[...]
| spark.gluten.velox.castFromVarcharAddTrimNode
| false | If true, will add a trim node which has the same
sementic as vanilla Spark to CAST-from-varchar.Otherwise, do nothing.
[...]
-| spark.gluten.velox.fs.s3a.connect.timeout
| 200s | Timeout for AWS s3 connection.
[...]
## Gluten Velox backend *experimental* configurations
diff --git
a/gluten-substrait/src/main/scala/org/apache/gluten/config/GlutenConfig.scala
b/gluten-substrait/src/main/scala/org/apache/gluten/config/GlutenConfig.scala
index 2be6b64218..371c2b9609 100644
---
a/gluten-substrait/src/main/scala/org/apache/gluten/config/GlutenConfig.scala
+++
b/gluten-substrait/src/main/scala/org/apache/gluten/config/GlutenConfig.scala
@@ -494,7 +494,6 @@ object GlutenConfig extends ConfigRegistry {
SPARK_S3_CONNECTION_MAXIMUM,
SPARK_S3_ENDPOINT_REGION,
SPARK_S3_AWS_IMDS_ENABLED,
- "spark.gluten.velox.fs.s3a.connect.timeout",
"spark.gluten.velox.fs.s3a.retry.mode",
"spark.gluten.velox.awsSdkLogLevel",
"spark.gluten.velox.s3UseProxyFromEnv",
@@ -604,7 +603,6 @@ object GlutenConfig extends ConfigRegistry {
(SPARK_S3_USE_INSTANCE_CREDENTIALS, "false"),
(SPARK_S3_RETRY_MAX_ATTEMPTS, "20"),
(SPARK_S3_CONNECTION_MAXIMUM, "15"),
- ("spark.gluten.velox.fs.s3a.connect.timeout", "200s"),
("spark.gluten.velox.fs.s3a.retry.mode", "legacy"),
(
"spark.gluten.sql.columnar.backend.velox.IOThreads",
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
