LuciferYang opened a new issue, #12621:
URL: https://github.com/apache/gluten/issues/12621
### Backend
VL (Velox) — the entry point is in `gluten-core`; the end-to-end failure
also reaches the Velox driver-start path.
### Bug description
`GlutenDriverPlugin.setPredefinedConfigs` reads the off-heap size in its
non-dynamic-sizing branch with the single-arg
`conf.getSizeAsBytes(SPARK_OFFHEAP_SIZE_KEY)`. `checkOffHeapSettings` returns
early for untracked memory mode (`spark.gluten.memory.untracked=true`) and
skips the requirement that the size be set, so in that mode the key can be
absent. `SparkConf.get(String)` throws `NoSuchElementException` on a missing
key (it reads the raw settings map and does not consult the `ConfigEntry`
default), so driver plugin init crashes instead of proceeding under untracked
semantics.
The same single-arg read exists on the Velox driver-start path:
`VeloxListenerApi.onDriverStart` also calls
`conf.getSizeAsBytes(SPARK_OFFHEAP_SIZE_KEY)`, and it runs after
`setPredefinedConfigs` (via `GlutenDriverPlugin.init` →
`Component.onDriverStart`). The dynamic-sizing branch already shields that
reader by doing `conf.set(SPARK_OFFHEAP_SIZE_KEY, "0")`; untracked mode did
not, so it was the one unshielded path.
The fix normalizes the key in the untracked branch: when
`spark.memory.offHeap.size` is absent, set it to `0`, mirroring the
dynamic-sizing branch. One change covers both readers (the plugin and
`VeloxListenerApi`), and normal mode is unaffected because
`checkOffHeapSettings` already guarantees the key is set there.
### Gluten version
main (1.8.0-SNAPSHOT)
### Spark version
Version-agnostic (applies to spark-3.3 / 3.4 / 3.5 / 4.0 / 4.1).
### Spark configurations
- `spark.plugins=org.apache.gluten.GlutenPlugin`
- `spark.gluten.memory.untracked=true`
- `spark.memory.offHeap.size` not set (dynamic off-heap sizing left at its
default of false).
### System information
N/A — logic issue in
`gluten-core/src/main/scala/org/apache/gluten/GlutenPlugin.scala`, independent
of OS/hardware.
### Relevant logs
```text
java.util.NoSuchElementException: spark.memory.offHeap.size
at org.apache.spark.SparkConf.$anonfun$get$1(SparkConf.scala:...)
at org.apache.spark.SparkConf.getSizeAsBytes(SparkConf.scala:...)
at
org.apache.gluten.GlutenDriverPlugin$.setPredefinedConfigs(GlutenPlugin.scala:141)
at org.apache.gluten.GlutenDriverPlugin.init(GlutenPlugin.scala:61)
```
### Fix direction
In the untracked/non-dynamic branch, set `spark.memory.offHeap.size` to `0`
when it is absent, so neither `setPredefinedConfigs` nor
`VeloxListenerApi.onDriverStart` hits `NoSuchElementException`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
