zhouyuan commented on code in PR #12367:
URL: https://github.com/apache/gluten/pull/12367#discussion_r3490702781
##########
backends-velox/src/main/scala/org/apache/gluten/backendsapi/velox/VeloxIteratorApi.scala:
##########
@@ -199,7 +228,16 @@ class VeloxIteratorApi extends IteratorApi with Logging {
iter => new ColumnarBatchInIterator(BackendsApiManager.getBackendName,
iter.asJava)
}
- val extraConf = Map(GlutenConfig.COLUMNAR_CUDF_ENABLED.key ->
enableCudf.toString).asJava
+ // Merge the fs.* keys captured on the driver (stored in
GlutenPartition.fsConf)
+ // into the extraConf passed to NativePlanEvaluator / VeloxRuntime.
+ // Runtimes.contextInstance() will call
GlutenConfig.getNativeSessionConf() which
+ // merges extraConf on top of SQLConf.get.getAllConfs. Because the
executor-side
+ // SQLConf never receives "fs.*" keys (Spark only propagates "spark.*"
keys via
+ // task local properties), this is the only path these credentials can
take to
+ // reach the native session config and ultimately the Velox ABFS connector.
+ val partitionFsConf = inputPartition.asInstanceOf[GlutenPartition].fsConf
+ val extraConf = (partitionFsConf +
+ (GlutenConfig.COLUMNAR_CUDF_ENABLED.key -> enableCudf.toString)).asJava
val transKernel =
NativePlanEvaluator.create(BackendsApiManager.getBackendName, extraConf)
Review Comment:
fixed by adding a digest on the key/password
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
