Copilot commented on code in PR #12163:
URL: https://github.com/apache/gluten/pull/12163#discussion_r3439975339
##########
gluten-substrait/src/main/scala/org/apache/gluten/execution/JoinExecTransformer.scala:
##########
@@ -105,6 +107,15 @@ trait HashJoinLikeExecTransformer extends BaseJoinExec
with TransformSupport {
(right, left)
}
+ protected def canonicalBuildHashTableId(plan: SparkPlan): String = plan
match {
+ case b: BroadcastQueryStageExec =>
+ canonicalBuildHashTableId(b.plan)
+ case r: ReusedExchangeExec =>
+ canonicalBuildHashTableId(r.child)
+ case other =>
+ other.id.toString
+ }
Review Comment:
`canonicalBuildHashTableId` currently returns the raw SparkPlan numeric `id`
as the hash-table/cache key. On the native side, this ID is also used as a
Velox `PlanNodeId` (see BHJ path in `SubstraitToVeloxPlan.cc`), while
`nextPlanNodeId()` generates numeric string IDs as well. Using plain numbers
makes collisions very likely, which can corrupt split routing and/or plan
metrics. Prefix the canonical ID with a stable string (similar to the old
`BuiltHashTable-` prefix) to keep it distinct from auto-generated node IDs.
##########
ep/build-velox/src/get-velox.sh:
##########
@@ -17,9 +17,9 @@
set -exu
CURRENT_DIR=$(cd "$(dirname "$BASH_SOURCE")"; pwd)
-VELOX_REPO=https://github.com/IBM/velox.git
-VELOX_BRANCH=dft-2026_06_12
-VELOX_ENHANCED_BRANCH=ibm-2026_06_12
+VELOX_REPO=https://github.com/JkSelf/velox.git
+VELOX_BRANCH=dft-2026_06_12-hashtable-cache
+VELOX_ENHANCED_BRANCH=ibm-2026_06_12-hashtable-cache
Review Comment:
Defaulting the Velox dependency to a personal fork (`JkSelf/velox`) makes
builds non-reproducible and introduces supply-chain / operational risk for
anyone building Gluten without custom flags. Please keep the default pinned to
an official / org-owned repository (e.g., `IBM/velox` or
`facebookincubator/velox`) and apply the HashTableCache changes via a pinned
commit/branch in that repo (or via `UPSTREAM_VELOX_PR_ID`/patching), while
still allowing personal forks only through `--velox_repo` for local testing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
