Weixin-Xu commented on code in PR #13492:
URL: https://github.com/apache/gluten/pull/13492#discussion_r4421627803
##########
backends-bolt/src/main/scala/org/apache/spark/api/python/ColumnarArrowEvalPythonExec.scala:
##########
@@ -368,44 +373,29 @@ case class ColumnarArrowEvalPythonExec(
val joinedVectors = (0 until inputCb.numCols).toArray.map(
i => inputCb.column(i)) ++ (0 until
outputCb.numCols).toArray.map(
i => outputCb.column(i))
- (0 until joinedVectors.length).foreach(
- i => {
-
adjustRefCnt(joinedVectors(i).asInstanceOf[ArrowWritableColumnVector], 1)
- })
val numRows = inputCb.numRows
numOutputBatches += 1
numOutputRows += numRows
val batch = new ColumnarBatch(joinedVectors, numRows)
Review Comment:
The joined result now uses independently owned input vectors, so its
reference counts no longer depend on when the upstream writer releases its
batch. Added regression coverage that retains the upstream input while joining
and offloading the result, then verifies that the result and upstream input can
be released independently.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
