Yohahaha commented on code in PR #11083:
URL:
https://github.com/apache/incubator-gluten/pull/11083#discussion_r2540187133
##########
backends-velox/src/main/scala/org/apache/spark/shuffle/ColumnarShuffleWriter.scala:
##########
@@ -203,7 +205,7 @@ class ColumnarShuffleWriter[K, V](
val columnarBatchHandle =
ColumnarBatches.getNativeHandle(BackendsApiManager.getBackendName,
cb)
val startTime = System.nanoTime()
- shuffleWriterJniWrapper.write(
+ val curBytesWritten = shuffleWriterJniWrapper.write(
Review Comment:
shuffle write bytes does not equals to batch size, so before logic was wrong.
> I wonder if this should be
it depends on where we count incremental bytes, currently jni call returns
total written bytes but spark only expose incremental interface.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
