Yohahaha commented on code in PR #11083:
URL:
https://github.com/apache/incubator-gluten/pull/11083#discussion_r2544070346
##########
backends-velox/src/main/scala/org/apache/spark/shuffle/ColumnarShuffleWriter.scala:
##########
@@ -203,7 +205,7 @@ class ColumnarShuffleWriter[K, V](
val columnarBatchHandle =
ColumnarBatches.getNativeHandle(BackendsApiManager.getBackendName,
cb)
val startTime = System.nanoTime()
- shuffleWriterJniWrapper.write(
+ val curBytesWritten = shuffleWriterJniWrapper.write(
Review Comment:
after this pr,
`Java_org_apache_gluten_vectorized_ShuffleWriterJniWrapper_write` will return
the bytes we written to shuffle service during this call which was compressed
and accurate.
`batch->numBytes` is not accurate right? and we did not use it in java side
before, so this pr wants to use the correct return value to enhance
observability.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
