FMX commented on code in PR #3377:
URL: https://github.com/apache/celeborn/pull/3377#discussion_r2224691344
##########
worker/src/main/scala/org/apache/celeborn/service/deploy/worker/FetchHandler.scala:
##########
@@ -580,19 +580,23 @@ class FetchHandler(
streamChunkSlice.chunkIndex,
streamChunkSlice.offset,
streamChunkSlice.len)
+ val bufSize = buf.size()
+ workerSource.updateHistogram(WorkerSource.FETCH_CHUNK_SIZE, bufSize)
chunkStreamManager.chunkBeingSent(streamChunkSlice.streamId)
client.getChannel.writeAndFlush(new ChunkFetchSuccess(streamChunkSlice,
buf))
.addListener(new GenericFutureListener[Future[_ >: Void]] {
override def operationComplete(future: Future[_ >: Void]): Unit = {
if (future.isSuccess) {
if (log.isDebugEnabled) {
logDebug(
- s"Sending ChunkFetchSuccess to $remoteAddr succeeded, chunk
$streamChunkSlice")
+ s"Sending ChunkFetchSuccess to $remoteAddr succeeded," +
+ s" chunk $streamChunkSlice, buf size: $bufSize")
}
workerSource.incCounter(WorkerSource.FETCH_CHUNK_SUCCESS_COUNT)
} else {
logWarning(
- s"Sending ChunkFetchSuccess to $remoteAddr failed, chunk
$streamChunkSlice",
+ s"Sending ChunkFetchSuccess to $remoteAddr failed," +
+ s" chunk $streamChunkSlice, buf size: $bufSize",
Review Comment:
It's okay to add a new metric here because, if you have enabled debug-level
logs, you'll receive a large number of logs.
Simply logging the size of a chunk buffer has no meaning.
As you have mentioned, the length of streamChunkSlice is always
Int.MAX_VALUE, we can remove these two fields someday.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
