[ https://issues.apache.org/jira/browse/CASSANDRA-16325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17656998#comment-17656998 ]
David Capwell commented on CASSANDRA-16325: ------------------------------------------- bq. I think if we updated the ProgressInfo object to store the delta from the last time progress was called I just looked into this and it would require light caching to the callers, such as {code} diff --git src/java/org/apache/cassandra/db/streaming/CassandraCompressedStreamReader.java src/java/org/apache/cassandra/db/streaming/CassandraCompressedStreamReader.java index dda874ba40..b9f4566aac 100644 --- src/java/org/apache/cassandra/db/streaming/CassandraCompressedStreamReader.java +++ src/java/org/apache/cassandra/db/streaming/CassandraCompressedStreamReader.java @@ -79,6 +79,7 @@ public class CassandraCompressedStreamReader extends CassandraStreamReader writer = createWriter(cfs, totalSize, repairedAt, pendingRepair, format); String filename = writer.getFilename(); int sectionIdx = 0; + long read = 0; for (SSTableReader.PartitionPositionBounds section : sections) { assert cis.chunkBytesRead() <= totalSize; @@ -93,7 +94,10 @@ public class CassandraCompressedStreamReader extends CassandraStreamReader { writePartition(deserializer, writer); // when compressed, report total bytes of compressed chunks read since remoteFile.size is the sum of chunks transferred - session.progress(filename + '-' + fileSeqNum, ProgressInfo.Direction.IN, cis.chunkBytesRead(), totalSize); + long bytes = cis.chunkBytesRead(); + long delta = bytes - read; + read = bytes; + session.progress(filename + '-' + fileSeqNum, ProgressInfo.Direction.IN, bytes, delta, totalSize); } assert in.getBytesRead() == sectionLength; } {code} So, if the reason to avoid using a listener was tracking progress info, we can compute the delta at the event creation side (non look hard), I personally find that better than embedding your custom metrics listener as it gives benefits to other listeners (such as removing the cache in StreamingState) > Update streaming metrics incrementally > -------------------------------------- > > Key: CASSANDRA-16325 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16325 > Project: Cassandra > Issue Type: Improvement > Components: Observability/Metrics > Reporter: Paulo Motta > Assignee: Isaac Reath > Priority: Normal > Labels: lhf > Fix For: 4.2 > > Time Spent: 10h 10m > Remaining Estimate: 0h > > Currently the inbound and outbound streamed bytes metrics are incremented > after each file is streamed, what doesn't represent the current number of > bytes streamed since it can take a long time for a large file to be streamed. > We should update the metric incrementally as data is streamed. -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org