bkonold commented on a change in pull request #1301: Samza-2478: Add new
metrics to track key and value size of records written to RocksDb
URL: https://github.com/apache/samza/pull/1301#discussion_r388770032
##########
File path:
samza-kv/src/main/scala/org/apache/samza/storage/kv/SerializedKeyValueStore.scala
##########
@@ -62,7 +62,10 @@ class SerializedKeyValueStore[K, V](
val keyBytes = toBytesOrNull(key, keySerde)
val valBytes = toBytesOrNull(value, msgSerde)
store.put(keyBytes, valBytes)
+ val keySizeBytes = if (keyBytes == null) 0 else keyBytes.length
val valSizeBytes = if (valBytes == null) 0 else valBytes.length
+ metrics.recordKeySizeBytes.update(keySizeBytes)
+ metrics.recordValueSizeBytes.update(valSizeBytes)
updatePutMetrics(1, valSizeBytes)
Review comment:
Is there a way we can roll together recording maximum record size into the
histogram tracking? When I implemented the metric for recording max I had
considered modifying histogram to record maximum in addition to percentiles,
but decided against it since the memory footprint of the histogram was
unjustifiable at the time. Now that we'd like those metrics anyway, maybe it is
justifiable now.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
