lakshmi-manasa-g commented on a change in pull request #1364:
URL: https://github.com/apache/samza/pull/1364#discussion_r427782229
##########
File path:
samza-azure/src/main/java/org/apache/samza/system/azureblob/utils/BlobMetadataContext.java
##########
@@ -25,10 +25,12 @@
public class BlobMetadataContext {
private final String streamName;
private final long blobSize;
+ private final long numberOfMessagesInBlob;
- public BlobMetadataContext(String streamName, long blobSize) {
+ public BlobMetadataContext(String streamName, long blobSize, long
numberOfMessagesInBlob) {
this.streamName = streamName;
this.blobSize = blobSize;
+ this.numberOfMessagesInBlob = numberOfMessagesInBlob;
Review comment:
This value will be given to the metadata generator Impl passed in by the
user of the SystemProducer through a config
(https://github.com/apache/samza/pull/1358).
So along with streamName, blobSize, this numberOfMessagesInBlob is
anticipated to be used to build metadata properties of the blob created.
Metadata proprties are kv pairs attached to the blob that can then be leveraged
for kusto ingestion. So I expect the usage to be something like
<stream:streamName, rawSize: blobSize, records: numberofMessages>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
