mynameborat commented on code in PR #1662:
URL: https://github.com/apache/samza/pull/1662#discussion_r1227250936
##########
samza-azure/src/main/java/org/apache/samza/system/azureblob/AzureBlobConfig.java:
##########
@@ -80,6 +80,12 @@ public class AzureBlobConfig extends MapConfig {
public static final String SYSTEM_MAX_FLUSH_THRESHOLD_SIZE =
SYSTEM_AZUREBLOB_PREFIX + "maxFlushThresholdSize";
private static final int SYSTEM_MAX_FLUSH_THRESHOLD_SIZE_DEFAULT = 10485760;
+ // initialization size of in-memory OutputStream
+ // This value should be between SYSTEM_INIT_BUFFER_SIZE_DEFAULT and
getMaxFlushThresholdSize() exclusive.
+ public static final String SYSTEM_INIT_BUFFER_SIZE = SYSTEM_AZUREBLOB_PREFIX
+ "initBufferSize.bytes";
+ // re-use size for parameterless constructor java.io.ByteArrayOutputStream()
+ public static final int SYSTEM_INIT_BUFFER_SIZE_DEFAULT = 32;
Review Comment:
Can you provide some context through comments here on why and how this value
is chosen and what its impact is?
E.g., what was the previous default provided through config or defaulted
somehow (through code) that caused issues so that folks can be mindful about
making changes to these defaults in future.
##########
samza-azure/src/main/java/org/apache/samza/system/azureblob/avro/AzureBlobAvroWriter.java:
##########
@@ -108,19 +109,32 @@ public class AzureBlobAvroWriter implements
AzureBlobWriter {
private final String blobURLPrefix;
private final long maxBlobSize;
private final long maxRecordsPerBlob;
+ private final int initBufferSize;
private final boolean useRandomStringInBlobName;
private final Object currentDataFileWriterLock = new Object();
private volatile long recordsInCurrentBlob = 0;
private BlobMetadataGeneratorFactory blobMetadataGeneratorFactory;
private Config blobMetadataGeneratorConfig;
private String streamName;
+ @Deprecated
Review Comment:
Is this constructor used somewhere else other than the factory? If not, I'd
go ahead and remove this code since the factory handles the logic on passing in
the buffer size.
The contract for this class is clear that going forward buffer size needs to
be provided.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
