VartulAgrawal commented on a change in pull request #1177: SAMZA-2339 : Update
KafkaSystemAdmin to adhere to the checkpoint replication factor
URL: https://github.com/apache/samza/pull/1177#discussion_r331292767
##########
File path:
samza-kafka/src/main/java/org/apache/samza/system/kafka/KafkaSystemAdmin.java
##########
@@ -540,6 +540,9 @@ public KafkaStreamSpec toKafkaSpec(StreamSpec spec) {
kafkaSpec =
new KafkaStreamSpec(spec.getId(), spec.getPhysicalName(),
systemName, 1, coordinatorStreamReplicationFactor,
coordinatorStreamProperties);
+ } else if (spec.isCheckpointStream()) {
+ kafkaSpec =
KafkaStreamSpec.fromSpec(StreamSpec.createCheckpointStreamSpec(spec.getPhysicalName(),
systemName))
+ .copyWithReplicationFactor(Integer.parseInt(new
KafkaConfig(config).getCheckpointReplicationFactor().get()));
Review comment:
The KafkaCheckpointManagerFactory is only created after the KafkaSystemAdmin
is initialised so by the time KafkaCheckpointManagerFactory is called the
replicationFactor is already reset to 2 (default for any general topic) and
hence it needs to be based to the spec here. Tested this in our current setup
too :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
