mynameborat commented on a change in pull request #951: SAMZA-2127: Upgrade to Kafka 2.0 URL: https://github.com/apache/samza/pull/951#discussion_r268891069
########## File path: samza-kafka/src/main/java/org/apache/samza/system/kafka/KafkaSystemAdmin.java ########## @@ -628,10 +589,12 @@ public void validateStream(StreamSpec streamSpec) throws StreamValidationExcepti @Override public void deleteMessages(Map<SystemStreamPartition, String> offsets) { if (deleteCommittedMessages) { - if (adminClientForDelete == null) { - adminClientForDelete = kafka.admin.AdminClient.create(createAdminClientProperties()); - } - KafkaSystemAdminUtilsScala.deleteMessages(adminClientForDelete, offsets); + Map<TopicPartition, RecordsToDelete> recordsToDelete = offsets.entrySet() + .stream() + .collect(Collectors.toMap(entry -> + new TopicPartition(entry.getKey().getStream(), entry.getKey().getPartition().getPartitionId()), + entry -> RecordsToDelete.beforeOffset(Long.parseLong(entry.getValue()) + 1))); + adminClient.deleteRecords(recordsToDelete); Review comment: Done 1 & 2 3. We have been using AdminClient for all flows except deletion path. We don't have any of the backoff configurations configurable for the other flows and I plan to keep it the same for deletion too. We can follow up if there is a need for it in a separate PR ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services