turcsanyip commented on PR #10538: URL: https://github.com/apache/nifi/pull/10538#issuecomment-3714697148
Thanks for sharing your thoughts and concerns @exceptionfactory. I agree that `ConsumerPartitionsUtil` is complex. However, it is worth noting that it was directly reused from the NiFi 1.x implementation where it was a single commit that worked properly. So I would not consider it legacy code that was getting more and more complicated over time. I also reviewed it in this round and I did not find how it could be simplified / changed. This brings us to your second concern about portability. The partitions need to be bound to a NiFi node and hostnames seem the only way to do it. So even if a separate processor were added, it would still have the portability issue. Considering that the change does not affect the existing users of the processor (no config change, portability issue does not arise if the feature is not used), I do not think it would be worth adding a new processor that requires duplicated code leading to additional maintenance effort and issues going forward. As far as I can recall, the original plan of migrating to Kafka 3 processors was to implement the core functionality in the first round and add the extra features later. Based on this, and considering that a separate implementation would also introduce complexity and non-portability issues via `ConsumerPartitionsUtil`, I believe we could add this feature to the current `ConsumeKafka` processor. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
