leesf commented on a change in pull request #1039: [HUDI-340]: made max events to read from kafka source configurable URL: https://github.com/apache/incubator-hudi/pull/1039#discussion_r350031504
########## File path: hudi-utilities/src/main/java/org/apache/hudi/utilities/deltastreamer/SourceFormatAdapter.java ########## @@ -126,4 +127,13 @@ public SourceFormatAdapter(Source source) { throw new IllegalArgumentException("Unknown source type (" + source.getSourceType() + ")"); } } + + /** + * This method is needed to set Config.DEFAULT_MAX_EVENTS_FROM_KAFKA_SOURCE to some lower value for test cases, + * gets called only from TestKafkaSource.java class + * @param maxEventsFromKafkaSource + */ + public void setDefaultMaxEventsFromKafkaSource(long maxEventsFromKafkaSource) { Review comment: > @leesf regarding your first point, basically I wanted to show a scenario where sourceLimit and Config.MAX_EVENTS_FROM_KAFKA_SOURCE_PROP are set to Long.MAX_VALUE and default value comes into play when the number of events in kafka topic is greater than that default value (i.e greater than 5M in normal case). That would require consuming those many events from kafka topic, hence I added the function to reset DEFAULT_MAX_EVENTS_FROM_KAFKA_SOURCE. IMHO, with lesser number of events in topic, this scenario is not depicted properly. WDYT? Make sense. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services