[
https://issues.apache.org/jira/browse/HUDI-340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17141560#comment-17141560
]
Pratyaksh Sharma commented on HUDI-340:
---------------------------------------
[~wangxianghu] I agree with Vinoth on this. We have already capped the value at
5M. If some user sets sourceLimit to a really huge value, then we can safely
assume he is purposely doing that and he has a strong reasoning behind setting
a real huge value for source limit. :)
> Increase Default max events to read from kafka source
> -----------------------------------------------------
>
> Key: HUDI-340
> URL: https://issues.apache.org/jira/browse/HUDI-340
> Project: Apache Hudi
> Issue Type: Improvement
> Components: DeltaStreamer
> Reporter: Pratyaksh Sharma
> Assignee: Pratyaksh Sharma
> Priority: Major
> Labels: pull-request-available
> Fix For: 0.5.1
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Right now, DEFAULT_MAX_EVENTS_TO_READ is set to 1M in case of kafka source in
> KafkaOffsetGen.java class. DeltaStreamer can handle much more incoming
> records than this.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
