kamalcph commented on PR #20913: URL: https://github.com/apache/kafka/pull/20913#issuecomment-4402964631
@jiafu1115 Thanks for addressing the review comments. I think we need the -1 logic, but rephrase the config description. (ie) For `remote.copy.lag.ms` and `remote.copy.lag.bytes` * Set the value to -1 to have the maximum lag in uploading the segment. When set to -1, the lag is based on the local retention settings. Once the segment breaches either local retention time/size, then it is eligible for upload to the remote storage. * Set the value to 0 to copy the segments eagerly / immediately and there is no delay. This should be default value for both the configs to retain the existing behavior. * We can maintain the other validation that `remote.copy.lag.ms/bytes` <= `local.retention.ms/bytes`. Valid value range [-1, 0, ..., local_retention]. To handle this case: ``` retention.ms = 2 days local.retention.ms = 6 hour retention.bytes = -1 no limit by default local.retention.bytes = -2 no limit by default ``` User has to set both the `remote.copy.lag.bytes=-1` and `remote.copy.lag.ms=5.5 hrs` config. The idea is that the user will configure both `remote.copy.lag.bytes` and `remote.copy.lag.ms` configs based on their workload. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
