[ 
https://issues.apache.org/jira/browse/KAFKA-70?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13102703#comment-13102703
 ] 

Jay Kreps commented on KAFKA-70:
--------------------------------

Actually, now that I think about it, not sure if just limiting the config to 
the size of one segment is enough. Intuitively here is what I think we want the 
config to mean: "Keep the least amount of data possible to guarantee a buffer 
for the consumer of at least N bytes". If we don't give that guarantee of that 
form there is no way to reason about the SLA for the consumer to pick up their 
data. For example if retention_size is 50*1024*1024+10 and segment size is 
5050*1024*1024, then the log may actually be trimmed as small as 10 bytes.

So Jun, I think the config approach doesn't work. I think we need to tweak the 
delete rule slightly.

> Introduce retention setting that depends on space
> -------------------------------------------------
>
>                 Key: KAFKA-70
>                 URL: https://issues.apache.org/jira/browse/KAFKA-70
>             Project: Kafka
>          Issue Type: New Feature
>    Affects Versions: 0.8
>              Labels: log, roll
>         Attachments: KAFKA-70.patch
>
>
> Currently there is this setting: 
> log.retention.hours 
> introduce: 
> log.retention.size 
> Semantic would be, either size is reached or time is reached, oldest messages 
> will be deleted. 
> This does not break back-ward compatibility and would make the system robust 
> under scenarios where message size is not deterministic over time.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Reply via email to