TL;DR broker replication should preserve file create time. My colleagues and I stumbled on an interesting issue in production. We use a time based retention policy instead of a size based policy and this normally works great (60 hours retention).
A few days ago we rebuilt a broker (os reinstall). The replication process took several hours and the files that were streamed in were technically 60 hours old, but all files received the current (wall clock) create time. After running for 2 days (wall time) we observed that this new broker was holding onto ~5 days of data. Even though it was in-sync to the leader, it had 35GB more of data. We scratched our heads for a few minutes and quickly came to the conclusion that time based retention must be using the create time of the file. To test our theory, we used "touch" to change the creation time of some files, and saw that they were deleted within 5 minutes. Feature request: When replicating data from another broker the create time should be preserved for all files. Is something like rsync an option? Best, Justin Venus
