Hello, One thing you need to check is that the file handler limit on your machine is set high enough for the number of files on disk plus sockets. Other than that, as long as you are consuming at the tail of the log most of the time it should be fine.
Guozhang On Wed, Nov 19, 2014 at 9:23 AM, Thunder Stumpges <[email protected]> wrote: > Hello, > > We have a 5 node cluster, each with a 5 disk JBOD storage running Kafka > 0.8.1. We are interested in the possibility of using the OffsetRequest API > taking a time to get offsets from a given time. Because that API can only > "find" times where the log segments roll, we would need to set > log.roll.hours =1 in order to have a useful granularity for that API. > > I am wondering what would be the drawbacks to this? I know it would create > many more files, but what affect would that have on the system(s)? Is there > a general rule of thumb for what would be "too many" files? > > -- -- Guozhang
