Thanks for the update. How much smaller did the BF get to ?
A ----------------- Aaron Morton Freelance Developer @aaronmorton http://www.thelastpickle.com On 13/03/2012, at 8:24 AM, Mick Semb Wever wrote: > >>>>> It's my understanding then for this use case that bloom filters are of >>>>> little importance and that i can > > > Ok. To summarise our actions to get us out of this situation, in hope > that it may help others one day, we did the following actions: > > 1) upgrade to 1.0.7 > 2) set fp_ratio=0.99 > 3) set index_interval=4096 > 4) restarted the node with Xmx30G > 5) run `nodetool scrub` > and monitor total size of bf files > using `du -hc *-Filter.db | grep total` > 6) restart node with original Xmx setting once total bf size is under > (scrub was running for >12hrs) > (remaining bloom filters can be rebuilt later from normal compact) > > Hopefully it will also eventuate that this cluster can run with a more > normal Xmx4G rather than the previous Xmx12G. > > (2) and (3) are very much dependent on our set up using hadoop where all > reads are get_range_slice with 16k rows per request. Both could be tuned > correctly but they're the numbers that worked first up. > > ~mck > > -- > "When there is no enemy within, the enemies outside can't hurt you." > African proverb > > | http://github.com/finn-no | http://tech.finn.no |
