Re: WAL size estimation

2019-06-26 Thread Todd Lipcon
Hey Pavel, I went back and looked at the source here. It appears that 24MB is the expected size for an index file -- each entry is 24 bytes and the index file should keep 1M entries. That said, for a "cold tablet" (in which you'd have only a small number of actual WAL files) I would expect only

Re: WAL size estimation

2019-06-19 Thread Pavel Martynov
> Try adding the '-p' flag here? That should show preallocated extents. Would be interesting to run it on some index file which is larger than 1MB, for example. # du -h --apparent-size index.00108 23M index.00108 # du -h index.00108 23M index.00108 # xfs_bmap -v -p

Re: WAL size estimation

2019-06-19 Thread Todd Lipcon
On Wed, Jun 19, 2019 at 12:49 AM Pavel Martynov wrote: > Hi Todd, thanks for the answer! > > > Any chance you've done something like copy the files away and back that > might cause them to lose their sparseness? > > No, I don't think so. Recently we experienced some problems with stability >

Re: WAL size estimation

2019-06-19 Thread Pavel Martynov
Hi Todd, thanks for the answer! > Any chance you've done something like copy the files away and back that might cause them to lose their sparseness? No, I don't think so. Recently we experienced some problems with stability with Kudu, and ran rebalance a couple of times, if this related. But we

WAL size estimation

2019-06-19 Thread Pavel Martynov
Hi guys! We want to buy SSDs for TServers WALs for our cluster. I'm working on capacity estimation for this SSDs using "Getting Started with Kudu" book, Chapter 4, Write-Ahead Log ( https://www.oreilly.com/library/view/getting-started-with/9781491980248/ch04.html