With zfs I get very good compression ratios: canoa:~# zfs get all zfs/couchdb NAME PROPERTY VALUE SOURCE zfs/couchdb type filesystem - zfs/couchdb creation Tue Aug 3 16:52 2010 - zfs/couchdb used 53.1G - zfs/couchdb available 36.0G - zfs/couchdb referenced 53.1G - zfs/couchdb compressratio 5.87x - zfs/couchdb mounted yes - zfs/couchdb quota none default zfs/couchdb reservation none default zfs/couchdb recordsize 128K default zfs/couchdb mountpoint /zfs/couchdb default zfs/couchdb sharenfs off default zfs/couchdb checksum on default zfs/couchdb compression gzip inherited from zfs zfs/couchdb atime off inherited from zfs zfs/couchdb devices on default ......
That's 5.87x using gzip (level 6 which is the default). I wouldn't be able to use my rather small (128Gb) solid state disks without it :-) Mike On Sun, 2010-09-19 at 10:42 +0300, Metin Akat wrote: > Hmm, then I start thinking that btrfs is really better than ext4, > especially if I want to do things like copy the files in order to > deploy (fast) another instance with already build view indexes. > > And would the couchdb files compress well? > > On Sat, Sep 18, 2010 at 12:23 PM, Wout Mertens <[email protected]> wrote: > > Everything except the 4KB blocksize I would say. While ZFS and Btrfs are > > very different on-disk, they are both Copy-On-Write filesystems with > > extents, compression and cheap snapshots. I don't know how the 4KB > > blocksize settings translates onto Btrfs. > > > > They are both really suited for the append-only workload CouchDB presents. > > > > Wout. > > > > On Sep 18, 2010, at 9:49 , Metin Akat wrote: > > > >> What part of this blog post is relevant to btrfs? > >> > >> On Fri, Sep 17, 2010 at 10:22 PM, Chris Anderson <[email protected]> wrote: > >>> On Thu, Sep 16, 2010 at 6:22 PM, Tyler Gillies <[email protected]> wrote: > >>>> Wow, thanks for the thought out writeup! > >>>> > >>> > >>> here's a blog post http://letsgetdugg.com/2010/06/25/couchdb-on-zfs/ > >>> > >>>> On Thu, Sep 16, 2010 at 12:47 PM, Randall Leeds > >>>> <[email protected]>wrote: > >>>> > >>>>> Disclaimer: I'm no file systems expert. > >>>>> > >>>>> I recommend something with extents otherwise you might take a big > >>>>> performance hit while couch deletes old db files after compaction. > >>>>> Compression sounds cool as long as you can do it really fast (are > >>>>> there setups where this happens in hardware?). > >>>>> > >>>>> reiserfs: > >>>>> According to wikipedia it "still uses the big kernel lock (BKL) — a > >>>>> global kernel-wide lock" which makes performance on multiple cores > >>>>> suffer. > >>>>> It's big benefit, as I always understood it, is being able to pack > >>>>> smile files together into single blocks. You will likely not have lots > >>>>> of small files with Couch :-P > >>>>> > >>>>> xfs: > >>>>> Delayed allocation might be a big performance win with a Couch. Since > >>>>> outstanding writes are committed together in chunks and then fsync'd > >>>>> all together I bet this feature would do good things for Couch > >>>>> performance. > >>>>> > >>>>> ext(3|4) > >>>>> I'd recommend ext4 over ext3. Delayed allocation like xfs as well as > >>>>> the multiblock allocator should make it much better than ext3. You > >>>>> also get extents. > >>>>> > >>>>> btrfs/zfs: > >>>>> Some of the features of each sound interesting, but nothing that > >>>>> stands out to me as "great for CouchDB". Snapshots and backups are > >>>>> cool, but Couch is doing this for you already in a sense due to the > >>>>> way the btree is appended: CouchDB documents are, in a sense, > >>>>> copy-on-write. Checksumming is cool if you think it's important for > >>>>> your data integrity. If you want snapshots for backup you can always > >>>>> use CouchDB replication. > >>>>> > >>>>> If you run any tests I'd be very, very interested in seeing your > >>>>> results. > >>>>> > >>>>> -Randall > >>>>> > >>>>> On Thu, Sep 16, 2010 at 03:11, Metin Akat <[email protected]> wrote: > >>>>>> I'm sure almost everybody out there is using ext4/3 (including me), > >>>>>> but what about filesystems like btrfs, zfs, reiserfs, xfs. Some of > >>>>>> them have very appealing feature-sets (like compression for example, > >>>>>> and we all know how greedy is couchdb for disk space). > >>>>>> And I know that for example btrfs is not yet "recommended for > >>>>>> production". But its time is coming. From what I see, Ubuntu 10.10 > >>>>>> works flawlessly on btrfs. > >>>>>> So I'd be happy if we have some discussion on the topic, instead of > >>>>>> "everybody uses ext4, just use it" kind of stuff :). > >>>>>> Couchdb was "alpha software" for years, and we all used it in > >>>>>> production, so we are not afraid of alpha/beta software, as long as > >>>>>> it's good :) > >>>>> > >>>> > >>>> > >>>> > >>>> -- > >>>> http://www.readwriteweb.com/about#tyler > >>>> > >>>> Ask me anything <http://tumble.pdxbrain.com/ask>! > >>>> > >>> > >>> > >>> > >>> -- > >>> Chris Anderson > >>> http://jchrisa.net > >>> http://couch.io > >>> > > > > >
