[lxc-users] lxd: -B backingstore equivalent?
Is there a -B btrfs equivalent in lxd? For example, with lxc, I would use: # lxc-create --template download --name test-container -B btrfs -B backingstore 'backingstore' is one of 'dir', 'lvm', 'loop', 'btrfs', 'zfs', or 'best'. The default is 'dir', meaning that the container root filesystem will be a directory under /var/lib/lxc/container/rootfs. How can I do the same with lxd (lxc command)? It seems to default to dir. # lxc launch images:ubuntu/trusty/amd64 test-container -- Tomasz Chmielewski http://wpkg.org ___ lxc-users mailing list lxc-users@lists.linuxcontainers.org http://lists.linuxcontainers.org/listinfo/lxc-users
Re: [lxc-users] lxd: -B backingstore equivalent?
Hi Tomasz, On Fri, Jun 05, 2015 at 07:22:25PM +0900, Tomasz Chmielewski wrote: Is there a -B btrfs equivalent in lxd? Yes, if you mount /var/lib/lxd as a btrfs subvolume, it should Just Work. For example, with lxc, I would use: # lxc-create --template download --name test-container -B btrfs -B backingstore 'backingstore' is one of 'dir', 'lvm', 'loop', 'btrfs', 'zfs', or 'best'. The default is 'dir', meaning that the container root filesystem will be a directory under /var/lib/lxc/container/rootfs. How can I do the same with lxd (lxc command)? It seems to default to dir. LVM support is Coming Soon, and making it fast and stable will likely be a primary focus. Tycho # lxc launch images:ubuntu/trusty/amd64 test-container -- Tomasz Chmielewski http://wpkg.org ___ lxc-users mailing list lxc-users@lists.linuxcontainers.org http://lists.linuxcontainers.org/listinfo/lxc-users ___ lxc-users mailing list lxc-users@lists.linuxcontainers.org http://lists.linuxcontainers.org/listinfo/lxc-users
Re: [lxc-users] lxd: -B backingstore equivalent?
Lxd will not be as flexible as lxc in many ways, including with respect to backing stores. What will become the recommended backing store is actualy not yet implemented, but will be soon - that is lvm with thinpools. You'll be able to either provide a vg with the default name, or specify a custom vg name, for all of lxd to use, and lxd will use lvm snapshots for quick container launches. In lieu of that, you currently can use btrfs - just make sure that /var/lib/lxd is on a btrfs. The first launch of a container from a new image will create a btrfs cache directory, and each launch from that image will be done using a btrfs snapshot. -serge Quoting Tomasz Chmielewski (man...@wpkg.org): Is there a -B btrfs equivalent in lxd? For example, with lxc, I would use: # lxc-create --template download --name test-container -B btrfs -B backingstore 'backingstore' is one of 'dir', 'lvm', 'loop', 'btrfs', 'zfs', or 'best'. The default is 'dir', meaning that the container root filesystem will be a directory under /var/lib/lxc/container/rootfs. How can I do the same with lxd (lxc command)? It seems to default to dir. # lxc launch images:ubuntu/trusty/amd64 test-container -- Tomasz Chmielewski http://wpkg.org ___ lxc-users mailing list lxc-users@lists.linuxcontainers.org http://lists.linuxcontainers.org/listinfo/lxc-users ___ lxc-users mailing list lxc-users@lists.linuxcontainers.org http://lists.linuxcontainers.org/listinfo/lxc-users
Re: [lxc-users] lxd: -B backingstore equivalent?
Quoting Mark Constable (ma...@renta.net): On Fri, 5 Jun 2015 07:58:02 AM Tycho Andersen wrote: LVM support is Coming Soon, and making it fast and stable will likely be a primary focus. On Fri, 5 Jun 2015 01:57:34 PM Serge Hallyn wrote: What will become the recommended backing store is actualy not yet implemented, but will be soon - that is lvm with thinpools. Does this mean that btrfs is considered a second class option with the primary focus and most of your future lxd backing store effort being put into LVM? It is, for a few reasons. I don't think the support will ever go away, but lvm is going to be the preferred backing store. If so then what would be the reason to abandon btrfs as a preferred backing store (if it ever was)? I think so. -serge ___ lxc-users mailing list lxc-users@lists.linuxcontainers.org http://lists.linuxcontainers.org/listinfo/lxc-users
Re: [lxc-users] lxd: -B backingstore equivalent?
On Sat, Jun 06, 2015 at 12:32:07AM +0900, Tomasz Chmielewski wrote: On 2015-06-06 00:19, Tycho Andersen wrote: # ls -l /var/lib/lxd lrwxrwxrwx 1 root root 8 Jun 5 10:15 /var/lib/lxd - /srv/lxd Ah, my best guess is that lxd doesn't follow the symlink correctly when detecting filesystems. Whatever the cause, if you file a bug we'll fix it, thanks. Can you point me to the bug filing system for linuxcontainers.org? We use github for lxc/lxd, so https://github.com/lxc/lxd/issues/new Tycho ___ lxc-users mailing list lxc-users@lists.linuxcontainers.org http://lists.linuxcontainers.org/listinfo/lxc-users
Re: [lxc-users] lxd: -B backingstore equivalent?
On 2015-06-06 00:19, Tycho Andersen wrote: # ls -l /var/lib/lxd lrwxrwxrwx 1 root root 8 Jun 5 10:15 /var/lib/lxd - /srv/lxd Ah, my best guess is that lxd doesn't follow the symlink correctly when detecting filesystems. Whatever the cause, if you file a bug we'll fix it, thanks. Can you point me to the bug filing system for linuxcontainers.org? The closest to contributing seems to be here: https://linuxcontainers.org/lxd/contribute/ but don't see any report an bug, issue tracker or anything similar. -- Tomasz Chmielewski http://wpkg.org ___ lxc-users mailing list lxc-users@lists.linuxcontainers.org http://lists.linuxcontainers.org/listinfo/lxc-users
Re: [lxc-users] lxd: -B backingstore equivalent?
On 2015-06-06 00:00, Tycho Andersen wrote: As I've checked, this is not the case (the container is created in a directory, not in btrfs subvolume; lxc-create -B btrfs creates it in a subvolume). Can you file a bug with info to reproduce? It should work as of 0.8. Before I file a bug report - that's how it works for me - /var/lib/lxd/ is a symbolic link to /srv/lxd, placed on a btrfs filesystem: # ls -l /var/lib/lxd lrwxrwxrwx 1 root root 8 Jun 5 10:15 /var/lib/lxd - /srv/lxd # mount|grep /srv /dev/sda4 on /srv type btrfs (rw,noatime,device=/dev/sda4,device=/dev/sdb4,compress=zlib) # lxc launch images:ubuntu/trusty/amd64 test-image Creating container...done Starting container...done error: exit status 1 Note that it errored when trying to start the container - I have to add lxc.aa_allow_incomplete = 1; otherwise, it won't start (is there some /etc/lxc/default.conf equivalent for lxd, where this could be set?). However, the container is already created in a directory, so I don't think the above error matters: # btrfs sub list /srv|grep lxd # btrfs sub list /srv|grep test-image # -- Tomasz Chmielewski http://wpkg.org ___ lxc-users mailing list lxc-users@lists.linuxcontainers.org http://lists.linuxcontainers.org/listinfo/lxc-users
Re: [lxc-users] lxd: -B backingstore equivalent?
On Sat, Jun 06, 2015 at 12:11:27AM +0900, Tomasz Chmielewski wrote: On 2015-06-06 00:00, Tycho Andersen wrote: As I've checked, this is not the case (the container is created in a directory, not in btrfs subvolume; lxc-create -B btrfs creates it in a subvolume). Can you file a bug with info to reproduce? It should work as of 0.8. Before I file a bug report - that's how it works for me - /var/lib/lxd/ is a symbolic link to /srv/lxd, placed on a btrfs filesystem: # ls -l /var/lib/lxd lrwxrwxrwx 1 root root 8 Jun 5 10:15 /var/lib/lxd - /srv/lxd Ah, my best guess is that lxd doesn't follow the symlink correctly when detecting filesystems. Whatever the cause, if you file a bug we'll fix it, thanks. # mount|grep /srv /dev/sda4 on /srv type btrfs (rw,noatime,device=/dev/sda4,device=/dev/sdb4,compress=zlib) # lxc launch images:ubuntu/trusty/amd64 test-image Creating container...done Starting container...done error: exit status 1 Note that it errored when trying to start the container - I have to add lxc.aa_allow_incomplete = 1; otherwise, it won't start (is there some /etc/lxc/default.conf equivalent for lxd, where this could be set?). Yes, there is a default profile that is applied if you don't specify one, you can edit it with: lxc profile edit default Tycho However, the container is already created in a directory, so I don't think the above error matters: # btrfs sub list /srv|grep lxd # btrfs sub list /srv|grep test-image # -- Tomasz Chmielewski http://wpkg.org ___ lxc-users mailing list lxc-users@lists.linuxcontainers.org http://lists.linuxcontainers.org/listinfo/lxc-users
Re: [lxc-users] lxd: -B backingstore equivalent?
On Fri, 5 Jun 2015 05:05:21 PM Serge Hallyn wrote: Does this mean that btrfs is considered a second class option It is, for a few reasons. Sorry to persist with this but would you mind elaborating briefly on some of those reasons or point me to further discussion please? We didn't want to depend on a single fs. Also, btrfs still has some performance issues (esp at fsync, which kills apt-get), I suspect a lot of performance issues revolve around unbalanced systems. and people still seem to hit corruption with it (though other people seem to run it rock-solid with no issues). Older war stories mostly revolve around folks letting their btrfs systems get to 100% full and/or involve earlier series 3 kernels and those earlier bad experiences are still being used as a reason why btrfs is not ready. I have invested heavily in btrfs so I am a little shocked at this news. If I want to stick to btrfs then would I be better off relying on legacy lxc? I don't think we'll be dropping the support we have. Sure, I wouldn't expect that, but it means that most future devel, testing, tutorials and example setups will be based on LVM instead of btrfs and that concerns me (not that my concerns matter in the real world.) We definately won't be adding support for zfs, overlayfs, etc. Good. Can you say a bit more about how your usage depends on btrfs? I can't compare btrfs to LVM because I've been using btrfs for so long now that I have forgotten all I knew about LVM... and very glad of that because btrfs is so much simpler and more flexible. I have a couple of dozen personal and professional systems and all run utopic and btrfs. The busiest server with 1000s of clients and 100's of vhost domains has been up for 6 months without any problems other than initial performance issues because the fs needed to be rebalanced. Once that was done, and once a month, it's been perfectly satisfactory. I also got caught out with sparse sqlite3 databases from Dspam but once they were regularly vacuumed that problem disappeared. I didn't notice that particular problem on the previous ext4/dell-raid system. Personally, my own pair of HP microservers for local backup were renovated from zfs to btrfs 3 months ago and have been working perfectly. Again, particularly so since being rebalanced. The ease of management and flexibility, especially being able to use send/receive to sync them, is just not (so easily) available without btrfs. The key points over LVM is being able to use disks of any size, online transition of raid personalities, file system (not hardware) level checksumming and... subvolumes. I guess my usage depends on btrfs is because of it's ease of use and flexibility to cover everything from a single laptop SSD through to various RAID configurations but short of enterprise level openstack-like systems. There the extra stability and performance of LVM is justified in 2015 (maybe 2016) but short of that fairly lofty niche enterprise level of need, this year, I believe btrfs is an overall superior fs solution and a perfect fit for lxc/lxd. Obviously IMHO. ___ lxc-users mailing list lxc-users@lists.linuxcontainers.org http://lists.linuxcontainers.org/listinfo/lxc-users
Re: [lxc-users] lxd: -B backingstore equivalent?
On Fri, Jun 05, 2015 at 11:36:37PM +0900, Tomasz Chmielewski wrote: On 2015-06-05 22:58, Tycho Andersen wrote: Hi Tomasz, On Fri, Jun 05, 2015 at 07:22:25PM +0900, Tomasz Chmielewski wrote: Is there a -B btrfs equivalent in lxd? Yes, if you mount /var/lib/lxd as a btrfs subvolume, it should Just Work. As I've checked, this is not the case (the container is created in a directory, not in btrfs subvolume; lxc-create -B btrfs creates it in a subvolume). Can you file a bug with info to reproduce? It should work as of 0.8. Thanks, Tycho lxd 0.9-0ubuntu2~ubuntu14.04.1~ppa1 -- Tomasz Chmielewski http://wpkg.org ___ lxc-users mailing list lxc-users@lists.linuxcontainers.org http://lists.linuxcontainers.org/listinfo/lxc-users
