I am working on transitioning my day-to-day software development work to Qubes. The primary challenge that I face is widespread use of Vagrant for provisioning development environments. I am aware of the challenges and concerns around hacking Qubes to achieve nested virtualization, so I am trying to avoid going down that road.
A potential alternative is to use Docker, because there are no inherent issues with using Linux containers inside virtual machines. Vagrant supports using Docker as a "provider," so this seems like a viable solution that could allow me to use my existing Vagrant-based development environments on Qubes with only minor modifications. I set up a new TemplateVM (based on Fedora 24) and AppVM to experiment with setting up a Vagrant+Docker-based development environment. It was surprisingly easy; however, after some initial testing, I realized an unfortunate truth: operations inside the Docker container are *very slow* - so slow as to create an unacceptable level of overhead for day-to-day development work. The slowdown appears to be due to slow disk IO. Running htop shows that processes in the container have status "D", meaning "disk sleep (uninterruptible)", for long periods of time. I tried switching the Docker storage backend to "overlay" instead of the Fedora default of devicemapper with loopback devices, which is a well-known technique for improving Docker performance on Fedora. This was an improvement, based on my experience as well as some simple dd-based IO benchmarks, but overall performance is still much too slow--borderline unusable. My gut feeling is that this is due to the combination of Qubes' overlay filesystem with Docker's overlay filesystem - I imagine that nesting COW filesystems is naturally a prescription for degraded performance. Unfortunately, I do not know enough about Qubes/Docker/filesystems to know of the best way to test this hypothesis. My questions for the list are: 1. Is anybody else successfully using Vagrant and/or Docker on Qubes? Do you have any tips/tricks to share? 2. Does anybody know or have any alternative theories as to what might be causing these performance problems? 3. Even better: does anybody have good advice for how to experimentally analyze these problems? Good ideas for how to analyze the source of the slowdown, ideas for benchmarks, useful analysis tools or techniques, etc.? I have some notes on my experience and a Docker environment for basic IO benchmarking, which I am happy to share if anybody's interested. Thanks, Garrett -- You received this message because you are subscribed to the Google Groups "qubes-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to qubes-users+unsubscr...@googlegroups.com. To post to this group, send email to qubes-users@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/qubes-users/8065ddd4-8153-3d13-dd15-edb7aafca62f%40freedom.press. For more options, visit https://groups.google.com/d/optout.
