Re: Debian, Qemu, KVM and Raspberry Pi
On 17/11/16 17:30, Lennart Sorensen wrote: On Thu, Nov 17, 2016 at 08:21:13AM +, Mark Morgan Lloyd wrote: I'm obviously watching these ongoing threads with a lot of interest :-) If I can ask two questions so that there's a summary in a single place ready for me to get back onto this: * Assuming a host kernel that has apparently been built with KVM etc., what's the best way to test that it exposes the required functionality? * What's the recommended Debian guest, and am I correct in assuming that the only indication of whether it's using KVM etc. is its speed of execution? I'm interested in embedding a low-traffic DMZ in a firewall, I think Qemu is adequate for this but wouldn't trust weaker containerisation. KVM on arm requires: CPUs booted in HYP mode (so boot loader has to be done right). That's been in the Raspbian loader for at least a few months. Kernel built with VGIC support (or in the case of the RPi 2 and 3 with emulated VGIC since it doesn't have the normal VGIC that most arm chips have). Think I've done that OK, at least for am RPi2. Either a 64 bit kernel or an lpae kernel. Probably safest ATM to stick to an RPi2, which means LPAE... I'll check. New enough qemu to have support for kvm on arm (not usually a problem anymore). RPi2/3 requires a patched one since the emulated VGIC apparently requires qemu to be tied to specific cores so that the first core can stay responsible for the VGIC emulation and not confuse qemu. I believe that can be forced without a patch. Really the RPi2/3 is just too weird to really support KVM due to the lack of standard expected arm cpu features. I don't expect it to ever have mainline kernel and qemu support due to those hardware deficiensies. OTOH it's cheap and fairly popular. If you have an arm system that boots with the cpu in HYP mode and have a kernel with KVM support enabled. I see the armhf lpae kernel has KVM support enabled in debian. I don't see it in the arm64 kernel config, so it is not enabled there yet. Probably should be. If you have that, then you should be able to run qemu with the -enable-kvm flag and it should work. I'll check, but won't be for a few days due to other pressures. -- Mark Morgan Lloyd markMLl .AT. telemetry.co .DOT. uk [Opinions above are the author's, not those of his employers or colleagues]
Re: Debian, Qemu, KVM and Raspberry Pi
On Thu, Nov 17, 2016 at 08:21:13AM +, Mark Morgan Lloyd wrote: > I'm obviously watching these ongoing threads with a lot of interest :-) > > If I can ask two questions so that there's a summary in a single place ready > for me to get back onto this: > > * Assuming a host kernel that has apparently been built with KVM etc., > what's the best way to test that it exposes the required functionality? > > * What's the recommended Debian guest, and am I correct in assuming that > the only indication of whether it's using KVM etc. is its speed of > execution? > > I'm interested in embedding a low-traffic DMZ in a firewall, I think Qemu is > adequate for this but wouldn't trust weaker containerisation. KVM on arm requires: CPUs booted in HYP mode (so boot loader has to be done right). Kernel built with VGIC support (or in the case of the RPi 2 and 3 with emulated VGIC since it doesn't have the normal VGIC that most arm chips have). Either a 64 bit kernel or an lpae kernel. New enough qemu to have support for kvm on arm (not usually a problem anymore). RPi2/3 requires a patched one since the emulated VGIC apparently requires qemu to be tied to specific cores so that the first core can stay responsible for the VGIC emulation and not confuse qemu. Really the RPi2/3 is just too weird to really support KVM due to the lack of standard expected arm cpu features. I don't expect it to ever have mainline kernel and qemu support due to those hardware deficiensies. If you have an arm system that boots with the cpu in HYP mode and have a kernel with KVM support enabled. I see the armhf lpae kernel has KVM support enabled in debian. I don't see it in the arm64 kernel config, so it is not enabled there yet. Probably should be. If you have that, then you should be able to run qemu with the -enable-kvm flag and it should work. -- Len Sorensen
Re: Debian, Qemu, KVM and Raspberry Pi
On 16/11/16 20:00, Lennart Sorensen wrote: On Wed, Nov 16, 2016 at 09:09:57AM +0100, Uwe Kleine-König wrote: AFAIK the RPi3 should be supported by the Debian arm64 kernel. So maybe the setup is easier there?! Doesn't solve that it needs VGIC emulation, which I highly doubt has gone into the mainline kernel. So KVM would still not be easy. Even looks like the current emulation patch is mutually exclusive with normal VGIC support in the kernel, so not something ready to go in at this point. I'm obviously watching these ongoing threads with a lot of interest :-) If I can ask two questions so that there's a summary in a single place ready for me to get back onto this: * Assuming a host kernel that has apparently been built with KVM etc., what's the best way to test that it exposes the required functionality? * What's the recommended Debian guest, and am I correct in assuming that the only indication of whether it's using KVM etc. is its speed of execution? I'm interested in embedding a low-traffic DMZ in a firewall, I think Qemu is adequate for this but wouldn't trust weaker containerisation. -- Mark Morgan Lloyd markMLl .AT. telemetry.co .DOT. uk [Opinions above are the author's, not those of his employers or colleagues]
Re: Debian, Qemu, KVM and Raspberry Pi
On Wed, Nov 16, 2016 at 09:09:57AM +0100, Uwe Kleine-König wrote: > AFAIK the RPi3 should be supported by the Debian arm64 kernel. So maybe > the setup is easier there?! Doesn't solve that it needs VGIC emulation, which I highly doubt has gone into the mainline kernel. So KVM would still not be easy. Even looks like the current emulation patch is mutually exclusive with normal VGIC support in the kernel, so not something ready to go in at this point. -- Len Sorensen
Re: Debian, Qemu, KVM and Raspberry Pi
On 11/08/2016 04:01 PM, Mark Morgan Lloyd wrote: > I prefer to run pukka Debian rather than Raspbian, approximately as > described at http://sjoerd.luon.net/posts/2015/02/debian-jessie-on-rpi2/ > > http://blog.flexvdi.com/2015/03/17/enabling-kvm-virtualization-on-the-raspberry-pi-2/ > and related pages describes getting Qemu+KVM running on an RPi2. > > Has anybody done this, are there comparable instructions for an RPi3, > and- above all- is there a straightforward kernel release suitable for > host and guest? AFAIK the RPi3 should be supported by the Debian arm64 kernel. So maybe the setup is easier there?! Best regards Uwe signature.asc Description: OpenPGP digital signature
Re: Debian, Qemu, KVM and Raspberry Pi
Ah. I'd get a Dell laptop a few years old, put in a fresh hard drive, and run OpenBSD instead of Linux. Much simpler and more reliable. I've been using it since about 2002, including on the only firewall machines I've built. I prefer the default FVWM to KDE for speed reasons but I've had no problems running KDE programs (Konqueror for example) under it. Gnome stuff works too, they should install the needed libraries. As far as I know, using a KDE or Gnome window manager works, they're just unneeded bloat. The black on black sounds like a colormap problem. Or a framebuffer problem where it's using some odd color depth instead of 24, I see a problem like that with some HDMI modes and Raspbian on my Pis. But I've mostly had no X at all on my firewall machines. I used to run full-size i386 machines retired from being Windows machines. One bottleneck may be that you need _two_ fast network interfaces plus a third one for control. A laptop with a built-in interface plus a Cardbus card would maybe work for the fast ones. Use the built-in (fastest) for the external network if you're going to be filtering packets. On 11/8/16, Mark Morgan Lloyd wrote: > On 08/11/16 16:00, Alan Corey wrote: >>> Has anybody done this, are there comparable instructions for an RPi3, >>> and- above all- is there a straightforward kernel release suitable for >>> host and guest? >> >> I posted a similar question on the Raspberry Pi forums here: >> https://www.raspberrypi.org/forums/viewtopic.php?f=63&t=154497&p=1010500#p1010500 > > Although the parts of the cited responses that I'm reading don't touch > on KVM etc. What I want to do is hide a mail (or other DMZ) server > inside a guest on a firewall system... which sounds a bit weird but > should be OK. > >> I got one response with some links in it but so far I'm still using >> Raspbian. It's very stable, which I care more about than being >> bleeding edge. I do the apt-get update and apt-get upgrade about once >> a month. I have 2 3Bs and a Zero, clone my SD card so I maintain 1 >> image. > > After experimentation we reverted to pukka Debian because we favour KDE, > and I was never able to get it grafted reliably onto Raspbian. > > I'm running with a USB-connected Seagate disc, that works well except > for cases where I reboot with e.g. a 3G 'phone tethered since it gets > the /dev/sd devices confused. I'm hoping that the new RPi3 firmware that > allows USB boot etc. improves that. > > I'm basically preparing an SD-Card using up-to-date Raspbian, and then > overwriting / with stuff from the link I gave earlier. Fix up locales > and run tasksel and that's about it. > > A colleague had Ubuntu aarch64 on an Odroid C2 and while it worked it > was definitely nothing to write home about- to the extent that he's now > reverted to my usual Debian mix and the hardware is back on my desk so > that I can investigate its I/O performance (which in principle is better > than that of an RPi3: it has Gbit Ethernet which isn't hung off the > USB). I'm not blaming the architecture for the problems, there's > definite distro packaging issues (e.g. remove no-longer-needed packages > and the desktop switches to black-on-black). > > - > > One of my ongoing jobs involves lots of L2TP stuff, if anybody's got the > standing please could they review > http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=842346 since it makes a > big difference to reliability. > > -- > Mark Morgan Lloyd > markMLl .AT. telemetry.co .DOT. uk > > [Opinions above are the author's, not those of his employers or colleagues] > > -- Credit is the root of all evil. - AB1JX
Re: Debian, Qemu, KVM and Raspberry Pi
> Has anybody done this, are there comparable instructions for an RPi3, > and- above all- is there a straightforward kernel release suitable for > host and guest? I posted a similar question on the Raspberry Pi forums here: https://www.raspberrypi.org/forums/viewtopic.php?f=63&t=154497&p=1010500#p1010500 I got one response with some links in it but so far I'm still using Raspbian. It's very stable, which I care more about than being bleeding edge. I do the apt-get update and apt-get upgrade about once a month. I have 2 3Bs and a Zero, clone my SD card so I maintain 1 image. -- Credit is the root of all evil. - AB1JX
Re: Debian, Qemu, KVM and Raspberry Pi
On Tue, Nov 08, 2016 at 03:01:09PM +, Mark Morgan Lloyd wrote: > I prefer to run pukka Debian rather than Raspbian, approximately as > described at http://sjoerd.luon.net/posts/2015/02/debian-jessie-on-rpi2/ > > http://blog.flexvdi.com/2015/03/17/enabling-kvm-virtualization-on-the-raspberry-pi-2/ > and related pages describes getting Qemu+KVM running on an RPi2. Wow I did not know it used a non standard interrupt controller. Ouch that would make it painful. > Has anybody done this, are there comparable instructions for an RPi3, and- > above all- is there a straightforward kernel release suitable for host and > guest? > > I've had it working after a fashion on an RPi2, but I found the process of > working out how the standard kernel was built and then merging patches from > GOK where to be painful. I suspect it would be quite similar between the Pi2 and pi3, although who knows, there could be more small differences showing up. I have only run kvm for arm on an AM572x which has a standard interrupt controller and hence was not too bad to get going. -- Len Sorensen
