Re: [PATCHv3] virtio-spec: 64 bit features, used/avail event, fixes
On Wed, 1 Jun 2011 13:25:48 +0300, "Michael S. Tsirkin" wrote: > Add an option to modify the notificatin > hand-off in virtio to be basically like Xen: > each side published an index, the other side only triggers > an event when it crosses that index value > (Xen event indexes start at 1, ours start at 0 for > backward-compatiblity, but that's minor). > > Since we've run out of bits in the 32 bit field, > I added another 32 bit and bit 31 enables that. OK. I've applied this, and published it as the 0.9 draft. Thanks! Rusty. ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linux-foundation.org/mailman/listinfo/virtualization
Re: [PATCHv3] virtio-spec: 64 bit features, used/avail event, fixes
On Thu, Jun 02, 2011 at 11:19:35AM +0930, Rusty Russell wrote: > On Wed, 1 Jun 2011 13:25:48 +0300, "Michael S. Tsirkin" > wrote: > > Add an option to modify the notificatin > > hand-off in virtio to be basically like Xen: > > each side published an index, the other side only triggers > > an event when it crosses that index value > > (Xen event indexes start at 1, ours start at 0 for > > backward-compatiblity, but that's minor). > > > > Since we've run out of bits in the 32 bit field, > > I added another 32 bit and bit 31 enables that. > > OK. I've applied this, and published it as the 0.9 draft. > > Thanks! > Rusty. There's something that is bothering me: each such change increases the size of the config. Now, on PCI it's in io space which is much constrained. It might not seem like 32 or 64 bytes is a lot, but in practice there's a problem with a bridged setup: bridges sometimes need resources pre-allocated for all devices that might land behind them. E.g. with 32 bridges, and 32 devices behind each one, the available 64K space gets us only 64 bytes per device. Thankfully even with this change, we are still below that limit. But for example, this seems to push the size of the config for virtio net from 32 to 34 bytes. The PCI BAR is a power of two, so it will be exactly 64 bytes then. io seems to work much better than memory on kvm, so we will need to stick to that for datapath. But maybe it's time to start putting non-datapath in memory? Thanks, -- MST ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linux-foundation.org/mailman/listinfo/virtualization
Re: [PATCHv3] virtio-spec: 64 bit features, used/avail event, fixes
Hi, > E.g. with 32 bridges, and 32 devices behind each one, > the available 64K space gets us only 64 bytes per device. 15 bridges (with io window enabled) max, the smallest io window you can assign to a bridge is 4k, and you need some space for the devices on the root bus ... cheers, Gerd ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linux-foundation.org/mailman/listinfo/virtualization
Re: [PATCHv3] virtio-spec: 64 bit features, used/avail event, fixes
On Wed, Aug 03, 2011 at 06:15:33PM +0200, Gerd Hoffmann wrote: > Hi, > > >E.g. with 32 bridges, and 32 devices behind each one, > >the available 64K space gets us only 64 bytes per device. > > 15 bridges (with io window enabled) max, the smallest io window you > can assign to a bridge is 4k, Hmm true, I missed that. So with 32 devs we get 256 bytes per device. We can still get low on that space when using multifunction devices though (there could be up to 256 functions behind a bridge, which only leaves 16 bytes per device). BTW, this limitation will be a problem for pci express devices. > and you need some space for the > devices on the root bus ... > > cheers, > Gerd ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linux-foundation.org/mailman/listinfo/virtualization
Re: [PATCHv3] virtio-spec: 64 bit features, used/avail event, fixes
On Wed, Aug 03, 2011 at 06:15:33PM +0200, Gerd Hoffmann wrote: > Hi, > > >E.g. with 32 bridges, and 32 devices behind each one, > >the available 64K space gets us only 64 bytes per device. > > 15 bridges (with io window enabled) max, the smallest io window you > can assign to a bridge is 4k, and you need some space for the > devices on the root bus ... > > cheers, > Gerd Hmm, wait, we could go 32 bit, then we are not limited to 32 bridges anymore, right? Does our bios support that? -- MST ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linux-foundation.org/mailman/listinfo/virtualization
Re: [PATCHv3] virtio-spec: 64 bit features, used/avail event, fixes
Hi, > Hmm, wait, we could go 32 bit, then we are not limited to 32 > bridges anymore, right? Does our bios support that? How you want go to 32bit? io space (not mmio) is fixed at 16bit in x86, isn't it? cheers, Gerd ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linux-foundation.org/mailman/listinfo/virtualization
