Hi Marcel
On 21/07/2017 01:33, Marcel Apfelbaum wrote:
On 20/07/2017 3:44, Kinsella, Ray wrote:
That's strange. Please ensure the virtio devices are working in
virtio 1.0 mode (disable-modern=0,disable-legacy=1).
Let us know any problems you see.
Not sure what yet, I will try scaling it with hotplugging tomorrow.
Updates?
I have managed to scale it to 128 devices.
The kernel does complain about IO address space exhaustion.
[ 83.697956] pci 0000:80:00.0: BAR 13: no space for [io size 0x1000]
[ 83.700958] pci 0000:80:00.0: BAR 13: failed to assign [io size 0x1000]
[ 83.701689] pci 0000:80:00.1: BAR 13: no space for [io size 0x1000]
[ 83.702378] pci 0000:80:00.1: BAR 13: failed to assign [io size 0x1000]
[ 83.703093] pci 0000:80:00.2: BAR 13: no space for [io size 0x1000]
I was surprised that I am running out of IO address space, as I am
disabling legacy virtio. I assumed that this would remove the need for
SeaBIOS to allocate the PCI Express Root Port IO address space. In any
case - it doesn't stop the virtio-net device coming up and working as
expected.
[ 668.692081] virtio_net virtio103 enp141s0f4: renamed from eth101
[ 668.707114] virtio_net virtio130 enp144s0f7: renamed from eth128
[ 668.719795] virtio_net virtio129 enp144s0f6: renamed from eth127
I encountered some issues in vhost, due to open file exhaustion but
resolved these with 'ulimit' in the usual way - burned alot of time on
that today.
When scaling up to 512 Virtio-net devices SeaBIOS appears to really slow
down when configuring PCI Config space - haven't manage to get this to
work yet.
Not really. All you have to do is to add a property to the pxb-pci/pxb
devices: pci_domain=x; then update the ACPI table to include the pxb
domain. You also have to tweak a little the pxb-pcie/pxb devices
to not share the bus numbers if pci_domain > 0.
Thanks for information, will add to the list.
Ray K
\
