On Thu, Jun 07, 2018 at 01:07:48PM -0500, Matt Riedemann wrote: > On 6/7/2018 12:56 PM, melanie witt wrote: > > Recently, we've received interest about increasing the maximum number of > > allowed volumes to attach to a single instance > 26. The limit of 26 is > > because of a historical limitation in libvirt (if I remember correctly) > > and is no longer limited at the libvirt level in the present day. So, > > we're looking at providing a way to attach more than 26 volumes to a > > single instance and we want your feedback. > > The 26 volumes thing is a libvirt driver restriction.
The original limitation of 26 disks was because at that time there was no 'virtio-scsi'. (With 'virtio-scsi', each of its controller allows upto 256 targets, and each target can use any LUN (Logical Unit Number) from 0 to 16383 (inclusive). Therefore, the maxium allowable disks on a single 'virtio-scsi' controller is 256 * 16384 == 4194304.) Source[1]. [...] > > Some ideas that have been discussed so far include: > > > > A) Selecting a new, higher maximum that still yields reasonable > > performance on a single compute host (64 or 128, for example). Pros: > > helps prevent the potential for poor performance on a compute host from > > attaching too many volumes. Cons: doesn't let anyone opt-in to a higher > > maximum if their environment can handle it. Option (A) can still be considered: We can limit it to 256 disks. Why? FWIW, I did some digging here: The upstream libguestfs project after some thorough testing, arrived at a limit of 256 disks, and suggest the same for Nova. And if anyone wants to increase that limit, the proposer should come up with a fully worked through test plan. :-) (Try doing any meaningful I/O to so many disks at once, and see how well that works out.) What more, the libguestfs upstream tests 256 disks, and even _that_ fails sometimes: https://bugzilla.redhat.com/show_bug.cgi?id=1478201 -- "kernel runs out of memory with 256 virtio-scsi disks" The above bug is fixed now in kernel-4.17.0-0.rc3.git1.2. (And also required a corresponding fix in QEMU[2], which is available from version v2.11.0 onwards.) [...] [1] https://lists.nongnu.org/archive/html/qemu-devel/2017-04/msg02823.html -- virtio-scsi limits [2] https://git.qemu.org/?p=qemu.git;a=commit;h=5c0919d -- /kashyap __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev