Re: New vendor branch for FreeBSD on Hyper-V. Please review.
Hi. As ATA driver maintainer I've took a look on that part of the code. I have some comments about what I see. Please keep me CC'd, as I am normally not on this list. 1) You are using machine dependent function do_cpuid() in machine independent ATA code. It will just not build on any architecture except amd64 and i386. You should periodically run `make universe` to be sure that you code does not breaks the build for some rare archs. As minimum that code should be wrapped into respective #ifdef's, but I would prefer it to be moved out somewhere to MD code (like MD code that does general VM detection now), as this doesn't really belong to the ATA driver. 2) You are disabling "ad" ATA disk driver. Here I have two objections: a) "ad" driver is obsoleted and not used since FreeBSD 9.0, when CAM-based ATA stack replaces it completely, wrapping only controller driver parts of old ata(4) into CAM SIM, which is now only one of four ATA/SATA SIMs in the system. Your change is effectively null now. b) I believe you are using wrong approach here. I believe instead of completely disabling disk driver you should disable only specific Hyper-V emulated ATA controller device instance. I've never used Hyper-V, but I guess it should support PCI device pass-through. Just imagine what happen if somebody wish to pass-through ATA/SATA controller? You will disable it also. 3) Looking on what I suppose is "Fast IDE" driver in the file hv_storvsc_drv_freebsd.c, it seems more like cut-down SAS, then IDE. It doesn't support ATA commands, it reports itself to CAM as SAS. The only differences I see are disabling LUNs and limiting queue to 1 slot. Just a smile: it is neither really "Fast" nor at all "IDE". :) 4) In you SAS driver in storvsc_action() you have hardcoded CTS_SCSI_FLAGS_TAG_ENB flag. While I guess it should not cause any problems, it wont allow user to control it if he decide to experiment with disabling it. -- Alexander Motin ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Proposal for better support of hypervisors and their synthetic drivers at boot-time
On 23.04.2013 17:07, Larry Melia wrote: I’m on a team of developers working on synthetic device drivers for FreeBSD on Hyper-V. In late March, I briefly mentioned (on this list) that we're trying to get our drivers included with the FreeBSD distribution. Also noted were some necessary patches to the ATA driver. The changes are necessary to achieve significant performance gains by replacing the native ATA driver with our synthetic storage driver when when Hyper-V is detected. Alexander Motin, the maintainer of the ATA code-base, however, expressed some concerns about making these modifications that are so specific to Hyper-V and the AMD64 build. We understand his concerns and have subsequently removed these patches from our vendor branch. Our drivers are now completely independent and require no changes to the kernel or other drivers. The only touch-point is the common makefile for modules/drivers. Removing our ATA patches, on the other hand, results in a huge performance loss. This is because the root file system is managed by the ATA driver, which is emulated under Hyper-V. Furthermore, there may be other native drivers that we wish to replace with synthetic ones at boot time (e.g., mouse driver, video driver), but, there appears to be no easy way to do this.Therefore, we would like to work with the developer community to come up with better solution that would improve support for synthetic drivers at boot-time. One possible solution: (1) Move the call to init_param1() (in sys/kern/subr_parm.c), which is used for hypervisor detection, to an earlier point in the boot process. Presently, it appears to be called after the ATA driver is selected, which is too late in the boot process. (This was discovered after some testing with the ATA driver.) Therefore, before the bus drivers and native controllers are detected and selected, discovery of a host hypervisor should be done first. (2) Extend the boot process to dynamically insert and prioritize synthetic drivers over native ones. Presently, FreeBSD supports the selection of generic drivers for a given device, but these can be overridden when a more specific driver is available. This priority scheme could be extended by first detecting the presence of a hypervisor, then allowing this priority mechanism to override the native drivers with their synthetic cousins (i.e., we only want to override a native driver when a specific hypervisor is detected and synthetic drivers for it have been configured). This could be arranged in a separate, table-driven, input file or perhaps, by extending the existing driver/module configuration file with additional mappings of native to synthetic drivers. (3) Upgrade the init_param1() function (in sys/kern/subr_parm.c) to use the more recent approach to hypervisor detection. This approach uses the CPU-identify functions to retrieve a unique signature consisting of a fixed string of ASCII characters. This was done on Linux about five years. For backward compatibility, however, the existing logic would be retained, but augmented with this new approach. It would also be conditionally added only for x86/AMD64 builds. When reviewing the changes we're not looking at a lot of lines of code.The difficultly lies, however, in where the changes are made. Obviously, we need to work closely with the committers responsible for the kernel modules we're looking to touch. Retrospectively, these upgrades only bring FreeBSD up to the same level of hypervisor support already present in Linux.There are other issues that we would also like to discuss, but the improvements described here are on the top of the list. We look forward to working more closely with the FreeBSD community and welcome your remarks and feedback on this proposal. I am sorry, I don't really understand what is your proposition, so I just describe my vision of the solution here, hoping it will be useful. You are right that FreeBSD has priority mechanism to select best driver for each device. That mechanism should be sufficient to prevent default OS driver from attaching to emulated ATA controller when Hyper-V environment is detected and paravirtual drivers are installed. To do that you should just create a very small dummy driver for PCI bus, implementing single probe() method. That probe() method should in your preferable way check that Hyper-V environment is active and match PCI ID of the device against the list of used by Hyper-V for emulated ATA controller. If both conditions match, the probe() method should return highest priority value to override any other ATA driver in system for this specific device. After that the dummy driver should just have empty attach() and detach() routines, not doing anything with the emulated device, relying on paravitrual driver to handle the disks. Since the dummy driver can be made machine-dependent, you may use there any VM detection mechanisms you like, even direct MSR access. Dummy driver ca
Re: ATA controller and Hyper-V "synthetic storage driver" handle the IDE disks devices ( but not for the CD-ROM) Re: Hyper-V on FreeBSD
On 07.05.2013 10:40, Victor Miasnikov wrote: prevent default OS driver from attaching to emulated ATA controller when Hyper-V Disable both primary and secondary ATA controller prevent use CD-ROM device in VM ( guest) Because "synthetic storage driver" not handle CD-ROM devices ( IMHO, even more: Hyper-V host work with CD-ROM only as "PCI bus" device) Then may be possibility of blocking/hiding specific ATA channels or devices could be investigated. http://lists.freebsd.org/pipermail/freebsd-virtualization/2013-April/001180.html == . . . On 23.04.2013 17:07, Larry Melia wrote: . . . Removing our ATA patches, on the other hand, results in a huge performance loss. This is because the root file system is managed by the ATA driver, which is emulated under Hyper-V. == Place on IDE disk only "\boot" ( and bootloader), other on SCSI . May be instead of splitting filesystem it could be better to boot from "SCSI" completely and forget about "ATA"? Or there is no option ROM for SCSI boot in Hyper-V? -- Alexander Motin ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: more granular detection and control to disable/enable PCI-ATA devices
Hi. On 04.06.2013 18:30, Larry Melia wrote: With you suggestions, I finally was able to get the override driver working-see https://github.com/FreeBSDonHyper-V/freebsd-snapshot/blob/hyperv-dev-ata-override/sys/dev/hyperv/stordisengage/hv_ata_pci_disengage.c. While it operates wonderfully, allowing our "enlightened" driver to improve performance, some versions of Hyper-V still use the native CD-ROM driver, because there is no "enlightened" support for it in the hypervisor. From my limited knowledge of the ATA drivers, it seems likely that the PCI-ATA driver be attached when a CD-ROM is detected, but lower-level drivers disabled (during a probe) when a hard drive is detected. On Hyper-V, therefore, a user would be able to configure a PCI/IDE virtual controller with two devices, the first device a hard disk and the second one a CD-ROM. The CD-ROM would operate via the native driver, whereas the hard disk would use the "enlightened" driver (to improve performance). Is there an easy way to add more granular detection, disabling the native ATA driver(s) selectively for hard drives, while allowing CD-ROM devices to be natively attached? Any suggestions would be very much appreciated. Unfortunately, CAM subsystem used for both ATA and SCSI stacks in FreeBSD 9.x and above is mostly unaware of "NewBus" infrastructure used for driver probe and attachment. That is why you can't replace driver for a single disk in the same way as you replaced driver for the whole controller. The highest level present in "NewBus" is ATA channel. So if disk and CD-ROM are always live on different channels, you can create dummy driver not for the whole controller (atapciX), but for single hardcoded ATA channel (ataX). Another possible way is to make controller driver not dummy, making it mostly duplicating default one, but filtering out unwanted devices. That may look like overkill, but it is not necessary so, because ATA stack is quite modularized, and you probably don't need to implement all ATA functionality such as mode setting, etc. Only thing that should be different in your driver is a reset method -- never reporting ATA disks to upper layers, only ATAPI devices. You may find number of drivers for example in sys/dev/ata/chipsets. The later way is definitely more complicated then just a few lines hack blocking CAM ada driver (ATA disk driver), but it still can be made modular and non-invasive. -- Alexander Motin ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: [Hyper-V][camlock] storvsc driver panics during boot with patches from camlock project
Hi. I took some look and think problems are in scan_for_luns() routine: - After the locking changes scanning normally uses different locks, not the SIM one. That probably caused panic. - But I think that scanning is simply not needed there -- FreeBSD CAM scans every new bus automatically on registration (Even for late registered buses it is done I think at least since FreeBSD 8). I think everything should just work if you remove scan_for_luns() at all. - If you still wish to force scan (due to having information about changed list of devices, etc), then you can make CAM do all the magic for you by calling xpt_rescan(). On 24.10.2013 08:34, Oleg Sidorkin wrote: Hello again. Camlock patches are now committed and -CURRENT on Hyper-V now panics with almost the same stacktrace: FreeBSD 11.0-CURRENT #16 r257016: Wed Oct 23 21:08:44 UTC 2013 olsi@current:/usr/obj/usr/src/sys/HYPERVKERNEL amd64 FreeBSD clang version 3.3 (tags/RELEASE_33/final 183502) 20130610 CPU: Intel(R) Core(TM) i5-2540M CPU @ 2.60GHz (1309.57-MHz K8-class CPU) Origin = "GenuineIntel" Id = 0x206a7 Family = 0x6 Model = 0x2a Stepping = 7 .. ZFS filesystem version: 5 ZFS storage pool version: features support (5000) Timecounters tick every 10.000 msec storvsc0 on vmbus0 kernel trap 12 with interrupts disabled Fatal trap 12: page fault while in kernel mode cpuid = 0; apic id = 00 fault virtual address = 0x20 fault code = supervisor read data, page not present instruction pointer = 0x20:0x804f58cc stack pointer = 0x28:0xfe011dd5f5d0 frame pointer = 0x28:0xfe011dd5f600 code segment= base 0x0, limit 0xf, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags= resume, IOPL = 0 current process = 0 (hv_control_1 taskq) [ thread pid 0 tid 100047 ] Stopped at turnstile_broadcast+0x8c: movq0x20(%rbx,%rax,1),%rdx db> bt Tracing pid 0 tid 100047 td 0xf8000331e000 turnstile_broadcast() at turnstile_broadcast+0x8c/frame 0xfe011dd5f600 __mtx_unlock_sleep() at __mtx_unlock_sleep+0x60/frame 0xfe011dd5f630 unlock_mtx() at unlock_mtx+0x2a/frame 0xfe011dd5f640 _sleep() at _sleep+0x18e/frame 0xfe011dd5f6c0 cam_periph_runccb() at cam_periph_runccb+0x9e/frame 0xfe011dd5f7f0 storvsc_attach() at storvsc_attach+0x6d4/frame 0xfe011dd5f890 device_attach() at device_attach+0x3a2/frame 0xfe011dd5f8f0 hv_vmbus_child_device_register() at hv_vmbus_child_device_register+0xdb/frame 0xfe011dd5f990 vmbus_channel_process_offer() at vmbus_channel_process_offer+0x133/frame 0xfe011dd5f9d0 work_item_callback() at work_item_callback+0x26/frame 0xfe011dd5f9f0 taskqueue_run_locked() at taskqueue_run_locked+0xe6/frame 0xfe011dd5fa40 taskqueue_thread_loop() at taskqueue_thread_loop+0xa8/frame 0xfe011dd5fa70 fork_exit() at fork_exit+0x9a/frame 0xfe011dd5fab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfe011dd5fab0 --- trap 0, rip = 0, rsp = 0xfe011dd5fb70, rbp = 0 --- Thanks On Tue, Sep 24, 2013 at 3:04 AM, Abhishek Gupta (LIS) wrote: Hi Oleg, Please give us some time. I shall look at it. Thanks for reporting. Regards, Abhishek -Original Message- From: owner-freebsd-virtualizat...@freebsd.org [mailto:owner-freebsd-virtualizat...@freebsd.org] On Behalf Of Oleg Sidorkin Sent: Monday, September 23, 2013 7:21 AM To: freebsd-virtualization@freebsd.org Cc: Alexander Motin Subject: [Hyper-V][camlock] storvsc driver panics during boot with patches from camlock project Hello. I'm running the latest current (amd64) under Hyper-V with hyper-v services enabled. If camlock patches are applied (http://people.freebsd.org/~mav/camlock_patches/camlock_20130906.patch), I'm hitting the following kernel panic during boot: FreeBSD 10.0-ALPHA2 #5 r255762M: Sun Sep 22 16:48:21 UTC 2013 olsi@current:/usr/obj/usr/src/sys/HYPERVKERNEL amd64 FreeBSD clang version 3.3 (tags/RELEASE_33/final 183502) 20130610 CPU: Intel(R) Core(TM) i5-2540M CPU @ 2.60GHz (1309.17-MHz K8-class CPU) Origin = "GenuineIntel" Id = 0x206a7 Family = 0x6 Model = 0x2a Stepping = 7 Timecounter "Hyper-V" frequency 1000 Hz quality 1000 ZFS NOTICE: Prefetch is disabled by default if less than 4GB of RAM is present; to enable, add "vfs.zfs.prefetch_disable=0" to /boot/loader.conf. ZFS filesystem version: 5 ZFS storage pool version: features support (5000) Timecounters tick every 10.000 msec storvsc0 on vmbus0 Netvsc initializing... SMP: AP CPU #3 Launched! SMP: AP CPU #2 Launched! SMP: AP CPU #1 Launched! kernel trap 12 with interrupts disabled Fatal trap 12: page fault while in kernel mode cpuid = 3; apic id = 03 fault virtual address = 0x20 fault code = supervisor read data, page not present instruction pointer = 0x20:0x804f444c stack poin
Re: [Hyper-V][camlock] storvsc driver panics during boot with patches from camlock project
Oleg, It was found that real problem is not there. You do not need manual scanning. You should remove it. All you really need is to properly report bus to CAM: @@ -1147,7 +1062,7 @@ cpi->hba_eng_cnt = 0; cpi->max_target = STORVSC_MAX_TARGETS; cpi->max_lun = sc->hs_drv_props->drv_max_luns_per_target; - cpi->initiator_id = 0; + cpi->initiator_id = cpi->max_lun + 1; cpi->bus_id = cam_sim_bus(sim); cpi->base_transfer_speed = 30; cpi->transport = XPORT_SAS; Default scanner skips initiator from the process, so initiator_id = 0 excluded from scan the only really used target. On 04.03.2014 12:37, Oleg Sidorkin wrote: Hi again. Disabling scan_for_luns leaves drives undetected. Calling xpt_rescan for each lun works for me. With the attached patch system boots and detects all configured drives. But also this patch introduces a race between drives detection and boot process, so sometimes system tries to mount undetected drive. I'm going to fix this by calling xpt_hold_boot() before xpt_rescan() and calling xpt_release_boot() in callback. Thanks On Thu, Oct 24, 2013 at 9:50 AM, Alexander Motin wrote: Hi. I took some look and think problems are in scan_for_luns() routine: - After the locking changes scanning normally uses different locks, not the SIM one. That probably caused panic. - But I think that scanning is simply not needed there -- FreeBSD CAM scans every new bus automatically on registration (Even for late registered buses it is done I think at least since FreeBSD 8). I think everything should just work if you remove scan_for_luns() at all. - If you still wish to force scan (due to having information about changed list of devices, etc), then you can make CAM do all the magic for you by calling xpt_rescan(). On 24.10.2013 08:34, Oleg Sidorkin wrote: Hello again. Camlock patches are now committed and -CURRENT on Hyper-V now panics with almost the same stacktrace: FreeBSD 11.0-CURRENT #16 r257016: Wed Oct 23 21:08:44 UTC 2013 olsi@current:/usr/obj/usr/src/sys/HYPERVKERNEL amd64 FreeBSD clang version 3.3 (tags/RELEASE_33/final 183502) 20130610 CPU: Intel(R) Core(TM) i5-2540M CPU @ 2.60GHz (1309.57-MHz K8-class CPU) Origin = "GenuineIntel" Id = 0x206a7 Family = 0x6 Model = 0x2a Stepping = 7 .. ZFS filesystem version: 5 ZFS storage pool version: features support (5000) Timecounters tick every 10.000 msec storvsc0 on vmbus0 kernel trap 12 with interrupts disabled Fatal trap 12: page fault while in kernel mode cpuid = 0; apic id = 00 fault virtual address = 0x20 fault code = supervisor read data, page not present instruction pointer = 0x20:0x804f58cc stack pointer = 0x28:0xfe011dd5f5d0 frame pointer = 0x28:0xfe011dd5f600 code segment= base 0x0, limit 0xf, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags= resume, IOPL = 0 current process = 0 (hv_control_1 taskq) [ thread pid 0 tid 100047 ] Stopped at turnstile_broadcast+0x8c: movq 0x20(%rbx,%rax,1),%rdx db> bt Tracing pid 0 tid 100047 td 0xf8000331e000 turnstile_broadcast() at turnstile_broadcast+0x8c/frame 0xfe011dd5f600 __mtx_unlock_sleep() at __mtx_unlock_sleep+0x60/frame 0xfe011dd5f630 unlock_mtx() at unlock_mtx+0x2a/frame 0xfe011dd5f640 _sleep() at _sleep+0x18e/frame 0xfe011dd5f6c0 cam_periph_runccb() at cam_periph_runccb+0x9e/frame 0xfe011dd5f7f0 storvsc_attach() at storvsc_attach+0x6d4/frame 0xfe011dd5f890 device_attach() at device_attach+0x3a2/frame 0xfe011dd5f8f0 hv_vmbus_child_device_register() at hv_vmbus_child_device_register+0xdb/frame 0xfe011dd5f990 vmbus_channel_process_offer() at vmbus_channel_process_offer+0x133/frame 0xfe011dd5f9d0 work_item_callback() at work_item_callback+0x26/frame 0xfe011dd5f9f0 taskqueue_run_locked() at taskqueue_run_locked+0xe6/frame 0xfe011dd5fa40 taskqueue_thread_loop() at taskqueue_thread_loop+0xa8/frame 0xfe011dd5fa70 fork_exit() at fork_exit+0x9a/frame 0xfe011dd5fab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfe011dd5fab0 --- trap 0, rip = 0, rsp = 0xfe011dd5fb70, rbp = 0 --- Thanks On Tue, Sep 24, 2013 at 3:04 AM, Abhishek Gupta (LIS) wrote: Hi Oleg, Please give us some time. I shall look at it. Thanks for reporting. Regards, Abhishek -Original Message- From: owner-freebsd-virtualizat...@freebsd.org [mailto:owner-freebsd-virtualizat...@freebsd.org] On Behalf Of Oleg Sidorkin Sent: Monday, September 23, 2013 7:21 AM To: freebsd-virtualization@freebsd.org Cc: Alexander Motin Subject: [Hyper-V][camlock] storvsc driver panics during boot with patches from camlock project Hello. I'm running the latest current (amd64) under Hyper-V with hyper-v services enabled. If cam
Re: bhyve 20150316 crashed
On 16.03.2015 10:56, Florian Smeets wrote: > On 3/16/2015 1:12 AM, Willem Jan Withagen wrote: >> Assertion failed: (err == 0), function pci_vtblk_proc, file >> /usr/srcs/src11/src/usr.sbin/bhyve/pci_virtio_block.c, line 276. >> >> I noticed some work by several people on the virtio part of the tree. >> >> This is with todays tree: >> bhyve00 11.0-CURRENT FreeBSD 11.0-CURRENT #72 r280010: >> En VM crashing is an older FreeBSD-head, whilest running >> svlite up. >> >> And err is actually 7, >> type = 1 >> > > I was about to report the same thing. I'm still running r280041 and > wanted to repro with the latest changes before reporting. > > I've just now updates 7 bhyves to r280120 and am putting some load on it. I also hit this problem, and I hope it should be fixed by r280126. -- Alexander Motin ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Bhyve storage improvements (was: Several bhyve quirks)
> I've always assumed virtio driver > emulated driver so it didn't occur > to me to try ahci-hd. I've just merged to FreeBSD stable/10 branch set of bhyve changes that should significantly improve situation in the storage area. virtio-blk driver was fixed to work asynchronously and not block virtual CPU, that should fix many problems with performance and interactivity. Both virtio-blk and ahci-hd drivers got ability to execute multiple (up to 8) requests same time, that should proportionally improve parallel random I/O performance on wide storages. At this point virtio-blk is indeed faster then ahci-hd on high IOPS, and they both are faster then before. On the other side ahci-hd driver now got TRIM support to allow freeing unused space on backing ZVOL. Unfortunately there is no any TRIM/UNMAP support in virtio-blk API to allow the same. Also both virtio-blk and ahci-hd drivers now report to guest logical and physical block sizes of underlying storage, that allow guests properly align partitions and I/Os for best compatibility and performance. -- Alexander Motin ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Bhyve storage improvements
On 27.03.2015 18:47, John Nielsen wrote: > Does anyone have plans (or know about any) to implement virtio-scsi support > in bhyve? That API does support TRIM and should retain most or all of the > low-overhead virtio goodness. I was thinking about that (not really a plans yet, just some thoughts), but haven't found a good motivation and understanding of whole possible infrastructure. I am not sure it worth to emulate SCSI protocol in addition to already done ATA in ahci-hd and simple block in virtio-blk just to get another, possibly faster then AHCI, block storage with TRIM/UNMAP. Really good SCSI disk emulation in CTL in kernel takes about 20K lines of code. It is pointless to duplicate it, and may be complicated for administration to just interface to it. Indeed I've seen virtio-blk being faster then ahci-hd in some tests, but those tests were highly synthetic. I haven't tested it on real workloads, but I have feeling that real difference may be not that large. If somebody wants to check -- more benchmarks are highly welcome! From the theoretical side I'd like to notice that both ATA and SCSI protocols on guests go through additional ATA/SCSI infrastructure (CAM in FreeBSD), absent in case pure block virtio-blk, so they have some more overhead by definition. Main potential benefit I see from using virtio-scsi is a possibility to pass through to client not a block device, but some real SCSI device. It can be some local DVD writer, or remote iSCSI storage. The last would be especially interesting for large production installations. But the main problem I see here is booting. To make user-level loader boot the kernel from DVD or iSCSI, bhyve has to implement its own SCSI initiator, like small second copy of CAM in user-level. Booting kernel from some other local block storage and then attaching to remote iSCSI storage for data can be much easier, but it is not convenient. It is possible to nt connect to iSCSI directly from user-level, but to make kernel CAM do it, and then make CAM provide both block layer for booting and SCSI layer for virtio-scsi, but I am not sure that it is very good from security point to make host system to see virtual disks. Though may be it could work if CAM could block kernel/GEOM access to them, alike it is done for ZVOLs now, supporting "geom" and "dev" modes. Though that complicates CAM and the whole infrastructure. -- Alexander Motin ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Bhyve storage improvements (was: Several bhyve quirks)
Hi, Julian. > I had some time to try it out today, but I'm still having issues: I've just made experiment alike to your with making bhyve to work on top of GEOM device instead of preferable "dev" mode of ZVOL. And I indeed reproduced the problem. But the problem that I see is not related to the block size. The block size is reported to the guest correctly as 4K, and as I can see it works as such at least in FreeBSD guest. The problem is in the way how bhyve inter-operates with block/GEOM devices. bhyve sends requests to the kernel with preadv()/pwritev() calls, specifying scatter/gather lists of buffer addresses provided by the guest. But GEOM code can not handle scatter/gather lists, only sequential buffer, and so single request is split into several. The problem is that splitting happens according to scatter/gather elements, and those elements in general case may not be multiple to the block size, that is fatal for GEOM and any block device. I am not yet sure how to fix this problem. The most straightforward way is to copy the data at some point to collect elements of scatter/gather list into something sequential to pass to GEOM, but that requires additional memory allocation, and the copying is not free. May be some cases could be optimized to work without copying but with some clever page mapping, but that seems absolutely not trivial. -- Alexander Motin ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Bhyve storage improvements (was: Several bhyve quirks)
On 06.04.2015 23:38, Alexander Motin wrote: >> I had some time to try it out today, but I'm still having issues: > > I've just made experiment alike to your with making bhyve to work on top > of GEOM device instead of preferable "dev" mode of ZVOL. And I indeed > reproduced the problem. But the problem that I see is not related to the > block size. The block size is reported to the guest correctly as 4K, and > as I can see it works as such at least in FreeBSD guest. > > The problem is in the way how bhyve inter-operates with block/GEOM > devices. bhyve sends requests to the kernel with preadv()/pwritev() > calls, specifying scatter/gather lists of buffer addresses provided by > the guest. But GEOM code can not handle scatter/gather lists, only > sequential buffer, and so single request is split into several. The > problem is that splitting happens according to scatter/gather elements, > and those elements in general case may not be multiple to the block > size, that is fatal for GEOM and any block device. > > I am not yet sure how to fix this problem. The most straightforward way > is to copy the data at some point to collect elements of scatter/gather > list into something sequential to pass to GEOM, but that requires > additional memory allocation, and the copying is not free. May be some > cases could be optimized to work without copying but with some clever > page mapping, but that seems absolutely not trivial. I've committed the workaround to FreeBSD head at r281700. -- Alexander Motin ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
[Differential] [Updated] D5473: ATA/ATAPI6 device emulation in bhyve
mav added a comment. It was definitely significant amount of work, but I am not sure what motivation was behind it. Why do we need it after already having much more featured AHCI-based ATA/ATAPI emulation? Are there any significant OS not supporting AHCI? There is significant amount of code duplication between implementations, plus this one obviously requires more work to be complete. REVISION DETAIL https://reviews.freebsd.org/D5473 EMAIL PREFERENCES https://reviews.freebsd.org/settings/panel/emailpreferences/ To: iateaca, grehan, neel, tychon, mav Cc: freebsd-virtualization-list ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
[Differential] [Commented On] D5473: ATA/ATAPI6 device emulation in bhyve
mav added a comment. In https://reviews.freebsd.org/D5473#117346, @iateaca wrote: > The motivation is to run older versions of operating systems such as FreeBSD 4 which does not have AHCI drivers. > What do you mean by code duplication ? I think only the ATAPI CDROM logic could be common but the current implementation from AHCI can not be used with the ATA data strcutures. If we want to achive this, I think a redesign of AHCI ATAPI is required too. I personally see quite little sense in supporting so old legacy guests. I agree that there can be "some cases", but I am not sure they worth the time spent and code size growth. Other then legacy guests support this code does not give us anything useful -- legacy ATA will be by definition much slower and less functional then its AHCI counterpart. It will require dozens of emulated register accesses per I/O, comparing to only several for AHCI, and won't support command queuing. Though obviously nice and clean unified implementation would look better. In your patch you are one more time reimplementing some subsets of ATA and ATAPI commands handling, already done much wider for AHCI. It would be much better to have single device emulation code, interfacing with different controller code parts. But it would also take much more time (you are rigth that it would require existing code redesign), for the same little reason, so I am not sure how good is that idea. It needs investigation. REVISION DETAIL https://reviews.freebsd.org/D5473 EMAIL PREFERENCES https://reviews.freebsd.org/settings/panel/emailpreferences/ To: iateaca, grehan, neel, tychon, mav Cc: freebsd-virtualization-list ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
