Re: Any way to trigger system_powerdown from a signal?
On 2024-07-24 19:35, Brett Neumeier wrote: I'm trying to set up supervision for a QEMU virtual machine on a machine that uses s6 and s6-rc for service management. I currently have my QEMU configured so that it shuts down gracefully when the "system_powerdown" monitor command is executed. With s6, though, the only idiomatic way that I can get the supervision frameowork to shut down a long-running process is to send it a signal. I've verified that I can get QEMU to terminate by sending it a SIGINT, SIGTERM, or SIGPWR; but all of those just cause it to terminate, they don't send an ACPI shutdown request to the guest operating system. Is there any way to trigger the same "system_powerdown" mechanism by sending QEMU a signal? If not, can anyone suggest a way to add a signal handler that does that? The method I usually use is to have the "monitor" listen for interactive human commands on a per machine AF_UNIX socket like /var/lib/qemu-wrapper/vmfoo/vmfoo.mon, then have my shutdown script does this (within appropriate robustness conditions and checks): # echo system_powerdown | socat - unix-connect:$socknam I suspect Red Hat libvirt does something similar. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: recover snapshot
On 2024-07-18 19:23, Pierrick Bouvier wrote: Hi Jakob, On 7/18/24 07:50, Jakob Bohm via wrote: On 2024-07-16 22:22, Leonel Garcia Rosas wrote: I accidentally delete the backing file. It was a very old 200GB file from 2018. The snapshot is 1 TB and I can't use it. My server is down. I tried to recover the file with all the tools I found (testdisk, photorec, rr-studios) with no luck. I just issue a qemu-img info on the snapshot and the error shows up, and I unmount the partition. Remount in ro and use the tools I mentioned. Is there something I can do to recover the files in the snapshot? It is the /home directory from my server and I have customers waiting for email and stuff. I'll pay for the advice or the service Thanks in advance! Leonel Well, you deleted all the virtual disk sectors that were unchanged since the backing file. To get back all disk sectors changed since the backing file, just repoint the qcow2 image to an all zero backing file . Interesting, it's an idea I suggested on IRC (where Leonel came first), but someone mentioned it would not work. Will snapshot apply the diff "blindly", without checking src block (in backing file) was the one expected? I believe qcow2 itself has no information about the expected sector contents, but the file system and applications in the virtual machine might. I don't know the snapshot/qcow2 implementation details, and I wonder if there is some kind of checksum regarding the original disk, or simply a list of changes (write this block with new content X at addr Y). DO NOT CREATE A NEW ALL ZERO 200MB FILE, it might overwrite any locally recoverable part of the old backing file. Instead perform the recovery on a different real disk . Also look at file recovery tools for the file system that stored the lost 200MB file. Ignore any AI Slop articles that just tells you, you should have made a backup or installed some delete protection software before the mistake. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: recover snapshot
On 2024-07-16 22:22, Leonel Garcia Rosas wrote: I accidentally delete the backing file. It was a very old 200GB file from 2018. The snapshot is 1 TB and I can't use it. My server is down. I tried to recover the file with all the tools I found (testdisk, photorec, rr-studios) with no luck. I just issue a qemu-img info on the snapshot and the error shows up, and I unmount the partition. Remount in ro and use the tools I mentioned. Is there something I can do to recover the files in the snapshot? It is the /home directory from my server and I have customers waiting for email and stuff. I'll pay for the advice or the service Thanks in advance! Leonel Well, you deleted all the virtual disk sectors that were unchanged since the backing file. To get back all disk sectors changed since the backing file, just repoint the qcow2 image to an all zero backing file . DO NOT CREATE A NEW ALL ZERO 200MB FILE, it might overwrite any locally recoverable part of the old backing file. Instead perform the recovery on a different real disk . Also look at file recovery tools for the file system that stored the lost 200MB file. Ignore any AI Slop articles that just tells you, you should have made a backup or installed some delete protection software before the mistake. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: Syncing the guest display resolution with the host in fullscreen
On 2024-07-08 08:00, Thomas Huth wrote: On 06/07/2024 00.46, Anton Shepelev wrote: Thomas Huth: I had a quick look, but if I got this right, 1366x768 is not a mode that fits into the standard EDID information, since 1366 is not dividable by 8. Thank you very much Thomas. That also explains why get- edid(1) does not work on this laptop. See also https://en.wikipedia.org/wiki/Extended_Display_Identification_Data#Limitations for some more information. It says: For 1366x768 pixel Wide XGA panels the nearest resolution expressible in the EDID standard timing descriptor syntax is 1360x765 pixels, typically leading to 3 pixel thin black bars. I can live with 3-pixel black bars, if QEMU will only enter a corresponding full-screen mode. What I cannot live with, is a scaled image with intepolative blur or other artefacts. QEMU already enables 1360x768 in the EDID information, see: https://gitlab.com/qemu-project/qemu/-/blob/master/hw/display/edid-generate.c#L18 ... but I've got no clue why guests don't offer that screen resolution ... I still assume this resolution is something special that the driver in the guest has to support explicitly. 1366x768 is a very common display size in real hardware, as it is the closestapproximation to 16:9 at 768p, see the discussion at https://en.wikipedia.org/wiki/Display_resolution_standards#1366x768 Commercial display drivers are likely to have it as a tested use case, possiblywith some non-EDID detection method.Maybe the DisplayID data format allows returning 1366 pixels as width even when delivered over an EDID connection. Duckducking for 1366x768 EDID returns some troubleshooting conversations about making various drivers (including the X11 nVidia driver) use it, but no explanation of what such monitors typically return in their EDID messages to drivers. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: I doubt here
Only if the BIOS matches the virtual motherboard provided by Qemu . For example, if the BIOS tries to configure a register controlling the USB controller on the real motherboard, it will only work if the virtual motherboard has the same USB controller at the same bus address. The Qemu virtual motherboards are based on specific old Intel chipsets combined with custom virtual chips such as the paravirtual disk controller. It's the same problem as installing a BIOS from a completely different motherboard on a different real motherboard, except you don't need a soldering iron to repair the damage to virtual hardware. On 2024-06-21 15:39, Lucas Machado Zainote wrote: Can I boot a nomral cumputer bios in qemu ? I mean a .bin file. let's say I have a motherborad bios file in this format can I use qemu to test it? thank you for the great software. Lucas Machado Zainote lucasmzain...@aol.com Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: qemu-img convert: Compression can not be disabled when converting from .qcow2 to .raw
Dear Sven, Note that qcow2 files contain data saying how large the virtual disk is and what blocks in the virtual disk correspond to what blocks in the qcow2 file. Thus adding extra all-0 blocks at the end of a qcow2 file using generic file manipulation tools like truncate or dd will not change the size or content of the virtual disk image that can be extracted as a raw file. Instead you need to resize the virtual disk inside the qcow2 file with "qemu-img resize" subcommand before converting to a raw image. Alternatively, you can use the generic file tools to change the size of the raw file directly. On 2024-06-21 16:46, Sven Ott wrote: Hi, I want to mount a VM image to a loop device and give it some excess space. To do so, I download a .qcow2 file, add some 0 bytes with truncate, and then convert the image from QCOW2 to RAW format with qemu-img convert, like so: ``` GUEST_IMG=focal-server-cloudimg-amd64 wget https://cloud-images.ubuntu.com/focal/current/$GUEST_IMG truncate -s 5G $GUEST_IMG.img qemu-img convert -f qcow2 -O raw $GUEST_IMG.img $GUEST_IMG.raw ``` The problem is that the convert command throws away the 0-bytes which have been appended earlier, leaving me with a .raw image of the original size. As per the man page, the resulting image can be optionally compressed with the -c flag, indicating that not providing said flag would lead to an uncompressed resulting image. I'm on Debian on x86_64; I've tried the qemu-img version 6.2 and 8.2 unsuccessfully so far. Any help would be appreciated! Sven Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: target/i386: fix pushed value of EFLAGS.RF
On 2024-06-11 00:44, Paolo Bonzini wrote:
On Tue, Jun 11, 2024 at 12:39 AM Robert Henry wrote:
Paolo:
Regarding your commit to QEMU
https://github.com/qemu/qemu/commit/69cb498c56263a5ae484fd4fef920d3d3eea04c8
Four years ago I reported a bug
https://gitlab.com/qemu-project/qemu/-/issues/249 and as part of cleaning up
prior to retirement, I want to get my patch published.
Oh, thanks for pointing that issue out. I'm happy to help.
I see that your patch has the issue that it doesn't affect PUSHL_RA/POPL_RA.
Also, can you confirm that this is needed:
+ if (/*old_semantics ||*/ cpl == 0) {
+val = cpu_ldq_kernel_ra(env, *sp, ra);
+ } else {
+val = cpu_ldq_data_ra(env, *sp, ra);
+ }
and you cannot just use "val = cpu_ldq_data_ra(env, *sp, ra)"?
Looking at that code, does this imply that Qemu fails to emulate Ring 1
and 2
of the x86 architecture?
Enjoy
Jakob
--
Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com
Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10
This public discussion message is non-binding and may contain errors.
WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: Query related with Emulation on QEMU
[Resending my reply from the right address...] No, No, Yes Sadly, there seems to still be no IA64 CPU emulation in Qemu (even the ability to run QEMU on actual IA64 hardware has been removed) or anywhere else, forcing all but the richest companies to not develop IA64 software while the architecture existed. For SunOS, look at the x86/x64 version of SunOS for emulation on x64 in KVM/HAXM mode. Or use qemu-system-sparc under Sparc Linux for hardware speed or under any other hardware architecture with only the quick TCG emulation speed. On 2024-05-30 08:10, Prabhu C wrote: Hi QEMU team, I have a query on QEMU emulation capabilities. I have a system with RHEL 8/CENTOS 8 installed with QEMU. Can I emulate any of below hardware platform on top of RHEL 8 or CENTOS 8 using QEMU? 1) HP-UX 11.23 ia64 2) HP-UX 11.31 ia 64 3) SunOS 5.10 If yes, Can you give me some reference materials to perform the emulation of the above hardware using QEMU? Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: Query related with Emulation on QEMU
No, No, Yes Sadly, there seems to still be no IA64 CPU emulation in Qemu (even the ability to run QEMU on actual IA64 hardware has been removed) or anywhere else, forcing all but the richest companies to not develop IA64 software while the architecture existed. For SunOS, look at the x86/x64 version of SunOS for emulation on x64 in KVM/HAXM mode. Or use qemu-system-sparc under Sparc Linux for hardware speed or under any other hardware architecture with only the quick TCG emulation speed. On 2024-05-30 08:10, Prabhu C wrote: Hi QEMU team, I have a query on QEMU emulation capabilities. I have a system with RHEL 8/CENTOS 8 installed with QEMU. Can I emulate any of below hardware platform on top of RHEL 8 or CENTOS 8 using QEMU? 1) HP-UX 11.23 ia64 2) HP-UX 11.31 ia 64 3) SunOS 5.10 If yes, Can you give me some reference materials to perform the emulation of the above hardware using QEMU? -- Jakob Bohm, CIO, partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Soborg, Denmark. direct: +45 31 13 16 10 This message is only for its intended recipient, delete if misaddressed. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: base format
On 2024-04-25 14:51, lacsaP Patatetom wrote: Le jeu. 25 avr. 2024 à 11:30, Peter Maydell <mailto:peter.mayd...@linaro.org>> a écrit : On Thu, 25 Apr 2024 at 09:55, lacsaP Patatetom mailto:patate...@gmail.com>> wrote: > > hi, > > when using `qemu-img create`, why do I have to specify the format of the base image ? > can't `qemu-img` detect it itself ? Image format detection isn't 100% reliable. Notably, a 'raw' format image could in theory look like any of the other image types if it happens to start with the right kind of data. thanks -- PMM OK for the raw format, which can be anything, but not for the qcow2 format, which is standardized (header). in the absence of details, `qemu-img` could use the qcow2 format by default ? qemu-img create [-b base [-F (qcow2*|raw)]] [-f (qcow2*|raw)] image [size] But it cannot safely tell if the base file (usually a raw image or raw physical disk partition) is qcow2 or araw file that just happens to begin with the magic bytes that represent qcow2. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: Freebsd 12 strange disk behaviour on QEMU machine
Using Linux Guests on older qemu versions, I have seen some bad disk speed effects of snapshot mechanisms, but not your particular scenario: 1. When creating a snapshot and until all snapshots are deleted, all disk writes are redirected to a delta storage area that keeps track of which virtual disk blocks are different from the snapshot contents and what that different block contents is. Read access thus gets overhead from doing the lookup and occasional redirected read. Write access gets much more overhead from adding delta entries and redirecting the write. This is inherent in the snapshot mechanism. 2. In snapshot mode I have seen general I/O (including network and/or basic CPU stuff) getting slowed down so hard that the outside cannot even ping the virtual machines, indicating that someone in the qemu team made the mistake of doing the disk snapshot overhead in the same execution thread as basic emulation tasks. Later qemu developers get so focused on spitting out replies to rookies that they fail to realize the fundamental problem. 3. I have not seen slowdowns happen after snapshot removal, but cannot rule out that someone made further mistakes in I/O handling. On 2023-09-20 15:40, Roland Giesler wrote: Has anyone here ever has a similar occurrence? It's quite perplexing and I'm surprised that I can't find anything about this on the web. On 2023/09/18 16:03, Roland Giesler wrote: We have a FreeBSD machine running an IRIS poller that behaves really strange. We had FreeBSD 12.3 at first, then changed to 12.4. Same result. We also changed from BIOS to UEFI. Same result. The storage is ceph RBD on NVMe SSD drives, so it's fast. The symptoms are as follows: After installation of the OS and the script (most perl) that IRIS uses, the service run perflectly. I then take a snapshot. If the machine is restarted for whatever else reason everything comes up, but the FreeBSD processes go into a "D state", meaning they are waiting for the disk. This happens very time. It's then not possible to fix the machine, although all seems quite normal and not config changes took place that we can detect. If I roll back the snapshot, things are normal for a while put soon processes go into D state again. I'm not sure that this is a qemu problem, but we have to start somewhere. Finally, I not have the VM running an a local volume (non ceph, standalone SSD) and there's no problem. This is actually an installation that was done on ceph and then the disk volume was moved to the current location. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: OpenGL 4
On 2023-04-17 16:50, Johannes Scheyerle wrote: Hey guys, I'm using an Macbook Air M1 and looking for a possibility to use Windows on this computer. The most important thing is, that OpenGL 4 or higher supported. Can your System do this? Best Jo Many versions of qemu-system allow providing the virtual machine with emulated "qxl" accelerated graphics hardware to be executed on a "spice" client. I strongly suspect that this will provide the desired 3D capabilities needed to run the Guest with Windows OpenGL drivers for "qxl" "hardware". Unfortunately, The qemu 5.2.0 manual page did not provide meaningful details of this, blindly assuming that all readers know those things already. Hopefully later releases have corrected that. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: Windows NT 4.0
On 2023-02-08 14:55, Wilson Birney wrote: I have a windows NT 4.0 raw image that when converted with qemu-img to qcow2 or vmdk fails to boot in qemu with no obvious error using the following: qemu-system-i386 -m 128 -cpu pentium -hda f.qcow2 It just hangs at 'Booting from Hard Disk" and pegs out a cpu core. This same image boots just fine in vmware Workstation and virtualbox when i convert to a vmdk. Any ideas on what could be causing this or what to do to diagnose it? Check the virtual hard disk interface. The NT 4.0 boot process relies on some files in the root directory providing details of that hardware, for example the driver for virtio-scsi or vmware-scsi. Or an indication to use standard IDE/SATA drivers. This also includes which partition contains Windows. Some of this is in the text file BOOT.INI, other parts are in code files in the root, such as a .SYS file for the SCSI driver. As a result, any virtual disk for running NT 4.0 under virtualization will be specific to the disk, display and CPU hardware of that VM configuration. Fortunately however, there are qemu utilities to mount the virtual disk under the host OS, where an appropriate file system driver can access the FAT, NTFS or HPFS contents for r/w and thus potentially change of this hardware information. To access registry settings it is more practical to mount the disk as a secondary disk in a VM that boots from a minimalist Windows system such as WinPE, from which the secondary disk registry files can be mounted elsewhere in the registry tree for editing. -- Jakob Bohm, CIO, partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Soborg, Denmark. direct: +45 31 13 16 10 This message is only for its intended recipient, delete if misaddressed. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: qemu won't log files (probably user error)
Dear angry bum, That the output is already being sent to stdout and simply needs to be captured like for any other command outputting messages there. On most operating systems, this would involve basic redirection $ qemu-system-x86_64 -cpu \? | more OR C:\foo> qemu-system-x86_64 -cpu ? | more On 2022-12-30 16:23, IndignantHomeless wrote: Could anyone explain how to get the Qemu to output a log file into a specified directory so I could cue the "-cpu ?" and "-M ?" commands without having to belligerently slam on the pause button? I've tried "-monitor stdio" invoking log page,unimp,guest_errors and it never outputs a file. I also tried "-D **\a log.txt" (**=variable path directory) with and without premade text file. Any further input would be nice! Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: If your networking is failing after updating to the latest git version of QEMU...
On 2022-09-29 08:34, Thomas Huth wrote: On 29/09/2022 04.32, Jason Wang wrote: On Thu, Sep 29, 2022 at 1:06 AM Philippe Mathieu-Daudé wrote: On 28/9/22 10:27, Thomas Huth wrote: ... it might have happened due to the removal of the "slirp" submodule from the git repository. For example if you see an error message like this: Parameter 'type' expects a netdev backend type this likely means that the "user" mode backend type is not available in your binary anymore. To fix it, simply install "libslirp-devel" (or libslirp-dev or however it is called) from your OS distribution and recompile. Thanks for the hint Thomas. I'm afraid many developers will miss your email. Jason, Marc-André, could we improve the buildsys check or display a more helpful information from the code instead? It looks to me we need to improve the build. I'm not sure there is anything to improve in the build system - configure/meson.build are just doing what they should: Pick the default value for "slirp" if the user did not explicitly specify "--enable-slirp". But the error message is not very helpful. It should rather say something like (partly suggested by Daniel in IRC yesterday already): Type 'user' is not a supported netdev backend by this QEMU build. Please check the spelling or whether it has been enabled at compilation time. ... or something like this. Someone interested to write a patch? Maybe a more actionable error message such as: Type 'user' is not a supported netdev backend by this QEMU build because the libslirp development files were not found during build of QEMU. The condition for this error message should be that both the user backend is not compiled AND the build system did not detect libslirp. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: Boot order not working with OVMF
On 2022-08-21 08:45, Christopher Snowhill (kode54) wrote: On Aug 20, 2022, at 7:30 PM, x...@trimaso.com.mx wrote: Hello. I noticed that "-boot order=dc" -for example- seems to be only functional in SeaBIOS, but totally ignored in OVMF. With this last one I need to use "-boot menu=on", press Esc, and manually edit boot order. Is there a reason for this? Oh, and forgot to mention in previous thread, I'm currently using QEMU 6.2. Thanks again for your attention. You should be specifying bootindex (integer) values on the boot devices themselves. At least that's what Proxmox does when it automates qemu-kvm command lines. IMHO it would be really good IN GENERAL if qemu promised to map equivalent options to each other as long as both syntaxes exist. So if there's both a boot order option and a bootindex option, both will be internally mapped to a single implementation variable or list, which would then be passed to emulated interfaces such as those read by SeaBIOS, OVMF, OpenFirmware etc. etc. The qemu man pages should say which syntax takes priority over the other if both are input. This would also apply to historic redesigns such as the change from device-specific options like "-hda" to generalized options like "-drive" or "-device". Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: linux shell comes out to qemu-monitor when I enter anything. (I have two uarts)
On 2022-07-20 14:35, Peter Maydell wrote: On Wed, 20 Jul 2022 at 13:14, Chan Kim wrote: Hi, Peter Maydell, Thank you for the advice. So I uncommented the stdio_in_use guard, and used this option (because the second uart will be used) -chardev null,mux=off,id=char0 -serial chardev:char0 -chardev stdio,mux=off,id=char1 -serial chardev:char1 And it gives me : qemu-system-aarch64: cannot use stdio by multiple character devices That isn't a full QEMU command line. Either something else on your command line is also using 'stdio', or you've passed an option that implicitly says "use stdio" (maybe -nographic ?), or you have code in your device model that's hardwiring use of the stdio chardev. -- PMM It would be a lot more helpful if your error message was explicit about which items were conflicting rather than demanding the full configuration from people affected. And don't fall for the temptation of printing out a list of potential reasons in the fixed error text, actually print the actual data triggering your error case. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: Software Safety Impact in Qemu
Please note that it is entirely possible for an exact version of a GPL program to be certified (by a 3rd party standards testing organization) to meet any specific standard. This does not constitute a warranty from the volunteer programmers, merely a valuable feature. One such example that was historically promoted by the FSF is POSIX certification of GNU systems. On 2022-07-01 12:47, Alex Bennée wrote: asif siddiqui writes: Hello All, This is regarding the functional safety impact of software being run in qemu. Is there any document or link reference which talks about qemu being ISO 26262 certified and functional safety compliant. Not that I'm aware of. The GPLv2 itself specifies: NO WARRANTY 11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION. 12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. If you want specific warranties that would the sort of thing you would talk about with a distributor under some sort of service contract. We also explicitly rule out emulation from providing a security boundary: Bugs affecting the non-virtualization use case are not considered security bugs at this time. Users with non-virtualization use cases must not rely on QEMU to provide guest isolation or any security guarantees. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: Share host memory with a guest running no operating system
On 2022-06-21 03:38, John Ramsden wrote: I'm attempting to share host memory with a guest running no operating system using `-object memory-backend-file,id=mem0,size=4G,mem-path=/dev/shm/qemu,share=on`. When I try this using a guest with an operating system such as Ubuntu, I see my object mem0 in `info mtree`, and I'm able to write to `/dev/shm/qemu` on the host and read the changed memory on the VM - similar to this blog post (I am not the author)https://blog.reds.ch/?p=1379. This doesn't seem to work when I have a guest without an operating system, there is no `mem0` device. In case it's relevant I am using an arm64 guest. Do I need a guest operating system to use this feature? It is important to understand the difference between the two qemu variants available for each architecture: In the qemu-variant named qemu-archname-system, your guest runs as bare metal program (like memtest86) that can be run on a computer without booting an operating system, there are no device or file names in the guest, only hardware addresses that access the various devices. Essentially, such programs are their own primitive OS. In the qemu-variant named just qemu-archname, your guest is a Linux application compiled for the wrong CPU, it sees exactly the same device names and files as the host machine, as that qemu program only simulates the CPU instruction set, not an entire machine, device simulation options make no sense for that qemu variant. -- Jakob Bohm, CIO, partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Soborg, Denmark. direct: +45 31 13 16 10 This message is only for its intended recipient, delete if misaddressed. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: trying to get qom-{get,set} working with gpio on qemu-x86_64
On 2022-02-25 21:48, John Snow wrote: On Thu, Feb 24, 2022 at 6:39 AM Ben Dooks wrote: On 24/02/2022 11:05, Peter Maydell wrote: On Thu, 24 Feb 2022 at 10:53, Ben Dooks wrote: I have been trying to get qom-get/qom-set scripts working to make an virtual GPIO device on qemu-system-x86_64. I have a working TMP105 and I can see the MAX7310 being instantiated with the relevant Linux devices showing up (i2c, gpio). I can read the sensor from QEmu console: (qemu) qom-get /machine/peripheral/sensor temperature 2 (qemu) qom-set /machine/peripheral/sensor temperature 19000 (qemu) qom-get /machine/peripheral/sensor temperature 19000 When trying to get or set anything from the GPIO chip it doesn't seem to be working. How do I get this working, I haven't found any useful reference online and need some help: (qemu) qom-set /machine/peripheral/gpio unnamed-gpio-out[0] 0 Error: Device '0' not found (qemu) qom-set /machine/peripheral/gpio unnamed-gpio-in[0] 0 Error: Insufficient permission to perform this operation The 'unnamed-gpio-out/in' properties are part of the mechanism by which a QEMU machine model can connect up GPIO lines of devices it creates (for instance, it might connect a device's outbound GPIO line to a GPIO controller). They appear in the monitor qom-set/qom-get APIs because they use the same underlying infrastructure as user-settable properties, but you can't do anything useful with them on the command line. If you need to wire up a device's GPIO lines, you need to do it in C code as part of the QEMU machine model implementation. Put another way, -device max7310,id=gpio,address=0x20 will work fine for "put a gpio controller at this i2c address, so that guests that expect to be able to talk to one there don't crash". You can't however actually wire up any of that controller's GPIO lines to anything (emulated LEDs, other devices, etc, etc). Ok, thanks. I was really trying to do something simple. I want to add the GPIO chip and just control the lines from an external script as part of a testing environment where the test framework can change a GPIO value and the configuration in the machine will notice this and take appropriate actions. Is it worth adding some mechanism to allow the command line to specify how gpios are created, or have an option to just export all the GPIO lines as generic GPIOs? I'll have to go see how much effort it would be to make our own machine file. I am also having issues with both sensor and the gpio via the qom-set, as so: $ sudo ./scripts/qmp/qom-set --socket ~/qmp.sock /machine/peripheral/sensor.temperature 20200 ExecuteError: Invalid parameter type for 'temperature', expected: integer Task was destroyed but it is pending! Is this also a known issue? I'm not sure what's going on there: it sounds like something is confused about types. I've cc'd the python qmp scripts maintainer. Howdy. The qom-set and qom-get tools have been around for quite a while longer than I have been taking care of them, and I wasn't sure if anyone was actually using them... They *do* have a defect where they assume anything you enter is a string and they send it that way over the wire. If the argument you are sending actually requires an integer, it's going to fail. I had a discussion recently on how we might be able to fix this, but since QOM data is ... semi-opaque and incomplete at times, there's no real way to know the correct type in advance. If it would help people out, I could change the way the scripts work to place the burden on the human user, i.e. qom-set \"20200\" qom-set 20200 would become explicitly different invocations. I really genuinely have no clue if anyone is relying on this script though, so I've been afraid to really do much with it. Maybe breaking some users of this script in order to prevent new users from learning it doesn't really work is a fine trade-off, though. Nice, but just in case there are robotic users (scripts, front ends etc.), I would recommend choosing a syntax for parsed mode that cannot be a valid input for the old unparsed (string only) mode. Depending on context (I don't know and use the specific command), this could be an option argument (if options cannot be valid strings) or an alternative command name, like qom-set2 . Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
qemu-img check fills terminal buffer with Leaked cluster 12345678 refcount=3 reference=2
Dear qemu experts, I recently ran qemu-img check on a large .qcow2 file, and got thousands (maybe millions) of output lines of the form: Leaked cluster 12345678 refcount=3 reference=2 This seems like a bug in qemu-img check code, as repeating the same message for every cluster in a huge range is excessive and prevents terminal buffers from retaining earlier output that may or may not happen before this barrage of noise (rerunning the check with a grep filter showed a few ERRORs about clusters with reference > refcount, and a lot of other clusters with reference=0). Furthermore, the message seems to be factually incorrect, as a cluster with at least one actual reference is not "leaked", even if its reference count is higher than reality. I kind of suspect this to be related to lazy_refcounts in some way, but can't be sure. Running qemu-img 5.2.0 (plus Debian patches), trying to sanity check the backup of a large qcow2 file (> 400GB). Not using libvirt because I am not running a Red Hat system. P.S. My backup method may be somewhat wonky, but there are no clear instructions on how to backup a savevm state without stopping the virtual machine or redefining the host storage stack in weird ways. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: Save qemu state
On 2021-12-01 21:53, Peter Maydell wrote: On Wed, 1 Dec 2021 at 19:28, Leek, Jim wrote: So, why doesn't QEMU support external checkpoints? (ie an option where checkpoints each get written to a new file.) If you really want to save to an external file I think you can do this by treating it as a migration (which is the same infrastructure underneath for snapshotting the machine state, but it sends it over eg a file descriptor or a socket rather than saving it into a qcow2 file). David Gilbert gives a set of commands that might work for this in this email thread: https://lore.kernel.org/qemu-devel/YRvjHiZmPkSuv%2F%2Fz@work-vm/ # (qemu) migrate_set_parameter max-bandwidth 100G # (qemu) migrate "exec:cat > myfile.mig" # (qemu) q # # qemu -incoming "exec:cat myfile.mig" I haven't tried them, but this sort of thing is probably a good place to start if you want to investigate more complicated things than "just save to the qcow2". My guess at the reason why savevm requires qcow2 is because qcow2 provides a builtin mechanism for taking a low-cost snapshot of the state of the disk image -- otherwise you would end up needing to put the whole of your raw disk image contents in the snapshot, or otherwise manually arrange to do something to keep a copy of them. So it probably didn't seem like a big deal to tie the vm save/restore to qcow2 back when that code was written. (Looking into the depths of the git history it looks like the very first incarnation of 'savevm' really did take a separate filename to write the state to. Commit faea38e78 in 2006 is when that changed.) These days, the migrate command provides the power and flexibility that is needed by the main use-case of the ability to checkpoint the VM, ie migrating the VM from one host to another. The use as a developer-tool convenience is something we get as a handy side-benefit, not something anybody's actively paid to improve the ergonomics of. This brings back the closely related topics of how to reliably backup the qcow2 file containing the savevm as part of scheduled system backups. Currently this suffers from the qcow2 code not guaranteeing non-write to the host file blocks containing the snapshot metadata and the snapshotted virtual disk blocks. -- Jakob Bohm, CIO, partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Soborg, Denmark. direct: +45 31 13 16 10 This message is only for its intended recipient, delete if misaddressed. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: Using LUKS format to connect to an encrypted iscsi volume with libiscsi
On 2021-10-06 20:52, Will Gorman wrote: I'm attempting to use qemu-kvm (qemu-kvm-ev-2.12.0-44.1.el7_8.1) to run a VM that will be able to use an iscsi volume that has been encrypted with LUKS. Below are the qemu command line arguments related to this volume: -object secret,id=scsi1-0-0-1-luks-secret0,file=/root/qemuluks.key \ -drive file.driver=iscsi,file.portal=$TARGET_IP:3260,file.target=$TARGET_IQN,file.lun=0,file.transport=tcp,file.initiator-name=iqn.1994-05.com.redhat:host1,key-secret=sec0,format=luks,if=none,id=drive-scsi1-0-0-1 \ -device scsi-block,bus=scsi1.0,channel=0,scsi-id=0,lun=1,drive=drive-scsi1-0-0-1,id=scsi1-0-0-1 \ I think (from the horribly incomplete documentation) that the built-in qemu LUKS encryption is ONLY for qcow2 disk image files, not for any kind of "raw" disk, even if remote over iSCSI. When running the VM with qemu-kvm, I get the following error: 2021-09-22T20:26:04.975007Z qemu-kvm: -device scsi-block,bus=scsi1.0,channel=0,scsi-id=0,lun=1,drive=drive-scsi1-0-0-1,id=scsi1-0-0-1: cannot get SG_IO version number: Operation not supported Is this a SCSI device? I think that it is at least using the key since if I intentionally provide an incorrect value for the key I get a different error about "Invalid password, cannot unlock any keyslot" but it gets further with the correct key. Is it supported to use LUKS with the iscsi driver and libiscsi? If so, are there any other configuration options I should be considering? Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Accessing the contents of a savevm snapshot
For purposes of doing minimum-interruption backups of virtual machines, I am looking for ways to access the contents of copying/reading a savevm snapshot while the VM continues running past the time of the snapshot. This will later be followed by a delvm HMP command to discard the temporary snapshot and resume non-qow use of the disk image file. However looking through the (scattered) qemu documentation, I see no commands to access the snapshot while the VM continues to run. Neither the disk contents snapshotted by the savevm command, nor the VM state also saved by the savevm command. What am I missing? Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
How to clean up unused qcow2 clusters
Hi, As is well known, qcow2 files contain a large number of reference counted clusters that correspond roughly to file system blocks. As snapshots are deleted, many clusters will become logically freed (they are no longer referenced by any image or snapshot). My main question is how to remove, or at least zero out, those unused host clusters, assuming the qcow2 file is not busy (because the virtual machine is stopped and the virtual disk is not being accessed with any other tool)? My secondary question is why the qcow2 specification (qcow2.txt) is not installed with the other documentation files, and why there is no corresponding web page (not even in the qemu Wiki)? Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: What does qemu-img convert -W documentation mean?
On 2021-05-26 15:40, Richard W.M. Jones wrote: Firstly there's a real interesting thread about nbdcopy vs qemu-img convert performance: https://listman.redhat.com/archives/libguestfs/2021-May/thread.html#00119 I have a question about qemu-img convert -W flag. The documentation says: -W Allow out-of-order writes to the destination. This option im‐ proves performance, but is only recommended for preallocated de‐ vices like host devices or other raw block devices. I don't understand why out of order writes are bad for things like raw files and qcow2 files, or for non-preallocated block devices. Wouldn't they always be a win? Rich. Out of order writes to things that allocate one block at a time from the lower layer would cause the mapping from virtual clusters to clusters at the lower layer to befragmented, literally out of order, which hurts performanceof future access. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: Converting QEMU .raw to VMDK VMware
On 2021-04-14 19:40, Terrance Battle wrote: Hi, I have a question, how do I go about converting a .raw snapshot to VMware VMDK? We’re looking to move the .raw snapshot to our new VMware environment for DevOps. Thanks, Look at qemu-img, it can convert between many virtual disk formats, including the generic .raw format and an older version of the VMWare VMDK format. Another option is to look at VMWare conversion tools, as they should support the generic raw format too. Since you seem new to this, please note that you may also have to reconfigure the OS inside the virtual machine to support the VMWare virtual hardware as opposed to the qemu virtual hardware. VMWare conversion tools may have options to do that appropriate for your current VMWare product versions. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: shared disk (DOS)
On 2021-04-13 11:42, Tomas By wrote: On Tue, 13 Apr 2021 11:05:54 +0200, Dennis Luehring wrote: Am 13.04.2021 um 11:00 schrieb Tomas By: Or, put another way, how do I set up a 1990s DOS local network? for example [...] No I think the MS stuff looks better. Please refer to one of my earlier posts where I gave a summary of how to make a Linux host machine provide the server side of the SMB network used by MS network client. It is important to note that modern versions of SAMBA (and equivalent Microsoft software) default to a security setting that rejects the weak password encryption used by many historic DOS era clients. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: shared disk (DOS)
On 2021-04-12 21:18, Tomas By wrote: On Mon, 12 Apr 2021 11:16:21 +0200,Peter Maydell wrote: No, that doesn't work. [...] On Mon, 12 Apr 2021 12:17:08 +0200, Peter Maydell wrote: In theory if the different guests all access entirely disjoint sections of the file this could be made to work. [...] They can access the exact same data also, it would just mean some delay for one or more of them, and maybe some arbitrary decision about the sequence. (But if they use SHARE.EXE on the DOS side, as I understand it to work, then there will not be concurrent access to the same DOS files. And then it's a question of the mapping from those to image file locations.) You still misunderstand SHARE.EXE (a DOS only thing). SHARE.exe makes DOS handle file locking calls (which cc:mail may happen to use for something). It does nothing for sharing disks. With network file systems such as SMB and/or NFS, SHARE.exe may also provide some of the glue needed for file locking calls from programs (like cc:mail) to reach the file server (which will have code to handle locking from multiple DOS machines accessing the same file). In general, you should treat qemu virtual machines almost like real machines. Wiring multiple machines to access a single physical disk will involve use of SCSI cables coming out the back of the computer and connected to a single physical disk. This has always been an exotic thing to do, and it was never supported by DOS. -- Jakob Bohm, CIO, partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Soborg, Denmark. direct: +45 31 13 16 10 This message is only for its intended recipient, delete if misaddressed. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: Question about harddrive
On 2021-04-11 00:31, esposito mikael wrote: Hi, I have juste a little question, i just start to use qemu for emulate arm computer. And i can't change the hard drive description for change QEMU HARDDISK for something like DELL. For information, i don't use a virtualise storage i boot from a modified img file. Thx for your help. Best regard *Cordialement,* *Mikael Esposito* *Qemu *hard drives have two names: The hardware model name (such as Seagate Barracuda or "Qemu Something"), I don't know an option to change that. The volume label written in the file system, this can be changed as an option to mkfs or with a file system specific utility such as e2label, ntfslabel etc. The software running on the virtual RPi probably includes the relevant command. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: shared disk (DOS)
On 2021-04-10 18:02, Tomas By wrote: On Sat, 10 Apr 2021 17:57:26 +0200, Peter Maydell wrote: This is a guest configuration question. There is no magic "tell QEMU to use this format for a shared disk image and it will all work" option. You'd need to run *one* guest connected to the disk, and have it serve access to that disk over the network, that your other guests access only as a network drive. The page I linked to seems to talk about sharing a disk image, though. /Tomas You are both wrong about SHARE.EXE, probably due to the sloppy documents written about it for many years. SHARE.EXE implements the parts of the DOS kernel that handles multiple programs (on ONE machine, not many) accessing the same file. It makes programs that do file locking etc. work at all. Running any kind of multitasking on top of DOS (like Topview or MSWindows or a DOS-based file server) makes it extra important. SHARE.EXE does nothing to help two DOS machines accessing the same disk at the same time. The only available DOS file systems are the FAT file system, the read only CD-ROM file system and network file systems. Thus for a writable disk, there is no DOS feature to share a disk image, while a read only disk (where the virtual hardware returns error for any attempt to write to disk) would work just fine, especially because of the long DOS history of booting from a read-only floppy and saving files on a writable floppy in the second drive. DOS sees (virtual) hard drives as really big floppies that cannot be popped out of the drive/reader before (virtual) power off. The network file system option goes like this: 1. Install Samba on the Linux host and share a read/write directory. (I assume you are not ready to deal with the limited Linux support for IPX or NetBEUI). 2. On a DOS boot image (which will be made read only later) install software to access SMB file systems, such as the Microsoft network client that was historically used to access Windows NT or OS/2 file servers. 3. Configure DOS to "map" the shared network location to an unused drive letter such as Z: on startup. 4. Finalize the configuration and shut down to make the disk image read only from the Linux side, typically by changing the qemu command line. 5. Optionally configure Qemu to make a temporary blank disk image available to each virtual machine as a second hard drive. Use that for temporary files in your DOS programs and scripts, typically by setting the various environment variables such as TMP, TEMP, TMPDIR etc. in AUTOEXEC.BAT. This is easier if you have an image of an empty disk that can be gunzipped to a file under a tmpfs just before starting qemu. Besides networking, other ways to get data in/out of DOS are these: S. Tell qemu to map the virtual serial port (at extremely high speed) to host (Linux) side files or software, then have DOS code read/write there. I hope qemu serial ports can be configured in a lossless mode that never overruns or underruns, even if the guest software can't keep up. F. Prepare a temporary FAT disk image (loop mount or mtools) for each qemu run and make it available as a floppy or 3rd disk, where DOS programs can read instructions and/or write results. Note that DOS does very little write caching, provided you wait 2 seconds from last program exit before power off. That's because back in 1980, the fastest guy at Microsoft couldn't change 5.25" floppies faster than 2 seconds. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
User level instructions for modern qcow2 encryption?
A few versions ago, qcow2 encryption was completely redesigned based on LUKS full disc encryption instead of the old ad-hoc design in older qcow2 versions. However, looking in the end user documentations (mostly man pages), I have been unable to find coherent high level documentation directed at qemu users (as opposed to design documents for qemu developers). Main questions: 1. From what minimum qemu released version (corresponding to official tarballs) is the modern qcow2 encryption and its user interface (qemu-* command lines and human monitor commands) fully implemented and stable? 2. Where can I find a complete configuration example using only qemu-* binaries (not libvirt, proxmox or other high level wrappers)? Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: Please help.. I wish to load .elf file directly for baremetal run..
You have to think like a real bare-metal programmer. First think of your computer as a real piece of hardware made from real chips with real data sheets and real PCB tracks connecting them. This provides you with a set of limitations and opportunities. Write (or borrow) your own code to load your linker sections at desired RAM addresses from the contents of your Flash EEPROM chip, this could be a simple as doing memcpy to specific RAM address ranges from specific ROM address ranges, or it could be code that parses some data structures in the ROM image to determine where to copy stuff (Pro tip: Initialize your DRAM controller before using the RAM for anything). Then write (or borrow) a script or program to pack your linker sections into the physical size of your EEPROM chip. Special Note: On the PC platform the ROM BIOS is the only true bare metal program, and does a lot of initial stuff before running the "almost bare metal" OS loader from the main disk image. On 2021-03-17 08:35, c...@etri.re.kr wrote: If I change the linker script so that the program loads and starts at RAM address, I can use -kernel test.elf and run the program. But I’m still curious if I can load the program at 0x0 (ROM area) and use some data at 0x4000 (RAM) like we can do in rtl simulation. Thanks! Chan Kim *From:*c...@etri.re.kr *Sent:* Wednesday, March 17, 2021 3:29 PM *To:* 'qemu-discuss' *Subject:* Please help.. I wish to load .elf file directly for baremetal run.. Hello all, Using the help from this email list, I’m using command below for my baremetal program test on qemu. (-s -S for connecting with gdb) : $aarch64-none-elf-objcopy -O binary test.elf test.bin $cp test.bin pflash.img; truncate -s 67108864 pflash.img //to cut it down to 64MB because the pflash size is 64MB) $qemu-system-aarch64 -machine type=virt,gic-version=3,secure=true,virtualization=true -cpu cortex-a72 -nographic -smp 1 -m 2048 -drive if=pflash,file=pflash.img,format=raw,readonly=on -s - The entry address is 0. The problem is this test.bin can become very big when I have some space between section. For example, if I want to put some initialized data (that can be changed) in RAM at 0x4000, this bin file becomes bigger than 1GB. (because it fills the in-between spaces in .bin file) I tried just using -kernel test.elf but it errors in rom_check_and_register_reset() inside qemu_init. And the arm64 virt machine places dtb in the first RAM area. Isn’t there any option though which I can just load the .elf file (each section to its address) and run from the entry address? and is it possible to disable dtb loading at the first RAM address? Thank you for any help. Chan Kim Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: qemu and Unicode characters on serial console
On 2021-03-14 11:10, Francois wrote: Hi qemu! I am wondering if it is possible to get Unicode support through a stdio, or pty, serial console. I am running qemu from CentOS stream qemu-kvm-core-4.2.0-35.module_el8.4.0+547+a85d02ba.x86_64 My terminal and locale use UTF-8. I am booting into the OVMF boot menu below, that lets me change the language to French, however characters like é è ç are badly rendered. I would expect the UEFI shell "edit" command to support unicode characters but I cannot test this either. My goal is to use Unicode characters in the boot entries (or "options de bottes" as it is funnily translated today) so they look nice. ``` /usr/libexec/qemu-kvm \ -m 512M \ -net none \ -machine q35 \ -pflash ./OVMF_CODE.secboot.fd \ -pflash ./OVMF_VARS.fd \ -serial stdio \ -debugcon file:debug.log -global isa-debugcon.iobase=0x402 ``` Cheers You need to consider which character encoding the UEFI BIOS code assumes for the serial port. Obvious possibilities that could all be used with French: 1. UTF-8 2. ISO8859-1 3. ISO8859-15 (same as -1, except the code for the Euro symbol) 4. MS1252 (Same as -1, but extra characters, including the Euro symbol on an unused number) 5. IBM 437 (Used by the oldest IBM BIOS) 6. IBM 850 (Fewer graphics characters for menus, more accented letters) If the UEFI BIOS uses one of these, but the software processing the serial output uses a different one, the result is very ugly MojiBake. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: IRQ question
On 2021-02-07 15:39, Weiss, Howard wrote: I am running QEMU on Ubuntu. My target system runs Windows 10. I am writing a simulated device to test the a Windows device driver If my Windows device driver is at interrupt level when it invokes my simulated device (eg reads/writes a port) does the code in my simulated device also run ar interrupt level? The Windows IRQL is a variable in the Windows per CPU scheduler code, it controls reentrancy of kernel mode Windows code (inside the virtual machine). Traditionally, it is not exposed in a (virtual) hardware register that qemu could easily monitor, though there might be a HyperV PV interface to expose it. If a Windows kernel mode device driver issues an I/O to a (virtual) IO-space or memory-mapped I/O port, the (virtual) hardware would normally react in the background on its own (virtual) hardware time, later causing something to happen outside the (virtual) CPU, which may at any later time cause a (virtual) input change, such as an I/O port getting a different value, a bus master device writing to (virtual) RAM or a (virtual) interrupt being sent through the (virtual) interrupt controller. That in turn would be picked up by either polling code in the Windows driver, or the Windows kernel invoking the driver to handle the event. So in short, a virtual device wouldn't know or care if the Windows kernel was at any particular IRQL when the virtual I/O port is signalled to do something, but the Windows driver would be designed to do certain things to the virtual device at specific IRQL to avoid certain race conditions against itself. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: Live migration fails with "Mismatched RAM page size ram-node0 (local) 2097152 != 1526773257204281392"
On 2021-02-02 15:21, Damir Chanyshev wrote: вт, 2 февр. 2021 г. в 17:01, Peter Maydell : Are you using exactly the same VM config on both source and destination ? The migration error messages are often rather opaque, but generally they mean "the source and the destination don't match". In this case I think the message means that the hugepage size on source and destination hosts is different. thanks -- PMM Yes, VM config exactly the same on both sides. On hugepage part, guest backed by 2M pages. Is the large number an ASCII string mistaken for a binary number? Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: Recompile QEMU with frame-pointers
On 2021-01-27 15:14, Salvatore Mazzarino wrote: I’m trying to profile my QEMU process but what I get is a stack full of unknown. I would then need to recompile QEMU with -fno-omit-frame-pointer. Do you know if there is a version already built for that purpose? I am not sure, but I suspect that compiler-generated frame pointer code would interfere with the TCG compilation of tiny code snippets to be pasted together at runtime by the translated code generator. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: interrupt 001E
On 2021-01-17 13:13, Tomas By wrote: Hi, On Sun, 17 Jan 2021 02:43:05 +0100, Berto Furth wrote: Interrupt 0x1E seems to be related to "SYSTEM DATA - DISKETTE PARAMETERS" according to Ralf Brown's interrupt list. Could a disk drive be misconfigured? On Sun, 17 Jan 2021 11:52:19 +0100, Berto Furth wrote: I'm afraid I don't really know what's happening (hopefully someone else will!). If I were to guess, I would say it probably has to do with file locking, or whatever is the equivalent concept in DOS. MS-DOS file locking is indeed called file locking and is handled by the DOS kernel extension named SHARE.EXE . It's conceptual logic is completely duplicated by the file sharing rules in Windows NT/Win32, actual API calls and flags can be found in the interrupt list and/or in the MS-DOS Encyclopedia. However given that your program files are named ADMIN.EXE, IMPORT.EXE and EXPORT.EXE, I suspect this to be MS-DOS era networking tools for one of the multiple competing networking stacks, one of which may have used INT 1E for some actual code (the diskette parameter table for PC BIOS is pure data pointed to by the INT 1E vector at real mode address 00078, not code that can be invoked). None of the interrupt APIs in the interrupt list do that though, so you may have to look at where these files are from and run whatever TSR/Driver they depend on. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: Sound error: Could not initialize DirectSoundCapture
On 2020-12-15 12:46, kazusanosuke_kaz wrote: Dear all! I appreciate your kindness. After all, I could hear the nostalgic startup sound by sb16 emulation. I'm relieved to hear the error is harmless. There still remains another problem. Sometimes I can hear the sound, sometimes I can't. It is unclear what causes this behavior. I have never been distressed in this way by actual machines, actual sb16 ISAPnP or other sound devices. There still remains yet another problem. FPU's benchmark score(1) is equivalent to classic pentium @ 30Mhz. On the other hand, the score of integer instructions is equivalent to classic pentium @ 166Mhz, The window system is so slow as you can see the movement to open a tree of Explorer, default file manager of windows 98. I wonder if it's becouse of the CPU emulation. It seems SSE is not used, vt-x doesn't work. The scores are far from them when I use VirtualBox using vt-x. (1) I used HDBench, the most famous software in Japan when people use windows 98 on actual machines. Please note that CPU emulation via VT-x on a Windows Host machine requires installing and using the Intel HAXM accellerator, which acts as the device driver for accessing VT-x (similar to what the kvm kernel module does on Linux). These problems are not urgent or important. Just FYI thanks, -- Yasuhide Omori Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: How to simulate a device which generates an interrupt every 8.3 ms
On 2020-12-10 12:07, Peter Maydell wrote: On Thu, 10 Dec 2020 at 05:28, Weiss, Howard wrote: Hi – I am writing a Windows 10 device driver which receives an interrupt from hardware every 8.3 ms. I am simulating the hardware device in a linux QEMU/KVM VM with Windows 10 installed. How do I program my simulated device to generate an interrupt every 8.3 ms? Under windows, I would generate a high resolution timer interrupt using the windows multi-media API. What is the QEMU/KVM equivalent? Use a QEMUTimer. You can set the expiry period in nanoseconds. Note that you should probably not expect QEMU's timing to be accurate to exactly 8.3ms. I'm guessing any mostly smooth 120Hz interrupt rate would be fine. Similar to the 55ms/18.2Hz PC Int08 signal that was historically 0x1 ticks per hour, 0x18 + epsilon ticks/day. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: AARCH64 sve and sve2 instruction emulation not calling plugin memory op
Have you considered repurposing whatever plugin interface is
used for x86 I/O port space? (By defining it for each
architecture without a predefined mapping to PCI bus I/O
port space).
On 2020-12-09 14:58, Richard Henderson wrote:
On 12/8/20 7:48 PM, Robert Henry wrote:
Richard:
I posted this item to qemu-discuss yesterday, but cc:ed you to the wrong
address.
...
It looks like the emulation of AARCH64 SVE and SVE2 instructions is not
calling the part of the plugin interface that is supposed to be called when
there is a memory operation, viz the function registered by a call to
qemu_plugin_register_vcpu_mem_cb
Nope, nor for most any other out-of-line memory operation performed by any
other qemu target. Starting with arm's own dc zva.
The plugin interface needs extension for this. How should I signal that 256
consecutive byte loads have occurred? How should I signal that the controlling
predicate was not all true, so only 250 of those 256 were actually active? How
should I signal 59 non-consecutive (gather) loads have occurred?
If the answer is simply that you want 256 or 250 or 59 plugin callbacks
respectively, then we might be able to force the memory operations into the
slow path, and hook the operation there. As if it were an i/o operation.
There are other changes to the slow path that need changing, because at present
I do not have enough bits to represent 256-bit alignment for (specifically)
arm32 neon "vld4 {d0-d3},[r0:256]". So perhaps I can work in the plugin thing
at the same time.
Thoughts?
r~
Enjoy
Jakob
--
Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com
Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10
This public discussion message is non-binding and may contain errors.
WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: Sound error: Could not initialize DirectSoundCapture
On 07/12/2020 15:13, Peter Maydell wrote: On Mon, 7 Dec 2020 at 13:56, kazusanosuke_kaz wrote: Hi, all! I've just installed qemu to my PC, Windows 8.1(x64) with Core i5-4670 / MSI MS-7817(OnBoard Sound) / GeForce GT 640 Windows 98SE was successfully installed on qemu, but sound device doesn't work ... "C:\Program Files\qemu\qemu-system-i386.exe" -device sb16 -m 512 Windows98SE.qcow2 dsound: Could not initialize DirectSoundCapture dsound: Reason: No sound driver is available for use, or the given GUID is not a valid DirectSound device ID (I don't use Windows, but here are some suggestions based on reading the source code.) Even though I choose another soundhw, the result is all the same That's because this error message is not related to what sound hardware is being given to the guest. It's the QEMU DirectSound audio backend complaining that when it asked the host Windows for a sound-input device (ie a microphone or similar) it got an error back. (Specifically, the IDirectSoundCapture_Initialize() call failed.) So the warnings are basically saying "sound input to QEMU is not going to work", and they'll be printed for any kind of sound hardware you give to the guest (sb16, ac97, ...). That said, looking at the code I think that the DirectSound backend attempts to continue even if audio input doesn't work. So if you only care about audio output, in theory these warnings should be harmless. You could try some of the suggestions in this stack overflow post: https://stackoverflow.com/questions/55601413/how-to-fix-could-not-initialize-directsoundcapture-in-android-studio (effectively "plug in a microphone" or "enable some windows-internal input device") to see if they help. If that makes the warnings go away and also makes sound output start working, then there might be a bug in the QEMU code that is supposed to handle "input doesn't work but proceed with output anyway". If the warnings go away but sound output still doesn't work, then the output problem is not related to the warnings. I sometimes use Windows, and the only time I tried a qemu on Windows, I could not find any working sound option, and all the help and error messages were useless, suggesting things that weren't explained, listing options that didn't work etc. Someone on the team needs to clean up sound support in the prebuilt Windows binaries that are linked to from qemu.org. -- Jakob Bohm, CIO, partner, WiseMo A/S. http://www.wisemo.com <http://www.wisemo.com> Transformervej 29, 2860 Soborg, Denmark. direct: +45 31 13 16 10 This message is only for its intended recipient, delete if misaddressed. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: Is it possible to control irq allocation in QEMU?
On 06/12/2020 10:45, Hao Lee wrote: Hi, I use the following options to simulate USB devices. -drive if=none,id=usbstick,file=usb.qcow2 \ -usb -device usb-ehci,id=ehci \ -device usb-tablet,bus=usb-bus.0 \ -device usb-storage,bus=ehci.0,drive=usbstick The contents in /proc/interrupts are: 0:515 IO-APIC 2-edge timer 1: 63 IO-APIC 1-edge i8042 8: 1 IO-APIC 8-edge rtc0 9: 0 IO-APIC 9-fasteoi acpi 10:127 IO-APIC 10-fasteoi ehci_hcd:usb1 11: 36 IO-APIC 11-fasteoi uhci_hcd:usb2 12: 5 IO-APIC 12-edge i8042 14: 93 IO-APIC 14-edge ata_piix 15: 6 IO-APIC 15-edge ata_piix I want to know why QEMU allocates irq-10 and irq-11 for the two USB devices. Can I make them above irq-15? Thanks! The classic PC architecture, as introduced by IBM PC/AT only has the 15 interrupt lines. The modern IO-APIC may have options to route PCI(e) interrupts to other x86 interrupt vectors, if activated by the operating system, in this case Linux. If QEMU emulates such an IO-APIC ability, Linux may have boot options to use it, or it might be configurable in the emulated SeaBIOS. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. http://www.wisemo.com Transformervej 29, 2860 Soborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: BUG:No Valid SPF Record Leading to Email Spoofing.
On 2020-11-03 16:09, Peter Maydell wrote: On Tue, 3 Nov 2020 at 14:23, Jakob Bohm wrote: I just checked, the project admins still haven't fixed the qemu.org DNS as per best practice (see my previous mail). qemu.org doesn't run a mail service anyway -- there are no qemu.org email addresses. Best current practice is to have DNS records telling potential mail recipients that no email addresses exist for a domain. This is a side effect of the ancient rule that any A record functions as an implicit delivery point for incoming mail, making it formally valid to send mail from any DNS domain name with an IP address. The current way of doing that is to add the following records: MX 0 . TXT "v=spf1 -all" Older software will recognize that TXT record as a request to reject SMTP connections with HELO or MAIL FROM specifying the DNS name, while the "MX 0 ." record is from a newer specification. As prohibited by DNS, these records are not needed for a DNS name that points to a CNAME, such as "www.qemu.org". Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: BUG:No Valid SPF Record Leading to Email Spoofing.
I just checked, the project admins still haven't fixed the qemu.org DNS
as per best practice (see my previous mail).
On 2020-11-03 01:09, Atik Islam wrote:
Hi There
any update ?
Thanks
On Fri, Mar 20, 2020 at 2:40 AM Atik Islam <mailto:atiki8...@gmail.com>> wrote:
Hi,
Severity : High.
Introduction:
There is a email spoofing vulnerability.Email spoofing is the
forgery of an email header so that the message appears to have
originated from someone or somewhere other than the actual source.
Email spoofing is a tactic used in phishing and spam campaigns
because people are more likely to open an email when they think it
has been sent by a legitimate source. The goal of email spoofing
is to get recipients to open, and possibly even respond to, a
solicitation.
Steps to Reproduce:
1.goto http://www.kitterman.com/spf/validate.html
<http://www.kitterman.com/spf/validate.html>
2.Enter domain name: www.qemu.org <http://www.qemu.org> and click
spf record if any under "Does my domain already have an SPF
record? What is it? Is it valid?"
3.You will see that no valid spf protection.
4.So that why i try to send email using qemu-discuss@nongnu.org
<mailto:qemu-discuss@nongnu.org> and i was successfully delivered
the messege to my email address.
In addition to above checking,
I used https://emkei.cz/ <https://emkei.cz/> and send a test mail
using www.qemu.orgdomain which was delivered successfully.This
further confirms that the emails spoofed.
Impact
An attacker would send a Fake email. The results can be more
dangerous.
Enjoy
Jakob
--
Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com
Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10
This public discussion message is non-binding and may contain errors.
WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: BUG:No Valid SPF Record Leading to Email Spoofing.
I just checked, the project admins still haven't fixed the qemu.org DNS as per best practice (see my previous mail). On 2020-11-03 01:09, Atik Islam wrote: Hi There any update ? Thanks On Fri, Mar 20, 2020 at 2:40 AM Atik Islam <mailto:atiki8...@gmail.com>> wrote: Hi, Severity : High. Introduction: There is a email spoofing vulnerability.Email spoofing is the forgery of an email header so that the message appears to have originated from someone or somewhere other than the actual source. Email spoofing is a tactic used in phishing and spam campaigns because people are more likely to open an email when they think it has been sent by a legitimate source. The goal of email spoofing is to get recipients to open, and possibly even respond to, a solicitation. Steps to Reproduce: 1.goto http://www.kitterman.com/spf/validate.html <http://www.kitterman.com/spf/validate.html> 2.Enter domain name: www.qemu.org <http://www.qemu.org> and click spf record if any under "Does my domain already have an SPF record? What is it? Is it valid?" 3.You will see that no valid spf protection. 4.So that why i try to send email using qemu-discuss@nongnu.org <mailto:qemu-discuss@nongnu.org> and i was successfully delivered the messege to my email address. In addition to above checking, I used https://emkei.cz/ <https://emkei.cz/> and send a test mail using www.qemu.orgdomain which was delivered successfully.This further confirms that the emails spoofed. Impact An attacker would send a Fake email. The results can be more dangerous. -- Jakob Bohm, CIO, partner, WiseMo A/S. https://www.wisemo.com <https://www.wisemo.com> Transformervej 29, 2860 Soborg, Denmark. direct: +45 31 13 16 10 This message is only for its intended recipient, delete if misaddressed. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: PS/2 emulation - timer?
On 2020-11-02 23:25, Peter Maydell wrote: On Mon, 2 Nov 2020 at 21:44, Kamil Jońca wrote: As I wrote earlier there is strange behavior of ps/2 mouse emulation (https://bugs.launchpad.net/qemu/+bug/1618301) I tried to read code (https://github.com/qemu/qemu/blob/master/hw/input/ps2.c) but cannot find anything wrong. The only thing I doubt is time relation issues (ps/2 description mentions about sample rate), but this was rather quick look. Not sure. But as a general rule of thumb, you're better off going with an absolute-position device (the standard one is the USB tablet device) rather than a relative-position one like the PS/2 mouse when you're giving the guest an emulated pointing device. A relative-position one will always tend to get out of sync with the host mouse, and an absolute-position one won't. thanks -- PMM Historically, there were some absolute-position devices that used the PS/2 port. These were identified by either a special bit in the position message, or specific responses to an ID query sent out the PS/2 port. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: Does the order of kernel parameters matter?
On 2020-10-14 20:39, Salvatore Mazzarino wrote: I'm facing some interesting issues. Changing the order of kernel parameters passed to the `-append`, it changes the outcome. Running my QEMU VM with the following works: -append "root=/dev/disk/by-id/virtio-rootfs rootflags=rw flatcar.first_boot=1 tsc=reliable no_timer_check= rcupdate.rcu_expedited=1 i8042.direct=1 i8042.dumbkbd=1 i8042.nopnp=1 i8042.noaux=1 noreplace-smp= reboot=k console=hvc0 console=hvc1 cryptomgr.notests= net.ifnames=0 pci=lastbus=0" However, if put the `root=` to the end then it does not work and the root volume is not found so not mounted. So I'm now wondering. the order does it really matter? And if so there is any logic behind it? Any rules to follow? This happens generally with the Linux kernel, probably because the mechanism for passing the options to the kernel has a character limit, causing the last of your long line to be thrown away. I suggest looking at the kernel documentation to see if there is a limit and then check if you exceed it. As for qemu, I suspect that qemu uses a generic method of passing the string into the VM, which has a much higher limit to support non-Linux kernels. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: Which qemu change corresponds to RedHat bug 1655408
On 2020-10-09 15:56, Max Reitz wrote: On 09.10.20 14:55, Jakob Bohm wrote: On 2020-10-09 10:48, Max Reitz wrote: [...] The error I got was specifically "Failed to lock byte 100" and VM not starting. The ISO file was on a R/W NFS3 share, but was itself R/O for the user that root was mapped to by linux-nfs-server via /etc/exports options, specifically the file iso file was mode 0444 in a 0755 directory, and the exports line was (simplified) /share1 :::/64(ro,sync,mp,subtree_check,anonuid=1000,anongid=1000) where :::/64 is the numeric IPv6 prefix of the LAN NFS kernel Server ran Debian Stretch kernel 4.19.0-0.bpo.8-amd64 #1 SMP Debian 4.19.98-1~bpo9+1 (2020-03-09) x86_64 GNU/Linux NFS client mount options were: rw,nosuid,nodev,noatime,vers=3,rsize=1048576,wsize=1048576,namlen=255, soft,proto=tcp6,timeo=600,retrans=6,sec=sys,mountaddr=:::::xxff:fexx:, mountvers=3,mountport=45327,mountproto=udp6,local_lock=none,addr=:::::xxff:fexx: NFS client ran Debian Buster kernel 4.19.0-0.bpo.6-amd64 #1 SMP Debian 4.19.67-2+deb10u2~bpo9+1 (2019-11-12) x86_64 with Debian qemu-system- x86 version 1:5.0-14~bpo10+1 Booting used SysV init and libvirt was not used. Copying the ISO to a local drive (where qemu-as-root had full capabilities to bypass file security) worked around the failure. I hope these details help reproduce the bug. I’ll try again, thanks. Can you perchance reproduce the bug with a more recent upstream kernel (e.g. 5.8)? I seem to recall there have been some locking bugs in the NFS code, perhaps there was something that was fixed by now. (Or at least 4.19.150, which seems to be the most recent 4.19.x according to kernel.org) And I still have no idea why qemu tried to lock bytes in a read-only raw image file, there is no block metadata to synchronize access to (like in qcow2), when the option explicitly said ",format=raw" to avoid attempts to access the iso file as any of the advanced virtual disk formats. I reasoned about that in my previous reply already, see below. It’s because just because an image file is read-only when opening it doesn’t mean that it’s going to stay that way. You’re correct that in the case of raw, this isn’t about metadata (as there isn’t any), but about guest data, which needs to be protected from concurrent access all the same, though. (As for “why does qemu try to lock, when the option explicitly said raw”; there is a dedicated option to turn off locking, and that is file.locking=off. I’m not suggesting that as a realistic work-around, I’m just adding that FYI in case you didn’t know and need something ASAP.) [...] The error message itself seams meaningless, as there is no particular reason to request file locks on a read-only raw disk image. Yes, there is. We must prevent a concurrent instance from writing to the image[1], and so we have to signal that somehow, which we do through file locks. I suppose it can be argued that if the image file itself is read-only (outside of qemu), there is no need for locks, because nothing could ever modify the image anyway. But wouldn’t it be possible to change the modifications after qemu has opened the image, or to remount some RO filesystem R/W? Perhaps we could automatically switch off file locks for a given image file when taking the first one fails, and the image is read-only. But first I’d rather know what exactly is causing the error you see to appear. [1] Technically, byte 100 is about being able to read valid data from the image, which is a constraint that’s only very rarely broken. But still, it’s a constraint that must be signaled. (You only see the failure on this byte, because the later bytes (like the one not preventing concurrent R/W access, 201) are not even attempted to be locked after the first lock fails.) (As for other instances writing to the image, you can allow that by setting the share-rw=on option on the guest device. This tells qemu that the guest will accept modifications from the outside. But that still won’t prevent qemu from having to take a shared lock on byte 100.) Max Theoretically, locking on a raw file needs to be protocol-compatible with loop-mounting the same raw file, so if the loop driver doesn't probe those magic byte offsets to prevent out-of-order block writes, then there is little point for the qemu raw driver to do so. This applies to both the loop driver in the host kernel and the loop driver on any other machine with file share access to the sane image file. As for upgrading, I will try newer kernels packaged for the Debian version used, once the current large batch job has completed, but I doubt it will make much difference given the principles I just stated. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-bin
Re: Which qemu change corresponds to RedHat bug 1655408
On 2020-10-09 10:48, Max Reitz wrote: On 08.10.20 18:49, Jakob Bohm wrote: (Top posting because previous reply did so): If the bug was closed as "can't reproduce", why was a very similar bug listed as fixed in RHSA-2019:2553-01 ? Hi, Which very similar bug do you mean? I can only guess that perhaps you mean 1603104 or 1551486. Bug 1603104 was about qemu not ignoring errors when releasing file locks fails (we should ignore errors then, because they're not fatal, and we often cannot return errors, so they ended up as aborts). (To give more context, this error generally appeared only when the storage the image is on somehow disappeared while qemu is running. E.g. when the connection to an NFS server was lost.) Bug 1551486 entailed a bit of a rewrite of the whole locking code, which may have resulted in the bug 1655408 no longer appearing for our QE team. But it was a different bug, as it wasn’t about any error, but just about the fact that qemu used more FDs than necessary. (Although I see 1655408 was reported for RHEL 8, whereas 1603104 and 1551486 (as part of RHSA-2019:2553) were reported for RHEL 7. The corresponding RHEL 8 bug for those two is 1694148.) Either way, both of those bugs are fixed in 5.0. 1655408 in contrast reports an error at startup; locking itself failed. I couldn’t reproduce it, and I still can’t; neither with the image mounted concurrently, nor with an RO NFS mount. (For example: exports: [...]/test-nfs-ro 127.0.0.1(ro,sync,no_subtree_check,fsid=0,insecure,crossmnt) $ for i in $(seq 100); do \ echo -e '\033[1m---\033[0m'; \ x86_64-softmmu/qemu-system-x86_64 \ -drive \ if=none,id=drv0,readonly=on,file=/mnt/tmp/arch.iso,format=raw \ -device ide-cd,drive=drv0 \ -enable-kvm -m 2048 -display none &; \ pid=$!; \ sleep 1; \ kill $pid; \ done (Where x86_64-softmmu/qemu-system-x86_64 is upstream 5.0.1.) All I see is something like: --- qemu-system-x86_64: terminating on signal 15 from pid 7278 (/bin/zsh) [2] 34103 [3] - 34095 terminated x86_64-softmmu/qemu-system-x86_64 -drive -device ide-cd,drive=drv0 -m 2048 So no file locking errors.) The error I got was specifically "Failed to lock byte 100" and VM not starting. The ISO file was on a R/W NFS3 share, but was itself R/O for the user that root was mapped to by linux-nfs-server via /etc/exports options, specifically the file iso file was mode 0444 in a 0755 directory, and the exports line was (simplified) /share1 :::/64(ro,sync,mp,subtree_check,anonuid=1000,anongid=1000) where :::/64 is the numeric IPv6 prefix of the LAN NFS kernel Server ran Debian Stretch kernel 4.19.0-0.bpo.8-amd64 #1 SMP Debian 4.19.98-1~bpo9+1 (2020-03-09) x86_64 GNU/Linux NFS client mount options were: rw,nosuid,nodev,noatime,vers=3,rsize=1048576,wsize=1048576,namlen=255, soft,proto=tcp6,timeo=600,retrans=6,sec=sys,mountaddr=:::::xxff:fexx:, mountvers=3,mountport=45327,mountproto=udp6,local_lock=none,addr=:::::xxff:fexx: NFS client ran Debian Buster kernel 4.19.0-0.bpo.6-amd64 #1 SMP Debian 4.19.67-2+deb10u2~bpo9+1 (2019-11-12) x86_64 with Debian qemu-system- x86 version 1:5.0-14~bpo10+1 Booting used SysV init and libvirt was not used. Copying the ISO to a local drive (where qemu-as-root had full capabilities to bypass file security) worked around the failure. I hope these details help reproduce the bug. And I still have no idea why qemu tried to lock bytes in a read-only raw image file, there is no block metadata to synchronize access to (like in qcow2), when the option explicitly said ",format=raw" to avoid attempts to access the iso file as any of the advanced virtual disk formats. On 2020-10-08 18:41, Philippe Mathieu-Daudé wrote: Hi Jakob, On 10/8/20 6:32 PM, Jakob Bohm wrote: Red Hat bugzilla bug 1655408 against qemu is listed by Red Hat as fixed in April 2019, but I cannot find the corresponding change on qemu.org (the Changelog in the wiki is not a traditional changelog and doesn't cover bugfix releases such as 5.0.1, the git commit log is too detailed to search, the Red Hat bugzilla and security advisory pages do not link red hat bugs back to upstream (launchpad) bugs or git changes. Here is the bug title (which also affects my Debian packaged qemu 5.0): VM can not boot up due to "Failed to lock byte 100" if cdrom has been mounted on the host Further observation: The basic problem is that qemu-system refuses to start with the error message "Failed to lock byte 100" when -drive points to a read-only ISO file. For the reporter of the Red Hat bug, that was a mount-induced read-only condition, in my case it is an NFS mount of a read-only directory. The error message itself seams meaningless, as there is no particular reason to request file locks on a read-only raw disk image. Yes, there is. We must pre
Re: Which qemu change corresponds to RedHat bug 1655408
(Top posting because previous reply did so): If the bug was closed as "can't reproduce", why was a very similar bug listed as fixed in RHSA-2019:2553-01 ? On 2020-10-08 18:41, Philippe Mathieu-Daudé wrote: Hi Jakob, On 10/8/20 6:32 PM, Jakob Bohm wrote: Red Hat bugzilla bug 1655408 against qemu is listed by Red Hat as fixed in April 2019, but I cannot find the corresponding change on qemu.org (the Changelog in the wiki is not a traditional changelog and doesn't cover bugfix releases such as 5.0.1, the git commit log is too detailed to search, the Red Hat bugzilla and security advisory pages do not link red hat bugs back to upstream (launchpad) bugs or git changes. Here is the bug title (which also affects my Debian packaged qemu 5.0): VM can not boot up due to "Failed to lock byte 100" if cdrom has been mounted on the host Further observation: The basic problem is that qemu-system refuses to start with the error message "Failed to lock byte 100" when -drive points to a read-only ISO file. For the reporter of the Red Hat bug, that was a mount-induced read-only condition, in my case it is an NFS mount of a read-only directory. The error message itself seams meaningless, as there is no particular reason to request file locks on a read-only raw disk image. my qemu-system-x86_64 invocation contains the option (on one line): -drive if=none,id=drive-ide0-1-0,readonly=on, file=/mnt/someshare/path/gparted-live-1.1.0-5-amd64.iso,format=raw https://bugzilla.redhat.com/show_bug.cgi?id=1655408 has been closed due to lack of reproducer. Can you amend your information to the BZ? It will likely be re-opened. Thanks! Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Which qemu change corresponds to RedHat bug 1655408
Red Hat bugzilla bug 1655408 against qemu is listed by Red Hat as fixed in April 2019, but I cannot find the corresponding change on qemu.org (the Changelog in the wiki is not a traditional changelog and doesn't cover bugfix releases such as 5.0.1, the git commit log is too detailed to search, the Red Hat bugzilla and security advisory pages do not link red hat bugs back to upstream (launchpad) bugs or git changes. Here is the bug title (which also affects my Debian packaged qemu 5.0): VM can not boot up due to "Failed to lock byte 100" if cdrom has been mounted on the host Further observation: The basic problem is that qemu-system refuses to start with the error message "Failed to lock byte 100" when -drive points to a read-only ISO file. For the reporter of the Red Hat bug, that was a mount-induced read-only condition, in my case it is an NFS mount of a read-only directory. The error message itself seams meaningless, as there is no particular reason to request file locks on a read-only raw disk image. my qemu-system-x86_64 invocation contains the option (on one line): -drive if=none,id=drive-ide0-1-0,readonly=on, file=/mnt/someshare/path/gparted-live-1.1.0-5-amd64.iso,format=raw Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: Remap a key
(Resend from my subscribed e-mail address) Correction, I meant "ctrl-break", earlier someone asked about ctrl-alt-del On 2020-10-07 21:12, Jakob Bohm wrote: There is no such mechanism in the qemu documentation, but there is this way: 1. Use the appropriate hot-key to switch to the qemu monitor screen 2. Type this command at the "(qemu)" prompt (qemu) sendkey ctrl-alt-delete This can of cause be automated by having software on the guest connect to the qemu monitor and send the command when you press your magic key. Another available method is to use the VNC console and a VNC client with a ctrl-alt-del function. On 2020-10-07 12:58, Will Senn wrote: I've been digging into the lack of a proper break key on laptops (my MacBook for example) and I was wondering if it would be possible to remap one of the existing keys on the laptop to the CTRL-BREAK key, which sends the scancode E0 46? I'd be fine with having CTRL-ALT-] map to it or pretty much anything not typically typed in DOS. But doing assembly programming in DOS without a break key is frustrating to no end. Pointers to qemu keymapping documentation would be helpful or any sort of 'here's a keymapping example' tip. Thanks, Will -- GPG Fingerprint: 68F4 B3BD 1730 555A 4462 7D45 3EAA 5B6D A982 BAAF -- Jakob Bohm, CIO, partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Soborg, Denmark. direct: +45 31 13 16 10 This message is only for its intended recipient, delete if misaddressed. WiseMo - Remote Service Management for PCs, Phones and Embedded Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: Remap a key
Correction, I meant "ctrl-break", earlier someone asked about ctrl-alt-del On 2020-10-07 21:12, Jakob Bohm wrote: There is no such mechanism in the qemu documentation, but there is this way: 1. Use the appropriate hot-key to switch to the qemu monitor screen 2. Type this command at the "(qemu)" prompt (qemu) sendkey ctrl-alt-delete This can of cause be automated by having software on the guest connect to the qemu monitor and send the command when you press your magic key. Another available method is to use the VNC console and a VNC client with a ctrl-alt-del function. On 2020-10-07 12:58, Will Senn wrote: I've been digging into the lack of a proper break key on laptops (my MacBook for example) and I was wondering if it would be possible to remap one of the existing keys on the laptop to the CTRL-BREAK key, which sends the scancode E0 46? I'd be fine with having CTRL-ALT-] map to it or pretty much anything not typically typed in DOS. But doing assembly programming in DOS without a break key is frustrating to no end. Pointers to qemu keymapping documentation would be helpful or any sort of 'here's a keymapping example' tip. Thanks, Will -- GPG Fingerprint: 68F4 B3BD 1730 555A 4462 7D45 3EAA 5B6D A982 BAAF -- Jakob Bohm, CIO, partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Soborg, Denmark. direct: +45 31 13 16 10 This message is only for its intended recipient, delete if misaddressed. WiseMo - Remote Service Management for PCs, Phones and Embedded -- Jakob Bohm, CIO, partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Soborg, Denmark. direct: +45 31 13 16 10 This message is only for its intended recipient, delete if misaddressed. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: Remap a key
There is no such mechanism in the qemu documentation, but there is this way: 1. Use the appropriate hot-key to switch to the qemu monitor screen 2. Type this command at the "(qemu)" prompt (qemu) sendkey ctrl-alt-delete This can of cause be automated by having software on the guest connect to the qemu monitor and send the command when you press your magic key. Another available method is to use the VNC console and a VNC client with a ctrl-alt-del function. On 2020-10-07 12:58, Will Senn wrote: I've been digging into the lack of a proper break key on laptops (my MacBook for example) and I was wondering if it would be possible to remap one of the existing keys on the laptop to the CTRL-BREAK key, which sends the scancode E0 46? I'd be fine with having CTRL-ALT-] map to it or pretty much anything not typically typed in DOS. But doing assembly programming in DOS without a break key is frustrating to no end. Pointers to qemu keymapping documentation would be helpful or any sort of 'here's a keymapping example' tip. Thanks, Will -- GPG Fingerprint: 68F4 B3BD 1730 555A 4462 7D45 3EAA 5B6D A982 BAAF -- Jakob Bohm, CIO, partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Soborg, Denmark. direct: +45 31 13 16 10 This message is only for its intended recipient, delete if misaddressed. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: Question about reduced memory by balloon driver (virtio-balloon)
On 2020-09-16 13:35, Henry lol wrote: Hi guys, When a guest's memory is dynamically reduced with "balloon" QMP API, I'm wondering whether the virtio-balloon driver in the guest is consuming such reduced memory. If so, how can I see that the memory virtio-balloon is consuming? and is it possible the virtio-balloon has such a huge memory even though it's a kernel module?? Thanks, I have not checked the source code for the qemu balloon driver, but the traditional way such balloon drivers work is: 1. Qemu tells the balloon how much memory to remove from Guest 2. Balloon driver software may check the overall memory use on the Guest and adjust the request accordingly. This is useful to grant the memory to whichever Guest needs it the most. 3. Balloon driver pretends that it is a very advanced hardware device (such as a shared-memory GPU) that needs a range of physical RAM to be accessed directly via the I/O bus for some hardware purpose, it asks for the amount needed. 4. The Guest OS hands that "phyisical" memory range to the device (or declares Guest memory full). 5. The Balloon driver hands the "physical" RAM addresses back to qemu, which can then removes host memory backing from that virtual machine address range. 6. The actual amount of physical memory can be seen as either the "non-paged memory use" of the balloon driver, as a special number in the Guest memory statistics (if the Guest kernel has integrated the concept directly), or by querying qemu on the host via one of the two "monitor" interfaces (The Human Monitoring Protocol or the machine monitoring protocol). -- Jakob Bohm, CIO, partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Soborg, Denmark. direct: +45 31 13 16 10 This message is only for its intended recipient, delete if misaddressed. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: different time in host/quest
On 16/08/2020 11:35, Valery Reznic wrote: Hello. My host is Fedora 32 x86_64. qemu is 4.2.0 When I run Fedora Core 3 i386 or x86_64 on under qemu time in the VM is 3 hours behind time on the host. When I running Fedora 12 of Fedora 29 under same qemu time in the VM is OK. It's not a problem with timezones: 1. Time zones on all VMs and host are same. 2. I also checked UTC time I wonder what may be a reason for such behaviour. By the way, when I add '-rtc base=' to the qemu invocation for FC3 it fixed problem with the difference in the host/guest times. Valery Maybe qemu sets the emulated (guest) battery RTC to local time (as required by Microsoft operating systems), while your Fedora guest assumes it is set to UTC, as is common for Linux systems. Perhaps there is a qemu option to choose between local and UTC guest RTC emulation. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. http://www.wisemo.com Transformervej 29, 2860 Soborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: host in the non admin policy
On 2020-07-29 09:38, alfiomarzu...@libero.it wrote: hello everybody, qemu is an fantastic work! I use it in my office where the pc is under strong policy without administrators privileges. all Pc have windows10, and in my, the virtualmachine with w7 work with some limit, type usb and network card, but I will fix with little bit patiente. Work all becouse I have copied the folder from an pc where I am admin, Ask if is possible for the windows version remove the installer becouse is an ostacle where we dont have need. it is possible? Thank you for the wonderful work. Alfio Please keep the installer for the Windows port, it is very convenient for those of us with admin privileges. Also add a README.txt explaining what to do after install, such as downloading HAXM from a specific Intel webpage and how to set up networking and sound for qemu as packaged in that installer. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: qemu-img measure
Proposed alternative implementation techniques: A. Check if the block count returned by stat is less than what the apparent file size (elsewhere in the stat structure) suggests. B. When converting from a full-size (or presumed full-size) raw image to an internally sparse format such as qcow2, do zero block detection on the fly as the blocks are copied, then write the block-absent information to the output metadata after copying the data and omitting the zero blocks. This would only do one pass over the data, and would get the result right if something is writing to the raw image during conversion. On 2020-07-27 00:08, Arik Hadas wrote: On Fri, Jul 24, 2020 at 5:00 PM Stefan Hajnoczi <mailto:stefa...@redhat.com>> wrote: On Thu, Jul 23, 2020 at 07:31:13PM +0300, Nir Soffer wrote: > On Thu, Jul 23, 2020 at 6:12 PM Arik Hadas mailto:aha...@redhat.com>> wrote: > > [root@lion01 8c98c94d-bc14-4e24-b89c-4d96c820056d]# qemu-img measure -O qcow2 /tmp/arik.qcow2 > > required size: 9359720448 > > fully allocated size: 16108814336 > > Now we have qcow2 image, so we don't depend on the file system capabilities. > This is the advantage of using advanced file format. > > > shouldn't the 'measure' command be a bit smarter than that? :) > > I think it cannot be smarter, but maybe qemu folks have a better answer. qemu-img measure checks that allocation status of blocks. As Nir said, if the file system (older NFS versions) doesn't support that then it doesn't know about zero blocks. It would be possible to add a slow loop that reads the entire input image but I'm not sure how useful it would be to read many gigabytes of data from the disk. The process would be slow and degrade performance of VMs by consuming I/O bandwidth. Not sure about the general case either but specifically for exporting VMs to OVAs it can be useful. Yes, it may be slow and degrade the performance of the VM for longer time (the next step when exporting a running VM is copying the measured disk(s) so the latter may happen anyway) but, arguably, still better than the alternatives of allocating (potentially) the virtual size of the disk within the OVA or copying the image twice. Maybe we can add an optional parameter that makes qemu-img measure to fall back to that slow loop when the blocks' allocation status is not provided by the file system? Stefan -- Jakob Bohm, CIO, partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Soborg, Denmark. direct: +45 31 13 16 10 This message is only for its intended recipient, delete if misaddressed. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: How to add files to an existing img partition.
On 28/06/2020 16:25, Frantisek Rysanek wrote: 2) Add files into that container. If possible, be able to resize the file holding the container, as well as resize the partitions defined inside. Mount the image in a VM, where you have some tool (partition magic? PQ magic?) that can shrink Windows filesystems (FAT16/FAT32/NTFS). Booted from a small "service" system disk. How about PartEd live, it is a Linux liveCD (Debian based) that contains a GUI for GNU parted and all the available Libre command line file system tools, including built in knowledge of how to invoke them for common tasks like the OP needs. It doesn't however include qemu-img or qemu-nbd to manipulate disk image files. While these can be installed to the liveCD RAM disk from Debian repositories, the PartEd GUI doesn't know how to invoke them on-the-fly. For XP disks, PartEd GUI knows about ntfsresize and some unspecified tool for (V)FAT filesystems. Most Linux-based tools can work directly with raw images, either directly or via the "loop" device as configured with losetup After that tool is finished, you should end up with some deallocated space at the end of your raw image file. You can then use qemu-img to shrink the image file - or so I understand. See also my previous note, if this stuff with raw images and resizing wouldn't be easier to solve with a more modern image format - provided that you can get a hypervisor for Android that can work with them. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. http://www.wisemo.com Transformervej 29, 2860 Soborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: Exporting qcow2 images as raw data from ova file with qemu-nbd
On 2020-06-23 16:08, Richard W.M. Jones wrote: On Tue, Jun 23, 2020 at 08:47:52AM -0500, Eric Blake wrote: On 6/22/20 5:21 PM, Nir Soffer wrote: And it works, but it exposes the qcow2 data. I want to raw data so I can upload the guest data to ovirt, where is may be converted to qcow2 format. Nir, can you use qemu-img convert and get a free conversion to your choice of format? This works fine over NBD as long as you don't try and write which I guess you don't want to do here. Richard suggested to try nbdkit tar plugin, but the plugin is not available on RHEL, and this adds additional dependency, when we already use qemu-nbd. Rich just rewrote the tar plugin to use python instead of perl, which means it is that much easier for a future RHEL to pull it in. We still ought to consider having a tar filter, either in place of or in addition to, the tar plugin (similar to how we recently converted nbdkit's ext4 support from a plugin to a filter) - having a tar filter would allow you to read a compressed ova file (by combining the xz and tar filters to decompress then extract a file). But right now, nbdkit doesn't support non-C filters (and given that our tar plugin was written first in perl and now in python, that still means translation to yet another language if the filter requires it to be in C). The reason it was in Perl and is now in Python (and not C), and also the reason it still a plugin, is that parsing tar files is very complex because of historical compatibility. If we accept that we cannot write a from-scratch tar file parser in C then we have to use an existing tool or library (‘tar’ itself was used by Perl, now we're using ‘tarfile.py’ from Python stdlib). Those tools require access to an actual local file. So I'm afraid this rewrite is hard work :-) Unless we accept that we only parse files created by a narrow range of tools, but the problem is that OVA files can be generated by a wide variety of tools. If you can supply the offset by some other means then of course using nbdkit-offset-filter or qemu's offset block layer is the solution. Note that the BSD implementation of tar (in C) is available as a library, which is also available on some Linux systems. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: Exporting qcow2 images as raw data from ova file with qemu-nbd
Why not use qemu-img convert directly, it doesn't expose the disk
content to any
interface except the disk image file(s) created.
On 2020-06-23 00:21, Nir Soffer wrote:
I'm trying to export qcow2 images from ova format using qemu-nbd.
I create 2 compressed qcow2 images, with different data:
$ qemu-img info disk1.qcow2
image: disk1.qcow2
file format: qcow2
virtual size: 200 MiB (209715200 bytes)
disk size: 384 KiB
...
$ qemu-img info disk2.qcow2
image: disk2.qcow2
file format: qcow2
virtual size: 200 MiB (209715200 bytes)
disk size: 384 KiB
...
And packed them in a tar file. This is not a valid ova but good enough
for this test:
$ tar tvf vm.ova
-rw-r--r-- nsoffer/nsoffer 454144 2020-06-22 21:34 disk1.qcow2
-rw-r--r-- nsoffer/nsoffer 454144 2020-06-22 21:34 disk2.qcow2
To get info about the disks in ova file, we can use:
$ python -c 'import tarfile; print(list({"name": m.name, "offset":
m.offset_data, "size": m.size} for m in tarfile.open("vm.ova")))'
[{'name': 'disk1.qcow2', 'offset': 512, 'size': 454144}, {'name':
'disk2.qcow2', 'offset': 455168, 'size': 454144}]
First I tried the obvious:
$ qemu-nbd --persistent --socket=/tmp/nbd.sock --read-only --offset=512 vm.ova
And it works, but it exposes the qcow2 data. I want to raw data so I
can upload the guest
data to ovirt, where is may be converted to qcow2 format.
$ qemu-img info --output json "nbd+unix://?socket=/tmp/nbd.sock"
{
"virtual-size": 209715200,
"filename": "nbd+unix://?socket=/tmp/nbd.sock",
"format": "qcow2",
...
}
Looking in qemu manual and qapi/block-core.json, I could construct this command:
$ qemu-nbd --persistent --socket=/tmp/nbd.sock --read-only
'json:{"driver": "qcow2", "file": {"driver": "raw", "offset": 512,
"size": 454144, "file": {"driver": "file", "filename": "vm.ova"}}}'
And it works:
$ qemu-img info --output json "nbd+unix://?socket=/tmp/nbd.sock"
{
"virtual-size": 209715200,
"filename": "nbd+unix://?socket=/tmp/nbd.sock",
"format": "raw"
}
$ qemu-img map --output json "nbd+unix://?socket=/tmp/nbd.sock"
[{ "start": 0, "length": 104857600, "depth": 0, "zero": false, "data":
true, "offset": 0},
{ "start": 104857600, "length": 104857600, "depth": 0, "zero": true,
"data": false, "offset": 104857600}]
$ qemu-img map --output json disk1.qcow2
[{ "start": 0, "length": 104857600, "depth": 0, "zero": false, "data": true},
{ "start": 104857600, "length": 104857600, "depth": 0, "zero": true,
"data": false}]
$ qemu-img convert -f raw -O raw nbd+unix://?socket=/tmp/nbd.sock disk1.raw
$ qemu-img info disk1.raw
image: disk1.raw
file format: raw
virtual size: 200 MiB (209715200 bytes)
disk size: 100 MiB
$ qemu-img compare disk1.raw disk1.qcow2
Images are identical.
I wonder if this is the best way to stack a qcow2 driver on top of a
raw driver exposing
a range from a tar file.
I found similar example for gluster in:
docs/system/device-url-syntax.rst.inc
Richard suggested to try nbdkit tar plugin, but the plugin is not
available on RHEL,
and this adds additional dependency, when we already use qemu-nbd.
Nir
Enjoy
Jakob
--
Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com
Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10
This public discussion message is non-binding and may contain errors.
WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: Error SSL qemu KVM on AMD 3900X
On 18/06/2020 05:21, supp...@vietpn.com wrote: OpenSSL: error:2D06D075:FIPS routines:fips_pkey_signature_test:test failure OpenSSL: error:2D08E06B:FIPS routines:FIPS_CHECK_EC:pairwise test failed OpenSSL: error:1409802B:SSL routines:ssl3_send_client_key_exchange:reason(43) Unable to establish SSL connection. When running on AMD chips the server cannot boot if the multiplier core is greater than 1. And all get SSL errors. Cant you help me! I'm running on CentOS 7.8.2003 This looks like a general OpenSSL issue. You seem to be running RedHat modified OpenSSL 1.0.x series with FIPS (US government) mode enabled. You will have to look for RedHat-specific solutions because the OpenSSL foundation has dropped support since Dec 31, 2019. OpenSSL upstream currently only supports OpenSSL 1.1.1 which has no FIPS mode. In early may, someone reported a problem with AMD CPU compatibility in OpenSSL 1.1.1, but no details have been posted on their user mailing list yet. Hope this helps. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. http://www.wisemo.com Transformervej 29, 2860 Soborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: Which kernel version meet the needs, when run QEMU 5.1 on kunpeng 920 ARM serever?
On 22/05/2020 22:19, Sid Spry wrote: On Thu, May 21, 2020, at 2:17 AM, linw...@ruijie.com.cn wrote: On Wed, 2020-05-20 at 11:10 -0500, Sid Spry wrote: On Wed, May 20, 2020, at 3:11 AM, linw...@ruijie.com.cn wrote: How can I figure out which kernel version will meet the needs if I am using Qemu 5.x on huawei KunPeng 920 ARM server, which will fully use the hardware supported feature on this kind of server? Is there any basic mathod out there to quickly find out? What do you mean by features? If you mean CPU virtualization features, -machine accel=kvm and -cpu host will enable everything that can be enabled. Hardware devices need to be explicitly passed in. If you use a modern distribution's kernel it will have modules for all qemu devices available by default. Hardware features which I mean is something like VHE, SMMU, GICv3(may be GICv4) etc. Are these all enable and can neatly support on the kernel version 4.14(Centos7 aarch64) and can be utilize by my virtual machine, should I try the latest version kernel 5.x, is that necessary? Ok, I leave exhaustive check to you, but based on mailing list postings circa 2015[0] those features are supported in some 4.x kernels. If it makes no difference otherwise I highly recommend always choosing the higher kernel version especially when tracking new hardware developments. By the time hardware is released typically it just works. Also -- the features listed are integral to KVM's operation. If they weren't supported I doubt KIM would function. [0]: https://lists.cs.columbia.edu/pipermail/kvmarm/2015-July/015619.html This info (with actual specificity) needs to be in the maintained qemu docs. Main README should state minimum kernel versions for each qemu-feature, from TCG to KVM hardware pass through of specific GPUs. qemu-system manpage should list extra host requirements for any option/device/feature that needs more than the minimum. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. http://www.wisemo.com Transformervej 29, 2860 Soborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: Using All Cores of CPU on Snapdragon Processor during x86-to-ARM User Space Emulation
On 13/05/2020 12:02, Alex Bennée wrote: Vijay Daita writes: Hello It is my understanding that one would be unable to do x86-to-ARM user space emulation while utilizing all cores because of x86 barriers. Actually the utilisation of multiple cores (often referred to at MTTCG) is a function of system emulation and you are correct for x86-on-ARM we don't enable MTTCG because we don't currently add barrier instructions to fully emulate the x86 memory model. However for linux-user we have always followed the guest threading model because the guest clone() is passed down to the host. However because the memory modelling isn't perfect you can run into problems because of the mismatch. I wanted to know if there is difference between what QEMU aims to do and using a interpreter of sorts to convert x86 instructions directly to ARM instructions so that when run on the system directly, the system can decide, itself, how to apportion the task. This is what the TCG does - it translates guest instructions into groups of host instructions. We could insert the extra barriers for all loads and stores but the effect would be to cripple performance. In an ideal world we would only do these for the load/store instructions involved in inter-thread synchronisation operations but that's a fairly tricky problem to solve. Especially because the x86 memory model traditionally has barrier/synchronization instructions automatically push through their ordering to all other cores/CPUs, and as a result, "barrier load" wasn't really a thing until CMPXCHG was introduced in a later CPU generation than the basic sync instructions (8086) and cache coherency mechanisms (80486). In fact, the LOCK barrier prefix triggers an #UD exception with most load instructions. The one exception to this lack was instruction decoding, where certain commonly used branch instructions were defined as implicitly picking up any changes in instruction memory. This of cause corresponds to the TCG checking for needed retranslation of buffers at those points. Additionally, x86 barriers generally guarantee total ordering relative to the barrier operation of all memory accesses that occur before or after in program order, which some other CPU families do not. I am new to this, so sorry if this doesn't make very much sense. Thank you Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. http://www.wisemo.com Transformervej 29, 2860 Soborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: QEMU vmvga + Windows - Which driver to use?
On 11/05/2020 10:14, TestingBot wrote: I am trying to find a driver for Windows 7/8/10 to use the "vmvga" display driver with QEMU+KVM. For Linux, this driver is available by default, and for OSX there's VmsVGA2, but where can I find the correct driver to use with Windows and QEMU 2.11.1 (included with Ubuntu 18.04). Right now, when I specify vmvga in libvirt, Windows will fall back to the default display adapter. It is not available in the virtio iso, nor can I seem to find it anywhere on the web. vmvga is the virtual hardware under VMWare. The Windows driver is part of the VMWare extensions usually installed from a disk image which is part of VMWare itself. It is thus mostly useful when the virtual machine is migrated between qemu and VMWare. I don't know id the driver can be legally extracted from VMWare products or not. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. http://www.wisemo.com Transformervej 29, 2860 Soborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: qemu 4.2.0 audiodev soundhw
As someone who doesn't always keep libvirt and qemu in sync (if using libvirt at all), keeping forward and backwards compatibility for command lines and other interfaces becomes quite critical. For example doing a quick site compile of a more recent qemu needs to work with whatever a distribution libvirt or local script invokes, even if later upgrading libvirt without recompiling qemu. In that light, option 2 is not just preferable, but necessary. It can however be simplified to silently assume a hard coded audiodev value, with the qemu-system man page explicitly stating that value, along with the version dependency of both syntax generations. In fact, over the years, I have found it excruciatingly difficult to find valid qemu documentation, as each feature effort tends to leave behind half-updated pages and a bunch of uncoordinated messages about what may or may not have been implemented in unspecified versions. On 20/04/2020 19:54, Idar Lund wrote: Hi, Thanks for your response! Yes, I agree with you on the options. If you guys decide on (3), I would suggest to make it dynamically like this; "-soundhw hda,audiodev=sound1". This would then copy the 'audiodev' (and possible other) parameter(s) to the '-device' option. My personal preference would be to recommend option number 1. The reason for this is that maintaining a shortcut like this makes it hard to maintain for developers when adding features and fixes bugs on other options. And of course documentation maintainers :) The second reason as I see it is that people tend to create a .sh script or similar to start their qemu virtual machines if they don't use libvirt/xml schema. And for that, a more verbose command would actually be easier to maintain for users since we then know where to put parameters like this. -Idar On Mon, Apr 20, 2020 at 4:44 PM Gerd Hoffmann <mailto:kra...@redhat.com>> wrote: On Fri, Apr 17, 2020 at 12:15:30PM +0100, Peter Maydell wrote: > On Fri, 17 Apr 2020 at 12:08, Idar Lund mailto:idarl...@gmail.com>> wrote: > > I'm using qemu-system-x86_64 with the following options: > > -audiodev pa,id=sound1,server=/run/user/1000/pulse/native \ > > -soundhw hda > > > > After upgrade to 4.2.0 (qemu-4.2.0-7.fc32) I get the following warning: > > (qemu) audio: Device hda: audiodev default parameter is deprecated, please specify audiodev=sound1 > > > > The documentation `man qemu-system-x86_64` seems to not reflect this. > > How am I supposed to use audiodev and soundhw? > > This looks like another question for you, Gerd... Hmm, good question how to proceed here best ... "-soundhw hda" is a shortcut for "-device intel-hda -device hda-duplex" You can use "-device intel-hda -device hda-duplex,audiodev=sound1" to make the warning go away. That is pretty verbose when compared to "-soundhw hda" though ... So the options I see are: (1) deprecate the -soundhw shortcut, expect users to use -device instead. (2) have -soundhw lookup the audiodev and add it automatically. Works only with a single audiodev, but that isn't different from what we have today. If you want do more complicated things you already have to use the more verbose -device command line. (3) add audiodev option to -soundhw. I don't like (3) much, our command line is already messy enough. So my personal preference would be (1) or (2) ... Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. http://www.wisemo.com Transformervej 29, 2860 Soborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: Cisco Switch Errors
On 18/04/2020 17:25, Randy Wheaton wrote: Good Morning, We have 5 servers running Centos7 QEMU and we are relocating them to a new Data Center and we're getting error disabled on the Cisco Fex ports when we turn up the port. Is there any support you could provide on what our configuration should be to get the servers up and running with an A side and B Side configuration so if lose one network path the other will take over. We don't have anyone left in the company that knows how this was configured but any help connecting to Cisco FEX 2k switches would be great. Thanks for your help. * * ** I'm no Cisco guy, but from general experience the most likely scenario is this: The Cisco switches detected that the virtual machine network addresses (MAC and/or IP) were suddenly trying to enter the network from the wrong switch or switch port, indicating a suspected attack from a compromised server (such attacks do exist and blocking them is good). In that case it is basically a matter of telling the Cisco switches that the authorized location of the virtual machines have changed. Not knowing Cisco gear or your setup, the enforcement may have been automatic from Cisco systems detecting the usual location, or may have been manually configured into the switches. Either way, the authorization would need to be done approximately the same way. Either way, this may involve appropriate work order, 2-person or other oversight procedures, as it is essentially automatic checking of such. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. http://www.wisemo.com Transformervej 29, 2860 Soborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: BUG:No Valid SPF Record Leading to Email Spoofing.
Clarification: Both qemu.org and www.qemu.org need (but lack) SPF records. Steps to reproduce: $ host -t TXT qemu.org qemu.org has no TXT record $ host -t TXT www.qemu.org www.qemu.org is an alias for qemu.org. Expected output (if no @qemu.org e-mail addresses): $ host -t TXT qemu.org qemu.org descriptive text "v=spf1 -all" $ host -t TXT www.qemu.org www.qemu.org is an alias for qemu.org. On 19/03/2020 21:40, Atik Islam wrote: Hi, Severity : High. Introduction: There is a email spoofing vulnerability.Email spoofing is the forgery of an email header so that the message appears to have originated from someone or somewhere other than the actual source. Email spoofing is a tactic used in phishing and spam campaigns because people are more likely to open an email when they think it has been sent by a legitimate source. The goal of email spoofing is to get recipients to open, and possibly even respond to, a solicitation. Steps to Reproduce: 1.goto http://www.kitterman.com/spf/validate.html 2.Enter domain name: www.qemu.org <http://www.qemu.org> and click spf record if any under "Does my domain already have an SPF record? What is it? Is it valid?" 3.You will see that no valid spf protection. 4.So that why i try to send email using qemu-discuss@nongnu.org <mailto:qemu-discuss@nongnu.org> and i was successfully delivered the messege to my email address. In addition to above checking, I used https://emkei.cz/ and send a test mail using www.qemu.orgdomain which was delivered successfully.This further confirms that the emails spoofed. Impact An attacker would send a Fake email. The results can be more dangerous. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. http://www.wisemo.com Transformervej 29, 2860 Soborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: QEMU-img convert question
On 2019-12-02 21:09, Ottavio Caruso wrote: On Mon, 2 Dec 2019 at 18:07, Carson, Jay B wrote: Does running the following command transfer any data to any cloud or external service provider: $ qemu-img convert -f raw -O qcow2 image.img image.qcow2 No. The image stays on the computer on which this command is run. Does this application work as a completely standalone application? I have no idea what that means. If you mean it is separate from qemu-system-, then yes, but it's usually part of the same package. Note that on Debian-based GNU/Linux systems, qemu-img is in the separate package qemu-utils along with qemu-nbd, qemu-io, ivshmem etc. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: Does reboot clear RAM?
On physical machines, the following mechanisms are common: 1. DRAM chips physically loose their contents after a few seconds of power off, but not on a CPU reset (reset button on typical board). A CPU reset does not have this effect. This also applies to the PC platform. 2. SRAM chips physically loose their contents after a few milliseconds of power off, otherwise as for DRAM. 3. On x86 and x86_64 PCs, the IBM compatible BIOS typically does a memory test and wipe during actual boot, but not upon a software initiated boot. This PC BIOS rule exists for the following two purposes: 3.1 Older guest operating systems use a software reset to switch the CPU from "protected mode" to "real mode" because the historical 80286 CPU chip had no other way to return to real mode and returning to real mode was needed to invoke BIOS APIs. 3.2 Signalling if such a non-wiping boot is desired (for speed or other reasons) is officially done by writing a magic value in one of the well-known BIOS global addresses, if this global address has not been set to one of those magic values, and the global RTC register with related semantics have not been so set either, the BIOS (in qemu's case SEABIOS) should do the wipe as part of the POST (Power-On-Self-Test), otherwise it should skip that and most other parts of the POST. On 11/11/2019 08:24, Joachim Durchholz wrote: What mechanisms do these platforms use? Via CPU, or via dedicated RST lines on the RAM? (I have no idea whether modern RAM has such a reset line, even.) P.S.: No need to reply to me directly, replying to list is enough to reach me. Am 10.11.19 um 19:17 schrieb Kent Dorfman: Many non-intel platforms do wipe memory on reset (RST). It's a security feature to make in more dificult for folks with jtag breakouts to examine secure code/data. On 11/10/19, Joachim Durchholz wrote: Do physical machines wipe memory? They don't on shutdown, and memory tends to retain bits for a while (seconds to hours, depending on temperature and RAM technology). I don't know what RAM chips do if they are reset. Do they even have a reset line nowadays? Am 09.11.19 um 10:56 schrieb Narcis Garcia via: In my humble opinion, Qemu should wipe any released RAM area because of guest shutdown or reboot, same as a physical machine, to guest finds it's booting in same conditions. El 8/11/19 a les 23:55, Nachammai Karuppiah ha escrit: Does qemu clear the physical address on guest reboot? Qemu is started with kernel parameters, ramoops.mem_address=0x7f0 ramoops.mem_size=0x10 but I do see that the memory even outside this region is not wiped away and is retained after rebooting the guest Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: How to clone CPUState in a new thread?
On 07/11/2019 01:44, Michael Goffioul wrote: Hi, I'm working on a project that wants to replace houdini (ARM-to-x86 translation layer for Android from Intel) with a free open-source implementation. I'm trying to leverage qemu user-mode to achieve that, but it requires code changes to allow executing dynamically loaded functions instead of running a single executable. Basic question: Isn't the qemu user-mode emulator already able to run a "single executable" that loads DLLs, creates dynamic code etc. in the emulated instruction set? The obvious exception would be to skip the ARM instruction set intermediary when translating Dalvik byte code from .dex files. From this perspective, emulated ARM thread creation would be just letting qemu emulate the ARM code that would be called, including letting qemu emulate the system calls such as "clone". A special case would be if houdini allows direct calls between ARM and x86 .so files. I don't know if qemu-user has the ability to expose host native DLLs to emulated code. In a nutshell, using ideas from unicorn-engine, I've enhanced CPUARMState with a stop address. Whenever this address is encountered in the translator, it generates a YIELD exception, which then makes the cpu_loop to exit. It works fine for simple cases, but I'm having trouble with multi-threading aspect. Threads created from the native/ARM side do seem to work properly. The problem is when a new Java thread (not created from native/ARM) attempts to execute native code. The QEMU engine has been initialized in the main thread, but new Java threads do not have access to thread-local variable thread_cpu. I've tried (maybe naively) to recreate what the clone syscall is doing to create a new CPUState/CPUArchState object, usable from the new thread, but executing any ARM code quickly lead to a crash. I suppose I'm doing something wrong, or missing something to properly initiale a new cpu. I'm hoping that someone could help me solve this problem. I've attached the current QEMU patch I'm using, most of the Android glue layer is in linux-user/main.c. It contains a set of utility functions that my Android native bridge implementation is using. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. http://www.wisemo.com Transformervej 29, 2860 Soborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: Creating image of OS to run with qemu.
On 31/10/2019 09:12, bilsch01 wrote: I have executable code for my simple OS in a binary file (jsec2.bin) created using nasm assembler. I have been running it from a flash drive with a boot sector that loads the executable to memory and jumps to it. File jsec2.bin is not bootable by itself. I want to start running it with qemu - no flash drive involved. I do not want qemu to use a virtual flash drive either. I want to use the qemu floppy image switch, -fda. However the -fda switch will only work if there is a aa55 boot sector mark at the end of the first sector of the image. qemu-system-i386 -fda jsec2.bin will not work. Here's some other possibilities: 1) I need an image of a bootable floppy with a bootsector that jumps to a file named jsec2.bin just like an msdos boot sector jumps to the file named IO.SYS contained in the file system on the disk. Is there a tool that I can adapt to this purpose? Look at the source code of the boot sector of the floppy/hard drive version of the "SysLinux" boot loader. (See wiki.syslinux.org) Or look at the source code for of the boot sector of the FreeDOS OS. Both are free systems that apparently use that technique of loading a file from FAT12/FAT16 file systems as used on floppies. 2) possibly there is some way to make a grub floppy image or iso that boots the executable in jsec2.bin. If there is a way using a grub tool? Actually, both of these ideas suck because I want to use qemu for running the code as I develop it and these ideas are slow and cumbersome. Any suggestions will be appreciated. Thanks Bill S. Option 3: If you can get a working copy of SysLinux 4.xx (not version 5 or above), you can write your OS as a "comboot" file that runs almost under raw BIOS but with a few helper features to get things started. Comboot files are loaded at address :0100 with the bytes from : to :00FF filled with some nice info. Option 4: Mark your code blob as a PC-style "extension ROM" (as found on some plug in cards like hard drive controllers or video cards). This involves adding a few simple headers, padding the file size to a power of 2 between 1KB and 64KB, and telling qemu this is a ROM to be mapped into address space at some address between C000: and EFC0: (Note: C000: to C000:7FFF are typically used for Video BIOS already). For more information, refer to your BIOS technical manual. As you are working in real mode, I have written the addresses in segment:offset notation. Option 5: Emulate the file format and startup entrypoint of Linux kernels as supported by all the Linux boot loaders (including the one built into qemu itself using the "--kernel" option). This is more complex and may involve running at least the beginning of your code in 32 bit mode. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [RFC PATCH] configure: deprecate 32 bit build hosts
On 02/10/2019 17:16, Richard Henderson wrote: On 10/2/19 2:10 AM, Daniel P. Berrangé wrote: GCC only implements int128_t for 64-bit targets. QEMU probes for that during configure and sets CONFIG_INT128 If I'm reading correctly include/qemu/int128.h then provides a fallback type based on a struct with two int64s. This has some inconvenience though as you have to use the the (inline) function calls for all the basic operands and will be less efficient when using the fallback. Presumably this is not viable for TCG ? A structure (for some ABIs) may be passed and returned by invisible reference. It's not impossible (nothing's impossible), but it adds previously unnecessary complexity to allocate that storage on the jit stack. Actually manipulating one 128-bit value consumes 4/6 of the i386 registers, which I can well imagine could wind up causing problems. Certainly manipulating two values at once is out of the question. That's less of a problem for arm and mips. Note that all newer i386 chips (specifically 486DX and up) include at least eight 64 bit registers and int operations in the floating point unit. Using those to open-code 128 bit arithmetic would probably require using at least 5/8 of those, this (like any other emulation of a CPU with more total register space) would probably require some spilling of virtual registers to some part of RAM. 486SX and older (all the way down to the 8086) also have those, but only if the x87 coprocessor is installed or emulated by the host OS kernel. Later i386 models have even more features for int128 emulation in their larger floating point units. Anyway, all of the pain points go away if we assume 64-bit. -- Jakob Bohm, CIO, partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Soborg, Denmark. direct: +45 31 13 16 10 This message is only for its intended recipient, delete if misaddressed. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [RFC PATCH] configure: deprecate 32 bit build hosts
On 26/09/2019 17:31, Alex Bennée wrote: Peter Maydell writes: On Thu, 26 Sep 2019 at 00:31, Alex Bennée wrote: The 32 bit hosts are already a second class citizen especially with support for running 64 bit guests under TCG. We are also limited by testing as actual working 32 bit machines are getting quite rare in developers personal menageries. For TCG supporting newer types like Int128 is a lot harder with 32 bit calling conventions compared to their larger bit sized cousins. Fundamentally address space is the most useful thing for the translator to have even for a 32 bit guest a 32 bit host is quite constrained. As far as I'm aware 32 bit KVM users are even less numerous. Even ILP32 doesn't make much sense given the address space QEMU needs to manage. For KVM we should wait until the kernel chooses to drop support, I think. I can certainly do that - although I'd still like to know who actually uses 32 bit kvm support these days. Note that the hardware support for kvm was apparently only in 64 bit AMD/IntelCPUs. I don't know if early Intel Xeons with 64 bit and kvm support ran faster in 32 bit mode, which would be a good reason to use a 32 bit host kernel on them these days, but otherwise I see little reason to treat 32 bit x86 kvm as a priority target. I don't know what was used to run 32 bit ARM binaries on x86 Android tablets with 32 bit x86 user space, but if it was qemu, it would have been TCG, not kvm. @@ -745,19 +744,22 @@ case "$cpu" in ;; armv*b|armv*l|arm) cpu="arm" -supported_cpu="yes" ;; I'll leave others to voice opinions about their architectures, but I still have 32-bit arm in my test set for builds, and I'm pretty sure we have users (raspi users, for a start). raspi is probably the most common one because of the 32 bit userspace they use even though they are running on 64 bit chips. Please note that not all releases of the raspi boards had 64 bit CPUs. Raspi 0, 1 and 2 used 32 bit cores (some later production runs of Raspi 2 actually had the Raspi 3 chip). Then there's the whole space of raspi-lookalike boards like the bananas (mostly 32 bit A7, some boards with 64 bit A53). LeMaker Etc. Though I don't currently run qemu on them, I happen to have some older gen dev boards used for various things, including the BeagleBone open hardware designs that are all based on a 32 bit Arm A8 core. Outside of dev boards, there's also the whole theme of running qemu on Arm based Androidsmartphonesas a way to keep the inner OS separate from the phone OS. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: KVM - Qcow2 image size behavior
On 25/09/2019 17:05, Julianno Nogueira wrote: Hello everyone! I would like to thank you all for the project and the tool developed across these years. I am a fan of yours! This is very important to make the new technology era a better and open place to work and live with! I´m facing a weird listing image file behaviour (or at least I think It is), that have a 500GB Qcow2 image: [root@server /]# qemu-img info /var/lib/libvirt/images/netvoltcloudserv.img image: /var/lib/libvirt/images/netvoltcloudserv.img file format: qcow2 virtual size: 500G (536870912000 bytes) disk size: 489G cluster_size: 65536 Format specific information: compat: 1.1 lazy refcounts: true - The Image size is 500GB, and that´s correct and exacly what I have set up. Now the problem is when I ls this image: [root@server /]# ls -la /var/lib/libvirt/images/ total 512593892 drwx--x--x. 4 root root 75 Set 16 22:46 . drwxr-xr-x. 10 root root 117 Jun 20 12:04 .. drwx--x--x. 2 root root 29 Jul 5 2018 discos_virtuais -rwx--x--x. 1 qemu qemu *1010408555008* Set 25 11:51 netvoltcloudserv.img drwx--x--x. 2 root root 6 Set 16 22:46 nsf-pool-1 *- The size of this image is 1TB!* *So my question based on this, is: * - Is this a "bug" or "expected" behavior? - Who to "fix the size" or make SISOP see it correctly (with 500GB)? SYSOP should know that most modern file systems (including the one containing your/var/lib/libvirt/images/ directory) support the feature known as "sparse files", wheresome parts of a file are not actually stored, but are treated as all zeroes when read. On unix/linux the true disk use can be seen with the standard utility "du" . On Windows, the resource kit tool diruse can usually show this info. In your case qemu itself claims the file only uses 489G of disk space, but I don'tknow if that corresponds to the reality as reported by the command du -ha /var/lib/libvirt/images/ As for qcow2 specifically, the format allows some qcow2 clusters (each 64K in yourimage) to be such sparse clusters, and this is often the default when creating newqcow2 image files. Finally note that the virtual disk size seen by your virtual machine (in this case500GB) may be less or more than the apparent size and actual disk use of the qcow2 file. A qcow2 file contains at least the following: The non-zero clusters from the virtual disk,snapshots of that image created via the libvirt snapshot commands, copies ofthe virtual machine ram (if included in snapshots of a running machine), andmetadata to keep track of where all this is in the qcow2 file. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] ANNOUNCE: emails from this mailing list will soon drop the [qemu-*] subject tag
The point of DKIM (and moreso DMARC) is to prevent spoofing in the absence of functioning PGP and S/MIME infrastructure. It is not as strong as end to end encryption, but benefits from being zero effort for most of the users protected, as it is handled almost entirely by mail system administrators. Unfortunately, the rushed implementation of the SPF+DKIM+DMARC combination meant that handling of mailing lists wasn't properly integrated in the specifications. Which in turn resulted in some pretty horrible implementation ideas by both the DMARC group (having each mail list server set up complex e-mail aliases for each participant) and MailMan (suggesting that mailing lists kick out any participants on a DMARC-compatible mail server). On 24/09/2019 16:06, Narcis Garcia wrote: Is far better mail servers use SPF and not DKIM. DKIM is a signature between servers, but valid author's signatures are far better done with PGP by author itself because is made by author and verified by reader. DKIM doesn't protect mail content autenticity at all, beginning with the content sent from MUA to MTA. PGP signatures are compatible with mailing lists footers and prefixes. DKIM has only sense to mail servers control mail contents, and being from same mail servers that contents can be hacked. El 10/9/19 a les 10:38, Peter Maydell ha escrit: Hi; this is an announcement to let you know that in future emails to all QEMU project mailing lists (including this one) will no longer have the [qemu-*] tag in their Subject line. We need to make this config change because having the mailing list server edit subject lines like this conflicts with the increasingly common DKIM/DMARC anti-email-forgery system. We used to work around this by having the list server rewrite email From addresses instead, but this has proven to have bad consequences (notably that patches applied from emails to QEMU can end up with incorrect authorship by accident). If you were using the Subject line tag to filter QEMU emails, you'll need to change your mail client's config to instead look at the "List-Id:" message header to identify list traffic (you can do this now without waiting for us to change the list config to drop the subject tags). Apologies for any inconvenience that this upcoming config change might cause you. thanks -- PMM Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] QEMU-i386-static question
The qemu-static chroot still uses the native (ARM) kernel and the native (ARM) device driver for your serial port. You just need to add the /dev/ttyUSB0 device file in the chroot. Either as a bind mount accessing the same file in the native root, or as a another device file inode created with the mknod command to match the other one. Then when x86 code opens "/dev/ttyUSB0" it will get the one in its chroot, which points to the same kernel mode device object as the one in the real (ARM) root. Each I/O call (open, close, read, write, ioctl etc.) will be passed from x86 code, through the qemulated x86 syscall to the identical ARM syscall, and then the ARM kernel will access the real USB device on the real USB bus. On 13/09/2019 14:22, Dr. Pedro E. Colla wrote: Hi Peter, Thank you very much for your kind and detailed response. As it's a cross architecture setup chroot operates on a different platform (ARM host vs. X86 guest) so using binfmt qemu-i386-static is invoked by the chroot command, therefore when executed qemu has been already executed. Under that configuration /dev/ttyUSB0 doesn't exists. I might install the USB Prolific serial port drivers once in X86 but I don't believe that would make the chrooted environment (under qemu) to share the port with the host, does it? Thanks in advance for your thoughts and tips on the subject. 73 de Pedro LU7DID On Fri, Sep 13, 2019 at 6:27 AM Peter Maydell wrote: On Thu, 12 Sep 2019 at 21:28, Dr. Pedro E. Colla wrote: I'm trying to implement a chroot environment on a Raspberry Pi with the ultimate intention to run Windows applications using wine. To do that first an x86 chroot environment using the subject emulator is used, then once inside a shell of it wine is invoked. The whole thing works surprisingly well despite my initial expectations given the small CPU resources available. However, I do need for the Windows program to access the serial port (/dev/ttyUSB0 at the host environment); the Windows problem is unable to use that port despite different ways to configure it for wine to recognize it; but ultimately I found the problem is the x86 debian running under qemu isn't seeing it at all. It seems this version of qemu doesn't handle the usual arguments to define character devices or usb devices. QEMU has two modes of operation: (1) system emulation, where we emulate an entire hardware system for the guest, including various devices. In this mode we allow the user to configure how our emulated devices hook up to the host system via the command line arguments you mention (2) user-mode emulation, where we emulate just a single guest Linux process, and pass through all the system calls it makes to the host. In this mode there is no need for any kind of command line options to configure devices, because there are no devices to configure. qemu-i386-static is the user-mode emulator, case (2). If you would like binaries you run under it to be able to see the host's /dev/ttyUSB0 you need to ensure that that (special) file exists in the chroot before you run QEMU. Then the guest binary will be able to open it, exactly as a host binary could. After that, getting it to be visible to Windows programs running under Wine is a Wine configuration question. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] Using qemu to run a physical machine in parallel
On 04/08/2019 17:26, Yassine Chaouche wrote: Dear qemu, I have installed Linux Mint on my machine on /dev/sda5. Later, I installed MX Linux on /dev/sda9, but the installation of MX eventually didn't work well as I tried to do things. I need to fix MX linux, but each I change something I need to reboot the machine to see if it fixed it, and this is cumbersome/tiresome/awkward. What I would like to do is boot on /dev/sda5 (Linux Mint) and run MX Linux (/dev/sda9) in parallel. Can I achieve this with qemu ? I don't want to reinstall MX as a virtual machine, I would like to run and fix the already installed one. Thanks for your tips ! In general, qemu can use your physical hard disk partition (/dev/sda9) as a the storage for a virtual machine disk (such as /dev/sda), thus almost allowing you to run your MX system under qemu, however there are some important differences between the virtual and real machine that you will need to work aroundyourself. 1. The virtual machine will have different "hardware", for example it will have a different network card (at least a different mac address) and a different graphics card. It will also have less memory because it has to fit inside the free memory not used by your Mint system. 2. If you mount /dev/sda9 as a virtual hard drive, the virtual machine will see it as a physical disk, not a partition of a physical disk. This may confuse the MX startup scripts and configuration. As an alternative, you can layer a qcow2 image on top of the full /dev/sda, thus showing the virtual machine the entire disk, but redirecting all disk writes by MX to the qcow2 file. Beware however that because the actual /dev/sda is changing every time your Mint system writes to it, the content of the Mint and shared partitions as seen by the virtual machine will be a garbled mess that you should not access. Once you find a working MX setup, you will have to copy out _only_ the MX-only partition content to the real /dev/sda partitions while not using or overwriting the partitions belonging to Mint or shared, probably by doing funny stuff with qemu-nbd and dd, very very carefully. Tip: To make a small shared partitions (such as boot or EFI) completely separate for the virtual machine, mount the qcow2 image with qemu-nbd (without the virtual machine running!) and very carefully dd the initial contents of the physical partition to the qcow2 partition. I am not sure which additional options are needed to prevent qcow2 from mapping those sectors back to the physical sectors that will change later. 3. If MX installs Linux kernels or bootloaders into common "boot" and/or "EFI" partitions, you will have to manually copy those files out to the Mint machine, then pass the kernels and initramfs images on the qemu command line to boot them. Some configuration tips may be gleaned from "PV" (Physical to Virtual) conversion instructions and scripts, but with the key difference of not actually trying to store a complete copy of your physical /dev/sda inside itself. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] qemu-sparc vs qemu-system-sparc
To clarify: qemu-sparc is for running a single *program* written for a SPARC CPU, but the same OS as yours (in John's case Linux for SPARC). qemu-system-sparc is for running an entire virtual *machine* containing an OS for a SPARC CPU, such as a hard drive containing Solaris for SPARC or Linux for SPARC. Both programs are useful, but for different things. On 17/07/2019 16:29, Mike Russo wrote: You are trying to virtualize SPARC on x86 so you need qemu-system-sparc. qemu-sparc would be for a user-mode emulation on a SPARC processor. I've done this before on CentOS 7 and it should definitely be possible to install qemu-system-sparc. yum should not have a problem figuring out the dependencies - make sure you're all up to date and don't have any broken packages. -Original Message- From: John Chludzinski mailto:john%20chludzinski%20%3cjohn.chludzin...@gmail.com%3e>> To: qemu-discuss@nongnu.org<mailto:qemu-discuss@nongnu.org> Subject: [Qemu-discuss] qemu-sparc vs qemu-system-sparc Date: Tue, 16 Jul 2019 17:43:46 -0400 I installed CentOS 7.6 then installed all things QEMU on my machine. I have a SPARC image I need to bringing up in a VM. I've been using *qemu-system-sparc*. $ qemu-system-sparc -m 256 -hda solaris_v2-qemu_v2.2.0.disk -nographic -bios ./openbios-sparc32 This is on a box on which I have Fedora-30 installed. Can I use *qemu-sparc* to bring up my Solaris image: *solaris_v2-qemu_v2.2.0.disk* ? If so, how? BTW, *qemu-sparc* come with (on CentOS 7.6): $ sudo yum install qemu* *PS> I've tried to install qemu-system-sparc on my CentOS box but ended up in a never-ending whack-a-mole game of dependencies.* Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] mipsel_bios.bin missing
On 01/07/2019 02:02, Andrew Pennebaker wrote: qemu-system-mips64el: Could not load MIPS bios 'mipsel_bios.bin' I am scratching my head why qemu offers an mips64el emulator with missing parts. At least other emulators like for ARM and PPC have standard packages for obtaining the BIOS firmware assets. Hmm, the wiki says that this file is not even necessary. The wiki advises creating a BIOS file consisting of 128KiB of zeroes. Then the wiki mentions something about using a kernel to avoid loading the BIOS file. Uh, I am not familiar enough with qemu to know what kind of kernel should be passed in. I just want to run a basic Debian mips64el guest. In general, qemu expects Linux kernels, such as the Debian kernels. Can we please get this error resolved in a future qemu release? I wonder if any of the "Open hardware router" projects have an open source MIPSel BIOS that qemu could distribute. Or if they all just run the Linux bootloader directly. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] Qemu on Windows (with haxm and whpx), minimal Ubuntu, SLIRP and OpenVPN: very slow
On 13/05/2019 16:35, Michael Fritscher wrote: Am 13.05.2019 um 16:10:33 schrieb Michael Fritscher: Am Montag, 13. Mai 2019 15:25 CEST, Michael Fritscher schrieb: Am 13.05.2019 um 10:32:48 schrieb Michael Fritscher: Good day, Scenario: Host: Windows 7 and 10 64 bit, Skylake+ CPUs Qemu Version: 3.1 (4.0 isn't available yet on https://qemu.weilnetz.de/) Accelerators: HAXM and WHPX Guest: Ubuntu 18.04.2 64 bit, minimal install Guest's tasks: OpenVPN, iptables, tc Configuration: -machine q35,accel=kvm:hvf:whpx:hax:tcg -m 512 -smp 1 -rtc base=utc -drive file=layertc_environmentmanager_amsvm.iso,if=virtio,media=cdrom -vga std -netdev user,id=natted, restrict=n, net=172.31.1.0/255.255.255.0, dhcpstart=172.31.1.1, host=172.31.1.3, dns=172.31.1.4 -device virtio-net,netdev=natted,mac=00:50:56:00:08:00 -nodefaults -monitor tcp:localhost: -kernel vmlinuz -initrd initrd.img -append "boot=casper textonly quiet nosplash nofb fb=false pti=off spectre_v2=off l1tf=off nospec_store_bypass_disable noprompt" (the iso is a casper livecd, but I start the linux kernel directly because of boot speed) Problem: It is slow... While a openssl speed -elapsed -evp aes-128-cbc is ok (500MB/sec and beyond), OpenVPN struggles. I get 70% virtual CPU usage , mostly on OpenVPN and system load while transfering 1-2 MB/s with a UDP video stream. The measured delay is also rather spiky and is about 30...40 ms. On the host I get a cpu usage of about 10% (=almost a core) I see this both on the server and the client side. On the player edition of the big commercial player, the results are _much_ better (10% virtual CPU usage, delay of 4-6 ms) How can I further debug this? Do you know any knobs I can try? Slirp, while there are warnings of being slow, doesn't seem to be the culprit as I can get easily >>10 MB/s through it. And I see the big CPU usage of OpenVPN Best regards, Michael Fritscher Hi again, after a bit of digging, openVPN seems to call read_hpet with about 1 kHz, which is, regarding to the program perf, the main culprit. Is a call rate of this frequency normal and qemu is just very slow to handle them, or is there another problem? Compare the boot-up dmesg of the guest OS kernel in the two scenarios. Assuming read_hpet is a kernel-level function, it may simply be that on VMware, the kernel or driver chooses to use a different "hardware" mechanism to implement whichever system call is being called 1000 times per second. Also note, that video stream software is more likely than OpenVPN to be making extremely frequent timer checks (UDP media streams time packet arrivals to manage buffering and smooth over the content in lost packets). Another frequent user of timing calls is performance checking tools. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] Best Intel hardware for qemu
As I wrote, qemu can pass disks (and disk partitions) through without passing the disk controller through. To qemu, a physical disk is just another virtual disk storage format. But if you want to pass through an entire PCI disk controller (with all its disks) for faster I/O, then VFIO is needed. On 10/04/2019 20:40, Lars Bonnesen wrote: I want to pass local disks to a VM in order to run freenas or similar. Regards, Lars. On Wed, Apr 10, 2019, 20:20 Jakob Bohm <mailto:jb-gnumli...@wisemo.com>> wrote: If you pass through the disk access to your SAN partitions as disk accesses to block devices (such as SAN client drivers) in the host machine, you don't need VFIO for that. This can handle nearly unlimited number of virtual machines without running out of PCI slots in the host machine. This is the equivalent of "passing through a SAN disk" in VMWare, but isn't artificially limited to SAN disks (for example, you can layer the Linux multipath drivers and/or the Linux disk encryption drivers between the virtual machine and the actual SAN). If you pass through the network access to your (iSCSI or NBD) SAN as network traffic via the general qemu/kvm network features, you don't need VFIO for that. This can handle nearly unlimited number of virtual machines without running out of PCI slots in the host machine. This is equivalent to using a "VMWare virtual switch". If you dedicate a physical SAN adapter (iSCSI, NBD, SAS or fibre channel) to each virtual machine and pass it through to that virtual machine, you need VFIO for that. As on VMWare, this will limit you to one virtual machine for each PCI slot in the motherboard. If you dedicate a physical network adapter (NIC) to each individual virtual machine and pass it through to PCI drivers in that virtual machine, you need VFIO for that. This too will limit you to one virtual machine for each PCI slot in the motherboard. As for passing through raw SCSI devices or busses, I don't know if the latest qemu versions have the ability to do this at a hardware-independent level like VMWare does (VM sends standard SCSI requests to qemu virtual SCSI adapter, qemu sends those same SCSI requests to real SCSI hardware via something like the Linux "SCSI generic" driver, optionally mapping at most the SCSI-level bus address). On 10/04/2019 19:42, Lars Bonnesen wrote: > But for sure I want passthru - for running virtualized SAN and such > > Any unofficial list? > > Regards, Lars. > > On Wed, Apr 10, 2019, 18:58 Friedrich Oslage mailto:friedrich%2bqemu-u...@oslage.de>> > wrote: > >> As long as it's VT-x capable and can run Linux, you're good to go. >> >> It only gets tricky once you start using VFIO (direct passthrough of >> host PCI devices to a VM, such as GPUs, NICs or NVMes for instance). You >> need VT-d support for that and both the CPU and mainboard have to >> support it. And not only do they have to support it, they have to >> support in a usable way with decent iommu group isolation and without >> weird bugs. There are no (official) compatibility lists for this, it's >> still mostly trial and error... >> >> Regards >> Friedrich >> >> On 4/10/19 2:26 PM, Lars Bonnesen wrote: >>> So I am coming from the VMware world (with comprehensive compatibillity >>> lists) but about to start a project with KVM/Qemu and I would like to >> setup >>> an inexpensive test setup for this purpose. >>> >>> I am thinking of buying one of SuperMicros IoT-servers like >>> >> https://www.supermicro.com/products/system/Mini-ITX/SYS-E300-9D-4CN8TP.cfm >>> Will this be a nice pick for Qemu? Or any VT-supportet system will work >>> fine? >>> >>> Regards, Lars. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] Best Intel hardware for qemu
If you pass through the disk access to your SAN partitions as disk accesses to block devices (such as SAN client drivers) in the host machine, you don't need VFIO for that. This can handle nearly unlimited number of virtual machines without running out of PCI slots in the host machine. This is the equivalent of "passing through a SAN disk" in VMWare, but isn't artificially limited to SAN disks (for example, you can layer the Linux multipath drivers and/or the Linux disk encryption drivers between the virtual machine and the actual SAN). If you pass through the network access to your (iSCSI or NBD) SAN as network traffic via the general qemu/kvm network features, you don't need VFIO for that. This can handle nearly unlimited number of virtual machines without running out of PCI slots in the host machine. This is equivalent to using a "VMWare virtual switch". If you dedicate a physical SAN adapter (iSCSI, NBD, SAS or fibre channel) to each virtual machine and pass it through to that virtual machine, you need VFIO for that. As on VMWare, this will limit you to one virtual machine for each PCI slot in the motherboard. If you dedicate a physical network adapter (NIC) to each individual virtual machine and pass it through to PCI drivers in that virtual machine, you need VFIO for that. This too will limit you to one virtual machine for each PCI slot in the motherboard. As for passing through raw SCSI devices or busses, I don't know if the latest qemu versions have the ability to do this at a hardware-independent level like VMWare does (VM sends standard SCSI requests to qemu virtual SCSI adapter, qemu sends those same SCSI requests to real SCSI hardware via something like the Linux "SCSI generic" driver, optionally mapping at most the SCSI-level bus address). On 10/04/2019 19:42, Lars Bonnesen wrote: But for sure I want passthru - for running virtualized SAN and such Any unofficial list? Regards, Lars. On Wed, Apr 10, 2019, 18:58 Friedrich Oslage wrote: As long as it's VT-x capable and can run Linux, you're good to go. It only gets tricky once you start using VFIO (direct passthrough of host PCI devices to a VM, such as GPUs, NICs or NVMes for instance). You need VT-d support for that and both the CPU and mainboard have to support it. And not only do they have to support it, they have to support in a usable way with decent iommu group isolation and without weird bugs. There are no (official) compatibility lists for this, it's still mostly trial and error... Regards Friedrich On 4/10/19 2:26 PM, Lars Bonnesen wrote: So I am coming from the VMware world (with comprehensive compatibillity lists) but about to start a project with KVM/Qemu and I would like to setup an inexpensive test setup for this purpose. I am thinking of buying one of SuperMicros IoT-servers like https://www.supermicro.com/products/system/Mini-ITX/SYS-E300-9D-4CN8TP.cfm Will this be a nice pick for Qemu? Or any VT-supportet system will work fine? Regards, Lars. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] Fw: Windows 10 qemu installation problem
On 31/01/2019 19:03, libr rop wrote: Hello, I am struck. Waiting for help. Please somebody give some clues where I am going wrong. Need this thing working. - Forwarded message - From: libr rop To: qemu-discuss@nongnu.org Sent: Wednesday, 30 January, 2019, 4:13:37 PM ISTSubject: Windows 10 qemu installation problem Dear All, I am new to qemu and want to give it a try. I downloaded qemu from https://qemu.weilnetz.de/w64/ and tried to install by running the installer. It asks for admin privileges as usual and completes the installation on my window 10 PC 64 bit pro. I have also set the correct path in environment variables to C:\Program Files\qemu But still when I check qemu from Windows PowerShell its throwing CommandNotFoundException. Restarted my machine multiple times. Also tried two previous versions of qemu, available at the download link. Tried it on a windows 7 PC also but the results are the same. My hardware supports virtualization. I can run hyper-v, vmware and virtualbox fine. Kindly help. Try looking in the installed directory. There may not be a command named only "qemu.exe", only ones named according to the exact kind of computer virtualized. For example qemu-system-i386.exe for a 32 bit PC, qemu-system-x86_64.exe for a 64 bit PC etc. I haven't tried qemu on Windows, so just guessing. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] Regarding QEMU Monitor "x /fmt addr" command
On 17/01/2019 19:57, Peter Maydell wrote:
On Thu, 17 Jan 2019 at 18:54, Phil Jones wrote:
Sorry, I meant address of an integer variable inside of a process. Thank you
for the answers, that confirms my suspicions. If I were to translate guest
process virtual address to Windows physical address, could I use that to read
value?
Only if you do the virtual-to-physical translation
exactly at the point when the relevant process is
in memory and when the memory is swapped in and not
out on disk, and you don't let the guest run between
the point when you do the translation and when you use
it. (Otherwise windows could swap something out or
change to a different process.) But if it worked to
do that then the 'x' command would work too.
thanks
-- PMM
Note that on Windows, there are low level ways to get (and optionally
lock) a range of process-address-space virtual memory addresses to
the (qemu virtualized) physical addresses.
These ways include:
1. Having a cooperating kernel module ("driver") make the appropriate
calls, as if it was planning to pass this memory to a piece of non-
standard DMA/bus-master hardware (not the higher level calls that
refer to specific busses like PCI or devices like virtio disk 2).
In the future, someone might add this to one of the qemu helper
client drivers, as something accessible from the gdb stub. Note
that a few undocumented calls may be needed to get a pointer to a
chosen process. Also note kernel mode driver signing problems.
2. Using the builtin WinDbg stub in the Windows kernel. This was
always meant to be accessed from a different (Windows) machine,
and as such can usually be mapped to a qemu device that can then
be forwarded to another virtual machine running just WinDbg and
associated data. WinDbg has command line versions too (kd.exe
for talking to the stub in the Windows kernel).
3. Using Windows-build specific symbol information (available via
a special "debug use only" "public symbol server" service) to
locate the memory mapping structures for processes and mapping
it to "physical" addresses, this latter method is the one that
has the fewest side effects on the virtual machine, but doing
this reliably requires some deep experience, or helper scripts
developed by someone else.
The above applies to all versions of NT-based Windows (not 100%
certain about the ARM ports though, but i386, AMD64, MIPS, PPC and
Alpha should be fine). For the VxD based Windows (Windows 3.x in
"enhanced mode", Windows 95,98 and Me) each of the 3 methods are
somewhat different, and I don't think there is a command line
version of the kernel mode debugger clients (WDEB386.exe).
Windows 3.x in "standard mode" or "real mode" generally keeps a
mostly 1:1 virtual to physical mapping.
Enjoy
Jakob
--
Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com
Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10
This public discussion message is non-binding and may contain errors.
WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] Issue related to mounting /dev/nbd0p1
Check cat /proc/partitions If the partitions are not listed there, the kernel does not recognize them, perhaps the kernel doesn't recognize that nbd0 is a "partitionable disk", and thus does not create the internal nbd0p1 and ndb0p2 devices. If the partitions are listed there, you just need to create the actual /dev/nbd0p1 and /dev/nbd0p2 file names with mknod or figure out why your "/dev/" management software (maybe udev, maybe something better) doesn't do that for you. On 29/11/2018 12:46, ramakanth varala wrote: Still with some errors.. [root@localhost ~]# kpartx -a /dev/nbd0 read error, sector 0 read error, sector 1 read error, sector 29 [root@localhost ~]# ls /dev/nbd nbd0 nbd1 nbd10 nbd11 nbd12 nbd13 nbd14 nbd15 nbd2 nbd3 nbd4 nbd5 nbd6 nbd7 nbd8 nbd9 On Thu, Nov 29, 2018 at 4:22 PM Nerijus Baliūnas < neri...@users.sourceforge.net> wrote: Please try kpartx -a /dev/nbd0 2018-11-29 12:38, ramakanth varala rašė: thanks for the reply .. But i get below error when i do .. [root@localhost ~]# partx -a /dev/nbd0 HDIO_GETGEO: Inappropriate ioctl for device On Thu, Nov 29, 2018 at 4:04 PM Nerijus Baliūnas < neri...@users.sourceforge.net> wrote: 2018-11-29 12:10, ramakanth varala rašė: [root@localhost ~]# mount /dev/nbd0p1 /home/test.1.3.debug/mnt/boot mount: special device /dev/nbd0p1 does not exist partx -a /dev/nbd0 Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] How to redirect QEMU -serial output to both a file and the terminal or a port?
On 11/11/2018 11:57, Ciro Santilli wrote: https://superuser.com/questions/1373226/how-to-redirect-qemu-serial-output-to-both-a-file-and-the-terminal-or-a-port I would like to be able to both interact with the system via the command line, but also get the output to a file at the same time. If I do: qemu-sysem-x86_64 -serial stdio |& tee file then it mostly works, but I would like to avoid any Bash operations and let QEMU do the heavy lifting for me. For example, I'm using Python, and it is not so simple to implement a reliable `tee` there. If I do: qemu-sysem-x86_64 -serial file:myfile It redirects to the file and I can't give any input. Is there a way to "combine" both `file:` and `stdio` to a single `-serial`? Multiple `-serial` entries just create multiple serial ports instead of modifying a single one. I'm also interested if it works with telnet as in: -serial tcp::1234,server,nowait Try using the socat and tee commands to set up a pty that writes to file and your I/O, then tell qemu to connect the serial port to your pty. This even works for multiple virtual serial ports, each with their own terminal window or ssh session. I'm not completely sure, but something like In process/shell connected to terminal window: socat PTY:link=/run/yourpath/vm1serial,wait-slave - | tee file In process/shell not necessarily connected to terminal window: qemu-system-x86_64 -serial /run/yourpath/vm1serial Where "yourpath" and "file" is up to you There is also the "-serial pty" option to qemu, but my copy of the qemu 2.8.0 manpage leaves it basically undocumented (it says it creates a pty, but not what it does with it). Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] (newbie) qemu unexpectedly closed the monitor: ftruncate: Invalid argument
On 26/10/2018 21:54, Archna Johnson wrote: Hello, While trying to power on a VM using KVM-qemu, I see this failure. Could someone please advise about what I am doing wrong? libvir: QEMU Driver error : internal error: qemu unexpectedly closed the monitor: ftruncate: Invalid argument 2018-10-26T19:18:06.394822Z qemu-kvm: -object memory-backend-file,id=memnvdimm0,prealloc=yes,mem-path=/dev/dax0.0,share=yes,size=4294967296: unable to map backing store for guest RAM: Invalid argument Did the qemu process exit at about this time? If so, the error message about "unexpectedly closed the monitor/ftruncate" message is probably just libvirt being surprised that qemu exited and you should focus on the other error message: "unable to map backing store for guest RAM: Invalid argument". Below is some information about the software and logs that I could pull: Thanks! Archna virsh version Compiled against library: libvirt 3.9.0 Using library: libvirt 3.9.0 Using API: QEMU 3.9.0 Running hypervisor: QEMU 2.10.0 Red Hat Enterprise Linux Server release 7.5 (Maipo) Linux 4.19.0-1.el7.elrepo.x86_64 #1 SMP Mon Oct 22 10:40:32 EDT 2018 x86_64 x86_64 x86_64 GNU/Linux ndctl version 63+ cat /var/log/libvirt/qemu/dax_vm.log 2018-10-26 19:18:06.341+: starting up libvirt version: 3.9.0, package: 14.el7_5.8 (Red Hat, Inc. <http://bugzilla.redhat.com/bugzilla>, 2018-09-04-07:27:24, x86-019.build.eng.bos.redhat.com), qemu version: 2.10.0(qemu-kvm-rhev-2.10.0-21.el7_5.6), hostname: LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin QEMU_AUDIO_DRV=none /usr/libexec/qemu-kvm -name guest=dax_vm,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-4-dax_vm/master-key.aes -machine pc-i440fx-rhel7.5.0,accel=kvm,usb=off,dump-guest-core=off,nvdimm=on -cpu host -m size=67108864k,slots=2,maxmem=75497472k -realtime mlock=off -smp 8,sockets=8,cores=1,threads=1 -numa node,nodeid=0,cpus=0-7,mem=65536 -object memory-backend-file,id=memnvdimm0,prealloc=yes,mem-path=/dev/dax0.0,share=yes,size=4294967296 -device nvdimm,node=0,memdev=memnvdimm0,id=nvdimm0,slot=0 -uuid 56012bbe-d950-11e8-8102-005056b801b1 -display none -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-4-dax_vm/monitor.sock,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -boot strict=on -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -device virtio-scsi-pci,id=scsi0,bus=pci.0,addr=0x2 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x3 -drive file=/dev/fox22_ds/dax_vm_DataONTAPv.raw,format=raw,if=none,id=drive-scsi0-0-0-0,cache=directsync -device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0,bootindex=1 -drive file=/dev/fox22_ds/dax_vm_coredisk,format=raw,if=none,id=drive-scsi0-0-0-1,cache=directsync -device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=1,drive=drive-scsi0-0-0-1,id=scsi0-0-0-1 -drive file=/dev/fox22_ds/dax_vm_root_1,format=raw,if=none,id=drive-scsi0-0-0-2,cache=directsync -device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=2,drive=drive-scsi0-0-0-2,id=scsi0-0-0-2 -drive file=/dev/fox22_ds/dax_vm_data_1,format=raw,if=none,id=drive-scsi0-0-0-3,cache=directsync -device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=3,drive=drive-scsi0-0-0-3,id=scsi0-0-0-3 -drive file=/dev/fox22_ds/dax_vm_config.iso,format=raw,if=none,id=drive-ide0-1-0,readonly=on -device ide-cd,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -netdev tap,fd=26,id=hostnet0,vhost=on,vhostfd=31 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:ce:70:2c,bus=pci.0,addr=0x5 -netdev tap,fd=32,id=hostnet1,vhost=on,vhostfd=33 -device virtio-net-pci,netdev=hostnet1,id=net1,mac=52:54:00:9c:a7:ac,bus=pci.0,addr=0x6 -chardev socket,id=charserial0,host=0.0.0.0,port=7200,telnet,server,nowait -device isa-serial,chardev=charserial0,id=serial0 -chardev socket,id=charserial1,host=0.0.0.0,port=7213,telnet,server,nowait -device isa-serial,chardev=charserial1,id=serial1 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x4 -msg timestamp=on 2018-10-26 19:18:06.341+: Domain id=4 is tainted: high-privileges 2018-10-26 19:18:06.341+: Domain id=4 is tainted: host-cpu ftruncate: Invalid argument 2018-10-26T19:18:06.394822Z qemu-kvm: -object memory-backend-file,id=memnvdimm0,prealloc=yes,mem-path=/dev/dax0.0,share=yes,size=4294967296: unable to map backing store for guest RAM: Invalid argument 2018-10-26 19:18:06.455+: shutting down, reason=failed snippets from the xml file for the VM: 75497472 71303168 71303168 8 hvm /dev/dax0.0 4194304 0 Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16
Re: [Qemu-discuss] Port-forwarding to a QEMU/KVM VM
On 21/09/2018 08:06, Jaap Winius wrote: Hi folks, For years I've maintained Debian Linux servers that run QEMU/KVM virtual machines along with ppp, bridge-utils (brctl) and iptables. In these cases it is simple to configure the latter to forward packets from the Internet, arriving on interface ppp0, over to VMs on the internal bridged interface, br0. This interface is configured like: auto enp35s0 iface enp35s0 inet manual auto br0 iface br0 inet static address 10.1.1.5/24 gateway 10.1.1.1 bridge_ports enp35s0 bridge_stp off bridge_fd 0 bridge_ageing 0 bridge_maxwait 2 The relevant iptables rules I use to forward HTTPS traffic on to the VM, 10.1.1.10, look like: iptables -t nat -A PREROUTING -i ppp0 \ -p tcp --dport 443 -j DNAT --to 10.1.1.10:443 iptables -A FORWARD -i ppp0 \ -p tcp -d 10.1.1.10 --dport 443 --syn -m state --state NEW -j ACCEPT However, this forwarding configuration stopped working after ppp and iptables were moved to a physically separate gateway machine. Now the packets from outside are still forwarded on to the VM (that uses a virtio network interface), which responds, but the replies never make it out of the bridged network segment. Using tcpdump, the reply packets can be detected on the VM and the host server, but not on the gateway. How can port-forwarding functionality best be restored in this case? To be honest, this problem seems more like something to do with brctl than with QEMU/KVM, but as brctl appears to be the bridge of choice in these environments, surely someone here has already encountered this problem and found a fix for it. And as I'm rather stumped on this one, I'd be very grateful if someone were to share their solution here. Check the following: Your VM MAC addresses are unique network wide and don't have the multicast bit set (so packets can be routed correctly by your layer 2 Ethernet switches). Your IP tables allow ARP requests (for IPv4) and neighbor discovery ICMPv6 (for IPv6) to travel between your VMs and the physical network (so other machines, including the gateway can find the IP-to-MAC mapping for your VMs). Your physical Ethernet switch is configured to accept that the cable to your machine behaves like a cable to another Ethernet switch (this is the default for most switches with a few annoying exceptions). If you have a spare machine with two ethernet ports, configure it to bridge the two ports with brctl and run wireshark on that, then "splice" it into the connection between your machine and the physical switch, then again between the physical switch and the gateway. Look if the packets are dropped on one side of the physical switch or the other. Also look for any Layer 2 and layer 3 packets that negotiate the routing of IP and MAC addresses. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] How to check cpu running mode?
On 18/08/2018 05:59, krishnaLee wrote:
Jakob:
I need more help,just now,I'm trigger a page fault in 64-bit mode,see
this picture:
https://github.com/krishna116/test/blob/master/test-qemu-in-64bit-mode.png
so I can write some system mode code accroding to this information,
but my follow code seems can't get the right answer, is my algorithm wrong?
//this is my algorithm in 64-bit mode:
#define CS_L_BIT 0x1<<(32+22-1)
//CS_selector_index=0x7; //0x38>>3
//GDTR_base=0x7E9; //0x7e9f598&~0x
int64* segment_descriptor_address=(int64*)(int64)
(*(0x7E9+0x7*8*2)); //GDTR_base+CS_selector_index*
sizeof(Segment_Descriptor*2)
if((*segment_descriptor_address)_L_BIT)
{
//it is 64bit mode
}else
{
//it is Compatibility mode
}
thank you,
krishna
Detecting if the CPU is in long mode (i.e. is running 64 bit code!) is
not useful if you are not going to write low level assembler or similar
code that does something different depending on it.
Because "being in long mode" means "running a 64 bit program or OS
kernel" and similar for the other cases my (completely untested) example
code tried to detect.
So most of the time, this is a difference you choose up front, not
something you detect.
Enjoy
Jakob
--
Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com
Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10
This public discussion message is non-binding and may contain errors.
WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] How to check cpu running mode?
On 17/08/2018 15:32, Jakob Bohm wrote: On 17/08/2018 12:27, krishnaLee wrote: Hi, as to my known,intel x64 cpu can work at 64-bit mode or Compatibility mode, so thery are two kind of x64-mode, I find QEMU can run at each of them, I want to write som C code to check it if it is running at 64-bit mode or Compatibility mode, what's the key? This mode is a bit in the segment descriptor referenced by the current CS register, the following or similar code should be able to check this(note that this code has to use only instructions that have the same encoding in all CPU modes!): ; UNTESTED HYPOTHETICAL CODE!!! ; First, check that we are not running in real or VM8086 ; mode, because then the following will not work ; (LAR will trigger an undefined opcode exception). ; Doing this is left as an exercise for the reader ; The following is valid in 16, 32 and 64 bit protected mode ; the instructions should have the same opcodes, they are ; shown here using the 32 bit mode Intel syntax .386 MOV EAX,CS LAR EAX,EAX JNZ SHORT modeerror SHR EAX,21 JCSHORT mode64 SHR EAX,1 JC SHORT mode32 mode16j: .286 ; Code here is 16 bit code and can use all 16 bit instructions JMP mode16 ; This can be a longer jmp now that we know the ; instruction encoding mode32j: .386 ; Code here is 32 bit code and can use all 32 bit instructions JMP mode32 ; This can be a longer jmp now that we know the ; instruction encoding mode64j: .amd64 ; Note, not sure what directive tells MASM to generate 64bit opcodes ; Code here is 64 bit code and can use all 64 bit instructions JMP mode64 ; This can be a longer jmp now that we know the ; instruction encoding modeerror: ; This really shouldn't happen! ; One possible way would be if something changed the CS descriptor ; in the descriptor table without reloading CS ; Code here is still restricted to mode independent instructions! ; Alternatively, a code sequence could be constructed that has different ; effects depending on CPU mode but doesn't directly query the CS ; descriptor table entry, avoiding the modeerror case above. ; UNTESTED HYPOTHETICAL CODE!!! ; Perhaps .386 XOR EAX,EAX ; XOR AX,AX if 16 bit mode NOT EAX ; 16 bit: AX is now 0h ; 32 bit: EAX is now 0h ; 64 bit: RAX is now 0h (UNTESTED) SHR EAX,16 JNC SHORT mode16j SHR EAX,16 JNC SHORT mode32j mode64j: ; as above mode32j: ; as above mode16j: .286 SMSW AX SHR AX,1 JNC SHORT mode16real ; Add code here to test if this is VM8086 mode or 16 bit ; protected mode ; First set up a stack, 4 byte aligned ; CODE MISSING ; Next do the classic tricks to detect if this is at least ; a 386, this involves using 16 bit PUSHF and POPF ; CODE MISSING ; The missing code should jump to mode16rj if < 386 ; Next use code to determine if in VM8086 mode, note that ; PUSHFD hides the desired EFLAGS bit from us, so another ; trick is needed ; CODE MISSING ; The missing code should jump to mode16rj if real mode mode16vj: .286 ; Code here is 16 bit code and can use all 16 bit instructions ; VM8086 mode JMP mode16v ; This can be a longer jmp now that we know the ; instruction encoding mode16rj: .286 ; Code here is 16 bit code and can use all 16 bit instructions ; Real mode, full CPU access! JMP mode16v ; This can be a longer jmp now that we know the ; instruction encoding Corrections: In the first example, the first shift should be SHR EAX,22 In the second example, the first shift should be SHR EAX,17 And the entire 16 bit case needs to be regrouped: mode16rj: .286 ; Code here is 16 bit code and can use all 16 bit instructions ; Real mode, full CPU access! ; May or may not be the unofficial large-segment real mode JMP mode16r ; This can be a longer jmp now that we know the ; instruction encoding mode16j: .286 ; 16 bit code, maybe protected, real, VM8086 or even ; the unofficial large-segment real mode ; First set up a stack, 4 byte aligned ; CODE MISSING OR ASSUME STACK ALREADY VALID ; Next do the classic tricks to detect if this is at least ; a 286, this involves using 16 bit PUSHF and POPF ; CODE MISSING ; The missing code should jump to mode16rj if < 286 SMSW AX SHR AX,1 JNC SHORT mode16rj ; So now it is protected mode or VM8086 mode ; First check if at least a 386 (we know it's at least 286) ; CODE MISSING ; The missing code should jump to mode16pj if a 286 ; Add code here to test if this is VM8086 mode or 16 bit ; protected mode ; Next us
Re: [Qemu-discuss] How to check cpu running mode?
On 17/08/2018 12:27, krishnaLee wrote: Hi, as to my known,intel x64 cpu can work at 64-bit mode or Compatibility mode, so thery are two kind of x64-mode, I find QEMU can run at each of them, I want to write som C code to check it if it is running at 64-bit mode or Compatibility mode, what's the key? This mode is a bit in the segment descriptor referenced by the current CS register, the following or similar code should be able to check this(note that this code has to use only instructions that have the same encoding in all CPU modes!): ; UNTESTED HYPOTHETICAL CODE!!! ; First, check that we are not running in real or VM8086 ; mode, because then the following will not work ; (LAR will trigger an undefined opcode exception). ; Doing this is left as an exercise for the reader ; The following is valid in 16, 32 and 64 bit protected mode ; the instructions should have the same opcodes, they are ; shown here using the 32 bit mode Intel syntax .386 MOV EAX,CS LAR EAX,EAX JNZ SHORT modeerror SHR EAX,21 JCSHORT mode64 SHR EAX,1 JC SHORT mode32 mode16j: .286 ; Code here is 16 bit code and can use all 16 bit instructions JMP mode16 ; This can be a longer jmp now that we know the ; instruction encoding mode32j: .386 ; Code here is 32 bit code and can use all 32 bit instructions JMP mode32 ; This can be a longer jmp now that we know the ; instruction encoding mode64j: .amd64 ; Note, not sure what directive tells MASM to generate 64bit opcodes ; Code here is 64 bit code and can use all 64 bit instructions JMP mode64 ; This can be a longer jmp now that we know the ; instruction encoding modeerror: ; This really shouldn't happen! ; One possible way would be if something changed the CS descriptor ; in the descriptor table without reloading CS ; Code here is still restricted to mode independent instructions! ; Alternatively, a code sequence could be constructed that has different ; effects depending on CPU mode but doesn't directly query the CS ; descriptor table entry, avoiding the modeerror case above. ; UNTESTED HYPOTHETICAL CODE!!! ; Perhaps .386 XOR EAX,EAX ; XOR AX,AX if 16 bit mode NOT EAX ; 16 bit: AX is now 0h ; 32 bit: EAX is now 0h ; 64 bit: RAX is now 0h (UNTESTED) SHR EAX,16 JNC SHORT mode16j SHR EAX,16 JNC SHORT mode32j mode64j: ; as above mode32j: ; as above mode16j: .286 SMSW AX SHR AX,1 JNC SHORT mode16real ; Add code here to test if this is VM8086 mode or 16 bit ; protected mode ; First set up a stack, 4 byte aligned ; CODE MISSING ; Next do the classic tricks to detect if this is at least ; a 386, this involves using 16 bit PUSHF and POPF ; CODE MISSING ; The missing code should jump to mode16rj if < 386 ; Next use code to determine if in VM8086 mode, note that ; PUSHFD hides the desired EFLAGS bit from us, so another ; trick is needed ; CODE MISSING ; The missing code should jump to mode16rj if real mode mode16vj: .286 ; Code here is 16 bit code and can use all 16 bit instructions ; VM8086 mode JMP mode16v ; This can be a longer jmp now that we know the ; instruction encoding mode16rj: .286 ; Code here is 16 bit code and can use all 16 bit instructions ; Real mode, full CPU access! JMP mode16v ; This can be a longer jmp now that we know the ; instruction encoding Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] qcow2 performanceimprove
On 17/08/2018 04:28, yang.bi...@zte.com.cn wrote: If there is no backing file or snapshot you still need to fill the>>>cluster with zeroes, and that's going to be slower with larger>>>clusters.>>>> If not fill zeroes and only write guest data ,what`s wrong could>> happen ?>The following could happen:> 1) Guest reads at offset [0, 4k] -> there's only zeroes> 2) Guest writes at offset [8k, 16k]> 3) Guest reads at offset [0, 4k] -> there's something else now>Berto Why could guest read the area at offset [0, 4k] has not be writen yet ? Yang.bin Because on any real computer (which qemu emulates), you can read any sector of a disk, even if it only contains whatever the disk factory put there during manufacture, usually something like 0x00, 0xA5, 0xFF etc. qcow2 unallocated virtual disk clusters contain 0x00 when read and there are commands that convert all-0x00 clusters back to being unallocated. So if a a guest machine issues a write to only part of a qcow2 unallocated cluster, the qcow2 needs to emulate that the rest of that qcow2 cluster still contains the 0x00. qcow2 might do that either by actually writing 0x00 to wherever it now allocated a full qcow2 cluster of storage in order to save the changed part, or create a complex data structure to remember which part of which cluster is virtually zero but not actually stored anywhere. Either way has speed overhead, but the first is usually the fastest. Now if the qcow2 file is created with the same qcow2 cluster size as the actual cluster size in the guest filesystem, and the start of the guest file system is aligned so all filesystem clusters start at disk offsets that are multiples of the filesystem cluster size (already required for physical disks with 4096-byte sectors), then this partial write scenario will happen vary rarely. That cluster size to cluster size alignment is generally a best practice for performance of thin provisioned virtual disks of all kinds, not just for qcow2 or other qemu formats. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] newbie question: how to connect to a running guest with qemu monitor
On 29/05/2018 21:15, Lentes, Bernd wrote: Hi, sorry for asking this propable silly question, but i googled and didn't find an appropriate answer. I have a SLES 12 SP3 host with a SLES 10 SP4 guest and some windows 7 guests. I'd like to connect to both os when they are running.. The command-line for the SLES 10 is: ps aux|grep kvm|grep -E 'chardev|mon' qemu 11131 4.2 53.0 13463948 8583652 ?Sl May03 1605:21 /usr/bin/qemu-system-x86_64 -machine accel=kvm -name guest=mausdb2,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-10-mausdb2/master-key.aes -machine pc-i440fx-2.9,accel=kvm,usb=off,dump-guest-core=off -m 8320 -realtime mlock=off -smp 2,sockets=2,cores=1,threads=1 -uuid 5db2211a-1b91-76a1-11dd-1bf1e85ace75 -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-10-mausdb2/monitor.sock,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -boot strict=on -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/var/lib/libvirt/images/mausdb/mausdb.dd,format=raw,if=none,id=drive-virtio-disk0,cache=none -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -drive file=/mnt/idg-2/SysAdmin_AG_Wurst/software_und_treiber/linux/knoppix/KNOPPIX_V8.1-2017-09-05-DE.iso,format=raw,if=none,id=drive-ide0-0-0,readonly=on,cache=none -device ide-cd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=2 -netdev tap,fd=24,id=hostnet0 -device e1000,netdev=hostnet0,id=net0,mac=52:54:00:37:92:aa,bus=pci.0,addr=0x3 -vnc 127.0.0.1:0 -device cirrus-vga,id=video0,bus=pci.0,addr=0x2 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 -msg timestamp=on I created the guest with virt-manager. Is there a way to connect with qemu to the console/GUI of a running guest ? It's very close, but not exact (you need to do it one command at a time, but there may be a libvirt programming interface to do this more easily for automation). # virsh qemu-monitor-command --help For example # virsh qemu-monitor-command $guestname --hmp help Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] Impact of incorrectly-mapped OVMF
On 20/04/2018 10:54, Laszlo Ersek wrote: ... Unfortunately this detection logic is extremely prone to mis-use (and esp. it was never meant to deal with case (4)) and to causing confusion. For this reason, I have had patches for a long time now that allow the user to compile "-bios" support out of OVMF, with a build flag (-DMEM_VARSTORE_EMU_ENABLE=FALSE). Then you will simply become unable to boot with "-bios" -- the firmware will just hang with a black screen. (If you capture the debug log (see OvmfPkg/README), you'll know what's up though.) Note that this would not be the default -- the default build wouldn't change. Wouldn't it be much more user-friendly for this case to output a cleartext error message to both screen and any non-screen console (such as qemu stdout or a real serial console on a physical machine), before halting the boot. Hopefully UEFI has a standard way to do that for early boot errors. A nice default Tianocore message could be: "UEFI BIOS flashed at wrong offset see your hardware manual. System halted". With an option for a motherboard build config (such as qemu) to specify a different text such as: "UEFI OVMF_CODE.fd passed to qemu -bios, pass it to -pflash instead, see qemu-system-x86_64(1). System halted." Docs/samples could show an override such as "UEFI BIOS flashed at wrong address, see https://help.example.com/badflash/ for recovery instructions. System halted." Similarly there can be a second error message: "NVRAM config flash chip not found, check for hardware damage. System halted." With the qemu board override "OVMF_VARS.fd not found in VM, please pass pass to qemu -, see qemu-system-x86_64(1). System halted." And doc/sample override: "Chip U123 not responding, check for mainboard damage. System halted." And a third: "UEFI NVRAM config flash chip is read only, check your Config jumper setting. Booting as configured." With a qemu board override referring to the relevant qemu option. (A secure pro motherboard may have such a hardware lockdown feature). etc. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. http://www.wisemo.com Transformervej 29, 2860 Soborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] KVM without microcode
On 11/04/2018 21:54, JT wrote: (I've also posted this to the KVM mailing list) Hey All A hopefully simple question: If a KVM Hypervisor is using a kernel that identifies itself as using "Full generic retpoline", does this mean that the hypervisor and other guests are safe from a malicious guest trying to exploit Spectre V2, even if we haven't updated our CPU microcode to support IBPB or IBRS? My confusion arrises from the Intel Retpoline PDF which states: "RET has this behavior on all processors which are based on the Intel=C2=AE microarchitecture codename Broadwell and earlier when updated with the latest microcode." https://software.intel.com/sites/default/files/managed/1d/46/Retpoline-A-Br= anch-Target-Injection-Mitigation.pdf I understand that RET has nothing to do with IBPB or IBRS, but how do I know if my CPU has this RET behaviour that retpoline can make use of? Thanks In general, the RetPoline workaround needs to be compiled into all potentially Spectre V2 affected software, including Guest kernels. This is because RetPoline is a code change that prevents some Spectre attacks from actually causing the code to speculatively do the wrong thing, even if the CPU is vulnerable. So RetPoline only protects the code that uses RetPoline wherever it would normally use an indirect branch. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] qemu-kvm and qemu-system-x86_64
On 22/03/2018 18:29, Deneau, Tom wrote: My situation: I want to run a KVM-accelerated qemu but I want to build everything involved from the latest sources and use those instead of any downloadable distro packages. (This is on SLES12-SP3 although I don't think the distro matters). I am able to build and install from * http://github.com/qemu/qemu using ../configure --target-list=x86_64-softmmu --enable-kvm --prefix=/usr --sysconfdir=/etc * http://github.com/libvirt/libvirt using ./configure --prefix=/usr --sysconfdir=/etc --localstatedir=/var Before running make install, I also removed the existing distro packages like libvirt, libvirt-client, qemu-kvm, etc. My question: In the installed distro packages there was an executable, /usr/libexec/qemu-kvm But after building and installing qemu from sources as above, I did not see any new qemu-kvm. Is there some other repository I need to build from to get qemu-kvm? I ask because I am starting vms using virsh define xyz.xml; virsh start xyz. And in the original .xml files I had the following under devices: /usr/libexec/qemu-kvm Can I replace that with /usr/local/bin/qemu-system-x86_64 And given that I am already using hvm Will I get the same KVM acceleration as I was previously getting by using qemu-kvm? -- Tom Deneau For many years now, qemu-kvm has been a small shell script that prefixes "--enable-kvm" to the other arguments. Something like (guessing since its no longer on my system): #!/bin/sh exec qemu-system-x86_64 --enable-kvm "$@" It was only a real program before kvm support was merged into standard qemu. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] Fwd: qemu VM cannot be killed
Does this represent a security vulnerability (or two)? Specifically: - If this can be triggered by code running in the Guest OS (as opposed to via the qemu commandline/config) it could be considered a violation of the sandboxing. - If this can be done when qemu is not started by root, it could be considered a violation of basic kernel security on the host, since a non-root process should have no way to cause a kernel hang that cannot be resolved by kill -9, regardless if the related syscalls are made by a qemu binary or by a custom exploit program. (Note: Such vulnerabilities would not normally be discussed in public, but since the report is already public there is no further harm). On 15/11/2017 15:38, Michael S. Tsirkin wrote: Yes - I suspect a packet is stuck somewhere in networking stack. This is what vhost is waiting for. Yes, host reboot is the only way out. RHEL disables zero copy tx in vhost to avoid these issues. On Wed, Nov 15, 2017 at 02:55:32PM +0100, Lukáš Kubín wrote: CC-ing Michael and Jason as I was suggested in OFTC:#virt forum. Thanks! -- Forwarded message -- From: Lukáš Kubín <lukas.ku...@gmail.com> Date: Wed, Nov 15, 2017 at 1:39 PM Subject: qemu VM cannot be killed To: qemu-discuss@nongnu.org Hi, we've experienced an issue with kvm instance which got stuck at reboot. It's an OpenStack environment, with OpenContrail networking (vrouter agent running on host), Ubuntu 16.04. Machine was first called to reboot from guest OS by user, had issues with NFS unmount during that, user sent a hard-reboot call from OpenStack again then. Then we (platform operator) got involved, tried to "virsh destroy" it with this output: error: Failed to destroy domain instance-4243 error: Failed to terminate process 140529 with SIGKILL: Device or resource busy Neither "kill -9" sent to the qemu process helped. Good guys at OFTC:#virt have guided me to collect the following traces and ask for help here: # cat /proc/140529/wchan vhost_net_ubuf_put_and_wait # cat /proc/140529/stack [] vhost_net_ubuf_put_and_wait+0x54/0xa0 [vhost_net] [] vhost_net_ioctl+0x354/0x8a0 [vhost_net] [] do_vfs_ioctl+0xa1/0x5f0 [] SyS_ioctl+0x79/0x90 [] entry_SYSCALL_64_fastpath+0x1e/0xa8 [] 0x The versions we use are: • kernel 4.8.0-41-generic • qemu-kvm 1:2.5+dfsg-5ubuntu10.2~xenial0+contrail1 • libvirt-bin 1.3.1-1ubuntu10.1~xenial1+contrail1 What can be the cause for this error? What can we do in such a situation to destroy the VM - is physical server reboot the only option? Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: [Qemu-discuss] Questions regarding Qemu and AARCH64
On 08/08/2017 10:22, sanjeev wrote: Hi I come from the windows world and my experience with all things linux is pretty limited. Trying to improve upon the same and understand and learn a few low level concepts about platform bringup, boot steps etc along with ARMv8 using Qemu. I have a few questions regarding the same: Question 1) I was told that I shouldn't use the -kernel option if I am running a bare metal code. I just created a file with a reset handler located at address Zero using a linker file and ran it with *qemu-system-aarch64 -s -S -machine virt -cpu cortex-a53 -kernel test.elf* and I could get my reset handler break point hit. Does qemu place my code at address zero in the system memory map and starts executing it but address 0x0 is reserved for boot flash as you mentioned earlier. Also is there a different option other than -kernel for bare metal. I believe something passed to -kernel and -initrd will be loaded according to the (emulated) platform specific way in which a Linux kernel would be loaded. Documentation for that may be found in the "arch" subdirectory of the cross-platform Linux kernel sources from kernel.org and/or in the "doc" subdirectory of said sources.Once the (emulated) machine hands over execution to that "Linux kernel", the code is no longer required to act as if it was Linux. Question 2) What exactly is virtio in Qemu virt board emulation. In qemu source virt.c, there's a system memory map with memory mapped range for PCIe. How is this to be used. Does qemu have any option to plug a PCIe device into the configuration and use the PCI address map range. I have limited knowledge of these lower level details. Trying to learn theconcepts of how a board is brought up in general along with some ARMv8 concepts. virtio is a set of completely fictional "hardware" devices that only exist in qemu-emulated virtual machines, and which are optimized for that implementation scenario rather than for being implemented with real hardware. This means that if an OS or bare metal program has an equally well written virtio device driver, this will typically outperform any emulation of similar real world hardware such as an AHCI-compliant SATA controller or a 16450- compatible serial port controller. Some virtio devices even perform functions that make no sense on real hardware, such as handing some of the assigned RAM back to the host machine. Most virtio devices will appear in the list of devices found via platform- standard enumeration mechanisms such as the PCI bus hardware detection mechanism. Thanks and Regards Sanjeev Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
