Re: recover snapshot

[Alberto Garcia]

On Thu, Jul 18, 2024 at 07:33:14PM +0200, Jakob Bohm via wrote:
> > > > I accidentally delete the backing file. It was a very old
> > > > 200GB file from 2018. The snapshot is 1 TB and I can't use it.
[...]
> > > > Is there something I can do to recover the files in the snapshot?
> > > Well, you deleted all the virtual disk sectors that were
> > > unchanged since the backing file.
> > > 
> > > To get back all disk sectors changed since the backing file,
> > > just repoint the qcow2 image to an all zero backing file .
> > 
> > Interesting, it's an idea I suggested on IRC (where Leonel came
> > first), but someone mentioned it would not work.
> > Will snapshot apply the diff "blindly", without checking src block
> > (in backing file) was the one expected?
> 
> I believe qcow2 itself has no information about the expected sector
> contents, but the file system and applications in the virtual
> machine might.

A qcow2 file is a collection of data clusters of the same size (you
can see the size with 'qemu-img info').

Each cluster can be allocated (it contains the actual data) or
unallocated (the data is read from the backing file, or is zero if
there is no backing file). There are no checksums or any kind of
information about the expected contents, you can actually replace one
backing file with a different one and QEMU won't notice (your virtual
machine will, of course).

If you deleted the backing file then the clusters that were
unallocated in the active file are now zeroes. That's the data that
you lost.

You can see the affected clusters with 'qemu-img map' but there is no
way to recover that information at the QEMU / qcow2 level.

Berto

Re: base format

[Alberto Garcia]

On Thu, Apr 25, 2024 at 02:51:28PM +0200, lacsaP Patatetom wrote:
> > > when using `qemu-img create`, why do I have to specify the
> > > format of the base image ? can't `qemu-img` detect it itself ?
> >
> > Image format detection isn't 100% reliable. Notably, a 'raw'
> > format image could in theory look like any of the other image
> > types if it happens to start with the right kind of data.
> >
> OK for the raw format, which can be anything, but not for the qcow2
> format, which is standardized (header).
> in the absence of details, `qemu-img` could use the qcow2 format by
> default ?

You probably don't want the integrity of your data to depend on
whether QEMU detects correctly the format of the backing file every
time you open it.

If the problem is that you have to type extra characters every time
you create an image that can be easily solved with a shell script or
an alias.

Berto

Re: Can "throttle-group"(added by "object-add") be lively updated through QMP cmd?

[Alberto Garcia]

On Tue, Jan 16, 2024 at 09:37:48AM +, Chun Feng Wu wrote:
>   * Is there any reason QEMU doesn’t provide QMP cmd like
>   “object-set” to update object?

You can use qom-set for throttle-group, here's a simple example:

   https://github.com/qemu/qemu/blob/v8.2.0/tests/qemu-iotests/184#L108

Berto

Re: Directory does not fit in FAT16 (capacity 516.06 MB)

[Alberto Garcia]

On Fri, Dec 15, 2023 at 09:15:22AM +0100, lacsaP Patatetom wrote:
> here's a little script following the exchanges above.
> perhaps it will be useful to others...

If you want a full disk image with its partition table I don't know if
you are aware that those tools can work with offsets, e.g.:

  $ truncate -s 500M disk.img
  $ echo -e 'label:gpt\n2048,,U' | sfdisk disk.img
  $ mkfs.fat --offset 2048 disk.img 
  $ mcopy -i disk.img@@1M -s /path/to/files ::

where 2048 is the offset of the first partition (2048 sectors = 1MB)

Berto

Re: Directory does not fit in FAT16 (capacity 516.06 MB)

[Alberto Garcia]

On Wed, Dec 13, 2023 at 04:33:00PM +0100, lacsaP Patatetom wrote:
> having said that, and if you respect the constraints ("what you
> should never do..."), vvfat is a really nice way of performing and
> repeating quick tests without having to go through the process of
> building/creating/updating disk images.

While I agree with what you say here and I also think that it would be
nice if that driver was maintained and had more features... really,
creating a temporary FAT image of a few hundred MBs with some files
inside takes less than a second in a modern computer, you can boot
QEMU like this and you won't probably even notice the difference:

   $QEMU -drive if=ide,file=$(create-image.sh),driver=raw

Berto

Re: Directory does not fit in FAT16 (capacity 516.06 MB)

[Alberto Garcia]

On Wed, Dec 13, 2023 at 01:27:49PM +0100, Narcis Garcia wrote:
> A FAT16 partition (0x06) can be as large as 2 GiB

That's why I say that there's probably a limitation in the QEMU vvfat
driver.

Berto

Re: Directory does not fit in FAT16 (capacity 516.06 MB)

[Alberto Garcia]

On Wed, Dec 13, 2023 at 11:46:51AM +0100, lacsaP Patatetom wrote:
> to quickly test my efi application `./esp/EFI/BOOT/BOOTX64.EFI`, I use the
> following command `qemu-system-x86_64 -accel kvm -machine q35 -m 1024 -bios
> /usr/share/edk2/x64/OVMF.fd -hda fat:./esp/ -snapshot` but it has recently
> failed with the error message given in subject.

I'm not familiar with the QEMU vvfat driver, but FAT16 does allow
larger filesystems, I suspect there's a limitation there?

> for various reasons, i can't purge/reinitailize/empty the folder :
> what other options do I have with qemu or do I have to start from a
> lighter folder ?

You can create a FAT image, put your files there using e.g mtools and
boot from that image:

$ fallocate -l 1G esp.img
$ mformat -i esp.img
$ mcopy -i esp.img -s /path/to/files ::

Berto

Re: swap

[Alberto Garcia]

For non-Spanish speakers: he's asking about how memory swapping works
on a VM. I'm answering his question and telling him to use English on
this mailing list, or write me directly to continue using Spanish.

Answer in Spanish below:

Hola Pablo,

esta es una lista de correo en inglés. Si necesitas responder a este
correo hazlo en inglés o escríbeme en privado.

El huésped usa la memoria de intercambio igual que un equipo normal
sin virtualizar. Si le asignas una partición de swap ésta se
almacenará dentro del disco qcow2. Si el huésped no tiene swap pues
sólo puede usar la memoria que el hayas asignado al arrancar la
máquina virtual.

Saludos,

Berto

On Mon, Nov 20, 2023 at 09:48:55AM +0100, Pablo Buzón Garrido wrote:
> Hola,
> 
> Tengo una pregunta acerca del comportamiento de qemu y la memoria de
> intercambio. Me gustaría saber de donde coge el sistema huésped su propia
> memoria de intercambio; si es de la misma RAM del host (es lo que me
> gustaría) o si es del disco duro (en este caso de qcow2) como lo haría un
> equipo normal sin virtualizar.
> 
> En mi caso lo que tengo es una orangepi 5 con un sistema ubuntu 22.04 arm64
> compatible con este equipo como Host. Y luego virtualizado con qemu tengo
> varios armbian (debian 12) arm64. He revisado con gparted en los huéspedes
> y veo que no tengo partición swap de intercambio, por lo que imagino se
> hará con la gestión de un fichero que no sé si se guarda en el disco duro
> virtual (qcow2) o si se guarda en la misma RAM del Host.
> 
> El Host he revisado y tampoco tiene partición swap de intercambio.
> 
> Muchas gracias
> Un saludo!

Re: Is it normal to get bigger qcow2 image after blockcopy?

[Alberto Garcia]

On Tue, Nov 07, 2023 at 05:32:22PM +0800, Han Han wrote:
> >>> Before blockcopy, check source image:  ❓
> >>>
> >>   # qemu-img info -U
> >>> /var/lib/avocado/data/avocado-vt/images/jeos-27-x86_64.qcow2  ❓
> >>>
> >>
> > Was this a compressed qcow2 image when you started? maybe you started with
> > an appliance image?
> >
> Hi, Nir, I have a question here: How to check if a qcow2 image is
> compressed?

$ qemu-img map /path/to/image.qcow2

If it says "File contains external, encrypted or compressed clusters"
and the image is not encrypted then that's probably it.

> The qcow2 file /var/lib/avocado/data/avocado-vt/images/copy.qcow2 is
> created without compress options, while the qemu-img info for it
> shows `compression type: zlib`. Is that mean the image is compressed
> as well?

No, in qcow2 there is actually no such thing as a compressed image as
opposed to an uncompressed image.

qcow2 images contain a collection of data clusters, and each
individual cluster can be either compressed or uncompressed.

That zlib that qemu-img reports is the algorithm that is used for
compressed clusters *IF* they exist, but it's perfectly possible that
an image does not have any compressed cluster.

Berto

Re: Performance Discrepancies and Limitations in Local Storage IOPS Testing

[Alberto Garcia]

On Thu, Aug 24, 2023 at 12:29:43PM +0200, Jan Wasilewski wrote:
> Ultimately, I discovered that the performance is significantly
> enhanced when the hypervisor is deployed on top of Ubuntu
> 22.04LTS. Under this setup, I was able to achieve around 100,000
> IOPS during my fio tests [1][2]. In contrast, conducting a similar
> test with the hypervisor deployed on Ubuntu 20.04LTS yielded
> significantly lower results, averaging around 20,000 IOPS [3][4].

That's quite a difference. How are you storing the virtual disks? Is
it qcow2 files? How are they created (i.e. what options are you
using?), what runtime options -if any- are you using with them (e.g.
l2-cache-size)?

Did you try to build QEMU 6.2 in Ubuntu 20.04 and see what happens
with the IOPS?

Berto

Re: Reopen qcow2 backing file from new location

[Alberto Garcia]

On Fri, Jun 23, 2023 at 11:50:10AM +0200, Stanislav Datskevych wrote:

> Unfortunately, change-backing-file only changes the string in the
> image's metadata and doesn't cause QEMU to reopen it on the running
> VM.  So in order to fully migrate VM to its new storage, the VM must
> be stopped and started.
> 
> Maybe there is a way to force qemu to reopen the backing file? 

I think that blockdev-reopen is what you are looking for:

   
https://www.qemu.org/docs/master/interop/qemu-storage-daemon-qmp-ref.html#qapidoc-780

It is option (3) from that list: first you open 
using blockdev-add and then use blockdev-reopen to make 'backing'
point to this new backing file.

Berto

Re: Using blockdev-mirror for Live Backup

[Alberto Garcia]

On Wed, May 24, 2023 at 06:35:02PM +0330, Ali Gholami Rudi wrote:
> If I prepare a patch, do you think it would be merged?

I cannot say because I'm not working on the project at the moment,
if you have an idea for an improvement I suggest that you write to
the qemu-block mailing list describing the problem in detail and the
solution that you propose.

Berto

Re: Using blockdev-mirror for Live Backup

[Alberto Garcia]

On Tue, May 16, 2023 at 11:58:30PM +0330, aliqr...@gmail.com wrote:

> We are executing hundreds of VMs in our hosts (dual AMD EPYC 7002
> CPUs + 2TB memory + 8 NVMe disks).  To cope with disk failures, we
> tried different Linux software RAID mechanisms.  At high disk usage,
> all of them faced unexpectedly high CPU load, mostly in the kernel.
> Therefore we decided not to use software RAID.

An alternative is the Quorum block driver:

   -drive if=virtio,driver=quorum,
  children.0.file.filename=disk0.qcow2,
  children.1.file.filename=disk1.qcow2,
  children.2.file.filename=disk2.qcow2,
  vote-threshold=2

   (this is supposed to go all in one line)

This is similar to RAID-1, but note that the disk images themselves
do not contain any kind of additional metadata and they don't know
that they are part of a Quorum array, it's your responsibility to put
identical images together.

Regards,

Berto

Re: dropping 32-bit host support

[Alberto Garcia]

On Fri, Mar 17, 2023 at 11:03:51AM +0300, Andrew Randrianasulu wrote:
> To be honest after sleeping over problem I found situation beyond
> ridiculous.

In the course of this thread you have had some of the most active and
knowledgeable QEMU developers explain you in a very reasonable way
the problems that the project is having and the reasons behind the
decision to deprecate 32-bit host support.

While it is perfectly understandable that you are unhappy with the
loss of a certain feature, your attitude towards the people who are
doing the actual hard work of keeping the project running has been
that of contempt ("the project needs different developer culture") and
of uniformed suggestions ("maybe the CI system should tweak the ccache
settings").

I would ask you to please reconsider before going on like this and to
think twice about whether venting your frustrations in this mailing
list is helpful or useful for anyone else.

Regards,

Berto

Re: Windows 98 ISO does not boot

[Alberto Garcia]

On Wed, Jan 11, 2023 at 12:03:53PM +0100, timothylegg wrote:
> I went to https://winworldpc.com/product/windows-98/98-second-edition and

From that page:

" Important: Only the OEM Full version is bootable. All others require an 
appropriate Windows 98 Boot Floppy. "

Berto

Re: Detecting swapped images in backing chain via qemu-img

[Alberto Garcia]

On Thu, Nov 24, 2022 at 01:04:17PM +0200, Plamen Dimitrov wrote:

> I apologize in advance if I am missing something obvious but if I
> have a backing chain like
> 
> A <- B <- C
> 
> and for some reason B is swapped with another very similar B layer,
> can qemu-img detect this in some way?

If you mean replacing B with another image with similar contents but
different from B (but the same metadata, backing chain information,
etc.), then no, because there's nothing wrong with that backing chain
from QEMU's point of view.

Berto

Re: how to use blockdev-add in hmp console

[Alberto Garcia]

On Tue, Feb 22, 2022 at 07:13:24PM +0800, Jiatong Shen wrote:

>   I am reading
> https://qemu.readthedocs.io/en/latest/interop/live-block-operations.html
> and try to learn blockdev-add command. I boot a virtual machine with
> -nographic option and entered hmp interactive console. but when I execute
> blockdev-add there, it fails with no such command. but on the given website
> looks like it could successfully translate to a qmp command.

What you see in that link is actually not HMP but QMP, using the
qmp-shell script, see scripts/qmp/qmp-shell

Berto

Re: How to use SHA256 in a QEMU device

[Alberto Garcia]

On Fri, Nov 20, 2020 at 09:33:18PM +, Weiss, Howard wrote:
> I am writing a device which uses SHA256
  [...]
> I have tried
> 
>   #include 

You don't have to use openssl, QEMU already provides those algorithms,
see include/crypto/hash.h

Berto

Re: Purpose of stacking block devices

[Alberto Garcia]

On Wed, Oct 21, 2020 at 08:23:17AM +0200, Raphael Pour wrote:

> could someone explain me the purpose of stacking block devices on
> top of each other?
> 
> A block device with driver file or qcow2 "is usually" stacked on top of
> a block device with driver raw as stated in the docs[1].

One of them specifies the format, i.e. how the data is structured
internally (qcow2, qed, raw, vmdk, ...) and the other one specifies
the protocol used to access that data (file, http, nbd, ...).

Berto

Re: Large qcow2 performance

[Alberto Garcia]

On Tue, Sep 24, 2019 at 02:45:10PM -0500, Andrew Martin wrote:
> Hi Berto,
> 
> Thank you for the detailed explanation! It is also worth noting that
> I am planning on hosting this qcow2 on a ZFS dataset (and ZFS does
> some in-memory caching with the ARC as well); does that have an
> impact on performance?
> 
> Moreover, would setting the cluster_size to match the ZFS recordsize
> be advisable?
> https://jrs-s.net/2018/03/13/zvol-vs-qcow2-with-kvm/

I don't have much experience with ZFS so I'm not the best to provide
advice, but for the record I see that the ZFS recordsize is at most
128KB, whereas qcow2 supports clusters up to 2MB.

A 6TB qcow2 image can hold up to 384MB of L2 metadata if you use 128KB
clusters. 384MB would be the largest possible size of the L2 cache,
although in practice you'll probably be fine with much less, but how
much less depends on your usage pattern.

Doubling the cluster size halves the size of the L2 metadata (and
therefore the maximum needed cache size) but, as I said earlier, it
makes allocation slower and increases the disk usage.

Berto

Re: Large qcow2 performance

[Alberto Garcia]

On Mon, Sep 23, 2019 at 03:17:39PM -0500, Andrew Martin wrote:

> I am considering creating a VM with a large qcow2, around 6TB in
> size. This Linux VM will be reading and writing a lot of small
> files to this very large qcow2 using the VirtIO driver. Are there
> any concerns or guidance about creating such a large qcow2, or
> performance issues with small files on a 6TB qcow2?

Briefly:

there's two important parameters that can affect performance:
cluster_size and l2_cache_size.

cluster_size is set when you create the image, l2_cache_size is a
runtime option.

If the L2 cache size is not large enough, I/O performance will suffer.
Check here for a summary (the defaults have changed since then, but
the basic description of the problem is correct):

   
https://blogs.igalia.com/berto/2015/12/17/improving-disk-io-performance-in-qemu-2-5-with-the-qcow2-l2-cache/

With small clusters you need a larger L2 cache, that is: QEMU needs
more memory in order to get the same performance.

Images with large clusters use less memory, but there are two
drawbacks:

- The cluster is the minimum unit of allocation, so even if you write
  512 bytes QEMU needs to allocate one complete cluster and fill it
  with data. This makes *allocation* much slower the larger the
  cluster size.

  See the table here for an example, but focus only on the first and
  last columns and ignore the two in the middle:

 https://lists.gnu.org/archive/html/qemu-block/2019-07/msg00496.html

  After a cluster is allocated an image with large clusters should
  give you the same (or possibly better) performance.

- Because of what I just said, images with large clusters waste more
  space on disk.

A hybrid approach that combines the benefits of large clusters
regarding memory usage with the benefits of small clusters regarding
allocation performance is currently under development, and I plan to
send patches soon.

Berto

Re: [Qemu-discuss] Monitor mode with -display curses

[Alberto Garcia]

On Tue, Aug 21, 2018 at 09:17:39PM +0200, Vincenzo Romano wrote:
> But I would then get "just" the monitor. Not the console.
> I need both.
> Thanks anyway.

I'm not sure if I'm following you... if you have '-display curses'
you'll already have the guest console on the terminal emulator where
you're running QEMU.

If you want to have both the console and the monitor in the same place
you can try with -nographic. This way you have the guest serial port
and the monitor in the same place and you can switch between them
(press 'C-a h' for help). You need to pass console=ttyS0 to the guest
kernel.

Berto

Re: [Qemu-discuss] Monitor mode with -display curses

[Alberto Garcia]

On Mon, Aug 20, 2018 at 06:15:21PM +0200, Vincenzo Romano wrote:

> Is there a way to get to the monitor commands when using the option
> "-display curses"?

Yes, you can use a socket, e.g:

  $QEMU -monitor unix:path=/path/to/socket,server,nowait

and to connect to it:

  socat UNIX-CONNECT:/path/to/socket STDIO

You can connect to the QMP monitor the same way, just use -qmp instead
of -monitor.

See the documenation of "-serial" on the QEMU manpage for more
examples.

Berto

Re: [Qemu-discuss] qcow2 performanceimprove

[Alberto Garcia]

On Fri, Aug 17, 2018 at 10:28:49AM +0800, yang.bi...@zte.com.cn wrote:

> > > > If there is no backing file or snapshot you still need to fill
> > > > the cluster with zeroes, and that's going to be slower with
> > > > larger clusters.
> > > If not fill zeroes and only write guest data ,what`s wrong could
> > > happen ?
> > The following could happen:
> > 1) Guest reads at offset  [0,   4k] -> there's only zeroes
> > 2) Guest writes at offset [8k, 16k]
> > 3) Guest reads at offset  [0,   4k] -> there's something else now

> Why could guest read the area at offset [0, 4k] has not be writen
> yet ?

Jakob already gave you some answers, but here's a simple one: because
it might have already been written.

If the guest wrote zeroes to [0, 1M] you can't generally assume that
there's an allocated 1MB cluster on the qcow2 file filled with zeroes.

 - QEMU can detect that the guest tried to write zeroes and decide to
   leave the cluster unallocated (see for example the "detect_zeroes"
   option, or the "WRITE SAME" SCSI command).
 - The qcow2 file could have been converted at some point, and
   zero-filled clusters could have been deallocated for efficiency.

Berto

Re: [Qemu-discuss] qcow2 performance improve

[Alberto Garcia]

On Thu 16 Aug 2018 10:20:12 AM CEST, yang.bi...@zte.com.cn wrote:
>>If there is no backing file or snapshot you still need to fill the
>>cluster with zeroes, and that's going to be slower with larger
>>clusters.
>
> If not fill zeroes and only write guest data ,what`s wrong could
> happen ?

The following could happen:

 1) Guest reads at offset  [0,   4k] -> there's only zeroes
 2) Guest writes at offset [8k, 16k]
 3) Guest reads at offset  [0,   4k] -> there's something else now

Berto

Re: [Qemu-discuss] vga through qemu in arm

[Alberto Garcia]

On Mon, May 14, 2018 at 02:40:42PM +0800, lizhuoyao wrote:
> hi everyone:
>   currently, I meet a problem about qemu.Use virt-install order to create a 
> domain, return a failed:
>   order: virt-install -n centos-gg -r 1024 --disk 
> centos-gg.img,format=qcow2,size=10 --cdrom 
> /home/Centos-7-aarch64-Everything.iso --graphics vnc, listen=0.0.0.0
> failed: this QEMU does not support 'cirrus' video device

Can you try with '--video=vga' ?

Berto

Re: [Qemu-discuss] QEMU make error

[Alberto Garcia]

On Fri 27 Oct 2017 02:46:54 AM CEST, Akiv Jhirad  wrote:
> When running make in the qemu build, I get the error:
>
> /home/rtems/pc_rtems/qemu/replay/replay-internal.c: In function
> 'replay_put_array':
>
> /home/rtems/pc_rtems/qemu/replay/replay-internal.c:65:15: error:
>
> ignoring return value of 'fwrite', declared with attribute
> warn_unused_result [-Werror=unused-result]
>
>  fwrite(buf, 1, size, replay_file);
>
>^

The return value of fwrite() should be checked, I would expect something
like this:

 if (replay_file) {
 replay_put_dword(size);
-fwrite(buf, 1, size, replay_file);
+if (fwrite(buf, 1, size, replay_file) != size) {
+error_report("replay write error");
+}
 }

Pavel, what do you say?

Berto

Re: [Qemu-discuss] QEMU caching modes

[Alberto Garcia]

On Fri, Mar 17, 2017 at 09:52:27PM +0100, Jan Schermer wrote:

> I'm trying to understand when/what IO is synchronous and why, and so
> far what I'm seeing I can only describe as "weird".
> 
> With cache=none, all IO should pass from the QEMU process to the device 
> as-is. Right?

Here's the summary of what the 'cache' option does in practice:

cache=none ->   direct I/O (the host page cache is bypassed).
See O_DIRECT in open(2)

cache=directsync   ->   direct I/O + flush after each write

cache=writethrough ->   flush after each write

cache=unsafe   ->   guests flushes are ignored

cache=writeback->   no special options (this is the default)

> If so, that's not what I see - I see (with btrace) write IO as
> asynchronous (W) in the guest, but synchronous (WS) on the host.

Looks like what you're seeing is the direct I/O.

Berto

Re: [Qemu-discuss] [Qemu-devel] Estimation of qcow2 image size converted from raw image

[Alberto Garcia]

On Wed 15 Feb 2017 04:57:12 PM CET, Nir Soffer wrote:
>>> Let's try this syntax:
>>>
>>>   $ qemu-img query-max-size -f raw -O qcow2 input.raw
>>>   1234678000
>>>
>>> As John explained, it is only an estimate.  But it will be a
>>> conservative maximum.
>>
>> This forces you to have an input file. It would be nice to be able to
>> get the same information by merely giving the desired capacity e.g
>>
>>   $ qemu-img query-max-size -O qcow2 20G
>
> Without a file, this will have to assume that all clusters will be
> allocated.

...and that there are no internal snapshots. I'm not sure if this is
very useful in general.

Berto

Re: [Qemu-discuss] VGA choices

[Alberto Garcia]

On Wed, Feb 01, 2017 at 01:25:17PM +0100, Vincenzo Romano wrote:

> I've tried to start a previously created VM (running Ubuntu Server
> 16.04) with just " -nographic -curses" as far as the console is
> concerned.
> So, no "-vga" and no "-vnc" option. And no -daemonize of course.
> The PTY shows immediately a black screen with a blue "800 x 600
> Graphic mode" in the center... and then nothing else, despite I have
> asked for a "splash-time=5".

Let's see:

   -display XXX
  QEMU displays the video output of the VM using the method
  that you choose here. By default it's usually 'gtk' or 'sdl'
  (depending on how QEMU was compiled), meaning that the video
  output of the VM will appear inside a GTK or SDL window.
  That's how you normally run QEMU as a desktop user.

   -curses, or -display curses
  QEMU displays the graphical output of the VM, but only when the
  VM is running in text mode.
  Otherwise it displays the "800 x 600 Graphic Mode" message.

   -vnc, or -display vnc
  QEMU works as a VNC server, so you can see the video output of
  the VM using a VNC client.

   -nographic, or -display none
  QEMU doesn't display the video output of the VM at all.

   -vga XXX
  The emulated graphics card that the VM can see. 
  That's independent from what QEMU displays, and therefore from
  the previous options.
  If you don't use -vga then QEMU will select the default one
  (-vga std).

> I think the BIOS itself is preventing the curses mode to work. How
> can I "fix"this?

What's happening is that your Ubuntu VM is not using text mode.

Berto

Re: [Qemu-discuss] WinXP Guest: New hardware found - VGA controller

[Alberto Garcia]

On Thu, Jan 26, 2017 at 12:18:53AM +0100, Joe wrote:
> Anyway the issue was related to a misunderstanding of qemu man page:
> 
>  -vga type
>Select type of VGA card to emulate. Valid values for type are
> 
>cirrus
>Cirrus Logic GD5446 Video card. All Windows versions
> starting from Windows 95 should recognize
>and use this graphic card. For optimal performances,
> use 16 bit color depth in the guest and
>the host OS.  (This one is the default)
> ---

Yeah, cirrus used to be the default, but not anymore. The docs need to
be fixed, I'll prepare a patch.

> I noticed "-vga std" seems better than "cirrus", due to a higher
> resolution: I'm trying to use autocad under XP virtualized by qemu
> and high resolution would be appreciated.

Yeah, I suppose 'std' should be the best option for XP.

Berto

Re: [Qemu-discuss] Multiple snapshots with one guest user

[Alberto Garcia]

On Wed, Jan 25, 2017 at 04:16:22PM +0100, Gianni Costanzi wrote:

> > > what's the purpose of specifying the size when you're creating
> > > an image with a backing file? Is it simply ignored?
> >
> > The (virtual) sizes of the new image and the backing image can be
> > different. If you don't specify anything then the new image will
> > have the same size.

> I still don't understand what it means. If I have a base image of
> 10 GB with and it is almost full of data (it has a 10GB partition
> almost full, for example) and then I create a new image with 2 GB of
> size based on the first one.

Then you get a 2GB image. When you create an image with qemu-img you
get the (virtual) size that you asked for.

When QEMU reads from the virtual disk and a particular chunk of data
is not in the new image then it reads it from the backing image.

The practical effect is that you have a new image with the first 2GB
of the backing image.

> If I boot my os from the new Image I still see 10 GB partition,
> right?

Well, the OS will probably say that you have a 2GB disk with a 10GB
partition, because that's what's written in the partition table, but
as soon as you try to use it you'll get an "attempt to access beyond
end of device".

Berto

Re: [Qemu-discuss] Multiple snapshots with one guest user

[Alberto Garcia]

On Wed, Jan 25, 2017 at 11:23:01AM +0100, Gianni Costanzi wrote:

> what's the purpose of specifying the size when you're creating an
> image with a backing file? Is it simply ignored?

The (virtual) sizes of the new image and the backing image can be
different. If you don't specify anything then the new image will have
the same size.

Berto

Re: [Qemu-discuss] Info about compressed images

[Alberto Garcia]

On Fri, Jan 13, 2017 at 06:57:53PM +0100, Gianni Costanzi wrote:
> Hi,
> I'm new here and I've started using QEMU when I've started playing
> with Unetlab/EVE Network simulator.
> 
> I have some questions about compression and images with a compressed
> backing file:

I think there's some misunderstanding here about compressed images in
QEMU. I'll try to clarify:

* You create a compressed image with 'qemu-img convert -c'. That is a
  copy of the original image with all the clusters compressed.

* The compression is read-only: QEMU will read the compressed clusters,
  but everything that it writes will be uncompressed (also if you
  rewrite compressed clusters).
  
* Therefore, there's no such thing as an image with compression
  enabled. In QEMU you don't compress an image, you compress
  individual clusters of data. An image can have a mix of compressed
  and uncompressed clusters.

Berto

Re: [Qemu-discuss] WinXP Guest: New hardware found - VGA controller

[Alberto Garcia]

On Mon, Jan 23, 2017 at 01:00:26AM +0100, Joe wrote:

> I'm trying Qemu (2.8.0) on Slackware GNU Linux 14.2 (64-bit) host.
> I've a Windows XP (guest) raw image which was working with Qemu
> times ago (I used it on an older Slackware version as host system).

The default hardware emulated by QEMU changes in each new version.

If it worked with an earlier version and now you find that the guest
is detecting new hardware, try forcing the emulation of a machine from
an older version of QEMU.

For example:

qemu-system-i386 -M pc-i440fx-1.6 -m 2G ...

Type 'qemu-system-i386 -M ?' for a complete list.

Berto

Re: [Qemu-discuss] Throttle in virtio-net

[Alberto Garcia]

On Mon 05 Sep 2016 10:52:40 AM CEST, Pradeep Kiruvale wrote:

>> > I am planning to implement throttling functionality for virtio-net
>> > driver using the throttling APIs that exist inside qemu.
>>
>> the problem with implementing throttling for the network is that
>> it's useless if you use the vhost_net kernel accelerator, because it
>> bypasses QEMU entirely:
>>
>>https://access.redhat.com/documentation/en-US/Red_Hat_
>> Enterprise_Linux/7/html/Virtualization_Tuning_and_Optimization_Guide/sect-
>> Virtualization_Tuning_Optimization_Guide-Networking-
>> Virtio_and_vhostnet.html
>
> Thanks for this very valuable information. What if someone wants to
> just use virtio-net without the vhost acceleration?

I think you can use tc just the same, as far as I'm aware that's what
libvirt does.

Berto

Re: [Qemu-discuss] Throttle in virtio-net

[Alberto Garcia]

On Fri, Sep 02, 2016 at 05:13:40PM +0200, Pradeep Kiruvale wrote:

> I am planning to implement throttling functionality for virtio-net
> driver using the throttling APIs that exist inside qemu.

Hi Pradeep,

the problem with implementing throttling for the network is that
it's useless if you use the vhost_net kernel accelerator, because it
bypasses QEMU entirely:

   
https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/Virtualization_Tuning_and_Optimization_Guide/sect-Virtualization_Tuning_Optimization_Guide-Networking-Virtio_and_vhostnet.html

libvirt implements this using tc:

   https://libvirt.org/formatdomain.html#elementDomain
   http://luxik.cdi.cz/~devik/qos/htb/manual/userg.htm#ceiling

Berto

Re: [Qemu-discuss] SSD tuning

[Alberto Garcia]

On Mon, Aug 08, 2016 at 04:12:06PM +, Tim Bell wrote:

> Any suggestions for how to tune KVM for SSDs on the hypervisor ?
> We’re seeing a significant drop (50%) of IOPS for random write
> from guest to host.

If you're using qcow2, you are doing random I/O and you have a
sufficiently big virtual disk, chances are you need to increase the
cache size:

Take a look at this:

https://blogs.igalia.com/berto/2015/12/17/improving-disk-io-performance-in-qemu-2-5-with-the-qcow2-l2-cache/

Berto

Re: [Qemu-discuss] [Qemu-devel] iolimits for virtio-9p

[Alberto Garcia]

On Fri 06 May 2016 09:39:13 AM CEST, Pradeep Kiruvale wrote:

>> The throttling API is currently only used by block devices, and the
>> only documentation out-there is the code itself...
>
> Thanks, I will have a look and get back to you if I have any further
> questions regarding this.

As I said in an earlier e-mail:

1. the common API is in throttle.h.

2. Once you configure the throttling settings you essentially only need
   to call throttle_schedule_timer() to see if a request needs to be
   throttled, and afterwards throttle_account() to register the I/O that
   has been peformed.

Berto

Re: [Qemu-discuss] iolimits for virtio-9p

[Alberto Garcia]

On Wed, Apr 27, 2016 at 09:29:02AM +0200, Pradeep Kiruvale wrote:

> Thanks for the reply. I am still in the early phase, I will let you
> know if any changes are needed for the APIs.
> 
> We might also have to implement throttle-group.c for 9p devices, if
> we want to apply throttle for group of devices.

Fair enough, but again please note that:

- throttle-group.c is not meant to be generic, but it's tied to
  BlockDriverState / BlockBackend.
- it is currently being rewritten:
  https://lists.gnu.org/archive/html/qemu-block/2016-04/msg00645.html

If you can explain your use case with a bit more detail we can try to
see what can be done about it.

Regards,

Berto

Re: [Qemu-discuss] iolimits for virtio-9p

[Alberto Garcia]

On Tue 19 Apr 2016 02:09:24 PM CEST, Pradeep Kiruvale wrote:

> We are planning to implement the io-limits for the virtio-9p driver
> i.e for fsdev devices.
> So, I am looking into the code base and how it has done for the block
> io devices.
>
> I would like to know how difficult is this and is there some one out
> there who has any plan to do this?

Hi,

as Stefan said already, the common API is in throttle.h.

It should be generic enough to be used in other parts of QEMU, but tell
me if you feel that it needs changes.

Once you configure the throttling settings you essentially only need to
call throttle_schedule_timer() to see if a request needs to be
throttled, and afterwards throttle_account() to register the I/O that
has been peformed.

You'll see that there's also throttle-group.c, but that's specific to
the block layer and not meant to be generic.

Berto

Re: [Qemu-discuss] Cannot start qemu because of gtk version issue

[Alberto Garcia]

On Tue, Mar 22, 2016 at 04:05:37PM +0800, Cao jin wrote:
> Hi,
> 
> I configured as following:
> ./configure --target-list=x86_64-softmmu --enable-debug
> 
> when run qemu, got following error:
> (qemu-system-x86_64:3561): Gtk-ERROR **: GTK+ 2.x symbols detected. Using
> GTK+ 2.x and GTK+ 3 in the same process is not supported

You're probably linking QEMU with GTK+ 2.x but one of the other
libraries that QEMU depends on uses GTK+ 3.x.

I suggest that you switch completely to GTK+ 3, by passing
--with-gtkabi=3.0 to the configure script.

Berto

Re: [Qemu-discuss] about block_stream

[Alberto Garcia]

On Mon, Mar 30, 2015 at 11:26:03AM +0800, yang.bi...@zte.com.cn wrote:

 Can block_stream merge any node to its direct leaf Or any other 
 alternative? Thanks!

I'm currently working on a patch that would allow block_stream to
write to any intermediate layer:

http://wiki.qemu.org/Features/Snapshots#Streaming_to_an_Intermediate_Layer_.5Bproposal.5D

Berto