Re: [Qemu-devel] [PATCH 04/14] qemu-nbd: Simplify --partition handling

2018-11-30 Thread Richard W.M. Jones
) > } > fd_size -= dev_offset; > > -if (partition != -1) { > + if (partition) { > ret = find_partition(blk, partition, &dev_offset, &fd_size); > if (ret < 0) { > error_report("Could not find partition %d: %s"

Re: [Qemu-devel] [PATCH 03/14] qemu-nbd: Fail earlier for -c/-d on non-linux

2018-11-30 Thread Richard W.M. Jones
device) { > +#if HAVE_NBD_DEVICE > int ret; > > ret = pthread_create(&client_thread, NULL, nbd_client_thread, > device); > @@ -1013,6 +1027,10 @@ int main(int argc, char **argv) > error_report("Failed to create client thread: %s", >

Re: [Qemu-devel] [PATCH 02/14] nbd/client: More consistent error messages

2018-11-30 Thread Richard W.M. Jones
ected %u (%s)", > + reply.type, nbd_rep_lookup(reply.type), > + NBD_REP_ACK, nbd_rep_lookup(NBD_REP_ACK)); > nbd_send_opt_abort(ioc); > return -1; > } The NBD protocol doc seems to use integers pretty consistently (and c

Re: [Qemu-devel] [PATCH 01/14] qemu-nbd: Use program name in error messages

2018-11-30 Thread Richard W.M. Jones
ption negotiation failed: Verify failed: No certificate was found. > +qemu-nbd: option negotiation failed: Verify failed: No certificate was found. > qemu-img: Could not open > 'driver=nbd,host=127.0.0.1,port=PORT,tls-creds=tls0': The certificate hasn't > got a known is

[Qemu-devel] [PATCH] scsi-disk: Fix crash if underlying host file or disk returns error.

2018-11-21 Thread Richard W.M. Jones
ailed. Kevin Wolf suggested this fix: < kwolf> Hm, should the final return false; in that patch actually be a return true? < kwolf> Because I think he didn't intend to change anything except BLOCK_ERROR_ACTION_IGNORE Signed-off-by: Richard W.M. Jones Bugli

Re: [Qemu-devel] [PATCH 2/2] scsi-disk: fix rerror/werror=ignore

2018-11-18 Thread Richard W.M. Jones
On Sat, Oct 13, 2018 at 11:53:23AM +0200, Paolo Bonzini wrote: > rerror=ignore was returning true from scsi_handle_rw_error but the callers > were not > calling scsi_req_complete when rerror=ignore returns true (this is the > correct thing > to do when true is returned after executing a passthrou

Re: [Qemu-devel] [Qemu-block] Change in qemu 2.12 causes qemu-img convert to NBD to write more data

2018-11-17 Thread Richard W.M. Jones
, Nov 7, 2018 at 7:55 PM Nir Soffer wrote: > > > > > > > >> On Wed, Nov 7, 2018 at 7:27 PM Kevin Wolf wrote: > > > >> > > > >>> Am 07.11.2018 um 15:56 hat Nir Soffer geschrieben: > > > >>> > Wed, Nov 7, 2018 at 4:36 PM Ri

Re: [Qemu-devel] [PATCH for-3.2 00/41] RFC: slirp: make it again a standalone project

2018-11-14 Thread Richard W.M. Jones
On Wed, Nov 14, 2018 at 03:46:39PM +0100, Markus Armbruster wrote: > The issue I have with SLIRP isn't that it solves a useless problem (au > contraire!), it's that it's a useless solution. Okay, that's an unfair > exaggeration, it's not useless, I just wouldn't trust it in production, > unless it

Re: [Qemu-devel] [PATCH for-3.2 00/41] RFC: slirp: make it again a standalone project

2018-11-14 Thread Richard W.M. Jones
On Wed, Nov 14, 2018 at 01:59:25PM +0100, Markus Armbruster wrote: > Marc-André Lureau writes: > > > Hi, > > > > Based-on: https://people.debian.org/~sthibault/qemu.git/ slirp branch > > > > This series goal is to allow building libslirp as an independent library. > > > > While looking at making

[Qemu-devel] ANNOUNCE: nbdkit 1.8 - an NBD server toolkit with stable plugin API and permissive license

2018-11-12 Thread Richard W.M. Jones
I’m pleased to announce the next stable release of nbdkit. NBD — Network Block Device — is a protocol for accessing Block Devices (hard disks and disk-like things) over a Network. nbdkit is a toolkit for creating NBD servers. The key features are: * Multithreaded NBD server written in C with g

Re: [Qemu-devel] Change in qemu 2.12 causes qemu-img convert to NBD to write more data

2018-11-07 Thread Richard W.M. Jones
On Wed, Nov 07, 2018 at 04:56:48PM +0200, Nir Soffer wrote: > Wed, Nov 7, 2018 at 4:36 PM Richard W.M. Jones wrote: > > > Another thing I tried was to change the NBD server (nbdkit) so that it > > doesn't advertise zero support to the client: > > > > $

Re: [Qemu-devel] Change in qemu 2.12 causes qemu-img convert to NBD to write more data

2018-11-07 Thread Richard W.M. Jones
Another thing I tried was to change the NBD server (nbdkit) so that it doesn't advertise zero support to the client: $ nbdkit --filter=log --filter=nozero memory size=6G logfile=/tmp/log \ --run './qemu-img convert ./fedora-28.img -n $nbd' $ grep '\.\.\.$' /tmp/log | sed 's/.*\([A-Z][a-z

[Qemu-devel] Change in qemu 2.12 causes qemu-img convert to NBD to write more data

2018-11-07 Thread Richard W.M. Jones
(I'm not going to claim this is a bug, but it causes a large, easily measurable performance regression in virt-v2v). In qemu 2.10, when you do ‘qemu-img convert’ to an NBD target, qemu interleaves write and zero requests. We can observe this as follows: $ virt-builder fedora-28 $ nbdkit --fi

Re: [Qemu-devel] [Libguestfs] How to emulate block I/O timeout on qemu side?

2018-11-06 Thread Richard W.M. Jones
On Tue, Nov 06, 2018 at 09:43:06AM +, Richard W.M. Jones wrote: > On Tue, Nov 06, 2018 at 09:14:57AM +0000, Richard W.M. Jones wrote: > > This link shows how to combine delay and error filters together: > > > > https://rwmj.wordpress.com/2018/11/04/nbd-graphical-viewer/

Re: [Qemu-devel] [Libguestfs] How to emulate block I/O timeout on qemu side?

2018-11-06 Thread Richard W.M. Jones
On Tue, Nov 06, 2018 at 09:14:57AM +, Richard W.M. Jones wrote: > This link shows how to combine delay and error filters together: > > https://rwmj.wordpress.com/2018/11/04/nbd-graphical-viewer/ Oops, that's in a forthcoming blog post not this one. Not enough caffein

Re: [Qemu-devel] [Libguestfs] How to emulate block I/O timeout on qemu side?

2018-11-06 Thread Richard W.M. Jones
On Tue, Nov 06, 2018 at 02:17:46PM +0800, Dongli Zhang wrote: > On 11/06/2018 01:49 AM, Eric Blake wrote: > > On 11/2/18 3:11 AM, Dongli Zhang wrote: > >> Hi, > >> > >> Is there any way to emulate I/O timeout on qemu side (not fault > >> injection in VM kernel) without modifying qemu source code? >

Re: [Qemu-devel] [PATCH v5 0/5] Connect a PCIe host and graphics support to RISC-V

2018-10-11 Thread Richard W.M. Jones
On Thu, Oct 11, 2018 at 09:01:14AM +0100, Richard W.M. Jones wrote: > Here are the settings we currently do NOT have in my RV kernel: > > CONFIG_HOTPLUG_PCI_PCIE=y > CONFIG_HOTPLUG_PCI=y > CONFIG_MEDIA_PCI_SUPPORT=y > CONFIG_PCI_ATS=y > CONFIG_PCI_DEBUG

Re: [Qemu-devel] [PATCH v5 0/5] Connect a PCIe host and graphics support to RISC-V

2018-10-11 Thread Richard W.M. Jones
On Thu, Oct 11, 2018 at 07:59:59AM +0200, Andrea Bolognani wrote: > On Wed, 2018-10-10 at 10:57 -0700, Alistair wrote: > > I use this monolithic config: > > https://github.com/alistair23/meta-riscv/blob/7a950aa705b439b5ec19bb6f094930888335ba7b/recipes-kernel/linux/files/freedom-u540/defconfig > >

Re: [Qemu-devel] [PATCH v5 0/5] Connect a PCIe host and graphics support to RISC-V

2018-10-11 Thread Richard W.M. Jones
On Thu, Oct 11, 2018 at 07:59:59AM +0200, Andrea Bolognani wrote: > On Wed, 2018-10-10 at 10:57 -0700, Alistair wrote: > > On 10/10/2018 05:26 AM, Andrea Bolognani wrote: > > > * what should libvirt look for to figure out whether or not a RISC-V > > >guest will have PCI support? For aarch64 we

Re: [Qemu-devel] [PATCH 0/2] nbd server: drop old-style negotiation

2018-10-04 Thread Richard W.M. Jones
On Thu, Oct 04, 2018 at 12:10:17PM +, Vladimir Sementsov-Ogievskiy wrote: > this is a bit more difficult and more overhead than option in Qemu.. But > I don't sure we should care about I think Eric's point is that dropping oldstyle in qemu-nbd isn't really a problem because nbdkit has no plan

Re: [Qemu-devel] [PATCH 0/2] nbd server: drop old-style negotiation

2018-10-03 Thread Richard W.M. Jones
FWIW I don't have anything to add - agree with what's been said already. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-p2v converts physical machines to virtual machines. Boot

[Qemu-devel] [PATCH] curl: Make sslverify=off disable host as well as peer verification.

2018-09-14 Thread Richard W.M. Jones
s target host name '8.8.8.8' Could not open backing image to determine size. Further information about the two settings is available here: https://curl.haxx.se/libcurl/c/CURLOPT_SSL_VERIFYPEER.html https://curl.haxx.se/libcurl/c/CURLOPT_SSL_VERIFYHOST.html Signed-off-by: Richard W.

Re: [Qemu-devel] nbd oldstyle negotiation

2018-08-28 Thread Richard W.M. Jones
Just following up on this old thread for documentation purposes: Was it finally decided to drop oldstyle from qemu client in 3.1? Was anything posted to make that happen? I don't see any patches on the list. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjone

[Qemu-devel] ANNOUNCE: nbdkit 1.6 - an NBD server toolkit with stable plugin API and permissive license

2018-08-28 Thread Richard W.M. Jones
NBD — Network Block Device — is a protocol for accessing Block Devices (hard disks and disk-like things) over a Network. nbdkit is a toolkit for creating NBD servers. The key features are: * Multithreaded NBD server written in C with good performance. * Minimal dependencies for the basic serve

Re: [Qemu-devel] [PATCH] i386: Disable TOPOEXT by default on "-cpu host"

2018-08-10 Thread Richard W.M. Jones
.no_autoenable_flags = CPUID_EXT3_TOPOEXT, > }, > [FEAT_C000_0001_EDX] = { > .feat_names = { Can confirm that this fixes the problem observed on the original AMD Phenom machine, using qemu from git (@6ad908053) + your patch. Therefore: Tested-by: Richard W.M. Jones Tha

Re: [Qemu-devel] [PULL 09/12] i386: Allow TOPOEXT to be enabled on older kernels

2018-08-09 Thread Richard W.M. Jones
On Mon, Jun 25, 2018 at 07:25:21PM -0300, Eduardo Habkost wrote: > From: Babu Moger > > Enabling TOPOEXT feature might cause compatibility issues if > older kernels does not set this feature. Lets set this feature > unconditionally. > > Signed-off-by: Babu Moger > Message-Id: <1528939107-17193-

Re: [Qemu-devel] Byte ordering of VM Generation ID in Windows VMs

2018-07-09 Thread Richard W.M. Jones
On Mon, Jul 09, 2018 at 08:36:47AM +0100, Richard W.M. Jones wrote: > On Thu, Jul 05, 2018 at 03:11:32PM +0100, Richard W.M. Jones wrote: > > VMware represents these internally as two signed 64 bit integers, eg: > > > > vm.genid = "-570734802784577186" > >

Re: [Qemu-devel] Byte ordering of VM Generation ID in Windows VMs

2018-07-09 Thread Richard W.M. Jones
On Mon, Jul 09, 2018 at 11:05:34AM +0200, Laszlo Ersek wrote: > On 07/09/18 09:36, Richard W.M. Jones wrote: > > On Thu, Jul 05, 2018 at 03:11:32PM +0100, Richard W.M. Jones wrote: > >> VMware represents these internally as two signed 64 bit integers, eg: &g

Re: [Qemu-devel] Byte ordering of VM Generation ID in Windows VMs

2018-07-09 Thread Richard W.M. Jones
On Thu, Jul 05, 2018 at 03:11:32PM +0100, Richard W.M. Jones wrote: > VMware represents these internally as two signed 64 bit integers, eg: > > vm.genid = "-570734802784577186" > vm.genidx = "-5042519231342505152" > > I am still trying to get verifi

Re: [Qemu-devel] Byte ordering of VM Generation ID in Windows VMs

2018-07-05 Thread Richard W.M. Jones
On Thu, Jul 05, 2018 at 08:10:15PM +0300, Michael S. Tsirkin wrote: > On Thu, Jul 05, 2018 at 05:43:43PM +0100, Richard W.M. Jones wrote: > > On Thu, Jul 05, 2018 at 04:20:33PM +0200, Laszlo Ersek wrote: > > > QEMU does the right thing. If other hypervisors don't do th

Re: [Qemu-devel] Byte ordering of VM Generation ID in Windows VMs

2018-07-05 Thread Richard W.M. Jones
On Thu, Jul 05, 2018 at 04:20:33PM +0200, Laszlo Ersek wrote: > QEMU does the right thing. If other hypervisors don't do this -- while > still taking and displaying the value in UUID / GUID textual format --, > they are wrong. The VMGENID spec from Microsoft >

Re: [Qemu-devel] Byte ordering of VM Generation ID in Windows VMs

2018-07-05 Thread Richard W.M. Jones
On Thu, Jul 05, 2018 at 04:54:16PM +0300, Michael S. Tsirkin wrote: > On Thu, Jul 05, 2018 at 01:52:31PM +0100, Richard W.M. Jones wrote: > > On Thu, Jul 05, 2018 at 01:39:29PM +0100, Richard W.M. Jones wrote: > > > I was doing a bit of investigation around how different hyperv

Re: [Qemu-devel] Byte ordering of VM Generation ID in Windows VMs

2018-07-05 Thread Richard W.M. Jones
On Thu, Jul 05, 2018 at 01:39:29PM +0100, Richard W.M. Jones wrote: > I was doing a bit of investigation around how different hypervisors > handle the VM Generation ID feature. QEMU's behaviour seems quite > strange, I wonder if this is a bug or expected? > > (1) I booted a

[Qemu-devel] Byte ordering of VM Generation ID in Windows VMs

2018-07-05 Thread Richard W.M. Jones
I was doing a bit of investigation around how different hypervisors handle the VM Generation ID feature. QEMU's behaviour seems quite strange, I wonder if this is a bug or expected? (1) I booted a Windows 2016 VM with: qemu-system-x86_64 -M pc,accel=kvm -m 2G -hda w2k16-mincore.img \ -de

[Qemu-devel] [PATCH v6] crypto: Implement TLS Pre-Shared Keys (PSK).

2018-07-03 Thread Richard W.M. Jones
ys \ --tls-creds tls0 \ image.qcow2 Example NBD server using nbdkit: $ nbdkit -n -e / -fv \ --tls=on --tls-psk=/tmp/keys/keys.psk \ file file=disk.img Signed-off-by: Richard W.M. Jones --- crypto/Makefile.objs | 1 + crypto/tlscredsp

[Qemu-devel] [PATCH v6] crypto: Implement TLS Pre-Shared Keys (PSK).

2018-07-03 Thread Richard W.M. Jones
v5 was here: https://lists.gnu.org/archive/html/qemu-devel/2018-06/msg08491.html https://lists.gnu.org/archive/html/qemu-devel/2018-07/msg00077.html v6: - Make ECDHE-PSK dependent on GnuTLS >= 3. - Retested against nbdkit. - Retested with internal unit tests. Rich.

Re: [Qemu-devel] [PATCH v5] crypto: Implement TLS Pre-Shared Keys (PSK).

2018-07-03 Thread Richard W.M. Jones
Hi Dan, I was on holiday yesterday so I guess we've missed the deadline. In any case I will post v6 with the requested change in a few minutes. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordp

Re: [Qemu-devel] [PATCH v5] crypto: Implement TLS Pre-Shared Keys (PSK).

2018-06-29 Thread Richard W.M. Jones
On Fri, Jun 29, 2018 at 06:03:43PM +0100, Daniel P. Berrangé wrote: > On Thu, Jun 28, 2018 at 07:46:24PM +0100, Richard W.M. Jones wrote: > > diff --git a/crypto/tlssession.c b/crypto/tlssession.c > > index 96a02deb69..50df64e0a9 100644 > > --- a/crypto/tlssession.c > >

[Qemu-devel] [PATCH v5] crypto: Implement TLS Pre-Shared Keys (PSK).

2018-06-28 Thread Richard W.M. Jones
ys \ --tls-creds tls0 \ image.qcow2 Example NBD server using nbdkit: $ nbdkit -n -e / -fv \ --tls=on --tls-psk=/tmp/keys/keys.psk \ file file=disk.img Signed-off-by: Richard W.M. Jones --- crypto/Makefile.objs | 1 + crypto/tlscredsp

[Qemu-devel] [PATCH v5] crypto: Implement TLS Pre-Shared Keys (PSK).

2018-06-28 Thread Richard W.M. Jones
v4 was here: https://lists.nongnu.org/archive/html/qemu-devel/2018-06/msg08464.html v5: - Checks that username is not used when endpoint=server. (Note it is optional when endpoint=client, defaulting to "qemu") Rich.

[Qemu-devel] [PATCH v4] crypto: Implement TLS Pre-Shared Keys (PSK).

2018-06-28 Thread Richard W.M. Jones
ys \ --tls-creds tls0 \ image.qcow2 Example NBD server using nbdkit: $ nbdkit -n -e / -fv \ --tls=on --tls-psk=/tmp/keys/keys.psk \ file file=disk.img Signed-off-by: Richard W.M. Jones --- crypto/Makefile.objs | 1 + crypto/tlscredsp

[Qemu-devel] [PATCH v4] crypto: Implement TLS Pre-Shared Keys (PSK).

2018-06-28 Thread Richard W.M. Jones
v4: - This addresses points raised by Eric and Dan's reviews. The only thing I didn't change was using --object in qemu-options.hx, since that would really be different from everything else in the file. (However I did change -object -> --object in the header docs) This version uses g_file_get_c

Re: [Qemu-devel] [PATCH v3] crypto: Implement TLS Pre-Shared Keys (PSK).

2018-06-28 Thread Richard W.M. Jones
On Thu, Jun 28, 2018 at 05:06:20PM +0100, Daniel P. Berrangé wrote: > On Thu, Jun 28, 2018 at 09:42:18AM -0500, Eric Blake wrote: > > Does it make sense to forbid this operation on servers (since it only makes > > sense for clients)? > > We can't validate that here, because we can't guarantee that

Re: [Qemu-devel] [PATCH v3] crypto: Implement TLS Pre-Shared Keys (PSK).

2018-06-28 Thread Richard W.M. Jones
On Thu, Jun 28, 2018 at 09:42:18AM -0500, Eric Blake wrote: > On 06/28/2018 08:22 AM, Richard W.M. Jones wrote: > >+while (fgets(line, sizeof line, fp) != NULL) { > >+if (strncmp(line, username, ulen) == 0 && line[ulen] == ':') { > > ...can&#x

[Qemu-devel] [PATCH v3] crypto: Implement TLS Pre-Shared Keys (PSK).

2018-06-28 Thread Richard W.M. Jones
v2 was here: https://lists.nongnu.org/archive/html/qemu-devel/2018-06/msg07404.html v2 -> v3: - Offer ECDHE-PSK key exchange algorithm too. - Added a regression test to tests/test-crypto-tlssession.c Rich.

[Qemu-devel] [PATCH v3] crypto: Implement TLS Pre-Shared Keys (PSK).

2018-06-28 Thread Richard W.M. Jones
ys \ --tls-creds tls0 \ image.qcow2 Example NBD server using nbdkit: $ nbdkit -n -e / -fv \ --tls=on --tls-psk=/tmp/keys/keys.psk \ file file=disk.img Signed-off-by: Richard W.M. Jones --- crypto/Makefile.objs | 1 + crypto/tlscredsp

[Qemu-devel] [PATCH v2] crypto: Implement TLS Pre-Shared Keys (PSK).

2018-06-26 Thread Richard W.M. Jones
v1 was here: https://lists.nongnu.org/archive/html/qemu-devel/2018-06/threads.html#07252 For v2: - Added documentation as suggested by Dan B. - Fixed a backwards test of creds->username which slipped into the previous version by accident because I was fiddling around with the code aft

[Qemu-devel] [PATCH v2] crypto: Implement TLS Pre-Shared Keys (PSK).

2018-06-26 Thread Richard W.M. Jones
ys \ --tls-creds tls0 \ image.qcow2 Example NBD server using nbdkit: $ nbdkit -n -e / -fv \ --tls=on --tls-psk=/tmp/keys/keys.psk \ file file=disk.img Signed-off-by: Richard W.M. Jones --- crypto/Makefile.objs | 1 + crypto/tlscred

[Qemu-devel] [PATCH] crypto: Implement TLS Pre-Shared Keys (PSK).

2018-06-25 Thread Richard W.M. Jones
ys \ --tls-creds tls0 \ image.qcow2 Example NBD server using nbdkit: $ nbdkit -n -e / -fv \ --tls=on --tls-psk=/tmp/keys/keys.psk \ file file=disk.img Signed-off-by: Richard W.M. Jones --- crypto/Makefile.objs | 1 + crypto/tlscred

[Qemu-devel] [PATCH] crypto: Implement TLS Pre-Shared Keys (PSK).

2018-06-25 Thread Richard W.M. Jones
TLS-PSK (Pre-Shared Keys) lets us set up TLS connections much more easily, especially for NBD. This is a "version 0" of the patch for now, mainly to solicit comments. It needs documentation at least. Rich.

Re: [Qemu-devel] [PATCH v2 3/4] block: Remove deprecated -drive option serial

2018-06-14 Thread Richard W.M. Jones
On Thu, Jun 14, 2018 at 11:55:43AM +0200, Kevin Wolf wrote: > The -drive option serial was deprecated in QEMU 2.10. It's time to > remove it. > > Tests need to be updated to set the serial number with -global instead > of using the -drive option. Libguestfs uses this option to set the disk serial

Re: [Qemu-devel] storing machine data in qcow images?

2018-06-11 Thread Richard W.M. Jones
On Wed, Jun 06, 2018 at 01:32:47PM +0200, Max Reitz wrote: > ext2? I wrote an nbdkit plugin for ext2/ext3/ext4 last week. https://github.com/libguestfs/nbdkit/tree/master/plugins/ext2 It uses libext2fs from e2fsprogs and I think there are some lessons for anyone who wants to use ext2 to store

Re: [Qemu-devel] storing machine data in qcow images?

2018-06-07 Thread Richard W.M. Jones
On Thu, Jun 07, 2018 at 12:02:29PM +0200, Andrea Bolognani wrote: > Something that I haven't seen mentioned in the thread - and this > looks like as good a point as any to jump in - is that for q35 > guests using EFI as well as aarch64 guests the "one click import" > experience requires not only hi

Re: [Qemu-devel] storing machine data in qcow images?

2018-06-06 Thread Richard W.M. Jones
On Wed, Jun 06, 2018 at 12:48:17PM +0100, Daniel P. Berrangé wrote: > On Wed, Jun 06, 2018 at 12:42:28PM +0100, Richard W.M. Jones wrote: > > On Wed, Jun 06, 2018 at 12:14:07PM +0100, Dr. David Alan Gilbert wrote: > > > The problem with having a separate file is that you e

Re: [Qemu-devel] storing machine data in qcow images?

2018-06-06 Thread Richard W.M. Jones
On Wed, Jun 06, 2018 at 12:14:07PM +0100, Dr. David Alan Gilbert wrote: > The problem with having a separate file is that you either have to copy > it around with the image or have an archive. If you have an archive > you have to have an unpacking step which then copies, potentially a lot > of dat

Re: [Qemu-devel] storing machine data in qcow images?

2018-06-06 Thread Richard W.M. Jones
On Wed, Jun 06, 2018 at 01:02:53PM +0200, Max Reitz wrote: > (I'm wondering if we could write a block driver that could provide such > a chunk allocation transparently to qcow2... Note that a qcow2 file > does not need to be continuous, so you could in theory indeed store the > qcow2 file and its

Re: [Qemu-devel] [Qemu-block] storing machine data in qcow images?

2018-06-05 Thread Richard W.M. Jones
On Tue, Jun 05, 2018 at 02:54:07PM -0500, Eric Blake wrote: > On 06/05/2018 02:47 PM, Michael S. Tsirkin wrote: > > >>>Layer 1: > >>>The string shall always be a JSON 'object'; i.e. of the form > >>> { "something": ... , "more": ... } > >>> > >>>The key strings shall be non-null and no

Re: [Qemu-devel] storing machine data in qcow images?

2018-05-29 Thread Richard W.M. Jones
On Tue, May 29, 2018 at 11:14:11AM -0300, Eduardo Habkost wrote: > On Tue, May 29, 2018 at 03:03:16PM +0100, Dr. David Alan Gilbert wrote: > > * Richard W.M. Jones (rjo...@redhat.com) wrote: > > > On Fri, May 18, 2018 at 06:09:56PM +0100, Daniel P. Berrangé wrote: > > &g

Re: [Qemu-devel] storing machine data in qcow images?

2018-05-28 Thread Richard W.M. Jones
On Mon, May 28, 2018 at 08:38:33PM +0200, Kevin Wolf wrote: > Just accessing the image file within a tar archive is possible and we > could write a block driver for that (I actually think we should do > this), but it restricts you because certain operations like resizing > aren't really possible in

Re: [Qemu-devel] storing machine data in qcow images?

2018-05-28 Thread Richard W.M. Jones
On Mon, May 28, 2018 at 10:20:54PM +0100, Richard W.M. Jones wrote: > On Mon, May 28, 2018 at 08:38:33PM +0200, Kevin Wolf wrote: > > Just accessing the image file within a tar archive is possible and we > > could write a block driver for that (I actually think we should do &g

Re: [Qemu-devel] storing machine data in qcow images?

2018-05-28 Thread Richard W.M. Jones
On Mon, May 28, 2018 at 08:10:32PM +0200, Max Reitz wrote: > As someone who is just naive and doesn't see the big picture, I don't > see what's wrong with using a tar file that contains the image and > additional data. FWIW an OVA file is exactly this: an uncompressed tar file containing disk imag

Re: [Qemu-devel] storing machine data in qcow images?

2018-05-24 Thread Richard W.M. Jones
On Thu, May 24, 2018 at 05:08:17PM +0200, Kevin Wolf wrote: > Am 24.05.2018 um 16:56 hat Michael S. Tsirkin geschrieben: > > On Thu, May 24, 2018 at 12:32:51PM +0100, Richard W.M. Jones wrote: > > > There is however a seed of a good idea in the thread: > > > > >

Re: [Qemu-devel] storing machine data in qcow images?

2018-05-24 Thread Richard W.M. Jones
I read the whole thread and the fundamental problem is that you're mixing layers. Let qcow2 be a disk image format, and let management layers deal with metadata and how to run qemu. What's going to happen when you have (eg) an OVA file containing qcow2 files, and the qcow2 files all have differen

Re: [Qemu-devel] storing machine data in qcow images?

2018-05-24 Thread Richard W.M. Jones
On Fri, May 18, 2018 at 06:09:56PM +0100, Daniel P. Berrangé wrote: > The closest to a cross-hypervisor standard is OVF which can store > metadata about required hardware for a VM. I'm pretty sure it does > not have the concept of machine types, but maybe it has a way for > people to define metadat

Re: [Qemu-devel] [PATCH 1/3] nbd: Add option to disallow listing exports

2018-04-16 Thread Richard W.M. Jones
On Mon, Apr 16, 2018 at 11:31:18AM +0100, Daniel P. Berrangé wrote: > Essentially this is abusing the export name as a crude authentication > token. There are NBD servers that expect NBD_OPT_LIST to always succeeed I guess you mean "NBD clients" ... > when they detect that the new style protocol

Re: [Qemu-devel] [PATCH 1/3] nbd: Add option to disallow listing exports

2018-04-13 Thread Richard W.M. Jones
alhost 10809 > Negotiation: .. > > E: listing not allowed by server. > Server said: Listing exports is forbidden > > Signed-off-by: Nir Soffer Tested-by: Richard W.M. Jones The code looks good to me too, so ACK. Rich. > blockdev-nbd.c | 2 +- > include/block/

Re: [Qemu-devel] [PATCH for-2.12 0/2] RISC-V: Mark FP status dirty

2018-04-03 Thread Richard W.M. Jones
On Wed, Mar 28, 2018 at 10:22:31AM +0800, Richard Henderson wrote: > Since it was my patch that broke FP state tracking in the > first place, I feel obligated to fix it again. I missed this patch, thanks Michael Clark for pointing it out to me. I've just tried it now using my test reproducer of t

Re: [Qemu-devel] [PULL 0/1] RISC-V: Critical fixes for QEMU 2.12

2018-04-01 Thread Richard W.M. Jones
On Fri, Mar 30, 2018 at 10:08:23AM -0700, Michael Clark wrote: > Hi Peter, > > I had tested Richard's proper fix but we didn't have a PR or the required > Reviewed-by and Signed-off-by so I made the PR for the conservative fix, "Richard" is me or Richard Henderson? Anyway if you meant Stefan O'R

Re: [Qemu-devel] [PATCH v6 26/26] RISC-V: Workaround for critical mstatus.FS MTTCG bug

2018-03-24 Thread Richard W.M. Jones
ersus misreporting mstatus.FS > resulting in floating point register file corruption. > > Cc: Palmer Dabbelt > Cc: Sagar Karandikar > Cc: Bastian Koppelmann > Cc: Richard W.M. Jones > Cc: Peter Maydell > Signed-off-by: Michael Clark I tested this by running qemu from gi

Re: [Qemu-devel] [PATCH v3] RISC-V: Fix riscv_isa_string memory size bug

2018-03-22 Thread Richard W.M. Jones
On Thu, Mar 22, 2018 at 02:06:24PM -0700, Michael Clark wrote: > On Mon, Mar 19, 2018 at 2:39 PM, Michael Clark wrote: > > On Mon, Mar 19, 2018 at 12:42 PM, Richard W.M. Jones > > wrote: > >> Did you see the problem with restoring floating point registers on > >&g

Re: [Qemu-devel] [Qemu-block] [Libguestfs] [PATCH] tests: regressions: make test-big-heap use a temporary empty file

2018-03-21 Thread Richard W.M. Jones
On Wed, Mar 21, 2018 at 03:58:05PM -0500, Eric Blake wrote: > On 03/21/2018 03:44 PM, Kevin Wolf wrote: > >>> > >>>You're right that file locking on a character device like /dev/null is > >>>not going to work as expected, but is it a case where fcntl() actually > >>>fails, or is it worse where the

Re: [Qemu-devel] [PATCH v3] RISC-V: Fix riscv_isa_string memory size bug

2018-03-19 Thread Richard W.M. Jones
On Mon, Mar 19, 2018 at 11:35:51AM -0700, Michael Clark wrote: > The RISC-V post-merge spec conformance and cleanup series has had a lot of > testing. I've been using it to compile QEMU inside of QEMU using the RISC-V > Fedora Image and its native RISC-V GCC toolchain running inside SMP Linux > 4.1

Re: [Qemu-devel] [PATCH v8 00/23] RISC-V QEMU Port Submission

2018-03-09 Thread Richard W.M. Jones
On Fri, Mar 09, 2018 at 04:43:34PM +, Peter Maydell wrote: > On 5 March 2018 at 08:41, Richard W.M. Jones wrote: > > > > The attached patch is also needed to avoid crashes during various > > math-heavy test suites. > > Applied to master, thanks. > > FYI,

Re: [Qemu-devel] [PULL] RISC-V QEMU Port Submission v8.2

2018-03-07 Thread Richard W.M. Jones
On Wed, Mar 07, 2018 at 01:09:29PM +1300, Michael Clark wrote: > Hopefully, this PR gets merged... I hope so too. We've been testing v8 (substantially the same as v8.2) extensively, including SMP. It's building hundreds of packages a day in the autobuilder, and being used for manual builds by se

Re: [Qemu-devel] [PATCH v8 00/23] RISC-V QEMU Port Submission

2018-03-05 Thread Richard W.M. Jones
The attached patch is also needed to avoid crashes during various math-heavy test suites. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-p2v converts physical machines to virtual

Re: [Qemu-devel] [PATCH 1/2] block: curl: Allow arbitrary HTTP request headers to be set.

2018-03-01 Thread Richard W.M. Jones
On Thu, Mar 01, 2018 at 04:11:18PM +, Daniel P. Berrangé wrote: > On Thu, Mar 01, 2018 at 01:58:55PM +0000, Richard W.M. Jones wrote: > > Allow arbitrary HTTP request headers to be set, like this: > > > > qemu-img create -f qcow2 \ > > -b

Re: [Qemu-devel] [PATCH 2/2] block: curl: Allow Certificate Authority bundle to be passed in.

2018-03-01 Thread Richard W.M. Jones
On Thu, Mar 01, 2018 at 03:34:38PM +, Daniel P. Berrangé wrote: > On Thu, Mar 01, 2018 at 01:58:56PM +0000, Richard W.M. Jones wrote: > > This allows a Certificate Authority bundle to be passed to the curl > > driver, allowing authentication against servers that check > &g

Re: [Qemu-devel] [PATCH 1/2] block: curl: Allow arbitrary HTTP request headers to be set.

2018-03-01 Thread Richard W.M. Jones
On Thu, Mar 01, 2018 at 03:24:48PM +, Nir Soffer wrote: > Other issue that you need to consider is that you cannot > create images via the http. Images are created only via > engine API/UI. We've got a separate bit of Python creating the image so I think we're ok here. > What ovirt-imageio gi

Re: [Qemu-devel] [PATCH 0/2] block: curl: Proof of concept for connecting to oVirt.

2018-03-01 Thread Richard W.M. Jones
On Thu, Mar 01, 2018 at 06:21:15AM -0800, no-re...@patchew.org wrote: > /tmp/qemu-test/src/block/curl.c: In function 'curl_open': > /tmp/qemu-test/src/block/curl.c:770:15: error: assignment discards 'const' > qualifier from pointer target type [-Werror=discarded-qualifiers] > s->cainfo = qemu

[Qemu-devel] [PATCH 2/2] block: curl: Allow Certificate Authority bundle to be passed in.

2018-03-01 Thread Richard W.M. Jones
d using some extra code. See https://github.com/oVirt/ovirt-engine-sdk/blob/master/sdk/examples/upload_ova_as_vm.py for some guidance). Signed-off-by: Richard W.M. Jones --- block/curl.c | 14 ++ qapi/block-core.json | 3 +++ 2 files changed, 17 insertions(+) diff --git a/block/

[Qemu-devel] [PATCH 0/2] block: curl: Proof of concept for connecting to oVirt.

2018-03-01 Thread Richard W.M. Jones
This is a proof of concept patch which needs a bunch more testing. I'm just posting it to get feedback. It's almost possible to connect qemu to an oVirt / RHV server and upload or download or otherwise modify disk images. However the curl driver is missing a couple of features to satisfy oVirt's

[Qemu-devel] [PATCH 1/2] block: curl: Allow arbitrary HTTP request headers to be set.

2018-03-01 Thread Richard W.M. Jones
LOPT_HTTPHEADER, thus pulling in curl API guarantees into qemu, but curl has had very strong API backward compatibility since the start of the project. Signed-off-by: Richard W.M. Jones --- block/curl.c | 21 - qapi/block-core.json | 10 -- 2 files changed, 28 ins

Re: [Qemu-devel] [PATCH v6 00/23] RISC-V QEMU Port Submission

2018-02-26 Thread Richard W.M. Jones
If anyone wants a simple way to test this, grab the latest bbl & stage4 disk image from here and boot it under qemu-system-riscv64 using the command line given in the readme.txt file: https://fedorapeople.org/groups/risc-v/disk-images/ I've added v6 to Fedora copr, and switched to using it for

Re: [Qemu-devel] [PATCH v5 00/23] RISC-V QEMU Port Submission

2018-02-17 Thread Richard W.M. Jones
I just want to mention that we've been running this patch set in production for a few days, doing hundreds of Fedora RISC-V builds with ‘-smp 4’ and it has been rock solid. Therefore: Tested-by: Richard W.M. Jones Rich. -- Richard Jones, Virtualization Group, Red Hat

Re: [Qemu-devel] [PATCH 3/3] block/ssh: Add basic .bdrv_truncate()

2018-02-15 Thread Richard W.M. Jones
The series looks fine to me: Reviewed-by: Richard W.M. Jones Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-builder quickly builds VMs from scratch http://libguestfs.org/virt

[Qemu-devel] ANNOUNCE: libguestfs 1.38 released

2018-02-09 Thread Richard W.M. Jones
I'm pleased to announce libguestfs 1.38, a library and a set of tools for accessing and modifying virtual machine disk images. This release represents about a year of work by many contributors. I'd like to call out in particular substantial contributions from: Cédric Bosdonnat, Pavel Butsykin, Mat

Re: [Qemu-devel] [PATCH 0/2] block/ssh: Implement .bdrv_refresh_filename()

2018-02-05 Thread Richard W.M. Jones
On Mon, Feb 05, 2018 at 09:22:30PM +0100, Max Reitz wrote: > This series implements .bdrv_refresh_filename() for the ssh block > driver, along with an appropriate .bdrv_dirname() so we don't chop off > query strings for backing files with relative filenames. > > This series depends on my “block: F

Re: [Qemu-devel] [PATCH 0/2] block/ssh: Implement .bdrv_refresh_filename()

2018-02-05 Thread Richard W.M. Jones
I assume the context of all this is making relative backing file names work in qcow2 files? Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-builder quickly builds VMs from scratch

Re: [Qemu-devel] [PATCH v1 00/21] RISC-V QEMU Port Submission v1

2018-01-03 Thread Richard W.M. Jones
On Thu, Jan 04, 2018 at 10:50:06AM +1300, Michael Clark wrote: > On Thu, Jan 4, 2018 at 12:35 AM, Richard W.M. Jones > wrote: > > > Just a few small points: > > > > (1) I've built Fedora RPMs from this patch set [approximately - I'm > > using a very

Re: [Qemu-devel] [PATCH v1 00/21] RISC-V QEMU Port Submission v1

2018-01-03 Thread Richard W.M. Jones
Just a few small points: (1) I've built Fedora RPMs from this patch set [approximately - I'm using a very slightly different / slightly older version, but it's not substantively different]: http://copr-fe.cloud.fedoraproject.org/coprs/rjones/riscv/ It works well for me building plenty of Fedor

Re: [Qemu-devel] [Qemu-block] [PATCH v3] ssh: switch from libssh2 to libssh

2017-12-19 Thread Richard W.M. Jones
On Tue, Dec 19, 2017 at 10:24:18AM -0500, Jeff Cody wrote: > On Mon, Dec 18, 2017 at 09:31:40PM +0000, Richard W.M. Jones wrote: > > On Mon, Dec 18, 2017 at 01:43:39PM -0500, John Snow wrote: > > > Hi, friendly ping: > > > > > > It's been over a month with

Re: [Qemu-devel] [Qemu-block] [PATCH v3] ssh: switch from libssh2 to libssh

2017-12-18 Thread Richard W.M. Jones
On Mon, Dec 18, 2017 at 01:43:39PM -0500, John Snow wrote: > Hi, friendly ping: > > It's been over a month with no replies, so it's safe to say this has > gotten lost in the 2.11 release shuffle. > > Recommend you re-send it for the 2.12 window. Actually there is an interop problem with the ssh

Re: [Qemu-devel] About the light VM solution!

2017-12-07 Thread Richard W.M. Jones
I did a bit of work on this back in early 2016 and wrote a paper which analyzes what Intel were doing with Clear Containers back then, and how it fitted with the more distribution-centric view of Fedora and Red Hat, ie that we ideally want a single qemu binary, a single kernel, a single SeaBIOS, et

Re: [Qemu-devel] [PULL 3/8] nbd-client: Refuse read-only client with BDRV_O_RDWR

2017-12-03 Thread Richard W.M. Jones
On Thu, Nov 09, 2017 at 10:59:34AM -0600, Eric Blake wrote: > The NBD spec says that clients should not try to write/trim to > an export advertised as read-only by the server. But we failed > to check that, and would allow the block layer to use NBD with > BDRV_O_RDWR even when the server is read-

Re: [Qemu-devel] [PATCH 1/1] vhost-scsi: add missing virtqueue_size parameter

2017-12-02 Thread Richard W.M. Jones
On Fri, Dec 01, 2017 at 04:15:38PM +0100, Eric Farman wrote: > Commit 5c0919d02066 ("virtio-scsi: Add virtqueue_size parameter allowing > virtqueue size to be set.") introduced a new parameter to virtio-scsi. > Later, commit 920036106044 ("vhost-user-scsi: add missing virtqueue_size > param") added

Re: [Qemu-devel] Effect of qemu-img convert -m and -W options

2017-11-16 Thread Richard W.M. Jones
On Thu, Nov 16, 2017 at 05:30:48PM +, Stefan Hajnoczi wrote: > On Thu, Nov 16, 2017 at 3:10 PM, Richard W.M. Jones wrote: > > Both local filesystems, but on different SATA devices. > > Okay. I'm curious what the strace -f output looks like (only the > preadv(2)/

Re: [Qemu-devel] Effect of qemu-img convert -m and -W options

2017-11-16 Thread Richard W.M. Jones
On Thu, Nov 16, 2017 at 03:51:29PM +0100, Peter Lieven wrote: > Afaik all writes to the same QCOW2 serialize because of the s->lock that > is held during the write. So its not suprising that there is no benefit from > mutliple threads as long as reading from the RAW file involves no delay. > Which

Re: [Qemu-devel] Effect of qemu-img convert -m and -W options

2017-11-16 Thread Richard W.M. Jones
On Thu, Nov 16, 2017 at 02:47:46PM +, Stefan Hajnoczi wrote: > The threads you observed are the thread pool that performs > preadv(2)/pwritev(2) syscalls. The Linux AIO API could be used instead > and does not use threads for read and write operations. I guess if I used AIO then I wouldn't ge

Re: [Qemu-devel] [Libguestfs] [qemu-img] support for XVA

2017-11-16 Thread Richard W.M. Jones
I think it's even possible to modify that shell script to pipe into nbdkit and from there to ‘qemu-img convert’. I'm too lazy to actually do that right now, but the basic idea is here: https://rwmj.wordpress.com/2014/10/14/streaming-nbd-server/ You'll probably have to add ‘-m 1’ to the qemu-im

Re: [Qemu-devel] [Libguestfs] [qemu-img] support for XVA

2017-11-16 Thread Richard W.M. Jones
On Thu, Nov 16, 2017 at 11:07:10AM +0100, Gandalf Corvotempesta wrote: > 2017-11-16 11:01 GMT+01:00 Richard W.M. Jones : > > As mentioned before you can use this to do a qemu-img convert using > > captive nbdkit: > > > > $ nbdkit -U - \ > >

<    1   2   3   4   5   6   7   8   9   10   >