[PULL 00/26] Next patches

2023-02-02 Thread Juan Quintela
The following changes since commit deabea6e88f7c4c3c12a36ee30051c6209561165:

  Merge tag 'for_upstream' of https://git.kernel.org/pub/scm/virt/kvm/mst/qemu 
into staging (2023-02-02 10:10:07 +)

are available in the Git repository at:

  https://gitlab.com/juan.quintela/qemu.git tags/next-pull-request

for you to fetch changes up to 5ee6d3d1eeccd85aa2a835e82b8d9e1b4f7441e1:

  migration: check magic value for deciding the mapping of channels (2023-02-02 
17:04:16 +0100)


Migration PULL request, new try

Hi

It includes:
- David Hildenbrand fixes for virtio-men
- David Gilbert canary to detect problems
- Fix for rdma return values (Fiona)
- Peter Xu uffd_open fixes
- Peter Xu show right downtime for postcopy
- manish.mishra msg fix fixes
- my vfio changes.

Please apply.

Please, apply.



David Hildenbrand (13):
  migration/ram: Fix populate_read_range()
  migration/ram: Fix error handling in ram_write_tracking_start()
  migration/ram: Don't explicitly unprotect when unregistering uffd-wp
  migration/ram: Rely on used_length for uffd_change_protection()
  migration/ram: Optimize ram_write_tracking_start() for
RamDiscardManager
  migration/savevm: Move more savevm handling into vmstate_save()
  migration/savevm: Prepare vmdesc json writer in
qemu_savevm_state_setup()
  migration/savevm: Allow immutable device state to be migrated early
(i.e., before RAM)
  migration/vmstate: Introduce VMSTATE_WITH_TMP_TEST() and
VMSTATE_BITMAP_TEST()
  migration/ram: Factor out check for advised postcopy
  virtio-mem: Fail if a memory backend with "prealloc=on" is specified
  virtio-mem: Migrate immutable properties early
  virtio-mem: Proper support for preallocation with migration

Dr. David Alan Gilbert (2):
  migration: Add canary to VMSTATE_END_OF_LIST
  migration: Perform vmsd structure check during tests

Fiona Ebner (1):
  migration/rdma: fix return value for qio_channel_rdma_{readv,writev}

Juan Quintela (4):
  migration: No save_live_pending() method uses the QEMUFile parameter
  migration: Split save_live_pending() into state_pending_*
  migration: Remove unused threshold_size parameter
  migration: simplify migration_iteration_run()

Peter Xu (3):
  migration: Fix migration crash when target psize larger than host
  util/userfaultfd: Add uffd_open()
  migration: Show downtime during postcopy phase

Zhenzhong Duan (1):
  migration/dirtyrate: Show sample pages only in page-sampling mode

manish.mishra (2):
  io: Add support for MSG_PEEK for socket channel
  migration: check magic value for deciding the mapping of channels

 docs/devel/migration.rst|  18 +--
 docs/devel/vfio-migration.rst   |   4 +-
 include/hw/virtio/virtio-mem.h  |   8 ++
 include/io/channel.h|   6 +
 include/migration/misc.h|   4 +-
 include/migration/register.h|  17 +--
 include/migration/vmstate.h |  35 +-
 include/qemu/userfaultfd.h  |   8 ++
 migration/channel.h |   5 +
 migration/migration.h   |   4 +
 migration/multifd.h |   2 +-
 migration/postcopy-ram.h|   2 +-
 migration/savevm.h  |  10 +-
 chardev/char-socket.c   |   4 +-
 hw/core/machine.c   |   4 +-
 hw/s390x/s390-stattrib.c|  11 +-
 hw/vfio/migration.c |  20 +--
 hw/virtio/virtio-mem.c  | 144 -
 io/channel-buffer.c |   1 +
 io/channel-command.c|   1 +
 io/channel-file.c   |   1 +
 io/channel-null.c   |   1 +
 io/channel-socket.c |  19 ++-
 io/channel-tls.c|   1 +
 io/channel-websock.c|   1 +
 io/channel.c|  16 ++-
 migration/block-dirty-bitmap.c  |  14 +--
 migration/block.c   |  13 +-
 migration/channel-block.c   |   1 +
 migration/channel.c |  45 +++
 migration/dirtyrate.c   |  10 +-
 migration/migration.c   | 119 --
 migration/multifd.c |  19 +--
 migration/postcopy-ram.c|  16 +--
 migration/ram.c | 120 +-
 migration/rdma.c|  16 ++-
 migration/savevm.c  | 187 
 migration/vmstate.c |   2 +
 scsi/qemu-pr-helper.c   |   2 +-
 tests/qtest/migration-test.c|   3 +-
 tests/qtest/tpm-emu.c   |   2 +-
 tests/unit/test-io-channel-socket.c |   1 +
 util/userfaultfd.c  |  13 +-
 util/vhost-user-server.c|   2 +-
 hw/vfio/trace-events|   2 +-
 migration/trace-events  |   7 +-
 46 files changed, 715 insertions(+), 226 deletions(-)

-- 
2.39.1




Re: [PULL 00/26] Next patches

2023-02-04 Thread Peter Maydell
On Thu, 2 Feb 2023 at 16:07, Juan Quintela  wrote:
>
> The following changes since commit deabea6e88f7c4c3c12a36ee30051c6209561165:
>
>   Merge tag 'for_upstream' of 
> https://git.kernel.org/pub/scm/virt/kvm/mst/qemu into staging (2023-02-02 
> 10:10:07 +)
>
> are available in the Git repository at:
>
>   https://gitlab.com/juan.quintela/qemu.git tags/next-pull-request
>
> for you to fetch changes up to 5ee6d3d1eeccd85aa2a835e82b8d9e1b4f7441e1:
>
>   migration: check magic value for deciding the mapping of channels 
> (2023-02-02 17:04:16 +0100)
>
> 
> Migration PULL request, new try

Fails to build on anything that isn't Linux:

In file included from ../migration/postcopy-ram.c:40:
/private/var/folders/76/zy5ktkns50v6gt5g8r0sf6scgn/T/cirrus-ci-build/include/qemu/userfaultfd.h:18:10:
fatal error: 'linux/userfaultfd.h' file not found

thanks
-- PMM



Re: [PULL 00/26] Next patches

2023-02-06 Thread Peter Xu
On Sat, Feb 04, 2023 at 10:19:34AM +, Peter Maydell wrote:
> On Thu, 2 Feb 2023 at 16:07, Juan Quintela  wrote:
> >
> > The following changes since commit deabea6e88f7c4c3c12a36ee30051c6209561165:
> >
> >   Merge tag 'for_upstream' of 
> > https://git.kernel.org/pub/scm/virt/kvm/mst/qemu into staging (2023-02-02 
> > 10:10:07 +)
> >
> > are available in the Git repository at:
> >
> >   https://gitlab.com/juan.quintela/qemu.git tags/next-pull-request
> >
> > for you to fetch changes up to 5ee6d3d1eeccd85aa2a835e82b8d9e1b4f7441e1:
> >
> >   migration: check magic value for deciding the mapping of channels 
> > (2023-02-02 17:04:16 +0100)
> >
> > 
> > Migration PULL request, new try
> 
> Fails to build on anything that isn't Linux:
> 
> In file included from ../migration/postcopy-ram.c:40:
> /private/var/folders/76/zy5ktkns50v6gt5g8r0sf6scgn/T/cirrus-ci-build/include/qemu/userfaultfd.h:18:10:
> fatal error: 'linux/userfaultfd.h' file not found

Oops, my fault.

Juan, please feel free to drop patch "util/userfaultfd: Add uffd_open()".
I'll respin with the whole set.

-- 
Peter Xu




Re: [PULL 00/26] Next patches

2023-02-06 Thread Juan Quintela
On Mon, Feb 6, 2023, 23:07 Peter Xu  wrote:

> On Sat, Feb 04, 2023 at 10:19:34AM +, Peter Maydell wrote:
> > On Thu, 2 Feb 2023 at 16:07, Juan Quintela  wrote:
> > >
> > > The following changes since commit
> deabea6e88f7c4c3c12a36ee30051c6209561165:
> > >
> > >   Merge tag 'for_upstream' of
> https://git.kernel.org/pub/scm/virt/kvm/mst/qemu into staging (2023-02-02
> 10:10:07 +)
> > >
> > > are available in the Git repository at:
> > >
> > >   https://gitlab.com/juan.quintela/qemu.git tags/next-pull-request
> > >
> > > for you to fetch changes up to
> 5ee6d3d1eeccd85aa2a835e82b8d9e1b4f7441e1:
> > >
> > >   migration: check magic value for deciding the mapping of channels
> (2023-02-02 17:04:16 +0100)
> > >
> > > 
> > > Migration PULL request, new try
> >
> > Fails to build on anything that isn't Linux:
> >
> > In file included from ../migration/postcopy-ram.c:40:
> >
> /private/var/folders/76/zy5ktkns50v6gt5g8r0sf6scgn/T/cirrus-ci-build/include/qemu/userfaultfd.h:18:10:
> > fatal error: 'linux/userfaultfd.h' file not found
>
> Oops, my fault.
>
> Juan, please feel free to drop patch "util/userfaultfd: Add uffd_open()".
> I'll respin with the whole set.
>


Fixed it already

> --
> Peter Xu
>
>


Re: [PULL 00/26] Next patches

2023-02-06 Thread Juan Quintela
Juan Quintela  wrote:
> The following changes since commit deabea6e88f7c4c3c12a36ee30051c6209561165:
>
>   Merge tag 'for_upstream' of 
> https://git.kernel.org/pub/scm/virt/kvm/mst/qemu into staging (2023-02-02 
> 10:10:07 +)
>
> are available in the Git repository at:
>
>   https://gitlab.com/juan.quintela/qemu.git tags/next-pull-request
>
> for you to fetch changes up to 5ee6d3d1eeccd85aa2a835e82b8d9e1b4f7441e1:
>
>   migration: check magic value for deciding the mapping of channels 
> (2023-02-02 17:04:16 +0100)
>
> 
> Migration PULL request, new try

NACK

Has the same problem that peter detected.
Rebased it and fix file to only be included on linux.




>
> Hi
>
> It includes:
> - David Hildenbrand fixes for virtio-men
> - David Gilbert canary to detect problems
> - Fix for rdma return values (Fiona)
> - Peter Xu uffd_open fixes
> - Peter Xu show right downtime for postcopy
> - manish.mishra msg fix fixes
> - my vfio changes.
>
> Please apply.
>
> Please, apply.
>
> 
>
> David Hildenbrand (13):
>   migration/ram: Fix populate_read_range()
>   migration/ram: Fix error handling in ram_write_tracking_start()
>   migration/ram: Don't explicitly unprotect when unregistering uffd-wp
>   migration/ram: Rely on used_length for uffd_change_protection()
>   migration/ram: Optimize ram_write_tracking_start() for
> RamDiscardManager
>   migration/savevm: Move more savevm handling into vmstate_save()
>   migration/savevm: Prepare vmdesc json writer in
> qemu_savevm_state_setup()
>   migration/savevm: Allow immutable device state to be migrated early
> (i.e., before RAM)
>   migration/vmstate: Introduce VMSTATE_WITH_TMP_TEST() and
> VMSTATE_BITMAP_TEST()
>   migration/ram: Factor out check for advised postcopy
>   virtio-mem: Fail if a memory backend with "prealloc=on" is specified
>   virtio-mem: Migrate immutable properties early
>   virtio-mem: Proper support for preallocation with migration
>
> Dr. David Alan Gilbert (2):
>   migration: Add canary to VMSTATE_END_OF_LIST
>   migration: Perform vmsd structure check during tests
>
> Fiona Ebner (1):
>   migration/rdma: fix return value for qio_channel_rdma_{readv,writev}
>
> Juan Quintela (4):
>   migration: No save_live_pending() method uses the QEMUFile parameter
>   migration: Split save_live_pending() into state_pending_*
>   migration: Remove unused threshold_size parameter
>   migration: simplify migration_iteration_run()
>
> Peter Xu (3):
>   migration: Fix migration crash when target psize larger than host
>   util/userfaultfd: Add uffd_open()
>   migration: Show downtime during postcopy phase
>
> Zhenzhong Duan (1):
>   migration/dirtyrate: Show sample pages only in page-sampling mode
>
> manish.mishra (2):
>   io: Add support for MSG_PEEK for socket channel
>   migration: check magic value for deciding the mapping of channels
>
>  docs/devel/migration.rst|  18 +--
>  docs/devel/vfio-migration.rst   |   4 +-
>  include/hw/virtio/virtio-mem.h  |   8 ++
>  include/io/channel.h|   6 +
>  include/migration/misc.h|   4 +-
>  include/migration/register.h|  17 +--
>  include/migration/vmstate.h |  35 +-
>  include/qemu/userfaultfd.h  |   8 ++
>  migration/channel.h |   5 +
>  migration/migration.h   |   4 +
>  migration/multifd.h |   2 +-
>  migration/postcopy-ram.h|   2 +-
>  migration/savevm.h  |  10 +-
>  chardev/char-socket.c   |   4 +-
>  hw/core/machine.c   |   4 +-
>  hw/s390x/s390-stattrib.c|  11 +-
>  hw/vfio/migration.c |  20 +--
>  hw/virtio/virtio-mem.c  | 144 -
>  io/channel-buffer.c |   1 +
>  io/channel-command.c|   1 +
>  io/channel-file.c   |   1 +
>  io/channel-null.c   |   1 +
>  io/channel-socket.c |  19 ++-
>  io/channel-tls.c|   1 +
>  io/channel-websock.c|   1 +
>  io/channel.c|  16 ++-
>  migration/block-dirty-bitmap.c  |  14 +--
>  migration/block.c   |  13 +-
>  migration/channel-block.c   |   1 +
>  migration/channel.c |  45 +++
>  migration/dirtyrate.c   |  10 +-
>  migration/migration.c   | 119 --
>  migration/multifd.c |  19 +--
>  migration/postcopy-ram.c|  16 +--
>  migration/ram.c | 120 +-
>  migration/rdma.c|  16 ++-
>  migration/savevm.c  | 187 
>  migration/vmstate.c |   2 +
>  scsi/qemu-pr-helper.c   |   2 +-
>  tests/qtest/migration-test.c|   3 +-
>  tests/qtest/