[pve-devel] [PATCH v3 qemu 1/2] fix #5409: backup: fix copy-before-write timeout

2024-04-29 Thread Fiona Ebner
The type for the copy-before-write timeout in nanoseconds was wrong. By being just uint32_t, a maximum of slightly over 4 seconds was possible. Larger values would overflow and thus the 45 seconds set by Proxmox's backup with fleecing, resulted in effectively 2 seconds timeout for copy-before-write

[pve-devel] [PATCH v3 qemu 2/2] backup: improve error when copy-before-write fails for fleecing

2024-04-29 Thread Fiona Ebner
With fleecing, failure for copy-before-write does not fail the guest write, but only sets the snapshot error that is associated to the copy-before-write filter, making further requests to the snapshot access fail with EACCES, which then also fails the job. But that error code is not the root cause

[pve-devel] applied: [PATCH v2 qemu 1/2] fix #5409: backup: fix copy-before-write timeout

2024-04-29 Thread Thomas Lamprecht
Am 29/04/2024 um 16:27 schrieb Fiona Ebner: > The type for the copy-before-write timeout in nanoseconds was wrong. > By being just uint32_t, a maximum of slightly over 4 seconds was > possible. Larger values would overflow and thus the 45 seconds set by > Proxmox's backup with fleecing, resulted in

[pve-devel] [PATCH docs] backup: mention where fleecing can be configured

2024-04-29 Thread Fiona Ebner
Reported in the community forum: https://forum.proxmox.com/threads/145955/post-658380 Signed-off-by: Fiona Ebner --- vzdump.adoc | 6 +- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/vzdump.adoc b/vzdump.adoc index 79d4bbc..9b1cb61 100644 --- a/vzdump.adoc +++ b/vzdump.adoc @@

[pve-devel] [PATCH v2 qemu 1/2] fix #5409: backup: fix copy-before-write timeout

2024-04-29 Thread Fiona Ebner
The type for the copy-before-write timeout in nanoseconds was wrong. By being just uint32_t, a maximum of slightly over 4 seconds was possible. Larger values would overflow and thus the 45 seconds set by Proxmox's backup with fleecing, resulted in effectively 2 seconds timeout for copy-before-write

[pve-devel] [PATCH v2 qemu 2/2] backup: improve error when copy-before-write fails for fleecing

2024-04-29 Thread Fiona Ebner
With fleecing, failure for copy-before-write does not fail the guest write, but only sets the snapshot error that is associated to the copy-before-write filter, making further requests to the snapshot access fail with EACCES, which then also fails the job. But that error code is not the root cause

[pve-devel] applied: [PATCH docs v2] fix #5429: network: override device names: include Type=ether

2024-04-29 Thread Thomas Lamprecht
Am 29/04/2024 um 09:49 schrieb Friedrich Weber: > Mention that the systemd link file should contain `Type=ether` in most > setup, to make sure it only applies to Ethernet devices and does not > ever apply to e.g. bridges or bonds which inherit the MAC address of > the Ethernet device. Mention that

[pve-devel] applied: [PATCH-SERIES qemu] update to QEMU 9.0.0

2024-04-29 Thread Thomas Lamprecht
Am 25/04/2024 um 17:21 schrieb Fiona Ebner: > QEMU 8.2.2 required many changes, in particular to the alloc-track > block driver. It should be the same as [0] just with backup fleecing > patches added in. See the patch for details. > > The only bigger change in QEMU 9.0.0 is that the AioContext loc

[pve-devel] [PATCH manager 2/2] ui: lxc: add deny read/write options for device passthrough

2024-04-29 Thread Filip Schauer
Add checkboxes to the device passthrough dialogue for restricting container read/write access to a passed through device. Signed-off-by: Filip Schauer --- www/manager6/lxc/DeviceEdit.js | 16 1 file changed, 16 insertions(+) diff --git a/www/manager6/lxc/DeviceEdit.js b/www/man

[pve-devel] [PATCH container 1/2] add deny read/write options for device passthrough

2024-04-29 Thread Filip Schauer
Add the deny_read and deny_write options for device passthrough, to restrict container access to devices. Signed-off-by: Filip Schauer --- src/PVE/LXC.pm| 13 - src/PVE/LXC/Config.pm | 10 ++ 2 files changed, 22 insertions(+), 1 deletion(-) diff --git a/src/PVE/LXC.p

[pve-devel] [PATCH container/manager 0/2] add deny read/write options for device passthrough

2024-04-29 Thread Filip Schauer
Add the deny_read and deny_write options for device passthrough, to restrict container access to devices. This allows for passing through a device in read-only mode without giving the container full access it. Up until now a container with a device passed through to it was granted full access to

Re: [pve-devel] [PATCH qemu] PVE backup: improve error when copy-before-write fails for fleecing

2024-04-29 Thread Fiona Ebner
Am 29.04.24 um 14:53 schrieb Fiona Ebner: > With fleecing, failure for copy-before-write does not fail the guest > write, but only sets the snapshot error that is associated to the > copy-before-write filter, making further requests to the snapshot > access fail with EACCES, which then also fails t

[pve-devel] [PATCH qemu] PVE backup: improve error when copy-before-write fails for fleecing

2024-04-29 Thread Fiona Ebner
With fleecing, failure for copy-before-write does not fail the guest write, but only sets the snapshot error that is associated to the copy-before-write filter, making further requests to the snapshot access fail with EACCES, which then also fails the job. But that error code is not the root cause

[pve-devel] [PATCH manager v3 9/9] ui: import: show size for dir-based storages

2024-04-29 Thread Dominik Csapak
since there we already have the size information Signed-off-by: Dominik Csapak --- www/manager6/storage/Browser.js | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/www/manager6/storage/Browser.js b/www/manager6/storage/Browser.js index 763abc70..c0b66acc 100644 --- a/www/manag

[pve-devel] [PATCH manager v3 6/9] ui: import: improve rendering of volume names

2024-04-29 Thread Dominik Csapak
in directory storages, we don't need the 'import/' part of the volumes, as that is implied in dir based storages Signed-off-by: Dominik Csapak --- www/manager6/Utils.js | 8 +++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/www/manager6/Utils.js b/www/manager6/Utils.js index

[pve-devel] [PATCH storage v3 01/10] copy OVF.pm from qemu-server

2024-04-29 Thread Dominik Csapak
copies the OVF.pm and relevant ovf tests from qemu-server. We need it here, and it uses PVE::Storage already, and since there is no intermediary package/repository we could put it, it seems fitting in here. Put it in a new GuestImport module Signed-off-by: Dominik Csapak --- src/PVE/GuestImport

[pve-devel] [PATCH qemu-server v3 4/4] api: create: add 'import-extraction-storage' parameter

2024-04-29 Thread Dominik Csapak
this is to override the target extraction storage for the option disk extraction for 'import-from'. This way if the storage does not supports the content type 'images', one can give an alternative one. Signed-off-by: Dominik Csapak --- PVE/API2/Qemu.pm | 56 +

[pve-devel] [PATCH qemu-server v3 3/4] api: create: implement extracting disks when needed for import-from

2024-04-29 Thread Dominik Csapak
when 'import-from' contains a disk image that needs extraction (currently only from an 'ova' archive), do that in 'create_disks' and overwrite the '$source' volid. Collect the names into a 'delete_sources' list, that we use later to clean it up again (either when we're finished with importing or i

[pve-devel] [PATCH manager v3 8/9] ui: guest import: change icon/text for non-esxi import storage

2024-04-29 Thread Dominik Csapak
since 'virtual guests' only make sense for a hypervisor, not e.g. a directory for OVAs also change the icon from 'desktop' to 'cloud-download' in the non-esxi case Signed-off-by: Dominik Csapak --- www/manager6/storage/Browser.js | 6 -- 1 file changed, 4 insertions(+), 2 deletions(-) diff

[pve-devel] [PATCH manager v3 5/9] ui: disable 'import' button for non importable formats

2024-04-29 Thread Dominik Csapak
importable formats are currently ova/ovf/vmx Signed-off-by: Dominik Csapak --- www/manager6/storage/Browser.js | 8 +++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/www/manager6/storage/Browser.js b/www/manager6/storage/Browser.js index 934ce706..822257e7 100644 --- a/www/ma

[pve-devel] [PATCH qemu-server v3 1/4] api: delete unused OVF.pm

2024-04-29 Thread Dominik Csapak
the api part was never in use by anything Signed-off-by: Dominik Csapak --- PVE/API2/Qemu/Makefile | 2 +- PVE/API2/Qemu/OVF.pm | 53 -- 2 files changed, 1 insertion(+), 54 deletions(-) delete mode 100644 PVE/API2/Qemu/OVF.pm diff --git a/PVE/API2/Qem

[pve-devel] [PATCH storage v3 08/10] api: allow ova upload/download

2024-04-29 Thread Dominik Csapak
introducing a separate regex that only contains ova, since upload/downloading ovfs does not make sense (since the disks are then missing). Signed-off-by: Dominik Csapak --- src/PVE/API2/Storage/Status.pm | 18 ++ src/PVE/Storage.pm | 11 +++ 2 files changed, 2

[pve-devel] [PATCH storage v3 05/10] ovf: implement parsing out firmware type

2024-04-29 Thread Dominik Csapak
it seems there is no part of the ovf standard that handles which type of bios there is (at least i could not find it). Every ovf/ova i tested either has no info about it, or has it in a vmware specific property which we parse here. Signed-off-by: Dominik Csapak --- src/PVE/GuestImport/OVF.pm

[pve-devel] [PATCH storage v3 03/10] plugin: dir: handle ova files for import

2024-04-29 Thread Dominik Csapak
since we want to handle ova files (which are only ovf+images bundled in a tar file) for import, add code that handles that. we introduce a valid volname for files contained in ovas like this: storage:import/archive.ova/disk-1.vmdk by basically treating the last part of the path as the name for

[pve-devel] [PATCH storage/qemu-server/manager v3] implement ova/ovf import for file based storages

2024-04-29 Thread Dominik Csapak
This series enables importing ova/ovf from directory based storages, inclusive upload/download via the webui (ova only). It also improves the ovf importer by parsing the ostype, nics, bootorder (and firmware from vmware exported files). I opted to move the OVF.pm to pve-storage, since there is no

[pve-devel] [PATCH qemu-server v3 2/4] use OVF from Storage

2024-04-29 Thread Dominik Csapak
and delete it here (incl tests; they live in pve-storage now). Signed-off-by: Dominik Csapak --- PVE/CLI/qm.pm | 4 +- PVE/QemuServer/Makefile | 1 - PVE/QemuServer/OVF.pm | 242 -- test/Makefile

[pve-devel] [PATCH manager v3 7/9] ui: guest import: add storage selector for ova extraction storage

2024-04-29 Thread Dominik Csapak
but only when we detect the 'ova-needs-extraction' warning. This can be used to select the storage where the disks contained in an OVA will be extracted to temporarily. Signed-off-by: Dominik Csapak --- www/manager6/window/GuestImport.js | 23 +++ 1 file changed, 23 insertion

[pve-devel] [PATCH manager v3 3/9] ui: enable import content type for relevant storages

2024-04-29 Thread Dominik Csapak
Signed-off-by: Dominik Csapak --- www/manager6/Utils.js| 1 + www/manager6/form/ContentTypeSelector.js | 2 +- www/manager6/storage/CephFSEdit.js | 2 +- www/manager6/storage/GlusterFsEdit.js| 2 +- 4 files changed, 4 insertions(+), 3 deletions(-) diff --git a/www/m

[pve-devel] [PATCH manager v3 2/9] ui: guest import: add ova-needs-extracting warning text

2024-04-29 Thread Dominik Csapak
Signed-off-by: Dominik Csapak --- www/manager6/window/GuestImport.js | 1 + 1 file changed, 1 insertion(+) diff --git a/www/manager6/window/GuestImport.js b/www/manager6/window/GuestImport.js index 4bedc211..76ba6dc8 100644 --- a/www/manager6/window/GuestImport.js +++ b/www/manager6/window/Gues

[pve-devel] [PATCH manager v3 1/9] ui: fix special 'import' icon for non-esxi storages

2024-04-29 Thread Dominik Csapak
we only want to show that icon in the tree when the storage is solely used for importing, not when it's just one of several content types. Signed-off-by: Dominik Csapak --- www/manager6/Utils.js | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/www/manager6/Utils.js b/www/manag

[pve-devel] [PATCH manager v3 4/9] ui: enable upload/download/remove buttons for 'import' type storages

2024-04-29 Thread Dominik Csapak
but only for non esxi ones, since that does not allow uploading/downloading there Signed-off-by: Dominik Csapak --- www/manager6/storage/Browser.js| 9 +++-- www/manager6/window/UploadToStorage.js | 1 + 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/www/manager6/stor

[pve-devel] [PATCH storage v3 10/10] add 'import' content type to 'check_volume_access'

2024-04-29 Thread Dominik Csapak
in the same branch as 'vztmpl' and 'iso' Signed-off-by: Dominik Csapak --- src/PVE/Storage.pm | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/PVE/Storage.pm b/src/PVE/Storage.pm index 31b2ad5..fe29842 100755 --- a/src/PVE/Storage.pm +++ b/src/PVE/Storage.pm @@ -540,7 +540

[pve-devel] [PATCH storage v3 09/10] plugin: enable import for nfs/btrfs/cifs/cephfs/glusterfs

2024-04-29 Thread Dominik Csapak
and reuse the DirPlugin implementation Signed-off-by: Dominik Csapak --- src/PVE/Storage/BTRFSPlugin.pm | 5 + src/PVE/Storage/CIFSPlugin.pm | 6 +- src/PVE/Storage/CephFSPlugin.pm| 6 +- src/PVE/Storage/GlusterfsPlugin.pm | 6 +- src/PVE/Storage/NFSPlugin.pm |

[pve-devel] [PATCH storage v3 07/10] ovf: implement parsing nics

2024-04-29 Thread Dominik Csapak
by iterating over the relevant parts and trying to parse out the 'ResourceSubType'. The content of that is not standardized, but I only ever found examples that are compatible with vmware, meaning it's either 'e1000', 'e1000e' or 'vmxnet3' (in various capitalizations; thus the `lc()`) As a fallbac

[pve-devel] [PATCH storage v3 06/10] ovf: implement rudimentary boot order

2024-04-29 Thread Dominik Csapak
simply add all parsed disks to the boot order in the order we encounter them (similar to the esxi plugin). Signed-off-by: Dominik Csapak --- src/PVE/GuestImport/OVF.pm | 6 +- src/test/run_ovf_tests.pl | 3 +++ 2 files changed, 8 insertions(+), 1 deletion(-) diff --git a/src/PVE/GuestImpor

[pve-devel] [PATCH storage v3 02/10] plugin: dir: implement import content type

2024-04-29 Thread Dominik Csapak
in DirPlugin and not Plugin (because of cyclic dependency of Plugin -> OVF -> Storage -> Plugin otherwise) only ovf is currently supported (though ova will be shown in import listing), expects the files to not be in a subdir, and adjacent to the ovf file. listed will be all ovf/qcow2/raw/vmdk fil

[pve-devel] [PATCH storage v3 04/10] ovf: implement parsing the ostype

2024-04-29 Thread Dominik Csapak
use the standards info about the ostypes to map to our own (see comment for link to the relevant part of the dmtf schema) every type that is not listed we map to 'other', so no need to have it in a list. Signed-off-by: Dominik Csapak --- src/PVE/GuestImport/OVF.pm | 69 +

Re: [pve-devel] [PATCH manager] ui: Remove pveACMEPluginView in favor of pmxACMEPluginView

2024-04-29 Thread Thomas Lamprecht
subject: ui: acme: switch plugin view over to the one from widget-toolkit (having internal xtypes in the subject already is not really that useful) Am 29/08/2023 um 13:00 schrieb Filip Schauer: > Remove pveACMEPluginView and use the ACMEPluginView from the > proxmox-widget-toolkit instead. This

Re: [pve-devel] [PATCH container] setup: support Ubuntu 24.04 Noble

2024-04-29 Thread Thomas Lamprecht
Am 29/04/2024 um 11:56 schrieb Fiona Ebner: > Am 29.04.24 um 11:36 schrieb Fiona Ebner: >> Am 29.04.24 um 11:23 schrieb Fiona Ebner: >>> Reported in the community forum: >>> https://forum.proxmox.com/threads/145848/#post-658694 >>> >>> Signed-off-by: Fiona Ebner >>> --- >>> >>> Minimally tested, t

Re: [pve-devel] [PATCH manager] ui: Remove pveACMEPluginView in favor of pmxACMEPluginView

2024-04-29 Thread Filip Schauer
bump On 29/08/2023 13:00, Filip Schauer wrote: Remove pveACMEPluginView and use the ACMEPluginView from the proxmox-widget-toolkit instead. This leaves pveACMEPluginEditor unused, so remove it as well. Signed-off-by: Filip Schauer --- www/manager6/dc/ACMEClusterView.js | 100 + www

Re: [pve-devel] [PATCH container] setup: support Ubuntu 24.04 Noble

2024-04-29 Thread Fiona Ebner
Am 29.04.24 um 11:36 schrieb Fiona Ebner: > Am 29.04.24 um 11:23 schrieb Fiona Ebner: >> Reported in the community forum: >> https://forum.proxmox.com/threads/145848/#post-658694 >> >> Signed-off-by: Fiona Ebner >> --- >> >> Minimally tested, that an upgrade from an existing 23.04 container >> wor

Re: [pve-devel] [PATCH container] setup: support Ubuntu 24.04 Noble

2024-04-29 Thread Fiona Ebner
Am 29.04.24 um 11:23 schrieb Fiona Ebner: > Reported in the community forum: > https://forum.proxmox.com/threads/145848/#post-658694 > > Signed-off-by: Fiona Ebner > --- > > Minimally tested, that an upgrade from an existing 23.04 container > works, there still is network and no obviously bad me

[pve-devel] [PATCH container] setup: support Ubuntu 24.04 Noble

2024-04-29 Thread Fiona Ebner
Reported in the community forum: https://forum.proxmox.com/threads/145848/#post-658694 Signed-off-by: Fiona Ebner --- Minimally tested, that an upgrade from an existing 23.04 container works, there still is network and no obviously bad messages in the container's journal. src/PVE/LXC/Setup/Ubu

[pve-devel] [PATCH docs v2] fix #5429: network: override device names: include Type=ether

2024-04-29 Thread Friedrich Weber
Mention that the systemd link file should contain `Type=ether` in most setup, to make sure it only applies to Ethernet devices and does not ever apply to e.g. bridges or bonds which inherit the MAC address of the Ethernet device. Mention that some setups may require other options. Reported in the