[pve-devel] applied-series: [PATCH zfsonlinux v2 0/2] Update to ZFS 2.2.4

2024-05-21 Thread Thomas Lamprecht
Am 07/05/2024 um 17:02 schrieb Stoiko Ivanov:
> v1->v2:
> Patch 2/2 (adaptation of arc_summary/arcstat patch) modified:
> * right after sending the v1 I saw a report where pinning kernel 6.2 (thus
>   ZFS 2.1) leads to a similar traceback - which I seem to have overlooked
>   when packaging 2.2.0 ...
>   adapted the patch by booting a VM with kernel 6.2 and the current
>   userspace and running arc_summary /arcstat -a until no traceback was
>   displayed with a single-disk pool.
> 
> original cover-letter for v1:
> This patchset updates ZFS to the recently released 2.2.4
> 
> We had about half of the patches already in 2.2.3-2, due to the needed
> support for kernel 6.8.
> 
> Compared to the last 2.2 point releases this one compares quite a few
> potential performance improvments:
> * for ZVOL workloads (relevant for qemu guests) multiple taskq were
>   introduced [1] - this change is active by default (can be put back to
>   the old behavior with explicitly setting `zvol_num_taskqs=1`
> * the interface for ZFS submitting operations to the kernel's block layer
>   was augmented to better deal with split-pages [2] - which should also
>   improve performance, and prevent unaligned writes which are rejected by
>   e.g. the SCSI subsystem. - The default remains with the current code
>   (`zfs_vdev_disk_classic=0` turns on the 'new' behavior...)
> * Speculative prefetching was improved [3], which introduced not kstats,
>   which are reported by`arc_summary` and `arcstat`, as before with the
>   MRU/MFU additions there was not guard for running the new user-space
>   with an old kernel resulting in Python exceptions of both tools.
>   I adapted the patch where Thomas fixed that back in the 2.1 release
>   times. - sending as separate patch for easier review - and I hope it's
>   ok that I dropped the S-o-b tag (as it's changed code) - glad to resend
>   it, if this should be adapted.
> 
> Minimally tested on 2 VMs (the arcstat/arc_summary changes by running with
> an old kernel and new user-space)
> 
> 
> [0] https://github.com/openzfs/zfs/releases/tag/zfs-2.2.4
> [1] https://github.com/openzfs/zfs/pull/15992
> [2] https://github.com/openzfs/zfs/pull/15588
> [3] https://github.com/openzfs/zfs/pull/16022
> 
> Stoiko Ivanov (2):
>   update zfs submodule to 2.2.4 and refresh patches
>   update arc_summary arcstat patch with new introduced values
> 
>  ...md-unit-for-importing-specific-pools.patch |   4 +-
>  ...-move-manpage-arcstat-1-to-arcstat-8.patch |   2 +-
>  ...-guard-access-to-freshly-introduced-.patch | 438 
>  ...-guard-access-to-l2arc-MFU-MRU-stats.patch | 113 ---
>  ...hten-bounds-for-noalloc-stat-availab.patch |   4 +-
>  ...rectly-handle-partition-16-and-later.patch |  52 --
>  ...-use-splice_copy_file_range-for-fall.patch | 135 
>  .../0014-linux-5.4-compat-page_size.patch | 121 
>  .../patches/0015-abd-add-page-iterator.patch  | 334 -
>  ...-existing-functions-to-vdev_classic_.patch | 349 -
>  ...v_disk-reorganise-vdev_disk_io_start.patch | 111 ---
>  ...-read-write-IO-function-configurable.patch |  69 --
>  ...e-BIO-filling-machinery-to-avoid-spl.patch | 671 --
>  ...dule-parameter-to-select-BIO-submiss.patch | 104 ---
>  ...se-bio_chain-to-submit-multiple-BIOs.patch | 363 --
>  ...on-t-use-compound-heads-on-Linux-4.5.patch |  96 ---
>  ...ault-to-classic-submission-for-2.2.x.patch |  90 ---
>  ...ion-caused-by-mmap-flushing-problems.patch | 104 ---
>  ...touch-vbio-after-its-handed-off-to-t.patch |  57 --
>  debian/patches/series |  16 +-
>  upstream  |   2 +-
>  21 files changed, 445 insertions(+), 2790 deletions(-)
>  create mode 100644 
> debian/patches/0009-arc-stat-summary-guard-access-to-freshly-introduced-.patch
>  delete mode 100644 
> debian/patches/0009-arc-stat-summary-guard-access-to-l2arc-MFU-MRU-stats.patch
>  delete mode 100644 
> debian/patches/0012-udev-correctly-handle-partition-16-and-later.patch
>  delete mode 100644 
> debian/patches/0013-Linux-6.8-compat-use-splice_copy_file_range-for-fall.patch
>  delete mode 100644 debian/patches/0014-linux-5.4-compat-page_size.patch
>  delete mode 100644 debian/patches/0015-abd-add-page-iterator.patch
>  delete mode 100644 
> debian/patches/0016-vdev_disk-rename-existing-functions-to-vdev_classic_.patch
>  delete mode 100644 
> debian/patches/0017-vdev_disk-reorganise-vdev_disk_io_start.patch
>  delete mode 100644 
> debian/patches/0018-vdev_disk-make-read-write-IO-function-configurable.patch
>  delete mode 100644 
> debian/patches/0019-vdev_disk-rewrite-BIO-filling-machinery-to-avoid-spl.patch
>  delete mode 100644 
> debian/patches/0020-vdev_disk-add-module-parameter-to-select-BIO-submiss.patch
>  delete mode 100644 
> debian/patches/0021-vdev_disk-use-bio_chain-to-submit-multiple-BIOs.patch
>  delete mode 100644 
> debian/patches/0022-abd_iter_page-don-t-use-compound-heads-on-Linux-4.5.patch
>  delete mode 100644 

[pve-devel] applied: [PATCH docs] network: override device names: suggest running update-initramfs

2024-05-21 Thread Thomas Lamprecht
Am 21/05/2024 um 14:55 schrieb Friedrich Weber:
> The initramfs-tools hook /usr/share/initramfs-tools/hooks/udev copies
> link files from /etc/systemd/network to the initramfs, where they take
> effect in early userspace. If the link files in the initramfs diverge
> from the link files in the rootfs, this can lead to confusing
> behavior, as reported in enterprise support. For instance:
> 
> - If an interface matches link files both in the initramfs and the
>   rootfs, it will be renamed twice during boot.
> - A leftover link file in the initramfs renaming an interface A to a
>   new name X may prevent a link file in the rootfs from renaming a
>   different interface B to the same name X (it will fail with "File
>   exists").
> 
> To avoid this confusion, mention the link files are copied to the
> initramfs, and suggest updating the initramfs after making changes to
> the link files.
> 
> Suggested-by: Hannes Laimer 
> Signed-off-by: Friedrich Weber 
> ---
>  pve-network.adoc | 12 ++--
>  1 file changed, 10 insertions(+), 2 deletions(-)
> 
>

applied, thanks!


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel



[pve-devel] applied: [PATCH v2 manager] api: add proxmox-firewall to versions pkg list

2024-05-21 Thread Thomas Lamprecht
Am 24/04/2024 um 13:35 schrieb Mira Limbeck:
> Signed-off-by: Mira Limbeck 
> ---
> v2:
>  - add `api: ` prefix to commit msg
> 
>  PVE/API2/APT.pm | 1 +
>  1 file changed, 1 insertion(+)
> 
>

applied, thanks!


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel



[pve-devel] applied-series: [PATCH proxmox-firewall 1/2] firewall: improve handling of ARP traffic for guests

2024-05-21 Thread Thomas Lamprecht
Am 15/05/2024 um 15:37 schrieb Stefan Hanreich:
> In order to be able to send outgoing ARP packets when the default
> policy is set to drop or reject, we need to explicitly allow ARP
> traffic in the outgoing chain of guests. We need to do this in the
> guest chain itself in order to be able to filter spoofed packets via
> the MAC filter.
> 
> Contrary to the out direction we can simply accept all incoming ARP
> traffic, since we do not do any MAC filtering for incoming traffic.
> Since we create fdb entries for every NIC, guests should only see ARP
> traffic for their MAC addresses anyway.
> 
> Signed-off-by: Stefan Hanreich 
> Originally-by: Laurent Guerby 
> ---
>  proxmox-firewall/resources/proxmox-firewall.nft   | 1 +
>  proxmox-firewall/src/firewall.rs  | 8 
>  .../tests/snapshots/integration_tests__firewall.snap  | 4 ++--
>  3 files changed, 7 insertions(+), 6 deletions(-)
> 
>

applied both patches, thanks!

I reworded the subject here too and re-ordered the git trailers, as they
should have a causal order where possible. I.e., if someone else made a
patch, or helped you to do so, their co-authored-by or originally-by is
normally before your signed-off-by, as your "signature" shows that all
above it is (to your best knowledge) correct w.r.t patch ownership and
description, and like on "real" documents that signature goes at the
bottom.


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel



Re: [pve-devel] [PATCH zfsonlinux v2 1/2] update zfs submodule to 2.2.4 and refresh patches

2024-05-21 Thread Max Carrara
On Tue May 7, 2024 at 5:02 PM CEST, Stoiko Ivanov wrote:
> mostly - drop all patches we had queued up to get kernel 6.8
> supported.
>
> Signed-off-by: Stoiko Ivanov 
> ---

See my reply to the cover letter.

Tested-by: Max Carrara 

>  ...md-unit-for-importing-specific-pools.patch |   4 +-
>  ...-move-manpage-arcstat-1-to-arcstat-8.patch |   2 +-
>  ...-guard-access-to-l2arc-MFU-MRU-stats.patch |  12 +-
>  ...hten-bounds-for-noalloc-stat-availab.patch |   4 +-
>  ...rectly-handle-partition-16-and-later.patch |  52 --
>  ...-use-splice_copy_file_range-for-fall.patch | 135 
>  .../0014-linux-5.4-compat-page_size.patch | 121 
>  .../patches/0015-abd-add-page-iterator.patch  | 334 -
>  ...-existing-functions-to-vdev_classic_.patch | 349 -
>  ...v_disk-reorganise-vdev_disk_io_start.patch | 111 ---
>  ...-read-write-IO-function-configurable.patch |  69 --
>  ...e-BIO-filling-machinery-to-avoid-spl.patch | 671 --
>  ...dule-parameter-to-select-BIO-submiss.patch | 104 ---
>  ...se-bio_chain-to-submit-multiple-BIOs.patch | 363 --
>  ...on-t-use-compound-heads-on-Linux-4.5.patch |  96 ---
>  ...ault-to-classic-submission-for-2.2.x.patch |  90 ---
>  ...ion-caused-by-mmap-flushing-problems.patch | 104 ---
>  ...touch-vbio-after-its-handed-off-to-t.patch |  57 --
>  debian/patches/series |  14 -
>  upstream  |   2 +-
>  20 files changed, 12 insertions(+), 2682 deletions(-)
>  delete mode 100644 
> debian/patches/0012-udev-correctly-handle-partition-16-and-later.patch
>  delete mode 100644 
> debian/patches/0013-Linux-6.8-compat-use-splice_copy_file_range-for-fall.patch
>  delete mode 100644 debian/patches/0014-linux-5.4-compat-page_size.patch
>  delete mode 100644 debian/patches/0015-abd-add-page-iterator.patch
>  delete mode 100644 
> debian/patches/0016-vdev_disk-rename-existing-functions-to-vdev_classic_.patch
>  delete mode 100644 
> debian/patches/0017-vdev_disk-reorganise-vdev_disk_io_start.patch
>  delete mode 100644 
> debian/patches/0018-vdev_disk-make-read-write-IO-function-configurable.patch
>  delete mode 100644 
> debian/patches/0019-vdev_disk-rewrite-BIO-filling-machinery-to-avoid-spl.patch
>  delete mode 100644 
> debian/patches/0020-vdev_disk-add-module-parameter-to-select-BIO-submiss.patch
>  delete mode 100644 
> debian/patches/0021-vdev_disk-use-bio_chain-to-submit-multiple-BIOs.patch
>  delete mode 100644 
> debian/patches/0022-abd_iter_page-don-t-use-compound-heads-on-Linux-4.5.patch
>  delete mode 100644 
> debian/patches/0023-vdev_disk-default-to-classic-submission-for-2.2.x.patch
>  delete mode 100644 
> debian/patches/0024-Fix-corruption-caused-by-mmap-flushing-problems.patch
>  delete mode 100644 
> debian/patches/0025-vdev_disk-don-t-touch-vbio-after-its-handed-off-to-t.patch
>
> diff --git 
> a/debian/patches/0007-Add-systemd-unit-for-importing-specific-pools.patch 
> b/debian/patches/0007-Add-systemd-unit-for-importing-specific-pools.patch
> index 8232978c..0600296f 100644
> --- a/debian/patches/0007-Add-systemd-unit-for-importing-specific-pools.patch
> +++ b/debian/patches/0007-Add-systemd-unit-for-importing-specific-pools.patch
> @@ -18,7 +18,7 @@ Signed-off-by: Thomas Lamprecht 
>  ---
>   etc/Makefile.am   |  1 +
>   etc/systemd/system/50-zfs.preset  |  1 +
> - etc/systemd/system/zfs-imp...@.service.in | 18 
> + etc/systemd/system/zfs-imp...@.service.in | 18 ++
>   3 files changed, 20 insertions(+)
>   create mode 100644 etc/systemd/system/zfs-imp...@.service.in


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel



[pve-devel] applied: [PATCH proxmox-firewall v2 1/1] firewall: properly reject ipv6 traffic

2024-05-21 Thread Thomas Lamprecht
Am 13/05/2024 um 14:14 schrieb Stefan Hanreich:
> ICMPv6 has different message types for rejecting traffic. With ICMP we
> used host-prohibited as rejection type, which doesn't exist in ICMPv6.
> Add an additional rule for IPv6, so it uses admin-prohibited.
> 
> Additionally, add a terminal drop statement in order to prevent any
> traffic that does not get matched from bypassing the reject chain.
> 
> Signed-off-by: Stefan Hanreich 
> ---
> Changes from v1 -> v2:
> * add a terminal drop statement to prevent any unmatched traffic from
>   bypassing the reject chain
> * properly match ICMPv6 traffic via l4proto
> 
>  proxmox-firewall/resources/proxmox-firewall.nft | 8 ++--
>  1 file changed, 6 insertions(+), 2 deletions(-)
> 
>

applied, with an updated commit subject (as per our guideline[0], using the
"firewall" tag inside a repo that has "firewall" already in the name is not
really adding much), thanks!

[0]: 
https://pve.proxmox.com/wiki/Developer_Documentation#Commits_and_Commit_Messages


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel



Re: [pve-devel] [PATCH zfsonlinux v2 2/2] update arc_summary arcstat patch with new introduced values

2024-05-21 Thread Max Carrara
On Tue May 7, 2024 at 5:02 PM CEST, Stoiko Ivanov wrote:
> ZFS 2.2.4 added new kstats for speculative prefetch in:
> 026fe796465e3da7b27d06ef5338634ee6dd30d8
>
> Adapt our patch introduced with ZFS 2.1 (for the then added MFU/MRU
> stats), to also deal with the now introduced values not being present
> (because an old kernel-module does not offer them).
>
> Signed-off-by: Stoiko Ivanov 
> ---

See my reply to the cover letter.

Reviewed-by: Max Carrara 
Tested-by: Max Carrara 

>  ...-guard-access-to-freshly-introduced-.patch | 438 ++
>  ...-guard-access-to-l2arc-MFU-MRU-stats.patch | 113 -
>  debian/patches/series |   2 +-
>  3 files changed, 439 insertions(+), 114 deletions(-)
>  create mode 100644 
> debian/patches/0009-arc-stat-summary-guard-access-to-freshly-introduced-.patch
>  delete mode 100644 
> debian/patches/0009-arc-stat-summary-guard-access-to-l2arc-MFU-MRU-stats.patch
>
> diff --git 
> a/debian/patches/0009-arc-stat-summary-guard-access-to-freshly-introduced-.patch
>  
> b/debian/patches/0009-arc-stat-summary-guard-access-to-freshly-introduced-.patch
> new file mode 100644
> index ..bc7db2a9
> --- /dev/null
> +++ 
> b/debian/patches/0009-arc-stat-summary-guard-access-to-freshly-introduced-.patch
> @@ -0,0 +1,438 @@
> +From  Mon Sep 17 00:00:00 2001
> +From: Thomas Lamprecht 
> +Date: Wed, 10 Nov 2021 09:29:47 +0100
> +Subject: [PATCH] arc stat/summary: guard access to freshly introduced stats
> +
> +l2arc MFU/MRU and zfetch past future and stride stats were introduced
> +in 2.1 and 2.2.4 respectively:
> +
> +commit 085321621e79a75bea41c2b6511da6ebfbf2ba0a added printing MFU
> +and MRU stats for 2.1 user space tools, but those keys are not
> +available in the 2.0 module. That means it may break the arcstat and
> +arc_summary tools after upgrade to 2.1 (user space), before a reboot
> +to the new 2.1 ZFS kernel-module happened, due to python raising a
> +KeyError on the dict access then.
> +
> +Move those two keys to a .get accessor with `0` as fallback, as it
> +should be better to show some possible wrong data for new stat-keys
> +than throwing an exception.
> +
> +also move l2_mfu_asize  l2_mru_asize l2_prefetch_asize
> +l2_bufc_data_asize l2_bufc_metadata_asize to .get accessor
> +(these are only present with a cache device in the pool)
> +
> +guard access to iohits and uncached state introduced in
> +792a6ee462efc15a7614f27e13f0f8aaa9414a08
> +
> +guard access to zfetch past future stride stats introduced in
> +026fe796465e3da7b27d06ef5338634ee6dd30d8
> +
> +These are present in the current kernel, but lead to an exception, if
> +running the new user-space with an old kernel module.
> +
> +Signed-off-by: Stoiko Ivanov 
> +---
> + cmd/arc_summary | 132 
> + cmd/arcstat.in  |  48 +-
> + 2 files changed, 90 insertions(+), 90 deletions(-)
> +
> +diff --git a/cmd/arc_summary b/cmd/arc_summary
> +index 100fb1987..30f5d23e9 100755
> +--- a/cmd/arc_summary
>  b/cmd/arc_summary
> +@@ -551,21 +551,21 @@ def section_arc(kstats_dict):
> + arc_target_size = arc_stats['c']
> + arc_max = arc_stats['c_max']
> + arc_min = arc_stats['c_min']
> +-meta = arc_stats['meta']
> +-pd = arc_stats['pd']
> +-pm = arc_stats['pm']
> +-anon_data = arc_stats['anon_data']
> +-anon_metadata = arc_stats['anon_metadata']
> +-mfu_data = arc_stats['mfu_data']
> +-mfu_metadata = arc_stats['mfu_metadata']
> +-mru_data = arc_stats['mru_data']
> +-mru_metadata = arc_stats['mru_metadata']
> +-mfug_data = arc_stats['mfu_ghost_data']
> +-mfug_metadata = arc_stats['mfu_ghost_metadata']
> +-mrug_data = arc_stats['mru_ghost_data']
> +-mrug_metadata = arc_stats['mru_ghost_metadata']
> +-unc_data = arc_stats['uncached_data']
> +-unc_metadata = arc_stats['uncached_metadata']
> ++meta = arc_stats.get('meta', 0)
> ++pd = arc_stats.get('pd', 0)
> ++pm = arc_stats.get('pm', 0)
> ++anon_data = arc_stats.get('anon_data', 0)
> ++anon_metadata = arc_stats.get('anon_metadata', 0)
> ++mfu_data = arc_stats.get('mfu_data', 0)
> ++mfu_metadata = arc_stats.get('mfu_metadata', 0)
> ++mru_data = arc_stats.get('mru_data', 0)
> ++mru_metadata = arc_stats.get('mru_metadata', 0)
> ++mfug_data = arc_stats.get('mfu_ghost_data', 0)
> ++mfug_metadata = arc_stats.get('mfu_ghost_metadata', 0)
> ++mrug_data = arc_stats.get('mru_ghost_data', 0)
> ++mrug_metadata = arc_stats.get('mru_ghost_metadata', 0)
> ++unc_data = arc_stats.get('uncached_data', 0)
> ++unc_metadata = arc_stats.get('uncached_metadata', 0)
> + bonus_size = arc_stats['bonus_size']
> + dnode_limit = arc_stats['arc_dnode_limit']
> + dnode_size = arc_stats['dnode_size']
> +@@ -655,13 +655,13 @@ def section_arc(kstats_dict):
> + prt_i1('L2 cached evictions:', f_bytes(arc_stats['evict_l2_cached']))
> +  

[pve-devel] [PATCH pve-ha-manager v3 5/8] env: notify: use named templates instead of passing template strings

2024-05-21 Thread Lukas Wagner
Signed-off-by: Lukas Wagner 
---
 debian/pve-ha-manager.install |  3 +++
 src/Makefile  |  1 +
 src/PVE/HA/Env/PVE2.pm|  4 ++--
 src/PVE/HA/NodeStatus.pm  | 20 +--
 src/PVE/HA/Sim/Env.pm |  3 ++-
 src/templates/Makefile| 10 ++
 src/templates/default/fencing-body.html.hbs   | 14 +
 src/templates/default/fencing-body.txt.hbs| 11 ++
 src/templates/default/fencing-subject.txt.hbs |  1 +
 9 files changed, 45 insertions(+), 22 deletions(-)
 create mode 100644 src/templates/Makefile
 create mode 100644 src/templates/default/fencing-body.html.hbs
 create mode 100644 src/templates/default/fencing-body.txt.hbs
 create mode 100644 src/templates/default/fencing-subject.txt.hbs

diff --git a/debian/pve-ha-manager.install b/debian/pve-ha-manager.install
index a7598a9..0ffbd8d 100644
--- a/debian/pve-ha-manager.install
+++ b/debian/pve-ha-manager.install
@@ -38,3 +38,6 @@
 /usr/share/perl5/PVE/HA/Usage/Static.pm
 /usr/share/perl5/PVE/Service/pve_ha_crm.pm
 /usr/share/perl5/PVE/Service/pve_ha_lrm.pm
+/usr/share/pve-manager/templates/default/fencing-body.html.hbs
+/usr/share/pve-manager/templates/default/fencing-body.txt.hbs
+/usr/share/pve-manager/templates/default/fencing-subject.txt.hbs
diff --git a/src/Makefile b/src/Makefile
index 87bb0de..56bd360 100644
--- a/src/Makefile
+++ b/src/Makefile
@@ -73,6 +73,7 @@ install: watchdog-mux pve-ha-crm pve-ha-lrm ha-manager.1 
pve-ha-crm.8 pve-ha-lrm
install -d $(DESTDIR)/$(MAN1DIR)
install -m 0644 ha-manager.1 $(DESTDIR)/$(MAN1DIR)
gzip -9 $(DESTDIR)/$(MAN1DIR)/ha-manager.1
+   $(MAKE) -C templates $@
 
 .PHONY: test
 test: 
diff --git a/src/PVE/HA/Env/PVE2.pm b/src/PVE/HA/Env/PVE2.pm
index fcb60a9..cb73bcf 100644
--- a/src/PVE/HA/Env/PVE2.pm
+++ b/src/PVE/HA/Env/PVE2.pm
@@ -221,10 +221,10 @@ sub log {
 }
 
 sub send_notification {
-my ($self, $subject, $text, $template_data, $metadata_fields) = @_;
+my ($self, $template_name, $template_data, $metadata_fields) = @_;
 
 eval {
-   PVE::Notify::error($subject, $text, $template_data, $metadata_fields);
+   PVE::Notify::error($template_name, $template_data, $metadata_fields);
 };
 
 $self->log("warning", "could not notify: $@") if $@;
diff --git a/src/PVE/HA/NodeStatus.pm b/src/PVE/HA/NodeStatus.pm
index e053c55..9e6d898 100644
--- a/src/PVE/HA/NodeStatus.pm
+++ b/src/PVE/HA/NodeStatus.pm
@@ -188,23 +188,6 @@ sub update {
}
 }
 
-my $body_template = {"subject-prefix"}/r;
 $subject = $subject =~ s/\{\{subject}}/$properties->{"subject"}/r;
 
diff --git a/src/templates/Makefile b/src/templates/Makefile
new file mode 100644
index 000..396759f
--- /dev/null
+++ b/src/templates/Makefile
@@ -0,0 +1,10 @@
+NOTIFICATION_TEMPLATES=
\
+   default/fencing-subject.txt.hbs \
+   default/fencing-body.txt.hbs\
+   default/fencing-body.html.hbs   \
+
+.PHONY: install
+install:
+   install -dm 0755 $(DESTDIR)/usr/share/pve-manager/templates/default
+   $(foreach i,$(NOTIFICATION_TEMPLATES), \
+   install -m644 $(i) $(DESTDIR)/usr/share/pve-manager/templates/$(i) 
;)
diff --git a/src/templates/default/fencing-body.html.hbs 
b/src/templates/default/fencing-body.html.hbs
new file mode 100644
index 000..1420348
--- /dev/null
+++ b/src/templates/default/fencing-body.html.hbs
@@ -0,0 +1,14 @@
+
+
+The node '{{node}}' failed and needs manual intervention.
+
+The PVE HA manager tries to fence it and recover the configured HA 
resources to
+a healthy node if possible.
+
+Current fence status: {{subject-prefix}}
+{{subject}}
+
+Overall Cluster status:
+{{object status-data}}
+
+
diff --git a/src/templates/default/fencing-body.txt.hbs 
b/src/templates/default/fencing-body.txt.hbs
new file mode 100644
index 000..e46a1fd
--- /dev/null
+++ b/src/templates/default/fencing-body.txt.hbs
@@ -0,0 +1,11 @@
+The node '{{node}}' failed and needs manual intervention.
+
+The PVE HA manager tries to 

[pve-devel] [PATCH proxmox-perl-rs v3 2/8] notify: don't pass config structs by reference

2024-05-21 Thread Lukas Wagner
proxmox_notify's api functions have been changed so that they take
ownership of config structs.

Signed-off-by: Lukas Wagner 
---
 common/src/notify.rs | 24 
 1 file changed, 12 insertions(+), 12 deletions(-)

diff --git a/common/src/notify.rs b/common/src/notify.rs
index d965417..00a6056 100644
--- a/common/src/notify.rs
+++ b/common/src/notify.rs
@@ -151,7 +151,7 @@ mod export {
 
 api::sendmail::add_endpoint(
  config,
- {
+SendmailConfig {
 name,
 mailto,
 mailto_user,
@@ -185,7 +185,7 @@ mod export {
 api::sendmail::update_endpoint(
  config,
 name,
- {
+SendmailConfigUpdater {
 mailto,
 mailto_user,
 from_address,
@@ -236,7 +236,7 @@ mod export {
 let mut config = this.config.lock().unwrap();
 api::gotify::add_endpoint(
  config,
- {
+GotifyConfig {
 name: name.clone(),
 server,
 comment,
@@ -244,7 +244,7 @@ mod export {
 filter: None,
 origin: None,
 },
- { name, token },
+GotifyPrivateConfig { name, token },
 )
 }
 
@@ -266,12 +266,12 @@ mod export {
 api::gotify::update_endpoint(
  config,
 name,
- {
+GotifyConfigUpdater {
 server,
 comment,
 disable,
 },
- { token },
+GotifyPrivateConfigUpdater { token },
 delete.as_deref(),
 digest.as_deref(),
 )
@@ -323,7 +323,7 @@ mod export {
 let mut config = this.config.lock().unwrap();
 api::smtp::add_endpoint(
  config,
- {
+SmtpConfig {
 name: name.clone(),
 server,
 port,
@@ -337,7 +337,7 @@ mod export {
 disable,
 origin: None,
 },
- { name, password },
+SmtpPrivateConfig { name, password },
 )
 }
 
@@ -366,7 +366,7 @@ mod export {
 api::smtp::update_endpoint(
  config,
 name,
- {
+SmtpConfigUpdater {
 server,
 port,
 mode,
@@ -378,7 +378,7 @@ mod export {
 comment,
 disable,
 },
- { password },
+SmtpPrivateConfigUpdater { password },
 delete.as_deref(),
 digest.as_deref(),
 )
@@ -427,7 +427,7 @@ mod export {
 let mut config = this.config.lock().unwrap();
 api::matcher::add_matcher(
  config,
- {
+MatcherConfig {
 name,
 match_severity,
 match_field,
@@ -464,7 +464,7 @@ mod export {
 api::matcher::update_matcher(
  config,
 name,
- {
+MatcherConfigUpdater {
 match_severity,
 match_field,
 match_calendar,
-- 
2.39.2



___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel



[pve-devel] [PATCH many v3 0/8] notifications: move template strings to template files

2024-05-21 Thread Lukas Wagner
These changes adapts the PVE notification stack to the changes introduced
in proxmox-notify 0.4.

The notification system uses handlebar templates to render the subject
and the body of notifications. Previously, the template strings were
defined inline at the call site. This patch series extracts the templates
into template files and installs them at
  /usr/share/pve-manager/templates/default

where they stored as -{body,subject}.{txt,html}.hbs

The 'default' part in the path is a preparation for translated
notifications and/or overridable notification templates.
Future work could provide notifications translated to e.g. German
in `templates/de` or similar. This will be a first for having
translated strings on the backend-side, so there is need for further
research.

Folke kindly did some off-list testing before this was posted, hence
his T-bs were included.

Bumps/dependencies:
  - libproxmox-rs-perl needs to have its proxmox-notify dep updated to 0.4
and breaks old libpve-notify-perl (versioned break)
  - libpve-notify-perl breaks old pve-manager and old pve-ha-manager (versioned 
break)

The versioned breaks are necessary due to changed semantics in the API (passing
a template name instead of template strings) and due to changes in how
templates are rendered (separate templates for HTML/plain text, whereas before
both were rendered from the same template string, with some magic from
handlebar helpers)

Changes since v2:
  - Dropped already applied patches for 'proxmox'
  - Rebased, quick smoke-test to check if anything broke
in the meanwhile

Changes since v1:
  - Incorporated feedback from @Fiona and @Fabian - thanks!
- most noteworthy: Change template path from /usr/share/proxmox-ve
  to /usr/share/pve-manager
- apart from that mostly just cosmetics/style

proxmox-perl-rs:

Lukas Wagner (3):
  notify: use file based notification templates
  notify: don't pass config structs by reference
  notify: adapt to Option> to Vec changes in proxmox_notify

 common/src/notify.rs | 48 +---
 1 file changed, 23 insertions(+), 25 deletions(-)


pve-cluster:

Lukas Wagner (1):
  notify: use named template instead of passing template strings

 src/PVE/Notify.pm | 29 -
 1 file changed, 12 insertions(+), 17 deletions(-)


pve-ha-manager:

Lukas Wagner (1):
  env: notify: use named templates instead of passing template strings

 debian/pve-ha-manager.install |  3 +++
 src/Makefile  |  1 +
 src/PVE/HA/Env/PVE2.pm|  4 ++--
 src/PVE/HA/NodeStatus.pm  | 20 +--
 src/PVE/HA/Sim/Env.pm |  3 ++-
 src/templates/Makefile| 10 ++
 src/templates/default/fencing-body.html.hbs   | 14 +
 src/templates/default/fencing-body.txt.hbs| 11 ++
 src/templates/default/fencing-subject.txt.hbs |  1 +
 9 files changed, 45 insertions(+), 22 deletions(-)
 create mode 100644 src/templates/Makefile
 create mode 100644 src/templates/default/fencing-body.html.hbs
 create mode 100644 src/templates/default/fencing-body.txt.hbs
 create mode 100644 src/templates/default/fencing-subject.txt.hbs


pve-manager:

Lukas Wagner (3):
  gitignore: ignore any test artifacts
  tests: remove vzdump_notification test
  notifications: use named templates instead of in-code templates

 .gitignore|   2 +
 Makefile  |   2 +-
 PVE/API2/APT.pm   |   9 +-
 PVE/API2/Replication.pm   |  20 +---
 PVE/VZDump.pm |  20 +---
 templates/Makefile|  24 +
 .../default/package-updates-body.html.hbs |   6 ++
 .../default/package-updates-body.txt.hbs  |   3 +
 .../default/package-updates-subject.txt.hbs   |   1 +
 templates/default/replication-body.html.hbs   |  18 
 templates/default/replication-body.txt.hbs|  12 +++
 templates/default/replication-subject.txt.hbs |   1 +
 templates/default/test-body.html.hbs  |   1 +
 templates/default/test-body.txt.hbs   |   1 +
 templates/default/test-subject.txt.hbs|   1 +
 templates/default/vzdump-body.html.hbs|  11 ++
 templates/default/vzdump-body.txt.hbs |  10 ++
 templates/default/vzdump-subject.txt.hbs  |   1 +
 test/Makefile |   6 +-
 test/vzdump_notification_test.pl  | 101 --
 20 files changed, 98 insertions(+), 152 deletions(-)
 create mode 100644 templates/Makefile
 create mode 100644 templates/default/package-updates-body.html.hbs
 create mode 100644 templates/default/package-updates-body.txt.hbs
 create mode 100644 templates/default/package-updates-subject.txt.hbs
 create mode 100644 templates/default/replication-body.html.hbs
 create mode 100644 

[pve-devel] [PATCH proxmox-perl-rs v3 3/8] notify: adapt to Option> to Vec changes in proxmox_notify

2024-05-21 Thread Lukas Wagner
Signed-off-by: Lukas Wagner 
---
 common/src/notify.rs | 16 
 1 file changed, 8 insertions(+), 8 deletions(-)

diff --git a/common/src/notify.rs b/common/src/notify.rs
index 00a6056..e1b006b 100644
--- a/common/src/notify.rs
+++ b/common/src/notify.rs
@@ -153,8 +153,8 @@ mod export {
  config,
 SendmailConfig {
 name,
-mailto,
-mailto_user,
+mailto: mailto.unwrap_or_default(),
+mailto_user: mailto_user.unwrap_or_default(),
 from_address,
 author,
 comment,
@@ -329,8 +329,8 @@ mod export {
 port,
 mode,
 username,
-mailto,
-mailto_user,
+mailto: mailto.unwrap_or_default(),
+mailto_user: mailto_user.unwrap_or_default(),
 from_address,
 author,
 comment,
@@ -429,10 +429,10 @@ mod export {
  config,
 MatcherConfig {
 name,
-match_severity,
-match_field,
-match_calendar,
-target,
+match_severity: match_severity.unwrap_or_default(),
+match_field: match_field.unwrap_or_default(),
+match_calendar: match_calendar.unwrap_or_default(),
+target: target.unwrap_or_default(),
 mode,
 invert_match,
 comment,
-- 
2.39.2



___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel



Re: [pve-devel] [PATCH zfsonlinux v2 0/2] Update to ZFS 2.2.4

2024-05-21 Thread Max Carrara
On Tue May 7, 2024 at 5:02 PM CEST, Stoiko Ivanov wrote:
> v1->v2:
> Patch 2/2 (adaptation of arc_summary/arcstat patch) modified:
> * right after sending the v1 I saw a report where pinning kernel 6.2 (thus
>   ZFS 2.1) leads to a similar traceback - which I seem to have overlooked
>   when packaging 2.2.0 ...
>   adapted the patch by booting a VM with kernel 6.2 and the current
>   userspace and running arc_summary /arcstat -a until no traceback was
>   displayed with a single-disk pool.

Testing
---

* Built and installed ZFS with those two patches on my test VM
  - Note: Couldn't install zfs-initramfs and zfs-dracut due to some
dependency issue
  - zfs-initramfs depends on initramfs-tools, but complained it wasn't
available (even though the package is installed ...)
  - zfs-dracut did the same for dracut
  - initramfs-tools then conflicts with the virtual linux-initramfs-tool
  - Removing zfs-initramfs from the packages to be installed "fixed"
this; all other packages then installed without any issue

* `arcstat -a` and `arc_summary` correctly displayed the new values
  while old kernel was still running

* Didn't encounter any exceptions

* VM also survived a reboot - same results for new kernel

* Didn't notice anything off overall while the VM was running - will
  holler if I find anything

Review
--

Looked specifically at patch 02; applied and diffed it on the upstream
ZFS sources checked out at tag `zfs-2.2.4`. What can I say, it's just
replacing calls to `obj.__getitem__()` with `obj.get('foo', 0)` - so,
pretty straightforward. (The original code could use a brush-up, but
that's beside the point.)

Summary
---

All in all, LGTM - haven't really looked at patch 01 in detail, so I'll
add my R-b tag only to patch 02. Good work!


>
> original cover-letter for v1:
> This patchset updates ZFS to the recently released 2.2.4
>
> We had about half of the patches already in 2.2.3-2, due to the needed
> support for kernel 6.8.
>
> Compared to the last 2.2 point releases this one compares quite a few
> potential performance improvments:
> * for ZVOL workloads (relevant for qemu guests) multiple taskq were
>   introduced [1] - this change is active by default (can be put back to
>   the old behavior with explicitly setting `zvol_num_taskqs=1`
> * the interface for ZFS submitting operations to the kernel's block layer
>   was augmented to better deal with split-pages [2] - which should also
>   improve performance, and prevent unaligned writes which are rejected by
>   e.g. the SCSI subsystem. - The default remains with the current code
>   (`zfs_vdev_disk_classic=0` turns on the 'new' behavior...)
> * Speculative prefetching was improved [3], which introduced not kstats,
>   which are reported by`arc_summary` and `arcstat`, as before with the
>   MRU/MFU additions there was not guard for running the new user-space
>   with an old kernel resulting in Python exceptions of both tools.
>   I adapted the patch where Thomas fixed that back in the 2.1 release
>   times. - sending as separate patch for easier review - and I hope it's
>   ok that I dropped the S-o-b tag (as it's changed code) - glad to resend
>   it, if this should be adapted.
>
> Minimally tested on 2 VMs (the arcstat/arc_summary changes by running with
> an old kernel and new user-space)
>
>
> [0] https://github.com/openzfs/zfs/releases/tag/zfs-2.2.4
> [1] https://github.com/openzfs/zfs/pull/15992
> [2] https://github.com/openzfs/zfs/pull/15588
> [3] https://github.com/openzfs/zfs/pull/16022
>
> Stoiko Ivanov (2):
>   update zfs submodule to 2.2.4 and refresh patches
>   update arc_summary arcstat patch with new introduced values
>
>  ...md-unit-for-importing-specific-pools.patch |   4 +-
>  ...-move-manpage-arcstat-1-to-arcstat-8.patch |   2 +-
>  ...-guard-access-to-freshly-introduced-.patch | 438 
>  ...-guard-access-to-l2arc-MFU-MRU-stats.patch | 113 ---
>  ...hten-bounds-for-noalloc-stat-availab.patch |   4 +-
>  ...rectly-handle-partition-16-and-later.patch |  52 --
>  ...-use-splice_copy_file_range-for-fall.patch | 135 
>  .../0014-linux-5.4-compat-page_size.patch | 121 
>  .../patches/0015-abd-add-page-iterator.patch  | 334 -
>  ...-existing-functions-to-vdev_classic_.patch | 349 -
>  ...v_disk-reorganise-vdev_disk_io_start.patch | 111 ---
>  ...-read-write-IO-function-configurable.patch |  69 --
>  ...e-BIO-filling-machinery-to-avoid-spl.patch | 671 --
>  ...dule-parameter-to-select-BIO-submiss.patch | 104 ---
>  ...se-bio_chain-to-submit-multiple-BIOs.patch | 363 --
>  ...on-t-use-compound-heads-on-Linux-4.5.patch |  96 ---
>  ...ault-to-classic-submission-for-2.2.x.patch |  90 ---
>  ...ion-caused-by-mmap-flushing-problems.patch | 104 ---
>  ...touch-vbio-after-its-handed-off-to-t.patch |  57 --
>  debian/patches/series |  16 +-
>  upstream  |   2 +-
>  21 files changed, 445 insertions(+), 

[pve-devel] [PATCH manager v3 8/8] notifications: use named templates instead of in-code templates

2024-05-21 Thread Lukas Wagner
This commit adapts notification sending for
- package update
- replication
- backups

to use named templates (installed in /usr/share/pve-manager/templates)
instead of passing template strings defined in code to the
notification stack.

Signed-off-by: Lukas Wagner 
---
 Makefile  |  2 +-
 PVE/API2/APT.pm   |  9 +--
 PVE/API2/Replication.pm   | 20 +---
 PVE/VZDump.pm | 20 ++--
 templates/Makefile| 24 +++
 .../default/package-updates-body.html.hbs |  6 +
 .../default/package-updates-body.txt.hbs  |  3 +++
 .../default/package-updates-subject.txt.hbs   |  1 +
 templates/default/replication-body.html.hbs   | 18 ++
 templates/default/replication-body.txt.hbs| 12 ++
 templates/default/replication-subject.txt.hbs |  1 +
 templates/default/test-body.html.hbs  |  1 +
 templates/default/test-body.txt.hbs   |  1 +
 templates/default/test-subject.txt.hbs|  1 +
 templates/default/vzdump-body.html.hbs| 11 +
 templates/default/vzdump-body.txt.hbs | 10 
 templates/default/vzdump-subject.txt.hbs  |  1 +
 17 files changed, 95 insertions(+), 46 deletions(-)
 create mode 100644 templates/Makefile
 create mode 100644 templates/default/package-updates-body.html.hbs
 create mode 100644 templates/default/package-updates-body.txt.hbs
 create mode 100644 templates/default/package-updates-subject.txt.hbs
 create mode 100644 templates/default/replication-body.html.hbs
 create mode 100644 templates/default/replication-body.txt.hbs
 create mode 100644 templates/default/replication-subject.txt.hbs
 create mode 100644 templates/default/test-body.html.hbs
 create mode 100644 templates/default/test-body.txt.hbs
 create mode 100644 templates/default/test-subject.txt.hbs
 create mode 100644 templates/default/vzdump-body.html.hbs
 create mode 100644 templates/default/vzdump-body.txt.hbs
 create mode 100644 templates/default/vzdump-subject.txt.hbs

diff --git a/Makefile b/Makefile
index 28295395..337682b3 100644
--- a/Makefile
+++ b/Makefile
@@ -10,7 +10,7 @@ DSC=$(PACKAGE)_$(DEB_VERSION).dsc
 DEB=$(PACKAGE)_$(DEB_VERSION)_$(DEB_HOST_ARCH).deb
 
 DESTDIR=
-SUBDIRS = aplinfo PVE bin www services configs network-hooks test
+SUBDIRS = aplinfo PVE bin www services configs network-hooks test templates
 
 all: $(SUBDIRS)
set -e && for i in $(SUBDIRS); do $(MAKE) -C $$i; done
diff --git a/PVE/API2/APT.pm b/PVE/API2/APT.pm
index 19f0baca..d0e7c544 100644
--- a/PVE/API2/APT.pm
+++ b/PVE/API2/APT.pm
@@ -238,12 +238,6 @@ __PACKAGE__->register_method({
return $pkglist;
 }});
 
-my $updates_available_subject_template = "New software packages available 
({{hostname}})";
-my $updates_available_body_template =  'update_database',
 path => 'update',
@@ -358,8 +352,7 @@ __PACKAGE__->register_method({
};
 
PVE::Notify::info(
-   $updates_available_subject_template,
-   $updates_available_body_template,
+   "package-updates",
$template_data,
$metadata_fields,
);
diff --git a/PVE/API2/Replication.pm b/PVE/API2/Replication.pm
index 0dc944c9..d84ac1ab 100644
--- a/PVE/API2/Replication.pm
+++ b/PVE/API2/Replication.pm
@@ -92,23 +92,6 @@ my sub _should_mail_at_failcount {
 return $i * 48 == $fail_count;
 };
 
-my $replication_error_subject_template = "Replication Job: '{{job-id}}' 
failed";
-my $replication_error_body_template = < 1024*1024;
 
 sub send_notification {
@@ -588,8 +574,7 @@ sub send_notification {
 
PVE::Notify::notify(
$severity,
-   $subject_template,
-   $body_template,
+   "vzdump",
$notification_props,
$fields,
$notification_config
@@ -600,8 +585,7 @@ sub send_notification {
# no email addresses were configured.
PVE::Notify::notify(
$severity,
-   $subject_template,
-   $body_template,
+   "vzdump",
$notification_props,
$fields,
);
diff --git a/templates/Makefile b/templates/Makefile
new file mode 100644
index ..236988c5
--- /dev/null
+++ b/templates/Makefile
@@ -0,0 +1,24 @@
+NOTIFICATION_TEMPLATES=\
+   default/test-subject.txt.hbs\
+   default/test-body.txt.hbs   \
+   default/test-body.html.hbs  \
+   default/vzdump-subject.txt.hbs  \
+   default/vzdump-body.txt.hbs \
+   default/vzdump-body.html.hbs\
+   default/replication-subject.txt.hbs \
+   

[pve-devel] [PATCH manager v3 7/8] tests: remove vzdump_notification test

2024-05-21 Thread Lukas Wagner
With the upcoming changes in how we send notifications, this one
really becomes pretty annoying to keep working. The location where
templates are looked up are defined in the proxmox_notify crate, so
there is no easy way to mock this for testing.
The test itself seemed not super valuable, mainly testing if
the backup logs are shortened if they ware too long - so they are just
removed.

Signed-off-by: Lukas Wagner 
---
 test/Makefile|   6 +-
 test/vzdump_notification_test.pl | 101 ---
 2 files changed, 1 insertion(+), 106 deletions(-)
 delete mode 100755 test/vzdump_notification_test.pl

diff --git a/test/Makefile b/test/Makefile
index 62d75050..743804c8 100644
--- a/test/Makefile
+++ b/test/Makefile
@@ -5,7 +5,7 @@ all:
 export PERLLIB=..
 
 .PHONY: check
-check: test-replication test-balloon test-vzdump-notification test-vzdump 
test-osd
+check: test-replication test-balloon test-vzdump test-osd
 
 .PHONY: test-balloon
 test-balloon:
@@ -17,10 +17,6 @@ test-replication: replication1.t replication2.t 
replication3.t replication4.t re
 replication%.t: replication_test%.pl
./$<
 
-.PHONY: test-vzdump-notification
-test-vzdump-notification:
-   ./vzdump_notification_test.pl
-
 .PHONY: test-vzdump
 test-vzdump: test-vzdump-guest-included test-vzdump-new
 
diff --git a/test/vzdump_notification_test.pl b/test/vzdump_notification_test.pl
deleted file mode 100755
index 631606bb..
--- a/test/vzdump_notification_test.pl
+++ /dev/null
@@ -1,101 +0,0 @@
-#!/usr/bin/perl
-
-use strict;
-use warnings;
-
-use lib '..';
-
-use Test::More tests => 3;
-use Test::MockModule;
-
-use PVE::VZDump;
-
-my $STATUS = qr/.*status.*/;
-my $NO_LOGFILE = qr/.*Could not open log file.*/;
-my $LOG_TOO_LONG = qr/.*Log output was too long.*/;
-my $TEST_FILE_PATH   = '/tmp/mail_test';
-my $TEST_FILE_WRONG_PATH = '/tmp/mail_test_wrong';
-
-sub prepare_mail_with_status {
-open(TEST_FILE, '>', $TEST_FILE_PATH); # Removes previous content
-print TEST_FILE "start of log file\n";
-print TEST_FILE "status: 0\% this should not be in the mail\n";
-print TEST_FILE "status: 55\% this should not be in the mail\n";
-print TEST_FILE "status: 100\% this should not be in the mail\n";
-print TEST_FILE "end of log file\n";
-close(TEST_FILE);
-}
-
-sub prepare_long_mail {
-open(TEST_FILE, '>', $TEST_FILE_PATH); # Removes previous content
-# 0.5 MB * 2 parts + the overview tables gives more than 1 MB mail
-print TEST_FILE "a" x (1024*1024);
-close(TEST_FILE);
-}
-
-my $result_text;
-my $result_properties;
-
-my $mock_notification_module = Test::MockModule->new('PVE::Notify');
-my $mocked_notify = sub {
-my ($severity, $title, $text, $properties, $metadata) = @_;
-
-$result_text = $text;
-$result_properties = $properties;
-};
-my $mocked_notify_short = sub {
-my (@params) = @_;
-return $mocked_notify->('', @params);
-};
-
-$mock_notification_module->mock(
-'notify' => $mocked_notify,
-'info' => $mocked_notify_short,
-'notice' => $mocked_notify_short,
-'warning' => $mocked_notify_short,
-'error' => $mocked_notify_short,
-);
-$mock_notification_module->mock('cfs_read_file', sub {
-my $path = shift;
-
-if ($path eq 'datacenter.cfg') {
-return {};
-} elsif ($path eq 'notifications.cfg' || $path eq 
'priv/notifications.cfg') {
-return '';
-} else {
-   die "unexpected cfs_read_file\n";
-}
-});
-
-my $MAILTO = ['test_addr...@proxmox.com'];
-my $SELF = {
-opts => { mailto => $MAILTO },
-cmdline => 'test_command_on_cli',
-};
-
-my $task = { state => 'ok', vmid => '100', };
-my $tasklist;
-sub prepare_test {
-$result_text = undef;
-$task->{tmplog} = shift;
-$tasklist = [ $task ];
-}
-
-{
-prepare_test($TEST_FILE_WRONG_PATH);
-PVE::VZDump::send_notification($SELF, $tasklist, 0, undef, undef, undef);
-like($result_properties->{logs}, $NO_LOGFILE, "Missing logfile is 
detected");
-}
-{
-prepare_test($TEST_FILE_PATH);
-prepare_mail_with_status();
-PVE::VZDump::send_notification($SELF, $tasklist, 0, undef, undef, undef);
-unlike($result_properties->{"status-text"}, $STATUS, "Status are not in 
text part of mails");
-}
-{
-prepare_test($TEST_FILE_PATH);
-prepare_long_mail();
-PVE::VZDump::send_notification($SELF, $tasklist, 0, undef, undef, undef);
-like($result_properties->{logs}, $LOG_TOO_LONG, "Text part of mails gets 
shortened");
-}
-unlink $TEST_FILE_PATH;
-- 
2.39.2



___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel



[pve-devel] [PATCH manager v3 6/8] gitignore: ignore any test artifacts

2024-05-21 Thread Lukas Wagner
Signed-off-by: Lukas Wagner 
---
 .gitignore | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/.gitignore b/.gitignore
index e8d1eb27..481ae1e0 100644
--- a/.gitignore
+++ b/.gitignore
@@ -9,3 +9,5 @@ dest/
 /www/mobile/pvemanager-mobile.js
 /www/touch/touch-[0-9]*/
 /pve-manager-[0-9]*/
+/test/.mocked_*
+/test/*.tmp
-- 
2.39.2



___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel



[pve-devel] [PATCH cluster v3 4/8] notify: use named template instead of passing template strings

2024-05-21 Thread Lukas Wagner
The notification system will now load template files from a defined
location. The template to use is now passed to proxmox_notify, instead
of separate template strings for subject/body.

Signed-off-by: Lukas Wagner 
---
 src/PVE/Notify.pm | 29 -
 1 file changed, 12 insertions(+), 17 deletions(-)

diff --git a/src/PVE/Notify.pm b/src/PVE/Notify.pm
index 872eb25..c514111 100644
--- a/src/PVE/Notify.pm
+++ b/src/PVE/Notify.pm
@@ -58,17 +58,16 @@ sub write_config {
 }
 
 my $send_notification = sub {
-my ($severity, $title, $message, $template_data, $fields, $config) = @_;
+my ($severity, $template_name, $template_data, $fields, $config) = @_;
 $config = read_config() if !defined($config);
-$config->send($severity, $title, $message, $template_data, $fields);
+$config->send($severity, $template_name, $template_data, $fields);
 };
 
 sub notify {
-my ($severity, $title, $message, $template_data, $fields, $config) = @_;
+my ($severity, $template_name, $template_data, $fields, $config) = @_;
 $send_notification->(
 $severity,
-$title,
-$message,
+$template_name,
 $template_data,
 $fields,
 $config
@@ -76,11 +75,10 @@ sub notify {
 }
 
 sub info {
-my ($title, $message, $template_data, $fields, $config) = @_;
+my ($template_name, $template_data, $fields, $config) = @_;
 $send_notification->(
 'info',
-$title,
-$message,
+$template_name,
 $template_data,
 $fields,
 $config
@@ -88,11 +86,10 @@ sub info {
 }
 
 sub notice {
-my ($title, $message, $template_data, $fields, $config) = @_;
+my ($template_name, $template_data, $fields, $config) = @_;
 $send_notification->(
 'notice',
-$title,
-$message,
+$template_name,
 $template_data,
 $fields,
 $config
@@ -100,11 +97,10 @@ sub notice {
 }
 
 sub warning {
-my ($title, $message, $template_data, $fields, $config) = @_;
+my ($template_name, $template_data, $fields, $config) = @_;
 $send_notification->(
 'warning',
-$title,
-$message,
+$template_name,
 $template_data,
 $fields,
 $config
@@ -112,11 +108,10 @@ sub warning {
 }
 
 sub error {
-my ($title, $message, $template_data, $fields, $config) = @_;
+my ($template_name, $template_data, $fields, $config) = @_;
 $send_notification->(
 'error',
-$title,
-$message,
+$template_name,
 $template_data,
 $fields,
 $config
-- 
2.39.2



___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel



[pve-devel] [PATCH proxmox-perl-rs v3 1/8] notify: use file based notification templates

2024-05-21 Thread Lukas Wagner
Instead of passing literal template strings to the notification
system, we now only pass an identifier. This identifier will be used
load the template files from a product-specific directory.

Signed-off-by: Lukas Wagner 
---
 common/src/notify.rs | 8 +++-
 1 file changed, 3 insertions(+), 5 deletions(-)

diff --git a/common/src/notify.rs b/common/src/notify.rs
index 8f9f38f..d965417 100644
--- a/common/src/notify.rs
+++ b/common/src/notify.rs
@@ -94,16 +94,14 @@ mod export {
 fn send(
 #[try_from_ref] this: ,
 severity: Severity,
-title: String,
-body: String,
+template_name: String,
 template_data: Option,
 fields: Option>,
 ) -> Result<(), HttpError> {
 let config = this.config.lock().unwrap();
-let notification = Notification::new_templated(
+let notification = Notification::from_template(
 severity,
-title,
-body,
+template_name,
 template_data.unwrap_or_default(),
 fields.unwrap_or_default(),
 );
-- 
2.39.2



___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel



[pve-devel] [PATCH docs] network: override device names: suggest running update-initramfs

2024-05-21 Thread Friedrich Weber
The initramfs-tools hook /usr/share/initramfs-tools/hooks/udev copies
link files from /etc/systemd/network to the initramfs, where they take
effect in early userspace. If the link files in the initramfs diverge
from the link files in the rootfs, this can lead to confusing
behavior, as reported in enterprise support. For instance:

- If an interface matches link files both in the initramfs and the
  rootfs, it will be renamed twice during boot.
- A leftover link file in the initramfs renaming an interface A to a
  new name X may prevent a link file in the rootfs from renaming a
  different interface B to the same name X (it will fail with "File
  exists").

To avoid this confusion, mention the link files are copied to the
initramfs, and suggest updating the initramfs after making changes to
the link files.

Suggested-by: Hannes Laimer 
Signed-off-by: Friedrich Weber 
---
 pve-network.adoc | 12 ++--
 1 file changed, 10 insertions(+), 2 deletions(-)

diff --git a/pve-network.adoc b/pve-network.adoc
index be8d63c..acdcf39 100644
--- a/pve-network.adoc
+++ b/pve-network.adoc
@@ -184,6 +184,13 @@ manpage] for more details.
 
 Then, you can assign a name using the `Name` option in the `[Link]` section.
 
+Link files are copied to the `initramfs`, so it is recommended to refresh the
+`initramfs` after adding, modifying, or removing a link file:
+
+
+# update-initramfs -u -k all
+
+
 For example, to assign the name `enwan0` to the Ethernet device with MAC
 address `aa:bb:cc:dd:ee:ff`, create a file
 `/etc/systemd/network/10-enwan0.link` with the following contents:
@@ -197,8 +204,9 @@ Type=ether
 Name=enwan0
 
 
-Do not forget to adjust `/etc/network/interfaces` to use the new name.
-You need to reboot the node for the change to take effect.
+Do not forget to adjust `/etc/network/interfaces` to use the new name, and
+refresh your `initramfs` as described above. You need to reboot the node for
+the change to take effect.
 
 NOTE: It is recommended to assign a name starting with `en` or `eth` so that
 {pve} recognizes the interface as a physical network device which can then be
-- 
2.39.2



___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel



Re: [pve-devel] [PATCH qemu-server v10 1/4] add C program to get hardware capabilities from CPUID

2024-05-21 Thread Thomas Lamprecht
Am 17/05/2024 um 13:21 schrieb Dominik Csapak:
> one small nit inline:
> 
> On 5/10/24 13:47, Markus Frank wrote:
>> diff --git a/query-machine-capabilities/Makefile 
>> b/query-machine-capabilities/Makefile
>> new file mode 100644
>> index 000..c5f6348
>> --- /dev/null
>> +++ b/query-machine-capabilities/Makefile
>> @@ -0,0 +1,21 @@
>> +DESTDIR=
>> +PREFIX=/usr
>> +SBINDIR=${PREFIX}/libexec/qemu-server
>> +SERVICEDIR=/lib/systemd/system
>> +
> 
> PREFIX is only used once here, so it's probably better inlining the value

No, having the PREFIX variable separate is a common pattern
that allows customizing installation.
Even if we do not need that ourselves, it's still not costing us really
anything to keep following that here and could make comparing changes
between two packages with binaries installed in different paths easier.
So while I wouldn't go through all our build systems and introduce this
variable if missing, I'd also not recommend developers to drop it, as it
is not better to do so.


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel



Re: [pve-devel] [PATCH cluster 0/2] fix #5461: qdevice cluster mgmt

2024-05-21 Thread Aaron Lauterer
v2 is available 
https://lists.proxmox.com/pipermail/pve-devel/2024-May/063878.html


On  2024-05-15  12:32, Aaron Lauterer wrote:

on clusters where the ssh known hosts info is only stored in the pmxcfs,
it broke the mgmt of the qdevice. By adding the necessary ssh options on
each intra cluster ssh call, we can fix this.

The second patch is optional and adjusts the touched code to fit better
into the max line lenghts.
I put it into a separate patch as I wasn't 100% sure how to format it,
   especially the first instance in that patch.


Aaron Lauterer (2):
   fix #5461: pvecm: ssh: adapt intra cluster ssh options
   pvecm: qdevice: adjust line lengths

  src/PVE/CLI/pvecm.pm | 36 
  1 file changed, 24 insertions(+), 12 deletions(-)




___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel