[pve-devel] applied: [PATCH manager 1/2] pvereport: dir2text: ignore special . and .. files

2023-11-02 Thread Thomas Lamprecht
On 03/10/2023 13:36, Aaron Lauterer wrote:
> So far this hasn't been an issue as each user of dir2text wanted files
> with a specific pattern. But if we want every file in the directory, we
> need to skip the special files '.' and '..'.
> 
> Signed-off-by: Aaron Lauterer 
> ---
>  PVE/Report.pm | 1 +
>  1 file changed, 1 insertion(+)
> 
>

applied, thanks!

I made an (only tangentially) related follow-up, adding a short hint
about what should get output to start of $text in dir2text, as it
seems IMO possible helpful when to get a bit more active hint that
there are no files when combing through logs.


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel



[pve-devel] aüüöoed: [PATCH docs] correct outdated info about gui header

2023-11-02 Thread Thomas Lamprecht
On 02/11/2023 16:08, Dominik Csapak wrote:
> the header part of the gui did change, by moving the user name into a
> button, which now contains more user specific actions. Update it to be
> correct again.
> 

applied, with some language follow-ups, thanks — keeping such things
up-to-date is important work!

> Signed-off-by: Dominik Csapak 
> ---
>  pve-gui.adoc | 18 ++
>  1 file changed, 10 insertions(+), 8 deletions(-)
> 
> diff --git a/pve-gui.adoc b/pve-gui.adoc
> index 9f63a7e..046fb9f 100644
> --- a/pve-gui.adoc
> +++ b/pve-gui.adoc
> @@ -101,21 +101,23 @@ search bar nearside you can search for specific objects 
> (VMs,
>  containers, nodes, ...). This is sometimes faster than selecting an
>  object in the resource tree.
>  
> -To the right of the search bar we see the identity (login name). The
> -gear symbol is a button opening the 'My Settings' dialog. There you
> -can customize some client side user interface setting (reset the saved
> -login name, reset saved layout).
> -
> -The rightmost part of the header contains four buttons:
> +The right part of the header contains four buttons:
>  
>  [horizontal]
> -Help :: Opens a new browser window showing the reference documentation.
> +Documentation :: Opens a new browser window showing the reference 
> documentation.
>  
>  Create VM :: Opens the virtual machine creation wizard.
>  
>  Create CT :: Open the container creation wizard.
>  
> -Logout :: Logout, and show the login dialog again.
> +Identity Menu :: Displays the identity and contains user specific 
> options.

IMO that name is rather odd.. changed to "User Menu"

I connected the following paragraph with the above definition-list entry
by placing a "+" on the empty line, that way they are separate paragraphs
but still indented on the same level, which IMO makes sense for the menu
child entries.

> +
> +The identity menu contains various user specific options, such as the 'My

"various user specific options" is a repetition from the list above, sticks
especially out because it's already a bit superfluous there too, i.e., what
else would the "user menu" contain? ^^

> +Settings' dialog. There you can customize some client side user interface
> +setting (reset the saved login name, reset saved layout).

I reworded to avoid the parenthesis, but they were mostly pre-exsiting,
so just mentioning for the record.

> +
> +It also contains a shortcut to the 'TFA', 'Language' and 'Color Theme'
> +settings, as well as the Logout option.
>  
>  
>  [[gui_my_settings]]



___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] applied: [PATCH zfsonlinux] add patch for spurious warning on `zfs mount -a`

2023-11-02 Thread Thomas Lamprecht
On 02/11/2023 18:33, Stoiko Ivanov wrote:
> reported in our community forum:
> https://forum.proxmox.com/threads/.135635/post-60036

The dot isn't required, but it seems the anchor # really is,
your link goes to an ancient post without that, switched it
to:

https://forum.proxmox.com/threads/135635/#post-60036

> 
> the small fix was merged upstream:
> https://github.com/openzfs/zfs/pull/15468
> 
> minimally tested by building with this patch and running
> `zfs mount -a` on an affected system.
> 
> Signed-off-by: Stoiko Ivanov 
> ---
> this patch fixes a cosmetic issue, but might help keep support requests down
> a bit.
> also quickly skimmed through the other patches in upstream/master and
> salsa.debian.org - but currently don't think anything needs to be pulled in
> urgently
> 
>  ...runcate_shares-without-etc-exports.d.patch | 76 +++
>  debian/patches/series |  1 +
>  2 files changed, 77 insertions(+)
>  create mode 100644 
> debian/patches/0012-Fix-nfs_truncate_shares-without-etc-exports.d.patch
> 
>

applied, thanks!


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel



[pve-devel] applied: [PATCH zfsonlinux] add patch for spurious warning on `zfs mount -a`

2023-11-02 Thread Thomas Lamprecht
On 02/11/2023 18:33, Stoiko Ivanov wrote:
> reported in our community forum:
> https://forum.proxmox.com/threads/.135635/post-60036
> 
> the small fix was merged upstream:
> https://github.com/openzfs/zfs/pull/15468
> 
> minimally tested by building with this patch and running
> `zfs mount -a` on an affected system.
> 
> Signed-off-by: Stoiko Ivanov 
> ---
> this patch fixes a cosmetic issue, but might help keep support requests down
> a bit.
> also quickly skimmed through the other patches in upstream/master and
> salsa.debian.org - but currently don't think anything needs to be pulled in
> urgently
> 
>  ...runcate_shares-without-etc-exports.d.patch | 76 +++
>  debian/patches/series |  1 +
>  2 files changed, 77 insertions(+)
>  create mode 100644 
> debian/patches/0012-Fix-nfs_truncate_shares-without-etc-exports.d.patch
> 
>

applied, thanks!


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel



[pve-devel] applied-series: [PATCH 00/12] installer: add crate for common code

2023-11-02 Thread Thomas Lamprecht
On 25/10/2023 17:59, Aaron Lauterer wrote:
> since work on the auto installer is happenning in parallel, now would be
> a good point to move commonly used code into its own crate. Otherwise
> the auto-installer will always have to play catch up with the ongoing
> development of the tui installer.
> 
> I tried to split up the commits as much as possible, but there are two
> larger ones, copying most the code over to the new repo and making the
> switch. The former because it is difficult to pull apart the parts that
> belong together. The latter needed to be this big as most local
> occurences needed to be removed at the same time to avoid dependency
> conflicts.
> 
> One last things that is missing, is the "InstallConfig" struct.
> This should also be part of the common crate, but I need to look further
> into how to make it possible that it can be created from structs of each
> crate (tui, auto) as implementing a ::From can only be done within the
> crate where the struct lives IIUC.
> 
> This series depends on the patches by Christoph to remove the global
> unsafe setup info, version 2 [0]. Without those patches applied first,
> this series will not apply.
> 
> [0] https://lists.proxmox.com/pipermail/pve-devel/2023-October/059628.html
> 
> Aaron Lauterer (12):
>   add proxmox-installer-common crate
>   common: copy common code from tui-installer
>   common: utils: add dependency for doc test
>   common: make InstallZfsOption public
>   common: disk_checks: make functions public
>   tui-installer: add dependency for new common crate
>   tui: switch to common crate
>   tui: remove now unused utils.rs
>   common: add installer_setup method
>   common: document installer_setup method
>   tui: use installer_setup from common cate
>   tui: remove unused read_json function

applied series, with Christoph's build-sys fix squashed into your first
patch, and his R-b added, thanks!


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel



[pve-devel] [PATCH zfsonlinux] add patch for spurious warning on `zfs mount -a`

2023-11-02 Thread Stoiko Ivanov
reported in our community forum:
https://forum.proxmox.com/threads/.135635/post-60036

the small fix was merged upstream:
https://github.com/openzfs/zfs/pull/15468

minimally tested by building with this patch and running
`zfs mount -a` on an affected system.

Signed-off-by: Stoiko Ivanov 
---
this patch fixes a cosmetic issue, but might help keep support requests down
a bit.
also quickly skimmed through the other patches in upstream/master and
salsa.debian.org - but currently don't think anything needs to be pulled in
urgently

 ...runcate_shares-without-etc-exports.d.patch | 76 +++
 debian/patches/series |  1 +
 2 files changed, 77 insertions(+)
 create mode 100644 
debian/patches/0012-Fix-nfs_truncate_shares-without-etc-exports.d.patch

diff --git 
a/debian/patches/0012-Fix-nfs_truncate_shares-without-etc-exports.d.patch 
b/debian/patches/0012-Fix-nfs_truncate_shares-without-etc-exports.d.patch
new file mode 100644
index ..7eb9721d
--- /dev/null
+++ b/debian/patches/0012-Fix-nfs_truncate_shares-without-etc-exports.d.patch
@@ -0,0 +1,76 @@
+From  Mon Sep 17 00:00:00 2001
+From: siv0 
+Date: Tue, 31 Oct 2023 21:57:54 +0100
+Subject: [PATCH] Fix nfs_truncate_shares without /etc/exports.d
+
+Calling nfs_reset_shares on Linux prints a warning:
+`failed to lock /etc/exports.d/zfs.exports.lock: No such file or
+directory`
+when /etc/exports.d does not exist. The directory gets created, when a
+filesystem is actually exported through nfs_toggle_share and
+nfs_init_share. The truncation of /etc/exports.d/zfs.exports happens
+unconditionally when calling `zfs mount -a` (via zfs_do_mount and
+share_mount in `cmd/zfs/zfs_main.c`).
+
+Fixing the issue only in the Linux part, since the exports file on
+freebsd is in `/etc/zfs/`, which seems present on 2 FreeBSD systems I
+have access to (through `/etc/zfs/compatibility.d/`), while a Debian
+box does not have the directory even if `/usr/sbin/exportfs` is
+present through the `nfs-kernel-server` package.
+
+The code for exports_available is copied from nfs_available above.
+
+Fixes: ede037cda73675f42b1452187e8dd3438fafc220
+("Make zfs-share service resilient to stale exports")
+
+Reviewed-by: Brian Atkinson 
+Reviewed-by: Brian Behlendorf 
+Signed-off-by: Stoiko Ivanov 
+Closes #15369
+Closes #15468
+(cherry picked from commit 41e55b476bcfc90f1ad81c02c5375367fdace9e9)
+Signed-off-by: Stoiko Ivanov 
+---
+ lib/libshare/os/linux/nfs.c | 18 ++
+ 1 file changed, 18 insertions(+)
+
+diff --git a/lib/libshare/os/linux/nfs.c b/lib/libshare/os/linux/nfs.c
+index 004946b0c..3dce81840 100644
+--- a/lib/libshare/os/linux/nfs.c
 b/lib/libshare/os/linux/nfs.c
+@@ -47,6 +47,7 @@
+ 
+ 
+ static boolean_t nfs_available(void);
++static boolean_t exports_available(void);
+ 
+ typedef int (*nfs_shareopt_callback_t)(const char *opt, const char *value,
+ void *cookie);
+@@ -539,6 +540,8 @@ nfs_commit_shares(void)
+ static void
+ nfs_truncate_shares(void)
+ {
++  if (!exports_available())
++  return;
+   nfs_reset_shares(ZFS_EXPORTS_LOCK, ZFS_EXPORTS_FILE);
+ }
+ 
+@@ -566,3 +569,18 @@ nfs_available(void)
+ 
+   return (avail == 1);
+ }
++
++static boolean_t
++exports_available(void)
++{
++  static int avail;
++
++  if (!avail) {
++  if (access(ZFS_EXPORTS_DIR, F_OK) != 0)
++  avail = -1;
++  else
++  avail = 1;
++  }
++
++  return (avail == 1);
++}
diff --git a/debian/patches/series b/debian/patches/series
index 710cbfbe..6a5ab10f 100644
--- a/debian/patches/series
+++ b/debian/patches/series
@@ -9,3 +9,4 @@
 0009-arc-stat-summary-guard-access-to-l2arc-MFU-MRU-stats.patch
 0010-zvol-Remove-broken-blk-mq-optimization.patch
 0011-Revert-zvol-Temporally-disable-blk-mq.patch
+0012-Fix-nfs_truncate_shares-without-etc-exports.d.patch
-- 
2.39.2



___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel



Re: [pve-devel] [PATCH v2 container 1/1] Add device passthrough

2023-11-02 Thread Filip Schauer



On 30/10/2023 14:34, Wolfgang Bumiller wrote:

On Tue, Oct 24, 2023 at 02:55:53PM +0200, Filip Schauer wrote:

Add a dev[n] argument to the container config to pass devices through to
a container. A device can be passed by its path. Alternatively a mapped
USB device can be passed through with usbmapping=.

Signed-off-by: Filip Schauer
---
  src/PVE/LXC.pm| 34 +++-
  src/PVE/LXC/Config.pm | 60 +++
  2 files changed, 93 insertions(+), 1 deletion(-)

diff --git a/src/PVE/LXC.pm b/src/PVE/LXC.pm
index c9b5ba7..a3ddb62 100644
--- a/src/PVE/LXC.pm
+++ b/src/PVE/LXC.pm
@@ -5,7 +5,8 @@ use warnings;
  
  use Cwd qw();

  use Errno qw(ELOOP ENOTDIR EROFS ECONNREFUSED EEXIST);
-use Fcntl qw(O_RDONLY O_WRONLY O_NOFOLLOW O_DIRECTORY);
+use Fcntl qw(O_RDONLY O_WRONLY O_NOFOLLOW O_DIRECTORY :mode);
+use File::Basename;
  use File::Path;
  use File::Spec;
  use IO::Poll qw(POLLIN POLLHUP);
@@ -639,6 +640,37 @@ sub update_lxc_config {
$raw .= "lxc.mount.auto = sys:mixed\n";
  }
  
+# Clear passthrough directory from previous run

+my $passthrough_dir = "/var/lib/lxc/$vmid/passthrough";
+File::Path::rmtree($passthrough_dir);

I think we need to make a few changes here.

First: we don't necessarily need this directory.
Having a device list would certainly be nice, but it makes more sense to
just have a file we can easily parse (possibly even just a json hash),
like the `devices` file we already create in the pre-start hook, except
prepared *for* the pre-start hook, which *should* be able to just
`mknod` the devices right into the container's `/dev` on startup.



Devices mknoded into the container's /dev directory in the pre-start
hook will not be visible in the container once it is fully started.
Meanwhile mknoding a device to a different path inside the container
works fine. It seems that LXC mounts over the /dev directory. This can
be solved by calling mknod in lxc-pve-autodev-hook, but this does not
work with unprivileged containers without the mknod capability.

So are bind mounts our only option without modifying LXC,
or am I overlooking something?



We'd also avoid "lingering" device nodes with potentially harmful
uid/permissions in /var, which is certainly better from a security POV.

But note that we do need the `lxc.cgroup2.*` entries before starting the
container in order to ensure the devices cgroup has the right
permissions.


+
+PVE::LXC::Config->foreach_passthrough_device($conf, sub {
+   my ($key, $sanitized_path) = @_;
+
+   my $absolute_path = "/$sanitized_path";
+   my ($mode, $rdev) = (stat($absolute_path))[2, 6];
+   die "Could not find major and minor ids of device $absolute_path.\n"
+   unless ($mode && $rdev);
+
+   my $major = PVE::Tools::dev_t_major($rdev);
+   my $minor = PVE::Tools::dev_t_minor($rdev);
+   my $device_type_char = S_ISBLK($mode) ? 'b' : 'c';
+   my $passthrough_device_path = "$passthrough_dir/$sanitized_path";
+   File::Path::make_path(dirname($passthrough_device_path));
+   PVE::Tools::run_command([
+   '/usr/bin/mknod',
+   '-m', '0660',
+   $passthrough_device_path,
+   $device_type_char,
+   $major,
+   $minor
+   ]);

It's probably worth adding a helper for the mknod syscall to
`PVE::Tools`, there are a bunch of syscalls in there already.


+   chown 10, 10, $passthrough_device_path if ($unprivileged);

^ This isn't necessarily the correct id. Users may have custom id
mappings.
`PVE::LXC::parse_id_maps($conf)` returns the mapping alongside the root
uid and gid. (See for example `sub mount_all` for how it's used.


+
+   $raw .= "lxc.cgroup2.devices.allow = $device_type_char $major:$minor 
rw\n";
+   $raw .= "lxc.mount.entry = $passthrough_device_path $sanitized_path none 
bind,create=file\n";
+});
+
  # WARNING: DO NOT REMOVE this without making sure that loop device nodes
  # cannot be exposed to the container with r/w access (cgroup perms).
  # When this is enabled mounts will still remain in the monitor's namespace
diff --git a/src/PVE/LXC/Config.pm b/src/PVE/LXC/Config.pm
index 56e1f10..edd813e 100644
--- a/src/PVE/LXC/Config.pm
+++ b/src/PVE/LXC/Config.pm
@@ -29,6 +29,7 @@ mkdir $lockdir;
  mkdir "/etc/pve/nodes/$nodename/lxc";
  my $MAX_MOUNT_POINTS = 256;
  my $MAX_UNUSED_DISKS = $MAX_MOUNT_POINTS;
+my $MAX_DEVICES = 256;
  
  # BEGIN implemented abstract methods from PVE::AbstractConfig
  
@@ -908,6 +909,49 @@ for (my $i = 0; $i < $MAX_UNUSED_DISKS; $i++) {

  }
  }
  
+PVE::JSONSchema::register_format('pve-lxc-dev-string', \&verify_lxc_dev_string);

+sub verify_lxc_dev_string {
+my ($dev, $noerr) = @_;
+
+if (
+   $dev =~m@/\.\.?/@  ||
+   $dev =~m@/\.\.?$@  ||
+   $dev !~ m!^/dev/!
+) {
+   return undef if $noerr;
+   die "$dev is not a valid device path\n";
+}
+
+return $dev;
+}
+
+my $dev_d

[pve-devel] [PATCH docs] correct outdated info about gui header

2023-11-02 Thread Dominik Csapak
the header part of the gui did change, by moving the user name into a
button, which now contains more user specific actions. Update it to be
correct again.

Signed-off-by: Dominik Csapak 
---
 pve-gui.adoc | 18 ++
 1 file changed, 10 insertions(+), 8 deletions(-)

diff --git a/pve-gui.adoc b/pve-gui.adoc
index 9f63a7e..046fb9f 100644
--- a/pve-gui.adoc
+++ b/pve-gui.adoc
@@ -101,21 +101,23 @@ search bar nearside you can search for specific objects 
(VMs,
 containers, nodes, ...). This is sometimes faster than selecting an
 object in the resource tree.
 
-To the right of the search bar we see the identity (login name). The
-gear symbol is a button opening the 'My Settings' dialog. There you
-can customize some client side user interface setting (reset the saved
-login name, reset saved layout).
-
-The rightmost part of the header contains four buttons:
+The right part of the header contains four buttons:
 
 [horizontal]
-Help :: Opens a new browser window showing the reference documentation.
+Documentation :: Opens a new browser window showing the reference 
documentation.
 
 Create VM :: Opens the virtual machine creation wizard.
 
 Create CT :: Open the container creation wizard.
 
-Logout :: Logout, and show the login dialog again.
+Identity Menu :: Displays the identity and contains user specific options.
+
+The identity menu contains various user specific options, such as the 'My
+Settings' dialog. There you can customize some client side user interface
+setting (reset the saved login name, reset saved layout).
+
+It also contains a shortcut to the 'TFA', 'Language' and 'Color Theme'
+settings, as well as the Logout option.
 
 
 [[gui_my_settings]]
-- 
2.39.2


___
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel