Re: [pve-devel] [RFC qemu/storage/qemu-server/container/manager 00/23] backup provider API

2024-07-28 Thread Dietmar Maurer
> In hyper-converged deployments, the node performing the backup is sourcing > ((nodes-1)/(nodes))*bytes) of backup data (i.e., ingress traffic) and then > sending 1*bytes to PBS (i.e., egress traffic). If PBS were to pull the data > from the nodes directly, the maximum load on any one host

Re: [pve-devel] [RFC qemu/storage/qemu-server/container/manager 00/23] backup provider API

2024-07-28 Thread Dietmar Maurer
> The biggest issue we see reported related to QEMU bitmaps is > persistence. The lack of durability results in unpredictable backup > behavior at scale. If a host, rack, or data center loses power, you're > in for a full backup cycle. Even if several VMs are powered off for > some reason, it

Re: [pve-devel] [RFC qemu/storage/qemu-server/container/manager 00/23] backup provider API

2024-07-28 Thread Dietmar Maurer
> Today, I believe the client is reading the data and pushing it to > PBS. In the case of CEPH, wouldn't this involve sourcing data from > multiple nodes and then sending it to PBS? Wouldn't it be more > efficient for PBS to read it directly from storage? In the case of > centralized storage, we'd

Re: [pve-devel] [RFC qemu/storage/qemu-server/container/manager 00/23] backup provider API

2024-07-27 Thread Dietmar Maurer
> Would adding support for offloading incremental difference detection > to the underlying storage be feasible with the API updates? The QEMU > bitmap strategy works for all storage devices but is far from > optimal. Sorry, but why do you think this is far from optimal?

[pve-devel] [PATCH pve-xtermjs v3] termproxy: allow to use unix sockets for auth requests

2024-07-24 Thread Dietmar Maurer
Remove ureq, because it does not support unix sockets. Signed-off-by: Dietmar Maurer --- Changes sinve v2: split out the command line help text change into patch: [PATCH pve-xtermjs] termproxy: fix the command line help text Changes since v1: - use extra --authsocket cli option - use

[pve-devel] [PATCH pve-xtermjs] termproxy: fix the command line help text

2024-07-24 Thread Dietmar Maurer
The need to be the first argument. Signed-off-by: Dietmar Maurer --- termproxy/src/cli.rs | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/termproxy/src/cli.rs b/termproxy/src/cli.rs index cc44655..adfd830 100644 --- a/termproxy/src/cli.rs +++ b/termproxy/src/cli.rs @@ -4,7

[pve-devel] [PATCH pve-xtermjs v2] termproxy: allow to use unix sockets for auth requests

2024-07-24 Thread Dietmar Maurer
Remove ureq, because it does not support unix sockets. Signed-off-by: Dietmar Maurer --- Changes since v1: - use extra --authsocket cli option - use single format!() instead of multiple push_str() - cleanup variable names termproxy/Cargo.toml | 2 +- termproxy/src/cli.rs | 26

[pve-devel] [PATCH pve-xtermjs] termproxy: allow to use unix sockets for auth requests

2024-07-23 Thread Dietmar Maurer
Remove ureq, because it does not support unix sockets. Signed-off-by: Dietmar Maurer --- termproxy/Cargo.toml | 2 +- termproxy/src/cli.rs | 29 + termproxy/src/main.rs | 59 +-- 3 files changed, 71 insertions(+), 19 deletions

[pve-devel] [PATCH proxmox] proxmox-acme: derive PartialEq for API types

2024-03-07 Thread Dietmar Maurer
Signed-off-by: Dietmar Maurer --- proxmox-acme/src/account.rs | 2 +- proxmox-acme/src/eab.rs | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/proxmox-acme/src/account.rs b/proxmox-acme/src/account.rs index e244c09..7f00143 100644 --- a/proxmox-acme/src/account.rs +++ b

[pve-devel] [PATCH proxmox] proxmox-acme: add api-types feature

2024-03-07 Thread Dietmar Maurer
Because AccountData is exposed via our API (currently as type Object). Signed-off-by: Dietmar Maurer --- proxmox-acme/Cargo.toml | 3 +++ proxmox-acme/src/account.rs | 7 +++ proxmox-acme/src/eab.rs | 5 + 3 files changed, 15 insertions(+) diff --git a/proxmox-acme/Cargo.toml b

Re: [pve-devel] [PATCH storage 1/1] storage/plugins: pass scfg to parse_volname

2024-03-01 Thread Dietmar Maurer
> On 29.2.2024 16:09 CET Roland Kammerer via pve-devel > wrote: > All in all, yes, this is specific for our use case, otherwise > parse_volname would already have that additional parameter as all the > other plugin functions, but I don't see where this would hurt existing > code, and it

Re: [pve-devel] [PATCH manager] ui: node: system: avoid using 'Syslog'

2024-02-16 Thread Dietmar Maurer
> while I don't mind (at all!) that that part of the UI/API is labelled syslog > (I don't think it's hard to understand that it gives you the system logs of > that node, and "syslog" is a bit like "Kleenex" in that regard ;)) - I do > have to disagree here ;) journald does a lot more than just

Re: [pve-devel] [PATCH manager] ui: node: system: avoid using 'Syslog'

2024-02-15 Thread Dietmar Maurer
I can also imaging using "Events" instead of "Syslog". - title: 'Syslog', + title: gettext('Events'), IMHO this is easier to translate. > With your change: > > - title: 'Syslog', > + title: gettext('System Log'), > > we now need to

Re: [pve-devel] [PATCH manager] ui: node: system: avoid using 'Syslog'

2024-02-15 Thread Dietmar Maurer
> The information gathered by the API call comes from the systemd > journal. While 'Syslog' could be interpreted as a shorthand for > "System Log", it's better to be explicit to avoid any confusion. > - title: 'Syslog', > + title: gettext('System Log'), From Wikipedia:

Re: [pve-devel] [RFC docs 13/13] vzdump: add section about backup fleecing

2024-01-25 Thread Dietmar Maurer
> >>Stupid question: Wouldn't It be much easier to add a simple IO-buffer > >>with limited capacity, implemented inside the RUST backup code? > > At work, we are running a backup cluster on remote location with hdd , > and a production cluster with super fast nvme, > and sometimes I have really

Re: [pve-devel] [RFC docs 13/13] vzdump: add section about backup fleecing

2024-01-25 Thread Dietmar Maurer
Stupid question: Wouldn't It be much easier to add a simple IO-buffer with limited capacity, implemented inside the RUST backup code? > +WARNING: Theoretically, the fleecing image can grow to the same size as the > +original image, e.g. if the guest re-writes a whole disk while the backup is >

Re: [pve-devel] [PATCH widget-toolkit] i18n: mark strings as translatable

2023-12-06 Thread Dietmar Maurer
> Do note the following examples: > > #: pve-manager/www/manager6/qemu/IPConfigEdit.js:97 > #: pve-manager/www/manager6/qemu/IPConfigEdit.js:155 > msgid "DHCP" > msgstr "بروتوكول التهيئة الآلية للمضيفين (DHCP)" > > #: pve-manager/www/manager6/qemu/IPConfigEdit.js:163 >

Re: [pve-devel] [PATCH manager] ui: mark strings translatable

2023-11-29 Thread Dietmar Maurer
; Am 28/11/2023 um 15:46 schrieb Dietmar Maurer: > > To be more clear, I would use: > > > > proxmox.Utils.defaultText + ' (' + gettext('Free') + ')' ___ pve-devel mailing list pve-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH manager] ui: mark strings translatable

2023-11-28 Thread Dietmar Maurer
> I'm taking on a lot of contributions to translations and the common complaint > I hear is that not all can be translated correctly due to such tricks (or just > missing gettext), most translators care much more about a correct translation > than some over-optimized ones than then break depending

Re: [pve-devel] [PATCH manager] ui: mark strings translatable

2023-11-28 Thread Dietmar Maurer
To be more clear, I would use: proxmox.Utils.defaultText + ' (' + gettext('Free') + ')' > On 28.11.2023 15:39 CET Dietmar Maurer wrote: > > > > The string `proxmox.Utils.defaultText + ' (free)'` was inlined as > > `Default (Free)` cutting translatable strings make

Re: [pve-devel] [PATCH manager] ui: mark strings translatable

2023-11-28 Thread Dietmar Maurer
> The string `proxmox.Utils.defaultText + ' (free)'` was inlined as > `Default (Free)` cutting translatable strings makes them harder or even > impossible to translate in certain languages. Well, this also duplicates the number of things to translate! So what languages are the problem exactly?

Re: [pve-devel] [PATCH widget-toolkit] node: repos: Use won't rather than don't

2023-11-24 Thread Dietmar Maurer
Do you also plan to fix those typos in the translation files? Else we need to re-tralstale them for all languages! > On 24.11.2023 15:58 CET Maximiliano Sandoval wrote: > > > It would be preferable to use "won't" but I would rather err on the safe > side when it comes to escapes in gettext. >

Re: [pve-devel] [PATCH v4 many 00/11] notifications: add SMTP endpoint

2023-11-09 Thread Dietmar Maurer
> > As a compromise, maybe we could just add a note to the docs > > that discusses the reliability aspects of 'sendmail' vs 'smtp' > > endpoints? > > > > Sure, for now adding a general hint to the documentation that they are > send one-shot only would be good. Ok for me.

Re: [pve-devel] [PATCH v4 many 00/11] notifications: add SMTP endpoint

2023-11-09 Thread Dietmar Maurer
> On 11/8/23 16:52, Dietmar Maurer wrote: > >> This patch series adds support for a new notification endpoint type, > >> smtp. As the name suggests, this new endpoint allows PVE to talk > >> to SMTP server directly, without using the system's MTA (postfix). > >

Re: [pve-devel] [PATCH v4 many 00/11] notifications: add SMTP endpoint

2023-11-08 Thread Dietmar Maurer
> This patch series adds support for a new notification endpoint type, > smtp. As the name suggests, this new endpoint allows PVE to talk > to SMTP server directly, without using the system's MTA (postfix). Isn't this totally unreliable? What if the server responds with a temporary error code?

Re: [pve-devel] [PATCH pve-ceph] fix compatibility with CPUs not supporting SSE 4.1 instructions

2023-09-18 Thread Dietmar Maurer
> One of our users ran into issues with running Ceph on older CPU > architectures [1]. This is apparently due to a bug in gcc-12 that > leads to SSE 4.1 instructions always being executed rather than > dynamically dispatching functions using those instructions. Cant we fix the GCC bug instead?

Re: [pve-devel] appllied: [PATCH] read firstline: only map ENOENT to undef, raise error otherwise

2023-06-12 Thread Dietmar Maurer
> One could argue that the case for not existent should return undef, > while an empty file should return an empty string, but for that we > might want to check all use-sites first. AFAIR I use this function many times assuming that it does not throw errors in case of empty files. That is quite

Re: [pve-devel] qemu 7.0 : fleecing backup (aka: local temp write cache"

2022-08-01 Thread Dietmar Maurer
> Can I use a small local fast PBS instance without need to keep the full > datastore chunks ? > > I have 300TB nvme in production, I don't want to buy 300TB nvme for backup. no, sorry. Do you want to use Ceph as temp backup storage, or simply an additional (node local) nvme?

Re: [pve-devel] qemu 7.0 : fleecing backup (aka: local temp write cache"

2022-07-31 Thread Dietmar Maurer
> This is really a blocker for me,I can't use pbs because I'm using nvme > is production, and a 7200k hdd backup in a remote 200km site with 5ms > latency. Why don't you use a local(fast) PBS instance, then sync to the slow remote? ___ pve-devel

Re: [pve-devel] vncpropxy question

2022-07-01 Thread Dietmar Maurer
> addendum: > > 'it doesn't do anything here' is not completely correct > for 'regular' vm displays it just does not set the ticket which > breaks the connection I think this ("break the connection") is important, because otherwise it would allow unecrypted VNC traffic over the network. I guess

[pve-devel] vncpropxy question

2022-06-30 Thread Dietmar Maurer
in qemu-server, I wonder why we set $ENV{LC_PVE_TICKET} conditionally? Does not make any sense to me, because it make all other connection failing... diff --git a/PVE/API2/Qemu.pm b/PVE/API2/Qemu.pm index 99b426e..c6a3ac1 100644 --- a/PVE/API2/Qemu.pm +++ b/PVE/API2/Qemu.pm @@ -2102,7 +2102,7

Re: [pve-devel] [PATCH pve-docs] added Memory Encryption documentation

2022-06-09 Thread Dietmar Maurer
Live migration works? > +Limitations: > + > +* Memory usage on host is always wrong and around 82% Usage > +* Snapshots do not work > +* edk2-OVMF required > +* Recommendable: VirtIO RNG for more entropy (VMs sometimes will not ___ pve-devel mailing

Re: [pve-devel] [PATCH proxmox-openid-rs] fix Open ID with Azure as provider

2022-03-31 Thread Dietmar Maurer
> let response = if let Method::POST = request.method { > -req.send(&*request.body) > +let bytes = request.body.as_slice(); > +req.send_bytes(bytes) Does this have the side effect of changing the transfer encoding? If so, it is worth to add an inline comment.

Re: [pve-devel] [PATCH qemu-server v2 4/4] parse vm config: remove "\s*" from multi-line comment regex

2022-02-24 Thread Dietmar Maurer
> On 02/24/2022 3:49 PM Stefan Sterz wrote: > > > To be consistent with PBS's implementation of multi-line comments > remove "\s*" here too. Since the regex isn't lazy .* matches > everything \s* would anyway. But the old regex trimm spaces from the end, so this is quite different!

Re: [pve-devel] auto ballooning && ksm thresholds both at 80%

2021-12-31 Thread Dietmar Maurer
> Currently both are at 80%, > that mean that ballooning is vm reducing memory fast, and ksm don't > have time to run. > > as ballooning is a lot more intrusive than ksm, I wonder if it couldn't > be set to something like 90%. That sounds reasonable to me, but can you see that theoretical effect

Re: [pve-devel] Groups for OpenID Connect?

2021-12-24 Thread Dietmar Maurer
> This endpoint here would be Google Workspace (i.e. Google's OIDC provider). > > Currently, in the Proxmox LDAP sync - it translates Google Groups (in the > Google Workspace domain) into LDAP groups, which is what we want. > > I'm not too familiar with the OIDC - I do know that Google

Re: [pve-devel] Groups for OpenID Connect?

2021-12-23 Thread Dietmar Maurer
> However, is there any support for groups in OpenID Connect, or a similar > concept? In OpenID, it is possible to request "scopes" from the server, which can then send additional data (claims). But I am unsure if and how people use those system to manage groups. So what kind of OpenID

[pve-devel] [proxmox-backup-qemu] update dependencies to latest proxmox-backup git version

2021-11-05 Thread Dietmar Maurer
Signed-off-by: Dietmar Maurer --- Cargo.toml | 15 +++ src/backup.rs | 11 +++ src/commands.rs | 15 +-- src/lib.rs | 11 ++- src/restore.rs | 13 ++--- src/shared_cache.rs | 2 +- src/upload_queue.rs | 6

Re: [pve-devel] building pve-docs - Can't locate PVE/RESTHandler.pm

2021-09-17 Thread Dietmar Maurer
> I don't know where to find the PVE::RESTHandler module. Any (working) PVE installation should have that installed... ___ pve-devel mailing list pve-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] Flow for i18n

2021-08-18 Thread Dietmar Maurer
Isn't it easy enough to edit a po file? I am not particularly keen to setup and maintain another tool/service. > On 08/18/2021 12:46 PM Claudio Ferreira wrote: > > > Hi for all > > My name is Claudio and I did some translations for many open source > projects, and I want to help in ProxMox

[pve-devel] [PATCH pve-docs] add OpenId Connect docu

2021-07-01 Thread Dietmar Maurer
--- pveum.adoc | 88 +- 1 file changed, 87 insertions(+), 1 deletion(-) diff --git a/pveum.adoc b/pveum.adoc index a1adbaa..9329583 100644 --- a/pveum.adoc +++ b/pveum.adoc @@ -29,7 +29,7 @@ endif::manvolnum[] Proxmox VE supports multiple

[pve-devel] [PATCH pve-access-control v2 4/5] api: implement openid API

2021-06-30 Thread Dietmar Maurer
This moves compute_api_permission() into RPCEnvironment.pm. --- src/PVE/API2/AccessControl.pm | 60 ++ src/PVE/API2/Makefile | 3 +- src/PVE/API2/OpenId.pm| 211 ++ src/PVE/RPCEnvironment.pm | 49 4 files changed, 270

[pve-devel] [PATCH pve-access-control v2 5/5] implement OpenID autocreate user feature

2021-06-30 Thread Dietmar Maurer
--- src/PVE/API2/OpenId.pm | 35 +++ 1 file changed, 31 insertions(+), 4 deletions(-) diff --git a/src/PVE/API2/OpenId.pm b/src/PVE/API2/OpenId.pm index d0b29fc..8384729 100644 --- a/src/PVE/API2/OpenId.pm +++ b/src/PVE/API2/OpenId.pm @@ -9,9 +9,10 @@ use

[pve-devel] [PATCH pve-access-control v2 2/5] add OpenId configuration

2021-06-30 Thread Dietmar Maurer
--- src/PVE/AccessControl.pm | 2 ++ src/PVE/Auth/Makefile| 3 +- src/PVE/Auth/OpenId.pm | 68 3 files changed, 72 insertions(+), 1 deletion(-) create mode 100755 src/PVE/Auth/OpenId.pm diff --git a/src/PVE/AccessControl.pm

[pve-devel] [PATCH pve-access-control v2 0/5] add OpenId realms

2021-06-30 Thread Dietmar Maurer
Changes in v2: - also check if user is expired (in check_user_enabled) - always die with newline - rename "user-attr" to "username-claim" Dietmar Maurer (5): check_user_enabled: also check if user is expired add OpenId configuration depend on libpve-rs-perl api:

[pve-devel] [PATCH pve-access-control v2 3/5] depend on libpve-rs-perl

2021-06-30 Thread Dietmar Maurer
--- debian/control | 2 ++ 1 file changed, 2 insertions(+) diff --git a/debian/control b/debian/control index 81a32bd..3ef748b 100644 --- a/debian/control +++ b/debian/control @@ -10,6 +10,7 @@ Build-Depends: debhelper (>= 12~), lintian, perl,

[pve-devel] [PATCH pve-access-control v2 1/5] check_user_enabled: also check if user is expired

2021-06-30 Thread Dietmar Maurer
--- src/PVE/AccessControl.pm | 16 +++- 1 file changed, 7 insertions(+), 9 deletions(-) diff --git a/src/PVE/AccessControl.pm b/src/PVE/AccessControl.pm index 2569a35..8628678 100644 --- a/src/PVE/AccessControl.pm +++ b/src/PVE/AccessControl.pm @@ -428,12 +428,10 @@ sub verify_token

[pve-devel] [PATCH pve-access-control 3/4] api: implement openid API

2021-06-24 Thread Dietmar Maurer
This moves compute_api_permission() into RPCEnvironment.pm. --- src/PVE/API2/AccessControl.pm | 60 ++ src/PVE/API2/Makefile | 3 +- src/PVE/API2/OpenId.pm| 214 ++ src/PVE/RPCEnvironment.pm | 49 4 files changed, 273

[pve-devel] [PATCH pve-access-control 1/4] add OpenId configuration

2021-06-24 Thread Dietmar Maurer
--- src/PVE/AccessControl.pm | 2 ++ src/PVE/Auth/Makefile| 3 +- src/PVE/Auth/OpenId.pm | 67 3 files changed, 71 insertions(+), 1 deletion(-) create mode 100755 src/PVE/Auth/OpenId.pm diff --git a/src/PVE/AccessControl.pm

[pve-devel] [PATCH pve-access-control 2/4] depend on libpve-rs-perl

2021-06-24 Thread Dietmar Maurer
--- debian/control | 2 ++ 1 file changed, 2 insertions(+) diff --git a/debian/control b/debian/control index 81a32bd..3ef748b 100644 --- a/debian/control +++ b/debian/control @@ -10,6 +10,7 @@ Build-Depends: debhelper (>= 12~), lintian, perl,

[pve-devel] [PATCH pve-manager] ui: implement OpenId login

2021-06-24 Thread Dietmar Maurer
--- PVE/HTTPServer.pm | 4 +- www/manager6/Utils.js | 8 +++ www/manager6/window/LoginWindow.js | 105 - 3 files changed, 114 insertions(+), 3 deletions(-) diff --git a/PVE/HTTPServer.pm b/PVE/HTTPServer.pm index 636b562b..dabdf7f3

[pve-devel] [PATCH pve-access-control 4/4] implement OpenID autocreate user feature

2021-06-24 Thread Dietmar Maurer
--- src/PVE/API2/OpenId.pm | 33 ++--- 1 file changed, 30 insertions(+), 3 deletions(-) diff --git a/src/PVE/API2/OpenId.pm b/src/PVE/API2/OpenId.pm index db9f9eb..3814895 100644 --- a/src/PVE/API2/OpenId.pm +++ b/src/PVE/API2/OpenId.pm @@ -9,9 +9,10 @@ use

Re: [pve-devel] RE : RE : [PATCH] [PATCH pve-access-control] SSO feature:login with SAMLv2

2021-06-02 Thread Dietmar Maurer
> On 06/02/2021 12:16 PM wb wrote: > > > > I also wonder why SAML? Would it be an option to use OpenId connect instead? > As I was able to use SAML, I know the functional part and therefore, if I > used SAML, it is only by ease. > > Switch to OpenID, why not. The time I set up a functional

Re: [pve-devel] RE : [PATCH] [PATCH pve-access-control] SSO feature:login with SAMLv2

2021-06-02 Thread Dietmar Maurer
> > I wonder why you want to store temporary data in /etc/pve/tmp/saml. > > Wouldn't it we good enough > > to store that on the local file system? > On the one hand, I enjoyed reusing your work. > On the other hand, I think it is more secure to put this kind of data in > /etc/pve/tmp/saml than

Re: [pve-devel] [PATCH] [PATCH pve-access-control] SSO feature: login with SAMLv2

2021-06-01 Thread Dietmar Maurer
Unfortunately, your code depends on code not packaged for Debian. Any idea how to replace that (cpanm Net::SAML2)? Or better, is there a 'rust' implementaion for SAML2? If so, we could make perl bindings for that and reuse the code with Proxmox Backup Server. Other ideas? > diff --git

Re: [pve-devel] [PATCH] [PATCH pve-access-control] SSO feature: login with SAMLv2

2021-06-01 Thread Dietmar Maurer
I wonder why you want to store temporary data in /etc/pve/tmp/saml. Wouldn't it we good enough to store that on the local file system? > On 05/27/2021 11:55 PM Julien BLAIS wrote: > > > Added a new endpoint usable by api2/html/access/saml?realm=$DOM > which allows to initiate a redirection

Re: [pve-devel] [PATCH] [PATCH pve-http-server] SSO feature: login with SAMLv2

2021-05-31 Thread Dietmar Maurer
I am trying to test your code, so I need a SAML Identity provider. What is the best OSS implementation for that? I tried lemonldap-ng, but there example configuration is a nightmare and I was unable to get that running. Is there anything else I can use to test?. - Dietmar

Re: [pve-devel] cfs-locked 'authkey' operation: pve cluster filesystem not online

2021-05-24 Thread Dietmar Maurer
Hi Julien, > Hello to all. > > I have the plan to implement the SSO authentication feature with the SAML > protocol. > However, I have an error that prevents me from validating the authentication > process. > It is about the locks. > The first step is to store the request_saml_id. If I try to

Re: [pve-devel] rust build questions

2021-05-05 Thread Dietmar Maurer
:api::api` > --> src/api/api_type_macros.rs:6:5 > | > 4 | use proxmox::api::api; > | ^ no `api` in `api` Sorry, this was a bug in the proxmox crate. Please update an test again: commit fa3b5374ed61da3c40a1fc58070d6a16c877c3af (HEAD -> master, origin/master, origin/

Re: [pve-devel] [PATCH v2 pve-access-control] fix #1500: permission path syntax check for access control

2021-04-19 Thread Dietmar Maurer
FYI, I do it without any regex in rust: https://git.proxmox.com/?p=proxmox-backup.git;a=blob;f=src/config/acl.rs;h=61e507ec42bf5a30f64f56564a1fb107d148fb7b;hb=HEAD#l272 I guess this is faster (at least in rust). > On 04/19/2021 9:16 AM Lorenz Stechauner wrote: > > > Syntax for permission

Re: [pve-devel] [PATCH manager 2/2] Close #1295: Make apt notifications configurable

2021-04-08 Thread Dietmar Maurer
> Is there a reason why we assume that users without subscription do not want > such notifications? > > As far as I see it, if we change it to > > $dccfg->{notify_updates} // 1 > Then (until they change something) > - users with active subscription should _continue_ to get notifications > -

Re: [pve-devel] [pbs-devel] [PATCH v2 proxmox-backup-qemu 05/11] access: use bigger cache and LRU chunk reader

2021-03-17 Thread Dietmar Maurer
What about using a memory mapped files as cache. That way, you do not need to care about available memory? > >> Maybe we could get the available memory and use that as hint, I mean as > >> memory > >> usage can be highly dynamic it will never be perfect, but better than just > >> ignoring > >>

Re: [pve-devel] [PATCH storage] fix #3203: report smart status correctly

2021-03-01 Thread Dietmar Maurer
This code is quite strange. Please can you use a normal: if .. then .. else .. > +push @$cmd, '-H' if $healthonly; > +push @$cmd, '-a', '-A', '-f', 'brief' if !$healthonly; ___ pve-devel mailing list pve-devel@lists.proxmox.com

Re: [pve-devel] [PATCH qemu-server] fix #3324: clone disk: use larger blocksize for EFI disk when possible

2021-03-01 Thread Dietmar Maurer
> >>> diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm > >>> index f401baf..e579cdf 100644 > >>> --- a/PVE/QemuServer.pm > >>> +++ b/PVE/QemuServer.pm > >>> @@ -6991,7 +6991,15 @@ sub clone_disk { > >>>   # that is given by the OVMF_VARS.fd > >>>   my $src_path =

Re: [pve-devel] [PATCH v2 ha-manager] ha-simulator: add xauth dependency

2021-02-09 Thread Dietmar Maurer
This is true for anything. X11 forwarding simply works that way. So I am quite unsure if we should add xauth here... Or is this a common practice (I am unaware of)? > > When installing the ha-simulator on a PVE node to start it via ssh with > x11 forwarding, the xauth package helps to avoid

Re: [pve-devel] Adding new data storage

2020-12-17 Thread Dietmar Maurer
> I would like to add new data storage. This storage would resemble ZFS > over iSCSI but will use different API to access storage. I am curios, what API exactly? ___ pve-devel mailing list pve-devel@lists.proxmox.com

Re: [pve-devel] Compiling PBS on arm64

2020-12-06 Thread Dietmar Maurer
> On 12/06/2020 8:41 PM Kamil Trzciński wrote: > > > I'm slightly progressing, but I stumbled across some `debcargo` problem. It > appears that > Proxmox uses their own fork of `debcargo`, which is needed in order to > build crates > without the usage of crates.io. I guess you can simply

Re: [pve-devel] [PATCH pve-manager]: api2:/cluster return intiger format for VMID

2020-12-01 Thread Dietmar Maurer
> On 12/01/2020 10:41 AM Dietmar Maurer wrote: > > > > for (my $i = 100; $i < 1; $i++) { > > - return $i if !defined($idlist->{$i}); > > + return int($i) if !defined($idlist->{$i}); > > IMO, this does not solve the problem, becau

Re: [pve-devel] [PATCH pve-manager]: api2:/cluster return intiger format for VMID

2020-12-01 Thread Dietmar Maurer
> for (my $i = 100; $i < 1; $i++) { > - return $i if !defined($idlist->{$i}); > + return int($i) if !defined($idlist->{$i}); IMO, this does not solve the problem, because $i is already and int. ___ pve-devel mailing list

Re: [pve-devel] [PATCH v3 manager] ad #3140: allow interface suffix in dns entries

2020-11-25 Thread Dietmar Maurer
> What text do you mean exactly? The interface name? > Arbitrary null-terminated byte string... Ok > (Yes I can name an interface "---" or  (poop-emoji)..., > neither of which our iface schema in JSONSchema.pm would allow...) great. ___ pve-devel

Re: [pve-devel] [PATCH v3 manager] ad #3140: allow interface suffix in dns entries

2020-11-25 Thread Dietmar Maurer
Thanks for the info. But what encoding does that text use? I cannot find that in RFC4007 (they only talk about strings and text). > > Answering myself, it is defined in RFC4007. > > > > But "man resolv.conf" say address must be RFC2373 ? > > It'll still work. It's a very common notation for

Re: [pve-devel] [PATCH v3 manager] ad #3140: allow interface suffix in dns entries

2020-11-25 Thread Dietmar Maurer
Answering myself, it is defined in RFC4007. But "man resolv.conf" say address must be RFC2373 ? > On 11/25/2020 12:08 PM Dietmar Maurer wrote: > > > What kind of format is that? RFC2373 does not mention it. Please ca

Re: [pve-devel] [PATCH v3 manager] ad #3140: allow interface suffix in dns entries

2020-11-25 Thread Dietmar Maurer
What kind of format is that? RFC2373 does not mention it. Please can you give me a hint? > On 11/25/2020 11:36 AM Wolfgang Bumiller wrote: > > > Signed-off-by: Wolfgang Bumiller > --- > changes to v2: > * use `for of` loop in verify_ip64_address_list > > www/manager6/Toolkit.js | 17

[pve-devel] applied: [PATCH v3 proxmox-backup-qemu 2/3] return version via rust fn instead of header define

2020-11-24 Thread Dietmar Maurer
applied (and built a new proxmox-backup-qemu package) ___ pve-devel mailing list pve-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH v3 0/3] Add proxmox-backup-qemu version to VM status output

2020-11-24 Thread Dietmar Maurer
> proxmox-backup-qemu is missing a not-pushed version bump commit, otherwise > I'd have applied this. sorry, just pushed the missing commit. ___ pve-devel mailing list pve-devel@lists.proxmox.com

[pve-devel] applied: [pbs-devel] [PATCH proxmox-backup v2 1/1] client: add 'snapshot notes show/update' command

2020-11-24 Thread Dietmar Maurer
applied ___ pve-devel mailing list pve-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] Improve container backup speed dramatically (factor 100-1000)

2020-11-19 Thread Dietmar Maurer
> Container backup is very slow compared to VM backup. I have a 500 GB > container (sftp server) with minimal changing files, but even the incremental > bakcups take 2 hours with heavy disk activity. Almost nothing is transfered > to the backup server. It seems that it it reads the whole

Re: [pve-devel] [RFC PATCH] fix #3106: correctly queue incoming connections

2020-11-03 Thread Dietmar Maurer
Answering myself, this works as expected. I now simply use Arc::new(()) to count references. > On 11/03/2020 6:45 PM Dietmar Maurer wrote: > > > > > +Ok((sock, _addr)) => { > > > +sock.set_nodelay(true).unwrap();

Re: [pve-devel] [RFC PATCH] fix #3106: correctly queue incoming connections

2020-11-03 Thread Dietmar Maurer
> > +Ok((sock, _addr)) => { > > +sock.set_nodelay(true).unwrap(); > > +let _ = set_tcp_keepalive(sock.as_raw_fd(), > > PROXMOX_BACKUP_TCP_KEEPALIVE_TIME); > > +let acceptor = Arc::clone(); > > +let

Re: [pve-devel] [RFC PATCH] fix #3106: correctly queue incoming connections

2020-11-03 Thread Dietmar Maurer
> On Tue, Nov 03, 2020 at 02:25:21PM +0100, Dietmar Maurer wrote: > > > > -let connections = > > > > proxmox_backup::tools::async_io::HyperAccept(connections); > > > > +let connections = accept_connections(listener, acceptor

Re: [pve-devel] [RFC PATCH] fix #3106: correctly queue incoming connections

2020-11-03 Thread Dietmar Maurer
> > -let connections = > > proxmox_backup::tools::async_io::HyperAccept(connections); > > +let connections = accept_connections(listener, acceptor); > > +let connections = > > hyper::server::accept::from_stream(connections); > > If we move the `from_stream`

[pve-devel] [RFC PATCH] fix #3106: correctly queue incoming connections

2020-11-03 Thread Dietmar Maurer
--- based, on Domink's patch, but with the following changes: - factor out code into separate function accept_connections() - no select with shutdown future (no needed) - remove sender2.send_timeout() - not sure why this was there? - restict number of spawned tasks Seems to work, but I

Re: [pve-devel] [PATCH pve-manager] pvestatd: stream host pressure counters

2020-10-23 Thread Dietmar Maurer
> I would like also to add some improvement for vm memory/cpu stats. > > for cpu, currently, we only monitor the qemu process cpu usage, but > with virtio-net + vhost-net, > we are missing vhost-* process cpu usage. (For vms will a lot of > traffic, this is really signifiant). > > I would like

[pve-devel] [PATCH proxmox-widget-toolkit] correctly encode upid paramenter (use encodeURIComponent)

2020-10-22 Thread Dietmar Maurer
--- src/window/TaskViewer.js | 8 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/src/window/TaskViewer.js b/src/window/TaskViewer.js index 0333472..31e1ebc 100644 --- a/src/window/TaskViewer.js +++ b/src/window/TaskViewer.js @@ -14,7 +14,7 @@

Re: [pve-devel] [PATCH pve-common 1/1] ProcFSTools: add read_pressure

2020-10-13 Thread Dietmar Maurer
> BTW, I'm currently playing with reading the rrd files, and I have notice than > lower precision is 1minute. > as pvestatd send values around each 10s, is this 1minute precision an average > of 6x10s values send by pvestatd ? Yes (we also store the MAX) > I'm currently working on a poc of vm

Re: [pve-devel] [PATCH pve-common 1/1] ProcFSTools: add read_pressure

2020-10-13 Thread Dietmar Maurer
> I have notice that it's possible to get pressure info for each vm/ct > through cgroups > > /sys/fs/cgroup/unified/qemu.slice/.scope/cpu.pressure > /sys/fs/cgroup/unified/lxc//cpu.pressure > > > Maybe it could be great to have some new rrd graphs for each vm/ct ? > They are very useful

Re: [pve-devel] [PATCH pve-manager] pvestatd: stream host pressure counters

2020-10-12 Thread Dietmar Maurer
I wonder if we want to store that information with RRD instead? > On 10/06/2020 1:58 PM Alexandre Derumier wrote: > > > Signed-off-by: Alexandre Derumier > --- > PVE/Service/pvestatd.pm | 4 > 1 file changed, 4 insertions(+) > > diff --git a/PVE/Service/pvestatd.pm

[pve-devel] applied: [PATCH pve-common 1/1] ProcFSTools: add read_pressure

2020-10-12 Thread Dietmar Maurer
applied ___ pve-devel mailing list pve-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] applied: [PATCH widget-toolkit] fix autoscrolling on browser zoom

2020-09-30 Thread Dietmar Maurer
applied ___ pve-devel mailing list pve-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH proxmox-backup] avoid chrono dependency, depend on proxmox 0.3.8

2020-09-14 Thread Dietmar Maurer
- remove chrono dependency - depend on proxmox 0.3.8 - remove epoch_now, epoch_now_u64 and epoch_now_f64 - remove tm_editor (moved to proxmox crate) - use new helpers from proxmox 0.3.8 * epoch_i64 and epoch_f64 * parse_rfc3339 * epoch_to_rfc3339_utc * strftime_local - BackupDir

Re: [pve-devel] corosync bug: cluster break after 1 node clean shutdown

2020-09-14 Thread Dietmar Maurer
> I wonder if something like pacemaker sbd could be implemented in proxmox as > extra layer of protection ? AFAIK Thomas already has patches to implement active fencing. But IMHO this will not solve the corosync problems.. ___ pve-devel mailing list

[pve-devel] [RFC proxmox-backup] avoid chrono dependency

2020-09-13 Thread Dietmar Maurer
depend on proxmox 0.3.7 --- Cargo.toml | 3 +- examples/download-speed.rs | 4 +- examples/upload-speed.rs | 2 +- src/api2/admin/datastore.rs| 28 ++--- src/api2/backup.rs | 2 +-

Re: [pve-devel] corosync bug: cluster break after 1 node clean shutdown

2020-09-04 Thread Dietmar Maurer
> do you think it could be possible to add an extra optionnal layer of security > check, not related to corosync ? I would try to find the bug instead. > I'm still afraid of this corosync bug since years, and still don't use HA. > (or I have tried to enable it 2months ago,and this give me a

Re: [pve-devel] [PATCH common] get_options: allow optional arguments "arg_params" if no ambiguity

2020-08-26 Thread Dietmar Maurer
> If we run out of passed arguments from the user but still had defined > "arg_params" (those params which went after the command in fixed > order without option -- dashes) we always errored out with "not > enough arguments". But, there are situations where the remaining > arg_params are all

Re: [pve-devel] More than 10 interfaces in lxc containers

2020-08-24 Thread Dietmar Maurer
> On 08/24/2020 12:54 PM Stephan Leemburg wrote: > > > On 24-08-2020 06:53, Dietmar Maurer wrote: > >> If I don't put a tag on the device, it seems to behave like a trunk. So, > >> that would solve my problem. _If_ the hosts where openvswitch enabled. >

Re: [pve-devel] More than 10 interfaces in lxc containers

2020-08-23 Thread Dietmar Maurer
> If I don't put a tag on the device, it seems to behave like a trunk. So, > that would solve my problem. _If_ the hosts where openvswitch enabled. I am unable to see why you need openvswitch for that? This also works with standard linux network. ___

Re: [pve-devel] More than 10 interfaces in lxc containers

2020-08-23 Thread Dietmar Maurer
> If it would be possible to provide a 'trunk' openvswitch interface to > the CT, then from within the CT vlan devices could be setup from the > trunk, but in the end that will still create 10+ interfaces in the > container itself. Cant you simply use a single network interface, then configure

Re: [pve-devel] More than 10 interfaces in lxc containers

2020-08-22 Thread Dietmar Maurer
> > For me, I have that need too for a firewall container. > > Why does your firewall need more the 10 interface? Sigh. too early in the morning... I wanted to ask: Why does your firewall need more than 10 interfaces? Normally, a firewall uses one interface per zone, and more than 10 zones

Re: [pve-devel] More than 10 interfaces in lxc containers

2020-08-22 Thread Dietmar Maurer
> For me, I have that need too for a firewall container. Why does your firewall need more the 10 interface? > Would you please consider raising the limit? No, unless someone can explain why that is required ;-) ___ pve-devel mailing list

Re: [pve-devel] Proxmox, Qemu and COLO feature...

2020-08-21 Thread Dietmar Maurer
> Could you guys tell me how the status of qemu COLO feature regarding > Proxmox VE? > Is there any plan for this? No plans for now ... ___ pve-devel mailing list pve-devel@lists.proxmox.com

  1   2   >