Re: [pve-devel] [PATCH manager 1/4] ceph: add perf data cache helpers

2019-11-05 Thread Dominik Csapak
On 11/5/19 6:33 PM, Thomas Lamprecht wrote: On 11/5/19 1:51 PM, Dominik Csapak wrote: add a helper to cache the ceph performance data inside pmxcfs with broadcast_node_kv, and also a helper to read it out merge the data from all nodes that sent performance data the '$perf_cache' variable actua

[pve-devel] [PATCH manager] add graphite tcp support

2019-11-05 Thread Martin Verges
This change allows sending statistics to graphite over TCP. So far only UDP is possible, which is not available in some environments, like behind a loadbalancer. Configuration example: ~ $ cat /etc/pve/status.cfg graphite: server 10.20.30.40 port 2003 path proxmox proto tcp

[pve-devel] applied-series: [PATCH storage 2/2] convert vmid to int to conform with return type declaration

2019-11-05 Thread Thomas Lamprecht
On 11/5/19 1:42 PM, Tim Marx wrote: > Signed-off-by: Tim Marx > --- > PVE/API2/Storage/Content.pm | 1 + > 1 file changed, 1 insertion(+) > applied both patches, thanks! ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-

Re: [pve-devel] [PATCH manager 1/4] ceph: add perf data cache helpers

2019-11-05 Thread Thomas Lamprecht
On 11/5/19 1:51 PM, Dominik Csapak wrote: > add a helper to cache the ceph performance data inside pmxcfs > with broadcast_node_kv, and also a helper to read it out > > merge the data from all nodes that sent performance data > > the '$perf_cache' variable actually serves two purposes, > the writ

Re: [pve-devel] [PATCH manager] add graphite tcp support

2019-11-05 Thread Martin Verges
sorry, the git send-email is awful ;/ I hope the patch and documentation patch is now to your satisfaction as it includes a timeout for tcp as well and therefore should not cause any trouble. We use deployed it on our cluster and so far it seems to work fine. However we do not have a test environm

[pve-devel] [PATCH manager] add graphite tcp support

2019-11-05 Thread Martin Verges
This change allows sending statistics to graphite over TCP. So far only UDP is possible, which is not available in some environments, like behind a loadbalancer. Configuration example: ~ $ cat /etc/pve/status.cfg graphite: server 10.20.30.40 port 2003 path proxmox proto tcp

[pve-devel] [PATCH docs] add documatation for graphite tcp support

2019-11-05 Thread Martin Verges
Signed-off-by: Martin Verges --- pve-external-metric-server.adoc | 10 +++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/pve-external-metric-server.adoc b/pve-external-metric-server.adoc index c9e1107..1497a44 100644 --- a/pve-external-metric-server.adoc +++ b/pve-external-

[pve-devel] [PATCH manager] add graphite tcp support

2019-11-05 Thread Martin Verges
This change allows sending statistics to graphite over TCP. So far only UDP is possible, which is not available in some environments, like behind a loadbalancer. Configuration example: ~ $ cat /etc/pve/status.cfg graphite: server 10.20.30.40 port 2003 path proxmox proto tcp

[pve-devel] applied-series: [PATCH container 0/5] update & cleanup for our hooks

2019-11-05 Thread Thomas Lamprecht
On 11/5/19 1:58 PM, Wolfgang Bumiller wrote: > This should deduplicate some code in our hooks and add support for > `lxc.hook.version`. That way users can use newer hook versions without > breaking the PVE hooks in the process. > > Patches 2 & 4 are best viewed with -w. I'm attaching -w of 2 below

[pve-devel] [PATCH container 4/5] autodev hook: use new helpers and whitespace fixup

2019-11-05 Thread Wolfgang Bumiller
Signed-off-by: Wolfgang Bumiller --- src/lxc-pve-autodev-hook | 93 1 file changed, 27 insertions(+), 66 deletions(-) diff --git a/src/lxc-pve-autodev-hook b/src/lxc-pve-autodev-hook index 4913870..3c45949 100755 --- a/src/lxc-pve-autodev-hook +++ b/src/l

[pve-devel] [PATCH container 0/5] update & cleanup for our hooks

2019-11-05 Thread Wolfgang Bumiller
This should deduplicate some code in our hooks and add support for `lxc.hook.version`. That way users can use newer hook versions without breaking the PVE hooks in the process. Patches 2 & 4 are best viewed with -w. I'm attaching -w of 2 below, 4 isn't that tragic. Wolfgang Bumiller (5): add PV

[pve-devel] [PATCH container 2/5] change hooks to use new helper

2019-11-05 Thread Wolfgang Bumiller
We now get rid of all the PVE::CLIHandler baggage which reduces the code a lot. It is also not compatible with the new lxc.hook.version=1 method of hooks! The new helper is specific to lxc hooks and supports both current `lxc.hook.version`s. Signed-off-by: Wolfgang Bumiller --- src/lxc-pve-auto

[pve-devel] [PATCH container 1/5] add PVE::LXC::Tools

2019-11-05 Thread Wolfgang Bumiller
Will contain lxc/container specific tools which should also be accessible within our lxc hook scripts. Signed-off-by: Wolfgang Bumiller --- src/PVE/LXC/Makefile | 7 +++- src/PVE/LXC/Tools.pm | 82 2 files changed, 88 insertions(+), 1 deletion(-) cr

[pve-devel] [PATCH container 5/5] config: whitelist lxc.hook.version

2019-11-05 Thread Wolfgang Bumiller
We currently don't depend on a particular version, although in the future we may want to enforce a minimum (at which point we'll need more than just a whitelist entry for this, but right now this will do...) Signed-off-by: Wolfgang Bumiller --- src/PVE/LXC/Config.pm | 1 + 1 file changed, 1 inse

[pve-devel] [PATCH container 3/5] tools: add device file iterator, and cgroup_do_write

2019-11-05 Thread Wolfgang Bumiller
Signed-off-by: Wolfgang Bumiller --- src/PVE/LXC/Tools.pm | 51 1 file changed, 51 insertions(+) diff --git a/src/PVE/LXC/Tools.pm b/src/PVE/LXC/Tools.pm index 18a1a22..bebd7d8 100644 --- a/src/PVE/LXC/Tools.pm +++ b/src/PVE/LXC/Tools.pm @@ -79,4 +79,

[pve-devel] [PATCH manager 4/4] gui: ceph-dashboard: display cached performance data

2019-11-05 Thread Dominik Csapak
this calls the api once when opening the dashboard, and fills the charts with that cached data, then it continues like previously with its interval Signed-off-by: Dominik Csapak --- www/manager6/ceph/Status.js | 18 ++ 1 file changed, 18 insertions(+) diff --git a/www/manager6/c

[pve-devel] [PATCH manager 1/4] ceph: add perf data cache helpers

2019-11-05 Thread Dominik Csapak
add a helper to cache the ceph performance data inside pmxcfs with broadcast_node_kv, and also a helper to read it out merge the data from all nodes that sent performance data the '$perf_cache' variable actually serves two purposes, the writer (will be pvestatd) uses it to broadcast only its valu

[pve-devel] [PATCH manager 2/4] ceph: cache perf data in pvestatd

2019-11-05 Thread Dominik Csapak
Signed-off-by: Dominik Csapak --- altough the rados call have a default timeout of 5 seconds, i am not sure if this is too much of a load we could only run that call e.g. every 3rd pvestatd loop, to minimize the load. combined with the merging of the data on the reader side, i guess we would have

[pve-devel] [PATCH manager 3/4] ceph: add 'iostat' api call

2019-11-05 Thread Dominik Csapak
this gets the cached ceph performance data back from pmxcfs Signed-off-by: Dominik Csapak --- PVE/API2/Cluster/Ceph.pm | 24 1 file changed, 24 insertions(+) diff --git a/PVE/API2/Cluster/Ceph.pm b/PVE/API2/Cluster/Ceph.pm index fa79d077..31373ef2 100644 --- a/PVE/API2/

[pve-devel] [PATCH manager 0/4] cache ceph performance data for the dashboard

2019-11-05 Thread Dominik Csapak
with this patch, we are able to directly fill the charts on the ceph dashboard with cached performance data, instead of always showing empty ones each time we load the dashboard Dominik Csapak (4): ceph: add perf data cache helpers ceph: cache perf data in pvestatd ceph: add 'iostat' api cal

[pve-devel] [PATCH 1/3] Copy the needed function form acme.sh

2019-11-05 Thread Wolfgang Link
For the thin wrapper around acme.sh DNS plugins, the required functions are copied. The project acme.sh can be found here. https://github.com/Neilpang/acme.sh --- src/proxmox-acme/proxmox-acme | 806 ++ 1 file changed, 806 insertions(+) create mode 100644 src/prox

[pve-devel] RFC V2 for ACME DNS Challenge

2019-11-05 Thread Wolfgang Link
These patches are just the acme.sh thin wrapper part for the reuse of the dnsapi. In this version I use curl instead of wget. Curl is already part of PMG, and PVE has libcurl packages dependency. This script has additionl an additional on - coreutils (an we ignore) - sed (must be install

[pve-devel] [PATCH 3/3] Implement functions setup and teardown. We use these functions for adding and removing a txt record through the dnsapi.

2019-11-05 Thread Wolfgang Link
--- src/proxmox-acme/proxmox-acme | 78 +++ 1 file changed, 78 insertions(+) diff --git a/src/proxmox-acme/proxmox-acme b/src/proxmox-acme/proxmox-acme index 662c39a..6da9436 100644 --- a/src/proxmox-acme/proxmox-acme +++ b/src/proxmox-acme/proxmox-acme @@ -1,5 +1,

[pve-devel] [PATCH 2/3] Remove unnecessary Code and fixes.

2019-11-05 Thread Wolfgang Link
This Code is not required in the Proxmox environment. We know in our environment what we have as a tool-change. Fix Code what does not work because variable or functions are missing. --- src/proxmox-acme/proxmox-acme | 476 +- 1 file changed, 120 insertions(+), 356

[pve-devel] [PATCH storage 2/2] convert vmid to int to conform with return type declaration

2019-11-05 Thread Tim Marx
Signed-off-by: Tim Marx --- PVE/API2/Storage/Content.pm | 1 + 1 file changed, 1 insertion(+) diff --git a/PVE/API2/Storage/Content.pm b/PVE/API2/Storage/Content.pm index 76c3b67..9f6ea10 100644 --- a/PVE/API2/Storage/Content.pm +++ b/PVE/API2/Storage/Content.pm @@ -101,6 +101,7 @@ __PACKAGE__->

[pve-devel] [PATCH storage 1/2] whitespace cleanup

2019-11-05 Thread Tim Marx
Signed-off-by: Tim Marx --- PVE/API2/Storage/Content.pm | 56 ++--- 1 file changed, 28 insertions(+), 28 deletions(-) diff --git a/PVE/API2/Storage/Content.pm b/PVE/API2/Storage/Content.pm index 9731720..76c3b67 100644 --- a/PVE/API2/Storage/Content.pm +++ b/PVE/A

Re: [pve-devel] [PATCH manager] add graphite tcp support

2019-11-05 Thread Dominik Csapak
On 11/5/19 1:21 PM, Thomas Lamprecht wrote: On 11/5/19 1:04 PM, Alexandre DERUMIER wrote: Hi, be carefull of timeouts, because it could hang pvestatd. (That's why it was implemented in udp for graphite && influxdb) That's true, as almost always there's a reason why seemingly simple things ar

Re: [pve-devel] [PATCH manager] add graphite tcp support

2019-11-05 Thread Thomas Lamprecht
On 11/5/19 1:04 PM, Alexandre DERUMIER wrote: > Hi, > > be carefull of timeouts, because it could hang pvestatd. > (That's why it was implemented in udp for graphite && influxdb) > That's true, as almost always there's a reason why seemingly simple things are not already done.. So we need to be

Re: [pve-devel] [PATCH manager] add graphite tcp support

2019-11-05 Thread Alexandre DERUMIER
Hi, be carefull of timeouts, because it could hang pvestatd. (That's why it was implemented in udp for graphite && influxdb) - Mail original - De: "Martin Verges" À: "pve-devel" Envoyé: Lundi 4 Novembre 2019 16:48:04 Objet: [pve-devel] [PATCH manager] add graphite tcp support This chan

[pve-devel] applied: [PATCH kernel] fix #2448: compile ISO8859-1 NLS as built-in

2019-11-05 Thread Thomas Lamprecht
Allows to mount VFAT devices even if the currently running kernel was removed before any VFAT, or other FS using the default Native Language Support module was mounted during the current uptime. This then could break updating the ESP partitions, which are mounted with VFAT in a postrm triggered st

Re: [pve-devel] [PATCH docs 10/11] Fix #1958: pveceph: add section Ceph maintenance

2019-11-05 Thread Aaron Lauterer
On 11/4/19 2:52 PM, Alwin Antreich wrote: Signed-off-by: Alwin Antreich --- pveceph.adoc | 54 1 file changed, 54 insertions(+) diff --git a/pveceph.adoc b/pveceph.adoc index 087c4d0..127e3bb 100644 --- a/pveceph.adoc +++ b/pveceph.adoc

Re: [pve-devel] [PATCH docs 08/11] pveceph: Reorganize TOC for new sections

2019-11-05 Thread Aaron Lauterer
In general wouldn't it be better to change the headlines to active voice? Create Monitors instead of Creating Monitors? More suggestions inline. On 11/4/19 2:52 PM, Alwin Antreich wrote: Put the previous added sections into subsection for a better outline of the TOC. With the rearrangement of

[pve-devel] applied: [PATCH docs 01/11] pveceph: old style commands to subcommands

2019-11-05 Thread Thomas Lamprecht
On 11/4/19 2:52 PM, Alwin Antreich wrote: > Replace remaining old style single commands with current subcommands > > Signed-off-by: Alwin Antreich > --- > pveceph.adoc | 12 ++-- > 1 file changed, 6 insertions(+), 6 deletions(-) > applied, thanks! __

Re: [pve-devel] [PATCH manager] add graphite tcp support

2019-11-05 Thread Dominik Csapak
Hi, thanks for the patch :) looks good for the most part, but afaics you did not sign our CLA yet, please see our Developer Documentation[0] for that. also it would be nice to extend our documentation for this in the pve-docs git repository[1], in the file 'pve-external-metric-server.adoc'[2], b

Re: [pve-devel] [PATCH docs 06/11] pveceph: add section - Destroying Ceph Pools

2019-11-05 Thread Aaron Lauterer
On 11/4/19 2:52 PM, Alwin Antreich wrote: Signed-off-by: Alwin Antreich --- pveceph.adoc | 21 +++ images/screenshot/gui-ceph-pools-destroy.png | Bin 0 -> 141532 bytes 2 files changed, 21 insertions(+) create mode 100644 images/screensho

Re: [pve-devel] [PATCH docs 05/11] pveceph: add section - Destroying Ceph Manager

2019-11-05 Thread Aaron Lauterer
On 11/4/19 2:52 PM, Alwin Antreich wrote: Signed-off-by: Alwin Antreich --- pveceph.adoc | 20 ++ .../screenshot/gui-ceph-manager-destroy.png | Bin 0 -> 153596 bytes 2 files changed, 20 insertions(+) create mode 100644 images/screensh

Re: [pve-devel] [PATCH docs 02/11] pveceph: add section - Destroying Ceph OSDs

2019-11-05 Thread Aaron Lauterer
On 11/5/19 10:40 AM, Aaron Lauterer wrote: To remove an OSD via the GUI first select the {PVE} node in the tree view. Select the **Ceph -> OSD** panel and select the OSD to destroy. Next click the **OUT** button. Once the OSD's status changed from `in` to `out` click the **STOP** button. A

Re: [pve-devel] [PATCH docs 03/11] pveceph: add section - Destroying Ceph Monitor

2019-11-05 Thread Aaron Lauterer
On 11/4/19 2:52 PM, Alwin Antreich wrote: Signed-off-by: Alwin Antreich --- pveceph.adoc | 19 ++ .../screenshot/gui-ceph-monitor-destroy.png | Bin 0 -> 154084 bytes 2 files changed, 19 insertions(+) create mode 100644 images/screensh

Re: [pve-devel] [PATCH docs 02/11] pveceph: add section - Destroying Ceph OSDs

2019-11-05 Thread Aaron Lauterer
Some suggestions inline. On 11/4/19 2:52 PM, Alwin Antreich wrote: Signed-off-by: Alwin Antreich --- pveceph.adoc | 31 + images/screenshot/gui-ceph-osd-destroy.png | Bin 0 -> 146184 bytes 2 files changed, 31 insertions(+) create mode 10

Re: [pve-devel] [PATCH pve-docs] Add section for ZFS Special Device

2019-11-05 Thread Aaron Lauterer
Nicely written. I have some suggestions inline: * splitting long sentences * adding more info as to what is valid for the size in special_small_blocks (taken from the zfs man page) * rewrote the last paragraph a bit On 10/22/19 12:33 PM, Fabian Ebner wrote: > Signed-off-by: Fabian Ebner > ---