Re: [pve-devel] [PATCH manager v2 0/5] ACME node adaptions for plugins

2020-05-06 Thread Dominik Csapak




On 5/6/20 8:11 PM, Thomas Lamprecht wrote:

On 5/6/20 4:31 PM, Dominik Csapak wrote:

this series adapts the node->certificates->acme panel to include
an 'accountselector' and to be able to add/edit/remove single domains,
including ones with a plugin

changes from v1:
* drop fieldLabel in ACMEAccountSelector
* reword 'Account' in 'Used Account'
* use different approach to change the account
   (use viewModel and a displayfield/combobox/editbutton to better
   see when the account actually changes)

Dominik Csapak (5):
   ui: add ACME selector formfields for account and plugins
   ui: Parser: add printACME
   ui: Utils: add helper functions for acme domains
   ui: node/ACME: add ACMEDomainEdit
   ui: node/ACME: rework ACME grid for plugin based domains

  www/manager6/Makefile|   2 +
  www/manager6/Parser.js   |   7 +
  www/manager6/Utils.js|  23 +
  www/manager6/form/ACMEAccountSelector.js |  21 +
  www/manager6/form/ACMEPluginSelector.js  |  19 +
  www/manager6/node/ACME.js| 617 ---
  6 files changed, 517 insertions(+), 172 deletions(-)
  create mode 100644 www/manager6/form/ACMEAccountSelector.js
  create mode 100644 www/manager6/form/ACMEPluginSelector.js



applied series, but this wasn't much tested, or the wrong thing was sent..


hi, sorry yes should have tested more (was a bit in a hurry yesterday)


I had to fix the acme undefined access exceptions I mentioned to you for v1
off-list,


i actually tested this, but did not trigger it, but looking at the code
now, yes i can see where it has tripped up


the "show button to go to account page" didn't worked at all (no code
doing it included),


yes, sorry, after my initial tests with this, i did actually never
remove my acme accounts again


I fixed that but threw it out for something different
all together.. 


looks good AFAICS


renamed: form/ACMEAPiSelector.js -> form/ACMEAPISelector.js > (Which I also 
mentioned for v1..) and threw in a few smaller followups.


but this was in my other series (which you applied the day before),

anyway, i would've been happy to send a v3 (you can ofc fix up my 
patches, but no need to generate more work for yourself :) )




The store glitches still on load for the Certificates ACME view, I increased
the interval to 10 seconds as I really did not wanted to investigate that too.


i think what you mean with 'glitch' is the loading mask on account 
verification... after looking at the thing again, i am not so sure 
anymore that we actually need that (we already have a list off accounts

in the accountselector). i'll see if i can improve that bit



The DC -> ACME ones could profit from update/diffstore to get changes more
live, even if seldom made - they are really cheap to query.



yes makes sense, i found also some other issues (backend as well)
i'll send some patches later today

thanks for fixing my bugs :)


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


Re: [pve-devel] [PATCH manager v2 0/5] ACME node adaptions for plugins

2020-05-06 Thread Thomas Lamprecht
On 5/6/20 4:31 PM, Dominik Csapak wrote:
> this series adapts the node->certificates->acme panel to include
> an 'accountselector' and to be able to add/edit/remove single domains,
> including ones with a plugin
> 
> changes from v1:
> * drop fieldLabel in ACMEAccountSelector
> * reword 'Account' in 'Used Account'
> * use different approach to change the account
>   (use viewModel and a displayfield/combobox/editbutton to better
>   see when the account actually changes)
> 
> Dominik Csapak (5):
>   ui: add ACME selector formfields for account and plugins
>   ui: Parser: add printACME
>   ui: Utils: add helper functions for acme domains
>   ui: node/ACME: add ACMEDomainEdit
>   ui: node/ACME: rework ACME grid for plugin based domains
> 
>  www/manager6/Makefile|   2 +
>  www/manager6/Parser.js   |   7 +
>  www/manager6/Utils.js|  23 +
>  www/manager6/form/ACMEAccountSelector.js |  21 +
>  www/manager6/form/ACMEPluginSelector.js  |  19 +
>  www/manager6/node/ACME.js| 617 ---
>  6 files changed, 517 insertions(+), 172 deletions(-)
>  create mode 100644 www/manager6/form/ACMEAccountSelector.js
>  create mode 100644 www/manager6/form/ACMEPluginSelector.js
> 

applied series, but this wasn't much tested, or the wrong thing was sent..
I had to fix the acme undefined access exceptions I mentioned to you for v1
off-list, the "show button to go to account page" didn't worked at all (no code
doing it included), I fixed that but threw it out for something different
all together.. renamed: form/ACMEAPiSelector.js -> form/ACMEAPISelector.js
(Which I also mentioned for v1..) and threw in a few smaller followups.

The store glitches still on load for the Certificates ACME view, I increased
the interval to 10 seconds as I really did not wanted to investigate that too.

The DC -> ACME ones could profit from update/diffstore to get changes more
live, even if seldom made - they are really cheap to query.

___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


Re: [pve-devel] pve-firewall 4.1 broke security group

2020-05-06 Thread Daniel Berteaud
- Le 6 Mai 20, à 17:46, Thomas Lamprecht t.lampre...@proxmox.com a écrit :

> On 5/6/20 5:28 PM, Thomas Lamprecht wrote:
>> On 5/6/20 5:21 PM, Daniel Berteaud wrote:
>>> Just opened [ https://bugzilla.proxmox.com/show_bug.cgi?id=2719 |
>>> https://bugzilla.proxmox.com/show_bug.cgi?id=2719 ]
>>> It's rather important, as it might cut remote access (and not everyone has 
>>> an
>>> IPMI console available)
>>>
>>> ++
>>>
>> 
>> I'll take a look, I know which patch just checking if revert or followup is
>> the better/quicker fix. Thanks for noticing us!
> 
> revert it was, rolled out an update as pve-firewall in version 4.1-2

That was a quick fix !
I confirm it's working again now.

Thanks

++

-- 
[ https://www.firewall-services.com/ ]  
Daniel Berteaud 
FIREWALL-SERVICES SAS, La sécurité des réseaux 
Société de Services en Logiciels Libres 
Tél : +33.5 56 64 15 32 
Matrix: @dani:fws.fr 
[ https://www.firewall-services.com/ | https://www.firewall-services.com ]

___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


Re: [pve-devel] pve-firewall 4.1 broke security group

2020-05-06 Thread Thomas Lamprecht
On 5/6/20 5:28 PM, Thomas Lamprecht wrote:
> On 5/6/20 5:21 PM, Daniel Berteaud wrote:
>> Just opened [ https://bugzilla.proxmox.com/show_bug.cgi?id=2719 | 
>> https://bugzilla.proxmox.com/show_bug.cgi?id=2719 ] 
>> It's rather important, as it might cut remote access (and not everyone has 
>> an IPMI console available) 
>>
>> ++ 
>>
> 
> I'll take a look, I know which patch just checking if revert or followup is
> the better/quicker fix. Thanks for noticing us!

revert it was, rolled out an update as pve-firewall in version 4.1-2

___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


Re: [pve-devel] pve-firewall 4.1 broke security group

2020-05-06 Thread Thomas Lamprecht
On 5/6/20 5:21 PM, Daniel Berteaud wrote:
> Just opened [ https://bugzilla.proxmox.com/show_bug.cgi?id=2719 | 
> https://bugzilla.proxmox.com/show_bug.cgi?id=2719 ] 
> It's rather important, as it might cut remote access (and not everyone has an 
> IPMI console available) 
> 
> ++ 
> 

I'll take a look, I know which patch just checking if revert or followup is
the better/quicker fix. Thanks for noticing us!

___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


Re: [pve-devel] applied: [PATCH v2 qemu-server 2/6] api: allow listing custom and default CPU models

2020-05-06 Thread Thomas Lamprecht
On 5/6/20 5:20 PM, Stefan Reiter wrote:
> On 5/6/20 4:52 PM, Thomas Lamprecht wrote:
>> On 5/4/20 12:58 PM, Stefan Reiter wrote:
>>> More API calls will follow for this path, for now add the 'index' call to
>>> list all custom and default CPU models.
>>>
>>> Any user can list the default CPU models, as these are public anyway, but
>>> custom models are restricted to users with Sys.Audit on /nodes.
>>>
>>> Signed-off-by: Stefan Reiter 
>>> ---
>>>   PVE/API2/Qemu/CPU.pm    | 61 +
>>>   PVE/API2/Qemu/Makefile  |  2 +-
>>>   PVE/QemuServer/CPUConfig.pm | 30 ++
>>>   3 files changed, 92 insertions(+), 1 deletion(-)
>>>   create mode 100644 PVE/API2/Qemu/CPU.pm
>>>
>>
>> applied, thanks!
>>
>> Albeit API call shouldn't say that it's return values are links to API sub
>> directory/paths if they ain't..
>>
> 
> Ah sorry, I removed this locally but must've forgotten to check it in.
> 
> It's probably best to remove this entirely for now, it's not just wrong now 
> but also wrong wrt. my upcoming changes... (I put the individual queries 
> under .../cpu/model/ , so I could also have .../cpu/supported-flags 
> etc. without mixing dynamic and static paths)

send a patch for whatever, and reply to series not which are, or will be
soon, outdated - thanks!


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] pve-firewall 4.1 broke security group

2020-05-06 Thread Daniel Berteaud
Just opened [ https://bugzilla.proxmox.com/show_bug.cgi?id=2719 | 
https://bugzilla.proxmox.com/show_bug.cgi?id=2719 ] 
It's rather important, as it might cut remote access (and not everyone has an 
IPMI console available) 

++ 

-- 


[ https://www.firewall-services.com/ ]  
Daniel Berteaud 
FIREWALL-SERVICES SAS, La sécurité des réseaux 
Société de Services en Logiciels Libres 
Tél : +33.5 56 64 15 32 
Matrix: @dani:fws.fr 
[ https://www.firewall-services.com/ | https://www.firewall-services.com ] 
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


Re: [pve-devel] applied: [PATCH v2 qemu-server 2/6] api: allow listing custom and default CPU models

2020-05-06 Thread Stefan Reiter

On 5/6/20 4:52 PM, Thomas Lamprecht wrote:

On 5/4/20 12:58 PM, Stefan Reiter wrote:

More API calls will follow for this path, for now add the 'index' call to
list all custom and default CPU models.

Any user can list the default CPU models, as these are public anyway, but
custom models are restricted to users with Sys.Audit on /nodes.

Signed-off-by: Stefan Reiter 
---
  PVE/API2/Qemu/CPU.pm| 61 +
  PVE/API2/Qemu/Makefile  |  2 +-
  PVE/QemuServer/CPUConfig.pm | 30 ++
  3 files changed, 92 insertions(+), 1 deletion(-)
  create mode 100644 PVE/API2/Qemu/CPU.pm



applied, thanks!

Albeit API call shouldn't say that it's return values are links to API sub
directory/paths if they ain't..



Ah sorry, I removed this locally but must've forgotten to check it in.

It's probably best to remove this entirely for now, it's not just wrong 
now but also wrong wrt. my upcoming changes... (I put the individual 
queries under .../cpu/model/ , so I could also have 
.../cpu/supported-flags etc. without mixing dynamic and static paths)


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] applied-series: [PATCH v3 0/3] Support all 8 corosync3 links in GUI

2020-05-06 Thread Thomas Lamprecht
On 3/23/20 1:41 PM, Stefan Reiter wrote:
> v2 -> v3:
> * add patch 1 (localization fix)
> * implement changes from Dominik's review:
>   * use 'let' in new code
>   * use references for element lookup
>   * some code style nits
>   * fix formatting (simpler in general with hbox, and also should work for all
> languages now)
>   * fix IPv6 address selection
> 
> Note: I didn't include any changes to onInputTypeChange as proposed by Dominik
> (using bindings instead of set-function calls). I couldn't quite wrap my head
> around the ExtJS side of that, so usually I'd like to talk this through in
> person, but since Dominik's on vacation and talking face-to-face right now 
> is...
> well, not recommended in general, I left it out for now.
> 
> This could easily be done in a followup though and wouldn't change the
> interface for the user, so I hope that's okay.
> 
> RFC -> v2:
> * rebased on master
> * slight rewording
> 
> 
> manager: Stefan Reiter (3):
>   gui/cluster: fix translation for cluster join button
>   gui/cluster: add CorosyncLinkEdit component to support up to 8 links
>   gui/cluster: add structured peerLinks to join info
> 
>  www/manager6/Makefile   |   1 +
>  www/manager6/dc/Cluster.js  |  13 +-
>  www/manager6/dc/ClusterEdit.js  | 194 ++---
>  www/manager6/dc/CorosyncLinkEdit.js | 425 
>  4 files changed, 534 insertions(+), 99 deletions(-)
>  create mode 100644 www/manager6/dc/CorosyncLinkEdit.js
> 

applied remaining parts of series with some style changes as followup,
please holler at me if something is off. Thanks!

___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] applied: [PATCH widget-toolkit] add missing htmlEncodes

2020-05-06 Thread Thomas Lamprecht
On 4/30/20 4:03 PM, Dominik Csapak wrote:
> username can include some special characters, so we have
> to escape them
> 
> Signed-off-by: Dominik Csapak 
> ---
>  window/TaskViewer.js | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
> 
> diff --git a/window/TaskViewer.js b/window/TaskViewer.js
> index 347542e..bbbf716 100644
> --- a/window/TaskViewer.js
> +++ b/window/TaskViewer.js
> @@ -127,6 +127,7 @@ Ext.define('Proxmox.window.TaskViewer', {
>   },
>   user: {
>   header: gettext('User name'),
> + renderer: Ext.String.htmlEncode,
>   required: true
>   },
>   node: {
> @@ -146,7 +147,8 @@ Ext.define('Proxmox.window.TaskViewer', {
>   renderer: Proxmox.Utils.render_timestamp
>   },
>   upid: {
> - header: gettext('Unique task ID')
> + header: gettext('Unique task ID'),
> + renderer: Ext.String.htmlEncode,
>   }
>   };
>  
> 

applied, thanks!

___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] applied: [PATCH manager] ui: fix missing htmlEncodes

2020-05-06 Thread Thomas Lamprecht
On 4/30/20 4:04 PM, Dominik Csapak wrote:
> username can include some special characters, so we have
> to escape them
> 
> Signed-off-by: Dominik Csapak 
> ---
>  www/manager6/Workspace.js  | 2 +-
>  www/manager6/dc/ACLView.js | 2 +-
>  www/manager6/dc/GroupView.js   | 1 +
>  www/manager6/dc/Log.js | 2 ++
>  www/manager6/dc/PermissionView.js  | 3 ++-
>  www/manager6/dc/TFAEdit.js | 1 +
>  www/manager6/dc/Tasks.js   | 1 +
>  www/manager6/dc/TokenEdit.js   | 1 +
>  www/manager6/dc/TokenView.js   | 4 ++--
>  www/manager6/dc/UserEdit.js| 1 +
>  www/manager6/dc/UserView.js| 4 ++--
>  www/manager6/form/GroupSelector.js | 1 +
>  www/manager6/form/TokenSelector.js | 1 +
>  www/manager6/form/UserSelector.js  | 1 +
>  www/manager6/window/Settings.js| 2 +-
>  15 files changed, 19 insertions(+), 8 deletions(-)
> 

applied, thanks!


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] applied: [PATCH v2 qemu-server 2/6] api: allow listing custom and default CPU models

2020-05-06 Thread Thomas Lamprecht
On 5/4/20 12:58 PM, Stefan Reiter wrote:
> More API calls will follow for this path, for now add the 'index' call to
> list all custom and default CPU models.
> 
> Any user can list the default CPU models, as these are public anyway, but
> custom models are restricted to users with Sys.Audit on /nodes.
> 
> Signed-off-by: Stefan Reiter 
> ---
>  PVE/API2/Qemu/CPU.pm| 61 +
>  PVE/API2/Qemu/Makefile  |  2 +-
>  PVE/QemuServer/CPUConfig.pm | 30 ++
>  3 files changed, 92 insertions(+), 1 deletion(-)
>  create mode 100644 PVE/API2/Qemu/CPU.pm
> 

applied, thanks!

Albeit API call shouldn't say that it's return values are links to API sub
directory/paths if they ain't..

___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] applied: [PATCH v2 qemu-server 1/6] api: check Sys.Audit permissions when setting a custom CPU model

2020-05-06 Thread Thomas Lamprecht
On 5/4/20 12:58 PM, Stefan Reiter wrote:
> Explicitly allows changing other properties than the cputype, even if
> the currently set cputype is not accessible by the user. This way, an
> administrator can assign a custom CPU type to a VM for a less privileged
> user without breaking edit functionality for them.
> 
> Signed-off-by: Stefan Reiter 
> ---
>  PVE/API2/Qemu.pm | 25 +
>  1 file changed, 25 insertions(+)
> 

applied, thanks!

___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] applied: [PATCH access-control 1/2] whitespace cleanup

2020-05-06 Thread Thomas Lamprecht
On 5/6/20 2:00 PM, Tim Marx wrote:
> Signed-off-by: Tim Marx 
> ---
>  PVE/API2/AccessControl.pm | 48 +++
>  1 file changed, 24 insertions(+), 24 deletions(-)
> 

applied, thanks!


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] applied: [PATCH manager 1/3] ui: whitespace cleanup

2020-05-06 Thread Thomas Lamprecht
On 5/6/20 2:00 PM, Tim Marx wrote:
> Signed-off-by: Tim Marx 
> ---
>  www/manager6/Workspace.js | 22 +++---
>  1 file changed, 11 insertions(+), 11 deletions(-)

applied, thanks!

___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] applied: [PATCH manager 3/3] ui: auth: clear ui capabilities on logout

2020-05-06 Thread Thomas Lamprecht
On 5/6/20 2:00 PM, Tim Marx wrote:
> Signed-off-by: Tim Marx 
> ---
>  www/manager6/Workspace.js | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/www/manager6/Workspace.js b/www/manager6/Workspace.js
> index 20d8c692..ffc5b175 100644
> --- a/www/manager6/Workspace.js
> +++ b/www/manager6/Workspace.js
> @@ -37,6 +37,7 @@ Ext.define('PVE.Workspace', {
>   var me = this;
>  
>   Proxmox.Utils.authClear();
> + Ext.state.Manager.clear('GuiCap');
>   Proxmox.UserName = null;
>   me.loginData = null;
>  
> 

applied, thanks!

___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


Re: [pve-devel] [PATCH manager] gui: never collapse notes for templates

2020-05-06 Thread Thomas Lamprecht
On 5/6/20 12:34 PM, Stefan Reiter wrote:
> There's no graphs on screen, so no reason to collapse the notes to save
> space. Besides, it looked a bit funky expanding the notes on smaller
> screens, since they always jumped to the bottom to fill the space...
> 
> Signed-off-by: Stefan Reiter 
> ---
>  www/manager6/panel/NotesView.js | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/www/manager6/panel/NotesView.js b/www/manager6/panel/NotesView.js
> index edf38e5d..90de7b5a 100644
> --- a/www/manager6/panel/NotesView.js
> +++ b/www/manager6/panel/NotesView.js
> @@ -105,7 +105,7 @@ Ext.define('PVE.panel.NotesView', {
>   me.callParent();
>   if (type === 'node') {
>   me.down('#tbar').setVisible(true);
> - } else {
> + } else if (me.pveSelNode.data.template !== 1) {
>   me.setCollapsible(true);
>   me.collapseDirection = 'right';
>  
> 

applied, thanks!

___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] applied: [PATCH manager v2] Fix #1210: ceph: extend pveceph purge

2020-05-06 Thread Thomas Lamprecht
On 5/5/20 5:27 PM, Alwin Antreich wrote:
> to clean service directories as well as disable and stop Ceph services.
> Addtionally provide the option to remove crash and log information.
> 
> This patch is also in addtion to #2607, as the current cleanup doesn't
> allow to re-configure Ceph, without manual steps during purge.
> 
> Signed-off-by: Alwin Antreich 
> ---
> v1 -> v2:
> * incorporate Thomas suggestions. Thanks.
>   - add warning for failed ceph connection
>   - use grep instead of map
>   - change $ceph variable name to $service in purge methods
> 
>  PVE/CLI/pveceph.pm | 48 ++-
>  PVE/Ceph/Tools.pm  | 71 ++
>  2 files changed, 100 insertions(+), 19 deletions(-)
> 

applied, thanks!

___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] applied: [PATCH v2 manager] Improve storage selection on restore

2020-05-06 Thread Thomas Lamprecht
On 5/6/20 10:14 AM, Fabian Ebner wrote:
> Previously, the blank '' would be passed along and lead to a
> parameter verfication failure.
> 
> For LXC the default behavior in the backend is to use 'local', so
> disallow blank and auto-select the first storage supporting'rootdir'
> instead.
> 
> For QEMU the default behavior in the backend is to use the
> original layout from the backup configuration file, which
> makes sense to use as the default in the GUI as well.
> 
> Signed-off-by: Fabian Ebner 
> ---
> 
> Changes from v1:
> * avoid unnecessary ?-operators
> * better emptyText
> 
>  www/manager6/window/Restore.js | 9 ++---
>  1 file changed, 6 insertions(+), 3 deletions(-)
> 

applied, thanks!

___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] [PATCH manager v2 2/5] ui: Parser: add printACME

2020-05-06 Thread Dominik Csapak
since we decode the domain list in parseACME into an array, we
have to join them again to a string when printing

otherwise printPropertyString attaches them just with ',' which
does not work here

Signed-off-by: Dominik Csapak 
---
 www/manager6/Parser.js | 7 +++
 1 file changed, 7 insertions(+)

diff --git a/www/manager6/Parser.js b/www/manager6/Parser.js
index 4cecb3e1..20d81d4a 100644
--- a/www/manager6/Parser.js
+++ b/www/manager6/Parser.js
@@ -5,6 +5,13 @@ Ext.define('PVE.Parser', { statics: {
 
 // this class only contains static functions
 
+printACME: function(value) {
+   if (Ext.isArray(value.domains)) {
+   value.domains = value.domains.join(';');
+   }
+   return PVE.Parser.printPropertyString(value);
+},
+
 parseACME: function(value) {
if (!value) {
return;
-- 
2.20.1


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] [PATCH manager v2 4/5] ui: node/ACME: add ACMEDomainEdit

2020-05-06 Thread Dominik Csapak
which expects a nodeconfig (for digest and domaincount)
and for the edit case, the parsed 'domain' object

this editwindow has three fields:
* type selector (standalone/dns)
* domain
* plugin (only for dns)

if the user chooses dns but there are already the maximum count of
acmedomainX entries, the type field gets invalid (with a error tooltip)

the onGetValues method is non-trivial, because of the mixing of
acmedomainX and acme.domain values, so we have to be careful
that we delete/edit the correct entry

Signed-off-by: Dominik Csapak 
---
 www/manager6/node/ACME.js | 136 ++
 1 file changed, 136 insertions(+)

diff --git a/www/manager6/node/ACME.js b/www/manager6/node/ACME.js
index 40ecc00e..a8bb39d6 100644
--- a/www/manager6/node/ACME.js
+++ b/www/manager6/node/ACME.js
@@ -230,6 +230,142 @@ Ext.define('PVE.node.ACMEAccountView', {
 }
 });
 
+Ext.define('PVE.node.ACMEDomainEdit', {
+extend: 'Proxmox.window.Edit',
+alias: 'widget.pveACMEDomainEdit',
+
+subject: gettext('Domain'),
+isCreate: false,
+
+items: [
+   {
+   xtype: 'inputpanel',
+   onGetValues: function(values) {
+   let me = this;
+   let win = me.up('pveACMEDomainEdit');
+   let nodeconfig = win.nodeconfig;
+   let olddomain = win.domain || {};
+
+   let params = {
+   digest: nodeconfig.digest,
+   };
+
+   let configkey = olddomain.configkey;
+   let acmeObj = PVE.Parser.parseACME(nodeconfig.acme) || {};
+
+   if (values.type === 'dns') {
+   if (!olddomain.configkey || olddomain.configkey === 'acme') 
{
+   // look for first free slot
+   for (let i = 0; i < PVE.Utils.acmedomain_count; i++) {
+   if (nodeconfig[`acmedomain${i}`] === undefined) {
+   configkey = `acmedomain${i}`;
+   break;
+   }
+   }
+   if (olddomain.domain) {
+   // we have to remove the domain from the acme 
domainlist
+   PVE.Utils.remove_domain_from_acme(acmeObj, 
olddomain.domain);
+   params.acme = PVE.Parser.printACME(acmeObj);
+   }
+   }
+
+   delete values.type;
+   params[configkey] = PVE.Parser.printPropertyString(values, 
'domain');
+   } else {
+   if (olddomain.configkey && olddomain.configkey !== 'acme') {
+   // delete the old dns entry
+   params.delete = [olddomain.configkey];
+   }
+
+   // add new, remove old and make entries unique
+   PVE.Utils.add_domain_to_acme(acmeObj, values.domain);
+   PVE.Utils.remove_domain_from_acme(acmeObj, 
olddomain.domain);
+   params.acme = PVE.Parser.printACME(acmeObj);
+   }
+
+   return params;
+   },
+   items: [
+   {
+   xtype: 'proxmoxKVComboBox',
+   name: 'type',
+   fieldLabel: gettext('Type'),
+   allowBlank: false,
+   comboItems: [
+   ['standalone', 'standalone'],
+   ['dns', 'DNS'],
+   ],
+   validator: function(value) {
+   let me = this;
+   let win = me.up('pveACMEDomainEdit');
+   let oldconfigkey = win.domain ? win.domain.configkey : 
undefined;
+   let val = me.getValue();
+   if (val === 'dns' && (!oldconfigkey || oldconfigkey === 
'acme')) {
+   // we have to check if there is a 'acmedomain' slot 
left
+   let found = false;
+   for (let i = 0; i < PVE.Utils.acmedomain_count; 
i++) {
+   if (!win.nodeconfig[`acmedomain${i}`]) {
+   found = true;
+   }
+   }
+   if (!found) {
+   return gettext('Only 5 Domains with type DNS 
can be configured');
+   }
+   }
+
+   return true;
+   },
+   listeners: {
+   change: function(cb, value) {
+   let me = this;
+   let view = me.up('pveACMEDomainEdit');
+   view.down('field[name=plugin]').setDisabled(value 
!== 'dns');
+   },
+   },
+   },
+   {
+   xtype: 'hidden',
+   

[pve-devel] [PATCH manager v2 5/5] ui: node/ACME: rework ACME grid for plugin based domains

2020-05-06 Thread Dominik Csapak
This is basically a complete rework of the ACME grid.
Instead of having an ObjectGrid, we now have a normal
GridPanel which allows us to show a row for each Domain.

But to achieve this, we need to manually fill the store with data
from the 'acme' and 'acmedomainX' entries of the node config.

We also add an AccountSelector to the tbar and a link to the
datacenter->acme panel (when there is no account)

this also removes the 'register account' and 'view account' buttons,
since those are now available in datacenter->acme

Signed-off-by: Dominik Csapak 
---
changes from v1:
* different approach for editing accounts
  uses now a displayfield/combobox and button to switch between
  display and edit, and only do it when the 'check' is clicked

* use viewmodel binding for account name, makes the code a little more
  compact

 www/manager6/node/ACME.js | 481 --
 1 file changed, 309 insertions(+), 172 deletions(-)

diff --git a/www/manager6/node/ACME.js b/www/manager6/node/ACME.js
index a8bb39d6..33159b2e 100644
--- a/www/manager6/node/ACME.js
+++ b/www/manager6/node/ACME.js
@@ -1,49 +1,3 @@
-Ext.define('PVE.node.ACMEEditor', {
-extend: 'Proxmox.window.Edit',
-xtype: 'pveACMEEditor',
-
-subject: gettext('Domains'),
-items: [
-   {
-   xtype: 'inputpanel',
-   items: [
-   {
-   xtype: 'textarea',
-   fieldLabel: gettext('Domains'),
-   emptyText: "domain1.example.com\ndomain2.example.com",
-   name: 'domains'
-   }
-   ],
-   onGetValues: function(values) {
-   if (!values.domains) {
-   return {
-   'delete': 'acme'
-   };
-   }
-   var domains = values.domains.split(/\n/).join(';');
-   return {
-   'acme': 'domains=' + domains
-   };
-   }
-   }
-],
-
-initComponent: function() {
-   var me = this;
-   me.callParent();
-
-   me.load({
-   success: function(response, opts) {
-   var res = PVE.Parser.parseACME(response.result.data.acme);
-   if (res) {
-   res.domains = res.domains.join(' ');
-   me.setValues(res);
-   }
-   }
-   });
-}
-});
-
 Ext.define('PVE.node.ACMEAccountCreate', {
 extend: 'Proxmox.window.Edit',
 
@@ -366,138 +320,330 @@ Ext.define('PVE.node.ACMEDomainEdit', {
 },
 });
 
+Ext.define('pve-acme-domains', {
+extend: 'Ext.data.Model',
+fields: ['domain', 'type', 'alias', 'plugin', 'configkey'],
+idProperty: 'domain',
+});
+
 Ext.define('PVE.node.ACME', {
-extend: 'Proxmox.grid.ObjectGrid',
-xtype: 'pveACMEView',
+extend: 'Ext.grid.Panel',
+alias: 'widget.pveACMEView',
 
 margin: '10 0 0 0',
 title: 'ACME',
 
+viewModel: {
+   data: {
+   account: null,
+   accountEditable: false,
+   },
+
+   formulas: {
+   editBtnIcon: (get) => {
+   return 'fa black fa-' + (get('accountEditable') ? 'check' : 
'pencil');
+   },
+   },
+},
+
+controller: {
+   xclass: 'Ext.app.ViewController',
+
+   addDomain: function() {
+   let me = this;
+   let view = me.getView();
+
+   Ext.create('PVE.node.ACMEDomainEdit', {
+   nodename: view.nodename,
+   nodeconfig: view.nodeconfig,
+   apiCallDone: function() {
+   me.reload();
+   },
+   }).show();
+   },
+
+   editDomain: function() {
+   let me = this;
+   let view = me.getView();
+
+   let selection = view.getSelection();
+   if (selection.length < 1) return;
+
+   Ext.create('PVE.node.ACMEDomainEdit', {
+   nodename: view.nodename,
+   nodeconfig: view.nodeconfig,
+   domain: selection[0].data,
+   apiCallDone: function() {
+   me.reload();
+   },
+   }).show();
+   },
+
+   removeDomain: function() {
+   let me = this;
+   let view = me.getView();
+   let selection = view.getSelection();
+   if (selection.length < 1) return;
+
+   let rec = selection[0].data;
+   let params = {};
+   if (rec.configkey !== 'acme') {
+   params.delete = rec.configkey;
+   } else {
+   let acme = PVE.Parser.parseACME(view.nodeconfig.acme);
+   PVE.Utils.remove_domain_from_acme(acme, rec.domain);
+   params.acme = PVE.Parser.printACME(acme);
+   }
+
+   Proxmox.Utils.API2Request({
+   method: 'PUT',
+   url: `/nodes/${view.nodename}/config`,
+   params,
+   success: function(response, opt) {
+   me.reload();
+ 

[pve-devel] [PATCH manager v2 3/5] ui: Utils: add helper functions for acme domains

2020-05-06 Thread Dominik Csapak
to convieniently add and remove domains from a parsed ACME object
they also make domains unique in the array

also add the count of configureable acmedomainX entries

Signed-off-by: Dominik Csapak 
---
 www/manager6/Utils.js | 23 +++
 1 file changed, 23 insertions(+)

diff --git a/www/manager6/Utils.js b/www/manager6/Utils.js
index 872b7c29..31e262c0 100644
--- a/www/manager6/Utils.js
+++ b/www/manager6/Utils.js
@@ -1354,6 +1354,29 @@ Ext.define('PVE.Utils', { utilities: {
}
 },
 
+acmedomain_count: 5,
+
+add_domain_to_acme: function(acme, domain) {
+   if (acme.domains === undefined) {
+   acme.domains = [domain];
+   } else {
+   acme.domains.push(domain);
+   acme.domains = acme.domains.filter((value, index, self) => {
+   return self.indexOf(value) === index;
+   });
+   }
+   return acme;
+},
+
+remove_domain_from_acme: function(acme, domain) {
+   if (acme.domains !== undefined) {
+   acme.domains = acme.domains.filter((value, index, self) => {
+   return self.indexOf(value) === index && value !== domain;
+   });
+   }
+   return acme;
+},
+
 handleStoreErrorOrMask: function(me, store, regex, callback) {
 
me.mon(store, 'load', function (proxy, response, success, operation) {
-- 
2.20.1


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] [PATCH manager v2 1/5] ui: add ACME selector formfields for account and plugins

2020-05-06 Thread Dominik Csapak
filter the plugins by type === 'dns'
and add a convenience method for ACMEAccountSelector to check if there
are any accounts

Signed-off-by: Dominik Csapak 
---
changes from v1:
* drop fieldLabel in ACMEAccountSelector
 www/manager6/Makefile|  2 ++
 www/manager6/form/ACMEAccountSelector.js | 21 +
 www/manager6/form/ACMEPluginSelector.js  | 19 +++
 3 files changed, 42 insertions(+)
 create mode 100644 www/manager6/form/ACMEAccountSelector.js
 create mode 100644 www/manager6/form/ACMEPluginSelector.js

diff --git a/www/manager6/Makefile b/www/manager6/Makefile
index e2214588..563e7dc5 100644
--- a/www/manager6/Makefile
+++ b/www/manager6/Makefile
@@ -73,6 +73,8 @@ JSSRC=
\
form/SDNControllerSelector.js   \
form/TFASelector.js \
form/ACMEAPiSelector.js \
+   form/ACMEAccountSelector.js \
+   form/ACMEPluginSelector.js  \
dc/Tasks.js \
dc/Log.js   \
panel/StatusPanel.js\
diff --git a/www/manager6/form/ACMEAccountSelector.js 
b/www/manager6/form/ACMEAccountSelector.js
new file mode 100644
index ..12d5848c
--- /dev/null
+++ b/www/manager6/form/ACMEAccountSelector.js
@@ -0,0 +1,21 @@
+Ext.define('PVE.form.ACMEAccountSelector', {
+extend: 'Ext.form.field.ComboBox',
+alias: 'widget.pveACMEAccountSelector',
+
+displayField: 'name',
+valueField: 'name',
+
+store: {
+   model: 'pve-acme-accounts',
+   autoLoad: true,
+},
+
+triggerAction: 'all',
+queryMode: 'local',
+allowBlank: false,
+editable: false,
+
+isEmpty: function() {
+   return this.getStore().getData().length === 0;
+}
+});
diff --git a/www/manager6/form/ACMEPluginSelector.js 
b/www/manager6/form/ACMEPluginSelector.js
new file mode 100644
index ..ccce97b8
--- /dev/null
+++ b/www/manager6/form/ACMEPluginSelector.js
@@ -0,0 +1,19 @@
+Ext.define('PVE.form.ACMEPluginSelector', {
+extend: 'Ext.form.field.ComboBox',
+alias: 'widget.pveACMEPluginSelector',
+
+fieldLabel: gettext('Plugin'),
+displayField: 'plugin',
+valueField: 'plugin',
+
+store: {
+   model: 'pve-acme-plugins',
+   autoLoad: true,
+   filters: item => item.data.type === 'dns',
+},
+
+triggerAction: 'all',
+queryMode: 'local',
+allowBlank: false,
+editable: false,
+});
-- 
2.20.1


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] [PATCH manager v2 0/5] ACME node adaptions for plugins

2020-05-06 Thread Dominik Csapak
this series adapts the node->certificates->acme panel to include
an 'accountselector' and to be able to add/edit/remove single domains,
including ones with a plugin

changes from v1:
* drop fieldLabel in ACMEAccountSelector
* reword 'Account' in 'Used Account'
* use different approach to change the account
  (use viewModel and a displayfield/combobox/editbutton to better
  see when the account actually changes)

Dominik Csapak (5):
  ui: add ACME selector formfields for account and plugins
  ui: Parser: add printACME
  ui: Utils: add helper functions for acme domains
  ui: node/ACME: add ACMEDomainEdit
  ui: node/ACME: rework ACME grid for plugin based domains

 www/manager6/Makefile|   2 +
 www/manager6/Parser.js   |   7 +
 www/manager6/Utils.js|  23 +
 www/manager6/form/ACMEAccountSelector.js |  21 +
 www/manager6/form/ACMEPluginSelector.js  |  19 +
 www/manager6/node/ACME.js| 617 ---
 6 files changed, 517 insertions(+), 172 deletions(-)
 create mode 100644 www/manager6/form/ACMEAccountSelector.js
 create mode 100644 www/manager6/form/ACMEPluginSelector.js

-- 
2.20.1


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] applied: [PATCH RESEND qemu-server] migrate: cleanup forwarding code

2020-05-06 Thread Thomas Lamprecht
On 5/5/20 1:07 PM, Fabian Grünbichler wrote:
> fixing the following two issues:
> - the legacy code path was never converted to the new fork_tunnel
> signature (which probably means that nothing triggers it in practice
> anymore?)
> - the NBD Unix socket got forwarded multiple times if more than one disk
> was migrated via NBD (this is harmless, but wrong)
> 
> for the second issue I opted to keep the code compatible with the
> possibility that Qemu starts supporting multiple NBD servers in the
> future (and the target node could thus return multiple UNIX socket
> paths). currently we can only start one NBD server on one socket, and
> each drive-mirror simply starts a new connection over that single
> socket.
> 
> I took the liberty of renaming the variables/keys since I found
> 'tunnel_addr' and 'sock_addr' rather confusing.
> 
> Reviewed-By: Mira Limbeck 
> Tested-By: Mira Limbeck 
> 
> Signed-off-by: Fabian Grünbichler 
> ---
> Resend for nbdstop context change
> 
>  PVE/QemuMigrate.pm | 41 ++---
>  1 file changed, 22 insertions(+), 19 deletions(-)
> 

applied, thanks!


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] applied-series: [PATCH container] vzdump: use new 'pbs' option

2020-05-06 Thread Thomas Lamprecht
On 5/6/20 10:57 AM, Fabian Grünbichler wrote:
> instead of storage config to determine whether we are in 'PBS mode'
> 
> Signed-off-by: Fabian Grünbichler 
> ---
> 
> Notes:
> requires a break on pve-manager << version setting this option,
> since the dependency is the other way round
> 
>  src/PVE/VZDump/LXC.pm | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/src/PVE/VZDump/LXC.pm b/src/PVE/VZDump/LXC.pm
> index 2d003d0..45a3d8f 100644
> --- a/src/PVE/VZDump/LXC.pm
> +++ b/src/PVE/VZDump/LXC.pm
> @@ -303,7 +303,7 @@ sub assemble {
>  my $firewall ="/etc/pve/firewall/$vmid.fw";
>  my $fwconftmp = "$tmpdir/etc/vzdump/pct.fw";
>  
> -if ($opts->{scfg}->{type} eq 'pbs') {
> +if ($self->{vzdump}->{opts}->{pbs}) {
>   # fixme: do not store pct.conf and fw.conf into $tmpdir
>   if (-e  $firewall) {
>   PVE::Tools::file_copy($firewall, $fwconftmp);
> @@ -356,7 +356,7 @@ sub archive {
>  
>  my $userns_cmd = $task->{userns_cmd};
>  
> -if ($opts->{scfg}->{type} eq 'pbs') {
> +if ($self->{vzdump}->{opts}->{pbs}) {
>  
>   my $rootdir = $default_mount_point;
>   my $param = [];
> 

applied series


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] applied: [PATCH docs v3] add documenation for ldap syncing

2020-05-06 Thread Thomas Lamprecht
On 5/4/20 3:32 PM, Dominik Csapak wrote:
> explaining the main Requirements and limitations, as well as the
> most important sync options
> 
> Signed-off-by: Dominik Csapak 
> ---
> changes from v2:
> * incorporated suggestions from aaron
> @aaron, regarding linking to character limitations,
> sadly no, this is a sub based format, so even if we would have a place
> were we could link to for formats (we don't afaik) since it's sub
> based there would be no auto-generated information
> 
> just for completeness, atm the limit is >2 and <60 characters, and no
> '/', ':'
> 
> (maybe we can change it to a regex?)
> 
>  pveum.adoc | 48 
>  1 file changed, 48 insertions(+)
> 

applied, thanks!


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] applied: [PATCH qemu-server] cfg2cmd: set audiodev parameter only on qemu>=4.2

2020-05-06 Thread Thomas Lamprecht
On 5/6/20 2:51 PM, Aaron Lauterer wrote:
> fixes behavior introduced with commit
> 940e2a3a06b3ea47aae144519e2aaa881a80e437
> 
> Qemu 4.1 will fail to start a guest with an audio device set with
> `Property '.audiodev' not found`.
> 
> Signed-off-by: Aaron Lauterer 
> ---
> 
> Users reported this problem in the forum today [0].
> 
> I was able to reproduce this problem which makes me somewhat dumbfounded
> how I missed that while testing the initial commit... :/
> 

I even asked you about that specific case ;-)
https://pve.proxmox.com/pipermail/pve-devel/2020-March/042643.html

applied

> [0] 
> https://forum.proxmox.com/threads/kein-kvm-start-mehr-nach-heutigem-update.69379
> 
>  PVE/QemuServer.pm | 15 +--
>  1 file changed, 9 insertions(+), 6 deletions(-)
> 
> diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm
> index cb96b71..c50f102 100644
> --- a/PVE/QemuServer.pm
> +++ b/PVE/QemuServer.pm
> @@ -2752,19 +2752,22 @@ sub conf_has_audio {
>  }
>  
>  sub audio_devs {
> -my ($audio, $audiopciaddr) = @_;
> +my ($audio, $audiopciaddr, $machine_version) = @_;
>  
>  my $devs = [];
>  
>  my $id = $audio->{dev_id};
> -my $audiodev = "audiodev=$audio->{backend_id}";
> +my $audiodev = "";
> +if (min_version($machine_version, 4, 2)) {
> + $audiodev = ",audiodev=$audio->{backend_id}";
> +}
>  
>  if ($audio->{dev} eq 'AC97') {
> - push @$devs, '-device', "AC97,id=${id}${audiopciaddr},$audiodev";
> + push @$devs, '-device', "AC97,id=${id}${audiopciaddr}$audiodev";
>  } elsif ($audio->{dev} =~ /intel\-hda$/) {
>   push @$devs, '-device', "$audio->{dev},id=${id}${audiopciaddr}";
> - push @$devs, '-device', 
> "hda-micro,id=${id}-codec0,bus=${id}.0,cad=0,$audiodev";
> - push @$devs, '-device', 
> "hda-duplex,id=${id}-codec1,bus=${id}.0,cad=1,$audiodev";
> + push @$devs, '-device', 
> "hda-micro,id=${id}-codec0,bus=${id}.0,cad=0$audiodev";
> + push @$devs, '-device', 
> "hda-duplex,id=${id}-codec1,bus=${id}.0,cad=1$audiodev";
>  } else {
>   die "unkown audio device '$audio->{dev}', implement me!";
>  }
> @@ -3275,7 +3278,7 @@ sub config_to_command {
>  
>  if (min_version($machine_version, 4, 0) && (my $audio = 
> conf_has_audio($conf))) {
>   my $audiopciaddr = print_pci_addr("audio0", $bridges, $arch, 
> $machine_type);
> - my $audio_devs = audio_devs($audio, $audiopciaddr);
> + my $audio_devs = audio_devs($audio, $audiopciaddr, $machine_version);
>   push @$devices, @$audio_devs;
>  }
>  
> 


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] [PATCH qemu-server] cfg2cmd: set audiodev parameter only on qemu>=4.2

2020-05-06 Thread Aaron Lauterer
fixes behavior introduced with commit
940e2a3a06b3ea47aae144519e2aaa881a80e437

Qemu 4.1 will fail to start a guest with an audio device set with
`Property '.audiodev' not found`.

Signed-off-by: Aaron Lauterer 
---

Users reported this problem in the forum today [0].

I was able to reproduce this problem which makes me somewhat dumbfounded
how I missed that while testing the initial commit... :/

[0] 
https://forum.proxmox.com/threads/kein-kvm-start-mehr-nach-heutigem-update.69379

 PVE/QemuServer.pm | 15 +--
 1 file changed, 9 insertions(+), 6 deletions(-)

diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm
index cb96b71..c50f102 100644
--- a/PVE/QemuServer.pm
+++ b/PVE/QemuServer.pm
@@ -2752,19 +2752,22 @@ sub conf_has_audio {
 }
 
 sub audio_devs {
-my ($audio, $audiopciaddr) = @_;
+my ($audio, $audiopciaddr, $machine_version) = @_;
 
 my $devs = [];
 
 my $id = $audio->{dev_id};
-my $audiodev = "audiodev=$audio->{backend_id}";
+my $audiodev = "";
+if (min_version($machine_version, 4, 2)) {
+   $audiodev = ",audiodev=$audio->{backend_id}";
+}
 
 if ($audio->{dev} eq 'AC97') {
-   push @$devs, '-device', "AC97,id=${id}${audiopciaddr},$audiodev";
+   push @$devs, '-device', "AC97,id=${id}${audiopciaddr}$audiodev";
 } elsif ($audio->{dev} =~ /intel\-hda$/) {
push @$devs, '-device', "$audio->{dev},id=${id}${audiopciaddr}";
-   push @$devs, '-device', 
"hda-micro,id=${id}-codec0,bus=${id}.0,cad=0,$audiodev";
-   push @$devs, '-device', 
"hda-duplex,id=${id}-codec1,bus=${id}.0,cad=1,$audiodev";
+   push @$devs, '-device', 
"hda-micro,id=${id}-codec0,bus=${id}.0,cad=0$audiodev";
+   push @$devs, '-device', 
"hda-duplex,id=${id}-codec1,bus=${id}.0,cad=1$audiodev";
 } else {
die "unkown audio device '$audio->{dev}', implement me!";
 }
@@ -3275,7 +3278,7 @@ sub config_to_command {
 
 if (min_version($machine_version, 4, 0) && (my $audio = 
conf_has_audio($conf))) {
my $audiopciaddr = print_pci_addr("audio0", $bridges, $arch, 
$machine_type);
-   my $audio_devs = audio_devs($audio, $audiopciaddr);
+   my $audio_devs = audio_devs($audio, $audiopciaddr, $machine_version);
push @$devices, @$audio_devs;
 }
 
-- 
2.20.1


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


Re: [pve-devel] [PATCH qemu-server] cfg2cmd: fix uninitialized value warning on OVMF w/o efidisk

2020-05-06 Thread Thomas Lamprecht
On 5/6/20 2:17 PM, Stefan Reiter wrote:
> It's possible to have a VM with OVMF but without an efidisk, so don't
> call parse_drive on a potential undef value.
> 
> Partial revert of 818c3b8d91 ("cfg2cmd: ovmf: code cleanup")
> 
> Signed-off-by: Stefan Reiter 
> ---
>  PVE/QemuServer.pm | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm
> index e9b094b..e76aee3 100644
> --- a/PVE/QemuServer.pm
> +++ b/PVE/QemuServer.pm
> @@ -3084,7 +3084,8 @@ sub config_to_command {
>   die "uefi base image '$ovmf_code' not found\n" if ! -f $ovmf_code;
>  
>   my ($path, $format);
> - if (my $d = parse_drive('efidisk0', $conf->{efidisk0})) {
> + if (my $efidisk = $conf->{efidisk0}) {
> + my $d = parse_drive('efidisk0', $efidisk);
>   my ($storeid, $volname) = PVE::Storage::parse_volume_id($d->{file}, 
> 1);
>   $format = $d->{format};
>   if ($storeid) {
> 

applied, thanks!

___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


Re: [pve-devel] [PATCH qemu-server] vzdump: fix template backup to stdout

2020-05-06 Thread Thomas Lamprecht
On 5/6/20 11:56 AM, Fabian Grünbichler wrote:
> redirecting to the saved STDOUT in case of a template backup or a VM
> without any disks failed because of the erroneous '=':
> 
> Backup of VM 123123 failed - command '/usr/bin/vma create -v -c [...]' failed:
> Bad filehandle: =5 at /usr/share/perl/5.28/IPC/Open3.pm line 58.
> 
> https://forum.proxmox.com/threads/vzdump-to-stdout.69364
> 
> Signed-off-by: Fabian Grünbichler 
> ---
>  PVE/VZDump/QemuServer.pm | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 

applied, thanks!


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] applied: [PATCH v2 qemu-server] rng: die when trying to pass through disconnected hwrng

2020-05-06 Thread Thomas Lamprecht
On 5/5/20 4:53 PM, Stefan Reiter wrote:
> If /dev/hwrng exists, but no actual generator is connected (or it is
> disabled on the host), QEMU will happily start the VM but crash as soon
> as the guest accesses the VirtIO RNG device.
> 
> To prevent this unfortunate behaviour, check if a useable hwrng is
> connected to the host before allowing the VM to be started.
> 
> While at it, clean up config_to_command by moving new and existing rng
> source checks to a seperate sub.
> 
> Signed-off-by: Stefan Reiter 
> ---
> 
> v2: Move to sub, clean up, extend comment
> 
>  PVE/QemuServer.pm | 26 --
>  1 file changed, 20 insertions(+), 6 deletions(-)
> 

applied, thanks!

___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] [PATCH qemu-server] cfg2cmd: fix uninitialized value warning on OVMF w/o efidisk

2020-05-06 Thread Stefan Reiter
It's possible to have a VM with OVMF but without an efidisk, so don't
call parse_drive on a potential undef value.

Partial revert of 818c3b8d91 ("cfg2cmd: ovmf: code cleanup")

Signed-off-by: Stefan Reiter 
---
 PVE/QemuServer.pm | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm
index e9b094b..e76aee3 100644
--- a/PVE/QemuServer.pm
+++ b/PVE/QemuServer.pm
@@ -3084,7 +3084,8 @@ sub config_to_command {
die "uefi base image '$ovmf_code' not found\n" if ! -f $ovmf_code;
 
my ($path, $format);
-   if (my $d = parse_drive('efidisk0', $conf->{efidisk0})) {
+   if (my $efidisk = $conf->{efidisk0}) {
+   my $d = parse_drive('efidisk0', $efidisk);
my ($storeid, $volname) = PVE::Storage::parse_volume_id($d->{file}, 
1);
$format = $d->{format};
if ($storeid) {
-- 
2.20.1


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] [PATCH manager 1/5] ui: add ACME selector formfields for account and plugins

2020-05-06 Thread Dominik Csapak
filter the plugins by type === 'dns'
and add a convenience method for ACMEAccountSelector to check if there
are any accounts

Signed-off-by: Dominik Csapak 
---
 www/manager6/Makefile|  2 ++
 www/manager6/form/ACMEAccountSelector.js | 22 ++
 www/manager6/form/ACMEPluginSelector.js  | 19 +++
 3 files changed, 43 insertions(+)
 create mode 100644 www/manager6/form/ACMEAccountSelector.js
 create mode 100644 www/manager6/form/ACMEPluginSelector.js

diff --git a/www/manager6/Makefile b/www/manager6/Makefile
index e2214588..563e7dc5 100644
--- a/www/manager6/Makefile
+++ b/www/manager6/Makefile
@@ -73,6 +73,8 @@ JSSRC=
\
form/SDNControllerSelector.js   \
form/TFASelector.js \
form/ACMEAPiSelector.js \
+   form/ACMEAccountSelector.js \
+   form/ACMEPluginSelector.js  \
dc/Tasks.js \
dc/Log.js   \
panel/StatusPanel.js\
diff --git a/www/manager6/form/ACMEAccountSelector.js 
b/www/manager6/form/ACMEAccountSelector.js
new file mode 100644
index ..5b533c1b
--- /dev/null
+++ b/www/manager6/form/ACMEAccountSelector.js
@@ -0,0 +1,22 @@
+Ext.define('PVE.form.ACMEAccountSelector', {
+extend: 'Ext.form.field.ComboBox',
+alias: 'widget.pveACMEAccountSelector',
+
+fieldLabel: gettext('Account'),
+displayField: 'name',
+valueField: 'name',
+
+store: {
+   model: 'pve-acme-accounts',
+   autoLoad: true,
+},
+
+triggerAction: 'all',
+queryMode: 'local',
+allowBlank: false,
+editable: false,
+
+isEmpty: function() {
+   return this.getStore().getData().length === 0;
+}
+});
diff --git a/www/manager6/form/ACMEPluginSelector.js 
b/www/manager6/form/ACMEPluginSelector.js
new file mode 100644
index ..ccce97b8
--- /dev/null
+++ b/www/manager6/form/ACMEPluginSelector.js
@@ -0,0 +1,19 @@
+Ext.define('PVE.form.ACMEPluginSelector', {
+extend: 'Ext.form.field.ComboBox',
+alias: 'widget.pveACMEPluginSelector',
+
+fieldLabel: gettext('Plugin'),
+displayField: 'plugin',
+valueField: 'plugin',
+
+store: {
+   model: 'pve-acme-plugins',
+   autoLoad: true,
+   filters: item => item.data.type === 'dns',
+},
+
+triggerAction: 'all',
+queryMode: 'local',
+allowBlank: false,
+editable: false,
+});
-- 
2.20.1


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] [PATCH manager 0/5] ACME node adaptions for plugins

2020-05-06 Thread Dominik Csapak
this series adapts the node->certificates->acme panel to include
an 'accountselector' and to be able to add/edit/remove single domains,
including ones with a plugin

Dominik Csapak (5):
  ui: add ACME selector formfields for account and plugins
  ui: Parser: add printACME
  ui: Utils: add helper functions for acme domains
  ui: node/ACME: add ACMEDomainEdit
  ui: node/ACME: rework ACME grid for plugin based domains

 www/manager6/Makefile|   2 +
 www/manager6/Parser.js   |   7 +
 www/manager6/Utils.js|  23 +
 www/manager6/form/ACMEAccountSelector.js |  22 +
 www/manager6/form/ACMEPluginSelector.js  |  19 +
 www/manager6/node/ACME.js| 556 ---
 6 files changed, 463 insertions(+), 166 deletions(-)
 create mode 100644 www/manager6/form/ACMEAccountSelector.js
 create mode 100644 www/manager6/form/ACMEPluginSelector.js

-- 
2.20.1


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] [PATCH manager 3/5] ui: Utils: add helper functions for acme domains

2020-05-06 Thread Dominik Csapak
to convieniently add and remove domains from a parsed ACME object
they also make domains unique in the array

also add the count of configureable acmedomainX entries

Signed-off-by: Dominik Csapak 
---
 www/manager6/Utils.js | 23 +++
 1 file changed, 23 insertions(+)

diff --git a/www/manager6/Utils.js b/www/manager6/Utils.js
index 872b7c29..31e262c0 100644
--- a/www/manager6/Utils.js
+++ b/www/manager6/Utils.js
@@ -1354,6 +1354,29 @@ Ext.define('PVE.Utils', { utilities: {
}
 },
 
+acmedomain_count: 5,
+
+add_domain_to_acme: function(acme, domain) {
+   if (acme.domains === undefined) {
+   acme.domains = [domain];
+   } else {
+   acme.domains.push(domain);
+   acme.domains = acme.domains.filter((value, index, self) => {
+   return self.indexOf(value) === index;
+   });
+   }
+   return acme;
+},
+
+remove_domain_from_acme: function(acme, domain) {
+   if (acme.domains !== undefined) {
+   acme.domains = acme.domains.filter((value, index, self) => {
+   return self.indexOf(value) === index && value !== domain;
+   });
+   }
+   return acme;
+},
+
 handleStoreErrorOrMask: function(me, store, regex, callback) {
 
me.mon(store, 'load', function (proxy, response, success, operation) {
-- 
2.20.1


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] [PATCH manager 2/5] ui: Parser: add printACME

2020-05-06 Thread Dominik Csapak
since we decode the domain list in parseACME into an array, we
have to join them again to a string when printing

otherwise printPropertyString attaches them just with ',' which
does not work here

Signed-off-by: Dominik Csapak 
---
 www/manager6/Parser.js | 7 +++
 1 file changed, 7 insertions(+)

diff --git a/www/manager6/Parser.js b/www/manager6/Parser.js
index 4cecb3e1..20d81d4a 100644
--- a/www/manager6/Parser.js
+++ b/www/manager6/Parser.js
@@ -5,6 +5,13 @@ Ext.define('PVE.Parser', { statics: {
 
 // this class only contains static functions
 
+printACME: function(value) {
+   if (Ext.isArray(value.domains)) {
+   value.domains = value.domains.join(';');
+   }
+   return PVE.Parser.printPropertyString(value);
+},
+
 parseACME: function(value) {
if (!value) {
return;
-- 
2.20.1


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] [PATCH manager 4/5] ui: node/ACME: add ACMEDomainEdit

2020-05-06 Thread Dominik Csapak
which expects a nodeconfig (for digest and domaincount)
and for the edit case, the parsed 'domain' object

this editwindow has three fields:
* type selector (standalone/dns)
* domain
* plugin (only for dns)

if the user chooses dns but there are already the maximum count of
acmedomainX entries, the type field gets invalid (with a error tooltip)

the onGetValues method is non-trivial, because of the mixing of
acmedomainX and acme.domain values, so we have to be careful
that we delete/edit the correct entry

Signed-off-by: Dominik Csapak 
---
 www/manager6/node/ACME.js | 136 ++
 1 file changed, 136 insertions(+)

diff --git a/www/manager6/node/ACME.js b/www/manager6/node/ACME.js
index 40ecc00e..a8bb39d6 100644
--- a/www/manager6/node/ACME.js
+++ b/www/manager6/node/ACME.js
@@ -230,6 +230,142 @@ Ext.define('PVE.node.ACMEAccountView', {
 }
 });
 
+Ext.define('PVE.node.ACMEDomainEdit', {
+extend: 'Proxmox.window.Edit',
+alias: 'widget.pveACMEDomainEdit',
+
+subject: gettext('Domain'),
+isCreate: false,
+
+items: [
+   {
+   xtype: 'inputpanel',
+   onGetValues: function(values) {
+   let me = this;
+   let win = me.up('pveACMEDomainEdit');
+   let nodeconfig = win.nodeconfig;
+   let olddomain = win.domain || {};
+
+   let params = {
+   digest: nodeconfig.digest,
+   };
+
+   let configkey = olddomain.configkey;
+   let acmeObj = PVE.Parser.parseACME(nodeconfig.acme) || {};
+
+   if (values.type === 'dns') {
+   if (!olddomain.configkey || olddomain.configkey === 'acme') 
{
+   // look for first free slot
+   for (let i = 0; i < PVE.Utils.acmedomain_count; i++) {
+   if (nodeconfig[`acmedomain${i}`] === undefined) {
+   configkey = `acmedomain${i}`;
+   break;
+   }
+   }
+   if (olddomain.domain) {
+   // we have to remove the domain from the acme 
domainlist
+   PVE.Utils.remove_domain_from_acme(acmeObj, 
olddomain.domain);
+   params.acme = PVE.Parser.printACME(acmeObj);
+   }
+   }
+
+   delete values.type;
+   params[configkey] = PVE.Parser.printPropertyString(values, 
'domain');
+   } else {
+   if (olddomain.configkey && olddomain.configkey !== 'acme') {
+   // delete the old dns entry
+   params.delete = [olddomain.configkey];
+   }
+
+   // add new, remove old and make entries unique
+   PVE.Utils.add_domain_to_acme(acmeObj, values.domain);
+   PVE.Utils.remove_domain_from_acme(acmeObj, 
olddomain.domain);
+   params.acme = PVE.Parser.printACME(acmeObj);
+   }
+
+   return params;
+   },
+   items: [
+   {
+   xtype: 'proxmoxKVComboBox',
+   name: 'type',
+   fieldLabel: gettext('Type'),
+   allowBlank: false,
+   comboItems: [
+   ['standalone', 'standalone'],
+   ['dns', 'DNS'],
+   ],
+   validator: function(value) {
+   let me = this;
+   let win = me.up('pveACMEDomainEdit');
+   let oldconfigkey = win.domain ? win.domain.configkey : 
undefined;
+   let val = me.getValue();
+   if (val === 'dns' && (!oldconfigkey || oldconfigkey === 
'acme')) {
+   // we have to check if there is a 'acmedomain' slot 
left
+   let found = false;
+   for (let i = 0; i < PVE.Utils.acmedomain_count; 
i++) {
+   if (!win.nodeconfig[`acmedomain${i}`]) {
+   found = true;
+   }
+   }
+   if (!found) {
+   return gettext('Only 5 Domains with type DNS 
can be configured');
+   }
+   }
+
+   return true;
+   },
+   listeners: {
+   change: function(cb, value) {
+   let me = this;
+   let view = me.up('pveACMEDomainEdit');
+   view.down('field[name=plugin]').setDisabled(value 
!== 'dns');
+   },
+   },
+   },
+   {
+   xtype: 'hidden',
+   

[pve-devel] [PATCH manager 5/5] ui: node/ACME: rework ACME grid for plugin based domains

2020-05-06 Thread Dominik Csapak
This is basically a complete rework of the ACME grid.
Instead of having an ObjectGrid, we now have a normal
GridPanel which allows us to show a row for each Domain.

But to achieve this, we need to manually fill the store with data
from the 'acme' and 'acmedomainX' entries of the node config.

We also add an AccountSelector to the tbar and a link to the
datacenter->acme panel (when there is no account)

this also removes the 'register account' and 'view account' buttons,
since those are now available in datacenter->acme

removes the old 'acmeeditor' since that is no longer needed

Signed-off-by: Dominik Csapak 
---
 www/manager6/node/ACME.js | 420 +++---
 1 file changed, 254 insertions(+), 166 deletions(-)

diff --git a/www/manager6/node/ACME.js b/www/manager6/node/ACME.js
index a8bb39d6..f6499d6e 100644
--- a/www/manager6/node/ACME.js
+++ b/www/manager6/node/ACME.js
@@ -1,49 +1,3 @@
-Ext.define('PVE.node.ACMEEditor', {
-extend: 'Proxmox.window.Edit',
-xtype: 'pveACMEEditor',
-
-subject: gettext('Domains'),
-items: [
-   {
-   xtype: 'inputpanel',
-   items: [
-   {
-   xtype: 'textarea',
-   fieldLabel: gettext('Domains'),
-   emptyText: "domain1.example.com\ndomain2.example.com",
-   name: 'domains'
-   }
-   ],
-   onGetValues: function(values) {
-   if (!values.domains) {
-   return {
-   'delete': 'acme'
-   };
-   }
-   var domains = values.domains.split(/\n/).join(';');
-   return {
-   'acme': 'domains=' + domains
-   };
-   }
-   }
-],
-
-initComponent: function() {
-   var me = this;
-   me.callParent();
-
-   me.load({
-   success: function(response, opts) {
-   var res = PVE.Parser.parseACME(response.result.data.acme);
-   if (res) {
-   res.domains = res.domains.join(' ');
-   me.setValues(res);
-   }
-   }
-   });
-}
-});
-
 Ext.define('PVE.node.ACMEAccountCreate', {
 extend: 'Proxmox.window.Edit',
 
@@ -366,138 +320,279 @@ Ext.define('PVE.node.ACMEDomainEdit', {
 },
 });
 
+Ext.define('pve-acme-domains', {
+extend: 'Ext.data.Model',
+fields: ['domain', 'type', 'alias', 'plugin', 'configkey'],
+idProperty: 'domain',
+});
+
 Ext.define('PVE.node.ACME', {
-extend: 'Proxmox.grid.ObjectGrid',
-xtype: 'pveACMEView',
+extend: 'Ext.grid.Panel',
+alias: 'widget.pveACMEView',
 
 margin: '10 0 0 0',
 title: 'ACME',
 
+controller: {
+   xclass: 'Ext.app.ViewController',
+
+   addDomain: function() {
+   let me = this;
+   let view = me.getView();
+
+   Ext.create('PVE.node.ACMEDomainEdit', {
+   nodename: view.nodename,
+   nodeconfig: view.nodeconfig,
+   apiCallDone: function() {
+   me.reload();
+   },
+   }).show();
+   },
+
+   editDomain: function() {
+   let me = this;
+   let view = me.getView();
+
+   let selection = view.getSelection();
+   if (selection.length < 1) return;
+
+   Ext.create('PVE.node.ACMEDomainEdit', {
+   nodename: view.nodename,
+   nodeconfig: view.nodeconfig,
+   domain: selection[0].data,
+   apiCallDone: function() {
+   me.reload();
+   },
+   }).show();
+   },
+
+   removeDomain: function() {
+   let me = this;
+   let view = me.getView();
+   let selection = view.getSelection();
+   if (selection.length < 1) return;
+
+   let rec = selection[0].data;
+   let params = {};
+   if (rec.configkey !== 'acme') {
+   params.delete = rec.configkey;
+   } else {
+   let acme = PVE.Parser.parseACME(view.nodeconfig.acme);
+   PVE.Utils.remove_domain_from_acme(acme, rec.domain);
+   params.acme = PVE.Parser.printACME(acme);
+   }
+
+   Proxmox.Utils.API2Request({
+   method: 'PUT',
+   url: `/nodes/${view.nodename}/config`,
+   params,
+   success: function(response, opt) {
+   me.reload();
+   },
+   failure: function(response, opt) {
+   Ext.Msg.alert(gettext('Error'), response.htmlStatus);
+   },
+   });
+   },
+
+   changeAccount: function(cb, value, oldvalue) {
+   if (value === oldvalue) return;
+   let me = this;
+   let view = me.getView();
+   let params = {};
+
+   let acme = PVE.Parser.parseACME(view.nodeconfig.acme);
+   acme.account = value;
+   

[pve-devel] [PATCH widget-toolkit] adapt auth utils for pve token authentication

2020-05-06 Thread Tim Marx
Signed-off-by: Tim Marx 
---
 Utils.js | 19 ---
 1 file changed, 16 insertions(+), 3 deletions(-)

diff --git a/Utils.js b/Utils.js
index 22eddd2..b190ce4 100644
--- a/Utils.js
+++ b/Utils.js
@@ -31,6 +31,10 @@ Ext.Ajax.on('beforerequest', function(conn, options) {
}
options.headers.CSRFPreventionToken = Proxmox.CSRFPreventionToken;
 }
+var token = window.localStorage.getItem("PVEAPIToken");
+if (token) {
+   options.headers.Authorization = 'PVEAPIToken=' + token;
+}
 });
 
 Ext.define('Proxmox.Utils', { utilities: {
@@ -195,19 +199,26 @@ Ext.define('Proxmox.Utils', { utilities: {
 },
 
 setAuthData: function(data) {
-   Proxmox.CSRFPreventionToken = data.CSRFPreventionToken;
Proxmox.UserName = data.username;
Proxmox.LoggedOut = data.LoggedOut;
// creates a session cookie (expire = null)
// that way the cookie gets deleted after the browser window is closed
-   Ext.util.Cookies.set(Proxmox.Setup.auth_cookie_name, data.ticket, null, 
'/', null, true);
+   if (data.ticket) {
+   Proxmox.CSRFPreventionToken = data.CSRFPreventionToken;
+   Ext.util.Cookies.set(Proxmox.Setup.auth_cookie_name, data.ticket, 
null, '/', null, true);
+   }
+
+   if (data.token) {
+   window.localStorage.setItem('PVEUserName', data.username);
+   window.localStorage.setItem('PVEAPIToken', data.token);
+   }
 },
 
 authOK: function() {
if (Proxmox.LoggedOut) {
return undefined;
}
-   return (Proxmox.UserName !== '') && 
Ext.util.Cookies.get(Proxmox.Setup.auth_cookie_name);
+   return Proxmox.UserName !== '' && 
(Ext.util.Cookies.get(Proxmox.Setup.auth_cookie_name) || 
window.localStorage.getItem("PVEAPIToken"));
 },
 
 authClear: function() {
@@ -215,6 +226,8 @@ Ext.define('Proxmox.Utils', { utilities: {
return undefined;
}
Ext.util.Cookies.clear(Proxmox.Setup.auth_cookie_name);
+   window.localStorage.removeItem("PVEAPIToken");
+   window.localStorage.removeItem("PVEUserName");
 },
 
 // comp.setLoading() is buggy in ExtJS 4.0.7, so we
-- 
2.20.1

___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] [PATCH access-control 1/2] whitespace cleanup

2020-05-06 Thread Tim Marx
Signed-off-by: Tim Marx 
---
 PVE/API2/AccessControl.pm | 48 +++
 1 file changed, 24 insertions(+), 24 deletions(-)

diff --git a/PVE/API2/AccessControl.pm b/PVE/API2/AccessControl.pm
index 5b63d2b..25230ac 100644
--- a/PVE/API2/AccessControl.pm
+++ b/PVE/API2/AccessControl.pm
@@ -32,36 +32,36 @@ eval {
 use base qw(PVE::RESTHandler);
 
 __PACKAGE__->register_method ({
-subclass => "PVE::API2::User",  
+subclass => "PVE::API2::User",
 path => 'users',
 });
 
 __PACKAGE__->register_method ({
-subclass => "PVE::API2::Group",  
+subclass => "PVE::API2::Group",
 path => 'groups',
 });
 
 __PACKAGE__->register_method ({
-subclass => "PVE::API2::Role",  
+subclass => "PVE::API2::Role",
 path => 'roles',
 });
 
 __PACKAGE__->register_method ({
-subclass => "PVE::API2::ACL",  
+subclass => "PVE::API2::ACL",
 path => 'acl',
 });
 
 __PACKAGE__->register_method ({
-subclass => "PVE::API2::Domains",  
+subclass => "PVE::API2::Domains",
 path => 'domains',
 });
 
 __PACKAGE__->register_method ({
-name => 'index', 
-path => '', 
+name => 'index',
+path => '',
 method => 'GET',
 description => "Directory index.",
-permissions => { 
+permissions => {
user => 'all',
 },
 parameters => {
@@ -80,7 +80,7 @@ __PACKAGE__->register_method ({
 },
 code => sub {
my ($param) = @_;
-
+
my $res = [];
 
my $ma = __PACKAGE__->method_attributes();
@@ -214,8 +214,8 @@ my $compute_api_permission = sub {
 };
 
 __PACKAGE__->register_method ({
-name => 'get_ticket', 
-path => 'ticket', 
+name => 'get_ticket',
+path => 'ticket',
 method => 'GET',
 permissions => { user => 'world' },
 description => "Dummy. Useful for formatters which want to provide a login 
page.",
@@ -224,14 +224,14 @@ __PACKAGE__->register_method ({
 },
 returns => { type => "null" },
 code => sub { return undef; }});
-  
+
 __PACKAGE__->register_method ({
-name => 'create_ticket', 
-path => 'ticket', 
+name => 'create_ticket',
+path => 'ticket',
 method => 'POST',
-permissions => { 
+permissions => {
description => "You need to pass valid credientials.",
-   user => 'world' 
+   user => 'world'
 },
 protected => 1, # else we can't access shadow files
 allowtoken => 0, # we don't want tokens to create tickets
@@ -250,7 +250,7 @@ __PACKAGE__->register_method ({
optional => 1,
completion => \::AccessControl::complete_realm,
}),
-   password => { 
+   password => {
description => "The secret password. This can also be a valid 
ticket.",
type => 'string',
},
@@ -266,7 +266,7 @@ __PACKAGE__->register_method ({
optional => 1,
maxLength => 64,
},
-   privs => { 
+   privs => {
description => "Verify ticket, and check if user have access 
'privs' on 'path'",
type => 'string' , format => 'pve-priv-list',
requires => 'path',
@@ -287,7 +287,7 @@ __PACKAGE__->register_method ({
 },
 code => sub {
my ($param) = @_;
-
+
my $username = $param->{username};
$username .= "\@$param->{realm}" if $param->{realm};
 
@@ -327,11 +327,11 @@ __PACKAGE__->register_method ({
 
 __PACKAGE__->register_method ({
 name => 'change_password',
-path => 'password', 
+path => 'password',
 method => 'PUT',
-permissions => { 
+permissions => {
description => "Each user is allowed to change his own password. A user 
can change the password of another user if he has 'Realm.AllocateUser' (on the 
realm of user ) and 'User.Modify' permission on /access/groups/ 
on a group where user  is member of.",
-   check => [ 'or', 
+   check => [ 'or',
   ['userid-param', 'self'],
   [ 'and',
 [ 'userid-param', 'Realm.AllocateUser'],
@@ -346,10 +346,10 @@ __PACKAGE__->register_method ({
additionalProperties => 0,
properties => {
userid => get_standard_option('userid-completed'),
-   password => { 
+   password => {
description => "The new password.",
type => 'string',
-   minLength => 5, 
+   minLength => 5,
maxLength => 64,
},
}
-- 
2.20.1

___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] [PATCH access-control 2/2] add ui capabilities endpoint

2020-05-06 Thread Tim Marx
Signed-off-by: Tim Marx 
---
 PVE/API2/AccessControl.pm | 29 +
 1 file changed, 29 insertions(+)

diff --git a/PVE/API2/AccessControl.pm b/PVE/API2/AccessControl.pm
index 25230ac..0a1b836 100644
--- a/PVE/API2/AccessControl.pm
+++ b/PVE/API2/AccessControl.pm
@@ -717,4 +717,33 @@ __PACKAGE__->register_method({
return $res;
 }});
 
+__PACKAGE__->register_method({
+name => 'uicapabilities',
+path => 'uicapabilities',
+method => 'GET',
+description => 'Retrieve user interface capabilities for calling 
user/token.',
+permissions => {
+   description => "Each user/token is allowed to retrieve their own 
capabilities.",
+   user => 'all',
+},
+parameters => {},
+returns => {
+   type => 'object',
+   properties => {
+   cap => {
+   type => 'object',
+   description => 'The user interface capabilities of the calling 
user/token'
+   }
+   },
+},
+code => sub {
+   my ($param) = @_;
+
+   my $rpcenv = PVE::RPCEnvironment::get();
+   my $userid = $rpcenv->get_user();
+   my $res->{cap} = &$compute_api_permission($rpcenv, $userid);
+
+   return $res;
+}});
+
 1;
-- 
2.20.1

___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] [PATCH manager 2/3] ui: auth: add api token authentication to login window

2020-05-06 Thread Tim Marx
Signed-off-by: Tim Marx 
---
 www/manager6/Workspace.js  |   5 ++
 www/manager6/window/LoginWindow.js | 129 ++---
 2 files changed, 105 insertions(+), 29 deletions(-)

diff --git a/www/manager6/Workspace.js b/www/manager6/Workspace.js
index 12ad70e4..20d8c692 100644
--- a/www/manager6/Workspace.js
+++ b/www/manager6/Workspace.js
@@ -72,6 +72,11 @@ Ext.define('PVE.Workspace', {
 
me.callParent();
 
+   var username = window.localStorage.getItem('PVEUserName');
+   if (username) {
+   Proxmox.UserName = username;
+   }
+
 if (!Proxmox.Utils.authOK()) {
me.showLogin();
} else {
diff --git a/www/manager6/window/LoginWindow.js 
b/www/manager6/window/LoginWindow.js
index e29b7352..5641e1ea 100644
--- a/www/manager6/window/LoginWindow.js
+++ b/www/manager6/window/LoginWindow.js
@@ -12,6 +12,8 @@ Ext.define('PVE.window.LoginWindow', {
var form = this.lookupReference('loginForm');
var unField = this.lookupReference('usernameField');
var saveunField = this.lookupReference('saveunField');
+   var tField = this.lookupReference('apitokenField');
+
var view = this.getView();
 
if (!form.isValid()) {
@@ -20,38 +22,61 @@ Ext.define('PVE.window.LoginWindow', {
 
view.el.mask(gettext('Please wait...'), 'x-mask-loading');
 
-   // set or clear username
-   var sp = Ext.state.Manager.getProvider();
-   if (saveunField.getValue() === true) {
-   sp.set(unField.getStateId(), unField.getValue());
+   if (tField.value !== '') {
+   var splitToken = tField.value.match(/^(.*)=(.*)$/);
+   var splitTokenID = splitToken[1].split('!');
+   Proxmox.Utils.API2Request({
+   url: '/api2/extjs/access/uicapabilities',
+   headers:{
+   Authorization: 'PVEAPIToken=' + tField.value
+   },
+   success: function(response, opts) {
+   var data = {
+   username: splitTokenID[0],
+   token: tField.value,
+   cap: response.result.data.cap
+   };
+   me.success(data);
+   },
+
+   failure: function(response, opts) {
+   me.failure(response);
+   }
+   });
} else {
-   sp.clear(unField.getStateId());
-   }
-   sp.set(saveunField.getStateId(), saveunField.getValue());
+   // set or clear username
+   var sp = Ext.state.Manager.getProvider();
+   if (saveunField.getValue() === true) {
+   sp.set(unField.getStateId(), unField.getValue());
+   } else {
+   sp.clear(unField.getStateId());
+   }
+   sp.set(saveunField.getStateId(), saveunField.getValue());
 
-   form.submit({
-   failure: function(f, resp){
-   me.failure(resp);
-   },
-   success: function(f, resp){
-   view.el.unmask();
+   form.submit({
+   failure: function(f, resp){
+   me.failure(resp);
+   },
+   success: function(f, resp){
+   view.el.unmask();
 
-   var data = resp.result.data;
-   if (Ext.isDefined(data.NeedTFA)) {
-   // Store first factor login information first:
-   data.LoggedOut = true;
-   Proxmox.Utils.setAuthData(data);
+   var data = resp.result.data;
+   if (Ext.isDefined(data.NeedTFA)) {
+   // Store first factor login information first:
+   data.LoggedOut = true;
+   Proxmox.Utils.setAuthData(data);
 
-   if (Ext.isDefined(data.U2FChallenge)) {
-   me.perform_u2f(data);
+   if (Ext.isDefined(data.U2FChallenge)) {
+   me.perform_u2f(data);
+   } else {
+   me.perform_otp();
+   }
} else {
-   me.perform_otp();
+   me.success(data);
}
-   } else {
-   me.success(data);
}
-   }
-   });
+   });
+   }
 
},
failure: function(resp) {
@@ -143,6 +168,31 @@ Ext.define('PVE.window.LoginWindow', {
}
});
},
+   onUsetokenChange: function(value) {
+   var uField = this.lookupReference('usernameField');
+   

[pve-devel] [PATCH manager 3/3] ui: auth: clear ui capabilities on logout

2020-05-06 Thread Tim Marx
Signed-off-by: Tim Marx 
---
 www/manager6/Workspace.js | 1 +
 1 file changed, 1 insertion(+)

diff --git a/www/manager6/Workspace.js b/www/manager6/Workspace.js
index 20d8c692..ffc5b175 100644
--- a/www/manager6/Workspace.js
+++ b/www/manager6/Workspace.js
@@ -37,6 +37,7 @@ Ext.define('PVE.Workspace', {
var me = this;
 
Proxmox.Utils.authClear();
+   Ext.state.Manager.clear('GuiCap');
Proxmox.UserName = null;
me.loginData = null;
 
-- 
2.20.1

___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] [PATCH manager 1/3] ui: whitespace cleanup

2020-05-06 Thread Tim Marx
Signed-off-by: Tim Marx 
---
 www/manager6/Workspace.js | 22 +++---
 1 file changed, 11 insertions(+), 11 deletions(-)

diff --git a/www/manager6/Workspace.js b/www/manager6/Workspace.js
index 01b462c7..12ad70e4 100644
--- a/www/manager6/Workspace.js
+++ b/www/manager6/Workspace.js
@@ -74,7 +74,7 @@ Ext.define('PVE.Workspace', {
 
 if (!Proxmox.Utils.authOK()) {
me.showLogin();
-   } else { 
+   } else {
if (me.loginData) {
me.onLogin(me.loginData);
}
@@ -88,7 +88,7 @@ Ext.define('PVE.Workspace', {
}
 
Ext.Ajax.request({
-   params: { 
+   params: {
username: Proxmox.UserName,
password: ticket
},
@@ -114,7 +114,7 @@ Ext.define('PVE.StdWorkspace', {
 // private
 setContent: function(comp) {
var me = this;
-   
+
var cont = me.child('#content');
 
var lay = cont.getLayout();
@@ -229,7 +229,7 @@ Ext.define('PVE.StdWorkspace', {
pool: 'pvePoolConfig'
};
var comp = {
-   xtype: tlckup[n.data.type || 'root'] || 
+   xtype: tlckup[n.data.type || 'root'] ||
'pvePanelConfig',
showSearch: (n.data.id === 'root') ||
Ext.isDefined(n.data.groupbyid),
@@ -245,7 +245,7 @@ Ext.define('PVE.StdWorkspace', {
}
});
 
-   selview.on('select', function(combo, records) { 
+   selview.on('select', function(combo, records) {
if (records) {
var view = combo.getViewFilter();
rtree.setViewFilter(view);
@@ -264,7 +264,7 @@ Ext.define('PVE.StdWorkspace', {
handler: function() {
var wiz = Ext.create('PVE.qemu.CreateWizard', {});
wiz.show();
-   } 
+   }
});
 
var createCT = Ext.createWidget('button', {
@@ -277,7 +277,7 @@ Ext.define('PVE.StdWorkspace', {
handler: function() {
var wiz = Ext.create('PVE.lxc.CreateWizard', {});
wiz.show();
-   } 
+   }
});
 
sprovider.on('statechange', function(sp, key, value) {
@@ -294,13 +294,13 @@ Ext.define('PVE.StdWorkspace', {
items: [
{
region: 'north',
-   layout: { 
+   layout: {
type: 'hbox',
align: 'middle'
},
-   baseCls: 'x-plain', 
+   baseCls: 'x-plain',
defaults: {
-   baseCls: 'x-plain'  
+   baseCls: 'x-plain'
},
border: false,
margin: '2 0 2 5',
@@ -331,7 +331,7 @@ Ext.define('PVE.StdWorkspace', {
text: gettext('Documentation'),
margin: '0 5 0 0'
},
-   createVM, 
+   createVM,
createCT,
{
pack: 'end',
-- 
2.20.1

___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] [PATCH manager] gui: never collapse notes for templates

2020-05-06 Thread Stefan Reiter
There's no graphs on screen, so no reason to collapse the notes to save
space. Besides, it looked a bit funky expanding the notes on smaller
screens, since they always jumped to the bottom to fill the space...

Signed-off-by: Stefan Reiter 
---
 www/manager6/panel/NotesView.js | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/www/manager6/panel/NotesView.js b/www/manager6/panel/NotesView.js
index edf38e5d..90de7b5a 100644
--- a/www/manager6/panel/NotesView.js
+++ b/www/manager6/panel/NotesView.js
@@ -105,7 +105,7 @@ Ext.define('PVE.panel.NotesView', {
me.callParent();
if (type === 'node') {
me.down('#tbar').setVisible(true);
-   } else {
+   } else if (me.pveSelNode.data.template !== 1) {
me.setCollapsible(true);
me.collapseDirection = 'right';
 
-- 
2.20.1


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] applied: [PATCH common] network: fix adding vlan tags to bridge

2020-05-06 Thread Fabian Grünbichler
Signed-off-by: Fabian Grünbichler 
---
makes starting VMs fail, see
https://forum.proxmox.com/threads/failed-to-start-vm-failed-to-remove-default-vlan-tags-of-tap104i0-command-sbin-bridge-bridge-vlan-del-dev-tap104i0-vid-1-4094-failed-exit-code.69375/

 src/PVE/Network.pm | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/src/PVE/Network.pm b/src/PVE/Network.pm
index 21c592c..ac49536 100644
--- a/src/PVE/Network.pm
+++ b/src/PVE/Network.pm
@@ -217,10 +217,10 @@ my $bridge_add_interface = sub {
 
if ($vlan_aware) {
if ($tag) {
-   eval { run_command(['/sbin/bridge', 'bridge', 'vlan', 'del', 'dev', 
$iface, 'vid', '1-4094']) };
+   eval { run_command(['/sbin/bridge', 'vlan', 'del', 'dev', $iface, 
'vid', '1-4094']) };
die "failed to remove default vlan tags of $iface - $@\n" if $@;
 
-   eval { run_command(['/sbin/bridge', 'bridge', 'vlan', 'add', 'dev', 
$iface, 'vid', $tag, 'pvid', 'untagged']) };
+   eval { run_command(['/sbin/bridge', 'vlan', 'add', 'dev', $iface, 
'vid', $tag, 'pvid', 'untagged']) };
die "unable to add vlan $tag to interface $iface - $@\n" if $@;
 
warn "Caution: Setting VLAN ID 1 on a VLAN aware bridge may be 
dangerous\n" if $tag == 1;
-- 
2.20.1


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


Re: [pve-devel] [PATCH v5 manager 1/2] vzdump: move remaining guest include logic to single method

2020-05-06 Thread Aaron Lauterer
I forgot to mention in the notes, that this patch and the other manager 
patch with the tests needs the following series applied beforehand!


https://pve.proxmox.com/pipermail/pve-devel/2020-May/043340.html

___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] [PATCH v5 container 2/2] vzdump: move include logic for mountpoints to method

2020-05-06 Thread Aaron Lauterer
Move the logic which mountpoints are included in the backup job to its
own method and adapt the VZDump code accordingly. This makes it possible
to develop other features around backup jobs.

Signed-off-by: Aaron Lauterer 
---

v4->v5:
* use new `foreach_volume`
* change $ret_volumes to $return_volumes
* in PVE::VZDump::LXC change variables to better names and and use
  dedicated variables instead of hash when used more than once

 src/PVE/LXC/Config.pm | 24 
 src/PVE/VZDump/LXC.pm | 25 -
 2 files changed, 40 insertions(+), 9 deletions(-)

diff --git a/src/PVE/LXC/Config.pm b/src/PVE/LXC/Config.pm
index 34f657a..dc50a5d 100644
--- a/src/PVE/LXC/Config.pm
+++ b/src/PVE/LXC/Config.pm
@@ -1549,4 +1549,28 @@ sub get_replicatable_volumes {
 return $volhash;
 }
 
+sub get_backup_volumes {
+my ($class, $conf) = @_;
+
+my $return_volumes = [];
+
+my $test_mountpoint = sub {
+   my ($key, $volume) = @_;
+
+   my $data = {};
+   my ($included, $reason) = $class->mountpoint_backup_enabled($key, 
$volume);
+
+   $data->{key} = $key;
+   $data->{included} = $included;
+   $data->{reason} = $reason;
+   $data->{data} = $volume;
+
+   push @$return_volumes, $data;
+};
+
+PVE::LXC::Config->foreach_volume($conf, $test_mountpoint);
+
+return $return_volumes;
+}
+
 1;
diff --git a/src/PVE/VZDump/LXC.pm b/src/PVE/VZDump/LXC.pm
index 2d003d0..7f97a00 100644
--- a/src/PVE/VZDump/LXC.pm
+++ b/src/PVE/VZDump/LXC.pm
@@ -120,18 +120,24 @@ sub prepare {
 $task->{rootgid} = $rootgid;
 
 my $volids = $task->{volids} = [];
-PVE::LXC::Config->foreach_volume($conf, sub {
-   my ($name, $data) = @_;
-   my $volid = $data->{volume};
+
+my $backup_volumes = PVE::LXC::Config->get_backup_volumes($conf);
+
+foreach my $current_volume (@{$backup_volumes}) {
+   my $name = $current_volume->{key};
+   my $included = $current_volume->{included};
+   my $data = $current_volume->{data};
+
+   my $volume = $data->{volume};
my $mount = $data->{mp};
my $type = $data->{type};
 
-   return if !$volid || !$mount;
+   next if !$volume || !$mount;
 
-   if (!PVE::LXC::Config->mountpoint_backup_enabled($name, $data)) {
+   if (!$included) {
push @$exclude_dirs, $mount;
$self->loginfo("excluding $type mount point $name ('$mount') from 
backup");
-   return;
+   next;
}
 
$data->{name} = $name;
@@ -140,10 +146,11 @@ sub prepare {
if ($conf->{template} && !defined($data->{ro})) {
$data->{ro} = 1;
}
+
+   $self->loginfo("including mount point $name ('$mount') in backup");
push @$disks, $data;
-   push @$volids, $volid
-   if $type eq 'volume';
-});
+   push @$volids, $volume if $included;
+}
 
 if ($mode eq 'snapshot') {
if (!PVE::LXC::Config->has_feature('snapshot', $conf, $storage_cfg, 
undef, undef, 1)) {
-- 
2.20.1


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] [PATCH v5 container 1/2] vzdump: add reason for mountpoint backup inclusion

2020-05-06 Thread Aaron Lauterer
Signed-off-by: Aaron Lauterer 
---
nothing changed to previous series'

 src/PVE/LXC/Config.pm | 24 +---
 1 file changed, 17 insertions(+), 7 deletions(-)

diff --git a/src/PVE/LXC/Config.pm b/src/PVE/LXC/Config.pm
index dcc8755..34f657a 100644
--- a/src/PVE/LXC/Config.pm
+++ b/src/PVE/LXC/Config.pm
@@ -49,13 +49,23 @@ sub cfs_config_path {
 sub mountpoint_backup_enabled {
 my ($class, $mp_key, $mountpoint) = @_;
 
-return 1 if $mp_key eq 'rootfs';
-
-return 0 if $mountpoint->{type} ne 'volume';
-
-return 1 if $mountpoint->{backup};
-
-return 0;
+my $enabled;
+my $reason;
+
+if ($mp_key eq 'rootfs') {
+   $enabled = 1;
+   $reason = 'rootfs';
+} elsif ($mountpoint->{type} ne 'volume') {
+   $enabled = 0;
+   $reason = 'not a volume';
+} elsif ($mountpoint->{backup}) {
+   $enabled = 1;
+   $reason = 'enabled';
+} else {
+   $enabled = 0;
+   $reason = 'disabled';
+}
+return wantarray ? ($enabled, $reason) : $enabled;
 }
 
 sub has_feature {
-- 
2.20.1


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] [PATCH qemu-server] vzdump: fix template backup to stdout

2020-05-06 Thread Fabian Grünbichler
redirecting to the saved STDOUT in case of a template backup or a VM
without any disks failed because of the erroneous '=':

Backup of VM 123123 failed - command '/usr/bin/vma create -v -c [...]' failed:
Bad filehandle: =5 at /usr/share/perl/5.28/IPC/Open3.pm line 58.

https://forum.proxmox.com/threads/vzdump-to-stdout.69364

Signed-off-by: Fabian Grünbichler 
---
 PVE/VZDump/QemuServer.pm | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/PVE/VZDump/QemuServer.pm b/PVE/VZDump/QemuServer.pm
index f122539..3a990cf 100644
--- a/PVE/VZDump/QemuServer.pm
+++ b/PVE/VZDump/QemuServer.pm
@@ -518,7 +518,7 @@ sub archive_vma {
$self->loginfo(join(' ', @$cmd));
 
if ($opts->{stdout}) {
-   $self->cmd($cmd, output => ">&=" . fileno($opts->{stdout}));
+   $self->cmd($cmd, output => ">&" . fileno($opts->{stdout}));
} else {
$self->cmd($cmd);
}
-- 
2.20.1


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] [PATCH v5 manager 1/2] vzdump: move remaining guest include logic to single method

2020-05-06 Thread Aaron Lauterer
The `guest include` logic handling `all` and `exclude` parameters was in
the `PVE::VZDump->exec_backup()` method. Moving this logic into the
`get_included_guests` method allows us to simplify and generalize it.

This helps to make the overall logic easier to test and develop other
features around vzdump backup jobs.

The method now returns a hash with node names as keys mapped to arrays
of VMIDs on these nodes that are included in the vzdump job.

The VZDump API call to create a new backup is adapted to use the new
method to create the list of local VMIDs and the skiplist.

Permission checks are kept where they are to be able to handle missing
permissions according to the current context. The old behavior to die
on a backup job when the user is missing the permission to a guest and
the job is not an 'all' or 'exclude' job is kept.

Signed-off-by: Aaron Lauterer 
---

v4 -> v5:
* move handling of `all` and `exlcude` cases from
  `PVE::VZDump->exec_backup` to `PVE::VZDump->get_included_guests`
* change return value to hash of node names with arrays of included
  VMIDs to be more general
* adapt API call to start a VZDump job accordingly. Creating the list of
  local VMIDs and the skiplist is handled right there again as so far
  this is the only place where that kind of separation is needed

v3 -> v4:
* reworked the whole logic according to the discussion with
fabian [1].
* re-added missing early exit

[1] https://pve.proxmox.com/pipermail/pve-devel/2020-March/042451.html

 PVE/API2/VZDump.pm | 18 ---
 PVE/VZDump.pm  | 76 +-
 2 files changed, 48 insertions(+), 46 deletions(-)

diff --git a/PVE/API2/VZDump.pm b/PVE/API2/VZDump.pm
index 68a3de89..77dac1b1 100644
--- a/PVE/API2/VZDump.pm
+++ b/PVE/API2/VZDump.pm
@@ -69,19 +69,26 @@ __PACKAGE__->register_method ({
return 'OK' if $param->{node} && $param->{node} ne $nodename;
 
my $cmdline = PVE::VZDump::Common::command_line($param);
-   my ($vmids, $skiplist) = PVE::VZDump->get_included_guests($param);
+
+   my $vmids_per_node = PVE::VZDump->get_included_guests($param);
+
+   my $vmids = $vmids_per_node->{$nodename} // [];
+
+   my $skiplist = [];
+
+   for my $current_node (keys %{$vmids_per_node}) {
+   push @{$skiplist}, @{$vmids_per_node->{$current_node}}
+   if $current_node ne $nodename;
+   }
 
if($param->{stop}){
PVE::VZDump::stop_running_backups();
return 'OK' if !scalar(@{$vmids});
}
 
-   # silent exit if specified VMs run on other nodes
+   # silent exit if specified guests run on other nodes
return "OK" if !scalar(@{$vmids});
 
-   my @exclude = PVE::Tools::split_list(extract_param($param, 'exclude'));
-   $param->{exclude} = PVE::VZDump::check_vmids(@exclude);
-
# exclude-path list need to be 0 separated
if (defined($param->{'exclude-path'})) {
my @expaths = split(/\0/, $param->{'exclude-path'} || '');
@@ -106,6 +113,7 @@ __PACKAGE__->register_method ({
die "interrupted by signal\n";
};
 
+   $param->{vmids} = $vmids;
my $vzdump = PVE::VZDump->new($cmdline, $param, $skiplist);
 
eval {
diff --git a/PVE/VZDump.pm b/PVE/VZDump.pm
index 68c08f47..cd278f4d 100644
--- a/PVE/VZDump.pm
+++ b/PVE/VZDump.pm
@@ -1039,29 +1039,18 @@ sub exec_backup {
if scalar(@{$self->{skiplist}});
 
 my $tasklist = [];
+my $vzdump_plugins =  {};
+foreach my $plugin (@{$self->{plugins}}) {
+   my $type = $plugin->type();
+   $vzdump_plugins->{$type} = $plugin;
+}
 
-if ($opts->{all}) {
-   foreach my $plugin (@{$self->{plugins}}) {
-   my $vmlist = $plugin->vmlist();
-   foreach my $vmid (sort @$vmlist) {
-   next if grep { $_ eq  $vmid } @{$opts->{exclude}};
-   next if !$rpcenv->check($authuser, "/vms/$vmid", [ 'VM.Backup' 
], 1);
-   push @$tasklist, { vmid => $vmid,  state => 'todo', plugin => 
$plugin, mode => $opts->{mode} };
-   }
-   }
-} else {
-   foreach my $vmid (sort @{$opts->{vmids}}) {
-   my $plugin;
-   foreach my $pg (@{$self->{plugins}}) {
-   my $vmlist = $pg->vmlist();
-   if (grep { $_ eq  $vmid } @$vmlist) {
-   $plugin = $pg;
-   last;
-   }
-   }
-   $rpcenv->check($authuser, "/vms/$vmid", [ 'VM.Backup' ]);
-   push @$tasklist, { vmid => $vmid,  state => 'todo', plugin => 
$plugin, mode => $opts->{mode} };
-   }
+my $vmlist = PVE::Cluster::get_vmlist();
+foreach my $vmid (sort @{$opts->{vmids}}) {
+   my $guest_type = $vmlist->{ids}->{$vmid}->{type};
+   my $plugin = $vzdump_plugins->{$guest_type};
+   next if !$rpcenv->check($authuser, "/vms/$vmid", [ 'VM.Backup' ], 
$opts->{all});
+   push @$tasklist, { vmid => $vmid,  state => 'todo', 

[pve-devel] [PATCH v5 manager 2/2] vzdump: test: adapt and add more tests that are possible now

2020-05-06 Thread Aaron Lauterer
Now, with the logic for `all` and `exclude` in the same single method,
additional tests for these cases are possible.

Adapt to hash return value of `get_included_guests`.

Signed-off-by: Aaron Lauterer 
---
 test/vzdump_guest_included_test.pl | 123 ++---
 1 file changed, 58 insertions(+), 65 deletions(-)

diff --git a/test/vzdump_guest_included_test.pl 
b/test/vzdump_guest_included_test.pl
index 378ad474..8b6db561 100755
--- a/test/vzdump_guest_included_test.pl
+++ b/test/vzdump_guest_included_test.pl
@@ -9,7 +9,7 @@ use warnings;
 
 use lib '..';
 
-use Test::More tests => 7;
+use Test::More tests => 9;
 use Test::MockModule;
 
 use PVE::VZDump;
@@ -78,84 +78,82 @@ $pve_api2tools->mock(
 
 my $tests = {};
 
-# is handled in the PVE::VZDump->exec_backup() method for now
-# $tests->{'Test all guests'} = {
-# expected_vmids => [ 100, 101, 112, 113, 200, 201, 212, 213 ],
-# expected_skiplist => [ ],
-# param => {
-#  all => 1,
-# }
-# };
-
-# is handled in the PVE::VZDump->exec_backup() method for now
-# $tests->{'Test all guests with cluster node limit'} = {
-# expected_vmids => [ 100, 101, 112, 113, 200, 201, 212, 213 ],
-# expected_skiplist => [],
-# param => {
-#  all => 1,
-#  node => 'node2',
-# }
-# };
-
-# is handled in the PVE::VZDump->exec_backup() method for now
-# $tests->{'Test all guests with local node limit'} = {
-# expected_vmids => [ 100, 101, 112, 113 ],
-# expected_skiplist => [ 200, 201, 212, 213 ],
-# param => {
-#  all => 1,
-#  node => 'node1',
-# }
-# };
-#
-# TODO: all test variants with exclude
-
-$tests->{'Test pool members'} = {
-expected_vmids => [ 100, 101 ],
-expected_skiplist => [ 200, 201 ],
+$tests->{'Test all guests'} = {
+expected => {
+   node1 => [ 100, 101, 112, 113 ],
+   node2 => [ 200, 201, 212, 213 ],
+},
 param => {
-   pool => 'testpool',
+   all => 1,
 }
 };
 
-$tests->{'Test pool members with cluster node limit'} = {
-expected_vmids => [ 100, 101, 200, 201 ],
-expected_skiplist => [],
+$tests->{'Test all guests with node limit'} = {
+expected => {
+   node2 => [ 200, 201, 212, 213 ],
+},
 param => {
-   pool => 'testpool',
+   all => 1,
node => 'node2',
 }
 };
 
-$tests->{'Test pool members with local node limit'} = {
-expected_vmids => [ 100, 101 ],
-expected_skiplist => [ 200, 201 ],
+$tests->{'Test exclude'} = {
+expected => {
+   node1 =>[ 101, 112, 113 ],
+   node2 => [ 201, 212,  213 ],
+},
 param => {
-   pool => 'testpool',
+   all => 1,
+   exclude => '100, 102, 200, 202',
+}
+};
+
+$tests->{'Test exclude with node limit'} = {
+expected => {
+   node1 =>[ 101, 112, 113 ],
+},
+param => {
+   all => 1,
+   exclude => '100, 102, 200, 202',
node => 'node1',
 }
 };
 
-$tests->{'Test selected VMIDs'} = {
-expected_vmids => [ 100 ],
-expected_skiplist => [ 201, 212 ],
+$tests->{'Test pool members'} = {
+expected => {
+   node1 =>[ 100, 101 ],
+   node2 => [ 200, 201 ],
+},
 param => {
-   vmid => '100, 201, 212',
+   pool => 'testpool',
 }
 };
 
+$tests->{'Test pool members with node limit'} = {
+expected => {
+   node2 => [ 200, 201 ],
+},
+param => {
+   pool => 'testpool',
+   node => 'node2',
+}
+};
 
-$tests->{'Test selected VMIDs with cluster node limit'} = {
-expected_vmids => [ 100, 201, 212 ],
-expected_skiplist => [],
+$tests->{'Test selected VMIDs'} = {
+expected => {
+   node1 =>[ 100 ],
+   node2 => [ 201, 212 ],
+},
 param => {
vmid => '100, 201, 212',
-   node => 'node2',
 }
 };
 
-$tests->{'Test selected VMIDs with local node limit'} = {
-expected_vmids => [ 100 ],
-expected_skiplist => [ 201, 212 ],
+$tests->{'Test selected VMIDs with node limit'} = {
+expected => {
+   node1 =>[ 100 ],
+},
 param => {
vmid => '100, 201, 212',
node => 'node1',
@@ -163,8 +161,8 @@ $tests->{'Test selected VMIDs with local node limit'} = {
 };
 
 $tests->{'Test selected VMIDs on other nodes'} = {
-expected_vmids => [],
-expected_skiplist => [ 201, 212 ],
+expected => {
+},
 param => {
vmid => '201, 212',
node => 'node1',
@@ -176,16 +174,11 @@ $tests->{'Test selected VMIDs on other nodes'} = {
 foreach my $testname (sort keys %{$tests}) {
 my $testdata = $tests->{$testname};
 
-note($testname);
-my $expected = [ $testdata->{expected_vmids}, 
$testdata->{expected_skiplist} ];
-
-my ($local, $cluster)  = 
PVE::VZDump->get_included_guests($testdata->{param});
-my $result = [ $local, $cluster ];
+# note($testname);
 
-# print "Expected: " . Dumper($expected);
-# print "Returned: " . Dumper($result);
+my $result  = PVE::VZDump->get_included_guests($testdata->{param});
 
-is_deeply($result, 

[pve-devel] [PATCH v5 series 0/5] add needed changes for backup detail view

2020-05-06 Thread Aaron Lauterer
This patch series provides needed changes to make other features around
VZDUMP backups possible.

It is moving the following logic into its own separate methods:
* which guests are included in a vzdump job
adds missing `all` and `exclude` handling. Needs the following
patches [0] to be applied beforehand!

* which volumes / mountpoints of a guest are included

changes from v4[1] to v5:
* incorporate feedback
* rebase on [0]
* extend tests to `all` and `exclude`

more in the individual patches

changes from v3[5] to v4:
* rework guest include logic in manager
* fix changed function call in QemuConfig.pm

[0] https://pve.proxmox.com/pipermail/pve-devel/2020-May/043340.html
[1] https://pve.proxmox.com/pipermail/pve-devel/2020-April/042753.html
[5] https://pve.proxmox.com/pipermail/pve-devel/2020-March/042420.html

manager: Aaron Lauterer (2):
  vzdump: move remaining guest include logic to single method
  vzdump: test: adapt and add more tests that are possible now

 PVE/API2/VZDump.pm | 18 --
 PVE/VZDump.pm  | 77 +++-
 test/vzdump_guest_included_test.pl | 96 +++---
 3 files changed, 109 insertions(+), 82 deletions(-)


qemu-server: Aaron Lauterer (1):
  vzdump: move include logic for volumes to method

 PVE/QemuConfig.pm| 31 
 PVE/VZDump/QemuServer.pm | 44 +---
 2 files changed, 54 insertions(+), 21 deletions(-)


container: Aaron Lauterer (2):
  vzdump: add reason for mountpoint backup inclusion
  vzdump: move include logic for mountpoints to method

 src/PVE/LXC/Config.pm | 48 ---
 src/PVE/VZDump/LXC.pm | 25 ++
 2 files changed, 57 insertions(+), 16 deletions(-)

--
2.20.1


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] [PATCH v5 qemu-server] vzdump: move include logic for volumes to method

2020-05-06 Thread Aaron Lauterer
Move the logic which volumes are included in the backup job to its own
method and adapt the VZDump code accordingly. This makes it possible to
develop other features around backup jobs.

Signed-off-by: Aaron Lauterer 
---
v4->v5:
* use new foreach_volume call
* change $ret_volumes to $return_volumes
* use dedicated variables in PVE::VZDump::QemuServer where hash is used
  more than once
* renamed $backup_included and other variables to (IMHO) better names

v3->v4: changed function call for `foreach_drive` to QemuServer::Drive

This can be changed to `foreach_volume` once the patches by Fabian Ebner
are through.
 PVE/QemuConfig.pm| 31 
 PVE/VZDump/QemuServer.pm | 44 +---
 2 files changed, 54 insertions(+), 21 deletions(-)

diff --git a/PVE/QemuConfig.pm b/PVE/QemuConfig.pm
index 240fc06..dac913f 100644
--- a/PVE/QemuConfig.pm
+++ b/PVE/QemuConfig.pm
@@ -165,6 +165,37 @@ sub get_replicatable_volumes {
 return $volhash;
 }
 
+sub get_backup_volumes {
+my ($class, $conf) = @_;
+
+my $return_volumes = [];
+
+my $test_volume = sub {
+   my ($key, $drive) = @_;
+
+   return if PVE::QemuServer::drive_is_cdrom($drive);
+
+   my $volume = {};
+   my $included = $drive->{backup} // 1;;
+   my $reason = defined($drive->{backup}) ? 'disabled' : 'enabled';
+
+   if ($key =~ m/^efidisk/ && (!defined($conf->{bios}) || $conf->{bios} ne 
'ovmf')) {
+   $included = 0;
+   $reason = "efidisk with non omvf bios";
+   }
+   $volume->{key} = $key;
+   $volume->{included} = $included;
+   $volume->{reason} = $reason;
+   $volume->{data} = $drive;
+
+   push @$return_volumes, $volume;
+};
+
+PVE::QemuConfig->foreach_volume($conf, $test_volume);
+
+return $return_volumes;
+}
+
 sub __snapshot_save_vmstate {
 my ($class, $vmid, $conf, $snapname, $storecfg, $statestorage, $suspend) = 
@_;
 
diff --git a/PVE/VZDump/QemuServer.pm b/PVE/VZDump/QemuServer.pm
index f122539..480979c 100644
--- a/PVE/VZDump/QemuServer.pm
+++ b/PVE/VZDump/QemuServer.pm
@@ -69,37 +69,39 @@ sub prepare {
 
 my $vollist = [];
 my $drivehash = {};
-PVE::QemuConfig->foreach_volume($conf, sub {
-   my ($ds, $drive) = @_;
-
-   return if PVE::QemuServer::drive_is_cdrom($drive);
-
-   my $volid = $drive->{file};
-
-   if (defined($drive->{backup}) && !$drive->{backup}) {
-   $self->loginfo("exclude disk '$ds' '$volid' (backup=no)");
-   return;
-   } elsif ($self->{vm_was_running} && $drive->{iothread}) {
+my $backup_volumes = PVE::QemuConfig->get_backup_volumes($conf);
+
+foreach my $volume (@{$backup_volumes}) {
+   my $name = $volume->{key};
+   my $data = $volume->{data};
+   my $volid = $data->{file};
+   my $size = $data->{size};
+
+   if (!$volume->{included}) {
+   if ($volume->{reason} eq 'efidisk with non omvf bios') {
+   $self->loginfo("excluding '$name' (efidisks can only be backed 
up when BIOS is set to 'ovmf')");
+   next;
+   }
+   $self->loginfo("exclude disk '$name' '$volid' (backup=no)");
+   next;
+   } elsif ($self->{vm_was_running} && $data->{iothread}) {
if (!PVE::QemuServer::Machine::runs_at_least_qemu_version($vmid, 4, 
0, 1)) {
-   die "disk '$ds' '$volid' (iothread=on) can't use backup feature 
with running QEMU " .
+   die "disk '$name' '$volid' (iothread=on) can't use backup 
feature with running QEMU " .
"version < 4.0.1! Either set backup=no for this drive or 
upgrade QEMU and restart VM\n";
}
-   } elsif ($ds =~ m/^efidisk/ && (!defined($conf->{bios}) || 
$conf->{bios} ne 'ovmf')) {
-   $self->loginfo("excluding '$ds' (efidisks can only be backed up 
when BIOS is set to 'ovmf')");
-   return;
} else {
-   my $log = "include disk '$ds' '$volid'";
-  if (defined $drive->{size}) {
-   my $readable_size = 
PVE::JSONSchema::format_size($drive->{size});
+   my $log = "include disk '$name' '$volid'";
+   if (defined $size) {
+   my $readable_size = PVE::JSONSchema::format_size($size);
$log .= " $readable_size";
-  }
+   }
$self->loginfo($log);
}
 
my ($storeid, $volname) = PVE::Storage::parse_volume_id($volid, 1);
push @$vollist, $volid if $storeid;
-   $drivehash->{$ds} = $drive;
-});
+   $drivehash->{$name} = $volume->{data};
+}
 
 PVE::Storage::activate_volumes($self->{storecfg}, $vollist);
 
-- 
2.20.1


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


Re: [pve-devel] [PATCH manager] vzdump: set 'pbs' option when backing up to PBS target

2020-05-06 Thread Thomas Lamprecht
On 5/6/20 10:57 AM, Fabian Grünbichler wrote:
> this unifies the logic into a single place instead of all over this
> module and the plugins.
> 
> it also fixes tons of 'uninitialized value' warnings when backing up
> with --dumpdir but no --storage set, since the existing conditions for
> PBS targets are missing a definedness check.
> 
> Signed-off-by: Fabian Grünbichler 
> ---
> 
> Notes:
> this commit alone does not break anything, but since the plugins in 
> qemu-server
> and pve-container can't have a versioned depends on pve-manager, we need 
> to
> break the old versions of pve-manager in those two packages to ensure we 
> get a
> version setting the now required option.

no we don't need that, this is experimental and we support only latest
versions together anway.

> 
>  PVE/VZDump.pm | 20 +++-
>  1 file changed, 11 insertions(+), 9 deletions(-)
> 
> diff --git a/PVE/VZDump.pm b/PVE/VZDump.pm
> index 6fd3aeed..623c6f8c 100644
> --- a/PVE/VZDump.pm
> +++ b/PVE/VZDump.pm
> @@ -86,6 +86,7 @@ sub storage_info {
>   return {
>   scfg => $scfg,
>   maxfiles => $scfg->{maxfiles},
> + pbs => 1,
>   };
>  } else {
>   return {
> @@ -459,6 +460,7 @@ sub new {
>   if ($@);
>   $opts->{dumpdir} = $info->{dumpdir};
>   $opts->{scfg} = $info->{scfg};
> + $opts->{pbs} = $info->{pbs};
>   $maxfiles //= $info->{maxfiles};
>  } elsif ($opts->{dumpdir}) {
>   $errors .= "dumpdir '$opts->{dumpdir}' does not exist"
> @@ -651,7 +653,7 @@ sub exec_backup_task {
>  my $pbs_group_name;
>  my $pbs_snapshot_name;
>  
> -if ($opts->{scfg}->{type} eq 'pbs') {
> +if ($self->{opts}->{pbs}) {
>   if ($vmtype eq 'lxc') {
>   $pbs_group_name = "ct/$vmid";
>   } elsif  ($vmtype eq 'qemu') {
> @@ -697,7 +699,7 @@ sub exec_backup_task {
>  
>   if ($maxfiles && !$opts->{remove}) {
>   my $count;
> - if ($opts->{scfg}->{type} eq 'pbs') {
> + if ($self->{opts}->{pbs}) {
>   my $res = 
> PVE::Storage::PBSPlugin::run_client_cmd($opts->{scfg}, $opts->{storage}, 
> 'snapshots', $pbs_group_name);
>   $count = scalar(@$res);
>   } else {
> @@ -710,7 +712,7 @@ sub exec_backup_task {
>   if $count >= $maxfiles;
>   }
>  
> - if ($opts->{scfg}->{type} ne 'pbs') {
> + if (!$self->{opts}->{pbs}) {
>   $task->{logfile} = "$opts->{dumpdir}/$basename.log";
>   }
>  
> @@ -720,7 +722,7 @@ sub exec_backup_task {
>   $ext .= ".${comp_ext}";
>   }
>  
> - if ($opts->{scfg}->{type} eq 'pbs') {
> + if ($self->{opts}->{pbs}) {
>   die "unable to pipe backup to stdout\n" if $opts->{stdout};
>   } else {
>   if ($opts->{stdout}) {
> @@ -735,7 +737,7 @@ sub exec_backup_task {
>  
>   $task->{vmtype} = $vmtype;
>  
> - if ($opts->{scfg}->{type} eq 'pbs') {
> + if ($self->{opts}->{pbs}) {
>   $task->{tmpdir} = "/var/tmp/vzdumptmp$$"; #fixme
>   } elsif ($opts->{tmpdir}) {
>   $task->{tmpdir} = "$opts->{tmpdir}/vzdumptmp$$";
> @@ -898,14 +900,14 @@ sub exec_backup_task {
>   }
>  
>   # fixme: ??
> - if ($opts->{scfg}->{type} eq 'pbs') {
> + if ($self->{opts}->{pbs}) {
>   debugmsg ('info', "creating pbs archive on storage 
> '$opts->{storage}'", $logfd);
>   } else {
>   debugmsg ('info', "creating archive '$task->{tarfile}'", $logfd);
>   }
>   $plugin->archive($task, $vmid, $task->{tmptar}, $comp);
>  
> - if ($opts->{scfg}->{type} eq 'pbs') {
> + if ($self->{opts}->{pbs}) {
>   # fixme: log size ?
>   debugmsg ('info', "pbs upload finished", $logfd);
>   } else {
> @@ -921,7 +923,7 @@ sub exec_backup_task {
>   # purge older backup
>   if ($maxfiles && $opts->{remove}) {
>  
> - if ($opts->{scfg}->{type} eq 'pbs') {
> + if ($self->{opts}->{pbs}) {
>   my $args = [$pbs_group_name, '--keep-last', $maxfiles];
>   my $logfunc = sub { my $line = shift; debugmsg ('info', $line, 
> $logfd); };
>   PVE::Storage::PBSPlugin::run_raw_client_cmd(
> @@ -1012,7 +1014,7 @@ sub exec_backup_task {
>  close ($logfd) if $logfd;
>  
>  if ($task->{tmplog}) {
> - if ($opts->{scfg}->{type} eq 'pbs') {
> + if ($self->{opts}->{pbs}) {
>   if ($task->{state} eq 'ok') {
>   my $param = [$pbs_snapshot_name, $task->{tmplog}];
>   PVE::Storage::PBSPlugin::run_raw_client_cmd(
> 



___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] [PATCH container] vzdump: use new 'pbs' option

2020-05-06 Thread Fabian Grünbichler
instead of storage config to determine whether we are in 'PBS mode'

Signed-off-by: Fabian Grünbichler 
---

Notes:
requires a break on pve-manager << version setting this option,
since the dependency is the other way round

 src/PVE/VZDump/LXC.pm | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/src/PVE/VZDump/LXC.pm b/src/PVE/VZDump/LXC.pm
index 2d003d0..45a3d8f 100644
--- a/src/PVE/VZDump/LXC.pm
+++ b/src/PVE/VZDump/LXC.pm
@@ -303,7 +303,7 @@ sub assemble {
 my $firewall ="/etc/pve/firewall/$vmid.fw";
 my $fwconftmp = "$tmpdir/etc/vzdump/pct.fw";
 
-if ($opts->{scfg}->{type} eq 'pbs') {
+if ($self->{vzdump}->{opts}->{pbs}) {
# fixme: do not store pct.conf and fw.conf into $tmpdir
if (-e  $firewall) {
PVE::Tools::file_copy($firewall, $fwconftmp);
@@ -356,7 +356,7 @@ sub archive {
 
 my $userns_cmd = $task->{userns_cmd};
 
-if ($opts->{scfg}->{type} eq 'pbs') {
+if ($self->{vzdump}->{opts}->{pbs}) {
 
my $rootdir = $default_mount_point;
my $param = [];
-- 
2.20.1


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] [PATCH qemu-server] vzdump: use new 'pbs' option

2020-05-06 Thread Fabian Grünbichler
instead of storage config to determine whether we are in 'PBS mode'

Signed-off-by: Fabian Grünbichler 
---

Notes:
requires a break on pve-manager << version setting this option,
since the dependency is the other way round.

 PVE/VZDump/QemuServer.pm | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/PVE/VZDump/QemuServer.pm b/PVE/VZDump/QemuServer.pm
index f122539..1bdc490 100644
--- a/PVE/VZDump/QemuServer.pm
+++ b/PVE/VZDump/QemuServer.pm
@@ -248,7 +248,7 @@ sub archive {
 my $opts = $self->{vzdump}->{opts};
 my $scfg = $opts->{scfg};
 
-if ($scfg->{type} eq 'pbs') {
+if ($self->{vzdump}->{opts}->{pbs}) {
$self->archive_pbs($task, $vmid);
 } else {
$self->archive_vma($task, $vmid, $filename, $comp);
-- 
2.20.1


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] [PATCH manager] vzdump: set 'pbs' option when backing up to PBS target

2020-05-06 Thread Fabian Grünbichler
this unifies the logic into a single place instead of all over this
module and the plugins.

it also fixes tons of 'uninitialized value' warnings when backing up
with --dumpdir but no --storage set, since the existing conditions for
PBS targets are missing a definedness check.

Signed-off-by: Fabian Grünbichler 
---

Notes:
this commit alone does not break anything, but since the plugins in 
qemu-server
and pve-container can't have a versioned depends on pve-manager, we need to
break the old versions of pve-manager in those two packages to ensure we 
get a
version setting the now required option.

 PVE/VZDump.pm | 20 +++-
 1 file changed, 11 insertions(+), 9 deletions(-)

diff --git a/PVE/VZDump.pm b/PVE/VZDump.pm
index 6fd3aeed..623c6f8c 100644
--- a/PVE/VZDump.pm
+++ b/PVE/VZDump.pm
@@ -86,6 +86,7 @@ sub storage_info {
return {
scfg => $scfg,
maxfiles => $scfg->{maxfiles},
+   pbs => 1,
};
 } else {
return {
@@ -459,6 +460,7 @@ sub new {
if ($@);
$opts->{dumpdir} = $info->{dumpdir};
$opts->{scfg} = $info->{scfg};
+   $opts->{pbs} = $info->{pbs};
$maxfiles //= $info->{maxfiles};
 } elsif ($opts->{dumpdir}) {
$errors .= "dumpdir '$opts->{dumpdir}' does not exist"
@@ -651,7 +653,7 @@ sub exec_backup_task {
 my $pbs_group_name;
 my $pbs_snapshot_name;
 
-if ($opts->{scfg}->{type} eq 'pbs') {
+if ($self->{opts}->{pbs}) {
if ($vmtype eq 'lxc') {
$pbs_group_name = "ct/$vmid";
} elsif  ($vmtype eq 'qemu') {
@@ -697,7 +699,7 @@ sub exec_backup_task {
 
if ($maxfiles && !$opts->{remove}) {
my $count;
-   if ($opts->{scfg}->{type} eq 'pbs') {
+   if ($self->{opts}->{pbs}) {
my $res = 
PVE::Storage::PBSPlugin::run_client_cmd($opts->{scfg}, $opts->{storage}, 
'snapshots', $pbs_group_name);
$count = scalar(@$res);
} else {
@@ -710,7 +712,7 @@ sub exec_backup_task {
if $count >= $maxfiles;
}
 
-   if ($opts->{scfg}->{type} ne 'pbs') {
+   if (!$self->{opts}->{pbs}) {
$task->{logfile} = "$opts->{dumpdir}/$basename.log";
}
 
@@ -720,7 +722,7 @@ sub exec_backup_task {
$ext .= ".${comp_ext}";
}
 
-   if ($opts->{scfg}->{type} eq 'pbs') {
+   if ($self->{opts}->{pbs}) {
die "unable to pipe backup to stdout\n" if $opts->{stdout};
} else {
if ($opts->{stdout}) {
@@ -735,7 +737,7 @@ sub exec_backup_task {
 
$task->{vmtype} = $vmtype;
 
-   if ($opts->{scfg}->{type} eq 'pbs') {
+   if ($self->{opts}->{pbs}) {
$task->{tmpdir} = "/var/tmp/vzdumptmp$$"; #fixme
} elsif ($opts->{tmpdir}) {
$task->{tmpdir} = "$opts->{tmpdir}/vzdumptmp$$";
@@ -898,14 +900,14 @@ sub exec_backup_task {
}
 
# fixme: ??
-   if ($opts->{scfg}->{type} eq 'pbs') {
+   if ($self->{opts}->{pbs}) {
debugmsg ('info', "creating pbs archive on storage 
'$opts->{storage}'", $logfd);
} else {
debugmsg ('info', "creating archive '$task->{tarfile}'", $logfd);
}
$plugin->archive($task, $vmid, $task->{tmptar}, $comp);
 
-   if ($opts->{scfg}->{type} eq 'pbs') {
+   if ($self->{opts}->{pbs}) {
# fixme: log size ?
debugmsg ('info', "pbs upload finished", $logfd);
} else {
@@ -921,7 +923,7 @@ sub exec_backup_task {
# purge older backup
if ($maxfiles && $opts->{remove}) {
 
-   if ($opts->{scfg}->{type} eq 'pbs') {
+   if ($self->{opts}->{pbs}) {
my $args = [$pbs_group_name, '--keep-last', $maxfiles];
my $logfunc = sub { my $line = shift; debugmsg ('info', $line, 
$logfd); };
PVE::Storage::PBSPlugin::run_raw_client_cmd(
@@ -1012,7 +1014,7 @@ sub exec_backup_task {
 close ($logfd) if $logfd;
 
 if ($task->{tmplog}) {
-   if ($opts->{scfg}->{type} eq 'pbs') {
+   if ($self->{opts}->{pbs}) {
if ($task->{state} eq 'ok') {
my $param = [$pbs_snapshot_name, $task->{tmplog}];
PVE::Storage::PBSPlugin::run_raw_client_cmd(
-- 
2.20.1


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


[pve-devel] [PATCH v2 manager] Improve storage selection on restore

2020-05-06 Thread Fabian Ebner
Previously, the blank '' would be passed along and lead to a
parameter verfication failure.

For LXC the default behavior in the backend is to use 'local', so
disallow blank and auto-select the first storage supporting'rootdir'
instead.

For QEMU the default behavior in the backend is to use the
original layout from the backup configuration file, which
makes sense to use as the default in the GUI as well.

Signed-off-by: Fabian Ebner 
---

Changes from v1:
* avoid unnecessary ?-operators
* better emptyText

 www/manager6/window/Restore.js | 9 ++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/www/manager6/window/Restore.js b/www/manager6/window/Restore.js
index 3f9ad3bc..9357958b 100644
--- a/www/manager6/window/Restore.js
+++ b/www/manager6/window/Restore.js
@@ -24,7 +24,11 @@ Ext.define('PVE.window.Restore', {
value: '',
fieldLabel: gettext('Storage'),
storageContent: (me.vmtype === 'lxc') ? 'rootdir' : 'images',
-   allowBlank: true
+   // when restoring a container without specifying a storage, the 
backend defaults
+   // to 'local', which is unintuitive and 'rootdir' might not even be 
allowed on it
+   allowBlank: me.vmtype !== 'lxc',
+   emptyText: (me.vmtype === 'lxc') ? '' : gettext('From backup 
configuration'),
+   autoSelect: me.vmtype === 'lxc',
});
 
var IDfield;
@@ -135,16 +139,15 @@ Ext.define('PVE.window.Restore', {
var submitBtn = Ext.create('Ext.Button', {
text: gettext('Restore'),
handler: function(){
-   var storage = storagesel.getValue();
var values = form.getValues();
 
var params = {
-   storage: storage,
vmid: me.vmid || values.vmid,
force: me.vmid ? 1 : 0
};
if (values.unique) { params.unique = 1; }
if (values.start) { params.start = 1; }
+   if (values.storage) { params.storage = values.storage; }
 
if (values.bwlimit !== undefined) {
params.bwlimit = values.bwlimit;
-- 
2.20.1


___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


Re: [pve-devel] [PATCH manager] Improve storage selection on restore

2020-05-06 Thread Fabian Ebner

On 5/5/20 1:40 PM, Thomas Lamprecht wrote:

On 5/5/20 1:20 PM, Fabian Ebner wrote:

Previously, the blank '' would be passed along and lead to a
parameter verfication failure.

For LXC the default behavior in the backend is to use 'local' as
the storage, so disallow blank and auto-select the first storage
supporting 'rootdir' instead.

For QEMU the default behavior in the backend is to use the
original layout from the backup configuration file, which
makes sense to use as the default in the GUI as well.

Signed-off-by: Fabian Ebner 
---
  www/manager6/window/Restore.js | 9 ++---
  1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/www/manager6/window/Restore.js b/www/manager6/window/Restore.js
index 3f9ad3bc..09c8a997 100644
--- a/www/manager6/window/Restore.js
+++ b/www/manager6/window/Restore.js
@@ -24,7 +24,11 @@ Ext.define('PVE.window.Restore', {
value: '',
fieldLabel: gettext('Storage'),
storageContent: (me.vmtype === 'lxc') ? 'rootdir' : 'images',
-   allowBlank: true
+   // when restoring a container without specifying a storage, the 
backend defaults
+   // to 'local', which is unintuitive and 'rootdir' might not even be 
allowed on it
+   allowBlank: (me.vmtype === 'lxc') ? false : true,
+   emptyText: (me.vmtype === 'lxc') ? '' : gettext('use old layout'),


1. We capitalize gettext strings normally, at least the first word
2. "Use old layout" is quite confusing, IMO, maybe "From Backup Configuration"
or "Backup Layout"



Yes, "From Backup Configuration" is better.


I mean in the longer run we want to probably stream line behavior between CT
and VM so that both try to use the ones from the backup config or even pass
a volume to storage map (as advanced feature).



+   autoSelect: (me.vmtype === 'lxc') ? true : false,


in general I like to avoid ternary operations if easily possible, for boolean
rvalues that's a must, me.vmtype === 'lxc' is already true or false ;-)




I thought it might make sense for readability in this case, because 
having the same expression four times in a row, one can read the first 
"column" as what happens for LXC and the second "column" as what happens 
for QEMU. But I can of course change it in v2.



});
  
  	var IDfield;

@@ -135,16 +139,15 @@ Ext.define('PVE.window.Restore', {
var submitBtn = Ext.create('Ext.Button', {
text: gettext('Restore'),
handler: function(){
-   var storage = storagesel.getValue();
var values = form.getValues();
  
  		var params = {

-   storage: storage,
vmid: me.vmid || values.vmid,
force: me.vmid ? 1 : 0
};
if (values.unique) { params.unique = 1; }
if (values.start) { params.start = 1; }
+   if (values.storage) { params.storage = values.storage; }
  
  		if (values.bwlimit !== undefined) {

params.bwlimit = values.bwlimit;





___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel