Re: [pve-devel] vm deletion succeeds even if storage deletion fails

[Stefan Priebe - Profihost AG]

Hi,

Am 15.01.19 um 08:19 schrieb Stefan Priebe - Profihost AG:
> 
> Am 15.01.19 um 08:15 schrieb Fabian Grünbichler:
>> On Mon, Jan 14, 2019 at 11:04:29AM +0100, Stefan Priebe - Profihost AG wrote:
>>> Hello,
>>>
>>> today i was wondering about some disk images while the vm was deleted.
>>>
>>> Inspecting the task history i found this log:
>>>
>>> trying to acquire cfs lock 'storage-cephstoroffice' ...
>>> trying to acquire cfs lock 'storage-cephstoroffice' ...
>>> trying to acquire cfs lock 'storage-cephstoroffice' ...
>>> trying to acquire cfs lock 'storage-cephstoroffice' ...
>>> trying to acquire cfs lock 'storage-cephstoroffice' ...
>>> trying to acquire cfs lock 'storage-cephstoroffice' ...
>>> trying to acquire cfs lock 'storage-cephstoroffice' ...
>>> trying to acquire cfs lock 'storage-cephstoroffice' ...
>>> trying to acquire cfs lock 'storage-cephstoroffice' ...
>>> Could not remove disk 'cephstoroffice:vm-202-disk-1', check manually:
>>> error with cfs lock 'storage-cephstoroffice': got lock request timeout
>>> trying to acquire cfs lock 'storage-cephstoroffice' ...
>>> trying to acquire cfs lock 'storage-cephstoroffice' ...
>>> trying to acquire cfs lock 'storage-cephstoroffice' ...
>>> trying to acquire cfs lock 'storage-cephstoroffice' ...
>>> trying to acquire cfs lock 'storage-cephstoroffice' ...
>>> trying to acquire cfs lock 'storage-cephstoroffice' ...
>>> trying to acquire cfs lock 'storage-cephstoroffice' ...
>>> trying to acquire cfs lock 'storage-cephstoroffice' ...
>>> trying to acquire cfs lock 'storage-cephstoroffice' ...
>>> error with cfs lock 'storage-cephstoroffice': got lock request timeout
>>> TASK OK
>>>
>>> so the vm was deleted but the storage was left unclean. Is this a known
>>> bug? If not can someone point me to the code so i can provide a patch.
>>
>> this was changed intentionally because users complained that there is no
>> way to delete a VM config that references an undeletable disk (e.g.,
>> because a storage is down).
> 
> hui - really? But this leaves you unused disks behind? What happens if
> another user get's the same VM id? This sounds a bit weird to me.
> 
>> if you want to improve this further, we could discuss adding a 'force'
>> parameter to the 'destroy_vm' API call (in PVE/API2/Qemu.pm) and
>> 'destroy_vm' in PVE/QemuServer.pm, and adapting the latter to only
>> ignore disk removal errors in case it is set?
>>
>> I am not sure how many people might rely on the current behaviour, which
>> has been like this since (the end of) 2016..

Can you point me to the commit / package where this change was
introduced? I've searched for it but couldn't finde it.

Stefan

>>
>> ___
>> pve-devel mailing list
>> pve-devel@pve.proxmox.com
>> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
>>
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] vm deletion succeeds even if storage deletion fails

[Stefan Priebe - Profihost AG]

Am 15.01.19 um 08:15 schrieb Fabian Grünbichler:
> On Mon, Jan 14, 2019 at 11:04:29AM +0100, Stefan Priebe - Profihost AG wrote:
>> Hello,
>>
>> today i was wondering about some disk images while the vm was deleted.
>>
>> Inspecting the task history i found this log:
>>
>> trying to acquire cfs lock 'storage-cephstoroffice' ...
>> trying to acquire cfs lock 'storage-cephstoroffice' ...
>> trying to acquire cfs lock 'storage-cephstoroffice' ...
>> trying to acquire cfs lock 'storage-cephstoroffice' ...
>> trying to acquire cfs lock 'storage-cephstoroffice' ...
>> trying to acquire cfs lock 'storage-cephstoroffice' ...
>> trying to acquire cfs lock 'storage-cephstoroffice' ...
>> trying to acquire cfs lock 'storage-cephstoroffice' ...
>> trying to acquire cfs lock 'storage-cephstoroffice' ...
>> Could not remove disk 'cephstoroffice:vm-202-disk-1', check manually:
>> error with cfs lock 'storage-cephstoroffice': got lock request timeout
>> trying to acquire cfs lock 'storage-cephstoroffice' ...
>> trying to acquire cfs lock 'storage-cephstoroffice' ...
>> trying to acquire cfs lock 'storage-cephstoroffice' ...
>> trying to acquire cfs lock 'storage-cephstoroffice' ...
>> trying to acquire cfs lock 'storage-cephstoroffice' ...
>> trying to acquire cfs lock 'storage-cephstoroffice' ...
>> trying to acquire cfs lock 'storage-cephstoroffice' ...
>> trying to acquire cfs lock 'storage-cephstoroffice' ...
>> trying to acquire cfs lock 'storage-cephstoroffice' ...
>> error with cfs lock 'storage-cephstoroffice': got lock request timeout
>> TASK OK
>>
>> so the vm was deleted but the storage was left unclean. Is this a known
>> bug? If not can someone point me to the code so i can provide a patch.
> 
> this was changed intentionally because users complained that there is no
> way to delete a VM config that references an undeletable disk (e.g.,
> because a storage is down).

hui - really? But this leaves you unused disks behind? What happens if
another user get's the same VM id? This sounds a bit weird to me.

> if you want to improve this further, we could discuss adding a 'force'
> parameter to the 'destroy_vm' API call (in PVE/API2/Qemu.pm) and
> 'destroy_vm' in PVE/QemuServer.pm, and adapting the latter to only
> ignore disk removal errors in case it is set?
> 
> I am not sure how many people might rely on the current behaviour, which
> has been like this since (the end of) 2016..
> 
> ___
> pve-devel mailing list
> pve-devel@pve.proxmox.com
> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
> 
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] vm deletion succeeds even if storage deletion fails

[Stefan Priebe - Profihost AG]

Hello,

today i was wondering about some disk images while the vm was deleted.

Inspecting the task history i found this log:

trying to acquire cfs lock 'storage-cephstoroffice' ...
trying to acquire cfs lock 'storage-cephstoroffice' ...
trying to acquire cfs lock 'storage-cephstoroffice' ...
trying to acquire cfs lock 'storage-cephstoroffice' ...
trying to acquire cfs lock 'storage-cephstoroffice' ...
trying to acquire cfs lock 'storage-cephstoroffice' ...
trying to acquire cfs lock 'storage-cephstoroffice' ...
trying to acquire cfs lock 'storage-cephstoroffice' ...
trying to acquire cfs lock 'storage-cephstoroffice' ...
Could not remove disk 'cephstoroffice:vm-202-disk-1', check manually:
error with cfs lock 'storage-cephstoroffice': got lock request timeout
trying to acquire cfs lock 'storage-cephstoroffice' ...
trying to acquire cfs lock 'storage-cephstoroffice' ...
trying to acquire cfs lock 'storage-cephstoroffice' ...
trying to acquire cfs lock 'storage-cephstoroffice' ...
trying to acquire cfs lock 'storage-cephstoroffice' ...
trying to acquire cfs lock 'storage-cephstoroffice' ...
trying to acquire cfs lock 'storage-cephstoroffice' ...
trying to acquire cfs lock 'storage-cephstoroffice' ...
trying to acquire cfs lock 'storage-cephstoroffice' ...
error with cfs lock 'storage-cephstoroffice': got lock request timeout
TASK OK

so the vm was deleted but the storage was left unclean. Is this a known
bug? If not can someone point me to the code so i can provide a patch.

Greets,
Stefan
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] firewall : possible bug/race when cluster.fw is replicated and rules are updated ?

[Stefan Priebe - Profihost AG]

Hi Alexandre,


Am 08.01.19 um 21:55 schrieb Alexandre DERUMIER:
>>> But, file_set_contents - which save_clusterfw_conf uses - does this 
>>> already[0],
>>> so maybe this is the "high-level fuse rename isn't atomic" bug again...
>>> May need to take a closer look tomorrow.
> 
> mmm, ok.
> 
> In my case, it was with a simple file copy (cp /tmp/cluster.fw 
> /etc/pve/firewall/cluster.fw). (I manage cluster.fw through puppet for 
> multiple cluster).
> can reproduce it too with a simple vi edition.
> 
> I think others users could trigger this too, if they use scripts to automate 
> ipset (blacklist, ).
> 
> Maybe could it be better to only disable firewall when option is setup with 
> "enabled:0", and if cluster.fw is missing, don't change any rules.
> ²²²


for those cases i use something like (pseudocode - i use salt not puppet):

- manage copy of file
- if file has changed trigger:
   - mv -v $managedfile $realfile


Greets,
Stefan

> 
> 
> 
> - Mail original -
> De: "Thomas Lamprecht" 
> À: "pve-devel" , "aderumier" 
> Envoyé: Mardi 8 Janvier 2019 20:58:51
> Objet: Re: [pve-devel] firewall : possible bug/race when cluster.fw is 
> replicated and rules are updated ?
> 
> Hi, 
> 
> On 1/8/19 7:37 PM, Alexandre DERUMIER wrote: 
>> I'm able to reproduce with: 
>> --- 
>> on 1 host: 
>>
>> cluster.fw: 
>> [OPTIONS] 
>>
>> enable: 1 
>> policy_in: ACCEPT 
>>
>>
>>
>>
>> #!/usr/bin/perl 
>>
>> use IO::File; 
>> use PVE::Firewall; 
>> use Data::Dumper; 
>> use Time::HiRes qw ( time alarm sleep usleep ); 
>>
>> while(1){ 
>>
>> $filename = "/etc/pve/firewall/cluster.fw"; 
>>
>> if (my $fh = IO::File->new($filename, O_RDONLY)) { 
>>
>> $cluster_conf = PVE::Firewall::parse_clusterfw_config($filename, $fh, 
>> $verbose); 
>> my $cluster_options = $cluster_conf->{options}; 
>>
>> if (!$cluster_options->{enable}) { 
>> print Dumper($cluster_options); 
>> die "error\n"; 
>> } 
>>
>> } 
>> usleep(100); 
>> }; 
>>
>>
>> the script is running fine. 
>>
>>
>> on another host, edit the file (simple open/write), 
>> then the script on first host, return 
>>
>> $VAR1 = {}; 
>> error 
> 
> that is expected, AFAICT, a modify operation shouldn't be: 
> * read FILE -> modify -> write FILE 
> but rather: 
> * read FILE -> modify -> write FILE.TMP -> move FILE.TMP to FILE 
> if it's wanted that always a valid content is read. Else yes, you may have a 
> small 
> time window where the file is truncated. 
> 
> But, file_set_contents - which save_clusterfw_conf uses - does this 
> already[0], 
> so maybe this is the "high-level fuse rename isn't atomic" bug again... 
> May need to take a closer look tomorrow. 
> 
> [0]: 
> https://git.proxmox.com/?p=pve-common.git;a=blob;f=src/PVE/Tools.pm;h=accf6539da94d2b5d5b6f4539310fe5c4d526c7e;hb=HEAD#l213
>  
> 
>>
>> - Mail original - 
>> De: "aderumier"  
>> À: "pve-devel"  
>> Envoyé: Mardi 8 Janvier 2019 19:15:06 
>> Objet: [pve-devel] firewall : possible bug/race when cluster.fw is 
>> replicated and rules are updated ? 
>>
>> Hi, 
>> I'm currently debugging a possible firewalling problem. 
>> I'm running some cephfs client in vm, firewalled by proxmox. 
>> cephfs client are really sensitive to network problem, and mainly with 
>> packets logss or dropped packets. 
>>
>> I'm really not sure, but I have currently puppet updating my cluster.fw, at 
>> regular interval, 
>> and sometimes, I have all the vm on a specific host (or multiple hosts), at 
>> the same time, have a small disconnect (maybe some second). 
>>
>>
>> I would like to known, if cluster.fw replication is atomic in /etc/pve/ ? 
>> or if they are any chance, that during file replication, the firewall try to 
>> read the file, 
>> it could be empty ? 
>>
>>
>> I just wonder (I'm really really not sure) if I could trigger this: 
>>
>>
>> sub update { 
>> my $code = sub { 
>>
>> my $cluster_conf = load_clusterfw_conf(); 
>> my $cluster_options = $cluster_conf->{options}; 
>>
>> if (!$cluster_options->{enable}) { 
>> PVE::Firewall::remove_pvefw_chains(); 
>> return; 
>> } 
>>
>>
>> cluster.conf not readable/absent/ , and remove_pvefw_chains called. 
>> then after some seconds, rules are applied again. 
>>
>>
>> I'm going to add some log to try to reproduce it. (BTW, it could be great to 
>> logs rules changed, maybe an audit log with a diff could be great) 
>> ___ 
>> pve-devel mailing list 
>> pve-devel@pve.proxmox.com 
>> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel 
>>
>> ___ 
>> pve-devel mailing list 
>> pve-devel@pve.proxmox.com 
>> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel 
> 
> ___
> pve-devel mailing list
> pve-devel@pve.proxmox.com
> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
> 
___
pve-devel mailing list
pve-devel@pve.proxmox.com

[pve-devel] tunnel replied 'ERR: resume failed - unable to find configuration file for VM 214 - no such machine' to command 'resume 214'

[Stefan Priebe - Profihost AG]

Hello,

since upgrading to PVE 5 i'm seeing the following error on a regular
basis - while stress testing migration (doing 100-200 migrations in a row):

2018-10-23 13:54:42 migration speed: 384.00 MB/s - downtime 113 ms
2018-10-23 13:54:42 migration status: completed
2018-10-23 13:54:42 ERROR: tunnel replied 'ERR: resume failed - unable
to find configuration file for VM 214 - no such machine' to command
'resume 214'
2018-10-23 13:54:47 ERROR: migration finished with problems (duration
00:00:31)

Is this something to debug further?

Greets,
Stefan
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] Unit 240.scope already exists.

[Stefan Priebe - Profihost AG]

Am 12.09.2018 um 10:19 schrieb Wolfgang Bumiller:
> On Wed, Sep 12, 2018 at 08:29:02AM +0200, Stefan Priebe - Profihost AG wrote:
>> Hello,
>>
>> i don't know whether this is a known bug but since Proxmox V5 i have
>> seen the following error message several times while trying to start a
>> vm after shutdown:
>> ERROR: start failed:
>> org.freedesktop.systemd1.UnitExists: Unit 240.scope already exists
> 
> We've seen this happen. I'm inclined to consider it a systemd issue
> since we perform a `systemctl stop` on the scope at startup. And
> particularly when the qemu process is already gone it's weird that this
> command returns before the scope is actually gone.
> To me this looks like a timing issue. (And the systemctl command line
> tool also doesn't give us a lot of choices regarding synchronization,
> eg. I wish there was a 'systemctl wait' to wait for a job to finish...).
> 
> Since the explicit unconditional `systemctl stop` doesn't seem to be
> doing the trick we may just have to talk to systemd more via dbus at
> startup.
> We already use dbus to create the scope in the first place, but waiting
> for jobs we don't know exist anymore after trying to make sure they
> actually don't hasn't felt too appealing to us...
> 
>>
>> I had a similiar problem under our ubuntu 18.04 workstations where the
>> only solution was to tun:
>>
>> rm -fv /run/systemd/transient/$name.scope
>> systemctl daemon-reload
> 
> When it's not a timing issue, a `systemctl stop $name.scope` should work
> as well. If it doesn't, I'd very much like to know why (and why
> systemctl doesn't tell you about it...).

yes that's most probably correct and yes i also think this is a systemd
issue.


What about implementing a retry / wait 1s for max 5s if the scope is
stopped but still exists?

Greets,
Stefan
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] Unit 240.scope already exists.

[Stefan Priebe - Profihost AG]

Hello,

i don't know whether this is a known bug but since Proxmox V5 i have
seen the following error message several times while trying to start a
vm after shutdown:
ERROR: start failed:
org.freedesktop.systemd1.UnitExists: Unit 240.scope already exists

I had a similiar problem under our ubuntu 18.04 workstations where the
only solution was to tun:

rm -fv /run/systemd/transient/$name.scope
systemctl daemon-reload

Greets,
Stefan
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] qemu config and balloon

[Stefan Priebe - Profihost AG]

Am 04.09.2018 um 10:10 schrieb Thomas Lamprecht:
> On 9/4/18 10:00 AM, Dominik Csapak wrote:
>> On 09/04/2018 09:24 AM, Stefan Priebe - Profihost AG wrote:
>>> I searched the PVE docs but didn't find anything.
>> [snip]
>> also in the qm manpage under 'Memory' there is a more detailed
>> description of the autoballooning
>>
> 
> I.e.: https://pve.proxmox.com/pve-docs/chapter-qm.html#qm_memory

thanks!

> 
> ___
> pve-devel mailing list
> pve-devel@pve.proxmox.com
> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
> 
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] qemu config and balloon

[Stefan Priebe - Profihost AG]

Hello list,

can anybody enlighten me where the difference between:
balloon: 0 and balloon not defined at all in the config file is?

I searched the PVE docs but didn't find anything. It seems balloon: 0 is
set if ballooning is disabled. And if the balloon value is identical to
the memory value balloon is not defined at all.

Greets,
Stefan
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH pve-docs] add ibpb, ssbd, virt-ssbd, amd-ssbd, amd-no-ssb, pdpe1gb cpu flags

[Stefan Priebe - Profihost AG]

Am 28.08.2018 um 10:47 schrieb Thomas Lamprecht:
> On 8/27/18 7:50 PM, Stefan Priebe - Profihost AG wrote:
>> I'm using them as a default since 2 weeks. No problems so far.
>>
> 
> for the backend this is probably OK.
> 
> The GUI part isn't as easy to make sane.
> 
> So there's all those flags, you have *no* guarantee to have any of them
> (even if virt-ssbd sounds like it)
> Intel gets ssbd or not, depending on microcode version (or future
> CPU models)
> AMD can have virt-ssbd, and additionally amd-ssbd (the later implies
> the former, but not vice versa).
> 
> The pdpe1gb flag is something completely different and not really security
> related, so I'd add it in another commit.. 
> 
> Problem is with migration, even in a HW homogeneous environment (all CPUs
> are the same model/revision) a microcode version difference can make it fail.
> 
> Migration from Intel to AMD or the other way is not possible, but this is
> the same with the already existing spec-ctrl, AFAIS.
> 
> So better to make a single SSBD flag in the GUI and map it to whatever we
> have available at start in the host CPU or make a CPU Flag selector exposing
> all those options?

I've handled it differently and made a datacenter option on my own out
of them. So i can set default cpu flags for each proxmox datacenter.
They're added to the customer ones. Not sure if this is something to
work for PVE in general.

Greets.
Stefan

> 
>>
>> Am 27.08.2018 um 18:01 schrieb Alexandre DERUMIER:
>>> any comments to add theses cpu flags ?
>>>
>>>
>>> - Mail original -
>>> De: "aderumier" 
>>> À: "pve-devel" 
>>> Envoyé: Lundi 20 Août 2018 18:26:50
>>> Objet: Re: [pve-devel] [PATCH pve-docs] add ibpb, ssbd, virt-ssbd, 
>>> amd-ssbd, amd-no-ssb, pdpe1gb cpu flags
>>>
>>> Sorry, it's for qemu-server package. 
>>>
>>> I'll rework the pve-docs tomorrow, with amd && intel flags 
>>>
>>>
>>> - Mail original - 
>>> De: "Alexandre Derumier"  
>>> À: "pve-devel"  
>>> Cc: "Alexandre Derumier"  
>>> Envoyé: Lundi 20 Août 2018 17:53:18 
>>> Objet: [PATCH pve-docs] add ibpb,ssbd,virt-ssbd,amd-ssbd,amd-no-ssb,pdpe1gb 
>>> cpu flags 
>>>
>>> see: https://www.berrange.com/tags/ssbd/ 
>>> --- 
>>> PVE/QemuServer.pm | 4 ++-- 
>>> 1 file changed, 2 insertions(+), 2 deletions(-) 
>>>
>>> diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm 
>>> index 1c0fba2..015f8f7 100644 
>>> --- a/PVE/QemuServer.pm 
>>> +++ b/PVE/QemuServer.pm 
>>> @@ -155,7 +155,7 @@ my $cpu_vendor_list = { 
>>> max => 'default', 
>>> }; 
>>>
>>> -my $cpu_flag = qr/[+-](pcid|spec-ctrl)/; 
>>> +my $cpu_flag = 
>>> qr/[+-](pcid|spec-ctrl|ibpb|ssbd|virt-ssbd|amd-ssbd|amd-no-ssb|pdpe1gb)/; 
>>>
>>> my $cpu_fmt = { 
>>> cputype => { 
>>> @@ -174,7 +174,7 @@ my $cpu_fmt = { 
>>> flags => { 
>>> description => "List of additional CPU flags separated by ';'." 
>>> . " Use '+FLAG' to enable, '-FLAG' to disable a flag." 
>>> - . " Currently supported flags: 'pcid', 'spec-ctrl'.", 
>>> + . " Currently supported flags: 'pcid', 'spec-ctrl', 'ibpb', 'ssbd', 
>>> 'virt-ssbd', 'amd-ssbd', 'amd-no-ssb', 'pdpe1gb'.", 
>>> format_description => '+FLAG[;-FLAG...]', 
>>> type => 'string', 
>>> pattern => qr/$cpu_flag(;$cpu_flag)*/, 
>>>
>> ___
>> pve-devel mailing list
>> pve-devel@pve.proxmox.com
>> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
>>
> 
> 
> 
> 
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH pve-docs] add ibpb, ssbd, virt-ssbd, amd-ssbd, amd-no-ssb, pdpe1gb cpu flags

[Stefan Priebe - Profihost AG]

I'm using them as a default since 2 weeks. No problems so far.

Greets,
Stefan

Am 27.08.2018 um 18:01 schrieb Alexandre DERUMIER:
> any comments to add theses cpu flags ?
> 
> 
> - Mail original -
> De: "aderumier" 
> À: "pve-devel" 
> Envoyé: Lundi 20 Août 2018 18:26:50
> Objet: Re: [pve-devel] [PATCH pve-docs] add ibpb, ssbd, virt-ssbd, amd-ssbd, 
> amd-no-ssb, pdpe1gb cpu flags
> 
> Sorry, it's for qemu-server package. 
> 
> I'll rework the pve-docs tomorrow, with amd && intel flags 
> 
> 
> - Mail original - 
> De: "Alexandre Derumier"  
> À: "pve-devel"  
> Cc: "Alexandre Derumier"  
> Envoyé: Lundi 20 Août 2018 17:53:18 
> Objet: [PATCH pve-docs] add ibpb,ssbd,virt-ssbd,amd-ssbd,amd-no-ssb,pdpe1gb 
> cpu flags 
> 
> see: https://www.berrange.com/tags/ssbd/ 
> --- 
> PVE/QemuServer.pm | 4 ++-- 
> 1 file changed, 2 insertions(+), 2 deletions(-) 
> 
> diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm 
> index 1c0fba2..015f8f7 100644 
> --- a/PVE/QemuServer.pm 
> +++ b/PVE/QemuServer.pm 
> @@ -155,7 +155,7 @@ my $cpu_vendor_list = { 
> max => 'default', 
> }; 
> 
> -my $cpu_flag = qr/[+-](pcid|spec-ctrl)/; 
> +my $cpu_flag = 
> qr/[+-](pcid|spec-ctrl|ibpb|ssbd|virt-ssbd|amd-ssbd|amd-no-ssb|pdpe1gb)/; 
> 
> my $cpu_fmt = { 
> cputype => { 
> @@ -174,7 +174,7 @@ my $cpu_fmt = { 
> flags => { 
> description => "List of additional CPU flags separated by ';'." 
> . " Use '+FLAG' to enable, '-FLAG' to disable a flag." 
> - . " Currently supported flags: 'pcid', 'spec-ctrl'.", 
> + . " Currently supported flags: 'pcid', 'spec-ctrl', 'ibpb', 'ssbd', 
> 'virt-ssbd', 'amd-ssbd', 'amd-no-ssb', 'pdpe1gb'.", 
> format_description => '+FLAG[;-FLAG...]', 
> type => 'string', 
> pattern => qr/$cpu_flag(;$cpu_flag)*/, 
> 
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] missing cpu flags? (CVE-2018-3639)

[Stefan Priebe - Profihost AG]

Am 20.08.2018 um 17:19 schrieb Alexandre DERUMIER:
> Hi Stefan,
> 
> thanks for the infos!
> 
> 
>>> At least ssbd is important for guest to mitigate CVE-2018-3639. 
> 
> This need qemu 3.0 :/
> 
> https://wiki.qemu.org/ChangeLog/3.0
> 
> "The 'ssbd', 'virt-ssbd', 'amd-ssbd' and 'amd-no-ssb' CPU feature flags are 
> added in relation to the "Speculative Store Bypass" hardware vulnerability 
> (CVE-2018-3639)"

You already answered yourself ;-) it's working fine with 2.11.2. I'm
already using it since a few days.

>>> It also seems to make sense to enable pdpe1gb 
> 
> is it related to a vulnerability ?
No.


> it's already possible to use hugepage currently with "hugepages: <1024 | 2 | 
> any>". But it's only on the qemu/hostside.
> I think pdpe1gb expose hugepage inside the guest, right ?
Yes.

Stefan

> 
> - Mail original -
> De: "Stefan Priebe, Profihost AG" 
> À: "pve-devel" 
> Envoyé: Vendredi 17 Août 2018 13:30:10
> Objet: [pve-devel] missing cpu flags? (CVE-2018-3639)
> 
> Hello, 
> 
> after researching l1tf mitigation for qemu and reading 
> https://www.berrange.com/posts/2018/06/29/cpu-model-configuration-for-qemu-kvm-on-x86-hosts/
>  
> 
> It seems pve misses at least the following cpu flag: 
> ssbd 
> 
> It also seems to make sense to enable pdpe1gb 
> 
> At least ssbd is important for guest to mitigate CVE-2018-3639. 
> 
> Greets, 
> Stefan 
> 
> Excuse my typo sent from my mobile phone. 
> ___ 
> pve-devel mailing list 
> pve-devel@pve.proxmox.com 
> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel 
> 
> ___
> pve-devel mailing list
> pve-devel@pve.proxmox.com
> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
> 
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] missing cpu flags? (CVE-2018-3639)

[Stefan Priebe - Profihost AG]

Hello,

after researching l1tf mitigation for qemu and reading 
https://www.berrange.com/posts/2018/06/29/cpu-model-configuration-for-qemu-kvm-on-x86-hosts/

It seems pve misses at least the following cpu flag:
ssbd

It also seems to make sense to enable pdpe1gb

At least ssbd is important for guest to mitigate CVE-2018-3639.

Greets,
Stefan

Excuse my typo sent from my mobile phone.
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] Hyperconverged Cloud / Qemu + Ceph on same node

[Stefan Priebe - Profihost AG]

Am 24.07.2018 um 09:25 schrieb Dietmar Maurer:
>> Am 23.07.2018 um 21:04 schrieb Alexandre DERUMIER:
>>> Personally, I think that a vm could take all cpu usage, or memory, and
>>> impact ceph cluster for other vms.
>>>
>>> we should give ceph some kind of (configurable) priority.
>>
>> Yes the problem is or might be. Most users will overcommit the cpu cores
>> with VMs. So theoretically the kvm processes might eat ALL CPU cores.
> 
> Really? I though we have a scheduler that give all processes a change to run.
> 
>> But while running ceph on the same node this should NEVER happen. So my
>> idea is to give cores exclusively to ceph.
> 
> Why exclusively? A scheduler can prioritize specific processes.

I'm not sure how this influences latency - it will move the processes
between cores and still fight against qemu. Even if the scheduler is
fast enough you still have the NUMA problem.

See for example:
http://tracker.ceph.com/projects/ceph/wiki/Tuning_for_All_Flash_Deployments#Ceph-Storage-Node-NUMA-Tuning

Greets,
Stefan

___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] Hyperconverged Cloud / Qemu + Ceph on same node

[Stefan Priebe - Profihost AG]

Am 23.07.2018 um 21:04 schrieb Alexandre DERUMIER:
> Personally, I think that a vm could take all cpu usage, or memory, and impact 
> ceph cluster for other vms.
> 
> we should give ceph some kind of (configurable) priority.

Yes the problem is or might be. Most users will overcommit the cpu cores
with VMs. So theoretically the kvm processes might eat ALL CPU cores.
But while running ceph on the same node this should NEVER happen. So my
idea is to give cores exclusively to ceph.

Another problem is NUMA currently we use single CPUs to prevent all the
NUMA mess for ceph. So if i want to run ceph on Dual Socket Machines i
also want to pin ceph exclusively to cores on the same CPU.

Greets,
Stefan

> - Mail original -
> De: "dietmar" 
> À: "pve-devel" , "aderumier" 
> Envoyé: Lundi 23 Juillet 2018 20:53:19
> Objet: Re: [pve-devel] Hyperconverged Cloud / Qemu + Ceph on same node
> 
> I am not sure CPU pinning helps. What problem do you want to solve 
> exactly? 
> 
>> maybe could we use cgroups ? (in ceph systemd units) 
>>
>> we already use them fo vm && ct (shares cpu option for priority) 
>>
>>
>> - Mail original - 
>> De: "Stefan Priebe, Profihost AG"  
>> À: "pve-devel"  
>> Envoyé: Lundi 23 Juillet 2018 13:49:17 
>> Objet: [pve-devel] Hyperconverged Cloud / Qemu + Ceph on same node 
>>
>> Hello, 
>>
>> after listening / reading: 
>> https://www.openstack.org/videos/vancouver-2018/high-performance-ceph-for-hyper-converged-telco-nfv-infrastructure
>>  
>>
>> and 
>> https://www.youtube.com/watch?v=0_V-L7_CDTs=youtu.be 
>> and 
>> https://arxiv.org/pdf/1802.08102.pdf 
>>
>> I was thinking about creating a Proxmox based Cloud with Ceph on the 
>> same nodes as proxmox. 
>>
>> What i'm missing is HOW do i get automatic CPU pinning for Qemu and 
>> Ceph? How can they live in parallel without manually adjusting cpu 
>> pinning lists? Has anybody already tried it? 
>>
>> Greets, 
>> Stefan 
>> ___ 
>> pve-devel mailing list 
>> pve-devel@pve.proxmox.com 
>> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel 
>>
>> ___ 
>> pve-devel mailing list 
>> pve-devel@pve.proxmox.com 
>> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel 
> 
> ___
> pve-devel mailing list
> pve-devel@pve.proxmox.com
> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
> 
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] Hyperconverged Cloud / Qemu + Ceph on same node

[Stefan Priebe - Profihost AG]

Hello,

after listening / reading:
https://www.openstack.org/videos/vancouver-2018/high-performance-ceph-for-hyper-converged-telco-nfv-infrastructure
and
https://www.youtube.com/watch?v=0_V-L7_CDTs=youtu.be
and
https://arxiv.org/pdf/1802.08102.pdf

I was thinking about creating a Proxmox based Cloud with Ceph on the
same nodes as proxmox.

What i'm missing is HOW do i get automatic CPU pinning for Qemu and
Ceph? How can they live in parallel without manually adjusting cpu
pinning lists? Has anybody already tried it?

Greets,
Stefan
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] cpuflag: pcid needed in guest for good performance after meltdown

[Stefan Priebe - Profihost AG]

Am 10.01.2018 um 08:08 schrieb Fabian Grünbichler:
> On Tue, Jan 09, 2018 at 08:47:10PM +0100, Stefan Priebe - Profihost AG wrote:
>> Am 09.01.2018 um 19:25 schrieb Fabian Grünbichler:
>>> On Tue, Jan 09, 2018 at 04:31:40PM +0100, Stefan Priebe - Profihost AG 
>>> wrote:
>>>>
>>>> Am 09.01.2018 um 16:18 schrieb Fabian Grünbichler:
>>>>> On Tue, Jan 09, 2018 at 02:58:24PM +0100, Fabian Grünbichler wrote:
>>>>>> On Mon, Jan 08, 2018 at 09:34:57PM +0100, Stefan Priebe - Profihost AG 
>>>>>> wrote:
>>>>>>> Hello,
>>>>>>>
>>>>>>> for meltdown mitigation and performance it's important to have the pcid
>>>>>>> flag passed down to the guest (f.e.
>>>>>>> https://groups.google.com/forum/m/#!topic/mechanical-sympathy/L9mHTbeQLNU).
>>>>>>>
>>>>>>> My host shows the flag:
>>>>>>> # grep ' pcid ' /proc/cpuinfo  | wc -l
>>>>>>> 56
>>>>>>>
>>>>>>> But the guest does not:
>>>>>>> # grep pcid /proc/cpuinfo
>>>>>>> #
>>>>>>>
>>>>>>> Guest was started with:
>>>>>>> -cpu IvyBridge,+kvm_pv_unhalt,+kvm_pv_eoi,enforce,vendor=GenuineIntel
>>>>>>>
>>>>>>> Is this something missing in host kernel or in PVE?
>>>>>>>
>>>>>>> Greets,
>>>>>>> Stefan
>>>>>>
>>>>>> we are preparing a patch for qemu-server to allow enabling the pcid
>>>>>> cpuflag on a VM config level (especially since most VMs will run with
>>>>>> kvm64, where changing the default is not an option!).
>>>>>>
>>>>>> switching it on by default for SandyBridge and co (which are missing it
>>>>>> in Qemu, but support it technically) will likely happen with the move to
>>>>>> Qemu 2.11, pending further feedback and review (it is not clear at all
>>>>>> how various guest OS handle getting the flag changed out under them when
>>>>>> migrating, and we don't want to integrate specific pinning support in a
>>>>>> rushed through manner).
>>>>>>
>>>>>> for now you can of course just patch a hardcoded +pcid in to the cpu
>>>>>> flag list on your hosts if you know your CPUs all support it and
>>>>>> stop/start your VMs to regain lost performance.
>>>>>>
>>>>>
>>>>> should be on pvetest for 5.x and 4.x shortly - please provide feedback!
>>>>>
>>>>> patches for exposing it somewhere on the GUI will follow (most likely
>>>>> tomorrow).
>>>>
>>>> Thanks! I would love to see it as a default for the specific qemu cpu
>>>> models.
>>>>
>>>> I mean we know exactly that sandybridge, ivybridge, ... support it or not?
>>>> ,
>>>
>>> yes we do. but we are not 100% sure they won't cause problems when
>>> live-migrating from without PCID to with PCID (on all guest operating
>>> systems), and instead of introducing one-off pinning mechanisms for this
>>> now in a rush, we give you a simple way (both from a code and from a
>>> user perspective) to set it on all your VMs manually and trigger
>>> shutdown/start cycles. the same option can be extended to support
>>> selectively enabling or disabling other CPU flags in the feature (which
>>> has been requested for non-security reasons a couple of times already).
>>>
>>> the default for SandyBridge and co will likely be changed with the next
>>> Qemu release (and accompagnying bump of machine type, which is already
>>> integrated into our live-migration mechanism).
>>
>> If i understand the patches correctly this means we can't use the old
>> CPU models any longer but need to use Haswell-IBRS,
>> Sandybridge-Haswell-IBRS, ...
>>
>> See:
>> http://lists.nongnu.org/archive/html/qemu-devel/2018-01/msg01562.html
>>
>> and
>>
>> http://lists.nongnu.org/archive/html/qemu-devel/2018-01/msg01563.html
>>
>> Stefan
> 
> IBRS is for the Spectre mitigation. since those are new machine types,
> they can get the PCID by default without any problems. the unclear ones
> are the existing Sandybridge (and co) without IBRS.

Yes sure i just wanted to say that qemu itself does not plan to extend
the existing types.

> 
> ___
> pve-devel mailing list
> pve-devel@pve.proxmox.com
> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
> 
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] cpuflag: pcid needed in guest for good performance after meltdown

[Stefan Priebe - Profihost AG]

Am 09.01.2018 um 19:25 schrieb Fabian Grünbichler:
> On Tue, Jan 09, 2018 at 04:31:40PM +0100, Stefan Priebe - Profihost AG wrote:
>>
>> Am 09.01.2018 um 16:18 schrieb Fabian Grünbichler:
>>> On Tue, Jan 09, 2018 at 02:58:24PM +0100, Fabian Grünbichler wrote:
>>>> On Mon, Jan 08, 2018 at 09:34:57PM +0100, Stefan Priebe - Profihost AG 
>>>> wrote:
>>>>> Hello,
>>>>>
>>>>> for meltdown mitigation and performance it's important to have the pcid
>>>>> flag passed down to the guest (f.e.
>>>>> https://groups.google.com/forum/m/#!topic/mechanical-sympathy/L9mHTbeQLNU).
>>>>>
>>>>> My host shows the flag:
>>>>> # grep ' pcid ' /proc/cpuinfo  | wc -l
>>>>> 56
>>>>>
>>>>> But the guest does not:
>>>>> # grep pcid /proc/cpuinfo
>>>>> #
>>>>>
>>>>> Guest was started with:
>>>>> -cpu IvyBridge,+kvm_pv_unhalt,+kvm_pv_eoi,enforce,vendor=GenuineIntel
>>>>>
>>>>> Is this something missing in host kernel or in PVE?
>>>>>
>>>>> Greets,
>>>>> Stefan
>>>>
>>>> we are preparing a patch for qemu-server to allow enabling the pcid
>>>> cpuflag on a VM config level (especially since most VMs will run with
>>>> kvm64, where changing the default is not an option!).
>>>>
>>>> switching it on by default for SandyBridge and co (which are missing it
>>>> in Qemu, but support it technically) will likely happen with the move to
>>>> Qemu 2.11, pending further feedback and review (it is not clear at all
>>>> how various guest OS handle getting the flag changed out under them when
>>>> migrating, and we don't want to integrate specific pinning support in a
>>>> rushed through manner).
>>>>
>>>> for now you can of course just patch a hardcoded +pcid in to the cpu
>>>> flag list on your hosts if you know your CPUs all support it and
>>>> stop/start your VMs to regain lost performance.
>>>>
>>>
>>> should be on pvetest for 5.x and 4.x shortly - please provide feedback!
>>>
>>> patches for exposing it somewhere on the GUI will follow (most likely
>>> tomorrow).
>>
>> Thanks! I would love to see it as a default for the specific qemu cpu
>> models.
>>
>> I mean we know exactly that sandybridge, ivybridge, ... support it or not?
>>,
> 
> yes we do. but we are not 100% sure they won't cause problems when
> live-migrating from without PCID to with PCID (on all guest operating
> systems), and instead of introducing one-off pinning mechanisms for this
> now in a rush, we give you a simple way (both from a code and from a
> user perspective) to set it on all your VMs manually and trigger
> shutdown/start cycles. the same option can be extended to support
> selectively enabling or disabling other CPU flags in the feature (which
> has been requested for non-security reasons a couple of times already).
> 
> the default for SandyBridge and co will likely be changed with the next
> Qemu release (and accompagnying bump of machine type, which is already
> integrated into our live-migration mechanism).

If i understand the patches correctly this means we can't use the old
CPU models any longer but need to use Haswell-IBRS,
Sandybridge-Haswell-IBRS, ...

See:
http://lists.nongnu.org/archive/html/qemu-devel/2018-01/msg01562.html

and

http://lists.nongnu.org/archive/html/qemu-devel/2018-01/msg01563.html

Stefan
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] cpuflag: pcid needed in guest for good performance after meltdown

[Stefan Priebe - Profihost AG]

Am 09.01.2018 um 16:18 schrieb Fabian Grünbichler:
> On Tue, Jan 09, 2018 at 02:58:24PM +0100, Fabian Grünbichler wrote:
>> On Mon, Jan 08, 2018 at 09:34:57PM +0100, Stefan Priebe - Profihost AG wrote:
>>> Hello,
>>>
>>> for meltdown mitigation and performance it's important to have the pcid
>>> flag passed down to the guest (f.e.
>>> https://groups.google.com/forum/m/#!topic/mechanical-sympathy/L9mHTbeQLNU).
>>>
>>> My host shows the flag:
>>> # grep ' pcid ' /proc/cpuinfo  | wc -l
>>> 56
>>>
>>> But the guest does not:
>>> # grep pcid /proc/cpuinfo
>>> #
>>>
>>> Guest was started with:
>>> -cpu IvyBridge,+kvm_pv_unhalt,+kvm_pv_eoi,enforce,vendor=GenuineIntel
>>>
>>> Is this something missing in host kernel or in PVE?
>>>
>>> Greets,
>>> Stefan
>>
>> we are preparing a patch for qemu-server to allow enabling the pcid
>> cpuflag on a VM config level (especially since most VMs will run with
>> kvm64, where changing the default is not an option!).
>>
>> switching it on by default for SandyBridge and co (which are missing it
>> in Qemu, but support it technically) will likely happen with the move to
>> Qemu 2.11, pending further feedback and review (it is not clear at all
>> how various guest OS handle getting the flag changed out under them when
>> migrating, and we don't want to integrate specific pinning support in a
>> rushed through manner).
>>
>> for now you can of course just patch a hardcoded +pcid in to the cpu
>> flag list on your hosts if you know your CPUs all support it and
>> stop/start your VMs to regain lost performance.
>>
> 
> should be on pvetest for 5.x and 4.x shortly - please provide feedback!
> 
> patches for exposing it somewhere on the GUI will follow (most likely
> tomorrow).

Thanks! I would love to see it as a default for the specific qemu cpu
models.

I mean we know exactly that sandybridge, ivybridge, ... support it or not?

Greets,
Stefan

___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] cpuflag: pcid needed in guest for good performance after meltdown

[Stefan Priebe - Profihost AG]

Am 09.01.2018 um 14:36 schrieb Alexandre DERUMIER:
>>> i've the first customers having impacts. Current badest example: instead
>>> of 12% cpu load 75% cpu load in guest.
> 
> Just curious, what kind of workload ?

MySQL only - small blocks

> I think with ceph and librbd, it'll hurt too for small block.

Yes i'm booting ceph as well with pti=off / have no updated them at all.

> (I think I'll disable the patch cluster/osd side, as it's in a secure 
> network, but client side will be impacted)

So the problem in my case is completely client impacted.

Greets,
Stefan

> 
> - Mail original -
> De: "Stefan Priebe, Profihost AG" <s.pri...@profihost.ag>
> À: "aderumier" <aderum...@odiso.com>
> Cc: "pve-devel" <pve-devel@pve.proxmox.com>
> Envoyé: Mardi 9 Janvier 2018 14:32:27
> Objet: Re: [pve-devel] cpuflag: pcid needed in guest for good performance 
> after meltdown
> 
> Am 09.01.2018 um 14:24 schrieb Alexandre DERUMIER: 
>> ok thanks ! 
> 
> i've the first customers having impacts. Current badest example: instead 
> of 12% cpu load 75% cpu load in guest. 
> 
>> - Mail original - 
>> De: "Stefan Priebe, Profihost AG" <s.pri...@profihost.ag> 
>> À: "aderumier" <aderum...@odiso.com> 
>> Cc: "pve-devel" <pve-devel@pve.proxmox.com> 
>> Envoyé: Mardi 9 Janvier 2018 14:10:27 
>> Objet: Re: [pve-devel] cpuflag: pcid needed in guest for good performance 
>> after meltdown 
>>
>> Am 09.01.2018 um 13:02 schrieb Alexandre DERUMIER: 
>>>>> You mean: 
>>>>> -cpu qemu64,+pcid 
>>>
>>> yes. 
>>>
>>> (don't known the perf difference between qemu64 (without pcid), and intel 
>>> cpu model (without pcid too) ? ) 
>>
>> The results are nearly the same. 
>>
>> Same VM as before now model qemu64,+pcid: 
>> real 0m13.870s 
>> user 0m7.128s 
>> sys 0m6.697s 
>>
>> qemu64: 
>> real 0m25.214s 
>> user 0m16.923s 
>> sys 0m8.956s 
>>
>> Stefan 
>>>
>>> - Mail original - 
>>> De: "Stefan Priebe, Profihost AG" <s.pri...@profihost.ag> 
>>> À: "aderumier" <aderum...@odiso.com> 
>>> Cc: "pve-devel" <pve-devel@pve.proxmox.com> 
>>> Envoyé: Mardi 9 Janvier 2018 12:57:30 
>>> Objet: Re: [pve-devel] cpuflag: pcid needed in guest for good performance 
>>> after meltdown 
>>>
>>> Am 09.01.2018 um 12:55 schrieb Alexandre DERUMIER: 
>>>>>> Yes - see an example which does a lot of syscalls: 
>>>>
>>>> and for qemu64 ? (is it possible to sent +pcid too ?) 
>>>
>>> You mean: 
>>> -cpu qemu64,+pcid 
>>> ? 
>>>
>>> Stefan 
>>>
>>>> - Mail original - 
>>>> De: "Stefan Priebe, Profihost AG" <s.pri...@profihost.ag> 
>>>> À: "aderumier" <aderum...@odiso.com> 
>>>> Cc: "pve-devel" <pve-devel@pve.proxmox.com> 
>>>> Envoyé: Mardi 9 Janvier 2018 12:20:04 
>>>> Objet: Re: [pve-devel] cpuflag: pcid needed in guest for good performance 
>>>> after meltdown 
>>>>
>>>> Am 09.01.2018 um 10:43 schrieb Alexandre DERUMIER: 
>>>>>>> That's bad as pcid is very important to performance for meltdown fixes 
>>>>>>> in the linux kernel. 
>>>>>
>>>>> I wonder the difference of performance for 
>>>>>
>>>>> - qemu64|kvm64 cpu model 
>>>>> - intel cpu model 
>>>>> - intel + pcid cpu model ? 
>>>>>
>>>>> (currently I'm running mainly qemu64 because I don't need advanced cpu 
>>>>> flag) 
>>>>>
>>>>> Do you have already done some benchmarks ? 
>>>>
>>>>
>>>> Yes - see an example which does a lot of syscalls: 
>>>>
>>>> no tasks running other than du: 
>>>>
>>>> no pcid: 
>>>> # time for i in $(seq 1 1 50); do du -sx /; done 
>>>> ... 
>>>> real 0m26.614s 
>>>> user 0m17.548s 
>>>> sys 0m9.056s 
>>>>
>>>>
>>>> kvm started with +pcid: 
>>>> # time for i in $(seq 1 1 50); do du -sx /; done 
>>>> ... 
>>>> real 0m14.734s 
>>>> user 0m7.755s 
>>>> sys 0m6.973s 
>>>>
>>>> Greets, 
>>>> Stefan 
>>

Re: [pve-devel] cpuflag: pcid needed in guest for good performance after meltdown

[Stefan Priebe - Profihost AG]

Am 09.01.2018 um 14:24 schrieb Alexandre DERUMIER:
> ok thanks !

i've the first customers having impacts. Current badest example: instead
of 12% cpu load 75% cpu load in guest.

> - Mail original -
> De: "Stefan Priebe, Profihost AG" <s.pri...@profihost.ag>
> À: "aderumier" <aderum...@odiso.com>
> Cc: "pve-devel" <pve-devel@pve.proxmox.com>
> Envoyé: Mardi 9 Janvier 2018 14:10:27
> Objet: Re: [pve-devel] cpuflag: pcid needed in guest for good performance 
> after meltdown
> 
> Am 09.01.2018 um 13:02 schrieb Alexandre DERUMIER: 
>>>> You mean: 
>>>> -cpu qemu64,+pcid 
>>
>> yes. 
>>
>> (don't known the perf difference between qemu64 (without pcid), and intel 
>> cpu model (without pcid too) ? ) 
> 
> The results are nearly the same. 
> 
> Same VM as before now model qemu64,+pcid: 
> real 0m13.870s 
> user 0m7.128s 
> sys 0m6.697s 
> 
> qemu64: 
> real 0m25.214s 
> user 0m16.923s 
> sys 0m8.956s 
> 
> Stefan 
>>
>> - Mail original - 
>> De: "Stefan Priebe, Profihost AG" <s.pri...@profihost.ag> 
>> À: "aderumier" <aderum...@odiso.com> 
>> Cc: "pve-devel" <pve-devel@pve.proxmox.com> 
>> Envoyé: Mardi 9 Janvier 2018 12:57:30 
>> Objet: Re: [pve-devel] cpuflag: pcid needed in guest for good performance 
>> after meltdown 
>>
>> Am 09.01.2018 um 12:55 schrieb Alexandre DERUMIER: 
>>>>> Yes - see an example which does a lot of syscalls: 
>>>
>>> and for qemu64 ? (is it possible to sent +pcid too ?) 
>>
>> You mean: 
>> -cpu qemu64,+pcid 
>> ? 
>>
>> Stefan 
>>
>>> - Mail original - 
>>> De: "Stefan Priebe, Profihost AG" <s.pri...@profihost.ag> 
>>> À: "aderumier" <aderum...@odiso.com> 
>>> Cc: "pve-devel" <pve-devel@pve.proxmox.com> 
>>> Envoyé: Mardi 9 Janvier 2018 12:20:04 
>>> Objet: Re: [pve-devel] cpuflag: pcid needed in guest for good performance 
>>> after meltdown 
>>>
>>> Am 09.01.2018 um 10:43 schrieb Alexandre DERUMIER: 
>>>>>> That's bad as pcid is very important to performance for meltdown fixes 
>>>>>> in the linux kernel. 
>>>>
>>>> I wonder the difference of performance for 
>>>>
>>>> - qemu64|kvm64 cpu model 
>>>> - intel cpu model 
>>>> - intel + pcid cpu model ? 
>>>>
>>>> (currently I'm running mainly qemu64 because I don't need advanced cpu 
>>>> flag) 
>>>>
>>>> Do you have already done some benchmarks ? 
>>>
>>>
>>> Yes - see an example which does a lot of syscalls: 
>>>
>>> no tasks running other than du: 
>>>
>>> no pcid: 
>>> # time for i in $(seq 1 1 50); do du -sx /; done 
>>> ... 
>>> real 0m26.614s 
>>> user 0m17.548s 
>>> sys 0m9.056s 
>>>
>>>
>>> kvm started with +pcid: 
>>> # time for i in $(seq 1 1 50); do du -sx /; done 
>>> ... 
>>> real 0m14.734s 
>>> user 0m7.755s 
>>> sys 0m6.973s 
>>>
>>> Greets, 
>>> Stefan 
>>>
>>>>
>>>> - Mail original - 
>>>> De: "Stefan Priebe, Profihost AG" <s.pri...@profihost.ag> 
>>>> À: "aderumier" <aderum...@odiso.com> 
>>>> Cc: "pve-devel" <pve-devel@pve.proxmox.com> 
>>>> Envoyé: Mardi 9 Janvier 2018 10:20:39 
>>>> Objet: Re: [pve-devel] cpuflag: pcid needed in guest for good performance 
>>>> after meltdown 
>>>>
>>>> Am 09.01.2018 um 09:18 schrieb Alexandre DERUMIER: 
>>>>> they are a discussion on qemu mailing currently about pcid 
>>>>>
>>>>>
>>>>> From paolo bonzini: 
>>>>>
>>>>> " 
>>>>> Note that PCID is still not supported for guests without EPT, so 
>>>>> this would break ept=0 with recent "-cpu" models. I'm not sure of 
>>>>> a way to fix it; probably it just has to be documented." 
>>>>
>>>> That's bad as pcid is very important to performance for meltdown fixes 
>>>> in the linux kernel. 
>>>>
>>>> Stefan 
>>>>
>>>>>
>>>>> - Mail original - 
>>>>> De: "Stefan Priebe, Profihost AG" <s.pri...@pro

Re: [pve-devel] cpuflag: pcid needed in guest for good performance after meltdown

[Stefan Priebe - Profihost AG]

Am 09.01.2018 um 13:02 schrieb Alexandre DERUMIER:
>>> You mean: 
>>> -cpu qemu64,+pcid
> 
> yes.
> 
> (don't known the perf difference between qemu64 (without pcid), and intel cpu 
> model (without pcid too) ? )

The results are nearly the same.

Same VM as before now model qemu64,+pcid:
real0m13.870s
user0m7.128s
sys 0m6.697s

qemu64:
real 0m25.214s
user 0m16.923s
sys 0m8.956s

Stefan
> 
> - Mail original -
> De: "Stefan Priebe, Profihost AG" <s.pri...@profihost.ag>
> À: "aderumier" <aderum...@odiso.com>
> Cc: "pve-devel" <pve-devel@pve.proxmox.com>
> Envoyé: Mardi 9 Janvier 2018 12:57:30
> Objet: Re: [pve-devel] cpuflag: pcid needed in guest for good performance 
> after meltdown
> 
> Am 09.01.2018 um 12:55 schrieb Alexandre DERUMIER: 
>>>> Yes - see an example which does a lot of syscalls: 
>>
>> and for qemu64 ? (is it possible to sent +pcid too ?) 
> 
> You mean: 
> -cpu qemu64,+pcid 
> ? 
> 
> Stefan 
> 
>> - Mail original - 
>> De: "Stefan Priebe, Profihost AG" <s.pri...@profihost.ag> 
>> À: "aderumier" <aderum...@odiso.com> 
>> Cc: "pve-devel" <pve-devel@pve.proxmox.com> 
>> Envoyé: Mardi 9 Janvier 2018 12:20:04 
>> Objet: Re: [pve-devel] cpuflag: pcid needed in guest for good performance 
>> after meltdown 
>>
>> Am 09.01.2018 um 10:43 schrieb Alexandre DERUMIER: 
>>>>> That's bad as pcid is very important to performance for meltdown fixes 
>>>>> in the linux kernel. 
>>>
>>> I wonder the difference of performance for 
>>>
>>> - qemu64|kvm64 cpu model 
>>> - intel cpu model 
>>> - intel + pcid cpu model ? 
>>>
>>> (currently I'm running mainly qemu64 because I don't need advanced cpu 
>>> flag) 
>>>
>>> Do you have already done some benchmarks ? 
>>
>>
>> Yes - see an example which does a lot of syscalls: 
>>
>> no tasks running other than du: 
>>
>> no pcid: 
>> # time for i in $(seq 1 1 50); do du -sx /; done 
>> ... 
>> real 0m26.614s 
>> user 0m17.548s 
>> sys 0m9.056s 
>>
>>
>> kvm started with +pcid: 
>> # time for i in $(seq 1 1 50); do du -sx /; done 
>> ... 
>> real 0m14.734s 
>> user 0m7.755s 
>> sys 0m6.973s 
>>
>> Greets, 
>> Stefan 
>>
>>>
>>> - Mail original - 
>>> De: "Stefan Priebe, Profihost AG" <s.pri...@profihost.ag> 
>>> À: "aderumier" <aderum...@odiso.com> 
>>> Cc: "pve-devel" <pve-devel@pve.proxmox.com> 
>>> Envoyé: Mardi 9 Janvier 2018 10:20:39 
>>> Objet: Re: [pve-devel] cpuflag: pcid needed in guest for good performance 
>>> after meltdown 
>>>
>>> Am 09.01.2018 um 09:18 schrieb Alexandre DERUMIER: 
>>>> they are a discussion on qemu mailing currently about pcid 
>>>>
>>>>
>>>> From paolo bonzini: 
>>>>
>>>> " 
>>>> Note that PCID is still not supported for guests without EPT, so 
>>>> this would break ept=0 with recent "-cpu" models. I'm not sure of 
>>>> a way to fix it; probably it just has to be documented." 
>>>
>>> That's bad as pcid is very important to performance for meltdown fixes 
>>> in the linux kernel. 
>>>
>>> Stefan 
>>>
>>>>
>>>> - Mail original - 
>>>> De: "Stefan Priebe, Profihost AG" <s.pri...@profihost.ag> 
>>>> À: "pve-devel" <pve-devel@pve.proxmox.com>, "aderumier" 
>>>> <aderum...@odiso.com> 
>>>> Envoyé: Mardi 9 Janvier 2018 08:35:00 
>>>> Objet: Re: [pve-devel] cpuflag: pcid needed in guest for good performance 
>>>> after meltdown 
>>>>
>>>> Am 08.01.2018 um 23:23 schrieb Alexandre DERUMIER: 
>>>>> I think it's not exposed in current cpu model 
>>>>>
>>>>> it can be enabled with "+pcid" 
>>>>>
>>>>>
>>>>> I don't known what it's the best way to add it in proxmox. 
>>>>> We could do it with qemu version upgrade, but qemu 2.10 are not ready. 
>>>>
>>>> Yes but that was always bad at least to me. We're limiting ourselfes if 
>>>> we can only add new qemu features with a new qemu version. 
>>>>
>>>> What about a version or featur

Re: [pve-devel] cpuflag: pcid needed in guest for good performance after meltdown

[Stefan Priebe - Profihost AG]

Am 09.01.2018 um 12:55 schrieb Alexandre DERUMIER:
>>> Yes - see an example which does a lot of syscalls:
> 
> and for qemu64 ? (is it possible to sent +pcid too ?)

You mean:
-cpu qemu64,+pcid
?

Stefan

> - Mail original -----
> De: "Stefan Priebe, Profihost AG" <s.pri...@profihost.ag>
> À: "aderumier" <aderum...@odiso.com>
> Cc: "pve-devel" <pve-devel@pve.proxmox.com>
> Envoyé: Mardi 9 Janvier 2018 12:20:04
> Objet: Re: [pve-devel] cpuflag: pcid needed in guest for good performance 
> after meltdown
> 
> Am 09.01.2018 um 10:43 schrieb Alexandre DERUMIER: 
>>>> That's bad as pcid is very important to performance for meltdown fixes 
>>>> in the linux kernel. 
>>
>> I wonder the difference of performance for 
>>
>> - qemu64|kvm64 cpu model 
>> - intel cpu model 
>> - intel + pcid cpu model ? 
>>
>> (currently I'm running mainly qemu64 because I don't need advanced cpu flag) 
>>
>> Do you have already done some benchmarks ? 
> 
> 
> Yes - see an example which does a lot of syscalls: 
> 
> no tasks running other than du: 
> 
> no pcid: 
> # time for i in $(seq 1 1 50); do du -sx /; done 
> ... 
> real 0m26.614s 
> user 0m17.548s 
> sys 0m9.056s 
> 
> 
> kvm started with +pcid: 
> # time for i in $(seq 1 1 50); do du -sx /; done 
> ... 
> real 0m14.734s 
> user 0m7.755s 
> sys 0m6.973s 
> 
> Greets, 
> Stefan 
> 
>>
>> - Mail original - 
>> De: "Stefan Priebe, Profihost AG" <s.pri...@profihost.ag> 
>> À: "aderumier" <aderum...@odiso.com> 
>> Cc: "pve-devel" <pve-devel@pve.proxmox.com> 
>> Envoyé: Mardi 9 Janvier 2018 10:20:39 
>> Objet: Re: [pve-devel] cpuflag: pcid needed in guest for good performance 
>> after meltdown 
>>
>> Am 09.01.2018 um 09:18 schrieb Alexandre DERUMIER: 
>>> they are a discussion on qemu mailing currently about pcid 
>>>
>>>
>>> From paolo bonzini: 
>>>
>>> " 
>>> Note that PCID is still not supported for guests without EPT, so 
>>> this would break ept=0 with recent "-cpu" models. I'm not sure of 
>>> a way to fix it; probably it just has to be documented." 
>>
>> That's bad as pcid is very important to performance for meltdown fixes 
>> in the linux kernel. 
>>
>> Stefan 
>>
>>>
>>> - Mail original - 
>>> De: "Stefan Priebe, Profihost AG" <s.pri...@profihost.ag> 
>>> À: "pve-devel" <pve-devel@pve.proxmox.com>, "aderumier" 
>>> <aderum...@odiso.com> 
>>> Envoyé: Mardi 9 Janvier 2018 08:35:00 
>>> Objet: Re: [pve-devel] cpuflag: pcid needed in guest for good performance 
>>> after meltdown 
>>>
>>> Am 08.01.2018 um 23:23 schrieb Alexandre DERUMIER: 
>>>> I think it's not exposed in current cpu model 
>>>>
>>>> it can be enabled with "+pcid" 
>>>>
>>>>
>>>> I don't known what it's the best way to add it in proxmox. 
>>>> We could do it with qemu version upgrade, but qemu 2.10 are not ready. 
>>>
>>> Yes but that was always bad at least to me. We're limiting ourselfes if 
>>> we can only add new qemu features with a new qemu version. 
>>>
>>> What about a version or feature field in the guest config which we only 
>>> update on a fresh vm start. 
>>>
>>> Something like this: 
>>> 123.conf: 
>>> ... 
>>> pve_qemu_version: 2.11-10 
>>> ... 
>>>
>>> This field is ONLY and ALWAYS updated in a fresh start not on migration. 
>>> This can be easily detected. 
>>>
>>> Than we can do stuff like 
>>> if (version_cmp($conf->{pve_qemu_version}, "2.11-11")) { 
>>> # enable pcid flag 
>>> } 
>>>
>>> Greets 
>>> Stefan 
>>>
>>>> Maybe add new cpumodel with +pcid enabled ? 
>>>> or add code to manage custom cpuflags and add a checkbox in cpu options ? 
>>>>
>>>>
>>>>
>>>> - Mail original - 
>>>> De: "Stefan Priebe, Profihost AG" <s.pri...@profihost.ag> 
>>>> À: "pve-devel" <pve-devel@pve.proxmox.com> 
>>>> Envoyé: Lundi 8 Janvier 2018 21:34:57 
>>>> Objet: [pve-devel] cpuflag: pcid needed in guest for good performance 
>>>> after meltdown 
>>>>
>>>> Hello, 
>>>>
>>>> for meltdown mitigation and performance it's important to have the pcid 
>>>> flag passed down to the guest (f.e. 
>>>> https://groups.google.com/forum/m/#!topic/mechanical-sympathy/L9mHTbeQLNU).
>>>>  
>>>>
>>>> My host shows the flag: 
>>>> # grep ' pcid ' /proc/cpuinfo | wc -l 
>>>> 56 
>>>>
>>>> But the guest does not: 
>>>> # grep pcid /proc/cpuinfo 
>>>> # 
>>>>
>>>> Guest was started with: 
>>>> -cpu IvyBridge,+kvm_pv_unhalt,+kvm_pv_eoi,enforce,vendor=GenuineIntel 
>>>>
>>>> Is this something missing in host kernel or in PVE? 
>>>>
>>>> Greets, 
>>>> Stefan 
>>>> ___ 
>>>> pve-devel mailing list 
>>>> pve-devel@pve.proxmox.com 
>>>> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel 
>>>>
>>>> ___ 
>>>> pve-devel mailing list 
>>>> pve-devel@pve.proxmox.com 
>>>> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel 
>>>>
>>>
>>
> 
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] cpuflag: pcid needed in guest for good performance after meltdown

[Stefan Priebe - Profihost AG]

Am 09.01.2018 um 10:43 schrieb Alexandre DERUMIER:
>>> That's bad as pcid is very important to performance for meltdown fixes
>>> in the linux kernel.
> 
> I wonder the difference of performance for
> 
> - qemu64|kvm64 cpu model
> - intel cpu model
> - intel + pcid cpu model ?
> 
> (currently I'm running mainly qemu64 because I don't need advanced cpu flag)
> 
> Do you have already done some benchmarks ?


Yes - see an example which does a lot of syscalls:

no tasks running other than du:

no pcid:
# time for i in $(seq 1 1 50); do du -sx /; done
...
real0m26.614s
user0m17.548s
sys 0m9.056s


kvm started with +pcid:
# time for i in $(seq 1 1 50); do du -sx /; done
...
real0m14.734s
user0m7.755s
sys 0m6.973s

Greets,
Stefan

> 
> - Mail original -
> De: "Stefan Priebe, Profihost AG" <s.pri...@profihost.ag>
> À: "aderumier" <aderum...@odiso.com>
> Cc: "pve-devel" <pve-devel@pve.proxmox.com>
> Envoyé: Mardi 9 Janvier 2018 10:20:39
> Objet: Re: [pve-devel] cpuflag: pcid needed in guest for good performance 
> after meltdown
> 
> Am 09.01.2018 um 09:18 schrieb Alexandre DERUMIER: 
>> they are a discussion on qemu mailing currently about pcid 
>>
>>
>> From paolo bonzini: 
>>
>> " 
>> Note that PCID is still not supported for guests without EPT, so 
>> this would break ept=0 with recent "-cpu" models. I'm not sure of 
>> a way to fix it; probably it just has to be documented." 
> 
> That's bad as pcid is very important to performance for meltdown fixes 
> in the linux kernel. 
> 
> Stefan 
> 
>>
>> - Mail original - 
>> De: "Stefan Priebe, Profihost AG" <s.pri...@profihost.ag> 
>> À: "pve-devel" <pve-devel@pve.proxmox.com>, "aderumier" 
>> <aderum...@odiso.com> 
>> Envoyé: Mardi 9 Janvier 2018 08:35:00 
>> Objet: Re: [pve-devel] cpuflag: pcid needed in guest for good performance 
>> after meltdown 
>>
>> Am 08.01.2018 um 23:23 schrieb Alexandre DERUMIER: 
>>> I think it's not exposed in current cpu model 
>>>
>>> it can be enabled with "+pcid" 
>>>
>>>
>>> I don't known what it's the best way to add it in proxmox. 
>>> We could do it with qemu version upgrade, but qemu 2.10 are not ready. 
>>
>> Yes but that was always bad at least to me. We're limiting ourselfes if 
>> we can only add new qemu features with a new qemu version. 
>>
>> What about a version or feature field in the guest config which we only 
>> update on a fresh vm start. 
>>
>> Something like this: 
>> 123.conf: 
>> ... 
>> pve_qemu_version: 2.11-10 
>> ... 
>>
>> This field is ONLY and ALWAYS updated in a fresh start not on migration. 
>> This can be easily detected. 
>>
>> Than we can do stuff like 
>> if (version_cmp($conf->{pve_qemu_version}, "2.11-11")) { 
>> # enable pcid flag 
>> } 
>>
>> Greets 
>> Stefan 
>>
>>> Maybe add new cpumodel with +pcid enabled ? 
>>> or add code to manage custom cpuflags and add a checkbox in cpu options ? 
>>>
>>>
>>>
>>> - Mail original - 
>>> De: "Stefan Priebe, Profihost AG" <s.pri...@profihost.ag> 
>>> À: "pve-devel" <pve-devel@pve.proxmox.com> 
>>> Envoyé: Lundi 8 Janvier 2018 21:34:57 
>>> Objet: [pve-devel] cpuflag: pcid needed in guest for good performance after 
>>> meltdown 
>>>
>>> Hello, 
>>>
>>> for meltdown mitigation and performance it's important to have the pcid 
>>> flag passed down to the guest (f.e. 
>>> https://groups.google.com/forum/m/#!topic/mechanical-sympathy/L9mHTbeQLNU). 
>>>
>>> My host shows the flag: 
>>> # grep ' pcid ' /proc/cpuinfo | wc -l 
>>> 56 
>>>
>>> But the guest does not: 
>>> # grep pcid /proc/cpuinfo 
>>> # 
>>>
>>> Guest was started with: 
>>> -cpu IvyBridge,+kvm_pv_unhalt,+kvm_pv_eoi,enforce,vendor=GenuineIntel 
>>>
>>> Is this something missing in host kernel or in PVE? 
>>>
>>> Greets, 
>>> Stefan 
>>> ___ 
>>> pve-devel mailing list 
>>> pve-devel@pve.proxmox.com 
>>> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel 
>>>
>>> ___ 
>>> pve-devel mailing list 
>>> pve-devel@pve.proxmox.com 
>>> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel 
>>>
>>
> 
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] cpuflag: pcid needed in guest for good performance after meltdown

[Stefan Priebe - Profihost AG]

Am 09.01.2018 um 09:18 schrieb Alexandre DERUMIER:
> they are a discussion on qemu mailing currently about pcid 
> 
> 
> From paolo bonzini:
> 
> " 
> Note that PCID is still not supported for guests without EPT, so 
> this would break ept=0 with recent "-cpu" models. I'm not sure of 
> a way to fix it; probably it just has to be documented."

That's bad as pcid is very important to performance for meltdown fixes
in the linux kernel.

Stefan

> 
> - Mail original -
> De: "Stefan Priebe, Profihost AG" <s.pri...@profihost.ag>
> À: "pve-devel" <pve-devel@pve.proxmox.com>, "aderumier" <aderum...@odiso.com>
> Envoyé: Mardi 9 Janvier 2018 08:35:00
> Objet: Re: [pve-devel] cpuflag: pcid needed in guest for good performance 
> after meltdown
> 
> Am 08.01.2018 um 23:23 schrieb Alexandre DERUMIER: 
>> I think it's not exposed in current cpu model 
>>
>> it can be enabled with "+pcid" 
>>
>>
>> I don't known what it's the best way to add it in proxmox. 
>> We could do it with qemu version upgrade, but qemu 2.10 are not ready. 
> 
> Yes but that was always bad at least to me. We're limiting ourselfes if 
> we can only add new qemu features with a new qemu version. 
> 
> What about a version or feature field in the guest config which we only 
> update on a fresh vm start. 
> 
> Something like this: 
> 123.conf: 
> ... 
> pve_qemu_version: 2.11-10 
> ... 
> 
> This field is ONLY and ALWAYS updated in a fresh start not on migration. 
> This can be easily detected. 
> 
> Than we can do stuff like 
> if (version_cmp($conf->{pve_qemu_version}, "2.11-11")) { 
> # enable pcid flag 
> } 
> 
> Greets 
> Stefan 
> 
>> Maybe add new cpumodel with +pcid enabled ? 
>> or add code to manage custom cpuflags and add a checkbox in cpu options ? 
>>
>>
>>
>> - Mail original - 
>> De: "Stefan Priebe, Profihost AG" <s.pri...@profihost.ag> 
>> À: "pve-devel" <pve-devel@pve.proxmox.com> 
>> Envoyé: Lundi 8 Janvier 2018 21:34:57 
>> Objet: [pve-devel] cpuflag: pcid needed in guest for good performance after 
>> meltdown 
>>
>> Hello, 
>>
>> for meltdown mitigation and performance it's important to have the pcid 
>> flag passed down to the guest (f.e. 
>> https://groups.google.com/forum/m/#!topic/mechanical-sympathy/L9mHTbeQLNU). 
>>
>> My host shows the flag: 
>> # grep ' pcid ' /proc/cpuinfo | wc -l 
>> 56 
>>
>> But the guest does not: 
>> # grep pcid /proc/cpuinfo 
>> # 
>>
>> Guest was started with: 
>> -cpu IvyBridge,+kvm_pv_unhalt,+kvm_pv_eoi,enforce,vendor=GenuineIntel 
>>
>> Is this something missing in host kernel or in PVE? 
>>
>> Greets, 
>> Stefan 
>> ___ 
>> pve-devel mailing list 
>> pve-devel@pve.proxmox.com 
>> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel 
>>
>> ___ 
>> pve-devel mailing list 
>> pve-devel@pve.proxmox.com 
>> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel 
>>
> 
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] cpuflag: pcid needed in guest for good performance after meltdown

[Stefan Priebe - Profihost AG]

Am 09.01.2018 um 09:44 schrieb Alexandre DERUMIER:
>>> What about a version or feature field in the guest config which we only
>>> update on a fresh vm start.
>>>
>>> Something like this:
>>> 123.conf:
>>> ...
>>> pve_qemu_version: 2.11-10
>>> ...
>>>
>>> This field is ONLY and ALWAYS updated in a fresh start not on migration.
>>> This can be easily detected.
> 
> how does it solve the problem, if you update at start ? (if you take the 
> current qemu version, and update the conf file).

You can introduce new cpu flags or different qemu start parameters based
on qemu-server pkg version instead of kvm version.

Keeping old behaviour was never supported by pve at least i've not seen
that.

> we should need something external to qemu, like proxmox_config_version or 
> something like that.
> But I'm not sure it's easy to maintain. (maybe some user want to keep an old 
> qemu version for example, with last qemu-server code)
> 
> As alternative, we could patch qemu to have something like version : 2.11.x.y 
>  (where x is the minor version from qemu, and y minor version for proxmox)
> 
> 
> 
> - Mail original -
> De: "Stefan Priebe, Profihost AG" <s.pri...@profihost.ag>
> À: "pve-devel" <pve-devel@pve.proxmox.com>, "aderumier" <aderum...@odiso.com>
> Envoyé: Mardi 9 Janvier 2018 08:35:00
> Objet: Re: [pve-devel] cpuflag: pcid needed in guest for good performance 
> after meltdown
> 
> Am 08.01.2018 um 23:23 schrieb Alexandre DERUMIER: 
>> I think it's not exposed in current cpu model 
>>
>> it can be enabled with "+pcid" 
>>
>>
>> I don't known what it's the best way to add it in proxmox. 
>> We could do it with qemu version upgrade, but qemu 2.10 are not ready. 
> 
> Yes but that was always bad at least to me. We're limiting ourselfes if 
> we can only add new qemu features with a new qemu version. 
> 
> What about a version or feature field in the guest config which we only 
> update on a fresh vm start. 
> 
> Something like this: 
> 123.conf: 
> ... 
> pve_qemu_version: 2.11-10 
> ... 
> 
> This field is ONLY and ALWAYS updated in a fresh start not on migration. 
> This can be easily detected. 
> 
> Than we can do stuff like 
> if (version_cmp($conf->{pve_qemu_version}, "2.11-11")) { 
> # enable pcid flag 
> } 
> 
> Greets 
> Stefan 
> 
>> Maybe add new cpumodel with +pcid enabled ? 
>> or add code to manage custom cpuflags and add a checkbox in cpu options ? 
>>
>>
>>
>> - Mail original - 
>> De: "Stefan Priebe, Profihost AG" <s.pri...@profihost.ag> 
>> À: "pve-devel" <pve-devel@pve.proxmox.com> 
>> Envoyé: Lundi 8 Janvier 2018 21:34:57 
>> Objet: [pve-devel] cpuflag: pcid needed in guest for good performance after 
>> meltdown 
>>
>> Hello, 
>>
>> for meltdown mitigation and performance it's important to have the pcid 
>> flag passed down to the guest (f.e. 
>> https://groups.google.com/forum/m/#!topic/mechanical-sympathy/L9mHTbeQLNU). 
>>
>> My host shows the flag: 
>> # grep ' pcid ' /proc/cpuinfo | wc -l 
>> 56 
>>
>> But the guest does not: 
>> # grep pcid /proc/cpuinfo 
>> # 
>>
>> Guest was started with: 
>> -cpu IvyBridge,+kvm_pv_unhalt,+kvm_pv_eoi,enforce,vendor=GenuineIntel 
>>
>> Is this something missing in host kernel or in PVE? 
>>
>> Greets, 
>> Stefan 
>> ___ 
>> pve-devel mailing list 
>> pve-devel@pve.proxmox.com 
>> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel 
>>
>> ___ 
>> pve-devel mailing list 
>> pve-devel@pve.proxmox.com 
>> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel 
>>
> 
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] cpuflag: pcid needed in guest for good performance after meltdown

[Stefan Priebe - Profihost AG]

Am 08.01.2018 um 23:23 schrieb Alexandre DERUMIER:
> I think it's not exposed in current cpu model
> 
> it can be enabled with "+pcid"
> 
> 
> I don't known what it's the best way to add it in proxmox.
> We could do it with qemu version upgrade, but qemu 2.10 are not ready.

Yes but that was always bad at least to me. We're limiting ourselfes if
we can only add new qemu features with a new qemu version.

What about a version or feature field in the guest config which we only
update on a fresh vm start.

Something like this:
123.conf:
...
pve_qemu_version: 2.11-10
...

This field is ONLY and ALWAYS updated in a fresh start not on migration.
This can be easily detected.

Than we can do stuff like
if (version_cmp($conf->{pve_qemu_version}, "2.11-11")) {
   # enable pcid flag
}

Greets
Stefan

> Maybe add new cpumodel with +pcid enabled ?
> or add code to manage custom cpuflags and add a checkbox in cpu options ?
> 
> 
> 
> - Mail original -
> De: "Stefan Priebe, Profihost AG" <s.pri...@profihost.ag>
> À: "pve-devel" <pve-devel@pve.proxmox.com>
> Envoyé: Lundi 8 Janvier 2018 21:34:57
> Objet: [pve-devel] cpuflag: pcid needed in guest for good performance after 
> meltdown
> 
> Hello, 
> 
> for meltdown mitigation and performance it's important to have the pcid 
> flag passed down to the guest (f.e. 
> https://groups.google.com/forum/m/#!topic/mechanical-sympathy/L9mHTbeQLNU). 
> 
> My host shows the flag: 
> # grep ' pcid ' /proc/cpuinfo | wc -l 
> 56 
> 
> But the guest does not: 
> # grep pcid /proc/cpuinfo 
> # 
> 
> Guest was started with: 
> -cpu IvyBridge,+kvm_pv_unhalt,+kvm_pv_eoi,enforce,vendor=GenuineIntel 
> 
> Is this something missing in host kernel or in PVE? 
> 
> Greets, 
> Stefan 
> ___ 
> pve-devel mailing list 
> pve-devel@pve.proxmox.com 
> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel 
> 
> ___
> pve-devel mailing list
> pve-devel@pve.proxmox.com
> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
> 
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] cpuflag: pcid needed in guest for good performance after meltdown

[Stefan Priebe - Profihost AG]

Am 08.01.2018 um 22:11 schrieb Michael Rasmussen:
> On Mon, 8 Jan 2018 21:34:57 +0100
> Stefan Priebe - Profihost AG <s.pri...@profihost.ag> wrote:
> 
>> Hello,
>>
>> for meltdown mitigation and performance it's important to have the pcid
>> flag passed down to the guest (f.e.
>> https://groups.google.com/forum/m/#!topic/mechanical-sympathy/L9mHTbeQLNU).
>>
> As I understand it it is only required given the guest also uses a
> kernel that take advantage of this feature and which is, for Linux,
> a kernel 4.14 post Meltdown fix?

sure but I think this is also backported to LTS kernels - and also SuSE
backported this into all their releases from SLES 11, 12 and OpenSuSE.

> 
> 
> 
> ___
> pve-devel mailing list
> pve-devel@pve.proxmox.com
> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
> 
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] cpuflag: pcid needed in guest for good performance after meltdown

[Stefan Priebe - Profihost AG]

Hello,

for meltdown mitigation and performance it's important to have the pcid
flag passed down to the guest (f.e.
https://groups.google.com/forum/m/#!topic/mechanical-sympathy/L9mHTbeQLNU).

My host shows the flag:
# grep ' pcid ' /proc/cpuinfo  | wc -l
56

But the guest does not:
# grep pcid /proc/cpuinfo
#

Guest was started with:
-cpu IvyBridge,+kvm_pv_unhalt,+kvm_pv_eoi,enforce,vendor=GenuineIntel

Is this something missing in host kernel or in PVE?

Greets,
Stefan
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] Updated qemu pkg needed for Meltdown and Spectre?

[Stefan Priebe - Profihost AG]

Here we go - attached is the relevant patch - extracted from the
opensuse src.rpm.

Greets,
Stefan


Am 04.01.2018 um 19:37 schrieb Alexandre DERUMIER:
> seem that for spectre, cpumodel=qemu64|kvm64 is ok.
> 
> but not for the 2 others cve
> 
> On 04/01/2018 19:13, Alexandre DERUMIER wrote:
>> Thanks Paolo !
>>
>> Do we need to update guest kernel too, if qemu use cpumodel=qemu64 ?
>>
>> (For example,  I have some very old guests where kernel update is not 
>> possible)
> 
> If you want to be protected against the other two CVEs (one of which is
> "Meltdown"), yes.
> 
> Paolo
> 
> 
> - Mail original -
> De: "Stefan Priebe, Profihost AG" <s.pri...@profihost.ag>
> À: "pve-devel" <pve-devel@pve.proxmox.com>
> Envoyé: Jeudi 4 Janvier 2018 19:25:44
> Objet: Re: [pve-devel] Updated qemu pkg needed for Meltdown and Spectre?
> 
> Thanks! But that means we can update the kernel on the host which makes the 
> host and vm jumping safe BUT multi user guests are still vulnerable as long 
> as there are no qemu patches even if the guest has a current kernel. 
> 
> Greets, 
> Stefan 
> 
> Excuse my typo sent from my mobile phone. 
> 
>> Am 04.01.2018 um 19:09 schrieb Alexandre DERUMIER <aderum...@odiso.com>: 
>>
>> From Paolo bonzini on qemu-devel 
>>
>> -- 
>> _posts/ 2018-01-04 -spectre.md | 60 
>>  
>> 1 file changed, 60 insertions(+) 
>> create mode 100644 _posts/ 2018-01-04 -spectre.md 
>>
>> diff --git a/_posts/ 2018-01-04 -spectre.md b/_posts/ 2018-01-04 -spectre.md 
>> new file mode 100644 
>> index 000..1be86d0 
>> --- /dev/null 
>> +++ b/_posts/ 2018-01-04 -spectre.md 
>> @@ -0,0 +1,60 @@ 
>> +--- 
>> +layout: post 
>> +title: "QEMU and the Spectre and Meltdown attacks" 
>> +date: 2018-01-04 18:00:00 + 
>> +author: Paolo Bonzini and Eduardo Habkost 
>> +categories: [meltdown, spectre, security, x86] 
>> +--- 
>> +As you probably know by now, three critical architectural flaws in CPUs 
>> have 
>> +been recently disclosed that allow user processes to read kernel or 
>> hypervisor 
>> +memory through cache side-channel attacks. These flaws, collectively 
>> +named _Meltdown_ and _Spectre_, affect in one way or another almost 
>> +all processors that perform out-of-order execution, including x86 (from 
>> +Intel and AMD), POWER, s390 and ARM processors. 
>> + 
>> +No microcode updates are required to block the _Meltdown_ attack; it is 
>> +enough to update the guest operating system to a version that separates 
>> +the user and kernel address spaces (known as _page table isolation_ for 
>> +the Linux kernel). Therefore, this post will focus on _Spectre_, and 
>> +especially on [CVE-2017-5715]( [ 
>> https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2017-5715 | 
>> https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2017-5715 ] ). 
>> + 
>> +Fixing or mitigating _Spectre_ in general, and CVE-2017-5715 in particular, 
>> +requires cooperation between the processor and the operating system kernel 
>> or 
>> +hypervisor; the processor can be updated through microcode or millicode 
>> +patches to provide the required functionality. CVE-2017-5715 allows guests 
>> +to read potentially sensitive data from hypervisor memory; however, 
>> __patching 
>> +the host kernel is sufficient to block this attack__. 
>> + 
>> +On the other hand, in order to protect the guest kernel from a malicious 
>> +userspace, updates are also needed to the guest kernel and, depending on 
>> +the processor architecture, to QEMU. Just like on bare-metal, the guest 
>> +kernel will use the new functionality provided by the microcode or 
>> millicode 
>> +updates. When running under a hypervisor, processor emulation is mostly out 
>> of 
>> +QEMU's scope, so QEMU's role in the fix is small, but nevertheless 
>> important. 
>> +In the case of KVM: 
>> + 
>> +* QEMU configures the hypervisor to emulate a specific processor model. 
>> +For x86, QEMU has to be aware of new CPUID bits introduced by the microcode 
>> +update, and it must provide them to guests depending on how the guest is 
>> +configured. 
>> + 
>> +* upon virtual machine migration, QEMU reads the CPU state on the source 
>> +and transmits it to the destination. For x86, QEMU has to be aware of new 
>> +model specific registers (MSRs). 
>> + 
>> +Right now, there are no public patches to KVM that expose the new CPUID 
>> bit

Re: [pve-devel] Updated qemu pkg needed for Meltdown and Spectre?

[Stefan Priebe - Profihost AG]

2.14.3
> 
> Alexandre Derumier 
> Ingénieur système et stockage 
> 
> Manager Infrastructure 
> 
> 
> Fixe : +33 3 59 82 20 10 
> 
> 
> 
> 125 Avenue de la république 
> 59110 La Madeleine 
> [ https://twitter.com/OdisoHosting ] [ https://twitter.com/mindbaz ] [ 
> https://www.linkedin.com/company/odiso ] [ 
> https://www.viadeo.com/fr/company/odiso ] [ 
> https://www.facebook.com/monsiteestlent ] 
> 
> [ https://www.monsiteestlent.com/ | MonSiteEstLent.com ] - Blog dédié à la 
> webperformance et la gestion de pics de trafic
> 
> - Mail original -
> De: "Fabian Grünbichler" <f.gruenbich...@proxmox.com>
> À: "pve-devel" <pve-devel@pve.proxmox.com>
> Envoyé: Jeudi 4 Janvier 2018 09:50:04
> Objet: Re: [pve-devel] Updated qemu pkg needed for Meltdown and Spectre?
> 
>> On Thu, Jan 04, 2018 at 07:17:54AM +0100, Stefan Priebe - Profihost AG 
>> wrote: 
>> Hello, 
>> 
>> as far as i can see at least SuSE updated qemu for Meltdown and Spectre 
>> to provide CPUID information to the guest. 
>> 
>> I think we need to patch qemu as well asap? Has anybody found the 
>> relevant patches? 
>> 
>> https://www.pro-linux.de/sicherheit/2/41859/preisgabe-von-informationen-in-qemu.html
>>  
>> 
>> Greets, 
>> Stefan 
> 
> there seem to be no public (qemu) patches yet, once there are, we will 
> review and include them. 
> 
> ___ 
> pve-devel mailing list 
> pve-devel@pve.proxmox.com 
> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel 
> 
> ___
> pve-devel mailing list
> pve-devel@pve.proxmox.com
> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] Updated qemu pkg needed for Meltdown and Spectre?

[Stefan Priebe - Profihost AG]

Hello,

as far as i can see at least SuSE updated qemu for Meltdown and Spectre
to provide CPUID information to the guest.

I think we need to patch qemu as well asap? Has anybody found the
relevant patches?

https://www.pro-linux.de/sicherheit/2/41859/preisgabe-von-informationen-in-qemu.html

Greets,
Stefan
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] dropped pkts with Qemu on tap interace (RX)

[Stefan Priebe - Profihost AG]

Hello,

happy new year to all!

Currently i'm trying to fix a problem where we have "random" missing
packets.

We're doing an ssh connect from machine a to machine b every 5 minutes
via rsync and ssh.

Sometimes it happens that we get this cron message:
"Connection to 192.168.0.2 closed by remote host.
rsync: connection unexpectedly closed (0 bytes received so far) [sender]
rsync error: unexplained error (code 255) at io.c(226) [sender=3.1.2]
ssh: connect to host 192.168.0.2 port 22: Connection refused"

The tap devices on the target vm shows dropped RX packages on BOTH tap
interfaces - strangely with the same amount of pkts?

# ifconfig tap317i0; ifconfig tap317i1
tap317i0  Link encap:Ethernet  HWaddr 6e:cb:65:94:bb:bf
  UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1
  RX packets:2238445 errors:0 dropped:13159 overruns:0 frame:0
  TX packets:9655853 errors:0 dropped:0 overruns:0 carrier:0
  collisions:0 txqueuelen:1000
  RX bytes:177991267 (169.7 MiB)  TX bytes:910412749 (868.2 MiB)

tap317i1  Link encap:Ethernet  HWaddr 96:f8:b5:d0:9a:07
  UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1
  RX packets:1516085 errors:0 dropped:13159 overruns:0 frame:0
  TX packets:1446964 errors:0 dropped:0 overruns:0 carrier:0
  collisions:0 txqueuelen:1000
  RX bytes:1597564313 (1.4 GiB)  TX bytes:3517734365 (3.2 GiB)

Any ideas how to inspect this issue?

Greets,
Stefan
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] pve api offline during log rotation

[Stefan Priebe - Profihost AG]

> Am 09.11.2017 um 15:03 schrieb Thomas Lamprecht <t.lampre...@proxmox.com>:
> 
>> On 11/09/2017 02:09 PM, Caspar Smit wrote:
>> If i look into the /etc/init.d/pveproxy script you will see that a
>> 'restart' (and reload for that matter) does a: $DAEMON restart
>> 
>> Where $DAEMON=/usr/bin/pveproxy
>> 
>> So basically a "/etc/init.d/pveproxy restart (or reload)" should do a
>> "/usr/bin/pveproxy restart"
>> 
> 
> Yes, true, thanks for the correction.
> 
>> Or is the init.d script skipped because the systemd wrapper picks it up as
>> "systemctl restart pveproxy" ? (it looks that way:)
>> 
> 
> Yeah, that's exactly the case, services files have always precedence and if
> there is none, systemd auto generates one and passes the requested operation
> (start, stop, restart) to this one, which changed the reload to a restart in
> this case. So I'm swapped it out with reload in the patch I sent.
> 
> cheers, Thomas

Works perfectly and applies cleanly to stable-4 branch.

Thanks!

Greets,
Stefan
> 
>> # /etc/init.d/pveproxy restart
>> [ ok ] Restarting pveproxy (via systemctl): pveproxy.service.
>> 
>> Kind regards,
>> Caspar
>> 
>> 2017-11-09 13:47 GMT+01:00 Thomas Lamprecht <t.lampre...@proxmox.com>:
>> 
>>>> On 11/09/2017 01:40 PM, Stefan Priebe - Profihost AG wrote:
>>>> *arg* sorry about that and thanks for resending your last paragraph. Yes
>>>> that's exactly the point.
>>>> 
>>>> Also thanks for the restart and systemctl explanation.
>>>> 
>>> 
>>> I have also have to apologize me, we're both (partly) right:
>>> log rotate configuration does a restart on the old init.d scripts not the
>>> pveproxy executable itself (should got rid of those init.d scripts already)
>>> which results really in a full stop/start cycle (as systemctl restart would
>>> have been).
>>> So yeah, it currently does the "wrong" restart, but even if it would do the
>>> "correct" one the problem is still there as of aforementioned reason with
>>> the graceful restart. So both have to get fixed, I look into it, thanks for
>>> the report and your patience with this.
>>> 
>>> cheers, Thomas
>>> 
>>> ___
>>> pve-devel mailing list
>>> pve-devel@pve.proxmox.com
>>> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
>>> 
>> ___
>> pve-devel mailing list
>> pve-devel@pve.proxmox.com
>> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
>> 
> 
> 
> ___
> pve-devel mailing list
> pve-devel@pve.proxmox.com
> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] pve api offline during log rotation

[Stefan Priebe - Profihost AG]

Sorry for the late reply. Is there a chance to get the fix backported to 4.4?

Greets,
Stefan

Excuse my typo sent from my mobile phone.

> Am 09.11.2017 um 13:40 schrieb Stefan Priebe - Profihost AG 
> <s.pri...@profihost.ag>:
> 
> *arg* sorry about that and thanks for resending your last paragraph. Yes
> that's exactly the point.
> 
> Also thanks for the restart and systemctl explanation.
> 
> Greets,
> Stefan
> 
>> Am 09.11.2017 um 13:35 schrieb Thomas Lamprecht:
>> Hi,
>> 
>>> On 11/09/2017 01:08 PM, Stefan Priebe - Profihost AG wrote:
>>> yes that's what i'm talking about. The logfile rotation script DOES a
>>> restart not a reload.
>>> 
>> 
>> No it doesn't do a systemd restart, it's a bit confusing - I know.
>> 
>>> See here:
>>> https://git.proxmox.com/?p=pve-manager.git;a=blob_plain;f=debian/pve.logrotate;hb=HEAD
>>> 
>> 
>> It does a `pveproxy restart`, which _is_ a real graceful "fast" restart,
>> not to be confused with `systemctl restart pveproxy` which does a full
>> stop first, and then a full new startup again.
>> 
>> `systemctl reload pveproxy` does the exact same as the logrotation, see:
>> 
>> https://git.proxmox.com/?p=pve-manager.git;a=blob_plain;f=bin/init.d/pveproxy.service
>> 
>> Thus the problem is not this but the one I described in the last paragraph
>> from my last answer:
>> 
>>> On 11/09/2017 11:03 AM, Thomas Lamprecht wrote:
>>> 
>>> We do a re-exec on "ourself" (from the daemons POV), and the intend is to
>>> leave non-idle child workers untouched, but the logic doing this is a bit
>>> flawed as all current worker child always receive a TERM signal.
>>> Here the HTTP server worker wait at least for active connection to end,
>>> but new ones do not get accepted. We directly restart after that, but yes,
>>> depending on load there can be a time window where no one is there to
>>> accept connections.
>>> I'd rather not send the TERM signal in the case where the
>>> "leave_children_open_on_reload" option is set and we're restarting but
>>> just restart, passing the current worker PIDs over to our new self
>>> (this gets already done). There on startup then start new workers and
>>> message the old ones to not  accept new connections and terminate
>>> gracefully as soon as possible. Now there is never a time where no active
>>> listening worker would there. I try to give it a look.
>>> 
>> 
>> cheers,
>> Thomas
>> 
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] pve api offline during log rotation

[Stefan Priebe - Profihost AG]

*arg* sorry about that and thanks for resending your last paragraph. Yes
that's exactly the point.

Also thanks for the restart and systemctl explanation.

Greets,
Stefan

Am 09.11.2017 um 13:35 schrieb Thomas Lamprecht:
> Hi,
> 
> On 11/09/2017 01:08 PM, Stefan Priebe - Profihost AG wrote:
>> yes that's what i'm talking about. The logfile rotation script DOES a
>> restart not a reload.
>>
> 
> No it doesn't do a systemd restart, it's a bit confusing - I know.
> 
>> See here:
>> https://git.proxmox.com/?p=pve-manager.git;a=blob_plain;f=debian/pve.logrotate;hb=HEAD
>>
> 
> It does a `pveproxy restart`, which _is_ a real graceful "fast" restart,
> not to be confused with `systemctl restart pveproxy` which does a full
> stop first, and then a full new startup again.
> 
> `systemctl reload pveproxy` does the exact same as the logrotation, see:
> 
> https://git.proxmox.com/?p=pve-manager.git;a=blob_plain;f=bin/init.d/pveproxy.service
> 
> Thus the problem is not this but the one I described in the last paragraph
> from my last answer:
> 
> On 11/09/2017 11:03 AM, Thomas Lamprecht wrote:
>>
>> We do a re-exec on "ourself" (from the daemons POV), and the intend is to
>> leave non-idle child workers untouched, but the logic doing this is a bit
>> flawed as all current worker child always receive a TERM signal.
>> Here the HTTP server worker wait at least for active connection to end,
>> but new ones do not get accepted. We directly restart after that, but yes,
>> depending on load there can be a time window where no one is there to
>> accept connections.
>> I'd rather not send the TERM signal in the case where the
>> "leave_children_open_on_reload" option is set and we're restarting but
>> just restart, passing the current worker PIDs over to our new self
>> (this gets already done). There on startup then start new workers and
>> message the old ones to not  accept new connections and terminate
>> gracefully as soon as possible. Now there is never a time where no active
>> listening worker would there. I try to give it a look.
>>
> 
> cheers,
> Thomas
> 
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] pve api offline during log rotation

[Stefan Priebe - Profihost AG]

Hello,

yes that's what i'm talking about. The logfile rotation script DOES a
restart not a reload.

See here:
https://git.proxmox.com/?p=pve-manager.git;a=blob_plain;f=debian/pve.logrotate;hb=HEAD

Greets,
Stefan
Am 09.11.2017 um 11:03 schrieb Thomas Lamprecht:
> Hi,
> 
> On 09/27/2017 08:21 AM, Stefan Priebe - Profihost AG wrote:
>> Am 21.09.2017 um 15:30 schrieb Thomas Lamprecht:
>>> On 09/20/2017 01:26 PM, Stefan Priebe - Profihost AG wrote:
>>>>> [snip]
>>>> thanks for the reply. Does this also apply to PVE 4? Sorry i missed that
>>>> info.
>>>>
>>>
>>> Yes. But it should happen more often with PVE 5.
>>> Affecting packages had the fix already backported and applied in git.
>>> It may need a little time until all get released to all debian repos,
>>> though.
>>
>> sorry for the late reply. I updated all pkgs to latest git stable-4 branch.
>>
>> But it still happens. On 06:25 i get a lot of:
>> 595 Connection refused
>> or
>> 500 Can't connect to XX.de:8006
>>
>> messages.
>>
>> The journal looks like this:
>> Sep 27 06:25:03 systemd[1]: Stopping PVE API Proxy Server...
>> Sep 27 06:25:04 pveproxy[47485]: received signal TERM
> 
> This is a `systemctl restart pveproxy` not a `systemctl reload pveproxy`
> The first does a full shutdown and then full startup cycle (with new PID)
> the latter does not a complete full shutdown but a rexec on itself.
> 
> On an update of pve-manager reload is used, so I get:
> 
> Nov 09 10:08:48 dev5 systemd[1]: Reloading PVE API Proxy Server.
> Nov 09 10:08:49 dev5 pvedaemon[1761]: restarting server
> Nov 09 10:08:49 dev5 pvedaemon[1761]: worker 11849 finished
> Nov 09 10:08:49 dev5 pvedaemon[1761]: worker 11848 finished
> Nov 09 10:08:49 dev5 pvedaemon[1761]: worker 11847 finished
> Nov 09 10:08:49 dev5 pvedaemon[1761]: starting 3 worker(s)
> Nov 09 10:08:49 dev5 pvedaemon[1761]: worker 19464 started
> Nov 09 10:08:49 dev5 pvedaemon[1761]: worker 19465 started
> Nov 09 10:08:49 dev5 pvedaemon[1761]: worker 19466 started
> Nov 09 10:08:49 dev5 pveproxy[19458]: send HUP to 14616
> Nov 09 10:08:49 dev5 pveproxy[14616]: received signal HUP
> Nov 09 10:08:49 dev5 pveproxy[14616]: server closing
> Nov 09 10:08:49 dev5 pveproxy[14616]: server shutdown (restart)
> Nov 09 10:08:49 dev5 pveproxy[15962]: worker exit
> Nov 09 10:08:49 dev5 pveproxy[15963]: worker exit
> Nov 09 10:08:49 dev5 pveproxy[15961]: worker exit
> Nov 09 10:08:49 dev5 systemd[1]: Reloaded PVE API Proxy Server.
> 
>> Sep 27 06:25:04 pveproxy[47485]: server closing
>> Sep 27 06:25:04 pveproxy[25445]: worker exit
>> Sep 27 06:25:04 pveproxy[24339]: worker exit
>> Sep 27 06:25:04 pveproxy[16705]: worker exit
>> Sep 27 06:25:04 pveproxy[52966]: worker exit
>> Sep 27 06:25:04 pveproxy[16550]: worker exit
>> Sep 27 06:25:04 pveproxy[18493]: worker exit
>> Sep 27 06:25:04 pveproxy[3140]: worker exit
>> Sep 27 06:25:04 pveproxy[12704]: worker exit
>> Sep 27 06:25:04 pveproxy[13839]: worker exit
>> Sep 27 06:25:04 pveproxy[47485]: worker 16705 finished
>> Sep 27 06:25:04 pveproxy[47485]: worker 13839 finished
>> Sep 27 06:25:04 pveproxy[47485]: worker 12704 finished
>> Sep 27 06:25:04 pveproxy[47485]: worker 17652 finished
>> Sep 27 06:25:04 pveproxy[47485]: worker 3140 finished
>> Sep 27 06:25:04 pveproxy[47485]: worker 52966 finished
>> Sep 27 06:25:04 pveproxy[47485]: worker 24339 finished
>> Sep 27 06:25:04 pveproxy[47485]: worker 25445 finished
>> Sep 27 06:25:04 pveproxy[47485]: worker 18493 finished
>> Sep 27 06:25:04 pveproxy[47485]: worker 16550 finished
>> Sep 27 06:25:04 pveproxy[47485]: server stopped
>> Sep 27 06:25:05 systemd[1]: Starting PVE API Proxy Server...
>> Sep 27 06:25:06 pveproxy[28612]: Using '/etc/pve/local/pveproxy-ssl.pem'
>> as certificate for the web interface.
>> Sep 27 06:25:06 pveproxy[28617]: starting server
>> Sep 27 06:25:06 pveproxy[28617]: starting 10 worker(s)
>> Sep 27 06:25:06 pveproxy[28617]: worker 28618 started
>> Sep 27 06:25:06 pveproxy[28617]: worker 28619 started
>> Sep 27 06:25:06 pveproxy[28617]: worker 28620 started
>> Sep 27 06:25:06 pveproxy[28617]: worker 28621 started
>> Sep 27 06:25:06 pveproxy[28617]: worker 28622 started
>> Sep 27 06:25:06 pveproxy[28617]: worker 28623 started
>> Sep 27 06:25:06 pveproxy[28617]: worker 28624 started
>> Sep 27 06:25:06 pveproxy[28617]: worker 28625 started
>> Sep 27 06:25:06 pveproxy[28617]: worker 28626 started
>> Sep 27 06:25:06 systemd[1]: Started PVE API Proxy Server.
>> Sep 27 06:25:06 pveproxy[28617]: worker 28628 started
>>

Re: [pve-devel] pve api offline during log rotation

[Stefan Priebe - Profihost AG]

Hello,

any news on this? Is this expected?

Thanks,
Stefan

Am 27.09.2017 um 08:21 schrieb Stefan Priebe - Profihost AG:
> Hi,
> 
> Am 21.09.2017 um 15:30 schrieb Thomas Lamprecht:
>> On 09/20/2017 01:26 PM, Stefan Priebe - Profihost AG wrote:
>>> Hi,
>>>
>>>
>>> Am 20.09.2017 um 10:36 schrieb Thomas Lamprecht:
>>>> On 09/20/2017 06:40 AM, Stefan Priebe - Profihost AG wrote:
>>>>> Nobody?
>>>>>
>>>>
>>>> We register the restart command from pveproxy with the $use_hup
>>>> parameter,
>>>> this then send a SIGHUP when calling pveproxy restart - which gets
>>>> mapped to
>>>> systemctl reload-or-restart pveproxy.service, which sends the HUP
>>>> signal to
>>>> the main process, which in turn does an exec on itself. So  a restart
>>>> is a
>>>> reload in this case.
>>>>
>>>> As we already use use our Daemons class' "leave_children_open_on_reload"
>>>> functionality to keep the workers running during a restart open
>>>> connection
>>>> should stay. But as the worker always get  a SIGTERM even with this
>>>> option
>>>> this is not always the case, I will investigate this behavior but this
>>>> is not
>>>> responsible for longer restart times.
>>>>
>>>> Is suspect that you suffered from a bug we fixed about a week ago [1]
>>>> where
>>>> worker signal handlers got overwritten and thus daemons could always be
>>>> stopped grafully. restarting may have been affected to by it.
>>>>
>>>> The fix is currently already in the no-subscription repo.
>>>>
>>>> If you still experience this behavior could you please do a
>>>> `pveproxy restart` and post the logs from during the restart,
>>>> something like:
>>>> # journalctl -u pveproxy.service --since -10min
>>>
>>>
>>> thanks for the reply. Does this also apply to PVE 4? Sorry i missed that
>>> info.
>>>
>>
>> Yes. But it should happen more often with PVE 5.
>> Affecting packages had the fix already backported and applied in git.
>> It may need a little time until all get released to all debian repos,
>> though.
> 
> sorry for the late reply. I updated all pkgs to latest git stable-4 branch.
> 
> But it still happens. On 06:25 i get a lot of:
> 595 Connection refused
> or
> 500 Can't connect to XX.de:8006
> 
> messages.
> 
> The journal looks like this:
> Sep 27 06:25:03 systemd[1]: Stopping PVE API Proxy Server...
> Sep 27 06:25:04 pveproxy[47485]: received signal TERM
> Sep 27 06:25:04 pveproxy[47485]: server closing
> Sep 27 06:25:04 pveproxy[25445]: worker exit
> Sep 27 06:25:04 pveproxy[24339]: worker exit
> Sep 27 06:25:04 pveproxy[16705]: worker exit
> Sep 27 06:25:04 pveproxy[52966]: worker exit
> Sep 27 06:25:04 pveproxy[16550]: worker exit
> Sep 27 06:25:04 pveproxy[18493]: worker exit
> Sep 27 06:25:04 pveproxy[3140]: worker exit
> Sep 27 06:25:04 pveproxy[12704]: worker exit
> Sep 27 06:25:04 pveproxy[13839]: worker exit
> Sep 27 06:25:04 pveproxy[47485]: worker 16705 finished
> Sep 27 06:25:04 pveproxy[47485]: worker 13839 finished
> Sep 27 06:25:04 pveproxy[47485]: worker 12704 finished
> Sep 27 06:25:04 pveproxy[47485]: worker 17652 finished
> Sep 27 06:25:04 pveproxy[47485]: worker 3140 finished
> Sep 27 06:25:04 pveproxy[47485]: worker 52966 finished
> Sep 27 06:25:04 pveproxy[47485]: worker 24339 finished
> Sep 27 06:25:04 pveproxy[47485]: worker 25445 finished
> Sep 27 06:25:04 pveproxy[47485]: worker 18493 finished
> Sep 27 06:25:04 pveproxy[47485]: worker 16550 finished
> Sep 27 06:25:04 pveproxy[47485]: server stopped
> Sep 27 06:25:05 systemd[1]: Starting PVE API Proxy Server...
> Sep 27 06:25:06 pveproxy[28612]: Using '/etc/pve/local/pveproxy-ssl.pem'
> as certificate for the web interface.
> Sep 27 06:25:06 pveproxy[28617]: starting server
> Sep 27 06:25:06 pveproxy[28617]: starting 10 worker(s)
> Sep 27 06:25:06 pveproxy[28617]: worker 28618 started
> Sep 27 06:25:06 pveproxy[28617]: worker 28619 started
> Sep 27 06:25:06 pveproxy[28617]: worker 28620 started
> Sep 27 06:25:06 pveproxy[28617]: worker 28621 started
> Sep 27 06:25:06 pveproxy[28617]: worker 28622 started
> Sep 27 06:25:06 pveproxy[28617]: worker 28623 started
> Sep 27 06:25:06 pveproxy[28617]: worker 28624 started
> Sep 27 06:25:06 pveproxy[28617]: worker 28625 started
> Sep 27 06:25:06 pveproxy[28617]: worker 28626 started
> Sep 27 06:25:06 systemd[1]: Started PVE API Proxy Server.
> Sep 27 06:25:06 pveproxy[28617]: worker 28628 started
> 
> The problem still is that pveproxy is stopped and started again. So
> there is a gap where no new connections get accepted.
> 
> Normal behaviour for other deamons i know is to use a special SIGNAL to
> just reopen the logs and do not stop and start the daemon.
> 
> Greets,
> Stefan
> 
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] pve api offline during log rotation

[Stefan Priebe - Profihost AG]

Hi,

Am 21.09.2017 um 15:30 schrieb Thomas Lamprecht:
> On 09/20/2017 01:26 PM, Stefan Priebe - Profihost AG wrote:
>> Hi,
>>
>>
>> Am 20.09.2017 um 10:36 schrieb Thomas Lamprecht:
>>> On 09/20/2017 06:40 AM, Stefan Priebe - Profihost AG wrote:
>>>> Nobody?
>>>>
>>>
>>> We register the restart command from pveproxy with the $use_hup
>>> parameter,
>>> this then send a SIGHUP when calling pveproxy restart - which gets
>>> mapped to
>>> systemctl reload-or-restart pveproxy.service, which sends the HUP
>>> signal to
>>> the main process, which in turn does an exec on itself. So  a restart
>>> is a
>>> reload in this case.
>>>
>>> As we already use use our Daemons class' "leave_children_open_on_reload"
>>> functionality to keep the workers running during a restart open
>>> connection
>>> should stay. But as the worker always get  a SIGTERM even with this
>>> option
>>> this is not always the case, I will investigate this behavior but this
>>> is not
>>> responsible for longer restart times.
>>>
>>> Is suspect that you suffered from a bug we fixed about a week ago [1]
>>> where
>>> worker signal handlers got overwritten and thus daemons could always be
>>> stopped grafully. restarting may have been affected to by it.
>>>
>>> The fix is currently already in the no-subscription repo.
>>>
>>> If you still experience this behavior could you please do a
>>> `pveproxy restart` and post the logs from during the restart,
>>> something like:
>>> # journalctl -u pveproxy.service --since -10min
>>
>>
>> thanks for the reply. Does this also apply to PVE 4? Sorry i missed that
>> info.
>>
> 
> Yes. But it should happen more often with PVE 5.
> Affecting packages had the fix already backported and applied in git.
> It may need a little time until all get released to all debian repos,
> though.

sorry for the late reply. I updated all pkgs to latest git stable-4 branch.

But it still happens. On 06:25 i get a lot of:
595 Connection refused
or
500 Can't connect to XX.de:8006

messages.

The journal looks like this:
Sep 27 06:25:03 systemd[1]: Stopping PVE API Proxy Server...
Sep 27 06:25:04 pveproxy[47485]: received signal TERM
Sep 27 06:25:04 pveproxy[47485]: server closing
Sep 27 06:25:04 pveproxy[25445]: worker exit
Sep 27 06:25:04 pveproxy[24339]: worker exit
Sep 27 06:25:04 pveproxy[16705]: worker exit
Sep 27 06:25:04 pveproxy[52966]: worker exit
Sep 27 06:25:04 pveproxy[16550]: worker exit
Sep 27 06:25:04 pveproxy[18493]: worker exit
Sep 27 06:25:04 pveproxy[3140]: worker exit
Sep 27 06:25:04 pveproxy[12704]: worker exit
Sep 27 06:25:04 pveproxy[13839]: worker exit
Sep 27 06:25:04 pveproxy[47485]: worker 16705 finished
Sep 27 06:25:04 pveproxy[47485]: worker 13839 finished
Sep 27 06:25:04 pveproxy[47485]: worker 12704 finished
Sep 27 06:25:04 pveproxy[47485]: worker 17652 finished
Sep 27 06:25:04 pveproxy[47485]: worker 3140 finished
Sep 27 06:25:04 pveproxy[47485]: worker 52966 finished
Sep 27 06:25:04 pveproxy[47485]: worker 24339 finished
Sep 27 06:25:04 pveproxy[47485]: worker 25445 finished
Sep 27 06:25:04 pveproxy[47485]: worker 18493 finished
Sep 27 06:25:04 pveproxy[47485]: worker 16550 finished
Sep 27 06:25:04 pveproxy[47485]: server stopped
Sep 27 06:25:05 systemd[1]: Starting PVE API Proxy Server...
Sep 27 06:25:06 pveproxy[28612]: Using '/etc/pve/local/pveproxy-ssl.pem'
as certificate for the web interface.
Sep 27 06:25:06 pveproxy[28617]: starting server
Sep 27 06:25:06 pveproxy[28617]: starting 10 worker(s)
Sep 27 06:25:06 pveproxy[28617]: worker 28618 started
Sep 27 06:25:06 pveproxy[28617]: worker 28619 started
Sep 27 06:25:06 pveproxy[28617]: worker 28620 started
Sep 27 06:25:06 pveproxy[28617]: worker 28621 started
Sep 27 06:25:06 pveproxy[28617]: worker 28622 started
Sep 27 06:25:06 pveproxy[28617]: worker 28623 started
Sep 27 06:25:06 pveproxy[28617]: worker 28624 started
Sep 27 06:25:06 pveproxy[28617]: worker 28625 started
Sep 27 06:25:06 pveproxy[28617]: worker 28626 started
Sep 27 06:25:06 systemd[1]: Started PVE API Proxy Server.
Sep 27 06:25:06 pveproxy[28617]: worker 28628 started

The problem still is that pveproxy is stopped and started again. So
there is a gap where no new connections get accepted.

Normal behaviour for other deamons i know is to use a special SIGNAL to
just reopen the logs and do not stop and start the daemon.

Greets,
Stefan
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] pve api offline during log rotation

[Stefan Priebe - Profihost AG]

Hi,


Am 20.09.2017 um 10:36 schrieb Thomas Lamprecht:
> On 09/20/2017 06:40 AM, Stefan Priebe - Profihost AG wrote:
>> Nobody?
>>
> 
> We register the restart command from pveproxy with the $use_hup parameter,
> this then send a SIGHUP when calling pveproxy restart - which gets
> mapped to
> systemctl reload-or-restart pveproxy.service, which sends the HUP signal to
> the main process, which in turn does an exec on itself. So  a restart is a
> reload in this case.
> 
> As we already use use our Daemons class' "leave_children_open_on_reload"
> functionality to keep the workers running during a restart open connection
> should stay. But as the worker always get  a SIGTERM even with this option
> this is not always the case, I will investigate this behavior but this
> is not
> responsible for longer restart times.
> 
> Is suspect that you suffered from a bug we fixed about a week ago [1] where
> worker signal handlers got overwritten and thus daemons could always be
> stopped grafully. restarting may have been affected to by it.
> 
> The fix is currently already in the no-subscription repo.
> 
> If you still experience this behavior could you please do a
> `pveproxy restart` and post the logs from during the restart,
> something like:
> # journalctl -u pveproxy.service --since -10min


thanks for the reply. Does this also apply to PVE 4? Sorry i missed that
info.

Greets,
Stefan

> 
> 
> [1]
> https://git.proxmox.com/?p=pve-common.git;a=commit;h=eead1ccaa509e8559e466ca5926c6625f27bff35
> 
> 
>> Stefan
>>
>> Excuse my typo sent from my mobile phone.
>>
>>> Am 12.09.2017 um 09:27 schrieb Stefan Priebe - Profihost AG
>>> <s.pri...@profihost.ag>:
>>>
>>> Hello,
>>>
>>> pveproxy has already a reload command - which seems to reopen the logs
>>> correctly. Is there any reason why restart is used in the postrotate
>>> part of logrotate?
>>>
>>> Greets,
>>> Stefan
>>>
>>>> Am 12.09.2017 um 09:13 schrieb Stefan Priebe - Profihost AG:
>>>> Hello,
>>>>
>>>> we're heavily using the pve api - doing a lot of calls every few
>>>> minutes.
>>>>
>>>> Currently the log rotation does a pveproxy restart which makes the API
>>>> unavailable for a few seconds. This is pretty bad.
>>>>
>>>> I see the following solution:
>>>> - add a special signal like SIGUSR1 to pveproxy and spiceproxy to just
>>>> reopen the logs
>>>>
>>>> Greets,
>>>> Stefan
>>>>
>> ___
>> pve-devel mailing list
>> pve-devel@pve.proxmox.com
>> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
>>
> 
> 
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] pve api offline during log rotation

[Stefan Priebe - Profihost AG]

Nobody?

Stefan

Excuse my typo sent from my mobile phone.

> Am 12.09.2017 um 09:27 schrieb Stefan Priebe - Profihost AG 
> <s.pri...@profihost.ag>:
> 
> Hello,
> 
> pveproxy has already a reload command - which seems to reopen the logs
> correctly. Is there any reason why restart is used in the postrotate
> part of logrotate?
> 
> Greets,
> Stefan
> 
>> Am 12.09.2017 um 09:13 schrieb Stefan Priebe - Profihost AG:
>> Hello,
>> 
>> we're heavily using the pve api - doing a lot of calls every few minutes.
>> 
>> Currently the log rotation does a pveproxy restart which makes the API
>> unavailable for a few seconds. This is pretty bad.
>> 
>> I see the following solution:
>> - add a special signal like SIGUSR1 to pveproxy and spiceproxy to just
>> reopen the logs
>> 
>> Greets,
>> Stefan
>> 
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] pve api offline during log rotation

[Stefan Priebe - Profihost AG]

Hello,

pveproxy has already a reload command - which seems to reopen the logs
correctly. Is there any reason why restart is used in the postrotate
part of logrotate?

Greets,
Stefan

Am 12.09.2017 um 09:13 schrieb Stefan Priebe - Profihost AG:
> Hello,
> 
> we're heavily using the pve api - doing a lot of calls every few minutes.
> 
> Currently the log rotation does a pveproxy restart which makes the API
> unavailable for a few seconds. This is pretty bad.
> 
> I see the following solution:
> - add a special signal like SIGUSR1 to pveproxy and spiceproxy to just
> reopen the logs
> 
> Greets,
> Stefan
> 
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] pve api offline during log rotation

[Stefan Priebe - Profihost AG]

Hello,

we're heavily using the pve api - doing a lot of calls every few minutes.

Currently the log rotation does a pveproxy restart which makes the API
unavailable for a few seconds. This is pretty bad.

I see the following solution:
- add a special signal like SIGUSR1 to pveproxy and spiceproxy to just
reopen the logs

Greets,
Stefan
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] lost tcp packets in bridge

[Stefan Priebe - Profihost AG]

Am 22.08.2017 um 08:16 schrieb Alexandre DERUMIER:
> Hi Stefan,
> 
> No I don't have this behaviour. I only remember probleme with old debian 
> wheezy with kernel 3.2,
> where I had this kind of problem. (seem to be a problem in virtio-net driver 
> in 3.2)
> 
> 
> Do you known exactly where the packet is lost ? (do you have used tcpdump on 
> tap interface too ?)
> 
> mtu size is enough big on bridge && taps vs mtu in your vms ?

currently i can't reproduce the issue - even i was for two days.
Currently i've no idea what has happend and will report back if it
happens again.

Thanks!

Greets,
Stefan


> 
> 
> - Mail original -
> De: "Stefan Priebe, Profihost AG" <s.pri...@profihost.ag>
> À: "pve-devel" <pve-devel@pve.proxmox.com>
> Envoyé: Mardi 22 Août 2017 08:04:18
> Objet: [pve-devel] lost tcp packets in bridge
> 
> Hello, 
> 
> while running PVE 4.4 i'm observing lost tcp packets on a new cluster. 
> Two VMs are connected to the same bridge on the same host using virtio 
> and linux as a guest. 
> 
> But the target guest does not receive all tcp packets some get lost. I 
> see them leaving on the source using tcpdump but they do not arrive at 
> the target network card (checked using tcp dump). This results in 
> sometimes awfully slow performance as tcp retransmits happen. 
> 
> I already flushed all iptables rules - but this did not help. I can't 
> reproduce the issue it just happens "sometimes" with some packets. 
> 
> Has anybody already ovserved something like this? 
> 
> Greets, 
> Stefan 
> ___ 
> pve-devel mailing list 
> pve-devel@pve.proxmox.com 
> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel 
> 
> ___
> pve-devel mailing list
> pve-devel@pve.proxmox.com
> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
> 
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] lost tcp packets in bridge

[Stefan Priebe - Profihost AG]

Hello,

while running PVE 4.4 i'm observing lost tcp packets on a new cluster.
Two VMs are connected to the same bridge on the same host using virtio
and linux as a guest.

But the target guest does not receive all tcp packets some get lost. I
see them leaving on the source using tcpdump but they do not arrive at
the target network card (checked using tcp dump). This results in
sometimes awfully slow performance as tcp retransmits happen.

I already flushed all iptables rules - but this did not help. I can't
reproduce the issue it just happens "sometimes" with some packets.

Has anybody already ovserved something like this?

Greets,
Stefan
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [RFC qemu-server stable-4] add workaround for pve 4.4 to 5.0 live migration

[Stefan Priebe - Profihost AG]

Hello,

Am 11.07.2017 um 16:40 schrieb Thomas Lamprecht:
> commit 85909c04c49879f5fffa366fc3233eee2b157e97 switched from cirrus
> to vga for non windows OSs.
> 
> This adds an artificial blocker on live migrations from PVE 4.X to
> PVE 5.0.
> Address it in PVE 4.4 by explicitly setting cirrus in the config if
> it would be used, so that a PVE 5.0 starts up the correct hardware in
> the case of a migration. Do it only on running VMs with online
> migration enabled.
> Do not clean it up, as this would then make live migration fail again
> until the VM had a real poweroff and power on cycle even from PVE 5.0
> to PVE 5.0.
> While not the nicest way it is a short and valid solution and doesn't
> adds hackery in new code. We may also touch the config only on the
> source site until migration phase 3 completes.

this is a pretty hacky approach i don't like.

What about doing something different which is a more general approach an
can be used for pve 6, 7 as well:
1.) write at each vm start pve-manager version to a field in the
 vm conf f.e. pveversion: 4.4

2.) while doing an online migration check if there are special things to
consider f. e. default vga = cirrus if !defined($conf->{vga} &&
$conf->{pveversion} < 5;

Greets,
Stefan

  $
> Signed-off-by: Thomas Lamprecht 
> ---
> 
> Another approach would be to check the machine type on the target side and if
> older than *-2.9 use cirrus, if not explicitly set already.
> But this would then add code on PVE 5.0 which we would need to maintain for a
> bit of time, so I'm not sure if that would be the nicer solution...
>  
>  PVE/QemuMigrate.pm | 8 
>  1 file changed, 8 insertions(+)
> 
> diff --git a/PVE/QemuMigrate.pm b/PVE/QemuMigrate.pm
> index e6f147e..6cfca0b 100644
> --- a/PVE/QemuMigrate.pm
> +++ b/PVE/QemuMigrate.pm
> @@ -429,6 +429,14 @@ sub phase1 {
>  
>  my $conf = $self->{vmconf};
>  
> +# HACK: avoid migration failure to 5.0 as the VGA default has changed
> +if (!defined($conf->{vga}) && $self->{running} && 
> $self->{opts}->{online}) {
> + my $winversion = PVE::QemuServer::windows_version($conf->{ostype});
> + if ($winversion < 6) {
> + $conf->{vga} = 'cirrus';
> + }
> +}
> +
>  # set migrate lock in config file
>  $conf->{lock} = 'migrate';
>  PVE::QemuConfig->write_config($vmid, $conf);
> 
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] spiceproxy CONNECT method

[Stefan Priebe - Profihost AG]

Hello,

it's an ubuntu 16.04 client using the following pkg:

$ dpkg -l | grep virt-viewer
ii  virt-viewer 1.0-1
amd64Displaying the
graphical console of a virtual machine

$ remote-viewer -V
Remote-Viewer Version 1.00

I haven't checked spice-client-gtk at all. I always used remote-viewer
from virt-viewer.

Greets,
Stefan

Am 05.07.2017 um 11:36 schrieb Thomas Lamprecht:
> Hi,
> 
> On 07/04/2017 08:40 AM, Stefan Priebe - Profihost AG wrote:
>> Hello,
>>
>> i'm doing some experiments with the spiceproxy. Currently just trying to
>> get it working.
>>
>> After downloading a spice connection file from the web gui. remote
>> viewer stops with:
>>
>> "failed to connect HTTP proxy connection failed: 501 method 'CONNECT'
>> not available"
>>
>> The access.log of the spice proxy shows:
>> CONNECT
>> pvespiceproxy:595b35b2:123:node1::5a12b84a5038dc4dd37c4eac5bd5af567c7bbe1f:61000
>>
>> HTTP/1.0
>>
>> pve-manager: 4.4-12
>> libpve-http-server-perl: 1.0-4
>> libpve-common-perl: 4.0-94
>>
>> Is this method missing? Does my remote-viewer do strange things?
> 
> Which remote viewer on what OS do you use to test?
> So that I can try to reproduce it. Currently it works with:
> spice-client-gtk 0.33-3.3 under Debian, as far as I can tell.
> 
> cheers,
> Thomas
> Sorry for any inconvenience caused, we're looking for an transparent
> solution here.
> 
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] spiceproxy CONNECT method

[Stefan Priebe - Profihost AG]

Hello,

the following commit fixed it for me.


commit 6ddcde393f8bdbb5aaf3d347213bf819c788478b
Author: Stefan Priebe <s.pri...@profihost.ag>
Date:   Tue Jul 4 11:14:19 2017 +0200

PVE/HTTPServer: add CONNECT method for spice

diff --git a/PVE/HTTPServer.pm b/PVE/HTTPServer.pm
index a6fef7b..2c956de 100755
--- a/PVE/HTTPServer.pm
+++ b/PVE/HTTPServer.pm
@@ -257,6 +257,7 @@ my $known_methods = {
 POST => 1,
 PUT => 1,
 DELETE => 1,
+CONNECT => 1,
 };

 my $split_abs_uri = sub {

Grüße,
Stefan Priebe

Am 04.07.2017 um 08:40 schrieb Stefan Priebe - Profihost AG:
> Hello,
> 
> i'm doing some experiments with the spiceproxy. Currently just trying to
> get it working.
> 
> After downloading a spice connection file from the web gui. remote
> viewer stops with:
> 
> "failed to connect HTTP proxy connection failed: 501 method 'CONNECT'
> not available"
> 
> The access.log of the spice proxy shows:
> CONNECT
> pvespiceproxy:595b35b2:123:node1::5a12b84a5038dc4dd37c4eac5bd5af567c7bbe1f:61000
> HTTP/1.0
> 
> pve-manager: 4.4-12
> libpve-http-server-perl: 1.0-4
> libpve-common-perl: 4.0-94
> 
> Is this method missing? Does my remote-viewer do strange things?
> 
> Thanks!
> 
> Greets,
> Stefan
> ___
> pve-devel mailing list
> pve-devel@pve.proxmox.com
> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
> 
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] spiceproxy CONNECT method

[Stefan Priebe - Profihost AG]

Hello,

i'm doing some experiments with the spiceproxy. Currently just trying to
get it working.

After downloading a spice connection file from the web gui. remote
viewer stops with:

"failed to connect HTTP proxy connection failed: 501 method 'CONNECT'
not available"

The access.log of the spice proxy shows:
CONNECT
pvespiceproxy:595b35b2:123:node1::5a12b84a5038dc4dd37c4eac5bd5af567c7bbe1f:61000
HTTP/1.0

pve-manager: 4.4-12
libpve-http-server-perl: 1.0-4
libpve-common-perl: 4.0-94

Is this method missing? Does my remote-viewer do strange things?

Thanks!

Greets,
Stefan
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] stress testing pve api / pveproxy

[Stefan Priebe - Profihost AG]

Am 27.04.2017 um 19:12 schrieb Dietmar Maurer:
> Hi Stefan,
> 
> is this already solved? Or do you still observe that problem?

Thanks for asking. It seems to be a client related timeout. Not on PVE
side. But what i did not understand is that i don't see it if i raise

$ua->timeout( $secs );

from 10 to 30 but the whole script only runs 1-3s on each run at all.
But it seems to work now.

Greets,
Stefan

> 
>> while stress testing the pve api i'm seeing pretty often a "500 read
>> timeout" response to my simple GET requests against the API.
>>
>> Around 1 out of 50 requests firing one request every 200ms wait for
>> answer fire next one.
>>
>> That one is coming from $response->status_line of a HTTP::Request->new()
>> object against the API.
>>
>> /usr/bin/pveproxy start -debug 1
>>
>> is printing many information especially not what is happening to my request.
>>
>> Any ideas? Well known bug? It's PVE 4.4.
> 
___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] stress testing pve api / pveproxy

[Stefan Priebe - Profihost AG]

Hello,

while stress testing the pve api i'm seeing pretty often a "500 read
timeout" response to my simple GET requests against the API.

Around 1 out of 50 requests firing one request every 200ms wait for
answer fire next one.

That one is coming from $response->status_line of a HTTP::Request->new()
object against the API.

/usr/bin/pveproxy start -debug 1

is printing many information especially not what is happening to my request.

Any ideas? Well known bug? It's PVE 4.4.

-- 
Thanks!
Stefan
___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] applied: [PATCH cluster v6] implement chown and chmod for user root group www-data and perm 0640

[Stefan Priebe - Profihost AG]

THX

Stefan

Am 05.04.2017 um 12:11 schrieb Wolfgang Bumiller:
> applied to both master and stable-4
> 
> On Tue, Apr 04, 2017 at 04:43:31PM +0200, Thomas Lamprecht wrote:
>> From: Stefan Priebe <s.pri...@profihost.ag>
>>
>> This allows us to use management software for files inside of /etc/pve.
>> e.g. saltstack which rely on being able to set uid,gid and chmod
>>
>> Signed-off-by: Thomas Lamprecht <t.lampre...@proxmox.com>
>> Signed-off-by: Stefan Priebe <s.pri...@profihost.ag>
>> ---
>>
>> changes v5 -> v6: adress Wolfgangs comments:
>> * explicitly allow only setting the mode we support, do not mask out 
>> sgid/suid
>>   bits
>> * use more fitting -EPERM not -EACCES, EACCES is for the case that there are
>>   access problems on the path traversal to the file, whereas EPERM is for
>>   access/permission problems for the file itself.
>>
>>  data/src/pmxcfs.c | 38 +-
>>  1 file changed, 37 insertions(+), 1 deletion(-)
>>
>> diff --git a/data/src/pmxcfs.c b/data/src/pmxcfs.c
>> index 1b6cbcc..0f09937 100644
>> --- a/data/src/pmxcfs.c
>> +++ b/data/src/pmxcfs.c
>> @@ -186,6 +186,40 @@ ret:
>>  return ret;
>>  }
>>  
>> +static int cfs_fuse_chmod(const char *path, mode_t mode)
>> +{
>> +int ret = -EPERM;
>> +
>> +cfs_debug("enter cfs_fuse_chmod %s", path);
>> +
>> +mode_t allowed_mode = (S_IRUSR | S_IWUSR);
>> +if (!path_is_private(path))
>> +allowed_mode |= (S_IRGRP);
>> +
>> +// allow only setting our supported modes (0600 for priv, 0640 for rest)
>> +if (mode == allowed_mode)
>> +ret = 0;
>> +
>> +cfs_debug("leave cfs_fuse_chmod %s (%d) mode: %o", path, ret, 
>> (int)mode);
>> +
>> +return ret;
>> +}
>> +
>> +static int cfs_fuse_chown(const char *path, uid_t user, gid_t group)
>> +{
>> +int ret = -EPERM;
>> +
>> +cfs_debug("enter cfs_fuse_chown %s", path);
>> +
>> +// we get -1 if no change should be made
>> +if ((user == 0 || user == -1) && (group == cfs.gid || group == -1))
>> +ret = 0;
>> +
>> +cfs_debug("leave cfs_fuse_chown %s (%d) (uid: %d; gid: %d)", path, ret, 
>> user, group);
>> +
>> +return ret;
>> +}
>> +
>>  static int cfs_fuse_mkdir(const char *path, mode_t mode)
>>  {
>>  cfs_debug("enter cfs_fuse_mkdir %s", path);
>> @@ -488,7 +522,9 @@ static struct fuse_operations fuse_ops = {
>>  .readlink = cfs_fuse_readlink,
>>  .utimens = cfs_fuse_utimens,
>>  .statfs = cfs_fuse_statfs,
>> -.init = cfs_fuse_init
>> +.init = cfs_fuse_init,
>> +.chown = cfs_fuse_chown,
>> +.chmod = cfs_fuse_chmod
>>  };
>>  
>>  static char *
>> -- 
>> 2.11.0
> 
> ___
> pve-devel mailing list
> pve-devel@pve.proxmox.com
> http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
> 
___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] PVE daemon / PVE proxy

[Stefan Priebe - Profihost AG]

Hi,

i have sometimes problems with the pve api - just closing the connection
or giving me a timeout and i wanted to debug this.

I wanted to raise the workers - but wasn't able to find an option to
change the workers of pveproxy and or pvedaemon.

Greets,
Stefan
___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] clustered /var/log/pve

[Stefan Priebe - Profihost AG]

Never mind.I found the culprit the file is just read to early by other nodes.

Stefan

Excuse my typo sent from my mobile phone.

> Am 22.03.2017 um 15:19 schrieb Stefan Priebe - Profihost AG 
> <s.pri...@profihost.ag>:
> 
> Hi,
> 
> this works fine with /var/log/pve/tasks which is the folder i wanted to
> share.
> 
> It seems to work for all logs which are now also available after migrations.
> 
> The only thing which is not working is the migration log itself. Which
> always get's a status of "unexpected status". Is there anything special
> with the migration log? All others get the correct state of OK.
> 
> Greets
> Stefan
> 
>> Am 11.03.2017 um 08:28 schrieb Dietmar Maurer:
>> 
>> 
>>> On March 10, 2017 at 9:24 PM Stefan Priebe - Profihost AG
>>> <s.pri...@profihost.ag> wrote:
>>> 
>>> 
>>> 
>>> Am 10.03.2017 um 21:20 schrieb Dietmar Maurer:
>>>>> Sure. Great. So there's no problem if all files got shared between
>>>>> nodes? 
>>>> 
>>>> Sorry, but I never tested that.
>>>> 
>>>>> I've never looked at the code for the active and index files...
>>>> 
>>>> I guess you would need some cluster wide locking, or use separate 
>>>> directories for each node...
>>> 
>>> flock should be enough. ocfs2 supports that. Does the current code make
>>> use of that? 
>> 
>> yes
>> 
>>> Haven't found it in pve-manager. Is it somewhere else?
>> 
>> pve-common/src/PVE/RESTEnvironment.pm
>> 
___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] clustered /var/log/pve

[Stefan Priebe - Profihost AG]

Hi,

this works fine with /var/log/pve/tasks which is the folder i wanted to
share.

It seems to work for all logs which are now also available after migrations.

The only thing which is not working is the migration log itself. Which
always get's a status of "unexpected status". Is there anything special
with the migration log? All others get the correct state of OK.

Greets
Stefan

Am 11.03.2017 um 08:28 schrieb Dietmar Maurer:
> 
> 
>> On March 10, 2017 at 9:24 PM Stefan Priebe - Profihost AG
>> <s.pri...@profihost.ag> wrote:
>>
>>
>>
>> Am 10.03.2017 um 21:20 schrieb Dietmar Maurer:
>>>> Sure. Great. So there's no problem if all files got shared between
>>>> nodes? 
>>>
>>> Sorry, but I never tested that.
>>>
>>>> I've never looked at the code for the active and index files...
>>>
>>> I guess you would need some cluster wide locking, or use separate 
>>> directories for each node...
>>
>> flock should be enough. ocfs2 supports that. Does the current code make
>> use of that? 
> 
> yes
> 
>> Haven't found it in pve-manager. Is it somewhere else?
> 
> pve-common/src/PVE/RESTEnvironment.pm
> 
___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH] implement chown and chmod for user root group www-data and perm 0640

[Stefan Priebe - Profihost AG]

Hi Thomas,

thanks and yes if you will do a V5 it would be great!

Stefan

Am 21.03.2017 um 10:46 schrieb Thomas Lamprecht:
> Hi,
> 
> On 03/20/2017 03:11 PM, Stefan Priebe wrote:
>> This allows us to use management software for files inside of /etc/pve.
>> f.e. saltstack which rely on being able to set uid,gid and chmod
>>
>> Reviewed-by: Thomas Lamprecht <t.lampre...@proxmox.com>
> 
> If you change the patch behavior (comments or other non semantic changes
> are normally fine) without previous discussion with the old reviewed-by
> people
> I'd suggest dropping them when sending it, as else it suggest that the
> new behavior
> and code was also already reviewed. I know I'm nitpicking for such a
> "small" change,
> but the cluster filesystem is a core part of ours and permission should
> work.
> Anyways, would be great for the next time :)
> 
>> Signed-off-by: Stefan Priebe <s.pri...@profihost.ag>
>> ---
>>   data/src/pmxcfs.c | 38 +-
>>   1 file changed, 37 insertions(+), 1 deletion(-)
>>
>> diff --git a/data/src/pmxcfs.c b/data/src/pmxcfs.c
>> index 1b6cbcc..5f45115 100644
>> --- a/data/src/pmxcfs.c
>> +++ b/data/src/pmxcfs.c
>> @@ -186,6 +186,40 @@ ret:
>>   return ret;
>>   }
>>   +static int cfs_fuse_chmod(const char *path, mode_t mode)
>> +{
>> +int ret = -EACCES;
>> +
>> +cfs_debug("enter cfs_fuse_chmod %s", path);
>> +
>> +// asserts 0640 or 0600, but allows setting UID and GID - some
>> programs need that
>> +if (path_is_private(path)) {
>> +if ((mode & ACCESSPERMS) == (S_IRUSR | S_IWUSR))
>> +ret = 0;
>> +} else if ((mode & ACCESSPERMS) == (S_IRUSR | S_IWUSR | S_IRGRP)) {
>> +ret = 0;
>> +}
>> +
> 
> With the new addition, may I propose another refactoring of above,
> as IMO nested if branches with no parenthesis in the inner level seems
> confusing.
> (I generally do not like the "no-parenthesis if one liners", but it
> seems they are
> widely used here, so I also do not want to break with the style.)
> 
> Whats your opinion on something like:
> 
> -- 
> static int cfs_fuse_chmod(const char *path, mode_t mode)
> {
> int ret = -EACCES;
> 
> cfs_debug("enter cfs_fuse_chmod %s", path);
> 
> mode_t allowed_mode = (S_IRUSR | S_IWUSR);
> if (!path_is_private(path))
> allowed_mode |= (S_IRGRP);
> 
> // ACCESSPERMS masks out UID and GID for the check, some
> programs need to set them
> if ((mode & ACCESSPERMS) == allowed_mode)
> ret = 0;
> 
> cfs_debug("leave cfs_fuse_chmod %s (%d) mode: %o", path, ret,
> (int)mode);
> 
> return ret;
> }
> -- 
> 
> Else, all looks good. I can also make a V5 with those changes on top of
> yours, if you prefer.
> 
> cheers,
> Thomas
> 
>> +cfs_debug("leave cfs_fuse_chmod %s (%d) mode: %o", path, ret,
>> (int)mode);
>> +
>> +return ret;
>> +}
>> +
>> +static int cfs_fuse_chown(const char *path, uid_t user, gid_t group)
>> +{
>> +int ret = -EACCES;
>> +
>> +cfs_debug("enter cfs_fuse_chown %s", path);
>> +
>> +// we get -1 if no change should be made
>> +if ((user == 0 || user == -1) && (group == cfs.gid || group == -1))
>> +ret = 0;
>> +
>> +cfs_debug("leave cfs_fuse_chown %s (%d) (uid: %d; gid: %d)",
>> path, ret, user, group);
>> +
>> +return ret;
>> +}
>> +
>>   static int cfs_fuse_mkdir(const char *path, mode_t mode)
>>   {
>>   cfs_debug("enter cfs_fuse_mkdir %s", path);
>> @@ -488,7 +522,9 @@ static struct fuse_operations fuse_ops = {
>>   .readlink = cfs_fuse_readlink,
>>   .utimens = cfs_fuse_utimens,
>>   .statfs = cfs_fuse_statfs,
>> -.init = cfs_fuse_init
>> +.init = cfs_fuse_init,
>> +.chown = cfs_fuse_chown,
>> +.chmod = cfs_fuse_chmod
>>   };
>> static char *
> 
> 
> ___
> pve-devel mailing list
> pve-devel@pve.proxmox.com
> http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH] implement chown and chmod for user root group www-data and perm 0640

[Stefan Priebe]

This allows us to use management software for files inside of /etc/pve.
f.e. saltstack which rely on being able to set uid,gid and chmod

Reviewed-by: Thomas Lamprecht <t.lampre...@proxmox.com>
Signed-off-by: Stefan Priebe <s.pri...@profihost.ag>
---
 data/src/pmxcfs.c | 38 +-
 1 file changed, 37 insertions(+), 1 deletion(-)

diff --git a/data/src/pmxcfs.c b/data/src/pmxcfs.c
index 1b6cbcc..5f45115 100644
--- a/data/src/pmxcfs.c
+++ b/data/src/pmxcfs.c
@@ -186,6 +186,40 @@ ret:
return ret;
 }
 
+static int cfs_fuse_chmod(const char *path, mode_t mode)
+{
+   int ret = -EACCES;
+
+   cfs_debug("enter cfs_fuse_chmod %s", path);
+
+   // asserts 0640 or 0600, but allows setting UID and GID - some programs 
need that
+   if (path_is_private(path)) {
+   if ((mode & ACCESSPERMS) == (S_IRUSR | S_IWUSR))
+   ret = 0;
+   } else if ((mode & ACCESSPERMS) == (S_IRUSR | S_IWUSR | S_IRGRP)) {
+   ret = 0;
+   }
+
+   cfs_debug("leave cfs_fuse_chmod %s (%d) mode: %o", path, ret, 
(int)mode);
+
+   return ret;
+}
+
+static int cfs_fuse_chown(const char *path, uid_t user, gid_t group)
+{
+   int ret = -EACCES;
+
+   cfs_debug("enter cfs_fuse_chown %s", path);
+
+   // we get -1 if no change should be made
+   if ((user == 0 || user == -1) && (group == cfs.gid || group == -1))
+   ret = 0;
+
+   cfs_debug("leave cfs_fuse_chown %s (%d) (uid: %d; gid: %d)", path, ret, 
user, group);
+
+   return ret;
+}
+
 static int cfs_fuse_mkdir(const char *path, mode_t mode)
 {
cfs_debug("enter cfs_fuse_mkdir %s", path);
@@ -488,7 +522,9 @@ static struct fuse_operations fuse_ops = {
.readlink = cfs_fuse_readlink,
.utimens = cfs_fuse_utimens,
.statfs = cfs_fuse_statfs,
-   .init = cfs_fuse_init
+   .init = cfs_fuse_init,
+   .chown = cfs_fuse_chown,
+   .chmod = cfs_fuse_chmod
 };
 
 static char *
-- 
2.1.4

___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] pve-cluster:i V4 implement chown and chmod for user root group www-data

[Stefan Priebe]

V4:
- allow chmod for priv path as well
___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] broken system / pve-firewall

[Stefan Priebe - Profihost AG]

Am 19.03.2017 um 21:42 schrieb Dietmar Maurer:
>> To me the main question is why does pve-cluster provide a default of 0
>> which disables iptables for bridges and makes pve-firewall useless for
>> linux bridges.
> 
> AFAIR this is for performance reasons ...

sure but pve-firewall isn't working in that case?

Greets,
Steefan
___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] broken system / pve-firewall

[Stefan Priebe - Profihost AG]

Hi,
Am 19.03.2017 um 14:44 schrieb Dietmar Maurer:
>> After digging around for some weeks i found out that the chain FORWARD
>> does not receive packets anymore?
> 
> And hints in syslog?

No the reason is simply that
net.bridge.bridge-nf-call-iptables
is 0 again. Most probably because /etc/sysctl.d is reinitialized for
some reason.

To me the main question is why does pve-cluster provide a default of 0
which disables iptables for bridges and makes pve-firewall useless for
linux bridges.

> Which kernel do you run exactly?
Tested with my own vanilla 4.4 kernel and with 4.4.44-1-pve. But again
this behaviour is expected with net.bridge.bridge-nf-call-iptables=0 for
all kernels.

Greets,
Stefan
___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] broken system / pve-firewall

[Stefan Priebe - Profihost AG]

Hello list,

i'm going crazy with a problem i don't understand.

After some time the pve-firewall stops working to me. It doesn't filter
any packets anymore. If i restart pve-firewall everything is fine again.

After digging around for some weeks i found out that the chain FORWARD
does not receive packets anymore?

It look like this - so NO packets get processed:
# iptables -L FORWARD -vnx
Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
pkts  bytes target prot opt in out source
   destination
   00 PVEFW-FORWARD  all  --  *  *   0.0.0.0/0
 0.0.0.0/0


Status output:
# systemctl status -l pve-firewall.service
● pve-firewall.service - Proxmox VE firewall
   Loaded: loaded (/lib/systemd/system/pve-firewall.service; enabled)
   Active: active (running) since Thu 2017-03-02 13:11:24 CET; 2 weeks 2
days ago
 Main PID: 3056 (pve-firewall)
   CGroup: /system.slice/pve-firewall.service
   └─3056 pve-firewal

Mar 02 13:11:24 dev-cluster pve-firewall[3056]: starting server
Mar 02 13:11:24 dev-cluster systemd[1]: Started Proxmox VE firewall.
Mar 08 19:42:06 dev-cluster pve-firewall[3056]: firewall update time
(5.055 seconds)
Mar 09 17:26:31 dev-cluster pve-firewall[3056]: ipcc_send_rec failed:
Transport endpoint is not connected
Mar 09 20:23:11 dev-cluster pve-firewall[3056]: ipcc_send_rec failed:
Transport endpoint is not connected
Mar 15 10:49:23 dev-cluster pve-firewall[3056]: firewall update time
(5.237 seconds)
Mar 17 08:17:57 dev-cluster pve-firewall[3056]: firewall update time
(5.063 seconds)

# systemctl restart pve-firewall.service
#

# iptables -L FORWARD -vnx
Chain FORWARD (policy ACCEPT 80 packets, 6543 bytes)
pkts  bytes target prot opt in out source
   destination
 32649611 PVEFW-FORWARD  all  --  *  *   0.0.0.0/0
 0.0.0.0/0

After the restart the FORWARD chain get's immediatly packets again.

I noticed that after the restart:
net.bridge.bridge-nf-call-ip6tables
net.bridge.bridge-nf-call-iptables

changed from 0 to 1 which makes sense.

but:
# cat /etc/sysctl.d/pve.conf
net.bridge.bridge-nf-call-ip6tables = 0
net.bridge.bridge-nf-call-iptables = 0
net.bridge.bridge-nf-call-arptables = 0
net.bridge.bridge-nf-filter-vlan-tagged = 0
fs.aio-max-nr = 1048576

# dpkg -S /etc/sysctl.d/pve.conf
pve-cluster: /etc/sysctl.d/pve.conf

Greets,
Stefan
___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] clustered /var/log/pve

[Stefan Priebe - Profihost AG]

Great thanks. Is there any reason why we don't use the existing pmxcfs for that 
path as well?

Stefan

Excuse my typo sent from my mobile phone.

> Am 11.03.2017 um 08:28 schrieb Dietmar Maurer <diet...@proxmox.com>:
> 
> 
> 
>> On March 10, 2017 at 9:24 PM Stefan Priebe - Profihost AG
>> <s.pri...@profihost.ag> wrote:
>> 
>> 
>> 
>> Am 10.03.2017 um 21:20 schrieb Dietmar Maurer:
>>>> Sure. Great. So there's no problem if all files got shared between
>>>> nodes? 
>>> 
>>> Sorry, but I never tested that.
>>> 
>>>> I've never looked at the code for the active and index files...
>>> 
>>> I guess you would need some cluster wide locking, or use separate 
>>> directories for each node...
>> 
>> flock should be enough. ocfs2 supports that. Does the current code make
>> use of that? 
> 
> yes
> 
>> Haven't found it in pve-manager. Is it somewhere else?
> 
> pve-common/src/PVE/RESTEnvironment.pm
> 
> ___
> pve-devel mailing list
> pve-devel@pve.proxmox.com
> http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] clustered /var/log/pve

[Stefan Priebe - Profihost AG]

Am 10.03.2017 um 21:20 schrieb Dietmar Maurer:
>> Sure. Great. So there's no problem if all files got shared between
>> nodes? 
> 
> Sorry, but I never tested that.
> 
>> I've never looked at the code for the active and index files...
> 
> I guess you would need some cluster wide locking, or use separate 
> directories for each node...

flock should be enough. ocfs2 supports that. Does the current code make
use of that? Haven't found it in pve-manager. Is it somewhere else?

Thanks!

Greets,
Stefan
___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] clustered /var/log/pve

[Stefan Priebe - Profihost AG]

Am 10.03.2017 um 20:54 schrieb Dietmar Maurer:
>> Is there any reason to no run /var/log/pve on ocfs2? So that it is
>> shared over all nodes?
> 
> never tried. But I guess it is no problem as long as ocfs2 works (cluster
> quorate).

Sure. Great. So there's no problem if all files got shared between
nodes? I've never looked at the code for the active and index files...

Greets,
Stefan

___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] clustered /var/log/pve

[Stefan Priebe - Profihost AG]

Hello,

i don't like that i don't have a complete task history of a VM after
migration.

Is there any reason to no run /var/log/pve on ocfs2? So that it is
shared over all nodes?

Greets,
Stefan
___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH] implement chown and chmod for user root group www-data and perm 0640

[Stefan Priebe - Profihost AG]

thanks for review. V4 sent.

Stefan

Am 10.03.2017 um 10:20 schrieb Thomas Lamprecht:
> small comment inline,
> 
> On 03/09/2017 08:17 PM, Stefan Priebe wrote:
>> This allows us to use management software for files inside of /etc/pve.
>> f.e. saltstack which rely on being able to set uid,gid and chmod
>>
>> Signed-off-by: Stefan Priebe <s.pri...@profihost.ag>
>> ---
>>   data/src/pmxcfs.c | 33 -
>>   1 file changed, 32 insertions(+), 1 deletion(-)
>>
>> diff --git a/data/src/pmxcfs.c b/data/src/pmxcfs.c
>> index 1b6cbcc..1204331 100644
>> --- a/data/src/pmxcfs.c
>> +++ b/data/src/pmxcfs.c
>> @@ -186,6 +186,35 @@ ret:
>>   return ret;
>>   }
>>   +static int cfs_fuse_chmod(const char *path, mode_t mode)
>> +{
>> +int ret = -EACCES;
>> +
>> +cfs_debug("enter cfs_fuse_chmod %s", path);
>> +
>> +// asserts 0640, but allows setting UID and GID - some programs
>> need that
>> +if ((mode & ACCESSPERMS) == (S_IRUSR | S_IWUSR | S_IRGRP))
>> +ret = 0;
>> +
>> +cfs_debug("leave cfs_fuse_chmod %s (%d) mode: %o", path, ret,
>> (int)mode);
>> +
>> +return ret;
>> +}
>> +
>> +static int cfs_fuse_chown(const char *path, uid_t user, gid_t group)
>> +{
>> +int ret = -EACCES;
>> +
>> +cfs_debug("enter cfs_fuse_chown %s", path);
> 
> Can we add the uid and gid to the debug message, I already needed them
> to review the patch:
> 
> cfs_debug("enter cfs_fuse_chown %s (uid: %d; gid: %d)", path, user, group);
> 
>> +
>> +if (user == 0 && group == cfs.gid)
> 
> If we do not change either group or user chmod uses `-1` for that
> parameter.
> So something like this:
> 
> // we get -1 if no change should be made
> if ((user == 0 || user == -1) && (group == cfs.gid || group == -1))
> 
> should be done here, this allows also:
> 
> chmod root /etc/pve/...
> chmod :www-data /etc/pve/...
> 
> else only
> chmod root:www-data /etc/pve/...
> 
> was allowed (all three are valid for us).
> 
> The rest looks good for me as is!
> 
> With the above changes made you may pickup my reviewed tag - if wanted:
> Reviewed-by Thomas Lamprecht <t.lampre...@proxmox.com>
> 
>> +ret = 0;
>> +
>> +cfs_debug("leave cfs_fuse_chown %s (%d)", path, ret);
>> +
>> +return ret;
>> +}
>> +
>>   static int cfs_fuse_mkdir(const char *path, mode_t mode)
>>   {
>>   cfs_debug("enter cfs_fuse_mkdir %s", path);
>> @@ -488,7 +517,9 @@ static struct fuse_operations fuse_ops = {
>>   .readlink = cfs_fuse_readlink,
>>   .utimens = cfs_fuse_utimens,
>>   .statfs = cfs_fuse_statfs,
>> -.init = cfs_fuse_init
>> +.init = cfs_fuse_init,
>> +.chown = cfs_fuse_chown,
>> +.chmod = cfs_fuse_chmod
>>   };
>> static char *
> 
> 
> ___
> pve-devel mailing list
> pve-devel@pve.proxmox.com
> http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH] implement chown and chmod for user root group www-data and perm 0640

[Stefan Priebe]

This allows us to use management software for files inside of /etc/pve.
f.e. saltstack which rely on being able to set uid,gid and chmod

Reviewed-by: Thomas Lamprecht <t.lampre...@proxmox.com>
Signed-off-by: Stefan Priebe <s.pri...@profihost.ag>
---
 data/src/pmxcfs.c | 34 +-
 1 file changed, 33 insertions(+), 1 deletion(-)

diff --git a/data/src/pmxcfs.c b/data/src/pmxcfs.c
index 1b6cbcc..9866b9c 100644
--- a/data/src/pmxcfs.c
+++ b/data/src/pmxcfs.c
@@ -186,6 +186,36 @@ ret:
return ret;
 }
 
+static int cfs_fuse_chmod(const char *path, mode_t mode)
+{
+   int ret = -EACCES;
+
+   cfs_debug("enter cfs_fuse_chmod %s", path);
+
+   // asserts 0640, but allows setting UID and GID - some programs need 
that
+   if ((mode & ACCESSPERMS) == (S_IRUSR | S_IWUSR | S_IRGRP))
+   ret = 0;
+
+   cfs_debug("leave cfs_fuse_chmod %s (%d) mode: %o", path, ret, 
(int)mode);
+
+   return ret;
+}
+
+static int cfs_fuse_chown(const char *path, uid_t user, gid_t group)
+{
+   int ret = -EACCES;
+
+   cfs_debug("enter cfs_fuse_chown %s", path);
+
+   // we get -1 if no change should be made
+   if ((user == 0 || user == -1) && (group == cfs.gid || group == -1))
+   ret = 0;
+
+   cfs_debug("leave cfs_fuse_chown %s (%d) (uid: %d; gid: %d)", path, ret, 
user, group);
+
+   return ret;
+}
+
 static int cfs_fuse_mkdir(const char *path, mode_t mode)
 {
cfs_debug("enter cfs_fuse_mkdir %s", path);
@@ -488,7 +518,9 @@ static struct fuse_operations fuse_ops = {
.readlink = cfs_fuse_readlink,
.utimens = cfs_fuse_utimens,
.statfs = cfs_fuse_statfs,
-   .init = cfs_fuse_init
+   .init = cfs_fuse_init,
+   .chown = cfs_fuse_chown,
+   .chmod = cfs_fuse_chmod
 };
 
 static char *
-- 
2.1.4

___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] V3 implement chown and chmod for user root group www-data and

[Stefan Priebe]

fixed the intend in fuse fuse_operations as well

___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH] implement chown and chmod for user root group www-data and perm 0640

[Stefan Priebe]

This allows us to use management software for files inside of /etc/pve.
f.e. saltstack which rely on being able to set uid,gid and chmod

Signed-off-by: Stefan Priebe <s.pri...@profihost.ag>
---
 data/src/pmxcfs.c | 33 -
 1 file changed, 32 insertions(+), 1 deletion(-)

diff --git a/data/src/pmxcfs.c b/data/src/pmxcfs.c
index 1b6cbcc..1204331 100644
--- a/data/src/pmxcfs.c
+++ b/data/src/pmxcfs.c
@@ -186,6 +186,35 @@ ret:
return ret;
 }
 
+static int cfs_fuse_chmod(const char *path, mode_t mode)
+{
+   int ret = -EACCES;
+
+   cfs_debug("enter cfs_fuse_chmod %s", path);
+
+   // asserts 0640, but allows setting UID and GID - some programs need 
that
+   if ((mode & ACCESSPERMS) == (S_IRUSR | S_IWUSR | S_IRGRP))
+   ret = 0;
+
+   cfs_debug("leave cfs_fuse_chmod %s (%d) mode: %o", path, ret, 
(int)mode);
+
+   return ret;
+}
+
+static int cfs_fuse_chown(const char *path, uid_t user, gid_t group)
+{
+   int ret = -EACCES;
+
+   cfs_debug("enter cfs_fuse_chown %s", path);
+
+   if (user == 0 && group == cfs.gid)
+   ret = 0;
+
+   cfs_debug("leave cfs_fuse_chown %s (%d)", path, ret);
+
+   return ret;
+}
+
 static int cfs_fuse_mkdir(const char *path, mode_t mode)
 {
cfs_debug("enter cfs_fuse_mkdir %s", path);
@@ -488,7 +517,9 @@ static struct fuse_operations fuse_ops = {
.readlink = cfs_fuse_readlink,
.utimens = cfs_fuse_utimens,
.statfs = cfs_fuse_statfs,
-   .init = cfs_fuse_init
+   .init = cfs_fuse_init,
+   .chown = cfs_fuse_chown,
+   .chmod = cfs_fuse_chmod
 };
 
 static char *
-- 
2.1.4

___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH] implement chown and chmod for user root group www-data and perm 0640

[Stefan Priebe - Profihost AG]

Hello Thomas,


Am 09.03.2017 um 18:09 schrieb Thomas Lamprecht:
> On 03/09/2017 05:26 PM, Stefan Priebe wrote:
>> This allows us to use management software for files inside of /etc/pve.
>> f.e. saltstack which rely on being able to set uid,gid and chmod
>>
>> Signed-off-by: Stefan Priebe <s.pri...@profihost.ag>
>> ---
>>   data/src/pmxcfs.c | 41 -
>>   1 file changed, 40 insertions(+), 1 deletion(-)
>>
>> diff --git a/data/src/pmxcfs.c b/data/src/pmxcfs.c
>> index 1b6cbcc..aa81808 100644
>> --- a/data/src/pmxcfs.c
>> +++ b/data/src/pmxcfs.c
>> @@ -186,6 +186,43 @@ ret:
>>   return ret;
>>   }
>>   +static int cfs_fuse_chmod(const char *path, mode_t mode)
>> +{
>> +  const mode_t pve_mode = S_IRUSR | S_IWUSR | S_IRGRP;
>> +  int mode_i = mode & (S_IRWXU | S_IRWXG | S_IRWXO);
>> +  int pve_mode_i = pve_mode & (S_IRWXU | S_IRWXG | S_IRWXO);
>> +
> why not directly setting the pve_mode_i variable to
> 
> (S_IRUSR | S_IWUSR | S_IRGRP);
> 
> The expression (S_IRWXU | S_IRWXG | S_IRWXO) equals 0777, so doing a
> binary and to pve_mode does not change anything, or am I mistaken?

*urg* was a leftofter from trying different variations ;-) Sorry.

>> +  cfs_debug("enter cfs_fuse_mode %s", path);
> did you mean:
> enter cfs_fuse_chmod

Yes, fixed for next patch.

>> +  int ret = -ENOSYS;
>> +
> EACCES would be more fitting here?

Sure i was just trying to keep it backward compatible. So currrently you
get not implemented so i kept it like that. Changed to EACCES for V2.

> 
> man 2 chmod
> 
> does not specifies ENOSYS, and implementing it and returning ENOSYS (aka
> not implemented) seems strange to me?

Right but it's the current behaviour of pmxcfs. Changed to EACCES anyway
for V2.

>> +  if (pve_mode_i == mode_i) {
>> +ret = 0;
>> +goto ret;
>> +  }
>> +
>> +  ret:
>> +cfs_debug("leave cfs_fuse_mode %s (%d) mode: %o pve_mode: %o",
>> path, ret, mode_i, pve_mode_i);
>> +
> 
> it looks like you mix intend styles here (above spaces and below tabs
> for same intend level),
> please avoid that

Fixed for V2.

> 
>> +return ret;
>> +}
> 
> While you adapt the C style we use in other calls, i.e. the ret: label
> and gotos,
> it over complicates stuff here as no cleanup must be made
Yes the idea was just to keep the style for each function.

> 
> Here a minimal implementation (totally untested™)
> -- 
> static int cfs_fuse_chmod(const char *path, mode_t mode)
> {
> cfs_debug("enter cfs_fuse_chmod %s", path);
> int ret = -EACCES;
> 
> // asserts 0640, but allows setting UID and GID - some programs need
> that
> if (mode & ACCESSPERMS == (S_IRUSR | S_IWUSR | S_IRGRP))
> {
> ret = 0;
> }
> cfs_debug("leave cfs_fuse_chmod %s (%d) mode: %o pve_mode: %o",
> path, ret, mode_i, pve_mode_i);
> 
> return ret;
> }
> -- 
> 
> 
>> +
>> +static int cfs_fuse_chown(const char *path, uid_t user, gid_t group)
>> +{
>> +cfs_debug("enter cfs_fuse_chown %s", path);
>> +
>> +int ret = -ENOSYS;
> Also EACCES here?
> 
> same here (above tabs and below spaces for same intend level),
> 
> rest looks ok.
>> +
>> +if (user == 0 && group == cfs.gid) {
>> +  ret = 0;
>> +  goto ret;
>> +}
>> +
>> +ret:
>> +  cfs_debug("leave cfs_fuse_chown %s (%d)", path, ret);
>> +
>> +return ret;
>> +}
>> +
>>   static int cfs_fuse_mkdir(const char *path, mode_t mode)
>>   {
>>   cfs_debug("enter cfs_fuse_mkdir %s", path);
>> @@ -488,7 +525,9 @@ static struct fuse_operations fuse_ops = {
>>   .readlink = cfs_fuse_readlink,
>>   .utimens = cfs_fuse_utimens,
>>   .statfs = cfs_fuse_statfs,
>> -.init = cfs_fuse_init
>> +.init = cfs_fuse_init,
>> +  .chown = cfs_fuse_chown,
>> +  .chmod = cfs_fuse_chmod
>>   };
>> static char *
> 
> 
> 
> ___
> pve-devel mailing list
> pve-devel@pve.proxmox.com
> http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH] implement chown and chmod for user root group www-data and perm 0640

[Stefan Priebe]

This allows us to use management software for files inside of /etc/pve.
f.e. saltstack which rely on being able to set uid,gid and chmod

Signed-off-by: Stefan Priebe <s.pri...@profihost.ag>
---
 data/src/pmxcfs.c | 33 -
 1 file changed, 32 insertions(+), 1 deletion(-)

diff --git a/data/src/pmxcfs.c b/data/src/pmxcfs.c
index 1b6cbcc..81d6159 100644
--- a/data/src/pmxcfs.c
+++ b/data/src/pmxcfs.c
@@ -186,6 +186,35 @@ ret:
return ret;
 }
 
+static int cfs_fuse_chmod(const char *path, mode_t mode)
+{
+   int ret = -EACCES;
+
+   cfs_debug("enter cfs_fuse_chmod %s", path);
+
+   // asserts 0640, but allows setting UID and GID - some programs need 
that
+   if ((mode & ACCESSPERMS) == (S_IRUSR | S_IWUSR | S_IRGRP))
+   ret = 0;
+
+   cfs_debug("leave cfs_fuse_chmod %s (%d) mode: %o", path, ret, 
(int)mode);
+
+   return ret;
+}
+
+static int cfs_fuse_chown(const char *path, uid_t user, gid_t group)
+{
+   int ret = -EACCES;
+
+   cfs_debug("enter cfs_fuse_chown %s", path);
+
+   if (user == 0 && group == cfs.gid)
+   ret = 0;
+
+   cfs_debug("leave cfs_fuse_chown %s (%d)", path, ret);
+
+   return ret;
+}
+
 static int cfs_fuse_mkdir(const char *path, mode_t mode)
 {
cfs_debug("enter cfs_fuse_mkdir %s", path);
@@ -488,7 +517,9 @@ static struct fuse_operations fuse_ops = {
.readlink = cfs_fuse_readlink,
.utimens = cfs_fuse_utimens,
.statfs = cfs_fuse_statfs,
-   .init = cfs_fuse_init
+   .init = cfs_fuse_init,
+  .chown = cfs_fuse_chown,
+  .chmod = cfs_fuse_chmod
 };
 
 static char *
-- 
2.1.4

___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH] implement chown and chmod for user root group www-data and perm 0640

[Stefan Priebe - Profihost AG]

Am 09.03.2017 um 17:35 schrieb Dietmar Maurer:
> To clarify things: this does not allow to change anything? It just allows
> chown class which would result in no change at all?

Sorry yes. But this returns success if a programm wants to chown or
chmod to the values pve-cluster already has / supports.

At least saltstack always sets chmod and chown values and fails it it
can't. Now it believes that it was successful while providing salt with
the correct values:
user: root
group: www-date
chmod 0640

Greets,
Stefan

> 
>> On March 9, 2017 at 5:26 PM Stefan Priebe <s.pri...@profihost.ag> wrote:
>>
>>
>> This allows us to use management software for files inside of /etc/pve.
>> f.e. saltstack which rely on being able to set uid,gid and chmod
>>
>> Signed-off-by: Stefan Priebe <s.pri...@profihost.ag>
>> ---
>>  data/src/pmxcfs.c | 41 -
>>  1 file changed, 40 insertions(+), 1 deletion(-)
>>
>> diff --git a/data/src/pmxcfs.c b/data/src/pmxcfs.c
>> index 1b6cbcc..aa81808 100644
>> --- a/data/src/pmxcfs.c
>> +++ b/data/src/pmxcfs.c
>> @@ -186,6 +186,43 @@ ret:
>>  return ret;
>>  }
>>  
>> +static int cfs_fuse_chmod(const char *path, mode_t mode)
>> +{
>> +  const mode_t pve_mode = S_IRUSR | S_IWUSR | S_IRGRP;
>> +  int mode_i = mode & (S_IRWXU | S_IRWXG | S_IRWXO);
>> +  int pve_mode_i = pve_mode & (S_IRWXU | S_IRWXG | S_IRWXO);
>> +
>> +  cfs_debug("enter cfs_fuse_mode %s", path);
>> +  int ret = -ENOSYS;
>> +
>> +  if (pve_mode_i == mode_i) {
>> +ret = 0;
>> +goto ret;
>> +  }
>> +
>> +  ret:
>> +cfs_debug("leave cfs_fuse_mode %s (%d) mode: %o pve_mode: %o", path, 
>> ret,
>> mode_i, pve_mode_i);
>> +
>> +return ret;
>> +}
>> +
>> +static int cfs_fuse_chown(const char *path, uid_t user, gid_t group)
>> +{
>> +cfs_debug("enter cfs_fuse_chown %s", path);
>> +
>> +int ret = -ENOSYS;
>> +
>> +if (user == 0 && group == cfs.gid) {
>> +  ret = 0;
>> +  goto ret;
>> +}
>> +
>> +ret:
>> +  cfs_debug("leave cfs_fuse_chown %s (%d)", path, ret);
>> +
>> +return ret;
>> +}
>> +
>>  static int cfs_fuse_mkdir(const char *path, mode_t mode)
>>  {
>>  cfs_debug("enter cfs_fuse_mkdir %s", path);
>> @@ -488,7 +525,9 @@ static struct fuse_operations fuse_ops = {
>>  .readlink = cfs_fuse_readlink,
>>  .utimens = cfs_fuse_utimens,
>>  .statfs = cfs_fuse_statfs,
>> -.init = cfs_fuse_init
>> +.init = cfs_fuse_init,
>> +  .chown = cfs_fuse_chown,
>> +  .chmod = cfs_fuse_chmod
>>  };
>>  
>>  static char *
>> -- 
>> 2.1.4
>>
>> ___
>> pve-devel mailing list
>> pve-devel@pve.proxmox.com
>> http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
> 
___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH] implement chown and chmod for user root group www-data and perm 0640

[Stefan Priebe]

This allows us to use management software for files inside of /etc/pve.
f.e. saltstack which rely on being able to set uid,gid and chmod

Signed-off-by: Stefan Priebe <s.pri...@profihost.ag>
---
 data/src/pmxcfs.c | 41 -
 1 file changed, 40 insertions(+), 1 deletion(-)

diff --git a/data/src/pmxcfs.c b/data/src/pmxcfs.c
index 1b6cbcc..aa81808 100644
--- a/data/src/pmxcfs.c
+++ b/data/src/pmxcfs.c
@@ -186,6 +186,43 @@ ret:
return ret;
 }
 
+static int cfs_fuse_chmod(const char *path, mode_t mode)
+{
+  const mode_t pve_mode = S_IRUSR | S_IWUSR | S_IRGRP;
+  int mode_i = mode & (S_IRWXU | S_IRWXG | S_IRWXO);
+  int pve_mode_i = pve_mode & (S_IRWXU | S_IRWXG | S_IRWXO);
+
+  cfs_debug("enter cfs_fuse_mode %s", path);
+  int ret = -ENOSYS;
+
+  if (pve_mode_i == mode_i) {
+ret = 0;
+goto ret;
+  }
+
+  ret:
+cfs_debug("leave cfs_fuse_mode %s (%d) mode: %o pve_mode: %o", path, ret, 
mode_i, pve_mode_i);
+
+   return ret;
+}
+
+static int cfs_fuse_chown(const char *path, uid_t user, gid_t group)
+{
+   cfs_debug("enter cfs_fuse_chown %s", path);
+
+   int ret = -ENOSYS;
+
+if (user == 0 && group == cfs.gid) {
+  ret = 0;
+  goto ret;
+}
+
+ret:
+  cfs_debug("leave cfs_fuse_chown %s (%d)", path, ret);
+
+return ret;
+}
+
 static int cfs_fuse_mkdir(const char *path, mode_t mode)
 {
cfs_debug("enter cfs_fuse_mkdir %s", path);
@@ -488,7 +525,9 @@ static struct fuse_operations fuse_ops = {
.readlink = cfs_fuse_readlink,
.utimens = cfs_fuse_utimens,
.statfs = cfs_fuse_statfs,
-   .init = cfs_fuse_init
+   .init = cfs_fuse_init,
+  .chown = cfs_fuse_chown,
+  .chmod = cfs_fuse_chmod
 };
 
 static char *
-- 
2.1.4

___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] applied: [PATCH firewall] simulator: make lxc/qemu optional

[Stefan Priebe - Profihost AG]

To avoid cycling deps an alternative might be forward decleration.

http://www.perlmonks.org/?node_id=1057957

Stefan

Excuse my typo sent from my mobile phone.

Am 06.02.2017 um 18:38 schrieb Dietmar Maurer :

>> An alternative might be
>> http://perldoc.perl.org/autouse.html
> 
> Thank for that interesting link. But It would prefer a way to avoid
> that cyclic package dependency at all. On the other way, the 
> current solution is not that bad ;-)
> 
> ___
> pve-devel mailing list
> pve-devel@pve.proxmox.com
> http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] applied: [PATCH firewall] simulator: make lxc/qemu optional

[Stefan Priebe - Profihost AG]

An alternative might be
http://perldoc.perl.org/autouse.html

Stefan

Excuse my typo sent from my mobile phone.

> Am 06.02.2017 um 18:20 schrieb Dietmar Maurer :
> 
> is there really no other way to solve this issue?
> 
> ___
> pve-devel mailing list
> pve-devel@pve.proxmox.com
> http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] pve-firewall / current git master

[Stefan Priebe - Profihost AG]

Hi,

sorry don't know how to teach thunderbird to not break lines. But i
could sent the mail again using pastebin. Just request. Sorry again.

Am 06.02.2017 um 14:59 schrieb Wolfgang Bumiller:
> First a general note (for everyone on the list actually):
> Please don't let your mail clients line-break command outputs, it steals
> way too much of my time reading this :-\.
> (And please prefer iptables-save style output over iptables -L...,
> iptables -L is just horrible. I'm so looking forward to when we can
> finally use `nft list ruleset` instead...)
> 
> Reply inline:
> 
> On Mon, Feb 06, 2017 at 11:25:44AM +0100, Stefan Priebe - Profihost AG wrote:
>> Hi,
>>
>> after upgrading my test cluster to latest git versions from 4.3. I've no
>> working firewall rules anymore. All chains contain an ACCEPT rule. But
>> i'm not sure whether this was also the case with 4.3. But it breaks the
>> rules.
>>
>> The chains is this one:
>> # iptables -L tap137i0-IN -vnx
>> Chain tap137i0-IN (1 references)
>> pkts  bytes target prot opt in out source
>>destination
>>00 DROP   udp  --  *  *   0.0.0.0/0 0.0.0.0/0 
>>udp dpt:67
>>00 ACCEPT tcp  --  *  *   0.0.0.0/0 0.0.0.0/0 
>>match-set PVEFW-0-officeips-v4 src tcp dpt:443
>>1   52 ACCEPT tcp  --  *  *   0.0.0.0/0 0.0.0.0/0 
>>match-set PVEFW-0-ph-networks-v4 src tcp dpt:22
>>   66 3040 GROUP-ph_default_group-IN  all  --  *  * 0.0.0.0/0 
>>0.0.0.0/0
>>   33 1716 ACCEPT all  --  *  *   0.0.0.0/0 0.0.0.0/0 
>>mark match 0x8000/0x8000
>>00 PVEFW-Drop  all  --  *  *   0.0.0.0/0 0.0.0.0/0
>>00 DROP   all  --  *  *   0.0.0.0/0 0.0.0.0/0
>>00all  --  *  *   0.0.0.0/0 0.0.0.0/0 
>>/* PVESIG:zR5Xk5kxEPWmHBeoIDiNXxCERrg */
>>
>> But all packets get accepted by:
>>   33 1716 ACCEPT all  --  *  *   0.0.0.0/0 0.0.0.0/0 
>>mark match 0x8000/0x8000
>>
>> what is this?
> 
> Our "sub"-chains (like groups) generally avoid using ACCEPT directly and
> instead set a mark and RETURN. (In many cases this is not strictly
> necessary but it is more flexible and could potentially allow more
> complex rules (like nesting groups or something, if we ever want that)).
> So the input rules of ph_default_group would be responsible for setting
> this bit in your case above.

Mhm that's even more strange. The default group is this one:
http://pastebin.com/raw/HAxJkhv7

So there's even a drop at the end of this group. So ACCEPT should not be
reachable at all.

My test is a tcp connect to port 3306 which works just fine.

Here both again:
Group:
http://pastebin.com/raw/HAxJkhv7

monitoring list:
http://pastebin.com/raw/4QeCYEVe

iptables tap in:
http://pastebin.com/raw/1QVTJG8K

Greets,
Stefan

> 
> ___
> pve-devel mailing list
> pve-devel@pve.proxmox.com
> http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
> 
___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] pve-firewall / current git master

[Stefan Priebe - Profihost AG]

Hi,

after upgrading my test cluster to latest git versions from 4.3. I've no
working firewall rules anymore. All chains contain an ACCEPT rule. But
i'm not sure whether this was also the case with 4.3. But it breaks the
rules.

The chains is this one:
# iptables -L tap137i0-IN -vnx
Chain tap137i0-IN (1 references)
pkts  bytes target prot opt in out source
   destination
   00 DROP   udp  --  *  *   0.0.0.0/0
 0.0.0.0/0udp dpt:67
   00 ACCEPT tcp  --  *  *   0.0.0.0/0
 0.0.0.0/0match-set PVEFW-0-officeips-v4 src tcp dpt:443
   1   52 ACCEPT tcp  --  *  *   0.0.0.0/0
 0.0.0.0/0match-set PVEFW-0-ph-networks-v4 src tcp dpt:22
  66 3040 GROUP-ph_default_group-IN  all  --  *  *
0.0.0.0/00.0.0.0/0
  33 1716 ACCEPT all  --  *  *   0.0.0.0/0
 0.0.0.0/0mark match 0x8000/0x8000
   00 PVEFW-Drop  all  --  *  *   0.0.0.0/0
  0.0.0.0/0
   00 DROP   all  --  *  *   0.0.0.0/0
 0.0.0.0/0
   00all  --  *  *   0.0.0.0/0
 0.0.0.0/0/* PVESIG:zR5Xk5kxEPWmHBeoIDiNXxCERrg */

But all packets get accepted by:
  33 1716 ACCEPT all  --  *  *   0.0.0.0/0
 0.0.0.0/0mark match 0x8000/0x8000

what is this?

Greets,
Stefan
___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] no subsciption repo hash sum mismatch

[Stefan Priebe - Profihost AG]

Hi,

W: Failed to fetch
http://download.proxmox.com/debian/dists/jessie/pve-no-subscription/binary-amd64/Packages
 Hash Sum mismatch

Stefan
___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] unlock VM through API?

[Stefan Priebe - Profihost AG]

Am 19.12.2016 um 08:40 schrieb Fabian Grünbichler:
> On Mon, Dec 19, 2016 at 07:23:35AM +0100, Stefan Priebe - Profihost AG wrote:
>> Anything wrong or a bug?
>>
>> Greets,
>> Stefan
> 
> nothing wrong. unlocking a VM is possible with the special "qm unlock"
> command (which is not exposed via the API IIRC) or by setting "skiplock"
> when updating the VM configuration (which skips the lock check that
> would otherwise prevent config modification).
> 
> e.g.: pvesh set nodes/nora/qemu/100/config -delete lock -skiplock
> 
> both methods are only available for root@pam

ah OK thanks.

> 
>> Excuse my typo sent from my mobile phone.
>>
>>> Am 16.12.2016 um 22:19 schrieb Stefan Priebe - Profihost AG 
>>> <s.pri...@profihost.ag>:
>>>
>>> Hello,
>>>
>>> is there a way to unlock a VM through the API?
>>>
>>> I tried it this way but this does not work:
>>>
>>> pve:/nodes/testnode1/qemu/100> set config -delete lock
>>> VM is locked (migrate)
>>>
>>> Greets,
>>> Stefan
>> ___
>> pve-devel mailing list
>> pve-devel@pve.proxmox.com
>> http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
> 
> ___
> pve-devel mailing list
> pve-devel@pve.proxmox.com
> http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
> 
___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] 3.10.0 add netpoll support to veth (patch removed)

[Stefan Priebe - Profihost AG]

I think you can simply remove it. It's already upstream and I'm not sure if 
there are users in 3.10.

Thanks.

Stefan

Excuse my typo sent from my mobile phone.

> Am 19.12.2016 um 07:08 schrieb Dietmar Maurer :
> 
> Hi Stefan,
> 
> I just updated the 3.10.0 kernel:
> 
> https://git.proxmox.com/?p=pve-kernel-3.10.0.git;a=shortlog;h=refs/heads/stable-3
> 
> But your patch that adds netpoll "support" to veth does not apply, so 
> I removed it for now. I can include it again if you send me a version
> that compiles with latest kernel. I am not sure is anybody still use that 
> kernel/feature, so I want to minimize efforts...
> 
> ___
> pve-devel mailing list
> pve-devel@pve.proxmox.com
> http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] unlock VM through API?

[Stefan Priebe - Profihost AG]

Anything wrong or a bug?

Greets,
Stefan

Excuse my typo sent from my mobile phone.

> Am 16.12.2016 um 22:19 schrieb Stefan Priebe - Profihost AG 
> <s.pri...@profihost.ag>:
> 
> Hello,
> 
> is there a way to unlock a VM through the API?
> 
> I tried it this way but this does not work:
> 
> pve:/nodes/testnode1/qemu/100> set config -delete lock
> VM is locked (migrate)
> 
> Greets,
> Stefan
___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] unlock VM through API?

[Stefan Priebe - Profihost AG]

Hello,

is there a way to unlock a VM through the API?

I tried it this way but this does not work:

pve:/nodes/testnode1/qemu/100> set config -delete lock
VM is locked (migrate)

Greets,
Stefan
___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] low network speed and high ping times after upgrade to PVE 4.4

[Stefan Priebe - Profihost AG]

Hello,

after upgrading a PVE cluster from 3.4 to 4.4 i have some higher volume
vms which have high ping times even when they're on the same node and
slow network speed tested with iperf.

Has anybody seen something like this before?

--- 192.168.0.11 ping statistics ---
20 packets transmitted, 20 received, 0% packet loss, time 19015ms
rtt min/avg/max/mdev = 0.170/6.428/32.135/7.640 ms

normal ping time would be between 0.1 - 0.2 but it jump very high and
quikly - while serving only around 6Mbit/s of traffic.

iperf shows only jumping 70 Mbit/s-300Mbit/s instead of 1000Mbit/s.

Greets,
Stefan
___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] PVE 4.x watchdog-mux.service

[Stefan Priebe - Profihost AG]

Hi,

since starting to upgrade some nodes to PVE 4.x i've seen that a lot of
them have a failed service watchdog-mux.

Is there any reasons why this one is enabled by default?

# systemctl --failed
  UNIT LOAD   ACTIVE SUBDESCRIPTION
● watchdog-mux.service loaded failed failed Proxmox VE watchdog multiplexer


# systemctl status -l watchdog-mux.service
● watchdog-mux.service - Proxmox VE watchdog multiplexer
   Loaded: loaded (/lib/systemd/system/watchdog-mux.service; static)
   Active: failed (Result: exit-code) since Mon 2016-12-05 14:57:35 CET;
48min ago
  Process: 1551 ExecStart=/usr/sbin/watchdog-mux (code=exited,
status=1/FAILURE)
 Main PID: 1551 (code=exited, status=1/FAILURE)

Dec 05 14:57:34 node1 watchdog-mux[1551]: watchdog open: No such file or
directory
Dec 05 14:57:34 node1 systemd[1]: Started Proxmox VE watchdog multiplexer.
Dec 05 14:57:35 node1systemd[1]: watchdog-mux.service: main process
exited, code=exited, status=1/FAILURE
Dec 05 14:57:35 node1 systemd[1]: Unit watchdog-mux.service entered
failed state.

Greets,
Stefan
___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] making the firewall more robust?

[Stefan Priebe - Profihost AG]

Am 29.11.2016 um 10:29 schrieb Dietmar Maurer:
>> So it seems that the whole firewall breaks if there is somewhere
>> something wrong.
>>
>> I think especially for the firewall it's important to jsut skip that
>> line but process all other values.
> 
> That is how it should work. If there is a bug, we need to fix it. So
> the first question is how to trigger that bug?

# cat 120.fw
[OPTIONS]

policy_in: DROP
log_level_in: nolog
enable: 1

[IPSET letsencrypt]

0.0.0.0/0 # All IP
all_ips

[RULES]

|IN ACCEPT -i net1 -source 0.0.0.0/0 -p tcp -dport  # netcat test
IN ACCEPT -i net1 -source 0.0.0.0/0 -p tcp -dport 80,443 # From all IP
to Port 80 and 443
GROUP ph_default_group -i net1

Stefan
___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] making the firewall more robust?

[Stefan Priebe - Profihost AG]

Am 29.11.2016 um 10:24 schrieb Fabian Grünbichler:
> On Tue, Nov 29, 2016 at 10:10:53AM +0100, Stefan Priebe - Profihost AG wrote:
>> Hello,
>>
>> today i've noticed that the firewall is nearly inactive on a node.
>>
>> systemctl status says:
>> Nov 29 10:07:05 node2 pve-firewall[2534]: status update error:
>> ipset_restore_cmdlist: ipset v6.23: Error in line 3: The value of the
>> CIDR parameter of the IP address is invalid
>> Nov 29 10:07:14 node2 pve-firewall[2534]: status update error:
>> ipset_restore_cmdlist: ipset v6.23: Error in line 3: The value of the
>> CIDR parameter of the IP address is invalid
>> Nov 29 10:07:24 node2 pve-firewall[2534]: status update error:
>> ipset_restore_cmdlist: ipset v6.23: Error in line 3: The value of the
>> CIDR parameter of the IP address is invalid
>>
>> So it seems that the whole firewall breaks if there is somewhere
>> something wrong.
>>
>> I think especially for the firewall it's important to jsut skip that
>> line but process all other values.
>>
>> How is your opinion? Any idea how to "fix" that?
> 
> that bug should already be fixed in git AFAIK.

Which one? Cannot find the commit. I'm ruinning pve-firewall 2.0-31

> there are two problems with partially applying firewall rules:
> - we don't know which rules are invalid (because of course we try to
>   generate valid rules, errors like the above are clearly bugs ;)) - we
>   could guess based on some error message by the underlying tools, but
>   that is error prone
> - applying some rules but not all can have as catastrophic consequences
>   as not applying any (e.g., if you miss a single ACCEPT rule because of
>   a bug, you might not be able to access your cluster at all!)

OK sure. But then we should may be send an email to root in case of a
failure? Currently nobody knows if such a failure happens. Also the
pve-firewall daemon does not fail itself. So even systemd says
pve-firewall is up and running.

Greets,
Stefan
___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] making the firewall more robust?

[Stefan Priebe - Profihost AG]

In this case an employee managed to create the following ipset:

# cat /var/lib/pve-firewall/ipsetcmdlist1
destroy PVEFW-120-letsencrypt-v4_swap
create PVEFW-120-letsencrypt-v4_swap hash:net family inet hashsize 64
maxelem 64
add PVEFW-120-letsencrypt-v4_swap 0.0.0.0/0
swap PVEFW-120-letsencrypt-v4_swap PVEFW-120-letsencrypt-v4
flush PVEFW-120-letsencrypt-v4_swap
destroy PVEFW-120-letsencrypt-v4_swap

which fails:
ipset_restore_cmdlist: ipset v6.23: Error in line 3: The value of the
CIDR parameter of the IP address is invalid

Stefan

Am 29.11.2016 um 10:10 schrieb Stefan Priebe - Profihost AG:
> Hello,
> 
> today i've noticed that the firewall is nearly inactive on a node.
> 
> systemctl status says:
> Nov 29 10:07:05 node2 pve-firewall[2534]: status update error:
> ipset_restore_cmdlist: ipset v6.23: Error in line 3: The value of the
> CIDR parameter of the IP address is invalid
> Nov 29 10:07:14 node2 pve-firewall[2534]: status update error:
> ipset_restore_cmdlist: ipset v6.23: Error in line 3: The value of the
> CIDR parameter of the IP address is invalid
> Nov 29 10:07:24 node2 pve-firewall[2534]: status update error:
> ipset_restore_cmdlist: ipset v6.23: Error in line 3: The value of the
> CIDR parameter of the IP address is invalid
> 
> So it seems that the whole firewall breaks if there is somewhere
> something wrong.
> 
> I think especially for the firewall it's important to jsut skip that
> line but process all other values.
> 
> How is your opinion? Any idea how to "fix" that?
> 
> Greets,
> Stefan
> 
___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] making the firewall more robust?

[Stefan Priebe - Profihost AG]

Hello,

today i've noticed that the firewall is nearly inactive on a node.

systemctl status says:
Nov 29 10:07:05 node2 pve-firewall[2534]: status update error:
ipset_restore_cmdlist: ipset v6.23: Error in line 3: The value of the
CIDR parameter of the IP address is invalid
Nov 29 10:07:14 node2 pve-firewall[2534]: status update error:
ipset_restore_cmdlist: ipset v6.23: Error in line 3: The value of the
CIDR parameter of the IP address is invalid
Nov 29 10:07:24 node2 pve-firewall[2534]: status update error:
ipset_restore_cmdlist: ipset v6.23: Error in line 3: The value of the
CIDR parameter of the IP address is invalid

So it seems that the whole firewall breaks if there is somewhere
something wrong.

I think especially for the firewall it's important to jsut skip that
line but process all other values.

How is your opinion? Any idea how to "fix" that?

Greets,
Stefan
___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] Something missing in http://pve.proxmox.com/wiki/HTTPS_Certificate_Configuration_(Version_4.x_and_newer) ?

[Stefan Priebe - Profihost AG]

Am 22.11.2016 um 14:38 schrieb Fabian Grünbichler:
> On Tue, Nov 22, 2016 at 01:57:47PM +0100, Fabian Grünbichler wrote:
>> On Tue, Nov 22, 2016 at 01:09:17PM +0100, Stefan Priebe - Profihost AG wrote:
>>> Hi,
>>>
>>> Am 22.11.2016 um 12:26 schrieb Fabian Grünbichler:
>>>> On Tue, Nov 22, 2016 at 12:11:22PM +0100, Stefan Priebe - Profihost AG 
>>>> wrote:
>>>>> Am 22.11.2016 um 11:56 schrieb Dietmar Maurer:
>>>>>> I think this commit should solve the issue:
>>>>>>
>>>>>> https://git.proxmox.com/?p=pve-manager.git;a=commitdiff;h=333dd203d5e07d9d3e20d3674a2e3ff2fc89fa8c
>>>>>>
>>>>>>> Please can you test with latest version from git?
>>>>>
>>>>> Already running that version ;-) But thank you for pointing me to this
>>>>> commit. If i revert that one it's working fine again.
>>>>>
>>>>> The issue in my case was that the verify in HTTPServer.pm verify_cb was
>>>>> failing.
>>>>>
>>>>> The documentation says:
>>>>> "fullchain.pem (your certificate and all intermediate certificates,
>>>>> excluding the root certificate, in PEM format)"
>>>>>
>>>>> With the full chain it's not working. I then removed the whole chain and
>>>>> only putted my final crt into that one and now it's working fine. With
>>>>> the full chain $depth was 2 in my case.
>>>>
>>>> I will do some more testing later on - is it possible that you put the
>>>> certificates into the file in the "wrong" order (leaf first, then
>>>> intermediate would be correct)?
>>>>
>>>> The pinning is supposed to only verify the last certificate in the chain
>>>> (as returned by the server), so whether you have a chain of depth 2 or 3
>>>> (self-signed root + leaf or root + ca + leaf) does not matter at all.
>>>> But when reading from a .pem file with multiple certificates OpenSSL
>>>> reads the first one in the file, so it's possible that in your case we
>>>> attempt to compare the leaf certificate's fingerprint (from the
>>>> connection / server) to the CA certificate's fingerprint (from the .pem
>>>> file), which obviously does not work.
>>>>
>>>> If you want, you could send me the certificate file off-list (only the
>>>> certificate please! unless those are from a test node with self-signed
>>>> certificates that you don't care about) and I will try to recreate your
>>>> setup for further tests...
>>>
>>> As this is a real life certificate which is not self signed but signed
>>> from the ca i'm not comfortable with that.
>>
>> understandable. but maybe you could confirm the order? you can just copy
>> the file and split at the end/start marker, with "openssl x509 -in
>> PEMFILE -noout -subject" you should get the subject of each resulting
>> PEMFILE
>>
>>>
>>> But may be you're already right and the order in the file is important.
>>>
>>> Wouldn't it be easier for users to just put the sigle .crt into this
>>> location and put the ca chain into pve-root-ca.pem ?
>>
>> that does not work (unfortunately) - AnyEvent cannot load the chain / CA
>> certificate from a separate file.. since the root certificate is not
>> supposed to be part of the chain sent by the server (it needs to be
>> trusted separately anyway), this is not a problem for the self-signed
>> chain with depth 2. but for the "regular" chain of depth 3 you need to
>> put the leaf/server certificate and the intermediate(s) into
>> pveproxy-ssl.pem (and the root into your OS/browser, if it is not
>> already there ;)).
>>
>> I'll test some more, and as a last workaround we could do the split of
>> pveproxy-ssl.pem in the verification code (and simply only take the one
>> that is not a CA certificate - this is easy to check because the CA
>> property is a certificate extension, but of course we have additional
>> overhead and complexity for parsing and splitting..).
> 
> seems like the error was a flipped default return value in one of the
> last refactorings before sending the patch - could you try the following
> patch?
> 
> a wrong order in pveproxy-ssl.pem would already completely break the
> whole pveproxy, so this cannot be the issue ;)
>
> 
> -- >8 --
> 
> Subject: [PATCH manager] fix SSL verify callback for certificate chains
> 
> igno

Re: [pve-devel] Something missing in http://pve.proxmox.com/wiki/HTTPS_Certificate_Configuration_(Version_4.x_and_newer) ?

[Stefan Priebe - Profihost AG]

Hi,

Am 22.11.2016 um 12:26 schrieb Fabian Grünbichler:
> On Tue, Nov 22, 2016 at 12:11:22PM +0100, Stefan Priebe - Profihost AG wrote:
>> Am 22.11.2016 um 11:56 schrieb Dietmar Maurer:
>>> I think this commit should solve the issue:
>>>
>>> https://git.proxmox.com/?p=pve-manager.git;a=commitdiff;h=333dd203d5e07d9d3e20d3674a2e3ff2fc89fa8c
>>>
>>>> Please can you test with latest version from git?
>>
>> Already running that version ;-) But thank you for pointing me to this
>> commit. If i revert that one it's working fine again.
>>
>> The issue in my case was that the verify in HTTPServer.pm verify_cb was
>> failing.
>>
>> The documentation says:
>> "fullchain.pem (your certificate and all intermediate certificates,
>> excluding the root certificate, in PEM format)"
>>
>> With the full chain it's not working. I then removed the whole chain and
>> only putted my final crt into that one and now it's working fine. With
>> the full chain $depth was 2 in my case.
> 
> I will do some more testing later on - is it possible that you put the
> certificates into the file in the "wrong" order (leaf first, then
> intermediate would be correct)?
> 
> The pinning is supposed to only verify the last certificate in the chain
> (as returned by the server), so whether you have a chain of depth 2 or 3
> (self-signed root + leaf or root + ca + leaf) does not matter at all.
> But when reading from a .pem file with multiple certificates OpenSSL
> reads the first one in the file, so it's possible that in your case we
> attempt to compare the leaf certificate's fingerprint (from the
> connection / server) to the CA certificate's fingerprint (from the .pem
> file), which obviously does not work.
> 
> If you want, you could send me the certificate file off-list (only the
> certificate please! unless those are from a test node with self-signed
> certificates that you don't care about) and I will try to recreate your
> setup for further tests...

As this is a real life certificate which is not self signed but signed
from the ca i'm not comfortable with that.

But may be you're already right and the order in the file is important.

Wouldn't it be easier for users to just put the sigle .crt into this
location and put the ca chain into pve-root-ca.pem ?

Greets,
Stefan

> 
> ___
> pve-devel mailing list
> pve-devel@pve.proxmox.com
> http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
> 
___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] Something missing in http://pve.proxmox.com/wiki/HTTPS_Certificate_Configuration_(Version_4.x_and_newer) ?

[Stefan Priebe - Profihost AG]

Am 22.11.2016 um 11:56 schrieb Dietmar Maurer:
> I think this commit should solve the issue:
> 
> https://git.proxmox.com/?p=pve-manager.git;a=commitdiff;h=333dd203d5e07d9d3e20d3674a2e3ff2fc89fa8c
> 
>> Please can you test with latest version from git?

Already running that version ;-) But thank you for pointing me to this
commit. If i revert that one it's working fine again.

The issue in my case was that the verify in HTTPServer.pm verify_cb was
failing.

The documentation says:
"fullchain.pem (your certificate and all intermediate certificates,
excluding the root certificate, in PEM format)"

With the full chain it's not working. I then removed the whole chain and
only putted my final crt into that one and now it's working fine. With
the full chain $depth was 2 in my case.

Greets,
Stefan

>>> On November 22, 2016 at 11:49 AM Stefan Priebe - Profihost AG
>>> <s.pri...@profihost.ag> wrote:
>>>
>>>
>>> Hi,
>>>
>>> while using a custom certificate was working fine for me with 3. I'm
>>> getting the following error message if i'm connected to node X and want
>>> to view the hw tab of a VM running on node Y.
>>>
>>> 596 ssl3_get_server_certificate: certificate verify failed
>>>
>>> Request
>>> URL:https://node1.X.de:8006/api2/json/nodes/nodeY/qemu/114/status/current
>>> Request Method:GET
>>> Status Code:596 ssl3_get_server_certificate: certificate verify failed
>>>
>>> I following the documentation here:
>>> http://pve.proxmox.com/wiki/HTTPS_Certificate_Configuration_(Version_4.x_and_newer)
>>>
>>> and reverted everything to default and started from fresh replacing
>>> pveproxy-ssl cert files.
>>>
>>> My browser connects fine and without any error to the Web GUI itself. So
>>> it only happens if pve proxies internally.
>>>
>>> Greets,
>>> Stefan
>>> ___
>>> pve-devel mailing list
>>> pve-devel@pve.proxmox.com
>>> http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
>>
>> ___
>> pve-devel mailing list
>> pve-devel@pve.proxmox.com
>> http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
> 
___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] Something missing in http://pve.proxmox.com/wiki/HTTPS_Certificate_Configuration_(Version_4.x_and_newer) ?

[Stefan Priebe - Profihost AG]

Hi,

while using a custom certificate was working fine for me with 3. I'm
getting the following error message if i'm connected to node X and want
to view the hw tab of a VM running on node Y.

596 ssl3_get_server_certificate: certificate verify failed

Request
URL:https://node1.X.de:8006/api2/json/nodes/nodeY/qemu/114/status/current
Request Method:GET
Status Code:596 ssl3_get_server_certificate: certificate verify failed

I following the documentation here:
http://pve.proxmox.com/wiki/HTTPS_Certificate_Configuration_(Version_4.x_and_newer)

and reverted everything to default and started from fresh replacing
pveproxy-ssl cert files.

My browser connects fine and without any error to the Web GUI itself. So
it only happens if pve proxies internally.

Greets,
Stefan
___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] since upgrade v3 => v4 moving qemu-server config files do not work

[Stefan Priebe - Profihost AG]

ignore me my fault...

Stefan

Am 22.11.2016 um 10:15 schrieb Stefan Priebe - Profihost AG:
> Hi,
> 
> in the past / with V3 i was able to move qemu-server VM config files
> around simply with mv.
> 
> Under v4 it seems this no longer works the file automagically move to
> their old location.
> 
> Here an example:
> [node4 ~]# for VM in $(ps aux|grep "kvm"|grep -- "-id"|sed -e "s/.*-id
> //" -e "s/ .*//"); do echo $VM; mv -v
> /etc/pve/nodes/node*/qemu-server/$VM.conf /etc/pve/qemu-server/; done
> 121
> ‘/etc/pve/nodes/node2/qemu-server/121.conf’ ->
> ‘/etc/pve/qemu-server/121.conf’
> 103
> ‘/etc/pve/nodes/node2/qemu-server/103.conf’ ->
> ‘/etc/pve/qemu-server/103.conf’
> 100
> mv: ‘/etc/pve/nodes/node4/qemu-server/100.conf’ and
> ‘/etc/pve/qemu-server/100.conf’ are the same file
> 115
> mv: ‘/etc/pve/nodes/node4/qemu-server/115.conf’ and
> ‘/etc/pve/qemu-server/115.conf’ are the same file
> 146
> mv: ‘/etc/pve/nodes/node4/qemu-server/146.conf’ and
> ‘/etc/pve/qemu-server/146.conf’ are the same file
> 111
> mv: ‘/etc/pve/nodes/node4/qemu-server/111.conf’ and
> ‘/etc/pve/qemu-server/111.conf’ are the same file
> 112
> mv: ‘/etc/pve/nodes/node4/qemu-server/112.conf’ and
> ‘/etc/pve/qemu-server/112.conf’ are the same file
> 
> wait some time
> 
> [node4 ~]# for VM in $(ps aux|grep "kvm"|grep -- "-id"|sed -e "s/.*-id
> //" -e "s/ .*//"); do echo $VM; mv -v
> /etc/pve/nodes/node*/qemu-server/$VM.conf /etc/pve/qemu-server/; done
> 121
> ‘/etc/pve/nodes/node2/qemu-server/121.conf’ ->
> ‘/etc/pve/qemu-server/121.conf’
> 103
> ‘/etc/pve/nodes/node2/qemu-server/103.conf’ ->
> ‘/etc/pve/qemu-server/103.conf’
> 100
> mv: ‘/etc/pve/nodes/node4/qemu-server/100.conf’ and
> ‘/etc/pve/qemu-server/100.conf’ are the same file
> 115
> mv: ‘/etc/pve/nodes/node4/qemu-server/115.conf’ and
> ‘/etc/pve/qemu-server/115.conf’ are the same file
> 146
> mv: ‘/etc/pve/nodes/node4/qemu-server/146.conf’ and
> ‘/etc/pve/qemu-server/146.conf’ are the same file
> 111
> mv: ‘/etc/pve/nodes/node4/qemu-server/111.conf’ and
> ‘/etc/pve/qemu-server/111.conf’ are the same file
> 112
> mv: ‘/etc/pve/nodes/node4/qemu-server/112.conf’ and
> ‘/etc/pve/qemu-server/112.conf’ are the same file
> 
> Stefan
> 
___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] since upgrade v3 => v4 moving qemu-server config files do not work

[Stefan Priebe - Profihost AG]

Hi,

in the past / with V3 i was able to move qemu-server VM config files
around simply with mv.

Under v4 it seems this no longer works the file automagically move to
their old location.

Here an example:
[node4 ~]# for VM in $(ps aux|grep "kvm"|grep -- "-id"|sed -e "s/.*-id
//" -e "s/ .*//"); do echo $VM; mv -v
/etc/pve/nodes/node*/qemu-server/$VM.conf /etc/pve/qemu-server/; done
121
‘/etc/pve/nodes/node2/qemu-server/121.conf’ ->
‘/etc/pve/qemu-server/121.conf’
103
‘/etc/pve/nodes/node2/qemu-server/103.conf’ ->
‘/etc/pve/qemu-server/103.conf’
100
mv: ‘/etc/pve/nodes/node4/qemu-server/100.conf’ and
‘/etc/pve/qemu-server/100.conf’ are the same file
115
mv: ‘/etc/pve/nodes/node4/qemu-server/115.conf’ and
‘/etc/pve/qemu-server/115.conf’ are the same file
146
mv: ‘/etc/pve/nodes/node4/qemu-server/146.conf’ and
‘/etc/pve/qemu-server/146.conf’ are the same file
111
mv: ‘/etc/pve/nodes/node4/qemu-server/111.conf’ and
‘/etc/pve/qemu-server/111.conf’ are the same file
112
mv: ‘/etc/pve/nodes/node4/qemu-server/112.conf’ and
‘/etc/pve/qemu-server/112.conf’ are the same file

wait some time

[node4 ~]# for VM in $(ps aux|grep "kvm"|grep -- "-id"|sed -e "s/.*-id
//" -e "s/ .*//"); do echo $VM; mv -v
/etc/pve/nodes/node*/qemu-server/$VM.conf /etc/pve/qemu-server/; done
121
‘/etc/pve/nodes/node2/qemu-server/121.conf’ ->
‘/etc/pve/qemu-server/121.conf’
103
‘/etc/pve/nodes/node2/qemu-server/103.conf’ ->
‘/etc/pve/qemu-server/103.conf’
100
mv: ‘/etc/pve/nodes/node4/qemu-server/100.conf’ and
‘/etc/pve/qemu-server/100.conf’ are the same file
115
mv: ‘/etc/pve/nodes/node4/qemu-server/115.conf’ and
‘/etc/pve/qemu-server/115.conf’ are the same file
146
mv: ‘/etc/pve/nodes/node4/qemu-server/146.conf’ and
‘/etc/pve/qemu-server/146.conf’ are the same file
111
mv: ‘/etc/pve/nodes/node4/qemu-server/111.conf’ and
‘/etc/pve/qemu-server/111.conf’ are the same file
112
mv: ‘/etc/pve/nodes/node4/qemu-server/112.conf’ and
‘/etc/pve/qemu-server/112.conf’ are the same file

Stefan
___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH] VZDump: die with error if plugin loading fails

[Stefan Priebe - Profihost AG]

Am 17.11.2016 um 08:42 schrieb Fabian Grünbichler:
> On Thu, Nov 17, 2016 at 07:01:24AM +0100, Dietmar Maurer wrote:
>>> It is really hard to review patches without descriptions. Please
>>> can you add minimal information?
>>
>> Oh, just saw you sent that in a separate mail - please ignore me!
> 
> Just as a side note - it makes a lot of sense to include the rationale
> behind a patch (or a short version of it) in the commit message itself,
> except for very trivial patches or where it is really obvious.  This not
> only helps understanding when looking at the code later, but also helps
> when reviewing the history of a specific part of the code.
> 
> Also, for single patches separate cover letters are IMHO a bit noisy, I
> prefer putting any extra remarks that should not be committed in the
> summary part of a patch (below the three dashes).

Thanks!

Will do so.

Stefan

> The following example is taken from our developer documentation in the
> wiki:
> 
> From 12345abcde Mon Sep 12 00:00:00 2001
> From: Git Commiter 
> Date: Fri, 7 Oct 2016 08:30:17 +0200
> Subject: [PATCH container 1/2] Fix #1013: this and that
> 
> Here is your commit message.
> It explains the bugfix and ends after thisline.
> 
> ---
>  ***HERE*** you can write your comments.
>  If this is a new version of an old patch, explain your changes here
> 
>  src/PVE/Tools.pm | 2 +-
> 
> diff --git a/src/PVE/Tools.pm b/src/PVE/Tools.pm
> (...)
> 
___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH] RBD: snap purge does not support automatic unprotect so list all snapshots and then unprotect and delete them

[Stefan Priebe - Profihost AG]

Am 17.11.2016 um 07:33 schrieb Dietmar Maurer:
> AFAIK we only protect base volumes, and we 'unprotect' that in the code.
> So what is the purpose of this patch?
> 

good question ;-) I just remember that i had this situation where i did
clones from snapshots which results in protected snapshots.

Greets,
Stefan
___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] VZDump/QemuServer: set bless clas correctly

[Stefan Priebe - Profihost AG]

Am 17.11.2016 um 07:20 schrieb Dietmar Maurer:
>> While blessing it is good practise to provide the class. This makes it also
>> possible to use
>> QemuServer as a base / parent class.
> 
> Why do you want another class (QemuServer) as base?

We've a custom class PHQemuServer which has VZDump/QemuServer as a base
class as we're doing backups in some different way. That's a private use
case but it's common to use it this way and add the class to the bless.

See also:
http://modernperlbooks.com/books/modern_perl/chapter_07.html#Ymxlc3NlZF9yZWZlcmVuY2Vz

> Isn't it better to remove that whole $class parameter?
> 
> sub new {
> my ($vzdump) = @_;

This would result in the need to call
VZDUmp::QemuServer::new(..);

never saw that before for creating objects ;-)

Stefan
___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH] allow --allow-shrink on RBD resize

[Stefan Priebe - Profihost AG]

Am 17.11.2016 um 06:44 schrieb Dietmar Maurer:
> This is dangerous and can destroy data. So is there a real reason
> to allow that?

Yes it can. That's the reason i've not exported this to the GUI. But it
is a real use case today. More and more FS supports shrinking.

f.e. Windows ntfs, ext4, btrfs, ...

So there are several cases where you want to shrink a volume. Without a
downtime of the server.

Greets,
Stefan

> 
>> On November 16, 2016 at 8:13 PM Stefan Priebe <s.pri...@profihost.ag> wrote:
>>
>>
>> Signed-off-by: Stefan Priebe <s.pri...@profihost.ag>
>> ---
>>  PVE/Storage/RBDPlugin.pm | 2 +-
>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/PVE/Storage/RBDPlugin.pm b/PVE/Storage/RBDPlugin.pm
>> index 09562fe..eb0c256 100644
>> --- a/PVE/Storage/RBDPlugin.pm
>> +++ b/PVE/Storage/RBDPlugin.pm
>> @@ -602,7 +602,7 @@ sub volume_resize {
>>  
>>  my ($vtype, $name, $vmid) = $class->parse_volname($volname);
>>  
>> -my $cmd = &$rbd_cmd($scfg, $storeid, 'resize', '--size',
>> ($size/1024/1024), $name);
>> +my $cmd = &$rbd_cmd($scfg, $storeid, 'resize', '--allow-shrink',
>> '--size', ($size/1024/1024), $name);
>>  run_rbd_command($cmd, errmsg => "rbd resize '$volname' error");
>>  return undef;
>>  }
>> -- 
>> 2.1.4
>>
>> ___
>> pve-devel mailing list
>> pve-devel@pve.proxmox.com
>> http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
> 
___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH] allow --allow-shrink on RBD resize

[Stefan Priebe - Profihost AG]

Am 16.11.2016 um 21:41 schrieb Alexandre DERUMIER:
> It is usefull if we growing the volume ?

no - but the function is called resize not grow ;-) So i use it also for
shrinking volumes.

> I'm seeing a refused tracker/pr here to disallow --alow-shrink on extend
> 
> http://tracker.ceph.com/issues/15991
> https://github.com/ceph/ceph/pull/9408

Both are closed or marked as refused so i don't think this will change
in the future.

Greets,
Stefan

> - Mail original -----
> De: "Stefan Priebe, Profihost AG" <s.pri...@profihost.ag>
> À: "pve-devel" <pve-devel@pve.proxmox.com>
> Envoyé: Mercredi 16 Novembre 2016 20:13:59
> Objet: [pve-devel] [PATCH] allow --allow-shrink on RBD resize
> 
> Signed-off-by: Stefan Priebe <s.pri...@profihost.ag> 
> --- 
> PVE/Storage/RBDPlugin.pm | 2 +- 
> 1 file changed, 1 insertion(+), 1 deletion(-) 
> 
> diff --git a/PVE/Storage/RBDPlugin.pm b/PVE/Storage/RBDPlugin.pm 
> index 09562fe..eb0c256 100644 
> --- a/PVE/Storage/RBDPlugin.pm 
> +++ b/PVE/Storage/RBDPlugin.pm 
> @@ -602,7 +602,7 @@ sub volume_resize { 
> 
> my ($vtype, $name, $vmid) = $class->parse_volname($volname); 
> 
> - my $cmd = &$rbd_cmd($scfg, $storeid, 'resize', '--size', ($size/1024/1024), 
> $name); 
> + my $cmd = &$rbd_cmd($scfg, $storeid, 'resize', '--allow-shrink', '--size', 
> ($size/1024/1024), $name); 
> run_rbd_command($cmd, errmsg => "rbd resize '$volname' error"); 
> return undef; 
> } 
> 
___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] VZDump: die with error if plugin loading fails

[Stefan Priebe]

This took me some time that a custom modification was preventing a whole plugin 
to fail loading. 
The warn also hides in the systemctl status -l / journal log. I think dying is 
better if a plugin
contains an error.

[PATCH] VZDump: die with error if plugin loading fails
___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel