Re: [pve-devel] Default cache mode for VM hard drives

2015-05-28 Thread Stanislav German-Evtushenko
Hi Dietmar, I did it couple of times already and everytime I had the same answer "upper layer problem". Well, as we've done this long way up to this point I would like to continue. I have just done the same test with mdadm and not DRBD. And what I found that this problem was reproducible on the s

[pve-devel] [PATCH] fix a regex typo in run_command

2015-05-28 Thread Wolfgang Bumiller
m/|/ is always true as it effectively matches 'nothing or nothing anywhere in a string' looks like it was supposed to be m/\|/ --- src/PVE/Tools.pm | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/PVE/Tools.pm b/src/PVE/Tools.pm index 1bc9eec..8e18087 100644 --- a/src/PVE/To

Re: [pve-devel] Default cache mode for VM hard drives

2015-05-28 Thread Dietmar Maurer
> I have just done the same test with mdadm and not DRBD. And what I found > that this problem was reproducible on the software raid too, just as it was > claimed by Lars Ellenberg. It means that problem is not only related to > DRBD but to O_DIRECT mode generally when we don't use host cache and a

Re: [pve-devel] Default cache mode for VM hard drives

2015-05-28 Thread Stanislav German-Evtushenko
Moreover, if you create ext3 on top of md0 and repeat then raid array becomes inconsistent too. # Additional steps: mkfs.ext3 /dev/md0 mkdir /tmp/ext3 mount /dev/md0 /tmp/ext3 ./a.out /tmp/ext3/testfile1 # and then: vbindiff /tmp/mdadm{1,2} #press enter multiple times to skip metadata On T

Re: [pve-devel] Default cache mode for VM hard drives

2015-05-28 Thread Dietmar Maurer
> The testcase is bellow. Oh, I missed that test case. I will try to reproduce that - give me some time. ___ pve-devel mailing list pve-devel@pve.proxmox.com http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] Default cache mode for VM hard drives

2015-05-28 Thread Alexandre DERUMIER
Hi, not sure it's related, but with O_DIRECT I think that the write need to be aligned with multiple of 4k block. (or 512bytes) (and I remember some bug with qemu and and 512b-logical/4k-physical disks http://pve.proxmox.com/pipermail/pve-devel/2012-November/004530.html I'm not an expert so I

Re: [pve-devel] Default cache mode for VM hard drives

2015-05-28 Thread Eneko Lacunza
Hi Stanislav, I really think you should have a look where the problem is. This is not Proxmox specific at all, so this list doesn't seem the place to diagnose it. If DRBD guys think it's an upper layer problem, maybe they can point you to the problematic layer, and then contact the mantainer

Re: [pve-devel] Default cache mode for VM hard drives

2015-05-28 Thread Stanislav German-Evtushenko
> not sure it's related, but with O_DIRECT I think that the write need to be aligned with multiple of 4k block. (or 512bytes) That is right and you just can't use O_DIRECT without alignment. You would just get an error on "write" system call. If you check drbd_oos_test.c you find posix_memalign the

Re: [pve-devel] Default cache mode for VM hard drives

2015-05-28 Thread Alexandre DERUMIER
>>I recall from ceph list that there were some barrier problems in kernels < >>2.6.33 . I don't know whether those are fixed in the kernel from RHEL6 >>Proxmox uses... About barrier, they are multiple layers (fs, lvm,md, virtio-blk,...) where it was buggy. It should be ok with any kernel > 2.6

Re: [pve-devel] Default cache mode for VM hard drives

2015-05-28 Thread Stanislav German-Evtushenko
Dietmar, fsync esures that data reaches underlying hardware but it does not help being sure that buffer is not changed until it is fully written. I will describe my understanding here why we get this problem with O_DIRECT and don't have without. ** Without O_DIRECT ** 1. Application tries to wri

Re: [pve-devel] Default cache mode for VM hard drives

2015-05-28 Thread Stanislav German-Evtushenko
Alexandre, This is all correct but not related to inconsistency issue. Stanislav On Thu, May 28, 2015 at 10:44 AM, Alexandre DERUMIER wrote: > >>That is right and you just can't use O_DIRECT without alignment. You > would just get an error on "write" system call. If you check > drbd_oos_test.c

Re: [pve-devel] Default cache mode for VM hard drives

2015-05-28 Thread Alexandre DERUMIER
>>Resume: when working in O_DIRECT mode QEMU has to wait until "write" system >>call is finished before changing this buffer OR QEMU has to create new buffer >>every time OR ... other ideas? AFAIK, only O_DSYNC can guarantee that data are really written to the last layer(disk platters) That's

Re: [pve-devel] Default cache mode for VM hard drives

2015-05-28 Thread Stanislav German-Evtushenko
Alexandre, > do you see the problem with qemu cache=directsync ? (O_DIRECT + O_DSYNC). Yes, it happens in less number of cases (may be 10 times less) but still happens. I have a reproducible case with Windows 7 and directsync. Stanislav On Thu, May 28, 2015 at 11:18 AM, Alexandre DERUMIER wrote

Re: [pve-devel] Default cache mode for VM hard drives

2015-05-28 Thread Stanislav German-Evtushenko
Alexandre, > That's why we need to use barrier or FUA in last kernel in guest, when using O_DIRECT, to be sure that guest filesystem is ok and datas are flushed at regular interval. The problems are: - Linux swap - no barrier or something similar - Windows - I have no idea what Windows does to en

Re: [pve-devel] Grub problems and ZFS

2015-05-28 Thread Emmanuel Kasper
On 05/27/2015 12:15 PM, Dietmar Maurer wrote: >> IMHO this way of bypassing os-prober is cleaner than adding a 'Conflict' >> in our zfs-grub package, since it minimizes the packages conflicts when >> adding our proxmox repo on top of debian's. >> >> [1] https://bugs.debian.org/cgi-bin/bugreport.c

Re: [pve-devel] Default cache mode for VM hard drives

2015-05-28 Thread Wolfgang Bumiller
I was able to reproduce the problem on my local machine (kernel 3.10). To be sure everything's correct I added some error checking to the code. I'm attaching the changed source (and the bdiff source). Transcript is below. I also added an fsync() before close() due to this section in close(2)'s NO

[pve-devel] High Performance SSH

2015-05-28 Thread dea
Hi all !!! Proxmox uses ssh to move data from nodes (ok, is possible to disable encryption but is not safe). I've find this... http://www.psc.edu/index.php/hpn-ssh What do you all think? Luca ___ pve-devel mailing list pve-devel@pve.proxmox.com htt

[pve-devel] [PATCH 3/4] Temporary disable loading the tabs of the main Config panel

2015-05-28 Thread Emmanuel Kasper
--- www/manager5/dc/Config.js | 2 ++ 1 file changed, 2 insertions(+) diff --git a/www/manager5/dc/Config.js b/www/manager5/dc/Config.js index 6c19805..aa9c0f8 100644 --- a/www/manager5/dc/Config.js +++ b/www/manager5/dc/Config.js @@ -100,6 +100,8 @@ Ext.define('PVE.dc.Config', {

[pve-devel] [PATCH 4/4] Add comment header for important classes, idea taken from Workspace.js

2015-05-28 Thread Emmanuel Kasper
--- www/manager5/dc/Config.js | 4 www/manager5/dc/Log.js| 4 www/manager5/dc/Tasks.js | 4 www/manager5/form/ViewSelector.js | 3 +++ www/manager5/panel/StatusPanel.js | 3 +++ www/manager5/tree/ResourceTree.js | 3 +++ 6 files changed, 21 insertions(+)

[pve-devel] [PATCH 1/4] Remove unused misspelled parameter

2015-05-28 Thread Emmanuel Kasper
--- www/manager/data/ResourceStore.js | 1 - www/manager/form/NodeSelector.js | 1 - www/manager/form/RealmComboBox.js | 1 - www/manager/form/ViewSelector.js | 1 - www/manager/node/TimeEdit.js | 2 +- www/manager5/data/ResourceStore.js | 1 - www/manager5/form/RealmComboBox.js | 1 -

[pve-devel] (no subject)

2015-05-28 Thread Emmanuel Kasper
This patch serie allows to load the bottom Statuspanel, and add some comment headers to the classes already added in manager5/ ___ pve-devel mailing list pve-devel@pve.proxmox.com http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH 2/4] Add necessary javascript includes to load Status Panel

2015-05-28 Thread Emmanuel Kasper
--- PVE/ExtJSIndex5.pm| 3 +++ www/manager5/Workspace.js | 2 +- 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/PVE/ExtJSIndex5.pm b/PVE/ExtJSIndex5.pm index 34952ac..0f1eb89 100644 --- a/PVE/ExtJSIndex5.pm +++ b/PVE/ExtJSIndex5.pm @@ -31,6 +31,7 @@ _EOD +

Re: [pve-devel] High Performance SSH

2015-05-28 Thread Dietmar Maurer
> I've find this... > > http://www.psc.edu/index.php/hpn-ssh > > What do you all think? This is great, but unfortunately ssh people rejected those patches (AFAIK). So default ssh tools from Debian does not have that features. ___ pve-devel mailing l

[pve-devel] [PATCH 2/3] implement hotplug for cpuunits

2015-05-28 Thread Alexandre Derumier
Signed-off-by: Alexandre Derumier --- PVE/QemuServer.pm | 4 1 file changed, 4 insertions(+) diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm index 21fa84c..a8177d7 100644 --- a/PVE/QemuServer.pm +++ b/PVE/QemuServer.pm @@ -3949,6 +3949,8 @@ sub vmconfig_hotplug_pending { } el

[pve-devel] [PATCH 3/3] remove old openvz fairscheduler code

2015-05-28 Thread Alexandre Derumier
Signed-off-by: Alexandre Derumier --- PVE/QemuServer.pm | 87 --- 1 file changed, 87 deletions(-) diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm index a8177d7..bf0792f 100644 --- a/PVE/QemuServer.pm +++ b/PVE/QemuServer.pm @@ -119,78 +119,6

[pve-devel] [PATCH 1/3] implement cgroups && cpu.shares

2015-05-28 Thread Alexandre Derumier
Signed-off-by: Alexandre Derumier --- PVE/QemuServer.pm | 60 +++ 1 file changed, 60 insertions(+) diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm index 22ff875..21fa84c 100644 --- a/PVE/QemuServer.pm +++ b/PVE/QemuServer.pm @@ -67,6 +67,58

[pve-devel] qemu-server: cgroups && cpu.shares implementation

2015-05-28 Thread Alexandre Derumier
Hi, Here the patches serie for implemtation of cpuunits through cgroups ___ pve-devel mailing list pve-devel@pve.proxmox.com http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] High Performance SSH

2015-05-28 Thread dea
Il Thu, 28 May 2015 12:02:21 +0200 (CEST), Dietmar Maurer scrisse > > I've find this... > > > > http://www.psc.edu/index.php/hpn-ssh > > > > What do you all think? > > This is great, but unfortunately ssh people rejected those patches > (AFAIK). So default ssh tools from Debian does not have t

Re: [pve-devel] High Performance SSH

2015-05-28 Thread Eneko Lacunza
On 28/05/15 12:38, dea wrote: Il Thu, 28 May 2015 12:02:21 +0200 (CEST), Dietmar Maurer scrisse I've find this... http://www.psc.edu/index.php/hpn-ssh What do you all think? This is great, but unfortunately ssh people rejected those patches (AFAIK). So default ssh tools from Debian does not

Re: [pve-devel] qemu-server: cgroups && cpu.shares implementation

2015-05-28 Thread Dietmar Maurer
> Here the patches serie for implemtation of cpuunits through cgroups If you implement it inside qemu-server you have a race-condition, because you do it too late and qemu already started the threads? Or maybe only parts of necessary threads are already created? So wouldn't it be easier to do it

Re: [pve-devel] Default cache mode for VM hard drives

2015-05-28 Thread Stanislav German-Evtushenko
Hi Eneko, Writing in QEMU-KVM are not thread-safe. I don't know if this is "by design" or just a bug but proving this information here is necessary to show that we should find a solution or work around for Proxmox. General problem is that using Proxmox VE with default settings can make any of Sof

Re: [pve-devel] qemu-server: cgroups && cpu.shares implementation

2015-05-28 Thread Stefan Priebe - Profihost AG
Am 28.05.2015 um 12:51 schrieb Dietmar Maurer: >> Here the patches serie for implemtation of cpuunits through cgroups > > If you implement it inside qemu-server you have a race-condition, because > you do it too late and qemu already started the threads? Or maybe only parts > of necessary threads

Re: [pve-devel] High Performance SSH

2015-05-28 Thread dea
> I don't think it is wise to play with security-related software in > the stack. If OpenBSD and Debian (or for the matter all the other > distros) haven't applied those patches, I'm sure there is some > reason, although maybe it being only "uncertainty". Yes, is true. But I think that from a

Re: [pve-devel] Default cache mode for VM hard drives

2015-05-28 Thread Alexandre DERUMIER
>>BTW: can anybody test drbd_oos_test.c against Ceph? I guess we will have the >>same result. I think they are no problem with ceph, qemu cache option only enable|disable rbd_cache. qemu rbd access is only userland, so host don't have any cache or buffer. When data is written to ceph, it's wri

Re: [pve-devel] qemu-server: cgroups && cpu.shares implementation

2015-05-28 Thread Alexandre DERUMIER
>>If you implement it inside qemu-server you have a race-condition, because >>you do it too late and qemu already started the threads? Or maybe only parts >>of necessary threads are already created? Yes maybe, I don't have see race when I have tested it. new threads should go automatically to t

Re: [pve-devel] Default cache mode for VM hard drives

2015-05-28 Thread Stanislav German-Evtushenko
Alexandre, The important point is whether O_DIRECT is used with Ceph or not. Don't you know? > qemu rbd access is only userland, so host don't have any cache or buffer. If RBD device does not use host cache then it is very likely that RBD utilizes O_DIRECT. I am not sure if there are other ways t

Re: [pve-devel] Default cache mode for VM hard drives

2015-05-28 Thread Eneko Lacunza
Hi Stanislav, On 28/05/15 13:10, Stanislav German-Evtushenko wrote: Alexandre, The important point is whether O_DIRECT is used with Ceph or not. Don't you know? > qemu rbd access is only userland, so host don't have any cache or buffer. If RBD device does not use host cache then it is very

Re: [pve-devel] Default cache mode for VM hard drives

2015-05-28 Thread Eneko Lacunza
Hi, I'm not kernel/IO expert in any way, but I think this test program has a race condition, so it is not helping us diagnose the problem. We're writing to buffer x while it is in use by write syscall. This is plainly wrong on userspace. Cheers Eneko On 28/05/15 11:27, Wolfgang Bumiller wr

Re: [pve-devel] qemu-server: cgroups && cpu.shares implementation

2015-05-28 Thread Alexandre DERUMIER
>>Not tested but what about this: >> >>fork() >># in child >>put current pid into cgroup >>exec kvm Yes, I think it should work, if we put the pid of the forked process in cgroups. Other child threads should go automaticaly to the parent cgroup. I have done tests with hotplug virtio-net with

Re: [pve-devel] Default cache mode for VM hard drives

2015-05-28 Thread Stanislav German-Evtushenko
Eneko, > I'm not kernel/IO expert in any way, but I think this test program has a race > condition, so it is not helping us diagnose the problem. > We're writing to buffer x while it is in use by write syscall. This is > plainly wrong on userspace. Yes, and this is exactly what is happening ins

Re: [pve-devel] Default cache mode for VM hard drives

2015-05-28 Thread Alexandre DERUMIER
>> qemu rbd access is only userland, so host don't have any cache or buffer. >>If RBD device does not use host cache then it is very likely that RBD >>utilizes O_DIRECT. I am not sure if there are other ways to avoid host cache. qemu use librbd to access directly to ceph, so host don't have any

Re: [pve-devel] Default cache mode for VM hard drives

2015-05-28 Thread Wolfgang Bumiller
> On May 28, 2015 at 1:31 PM Eneko Lacunza wrote: > I'm not kernel/IO expert in any way, but I think this test program has a > race condition, so it is not helping us diagnose the problem. > > We're writing to buffer x while it is in use by write syscall. This is > plainly wrong on userspace.

Re: [pve-devel] Default cache mode for VM hard drives

2015-05-28 Thread Dietmar Maurer
> I'm not kernel/IO expert in any way, but I think this test program has a > race condition, so it is not helping us diagnose the problem. > > We're writing to buffer x while it is in use by write syscall. This is > plainly wrong on userspace. For this test, we do not care about userspace seman

Re: [pve-devel] Default cache mode for VM hard drives

2015-05-28 Thread Stanislav German-Evtushenko
Alexandre, > qemu use librbd to access directly to ceph, so host don't have any /dev/rbd.. or filesystem mount. Ah, I understand, this is not a normal block device but userspace lib. > ceph use O_DIRECT+O_DYNC to write to the journal of osds. Is this done inside KVM process? If so then KVM keeps

Re: [pve-devel] Default cache mode for VM hard drives

2015-05-28 Thread Dietmar Maurer
> > ceph use O_DIRECT+O_DYNC to write to the journal of osds. > Is this done inside KVM process? If so then KVM keeps buffer for this > O_DIRECT writing. Therefore if multiple threads can access (and change) > this buffer at the same time then the similar issue can happen in theory. It only happen

Re: [pve-devel] Default cache mode for VM hard drives

2015-05-28 Thread Stanislav German-Evtushenko
Dietmar, >> I'm not kernel/IO expert in any way, but I think this test program has a >> race condition, so it is not helping us diagnose the problem. >> >> We're writing to buffer x while it is in use by write syscall. This is >> plainly wrong on userspace. > > For this test, we do not care about

Re: [pve-devel] Default cache mode for VM hard drives

2015-05-28 Thread Eneko Lacunza
On 28/05/15 13:44, Stanislav German-Evtushenko wrote: Eneko, > I'm not kernel/IO expert in any way, but I think this test program has a race condition, so it is not helping us diagnose the problem. > We're writing to buffer x while it is in use by write syscall. This is plainly wrong on usersp

Re: [pve-devel] Default cache mode for VM hard drives

2015-05-28 Thread Wolfgang Bumiller
> On May 28, 2015 at 2:07 PM Stanislav German-Evtushenko > wrote: > With O_DIRECT we have to trust our user space application because data > is getting by kernel directly from the application memory. We can > think that kernel could copy buffer from user space before writing it > to block device h

Re: [pve-devel] Default cache mode for VM hard drives

2015-05-28 Thread Eneko Lacunza
On 28/05/15 13:49, Dietmar Maurer wrote: I'm not kernel/IO expert in any way, but I think this test program has a race condition, so it is not helping us diagnose the problem. We're writing to buffer x while it is in use by write syscall. This is plainly wrong on userspace. For this test, we do

Re: [pve-devel] Default cache mode for VM hard drives

2015-05-28 Thread Stanislav German-Evtushenko
Eneko, > Note that without O_DIRECT you won't get a "correct" result either; disk > may end not containing the data in the buffer when write was called. > Softmirror data will be identically uncertain :) You are right. That is why I suppose there is a bug (operations with buffer are not ensured t

Re: [pve-devel] qemu-server: cgroups && cpu.shares implementation

2015-05-28 Thread Alexandre DERUMIER
Another way could be use launch qemu with systemd-run, I think we can specify cgroup (slice in systemd) directly. - Mail original - De: "aderumier" À: "Stefan Priebe" Cc: "pve-devel" Envoyé: Jeudi 28 Mai 2015 13:36:25 Objet: Re: [pve-devel] qemu-server: cgroups && cpu.shares implement

Re: [pve-devel] Default cache mode for VM hard drives

2015-05-28 Thread Eneko Lacunza
On 28/05/15 15:01, Stanislav German-Evtushenko wrote: Note that without O_DIRECT you won't get a "correct" result either; disk may end not containing the data in the buffer when write was called. Softmirror data will be identically uncertain :) You are right. That is why I suppose there is a bug

Re: [pve-devel] qemu-server: cgroups && cpu.shares implementation

2015-05-28 Thread Alexandre DERUMIER
>>Another way could be use launch qemu with systemd-run, I think we can specify >>cgroup (slice in systemd) directly. for example: systemd-run --remain-after-exit --slice=qemu --unit=100 -p CPUShares=499 /usr/bin/kvm -id 100 Seem better than hacking qemu ? - Mail original - De

Re: [pve-devel] qemu-server: cgroups && cpu.shares implementation

2015-05-28 Thread Alexandre DERUMIER
better with scope: systemd-run --scope --slice=qemu --unit 100 -p CPUShares=499 /usr/bin/kvm -id 100 ... like this, cgroup is autoremoved on process stop. - Mail original - De: "aderumier" À: "dietmar" Cc: "pve-devel" Envoyé: Jeudi 28 Mai 2015 15:21:10 Objet: Re: [pve-devel] qemu-

Re: [pve-devel] Default cache mode for VM hard drives

2015-05-28 Thread Stanislav German-Evtushenko
On Thu, May 28, 2015 at 4:17 PM, Eneko Lacunza wrote: > On 28/05/15 15:01, Stanislav German-Evtushenko wrote: >>> >>> Note that without O_DIRECT you won't get a "correct" result either; disk >>> may end not containing the data in the buffer when write was called. >>> Softmirror data will be identi

Re: [pve-devel] Default cache mode for VM hard drives

2015-05-28 Thread Wolfgang Bumiller
> On May 28, 2015 at 3:17 PM Eneko Lacunza wrote: > On 28/05/15 15:01, Stanislav German-Evtushenko wrote: > >> Note that without O_DIRECT you won't get a "correct" result either; disk > >> may end not containing the data in the buffer when write was called. > >> Softmirror data will be identically

[pve-devel] [PATCH 3/3] remove old openvz fairscheduler code

2015-05-28 Thread Alexandre Derumier
Signed-off-by: Alexandre Derumier --- PVE/QemuServer.pm | 87 --- 1 file changed, 87 deletions(-) diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm index fe40140..94b9176 100644 --- a/PVE/QemuServer.pm +++ b/PVE/QemuServer.pm @@ -78,78 +78,6 @

[pve-devel] [PATCH 2/3] implement hotplug for cpuunits

2015-05-28 Thread Alexandre Derumier
Signed-off-by: Alexandre Derumier --- PVE/QemuServer.pm | 15 +++ 1 file changed, 15 insertions(+) diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm index 3cd4475..fe40140 100644 --- a/PVE/QemuServer.pm +++ b/PVE/QemuServer.pm @@ -67,6 +67,17 @@ PVE::JSONSchema::register_standard_o

Re: [pve-devel] Default cache mode for VM hard drives

2015-05-28 Thread Eneko Lacunza
On 28/05/15 15:32, Stanislav German-Evtushenko wrote: What does it mean that operations with buffer are not ensured to be thread-safe in qemu? O_DIRECT doesn't guarantee that buffer reading is finished when write returns if I read "man -s 2 open" correctly. The statement seems to be not correc

[pve-devel] [PATCH 1/3] implement cgroups through systemd-run

2015-05-28 Thread Alexandre Derumier
Signed-off-by: Alexandre Derumier --- PVE/QemuServer.pm | 9 + 1 file changed, 9 insertions(+) diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm index 22ff875..3cd4475 100644 --- a/PVE/QemuServer.pm +++ b/PVE/QemuServer.pm @@ -2618,6 +2618,15 @@ sub config_to_command { my $hotplug_

[pve-devel] qemu-server : cgroups implementation through systemd-run

2015-05-28 Thread Alexandre Derumier
I have tested it, it's working fine. (I don't known how HA will be managed but it should work too) ___ pve-devel mailing list pve-devel@pve.proxmox.com http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH] working towards fixin non-root builds

2015-05-28 Thread Wolfgang Bumiller
This first patch should still work with the existing scripts and allow us to gradually change the startup of our daemon scripts so that they do not try to perform tasks that need root before handling the autogenerated build-time utiliity commands like 'printmanpod'. We can then move mkdirs, chown

[pve-devel] [PATCH] defer some daemon setup routines

2015-05-28 Thread Wolfgang Bumiller
A first step towards untangling some of the intermingled data and functionality setup tasks for the daemons: Daemon::new now only validates and untaints arguments, but doesn't perform any actions such as setuid/setgid until the new Daemon::setup method which is now executed from Daemon::start righ

[pve-devel] [PATCH] defer some daemon setup routines

2015-05-28 Thread Wolfgang Bumiller
A first step towards untangling some of the intermingled data and functionality setup tasks for the daemons: Daemon::new now only validates and untaints arguments, but doesn't perform any actions such as setuid/setgid until the new Daemon::setup method which is now executed from Daemon::start righ

[pve-devel] [PATCH v2] working towards fixing non-root builds

2015-05-28 Thread Wolfgang Bumiller
Okay things are running again. I noticed the first git --amend didn't make it into the previous patches either so here's the updated and working patch. Wolfgang Bumiller (1): defer some daemon setup routines src/PVE/CLIHandler.pm | 4 +- src/PVE/Daemon.pm | 116 +-

Re: [pve-devel] qemu-server: cgroups && cpu.shares implementation

2015-05-28 Thread Dietmar Maurer
> Another way could be use launch qemu with systemd-run, I think we can specify > cgroup (slice in systemd) directly. What would be the advantage? Fr me that just makes things more complex? ___ pve-devel mailing list pve-devel@pve.proxmox.com http://pve

Re: [pve-devel] qemu-server: cgroups && cpu.shares implementation

2015-05-28 Thread Dietmar Maurer
> systemd-run --scope --slice=qemu --unit 100 -p CPUShares=499 /usr/bin/kvm -id > 100 ... > > > like this, cgroup is autoremoved on process stop. Interesting. I need toö take a look at the code in 'systemd-run' to see how complex that is. Are there any disadvantages? ___

Re: [pve-devel] Default cache mode for VM hard drives

2015-05-28 Thread Dietmar Maurer
> This is not okay and this is what is actually happening: > 0. set_buffer > 1. start_writing_with_o_direct_from_buffer > 2. change_buffer (we can do this only in another thread) > 3. finish_writing_with_o_direct_from_buffer > 4. change_buffer > 5. start_writing_with_o_direct_from_buffer > 6. chang

Re: [pve-devel] Default cache mode for VM hard drives

2015-05-28 Thread Dietmar Maurer
> > But there is currently only one io-thread in qemu, so this > > cannot happen with qemu if above is the only problem? > > But there are other threads, right? Buffer can be changed by another > thread where guest OS itself is running. No, AFAIK there is only one thread doing all IO (currently).

Re: [pve-devel] Default cache mode for VM hard drives

2015-05-28 Thread Stanislav German-Evtushenko
On Thu, May 28, 2015 at 6:35 PM, Dietmar Maurer wrote: >> This is not okay and this is what is actually happening: >> 0. set_buffer >> 1. start_writing_with_o_direct_from_buffer >> 2. change_buffer (we can do this only in another thread) >> 3. finish_writing_with_o_direct_from_buffer >> 4. change_

Re: [pve-devel] Default cache mode for VM hard drives

2015-05-28 Thread Dietmar Maurer
> If you provide a buffer to the kernel, that you change while it is > working with it, I don't know why you expect a reliable/predictable > result? Specially (but not only) if you tell it not to make a copy!! > > Note that without O_DIRECT you won't get a "correct" result either; disk > may en

Re: [pve-devel] [PATCH v2] working towards fixing non-root builds

2015-05-28 Thread Dietmar Maurer
applied. But we should implement/use the new functionality asap to see if everything works as expecteced. On 05/28/2015 04:54 PM, Wolfgang Bumiller wrote: Okay things are running again. I noticed the first git --amend didn't make it into the previous patches either so here's the updated and work

Re: [pve-devel] pve-devel Digest, Vol 60, Issue 45

2015-05-28 Thread Stanislav German-Evtushenko
> Message: 9 > Date: Thu, 28 May 2015 17:47:54 +0200 (CEST) > From: Dietmar Maurer > To: Stanislav German-Evtushenko > Cc: pve-devel > Subject: Re: [pve-devel] Default cache mode for VM hard drives > Message-ID: > <851217848.227.1432828074143.javamail.open-xcha...@ronja.mits.lan> > Conte

Re: [pve-devel] Default cache mode for VM hard drives

2015-05-28 Thread Stanislav German-Evtushenko
On Thu, May 28, 2015 at 6:47 PM, Dietmar Maurer wrote: >> > But there is currently only one io-thread in qemu, so this >> > cannot happen with qemu if above is the only problem? >> >> But there are other threads, right? Buffer can be changed by another >> thread where guest OS itself is running. >

Re: [pve-devel] Default cache mode for VM hard drives

2015-05-28 Thread Dietmar Maurer
> Each kvm process have multiple threads and the number of them is > changing in time. AFAIK all disk IO is done by a single, dedicated thread. ___ pve-devel mailing list pve-devel@pve.proxmox.com http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-deve

Re: [pve-devel] Default cache mode for VM hard drives

2015-05-28 Thread Stanislav German-Evtushenko
On Thu, May 28, 2015 at 7:19 PM, Dietmar Maurer wrote: >> Each kvm process have multiple threads and the number of them is >> changing in time. > > AFAIK all disk IO is done by a single, dedicated thread. I tried to read qemu-kvm code but it is difficult for me as I have never written C code. Wha

Re: [pve-devel] Default cache mode for VM hard drives

2015-05-28 Thread Dietmar Maurer
> On Thu, May 28, 2015 at 7:19 PM, Dietmar Maurer wrote: > >> Each kvm process have multiple threads and the number of them is > >> changing in time. > > > > AFAIK all disk IO is done by a single, dedicated thread. > > I tried to read qemu-kvm code but it is difficult for me as I have > never wri

Re: [pve-devel] qemu-server: cgroups && cpu.shares implementation

2015-05-28 Thread Alexandre DERUMIER
>>Interesting. I need toö take a look at the code in 'systemd-run' to see how >>complex that is. Are there any disadvantages? I don't see any disadvantages. from systemd-run: " If a command is run as transient scope unit, it will be started directly by systemd-run and thus inherit the execution

Re: [pve-devel] Default cache mode for VM hard drives

2015-05-28 Thread Stanislav German-Evtushenko
> Your test code was a really interesting example, and lead to > really unexpected results (at least for me). But as Eneko already > mentioned, nobody would write such code. It is simply not thread save, > and I think qemu does it correctly. I have written that code only because nobody wanted to t

Re: [pve-devel] Default cache mode for VM hard drives

2015-05-28 Thread Dietmar Maurer
> However now I think it is going to be even easier to reproduce in a > VM. My guess if you install a VM with virtual drive on DRBD or MD RAID > and cache=none, create ext3 or ext4 partition inside this VM and run > my code inside then you will get inconsistence. May be you need to run > it not onc

Re: [pve-devel] Default cache mode for VM hard drives

2015-05-28 Thread Stanislav German-Evtushenko
On May 28, 2015 8:12 PM, "Stanislav German-Evtushenko" gmail.com > wrote: > > > Your test code was a really interesting example, and lead to > > really unexpected results (at least for me). But as Eneko already > > mentioned, nobody would write such code. It is simply not thread save, > > and I thi

Re: [pve-devel] High Performance SSH

2015-05-28 Thread Andreas Steinel
A little bit off-topic, but there is hope: This week, I almost saturated a 1 Gbit network link between two brand new Dell Servers with 3.2 GHz Xeon E5-2667v3 CPUs. I got 105 MB/sec using standard SSH/SCP. So we finally have single-thread-performance that is fast enough for encryption on gigabit. P

Re: [pve-devel] qemu-server: cgroups && cpu.shares implementation

2015-05-28 Thread Dietmar Maurer
> So it's like a temp service management from systemd. > > Also as advantage, is it also possible to catch qemu crash with journalctl ? What do you want to do? Restart if crached? ___ pve-devel mailing list pve-devel@pve.proxmox.com http://pve.proxmox.

Re: [pve-devel] qemu-server : cgroups implementation through systemd-run

2015-05-28 Thread Dietmar Maurer
applied, thanks! On 05/28/2015 03:59 PM, Alexandre Derumier wrote: I have tested it, it's working fine. (I don't known how HA will be managed but it should work too) ___ pve-devel mailing list pve-devel@pve.proxmox.com http://pve.proxmox.com/cgi-bin/

Re: [pve-devel] qemu-server : cgroups implementation through systemd-run

2015-05-28 Thread Dietmar Maurer
Are there other things we can set this way? For example can we limit CPU with cfs_period_us and cfs_quota_us? I guess that would be interesting for hosters. On 05/28/2015 03:59 PM, Alexandre Derumier wrote: I have tested it, it's working fine. (I don't known how HA will be managed but it sho

Re: [pve-devel] qemu-server : cgroups implementation through systemd-run

2015-05-28 Thread Dietmar Maurer
Answering myself, I found we can set this with CPUQuota=XX% So we can directly map this to the 'cpulimit' parameter? On 05/29/2015 08:33 AM, Dietmar Maurer wrote: Are there other things we can set this way? For example can we limit CPU with cfs_period_us and cfs_quota_us? I guess that woul