Re: [pve-devel] [PATCH] iothreads : create 1 iothread by virtio device
> I going to have a full ssd ceph cluster next month (18x s 3500 on 3 hosts), > with more powerfull nodes and more powerfull clients (20 cores 3,1Gghz) > > I'll do benchmarks and post results with differents setup. Would be interesting to additionally benchmark with sheepdog, so that we can compare ;-) ___ pve-devel mailing list pve-devel@pve.proxmox.com http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] [PATCH] bump to 0.9 v2
>>I just updated to 0.9.1, which is latest stable release. Or is there a reason >>to keep 0.9.0? I just miss the release announcement. >>If not, please can you test 0.9.1? I'll test it today - Mail original - De: "dietmar" À: "aderumier" , "pve-devel" Envoyé: Mercredi 28 Janvier 2015 06:38:36 Objet: Re: [pve-devel] [PATCH] bump to 0.9 v2 > > + * Bump to sheepdog 0.9.0 > > Is the sheepdog storage format considered stable now? Any information about > that? I just updated to 0.9.1, which is latest stable release. Or is there a reason to keep 0.9.0? If not, please can you test 0.9.1? ___ pve-devel mailing list pve-devel@pve.proxmox.com http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] qemu-server : memory hotplug patch v10
>>Yes. Please can you try to make a patch for that?
Yes, sure.
I'll try to send patches today or tomorrow
- Mail original -
De: "dietmar"
À: "aderumier" , "pve-devel"
Envoyé: Mercredi 28 Janvier 2015 06:56:49
Objet: Re: [pve-devel] qemu-server : memory hotplug patch v10
> (Note that I think that hotplug->{memory} itself shouldn't be "hotpluggagle",
> because we need to restart the vm in this case to get support for dimm
> memory)
Yes. Please can you try to make a patch for that?
___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] [PATCH] iothreads : create 1 iothread by virtio device
>>On our SSD pool which has 24x Intel S3700 across 4 hosts, the most we can get >>inside a guest using virtio-scsi is ~9000IOPS @ 4K size. >>In guest CPU usage is around 7% yet the guest is using 100% for each >>allocated core on the host, which makes me think that the multiple IO threads >>patch will unleash the performance for us. Hi Andrew, well Indeed, I have notice same result than you. )librbd is really cpu intensive, and qemu use 1global thread by default for all ios. That mean that your 9000iops is related to the cpu of 1core.( the more cpu frequency you'll have, the more iops you'll get). (you can add 10 disk to the vm, you'll 1 thread for all disks) So, I solution is to use the krbd kernel module. I have sent a patch to the mailing some time ago, not yet in master. with krbd kernel module, I can reach around 4iops, with default qemu without iothread. http://pve.proxmox.com/pipermail/pve-devel/2014-November/013175.html (krbd use 4x less cpu than librbd, I have sent a bug report to inktank, they are looking at it currently) Now, with librbd + iothread (note that currently it's works fine with virtio-blk, but for virtio-scsi it's not stable yet), you can use 1 thread for each disk. With 1 disk, I think I'm around 15000iops vs 9000iops. with X disks, I'm around X disks x 15000iops. For virtio-scsi, in my patches series, I have also support for multiqueues in virtio-scsi, which also help. This require a recent guest kernel. I have seen some improvement, for sequential reads for small blocks, which aggregate small blocks in bigger one, so less ios to ceph, less cpu. (1->5iops for this workload) Also, to reduce cpu usage and improve performance, you can improve the ceph.conf with debug lockdep = 0/0 debug context = 0/0 debug crush = 0/0 debug buffer = 0/0 debug timer = 0/0 debug journaler = 0/0 debug osd = 0/0 debug optracker = 0/0 debug objclass = 0/0 debug filestore = 0/0 debug journal = 0/0 debug ms = 0/0 debug monc = 0/0 debug tp = 0/0 debug auth = 0/0 debug finisher = 0/0 debug heartbeatmap = 0/0 debug perfcounter = 0/0 debug asok = 0/0 debug throttle = 0/0 cephx sign messages = false cephx require signatures = false From my benchmark, the more iops I was able to reach with 1 vm was. 3 x 4iops with 3 virtio-blk disk + iothreads + krbd This was will small cpu CPU E5-2603 v2 @ 1.80GHz. I going to have a full ssd ceph cluster next month (18x s 3500 on 3 hosts), with more powerfull nodes and more powerfull clients (20 cores 3,1Gghz) I'll do benchmarks and post results with differents setup. If you have time, maybe can you test my krbd patch ? (Note that discard support in not yet available in krbd in kernel 3.10) Hope It's help. Alexandre (I'm also going to try to test vhost-scsi feature of qemu soon, which is some kind of full passthrough, bypassing all emulation layers) - Mail original - De: "Andrew Thrift" À: "aderumier" Envoyé: Jeudi 29 Janvier 2015 02:55:11 Objet: Re: [pve-devel] [PATCH] iothreads : create 1 iothread by virtio device Hi Alexandre, Without this patch what IOPS do you get ? On our SSD pool which has 24x Intel S3700 across 4 hosts, the most we can get inside a guest using virtio-scsi is ~9000IOPS @ 4K size. In guest CPU usage is around 7% yet the guest is using 100% for each allocated core on the host, which makes me think that the multiple IO threads patch will unleash the performance for us. Regards, Andrew On Fri, Jan 16, 2015 at 8:43 AM, Dietmar Maurer < diet...@proxmox.com > wrote: > The patch series for block jobs was here > http://lists.gnu.org/archive/html/qemu-devel/2014-10/msg02191.html > > > (I'm not sure about the difference between proxmox backup patches and qemu > implemention) We have a bunch of patches above that (vma). ___ pve-devel mailing list pve-devel@pve.proxmox.com http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel ___ pve-devel mailing list pve-devel@pve.proxmox.com http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
[pve-devel] [PATCH] zfs: fixV2 wait by alloc_image
Signed-off-by: Wolfgang Link
---
PVE/Storage/ZFSPlugin.pm |9 -
PVE/Storage/ZFSPoolPlugin.pm |7 +++
2 files changed, 15 insertions(+), 1 deletion(-)
diff --git a/PVE/Storage/ZFSPlugin.pm b/PVE/Storage/ZFSPlugin.pm
index 581ef6c..d8acce9 100644
--- a/PVE/Storage/ZFSPlugin.pm
+++ b/PVE/Storage/ZFSPlugin.pm
@@ -265,7 +265,14 @@ sub clone_image {
sub alloc_image {
my ($class, $storeid, $scfg, $vmid, $fmt, $name, $size) = @_;
-my $volname = $class->SUPER::alloc_image($storeid, $scfg, $vmid, $fmt,
$name, $size);
+die "unsupported format '$fmt'" if $fmt ne 'raw';
+
+die "illegal name '$name' - sould be 'vm-$vmid-*'\n"
+if $name && $name !~ m/^vm-$vmid-/;
+
+my $volname = $class->zfs_find_free_diskname($storeid, $scfg, $vmid) if
!$name;
+
+$class->zfs_create_zvol($scfg, $name, $size);
my $guid = $class->zfs_create_lu($scfg, $volname);
$class->zfs_add_lun_mapping_entry($scfg, $volname, $guid);
diff --git a/PVE/Storage/ZFSPoolPlugin.pm b/PVE/Storage/ZFSPoolPlugin.pm
index 10da7f7..490b61c 100644
--- a/PVE/Storage/ZFSPoolPlugin.pm
+++ b/PVE/Storage/ZFSPoolPlugin.pm
@@ -184,6 +184,13 @@ sub alloc_image {
$name = $class->zfs_find_free_diskname($storeid, $scfg, $vmid) if !$name;
$class->zfs_create_zvol($scfg, $name, $size);
+run_command ("udevadm trigger --subsystem-match block");
+run_command ("udevadm settle --timeout 5");
+
+for (1..10) {
+ last if -e "/dev/zvol/$scfg->{pool}/$name" ;
+ Time::HiRes::usleep(100);
+}
return $name;
}
--
1.7.10.4
___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
[pve-devel] [PATCH] Fix: disable root
From: Wolfgang Link
root can now be disabled in GUI.
Signed-off-by: Wolfgang Link
---
PVE/AccessControl.pm | 10 --
1 file changed, 4 insertions(+), 6 deletions(-)
diff --git a/PVE/AccessControl.pm b/PVE/AccessControl.pm
index db85d08..a1b4971 100644
--- a/PVE/AccessControl.pm
+++ b/PVE/AccessControl.pm
@@ -356,8 +356,6 @@ sub check_user_enabled {
return 1 if $data->{enable};
-return 1 if $username eq 'root@pam'; # root is always enabled
-
die "user '$username' is disabled\n" if !$noerr;
return undef;
@@ -695,10 +693,10 @@ sub userconfig_force_defaults {
$cfg->{roles}->{$r} = $special_roles->{$r};
}
-# fixme: remove 'root' group (not required)?
-
-# add root user
-$cfg->{users}->{'root@pam'}->{enable} = 1;
+# add root user if not exists
+if (!$cfg->{users}->{'root@pam'}) {
+ $cfg->{users}->{'root@pam'}->{enable} = 1;
+}
}
sub parse_user_config {
--
1.7.10.4
___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] [PATCH] memory hotplug patch v10
> Is there already a release plan/date for qemu 2.3? answering myself: http://wiki.qemu.org/Planning/2.3 ___ pve-devel mailing list pve-devel@pve.proxmox.com http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] [PATCH] zfs: fix wait by alloc_image
> diff --git a/PVE/Storage/ZFSPoolPlugin.pm b/PVE/Storage/ZFSPoolPlugin.pm
> index 10da7f7..ed10484 100644
> --- a/PVE/Storage/ZFSPoolPlugin.pm
> +++ b/PVE/Storage/ZFSPoolPlugin.pm
> @@ -185,6 +185,13 @@ sub alloc_image {
>
> $class->zfs_create_zvol($scfg, $name, $size);
>
> +run_command ("udevadm trigger --subsystem-match block");
> +run_command ("udevadm settle --timeout 5");
Oh, we cannot do that, because ZFSPluging is also using this method:
sub alloc_image {
my ($class, $storeid, $scfg, $vmid, $fmt, $name, $size) = @_;
my $volname = $class->SUPER::alloc_image($storeid, $scfg, $vmid, $fmt,
$name, $size);
___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] [PATCH] memory hotplug patch v10
> A patch has been sent to qemu-mailing, but not yet applied > http://lists.gnu.org/archive/html/qemu-devel/2014-11/msg02050.html > > I don't known if we want to apply it on pve-qemu-kvm, or wait for qemu 2.3. > (I think both unplug and balloon fix will be ready for qemu 2.3) For me, current qemu is quite unstable. I get random errors in live migration and snapshot save code. I am just compiling actual qemu master to see if there are improvements. Is there already a release plan/date for qemu 2.3? ___ pve-devel mailing list pve-devel@pve.proxmox.com http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] [PATCH] zfs: fix wait by alloc_image
> +run_command ("udevadm trigger --subsystem-match block");
> +run_command ("udevadm settle --timeout 5");
>
reasonable
> +for (1..4) {
> + Time::HiRes::usleep(100);
> +}
but I do not understand this one?
___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
[pve-devel] [PATCH] Fix: disable root
From: Wolfgang Link
root can now be disabled in GUI.
Signed-off-by: Wolfgang Link
---
PVE/AccessControl.pm | 10 --
1 file changed, 4 insertions(+), 6 deletions(-)
diff --git a/PVE/AccessControl.pm b/PVE/AccessControl.pm
index db85d08..a1b4971 100644
--- a/PVE/AccessControl.pm
+++ b/PVE/AccessControl.pm
@@ -356,8 +356,6 @@ sub check_user_enabled {
return 1 if $data->{enable};
-return 1 if $username eq 'root@pam'; # root is always enabled
-
die "user '$username' is disabled\n" if !$noerr;
return undef;
@@ -695,10 +693,10 @@ sub userconfig_force_defaults {
$cfg->{roles}->{$r} = $special_roles->{$r};
}
-# fixme: remove 'root' group (not required)?
-
-# add root user
-$cfg->{users}->{'root@pam'}->{enable} = 1;
+# add root user if not exists
+if (!$cfg->{users}->{'root@pam'}) {
+ $cfg->{users}->{'root@pam'}->{enable} = 1;
+}
}
sub parse_user_config {
--
1.7.10.4
___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] [PATCH] memory hotplug patch v10
>>Please can you factor out the code to compute dimm sizes into a separate
>>method?
>>I don't want that code duplicated 2 times.
Yes, no problem, I thinked to do it this week.
also ballooning don't work correctly currently with dimm memory,
because total memory compute don't take dimm values.
A patch has been sent to qemu-mailing, but not yet applied
http://lists.gnu.org/archive/html/qemu-devel/2014-11/msg02050.html
I don't known if we want to apply it on pve-qemu-kvm, or wait for qemu 2.3.
(I think both unplug and balloon fix will be ready for qemu 2.3)
- Mail original -
De: "dietmar"
À: "aderumier" , "pve-devel"
Envoyé: Mercredi 28 Janvier 2015 07:09:59
Objet: Re: [pve-devel] [PATCH] memory hotplug patch v10
applied, thanks!
> + if ($hotplug_features->{memory}) {
> + my $dimm_id = 0;
> + my $dimm_size = 512;
> + my $current_size = $static_memory;
> + for (my $j = 0; $j < 8; $j++) {
> + for (my $i = 0; $i < 32; $i++) {
> + my $name = "dimm${dimm_id}";
> + $dimm_id++;
Please can you factor out the code to compute dimm sizes into a separate
method?
I don't want that code duplicated 2 times.
___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
[pve-devel] [PATCH] zfs: fix wait by alloc_image
Signed-off-by: Wolfgang Link
---
PVE/Storage/ZFSPoolPlugin.pm |7 +++
1 file changed, 7 insertions(+)
diff --git a/PVE/Storage/ZFSPoolPlugin.pm b/PVE/Storage/ZFSPoolPlugin.pm
index 10da7f7..ed10484 100644
--- a/PVE/Storage/ZFSPoolPlugin.pm
+++ b/PVE/Storage/ZFSPoolPlugin.pm
@@ -185,6 +185,13 @@ sub alloc_image {
$class->zfs_create_zvol($scfg, $name, $size);
+run_command ("udevadm trigger --subsystem-match block");
+run_command ("udevadm settle --timeout 5");
+
+for (1..4) {
+ Time::HiRes::usleep(100);
+}
+
return $name;
}
--
1.7.10.4
___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
