[Aoetools-discuss] aoe6-79 for kernel 3.2.0 / ubuntu 12.04 LTS
Hi there,
maybe someone is interested.
I tried to install aoe6-79 via dkms into my ubuntu 12.04.
The tarball doesn't compile out of the box under kernel version 3.2.0
So I patched the source till it compiles.
Here is my diff:
diff -pur aoe6-79.orig/linux/drivers/block/aoe/aoeblk.c
aoe6-79/linux/drivers/block/aoe/aoeblk.c
--- aoe6-79.orig/linux/drivers/block/aoe/aoeblk.c 2012-02-09
21:43:04.0 +0100
+++ aoe6-79/linux/drivers/block/aoe/aoeblk.c2012-04-26 11:35:31.931975631
+0200
@@ -10,6 +10,8 @@
#include
#include
#include
+#include
+#include
#include
#include "aoe.h"
#include "disk_attr.h"
diff -pur aoe6-79.orig/linux/drivers/block/aoe/aoechr.c
aoe6-79/linux/drivers/block/aoe/aoechr.c
--- aoe6-79.orig/linux/drivers/block/aoe/aoechr.c 2012-02-09
21:43:04.0 +0100
+++ aoe6-79/linux/drivers/block/aoe/aoechr.c2012-04-26 11:35:41.879975413
+0200
@@ -9,6 +9,7 @@
#include
#include
#include
+#include
#include
#include "aoe.h"
diff -pur aoe6-79.orig/linux/drivers/block/aoe/aoemain.c
aoe6-79/linux/drivers/block/aoe/aoemain.c
--- aoe6-79.orig/linux/drivers/block/aoe/aoemain.c 2012-02-09
21:43:04.0 +0100
+++ aoe6-79/linux/drivers/block/aoe/aoemain.c 2012-04-26 12:15:13.967923384
+0200
@@ -15,9 +15,7 @@
MODULE_LICENSE("GPL");
MODULE_AUTHOR("Sam Hopkins ");
MODULE_DESCRIPTION("AoE block/char driver for 2.6.2 and newer 2.6 kernels");
-/* this creates warning "Warning: could not find versions for
.tmp_versions/aoe.mod"
MODULE_VERSION(VERSION);
-*/
/* modinfo sometimes works, but this will be in sysfs */
static char version[] = VERSION;
And my /usr/src/aoe6-79/dkms.conf is:
==
PACKAGE_NAME="aoe6"
PACKAGE_VERSION="79"
AUTOINSTALL="yes"
REMAKE_INITRD="yes"
MAKE[0]="make KDIR=${kernel_source_dir}
INSTDIR=/lib/modules/${kernelver}/drivers/block/aoe"
BUILT_MODULE_NAME[0]="aoe"
BUILT_MODULE_LOCATION[0]="linux/drivers/block/aoe/"
DEST_MODULE_LOCATION[0]="/kernel/drivers/block/aoe"
==
You can add and build the module with
# dkms build aoe6/79
and install with
# dkms install aoe6/79
It seems to work for me.
Lars
--
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
___
Aoetools-discuss mailing list
Aoetools-discuss@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/aoetools-discuss
Re: [Aoetools-discuss] AoE blog article
On Wed, Apr 25, 2012 at 01:47:23PM +0200, Lars Täuber spake thusly: > Tracy Reed schrieb: > > Reasons why I am currently migrating away from AoE to iSCSI (*sigh*): > > > > 1. Disk alignment between Xen VMs and the target. > > What do you mean by that? I mean that when I configure a Xen VM to use an AoE block device I always had mis-aligned writes. The Xen dom0 has the block device in /dev/etherd and I put the block device in the Xen VM config file and make it /dev/xvda inside the VM. If I access the device from dom0 everything is fine. Very fast writes, no misalignment. But accessing the block device from within the VM causes the problem. This makes no sense to me and I don't see anything that could cause alignment to change but it clearly did somehow. This got to be very noticeable performance-wise and when doing a pure-write benchmark while running iostat on the target I could see lots of reads happening to backfill partial pages due to the misaligned write. -- Tracy Reed pgp9Cv6KCRmlg.pgp Description: PGP signature -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/___ Aoetools-discuss mailing list Aoetools-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/aoetools-discuss
Re: [Aoetools-discuss] AoE blog article
Den 26/04/2012 kl. 13.02 skrev Tracy Reed: >>> >>> 1. Disk alignment between Xen VMs and the target. >> >> What do you mean by that? > > I mean that when I configure a Xen VM to use an AoE block device I always had > mis-aligned writes. The Xen dom0 has the block device in /dev/etherd and I put > the block device in the Xen VM config file and make it /dev/xvda inside the > VM. > > If I access the device from dom0 everything is fine. Very fast writes, no > misalignment. But accessing the block device from within the VM causes the > problem. This makes no sense to me and I don't see anything that could cause > alignment to change but it clearly did somehow. This got to be very noticeable > performance-wise and when doing a pure-write benchmark while running iostat on > the target I could see lots of reads happening to backfill partial pages due > to > the misaligned write. FYI: Using XenServer 5.6 SP2 with AoE-targets (both coraid appliances and ggaoed) I get the same diskspeeds in XenServer dom0 and VM domU. So /Lars -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Aoetools-discuss mailing list Aoetools-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/aoetools-discuss
Re: [Aoetools-discuss] AoE blog article
Very interesting answer, thank you. Partition alignment, and IO performance in general, is something I'm trying to get better grips on. FWIW, we're on AoE 47 (stock Debian kernel), with AoE clients using two AoE devices as a backing device for a RAID1 device. It works well enough, but performance isn't exactly stellar. I have some questions that would be grateful to get the answer to. * How will using iSCSI change the partition alignment problem ? I'm guessing you have found the difference the only way that works; by trying. It seems unclear why AoE would cause the unaligned writes, so maybe there isn't a good answer to this question. * How do you actually see from the output of iostat that you're writes aren't aligned ? I usually use "iostat -dxm 1" to get an overview, but I don't fully understand all the output. In a non-AoE context, I have a dm RAID6, where I see all the disks and the md device when using iostat. I can there see the md device hitting 100% utilization, while the disk don't go above ~60%. Is that a symptom of unaligned writes ? Thanks for your insightful answers. On Thu, Apr 26, 2012 at 13:02, Tracy Reed wrote: > On Wed, Apr 25, 2012 at 01:47:23PM +0200, Lars Täuber spake thusly: >> Tracy Reed schrieb: >> > Reasons why I am currently migrating away from AoE to iSCSI (*sigh*): >> > >> > 1. Disk alignment between Xen VMs and the target. >> >> What do you mean by that? > > I mean that when I configure a Xen VM to use an AoE block device I always had > mis-aligned writes. The Xen dom0 has the block device in /dev/etherd and I put > the block device in the Xen VM config file and make it /dev/xvda inside the > VM. > > If I access the device from dom0 everything is fine. Very fast writes, no > misalignment. But accessing the block device from within the VM causes the > problem. This makes no sense to me and I don't see anything that could cause > alignment to change but it clearly did somehow. This got to be very noticeable > performance-wise and when doing a pure-write benchmark while running iostat on > the target I could see lots of reads happening to backfill partial pages due > to > the misaligned write. > > -- > Tracy Reed > > -- > Live Security Virtual Conference > Exclusive live event will cover all the ways today's security and > threat landscape has changed and how IT managers can respond. Discussions > will include endpoint security, mobile security and the latest in malware > threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ > ___ > Aoetools-discuss mailing list > Aoetools-discuss@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/aoetools-discuss > -- Vennlig hilsen Torbjørn Thorsen Utvikler / driftstekniker Trollweb Solutions AS - Professional Magento Partner www.trollweb.no Telefon dagtid: +47 51215300 Telefon kveld/helg: For kunder med Serviceavtale Besøksadresse: Luramyrveien 40, 4313 Sandnes Postadresse: Maurholen 57, 4316 Sandnes Husk at alle våre standard-vilkår alltid er gjeldende -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Aoetools-discuss mailing list Aoetools-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/aoetools-discuss
Re: [Aoetools-discuss] aoe6-79 for kernel 3.2.0 / ubuntu 12.04 LTS
Hall! Something went wrong, because I don't have the device files /dev/etherd/discover /dev/etherd/interface /dev/etherd/flush /dev/etherd/revalidate with version v79 any more. And I made some simple speed tests with dd and the new and old version of aoe. There is no significant difference with my setup. On the target side it's ggaoed servicing an 128MiB ramdisk. Here the summary: dmesg: e1000e: eth1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None # /usr/src/aoe/aoetools-33/aoe-sancheck Probing...done. == INTERFACE SUMMARY == NameStatus MTU PCI ID eth1UP 90008086:10d3 == DEVICE SUMMARY == Device MacsPayload Local Interfaces e3.1 18704eth1 e3.2 18704eth1 writing with dd (oflag=direct) from local file to ramdisk via aoe (ggaoed) e.g. dd if=/local/file of=/dev/etherd/e3.2 bs=1M oflag=direct reading from ramdisk via aoe to /dev/null dd if=/dev/etherd/e3.2 of=/dev/null writing bs=512 bs=8192 bs=1M reading [MB/s] aoe v47: ~3.3~32 ~113 ~120 aoe v79: ~3.2~18 ~115 ~120 So I'll revert back to v47. How fast are your setups? Lars -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Aoetools-discuss mailing list Aoetools-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/aoetools-discuss
Re: [Aoetools-discuss] AoE blog article
Am Thu, 26 Apr 2012 13:17:40 +0200 Lars Erik Dangvard Jensen schrieb: > FYI: > Using XenServer 5.6 SP2 with AoE-targets (both coraid appliances and > ggaoed) I get the same diskspeeds in XenServer dom0 and VM domU. This is my experience too. But with xen_major : 4 xen_minor : 0 xen_extra : .2_52-0.2.1 xen_caps : xen-3.0-x86_64 xen-3.0-x86_32p aoe v47 1GBit => 10GBit ggaoed offering LVs on DRBD on RAID6s Lars -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Aoetools-discuss mailing list Aoetools-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/aoetools-discuss
Re: [Aoetools-discuss] AoE blog article
On Thu, Apr 26, 2012 at 01:17:43PM +0200, Torbjørn Thorsen spake thusly: > * How will using iSCSI change the partition alignment problem ? > I'm guessing you have found the difference the only way that works; by trying. > It seems unclear why AoE would cause the unaligned writes, so maybe there > isn't > a good answer to this question. I presume it changes the partition alignment problem but not actually introducing one. I don't know how it is possible that an alignment problem is being introduced by using a VM. It makes no sense to me. But the performance is what it is. And when doing writes I see lots of corresponding reads indicative of an alignment problem. And it isn't necessary a partition alignment problem: I'm pretty sure that at some point I actually exported a raw unpartitioned hard drive into then Xen VM and still had issues. It almost seems like Xen itself is causing it. > * How do you actually see from the output of iostat that you're writes > aren't aligned ? > I usually use "iostat -dxm 1" to get an overview, but I don't fully > understand all the output. I run iostat and watch the "rrqm/s wrqm/s r/s w/s" columns. If I am inside the VM and I do a: dd if=/dev/zero of=testfile bs=4M then in the iostat output I would expect to see a lot of write requests merged (wrqm/s) and writes (w/s) per second. I would expect to see few if any reads. Instead I see a significant number of reads perhaps totalling around 1/4 as many writes. This causes lots of extra seeking which really slows down IO. > In a non-AoE context, I have a dm RAID6, where I see all the disks and > the md device > when using iostat. > I can there see the md device hitting 100% utilization, while the disk > don't go above ~60%. > Is that a symptom of unaligned writes ? Not necessarily. The main indicator of unaligned writes is seeing a lot of reads when only writes should be happening. -- Tracy Reed pgpr0UQ5TOEvP.pgp Description: PGP signature -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/___ Aoetools-discuss mailing list Aoetools-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/aoetools-discuss
Re: [Aoetools-discuss] AoE blog article
On Thu, Apr 26, 2012 at 03:46:17PM +0200, Lars Täuber spake thusly: > 10GBit ggaoed offering LVs on DRBD on RAID6s I have always used the vblade from Coraid, not ggaoed. I would give it a try but I really don't want to invest any more time in this issue and whether ggaoed is going to be supported long-term. -- Tracy Reed Digital signature attached for your safety. CopilotcoProfessionally Managed PCI Compliant Secure Hosting 866-MY-COPILOT x101 http://copilotco.com pgpktaukxW9Jz.pgp Description: PGP signature -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/___ Aoetools-discuss mailing list Aoetools-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/aoetools-discuss
