Re: [Aoetools-discuss] AoE blog article
On Wed, Apr 25, 2012 at 01:47:23PM +0200, Lars Täuber spake thusly: Tracy Reed tr...@ultraviolet.org schrieb: Reasons why I am currently migrating away from AoE to iSCSI (*sigh*): 1. Disk alignment between Xen VMs and the target. What do you mean by that? I mean that when I configure a Xen VM to use an AoE block device I always had mis-aligned writes. The Xen dom0 has the block device in /dev/etherd and I put the block device in the Xen VM config file and make it /dev/xvda inside the VM. If I access the device from dom0 everything is fine. Very fast writes, no misalignment. But accessing the block device from within the VM causes the problem. This makes no sense to me and I don't see anything that could cause alignment to change but it clearly did somehow. This got to be very noticeable performance-wise and when doing a pure-write benchmark while running iostat on the target I could see lots of reads happening to backfill partial pages due to the misaligned write. -- Tracy Reed pgp9Cv6KCRmlg.pgp Description: PGP signature -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/___ Aoetools-discuss mailing list Aoetools-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/aoetools-discuss
Re: [Aoetools-discuss] AoE blog article
On Tue, Apr 24, 2012 at 03:50:33PM +, Jeff Sturm spake thusly: (AoE multipath really demonstrates the advantages of a simple protocol--there's nothing involved in making it work other than plugging in additional interfaces.) Agree completely. Simplicity has always been one of the primary reasons I used AoE for so long. Reasons why I am currently migrating away from AoE to iSCSI (*sigh*): 1. Disk alignment between Xen VMs and the target. I've never figured it out and got it working reliably. I've played with partitioning and disk geometry and partition offsets and all kinds of things. I've just never made it work properly and can't pay the performance penalty. Direct machine-to-machine AoE is blazing fast and typically faster than iSCSI. But my use case has always been VMs, from the very beginning of my initial use of AoE. I've really only used it outside of VMs just once. 2. Lack of integration with RHEL/CentOS. iSCSI gets all the work. I can write init scripts and fix this stuff myself if necessary, and have to some degree, it's just more work. Multipath was important to us for reliability, so the loss of a single switch would not impact the storage device. I've been using LACP but with multipath you can span switches without needing fancy expensive stacking switches which is pretty cool. I really should consider whether I want to go multipath with iSCSI. -- Tracy Reed pgp72iBSsRUb5.pgp Description: PGP signature -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/___ Aoetools-discuss mailing list Aoetools-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/aoetools-discuss
[Aoetools-discuss] AoE blog article
Here's a good blog article on some of the current problems with AoE that are killing its adoption and therefore utility to potential Coraid customers: http://www.typinganimal.net/wp/2012/04/16/red-hat-just-doesnt-get-aoe/ -- Tracy Reed pgpHVHTwd28x9.pgp Description: PGP signature -- For Developers, A Lot Can Happen In A Second. Boundary is the first to Know...and Tell You. Monitor Your Applications in Ultra-Fine Resolution. Try it FREE! http://p.sf.net/sfu/Boundary-d2dvs2 ___ Aoetools-discuss mailing list Aoetools-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/aoetools-discuss
Re: [Aoetools-discuss] Project alive?
On Wed, Apr 18, 2012 at 08:54:21AM -0600, Joshua J. Kugler spake thusly: So the last release for aoetools was 2010-09-21, and the last release for kvblade was 2006-10-02. Are these public projects still maintained? Is there any progress/maintenance on kvblade? It is pretty much feature complete and there are no showstopper bugs that I am aware of. So there hasn't been much work done on it. I'm don't know what Coraid's long-term plans for it are. -- Tracy Reed pgpfLNd456qfi.pgp Description: PGP signature -- Better than sec? Nothing is better than sec when it comes to monitoring Big Data applications. Try Boundary one-second resolution app monitoring today. Free. http://p.sf.net/sfu/Boundary-dev2dev___ Aoetools-discuss mailing list Aoetools-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/aoetools-discuss
Re: [Aoetools-discuss] Throughput for raw AoE device versus filesystem
On Tue, Jul 05, 2011 at 05:03:40PM +0200, Torbjørn Thorsen spake thusly: I'm setting up a AoE-based SAN, and I'm not quite sure I've reached a good performance level. I can read and write the raw AoE device (/dev/etherd/*) at more or less line-speed on my 1gig Ethernet adapters. This means I'm seeing I/O rates of 100 to 120 MB/s when using dd or something similar. This is in line with what I get also. Sounds like your performance level is as expected (very good). However, when I put a filesystem on there, I'm seeing rates of 55 to 70 MB/s. I've tested mostly by using rsync, cp or dd, but I tried bonnie and saw much the same results. Yep. You are most likely running into physical limitations of the disk. Since I'm seeing line-speed when using the device directly, I guess this means that the configuration is more or less okay. Yep. What kind of performance are you guys seeing on your filesystems when using 1gig Ethernet adapters ? The speed of the network is not nearly as important as the speed of the disk hardware. I get performance similar to yours when doing streaming reads/writes to at least two disks. A single 7200rpm drive can typically do 70MB/s so you usually need to gang up at least two of these in a mirror or stripe. Many more smaller disks are necessary for higher IOPS. Fortunately, this is a problem completely independent of AoE so lots of people know how to solve it. These days I deploy SuperMicro 24 bay 2.5 servers stuffed full of 10k RPM disks. This seems to get me the most reasonable bang/buck while providing the kind of IOPS I need to run databases, mail servers, etc. The giant/cheap 2T disks you can buy these days are great for archival and backup storage but for actual data processing the advice has been the same for many years: Throw lots of spindles at the problem. -- Tracy Reed Digital signature attached for your safety. CopilotcoProfessionally Managed PCI Compliant Secure Hosting 866-MY-COPILOT x101 http://copilotco.com pgpy50WeWKhxL.pgp Description: PGP signature -- All of the data generated in your IT infrastructure is seriously valuable. Why? It contains a definitive record of application performance, security threats, fraudulent activity, and more. Splunk takes this data and makes sense of it. IT sense. And common sense. http://p.sf.net/sfu/splunk-d2d-c2___ Aoetools-discuss mailing list Aoetools-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/aoetools-discuss
Re: [Aoetools-discuss] AoE seems slow compared to iSCSI, what am I doing wrong ?
On Mon, May 09, 2011 at 05:53:49PM +0200, Torbjørn Thorsen spake thusly: As I applied all your advice at the same time, I'm not quite sure what made the biggest impact Oops...And I almost mentioned be sure to do each one at a time then test so we would know which one did it. :) The result was that my AoE benchmarks now outperform the iSCSI alternative. Excellent, just as it should be! Just out of curiosity, by how much does AoE outperform iSCSI? iSCSI might be made to go a bit faster if as much effort were put into optimizing it as we have AoE. I'm using HP ProCurve 2510s. That's what I use also. Flow control in the switch was not enabled, and I can't remember seeing that one mentioned too many places either. Yeah, it helps the switch out a lot. The only time you should not use flow control is if you are using one of either VLAN tagging or bonding, I forget which and why. Wouldn't you know, I had not included the respective ports in the jumbo-enabled VLAN. After enabling the ports on the switch, I verified I had a 9000 MTU by using ping with the prohibit fragmentation option enabled and sending frames bigger than 1500 bytes. Yep, that would do it. As for data safety, should I consider running vblade with the O_SYNC option enabled ? You might want to give it a try but I have found that it hurt performance unacceptably unless you have a hardware RAID controller with battery backed cache. Using O_SYNC works out well in that case because the controller will cache for you reliably and not lose the data in a power failure. I will be exporting block devices to be used as the filesystem for Xen instances, and I wouldn't want to lie to MySQL living in Xen about data being on disk. That is exactly what I do also. Using the raw device here as well, I don't see the need for partitioning at this level. Good. Glad you got your performance issue fixed. I have strugged with many myself over nearly 6 years of using AoE and I what I gave you is my standard formula for always getting max performance. -- Tracy Reed pgpaMcdactpfb.pgp Description: PGP signature -- WhatsUp Gold - Download Free Network Management Software The most intuitive, comprehensive, and cost-effective network management toolset available today. Delivers lowest initial acquisition cost and overall TCO of any competing solution. http://p.sf.net/sfu/whatsupgold-sd___ Aoetools-discuss mailing list Aoetools-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/aoetools-discuss
Re: [Aoetools-discuss] Ethernet CRC-32 and Data integrity?
On Wed, Nov 10, 2010 at 11:09:16AM -0600, David Leach spake thusly: I've been looking at AoE and I'm trying to understand what affect the Ethernet CRC-32 data integrity checking has on the AoE communications? I too have been wanting to better understand the error correction facilities of AoE. So far I have never run into any trouble that I am aware of. Has anyone done any analysis or have any response to AoE's ultimate reliability for missed error detection of bad frames? I suspect it is sufficiently low as to not be an issue. You have to multiply the chance of an error by the chance that the error would not be caught due to being a 32 bit crc. I think that there is a further problem to understand and that is with network connection points. AoE is not routable but that doesn't mean you can't use network switches to interconnect initiators with their AoE targets and at these switches there seems to be a possible error point introduced which AoE isn't protecting against? Are there best practices for AoE installations to protect against these error points? AoE is not layer 3 routable. It is routable in general with spanning tree etc. You can also implement an ethernet tunnel (being careful of MTU concerns etc). I think the genious of AoE vs iSCSI is in adhering to the separation of concerns of each of the layers of the network stack. -- Tracy Reed http://tracyreed.org pgp84YiSewSQD.pgp Description: PGP signature -- The Next 800 Companies to Lead America's Growth: New Video Whitepaper David G. Thomson, author of the best-selling book Blueprint to a Billion shares his insights and actions to help propel your business during the next growth cycle. Listen Now! http://p.sf.net/sfu/SAP-dev2dev___ Aoetools-discuss mailing list Aoetools-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/aoetools-discuss
Re: [Aoetools-discuss] CORAID HBA with vblade
On Wed, Oct 13, 2010 at 08:14:04PM +0200, Roland Kletzing spake thusly: For ESXi, HBA is a must have (vmware is closed source...) i can`t follow that argument.. what has closed source source to do with that ? In this case what it means is you cannot compile and run the Linux kernel initiator on it. Not easily anyway. And it certainly wouldn't be supported. The original poster was worried about being locked into Coraid if he bought Coraid storage and HBA's. He can make his own storage using whiteboxes and vblade if he wanted to support/tune it himself (doable but not trivial). But when it comes to the initiator he has little choice but to buy Coraid HBAs due to the nature of ESXi. btw - many people believe esx is closed source, but many parts of esx are open source, including many parts of the esx kernel, which has many drivers derived from linux drivers. It doesn't take too many closed parts or heavily modified non-community supported forked drivers derived from the open source drivers to remove all of the utility of being open source. so, the question is if an AoE HBA is really a necissity or just an artificial hack to get AoE into ESX without signinig NDA´s for driver development. I doubt this question has little practical value for the original poster. -- Tracy Reed http://tracyreed.org pgp0QOTWqk8C5.pgp Description: PGP signature -- Beautiful is writing same markup. Internet Explorer 9 supports standards for HTML5, CSS3, SVG 1.1, ECMAScript5, and DOM L2 L3. Spend less time writing and rewriting code and more time creating great experiences on the web. Be a part of the beta today. http://p.sf.net/sfu/beautyoftheweb___ Aoetools-discuss mailing list Aoetools-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/aoetools-discuss
Re: [Aoetools-discuss] AOE
On Tue, May 11, 2010 at 10:54:21PM +0200, Stefan Priebe - allied internet ag spake thusly: I'm using a Raid 10 and i already know that document. I played around with fdisk and different sectors, heads, ... but nothing helped to me. I'm shure this is an alignment problem but i don't know what to try. You may need to align the start of data of your partition on a 64 cylinder boundary. I don't have a web browser handy but google for linux raid alignment and you should find some pointers which may help. -- Tracy Reed http://tracyreed.org pgpidUtKF2xE6.pgp Description: PGP signature -- ___ Aoetools-discuss mailing list Aoetools-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/aoetools-discuss
Re: [Aoetools-discuss] AOE
On Wed, May 12, 2010 at 08:57:55AM +0200, Stefan Priebe - allied internet ag spake thusly: I'm using a Hardware Raid 10 Controller with 512MB Cache. Could you give me an example how to tell the target to use direct I/O? The vblade manpage says: -d The -d flag selects O_DIRECT mode for accessing the underlying block device. -- Tracy Reed http://tracyreed.org pgp8kPWbxrw3u.pgp Description: PGP signature -- ___ Aoetools-discuss mailing list Aoetools-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/aoetools-discuss
Re: [Aoetools-discuss] AOE
On Wed, May 12, 2010 at 09:46:58AM -0400, Jesse Becker spake thusly: I have to admit that I find this statement more than a little amusing, with the AoE spec at 12 pages, and iSCSI weighing in at over 20 times that[2]. Ease of use counts for something. The only time I've seen a need to worry about alightment issues--which I'd expect would plague iSCSI as well under certain cases--is when you are using LVM. I've yet to see a problem when using basic AoE block devices. How does LVM affect alignment? Doesn't LVM put its info at the end of the disk? If not how do you compensate for it? -- Tracy Reed -- ___ Aoetools-discuss mailing list Aoetools-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/aoetools-discuss
Re: [Aoetools-discuss] AOE
On Tue, May 11, 2010 at 04:09:41PM +0200, Stefan Priebe - allied internet ag spake thusly: i was trying to replace iSCSI with AOE - but when i was using AOE the target system was continually reading while i was just writing to the disk. When switching from AOE to iSCSI without changing anything else everything was fine again. You are probably having the infamous AoE alignment issue. It has really been bugging me lately too. I would say this is probably my biggest hassle in using AoE. I really wish this could be fixed in the target. There are workarounds (playing with disk geometry) but you have to be very careful to ensure alignment all throughout the various layers of your IO system. And if you are doing RAID 5 be aware of the RAID 5 write hole. The AoE alignment issue has been discussed previously. This might be of use to you: http://copilotco.com/Virtualization/wiki/aoe-caching-alignment.pdf/at_download/file -- Tracy Reed http://tracyreed.org pgpYmKSilaEYK.pgp Description: PGP signature -- ___ Aoetools-discuss mailing list Aoetools-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/aoetools-discuss
[Aoetools-discuss] domU is causing misaligned disk writes
Anyone know why my xen xvda devices would be doing (apparently) unaligned writes to my SAN causing horrible performance and massive seeking and lots of reading for page cache backfill? BUT writing to the device in the dom0 is very fast and causes no extra reads? I am running the 2.6.18-164.11.1.el5xen xen/kernel which came with CentOS 5.4 After spending a lot of time banging my head on this I seem to have finally tracked it down to a difference between domU and dom0. I never would have thought it would be this but it is extremely reproduceable. We're talking a difference of 4-5x in write speed. Reads are equally fast everywhere. I am using AoE v72 kernel module (initiator) on a Dell R610's to talk to vblade-19 (target) on Dell R710's all running CentOS 5.4. I have striped two 7200 RPM SATA disks and exported the md with AoE (although I have done these tests with individual disks also). Read performance is excellent: # dd of=/dev/null if=/dev/xvdg1 bs=4096 count=300 300+0 records in 300+0 records out 1228800 bytes (12 GB) copied, 106.749 seconds, 115 MB/s I dropped the cache with: echo 1 /proc/sys/vm/drop_caches on both target and initiator before starting the test. This is great for just a single gig-e link. This suggests that the network is fine. However, write performance is odious. Typically around 20MB/s. It should be more like 70MB/s per disk or better (7200rpm SATA) and max out my gig-e with write performance similar to the above read performance. I mentioned above that these are unaligned writes because when running iostat on the target machine I can see lots of reads happening which are surely causing seeks and killing performance. Typical is something like 8MB/s of reads while doing 16MB/s of writes. HOWEVER, if I do the writes from the dom0 the performance is excellent: # dd if=/dev/zero of=/dev/etherd/e6.2 bs=4096 count=300 300+0 records in 300+0 records out 1228800 bytes (12 GB) copied, 104.679 seconds, 117 MB/s And I see no reads happening on the disks being written to in iostat. Purely streaming writes at high speeds. I have had AoE working very well with Xen previously although not with this particular hardware/xen/aoe version. Also it occurs to me that in the past when I have done this I network booted the domU's and they got root over AoE using a complicated initrd that I cooked up. In the last year or so I decided that it was too complicated and went to booting my dom0's from compact flash with the AoE driver in the dom0 instead of the domU. I now handing the domU xvd's from the AoE driver in dom0. I strongly suspect that this is why things worked great before but stink now. Unfortunately I don't have a working network boot initrd setup like I used to and although I still have all of the code etc. it would take a while to set up. I don't want to run that setup in production anymore anyway if I can help it. I have tried manually aligning the disk by setting the beginning of data on the partition from 63 to 64 (although this is usually done for RAID alignment) and I have tried changing the disk geometry to account for the extra partition table which causes a half-block page-cache misalignment as described by the ever insightful Kelsey Hudson in his writeup on the issue here: http://copilotco.com/Virtualization/wiki/aoe-caching-alignment.pdf/at_download/file All to no avail. What am I missing here? Why is domU apparently fudging my writes? -- Tracy Reed http://tracyreed.org pgpRIn3Fmcl6X.pgp Description: PGP signature -- Download Intel#174; Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev___ Aoetools-discuss mailing list Aoetools-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/aoetools-discuss
Re: [Aoetools-discuss] [Xen-devel] domU is causing misaligned disk writes
On Tue, Apr 20, 2010 at 11:49:55AM +0300, Pasi Kärkkäinen spake thusly: Please paste your domU partition table: sfdisk -d /dev/xvda I have tried many different things including dd straight to the raw unpartitioned device. That should not be affected by partitioning/lvm/filesystem problems right? Are you using filesystems on normal partitions, or LVM in the domU? I'm pretty sure this is a domU partitioning problem. I have done all of the above. Here I am an xvdg device in my domU to which I am directly doing a dd to, no partitioning or anything: # dd if=/dev/zero of=/dev/xvdg bs=4096 count=300 300+0 records in 300+0 records out 1228800 bytes (12 GB) copied, 449.109 seconds, 27.4 MB/s # /sbin/sfdisk -d /dev/xvdg sfdisk: ERROR: sector 0 does not have an msdos signature /dev/xvdg: unrecognized partition table type No partitions found and running iostat on the target shows the following: Device: rrqm/s wrqm/s r/s w/srkB/swkB/s avgrq-sz avgqu-sz await svctm %util sda 0.00 3474.60 1070.60 46.40 4311.20 13680.0032.21 2.081.83 0.49 54.32 sdb 0.00 3376.00 1060.20 45.60 4289.60 13686.4032.51 2.462.23 0.53 58.12 Or I can partition it with a geometry of 248 heads and 56 sectors which is a multiple of 8 which should avoid the misalignment due to the extra partition table (there is a partition on the physical disk on the target already then I create a logical volume to export to the initiator which then puts its own partition in it which causes misalignment): dd if=/dev/zero of=/dev/xvdg1 bs=4096 count=300 300+0 records in 300+0 records out 1228800 bytes (12 GB) copied, 445.338 seconds, 27.6 MB/s # /sbin/sfdisk -d /dev/xvdg # partition table of /dev/xvdg unit: sectors /dev/xvdg1 : start= 56, size=566227592, Id=8e /dev/xvdg2 : start=0, size=0, Id= 0 /dev/xvdg3 : start=0, size=0, Id= 0 /dev/xvdg4 : start=0, size=0, Id= 0 Device: rrqm/s wrqm/s r/s w/srkB/swkB/s avgrq-sz avgqu-sz await svctm %util sda 0.00 3472.20 1188.20 51.00 4805.60 14097.6030.51 2.712.13 0.52 64.02 sdb 0.00 3472.40 1187.00 52.00 4784.00 14092.8030.47 2.822.22 0.56 68.80 Or I can take a standard partition geometry and set it to start at 64 instead of 63 like so many RAID alignment pages talk about: It is taking even longer this time and I am tired of waiting for dd before sending off this email but suffice it to say it is painfully slow. # /sbin/sfdisk -d /dev/xvdg # partition table of /dev/xvdg unit: sectors /dev/xvdg1 : start= 64, size=566226926, Id=83 /dev/xvdg2 : start=0, size=0, Id= 0 /dev/xvdg3 : start=0, size=0, Id= 0 /dev/xvdg4 : start=0, size=0, Id= 0 Device: rrqm/s wrqm/s r/s w/srkB/swkB/s avgrq-sz avgqu-sz await svctm %util sda 0.00 1832.73 1234.73 30.94 4991.62 7864.2720.31 1.521.23 0.47 59.82 sdb 0.00 1835.13 1219.76 30.54 4916.57 7839.5220.40 1.271.04 0.45 56.67 I would not be at all surprised if you are right about it being a domU partitioning problem. But every scheme I have tried has failed to work properly. Appreciate any pointers. -- Tracy Reed http://tracyreed.org pgp3rUhOUpxW7.pgp Description: PGP signature -- ___ Aoetools-discuss mailing list Aoetools-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/aoetools-discuss
Re: [Aoetools-discuss] [Xen-devel] domU is causing misaligned disk writes
On Tue, Apr 20, 2010 at 10:54:42PM +0300, Pasi Kärkkäinen spake thusly: Please try with bs=1024k and maybe with bs=64k aswell. 4k blocksize transfer will always be slower in domU than in dom0 since virtual disk abstraction makes some overhead, which is more visible with small blocksizes. But overhead in domU wouldn't be causing all of these reads. I am doing a test with bs=64k now: Device: rrqm/s wrqm/s r/s w/srkB/swkB/s avgrq-sz avgqu-sz await svctm %util sda 0.00 3258.20 964.00 54.00 3903.20 13652.80 34.49 2.752.71 0.68 68.88 sdb 0.00 3270.20 974.80 54.00 3940.80 13710.40 34.31 2.422.38 0.55 56.22 So the speed is the same to the partitioned disk than to the raw disk? What disk backend are you using in dom0? phy:? tap:aio: ? Yes. I am using phy: disk backend. Should I be using something else? -- Tracy Reed http://tracyreed.org pgpI4OMgBXznq.pgp Description: PGP signature -- ___ Aoetools-discuss mailing list Aoetools-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/aoetools-discuss
Re: [Aoetools-discuss] [Xen-devel] domU is causing misaligned disk writes
On Tue, Apr 20, 2010 at 01:41:51PM -0700, Brendan Cully spake thusly: You could also be limited by the size of the block request ring (I believe the ring is normally only one page) -- the ring needs to be large enough to handle the bandwidth delay product, and AoE means the delay is probably higher than normal. Interesting. Any easy way to increase this as a test? Do you get better performance against a local partition? You mean a local partition on local disk in the dom0 given to a domU as xvd? Let's see... I just created a 20G logical volume on the dom0: # /usr/sbin/lvcreate -n test -L20G sysvg Added it to the domain config file to be /dev/xvdi and rebooted. phy:/dev/sysvg/test,xvdi,w I know you can attach block devices on the fly but this has not been entirely reliable for me in the past so I reboot now. In domU against xvdi which is /dev/sysvg/test in dom0: # dd if=/dev/zero of=/dev/xvdi bs=4096 count=100 100+0 records in 100+0 records out 409600 bytes (4.1 GB) copied, 99.3749 seconds, 41.2 MB/s And iostat on dom0 shows: Device: rrqm/s wrqm/s r/s w/srkB/swkB/s avgrq-sz avgqu-sz await svctm %util sda 0.00 824.00 4.00 101.0016.00 39936.00 760.99 3.19 31.31 9.52 100.00 In dom0 against the local disk to demonstrate native performance: # dd if=/dev/zero of=/dev/sysvg/test bs=4096 count=100 100+0 records in 100+0 records out 409600 bytes (4.1 GB) copied, 84.9047 seconds, 48.2 MB/s Device: rrqm/s wrqm/s r/s w/srkB/swkB/s avgrq-sz avgqu-sz await svctm %util sda 7.00 11104.00 5.00 96.0048.00 48144.00 954.30 133.92 1172.40 9.94 100.40 Virtually no reads happening. This disk seems a bit slow (older 80G sata disk) but otherwise normal. I don't see anything indicating alignment issues. -- Tracy Reed http://tracyreed.org pgpoPja4GCdfF.pgp Description: PGP signature -- ___ Aoetools-discuss mailing list Aoetools-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/aoetools-discuss
Re: [Aoetools-discuss] [ANNOUNCE] ggaoed 1.0
On Thu, Nov 12, 2009 at 11:46:18AM +0100, Gabor Gombas spake thusly: I've released ggaoed 1.0, my AoE target implementation for Linux. It is available at: Nice! How does this compare with the official coraid vblade target? Does it also implement any of your highlights? Any benchmarks yet? -- Tracy Reed http://tracyreed.org pgp0Zb287NUVW.pgp Description: PGP signature -- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july___ Aoetools-discuss mailing list Aoetools-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/aoetools-discuss
[Aoetools-discuss] Retransmit issues
I currently have two AoE SANs deployed and they both have the same problem. So I must be missing something somewhere. I originally wrote about this last November: http://www.mail-archive.com/aoetools-discuss@lists.sourceforge.net/msg00136.html And never got the problem solved. I also did not get around to trying the patch Ed suggested. But I have a feeling there has got to be something I am doing wrong in the setup here. Performance didn't really matter much on that deployment at the time although it is becoming more important and I have just set up a second SAN with the same issue where I really do need it to perform. I have set up AoE SANs a few times before and got great performance. I'm not sure what could possibly be different this time. vblade-19 on the target side AoE v72 kernel module on the initiator. Using mtu 9000 on all of the interfaces involved. HP ProCurve 2810 switch with a dedicated VLAN for the AoE SAN. The switch is set up for 9000 MTU also. The initiator says: aoe: e0.0: setting 8704 byte data frames aoe: e1.0: setting 8704 byte data frames so I know it is getting the MTU right on that side. The initiator has a vlan interface for the SAN which then goes over the bonded link. cat /dev/etherd/err on the initiator produces lots of: unexpected rsp e2.0tag=7e426...@102e56f91 s=0024e860c18a d=00219b916485 retransmit e2.0 oldtag=00736...@102e56faa newtag=00826faa s=00219b916485 d=0024e860c18a nout=1 unexpected rsp e2.0tag=00736...@102e56fad s=0024e860c18a d=00219b916485 retransmit e2.0 oldtag=083b7...@102e5700e newtag=083c700e s=00219b916485 d=0024e860c18a nout=1 unexpected rsp e2.0tag=083b7...@102e57016 s=0024e860c18a d=00219b916485 retransmit e2.0 oldtag=123d7...@102e5708b newtag=1245708b s=00219b916485 d=0024e860c18a nout=1 unexpected rsp e2.0tag=123d7...@102e57095 s=0024e860c18a d=00219b916485 retransmit e2.0 oldtag=17387...@102e570d6 newtag=174170d6 s=00219b916485 d=0024e860c18a nout=1 unexpected rsp e2.0tag=17387...@102e570dc s=0024e860c18a d=00219b916485 retransmit e2.0 oldtag=20c87...@102e57153 newtag=20cf7153 s=00219b916485 d=0024e860c18a nout=1 unexpected rsp e2.0tag=20c87...@102e5715b s=0024e860c18a d=00219b916485 retransmit e2.0 oldtag=2aae7...@102e571d0 newtag=2ab471d0 s=00219b916485 d=0024e860c18a nout=1 At this point I'm at a loss for what the problem could be. -- Tracy Reed http://tracyreed.org pgpAoeLHTTfVL.pgp Description: PGP signature -- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july___ Aoetools-discuss mailing list Aoetools-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/aoetools-discuss
Re: [Aoetools-discuss] Boot FreeBSD from AoE
On Fri, May 01, 2009 at 11:25:03AM +0200, Matthias Teege spake thusly: Is it possible to boot FreeBSD from AoE target? Does FreeBSD have some sort of initrd system like Linux does? I imagine it must have. I have Linux booting from AoE by PXE booting a kernel and initrd. The initrd loads the aoe module and sets up any necessary networking to make the AoE root fs accessable and then continues with the normal boot process. I bet FreeBSD can do similar. You will probably have to make the FreeBSD initrd yourself just like I did for Linux. -- Tracy Reed http://tracyreed.org pgpQQdDmn49j2.pgp Description: PGP signature -- Register Now Save for Velocity, the Web Performance Operations Conference from O'Reilly Media. Velocity features a full day of expert-led, hands-on workshops and two days of sessions from industry leaders in dedicated Performance Operations tracks. Use code vel09scf and Save an extra 15% before 5/3. http://p.sf.net/sfu/velocityconf___ Aoetools-discuss mailing list Aoetools-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/aoetools-discuss
