Re: [Xen-devel] Xen blktap driver for Ceph RBD : Anybody wants to test ? :p

2013-12-03 Thread Sylvain Munaut
Hi, What sort of memory are your instances using? I just had a look. Around 120 Mb. Which indeed is a bit higher that I'd like. I haven't turned on any caching so I assume it's disabled. Yes. Cheers, Sylvain -- To unsubscribe from this list: send the line unsubscribe ceph-devel in

RE: [Xen-devel] Xen blktap driver for Ceph RBD : Anybody wants to test ? :p

2013-11-30 Thread James Harper
Hi James, Are you still working on this in any way? Well I'm using it, but I haven't worked on it. I never was able to reproduce any issue with it locally ... In prod, I do run it with cache disabled though since I never took the time to check using the cache was safe in the various

RE: [Xen-devel] Xen blktap driver for Ceph RBD : Anybody wants to test ? :p

2013-11-29 Thread James Harper
...@vger.kernel.org] On Behalf Of Sylvain Munaut Sent: Saturday, 20 April 2013 12:41 AM To: Pasi Kärkkäinen Cc: ceph-devel@vger.kernel.org; xen-de...@lists.xen.org Subject: Re: [Xen-devel] Xen blktap driver for Ceph RBD : Anybody wants to test ? :p If you have time to write up some lines about steps

Re: [Xen-devel] Xen blktap driver for Ceph RBD : Anybody wants to test ? :p

2013-11-29 Thread Sylvain Munaut
Hi James, Are you still working on this in any way? Well I'm using it, but I haven't worked on it. I never was able to reproduce any issue with it locally ... In prod, I do run it with cache disabled though since I never took the time to check using the cache was safe in the various failure

Re: [Xen-devel] Xen blktap driver for Ceph RBD : Anybody wants to test ? :p

2013-08-16 Thread Frederik Thuysbaert
Hi Sylvain, I'm not quite sure what u mean, can u give some more information on how I do this? I compiled tapdisk with ./configure CFLAGS=-g, but I'm not sure this is what u meant. Yes, ./configure CFLAGS=-g LDFLAGS=-g is a good start. ... Then once you have a core file, you can use gdb

RE: [Xen-devel] Xen blktap driver for Ceph RBD : Anybody wants to test ? :p

2013-08-15 Thread James Harper
Hi, I just tested with tap2:aio and that worked (had an old image of the VM on lvm still so just tested with that). Switching back to rbd and it crashes every time, just as postgres is starting in the vm. Booting into single user mode, waiting 30 seconds, then letting the boot

RE: [Xen-devel] Xen blktap driver for Ceph RBD : Anybody wants to test ? :p

2013-08-15 Thread James Harper
I just had a crash since upgrading to dumpling, and will disable merging tonight. Still crashes with merging disabled. James -- To unsubscribe from this list: send the line unsubscribe ceph-devel in the body of a message to majord...@vger.kernel.org More majordomo info at

Re: [Xen-devel] Xen blktap driver for Ceph RBD : Anybody wants to test ? :p

2013-08-14 Thread Frederik Thuysbaert
On 13-08-13 17:39, Sylvain Munaut wrote: It's actually strange that it changes anything at all. Can you try adding a ERROR(HERE\n); in that error path processing and check syslog to see if it's triggered at all ? A traceback would be great if you can get a core file. And possibly compile

Re: [Xen-devel] Xen blktap driver for Ceph RBD : Anybody wants to test ? :p

2013-08-14 Thread Sylvain Munaut
Hi, I just tested with tap2:aio and that worked (had an old image of the VM on lvm still so just tested with that). Switching back to rbd and it crashes every time, just as postgres is starting in the vm. Booting into single user mode, waiting 30 seconds, then letting the boot continue it

RE: [Xen-devel] Xen blktap driver for Ceph RBD : Anybody wants to test ? :p

2013-08-14 Thread James Harper
Hi, I just tested with tap2:aio and that worked (had an old image of the VM on lvm still so just tested with that). Switching back to rbd and it crashes every time, just as postgres is starting in the vm. Booting into single user mode, waiting 30 seconds, then letting the boot continue

Re: [Xen-devel] Xen blktap driver for Ceph RBD : Anybody wants to test ? :p

2013-08-14 Thread Sylvain Munaut
Hi Frederik, A traceback would be great if you can get a core file. And possibly compile tapdisk with debug symbols. I'm not quite sure what u mean, can u give some more information on how I do this? I compiled tapdisk with ./configure CFLAGS=-g, but I'm not sure this is what u meant. Yes,

Re: [Xen-devel] Xen blktap driver for Ceph RBD : Anybody wants to test ? :p

2013-08-13 Thread Sylvain Munaut
FWIW, I can confirm via printf's that this error path is never hit in at least some of the crashes I'm seeing. Ok thanks. Are you using cache btw ? Cheers, Sylvain -- To unsubscribe from this list: send the line unsubscribe ceph-devel in the body of a message to

RE: [Xen-devel] Xen blktap driver for Ceph RBD : Anybody wants to test ? :p

2013-08-13 Thread James Harper
FWIW, I can confirm via printf's that this error path is never hit in at least some of the crashes I'm seeing. Ok thanks. Are you using cache btw ? I hope not. How could I tell? It's not something I've explicitly enabled. Thanks James -- To unsubscribe from this list: send the

Re: [Xen-devel] Xen blktap driver for Ceph RBD : Anybody wants to test ? :p

2013-08-13 Thread Sylvain Munaut
Hi, I hope not. How could I tell? It's not something I've explicitly enabled. It's disabled by default. So you'd have to have enabled it either in ceph.conf or directly in the device path in the xen config. (option is 'rbd cache', http://ceph.com/docs/next/rbd/rbd-config-ref/ ) Cheers,

Re: [Xen-devel] Xen blktap driver for Ceph RBD : Anybody wants to test ? :p

2013-08-13 Thread Frederik Thuysbaert
Hi, I have been testing this a while now, and just finished testing your untested patch. The rbd caching problem still persists. The system I am testing on has the following characteristics: Dom0: - Linux xen-001 3.2.0-4-amd64 #1 SMP Debian 3.2.46-1 x86_64 - Most recent git checkout

Re: [Xen-devel] Xen blktap driver for Ceph RBD : Anybody wants to test ? :p

2013-08-13 Thread Sylvain Munaut
Hi, I have been testing this a while now, and just finished testing your untested patch. The rbd caching problem still persists. Yes, I wouldn't expect to change anything for caching. But I still don't understand why caching would change anything at all ... all of it should be handled within

RE: [Xen-devel] Xen blktap driver for Ceph RBD : Anybody wants to test ? :p

2013-08-13 Thread James Harper
-company.com] Sent: Tuesday, 13 August 2013 7:20 PM To: James Harper Cc: Pasi Kärkkäinen; ceph-devel@vger.kernel.org; xen-de...@lists.xen.org Subject: Re: [Xen-devel] Xen blktap driver for Ceph RBD : Anybody wants to test ? :p Hi, I hope not. How could I tell? It's not something I've explicitly

RE: [Xen-devel] Xen blktap driver for Ceph RBD : Anybody wants to test ? :p

2013-08-13 Thread James Harper
I think I have a separate problem too - tapdisk will segfault almost immediately upon starting but seemingly only for Linux PV DomU's. Once it has started doing this I have to wait a few hours to a day before it starts working again. My Windows DomU's appear to be able to start normally though.

Re: [Xen-devel] Xen blktap driver for Ceph RBD : Anybody wants to test ? :p

2013-08-13 Thread Sylvain Munaut
On Wed, Aug 14, 2013 at 1:39 AM, James Harper james.har...@bendigoit.com.au wrote: I think I have a separate problem too - tapdisk will segfault almost immediately upon starting but seemingly only for Linux PV DomU's. Once it has started doing this I have to wait a few hours to a day before

RE: [Xen-devel] Xen blktap driver for Ceph RBD : Anybody wants to test ? :p

2013-08-13 Thread James Harper
On Wed, Aug 14, 2013 at 1:39 AM, James Harper james.har...@bendigoit.com.au wrote: I think I have a separate problem too - tapdisk will segfault almost immediately upon starting but seemingly only for Linux PV DomU's. Once it has started doing this I have to wait a few hours to a day

RE: [Xen-devel] Xen blktap driver for Ceph RBD : Anybody wants to test ? :p

2013-08-13 Thread James Harper
On Wed, Aug 14, 2013 at 1:39 AM, James Harper james.har...@bendigoit.com.au wrote: I think I have a separate problem too - tapdisk will segfault almost immediately upon starting but seemingly only for Linux PV DomU's. Once it has started doing this I have to wait a few hours to a

Re: [Xen-devel] Xen blktap driver for Ceph RBD : Anybody wants to test ? :p

2013-08-12 Thread Sylvain Munaut
Hi, tapdisk[9180]: segfault at 7f7e3a5c8c10 ip 7f7e387532d4 sp 7f7e3a5c8c10 error 4 in libpthread-2.13.so[7f7e38748000+17000] tapdisk:9180 blocked for more than 120 seconds. tapdisk D 88043fc13540 0 9180 1 0x You can try generating a core file by

RE: [Xen-devel] Xen blktap driver for Ceph RBD : Anybody wants to test ? :p

2013-08-12 Thread James Harper
tapdisk[9180]: segfault at 7f7e3a5c8c10 ip 7f7e387532d4 sp 7f7e3a5c8c10 error 4 in libpthread-2.13.so[7f7e38748000+17000] tapdisk:9180 blocked for more than 120 seconds. tapdisk D 88043fc13540 0 9180 1 0x You can try generating a core file by

RE: [Xen-devel] Xen blktap driver for Ceph RBD : Anybody wants to test ? :p

2013-08-12 Thread James Harper
Here's an (untested yet) patch in the rbd error path: diff --git a/drivers/block-rbd.c b/drivers/block-rbd.c index 68fbed7..ab2d2c5 100644 --- a/drivers/block-rbd.c +++ b/drivers/block-rbd.c @@ -560,6 +560,9 @@ err: if (c) rbd_aio_release(c); +

RE: [Xen-devel] Xen blktap driver for Ceph RBD : Anybody wants to test ? :p

2013-08-10 Thread James Harper
Hi, I've had a few occasions where tapdisk has segfaulted: tapdisk[9180]: segfault at 7f7e3a5c8c10 ip 7f7e387532d4 sp 7f7e3a5c8c10 error 4 in libpthread-2.13.so[7f7e38748000+17000] tapdisk:9180 blocked for more than 120 seconds. tapdisk D 88043fc13540 0

Re: [Xen-devel] Xen blktap driver for Ceph RBD : Anybody wants to test ? :p

2013-08-09 Thread Sylvain Munaut
Hi, I've had a few occasions where tapdisk has segfaulted: tapdisk[9180]: segfault at 7f7e3a5c8c10 ip 7f7e387532d4 sp 7f7e3a5c8c10 error 4 in libpthread-2.13.so[7f7e38748000+17000] tapdisk:9180 blocked for more than 120 seconds. tapdisk D 88043fc13540 0 9180 1

RE: [Xen-devel] Xen blktap driver for Ceph RBD : Anybody wants to test ? :p

2013-08-08 Thread James Harper
Yes the procedure didn't change. If you're on debian I could also sent your prebuilt .deb for blktap and for a patched xen version that includes userspace RBD support. If you have any issue, I can be found on ceph's IRC under 'tnt' nick. I've had a few occasions where tapdisk has

Re: [Xen-devel] Xen blktap driver for Ceph RBD : Anybody wants to test ? :p

2013-08-05 Thread Sylvain Munaut
Hi, Yes the procedure didn't change. If you're on debian I could also sent your prebuilt .deb for blktap and for a patched xen version that includes userspace RBD support. If you have any issue, I can be found on ceph's IRC under 'tnt' nick. Cheers, Sylvain -- To unsubscribe from this

RE: [Xen-devel] Xen blktap driver for Ceph RBD : Anybody wants to test ? :p

2013-08-05 Thread James Harper
Yes the procedure didn't change. If you're on debian I could also sent your prebuilt .deb for blktap and for a patched xen version that includes userspace RBD support. It's working great so far. I just pulled the source and built it then copied blktap in. For some reason I already had

Re: [Xen-devel] Xen blktap driver for Ceph RBD : Anybody wants to test ? :p

2013-08-05 Thread Sylvain Munaut
Hi, It's working great so far. I just pulled the source and built it then copied blktap in. Good to hear :) I've been using it more and more recently and it'll been good for me too, even with live migrations. For some reason I already had a tapdisk in /usr/sbin, as well as the one in

RE: [Xen-devel] Xen blktap driver for Ceph RBD : Anybody wants to test ? :p

2013-08-05 Thread James Harper
For some reason I already had a tapdisk in /usr/sbin, as well as the one in /usr/bin, which confused the issue for a while. I must have installed something manually but I don't remember what. What distribution are you using ? Debian Wheezy James -- To unsubscribe from this list:

Re: [Xen-devel] Xen blktap driver for Ceph RBD : Anybody wants to test ? :p

2013-08-05 Thread Pasi Kärkkäinen
On Mon, Aug 05, 2013 at 01:01:35PM +0200, Sylvain Munaut wrote: Any chance this will be rolled into the main blktap sources? I'd like to ... but I ave no idea how or even who to contact for that ... blktap is so fragmented ... You have blktap2 which is in the man Xen tree. But that's

Re: [Xen-devel] Xen blktap driver for Ceph RBD : Anybody wants to test ? :p

2013-08-05 Thread Sylvain Munaut
I think I saw an announcement recently on xen-devel that blktap3 development has been stopped.. Oh :( In the mail it speaks about QEMU but is it possible to use the QEMU driver model when booting PV domains ? (and not PVHVM). Cheers, Sylvain -- To unsubscribe from this list: send the

Re: [Xen-devel] Xen blktap driver for Ceph RBD : Anybody wants to test ? :p

2013-08-05 Thread George Dunlap
On Mon, Aug 5, 2013 at 1:03 PM, Sylvain Munaut s.mun...@whatever-company.com wrote: I think I saw an announcement recently on xen-devel that blktap3 development has been stopped.. Oh :( In the mail it speaks about QEMU but is it possible to use the QEMU driver model when booting PV domains

Re: [Xen-devel] Xen blktap driver for Ceph RBD : Anybody wants to test ? :p

2013-08-05 Thread Sylvain Munaut
Hi George, Yes; qemu knows how to be a Xen PV block back-end. Very interesting. Is there documentation about this somewhere ? I had a look some time ago and it was really not very clear. Things like what Xen version support this. And with which features ( indirect descriptors, persistent

Re: [Xen-devel] Xen blktap driver for Ceph RBD : Anybody wants to test ? :p

2013-08-05 Thread George Dunlap
On 05/08/13 14:55, Sylvain Munaut wrote: Hi George, Yes; qemu knows how to be a Xen PV block back-end. Very interesting. Is there documentation about this somewhere ? I had a look some time ago and it was really not very clear. Things like what Xen version support this. And with which

Re: [Xen-devel] Xen blktap driver for Ceph RBD : Anybody wants to test ? :p

2013-08-05 Thread Wei Liu
On Mon, Aug 05, 2013 at 03:04:47PM +0100, George Dunlap wrote: On 05/08/13 14:55, Sylvain Munaut wrote: Hi George, Yes; qemu knows how to be a Xen PV block back-end. Very interesting. Is there documentation about this somewhere ? I had a look some time ago and it was really not very

Re: [Xen-devel] Xen blktap driver for Ceph RBD : Anybody wants to test ? :p

2013-08-05 Thread George Dunlap
On 05/08/13 16:18, Wei Liu wrote: On Mon, Aug 05, 2013 at 03:04:47PM +0100, George Dunlap wrote: On 05/08/13 14:55, Sylvain Munaut wrote: Hi George, Yes; qemu knows how to be a Xen PV block back-end. Very interesting. Is there documentation about this somewhere ? I had a look some time ago

Re: [Xen-devel] Xen blktap driver for Ceph RBD : Anybody wants to test ? :p

2013-08-05 Thread Wei Liu
On Mon, Aug 05, 2013 at 04:20:20PM +0100, George Dunlap wrote: On 05/08/13 16:18, Wei Liu wrote: On Mon, Aug 05, 2013 at 03:04:47PM +0100, George Dunlap wrote: On 05/08/13 14:55, Sylvain Munaut wrote: Hi George, Yes; qemu knows how to be a Xen PV block back-end. Very interesting. Is

RE: [Xen-devel] Xen blktap driver for Ceph RBD : Anybody wants to test ? :p

2013-07-31 Thread James Harper
I'm about to start trying this out. Has anything changed since this email http://www.mail-archive.com/ceph-devel@vger.kernel.org/msg13984.html ? Thanks James -- To unsubscribe from this list: send the line unsubscribe ceph-devel in the body of a message to majord...@vger.kernel.org More

Re: [Xen-devel] Xen blktap driver for Ceph RBD : Anybody wants to test ? :p

2013-07-01 Thread Sylvain Munaut
Hi again, However when rbd cache is enabled with: [client] rbd_cache = true the tapdisk process crashes if I do this in the domU: dd if=/dev/xvda bs=1M /dev/null I tested this locally and couldn't reproduce the issue. Doing reads doesn't do anything bad AFAICT. Doing writes OTOH seems to

Re: [Xen-devel] Xen blktap driver for Ceph RBD : Anybody wants to test ? :p

2013-07-01 Thread Nathan O'Sullivan
I've installed debug symbols, perhaps that will give a better idea what is going on? #0 __GI___libc_free (mem=0x7f516065) at malloc.c:2970 #1 0x7f515f3ac84b in ~raw_posix_aligned (this=0x7f513c418f20, __in_chrg=optimised out) at common/buffer.cc:152 #2

Re: [Xen-devel] Xen blktap driver for Ceph RBD : Anybody wants to test ? :p

2013-06-21 Thread Nathan O'Sullivan
I've been testing this on Ubuntu 12.04.02 64-bit with kernel 3.2.0-48 and ceph 0.61.4 With rbd cache disabled, it works well enough in initial testing. However when rbd cache is enabled with: [client] rbd_cache = true the tapdisk process crashes if I do this in the domU: dd if=/dev/xvda bs=1M

Re: [Xen-devel] Xen blktap driver for Ceph RBD : Anybody wants to test ? :p

2013-06-21 Thread Sylvain Munaut
Hi, I've been testing this on Ubuntu 12.04.02 64-bit with kernel 3.2.0-48 and ceph 0.61.4 Thanks for testing :) However when rbd cache is enabled with: [client] rbd_cache = true the tapdisk process crashes if I do this in the domU: dd if=/dev/xvda bs=1M /dev/null Interesting. I'm

Re: [Xen-devel] Xen blktap driver for Ceph RBD : Anybody wants to test ? :p

2013-04-19 Thread Pasi Kärkkäinen
On Thu, Apr 18, 2013 at 05:05:29PM +0200, Sylvain Munaut wrote: Hi, Hi, I've been working on getting a working blktap driver allowing to access ceph RBD block devices without relying on the RBD kernel driver and it finally got to a point where, it works and is testable. Great! Ceph

Re: [Xen-devel] Xen blktap driver for Ceph RBD : Anybody wants to test ? :p

2013-04-19 Thread Sylvain Munaut
If you have time to write up some lines about steps required to test this, that'd be nice, it'll help people to test this stuff. To quickly test, I compiled the package and just replaced the tapdisk binary from my normal blktap install with the newly compiled one. Then you need to setup a RBD