[PATCH 2/2] updated test to cover "." directory stat

2011-05-11 Thread Brian Chrisman
Signed-off-by: Brian Chrisman --- src/client/testceph.cc | 17 + 1 files changed, 17 insertions(+), 0 deletions(-) diff --git a/src/client/testceph.cc b/src/client/testceph.cc index da4b7a2..520fa75 100644 --- a/src/client/testceph.cc +++ b/src/client/testceph.cc @@ -174,6 +17

[PATCH 1/2] Add analogous special case for "." directory alongside ".." in _lookup

2011-05-11 Thread Brian Chrisman
Signed-off-by: Brian Chrisman --- src/client/Client.cc |5 + 1 files changed, 5 insertions(+), 0 deletions(-) diff --git a/src/client/Client.cc b/src/client/Client.cc index 10a6829..6ab4643 100644 --- a/src/client/Client.cc +++ b/src/client/Client.cc @@ -3267,6 +3267,11 @@ int Client::_

[PATCH 0/2] Update Client to handle self directories

2011-05-11 Thread Brian Chrisman
libceph lookup of the self-referencing '.' directory fails. Patch makes Client class handle '.' specially like it does '..'. testceph updated to check the special cases of lstat(.) and lstat(/.). Brian Chrisman (2): Add analogous special case for "." directory alongside ".." in _lookup upd

Re: [PATCH] mds: issue caps on file update finish if no client cap specified

2011-05-11 Thread Henry C Chang
I created a ticket - http://tracker.newdream.net/issues/1084 and uploaded the mds log where I found the problem. (Full log is huge. I just uploaded the critical part. Let me know if you need the full one.) Henry 2011/5/12 Henry C Chang : >> This one I'm not sure about.  Do you have an MDS log for

Re: [PATCH] mds: issue caps on file update finish if no client cap specified

2011-05-11 Thread Henry C Chang
> This one I'm not sure about.  Do you have an MDS log for this case?  I > would expect the cap issue to happen when we drop_locks(mut) and the lock > state changes. > OK. I'll try to reproduce it with debug log on. Henry -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in

Re: [PATCH 0/3] d_prune

2011-05-11 Thread Sage Weil
On Wed, 11 May 2011, Sage Weil wrote: > [half written email from wrong patch directory] Hi Al, Christoph, Once the dentry_unhash series is merged, the VFS won't be doing any hashing or unhashing of dentries on behalf of file systems; that will almost solely their responsibility the respective i_o

[PATCH 2/3] ceph: clear parent D_COMPLETE flag when on dentry prune

2011-05-11 Thread Sage Weil
When the VFS prunes a dentry from the cache, clear the D_COMPLETE flag on the parent dentry. Do this for the live and snapshotted namespaces. Do not bother for the .snap dir contents, since we do not cache that. Signed-off-by: Sage Weil --- fs/ceph/dir.c | 28 f

[PATCH 3/3] ceph: switch I_COMPLETE to D_COMPLETE

2011-05-11 Thread Sage Weil
We used to use a flag on the directory inode to track whether the dcache contents for a directory were a complete cached copy. Switch to a dentry flag CEPH_D_COMPLETE that can be safely updated by ->d_prune(). Signed-off-by: Sage Weil --- fs/ceph/caps.c |8 ++ fs/ceph/dir.c

[PATCH 0/3] d_prune

2011-05-11 Thread Sage Weil
The Ceph client is told by the server when it has the entire contents of a directory in cache, and is notified prior to any changes. However, the current VFS infrastructure does not allow the client to handle a lookup on a non-existent entry in a non-racy way. The first patch addes a new d_pru

[PATCH 1/3] vfs: add d_prune dentry operation

2011-05-11 Thread Sage Weil
This adds a d_prune dentry operation that is called by the VFS prior to pruning (i.e. unhashing and killing) a hashed dentry from the dcache. This will be used by Ceph to maintain a flag indicating whether the complete contents of a directory are contained in the dcache, allowing it to satisfy loo

[GIT PULL] Ceph fixes for 2.6.39

2011-05-11 Thread Sage Weil
Hi Linus, Please pull the following bug fixes from git://git.kernel.org/pub/scm/linux/kernel/git/sage/ceph-client.git for-linus These patches came in pretty late but fix a few crashes and deadlocks that turned up under blogbench. Thanks! sage Henry C Chang (3): ceph: print debug mess

Re: rbd create error with 0.26

2011-05-11 Thread Josh Durgin
On 05/10/2011 06:54 PM, Simon Tian wrote: If helpless, I will got more trace info. BTW, where could I get the debug packages? Unfortunately, the backtrace isn't very useful without debugging symbols, and I don't see any Fedora packages that include them. You can create a package with debuggi

rados pool object listing suckage

2011-05-11 Thread Sage Weil
The ability to list objects in rados pools is something we originally added to round out the librados interface and to support the requirements of radosgw (S3 and swift both let you list objects in buckets). The current interface is stateless. You iterate over the PGs in the pool, and for eac

Re: OSD Crash

2011-05-11 Thread Colin McCabe
On Wed, May 11, 2011 at 1:47 PM, Mark Nigh wrote: > Some additional testing shows that the underlying filesystem btrfs does fail > thus the daemon appropriately fails. > > The way I am simulating a failed HDD is by removing the HDD. The failure is > working, > but the problem is when I reinsert

RE: OSD Crash

2011-05-11 Thread Sage Weil
On Wed, 11 May 2011, Mark Nigh wrote: > Some additional testing shows that the underlying filesystem btrfs does > fail thus the daemon appropriately fails. > > The way I am simulating a failed HDD is by removing the HDD. The failure > is working, but the problem is when I reinsert the HDD. I thi

RE: OSD Crash

2011-05-11 Thread Mark Nigh
Some additional testing shows that the underlying filesystem btrfs does fail thus the daemon appropriately fails. The way I am simulating a failed HDD is by removing the HDD. The failure is working, but the problem is when I reinsert the HDD. I think I see the BTRFS filesystem recovery (btrfs f

Re: [PATCH] mds: issue caps on file update finish if no client cap specified

2011-05-11 Thread Sage Weil
On Wed, 11 May 2011, Henry C Chang wrote: > Fix the following scenario that happens occasionally when running > blogbench: > > client released caps on one inode. Then, the inode's ifile was wrlocked > during updating client range. Before the update had finished, client > re-opened the file again

Re: [PATCH 1/3] ceph: print debug message before put mds session

2011-05-11 Thread Sage Weil
Applied all three of these. Thanks, Henry! I'll send them to Linus today or tomorrow so they'll make 2.6.39. sage On Wed, 11 May 2011, Henry C Chang wrote: > The mds session, s, could be freed during ceph_put_mds_session. > Move dout before ceph_put_mds_session. > > Signed-off-by: Henry C C

Re: [PATCH 0/2] adding libceph xattr support

2011-05-11 Thread Sage Weil
Thanks Brian, applied these. (Also broke out the namespace thing into a separate patch.) sage On Tue, 10 May 2011, Brian Chrisman wrote: > Expands libceph to handle xattr calls including underlying Client methods. > testceph is expanded to verify libceph xattr calls work. > > Brian Chrisman (

Re: Kernel 2.6.38.6 page allocation failure (ixgbe)

2011-05-11 Thread Stefan Majer
Hi Sage, after some digging we set sysctl -w vm.min_free_kbytes=262144 default was around 16000 This solved our problem and rados bench survived a 5 minute torture with no single failure: min lat: 0.036177 max lat: 299.924 avg lat: 0.553904 sec Cur ops started finished avg MB/s cur MB/s

[PATCH] mds: issue caps on file update finish if no client cap specified

2011-05-11 Thread Henry C Chang
Fix the following scenario that happens occasionally when running blogbench: client released caps on one inode. Then, the inode's ifile was wrlocked during updating client range. Before the update had finished, client re-opened the file again for reading. Since ifile was wrlocked, the client was n

[PATCH 1/3] ceph: print debug message before put mds session

2011-05-11 Thread Henry C Chang
The mds session, s, could be freed during ceph_put_mds_session. Move dout before ceph_put_mds_session. Signed-off-by: Henry C Chang --- fs/ceph/mds_client.c |2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/fs/ceph/mds_client.c b/fs/ceph/mds_client.c index f60b07b..d0fae4

[PATCH 3/3] ceph: do not use i_wrbuffer_ref as refcount for Fb cap

2011-05-11 Thread Henry C Chang
We increments i_wrbuffer_ref when taking the Fb cap. This breaks the dirty page accounting and causes looping in __ceph_do_pending_vmtruncate, and ceph client hangs. This bug can be reproduced occasionally by running blogbench. Add a new field i_wb_ref to inode and dedicate it to Fb reference cou

Re: Kernel 2.6.38.6 page allocation failure (ixgbe)

2011-05-11 Thread Stefan Majer
Hi Sage, we were running rados bench like this: # rados -p data bench 60 write -t 128 Maintaining 128 concurrent writes of 4194304 bytes for at least 60 seconds. sec Cur ops started finished avg MB/s cur MB/s last lat avg lat 0 0 0 0 0 0

[PATCH 2/3] ceph: fix list_add in ceph_put_snap_realm

2011-05-11 Thread Henry C Chang
Signed-off-by: Henry C Chang --- fs/ceph/snap.c |2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/fs/ceph/snap.c b/fs/ceph/snap.c index e86ec11..24067d6 100644 --- a/fs/ceph/snap.c +++ b/fs/ceph/snap.c @@ -206,7 +206,7 @@ void ceph_put_snap_realm(struct ceph_mds_client *md

OSD Crash

2011-05-11 Thread Mark Nigh
I was performing a few failure test with the osd by removing a HDD from one of the osd host. All was well, the cluster noticed the failure and re-balanced data but when I replace the HDD into the host, the cosd crashed. Here is my setup. 6 osd host with 4 HDDs each (4 cosd daemons running for ea