Re: Blueprint: inline data support (step 2)

2013-07-31 Thread Li Wang
Sure, just being busy with some other things, I will post it before the summit. Cheers, Li Wang On 07/31/2013 08:17 AM, Loic Dachary wrote: Hi, It would be nice to have URLs to the current implementation and the benchmark results you got in the blueprint. http://wiki.ceph.com/01Planning

Re: Read ahead affect Ceph read performance much

2013-07-31 Thread Li Wang
determines how much optimization space are there. More OSDs, More performance potential to exploit, so maybe you could try more OSDs. Would like to hear your further test results. Cheers, Li Wang On 07/31/2013 12:42 PM, Chen, Xiaoxi wrote: My 0.02, we have done some readahead test tuning on server

Re: Blueprint: inline data support (step 2)

2013-08-10 Thread Li Wang
Hi Sage, I am on holiday, actually including the CDS day:) I will take care of it later. Thanks for your comments. Cheers, Li Wang On 08/10/2013 02:18 AM, Sage Weil wrote: Hi Li, Thanks for discussing this at the summit! As I mentioned, I think email will be the easiest way to detail my

[PATCH v4] Ceph-fuse: Fallocate and punch hole support

2013-08-13 Thread Li Wang
This patch implements fallocate and punch hole support for Ceph fuse client. Signed-off-by: Yunchuan Wen yunchuan...@ubuntukylin.com Signed-off-by: Li Wang liw...@ubuntukylin.com --- Since the i_size is untrustable without Fs cap, we'd better let the fallocate go without checking if it beyond

[PATCH] Ceph-qa: change the fsx.sh to support hole punching test

2013-08-13 Thread Li Wang
This patch change the fsx.sh to pull better fsx.c from xfstests site to support hole punching test. Signed-off-by: Yunchuan Wen yunchuan...@ubuntukylin.com Signed-off-by: Li Wang liw...@ubuntukylin.com --- qa/workunits/suites/fsx.sh |6 -- 1 file changed, 4 insertions(+), 2 deletions

Re: [PATCH v4] Ceph-fuse: Fallocate and punch hole support

2013-08-14 Thread Li Wang
exists, at this time, who is the first? The order is file location or generating time? On 08/14/2013 12:44 PM, Sage Weil wrote: On Wed, 14 Aug 2013, Li Wang wrote: This patch implements fallocate and punch hole support for Ceph fuse client. Signed-off-by: Yunchuan Wen yunchuan...@ubuntukylin.com

[PATCH v4] Ceph: Punch hole support for kernel client

2013-08-14 Thread Li Wang
This patch implements fallocate and punch hole support for Ceph kernel client. Signed-off-by: Li Wang liw...@ubuntukylin.com Signed-off-by: Yunchuan Wen yunchuan...@ubuntukylin.com --- Passed the fsx test from xfstests. Truncate rather than delete the first object. Thanks go to Sage and Zheng

[PATCH v5] Ceph: Punch hole support for kernel client

2013-08-14 Thread Li Wang
This patch implements fallocate and punch hole support for Ceph kernel client. Signed-off-by: Li Wang liw...@ubuntukylin.com Signed-off-by: Yunchuan Wen yunchuan...@ubuntukylin.com --- Against v3: Passed the fsx test from xfstests. Truncate rather than delete the first object. Thanks go to Sage

[PATCH v5] Ceph-fuse: Fallocate and punch hole support

2013-08-14 Thread Li Wang
This patch implements fallocate and punch hole support for Ceph fuse client. Signed-off-by: Yunchuan Wen yunchuan...@ubuntukylin.com Signed-off-by: Li Wang liw...@ubuntukylin.com --- Enable libcephfs to not delete the first object by passing in a flag. --- src/client/Client.cc | 93

[PATCH] Ceph: Remove useless variable revoked_rdcache

2013-08-15 Thread Li Wang
Cleanup in handle_cap_grant() Signed-off-by: Li Wang liw...@ubuntukylin.com --- fs/ceph/caps.c |5 + 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/fs/ceph/caps.c b/fs/ceph/caps.c index 25442b4..8c90cae 100644 --- a/fs/ceph/caps.c +++ b/fs/ceph/caps.c @@ -2361,7 +2361,6

Re: [PATCH v5] Ceph-fuse: Fallocate and punch hole support

2013-08-19 Thread Li Wang
Hi Sage, In general, these look ok to us. One thing to discuss is that for libcephfs, it works in its own way, in spite of the FALLOC_FL_PUNCH_HOLE defined or not. Shall we limit it? Cheers, Li Wang On 08/16/2013 02:06 PM, Sage Weil wrote: Hi Li- I pushed a couple small fixes on top

Performance results on inline data support

2013-09-30 Thread Li Wang
time is measured with/without inline data support. The results are as follows (seconds), #files withoutwith 100017.3674 8.7186 200035.4848 17.7646 300053.2164 26.4374 Cheers, Li Wang -- To unsubscribe from this list: send the line unsubscribe ceph-devel in the body

Re: Performance results on inline data support

2013-10-01 Thread Li Wang
Currently it is 4KB, but we will implement it as a tunable parameter. Cheers, Li Wang On 09/30/2013 08:39 PM, Mark Nelson wrote: On 09/30/2013 03:34 AM, Li Wang wrote: Hi, We did a performance test on inline data support, the Ceph cluster is composed of 1 MDS, 1 MON, 6 OSD on a HPC cluster

Re: Performance results on inline data support

2013-10-01 Thread Li Wang
of the MDS's metadata cache, at least, leave less space for metadata cache. Cheers, Li Wang On 10/01/2013 10:29 PM, Mark Nelson wrote: Great. I'd be curious to know what the right limits are. :) Mark On 10/01/2013 09:21 AM, Li Wang wrote: Currently it is 4KB, but we will implement it as a tunable

Re: [RFC PATCH] ceph: add acl for cephfs

2013-10-17 Thread Li Wang
Hi, I did not take a careful look at the code. But it seems to me that for Ceph, the same directory could be mounted to different clients with totally different uid/gid, not aware of each other. The ACL won't make much sense and may not work properly. On 10/17/2013 04:27 PM, Guangliang

[PATCH] Fix memory leak in Backtrace::print()

2013-10-30 Thread Li Wang
Free already allocated memory if short of memory Signed-off-by: Li Wang liw...@ubuntukylin.com --- src/common/BackTrace.cc |6 -- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/src/common/BackTrace.cc b/src/common/BackTrace.cc index 6cace65..886ed6a 100644 --- a/src/common

[PATCH] ceph: Fix memory leak in chain_listxattr()

2013-10-30 Thread Li Wang
Free allocated memory before return Signed-off-by: Li Wang liw...@ubuntukylin.com --- src/os/chain_xattr.cc |4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/src/os/chain_xattr.cc b/src/os/chain_xattr.cc index 62733e3..0859564 100644 --- a/src/os/chain_xattr.cc +++ b/src

[RESEND PATCH] ceph: Fix memory leak in chain_listxattr

2013-10-30 Thread Li Wang
Free allocated memory before return Signed-off-by: Li Wang liw...@ubuntukylin.com --- src/os/chain_xattr.cc |2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/os/chain_xattr.cc b/src/os/chain_xattr.cc index 62733e3..52ae8db 100644 --- a/src/os/chain_xattr.cc +++ b/src/os

[PATCH] ceph: Fix memory leak in chain_flistxattr()

2013-10-30 Thread Li Wang
Free allocated memory before return Signed-off-by: Li Wang liw...@ubuntukylin.com --- src/os/chain_xattr.cc |2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/os/chain_xattr.cc b/src/os/chain_xattr.cc index 62733e3..d0a149c 100644 --- a/src/os/chain_xattr.cc +++ b/src/os

[PATCH] ceph: Release resource before return in BackedObject::download()

2013-10-30 Thread Li Wang
Close file before return Signed-off-by: Li Wang liw...@ubuntukylin.com --- src/tools/rados/rados_sync.cc |1 + 1 file changed, 1 insertion(+) diff --git a/src/tools/rados/rados_sync.cc b/src/tools/rados/rados_sync.cc index 03293d3..d2d6ab5 100644 --- a/src/tools/rados/rados_sync.cc +++ b

Re: coverity scan - a plea for help!

2013-10-31 Thread Li Wang
Hi Sage, If you trust, I think we can try this job. Cheers, Li Wang On 11/01/2013 07:30 AM, Sage Weil wrote: Hi everyone, When I send this out several months ago, Danny Al-Gaaf stepped up and submitted an amazing number of patches cleaning up the most concerning issues that Coverity had

[PATCH] ceph: Update the pages in fscache in writepages() path

2013-10-31 Thread Li Wang
Currently, the pages in fscache only are updated in writepage() path, add the process in writepages(). Signed-off-by: Min Chen minc...@ubuntukylin.com Signed-off-by: Li Wang liw...@ubuntukylin.com Signed-off-by: Yunchuan Wen yunchuan...@ubuntukylin.com --- fs/ceph/addr.c |8 +--- 1 file

[RFC PATCH] ceph: Capture stride readahead

2013-11-01 Thread Li Wang
Enable ceph to capture stride readahead, the algorithm is simple and straightforward: prefetch the next stripe if hit. In the future, it may be implemented as enabled only when user requests explicitly as a mount option. Signed-off-by: Yunchuan Wen yunchuan...@ubuntukylin.com Signed-off-by: Li

[RFC PATCH] ceph: Write through cache support based on fscache

2013-11-01 Thread Li Wang
. Signed-off-by: Min Chen minc...@ubuntukylin.com Signed-off-by: Li Wang liw...@ubuntukylin.com Signed-off-by: Yunchuan Wen yunchuan...@ubuntukylin.com --- fs/ceph/addr.c | 10 +++--- fs/ceph/cache.c | 29 + fs/ceph/cache.h | 13 + 3 files changed, 49

Re: [RFC PATCH] ceph: Write through cache support based on fscache

2013-11-02 Thread Li Wang
will not bother too much. So our goal is to implement both, we plan to submit it as a blueprint at the incoming CDS. It would be great if you could help review and give comments on our codes during the development. Again, thanks very much. Cheers, Li Wang On 11/02/2013 12:51 AM, Milosz Tanski

Re: librados integrity tests.

2013-11-05 Thread Li Wang
RADOS does not depend on MDS, however, MDS does depend on RADOS to permanently store metadata. It is not very clear what do you define the integrity of librados. On 11/04/2013 12:16 PM, Bill Hastings wrote: Hi All We are planning to test Ceph for certain use cases. But how do folks usually

[PATCH v2] Ceph: Fix memory leak in chain_flistxattr()

2013-11-06 Thread Li Wang
Free allocated memory before return. Signed-off-by: Li Wang liw...@ubuntukylin.com --- src/os/chain_xattr.cc | 14 +- 1 file changed, 9 insertions(+), 5 deletions(-) diff --git a/src/os/chain_xattr.cc b/src/os/chain_xattr.cc index 8ca8156..c020c9d 100644 --- a/src/os

Waiters on OSD operations will hang if replies invalid?

2013-11-06 Thread Li Wang
For ceph_sync_write()/ceph_osdc_readpages()/ceph_osdc_writepages(), the user process or kernel thread will wait for the pending OSD requests to complete on the corresponding req-r_completion. But it seems they only are waked up in handle_reply() and provided the replies are correct. What about

Re: [PATCH v4] ceph: add acl for cephfs

2013-11-08 Thread Li Wang
Hi, It seems to me there are three issues, you can take a look below if they are really there, On 11/08/2013 01:23 PM, Guangliang Zhao wrote: v4: check the validity before set/get_cached_acl() v3: handle the attr change in ceph_set_acl() v2: remove some redundant code in ceph_setattr()

[PATCH] Ceph: allocate non-zero page to fscache in readpage()

2013-11-08 Thread Li Wang
ceph_osdc_readpages() returns number of bytes read, currently, the code only allocate full-zero page into fscache, this patch fixes this. Signed-off-by: Li Wang liw...@ubuntukylin.com --- fs/ceph/addr.c |2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/ceph/addr.c b/fs

[PATCH 7/7] Cifs: Uncaching no-data page in readpage()

2013-11-11 Thread Li Wang
Currently, if one page allocated into fscache in readpage(), however, with no-data read, it is not uncached. This patch fixes this. Signed-off-by: Li Wang liw...@ubuntukylin.com --- fs/cifs/file.c |4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/fs/cifs/file.c b/fs/cifs

[PATCH 5/7] Cifs: Introduce routine for uncaching single no-data page

2013-11-11 Thread Li Wang
Introduce a routine for uncaching single no-data page, typically in readpage(). Signed-off-by: Li Wang liw...@ubuntukylin.com --- fs/cifs/fscache.h | 13 + 1 file changed, 13 insertions(+) diff --git a/fs/cifs/fscache.h b/fs/cifs/fscache.h index 24794b6..c712f42 100644 --- a/fs

[PATCH 6/7] Cifs: Implement uncaching single no-data page

2013-11-11 Thread Li Wang
Implement the routine for uncaching single no-data page, typically in readpage(). Signed-off-by: Li Wang liw...@ubuntukylin.com --- fs/cifs/fscache.c |7 +++ 1 file changed, 7 insertions(+) diff --git a/fs/cifs/fscache.c b/fs/cifs/fscache.c index 8d4b7bc..168f184 100644 --- a/fs/cifs

[PATCH 3/7] Ceph: Introduce routine for uncaching single no-data page

2013-11-11 Thread Li Wang
Introduce a routine for uncaching single no-data page, typically in readpage(). Signed-off-by: Li Wang liw...@ubuntukylin.com --- fs/ceph/cache.h | 13 + 1 file changed, 13 insertions(+) diff --git a/fs/ceph/cache.h b/fs/ceph/cache.h index ba94940..eb0ec76 100644 --- a/fs/ceph

[PATCH 0/7] Cifs and Ceph: Uncache single no-data page in readpage()

2013-11-11 Thread Li Wang
Currently, the page allocated into fscache in readpage() for Cifs and Ceph does not be uncached if no data read due to io error. This patch fixes this. fscache_readpages_cancel() is for this kind of job but taking list read * as input, so a new routine take page * as input is introduced. Li

[PATCH 1/7] Fscache: Introduce new API fscache_readpage_cancel()

2013-11-11 Thread Li Wang
Introduce a new API fscache_readpage_cancel() for uncaching one single no-data page from fscache. Signed-off-by: Li Wang liw...@ubuntukylin.com --- include/linux/fscache.h | 11 +++ 1 file changed, 11 insertions(+) diff --git a/include/linux/fscache.h b/include/linux/fscache.h index

[PATCH 2/7] Fscache: Implement uncaching single no-data page

2013-11-11 Thread Li Wang
Similar to the routine for multiple pages except that it takes page * as input rather than list head *. Signed-off-by: Li Wang liw...@ubuntukylin.com --- fs/fscache/page.c |8 1 file changed, 8 insertions(+) diff --git a/fs/fscache/page.c b/fs/fscache/page.c index 7f5c658..0c69f72

[PATCH 4/7] Ceph: Uncaching no-data page in readpage()

2013-11-11 Thread Li Wang
Currently, if one page allocated into fscache in readpage(), however, with no-data read, it is not uncached. This patch fixes this. Signed-off-by: Li Wang liw...@ubuntukylin.com --- fs/ceph/addr.c |1 + 1 file changed, 1 insertion(+) diff --git a/fs/ceph/addr.c b/fs/ceph/addr.c index

Re: [PATCH v5] ceph: add acl for cephfs

2013-11-12 Thread Li Wang
acl also as mount-time option. Maybe you could take it into consideration. Cheers, Li Wang On 11/11/2013 03:18 PM, Guangliang Zhao wrote: v5: handle the roll back in ceph_set_acl(), correct ceph_get/set_cached_acl() v4: check the validity before set/get_cached_acl() v3: handle the attr

[RFC] Ceph encryption support

2013-11-12 Thread Li Wang
, with encryption enabled, the same file is not allowed by opened by the second writer, alternatively, we enforce O_LAZYIO on the file, but application is supposed to be aware of this. We plan to submit it as a blueprint for the incoming CDS, comments are welcome. Cheers, Li Wang -- To unsubscribe from

[PATCH] Ceph: Avoid data inconsistency due to d-cache aliasing in readpage()

2013-11-12 Thread Li Wang
If the length of data to be read in readpage() is exactly PAGE_CACHE_SIZE, the original code does not flush d-cache for data consistency after finishing reading. This patches fixes this. Signed-off-by: Li Wang liw...@ubuntukylin.com --- fs/ceph/addr.c |8 ++-- 1 file changed, 6

Re: [PATCH] Ceph: Avoid data inconsistency due to d-cache aliasing in readpage()

2013-11-13 Thread Li Wang
Hi Yan, zero_user_segment() has invoked flush_dcache_page() for us, we donnot wanna flush d-cache twice. Cheers, Li Wang On 11/13/2013 09:19 PM, Yan, Zheng wrote: On Wed, Nov 13, 2013 at 3:22 PM, Li Wang liw...@ubuntukylin.com wrote: If the length of data to be read in readpage

[PATCH 2/2] mds: Release resource before return

2013-11-17 Thread Li Wang
Close file before return. Fix coverity issue: CID 1019571 Signed-off-by: Li Wang liw...@ubuntukylin.com Reported-by: Xianxia Xiao xianxiax...@ubuntukylin.com --- src/mds/MDCache.cc |2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/mds/MDCache.cc b/src/mds/MDCache.cc

[PATCH 0/2] Fix resource leak

2013-11-17 Thread Li Wang
Fix two coverity issues. Li Wang (2): rbd: Release resource before return mds: Release resource before return src/mds/MDCache.cc |2 +- src/rbd.cc |6 +++--- 2 files changed, 4 insertions(+), 4 deletions(-) -- 1.7.9.5 -- To unsubscribe from this list: send the line

Re: [RFC] Ceph encryption support

2013-11-20 Thread Li Wang
Hi Gregory, Thanks for your comments. On 11/13/2013 01:58 AM, Gregory Farnum wrote: On Tue, Nov 12, 2013 at 6:10 AM, Li Wang liw...@ubuntukylin.com wrote: Hi, We want to implement encryption support for Ceph. Currently, we have the draft design, 1 When user mount a ceph directory

Re: [RFC] Ceph encryption support

2013-11-20 Thread Li Wang
Hi Alex, Thanks for your comments. On 11/13/2013 09:07 AM, Alex Elsayed wrote: Li Wang wrote: Hi, We want to implement encryption support for Ceph. Currently, we have the draft design, 1 When user mount a ceph directory for the first time, he can specify a passphrase

[PATCH 02/18] ceph: Add inline state definition

2013-11-27 Thread Li Wang
Signed-off-by: Yunchuan Wen yunchuan...@ubuntukylin.com Signed-off-by: Li Wang liw...@ubuntukylin.com --- src/include/ceph_fs.h |3 +++ 1 file changed, 3 insertions(+) diff --git a/src/include/ceph_fs.h b/src/include/ceph_fs.h index 47ec1f1..07a78b8 100644 --- a/src/include/ceph_fs.h +++ b

[PATCH 04/18] mds: Add inline encode/decode to inode_t

2013-11-27 Thread Li Wang
Signed-off-by: Yunchuan Wen yunchuan...@ubuntukylin.com Signed-off-by: Li Wang liw...@ubuntukylin.com --- src/mds/mdstypes.cc | 12 ++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/src/mds/mdstypes.cc b/src/mds/mdstypes.cc index df6cd8e..01a04e8 100644 --- a/src/mds

[PATCH 10/18] mds: Push inline data to client in inodestat

2013-11-27 Thread Li Wang
Signed-off-by: Yunchuan Wen yunchuan...@ubuntukylin.com Signed-off-by: Li Wang liw...@ubuntukylin.com --- src/mds/CInode.cc | 15 +++ src/messages/MClientReply.h |7 +++ 2 files changed, 22 insertions(+) diff --git a/src/mds/CInode.cc b/src/mds/CInode.cc index

[PATCH 07/18] mds: Add inline fields to Capability

2013-11-27 Thread Li Wang
Signed-off-by: Yunchuan Wen yunchuan...@ubuntukylin.com Signed-off-by: Li Wang liw...@ubuntukylin.com --- src/mds/Capability.h |2 ++ 1 file changed, 2 insertions(+) diff --git a/src/mds/Capability.h b/src/mds/Capability.h index fb6b3dc..995ea3a 100644 --- a/src/mds/Capability.h +++ b/src

[PATCH 06/18] osdc: Add write method with truncate parameters

2013-11-27 Thread Li Wang
Signed-off-by: Yunchuan Wen yunchuan...@ubuntukylin.com Signed-off-by: Li Wang liw...@ubuntukylin.com --- src/osdc/Objecter.h | 10 +- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/src/osdc/Objecter.h b/src/osdc/Objecter.h index 41973dd..40f03de 100644 --- a/src/osdc

[PATCH 09/18] ceph: Add inline fields to InodeStat

2013-11-27 Thread Li Wang
Signed-off-by: Yunchuan Wen yunchuan...@ubuntukylin.com Signed-off-by: Li Wang liw...@ubuntukylin.com --- src/messages/MClientReply.h |2 ++ 1 file changed, 2 insertions(+) diff --git a/src/messages/MClientReply.h b/src/messages/MClientReply.h index 896245f..47908e9 100644 --- a/src/messages

[PATCH 12/18] client: Add inline fields to Inode

2013-11-27 Thread Li Wang
Signed-off-by: Yunchuan Wen yunchuan...@ubuntukylin.com Signed-off-by: Li Wang liw...@ubuntukylin.com --- src/client/Inode.h |5 + 1 file changed, 5 insertions(+) diff --git a/src/client/Inode.h b/src/client/Inode.h index cc054a6..bb17706 100644 --- a/src/client/Inode.h +++ b/src/client

[PATCH 17/18] client: Write inline data path

2013-11-27 Thread Li Wang
Signed-off-by: Yunchuan Wen yunchuan...@ubuntukylin.com Signed-off-by: Li Wang liw...@ubuntukylin.com --- src/client/Client.cc | 55 +- 1 file changed, 54 insertions(+), 1 deletion(-) diff --git a/src/client/Client.cc b/src/client/Client.cc index

[PATCH 15/18] client: Add inline data migration helper

2013-11-27 Thread Li Wang
Signed-off-by: Yunchuan Wen yunchuan...@ubuntukylin.com Signed-off-by: Li Wang liw...@ubuntukylin.com --- src/client/Client.cc | 41 + src/client/Client.h |3 +++ 2 files changed, 44 insertions(+) diff --git a/src/client/Client.cc b/src/client

[PATCH 05/18] ceph: Add inline fields to MClientCaps

2013-11-27 Thread Li Wang
Signed-off-by: Yunchuan Wen yunchuan...@ubuntukylin.com Signed-off-by: Li Wang liw...@ubuntukylin.com --- src/messages/MClientCaps.h | 19 ++- 1 file changed, 18 insertions(+), 1 deletion(-) diff --git a/src/messages/MClientCaps.h b/src/messages/MClientCaps.h index 117f241

[PATCH 11/18] mds: Receive updated inline data from client

2013-11-27 Thread Li Wang
Signed-off-by: Yunchuan Wen yunchuan...@ubuntukylin.com Signed-off-by: Li Wang liw...@ubuntukylin.com --- src/mds/Locker.cc |7 +++ 1 file changed, 7 insertions(+) diff --git a/src/mds/Locker.cc b/src/mds/Locker.cc index 63e0e08..4b02a56 100644 --- a/src/mds/Locker.cc +++ b/src/mds

[PATCH v3 00/18] ceph: Inline data support

2013-11-27 Thread Li Wang
This patch implements inline data support for Ceph. It is also available to be pulled from: https://github.com/kylinstorage/ceph.git inline Signed-off-by: Yunchuan Wen yunchuan...@ubuntukylin.com Signed-off-by: Li Wang liw...@ubuntukylin.com --- Against v2: Streamline the inline data migration

[PATCH 01/18] ceph: Add inline data feature

2013-11-27 Thread Li Wang
Signed-off-by: Yunchuan Wen yunchuan...@ubuntukylin.com Signed-off-by: Li Wang liw...@ubuntukylin.com --- src/ceph_mds.cc |1 + src/include/ceph_features.h |2 ++ 2 files changed, 3 insertions(+) diff --git a/src/ceph_mds.cc b/src/ceph_mds.cc index 88b807b..dac676f 100644

[PATCH 08/18] mds: Push inline data to client in cap message

2013-11-27 Thread Li Wang
Signed-off-by: Yunchuan Wen yunchuan...@ubuntukylin.com Signed-off-by: Li Wang liw...@ubuntukylin.com --- src/mds/CInode.cc |7 +++ 1 file changed, 7 insertions(+) diff --git a/src/mds/CInode.cc b/src/mds/CInode.cc index c8b00ef..4756865 100644 --- a/src/mds/CInode.cc +++ b/src/mds

[PATCH 14/18] client: Push inline data to mds by send cap

2013-11-27 Thread Li Wang
Signed-off-by: Yunchuan Wen yunchuan...@ubuntukylin.com Signed-off-by: Li Wang liw...@ubuntukylin.com --- src/client/Client.cc |5 + 1 file changed, 5 insertions(+) diff --git a/src/client/Client.cc b/src/client/Client.cc index 19d31e0..3beab8f 100644 --- a/src/client/Client.cc +++ b/src

[PATCH 18/18] client: Fallocate inline data path

2013-11-27 Thread Li Wang
Signed-off-by: Yunchuan Wen yunchuan...@ubuntukylin.com Signed-off-by: Li Wang liw...@ubuntukylin.com --- src/client/Client.cc | 99 ++ 1 file changed, 76 insertions(+), 23 deletions(-) diff --git a/src/client/Client.cc b/src/client/Client.cc

[PATCH 2/2] ceph: Add necessary clean up if invalid reply received in handle_reply()

2013-11-27 Thread Li Wang
Wake up possible waiters, invoke the call back if any, unregister the request Signed-off-by: Li Wang liw...@ubuntukylin.com Signed-off-by: Yunchuan Wen yunchuan...@ubuntukylin.com --- net/ceph/osd_client.c |7 +++ 1 file changed, 7 insertions(+) diff --git a/net/ceph/osd_client.c b/net

[PATCH 1/2] ceph: Clean up if error occurred in finish_read()

2013-11-27 Thread Li Wang
Clean up if error occurred rather than going through normal process Signed-off-by: Li Wang liw...@ubuntukylin.com Signed-off-by: Yunchuan Wen yunchuan...@ubuntukylin.com --- fs/ceph/addr.c |3 +++ 1 file changed, 3 insertions(+) diff --git a/fs/ceph/addr.c b/fs/ceph/addr.c index 1e561c0

[PATCH 0/2] ceph: Add clean up if invalid osd reply received

2013-11-27 Thread Li Wang
Signed-off-by: Li Wang liw...@ubuntukylin.com Signed-off-by: Yunchuan Wen yunchuan...@ubuntukylin.com Li Wang (2): ceph: Clean up if error occurred in finish_read() ceph: Add necessary clean up if invalid reply received in handle_reply() fs/ceph/addr.c|3 +++ net/ceph

Re: /usr/bin/ld: cannot find -lboost_program_options

2013-12-01 Thread Li Wang
Please install libboost-program-options-dev package before compiling for example, for Ubuntu, sudo apt-get install libboost-program-options-dev On 12/02/2013 09:57 AM, charles L wrote: Pls can some1 help? Im compiling ceph...i did the make -j2 command and got this cannot find

Re: [PATCH 17/18] client: Write inline data path

2013-12-02 Thread Li Wang
the inline threshold. Cheers, Li Wang On 11/28/2013 11:02 AM, Yan, Zheng wrote: On Wed, Nov 27, 2013 at 9:40 PM, Li Wang liw...@ubuntukylin.com wrote: Signed-off-by: Yunchuan Wen yunchuan...@ubuntukylin.com Signed-off-by: Li Wang liw...@ubuntukylin.com --- src/client/Client.cc | 55

Re: [PATCH 17/18] client: Write inline data path

2013-12-02 Thread Li Wang
length be zero, that will capture some situations to almost eliminate the migration overhead (v4); (3) It could be implicitly turned off at mount time by client (v4); (4) It could be turned off globally by configuring the mds(v4). v4 is coming soon. Cheers, Li Wang On 11/30/2013 01:01 AM, Matt W

[PATCH v4] ceph: Inline data support

2013-12-02 Thread Li Wang
This patch implements inline data support for Ceph. Review at: https://github.com/ceph/ceph/pull/884 Pull at: https://github.com/kylinstorage/ceph.git inline Signed-off-by: Yunchuan Wen yunchuan...@ubuntukylin.com Signed-off-by: Li Wang liw...@ubuntukylin.com --- Against v3: Add the inline switch

Re: [PATCH 1/3] libceph: block I/O when PAUSE or FULL osd map flags are set

2013-12-06 Thread Li Wang
I just had a quick look, did not think it thoroughly. (1) If possible, there is a race condition, that a former write get blocked by FULL, a latter write is lucky to be sent to osd after FULL - NOFULL, then the former write is resent, to cause the old data overwrite the new data. (2) If it

Re: [PATCH 1/2] No MDS mount error fix

2013-12-09 Thread Li Wang
Personally, I don't think there is issue for current implementation, either. If no ACTIVE mds, the mount process put to wait, until updated MDS map received and with active mds present indicated in the map, it will be waked up and go on the mount process, otherwise, EIO returned if timeout. If

Re: [PATCH 1/2] No MDS mount error fix

2013-12-09 Thread Li Wang
the client be uselessly waked up ... On 2013/12/9 22:26, Li Wang wrote: Personally, I don't think there is issue for current implementation, either. If no ACTIVE mds, the mount process put to wait, until updated MDS map received and with active mds present indicated in the map, it will be waked

Re: [PATCH 1/2] No MDS mount error fix

2013-12-09 Thread Li Wang
by ceph.mount client for printing simple message about what's going on. 2013/12/9 Li Wang liw...@ubuntukylin.com mailto:liw...@ubuntukylin.com Personally, I don't think there is issue for current implementation, either. If no ACTIVE mds, the mount process put to wait, until updated MDS map

[PATCH 2/2] ceph fscache: Uncaching no data page from fscache in readpage()

2013-12-19 Thread Li Wang
Currently, if one new page allocated into fscache in readpage(), however, with no data read into due to error encountered during reading from OSDs, the slot in fscache is not uncached. This patch fixes this. Signed-off-by: Li Wang liw...@ubuntukylin.com --- fs/ceph/addr.c |1 + 1 file

[PATCH 1/2] ceph fscache: Introduce a routine for uncaching single no data page from fscache

2013-12-19 Thread Li Wang
Signed-off-by: Li Wang liw...@ubuntukylin.com --- fs/ceph/cache.h | 13 + 1 file changed, 13 insertions(+) diff --git a/fs/ceph/cache.h b/fs/ceph/cache.h index ba94940..da95f61 100644 --- a/fs/ceph/cache.h +++ b/fs/ceph/cache.h @@ -67,6 +67,14 @@ static inline int

[PATCH 0/2] ceph fscache: uncaching single no data page when error

2013-12-19 Thread Li Wang
Currently, if one new page allocated into fscache in readpage(), however, with no data read into due to error encountered during reading from OSDs, the slot in fscache is not uncached. This patch fixes this. Li Wang (2): ceph: Introduce a routine for uncaching single no data page from

[PATCH 2/3] Ceph fscache: Update object store limit after file writing

2013-12-26 Thread Li Wang
From: Yunchuan Wen yunchuan...@ubuntukylin.com Synchronize object-store_limit[_l] with new inode-i_size after file writing. Signed-off-by: Yunchuan Wen yunchuan...@ubuntukylin.com Signed-off-by: Min Chen minc...@ubuntukylin.com Signed-off-by: Li Wang liw...@ubuntukylin.com --- fs/ceph/file.c

[PATCH 3/3] Ceph fscache: Wait for completion of object initialization

2013-12-26 Thread Li Wang
, then immediately followed by a writing, the initialization may have not completed, the code will reach the ASSERT in fscache_submit_exclusive_op() to cause kernel bug. Signed-off-by: Yunchuan Wen yunchuan...@ubuntukylin.com Signed-off-by: Min Chen minc...@ubuntukylin.com Signed-off-by: Li Wang liw

[PATCH 0/3] Ceph fscache: Fix kernel panic due to a race

2013-12-26 Thread Li Wang
From: Yunchuan Wen yunchuan...@ubuntukylin.com The following scripts could easily panic the kernel, #!/bin/bash mount -t ceph -o fsc MONADDR:/ cephfs rm -rf cephfs/foo dd if=/dev/zero of=cephfs/foo bs=8 count=512 echo 3 /proc/sys/vm/drop_caches dd if=cephfs/foo of=/dev/null bs=8 count=1024

[PATCH v5] ceph: Inline data support

2013-12-30 Thread Li Wang
This patch implements inline data support for Ceph. Review at: https://github.com/ceph/ceph/pull/1021 Pull at: https://github.com/kylinstorage/ceph.git wip-inline Signed-off-by: Yunchuan Wen yunchuan...@ubuntukylin.com Signed-off-by: Li Wang liw...@ubuntukylin.com --- Against v4: Forbid old mds

[PATCH v6] ceph: Inline data support

2014-01-14 Thread Li Wang
This patch implements inline data support for Ceph. Review at: https://github.com/ceph/ceph/pull/1081 Pull at: https://github.com/kylinstorage/ceph.git wip-inline Li Wang (23): ceph: Add inline data feature ceph: Add inline state definition Signed-off-by: Yunchuan Wen yunchuan

[PATCH] Cephfs Quota Support

2014-01-21 Thread Li Wang
-by: Li Wang liw...@ubuntukylin.com --- Yunchuan Wen (25): ceph: Add quota feature flags ceph: Add quota_info_t to store quota info mds: Add quota field to inode_t mds: Shutdown old mds without quota support mds: Handle quota update ceph: Add MClientQuota message type ceph: Add

Re: CephFS standup

2014-01-30 Thread Li Wang
will be the Year of the Linux Desktop^W^W^WCephFS! To that end, we should schedule a daily standup to coordinate development activities. The regular participants are probably: Zheng Yan (Shanghai, China) Li Wang (Changsha, China) Sage Weil

Re: Google Summer of Code 2014

2014-02-12 Thread Li Wang
:) Cheers, Li Wang On 2014/2/13 5:17, Patrick McGarry wrote: Hey Ceph developers, We are getting ready to submit our project list to be a mentoring organization for GSoC 2014 and Sage suggested that perhaps there might be a few more mentors/projects out there. At the very least perhaps some of you

Re: New Defects reported by Coverity Scan for ceph (fwd)

2014-03-03 Thread Li Wang
Then it seems that Coverity is only able to perform intra-procedure check, is there any inter-procedure check option to turn on? On 2014/3/4 6:53, John Spray wrote: On Mon, Mar 3, 2014 at 10:23 PM, Sage Weil s...@inktank.com wrote: ** CID 1188299: Data race condition (MISSING_LOCK)

Re: contraining crush placement possibilities

2014-03-06 Thread Li Wang
Provided 3 osds are down simultaneously On 2014/3/7 11:51, Li Wang wrote: Just had a quick look. It seems crush could meet the demand, say, if we have 100 osds, replica_num is 3, then we partition the 100 osds into 3 trees, 'take' iterates on the 3 trees, for each tree, select 1 osd

Re: contraining crush placement possibilities

2014-03-06 Thread Li Wang
Just had a quick look. It seems crush could meet the demand, say, if we have 100 osds, replica_num is 3, then we partition the 100 osds into 3 trees, 'take' iterates on the 3 trees, for each tree, select 1 osd. Then the probability of losing data is at most n*n*n/Cn3, can we make it better?

Re: contraining crush placement possibilities

2014-03-10 Thread Li Wang
pgp_num is the upper bound of number of OSD combinations, right? so we can reduce pgp_num to constrain the possible combinations, and the data loss probability is only dependent on pgp_num, say, pgp_num/Pn(replica_num) (Since (a, b) and (b, a) are different pgs, so it is permutation rather than

Re: What should the Ceph Foundation be about ?

2014-05-06 Thread Li Wang
On 2014/5/6 11:54, Sage Weil wrote: On Mon, 5 May 2014, Justin Erenkrantz wrote: On Thu, May 1, 2014 at 12:32 PM, Patrick McGarry patr...@inktank.com wrote: People like Jim Jagielski, Brian Stevens, Michael Tiemann, and many others are bound to have years of experience that can help us make

Split Swift code with S3 code in Radosgw

2013-03-27 Thread Li Wang
to Swift API and make it easier to add Swift specific features, such as custom metadata, expiring object, negative acl rule, so what is the Ceph community's response? Cheers, Li Wang

[PATCH] Swift ACL .rlistings support

2013-04-02 Thread Li Wang
This patch implements the Swift ACL .rlistings for Radosgw, it should be seamlessly compatible with earlier version as well as S3. Signed-off-by: Yunchuan Wen yunchuan...@ubuntukylin.com Signed-off-by: Li Wang liw...@ubuntukylin.com --- src/rgw/rgw_acl.cc |3 +++ src/rgw/rgw_acl.h

[PATCH] radosgw: receiving unexpected error code while accessing an non-existing object by authorized not-owner user

2013-04-16 Thread Li Wang
:49 GMT Server: Apache/2.2.22 (Ubuntu) Accept-Ranges: bytes Content-Length: 12 Vary: Accept-Encoding Content-Type: text/plain; charset=utf-8 Signed-off-by: Yunchuan Wen yunchuan...@ubuntukylin.com Signed-off-by: Li Wang liw...@ubuntukylin.com --- src/rgw/rgw_acl.cc| 11 +-- src/rgw

[PATCH V2] radosgw: receiving unexpected error code while accessing an non-existing object by authorized not-owner user

2013-04-18 Thread Li Wang
:49 GMT Server: Apache/2.2.22 (Ubuntu) Accept-Ranges: bytes Content-Length: 12 Vary: Accept-Encoding Content-Type: text/plain; charset=utf-8 Signed-off-by: Yunchuan Wen yunchuan...@ubuntukylin.com Signed-off-by: Li Wang liw...@ubuntukylin.com --- src/rgw/rgw_op.cc |2 +- 1 file changed, 1

[PATCH v3] radosgw: receiving unexpected error code while accessing an non-existing object by authorized not-owner user

2013-04-20 Thread Li Wang
:49 GMT Server: Apache/2.2.22 (Ubuntu) Accept-Ranges: bytes Content-Length: 12 Vary: Accept-Encoding Content-Type: text/plain; charset=utf-8 Signed-off-by: Yunchuan Wen yunchuan...@ubuntukylin.com Signed-off-by: Li Wang liw...@ubuntukylin.com Reviewed-by: Yehuda Sadeh yeh...@inktank.com --- Changes

[PATCH v4] radosgw: receiving unexpected error code while accessing an non-existing object by authorized not-owner user

2013-04-25 Thread Li Wang
:49 GMT Server: Apache/2.2.22 (Ubuntu) Accept-Ranges: bytes Content-Length: 12 Vary: Accept-Encoding Signed-off-by: Yunchuan Wen yunchuan...@ubuntukylin.com Signed-off-by: Li Wang liw...@ubuntukylin.com Reviewed-by: Yehuda Sadeh yeh...@inktank.com --- src/rgw/rgw_acl.cc|4 ++-- src/rgw

Re: A question regarding the unlink() code

2013-04-29 Thread Li Wang
HI Yehuda, Thanks for the answer. We are planning to use the CEPH_OSD_OP_DELETE to implement fallocate/hole punching, the details are described in the blueprint (wiki.ceph.com). Your comments/suggestions are appreciated. Regards, Li Wang On 04/29/2013 10:23 PM, Yehuda Sadeh wrote

[PATCH] Fix some little/big endian issues

2013-05-15 Thread Li Wang
Ceph uses little endian, this patch fixes some endian issues while Ceph running on big endian machine. Signed-off-by: Yunchuan Wen yunchuan...@ubuntukylin.com Signed-off-by: Li Wang liw...@ubuntukylin.com --- src/include/encoding.h | 12 +++- 1 file changed, 7 insertions(+), 5

[PATCH 0/2] Kernel file system client support for punch hole

2013-06-14 Thread Li Wang
This patch implements punch hole (fallocate) support for Ceph kernel file system client. We prepared two patches based on different kernel versions, one against kernel 3.8-rc3, the other against the latest 3.10-rc5. It is because unfortunately, we failed to set up a workable Ceph system with the

[PATCH 2/2] Punch hole support against 3.10-rc5

2013-06-14 Thread Li Wang
This patch implements punch hole (fallocate) support against Linux kernel 3.10-rc5. Signed-off-by: Li Wang liw...@ubuntukylin.com Signed-off-by: Yunchuan Wen yunchuan...@ubuntukylin.com --- fs/ceph/file.c| 245 + net/ceph/osd_client.c

[PATCH 1/2] Punch hole support against 3.8-rc3

2013-06-14 Thread Li Wang
This patch implements punch hole (fallocate) support against Linux kernel 3.8-rc3. Signed-off-by: Li Wang liw...@ubuntukylin.com Signed-off-by: Yunchuan Wen yunchuan...@ubuntukylin.com --- fs/ceph/file.c| 248 + net/ceph/osd_client.c

  1   2   >