RE: Configuration of your benchmark for the performance report with functrace

2015-12-16 Thread James (Fei) Liu-SSI
It should be easy to port functrace to support infernalis , right? Regards, James -Original Message- From: Blinick, Stephen L [mailto:stephen.l.blin...@intel.com] Sent: Wednesday, December 16, 2015 4:01 PM To: James (Fei) Liu-SSI Subject: RE: Configuration of your benchmark for the p

RE: Ceph_objectstore_bench crashed on keyvaluestore bench with ceph master branch

2015-12-03 Thread James (Fei) Liu-SSI
Hi Haomai, Code was reviewed and fixes was verified in my bench. Thanks for your quick turnaround. Regards, James -Original Message- From: Haomai Wang [mailto:haomaiw...@gmail.com] Sent: Wednesday, December 02, 2015 9:49 PM To: James (Fei) Liu-SSI Cc: ceph-devel Subject: Re

Ceph_objectstore_bench crashed on keyvaluestore bench with ceph master branch

2015-12-02 Thread James (Fei) Liu-SSI
Hi Haomai, I happened to run ceph_objectstore_bench against key value store on master branch. It always crashed at finisher_thread_entry : assert(!ls_rval.empty()); It looks like the completion not only has null entry in the finisher queue , but also has none entry in the

RE: queue_transaction interface + unique_ptr + performance

2015-12-02 Thread James (Fei) Liu-SSI
Hi Somnath, Great findings. As you mentioned, unique_ptr and smart_ptr always have a well known problem about the performance comparing to conventional pointer , Got a chance to run your interesting code and close to what you find. But I did not find any place in filestore/newstore

RE: 11/11/2015 Weekly Ceph Performance Meeting IS ON!

2015-11-11 Thread James (Fei) Liu-SSI
Hi Mark, Have been busy this morning and missed the meeting today. Would be possible to upload the recording video to the site at your most convenient time? Thanks, James -Original Message- From: ceph-devel-ow...@vger.kernel.org [mailto:ceph-devel-ow...@vger.kernel.org] On

RE: newstore direction

2015-10-22 Thread James (Fei) Liu-SSI
Hi Sage and other fellow cephers, I truly share the pains with you all about filesystem while I am working on objectstore to improve the performance. As mentioned , there is nothing wrong with filesystem. Just the Ceph as one of use case need more supports but not provided in near future

RE: newstore direction

2015-10-20 Thread James (Fei) Liu-SSI
approach . Regards, James -Original Message- From: ceph-devel-ow...@vger.kernel.org [mailto:ceph-devel-ow...@vger.kernel.org] On Behalf Of James (Fei) Liu-SSI Sent: Tuesday, October 20, 2015 1:33 PM To: Sage Weil Cc: Somnath Roy; ceph-devel@vger.kernel.org Subject: RE: newstore

RE: newstore direction

2015-10-19 Thread James (Fei) Liu-SSI
Hi Sage and Somnath, In my humble opinion, There is another more aggressive solution than raw block device base keyvalue store as backend for objectstore. The new key value SSD device with transaction support would be ideal to solve the issues. First of all, it is raw SSD device. Secondly

RE: Backend ObjectStore engine performance bench with FIO

2015-09-30 Thread James (Fei) Liu-SSI
be the bottleneck. Anything else in your mind will be the bottleneck? Regards, James -Original Message- From: Casey Bodley [mailto:cbod...@redhat.com] Sent: Wednesday, September 30, 2015 6:57 AM To: Xiaoxi Chen Cc: James (Fei) Liu-SSI; ceph-devel@vger.kernel.org Subject: Re: Backend

RE: loadable objectstore

2015-09-29 Thread James (Fei) Liu-SSI
Hi Varada, Can you use command nm to see whether " tracepoint_dlopen" is defined in "./.libs/libceph_filestore.so"? Regards, James -Original Message- From: Varada Kari [mailto:varada.k...@sandisk.com] Sent: Monday, September 28, 2015 6:21 PM To: James (Fei

RE: Backend ObjectStore engine performance bench with FIO

2015-09-29 Thread James (Fei) Liu-SSI
Hi Casey, Great job. Regards, James -Original Message- From: Casey Bodley [mailto:cbod...@redhat.com] Sent: Tuesday, September 29, 2015 1:06 PM To: James (Fei) Liu-SSI Cc: ceph-devel@vger.kernel.org Subject: Re: Backend ObjectStore engine performance bench with FIO Hi James

RE: loadable objectstore

2015-09-29 Thread James (Fei) Liu-SSI
-Original Message- From: Varada Kari [mailto:varada.k...@sandisk.com] Sent: Tuesday, September 29, 2015 10:14 AM To: James (Fei) Liu-SSI; Sage Weil; Matt W. Benjamin; Loic Dachary Cc: ceph-devel Subject: RE: loadable objectstore Hi James, Symbol is present in bss. This is what I find $ nm

RE: loadable objectstore

2015-09-29 Thread James (Fei) Liu-SSI
Hi Varada, Let me know when you complete the rebase. Then I can help you over here to reproduce and fix the issue. Regards, James -Original Message- From: Varada Kari [mailto:varada.k...@sandisk.com] Sent: Tuesday, September 29, 2015 10:38 AM To: James (Fei) Liu-SSI; Sage Weil

RE: loadable objectstore

2015-09-28 Thread James (Fei) Liu-SSI
Hi Varada, Have you rebased the pull request to master already? Thanks, James -Original Message- From: ceph-devel-ow...@vger.kernel.org [mailto:ceph-devel-ow...@vger.kernel.org] On Behalf Of Varada Kari Sent: Friday, September 11, 2015 3:28 AM To: Sage Weil; Matt W. Benjamin;

RE: Newstore create failed with fio_objectstore

2015-09-18 Thread James (Fei) Liu-SSI
. thanks Regards, James -Original Message- From: James (Fei) Liu-SSI Sent: Friday, September 18, 2015 2:55 PM To: 'Chen, Xiaoxi' Cc: 'ceph-devel@vger.kernel.org' Subject: RE: Newstore create failed with fio_objectstore Hi Xiaoxi and Cephers, Thanks for your feedback. I am trying to get

RE: Newstore create failed with fio_objectstore

2015-09-18 Thread James (Fei) Liu-SSI
Hi Xiaoxi and Cephers, Thanks for your feedback. I am trying to get newstore works with rocksdb in fio-objectstore plugin.   Here are steps I  did: 1. Reconfigure the ceph with RocksDB build in ./configure --with-fio-dir=./src/fio/ --with-librocksdb-static 2. sudo ./fio/fio

RE: loadable objectstore

2015-09-14 Thread James (Fei) Liu-SSI
/ELF_Format.pdf [2]https://gcc.gnu.org/ml/gcc/2004-06/msg01956.html Regards, James -Original Message- From: Allen Samuels [mailto:allen.samu...@sandisk.com] Sent: Saturday, September 12, 2015 1:35 PM To: Varada Kari; James (Fei) Liu-SSI; Sage Weil; Matt W. Benjamin; Loic Dachary Cc: ceph-devel

RE: About Fio backend with ObjectStore API

2015-09-14 Thread James (Fei) Liu-SSI
15 at 4:08 AM, Casey Bodley <cbod...@redhat.com> wrote: >> Hi James, >> >> I just looked back at the results you posted, and saw that you were using >> iodepth=1. Setting this higher should help keep the FileStore busy. >> >> Casey >> >> - Original M

RE: loadable objectstore

2015-09-11 Thread James (Fei) Liu-SSI
Hi Varada, Got a chance to go through the code. Great job. It is much cleaner . Several questions: 1. What you think about the performance impact with the new implementation? Such as dynamic library vs static link? 2. Could any vendor just provide a objectstore interfaces complied dynamic

RE: About Fio backend with ObjectStore API

2015-09-11 Thread James (Fei) Liu-SSI
objectstore_aio_open, .close_file = fio_objectstore_aio_close, }; Let me know what you think. Regards, James -Original Message- From: Casey Bodley [mailto:cbod...@redhat.com] Sent: Friday, September 11, 2015 7:28 AM To: James (Fei) Liu-SSI Cc: Haomai Wang; ceph-d

RE: About Fio backend with ObjectStore API

2015-09-10 Thread James (Fei) Liu-SSI
: James (Fei) Liu-SSI Cc: Casey Bodley; ceph-devel@vger.kernel.org Subject: Re: About Fio backend with ObjectStore API On Sat, Sep 5, 2015 at 4:29 AM, James (Fei) Liu-SSI <james@ssi.samsung.com> wrote: > Hi Casey, >Thanks. I even got a compiling error with fio-objectstore branc

RE: About Fio backend with ObjectStore API

2015-09-09 Thread James (Fei) Liu-SSI
objectstore=filestore directory=/mnt/fio_ceph_filestore filestore_journal=/var/lib/ceph/osd/journal-ram/fio_ceph_filestore1 -Original Message- From: Haomai Wang [mailto:haomaiw...@gmail.com] Sent: Tuesday, September 08, 2015 7:28 PM To: James (Fei) Liu-SSI Cc: Casey Bodley Subject: Re: About

RE: About Fio backend with ObjectStore API

2015-09-04 Thread James (Fei) Liu-SSI
-OptiPlex-7010:~/WorkSpace/ceph_fio$ git branch * fio-objectstore master Regards, James -Original Message- From: Casey Bodley [mailto:cbod...@redhat.com] Sent: Thursday, September 03, 2015 10:44 AM To: James (Fei) Liu-SSI Cc: ceph-devel@vger.kernel.org Subject: Re: About Fio backend

RE: About Fio backend with ObjectStore API

2015-09-02 Thread James (Fei) Liu-SSI
Hi Haomai and Case, Do you have any fixes for that segfault? Thanks, James -Original Message- From: Haomai Wang [mailto:haomaiw...@gmail.com] Sent: Wednesday, July 22, 2015 6:07 PM To: Casey Bodley Cc: Casey Bodley; Matt W. Benjamin; James (Fei) Liu-SSI; ceph-devel

RE: About Fio backend with ObjectStore API

2015-07-09 Thread James (Fei) Liu-SSI
} -Original Message- From: Casey Bodley [mailto:cbod...@gmail.com] Sent: Thursday, July 09, 2015 9:19 AM To: James (Fei) Liu-SSI Cc: Haomai Wang; ceph-devel@vger.kernel.org Subject: Re: About Fio backend with ObjectStore API Hi James, In the job file src/test/filestore.fio, you can modify

RE: About Fio backend with ObjectStore API

2015-07-09 Thread James (Fei) Liu-SSI
to reflect that the whole objectstore will be supported by fio_ceph_objectstore.cc. what you think? Let me know if you need any help from my side. Regards, James -Original Message- From: Casey Bodley [mailto:cbod...@gmail.com] Sent: Thursday, July 09, 2015 12:32 PM To: James (Fei) Liu-SSI

RE: About Fio backend with ObjectStore API

2015-07-08 Thread James (Fei) Liu-SSI
] On Behalf Of James (Fei) Liu-SSI Sent: Tuesday, June 30, 2015 2:19 PM To: Casey Bodley Cc: Haomai Wang; ceph-devel@vger.kernel.org Subject: RE: About Fio backend with ObjectStore API Hi Casey, Thanks a lot. Regards, James -Original Message- From: Casey Bodley [mailto:cbod

RE: About Fio backend with ObjectStore API

2015-07-07 Thread James (Fei) Liu-SSI
on this and got it pushed upstream into fio proper. :D Mark On 06/30/2015 04:19 PM, James (Fei) Liu-SSI wrote: Hi Casey, Thanks a lot. Regards, James -Original Message- From: Casey Bodley [mailto:cbod...@gmail.com] Sent: Tuesday, June 30, 2015 2:16 PM To: James (Fei

RE: About Fio backend with ObjectStore API

2015-06-30 Thread James (Fei) Liu-SSI
Hi Haomai, What are you trying to ask is to benchmark local objectstore(like kvstore/filestore/newstore) locally with FIO(ObjectStore engine)? You want to purely compare the performance locally for these objectstores, right? Regards, James -Original Message- From:

RE: About Fio backend with ObjectStore API

2015-06-30 Thread James (Fei) Liu-SSI
Hi Casey, Thanks a lot. Regards, James -Original Message- From: Casey Bodley [mailto:cbod...@gmail.com] Sent: Tuesday, June 30, 2015 2:16 PM To: James (Fei) Liu-SSI Cc: Haomai Wang; ceph-devel@vger.kernel.org Subject: Re: About Fio backend with ObjectStore API Hi, When Danny Al

RE: Inline dedup/compression

2015-06-30 Thread James (Fei) Liu-SSI
] Sent: Tuesday, June 30, 2015 8:50 AM To: Allen Samuels; James (Fei) Liu-SSI; Haomai Wang Cc: ceph-devel Subject: RE: Inline dedup/compression - Reference count has to be maintained as an attribute of the object - As mentioned in the write workflow, duplicate segment writes increment the reference

RE: Inline dedup/compression

2015-06-29 Thread James (Fei) Liu-SSI
want to save space/cost as data grows. Regards, James -Original Message- From: Gregory Farnum [mailto:g...@gregs42.com] Sent: Monday, June 29, 2015 4:01 AM To: James (Fei) Liu-SSI Cc: Haomai Wang; ceph-devel Subject: Re: Inline dedup/compression We discuss this periodically

RE: Inline dedup/compression

2015-06-29 Thread James (Fei) Liu-SSI
into details more for each solution. I really like your idea for dedupe in OSD side by the way. Let me think more about it. Regards, James -Original Message- From: Haomai Wang [mailto:haomaiw...@gmail.com] Sent: Friday, June 26, 2015 8:55 PM To: James (Fei) Liu-SSI Cc: ceph-devel Subject: Re

RE: Inline dedup/compression

2015-06-26 Thread James (Fei) Liu-SSI
. It is not apple to apple comparison. But it is good reference. The datacenters need cost effective solution. Regards, James -Original Message- From: Haomai Wang [mailto:haomaiw...@gmail.com] Sent: Thursday, June 25, 2015 8:08 PM To: James (Fei) Liu-SSI Cc: ceph-devel Subject: Re: Inline dedup