RE: queue_transaction interface + unique_ptr + performance

2015-12-03 Thread Somnath Roy
y; Samuel Just (sam.j...@inktank.com); ceph-devel@vger.kernel.org Subject: Re: queue_transaction interface + unique_ptr + performance Hi, To those writing benchmarks, please note that the standard library provides std::make_shared() as an optimization for shared_ptr. It creates the object and i

RE: Ceph_objectstore_bench crashed on keyvaluestore bench with ceph master branch

2015-12-03 Thread James (Fei) Liu-SSI
Hi Haomai, Code was reviewed and fixes was verified in my bench. Thanks for your quick turnaround. Regards, James -Original Message- From: Haomai Wang [mailto:haomaiw...@gmail.com] Sent: Wednesday, December 02, 2015 9:49 PM To: James (Fei) Liu-SSI Cc: ceph-devel Subject: Re

Re: queue_transaction interface + unique_ptr + performance

2015-12-03 Thread Casey Bodley
Hi, To those writing benchmarks, please note that the standard library provides std::make_shared() as an optimization for shared_ptr. It creates the object and its shared storage in a single allocation, instead of the two allocations required for "std::shared_ptr fs (new Foo())". Casey - Ori

Re: Compiling for FreeBSD, runtimes for seperate tests.

2015-12-03 Thread Willem Jan Withagen
On 3-12-2015 10:50, Willem Jan Withagen wrote: On 2-12-2015 22:10, Willem Jan Withagen wrote: Running gmake check Now I start wondering how long certain tests are able to run: I've killed: unittest_chain_xattr because it was running already voor 12 hours. And unittest_lfnindex is also

Re: Compiling for FreeBSD, missing rbd

2015-12-03 Thread Willem Jan Withagen
On 3-12-2015 13:34, Mykola Golub wrote: > On Wed, Dec 02, 2015 at 11:47:04PM +0100, Willem Jan Withagen wrote: >> >> Running gmake check is starting to work. >> reporting still thinks there are no successful tests >> but tests themselves report OKE >> >> But where I really can not get it

Re: Compiling for FreeBSD, missing rbd

2015-12-03 Thread Mykola Golub
On Wed, Dec 02, 2015 at 11:47:04PM +0100, Willem Jan Withagen wrote: > > Running gmake check is starting to work. > reporting still thinks there are no successful tests > but tests themselves report OKE > > But where I really can not get it to work is with testing rbd. > > Is starts

RE: queue_transaction interface + unique_ptr + performance

2015-12-03 Thread Sage Weil
1- I agree we should avoid shared_ptr whenever possible. 2- unique_ptr should not have any more overhead than a raw pointer--the compiler is enforcing the single-owner semantics. See for example https://msdn.microsoft.com/en-us/library/hh279676.aspx "It is exactly is efficient as a raw

Re: CodingStyle on existing code

2015-12-03 Thread Joao Eduardo Luis
On 12/01/2015 03:18 PM, Sage Weil wrote: > On Tue, 1 Dec 2015, Wido den Hollander wrote: >> >> On 01-12-15 16:00, Gregory Farnum wrote: >>> On Tue, Dec 1, 2015 at 5:47 AM, Loic Dachary wrote: On 01/12/2015 14:10, Wido den Hollander wrote: > Hi, > > While working on mon/P

Re: Compiling for FreeBSD, runtimes for seperate tests.

2015-12-03 Thread Willem Jan Withagen
On 2-12-2015 22:10, Willem Jan Withagen wrote: Running gmake check Now I start wondering how long certain tests are able to run: I've killed: unittest_chain_xattr because it was running already voor 12 hours. And unittest_lfnindex is also running for > 20 minutes Is there a way

RE: ceph encoding optimization

2015-12-03 Thread piotr.da...@ts.fujitsu.com
> -Original Message- > From: Xinze Chi (信泽) [mailto:xmdx...@gmail.com] > Sent: Thursday, December 03, 2015 10:18 AM > To: Milosz Tanski > Cc: Sage Weil; Gregory Farnum; Dan Mick; Haomai Wang; Dałek, Piotr; ceph- > de...@vger.kernel.org > Subject: Re: ceph encoding opti

Re: ceph encoding optimization

2015-12-03 Thread 信泽
I write a new patch about this https://github.com/XinzeChi/ceph/commit/06eb471e463a4687e251273d0b5dfe170acbef2d If you use __attribute__ ((packed)) after struct, we could encode many struct member in a batch. There is not compatibility problem if we keep the order of members defined in struct. Wa

Re: [ceph-users] How long will the logs be kept?

2015-12-03 Thread Jan Schermer
---邮件原件- > 发件人: huang jun [mailto:hjwsm1...@gmail.com] > 发送时间: 2015年12月3日 13:19 > 收件人: wukongming 12019 (RD) > 抄送: ceph-devel@vger.kernel.org; ceph-us...@lists.ceph.com > 主题: Re: How long will the logs be kept? > > it will rotate every week by default, you can see the logrotate f

RE: queue_transaction interface + unique_ptr + performance

2015-12-03 Thread piotr.da...@ts.fujitsu.com
> -Original Message- > From: ceph-devel-ow...@vger.kernel.org [mailto:ceph-devel- > ow...@vger.kernel.org] On Behalf Of Somnath Roy > Sent: Thursday, December 03, 2015 3:13 AM > > *Also*, in this way we are unnecessary adding another smart pointer > overhead in the Ceph IO path. > As I com

RE: queue_transaction interface + unique_ptr + performance

2015-12-02 Thread Somnath Roy
d for shared_ptr overhead is >2X. Thanks & Regards Somnath -Original Message- From: Somnath Roy Sent: Wednesday, December 02, 2015 7:59 PM To: 'James (Fei) Liu-SSI'; Sage Weil (s...@newdream.net); Samuel Just (sam.j...@inktank.com) Cc: ceph-devel@vger.kernel.org Subject:

Re: Ceph_objectstore_bench crashed on keyvaluestore bench with ceph master branch

2015-12-02 Thread Haomai Wang
thanks! Fixed in https://github.com/ceph/ceph/pull/6783. plz review On Thu, Dec 3, 2015 at 3:19 AM, James (Fei) Liu-SSI wrote: > Hi Haomai, >I happened to run ceph_objectstore_bench against key value store on master > branch. It always crashed at finisher_thread_entry : > assert(!ls_rval.e

Re: Troublesome documentation link on http://ceph.com

2015-12-02 Thread Dan Mick
On 12/01/2015 04:44 AM, Nathan Cutler wrote: > So I go to http://ceph.com and click on the "Documentation" link. In the > HTML source code the linked URL is http://ceph.com/docs but the HTTP > server rewrites this to: > > http://docs.ceph.com/docs/v0.80.5/ > > Could someone arrange for this to be

Re: queue_transaction interface + unique_ptr

2015-12-02 Thread Haomai Wang
On Thu, Dec 3, 2015 at 8:17 AM, Somnath Roy wrote: > Hi Sage/Sam, > As discussed in today's performance meeting , I am planning to change the > queue_transactions() interface to the following. > > int queue_transactions(Sequencer *osr, list& tls, > Context *onreadable,

Re: How long will the logs be kept?

2015-12-02 Thread huang jun
it will rotate every week by default, you can see the logrotate file /etc/ceph/logrotate.d/ceph 2015-12-03 12:37 GMT+08:00 Wukongming : > Hi ,All > Is there anyone who knows How long or how many days will the logs.gz > (mon/osd/mds)be kept, maybe before flushed? > > --

RE: queue_transaction interface + unique_ptr + performance

2015-12-02 Thread Somnath Roy
); Samuel Just (sam.j...@inktank.com) Cc: ceph-devel@vger.kernel.org Subject: RE: queue_transaction interface + unique_ptr + performance Hi Somnath, Great findings. As you mentioned, unique_ptr and smart_ptr always have a well known problem about the performance comparing to conventional pointer , Got

RE: queue_transaction interface + unique_ptr + performance

2015-12-02 Thread James (Fei) Liu-SSI
Message- From: ceph-devel-ow...@vger.kernel.org [mailto:ceph-devel-ow...@vger.kernel.org] On Behalf Of Somnath Roy Sent: Wednesday, December 02, 2015 6:13 PM To: Somnath Roy; Sage Weil (s...@newdream.net); Samuel Just (sam.j...@inktank.com) Cc: ceph-devel@vger.kernel.org Subject: RE

Re: tarball for 10.0.0

2015-12-02 Thread Dan Mick
On 11/30/2015 11:57 AM, Deneau, Tom wrote: > I did not see the source tarball for 10.0.0 at > http://download.ceph.com/tarballs/ceph-10.0.0.tar.gz > It's there now, FWIW. -- Dan Mick Red Hat, Inc. Ceph docs: http://ceph.com/docs -- To unsubscribe from this list: send the line "unsubscribe cep

RE: queue_transaction interface + unique_ptr + performance

2015-12-02 Thread Somnath Roy
*Also*, in this way we are unnecessary adding another smart pointer overhead in the Ceph IO path. As I communicated sometimes back (probably 2 years now :-) ) in the community, profiler is showing these smart pointers (shared_ptr) as one of the hot spot. Now, I decided to actually measure this..

Re: Compiling for FreeBSD

2015-12-02 Thread Yan, Zheng
On Thu, Dec 3, 2015 at 4:52 AM, Willem Jan Withagen wrote: > On 2-12-2015 15:13, Yan, Zheng wrote: >> see https://github.com/ceph/ceph/pull/6770. The code can be compiled >> on FreeBSD/OSX, most client programs can connect to ceph servers on >> Linux. > > Hi, > > I do like some of the inline compi

Re: Compiling for FreeBSD, missing rbd

2015-12-02 Thread Willem Jan Withagen
Running gmake check is starting to work. reporting still thinks there are no successful tests but tests themselves report OKE But where I really can not get it to work is with testing rbd. Is starts with the simple: /bin/sh: rbd: not found And whatever I'm trying in co

Re: Compiling for FreeBSD

2015-12-02 Thread Willem Jan Withagen
On 1-12-2015 19:51, Willem Jan Withagen wrote: > On 1-12-2015 17:24, Willem Jan Withagen wrote: >> I think in the short run it will not be the code that is going to be a >> major porting pain. But getting the run-time environment ironed out is >> just plain (hard) work. Things where /bin/sh expects

Re: Compiling for FreeBSD

2015-12-02 Thread Willem Jan Withagen
On 2-12-2015 15:13, Yan, Zheng wrote: > see https://github.com/ceph/ceph/pull/6770. The code can be compiled > on FreeBSD/OSX, most client programs can connect to ceph servers on > Linux. Hi, I do like some of the inline compiler tests. I guess that the way the errno's are done like the other OS

Re: Suggestions on tracker 13578

2015-12-02 Thread John Spray
arnum; Vimal >> Cc: ceph-devel >> Subject: Re: Suggestions on tracker 13578 >> >> >> On 12/02/2015 12:23 PM, Gregory Farnum wrote: >> > On Tue, Dec 1, 2015 at 5:23 AM, Vimal wrote: >> >> Hello, >> >> >> >> This mail is to di

RE: Suggestions on tracker 13578

2015-12-02 Thread Paul Von-Stamwitz
> -Original Message- > From: ceph-devel-ow...@vger.kernel.org [mailto:ceph-devel- > ow...@vger.kernel.org] On Behalf Of Mark Nelson > Sent: Wednesday, December 02, 2015 11:04 AM > To: Gregory Farnum; Vimal > Cc: ceph-devel > Subject: Re: Suggestions on tracker 13578

Re: Suggestions on tracker 13578

2015-12-02 Thread Mark Nelson
On 12/02/2015 12:23 PM, Gregory Farnum wrote: On Tue, Dec 1, 2015 at 5:23 AM, Vimal wrote: Hello, This mail is to discuss the feature request at http://tracker.ceph.com/issues/13578. If done, such a tool should help point out several mis-configurations that may cause problems in a cluster l

Re: Suggestions on tracker 13578

2015-12-02 Thread Gregory Farnum
On Tue, Dec 1, 2015 at 5:23 AM, Vimal wrote: > Hello, > > This mail is to discuss the feature request at > http://tracker.ceph.com/issues/13578. > > If done, such a tool should help point out several mis-configurations that > may cause problems in a cluster later. > > Some of the suggestions are:

Re: Compiling for FreeBSD

2015-12-02 Thread Yan, Zheng
see https://github.com/ceph/ceph/pull/6770. The code can be compiled on FreeBSD/OSX, most client programs can connect to ceph servers on Linux. Regards Yan. Zheng On Wed, Dec 2, 2015 at 2:43 AM, Willem Jan Withagen wrote: > On 1-12-2015 19:36, Sage Weil wrote: >> >> On Tue, 1 Dec 2015, Alan Some

Re: RGW S3 Website hosting, non-clean code for early review

2015-12-02 Thread Javier Muñoz
me major > work on this in August, probably week 2/3, depending on other > interrupts/priorities. > > It's almost complete, the only missing features are: > - error page fetcher gets confused on conditional requests (I think a > different approach is needed, esp re range &

Re: why my cluster become unavailable (min_size of pool)

2015-12-02 Thread Libin Wu
t osd as lost, and accept that you may have lost some recent > writes to the data). > > sage > > > > > >> >> We use 0.80.10 version. >> >> Thanks! >> >> >> -- >> hzwulibin >> 2015-11-26 >> >&g

Re: How to open clog debug

2015-12-01 Thread Zhiqiang Wang
Hi Joao, 2015-12-01 22:34 GMT+08:00 Joao Eduardo Luis : > On 12/01/2015 09:13 AM, Zhiqiang Wang wrote: >> Looks like in the current code, the clog generated by OSD and MDS are >> not sent to MON. When the client subscribes to the log monitor, it >> won't get the OSD and MDS clogs. It only gets tho

Re: Fwd: Fwd: [newstore (again)] how disable double write WAL

2015-12-01 Thread Sage Weil
Hi David, On Tue, 1 Dec 2015, David Casier wrote: > Hi Sage, > With a standard disk (4 to 6 TB), and a small flash drive, it's easy > to create an ext4 FS with metadata on flash > > Example with sdg1 on flash and sdb on hdd : > > size_of() { > blockdev --getsize $1 > } > > mkdmsetup() { > _

Re: [PATCH] rbd: don't put snap_context twice in rbd_queue_workfn()

2015-12-01 Thread Josh Durgin
On 12/01/2015 12:04 PM, Ilya Dryomov wrote: Commit 4e752f0ab0e8 ("rbd: access snapshot context and mapping size safely") moved ceph_get_snap_context() out of rbd_img_request_create() and into rbd_queue_workfn(), adding a ceph_put_snap_context() to the error path in rbd_queue_workfn(). However, r

Re: Compiling for FreeBSD

2015-12-01 Thread Willem Jan Withagen
On 1-12-2015 17:24, Willem Jan Withagen wrote: I think in the short run it will not be the code that is going to be a major porting pain. But getting the run-time environment ironed out is just plain (hard) work. Things where /bin/sh expects certain bash-isms. Where paths have not been setup to t

Re: Compiling for FreeBSD

2015-12-01 Thread Willem Jan Withagen
On 1-12-2015 19:36, Sage Weil wrote: On Tue, 1 Dec 2015, Alan Somers wrote: On Tue, Dec 1, 2015 at 11:08 AM, Willem Jan Withagen wrote: On 1-12-2015 18:22, Alan Somers wrote: I did some work porting Ceph to FreeBSD, but got distracted and stopped about two years ago. You may find this port

Re: Compiling for FreeBSD

2015-12-01 Thread Willem Jan Withagen
On 1-12-2015 19:21, Alan Somers wrote: On Tue, Dec 1, 2015 at 11:08 AM, Willem Jan Withagen wrote: On 1-12-2015 18:22, Alan Somers wrote: I did some work porting Ceph to FreeBSD, but got distracted and stopped about two years ago. You may find this port useful, though it will probably need t

Re: Compiling for FreeBSD

2015-12-01 Thread Sage Weil
On Tue, 1 Dec 2015, Alan Somers wrote: > On Tue, Dec 1, 2015 at 11:08 AM, Willem Jan Withagen wrote: > > On 1-12-2015 18:22, Alan Somers wrote: > >> > >> I did some work porting Ceph to FreeBSD, but got distracted and > >> stopped about two years ago. You may find this port useful, though it > >>

Re: Compiling for FreeBSD

2015-12-01 Thread Alan Somers
On Tue, Dec 1, 2015 at 11:08 AM, Willem Jan Withagen wrote: > On 1-12-2015 18:22, Alan Somers wrote: >> >> I did some work porting Ceph to FreeBSD, but got distracted and >> stopped about two years ago. You may find this port useful, though it >> will probably need to be updated: >> >> https://pe

Re: Compiling for FreeBSD

2015-12-01 Thread Willem Jan Withagen
On 1-12-2015 18:22, Alan Somers wrote: I did some work porting Ceph to FreeBSD, but got distracted and stopped about two years ago. You may find this port useful, though it will probably need to be updated: https://people.freebsd.org/~asomers/ports/net/ceph/ I'll chcek that one as well... A

Re: Compiling for FreeBSD

2015-12-01 Thread Alan Somers
27;make check' from src/... that's what we'd actually >>>> want the gitbuilder to do. >>> >>> >>> I was running that at the moment >>> Found the suggestion on the developers pages, in the manual section. >>> Sort of hidden at

Re: Compiling for FreeBSD

2015-12-01 Thread Willem Jan Withagen
ction. Sort of hidden at the bottom. :) Did kill it in between, but now when I run it, it just only generates the report. So I just went make clean, which is rather too much... But could not really figure out the makefiles in test (yet) How do I reset the test results? I don't think there is an

Re: Cache Tiering Investigation and Potential Patch

2015-12-01 Thread Mark Nelson
proxied to take advantage of larger read cache. Nick -Original Message- From: ceph-devel-ow...@vger.kernel.org [mailto:ceph-devel- ow...@vger.kernel.org] On Behalf Of Sage Weil Sent: 25 November 2015 20:41 To: Nick Fisk Cc: 'ceph-users' ; ceph-devel@vger.kernel.org; 'Mark

Re: CodingStyle on existing code

2015-12-01 Thread Sage Weil
On Tue, 1 Dec 2015, Wido den Hollander wrote: > > On 01-12-15 16:00, Gregory Farnum wrote: > > On Tue, Dec 1, 2015 at 5:47 AM, Loic Dachary wrote: > >> > >> > >> On 01/12/2015 14:10, Wido den Hollander wrote: > >>> Hi, > >>> > >>> While working on mon/PGMonitor.cc I see that there is a lot of > >

Re: CodingStyle on existing code

2015-12-01 Thread Wido den Hollander
On 01-12-15 16:00, Gregory Farnum wrote: > On Tue, Dec 1, 2015 at 5:47 AM, Loic Dachary wrote: >> >> >> On 01/12/2015 14:10, Wido den Hollander wrote: >>> Hi, >>> >>> While working on mon/PGMonitor.cc I see that there is a lot of >>> inconsistency on the code. >>> >>> A lot of whitespaces, inden

Re: CodingStyle on existing code

2015-12-01 Thread Gregory Farnum
On Tue, Dec 1, 2015 at 5:47 AM, Loic Dachary wrote: > > > On 01/12/2015 14:10, Wido den Hollander wrote: >> Hi, >> >> While working on mon/PGMonitor.cc I see that there is a lot of >> inconsistency on the code. >> >> A lot of whitespaces, indentation which is not correct, well, a lot of >> things.

Re: Compiling for FreeBSD

2015-12-01 Thread Sage Weil
g that at the moment > Found the suggestion on the developers pages, in the manual section. > Sort of hidden at the bottom. :) > > Did kill it in between, but now when I run it, it just only generates the > report. > So I just went make clean, which is rather too much... >

Re: How to open clog debug

2015-12-01 Thread Joao Eduardo Luis
On 12/01/2015 09:13 AM, Zhiqiang Wang wrote: > Looks like in the current code, the clog generated by OSD and MDS are > not sent to MON. When the client subscribes to the log monitor, it > won't get the OSD and MDS clogs. It only gets those generated by MON. Most likely, you need to set 'clog_to_mo

Re: CodingStyle on existing code

2015-12-01 Thread Loic Dachary
On 01/12/2015 14:10, Wido den Hollander wrote: > Hi, > > While working on mon/PGMonitor.cc I see that there is a lot of > inconsistency on the code. > > A lot of whitespaces, indentation which is not correct, well, a lot of > things. > > Is this something we want to fix? With some scripts we c

Re: Compiling for FreeBSD

2015-12-01 Thread Willem Jan Withagen
On 1-12-2015 14:30, Sage Weil wrote: On Tue, 1 Dec 2015, Willem Jan Withagen wrote: On 30-11-2015 14:21, Sage Weil wrote: The problem with all of the porting code in general is that it is doomed to break later on if we don't have (at least) ongoing build tests. In order for a FreeBSD or OSX po

Re: Compiling for FreeBSD

2015-12-01 Thread Sage Weil
On Tue, 1 Dec 2015, Willem Jan Withagen wrote: > On 30-11-2015 14:21, Sage Weil wrote: > > The problem with all of the porting code in general is that it is doomed > > to break later on if we don't have (at least) ongoing build tests. In > > order for a FreeBSD or OSX port to continue working we n

Re: Compiling for FreeBSD

2015-12-01 Thread Willem Jan Withagen
On 1-12-2015 13:22, Mykola Golub wrote: On Tue, Dec 01, 2015 at 10:42:57AM +0100, Willem Jan Withagen wrote: On 30-11-2015 17:20, Gregory Farnum wrote: Installed the results: gmake install And looked at what it delivered: zfs diff And I have to be honest that I'm not really enjoying

Re: Compiling for FreeBSD

2015-12-01 Thread Mykola Golub
On Tue, Dec 01, 2015 at 10:42:57AM +0100, Willem Jan Withagen wrote: > On 30-11-2015 17:20, Gregory Farnum wrote: > >>Installed the results: > >>gmake install > >> > >>And looked at what it delivered: > >> zfs diff > >> > >>And I have to be honest that I'm not really enjoying all the ceph-

Re: Compiling for FreeBSD

2015-12-01 Thread Willem Jan Withagen
On 30-11-2015 14:21, Sage Weil wrote: The problem with all of the porting code in general is that it is doomed to break later on if we don't have (at least) ongoing build tests. In order for a FreeBSD or OSX port to continue working we need VMs that run either gitbuilder or a jenkins job or simi

Re: Compiling for FreeBSD

2015-12-01 Thread Willem Jan Withagen
On 30-11-2015 17:20, Gregory Farnum wrote: Installed the results: gmake install And looked at what it delivered: zfs diff And I have to be honest that I'm not really enjoying all the ceph-test stuff in my /usr/local/bin Would prefer it to go into something like: /usr/loca

Re: How to open clog debug

2015-12-01 Thread Zhiqiang Wang
Looks like in the current code, the clog generated by OSD and MDS are not sent to MON. When the client subscribes to the log monitor, it won't get the OSD and MDS clogs. It only gets those generated by MON. 2015-11-30 21:24 GMT+08:00 Sage Weil : > On Mon, 30 Nov 2015, Wukongming wrote: >> Hi, All

Re: Compiling for FreeBSD

2015-11-30 Thread Gregory Farnum
On Mon, Nov 30, 2015 at 11:04 AM, Willem Jan Withagen wrote: > On 30-11-2015 15:40, Willem Jan Withagen wrote: >> On 30-11-2015 15:13, Mykola Golub wrote: >>> On Mon, Nov 30, 2015 at 12:53:54PM +0100, Willem Jan Withagen wrote: >>> > git clone --recursive -b wip-freebsd https://github.com/troc

Re: Compiling for FreeBSD

2015-11-30 Thread Willem Jan Withagen
On 30-11-2015 15:40, Willem Jan Withagen wrote: > On 30-11-2015 15:13, Mykola Golub wrote: >> On Mon, Nov 30, 2015 at 12:53:54PM +0100, Willem Jan Withagen wrote: >> git clone --recursive -b wip-freebsd https://github.com/trociny/ceph.git cd ceph ./install-deps.sh ./do_autogen.s

Re: RFC: teuthology field in commit messages

2015-11-30 Thread Gregory Farnum
On Sun, Nov 29, 2015 at 6:15 PM, Loic Dachary wrote: > > > On 29/11/2015 23:55, John Spray wrote: >> On Sun, Nov 29, 2015 at 9:25 PM, Loic Dachary wrote: >>> >>> >>> On 29/11/2015 21:47, John Spray wrote: On Sun, Nov 29, 2015 at 8:25 PM, Loic Dachary wrote: > > > On 29/11/2015 2

Re: Compiling for FreeBSD

2015-11-30 Thread Willem Jan Withagen
On 30-11-2015 15:13, Mykola Golub wrote: > On Mon, Nov 30, 2015 at 12:53:54PM +0100, Willem Jan Withagen wrote: > >>> git clone --recursive -b wip-freebsd https://github.com/trociny/ceph.git >>> cd ceph >>> ./install-deps.sh >>> ./do_autogen.sh >>> gmake >>> cd src >>> ./vstart.sh # start dev clus

Re: Compiling for FreeBSD

2015-11-30 Thread Willem Jan Withagen
On 30-11-2015 15:13, Mykola Golub wrote: > On Mon, Nov 30, 2015 at 12:53:54PM +0100, Willem Jan Withagen wrote: > >>> git clone --recursive -b wip-freebsd https://github.com/trociny/ceph.git >>> cd ceph >>> ./install-deps.sh >>> ./do_autogen.sh >>> gmake >>> cd src >>> ./vstart.sh # start dev clus

Re: Compiling for FreeBSD

2015-11-30 Thread Mykola Golub
On Mon, Nov 30, 2015 at 12:53:54PM +0100, Willem Jan Withagen wrote: > > git clone --recursive -b wip-freebsd https://github.com/trociny/ceph.git > > cd ceph > > ./install-deps.sh > > ./do_autogen.sh > > gmake > > cd src > > ./vstart.sh # start dev cluster > > ./ceph -s # check it works > > > >

Re: Compiling for FreeBSD

2015-11-30 Thread Willem Jan Withagen
On 30-11-2015 14:21, Sage Weil wrote: > The problem with all of the porting code in general is that it is doomed > to break later on if we don't have (at least) ongoing build tests. In > order for a FreeBSD or OSX port to continue working we need VMs that run > either gitbuilder or a jenkins jo

Re: How to open clog debug

2015-11-30 Thread Sage Weil
On Mon, 30 Nov 2015, Wukongming wrote: > Hi, All > > Does anyone know how to open clog debug? It's usually something like monc->clog.debug() << "hi there\n"; sage -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majord...@vger.

Re: Compiling for FreeBSD

2015-11-30 Thread Sage Weil
The problem with all of the porting code in general is that it is doomed to break later on if we don't have (at least) ongoing build tests. In order for a FreeBSD or OSX port to continue working we need VMs that run either gitbuilder or a jenkins job or similar so that we can tell when it brea

Re: Compiling for FreeBSD

2015-11-30 Thread Willem Jan Withagen
On 30-11-2015 07:58, Mykola Golub wrote: > On Mon, Nov 30, 2015 at 11:46:27AM +0800, Yan, Zheng wrote: >> On Mon, Nov 30, 2015 at 2:57 AM, Willem Jan Withagen >> wrote: ... Hi Mykola, > I have a branch in my repo that contains patches to make compilation > on FreeBSD easier: > > https://github

Re: Compiling for FreeBSD

2015-11-29 Thread Mykola Golub
On Mon, Nov 30, 2015 at 11:46:27AM +0800, Yan, Zheng wrote: > On Mon, Nov 30, 2015 at 2:57 AM, Willem Jan Withagen wrote: > > > > On 29-11-2015 19:08, Haomai Wang wrote: > > > > > I guess we still expect FreeBSD support, which version do you test to > > > compile? I'd like to help to make bsd work

Re: Bucket namespaces pull req. 5872

2015-11-29 Thread Pete Zaitcev
On Mon, 26 Oct 2015 16:09:41 +0100 Radoslaw Zarzynski wrote: > > The rgw_swift_account_in_url should be possible to incorporate > > in a compatible fashion (it does not add an extra next_tok()). > > According to "rgw_swift_account_in_url": I don’t see viable method for > deducing whether two tok

Re: Compiling for FreeBSD

2015-11-29 Thread Yan, Zheng
On Mon, Nov 30, 2015 at 2:57 AM, Willem Jan Withagen wrote: > > On 29-11-2015 19:08, Haomai Wang wrote: > > > I guess we still expect FreeBSD support, which version do you test to > > compile? I'd like to help to make bsd work :-) > > I considered it is best to develop against HEAD aka: > 11.0-CUR

Re: RFC: teuthology field in commit messages

2015-11-29 Thread Loic Dachary
On 29/11/2015 23:55, John Spray wrote: > On Sun, Nov 29, 2015 at 9:25 PM, Loic Dachary wrote: >> >> >> On 29/11/2015 21:47, John Spray wrote: >>> On Sun, Nov 29, 2015 at 8:25 PM, Loic Dachary wrote: On 29/11/2015 21:08, John Spray wrote: > On Sat, Nov 28, 2015 at 3:56 PM, Loi

Re: RFC: teuthology field in commit messages

2015-11-29 Thread John Spray
On Sun, Nov 29, 2015 at 9:25 PM, Loic Dachary wrote: > > > On 29/11/2015 21:47, John Spray wrote: >> On Sun, Nov 29, 2015 at 8:25 PM, Loic Dachary wrote: >>> >>> >>> On 29/11/2015 21:08, John Spray wrote: On Sat, Nov 28, 2015 at 3:56 PM, Loic Dachary wrote: > Hi Ceph, > > An opt

Re: RFC: teuthology field in commit messages

2015-11-29 Thread Loic Dachary
On 29/11/2015 21:47, John Spray wrote: > On Sun, Nov 29, 2015 at 8:25 PM, Loic Dachary wrote: >> >> >> On 29/11/2015 21:08, John Spray wrote: >>> On Sat, Nov 28, 2015 at 3:56 PM, Loic Dachary wrote: Hi Ceph, An optional teuthology field could be added to a commit message like so:

Re: RFC: teuthology field in commit messages

2015-11-29 Thread John Spray
On Sun, Nov 29, 2015 at 8:25 PM, Loic Dachary wrote: > > > On 29/11/2015 21:08, John Spray wrote: >> On Sat, Nov 28, 2015 at 3:56 PM, Loic Dachary wrote: >>> Hi Ceph, >>> >>> An optional teuthology field could be added to a commit message like so: >>> >>> teuthology: --suite rbd >>> >>> to state

Re: RFC: teuthology field in commit messages

2015-11-29 Thread Loic Dachary
On 29/11/2015 21:08, John Spray wrote: > On Sat, Nov 28, 2015 at 3:56 PM, Loic Dachary wrote: >> Hi Ceph, >> >> An optional teuthology field could be added to a commit message like so: >> >> teuthology: --suite rbd >> >> to state that this commit should be tested with the rbd suite. It could be

Re: RFC: teuthology field in commit messages

2015-11-29 Thread John Spray
On Sat, Nov 28, 2015 at 3:56 PM, Loic Dachary wrote: > Hi Ceph, > > An optional teuthology field could be added to a commit message like so: > > teuthology: --suite rbd > > to state that this commit should be tested with the rbd suite. It could be > parsed by bots and humans. > > It would make it

Re: Compiling for FreeBSD

2015-11-29 Thread Willem Jan Withagen
On 29-11-2015 19:08, Haomai Wang wrote: > I guess we still expect FreeBSD support, which version do you test to > compile? I'd like to help to make bsd work :-) I considered it is best to develop against HEAD aka: 11.0-CURRENT FreeBSD 11.0-CURRENT #0 r291381: Sat Nov 28 14:22:54 CET 2015 I'm also

Re: Compiling for FreeBSD

2015-11-29 Thread Haomai Wang
On Mon, Nov 30, 2015 at 1:44 AM, Willem Jan Withagen wrote: > Hi, > > Not unlike many others running FreeBSD I'd like to see if I/we can get > Ceph to build and run on FreeBSD. If not all components than at least > certain components. > > With compilation I do get quite some way, even with the CLA

Re: RFC: teuthology field in commit messages

2015-11-29 Thread Loic Dachary
Hi Joao, On 29/11/2015 12:51, Joao Eduardo Luis wrote: > On 11/28/2015 03:56 PM, Loic Dachary wrote: >> Hi Ceph, >> >> An optional teuthology field could be added to a commit message like so: >> >> teuthology: --suite rbd >> >> to state that this commit should be tested with the rbd suite. It coul

Re: RFC: teuthology field in commit messages

2015-11-29 Thread Joao Eduardo Luis
On 11/28/2015 03:56 PM, Loic Dachary wrote: > Hi Ceph, > > An optional teuthology field could be added to a commit message like so: > > teuthology: --suite rbd > > to state that this commit should be tested with the rbd suite. It could be > parsed by bots and humans. > > It would make it easy

Re: [PATCH] mm: Allow GFP_IOFS for page_cache_read page cache allocation

2015-11-27 Thread Michal Hocko
On Fri 27-11-15 17:40:41, Vlastimil Babka wrote: > On 11/26/2015 04:08 PM, Michal Hocko wrote: > >On Thu 12-11-15 10:53:01, Jan Kara wrote: > >>On Wed 11-11-15 15:13:53, mho...@kernel.org wrote: > >>> > >>>Hi, > >>>this has been posted previously as a part of larger GFP_NOFS related > >>>patch set

Re: [PATCH] mm: Allow GFP_IOFS for page_cache_read page cache allocation

2015-11-27 Thread Vlastimil Babka
On 11/26/2015 04:08 PM, Michal Hocko wrote: On Thu 12-11-15 10:53:01, Jan Kara wrote: On Wed 11-11-15 15:13:53, mho...@kernel.org wrote: Hi, this has been posted previously as a part of larger GFP_NOFS related patch set (http://lkml.kernel.org/r/1438768284-30927-1-git-send-email-mhocko%40kern

Re: data-at-rest compression

2015-11-27 Thread Alyona Kiselyova
Hi All, I created a separate PR for common plugin interface, corresponding to 2 Sage's commits and one my bug fix commit: https://github.com/ceph/ceph/pull/6696 Also I updated compression plugins commit with this patch: https://github.com/ceph/ceph/pull/6361 Igor will wait to update his PR (https:/

Re: Quick comparison of Hammer & Infernalis on NVMe

2015-11-27 Thread Jens Rosenboom
2015-11-26 15:02 GMT+01:00 Alexandre DERUMIER : > Thanks for sharing ! > > Results are impressive, Great to see that write are finally improving. > > I just wonder how much you could have with rbd_cache=false. Do you have rbd_cache_writethrough_until_flush set to false? Otherwise the fio tests wil

Re: What's the difference between Paxos::trim() and PaxosService::trim?

2015-11-27 Thread Joao Eduardo Luis
Hi Xiaoxi, cc-ing ceph-devel for posterity ;) On 11/27/2015 08:25 AM, Chen, Xiaoxi wrote: > Hi joao, > > I am a bit confuse about the difference between in > Paxos::trim() and PaxosService::trim? Seems they did same work basically? In essence, both have the same purpose: remove no long

Re: rgw/civetweb privileged port bind

2015-11-26 Thread Karol Mroz
On Thu, Nov 26, 2015 at 11:38:26AM -0800, Sage Weil wrote: > On Thu, 26 Nov 2015, Karol Mroz wrote: > > Hello, > > > > As I understand it, with the release of infernalis, ceph > > daemons are no longer being run as root. Thus, rgw/civetweb > > is unable to bind to privileged ports: > > > > http:/

Re: rgw/civetweb privileged port bind

2015-11-26 Thread Sage Weil
On Thu, 26 Nov 2015, Karol Mroz wrote: > Hello, > > As I understand it, with the release of infernalis, ceph > daemons are no longer being run as root. Thus, rgw/civetweb > is unable to bind to privileged ports: > > http://tracker.ceph.com/issues/13600 > > We encountered this problem as well in

Re: [PATCH] mm: Allow GFP_IOFS for page_cache_read page cache allocation

2015-11-26 Thread Michal Hocko
On Thu 12-11-15 10:53:01, Jan Kara wrote: > On Wed 11-11-15 15:13:53, mho...@kernel.org wrote: > > From: Michal Hocko > > > > page_cache_read has been historically using page_cache_alloc_cold to > > allocate a new page. This means that mapping_gfp_mask is used as the > > base for the gfp_mask. Ma

Re: Quick comparison of Hammer & Infernalis on NVMe

2015-11-26 Thread Alexandre DERUMIER
Thanks for sharing ! Results are impressive, Great to see that write are finally improving. I just wonder how much you could have with rbd_cache=false. Also, with such high load, maybe using jemalloc on fio could help too (I have seen around 20% improvement on fio client) LD_PRELOAD=${JEMALLO

Re: block-rbd: One function call less in rbd_dev_probe_parent() after error detection

2015-11-26 Thread SF Markus Elfring
>> * Why was the function "rbd_dev_probe_parent" implemented in the way >> that it relies on a sanity check in the function "rbd_dev_destroy" then? > > Because it's not a bad thing? There are different opinions about this implementation detail. > What's wrong with an init to NULL, a possible

Re: why my cluster become unavailable (min_size of pool)

2015-11-26 Thread Sage Weil
-- > hzwulibin > 2015-11-26 > > - > "hzwulibin" > ?2015-11-23 09:00 > Sage Weil,Haomai Wang > ???ceph-devel > ???Re: why my cluster become unavailable > > Hi, Sage > > Tha

Re: block-rbd: One function call less in rbd_dev_probe_parent() after error detection

2015-11-26 Thread Ilya Dryomov
On Thu, Nov 26, 2015 at 8:54 AM, SF Markus Elfring wrote: >>> I interpreted the eventual passing of a null pointer to the >>> rbd_dev_destroy() >>> function as an indication for further source code adjustments. >> >> If all error paths could be adjusted so that NULL pointers are never passed >>

RE: Scaling Ceph reviews and testing

2015-11-26 Thread piotr.da...@ts.fujitsu.com
> So, my proposition is to postpone QA'ing performance pull > requests until someone unrelated to PR author (or even author's company) > can confirm that claims in that particular PR are true. Providing code snippet > that shows the perf difference (or provide a way to verify those claims in > repr

RE: Scaling Ceph reviews and testing

2015-11-26 Thread piotr.da...@ts.fujitsu.com
> -Original Message- > From: Podoski, Igor > Sent: Thursday, November 26, 2015 10:25 AM > > > But correctness and reliability regressions are one thing, performance > > regressions are another one. I already see PRs that promise > > performance increase, when at (my) first glance it looks

RE: Scaling Ceph reviews and testing

2015-11-26 Thread igor.podo...@ts.fujitsu.com
> -Original Message- > From: ceph-devel-ow...@vger.kernel.org [mailto:ceph-devel- > ow...@vger.kernel.org] On Behalf Of Dalek, Piotr > Sent: Thursday, November 26, 2015 9:56 AM > To: Gregory Farnum; ceph-devel > Subject: RE: Scaling Ceph reviews and testing > >

RE: Scaling Ceph reviews and testing

2015-11-26 Thread piotr.da...@ts.fujitsu.com
> -Original Message- > From: ceph-devel-ow...@vger.kernel.org [mailto:ceph-devel- > ow...@vger.kernel.org] On Behalf Of Gregory Farnum > Sent: Wednesday, November 25, 2015 11:14 PM > > It has been a long-standing requirement that all code be tested by > teuthology before being merged to mas

Re: Re: why my cluster become unavailable (min_size of pool)

2015-11-26 Thread hzwulibin
Hi, haomai Thanks for quick reply, your explain make sense for me. Thanks! -- hzwulibin 2015-11-26 - 发件人:Haomai Wang 发送日期:2015-11-26 16:00 收件人:hzwulibin 抄送:Sage Weil,ceph-devel 主题:Re

Re: why my cluster become unavailable (min_size of pool)

2015-11-26 Thread Haomai Wang
s! > > > -- > hzwulibin > 2015-11-26 > > - > 发件人:"hzwulibin" > 发送日期:2015-11-23 09:00 > 收件人:Sage Weil,Haomai Wang > 抄送:ceph-devel > 主题:Re: why my cluster become unavailable >

Re: block-rbd: One function call less in rbd_dev_probe_parent() after error detection

2015-11-25 Thread SF Markus Elfring
>> I interpreted the eventual passing of a null pointer to the rbd_dev_destroy() >> function as an indication for further source code adjustments. > > If all error paths could be adjusted so that NULL pointers are never passed > in, > destroy functions wouldn't need to have a NULL check, would th

<    1   2   3   4   5   6   7   8   9   10   >