[PATCH]ECBackend: Don't directyly use get_recovery_chunk_size() in RecoveryOp::WRITING state.

2014-07-30 Thread Ma, Jianpeng
We cannot guarantee that conf-osd_recovery_max_chunk don't change when recoverying a erasure object. If change between RecoveryOp::READING and RecoveryOp::WRITING, it can cause this bug: 2014-07-30 10:12:09.599220 7f7ff26c0700 -1 osd/ECBackend.cc: In function 'void

[PATCH]ECBackend: Using ROUND_UP_TO to refactor get_recovery_chunk_size()

2014-07-30 Thread Ma, Jianpeng
Signed-off-by: Ma Jianpeng jianpeng...@intel.com --- src/osd/ECBackend.h | 6 ++ 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/src/osd/ECBackend.h b/src/osd/ECBackend.h index 28bcf8a..0e96407 100644 --- a/src/osd/ECBackend.h +++ b/src/osd/ECBackend.h @@ -149,10 +149,8 @@

Re: [PATCH 6/9] gpiolib: devres: use correct structure type name in sizeof

2014-07-30 Thread Alexandre Courbot
On Wed, Jul 30, 2014 at 12:16 AM, Julia Lawall julia.law...@lip6.fr wrote: From: Julia Lawall julia.law...@lip6.fr Correct typo in the name of the type given to sizeof. Because it is the size of a pointer that is wanted, the typo has no impact on compilation or execution. This problem was

Re: [PATCH 6/9] gpiolib: devres: use correct structure type name in sizeof

2014-07-30 Thread Julia Lawall
On Wed, 30 Jul 2014, Alexandre Courbot wrote: On Wed, Jul 30, 2014 at 12:16 AM, Julia Lawall julia.law...@lip6.fr wrote: From: Julia Lawall julia.law...@lip6.fr Correct typo in the name of the type given to sizeof. Because it is the size of a pointer that is wanted, the typo has no

Non existing monitor

2014-07-30 Thread Aanchal Agrawal
Hi, We found a case(bug?) in ceph mon code, where in, an attempt to start a non-existing monitor is throwing up a levelDB error saying failed to create new leveldb store, instead we thought an appropriate message say No Monitor present with that id would do, by checking for the monitor

Re: giant and hammer dates

2014-07-30 Thread Gregory Farnum
On Tue, Jul 29, 2014 at 7:11 PM, Sage Weil sw...@redhat.com wrote: We've talked a bit about moving to a ~4 month (instead of 3 month) cadence. I'm still inclined in this direction because it means fewer stable releases that we will be maintaining and a longer and (hopefully) more productive

Re: giant and hammer dates

2014-07-30 Thread Sage Weil
On Wed, 30 Jul 2014, Loic Dachary wrote: Hi Sage, From my (biased) point of view, the upside is that it will give me more time to complete the locally repairable code for Giant ;-). The downside is that it puts a little less pressure to improve the tools and methods that make a rapid

Re: bucket index sharding - IO throttle

2014-07-30 Thread Guang Yang
+ceph-devel. Thanks, Guang On Jul 29, 2014, at 10:20 PM, Guang Yang yguan...@outlook.com wrote: Hi Yehuda, Per you review comment in terms of IO throttling for bucket index operation, I prototyped the below code (details still need to polish), can you take a look if that is right way to

Re: giant and hammer dates

2014-07-30 Thread Loic Dachary
Hi Sage, Thanks for taking the time to write this overview of the release cycle tools and their evolutions : I did not realize so much work was going on :-) Cheers On 30/07/2014 20:22, Sage Weil wrote: On Wed, 30 Jul 2014, Loic Dachary wrote: Hi Sage, From my (biased) point of view, the

Re: fscache recursive hang -- similar to loopback NFS issues

2014-07-30 Thread Milosz Tanski
I don't think that fixing a dead lock should impose a somewhat un-explainable high latency for the for the end user (or system admin). With old drives such latencies (second plus) were not unexpected. - Milosz On Tue, Jul 29, 2014 at 10:19 PM, NeilBrown ne...@suse.de wrote: On Tue, 29 Jul 2014

Re: giant and hammer dates

2014-07-30 Thread Sage Weil
On Wed, 30 Jul 2014, Gregory Farnum wrote: On Tue, Jul 29, 2014 at 7:11 PM, Sage Weil sw...@redhat.com wrote: We've talked a bit about moving to a ~4 month (instead of 3 month) cadence. I'm still inclined in this direction because it means fewer stable releases that we will be maintaining

Re: giant and hammer dates

2014-07-30 Thread Mark Nelson
On 07/30/2014 09:22 AM, Gregory Farnum wrote: On Tue, Jul 29, 2014 at 7:11 PM, Sage Weil sw...@redhat.com wrote: We've talked a bit about moving to a ~4 month (instead of 3 month) cadence. I'm still inclined in this direction because it means fewer stable releases that we will be maintaining

Re: bucket index sharding - IO throttle

2014-07-30 Thread Yehuda Sadeh
Can you send this code through a github pull request (or at least as a patch)? It'lll be easier to review and comment. Thanks, Yehuda On Wed, Jul 30, 2014 at 7:58 AM, Guang Yang yguan...@outlook.com wrote: +ceph-devel. Thanks, Guang On Jul 29, 2014, at 10:20 PM, Guang Yang

Re: First attempt at rocksdb monitor store stress testing

2014-07-30 Thread Mark Nelson
Hi Xinxin, Yes, that did work. I was able to observe the log and figure out the stall: Too many files open in the level0-level1 compaction thread. Similar to the issue that we've seen the past with leveldb. Setting a higher ulimit fixed the problem. With leveled compaction on spinning

RE: Pull Request for ISA EC plug-in

2014-07-30 Thread Sage Weil
On Mon, 28 Jul 2014, Andreas Joachim Peters wrote: Hi Sage, I fixed that. I missed '$(srcdir)' in the assembler and C/C++ include statements (I always compiled in src dir ..) You can retry. I think there are still a few issues: make[4]: ***

Re: [ceph-users] Calamari Goes Open Source

2014-07-30 Thread John Spray
On Wed, Jul 30, 2014 at 3:40 PM, Dan Ryder (daryder) dary...@cisco.com wrote: I had similar issues, I tried many different ways to use vagrant but couldn’t build packages successfully. I’m not sure how reliable this is, but if you are looking to get Calamari packages quickly, you can skip the

RE: Pull Request for ISA EC plug-in

2014-07-30 Thread Andreas Joachim Peters
Hi Sage, my guess it comes from the yasm include option which evt. is non-standard. Mine documents '-i' and '-I' while I found on the web that probably only '-I' is the portable one. I have changed that, still I can only guess, since I cannot reproduce. Please rerun. Thanks, Andreas.

Re: giant and hammer dates

2014-07-30 Thread Eric Eastman
Personally I'd prefer a longer testing and bugfix period for named releases. 4 months of development plus 2 months of testing. I would also like to see a 6 month release scheduled for named releases. As someone trying to build a product with the Ceph software, I would like to see less, and

RE: Pull Request for ISA EC plug-in

2014-07-30 Thread Sage Weil
On Wed, 30 Jul 2014, Andreas Joachim Peters wrote: Hi Sage, my guess it comes from the yasm include option which evt. is non-standard. Mine documents '-i' and '-I' while I found on the web that probably only '-I' is the portable one. I have changed that, still I can only guess, since I

Re: Forcing Ceph into mapping all objects to a single PG

2014-07-30 Thread Daniel Hofmann
Follow up: I wrote a small Python script for evaluating the object to placement mapping's quality. I'm using a Chi-Squared statistical hypothesis test. Here is the script and also an example output for a few rados benchmark writes: https://gist.github.com/daniel-j-h/d7d87dfe5de3c5bbfd0f Maybe

RE: Pull Request for ISA EC plug-in

2014-07-30 Thread Ma, Jianpeng
Hi, At my machine, I also met this bug. But I modify this, it can work. diff --git a/src/erasure-code/isa/Makefile.am b/src/erasure-code/isa/Makefile.am index eebffea..dee593b 100644 --- a/src/erasure-code/isa/Makefile.am +++ b/src/erasure-code/isa/Makefile.am @@ -34,7 +34,7 @@

RE: First attempt at rocksdb monitor store stress testing

2014-07-30 Thread Shu, Xinxin
Hi mark, Which way do you used to set a higher limitation? use 'ulimit' command or enlarge rocksdb_max_open_files config option? Cheers, xinxin -Original Message- From: Mark Nelson [mailto:mark.nel...@inktank.com] Sent: Thursday, July 31, 2014 1:35 AM To: Shu, Xinxin;

RE: First attempt at rocksdb monitor store stress testing

2014-07-30 Thread Shu, Xinxin
Hi mark , There are four tables in your report , do you run four tests cases or only run a single case (read/write mix) , if you run a single mix case , but latency in the fourth table is not same with other three tables . In leveldb and rocksdb, level0 and level1 should be same size? to

RE: First attempt at rocksdb monitor store stress testing

2014-07-30 Thread Shu, Xinxin
Does your report base on wip-rocksdb-mark branch? Cheers, xinxin -Original Message- From: ceph-devel-ow...@vger.kernel.org [mailto:ceph-devel-ow...@vger.kernel.org] On Behalf Of Mark Nelson Sent: Tuesday, July 29, 2014 12:56 AM To: Shu, Xinxin; ceph-devel@vger.kernel.org Subject: Re:

RE: First attempt at rocksdb monitor store stress testing

2014-07-30 Thread Sage Weil
By the way, I'm getting closer to getting wip-rocksdb in a state where it can be merged, but it is failing to build due to this line: $(shell (./build_tools/build_detect_version)) in Makefile.am which results in automake: warnings are treated as errors warning: Makefile.am:59: shell

[PATCH] daemons: write pid file even when told not to daemonize

2014-07-30 Thread Alexandre Oliva
systemd wants to run daemons in foreground, but daemons wouldn't write out the pid file with -f. Fixed. Signed-off-by: Alexandre Oliva ol...@gnu.org --- src/ceph_mon.cc |3 +-- src/common/config.cc |2 -- src/global/global_init.cc | 10 +- 3 files changed, 10

[PATCH] init: exec daemons or wrappers after setting up environment

2014-07-30 Thread Alexandre Oliva
Since with systemd we want to start daemons in foreground, adding an exec before the daemon (or wrapper) name saves us a running shell just waiting for the daemon to complete. There's no reason to not do so even when not using systemd, so, done... Signed-off-by: Alexandre Oliva ol...@gnu.org ---

Re: [PATCH 2/9] CAPI: use correct structure type name in sizeof

2014-07-30 Thread David Miller
From: Julia Lawall julia.law...@lip6.fr Date: Tue, 29 Jul 2014 17:16:44 +0200 From: Julia Lawall julia.law...@lip6.fr Correct typo in the name of the type given to sizeof. Because it is the size of a pointer that is wanted, the typo has no impact on compilation or execution. This

Re: [PATCH 3/9] i40e: use correct structure type name in sizeof

2014-07-30 Thread David Miller
From: Julia Lawall julia.law...@lip6.fr Date: Tue, 29 Jul 2014 17:16:45 +0200 From: Julia Lawall julia.law...@lip6.fr Correct typo in the name of the type given to sizeof. Because it is the size of a pointer that is wanted, the typo has no impact on compilation or execution. This

Re: Pull Request for ISA EC plug-in

2014-07-30 Thread Joe Young
Hi, I also did a quick verify on my precise box, upgrading to yasm1.2 with '-i' include option can do the work, while with yasm1.1(default in precise) it is not able to work with '-i' or '-I' . Seems like we should do a check on yasm version? thanks, /j On Thu, Jul 31, 2014 at 8:43 AM, Ma,

Re: [PATCH] daemons: write pid file even when told not to daemonize

2014-07-30 Thread Loic Dachary
Hi Alexandre, With this patch ceph-osd -f will try to create the default pid file : this is a non backward compatible change. Maybe there is a way for systemd to capture the pid of the process and store it instead of requiring the deamon to create the pid file ? Cheers On 31/07/2014 08:08,

Re: [PATCH] daemons: write pid file even when told not to daemonize

2014-07-30 Thread Sage Weil
On Thu, 31 Jul 2014, Loic Dachary wrote: Hi Alexandre, With this patch ceph-osd -f will try to create the default pid file : this is a non backward compatible change. Maybe there is a way for systemd to capture the pid of the process and store it instead of requiring the deamon to create

KeyFileStore ?

2014-07-30 Thread Sage Weil
After the latest set of bug fixes to the FileStore file naming code I am newly inspired to replace it with something less complex. Right now I'm mostly thinking about HDDs, although some of this may map well onto hybrid SSD/HDD as well. It may or may not make sense for pure flash. Anyway,

RE: Pull Request for ISA EC plug-in

2014-07-30 Thread Sage Weil
On Thu, 31 Jul 2014, Ma, Jianpeng wrote: Hi, At my machine, I also met this bug. But I modify this, it can work. diff --git a/src/erasure-code/isa/Makefile.am b/src/erasure-code/isa/Makefile.am index eebffea..dee593b 100644 --- a/src/erasure-code/isa/Makefile.am +++

Re: KeyFileStore ?

2014-07-30 Thread Mark Kirkwood
On 31/07/14 17:25, Sage Weil wrote: After the latest set of bug fixes to the FileStore file naming code I am newly inspired to replace it with something less complex. Right now I'm mostly thinking about HDDs, although some of this may map well onto hybrid SSD/HDD as well. It may or may not