Re: [ceph-users] CTDB Cluster Samba on Cephfs

2013-04-16 Thread Sam Lang
On Fri, Apr 12, 2013 at 6:14 PM, Jeremy Allison j...@samba.org wrote: On Wed, Apr 03, 2013 at 03:53:58PM -0500, Sam Lang wrote: On Thu, Mar 28, 2013 at 6:32 AM, Kai Blin k...@samba.org wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 2013-03-28 09:16, Volker Lendecke wrote

[PATCH] ceph: Use pseudo-random numbers to choose mds

2013-04-10 Thread Sam Lang
We don't need to use up entropy to choose an mds, so use prandom_u32() to get a pseudo-random number. Also, we don't need to choose a random mds if only one mds is available, so add special casing for the common case. Fixes http://tracker.ceph.com/issues/3579 Signed-off-by: Sam Lang sam.l

Re: [ceph-users] CTDB Cluster Samba on Cephfs

2013-04-03 Thread Sam Lang
On Thu, Mar 28, 2013 at 6:32 AM, Kai Blin k...@samba.org wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 2013-03-28 09:16, Volker Lendecke wrote: On Wed, Mar 27, 2013 at 10:43:36PM -0700, Matthieu Patou wrote: On 03/27/2013 10:41 AM, Marco Aroldi wrote: Hi list, I'm trying to create

next branch mistakenly updated

2013-03-26 Thread Sam Lang
All, I mistakenly updated the 'next' branch with a bunch of commits from master that didn't belong in the next branch. I have force pushed next as it was before the update, but this was a non-linear change to the branch, so anyone who fetched in the interim will need to re-fetch and rebase next.

Re: Mutiple network port for a single osd

2013-03-15 Thread Sam Lang
On Wed, Mar 6, 2013 at 11:27 AM, Alex Sla 4k3...@gmail.com wrote: Hello guys, first of all, thank you for Ceph. Such an amazon technology. Great pleasure to work with. Then to my question. I review Ceph in scope of security and need to understand how the single osd is create network port

Re: Unit testing questions for FileStore::_detect_fs

2013-02-16 Thread Sam Lang
On Sun, Feb 10, 2013 at 4:36 AM, Loic Dachary l...@dachary.org wrote: Hi, On 02/06/2013 05:47 PM, Sage Weil wrote: On Wed, 6 Feb 2013, Loic Dachary wrote: Hi, The patch below adds unit tests for FileStore::_detect_fs, but it needs to run as root in order to mount the ext3, ext4 and btrfs

Re: Crash and strange things on MDS

2013-02-16 Thread Sam Lang
On Fri, Feb 15, 2013 at 7:02 PM, Kevin Decherf ke...@kdecherf.com wrote: On Wed, Feb 13, 2013 at 10:19:36AM -0800, Gregory Farnum wrote: On Wed, Feb 13, 2013 at 3:47 AM, Kevin Decherf ke...@kdecherf.com wrote: On Mon, Feb 11, 2013 at 12:25:59PM -0800, Gregory Farnum wrote: On Mon, Feb 11,

Re: [PATCH] config: Add small note about default number of PGs

2013-02-15 Thread Sam Lang
On Sat, Feb 9, 2013 at 1:55 PM, Wido den Hollander w...@42on.com wrote: From: Wido den Hollander w...@widodh.nl It's still not clear to end users this should go into the mon or global section of ceph.conf Until this gets resolved document it here as well for the people who look up their

Re: osd down (for 2 about 2 minutes) error after adding a new host to my cluster

2013-02-15 Thread Sam Lang
On Mon, Feb 11, 2013 at 7:39 PM, Isaac Otsiabah zmoo...@yahoo.com wrote: Yes, there were osd daemons running on the same node that the monitor was running on. If that is the case then i will run a test case with the monitor running on a different node where no osd is running and see what

Re: optimizing ceph-fuse performance

2013-02-13 Thread Sam Lang
On Wed, Feb 13, 2013 at 6:31 AM, femi anjorin femi.anjo...@gmail.com wrote: Hi, Pls can somebody help interprete what this configuration signify. [global] debug ms = 0 [mon] debug mon = 20 debug paxos = 20 debug auth = 20 [osd] debug osd = 20

Re: Crash and strange things on MDS

2013-02-11 Thread Sam Lang
On Mon, Feb 11, 2013 at 7:05 AM, Kevin Decherf ke...@kdecherf.com wrote: On Mon, Feb 04, 2013 at 07:01:54PM +0100, Kevin Decherf wrote: Hey everyone, It's my first post here to expose a potential issue I found today using Ceph 0.56.1. The cluster configuration is, briefly: 27 osd of ~900GB

Re: page allocation failures on osd nodes

2013-01-29 Thread Sam Lang
On Tue, Jan 29, 2013 at 2:21 PM, Andrey Korolyov and...@xdel.ru wrote: On Mon, Jan 28, 2013 at 8:55 PM, Andrey Korolyov and...@xdel.ru wrote: On Mon, Jan 28, 2013 at 5:48 PM, Sam Lang sam.l...@inktank.com wrote: On Sun, Jan 27, 2013 at 2:52 PM, Andrey Korolyov and...@xdel.ru wrote: Ahem. once

Re: page allocation failures on osd nodes

2013-01-28 Thread Sam Lang
On Sun, Jan 27, 2013 at 2:52 PM, Andrey Korolyov and...@xdel.ru wrote: Ahem. once on almost empty node same trace produced by qemu process(which was actually pinned to the specific numa node), so seems that`s generally is a some scheduler/mm bug, not directly related to the osd processes. In

Re: RadosGW performance and disk space usage

2013-01-25 Thread Sam Lang
On Thu, Jan 24, 2013 at 9:27 AM, Cesar Mello cme...@gmail.com wrote: Hi! I have successfully prototyped read/write access to ceph from Windows using the S3 API, thanks so much for the help. Now I would like to do some prototypes targeting performance evaluation. My scenario typically

Re: page allocation failures on osd nodes

2013-01-25 Thread Sam Lang
On Fri, Jan 25, 2013 at 10:07 AM, Andrey Korolyov and...@xdel.ru wrote: Sorry, I have written too less yesterday because of being sleepy. That`s obviously a cache pressure since dropping caches resulted in disappearance of this errors for a long period. I`m not very familiar with kernel memory

Re: osd down (for 2 about 2 minutes) error after adding a new host to my cluster

2013-01-25 Thread Sam Lang
On Fri, Jan 25, 2013 at 11:51 AM, Isaac Otsiabah zmoo...@yahoo.com wrote: Gregory, the network physical layout is simple, the two networks are separate. the 192.168.0 and the 192.168.1 are not subnets within a network. Hi Isaac, Could you send us your routing tables on the osds (route -n).

Re: Concepts of whole cluster snapshots/backups and backups in general.

2013-01-24 Thread Sam Lang
On Mon, Jan 21, 2013 at 1:43 PM, Michael Grosser m...@seetheprogress.com wrote: For anyone looking for a solution I want to outline the solution I will go with in my coming setup. First I wanna say I'm looking forward to the geo replication feature, which hopefully features async replication

Re: Understanding Ceph

2013-01-24 Thread Sam Lang
On Thu, Jan 24, 2013 at 9:28 AM, Dimitri Maziuk dmaz...@bmrb.wisc.edu wrote: On 1/24/2013 8:20 AM, Sam Lang wrote: Yep it means that you only have one OSD with replication level of 2. If you had a rep level of 3, you would see degraded (66.667%). If you just want to make the message go away

Re: Understanding Ceph

2013-01-24 Thread Sam Lang
On Thu, Jan 24, 2013 at 9:45 AM, Dimitri Maziuk dmaz...@bmrb.wisc.edu wrote: One other question I have left (so far) is: I read and tried to follow http://ceph.com/docs/master/install/rpm/ and http://ceph.com/docs/master/start/quick-start/ on centos 6.3. mkcephfs step fails without rbd

Re: python examples for librados

2013-01-24 Thread Sam Lang
On Wed, Jan 23, 2013 at 3:09 PM, Nick Bartos n...@pistoncloud.com wrote: I just recently found out that ceph has some python bindings (yay!). I see there are a couple of examples for using the rbd bindings here: http://ceph.com/docs/master/rbd/librbdpy/ But that doesn't really include much

Re: Understanding Ceph

2013-01-23 Thread Sam Lang
On Sun, Jan 20, 2013 at 10:39 AM, Dimitri Maziuk dmaz...@bmrb.wisc.edu wrote: On 1/19/2013 12:16 PM, Sage Weil wrote: We generally recommend the KVM+librbd route, as it is easier to manage the dependencies, and is well integrated with libvirt. FWIW this is what OpenStack and CloudStack

Re: ceph stability

2012-12-20 Thread Sam Lang
On 12/19/2012 10:02 PM, Roman Hlynovskiy wrote: Hello Sage, Yes, I can easily reproduce it. how can I open a bug on the tracker? I can't find the link for this. log is ready to be uploaded. Hi Roman, The bug tracker is at: http://tracker.newdream.net/ Once you've created an account, you

Re: lockdeps

2012-12-11 Thread Sam Lang
Also, we should probably just setup hell grind for detecting deadlocks. -sam On Dec 11, 2012, at 4:25 PM, Sam Lang sam.l...@inktank.com wrote: I've been puzzling over a failure in teuthology where lockdeps were enabled and reported a lock cycle. The output of the found cycle is below. I

lockdeps

2012-12-11 Thread Sam Lang
I've been puzzling over a failure in teuthology where lockdeps were enabled and reported a lock cycle. The output of the found cycle is below. I think the issue is actually erroneous, as it reports a found cycle, but the two dependencies that cause the cycle occur in separate threads. Its

Re: [ceph-commit] [ceph/ceph] e6a154: osx: compile on OSX

2012-12-10 Thread Sam Lang
On 12/10/2012 07:01 AM, Sage Weil wrote: On Mon, 10 Dec 2012, Jimmy Tang wrote: On 9 Dec 2012, at 18:22, Noah Watkins wrote: On Sun, Dec 9, 2012 at 10:05 AM, Gregory Farnum g...@inktank.com wrote: Oooh, very nice! Do you have a list of the dependencies that you actually needed to install?

Re: Client crash on getcwd with non-default root mount

2012-11-29 Thread Sam Lang
On 11/29/2012 01:52 PM, Noah Watkins wrote: I'm getting the assert failure below with the following test: ceph_mount(cmount, /otherdir); This should fail with ENOENT if you check the return code. -sam ceph_getcwd(cmount); -- client/Inode.h: In function 'Dentry*

Re: Client crash on getcwd with non-default root mount

2012-11-29 Thread Sam Lang
Watkins jayh...@cs.ucsc.edu wrote: Oh, let me clarify. /otherdir exists, and the mount succeeds. - Noah On Thu, Nov 29, 2012 at 11:58 AM, Sam Lang sam.l...@inktank.com wrote: On 11/29/2012 01:52 PM, Noah Watkins wrote: I'm getting the assert failure below with the following test: ceph_mount

Re: Hadoop and Ceph client/mds view of modification time

2012-11-27 Thread Sam Lang
On 11/27/2012 11:07 AM, Gregory Farnum wrote: On Tuesday, November 27, 2012 at 8:45 AM, Sam Lang wrote: Hi Noah, I was able to reproduce your issue with a similar test using the fuse client and the clock_offset option for the mds. This is what I see happening: clientA's clock is a few

Re: Hadoop and Ceph client/mds view of modification time

2012-11-27 Thread Sam Lang
On 11/27/2012 11:03 AM, Sage Weil wrote: On Tue, 27 Nov 2012, Sam Lang wrote: Hi Noah, I was able to reproduce your issue with a similar test using the fuse client and the clock_offset option for the mds. This is what I see happening: clientA's clock is a few seconds behind the mds clock

Re: Hadoop and Ceph client/mds view of modification time

2012-11-27 Thread Sam Lang
On 11/27/2012 12:01 PM, Sage Weil wrote: On Tue, 27 Nov 2012, David Zafman wrote: On Nov 27, 2012, at 9:03 AM, Sage Weil s...@inktank.com wrote: On Tue, 27 Nov 2012, Sam Lang wrote: 3. When a client acquires the cap for a file, have the mds provide its current time as well. As the client

Re: Hadoop and Ceph client/mds view of modification time

2012-11-27 Thread Sam Lang
On 11/27/2012 01:38 PM, David Zafman wrote: On Nov 27, 2012, at 11:05 AM, Sam Lang sam.l...@inktank.com wrote: On 11/27/2012 12:01 PM, Sage Weil wrote: On Tue, 27 Nov 2012, David Zafman wrote: On Nov 27, 2012, at 9:03 AM, Sage Weil s...@inktank.com wrote: On Tue, 27 Nov 2012, Sam Lang

Re: bobtail release candidates

2012-11-26 Thread Sam Lang
On 11/26/2012 07:47 AM, Wido den Hollander wrote: Hi, On 11/26/2012 01:57 AM, Sage Weil wrote: Hi all, There are automatic builds of the prerelease bobtail code available under the 'next' branch. For debs, http://ceph.com/docs/master/install/debian/#add-development-testing-packages For

Re: bobtail release candidates

2012-11-26 Thread Sam Lang
On 11/26/2012 08:36 AM, Wido den Hollander wrote: On 11/26/2012 10:26 PM, Sam Lang wrote: On 11/26/2012 07:47 AM, Wido den Hollander wrote: Hi, On 11/26/2012 01:57 AM, Sage Weil wrote: Hi all, There are automatic builds of the prerelease bobtail code available under the 'next' branch

Re: Hadoop and Ceph client/mds view of modification time

2012-11-20 Thread Sam Lang
On 11/20/2012 01:44 PM, Noah Watkins wrote: This is a description of the clock synchronization issue we are facing in Hadoop: Components of Hadoop use mtime as a versioning mechanism. Here is an example where Client B tests the expected 'version' of a file created by Client A: Client A:

objectcacher lru eviction causes assert

2012-11-19 Thread Sam Lang
Hi All, We've been fixing a number of objectcacher bugs to handle races between slow osd commit replies and various other operations like truncate. I ran into another problem earlier today with a race between an object getting evicted from the lru cache (via readx - trim) and the osd

Re: Cephfs losing files and corrupting others

2012-11-01 Thread Sam Lang
On Thu 01 Nov 2012 11:22:59 AM CDT, Nathan Howell wrote: We have a small (3 node) Ceph cluster that occasionally has issues. It loses files and directories, truncates them or fills the contents with NULL bytes. So far we haven't been able to build a repro case but it seems to happen when bulk

Re: Assertion failure in ceph_readlink()

2012-11-01 Thread Sam Lang
On 11/01/2012 05:38 PM, Noah Watkins wrote: I'm getting the following assertion failure when running a test that creates a symlink and then tries to read it using ceph_readlink(). This is the failure, and the test is shown below (and is in wip-java-symlinks). Also note that if the test below

Re: Assertion failure in ceph_readlink()

2012-11-01 Thread Sam Lang
On 11/01/2012 06:22 PM, Noah Watkins wrote: filepath path(relpath); Inode *in; - int r = path_walk(path, in); + int r = path_walk(path, in, false); if (r 0) return r; Fixes both cases. Thanks! I discovered a few more bugs in path_walk() for the symlink case while

Re: active MDS and disk write cache

2012-10-31 Thread Sam Lang
On 10/31/2012 12:02 PM, Matt Weil wrote: I have a system with a bunch or ram that I want to remain the active MDS but still have a backup. This config doesn't seem to be working. I can make linuscs92 the active by stopping and starting the mds on linuscs95. It would be nice for linuscs92 to be

Re: Proposed change to vstart.sh

2012-10-23 Thread Sam Lang
On 10/22/2012 11:28 AM, Yehuda Sadeh wrote: On Mon, Oct 22, 2012 at 9:19 AM, Sam Lang sam.l...@inktank.com wrote: On 10/22/2012 11:14 AM, Yehuda Sadeh wrote: On Mon, Oct 22, 2012 at 9:08 AM, Sam Lang sam.l...@inktank.com wrote: On 10/22/2012 10:13 AM, Joe Buck wrote: On 10/21/2012 02:35

Re: Proposed change to vstart.sh

2012-10-22 Thread Sam Lang
On 10/22/2012 10:13 AM, Joe Buck wrote: On 10/21/2012 02:35 PM, Sage Weil wrote: Also, we should fix the auth none usage at some point. :) I made some changes a while back to fix the auth none problems with vstart

Re: Proposed change to vstart.sh

2012-10-22 Thread Sam Lang
On 10/22/2012 11:14 AM, Yehuda Sadeh wrote: On Mon, Oct 22, 2012 at 9:08 AM, Sam Lang sam.l...@inktank.com wrote: On 10/22/2012 10:13 AM, Joe Buck wrote: On 10/21/2012 02:35 PM, Sage Weil wrote: Also, we should fix the auth none usage at some point. :) I made some changes a while back

Re: Help...MDS Continuously Segfaulting

2012-10-17 Thread Sam Lang
, -sam On 2012/10/17 at 07:34, Sam Lang sam.l...@inktank.com wrote: On 10/16/2012 06:04 PM, Gregory Farnum wrote: Okay, that's the right debugging but it wasn't quite as helpful on its own as I expected. Can you get a core dump (you might already have one, depending on system settings

Re: Help...MDS Continuously Segfaulting

2012-10-17 Thread Sam Lang
with -g on CXXFLAGS: ./configure CXXFLAGS=-g -sam -Nick On 2012/10/17 at 09:53, Sam Lang sam.l...@inktank.com wrote: On 10/17/2012 09:42 AM, Nick Couchman wrote: Thanks...here's the backtrace: (gdb) bt #0 0x004dcfea in ESession::replay(MDS*) () #1 0x006a2446 in MDLog

Re: chaos monkeys

2012-10-09 Thread Sam Lang
On 10/09/2012 12:16 PM, Gregory Farnum wrote: also moved to ceph-devel On Tue, Oct 9, 2012 at 9:59 AM, Sam Lang sam.l...@inktank.com wrote: On 10/09/2012 11:46 AM, Gregory Farnum wrote: On Tue, Oct 9, 2012 at 9:43 AM, Sam Lang sam.l...@inktank.com wrote: Could we add some other chaos

Re: heartbeat logic

2011-08-04 Thread Sam Lang
On 08/03/2011 11:28 PM, Sage Weil wrote: On Wed, 3 Aug 2011, Sam Lang wrote: During startup of an osd cluster with 37 osds, within the first few seconds I see osds getting marked down, even though the osd processes remain running and seem to be just fine. The up count fluctuates for a while

heartbeat logic

2011-08-03 Thread Sam Lang
During startup of an osd cluster with 37 osds, within the first few seconds I see osds getting marked down, even though the osd processes remain running and seem to be just fine. The up count fluctuates for a while but seems to stabilize eventually at around 30 up osds, while 7 or so remain

syslog problems

2011-06-15 Thread Sam Lang
In my ceph setup, I had logs being written to the default location (/var/log/ceph/) and eventually would get monitor or osd crashes because the disk would fill up with logs. So I started writing the logs to syslog, and now the local disk doesn't fill up, but I still get similar errors to

Re: syslog problems

2011-06-15 Thread Sam Lang
On 06/15/2011 09:50 AM, Wido den Hollander wrote: Hi, On Wed, 2011-06-15 at 09:26 -0500, Sam Lang wrote: In my ceph setup, I had logs being written to the default location (/var/log/ceph/) and eventually would get monitor or osd crashes because the disk would fill up with logs. So I started

Re: syslog problems

2011-06-15 Thread Sam Lang
On 06/15/2011 02:23 PM, Wido den Hollander wrote: Hi, On Wed, 2011-06-15 at 10:05 -0500, Sam Lang wrote: On 06/15/2011 09:50 AM, Wido den Hollander wrote: Hi, On Wed, 2011-06-15 at 09:26 -0500, Sam Lang wrote: In my ceph setup, I had logs being written to the default location (/var/log/ceph