Re: How to create new empty journal?

2011-06-09 Thread Gregory Farnum
For all of these use cases you're going to want to flush the journal to permanent store to make sure you don't lose any data. 1) Shut down the cosd. 2) Start it up again with the --flush-journal argument. It will run in the foreground and flush out any data in the journal. You're all set and can

Re: Bug while creating new cluster

2011-06-09 Thread Gregory Farnum
What version of the code are you using? Master was broken for a couple hours today but should be fixed now. On Thu, Jun 9, 2011 at 6:17 PM, Dyweni - Ceph-Devel ys3fpfe2y...@dyweni.com wrote: Hi All, Just tripped over this bug: *** Caught signal (Segmentation fault) **  in thread 0xb6fff6d0

Re: Degraded PGs blocking open()?

2011-06-06 Thread Gregory Farnum
2011/6/6 Székelyi Szabolcs szeke...@niif.hu: Hi all, I have a three node ceph setup, two nodes playing all three roles (OSD, MDS, MON), and one being just a monitor (which happens to be the client I'm using the filesystem from). I want to achieve high availablity by mirroring all data

Re: ceph_tool_common_init failed

2011-06-02 Thread Gregory Farnum
2011/6/2 biyan chen riby.c...@gmail.com: 2011-06-02 16:27:35.010776 7f3f53940720 unable to authenticate as client.admin That's your problem. You don't have your keys set up properly to authenticate as the admin on those nodes. :) I suspect permissions problems are the cause of your other issues

Re: File only read, can't write, problem

2011-06-01 Thread Gregory Farnum
/]# ll total 112 drwxr-xr-x. 1 root root 0 Jun 1 20:05 1 I to execute the command : mount -o remount, rw /1 Still hint without permission 2011/5/27 Gregory Farnum gr...@hq.newdream.net: I'm not quite sure what you're saying here -- can you give a more precise description

Re: Able to mount some but not all nodes

2011-05-28 Thread Gregory Farnum
Mounting a Ceph cluster isn't quite like mounting an NFS machine. You mount by connecting to one of the monitor servers, and then you dynamically connect and disconnect to the other machines as needed. :) So if you can only connect to your first server, using the monitor address, then that's

Re: OSD crash

2011-05-27 Thread Gregory Farnum
This is an interesting one -- the invariant that assert is checking isn't too complicated (that the object lives on the RecoveryWQ's queue) and seems to hold everywhere the RecoveryWQ is called. And the functions modifying the queue are always called under the workqueue lock, and do maintenance if

Re: fs selection.

2011-05-27 Thread Gregory Farnum
You are really putting this stuff through the wringer -- nobody else hits these issues so frequently! Can you remind us of your test setup? I'd like to figure out how you're killing these FSes so badly. And just randomly, I'd try ext4 before xfs. :) -Greg On Fri, May 27, 2011 at 4:59 AM, Fyodor

Re: Quota function of radosgw

2011-05-27 Thread Gregory Farnum
On Fri, May 27, 2011 at 9:06 AM, Sylar Shen kimul...@gmail.com wrote: Hi developers, Recently I am testing radosgw about its functions. Then I am curious that will radosgw implement the function or API of quota in the future? I've checked the mail archives and found that the quota issue had

Re: Minimum Recommended Hardware

2011-05-26 Thread Gregory Farnum
On Thu, May 19, 2011 at 1:27 AM, Dyweni - Ceph-Devel ys3fpfe2y...@dyweni.com wrote: 4 x OSD  OSD1 =    P3 1Ghz, 256K Cache    2GB RAM    80GB IDE (dd_rescue can read about 45MB/s sustained)    1Gb NIC (e1000 driver)  OSD2 =    P3 550Mhz, 512K Cache    768MB RAM    40GB IDE (dd_rescue can

Re: Does anyone use IDE tool for Ceph development

2011-05-26 Thread Gregory Farnum
On Thu, May 19, 2011 at 3:56 AM, Qian Wei hzqian...@gmail.com wrote: Hello friends! I am learning ceph. Does anyone use IDE tool for Ceph development? Can you tell me something about IDE for ceph? Thanks very much for your reading my e-mail. I use Eclipse in my day-to-day work on Ceph, and it

Re: question about pg mapping

2011-05-26 Thread Gregory Farnum
On Fri, May 20, 2011 at 3:05 AM, Henry C Chang henry.cy.ch...@gmail.com wrote: Hi all, In my ceph cluster, I saw a pg membership change as below: At osdmap epch 160, pg 1.184 was mapped to [4, 6].        osdmaptool: osdmap file '160'         parsed '1.184' - 1.184        1.184 raw [4,6]

Re: libceph: get_reply unknown tid

2011-05-26 Thread Gregory Farnum
On Fri, May 20, 2011 at 10:59 AM, Fyodor Ustinov u...@ufm.su wrote: Hi! How should I respond to the message libceph: get_reply unknown tid 68724 from osd0 in syslog? That means something in the messaging layers got a bit busted, and you should generally respond by filing bug reports. :) What

Re: File only read, can't write, problem

2011-05-26 Thread Gregory Farnum
I'm not quite sure what you're saying here -- can you give a more precise description of the problem? On Tue, May 24, 2011 at 12:44 AM, biyan chen riby.c...@gmail.com wrote: mkcephfs -c /usr/local/etc/ceph/ceph.conf --allhosts --mkbtrfs -k /usr/local/etc/ceph/keyring.bin /etc/init.d/ceph -a

Re: libceph: get_reply unknown tid

2011-05-26 Thread Gregory Farnum
On Thu, May 26, 2011 at 3:30 PM, Fyodor Ustinov u...@ufm.su wrote: On 05/26/2011 08:37 PM, Gregory Farnum wrote: That means something in the messaging layers got a bit busted, and you should generally respond by filing bug reports.:)  What version are you on right now? I think we fixed some

Re: rbd create error with 0.26

2011-05-16 Thread Gregory Farnum
On Sun, May 15, 2011 at 11:44 PM, Simon Tian aixt2...@gmail.com wrote: Great!   Thx very much... It's just that reason!!! I just did rados -p data bench 60 write -t 1 -b 1024, the speed is more or less the same. So I changed the IO_SIZE to 1MB and above in my test codes, the speed became

Re: How to enable the debug symbols

2011-05-16 Thread Gregory Farnum
Did you actually re-build the code? Configure just sets up some of the scripts: 1) ./configure #(with desired environment settings) 2) ./autogen.sh 3) make #(hopefully -j8 or something) Then you'll need to make sure that the built library is the one you're linking with -- how to do this varies a

Re: The relationship between the number of cosd and memory cache

2011-05-13 Thread Gregory Farnum
On Fri, May 13, 2011 at 1:00 AM, Sylar Shen kimul...@gmail.com wrote: Hi developers, I have a question between the numbers of cosd and memory cache in a server. Here is my environment I got 15 servers as OSDes and each one has 8 cores CPU, 16GB RAM and 1TB*12 Disks. I only have 1 MDS and

Re: rados pool object listing suckage

2011-05-12 Thread Gregory Farnum
So it occurs to me that one thing we could do in the Objecter layer, to make this a bit more sane, is to optionally enable caching of the missing objects. Then the OSD could send out all the missing objects with a flag saying that they're missing, the Objecter could cache this list, go through the

Re: [PATCH 1/2] support for xattrs in libceph

2011-05-10 Thread Gregory Farnum
be misinterpreting that.  I can look at that again. On Tue, May 10, 2011 at 12:44 PM, Gregory Farnum gr...@hq.newdream.net wrote: What are the 'l'-prefixed functions for? They look like they're just duplicates except for the prefix. -Greg Ah, right. I didn't notice the different flag

Re: [PATCH 1/2] support for xattrs in libceph

2011-05-10 Thread Gregory Farnum
On Tue, May 10, 2011 at 1:34 PM, Yehuda Sadeh Weinraub yehud...@gmail.com wrote: lsetxattr and friends are known linux API functions, and have corresponding system calls with the same name. I say let's keep them as is. Heh. Should probably just ignore clueless me in that case. :) -- To

Re: [ceph-commit] ceph.git branch master updated. v0.27.1-331-gd9ea95f

2011-05-09 Thread Gregory Farnum
Ermm, doesn't rados sync do a whole tree sync or something? I think we want to maintain a simple get/set object capability and the rados tool is a pretty logical place for that. -Greg On Mon, May 9, 2011 at 3:21 PM, Colin McCabe cmcc...@ceph-tracker.dreamhostps.com wrote: The branch, master has

Re: [ceph-commit] ceph.git branch master updated. v0.27.1-331-gd9ea95f

2011-05-09 Thread Gregory Farnum
Ermm, doesn't rados sync do a whole tree sync or something? I think we want to maintain a simple get/set object capability and the rados tool is a pretty logical place for that. -Greg On Mon, May 9, 2011 at 3:21 PM, Colin McCabe cmcc...@ceph-tracker.dreamhostps.com wrote: The branch, master has

Re: [ceph-commit] ceph.git branch master updated. v0.27.1-331-gd9ea95f

2011-05-09 Thread Gregory Farnum
On Mon, May 9, 2011 at 4:24 PM, Gregory Farnum gregory.far...@dreamhost.com wrote: Ermm, doesn't rados sync do a whole tree sync or something? I think we want to maintain a simple get/set object capability and the rados tool is a pretty logical place for that. -Greg Sorry, Gmail got confused

Re: Unable to Add Monitor

2011-05-09 Thread Gregory Farnum
Ah, yeah, it sounds like you broke your mon map by trying to change the name of your active monitor. I'm pretty sure to make that work you would need to add the monitor under the new name and then remove the old name! Let us know if you run into any other trouble, you're probably touching a lot

Re: replication write speed

2011-05-08 Thread Gregory Farnum
On Sun, May 8, 2011 at 7:43 PM, Simon Tian aixt2...@gmail.com wrote: Hi folks,      I am testing the replication performance of ceph-0.26 with libceph, write 1G data in with ceph_write() and read it out with ceph_read(), rep_size    1                               2    3                    

Re: replication write speed

2011-05-08 Thread Gregory Farnum
On Sun, May 8, 2011 at 8:04 PM, Simon Tian aixt2...@gmail.com wrote: For primary copy, I think when the replication size is 3, 4, or even more, the writing speed should also near with 2 replication. Because the 2nd, 3rd, 4th, ... replication are written parallelly.  The speed I got for 3, 4

Re: rbd and Metaserver

2011-05-05 Thread Gregory Farnum
Nope, the MDS is only required for the POSIX-compliant Ceph layer. -Greg On Thursday, May 5, 2011 at 10:16 AM, Fyodor Ustinov wrote: Hi! If I only want to use rbd - requires metaserver? WBR, Fyodor. -- To unsubscribe from this list: send the line unsubscribe ceph-devel in the body

Re: Object size

2011-05-02 Thread Gregory Farnum
On Thu, Apr 28, 2011 at 4:55 PM, Zenon Panoussis ora...@provocation.net wrote: On 04/28/2011 10:02 PM, Gregory Farnum wrote: [various explanations] Thanks Greg, that's very helpful towards graspings ceph's workings. I'll put it in the wiki. The relation between these reports and your data

Re: Object size

2011-04-29 Thread Gregory Farnum
On Thu, Apr 28, 2011 at 4:55 PM, Zenon Panoussis ora...@provocation.net wrote: On 04/28/2011 10:02 PM, Gregory Farnum wrote: 1) You have 9791 MB of data in the filesystem. 2) You have (1MB - 9791 MB=) 2431MB of metadata maintaining the Ceph tree. 3) RADOS is using 2MB of disk space

Re: Object size

2011-04-28 Thread Gregory Farnum
On Thursday, April 28, 2011 at 11:51 AM, Zenon Panoussis wrote: What is the current default object size? A paper from 2004 by Sage et al speaks of 1 MB, a later one of 8 MB and http://diaspora.gen.nz/~rodgerd/archives/1219-Ceph.html says 4 MB. Is there a way to configure it? And is there any

Re: ceph block pointer abstractions

2011-04-27 Thread Gregory Farnum
On Wednesday, April 27, 2011 at 12:42 PM, Fabio Kaminski wrote: Ok, thats the way it should be.. :) But specializing a little more the question.. whats the data partition scheme between nodes, how can a user control it.. block level? file level? suppose that i have agregations that i want it

Re: Better debug log levels for autotest jobs

2011-04-27 Thread Gregory Farnum
Looking at what we have here, we are unlikely to need: auth, paxos, monc above 5 (or probably 1). The code these run is pretty stable and hasn't exhibited significant issues in a long time IIRC. Unfortunately the MDS is one we're likely to want to keep high, but I suppose that with debug mds 10

Re: long object names

2011-04-21 Thread Gregory Farnum
I really don't see how pushing the naming complexity into the local filesystem, where it adds lots of otherwise-useless inodes and dentries, is going to help us. I like what Yehuda has here for its relative simplicity -- though I think we should just up the hash size enough that we don't need

Re: libceph API

2011-04-20 Thread Gregory Farnum
On Wednesday, April 20, 2011 at 12:02 PM, Colin McCabe wrote: Hi guys, I pushed a branch with a revised libceph API. The main changes: * similar appearance to librados. It should be intuitive if you've used librados. * The new API allows multiple ceph clusters to be open at once (although

Re: Suicide

2011-04-19 Thread Gregory Farnum
On Tue, Apr 19, 2011 at 3:45 AM, Zenon Panoussis ora...@provocation.net wrote: It's a bit more complicated than that. While we could probably do a better job of controlling bandwidths, there are a lot of pieces devoted to handling changes in disk performance and preventing the OSD from

Re: Suicide

2011-04-18 Thread Gregory Farnum
I looked through your logs a bit and noticed that the OSD on node01 is crashing due to high latencies on disk access (I think the defaults for this case are it asserts out if there's no progress after 10 minutes or something). Based on that, I pretty much have to guess that there's just too

Re: Suicide

2011-04-18 Thread Gregory Farnum
On Monday, April 18, 2011 at 3:38 PM, Zenon Panoussis wrote: On 04/18/2011 11:21 PM, Gregory Farnum wrote: I looked through your logs a bit and noticed that the OSD on node01 is crashing due to high latencies on disk access (I think the defaults for this case are it asserts out

Re: can anyone give me some advice

2011-04-15 Thread Gregory Farnum
On Thu, Apr 14, 2011 at 8:47 PM, Qian Wei hzqian...@gmail.com wrote: Hello everyonel:        Recently, I have put up a simple test platform. Now I want to test the MDS cluster's performance,but I don't how to get it. Can anyone give me some advices or some tools may help me get it. Any kind

Re: Suicide

2011-04-15 Thread Gregory Farnum
I looked at your logs and they're very interesting. I suspect the reason your journal broke is because your logging partition filled up, although I can't be sure. We have seen a similar error once before but were unable to diagnose it because there wasn't enough logging, and it looks like we

Re: can mds prepare inodes?

2011-04-14 Thread Gregory Farnum
On Thu, Apr 14, 2011 at 2:46 AM, huang jun hjwsm1...@gmail.com wrote: hi developers,   I have a question about the prealloc_inos variable in SessionMap.h   when the client first request to allocate an inode, prepare_new_inode function will prepare new inodes to the session. so if  the

Re: osd stops

2011-04-13 Thread Gregory Farnum
On Wednesday, April 13, 2011 at 5:12 AM, Martin Wilderoth wrote: The statistics of the disks, this is after the crash of osd2 and osd4. /dev/sdc 143373312 124954676 18418636 88% /data/osd0 /dev/sdd 143373312 137639524 5733788 97% /data/osd1 /dev/sdc 143373312 120350584 23022728 84%

Re: osd stops

2011-04-12 Thread Gregory Farnum
On Tuesday, April 12, 2011 at 11:05 AM, Martin Wilderoth wrote: Thanks for the answer, now I know the reson. Some of my osd had 90% of data, dmesg also shows error with the btrfs on the hosts. I will run the test with another file system ext3 :-) or is any other filesystem better. It's a

Re: mds and cpu

2011-04-11 Thread Gregory Farnum
On Fri, Apr 8, 2011 at 4:28 PM, Fyodor Ustinov u...@ufm.su wrote: Hi. Looks as if mds uses only 1 cpu on the connection. This is so, or I made a mistake somewhere in the settings? This is basically right. The MDS has a bit of threading but most operations depend on a global lock so it's

Re: How to test HA(HighAvailability) of MDS

2011-04-11 Thread Gregory Farnum
On Monday, April 11, 2011 at 8:35 AM, AnnyRen wrote: Hi, I want to test the reliability of MDS in Ceph File System, there are three MONs, two MDSes and three OSD in my test environment. With ceph -s, I saw the MDS's status is 2011-04-11 23:30:38.001177 mds e5: 1/1/1 up {0=up:active}, 1

Re: Where I can find the definition/meanings of options parameter in /src/common/config.cc

2011-04-07 Thread Gregory Farnum
Unfortunately a lot of the config options aren't really defined or described anywhere outside the code. Some of them are described in the wiki at http://ceph.newdream.net/wiki/Ceph.conf, and possibly on other pages (use the search option). Feel free to ask about specific options you're interested

inject args and g_conf (was: Where I can find the definition/meanings...)

2011-04-07 Thread Gregory Farnum
On Thursday, April 7, 2011 at 5:21 PM, Colin McCabe wrote: I'm also aware that injectargs changes configuration values that other threads may be reading, without using a lock or an atomic update. So far, this doesn't seem to have created any problems, but it is technically incorrect and a

Re: Question about librados

2011-04-05 Thread Gregory Farnum
On Mon, Apr 4, 2011 at 11:44 PM, Rutger ter Borg rut...@terborg.net wrote: thanks for the elaborate answer! With respect to the compile error, I got this in an empty file with just including rados/librados.hpp $ g++ ./rados.cpp In file included from /usr/include/rados/buffer.h:55:0,          

Re: Question about librados

2011-04-04 Thread Gregory Farnum
On Mon, Apr 4, 2011 at 4:33 AM, Rutger ter Borg rut...@terborg.net wrote: Hello, I'm in the progress of evaluating librados as an object store. I'm using Debian's latest packages as of today, and noted that I need to define NO_ATOMIC_OPS to get something compiled that includes librados.hpp.

Re: increase duplication level decrease performance

2011-04-04 Thread Gregory Farnum
On Monday, April 4, 2011 at 2:55 PM, Fyodor Ustinov wrote: Hi. ceph osd pool set data size 1 dd if=/dev/zero of=aaa bs=1024000 count=4000 409600 bytes (4.1 GB) copied, 31.3153 s, 131 MB/s ceph osd pool set data size 2 409600 bytes (4.1 GB) copied, 72.7146 s, 56.3 MB/s ceph

Re: increase duplication level decrease performance

2011-04-04 Thread Gregory Farnum
On Monday, April 4, 2011 at 4:28 PM, Fyodor Ustinov wrote: On 04/05/2011 01:52 AM, Gregory Farnum wrote: Data replicas don't service reads unless you specifically configure it for that (it doesn't provide POSIX semantics if you do that), and each replica needs to get the data written

Re: HEALTH_WARNING

2011-04-03 Thread Gregory Farnum
On Sat, Apr 2, 2011 at 3:55 AM, Martin Wilderoth martin.wilder...@linserv.se wrote: Hello, I have seperate partitions for my osd and the btrfs file system. I also use SSD-disk for journaling. But I got problem when the root system was filled up with logfiles on one host, the file system

Re: vstart problem in master branch

2011-03-31 Thread Gregory Farnum
I think that's the user_xattr error (if you don't have user xattrs enabled on the mount). If not there were some issues on master yesterday or two days ago that I think are fixed now; somebody else should know more about what those were. -Greg On Thursday, March 31, 2011 at 10:59 AM, Noah

Re: a question about laggy mds

2011-03-24 Thread Gregory Farnum
are archived and indexed by search engines so other people can get their answer by google instead of email! :) Thanks . - Original Message - From: Gregory Farnum gregory.far...@dreamhost.com To: huang jun hjwsm1...@gmail.com Cc: ceph-devel@vger.kernel.org Sent: Thursday, March 24, 2011 10:27

Re: Object storage service beta

2011-03-23 Thread Gregory Farnum
Hi Wilfrid, If you used a promo code you should be able to create a Ceph Object Storage user. This will give you access to the playground via our S3 lookalike service, which you can set up to serve images the same way you set up Amazon S3 to do so. You can also create users to use the Ceph

Re: a question about laggy mds

2011-03-23 Thread Gregory Farnum
2011/3/23 huang jun hjwsm1...@gmail.com: hi, yes,the mds is overloaded. there are four clients in the ceph cluster,and each of them send a great many of requests to the mds. i think the standby should become active and share the workload rather than take over the laggy one in this case. what

Re: Disk allocation

2011-03-21 Thread Gregory Farnum
On Sat, Mar 19, 2011 at 11:43 PM, Martin Wilderoth martin.wilder...@linserv.se wrote: I have a small ceph cluster with 4 osd ( 2 disks on 2 hosts). I have been adding and removing files from the file system, mounted as ceph on an other host. Now I have removed most of the data on the file

Re: a problem about installion ceph on Centos 5.5

2011-03-21 Thread Gregory Farnum
You need to include the port to connect to. By default this will be 6789: [root@ceph86 /]# mount -t ceph 10.65.1.86:6789:/ /mnt/ceph That is a pretty terrible error message though; I'll put a note in the tracker! It's also possible you have other issues based on that libceph error (did you

Re: RPM issues

2011-03-21 Thread Gregory Farnum
On Mon, Mar 21, 2011 at 9:22 AM, Colin McCabe cmcc...@alumni.cmu.edu wrote: The more annoying dependencies are things like GTK2, which don't really affect the behavior of the rest of the project, but which will have to be mandatory in the RPM, it seems. Also, as I said, tcmalloc will present

File Deletion (Was: Disk allocation)

2011-03-21 Thread Gregory Farnum
On Monday, March 21, 2011 at 10:32 AM, Ben De Luca wrote: Sorry to jump into the converstation, how slow can the deletion of files actually be? One of the tests I ran a few weeks ago had me generating files, deleting them and then writing them again from a number of clients. I noticed that

Re: Disk allocation

2011-03-21 Thread Gregory Farnum
:-) - Ursprungligt meddelande - Från: Ben De Luca bdel...@gmail.com Till: Gregory Farnum gregory.far...@dreamhost.com Kopia: Martin Wilderoth martin.wilder...@linserv.se, ceph-devel@vger.kernel.org Skickat: måndag, 21 mar 2011 18:32:46 Ämne: Re: Disk allocation

Re: RE: A Few Questions about Ceph

2011-03-21 Thread Gregory Farnum
On Monday, March 21, 2011 at 4:43 AM, Wido den Hollander wrote: Hi, On Sun, 2011-03-20 at 13:51 -0500, Mark Nigh wrote: Wido, Thanks for your input. Has there be any talk/discussions around support on the upcoming release of v1.0. I see in the roadmap that is slated for April. I

Re: Wiki abuse

2011-03-18 Thread Gregory Farnum
Yeah, we put in a ticket for this a couple days ago and it's being worked on. :) -Greg On Fri, Mar 18, 2011 at 3:24 AM, Lars Täuber taeu...@bbaw.de wrote: Hello developpers, please don't mind me writing to this list. I can't find a contact for the one responsible for the wiki. Can you tell

Re: Building the Ceph client on Lucid Lynx

2011-03-18 Thread Gregory Farnum
On Friday, March 18, 2011 at 3:45 PM, Tommi Virtanen wrote: On Fri, Mar 18, 2011 at 10:23:37PM +, Steena Monteiro wrote: I'm trying to build the Ceph client on Ubuntu 10.04.2 with kernel 2.6.32-28-generic. make -C install gives: /usr/src/ceph-client-standalone/ceph/super.c:870:

Re: Building the Ceph client on Lucid Lynx

2011-03-18 Thread Gregory Farnum
On Friday, March 18, 2011 at 3:50 PM, Gregory Farnum wrote: On Friday, March 18, 2011 at 3:45 PM, Tommi Virtanen wrote: On Fri, Mar 18, 2011 at 10:23:37PM +, Steena Monteiro wrote: I'm trying to build the Ceph client on Ubuntu 10.04.2 with kernel 2.6.32-28-generic. make -C

Re: Facing some basic stability issues

2011-03-11 Thread Gregory Farnum
On Fri, Mar 11, 2011 at 6:31 AM, Ajit K Jena a...@iitb.ac.in wrote: Hi Greg, I found that the mds process dies during the weird 'untar' behavior. I am trying to analyze the logs. May be I will find something there. Can you send us the backtrace? It should be printed out at the end of the mds

Re: cosd multi-second stalls cause wrongly marked me down

2011-03-11 Thread Gregory Farnum
On Friday, March 11, 2011 at 10:51 AM, Jim Schutt wrote: I'm also confused. Here's the code I ran: void put() { //generic_dout(0) RefCountedObject::put this nref.read() - (nref.read() - 1) dendl; if (nref.dec() == 0) { utime_t s = g_clock.now(); delete this; utime_t e

Re: Facing some basic stability issues

2011-03-10 Thread Gregory Farnum
Hmmm, your conf file isn't obviously broken but you shouldn't be having any trouble with an untar, either. When the untar hangs, have you checked to make sure none of the daemons have crashed? Are they all idle, or is one of them using a lot of CPU? -Greg On Thursday, March 10, 2011 at 3:04 AM,

Re: osd cannot find cosd process after reboot

2011-03-10 Thread Gregory Farnum
Hmm, this does seem odd. How do you have Ceph installed and configured? Just to be clear: You have one machine hosting a monitor, MDS, and OSD. You reboot that machine and the MDS and monitor come back up, but the OSD doesn't. And you appear to have an OSD running somewhere else? This doesn't

Re: osd cannot find cosd process after reboot

2011-03-10 Thread Gregory Farnum
2011/3/10 huang jun hjwsm1...@gmail.com: hi Greg: here is my configure file: snip and i find something wrong  in /var/log/ceph/osd.0.log 2011-03-11 08:42:23.888496 7f207255d6f0 osd0 0 mounting /data/osd0 2011-03-11 08:42:23.888517 7f207255d6f0 filestore(/data/osd0) basedir /data/osd0 journal

Re: cosd multi-second stalls cause wrongly marked me down

2011-03-09 Thread Gregory Farnum
On Wednesday, March 9, 2011 at 8:02 AM, Jim Schutt wrote: Heh. Evidently I didn't do the instrumentation right. Or more specifically, I didn't look in the right places for the result I needed. Once I understood the code/logging enough to write a script to look for delayed osd_ping message

Re: cosd multi-second stalls cause wrongly marked me down

2011-03-09 Thread Gregory Farnum
On Wednesday, March 9, 2011 at 10:36 AM, Jim Schutt wrote: Here's another example with more debugging. The PG count during this interval is: 2011-03-09 10:35:58.306942 pg v379: 25344 pgs: 25344 active+clean; 12119 MB data, 12025 MB used, 44579 GB / 44787 GB avail 2011-03-09

Re: NFS of Gluster and Ceph

2011-03-09 Thread Gregory Farnum
Sylar: Did you run this using Ceph's FUSE or in-kernel client? If this is on cfuse, the results don't surprise me -- it's not very well optimized! If this is using the kernel client, I'd expect much faster results on the buffered write test -- I think speeds approaching the network interface

Re: Issue with replication level.

2011-03-08 Thread Gregory Farnum
On Tuesday, March 8, 2011 at 12:52 AM, Upendra Moturi wrote: Hi I am having issue witrh replication levels. Even though i have set data @2x and mds@1x replication,the data copied is occupying 4x space How are you measuring the data use as being at 4x? Keep in mind that data will be

Re: monitor dispatch queue seems backed up?

2011-03-04 Thread Gregory Farnum
On Friday, March 4, 2011 at 11:48 AM, Sage Weil wrote: Hi, I'm continuing my testing of the master branch (commit 1ed2d8c587) against 96 osds. I'm having trouble getting a new filesystem to start up. FWIW this size filesystem starts up in a minute or two at most under the stable

Re: about cpu_profiler

2011-03-04 Thread Gregory Farnum
On Friday, March 4, 2011 at 12:08 PM, Colin McCabe wrote: tl;dr: cpu_profiler may randomly crash your program on x86_64. The problem should be resolved in the future. From what I've read, it's unfortunately not out of date. However, it manifests itself as a crash so it's not the reason that

Re: monitor dispatch queue seems backed up?

2011-03-04 Thread Gregory Farnum
On Friday, March 4, 2011 at 12:53 PM, Yehuda Sadeh Weinraub wrote: you mean 'osd pg bits' in the ceph.conf? Sorry, yes. Jim, that 3 second gap is a configurable timeout/retry period; I don't remember exactly where it's set up. I suspect this failed reads problem is some kind of fallout from

Re: cosd multi-second stalls cause wrongly marked me down

2011-03-02 Thread Gregory Farnum
On Wednesday, March 2, 2011 at 3:25 PM, Jim Schutt wrote: On Wed, 2011-03-02 at 16:20 -0700, Gregory Farnum wrote: On Wed, Mar 2, 2011 at 2:57 PM, Jim Schutt jasc...@sandia.gov wrote: On Wed, 2011-03-02 at 14:59 -0700, Jim Schutt wrote: On Wed, 2011-03-02 at 14:45 -0700, Sage

Re: building Ceph on various architectures

2011-02-27 Thread Gregory Farnum
On Saturday, February 26, 2011 at 11:52 AM, Gregory Farnum wrote: On Saturday, February 26, 2011 at 11:40 AM, Laszlo Boszormenyi wrote: On Sat, 2011-02-26 at 10:28 -0800, Gregory Farnum wrote: This looks like an error that came up (and got fixed) on Ubuntu several months ago: https

Re: building Ceph on various architectures

2011-02-26 Thread Gregory Farnum
On Saturday, February 26, 2011 at 3:41 AM, Laszlo Boszormenyi wrote: Hi List, Being the Debian packager of Ceph, I ask for help about building 0.24.3 on armel. It fails with: g++ -DHAVE_CONFIG_H -I. -Wall -D__CEPH__ -D_FILE_OFFSET_BITS=64 -D_REENTRANT -D_THREAD_SAFE -rdynamic -g -O2 -MT

Re: building Ceph on various architectures

2011-02-26 Thread Gregory Farnum
On Saturday, February 26, 2011 at 11:40 AM, Laszlo Boszormenyi wrote: On Sat, 2011-02-26 at 10:28 -0800, Gregory Farnum wrote: This looks like an error that came up (and got fixed) on Ubuntu several months ago: https://bugs.launchpad.net/ubuntu/+source/ceph/+bug/635388 Yes, it looks

Re: [ceph-commit] ceph.git branch librados_api updated. v0.24.3-788-gd15fe7f

2011-02-25 Thread Gregory Farnum
On Thursday, February 24, 2011 at 6:19 PM, Colin McCabe wrote: The branch, librados_api has been updated via d15fe7f90bf488c6251be4633b104a822ebd134f (commit) from e3f3177febf07bf3b311414167fdf4ccbf07c472 (commit) Those revisions listed above that are new to this repository have not

Re: problem with ceph init scripts

2011-02-25 Thread Gregory Farnum
Sorry, we kind of lost this in the shuffle! Did you manage to resolve this issue? From what I see in your steps below, it looks like you're doing things in the wrong order -- mkcephfs should always be the first thing you do. If you really have nodes running before you do mkcephfs I suspect you

Re: remove PG

2011-02-23 Thread Gregory Farnum
On Wednesday, February 23, 2011 at 7:57 AM, H Chang wrote: Hi, I was browsing PG management code in Ceph, and have one question. It appears that an OSD could send a PG removal request to another OSD. The recipient OSD would then remove that particular PG from its pg_map. I'd like to know

Re: cosd multi-second stalls cause wrongly marked me down

2011-02-23 Thread Gregory Farnum
On Wednesday, February 23, 2011 at 9:52 AM, Jim Schutt wrote: On Thu, 2011-02-17 at 09:11 -0700, Sage Weil wrote: On Thu, 17 Feb 2011, Jim Schutt wrote: Hi Sage, On Wed, 2011-02-16 at 17:54 -0700, Sage Weil wrote: On Wed, 16 Feb 2011, Sage Weil wrote: shouldn't affect

Re: cosd multi-second stalls cause wrongly marked me down

2011-02-23 Thread Gregory Farnum
On Wednesday, February 23, 2011 at 10:54 AM, Sage Weil wrote: On Wed, 23 Feb 2011, Gregory Farnum wrote: I have managed to get OSDs wrongly marking each other down during startup when they're peering large numbers of PGs/pools, as they disagree on who they need to be heartbeating (due

Re: cosd multi-second stalls cause wrongly marked me down

2011-02-23 Thread Gregory Farnum
On Wednesday, February 23, 2011 at 11:23 AM, Jim Schutt wrote: I have managed to get OSDs wrongly marking each other down during startup when they're peering large numbers of PGs/pools, as they disagree on who they need to be heartbeating (due to the slow handling of new osd maps and pg

Re: cosd multi-second stalls cause wrongly marked me down

2011-02-18 Thread Gregory Farnum
On Thursday, February 17, 2011 at 11:13 PM, Sage Weil wrote: On Thu, 17 Feb 2011, Jim Schutt wrote: Why should it take 28 seconds to add a new timer event? Huh.. that is pretty weird. I see multiple sync in there, too, so it's not like something was somehow blocking on a btrfs commit.

Re: dedupe and compression in ceph

2011-02-18 Thread Gregory Farnum
On Friday, February 18, 2011 at 5:30 AM, Kiran Patil wrote: Hello, May I know when would it be possible to use the underlying btrfs features such as dedupe(offline) and compression for ceph cosds ? Is it possible to access btrfs features in cosd seamlessly ? Well, there are two ways to run

Re: cosd multi-second stalls cause wrongly marked me down

2011-02-16 Thread Gregory Farnum
On Wednesday, February 16, 2011 at 1:25 PM, Jim Schutt wrote: Hi, I've been testing v0.24.3 w/ 64 clients against 1 mon, 1 mds, 96 osds. Under heavy write load I see: [WRN] map e7 wrongly marked me down or wrong addr I was able to sort through the logs and discover that when this

Re: question about socket conection among osd,mds,mon

2011-02-14 Thread Gregory Farnum
On Monday, February 14, 2011 at 5:37 AM, Longguang Yue wrote: osd e35: 3 osds: 3 up, 3 in -- 2 blacklisted MDSes ceph can recover blacklisted osds by itself, but I want to recover blacklist osds as soon as possible. How to use the command, 'ceph osd blacklist rm host:port'

Re: [PATCH 17/46] fs: Use rename lock and RCU for multi-step operations

2011-02-07 Thread Gregory Farnum
On Mon, Feb 7, 2011 at 1:31 PM, Jim Schutt jasc...@sandia.gov wrote: On Mon, 2011-02-07 at 14:04 -0700, Yehuda Sadeh Weinraub wrote: It does look like this specific problem. You can try cherry-pick commit 9c3db35 off the ceph git. It is just a temporary workaround, and it wasn't tested too

Re: gcov/lcov

2011-02-01 Thread Gregory Farnum
On Tue, Feb 1, 2011 at 9:27 AM, Tommi Virtanen tommi.virta...@dreamhost.com wrote: That has been in the plans from day one. Line branch coverage are awesome tools. The first time I used Clover I think my team and I spent 2 hours picking our jaws up off the floor. They don't really discuss that

Re: Long time to start the ceph osd and mount

2011-02-01 Thread Gregory Farnum
On Mon, Jan 31, 2011 at 9:01 PM, DongJin Lee dongjin@auckland.ac.nz wrote: I'm using the unstable version dated 20th-Jan. When I was starting up multiple OSDs, e.g., 3 or more. iostat shows, after the start, the OSDs utilizes to 100% themselves (there's not much of traffic going on) If

Re: some thoughts about scrub

2011-01-31 Thread Gregory Farnum
2011/1/31 Cláudio Martins c...@ist.utl.pt:  So, I do agree that, as Brian Chrisman pointed out, background scrub is always important as it helps to prevent your data redundancy going bad without you knowing about it. I also agree with that sys. admin. notification is important in either case.

Re: 答复: question about logger.cc

2011-01-26 Thread Gregory Farnum
On Wed, Jan 26, 2011 at 5:36 PM, Longguang Yue longguang_...@tcloudcomputing.com wrote: I update 0.23.1  to 0.24.2.   I do not success in running ceph 0.24* version.!! L meet lots of error.   This is step: 1.make ceph.rpm ---ok 2.mkcephfs ;first time is

Re: Cloning rados block devices

2011-01-24 Thread Gregory Farnum
On Sun, Jan 23, 2011 at 6:07 AM, Chris Webb ch...@arachsys.com wrote: One feature I would really like to be able to export to users is an ability to make copy-on-write clones of virtual hard drives, in a Ceph context generating a new rbd image from an existing one, or from a snapshot of an

Re: how to do when data size big than osd disk.

2011-01-24 Thread Gregory Farnum
On Sun, Jan 23, 2011 at 8:28 PM, Longguang Yue longguang_...@tcloudcomputing.com wrote: 2 out of 3 is down,so the live osd is full, data lose. How to solve the case? Well, there's no way to magically make Ceph store more data than the OSD can hold. However, if you can bring the other OSDs back

Re: Ceph 0.23.2 Consolidated questions

2011-01-21 Thread Gregory Farnum
On Thu, Jan 20, 2011 at 9:13 AM, tsuna tsuna...@gmail.com wrote: On Tue, Jan 18, 2011 at 11:54 AM, Gregory Farnum gr...@hq.newdream.net wrote: Are you running on 32-bit machines? It looks like we're accidentally overflowing a 32-bit value, but I don't have any 32-bit machines handy to test

Re: ceph hang.this is my environment

2011-01-20 Thread Gregory Farnum
Based on what you've told us it looks like maybe you're running into a btrfs bug of some kind. We're working on getting the most recent version of btrfs onto some nodes here to check it out as we've been hearing about sync issues with btrfs for a few days now, but we haven't managed it yet.

<    6   7   8   9   10   11   12   >