[ceph-users] rbd import so slow

2014-01-09 Thread Chu Duc Minh
When using RBD backend for Openstack volume, i can easily surpass 200MB/s. But when using rbd import command, eg: # rbd import --pool test Debian.raw volume-Debian-1 --new-format --id volumes I only can import at speed ~ 30MB/s I don't know why rbd import slow? What can i do to improve import

[ceph-users] Ceph Performance

2014-01-09 Thread Bradley Kite
Hi there, I am new to Ceph and still learning its performance capabilities, but I would like to share my performance results in the hope that they are useful to others, and also to see if there is room for improvement in my setup. Firstly, a little about my setup: 3 servers (quad-core CPU, 16GB

[ceph-users] How to increase filesizelimit ?

2014-01-09 Thread Markus Goldberg
Hi, i want to use my ceph for a backup-repository holding virtual-tapes. When i copy files from my existing backup-system to ceph all files are cut at 1TB. The biggest files are arounf 5TB for now. So i'm afraid, that the actual filesizelimit is set to 1TB. How can I increase this limit ? Can

[ceph-users] backfilling after OSD marked out _and_ OSD removed

2014-01-09 Thread Dan Van Der Ster
Hi, I’m slightly confused about one thing we are observing at the moment. We’re testing the shutdown/removal of OSD servers and noticed twice as much backfilling as expected. This is what we did: 1. service ceph stop on some OSD servers. 2. ceph osd out for the above OSDs (to avoid waiting for

Re: [ceph-users] How to increase filesizelimit ?

2014-01-09 Thread Wido den Hollander
On 01/09/2014 12:06 PM, Markus Goldberg wrote: Hi, i want to use my ceph for a backup-repository holding virtual-tapes. When i copy files from my existing backup-system to ceph all files are cut at 1TB. The biggest files are arounf 5TB for now. CephFS? RBD? Or the RADOS Gateway? So i'm

[ceph-users] documentation comment

2014-01-09 Thread Wolfgang Hennerbichler
I think I found a comment in the documentation that's not inteded to be there: http://ceph.com/docs/master/rbd/rbd-snapshot/ For the rollback section, you could mention that rollback means overwriting the current version with data from a snapshot, and takes longer with larger images. So cloning

Re: [ceph-users] How to increase filesizelimit ?

2014-01-09 Thread Sage Weil
Hi Markus, There is a compile time limit of 1 tb per file in cephfs. We can increase that pretty easily. I need to check whether it can be safely switched to a configurable... sage Markus Goldberg goldb...@uni-hildesheim.de wrote: Hi, i want to use my ceph for a backup-repository holding

Re: [ceph-users] The Kraken has been released!

2014-01-09 Thread Yann ROBIN
I've installed kraken on my cluster and it works fine ! Thanks for this release :) From: ceph-users-boun...@lists.ceph.com [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of Don Talton (dotalton) Sent: jeudi 9 janvier 2014 06:31 To: ceph-us...@ceph.com Subject: [ceph-users] The Kraken has

Re: [ceph-users] The Kraken has been released!

2014-01-09 Thread Karan Singh
Excellent Stuff Donald . Thanks :-) Many Thanks Karan Singh - Original Message - From: Don Talton (dotalton) dotal...@cisco.com To: ceph-us...@ceph.com Sent: Thursday, 9 January, 2014 7:31:16 AM Subject: [ceph-users] The Kraken has been released! The first phase of Kraken

Re: [ceph-users] backfilling after OSD marked out _and_ OSD removed

2014-01-09 Thread Dan Van Der Ster
Here’s a more direct question. Given this osd tree: # ceph osd tree |head # idweight type name up/down reweight -1 2952root default -2 2952room 0513-R-0050 -3 262.1 rack RJ35 ... -14 135.8 rack RJ57 -51 0

Re: [ceph-users] ceph-deploy, single mon not in quorum

2014-01-09 Thread Travis Rhoden
HI Mordur, I'm definitely straining my memory on this one, but happy to help if I can? I'm pretty sure I did not figure it out -- you can see I didn't get any feedback from the list. What I did do, however, was uninstall everything and try the same setup with mkcephfs, which worked fine at the

Re: [ceph-users] ceph-deploy, single mon not in quorum

2014-01-09 Thread Alfredo Deza
On Thu, Jan 9, 2014 at 9:45 AM, Travis Rhoden trho...@gmail.com wrote: HI Mordur, I'm definitely straining my memory on this one, but happy to help if I can? I'm pretty sure I did not figure it out -- you can see I didn't get any feedback from the list. What I did do, however, was uninstall

Re: [ceph-users] ceph-deploy, single mon not in quorum

2014-01-09 Thread Travis Rhoden
On Thu, Jan 9, 2014 at 9:48 AM, Alfredo Deza alfredo.d...@inktank.com wrote: On Thu, Jan 9, 2014 at 9:45 AM, Travis Rhoden trho...@gmail.com wrote: HI Mordur, I'm definitely straining my memory on this one, but happy to help if I can? I'm pretty sure I did not figure it out -- you can see I

Re: [ceph-users] Ceph Performance

2014-01-09 Thread Christian Kauhaus
Am 09.01.2014 10:25, schrieb Bradley Kite: 3 servers (quad-core CPU, 16GB RAM), each with 4 SATA 7.2K RPM disks (4TB) plus a 160GB SSD. [...] By comparison, a 12-disk RAID5 iscsi SAN is doing ~4000 read iops and ~2000 iops write (but with 15KRPM SAS disks). I think that comparing Ceph on

Re: [ceph-users] ceph-deploy, single mon not in quorum

2014-01-09 Thread Mordur Ingolfsson
Hi guys, thank you very much for your feedback. I'm new to Ceph, so I ask you to be patient with my newbie-ness. I'm dealing with the same issue although I'm not using ceph-deploy. I installed manually (for learning purposes) a small test cluster of three nodes, one to host the single mon and

Re: [ceph-users] Ceph Performance

2014-01-09 Thread Bradley Kite
On 9 January 2014 15:44, Christian Kauhaus k...@gocept.com wrote: Am 09.01.2014 10:25, schrieb Bradley Kite: 3 servers (quad-core CPU, 16GB RAM), each with 4 SATA 7.2K RPM disks (4TB) plus a 160GB SSD. [...] By comparison, a 12-disk RAID5 iscsi SAN is doing ~4000 read iops and ~2000

[ceph-users] RBD kernel driver deadlock with emperor

2014-01-09 Thread Stephen Taylor
I've recently run into an issue with the RBD kernel client in emperor where I'm mapping and formatting an image, then repeatedly mounting it, writing data to it, unmounting it, and snapshotting it. Nearly every time (with only one exception so far), the driver appears to deadlock after the

Re: [ceph-users] RBD kernel driver deadlock with emperor

2014-01-09 Thread Jean-Tiare LE BIGOT
I had the same issue a couple of days ago. I fixed it by applying pending patches at https://git.kernel.org/cgit/linux/kernel/git/sage/ceph-client.git/?h=for-stable-3.10.24 According to recent mails, they should be included in next maintenance linux release 3.10.26 On 01/09/14 17:44,

Re: [ceph-users] Ceph Performance

2014-01-09 Thread Mark Nelson
On 01/09/2014 10:43 AM, Bradley Kite wrote: On 9 January 2014 15:44, Christian Kauhaus k...@gocept.com mailto:k...@gocept.com wrote: Am 09.01.2014 10:25, schrieb Bradley Kite: 3 servers (quad-core CPU, 16GB RAM), each with 4 SATA 7.2K RPM disks (4TB) plus a 160GB SSD.

Re: [ceph-users] documentation comment

2014-01-09 Thread John Wilkins
Thanks Wolfgang. The fix should be in shortly. On Thu, Jan 9, 2014 at 3:52 AM, Wolfgang Hennerbichler wo...@wogri.comwrote: I think I found a comment in the documentation that's not inteded to be there: http://ceph.com/docs/master/rbd/rbd-snapshot/ For the rollback section, you could

Re: [ceph-users] backfilling after OSD marked out _and_ OSD removed

2014-01-09 Thread Gregory Farnum
On Thu, Jan 9, 2014 at 6:27 AM, Dan Van Der Ster daniel.vanders...@cern.ch wrote: Here’s a more direct question. Given this osd tree: # ceph osd tree |head # idweight type name up/down reweight -1 2952root default -2 2952room 0513-R-0050 -3 262.1

Re: [ceph-users] backfilling after OSD marked out _and_ OSD removed

2014-01-09 Thread Dan Van Der Ster
Thanks Greg. One thought I had is that I might try just crush rm'ing the OSD instead of or just after marking it out... That should avoid the double rebalance, right? Cheers, Dan On Jan 9, 2014 7:57 PM, Gregory Farnum g...@inktank.com wrote: On Thu, Jan 9, 2014 at 6:27 AM, Dan Van Der Ster

Re: [ceph-users] The Kraken has been released!

2014-01-09 Thread David Moreau Simard
Awesome. You guys feel free to contribute to https://github.com/dmsimard/python-cephclient as well :) David Moreau Simard IT Architecture Specialist http://iweb.com On Jan 9, 2014, at 12:31 AM, Don Talton (dotalton) dotal...@cisco.com wrote: The first phase of Kraken (free) dashboard for

Re: [ceph-users] How to deploy ceph with a Debian version other than stable (Hello James Page ^o^)

2014-01-09 Thread Andrew Woodward
The ceph-deploy git master should now handle wheezy/sid, jessie/sid combinations now. On Wed, Jan 8, 2014 at 11:23 AM, Alfredo Deza alfredo.d...@inktank.comwrote: On Wed, Jan 8, 2014 at 2:00 PM, Gilles Mocellin gilles.mocel...@nuagelibre.org wrote: Le 08/01/2014 02:46, Christian Balzer a

Re: [ceph-users] ceph-deploy Platform is not supported: debian

2014-01-09 Thread Andrew Woodward
The ceph-deploy git master should be able to handle wheezy/sid jessie/sid strings now (and anything else following a/b syntax and not raise unless we have flagged the version for non-support) On Sun, Dec 1, 2013 at 6:38 PM, James Harper james.har...@bendigoit.com.auwrote: On Sun, Dec 1,

Re: [ceph-users] Ceph Performance

2014-01-09 Thread Kai Zhang
Hi Bradley, I did the similar benchmark recently, and the result is no better than yours. My setup: 3 servers(CPU: Intel Xeon E5-2609 0 @ 2.40GHz, RAM: 32GB), I used only 2 SATA 7.2k RPM disk(2 TB) plus a 400 GB SSD for OSD in total. Servers are connected with 10gbps ethernet. Replication

Re: [ceph-users] admin user for the object gateway?

2014-01-09 Thread Blair Nilsson
Emperor Mostly our evaluation is going pretty well, but things like this are driving us crazy. Less crazy then when we were evaluating swift however :) The part I am most worried about is we have no idea how to even start finding out what is happening here. The admin interface has very little

[ceph-users] Inconsistent pgs after update to 0.73 - 0.74

2014-01-09 Thread Mark Kirkwood
I've noticed this on 2 (development) clusters that I have with pools having size 1. I guess my first question would be - is this expected? Here's some detail from one of the clusters: $ ceph -v ceph version 0.74-621-g6fac2ac (6fac2acc5e6f77651ffcd7dc7aa833713517d8a6) $ ceph osd dump epoch 104

Re: [ceph-users] Inconsistent pgs after update to 0.73 - 0.74

2014-01-09 Thread David Zafman
With pool size of 1 the scrub can still do some consistency checking. These are things like missing attributes, on-disk size doesn’t match attribute size, non-clone without a head, expected clone. You could check the osd logs to see what they were. The pg below only had 1 object in error

[ceph-users] ceph安装访问问题

2014-01-09 Thread liuweid
Hello: I encountered when installing a problem can not be resolved, I hope ceph can help me, thank you very much! I installed the 0.56.4 version, 1, yum install dependencies 2, compiled 0.56.4.tar.gz 3, make make install 4 start 5,

Re: [ceph-users] ceph安装访问问题

2014-01-09 Thread hnuzhou...@gmail.com
Hi,I recommend you deploy ceph in ubuntu system and using ceph-deploy. you can find guide in http://ceph.com/docs/master/start/quick-ceph-deploy/ hnuzhou...@gmail.com 发件人: liuweid 发送时间: 2014-01-10 09:32 收件人: ceph-users 主题: [ceph-users]ceph安装访问问题 Hello: I encountered when installing a

[ceph-users] trying to understand stuck_unclean

2014-01-09 Thread YIP Wai Peng
Dear all, I have some pgs that are stuck_unclean, I'm trying to understand why. Hopefully someone can help me shed some light on it. For example, one of them is # ceph pg dump_stuck unclean 1.fa 0 0 0 0 0 0 0 active+remapped 2014-01-10 11:18:53.147842 0'0 6452:4272 [7] [7,3] 0'0 2014-01-09

Re: [ceph-users] Inconsistent pgs after update to 0.73 - 0.74

2014-01-09 Thread Mark Kirkwood
On 10/01/14 16:18, David Zafman wrote: With pool size of 1 the scrub can still do some consistency checking. These are things like missing attributes, on-disk size doesn’t match attribute size, non-clone without a head, expected clone. You could check the osd logs to see what they were.

Re: [ceph-users] trying to understand stuck_unclean

2014-01-09 Thread Wido den Hollander
On 01/10/2014 05:13 AM, YIP Wai Peng wrote: Dear all, I have some pgs that are stuck_unclean, I'm trying to understand why. Hopefully someone can help me shed some light on it. For example, one of them is # ceph pg dump_stuck unclean 1.fa000active+remapped2014-01-10

Re: [ceph-users] trying to understand stuck_unclean

2014-01-09 Thread YIP Wai Peng
Hi Wido, Thanks for the reply. I've dumped the query below. recovery_state doesn't say anything, there are also no missing or unfounded objects. What else could be wrong? - WP P.S: I am running tunables optimal already. { state: active+remapped, epoch: 6500, up: [ 7], acting: [