[ceph-users] Question about the Ceph's performance with spdk

Hi all, I’m new in Ceph, and I wonder to know the performance report exactly about Ceph’s spdk, but I couldn’t find it. The most thing I want to know is the performance improvement before spdk and after. Thanks, Datong Li ___ ceph-users mailing list

Re: [ceph-users] Cache mode readforward mode will eat your babies?

On Thu, 8 Jun 2017 15:29:05 +1000 Brad Hubbard wrote: > On Thu, Jun 8, 2017 at 3:10 PM, Christian Balzer wrote: > > On Thu, 8 Jun 2017 14:21:43 +1000 Brad Hubbard wrote: > > > >> On Thu, Jun 8, 2017 at 1:06 PM, Christian Balzer wrote: > >> > > >> > Hello, > >> > > >> > New cluster, Jewel, se

Re: [ceph-users] Cache mode readforward mode will eat your babies?

On Thu, Jun 8, 2017 at 3:10 PM, Christian Balzer wrote: > On Thu, 8 Jun 2017 14:21:43 +1000 Brad Hubbard wrote: > >> On Thu, Jun 8, 2017 at 1:06 PM, Christian Balzer wrote: >> > >> > Hello, >> > >> > New cluster, Jewel, setting up cache-tiering: >> > --- >> > Error EPERM: 'readforward' is not a w

Re: [ceph-users] PG that should not be on undersized+degraded on multi datacenter Ceph cluster

On Thu, Jun 8, 2017 at 2:59 PM, Alejandro Comisario wrote: > ha! > is there ANY way of knowing when this peering maximum has been reached for a > PG? Not currently AFAICT. It takes place deep in this c code that is shared between the kernel and userspace implementations. https://github.com/ceph

Re: [ceph-users] Cache mode readforward mode will eat your babies?

On Thu, 8 Jun 2017 14:21:43 +1000 Brad Hubbard wrote: > On Thu, Jun 8, 2017 at 1:06 PM, Christian Balzer wrote: > > > > Hello, > > > > New cluster, Jewel, setting up cache-tiering: > > --- > > Error EPERM: 'readforward' is not a well-supported cache mode and may > > corrupt your data. pass --ye

Re: [ceph-users] PG that should not be on undersized+degraded on multi datacenter Ceph cluster

ha! is there ANY way of knowing when this peering maximum has been reached for a PG? On Jun 7, 2017 20:21, "Brad Hubbard" wrote: > On Wed, Jun 7, 2017 at 5:13 PM, Peter Maloney > wrote: > > > > > Now if only there was a log or warning seen in ceph -s that said the > > tries was exceeded, > > Ch

Re: [ceph-users] Cache mode readforward mode will eat your babies?

On Thu, Jun 8, 2017 at 1:06 PM, Christian Balzer wrote: > > Hello, > > New cluster, Jewel, setting up cache-tiering: > --- > Error EPERM: 'readforward' is not a well-supported cache mode and may corrupt > your data. pass --yes-i-really-mean-it to force. > --- > > That's new and certainly wasn't

[ceph-users] Cache mode readforward mode will eat your babies?

Hello, New cluster, Jewel, setting up cache-tiering: --- Error EPERM: 'readforward' is not a well-supported cache mode and may corrupt your data. pass --yes-i-really-mean-it to force. --- That's new and certainly wasn't there in Hammer, nor did it whine about this when upgrading my test cluste

Re: [ceph-users] PG that should not be on undersized+degraded on multi datacenter Ceph cluster

On Wed, Jun 7, 2017 at 5:13 PM, Peter Maloney wrote: > > Now if only there was a log or warning seen in ceph -s that said the > tries was exceeded, Challenge accepted. > ___ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com

[ceph-users] CephFS Snapshot questions

I have a couple of CephFS snapshot questions - Is there any functionality similar to rbd clone/flatten such that the snapshot can be made writable? Or is that as simple as copying the .snap/ to another cluster? - If the first object write since the snapid was created is a user

Re: [ceph-users] Single External Journal

SSDs don't work like that. If you have a 100GB SSD and a 10GB partition and don't use the rest of your disk... your 10GB partition isn't limited to 10% of the total SSDs speed. You can use the full speed of the SSD with the 10GB partition. There is literally no difference if you have 3x 5GB part

Re: [ceph-users] PG that should not be on undersized+degraded on multi datacenter Ceph cluster

Peter, hi ... what happened to me is exactly what happened to you, thanks so much for pointing that out! I'm amazed on how you realized that was the problem !! Maybe that will help me troubleshoot a little more pro. best. On Wed, Jun 7, 2017 at 5:06 PM, Alejandro Comisario wrote: > Peter, hi. >

Re: [ceph-users] Requests blocked in degraded erasure coded pool

> The CRUSH rule min_size is a completely different thing from the pool > min_size. If you set the pool min_size to 2 I *think* it will do what you > expect. >> If you set min_size 2 before taking the OSDs down, that does seem odd. Good to know, I got confused by the same names. I will try to se

Re: [ceph-users] PG that should not be on undersized+degraded on multi datacenter Ceph cluster

Peter, hi. thanks for the reply, let me check that out, and get back to you On Wed, Jun 7, 2017 at 4:13 AM, Peter Maloney wrote: > On 06/06/17 19:23, Alejandro Comisario wrote: >> Hi all, i have a multi datacenter 6 nodes (6 osd) ceph jewel cluster. >> There are 3 pools in the cluster, all three

Re: [ceph-users] Requests blocked in degraded erasure coded pool

On Wed, Jun 7, 2017 at 12:59 PM Jonas Jaszkowic < jonasjaszko...@googlemail.com> wrote: > If you set min_size 2 before taking the OSDs down, that does seem odd. > > > I think I don’t get the exact concept of min_size in the crush Crush > ruleset. The documentation ( > http://docs.ceph.com/docs/mas

Re: [ceph-users] Requests blocked in degraded erasure coded pool

> If you set min_size 2 before taking the OSDs down, that does seem odd. I think I don’t get the exact concept of min_size in the crush Crush ruleset. The documentation (http://docs.ceph.com/docs/master/rados/operations/crush-map/ ) s

Re: [ceph-users] Requests blocked in degraded erasure coded pool

On Wed, Jun 7, 2017 at 12:30 PM Jonas Jaszkowic < jonasjaszko...@googlemail.com> wrote: > > Am 07.06.2017 um 20:29 schrieb Gregory Farnum : > > We prevent PGs from going active (and serving writes or reads) when they > have less than "min_size" OSDs participating. This is generally set so that > w

Re: [ceph-users] Single External Journal

HI Reed, That is what I'm doing right now, separate journal SSD partition for each OSD HDD. I just thinking of possibility to have better peak write performance without consuming more SSD capacity due to possibility of using larger journal of an OSD if other OSDs don't use it at same time. For exa

[ceph-users] Single External Journal

Hi, Is it possible to have single external journal for more than one OSD without doing any partitioning or at least only single partition on journal disk? For example, I want to have single SSD as external journal for some OSD HDDs but without doing any partitioning or at least only single partiti

Re: [ceph-users] Requests blocked in degraded erasure coded pool

Whoops, sent that too early. Let me try again. On Wed, Jun 7, 2017 at 3:24 AM Jonas Jaszkowic < jonasjaszko...@googlemail.com> wrote: > Thank you for your feedback! Do you have more information on *why* at > least > k+1 nodes need to be active in order for the cluster to work at this point? > > A

Re: [ceph-users] Requests blocked in degraded erasure coded pool

[ Re-adding ceph-users ] On Wed, Jun 7, 2017 at 3:24 AM Jonas Jaszkowic < jonasjaszko...@googlemail.com> wrote: > Thank you for your feedback! Do you have more information on *why* at > least > k+1 nodes need to be active in order for the cluster to work at this point? > Actually, I misread your

Re: [ceph-users] Lumionous: bluestore 'tp_osd_tp thread tp_osd_tp' had timed out after 60

Hi Mark & List, Unfortunately, even when using yesterdays master version of ceph, I'm still seeing OSDs go down, same error as before: OSD log shows lots of entries like this: (osd38) 2017-06-07 16:48:46.070564 7f90b58c3700 1 heartbeat_map is_healthy 'tp_osd_tp thread tp_osd_tp' had timed out a

Re: [ceph-users] would rbd cascade clone affect performance?

Assuming the RBD object-map feature is *not* enabled, if the associated backing object was not overwritten in rbd2 nor rbd3, every read operation to that object would involve first attempting to read from rbd3's object, then rbd2's, followed by rbd1's, which would introduce extra latency. The first

Re: [ceph-users] Write back mode Cach-tier behavior

That's very likely what I tried to do too. Since I can't, I'll have to live with the "all in". haha Regards, Webert Lima DevOps Engineer at MAV Tecnologia *Belo Horizonte - Brasil* On Wed, Jun 7, 2017 at 2:53 AM, Christian Balzer wrote: > > Hello, > > On Tue, 6 Jun 2017 08:58:07 -0300 Webert

Re: [ceph-users] RGW: Truncated objects and bad error handling

2017-06-01 18:52 GMT+00:00 Gregory Farnum : > > > On Thu, Jun 1, 2017 at 2:03 AM Jens Rosenboom wrote: >> >> On a large Hammer-based cluster (> 1 Gobjects) we are seeing a small >> amount of objects being truncated. All of these objects are between >> 512kB and 4MB in size and they are not uploade

Re: [ceph-users] design guidance

Hello, On Tue, 6 Jun 2017 20:59:40 -0400 Daniel K wrote: > Christian, > > Thank you for the tips -- I certainly googled my eyes out for a good while > before asking -- maybe my google-fu wasn't too good last night. > > > I love using IB, alas with just one port per host you're likely best off

[ceph-users] would rbd cascade clone affect performance?

Hi all: after clone from base image snapshot,we get a new rbd named "rbd1", now we can snapshot rbd1, then clone from "rbd1" we get a new "rbd2", clone from "rbd2" we get "rbd3"... would rbd cascade clone affect performance?___ ceph-users mailing list

Re: [ceph-users] PG that should not be on undersized+degraded on multi datacenter Ceph cluster

On 06/06/17 19:23, Alejandro Comisario wrote: > Hi all, i have a multi datacenter 6 nodes (6 osd) ceph jewel cluster. > There are 3 pools in the cluster, all three with size 3 and min_size 2. > > Today, i shut down all three nodes (controlled and in order) on > datacenter "CPD2" just to validate th