[ceph-users] rgw: Moving index objects to the right index_pool

Hi List, we want to brush up our cluster and correct things, that have been changed over time. When we started with bobtail, we put all index objects together with data into the pool rgw.buckets: root@cephadmin:~# radosgw-admin metadata get bucket:some-bucket { "key": "bucket:some-bucket",

Re: [ceph-users] Newbie question: stretch ceph cluster

Hi, Thanks for your advice, -Original Message- From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of Luis Periquito Sent: Friday, February 09, 2018 11:34 PM To: Kai Wagner Cc: Ceph Users Subject: Re: [ceph-users] Newbie question: stretch ceph cluster On Fri, Feb 9,

Re: [ceph-users] Newbie question: stretch ceph cluster

Hi, Thanks a lot, From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of Kai Wagner Sent: Friday, February 09, 2018 11:00 PM To: ceph-users@lists.ceph.com Subject: Re: [ceph-users] Newbie question: stretch ceph cluster Hi and welcome, On 09.02.2018 15:46, ST Wong (ITSC)

Re: [ceph-users] osd crush reweight 0 on "out" OSD causes backfilling?

Step 1.5 wait until the cluster is done backfilling. CRUSH is an algorithm. By changing the weight of any object it gives every PG in the entire cluster a chance to move. This is because CRUSH is deterministic and why every client can find any inject on any osd. The amount of data that moves is

Re: [ceph-users] BlueStore & Journal

Bluestore uses BlocksDB and WAL. When you look through the Ceph documentation make sure you're looking at the proper branch for the version you are using, most likely luminous. All ceph documentation URLs are versioned with Jewel, Hammer, master, etc. There have been a lot of discussions here on

Re: [ceph-users] Deployment with Xen

Hi David!! Thanks a lot for your answer. But what happens when you have... imagine two monitors or more and one of them becomes unreponsive?. Another one is used after a timeout or... what happens when a client wants to access to some data, needs to query for that (for knowing where the info

[ceph-users] BlueStore & Journal

All; I'm sorry if this question has been asked before. I'm reading through Ceph's documentation in preparation to build a cluster, and O keep coming across the recommendation to place journals on SSDs. Does BlueStore uses journals, or was this a nod to improving XFS and BTRFS performance?

Re: [ceph-users] Help rebalancing OSD usage, Luminus 1.2.2

It may work fine, but I would suggest limiting the number of operations going on at the same time. Bryan From: Bryan Banister Date: Tuesday, February 13, 2018 at 1:16 PM To: Bryan Stillwell , Janne Johansson Cc: Ceph

Re: [ceph-users] Help rebalancing OSD usage, Luminus 1.2.2

Thanks for the response Bryan! Would it be good to go ahead and do the increase up to 4096 PGs for thee pool given that it's only at 52% done with the rebalance backfilling operations? Thanks in advance!! -Bryan -Original Message- From: Bryan Stillwell [mailto:bstillw...@godaddy.com]

Re: [ceph-users] ceph iscsi kernel 4.15 - "failed with 500"

It looks that that package was configured to auto-delete on shaman. I've submitted a fix so it shouldn't happen again in the future, but in the meantime I pushed and built python-rtslib-2.1.fb67-1 [1]. [1] https://shaman.ceph.com/repos/python-rtslib/ On Tue, Feb 13, 2018 at 2:09 PM, Steven

[ceph-users] [Off-Topic] Ceph & ARM

Hey Cephers, A hardware vendor contacted me recently to ask about companies (or contractors) able to work with them on Ceph & ARM projects for customers requiring global coverage. If you have interest on this please contact me off-list for more details. Kindest regards, Leo -- Leonardo Vaz

Re: [ceph-users] osd crush reweight 0 on "out" OSD causes backfilling?

Thanks! I'm still puzzled as to _what_ data is moving if the OSD was previously "out" and didn't host any PG (according to pg dump). The host only had one other OSD which was already "out" and had zero weight. It looks like Ceph is moving some other data, which wasn't hosted on the re-weighted

Re: [ceph-users] Deployment with Xen

Monitors are not required for accessing data from the Ceph cluster. Clients will ask a monitor for a current OSD map and then use that OSD map to communicate with the OSDs directly for all reads and writes. The map includes the crush map which has all of the information a client needs to know

[ceph-users] Deployment with Xen

Good afternoon, As I'm new to Ceph I was wondering what could be the most proper way to use it with Xen hypervisor (with a plain Linux installation, Centos, for instance). Have read the less proper one is to just mount the /dev/rbdX device in a mount point and just showing that space to the

[ceph-users] ceph iscsi kernel 4.15 - "failed with 500"

Hi, I noticed a new ceph kernel (4.15.0-ceph-g1c778f43da52) was made available so I have upgraded my test environment Now the iSCSI gateway stopped working - ERROR [rbd-target-api:1430:call_api()] - _disk change on osd02 failed with 500 So I was thinking that I have to pudate all the packages I

Re: [ceph-users] Help rebalancing OSD usage, Luminus 1.2.2

Bryan, Based off the information you've provided so far, I would say that your largest pool still doesn't have enough PGs. If you originally had only 512 PGs for you largest pool (I'm guessing .rgw.buckets has 99% of your data), then on a balanced cluster you would have just ~11.5 PGs per OSD

Re: [ceph-users] Mapping faulty pg to file on cephfs

Go to the zeroth object of the inode and there will be a “backtrace” xattr that contains the path. It will be somewhat mangled so you may need to he dump it or something. On Tue, Feb 13, 2018 at 3:14 AM Josef Zelenka < josef.zele...@cloudevelops.com> wrote: > Oh, sorry, forgot to mention - this

Re: [ceph-users] Understanding/correcting sudden onslaught of unfound objects

I'm replying to myself here, but it's probably worth mentioning that after this started, I did bring back the failed host, though with "ceph osd weight 0" to avoid more data movement. For inconsistent pgs containing unfound objects, the output of "ceph pg query" does then show the original

Re: [ceph-users] osd crush reweight 0 on "out" OSD causes backfilling?

An out osd still has a crush weight. Removing that osd or weighting it to 0 will change the weight of the host that it's in. That is why data moves again. There is a thread in the ML started by Sage about possible ways to confront the double data shift when drives fail. Data moving of when it goes

[ceph-users] [luminous12.2.2]Cache tier doesn't work properly

Hi list, I am testing cache tier in writeback mode by rados bench. The test resutl is confusing.The write performance is worse than without a cache tier. For my understanding the pool with cache tier in writeback mode should performace like all ssd pool(client get ack after data write to hot

Re: [ceph-users] NFS-Ganesha: Files disappearing?

When this happens, I see this log line from the rgw component in the FSAL: 2018-02-13 12:24:15.434086 7ff4e2ffd700  0 lookup_handle handle lookup failed <13234489286997512229,9160472602707183340>(need persistent handles) For a short time, I cannot stat the mentioned directories. After a

Re: [ceph-users] osd crush reweight 0 on "out" OSD causes backfilling?

Thanks for your input John! This doesn't really match the doc [1], which suggests just taking them out and only using "reweight" in case of issues (with small clusters). Is "reweight" considered a must before removing and OSD? Cheers On 13/02/18 12:34, John Petrini wrote: > The rule of thumb

Re: [ceph-users] osd crush reweight 0 on "out" OSD causes backfilling?

Thanks for your input John! This doesn't really match the doc [1], which suggests just taking them out and only using "reweight" in case of issues (with small clusters). Is "reweight" considered a must before removing and OSD? Cheers On 13/02/18 12:34, John Petrini wrote: > The rule of thumb

Re: [ceph-users] osd crush reweight 0 on "out" OSD causes backfilling?

The rule of thumb is to reweight to 0 prior to marking out. This should avoid causing data movement twice as you're experiencing. ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[ceph-users] osd crush reweight 0 on "out" OSD causes backfilling?

Hi all, I'm in the process of decommissioning some OSDs and thought I'd previously migrated all data off them by marking them "out" (which did trigger a fair amount of remapping as expected). Looking at the pgmap ('ceph pg dump') confirmed that none of the "out" OSDs was hosting any more PGs

Re: [ceph-users] Question about Erasure-coding clusters and resiliency

Hi Tim, With the current setup you can only handle 1 host failure without loosing any data, BUT everything will probably freeze until you bring the failed node (or the OSD"s in it) back up. Your setup indicates k=6, m=2 and all 8 shards are distributed to 4 hosts (2 shards/osds per host). Be

Re: [ceph-users] Mapping faulty pg to file on cephfs

Oh, sorry, forgot to mention - this cluster is running jewel :( On 13/02/18 12:10, John Spray wrote: On Tue, Feb 13, 2018 at 10:38 AM, Josef Zelenka wrote: Hi everyone, one of the clusters we are running for a client recently had a power outage, it's currently

Re: [ceph-users] Mapping faulty pg to file on cephfs

On Tue, Feb 13, 2018 at 10:38 AM, Josef Zelenka wrote: > Hi everyone, one of the clusters we are running for a client recently had a > power outage, it's currently in a working state, however 3 pgs were left > inconsistent atm, with this type of error in the

[ceph-users] Mapping faulty pg to file on cephfs

Hi everyone, one of the clusters we are running for a client recently had a power outage, it's currently in a working state, however 3 pgs were left inconsistent atm, with this type of error in the log(when i attempt to ceph pg repair it) 2018-02-13 09:47:17.534912 7f3735626700 -1

Re: [ceph-users] Bluestore with so many small files

Hi Behnam, On 2/12/2018 4:06 PM, Behnam Loghmani wrote: Hi there, I am using ceph Luminous 12.2.2 with: 3 osds (each osd is 100G) - no WAL/DB separation. 3 mons 1 rgw cluster size 3 I stored lots of thumbnails with very small size on ceph with radosgw. Actual size of files is something

Re: [ceph-users] rbd feature overheads

On Tue, Feb 13, 2018 at 1:24 AM, Blair Bethwaite wrote: > Thanks Ilya, > > We can probably handle ~6.2MB for a 100TB volume. Is it reasonable to expect > a librbd client such as QEMU to only hold one object-map per guest? Yes, I think so. Thanks, Ilya