[ceph-users] Re: Ceph Snapshot Children not exists / children relation broken

2020-08-03 Thread Torsten Ennenbach
Hi Jason. Well, I don't tried that, because I am afraid to break something :/ I don’t really understand what are you doing there :( Thanks anyways. Regards Torsten > Am 31.07.2020 um 16:46 schrieb Torsten Ennenbach : > > Wow, > > Thx I will try this asap. Thats a … solution… > Unfortunatel

[ceph-users] Running fstrim (discard) inside KVM machine with RBD as disk device corrupts ext4 filesystem

2020-08-03 Thread Georg Schönberger
Hey Ceph users, we are currently facing some serious problems on our Ceph Cluster with libvirt (KVM), RBD devices and FSTRIM running inside VMs. The problem is right after running the fstrim command inside the VM the ext4 filesystem is corrupted and read-only with the following error message:

[ceph-users] Re: snaptrim blocks IO on ceph nautilus

2020-08-03 Thread Manuel Lausch
Hi, the problem still exists and I don't know whats the reason and how to fix it. I figured out, that only about 20 OSDs was affected. After I did a ceph daemon osd. compact on this the problem was gone. I compacted all OSDs in the hope my issue will be fixed with this. But over the weekend I ru

[ceph-users] Re: Problems with long taking deep-scrubbing processes causing PG_NOT_DEEP_SCRUBBED

2020-08-03 Thread Carsten Grommel - Profihost AG
One way this can happen is if you have the default setting osd_scrub_during_recovery=false Seems like the default setting is active If you’ve been doing a lot of [re]balancing, drive replacements, topology changes, expansions, etc. scrubs can be starved especially if you’re doing EC

[ceph-users] Re: Problems with long taking deep-scrubbing processes causing PG_NOT_DEEP_SCRUBBED

2020-08-03 Thread Carsten Grommel - Profihost AG
Yeah we tried that already. The health_warn will remain thus it seems that this will not reset the timer. Am 31.07.20 um 19:52 schrieb c...@elchaka.de: What happen when you do start a scrub manual? Imo ceph osd deep-scrub xyz Hth Mehmet Am 31. Juli 2020 15:35:49 MESZ schrieb Carsten Grom

[ceph-users] Crush Map and CEPH meta data locations

2020-08-03 Thread Edward kalk
The metadata that tells CEPH where all data is located, my understanding is the crush map. Where is it stored, is it redundantly distributed so as to protect from node failure? What safeguards the critical cluster metadata? -Ed ___ ceph-users mailing l

[ceph-users] Re: Running fstrim (discard) inside KVM machine with RBD as disk device corrupts ext4 filesystem

2020-08-03 Thread Jason Dillaman
On Mon, Aug 3, 2020 at 4:11 AM Georg Schönberger wrote: > > Hey Ceph users, > > we are currently facing some serious problems on our Ceph Cluster with > libvirt (KVM), RBD devices and FSTRIM running inside VMs. > > The problem is right after running the fstrim command inside the VM the > ext4 file

[ceph-users] Re: Ceph does not recover from OSD restart

2020-08-03 Thread Eric Smith
Can you post the output of these commands: ceph osd pool ls detail ceph osd tree ceph osd crush rule dump -Original Message- From: Frank Schilder Sent: Monday, August 3, 2020 9:19 AM To: ceph-users Subject: [ceph-users] Re: Ceph does not recover from OSD restart After moving the newl

[ceph-users] RadosGW/Keystone intergration issues

2020-08-03 Thread Vladimir Sigunov
Hello community, Im trying to integrate ceph RadosGW with OpenStack Keystone. Everything is working as expected, but when I try to reach public buckets with public link generated in Horizon, I get a permanent error ‘NoSuchBucket’. However, this bucket and all it’s content does exists: I can access

[ceph-users] Re: Ceph does not recover from OSD restart

2020-08-03 Thread Eric Smith
You said you had to move some OSDs out and back in for Ceph to go back to normal (The OSDs you added). Which OSDs were added? -Original Message- From: Frank Schilder Sent: Monday, August 3, 2020 9:55 AM To: Eric Smith ; ceph-users Subject: Re: Ceph does not recover from OSD restart Hi

[ceph-users] HEALTH_WARN crush map has legacy tunables (require firefly, min is hammer)

2020-08-03 Thread Mike Garza
Hi, I've been tasked with moving Jewel clusters to Nautilus. After the final upgrade Ceph Health warns about legacy tunables. On clusters running SSD's I enabled the optimal flag. Which took weeks to chug through remappings. My remaining clusters run HDD's. Does anyone have experience with using t

[ceph-users] Re: mimic: much more raw used than reported

2020-08-03 Thread Frank Schilder
Hi all, quick update: looks like copying OSDs does indeed deflate the objects with partial overwrites in an EC pool again: osd df tree blue stats ID SIZEUSE alloc store 878.96.6 6.64.6 <-- old disk with inflated objects 294 111.9 1.92.0 <-- new di

[ceph-users] Ceph does not recover from OSD restart

2020-08-03 Thread Frank Schilder
Dear cephers, I have a serious issue with degraded objects after an OSD restart. The cluster was in a state of re-balancing after adding disks to each host. Before restart I had "X/Y objects misplaced". Apart from that, health was OK. I now restarted all OSDs of one host and the cluster does no

[ceph-users] Re: RadosGW/Keystone intergration issues

2020-08-03 Thread Vladimir Sigunov
Hello, community Im still investigating the RGW/Keystone integration issue. In the addition to the info below, in radosGW log-file I found For the bucket with public URL (which fails): 2020-08-03T16:26:54.317+ 7fd4d6c9a700 20 req 115 0s swift:list_bucket rgw::auth::swift::DefaultStrategy: tryi

[ceph-users] Re: Ceph does not recover from OSD restart

2020-08-03 Thread Frank Schilder
After moving the newly added OSDs out of the crush tree and back in again, I get to exactly what I want to see: cluster: id: e4ece518-f2cb-4708-b00f-b6bf511e91d9 health: HEALTH_WARN norebalance,norecover flag(s) set 53030026/1492404361 objects misplaced (3.55

[ceph-users] Re: Ceph does not recover from OSD restart

2020-08-03 Thread Frank Schilder
Hi Eric, thanks for your fast response. Below the output, shortened a bit as indicated. Disks have been added to pool 11 'sr-rbd-data-one-hdd' only, this is the only pool with remapped PGs and is also the only pool experiencing the "loss of track" to objects. Every other pool recovers from rest

[ceph-users] LDAP integration

2020-08-03 Thread jhamster
Hello, I am looking into connecting my rados gateway to LDAP and found the following documentation. https://docs.ceph.com/docs/master/radosgw/ldap-auth/ I would like to allow an LDAP group to have access to create and manage buckets. The questions I still have are the following: -Do the LDAP

[ceph-users] Re: Ceph does not recover from OSD restart

2020-08-03 Thread Frank Schilder
As a side effect of the restart, also the leader sees blocked ops that never get cleared. I need to restart the mon daemon: cluster: id: e4ece518-f2cb-4708-b00f-b6bf511e91d9 health: HEALTH_WARN noout,norebalance flag(s) set 53242005/1492479251 objects misplac

[ceph-users] Module crash has failed (Octopus)

2020-08-03 Thread Andrei Mikhailovsky
Hello everyone, I am running my Octopus 15.2.4 version and a couple of days ago noticed an ERROR state on the cluster with the following message: Module 'crash' has failed: dictionary changed size during iteration I couldn't find much info on this error. I've tried restarting the mon servers

[ceph-users] Re: Ceph does not recover from OSD restart

2020-08-03 Thread Frank Schilder
Hi Eric, the procedure for re-discovering all objects is: # Flag: norebalance ceph osd crush move osd.288 host=bb-04 ceph osd crush move osd.289 host=bb-05 ceph osd crush move osd.290 host=bb-06 ceph osd crush move osd.291 host=bb-21 ceph osd crush move osd.292 host=bb-07 ceph osd crush move osd

[ceph-users] Re: Ceph does not recover from OSD restart

2020-08-03 Thread Frank Schilder
Sorry for the many small e-mails: requested IDs in the commands, 288-296. One new OSD per host. Best regards, = Frank Schilder AIT Risø Campus Bygning 109, rum S14 From: Frank Schilder Sent: 03 August 2020 16:59:04 To: Eric Smith; ceph-us

[ceph-users] Re: Crush Map and CEPH meta data locations

2020-08-03 Thread Gregor Krmelj
The CRUSH map makes up the so called "Cluster Map", for which the Ceph Monitors maintain a master copy. This is precisely why you have multiple monitors - for high availability in case a monitor goes down. This is all explained quite well in the architecture documentation: https://docs.ceph.com

[ceph-users] Re: RadosGW/Keystone intergration issues

2020-08-03 Thread Matthew Oliver
This PR should fix it. It's been sitting in need of QA for a while. Hopefully it lands soon so we can backport it. https://github.com/ceph/ceph/pull/35984 Matt On Mon, Aug 3, 2020 at 11:42 PM Vladimir Sigunov wrote: > Hello community, > Im trying to integrate ceph RadosGW with OpenStack Keysto

[ceph-users] Epson printer making weird sound? Reach tech support by contacting Epson Customer Service.

2020-08-03 Thread mary smith
Sometimes an issue can arise when you might hear weird sounds coming from the printer due to some tech glitch. If that happens, then you can get the help by going to the tech help sites or you can call the Epson Customer Service to get the problem resolved. https://www.epsonprintersupportpro.net

[ceph-users] Talk to a Cash App Representative to revive a shut account?

2020-08-03 Thread mary smith
Here, you will get a bit of uplifting news, that weather this application deactivated your Cash app account because of any reason, so you can easily revive your shut account as well as you can also reactivate your Cash Card (Visa charge card). In this way, here are many features that make it pop

[ceph-users] Re: Running fstrim (discard) inside KVM machine with RBD as disk device corrupts ext4 filesystem

2020-08-03 Thread Georg Schönberger
On 03.08.20 14:56, Jason Dillaman wrote: On Mon, Aug 3, 2020 at 4:11 AM Georg Schönberger wrote: Hey Ceph users, we are currently facing some serious problems on our Ceph Cluster with libvirt (KVM), RBD devices and FSTRIM running inside VMs. The problem is right after running the fstrim comma