[ceph-users] help me enable ceph iscsi gatewaty in ceph octopus

2020-08-04 Thread David Thuong
Please help me enable ceph iscsi gatewaty in ceph octopus . when i install ceph complete . i see iscsi gateway not enable. please help me config it ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to

[ceph-users] rados_connect timeout

2020-08-04 Thread Daniel Mezentsev
Hi all, Can somebody point me to the timeout parameter for rados_connect function. When monitors are not available it hangs undefinetely. Daniel Mezentsev, founder (+1) 604 313 8592. Soleks Data Group. Shaping the clouds. ___ ceph-users mailing

[ceph-users] help with deleting errant iscsi gateway

2020-08-04 Thread Sharad Mehrotra
Hi: I am using ceph nautilus with CentOS 7.6 and working on adding a pair of iscsi gateways in our cluster, following the documentation here: https://docs.ceph.com/docs/master/rbd/iscsi-target-cli/ I was in the "Configuring" section, step #3, "Create the iSCSI gateways" and ran into problems.

[ceph-users] Apparent bucket corruption error: get_bucket_instance_from_oid failed

2020-08-04 Thread Wesley Dillingham
Long running cluster, currently running 14.2.6 I have a certain user whose buckets have become corrupted in that the following commands: radosgw-admin bucket check --bucket radosgw-admin bucket list --bucket= return with the following: ERROR: could not init bucket: (2) No such file or

[ceph-users] RGW unable to delete a bucket

2020-08-04 Thread Andrei Mikhailovsky
Hi I am trying to delete a bucket using the following command: # radosgw-admin bucket rm --bucket= --purge-objects However, in console I get the following messages. About 100+ of those messages per second. 2020-08-04T17:11:06.411+0100 7fe64cacf080 1

[ceph-users] Re: Ceph does not recover from OSD restart

2020-08-04 Thread Frank Schilder
If with monitor log you mean the cluster log /var/log/ceph/ceph.log, I should have all of it. Please find a tgz-file here: https://files.dtu.dk/u/tFCEZJzQhH2mUIRk/logs.tgz?l (valid 100 days). Contents: logs/ceph-2020-08-03.log - cluster log for the day of restart

[ceph-users] Re: Ceph does not recover from OSD restart

2020-08-04 Thread Frank Schilder
Hi Eric, thanks for the clarification, I did misunderstand you. > You should not have to move OSDs in and out of the CRUSH tree however > in order to solve any data placement problems (This is the baffling part). Exactly. Should I create a tracker issue? I think this is not hard to reproduce

[ceph-users] Re: Ceph does not recover from OSD restart

2020-08-04 Thread Frank Schilder
Hi Erik, I added the disks and started the rebalancing. When I run into the issue, ca. 3 days after start of rebalancing, it was about 25% done. The cluster does not go to HEALTH_OK before the rebalancing is finished, it shows the "xxx objects misplaced" warning. The OSD crush locations for

[ceph-users] Re: block.db/block.wal device performance dropped after upgrade to 14.2.10

2020-08-04 Thread Eneko Lacunza
Hi Vladimir, What Kingston SSD model? El 4/8/20 a las 12:22, Vladimir Prokofev escribió: Here's some more insight into the issue. Looks like the load is triggered because of a snaptrim operation. We have a backup pool that serves as Openstack cinder-backup storage, performing snapshot backups

[ceph-users] Re: Ceph does not recover from OSD restart

2020-08-04 Thread Frank Schilder
Hi Eric, > Have you adjusted the min_size for pool sr-rbd-data-one-hdd Yes. For all EC pools located in datacenter ServerRoom, we currently set min_size=k=6, because we lack physical servers. Hosts ceph-21 and ceph-22 are logical but not physical, disks in these buckets are co-located such

[ceph-users] Re: Crush Map and CEPH meta data locations

2020-08-04 Thread Gregor Krmelj
Ah, really good question :) I believe it is stored locally on the monitor host. Saving the cluster map into RADOS would result in a chicken or egg problem. This is supported by the following two sections in the docs: 1.

[ceph-users] HEALTH_WARN crush map has legacy tunables (require firefly, min is hammer)

2020-08-04 Thread Mike Garza
Hi, I've been tasked with moving Jewel clusters to Nautilus. After the final upgrade Ceph Health warns about legacy tunables. On clusters running SSD's I enabled the optimal flag. Which took weeks to chug through remappings. My remaining clusters run HDD's. Does anyone have experience with using

[ceph-users] Re: Module crash has failed (Octopus)

2020-08-04 Thread Andrei Mikhailovsky
Thanks Michael. I will try it. Cheers Andrei - Original Message - > From: "Michael Fladischer" > To: "ceph-users" > Sent: Tuesday, 4 August, 2020 08:51:52 > Subject: [ceph-users] Re: Module crash has failed (Octopus) > Hi Andrei, > > Am 03.08.2020 um 16:26 schrieb Andrei

[ceph-users] Re: Ceph does not recover from OSD restart

2020-08-04 Thread Eric Smith
Do you have any monitor / OSD logs from the maintenance when the issues occurred? Original message From: Frank Schilder Date: 8/4/20 8:07 AM (GMT-05:00) To: Eric Smith , ceph-users Subject: Re: Ceph does not recover from OSD restart Hi Eric, thanks for the clarification, I

[ceph-users] Re: block.db/block.wal device performance dropped after upgrade to 14.2.10

2020-08-04 Thread Vladimir Prokofev
I really would not focus that much on a particular device model. Yes, Kingston SSDs are slower for reads, we knew that since we tested them. But that was before they were used as block.db devices, they first were intended purely as block.wal devices. This was even before bluestore actually, so

[ceph-users] Re: Crush Map and CEPH meta data locations

2020-08-04 Thread Edward kalk
Thank you Gregor for the reply. I have read that page. It does say what a Crush map is and how it’s used by monitors and OSDs, but does not say how or where the map is stored in the system. Is it replicated on all OSD, vis a distributed hidden pool? Is it stored on the local linux disk of the

[ceph-users] Re: block.db/block.wal device performance dropped after upgrade to 14.2.10

2020-08-04 Thread Vladimir Prokofev
> What Kingston SSD model? === START OF INFORMATION SECTION === Model Family: SandForce Driven SSDs Device Model: KINGSTON SE50S3100G Serial Number: LU WWN Device Id: Firmware Version: 611ABBF0 User Capacity:100,030,242,816 bytes [100 GB] Sector

[ceph-users] Re: Ceph does not recover from OSD restart

2020-08-04 Thread Eric Smith
All seems in order then - when you ran into your maintenance issue, how long was if after you added the new OSDs and did Ceph ever get to HEALTH_OK so it could trim PG history? Also did the OSDs just start back up in the wrong place in the CRUSH tree? -Original Message- From: Frank

[ceph-users] Re: Ceph does not recover from OSD restart

2020-08-04 Thread Eric Smith
Have you adjusted the min_size for pool sr-rbd-data-one-hdd at all? Also can you send the output of "ceph osd erasure-code-profile ls" and for each EC profile, "ceph osd erasure-code-profile get "? -Original Message- From: Frank Schilder Sent: Monday, August 3, 2020 11:05 AM To: Eric

[ceph-users] Re: block.db/block.wal device performance dropped after upgrade to 14.2.10

2020-08-04 Thread Vladimir Prokofev
Here's some more insight into the issue. Looks like the load is triggered because of a snaptrim operation. We have a backup pool that serves as Openstack cinder-backup storage, performing snapshot backups every night. Old backups are also deleted every night, so snaptrim is initiated. This

[ceph-users] block.db/block.wal device performance dropped after upgrade to 14.2.10

2020-08-04 Thread Vladimir Prokofev
Good day, cephers! We've recently upgraded our cluster from 14.2.8 to 14.2.10 release, also performing full system packages upgrade(Ubuntu 18.04 LTS). After that performance significantly dropped, main reason beeing that journal SSDs are now have no merges, huge queues, and increased latency.

[ceph-users] save some descriptions with rbd snapshots possible?

2020-08-04 Thread Marc Roos
Is it already possible to save some descriptions when creating an rbd snapshot? ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Avast Battery Saver

2020-08-04 Thread Avaknights14
If your system runs out of battery very rapidly then avast is the best solution for you. It is one of the most trusted battery savers that stops applications that you are not using, will speed up your operating device and will extend your device’s battery life. It also tells how much time you

[ceph-users] Re: Module crash has failed (Octopus)

2020-08-04 Thread Michael Fladischer
Hi Andrei, Am 03.08.2020 um 16:26 schrieb Andrei Mikhailovsky: Module 'crash' has failed: dictionary changed size during iteration I had the same error after upgrading to Octopus and I fixed it by stopping all MGRs, removing /var/lib/ceph/crash/posted on all MGR nodes (make a backup copy on

[ceph-users] Re: Running fstrim (discard) inside KVM machine with RBD as disk device corrupts ext4 filesystem

2020-08-04 Thread Georg Schönberger
On 03.08.20 14:56, Jason Dillaman wrote: On Mon, Aug 3, 2020 at 4:11 AM Georg Schönberger wrote: Hey Ceph users, we are currently facing some serious problems on our Ceph Cluster with libvirt (KVM), RBD devices and FSTRIM running inside VMs. The problem is right after running the fstrim