Re: [ceph-users] Ceph MDS randomly hangs with no useful error message

2020-01-20 Thread Yan, Zheng
On Tue, Jan 21, 2020 at 12:09 AM Janek Bevendorff wrote: > > Hi, I did as you asked and created a thread dump with GDB on the > blocking MDS. Here's the result: https://pastebin.com/pPbNvfdb > I don't find any clue from the backtrace. please run 'ceph daemon mds. dump_historic_ops' and ''ceph

Re: [ceph-users] low io with enterprise SSDs ceph luminous - can we expect more? [klartext]

2020-01-20 Thread Eric K. Miller
Hi Vitaliy, > You say you don't have access to raw drives. What does it mean? Do you > run Ceph OSDs inside VMs? In that case you should probably disable > Micron caches on the hosts, not just in VMs. Sorry, I should have been more clear. This cluster is in production, so I needed to schedule a

[ceph-users] lists and gmail

2020-01-20 Thread Sasha Litvak
It seems that people now split between new and old list servers. Regardless of either one of them, I am missing a number of messages that appear on archive pages but never seem to make to my inbox. And no they are not in my junk folder. I wonder if some of my questions are not getting a response

Re: [ceph-users] Ceph MDS randomly hangs with no useful error message

2020-01-20 Thread Janek Bevendorff
Hi, I did as you asked and created a thread dump with GDB on the blocking MDS. Here's the result: https://pastebin.com/pPbNvfdb On 17/01/2020 13:07, Yan, Zheng wrote: On Fri, Jan 17, 2020 at 4:47 PM Janek Bevendorff wrote: Hi, We have a CephFS in our cluster with 3 MDS to which > 300 client

Re: [ceph-users] CephsFS client hangs if one of mount-used MDS goes offline

2020-01-20 Thread Anton Aleksandrov
Yes, another MDS takes it over and even comes back, but client does not always "unfreeze". Weird, i see some different versions.. ceph versions {     "mon": {     "ceph version 13.2.6 (7b695f835b03642f85998b2ae7b6dd093d9fbce4) mimic (stable)": 2,     "ceph version 13.2.8 (5579a94fafbc1

Re: [ceph-users] CephsFS client hangs if one of mount-used MDS goes offline

2020-01-20 Thread Wido den Hollander
On 1/20/20 4:17 PM, Anton Aleksandrov wrote: > Hello community, > > We have very small ceph cluster of just 12 OSDs (1 per small server), 3 > MDS (one active) and 1 cephFS client. > Which version of Ceph? $ ceph versions > CephFS client is running Centos7, kernel 3.10.0-957.27.2.el7.x86_64.

[ceph-users] CephsFS client hangs if one of mount-used MDS goes offline

2020-01-20 Thread Anton Aleksandrov
Hello community, We have very small ceph cluster of just 12 OSDs (1 per small server), 3 MDS (one active) and 1 cephFS client. CephFS client is running Centos7, kernel 3.10.0-957.27.2.el7.x86_64. We created 3 MDS servers for redundancy and we mount our filesystem by connecting to 3 of them.

Re: [ceph-users] low io with enterprise SSDs ceph luminous - can we expect more? [klartext]

2020-01-20 Thread vitalif
Hi Eric, You say you don't have access to raw drives. What does it mean? Do you run Ceph OSDs inside VMs? In that case you should probably disable Micron caches on the hosts, not just in VMs. Yes, disabling the write cache only takes place upon a power cycle... or upon the next hotplug of th

[ceph-users] ceph 14.2.6 problem with default args to rbd (--name)

2020-01-20 Thread Rainer Krienke
Hello, I am fighting with rbd and CEPH_ARGS in order to make typing easier on a client. First I created a keyring on one of the ceph nodes: # ceph auth add client.rainer mon 'profile rbd' osd 'profile rbd' added key for client.rainer Then I added this keyring to /etc/ceph/ceph.keyring on a clie

Re: [ceph-users] low io with enterprise SSDs ceph luminous - can we expect more? [klartext]

2020-01-20 Thread Sasha Litvak
So hdparam -W 0 /dev/sdx doesn't work or it makes no difference? Also I am not sure I understand why it should happen before OSD have been started. At least in my experience hdparam does it to hardware regardless. On Mon, Jan 20, 2020, 2:25 AM Frank Schilder wrote: > We are using Micron 5200 PR

[ceph-users] S3 Bucket usage up 150% diference between rgw-admin and external metering tools.

2020-01-20 Thread EDH - Manuel Rios
Hi Cephs Several nodes of our Ceph 14.2.5 are fully dedicated to host cold storage / backups information. Today checking the data usage with a customer found that rgw-admin is reporting: { "bucket": "XX", "tenant": "", "zonegroup": "4d8c7c5f-ca40-4ee3-b5bb-b2cad90bd007", "pl

Re: [ceph-users] Luminous Bluestore OSDs crashing with ASSERT

2020-01-20 Thread Igor Fedotov
Hi Stefan, these lines are result of transaction dump performed on a failure during transaction submission (which is shown as "submit_transaction error: Corruption: block checksum mismatch code = 2" Most probably they are out of interest (checksum errors are unlikely to be caused by transact

Re: [ceph-users] OSD up takes 15 minutes after machine restarts

2020-01-20 Thread Igor Fedotov
No, bluestore_fsck_on_mount_deep is applied when bluestore_fsck_on_mount is set to true only. Hence there is no fsck on mount in your case. Thanks, Igor On 1/20/2020 10:25 AM, huxia...@horebdata.cn wrote: HI, Igor, does this could cause the problem? -

Re: [ceph-users] Issues with Nautilus 14.2.6 ceph-volume lvm batch --bluestore ?

2020-01-20 Thread Janne Johansson
Den mån 20 jan. 2020 kl 09:03 skrev Dave Hall : > Hello, > Since upgrading to Nautilus (+ Debian 10 Backports), when I issue > 'ceph-volume lvm batch --bluestore ' it fails with > > bluestore(/var/lib/ceph/osd/ceph-0/) _read_fsid unparsable uuid > > I previously had Luminous + Debian 9 running

Re: [ceph-users] low io with enterprise SSDs ceph luminous - can we expect more? [klartext]

2020-01-20 Thread Frank Schilder
We are using Micron 5200 PRO, 1.92TB for RBD images on KVM and are very happy with the performance. We are using EC 6+2 pools, which really eat up IOPs. Still, we get enough performance out to run 20-50 VMs per disk, which results in good space utilisation as well since our default image size is