[ceph-users] Re: Random heartbeat_map timed out

2020-12-23 Thread Igor Fedotov
Hi Seena, one of the frequent cause for such a timeout is slow RocksDB operationing. Which in turn might be caused by bluefs_buffered_io set to false and/or DB "fragmentation" after massive data removal. Hence the potential workarounds are adjusting bluefs_buffered_io and manual RocksDB comp

[ceph-users] Re: Random heartbeat_map timed out

2020-12-23 Thread Seena Fallah
I have enabled bluefs_buffered_io on some of my OSD nodes and disable some others based on the server node situation and I'm experiencing this issue on both of them! How can manual RocksDB compaction help? Can you please share with me the topic names for this issue on the mailing list? On Wed, D

[ceph-users] Re: Random heartbeat_map timed out

2020-12-26 Thread Seena Fallah
I found some strange behavior in BlueFS! It seems on each heartbeat timed out on my OSD nodes ceph_bluefs_read_prefetch_bytes will raise to 400MB/s and it's too huge because in normal it has just 20MB sometimes 1KB/s on it also client throughput is just 20MB on its disk! Is there any way to find ou