Re: [ceph-users] understanding the bluestore blob, chunk and compression params

2019-06-21 Thread Igor Fedotov
Actually there are two issues here - the first one (fixed by #28688) is unloaded OSD compression settings when OSD compression mode = none and pool one isn't none. Submitted https://github.com/ceph/ceph/pull/28688 to fix this part. The second - OSD doesn't see pool settings after restart until

Re: [ceph-users] understanding the bluestore blob, chunk and compression params

2019-06-21 Thread Igor Fedotov
On 6/20/2019 10:12 PM, Dan van der Ster wrote: I will try to reproduce with logs and create a tracker once I find the smoking gun... It's very strange -- I had the osd mode set to 'passive', and pool option set to 'force', and the osd was compressing objects for around 15 minutes. Then suddenl

Re: [ceph-users] understanding the bluestore blob, chunk and compression params

2019-06-21 Thread Dan van der Ster
http://tracker.ceph.com/issues/40480 On Thu, Jun 20, 2019 at 9:12 PM Dan van der Ster wrote: > > I will try to reproduce with logs and create a tracker once I find the > smoking gun... > > It's very strange -- I had the osd mode set to 'passive', and pool > option set to 'force', and the osd was

Re: [ceph-users] understanding the bluestore blob, chunk and compression params

2019-06-20 Thread Dan van der Ster
I will try to reproduce with logs and create a tracker once I find the smoking gun... It's very strange -- I had the osd mode set to 'passive', and pool option set to 'force', and the osd was compressing objects for around 15 minutes. Then suddenly it just stopped compressing, until I did 'ceph da

Re: [ceph-users] understanding the bluestore blob, chunk and compression params

2019-06-20 Thread Frank Schilder
Typo below, I meant "I doubled bluestore_compression_min_blob_size_hdd ..." From: Frank Schilder Sent: 20 June 2019 19:02 To: Dan van der Ster; ceph-users Subject: Re: [ceph-users] understanding the bluestore blob, chunk and compression para

Re: [ceph-users] understanding the bluestore blob, chunk and compression params

2019-06-20 Thread Igor Fedotov
On 6/20/2019 8:55 PM, Dan van der Ster wrote: On Thu, Jun 20, 2019 at 6:55 PM Igor Fedotov wrote: Hi Dan, bluestore_compression_max_blob_size is applied for objects marked with some additional hints only: if ((alloc_hints & CEPH_OSD_ALLOC_HINT_FLAG_SEQUENTIAL_READ) && (alloc_hin

Re: [ceph-users] understanding the bluestore blob, chunk and compression params

2019-06-20 Thread Dan van der Ster
On Thu, Jun 20, 2019 at 6:55 PM Igor Fedotov wrote: > > Hi Dan, > > bluestore_compression_max_blob_size is applied for objects marked with > some additional hints only: > >if ((alloc_hints & CEPH_OSD_ALLOC_HINT_FLAG_SEQUENTIAL_READ) && >(alloc_hints & CEPH_OSD_ALLOC_HINT_FLAG_RANDOM_RE

Re: [ceph-users] understanding the bluestore blob, chunk and compression params

2019-06-20 Thread Frank Schilder
June 2019 17:23:51 To: ceph-users Subject: Re: [ceph-users] understanding the bluestore blob, chunk and compression params P.S. I know this has been discussed before, but the compression_(mode|algorithm) pool options [1] seem completely broken -- With the pool mode set to force, we see that sometimes

Re: [ceph-users] understanding the bluestore blob, chunk and compression params

2019-06-20 Thread Igor Fedotov
I'd like to see more details (preferably backed with logs) on this... On 6/20/2019 6:23 PM, Dan van der Ster wrote: P.S. I know this has been discussed before, but the compression_(mode|algorithm) pool options [1] seem completely broken -- With the pool mode set to force, we see that sometimes t

Re: [ceph-users] understanding the bluestore blob, chunk and compression params

2019-06-20 Thread Igor Fedotov
Hi Dan, bluestore_compression_max_blob_size is applied for objects marked with some additional hints only:   if ((alloc_hints & CEPH_OSD_ALLOC_HINT_FLAG_SEQUENTIAL_READ) &&   (alloc_hints & CEPH_OSD_ALLOC_HINT_FLAG_RANDOM_READ) == 0 &&   (alloc_hints & (CEPH_OSD_ALLOC_HINT_FLAG_IMMUTAB

Re: [ceph-users] understanding the bluestore blob, chunk and compression params

2019-06-20 Thread Dan van der Ster
P.S. I know this has been discussed before, but the compression_(mode|algorithm) pool options [1] seem completely broken -- With the pool mode set to force, we see that sometimes the compression is invoked and sometimes it isn't. AFAICT, the only way to compress every object is to set bluestore_com

[ceph-users] understanding the bluestore blob, chunk and compression params

2019-06-20 Thread Dan van der Ster
Hi all, I'm trying to compress an rbd pool via backfilling the existing data, and the allocated space doesn't match what I expect. Here is the test: I marked osd.130 out and waited for it to erase all its data. Then I set (on the pool) compression_mode=force and compression_algorithm=zstd. Then I