[ceph-users] Re: mon log file grows huge

2023-07-10 Thread Ben
just rechecked debug_mon is by default 1/5. mgr/cephadm log_to_cluster level has been set to critical from debug. Wonder how to set others' level. Haven't got a clue to do that. Thanks, Ben Wesley Dillingham 于2023年7月10日周一 23:21写道: > At what level do you have logging set to for your mons? That

[ceph-users] radosgw + keystone breaks when projects have - in their names

2023-07-10 Thread Andrew Bogott
I'm in the process of adding the radosgw service to our OpenStack cloud and hoping to re-use keystone for discovery and auth. Things seem to work fine with many keystone tenants, but as soon as we try to do something in a project with a '-' in its name everything fails. Here's an example,

[ceph-users] Re: MON sync time depends on outage duration

2023-07-10 Thread Dan van der Ster
Oh yes, sounds like purging the rbd trash will be the real fix here! Good luck! __ Clyso GmbH | Ceph Support and Consulting | https://www.clyso.com On Mon, Jul 10, 2023 at 6:10 AM Eugen Block wrote: > Hi, > I got a customer response with

[ceph-users] Re: Planning cluster

2023-07-10 Thread Dan van der Ster
Hi Jan, On Sun, Jul 9, 2023 at 11:17 PM Jan Marek wrote: > Hello, > > I have a cluster, which have this configuration: > > osd pool default size = 3 > osd pool default min size = 1 > Don't use min_size = 1 during regular stable operations. Instead, use min_size = 2 to ensure data safety, and

[ceph-users] Re: RBD with PWL cache shows poor performance compared to cache device

2023-07-10 Thread Matthew Booth
On Thu, 6 Jul 2023 at 12:54, Mark Nelson wrote: > > > On 7/6/23 06:02, Matthew Booth wrote: > > On Wed, 5 Jul 2023 at 15:18, Mark Nelson wrote: > >> I'm sort of amazed that it gave you symbols without the debuginfo > >> packages installed. I'll need to figure out a way to prevent that. > >>

[ceph-users] Re: Reef release candidate - v18.1.2

2023-07-10 Thread Laura Flores
Hi Stefan, Yes, please create a tracker. I will take a look at the issue, Thanks, Laura Flores On Mon, Jul 10, 2023 at 10:50 AM Stefan Kooman wrote: > On 6/30/23 18:36, Yuri Weinstein wrote: > > > This RC has gone thru partial testing due to issues we are > > experiencing in the sepia lab. >

[ceph-users] Re: Reef release candidate - v18.1.2

2023-07-10 Thread Stefan Kooman
On 6/30/23 18:36, Yuri Weinstein wrote: This RC has gone thru partial testing due to issues we are experiencing in the sepia lab. Please try it out and report any issues you encounter. Happy testing! I tested the RC (v18.1.2) this afternoon. I tried out the new "read balancer". I hit asserts

[ceph-users] Re: mon log file grows huge

2023-07-10 Thread Wesley Dillingham
At what level do you have logging set to for your mons? That is a high volume of logs for the mon to generate. You can ask all the mons to print their debug logging level with: "ceph tell mon.* config get debug_mon" The default is 1/5 What is the overall status of your cluster? Is it healthy?

[ceph-users] Re: ceph quota qustion

2023-07-10 Thread Casey Bodley
On Mon, Jul 10, 2023 at 10:40 AM wrote: > > Hi, > > yes, this is incomplete multiparts problem. > > Then, how do admin delete the incomplete multipart object? > I mean > 1. can admin find incomplete job and incomplete multipart object? > 2. If first question is possible, then can admin delete all

[ceph-users] Re: ceph quota qustion

2023-07-10 Thread sejun21 . kim
Hi, yes, this is incomplete multiparts problem. Then, how do admin delete the incomplete multipart object? I mean 1. can admin find incomplete job and incomplete multipart object? 2. If first question is possible, then can admin delete all the job or object at once?

[ceph-users] mon log file grows huge

2023-07-10 Thread Ben
Hi, In our cluster monitors' log grows to couple GBs in days. There are quite many debug message from rocksdb, osd, mgr and mds. These should not be necessary with a well-run cluster. How could I close these logging? Thanks, Ben ___ ceph-users mailing

[ceph-users] Re: MON sync time depends on outage duration

2023-07-10 Thread Eugen Block
Hi, I got a customer response with payload size 4096, that made things even worse. The mon startup time was now around 40 minutes. My doubts wrt decreasing the payload size seem confirmed. Then I read Dan's response again which also mentions that the default payload size could be too

[ceph-users] Re: CEPH orch made osd without WAL

2023-07-10 Thread Jan Marek
Hello Eugen, Dne Po, čec 10, 2023 at 10:02:58 CEST napsal Eugen Block: > It's fine, you don't need to worry about the WAL device, it is automatically > created on the nvme if the DB is there. Having a dedicated WAL device would > only make sense if for example your data devices are on HDD, your

[ceph-users] Re: CEPH orch made osd without WAL

2023-07-10 Thread Eugen Block
It's fine, you don't need to worry about the WAL device, it is automatically created on the nvme if the DB is there. Having a dedicated WAL device would only make sense if for example your data devices are on HDD, your rocksDB on "regular" SSDs and you also have nvme devices. But since you

[ceph-users] Re: CEPH orch made osd without WAL

2023-07-10 Thread Jan Marek
Hello Eugen, I've tried to specify dedicated WAL device, but I have only /dev/nvme0n1 , so I cannot write a correct YAML file... Dne Po, čec 10, 2023 at 09:12:29 CEST napsal Eugen Block: > Yes, because you did *not* specify a dedicated WAL device. This is also > reflected in the OSD metadata: >

[ceph-users] Re: Are replicas 4 or 6 safe during network partition? Will there be split-brain?

2023-07-10 Thread Robert Sander
Hi, On 07.07.23 16:52, jcic...@cloudflare.com wrote: There are two sites, A and B. There are 5 mons, 2 in A, 3 in B. Looking at just one PG and 4 replicas, we have 2 replicas in site A and 2 replicas in site B. Site A holds the primary OSD for this PG. When a network split happens, I/O

[ceph-users] Re: CEPH orch made osd without WAL

2023-07-10 Thread Joachim Kraftmayer - ceph ambassador
you can also test it directly with ceph bench, if the WAL is on the flash device: https://www.clyso.com/blog/verify-ceph-osd-db-and-wal-setup/ Joachim ___ ceph ambassador DACH ceph consultant since 2012 Clyso GmbH - Premier Ceph Foundation Member

[ceph-users] Re: CEPH orch made osd without WAL

2023-07-10 Thread Eugen Block
Yes, because you did *not* specify a dedicated WAL device. This is also reflected in the OSD metadata: $ ceph osd metadata 6 | grep dedicated "bluefs_dedicated_db": "1", "bluefs_dedicated_wal": "0" Only if you had specified a dedicated WAL device you would see it in the lvm list

[ceph-users] Re: CEPH orch made osd without WAL

2023-07-10 Thread Jan Marek
Hello, but when I try to list devices config with ceph-volume, I can see a DB devices, but no WAL devices: ceph-volume lvm list == osd.8 === [db] /dev/ceph-5aa92e38-077b-48e2-bda6-5b7db7b7701c/osd-db-bfd11468-d109-4f85-9723-75976f51bfb9 block device

[ceph-users] Planning cluster

2023-07-10 Thread Jan Marek
Hello, I have a cluster, which have this configuration: osd pool default size = 3 osd pool default min size = 1 I have 5 monitor nodes and 7 OSD nodes. I have changed a crush map to divide ceph cluster to two datacenters - in the first one will be a part of cluster with 2 copies of data and in

[ceph-users] Re: CEPH orch made osd without WAL

2023-07-10 Thread Eugen Block
Hi, if you don't specify a different device for WAL it will be automatically colocated on the same device as the DB. So you're good with this configuration. Regards, Eugen Zitat von Jan Marek : Hello, I've tried to add to CEPH cluster OSD node with a 12 rotational disks and 1 NVMe. My

[ceph-users] CEPH orch made osd without WAL

2023-07-10 Thread Jan Marek
Hello, I've tried to add to CEPH cluster OSD node with a 12 rotational disks and 1 NVMe. My YAML was this: service_type: osd service_id: osd_spec_default service_name: osd.osd_spec_default placement: host_pattern: osd8 spec: block_db_size: 64G data_devices: rotational: 1 db_devices: