[ceph-users] Reef release candidate - v18.1.2
Hi everyone, This is the second release candidate for Reef. The Reef release comes with a new RockDB version (7.9.2) [0], which incorporates several performance improvements and features. Our internal testing doesn't show any side effects from the new version, but we are very eager to hear community feedback on it. This is the first release to have the ability to tune RockDB settings per column family [1], which allows for more granular tunings to be applied to different kinds of data stored in RocksDB. A new set of settings has been used in Reef to optimize performance for most kinds of workloads with a slight penalty in some cases, outweighed by large improvements in use cases such as RGW, in terms of compactions and write amplification. We would highly encourage community members to give these a try against their performance benchmarks and use cases. The detailed list of changes in terms of RockDB and BlueStore can be found in https://pad.ceph.com/p/reef-rc-relnotes. If any of our community members would like to help us with performance investigations or regression testing of the Reef release candidate, please feel free to provide feedback via email or in https://pad.ceph.com/p/reef_scale_testing. For more active discussions, please use the #ceph-at-scale slack channel in ceph-storage.slack.com. This RC has gone thru partial testing due to issues we are experiencing in the sepia lab. Please try it out and report any issues you encounter. Happy testing! Thanks, YuriW ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
[ceph-users] Re: [multisite] The purpose of zonegroup
cc Zac, who has been working on multisite docs in https://tracker.ceph.com/issues/58632 On Fri, Jun 30, 2023 at 11:37 AM Alexander E. Patrakov wrote: > > Thanks! This is something that should be copy-pasted at the top of > https://docs.ceph.com/en/latest/radosgw/multisite/ > > Actually, I reported a documentation bug for something very similar. > > On Fri, Jun 30, 2023 at 11:30 PM Casey Bodley wrote: > > > > you're correct that the distinction is between metadata and data; > > metadata like users and buckets will replicate to all zonegroups, > > while object data only replicates within a single zonegroup. any given > > bucket is 'owned' by the zonegroup that creates it (or overridden by > > the LocationConstraint on creation). requests for data in that bucket > > sent to other zonegroups should redirect to the zonegroup where it > > resides > > > > the ability to create multiple zonegroups can be useful in cases where > > you want some isolation for the datasets, but a shared namespace of > > users and buckets. you may have several connected sites sharing > > storage, but only require a single backup for purposes of disaster > > recovery. there it could make sense to create several zonegroups with > > only two zones each to avoid replicating all objects to all zones > > > > in other cases, it could make more sense to isolate things in separate > > realms with a single zonegroup each. zonegroups just provide some > > flexibility to control the isolation of data and metadata separately > > > > On Thu, Jun 29, 2023 at 5:48 PM Yixin Jin wrote: > > > > > > Hi folks, > > > In the multisite environment, we can get one realm that contains multiple > > > zonegroups, each in turn can have multiple zones. However, the purpose of > > > zonegroup isn't clear to me. It seems that when a user is created, its > > > metadata is synced to all zones within the same realm, regardless whether > > > they are in different zonegroups or not. The same happens to buckets. > > > Therefore, what is the purpose of having zonegroups? Wouldn't it be > > > easier to just have realm and zones? > > > Thanks,Yixin > > > ___ > > > ceph-users mailing list -- ceph-users@ceph.io > > > To unsubscribe send an email to ceph-users-le...@ceph.io > > > > > ___ > > ceph-users mailing list -- ceph-users@ceph.io > > To unsubscribe send an email to ceph-users-le...@ceph.io > > > > -- > Alexander E. Patrakov > ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
[ceph-users] Re: [multisite] The purpose of zonegroup
Thanks! This is something that should be copy-pasted at the top of https://docs.ceph.com/en/latest/radosgw/multisite/ Actually, I reported a documentation bug for something very similar. On Fri, Jun 30, 2023 at 11:30 PM Casey Bodley wrote: > > you're correct that the distinction is between metadata and data; > metadata like users and buckets will replicate to all zonegroups, > while object data only replicates within a single zonegroup. any given > bucket is 'owned' by the zonegroup that creates it (or overridden by > the LocationConstraint on creation). requests for data in that bucket > sent to other zonegroups should redirect to the zonegroup where it > resides > > the ability to create multiple zonegroups can be useful in cases where > you want some isolation for the datasets, but a shared namespace of > users and buckets. you may have several connected sites sharing > storage, but only require a single backup for purposes of disaster > recovery. there it could make sense to create several zonegroups with > only two zones each to avoid replicating all objects to all zones > > in other cases, it could make more sense to isolate things in separate > realms with a single zonegroup each. zonegroups just provide some > flexibility to control the isolation of data and metadata separately > > On Thu, Jun 29, 2023 at 5:48 PM Yixin Jin wrote: > > > > Hi folks, > > In the multisite environment, we can get one realm that contains multiple > > zonegroups, each in turn can have multiple zones. However, the purpose of > > zonegroup isn't clear to me. It seems that when a user is created, its > > metadata is synced to all zones within the same realm, regardless whether > > they are in different zonegroups or not. The same happens to buckets. > > Therefore, what is the purpose of having zonegroups? Wouldn't it be easier > > to just have realm and zones? > > Thanks,Yixin > > ___ > > ceph-users mailing list -- ceph-users@ceph.io > > To unsubscribe send an email to ceph-users-le...@ceph.io > > > ___ > ceph-users mailing list -- ceph-users@ceph.io > To unsubscribe send an email to ceph-users-le...@ceph.io -- Alexander E. Patrakov ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
[ceph-users] Re: [multisite] The purpose of zonegroup
you're correct that the distinction is between metadata and data; metadata like users and buckets will replicate to all zonegroups, while object data only replicates within a single zonegroup. any given bucket is 'owned' by the zonegroup that creates it (or overridden by the LocationConstraint on creation). requests for data in that bucket sent to other zonegroups should redirect to the zonegroup where it resides the ability to create multiple zonegroups can be useful in cases where you want some isolation for the datasets, but a shared namespace of users and buckets. you may have several connected sites sharing storage, but only require a single backup for purposes of disaster recovery. there it could make sense to create several zonegroups with only two zones each to avoid replicating all objects to all zones in other cases, it could make more sense to isolate things in separate realms with a single zonegroup each. zonegroups just provide some flexibility to control the isolation of data and metadata separately On Thu, Jun 29, 2023 at 5:48 PM Yixin Jin wrote: > > Hi folks, > In the multisite environment, we can get one realm that contains multiple > zonegroups, each in turn can have multiple zones. However, the purpose of > zonegroup isn't clear to me. It seems that when a user is created, its > metadata is synced to all zones within the same realm, regardless whether > they are in different zonegroups or not. The same happens to buckets. > Therefore, what is the purpose of having zonegroups? Wouldn't it be easier to > just have realm and zones? > Thanks,Yixin > ___ > ceph-users mailing list -- ceph-users@ceph.io > To unsubscribe send an email to ceph-users-le...@ceph.io > ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
[ceph-users] Re: Quincy osd bench in order to define osd_mclock_max_capacity_iops_[hdd|ssd]
Hi Luis,
Thank you for sharing your tricks :)
OK it's clever.
You by-pass a destroying disk fio bench with a test on a unique PG on a
single OSD, and then do some rados bench.
In this way you should get a more realistic ceph IOPS !
Rafael
Le 30/06/2023 à 15:00, Luis Domingues a écrit :
Hi Rafael.
We faced the exact same issue. And we did a bunch of tests and question.
We started with some FIOs, but results where quite meh once in production. Ceph
bench did not seemed very reliable.
What we ended up doing and seems to hold up quite nicely, is the above. It's
probably not the best but works for use.
Step 1: We put the disks we want to test in a small dev/test cluster we have.
So we can mess up with the configs, and go to the not so prod friendly way to
configure the cluster. And we can make sure no other activity other than our
test is running.
Step 2: Create a few pools, that have this particularities:
- Only 1 PG
- No replication at all. The only PG of the pool lives in only 1 disk.
- Upmap the PG to the OSD you want to test.
- Repeat for a few disks.
Step 2 needs to enable some options of ceph, as it will not allow you to do it
by default. I do not remember the exact options, but you can find them on the
documentation.
Sept 3: set mclock as high_client_ops. So mclock will virtually not limit
client ops.
Step 4: Run a few ceph rados bench on the different disks.
rados bench write -t -b -p
we used 300 secondes to be able do perfom various tests without
taking a week, but letting the cluster writing a bunch of stuff anyway.
we used 100, it worked well, and previous tests for other purposes
showed 100 was nice on our installation.
the pool with the disk you want to test
we used 128k and 4M. Feel free to experiment with other values.
But in our use case it was what we used.
Somewhere in the output of the bench, after it finishes, will be Average IOPS.
This average is more or less what the disk is capable of handling. Then we put
some number close to that one. If we have two types of disks that are close to
each others, we put the smaller value for all disks. And we set it as a global
configuration on ceph instead of going disk by disk.
It's probably not perfect and looks very like we tinkered something, but it's
the best solution for testing that so far. And most important, results between
tests where a lot more consistent than ceph bench or fio.
Hope this will help you.
Luis Domingues
Proton AG
--- Original Message ---
On Friday, June 30th, 2023 at 12:15, Rafael Diaz Maurin
wrote:
Hello,
I've just upgraded a Pacific cluster into Quincy, and all my osd have
the low value osd_mclock_max_capacity_iops_hdd : 315.00.
The manuel does not explain how to benchmark the OSD with fio or ceph
bench with good options.
Can someone have the good ceph bench options or fio options in order to
configure osd_mclock_max_capacity_iops_hdd for each osd ?
I ran this bench various times on the same OSD (class hdd) and I obtain
different results.
ceph tell ${osd} cache drop
ceph tell ${osd} bench 12288000 4096 4194304 100
example :
osd.21 (hdd): osd_mclock_max_capacity_iops_hdd = 315.00
bench 1 : 3006.2271379745534
bench 2 : 819.503206458996
bench 3 : 946.5406320134085
How can I succeed in getting the good values for the
osd_mclock_max_capacity_iops_[hdd|ssd] options ?
Thank you for your help,
Rafael
smime.p7s
Description: Signature cryptographique S/MIME
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io
[ceph-users] db/wal pvmoved ok, but gui show old metadatas
Hello,
we have a Ceph 17.2.5 cluster with a total of 26 nodes, where 15 nodes that
have faulty NVMe drives,
where the db/wal resides (one NVMe for the first 6 OSDs and another for the
remaining 6).
We replaced them with new drives and pvmoved it to avoid losing the OSDs.
So far, there are no issues, and the OSDs are functioning properly.
ceph see the correct news disks
root@node02:/# ceph daemon osd.26 list_devices
[
{
"device": "/dev/nvme0n1",
"device_id": "INTEL_SSDPEDME016T4S_CVMD516500851P6KGN"
},
{
"device": "/dev/sdc",
"device_id": "SEAGATE_ST18000NM004J_ZR52TT83C148JFSJ"
}
]
However, the Cephadm GUI still shows the old NVMe drives and hasn't recognized
the device change.
How can we make the GUI and Cephadm recognize the new devices?
I tried restarting the managers, thinking that it would rescan the OSDs during
startup, but it didn't work.
If you have any ideas, I would appreciate it.
Should I perform something like that: ceph orch daemon reconfig osd.*
Thank you for your help.
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io
[ceph-users] Re: Quincy osd bench in order to define osd_mclock_max_capacity_iops_[hdd|ssd]
Hi Rafael.
We faced the exact same issue. And we did a bunch of tests and question.
We started with some FIOs, but results where quite meh once in production. Ceph
bench did not seemed very reliable.
What we ended up doing and seems to hold up quite nicely, is the above. It's
probably not the best but works for use.
Step 1: We put the disks we want to test in a small dev/test cluster we have.
So we can mess up with the configs, and go to the not so prod friendly way to
configure the cluster. And we can make sure no other activity other than our
test is running.
Step 2: Create a few pools, that have this particularities:
- Only 1 PG
- No replication at all. The only PG of the pool lives in only 1 disk.
- Upmap the PG to the OSD you want to test.
- Repeat for a few disks.
Step 2 needs to enable some options of ceph, as it will not allow you to do it
by default. I do not remember the exact options, but you can find them on the
documentation.
Sept 3: set mclock as high_client_ops. So mclock will virtually not limit
client ops.
Step 4: Run a few ceph rados bench on the different disks.
rados bench write -t -b -p
we used 300 secondes to be able do perfom various tests without
taking a week, but letting the cluster writing a bunch of stuff anyway.
we used 100, it worked well, and previous tests for other purposes
showed 100 was nice on our installation.
the pool with the disk you want to test
we used 128k and 4M. Feel free to experiment with other values.
But in our use case it was what we used.
Somewhere in the output of the bench, after it finishes, will be Average IOPS.
This average is more or less what the disk is capable of handling. Then we put
some number close to that one. If we have two types of disks that are close to
each others, we put the smaller value for all disks. And we set it as a global
configuration on ceph instead of going disk by disk.
It's probably not perfect and looks very like we tinkered something, but it's
the best solution for testing that so far. And most important, results between
tests where a lot more consistent than ceph bench or fio.
Hope this will help you.
Luis Domingues
Proton AG
--- Original Message ---
On Friday, June 30th, 2023 at 12:15, Rafael Diaz Maurin
wrote:
> Hello,
>
> I've just upgraded a Pacific cluster into Quincy, and all my osd have
> the low value osd_mclock_max_capacity_iops_hdd : 315.00.
>
> The manuel does not explain how to benchmark the OSD with fio or ceph
> bench with good options.
> Can someone have the good ceph bench options or fio options in order to
> configure osd_mclock_max_capacity_iops_hdd for each osd ?
>
> I ran this bench various times on the same OSD (class hdd) and I obtain
> different results.
> ceph tell ${osd} cache drop
> ceph tell ${osd} bench 12288000 4096 4194304 100
>
> example :
> osd.21 (hdd): osd_mclock_max_capacity_iops_hdd = 315.00
> bench 1 : 3006.2271379745534
> bench 2 : 819.503206458996
> bench 3 : 946.5406320134085
>
> How can I succeed in getting the good values for the
> osd_mclock_max_capacity_iops_[hdd|ssd] options ?
>
> Thank you for your help,
>
> Rafael
>
>
>
> ___
> ceph-users mailing list -- ceph-users@ceph.io
> To unsubscribe send an email to ceph-users-le...@ceph.io
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io
[ceph-users] Re: Help needed to configure erasure coding LRC plugin
I created a tracker issue, maybe that will get some attention: https://tracker.ceph.com/issues/61861 Zitat von Michel Jouvin : Hi Eugen, Thank you very much for these detailed tests that match what I observed and reported earlier. I'm happy to see that we have the same understanding of how it should work (based on the documentation). Is there any other way that this list to enter in contact with the plugin developers as it seems they are not following this (very high volume) list... Or may somebody pass the email thread to one of them? Help would be really appreciated. Cheers, Michel Le 19/06/2023 à 14:09, Eugen Block a écrit : Hi, I have a real hardware cluster for testing available now. I'm not sure whether I'm completely misunderstanding how it's supposed to work or if it's a bug in the LRC plugin. This cluster has 18 HDD nodes available across 3 rooms (or DCs), I intend to use 15 nodes to be able to recover if one node fails. Given that I need one additional locality chunk per DC I need a profile with k + m = 12. So I chose k=9, m=3, l=4 which creates 15 chunks in total across those 3 DCs, one chunk per host, I checked the chunk placement and it is correct. This is the profile I created: ceph osd erasure-code-profile set lrc1 plugin=lrc k=9 m=3 l=4 crush-failure-domain=host crush-locality=room crush-device-class=hdd I created a pool with only one PG to make the output more readable. This profile should allow the cluster to sustain the loss of three chunks, the results are interesting. This is what I tested: 1. I stopped all OSDs on one host and the PG was still active with one missing chunk, everything's good. 2. Stopping a second host in the same DC resulted in the PG being marked as "down". That was unexpected since with m=3 I expected the PG to still be active but degraded. Before test #3 I started all OSDs to have the PG active+clean again. 3. I stopped one host per DC, so in total 3 chunks were missing and the PG was still active. Apparently, this profile is able to sustain the loss of m chunks, but not an entire DC. I get the impression (and I also discussed this with a colleague) that LRC with this implementation is either designed to loose only single OSDs which can be recovered quicker with fewer surviving OSDs and saving bandwidth. Or this is a bug because according to the low-level description [1] the algorithm works its way up in the reverse order within the configured layers, like in this example (not displaying my k, m, l requirements, just for reference): chunk nr 01234567 step 1 _cDD_cDD step 2 cDDD step 3 cDDD So if a whole DC fails and the chunks from step 3 can not be recovered, and maybe step 2 also fails, but eventually step 1 contains the actual k and m chunks which should sustain the loss of an entire DC. My impression is that the algorithm somehow doesn't arrive at step 1 and therefore the PG stays down although there are enough surviving chunks. I'm not sure if my observations and conclusion are correct, I'd love to have a comment from the developers on this topic. But in this state I would not recommend to use the LRC plugin when the resiliency requirements are to sustain the loss of an entire DC. Thanks, Eugen [1] https://docs.ceph.com/en/latest/rados/operations/erasure-code-lrc/#low-level-plugin-configuration Zitat von Michel Jouvin : Hi, I realize that the crushmap I attached to one of my email, probably required to understand the discussion here, has been stripped down by mailman. To avoid poluting the thread with a long output, I put it on at https://box.in2p3.fr/index.php/s/J4fcm7orfNE87CX. Download it if you are interested. Best regards, Michel Le 21/05/2023 à 16:07, Michel Jouvin a écrit : Hi Eugen, My LRC pool is also somewhat experimental so nothing really urgent. If you manage to do some tests that help me to understand the problem I remain interested. I propose to keep this thread for that. Zitat, I shared my crush map in the email you answered if the attachment was not suppressed by mailman. Cheers, Michel Sent from my mobile Le 18 mai 2023 11:19:35 Eugen Block a écrit : Hi, I don’t have a good explanation for this yet, but I’ll soon get the opportunity to play around with a decommissioned cluster. I’ll try to get a better understanding of the LRC plugin, but it might take some time, especially since my vacation is coming up. :-) I have some thoughts about the down PGs with failure domain OSD, but I don’t have anything to confirm it yet. Zitat von Curt : Hi, I've been following this thread with interest as it seems like a unique use case to expand my knowledge. I don't use LRC or anything outside basic erasure coding. What is your current crush steps rule? I know you made changes since your first post and had some thoughts I wanted to share, but wanted to see your rule first so I could try
[ceph-users] Re: ceph-fuse crash
If the crash is easily reproducible at your end, could you set debug_client
to 20 in the client-side conf file and then reattempt the operation.
You could then send over the collected logs and we could take a look at
them.
FYI - there's also a bug tracker that has identified a similar problem:
https://tracker.ceph.com/issues/56288, but we don't have the logs for it.
On Fri, Jun 30, 2023 at 11:16 AM wrote:
> Hi,
>
> I've deployed a ceph-quincy for HPC. Recently, I always encounter the
> problem of ceph-fuse crash
> kernel version is 4.18.0-348.el8.0.2.x86_64
>
> here is part of ceph-fuse log:
>
>-59> 2023-06-28T09:51:00.452+0800 155546ff7700 3 client.159239
> ll_lookup 0x200017f674a.head anaconda3
>-58> 2023-06-28T09:51:00.452+0800 15554cc49700 3 client.159239
> ll_opendir 0x10003e1408d.head
>-57> 2023-06-28T09:51:00.452+0800 15554cc49700 3 client.159239
> may_open 0x1554e79123d0 = 0
>-56> 2023-06-28T09:51:00.452+0800 15554cc49700 3 client.159239
> ll_opendir 0x10003e1408d.head = 0 (0x155328079380)
>-55> 2023-06-28T09:51:00.453+0800 1555473f9700 3 client.159239
> seekdir(0x155328079380, 0)
>-54> 2023-06-28T09:51:00.452+0800 155546bf5700 5 client.159239
> put_cap_ref dropped last FILE_CACHE ref on 0x20004a6e548.head(faked_ino=0
> nref=14 ll_ref=2 cap_refs={1024=0,2048=1} open={1=1} mode=100644
> size=5626/0 nlink=1 btime=2023-06-05T14:38:36.471178+0800
> mtime=2023-06-05T14:38:36.471178+0800 ctime=2023-06-05T14:38:36.471178+0800
> change_attr=1 caps=pAsLsXsFscr(0=pAsLsXsFscr) objectset[0x20004a6e548 ts
> 0/0 objects 1 dirty_or_tx 0] 0x1554e7902300)
>-53> 2023-06-28T09:51:00.453+0800 155546bf5700 3 client.159239 ll_read
> 0x15531806b970 0~8192 = 5626
>-52> 2023-06-28T09:51:00.453+0800 155546ff7700 3 client.159239
> may_lookup 0x155420004840 = 0
>-51> 2023-06-28T09:51:00.453+0800 1554a5dee700 3 client.159239
> ll_lookup 0x10003e1406f.head html
>-50> 2023-06-28T09:51:00.453+0800 155546ff7700 3 client.159239
> ll_lookup 0x200017f674a.head anaconda3 -> 0 (100019b7a43)
>-49> 2023-06-28T09:51:00.453+0800 1554a5dee700 3 client.159239
> may_lookup 0x1554c1a89e40 = 0
>-48> 2023-06-28T09:51:00.453+0800 1554a7dfe700 3 client.159239
> ll_flush 0x15531806b970 0x20004a6e548
>-47> 2023-06-28T09:51:00.453+0800 1555469f4700 3 client.159239
> ll_lookup 0x100019b7a43.head envs
>-46> 2023-06-28T09:51:00.453+0800 1555469f4700 3 client.159239
> may_lookup 0x155420343eb0 = 0
>-45> 2023-06-28T09:51:00.453+0800 1555469f4700 3 client.159239
> ll_lookup 0x100019b7a43.head envs -> 0 (200035de5d8)
>-44> 2023-06-28T09:51:00.453+0800 155545fef700 3 client.159239
> ll_release (fh)0x15531806b970 0x20004a6e548
>-43> 2023-06-28T09:51:00.453+0800 155546bf5700 3 client.159239
> seekdir(0x15544c054010, 1152360438801891331)
>-42> 2023-06-28T09:51:00.453+0800 155546ff7700 3 client.159239
> seekdir(0x15544c029ee0, 1152690945930559491)
>-41> 2023-06-28T09:51:00.453+0800 15554cc49700 3 client.159239
> ll_releasedir 0x15544c029ee0
>-40> 2023-06-28T09:51:00.453+0800 1555452da700 3 client.159239
> ll_lookup 0x20004a66459.head tests
>-39> 2023-06-28T09:51:00.453+0800 15554c244700 3 client.159239
> ll_lookup 0x100040bd04e.head att5410-w -> 0 (200047cbc02)
>-38> 2023-06-28T09:51:00.453+0800 1554a7dfe700 3 client.159239
> ll_lookup 0x200035de5d8.head steven-colossal
>-37> 2023-06-28T09:51:00.453+0800 1554a7dfe700 3 client.159239
> may_lookup 0x1554c20229c0 = 0
>-36> 2023-06-28T09:51:00.453+0800 1554a7dfe700 3 client.159239
> ll_lookup 0x200035de5d8.head steven-colossal -> 0 (100040bcf3a)
>-35> 2023-06-28T09:51:00.453+0800 1555452da700 3 client.159239
> may_lookup 0x155533c970f0 = 0
>-34> 2023-06-28T09:51:00.453+0800 1555452da700 3 client.159239
> ll_lookup 0x20004a66459.head tests -> 0 (20004a6e3e0)
>-33> 2023-06-28T09:51:00.453+0800 155545bed700 3 client.159239
> ll_releasedir 0x15544c054010
>-32> 2023-06-28T09:51:00.453+0800 155546bf5700 3 client.159239
> ll_getattr 0x200047cbc02.head = 0
>-31> 2023-06-28T09:51:00.453+0800 1555452da700 3 client.159239
> ll_lookup 0x200017f674a.head anaconda3
>-30> 2023-06-28T09:51:00.453+0800 1555452da700 3 client.159239
> may_lookup 0x155420004840 = 0
>-29> 2023-06-28T09:51:00.453+0800 1555452da700 3 client.159239
> ll_lookup 0x200017f674a.head anaconda3 -> 0 (100019b7a43)
>-28> 2023-06-28T09:51:00.453+0800 155545fef700 3 client.159239
> ll_lookup 0x100040bcf3a.head lib
>-27> 2023-06-28T09:51:00.453+0800 155545fef700 3 client.159239
> may_lookup 0x1554d7d1d240 = 0
>-26> 2023-06-28T09:51:00.453+0800 155545fef700 3 client.159239
> ll_lookup 0x100040bcf3a.head lib -> 0 (100040bd018)
>-25> 2023-06-28T09:51:00.453+0800 1555469f4700 3 client.159239
> ll_lookup 0x20004a6e3e0.head test_trainer
>-24> 2023-06-28T09:51:00.453+0800 1555469f4700 3 client.159239
> may_lookup 0x155531e5a6c0 = 0
>-23> 2023-06-28T09:51:00.453+0800 15554cc49700 3
[ceph-users] Quincy osd bench in order to define osd_mclock_max_capacity_iops_[hdd|ssd]
Hello,
I've just upgraded a Pacific cluster into Quincy, and all my osd have
the low value osd_mclock_max_capacity_iops_hdd : 315.00.
The manuel does not explain how to benchmark the OSD with fio or ceph
bench with good options.
Can someone have the good ceph bench options or fio options in order to
configure osd_mclock_max_capacity_iops_hdd for each osd ?
I ran this bench various times on the same OSD (class hdd) and I obtain
different results.
ceph tell ${osd} cache drop
ceph tell ${osd} bench 12288000 4096 4194304 100
example :
osd.21 (hdd): osd_mclock_max_capacity_iops_hdd = 315.00
bench 1 : 3006.2271379745534
bench 2 : 819.503206458996
bench 3 : 946.5406320134085
How can I succeed in getting the good values for the
osd_mclock_max_capacity_iops_[hdd|ssd] options ?
Thank you for your help,
Rafael
smime.p7s
Description: Signature cryptographique S/MIME
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io
[ceph-users] Re: RadosGW strange behavior when using a presigned url generated by SDK PHP
On Fri, Jun 30, 2023 at 01:21:57AM -, Huy Nguyen wrote: > Thanks for your reply, > Yes, my setup is like the following: > RGWs (port 8084) -> Nginx (80, 443) > > So this why it make me confuse when :8084 appear with the domain. > > And this behavior only occurs with PHP's generated url, not in Boto3 Put tcpdump or something else between nginx & RGW and capture the transaction when using Boto3 vs PHP. I'm relatively sure it's nginx adding it for you. -- Robin Hugh Johnson Gentoo Linux: Dev, Infra Lead, Foundation Treasurer E-Mail : robb...@gentoo.org GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85 GnuPG FP : 7D0B3CEB E9B85B1F 825BCECF EE05E6F6 A48F6136 signature.asc Description: PGP signature ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
