Den tors 16 maj 2024 kl 07:47 skrev Jayanth Reddy :
>
> Hello Community,
> In addition, we've 3+ Gbps links and the average object size is 200
> kilobytes. So the utilization is about 300 Mbps to ~ 1.8 Gbps and not more
> than that.
> We seem to saturate the link when the secondary zone fetches
Den tis 23 apr. 2024 kl 11:32 skrev Frédéric Nass
:
> Ceph is strongly consistent. Either you read/write objects/blocs/files with
> an insured strong consistency OR you don't. Worst thing you can expect from
> Ceph, as long as it's been properly designed, configured and operated is a
>
Den mån 15 apr. 2024 kl 13:09 skrev Mitsumasa KONDO :
> Hi Menguy-san,
>
> Thank you for your reply. Users who use large IO with tiny volumes are a
> nuisance to cloud providers.
>
> I confirmed my ceph cluster with 40 SSDs. Each OSD on 1TB SSD has about 50
> placement groups in my cluster.
Den tors 11 apr. 2024 kl 15:55 skrev :
>
> I have mapped port 32505 to 23860, however when connect via s3cmd it fails
> with "ERROR: S3 Temporary Error: Request failed for: /. Please try again
> later." .
> has anyone ecounted same issue?
>
> [root@vm-04 ~]# s3cmd ls
> WARNING: Retrying failed
Den tis 9 apr. 2024 kl 10:39 skrev Eugen Block :
> I'm trying to estimate the possible impact when large PGs are
> splitted. Here's one example of such a PG:
>
> PG_STAT OBJECTS BYTES OMAP_BYTES* OMAP_KEYS* LOG DISK_LOGUP
> 86.3ff277708 4144030984090 0
Den tors 4 apr. 2024 kl 06:11 skrev Zakhar Kirpichenko :
> Any comments regarding `osd noin`, please?
> >
> > I'm adding a few OSDs to an existing cluster, the cluster is running with
> > `osd noout,noin`:
> >
> > cluster:
> > id: 3f50555a-ae2a-11eb-a2fc-ffde44714d86
> > health:
> Hi every one,
> I'm new to ceph and I'm still studying it.
> In my company we decided to test ceph for possible further implementations.
>
> Although I undestood its capabilities I'm still doubtful about how to
> setup replication.
Default settings in ceph will give you replication = 3, which
> Sure! I think Wido just did it all unofficially, but afaik we've lost
> all of those records now. I don't know if Wido still reads the mailing
> list but he might be able to chime in. There was a ton of knowledge in
> the irc channel back in the day. With slack, it feels like a lot of
>
> Now we are using the GetBucketInfo from the AdminOPS api -
> https://docs.ceph.com/en/quincy/radosgw/adminops/#id44 with the stats=true
> option GET /admin/bucket?stats=1 which returns all buckets with the number of
> objects and size we then parse. We also use it for the tracking of newly
>
Den mån 4 mars 2024 kl 11:30 skrev Ml Ml :
>
> Hello,
>
> i wonder why my autobalancer is not working here:
I think the short answer is "because you have so wildly varying sizes
both for drives and hosts".
If your drive sizes span from 0.5 to 9.5, there will naturally be
skewed data, and it is
Den mån 12 feb. 2024 kl 14:12 skrev Murilo Morais :
>
> Good morning and happy holidays everyone!
>
> Guys, what would be the best strategy to increase the number of PGs in a
> POOL that is already in production?
"ceph osd pool set pg_num " and let the pool get pgp_nums increased slowly by
;>>> cluster. Disable the pg autoscaler for and increase the volumes pool to
> >>>>>> pg_num 256. Then likely re-asses and make the next power of 2 jump to
> >>>>>> 512
> >>>>>> and probably beyond.
> >>>>>>
> I’ve heard conflicting asserts on whether the write returns with min_size
> shards have been persisted, or all of them.
I think it waits until all replicas have written the data, but from
simplistic tests with fast network and slow drives, the extra time
taken to write many copies is not
> If there is a (planned) documentation of manual rgw bootstrapping,
> it would be nice to have also the names of required pools listed there.
It will depend on several things, like if you enable swift users, I
think they get a pool of their own, so I guess one would need to look
in the
Den mån 29 jan. 2024 kl 12:58 skrev Michel Niyoyita :
>
> Thank you Frank ,
>
> All disks are HDDs . Would like to know if I can increase the number of PGs
> live in production without a negative impact to the cluster. if yes which
> commands to use .
Yes. "ceph osd pool set pg_num "
where the
Den mån 29 jan. 2024 kl 10:38 skrev Eugen Block :
>
> Ah, you probably have dedicated RGW servers, right?
They are VMs, but yes.
--
May the most significant bit of your life be positive.
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe
Den mån 29 jan. 2024 kl 09:35 skrev Eugen Block :
But your (cephadm managed) cluster will
> complain about "stray daemons". There doesn't seem to be a way to
> deploy rgw daemons manually with the cephadm tool so it wouldn't be
> stray. Is there a specific reason not to use the orchestrator for
Den mån 29 jan. 2024 kl 08:11 skrev Jan Kasprzak :
>
> Hi all,
>
> how can radosgw be deployed manually? For Ceph cluster deployment,
> there is still (fortunately!) a documented method which works flawlessly
> even in Reef:
>
>
Den sön 28 jan. 2024 kl 23:02 skrev Adrian Sevcenco :
>
> >> is it wrong to think of PGs like a kind of object bucket (S3 like)?
> >
> > Mostly, yes.
> so .. in a PG there are no "file data" but pieces of "file data"?
> so 100 GB file with 2x replication will be placed in more than 2 PGs?
> Is
Den tors 25 jan. 2024 kl 17:47 skrev Robert Sander
:
> > forth), so this is why "ceph df" will tell you a pool has X free
> > space, where X is "smallest free space on the OSDs on which this pool
> > lies, times the number of OSDs". Given the pseudorandom placement of
> > objects to PGs, there is
Den tors 25 jan. 2024 kl 11:57 skrev Henry lol :
>
> It's reasonable enough.
> actually, I expected the client to have just? thousands of
> "PG-to-OSDs" mappings.
Yes, but filename to PG is done with a pseudorandom algo.
> Nevertheless, it’s so heavy that the client calculates location on
>
Den tors 25 jan. 2024 kl 03:05 skrev Henry lol :
>
> Do you mean object location (osds) is initially calculated only using its
> name and crushmap,
> and then the result is reprocessed with the map of the PGs?
>
> and I'm still skeptical about computation on the client-side.
> is it possible to
Den ons 10 jan. 2024 kl 19:20 skrev huxia...@horebdata.cn
:
> Dear Ceph folks,
>
> I am responsible for two Ceph clusters, running Nautilius 14.2.22 version,
> one with replication 3, and the other with EC 4+2. After around 400 days
> runing quietly and smoothly, recently the two clusters
Den tis 26 dec. 2023 kl 08:45 skrev Phong Tran Thanh :
>
> Hi community,
>
> I am running ceph with block rbd with 6 nodes, erasure code 4+2 with
> min_size of pool is 4.
>
> When three osd is down, and an PG is state down, some pools is can't write
> data, suppose three osd can't start and pg
Den ons 13 dec. 2023 kl 10:57 skrev Rok Jaklič :
> Hi,
>
> shouldn't etag of a "parent" object change when "child" objects are added
> on s3?
>
> Example:
> 1. I add an object to test bucket: "example/" - size 0
> "example/" has an etag XYZ1
> 2. I add an object to test bucket:
>
> Based on our observation of the impact of the balancer on the
> performance of the entire cluster, we have drawn conclusions that we
> would like to discuss with you.
>
> - A newly created pool should be balanced before being handed over
> to the user. This, I believe, is quite evident.
>
Den tors 30 nov. 2023 kl 17:35 skrev Francisco Arencibia Quesada <
arencibia.franci...@gmail.com>:
> Hello again guys,
>
> Can you recommend me a book that explains best practices with Ceph,
> for example is it okay to have mon,mgr, osd in the same virtual machine,
>
OSDs can need very much RAM
Looking up the "manual installation" parts might help, if you can't
get the container stuff going for $reasons.
Den mån 27 nov. 2023 kl 00:45 skrev Leo28C :
>
> I'm pulling my hair trying to get a simple cluster going. I first tried
> Gluster but I have an old system that can't handle the latest
Den fre 24 nov. 2023 kl 10:25 skrev Frank Schilder :
>
> Hi Denis,
>
> I would agree with you that a single misconfigured host should not take out
> healthy hosts under any circumstances. I'm not sure if your incident is
> actually covered by the devs comments, it is quite possible that you
Den fre 24 nov. 2023 kl 08:53 skrev Nguyễn Hữu Khôi :
>
> Hello.
> I have 10 nodes. My goal is to ensure that I won't lose data if 2 nodes
> fail.
Now you are mixing terms here.
There is a difference between "cluster stops" and "losing data".
If you have EC 8+2 and min_size 9, then when you
Den fre 24 nov. 2023 kl 06:15 skrev Miroslav Svoboda
:
> please, is it better to reduce the default object size from 4MB to some
> smaller value for the rbd image where there will be a lot of small mail
> and webhosting files?
Many of the defaults for rbd caching seems to be way above the 4M mark
> Now 25 years later lot of people recommend to use replica so if I buy XTo
> I'm only going to have X/3 To (vs raidz2 where I loose 2 disks over 9-12
> disks).
As seen from other answers, it changes which performance and space
usage you want go have, but there are other factors too. Replica = 3
Den tors 16 nov. 2023 kl 08:43 skrev Nguyễn Hữu Khôi
:
>
> Hello,
> Yes, I see it does not exceed RSS but I see in "ceph orch ps". it is over
> target. Does Mem Use include cache, I am right?
>
> NAMEHOST PORTSSTATUS REFRESHED
> AGE MEM USE MEM LIM
Den tors 16 nov. 2023 kl 00:30 skrev Giuliano Maggi
:
> Hi,
>
> I’d like to remove some “spurious" data:
>
> root@nerffs03:/# ceph df
> --- RAW STORAGE ---
> CLASS SIZEAVAILUSED RAW USED %RAW USED
> hdd1.0 PiB 1.0 PiB 47 GiB47 GiB 0
> TOTAL 1.0 PiB 1.0 PiB 47
> Docs question: https://tracker.ceph.com/issues/11385: Can a member of the
> community just raise a PR attempting to standardize commands, without
> coordinating with a team?
In this case I think I would recommend having both "rm" and "del" do
the same thing. I agree that this kind of mixup
Den tors 2 nov. 2023 kl 23:46 skrev V A Prabha :
>
> Is it possible to move the OSDs safe (making the OSDs out and move the content
> to other OSDs and remove it and map it fresh to other nodes which is less
> loaded)
> As the client feels that using 3 replicas and holding these much spare
>
Den ons 1 nov. 2023 kl 17:51 skrev Thomas Bennett :
>
> To update my own question, it would seem that Principle should be
> defined like this:
>
>- "Principal": {"AWS": ["arn:aws:iam::Tenant1:user/readwrite"]}
>
> And resource should:
> "Resource": [ "arn:aws:s3:::backups"]
>
> Is it
>
>
> The documentation describes that I could set a device class for an OSD with
> a command like:
>
> `ceph osd crush set-device-class CLASS OSD_ID [OSD_ID ..]`
>
> Class names can be arbitrary strings like 'big_nvme". Before setting a new
> device class to an OSD that already has an assigned
>
>
> One particular thing: The upgrade instructions in various places on
> docs.ceph.com say something like
>
> Upgrade monitors by installing the new packages and restarting the monitor
> daemons.
>
>
> To me this is kind of vague. Perhaps there is a different concept fo
> 'packages' within
Den tis 3 okt. 2023 kl 11:59 skrev Thomas Bennett :
> Hi,
>
> I'm running a Ceph 17.2.5 Rados Gateway and I have a user with more than
> 1000 buckets.
>
> When the client tries to list all their buckets using s3cmd, rclone and
> python boto3, they all three only ever return the first 1000 bucket
Den ons 27 sep. 2023 kl 15:32 skrev Nicolas FONTAINE :
> Hi everyone,
> Is there a way to specify which MGR and which MDS should be the active one?
>
At least for the mgr, you can just fail-over until it lands on the one you
want it to be running on.
--
May the most significant bit of your
Den mån 21 aug. 2023 kl 12:28 skrev Boris Behrens :
>
> Hi,
> I need to migrate a storage cluster to a new network.
>
> I added the new network to the ceph config via:
> ceph config set global public_network "old_network/64, new_network/64"
> I've added a set of new mon daemons with IP addresses
Den tors 15 juni 2023 kl 18:04 skrev 胡 玮文 :
> Hi Angelo,
>
> From my experience, I guess the objects written to degraded pg is
> immediately degraded. As the total number of objects is increasing, I think
> the increase of degraded objects is normal.
> >
> > I'm trying to understand what is
Den tors 8 juni 2023 kl 09:43 skrev Marc :
> > I bumped into an very interesting challenge, how to secure erase a rbd
> > image data without any encryption?
As Darren replied while I was typing this, you can't have dangerous
data written all over a cluster which automatically moves data around,
> I have a very old Ceph cluster running the old dumpling version 0.67.1. One
> of the three monitors suffered a hardware failure and I am setting up a new
> server to replace the third monitor running Ubuntu 22.04 LTS (all the other
> monitors are using the old Ubuntu 12.04 LTS).
> - Try to
If you can stop the rgws, you can make a new pool with 32 PGs and then
rados cppool this one over the new one, then rename them so this one
has the right name (and application) and start the rgws again.
Den mån 5 juni 2023 kl 16:43 skrev Louis Koo :
>
> ceph version is 16.2.13;
>
> The pg_num is
>undergo deepscrub and regular scrub cannot be completed in a timely manner. I
>have noticed that these PGs appear to be concentrated on a single OSD. I am
>seeking your guidance on how to address this issue and would appreciate any
>insights or suggestions you may have.
>
The usual "see if
Den tors 4 maj 2023 kl 17:07 skrev :
>
> The radosgw has been configured like this:
>
> [client.rgw.ceph1]
> host = ceph1
> rgw_frontends = beast port=8080 ssl_port=443 ssl_certificate=/root/ssl/ca.crt
> ssl_private_key=/root/ssl/ca.key
> #rgw_frontends = beast port=8080 ssl_port=443
>
Den tors 4 maj 2023 kl 10:39 skrev huxia...@horebdata.cn
:
> Dear Ceph folks,
>
> I am writing to ask for advice on best practice of expanding ceph cluster. We
> are running an 8-node Ceph cluster and RGW, and would like to add another 10
> node, each of which have 10x 12TB HDD. The current
Den fre 28 apr. 2023 kl 14:51 skrev Niklas Hambüchen :
>
> Hi all,
>
> > Scrubs only read data that does exist in ceph as it exists, not every
> > sector of the drive, written or not.
>
> Thanks, this does explain it.
>
> I just discovered:
>
> ZFS had this problem in the past:
>
> *
>
Den ons 26 apr. 2023 kl 21:20 skrev Niklas Hambüchen :
> > 100MB/s is sequential, your scrubbing is random. afaik everything is random.
>
> Is there any docs that explain this, any code, or other definitive answer?
> Also wouldn't it make sense that for scrubbing to be able to read the disk
>
Den tis 25 apr. 2023 kl 15:02 skrev Boris Behrens :
>
> We have a customer that tries to use veeam with our rgw objectstorage and
> it seems to be blazingly slow.
> What also seems to be strange, that veeam sometimes show "bucket does not
> exist" or "permission denied".
> I've tested parallel and
Den ons 19 apr. 2023 kl 00:55 skrev Murilo Morais :
> Good evening everyone!
> Guys, about the P420 RAID controller, I have a question about the operation
> mode: What would be better: HBA or RAID-0 with BBU (active write cache)?
As already said, always give ceph (and zfs and btrfs..) the raw
Den lör 15 apr. 2023 kl 15:47 skrev mahnoosh shahidi :
>
> Hi,
>
> I observed duplicate object names in the result of the admin list bucket on
> 15.2.12 cluster. I used the following command and some of the object names
> in the result list appeared more than once. There is no versioning config
>
Den fre 14 apr. 2023 kl 18:04 skrev Will Nilges :
>
> Hello!
> I'm trying to Install the ceph-common package on a Rocky Linux 9 box so
> that I can connect to our ceph cluster and mount user directories. I've
> added the ceph repo to yum.repos.d, but when I run `dnf install
> ceph-common`, I get
Den mån 10 apr. 2023 kl 22:31 skrev mhnx :
> Hello.
> I have a 10 node cluster. I want to create a non-replicated pool
> (replication 1) and I want to ask some questions about it:
>
> Let me tell you my use case:
> - I don't care about losing data,
> - All of my data is JUNK and these junk files
Den tors 23 mars 2023 kl 15:18 skrev Geert Kloosterman
:
> Hi all,
> Until Ceph Pacific, installing just the "ceph" package was enough to get
> everything needed to deploy Ceph.
> However, with Quincy, ceph-volume was split off into its own package, and it
> is not automatically installed
Den mån 20 mars 2023 kl 09:45 skrev Marc :
>
> > While
> > reading, we barely hit the mark of 100MB/s; we would expect at least
> > something similar to the write speed. These tests are being performed in
> > a
> > pool with a replication factor of 3.
> >
> >
>
> You don't even describe how you
Den tors 16 mars 2023 kl 06:42 skrev Norman :
> Janne,
>
> Thanks for your reply. To reduce the cost of recovering OSDs while
> WAL/DB device is down, maybe I have no
> choice but add more WAL/DB devices.
We do run one ssd-or-nvme for several OSD hdd drives and have not seen
this as a problem in
> hi, everyone,
> I have a question about repairing the broken WAL/DB device.
>
> I have a cluster with 8 OSDs, and 4 WAL/DB devices(1 OSD per WAL/DB
> device), and hwo can I repair the OSDs quickly if
>
> one WAL/DB device breaks down without rebuilding the them? Thanks.
I think this is one of
Den lör 4 mars 2023 kl 08:08 skrev :
> ceph 16.2.11,
> is safe to enable scrub and deep scrub during backfilling ?
> I have log recovery-backfilling due to a new crushmap , backfilling is going
> slow and deep scrub interval as expired so I have many pgs not deep-scrubbed
> in time.
It is safe
Den fre 3 mars 2023 kl 01:07 skrev :
> it is unclear for us what min_size means besides what it does. i hope someone
> can clear this up :)
> someone pointed out "split brain" but I am unsure about this.
>
> i think what happens in the worst case is this:
> only 1 PG is available, client writes
Den tors 2 mars 2023 kl 08:09 skrev Eugen Block :
> if your failure domain is "host" and you have enough redundancy (e.g.
> replicated size 3 or proper erasure-code profiles and rulesets) you
> should be able to reboot without any issue. Depending on how long the
> reboot would take, you could set
Den tis 28 feb. 2023 kl 18:13 skrev Dave Ingram :
> There are also several
> scrub errors. In short, it's a complete wreck.
>
> health: HEALTH_ERR
> 3 scrub errors
> Possible data damage: 3 pgs inconsistent
> [root@ceph-admin davei]# ceph health detail
> HEALTH_ERR 3
> The problem I’m seeing is after setting up RadosGW I can only upload to “S3”
> at around 25MBs with the official AWS CLI. Using s3cmd is slightly better at
> around 45MB/s. I’m going directly to the RadosGW instance with no load
> balancers in between and no ssl enabled. Just trying to figure
ust having WAL/DB separate might make the
OSD live lots longer since they bear the load of said smaller writes
for metadata.
Den mån 30 jan. 2023 kl 15:34 skrev Manuel Holtgrewe :
>
> OK. How much data will be written to the WAL and elsewhere?
>
> On Mon, Jan 30, 2023 at 3:17 PM
> I'm concerned with the potential increased NVME wear. Assuming writes of
> multiples of the block size, when I write 1GB of data to the CephFS, how much
> data is written to the disks?
In that case, repl=3 will write 1GB to three PGs. EC 8+3 would write
125M (ie, 1/8 of a GB) to 11 (8+3)
> is there information available anywhere about the write amplification for
> CephFS? I found quite some material on write amplification of VMs using
> journaled file system on top of RBD but nothing as it relates to CephFS?
>
> From my understanding I would expect the following:
>
> - for X-rep,
> Well, that's the thing: there are a whole bunch of ceph-guest-XX.log
> files in /var/log/ceph/; most of them are empty, a handful are up to 250
> Kb in size, and this one () keeps on growing - and where not sure where
> they're coming from (ie there's nothing that we can see in the conf files.
> Thank you for the prompt response.
>
> > Such a heterogenous setup is only possible with a large number of OSDs
>
> A heterogeneous setup is exactly what I am testing for. I wonder if the
> balancer will actually consider the WEIGHT when allocating the data to OSD.
It does, but only when the
Den tis 13 dec. 2022 kl 17:47 skrev Vladimir Brik
:
>
> Hello
>
> I have a bunch of HDD OSDs with DB/WAL devices on SSD. If
> the current trends continue, the DB/WAL devices will become
> full before the HDDs completely fill up (e.g. a 50% full HDD
> has DB/WAL device that is about 65% full).
>
>
Den mån 12 dec. 2022 kl 21:18 skrev Benjamin.Zieglmeier
:
> We are in the process of building new stage (non-production) Ceph RGW
> clusters hosting s3 buckets. We are looking to have our customers migrate
> their non-production buckets to these new clusters. We want to help ease the
>
Perhaps run "iostat -xtcy 5" on the OSD hosts to
see if any of the drives have weirdly high utilization despite low
iops/requests?
Den tis 6 dec. 2022 kl 10:02 skrev Boris Behrens :
>
> Hi Sven,
> I am searching really hard for defect hardware, but I am currently out of
> ideas:
> - checked
Den lör 3 dec. 2022 kl 22:52 skrev Sebastian :
>
> One thing to this discussion.
> I had a lot of problems with my clusters. I spent some time debugging.
> What I found and what I confirmed on AMD nodes, everything starts working
> like a charm when I added to kernel param iommu=pt
> Plus some
> my OSDs are running odroid-hc4's and they only have about 4GB of memory,
> and every 10 minutes a random OSD crashes due to out of memory. Sadly the
> whole machine gets unresponsive when the memory gets completely full, so no
> ssh access or prometheus output in the meantime.
> I've set the
> In Ceph/Libvirt docs only cachmodes writetrough and writeback are discussed.
> My clients's disks are all set to writeback in the libvirt client
> xml-definition.
>
> For a backup operation, I notice a severe lag on one of my VM's. Such a
> backup operation that takes 1 to 2 hours (on a same
Den ons 16 nov. 2022 kl 08:00 skrev Szabo, Istvan (Agoda)
:
>
> Hi,
>
> I have 3 mons in my cluster and I need to move to another cage one of them.
> I guess it is not an issue to have one mon down for an hour, is it?
If nothing else changes (ie, no OSDs,rgws,mds's fall off the net), no
new
Den mån 7 nov. 2022 kl 13:04 skrev Sagittarius-A Black Hole
:
> I'm using the Tailscale VPN, which is based on wireguard and want to
> make the Ceph cluster available offsite.
> Now I have seen that ceph does not bind the mon to the tailscale
> interface, so it is not available over it. Nmap shows
Den fre 4 nov. 2022 kl 13:37 skrev Murilo Morais :
> Hi Tyler, thanks for clarifying, it makes total sense now.
> Hypothetically, if there are any failures and most stop, how can I
> re-initialize the cluster in its current state or what can be done in this
> kind of case?
>
Just add one more mon
Den fre 4 nov. 2022 kl 10:48 skrev Szabo, Istvan (Agoda)
:
> Hi,
> One of my user told me that they can upload bigger files to the bucket than
> the limit. My question is to the developers mainly what’s the reason to set
> the rgw_bucket_quota_ttl=600 and rgw_user_quota_bucket_sync_interval=180?
Den fre 28 okt. 2022 kl 09:56 skrev Szabo, Istvan (Agoda)
:
> I found this long time back tracker https://tracker.ceph.com/issues/23470
> which I guess some way show that it is possible but haven't really found any
> documentation in ceph, how to do properly.
Yes, RGW supports them, more or
Den tors 20 okt. 2022 kl 18:57 skrev Wyll Ingersoll
:
> What network does radosgw use when it reads/writes the objects to the cluster?
Everything in ceph EXCEPT osd<->osd traffic uses the public network.
Anything that isn't backfills or replication betweens OSDs is always
using the public
> Hi,
> I've seen Dan's talk:
> https://www.youtube.com/watch?v=0i7ew3XXb7Q
> and other similar ones that talk about CLUSTER size.
> But, I see nothing (perhaps I have not looked hard enough), on any
> recommendations regarding max POOL size.
> So, are there any limitations on a given pool that
Den fre 14 okt. 2022 kl 12:10 skrev Szabo, Istvan (Agoda)
:
> I've added 5 more nodes to my cluster and got this issue.
> HEALTH_WARN 2 backfillfull osd(s); 17 pool(s) backfillfull; Low space
> hindering backfill (add storage if this doesn't resolve itself): 4 pgs
> backfill_toofull
>
> I've changed some elements of the config now and the results are much better
> but still quite poor relative to what I would consider normal SSD performance.
The number of PGs has been increased from 128 to 256. Not yet run JJ Balancer.
> In terms of performance, I measured the time it takes
> osd_memory_target = 2147483648
>
> Based on some reading, I'm starting to understand a little about what can be
> tweaked. For example, I think the osd_memory_target looks low. I also think
> the DB/WAL should be on dedicated disks or partitions, but have no idea what
> procedure
> Thanks for the quick response!
>
> What if the node is down? The client cannot even connect to the mgr.
Then this mgr would not be in the list of possible mgrs to connect to at all.
--
May the most significant bit of your life be positive.
___
Den tors 6 okt. 2022 kl 10:40 skrev Zhongzhou Cai :
> Hi folks,
> I have ceph-mgr bootstrapped on three nodes, and they are running in HA.
> When the active mgr node goes down, it will failover to one of the
> standbys. I'm wondering if there is a way for the client to be aware of the
> leadership
> Hello
>
> As previously describe here, we have a full-flash NVME ceph cluster (16.2.6)
> with currently only cephfs service configured.
[...]
> We noticed that cephfs_metadata pool had only 16 PG, we have set
> autoscale_mode to off and increase the number of PG to 256 and with this
> change,
> # ceph tell osd.2 config get osd_max_backfills
> {
> "osd_max_backfills": "1000"
> }
>
> makes little sense to me.
This means you have the mClock IO scheduler, and it gives back this
value since you are meant to change the mClock priorities and not the
number of backfills.
Some more info
> There is definitely something wrong in how my cluster manages
> osd_memory_target. For example, this is the situation for OSD 16:
> The memory limit seems to be correctly set (I disabled the memory
> autotune on the host, set the limit manually with --force and rebooted
> the host) but
> I used to create Bluestore OSDs using commands such as this one:
>
> ceph-volume lvm create --bluestore --data ceph-block-50/block-50 --block.db
> ceph-db-50-54/db-50
> with the goal of having block.db and wal.db co-located on the same LV
> (ceph-db-50-54/db-5 in my example, which is on a SSD
Den tors 29 sep. 2022 kl 17:57 skrev Matt Vandermeulen :
>
> I think you're likely to get a lot of mixed opinions and experiences
> with this question. I might suggest trying to grab a few samples from
> different vendors, and making sure they meet your needs (throw some
> workloads at them,
> Many thanks for any hint helping to get missing 7 OSDs up ASAP.
Not sure if it "helps", but I would try "ceph-volume lvm activate
--all" if those were on lvm, I guess ceph-volume simple and raw might
have similar command to search for and start everything that looks
like a ceph OSD.
Perhaps
> > ceph osd setcrushmap -i /tmp/crush.new
> >
> > Note: If you are overwriting your current rule, your data will need to
> > rebalance as soon as your set the crushmap, close to 100% of your
> > objects will move. If you create a new rule, you can set your pool to
> > use the new pool id anytime
Den lör 24 sep. 2022 kl 23:38 skrev Murilo Morais :
> I'm relatively new to Ceph. I set up a small cluster with two hosts with 12
> disks each host, all 3 TB SAS 7500 RPM and two 10 Gigabit interfaces. I
> created a pool in replicated mode and configured it to use two replicas.
>
> What I'm
Den ons 14 sep. 2022 kl 11:08 skrev gagan tiwari
:
> Yes. To start with we only have one HP server with DAS. Which I am planning
> to set up as ceph on. We can have one more server later.
>
> But I think you are correct. I will use ZFS file systems on it and NFS export
> all the data to all
Den ons 14 sep. 2022 kl 10:14 skrev gagan tiwari
:
>
> Sorry. I meant SSD Solid state disks.
>> > We have a HP storage server with 12 SDD of 5T each and have set-up hardware
>> > RAID6 on these disks.
>>
>> You have only one single machine?
>> If so, run zfs on it and export storage as NFS.
Den ons 14 sep. 2022 kl 08:54 skrev gagan tiwari
:
> Hi Guys,
> I am new to Ceph and storage. We have a requirement of
> managing around 40T of data which will be accessed by around 100 clients
> all running RockyLinux9.
>
> We have a HP storage server with 12 SDD of 5T each and
Den ons 20 juli 2022 kl 11:22 skrev Jesper Lykkegaard Karlsen :
> Thanks for you answer Janne.
> Yes, I am also running "ceph osd reweight" on the "nearfull" osds, once they
> get too close for comfort.
>
> But I just though a continuous prioritization of rebalancing PGs, could make
> this
1 - 100 of 334 matches
Mail list logo
