Patrick,
Thank you for reaching out.
> Have you tried remounting? Updating auth credentials usually requires remount.
Yes, we have tried both changing caps on our original user as well as
using the client.admin user with remounts between each change.
> Next thing to do is collect mds logs:
>
From my experience with Ceph in production over the last +8 years, it
is only a matter of time that pools with replication size of 2 or
erasure coding with m = 1 will lead to service outages and/or data loss
and cause problems in day2 operations.
___
Clyso GmbH
Hello David,
On Fri, Aug 20, 2021 at 7:17 AM David Prude wrote:
>
> Hello,
>
>We have a cluster initially installed with pacific 16.2.5 within
> which we have a single cephfs volume created as such:
>
> ceph fs volume create dncephfs
>
> We have a client.admin client:
>
>
Doesn’t look like it:
[root@cxcto-c240-j27-01 ~]# ceph orch upgrade status
{
"target_image": null,
"in_progress": false,
"services_complete": [],
"progress": null,
"message": ""
}
I did an upgrade from 16.2.4 to 16.2.5 about 4 weeks ago when the 16.2.5
release came out, but
Satoru;
Ok. What your cluster is telling you, then, is that it doesn't know which
replica is the "most current" or "correct" replica. You will need to determine
that, and let ceph know which one to use as the "good" replica. Unfortunately,
I can't help you with this. In fact, if this is
Hello Joshua,
On Tue, Aug 10, 2021 at 11:44 AM Joshua West wrote:
>
> Related: Where can I find MDS numeric state references for ceph mds
> set_state GID ?
>
> Like a dummy I accidentally upgraded to the ceph dev branch (quincy?),
> and have been having nothing but trouble since. This wasn't
Hi Dominic,
2021年8月21日(土) 1:05 :
> Satoru;
>
> You said " after restarting all nodes one by one." After each reboot, did
> you allow the cluster the time necessary to come back to a "HEALTH_OK"
> status?
>
No, the we rebooted with the following policy.
1. Reboot one machine.
2. Wait until
Hello Thomas
On Mon, Aug 9, 2021 at 12:04 PM Thomas Hukkelberg
wrote:
> Today we suddenly experience multiple MDS crashes during the day with an
> error we have not seen earlier. We run octopus 15.2.13 with 4 ranks and 4
> standby-reply MDSes and 1 passive standby. Any input on how to
We launch a local registry for cases like these and mirror the relevant
containers there. This keeps copies of the images closer to the target cluster
and reduces load on the public registries. Its not that much different from
mirroring a yum/apt repo locally to speed up access. For large
Thanks all for the replies :)
Indeed it looks like docker hub has been deprecated, and we are now on quay.io
although I would say the communications were a bit lacking, and it's been
implemented really quick leaving things like Cephadmin on the Octopus branch
broken (pending backport) for use
Hi,
you can just set the config option with 'ceph config set ...' after
your cluster has been bootstrapped. See [1] for more details about the
config store.
[1]
https://docs.ceph.com/en/latest/rados/configuration/ceph-conf/#monitor-configuration-database
Zitat von Dong Xie :
Dear
What is the output of 'ceph orch upgrade status'? Did you (maybe
accidentally) start an update? You can stop it with 'ceph orch upgrade
stop'.
Zitat von "Paul Giralt (pgiralt)" :
The output of my ’ceph status’ shows the following:
progress:
Updating node-exporter deployment (-1 ->
Satoru;
You said " after restarting all nodes one by one." After each reboot, did you
allow the cluster the time necessary to come back to a "HEALTH_OK" status?
Thank you,
Dominic L. Hilsbos, MBA
Vice President – Information Technology
Perform Air International Inc.
dhils...@performair.com
2021年8月21日(土) 0:25 Satoru Takeuchi :
...
> # additional information
>
I forgot to write the important information. All my data have 3 copies.
Regards,
Satoru
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to
The output of my ’ceph status’ shows the following:
progress:
Updating node-exporter deployment (-1 -> 14) (0s)
[]
Updating crash deployment (-1 -> 14) (0s)
[]
Updating crash deployment (-1 -> 14) (0s)
Hi,
I found an `active+recovery_unfound+undersized+degraded+remapped` pg
after restarting
all nodes one by one. Could anyone give some hints why this problem
happened and how
to restore my data?
I read some documents and searched Ceph issues, but I couldn't find
enough information.
Igor,
We've hit this again on ceph 15.2.13 using the default allocator. Once again,
configuring the OSDs to use the bitmap allocator has fixed up the issue.
I'm still trying to gather the full set of debug logs from the crash. I think
again the fact I'm running in containers is the issue here;
Dear All, Early days of my venture with Ceph. I understand that one should have at least two hosts to truly appreciate the design of Ceph. But as a baby step having one playground on a single host is really unavoidable. My env:A single Azure VMUbuntu 20.04Installed cephadm via apt per official
Hi all,
finally my (test) cluster is up and running!
Thank you very much for all the people here in list that helped me.
I now have some final questions:
1. In my cluster I have three monitors; when one monitor is down (I
simply shut down) raising a ceph -s underline that there are two
Erik Lindahl writes:
>> On 20 Aug 2021, at 10:39, Nico Schottelius
>> wrote:
>>
>> I believe mid term everyone will need to provide their own image
>> registries, as the approach of "everything is at dockerhub|quay"
>> does not scale well.
>
> Yeah, this particular issue is not hard to fix
Hi Yuval,
Thanks a lot for clarifying my doubts and great help indeed.
I will follow the same recommendations to setup S3 bucket notifications.
Best regards,
Sanjeev
From: Yuval Lifshitz
Sent: Friday, August 20, 2021 3:40 PM
To: Sanjeev Jha
Cc:
They will upload from the same network segment to the same network where the
cluster is located.
Istvan Szabo
Senior Infrastructure Engineer
---
Agoda Services Co., Ltd.
e: istvan.sz...@agoda.com
---
Hi,
1. In my cluster I have three monitors; when one monitor is down (I
simply shut down) raising a ceph -s underline that there are two
monitors alive and one down; when 2/3 of monitors down the cluster
became unresponsive (ceph -s remains stuck); is this normal?
yes, this is expected.
Eh. I did not catch that one.
So it seems that https://quay.io/repository/ceph/ceph?tab=tags has
15.2.14, but if the repository / container link is not updated, that
would explain the error message.
I'll have to check whether rook also needs to be updated to reference to
quay.io by default.
I
So 1x5GB so like 50TB/file?
Istvan Szabo
Senior Infrastructure Engineer
---
Agoda Services Co., Ltd.
e: istvan.sz...@agoda.com
---
-Original Message-
From: Janne Johansson
Sent: Friday,
S3cmd chunks 15MB.
“ Max put size for one piece is 5G above, times 10k pieces.”
I don’t think so, max put size I think if not multipart file, isn’t it?
Istvan Szabo
Senior Infrastructure Engineer
---
Agoda Services Co., Ltd.
e:
Hello,
We have a cluster initially installed with pacific 16.2.5 within
which we have a single cephfs volume created as such:
ceph fs volume create dncephfs
We have a client.admin client:
[client.admin]
key = REDACTED
caps mds = "allow *"
Nigel Williams writes:
> not showing via podman either:
A note to make it maybe a bit easier:
You can see on
https://hub.docker.com/r/ceph/ceph/tags?page=1=last_updated
which images / tags exist. 15.2.14 has not yet been tagged/pushed.
Best regards,
Nico
--
Sustainable and modern
Hi,
These are the values in octopus:
"rgw_max_put_size": "5368709120",
"rgw_multipart_part_upload_limit": "1",
"rgw_multipart_min_part_size": "5242880",
Correct me if I'm wrong but the multipart parts size is 15MB so 1 means
maximum size of one object is 150GB.
What is the
On Thu, Aug 19, 2021 at 6:30 PM Sanjeev Jha wrote:
> Hi Yuval,
>
> Thanks very much for your reply.
>
> I am using AMQP 0.9.1.
>
> Can I use aws sns create-topic command to create a topic in Ceph's RadosGW
> ?
>
yes. note that you define the topic on the RGW, not the rabbitmq broker.
e.g.
aws
> On 20 Aug 2021, at 10:39, Nico Schottelius
> wrote:
>
> I believe mid term everyone will need to provide their own image
> registries, as the approach of "everything is at dockerhub|quay"
> does not scale well.
Yeah, this particular issue is not hard to fix technically (and I just checked
Den fre 20 aug. 2021 kl 10:45 skrev Marc :
>
> > > S3cmd chunks 15MB.
>
> There seems to be an s5cmd, which should be much much faster than s3cmd.
There is both s4cmd, s5cmd, minio-mc and rclone which all have some
things that make them "better" than s3cmd in various ways, at the
expense of
Den fre 20 aug. 2021 kl 10:34 skrev Szabo, Istvan (Agoda)
:
>
> So 1x5GB so like 50TB/file?
Yes. Aws seems to have a max of 1000 pieces, which gives them a max
size of 5TB, but from your numbers, it seems you have higher limits
than aws.
--
May the most significant bit of your life be
> > S3cmd chunks 15MB.
>
There seems to be an s5cmd, which should be much much faster than s3cmd.
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io
> > Strange to change this in a patch version. This could have all kinds
> of nasty implications during upgrade. Firewalls being a prominent one.
> Or am I misunderstanding the change?
> >
> > It's also not mentioned prominently in the release notes.
>
> I'm not entirely sure whether one should
Hello,
> Recently, I just noticed that there are a lot of logs about Broken pipe
> error from all RGW nodes.
> We set up the cluster by using Ceph-ansible script. The current version of
> cluster is Octopus (15.2.13). After checking the configuration in RGW
> nodes, I see that there is a config
Den fre 20 aug. 2021 kl 09:20 skrev Szabo, Istvan (Agoda)
:
>
> S3cmd chunks 15MB.
No, s3cmd chunks the size you tell it to, but defaults to some value.
--multipart-chunk-size-mb=SIZE
Size of each chunk of a multipart upload. Files bigger
than
> On 20 Aug 2021, at 09:17, Hans van den Bogert wrote:
>
> Strange to change this in a patch version. This could have all kinds of
> nasty implications during upgrade. Firewalls being a prominent one. Or am I
> misunderstanding the change?
>
> It's also not mentioned prominently in the
Hi,
this seems to be a reoccuring issue, I had the same just yesterday in
my lab environment running on 15.2.13. If I don't specify other
criteria in the yaml file then I'll end up with standalone OSDs
instead of the desired rocksDB on SSD. Maybe this is still a bug, I
didn't check. My
Strange to change this in a patch version. This could have all kinds of
nasty implications during upgrade. Firewalls being a prominent one. Or
am I misunderstanding the change?
It's also not mentioned prominently in the release notes.
Hans
On 8/20/21 8:54 AM, Stefan Fleischmann wrote:
If
Den fre 20 aug. 2021 kl 08:32 skrev Szabo, Istvan (Agoda)
:
> These are the values in octopus:
> "rgw_max_put_size": "5_368_709_120",
> "rgw_multipart_part_upload_limit": "1",
> "rgw_multipart_min_part_size": "5242880",
>
> Correct me if I'm wrong but the multipart parts size is
If you have a look at https://hub.docker.com/r/ceph/ceph
"As of August 2021, new ceph/ceph container images are pushed to quay.io
registry only. Docker hub won't receive new content for that specific image but
current images remain available."
Looks like everyone has to pull from quay.io
On Sun, 15 Aug 2021 at 00:10, Jadkins21 wrote:
> Am I just being too impatient ? or did I miss something around docker
> being discontinued for cephadmin ? (hope not, it's great)
>
not showing via podman either:
root@rnk-00:~# podman pull docker.io/ceph/ceph:v15.2.14
Trying to pull
43 matches
Mail list logo
