Hello !
It seems like ceph-volume from Ceph Pacific 16.2.11 has a problem with
same LV names in different VG.
I use ceph-ansible (stable-6), with a pre-existing LVM configuration.
Here's the error :
TASK [ceph-osd : include_tasks scenarios/lvm.yml]
Le 2023-02-06 14:11, Eugen Block a écrit :
What does the active mgr log when you try to access the dashboard?
Please paste your rgw config settings as well.
Ah, Sorry to hijack, but I also can't access Object Storage menus in the
Dashboard since upgrading from 16.2.10 to 16.2.11.
Here are th
Hi Cephers,
I have large OMAP objects on one of my cluster (certainly due to a big
bucket deletion, and things not completely purged).
Since there is no tool to either reconstruct index from data or purge
unused index, I thought I can use mutlisite replication.
As I am in a multisite config
Hi,
My problem is the opposite !
I don't use SSL on RGWs, because I use a load balancer with HTTPS endpoint.
so no problem with certificates and IP adresses.
With 16.2.11, it does not work anymore because it uses DNS names, and those
names are resolving to a management IP, which is not the networ
I forgot, there's a similar bug around that :
https://tracker.ceph.com/issues/58811
Le mardi 11 avril 2023, 22:45:28 CEST Gilles Mocellin a écrit :
> Hi,
>
> My problem is the opposite !
> I don't use SSL on RGWs, because I use a load balancer with HTTPS endpoint
Hello cephers !
As I was asking in another thread ([RGW] Rebuilding a non master zone),
I try to find the best way to rebuild a zone in a multisite config.
The goal is to get rid of remaining Large OMAP objects.
The simplest way, as I can rely only on the primary zone, is to :
- remove the zone
Le jeudi 13 avril 2023, 18:20:27 CEST Chris Palmer a écrit :
> Hi
Hello,
> I have 3 Ceph clusters, all configured similarly, which have been happy
> for some months on 17.2.5:
>
> 1. A test cluster
> 2. A small production cluster
> 3. A larger production cluster
>
> All are debian 11 built
Hi Cephers,
In a multisite config, with one zonegroup and 2 zones, when I look at
`radiosgw-admin zonegroup get`,
I see by defaut these two parameters :
"log_meta": "false",
"log_data": "true",
Where can I find documentation on these, I can't find.
I set log_meta to tr
quot;
>
> I hope that helps.
> Rich
>
> On Mon, 5 Jun 2023 at 20:42, Gilles Mocellin
>
> wrote:
> > Hi Cephers,
> >
> > In a multisite config, with one zonegroup and 2 zones, when I look at
> > `radiosgw-admin zonegroup get`,
> >
&
Hello,
I still have large OMAP objects since a year.
These objects are probably from an ancient bucket that has been removed.
So I cannot use bilog trim. Depp-scrub dos nothing.
Also, even if I don't have a huge cluster (my Object Storage pools is
only arounde 10TB), the rgw-orphan-list is too
Hello Cephers,
I was using Ceph with OpenStack, and users could add, remove credentials
with `openstack ec2 credentials` commands.
But, we are moving our Object Storage service to a new cluster, and
didn't want to tie it with OpenStack.
Is there a way to have a bit of self service for Rados G
Le lundi 9 octobre 2023, 17:12:53 CEST Casey Bodley a écrit :
> On Mon, Oct 9, 2023 at 9:16 AM Gilles Mocellin
>
> wrote:
> > Hello Cephers,
> >
> > I was using Ceph with OpenStack, and users could add, remove credentials
> > with `openstack ec2 credentials` com
Le dimanche 4 février 2024, 09:29:04 CET duluxoz a écrit :
> Hi Cedric,
>
> That's what I thought - the access method shouldn't make a difference.
>
> No, no lvs details at all - I mean, yes, the osds show up with the lvs
> command on the ceph node(s), but not on the individual pools/images (on
Perhaps there is a partition table on your device.
What does show :
fdisk -l /dev/rbd0
If there is, you can create additional devices with :
kpartx /dev/rbd0
And you'll have /dev/rbd0p1 wich will perhaps bee a PV.
Le 5 février 2024 07:51:24 GMT+01:00, duluxoz a écrit :
>Mounting/Mapping command
Le dimanche 4 février 2024, 09:29:04 CET duluxoz a écrit :
> Hi Cedric,
>
> That's what I thought - the access method shouldn't make a difference.
>
> No, no lvs details at all - I mean, yes, the osds show up with the lvs
> command on the ceph node(s), but not on the individual pools/images (on
Le jeudi 11 avril 2024, 23:29:30 CEST Casey Bodley a écrit :
> unfortunately, this cloud sync module only exports data from ceph to a
> remote s3 endpoint, not the other way around:
>
> "This module syncs zone data to a remote cloud service. The sync is
> unidirectional; data is not synced back fr
Le jeudi 11 avril 2024, 23:44:05 CEST Gilles Mocellin a écrit :
> Le jeudi 11 avril 2024, 23:29:30 CEST Casey Bodley a écrit :
>
> > unfortunately, this cloud sync module only exports data from ceph to a
> > remote s3 endpoint, not the other way around:
> >
> > &qu
Hello Cephers,
On a new cluster, I only have 2 RBD block images, and the Dashboard
doesn't manage to list them correctly.
I have this message :
Warning
Displaying previously cached data for pool veeam-repos.
Sometime it disappears, but as soon as I reload or return to the listing
page, it
Hi !
I respond to the list, as it may help others.
I also reorder the response.
> On Mon, Jan 18, 2021 at 2:41 PM Gilles Mocellin <
>
> gilles.mocel...@nuagelibre.org> wrote:
> > Hello Cephers,
> >
> > On a new cluster, I only have 2 RBD block images, and th
Hello,
As we know, with 64k for bluestore_min_alloc_size_hdd (I'm only using
HDDs),
in certain conditions, especially with erasure coding,
there's a leak of space while writing objects smaller than 64k x k
(EC:k+m).
Every object is divided in k elements, written on different OSD.
My main us
5:11 AM Gilles Mocellin
wrote:
Hello,
As we know, with 64k for bluestore_min_alloc_size_hdd (I'm only
using
HDDs),
in certain conditions, especially with erasure coding,
there's a leak of space while writing objects smaller than 64k x k
(EC:k+m).
Every object is divided in k element
2021, at 13:12, Gilles Mocellin
wrote:
Hello,
As we know, with 64k for bluestore_min_alloc_size_hdd (I'm only using
HDDs),
in certain conditions, especially with erasure coding,
there's a leak of space while writing objects smaller than 64k x k
(EC:k+m).
Every object is divided
imestamp: Fri Jan 15 18:25:13 2021
access_timestamp: Fri Jan 15 18:25:13 2021
modify_timestamp: Fri Jan 15 19:22:54 2021
Kind Regards,Ernesto
On Thu, Jan 21, 2021 at 11:02 PM Gilles Mocellin
wrote:
Hi !
I respond to the list, as it may help others.
I also reorder the respo
Hello,
Since I upgrade to Ceph Octopus v15.2.11, on Ubuntu 18.04,
Radosgw crash straight at start.
On Two clusters, one Lab, and some test on a production cluster, shows
the same crash for radosgw.
As I don't find any similar bug in the Tracker, neither in this mailing
list... Am I alone ?
, so the next point releases should resolve the
crashes
On Fri, May 7, 2021 at 10:51 AM Gilles Mocellin
wrote:
Hello,
Since I upgrade to Ceph Octopus v15.2.11, on Ubuntu 18.04,
Radosgw crash straight at start.
On Two clusters, one Lab, and some test on a production cluster, shows
the same
Hello Cephers,
On a capacitive Ceph cluster (13 nodes, 130 OSDs 8To HDD), I'm migrating a 40
To image from a 3+2 EC pool to a 8+2 one.
The use case is Veeam backup on XFS filesystems, mounted via KRBD.
Backups are running, and I can see 200MB/s Throughput.
But my migration (rbd migrate prep
Le 2021-06-22 20:21, Gilles Mocellin a écrit :
Hello Cephers,
On a capacitive Ceph cluster (13 nodes, 130 OSDs 8To HDD), I'm
migrating a 40
To image from a 3+2 EC pool to a 8+2 one.
The use case is Veeam backup on XFS filesystems, mounted via KRBD.
Backups are running, and I can see
Le 2021-06-23 14:51, Alexander E. Patrakov a écrit :
вт, 22 июн. 2021 г. в 23:22, Gilles Mocellin
:
Hello Cephers,
On a capacitive Ceph cluster (13 nodes, 130 OSDs 8To HDD), I'm
migrating a 40
To image from a 3+2 EC pool to a 8+2 one.
The use case is Veeam backup on XFS filesy
Hello,
As a follow-up of the thread "RBD migration between 2 EC pools : very slow".
I'm running Octopus 15.2.13.
RBD migration seems really fragile.
I started a migration to change the data pool (from an EC 3+2 to an EC 8+2) :
- rbd migration prepare
- rbd migration execute
=> 4% after 6h, and
Hello Cephers,
I'm disappointed. I thought I'v had found a good way to migrate from one data
pool to
another, without too much downtime.
I use XFS on RBD, via KRBD to store backups (see another thread). XFS with
reflink and crc
(accelerate Veeam merges).
Also, I want to migrate from a EC k3m2
Just some more informations :
I use Ceph Octopus 15.2.13 on Ubuntu 18.04, deployed with ceph-ansible,
no container (except grafana/prometheus/node_exporter).
Le 2021-07-12 23:42, Gilles Mocellin a écrit :
Hello Cephers,
I'm disappointed. I thought I'v had found a good way to migrat
Le vendredi 27 août 2021, 09:18:01 CEST Francesco Piraneo G. a écrit :
> > For this August Debian testing became a Debian stable with LTS support.
> >
> > But I see that only sid repo exists, no testing and no new stable
> > bullseye.
> >
> > May be some one knows, when there are plans to have a
Hello,
We see large omap objects warnings on the RGW bucket index pool.
The objects OMAP keys are about objects in one identified big bucket.
Context :
=
We use S3 storage for an application, with ~1,5 M objects.
The production cluster is "replicated" with rclone cron jobs on another
d
Hello Cephers,
Anyone who had that problem find a workaround ?
I thing this bug represent well what's all about :
https://tracker.ceph.com/issues/51429[1]
We have a cluster that it this and cannot auto-reshard. LARGE OMAP Objects
stays there and
cannot be removed also by pg deep scrub.
As
Le lundi 10 janvier 2022, 11:42:11 CET Matthew Vernon a ?crit :
> Hi,
>
> On 07/01/2022 18:39, Gilles Mocellin wrote:
> > Anyone who had that problem find a workaround ?
>
> Are you trying to reshard a bucket in a multisite setup? That isn't
> expected to work (and,
Le 2022-01-11 00:33, Gilles Mocellin a écrit :
Le lundi 10 janvier 2022, 11:42:11 CET Matthew Vernon a ?crit :
Hi,
On 07/01/2022 18:39, Gilles Mocellin wrote:
> Anyone who had that problem find a workaround ?
Are you trying to reshard a bucket in a multisite setup? That isn't
expected
Hello !
On our Octopus (v15.2.15) cluster, mainly used for OpenStack,
We had several OSD crash.
Some would not restart with "no available blob id" assertion.
We found several related bugs :
https://tracker.ceph.com/issues/48216
https://tracker.ceph.com/issues/38272
The workaround that works is
ter's
usage? So major use case is RBD, right? Replicated or EC pools? How
often snapshots are taken if any?
Thanks,
Igor
On 3/3/2022 1:45 PM, Gilles Mocellin wrote:
Hello !
On our Octopus (v15.2.15) cluster, mainly used for OpenStack,
We had several OSD crash.
Some would not restart wi
Hello Cephers,
We've just configured multi-site between an existing Octopus cluster and
a new one in another Datacenter.
On our biggest bucket (replic_cfn_prod/cfb : 1.4 M objects, 670 GB) we
have many errors like that on the new site :
2022-03-15T10:21:00.800+0100 7f9834750700 1 == st
Hum, it seems I hit that bug : https://tracker.ceph.com/issues/50785
Merged in Pacific, not backported to Octopus, at least not linked to
this issue.
One more reason to migrate to Pacific...
Le 2022-03-15 10:43, Gilles Mocellin a écrit :
Hello Cephers,
We've just configured multi
rge omap in our RGW
index pool...
>
> On Wed, Dec 1, 2021 at 4:32 AM Gilles Mocellin <
>
> gilles.mocel...@nuagelibre.org> wrote:
> > Hello,
> >
> > We see large omap objects warnings on the RGW bucket index pool.
> > The objects OMAP k
Some more information about our issue (I work with Wissem).
As the OSD are crashing only on one node, we focus on it.
We found that it's the only node where we also see taht kind of error in
the OSD logs :
2022-04-08T11:38:26.464+0200 7fadaf877700 0 bad crc in data 3052515915
!= exp 38845088
nge the AOC cable (SFP+
+ Fibre).
But, it's not a Ceph problem. Just a hardware one, that only Ceph has
caught !
Le 2022-04-08 11:53, Gilles Mocellin a écrit :
Some more information about our issue (I work with Wissem).
As the OSD are crashing only on one node, we focus on it.
We found th
Le lundi 11 avril 2022, 10:26:31 CEST Gilles Mocellin a écrit :
> Just a follow-up.
>
> I've found that a specific network interface is causing this.
> We have bonds :
> - 1 management bond0
> - 1 storage access on bond1
> - 1 storage replication on bond2
>
> A
Le lundi 25 avril 2022, 17:46:04 CEST Chris Page a ?crit :
> Hi,
>
> Every now and then I am getting the following logs -
>
> pve01 2022-04-25T16:41:03.109+0100 7ff35b6da700 0 bad crc in data
> 3860390385 != exp 919468086 from v1:10.0.0.111:0/873787122
> pve01 2022-04-25T16:41:04.361+0100 7fb0e2
Le 2022-04-27 08:43, Konstantin Shalygin a écrit :
Just for memo record, what is your network card and driver?
ethtool -i eth0
Of course.
It's a Dell card, Broadcom Adv. Dual 25Gb Ethernet.
$ sudo ethtool -i enp129s0f1np1
driver: bnxt_en
version: 1.8.0
firmware-version: 218.0.219.13/pkg 21.8
Hello Cephers,
I'm trying to diagnose who's doing what on our cluster, which suffer
from SLOW_OPS, High latency periods since Pacific.
And I can't see all pool / images in RBD stats.
I had activated RBD image stats while running Octopus, now it seems we
only need to define mgr/prometheus/rbd_
Hi,
In our Ceph Pacific clusters (16.2.10) (1 for OpenStack and S3, 2 for
backup on RBD and S3),
since the upgrade to Pacific, we have regularly the MGR not responding,
not seen anymore in ceph status.
The process is still there.
Noting in the MGR log, just no more logs.
Restarting the servic
> just with similar symptoms.
>
> Regards,
> Eugen
>
> Zitat von Gilles Mocellin :
> > Hi,
> >
> > In our Ceph Pacific clusters (16.2.10) (1 for OpenStack and S3, 2
> > for backup on RBD and S3),
> > since the upgrade to Pacific, we have regularly
Hi !
I'm stuck with "no available blob id" during the start of an OSD.
It seems there's a workaround back-ported only in nautilus (Bug
https://tracker.ceph.com/issues/38272), but I use mimic for now.
Someone has an operational workaround ?
Or should I recreate my OSD ?
And what is the easiest way
Hello,
I don't use netplan, and still on Ubuntu 16.04.
But I use VLAN? on the bond, not directly on the interfaces :
bond0 :
- enp179s0f0
- enp179s0f1
Then I use bond0.323 and bond0.324.
(I use a bridge on top to be more like my OpenStack cluster, and with more
friendly names : br-mgmt, br-sto
parate (2 nics,
1 bond each).
>> On 31 Mar 2020, at 18:33, Gilles Mocellin
>> wrote:
>>
>> Hello,
>>
>> I don't use netplan, and still on Ubuntu 16.04.
>> But I use VLAN? on the bond, not directly on the interfaces :
>>
>> bond0 :
>>
52 matches
Mail list logo
