[ceph-users] Pacific Bug?

2024-02-13 Thread Alex
Hello Ceph Gurus! I'm running Ceph Pacific version. if I run ceph orch host ls --label osds shows all hosts label osds or ceph orch host ls --host-pattern host1 shows just host1 it works as expected But combining the two the label tag seems to "take over" ceph orch host ls --label osds --host-pa

[ceph-users] Re: Pacific Bug?

2024-02-14 Thread Alex
Thank you. Appreciate the prompt response. ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Failed adding back a node

2024-03-27 Thread Alex
Hello. We're rebuilding our OSD nodes. Once cluster worked without any issues, this one is being stubborn I attempted to add one back to the cluster and seeing the error below in out logs: cephadm ['--image', 'registry.redhat.io/rhceph/rhceph-5-rhel8:16.2.10-160', 'pull'] 2024-03-27 19:30:53,901

[ceph-users] Re: Failed adding back a node

2024-03-27 Thread Alex
Thanks! Is there a way of trying out the update on one osd first to make sure we don't nuke the entire production cluster? ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: Failed adding back a node

2024-03-27 Thread Alex
Hi Adam! In addition to my earlier question of is there a way of trying a more targeted upgrade first so we don't risk accidentally breaking the entire production cluster, `ceph config dump | grep container_image` shows: global basic container_image registry.redhat.io/rhceph/rhceph-5-rhel8@

[ceph-users] Re: Failed adding back a node

2024-03-28 Thread Alex
Thanks so much! I'll give it a shot. ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: Pacific Bug?

2024-03-29 Thread Alex
Hi again Adam :-) Would you happen to have the Bug Tracker issue for label bug? Thanks. ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: Failed adding back a node

2024-04-02 Thread Alex
Hi Adam. Re-deploying didn't work, but `ceph config dump` showed one of the container_images specified 16.2.10-160. After we removed that var, it instantly redeployed the OSDs. Thanks again for your help. ___ ceph-users mailing list -- ceph-users@ceph.i

[ceph-users] Monitoring

2024-06-18 Thread Alex
Good morning. Our RH Ceph comes with Prometheus monitoring "built in". How does everyone interstate that into their existing monitoring infrastructure so Ceph and other servers are all under one dashboard? Thanks, Alex. ___ ceph-users ma

[ceph-users] Re: Monitoring

2024-06-18 Thread Alex
Alright, thanks. ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: Monitoring

2024-06-18 Thread Alex
Thanks ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: Monitoring

2024-06-18 Thread Alex
But how do you combine it with Prometheus node exporter built into Ceph? ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: Lot of spams on the list

2024-06-24 Thread Alex
They seem to use the same few email address and then make new once. It should be possible to block them once a day to at least cut down the volume of emails but not completely block? ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an

[ceph-users] Re: Lot of spams on the list

2024-06-24 Thread Alex
Thanks Anthony! ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: Paid support options?

2024-08-23 Thread Alex
I'll jump on this thread as well. There's a slight possibility we may want to outsource the management of our Ceph cluster. It's too early to seriously discuss, but since this thread conveniently came today up as we were talking about this topic at work, I'll also ask who can do support, but the c

[ceph-users] Re: octopus rbd cluster just stopped out of nowhere (>20k slow ops)

2022-12-03 Thread Alex Gorbachev
luck! -- Alex Gorbachev https://alextelescope.blogspot.com On Fri, Dec 2, 2022 at 7:54 PM Boris Behrens wrote: > hi, > maybe someone here can help me to debug an issue we faced today. > > Today one of our clusters came to a grinding halt with 2/3 of our OSDs > reporting slow ops

[ceph-users] Re: octopus rbd cluster just stopped out of nowhere (>20k slow ops)

2022-12-04 Thread Alex Gorbachev
isolate an area that is problematic. Sometimes rebooting OSD hosts one at a time. Rebooting switches (if stacked/MLAG) one at a time. Something has got to be there, which makes the problem go away. -- Alex Gorbachev https://alextelescope.blogspot.com On Sun, Dec 4, 2022 at 6:08 AM Boris Behr

[ceph-users] Re: octopus rbd cluster just stopped out of nowhere (>20k slow ops)

2022-12-07 Thread Alex Gorbachev
Hi Boris, You have sysfs access in /sys/block//device - this will show a lot of settings. You can go to this directory on CentOS vs. Ubuntu, and see if any setting is different? -- Alex Gorbachev https://alextelescope.blogspot.com On Wed, Dec 7, 2022 at 5:14 AM Boris Behrens wrote: >

[ceph-users] Re: Trying to throttle global backfill

2023-03-08 Thread Alex Gorbachev
How about sleep: ceph tell osd.* injectargs '--osd-recovery-sleep 0.5' You can raise this number more (in seconds) to obtain your desired throughput -- Alex Gorbachev ISS/Storcium On Wed, Mar 8, 2023 at 6:56 PM Rice, Christian wrote: > I have a large number of misplaced o

[ceph-users] Re: Ganesha NFS: Files disappearing

2023-03-15 Thread Alex Walender
compile it yourself? Best Regards, Alex Walender Am 15.03.23 um 12:02 schrieb Patrick Schlangen: Hi, today I saw a strange situation where files which were copied to a cephfs via Ganesha NFS (deployed via cephadm) disappeared from the NFS directory and then did not show up anymore until I

[ceph-users] Re: Ceph performance problems

2023-03-22 Thread Alex Gorbachev
r workload goes to 5000 IOPS max, you do not need a million IOPS. If you need good latency response, buy the best NVMe drives possible for your use case, because latency will always go all the way to the drive itself. Hope this helps, and others can likely address cephfs aspects for you. -- Alex

[ceph-users] Re: 5 host setup with NVMe's and HDDs

2023-03-29 Thread Alex Gorbachev
example command we used to create OSDs on all 20TB drives on a given Proxmox VE host: lsblk|grep 18.2|grep disk|awk '{print $1}'|xargs -I {} pveceph osd create /dev/{} --encrypted 1 --crush-device-class hdd -db_dev /dev/nvme0n1 --db_dev_size 300 -- Alex Gorbachev ISS/Storci

[ceph-users] Re: Could you please explain the PG concept

2023-04-25 Thread Alex Gorbachev
s_pgs https://www.sebastien-han.fr/blog/2012/10/15/ceph-data-placement/ -- Alex Gorbachev ISS Storcium On Tue, Apr 25, 2023 at 6:10 PM wodel youchi wrote: > Hi, > > I am learning Ceph and I am having a hard time understanding PG and PG > calculus . > > I know that a PG is a col

[ceph-users] Re: architecture help (iscsi, rbd, backups?)

2023-04-27 Thread Alex Gorbachev
must use VMWare, you can present datastores via NFS as well, this has a lot of indirection but is easier to manage. -- Alex Gorbachev ISS Storcium https://www.iss-integration.com On Thu, Apr 27, 2023 at 5:06 PM Angelo Höngens wrote: > Hey guys and girls, > > I'm working on a pr

[ceph-users] Re: architecture help (iscsi, rbd, backups?)

2023-04-29 Thread Alex Gorbachev
Hi Angelo, You can always use Samba to serve shares, it works well with AD, if that is needed. You may want to benchmark your prototypes in an as close to production setting as possible. -- Alex Gorbachev ISS Storcium iss-integration.com On Sat, Apr 29, 2023 at 10:58 PM Angelo Hongens wrote

[ceph-users] Re: backing up CephFS

2023-04-30 Thread Alex Gorbachev
bitmapping (loosely similar to VMWare change block tracking) with Proxmox backup server. Filesystem snapshots IMO are *not* a good backup method, they are useful for point in time consistent data, or to have a back-out point for e.g. revertable changes. -- Alex Gorbachev www.iss-integration.com ISS

[ceph-users] Re: rbd export with export-format 2 exports all snapshots?

2023-08-27 Thread Alex Gorbachev
Tony, From what I recall having worked with snapshots a while ago, you would want export-diff to achieve a differential export. "export" will always go for a full image. -- Alex Gorbachev https://alextelescope.blogspot.com On Sun, Aug 27, 2023 at 8:03 PM Tony Liu wrote: >

[ceph-users] Re: OSD not starting

2023-11-04 Thread Alex Gorbachev
Hi Amudhan, Have you checked the time sync? This could be an issue: https://tracker.ceph.com/issues/17170 -- Alex Gorbachev Intelligent Systems Services Inc. http://www.iss-integration.com https://www.linkedin.com/in/alex-gorbachev-iss/ On Sat, Nov 4, 2023 at 11:22 AM Amudhan P wrote

[ceph-users] Seagate Exos power settings - any experiences at your sites?

2023-11-07 Thread Alex Gorbachev
em to be a huge issue if all defaults on Exos are wrong (stop-and-go on all Ceph/ZFS workloads). -- Best regards, Alex Gorbachev -- Intelligent Systems Services Inc. http://www.iss-integration.com https://www.linkedin.com/in/alex-gorbachev-iss/ ___ ceph-use

[ceph-users] Re: Ceph as rootfs?

2024-01-04 Thread Alex Gorbachev
discussed here: https://www.spinics.net/lists/ceph-users/msg66370.html -- Alex Gorbachev iss-integration.com On Wed, Jan 3, 2024 at 10:10 PM Jeremy Hansen wrote: > Is it possible to use Ceph as a root filesystem for a pxe booted host? > &g

[ceph-users] Re: RBD Image Returning 'Unknown Filesystem LVM2_member' On Mount - Help Please

2024-02-04 Thread Alex Gorbachev
hange -a y and see the LVs -- Alex Gorbachev www.iss-integration.com On Sun, Feb 4, 2024 at 2:55 AM duluxoz wrote: > Hi All, > > All of this is using the latest version of RL and Ceph Reef > > I've got an existing RBD Image (with data on it - not "critical" as I

[ceph-users] Re: RBD Image Returning 'Unknown Filesystem LVM2_member' On Mount - Help Please

2024-02-05 Thread Alex Gorbachev
emove it), and you can see the PV - you should also see the VG in vgs and LV in lvs. Then you mount the LV: mount /dev// / -- Alex Gorbachev www.iss-integration.com On Mon, Feb 5, 2024 at 1:04 AM duluxoz wrote: > ~~~ > Hello, > I think that /dev/rbd* devices are flitered "out&quo

[ceph-users] Re: Performance improvement suggestion

2024-02-20 Thread Alex Gorbachev
ptions for storage that are robust and do not require as much investment. Use ZFS, with NFS if needed. Use bcache/flashcache, or something similar on the client side. Use proper RAM caching in databases and applications. -- Alex Gorbachev Intelligent Systems Services Inc. STORCIUM On Tue, Feb 20,

[ceph-users] Re: OSD with dm-crypt?

2024-02-26 Thread Alex Gorbachev
If you are using a service spec, just set encrypted: true If using ceph-volume, pass this flag: --dmcrypt You can verify similar to https://smithfarm-thebrain.blogspot.com/2020/03/how-to-verify-that-encrypted-osd-is.html -- Alex Gorbachev ISS/Storcium On Mon, Feb 26, 2024 at 10:25 PM

[ceph-users] Re: OSD with dm-crypt?

2024-02-27 Thread Alex Gorbachev
Not possible to set after the fact, to my knowledge. The OSD must be set to dm-crypt at the creation stage. -- Alex Gorbachev ISS/Storcium On Mon, Feb 26, 2024 at 11:45 PM Michael Worsham < mwors...@datadimensions.com> wrote: > I was setting up the Ceph cluster via this URL

[ceph-users] cephadm module fails to load with "got an unexpected keyword argument"

2024-08-19 Thread Alex Sanderson
place": false, "force": true, "zap": true, "hostname": "gsceph1osd07", "original_weight": 4, "drain_started_at": "2024-08-10T09:54:49.132830Z", "drain_stopped_at": null, "drain_done_at": null, "process_started_at": "2024-08-10T09:54:34.367655Z"}] I thought I should put this out there in case anyone else was having a weird issue with a keyword argument problem.  It did not fix the problem with the dashboard, still working on that. Alex ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] ceph-fuse false passed X_OK check

2020-12-16 Thread Alex Taylor
Hi Cephers, I'm using VSCode remote development with a docker server. It worked OK but fails to start the debugger after /root mounted by ceph-fuse. The log shows that the binary passes access X_OK check but cannot be actually executed. see: ``` strace_log: access("/root/.vscode-server/extension

[ceph-users] Re: ceph-fuse false passed X_OK check

2020-12-16 Thread Alex Taylor
Sorry forgot to mention, the ceph version is Luminous v12.2.13 On Thu, Dec 17, 2020 at 9:45 AM Alex Taylor wrote: > > Hi Cephers, > > I'm using VSCode remote development with a docker server. It worked OK > but fails to start the debugger after /root mounted by ceph-fuse.

[ceph-users] Fwd: ceph-fuse false passed X_OK check

2020-12-18 Thread Alex Taylor
cc to the list On Thu, Dec 17, 2020 at 11:39 AM Patrick Donnelly wrote: > > On Wed, Dec 16, 2020 at 5:46 PM Alex Taylor wrote: > > > > Hi Cephers, > > > > I'm using VSCode remote development with a docker server. It worked OK > > but fails to start the

[ceph-users] Fwd: ceph-fuse false passed X_OK check

2020-12-23 Thread Alex Taylor
Hi Patrick, Any updates? Looking forward to your reply :D On Thu, Dec 17, 2020 at 11:39 AM Patrick Donnelly wrote: > > On Wed, Dec 16, 2020 at 5:46 PM Alex Taylor wrote: > > > > Hi Cephers, > > > > I'm using VSCode remote development with a docker server.

[ceph-users] Re: Large rbd

2021-01-21 Thread Alex Gorbachev
e itself - the parallel IOs are not necessarily aligned on RBD boundaries, but for our purposes, and using all SSD OSDs this is enough. Alex Gorbachev iss-integration.com > ___ > ceph-users mailing list -- ceph-users@ceph.io > To unsubscrib

[ceph-users] Re: Using RBD to pack billions of small files

2021-01-31 Thread Alex Gorbachev
your endeavor! -- Alex Gorbachev ISS/Storcium On Sat, Jan 30, 2021 at 10:01 AM Loïc Dachary wrote: > Bonjour, > > In the context Software Heritage (a noble mission to preserve all source > code)[0], artifacts have an average size of ~3KB and there are billions of > them. They n

[ceph-users] Re: Using RBD to pack billions of small files

2021-02-01 Thread Alex Gorbachev
Hi Loïc, Does not borg need a file system to write its files to? We do replicate the chunks incrementally with rsync, and that is a very nice and, importantly, idempotent way, to sync up data to a second site. -- Alex Gorbachev ISS/Storcium On Mon, Feb 1, 2021 at 2:43 AM Loïc Dachary wrote

[ceph-users] Re: Backups of monitor [EXT]

2021-02-15 Thread Alex Gorbachev
there is HA and the ability to bring a mon back from a short while ago, in which case it will synchronize with the rest of the cluster quickly - we had tested this successfully in the lab. It does not seem like going to 5 mons is necessary for you. -- Alex Gorbachev I <ht

[ceph-users] Re: Can't get one OSD (out of 14) to start

2021-04-16 Thread Alex Gorbachev
best practice, and restore your objects. Best regards, -- Alex Gorbachev ISS/Storcium On Fri, Apr 16, 2021 at 10:51 AM Mark Johnson wrote: > I ran an fsck on the problem OSD and found and repaired a couple of > errors. Remounted and started the OSD but it crashed again shortly after >

[ceph-users] Re: Upgrade tips from Luminous to Nautilus?

2021-04-29 Thread Alex Gorbachev
Mark, My main note was to make sure and NOT enable msgr2 until all OSDs are upgraded to Nautilus. I made that mistake early in the lab, and had to work hard to get it back together. Otherwise, pretty smooth process. -- Alex Gorbachev ISS/Storcium On Thu, Apr 29, 2021 at 4:58 AM Mark

[ceph-users] Re: Can not mount rbd device anymore

2021-06-22 Thread Alex Gorbachev
send an email to ceph-users-le...@ceph.io Have you checked the status of the cluster (ceph -s)? Are there any OSD issues, network problems (can you ping your MONs, OSD hosts)? Check your syslog on the client for any timeout entries. That should be a good start, and give you

[ceph-users] Re: Can not mount rbd device anymore

2021-06-22 Thread Alex Gorbachev
and > accessible? > Maybe I have a Filesystem problem? > I would look in syslog, dmesg, and use a -v flag with mount to see if anything else comes up. Maybe parted /dev/rbd0 to see if you can access the partition table. xfs_repair -L can address any issues with XFS, but it should not break

[ceph-users] Maturity of Cephadm vs ceph-ansible for new Pacific deployments

2021-07-30 Thread Alex Petty
I see less risk in using ceph-ansible because it's easily possible to move from ceph-ansible to cephadm, but not the other way. Does anyone have specific positive or negative feedback about Cephadm on recent Ceph versions? Thanks, Alex ___ ceph-use

[ceph-users] Tool to cancel pending backfills

2021-09-25 Thread Alex Gorbachev
decisions were made by balancer, or other tools) - in my case this was a necessity to reweight many OSDs back to 1. This tool saved many days of waiting for an unneeded rebalance. I found the tool at https://golangrepo.com/repo/digitalocean-pgremapper -- Alex Gorbachev https://alextelescope.blogspot.com

[ceph-users] Re: Cluster down

2021-10-13 Thread Alex Gorbachev
Hi Jorge, This looks like a corosync problem to me. If corosync loses connectivity, the Proxmox nodes would fence and reboot. Ideally, you'd have a second ring on different switch(es), even a cheap 1Gb switch will do. -- Alex Gorbachev ISS - Storcium On Wed, Oct 13, 2021 at 7:07 AM Jor

[ceph-users] Re: A change in Ceph leadership...

2021-10-19 Thread Alex Gorbachev
rever appreciate your brilliance, persistence, and pursuit of excellence. -- Alex Gorbachev ISS/Storcium On Fri, Oct 15, 2021 at 10:40 AM Sage Weil wrote: > This fall I will be stepping back from a leadership role in the Ceph > project. My primary focus during the next two months will

[ceph-users] Re: Cluster down

2021-10-19 Thread Alex Gorbachev
on its own VLAN. ring1 goes onto cheap 1Gb switches, and it will take over if needed. -- Alex Gorbachev ISS - Storcium On Tue, Oct 19, 2021 at 6:57 AM Jorge JP wrote: > Hello Alex, > > I don't understand the second part of your response. > > All my ceph nodes are conne

[ceph-users] Pg autoscaling and device_health_metrics pool pg sizing

2021-11-01 Thread Alex Petty
level (512, 1024) or will autoscaling size this pool up? I have never seen any output from ceph osd pool autoscale-status when I am trying to see autoscaling information. I’d appreciate some guidance about configuring PGs on Pacific. Thanks, Alex

[ceph-users] Re: RBDMAP clients rendering theirselfs as "Jewel" in "Luminous" ceph cluster

2021-12-08 Thread Alex Gorbachev
@ceph.io/thread/RUBXOY2L4JD7AYXHTTTXNJI4BPE6S7TX/ http://lists.ceph.com/pipermail/ceph-users-ceph.com/2018-May/027002.html -- Alex Gorbachev https://alextelescope.blogspot.com On Wed, Dec 8, 2021 at 8:38 AM Kamil Kuramshin wrote: > I have already tried kernel 4.19 - newest available from > s

[ceph-users] MDS Journal Replay Issues / Ceph Disaster Recovery Advice/Questions

2022-01-20 Thread Alex Jackson
Hello Ceph Users, I wanted to hopefully get some advice or at least get some questions answered about the Ceph Disaster Recovery Process detailed in the docs. The questions I have are as follows: - Do all the steps need to be performed or can I check the status of the MDS after each until it rec

[ceph-users] Laggy OSDs

2022-03-29 Thread Alex Closs
;t show signs of problems). Happy to provide logs, let me know what would be useful. Thanks for reading this wall :) -Alex MIT CSAIL he/they ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: Laggy OSDs

2022-03-29 Thread Alex Closs
Hi - I've been bitten by that too and checked, and that *did* happen but I swapped them off a while ago. Thanks for your quick reply :) -Alex On Mar 29, 2022, 6:26 PM -0400, Arnaud M , wrote: > Hello > > is swap enabled on your host ? Is swap used ? > > For our cluster we ten

[ceph-users] Re: Pool with ghost used space

2022-04-12 Thread Alex Gorbachev
Hi Joao, I have seen something like this in Luminous after increasing the size from 1 to 3, almost looks like an extra copy is being kept. I was never able to resolve this without recreating the pool. -- Alex Gorbachev On Mon, Apr 11, 2022 at 9:13 PM Joao Victor Rodrigues Soares wrote: >

[ceph-users] Re: Recommendations on books

2022-04-27 Thread Alex Gorbachev
There's this one: https://www.amazon.com/Learning-Ceph-Karan-Singh/dp/1783985623 -- Alex Gorbachev https://alextelescope.blogspot.com On Tue, Apr 26, 2022 at 11:17 PM Angelo Höngens wrote: > Hey guys and girls, > > Can you recommend some books to get started with ceph? I kno

[ceph-users] Re: Newer linux kernel cephfs clients is more trouble?

2022-05-11 Thread Alex Closs
Hey y'all - As a datapoint, I *don't* see this issue on 5.17.4-200.fc35.x86_64. Hosts are Fedora 35 server, with 17.2.0. Happy to test or provide more data from this cluster if it would be helpful. -Alex On May 11, 2022, 2:02 PM -0400, David Rivera , wrote: > Hi, > > My ex

[ceph-users] Re: Cannot repair inconsistent PG

2020-05-27 Thread Alex Gorbachev
Erasure > >> Codings. But this is just a wild guess. > >> > >> I found a similar problem in this mailing list - > >> > http://lists.ceph.com/pipermail/ceph-users-ceph.com/2018-April/026304.html > >> > >> Unfortunately the solution of waitin

[ceph-users] Re: Thank you!

2020-07-20 Thread Alex Gorbachev
..@performair.com > > www.PerformAir.com We have lived through multiple lightning strikes, human errors, and power spikes in clients' datacenters, and have never lost data. We only use 3x replication, at least for the time being and all our data is critical. -- Alex Gorbachev Inte

[ceph-users] Re: Proxmox+Ceph Benchmark 2020

2020-10-13 Thread Alex Gorbachev
Alwin, this is excellent info. We have a lab on AMD with a similar setup with NVMe on Proxmox, and will try these benchmarks as well. -- Alex Gorbachev Intelligent Systems Services Inc. STORCIUM On Tue, Oct 13, 2020 at 6:18 AM Alwin Antreich wrote: > Hello fellow Ceph users, > >

[ceph-users] Re: monitor sst files continue growing

2020-10-29 Thread Alex Gracie
We hit this issue over the weekend on our HDD backed EC Nautilus cluster while removing a single OSD. We also did not have any luck using compaction. The mon-logs filled up our entire root disk on the mon servers and we were running on a single monitor for hours while we tried to finish recovery

[ceph-users] 150mb per sec on NVMe pool

2020-11-10 Thread Alex L
Hi, I have invested in SAMSUNG PM983 (MZ1LB960HAJQ-7) x3 to run a fast pool on. However I am only getting 150mb/sec from these. vfio results directly on the NVMe's: https://docs.google.com/spreadsheets/d/1LXupjEUnNdf011QNr24pkAiDBphzpz5_MwM0t9oAl54/edit?usp=sharing Config and Results of cep

[ceph-users] Re: how many monitor should to deploy in a 1000+ osd cluster

2019-09-25 Thread Alex Gorbachev
are welcome! > don't know this one -- Alex Gorbachev Intelligent Systems Services Inc. > ___ > ceph-users mailing list -- ceph-users@ceph.io > To unsubscribe send an email to ceph-users-le...@ceph.io > _

[ceph-users] Cephfs + docker

2019-09-25 Thread Alex L
Hi, I am trying to figure out why my portainer and pi-hole in docker keeps getting broken databases. All other docker applications are working flawlessly but not these. I am running Ubuntu 18.04 + kernel ceph mount for the data directory. Have looked at how others do it, and they seem to all u

[ceph-users] Re: Cephfs + docker

2019-09-26 Thread Alex Lupsa
't have any problems at all, neither in Portainer nor any other > container. > > The only idea I have right now is to test the FUSE client. If that works, > upgrade to the HWE kernel and test if that works too. > > Best regards, > Patrick > > Alex L mailto:alexut.vo..

[ceph-users] Re: 14.2.4 Deduplication

2019-10-09 Thread Alex Gorbachev
ks > vs snapshots" and https://github.com/ceph/ceph/pull/29283) > -Greg > I was holding off, as this is really not a part of Ceph, but we have done some preliminary testing of VDO as a client on top of RBD, and it looked fine. We also run borg on top of RBD, which is also stabl

[ceph-users] Nautilus power outage - 2/3 mons and mgrs dead and no cephfs

2019-10-11 Thread Alex L
2/3 mons and mgrs are damaged but without any errors. I have loads stored on cephfs so would very much like to get that running as a first priority. Thanks! Alex Info about the home cluster: I run 23 osds on 3 hosts. 6 of these are a SSD cache layer for the spinning rust, as well as the metada

[ceph-users] Re: Nautilus power outage - 2/3 mons and mgrs dead and no cephfs

2019-10-17 Thread Alex L
Hi, I am still having issues accessing my cephfs and managed to pull out more interesting logs, I also have enabled logs to 20/20 that I intend to upload as soon as my ceph tracker account gets accepted. Oct 17 16:35:22 pve21 kernel: libceph: read_partial_message 8ae0e636 signature chec

[ceph-users] Re: Nautilus power outage - 2/3 mons and mgrs dead and no cephfs

2019-10-17 Thread Alex L
Final update. I switched the below from false and everything magically started working! cephx_require_signatures = true cephx_cluster_require_signatures = true cephx_sign_messages = true ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe se

[ceph-users] Re: Nautilus power outage - 2/3 mons and mgrs dead and no cephfs

2019-10-22 Thread Alex L
Hi Wido, It was one of the first thing I checked yes, and it was synched properly. I have the full logs but since everything works now, I am unsure if I should upload them to the tracker ? Thanks, A ___ ceph-users mailing list -- ceph-users@ceph.io To

[ceph-users] Re: Building a petabyte cluster from scratch

2019-12-03 Thread Alex Gorbachev
FYI for ZFS on RBD. https://github.com/zfsonlinux/zfs/issues/3324 We go for a more modest setting with async to 64, not 2. -- Alex Gorbachev Intelligent Systems Services Inc. On Tue, Dec 3, 2019 at 3:07 PM Fabien Sirjean wrote: > Hi Ceph users ! > > After years of using Ceph, w

[ceph-users] Migrate journal to Nvme from old SSD journal drive?

2020-02-04 Thread Alex L
ar on what migration does, nor has the same concept of a DEVICE (/dev/sda is a device for me) it seems. Thanks in advance, Alex [1] - Performance stats: https://docs.google.com/spreadsheets/d/1LXupjEUnNdf011QNr24pkAiDBphzpz5_MwM0t9oAl54/edit?usp=sharing _

[ceph-users] Octopus release announcement

2020-03-02 Thread Alex Chalkias
Hello, I was looking for an official announcement for Octopus release, as the latest update (back in Q3/2019) on the subject said it was scheduled for March 1st. Any updates on that? BR, -- Alex Chalkias *Product Manager* alex.chalk...@canonical.com +33 766599367 *Canonical | **Ubuntu

[ceph-users] Re: Octopus release announcement

2020-03-02 Thread Alex Chalkias
Thanks for the update. Are you doing a beta-release prior to the official launch? On Mon, Mar 2, 2020 at 7:12 PM Sage Weil wrote: > It's getting close. My guess is 1-2 weeks away. > > On Mon, 2 Mar 2020, Alex Chalkias wrote: > > > Hello, > > > > I was looki

[ceph-users] Re: Upgrade Luminous to Nautilus on a Debian system

2020-04-29 Thread Alex Gorbachev
(and it's primarily from not reading the docs very carefully) is that you should NOT enable the messenger 2 protocol until all OSDs have been updated. In other words, Ceph will complain about not running msgr2, but you should leave it like that until all OSDs are on Nautilus. Then you run: c

[ceph-users] Re: Upgrade Luminous to Nautilus on a Debian system

2020-04-30 Thread Alex Gorbachev
Herve, On Wed, Apr 29, 2020 at 2:57 PM Herve Ballans wrote: > Hi Alex, > > Thanks a lot for your tips. I note that for my planned upgrade. > > I take the opportunity here to add a complementary question regarding the > require-osd-release functionality (ceph osd require-os

[ceph-users] Re: What's the best practice for Erasure Coding

2020-05-03 Thread Alex Gorbachev
with filestore have always been needed to sustain a reasonably consistent transfer rate. Very much appreciate any reference info as to your design. Best regards, Alex On Mon, Jul 8, 2019 at 4:30 AM Frank Schilder wrote: > Hi David, >> >> I'm running a cluster with bluestor

[ceph-users] pg_autoscaler on cache will not work

2020-05-04 Thread Alex L
cs? Is this a bug of pg_autoscaler ? BR, Alex ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: Ceph meltdown, need help

2020-05-05 Thread Alex Gorbachev
r 1570.790824, current state > active+undersized+degraded, last acting [63,84] > > pg 3.66 is stuck undersized for 733.682973, current state > undersized+degraded+peered, last acting [63] > > pg 3.68 is stuck undersized for 1570.624462, current state > active+

[ceph-users] Re: Ceph meltdown, need help

2020-05-05 Thread Alex Gorbachev
le_connect_reply connect got BADAUTHORIZER > I had that when my time was off on MONs. We had some NTP problems once at a client site following major power outage, and I recall this exact message. Check your time sync. -- Alex Gorbachev Intelligent Systems Services Inc. > > Str

[ceph-users] Re: Cannot mount RBD on client

2024-06-21 Thread Alex from North
Hi Etienne, indeed, even ```rados ls --pool test``` hangs on the same instruction futex(0x7ffc2de0cb10, FUTEX_WAIT_BITSET_PRIVATE, 0, {tv_sec=10215, tv_nsec=619004859}, FUTEX_BITSET_MATCH_ANY Yes, by netcat I have checked from client side and all OSD ports are succeed ___

[ceph-users] cannot delete service by ceph orchestrator

2024-06-29 Thread Alex from North
Hi everybody! never seen this before and google stay silent. Just found the same question in 2021 but no answer there ((( so, by ceph orch ls I see: root@ceph1:~/ceph-rollout# ceph orch ls NAME PORTSRUNNING REFRESHED AGE PLACEMENT alertmanager

[ceph-users] Re: cannot delete service by ceph orchestrator

2024-06-29 Thread Alex from North
Ah! I guess I got it! So, once all OSDs (made by specification I'd like to delete) are gone - service will disappear as well, right? ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Multisite Pubsub - Duplicates Growing Uncontrollably

2021-10-11 Thread Alex Hussein-Kershaw
uggestions as to why this is occurring? Thanks, Alex ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: [EXTERNAL] Re: Multisite Pubsub - Duplicates Growing Uncontrollably

2021-10-19 Thread Alex Hussein-Kershaw
e network layer, but is getting far enough to cause a pubsub event such that we end up with the duplicates? I saw your email to Dave - great to hear that the pull functionality won't be lost should pubsub be deprecated! Best wishes, Alex -Original Message- From: Yuval Lifshitz S

[ceph-users] Ceph `realm pull` permission denied error

2020-07-13 Thread Alex Hussein-Kershaw
are correct (or atleast `radosgw-admin period get` on the primary site thinks so). Thanks for your help, Alex ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: Ceph `realm pull` permission denied error

2020-07-13 Thread Alex Hussein-Kershaw
Hi Zhenshi, Thanks for the suggestion, unfortunately I have tried this already and had no luck ☹ Best wishes, Alex From: Zhenshi Zhou Sent: 13 July 2020 10:58 To: Alex Hussein-Kershaw Cc: ceph-users@ceph.io Subject: Re: [ceph-users] Ceph `realm pull` permission denied error NOTE: Message is

[ceph-users] Re: Ceph `realm pull` permission denied error

2020-07-13 Thread Alex Hussein-Kershaw
can save some time from my troubles! Thanks, Alex From: Alex Hussein-Kershaw Sent: 13 July 2020 12:22 To: Zhenshi Zhou Cc: ceph-users@ceph.io Subject: RE: [ceph-users] Ceph `realm pull` permission denied error Hi Zhenshi, Thanks for the suggestion, unfortunately I have tried this already and

[ceph-users] S3 bucket lifecycle not deleting old objects

2020-07-28 Thread Alex Hussein-Kershaw
t;, "Owner": { "DisplayName": "hera EAS S3 user", "ID": "hera" }, "IsLatest": false, "Size": 4256 -- ], "NextKeyMarker": "85/49/20200604163626B4C712312312302641-Subscriber", "MaxKeys": 1000, "Prefix": "", "KeyMarker": "84/46/2020060508501821902143658709-Subscriber", "DeleteMarkers": [ { "Owner": { "DisplayName": "hera EAS S3 user", "ID": "hera" }, So those objects still being present seems to be in conflict with the config I have set? Thanks, Alex ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: [EXTERNAL] Re: S3 bucket lifecycle not deleting old objects

2020-07-29 Thread Alex Hussein-Kershaw
ing daft here? Also for info: [qs-admin@portala0 ceph]$ radosgw-admin lc list [ { "bucket": ":ahk-test:22bef6b9-67c8-41e6-9e51-17eaddf906fb.1444202.1", "status": "UNINITIAL" } } Thanks, Alex -Original Message- From: Ro

[ceph-users] RGW Lifecycle Processing and Promote Master Process

2020-08-14 Thread Alex Hussein-Kershaw
be in? Does anyone have any experience with this? Thanks, Alex ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: RGW Lifecycle Processing and Promote Master Process

2020-08-19 Thread Alex Hussein-Kershaw
It's a bug that is causing this to not happen automatically. Kindest regards, Alex -Original Message- From: Casey Bodley Sent: 19 August 2020 15:43 To: Alex Hussein-Kershaw Cc: ceph-users@ceph.io Subject: Re: [ceph-users] RGW Lifecycle Processing and Promote Master Process NOTE:

[ceph-users] S3 Deletes in Multisite Sometimes Not Syncing

2022-12-22 Thread Alex Hussein-Kershaw (HE/HIM)
something we can do it mitigate it? As I understand, we can’t return to a versioning disabled state for this bucket. Thanks, and Season’s Greetings 😊 Alex Kershaw | alex...@microsoft.com<mailto:alex...@microsoft.com> Software Engineer | Azure for Ope

[ceph-users] Re: [EXTERNAL] Re: S3 Deletes in Multisite Sometimes Not Syncing

2023-01-03 Thread Alex Hussein-Kershaw (HE/HIM)
Hi Matthew, That's interesting to hear - especially that you are not using bucket versioning and are seeing the same issue. I was hoping this might go away if I turned off versioning, but if that's not the case this gets a bit more worrying for us! Thanks, Alex -Origin

[ceph-users] Is a direct Octopus to Reef Upgrade Possible?

2024-02-23 Thread Alex Hussein-Kershaw (HE/HIM)
Any advice on this very welcome. Many thanks, Alex ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Multisite RGW with Self-signed CA & Disconnected Upgrade

2024-06-30 Thread Alex Hussein-Kershaw (HE/HIM)
internet access. I can install the docker image directly by copying up the file, but I don't see a way to invoke the "ceph orch upgrade" command with a locally available docker image. Am I missing the obvious? Many thanks, Alex ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io

  1   2   >