[ceph-users] Re: Snapshot automation/scheduling for rbd?

2024-02-05 Thread Jeremy Hansen
hat there are > limitations with KVM and disk snapshots but good to give it a try. > > Thanks > > > Get Outlook for Android (https://aka.ms/AAb9ysg) > From: Jeremy Hansen > Sent: Saturday, February 3, 2024 11:39:19 PM > To: ceph-users@ceph.io > Subject: [ceph-users] Re: Sn

[ceph-users] Re: Snapshot automation/scheduling for rbd?

2024-02-05 Thread Jeremy Hansen
to > the vm to freeze the fs if the vm supports it. > > > > > > Am I just off base here or missing something obvious? > > > > Thanks > > > > > > > > > > On Thursday, Feb 01, 2024 at 2:13 AM, Jeremy Hansen > <mailto:jer...@skidrow.l

[ceph-users] Re: Snapshot automation/scheduling for rbd?

2024-02-03 Thread Jeremy Hansen
Am I just off base here or missing something obvious? Thanks > On Thursday, Feb 01, 2024 at 2:13 AM, Jeremy Hansen (mailto:jer...@skidrow.la)> wrote: > Can rbd image snapshotting be scheduled like CephFS snapshots? Maybe I missed > it in the documentation but it looked lik

[ceph-users] Snapshot automation/scheduling for rbd?

2024-02-01 Thread Jeremy Hansen
Can rbd image snapshotting be scheduled like CephFS snapshots? Maybe I missed it in the documentation but it looked like scheduling snapshots wasn’t a feature for block images. I’m still running Pacific. We’re trying to devise a sufficient backup plan for Cloudstack and other things residing in

[ceph-users] Upgrading from 16.2.11?

2024-01-04 Thread Jeremy Hansen
I’d like to upgrade from 16.2.11 to the latest version. Is it possible to do this in one jump or do I need to go from 16.2.11 -> 16.2.14 -> 17.1.0 -> 17.2.7 -> 18.1.0 -> 18.2.1? I’m using cephadm. Thanks -jeremy signature.asc Description: PGP signature

[ceph-users] Ceph as rootfs?

2024-01-03 Thread Jeremy Hansen
Is it possible to use Ceph as a root filesystem for a pxe booted host? Thanks ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: Stray host/daemon

2023-12-01 Thread Jeremy Hansen
Found my previous post regarding this issue. Fixed by restarting mgr daemons. -jeremy > On Friday, Dec 01, 2023 at 3:04 AM, Me (mailto:jer...@skidrow.la)> wrote: > I think I ran in to this before but I forget the fix: > > HEALTH_WARN 1 stray host(s) with 1 daemon(s) not managed by cephadm >

[ceph-users] Stray host/daemon

2023-12-01 Thread Jeremy Hansen
I think I ran in to this before but I forget the fix: HEALTH_WARN 1 stray host(s) with 1 daemon(s) not managed by cephadm [WRN] CEPHADM_STRAY_HOST: 1 stray host(s) with 1 daemon(s) not managed by cephadm stray host cn06.ceph.fu.intra has 1 stray daemons: ['mon.cn03'] Pacific 16.2.11 How do I

[ceph-users] Re: Removed host still active, sort of?

2023-06-11 Thread Jeremy Hansen
Got around this issue by restarting the mgr daemons. -jeremy > On Saturday, Jun 10, 2023 at 11:26 PM, Me (mailto:jer...@skidrow.la)> wrote: > I see this in the web interface in Hosts and under cn03’s devices tab > > SAMSUNG_HD502HI_S1VFJ9ASB08190 > Unknown > n/a > sdg > mon.cn04 > > > 1 total >

[ceph-users] Re: Removed host still active, sort of?

2023-06-11 Thread Jeremy Hansen
I see this in the web interface in Hosts and under cn03’s devices tab SAMSUNG_HD502HI_S1VFJ9ASB08190 Unknown n/a sdg mon.cn04 1 total Which doesn’t make sense. There is no daemons running on this host and I noticed the daemon lists looks like its one that should be on another node. There is

[ceph-users] Re: Removed host still active, sort of?

2023-06-11 Thread Jeremy Hansen
I also see this error in the logs: 6/10/23 11:09:01 PM[ERR]host cn03.ceph does not exist Traceback (most recent call last): File "/usr/share/ceph/mgr/orchestrator/_interface.py", line 125, in wrapper return OrchResult(f(*args, **kwargs)) File "/usr/share/ceph/mgr/cephadm/module.py", line 1625,

[ceph-users] Removed host still active, sort of?

2023-06-10 Thread Jeremy Hansen
I’m going through the process of transitioning to new hardware. Pacific 16.2.11. I drained the host, all daemons were removed. Did the ceph orch host rm [ceph: root@cn01 /]# ceph orch host rm cn03.ceph Error EINVAL: host cn03.ceph does not exist Yet I see it here: ceph osd crush tree |grep

[ceph-users] Re: Ceph drain not removing daemons

2023-06-09 Thread Jeremy Hansen
Figured out how to cleanly relocate daemons via the interface. All is good. -jeremy > On Friday, Jun 09, 2023 at 2:04 PM, Me (mailto:jer...@skidrow.la)> wrote: > I’m doing a drain on a host using cephadm, Pacific, 16.2.11. > > ceph orch host drain > > removed all the OSDs, but these daemons

[ceph-users] Ceph drain not removing daemons

2023-06-09 Thread Jeremy Hansen
I’m doing a drain on a host using cephadm, Pacific, 16.2.11. ceph orch host drain removed all the OSDs, but these daemons remain: grafana.cn06 cn06.ceph.la1 *:3000 stopped 5m ago 18M - - mds.btc.cn06.euxhdu cn06.ceph.la1 running (2d) 5m ago 17M 29.4M - 16.2.11 de4b0b384ad4 017f7ef441ff

[ceph-users] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin

2023-03-03 Thread Jeremy Hansen
3/3/23 2:13:53 AM[WRN]unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin I keep seeing this warning in the logs. I’m not really sure what action to take to resolve this issue. Thanks -jeremy signature.asc

[ceph-users] Re: Upgrade not doing anything...

2023-02-27 Thread Jeremy Hansen
; > (http://quay.io/ceph/ceph:v16.2.11)", > > > "in_progress": true, > > > "services_complete": [], > > > "progress": "", > > > "message": "" > > > } > > > > > > Hasn’t ch

[ceph-users] Re: Upgrade not doing anything...

2023-02-27 Thread Jeremy Hansen
W cephadm, does > that return anything or just hang, also what about ceph health detail? You > can always try ceph orch upgrade pause and then orch upgrade resume, might > kick something loose, so to speak. > On Tue, Feb 28, 2023, 10:39 Jeremy Hansen (mailto:jer...@skidrow.la)> wrote:

[ceph-users] Re: Upgrade not doing anything...

2023-02-27 Thread Jeremy Hansen
PM, Curt (mailto:light...@gmail.com)> wrote: > What does Ceph orch upgrade status return? > On Tue, Feb 28, 2023, 10:16 Jeremy Hansen (mailto:jer...@skidrow.la)> wrote: > > I’m trying to upgrade from 16.2.7 to 16.2.11. Reading the documentation, I > > cut and paste the orch

[ceph-users] Upgrade not doing anything...

2023-02-27 Thread Jeremy Hansen
I’m trying to upgrade from 16.2.7 to 16.2.11. Reading the documentation, I cut and paste the orchestrator command to begin the upgrade, but I mistakenly pasted directly from the docs and it initiated an “upgrade” to 16.2.6. I stopped the upgrade per the docs and reissued the command specifying

[ceph-users] 1 stray daemon(s) not managed by cephadm

2022-07-25 Thread Jeremy Hansen
How do I track down what is the stray daemon? Thanks -jeremy ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Two osd's assigned to one device

2022-07-25 Thread Jeremy Hansen
I have a situation (not sure how it happened), but Ceph believe I have two OSD's assigned to a single device. I tried to delete osd.2 and osd.3, but it just hangs. I'm also trying to zap sdc, which claims it does not have an osd, but I'm unable to zap it. Any suggestions? /dev/sdb HDD TOSHIBA

[ceph-users] Re: Issues after a shutdown

2022-07-25 Thread Jeremy Hansen
> On Mon, 25 Jul 2022, 23:05 Jeremy Hansen, > wrote: > >> That results in packet loss: >> >> [root@cn01 ~]# ping -M do -s 8972 192.168.30.14 >> PING 192.168.30.14 (192.168.30.14) 8972(9000) bytes of data. >> ^C >> --- 192.168.30.14 ping statistics ---

[ceph-users] Re: Issues after a shutdown

2022-07-25 Thread Jeremy Hansen
to figure out. Hmmm. Thank you. On Mon, Jul 25, 2022 at 3:01 PM Sean Redmond wrote: > Looks good, just confirm it with a large ping with don't fragment flag set > between each host. > > ping -M do -s 8972 [destination IP] > > > On Mon, 25 Jul 2022, 22:56 Jeremy Hanse

[ceph-users] Re: Issues after a shutdown

2022-07-25 Thread Jeremy Hansen
Does ceph do any kind of io fencing if it notices an anomaly? Do I need to do something to re-enable these hosts if they get marked as bad? On Mon, Jul 25, 2022 at 2:56 PM Jeremy Hansen wrote: > MTU is the same across all hosts: > > - cn01.ceph.la1.clx.corp- > e

[ceph-users] Re: Issues after a shutdown

2022-07-25 Thread Jeremy Hansen
errors 0 dropped 0 overruns 0 carrier 0 collisions 0 10G. On Mon, Jul 25, 2022 at 2:51 PM Sean Redmond wrote: > Is the MTU in n the new rack set correctly? > > On Mon, 25 Jul 2022, 11:30 Jeremy Hansen, > wrote: > >> I transitioned some servers to a new rack and now I'

[ceph-users] Re: [Warning Possible spam] Re: Issues after a shutdown

2022-07-25 Thread Jeremy Hansen
ce that it contains instructions, starting at > "Please make sure that the host is reachable ...". How about starting to > follow those? > > Best regards, > = > Frank Schilder > AIT Risø Campus > Bygning 109, rum S14 > > _

[ceph-users] Re: Issues after a shutdown

2022-07-25 Thread Jeremy Hansen
d+degraded, last acting [26,4] pg 12.7f is stuck undersized for 35m, current state active+undersized+degraded, last acting [9,14] On Mon, Jul 25, 2022 at 12:43 PM Jeremy Hansen < farnsworth.mcfad...@gmail.com> wrote: > Pretty desperate here. Can someone suggest what I might be able to do to &

[ceph-users] Re: Issues after a shutdown

2022-07-25 Thread Jeremy Hansen
osd.34 30689 start_boot > > > > At this point it just keeps printing start_boot, but the dashboard has it > > marked as "in" but "down". > > > > On these three hosts that moved, there were a bunch marked as "out" and > > "down

[ceph-users] Issues after a shutdown

2022-07-25 Thread Jeremy Hansen
I transitioned some servers to a new rack and now I'm having major issues with Ceph upon bringing things back up. I believe the issue may be related to the ceph nodes coming back up with different IPs before VLANs were set. That's just a guess because I can't think of any other reason this would

[ceph-users] Network issues with a CephFS client mount via a Cloudstack instance

2021-08-30 Thread Jeremy Hansen
I’m going to also post this to the Cloudstack list as well. Attempting to rsync a large file to the Ceph volume, the instance becomes unresponsive at the network level. It eventually returns but it will continually drop offline as the file copies. Dmesg shows this on the Cloudstack host

[ceph-users] Re: Only 2/5 mon services running

2021-06-07 Thread Jeremy Hansen
-jeremy > On Jun 7, 2021, at 7:53 PM, Jeremy Hansen wrote: > > Signed PGP part > > In an attempt to troubleshoot why only 2/5 mon services were running, I > believe I’ve broke something: > > [ceph: root@cn01 /]# ceph orch ls > NAME PORTS RUNNING

[ceph-users] Only 2/5 mon services running

2021-06-07 Thread Jeremy Hansen
In an attempt to troubleshoot why only 2/5 mon services were running, I believe I’ve broke something: [ceph: root@cn01 /]# ceph orch ls NAME PORTS RUNNING REFRESHED AGE PLACEMENT alertmanager 1/1 81s ago9d count:1 crash

[ceph-users] Re: Global Recovery Event

2021-06-07 Thread Jeremy Hansen
This seems to have recovered on its own. Thank you -jeremy > On Jun 7, 2021, at 5:44 PM, Neha Ojha wrote: > > On Mon, Jun 7, 2021 at 5:24 PM Jeremy Hansen <mailto:jer...@skidrow.la>> wrote: >> >> >> I’m seeing this in my health status: >> >

[ceph-users] Re: CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s)

2021-06-07 Thread Jeremy Hansen
cephadm rm-daemon --name osd.29 on the node with the stale daemon did the trick. -jeremy > On Jun 7, 2021, at 2:24 AM, Jeremy Hansen wrote: > > Signed PGP part > So I found the failed daemon: > > [root@cn05 ~]# systemctl | grep 29 > > ● ceph-bfa2ad58-c049-11eb-

[ceph-users] Re: CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s)

2021-06-07 Thread Jeremy Hansen
this osd, so this is perhaps left over from a previous osd.29 on this host. How would I go about removing this cleanly and more important, in a way that Ceph is aware of the change, therefore clearing the warning. Thanks -jeremy > On Jun 7, 2021, at 1:54 AM, Jeremy Hansen wrote: > &g

[ceph-users] Re: CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s)

2021-06-07 Thread Jeremy Hansen
… ceph osd ls 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 16 17 18 20 22 23 24 26 27 31 33 34 So how would I approach fixing this? > On Jun 7, 2021, at 1:10 AM, 赵贺东 wrote: > > Hello Jeremy Hansen, > > try: > ceph log last cephadm > > or see files below > /var/log/ceph/cepha

[ceph-users] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s)

2021-06-07 Thread Jeremy Hansen
What’s the proper way to track down where this error is coming from? Thanks. 6/7/21 12:40:00 AM [WRN] [WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 6/7/21 12:40:00 AM [WRN] Health detail: HEALTH_WARN 1 failed cephadm daemon(s) signature.asc Description: Message signed with

[ceph-users] HEALTH_WARN Reduced data availability: 33 pgs inactive

2021-05-28 Thread Jeremy Hansen
I’m trying to understand this situation: ceph health detail HEALTH_WARN Reduced data availability: 33 pgs inactive [WRN] PG_AVAILABILITY: Reduced data availability: 33 pgs inactive pg 1.0 is stuck inactive for 20h, current state unknown, last acting [] pg 2.0 is stuck inactive for 20h,

[ceph-users] Re: Remapping OSDs under a PG

2021-05-28 Thread Jeremy Hansen
id 24 name 'osd.24' weight 3.63869 at location {datacenter=la1,host=cn06,rack=rack1,room=room1,root=default,row=6}: no change My end goal is to create a crush map that is away of two separate racks with independent UPS power to increase our availability in the event of power going out on one of our racks.

[ceph-users] Re: Remapping OSDs under a PG

2021-05-28 Thread Jeremy Hansen
I’m continuing to read and it’s becoming more clear. The CRUSH map seems pretty amazing! -jeremy > On May 28, 2021, at 1:10 AM, Jeremy Hansen wrote: > > Thank you both for your response. So this leads me to the next question: > > ceph osd crush rule create-replicated

[ceph-users] Re: Remapping OSDs under a PG

2021-05-28 Thread Jeremy Hansen
t; Create a crush rule that only chooses non-ssd drives, then > ceph osd pool set crush_rule YourNewRuleName > and it will move over to the non-ssd OSDs. > > Den fre 28 maj 2021 kl 02:18 skrev Jeremy Hansen : >> >> >> I’m very new to Ceph so if this question makes

[ceph-users] Remapping OSDs under a PG

2021-05-27 Thread Jeremy Hansen
I’m very new to Ceph so if this question makes no sense, I apologize. Continuing to study but I thought an answer to this question would help me understand Ceph a bit more. Using cephadm, I set up a cluster. Cephadm automatically creates a pool for Ceph metrics. It looks like one of my ssd