yes, "ten per second".
could anyone give me some suggestions on what to poke at?
I have already tried simplifying the ALUA setup at the vmware CLI level.
At the path level, I have tried setting
VMW_PSP_FIXED_AP (VMWSATP_DEFAULT_AA)
or
Fixed(VMware) (VMSATP_DEFAULT_AA)
but no
wer, using the same hardware. In some use cases, slower
than 100io/s.
Could anyone suggest a reason for this? (and ideally, how to retune?)
Did the required minimum effective size of the WAL on SSD grow between
releases, for example?
Target use is an iSCSI storage pool.
--
Philip Brown| Sr. Li
osd.9 config show
I cant even get useful google hits on things like
"ceph find rbd image stripe-unit"
all the pages seem to detail how to set the value, but not how to query it.
Can anyone point me in the right direction?
--
Philip Brown| Sr. Linux System Administrator | Medat
I would appreciate it if anyone could call out specific features involved here.
"upgrade because it's better" doesnt usually fly in cost justification writeups.
- Original Message -
From: "Andrew Ferris"
To: "ceph-users" , "Philip Brown"
"you have to use v6.5 or later, because X
happens", would be very helpful to me when doing a writeup for potential
deployment plans.
--
Philip Brown| Sr. Linux System Administrator | Medata, Inc.
5 Peters Canyon Rd Suite 250
Irvine CA 92606
Office 714.918.1310| Fax 714.918.132
ovide a broken gwcli?
--
Philip Brown| Sr. Linux System Administrator | Medata, Inc.
5 Peters Canyon Rd Suite 250
Irvine CA 92606
Office 714.918.1310| Fax 714.918.1325
pbr...@medata.com| www.medata.com
___
ceph-users mailing list -- ceph-users@ceph.io
To unsub
erm
use ceph-ansible? :)
go to github, and find the correct branch associated with the particular
release of ceph you want to use.
then try to follow the ceph-ansible docs on setup.
For example, to use ceph octopus, you are best off with the "STABLE-5" branch.
After that, a hint:
Just try
AAAND final update: problem fixed.
I had enabled
create_crush_tree: true
in group_vars/osd.yml
but I had neglected to ALSO set
crush_rule_config: true
So now it's all happy
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an
chassis: 'hostA'
root: 'default'
--
Philip Brown| Sr. Linux System Administrator | Medata, Inc.
5 Peters Canyon Rd Suite 250
Irvine CA 92606
Office 714.918.1310| Fax 714.918.1325
pbr...@medata.com| www.medata.com
___
ceph-users mailing li
:
hosts:
hostA1:
osd_crush_location:
host: 'hostA1'
chassis: 'hostA'
root: 'default'
hostA2:
osd_crush_location:
host: 'hostA2'
chassis: 'hostA'
root: 'default'
--
Philip Brown| Sr. Linux
Where does it read it from?
does it keep it in the block.db lv, or the block dev lv, or both?
I removed the vg from the block dev and did wipefs, if I recall.
- Original Message -
From: "Eugen Block"
To: "Philip Brown"
Cc: "ceph-users"
Sent: Tue
ing a 'ceph tell
osd.7 bench'? It sounds very strange, I haven't seen that.
Zitat von Philip Brown :
> I am in a situation where I see conflicting information.
> On the one hand,
> ls -l /var/lib/ceph/osd/ceph-7
> shows a symlink for block device, but no block.db
>
> On t
:00:18 PM
Subject: [ceph-users] Re: bug in ceph-volume create
On 4/5/2021 3:49 PM, Philip Brown wrote:
>
> As soon as you have an HDD fail... you will need to recreate the OSD.. and
> you are then stuck. Because you cant use batch mode for it...
> and you cant do it more granularly,
something that isnt supposed to be there in the
first place?!
(This is a bug all the way back in nautilus, through latest, I believe)
--
Philip Brown| Sr. Linux System Administrator | Medata, Inc.
5 Peters Canyon Rd Suite 250
Irvine CA 92606
Office 714.918.1310| Fax 714.918.1325
pbr...@
?
(This is currently ceph nautilus)
--
Philip Brown| Sr. Linux System Administrator | Medata, Inc.
5 Peters Canyon Rd Suite 250
Irvine CA 92606
Office 714.918.1310| Fax 714.918.1325
pbr...@medata.com| www.medata.com
___
ceph-users mailing list
Yoo have conflated "ceph orch host add" and
"ceph orch host label add"
This is not valid syntax:
ceph orch host add osdev-ctrl3 mon
The docs imply that the trailing "mon" would be ignored, since there is no 6th
argument to that command.
rchestrate a replacement?
otherwise, in my opinion, that flag really needs to be renamed to something
else.
- Original Message -
From: "David Orman"
To: "Eugen Block"
Cc: "Stefan Kooman" , "ceph-users" , "Philip
Brown"
Sent: Thursday, M
sage -
From: "Eugen Block"
To: "Stefan Kooman"
Cc: "ceph-users" , "Philip Brown"
Sent: Friday, March 19, 2021 2:19:55 PM
Subject: [BULK] Re: [ceph-users] Re: ceph octopus mysterious OSD crash
I am quite sure that this case is covered by cephadm alrea
ith this... but it is not here.
- Original Message -
From: "Stefan Kooman"
To: "Philip Brown"
Cc: "ceph-users"
Sent: Friday, March 19, 2021 12:04:30 PM
Subject: Re: [ceph-users] ceph octopus mysterious OSD crash
On 3/19/21 7:47 PM, Philip Brown wr
are already configured, and the SSD is
already mostly sliced.
- Original Message -
From: "Stefan Kooman"
To: "Philip Brown"
Cc: "ceph-users"
Sent: Friday, March 19, 2021 11:34:58 AM
Subject: Re: [ceph-users] ceph octopus mysterious OSD crash
So make sur
. now what can I do?
- Original Message -
From: "Stefan Kooman"
To: "Philip Brown"
Cc: "ceph-users"
Sent: Friday, March 19, 2021 9:58:56 AM
Subject: Re: [ceph-users] ceph octopus mysterious OSD crash
On 3/19/21 3:53 PM, Philip Brown wrote:
> mkay.
> Sooo.
arate db device. "
Has this been fixed yet?
Is it GOING to be fixed?
--
Philip Brown| Sr. Linux System Administrator | Medata, Inc.
5 Peters Canyon Rd Suite 250
Irvine CA 92606
Office 714.918.1310| Fax 714.918.1325
pbr...@medata.com| www.
mkay.
Sooo... what's the new and nifty proper way to clean this up?
The outsider's view is,
"I should just be able to run 'ceph orch osd rm 33'"
but that returns
Unable to find OSDs: ['33']
- Original Message -
From: "Stefan Kooman"
To: "Philip Brown
Unfortunately, the pod wont stay up. So "podman logs" wont work for it.
it is not even visible with "podman ps -a"
- Original Message -
From: "胡 玮文"
To: "Philip Brown"
Cc: "ceph-users"
Sent: Thursday, March 18, 2021 5:56:20 PM
Sub
client(hunting): handle_auth_bad_method server allowed_methods [2] but i
only support [2]
Not the best error message :-}
Now what do I need to do?
- Original Message -
From: "Stefan Kooman"
To: "Philip Brown" , "ceph-users"
Sent: Thursday, March 18
-xx-xx-xx@osd.33.service: main process exited, code=exited,
status=1/FAILURE
bash[1611797]: ceph-xx-xx-xx-xx-osd.33-deactivate
and eventually it just gives up.
smartctl -a doesnt show any errors on the HDD
dmesg doesnt show anything.
So... what do I do?
--
Philip Brown| Sr. Linux System
To whom it may concern:
failure: repodata/repomd.xml from Ceph: [Errno 256] No more mirrors to try.
https://download.ceph.com/rpm-octopus/el7/x86_64/repodata/repomd.xml: [Errno
14] curl#60 - "Peer's Certificate has expired."
___
ceph-users mailing
uot;: 1838126860,
"read_random_buffer_count": 55997,
"read_random_buffer_bytes": 227830937,
"read_count": 26224,
"read_bytes": 1199878580,
"read_prefetch_count": 26023,
"read_prefetch_bytes": 1194535977
more specific search term.
- Original Message -
From: "Maged Mokhtar"
To: "Philip Brown"
Cc: "ceph-users"
Sent: Friday, March 12, 2021 8:04:06 AM
Subject: Re: [ceph-users] Question about delayed write IOs, octopus, mixed
storage
as a side issue, i do no
"First it is not a good idea to mix SSD/HDD OSDs in the same pool,"
Sorry for not being explicit.
I used the cephadm/ceph orch facilities and told them "go set up all my disks".
SO they automatically set up the SSDs to be WAL devices or whatever.
0 0
--
Philip Brown| Sr. Linux System Administrator | Medata, Inc.
5 Peters Canyon Rd Suite 250
Irvine CA 92606
Office 714.918.1310| Fax 714.918.1325
pbr...@medata.com| www.medata.com
___
ceph-users mailing list -- ceph-users@ceph.io
# cephadm version
ceph version 15.2.9 (357616cbf726abb779ca75a551e8d02568e15b17) octopus (stable)
Simply commenting out that line makes it complete the cluster init like I
remember.
--
Philip Brown| Sr. Linux System Administrator | Medata, Inc.
5 Peters Canyon Rd Suite 250
Irvine CA 926
essage -
From: "Philip Brown"
To: "ceph-users"
Sent: Friday, February 19, 2021 3:36:50 PM
Subject: How to get ceph-volume to take pre-existing, working auth?
I'm trying to use ceph-volume to do various things.
It works fine locally, for things like
ceph-volume lvm zap
Bu
run from the same command prompt,
/usr/bin/ceph osd tree -f json
it works fine.
How can i get ceph-volume to just use the creds that are already working
somewhere?
--
Philip Brown| Sr. Linux System Administrator | Medata, Inc.
5 Peters Canyon Rd Suite 250
Irvine CA 92606
Office
ply osd -i howIreallywantOSDs.yml --host=somenewhostname
then it seems to work quite nicely.
I just find it rather surprising that the official fancy new orchestration tool
doesnt do the obvious right thing out of the box.
(Whereas ceph-ansible does)
- Original Message -
From: "Tony Liu"
or a DriveGroup specification is
used, a cephadm service is created"
However, using "ceph orch ps", I dont see any relevantly named service.
Where else should I be looking?
--
Philip Brown| Sr. Linux System Administrator | Medata, Inc.
5 Peters Canyon Rd Suite 250
Irvine CA 9
ever by hand, when a
computer should be perfectly capable of auto generating this stuff itself
--
Philip Brown| Sr. Linux System Administrator | Medata, Inc.
5 Peters Canyon Rd Suite 250
Irvine CA 92606
Office 714.918.1310| Fax 714.918.1325
pbr...@medata.com| www.medata.com
__
ive always run it against the block dev
- Original Message -
From: "Matt Wilder"
To: "Philip Brown"
Cc: "ceph-users"
Sent: Wednesday, February 10, 2021 12:06:55 PM
Subject: Re: [ceph-users] Re: Device is not available after zap
Are you running zap on the
Sorry, not much to say other than a "me too".
i spent a week testing ceph configurations.. it should have only been 2 days.
but a huge amount of my time was wasted because I needed to do a full reboot on
the hardware.
on a related note: sometimes "zap" didnt fully clean things up. I had to
to the old commandline, including
rbd pool init testpool
So maybe there's something that the command line is doing, that the gui SHOULD
be doing, but isnt.
- Original Message -
From: "Philip Brown"
To: "ceph-users"
Sent: Tuesday, December 22, 2020 4:43:32 PM
Su
timed out
and no errors in dmesg output
if I try to disable those features anyway, I get
librbd::Operations: one or more requested features are already disabled(22)
Invalid argument
nothing in /var/log/ceph/cephadm.log either
Any suggestions?
--
Philip Brown| Sr. Linux System Administrator | Medat
ph-version 15.2.8
--
Philip Brown| Sr. Linux System Administrator | Medata, Inc.
5 Peters Canyon Rd Suite 250
Irvine CA 92606
Office 714.918.1310| Fax 714.918.1325
pbr...@medata.com| www.medata.com
___
ceph-users mailing list -- ceph-use
ople who have done multi homing under octopus, would be
appreciated.
Note that my initial proof-of-concept cluster is just 3 physical nodes, so
everything needs to live on them.
--
Philip Brown| Sr. Linux System Administrator | Medata, Inc.
5 Peters Canyon Rd Suite 250
Irvine CA 9260
, as viewed via "cephadm shell"
So, what are the options here?
--
Philip Brown| Sr. Linux System Administrator | Medata, Inc.
5 Peters Canyon Rd Suite 250
Irvine CA 92606
Office 714.918.1310| Fax 714.918.1325
pbr...@medata.com| www.medata.com
__
if I cant control the behaviour who is to say it
wont mysteriously come back?
- Original Message -
From: "Philip Brown"
To: "Sebastian Trojanowski"
Cc: "ceph-users"
Sent: Thursday, December 17, 2020 9:02:05 AM
Subject: Re: [ceph-users] Re: performanc
alse
in /etc/ceph/ceph.conf should work also.
Except it doesnt.
Even after fully shutting down every node in the ceph cluster and doing a cold
startup.
is that a bug?
- Original Message -----
From: "Jason Dillaman"
To: "Philip Brown"
Cc: "dillaman" , "c
I am happy to say, this seems to have been the solution.
After running
ceph config set global rbd_cache false
I can now run the full 256 thread varient,
fio --direct=1 --rw=randwrite --bs=4k --ioengine=libaio --filename=/dev/rbd0
--iodepth=256 --numjobs=1 --time_based --group_reporting
command line that actually *works*, for the "rbd config"
variant, etc.
- Original Message -
From: "Jason Dillaman"
To: "Philip Brown"
Cc: "ceph-users"
Sent: Thursday, December 17, 2020 7:48:22 AM
Subject: Re: [ceph-users] Re: bug? cant turn off rbd
Huhhh...
Its unfortunate that every google search i did for turning off rbd cache,
specified "put it in the [client] section".
Doh.
Maybe this would make a good candidate to update the ceph rbd docs?
Speaking of which.. what is the *exact* syntax for that command please?
None of the below
d_cache": "true",
What else am I supposed to do???
--
Philip Brown| Sr. Linux System Administrator | Medata, Inc.
5 Peters Canyon Rd Suite 250
Irvine CA 92606
Office 714.918.1310| Fax 714.918.1325
pbr...@medata.com| www.medata.com
___
c
btw, I also tried putting
[client]
rbd cache = false
in the /etc/ceph/ceph.conf file on the main node, then doing
systemctl stop ceph.target
systemctl status ceph.target
on the main node.
but after restart, it tells me rbd cache is still enabled
# ceph --admin-daemon
e your cache/buffers are full and need flush. It
could harmful your env.
BR,
Sebastian
On 11.12.2020 19:08, Philip Brown wrote:
>
> I have a new 3 node octopus cluster, set up on SSDs.
>
> I'm running fio to benchmark the setup, with
>
> fio --filename=/dev/rbd0 --direct=1
not valid
Aborted (core dumped)
It would be nice if it did a more user-friendly arg check, and said "you need
to specify pool" instead of coredumping.
- Original Message -
From: "Jason Dillaman"
To: "Philip Brown"
Cc: "ceph-users"
Sent: Tuesday,
It wont be on the same node...
but since as you saw, the problem still shows up with iodepth=32 seems
we're still in the same problem ball park
also... there may be 100 client machines.. but each client can have anywhere
between 1-30 threads running at a time.
as far as fio using the rados
o: "Philip Brown"
Cc: "ceph-users"
Sent: Monday, December 14, 2020 10:19:48 AM
Subject: Re: [ceph-users] performance degredation every 30 seconds
On Mon, Dec 14, 2020 at 12:46 PM Philip Brown wrote:
>
> Further experimentation with fio's -rw flag, setting to rw=re
?
- Original Message -
From: "Philip Brown"
To: "dillaman"
Cc: "ceph-users"
Sent: Monday, December 14, 2020 9:01:21 AM
Subject: Re: [ceph-users] performance degredation every 30 seconds
Aha Insightful question!
running rados bench write to the same pool,
5200 Pro SSDs on all nodes.
- Original Message -
From: "Jason Dillaman"
To: "Philip Brown"
Cc: "ceph-users"
Sent: Monday, December 14, 2020 8:33:09 AM
Subject: Re: [ceph-users] performance degredation every 30 seconds
On Mon, Dec 14, 2020 at 11:28 AM Phi
=4457 IOPS][eta
00m:47s]
Jobs: 1 (f=1): [m(1)][62.8%][r=18.1MiB/s,w=18.7MiB/s][r=4640,w=4783 IOPS][eta
00m:45s]
Jobs: 1 (f=1): [m(1)][64.5%][r=7896KiB/s,w=8300KiB/s][r=1974,w=2075 IOPS][eta
00m:43s]
Jobs: 1 (f=1): [m(1)][66.1%][r=47.8MiB/s,w=47.3MiB/s][r=12.2k,w=12.1k IOPS][eta
00m:41s]
--
Philip
oops.
I posted this to the "Old" list, but supposedly this is the new list and the
better place to ask questions?
A google search didnt seem to find the answer on this, so thought I'd ask here:
what determines if an rdb is "100% busy"?
I have some backend OSDs, and an iSCSI gateway, serving
59 matches
Mail list logo
