> Op 12 juli 2016 om 8:47 schreef Christian Balzer :
>
>
>
> Hello,
>
> On Tue, 12 Jul 2016 08:39:16 +0200 (CEST) Wido den Hollander wrote:
>
> > Hi,
> >
> > I am upgrading a 1800 OSD cluster from Hammer 0.94.5 to 0.94.7 prior to
> > going to Jewel and while doing so I see the monitors bein
Hello,
On Tue, 12 Jul 2016 08:39:16 +0200 (CEST) Wido den Hollander wrote:
> Hi,
>
> I am upgrading a 1800 OSD cluster from Hammer 0.94.5 to 0.94.7 prior to going
> to Jewel and while doing so I see the monitors being flooded with these
> messages:
>
Google is your friend (and so is the NSA):
Hi,
I am upgrading a 1800 OSD cluster from Hammer 0.94.5 to 0.94.7 prior to going
to Jewel and while doing so I see the monitors being flooded with these
messages:
2016-07-12 08:28:12.919748 osd.1200 [WRN] failed to encode map e130549 with
expected crc
2016-07-12 08:28:12.921943 osd.1338 [WRN]
Hello,
On Tue, 12 Jul 2016 03:43:41 + Robin Percy wrote:
> First off, thanks for the great response David.
>
Yes, that was a very good writeup.
> If I understand correctly, you're saying there are two distinct costs to
> consider: peering, and backfilling. The backfilling cost is a functio
and this- after starting the osd manually
root@cephosd01:~# df
Filesystem 1K-blocksUsed Available Use% Mounted on
/dev/dm-0 15616412 1583180 13216900 11% /
udev 10240 0 10240 0% /dev
tmpfs 496564636 45020 10% /run
tmpfs
Hi,
while rebalancing a drive experienced read errors so I think leveldb was
corrupted. Unfortunately there's currently no second copy which is
up2date so I can forget this pg. Only one pg is affected (I moved all
other pg's away as they had active copies on another osd.
In "daily business" this
Hi Oliver,
On Friday, July 8, 2016, Oliver Dzombic wrote:
> Hi,
>
> does anyone have experience how to connect vmware with ceph smart ?
>
> iSCSI multipath does not really worked well.
> NFS could be, but i think thats just too much layers in between to have
> some useable performance.
>
> Syste
You are welcome. But please don't send test message to a public list. :)
2016-07-12 11:07 GMT+08:00 xiongnuwang :
> I have joined。
>
> ___
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
>
On Tue, Jul 12, 2016 at 1:07 AM, Gregory Farnum wrote:
> Oh, is this one of your custom-built packages? Are they using
> tcmalloc? That difference between VSZ and RSS looks like a glibc
> malloc problem.
> -Greg
>
ceph-fuse at http://download.ceph.com/rpm-jewel/el7/x86_64/ is not
linked to libtcm
First off, thanks for the great response David.
If I understand correctly, you're saying there are two distinct costs to
consider: peering, and backfilling. The backfilling cost is a function of
the amount of data in our pool, and therefore won't benefit from
incremental steps. But the peering cos
I have joined。___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Hi All
I use the Ceph stable version v10.2.2. When I begin to compile the source
code, I use the make && make install. I am sure that the command make
builds successfully. But the command make install would always appear the
same issue for Installing
/usr/local/lib/python2.7/dist-packages/ceph_det
Hello,
On Mon, 11 Jul 2016 16:19:58 +0200 Mateusz Skała wrote:
> Hello Cephers.
>
> Can someone help me in my cache tier configuration? I have 4 same SSD drives
> 176GB (184196208K) in SSD pool, how to determine target_max_bytes?
What exact SSD models are these?
What version of Ceph?
> I ass
Hello,
In this context my first question would also be, how does one wind up with
such a lock contention in the first place?
And how to safely resolve this?
Both of which are not Ceph problems, but those of the client stack being
used or of knowledgeable, 24/7 monitoring and management.
Net-spl
On Mon, Jul 11, 2016 at 04:53:36PM +0200, Lionel Bouton wrote:
> Le 11/07/2016 11:56, Brad Hubbard a écrit :
> > On Mon, Jul 11, 2016 at 7:18 PM, Lionel Bouton
> > wrote:
> >> Le 11/07/2016 04:48, 한승진 a écrit :
> >>> Hi cephers.
> >>>
> >>> I need your help for some issues.
> >>>
> >>> The ceph cl
Hello,
scrub settings will only apply to new scrubs, not running ones, as you
found out.
On Mon, 11 Jul 2016 15:37:49 +0300 K K wrote:
>
> I have tested windows instance Crystal Disk Mark. Result is:
>
Again, when running a test like this, check with atop/iostat how your
OSDs/HDDs are doing
Can anyone explain or at least refer to the lines of the codes in librd by
which objects are created? I need to know the relation between objects and
fio's iodepth...
Thanks in advance
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ce
root@cephosd01:~# fdisk -l /dev/sdb
Disk /dev/sdb: 50 GiB, 53687091200 bytes, 104857600 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 87B152E0-EB5D-4EB0-8FF
When you increase your PGs you're already going to be moving around all of your
data. Doing a full doubling of your PGs from 64 -> 128 -> 256 -> ... -> 2048
over and over and letting it backfill to healthy every time is a lot of extra
data movement that isn't needed.
I would recommend setting
Unfortunately that is correct -- the exclusive lock automatically
transitions upon request in order to handle QEMU live migration. There
is some on-going work to deeply integrate locking support into QEMU
which would solve this live migration case and librbd could internally
disable automatic lock
Hello,
I'm looking for some advice on how to most safely increase the pgs in our
primary ceph pool.
A bit of background: We're running ceph 0.80.9 and have a cluster of 126
OSDs with only 64 pgs allocated to the pool. As a result, 2 OSDs are now
88% full, while the pool is only showing as 6% used
I'm using this setup with ESXi 5.1 and I get very good performance. I
suspect you have other issues. Reliability is another story (see Nick's
posts on tgt and HA to get an idea of the awful problems you can have), but
for my test labs the risk is acceptable.
One change I found helpful is to run
On Mon, Jul 11, 2016 at 11:15 AM, Bob R wrote:
> George,
>
> Check the instructions here which should allow you to test your crush rules
> without applying them to your cluster.
> http://dachary.org/?p=3189
>
> also, fwiw, we are not using an 'emit' after each choose (note these rules
> are not im
George,
Check the instructions here which should allow you to test your crush rules
without applying them to your cluster.
http://dachary.org/?p=3189
also, fwiw, we are not using an 'emit' after each choose (note these rules
are not implementing what you're trying to)-
# rules
rule replicated_rul
Oh, is this one of your custom-built packages? Are they using
tcmalloc? That difference between VSZ and RSS looks like a glibc
malloc problem.
-Greg
On Mon, Jul 11, 2016 at 12:04 AM, Goncalo Borges
wrote:
> Hi John...
>
> Thank you for replying.
>
> Here is the result of the tests you asked but I
I'm not looking at the docs, but I think you need an "emit" statement after
every choose.
-Greg
On Monday, July 11, 2016, George Shuklin wrote:
> Hello.
>
> I want to try CRUSH rule with following idea:
> take one OSD from root with SSD drives (and use it as primary).
> take two OSD from root wi
Hi Mike,
i was trying:
https://ceph.com/dev-notes/adding-support-for-rbd-to-stgt/
ONE target, from different OSD servers directly, to multiple vmware esxi
servers.
A config looked like:
#cat iqn.ceph-cluster_netzlaboranten-storage.conf
driver iscsi
bs-type rbd
backing-store rbd/vmware-storag
Mount OSD data to temp. directory (e.g. /mnt) and check where journal
pointing (ls -la /mnt/journal). It can lead to a different location from
your "--osd-journal=" instruction to --mkjournal.
On 07/11/2016 05:46 PM, William Josefsson wrote:
Hi All,
Initially, I used /dev/disk/by-partuuid/xxx
Hello.
I want to try CRUSH rule with following idea:
take one OSD from root with SSD drives (and use it as primary).
take two OSD from root with HDD drives.
I've created this rule:
rule rule_mix {
ruleset 2
type replicated
min_size 2
max_size 10
step take
Thanks!
I need to create a VM having qcow2 image file as 6.7 GB but raw image as
600GB which is too big.
Is there a way that i need not to convert qcow2 file to raw and it works
well with rbd?
Regards
Gaurav Goyal
On Mon, Jul 11, 2016 at 11:46 AM, Kees Meijs wrote:
> Glad to hear it works now
Check out partition type for data partition for ceph.
fdisk -l /dev/sdc
On 07/11/2016 04:03 PM, Dirk Laurenz wrote:
hmm, helps partially ... running
/usr/sbin/ceph-disk trigger /dev/sdc1 or sdb1 works and brings osd up..
systemctl enable does not help
Am 11.07.2016 um 14:49 schrieb G
Glad to hear it works now! Good luck with your setup.
Regards,
Kees
On 11-07-16 17:29, Gaurav Goyal wrote:
> Hello it worked for me after removing the following parameter from
> /etc/nova/nova.conf file
___
ceph-users mailing list
ceph-users@lists.ceph
On 07/08/2016 02:22 PM, Oliver Dzombic wrote:
> Hi,
>
> does anyone have experience how to connect vmware with ceph smart ?
>
> iSCSI multipath does not really worked well.
Are you trying to export rbd images from multiple iscsi targets at the
same time or just one target?
For the HA/multiple t
Hi Goncalo,
On Fri, Jul 8, 2016 at 3:01 AM, Goncalo Borges
wrote:
> 5./ I have noticed that ceph-fuse (in 10.2.2) consumes about 1.5 GB of
> virtual memory when there is no applications using the filesystem.
>
> 7152 root 20 0 1108m 12m 5496 S 0.0 0.0 0:00.04 ceph-fuse
>
> When I onl
Hello it worked for me after removing the following parameter from
/etc/nova/nova.conf file
[root@OSKVM1 ~]# cat /etc/nova/nova.conf|grep hw_disk_discard
#hw_disk_discard=unmap
Though as per ceph documentation, for KILO version we must set this
parameter. I am using Liberty but i am not sure if
Situation is --> I have installed openstack setup (Liberty) for my lab.
Dear Ceph users,
I need your suggestion for my ceph design.
I have
Host 1 --> Controller + Compute1
Host 2 --> Compute 2
DELL SAN storage is attached to both hosts as
[root@OSKVM1 ~]# iscsiadm -m node
10.35.0.3:3260,1
iq
Le 11/07/2016 11:56, Brad Hubbard a écrit :
> On Mon, Jul 11, 2016 at 7:18 PM, Lionel Bouton
> wrote:
>> Le 11/07/2016 04:48, 한승진 a écrit :
>>> Hi cephers.
>>>
>>> I need your help for some issues.
>>>
>>> The ceph cluster version is Jewel(10.2.1), and the filesytem is btrfs.
>>>
>>> I run 1 Mon a
Hi All,
Initially, I used /dev/disk/by-partuuid/xxx-- when make the
journal for the OSD, i.e.
sudo ceph-osd -i 30 --mkjournal
--osd-journal=/dev/disk/by-partuuid/2fe31ba2-1ac6-4729-9fdc-63432f50357
Then, I try to use /dev/sdx5 format and it works, i.e.
sudo ceph-osd -i 30 --mkjournal
Hello Cephers.
Can someone help me in my cache tier configuration? I have 4 same SSD drives
176GB (184196208K) in SSD pool, how to determine target_max_bytes? I assume
that should be (4 drives* 188616916992 bytes )/ 3 replica = 251489222656
bytes *85% (because of full disk warning)
It will be 213
Hi Everyone,
I have a problem with OSD stuck in booting state.
sudo ceph daemon osd.7 status
{
"cluster_fsid": "724e501f-f4a3-4731-a832-c73685aabd21",
"osd_fsid": "058cac6e-6c66-4eeb-865b-3d22f0e91a99",
"whoami": 7,
"state": "booting",
"oldest_map": 1255,
"newest_map": 249
hmm, helps partially ... running
/usr/sbin/ceph-disk trigger /dev/sdc1 or sdb1 works and brings osd up..
systemctl enable does not help
Am 11.07.2016 um 14:49 schrieb George Shuklin:
Short story how OSDs are started in systemd environments:
Ceph OSD parittions has specific typecode (pa
> -Original Message-
> From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of
> Daniel Gryniewicz
> Sent: 11 July 2016 13:38
> To: ceph-users@lists.ceph.com
> Subject: Re: [ceph-users] OSPF to the host
>
> On 07/11/2016 08:23 AM, Saverio Proto wrote:
> >> I'm looking at
On 07/11/2016 09:57 AM, kefu chai wrote:
Hi Cephers,
I am proposing drop the support of i386. as we don't compile Ceph with
any i386 gitbuilder now[1] and hence don't test the i386 builds on
sepia on a regular basis. Also, based on the assumption that people
don't use i386 in production, I think
Short story how OSDs are started in systemd environments:
Ceph OSD parittions has specific typecode (partition type
4FBD7E29-9D25-41B8-AFD0-062C0CEFF05D). It handled by udev rules shipped
by ceph package:
/lib/udev/rules.d/95-ceph-osd.rules
It set up proper owner/group for this disk ('ceph' i
Hi,
what i do to reproduce the failure:
root@cephadmin:~# ceph osd tree
ID WEIGHT TYPE NAME UP/DOWN REWEIGHT PRIMARY-AFFINITY
-1 0.26340 root default
-2 0.08780 host cephosd01
0 0.04390 osd.0 up 1.0 1.0
1 0.04390 osd.1 up 1.0
On 07/11/2016 08:23 AM, Saverio Proto wrote:
I'm looking at the Dell S-ON switches which we can get in a Cumulus
version. Any pro's and con's of using Cumulus vs old school switch OS's you
may have come across?
Nothing to declare here. Once configured properly the hardware works
as expected
I have tested windows instance Crystal Disk Mark. Result is:
Sequential Read : 43.049 MB/s
Sequential Write : 45.181 MB/s
Random Read 512KB : 78.660 MB/s
Random Write 512KB : 39.292 MB/s
Random Read 4KB (QD=1) : 3.511 MB/s [ 857.3 IOPS]
Random Write 4KB (QD=1) : 1.380 MB/s [ 337.0 IOPS]
Random Re
Hi Dirk,
without any information, its impossible to tell you anything.
Please provide us some detail information about what is going wrong,
including error messages and so on.
As an admin you should be enough familar with your system to give us
more information but just "its not working". As you
Hello,
i'm new to ceph an try to do some first steps with ceph to understand
concepts.
my setup is at first completly in vm
i deployed (with ceph-deploy) three monitors and three osd hosts. (3+3 vms)
my frist test was to find out, if everything comes back online after a
system restart
> I'm looking at the Dell S-ON switches which we can get in a Cumulus
> version. Any pro's and con's of using Cumulus vs old school switch OS's you
> may have come across?
Nothing to declare here. Once configured properly the hardware works
as expected. I never used Dell, I used switches from
Merge happens either due to movement of objects due to CRUSH recalculation
(when cluster grows or shrinks due to various reasons) or deletion of
objects.
Split happens when portions of objects/volumes get populated that were
previously sparse. Each RADOS object by default is 4MB chunk and volumes
Thank you for the confirmation, John!
As we have both CIFS&NFS users, I was wishing the feature should be implemented
at the CephFS layer :<
Regards,
---Sandy
> -Original Message-
> From: John Spray [mailto:jsp...@redhat.com]
> Sent: Monday, July 11, 2016 7:28 PM
> To: xusangdi 11976 (RD
I believe splitting will happen on writes, merging I think only happens on
deletions.
From: Paul Renner [mailto:renner...@gmail.com]
Sent: 10 July 2016 19:40
To: n...@fisk.me.uk
Cc: ceph-users@lists.ceph.com
Subject: Re: [ceph-users] Filestore merge and split
Thanks...
Do you know when s
On Mon, Jul 11, 2016 at 8:04 AM, Goncalo Borges
wrote:
> Hi John...
>
> Thank you for replying.
>
> Here is the result of the tests you asked but I do not see nothing abnormal.
Thanks for running through that. Yes, nothing in the output struck me
as unreasonable either :-/
> Actually, your sugg
On Mon, Jul 11, 2016 at 9:28 AM, Xusangdi wrote:
> Hi Cephers,
>
>
>
> I’m planning to set up samba/nfs based on CephFS kernel mount. The
> WORM(write once read many) feature is required but I’m not
>
> sure if CephFS officially supports it, any suggestions? Thanks in advance.
There's nothing in
Hello everyone,
Last week, while deploying new disks in our cluster, we bump into what
we believe is a kernel bug. Now everything is working fine, though we
wanted to share our experience and see if other people have experienced
similar behaviour.
Steps we followed were:
1) First we removed DNE
Hello again
Any thoughts on this issue?
Cheers
Andrei
> From: "Andrei Mikhailovsky"
> To: "ceph-users"
> Sent: Wednesday, 22 June, 2016 18:02:28
> Subject: [ceph-users] Error EPERM when running ceph tell command
> Hi
> I am trying to run an osd level benchmark but get the following erro
On Mon, Jul 11, 2016 at 7:18 PM, Lionel Bouton
wrote:
> Le 11/07/2016 04:48, 한승진 a écrit :
>> Hi cephers.
>>
>> I need your help for some issues.
>>
>> The ceph cluster version is Jewel(10.2.1), and the filesytem is btrfs.
>>
>> I run 1 Mon and 48 OSD in 4 Nodes(each node has 12 OSDs).
>>
>> I've
I have change those params to all OSD and restart its:
osd_scrub_end_hour = 6
osd_scrub_load_threshold = 2.5
osd_scrub_sleep = 0.1
but ceph status still show deep scrub:
ceph status
cluster 830beb43-9898-4fa9-98c1-ee04c1cdf69c
health HEALTH_OK
monmap e6: 3 mons at
{block01=10.30.9.21:6789/0,obj
Le 11/07/2016 04:48, 한승진 a écrit :
> Hi cephers.
>
> I need your help for some issues.
>
> The ceph cluster version is Jewel(10.2.1), and the filesytem is btrfs.
>
> I run 1 Mon and 48 OSD in 4 Nodes(each node has 12 OSDs).
>
> I've experienced one of OSDs was killed himself.
>
> Always it issued s
Hi Manual,
This is a well known issue. You are definitely not the first one to hit this
problem. Before Jewel i (and other as well) added the line
ceph-disk activate all
to /etc/rc.local to get the OSD’s running at boot. In Jewel, however, this
doesn’t work anymore. Now i add these line to /et
Additional.
OSD params aplying to scrub:
"osd_scrub_invalid_stats": "true",
"osd_scrub_begin_hour": "0",
"osd_scrub_end_hour": "24",
"osd_scrub_load_threshold": "0.5",
"osd_scrub_min_interval": "86400",
"osd_scrub_max_interval": "604800",
"osd_scrub_interval_randomize_ratio": "0.5",
"osd_scrub_chu
> Still don't know your Ceph version, is it the latest Jewel?
10.1.2-0ubuntu1
> Check the ML archives, I remember people having performance issues with the
4.4 kernels.
Yes, I try today to find something
> These are OSD parameters, you need to query an OSD daemon.
There are:
ceph --admin-daemon
Hi,
I think there's still something misconfigured:
> Invalid: 400 Bad Request: Unknown scheme 'file' found in URI (HTTP 400)
It seems the RBD backend is not used as expected.
Have you configured both Cinder _and_ Glance to use Ceph?
Regards,
Kees
On 08-07-16 17:33, Gaurav Goyal wrote:
>
> I re
Hi Cephers,
I’m planning to set up samba/nfs based on CephFS kernel mount. The WORM(write
once read many) feature is required but I’m not
sure if CephFS officially supports it, any suggestions? Thanks in advance.
Regards,
---Sandy
Hello,
On Mon, 11 Jul 2016 09:54:59 +0300 K K wrote:
>
> > I hope the fastest of these MONs (CPU and storage) has the lowest IP
> > number and thus is the leader.
> no, the lowest IP has slowest CPU. But zabbix didn't show any load at all
> mons.
In your use case and configuration no surprise
On 07/11/2016 05:04 PM, Goncalo Borges wrote:
Hi John...
Thank you for replying.
Here is the result of the tests you asked but I do not see nothing
abnormal. Actually, your suggestions made me see that:
1) ceph-fuse 9.2.0 is presenting the same behaviour but with less
memory consumption,
Hi John...
Thank you for replying.
Here is the result of the tests you asked but I do not see nothing
abnormal. Actually, your suggestions made me see that:
1) ceph-fuse 9.2.0 is presenting the same behaviour but with less memory
consumption, probably, less enought so that it doesn't brake c
68 matches
Mail list logo
