Re: [ceph-users] Correct syntax for "mon host" line in ceph.conf?

So the problem was an issue with trying to use "master" branch of ceph-ansible, instead of a tagged branch... From: Sebastien Han [mailto:s...@redhat.com] Sent: Friday, February 01, 2019 9:40 AM To: Will Dennis Cc: ceph-ansi...@lists.ceph.com Subject: Re: [Ceph-ansibl

[ceph-users] Correct syntax for "mon host" line in ceph.conf?

I am using the "ceph-ansible" set of Ansible playbooks to try to get a test cluster up and running (in Vagrant.) I am deploying Mimic (13.2.4) on Ubuntu 16.04, with one (for now) monitor, and three osd servers. I have a play in the Ansible that is erroring out, and in troubleshooting what that

[ceph-users] ceph-ansible - where to ask questions?

Hi all, Trying to utilize the 'ceph-ansible' project (https://github.com/ceph/ceph-ansible ) to deploy some Ceph servers in a Vagrant testbed; hitting some issues with some of the plays - where is the right (best) venue to ask questions about this? Thanks, Will

Re: [ceph-users] Questions about using existing HW for PoC cluster

The hope is to be able to provide scale-out storage, that will be performant enough to use as a primary fs-based data store for research data (right now we mount via NFS on our cluster nodes, may do that with Ceph or perhaps do native cephfs access from the cluster nodes.) Right now I’m still

Re: [ceph-users] Questions about using existing HW for PoC cluster

Thanks again! Will -Original Message- From: Anthony D'Atri [mailto:a...@dreamsnake.net] Sent: Sunday, January 27, 2019 6:32 PM To: Will Dennis Cc: ceph-users Subject: Re: [ceph-users] Questions about using existing HW for PoC cluster > Been reading "Learning Ceph - Second Edition” A

Re: [ceph-users] Questions about using existing HW for PoC cluster

Been reading "Learning Ceph - Second Edition" (https://learning.oreilly.com/library/view/learning-ceph-/9781787127913/8f98bac7-44d4-45dc-b672-447d162ea604.xhtml) and in Ch. 4 I read this: "We've noted that Ceph OSDs built with the new BlueStore back end do not require journals. One might

[ceph-users] Questions about using existing HW for PoC cluster

Hi all, Kind of new to Ceph (have been using 10.2.11 on a 3-node Proxmox 4.x cluster [hyperconverged], works great!) and now I'm thinking of perhaps using it for a bigger data storage project at work, a PoC at first, but built as correctly as possible for performance and availability. I have

Re: [ceph-users] CephFS - How to handle "loaded dup inode" errors

> > errors > > > > > > > > On 07/06/2018 01:47 PM, John Spray wrote: > > > > > > On Fri, Jul 6, 2018 at 12:19 PM Wido den Hollander > > > wrote: > > > > > > > > > > > > > > > > >

Re: [ceph-users] CephFS - How to handle "loaded dup inode" errors

l journal reset cephfs-table-tool all reset session cephfs-table-tool all reset inode cephfs-table-tool all take_inos 10 I'm worried that my FS is corrupt because files are not linked correctly and have different content then they should. Please help. On Thu, 2018-07-05 at 10:35 +0200, De

Re: [ceph-users] CephFS MDS server stuck in "resolve" state

Hi, On Thu, 2018-07-05 at 09:55 +0800, Yan, Zheng wrote: > On Wed, Jul 4, 2018 at 7:02 PM Dennis Kramer (DBS) > wrote: > > > > > > Hi, > > > > I have managed to get cephfs mds online again...for a while. > > > > These topics covers more

[ceph-users] CephFS - How to handle "loaded dup inode" errors

Hi, I'm getting a bunch of "loaded dup inode" errors in the MDS logs. How can this be fixed? logs: 2018-07-05 10:20:05.591948 mds.mds05 [ERR] loaded dup inode 0x1991921 [2,head] v160 at , but inode 0x1991921.head v146 already exists at

Re: [ceph-users] CephFS MDS server stuck in "resolve" state

Wed, Jun 27, 2018 at 6:16 PM Dennis Kramer (DT) > wrote: > > > > > > Hi, > > > > Currently i'm running Ceph Luminous 12.2.5. > > > > This morning I tried running Multi MDS with: > > ceph fs set max_mds 2 > > > > I have 5 MDS serve

[ceph-users] CephFS MDS server stuck in "resolve" state

Hi, Currently i'm running Ceph Luminous 12.2.5. This morning I tried running Multi MDS with: ceph fs set max_mds 2 I have 5 MDS servers. After running above command, I had 2 active MDSs, 2 standby-active and 1 standby. And after trying a failover on one of the active MDSs, a standby-active

[ceph-users] pool recovery_priority not working as expected

tried the other way around: it did not work either) but it recovers pool_b. The setting seems not to be documented. Any ideas? Regards, Dennis ceph version 10.2.10 ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi

[ceph-users] Recovery priority

Hi, lets assume we have size=3 min_size=2 and lost some osds and now have some placement groups with only one copy left. Is there a setting to tell ceph to start recovering those pgs first in order to reach min_size and so get the cluster online faster? Regards, Dennis

Re: [ceph-users] ceph , VMWare , NFS-ganesha

Hi, we use PetaSAN for our VMWare-Cluster. It provides an webinterface for management and does clustered active-active ISCSI. For us the easy management was the point to choose this, so we need not to think about how to configure ISCSI... Regards, Dennis Am 28.05.2018 um 21:42 schrieb

[ceph-users] erasure coding chunk distribution

in advance, Dennis ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] ceph-disk removal roadmap (was ceph-disk is now deprecated)

uot;, line 416, in _populate self.append(Volume(**lv_item)) File "/usr/lib/python2.7/dist-packages/ceph_volume/api/lvm.py", line 638, in __init__ self.tags = parse_tags(kw['lv_tags']) File "/usr/lib/python2.7/dist-packages/ceph_volume/api/lvm.py", line 66, in parse_tags key, value = tag_assignment.split('=', 1) ValueError: need more than 1 value to unpack -- Dennis Lijnsveld BIT BV - http://www.bit.nl Kvk: 09090351 ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[ceph-users] how does recovery work

unavailable osds? Thanks in advance, Dennis ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Problem copying a file with ceph-fuse

Thanks for your response. But yes, the netwerk is OK. But i will double check to be sure. Then again, If I copy other (big) files from the same client everything works without any issues. The problem is isolated to a specific file. With a mis-configured network I would see this kind of issues

[ceph-users] Problem copying a file with ceph-fuse

Hi all, I have an issue that when I copy a specific file with ceph-fuse on cephfs (within the same directory) it stalls after a couple of GB of data. Nothing happens. No error, it just "hangs". When I copy the same file with the cephfs kernel client it works without issues. I'm running Jewel

Re: [ceph-users] Replacing a failed OSD

Hi Jim, I'm using a location script for OSDs, so when I add an OSD this script will determine its place in the cluster and in which bucket it belongs. In your ceph.conf there is a setting you can use: osd_crush_location_hook = With regards, On 09/14/2016 09:30 PM, Jim Kilborn wrote: > Reed, >

Re: [ceph-users] cephfs/ceph-fuse: mds0: Client XXX:XXX failingtorespond to capability release

Hi Burkhard, Thank you for your reply, see inline: On Wed, 14 Sep 2016, Burkhard Linke wrote: Hi, On 09/14/2016 12:43 PM, Dennis Kramer (DT) wrote: Hi Goncalo, Thank you. Yes, i have seen that thread, but I have no near full osds and my mds cache size is pretty high. You can use

Re: [ceph-users] cephfs/ceph-fuse: mds0: Client XXX:XXX failing to respond to capability release

Hi Goncalo, Thank you. Yes, i have seen that thread, but I have no near full osds and my mds cache size is pretty high. On Wed, 14 Sep 2016, Goncalo Borges wrote: Hi Dennis Have you checked http://lists.ceph.com/pipermail/ceph-users-ceph.com/2016-February/007207.html ? The issue

[ceph-users] cephfs/ceph-fuse: mds0: Client XXX:XXX failing to respond to capability release

Hi All, Running Ubuntu 16.04, with version JEWEL ceph version 10.2.2 (45107e21c568dd033c2f0a3107dec8f0b0e58374) In our environment we are running cephfs and our clients are connecting through ceph-fuse. Since I have upgraded from Hammer to Jewel I was haunted by the ceph-fuse segfaults, which

Re: [ceph-users] jewel blocked requests

I also have this problem. Is it perhaps possible to block clients entirely if it is not using a specific version of Ceph? BTW, I often stumble upon the cephfs problem: "client failing to respond to capability release", which result in blocked requests aswell. But i'm not entirely sure if you run

Re: [ceph-users] ceph-fuse "Transport endpoint is not connected" on Jewel 10.2.2

Hi all, I just want to confirm that the patch works in our environment. Thanks! On 08/30/2016 02:04 PM, Dennis Kramer (DBS) wrote: > Awesome Goncalo, that is very helpful. > > Cheers. > > On 08/30/2016 01:21 PM, Goncalo Borges wrote: >> Hi Dennis. >> >> That

Re: [ceph-users] ceph-fuse "Transport endpoint is not connected" on Jewel 10.2.2

Awesome Goncalo, that is very helpful. Cheers. On 08/30/2016 01:21 PM, Goncalo Borges wrote: > Hi Dennis. > > That is the first issue we saw and has nothing to do with the amd processors > (which only relates to the second issue we saw). So the fix in the patch > > https

Re: [ceph-users] ceph-fuse "Transport endpoint is not connected" on Jewel 10.2.2

server? On 08/25/2016 02:45 AM, Goncalo Borges wrote: > Hi Dennis... > > We use ceph-fuse in 10.2.2 and we saw two main issues with it immediately > after > upgrading from Infernalis to Jewel. > > In our case, we are enabling ceph-fuse in a heavily used Linux cluster, and >

Re: [ceph-users] ceph-fuse "Transport endpoint is not connected" on Jewel 10.2.2

On 08/29/2016 08:31 PM, Gregory Farnum wrote: > On Sat, Aug 27, 2016 at 3:01 AM, Francois Lafont > wrote: >> Hi, >> >> I had exactly the same error in my production ceph client node with >> Jewel 10.2.1 in my case. >> >> In the client node : >> - Ubuntu 14.04 >>

[ceph-users] ceph-fuse "Transport endpoint is not connected" on Jewel 10.2.2

Hi all, Running ceph version 10.2.2 (45107e21c568dd033c2f0a3107dec8f0b0e58374) on Ubuntu 16.04LTS. Currently I have the weirdest thing, I have a bunch of linux clients, mostly debian based (Ubuntu/Mint). They all use version 10.2.2 of ceph-fuse. I'm running cephfs since Hammer without any

Re: [ceph-users] cephfs - inconsistent nfs and samba directory listings

Hi all, Just wondering if the original issue has been resolved. I have the same problems with inconsistent nfs and samba directory listings. I'm running Hammer. Is it a confirmed seekdir bug in de kernel client? On 01/14/2016 04:05 AM, Yan, Zheng wrote: > On Thu, Jan 14, 2016 at 3:37 AM, Mike

Re: [ceph-users] Ceph noob - getting error when I try to "ceph-deploy osd activate" on a node

> On Jul 17, 2016, at 4:21 PM, Ruben Kerkhof wrote: > > Yes, that's it. You should see osd processes running, and the osd's > should be marked 'up' when you run 'ceph osd tree’. Looks like I’m good then: [wdennis@ceph2 ~]$ sudo ceph osd tree ID WEIGHT TYPE NAME

Re: [ceph-users] Ceph noob - getting error when I try to "ceph-deploy osd activate" on a node

> On Jul 17, 2016, at 7:05 AM, Ruben Kerkhof wrote: > > First, there's an issue with the version of parted in CentOS 7.2: > https://bugzilla.redhat.com/1339705 Saw this sort of thing: [ceph2][WARNIN] update_partition: Calling

Re: [ceph-users] Ceph noob - getting error when I try to "ceph-deploy osd activate" on a node

-v activate --mark-init systemd --mount /dev/sdc Still the same sort of error… no output from the blkid command used. Again, if I use either “-s PTTYPE” or get rid of the “-s TYPE” altogether, I get a value returned (“gpt”)… > On Jul 16, 2016, at 11:10 AM, Will Dennis <willard.den...@gmail.co

Re: [ceph-users] Ceph noob - getting error when I try to "ceph-deploy osd activate" on a node

owing the instructions in http://docs.ceph.com/docs/master/start/quick-ceph-deploy/ <http://docs.ceph.com/docs/master/start/quick-ceph-deploy/> from the beginning, and see if that works better. > On Jul 16, 2016, at 10:44 AM, Will Dennis <willard.den...@gmail.com> wrote: > &

Re: [ceph-users] Ceph noob - getting error when I try to "ceph-deploy osd activate" on a node

thought that using sysvinit was strange too, but I didn’t know what to expect… Wonder where that went off the rails? > On Jul 16, 2016, at 3:45 AM, Wido den Hollander <w...@42on.com> wrote: > > >> Op 15 juli 2016 om 21:19 schreef Will Dennis <willard.den...@gmail.

[ceph-users] Ceph noob - getting error when I try to "ceph-deploy osd activate" on a node

Hi all, Background: Completely new to Ceph; trying it out on three VMs I have, following install instructions found at http://docs.ceph.com/docs/master/start/quick-ceph-deploy/ I am at the point in the docs where it says to run

[ceph-users] CephFS Path restriction

-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, I'm trying to restrict clients to mount a specific path in CephFS. I've been using the official doc for this: http://docs.ceph.com/docs/master/cephfs/client-auth/ After setting these cap restrictions, the client can still mount and use all

Re: [ceph-users] CephFS Path restriction

-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Ah, that explains alot. Thank you. Yes, it was a bit confusing for which version it applied to. Awesome addition by the way, I like the path parameter! Cheers. On 12/08/2015 03:15 PM, John Spray wrote: > On Tue, Dec 8, 2015 at 1:43 PM, Den

Re: [ceph-users] State of nfs-ganesha CEPH fsal

-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Sorry for raising this topic from the dead, but i'm having the same issues with NFS-GANESHA /w the wrong user/group information. Do you maybe have a working ganesha.conf? I'm assuming I might mis-configured something in this file. It's also nice to

Re: [ceph-users] The project of ceph client file system porting from Linux to AIX

projects at hand now, so the AIX cephfs project is in pending status... It's open, anyone can make changes from this project if he want. On Thu, Mar 5, 2015 at 3:11 PM, Ketor D d.ke...@gmail.com wrote: Hi Dennis, I am interested in your project. I wrote a Win32 cephfs client https

[ceph-users] Question about notification of OSD down in client side

Hello, Is there some way to make the client(via RADOS API or something like that) to get the notification of an event (for example, an OSD down) happened in the cluster? -- Den ___ ceph-users mailing list ceph-users@lists.ceph.com

[ceph-users] Concurrent access of the object via Rados API...

Hello, For the case of multiple clients(separate process) accessing an object in the cluster, is the exclusive protection of the shared object needed for the caller? Process A::rados::ioctx.write(obj, ...) Process B::rados::ioctx.write(obj, ...) or: Process A:: mylock.lock();

[ceph-users] Question about ceph exclusive object?

Hello, What exactly does the parameter 'bool exclusive' mean in the int librados::IoCtxImpl::create(const object_t oid, bool exclusive)? I can't find any doc to describe this :-( -- Den ___ ceph-users mailing list ceph-users@lists.ceph.com

Re: [ceph-users] Status of SAMBA VFS

-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Fri, 6 Feb 2015, Gregory Farnum wrote: On Fri, Feb 6, 2015 at 7:11 AM, Dennis Kramer (DT) den...@holmes.nl wrote: On Fri, 6 Feb 2015, Gregory Farnum wrote: On Fri, Feb 6, 2015 at 6:39 AM, Dennis Kramer (DT) den...@holmes.nl wrote: I've

Re: [ceph-users] Introducing Learning Ceph : The First ever Book on Ceph

-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Awesome! Just bought the paper back copy. The sample looked very good. Thanks! Grt, On Fri, 6 Feb 2015, Karan Singh wrote: Hello Community Members I am happy to introduce the first book on Ceph with the title ?Learning Ceph?. Me and many folks

Re: [ceph-users] Cache pressure fail

On Fri, 7 Nov 2014, Gregory Farnum wrote: Did you upgrade your clients along with the MDS? This warning indicates the MDS asked the clients to boot some inboxes out of cache and they have taken too long to do so. It might also just mean that you're actively using more inodes at any given time

Re: [ceph-users] Cache pressure fail

On Wed, 11 Feb 2015, Wido den Hollander wrote: On 11-02-15 12:57, Dennis Kramer (DT) wrote: On Fri, 7 Nov 2014, Gregory Farnum wrote: Did you upgrade your clients along with the MDS? This warning indicates the MDS asked the clients to boot some inboxes out of cache and they have taken too

[ceph-users] Status of SAMBA VFS

Hi, Is the Samba VFS module for CephFS actively maintained at this moment? I haven't seen much updates in the ceph/samba git repo. With regards, ___ ceph-users mailing list ceph-users@lists.ceph.com

Re: [ceph-users] Status of SAMBA VFS

this module instead of an re-export with the kernel client. So I hope the VFS module will be maintained actively again any time soon. On Fri, 6 Feb 2015, Sage Weil wrote: On Fri, 6 Feb 2015, Dennis Kramer (DT) wrote: Hi, Is the Samba VFS module for CephFS actively maintained at this moment? I haven't

Re: [ceph-users] Status of SAMBA VFS

On Fri, 6 Feb 2015, Gregory Farnum wrote: On Fri, Feb 6, 2015 at 6:39 AM, Dennis Kramer (DT) den...@holmes.nl wrote: I've used the upstream module for our production cephfs cluster, but i've noticed a bug where timestamps aren't being updated correctly. Modified files are being reset

[ceph-users] How to notify an object watched by client via ceph class API

Hello, Just as the subject indicated, how can I notify an object watcher within a class '.so' loaded into osd deamon. I checked the code and found the below op code defined already:CEPH_OSD_OP_NOTIFY, but this op code is not used by existing ceph class api, I guest there must be a simple solution

Re: [ceph-users] How to notify an object watched by client via ceph class API

On Thu, Feb 5, 2015 at 4:51 PM, Sage Weil s...@newdream.net wrote: On Thu, 5 Feb 2015, Dennis Chen wrote: Hello, Just as the subject indicated, how can I notify an object watcher within a class '.so' loaded into osd deamon. I checked the code and found the below op code defined

Re: [ceph-users] How to notify an object watched by client via ceph class API

On Thu, Feb 5, 2015 at 6:26 PM, Sage Weil s...@newdream.net wrote: On Thu, 5 Feb 2015, Dennis Chen wrote: On Thu, Feb 5, 2015 at 4:51 PM, Sage Weil s...@newdream.net wrote: On Thu, 5 Feb 2015, Dennis Chen wrote: Hello, Just as the subject indicated, how can I notify an object watcher

[ceph-users] Question about output message and object update for ceph class

Hello, I write a ceph client using rados lib to execute a funcution upon the object. CLIENT SIDE CODE === int main() { ... strcpy(in, from client); err = rados_exec(io, objname, devctl, devctl_op, in, strlen(in), out, 128); if (err 0) { fprintf(stderr,

Re: [ceph-users] Question about output message and object update for ceph class

I take back the question, because I just found that for a succeed write opetion in the class, *no* data in the out buffer... On Wed, Feb 4, 2015 at 5:44 PM, Dennis Chen kernel.org@gmail.com wrote: Hello, I write a ceph client using rados lib to execute a funcution upon the object. CLIENT

Re: [ceph-users] Question about primary OSD of a pool

Hello Sudarshan, Thanks, it should be useful when I want to appoint the specific OSD as primary ;-) On Mon, Feb 2, 2015 at 3:50 PM, Sudarshan Pathak sushan@gmail.com wrote: Hello Dennis, You can create CRUSH rule to select one of osd as primary as: rule ssd-primary

[ceph-users] Question about primary OSD of a pool

Hello, If I write 2 different objects, eg, john and paul respectively to a same pool like testpool in the cluster, is the primary OSD calculated by CRUSH for the 2 objects the same? -- Den ___ ceph-users mailing list ceph-users@lists.ceph.com

Re: [ceph-users] Question about primary OSD of a pool

On Mon, Feb 2, 2015 at 12:04 AM, Loic Dachary l...@dachary.org wrote: On 01/02/2015 14:47, Dennis Chen wrote: Hello, If I write 2 different objects, eg, john and paul respectively to a same pool like testpool in the cluster, is the primary OSD calculated by CRUSH for the 2 objects the same

Re: [ceph-users] Question about primary OSD of a pool

/man/8/ceph for the ceph command. Cheers JC While moving. Excuse unintended typos. On Feb 1, 2015, at 08:04, Loic Dachary l...@dachary.org wrote: On 01/02/2015 14:47, Dennis Chen wrote: Hello, If I write 2 different objects, eg, john and paul respectively to a same pool like testpool

Re: [ceph-users] Question about primary OSD of a pool

Hi Sudarshan, Some hints for doing that ? On Mon, Feb 2, 2015 at 1:03 PM, Sudarshan Pathak sushan@gmail.com wrote: BTW, you can make crush to always choose the same OSD as primary. Regards, Sudarshan On Mon, Feb 2, 2015 at 9:26 AM, Dennis Chen kernel.org@gmail.com wrote: Thanks

[ceph-users] Question about ceph class usage

Hello, I found the document of ceph class usage is very few, below is the only one which can almost address my needs-- http://ceph.com/rados/dynamic-object-interfaces-with-lua/ But still some questions confusing me left there: 1. How to make the OSD to load the class lib? or what's the process

Re: [ceph-users] Problems during first install

of 0 you know this is your problem and you can fix it by adjusting the weight to something more reasonable. Regards, Dennis ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[ceph-users] PGs inconsistency, deep-scrub / repair won't fix (v0.80.1)

Hi all, A couple of weeks ago i've upgraded from emperor to firefly. I'm using Cloudstack /w CEPH as the storage backend for VMs and templates. Since the upgrade, ceph is in a HEALTH_ERR with 500+ pgs inconsistent and 2000+ scrub errors. Not sure if it has the do with firefly though, but the

[ceph-users] About Ceph Performance Counters

much information from web site except(http://ceph.com/docs/next/dev/perf_counters/), did you have more about this? Regards, Dennis ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[ceph-users] ceph and collectd

when collectd tried to get version from ceph side via *.asok, system call--poll seemed to timed out. anyone can help me ? Thank you! Regards, Dennis ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users

[ceph-users] rbd command error librbd::ImageCtx: error finding header

can be help about this? BRs, Dennis ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] rbd command error librbd::ImageCtx: error finding header

-img with this command: root@~# rbd create odm-kvm-img --size 10240 --pool mypool BRs, Dennis On 04/24/2013 11:28 AM, Chris Hoy Poy wrote: Hi, try # rbd -p mypool --image odm-kvm-img info cheers //Chris - Original Message - From: Dennis Chen xsc...@tnsoft.com.cn To: ceph-us...@ceph.com

Re: [ceph-users] rbd command error librbd::ImageCtx: error finding header

specify the -p mypool will create another error msg looks like: root@~# rbd --image odm-kvm-img -p mypool info 2013-04-24 10:44:29.442715 7f8d9134b780 -1 rbd: error opening image librbd: Error listing snapshots: (95) Operation not supportedodm-kvm-img : (95) Operation not supported BRs, Dennis