Re: [ceph-users] CephFS in the wild

On Wed, Jun 1, 2016 at 1:50 PM, Brady Deetz wrote: > Question: > I'm curious if there is anybody else out there running CephFS at the scale > I'm planning for. I'd like to know some of the issues you didn't expect that > I should be looking out for. I'd also like to simply see

Re: [ceph-users] ceph-fuse performance about hammer and jewel

Yan, Zheng: Thanks for your reply. But change into jewel, application read/write disk slowly. confirms the fio tested iops. Does there any other possibles? 在 16/6/1 21:39, Yan, Zheng 写道: On Wed, Jun 1, 2016 at 6:52 PM, qisy wrote: my test fio fio

Re: [ceph-users] CephFS: slow writes over NFS when fs is mounted with kernel driver but fast with Fuse

On Fri, Jun 3, 2016 at 10:43 PM, Jan Schermer wrote: > I'd be worried about it getting "fast" all of sudden. Test crash > consistency. > If you test something like file creation you should be able to estimate if > it should be that fast. (So it should be some fraction of

Re: [ceph-users] Jewel upgrade - rbd errors after upgrade

Thanks Jason. I don’t have anything specified explicitly for osd class dir. I suspect it might be related to the OSDs being restarted during the package upgrade process before all libraries are upgraded. > -Original Message- > From: Jason Dillaman [mailto:jdill...@redhat.com] >

Re: [ceph-users] Jewel upgrade - rbd errors after upgrade

I couldn't find anything wrong with the packages and everything seemed installed ok. Once I restarted the OSDs the directory issue went away but the error started moving to other rbd output, and the same class open error occurred on other OSDs. I have gone through and bounced all the OSDs

Re: [ceph-users] Jewel upgrade - rbd errors after upgrade

Odd -- sounds like you might have Jewel and Infernalis class objects and OSDs intermixed. I would double-check your installation and see if your configuration has any overload for "osd class dir". On Sun, Jun 5, 2016 at 10:28 PM, Adrian Saul wrote: > > I have

Re: [ceph-users] Jewel upgrade - rbd errors after upgrade

I have traced it back to an OSD giving this error: 2016-06-06 12:18:14.315573 7fd714679700 -1 osd.20 23623 class rbd open got (5) Input/output error 2016-06-06 12:19:49.835227 7fd714679700 0 _load_class could not open class /usr/lib64/rados-classes/libcls_rbd.so (dlopen failed):

Re: [ceph-users] Jewel upgrade - rbd errors after upgrade

The rbd_directory object is empty -- all data is stored as omap key/value pairs which you can list via "rados listomapvals rbd_directory". What is the output when you run "rbd ls --debug-ms=1 glebe-sata" and "rbd info --debug-ms=1 glebe-sata/hypervtst-lun04"? I am interested in the lines that

Re: [ceph-users] Jewel upgrade - rbd errors after upgrade

Seems like my rbd_directory is empty for some reason: [root@ceph-glb-fec-02 ceph]# rados get -p glebe-sata rbd_directory /tmp/dir [root@ceph-glb-fec-02 ceph]# strings /tmp/dir [root@ceph-glb-fec-02 ceph]# ls -la /tmp/dir -rw-r--r--. 1 root root 0 Jun 6 11:12 /tmp/dir [root@ceph-glb-fec-02

Re: [ceph-users] Jewel upgrade - rbd errors after upgrade

No - it throws a usage error - if I add a file argument after it works: [root@ceph-glb-fec-02 ceph]# rados -p glebe-sata get rbd_id.hypervtst-lun04 /tmp/crap [root@ceph-glb-fec-02 ceph]# cat /tmp/crap 109eb01f5f89de stat works: [root@ceph-glb-fec-02 ceph]# rados -p glebe-sata stat

Re: [ceph-users] Jewel upgrade - rbd errors after upgrade

Are you able to successfully run the following command successfully? rados -p glebe-sata get rbd_id.hypervtst-lun04 On Sun, Jun 5, 2016 at 8:49 PM, Adrian Saul wrote: > > I upgraded my Infernalis semi-production cluster to Jewel on Friday. While > the upgrade

[ceph-users] Jewel upgrade - rbd errors after upgrade

I upgraded my Infernalis semi-production cluster to Jewel on Friday. While the upgrade went through smoothly (aside from a time wasting restorecon /var/lib/ceph in the selinux package upgrade) and the services continued running without interruption. However this morning when I went to create

Re: [ceph-users] Best upgrade strategy

If your monitor nodes are separate from the osd nodes, I'd get ceph upgraded to the latest point release of your current line (0.94.7). Upgrade monitors, then osds, then other dependent services (mds, rgw, qemu). Once everything is happy again, I'd run OS and ceph upgrades together, starting with

[ceph-users] Best upgrade strategy

Hi, we are running a cluster with 6 storage nodes(72 osds) and 3 monitors. The osds and and monitors are running on Ubuntu 14.04 and with ceph 0.94.5. We want to upgrade the cluster to Jewel and at the same time the OS to Ubuntu 16.04. What would be the best way to this? First to upgrade the OS

[ceph-users] Disaster recovery and backups

Let's assume that everything went very very bad and i have to manually recover a cluster with an unconfigured ceph. 1. How can i recover datas directly from raw disks? Is this possible? 2. How can i restore a ceph cluster (and have data back) by using existing disks? 3. How do you manage backups

Re: [ceph-users] rados complexity

Ok, seems like my problem could be cephfs-related. I have 16 cephfs clients that do heavy, sub-optimal writes simultaneously. The cluster have no problems handling the load up until circa 2 kobjects.  Above this threshold the OSDs start to go down randomly and eventually get killed by the

[ceph-users] no osds in jewel

Hi, When I run below script to install Ceph (10.2.0), I met an error "no osds". Hammer was installed by the script. So I think I miss new thing, which was released since Hammer. Do you know what I miss? --- The script --- #!/bin/sh set -x ceph-deploy new csElsa echo "osd pool default size =

Re: [ceph-users] rados complexity

We've got a simple cluster having 45 OSDs, have above 5 kobjects and did not have any issues so far. Our cluster does mainly serve some rados pools for an application which usually writes data once and reads it multiple times. - Sven Am Sonntag, den 05.06.2016, 18:47 +0200 schrieb Mykola

[ceph-users] rados complexity

Are there any ceph users with pools containing >2 kobjects? If so, have you noticed any instabilities of the clusters once this threshold is reached? -Mykola___ ceph-users mailing list ceph-users@lists.ceph.com

[ceph-users] RGW AWS4 SignatureDoesNotMatch when requests with port != 80 or != 443

Hi! I get the error " SignatureDoesNotMatch" when I used presigned url with endpoint port != 80 and != 443. For example, if I use host http://192.168.1.1: then this is what I have in RGW log: // RGWEnv::set(): HTTP_HOST: 192.168.1.1: // RGWEnv::set(): SERVER_PORT: //

Re: [ceph-users] 403 AccessDenied with presigned url in Jewel AWS4.

Thank Robin H. Johnson! I've set "debug rgw = 20" in RGW config file and I have seen "NOTICE: now = 1464998270, now_req = 1464973070, exp = 3600" in RGW log file. I see that now is the local time on the RGW server (my timezone is UTC + 7) and now_req is UTC time. This leads to one error in src/

Re: [ceph-users] Older Ceph packages for Ubuntu 12.04 (Precise Pangolin) to recompile libvirt with RBD support

Hi, Anyone can assist on this? Looking forward to your reply, thank you. Cheers. On Fri, Jun 3, 2016 at 11:56 AM, Cloud List wrote: > Dear all, > > I am trying to setup older version of CloudStack 4.2.0 on Ubuntu 12.04 to > use Ceph RBD as primary storage for our