Re: [ceph-users] corrupted rbd filesystems since jewel

2017-05-04 Thread Jason Dillaman
Odd. Can you re-run "rbd rm" with "--debug-rbd=20" added to the command and post the resulting log to a new ticket at [1]? I'd also be interested if you could re-create that "librbd::object_map::InvalidateRequest" issue repeatably. [1] http://tracker.ceph.com/projects/rbd/issues On Thu, May 4, 20

Re: [ceph-users] corrupted rbd filesystems since jewel

2017-05-08 Thread Jason Dillaman
t upgraded to jewel AND got enabled exclusive lock. >> >> Greets, >> Stefan >> >> Am 04.05.2017 um 14:20 schrieb Jason Dillaman: >>> Odd. Can you re-run "rbd rm" with "--debug-rbd=20" added to the >>> command and post the result

Re: [ceph-users] Ceph memory overhead when used with KVM

2017-05-08 Thread Jason Dillaman
ad > """ > > Then let it run: > $> while true; do fio stress.fio; rm /root/test.dat; done > > I had this running over a weekend. > > Cheers > Sebastian > > On Tuesday, May 02, 2017 02:51:06 PM Jason Dillaman wrote: >> Can you share the fio job

Re: [ceph-users] Read from Replica Osds?

2017-05-08 Thread Jason Dillaman
librbd can optionally read from replicas for snapshots and parent images (i.e. known read-only data). This is controlled via the following configuration options: rbd_balance_snap_reads rbd_localize_snap_reads rbd_balance_parent_reads rbd_localize_parent_reads Direct users of the librados API can

Re: [ceph-users] corrupted rbd filesystems since jewel

2017-05-11 Thread Jason Dillaman
ew crashes. > > What still makes me wonder are those > librbd::object_map::InvalidateRequest: 0x7f7860004410 should_complete: r=0 > > messages. > > Greets, > Stefan > > Am 08.05.2017 um 14:50 schrieb Stefan Priebe - Profihost AG: >> Hi, >> Am 08.05.2017 um 14

Re: [ceph-users] corrupted rbd filesystems since jewel

2017-05-14 Thread Jason Dillaman
elated to fstrim and discard. I cannot reproduce it for >> images were we don't use trim - but it's still the case it's working >> fine for images created with jewel and it is not for images pre jewel. >> The only difference i can find is that the images cre

Re: [ceph-users] corrupted rbd filesystems since jewel

2017-05-15 Thread Jason Dillaman
m 14.05.2017 um 19:33 schrieb Stefan Priebe - Profihost AG: >> Hello Jason, >> >> Am 14.05.2017 um 14:04 schrieb Jason Dillaman: >>> It appears as though there is client.27994090 at 10.255.0.13 that >>> currently owns the exclusive lock on that image. I am assuming the

Re: [ceph-users] corrupted rbd filesystems since jewel

2017-05-15 Thread Jason Dillaman
;>> >>> The problem only seems to occur at all if a client has connected to >>> hammer without exclusive lock. Than got upgraded to jewel and exclusive >>> lock gets enabled. >>> >>> Greets, >>> Stefan >>> >>> Am 14.05.2017 um

Re: [ceph-users] corrupted rbd filesystems since jewel

2017-05-15 Thread Jason Dillaman
On Mon, May 15, 2017 at 3:54 PM, Stefan Priebe - Profihost AG wrote: > Would it be possible that the problem is the same you fixed? No, I would not expect it to be related to the other issues you are seeing. The issue I just posted a fix against only occurs when a client requests the lock from th

Re: [ceph-users] Ceph memory overhead when used with KVM

2017-05-16 Thread Jason Dillaman
t; > [iometer_just_write] >> > stonewall >> > bs=4M >> > rw=write >> > >> > [iometer_just_read] >> > stonewall >> > bs=4M >> > rw=read >> > """ >> > >> > Then let it run: >> > $&g

Re: [ceph-users] corrupted rbd filesystems since jewel

2017-05-16 Thread Jason Dillaman
On Tue, May 16, 2017 at 2:12 AM, Stefan Priebe - Profihost AG wrote: > 3.) it still happens on pre jewel images even when they got restarted / > killed and reinitialized. In that case they've the asok socket available > for now. Should i issue any command to the socket to get log out of the > hang

Re: [ceph-users] corrupted rbd filesystems since jewel

2017-05-16 Thread Jason Dillaman
ot; > ] > } > ], > "linger_ops": [ > { > "linger_id": 1, > "pg": "5.5f3bd635", > "osd": 17, > "object_id": "rbd_header.e10ca56b

Re: [ceph-users] corrupted rbd filesystems since jewel

2017-05-16 Thread Jason Dillaman
On Tue, May 16, 2017 at 3:37 PM, Stefan Priebe - Profihost AG wrote: > We've enabled the op tracker for performance reasons while using SSD > only storage ;-( Disabled you mean? > Can enable the op tracker using ceph osd tell? Than reproduce the > problem. Check what has stucked again? Or should

Re: [ceph-users] corrupted rbd filesystems since jewel

2017-05-16 Thread Jason Dillaman
"target_object_locator": "@5", > "paused": 0, > "used_replica": 0, > "precalc_pgid": 0, > "snapid": "head", > "registered": "1"

Re: [ceph-users] corrupted rbd filesystems since jewel

2017-05-17 Thread Jason Dillaman
The VM was running until 2017-05-17 12:10 but there was no I/O for 10 min. > > Greets, > Stefan > > Am 16.05.2017 um 22:54 schrieb Jason Dillaman: >> It looks like it's just a ping message in that capture. >> >> Are you saying that you restarted OSD 46 and the

Re: [ceph-users] corrupted rbd filesystems since jewel

2017-05-17 Thread Jason Dillaman
all debug symbols and issue a gdb: "thread apply all bt full"? > > Does it help? > > Greets, > Stefan > > Am 17.05.2017 um 15:12 schrieb Jason Dillaman: >> Perfect librbd log capture. I can see that a remove request to object >> rbd_data.e10ca56b8b4567.

Re: [ceph-users] corrupted rbd filesystems since jewel

2017-05-17 Thread Jason Dillaman
On Wed, May 17, 2017 at 10:21 AM, Stefan Priebe - Profihost AG wrote: > You mean the request no matter if it is successful or not? Which log > level should be set to 20? I'm hoping you can re-create the hung remove op when OSD logging is increased -- "debug osd = 20" would be nice if you can tur

Re: [ceph-users] corrupted rbd filesystems since jewel

2017-05-17 Thread Jason Dillaman
On Wed, May 17, 2017 at 10:25 AM, Stefan Priebe - Profihost AG wrote: > issue the delete request and send you the log? Yes, please. -- Jason ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] corrupted rbd filesystems since jewel

2017-05-17 Thread Jason Dillaman
"linger_id": 1, > "pg": "2.f0709c34", > "osd": 23, > "object_id": "rbd_header.21aafa6b8b4567", > "object_locator": "@2", > "targe

Re: [ceph-users] corrupted rbd filesystems since jewel

2017-05-17 Thread Jason Dillaman
b Stefan Priebe - Profihost AG: >> Hello Jason, >> >> the command >> # rados -p cephstor6 rm rbd_data.21aafa6b8b4567.0aaa >> >> hangs as well. Doing absolutely nothing... waiting forever. >> >> Greets, >> Stefan >> >> Am

Re: [ceph-users] corrupted rbd filesystems since jewel

2017-05-17 Thread Jason Dillaman
Stefan Priebe - Profihost AG: >> Hi, >> >> that command does not exist. >> >> But at least ceph -s permanently reports 1 pg in scrubbing with no change. >> >> Log attached as well. >> >> Greets, >> Stefan >> Am 17.05.2017 um 20:20

Re: [ceph-users] corrupted rbd filesystems since jewel

2017-05-17 Thread Jason Dillaman
226 2017-05-10 03:43:20.849784 171715'10548192 >2017-05-04 14:27:39.210713 > > So it seems the same scrub is stuck again... even after restarting the > osd. It just took some time until the scrub of this pg happened again. > > Greets, > Stefan > Am 17.05.2017 um

Re: [ceph-users] corrupted rbd filesystems since jewel

2017-05-18 Thread Jason Dillaman
t; Greets, > Stefan > > Am 17.05.2017 um 21:26 schrieb Stefan Priebe - Profihost AG: >> Am 17.05.2017 um 21:21 schrieb Jason Dillaman: >>> Any chance you still have debug logs enabled on OSD 23 after you >>> restarted it and the scrub froze again? >> >> N

Re: [ceph-users] Snap rollback failed with exclusive-lock enabled

2017-05-22 Thread Jason Dillaman
That's by design -- it doesn't make sense to live-rollback a block device when you have a running VM actively accessing the device. Once you shut-down the VM, rollback should work just fine. On Mon, May 22, 2017 at 5:29 AM, Lijie wrote: > Hi All, > > When I do a snap rollback command with exclus

Re: [ceph-users] corrupted rbd filesystems since jewel

2017-05-22 Thread Jason Dillaman
er an osd restart. Any further ideas? > > Coredump of the OSD with hanging scrub? > > Greets, > Stefan > > Am 18.05.2017 um 17:26 schrieb Jason Dillaman: >> I'm unfortunately out of ideas at the moment. I think the best chance >> of figuring out what is wrong is to

Re: [ceph-users] corrupted rbd filesystems since jewel

2017-05-22 Thread Jason Dillaman
llo Jason, > > should i do a coredump or a thread apply all bt? > > Don't know what is better. > > Greets, > Stefan > > Am 22.05.2017 um 15:19 schrieb Jason Dillaman: >> If you cannot recreate with debug logging enabled, that might be the >> next best opt

Re: [ceph-users] 答复: Snap rollback failed with exclusive-lock enabled

2017-05-22 Thread Jason Dillaman
But if I don't execute creating ext4 filesystem,snap can rollback with > exclusive-lock enabled successfully . > > So can u explain it? > > -邮件原件- > 发件人: Jason Dillaman [mailto:jdill...@redhat.com] > 发送时间: 2017年5月22日 19:46 > 收件人: lijie 11803 (RD) > 抄送: Sage Weil; ce

Re: [ceph-users] corrupted rbd filesystems since jewel

2017-05-22 Thread Jason Dillaman
2017-05-19 23:40:14.150887 451238'20721095 >2017-05-17 21:25:09.174598 > > Greets, > Stefan > > Am 22.05.2017 um 15:32 schrieb Jason Dillaman: >> If you have the debug symbols installed, I'd say "thread apply all bt" >> in addition to a "

Re: [ceph-users] How does rbd preserve the consistency of WRITE requests that span across multiple objects?

2017-05-24 Thread Jason Dillaman
Just like a regular block device, re-orders are permitted between write barriers/flushes. For example, if I had a HDD with 512 byte sectors and I attempted to write 4K, there is no guarantee what the disk will look like if you had a crash mid-write or if you concurrently issued an overlapping write

Re: [ceph-users] 答复: How does rbd preserve the consistency of WRITE requests that span across multiple objects?

2017-05-24 Thread Jason Dillaman
ult of those two writes are as expected? Does it merge those two > operations, or synchronously issue those writes to the disk? If the latter, > does the file system insert some other operations, like io barrier, between > those to writes so that the underlying storage system is aw

Re: [ceph-users] would rbd cascade clone affect performance?

2017-06-07 Thread Jason Dillaman
Assuming the RBD object-map feature is *not* enabled, if the associated backing object was not overwritten in rbd2 nor rbd3, every read operation to that object would involve first attempting to read from rbd3's object, then rbd2's, followed by rbd1's, which would introduce extra latency. The first

Re: [ceph-users] A Questions about rbd-mirror

2017-06-15 Thread Jason Dillaman
s/troubleshooting/log-and-debug/ On Thu, Jun 15, 2017 at 3:06 AM, YuShengzuo wrote: > Hi Jason Dillaman, > > > > I have a question about rbd-mirror : > > Recently, I begin to use the feature , but I can’t find logs > about it.(it running well) > >

Re: [ceph-users] Sparse file info in filestore not propagated to other OSDs

2017-06-21 Thread Jason Dillaman
On Wed, Jun 21, 2017 at 3:05 AM, Piotr Dałek wrote: > I saw that RBD (librbd) does that - replacing writes with discards when > buffer contains only zeros. Some code that does the same in librados could > be added and it shouldn't impact performance much, current implementation of > mem_is_zero is

Re: [ceph-users] red IO hang (was disk timeouts in libvirt/qemu VMs...)

2017-06-21 Thread Jason Dillaman
Are some or many of your VMs issuing periodic fstrims to discard unused extents? On Wed, Jun 21, 2017 at 2:36 PM, Hall, Eric wrote: > After following/changing all suggested items (turning off exclusive-lock > (and associated object-map and fast-diff), changing host cache behavior, > etc.) this is

Re: [ceph-users] red IO hang (was disk timeouts in libvirt/qemu VMs...)

2017-06-21 Thread Jason Dillaman
ges so yes, there is the default > “/sbin/fstrim –all” in /etc/cron.weekly/fstrim. > > -- > Eric > > On 6/21/17, 1:58 PM, "Jason Dillaman" wrote: > > Are some or many of your VMs issuing periodic fstrims to discard > unused extents? > > On Wed,

Re: [ceph-users] red IO hang (was disk timeouts in libvirt/qemu VMs...)

2017-06-23 Thread Jason Dillaman
thodology are welcome. > > > We occasionally see blocked requests in a running log (ceph –w > log), > but not correlated with hung VM IO. Scrubbing doesn’t seem correlated either. > > -- > Eric > > On 6/21/17, 2:55 PM, "Jason Dillaman" wrote:

Re: [ceph-users] Which one should I sacrifice: Tunables or Kernel-rbd?

2017-06-23 Thread Jason Dillaman
CentOS 7.3's krbd supports Jewel tunables (CRUSH_TUNABLES5) and does not support NBD since that driver is disabled out-of-the-box. As an alternative for NBD, the goal is to also offer LIO/TCMU starting with Luminous and the next point release of CentOS (or a vanilla >=4.12-ish kernel). On Fri, Jun

Re: [ceph-users] red IO hang (was disk timeouts in libvirt/qemu VMs...)

2017-06-23 Thread Jason Dillaman
On Fri, Jun 23, 2017 at 8:47 AM, Hall, Eric wrote: > I have debug logs. Should I open a RBD tracker ticket at > http://tracker.ceph.com/projects/rbd/issues for this? Yes, please. You might need to use the "ceph-post-file" utility if the logs are too large to attach to the ticket. In that case,

Re: [ceph-users] cannot open /dev/xvdb: Input/output error

2017-06-25 Thread Jason Dillaman
Are you using librbd via QEMU or krbd? If librbd, what errors are noted in the instance's librbd log file? On Sun, Jun 25, 2017 at 4:30 AM, Massimiliano Cuttini wrote: > After 4 months of test we decided to go live and store real VDI in > production. > However just the same day something went su

Re: [ceph-users] Snapshot removed, cluster thrashed...

2017-06-26 Thread Jason Dillaman
Restoring a snapshot involves copying the entire image from the snapshot revision to the HEAD revision. The faster approach would be to just create a clone from the snapshot. 2017-06-26 10:59 GMT-04:00 Marco Gaiarin : > Mandi! Lindsay Mathieson > In chel di` si favelave... > >> Have you tried re

Re: [ceph-users] Multi Tenancy in Ceph RBD Cluster

2017-06-26 Thread Jason Dillaman
On Mon, Jun 26, 2017 at 2:55 PM, Mayank Kumar wrote: > Thanks David, few more questions:- > - Is there a way to limit the capability of the keyring which is used to > map/unmap/lock to only allow those operations and nothing else using that > specific keyring Since RBD is basically just a collect

Re: [ceph-users] qemu-kvm vms start or reboot hang long time while using the rbd mapped image

2017-06-26 Thread Jason Dillaman
May I ask why you are using krbd with QEMU instead of librbd? On Fri, Jun 16, 2017 at 12:18 PM, 码云 wrote: > Hi All, > Recently.I meet a question and I did'nt find out any thing for explain it. > > Ops process like blow: > ceph 10.2.5 jewel, qemu 2.5.0 centos 7.2 x86_64 > create pool rbd_vms 3

Re: [ceph-users] qemu-kvm vms start or reboot hang long time whileusing the rbd mapped image

2017-06-27 Thread Jason Dillaman
lots of place in the > centos7.3, > are they fixed for something? > Tks and Rgds. > > > > ------ 原始邮件 -- > 发件人: "Jason Dillaman";; > 发送时间: 2017年6月27日(星期二) 上午7:28 > 收件人: "码云"; > 抄送: "ceph-users"; > 主题: Re: [cep

Re: [ceph-users] Performance issue with small files, and weird "workaround"

2017-06-27 Thread Jason Dillaman
Have you tried blktrace to determine if there are differences in the IO patterns to the rbd-backed virtio-scsi block device (direct vs indirect through loop)? On Tue, Jun 27, 2017 at 3:17 PM, Ruben Rodriguez wrote: > > We are setting a new set of servers to run the FSF/GNU infrastructure, > and w

Re: [ceph-users] rbd-fuse performance

2017-06-27 Thread Jason Dillaman
On Tue, Jun 27, 2017 at 7:17 PM, Daniel K wrote: > I'm trying to find a good way to mount ceph rbd images for export by > LIO/targetcli I would eventually recommend just directly serving the RBD images via LIO/TCMU. This is still a work-in-progress but it's being actively worked on with the goal

Re: [ceph-users] qemu-img convert vs rbd import performance

2017-06-28 Thread Jason Dillaman
Given that your time difference is roughly 10x, best guess is that qemu-img is sending the IO operations synchronously (queue depth = 1), whereas, by default, "rbd import" will send up to 10 write requests in parallel to the backing OSDs. Such an assumption assumes that you have really high latency

Re: [ceph-users] qemu-img convert vs rbd import performance

2017-06-28 Thread Jason Dillaman
working fine. We have not changed any queue depth setting > on that setup either. If it turns out to be queue depth how can we set queue > setting for qemu-img convert operation? > > Thank you. > > Sent from my iPhone > >> On Jun 28, 2017, at 7:56 PM, Jason Dillaman wrot

Re: [ceph-users] Ceph mount rbd

2017-06-28 Thread Jason Dillaman
... additionally, the forthcoming 4.12 kernel release will support non-cooperative exclusive locking. By default, since 4.9, when the exclusive-lock feature is enabled, only a single client can write to the block device at a time -- but they will cooperatively pass the lock back and forth upon writ

Re: [ceph-users] Question about rbd-mirror

2017-06-29 Thread Jason Dillaman
On Wed, Jun 28, 2017 at 11:42 PM, YuShengzuo wrote: > Hi Jason Dillaman, > > > > I am using rbd-mirror now (release Jewel). > > > > 1. > > And in many webs or other information introduced rbd-mirror notices that two > ceph cluster should be the ‘same fsid’. &g

Re: [ceph-users] slow cluster perfomance during snapshot restore

2017-06-29 Thread Jason Dillaman
On Thu, Jun 29, 2017 at 1:33 PM, Gregory Farnum wrote: > I'm not sure if there are built-in tunable commands available (check the > manpages? Or Jason, do you know?), but if not you can use any generic > tooling which limits how much network traffic the RBD command can run. Long-running RBD actio

Re: [ceph-users] Note about rbd_aio_write usage

2017-07-06 Thread Jason Dillaman
Pre-Luminous also copies the provided buffer when using the C API -- it just copies it at a later point and not immediately. The eventual goal is to eliminate the copy completely, but that requires some additional plumbing work deep down within the librados messenger layer. On Thu, Jul 6, 2017 at

Re: [ceph-users] Note about rbd_aio_write usage

2017-07-06 Thread Jason Dillaman
. On Thu, Jul 6, 2017 at 9:33 AM, Piotr Dałek wrote: > On 17-07-06 03:03 PM, Jason Dillaman wrote: >> >> On Thu, Jul 6, 2017 at 8:26 AM, Piotr Dałek >> wrote: >>> >>> Hi, >>> >>> If you're using "rbd_aio_write()" in your co

Re: [ceph-users] Note about rbd_aio_write usage

2017-07-06 Thread Jason Dillaman
On Thu, Jul 6, 2017 at 10:22 AM, Piotr Dałek wrote: > So I really see two problems here: lack of API docs and > backwards-incompatible change in API behavior. Docs are always in need of update, so any pull requests would be greatly appreciated. However, I disagree that the behavior has substanti

Re: [ceph-users] Note about rbd_aio_write usage

2017-07-06 Thread Jason Dillaman
On Thu, Jul 6, 2017 at 11:46 AM, Piotr Dałek wrote: > How about a hybrid solution? Keep the old rbd_aio_write contract (don't copy > the buffer with the assumption that it won't change) and instead of > constructing bufferlist containing bufferptr to copied data, construct a > bufferlist containin

Re: [ceph-users] Note about rbd_aio_write usage

2017-07-06 Thread Jason Dillaman
On Thu, Jul 6, 2017 at 3:25 PM, Piotr Dałek wrote: > Is that deep copy an equivalent of what > Jewel librbd did at unspecified point of time, or extra one? It's equivalent / replacement -- not an additional copy. This was changed to support scatter/gather IO API methods which the latest version o

Re: [ceph-users] krbd journal support

2017-07-06 Thread Jason Dillaman
There are no immediate plans to support the RBD journaling in krbd. The journaling feature requires a lot of code and, with limited resources, the priority has been to provide alternative block device options that pass-through to librbd for such use-cases and to optimize the performance of librbd /

Re: [ceph-users] Note about rbd_aio_write usage

2017-07-07 Thread Jason Dillaman
On Fri, Jul 7, 2017 at 2:48 AM, Piotr Dałek wrote: > Is this: > https://github.com/yuyuyu101/ceph/commit/794b49b5b860c538a349bdadb16bb6ae97ad9c20#commitcomment-15707924 > the issue you mention? Because at this point I'm considering switching to > C++ API and passing static bufferptr buried in my b

Re: [ceph-users] RBD journaling benchmarks

2017-07-10 Thread Jason Dillaman
On Mon, Jul 10, 2017 at 1:39 PM, Maged Mokhtar wrote: > These are significant differences, to the point where it may not make sense > to use rbd journaling / mirroring unless there is only 1 active client. I interpreted the results as the same RBD image was being concurrently used by two fio jobs

Re: [ceph-users] RBD journaling benchmarks

2017-07-12 Thread Jason Dillaman
On Mon, Jul 10, 2017 at 3:41 PM, Maged Mokhtar wrote: > On 2017-07-10 20:06, Mohamad Gebai wrote: > > > On 07/10/2017 01:51 PM, Jason Dillaman wrote: > > On Mon, Jul 10, 2017 at 1:39 PM, Maged Mokhtar wrote: > > These are significant differences, to the point where it may

Re: [ceph-users] qemu-img convert vs rbd import performance

2017-07-13 Thread Jason Dillaman
I'll refer you to the original thread about this [1] that was awaiting an answer. I would recommend dropping the "-t none" option since that might severely slow down sequential write operations if "qemu-img convert" is performing 512 byte IO operations. You might also want to consider adding the "-

Re: [ceph-users] qemu-img convert vs rbd import performance

2017-07-13 Thread Jason Dillaman
On Thu, Jul 13, 2017 at 8:57 AM, Irek Fasikhov wrote: > rbd readahead disable after bytes = 0 There isn't any reading from an RBD image in this example -- plus readahead disables itself automatically after the first 50MBs of IO (i.e. after the OS should have had enough time to start its own

Re: [ceph-users] 答复: No "snapset" attribute for clone object

2017-07-13 Thread Jason Dillaman
Quite possibly the same as this issue? [1] [1] http://tracker.ceph.com/issues/17445 On Thu, Jul 13, 2017 at 8:13 AM, 许雪寒 wrote: > By the way, we are using hammer version's rbd command to export-diff rbd > images on Jewel version's cluster. > > -邮件原件- > 发件人: ceph-users [mailto:ceph-users

Re: [ceph-users] RBD journaling benchmarks

2017-07-13 Thread Jason Dillaman
On Thu, Jul 13, 2017 at 10:58 AM, Maged Mokhtar wrote: > The case also applies to active/passive iSCSI.. you still have many > initiators/hypervisors writing concurrently to the same rbd image using a > clustered file system (csv/vmfs). Except from that point-of-view, there is only a single RBD c

Re: [ceph-users] 答复: 答复: No "snapset" attribute for clone object

2017-07-14 Thread Jason Dillaman
at 3:02 AM, 许雪寒 wrote: > Yes, I believe so. Is there any workarounds? > > -邮件原件- > 发件人: Jason Dillaman [mailto:jdill...@redhat.com] > 发送时间: 2017年7月13日 21:13 > 收件人: 许雪寒 > 抄送: ceph-users@lists.ceph.com > 主题: Re: [ceph-users] 答复: No "snapset" attribute for c

Re: [ceph-users] Ceph mount rbd

2017-07-14 Thread Jason Dillaman
> > In the moment, in my enviroment testing with ceph, using the version >> > 4.10 of kernel and i mount the system in two machines in the same >> > time, in production enviroment, i could serious problem with this >> > comportament. >> > >> > How c

Re: [ceph-users] RBD cache being filled up in small increases instead of 4MB

2017-07-15 Thread Jason Dillaman
On Sat, Jul 15, 2017 at 9:43 AM, Nick Fisk wrote: > Unless you tell the rbd client to not disable readahead after reading the 1st > x number of bytes (rbd readahead disable after bytes=0), it will stop reading > ahead and will only cache exactly what is requested by the client. The default is t

Re: [ceph-users] iSCSI production ready?

2017-07-17 Thread Jason Dillaman
On Sat, Jul 15, 2017 at 11:01 PM, Alvaro Soto wrote: > Hi guys, > does anyone know any news about in what release iSCSI interface is going to > be production ready, if not yet? There are several flavors of RBD iSCSI implementations that are in-use by the community. We are working to solidify the

Re: [ceph-users] RBD cache being filled up in small increases instead of 4MB

2017-07-17 Thread Jason Dillaman
On Sat, Jul 15, 2017 at 5:35 PM, Ruben Rodriguez wrote: > > > On 15/07/17 15:33, Jason Dillaman wrote: >> On Sat, Jul 15, 2017 at 9:43 AM, Nick Fisk wrote: >>> Unless you tell the rbd client to not disable readahead after reading the >>> 1st x number of bytes (

Re: [ceph-users] RBD cache being filled up in small increases instead of 4MB

2017-07-17 Thread Jason Dillaman
Are you 100% positive that your files are actually stored sequentially on the block device? I would recommend running blktrace to verify the IO pattern from your use-case. On Sat, Jul 15, 2017 at 5:42 PM, Ruben Rodriguez wrote: > > > On 15/07/17 09:43, Nick Fisk wrote: >>> -Original Message--

Re: [ceph-users] RBD cache being filled up in small increases instead of 4MB

2017-07-17 Thread Jason Dillaman
On Sat, Jul 15, 2017 at 8:00 PM, Ruben Rodriguez wrote: > > > On 14/07/17 18:43, Ruben Rodriguez wrote: >> How to reproduce... > > I'll provide more concise details on how to test this behavior: > > Ceph config: > > [client] > rbd readahead max bytes = 0 # we don't want forced readahead to fool us

Re: [ceph-users] qemu-img convert vs rbd import performance

2017-07-20 Thread Jason Dillaman
> > Parameters Time taken > -t writeback 38mins > -t none 38 mins > -S 4k 38 mins > With client options mentions by Irek Fasikhov 40 mins > The time taken is almost the same. > > On Thu, Jul 13, 2017 at 6:40 PM, Jason Dillaman > wrote: > >> On Thu, Jul 13

Re: [ceph-users] Is it possible to get IO usage (read / write bandwidth) by client or RBD image?

2017-07-21 Thread Jason Dillaman
Nothing is built-in for this yet, but it's on the roadmap for a future release [1]. [1] http://pad.ceph.com/p/ceph-top On Thu, Jul 20, 2017 at 9:52 AM, Stéphane Klein wrote: > Hi, > > is it possible to get IO stats (read / write bandwidth) by client or image? > > I see this thread > http://lists

Re: [ceph-users] Exclusive-lock Ceph

2017-07-24 Thread Jason Dillaman
You will need to pass the "exclusive" option when running "rbd map" (and be running kernel >= 4.12). On Mon, Jul 24, 2017 at 8:42 AM, wrote: > I'm testing ceph in my enviroment, but the feature exclusive lock don't > works fine for me or maybe i'm doing something wrong. > > I testing in two mach

Re: [ceph-users] Restore RBD image

2017-07-24 Thread Jason Dillaman
Increasing the size of an image only issues a single write to update the image size metadata in the image header. That operation is atomic and really shouldn't be able to do what you are saying. Regardless, since this is a grow operation, just re-run the resize to update the metadata again. On Mo

Re: [ceph-users] how to map rbd using rbd-nbd on boot?

2017-07-24 Thread Jason Dillaman
Your google-fu hasn't failed -- that is a missing feature. I've opened a new feature-request tracker ticket to get support for that. [1] http://tracker.ceph.com/issues/20762 On Fri, Jul 21, 2017 at 5:04 PM, Daniel K wrote: > Once again my google-fu has failed me and I can't find the 'correct' wa

Re: [ceph-users] Exclusive-lock Ceph

2017-07-24 Thread Jason Dillaman
t;exclusive" map option that is only available starting with kernel 4.12. > 2 The comand with exclusive > would this ? > > rbd map --exclusive test-xlock3 Yes, that should be it. > Thanks a Lot, > > Marcelo > > > Em 24/07/2017, Jason Dillaman escreveu: >

Re: [ceph-users] oVirt/RHEV and Ceph

2017-07-24 Thread Jason Dillaman
oVirt 3.6 added Cinder/RBD integration [1] and it looks like they are currently working on integrating Cinder within a container to simplify the integration [2]. [1] http://www.ovirt.org/develop/release-management/features/storage/cinder-integration/ [2] http://www.ovirt.org/develop/release-mana

Re: [ceph-users] ceph v10.2.9 - rbd cli deadlock ?

2017-07-26 Thread Jason Dillaman
While I cannot reproduce what you are seeing, I can see how it could theoretically be possible for this to deadlock on a thread shutdown if the process was being shutdown before the service thread had a chance to actually start executing. I've opened a tracker ticket for the issue [1]. [1] http://

Re: [ceph-users] Ceph - OpenStack space efficiency

2017-08-01 Thread Jason Dillaman
You could just use the "rbd du" command to calculate the real disk usage of images / snapshots and compare that to the thin-provisioned size of the images. On Mon, Jul 31, 2017 at 11:28 PM, Italo Santos wrote: > Hello everyone, > > As we know the Openstack ceph integration uses ceph rbd snapshot

Re: [ceph-users] iSCSI production ready?

2017-08-02 Thread Jason Dillaman
e its configuration within ceph-ansible. On Wed, Aug 2, 2017 at 12:02 PM, Дмитрий Глушенок wrote: > Will it be a separate project? There is a third RC for Luminous without a > word about iSCSI Gateway. > > 17 июля 2017 г., в 14:54, Jason Dillaman написал(а): > > On Sat, Jul 15, 2

Re: [ceph-users] broken parent/child relationship

2017-08-07 Thread Jason Dillaman
Does the image "tyr-p0/a56eae5f-fd35-4299-bcdc-65839a00f14c" have snapshots? If the deep-flatten feature isn't enabled, the flatten operation is not able to dissociate child images from parents when those child images have one or more snapshots. On Fri, Aug 4, 2017 at 2:30 PM, Shawn Edwards wrote

Re: [ceph-users] broken parent/child relationship

2017-08-07 Thread Jason Dillaman
ned on for that image. > > Deep-flatten cannot be added to an rbd after creation, correct? What are my > options here? > > On Mon, Aug 7, 2017 at 3:32 PM Jason Dillaman wrote: >> >> Does the image "tyr-p0/a56eae5f-fd35-4299-bcdc-65839a00f14c" have >> snapsh

Re: [ceph-users] hammer(0.94.5) librbd dead lock, i want to how to resolve

2017-08-07 Thread Jason Dillaman
I am not sure what you mean by "I stop ceph" (stopped all the OSDs?) -- and I am not sure how you are seeing ETIMEDOUT errors on a "rbd_write" call since it should just block assuming you are referring to stopping the OSDs. What is your use-case? Are you developing your own application on top of li

Re: [ceph-users] 答复: hammer(0.94.5) librbd dead lock,i want to how to resolve

2017-08-08 Thread Jason Dillaman
lient_mount_timeout = 75 > > -邮件原件- > 发件人: Jason Dillaman [mailto:jdill...@redhat.com] > 发送时间: 2017年8月8日 7:58 > 收件人: shilu 09816 (RD) > 抄送: ceph-users > 主题: Re: hammer(0.94.5) librbd dead lock,i want to how to resolve > > I am not sure what you mean by "I stop

Re: [ceph-users] Iscsi configuration

2017-08-08 Thread Jason Dillaman
We are working hard to formalize active/passive iSCSI configuration across Linux/Windows/ESX via LIO. We have integrated librbd into LIO's tcmu-runner and have developed a set of support applications to managing the clustered configuration of your iSCSI targets. There is some preliminary documentat

Re: [ceph-users] Iscsi configuration

2017-08-09 Thread Jason Dillaman
system and don't need nor want an added layer of complexity in the long term. On Wed, Aug 9, 2017 at 12:42 PM, Samuel Soulard wrote: > Hmm :( Even for an Active/Passive configuration? I'm guessing we will need > to do something with Pacemaker in the meantime? > > On Wed, Aug 9,

Re: [ceph-users] Possible to change RBD-Caching settings while rbd device is in use ?

2015-12-14 Thread Jason Dillaman
Sorry, none of the librbd configuration properties can be live-updated currently. -- Jason Dillaman - Original Message - > From: "Daniel Schwager" > To: "ceph-us...@ceph.com" > Sent: Friday, December 11, 2015 3:35:11 AM > Subject: [ceph-users

Re: [ceph-users] rbd du

2015-12-18 Thread Jason Dillaman
/19/real-size-of-a-ceph-rbd-image/ -- Jason Dillaman - Original Message - > From: "Allen Liao" > To: ceph-users@lists.ceph.com > Sent: Friday, August 21, 2015 3:24:54 PM > Subject: [ceph-users] rbd du > Hi all, > The online manual ( http://ceph.com/docs

Re: [ceph-users] letting and Infernalis

2016-01-04 Thread Jason Dillaman
tracing = false # enable librbd LTTng tracing You can dynamically enable LTTng on a running process via the admin socket as well. I created a tracker ticket for updating the documentation [1]. [1] http://tracker.ceph.com/issues/14219 -- Jason Dillaman - Original Message

Re: [ceph-users] How to do quiesced rbd snapshot in libvirt?

2016-01-04 Thread Jason Dillaman
I am surprised by the error you are seeing with exclusive lock enabled. The rbd CLI should be able to send the 'snap create' request to QEMU without an error. Are you able to provide "debug rbd = 20" logs from shortly before and after your snapshot attempt?

Re: [ceph-users] rbd bench-write vs dd performance confusion

2016-01-04 Thread Jason Dillaman
Hammer but it looks like it was missed. I will open a new tracker ticket to start that process. [1] https://github.com/ceph/ceph/commit/333f3a01a9916c781f266078391c580efb81a0fc -- Jason Dillaman - Original Message - > From: "Emile Snyder" > To: ceph-users@lists.ceph.co

Re: [ceph-users] can rbd block_name_prefix be changed?

2016-01-08 Thread Jason Dillaman
It's constant for an RBD image and is tied to the image's internal unique ID. -- Jason Dillaman - Original Message - > From: "min fang" > To: "ceph-users" > Sent: Friday, January 8, 2016 4:50:08 AM > Subject: [ceph-users] can rbd bl

Re: [ceph-users] Unable to see LTTng tracepoints in Ceph

2016-01-08 Thread Jason Dillaman
What tracing events do you see being generated from ceph-mon? I didn't realize it had any registered tracepoint events. -- Jason Dillaman - Original Message - > From: "Aakanksha Pudipeddi-SSI" > To: ceph-users@lists.ceph.com > Sent: Wednesday, January

Re: [ceph-users] 回复: can rbd block_name_prefix be changed?

2016-01-12 Thread Jason Dillaman
It's unique per-pool. -- Jason Dillaman - Original Message - > From: "louisfang2013" > To: "Jason Dillaman" > Cc: "ceph-users" > Sent: Tuesday, January 12, 2016 5:56:18 AM > Subject: 回复:[ceph-users] can rbd block_name_pref

Re: [ceph-users] How to do quiesced rbd snapshot in libvirt?

2016-01-13 Thread Jason Dillaman
Definitely would like to see the "debug rbd = 20" logs from 192.168.254.17 when this occurs. If you are co-locating your OSDs, MONs, and qemu-kvm processes, make sure your ceph.conf has "log file = " defined in the [global] or [client] section. -- Jason Dillaman --

Re: [ceph-users] How to do quiesced rbd snapshot in libvirt?

2016-01-14 Thread Jason Dillaman
I would need to see the log from the point where you've frozen the disks until the point when you attempt to create a snapshot. The logs below just show normal IO. I've opened a new ticket [1] where you can attach the logs. [1] http://tracker.ceph.com/issues/14373 -- Jaso

Re: [ceph-users] v10.0.2 released

2016-01-14 Thread Jason Dillaman
krbd. I wouldn't see this as a replacement for krbd, but rather another tool to support certain RBD use-cases [2]. [1] http://docs.ceph.com/docs/master/man/8/rbd/#commands [2] https://github.com/ceph/ceph/pull/6595 -- Jason Dillaman - Original Message - > From: "Bill S

Re: [ceph-users] rbd snap ls: how much locking is involved?

2016-01-21 Thread Jason Dillaman
Are you performing a lot of 'rbd export-diff' or 'rbd diff' operations? I can't speak to whether or not list-snaps is related to your blocked requests, but I can say that operation is only issued when performing RBD diffs. -- Jason Dillaman - Original Message

Re: [ceph-users] Ceph RBD bench has a strange behaviour when RBD client caching is active

2016-01-25 Thread Jason Dillaman
What release are you testing? You might be hitting this issue [1] where 'rbd bench-write' would issue the same IO request repeatedly. With writeback cache enabled, this would result in virtually no ops issued to the backend. [1] http://tracker.ceph.com/issues/14283 -- Jaso

Re: [ceph-users] Ceph RBD bench has a strange behaviour when RBD client caching is active

2016-01-25 Thread Jason Dillaman
Correct -- it was a bug in 'rbd bench-write' only. -- Jason Dillaman - Original Message - > From: "J-P Methot" > To: "Jason Dillaman" > Cc: ceph-users@lists.ceph.com > Sent: Monday, January 25, 2016 2:10:19 PM > Subject: Re: [ceph-

<    1   2   3   4   5   6   7   8   9   >