Re: Speeding up rbd_stat() in libvirt

2016-01-04 Thread Jason Dillaman
and image loading is asynchronous under the hood already. -- Jason Dillaman - Original Message - > From: "Wido den Hollander" <w...@42on.com> > To: ceph-devel@vger.kernel.org > Sent: Monday, December 28, 2015 8:48:40 AM > Subject: Speeding up r

Re: Is rbd_discard enough to wipe an RBD image?

2015-12-21 Thread Jason Dillaman
specify an offset somewhere in the middle of an object (i.e. the whole object cannot be deleted nor can it be truncated) -- this is the partial discard case controlled by that configuration param. -- Jason Dillaman - Original Message - > From: "Alexandre DERUMIER&

Re: The max single write IOPS on single RBD

2015-12-14 Thread Jason Dillaman
If you are testing with "iodepth=1", I'd recommend testing with "rbd non blocking aio = false" in your Ceph config file to see if that improves your single-threaded IO performance. -- Jason Dillaman - Original Message - > From: "Zhi Zhang" <zh

Re: About rbd journal

2015-12-08 Thread Jason Dillaman
is available and stable, periodic RBD incremental diff exports are probably the best backup alternative. -- Jason Dillaman - Original Message - > From: "changtao381" <changtao...@163.com> > To: dilla...@redhat.com > Cc: ceph-devel@vger.kernel.org > Sent: Tuesday,

Re: CEPH_RBD_API: options on image create

2015-10-15 Thread Jason Dillaman
399/functions/setsockopt.html > > Note option type documented for every option there, and it works > fairly well. > > Following a common practice is an additional argument to this > approach to me. Except for the following cases: sizeof(char*) == sizeof(uint32_t) (32bit) sizeof(char*

Re: CEPH_RBD_API: options on image create

2015-10-15 Thread Jason Dillaman
> > But we don't need them to match between different platforms, no? Is > linking 64bit code with 32bit possible (supported)? > > Also, for this particular (char*) case, length would actually be the > length of the string, not the pointer length. From my example: > > const char*

Re: CEPH_RBD_API: options on image create

2015-10-14 Thread Jason Dillaman
if someone passes a 2- or 8-byte int or a 4-byte char* string? Therefore, I would vote for passing strings a la librados rados_conf_set. Perhaps rbd_create4 and rbd_clone3 should move the order and features options to rbd_image_options_t as well? -- Jason Dillaman - Original Message

Re: lltng enabled by default and qemu apparmor|selinux problems

2015-10-12 Thread Jason Dillaman
to use LTTng-UST (i.e. for generating RBD replay traces) can enable the support and adjust their SElinux / AppArmor rules to accommodate. [1] https://github.com/ceph/ceph/pull/6135 -- Jason Dillaman - Original Message - > From: "Paul HEWLETT (Paul)" <paul.hewl...@al

Re: Running the rbd suite on OpenStack

2015-09-30 Thread Jason Dillaman
-- Jason Dillaman - Original Message - > From: "Loic Dachary" <l...@dachary.org> > To: "Jason Dillaman" <dilla...@redhat.com> > Cc: "Ceph Development" <ceph-devel@vger.kernel.org> > Sent: Wednesday, September 30, 2015 5:34:59 PM

Re: RBD mirroring CLI proposal ...

2015-09-23 Thread Jason Dillaman
> In this case the commands look a little confusing to me, as from their > names I would rather think they enable/disable mirror for existent > images too. Also, I don't see a command to check what current > behaviour is. And, I suppose it would be useful if we could configure > other default

Re: RBD mirroring CLI proposal ...

2015-09-23 Thread Jason Dillaman
> > > * rbd mirror pool add > > > This will register a remote cluster/pool as a peer to the > > > current, > > > local pool. All existing mirrored images and all future mirrored > > > images will have this peer registered as a journal client. > > > > > > * rbd

Re: RBD mirroring CLI proposal ...

2015-09-23 Thread Jason Dillaman
> > In this case the commands look a little confusing to me, as from their > > names I would rather think they enable/disable mirror for existent > > images too. Also, I don't see a command to check what current > > behaviour is. And, I suppose it would be useful if we could configure > > other

Re: RBD mirroring CLI proposal ...

2015-09-23 Thread Jason Dillaman
> So a pool policy is just a set of feature bits? It would have to store additional details as well. > I think Cinder at least creates images with rbd_default_features from > ceph.conf and adds in layering if it's not set, meaning there is no > interface for passing through feature bits (or

Re: RBD mirroring CLI proposal ...

2015-09-22 Thread Jason Dillaman
> I guess before attaching I will need to disable journaling for volumes > (because it was automatically enabled previously with 'rbd mirror pool > enable')? Will it automatically enable mirroring feature for the > attached images? I was planning to just fail the attach if journaling was a

RBD mirroring CLI proposal ...

2015-09-21 Thread Jason Dillaman
This will import the JSON-formated journal * rbd journal reset This will delete all entries from the journal where is [/]journal name -- Jason Dillaman [1] http://tracker.ceph.com/projects/ceph/wiki/RBD_Async_Mirroring -- To unsubscribe from this list: send the line

Re: bug with cache/tiering and snapshot reads

2015-09-08 Thread Jason Dillaman
it every time > using the script from the description. I kind of remember something similar > but it involved erasure coded pools and is probably different. I believe it's related to this thread [1]. There is an cache tier / EC pool issue wrt proxied writes, but this is different. --

Re: hello, I am confused about a question of rbd

2015-07-23 Thread Jason Dillaman
According to the git history, support for zero MB images for the create/resize commands was explicitly added by commit 08f47a4. Dan Mick or Josh Durgin could probably better explain the history behind the change since it was before my time. -- Jason Dillaman Red Hat dilla...@redhat.com

Re: upstream/firefly exporting the same snap 2 times results in different exports

2015-07-21 Thread Jason Dillaman
: log file = /valid/path/to/logs/$name.$pid.log debug rbd = 20 I opened a ticket [1] where you can attach the logs (if they aren't too large). [1] http://tracker.ceph.com/issues/12422 -- Jason Dillaman Red Hat dilla...@redhat.com http://www.redhat.com - Original Message

Re: upstream/firefly exporting the same snap 2 times results in different exports

2015-07-21 Thread Jason Dillaman
name. -- Jason Dillaman Red Hat dilla...@redhat.com http://www.redhat.com -- To unsubscribe from this list: send the line unsubscribe ceph-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: upstream/firefly exporting the same snap 2 times results in different exports

2015-07-21 Thread Jason Dillaman
Any chance that the snapshot was just created prior to the first export and you have a process actively writing to the image? -- Jason Dillaman Red Hat dilla...@redhat.com http://www.redhat.com - Original Message - From: Stefan Priebe - Profihost AG s.pri...@profihost.ag

Re: About Adding eventfd support for LibRBD

2015-07-13 Thread Jason Dillaman
need to use eventfd. -- Jason Dillaman Red Hat dilla...@redhat.com http://www.redhat.com - Original Message - From: Haomai Wang haomaiw...@gmail.com To: Josh Durgin jdur...@redhat.com Cc: ceph-devel@vger.kernel.org, Jason Dillaman dilla...@redhat.com Sent: Thursday, July 9, 2015

Re: About Adding eventfd support for LibRBD

2015-07-13 Thread Jason Dillaman
But it doesn't provide an easily compassable way of integrating waiting on other events in the application. eventfd is easy to embed in your (e)pool loop or any kind of event library (libev). Agreed -- which is why I asked about the proposed design since it appears (to me) that everything is

Re: About Adding eventfd support for LibRBD

2015-07-13 Thread Jason Dillaman
Sorry, I'm not following. Even we have poll_io_events, we need to when to call poll_io_events. I guess you mean we could notify user's side fd in rbd callback. Yes, we could do this. But a extra rbd callback could be omitted if we embed standard notification methods, we can get performance

Re: librbd cacher lock protection?

2015-06-26 Thread Jason Dillaman
extents populated. -- Jason Dillaman Red Hat dilla...@redhat.com http://www.redhat.com - Original Message - From: Maciej Patelczyk maciej.patelc...@intel.com To: Jason Dillaman dilla...@redhat.com Cc: ceph-devel@vger.kernel.org Sent: Friday, June 26, 2015 12:00:15 PM Subject: RE

Re: librbd cacher lock protection?

2015-06-23 Thread Jason Dillaman
'ObjectCacher::bh_read_finish', which verifies that the BH is still in the rx state (and that the transaction ids match) before overwriting the data. The pending client read request will then complete and will be provided the latest contents of the BH(s). -- Jason Dillaman Red Hat dilla...@redhat.com

Re: make check run time

2015-06-19 Thread Jason Dillaman
--Jason Dillaman Red Hat dilla...@redhat.com http://www.redhat.com - Loic Dachary l...@dachary.org wrote: Hi, Below is the sorted list of the longest running tests (measured by comparing the .log creation and modification times after a successfully run), in seconds. Note that when

Re: RBD journal draft design

2015-06-09 Thread Jason Dillaman
I must not be being clear. Tell me if this scenario is possible: * Client A writes to file foo many times and it is journaled to object set 1. * Client B writes to file bar many times and it starts journaling to object set 1, but hits the end and moves on to object set 2. * Client A hits a

Re: [ceph-users] rbd_cache, limiting read on high iops around 40k

2015-06-09 Thread Jason Dillaman
In the past we've hit some performance issues with RBD cache that we've fixed, but we've never really tried pushing a single VM beyond 40+K read IOPS in testing (or at least I never have). I suspect there's a couple of possibilities as to why it might be slower, but perhaps joshd can chime

Re: RBD journal draft design

2015-06-04 Thread Jason Dillaman
...Actually, doesn't *not* forcing a coordinated move from one object set to another mean that you don't actually have an ordering guarantee across tags if you replay the journal objects in order? The ordering between tags was meant to be a soft ordering guarantee (since any number of

Re: RBD journal draft design

2015-06-04 Thread Jason Dillaman
A successful append will indicate whether or not the journal is now full (larger than the max object size), indicating to the client that a new journal object should be used. If the journal is too large, an error code responce would alert the client that it needs to write to the current

Re: RBD journal draft design

2015-06-03 Thread Jason Dillaman
In contrast to the current journal code used by CephFS, the new journal code will use sequence numbers to identify journal entries, instead of offsets within the journal. Am I misremembering what actually got done with our journal v2 format? I think this is done — or at least we made a

Re: RBD journal draft design

2015-06-03 Thread Jason Dillaman
A new journal object class method will be used to submit journal entry append requests. This will act as a gatekeeper for the concurrent client case. A successful append will indicate whether or not the journal is now full (larger than the max object size), indicating to the client that

RBD journal draft design

2015-06-02 Thread Jason Dillaman
Journal Object ~~~ Data * 1..N: Journal Entry Journal Entries ~~~ Header * version * tag * sequence number * data size Data * raw payload Footer * CRC of journal entry header + data [1] http://comments.gmane.org/gmane.comp.file-systems.ceph.devel/24929 -- Jason Dillaman

Re: Regarding rbd --image-shared and --shared tag

2015-03-09 Thread Jason Dillaman
on a single image for backwards compatibility with krbd (since it does not yet support this new feature) or other use-cases where the image can be opened concurrently by two or more clients. -- Jason Dillaman Red Hat dilla...@redhat.com http://www.redhat.com - Original Message - From

Re: RBD tests on the next dumpling release

2015-02-09 Thread Jason Dillaman
I would agree with your assessment that http://tracker.ceph.com/issues/10560#teuthology-runs-on-3944c77c404c4a05886fe8276d5d0dd7e4f20410-6-february sounds like a repeat of http://tracker.ceph.com/issues/4959. Josh, thoughts? -- Jason Dillaman Red Hat dilla...@redhat.com http

Re: RBD tests on the next dumpling release

2015-02-06 Thread Jason Dillaman
a dumpling pull request for that issue ASAP. -- Jason Dillaman Red Hat dilla...@redhat.com http://www.redhat.com - Original Message - From: Loic Dachary l...@dachary.org To: Jason Dillaman dilla...@redhat.com, Josh Durgin jdur...@redhat.com Cc: Ceph Development ceph-devel@vger.kernel.org