[dm-devel] [PATCH] thin_dump: added --device-id, --skip-mappings, and new output --format's

2016-03-14 Thread Eric Wheeler
Hi Joe,

Please review the patch below when you have a moment.  I am interested in 
your feedback, and also interested in having this functionality merged 
upstream.  This was written against thin-provisioning-tools.git tag 
v0.5.6.

We use thin_dump on live dm-thin metadata snapshots all the time.  In our 
case, we want to dump the XML for new (snapshot) volumes instead of 
dumping the entire 16gb metadata device (37.8% used) which takes ~20-30 
minutes instead of ~5 seconds for a single volume with --device-id.

I started by adding --device-id to pass the `block_address dev_id` and 
skip the call to emit_mappings() within mapping_tree_emitter::visit() 
unless the dev_id matches --device-id as passed to thin_dump.  

It is sometimes nice to list all of the device supers without waiting for 
emit_mappings() so I added the --skip-mappings option.  This allows you to 
get the dev_id without reading the mappings:
thin_dump --skip-mappings -m /dev/mapper/vg-p_tmeta

Like `bool repair`, I added skip_mappings and dev_id as arguments to 
thin_provisioning::metadata_dump() and passed them down to 
mapping_tree_emitter with new members set_skip_mappings() and set_dev_id() 
to set private attributes (default values are assigned in 
metadata_dumper.h in case of backward compatible calls).

We work with the device metadata in a simplified format, and without 
superblock information:

origin_offset:length:tdata_offset (hence, "o:L:d" or "old") 

Therefore, the output --format 'old' was added.  I added the format 'null' 
as well for benchmarking purposes so the ostream needn't write large 
volumes of text to /dev/null.  

Benchmarks of the various formats on our sample metadata snapshot on an 
idle machine:

for i in xml human_readable old null; do
time thin_dump -m -f $i /dev/mapper/vg-p_tmeta -o /dev/null
done

realusersys
xml 29:01.2725:35.8701:54.49
human   27:39.8124:12.9001:52.39
old 27:07.7123:40.6401:52.97
null23:39.3821:17.2200:49.04

I have this as a branch in bitbucket if you prefer to see it online:
  
https://bitbucket.org/ewheelerinc/thin-provisioning-tools/branch/v0.5.6-device-id
  or
git pull https://bitbucket.org/ewheelerinc/thin-provisioning-tools.git 
v0.5.6-device-id


--
Eric Wheeler


diff --git a/thin-provisioning/thin_dump.cc b/thin-provisioning/thin_dump.cc
index 191acb5..7df5d85 100644
--- a/thin-provisioning/thin_dump.cc
+++ b/thin-provisioning/thin_dump.cc
@@ -22,6 +22,8 @@
 #include 
 
 #include "human_readable_format.h"
+#include "old_format.h"
+#include "null_format.h"
 #include "metadata_dumper.h"
 #include "metadata.h"
 #include "xml_format.h"
@@ -36,6 +38,8 @@ using namespace thin_provisioning;
 struct flags {
bool find_metadata_snap;
bool repair;
+   block_address dev_id;
+   bool skip_mappings;
 };
 
 namespace {
@@ -49,12 +53,16 @@ namespace {
e = create_xml_emitter(out);
else if (format == "human_readable")
e = create_human_readable_emitter(out);
+   else if (format == "old")
+   e = create_old_emitter(out);
+   else if (format == "null")
+   e = create_null_emitter(out);
else {
cerr << "unknown format '" << format << "'" << 
endl;
exit(1);
}
 
-   metadata_dump(md, e, flags.repair);
+   metadata_dump(md, e, flags.repair, flags.skip_mappings, 
flags.dev_id);
 
} catch (std::exception &e) {
cerr << e.what() << endl;
@@ -77,10 +85,12 @@ namespace {
out << "Usage: " << cmd << " [options] {device|file}" << endl
<< "Options:" << endl
<< "  {-h|--help}" << endl
-   << "  {-f|--format} {xml|human_readable}" << endl
+   << "  {-f|--format} {xml|human_readable|old|null}" << endl
<< "  {-r|--repair}" << endl
<< "  {-m|--metadata-snap} [block#]" << endl
<< "  {-o }" << endl
+   << "  {-d device_id}" << endl
+   << "  {-s|--skip-mappings}" << endl
<< "  {-V|--version}" << endl;
}
 }
@@ -89,17 +99,20 @@ int thin_dump_main(int argc, char **argv)
 {
int c;
char const *output = NULL;
-   const char shortopts[] = "hm::o:f:rV";
+   const char shortopts[] = "hm::o:sd:f:rV";
char *end_ptr;
string format = "xml";
block_address metadata_snap = 0;
struct flags flags;
-   flags.find_metadata_snap = flags.repair = false;
+   flags.find_metadata_snap = flags.repair = flags.skip_mappings = false;
+   flags.dev_id = -

Re: [dm-devel] block: transfer source bio's cgroup tags to clone via bio_associate_blkcg()

2016-03-14 Thread Nikolay Borisov
>
> Great.
>
> Jens, can you pick up the patch in question ("[PATCH] block: transfer
> source bio's cgroup tags to clone via bio_associate_blkcg()") that I
> posted in this thread?

And what about Vivek's patch of associating the source bio with the io
context of the process issuing the io?
Would that help in the DIO case?

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel


Re: [dm-devel] [PATCH RESEND v2 16/18] fuse: Support fuse filesystems outside of init_user_ns

2016-03-14 Thread Miklos Szeredi
On Wed, Mar 9, 2016 at 6:07 PM, Seth Forshee  wrote:
> On Wed, Mar 09, 2016 at 04:51:42PM +0100, Miklos Szeredi wrote:
>> On Wed, Mar 9, 2016 at 4:25 PM, Seth Forshee  
>> wrote:
>> > On Wed, Mar 09, 2016 at 03:48:22PM +0100, Miklos Szeredi wrote:
>>
>> >> Can't we use current_cred()->uid/gid? Or fsuid/fsgid maybe?
>> >
>> > That would be a departure from the current behavior in the !allow_other
>> > case for unprivileged users. Since those mounts are done by an suid
>> > helper all of those ids would be root in the userns, wouldn't they?
>>
>> Well, actually this is what the helper does:
>>
>> sprintf(d, "fd=%i,rootmode=%o,user_id=%u,group_id=%u",
>> fd, rootmode, getuid(), getgid());
>
> Sorry, I was thinking of euid. So this may not be a problem.
>
>> So it just uses the current uid/gid.  Apparently no reason to do this
>> in userland, we could just as well set these in the kernel.  Except
>> for possible backward compatibility problems for things not using the
>> helper.
>>
>> BUT if the mount is unprivileged or it's a userns mount, or anything
>> previously not possible, then we are not constrained by the backward
>> compatibility issues, and can go with the saner solution.
>>
>> Does that not make sense?
>
> But we generally do want backwards compatibility, and we want userspace
> software to be able to expect the same behavior whether or not it's
> running in a user namespaced container. Obviously we can't always have
> things 100% identical, but we shouldn't break things unless we really
> need to.
>
> However it may be that this isn't actually going to break assumptions of
> existing software like I had feared. My preference is still to not
> change any userspace-visible behaviors since we never know what software
> might have made assumptions based on those behaviors. But if you're
> confident that it won't break anything I'm willing to give it a try.

I'm quite confident it won't make a difference.

Thanks,
Miklos

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel


Re: [dm-devel] block: transfer source bio's cgroup tags to clone via bio_associate_blkcg()

2016-03-14 Thread Mike Snitzer
On Mon, Mar 14 2016 at 11:31am -0400,
Nikolay Borisov  wrote:

> 
> 
> On 03/14/2016 05:08 PM, Nikolay Borisov wrote:
> > 
> > 
> > On 03/02/2016 07:56 PM, Mike Snitzer wrote:
> >> On Wed, Mar 02 2016 at 11:06P -0500,
> >> Tejun Heo  wrote:
> >>
> >>> Hello,
> >>>
> >>> On Thu, Feb 25, 2016 at 09:53:14AM -0500, Mike Snitzer wrote:
>  Right, LVM created devices are bio-based DM devices in the kernel.
>  bio-based block devices do _not_ have an IO scheduler.  Their underlying
>  request-based device does.
> >>>
> >>> dm devices are not the actual resource source, so I don't think it'd
> >>> work too well to put io controllers on them (can't really do things
> >>> like proportional control without owning the queue).
> >>>
>  I'm not well-versed on the top-level cgroup interface and how it maps to
>  associated resources that are established in the kernel.  But it could
>  be that the configuration of blkio cgroup against a bio-based LVM device
>  needs to be passed through to the underlying request-based device
>  (e.g. /dev/sda4 in Chris's case)?
> 
>  I'm also wondering whether the latest cgroup work that Tejun has just
>  finished (afaik to support buffered IO in the IO controller) will afford
>  us a more meaningful reason to work to make cgroups' blkio controller
>  actually work with bio-based devices like LVM's DM devices?
> 
>  I'm very much open to advice on how to proceed with investigating this
>  integration work.  Tejun, Vivek, anyone else: if you have advice on next
>  steps for DM on this front _please_ yell, thanks!
> >>>
> >>> I think the only thing necessary is dm transferring bio cgroup tags to
> >>> the bio's that it ends up passing down the stack.  Please take a look
> >>> at fs/btrfs/extent_io.c::btrfs_bio_clone() for an example.  We
> >>> probably should introduce a wrapper for this so that each site doesn't
> >>> need to ifdef it.
> >>>
> >>> Thanks.
> >>
> >> OK, I think this should do it.  Nikolay and/or others can you test this
> >> patch using blkio cgroups controller with LVM devices and report back?
> >>
> >> From: Mike Snitzer 
> >> Date: Wed, 2 Mar 2016 12:37:39 -0500
> >> Subject: [PATCH] block: transfer source bio's cgroup tags to clone via 
> >> bio_associate_blkcg()
> >>
> >> Move btrfs_bio_clone()'s support for transferring a source bio's cgroup
> >> tags to a clone into both bio_clone_bioset() and __bio_clone_fast().
> >> The former is used by btrfs (MD and blk-core also use it via bio_split).
> >> The latter is used by both DM and bcache.
> >>
> >> This should enable the blkio cgroups controller to work with all
> >> stacking bio-based block devices.
> >>
> >> Reported-by: Nikolay Borisov 
> >> Suggested-by: Tejun Heo 
> >> Signed-off-by: Mike Snitzer 
> >> ---
> >>  block/bio.c  | 10 ++
> >>  fs/btrfs/extent_io.c |  6 --
> >>  2 files changed, 10 insertions(+), 6 deletions(-)
> > 
> > 
> > So I had a chance to test the settings here is what I got when running 
> > 2 container, using LVM-thin for their root device and having applied 
> > your patch: 
> > 
> > When the 2 containers are using the same blkio.weight values (500) I 
> > get the following from running DD simultaneously on the 2 containers: 
> > 
> > [root@c1501 ~]# dd if=test.img of=test2.img bs=1M count=3000 oflag=direct
> > 3000+0 records in
> > 3000+0 records out
> > 3145728000 bytes (3.1 GB) copied, 165.171 s, 19.0 MB/s
> > 
> > [root@c1500 ~]# dd if=test.img of=test2.img bs=1M count=3000 oflag=direct
> > 3000+0 records in
> > 3000+0 records out
> > 3145728000 bytes (3.1 GB) copied, 166.165 s, 18.9 MB/s
> > 
> > Also iostat showed the 2 volumes using almost the same amount of 
> > IO (around 20mb r/w). I then increase the weight for c1501 to 1000 i.e. 
> > twice the bandwidth that c1500 has, so I would expect its dd to complete
> > twice as fast: 
> > 
> > [root@c1501 ~]# dd if=test.img of=test2.img bs=1M count=3000 oflag=direct
> > 3000+0 records in
> > 3000+0 records out
> > 3145728000 bytes (3.1 GB) copied, 150.892 s, 20.8 MB/s
> > 
> > 
> > [root@c1500 ~]# dd if=test.img of=test2.img bs=1M count=3000 oflag=direct
> > 3000+0 records in
> > 3000+0 records out
> > 3145728000 bytes (3.1 GB) copied, 157.167 s, 20.0 MB/s
> > 
> > Now repeating the same tests but this time using the page-cache 
> > (echo 3 > /proc/sys/vm/drop_caches) was executed before each test run: 
> > 
> > With equal weights (500):
> > [root@c1501 ~]# dd if=test.img of=test2.img bs=1M count=3000
> > 3000+0 records in
> > 3000+0 records out
> > 3145728000 bytes (3.1 GB) copied, 114.923 s, 27.4 MB/s
> > 
> > [root@c1500 ~]# dd if=test.img of=test2.img bs=1M count=3000
> > 3000+0 records in
> > 3000+0 records out
> > 3145728000 bytes (3.1 GB) copied, 120.245 s, 26.2 MB/s
> > 
> > With (c1501's weight equal to twice that of c1500 (1000)):
> > 
> > [root@c1501 ~]# dd if=test.img of=test2.img bs=1M count=3000
> > 3000+0 records in
> > 3000+0 records out
> >

Re: [dm-devel] [PATCH] block: transfer source bio's cgroup tags to clone via bio_associate_blkcg()

2016-03-14 Thread Nikolay Borisov


On 03/14/2016 05:08 PM, Nikolay Borisov wrote:
> 
> 
> On 03/02/2016 07:56 PM, Mike Snitzer wrote:
>> On Wed, Mar 02 2016 at 11:06P -0500,
>> Tejun Heo  wrote:
>>
>>> Hello,
>>>
>>> On Thu, Feb 25, 2016 at 09:53:14AM -0500, Mike Snitzer wrote:
 Right, LVM created devices are bio-based DM devices in the kernel.
 bio-based block devices do _not_ have an IO scheduler.  Their underlying
 request-based device does.
>>>
>>> dm devices are not the actual resource source, so I don't think it'd
>>> work too well to put io controllers on them (can't really do things
>>> like proportional control without owning the queue).
>>>
 I'm not well-versed on the top-level cgroup interface and how it maps to
 associated resources that are established in the kernel.  But it could
 be that the configuration of blkio cgroup against a bio-based LVM device
 needs to be passed through to the underlying request-based device
 (e.g. /dev/sda4 in Chris's case)?

 I'm also wondering whether the latest cgroup work that Tejun has just
 finished (afaik to support buffered IO in the IO controller) will afford
 us a more meaningful reason to work to make cgroups' blkio controller
 actually work with bio-based devices like LVM's DM devices?

 I'm very much open to advice on how to proceed with investigating this
 integration work.  Tejun, Vivek, anyone else: if you have advice on next
 steps for DM on this front _please_ yell, thanks!
>>>
>>> I think the only thing necessary is dm transferring bio cgroup tags to
>>> the bio's that it ends up passing down the stack.  Please take a look
>>> at fs/btrfs/extent_io.c::btrfs_bio_clone() for an example.  We
>>> probably should introduce a wrapper for this so that each site doesn't
>>> need to ifdef it.
>>>
>>> Thanks.
>>
>> OK, I think this should do it.  Nikolay and/or others can you test this
>> patch using blkio cgroups controller with LVM devices and report back?
>>
>> From: Mike Snitzer 
>> Date: Wed, 2 Mar 2016 12:37:39 -0500
>> Subject: [PATCH] block: transfer source bio's cgroup tags to clone via 
>> bio_associate_blkcg()
>>
>> Move btrfs_bio_clone()'s support for transferring a source bio's cgroup
>> tags to a clone into both bio_clone_bioset() and __bio_clone_fast().
>> The former is used by btrfs (MD and blk-core also use it via bio_split).
>> The latter is used by both DM and bcache.
>>
>> This should enable the blkio cgroups controller to work with all
>> stacking bio-based block devices.
>>
>> Reported-by: Nikolay Borisov 
>> Suggested-by: Tejun Heo 
>> Signed-off-by: Mike Snitzer 
>> ---
>>  block/bio.c  | 10 ++
>>  fs/btrfs/extent_io.c |  6 --
>>  2 files changed, 10 insertions(+), 6 deletions(-)
> 
> 
> So I had a chance to test the settings here is what I got when running 
> 2 container, using LVM-thin for their root device and having applied 
> your patch: 
> 
> When the 2 containers are using the same blkio.weight values (500) I 
> get the following from running DD simultaneously on the 2 containers: 
> 
> [root@c1501 ~]# dd if=test.img of=test2.img bs=1M count=3000 oflag=direct
> 3000+0 records in
> 3000+0 records out
> 3145728000 bytes (3.1 GB) copied, 165.171 s, 19.0 MB/s
> 
> [root@c1500 ~]# dd if=test.img of=test2.img bs=1M count=3000 oflag=direct
> 3000+0 records in
> 3000+0 records out
> 3145728000 bytes (3.1 GB) copied, 166.165 s, 18.9 MB/s
> 
> Also iostat showed the 2 volumes using almost the same amount of 
> IO (around 20mb r/w). I then increase the weight for c1501 to 1000 i.e. 
> twice the bandwidth that c1500 has, so I would expect its dd to complete
> twice as fast: 
> 
> [root@c1501 ~]# dd if=test.img of=test2.img bs=1M count=3000 oflag=direct
> 3000+0 records in
> 3000+0 records out
> 3145728000 bytes (3.1 GB) copied, 150.892 s, 20.8 MB/s
> 
> 
> [root@c1500 ~]# dd if=test.img of=test2.img bs=1M count=3000 oflag=direct
> 3000+0 records in
> 3000+0 records out
> 3145728000 bytes (3.1 GB) copied, 157.167 s, 20.0 MB/s
> 
> Now repeating the same tests but this time using the page-cache 
> (echo 3 > /proc/sys/vm/drop_caches) was executed before each test run: 
> 
> With equal weights (500):
> [root@c1501 ~]# dd if=test.img of=test2.img bs=1M count=3000
> 3000+0 records in
> 3000+0 records out
> 3145728000 bytes (3.1 GB) copied, 114.923 s, 27.4 MB/s
> 
> [root@c1500 ~]# dd if=test.img of=test2.img bs=1M count=3000
> 3000+0 records in
> 3000+0 records out
> 3145728000 bytes (3.1 GB) copied, 120.245 s, 26.2 MB/s
> 
> With (c1501's weight equal to twice that of c1500 (1000)):
> 
> [root@c1501 ~]# dd if=test.img of=test2.img bs=1M count=3000
> 3000+0 records in
> 3000+0 records out
> 3145728000 bytes (3.1 GB) copied, 99.0181 s, 31.8 MB/s
> 
> [root@c1500 ~]# dd if=test.img of=test2.img bs=1M count=3000
> 3000+0 records in
> 3000+0 records out
> 3145728000 bytes (3.1 GB) copied, 122.872 s, 25.6 MB/s

And another test which makes it obvious that your patch works:

[root@c1501 ~]# dd i

Re: [dm-devel] [PATCH] block: transfer source bio's cgroup tags to clone via bio_associate_blkcg()

2016-03-14 Thread Nikolay Borisov


On 03/02/2016 07:56 PM, Mike Snitzer wrote:
> On Wed, Mar 02 2016 at 11:06P -0500,
> Tejun Heo  wrote:
> 
>> Hello,
>>
>> On Thu, Feb 25, 2016 at 09:53:14AM -0500, Mike Snitzer wrote:
>>> Right, LVM created devices are bio-based DM devices in the kernel.
>>> bio-based block devices do _not_ have an IO scheduler.  Their underlying
>>> request-based device does.
>>
>> dm devices are not the actual resource source, so I don't think it'd
>> work too well to put io controllers on them (can't really do things
>> like proportional control without owning the queue).
>>
>>> I'm not well-versed on the top-level cgroup interface and how it maps to
>>> associated resources that are established in the kernel.  But it could
>>> be that the configuration of blkio cgroup against a bio-based LVM device
>>> needs to be passed through to the underlying request-based device
>>> (e.g. /dev/sda4 in Chris's case)?
>>>
>>> I'm also wondering whether the latest cgroup work that Tejun has just
>>> finished (afaik to support buffered IO in the IO controller) will afford
>>> us a more meaningful reason to work to make cgroups' blkio controller
>>> actually work with bio-based devices like LVM's DM devices?
>>>
>>> I'm very much open to advice on how to proceed with investigating this
>>> integration work.  Tejun, Vivek, anyone else: if you have advice on next
>>> steps for DM on this front _please_ yell, thanks!
>>
>> I think the only thing necessary is dm transferring bio cgroup tags to
>> the bio's that it ends up passing down the stack.  Please take a look
>> at fs/btrfs/extent_io.c::btrfs_bio_clone() for an example.  We
>> probably should introduce a wrapper for this so that each site doesn't
>> need to ifdef it.
>>
>> Thanks.
> 
> OK, I think this should do it.  Nikolay and/or others can you test this
> patch using blkio cgroups controller with LVM devices and report back?
> 
> From: Mike Snitzer 
> Date: Wed, 2 Mar 2016 12:37:39 -0500
> Subject: [PATCH] block: transfer source bio's cgroup tags to clone via 
> bio_associate_blkcg()
> 
> Move btrfs_bio_clone()'s support for transferring a source bio's cgroup
> tags to a clone into both bio_clone_bioset() and __bio_clone_fast().
> The former is used by btrfs (MD and blk-core also use it via bio_split).
> The latter is used by both DM and bcache.
> 
> This should enable the blkio cgroups controller to work with all
> stacking bio-based block devices.
> 
> Reported-by: Nikolay Borisov 
> Suggested-by: Tejun Heo 
> Signed-off-by: Mike Snitzer 
> ---
>  block/bio.c  | 10 ++
>  fs/btrfs/extent_io.c |  6 --
>  2 files changed, 10 insertions(+), 6 deletions(-)


So I had a chance to test the settings here is what I got when running 
2 container, using LVM-thin for their root device and having applied 
your patch: 

When the 2 containers are using the same blkio.weight values (500) I 
get the following from running DD simultaneously on the 2 containers: 

[root@c1501 ~]# dd if=test.img of=test2.img bs=1M count=3000 oflag=direct
3000+0 records in
3000+0 records out
3145728000 bytes (3.1 GB) copied, 165.171 s, 19.0 MB/s

[root@c1500 ~]# dd if=test.img of=test2.img bs=1M count=3000 oflag=direct
3000+0 records in
3000+0 records out
3145728000 bytes (3.1 GB) copied, 166.165 s, 18.9 MB/s

Also iostat showed the 2 volumes using almost the same amount of 
IO (around 20mb r/w). I then increase the weight for c1501 to 1000 i.e. 
twice the bandwidth that c1500 has, so I would expect its dd to complete
twice as fast: 

[root@c1501 ~]# dd if=test.img of=test2.img bs=1M count=3000 oflag=direct
3000+0 records in
3000+0 records out
3145728000 bytes (3.1 GB) copied, 150.892 s, 20.8 MB/s


[root@c1500 ~]# dd if=test.img of=test2.img bs=1M count=3000 oflag=direct
3000+0 records in
3000+0 records out
3145728000 bytes (3.1 GB) copied, 157.167 s, 20.0 MB/s

Now repeating the same tests but this time using the page-cache 
(echo 3 > /proc/sys/vm/drop_caches) was executed before each test run: 

With equal weights (500):
[root@c1501 ~]# dd if=test.img of=test2.img bs=1M count=3000
3000+0 records in
3000+0 records out
3145728000 bytes (3.1 GB) copied, 114.923 s, 27.4 MB/s

[root@c1500 ~]# dd if=test.img of=test2.img bs=1M count=3000
3000+0 records in
3000+0 records out
3145728000 bytes (3.1 GB) copied, 120.245 s, 26.2 MB/s

With (c1501's weight equal to twice that of c1500 (1000)):

[root@c1501 ~]# dd if=test.img of=test2.img bs=1M count=3000
3000+0 records in
3000+0 records out
3145728000 bytes (3.1 GB) copied, 99.0181 s, 31.8 MB/s

[root@c1500 ~]# dd if=test.img of=test2.img bs=1M count=3000
3000+0 records in
3000+0 records out
3145728000 bytes (3.1 GB) copied, 122.872 s, 25.6 MB/s

I'd say that for buffered IO your patch does indeed make a difference, 
and this sort of aligns with what Vivek said about the patch
working for buffered writes but not for direct. 

I will proceed now and test his patch applied for the case of 
direct writes. 

Hope this helps. 


--
dm-devel mailing list
dm-de

Re: [dm-devel] dm-cache: blocks don't get cached on 3.18.21-17.el6.x86_64

2016-03-14 Thread Thanos Makatos
On 14 March 2016 at 12:44, Joe Thornber  wrote:
> On Mon, Mar 14, 2016 at 09:54:06AM +, Thanos Makatos wrote:
>> (I've already reported this issue to centos and centos-devel. and
>> waited long enough but didn't get any reply.)
>>
>> I'm evaluating dm-cache on CentOS 6 kernels  3.18.21-17.el6.x86_64 (Xen 4) 
>> and
>> 2.6.32-573.7.1.el6.x86_64 (KVM). The test I do is a simple sequential
>> read using dd(1). read_promote_adjustment and sequential_threshold
>> have been set to 1 and 0, respectively. For the 2.6.32 kernel, all
>> seems to be working fine, "#usedcacheblocks" correctly reflects the
>> number of cached blocks based on what I'm reading with dd(1),
>> performance is pretty much native SSD performance. However, the same
>> test on the 3.18.21-17.el6.x86_64 kernel results in "#usedcacheblocks"
>> being stuck to "2" and no performance improvement is observed.
>>
>> Any ideas what could be wrong? How can I further debug this?
>
> There may not be anything wrong, dm-cache is very cagey these days
> about promoting blocks to the ssd without evidence that they're
> hotspots.  Hitting blocks once with a dd is not enough.

I've been hitting the same block many many times, still it doesn't get
promoted. Is there a foolproof way that results in blocks getting
cached?

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel


Re: [dm-devel] dm-cache: blocks don't get cached on 3.18.21-17.el6.x86_64

2016-03-14 Thread Joe Thornber
On Mon, Mar 14, 2016 at 09:54:06AM +, Thanos Makatos wrote:
> (I've already reported this issue to centos and centos-devel. and
> waited long enough but didn't get any reply.)
> 
> I'm evaluating dm-cache on CentOS 6 kernels  3.18.21-17.el6.x86_64 (Xen 4) and
> 2.6.32-573.7.1.el6.x86_64 (KVM). The test I do is a simple sequential
> read using dd(1). read_promote_adjustment and sequential_threshold
> have been set to 1 and 0, respectively. For the 2.6.32 kernel, all
> seems to be working fine, "#usedcacheblocks" correctly reflects the
> number of cached blocks based on what I'm reading with dd(1),
> performance is pretty much native SSD performance. However, the same
> test on the 3.18.21-17.el6.x86_64 kernel results in "#usedcacheblocks"
> being stuck to "2" and no performance improvement is observed.
> 
> Any ideas what could be wrong? How can I further debug this?

There may not be anything wrong, dm-cache is very cagey these days
about promoting blocks to the ssd without evidence that they're
hotspots.  Hitting blocks once with a dd is not enough.

- Joe

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel


[dm-devel] patch:multipath: implement option '-u' for uevents

2016-03-14 Thread Zhangguanghui
This patch implements an option '-u' which uses the information from the 
program environment when checking the device.
the function don't be found some problems.
Whiling using the command 'udevadm test /sys/block/sdb', some error logs, such 
as/sbin/multipath -u sdb'(err)
So I don't know what is that?  Is it a potential risk ? Thanks a lots.

udev_rules_apply_to_event: PROGRAM '/sbin/multipath -u sdb' 
/lib/udev/rules.d/95-multipath.rules:20
udev_event_spawn: starting '/sbin/multipath -u sdb'
spawn_read: '/sbin/multipath -u sdb'(err) 'libudev: 
udev_device_new_from_syspath: device 0x55ee912a0b50 has devpath 
'/devices/platform/host72/session70/target72:0:0/72:0:0:0''
spawn_read: '/sbin/multipath -u sdb'(err) 'libudev: 
udev_device_new_from_syspath: device 0x55ee912a1ee0 has devpath 
'/devices/platform/host72/session70/target72:0:0''
spawn_read: '/sbin/multipath -u sdb'(err) 'libudev: 
udev_device_new_from_syspath: device 0x55ee912a2980 has devpath 
'/devices/platform/host72/session70''
spawn_read: '/sbin/multipath -u sdb'(err) 'libudev: 
udev_device_new_from_syspath: device 0x55ee912a1b70 has devpath 
'/devices/platform/host72''
spawn_read: '/sbin/multipath -u sdb'(err) 'libudev: 
udev_device_new_from_syspath: device 0x55ee912a4230 has devpath 
'/devices/platform''
spawn_read: '/sbin/multipath -u sdb'(err) 'libudev: 
udev_device_new_from_syspath: device 0x55ee912a2f90 has devpath 
'/devices/platform/host72/session70/iscsi_session/session70''
spawn_read: '/sbin/multipath -u sdb'(out) 'sdb is a valid multipath device path'
spawn_wait: '/sbin/multipath -u sdb' [42931] exit with return code 0



zhangguanghui
-
本邮件及其附件含有杭州华三通信技术有限公司的保密信息,仅限于发送给上面地址中列出
的个人或群组。禁止任何其他人以任何形式使用(包括但不限于全部或部分地泄露、复制、
或散发)本邮件中的信息。如果您错收了本邮件,请您立即电话或邮件通知发件人并删除本
邮件!
This e-mail and its attachments contain confidential information from H3C, 
which is
intended only for the person or entity whose address is listed above. Any use 
of the
information contained herein in any way (including, but not limited to, total 
or partial
disclosure, reproduction, or dissemination) by persons other than the intended
recipient(s) is prohibited. If you receive this e-mail in error, please notify 
the sender
by phone or email immediately and delete it!
--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel

[dm-devel] dm-cache: blocks don't get cached on 3.18.21-17.el6.x86_64

2016-03-14 Thread Thanos Makatos
(I've already reported this issue to centos and centos-devel. and
waited long enough but didn't get any reply.)

I'm evaluating dm-cache on CentOS 6 kernels  3.18.21-17.el6.x86_64 (Xen 4) and
2.6.32-573.7.1.el6.x86_64 (KVM). The test I do is a simple sequential
read using dd(1). read_promote_adjustment and sequential_threshold
have been set to 1 and 0, respectively. For the 2.6.32 kernel, all
seems to be working fine, "#usedcacheblocks" correctly reflects the
number of cached blocks based on what I'm reading with dd(1),
performance is pretty much native SSD performance. However, the same
test on the 3.18.21-17.el6.x86_64 kernel results in "#usedcacheblocks"
being stuck to "2" and no performance improvement is observed.

Any ideas what could be wrong? How can I further debug this?

-- 
Thanos Makatos

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel