Re: [dm-devel] extracting thin mappings in real time
> Instead I gave a second thought to the solution of using dm-thin's > public functions to access it's metadata, namely dm_thin_find_block. > The problem is that if the mapping is not in memory I'll have to read > it from disk, outside the critical path, and this adds complexity. > However, since the mapping was just established it should be cached so > it should mostly work (not being able to obtain the mapping every now > and then is acceptable). Does this make sense? I call dm_thin_find_block right after I've received the write completion and if I don't set can_issue_io to 1 it _always_ fails. If I do set it to 1 then it always succeeds but reads 20 KB from the metadata device, even when repeatedly called. I can see that internally it uses dm-bufio, which is supposed to some caching, no? Am I missing something? -- dm-devel mailing list dm-devel@redhat.com https://www.redhat.com/mailman/listinfo/dm-devel
Re: [dm-devel] extracting thin mappings in real time
On Thu, 4 Oct 2018 at 13:56, Joe Thornber wrote: > > On Wed, Oct 03, 2018 at 04:47:41PM +0100, Thanos Makatos wrote: > > poo metadata object can return this information? I've started looking > > at thin_bio_map(), is this the best place to start? > > See thin-metadata.h > > - Joe I started looking at piggybacking the mapping in bio->bi_pivate in provision_block and it looks ugly. First, I'll have to introduce a new bio flag specifically for this reason: whenever dm-thin sees this it should store the mapping in the bio->bi_private. Second, the bio passed to dm-thin is cloned so I'll have to find the original bio (this requires making struct dm_io public). And third, the original bio can be split to multiple cloned bios so the structure in bio->bi_pivate should accomodate this. Instead I gave a second thought to the solution of using dm-thin's public functions to access it's metadata, namely dm_thin_find_block. The problem is that if the mapping is not in memory I'll have to read it from disk, outside the critical path, and this adds complexity. However, since the mapping was just established it should be cached so it should mostly work (not being able to obtain the mapping every now and then is acceptable). Does this make sense? -- dm-devel mailing list dm-devel@redhat.com https://www.redhat.com/mailman/listinfo/dm-devel
Re: [dm-devel] extracting thin mappings in real time
On Wed, Oct 03, 2018 at 04:47:41PM +0100, Thanos Makatos wrote: > poo metadata object can return this information? I've started looking > at thin_bio_map(), is this the best place to start? See thin-metadata.h - Joe -- dm-devel mailing list dm-devel@redhat.com https://www.redhat.com/mailman/listinfo/dm-devel
[dm-devel] extracting thin mappings in real time
I have a kernel module that sits on top of a thin device mapper target that receives block I/O requests and re-submits then to the thin target. I would like to implement the following functionality: whenever I receive a write completion from the thin target (assuming that it's the first time a block written to) I would like to extract the newly-established mapping of that virtual block. I know that I can do this using thin_dump, however this involves: (1) spawning a process (2) reserving/releasing a metadata snapshot, and (3) dumping _all_ the mappings. In other words, it's far to heavyweight for my performance requirements. Ideally I would like to be able to obtain the mapping in kernel space. I had a look at thin_dump and from what I understand it directly reads the B-tree from the disk? Is there some kernel function that already does this? E.g. given a thin LBA return the physical block address. Also, regarding having to have reserved a metadata snapshot, is this necessary for obtaining mappings? Aren't mappings immutable once established ? -- Thanos Makatos -- dm-devel mailing list dm-devel@redhat.com https://www.redhat.com/mailman/listinfo/dm-devel
Re: [dm-devel] extracting thin mappings in real time
On Wed, Oct 3, 2018 at 3:47 PM Joe Thornber wrote: > > On Wed, Oct 03, 2018 at 03:13:36PM +0100, Thanos Makatos wrote: > > > Could you say more about why you want to do this? > > > > > > > So that I can directly read the data block without having to pass through > > dm-thin, e.g. there might be a more direct datapath to the physical block > > device. > > > > Mappings don't change if you're not using snapshots. If you are, then any > > > write > > > to a shared block will trigger a copy on write operation, and subsequently > > > a new > > > mapping. > > > > So for backups etc. I made the metadata snapshot available, which gives you > > > a > > > view of all the metadata frozen at a point in time. If you're using the > > > metadata > > > snap you still need to guarantee the mappings for the devices that you're > > > interested > > > in are not changing. eg, instead of backing up a live thin, take a > > > snapshot of the thin > > > and back this up instead. > > > > > > > For now I can guarantee that these mappings don't change (no snapshots, > > backups, etc), so it's safe to extract the mapping(s) without having > > acquired the meradata snapshot, correct? If this is the case, then how can > > I extract individual mappins? (Either from kernel space or from user > > space.) Are there specific dm-thin kernel functions I should start looking > > at? > > Why is this thin-specific? What happens if there are two thin-pools under > your code? Do you want the final physical location? The whole dm stack? You're right, this isn't thin-specific. Currently there will be only one thin pool per physical device so it's simple enough. Indeed I want the physical location, knowing the location in the data LV would suffice. >> Efficiently reading the metadata requires a cache of metadata blocks. Thinp >> uses dm-bufio > to provide this. It seems very wasteful for you to duplicate this. So your > options are: > > > i) querying the pool metadata object directly. Currently there is no way to > get hold of the >metadata object, you would need to make any queries off the io path, since > they can block. Do you mean that the pool metadata object is internal to dm-thin so that other kernel modules cannot access it? If so then I suppose I can add functions do dm-thin that return an opaque pointer to it (or something like that). Is there some specific function that given the poo metadata object can return this information? I've started looking at thin_bio_map(), is this the best place to start? Also, is option (i) doable from userspace? Is there logic in thin_dump to access the on-disk metadata? I suppose I will have to take care when reading the metadata as they might not have been committed to disk. >> ii) or examining the completed bios, you'd probably need to add some per bio >> data to give context. I think option (ii) sounds like the most efficient one but most invasive. -- dm-devel mailing list dm-devel@redhat.com https://www.redhat.com/mailman/listinfo/dm-devel
Re: [dm-devel] extracting thin mappings in real time
On Wed, Oct 03, 2018 at 03:13:36PM +0100, Thanos Makatos wrote: > > Could you say more about why you want to do this? > > > > So that I can directly read the data block without having to pass through > dm-thin, e.g. there might be a more direct datapath to the physical block > device. > > Mappings don't change if you're not using snapshots. If you are, then any > > write > > to a shared block will trigger a copy on write operation, and subsequently > > a new > > mapping. > > So for backups etc. I made the metadata snapshot available, which gives you > > a > > view of all the metadata frozen at a point in time. If you're using the > > metadata > > snap you still need to guarantee the mappings for the devices that you're > > interested > > in are not changing. eg, instead of backing up a live thin, take a > > snapshot of the thin > > and back this up instead. > > > > For now I can guarantee that these mappings don't change (no snapshots, > backups, etc), so it's safe to extract the mapping(s) without having > acquired the meradata snapshot, correct? If this is the case, then how can > I extract individual mappins? (Either from kernel space or from user > space.) Are there specific dm-thin kernel functions I should start looking > at? Why is this thin-specific? What happens if there are two thin-pools under your code? Do you want the final physical location? The whole dm stack? Efficiently reading the metadata requires a cache of metadata blocks. Thinp uses dm-bufio to provide this. It seems very wasteful for you to duplicate this. So your options are: i) querying the pool metadata object directly. Currently there is no way to get hold of the metadata object, you would need to make any queries off the io path, since they can block. ii) or examining the completed bios, you'd probably need to add some per bio data to give context. - Joe -- dm-devel mailing list dm-devel@redhat.com https://www.redhat.com/mailman/listinfo/dm-devel
Re: [dm-devel] extracting thin mappings in real time
> Could you say more about why you want to do this? > So that I can directly read the data block without having to pass through dm-thin, e.g. there might be a more direct datapath to the physical block device. Mappings don't change if you're not using snapshots. If you are, then any > write > to a shared block will trigger a copy on write operation, and subsequently > a new > mapping. So for backups etc. I made the metadata snapshot available, which gives you > a > view of all the metadata frozen at a point in time. If you're using the > metadata > snap you still need to guarantee the mappings for the devices that you're > interested > in are not changing. eg, instead of backing up a live thin, take a > snapshot of the thin > and back this up instead. > For now I can guarantee that these mappings don't change (no snapshots, backups, etc), so it's safe to extract the mapping(s) without having acquired the meradata snapshot, correct? If this is the case, then how can I extract individual mappins? (Either from kernel space or from user space.) Are there specific dm-thin kernel functions I should start looking at? -- dm-devel mailing list dm-devel@redhat.com https://www.redhat.com/mailman/listinfo/dm-devel
Re: [dm-devel] extracting thin mappings in real time
On Wed, Oct 03, 2018 at 01:40:22PM +0100, Thanos Makatos wrote: > I have a kernel module that sits on top of a thin device mapper target that > receives block I/O requests and re-submits then to the thin target. I would > like to implement the following functionality: whenever I receive a write > completion from the thin target (assuming that it's the first time a block > written to) I would like to extract the newly-established mapping of that > virtual block. > > I know that I can do this using thin_dump, however this involves: > (1) spawning a process > (2) reserving/releasing a metadata snapshot, and > (3) dumping _all_ the mappings. > > In other words, it's far to heavyweight for my performance requirements. > > Ideally I would like to be able to obtain the mapping in kernel space. I > had a look at thin_dump and from what I understand it directly reads the > B-tree from the disk? Is there some kernel function that already does this? > E.g. given a thin LBA return the physical block address. > > Also, regarding having to have reserved a metadata snapshot, is this > necessary for obtaining mappings? Aren't mappings immutable once established > ? Could you say more about why you want to do this? Mappings don't change if you're not using snapshots. If you are, then any write to a shared block will trigger a copy on write operation, and subsequently a new mapping. So for backups etc. I made the metadata snapshot available, which gives you a view of all the metadata frozen at a point in time. If you're using the metadata snap you still need to guarantee the mappings for the devices that you're interested in are not changing. eg, instead of backing up a live thin, take a snapshot of the thin and back this up instead. - Joe -- dm-devel mailing list dm-devel@redhat.com https://www.redhat.com/mailman/listinfo/dm-devel