Re: ext4 extents: How to determine if an extent points to a hole.
Hi Greg, Thanks a lot for such nice explanation. But I still have the same query lingering... On Mon, Sep 14, 2009 at 9:36 AM, Greg Freemyer wrote: > On Sun, Sep 13, 2009 at 1:49 PM, SandeepKsinha > wrote: >> On Sun, Sep 13, 2009 at 11:08 PM, Manish Katiyar wrote: >>> On Sun, Sep 13, 2009 at 10:12 PM, SandeepKsinha >>> wrote: Hi, On Sun, Sep 13, 2009 at 10:07 PM, Manish Katiyar wrote: > On Sun, Sep 13, 2009 at 5:44 PM, SandeepKsinha > wrote: >> Hi all, >> >> Looking at the flags in the extent info, Is there any specifc flags >> which indicates an extent to be a HOLE?? > > I am not sure if I understand the question correctly .. why would > you need that ? Can you give an example where it should be used ?? > Look at e4defrag.c, it checks the file size and allocates the same number of blocks for donor inode. Which will eventually make a holey file into a normal one. Any tool/application should make sure that they leave a sparse file as sparse. I think, as suggested by Greg Freemyer, we can use BMAP ioctl to get such information. >>> >>> Yes, but I think bmap would be costly if the file is large and is not >>> holey :-( but that would be probably same calling fiemap if the >>> file is completely fragmented such that each extent size is 1. >>> >> >> Since, ext2/ext3 did not have mutli block allocation thats why this is >> the only way that we might have. >> But generally most of the new features work on with extent based files on >> ext4. >> >> I am still wondering that how to we represent a hole using extents in >> a extent based file. >> Just like we had a convention of having the block number 0 in case of holes. >> >> Similarly, what do we look at to figure out if its a hole or not. BMAP >> is one way. But since, in a extent based file, we have only extents, >> there should be some flag to indicate the same. > > Sandeep, > > If you look at e4defrag, it first gets a list of all the extents. I'm > pretty sure extents only exist for allocated extents. Holes do not > have any associated extents. > This is what my actual question is, think of this situation... |-|-|-|--| 0 500 7001200 3000 [Logical block numbers for an inode] In this situation you will have four extents for sure. ext_1 -> 0 -- 500 ext_2 -> 501 --700 [ This will be an initialized extent] ext_3 -> 701 -- 1200 ext_4 -> 1200 -- 3000 After looking at the sources and some comments in the ext4 source code, I could figure out that holes would be having an initialized extent. Reference: http://lxr.linux.no/#linux+v2.6.31/fs/ext4/extents.c#L2843 I think we cannot have a mixture of both a BMAP and an EMAP, it will be either. > Then it calls merge_extents to create extent groups. In e4defrag > terminology, an extent group is a collection of all the logically > contiguous extents. I don't know if the kernel uses that terminology > or not. > Hope they are not merging together any initialized and uninitialized extents together, since they can be logically contiguous. Or rather they will be. > In other words in e4defrag terminology a sparse file is a series of: > > extent group - hole - extent group - hole - extent group - etc. > > Then e4defrag creates a donor file with exactly the same allocated > block areas by calling fallocate on the donor file for each extent > group with the same starting offset and length as the extent group. > This is true and should be applicable to initialized extents as well. I fear if they are > Thus the donor file ends up have exactly the same holes as the > original file. Then the donor blocks are used to defrag the original > file by calling move_extent. In the kernel, the move_extent logic > looks for holes and only replaces blocks that are allocated in the > original file. > This is true. I am sure of the kernel logic. > Greg > -- Regards, Sandeep. “To learn is to change. Education is a process that changes the learner.” -- To unsubscribe from this list: send an email with "unsubscribe kernelnewbies" to ecar...@nl.linux.org Please read the FAQ at http://kernelnewbies.org/FAQ
Modules required for booting without initrd
Hi, I want to boot my laptop without initrd, I tried to compile 2.6.30 kernel from debian back port repository, by building ext3 driver into kernel but during booting i am getting "vfs: could not mount root fs unkown-block(8,1)". My menu.lst: title Debian GNU/Linux, kernel 2.6.30-130909f-we-test root (hd0,0) kernel /boot/vmlinuz-2.6.30-130909f-we-test root=/dev/sda1 ro fdisk -l: http://paste.debian.net/46476/ lsmod: http://paste.debian.net/46474/ lspci: http://paste.debian.net/46475/ What are the modules should be loaded into the kernel to boot without initrd? and how to find them? Thanks, Alagunambi Welkin
Re: Linux Podcast: http://www.linuxreality.com/archives.php
Hello, Peter and all There is also another resource about Linux podcast. Linux Journal Podcast: http://podcast.com/show/55601/ There are a lot of episode for beginners. Enjoy yourself. On Sat, Sep 12, 2009 at 1:18 AM, Peter Teoh wrote: > Just sharing some podcasting: > > > http://www.linuxreality.com/archives.php > > this is new to me.. > > but others which u may already know: > > http://podcasts.jonmasters.org/kernel/kernel.xml (this is heavily > downloaded and very popular). > > http://www.thelinuxlink.net/ > > Timesys Podcast: > > https://linuxlink.timesys.com/3/podcast (interface is better than > the above link, but content wise not sure if there is any overlapping) > > > > -- > Regards, > Peter Teoh > > -- > To unsubscribe from this list: send an email with > "unsubscribe kernelnewbies" to ecar...@nl.linux.org > Please read the FAQ at http://kernelnewbies.org/FAQ > > -- Adam Jiang -- e-mail:jiang.a...@gmail.com http://www.adamjiang.com/wiki/
Re: ext4 extents: How to determine if an extent points to a hole.
On Sun, Sep 13, 2009 at 1:49 PM, SandeepKsinha wrote: > On Sun, Sep 13, 2009 at 11:08 PM, Manish Katiyar wrote: >> On Sun, Sep 13, 2009 at 10:12 PM, SandeepKsinha >> wrote: >>> Hi, >>> >>> >>> On Sun, Sep 13, 2009 at 10:07 PM, Manish Katiyar wrote: On Sun, Sep 13, 2009 at 5:44 PM, SandeepKsinha wrote: > Hi all, > > Looking at the flags in the extent info, Is there any specifc flags > which indicates an extent to be a HOLE?? I am not sure if I understand the question correctly .. why would you need that ? Can you give an example where it should be used ?? >>> >>> Look at e4defrag.c, >>> it checks the file size and allocates the same number of blocks for >>> donor inode. Which will eventually make a holey file into a normal >>> one. >>> Any tool/application should make sure that they leave a sparse file as >>> sparse. >>> >>> I think, as suggested by Greg Freemyer, we can use BMAP ioctl to get >>> such information. >> >> Yes, but I think bmap would be costly if the file is large and is not >> holey :-( but that would be probably same calling fiemap if the >> file is completely fragmented such that each extent size is 1. >> > > Since, ext2/ext3 did not have mutli block allocation thats why this is > the only way that we might have. > But generally most of the new features work on with extent based files on > ext4. > > I am still wondering that how to we represent a hole using extents in > a extent based file. > Just like we had a convention of having the block number 0 in case of holes. > > Similarly, what do we look at to figure out if its a hole or not. BMAP > is one way. But since, in a extent based file, we have only extents, > there should be some flag to indicate the same. Sandeep, If you look at e4defrag, it first gets a list of all the extents. I'm pretty sure extents only exist for allocated extents. Holes do not have any associated extents. Then it calls merge_extents to create extent groups. In e4defrag terminology, an extent group is a collection of all the logically contiguous extents. I don't know if the kernel uses that terminology or not. In other words in e4defrag terminology a sparse file is a series of: extent group - hole - extent group - hole - extent group - etc. Then e4defrag creates a donor file with exactly the same allocated block areas by calling fallocate on the donor file for each extent group with the same starting offset and length as the extent group. Thus the donor file ends up have exactly the same holes as the original file. Then the donor blocks are used to defrag the original file by calling move_extent. In the kernel, the move_extent logic looks for holes and only replaces blocks that are allocated in the original file. Greg -- To unsubscribe from this list: send an email with "unsubscribe kernelnewbies" to ecar...@nl.linux.org Please read the FAQ at http://kernelnewbies.org/FAQ
Re: e4defrag: file extent map : redundant loop???
Hi,
Manish Katiyar wrote:
> On Sun, Sep 13, 2009 at 5:26 PM, SandeepKsinha
wrote:
>> Hi all,
>>
>> Here is a code snippet from e4defrag.c, can be located @
>>
http://git.kernel.org/?p=fs/ext2/e2fsprogs.git;a=blob_plain;f=misc/e4defrag.c;hb=next
>>
>>
>> /*
>> * get_file_extents() - Get file's extent list.
>> *
>> * @fd: defrag target file's descriptor.
>> * @ext_list_head: the head of the extent list.
>> */
>> static int get_file_extents(int fd, struct fiemap_extent_list
**ext_list_head)
>> {
>> __u32 i;
>> int ret;
>> int ext_buf_size, fie_buf_size;
>> __u64 pos = 0;
>> struct fiemap *fiemap_buf = NULL;
>> struct fiemap_extent *ext_buf = NULL;
>> struct fiemap_extent_list *ext_list = NULL;
>>
>> /* Convert units, in bytes.
>> * Be careful : now, physical block number in extent is 48bit,
>> * and the maximum blocksize for ext4 is 4K(12bit),
>> * so there is no overflow, but in future it may be changed.
>> */
>>
>> /* Alloc space for fiemap */
>> ext_buf_size = EXTENT_MAX_COUNT * sizeof(struct fiemap_extent);
>> fie_buf_size = sizeof(struct fiemap) + ext_buf_size;
>>
>> fiemap_buf = malloc(fie_buf_size);
>> if (fiemap_buf == NULL)
>> return -1;
>>
>> ext_buf = fiemap_buf->fm_extents;
>> memset(fiemap_buf, 0, fie_buf_size);
>> fiemap_buf->fm_length = FIEMAP_MAX_OFFSET;
>> fiemap_buf->fm_flags |= FIEMAP_FLAG_SYNC;
>> fiemap_buf->fm_extent_count = EXTENT_MAX_COUNT;
>>
>> do {
>> fiemap_buf->fm_start = pos;
>> memset(ext_buf, 0, ext_buf_size);
>> ret = ioctl(fd, FS_IOC_FIEMAP, fiemap_buf);
>> if (ret < 0)
>> goto out;
>> for (i = 0; i < fiemap_buf->fm_mapped_extents; i++) {
>> ext_list = NULL;
>> ext_list = malloc(sizeof(struct fiemap_extent_list));
>> if (ext_list == NULL)
>> goto out;
>>
>> ext_list->data.physical = ext_buf[i].fe_physical
>> / block_size;
>> ext_list->data.logical = ext_buf[i].fe_logical
>> / block_size;
>> ext_list->data.len = ext_buf[i].fe_length
>> / block_size;
>>
>> ret = insert_extent_by_physical(
>> ext_list_head, ext_list);
>> if (ret < 0) {
>> FREE(ext_list);
>> goto out;
>> }
>> }
>> /* Record file's logical offset this time */
>> pos = ext_buf[EXTENT_MAX_COUNT-1].fe_logical +
>> ext_buf[EXTENT_MAX_COUNT-1].fe_length;
>> /*
>> * If fm_extents array has been filled and
>> * there are extents left, continue to cycle.
>> */
>> } while (fiemap_buf->fm_mapped_extents
>> == EXTENT_MAX_COUNT &&
>> !(ext_buf[EXTENT_MAX_COUNT-1].fe_flags
>> & FIEMAP_EXTENT_LAST));
>>
>> FREE(fiemap_buf);
>> return 0;
>> out:
>> FREE(fiemap_buf);
>> return -1;
>> }
>>
>> Why do we have a while loop here, we have already made sure that we
>> are initially itself allocating memory for EXTENT_MAX_COUNT?
>> This is the maximum number of extents supported by the file system.
> Sandeep,
>
> EXTENT_MAX_COUNT doesn't look to be the max extents supported by the
> filesystem. It is the max chunk supported by e4defrag. Max supported
> is FIEMAP_MAX_EXTENTS which is much larger.
Exactly.
Thanks for your explanation, Manish. :-)
Regards,
Akira Fujita
>> Is
>> this loop not redundant ?
>
> No, the loop just tries to get more extents as long as you haven't hit
> the last one.
>
>>
>>
>>
>> --
>> Regards,
>> Sandeep.
>>
>>
>>
>>
>>
>>
>> “To learn is to change. Education is a process that changes the learner.”
>>
>> --
>> To unsubscribe from this list: send an email with
>> "unsubscribe kernelnewbies" to ecar...@nl.linux.org
>> Please read the FAQ at http://kernelnewbies.org/FAQ
>>
>>
>
>
>
--
To unsubscribe from this list: send an email with
"unsubscribe kernelnewbies" to ecar...@nl.linux.org
Please read the FAQ at http://kernelnewbies.org/FAQ
Re: ext4 extents: How to determine if an extent points to a hole.
On Sun, Sep 13, 2009 at 10:12 PM, SandeepKsinha wrote: > Hi, > > > On Sun, Sep 13, 2009 at 10:07 PM, Manish Katiyar wrote: >> On Sun, Sep 13, 2009 at 5:44 PM, SandeepKsinha >> wrote: >>> Hi all, >>> >>> Looking at the flags in the extent info, Is there any specifc flags >>> which indicates an extent to be a HOLE?? >> >> I am not sure if I understand the question correctly .. why would >> you need that ? Can you give an example where it should be used ?? >> > > Look at e4defrag.c, > it checks the file size and allocates the same number of blocks for > donor inode. Which will eventually make a holey file into a normal > one. > Any tool/application should make sure that they leave a sparse file as sparse. > > I think, as suggested by Greg Freemyer, we can use BMAP ioctl to get > such information. Yes, but I think bmap would be costly if the file is large and is not holey :-( but that would be probably same calling fiemap if the file is completely fragmented such that each extent size is 1. > > >>> >>> I could only find the following ones... >>> >>> 39#define FIEMAP_MAX_OFFSET (~0ULL) >>> 40 >>> 41#define FIEMAP_FLAG_SYNC 0x0001 /* sync file data before map >>> */ >>> 42#define FIEMAP_FLAG_XATTR 0x0002 /* map extended attribute >>> tree */ >>> 43 >>> 44#define FIEMAP_FLAGS_COMPAT (FIEMAP_FLAG_SYNC | FIEMAP_FLAG_XATTR) >>> 45 >>> 46#define FIEMAP_EXTENT_LAST 0x0001 /* Last extent >>> in file. */ >>> 47#define FIEMAP_EXTENT_UNKNOWN 0x0002 /* Data >>> location unknown. */ >>> 48#define FIEMAP_EXTENT_DELALLOC 0x0004 /* Location >>> still pending. >>> 49 * Sets >>> EXTENT_UNKNOWN. */ >>> 50#define FIEMAP_EXTENT_ENCODED 0x0008 /* Data can not be >>> read >>> 51 * while fs is >>> unmounted */ >>> 52#define FIEMAP_EXTENT_DATA_ENCRYPTED 0x0080 /* Data is >>> encrypted by fs. >>> 53 * Sets >>> EXTENT_NO_BYPASS. */ >>> 54#define FIEMAP_EXTENT_NOT_ALIGNED 0x0100 /* Extent >>> offsets may not be >>> 55 * block aligned. */ >>> 56#define FIEMAP_EXTENT_DATA_INLINE 0x0200 /* Data mixed >>> with metadata. >>> 57 * Sets >>> EXTENT_NOT_ALIGNED.*/ >>> 58#define FIEMAP_EXTENT_DATA_TAIL 0x0400 /* Multiple >>> files in block. >>> 59 * Sets >>> EXTENT_NOT_ALIGNED.*/ >>> 60#define FIEMAP_EXTENT_UNWRITTEN 0x0800 /* Space allocated, >>> but >>> 61 * no data >>> (i.e. zero). */ >>> 62#define FIEMAP_EXTENT_MERGED 0x1000 /* File does not >>> natively >>> 63 * support >>> extents. Result >>> 64 * merged for >>> efficiency. */ >>> >>> >>> -- >>> Regards, >>> Sandeep. >>> >>> >>> >>> >>> >>> >>> “To learn is to change. Education is a process that changes the learner.” >>> >>> -- >>> To unsubscribe from this list: send an email with >>> "unsubscribe kernelnewbies" to ecar...@nl.linux.org >>> Please read the FAQ at http://kernelnewbies.org/FAQ >>> >>> >> >> >> >> -- >> Thanks - >> Manish >> > > > > -- > Regards, > Sandeep. > > > > > > > “To learn is to change. Education is a process that changes the learner.” > -- Thanks - Manish -- To unsubscribe from this list: send an email with "unsubscribe kernelnewbies" to ecar...@nl.linux.org Please read the FAQ at http://kernelnewbies.org/FAQ
Re: ext4 extents: How to determine if an extent points to a hole.
On Sun, Sep 13, 2009 at 11:08 PM, Manish Katiyar wrote: > On Sun, Sep 13, 2009 at 10:12 PM, SandeepKsinha > wrote: >> Hi, >> >> >> On Sun, Sep 13, 2009 at 10:07 PM, Manish Katiyar wrote: >>> On Sun, Sep 13, 2009 at 5:44 PM, SandeepKsinha >>> wrote: Hi all, Looking at the flags in the extent info, Is there any specifc flags which indicates an extent to be a HOLE?? >>> >>> I am not sure if I understand the question correctly .. why would >>> you need that ? Can you give an example where it should be used ?? >>> >> >> Look at e4defrag.c, >> it checks the file size and allocates the same number of blocks for >> donor inode. Which will eventually make a holey file into a normal >> one. >> Any tool/application should make sure that they leave a sparse file as >> sparse. >> >> I think, as suggested by Greg Freemyer, we can use BMAP ioctl to get >> such information. > > Yes, but I think bmap would be costly if the file is large and is not > holey :-( but that would be probably same calling fiemap if the > file is completely fragmented such that each extent size is 1. > Since, ext2/ext3 did not have mutli block allocation thats why this is the only way that we might have. But generally most of the new features work on with extent based files on ext4. I am still wondering that how to we represent a hole using extents in a extent based file. Just like we had a convention of having the block number 0 in case of holes. Similarly, what do we look at to figure out if its a hole or not. BMAP is one way. But since, in a extent based file, we have only extents, there should be some flag to indicate the same. > >> >> I could only find the following ones... 39#define FIEMAP_MAX_OFFSET (~0ULL) 40 41#define FIEMAP_FLAG_SYNC 0x0001 /* sync file data before map */ 42#define FIEMAP_FLAG_XATTR 0x0002 /* map extended attribute tree */ 43 44#define FIEMAP_FLAGS_COMPAT (FIEMAP_FLAG_SYNC | FIEMAP_FLAG_XATTR) 45 46#define FIEMAP_EXTENT_LAST 0x0001 /* Last extent in file. */ 47#define FIEMAP_EXTENT_UNKNOWN 0x0002 /* Data location unknown. */ 48#define FIEMAP_EXTENT_DELALLOC 0x0004 /* Location still pending. 49 * Sets EXTENT_UNKNOWN. */ 50#define FIEMAP_EXTENT_ENCODED 0x0008 /* Data can not be read 51 * while fs is unmounted */ 52#define FIEMAP_EXTENT_DATA_ENCRYPTED 0x0080 /* Data is encrypted by fs. 53 * Sets EXTENT_NO_BYPASS. */ 54#define FIEMAP_EXTENT_NOT_ALIGNED 0x0100 /* Extent offsets may not be 55 * block aligned. */ 56#define FIEMAP_EXTENT_DATA_INLINE 0x0200 /* Data mixed with metadata. 57 * Sets EXTENT_NOT_ALIGNED.*/ 58#define FIEMAP_EXTENT_DATA_TAIL 0x0400 /* Multiple files in block. 59 * Sets EXTENT_NOT_ALIGNED.*/ 60#define FIEMAP_EXTENT_UNWRITTEN 0x0800 /* Space allocated, but 61 * no data (i.e. zero). */ 62#define FIEMAP_EXTENT_MERGED 0x1000 /* File does not natively 63 * support extents. Result 64 * merged for efficiency. */ -- Regards, Sandeep. “To learn is to change. Education is a process that changes the learner.” -- To unsubscribe from this list: send an email with "unsubscribe kernelnewbies" to ecar...@nl.linux.org Please read the FAQ at http://kernelnewbies.org/FAQ >>> >>> >>> >>> -- >>> Thanks - >>> Manish >>> >> >> >> >> -- >> Regards, >> Sandeep. >> >> >> >> >> >> >> “To learn is to change. Education is a process that changes the learner.” >> > > > > -- > Thanks - > Manish > -- Regards, Sandeep. “To learn is to change. Education is a process that changes the learner.” -- To unsubscribe from this list: send an email with "unsubscribe kernelnewbies" to ecar...@nl.linux.org Please read the FAQ at http://kernelnewbies.org/FAQ
Re: e4defrag: file extent map : redundant loop???
On Sun, Sep 13, 2009 at 5:26 PM, SandeepKsinha wrote:
> Hi all,
>
> Here is a code snippet from e4defrag.c, can be located @
> http://git.kernel.org/?p=fs/ext2/e2fsprogs.git;a=blob_plain;f=misc/e4defrag.c;hb=next
>
>
> /*
> * get_file_extents() - Get file's extent list.
> *
> * @fd: defrag target file's descriptor.
> * @ext_list_head: the head of the extent list.
> */
> static int get_file_extents(int fd, struct fiemap_extent_list **ext_list_head)
> {
> __u32 i;
> int ret;
> int ext_buf_size, fie_buf_size;
> __u64 pos = 0;
> struct fiemap *fiemap_buf = NULL;
> struct fiemap_extent *ext_buf = NULL;
> struct fiemap_extent_list *ext_list = NULL;
>
> /* Convert units, in bytes.
> * Be careful : now, physical block number in extent is 48bit,
> * and the maximum blocksize for ext4 is 4K(12bit),
> * so there is no overflow, but in future it may be changed.
> */
>
> /* Alloc space for fiemap */
> ext_buf_size = EXTENT_MAX_COUNT * sizeof(struct fiemap_extent);
> fie_buf_size = sizeof(struct fiemap) + ext_buf_size;
>
> fiemap_buf = malloc(fie_buf_size);
> if (fiemap_buf == NULL)
> return -1;
>
> ext_buf = fiemap_buf->fm_extents;
> memset(fiemap_buf, 0, fie_buf_size);
> fiemap_buf->fm_length = FIEMAP_MAX_OFFSET;
> fiemap_buf->fm_flags |= FIEMAP_FLAG_SYNC;
> fiemap_buf->fm_extent_count = EXTENT_MAX_COUNT;
>
> do {
> fiemap_buf->fm_start = pos;
> memset(ext_buf, 0, ext_buf_size);
> ret = ioctl(fd, FS_IOC_FIEMAP, fiemap_buf);
> if (ret < 0)
> goto out;
> for (i = 0; i < fiemap_buf->fm_mapped_extents; i++) {
> ext_list = NULL;
> ext_list = malloc(sizeof(struct fiemap_extent_list));
> if (ext_list == NULL)
> goto out;
>
> ext_list->data.physical = ext_buf[i].fe_physical
> / block_size;
> ext_list->data.logical = ext_buf[i].fe_logical
> / block_size;
> ext_list->data.len = ext_buf[i].fe_length
> / block_size;
>
> ret = insert_extent_by_physical(
> ext_list_head, ext_list);
> if (ret < 0) {
> FREE(ext_list);
> goto out;
> }
> }
> /* Record file's logical offset this time */
> pos = ext_buf[EXTENT_MAX_COUNT-1].fe_logical +
> ext_buf[EXTENT_MAX_COUNT-1].fe_length;
> /*
> * If fm_extents array has been filled and
> * there are extents left, continue to cycle.
> */
> } while (fiemap_buf->fm_mapped_extents
> == EXTENT_MAX_COUNT &&
> !(ext_buf[EXTENT_MAX_COUNT-1].fe_flags
> & FIEMAP_EXTENT_LAST));
>
> FREE(fiemap_buf);
> return 0;
> out:
> FREE(fiemap_buf);
> return -1;
> }
>
> Why do we have a while loop here, we have already made sure that we
> are initially itself allocating memory for EXTENT_MAX_COUNT?
> This is the maximum number of extents supported by the file system.
Sandeep,
EXTENT_MAX_COUNT doesn't look to be the max extents supported by the
filesystem. It is the max chunk supported by e4defrag. Max supported
is FIEMAP_MAX_EXTENTS which is much larger.
> Is
> this loop not redundant ?
No, the loop just tries to get more extents as long as you haven't hit
the last one.
>
>
>
>
> --
> Regards,
> Sandeep.
>
>
>
>
>
>
> “To learn is to change. Education is a process that changes the learner.”
>
> --
> To unsubscribe from this list: send an email with
> "unsubscribe kernelnewbies" to ecar...@nl.linux.org
> Please read the FAQ at http://kernelnewbies.org/FAQ
>
>
--
Thanks -
Manish
--
To unsubscribe from this list: send an email with
"unsubscribe kernelnewbies" to ecar...@nl.linux.org
Please read the FAQ at http://kernelnewbies.org/FAQ
Re: ext4 extents: How to determine if an extent points to a hole.
On Sun, Sep 13, 2009 at 5:44 PM, SandeepKsinha wrote: > Hi all, > > Looking at the flags in the extent info, Is there any specifc flags > which indicates an extent to be a HOLE?? I am not sure if I understand the question correctly .. why would you need that ? Can you give an example where it should be used ?? > > I could only find the following ones... > > 39#define FIEMAP_MAX_OFFSET (~0ULL) > 40 > 41#define FIEMAP_FLAG_SYNC 0x0001 /* sync file data before map */ > 42#define FIEMAP_FLAG_XATTR 0x0002 /* map extended attribute tree > */ > 43 > 44#define FIEMAP_FLAGS_COMPAT (FIEMAP_FLAG_SYNC | FIEMAP_FLAG_XATTR) > 45 > 46#define FIEMAP_EXTENT_LAST 0x0001 /* Last extent > in file. */ > 47#define FIEMAP_EXTENT_UNKNOWN 0x0002 /* Data > location unknown. */ > 48#define FIEMAP_EXTENT_DELALLOC 0x0004 /* Location > still pending. > 49 * Sets > EXTENT_UNKNOWN. */ > 50#define FIEMAP_EXTENT_ENCODED 0x0008 /* Data can not be read > 51 * while fs is > unmounted */ > 52#define FIEMAP_EXTENT_DATA_ENCRYPTED 0x0080 /* Data is > encrypted by fs. > 53 * Sets > EXTENT_NO_BYPASS. */ > 54#define FIEMAP_EXTENT_NOT_ALIGNED 0x0100 /* Extent > offsets may not be > 55 * block aligned. */ > 56#define FIEMAP_EXTENT_DATA_INLINE 0x0200 /* Data mixed > with metadata. > 57 * Sets > EXTENT_NOT_ALIGNED.*/ > 58#define FIEMAP_EXTENT_DATA_TAIL 0x0400 /* Multiple > files in block. > 59 * Sets > EXTENT_NOT_ALIGNED.*/ > 60#define FIEMAP_EXTENT_UNWRITTEN 0x0800 /* Space allocated, but > 61 * no data > (i.e. zero). */ > 62#define FIEMAP_EXTENT_MERGED 0x1000 /* File does not > natively > 63 * support > extents. Result > 64 * merged for > efficiency. */ > > > -- > Regards, > Sandeep. > > > > > > > “To learn is to change. Education is a process that changes the learner.” > > -- > To unsubscribe from this list: send an email with > "unsubscribe kernelnewbies" to ecar...@nl.linux.org > Please read the FAQ at http://kernelnewbies.org/FAQ > > -- Thanks - Manish -- To unsubscribe from this list: send an email with "unsubscribe kernelnewbies" to ecar...@nl.linux.org Please read the FAQ at http://kernelnewbies.org/FAQ
Re: ext4 extents: How to determine if an extent points to a hole.
Hi, On Sun, Sep 13, 2009 at 10:07 PM, Manish Katiyar wrote: > On Sun, Sep 13, 2009 at 5:44 PM, SandeepKsinha > wrote: >> Hi all, >> >> Looking at the flags in the extent info, Is there any specifc flags >> which indicates an extent to be a HOLE?? > > I am not sure if I understand the question correctly .. why would > you need that ? Can you give an example where it should be used ?? > Look at e4defrag.c, it checks the file size and allocates the same number of blocks for donor inode. Which will eventually make a holey file into a normal one. Any tool/application should make sure that they leave a sparse file as sparse. I think, as suggested by Greg Freemyer, we can use BMAP ioctl to get such information. >> >> I could only find the following ones... >> >> 39#define FIEMAP_MAX_OFFSET (~0ULL) >> 40 >> 41#define FIEMAP_FLAG_SYNC 0x0001 /* sync file data before map */ >> 42#define FIEMAP_FLAG_XATTR 0x0002 /* map extended attribute tree >> */ >> 43 >> 44#define FIEMAP_FLAGS_COMPAT (FIEMAP_FLAG_SYNC | FIEMAP_FLAG_XATTR) >> 45 >> 46#define FIEMAP_EXTENT_LAST 0x0001 /* Last extent >> in file. */ >> 47#define FIEMAP_EXTENT_UNKNOWN 0x0002 /* Data >> location unknown. */ >> 48#define FIEMAP_EXTENT_DELALLOC 0x0004 /* Location >> still pending. >> 49 * Sets >> EXTENT_UNKNOWN. */ >> 50#define FIEMAP_EXTENT_ENCODED 0x0008 /* Data can not be read >> 51 * while fs is >> unmounted */ >> 52#define FIEMAP_EXTENT_DATA_ENCRYPTED 0x0080 /* Data is >> encrypted by fs. >> 53 * Sets >> EXTENT_NO_BYPASS. */ >> 54#define FIEMAP_EXTENT_NOT_ALIGNED 0x0100 /* Extent >> offsets may not be >> 55 * block aligned. */ >> 56#define FIEMAP_EXTENT_DATA_INLINE 0x0200 /* Data mixed >> with metadata. >> 57 * Sets >> EXTENT_NOT_ALIGNED.*/ >> 58#define FIEMAP_EXTENT_DATA_TAIL 0x0400 /* Multiple >> files in block. >> 59 * Sets >> EXTENT_NOT_ALIGNED.*/ >> 60#define FIEMAP_EXTENT_UNWRITTEN 0x0800 /* Space allocated, but >> 61 * no data >> (i.e. zero). */ >> 62#define FIEMAP_EXTENT_MERGED 0x1000 /* File does not >> natively >> 63 * support >> extents. Result >> 64 * merged for >> efficiency. */ >> >> >> -- >> Regards, >> Sandeep. >> >> >> >> >> >> >> “To learn is to change. Education is a process that changes the learner.” >> >> -- >> To unsubscribe from this list: send an email with >> "unsubscribe kernelnewbies" to ecar...@nl.linux.org >> Please read the FAQ at http://kernelnewbies.org/FAQ >> >> > > > > -- > Thanks - > Manish > -- Regards, Sandeep. “To learn is to change. Education is a process that changes the learner.” -- To unsubscribe from this list: send an email with "unsubscribe kernelnewbies" to ecar...@nl.linux.org Please read the FAQ at http://kernelnewbies.org/FAQ
PCI Issues
Hello, 1. Can several drivers can handle the same PCI device? 2. How can I know which driver currently handles my PCI devices? (lsmod show few drivers that can show a few candidate but I want to know the actual driver that register to that PCI). Regards, Mena
Re: spinlocks and copy_to_user()
Hi, On Sun, Sep 13, 2009 at 3:53 PM, Mark Ryden wrote: > Hello, > > I have a question about spinlocks and copy_to_user(): > copy_to_user() can sleep > so it should not be called when a spin_lock is held. > > However, I have data (a structure, not atomic variable) which can be > changed in interrupt context > in my module; so this data is protected by spinlocks. > > I use ioctl to pass this data to user space; this ioctl() calls > copy_to_user(); when I call spin_lock_bh() > before cp[y_to_user() in this ioctl , I get: > Use a local variable in ioctl: In the critical section copy your data into this variable using memcpy. Outside the critical section call copy_to_user. > "kernel: BUG: sleeping function called from invalid > context at include/linux/kernel.h:155". > > How should I call copy_to_user() so that it won't pass wrong data to > user space ? If I remove the > spin_lock_bh() before the call to copy_to_user() , I have a chance to > copy wrong data (to copy data while it is changing in my module ). > > Regards, > Mark > > -- > To unsubscribe from this list: send an email with > "unsubscribe kernelnewbies" to ecar...@nl.linux.org > Please read the FAQ at http://kernelnewbies.org/FAQ > > -- To unsubscribe from this list: send an email with "unsubscribe kernelnewbies" to ecar...@nl.linux.org Please read the FAQ at http://kernelnewbies.org/FAQ
spinlocks and copy_to_user()
Hello, I have a question about spinlocks and copy_to_user(): copy_to_user() can sleep so it should not be called when a spin_lock is held. However, I have data (a structure, not atomic variable) which can be changed in interrupt context in my module; so this data is protected by spinlocks. I use ioctl to pass this data to user space; this ioctl() calls copy_to_user(); when I call spin_lock_bh() before cp[y_to_user() in this ioctl , I get: "kernel: BUG: sleeping function called from invalid context at include/linux/kernel.h:155". How should I call copy_to_user() so that it won't pass wrong data to user space ? If I remove the spin_lock_bh() before the call to copy_to_user() , I have a chance to copy wrong data (to copy data while it is changing in my module ). Regards, Mark -- To unsubscribe from this list: send an email with "unsubscribe kernelnewbies" to ecar...@nl.linux.org Please read the FAQ at http://kernelnewbies.org/FAQ
e4defrag: file extent map : redundant loop???
Hi all,
Here is a code snippet from e4defrag.c, can be located @
http://git.kernel.org/?p=fs/ext2/e2fsprogs.git;a=blob_plain;f=misc/e4defrag.c;hb=next
/*
* get_file_extents() - Get file's extent list.
*
* @fd: defrag target file's descriptor.
* @ext_list_head: the head of the extent list.
*/
static int get_file_extents(int fd, struct fiemap_extent_list **ext_list_head)
{
__u32 i;
int ret;
int ext_buf_size, fie_buf_size;
__u64 pos = 0;
struct fiemap *fiemap_buf = NULL;
struct fiemap_extent*ext_buf = NULL;
struct fiemap_extent_list *ext_list = NULL;
/* Convert units, in bytes.
* Be careful : now, physical block number in extent is 48bit,
* and the maximum blocksize for ext4 is 4K(12bit),
* so there is no overflow, but in future it may be changed.
*/
/* Alloc space for fiemap */
ext_buf_size = EXTENT_MAX_COUNT * sizeof(struct fiemap_extent);
fie_buf_size = sizeof(struct fiemap) + ext_buf_size;
fiemap_buf = malloc(fie_buf_size);
if (fiemap_buf == NULL)
return -1;
ext_buf = fiemap_buf->fm_extents;
memset(fiemap_buf, 0, fie_buf_size);
fiemap_buf->fm_length = FIEMAP_MAX_OFFSET;
fiemap_buf->fm_flags |= FIEMAP_FLAG_SYNC;
fiemap_buf->fm_extent_count = EXTENT_MAX_COUNT;
do {
fiemap_buf->fm_start = pos;
memset(ext_buf, 0, ext_buf_size);
ret = ioctl(fd, FS_IOC_FIEMAP, fiemap_buf);
if (ret < 0)
goto out;
for (i = 0; i < fiemap_buf->fm_mapped_extents; i++) {
ext_list = NULL;
ext_list = malloc(sizeof(struct fiemap_extent_list));
if (ext_list == NULL)
goto out;
ext_list->data.physical = ext_buf[i].fe_physical
/ block_size;
ext_list->data.logical = ext_buf[i].fe_logical
/ block_size;
ext_list->data.len = ext_buf[i].fe_length
/ block_size;
ret = insert_extent_by_physical(
ext_list_head, ext_list);
if (ret < 0) {
FREE(ext_list);
goto out;
}
}
/* Record file's logical offset this time */
pos = ext_buf[EXTENT_MAX_COUNT-1].fe_logical +
ext_buf[EXTENT_MAX_COUNT-1].fe_length;
/*
* If fm_extents array has been filled and
* there are extents left, continue to cycle.
*/
} while (fiemap_buf->fm_mapped_extents
== EXTENT_MAX_COUNT &&
!(ext_buf[EXTENT_MAX_COUNT-1].fe_flags
& FIEMAP_EXTENT_LAST));
FREE(fiemap_buf);
return 0;
out:
FREE(fiemap_buf);
return -1;
}
Why do we have a while loop here, we have already made sure that we
are initially itself allocating memory for EXTENT_MAX_COUNT?
This is the maximum number of extents supported by the file system. Is
this loop not redundant ?
--
Regards,
Sandeep.
“To learn is to change. Education is a process that changes the learner.”
--
To unsubscribe from this list: send an email with
"unsubscribe kernelnewbies" to ecar...@nl.linux.org
Please read the FAQ at http://kernelnewbies.org/FAQ
ext4 extents: How to determine if an extent points to a hole.
Hi all, Looking at the flags in the extent info, Is there any specifc flags which indicates an extent to be a HOLE?? I could only find the following ones... 39#define FIEMAP_MAX_OFFSET (~0ULL) 40 41#define FIEMAP_FLAG_SYNC0x0001 /* sync file data before map */ 42#define FIEMAP_FLAG_XATTR 0x0002 /* map extended attribute tree */ 43 44#define FIEMAP_FLAGS_COMPAT (FIEMAP_FLAG_SYNC | FIEMAP_FLAG_XATTR) 45 46#define FIEMAP_EXTENT_LAST 0x0001 /* Last extent in file. */ 47#define FIEMAP_EXTENT_UNKNOWN 0x0002 /* Data location unknown. */ 48#define FIEMAP_EXTENT_DELALLOC 0x0004 /* Location still pending. 49* Sets EXTENT_UNKNOWN. */ 50#define FIEMAP_EXTENT_ENCODED 0x0008 /* Data can not be read 51* while fs is unmounted */ 52#define FIEMAP_EXTENT_DATA_ENCRYPTED0x0080 /* Data is encrypted by fs. 53* Sets EXTENT_NO_BYPASS. */ 54#define FIEMAP_EXTENT_NOT_ALIGNED 0x0100 /* Extent offsets may not be 55* block aligned. */ 56#define FIEMAP_EXTENT_DATA_INLINE 0x0200 /* Data mixed with metadata. 57* Sets EXTENT_NOT_ALIGNED.*/ 58#define FIEMAP_EXTENT_DATA_TAIL 0x0400 /* Multiple files in block. 59* Sets EXTENT_NOT_ALIGNED.*/ 60#define FIEMAP_EXTENT_UNWRITTEN 0x0800 /* Space allocated, but 61* no data (i.e. zero). */ 62#define FIEMAP_EXTENT_MERGED0x1000 /* File does not natively 63* support extents. Result 64* merged for efficiency. */ -- Regards, Sandeep. “To learn is to change. Education is a process that changes the learner.” -- To unsubscribe from this list: send an email with "unsubscribe kernelnewbies" to ecar...@nl.linux.org Please read the FAQ at http://kernelnewbies.org/FAQ
