Re: [U-Boot] [PATCH] ext4fs: Add ext4 extent tree cache

[Tom Rini]

On Mon, May 14, 2018 at 05:21:38PM +0200, Marek Vasut wrote:

> On 05/14/2018 05:14 PM, Evan Thompson wrote:
> > On Sun, May 13, 2018 at 12:03 PM, Marek Vasut  wrote:
> >>
> >> btw doesn't CONFIG_BLOCK_CACHE offer similar service, but for everyone
> >> and on block level ? I recall looking for ext4 fs speed up , but then
> >> ultimately used CONFIG_BLOCK_CACHE which did it for me.
> > 
> > I was unaware of the existence of CONFIG_BLOCK_CACHE - yes, that will
> > solve the same problem by caching at the block level instead of the
> > filesystem level. So my patch is redundant, and ext4 users should just
> > be advised to turn on CONFIG_BLOCK_CACHE.
> 
> Maybe this block cache should be default=y (if BLK is enabled?) or we
> should advertise it more somehow. I ran into the exact same issue and
> started looking into the ext4 patches myself only to find the block
> cache later on.

Agreed.  I'm right now testing a patch or two to do that and see what
the size cost is.  Thanks all!

-- 
Tom


signature.asc
Description: PGP signature
___
U-Boot mailing list
U-Boot@lists.denx.de
https://lists.denx.de/listinfo/u-boot

Re: [U-Boot] [PATCH] ext4fs: Add ext4 extent tree cache

[Marek Vasut]

On 05/14/2018 05:14 PM, Evan Thompson wrote:
> On Sun, May 13, 2018 at 12:03 PM, Marek Vasut  wrote:
>>
>> btw doesn't CONFIG_BLOCK_CACHE offer similar service, but for everyone
>> and on block level ? I recall looking for ext4 fs speed up , but then
>> ultimately used CONFIG_BLOCK_CACHE which did it for me.
> 
> I was unaware of the existence of CONFIG_BLOCK_CACHE - yes, that will
> solve the same problem by caching at the block level instead of the
> filesystem level. So my patch is redundant, and ext4 users should just
> be advised to turn on CONFIG_BLOCK_CACHE.

Maybe this block cache should be default=y (if BLK is enabled?) or we
should advertise it more somehow. I ran into the exact same issue and
started looking into the ext4 patches myself only to find the block
cache later on.

-- 
Best regards,
Marek Vasut
___
U-Boot mailing list
U-Boot@lists.denx.de
https://lists.denx.de/listinfo/u-boot

Re: [U-Boot] [PATCH] ext4fs: Add ext4 extent tree cache

[Evan Thompson]

On Sun, May 13, 2018 at 12:03 PM, Marek Vasut  wrote:
>
> btw doesn't CONFIG_BLOCK_CACHE offer similar service, but for everyone
> and on block level ? I recall looking for ext4 fs speed up , but then
> ultimately used CONFIG_BLOCK_CACHE which did it for me.

I was unaware of the existence of CONFIG_BLOCK_CACHE - yes, that will
solve the same problem by caching at the block level instead of the
filesystem level. So my patch is redundant, and ext4 users should just
be advised to turn on CONFIG_BLOCK_CACHE.

Thanks,
Evan
___
U-Boot mailing list
U-Boot@lists.denx.de
https://lists.denx.de/listinfo/u-boot

Re: [U-Boot] [PATCH] ext4fs: Add ext4 extent tree cache

[Marek Vasut]

On 05/11/2018 07:53 PM, Evan Thompson wrote:
> On Fri, May 11, 2018 at 10:18 AM, Tom Rini  wrote:
>> On Fri, May 11, 2018 at 10:14:38AM -0700, Evan Thompson wrote:
>>> On Thu, May 10, 2018 at 3:02 PM, Tom Rini  wrote:
 On Mon, Mar 26, 2018 at 04:05:24PM -0700, evan.g.thomp...@gmail.com wrote:

> +
> +#ifndef CONFIG_EXT4_EXTENT_CACHE_SIZE
> +#define CONFIG_EXT4_EXTENT_CACHE_SIZE 5
> +#endif

 This needs to be done in Kconfig.
>>>
>>> Ok, I'll fix that.
>>>
 Also, is there any sort of testing we could add to
 test/fs/fs-test.sh for this code?  Thanks!
>>>
>>> I can't think of a straightforward way, as this patch just provides a
>>> speedup for certain cases. The functionality of reading / writing
>>> files is unchanged.
>>
>> OK.  And the test suite passes as well as before, yes?
> 
> Yes, I'm seeing the same passes and fails as before.

btw doesn't CONFIG_BLOCK_CACHE offer similar service, but for everyone
and on block level ? I recall looking for ext4 fs speed up , but then
ultimately used CONFIG_BLOCK_CACHE which did it for me.

-- 
Best regards,
Marek Vasut
___
U-Boot mailing list
U-Boot@lists.denx.de
https://lists.denx.de/listinfo/u-boot

Re: [U-Boot] [PATCH] ext4fs: Add ext4 extent tree cache

[Evan Thompson]

On Fri, May 11, 2018 at 10:18 AM, Tom Rini  wrote:
> On Fri, May 11, 2018 at 10:14:38AM -0700, Evan Thompson wrote:
>> On Thu, May 10, 2018 at 3:02 PM, Tom Rini  wrote:
>> > On Mon, Mar 26, 2018 at 04:05:24PM -0700, evan.g.thomp...@gmail.com wrote:
>> >
>> >> +
>> >> +#ifndef CONFIG_EXT4_EXTENT_CACHE_SIZE
>> >> +#define CONFIG_EXT4_EXTENT_CACHE_SIZE 5
>> >> +#endif
>> >
>> > This needs to be done in Kconfig.
>>
>> Ok, I'll fix that.
>>
>> > Also, is there any sort of testing we could add to
>> > test/fs/fs-test.sh for this code?  Thanks!
>>
>> I can't think of a straightforward way, as this patch just provides a
>> speedup for certain cases. The functionality of reading / writing
>> files is unchanged.
>
> OK.  And the test suite passes as well as before, yes?

Yes, I'm seeing the same passes and fails as before.

- Evan
___
U-Boot mailing list
U-Boot@lists.denx.de
https://lists.denx.de/listinfo/u-boot

Re: [U-Boot] [PATCH] ext4fs: Add ext4 extent tree cache

[Tom Rini]

On Fri, May 11, 2018 at 10:14:38AM -0700, Evan Thompson wrote:
> On Thu, May 10, 2018 at 3:02 PM, Tom Rini  wrote:
> > On Mon, Mar 26, 2018 at 04:05:24PM -0700, evan.g.thomp...@gmail.com wrote:
> >
> >> +
> >> +#ifndef CONFIG_EXT4_EXTENT_CACHE_SIZE
> >> +#define CONFIG_EXT4_EXTENT_CACHE_SIZE 5
> >> +#endif
> >
> > This needs to be done in Kconfig.
> 
> Ok, I'll fix that.
> 
> > Also, is there any sort of testing we could add to
> > test/fs/fs-test.sh for this code?  Thanks!
> 
> I can't think of a straightforward way, as this patch just provides a
> speedup for certain cases. The functionality of reading / writing
> files is unchanged.

OK.  And the test suite passes as well as before, yes?

-- 
Tom


signature.asc
Description: PGP signature
___
U-Boot mailing list
U-Boot@lists.denx.de
https://lists.denx.de/listinfo/u-boot

Re: [U-Boot] [PATCH] ext4fs: Add ext4 extent tree cache

[Evan Thompson]

On Thu, May 10, 2018 at 3:02 PM, Tom Rini  wrote:
> On Mon, Mar 26, 2018 at 04:05:24PM -0700, evan.g.thomp...@gmail.com wrote:
>
>> +
>> +#ifndef CONFIG_EXT4_EXTENT_CACHE_SIZE
>> +#define CONFIG_EXT4_EXTENT_CACHE_SIZE 5
>> +#endif
>
> This needs to be done in Kconfig.

Ok, I'll fix that.

> Also, is there any sort of testing we could add to test/fs/fs-test.sh for 
> this code?  Thanks!

I can't think of a straightforward way, as this patch just provides a
speedup for certain cases. The functionality of reading / writing
files is unchanged.

- Evan
___
U-Boot mailing list
U-Boot@lists.denx.de
https://lists.denx.de/listinfo/u-boot

Re: [U-Boot] [PATCH] ext4fs: Add ext4 extent tree cache

[Tom Rini]

On Mon, Mar 26, 2018 at 04:05:24PM -0700, evan.g.thomp...@gmail.com wrote:

> From: Evan Thompson 
> 
> In ext4, the file inode can store up to 4 extents. If a file requires
> more (due to size or fragmentation), an extent index tree is used.
> 
> Currently, u-boot reads a node from each level of the extent tree
> for every block read, which is very inefficient when extent tree
> depth is > 0.
> 
> This patch adds a cache for the extent tree. We cache the 1
> most-recently-seen node at each extent tree level. The typical workload
> is sequential block access, so once we leave a given tree node, it will
> not be revisited. Therefore, it makes sense to just cache one node
> per tree level.
> 
> Cached blocks are lazily allocated. The typical case is extent tree
> depth = 0, in which case no caching is needed and no allocations will
> occur.
> 
> For files with extent tree depth = 1, this patch produces a ~10x
> improvement in read speed. For deeper extent trees, the improvement is
> larger. On my test device, a 3MB file which previously took 9s to read
> now takes 150ms.
> 
> Cache size is configurable with CONFIG_EXT4_EXTENT_CACHE_SIZE. However
> the default of 5 (the maximum depth of well-formed extent trees) is
> recommended.
> 
> Signed-off-by: Evan Thompson 
> ---
>  doc/README.ext4   |  7 +
>  fs/ext4/ext4_common.c | 74 
> +++
>  2 files changed, 76 insertions(+), 5 deletions(-)
> 
> diff --git a/doc/README.ext4 b/doc/README.ext4
> index 8ecd21eee3..a501b92396 100644
> --- a/doc/README.ext4
> +++ b/doc/README.ext4
> @@ -33,6 +33,13 @@ In addition, to get the write access command "ext4write", 
> enable:
>  which automatically selects CONFIG_EXT4_WRITE if it wasn't defined
>  already.
>  
> +For files with extents, an ext4 extent tree cache improves performance:
> +
> +  CONFIG_EXT4_EXTENT_CACHE_SIZE <#>
> +
> +The above cache size defaults to 5 if not defined. This default is
> +strongly recommended. 0 will turn off extent caching.
> +
>  Also relevant are the generic filesystem commands, selected by:
>  
>CONFIG_CMD_FS_GENERIC
> diff --git a/fs/ext4/ext4_common.c b/fs/ext4/ext4_common.c
> index e3cc30a1e0..61aade1ced 100644
> --- a/fs/ext4/ext4_common.c
> +++ b/fs/ext4/ext4_common.c
> @@ -1523,6 +1523,47 @@ void ext4fs_allocate_blocks(struct ext2_inode 
> *file_inode,
>  
>  #endif
>  
> +/* Extent tree cache caches one entry per tree level
> + * eg, ext_block->eh_depth is used as the index into the cache
> + *
> + * If the tree is deeper than CONFIG_EXT4_EXTENT_CACHE_SIZE (very unlikely),
> + * file read performance will be impacted by repeated re-reads
> + * of those index nodes.
> + */
> +
> +#ifndef CONFIG_EXT4_EXTENT_CACHE_SIZE
> +#define CONFIG_EXT4_EXTENT_CACHE_SIZE 5
> +#endif

This needs to be done in Kconfig.  Also, is there any sort of testing we
could add to test/fs/fs-test.sh for this code?  Thanks!

-- 
Tom


signature.asc
Description: PGP signature
___
U-Boot mailing list
U-Boot@lists.denx.de
https://lists.denx.de/listinfo/u-boot