Re: UBSAN array-index-out-of-bounds in read_blocklist

On 08/04/2024 01:56, Ubisectech Sirius wrote: Hello. We are Ubisectech Sirius Team, the vulnerability lab of China ValiantSec. Recently, our team has discovered a issue in Linux kernel 6.8. Attached to the email were a PoC file of the issue. Hello, I have found the cause of the problem and

Re: [PATCH] squashfs: fix inode lookup sanity checks

On 26/02/2021 09:29, Sean Nyekjaer wrote: When mouting a squashfs image created without inode compression it fails with: "unable to read inode lookup table" It turns out that the BLOCK_OFFSET is missing when checking the SQUASHFS_METADATA_SIZE agaist the actual size. Fixes: eabac19e40c0

Re: [PATCH] Squashfs: fix xattr id and id lookup sanity checks

> On 03/03/2021 00:34 Andrew Morton wrote: > > > On Mon, 1 Mar 2021 07:27:57 + (GMT) Phillip Lougher > wrote: > > > The checks for maximum metadata block size is > > missing SQUASHFS_BLOCK_OFFSET (the two byte length > > count). > > What are

[PATCH] Squashfs: fix xattr id and id lookup sanity checks

The checks for maximum metadata block size is missing SQUASHFS_BLOCK_OFFSET (the two byte length count). Cc: sta...@vger.kernel.org Signed-off-by: Phillip Lougher --- fs/squashfs/id.c | 6 -- fs/squashfs/xattr_id.c | 6 -- 2 files changed, 8 insertions(+), 4 deletions(-) diff

Re: [PATCH] squashfs: fix inode lookup sanity checks

TADATA_SIZE agaist the actual size. > > Fixes: eabac19e40c0 ("squashfs: add more sanity checks in inode lookup") > CC: sta...@vger.kernel.org > Signed-off-by: Sean Nyekjaer Acked-by: Phillip Lougher > --- > fs/squashfs/export.c | 8 ++-- > fs/squashfs/squ

[PATCH V2 2/4] Squashfs: add more sanity checks in id lookup

d-by: syzbot+b06d57ba83f604522...@syzkaller.appspotmail.com Reported-by: syzbot+c021ba012da41ee98...@syzkaller.appspotmail.com Reported-by: syzbot+5024636e8b5fd19f0...@syzkaller.appspotmail.com Reported-by: syzbot+bcbc661df46657d0f...@syzkaller.appspotmail.com Signed-off-by: Phillip Lougher Cc: sta...@vg

[PATCH V2 0/4] Squashfs: fix BIO migration regression and add sanity checks

Hi all, Patch [1/4] fixes a regression introduced by the "migrate from ll_rw_block usage to BIO" patch, which has produced a number of Sysbot/Syzkaller reports. Patches [2/4], [3/4], and [4/4] fix a number of filesystem corruption issues which have produced Sysbot reports in the id, inode and

[PATCH V2 1/4] Squashfs: avoid out of bounds writes in decompressors

system 2. It ensured that the "length" field read from the filesystem was within the expected maximum length. Without this any corrupted values can over-run allocated buffers. Reported-by: syzbot+6fba78f99b9afd4b5...@syzkaller.appspotmail.com Signed-off-by: Phillip Lougher

[PATCH V2 4/4] Squashfs: add more sanity checks in xattr id lookup

corruption of the xattr_ids count. This can either lead to the above kmalloc failure, or a smaller than expected table to be read. 3. It checks the contents of the index table for corruption. Reported-by: syzbot+2ccea6339d3683608...@syzkaller.appspotmail.com Signed-off-by: Phillip Loughe

[PATCH V2 3/4] Squashfs: add more sanity checks in inode lookup

zkaller.appspotmail.com Signed-off-by: Phillip Lougher Cc: sta...@vger.kernel.org --- fs/squashfs/export.c | 41 + 1 file changed, 33 insertions(+), 8 deletions(-) diff --git a/fs/squashfs/export.c b/fs/squashfs/export.c index ae2c87bb0fbe..3f134ba86a45 100

Greylisting on linux-kernel broken?

, and they have not been delivered either. Is there something wrong here? It is impossible to get an empty email to autoans...@vger.kernel.org wrong so it is delayed AFAIK. I have used http://vger.kernel.org/mxverify.html to check that the MX records appear correct. Thanks for any help. Phillip Lougher

[PATCH] squashfs: avoid bio_alloc() failure with 1Mbyte blocks

256 pages, but, because blocks are not aligned to page boundaries, it may require 257 pages to read. Bio_kmalloc() can handle 1024 pages, and so use this for the edge condition. Reported-by: Nicolas Prochazka Reported-by: Tomoatsu Shimada Signed-off-by: Phillip Lougher --- fs/squashfs/block.c | 6

[PATCH] squashfs: fix length field overlap check in metadata reading

that edge case. Reported-by: Bernd Amend Signed-off-by: Phillip Lougher --- fs/squashfs/block.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/squashfs/block.c b/fs/squashfs/block.c index 64f61330564a..76bb1c846845 100644 --- a/fs/squashfs/block.c +++ b/fs/squashfs/block.c

Re: Regression: squashfs issues since change "squashfs: migrate from ll_rw_block usage to BIO"

et <= bvec->bv_len - 1) { This check is wrong, it should be + if (offset < bvec->bv_len - 1) { Phillip > Are you able to check that the below fixes things up? > > Thanks. > > > From: Andrew Morton > Subject: revert "squashfs: migrate from ll

[ANN] Squashfs tools 4.4 released

Hi, I'm pleased to announce the release of Squashfs tools 4.4. This is the first release in over 5 years, and there are substantial improvements: reproducible builds, new compressors, CVE fixes, security hardening and new options for Mksquashfs/Unsquashfs. The new release can be downloaded here:

Re: Bug#921146: Program mksquashfs from squashfs-tools 1:4.3-11 does not make use all CPU cores

On Thu, Aug 1, 2019 at 11:26 PM Don Armstrong wrote: > > On Thu, 01 Aug 2019, Phillip Lougher wrote: > > That patch is a laughable piece of rubbish. I believe both you > > people (the Debian maintainer and author) are in total denial > > about your incompetence in this ma

Re: [PATCH 0/2] Make squashfs fragments' cache size more configurable

hangeable option allowing users to insert the Squashfs module in such a way that will break its performance. So NACK. Phillip Lougher (Squashfs maintainer) > Now make it be configured when booting or inserting module. > Actually, it's better that a config option in a number format > in

Re: [PATCH 0/2] Make squashfs fragments' cache size more configurable

owing users to insert the Squashfs module in such a way that will break its performance. So NACK. Phillip Lougher (Squashfs maintainer) > Now make it be configured when booting or inserting module. > Actually, it's better that a config option in a number format > in .config file cat be reconf

Re: [PATCH 1/5] Squashfs: remove the FILE_CACHE option

= file_direct.o page_actor.o > squashfs-$(CONFIG_SQUASHFS_DECOMP_SINGLE) += decompressor_single.o > squashfs-$(CONFIG_SQUASHFS_DECOMP_MULTI) += decompressor_multi.o > squashfs-$(CONFIG_SQUASHFS_DECOMP_MULTI_PERCPU) += > decompressor_multi_percpu.o > diff --git a/fs/squashfs/file_

Re: [PATCH 1/5] Squashfs: remove the FILE_CACHE option

.o > squashfs-$(CONFIG_SQUASHFS_DECOMP_MULTI) += decompressor_multi.o > squashfs-$(CONFIG_SQUASHFS_DECOMP_MULTI_PERCPU) += > decompressor_multi_percpu.o > diff --git a/fs/squashfs/file_cache.c b/fs/squashfs/file_cache.c > deleted file mode 100644 > index f2310d2a2019.. &

Re: [GIT PULL v2] zstd support (lib, btrfs, squashfs, nocrypto)

On Mon, Sep 11, 2017 at 8:24 PM, Chris Mason wrote: > Hi Linus, > > Nick Terrell's patch series to add zstd support to the kernel has been > floating around for a while. After talking with Dave Sterba, Herbert > and Phillip, we decided to send the whole thing in as one pull request.

Re: [GIT PULL v2] zstd support (lib, btrfs, squashfs, nocrypto)

On Mon, Sep 11, 2017 at 8:24 PM, Chris Mason wrote: > Hi Linus, > > Nick Terrell's patch series to add zstd support to the kernel has been > floating around for a while. After talking with Dave Sterba, Herbert > and Phillip, we decided to send the whole thing in as one pull request. As the

Re: [PATCH v4 4/5] squashfs: Add zstd support

On Sun, Aug 13, 2017 at 9:31 AM, Geert Uytterhoeven wrote: > On Fri, Aug 4, 2017 at 10:19 PM, Nick Terrell wrote: >> --- a/fs/squashfs/decompressor.c >> +++ b/fs/squashfs/decompressor.c >> @@ -65,6 +65,12 @@ static const struct squashfs_decompressor >>

Re: [PATCH v4 4/5] squashfs: Add zstd support

On Sun, Aug 13, 2017 at 9:31 AM, Geert Uytterhoeven wrote: > On Fri, Aug 4, 2017 at 10:19 PM, Nick Terrell wrote: >> --- a/fs/squashfs/decompressor.c >> +++ b/fs/squashfs/decompressor.c >> @@ -65,6 +65,12 @@ static const struct squashfs_decompressor >> squashfs_zlib_comp_ops = { >> }; >>

Re: [PATCH v3 4/4] squashfs: Add zstd support

On Mon, Jul 31, 2017 at 2:50 AM, Phillip Lougher <phillip.loug...@gmail.com> wrote: > On Thu, Jul 20, 2017 at 10:27 PM, Nick Terrell <terre...@fb.com> wrote: >> Add zstd compression and decompression support to SquashFS. zstd is a >> great fit for SquashFS becaus

Re: [PATCH v3 4/4] squashfs: Add zstd support

On Mon, Jul 31, 2017 at 2:50 AM, Phillip Lougher wrote: > On Thu, Jul 20, 2017 at 10:27 PM, Nick Terrell wrote: >> Add zstd compression and decompression support to SquashFS. zstd is a >> great fit for SquashFS because it can compress at ratios approaching xz, >> whil

Re: [PATCH v3 4/4] squashfs: Add zstd support

On Thu, Jul 20, 2017 at 10:27 PM, Nick Terrell wrote: > Add zstd compression and decompression support to SquashFS. zstd is a > great fit for SquashFS because it can compress at ratios approaching xz, > while decompressing twice as fast as zlib. For SquashFS in particular, > it

Re: [PATCH v3 4/4] squashfs: Add zstd support

On Thu, Jul 20, 2017 at 10:27 PM, Nick Terrell wrote: > Add zstd compression and decompression support to SquashFS. zstd is a > great fit for SquashFS because it can compress at ratios approaching xz, > while decompressing twice as fast as zlib. For SquashFS in particular, > it can decompress as

Re: squashfs-tools: Add -offset option to skip n bytes at the start of the file

On Sun, Jan 29, 2017 at 2:00 PM, Ritesh Raj Sarraf wrote: > On Sun, 2017-01-29 at 03:33 -0800, probonopd wrote: >> Yes, I am not happy about that. >> You could either push your change to the squashfs folks >> Sent a PR half a year ago but no reaction:

Re: squashfs-tools: Add -offset option to skip n bytes at the start of the file

On Sun, Jan 29, 2017 at 2:00 PM, Ritesh Raj Sarraf wrote: > On Sun, 2017-01-29 at 03:33 -0800, probonopd wrote: >> Yes, I am not happy about that. >> You could either push your change to the squashfs folks >> Sent a PR half a year ago but no reaction: plougher/squashfs-tools#13 - is >> something

Re: [RFC] Using squashfs, kernel will hung task with no free memory?

On 22/01/15 02:28, long.wanglong wrote: hi， I have encountered kernel hung task when running stability and stress test. test scenarios: 1)the kernel hungtask settings are following: hung_task_panic = 1 hung_task_timeout_secs = 120 2)the rootfs

Re: [RFC] Using squashfs, kernel will hung task with no free memory?

On 22/01/15 02:28, long.wanglong wrote: hi， I have encountered kernel hung task when running stability and stress test. test scenarios: 1)the kernel hungtask settings are following: hung_task_panic = 1 hung_task_timeout_secs = 120 2)the rootfs

[GIT PULL] Squashfs updates for 3.19

, and the necessary configuration option. Phillip Lougher (2): Squashfs: add LZ4 compression support Squashfs: Add LZ4 compression configuration option Documentation/filesystems/squashfs.txt |8 +- fs/squashfs/Kconfig

[GIT PULL] Squashfs updates for 3.19

, and the necessary configuration option. Phillip Lougher (2): Squashfs: add LZ4 compression support Squashfs: Add LZ4 compression configuration option Documentation/filesystems/squashfs.txt |8 +- fs/squashfs/Kconfig

Re: [PATCH 0/2] Squashfs: add LZ4 compression

On 12/12/14 15:56, Bruno Wolff III wrote: On Fri, Dec 12, 2014 at 13:23:19 +0100, toki clover wrote: Now, I did not see any Linux FS devs activity/response to this... What a waste of time because if those patch don't make it for this merge window, rebasing/reposting will be, again,

Re: [PATCH 0/2] Squashfs: add LZ4 compression

On 12/12/14 15:56, Bruno Wolff III wrote: On Fri, Dec 12, 2014 at 13:23:19 +0100, toki clover tokiclo...@gmail.com wrote: Now, I did not see any Linux FS devs activity/response to this... What a waste of time because if those patch don't make it for this merge window, rebasing/reposting will

[PATCH 2/2 V2] Squashfs: Add LZ4 compression configuration option

Add the glue code, and also update the documentation. Signed-off-by: Phillip Lougher --- V2 patch: when porting the code to the latest -rc I forgot to update the case where CONFIG_SQUASHFS_LZ4 is not defined, leading to a build error. Trivial change: +#ifndef CONFIG_SQUASHFS_LZ4 +static const

[PATCH 1/2] Squashfs: add LZ4 compression support

Add support for reading file systems compressed with the LZ4 compression algorithm. This patch adds the LZ4 decompressor wrapper code. Signed-off-by: Phillip Lougher --- fs/squashfs/lz4_wrapper.c | 142 + fs/squashfs/squashfs_fs.h |1 + 2 files

[PATCH 0/2] Squashfs: add LZ4 compression

requests to repost these patches and to get them mainlined. My intention is to submit them in the next kernel merge window. If you want LZ4 support in Squashfs now is a good time to publically support the inclusion of these patches. Phillip Lougher (2): Squashfs: add LZ4 compression support

[PATCH 2/2] Squashfs: Add LZ4 compression configuration option

Add the glue code, and also update the documentation. Signed-off-by: Phillip Lougher --- Documentation/filesystems/squashfs.txt |8 fs/squashfs/Kconfig| 15 +++ fs/squashfs/Makefile |1 + fs/squashfs/decompressor.c

[PATCH 2/2] Squashfs: Add LZ4 compression configuration option

Add the glue code, and also update the documentation. Signed-off-by: Phillip Lougher phil...@squashfs.org.uk --- Documentation/filesystems/squashfs.txt |8 fs/squashfs/Kconfig| 15 +++ fs/squashfs/Makefile |1 + fs/squashfs

[PATCH 1/2] Squashfs: add LZ4 compression support

Add support for reading file systems compressed with the LZ4 compression algorithm. This patch adds the LZ4 decompressor wrapper code. Signed-off-by: Phillip Lougher phil...@squashfs.org.uk --- fs/squashfs/lz4_wrapper.c | 142 + fs/squashfs

[PATCH 0/2] Squashfs: add LZ4 compression

requests to repost these patches and to get them mainlined. My intention is to submit them in the next kernel merge window. If you want LZ4 support in Squashfs now is a good time to publically support the inclusion of these patches. Phillip Lougher (2): Squashfs: add LZ4 compression support

[PATCH 2/2 V2] Squashfs: Add LZ4 compression configuration option

Add the glue code, and also update the documentation. Signed-off-by: Phillip Lougher phil...@squashfs.org.uk --- V2 patch: when porting the code to the latest -rc I forgot to update the case where CONFIG_SQUASHFS_LZ4 is not defined, leading to a build error. Trivial change: +#ifndef

[ANN] Squashfs tools 4.3 released

Hi, I'm pleased to announce the release of Squashfs tools 4.3. This is the first release in over 3 years, and there are substantial improvements to stability, new compression options and compressors, speed optimisations, new options for Mksquashfs/Unsquashfs, and fixes for CVE-2012-4024 and

[ANN] Squashfs tools 4.3 released

Hi, I'm pleased to announce the release of Squashfs tools 4.3. This is the first release in over 3 years, and there are substantial improvements to stability, new compression options and compressors, speed optimisations, new options for Mksquashfs/Unsquashfs, and fixes for CVE-2012-4024 and

Re: [PATCH] xz: make XZ_DEC_BCJ filters non-optional

On 05/03/14 16:24, Lasse Collin wrote: On 2014-03-05 Phillip Lougher wrote: (BTW Kyle you should have CC'd me on the patch as a courtesy). I could have done that too but somehow I didn't, sorry. np But speaking as the Squashfs author, the lack of BCJ support for an architecture creates

Re: [PATCH] xz: make XZ_DEC_BCJ filters non-optional

On 05/03/14 16:24, Lasse Collin wrote: On 2014-03-05 Phillip Lougher wrote: (BTW Kyle you should have CC'd me on the patch as a courtesy). I could have done that too but somehow I didn't, sorry. np But speaking as the Squashfs author, the lack of BCJ support for an architecture creates

Re: [PATCH] Squashfs: add asynchronous read support

On 16/12/13 05:30, Chanho Min wrote: This patch removes synchronous wait for the up-to-date of buffer in the file system level. Instead all operations after submit_bh are moved into the End-of-IO handler and its associated workeque. It decompresses/copies data into pages and unlock them

Re: [PATCH] Squashfs: add asynchronous read support

On 16/12/13 05:30, Chanho Min wrote: This patch removes synchronous wait for the up-to-date of buffer in the file system level. Instead all operations after submit_bh are moved into the End-of-IO handler and its associated workeque. It decompresses/copies data into pages and unlock them

[GIT PULL] Squashfs bug fixes for 3.13

) Just a single bug fix to the new "directly decompress into the page cache" code. -------- Phillip Lougher (1): Squashfs: fix failure to unlock pages on decompress error fs/squashfs/file_direct.c | 5 +++

[GIT PULL] Squashfs bug fixes for 3.13

) Just a single bug fix to the new directly decompress into the page cache code. Phillip Lougher (1): Squashfs: fix failure to unlock pages on decompress error fs/squashfs/file_direct.c | 5 - 1 file

[PATCH] Squashfs: fix failure to unlock pages on decompress error

Direct decompression into the page cache. If we fall back to using an intermediate buffer (because we cannot grab all the page cache pages) and we get a decompress fail, we forgot to release the pages. Reported-by: Roman Peniaev Signed-off-by: Phillip Lougher --- fs/squashfs/file_direct.c

Re: [PATCH 6/7] Squashfs: Directly decompress into the page cache for file data

On 22/11/13 07:40, Roman Peniaev wrote: Hello, Phillip. one remark below: +static int squashfs_read_cache(struct page *target_page, u64 block, int bsize, + int pages, struct page **page) +{ + struct inode *i = target_page->mapping->host; + struct squashfs_cache_entry

Re: [PATCH 6/7] Squashfs: Directly decompress into the page cache for file data

On 22/11/13 07:40, Roman Peniaev wrote: Hello, Phillip. one remark below: +static int squashfs_read_cache(struct page *target_page, u64 block, int bsize, + int pages, struct page **page) +{ + struct inode *i = target_page-mapping-host; + struct squashfs_cache_entry

[PATCH] Squashfs: fix failure to unlock pages on decompress error

Direct decompression into the page cache. If we fall back to using an intermediate buffer (because we cannot grab all the page cache pages) and we get a decompress fail, we forgot to release the pages. Reported-by: Roman Peniaev r.peni...@gmail.com Signed-off-by: Phillip Lougher phil

Re: [RFC PATCH] double free in decompressor.c

On 22/11/13 21:50, Geyslan Gregório Bem wrote: Coverity caught double free possibility (CID 1130962). I can patch this, but I have to know if is correct to free comp_opts in the function squashfs_decompressor_create() or it had to be done in the caller. My bet is the caller. 128void

Re: [RFC PATCH] double free in decompressor.c

On 22/11/13 21:50, Geyslan Gregório Bem wrote: Coverity caught double free possibility (CID 1130962). I can patch this, but I have to know if is correct to free comp_opts in the function squashfs_decompressor_create() or it had to be done in the caller. My bet is the caller. 128void

[GIT PULL] Squashfs updates for 3.13

): squashfs: Enhance parallel I/O Phillip Lougher (6): Squashfs: Refactor decompressor interface and code Squashfs: add multi-threaded decompression using percpu variable Squashfs: Generalise paging handling in the decompressors Squashfs: Restructure squashfs_readpage

Re: [PATCH 3/7] Squashfs: add multi-threaded decompression using percpu variables (V2)

On 20/11/13 08:33, Minchan Kim wrote: On Wed, Nov 20, 2013 at 01:48:06AM +, Phillip Lougher wrote: Add a multi-threaded decompression implementation which uses percpu variables. Using percpu variables has advantages and disadvantages over implementations which do not use percpu variables

Re: [PATCH 3/7] Squashfs: add multi-threaded decompression using percpu variables (V2)

On 20/11/13 08:33, Minchan Kim wrote: On Wed, Nov 20, 2013 at 01:48:06AM +, Phillip Lougher wrote: Add a multi-threaded decompression implementation which uses percpu variables. Using percpu variables has advantages and disadvantages over implementations which do not use percpu variables

[GIT PULL] Squashfs updates for 3.13

): squashfs: Enhance parallel I/O Phillip Lougher (6): Squashfs: Refactor decompressor interface and code Squashfs: add multi-threaded decompression using percpu variable Squashfs: Generalise paging handling in the decompressors Squashfs: Restructure squashfs_readpage

[PATCH 7/7] Squashfs: Check stream is not NULL in decompressor_multi.c

Fix static checker complaint that stream is not checked in squashfs_decompressor_destroy(). Reported-by: Dan Carpenter Signed-off-by: Phillip Lougher Reviewed-by: Minchan Kim --- fs/squashfs/decompressor_multi.c | 7 +++ 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/fs

[PATCH 4/7] Squashfs: Generalise paging handling in the decompressors

-off-by: Phillip Lougher Reviewed-by: Minchan Kim --- fs/squashfs/block.c | 27 ++ fs/squashfs/cache.c | 28 +++ fs/squashfs/decompressor.c | 14 -- fs/squashfs/decompressor.h | 5 ++-- fs

[PATCH 6/7] Squashfs: Directly decompress into the page cache for file data

dd if=file2 of=/dev/null bs=4096 & dd if=file3 of=/dev/null bs=4096 & dd if=file4 of=/dev/null bs=4096 Before: 629145600 bytes (629 MB) copied, 45.8046 s, 13.7 MB/s After: 629145600 bytes (629 MB) copied, 9.29414 s, 67.7 MB/s Signed-off-by: Phillip Lougher Reviewed-by:

[PATCH 5/7] Squashfs: Restructure squashfs_readpage()

-by: Phillip Lougher Reviewed-by: Minchan Kim --- fs/squashfs/Makefile | 2 +- fs/squashfs/file.c | 142 --- fs/squashfs/file_cache.c | 38 + fs/squashfs/squashfs.h | 7 +++ 4 files changed, 118 insertions(+), 71 deletions(-) create

[PATCH 3/7] Squashfs: add multi-threaded decompression using percpu variables (V2)

-by: Phillip Lougher --- fs/squashfs/Kconfig | 57 ++- fs/squashfs/Makefile| 10 +--- fs/squashfs/decompressor_multi_percpu.c | 98 + 3 files changed, 145 insertions(+), 20 deletions(-) create mode 100644 fs

[PATCH 2/7] squashfs: Enhance parallel I/O

of=/dev/null & dd if=test/test2.dat of=/dev/null & dd if=test/test3.dat of=/dev/null & dd if=test/test4.dat of=/dev/null & old : 1m39s -> new : 9s * From v1 * Change comp_strm with decomp_strm - Phillip * Change/add comments - Phillip Signed-off-by: Minchan Kim Signed-off-

[PATCH 1/7] Squashfs: Refactor decompressor interface and code

of the functions in decompressor_single.c Signed-off-by: Phillip Lougher Reviewed-by: Minchan Kim --- fs/squashfs/Makefile | 2 +- fs/squashfs/block.c | 11 +++-- fs/squashfs/decompressor.c| 47 ++--- fs/squashfs/decompressor.h| 21

[PATCH 0/7] Squashfs: performance enhancement patches

): squashfs: Enhance parallel I/O Phillip Lougher (6): Squashfs: Refactor decompressor interface and code Squashfs: Add multi-threaded decompression using percpu variables (V2) Squashfs: Generalise paging handling in the decompressors Squashfs: Restructure squashfs_readpage

[PATCH 0/7] Squashfs: performance enhancement patches

): squashfs: Enhance parallel I/O Phillip Lougher (6): Squashfs: Refactor decompressor interface and code Squashfs: Add multi-threaded decompression using percpu variables (V2) Squashfs: Generalise paging handling in the decompressors Squashfs: Restructure squashfs_readpage

[PATCH 1/7] Squashfs: Refactor decompressor interface and code

of the functions in decompressor_single.c Signed-off-by: Phillip Lougher phil...@squashfs.org.uk Reviewed-by: Minchan Kim minc...@kernel.org --- fs/squashfs/Makefile | 2 +- fs/squashfs/block.c | 11 +++-- fs/squashfs/decompressor.c| 47 ++--- fs

[PATCH 2/7] squashfs: Enhance parallel I/O

Signed-off-by: Phillip Lougher phil...@squashfs.org.uk --- fs/squashfs/Kconfig | 13 +++ fs/squashfs/Makefile | 9 +- fs/squashfs/decompressor_multi.c | 200 +++ 3 files changed, 221 insertions(+), 1 deletion(-) create mode 100644 fs

[PATCH 6/7] Squashfs: Directly decompress into the page cache for file data

if=file2 of=/dev/null bs=4096 dd if=file3 of=/dev/null bs=4096 dd if=file4 of=/dev/null bs=4096 Before: 629145600 bytes (629 MB) copied, 45.8046 s, 13.7 MB/s After: 629145600 bytes (629 MB) copied, 9.29414 s, 67.7 MB/s Signed-off-by: Phillip Lougher phil...@squashfs.org.uk

[PATCH 5/7] Squashfs: Restructure squashfs_readpage()

-by: Phillip Lougher phil...@squashfs.org.uk Reviewed-by: Minchan Kim minc...@kernel.org --- fs/squashfs/Makefile | 2 +- fs/squashfs/file.c | 142 --- fs/squashfs/file_cache.c | 38 + fs/squashfs/squashfs.h | 7 +++ 4 files changed

[PATCH 3/7] Squashfs: add multi-threaded decompression using percpu variables (V2)

-by: Phillip Lougher phil...@squashfs.org.uk --- fs/squashfs/Kconfig | 57 ++- fs/squashfs/Makefile| 10 +--- fs/squashfs/decompressor_multi_percpu.c | 98 + 3 files changed, 145 insertions(+), 20 deletions

[PATCH 7/7] Squashfs: Check stream is not NULL in decompressor_multi.c

Fix static checker complaint that stream is not checked in squashfs_decompressor_destroy(). Reported-by: Dan Carpenter dan.carpen...@oracle.com Signed-off-by: Phillip Lougher phil...@squashfs.org.uk Reviewed-by: Minchan Kim minc...@kernel.org --- fs/squashfs/decompressor_multi.c | 7 +++ 1

[PATCH 4/7] Squashfs: Generalise paging handling in the decompressors

-off-by: Phillip Lougher phil...@squashfs.org.uk Reviewed-by: Minchan Kim minc...@kernel.org --- fs/squashfs/block.c | 27 ++ fs/squashfs/cache.c | 28 +++ fs/squashfs/decompressor.c | 14 -- fs/squashfs

Re: [PATCH 3/6] Squashfs: add multi-threaded decompression using percpu variables

CCing Junjiro Okijima and Stephen Hemminger On 08/11/13 02:42, Minchan Kim wrote: Hello Phillip, On Thu, Nov 07, 2013 at 08:24:22PM +, Phillip Lougher wrote: Add a multi-threaded decompression implementation which uses percpu variables. Using percpu variables has advantages

Re: [PATCH 3/6] Squashfs: add multi-threaded decompression using percpu variables

CCing Junjiro Okijima and Stephen Hemminger On 08/11/13 02:42, Minchan Kim wrote: Hello Phillip, On Thu, Nov 07, 2013 at 08:24:22PM +, Phillip Lougher wrote: Add a multi-threaded decompression implementation which uses percpu variables. Using percpu variables has advantages

[PATCH 4/4] Squashfs: Check stream is not NULL in decompressor_multi.c

Fix static checker complaint that stream is not checked in squashfs_decompressor_destroy(). Reported-by: Dan Carpenter Signed-off-by: Phillip Lougher --- fs/squashfs/decompressor_multi.c |7 +++ 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/fs/squashfs

[PATCH 3/4] Squashfs: Directly decompress into the page cache for file data (V3)

mempcpy typo * squashfs_readpage_block(): Do not zero pages on decompress error * squashfs_read_cache(): free page in caller not here * page_actor.c: add comment regarding sleeping * page_actor.c: optimise code slightly Signed-off-by: Phillip Lougher --- fs/squashfs/Kconfig | 28 fs/squas

[PATCH 1/4] Squashfs: Refactor decompressor interface and code (V3)

of the functions in decompressor_single.c V3: * decompressor_single.c: Remove kfree(comp_opts) in error path * of squashfs_decompressor_create() * Double free of comp_opts found by static analysis, reported by * Dan Carpenter Signed-off-by: Phillip Lougher --- fs/squashfs/Makefile

[PATCH 2/4] Squashfs: Generalise paging handling in the decompressors (V3)

: slightly optimise code in page_actor.h Signed-off-by: Phillip Lougher --- fs/squashfs/block.c | 27 + fs/squashfs/cache.c | 28 ++ fs/squashfs/decompressor.c | 14 +++-- fs/squashfs/decompressor.h

[PATCH 0/4] Squashfs: performance improvement patches (V3)

Hi, This is a partial patchset of V3 patches following review by myself and others. Changes are described in the individual patches. There is one additional patch which fixes a static checker error. Squashfs: Refactor decompressor interface and code (V3) Squashfs: Generalise paging handling in

[PATCH 0/4] Squashfs: performance improvement patches (V3)

Hi, This is a partial patchset of V3 patches following review by myself and others. Changes are described in the individual patches. There is one additional patch which fixes a static checker error. Squashfs: Refactor decompressor interface and code (V3) Squashfs: Generalise paging handling in

[PATCH 1/4] Squashfs: Refactor decompressor interface and code (V3)

of the functions in decompressor_single.c V3: * decompressor_single.c: Remove kfree(comp_opts) in error path * of squashfs_decompressor_create() * Double free of comp_opts found by static analysis, reported by * Dan Carpenter Signed-off-by: Phillip Lougher phil...@squashfs.org.uk --- fs

[PATCH 2/4] Squashfs: Generalise paging handling in the decompressors (V3)

: slightly optimise code in page_actor.h Signed-off-by: Phillip Lougher phil...@squashfs.org.uk --- fs/squashfs/block.c | 27 + fs/squashfs/cache.c | 28 ++ fs/squashfs/decompressor.c | 14 +++-- fs/squashfs

[PATCH 4/4] Squashfs: Check stream is not NULL in decompressor_multi.c

Fix static checker complaint that stream is not checked in squashfs_decompressor_destroy(). Reported-by: Dan Carpenter dan.carpen...@oracle.com Signed-off-by: Phillip Lougher phil...@squashfs.org.uk --- fs/squashfs/decompressor_multi.c |7 +++ 1 file changed, 3 insertions(+), 4 deletions

[PATCH 3/4] Squashfs: Directly decompress into the page cache for file data (V3)

* squashfs_readpage_block(): Do not zero pages on decompress error * squashfs_read_cache(): free page in caller not here * page_actor.c: add comment regarding sleeping * page_actor.c: optimise code slightly Signed-off-by: Phillip Lougher phil...@squashfs.org.uk --- fs/squashfs/Kconfig | 28 fs

[PATCH 4/6] Squashfs: Generalise paging handling in the decompressors (V2)

update the code in decompressor_multi*.c Signed-off-by: Phillip Lougher --- fs/squashfs/block.c | 27 +--- fs/squashfs/cache.c | 28 +--- fs/squashfs/decompressor.c | 14 ++-- fs/squashfs/decompressor.h

[PATCH 6/6] Squashfs: Directly decompress into the page cache for file data (V2)

o grab pages could be because we've been VM reclaimed, but the other pages are still in the page cache and uptodate. * Make Kconfig option a choice, making the either-other nature of the option more explicit, and also tidying up the ifdef in the Makefile Signed-off-by: Phillip Loug

[PATCH 1/6] Squashfs: Refactor decompressor interface and code (V2)

offset, - length, srclength, pages); + return msblk->decompressor->comp_opts ? + msblk->decompressor->comp_opts(msblk, buff, length) : NULL; } #ifdef CONFIG_SQUASHFS_XZ diff --git a/fs/squashfs/decompressor_single.c b/fs/squashfs/decompressor_sing

[PATCH 2/6] Squashfs: enhance parallel I/O

of=/dev/null & dd if=test/test2.dat of=/dev/null & dd if=test/test3.dat of=/dev/null & dd if=test/test4.dat of=/dev/null & old : 1m39s -> new : 9s * From v1 * Change comp_strm with decomp_strm - Phillip * Change/add comments - Phillip Signed-off-by: Minchan Kim Signed-off-

[PATCH 3/6] Squashfs: add multi-threaded decompression using percpu variables

cores. Disadvantages: it limits decompression to one thread per core. Signed-off-by: Phillip Lougher --- fs/squashfs/Kconfig | 57 + fs/squashfs/Makefile| 10 +-- fs/squashfs/decompressor_multi_percpu.c | 105

[PATCH 5/6] Squashfs: restructure squashfs_readpage()

-by: Phillip Lougher --- fs/squashfs/Makefile |2 +- fs/squashfs/file.c | 142 +++--- fs/squashfs/file_cache.c | 38 + fs/squashfs/squashfs.h |7 +++ 4 files changed, 118 insertions(+), 71 deletions(-) create mode 100644 fs

[PATCH 0/6] Squashfs performance improvements

. Minchan Kim (1): Squashfs: enhance parallel I/O Phillip Lougher (5): Squashfs: Refactor decompressor interface and code (V2) Squashfs: add multi-threaded decompression using percpu variables Squashfs: Generalise paging handling

[PATCH 0/6] Squashfs performance improvements

. Minchan Kim (1): Squashfs: enhance parallel I/O Phillip Lougher (5): Squashfs: Refactor decompressor interface and code (V2) Squashfs: add multi-threaded decompression using percpu variables Squashfs: Generalise paging handling

[PATCH 5/6] Squashfs: restructure squashfs_readpage()

-by: Phillip Lougher phil...@squashfs.org.uk --- fs/squashfs/Makefile |2 +- fs/squashfs/file.c | 142 +++--- fs/squashfs/file_cache.c | 38 + fs/squashfs/squashfs.h |7 +++ 4 files changed, 118 insertions(+), 71 deletions(-) create

[PATCH 3/6] Squashfs: add multi-threaded decompression using percpu variables

cores. Disadvantages: it limits decompression to one thread per core. Signed-off-by: Phillip Lougher phil...@squashfs.org.uk --- fs/squashfs/Kconfig | 57 + fs/squashfs/Makefile| 10 +-- fs/squashfs/decompressor_multi_percpu.c | 105

  1   2   3   >