RE: [PATCH] staging: erofs: fix an error handling in erofs_readdir()
> Subject: Re: [PATCH] staging: erofs: fix an error handling in erofs_readdir() > > Hi Philip, > > On Tue, Aug 20, 2019 at 02:50:38PM +0800, Philip Li wrote: > > On Sun, Aug 18, 2019 at 09:25:04PM +0800, Gao Xiang wrote: > > > On Sun, Aug 18, 2019 at 09:17:52PM +0800, kbuild test robot wrote: > > > > Hi Gao, > > > > > > > > I love your patch! Yet something to improve: > > > > > > > > [auto build test ERROR on linus/master] > > > > [cannot apply to v5.3-rc4 next-20190816] > > > > [if your patch is applied to the wrong git tree, please drop us a note > > > > to help > improve the system] > > > > > > ... those patches should be applied to staging tree > > > since linux-next has not been updated yet... > > thanks for the feedback, we will consider this to our todo list. > > Yes, many confusing reports anyway... > (Just my personal suggestion, maybe we can add some hints on the patch email > to indicate which tree can be applied successfully for ci in the future...) thanks, this is good idea. On the other side, we support to add --base option to git format-patch to automatically suggest the base, refer to https://stackoverflow.com/a/37406982 for detail. Meanwhile, we will enhance the internal logic to find suitable base if possible. > > Thanks, > Gao Xiang > > > > > > > > > Thanks, > > > Gao Xiang > > > > > > > > > > > url: > > > > https://github.com/0day-ci/linux/commits/Gao-Xiang/staging-erofs-fix- > an-error-handling-in-erofs_readdir/20190818-191344 > > > > config: arm64-allyesconfig (attached as .config) > > > > compiler: aarch64-linux-gcc (GCC) 7.4.0 > > > > reproduce: > > > > wget https://raw.githubusercontent.com/intel/lkp- > tests/master/sbin/make.cross -O ~/bin/make.cross > > > > chmod +x ~/bin/make.cross > > > > # save the attached .config to linux build tree > > > > GCC_VERSION=7.4.0 make.cross ARCH=arm64 > > > > > > > > If you fix the issue, kindly add following tag > > > > Reported-by: kbuild test robot > > > > > > > > All errors (new ones prefixed by >>): > > > > > > > >drivers/staging/erofs/dir.c: In function 'erofs_readdir': > > > > >> drivers/staging/erofs/dir.c:110:11: error: 'EFSCORRUPTED' undeclared > (first use in this function); did you mean 'FS_NRSUPER'? > > > >err = -EFSCORRUPTED; > > > > ^~~~ > > > > FS_NRSUPER > > > >drivers/staging/erofs/dir.c:110:11: note: each undeclared identifier > > > > is > reported only once for each function it appears in > > > > > > > > vim +110 drivers/staging/erofs/dir.c > > > > > > > > 85 > > > > 86 static int erofs_readdir(struct file *f, struct dir_context > > > > *ctx) > > > > 87 { > > > > 88 struct inode *dir = file_inode(f); > > > > 89 struct address_space *mapping = dir->i_mapping; > > > > 90 const size_t dirsize = i_size_read(dir); > > > > 91 unsigned int i = ctx->pos / EROFS_BLKSIZ; > > > > 92 unsigned int ofs = ctx->pos % EROFS_BLKSIZ; > > > > 93 int err = 0; > > > > 94 bool initial = true; > > > > 95 > > > > 96 while (ctx->pos < dirsize) { > > > > 97 struct page *dentry_page; > > > > 98 struct erofs_dirent *de; > > > > 99 unsigned int nameoff, maxsize; > > > >100 > > > >101 dentry_page = read_mapping_page(mapping, i, > NULL); > > > >102 if (dentry_page == ERR_PTR(-ENOMEM)) { > > > >103 errln("no memory to readdir of logical > block %u of nid %llu", > > > >104i, EROFS_V(dir)->nid); > > > >105 err = -ENOMEM; > > > >106 break; > > > >107 } else if (IS_ERR(dentry_page)) { > > > >108 errln("fail to readdir of logical block > > > > %u of > nid %llu", > > > >109i, EROFS_V(dir)->nid); > > > > > 110 err = -EFSCORRUPTED; > > > >111 break; > > > >112 } > > > >113 > > > >114 de = (struct erofs_dirent *)kmap(dentry_page); > > > >115 > > > >116 nameoff = le16_to_cpu(de->nameoff); > > > >117 > > > >118 if (unlikely(nameoff < sizeof(struct > > > > erofs_dirent) || > > > >119 nameoff >= PAGE_SIZE)) { > > > >120 errln("%s, invalid de[0].nameoff %u", > > > >121__func__, nameoff); > > > >122 > > > >123 err = -EIO; > > > >124 goto skip_this; > > > >125 } > > > >126 > > > >127 maxsize = min_t(unsigned int, > > > >128 dirsize - ctx->pos + ofs, > PAGE
Re: [PATCH] staging: erofs: fix an error handling in erofs_readdir()
Hi Philip, On Tue, Aug 20, 2019 at 02:50:38PM +0800, Philip Li wrote: > On Sun, Aug 18, 2019 at 09:25:04PM +0800, Gao Xiang wrote: > > On Sun, Aug 18, 2019 at 09:17:52PM +0800, kbuild test robot wrote: > > > Hi Gao, > > > > > > I love your patch! Yet something to improve: > > > > > > [auto build test ERROR on linus/master] > > > [cannot apply to v5.3-rc4 next-20190816] > > > [if your patch is applied to the wrong git tree, please drop us a note to > > > help improve the system] > > > > ... those patches should be applied to staging tree > > since linux-next has not been updated yet... > thanks for the feedback, we will consider this to our todo list. Yes, many confusing reports anyway... (Just my personal suggestion, maybe we can add some hints on the patch email to indicate which tree can be applied successfully for ci in the future...) Thanks, Gao Xiang > > > > > Thanks, > > Gao Xiang > > > > > > > > url: > > > https://github.com/0day-ci/linux/commits/Gao-Xiang/staging-erofs-fix-an-error-handling-in-erofs_readdir/20190818-191344 > > > config: arm64-allyesconfig (attached as .config) > > > compiler: aarch64-linux-gcc (GCC) 7.4.0 > > > reproduce: > > > wget > > > https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross > > > -O ~/bin/make.cross > > > chmod +x ~/bin/make.cross > > > # save the attached .config to linux build tree > > > GCC_VERSION=7.4.0 make.cross ARCH=arm64 > > > > > > If you fix the issue, kindly add following tag > > > Reported-by: kbuild test robot > > > > > > All errors (new ones prefixed by >>): > > > > > >drivers/staging/erofs/dir.c: In function 'erofs_readdir': > > > >> drivers/staging/erofs/dir.c:110:11: error: 'EFSCORRUPTED' undeclared > > > >> (first use in this function); did you mean 'FS_NRSUPER'? > > >err = -EFSCORRUPTED; > > > ^~~~ > > > FS_NRSUPER > > >drivers/staging/erofs/dir.c:110:11: note: each undeclared identifier > > > is reported only once for each function it appears in > > > > > > vim +110 drivers/staging/erofs/dir.c > > > > > > 85 > > > 86static int erofs_readdir(struct file *f, struct dir_context > > > *ctx) > > > 87{ > > > 88struct inode *dir = file_inode(f); > > > 89struct address_space *mapping = dir->i_mapping; > > > 90const size_t dirsize = i_size_read(dir); > > > 91unsigned int i = ctx->pos / EROFS_BLKSIZ; > > > 92unsigned int ofs = ctx->pos % EROFS_BLKSIZ; > > > 93int err = 0; > > > 94bool initial = true; > > > 95 > > > 96while (ctx->pos < dirsize) { > > > 97struct page *dentry_page; > > > 98struct erofs_dirent *de; > > > 99unsigned int nameoff, maxsize; > > >100 > > >101dentry_page = read_mapping_page(mapping, i, > > > NULL); > > >102if (dentry_page == ERR_PTR(-ENOMEM)) { > > >103errln("no memory to readdir of logical > > > block %u of nid %llu", > > >104 i, EROFS_V(dir)->nid); > > >105err = -ENOMEM; > > >106break; > > >107} else if (IS_ERR(dentry_page)) { > > >108errln("fail to readdir of logical block > > > %u of nid %llu", > > >109 i, EROFS_V(dir)->nid); > > > > 110err = -EFSCORRUPTED; > > >111break; > > >112} > > >113 > > >114de = (struct erofs_dirent *)kmap(dentry_page); > > >115 > > >116nameoff = le16_to_cpu(de->nameoff); > > >117 > > >118if (unlikely(nameoff < sizeof(struct > > > erofs_dirent) || > > >119 nameoff >= PAGE_SIZE)) { > > >120errln("%s, invalid de[0].nameoff %u", > > >121 __func__, nameoff); > > >122 > > >123err = -EIO; > > >124goto skip_this; > > >125} > > >126 > > >127maxsize = min_t(unsigned int, > > >128dirsize - ctx->pos + ofs, > > > PAGE_SIZE); > > >129 > > >130/* search dirents at the arbitrary position */ > > >131if (unlikely(initial)) { > > >132initial = false; > > >133 > > >134ofs = roundup(ofs, sizeof(struct > > > erofs_dirent)); > > >135if (unlikely(ofs >= nameoff)) > > >136
Re: [PATCH] staging: erofs: fix an error handling in erofs_readdir()
On Sun, Aug 18, 2019 at 09:25:04PM +0800, Gao Xiang wrote: > On Sun, Aug 18, 2019 at 09:17:52PM +0800, kbuild test robot wrote: > > Hi Gao, > > > > I love your patch! Yet something to improve: > > > > [auto build test ERROR on linus/master] > > [cannot apply to v5.3-rc4 next-20190816] > > [if your patch is applied to the wrong git tree, please drop us a note to > > help improve the system] > > ... those patches should be applied to staging tree > since linux-next has not been updated yet... thanks for the feedback, we will consider this to our todo list. > > Thanks, > Gao Xiang > > > > > url: > > https://github.com/0day-ci/linux/commits/Gao-Xiang/staging-erofs-fix-an-error-handling-in-erofs_readdir/20190818-191344 > > config: arm64-allyesconfig (attached as .config) > > compiler: aarch64-linux-gcc (GCC) 7.4.0 > > reproduce: > > wget > > https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O > > ~/bin/make.cross > > chmod +x ~/bin/make.cross > > # save the attached .config to linux build tree > > GCC_VERSION=7.4.0 make.cross ARCH=arm64 > > > > If you fix the issue, kindly add following tag > > Reported-by: kbuild test robot > > > > All errors (new ones prefixed by >>): > > > >drivers/staging/erofs/dir.c: In function 'erofs_readdir': > > >> drivers/staging/erofs/dir.c:110:11: error: 'EFSCORRUPTED' undeclared > > >> (first use in this function); did you mean 'FS_NRSUPER'? > >err = -EFSCORRUPTED; > > ^~~~ > > FS_NRSUPER > >drivers/staging/erofs/dir.c:110:11: note: each undeclared identifier is > > reported only once for each function it appears in > > > > vim +110 drivers/staging/erofs/dir.c > > > > 85 > > 86 static int erofs_readdir(struct file *f, struct dir_context > > *ctx) > > 87 { > > 88 struct inode *dir = file_inode(f); > > 89 struct address_space *mapping = dir->i_mapping; > > 90 const size_t dirsize = i_size_read(dir); > > 91 unsigned int i = ctx->pos / EROFS_BLKSIZ; > > 92 unsigned int ofs = ctx->pos % EROFS_BLKSIZ; > > 93 int err = 0; > > 94 bool initial = true; > > 95 > > 96 while (ctx->pos < dirsize) { > > 97 struct page *dentry_page; > > 98 struct erofs_dirent *de; > > 99 unsigned int nameoff, maxsize; > >100 > >101 dentry_page = read_mapping_page(mapping, i, > > NULL); > >102 if (dentry_page == ERR_PTR(-ENOMEM)) { > >103 errln("no memory to readdir of logical > > block %u of nid %llu", > >104i, EROFS_V(dir)->nid); > >105 err = -ENOMEM; > >106 break; > >107 } else if (IS_ERR(dentry_page)) { > >108 errln("fail to readdir of logical block > > %u of nid %llu", > >109i, EROFS_V(dir)->nid); > > > 110 err = -EFSCORRUPTED; > >111 break; > >112 } > >113 > >114 de = (struct erofs_dirent *)kmap(dentry_page); > >115 > >116 nameoff = le16_to_cpu(de->nameoff); > >117 > >118 if (unlikely(nameoff < sizeof(struct > > erofs_dirent) || > >119 nameoff >= PAGE_SIZE)) { > >120 errln("%s, invalid de[0].nameoff %u", > >121__func__, nameoff); > >122 > >123 err = -EIO; > >124 goto skip_this; > >125 } > >126 > >127 maxsize = min_t(unsigned int, > >128 dirsize - ctx->pos + ofs, > > PAGE_SIZE); > >129 > >130 /* search dirents at the arbitrary position */ > >131 if (unlikely(initial)) { > >132 initial = false; > >133 > >134 ofs = roundup(ofs, sizeof(struct > > erofs_dirent)); > >135 if (unlikely(ofs >= nameoff)) > >136 goto skip_this; > >137 } > >138 > >139 err = erofs_fill_dentries(ctx, de, &ofs, > > nameoff, maxsize); > >140 skip_this: > >141 kunmap(dentry_page); > >142 > >143 put_page(dentry_page); > >144 > >145 c
Re: [PATCH] erofs: move erofs out of staging
Hi Qu, On Tue, Aug 20, 2019 at 02:04:46PM +0800, Qu Wenruo wrote: > [...] > > And performance is another point. > That tree-checker in btrfs is as fast/slow as CRC32. > Not sure how it would be for dm-verity, but I guess it's slower than > CRC32 if using any strong hash. Just a word, dm-verity can do simple CRC32 without hash tree in priciple as well. It is only a time problem. > > Anyway, for a RO fs, if it's relying on dm-verify then that's OK for > real-world usage. > But as a standalone fs, even it's RO, a verification layer would be a > great plus. Yes, again, it's on my schedule at least. > > At least when new student developers try fuzzed images on the fs, it > would be a good surprise other than tons of new bug reports. > > > > > > >>> > [...] > >>> > >>> Yes, we will do such a debugging tool of course. Actually Li Guifu is now > >>> developping a erofs-fuse to support old linux versions or other OSes for > >>> archiveing only use, we will base on that code to develop a better fuzzer > >>> tool as well. > >> > >> Personally speaking, debugging tool is way more important than a running > >> kernel module/fuse. > >> It's human trying to write the code, most of time is spent educating > >> code readers, thus debugging tool is way more important than dead cold > >> code. > > > > Agree, Xiang and I have no time to developing this feature now, we are glad > > very much if you could help > > us to do it ;) > > In fact, since the fs is a RO fs, it could be pretty good educational > example for any fs newbies. Thus a debug tool which can show the full > metadata of the fs can really be helpful. Yes, I think EROFS will be helpful for all fs newbies to understand the basic fs concepts, and I have received contributions from some new kernel developers. And again, I will do in my spare time. I'm dedicated to playing with EROFS in my spare time (rather than only at work to maintain for our products), so I will do and don't worry about that. And I agree with Miao Xie on one word, you can join us as well for educational reason or whatever if you have some extra time... :) Thanks, Gao Xiang > > In fact, btrfs-debug-tree (now "btrfs ins dump-tree") leads my way to > btrfs, and still one of my favourite tool to debug. > > Thanks, > Qu > > > ___ > devel mailing list > de...@linuxdriverproject.org > http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel ___ devel mailing list de...@linuxdriverproject.org http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel
Re: [PATCH] erofs: move erofs out of staging
[...] >> The same tool exists for btrfs, although lacks the write ability, but >> that dump is more comprehensive and a great tool to learn the on-disk >> format. >> >> >> And for the fuzzing defending part, just a few kernel releases ago, >> there is none for btrfs, and now we have a full static verification >> layer to cover (almost) all on-disk data at read and write time. >> (Along with enhanced runtime check) >> >> We have covered from vague values inside tree blocks and invalid/missing >> cross-ref find at runtime. >> >> Currently the two layered check works pretty fine (well, sometimes too >> good to detect older, improper behaved kernel). >> - Tree blocks with vague data just get rejected by verification layer >> So that all members should fit on-disk format, from alignment to >> generation to inode mode. >> >> The error will trigger a good enough (TM) error message for developer >> to read, and if we have other copies, we retry other copies just as >> we hit a bad copy. >> >> - At runtime, we have much less to check >> Only cross-ref related things can be wrong now. since everything >> inside a single tree block has already be checked. >> >> In fact, from my respect of view, such read time check should be there >> from the very beginning. >> It acts kinda of a on-disk format spec. (In fact, by implementing the >> verification layer itself, it already exposes a lot of btrfs design >> trade-offs) >> >> Even for a fs as complex (buggy) as btrfs, we only take 1K lines to >> implement the verification layer. >> So I'd like to see every new mainlined fs to have such ability. > > It is a good idea. In fact, we already have a verification layer which was > implemented > as a device mapper sub-module. I think it is enough for a read-only > filesystem because > it is simple, flexible and independent(we can modify the filesystem layout > without > verification module modification). If you're talking about dm-verity, then IMHO they are with completely different objective. For dm-verity it's more like authentication. Without proper key (authentication), no one can modify the data without being caught. That's why I hate such thing, it's not open at all, *as bad as locked bootloader*. While the tree-checker (the layer in btrfs) is more like a sensitive and sometimes overreacting detector, find anything wrong, then reject the offending tree block. The original objective of tree-checker is to free coder from defensive coding, providing a centralized verification service, thus we don't need to verify tree blocks randomly using ugly BUG_ON()s. (But unfortunately, a lot of BUG_ON()s are still kept as is, as it takes more time to persuade reviewers that those BUG_ON()s are impossible to hit anymore) Tree-checker can not only detect suspicious metadata (either caused by mem bit flip or poorly crafted image), but also bad *kernel* behavior or even runtime bitflip. (Well, it only works for RW fs, so not really helpful for a RO fs). And performance is another point. That tree-checker in btrfs is as fast/slow as CRC32. Not sure how it would be for dm-verity, but I guess it's slower than CRC32 if using any strong hash. Anyway, for a RO fs, if it's relying on dm-verify then that's OK for real-world usage. But as a standalone fs, even it's RO, a verification layer would be a great plus. At least when new student developers try fuzzed images on the fs, it would be a good surprise other than tons of new bug reports. > > >>> [...] >>> >>> Yes, we will do such a debugging tool of course. Actually Li Guifu is now >>> developping a erofs-fuse to support old linux versions or other OSes for >>> archiveing only use, we will base on that code to develop a better fuzzer >>> tool as well. >> >> Personally speaking, debugging tool is way more important than a running >> kernel module/fuse. >> It's human trying to write the code, most of time is spent educating >> code readers, thus debugging tool is way more important than dead cold code. > > Agree, Xiang and I have no time to developing this feature now, we are glad > very much if you could help > us to do it ;) In fact, since the fs is a RO fs, it could be pretty good educational example for any fs newbies. Thus a debug tool which can show the full metadata of the fs can really be helpful. In fact, btrfs-debug-tree (now "btrfs ins dump-tree") leads my way to btrfs, and still one of my favourite tool to debug. Thanks, Qu signature.asc Description: OpenPGP digital signature ___ devel mailing list de...@linuxdriverproject.org http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel
Re: [PATCH] erofs: move erofs out of staging
On Tue, Aug 20, 2019 at 11:33:51AM +0800, Miao Xie wrote: > > > on 2019/8/20 at 8:55, Qu Wenruo wrote: > > [...] > I have made a simple fuzzer to inject messy in inode metadata, > dir data, compressed indexes and super block, > https://git.kernel.org/pub/scm/linux/kernel/git/xiang/erofs-utils.git/commit/?h=experimental-fuzzer > > I am testing with some given dirs and the following script. > Does it look reasonable? > > # !/bin/bash > > mkdir -p mntdir > > for ((i=0; i<1000; ++i)); do > mkfs/mkfs.erofs -F$i testdir_fsl.fuzz.img testdir_fsl > /dev/null 2>&1 > >>> > >>> mkfs fuzzes the image? Er > >> > >> Thanks for your reply. > >> > >> First, This is just the first step of erofs fuzzer I wrote yesterday > >> night... > >> > >>> > >>> Over in XFS land we have an xfs debugging tool (xfs_db) that knows how > >>> to dump (and write!) most every field of every metadata type. This > >>> makes it fairly easy to write systematic level 0 fuzzing tests that > >>> check how well the filesystem reacts to garbage data (zeroing, > >>> randomizing, oneing, adding and subtracting small integers) in a field. > >>> (It also knows how to trash entire blocks.) > > > > The same tool exists for btrfs, although lacks the write ability, but > > that dump is more comprehensive and a great tool to learn the on-disk > > format. > > > > > > And for the fuzzing defending part, just a few kernel releases ago, > > there is none for btrfs, and now we have a full static verification > > layer to cover (almost) all on-disk data at read and write time. > > (Along with enhanced runtime check) > > > > We have covered from vague values inside tree blocks and invalid/missing > > cross-ref find at runtime. > > > > Currently the two layered check works pretty fine (well, sometimes too > > good to detect older, improper behaved kernel). > > - Tree blocks with vague data just get rejected by verification layer > > So that all members should fit on-disk format, from alignment to > > generation to inode mode. > > > > The error will trigger a good enough (TM) error message for developer > > to read, and if we have other copies, we retry other copies just as > > we hit a bad copy. > > > > - At runtime, we have much less to check > > Only cross-ref related things can be wrong now. since everything > > inside a single tree block has already be checked. > > > > In fact, from my respect of view, such read time check should be there > > from the very beginning. > > It acts kinda of a on-disk format spec. (In fact, by implementing the > > verification layer itself, it already exposes a lot of btrfs design > > trade-offs) > > > > Even for a fs as complex (buggy) as btrfs, we only take 1K lines to > > implement the verification layer. > > So I'd like to see every new mainlined fs to have such ability. > > It is a good idea. In fact, we already have a verification layer which was > implemented > as a device mapper sub-module. I think it is enough for a read-only > filesystem because > it is simple, flexible and independent(we can modify the filesystem layout > without > verification module modification). > > > >> > >> Actually, compared with XFS, EROFS has rather simple on-disk format. > >> What we inject one time is quite deterministic. > >> > >> The first step just purposely writes some random fuzzed data to > >> the base inode metadata, compressed indexes, or dir data field > >> (one round one field) to make it validity and coverability. > >> > >>> > >>> You might want to write such a debugging tool for erofs so that you can > >>> take apart crashed images to get a better idea of what went wrong, and > >>> to write easy fuzzing tests. > >> > >> Yes, we will do such a debugging tool of course. Actually Li Guifu is now > >> developping a erofs-fuse to support old linux versions or other OSes for > >> archiveing only use, we will base on that code to develop a better fuzzer > >> tool as well. > > > > Personally speaking, debugging tool is way more important than a running > > kernel module/fuse. > > It's human trying to write the code, most of time is spent educating > > code readers, thus debugging tool is way more important than dead cold code. > > Agree, Xiang and I have no time to developing this feature now, we are glad > very much if you could help > us to do it ;) I can speed all my spare time for this... As I said before, All HUAWEI smartphone products will continue using this filesystem, and maintaining this filesystem is one of our paid jobs, but since our Android products is based on dm-verity + EROFS, it's only on my personal time schedule (bosses care more about Android and money) and I will do that in my spare time of course. Thanks, Gao Xiang > > Thanks > Miao > > > > > Thanks, > > Qu > >> > >> Thanks, > >> Gao Xiang > >> > >>> > >>> --D > >>> > umount mntdir > mount -t erofs -o loop testdir_fsl.fuzz.img mntdir > for
Re: [PATCH] erofs: move erofs out of staging
on 2019/8/20 at 8:55, Qu Wenruo wrote: > [...] I have made a simple fuzzer to inject messy in inode metadata, dir data, compressed indexes and super block, https://git.kernel.org/pub/scm/linux/kernel/git/xiang/erofs-utils.git/commit/?h=experimental-fuzzer I am testing with some given dirs and the following script. Does it look reasonable? # !/bin/bash mkdir -p mntdir for ((i=0; i<1000; ++i)); do mkfs/mkfs.erofs -F$i testdir_fsl.fuzz.img testdir_fsl > /dev/null 2>&1 >>> >>> mkfs fuzzes the image? Er >> >> Thanks for your reply. >> >> First, This is just the first step of erofs fuzzer I wrote yesterday night... >> >>> >>> Over in XFS land we have an xfs debugging tool (xfs_db) that knows how >>> to dump (and write!) most every field of every metadata type. This >>> makes it fairly easy to write systematic level 0 fuzzing tests that >>> check how well the filesystem reacts to garbage data (zeroing, >>> randomizing, oneing, adding and subtracting small integers) in a field. >>> (It also knows how to trash entire blocks.) > > The same tool exists for btrfs, although lacks the write ability, but > that dump is more comprehensive and a great tool to learn the on-disk > format. > > > And for the fuzzing defending part, just a few kernel releases ago, > there is none for btrfs, and now we have a full static verification > layer to cover (almost) all on-disk data at read and write time. > (Along with enhanced runtime check) > > We have covered from vague values inside tree blocks and invalid/missing > cross-ref find at runtime. > > Currently the two layered check works pretty fine (well, sometimes too > good to detect older, improper behaved kernel). > - Tree blocks with vague data just get rejected by verification layer > So that all members should fit on-disk format, from alignment to > generation to inode mode. > > The error will trigger a good enough (TM) error message for developer > to read, and if we have other copies, we retry other copies just as > we hit a bad copy. > > - At runtime, we have much less to check > Only cross-ref related things can be wrong now. since everything > inside a single tree block has already be checked. > > In fact, from my respect of view, such read time check should be there > from the very beginning. > It acts kinda of a on-disk format spec. (In fact, by implementing the > verification layer itself, it already exposes a lot of btrfs design > trade-offs) > > Even for a fs as complex (buggy) as btrfs, we only take 1K lines to > implement the verification layer. > So I'd like to see every new mainlined fs to have such ability. It is a good idea. In fact, we already have a verification layer which was implemented as a device mapper sub-module. I think it is enough for a read-only filesystem because it is simple, flexible and independent(we can modify the filesystem layout without verification module modification). >> >> Actually, compared with XFS, EROFS has rather simple on-disk format. >> What we inject one time is quite deterministic. >> >> The first step just purposely writes some random fuzzed data to >> the base inode metadata, compressed indexes, or dir data field >> (one round one field) to make it validity and coverability. >> >>> >>> You might want to write such a debugging tool for erofs so that you can >>> take apart crashed images to get a better idea of what went wrong, and >>> to write easy fuzzing tests. >> >> Yes, we will do such a debugging tool of course. Actually Li Guifu is now >> developping a erofs-fuse to support old linux versions or other OSes for >> archiveing only use, we will base on that code to develop a better fuzzer >> tool as well. > > Personally speaking, debugging tool is way more important than a running > kernel module/fuse. > It's human trying to write the code, most of time is spent educating > code readers, thus debugging tool is way more important than dead cold code. Agree, Xiang and I have no time to developing this feature now, we are glad very much if you could help us to do it ;) Thanks Miao > > Thanks, > Qu >> >> Thanks, >> Gao Xiang >> >>> >>> --D >>> umount mntdir mount -t erofs -o loop testdir_fsl.fuzz.img mntdir for j in `find mntdir -type f`; do md5sum $j > /dev/null done done Thanks, Gao Xiang > > Thanks, > Gao Xiang > > ___ devel mailing list de...@linuxdriverproject.org http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel
Re: [PATCH] erofs: move erofs out of staging
On 2019/8/20 上午10:24, Chao Yu wrote: > On 2019/8/20 8:55, Qu Wenruo wrote: >> [...] > I have made a simple fuzzer to inject messy in inode metadata, > dir data, compressed indexes and super block, > https://git.kernel.org/pub/scm/linux/kernel/git/xiang/erofs-utils.git/commit/?h=experimental-fuzzer > > I am testing with some given dirs and the following script. > Does it look reasonable? > > # !/bin/bash > > mkdir -p mntdir > > for ((i=0; i<1000; ++i)); do > mkfs/mkfs.erofs -F$i testdir_fsl.fuzz.img testdir_fsl > /dev/null 2>&1 mkfs fuzzes the image? Er >>> >>> Thanks for your reply. >>> >>> First, This is just the first step of erofs fuzzer I wrote yesterday >>> night... >>> Over in XFS land we have an xfs debugging tool (xfs_db) that knows how to dump (and write!) most every field of every metadata type. This makes it fairly easy to write systematic level 0 fuzzing tests that check how well the filesystem reacts to garbage data (zeroing, randomizing, oneing, adding and subtracting small integers) in a field. (It also knows how to trash entire blocks.) >> >> The same tool exists for btrfs, although lacks the write ability, but >> that dump is more comprehensive and a great tool to learn the on-disk >> format. >> >> >> And for the fuzzing defending part, just a few kernel releases ago, >> there is none for btrfs, and now we have a full static verification >> layer to cover (almost) all on-disk data at read and write time. >> (Along with enhanced runtime check) >> >> We have covered from vague values inside tree blocks and invalid/missing >> cross-ref find at runtime. >> >> Currently the two layered check works pretty fine (well, sometimes too >> good to detect older, improper behaved kernel). >> - Tree blocks with vague data just get rejected by verification layer >> So that all members should fit on-disk format, from alignment to >> generation to inode mode. >> >> The error will trigger a good enough (TM) error message for developer >> to read, and if we have other copies, we retry other copies just as >> we hit a bad copy. >> >> - At runtime, we have much less to check >> Only cross-ref related things can be wrong now. since everything >> inside a single tree block has already be checked. >> >> In fact, from my respect of view, such read time check should be there >> from the very beginning. >> It acts kinda of a on-disk format spec. (In fact, by implementing the >> verification layer itself, it already exposes a lot of btrfs design >> trade-offs) >> >> Even for a fs as complex (buggy) as btrfs, we only take 1K lines to >> implement the verification layer. >> So I'd like to see every new mainlined fs to have such ability. > > Out of curiosity, it looks like every mainstream filesystem has its own > fuzz/injection tool in their tool-set, if it's really such a generic > requirement, why shouldn't there be a common tool to handle that, let > specified > filesystem fill the tool's callback to seek a node/block and supported fields > can be fuzzed in inode. It could be possible for XFS/EXT* to share the same infrastructure without much hassle. (If not considering external journal) But for btrfs, it's like a regular fs on a super large dm-linear, which further builds its chunks on different dm-raid1/dm-linear/dm-raid56. So not sure if it's possible for btrfs, as it contains its logical address layer bytenr (the most common one) along with per-chunk physical mapping bytenr (in another tree). It may depends on the granularity. But definitely a good idea to do so in a generic way. Currently we depend on super kind student developers/reporters on such fuzzed images, and developers sometimes get inspired by real world corruption (or his/her mood) to add some valid but hard-to-hit corner case check. Thanks, Qu > It can help to avoid redundant work whenever Linux > welcomes a new filesystem > > Thanks, > >> >>> >>> Actually, compared with XFS, EROFS has rather simple on-disk format. >>> What we inject one time is quite deterministic. >>> >>> The first step just purposely writes some random fuzzed data to >>> the base inode metadata, compressed indexes, or dir data field >>> (one round one field) to make it validity and coverability. >>> You might want to write such a debugging tool for erofs so that you can take apart crashed images to get a better idea of what went wrong, and to write easy fuzzing tests. >>> >>> Yes, we will do such a debugging tool of course. Actually Li Guifu is now >>> developping a erofs-fuse to support old linux versions or other OSes for >>> archiveing only use, we will base on that code to develop a better fuzzer >>> tool as well. >> >> Personally speaking, debugging tool is way more important than a running >> kernel module/fuse. >> It's human trying to write the code, most of time is spent educating >> code readers, thus debugging tool is
Re: [RFC PATCH 08/15] drivers/acrn: add VM memory management for ACRN char device
On 2019年08月19日 15:39, Dan Carpenter wrote: On Mon, Aug 19, 2019 at 01:32:54PM +0800, Zhao, Yakui wrote: In fact as this driver is mainly used for embedded IOT usage, it doesn't handle the complex cleanup when such error is encountered. Instead the clean up is handled in free_guest_vm. A use after free here seems like a potential security problem. Security matters for IoT... :( Thanks for pointing out the issue. The cleanup will be considered carefully. regards, dan carpenter ___ devel mailing list de...@linuxdriverproject.org http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel
Re: [RFC PATCH 15/15] drivers/acrn: add the support of offline SOS cpu
On 2019年08月19日 18:34, Dan Carpenter wrote: On Fri, Aug 16, 2019 at 10:25:56AM +0800, Zhao Yakui wrote: diff --git a/drivers/staging/acrn/acrn_dev.c b/drivers/staging/acrn/acrn_dev.c index 0602125..6868003 100644 --- a/drivers/staging/acrn/acrn_dev.c +++ b/drivers/staging/acrn/acrn_dev.c @@ -588,6 +588,41 @@ static const struct file_operations fops = { #define SUPPORT_HV_API_VERSION_MAJOR 1 #define SUPPORT_HV_API_VERSION_MINOR 0 +static ssize_t +offline_cpu_store(struct device *dev, + struct device_attribute *attr, + const char *buf, size_t count) +{ +#ifdef CONFIG_X86 + u64 cpu, lapicid; + + if (kstrtoull(buf, 0, &cpu) < 0) + return -EINVAL; Thanks for the review. Make sense. The error code will be preserved. Preserve the error code. ret = kstrtoull(buf, 0, &cpu); if (ret) return ret; + + if (cpu_possible(cpu)) { You can't pass unchecked cpu values to cpu_possible() or it results in an out of bounds read if cpu is >= than nr_cpu_ids. OK. It will add the check of "cpu < num_possibles_cpu()" to avoid the out of bounds. regards, dan carpenter ___ devel mailing list de...@linuxdriverproject.org http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel
Re: [PATCH] erofs: move erofs out of staging
On 2019/8/20 8:55, Qu Wenruo wrote: > [...] I have made a simple fuzzer to inject messy in inode metadata, dir data, compressed indexes and super block, https://git.kernel.org/pub/scm/linux/kernel/git/xiang/erofs-utils.git/commit/?h=experimental-fuzzer I am testing with some given dirs and the following script. Does it look reasonable? # !/bin/bash mkdir -p mntdir for ((i=0; i<1000; ++i)); do mkfs/mkfs.erofs -F$i testdir_fsl.fuzz.img testdir_fsl > /dev/null 2>&1 >>> >>> mkfs fuzzes the image? Er >> >> Thanks for your reply. >> >> First, This is just the first step of erofs fuzzer I wrote yesterday night... >> >>> >>> Over in XFS land we have an xfs debugging tool (xfs_db) that knows how >>> to dump (and write!) most every field of every metadata type. This >>> makes it fairly easy to write systematic level 0 fuzzing tests that >>> check how well the filesystem reacts to garbage data (zeroing, >>> randomizing, oneing, adding and subtracting small integers) in a field. >>> (It also knows how to trash entire blocks.) > > The same tool exists for btrfs, although lacks the write ability, but > that dump is more comprehensive and a great tool to learn the on-disk > format. > > > And for the fuzzing defending part, just a few kernel releases ago, > there is none for btrfs, and now we have a full static verification > layer to cover (almost) all on-disk data at read and write time. > (Along with enhanced runtime check) > > We have covered from vague values inside tree blocks and invalid/missing > cross-ref find at runtime. > > Currently the two layered check works pretty fine (well, sometimes too > good to detect older, improper behaved kernel). > - Tree blocks with vague data just get rejected by verification layer > So that all members should fit on-disk format, from alignment to > generation to inode mode. > > The error will trigger a good enough (TM) error message for developer > to read, and if we have other copies, we retry other copies just as > we hit a bad copy. > > - At runtime, we have much less to check > Only cross-ref related things can be wrong now. since everything > inside a single tree block has already be checked. > > In fact, from my respect of view, such read time check should be there > from the very beginning. > It acts kinda of a on-disk format spec. (In fact, by implementing the > verification layer itself, it already exposes a lot of btrfs design > trade-offs) > > Even for a fs as complex (buggy) as btrfs, we only take 1K lines to > implement the verification layer. > So I'd like to see every new mainlined fs to have such ability. Out of curiosity, it looks like every mainstream filesystem has its own fuzz/injection tool in their tool-set, if it's really such a generic requirement, why shouldn't there be a common tool to handle that, let specified filesystem fill the tool's callback to seek a node/block and supported fields can be fuzzed in inode. It can help to avoid redundant work whenever Linux welcomes a new filesystem Thanks, > >> >> Actually, compared with XFS, EROFS has rather simple on-disk format. >> What we inject one time is quite deterministic. >> >> The first step just purposely writes some random fuzzed data to >> the base inode metadata, compressed indexes, or dir data field >> (one round one field) to make it validity and coverability. >> >>> >>> You might want to write such a debugging tool for erofs so that you can >>> take apart crashed images to get a better idea of what went wrong, and >>> to write easy fuzzing tests. >> >> Yes, we will do such a debugging tool of course. Actually Li Guifu is now >> developping a erofs-fuse to support old linux versions or other OSes for >> archiveing only use, we will base on that code to develop a better fuzzer >> tool as well. > > Personally speaking, debugging tool is way more important than a running > kernel module/fuse. > It's human trying to write the code, most of time is spent educating > code readers, thus debugging tool is way more important than dead cold code. > > Thanks, > Qu >> >> Thanks, >> Gao Xiang >> >>> >>> --D >>> umount mntdir mount -t erofs -o loop testdir_fsl.fuzz.img mntdir for j in `find mntdir -type f`; do md5sum $j > /dev/null done done Thanks, Gao Xiang > > Thanks, > Gao Xiang > > ___ devel mailing list de...@linuxdriverproject.org http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel
Re: [PATCH] erofs: move erofs out of staging
Hi Qu, On Tue, Aug 20, 2019 at 08:55:32AM +0800, Qu Wenruo wrote: > [...] > >>> I have made a simple fuzzer to inject messy in inode metadata, > >>> dir data, compressed indexes and super block, > >>> https://git.kernel.org/pub/scm/linux/kernel/git/xiang/erofs-utils.git/commit/?h=experimental-fuzzer > >>> > >>> I am testing with some given dirs and the following script. > >>> Does it look reasonable? > >>> > >>> # !/bin/bash > >>> > >>> mkdir -p mntdir > >>> > >>> for ((i=0; i<1000; ++i)); do > >>> mkfs/mkfs.erofs -F$i testdir_fsl.fuzz.img testdir_fsl > /dev/null 2>&1 > >> > >> mkfs fuzzes the image? Er > > > > Thanks for your reply. > > > > First, This is just the first step of erofs fuzzer I wrote yesterday > > night... > > > >> > >> Over in XFS land we have an xfs debugging tool (xfs_db) that knows how > >> to dump (and write!) most every field of every metadata type. This > >> makes it fairly easy to write systematic level 0 fuzzing tests that > >> check how well the filesystem reacts to garbage data (zeroing, > >> randomizing, oneing, adding and subtracting small integers) in a field. > >> (It also knows how to trash entire blocks.) > > The same tool exists for btrfs, although lacks the write ability, but > that dump is more comprehensive and a great tool to learn the on-disk > format. > > > And for the fuzzing defending part, just a few kernel releases ago, > there is none for btrfs, and now we have a full static verification > layer to cover (almost) all on-disk data at read and write time. > (Along with enhanced runtime check) > > We have covered from vague values inside tree blocks and invalid/missing > cross-ref find at runtime. > > Currently the two layered check works pretty fine (well, sometimes too > good to detect older, improper behaved kernel). > - Tree blocks with vague data just get rejected by verification layer > So that all members should fit on-disk format, from alignment to > generation to inode mode. > > The error will trigger a good enough (TM) error message for developer > to read, and if we have other copies, we retry other copies just as > we hit a bad copy. > > - At runtime, we have much less to check > Only cross-ref related things can be wrong now. since everything > inside a single tree block has already be checked. > > In fact, from my respect of view, such read time check should be there > from the very beginning. > It acts kinda of a on-disk format spec. (In fact, by implementing the > verification layer itself, it already exposes a lot of btrfs design > trade-offs) > > Even for a fs as complex (buggy) as btrfs, we only take 1K lines to > implement the verification layer. > So I'd like to see every new mainlined fs to have such ability. It's already on our schedule, but we have limited manpower. Rome was not built in a day, as I mentioned eariler, we are doing our best. In principle, all the new Linux features on-disk can build their debugging tools, not only for file systems. You can hardly let your newborn baby go to university immediately. We're developping out of our interests for Linux community (our high level bosses care nothing except for money, you know) and we hope to better join in and contribute to Linux community, we need more time to enrich our eco-system in our spare time. All HUAWEI smartphone products will continue using this filesystem, and its performance and stability is proven by our 10+ millions products, and maintaining this filesystem is one of our paid jobs. > > > > > Actually, compared with XFS, EROFS has rather simple on-disk format. > > What we inject one time is quite deterministic. > > > > The first step just purposely writes some random fuzzed data to > > the base inode metadata, compressed indexes, or dir data field > > (one round one field) to make it validity and coverability. > > > >> > >> You might want to write such a debugging tool for erofs so that you can > >> take apart crashed images to get a better idea of what went wrong, and > >> to write easy fuzzing tests. > > > > Yes, we will do such a debugging tool of course. Actually Li Guifu is now > > developping a erofs-fuse to support old linux versions or other OSes for > > archiveing only use, we will base on that code to develop a better fuzzer > > tool as well. > > Personally speaking, debugging tool is way more important than a running > kernel module/fuse. > It's human trying to write the code, most of time is spent educating > code readers, thus debugging tool is way more important than dead cold code. Debugging tools and erofs-fuse share common code, that is to parse the filesystem. That was the main point that I want to say. Thanks, Gao Xiang > > Thanks, > Qu > > > > Thanks, > > Gao Xiang > > > >> > >> --D > >> > >>> umount mntdir > >>> mount -t erofs -o loop testdir_fsl.fuzz.img mntdir > >>> for j in `find mntdir -type f`; do > >>> md5sum $j > /dev/null > >>> done > >>> done > >>> > >>> Thanks, > >>> Gao
Re: [PATCH] erofs: move erofs out of staging
[...] >>> I have made a simple fuzzer to inject messy in inode metadata, >>> dir data, compressed indexes and super block, >>> https://git.kernel.org/pub/scm/linux/kernel/git/xiang/erofs-utils.git/commit/?h=experimental-fuzzer >>> >>> I am testing with some given dirs and the following script. >>> Does it look reasonable? >>> >>> # !/bin/bash >>> >>> mkdir -p mntdir >>> >>> for ((i=0; i<1000; ++i)); do >>> mkfs/mkfs.erofs -F$i testdir_fsl.fuzz.img testdir_fsl > /dev/null 2>&1 >> >> mkfs fuzzes the image? Er > > Thanks for your reply. > > First, This is just the first step of erofs fuzzer I wrote yesterday night... > >> >> Over in XFS land we have an xfs debugging tool (xfs_db) that knows how >> to dump (and write!) most every field of every metadata type. This >> makes it fairly easy to write systematic level 0 fuzzing tests that >> check how well the filesystem reacts to garbage data (zeroing, >> randomizing, oneing, adding and subtracting small integers) in a field. >> (It also knows how to trash entire blocks.) The same tool exists for btrfs, although lacks the write ability, but that dump is more comprehensive and a great tool to learn the on-disk format. And for the fuzzing defending part, just a few kernel releases ago, there is none for btrfs, and now we have a full static verification layer to cover (almost) all on-disk data at read and write time. (Along with enhanced runtime check) We have covered from vague values inside tree blocks and invalid/missing cross-ref find at runtime. Currently the two layered check works pretty fine (well, sometimes too good to detect older, improper behaved kernel). - Tree blocks with vague data just get rejected by verification layer So that all members should fit on-disk format, from alignment to generation to inode mode. The error will trigger a good enough (TM) error message for developer to read, and if we have other copies, we retry other copies just as we hit a bad copy. - At runtime, we have much less to check Only cross-ref related things can be wrong now. since everything inside a single tree block has already be checked. In fact, from my respect of view, such read time check should be there from the very beginning. It acts kinda of a on-disk format spec. (In fact, by implementing the verification layer itself, it already exposes a lot of btrfs design trade-offs) Even for a fs as complex (buggy) as btrfs, we only take 1K lines to implement the verification layer. So I'd like to see every new mainlined fs to have such ability. > > Actually, compared with XFS, EROFS has rather simple on-disk format. > What we inject one time is quite deterministic. > > The first step just purposely writes some random fuzzed data to > the base inode metadata, compressed indexes, or dir data field > (one round one field) to make it validity and coverability. > >> >> You might want to write such a debugging tool for erofs so that you can >> take apart crashed images to get a better idea of what went wrong, and >> to write easy fuzzing tests. > > Yes, we will do such a debugging tool of course. Actually Li Guifu is now > developping a erofs-fuse to support old linux versions or other OSes for > archiveing only use, we will base on that code to develop a better fuzzer > tool as well. Personally speaking, debugging tool is way more important than a running kernel module/fuse. It's human trying to write the code, most of time is spent educating code readers, thus debugging tool is way more important than dead cold code. Thanks, Qu > > Thanks, > Gao Xiang > >> >> --D >> >>> umount mntdir >>> mount -t erofs -o loop testdir_fsl.fuzz.img mntdir >>> for j in `find mntdir -type f`; do >>> md5sum $j > /dev/null >>> done >>> done >>> >>> Thanks, >>> Gao Xiang >>> Thanks, Gao Xiang signature.asc Description: OpenPGP digital signature ___ devel mailing list de...@linuxdriverproject.org http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel
driverdev-devel@linuxdriverproject.org
Good Day , My name is Mr. Ryan Rogers the Independent Financial Consultant. We are contacting you concerning funding of your business project. We are interested to partnership with you as we are seeking to diversify our financial portfolio into viable and lucrative business projects that worth it, We are most interested in partnerships business ventures in Medical and Health care projects, Real estate projects, mining projects, agricultural projects renewable energy projects, Oil and Gas, start-up projects and business expansions / Loan with lower rate, Your swift response is highly needed. Best Regard, Ryan Roger ___ devel mailing list de...@linuxdriverproject.org http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel
Re: [PATCH] erofs: move erofs out of staging
Hi Darrick, On Mon, Aug 19, 2019 at 09:09:23AM -0700, Darrick J. Wong wrote: > On Mon, Aug 19, 2019 at 04:14:11AM +0800, Gao Xiang wrote: > > Hi all, > > > > On Mon, Aug 19, 2019 at 02:16:55AM +0800, Gao Xiang wrote: > > > Hi Hch, > > > > > > On Sun, Aug 18, 2019 at 10:47:02AM -0700, Christoph Hellwig wrote: > > > > On Sun, Aug 18, 2019 at 10:29:38AM -0700, Eric Biggers wrote: > > > > > Not sure what you're even disagreeing with, as I *do* expect new > > > > > filesystems to > > > > > be held to a high standard, and to be written with the assumption > > > > > that the > > > > > on-disk data may be corrupted or malicious. We just can't expect the > > > > > bar to be > > > > > so high (e.g. no bugs) that it's never been attained by *any* > > > > > filesystem even > > > > > after years/decades of active development. If the developers were > > > > > careful, the > > > > > code generally looks robust, and they are willing to address such > > > > > bugs as they > > > > > are found, realistically that's as good as we can expect to get... > > > > > > > > Well, the impression I got from Richards quick look and the reply to it > > > > is > > > > that there is very little attempt to validate the ondisk data structure > > > > and there is absolutely no priority to do so. Which is very different > > > > from there is a bug or two here and there. > > > > > > As my second reply to Richard, I didn't fuzz all the on-disk fields for > > > EROFS. > > > and as my reply to Richard / Greg, current EROFS is used on the top of > > > dm-verity. > > > > > > I cannot say how well EROFS will be performed on malformed images (and > > > you can > > > also find the bug richard pointed out is a miswritten break->continue by > > > myself). > > > > > > I posted the upstream EROFS post on July 4, 2019 and a month and a half > > > later, > > > no one can tell me (yes, thanks for kind people reply me about their > > > suggestion) > > > what we should do next (you can see these emails, I sent many times) to > > > meet > > > the minimal upstream requirements and rare people can even dip into my > > > code. > > > > > > That is all I want to say. I will work on autofuzz these days, and I want > > > to > > > know how to meet your requirements on this (you can tell us your standard, > > > how well should we do). > > > > > > OK, you don't reply to my post once, I have no idea how to get your first > > > reply. > > > > I have made a simple fuzzer to inject messy in inode metadata, > > dir data, compressed indexes and super block, > > https://git.kernel.org/pub/scm/linux/kernel/git/xiang/erofs-utils.git/commit/?h=experimental-fuzzer > > > > I am testing with some given dirs and the following script. > > Does it look reasonable? > > > > # !/bin/bash > > > > mkdir -p mntdir > > > > for ((i=0; i<1000; ++i)); do > > mkfs/mkfs.erofs -F$i testdir_fsl.fuzz.img testdir_fsl > /dev/null 2>&1 > > mkfs fuzzes the image? Er Thanks for your reply. First, This is just the first step of erofs fuzzer I wrote yesterday night... > > Over in XFS land we have an xfs debugging tool (xfs_db) that knows how > to dump (and write!) most every field of every metadata type. This > makes it fairly easy to write systematic level 0 fuzzing tests that > check how well the filesystem reacts to garbage data (zeroing, > randomizing, oneing, adding and subtracting small integers) in a field. > (It also knows how to trash entire blocks.) Actually, compared with XFS, EROFS has rather simple on-disk format. What we inject one time is quite deterministic. The first step just purposely writes some random fuzzed data to the base inode metadata, compressed indexes, or dir data field (one round one field) to make it validity and coverability. > > You might want to write such a debugging tool for erofs so that you can > take apart crashed images to get a better idea of what went wrong, and > to write easy fuzzing tests. Yes, we will do such a debugging tool of course. Actually Li Guifu is now developping a erofs-fuse to support old linux versions or other OSes for archiveing only use, we will base on that code to develop a better fuzzer tool as well. Thanks, Gao Xiang > > --D > > > umount mntdir > > mount -t erofs -o loop testdir_fsl.fuzz.img mntdir > > for j in `find mntdir -type f`; do > > md5sum $j > /dev/null > > done > > done > > > > Thanks, > > Gao Xiang > > > > > > > > Thanks, > > > Gao Xiang > > > ___ devel mailing list de...@linuxdriverproject.org http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel
Re: [PATCH 2/2] staging: rtl8192e: rtllib_crypt_ccmp.c: Use crypto API ccm(aes)
On 8/16/19 1:59 AM, Christina Quast wrote: Use ccm(aes) aead transform instead of invoking the AES block cipher block by block. Signed-off-by: Christina Quast --- drivers/staging/rtl8192e/Kconfig | 1 + drivers/staging/rtl8192e/rtllib_crypt_ccmp.c | 187 --- 2 files changed, 78 insertions(+), 110 deletions(-) Tested-by: Larry Finger This change for the RTL8192E works. I do not have the hardware for testing the equivalent change for r8192u, but as the changes look the same, that one is likely OK as well. Thanks for the change, Larry ___ devel mailing list de...@linuxdriverproject.org http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel
[PATCH] staging: rtl8723bs: core: Remove unneeded declaration WFD_OUI
Remove unneeded declaration "extern unsigned char WFD_OUI" Signed-off-by: Hariprasad Kelam --- drivers/staging/rtl8723bs/core/rtw_ap.c | 1 - 1 file changed, 1 deletion(-) diff --git a/drivers/staging/rtl8723bs/core/rtw_ap.c b/drivers/staging/rtl8723bs/core/rtw_ap.c index 02f5478..6d18d23 100644 --- a/drivers/staging/rtl8723bs/core/rtw_ap.c +++ b/drivers/staging/rtl8723bs/core/rtw_ap.c @@ -13,7 +13,6 @@ extern unsigned char RTW_WPA_OUI[]; extern unsigned char WMM_OUI[]; extern unsigned char WPS_OUI[]; extern unsigned char P2P_OUI[]; -extern unsigned char WFD_OUI[]; void init_mlme_ap_info(struct adapter *padapter) { -- 2.7.4 ___ devel mailing list de...@linuxdriverproject.org http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel
Re: [PATCH] erofs: move erofs out of staging
On Mon, Aug 19, 2019 at 04:14:11AM +0800, Gao Xiang wrote: > Hi all, > > On Mon, Aug 19, 2019 at 02:16:55AM +0800, Gao Xiang wrote: > > Hi Hch, > > > > On Sun, Aug 18, 2019 at 10:47:02AM -0700, Christoph Hellwig wrote: > > > On Sun, Aug 18, 2019 at 10:29:38AM -0700, Eric Biggers wrote: > > > > Not sure what you're even disagreeing with, as I *do* expect new > > > > filesystems to > > > > be held to a high standard, and to be written with the assumption that > > > > the > > > > on-disk data may be corrupted or malicious. We just can't expect the > > > > bar to be > > > > so high (e.g. no bugs) that it's never been attained by *any* > > > > filesystem even > > > > after years/decades of active development. If the developers were > > > > careful, the > > > > code generally looks robust, and they are willing to address such bugs > > > > as they > > > > are found, realistically that's as good as we can expect to get... > > > > > > Well, the impression I got from Richards quick look and the reply to it is > > > that there is very little attempt to validate the ondisk data structure > > > and there is absolutely no priority to do so. Which is very different > > > from there is a bug or two here and there. > > > > As my second reply to Richard, I didn't fuzz all the on-disk fields for > > EROFS. > > and as my reply to Richard / Greg, current EROFS is used on the top of > > dm-verity. > > > > I cannot say how well EROFS will be performed on malformed images (and you > > can > > also find the bug richard pointed out is a miswritten break->continue by > > myself). > > > > I posted the upstream EROFS post on July 4, 2019 and a month and a half > > later, > > no one can tell me (yes, thanks for kind people reply me about their > > suggestion) > > what we should do next (you can see these emails, I sent many times) to meet > > the minimal upstream requirements and rare people can even dip into my code. > > > > That is all I want to say. I will work on autofuzz these days, and I want to > > know how to meet your requirements on this (you can tell us your standard, > > how well should we do). > > > > OK, you don't reply to my post once, I have no idea how to get your first > > reply. > > I have made a simple fuzzer to inject messy in inode metadata, > dir data, compressed indexes and super block, > https://git.kernel.org/pub/scm/linux/kernel/git/xiang/erofs-utils.git/commit/?h=experimental-fuzzer > > I am testing with some given dirs and the following script. > Does it look reasonable? > > # !/bin/bash > > mkdir -p mntdir > > for ((i=0; i<1000; ++i)); do > mkfs/mkfs.erofs -F$i testdir_fsl.fuzz.img testdir_fsl > /dev/null 2>&1 mkfs fuzzes the image? Er Over in XFS land we have an xfs debugging tool (xfs_db) that knows how to dump (and write!) most every field of every metadata type. This makes it fairly easy to write systematic level 0 fuzzing tests that check how well the filesystem reacts to garbage data (zeroing, randomizing, oneing, adding and subtracting small integers) in a field. (It also knows how to trash entire blocks.) You might want to write such a debugging tool for erofs so that you can take apart crashed images to get a better idea of what went wrong, and to write easy fuzzing tests. --D > umount mntdir > mount -t erofs -o loop testdir_fsl.fuzz.img mntdir > for j in `find mntdir -type f`; do > md5sum $j > /dev/null > done > done > > Thanks, > Gao Xiang > > > > > Thanks, > > Gao Xiang > > ___ devel mailing list de...@linuxdriverproject.org http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel
Re: [PATCH 6/6] staging: erofs: avoid endless loop of invalid lookback distance 0
On 2019-8-19 18:34, Gao Xiang wrote: > As reported by erofs-utils fuzzer, Lookback distance should > be a positive number, so it should be actually looked back > rather than spinning. > > Fixes: 02827e1796b3 ("staging: erofs: add erofs_map_blocks_iter") > Cc: # 4.19+ > Signed-off-by: Gao Xiang Reviewed-by: Chao Yu Thanks, ___ devel mailing list de...@linuxdriverproject.org http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel
Re: [PATCH 5/6] staging: erofs: detect potential multiref due to corrupted images
On 2019-8-19 18:34, Gao Xiang wrote: > As reported by erofs-utils fuzzer, currently, multiref > (ondisk deduplication) hasn't been supported for now, > we should forbid it properly. > > Fixes: 3883a79abd02 ("staging: erofs: introduce VLE decompression support") > Cc: # 4.19+ > Signed-off-by: Gao Xiang > --- > drivers/staging/erofs/zdata.c | 16 ++-- > 1 file changed, 14 insertions(+), 2 deletions(-) > > diff --git a/drivers/staging/erofs/zdata.c b/drivers/staging/erofs/zdata.c > index aae2f2b8353f..5b6fef5181af 100644 > --- a/drivers/staging/erofs/zdata.c > +++ b/drivers/staging/erofs/zdata.c > @@ -816,8 +816,16 @@ static int z_erofs_decompress_pcluster(struct > super_block *sb, > pagenr = z_erofs_onlinepage_index(page); > > DBG_BUGON(pagenr >= nr_pages); > - DBG_BUGON(pages[pagenr]); > > + /* > + * currently EROFS doesn't support multiref(dedup), > + * so here erroring out one multiref page. > + */ > + if (unlikely(pages[pagenr])) { > + DBG_BUGON(1); > + SetPageError(pages[pagenr]); > + z_erofs_onlinepage_endio(pages[pagenr]); Should set err meanwhile? > + } > pages[pagenr] = page; > } > z_erofs_pagevec_ctor_exit(&ctor, true); > @@ -849,7 +857,11 @@ static int z_erofs_decompress_pcluster(struct > super_block *sb, > pagenr = z_erofs_onlinepage_index(page); > > DBG_BUGON(pagenr >= nr_pages); > - DBG_BUGON(pages[pagenr]); > + if (unlikely(pages[pagenr])) { > + DBG_BUGON(1); > + SetPageError(pages[pagenr]); > + z_erofs_onlinepage_endio(pages[pagenr]); > + } > pages[pagenr] = page; > > overlapped = true; > ___ devel mailing list de...@linuxdriverproject.org http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel
Re: [PATCH 4/6] staging: erofs: avoid loop in submit chains
On 2019-8-19 18:34, Gao Xiang wrote: > As reported by erofs-utils fuzzer, 2 conditions > can happen in corrupted images, which can cause > unexpected behaviors. > - access the same pcluster one more time; > - access the tail end pcluster again, e.g. > _ access again (will trigger tail merging) >| > 1 2 3 1 2 -> 1 2 3 1 > |_ tail end of the chain\___/ (unexpected behavior) > Let's detect and avoid them now. > > Signed-off-by: Gao Xiang Reviewed-by: Chao Yu Thanks, ___ devel mailing list de...@linuxdriverproject.org http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel
Re: [PATCH 2/6] staging: erofs: cannot set EROFS_V_Z_INITED_BIT if fill_inode_lazy fails
On 2019-8-19 18:34, Gao Xiang wrote: > As reported by erofs-utils fuzzer, unsupported compressed > clustersize will make fill_inode_lazy fail, for such case > we cannot set EROFS_V_Z_INITED_BIT since we need return > failure for each z_erofs_map_blocks_iter(). > > Fixes: 152a333a5895 ("staging: erofs: add compacted compression indexes > support") > Cc: # 5.3+ > Signed-off-by: Gao Xiang Reviewed-by: Chao Yu Thanks, ___ devel mailing list de...@linuxdriverproject.org http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel
Re: [PATCH 3/6] staging: erofs: add two missing erofs_workgroup_put for corrupted images
On 2019-8-19 18:34, Gao Xiang wrote: > As reported by erofs-utils fuzzer, these error handling > path will be entered to handle corrupted images. > > Lack of erofs_workgroup_puts will cause unmounting > unsuccessfully. > > Fix these return values to EFSCORRUPTED as well. > > Fixes: 3883a79abd02 ("staging: erofs: introduce VLE decompression support") > Cc: # 4.19+ > Signed-off-by: Gao Xiang Reviewed-by: Chao Yu Thanks, ___ devel mailing list de...@linuxdriverproject.org http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel
Re: [PATCH 1/6] staging: erofs: some compressed cluster should be submitted for corrupted images
On 2019-8-19 18:34, Gao Xiang wrote: > As reported by erofs_utils fuzzer, a logical page can belong > to at most 2 compressed clusters, if one compressed cluster > is corrupted, but the other has been ready in submitting chain. > > The chain needs to submit anyway in order to keep the page > working properly (page unlocked with PG_error set, PG_uptodate > not set). > > Let's fix it now. > > Fixes: 3883a79abd02 ("staging: erofs: introduce VLE decompression support") > Cc: # 4.19+ > Signed-off-by: Gao Xiang Reviewed-by: Chao Yu Thanks, ___ devel mailing list de...@linuxdriverproject.org http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel
Re: [PATCH 1/6] staging: erofs: some compressed cluster should be submitted for corrupted images
On 2019-8-19 18:34, Gao Xiang wrote: > As reported by erofs_utils fuzzer, a logical page can belong > to at most 2 compressed clusters, if one compressed cluster > is corrupted, but the other has been ready in submitting chain. > > The chain needs to submit anyway in order to keep the page > working properly (page unlocked with PG_error set, PG_uptodate > not set). > > Let's fix it now. > > Fixes: 3883a79abd02 ("staging: erofs: introduce VLE decompression support") > Cc: # 4.19+ > Signed-off-by: Gao Xiang Reviewed-by: Chao Yu Thanks, ___ devel mailing list de...@linuxdriverproject.org http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel
[PATCH 2/6] staging: erofs: cannot set EROFS_V_Z_INITED_BIT if fill_inode_lazy fails
As reported by erofs-utils fuzzer, unsupported compressed clustersize will make fill_inode_lazy fail, for such case we cannot set EROFS_V_Z_INITED_BIT since we need return failure for each z_erofs_map_blocks_iter(). Fixes: 152a333a5895 ("staging: erofs: add compacted compression indexes support") Cc: # 5.3+ Signed-off-by: Gao Xiang --- drivers/staging/erofs/zmap.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/drivers/staging/erofs/zmap.c b/drivers/staging/erofs/zmap.c index b61b9b5950ac..7408e86823a4 100644 --- a/drivers/staging/erofs/zmap.c +++ b/drivers/staging/erofs/zmap.c @@ -85,12 +85,11 @@ static int fill_inode_lazy(struct inode *inode) vi->z_physical_clusterbits[1] = vi->z_logical_clusterbits + ((h->h_clusterbits >> 5) & 7); + set_bit(EROFS_V_Z_INITED_BIT, &vi->flags); unmap_done: kunmap_atomic(kaddr); unlock_page(page); put_page(page); - - set_bit(EROFS_V_Z_INITED_BIT, &vi->flags); out_unlock: clear_and_wake_up_bit(EROFS_V_BL_Z_BIT, &vi->flags); return err; -- 2.17.1 ___ devel mailing list de...@linuxdriverproject.org http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel
[PATCH 5/6] staging: erofs: detect potential multiref due to corrupted images
As reported by erofs-utils fuzzer, currently, multiref (ondisk deduplication) hasn't been supported for now, we should forbid it properly. Fixes: 3883a79abd02 ("staging: erofs: introduce VLE decompression support") Cc: # 4.19+ Signed-off-by: Gao Xiang --- drivers/staging/erofs/zdata.c | 16 ++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/drivers/staging/erofs/zdata.c b/drivers/staging/erofs/zdata.c index aae2f2b8353f..5b6fef5181af 100644 --- a/drivers/staging/erofs/zdata.c +++ b/drivers/staging/erofs/zdata.c @@ -816,8 +816,16 @@ static int z_erofs_decompress_pcluster(struct super_block *sb, pagenr = z_erofs_onlinepage_index(page); DBG_BUGON(pagenr >= nr_pages); - DBG_BUGON(pages[pagenr]); + /* +* currently EROFS doesn't support multiref(dedup), +* so here erroring out one multiref page. +*/ + if (unlikely(pages[pagenr])) { + DBG_BUGON(1); + SetPageError(pages[pagenr]); + z_erofs_onlinepage_endio(pages[pagenr]); + } pages[pagenr] = page; } z_erofs_pagevec_ctor_exit(&ctor, true); @@ -849,7 +857,11 @@ static int z_erofs_decompress_pcluster(struct super_block *sb, pagenr = z_erofs_onlinepage_index(page); DBG_BUGON(pagenr >= nr_pages); - DBG_BUGON(pages[pagenr]); + if (unlikely(pages[pagenr])) { + DBG_BUGON(1); + SetPageError(pages[pagenr]); + z_erofs_onlinepage_endio(pages[pagenr]); + } pages[pagenr] = page; overlapped = true; -- 2.17.1 ___ devel mailing list de...@linuxdriverproject.org http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel
[PATCH 6/6] staging: erofs: avoid endless loop of invalid lookback distance 0
As reported by erofs-utils fuzzer, Lookback distance should be a positive number, so it should be actually looked back rather than spinning. Fixes: 02827e1796b3 ("staging: erofs: add erofs_map_blocks_iter") Cc: # 4.19+ Signed-off-by: Gao Xiang --- drivers/staging/erofs/zmap.c | 6 ++ 1 file changed, 6 insertions(+) diff --git a/drivers/staging/erofs/zmap.c b/drivers/staging/erofs/zmap.c index 7408e86823a4..774dacbc5b32 100644 --- a/drivers/staging/erofs/zmap.c +++ b/drivers/staging/erofs/zmap.c @@ -350,6 +350,12 @@ static int vle_extent_lookback(struct z_erofs_maprecorder *m, switch (m->type) { case Z_EROFS_VLE_CLUSTER_TYPE_NONHEAD: + if (unlikely(!m->delta[0])) { + errln("invalid lookback distance 0 at nid %llu", + vi->nid); + DBG_BUGON(1); + return -EFSCORRUPTED; + } return vle_extent_lookback(m, m->delta[0]); case Z_EROFS_VLE_CLUSTER_TYPE_PLAIN: map->m_flags &= ~EROFS_MAP_ZIPPED; -- 2.17.1 ___ devel mailing list de...@linuxdriverproject.org http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel
[PATCH 3/6] staging: erofs: add two missing erofs_workgroup_put for corrupted images
As reported by erofs-utils fuzzer, these error handling path will be entered to handle corrupted images. Lack of erofs_workgroup_puts will cause unmounting unsuccessfully. Fix these return values to EFSCORRUPTED as well. Fixes: 3883a79abd02 ("staging: erofs: introduce VLE decompression support") Cc: # 4.19+ Signed-off-by: Gao Xiang --- drivers/staging/erofs/zdata.c | 6 -- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/drivers/staging/erofs/zdata.c b/drivers/staging/erofs/zdata.c index 87b0c96caf8f..23283c97fd3b 100644 --- a/drivers/staging/erofs/zdata.c +++ b/drivers/staging/erofs/zdata.c @@ -357,14 +357,16 @@ static struct z_erofs_collection *cllookup(struct z_erofs_collector *clt, cl = z_erofs_primarycollection(pcl); if (unlikely(cl->pageofs != (map->m_la & ~PAGE_MASK))) { DBG_BUGON(1); - return ERR_PTR(-EIO); + erofs_workgroup_put(grp); + return ERR_PTR(-EFSCORRUPTED); } length = READ_ONCE(pcl->length); if (length & Z_EROFS_PCLUSTER_FULL_LENGTH) { if ((map->m_llen << Z_EROFS_PCLUSTER_LENGTH_BIT) > length) { DBG_BUGON(1); - return ERR_PTR(-EIO); + erofs_workgroup_put(grp); + return ERR_PTR(-EFSCORRUPTED); } } else { unsigned int llen = map->m_llen << Z_EROFS_PCLUSTER_LENGTH_BIT; -- 2.17.1 ___ devel mailing list de...@linuxdriverproject.org http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel
[PATCH 4/6] staging: erofs: avoid loop in submit chains
As reported by erofs-utils fuzzer, 2 conditions can happen in corrupted images, which can cause unexpected behaviors. - access the same pcluster one more time; - access the tail end pcluster again, e.g. _ access again (will trigger tail merging) | 1 2 3 1 2 -> 1 2 3 1 |_ tail end of the chain\___/ (unexpected behavior) Let's detect and avoid them now. Signed-off-by: Gao Xiang --- drivers/staging/erofs/zdata.c | 13 - 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/drivers/staging/erofs/zdata.c b/drivers/staging/erofs/zdata.c index 23283c97fd3b..aae2f2b8353f 100644 --- a/drivers/staging/erofs/zdata.c +++ b/drivers/staging/erofs/zdata.c @@ -132,7 +132,7 @@ enum z_erofs_collectmode { struct z_erofs_collector { struct z_erofs_pagevec_ctor vector; - struct z_erofs_pcluster *pcl; + struct z_erofs_pcluster *pcl, *tailpcl; struct z_erofs_collection *cl; struct page **compressedpages; z_erofs_next_pcluster_t owned_head; @@ -353,6 +353,11 @@ static struct z_erofs_collection *cllookup(struct z_erofs_collector *clt, return NULL; pcl = container_of(grp, struct z_erofs_pcluster, obj); + if (clt->owned_head == &pcl->next || pcl == clt->tailpcl) { + DBG_BUGON(1); + erofs_workgroup_put(grp); + return ERR_PTR(-EFSCORRUPTED); + } cl = z_erofs_primarycollection(pcl); if (unlikely(cl->pageofs != (map->m_la & ~PAGE_MASK))) { @@ -381,6 +386,9 @@ static struct z_erofs_collection *cllookup(struct z_erofs_collector *clt, } } mutex_lock(&cl->lock); + /* used to check tail merging loop due to corrupted images */ + if (clt->owned_head == Z_EROFS_PCLUSTER_TAIL) + clt->tailpcl = pcl; clt->mode = try_to_claim_pcluster(pcl, &clt->owned_head); clt->pcl = pcl; clt->cl = cl; @@ -434,6 +442,9 @@ static struct z_erofs_collection *clregister(struct z_erofs_collector *clt, kmem_cache_free(pcluster_cachep, pcl); return ERR_PTR(-EAGAIN); } + /* used to check tail merging loop due to corrupted images */ + if (clt->owned_head == Z_EROFS_PCLUSTER_TAIL) + clt->tailpcl = pcl; clt->owned_head = &pcl->next; clt->pcl = pcl; clt->cl = cl; -- 2.17.1 ___ devel mailing list de...@linuxdriverproject.org http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel
[PATCH 0/6] staging: erofs: first stage of corrupted compressed images
Hi all, I have fuzzed EROFS for about a day and observed the following issues due to corrupted compression images by my first fuzzer (It seems ok for uncompressed images for now). Now it can survive for 10+ minutes on my PC (Let me send out what I'm done and I will dig it more deeply...) All the fixes are trivial. Note that those have dependency on EFSCORRUPTED, so for-next is needed and I will manually backport them by hand due to many cleanup patches... Thanks, Gao Xiang Gao Xiang (6): staging: erofs: some compressed cluster should be submitted for corrupted images staging: erofs: cannot set EROFS_V_Z_INITED_BIT if fill_inode_lazy fails staging: erofs: add two missing erofs_workgroup_put for corrupted images staging: erofs: avoid loop in submit chains staging: erofs: detect potential multiref due to corrupted images staging: erofs: avoid endless loop of invalid lookback distance 0 drivers/staging/erofs/zdata.c | 46 ++- drivers/staging/erofs/zmap.c | 9 +-- 2 files changed, 42 insertions(+), 13 deletions(-) -- 2.17.1 ___ devel mailing list de...@linuxdriverproject.org http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel
[PATCH 1/6] staging: erofs: some compressed cluster should be submitted for corrupted images
As reported by erofs_utils fuzzer, a logical page can belong to at most 2 compressed clusters, if one compressed cluster is corrupted, but the other has been ready in submitting chain. The chain needs to submit anyway in order to keep the page working properly (page unlocked with PG_error set, PG_uptodate not set). Let's fix it now. Fixes: 3883a79abd02 ("staging: erofs: introduce VLE decompression support") Cc: # 4.19+ Signed-off-by: Gao Xiang --- drivers/staging/erofs/zdata.c | 11 +-- 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/drivers/staging/erofs/zdata.c b/drivers/staging/erofs/zdata.c index 2d7aaf98f7de..87b0c96caf8f 100644 --- a/drivers/staging/erofs/zdata.c +++ b/drivers/staging/erofs/zdata.c @@ -1307,19 +1307,18 @@ static int z_erofs_vle_normalaccess_readpage(struct file *file, err = z_erofs_do_read_page(&f, page, &pagepool); (void)z_erofs_collector_end(&f.clt); - if (err) { + /* if some compressed cluster ready, need submit them anyway */ + z_erofs_submit_and_unzip(inode->i_sb, &f.clt, &pagepool, true); + + if (err) errln("%s, failed to read, err [%d]", __func__, err); - goto out; - } - z_erofs_submit_and_unzip(inode->i_sb, &f.clt, &pagepool, true); -out: if (f.map.mpage) put_page(f.map.mpage); /* clean up the remaining free pages */ put_pages_list(&pagepool); - return 0; + return err; } static bool should_decompress_synchronously(struct erofs_sb_info *sbi, -- 2.17.1 ___ devel mailing list de...@linuxdriverproject.org http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel
Re: [RFC PATCH 15/15] drivers/acrn: add the support of offline SOS cpu
On Fri, Aug 16, 2019 at 10:25:56AM +0800, Zhao Yakui wrote: > diff --git a/drivers/staging/acrn/acrn_dev.c b/drivers/staging/acrn/acrn_dev.c > index 0602125..6868003 100644 > --- a/drivers/staging/acrn/acrn_dev.c > +++ b/drivers/staging/acrn/acrn_dev.c > @@ -588,6 +588,41 @@ static const struct file_operations fops = { > #define SUPPORT_HV_API_VERSION_MAJOR 1 > #define SUPPORT_HV_API_VERSION_MINOR 0 > > +static ssize_t > +offline_cpu_store(struct device *dev, > + struct device_attribute *attr, > + const char *buf, size_t count) > +{ > +#ifdef CONFIG_X86 > + u64 cpu, lapicid; > + > + if (kstrtoull(buf, 0, &cpu) < 0) > + return -EINVAL; Preserve the error code. ret = kstrtoull(buf, 0, &cpu); if (ret) return ret; > + > + if (cpu_possible(cpu)) { You can't pass unchecked cpu values to cpu_possible() or it results in an out of bounds read if cpu is >= than nr_cpu_ids. regards, dan carpenter ___ devel mailing list de...@linuxdriverproject.org http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel
Re: [RFC PATCH 08/15] drivers/acrn: add VM memory management for ACRN char device
On Mon, Aug 19, 2019 at 10:39:58AM +0300, Dan Carpenter wrote: > On Mon, Aug 19, 2019 at 01:32:54PM +0800, Zhao, Yakui wrote: > > In fact as this driver is mainly used for embedded IOT usage, it doesn't > > handle the complex cleanup when such error is encountered. Instead the clean > > up is handled in free_guest_vm. > > A use after free here seems like a potential security problem. Security > matters for IoT... :( Yeah, the "S" in "IoT" stands for security. -- Regards/Gruss, Boris. Good mailing practices for 400: avoid top-posting and trim the reply. ___ devel mailing list de...@linuxdriverproject.org http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel
Re: [PATCH] erofs: move erofs out of staging
Hi Richard, On Mon, Aug 19, 2019 at 09:35:43AM +0200, Richard Weinberger wrote: > - Ursprüngliche Mail - > > I have made a simple fuzzer to inject messy in inode metadata, > > dir data, compressed indexes and super block, > > https://git.kernel.org/pub/scm/linux/kernel/git/xiang/erofs-utils.git/commit/?h=experimental-fuzzer > > > > I am testing with some given dirs and the following script. > > Does it look reasonable? > > I think that's a very good start. :-) I have been testing with this tools for hours, it seems strong against corrupted images without compression. I'm now struggling with corrupted images with compression, hopefully most of them can be fixed trivially... I will send the bunch of fixes later... Let me dig into it more... Thanks for your reply :-) Thanks, Gao Xiang > > Thanks, > //richard ___ devel mailing list de...@linuxdriverproject.org http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel
Re: [RFC PATCH 08/15] drivers/acrn: add VM memory management for ACRN char device
On Mon, Aug 19, 2019 at 01:32:54PM +0800, Zhao, Yakui wrote: > In fact as this driver is mainly used for embedded IOT usage, it doesn't > handle the complex cleanup when such error is encountered. Instead the clean > up is handled in free_guest_vm. A use after free here seems like a potential security problem. Security matters for IoT... :( regards, dan carpenter ___ devel mailing list de...@linuxdriverproject.org http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel
Re: [PATCH v3 1/3] ARM: dts: imx6ul: Add csi node
On Wed, Jul 31, 2019 at 06:32:57PM +0200, Sébastien Szymanski wrote: > Add csi node for i.MX6UL SoC. > > Reviewed-by: Fabio Estevam > Signed-off-by: Sébastien Szymanski Applied, thanks. ___ devel mailing list de...@linuxdriverproject.org http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel
Re: [PATCH] erofs: move erofs out of staging
- Ursprüngliche Mail - > On Sun, Aug 18, 2019 at 10:29:38AM -0700, Eric Biggers wrote: >> Not sure what you're even disagreeing with, as I *do* expect new filesystems >> to >> be held to a high standard, and to be written with the assumption that the >> on-disk data may be corrupted or malicious. We just can't expect the bar to >> be >> so high (e.g. no bugs) that it's never been attained by *any* filesystem even >> after years/decades of active development. If the developers were careful, >> the >> code generally looks robust, and they are willing to address such bugs as >> they >> are found, realistically that's as good as we can expect to get... > > Well, the impression I got from Richards quick look and the reply to it is > that there is very little attempt to validate the ondisk data structure > and there is absolutely no priority to do so. Which is very different > from there is a bug or two here and there. On the plus side, everything I reported got fixed within hours. Thanks, //richard ___ devel mailing list de...@linuxdriverproject.org http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel
Re: [PATCH] erofs: move erofs out of staging
- Ursprüngliche Mail - > I have made a simple fuzzer to inject messy in inode metadata, > dir data, compressed indexes and super block, > https://git.kernel.org/pub/scm/linux/kernel/git/xiang/erofs-utils.git/commit/?h=experimental-fuzzer > > I am testing with some given dirs and the following script. > Does it look reasonable? I think that's a very good start. :-) Thanks, //richard ___ devel mailing list de...@linuxdriverproject.org http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel