Re: [BUG] imx6qdl: degraded eMMC write performance
On Tue, Dec 10, 2019 at 03:44:52PM +0100, Hubert Feurstein wrote: > Hi, > > I've updated barebox for our custom platform from v2015.06.0 to > v2019.12.0. With the new version I have noticed a much worse write > performance onto the onboard eMMC. > > With v2015.06.0 the indicated progress of the copy command is very > smooth. Calling "cp -v /dev/zero /dev/mmc3.root" takes about 80 > seconds for 256MB. But with v2019.12.0 the progress is very bumpy and > the copy takes about 280 seconds. > > I've tracked this down to this commit which destroys the performance: > "block: Adjust cache sizes" (b6fef20c1215c6ef0004f6af4a9c4b77af51dc43) We could just revert this patch. I can't find any workload that gets faster with b6fef20c1215. It's rather the other way round. Sascha -- Pengutronix e.K. | | Steuerwalder Str. 21 | http://www.pengutronix.de/ | 31137 Hildesheim, Germany | Phone: +49-5121-206917-0| Amtsgericht Hildesheim, HRA 2686 | Fax: +49-5121-206917- | ___ barebox mailing list barebox@lists.infradead.org http://lists.infradead.org/mailman/listinfo/barebox
Re: [BUG] imx6qdl: degraded eMMC write performance
Am Di., 10. Dez. 2019 um 17:18 Uhr schrieb Sascha Hauer : [...] > > > We could adjust RW_BUF_SIZE (used by copy_file as buffer size) to a full > > > chunk size (16KiB). Does this give you back some lost performance? > > No, changing RW_BUF_SIZE did not help. > > And I also see why :-/ > > block_op_write() works around block sizes (512bytes), not around chunk > sizes. This means we always read before we write. This could probably be > optimized somehow, but this would only speed up the write case you > described in your initial mail. It seems what you are more interested in > is the read performance. Well, I'm interested in both. Of course the read-performance is important for booting. But the write-performance is important for production. It simply matters if you have to wait for another 200 seconds for a device to get programmed. Regards Hubert ___ barebox mailing list barebox@lists.infradead.org http://lists.infradead.org/mailman/listinfo/barebox
Re: [BUG] imx6qdl: degraded eMMC write performance
On Tue, Dec 10, 2019 at 04:43:54PM +0100, Hubert Feurstein wrote: > Hi Sascha, > > Am Di., 10. Dez. 2019 um 16:10 Uhr schrieb Sascha Hauer > : > [...] > > Well the commit message says it: > > | commit b6fef20c1215c6ef0004f6af4a9c4b77af51dc43 > > | Author: Sascha Hauer > > | Date: Thu Mar 29 13:49:45 2018 +0200 > > | > > | block: Adjust cache sizes > > | > > | Use four times more cache entries and divide the memory for each entry > > | by four. This lowers the linear read throughput somewhat but increases > > | the access speed for filesystems. > Yes, I know. I've read the commit message. But this patch even costs me ~200ms > in boottime (loading kernel and dts from ext4 root partition). > > [...] > > We could adjust RW_BUF_SIZE (used by copy_file as buffer size) to a full > > chunk size (16KiB). Does this give you back some lost performance? > No, changing RW_BUF_SIZE did not help. And I also see why :-/ block_op_write() works around block sizes (512bytes), not around chunk sizes. This means we always read before we write. This could probably be optimized somehow, but this would only speed up the write case you described in your initial mail. It seems what you are more interested in is the read performance. We could make the number of chunks and the chunk size configurable during runtime. This is less than ideal, but at least we could get a better performance with manageable effort. Sascha -- Pengutronix e.K. | | Steuerwalder Str. 21 | http://www.pengutronix.de/ | 31137 Hildesheim, Germany | Phone: +49-5121-206917-0| Amtsgericht Hildesheim, HRA 2686 | Fax: +49-5121-206917- | ___ barebox mailing list barebox@lists.infradead.org http://lists.infradead.org/mailman/listinfo/barebox
Re: [BUG] imx6qdl: degraded eMMC write performance
Hi Sascha, Am Di., 10. Dez. 2019 um 16:10 Uhr schrieb Sascha Hauer : [...] > Well the commit message says it: > | commit b6fef20c1215c6ef0004f6af4a9c4b77af51dc43 > | Author: Sascha Hauer > | Date: Thu Mar 29 13:49:45 2018 +0200 > | > | block: Adjust cache sizes > | > | Use four times more cache entries and divide the memory for each entry > | by four. This lowers the linear read throughput somewhat but increases > | the access speed for filesystems. Yes, I know. I've read the commit message. But this patch even costs me ~200ms in boottime (loading kernel and dts from ext4 root partition). [...] > We could adjust RW_BUF_SIZE (used by copy_file as buffer size) to a full > chunk size (16KiB). Does this give you back some lost performance? No, changing RW_BUF_SIZE did not help. Regards Hubert ___ barebox mailing list barebox@lists.infradead.org http://lists.infradead.org/mailman/listinfo/barebox
Re: [BUG] imx6qdl: degraded eMMC write performance
Hi Hubert, On Tue, Dec 10, 2019 at 03:44:52PM +0100, Hubert Feurstein wrote: > Hi, > > I've updated barebox for our custom platform from v2015.06.0 to > v2019.12.0. With the new version I have noticed a much worse write > performance onto the onboard eMMC. > > With v2015.06.0 the indicated progress of the copy command is very > smooth. Calling "cp -v /dev/zero /dev/mmc3.root" takes about 80 > seconds for 256MB. But with v2019.12.0 the progress is very bumpy and > the copy takes about 280 seconds. > > I've tracked this down to this commit which destroys the performance: > "block: Adjust cache sizes" (b6fef20c1215c6ef0004f6af4a9c4b77af51dc43) Well the commit message says it: | commit b6fef20c1215c6ef0004f6af4a9c4b77af51dc43 | Author: Sascha Hauer | Date: Thu Mar 29 13:49:45 2018 +0200 | | block: Adjust cache sizes | | Use four times more cache entries and divide the memory for each entry | by four. This lowers the linear read throughput somewhat but increases | the access speed for filesystems. The barebox block layer is quite simple. It always handles data in chunks of a hardcoded size. For a high throughput these should be large as this increases the buffer size we can write in the MMC controller in one go. Increasing the size is bad for filesystems though, as for each small block a fs wants to read at minimum a whole chunk must be read, or for write, read-modified-written. I remember I once tried to make the caching more intelligent. It seems I didn't even get far enough to store a branch in my working copy :( I just saw that in block_op_write() we must first read any partially written chunk, then modify it and write it back. Only full chunks can be written without reading them first. This means we should gain performance when we enter block_op_write() with only full chunks. We could adjust RW_BUF_SIZE (used by copy_file as buffer size) to a full chunk size (16KiB). Does this give you back some lost performance? Sascha -- Pengutronix e.K. | | Steuerwalder Str. 21 | http://www.pengutronix.de/ | 31137 Hildesheim, Germany | Phone: +49-5121-206917-0| Amtsgericht Hildesheim, HRA 2686 | Fax: +49-5121-206917- | ___ barebox mailing list barebox@lists.infradead.org http://lists.infradead.org/mailman/listinfo/barebox
[BUG] imx6qdl: degraded eMMC write performance
Hi, I've updated barebox for our custom platform from v2015.06.0 to v2019.12.0. With the new version I have noticed a much worse write performance onto the onboard eMMC. With v2015.06.0 the indicated progress of the copy command is very smooth. Calling "cp -v /dev/zero /dev/mmc3.root" takes about 80 seconds for 256MB. But with v2019.12.0 the progress is very bumpy and the copy takes about 280 seconds. I've tracked this down to this commit which destroys the performance: "block: Adjust cache sizes" (b6fef20c1215c6ef0004f6af4a9c4b77af51dc43) Regards Hubert ___ barebox mailing list barebox@lists.infradead.org http://lists.infradead.org/mailman/listinfo/barebox