Re: [PATCH 1/4] lib: Update LZ4 compressor module
On Sat, Jan 21, 2017 at 04:09:08PM +0100, Sven Schmidt wrote: > This patch updates LZ4 kernel module to LZ4 v1.7.3 by Yann Collet. > The kernel module is inspired by the previous work by Chanho Min. > The updated LZ4 module will not break existing code since there were alias > methods added to ensure backwards compatibility. > > API changes: > > New method LZ4_compress_fast which differs from the variant available > in kernel by the new acceleration parameter, > allowing to trade compression ratio for more compression speed > and vice versa. > > LZ4_decompress_fast is the respective decompression method, featuring a very > fast decoder (multiple GB/s per core), able to reach RAM speed in multi-core > systems. The decompressor allows to decompress data compressed with > LZ4 fast as well as the LZ4 HC (high compression) algorithm. > > Also the useful functions LZ4_decompress_safe_partial > LZ4_compress_destsize were added. The latter reverses the logic by trying to > compress as much data as possible from source to dest while the former aims > to decompress partial blocks of data. > > A bunch of streaming functions were also added > which allow compressig/decompressing data in multiple steps > (so called "streaming mode"). > > The methods lz4_compress and lz4_decompress_unknownoutputsize > are now known as LZ4_compress_default respectivley LZ4_decompress_safe. > The old methods are still available for providing backwards compatibility. > > Signed-off-by: Sven Schmidt <4ssch...@informatik.uni-hamburg.de> Please fix up so that it doesn't break the build as reported by the kbuild tool... thanks, greg k-h -- To unsubscribe from this list: send the line "unsubscribe linux-crypto" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 1/4] lib: Update LZ4 compressor module
Hi Sven, [auto build test WARNING on linus/master] [also build test WARNING on v4.10-rc4 next-20170120] [if your patch is applied to the wrong git tree, please drop us a note to help improve the system] url: https://github.com/0day-ci/linux/commits/Sven-Schmidt/Update-LZ4-compressor-module/20170121-231418 reproduce: # apt-get install sparse make ARCH=x86_64 allmodconfig make C=1 CF=-D__CHECK_ENDIAN__ sparse warnings: (new ones prefixed by >>) include/linux/compiler.h:253:8: sparse: attribute 'no_sanitize_address': unknown attribute >> lib/lz4/lz4_decompress.c:521:21: sparse: incompatible types for operation (>) lib/lz4/lz4_decompress.c:521:21:left side has type unsigned long [usertype] *src_len lib/lz4/lz4_decompress.c:521:21:right side has type int vim +521 lib/lz4/lz4_decompress.c 505 return -1; 506 } 507 EXPORT_SYMBOL(lz4_decompress_unknownoutputsize); 508 509 int lz4_decompress(const unsigned char *src, size_t *src_len, 510 unsigned char *dest, size_t actual_dest_len) { 511 *src_len = LZ4_decompress_fast(src, dest, (int)actual_dest_len); 512 513 /* 514 * Prior lz4_decompress will return 515 * 0 for success and a negative result for error 516 * new LZ4_decompress_fast returns 517 * - the length of data read on success 518 * - and also a negative result on error 519 * meaning when result > 0, we just return 0 here 520 */ > 521 if (src_len > 0) 522 return 0; 523 else 524 return -1; 525 } 526 EXPORT_SYMBOL(lz4_decompress); 527 528 MODULE_LICENSE("Dual BSD/GPL"); 529 MODULE_DESCRIPTION("LZ4 decompressor"); --- 0-DAY kernel test infrastructureOpen Source Technology Center https://lists.01.org/pipermail/kbuild-all Intel Corporation -- To unsubscribe from this list: send the line "unsubscribe linux-crypto" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 1/4] lib: Update LZ4 compressor module
Hi Sven, [auto build test ERROR on linus/master] [also build test ERROR on v4.10-rc4 next-20170120] [if your patch is applied to the wrong git tree, please drop us a note to help improve the system] url: https://github.com/0day-ci/linux/commits/Sven-Schmidt/Update-LZ4-compressor-module/20170121-231418 config: x86_64-lkp (attached as .config) compiler: gcc-6 (Debian 6.2.0-3) 6.2.0 20160901 reproduce: # save the attached .config to linux build tree make ARCH=x86_64 Note: the linux-review/Sven-Schmidt/Update-LZ4-compressor-module/20170121-231418 HEAD 0472409e2a1c442b51502961aa6d83b866218953 builds fine. It only hurts bisectibility. All errors (new ones prefixed by >>): In file included from lib/decompress_unlz4.c:19:0: lib/decompress_unlz4.c: In function 'unlz4': >> lib/decompress_unlz4.c:75:22: error: implicit declaration of function >> 'lz4_compressbound' [-Werror=implicit-function-declaration] inp = large_malloc(lz4_compressbound(uncomp_chunksize)); ^ include/linux/decompress/mm.h:83:33: note: in definition of macro 'large_malloc' #define large_malloc(a) vmalloc(a) ^ cc1: some warnings being treated as errors vim +/lz4_compressbound +75 lib/decompress_unlz4.c e76e1fdf Kyungsik Lee 2013-07-08 13 #include "lz4/lz4_decompress.c" e76e1fdf Kyungsik Lee 2013-07-08 14 #else e76e1fdf Kyungsik Lee 2013-07-08 15 #include e76e1fdf Kyungsik Lee 2013-07-08 16 #endif e76e1fdf Kyungsik Lee 2013-07-08 17 #include e76e1fdf Kyungsik Lee 2013-07-08 18 #include e76e1fdf Kyungsik Lee 2013-07-08 @19 #include e76e1fdf Kyungsik Lee 2013-07-08 20 #include e76e1fdf Kyungsik Lee 2013-07-08 21 e76e1fdf Kyungsik Lee 2013-07-08 22 #include e76e1fdf Kyungsik Lee 2013-07-08 23 e76e1fdf Kyungsik Lee 2013-07-08 24 /* e76e1fdf Kyungsik Lee 2013-07-08 25 * Note: Uncompressed chunk size is used in the compressor side e76e1fdf Kyungsik Lee 2013-07-08 26 * (userspace side for compression). e76e1fdf Kyungsik Lee 2013-07-08 27 * It is hardcoded because there is not proper way to extract it e76e1fdf Kyungsik Lee 2013-07-08 28 * from the binary stream which is generated by the preliminary e76e1fdf Kyungsik Lee 2013-07-08 29 * version of LZ4 tool so far. e76e1fdf Kyungsik Lee 2013-07-08 30 */ e76e1fdf Kyungsik Lee 2013-07-08 31 #define LZ4_DEFAULT_UNCOMPRESSED_CHUNK_SIZE (8 << 20) e76e1fdf Kyungsik Lee 2013-07-08 32 #define ARCHIVE_MAGICNUMBER 0x184C2102 e76e1fdf Kyungsik Lee 2013-07-08 33 d97b07c5 Yinghai Lu 2014-08-08 34 STATIC inline int INIT unlz4(u8 *input, long in_len, d97b07c5 Yinghai Lu 2014-08-08 35long (*fill)(void *, unsigned long), d97b07c5 Yinghai Lu 2014-08-08 36long (*flush)(void *, unsigned long), d97b07c5 Yinghai Lu 2014-08-08 37u8 *output, long *posp, e76e1fdf Kyungsik Lee 2013-07-08 38void (*error) (char *x)) e76e1fdf Kyungsik Lee 2013-07-08 39 { e76e1fdf Kyungsik Lee 2013-07-08 40int ret = -1; e76e1fdf Kyungsik Lee 2013-07-08 41size_t chunksize = 0; e76e1fdf Kyungsik Lee 2013-07-08 42size_t uncomp_chunksize = LZ4_DEFAULT_UNCOMPRESSED_CHUNK_SIZE; e76e1fdf Kyungsik Lee 2013-07-08 43u8 *inp; e76e1fdf Kyungsik Lee 2013-07-08 44u8 *inp_start; e76e1fdf Kyungsik Lee 2013-07-08 45u8 *outp; d97b07c5 Yinghai Lu 2014-08-08 46long size = in_len; e76e1fdf Kyungsik Lee 2013-07-08 47 #ifdef PREBOOT e76e1fdf Kyungsik Lee 2013-07-08 48size_t out_len = get_unaligned_le32(input + in_len); e76e1fdf Kyungsik Lee 2013-07-08 49 #endif e76e1fdf Kyungsik Lee 2013-07-08 50size_t dest_len; e76e1fdf Kyungsik Lee 2013-07-08 51 e76e1fdf Kyungsik Lee 2013-07-08 52 e76e1fdf Kyungsik Lee 2013-07-08 53if (output) { e76e1fdf Kyungsik Lee 2013-07-08 54outp = output; e76e1fdf Kyungsik Lee 2013-07-08 55} else if (!flush) { e76e1fdf Kyungsik Lee 2013-07-08 56error("NULL output pointer and no flush function provided"); e76e1fdf Kyungsik Lee 2013-07-08 57goto exit_0; e76e1fdf Kyungsik Lee 2013-07-08 58} else { e76e1fdf Kyungsik Lee 2013-07-08 59outp = large_malloc(uncomp_chunksize); e76e1fdf Kyungsik Lee 2013-07-08 60if (!outp) { e76e1fdf Kyungsik Lee 2013-07-08 61error("Could not allocate output buffer"); e76e1fdf Kyungsik Lee 2013-07-08 62goto exit_0; e76e1fdf Kyungsik Lee 2013-07-08 63} e76e1fdf Kyungsik Lee 2013-07-08 64} e76e1fdf Kyungsik Lee 2013-07-08 65 e76e1fdf Kyungsik Lee 2013-07-08 66if (input && fill) { e76e1fdf Kyungsik Lee 2013-07-08 67error("Both input pointer and fill function provided,"); e76e1fdf Kyungsik Lee 2013-07-08 68goto exit_1; e76e1fdf Kyungsik Lee 2013-07-08 69} else if (input) { e76e1fdf Kyungsi
Re: [PATCH 1/4] lib: Update LZ4 compressor module
Hi Sven, [auto build test ERROR on linus/master] [also build test ERROR on v4.10-rc4 next-20170120] [if your patch is applied to the wrong git tree, please drop us a note to help improve the system] url: https://github.com/0day-ci/linux/commits/Sven-Schmidt/Update-LZ4-compressor-module/20170121-231418 config: i386-randconfig-r0-201703 (attached as .config) compiler: gcc-5 (Debian 5.4.1-2) 5.4.1 20160904 reproduce: # save the attached .config to linux build tree make ARCH=i386 Note: the linux-review/Sven-Schmidt/Update-LZ4-compressor-module/20170121-231418 HEAD 0472409e2a1c442b51502961aa6d83b866218953 builds fine. It only hurts bisectibility. All errors (new ones prefixed by >>): fs/pstore/platform.c: In function 'allocate_lz4': >> fs/pstore/platform.c:369:20: error: implicit declaration of function >> 'lz4_compressbound' [-Werror=implicit-function-declaration] big_oops_buf_sz = lz4_compressbound(psinfo->bufsize); ^ cc1: some warnings being treated as errors vim +/lz4_compressbound +369 fs/pstore/platform.c 8cfc8ddc Geliang Tang 2016-02-18 363 8cfc8ddc Geliang Tang 2016-02-18 364 return outlen; 8cfc8ddc Geliang Tang 2016-02-18 365 } 8cfc8ddc Geliang Tang 2016-02-18 366 8cfc8ddc Geliang Tang 2016-02-18 367 static void allocate_lz4(void) 8cfc8ddc Geliang Tang 2016-02-18 368 { 8cfc8ddc Geliang Tang 2016-02-18 @369 big_oops_buf_sz = lz4_compressbound(psinfo->bufsize); 8cfc8ddc Geliang Tang 2016-02-18 370 big_oops_buf = kmalloc(big_oops_buf_sz, GFP_KERNEL); 8cfc8ddc Geliang Tang 2016-02-18 371 if (big_oops_buf) { 8cfc8ddc Geliang Tang 2016-02-18 372 workspace = kmalloc(LZ4_MEM_COMPRESS, GFP_KERNEL); :: The code at line 369 was first introduced by commit :: 8cfc8ddc99df9509a46043b14af81f5c6a223eab pstore: add lzo/lz4 compression support :: TO: Geliang Tang :: CC: Kees Cook --- 0-DAY kernel test infrastructureOpen Source Technology Center https://lists.01.org/pipermail/kbuild-all Intel Corporation .config.gz Description: application/gzip