Hey Vidya Sagar, *- Is the code actually using this compression library? Can this vulnerability issue be ignored?*
I glanced at the LZ4 in Flink. IIUC, LZ4 is used to compress blocks in batch table which was introduced by FLINK-11858[1], FLINK-23447[2] bumped it to 1.8. So, LZ4 is actually used by some code. *- * *would it be ok if we upgrade the version of LZ4 in our local cloned code base?* I guess you can refer to FLINK-23447[2] to upgrade it. I am not familiar with batch mode, AFAIK, flink-table-runtime[3] would definitely be affected. [1] https://issues.apache.org/jira/browse/FLINK-11858 [2] https://issues.apache.org/jira/browse/FLINK-23447 [3] https://github.com/apache/flink/blob/master/flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/runtime/operators/sort/BinaryExternalSorter.java#L213 Martijn Visser <martijnvis...@apache.org> 于2022年12月9日周五 18:23写道： > Hi Vidya, > > Please keep in mind that the Flink project is driven by volunteers. If > you're noticing an outdated version for the lz4 compression library and an > update is required, it would be great if you can open the PR to update that > dependency yourself. > > Best regards, > > Martijn > > On Thu, Dec 8, 2022 at 10:31 PM Vidya Sagar Mula <mulasa...@gmail.com> > wrote: > >> Thank you Yanfei for taking this issue as a bug and planning a fix in the >> upcoming version. >> >> I have another vulnerability bug coming on our product. It is related to >> the "LZ4" compression library version. Can you please take a look at this >> link? >> https://nvd.nist.gov/vuln/detail/CVE-2019-17543 >> >> I have noticed that, Flink code base is using "*<lz4.version>1.8.0</lz4* >> *.version>*". Vulnerability is present for the versions *before 1.9.2.* >> >> https://github.com/apache/flink/blob/master/pom.xml >> >> Can you please look into this issue also and address it in the coming >> releases? >> >> Questions: >> ----------- >> - Is the code actually using this compression library? Can this >> vulnerability issue be ignored? >> >> - Can you please let me know if this is going to be addressed. If yes, >> until we move to the new Flink version to get the latest changes, would it >> be ok if we upgrade the version of LZ4 in our local cloned code base? I >> would like to understand the impact if we make changes in our local Flink >> code with regards to testing efforts and any other affected modules? >> >> Can you please clarify this? >> >> Thanks, >> Vidya Sagar. >> >> >> On Wed, Dec 7, 2022 at 7:59 AM Yanfei Lei <fredia...@gmail.com> wrote: >> >>> Hi Vidya Sagar, >>> >>> Thanks for bringing this up. >>> >>> The RocksDB state backend defaults to Snappy[1]. If the compression >>> option is not specifically configured, this vulnerability of ZLIB has no >>> effect on the Flink application for the time being. >>> >>> *> is there any plan in the coming days to address this? * >>> >>> The FRocksDB 6.20.3-ververica-1.0 >>> <https://mvnrepository.com/artifact/com.ververica/frocksdbjni/6.20.3-ververica-1.0> >>> does >>> depend on ZLIB 1.2.11, FLINK-30321 is created to address this. >>> >>> *> If this needs to be fixed, is there any plan from Ververica to >>> address this vulnerability?* >>> >>> Yes, we plan to publish a new version of FRocksDB[3] in Flink 1.17, and >>> FLINK-30321 >>> would be included in the new release. >>> >>> *> how to address this vulnerability issue as this is coming as a high >>> severity blocking issue to our product.* >>> >>> As a kind of mitigation, don't configure ZLIB compression for RocksDB >>> state backend. >>> If ZLIB must be used now and your product can't wait, maybe you can >>> refer to this release document[4] to release your own version. >>> >>> [1] https://github.com/facebook/rocksdb/wiki/Compression >>> [2] https://issues.apache.org/jira/browse/FLINK-30321 >>> [3] https://cwiki.apache.org/confluence/display/FLINK/1.17+Release >>> [4] >>> https://github.com/ververica/frocksdb/blob/FRocksDB-6.20.3/FROCKSDB-RELEASE.md >>> >>> -- >>> Best, >>> Yanfei >>> Ververica (Alibaba) >>> >>> Vidya Sagar Mula <mulasa...@gmail.com> 于2022年12月7日周三 06:47写道： >>> >>>> Hi, >>>> >>>> There is a ZLIB vulnerability reported by the official National >>>> Vulnerability Database. This vulnerability causes memory corruption while >>>> deflating with ZLIB version less than 1.2.12. >>>> Here is the link for details... >>>> >>>> >>>> https://nvd.nist.gov/vuln/detail/cve-2018-25032#vulnCurrentDescriptionTitle >>>> >>>> *How is it linked to Flink?: * >>>> In the Flink statebackend rocksdb, there is ZLIB version 1.2.11 is used >>>> as part of the .so file. Hence, there is vulnerability exposure here. >>>> >>>> *Flink code details/links:* >>>> I am seeing the latest Flink code base where the statebackend rocksdb >>>> library *(frocksdbjni)* is coming from Ververica. The pom.xml >>>> dependency snapshot is here >>>> >>>> >>>> https://github.com/apache/flink/blob/master/flink-state-backends/flink-statebackend-rocksdb/pom.xml >>>> >>>> <dependency> >>>> >>>> <groupId>com.ververica</groupId> >>>> >>>> <artifactId>frocksdbjni</artifactId> >>>> >>>> <version>6.20.3-ververica-1.0</version> >>>> >>>> </dependency> >>>> >>>> >>>> When I see the frocksdbjni code base, the makefile is pointing to >>>> ZLIB_VER=1.2.11. This ZLIB version is vulnerable as per the NVD. >>>> >>>> https://github.com/ververica/frocksdb/blob/FRocksDB-6.20.3/Makefile >>>> >>>> *Questions:* >>>> >>>> - This vulnerability is marked as HIGH severity. How is it addressed at >>>> the Flink/Flink Stateback RocksDb? If not now, is there any plan in the >>>> coming days to address this? >>>> >>>> - As the Statebackend RocksDb is coming from Ververica, I am not seeing >>>> any latest artifacts published from them. As per the Maven Repository, the >>>> latest version is 6.20.3-ververica-1.0 >>>> <https://mvnrepository.com/artifact/com.ververica/frocksdbjni/6.20.3-ververica-1.0> >>>> and >>>> this is the one used in the Flink code base. >>>> >>>> https://mvnrepository.com/artifact/com.ververica/frocksdbjni >>>> >>>> If this needs to be fixed, is there any plan from Ververica to address >>>> this vulnerability? >>>> >>>> - From the Flink user perspective, it is not simple to make the changes >>>> to .so file locally. How are the Flink user companies addressing this >>>> vulnerability as it needs changes to the .SO file? >>>> >>>> Overall, my main question to the community is, how to address this >>>> vulnerability issue as this is coming as a high severity blocking issue to >>>> our product. >>>> >>>> Please provide the inputs/suggestions at the earliest. >>>> >>>> Thanks, >>>> Vidya Sagar. >>>> >>>> >>>> >>>> >>>> >>> -- Best, Yanfei
