FrankChen021 commented on PR #20228: URL: https://github.com/apache/druid/pull/20228#issuecomment-5519314433
> > removes decoding for legacy pre-1.0 v0.4-v0.7 frames > > @FrankChen021 what does this mean? Assuming this means that it can't read stuff written with zstd v0.4-0.7, the first version of zstd druid supported was `1.3.3-1` in #6224, so is this an actual problem? You’re right that Druid itself never produced these frames. The affected formats were produced by upstream native Zstd/libzstd 0.4.x–0.7.x; Druid first added zstd support at 1.3.3-1. However, that support was for decompressing externally supplied .zst files. Druid still documents zst as a supported batch-ingestion format, and the current path delegates these files to zstd-jni. I verified that the upstream xml_v04.zst fixture decodes to 5,345,280 bytes with zstd-jni 1.5.7-11 but fails with Unknown frame descriptor on 1.5.7-15. The 1.5.7-16 README confirms that 1.5.7-12 was the last release with legacy-format support. So there is no compatibility issue for Druid-generated segments, but there is a conditional compatibility regression for users ingesting externally generated v0.4–v0.7 .zst files. We do not have evidence that such files exist in users' Druid deployments. If Druid’s .zst contract is intended to cover only the final standard format, this can be considered out of scope; otherwise we should retain a legacy-capable version or explicitly document the migration. Or if the 1.5.7.11 has CVEs that we must resolve. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
