FrankChen021 commented on PR #20382: URL: https://github.com/apache/druid/pull/20382#issuecomment-5739322447
Two things that I would like to ask for clarification: In this PR(and other object storage like deep storage), the new property`druid.storage.zip` is used to determine whether segment files are compressed as `index.zip` or not. However, in #18982 , we introduced a new property `druid.storage.compressionFormat` for HDFS deep storage to indicate which compression format is used for compression(zip and lz4), that PR didn't bring the change to object storage like S3, Azure ang google cloud(because we don't use object storage), but architecturally, that property should apply to all deep storages. So I think we should use the `druid.storage.compressionFormat`( with a new `NONE` enum supported ) to replace the `druid.storage.zip`. Secondly, do you have any data that shows the segment size without compression as index.zip in the deep storage? As [commented in #18982](https://github.com/apache/druid/pull/18982#issuecomment-3846035169), zip 0 (which does not comrpess but only pack files together) has a size of 11GiB, while the default zip level currently we are using now shows the size is about 2.48GB, this means that if segments are not compressed, file size in deep storage should be much larger. I'm not sure if such file size increasing is consistent in your case. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
