yandrey321 commented on PR #10651: URL: https://github.com/apache/ozone/pull/10651#issuecomment-5240984168
> > I suggest referencing OzoneClientKeyReadWriteListOps, where key names can be deterministically calculated or generated for benchmarking instead of storing and computing content digests/histories. > > Thanks @chungen0126 for the suggestion! > > Fair question on the client-side cost. My reading is that `ockrw` is a pure benchmark, it reads and discards the bytes, so it needs no digest. This one is in the validator family (`dfsv` / `ockv`), which already digests every read, and [HDDS-14524](https://issues.apache.org/jira/browse/HDDS-14524) explicitly asks to keep a hash per file and validate it on read-back. Dropping validation would make this overlap with `ockrw`, so I'd rather keep it. > > MD5 here isn't a deliberate choice, it's just what the shared helper in `BaseFreonGenerator` uses. I switched to it per your earlier suggestion to reuse it. > > That said, I agree MD5 is not the ideal choice on the hot path, and @yandrey321 raised the same thing earlier. Since that helper is shared with the existing validators, changing it in place would silently change them too (I originally wanted to keep the PR scoped), so what do you think about a --digest option scoped to this command, defaulting to CRC32C? > > Does that direction sound reasonable? 🙏 I'd suggest replacing MD5 with CRC32, which is faster comparing with MD5. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
