Good news! Just added `pure Go` build mode <https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/README.md#pure-go-cgo_enabled0> to VictoriaMetrics. It uses github.com/klauspost/compress/zstd instead of github.com/valyala/gozstd. Preliminary results show slightly lower compression ratio at slightly better compression speed. This is because VictoriaMetrics requests higher compression levels from zstd, which aren't implemented yet in pure Go zstd compressor. So it falls back to lower compression level with faster speed.
The decompression speed is also slightly lower for small blocks due to the allocation inside DecodeAll <https://github.com/klauspost/compress/blob/415e5345507ece8609410be3d156d0e37e70b3b6/zstd/decoder.go#L292> . On Thursday, July 11, 2019 at 7:10:34 PM UTC+3, Klaus Post wrote: > > On Thursday, 11 July 2019 17:37:09 UTC+2, Aliaksandr Valialkin wrote: >> >> >> >> This is really great idea! Will try implementing it. >> >> Does github.com/klauspost/compress support all the levels for data >> decompression? VictoriaMetrics varies compression level depending on the >> data type. It would be great if github.com/klauspost/compress >> <https://www.google.com/url?q=https%3A%2F%2Fgithub.com%2Fklauspost%2Fcompress&sa=D&sntz=1&usg=AFQjCNGoED7U_RlamcOX9HL71ukxaq1QkA> >> could >> decompress data compressed by the upstream zstd on higher compression >> levels. >> > > Decompression will work for all input. It is implementing the full spec. > > Compression has "Fastest" and "Default" implemented, roughly matching > level 1 and 3 in zstd in speed and performance. I plan on adding something > around level 7-9 (as Better) and level 19 (as Best). But for it to be > useful I have mainly focused on the fastest modes. I also am planning more > concurrency in compression and decompression for streams - blocks will > probably remain as single goroutines. For now I am taking a small break and > having a bit of fun revisiting deflate and experimenting with Snappy. > > If there is anything I can do to help feel free to ping me. > > > /Klaus > > >> >> -- >> Best Regards, >> >> Aliaksandr >> > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/1b1d21b6-5263-499f-a625-641910b8e079%40googlegroups.com.
