Re: [PR] Add GZIP Compression Codec (#11434) [pinot]

2024-04-05 Thread via GitHub
marregui commented on code in PR #12668: URL: https://github.com/apache/pinot/pull/12668#discussion_r1554062252 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/io/compression/GzipDecompressor.java: ## @@ -0,0 +1,63 @@ +/** + * Licensed to the Apache Software

Re: [PR] Add GZIP Compression Codec (#11434) [pinot]

2024-04-04 Thread via GitHub
Jackie-Jiang commented on code in PR #12668: URL: https://github.com/apache/pinot/pull/12668#discussion_r1552693269 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/io/compression/GzipDecompressor.java: ## @@ -0,0 +1,63 @@ +/** + * Licensed to the Apache Softw

Re: [PR] Add GZIP Compression Codec (#11434) [pinot]

2024-04-04 Thread via GitHub
marregui commented on PR #12668: URL: https://github.com/apache/pinot/pull/12668#issuecomment-2036617662 @siddharthteotia afaik the api here is to receive two buffers for the decompress/compress operations, one containing the bytes to be processed, the other to hold the results. I w

Re: [PR] Add GZIP Compression Codec (#11434) [pinot]

2024-04-04 Thread via GitHub
siddharthteotia commented on PR #12668: URL: https://github.com/apache/pinot/pull/12668#issuecomment-2036354732 Thank you for the contribution. I am curious if we can put together a simple table (esp for STRING dimensions) highlighting the comp ratio and decomp speed for LZ4, SNAPPY,

Re: [PR] Add GZIP Compression Codec (#11434) [pinot]

2024-04-03 Thread via GitHub
marregui commented on code in PR #12668: URL: https://github.com/apache/pinot/pull/12668#discussion_r1549817440 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/io/compression/GzipDecompressor.java: ## @@ -0,0 +1,63 @@ +/** + * Licensed to the Apache Software

Re: [PR] Add GZIP Compression Codec (#11434) [pinot]

2024-04-03 Thread via GitHub
marregui commented on code in PR #12668: URL: https://github.com/apache/pinot/pull/12668#discussion_r1549129093 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/io/compression/GzipDecompressor.java: ## @@ -0,0 +1,63 @@ +/** + * Licensed to the Apache Software

Re: [PR] Add GZIP Compression Codec (#11434) [pinot]

2024-03-28 Thread via GitHub
Jackie-Jiang commented on code in PR #12668: URL: https://github.com/apache/pinot/pull/12668#discussion_r1543370962 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/io/compression/GzipDecompressor.java: ## @@ -0,0 +1,63 @@ +/** + * Licensed to the Apache Softw

Re: [PR] Add GZIP Compression Codec (#11434) [pinot]

2024-03-28 Thread via GitHub
Jackie-Jiang merged PR #12668: URL: https://github.com/apache/pinot/pull/12668 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot

Re: [PR] Add GZIP Compression Codec (#11434) [pinot]

2024-03-27 Thread via GitHub
marregui commented on code in PR #12668: URL: https://github.com/apache/pinot/pull/12668#discussion_r1541469193 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/readers/forward/BaseChunkForwardIndexReader.java: ## @@ -446,9 +446,10 @@ public void

Re: [PR] Add GZIP Compression Codec (#11434) [pinot]

2024-03-26 Thread via GitHub
Jackie-Jiang commented on code in PR #12668: URL: https://github.com/apache/pinot/pull/12668#discussion_r1539859364 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/readers/forward/BaseChunkForwardIndexReader.java: ## @@ -446,9 +446,10 @@ public

Re: [PR] Add GZIP Compression Codec (#11434) [pinot]

2024-03-26 Thread via GitHub
marregui commented on code in PR #12668: URL: https://github.com/apache/pinot/pull/12668#discussion_r1539288679 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/io/compression/GzipCompressor.java: ## @@ -0,0 +1,62 @@ +/** + * Licensed to the Apache Software Fo

Re: [PR] Add GZIP Compression Codec (#11434) [pinot]

2024-03-26 Thread via GitHub
marregui commented on code in PR #12668: URL: https://github.com/apache/pinot/pull/12668#discussion_r1539258153 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/io/compression/GzipCompressor.java: ## @@ -0,0 +1,62 @@ +/** + * Licensed to the Apache Software Fo

Re: [PR] Add GZIP Compression Codec (#11434) [pinot]

2024-03-25 Thread via GitHub
Jackie-Jiang commented on code in PR #12668: URL: https://github.com/apache/pinot/pull/12668#discussion_r1538361504 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/io/compression/GzipDecompressor.java: ## @@ -0,0 +1,59 @@ +/** + * Licensed to the Apache Softw

Re: [PR] Add GZIP Compression Codec (#11434) [pinot]

2024-03-22 Thread via GitHub
marregui commented on PR #12668: URL: https://github.com/apache/pinot/pull/12668#issuecomment-2014724301 [benchmark_no_dict_string_compression.txt](https://github.com/apache/pinot/files/14720625/benchmark_no_dict_string_compression.txt) I finally managed to run the benchmark for strin

Re: [PR] Add GZIP Compression Codec (#11434) [pinot]

2024-03-22 Thread via GitHub
marregui commented on code in PR #12668: URL: https://github.com/apache/pinot/pull/12668#discussion_r1535312725 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/io/compression/GZIPCompressor.java: ## @@ -0,0 +1,62 @@ +/** + * Licensed to the Apache Software Fo

Re: [PR] Add GZIP Compression Codec (#11434) [pinot]

2024-03-22 Thread via GitHub
marregui commented on code in PR #12668: URL: https://github.com/apache/pinot/pull/12668#discussion_r1535312278 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/io/compression/GZIPCompressor.java: ## @@ -0,0 +1,62 @@ +/** + * Licensed to the Apache Software Fo

Re: [PR] Add GZIP Compression Codec (#11434) [pinot]

2024-03-20 Thread via GitHub
marregui commented on PR #12668: URL: https://github.com/apache/pinot/pull/12668#issuecomment-2010137824 [benchmark_no_dict_integer_compression.txt](https://github.com/apache/pinot/files/14670316/benchmark_no_dict_integer_compression.txt) [benchmark_no_dict_long_compression.txt](https://

Re: [PR] Add GZIP Compression Codec (#11434) [pinot]

2024-03-20 Thread via GitHub
marregui commented on PR #12668: URL: https://github.com/apache/pinot/pull/12668#issuecomment-2009186253 > Thanks for the contribution! Can you please take a look at the thread safety issue? It will be good if you can also share some perf numbers since you've added it to the benchmark class

Re: [PR] Add GZIP Compression Codec (#11434) [pinot]

2024-03-19 Thread via GitHub
codecov-commenter commented on PR #12668: URL: https://github.com/apache/pinot/pull/12668#issuecomment-2008069935 ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/12668?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&u

Re: [PR] Add GZIP Compression Codec (#11434) [pinot]

2024-03-19 Thread via GitHub
Jackie-Jiang commented on code in PR #12668: URL: https://github.com/apache/pinot/pull/12668#discussion_r1530997294 ## pinot-segment-local/src/main/java/org/apache/pinot/segment/local/io/compression/GZIPCompressor.java: ## @@ -0,0 +1,62 @@ +/** + * Licensed to the Apache Softwar

[PR] Add GZIP Compression Codec (#11434) [pinot]

2024-03-18 Thread via GitHub
marregui opened a new pull request, #12668: URL: https://github.com/apache/pinot/pull/12668 As per the request in https://github.com/apache/pinot/issues/11434 . -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the U