[GitHub] [parquet-mr] shangxinli commented on pull request #959: PARQUET-2126: Make cached (de)compressors thread-safe

2022-07-24 Thread GitBox
shangxinli commented on PR #959: URL: https://github.com/apache/parquet-mr/pull/959#issuecomment-1193390004 @theosib-amazon, I am not concerned if release/close isn't called and I agree the caller must call release/close after finishing. My question is that before release/close is called, t

[GitHub] [parquet-mr] shangxinli commented on pull request #959: PARQUET-2126: Make cached (de)compressors thread-safe

2022-07-19 Thread GitBox
shangxinli commented on PR #959: URL: https://github.com/apache/parquet-mr/pull/959#issuecomment-1189198163 @theosib-amazon Do you still have time for addressing the feedback? I think we are very close to merge. -- This is an automated message from the Apache Git Service. To respond to t

[GitHub] [parquet-mr] shangxinli commented on pull request #959: PARQUET-2126: Make cached (de)compressors thread-safe

2022-06-09 Thread GitBox
shangxinli commented on PR #959: URL: https://github.com/apache/parquet-mr/pull/959#issuecomment-1151340364 Thanks for addressing the feedback! What I meant was that ideally when 'Threads terminate' happens, it should clean up the compressor/decompressor immediately. I understand we won't l

[GitHub] [parquet-mr] shangxinli commented on pull request #959: PARQUET-2126: Make cached (de)compressors thread-safe

2022-05-16 Thread GitBox
shangxinli commented on PR #959: URL: https://github.com/apache/parquet-mr/pull/959#issuecomment-1127847443 My question is when a thread exits, we don't have a corresponding evict operation on the map. Using thread pool might be OK if the thread object is not changed, but not sure if there

[GitHub] [parquet-mr] shangxinli commented on pull request #959: PARQUET-2126: Make cached (de)compressors thread-safe

2022-05-15 Thread GitBox
shangxinli commented on PR #959: URL: https://github.com/apache/parquet-mr/pull/959#issuecomment-1127067783 If we change it to be per thread, then would it be a problem in the scenario where short living threads come and go. When the thread stopped, we might not know and leak here.