[jira] [Updated] (HIVE-3809) Concurrency issue in RCFile: multiple threads can use the same decompressor
[ https://issues.apache.org/jira/browse/HIVE-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Bautin updated HIVE-3809: - Attachment: 0001-HIVE-3809-Concurrency-issue-in-RCFile-multiple-threa.patch > Concurrency issue in RCFile: multiple threads can use the same decompressor > --- > > Key: HIVE-3809 > URL: https://issues.apache.org/jira/browse/HIVE-3809 > Project: Hive > Issue Type: Bug >Reporter: Mikhail Bautin >Assignee: Mikhail Bautin >Priority: Critical > Fix For: 0.11.0 > > Attachments: > 0001-HIVE-3809-Concurrency-issue-in-RCFile-multiple-threa.patch, > 0001-HIVE-3809-Decompressors-should-only-be-returned-to-t.patch, D7419.1.patch > > > RCFile is not thread-safe, even if each reader is only used by one thread as > intended, because it is possible to return decompressors to the pool multiple > times by calling close on the reader multiple times. Then, different threads > can pick up the same decompressor twice from the pool, resulting in > decompression failures. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3809) Concurrency issue in RCFile: multiple threads can use the same decompressor
[ https://issues.apache.org/jira/browse/HIVE-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-3809: --- Resolution: Fixed Fix Version/s: 0.11.0 Status: Resolved (was: Patch Available) Committed to trunk. Thanks, Mikhail! > Concurrency issue in RCFile: multiple threads can use the same decompressor > --- > > Key: HIVE-3809 > URL: https://issues.apache.org/jira/browse/HIVE-3809 > Project: Hive > Issue Type: Bug >Reporter: Mikhail Bautin >Assignee: Mikhail Bautin >Priority: Critical > Fix For: 0.11.0 > > Attachments: > 0001-HIVE-3809-Decompressors-should-only-be-returned-to-t.patch, D7419.1.patch > > > RCFile is not thread-safe, even if each reader is only used by one thread as > intended, because it is possible to return decompressors to the pool multiple > times by calling close on the reader multiple times. Then, different threads > can pick up the same decompressor twice from the pool, resulting in > decompression failures. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3809) Concurrency issue in RCFile: multiple threads can use the same decompressor
[ https://issues.apache.org/jira/browse/HIVE-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-3809: --- Status: Patch Available (was: Open) +1 > Concurrency issue in RCFile: multiple threads can use the same decompressor > --- > > Key: HIVE-3809 > URL: https://issues.apache.org/jira/browse/HIVE-3809 > Project: Hive > Issue Type: Bug >Reporter: Mikhail Bautin >Assignee: Mikhail Bautin >Priority: Critical > Attachments: > 0001-HIVE-3809-Decompressors-should-only-be-returned-to-t.patch, D7419.1.patch > > > RCFile is not thread-safe, even if each reader is only used by one thread as > intended, because it is possible to return decompressors to the pool multiple > times by calling close on the reader multiple times. Then, different threads > can pick up the same decompressor twice from the pool, resulting in > decompression failures. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3809) Concurrency issue in RCFile: multiple threads can use the same decompressor
[ https://issues.apache.org/jira/browse/HIVE-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-3809: --- Assignee: Mikhail Bautin > Concurrency issue in RCFile: multiple threads can use the same decompressor > --- > > Key: HIVE-3809 > URL: https://issues.apache.org/jira/browse/HIVE-3809 > Project: Hive > Issue Type: Bug >Reporter: Mikhail Bautin >Assignee: Mikhail Bautin >Priority: Critical > Attachments: > 0001-HIVE-3809-Decompressors-should-only-be-returned-to-t.patch, D7419.1.patch > > > RCFile is not thread-safe, even if each reader is only used by one thread as > intended, because it is possible to return decompressors to the pool multiple > times by calling close on the reader multiple times. Then, different threads > can pick up the same decompressor twice from the pool, resulting in > decompression failures. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3809) Concurrency issue in RCFile: multiple threads can use the same decompressor
[ https://issues.apache.org/jira/browse/HIVE-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Bautin updated HIVE-3809: - Attachment: 0001-HIVE-3809-Decompressors-should-only-be-returned-to-t.patch Attaching a manually generated patch. > Concurrency issue in RCFile: multiple threads can use the same decompressor > --- > > Key: HIVE-3809 > URL: https://issues.apache.org/jira/browse/HIVE-3809 > Project: Hive > Issue Type: Bug >Reporter: Mikhail Bautin >Priority: Critical > Attachments: > 0001-HIVE-3809-Decompressors-should-only-be-returned-to-t.patch, D7419.1.patch > > > RCFile is not thread-safe, even if each reader is only used by one thread as > intended, because it is possible to return decompressors to the pool multiple > times by calling close on the reader multiple times. Then, different threads > can pick up the same decompressor twice from the pool, resulting in > decompression failures. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3809) Concurrency issue in RCFile: multiple threads can use the same decompressor
[ https://issues.apache.org/jira/browse/HIVE-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HIVE-3809: -- Attachment: D7419.1.patch mbautin requested code review of "[jira] [HIVE-3809] Concurrency issue in RCFile: multiple threads can use the same decompressor". Reviewers: kevinwilfong, njain, ashutoshc, cwsteinbach, edwardcapriolo, JIRA Making sure that decompressors are only returned to the pool once when readers are closed in RCFile. Otherwise the same decompressor can end up in the pool multiple times, and multiple threads would pick it up and try to use it concurrently. TEST PLAN Unit tests REVISION DETAIL https://reviews.facebook.net/D7419 AFFECTED FILES ql/src/java/org/apache/hadoop/hive/ql/io/RCFile.java MANAGE HERALD DIFFERENTIAL RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/17709/ To: kevinwilfong, njain, ashutoshc, cwsteinbach, edwardcapriolo, JIRA, mbautin > Concurrency issue in RCFile: multiple threads can use the same decompressor > --- > > Key: HIVE-3809 > URL: https://issues.apache.org/jira/browse/HIVE-3809 > Project: Hive > Issue Type: Bug >Reporter: Mikhail Bautin >Priority: Critical > Attachments: D7419.1.patch > > > RCFile is not thread-safe, even if each reader is only used by one thread as > intended, because it is possible to return decompressors to the pool multiple > times by calling close on the reader multiple times. Then, different threads > can pick up the same decompressor twice from the pool, resulting in > decompression failures. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
