Alex Behm has uploaded a new change for review. http://gerrit.cloudera.org:8080/6127
Change subject: IMPALA-4902: Copy parameters map in HdfsPartition.toThrift(). ...................................................................... IMPALA-4902: Copy parameters map in HdfsPartition.toThrift(). The bug: When generating the toThrift() of an HdfsTable, each THdfsPartition used to contain a reference to its partition's parameters map. As a result, one thread trying to serialize a thrift table returned by toThrift() could conflict with another thread updating the parameters maps of the table partitions. The fix: Create a shallow copy of the parameters map in HdfsPartition.toThrift(). This means that toThrift() itself must be protected from concurrent modifications to the parameters map. Callers of toThrift() are now required to hold the table lock. One place where the lock was not already held needed to be adjusted. Testing: - I was unable to reproduce the issue locally, but the stacks from the JIRAs point directly to the parameters map, and the races are pretty obvious from looking at the code. - Passed a core/hdfs private run. Change-Id: Ic11277ad5512d2431cd3cc791715917c95395ddf --- M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java M fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java M fe/src/main/java/org/apache/impala/catalog/Table.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/test/java/org/apache/impala/catalog/CatalogObjectToFromThriftTest.java 5 files changed, 123 insertions(+), 101 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/27/6127/1 -- To view, visit http://gerrit.cloudera.org:8080/6127 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ic11277ad5512d2431cd3cc791715917c95395ddf Gerrit-PatchSet: 1 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Alex Behm <alex.b...@cloudera.com>
