Bharath Vissapragada has uploaded this change for review. ( http://gerrit.cloudera.org:8080/12572
Change subject: IMPALA-8243: Fix racy access to nonPartFieldSchemas_ ...................................................................... IMPALA-8243: Fix racy access to nonPartFieldSchemas_ ** Please refer to the jira for the full stacktrace. When constructing the HMS partition state for a given partition, we are leaking a reference to HdfsTable#nonPartFieldSchemas_ list. The construction happens under a table lock. But once the thread exits the lock scope, the source list could be racily modified by another operation (say refresh) and that interferes with the original thread if it tries to access the list.` The fix is to make a shallow copy of the source list so that any changes to the list do not affect the original caller. This was found by a stress test under heavy concurrency of refresh operations + GetPartialCatalogObject() calls. Testing: --------- - I tried a bunch of combinations of operations in the unit-test framework but I couldn't reproduce the stack trace, probably because the operations are very short-lived - However, after deploying this patched jar on the stress test cluster, this exception never happened again. Change-Id: I7d68b54af2ba954cf0ffa7b2533cde7be835be77 --- M fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java 1 file changed, 4 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/72/12572/1 -- To view, visit http://gerrit.cloudera.org:8080/12572 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I7d68b54af2ba954cf0ffa7b2533cde7be835be77 Gerrit-Change-Number: 12572 Gerrit-PatchSet: 1 Gerrit-Owner: Bharath Vissapragada <bhara...@cloudera.com>
