Adar Dembo has submitted this change and it was merged. Change subject: KUDU-2137: protect against concurrent schema version change and tablet drop ......................................................................
KUDU-2137: protect against concurrent schema version change and tablet drop Try as I might, I could not reproduce the failure in the bug report. I looped the failed test several thousand times. I also looped the entire alter_table-test suite a thousand times. Finally, I wrote a unit test that hammers one table with concurrent add column, add partition, and drop partition operations. Nothing worked. So, here's my best guess at what's going on: if a tablet is dropped while the master is processing its report, it's conceivable that we could wind up in TabletInfo::set_reported_schema_version() with the table spinlock held just after TableInfo::AddRemoveTablets() dropped the tablet. This would cause us to decrement the tablet's "old" schema version from the table's count map twice: once when dropping the tablet and a second time in set_reported_schema_version(). The fix is straight-forward: after acquiring both spinlocks, double check that the tablet is still a member of the table. We need to take the tablet metadata lock to do this, but the saving grace is that tablet partition keys are immutable, so this lock acquisition needn't overlap with the spinlocks. Change-Id: I371fc310a97ae94ec2ebf04405db99c5f2937e1a Reviewed-on: http://gerrit.cloudera.org:8080/7996 Tested-by: Kudu Jenkins Reviewed-by: Dan Burkert <danburk...@apache.org> Reviewed-by: Alexey Serbin <aser...@cloudera.com> --- M src/kudu/master/catalog_manager.cc 1 file changed, 21 insertions(+), 2 deletions(-) Approvals: Dan Burkert: Looks good to me, approved Alexey Serbin: Looks good to me, approved Kudu Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/7996 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: I371fc310a97ae94ec2ebf04405db99c5f2937e1a Gerrit-PatchSet: 6 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: Adar Dembo <a...@cloudera.com> Gerrit-Reviewer: Adar Dembo <a...@cloudera.com> Gerrit-Reviewer: Alexey Serbin <aser...@cloudera.com> Gerrit-Reviewer: Dan Burkert <danburk...@apache.org> Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Todd Lipcon <t...@apache.org>
