Hello Alexey Serbin, Kudu Jenkins, Adar Dembo, Grant Henke, Hao Hao,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/13384
to look at the new patch set (#2).
Change subject: hms: fix flakiness of master-stress-test from managed tables
......................................................................
hms: fix flakiness of master-stress-test from managed tables
The switch to managed tables opened the door for the race described in
HIVE-21759, described with more Kudu-specific details below:
1. Master A* sends a create table request for T.
2. The HMS creates a directory for that table and an entry in the HMS
catalog.
3. Before A* can write this to the Kudu catalog, master B* becomes
leader. A's write fails, and A sends a drop request to the HMS to
roll back the created metadata for T.
4. The HMS receives A's drop request, and the HMS entry is dropped.
Note: In the process of dropping a table from the HMS, the last thing
the operation will do is drop any underlying data in the HMS (the
directory, in this case).
5. The create request for T is retried by B*. Because the HMS entry for
T is already dropped in the HMS, the request will be processed.
6. The HMS finally drops the underlying directory for T created by A.
7. Along the path of creating the table, the request from B* hits a "Not
found" exception, expecting for the directory for T to exist.
8. The create table request is failed, since the HMS transaction to
create the table is failed.
The end result is that the table doesn't exist in Kudu or in the HMS
following this sequence of events. This patch addresses this by catching
such errors with the HMS enabled and checking that an underlying Kudu
table and HMS entry is not created if so.
Without this patch, master-stress-test failed 24/300 times with 7 stress
threads. With it, it failed 9/300 times (mostly due to KUDU-2779).
Change-Id: I1233e5b133035d36f98ce08265c0273570caa454
---
M src/kudu/integration-tests/master-stress-test.cc
1 file changed, 36 insertions(+), 0 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/84/13384/2
--
To view, visit http://gerrit.cloudera.org:8080/13384
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I1233e5b133035d36f98ce08265c0273570caa454
Gerrit-Change-Number: 13384
Gerrit-PatchSet: 2
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <a...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <aser...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Grant Henke <granthe...@apache.org>
Gerrit-Reviewer: Hao Hao <hao....@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
