Todd Lipcon has uploaded a new change for review. http://gerrit.cloudera.org:8080/5513
Change subject: KUDU-1775 (part 3): enforce max cell size and max PK size ...................................................................... KUDU-1775 (part 3): enforce max cell size and max PK size This adds limits on the size of any individual cell as well as a limit on the maximum size of an encoded key. * Large cells are known to cause problems such as KUDU-1524 (crash during flush). * Large PKs have been seen to cause problems where the key column's cfile footer increases beyond the hard-coded maximum size of 64KB. If a user attempts to insert a non-conforming row, or update a cell to be larger than the maximum, they will receive an InvalidArgument error for that row. This patch takes the approach of doing the validation at the tablet layer rather than somewhere higher up the stack. The reasoning is that we don't want to reject an entire batch of operations due to one bad row, and we are not well equipped to set per-row errors anywhere except at the tablet layer. Because this is an incompatible change (it's possible that users have been successfully using larger cells) the limits are added as configurable flags, but tagged as unsafe. A new test verifies that, if the user has insertions in the WAL that were previously allowed but now are disallowed, the bootstrap fails without any crashes. It also tests that by raising the flag to a higher value, bootstraps will succeed. Making this test pass involved a small fix in bootstrap to properly handle the case where an Apply failed on restart the previously succeeded. Change-Id: Ib6b94cffd9c3efbe80a4b31e9272b376a4e41d15 Reviewed-on: http://gerrit.cloudera.org:8080/5475 Tested-by: Kudu Jenkins Reviewed-by: Adar Dembo <[email protected]> (cherry picked from commit 0259586af3611862645ffcbccca9e4f928e1095b) --- M docs/schema_design.adoc M src/kudu/client/client-test.cc M src/kudu/integration-tests/raft_consensus-itest.cc M src/kudu/integration-tests/test_workload.cc M src/kudu/integration-tests/ts_recovery-itest.cc M src/kudu/tablet/tablet.cc M src/kudu/tablet/tablet.h M src/kudu/tablet/tablet_bootstrap.cc M src/kudu/tserver/tablet_server-test.cc 9 files changed, 276 insertions(+), 41 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/13/5513/1 -- To view, visit http://gerrit.cloudera.org:8080/5513 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ib6b94cffd9c3efbe80a4b31e9272b376a4e41d15 Gerrit-PatchSet: 1 Gerrit-Project: kudu Gerrit-Branch: branch-1.2.x Gerrit-Owner: Todd Lipcon <[email protected]>
