This is an automated email from the ASF dual-hosted git repository.
mgreber pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/kudu.git
commit 0804a95555b8bf5a1f685ce6601acb36dbffe43e
Author: Ashwani Raina <ara...@cloudera.com>
AuthorDate: Fri Apr 26 18:07:33 2024 +0530
KUDU-3568 Fix budgeting constraint test by enabling preset factor
In the failing test TestRowSetCompactionSkipWithBudgetingConstraints
the rowset compaction went ahead even with high memory budget
requirement (at least as per test expectations). The test bases
it memory requirements on number of factor, one of which is
rowset_compaction_delta_memory_factor.
It is set to a very high value so that node on which test is
running is not able to satisfy the memory requirements and
hence skipping of compaction becomes necessary.
Failure can happen due to a bug in the test where we are using
rowset_compaction_delta_memory_factor without setting
rowset_compaction_enforce_preset_factor to true.
Since rowset_compaction_enforce_preset_factor is set to false
by default, which means compaction policy relies on metrics
(instead of memory factor) to come up with memory budgeting numbers.
The test passes in most runs, because the metrics,
(i.e. compact_rs_mem_usage_to_deltas_size_ratio) that compaction
policy relies on, happens to be 0. Due to this, memory budgeting
logic chooses the rowset_compaction_delta_memory_factor
(set to a very high value in this test) as the factor and since the
node doesn't have so much memory, it skips compaction as expected.
In a nutshell, rowset_compaction_delta_memory_factor is used when
when rowset_compaction_enforce_preset_factor is true.
If preset is set to false, test will rely on metrics instead of memory
factor defined by rowset_compaction_delta_memory_factor.
Change-Id: I9df218cd2d8ef3709793db267d5a0d651421dbb6
Reviewed-on: http://gerrit.cloudera.org:8080/21360
Reviewed-by: Attila Bukor <abu...@apache.org>
Reviewed-by: Zoltan Martonka <zmarto...@cloudera.com>
Tested-by: Marton Greber <greber...@gmail.com>
---
src/kudu/tablet/compaction-test.cc | 2 ++
1 file changed, 2 insertions(+)
diff --git a/src/kudu/tablet/compaction-test.cc
b/src/kudu/tablet/compaction-test.cc
index 6374bae64..a0d2c35d5 100644
--- a/src/kudu/tablet/compaction-test.cc
+++ b/src/kudu/tablet/compaction-test.cc
@@ -95,6 +95,7 @@ DEFINE_uint32(merge_benchmark_num_rowsets, 3,
DEFINE_uint32(merge_benchmark_num_rows_per_rowset, 500000,
"Number of rowsets as input to the merge");
+DECLARE_bool(rowset_compaction_enforce_preset_factor);
DECLARE_bool(rowset_compaction_memory_estimate_enabled);
DECLARE_bool(rowset_compaction_ancient_delta_threshold_enabled);
DECLARE_double(rowset_compaction_delta_memory_factor);
@@ -880,6 +881,7 @@ void TestCompaction::AddUpdateAndDelete(
void TestCompaction::TestRowSetCompactionWithOrWithoutBudgetingConstraints(
bool budgeting_constraints_applied) {
FLAGS_rowset_compaction_memory_estimate_enabled = true;
+ FLAGS_rowset_compaction_enforce_preset_factor = true;
FLAGS_rowset_compaction_ancient_delta_threshold_enabled = true;
// Ensure memory budgeting applies
