SaketaChalamchala commented on code in PR #10581:
URL: https://github.com/apache/ozone/pull/10581#discussion_r3456128486
##########
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/snapshot/TestSnapshotDiffManager.java:
##########
@@ -1170,105 +1167,110 @@ private void uploadSnapshotDiffJobToDb(SnapshotInfo
fromSnapshot,
private static Stream<Arguments> threadPoolFullScenarios() {
return Stream.of(
- Arguments.of("When there is a wait time between job batches",
- 500L, 45, 0),
- Arguments.of("When there is no wait time between job batches",
- 0L, 20, 25)
+ Arguments.of("When the pool drains between job batches",
+ true, 45, 0),
+ // 10 running + 10 queued = 20 accepted, remaining 25 rejected
+ Arguments.of("When the pool does not drain between job batches",
+ false, 20, 25)
);
}
@ParameterizedTest(name = "{0}")
@MethodSource("threadPoolFullScenarios")
public void testThreadPoolIsFull(String description,
- long waitBetweenBatches,
+ boolean drainBetweenBatches,
int expectInProgressJobsCount,
int expectRejectedJobsCount)
throws Exception {
- ExecutorService executorService = new ThreadPoolExecutor(100, 100, 0,
- TimeUnit.MILLISECONDS, new SynchronousQueue<>()
- );
-
- List<SnapshotInfo> snapshotInfos = new ArrayList<>();
-
- for (int i = 0; i < 10; i++) {
- UUID snapshotId = UUID.randomUUID();
- String snapshotName = "snap-" + snapshotId;
- SnapshotInfo snapInfo = new SnapshotInfo.Builder()
- .setSnapshotId(snapshotId)
- .setVolumeName(VOLUME_NAME)
- .setBucketName(BUCKET_NAME)
- .setName(snapshotName)
- .setSnapshotPath("fromSnapshotPath")
- .build();
- snapshotInfos.add(snapInfo);
-
- when(snapshotInfoTable.get(getTableKey(VOLUME_NAME, BUCKET_NAME,
- snapshotName))).thenReturn(snapInfo);
- }
-
+ List<SnapshotInfo> snapshotInfos = createTestSnapshots(10);
SnapshotDiffManager spy = spy(snapshotDiffManager);
- for (int i = 0; i < snapshotInfos.size(); i++) {
- for (int j = i + 1; j < snapshotInfos.size(); j++) {
- String fromSnapshotName = snapshotInfos.get(i).getName();
- String toSnapshotName = snapshotInfos.get(j).getName();
+ CountDownLatch blockWorkers = new CountDownLatch(1);
+ AtomicInteger completedJobs = new AtomicInteger(0);
+ doAnswer(invocation -> {
+ blockWorkers.await();
+ completedJobs.incrementAndGet();
+ return null;
+ }).when(spy).generateSnapshotDiffReport(anyString(), anyString(),
+ eq(VOLUME_NAME), eq(BUCKET_NAME), anyString(), anyString(),
+ eq(false), eq(false));
- doAnswer(invocation -> {
- Thread.sleep(250L);
- return null;
- }).when(spy).generateSnapshotDiffReport(anyString(), anyString(),
- eq(VOLUME_NAME), eq(BUCKET_NAME), eq(fromSnapshotName),
- eq(toSnapshotName), eq(false), eq(false));
- }
+ if (drainBetweenBatches) {
+ blockWorkers.countDown();
}
- List<Future<SnapshotDiffResponse>> futures = new ArrayList<>();
- for (int i = 0; i < snapshotInfos.size(); i++) {
- for (int j = i + 1; j < snapshotInfos.size(); j++) {
- String fromSnapshotName = snapshotInfos.get(i).getName();
- String toSnapshotName = snapshotInfos.get(j).getName();
-
- Future<SnapshotDiffResponse> future = executorService.submit(
- () -> submitJob(spy, fromSnapshotName, toSnapshotName));
- futures.add(future);
+ try {
+ List<SnapshotDiffResponse> responses = new ArrayList<>();
+ int totalSubmitted = 0;
+ for (int i = 0; i < snapshotInfos.size(); i++) {
+ for (int j = i + 1; j < snapshotInfos.size(); j++) {
+ String fromSnapshotName = snapshotInfos.get(i).getName();
+ String toSnapshotName = snapshotInfos.get(j).getName();
+ responses.add(submitJob(spy, fromSnapshotName, toSnapshotName));
+ totalSubmitted++;
+ }
+ if (drainBetweenBatches) {
+ final int expected = totalSubmitted;
Review Comment:
Not in scope for the current PR to fix flakiness but may be considered as a
follow up test improvement:
In the `drainBetweenBatches` scenario would a better check be to
1. Keep the latch closed until `full_thread_pool_size` jobs are submitted
2. Open the latch
3. Submit new jobs when `totalSubmitted - completedJobs.get() <
full_thread_pool_size`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
