rich7420 commented on code in PR #11843:
URL: https://github.com/apache/ozone/pull/11843#discussion_r4548882117


##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/SCMClientProtocolServer.java:
##########
@@ -1772,6 +1774,49 @@ public List<Long> suppressContainers(List<Long> 
containerIds, boolean suppress)
     return failedContainerIDs;
   }
 
+  @Override
+  public String submitContainerIdExport(ContainerID startContainerId, 
HddsProtos.LifeCycleState lifeCycleState,
+      ContainerHealthState healthState, long maxRows, int batchSize, int 
partSize) throws IOException {
+    getScm().checkAdminAccess(getRemoteUser(), false);

Review Comment:
   No test goes through the RPC layer yet, so flipping this `false` to `true`, 
or dropping the not-found `NonRetriableException` below, would go unnoticed. 
Could you add an ACL test and a translator round trip, like the 
`GetExistingContainersInBatch` ones in `TestSCMClientProtocolServer` and 
`TestStorageContainerLocationProtocolServerSideTranslatorPB`?



##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/export/ContainerExportManager.java:
##########
@@ -153,7 +155,7 @@ public ExportJob submitJob(ContainerID start, 
LifeCycleState lifeCycleState, Con
     final ExportScope scope = ExportScope.of(lifeCycleState, healthState);
     final ExportJob job = new ExportJob(scope, start, sizing);
     if (!runningJob.compareAndSet(null, job)) {
-      throw new IOException("Another container ID export is already running");
+      throw new NonRetriableException("Another container ID export is already 
running");

Review Comment:
   Not a blocker: could the message include the running job's id? There's no 
list RPC, so whoever hits this can't look up that job's status. Note 
`runningJob.get()` may already be null here.



##########
hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/container/export/TestContainerExportManager.java:
##########
@@ -475,6 +478,106 @@ public void 
testCompletedArchivesTrimmedAndOthersKeptOnRestart() throws Exceptio
     assertEquals(1, Stream.of(first, second).filter(job -> 
archiveFile(job).exists()).count());
   }
 
+  @Test
+  public void testContainerExportStatusOfSucceededJob() throws Exception {
+    final ExportJob job = exportOneContainer(ContainerHealthState.MISSING, 1);
+
+    final GetContainerIdExportStatusResponseProto status = 
statusOf(job.getId());
+    assertEquals(ExecutionState.SUCCEEDED, status.getState());
+    assertTrue(isDone(status));
+    assertEquals(job.getId().toString(), status.getJobId());
+    assertEquals(ContainerHealthState.MISSING.name(), status.getHealthState());
+    assertFalse(status.hasLifecycleState());
+    assertEquals(1, status.getIdsWritten());
+    // The sizing the job runs with, defaults applied.
+    assertEquals(0, status.getMaxRows());
+    assertEquals(TEST_BATCH_SIZE, status.getBatchSize());
+    assertEquals(TEST_PART_SIZE, status.getPartSize());
+    assertEquals(archiveFile(job).getAbsolutePath(), status.getTarPath());
+    assertFalse(status.hasErrorMessage());
+    // Elapsed time stops once the job is done.
+    final long elapsed = status.getElapsedMs();
+    Thread.sleep(50);
+    assertEquals(elapsed, statusOf(job.getId()).getElapsedMs());
+  }
+
+  @Test
+  public void testContainerExportStatusOfJobWithoutMatchesHasNoTarPath() 
throws Exception {
+    when(containerManager.getContainerIDs(
+        eq(ContainerID.valueOf(0)), anyInt(), eq(LifeCycleState.OPEN), 
isNull()))
+        .thenReturn(Collections.emptyList());
+    final ExportJob job = submit(ContainerID.valueOf(0), LifeCycleState.OPEN, 
null);
+    waitForDone(job.getId());
+
+    final GetContainerIdExportStatusResponseProto status = 
statusOf(job.getId());
+    assertEquals(ExecutionState.SUCCEEDED, status.getState());
+    assertEquals(LifeCycleState.OPEN, status.getLifecycleState());
+    assertFalse(status.hasHealthState());
+    assertEquals(0, status.getIdsWritten());
+    assertFalse(status.hasTarPath());
+  }
+
+  @Test
+  public void testContainerExportStatusOfFailedJob() throws Exception {
+    when(containerManager.getContainerIDs(
+        eq(ContainerID.valueOf(0)), anyInt(), isNull(), 
eq(ContainerHealthState.MISSING)))
+        .thenThrow(new RuntimeException("container listing failed"));
+    final ExportJob job = submit(ContainerID.valueOf(0), null, 
ContainerHealthState.MISSING);
+    waitForDone(job.getId());
+
+    final GetContainerIdExportStatusResponseProto status = 
statusOf(job.getId());
+    assertEquals(ExecutionState.FAILED, status.getState());
+    assertTrue(isDone(status));
+    assertTrue(status.getErrorMessage().contains("container listing failed"));
+    assertEquals(0, status.getIdsWritten());
+    assertFalse(status.hasTarPath());
+  }
+
+  @Test
+  public void 
testContainerExportStatusOfFailureWithoutMessageNamesTheException() throws 
Exception {
+    when(containerManager.getContainerIDs(
+        eq(ContainerID.valueOf(0)), anyInt(), isNull(), 
eq(ContainerHealthState.MISSING)))
+        .thenThrow(new IllegalStateException());
+    final ExportJob job = submit(ContainerID.valueOf(0), null, 
ContainerHealthState.MISSING);
+    waitForDone(job.getId());
+
+    final GetContainerIdExportStatusResponseProto status = 
statusOf(job.getId());
+    assertEquals(ExecutionState.FAILED, status.getState());
+    assertTrue(status.getErrorMessage().contains("IllegalStateException"), 
status.getErrorMessage());
+  }
+
+  @Test
+  public void testContainerExportStatusOfRunningJob() throws Exception {
+    final CountDownLatch listing = new CountDownLatch(1);
+    final CountDownLatch release = new CountDownLatch(1);
+    when(containerManager.getContainerIDs(
+        eq(ContainerID.valueOf(0)), anyInt(), isNull(), 
eq(ContainerHealthState.MISSING)))
+        .thenAnswer(invocation -> {
+          listing.countDown();
+          release.await();
+          return Collections.emptyList();
+        });
+    final ExportJob job = exportManager.submitJob(ContainerID.valueOf(0), 
null, ContainerHealthState.MISSING, 5, 1, 2);
+    assertTrue(listing.await(30, TimeUnit.SECONDS));
+
+    final GetContainerIdExportStatusResponseProto status = 
statusOf(job.getId());
+    assertEquals(ExecutionState.RUNNING, status.getState());
+    assertFalse(isDone(status));
+    assertEquals(5, status.getMaxRows());
+    assertEquals(1, status.getBatchSize());
+    assertEquals(2, status.getPartSize());
+    assertFalse(status.hasTarPath());
+    assertFalse(status.hasErrorMessage());
+
+    release.countDown();
+    waitForDone(job.getId());
+  }
+
+  @Test
+  public void testContainerExportStatusOfUnknownJobIsNull() {

Review Comment:
   Nit: this is the same as `testGetExportStatusUnknownJobReturnsNull`, since 
`statusOf()` just calls `getContainerExportStatus(id.toString())`. Could we 
keep only one?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to