[
https://issues.apache.org/jira/browse/HDFS-16477?focusedWorklogId=748249&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-748249
]
ASF GitHub Bot logged work on HDFS-16477:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 26/Mar/22 18:55
Start Date: 26/Mar/22 18:55
Worklog Time Spent: 10m
Work Description: ayushtkn commented on a change in pull request #4009:
URL: https://github.com/apache/hadoop/pull/4009#discussion_r835797579
##########
File path:
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/sps/TestExternalStoragePolicySatisfier.java
##########
@@ -441,6 +442,8 @@ private void doTestWhenStoragePolicySetToCOLD() throws
Exception {
hdfsCluster.triggerHeartbeats();
dfs.satisfyStoragePolicy(new Path(FILE));
+ // Assert metrics.
+ assertEquals(1, hdfsCluster.getNamesystem().getPendingSPSPaths());
// Wait till namenode notified about the block location details
DFSTestUtil.waitExpectedStorageType(FILE, StorageType.ARCHIVE, 3, 35000,
dfs);
Review comment:
```
// Wait till namenode notified about the block location details
DFSTestUtil.waitExpectedStorageType(FILE, StorageType.ARCHIVE, 3, 35000,
dfs);
```
Here you are waiting for SPS to process the path and move the blocks to the
correct place, once this is done, whether ``getPendingSPSPaths`` will still
return 1? I suppose no, right? the path got processed so the count should
reduce to 0.
So, my take is you don't have a control on ``
DFSTestUtil.waitExpectedStorageType(FILE, StorageType.ARCHIVE, 3, 35000,
dfs);``, if by chance SPS process that path before your assertion
then the test will fail.
I haven't gone through the code, but that is what I felt in my initial
pass, if it doesn't work this way do lemme know
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 748249)
Time Spent: 4h (was: 3h 50m)
> [SPS]: Add metric PendingSPSPaths for getting the number of paths to be
> processed by SPS
> ----------------------------------------------------------------------------------------
>
> Key: HDFS-16477
> URL: https://issues.apache.org/jira/browse/HDFS-16477
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: tomscut
> Assignee: tomscut
> Priority: Major
> Labels: pull-request-available
> Time Spent: 4h
> Remaining Estimate: 0h
>
> Currently we have no idea how many paths are waiting to be processed when
> using the SPS feature. We should add metric PendingSPSPaths for getting the
> number of paths to be processed by SPS in NameNode.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
