[jira] [Work started] (HDFS-16429) Add DataSetLockManager to maintain locks for FsDataSetImpl
[ https://issues.apache.org/jira/browse/HDFS-16429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-16429 started by Mingxiang Li. --- > Add DataSetLockManager to maintain locks for FsDataSetImpl > -- > > Key: HDFS-16429 > URL: https://issues.apache.org/jira/browse/HDFS-16429 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: hdfs >Affects Versions: 3.2.0 >Reporter: Mingxiang Li >Assignee: Mingxiang Li >Priority: Major > Labels: pull-request-available > Fix For: 3.2.0 > > Time Spent: 10m > Remaining Estimate: 0h > > 1、Use lockManager to maintain two level lock for FsDataSetImpl. > The simple lock model like this.Parts of implemented as follows > * As for finalizeReplica(),append(),createRbw()First get BlockPoolLock > read lock，and then get BlockPoolLock-volume-lock write lock. > * As for getStoredBlock(),getMetaDataInputStream()First get > BlockPoolLock read lock，and the then get BlockPoolLock-volume-lock read lock. > * As for deepCopyReplica(),getBlockReports() get the BlockPoolLock read lock. > * As for delete hold the BlockPoolLock write lock. > 2、Make LightWeightResizableGSet become thread safe.It not become performance > bottleneck if we make it thread safe.We can reduce lock grain size for > ReplicaMap when make LightWeightResizableGSet thread safe. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15180) DataNode FsDatasetImpl Fine-Grained Locking via BlockPool.
[ https://issues.apache.org/jira/browse/HDFS-15180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingxiang Li updated HDFS-15180: Parent: HDFS-15382 Issue Type: Sub-task (was: Improvement) > DataNode FsDatasetImpl Fine-Grained Locking via BlockPool. > --- > > Key: HDFS-15180 > URL: https://issues.apache.org/jira/browse/HDFS-15180 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: datanode >Affects Versions: 3.2.0 >Reporter: Qi Zhu >Assignee: Mingxiang Li >Priority: Major > Labels: pull-request-available > Attachments: HDFS-15180.001.patch, HDFS-15180.002.patch, > HDFS-15180.003.patch, HDFS-15180.004.patch, > image-2020-03-10-17-22-57-391.png, image-2020-03-10-17-31-58-830.png, > image-2020-03-10-17-34-26-368.png, image-2020-04-09-11-20-36-459.png > > Time Spent: 40m > Remaining Estimate: 0h > > Now the FsDatasetImpl datasetLock is heavy, when their are many namespaces in > big cluster. If we can split the FsDatasetImpl datasetLock via blockpool. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-16429) Add DataSetLockManager to maintain locks for FsDataSetImpl
[ https://issues.apache.org/jira/browse/HDFS-16429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingxiang Li updated HDFS-16429: Parent: HDFS-15382 Issue Type: Sub-task (was: Improvement) > Add DataSetLockManager to maintain locks for FsDataSetImpl > -- > > Key: HDFS-16429 > URL: https://issues.apache.org/jira/browse/HDFS-16429 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: hdfs >Affects Versions: 3.2.0 >Reporter: Mingxiang Li >Assignee: Mingxiang Li >Priority: Major > Labels: pull-request-available > Fix For: 3.2.0 > > Time Spent: 10m > Remaining Estimate: 0h > > 1、Use lockManager to maintain two level lock for FsDataSetImpl. > The simple lock model like this.Parts of implemented as follows > * As for finalizeReplica(),append(),createRbw()First get BlockPoolLock > read lock，and then get BlockPoolLock-volume-lock write lock. > * As for getStoredBlock(),getMetaDataInputStream()First get > BlockPoolLock read lock，and the then get BlockPoolLock-volume-lock read lock. > * As for deepCopyReplica(),getBlockReports() get the BlockPoolLock read lock. > * As for delete hold the BlockPoolLock write lock. > 2、Make LightWeightResizableGSet become thread safe.It not become performance > bottleneck if we make it thread safe.We can reduce lock grain size for > ReplicaMap when make LightWeightResizableGSet thread safe. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-15180) DataNode FsDatasetImpl Fine-Grained Locking via BlockPool.
[ https://issues.apache.org/jira/browse/HDFS-15180?focusedWorklogId=710305&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-710305 ] ASF GitHub Bot logged work on HDFS-15180: - Author: ASF GitHub Bot Created on: 18/Jan/22 07:49 Start Date: 18/Jan/22 07:49 Worklog Time Spent: 10m Work Description: MingXiangLi commented on pull request #3889: URL: https://github.com/apache/hadoop/pull/3889#issuecomment-1015149020 > 感谢@MingXiangLi的作品。现在是一个很大的进步。 快速浏览一下，我认为最好将其拆分为不同的独立JIRA和PR，并且合并更顺畅。 cc @jojochuang @sodonnel @goiri @ayushtkn你介意看看并提供一些其他建议吗？谢谢。 Good advice. Add sub task to this pr. https://github.com/apache/hadoop/pull/3900 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 710305) Time Spent: 40m (was: 0.5h) > DataNode FsDatasetImpl Fine-Grained Locking via BlockPool. > --- > > Key: HDFS-15180 > URL: https://issues.apache.org/jira/browse/HDFS-15180 > Project: Hadoop HDFS > Issue Type: Improvement > Components: datanode >Affects Versions: 3.2.0 >Reporter: Qi Zhu >Assignee: Mingxiang Li >Priority: Major > Labels: pull-request-available > Attachments: HDFS-15180.001.patch, HDFS-15180.002.patch, > HDFS-15180.003.patch, HDFS-15180.004.patch, > image-2020-03-10-17-22-57-391.png, image-2020-03-10-17-31-58-830.png, > image-2020-03-10-17-34-26-368.png, image-2020-04-09-11-20-36-459.png > > Time Spent: 40m > Remaining Estimate: 0h > > Now the FsDatasetImpl datasetLock is heavy, when their are many namespaces in > big cluster. If we can split the FsDatasetImpl datasetLock via blockpool. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16429) Add DataSetLockManager to maintain locks for FsDataSetImpl
[ https://issues.apache.org/jira/browse/HDFS-16429?focusedWorklogId=710304&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-710304 ] ASF GitHub Bot logged work on HDFS-16429: - Author: ASF GitHub Bot Created on: 18/Jan/22 07:47 Start Date: 18/Jan/22 07:47 Worklog Time Spent: 10m Work Description: MingXiangLi opened a new pull request #3900: URL: https://github.com/apache/hadoop/pull/3900 this pr is sub task of the https://github.com/apache/hadoop/pull/3889 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 710304) Remaining Estimate: 0h Time Spent: 10m > Add DataSetLockManager to maintain locks for FsDataSetImpl > -- > > Key: HDFS-16429 > URL: https://issues.apache.org/jira/browse/HDFS-16429 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs >Affects Versions: 3.2.0 >Reporter: Mingxiang Li >Assignee: Mingxiang Li >Priority: Major > Fix For: 3.2.0 > > Time Spent: 10m > Remaining Estimate: 0h > > 1、Use lockManager to maintain two level lock for FsDataSetImpl. > The simple lock model like this.Parts of implemented as follows > * As for finalizeReplica(),append(),createRbw()First get BlockPoolLock > read lock，and then get BlockPoolLock-volume-lock write lock. > * As for getStoredBlock(),getMetaDataInputStream()First get > BlockPoolLock read lock，and the then get BlockPoolLock-volume-lock read lock. > * As for deepCopyReplica(),getBlockReports() get the BlockPoolLock read lock. > * As for delete hold the BlockPoolLock write lock. > 2、Make LightWeightResizableGSet become thread safe.It not become performance > bottleneck if we make it thread safe.We can reduce lock grain size for > ReplicaMap when make LightWeightResizableGSet thread safe. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-16429) Add DataSetLockManager to maintain locks for FsDataSetImpl
[ https://issues.apache.org/jira/browse/HDFS-16429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDFS-16429: -- Labels: pull-request-available (was: ) > Add DataSetLockManager to maintain locks for FsDataSetImpl > -- > > Key: HDFS-16429 > URL: https://issues.apache.org/jira/browse/HDFS-16429 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs >Affects Versions: 3.2.0 >Reporter: Mingxiang Li >Assignee: Mingxiang Li >Priority: Major > Labels: pull-request-available > Fix For: 3.2.0 > > Time Spent: 10m > Remaining Estimate: 0h > > 1、Use lockManager to maintain two level lock for FsDataSetImpl. > The simple lock model like this.Parts of implemented as follows > * As for finalizeReplica(),append(),createRbw()First get BlockPoolLock > read lock，and then get BlockPoolLock-volume-lock write lock. > * As for getStoredBlock(),getMetaDataInputStream()First get > BlockPoolLock read lock，and the then get BlockPoolLock-volume-lock read lock. > * As for deepCopyReplica(),getBlockReports() get the BlockPoolLock read lock. > * As for delete hold the BlockPoolLock write lock. > 2、Make LightWeightResizableGSet become thread safe.It not become performance > bottleneck if we make it thread safe.We can reduce lock grain size for > ReplicaMap when make LightWeightResizableGSet thread safe. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16430) Validate maximum blocks in EC group when adding an EC policy
[ https://issues.apache.org/jira/browse/HDFS-16430?focusedWorklogId=710303&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-710303 ] ASF GitHub Bot logged work on HDFS-16430: - Author: ASF GitHub Bot Created on: 18/Jan/22 07:35 Start Date: 18/Jan/22 07:35 Worklog Time Spent: 10m Work Description: cndaimin opened a new pull request #3899: URL: https://github.com/apache/hadoop/pull/3899 HDFS EC adopts the last 4 bits of block ID to store the block index in EC block group. Therefore maximum blocks in EC block group is `2^4=16`, and which is defined here: `HdfsServerConstants#MAX_BLOCKS_IN_GROUP`. Currently there is no limitation or warning when adding a bad EC policy with `numDataUnits + numParityUnits > 16`. It only results in read/write error on EC file with bad EC policy. To users this is not very straightforward. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 710303) Remaining Estimate: 0h Time Spent: 10m > Validate maximum blocks in EC group when adding an EC policy > > > Key: HDFS-16430 > URL: https://issues.apache.org/jira/browse/HDFS-16430 > Project: Hadoop HDFS > Issue Type: Improvement > Components: ec, erasure-coding >Affects Versions: 3.3.0, 3.3.1 >Reporter: daimin >Assignee: daimin >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > HDFS EC adopts the last 4 bits of block ID to store the block index in EC > block group. Therefore maximum blocks in EC block group is 2^4=16, and which > is defined here: HdfsServerConstants#MAX_BLOCKS_IN_GROUP. > Currently there is no limitation or warning when adding a bad EC policy with > numDataUnits + numParityUnits > 16. It only results in read/write error on EC > file with bad EC policy. To users this is not very straightforward. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-16430) Validate maximum blocks in EC group when adding an EC policy
[ https://issues.apache.org/jira/browse/HDFS-16430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDFS-16430: -- Labels: pull-request-available (was: ) > Validate maximum blocks in EC group when adding an EC policy > > > Key: HDFS-16430 > URL: https://issues.apache.org/jira/browse/HDFS-16430 > Project: Hadoop HDFS > Issue Type: Improvement > Components: ec, erasure-coding >Affects Versions: 3.3.0, 3.3.1 >Reporter: daimin >Assignee: daimin >Priority: Minor > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > HDFS EC adopts the last 4 bits of block ID to store the block index in EC > block group. Therefore maximum blocks in EC block group is 2^4=16, and which > is defined here: HdfsServerConstants#MAX_BLOCKS_IN_GROUP. > Currently there is no limitation or warning when adding a bad EC policy with > numDataUnits + numParityUnits > 16. It only results in read/write error on EC > file with bad EC policy. To users this is not very straightforward. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-16429) Add DataSetLockManager to maintain locks for FsDataSetImpl
Mingxiang Li created HDFS-16429: --- Summary: Add DataSetLockManager to maintain locks for FsDataSetImpl Key: HDFS-16429 URL: https://issues.apache.org/jira/browse/HDFS-16429 Project: Hadoop HDFS Issue Type: Improvement Components: hdfs Affects Versions: 3.2.0 Reporter: Mingxiang Li Assignee: Mingxiang Li Fix For: 3.2.0 1、Use lockManager to maintain two level lock for FsDataSetImpl. The simple lock model like this.Parts of implemented as follows * As for finalizeReplica(),append(),createRbw()First get BlockPoolLock read lock，and then get BlockPoolLock-volume-lock write lock. * As for getStoredBlock(),getMetaDataInputStream()First get BlockPoolLock read lock，and the then get BlockPoolLock-volume-lock read lock. * As for deepCopyReplica(),getBlockReports() get the BlockPoolLock read lock. * As for delete hold the BlockPoolLock write lock. 2、Make LightWeightResizableGSet become thread safe.It not become performance bottleneck if we make it thread safe.We can reduce lock grain size for ReplicaMap when make LightWeightResizableGSet thread safe. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-16430) Validate maximum blocks in EC group when adding an EC policy
daimin created HDFS-16430: - Summary: Validate maximum blocks in EC group when adding an EC policy Key: HDFS-16430 URL: https://issues.apache.org/jira/browse/HDFS-16430 Project: Hadoop HDFS Issue Type: Improvement Components: ec, erasure-coding Affects Versions: 3.3.1, 3.3.0 Reporter: daimin Assignee: daimin HDFS EC adopts the last 4 bits of block ID to store the block index in EC block group. Therefore maximum blocks in EC block group is 2^4=16, and which is defined here: HdfsServerConstants#MAX_BLOCKS_IN_GROUP. Currently there is no limitation or warning when adding a bad EC policy with numDataUnits + numParityUnits > 16. It only results in read/write error on EC file with bad EC policy. To users this is not very straightforward. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16428) Source path setted storagePolicy will cause wrong typeConsumed in rename operation
[ https://issues.apache.org/jira/browse/HDFS-16428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17477588#comment-17477588 ] lei w commented on HDFS-16428: -- [~LiJinglun] [~ayushtkn] [~xiaoyuyao] Looking forward to your comments！ > Source path setted storagePolicy will cause wrong typeConsumed in rename > operation > --- > > Key: HDFS-16428 > URL: https://issues.apache.org/jira/browse/HDFS-16428 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs, namenode >Reporter: lei w >Priority: Major > Attachments: example.txt > > > When compute quota in rename operation , we use storage policy of the target > directory to compute src quota usage. This will cause wrong value of > typeConsumed when source path was setted storage policy. I provided a unit > test to present this situation. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-16428) Source path setted storagePolicy will cause wrong typeConsumed in rename operation
[ https://issues.apache.org/jira/browse/HDFS-16428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lei w updated HDFS-16428: - Summary: Source path setted storagePolicy will cause wrong typeConsumed in rename operation (was: When source path setted storagePolicy in rename operation will cause wrong typeConsumed ) > Source path setted storagePolicy will cause wrong typeConsumed in rename > operation > --- > > Key: HDFS-16428 > URL: https://issues.apache.org/jira/browse/HDFS-16428 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs, namenode >Reporter: lei w >Priority: Major > Attachments: example.txt > > > When compute quota in rename operation , we use storage policy of the target > directory to compute src quota usage. This will cause wrong value of > typeConsumed when source path was setted storage policy. I provided a unit > test to present this situation. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16424) Add a new rpcMetric called requeueTime
[
https://issues.apache.org/jira/browse/HDFS-16424?focusedWorklogId=710212&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-710212
]
ASF GitHub Bot logged work on HDFS-16424:
-
Author: ASF GitHub Bot
Created on: 18/Jan/22 01:56
Start Date: 18/Jan/22 01:56
Worklog Time Spent: 10m
Work Description: hfutatzhanghb commented on a change in pull request
#3884:
URL: https://github.com/apache/hadoop/pull/3884#discussion_r786365987
##
File path:
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java
##
@@ -3158,6 +3161,9 @@ private void requeueCall(Call call)
throws IOException, InterruptedException {
try {
internalQueueCall(call, false);
+long deltaNanos = Time.monotonicNowNanos() - call.timestampNanos;
Review comment:
Hi, @tasanuma , thanks a lot for reviewing. yes, it is the same time
when calculated in internalQueueCall(call, false).
But in fact， this time is REQUEUE time , original implementation does not
record this time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 710212)
Time Spent: 1h 10m (was: 1h)
> Add a new rpcMetric called requeueTime
> --
>
> Key: HDFS-16424
> URL: https://issues.apache.org/jira/browse/HDFS-16424
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
>Reporter: zhanghaobo
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> When observer namenode read enabled, if `call.getClientStateId() >
> alignmentContext.getLastSeenStateId()`, the rpc call will be requeued to
> callQueue.
> we need to know rpc requeue metrics to monitor requeue call and its
> performance.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16424) Add a new rpcMetric called requeueTime
[
https://issues.apache.org/jira/browse/HDFS-16424?focusedWorklogId=710211&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-710211
]
ASF GitHub Bot logged work on HDFS-16424:
-
Author: ASF GitHub Bot
Created on: 18/Jan/22 01:48
Start Date: 18/Jan/22 01:48
Worklog Time Spent: 10m
Work Description: tasanuma commented on a change in pull request #3884:
URL: https://github.com/apache/hadoop/pull/3884#discussion_r786363769
##
File path:
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java
##
@@ -3158,6 +3161,9 @@ private void requeueCall(Call call)
throws IOException, InterruptedException {
try {
internalQueueCall(call, false);
+long deltaNanos = Time.monotonicNowNanos() - call.timestampNanos;
Review comment:
Isn't it the same time as ENQUEUE calculated in `internalQueueCall(call,
false)` of the before line?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 710211)
Time Spent: 1h (was: 50m)
> Add a new rpcMetric called requeueTime
> --
>
> Key: HDFS-16424
> URL: https://issues.apache.org/jira/browse/HDFS-16424
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
>Reporter: zhanghaobo
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
> When observer namenode read enabled, if `call.getClientStateId() >
> alignmentContext.getLastSeenStateId()`, the rpc call will be requeued to
> callQueue.
> we need to know rpc requeue metrics to monitor requeue call and its
> performance.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16426) fix nextBlockReportTime when trigger full block report force
[
https://issues.apache.org/jira/browse/HDFS-16426?focusedWorklogId=710208&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-710208
]
ASF GitHub Bot logged work on HDFS-16426:
-
Author: ASF GitHub Bot
Created on: 18/Jan/22 01:41
Start Date: 18/Jan/22 01:41
Worklog Time Spent: 10m
Work Description: tasanuma commented on a change in pull request #3887:
URL: https://github.com/apache/hadoop/pull/3887#discussion_r786361910
##
File path:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BPServiceActor.java
##
@@ -1304,12 +1305,18 @@ long scheduleBlockReport(long delay, boolean
isRegistration) {
* Numerical overflow is possible here.
*/
void scheduleNextBlockReport() {
+ scheduleNextBlockReport(false);
+}
+
+void scheduleNextBlockReport(boolean forceFullBr) {
// If we have sent the first set of block reports, then wait a random
// time before we start the periodic block reports.
if (resetBlockReportTime) {
nextBlockReportTime.getAndSet(monotonicNow() +
ThreadLocalRandom.current().nextInt((int)
(blockReportIntervalMs)));
resetBlockReportTime = false;
+ } else if (forceFullBr) {
+nextBlockReportTime.getAndSet(monotonicNow() + blockReportIntervalMs);
Review comment:
@liubingxing @tomscut
Thanks for the discussion. IMHO, I prefer to use a random time to keep the
randomness.
I think HDFS-15167 causes this problem. Before HDFS-15167,
`resetBlockReportTime` is true when triggering full block report by force, and
it used a random time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 710208)
Time Spent: 1h 20m (was: 1h 10m)
> fix nextBlockReportTime when trigger full block report force
>
>
> Key: HDFS-16426
> URL: https://issues.apache.org/jira/browse/HDFS-16426
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: qinyuren
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> When we trigger full block report force by command line, the next block
> report time will be set like this:
> nextBlockReportTime.getAndAdd(blockReportIntervalMs);
> nextBlockReportTime will larger than blockReportIntervalMs.
> If we trigger full block report twice, the nextBlockReportTime will larger
> than 2 * blockReportIntervalMs. This is obviously not what we want.
> We fix the nextBlockReportTime = now + blockReportIntervalMs after full block
> report trigger by command line.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16423) balancer should not get blocks on stale storages
[ https://issues.apache.org/jira/browse/HDFS-16423?focusedWorklogId=710205&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-710205 ] ASF GitHub Bot logged work on HDFS-16423: - Author: ASF GitHub Bot Created on: 18/Jan/22 01:27 Start Date: 18/Jan/22 01:27 Worklog Time Spent: 10m Work Description: tasanuma commented on pull request #3883: URL: https://github.com/apache/hadoop/pull/3883#issuecomment-1014992750 @liubingxing Thanks for the PR. It is a kind of bug, and I don't think we need to make it configurable. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 710205) Time Spent: 1h 50m (was: 1h 40m) > balancer should not get blocks on stale storages > > > Key: HDFS-16423 > URL: https://issues.apache.org/jira/browse/HDFS-16423 > Project: Hadoop HDFS > Issue Type: Improvement > Components: balancer & mover >Reporter: qinyuren >Assignee: qinyuren >Priority: Major > Labels: pull-request-available > Attachments: image-2022-01-13-17-18-32-409.png > > Time Spent: 1h 50m > Remaining Estimate: 0h > > We have met a problems as described in HDFS-16420 > We found that balancer copied a block multi times without deleting the source > block if this block was placed in a stale storage. And resulting a block with > many copies, but these redundant copies are not deleted until the storage > become not stale. > > !image-2022-01-13-17-18-32-409.png|width=657,height=275! -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16399) Reconfig cache report parameters for datanode
[ https://issues.apache.org/jira/browse/HDFS-16399?focusedWorklogId=710192&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-710192 ] ASF GitHub Bot logged work on HDFS-16399: - Author: ASF GitHub Bot Created on: 17/Jan/22 23:58 Start Date: 17/Jan/22 23:58 Worklog Time Spent: 10m Work Description: tomscut commented on pull request #3841: URL: https://github.com/apache/hadoop/pull/3841#issuecomment-1014961403 Hi @tasanuma @Hexiaoqiao @virajjasani , could you please take a look at this. Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 710192) Time Spent: 1h (was: 50m) > Reconfig cache report parameters for datanode > - > > Key: HDFS-16399 > URL: https://issues.apache.org/jira/browse/HDFS-16399 > Project: Hadoop HDFS > Issue Type: New Feature >Reporter: tomscut >Assignee: tomscut >Priority: Major > Labels: pull-request-available > Time Spent: 1h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16399) Reconfig cache report parameters for datanode
[ https://issues.apache.org/jira/browse/HDFS-16399?focusedWorklogId=710191&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-710191 ] ASF GitHub Bot logged work on HDFS-16399: - Author: ASF GitHub Bot Created on: 17/Jan/22 23:58 Start Date: 17/Jan/22 23:58 Worklog Time Spent: 10m Work Description: tomscut removed a comment on pull request #3841: URL: https://github.com/apache/hadoop/pull/3841#issuecomment-1013603684 Hi @tasanuma @Hexiaoqiao @virajjasani , could you please take a look at this. Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 710191) Time Spent: 50m (was: 40m) > Reconfig cache report parameters for datanode > - > > Key: HDFS-16399 > URL: https://issues.apache.org/jira/browse/HDFS-16399 > Project: Hadoop HDFS > Issue Type: New Feature >Reporter: tomscut >Assignee: tomscut >Priority: Major > Labels: pull-request-available > Time Spent: 50m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16402) HeartbeatManager may cause incorrect stats
[
https://issues.apache.org/jira/browse/HDFS-16402?focusedWorklogId=710188&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-710188
]
ASF GitHub Bot logged work on HDFS-16402:
-
Author: ASF GitHub Bot
Created on: 17/Jan/22 23:56
Start Date: 17/Jan/22 23:56
Worklog Time Spent: 10m
Work Description: tomscut commented on pull request #3839:
URL: https://github.com/apache/hadoop/pull/3839#issuecomment-1014960835
Hi @Hexiaoqiao , I added an unit test for this change. Please take a look.
Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 710188)
Time Spent: 2h (was: 1h 50m)
> HeartbeatManager may cause incorrect stats
> --
>
> Key: HDFS-16402
> URL: https://issues.apache.org/jira/browse/HDFS-16402
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: tomscut
>Assignee: tomscut
>Priority: Major
> Labels: pull-request-available
> Attachments: image-2021-12-29-08-25-44-303.png,
> image-2021-12-29-08-25-54-441.png
>
> Time Spent: 2h
> Remaining Estimate: 0h
>
> After reconfig {*}dfs.datanode.data.dir{*}, we found that the stats of the
> Namenode Web became *negative* and there were many NPE in namenode logs. This
> problem has been solved by HDFS-14042.
> !image-2021-12-29-08-25-54-441.png|width=681,height=293!
> !image-2021-12-29-08-25-44-303.png|width=677,height=180!
> However, if *HeartbeatManager#updateHeartbeat* and
> *HeartbeatManager#updateLifeline* throw other exceptions, stats errors can
> also occur. We should ensure that *stats.subtract()* and *stats.add()* are
> transactional.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16402) HeartbeatManager may cause incorrect stats
[
https://issues.apache.org/jira/browse/HDFS-16402?focusedWorklogId=710190&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-710190
]
ASF GitHub Bot logged work on HDFS-16402:
-
Author: ASF GitHub Bot
Created on: 17/Jan/22 23:56
Start Date: 17/Jan/22 23:56
Worklog Time Spent: 10m
Work Description: tomscut commented on pull request #3839:
URL: https://github.com/apache/hadoop/pull/3839#issuecomment-1014961076
The failed unit test `TestDataNodeHotSwapVolumes` is unrelated to the change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 710190)
Time Spent: 2h 10m (was: 2h)
> HeartbeatManager may cause incorrect stats
> --
>
> Key: HDFS-16402
> URL: https://issues.apache.org/jira/browse/HDFS-16402
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: tomscut
>Assignee: tomscut
>Priority: Major
> Labels: pull-request-available
> Attachments: image-2021-12-29-08-25-44-303.png,
> image-2021-12-29-08-25-54-441.png
>
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> After reconfig {*}dfs.datanode.data.dir{*}, we found that the stats of the
> Namenode Web became *negative* and there were many NPE in namenode logs. This
> problem has been solved by HDFS-14042.
> !image-2021-12-29-08-25-54-441.png|width=681,height=293!
> !image-2021-12-29-08-25-44-303.png|width=677,height=180!
> However, if *HeartbeatManager#updateHeartbeat* and
> *HeartbeatManager#updateLifeline* throw other exceptions, stats errors can
> also occur. We should ensure that *stats.subtract()* and *stats.add()* are
> transactional.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-16428) When source path setted storagePolicy in rename operation will cause wrong typeConsumed
[ https://issues.apache.org/jira/browse/HDFS-16428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lei w updated HDFS-16428: - Attachment: example.txt > When source path setted storagePolicy in rename operation will cause wrong > typeConsumed > > > Key: HDFS-16428 > URL: https://issues.apache.org/jira/browse/HDFS-16428 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs, namenode >Reporter: lei w >Priority: Major > Attachments: example.txt > > > When compute quota in rename operation , we use storage policy of the target > directory to compute src quota usage. This will cause wrong value of > typeConsumed when source path was setted storage policy. I provided a unit > test to present this situation. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-16428) When source path setted storagePolicy in rename operation will cause wrong typeConsumed
lei w created HDFS-16428: Summary: When source path setted storagePolicy in rename operation will cause wrong typeConsumed Key: HDFS-16428 URL: https://issues.apache.org/jira/browse/HDFS-16428 Project: Hadoop HDFS Issue Type: Bug Components: hdfs, namenode Reporter: lei w When compute quota in rename operation , we use storage policy of the target directory to compute src quota usage. This will cause wrong value of typeConsumed when source path was setted storage policy. I provided a unit test to present this situation. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-15180) DataNode FsDatasetImpl Fine-Grained Locking via BlockPool.
[ https://issues.apache.org/jira/browse/HDFS-15180?focusedWorklogId=709924&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-709924 ] ASF GitHub Bot logged work on HDFS-15180: - Author: ASF GitHub Bot Created on: 17/Jan/22 13:15 Start Date: 17/Jan/22 13:15 Worklog Time Spent: 10m Work Description: Hexiaoqiao commented on pull request #3889: URL: https://github.com/apache/hadoop/pull/3889#issuecomment-1014523383 Thanks @MingXiangLi for your works. It is a great progress now. Take a quick glance, I think it is better to split it to different separate JIRA and PR, and more smooth to merge. cc @jojochuang @sodonnel @goiri @ayushtkn Would you mind to take a look and give some other suggestions? Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 709924) Time Spent: 0.5h (was: 20m) > DataNode FsDatasetImpl Fine-Grained Locking via BlockPool. > --- > > Key: HDFS-15180 > URL: https://issues.apache.org/jira/browse/HDFS-15180 > Project: Hadoop HDFS > Issue Type: Improvement > Components: datanode >Affects Versions: 3.2.0 >Reporter: Qi Zhu >Assignee: Mingxiang Li >Priority: Major > Labels: pull-request-available > Attachments: HDFS-15180.001.patch, HDFS-15180.002.patch, > HDFS-15180.003.patch, HDFS-15180.004.patch, > image-2020-03-10-17-22-57-391.png, image-2020-03-10-17-31-58-830.png, > image-2020-03-10-17-34-26-368.png, image-2020-04-09-11-20-36-459.png > > Time Spent: 0.5h > Remaining Estimate: 0h > > Now the FsDatasetImpl datasetLock is heavy, when their are many namespaces in > big cluster. If we can split the FsDatasetImpl datasetLock via blockpool. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15929) Replace RAND_pseudo_bytes in util.cc
[
https://issues.apache.org/jira/browse/HDFS-15929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17477123#comment-17477123
]
Mikhail Epikhin commented on HDFS-15929:
[~gaurava] It would be lovely, because current stable branches are unbuildable.
> Replace RAND_pseudo_bytes in util.cc
>
>
> Key: HDFS-15929
> URL: https://issues.apache.org/jira/browse/HDFS-15929
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: libhdfs++
>Reporter: Gautham Banasandra
>Assignee: Gautham Banasandra
>Priority: Critical
> Labels: pull-request-available
> Fix For: 3.4.0
>
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> RAND_pseudo_bytes was deprecated in OpenSSL 1.1.1. We get the following
> warning during compilation that it's deprecated -
> {code}
> /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/lib/common/util.cc:
> In function ‘std::string hdfs::GetRandomClientName()’:
> /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/lib/common/util.cc:78:31:
> warning: ‘int RAND_pseudo_bytes(unsigned char*, int)’ is deprecated
> [-Wdeprecated-declarations]
>78 | RAND_pseudo_bytes(&buf[0], 8);
> | ^
> In file included from /usr/include/openssl/e_os2.h:13,
> from /usr/include/openssl/ossl_typ.h:19,
> from /usr/include/openssl/rand.h:14,
> from
> /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/lib/common/util.h:29,
> from
> /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/lib/common/util.cc:19:
> /usr/include/openssl/rand.h:44:1: note: declared here
>44 | DEPRECATEDIN_1_1_0(int RAND_pseudo_bytes(unsigned char *buf, int num))
> | ^~
> /mnt/c/Users/Gautham/projects/apache/wsl/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/lib/common/util.cc:78:31:
> warning: ‘int RAND_pseudo_bytes(unsigned char*, int)’ is deprecated
> [-Wdeprecated-declarations]
>78 | RAND_pseudo_bytes(&buf[0], 8);
> | ^
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16423) balancer should not get blocks on stale storages
[ https://issues.apache.org/jira/browse/HDFS-16423?focusedWorklogId=709820&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-709820 ] ASF GitHub Bot logged work on HDFS-16423: - Author: ASF GitHub Bot Created on: 17/Jan/22 10:03 Start Date: 17/Jan/22 10:03 Worklog Time Spent: 10m Work Description: liubingxing edited a comment on pull request #3883: URL: https://github.com/apache/hadoop/pull/3883#issuecomment-1014195578 @tasanuma @Hexiaoqiao Please take a look. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 709820) Time Spent: 1h 40m (was: 1.5h) > balancer should not get blocks on stale storages > > > Key: HDFS-16423 > URL: https://issues.apache.org/jira/browse/HDFS-16423 > Project: Hadoop HDFS > Issue Type: Improvement > Components: balancer & mover >Reporter: qinyuren >Assignee: qinyuren >Priority: Major > Labels: pull-request-available > Attachments: image-2022-01-13-17-18-32-409.png > > Time Spent: 1h 40m > Remaining Estimate: 0h > > We have met a problems as described in HDFS-16420 > We found that balancer copied a block multi times without deleting the source > block if this block was placed in a stale storage. And resulting a block with > many copies, but these redundant copies are not deleted until the storage > become not stale. > > !image-2022-01-13-17-18-32-409.png|width=657,height=275! -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16426) fix nextBlockReportTime when trigger full block report force
[ https://issues.apache.org/jira/browse/HDFS-16426?focusedWorklogId=709818&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-709818 ] ASF GitHub Bot logged work on HDFS-16426: - Author: ASF GitHub Bot Created on: 17/Jan/22 10:03 Start Date: 17/Jan/22 10:03 Worklog Time Spent: 10m Work Description: liubingxing commented on pull request #3887: URL: https://github.com/apache/hadoop/pull/3887#issuecomment-1014341899 @tasanuma @Hexiaoqiao Please take a look. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 709818) Time Spent: 1h 10m (was: 1h) > fix nextBlockReportTime when trigger full block report force > > > Key: HDFS-16426 > URL: https://issues.apache.org/jira/browse/HDFS-16426 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: qinyuren >Priority: Major > Labels: pull-request-available > Time Spent: 1h 10m > Remaining Estimate: 0h > > When we trigger full block report force by command line, the next block > report time will be set like this: > nextBlockReportTime.getAndAdd(blockReportIntervalMs); > nextBlockReportTime will larger than blockReportIntervalMs. > If we trigger full block report twice, the nextBlockReportTime will larger > than 2 * blockReportIntervalMs. This is obviously not what we want. > We fix the nextBlockReportTime = now + blockReportIntervalMs after full block > report trigger by command line. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
