Re: [PR] HDFS-17250. EditLogTailer#triggerActiveLogRoll should handle thread Interrupted [hadoop]
hadoop-yetus commented on PR #6266:
URL: https://github.com/apache/hadoop/pull/6266#issuecomment-1806850727
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 54s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 49m 22s | | trunk passed |
| +1 :green_heart: | compile | 1m 24s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 1m 14s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 1m 11s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 24s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 9s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 36s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 3m 17s | | trunk passed |
| +1 :green_heart: | shadedclient | 40m 44s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 12s | | the patch passed |
| +1 :green_heart: | compile | 1m 15s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 1m 15s | | the patch passed |
| +1 :green_heart: | compile | 1m 7s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 1m 7s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 1s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 13s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 57s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 30s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 3m 21s | | the patch passed |
| +1 :green_heart: | shadedclient | 40m 30s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 240m 56s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6266/1/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 0m 42s | | The patch does not
generate ASF License warnings. |
| | | 397m 27s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.hdfs.server.datanode.TestDirectoryScanner |
| | hadoop.hdfs.TestDFSUtil |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6266/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6266 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 91bee368e523 4.15.0-213-generic #224-Ubuntu SMP Mon Jun 19
13:30:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 4774224af6701bcd041da7cc1b47efcb7ea8aff9 |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6266/1/testReport/ |
| Max. process+thread count | 2363 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6266/1/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically ge
Re: [PR] HDFS-17250. EditLogTailer#triggerActiveLogRoll should handle thread Interrupted [hadoop]
hadoop-yetus commented on PR #6266:
URL: https://github.com/apache/hadoop/pull/6266#issuecomment-1807093430
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 18m 11s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 49m 12s | | trunk passed |
| +1 :green_heart: | compile | 1m 22s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 1m 15s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 1m 11s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 22s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 10s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 36s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 3m 18s | | trunk passed |
| +1 :green_heart: | shadedclient | 40m 28s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 10s | | the patch passed |
| +1 :green_heart: | compile | 1m 15s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 1m 15s | | the patch passed |
| +1 :green_heart: | compile | 1m 8s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 1m 8s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 2s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 14s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 56s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 32s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 3m 21s | | the patch passed |
| +1 :green_heart: | shadedclient | 40m 33s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 239m 56s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6266/2/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 0m 43s | | The patch does not
generate ASF License warnings. |
| | | 412m 54s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.hdfs.server.datanode.TestDirectoryScanner |
| | hadoop.hdfs.TestDFSUtil |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6266/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6266 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux f5319d6216b0 4.15.0-213-generic #224-Ubuntu SMP Mon Jun 19
13:30:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 27e67f4d05631ca8974ac77127198fe5fbe61072 |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6266/2/testReport/ |
| Max. process+thread count | 2354 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6266/2/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically ge
Re: [PR] HDFS-17250. EditLogTailer#triggerActiveLogRoll should handle thread Interrupted [hadoop]
haiyang1987 commented on PR #6266: URL: https://github.com/apache/hadoop/pull/6266#issuecomment-1807711506 Hi @Hexiaoqiao @ZanderXu @ayushtkn @zhangshuyan0 Would you mind to take a review this pr when you have free time? thank you very much~ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HDFS-17250. EditLogTailer#triggerActiveLogRoll should handle thread Interrupted [hadoop]
haiyang1987 commented on PR #6266: URL: https://github.com/apache/hadoop/pull/6266#issuecomment-1812423453 Hi @tomscut @xinglin Would you mind to also take a review this pr when you have free time? thank you very much~ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HDFS-17250. EditLogTailer#triggerActiveLogRoll should handle thread Interrupted [hadoop]
haiyang1987 commented on PR #6266: URL: https://github.com/apache/hadoop/pull/6266#issuecomment-1813787555 > Thanks @haiyang1987 for your good work here. LGTM. +1 from my side. Let's wait if other folks would like to be involved here. Thanks. Thanks @Hexiaoqiao for your review it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HDFS-17250. EditLogTailer#triggerActiveLogRoll should handle thread Interrupted [hadoop]
xinglin commented on PR #6266:
URL: https://github.com/apache/hadoop/pull/6266#issuecomment-1817339327
qq: which version of hadoop did you run? Does `triggerActiveLogRoll()`
actually try to connect other NN as well? Is your issue happening for trunk? By
reading the source code, that does not seem to be the case.
log line of "Triggering log rolling to the remote NameNode, ": is this
truncated in your paste? seems to be incomplete compared to trunk.
```
/**
* Trigger the active node to roll its logs.
*/
@VisibleForTesting
void triggerActiveLogRoll() {
Future future = null;
try {
...
}
Callable getNameNodeProxy() {
return new MultipleNameNodeProxy() {
@Override
protected Void doWork() throws IOException {
LOG.info("Triggering log rolling to the remote NameNode, " +
"active NameNode = {}", currentNN.getIpcAddress());
cachedActiveProxy.rollEditLog();
return null;
}
};
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HDFS-17250. EditLogTailer#triggerActiveLogRoll should handle thread Interrupted [hadoop]
haiyang1987 commented on PR #6266:
URL: https://github.com/apache/hadoop/pull/6266#issuecomment-1817514156
> qq: which version of hadoop did you run? Does `triggerActiveLogRoll()`
actually try to connect to other NNs as well or only activeNN? Is your issue
happening for trunk? By reading the source code, that does not seem to be the
case: It seems `triggerActiveLogRoll` should only contact activeNN.
>
> log line of "Triggering log rolling to the remote NameNode, ": is this
truncated in your paste? seems to be incomplete compared to trunk.
>
> ```
> /**
>* Trigger the active node to roll its logs.
>*/
> @VisibleForTesting
> void triggerActiveLogRoll() {
> Future future = null;
> try {
> ...
>}
>
> Callable getNameNodeProxy() {
> return new MultipleNameNodeProxy() {
> @Override
> protected Void doWork() throws IOException {
> LOG.info("Triggering log rolling to the remote NameNode, " +
> "active NameNode = {}", currentNN.getIpcAddress());
> cachedActiveProxy.rollEditLog();
> return null;
> }
> };
> ```
Thanks @xinglin for your comment.
The current version is hadoop 3.3, triggerActiveLogRoll() will actually try
to connect to other NNs until active nn is requested, and recorded as
cachedActiveProxy
for this log
```
LOG.info("Triggering log rolling to the remote NameNode, " + "active
NameNode = {}", currentNN.getIpcAddress());
```
It is from by PR: HDFS-1630, it only record the remote NameNode address in
logs. this PR is not introduced in our current version.
trunk branch still has the problem described in the current issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HDFS-17250. EditLogTailer#triggerActiveLogRoll should handle thread Interrupted [hadoop]
xinglin commented on PR #6266: URL: https://github.com/apache/hadoop/pull/6266#issuecomment-1817623782 thanks @haiyang1987. The name of`getActiveNodeProxy()` caused the confusion. It should probably have been named as `getAliveNodeProxy`. ActiveNN has a specific meaning in the context of HA-NN. `getActiveNodeProxy` indeed iterates over all NNs. Neither `MultipleNameNodeProxy.call() or MultipleNameNodeProxy.getActiveNodeProxy()` catch InterruptedExecution in their try_catch block. have two dummy questions Question 1: when we submit a task to an executorService and later on we cancel it due to timeout, and we don't capture interrupted exception in our task, what state will the thread from executorService be left? Do we have to capture interrupted exception in our task in this case? ob1 trying to connect to ob2. we timed out after 60 sec. Are you saying the thread in the executorService would still be in Interrupted state, even though we have throw up the Interrupted Execution, as showed in the log (also because it is not captured within MultipleNameNodeProxy)? This seems to be contradicting to a statement I found from below blog, which says when an interruptedExecution is threw, the interruption status of that thread will be cleared. "Before a blocking code throws an InterruptedException, it marks the interruption status as false." https://praveer09.github.io/technology/2015/12/06/understanding-thread-interruption-in-java/ Another question: assuming the thread in the executorService is still in interrupted state, then how is it gotten cleared by your PR? `Thread.currentThread().isInterrupted()` will not clear the interrupt flag for a thread, as the Oracle Java doc says. > When a thread checks for an interrupt by invoking the static method Thread.interrupted, interrupt status is cleared. The non-static isInterrupted method, which is used by one thread to query the interrupt status of another, does not change the interrupt status flag. https://docs.oracle.com/javase/tutorial/essential/concurrency/interrupt.html#:~:text=When%20a%20thread%20checks%20for,change%20the%20interrupt%20status%20flag. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HDFS-17250. EditLogTailer#triggerActiveLogRoll should handle thread Interrupted [hadoop]
xinglin commented on PR #6266: URL: https://github.com/apache/hadoop/pull/6266#issuecomment-1817703121 `InterruptedIOException` is a subClass of `IOException`. In the try_catch block in `getActiveNodeProxy`, we do catch for `IOException`. So, interruptedIOException will be captured there. https://docs.oracle.com/javase/8/docs/api/java/io/InterruptedIOException.html What I think should be happening is as following. main_Thread calling `triggerActiveLogRoll`, wait for 60 secs, timeout, cancel this task, and return. MultipleNameNodeProxy.call() thread: -> getActiveNodeProxy() -> nnLookup.next = ob2 (down node) -> RPC.waitForProxy(ob2) -> after 60 secs, interrupted. -> output "Failed to reach ob2", increment nnLoopCount. -> nnLookup.next = n1 (live node). then, it should succeed to connect to n1. This does not seem to be the case from the logs you shared. thoughts? A possible fix might be to explicitly capture `InterruptedIOException` in `getActiveNodeProxy`, and just finish for this thread (assuming all `InterruptedIOExceptions` are invoked from `triggerActiveLogRoll`). For the following triggerActiveLogRoll calls, we should be good, since we will move the nnLookup to next one. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HDFS-17250. EditLogTailer#triggerActiveLogRoll should handle thread Interrupted [hadoop]
ZanderXu commented on PR #6266: URL: https://github.com/apache/hadoop/pull/6266#issuecomment-1818151590 @xinglin Thanks for your review. > when we submit a task to an executorService and later on we cancel it due to timeout, and we don't capture interrupted exception in our task, what state will the thread from executorService be left? Do we have to capture interrupted exception in our task in this case? This is very nice question, you can refer to the source code of `ThreadPoolExecutor#Worker#runWorker()`, the worker in the `ThreadPoolExecutor` will clear the `Interrupt` flag. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HDFS-17250. EditLogTailer#triggerActiveLogRoll should handle thread Interrupted [hadoop]
haiyang1987 commented on PR #6266: URL: https://github.com/apache/hadoop/pull/6266#issuecomment-1818872859 Thanks @xinglin @ZanderXu for your detailed review. > > when we submit a task to an executorService and later on we cancel it due to timeout, and we don't capture interrupted exception in our task, what state will the thread from executorService be left? Do we have to capture interrupted exception in our task in this case? > > This is very nice question, you can refer to the source code of `ThreadPoolExecutor#Worker#runWorker()`, the worker in the `ThreadPoolExecutor` will clear the `Interrupt` flag. Yeah, totally agree with this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HDFS-17250. EditLogTailer#triggerActiveLogRoll should handle thread Interrupted [hadoop]
haiyang1987 commented on PR #6266: URL: https://github.com/apache/hadoop/pull/6266#issuecomment-1818874038 > `InterruptedIOException` is a subClass of `IOException`. In the try_catch block in `getActiveNodeProxy`, we do catch for `IOException`. So, interruptedIOException will be captured there. > > https://docs.oracle.com/javase/8/docs/api/java/io/InterruptedIOException.html > > What I think should be happening is as following. > > ``` > main_Thread calling `triggerActiveLogRoll`, wait for 60 secs, timeout, cancel this task, and return. > > MultipleNameNodeProxy.call() thread: >-> getActiveNodeProxy() > -> nnLookup.next = ob2 (down node) > -> RPC.waitForProxy(ob2) > -> after 60 secs, interrupted. > -> output "Failed to reach ob2", increment nnLoopCount. Ideally, we should just stop here, since we already time out. > -> nnLookup.next = n1 (live node). > then, it should succeed to connect to n1. > ``` > > This does not seem to be the case from the logs you shared. thoughts? > > A possible fix might be to explicitly capture `InterruptedIOException` in `getActiveNodeProxy`, and just finish for this thread (assuming all `InterruptedIOExceptions` are invoked from `triggerActiveLogRoll`). For the following triggerActiveLogRoll calls, we should be good, since we will move the nnLookup to next one. Yeah, For MultipleNameNodeProxy#call() explicitly capture InterruptedIOException and then exit execution is also a solution -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HDFS-17250. EditLogTailer#triggerActiveLogRoll should handle thread Interrupted [hadoop]
xinglin commented on PR #6266: URL: https://github.com/apache/hadoop/pull/6266#issuecomment-1822167954 Feel free to commit this PR. did not intend to block here. I will spend more time on my own trying to understand the code/and the change . -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HDFS-17250. EditLogTailer#triggerActiveLogRoll should handle thread Interrupted [hadoop]
haiyang1987 commented on PR #6266: URL: https://github.com/apache/hadoop/pull/6266#issuecomment-183143 Hi @Hexiaoqiao @ZanderXu could you mind to push this modification forward when you have free time ? Thank you very much. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HDFS-17250. EditLogTailer#triggerActiveLogRoll should handle thread Interrupted [hadoop]
Hexiaoqiao commented on PR #6266: URL: https://github.com/apache/hadoop/pull/6266#issuecomment-1837453460 Hi @xinglin Do you have anymore concerns? If not, I will to push this RP forwards. Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HDFS-17250. EditLogTailer#triggerActiveLogRoll should handle thread Interrupted [hadoop]
xinglin commented on PR #6266: URL: https://github.com/apache/hadoop/pull/6266#issuecomment-1837558511 @Hexiaoqiao, no, feel free to merge this PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HDFS-17250. EditLogTailer#triggerActiveLogRoll should handle thread Interrupted [hadoop]
Hexiaoqiao commented on PR #6266: URL: https://github.com/apache/hadoop/pull/6266#issuecomment-1838632135 Committed to trunk. Thanks @haiyang1987 @ZanderXu and @xinglin . -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HDFS-17250. EditLogTailer#triggerActiveLogRoll should handle thread Interrupted [hadoop]
Hexiaoqiao merged PR #6266: URL: https://github.com/apache/hadoop/pull/6266 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HDFS-17250. EditLogTailer#triggerActiveLogRoll should handle thread Interrupted [hadoop]
haiyang1987 commented on PR #6266: URL: https://github.com/apache/hadoop/pull/6266#issuecomment-1839884692 Thanks @Hexiaoqiao @ZanderXu @xinglin for your review and merge. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
