[jira] [Commented] (HDFS-16659) JournalNode should throw NewerTxnIdException if SinceTxId is bigger than HighestWrittenTxId
[
https://issues.apache.org/jira/browse/HDFS-16659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17603520#comment-17603520
]
ASF GitHub Bot commented on HDFS-16659:
---
ZanderXu commented on PR #4560:
URL: https://github.com/apache/hadoop/pull/4560#issuecomment-1245257209
@xkrogen Sir, I create a new ticket HDFS-16771 to resolve this problems. If
you have any good ideas, please share with me. Thanks.
> JournalNode should throw NewerTxnIdException if SinceTxId is bigger than
> HighestWrittenTxId
> ---
>
> Key: HDFS-16659
> URL: https://issues.apache.org/jira/browse/HDFS-16659
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: ZanderXu
>Assignee: ZanderXu
>Priority: Critical
> Labels: pull-request-available
> Fix For: 3.4.0
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> JournalNode should throw `CacheMissException` if `sinceTxId` is bigger than
> `highestWrittenTxId` during handling `getJournaledEdits` rpc from NNs.
> Current logic may cause in-progress EditlogTailer cannot replay any Edits
> from JournalNodes in some corner cases, resulting in ObserverNameNode cannot
> handle requests from clients.
> Suppose there are 3 journalNodes, JN0 ~ JN1.
> * JN0 has some abnormal cases when Active Namenode is syncing 10 Edits with
> first txid 11
> * NameNode just ignore the abnormal JN0 and continue to sync Edits to Journal
> 1 and 2
> * JN0 backed to health
> * NameNode continue sync 10 Edits with first txid 21.
> * At this point, there are no Edits 11 ~ 30 in the cache of JN0
> * Observer NameNode try to select EditLogInputStream through
> `getJournaledEdits` with since txId 21
> * Journal 2 has some abnormal cases and caused a slow response
> The expected result is: Response should contain 20 Edits from txId 21 to txId
> 30 from JN1 and JN2. Because Active NameNode successfully write these Edits
> to JN1 and JN2 and failed write these edits to JN0.
> But in the current implementation, the response is [Response(0) from JN0,
> Response(10) from JN1], because there are some abnormal cases in JN2, such
> as GC, bad network, cause a slow response. So the `maxAllowedTxns` will be
> 0, NameNode will not replay any Edits.
> As above, the root case is that JournalNode should throw Miss Cache Exception
> when `sinceTxid` is more than `highestWrittenTxId`.
> And the bug code as blew:
> {code:java}
> if (sinceTxId > getHighestWrittenTxId()) {
> // Requested edits that don't exist yet; short-circuit the cache here
> metrics.rpcEmptyResponses.incr();
> return
> GetJournaledEditsResponseProto.newBuilder().setTxnCount(0).build();
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16659) JournalNode should throw NewerTxnIdException if SinceTxId is bigger than HighestWrittenTxId
[
https://issues.apache.org/jira/browse/HDFS-16659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17603505#comment-17603505
]
ASF GitHub Bot commented on HDFS-16659:
---
ZanderXu commented on PR #4560:
URL: https://github.com/apache/hadoop/pull/4560#issuecomment-1245232780
@xkrogen After deep thinking and do some verifications, I found there are
two places should be fixed for the case that `sinceTxId = highestTxId + 1`.
Currently Journal throws one `NewerTxnIdException` to namenode, we expect
namenode can catch `NewerTxnIdException` during `selectRpcInputStreams` and
ignore it.
But the namenode throws a `QuorumException` during `selectRpcInputStreams`
because there are a majority of `NewerTxnIdException`. Then the namenode
fallbacks to `selectStreamingInputStreams`.
Beside this problem, JournalNodeRpcServer shouldn't print any logs about
`NewerTxnIdException` when `sinceTxId = highestTxId + 1`, but it should print
some logs about `NewerTxnIdException` when `sinceTxId > highestTxId + 1`.
So as above cases, how about handling them differently? such as
```
long highestTxId = getHighestWrittenTxId();
if (sinceTxId == highestTxId + 1) {
// This is normal case and will return one response with 0 txnCount.
metrics.rpcEmptyResponses.incr();
return
GetJournaledEditsResponseProto.newBuilder().setTxnCount(0).build();
} else if (sinceTxId > highestTxId) {
// Requested edits that don't exist yet and is newer than highestTxId.
metrics.rpcEmptyResponses.incr();
throw new NewerTxnIdException(
"Highest txn ID available in the journal is %d, but requested txns
starting at %d.",
highestTxId, sinceTxId);
}
```
> JournalNode should throw NewerTxnIdException if SinceTxId is bigger than
> HighestWrittenTxId
> ---
>
> Key: HDFS-16659
> URL: https://issues.apache.org/jira/browse/HDFS-16659
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: ZanderXu
>Assignee: ZanderXu
>Priority: Critical
> Labels: pull-request-available
> Fix For: 3.4.0
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> JournalNode should throw `CacheMissException` if `sinceTxId` is bigger than
> `highestWrittenTxId` during handling `getJournaledEdits` rpc from NNs.
> Current logic may cause in-progress EditlogTailer cannot replay any Edits
> from JournalNodes in some corner cases, resulting in ObserverNameNode cannot
> handle requests from clients.
> Suppose there are 3 journalNodes, JN0 ~ JN1.
> * JN0 has some abnormal cases when Active Namenode is syncing 10 Edits with
> first txid 11
> * NameNode just ignore the abnormal JN0 and continue to sync Edits to Journal
> 1 and 2
> * JN0 backed to health
> * NameNode continue sync 10 Edits with first txid 21.
> * At this point, there are no Edits 11 ~ 30 in the cache of JN0
> * Observer NameNode try to select EditLogInputStream through
> `getJournaledEdits` with since txId 21
> * Journal 2 has some abnormal cases and caused a slow response
> The expected result is: Response should contain 20 Edits from txId 21 to txId
> 30 from JN1 and JN2. Because Active NameNode successfully write these Edits
> to JN1 and JN2 and failed write these edits to JN0.
> But in the current implementation, the response is [Response(0) from JN0,
> Response(10) from JN1], because there are some abnormal cases in JN2, such
> as GC, bad network, cause a slow response. So the `maxAllowedTxns` will be
> 0, NameNode will not replay any Edits.
> As above, the root case is that JournalNode should throw Miss Cache Exception
> when `sinceTxid` is more than `highestWrittenTxId`.
> And the bug code as blew:
> {code:java}
> if (sinceTxId > getHighestWrittenTxId()) {
> // Requested edits that don't exist yet; short-circuit the cache here
> metrics.rpcEmptyResponses.incr();
> return
> GetJournaledEditsResponseProto.newBuilder().setTxnCount(0).build();
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16659) JournalNode should throw NewerTxnIdException if SinceTxId is bigger than HighestWrittenTxId
[
https://issues.apache.org/jira/browse/HDFS-16659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17601082#comment-17601082
]
ASF GitHub Bot commented on HDFS-16659:
---
ZanderXu commented on PR #4560:
URL: https://github.com/apache/hadoop/pull/4560#issuecomment-1238805756
@xkrogen Sir, thank you for your patient answers and reviews, as well as
nice suggestions.
> JournalNode should throw NewerTxnIdException if SinceTxId is bigger than
> HighestWrittenTxId
> ---
>
> Key: HDFS-16659
> URL: https://issues.apache.org/jira/browse/HDFS-16659
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: ZanderXu
>Assignee: ZanderXu
>Priority: Critical
> Labels: pull-request-available
> Fix For: 3.4.0
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> JournalNode should throw `CacheMissException` if `sinceTxId` is bigger than
> `highestWrittenTxId` during handling `getJournaledEdits` rpc from NNs.
> Current logic may cause in-progress EditlogTailer cannot replay any Edits
> from JournalNodes in some corner cases, resulting in ObserverNameNode cannot
> handle requests from clients.
> Suppose there are 3 journalNodes, JN0 ~ JN1.
> * JN0 has some abnormal cases when Active Namenode is syncing 10 Edits with
> first txid 11
> * NameNode just ignore the abnormal JN0 and continue to sync Edits to Journal
> 1 and 2
> * JN0 backed to health
> * NameNode continue sync 10 Edits with first txid 21.
> * At this point, there are no Edits 11 ~ 30 in the cache of JN0
> * Observer NameNode try to select EditLogInputStream through
> `getJournaledEdits` with since txId 21
> * Journal 2 has some abnormal cases and caused a slow response
> The expected result is: Response should contain 20 Edits from txId 21 to txId
> 30 from JN1 and JN2. Because Active NameNode successfully write these Edits
> to JN1 and JN2 and failed write these edits to JN0.
> But in the current implementation, the response is [Response(0) from JN0,
> Response(10) from JN1], because there are some abnormal cases in JN2, such
> as GC, bad network, cause a slow response. So the `maxAllowedTxns` will be
> 0, NameNode will not replay any Edits.
> As above, the root case is that JournalNode should throw Miss Cache Exception
> when `sinceTxid` is more than `highestWrittenTxId`.
> And the bug code as blew:
> {code:java}
> if (sinceTxId > getHighestWrittenTxId()) {
> // Requested edits that don't exist yet; short-circuit the cache here
> metrics.rpcEmptyResponses.incr();
> return
> GetJournaledEditsResponseProto.newBuilder().setTxnCount(0).build();
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16659) JournalNode should throw NewerTxnIdException if SinceTxId is bigger than HighestWrittenTxId
[
https://issues.apache.org/jira/browse/HDFS-16659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17600904#comment-17600904
]
ASF GitHub Bot commented on HDFS-16659:
---
xkrogen commented on PR #4560:
URL: https://github.com/apache/hadoop/pull/4560#issuecomment-1238439633
Merged to trunk. Thanks for the contribution @ZanderXu !
> JournalNode should throw NewerTxnIdException if SinceTxId is bigger than
> HighestWrittenTxId
> ---
>
> Key: HDFS-16659
> URL: https://issues.apache.org/jira/browse/HDFS-16659
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: ZanderXu
>Assignee: ZanderXu
>Priority: Critical
> Labels: pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
> JournalNode should throw `CacheMissException` if `sinceTxId` is bigger than
> `highestWrittenTxId` during handling `getJournaledEdits` rpc from NNs.
> Current logic may cause in-progress EditlogTailer cannot replay any Edits
> from JournalNodes in some corner cases, resulting in ObserverNameNode cannot
> handle requests from clients.
> Suppose there are 3 journalNodes, JN0 ~ JN1.
> * JN0 has some abnormal cases when Active Namenode is syncing 10 Edits with
> first txid 11
> * NameNode just ignore the abnormal JN0 and continue to sync Edits to Journal
> 1 and 2
> * JN0 backed to health
> * NameNode continue sync 10 Edits with first txid 21.
> * At this point, there are no Edits 11 ~ 30 in the cache of JN0
> * Observer NameNode try to select EditLogInputStream through
> `getJournaledEdits` with since txId 21
> * Journal 2 has some abnormal cases and caused a slow response
> The expected result is: Response should contain 20 Edits from txId 21 to txId
> 30 from JN1 and JN2. Because Active NameNode successfully write these Edits
> to JN1 and JN2 and failed write these edits to JN0.
> But in the current implementation, the response is [Response(0) from JN0,
> Response(10) from JN1], because there are some abnormal cases in JN2, such
> as GC, bad network, cause a slow response. So the `maxAllowedTxns` will be
> 0, NameNode will not replay any Edits.
> As above, the root case is that JournalNode should throw Miss Cache Exception
> when `sinceTxid` is more than `highestWrittenTxId`.
> And the bug code as blew:
> {code:java}
> if (sinceTxId > getHighestWrittenTxId()) {
> // Requested edits that don't exist yet; short-circuit the cache here
> metrics.rpcEmptyResponses.incr();
> return
> GetJournaledEditsResponseProto.newBuilder().setTxnCount(0).build();
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16659) JournalNode should throw NewerTxnIdException if SinceTxId is bigger than HighestWrittenTxId
[
https://issues.apache.org/jira/browse/HDFS-16659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17600903#comment-17600903
]
ASF GitHub Bot commented on HDFS-16659:
---
xkrogen merged PR #4560:
URL: https://github.com/apache/hadoop/pull/4560
> JournalNode should throw NewerTxnIdException if SinceTxId is bigger than
> HighestWrittenTxId
> ---
>
> Key: HDFS-16659
> URL: https://issues.apache.org/jira/browse/HDFS-16659
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: ZanderXu
>Assignee: ZanderXu
>Priority: Critical
> Labels: pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
> JournalNode should throw `CacheMissException` if `sinceTxId` is bigger than
> `highestWrittenTxId` during handling `getJournaledEdits` rpc from NNs.
> Current logic may cause in-progress EditlogTailer cannot replay any Edits
> from JournalNodes in some corner cases, resulting in ObserverNameNode cannot
> handle requests from clients.
> Suppose there are 3 journalNodes, JN0 ~ JN1.
> * JN0 has some abnormal cases when Active Namenode is syncing 10 Edits with
> first txid 11
> * NameNode just ignore the abnormal JN0 and continue to sync Edits to Journal
> 1 and 2
> * JN0 backed to health
> * NameNode continue sync 10 Edits with first txid 21.
> * At this point, there are no Edits 11 ~ 30 in the cache of JN0
> * Observer NameNode try to select EditLogInputStream through
> `getJournaledEdits` with since txId 21
> * Journal 2 has some abnormal cases and caused a slow response
> The expected result is: Response should contain 20 Edits from txId 21 to txId
> 30 from JN1 and JN2. Because Active NameNode successfully write these Edits
> to JN1 and JN2 and failed write these edits to JN0.
> But in the current implementation, the response is [Response(0) from JN0,
> Response(10) from JN1], because there are some abnormal cases in JN2, such
> as GC, bad network, cause a slow response. So the `maxAllowedTxns` will be
> 0, NameNode will not replay any Edits.
> As above, the root case is that JournalNode should throw Miss Cache Exception
> when `sinceTxid` is more than `highestWrittenTxId`.
> And the bug code as blew:
> {code:java}
> if (sinceTxId > getHighestWrittenTxId()) {
> // Requested edits that don't exist yet; short-circuit the cache here
> metrics.rpcEmptyResponses.incr();
> return
> GetJournaledEditsResponseProto.newBuilder().setTxnCount(0).build();
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16659) JournalNode should throw NewerTxnIdException if SinceTxId is bigger than HighestWrittenTxId
[
https://issues.apache.org/jira/browse/HDFS-16659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17599594#comment-17599594
]
ASF GitHub Bot commented on HDFS-16659:
---
hadoop-yetus commented on PR #4560:
URL: https://github.com/apache/hadoop/pull/4560#issuecomment-1235675888
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 42s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 38m 45s | | trunk passed |
| +1 :green_heart: | compile | 1m 34s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 1m 32s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 23s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 38s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 19s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 43s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 39s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 6s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 19s | | the patch passed |
| +1 :green_heart: | compile | 1m 24s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 1m 24s | | the patch passed |
| +1 :green_heart: | compile | 1m 16s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 1m 16s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 58s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 22s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 57s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 29s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 16s | | the patch passed |
| +1 :green_heart: | shadedclient | 22m 28s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 237m 30s | | hadoop-hdfs in the patch
passed. |
| +1 :green_heart: | asflicense | 1m 0s | | The patch does not
generate ASF License warnings. |
| | | 346m 21s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4560/6/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4560 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux f9bd71d68f1c 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 095e80b0dbfb99371d2489fb37844f847c6cf820 |
| Default Java | Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4560/6/testReport/ |
| Max. process+thread count | 3519 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4560/6/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
> JournalNode should throw
[jira] [Commented] (HDFS-16659) JournalNode should throw NewerTxnIdException if SinceTxId is bigger than HighestWrittenTxId
[
https://issues.apache.org/jira/browse/HDFS-16659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17599366#comment-17599366
]
ASF GitHub Bot commented on HDFS-16659:
---
hadoop-yetus commented on PR #4560:
URL: https://github.com/apache/hadoop/pull/4560#issuecomment-1235284101
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 48s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 39m 59s | | trunk passed |
| +1 :green_heart: | compile | 1m 38s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 1m 30s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 18s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 45s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 16s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 39s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 44s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 30s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 19s | | the patch passed |
| +1 :green_heart: | compile | 1m 23s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 1m 23s | | the patch passed |
| +1 :green_heart: | compile | 1m 24s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 1m 24s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 58s |
[/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4560/5/artifact/out/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 17 unchanged -
0 fixed = 18 total (was 17) |
| +1 :green_heart: | mvnsite | 1m 30s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 57s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 31s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 21s | | the patch passed |
| +1 :green_heart: | shadedclient | 22m 25s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 253m 4s | | hadoop-hdfs in the patch
passed. |
| +1 :green_heart: | asflicense | 1m 12s | | The patch does not
generate ASF License warnings. |
| | | 364m 30s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4560/5/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4560 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 12111e311603 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / fc9b2d6b18e0571cc4fce42c3d134ff263039b55 |
| Default Java | Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4560/5/testReport/ |
| Max. process+thread count | 2920 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
[jira] [Commented] (HDFS-16659) JournalNode should throw NewerTxnIdException if SinceTxId is bigger than HighestWrittenTxId
[
https://issues.apache.org/jira/browse/HDFS-16659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17599240#comment-17599240
]
ASF GitHub Bot commented on HDFS-16659:
---
ZanderXu commented on code in PR #4560:
URL: https://github.com/apache/hadoop/pull/4560#discussion_r961258064
##
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/qjournal/client/TestQuorumJournalManager.java:
##
@@ -1122,41 +1115,50 @@ public void testSelectViaRpcTwoJNsError() throws
Exception {
* 4. Observer NameNode try to select EditLogInputStream vis PRC with start
txId 21.
* 5. Journal 1 has some abnormal cases caused slow response.
*
- * And the expected selecting result is: Response should contain 20 Edits
from txId 21 to txId 40.
- * Because there is no Edits from id 21 to 40 in the cache of JournalNode0.
+ * The expected result should contain txn 21 - 40.
*/
@Test
- public void testSelectViaRpcAfterJNJitter() throws Exception {
+ public void testSelectViaRPCAfterJNJitter() throws Exception {
EditLogOutputStream stm = qjm.startLogSegment(
1, NameNodeLayoutVersion.CURRENT_LAYOUT_VERSION);
SettableFuture slowLog = SettableFuture.create();
Mockito.doReturn(slowLog).when(spies.get(0))
-.sendEdits(eq(1L), eq(11L), eq(1), Mockito.any());
+.sendEdits(eq(1L), eq(11L), eq(10), Mockito.any());
+// Successfully write these edits to JN0 ~ JN2
writeTxns(stm, 1, 10);
+// Failed write these edits to JN0, but successfully write them to JN1 ~
JN2
writeTxns(stm, 11, 10);
-writeTxns(stm, 21, 10);
-writeTxns(stm, 31, 10);
-ListeningExecutorService service = MoreExecutors.listeningDecorator(
-Executors.newSingleThreadExecutor());
-Mockito.doAnswer(invocation -> service.submit(
-() -> {
- ByteArrayOutputStream byteStream = new ByteArrayOutputStream();
- EditLogFileOutputStream.writeHeader(
- NameNodeLayoutVersion.CURRENT_LAYOUT_VERSION,
- new DataOutputStream(byteStream));
- byteStream.write(createTxnData(21, 20));
- Thread.sleep(3000);
- return GetJournaledEditsResponseProto.newBuilder()
- .setTxnCount(20)
- .setEditLog(ByteString.copyFrom(byteStream.toByteArray()))
- .build();
-})
-).when(spies.get(1)).getJournaledEdits(21,
+// Successfully write these edits to JN1 ~ JN2
+writeTxns(stm, 21, 20);
+
+Semaphore semaphore = new Semaphore(0);
+
+
Mockito.doAnswer((Answer>)
invocation -> {
+ semaphore.release(1);
+ @SuppressWarnings("unchecked")
+ ListenableFuture result =
+ (ListenableFuture)
invocation.callRealMethod();
+ return result;
+}).when(spies.get(0)).getJournaledEdits(21,
+QuorumJournalManager.QJM_RPC_MAX_TXNS_DEFAULT);
+
+
Mockito.doAnswer((Answer>)
invocation -> {
+ semaphore.release(1);
+ @SuppressWarnings("unchecked")
+ ListenableFuture result =
+ (ListenableFuture)
invocation.callRealMethod();
+ return result;
+}).when(spies.get(1)).getJournaledEdits(21,
QuorumJournalManager.QJM_RPC_MAX_TXNS_DEFAULT);
-GetJournaledEditsResponseProto responseProto = spies.get(2)
-.getJournaledEdits(21, 5000).get();
-assertEquals(20, responseProto.getTxnCount());
+
Mockito.doAnswer((Answer>)
invocation -> {
+ semaphore.acquire(2);
+ @SuppressWarnings("unchecked")
+ ListenableFuture result =
+ (ListenableFuture)
invocation.callRealMethod();
+ return result;
+}).when(spies.get(2)).getJournaledEdits(21,
+QuorumJournalManager.QJM_RPC_MAX_TXNS_DEFAULT);
Review Comment:
Nice suggestion, thanks.
> JournalNode should throw NewerTxnIdException if SinceTxId is bigger than
> HighestWrittenTxId
> ---
>
> Key: HDFS-16659
> URL: https://issues.apache.org/jira/browse/HDFS-16659
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: ZanderXu
>Assignee: ZanderXu
>Priority: Critical
> Labels: pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
> JournalNode should throw `CacheMissException` if `sinceTxId` is bigger than
> `highestWrittenTxId` during handling `getJournaledEdits` rpc from NNs.
> Current logic may cause in-progress EditlogTailer cannot replay any Edits
> from JournalNodes in some corner cases, resulting in ObserverNameNode cannot
> handle requests from clients.
> Suppose there are 3 journalNodes, JN0 ~ JN1.
> * JN0 has some abnormal cases when Active Namenode is syncing 10 Edits with
> first txid 11
> * NameNode just ignore the abnormal JN0 and continue to sync Edits to Journal
> 1 and 2
> * JN0 backed to health
> * NameNode
[jira] [Commented] (HDFS-16659) JournalNode should throw NewerTxnIdException if SinceTxId is bigger than HighestWrittenTxId
[
https://issues.apache.org/jira/browse/HDFS-16659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17599239#comment-17599239
]
ASF GitHub Bot commented on HDFS-16659:
---
ZanderXu commented on code in PR #4560:
URL: https://github.com/apache/hadoop/pull/4560#discussion_r961257965
##
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/qjournal/client/TestQuorumJournalManager.java:
##
@@ -1101,6 +1113,59 @@ public void testSelectViaRpcTwoJNsError() throws
Exception {
}
}
+ /**
+ * Test selecting EditLogInputStream after some journalNode jitter.
+ * And the corner case as below:
+ * 1. Journal 0 has some abnormal cases when journaling Edits with start
txId 11.
+ * 2. NameNode just ignore the abnormal journal 0 and continue to write
Edits to Journal 1 and 2.
+ * 3. Journal 0 backed to health.
+ * 4. Observer NameNode try to select EditLogInputStream vis PRC with start
txId 21.
+ * 5. Journal 1 has some abnormal cases caused slow response.
+ *
+ * And the expected selecting result is: Response should contain 20 Edits
from txId 21 to txId 40.
+ * Because there is no Edits from id 21 to 40 in the cache of JournalNode0.
Review Comment:
Copy, Sir. I have updated it.
> JournalNode should throw NewerTxnIdException if SinceTxId is bigger than
> HighestWrittenTxId
> ---
>
> Key: HDFS-16659
> URL: https://issues.apache.org/jira/browse/HDFS-16659
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: ZanderXu
>Assignee: ZanderXu
>Priority: Critical
> Labels: pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
> JournalNode should throw `CacheMissException` if `sinceTxId` is bigger than
> `highestWrittenTxId` during handling `getJournaledEdits` rpc from NNs.
> Current logic may cause in-progress EditlogTailer cannot replay any Edits
> from JournalNodes in some corner cases, resulting in ObserverNameNode cannot
> handle requests from clients.
> Suppose there are 3 journalNodes, JN0 ~ JN1.
> * JN0 has some abnormal cases when Active Namenode is syncing 10 Edits with
> first txid 11
> * NameNode just ignore the abnormal JN0 and continue to sync Edits to Journal
> 1 and 2
> * JN0 backed to health
> * NameNode continue sync 10 Edits with first txid 21.
> * At this point, there are no Edits 11 ~ 30 in the cache of JN0
> * Observer NameNode try to select EditLogInputStream through
> `getJournaledEdits` with since txId 21
> * Journal 2 has some abnormal cases and caused a slow response
> The expected result is: Response should contain 20 Edits from txId 21 to txId
> 30 from JN1 and JN2. Because Active NameNode successfully write these Edits
> to JN1 and JN2 and failed write these edits to JN0.
> But in the current implementation, the response is [Response(0) from JN0,
> Response(10) from JN1], because there are some abnormal cases in JN2, such
> as GC, bad network, cause a slow response. So the `maxAllowedTxns` will be
> 0, NameNode will not replay any Edits.
> As above, the root case is that JournalNode should throw Miss Cache Exception
> when `sinceTxid` is more than `highestWrittenTxId`.
> And the bug code as blew:
> {code:java}
> if (sinceTxId > getHighestWrittenTxId()) {
> // Requested edits that don't exist yet; short-circuit the cache here
> metrics.rpcEmptyResponses.incr();
> return
> GetJournaledEditsResponseProto.newBuilder().setTxnCount(0).build();
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16659) JournalNode should throw NewerTxnIdException if SinceTxId is bigger than HighestWrittenTxId
[
https://issues.apache.org/jira/browse/HDFS-16659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17599204#comment-17599204
]
ASF GitHub Bot commented on HDFS-16659:
---
xkrogen commented on code in PR #4560:
URL: https://github.com/apache/hadoop/pull/4560#discussion_r961194399
##
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/qjournal/client/TestQuorumJournalManager.java:
##
@@ -1122,41 +1115,50 @@ public void testSelectViaRpcTwoJNsError() throws
Exception {
* 4. Observer NameNode try to select EditLogInputStream vis PRC with start
txId 21.
* 5. Journal 1 has some abnormal cases caused slow response.
*
- * And the expected selecting result is: Response should contain 20 Edits
from txId 21 to txId 40.
- * Because there is no Edits from id 21 to 40 in the cache of JournalNode0.
+ * The expected result should contain txn 21 - 40.
*/
@Test
- public void testSelectViaRpcAfterJNJitter() throws Exception {
+ public void testSelectViaRPCAfterJNJitter() throws Exception {
EditLogOutputStream stm = qjm.startLogSegment(
1, NameNodeLayoutVersion.CURRENT_LAYOUT_VERSION);
SettableFuture slowLog = SettableFuture.create();
Mockito.doReturn(slowLog).when(spies.get(0))
-.sendEdits(eq(1L), eq(11L), eq(1), Mockito.any());
+.sendEdits(eq(1L), eq(11L), eq(10), Mockito.any());
+// Successfully write these edits to JN0 ~ JN2
writeTxns(stm, 1, 10);
+// Failed write these edits to JN0, but successfully write them to JN1 ~
JN2
writeTxns(stm, 11, 10);
-writeTxns(stm, 21, 10);
-writeTxns(stm, 31, 10);
-ListeningExecutorService service = MoreExecutors.listeningDecorator(
-Executors.newSingleThreadExecutor());
-Mockito.doAnswer(invocation -> service.submit(
-() -> {
- ByteArrayOutputStream byteStream = new ByteArrayOutputStream();
- EditLogFileOutputStream.writeHeader(
- NameNodeLayoutVersion.CURRENT_LAYOUT_VERSION,
- new DataOutputStream(byteStream));
- byteStream.write(createTxnData(21, 20));
- Thread.sleep(3000);
- return GetJournaledEditsResponseProto.newBuilder()
- .setTxnCount(20)
- .setEditLog(ByteString.copyFrom(byteStream.toByteArray()))
- .build();
-})
-).when(spies.get(1)).getJournaledEdits(21,
+// Successfully write these edits to JN1 ~ JN2
+writeTxns(stm, 21, 20);
+
+Semaphore semaphore = new Semaphore(0);
+
+
Mockito.doAnswer((Answer>)
invocation -> {
+ semaphore.release(1);
+ @SuppressWarnings("unchecked")
+ ListenableFuture result =
+ (ListenableFuture)
invocation.callRealMethod();
+ return result;
+}).when(spies.get(0)).getJournaledEdits(21,
+QuorumJournalManager.QJM_RPC_MAX_TXNS_DEFAULT);
+
+
Mockito.doAnswer((Answer>)
invocation -> {
+ semaphore.release(1);
+ @SuppressWarnings("unchecked")
+ ListenableFuture result =
+ (ListenableFuture)
invocation.callRealMethod();
+ return result;
+}).when(spies.get(1)).getJournaledEdits(21,
QuorumJournalManager.QJM_RPC_MAX_TXNS_DEFAULT);
-GetJournaledEditsResponseProto responseProto = spies.get(2)
-.getJournaledEdits(21, 5000).get();
-assertEquals(20, responseProto.getTxnCount());
+
Mockito.doAnswer((Answer>)
invocation -> {
+ semaphore.acquire(2);
+ @SuppressWarnings("unchecked")
+ ListenableFuture result =
+ (ListenableFuture)
invocation.callRealMethod();
+ return result;
+}).when(spies.get(2)).getJournaledEdits(21,
+QuorumJournalManager.QJM_RPC_MAX_TXNS_DEFAULT);
Review Comment:
Nice!! Looks really good except for being pretty verbose. Can we create a
helper method like:
```
private void spyGetJournaledEdits(int jnSpyIdx, long fromTxId, Runnable
preHook) { ... }
```
Then here we can just have
```
spyGetJournaledEdits(0, 21, () -> semaphore.release(1));
spyGetJournaledEdits(1, 21, () -> semaphore.release(1));
spyGetJournaledEdits(2, 21, () -> semaphore.acquireUninterruptibly(2));
```
> JournalNode should throw NewerTxnIdException if SinceTxId is bigger than
> HighestWrittenTxId
> ---
>
> Key: HDFS-16659
> URL: https://issues.apache.org/jira/browse/HDFS-16659
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: ZanderXu
>Assignee: ZanderXu
>Priority: Critical
> Labels: pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
> JournalNode should throw `CacheMissException` if `sinceTxId` is bigger than
> `highestWrittenTxId` during handling `getJournaledEdits` rpc from NNs.
> Current logic may cause
[jira] [Commented] (HDFS-16659) JournalNode should throw NewerTxnIdException if SinceTxId is bigger than HighestWrittenTxId
[
https://issues.apache.org/jira/browse/HDFS-16659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17599203#comment-17599203
]
ASF GitHub Bot commented on HDFS-16659:
---
xkrogen commented on code in PR #4560:
URL: https://github.com/apache/hadoop/pull/4560#discussion_r961188653
##
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/qjournal/client/TestQuorumJournalManager.java:
##
@@ -1101,6 +1113,59 @@ public void testSelectViaRpcTwoJNsError() throws
Exception {
}
}
+ /**
+ * Test selecting EditLogInputStream after some journalNode jitter.
+ * And the corner case as below:
+ * 1. Journal 0 has some abnormal cases when journaling Edits with start
txId 11.
+ * 2. NameNode just ignore the abnormal journal 0 and continue to write
Edits to Journal 1 and 2.
+ * 3. Journal 0 backed to health.
+ * 4. Observer NameNode try to select EditLogInputStream vis PRC with start
txId 21.
+ * 5. Journal 1 has some abnormal cases caused slow response.
+ *
+ * And the expected selecting result is: Response should contain 20 Edits
from txId 21 to txId 40.
+ * Because there is no Edits from id 21 to 40 in the cache of JournalNode0.
Review Comment:
re: (1), sorry, I had my editor set to 80 chars instead of 100. line length
is fine.
Current diff doesn't address fix (2), and I don't think we really made
progress on (3). I am trying to say that in the steps 1~5, we never talk about
writing txns above 21, but then you say that the the expected result should
have txn 21 - 40. Since we never mentioned writing txns above 21, I would
expect that the result doesn't contain any txns higher than 21 ... So we need
to add more steps when describing the case to show when we were all of the txns.
> JournalNode should throw NewerTxnIdException if SinceTxId is bigger than
> HighestWrittenTxId
> ---
>
> Key: HDFS-16659
> URL: https://issues.apache.org/jira/browse/HDFS-16659
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: ZanderXu
>Assignee: ZanderXu
>Priority: Critical
> Labels: pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
> JournalNode should throw `CacheMissException` if `sinceTxId` is bigger than
> `highestWrittenTxId` during handling `getJournaledEdits` rpc from NNs.
> Current logic may cause in-progress EditlogTailer cannot replay any Edits
> from JournalNodes in some corner cases, resulting in ObserverNameNode cannot
> handle requests from clients.
> Suppose there are 3 journalNodes, JN0 ~ JN1.
> * JN0 has some abnormal cases when Active Namenode is syncing 10 Edits with
> first txid 11
> * NameNode just ignore the abnormal JN0 and continue to sync Edits to Journal
> 1 and 2
> * JN0 backed to health
> * NameNode continue sync 10 Edits with first txid 21.
> * At this point, there are no Edits 11 ~ 30 in the cache of JN0
> * Observer NameNode try to select EditLogInputStream through
> `getJournaledEdits` with since txId 21
> * Journal 2 has some abnormal cases and caused a slow response
> The expected result is: Response should contain 20 Edits from txId 21 to txId
> 30 from JN1 and JN2. Because Active NameNode successfully write these Edits
> to JN1 and JN2 and failed write these edits to JN0.
> But in the current implementation, the response is [Response(0) from JN0,
> Response(10) from JN1], because there are some abnormal cases in JN2, such
> as GC, bad network, cause a slow response. So the `maxAllowedTxns` will be
> 0, NameNode will not replay any Edits.
> As above, the root case is that JournalNode should throw Miss Cache Exception
> when `sinceTxid` is more than `highestWrittenTxId`.
> And the bug code as blew:
> {code:java}
> if (sinceTxId > getHighestWrittenTxId()) {
> // Requested edits that don't exist yet; short-circuit the cache here
> metrics.rpcEmptyResponses.incr();
> return
> GetJournaledEditsResponseProto.newBuilder().setTxnCount(0).build();
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16659) JournalNode should throw NewerTxnIdException if SinceTxId is bigger than HighestWrittenTxId
[
https://issues.apache.org/jira/browse/HDFS-16659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17599201#comment-17599201
]
ASF GitHub Bot commented on HDFS-16659:
---
xkrogen commented on code in PR #4560:
URL: https://github.com/apache/hadoop/pull/4560#discussion_r961189978
##
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/qjournal/client/TestQuorumJournalManager.java:
##
@@ -1101,6 +1113,59 @@ public void testSelectViaRpcTwoJNsError() throws
Exception {
}
}
+ /**
+ * Test selecting EditLogInputStream after some journalNode jitter.
+ * And the corner case as below:
+ * 1. Journal 0 has some abnormal cases when journaling Edits with start
txId 11.
+ * 2. NameNode just ignore the abnormal journal 0 and continue to write
Edits to Journal 1 and 2.
+ * 3. Journal 0 backed to health.
+ * 4. Observer NameNode try to select EditLogInputStream vis PRC with start
txId 21.
+ * 5. Journal 1 has some abnormal cases caused slow response.
+ *
+ * And the expected selecting result is: Response should contain 20 Edits
from txId 21 to txId 40.
+ * Because there is no Edits from id 21 to 40 in the cache of JournalNode0.
Review Comment:
Can we just copy your description from the PR?
> Suppose there are 3 journalNodes, JN0 ~ JN1.
> JN0 has some abnormal cases when Active Namenode is syncing 10 Edits with
first txid 11
NameNode just ignore the abnormal JN0 and continue to sync Edits to Journal
1 and 2
JN0 backed to health
NameNode continue sync 10 Edits with first txid 21.
At this point, there are no Edits 11 ~ 30 in the cache of JN0
Observer NameNode try to select EditLogInputStream through getJournaledEdits
with since txId 21
Journal 2 has some abnormal cases and caused a slow response
The expected result is: Response should contain 20 Edits from txId 21 to
txId 30 from JN1 and JN2. Because Active NameNode successfully write these
Edits to JN1 and JN2 and failed write these edits to JN0.
> JournalNode should throw NewerTxnIdException if SinceTxId is bigger than
> HighestWrittenTxId
> ---
>
> Key: HDFS-16659
> URL: https://issues.apache.org/jira/browse/HDFS-16659
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: ZanderXu
>Assignee: ZanderXu
>Priority: Critical
> Labels: pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
> JournalNode should throw `CacheMissException` if `sinceTxId` is bigger than
> `highestWrittenTxId` during handling `getJournaledEdits` rpc from NNs.
> Current logic may cause in-progress EditlogTailer cannot replay any Edits
> from JournalNodes in some corner cases, resulting in ObserverNameNode cannot
> handle requests from clients.
> Suppose there are 3 journalNodes, JN0 ~ JN1.
> * JN0 has some abnormal cases when Active Namenode is syncing 10 Edits with
> first txid 11
> * NameNode just ignore the abnormal JN0 and continue to sync Edits to Journal
> 1 and 2
> * JN0 backed to health
> * NameNode continue sync 10 Edits with first txid 21.
> * At this point, there are no Edits 11 ~ 30 in the cache of JN0
> * Observer NameNode try to select EditLogInputStream through
> `getJournaledEdits` with since txId 21
> * Journal 2 has some abnormal cases and caused a slow response
> The expected result is: Response should contain 20 Edits from txId 21 to txId
> 30 from JN1 and JN2. Because Active NameNode successfully write these Edits
> to JN1 and JN2 and failed write these edits to JN0.
> But in the current implementation, the response is [Response(0) from JN0,
> Response(10) from JN1], because there are some abnormal cases in JN2, such
> as GC, bad network, cause a slow response. So the `maxAllowedTxns` will be
> 0, NameNode will not replay any Edits.
> As above, the root case is that JournalNode should throw Miss Cache Exception
> when `sinceTxid` is more than `highestWrittenTxId`.
> And the bug code as blew:
> {code:java}
> if (sinceTxId > getHighestWrittenTxId()) {
> // Requested edits that don't exist yet; short-circuit the cache here
> metrics.rpcEmptyResponses.incr();
> return
> GetJournaledEditsResponseProto.newBuilder().setTxnCount(0).build();
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16659) JournalNode should throw NewerTxnIdException if SinceTxId is bigger than HighestWrittenTxId
[
https://issues.apache.org/jira/browse/HDFS-16659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17599199#comment-17599199
]
ASF GitHub Bot commented on HDFS-16659:
---
xkrogen commented on code in PR #4560:
URL: https://github.com/apache/hadoop/pull/4560#discussion_r961188653
##
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/qjournal/client/TestQuorumJournalManager.java:
##
@@ -1101,6 +1113,59 @@ public void testSelectViaRpcTwoJNsError() throws
Exception {
}
}
+ /**
+ * Test selecting EditLogInputStream after some journalNode jitter.
+ * And the corner case as below:
+ * 1. Journal 0 has some abnormal cases when journaling Edits with start
txId 11.
+ * 2. NameNode just ignore the abnormal journal 0 and continue to write
Edits to Journal 1 and 2.
+ * 3. Journal 0 backed to health.
+ * 4. Observer NameNode try to select EditLogInputStream vis PRC with start
txId 21.
+ * 5. Journal 1 has some abnormal cases caused slow response.
+ *
+ * And the expected selecting result is: Response should contain 20 Edits
from txId 21 to txId 40.
+ * Because there is no Edits from id 21 to 40 in the cache of JournalNode0.
Review Comment:
Current diff doesn't address (1) or (2), and I don't think we really made
progress on (3). I am trying to say that in the steps 1~5, we never talk about
writing txns above 21, but then you say that the the expected result should
have txn 21 - 40. Since we never mentioned writing txns above 21, I would
expect that the result doesn't contain any txns higher than 21 ... So we need
to add more steps when describing the case to show when we were all of the txns.
> JournalNode should throw NewerTxnIdException if SinceTxId is bigger than
> HighestWrittenTxId
> ---
>
> Key: HDFS-16659
> URL: https://issues.apache.org/jira/browse/HDFS-16659
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: ZanderXu
>Assignee: ZanderXu
>Priority: Critical
> Labels: pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
> JournalNode should throw `CacheMissException` if `sinceTxId` is bigger than
> `highestWrittenTxId` during handling `getJournaledEdits` rpc from NNs.
> Current logic may cause in-progress EditlogTailer cannot replay any Edits
> from JournalNodes in some corner cases, resulting in ObserverNameNode cannot
> handle requests from clients.
> Suppose there are 3 journalNodes, JN0 ~ JN1.
> * JN0 has some abnormal cases when Active Namenode is syncing 10 Edits with
> first txid 11
> * NameNode just ignore the abnormal JN0 and continue to sync Edits to Journal
> 1 and 2
> * JN0 backed to health
> * NameNode continue sync 10 Edits with first txid 21.
> * At this point, there are no Edits 11 ~ 30 in the cache of JN0
> * Observer NameNode try to select EditLogInputStream through
> `getJournaledEdits` with since txId 21
> * Journal 2 has some abnormal cases and caused a slow response
> The expected result is: Response should contain 20 Edits from txId 21 to txId
> 30 from JN1 and JN2. Because Active NameNode successfully write these Edits
> to JN1 and JN2 and failed write these edits to JN0.
> But in the current implementation, the response is [Response(0) from JN0,
> Response(10) from JN1], because there are some abnormal cases in JN2, such
> as GC, bad network, cause a slow response. So the `maxAllowedTxns` will be
> 0, NameNode will not replay any Edits.
> As above, the root case is that JournalNode should throw Miss Cache Exception
> when `sinceTxid` is more than `highestWrittenTxId`.
> And the bug code as blew:
> {code:java}
> if (sinceTxId > getHighestWrittenTxId()) {
> // Requested edits that don't exist yet; short-circuit the cache here
> metrics.rpcEmptyResponses.incr();
> return
> GetJournaledEditsResponseProto.newBuilder().setTxnCount(0).build();
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16659) JournalNode should throw NewerTxnIdException if SinceTxId is bigger than HighestWrittenTxId
[
https://issues.apache.org/jira/browse/HDFS-16659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17599192#comment-17599192
]
ASF GitHub Bot commented on HDFS-16659:
---
ZanderXu commented on PR #4560:
URL: https://github.com/apache/hadoop/pull/4560#issuecomment-1234914937
@xkrogen Sir, sorry to ping you again. Please help me review this patch
again.
> JournalNode should throw NewerTxnIdException if SinceTxId is bigger than
> HighestWrittenTxId
> ---
>
> Key: HDFS-16659
> URL: https://issues.apache.org/jira/browse/HDFS-16659
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: ZanderXu
>Assignee: ZanderXu
>Priority: Critical
> Labels: pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
> JournalNode should throw `CacheMissException` if `sinceTxId` is bigger than
> `highestWrittenTxId` during handling `getJournaledEdits` rpc from NNs.
> Current logic may cause in-progress EditlogTailer cannot replay any Edits
> from JournalNodes in some corner cases, resulting in ObserverNameNode cannot
> handle requests from clients.
> Suppose there are 3 journalNodes, JN0 ~ JN1.
> * JN0 has some abnormal cases when Active Namenode is syncing 10 Edits with
> first txid 11
> * NameNode just ignore the abnormal JN0 and continue to sync Edits to Journal
> 1 and 2
> * JN0 backed to health
> * NameNode continue sync 10 Edits with first txid 21.
> * At this point, there are no Edits 11 ~ 30 in the cache of JN0
> * Observer NameNode try to select EditLogInputStream through
> `getJournaledEdits` with since txId 21
> * Journal 2 has some abnormal cases and caused a slow response
> The expected result is: Response should contain 20 Edits from txId 21 to txId
> 30 from JN1 and JN2. Because Active NameNode successfully write these Edits
> to JN1 and JN2 and failed write these edits to JN0.
> But in the current implementation, the response is [Response(0) from JN0,
> Response(10) from JN1], because there are some abnormal cases in JN2, such
> as GC, bad network, cause a slow response. So the `maxAllowedTxns` will be
> 0, NameNode will not replay any Edits.
> As above, the root case is that JournalNode should throw Miss Cache Exception
> when `sinceTxid` is more than `highestWrittenTxId`.
> And the bug code as blew:
> {code:java}
> if (sinceTxId > getHighestWrittenTxId()) {
> // Requested edits that don't exist yet; short-circuit the cache here
> metrics.rpcEmptyResponses.incr();
> return
> GetJournaledEditsResponseProto.newBuilder().setTxnCount(0).build();
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16659) JournalNode should throw NewerTxnIdException if SinceTxId is bigger than HighestWrittenTxId
[
https://issues.apache.org/jira/browse/HDFS-16659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17585276#comment-17585276
]
ASF GitHub Bot commented on HDFS-16659:
---
hadoop-yetus commented on PR #4560:
URL: https://github.com/apache/hadoop/pull/4560#issuecomment-1228293430
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 51s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 37m 56s | | trunk passed |
| +1 :green_heart: | compile | 1m 36s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 1m 30s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 18s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 45s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 22s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 36s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 34s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 11s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 17s | | the patch passed |
| +1 :green_heart: | compile | 1m 28s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 1m 28s | | the patch passed |
| +1 :green_heart: | compile | 1m 18s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 1m 18s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 57s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 19s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 53s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 30s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 26s | | the patch passed |
| +1 :green_heart: | shadedclient | 22m 33s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 239m 21s | | hadoop-hdfs in the patch
passed. |
| +1 :green_heart: | asflicense | 1m 3s | | The patch does not
generate ASF License warnings. |
| | | 347m 47s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4560/4/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4560 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 0a38b1c3eb4c 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 69e3d9e8f81581c3bf96ed4cf34bc4226a74586a |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4560/4/testReport/ |
| Max. process+thread count | 2936 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4560/4/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
> JournalNode
[jira] [Commented] (HDFS-16659) JournalNode should throw NewerTxnIdException if SinceTxId is bigger than HighestWrittenTxId
[
https://issues.apache.org/jira/browse/HDFS-16659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17585127#comment-17585127
]
ASF GitHub Bot commented on HDFS-16659:
---
ZanderXu commented on code in PR #4560:
URL: https://github.com/apache/hadoop/pull/4560#discussion_r955628327
##
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/qjournal/client/TestQuorumJournalManager.java:
##
@@ -1101,6 +1113,59 @@ public void testSelectViaRpcTwoJNsError() throws
Exception {
}
}
+ /**
+ * Test selecting EditLogInputStream after some journalNode jitter.
+ * And the corner case as below:
+ * 1. Journal 0 has some abnormal cases when journaling Edits with start
txId 11.
+ * 2. NameNode just ignore the abnormal journal 0 and continue to write
Edits to Journal 1 and 2.
+ * 3. Journal 0 backed to health.
+ * 4. Observer NameNode try to select EditLogInputStream vis PRC with start
txId 21.
+ * 5. Journal 1 has some abnormal cases caused slow response.
+ *
+ * And the expected selecting result is: Response should contain 20 Edits
from txId 21 to txId 40.
+ * Because there is no Edits from id 21 to 40 in the cache of JournalNode0.
+ */
+ @Test
+ public void testSelectViaRpcAfterJNJitter() throws Exception {
+EditLogOutputStream stm = qjm.startLogSegment(
+1, NameNodeLayoutVersion.CURRENT_LAYOUT_VERSION);
+SettableFuture slowLog = SettableFuture.create();
+Mockito.doReturn(slowLog).when(spies.get(0))
+.sendEdits(eq(1L), eq(11L), eq(1), Mockito.any());
+writeTxns(stm, 1, 10);
+writeTxns(stm, 11, 10);
+writeTxns(stm, 21, 10);
+writeTxns(stm, 31, 10);
+ListeningExecutorService service = MoreExecutors.listeningDecorator(
+Executors.newSingleThreadExecutor());
+Mockito.doAnswer(invocation -> service.submit(
+() -> {
+ ByteArrayOutputStream byteStream = new ByteArrayOutputStream();
+ EditLogFileOutputStream.writeHeader(
+ NameNodeLayoutVersion.CURRENT_LAYOUT_VERSION,
+ new DataOutputStream(byteStream));
+ byteStream.write(createTxnData(21, 20));
+ Thread.sleep(3000);
Review Comment:
Nice suggestion, I learned a lot. Thanks, Sir.
##
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/qjournal/client/TestQuorumJournalManager.java:
##
@@ -1101,6 +1113,59 @@ public void testSelectViaRpcTwoJNsError() throws
Exception {
}
}
+ /**
+ * Test selecting EditLogInputStream after some journalNode jitter.
+ * And the corner case as below:
+ * 1. Journal 0 has some abnormal cases when journaling Edits with start
txId 11.
+ * 2. NameNode just ignore the abnormal journal 0 and continue to write
Edits to Journal 1 and 2.
+ * 3. Journal 0 backed to health.
+ * 4. Observer NameNode try to select EditLogInputStream vis PRC with start
txId 21.
+ * 5. Journal 1 has some abnormal cases caused slow response.
+ *
+ * And the expected selecting result is: Response should contain 20 Edits
from txId 21 to txId 40.
+ * Because there is no Edits from id 21 to 40 in the cache of JournalNode0.
+ */
+ @Test
+ public void testSelectViaRpcAfterJNJitter() throws Exception {
+EditLogOutputStream stm = qjm.startLogSegment(
+1, NameNodeLayoutVersion.CURRENT_LAYOUT_VERSION);
+SettableFuture slowLog = SettableFuture.create();
+Mockito.doReturn(slowLog).when(spies.get(0))
+.sendEdits(eq(1L), eq(11L), eq(1), Mockito.any());
+writeTxns(stm, 1, 10);
+writeTxns(stm, 11, 10);
+writeTxns(stm, 21, 10);
+writeTxns(stm, 31, 10);
+ListeningExecutorService service = MoreExecutors.listeningDecorator(
+Executors.newSingleThreadExecutor());
+Mockito.doAnswer(invocation -> service.submit(
+() -> {
+ ByteArrayOutputStream byteStream = new ByteArrayOutputStream();
+ EditLogFileOutputStream.writeHeader(
+ NameNodeLayoutVersion.CURRENT_LAYOUT_VERSION,
+ new DataOutputStream(byteStream));
+ byteStream.write(createTxnData(21, 20));
+ Thread.sleep(3000);
+ return GetJournaledEditsResponseProto.newBuilder()
+ .setTxnCount(20)
+ .setEditLog(ByteString.copyFrom(byteStream.toByteArray()))
+ .build();
+})
+).when(spies.get(1)).getJournaledEdits(21,
+QuorumJournalManager.QJM_RPC_MAX_TXNS_DEFAULT);
+
+GetJournaledEditsResponseProto responseProto = spies.get(2)
+.getJournaledEdits(21, 5000).get();
+assertEquals(20, responseProto.getTxnCount());
Review Comment:
remove it.
> JournalNode should throw NewerTxnIdException if SinceTxId is bigger than
> HighestWrittenTxId
> ---
>
> Key: HDFS-16659
> URL:
[jira] [Commented] (HDFS-16659) JournalNode should throw NewerTxnIdException if SinceTxId is bigger than HighestWrittenTxId
[
https://issues.apache.org/jira/browse/HDFS-16659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17585125#comment-17585125
]
ASF GitHub Bot commented on HDFS-16659:
---
ZanderXu commented on code in PR #4560:
URL: https://github.com/apache/hadoop/pull/4560#discussion_r955627989
##
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/qjournal/client/TestQuorumJournalManager.java:
##
@@ -1101,6 +1113,59 @@ public void testSelectViaRpcTwoJNsError() throws
Exception {
}
}
+ /**
+ * Test selecting EditLogInputStream after some journalNode jitter.
+ * And the corner case as below:
+ * 1. Journal 0 has some abnormal cases when journaling Edits with start
txId 11.
+ * 2. NameNode just ignore the abnormal journal 0 and continue to write
Edits to Journal 1 and 2.
+ * 3. Journal 0 backed to health.
+ * 4. Observer NameNode try to select EditLogInputStream vis PRC with start
txId 21.
+ * 5. Journal 1 has some abnormal cases caused slow response.
+ *
+ * And the expected selecting result is: Response should contain 20 Edits
from txId 21 to txId 40.
+ * Because there is no Edits from id 21 to 40 in the cache of JournalNode0.
+ */
+ @Test
+ public void testSelectViaRpcAfterJNJitter() throws Exception {
Review Comment:
JN0 contains txns 1-10 and miss 11 ~ 40.
> JournalNode should throw NewerTxnIdException if SinceTxId is bigger than
> HighestWrittenTxId
> ---
>
> Key: HDFS-16659
> URL: https://issues.apache.org/jira/browse/HDFS-16659
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: ZanderXu
>Assignee: ZanderXu
>Priority: Critical
> Labels: pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
> JournalNode should throw `CacheMissException` if `sinceTxId` is bigger than
> `highestWrittenTxId` during handling `getJournaledEdits` rpc from NNs.
> Current logic may cause in-progress EditlogTailer cannot replay any Edits
> from JournalNodes in some corner cases, resulting in ObserverNameNode cannot
> handle requests from clients.
> Suppose there are 3 journalNodes, JN0 ~ JN1.
> * JN0 has some abnormal cases when Active Namenode is syncing 10 Edits with
> first txid 11
> * NameNode just ignore the abnormal JN0 and continue to sync Edits to Journal
> 1 and 2
> * JN0 backed to health
> * NameNode continue sync 10 Edits with first txid 21.
> * At this point, there are no Edits 11 ~ 30 in the cache of JN0
> * Observer NameNode try to select EditLogInputStream through
> `getJournaledEdits` with since txId 21
> * Journal 2 has some abnormal cases and caused a slow response
> The expected result is: Response should contain 20 Edits from txId 21 to txId
> 30 from JN1 and JN2. Because Active NameNode successfully write these Edits
> to JN1 and JN2 and failed write these edits to JN0.
> But in the current implementation, the response is [Response(0) from JN0,
> Response(10) from JN1], because there are some abnormal cases in JN2, such
> as GC, bad network, cause a slow response. So the `maxAllowedTxns` will be
> 0, NameNode will not replay any Edits.
> As above, the root case is that JournalNode should throw Miss Cache Exception
> when `sinceTxid` is more than `highestWrittenTxId`.
> And the bug code as blew:
> {code:java}
> if (sinceTxId > getHighestWrittenTxId()) {
> // Requested edits that don't exist yet; short-circuit the cache here
> metrics.rpcEmptyResponses.incr();
> return
> GetJournaledEditsResponseProto.newBuilder().setTxnCount(0).build();
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16659) JournalNode should throw NewerTxnIdException if SinceTxId is bigger than HighestWrittenTxId
[
https://issues.apache.org/jira/browse/HDFS-16659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17585128#comment-17585128
]
ASF GitHub Bot commented on HDFS-16659:
---
ZanderXu commented on code in PR #4560:
URL: https://github.com/apache/hadoop/pull/4560#discussion_r955628485
##
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/qjournal/client/TestQuorumJournalManager.java:
##
@@ -1101,6 +1113,59 @@ public void testSelectViaRpcTwoJNsError() throws
Exception {
}
}
+ /**
+ * Test selecting EditLogInputStream after some journalNode jitter.
+ * And the corner case as below:
+ * 1. Journal 0 has some abnormal cases when journaling Edits with start
txId 11.
+ * 2. NameNode just ignore the abnormal journal 0 and continue to write
Edits to Journal 1 and 2.
+ * 3. Journal 0 backed to health.
+ * 4. Observer NameNode try to select EditLogInputStream vis PRC with start
txId 21.
+ * 5. Journal 1 has some abnormal cases caused slow response.
+ *
+ * And the expected selecting result is: Response should contain 20 Edits
from txId 21 to txId 40.
+ * Because there is no Edits from id 21 to 40 in the cache of JournalNode0.
+ */
+ @Test
+ public void testSelectViaRpcAfterJNJitter() throws Exception {
+EditLogOutputStream stm = qjm.startLogSegment(
+1, NameNodeLayoutVersion.CURRENT_LAYOUT_VERSION);
+SettableFuture slowLog = SettableFuture.create();
+Mockito.doReturn(slowLog).when(spies.get(0))
+.sendEdits(eq(1L), eq(11L), eq(1), Mockito.any());
+writeTxns(stm, 1, 10);
+writeTxns(stm, 11, 10);
+writeTxns(stm, 21, 10);
+writeTxns(stm, 31, 10);
+ListeningExecutorService service = MoreExecutors.listeningDecorator(
+Executors.newSingleThreadExecutor());
+Mockito.doAnswer(invocation -> service.submit(
+() -> {
+ ByteArrayOutputStream byteStream = new ByteArrayOutputStream();
+ EditLogFileOutputStream.writeHeader(
+ NameNodeLayoutVersion.CURRENT_LAYOUT_VERSION,
+ new DataOutputStream(byteStream));
+ byteStream.write(createTxnData(21, 20));
+ Thread.sleep(3000);
+ return GetJournaledEditsResponseProto.newBuilder()
+ .setTxnCount(20)
+ .setEditLog(ByteString.copyFrom(byteStream.toByteArray()))
+ .build();
+})
+).when(spies.get(1)).getJournaledEdits(21,
+QuorumJournalManager.QJM_RPC_MAX_TXNS_DEFAULT);
Review Comment:
Nice suggestion. Thanks, Sir.
> JournalNode should throw NewerTxnIdException if SinceTxId is bigger than
> HighestWrittenTxId
> ---
>
> Key: HDFS-16659
> URL: https://issues.apache.org/jira/browse/HDFS-16659
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: ZanderXu
>Assignee: ZanderXu
>Priority: Critical
> Labels: pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
> JournalNode should throw `CacheMissException` if `sinceTxId` is bigger than
> `highestWrittenTxId` during handling `getJournaledEdits` rpc from NNs.
> Current logic may cause in-progress EditlogTailer cannot replay any Edits
> from JournalNodes in some corner cases, resulting in ObserverNameNode cannot
> handle requests from clients.
> Suppose there are 3 journalNodes, JN0 ~ JN1.
> * JN0 has some abnormal cases when Active Namenode is syncing 10 Edits with
> first txid 11
> * NameNode just ignore the abnormal JN0 and continue to sync Edits to Journal
> 1 and 2
> * JN0 backed to health
> * NameNode continue sync 10 Edits with first txid 21.
> * At this point, there are no Edits 11 ~ 30 in the cache of JN0
> * Observer NameNode try to select EditLogInputStream through
> `getJournaledEdits` with since txId 21
> * Journal 2 has some abnormal cases and caused a slow response
> The expected result is: Response should contain 20 Edits from txId 21 to txId
> 30 from JN1 and JN2. Because Active NameNode successfully write these Edits
> to JN1 and JN2 and failed write these edits to JN0.
> But in the current implementation, the response is [Response(0) from JN0,
> Response(10) from JN1], because there are some abnormal cases in JN2, such
> as GC, bad network, cause a slow response. So the `maxAllowedTxns` will be
> 0, NameNode will not replay any Edits.
> As above, the root case is that JournalNode should throw Miss Cache Exception
> when `sinceTxid` is more than `highestWrittenTxId`.
> And the bug code as blew:
> {code:java}
> if (sinceTxId > getHighestWrittenTxId()) {
> // Requested edits that don't exist yet; short-circuit the cache here
> metrics.rpcEmptyResponses.incr();
> return
> GetJournaledEditsResponseProto.newBuilder().setTxnCount(0).build();
> }
> {code}
--
[jira] [Commented] (HDFS-16659) JournalNode should throw NewerTxnIdException if SinceTxId is bigger than HighestWrittenTxId
[
https://issues.apache.org/jira/browse/HDFS-16659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17585126#comment-17585126
]
ASF GitHub Bot commented on HDFS-16659:
---
ZanderXu commented on code in PR #4560:
URL: https://github.com/apache/hadoop/pull/4560#discussion_r955628159
##
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/qjournal/client/TestQuorumJournalManager.java:
##
@@ -1101,6 +1113,59 @@ public void testSelectViaRpcTwoJNsError() throws
Exception {
}
}
+ /**
+ * Test selecting EditLogInputStream after some journalNode jitter.
+ * And the corner case as below:
+ * 1. Journal 0 has some abnormal cases when journaling Edits with start
txId 11.
+ * 2. NameNode just ignore the abnormal journal 0 and continue to write
Edits to Journal 1 and 2.
+ * 3. Journal 0 backed to health.
+ * 4. Observer NameNode try to select EditLogInputStream vis PRC with start
txId 21.
+ * 5. Journal 1 has some abnormal cases caused slow response.
+ *
+ * And the expected selecting result is: Response should contain 20 Edits
from txId 21 to txId 40.
+ * Because there is no Edits from id 21 to 40 in the cache of JournalNode0.
+ */
+ @Test
+ public void testSelectViaRpcAfterJNJitter() throws Exception {
+EditLogOutputStream stm = qjm.startLogSegment(
+1, NameNodeLayoutVersion.CURRENT_LAYOUT_VERSION);
+SettableFuture slowLog = SettableFuture.create();
+Mockito.doReturn(slowLog).when(spies.get(0))
+.sendEdits(eq(1L), eq(11L), eq(1), Mockito.any());
+writeTxns(stm, 1, 10);
+writeTxns(stm, 11, 10);
+writeTxns(stm, 21, 10);
+writeTxns(stm, 31, 10);
Review Comment:
```
// Successfully write these edits to JN0 ~ JN2
writeTxns(stm, 1, 10);
// Failed write these edits to JN0, but successfully write them to JN1 ~ JN2
writeTxns(stm, 11, 10);
// Successfully write these edits to JN1 ~ JN2
writeTxns(stm, 21, 20);
```
##
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/qjournal/client/TestQuorumJournalManager.java:
##
@@ -1101,6 +1113,59 @@ public void testSelectViaRpcTwoJNsError() throws
Exception {
}
}
+ /**
+ * Test selecting EditLogInputStream after some journalNode jitter.
+ * And the corner case as below:
+ * 1. Journal 0 has some abnormal cases when journaling Edits with start
txId 11.
+ * 2. NameNode just ignore the abnormal journal 0 and continue to write
Edits to Journal 1 and 2.
+ * 3. Journal 0 backed to health.
+ * 4. Observer NameNode try to select EditLogInputStream vis PRC with start
txId 21.
+ * 5. Journal 1 has some abnormal cases caused slow response.
+ *
+ * And the expected selecting result is: Response should contain 20 Edits
from txId 21 to txId 40.
+ * Because there is no Edits from id 21 to 40 in the cache of JournalNode0.
+ */
+ @Test
+ public void testSelectViaRpcAfterJNJitter() throws Exception {
+EditLogOutputStream stm = qjm.startLogSegment(
+1, NameNodeLayoutVersion.CURRENT_LAYOUT_VERSION);
+SettableFuture slowLog = SettableFuture.create();
+Mockito.doReturn(slowLog).when(spies.get(0))
+.sendEdits(eq(1L), eq(11L), eq(1), Mockito.any());
+writeTxns(stm, 1, 10);
+writeTxns(stm, 11, 10);
+writeTxns(stm, 21, 10);
+writeTxns(stm, 31, 10);
+ListeningExecutorService service = MoreExecutors.listeningDecorator(
+Executors.newSingleThreadExecutor());
Review Comment:
remove it
> JournalNode should throw NewerTxnIdException if SinceTxId is bigger than
> HighestWrittenTxId
> ---
>
> Key: HDFS-16659
> URL: https://issues.apache.org/jira/browse/HDFS-16659
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: ZanderXu
>Assignee: ZanderXu
>Priority: Critical
> Labels: pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
> JournalNode should throw `CacheMissException` if `sinceTxId` is bigger than
> `highestWrittenTxId` during handling `getJournaledEdits` rpc from NNs.
> Current logic may cause in-progress EditlogTailer cannot replay any Edits
> from JournalNodes in some corner cases, resulting in ObserverNameNode cannot
> handle requests from clients.
> Suppose there are 3 journalNodes, JN0 ~ JN1.
> * JN0 has some abnormal cases when Active Namenode is syncing 10 Edits with
> first txid 11
> * NameNode just ignore the abnormal JN0 and continue to sync Edits to Journal
> 1 and 2
> * JN0 backed to health
> * NameNode continue sync 10 Edits with first txid 21.
> * At this point, there are no Edits 11 ~ 30 in the cache of JN0
> * Observer NameNode try to select EditLogInputStream through
> `getJournaledEdits` with since txId 21
> * Journal 2 has some
[jira] [Commented] (HDFS-16659) JournalNode should throw NewerTxnIdException if SinceTxId is bigger than HighestWrittenTxId
[
https://issues.apache.org/jira/browse/HDFS-16659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17585123#comment-17585123
]
ASF GitHub Bot commented on HDFS-16659:
---
ZanderXu commented on code in PR #4560:
URL: https://github.com/apache/hadoop/pull/4560#discussion_r955627539
##
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/qjournal/server/Journal.java:
##
@@ -750,10 +750,13 @@ public GetJournaledEditsResponseProto
getJournaledEdits(long sinceTxId,
"is a requirement to fetch journaled edits via RPC. Please enable " +
"it via " + DFSConfigKeys.DFS_HA_TAILEDITS_INPROGRESS_KEY);
}
-if (sinceTxId > getHighestWrittenTxId()) {
- // Requested edits that don't exist yet; short-circuit the cache here
- metrics.rpcEmptyResponses.incr();
Review Comment:
recover it.
> JournalNode should throw NewerTxnIdException if SinceTxId is bigger than
> HighestWrittenTxId
> ---
>
> Key: HDFS-16659
> URL: https://issues.apache.org/jira/browse/HDFS-16659
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: ZanderXu
>Assignee: ZanderXu
>Priority: Critical
> Labels: pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
> JournalNode should throw `CacheMissException` if `sinceTxId` is bigger than
> `highestWrittenTxId` during handling `getJournaledEdits` rpc from NNs.
> Current logic may cause in-progress EditlogTailer cannot replay any Edits
> from JournalNodes in some corner cases, resulting in ObserverNameNode cannot
> handle requests from clients.
> Suppose there are 3 journalNodes, JN0 ~ JN1.
> * JN0 has some abnormal cases when Active Namenode is syncing 10 Edits with
> first txid 11
> * NameNode just ignore the abnormal JN0 and continue to sync Edits to Journal
> 1 and 2
> * JN0 backed to health
> * NameNode continue sync 10 Edits with first txid 21.
> * At this point, there are no Edits 11 ~ 30 in the cache of JN0
> * Observer NameNode try to select EditLogInputStream through
> `getJournaledEdits` with since txId 21
> * Journal 2 has some abnormal cases and caused a slow response
> The expected result is: Response should contain 20 Edits from txId 21 to txId
> 30 from JN1 and JN2. Because Active NameNode successfully write these Edits
> to JN1 and JN2 and failed write these edits to JN0.
> But in the current implementation, the response is [Response(0) from JN0,
> Response(10) from JN1], because there are some abnormal cases in JN2, such
> as GC, bad network, cause a slow response. So the `maxAllowedTxns` will be
> 0, NameNode will not replay any Edits.
> As above, the root case is that JournalNode should throw Miss Cache Exception
> when `sinceTxid` is more than `highestWrittenTxId`.
> And the bug code as blew:
> {code:java}
> if (sinceTxId > getHighestWrittenTxId()) {
> // Requested edits that don't exist yet; short-circuit the cache here
> metrics.rpcEmptyResponses.incr();
> return
> GetJournaledEditsResponseProto.newBuilder().setTxnCount(0).build();
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16659) JournalNode should throw NewerTxnIdException if SinceTxId is bigger than HighestWrittenTxId
[
https://issues.apache.org/jira/browse/HDFS-16659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17585124#comment-17585124
]
ASF GitHub Bot commented on HDFS-16659:
---
ZanderXu commented on code in PR #4560:
URL: https://github.com/apache/hadoop/pull/4560#discussion_r955627637
##
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/qjournal/server/Journal.java:
##
@@ -750,10 +750,13 @@ public GetJournaledEditsResponseProto
getJournaledEdits(long sinceTxId,
"is a requirement to fetch journaled edits via RPC. Please enable " +
"it via " + DFSConfigKeys.DFS_HA_TAILEDITS_INPROGRESS_KEY);
}
-if (sinceTxId > getHighestWrittenTxId()) {
- // Requested edits that don't exist yet; short-circuit the cache here
- metrics.rpcEmptyResponses.incr();
- return
GetJournaledEditsResponseProto.newBuilder().setTxnCount(0).build();
+long highestTxId = getHighestWrittenTxId();
+if (sinceTxId > highestTxId) {
+ // Requested edits that don't exist yet and is newer than highestTxId.
+ throw new NewerTxnIdException(
+ "Highest txn ID available in the journal is %d, but requested txns "
+
+ "staring at %d. Maybe the journal is not healthy, just skip it.",
Review Comment:
fix it.
> JournalNode should throw NewerTxnIdException if SinceTxId is bigger than
> HighestWrittenTxId
> ---
>
> Key: HDFS-16659
> URL: https://issues.apache.org/jira/browse/HDFS-16659
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: ZanderXu
>Assignee: ZanderXu
>Priority: Critical
> Labels: pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
> JournalNode should throw `CacheMissException` if `sinceTxId` is bigger than
> `highestWrittenTxId` during handling `getJournaledEdits` rpc from NNs.
> Current logic may cause in-progress EditlogTailer cannot replay any Edits
> from JournalNodes in some corner cases, resulting in ObserverNameNode cannot
> handle requests from clients.
> Suppose there are 3 journalNodes, JN0 ~ JN1.
> * JN0 has some abnormal cases when Active Namenode is syncing 10 Edits with
> first txid 11
> * NameNode just ignore the abnormal JN0 and continue to sync Edits to Journal
> 1 and 2
> * JN0 backed to health
> * NameNode continue sync 10 Edits with first txid 21.
> * At this point, there are no Edits 11 ~ 30 in the cache of JN0
> * Observer NameNode try to select EditLogInputStream through
> `getJournaledEdits` with since txId 21
> * Journal 2 has some abnormal cases and caused a slow response
> The expected result is: Response should contain 20 Edits from txId 21 to txId
> 30 from JN1 and JN2. Because Active NameNode successfully write these Edits
> to JN1 and JN2 and failed write these edits to JN0.
> But in the current implementation, the response is [Response(0) from JN0,
> Response(10) from JN1], because there are some abnormal cases in JN2, such
> as GC, bad network, cause a slow response. So the `maxAllowedTxns` will be
> 0, NameNode will not replay any Edits.
> As above, the root case is that JournalNode should throw Miss Cache Exception
> when `sinceTxid` is more than `highestWrittenTxId`.
> And the bug code as blew:
> {code:java}
> if (sinceTxId > getHighestWrittenTxId()) {
> // Requested edits that don't exist yet; short-circuit the cache here
> metrics.rpcEmptyResponses.incr();
> return
> GetJournaledEditsResponseProto.newBuilder().setTxnCount(0).build();
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16659) JournalNode should throw NewerTxnIdException if SinceTxId is bigger than HighestWrittenTxId
[
https://issues.apache.org/jira/browse/HDFS-16659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17584972#comment-17584972
]
ASF GitHub Bot commented on HDFS-16659:
---
xkrogen commented on code in PR #4560:
URL: https://github.com/apache/hadoop/pull/4560#discussion_r955266389
##
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/qjournal/server/Journal.java:
##
@@ -750,10 +750,13 @@ public GetJournaledEditsResponseProto
getJournaledEdits(long sinceTxId,
"is a requirement to fetch journaled edits via RPC. Please enable " +
"it via " + DFSConfigKeys.DFS_HA_TAILEDITS_INPROGRESS_KEY);
}
-if (sinceTxId > getHighestWrittenTxId()) {
- // Requested edits that don't exist yet; short-circuit the cache here
- metrics.rpcEmptyResponses.incr();
- return
GetJournaledEditsResponseProto.newBuilder().setTxnCount(0).build();
+long highestTxId = getHighestWrittenTxId();
+if (sinceTxId > highestTxId) {
+ // Requested edits that don't exist yet and is newer than highestTxId.
+ throw new NewerTxnIdException(
+ "Highest txn ID available in the journal is %d, but requested txns "
+
+ "staring at %d. Maybe the journal is not healthy, just skip it.",
Review Comment:
typo: `staring` -> `starting`
We expect this to happen more frequently when the JNs are healthy than when
they are unhealthy, right? Maybe we should remove the part about "Maybe the
journal is not healthy" ?
##
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/qjournal/server/JournaledEditsCache.java:
##
@@ -413,5 +413,4 @@ long getCacheMissAmount() {
}
}
-
Review Comment:
Can you undo this whitespace change please
##
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/qjournal/client/TestQuorumJournalManager.java:
##
@@ -53,6 +63,8 @@
import org.apache.hadoop.hdfs.qjournal.MiniJournalCluster;
import org.apache.hadoop.hdfs.qjournal.QJMTestUtil;
import
org.apache.hadoop.hdfs.qjournal.protocol.QJournalProtocolProtos.SegmentStateProto;
+import
+
org.apache.hadoop.hdfs.qjournal.protocol.QJournalProtocolProtos.GetJournaledEditsResponseProto;
Review Comment:
leave on 1 line, we ignore line length limitation for imports
##
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/qjournal/client/TestQuorumJournalManager.java:
##
@@ -1101,6 +1113,59 @@ public void testSelectViaRpcTwoJNsError() throws
Exception {
}
}
+ /**
+ * Test selecting EditLogInputStream after some journalNode jitter.
+ * And the corner case as below:
+ * 1. Journal 0 has some abnormal cases when journaling Edits with start
txId 11.
+ * 2. NameNode just ignore the abnormal journal 0 and continue to write
Edits to Journal 1 and 2.
+ * 3. Journal 0 backed to health.
+ * 4. Observer NameNode try to select EditLogInputStream vis PRC with start
txId 21.
+ * 5. Journal 1 has some abnormal cases caused slow response.
+ *
+ * And the expected selecting result is: Response should contain 20 Edits
from txId 21 to txId 40.
+ * Because there is no Edits from id 21 to 40 in the cache of JournalNode0.
+ */
+ @Test
+ public void testSelectViaRpcAfterJNJitter() throws Exception {
Review Comment:
This test passes even if I revert your production changes from `Journal`,
can you check again?
##
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/qjournal/client/TestQuorumJournalManager.java:
##
@@ -1101,6 +1113,59 @@ public void testSelectViaRpcTwoJNsError() throws
Exception {
}
}
+ /**
+ * Test selecting EditLogInputStream after some journalNode jitter.
+ * And the corner case as below:
+ * 1. Journal 0 has some abnormal cases when journaling Edits with start
txId 11.
+ * 2. NameNode just ignore the abnormal journal 0 and continue to write
Edits to Journal 1 and 2.
+ * 3. Journal 0 backed to health.
+ * 4. Observer NameNode try to select EditLogInputStream vis PRC with start
txId 21.
+ * 5. Journal 1 has some abnormal cases caused slow response.
+ *
+ * And the expected selecting result is: Response should contain 20 Edits
from txId 21 to txId 40.
+ * Because there is no Edits from id 21 to 40 in the cache of JournalNode0.
+ */
+ @Test
+ public void testSelectViaRpcAfterJNJitter() throws Exception {
+EditLogOutputStream stm = qjm.startLogSegment(
+1, NameNodeLayoutVersion.CURRENT_LAYOUT_VERSION);
+SettableFuture slowLog = SettableFuture.create();
+Mockito.doReturn(slowLog).when(spies.get(0))
+.sendEdits(eq(1L), eq(11L), eq(1), Mockito.any());
+writeTxns(stm, 1, 10);
+writeTxns(stm, 11, 10);
+writeTxns(stm, 21, 10);
+writeTxns(stm, 31, 10);
+ListeningExecutorService service = MoreExecutors.listeningDecorator(
+
[jira] [Commented] (HDFS-16659) JournalNode should throw NewerTxnIdException if SinceTxId is bigger than HighestWrittenTxId
[
https://issues.apache.org/jira/browse/HDFS-16659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17584712#comment-17584712
]
ASF GitHub Bot commented on HDFS-16659:
---
hadoop-yetus commented on PR #4560:
URL: https://github.com/apache/hadoop/pull/4560#issuecomment-1226954004
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 44s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 38m 30s | | trunk passed |
| +1 :green_heart: | compile | 1m 42s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 1m 30s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 18s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 46s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 25s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 41s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 34s | | trunk passed |
| +1 :green_heart: | shadedclient | 22m 54s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 20s | | the patch passed |
| +1 :green_heart: | compile | 1m 26s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 1m 26s | | the patch passed |
| +1 :green_heart: | compile | 1m 20s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 1m 20s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 59s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 25s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 55s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 30s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 23s | | the patch passed |
| +1 :green_heart: | shadedclient | 22m 22s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 237m 56s | | hadoop-hdfs in the patch
passed. |
| +1 :green_heart: | asflicense | 1m 6s | | The patch does not
generate ASF License warnings. |
| | | 347m 4s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4560/3/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4560 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux b6ef6639ef5c 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 08e0dae7ddf43a22e699a34b21dbe7e32755969c |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4560/3/testReport/ |
| Max. process+thread count | 3832 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4560/3/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
> JournalNode
