[jira] [Commented] (RATIS-214) Support timeout using withDeadlineAfter for the blockingStub(Grpc)
[ https://issues.apache.org/jira/browse/RATIS-214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16407409#comment-16407409 ] Tsz Wo Nicholas Sze commented on RATIS-214: --- +1 the 004 patch looks good. > Support timeout using withDeadlineAfter for the blockingStub(Grpc) > -- > > Key: RATIS-214 > URL: https://issues.apache.org/jira/browse/RATIS-214 > Project: Ratis > Issue Type: Task >Reporter: Lokesh Jain >Assignee: Lokesh Jain >Priority: Major > Attachments: RATIS-214.004.patch > > > This jira aims to add rpc request timeout for Grpc. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (RATIS-176) Log Appender should throw an Exception in case append entry size exceeds the maxBufferSize configured
[ https://issues.apache.org/jira/browse/RATIS-176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shashikant Banerjee updated RATIS-176: -- Priority: Minor (was: Major) > Log Appender should throw an Exception in case append entry size exceeds the > maxBufferSize configured > -- > > Key: RATIS-176 > URL: https://issues.apache.org/jira/browse/RATIS-176 > Project: Ratis > Issue Type: Bug >Reporter: Shashikant Banerjee >Assignee: Shashikant Banerjee >Priority: Minor > Attachments: RATIS-176.001.patch > > > LogAppender while adding append entry in LogEntryBuffer, checks whether the > total allocated for all entries does not exceed the maxBufferSize allocated. > In case, the size exceeds the limit ,entries are not added to the buffer but > no exception is thrown . This case needs to be handled. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (RATIS-176) Log Appender should throw an Exception in case append entry size exceeds the maxBufferSize configured
[
https://issues.apache.org/jira/browse/RATIS-176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16406525#comment-16406525
]
Hadoop QA commented on RATIS-176:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} docker {color} | {color:red} 2m
2s{color} | {color:red} Docker failed to build yetus/ratis:date2018-03-20.
{color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | RATIS-176 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12915344/RATIS-176.001.patch |
| Console output |
https://builds.apache.org/job/PreCommit-RATIS-Build/145/console |
| Powered by | Apache Yetus 0.5.0 http://yetus.apache.org |
This message was automatically generated.
> Log Appender should throw an Exception in case append entry size exceeds the
> maxBufferSize configured
> --
>
> Key: RATIS-176
> URL: https://issues.apache.org/jira/browse/RATIS-176
> Project: Ratis
> Issue Type: Bug
>Reporter: Shashikant Banerjee
>Assignee: Shashikant Banerjee
>Priority: Major
> Attachments: RATIS-176.001.patch
>
>
> LogAppender while adding append entry in LogEntryBuffer, checks whether the
> total allocated for all entries does not exceed the maxBufferSize allocated.
> In case, the size exceeds the limit ,entries are not added to the buffer but
> no exception is thrown . This case needs to be handled.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (RATIS-176) Log Appender should throw an Exception in case append entry size exceeds the maxBufferSize configured
[ https://issues.apache.org/jira/browse/RATIS-176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shashikant Banerjee updated RATIS-176: -- Attachment: RATIS-176.001.patch > Log Appender should throw an Exception in case append entry size exceeds the > maxBufferSize configured > -- > > Key: RATIS-176 > URL: https://issues.apache.org/jira/browse/RATIS-176 > Project: Ratis > Issue Type: Bug >Reporter: Shashikant Banerjee >Assignee: Shashikant Banerjee >Priority: Major > Attachments: RATIS-176.001.patch > > > LogAppender while adding append entry in LogEntryBuffer, checks whether the > total allocated for all entries does not exceed the maxBufferSize allocated. > In case, the size exceeds the limit ,entries are not added to the buffer but > no exception is thrown . This case needs to be handled. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (RATIS-209) StateMachine updater may miss writeLog Request after a new Leader is chosen.
[
https://issues.apache.org/jira/browse/RATIS-209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16406433#comment-16406433
]
Lokesh Jain commented on RATIS-209:
---
I think the issue is in Ratis. The appendEntries are async, therefore the
follower can append the entries in future and return the next index. In such a
case the leader will see that the entry has been added to the log for majority
of the followers and will commit it. But the follower may not have yet written
the state machine data, which leads to the null pointer exception.
I think we need to make sure that the state machine data has been written
before moving on to applyTransaction in state machine.
> StateMachine updater may miss writeLog Request after a new Leader is chosen.
>
>
> Key: RATIS-209
> URL: https://issues.apache.org/jira/browse/RATIS-209
> Project: Ratis
> Issue Type: Bug
>Affects Versions: 0.2.0-alpha
>Reporter: Mukul Kumar Singh
>Assignee: Mukul Kumar Singh
>Priority: Major
>
> This issue is happening on ozone for write chunk request.
> 1) currently write chunk request is processed in two phases, in the first
> phase the user data is written to the follower as part of
> {{writeStateMachineData}} and then the entry is committed to the follower as
> part of {{commit}}.
> 2) The issue which is hit right now is the case where a)
> {{writeStateMachineData}} didn't happen for a particular chunk however b) the
> commit entry is still processed. this leads to a case where a corresponding
> stateMachineFuture is not present in the hashmap.
> {code}
> 2018-02-12 00:26:30,097 INFO org.apache.ratis.server.impl.FollowerState:
> 172.26.32.228_9858 changes to CANDIDATE, lastRpcTime:1773,
> electionTimeout:873ms
> 2018-02-12 00:26:30,098 INFO org.apache.ratis.server.impl.RaftServerImpl:
> 172.26.32.228_9858 changes role from FOLLOWER to CANDIDATE at term 3 for
> changeToCandidate
> 2018-02-12 00:26:30,100 INFO org.apache.ratis.server.impl.RaftServerImpl:
> 172.26.32.228_9858: change Leader from 172.26.32.232_9858 to null at term 3
> for initElection
> 2018-02-12 00:26:32,869 INFO org.apache.ratis.server.impl.LeaderElection:
> 172.26.32.228_9858: begin an election in Term 4
> 2018-02-12 00:26:32,901 INFO
> org.apache.ratis.grpc.server.RaftServerProtocolService: 172.26.32.228_9858:
> appendEntries completed
> 2018-02-12 00:26:33,217 INFO org.apache.ratis.server.impl.LeaderElection:
> 172.26.32.228_9858: Election REJECTED; received 2 response(s)
> [172.26.32.228_9858<-172.26.32.230_9858#0:FAIL-t4,
> 172.26.32.228_9858<-172.26.32.232_9858#0:FAIL-t4] and 0 exception(s);
> 172.26.32.228_9858:t4, leader=null, voted=172.26.32.228_9858, raftlog=[(t:3,
> i:10711)], conf=[172.26.32.228_9858:172.26.32.228:9858,
> 172.26.32.230_9858:172.26.32.230:9858,
> 172.26.32.232_9858:172.26.32.232:9858], old=null
> 2018-02-12 00:26:33,217 INFO org.apache.ratis.server.impl.RaftServerImpl:
> 172.26.32.228_9858 changes role from CANDIDATE to FOLLOWER at term 4 for
> changeToFollower
> 2018-02-12 00:26:39,518 INFO org.apache.ratis.server.impl.FollowerState:
> 172.26.32.228_9858 changes to CANDIDATE, lastRpcTime:5624,
> electionTimeout:975ms
> 2018-02-12 00:26:39,518 INFO org.apache.ratis.server.impl.RaftServerImpl:
> 172.26.32.228_9858 changes role from FOLLOWER to CANDIDATE at term 5 for
> changeToCandidate
> 2018-02-12 00:26:39,518 INFO org.apache.ratis.server.impl.RaftServerImpl:
> 172.26.32.228_9858 changes role from CANDIDATE to FOLLOWER at term 5 for
> changeToFollower
> 2018-02-12 00:26:39,520 INFO org.apache.ratis.server.impl.RaftServerImpl:
> 172.26.32.228_9858: change Leader from null to 172.26.32.232_9858 at term 5
> for appendEntries
> {code}
> {code}
> 2018-02-12 00:31:12,400 ERROR
> org.apache.ratis.server.impl.StateMachineUpdater: Terminating with exit
> status 2: StateMachineUpdater-172.26.32.228_9858: the StateMachineUpdater
> hits Throwable
> java.lang.NullPointerException
> at
> org.apache.hadoop.ozone.container.common.transport.server.ratis.ContainerStateMachine.applyTransaction(ContainerStateMachine.java:254)
> at
> org.apache.ratis.server.impl.RaftServerImpl.applyLogToStateMachine(RaftServerImpl.java:1001)
> at
> org.apache.ratis.server.impl.StateMachineUpdater.run(StateMachineUpdater.java:151)
> at java.lang.Thread.run(Thread.java:745)
> 2018-02-12 00:31:12,406 INFO org.apache.hadoop.hdfs.server.datanode.DataNode:
> SHUTDOWN_MSG:
> /
> SHUTDOWN_MSG: Shutting down DataNode at
> y128.l42scl.hortonworks.com/172.26.32.228
> /
> *** shutting down gRPC server since JVM is shutting down
> *** server shut down
>
[jira] [Commented] (RATIS-214) Support timeout using withDeadlineAfter for the blockingStub(Grpc)
[
https://issues.apache.org/jira/browse/RATIS-214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16406233#comment-16406233
]
Hadoop QA commented on RATIS-214:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m
32s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
21s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
57s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
43s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
21s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
36s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
5s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 18s{color} | {color:orange} root: The patch generated 16 new + 268 unchanged
- 0 fixed = 284 total (was 268) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 6m 4s{color}
| {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
11s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 15m 25s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | ratis.server.simulation.TestRaftWithSimulatedRpc |
| | ratis.server.impl.TestRaftServerJmx |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/ratis:date2018-03-20
|
| JIRA Issue | RATIS-214 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12915307/RATIS-214.004.patch |
| Optional Tests | asflicense javac javadoc unit findbugs checkstyle
compile |
| uname | Linux 51856aba40b0 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-RATIS-Build/yetus-personality.sh
|
| git revision | master / cbfa28a |
| Default Java | 1.8.0_151 |
| checkstyle |
https://builds.apache.org/job/PreCommit-RATIS-Build/144/artifact/out/diff-checkstyle-root.txt
|
| unit |
https://builds.apache.org/job/PreCommit-RATIS-Build/144/artifact/out/patch-unit-root.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-RATIS-Build/144/testReport/ |
| modules | C: ratis-client ratis-server ratis-grpc U: . |
| Console output |
https://builds.apache.org/job/PreCommit-RATIS-Build/144/console |
| Powered by | Apache Yetus 0.5.0 http://yetus.apache.org |
This message was automatically generated.
> Support timeout using withDeadlineAfter for the blockingStub(Grpc)
> --
>
> Key: RATIS-214
> URL: https://issues.apache.org/jira/browse/RATIS-214
> Project: Ratis
> Issue Type: Task
>Reporter: Lokesh Jain
>Assignee: Lokesh Jain
>
[jira] [Commented] (RATIS-214) Support timeout using withDeadlineAfter for the blockingStub(Grpc)
[ https://issues.apache.org/jira/browse/RATIS-214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16406198#comment-16406198 ] Lokesh Jain commented on RATIS-214: --- [~szetszwo] Thanks for reviewing the patch! v4 patch addresses your comments. > Support timeout using withDeadlineAfter for the blockingStub(Grpc) > -- > > Key: RATIS-214 > URL: https://issues.apache.org/jira/browse/RATIS-214 > Project: Ratis > Issue Type: Task >Reporter: Lokesh Jain >Assignee: Lokesh Jain >Priority: Major > Attachments: RATIS-214.004.patch > > > This jira aims to add rpc request timeout for Grpc. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (RATIS-214) Support timeout using withDeadlineAfter for the blockingStub(Grpc)
[ https://issues.apache.org/jira/browse/RATIS-214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated RATIS-214: -- Attachment: RATIS-214.004.patch > Support timeout using withDeadlineAfter for the blockingStub(Grpc) > -- > > Key: RATIS-214 > URL: https://issues.apache.org/jira/browse/RATIS-214 > Project: Ratis > Issue Type: Task >Reporter: Lokesh Jain >Assignee: Lokesh Jain >Priority: Major > Attachments: RATIS-214.004.patch > > > This jira aims to add rpc request timeout for Grpc. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (RATIS-214) Support timeout using withDeadlineAfter for the blockingStub(Grpc)
[ https://issues.apache.org/jira/browse/RATIS-214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated RATIS-214: -- Attachment: (was: RATIS-214.002.patch) > Support timeout using withDeadlineAfter for the blockingStub(Grpc) > -- > > Key: RATIS-214 > URL: https://issues.apache.org/jira/browse/RATIS-214 > Project: Ratis > Issue Type: Task >Reporter: Lokesh Jain >Assignee: Lokesh Jain >Priority: Major > Attachments: RATIS-214.004.patch > > > This jira aims to add rpc request timeout for Grpc. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (RATIS-214) Support timeout using withDeadlineAfter for the blockingStub(Grpc)
[ https://issues.apache.org/jira/browse/RATIS-214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated RATIS-214: -- Attachment: (was: RATIS-214.003.patch) > Support timeout using withDeadlineAfter for the blockingStub(Grpc) > -- > > Key: RATIS-214 > URL: https://issues.apache.org/jira/browse/RATIS-214 > Project: Ratis > Issue Type: Task >Reporter: Lokesh Jain >Assignee: Lokesh Jain >Priority: Major > Attachments: RATIS-214.004.patch > > > This jira aims to add rpc request timeout for Grpc. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (RATIS-214) Support timeout using withDeadlineAfter for the blockingStub(Grpc)
[ https://issues.apache.org/jira/browse/RATIS-214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated RATIS-214: -- Attachment: (was: RATIS-214.001.patch) > Support timeout using withDeadlineAfter for the blockingStub(Grpc) > -- > > Key: RATIS-214 > URL: https://issues.apache.org/jira/browse/RATIS-214 > Project: Ratis > Issue Type: Task >Reporter: Lokesh Jain >Assignee: Lokesh Jain >Priority: Major > Attachments: RATIS-214.002.patch, RATIS-214.003.patch > > > This jira aims to add rpc request timeout for Grpc. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
