[jira] [Updated] (RATIS-291) Raft Server should fail themselves when a raft storage directory fails
[ https://issues.apache.org/jira/browse/RATIS-291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated RATIS-291: -- Attachment: RATIS-291.02_commit.patch > Raft Server should fail themselves when a raft storage directory fails > -- > > Key: RATIS-291 > URL: https://issues.apache.org/jira/browse/RATIS-291 > Project: Ratis > Issue Type: Bug > Components: server >Reporter: Mukul Kumar Singh >Assignee: Shashikant Banerjee >Priority: Major > Labels: ozone > Fix For: 0.3.0 > > Attachments: RATIS-291.02.patch, RATIS-291.02_commit.patch > > > A Raft server uses a storage directory to store the write ahead log. If this > log is lost because of a reason, then this node should fail itself. > For a follower, if raft log location has failed, then the follower will not > be able to append any entries. This node will now be lagging behind the > follower and will eventually be notified via notifySlowness. > For a leader where the raft log disk has failed, the leader will not append > any new entries to its log. However with respect to the raft ring, the leader > will still remain healthy. This jira proposes to add a new api to identify a > leader with failed node. > Also this jira also proposes to add a new api to the statemachine, so that > state machine implementation can provide methods to verify the raft log > location. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (RATIS-291) Raft Server should fail themselves when a raft storage directory fails
[ https://issues.apache.org/jira/browse/RATIS-291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16590895#comment-16590895 ] Tsz Wo Nicholas Sze commented on RATIS-291: --- RATIS-291.02_commit.patch: to be committed. > Raft Server should fail themselves when a raft storage directory fails > -- > > Key: RATIS-291 > URL: https://issues.apache.org/jira/browse/RATIS-291 > Project: Ratis > Issue Type: Bug > Components: server >Reporter: Mukul Kumar Singh >Assignee: Shashikant Banerjee >Priority: Major > Labels: ozone > Fix For: 0.3.0 > > Attachments: RATIS-291.02.patch, RATIS-291.02_commit.patch > > > A Raft server uses a storage directory to store the write ahead log. If this > log is lost because of a reason, then this node should fail itself. > For a follower, if raft log location has failed, then the follower will not > be able to append any entries. This node will now be lagging behind the > follower and will eventually be notified via notifySlowness. > For a leader where the raft log disk has failed, the leader will not append > any new entries to its log. However with respect to the raft ring, the leader > will still remain healthy. This jira proposes to add a new api to identify a > leader with failed node. > Also this jira also proposes to add a new api to the statemachine, so that > state machine implementation can provide methods to verify the raft log > location. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (RATIS-291) Raft Server should fail themselves when a raft storage directory fails
[ https://issues.apache.org/jira/browse/RATIS-291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated RATIS-291: -- Affects Version/s: (was: 0.3.0) > Raft Server should fail themselves when a raft storage directory fails > -- > > Key: RATIS-291 > URL: https://issues.apache.org/jira/browse/RATIS-291 > Project: Ratis > Issue Type: Bug > Components: server >Reporter: Mukul Kumar Singh >Assignee: Shashikant Banerjee >Priority: Major > Labels: ozone > Fix For: 0.3.0 > > Attachments: RATIS-291.02.patch > > > A Raft server uses a storage directory to store the write ahead log. If this > log is lost because of a reason, then this node should fail itself. > For a follower, if raft log location has failed, then the follower will not > be able to append any entries. This node will now be lagging behind the > follower and will eventually be notified via notifySlowness. > For a leader where the raft log disk has failed, the leader will not append > any new entries to its log. However with respect to the raft ring, the leader > will still remain healthy. This jira proposes to add a new api to identify a > leader with failed node. > Also this jira also proposes to add a new api to the statemachine, so that > state machine implementation can provide methods to verify the raft log > location. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (RATIS-291) Raft Server should fail themselves when a raft storage directory fails
[ https://issues.apache.org/jira/browse/RATIS-291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16590893#comment-16590893 ] Tsz Wo Nicholas Sze commented on RATIS-291: --- +1 the 002 patch looks good. Since the new stepDownLeader() method is only used once and the code is already synchronized in appendTransaction(..), I will get rid of stepDownLeader() when committing the patch. > Raft Server should fail themselves when a raft storage directory fails > -- > > Key: RATIS-291 > URL: https://issues.apache.org/jira/browse/RATIS-291 > Project: Ratis > Issue Type: Bug > Components: server >Affects Versions: 0.3.0 >Reporter: Mukul Kumar Singh >Assignee: Shashikant Banerjee >Priority: Major > Labels: ozone > Fix For: 0.3.0 > > Attachments: RATIS-291.02.patch > > > A Raft server uses a storage directory to store the write ahead log. If this > log is lost because of a reason, then this node should fail itself. > For a follower, if raft log location has failed, then the follower will not > be able to append any entries. This node will now be lagging behind the > follower and will eventually be notified via notifySlowness. > For a leader where the raft log disk has failed, the leader will not append > any new entries to its log. However with respect to the raft ring, the leader > will still remain healthy. This jira proposes to add a new api to identify a > leader with failed node. > Also this jira also proposes to add a new api to the statemachine, so that > state machine implementation can provide methods to verify the raft log > location. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (RATIS-272) Design ideal API
[
https://issues.apache.org/jira/browse/RATIS-272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16590818#comment-16590818
]
ASF GitHub Bot commented on RATIS-272:
--
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/incubator-ratis/pull/4#discussion_r212458341
--- Diff:
ratis-logservice/src/main/java/org/apache/ratis/logservice/api/LogStream.java
---
@@ -0,0 +1,109 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.ratis.logservice.api;
+
+import java.util.concurrent.CompletableFuture;
+
+/**
+ * A distributed log with "infinite" length that supports reads and writes.
+ */
+public interface LogStream {
+
+ /**
+ * Returns the unique name to identify this log.
+ */
+ LogName getName();
+
+ /**
+ * Returns the size of this LogStream in bytes.
+ */
+ long getSizeInBytes();
+
+ /**
+ * Returns the number of records in this LogStream.
+ */
+ long getSizeInRecords();
+
+ /**
+ * Creates a reader to read this LogStream asynchronously.
+ *
+ * @return An asynchronous reader
+ */
+ AsyncLogReader createAsyncReader();
+
+ /**
+ * Creates a writer to write to this LogStream asynchronously.
+ *
+ * @return An asynchronous writer
+ */
+ AsyncLogWriter createAsyncWriter();
+
+ /**
+ * Creates a reader to read this LogStream synchronously.
+ *
+ * @return A synchronous reader
+ */
+ LogReader createReader();
+
+ /**
+ * Creates a write to write to this LogStream synchronously.
+ *
+ * @return A synchronous writer
+ */
+ LogWriter createWriter();
+
+ /**
+ * Removes the elements in this LogStream prior to the given recordId.
+ *
+ * @param recordId A non-negative recordId for this LogStream
+ */
+ CompletableFuture truncateBefore(long recordId);
--- End diff --
A LogStream is a sequence of entries: and RecordId is
monotonically increasing. Truncation is the removal of entries from the "head"
of the stream.
e.g. if we have a log with recordIds from `[0,100]`, we might say
`truncateBefore(10, false)` which would remove entries 0, 1, 2, ..., 9. The
"head" of the log would then be `recordId=10`
> Design ideal API
>
>
> Key: RATIS-272
> URL: https://issues.apache.org/jira/browse/RATIS-272
> Project: Ratis
> Issue Type: Sub-task
> Components: LogService
>Reporter: Josh Elser
>Priority: Major
>
> With influence from Apache DistributedLog, Kafka, and BookKeeper, design an
> API that balances the ideal notion of what a distribute log system should
> look like, but also considers the needs of HBase to replace a WAL.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (RATIS-291) Raft Server should fail themselves when a raft storage directory fails
[
https://issues.apache.org/jira/browse/RATIS-291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16590713#comment-16590713
]
Hadoop QA commented on RATIS-291:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 4m
10s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
58s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
20s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
36s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 15s{color} | {color:orange} root: The patch generated 2 new + 333 unchanged
- 0 fixed = 335 total (was 333) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 15m 12s{color}
| {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 27m 24s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | ratis.server.simulation.TestRaftWithSimulatedRpc |
| | ratis.TestRaftServerLeaderElectionTimeout |
| | ratis.TestRaftServerSlownessDetection |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/ratis:date2018-08-23
|
| JIRA Issue | RATIS-291 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12936868/RATIS-291.02.patch |
| Optional Tests | asflicense javac javadoc unit findbugs checkstyle
compile |
| uname | Linux 98b90ec6b5c1 4.4.0-133-generic #159-Ubuntu SMP Fri Aug 10
07:31:43 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-RATIS-Build/yetus-personality.sh
|
| git revision | master / 0581246 |
| Default Java | 1.8.0_181 |
| checkstyle |
https://builds.apache.org/job/PreCommit-RATIS-Build/311/artifact/out/diff-checkstyle-root.txt
|
| unit |
https://builds.apache.org/job/PreCommit-RATIS-Build/311/artifact/out/patch-unit-root.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-RATIS-Build/311/testReport/ |
| modules | C: ratis-server U: ratis-server |
| Console output |
https://builds.apache.org/job/PreCommit-RATIS-Build/311/console |
| Powered by | Apache Yetus 0.5.0 http://yetus.apache.org |
This message was automatically generated.
> Raft Server should fail themselves when a raft storage directory fails
> --
>
> Key: RATIS-291
> URL: https://issues.apache.org/jira/browse/RATIS-291
> Project: Ratis
> Issue Type: Bug
> Components: server
>Affects Versions: 0.3.0
>Reporter: Mukul Kumar Singh
>Assignee: Shashikant Banerjee
>Priority: Major
> Labels: ozone
>
[jira] [Updated] (RATIS-291) Raft Server should fail themselves when a raft storage directory fails
[ https://issues.apache.org/jira/browse/RATIS-291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shashikant Banerjee updated RATIS-291: -- Attachment: RATIS-291.02.patch > Raft Server should fail themselves when a raft storage directory fails > -- > > Key: RATIS-291 > URL: https://issues.apache.org/jira/browse/RATIS-291 > Project: Ratis > Issue Type: Bug > Components: server >Affects Versions: 0.3.0 >Reporter: Mukul Kumar Singh >Assignee: Shashikant Banerjee >Priority: Major > Labels: ozone > Fix For: 0.3.0 > > Attachments: RATIS-291.02.patch > > > A Raft server uses a storage directory to store the write ahead log. If this > log is lost because of a reason, then this node should fail itself. > For a follower, if raft log location has failed, then the follower will not > be able to append any entries. This node will now be lagging behind the > follower and will eventually be notified via notifySlowness. > For a leader where the raft log disk has failed, the leader will not append > any new entries to its log. However with respect to the raft ring, the leader > will still remain healthy. This jira proposes to add a new api to identify a > leader with failed node. > Also this jira also proposes to add a new api to the statemachine, so that > state machine implementation can provide methods to verify the raft log > location. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (RATIS-291) Raft Server should fail themselves when a raft storage directory fails
[ https://issues.apache.org/jira/browse/RATIS-291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16590680#comment-16590680 ] Shashikant Banerjee commented on RATIS-291: --- Thanks [~szetszwo], for the review. I think its really not required to step down the leader in case the server already getting terminated. Updated patch v2. > Raft Server should fail themselves when a raft storage directory fails > -- > > Key: RATIS-291 > URL: https://issues.apache.org/jira/browse/RATIS-291 > Project: Ratis > Issue Type: Bug > Components: server >Affects Versions: 0.3.0 >Reporter: Mukul Kumar Singh >Assignee: Shashikant Banerjee >Priority: Major > Labels: ozone > Fix For: 0.3.0 > > Attachments: RATIS-291.02.patch > > > A Raft server uses a storage directory to store the write ahead log. If this > log is lost because of a reason, then this node should fail itself. > For a follower, if raft log location has failed, then the follower will not > be able to append any entries. This node will now be lagging behind the > follower and will eventually be notified via notifySlowness. > For a leader where the raft log disk has failed, the leader will not append > any new entries to its log. However with respect to the raft ring, the leader > will still remain healthy. This jira proposes to add a new api to identify a > leader with failed node. > Also this jira also proposes to add a new api to the statemachine, so that > state machine implementation can provide methods to verify the raft log > location. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (RATIS-291) Raft Server should fail themselves when a raft storage directory fails
[ https://issues.apache.org/jira/browse/RATIS-291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shashikant Banerjee updated RATIS-291: -- Attachment: (was: RATIS-291.01.patch) > Raft Server should fail themselves when a raft storage directory fails > -- > > Key: RATIS-291 > URL: https://issues.apache.org/jira/browse/RATIS-291 > Project: Ratis > Issue Type: Bug > Components: server >Affects Versions: 0.3.0 >Reporter: Mukul Kumar Singh >Assignee: Shashikant Banerjee >Priority: Major > Labels: ozone > Fix For: 0.3.0 > > Attachments: RATIS-291.02.patch > > > A Raft server uses a storage directory to store the write ahead log. If this > log is lost because of a reason, then this node should fail itself. > For a follower, if raft log location has failed, then the follower will not > be able to append any entries. This node will now be lagging behind the > follower and will eventually be notified via notifySlowness. > For a leader where the raft log disk has failed, the leader will not append > any new entries to its log. However with respect to the raft ring, the leader > will still remain healthy. This jira proposes to add a new api to identify a > leader with failed node. > Also this jira also proposes to add a new api to the statemachine, so that > state machine implementation can provide methods to verify the raft log > location. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (RATIS-291) Raft Server should fail themselves when a raft storage directory fails
[ https://issues.apache.org/jira/browse/RATIS-291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16590620#comment-16590620 ] Tsz Wo Nicholas Sze commented on RATIS-291: --- Thanks [~shashikant]. Question: In RaftLogWorker, we are going to call ExitUtils.terminate(..). Is it still useful to stepDownLeader()? > Raft Server should fail themselves when a raft storage directory fails > -- > > Key: RATIS-291 > URL: https://issues.apache.org/jira/browse/RATIS-291 > Project: Ratis > Issue Type: Bug > Components: server >Affects Versions: 0.3.0 >Reporter: Mukul Kumar Singh >Assignee: Shashikant Banerjee >Priority: Major > Labels: ozone > Fix For: 0.3.0 > > Attachments: RATIS-291.01.patch > > > A Raft server uses a storage directory to store the write ahead log. If this > log is lost because of a reason, then this node should fail itself. > For a follower, if raft log location has failed, then the follower will not > be able to append any entries. This node will now be lagging behind the > follower and will eventually be notified via notifySlowness. > For a leader where the raft log disk has failed, the leader will not append > any new entries to its log. However with respect to the raft ring, the leader > will still remain healthy. This jira proposes to add a new api to identify a > leader with failed node. > Also this jira also proposes to add a new api to the statemachine, so that > state machine implementation can provide methods to verify the raft log > location. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (RATIS-300) Support multiple RaftServerImpl in RaftServerProxy
[
https://issues.apache.org/jira/browse/RATIS-300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16590580#comment-16590580
]
Hadoop QA commented on RATIS-300:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m
25s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 16 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
11s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
31s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
5s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
53s{color} | {color:red} root in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
49s{color} | {color:red} root in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 49s{color}
| {color:red} root in the patch failed. {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 28s{color} | {color:orange} root: The patch generated 124 new + 905
unchanged - 39 fixed = 1029 total (was 944) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 15m 6s{color}
| {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
7s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 4s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/ratis:date2018-08-23
|
| JIRA Issue | RATIS-300 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12936720/r300_20180822.patch |
| Optional Tests | asflicense javac javadoc unit findbugs checkstyle
compile |
| uname | Linux 035e9c685603 3.13.0-153-generic #203-Ubuntu SMP Thu Jun 14
08:52:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-RATIS-Build/yetus-personality.sh
|
| git revision | master / 0581246 |
| Default Java | 1.8.0_181 |
| mvninstall |
https://builds.apache.org/job/PreCommit-RATIS-Build/310/artifact/out/patch-mvninstall-root.txt
|
| compile |
https://builds.apache.org/job/PreCommit-RATIS-Build/310/artifact/out/patch-compile-root.txt
|
| javac |
https://builds.apache.org/job/PreCommit-RATIS-Build/310/artifact/out/patch-compile-root.txt
|
| checkstyle |
https://builds.apache.org/job/PreCommit-RATIS-Build/310/artifact/out/diff-checkstyle-root.txt
|
| unit |
https://builds.apache.org/job/PreCommit-RATIS-Build/310/artifact/out/patch-unit-root.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-RATIS-Build/310/testReport/ |
| modules | C: ratis-common ratis-server ratis-grpc ratis-examples U: . |
| Console output |
https://builds.apache.org/job/PreCommit-RATIS-Build/310/console |
| Powered by | Apache Yetus 0.5.0 http://yetus.apache.org |
This message was automatically generated.
> Support multiple RaftServerImpl in RaftServerProxy
> --
>
> Key: RATIS-300
> URL:
[jira] [Commented] (RATIS-272) Design ideal API
[
https://issues.apache.org/jira/browse/RATIS-272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16590538#comment-16590538
]
ASF GitHub Bot commented on RATIS-272:
--
Github user VladRodionov commented on a diff in the pull request:
https://github.com/apache/incubator-ratis/pull/4#discussion_r212385674
--- Diff:
ratis-logservice/src/main/java/org/apache/ratis/logservice/api/LogReader.java
---
@@ -0,0 +1,53 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.ratis.logservice.api;
+
+import java.io.IOException;
+import java.nio.ByteBuffer;
+import java.util.List;
+
+/**
+ * Synchronous client interface to read from a LogStream.
+ */
+public interface LogReader extends AutoCloseable {
+
+ /**
+ * Seeks to the position before the record at the provided {@code
offset} in the LogStream.
+ *
+ * @param offset A non-negative, offset in the LogStream
+ * @return A future for when the operation is completed.
+ */
+ void seek(long recordId) throws IOException;
--- End diff --
I meant what I meant: logs are not a message queues. Period. There is no
need for random access in log file to grab a particular log record, but there
is such a need in message queue. As for seeking support in Raft log - there is
no such requirement to implement this efficiently, quite contrary, afair,
protocol describes very stupid and lazy algo of synching two logs: leader goes
backward one record at a time and compares it with a lagging follower records.
Do not bring please functionality we (Log service users) are not asking
for.
> Design ideal API
>
>
> Key: RATIS-272
> URL: https://issues.apache.org/jira/browse/RATIS-272
> Project: Ratis
> Issue Type: Sub-task
> Components: LogService
>Reporter: Josh Elser
>Priority: Major
>
> With influence from Apache DistributedLog, Kafka, and BookKeeper, design an
> API that balances the ideal notion of what a distribute log system should
> look like, but also considers the needs of HBase to replace a WAL.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (RATIS-272) Design ideal API
[
https://issues.apache.org/jira/browse/RATIS-272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16589806#comment-16589806
]
ASF GitHub Bot commented on RATIS-272:
--
Github user ss77892 commented on a diff in the pull request:
https://github.com/apache/incubator-ratis/pull/4#discussion_r212202468
--- Diff:
ratis-logservice/src/main/java/org/apache/ratis/logservice/api/LogService.java
---
@@ -0,0 +1,53 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.ratis.logservice.api;
+
+import java.util.Iterator;
+import java.util.concurrent.CompletableFuture;
+
+/**
+ * Entry point for interacting with the Ratis LogService.
+ */
+public interface LogService {
+
+ /**
+ * Creates a new {@link LogStream} identified by the given name. Throws
+ * an exception if a {@link LogStream} with the given name already
exists.
+ *
+ * @param name Unique name for this LogStream.
+ */
+ CompletableFuture createLog(LogName name);
+
+ /**
+ * Fetches the {@link LogStream} identified by the given name.
+ *
+ * @param name The name of the LogStream
+ */
+ CompletableFuture getLog(LogName name);
+
+ /**
+ * Lists all {@link LogStream} instances known by this LogService.
+ */
+ CompletableFuture> listLogs();
--- End diff --
I would better ask whether this method should be here. Ratis is an embedded
library that allows creating a SINGLE distributed state machine. That means a
SINGLE log. A distributed log service that would allow creating a number of
logs (i.e. state machines) sounds like an application that would include state
machines creation/management. That still can be a part of the embedded library,
but it would make everything more complicated and less flexible (i.e. different
use cases may require special rules for quorum members like rack awareness,
latency measurement, RS Groups in case of HBase, etc).
> Design ideal API
>
>
> Key: RATIS-272
> URL: https://issues.apache.org/jira/browse/RATIS-272
> Project: Ratis
> Issue Type: Sub-task
> Components: LogService
>Reporter: Josh Elser
>Priority: Major
>
> With influence from Apache DistributedLog, Kafka, and BookKeeper, design an
> API that balances the ideal notion of what a distribute log system should
> look like, but also considers the needs of HBase to replace a WAL.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (RATIS-272) Design ideal API
[
https://issues.apache.org/jira/browse/RATIS-272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16589792#comment-16589792
]
ASF GitHub Bot commented on RATIS-272:
--
Github user ss77892 commented on a diff in the pull request:
https://github.com/apache/incubator-ratis/pull/4#discussion_r212199898
--- Diff:
ratis-logservice/src/main/java/org/apache/ratis/logservice/api/LogReader.java
---
@@ -0,0 +1,53 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.ratis.logservice.api;
+
+import java.io.IOException;
+import java.nio.ByteBuffer;
+import java.util.List;
+
+/**
+ * Synchronous client interface to read from a LogStream.
+ */
+public interface LogReader extends AutoCloseable {
+
+ /**
+ * Seeks to the position before the record at the provided {@code
offset} in the LogStream.
+ *
+ * @param offset A non-negative, offset in the LogStream
+ * @return A future for when the operation is completed.
+ */
+ void seek(long recordId) throws IOException;
--- End diff --
One more thing. Actually, such seek operation should be already implemented
under the hood because it's required for quick sync up for lagging quorum nodes
due network problems/restarts ( didn't dig into the code yet, but almost 100%
sure why segments were implemented there). All we need is to provide a facade
that can be used outside.
> Design ideal API
>
>
> Key: RATIS-272
> URL: https://issues.apache.org/jira/browse/RATIS-272
> Project: Ratis
> Issue Type: Sub-task
> Components: LogService
>Reporter: Josh Elser
>Priority: Major
>
> With influence from Apache DistributedLog, Kafka, and BookKeeper, design an
> API that balances the ideal notion of what a distribute log system should
> look like, but also considers the needs of HBase to replace a WAL.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (RATIS-272) Design ideal API
[
https://issues.apache.org/jira/browse/RATIS-272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16589774#comment-16589774
]
ASF GitHub Bot commented on RATIS-272:
--
Github user ss77892 commented on a diff in the pull request:
https://github.com/apache/incubator-ratis/pull/4#discussion_r212197492
--- Diff:
ratis-logservice/src/main/java/org/apache/ratis/logservice/api/LogReader.java
---
@@ -0,0 +1,53 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.ratis.logservice.api;
+
+import java.io.IOException;
+import java.nio.ByteBuffer;
+import java.util.List;
+
+/**
+ * Synchronous client interface to read from a LogStream.
+ */
+public interface LogReader extends AutoCloseable {
+
+ /**
+ * Seeks to the position before the record at the provided {@code
offset} in the LogStream.
+ *
+ * @param offset A non-negative, offset in the LogStream
+ * @return A future for when the operation is completed.
+ */
+ void seek(long recordId) throws IOException;
--- End diff --
That's why most of the implementations of distributed logs (pub/sub model)
actually are using segments/slices/whatever they call it and why usually they
are relatively small. Usually, there is a separate index that maintains the map
between id and segment OR it keeps start id for the segment in its name. What
do you mean by 'separate logs from message queues'?
> Design ideal API
>
>
> Key: RATIS-272
> URL: https://issues.apache.org/jira/browse/RATIS-272
> Project: Ratis
> Issue Type: Sub-task
> Components: LogService
>Reporter: Josh Elser
>Priority: Major
>
> With influence from Apache DistributedLog, Kafka, and BookKeeper, design an
> API that balances the ideal notion of what a distribute log system should
> look like, but also considers the needs of HBase to replace a WAL.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
