[jira] [Commented] (RATIS-139) RaftLogWorker should use two task queues.
[
https://issues.apache.org/jira/browse/RATIS-139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16245077#comment-16245077
]
Jing Zhao commented on RATIS-139:
-
I think there're two problems here:
# when-to-sync: this is the same problem in {{FSEditLogAsync.java}} in HDFS.
And I think we're using very similar approach in RaftLogWorker.
# the efficiency of the queue/buffer: in HDFS we need double buffer to "allow
edits to progress concurrently to flushes without allocating new buffers each
time." Here we actually do not have memory allocation issue. Further
improvement may be to use a more efficient queue-like data structure to replace
the current ArrayBlockingQueue, but I'm not sure if we should use double buffer.
BTW, for the forcesyncnum, the current hdfs has similar number which is
EditsDoubleBuffer#initBufferSize (or FileJournalManager#outputBufferCapacity).
Maybe we can also use the number of data size instead of log entry number as
the indicator of forcesync#.
> RaftLogWorker should use two task queues.
> --
>
> Key: RATIS-139
> URL: https://issues.apache.org/jira/browse/RATIS-139
> Project: Ratis
> Issue Type: Improvement
>Reporter: Jitendra Nath Pandey
>Assignee: Jing Zhao
>
> {code}
> private boolean shouldFlush() {
> return pendingFlushNum >= forceSyncNum ||
> (pendingFlushNum > 0 && queue.isEmpty());
> }
> {code}
> It is possible that queue is often non-empty, but not full to trigger force
> sync. We could use two queues similar to two buffer approach in HDFS edit
> logs.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (RATIS-123) Add CLI for FileStore example
[ https://issues.apache.org/jira/browse/RATIS-123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16245028#comment-16245028 ] Tsz Wo Nicholas Sze commented on RATIS-123: --- Once we have RATIS-95 (which implements a CLI for the arithmetic example), we could do something similar to FileStore. > Add CLI for FileStore example > - > > Key: RATIS-123 > URL: https://issues.apache.org/jira/browse/RATIS-123 > Project: Ratis > Issue Type: New Feature >Reporter: Tsz Wo Nicholas Sze > Attachments: FileStoreServer.java > > > In RATIS-122, we adds a File Store example which support the following > operations > - Write(path, data) > - Read(path) > - Delete(path) > In this JIRA, we add some CLIs so that servers and the clients can be run in > command line. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (RATIS-95) Executable Jar for the ratis examples
[
https://issues.apache.org/jira/browse/RATIS-95?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16245025#comment-16245025
]
Tsz Wo Nicholas Sze commented on RATIS-95:
--
{{example/src/main/bin}} sounds since it won't confuse the users
- Let's use ls and do not hard code the file version.
{code}
ARTIFACT=`ls target/ratis-examples-*.jar`
{code}
- The script files need a license header.
- It is a good starting point to have
org.apache.ratis.examples.arithmetic.Runner as the main class. It probably is
better to have a class org.apache.ratis.examples.Runner to run all the examples
(so that it can choose which example to run). This can be done in this JIRA or
separately.
Thanks a lot, I am very excited about your patch.
> Executable Jar for the ratis examples
> -
>
> Key: RATIS-95
> URL: https://issues.apache.org/jira/browse/RATIS-95
> Project: Ratis
> Issue Type: Improvement
>Reporter: Elek, Marton
>Assignee: Elek, Marton
> Attachments: RATIS-95.wip.patch
>
>
> The current example project shows an example implementation of the base
> interfaces. I suggest to create simple CLI application for the test (just an
> additional class with main and argument parsing) to make it easier to
> demonstrate how a ratis cluster could be run.
> For example:
> {code}
> java -jar ratis-examples-uber.jar --port 2323 --id node2 --peers
> node3:localhost:4566,node1:localhost:3456
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (RATIS-139) RaftLogWorker should use two task queues.
[
https://issues.apache.org/jira/browse/RATIS-139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16245011#comment-16245011
]
Jitendra Nath Pandey commented on RATIS-139:
The wait for the log-sync adds to the operation latency. Therefore to keep
latency low, it will help to flush as soon as possible, instead of waiting till
forcesyncnum. I think configuration of forcesyncnum will be tricky. We want to
flush asap, but at the same time batch as much as together for throughput. This
is a similar requirement as in HDFS edit logs, which uses two buffers.
I don't have measurements. It is a suggested improvement based on observation
in the code. I think it will help latency and would avoid need to tune
forcesyncnum.
> RaftLogWorker should use two task queues.
> --
>
> Key: RATIS-139
> URL: https://issues.apache.org/jira/browse/RATIS-139
> Project: Ratis
> Issue Type: Improvement
>Reporter: Jitendra Nath Pandey
>Assignee: Jing Zhao
>
> {code}
> private boolean shouldFlush() {
> return pendingFlushNum >= forceSyncNum ||
> (pendingFlushNum > 0 && queue.isEmpty());
> }
> {code}
> It is possible that queue is often non-empty, but not full to trigger force
> sync. We could use two queues similar to two buffer approach in HDFS edit
> logs.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (RATIS-135) Closed segments contribute to leaked memory as eviction policies are not honoured
[
https://issues.apache.org/jira/browse/RATIS-135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tsz Wo Nicholas Sze updated RATIS-135:
--
+1
> Closed segments contribute to leaked memory as eviction policies are not
> honoured
> -
>
> Key: RATIS-135
> URL: https://issues.apache.org/jira/browse/RATIS-135
> Project: Ratis
> Issue Type: Bug
>Reporter: Mukul Kumar Singh
>Assignee: Mukul Kumar Singh
> Attachments: RATIS-135.001.patch, RATIS-135.002.patch
>
>
> Closed segments leak memory as eviction does not remove these segments from
> closed segments list.
> {code}
> void evictCache(long[] followerIndices, long flushedIndex,
> long lastAppliedIndex) {
> List toEvict = evictionPolicy.evict(followerIndices,
> flushedIndex, lastAppliedIndex, closedSegments, maxCachedSegments);
> for (LogSegment s : toEvict) {
> s.evictCache();
> }
> {code}
> After the segments have been marked for eviction, they should be removed from
> queue as well.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (RATIS-137) RaftBasicTests.testBasicAppendEntries may fail
[
https://issues.apache.org/jira/browse/RATIS-137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tsz Wo Nicholas Sze reassigned RATIS-137:
-
Assignee: Lokesh Jain
> RaftBasicTests.testBasicAppendEntries may fail
> --
>
> Key: RATIS-137
> URL: https://issues.apache.org/jira/browse/RATIS-137
> Project: Ratis
> Issue Type: Bug
>Reporter: Tsz Wo Nicholas Sze
>Assignee: Lokesh Jain
>
> [~atrivedi] reported in RATIS-72 that the test may fail.
> {code}
> TestRaftWithHadoopRpc>RaftBasicTests.testBasicAppendEntries:127->RaftBasicTests.lambda$testBasicAppendEntries$1:127
> expected:<10> but was:<11>
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (RATIS-139) RaftLogWorker should use two task queues.
[
https://issues.apache.org/jira/browse/RATIS-139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16244933#comment-16244933
]
Jing Zhao commented on RATIS-139:
-
[~jnp], have you seen a performance penalty with the current implementation?
Here the scenario is different from editlog in HDFS, since the queue is a
pending task queue instead of a pure data buffer. If you look at the types of
the task, all the other tasks other than writelog may trigger a sync
automatically. And if the queue contains more writelog task, then we can wait
till the pendingflushnum hit forcesyncnum, or till the queue is empty, or we
hit a finalizeSegment/truncate taks. Thoughts?
> RaftLogWorker should use two task queues.
> --
>
> Key: RATIS-139
> URL: https://issues.apache.org/jira/browse/RATIS-139
> Project: Ratis
> Issue Type: Improvement
>Reporter: Jitendra Nath Pandey
>
> {code}
> private boolean shouldFlush() {
> return pendingFlushNum >= forceSyncNum ||
> (pendingFlushNum > 0 && queue.isEmpty());
> }
> {code}
> It is possible that queue is often non-empty, but not full to trigger force
> sync. We could use two queues similar to two buffer approach in HDFS edit
> logs.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (RATIS-139) RaftLogWorker should use two task queues.
[
https://issues.apache.org/jira/browse/RATIS-139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jing Zhao reassigned RATIS-139:
---
Assignee: Jing Zhao
> RaftLogWorker should use two task queues.
> --
>
> Key: RATIS-139
> URL: https://issues.apache.org/jira/browse/RATIS-139
> Project: Ratis
> Issue Type: Improvement
>Reporter: Jitendra Nath Pandey
>Assignee: Jing Zhao
>
> {code}
> private boolean shouldFlush() {
> return pendingFlushNum >= forceSyncNum ||
> (pendingFlushNum > 0 && queue.isEmpty());
> }
> {code}
> It is possible that queue is often non-empty, but not full to trigger force
> sync. We could use two queues similar to two buffer approach in HDFS edit
> logs.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (RATIS-139) RaftLogWorker should use two task queues.
[
https://issues.apache.org/jira/browse/RATIS-139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jing Zhao updated RATIS-139:
Issue Type: Improvement (was: Bug)
> RaftLogWorker should use two task queues.
> --
>
> Key: RATIS-139
> URL: https://issues.apache.org/jira/browse/RATIS-139
> Project: Ratis
> Issue Type: Improvement
>Reporter: Jitendra Nath Pandey
>
> {code}
> private boolean shouldFlush() {
> return pendingFlushNum >= forceSyncNum ||
> (pendingFlushNum > 0 && queue.isEmpty());
> }
> {code}
> It is possible that queue is often non-empty, but not full to trigger force
> sync. We could use two queues similar to two buffer approach in HDFS edit
> logs.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (RATIS-129) Compile protobuf and shade if the shaded source directory is missing
[ https://issues.apache.org/jira/browse/RATIS-129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated RATIS-129: -- Fix Version/s: (was: 0.2.0-alpha) > Compile protobuf and shade if the shaded source directory is missing > > > Key: RATIS-129 > URL: https://issues.apache.org/jira/browse/RATIS-129 > Project: Ratis > Issue Type: Improvement >Reporter: Tsz Wo Nicholas Sze >Assignee: Tsz Wo Nicholas Sze > Attachments: r129_20171103.patch > > > Currently, we uses skipShade to activate protobuf compilation and shading. > It is better to check if the corresponding shaded source directory is missing. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (RATIS-139) RaftLogWorker should use two task queues.
Jitendra Nath Pandey created RATIS-139:
--
Summary: RaftLogWorker should use two task queues.
Key: RATIS-139
URL: https://issues.apache.org/jira/browse/RATIS-139
Project: Ratis
Issue Type: Bug
Reporter: Jitendra Nath Pandey
{code}
private boolean shouldFlush() {
return pendingFlushNum >= forceSyncNum ||
(pendingFlushNum > 0 && queue.isEmpty());
}
{code}
It is possible that queue is often non-empty, but not full to trigger force
sync. We could use two queues similar to two buffer approach in HDFS edit logs.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (RATIS-122) Add a FileStore example
[ https://issues.apache.org/jira/browse/RATIS-122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16244891#comment-16244891 ] Jitendra Nath Pandey commented on RATIS-122: +1, the latest patch looks good to me. > Add a FileStore example > --- > > Key: RATIS-122 > URL: https://issues.apache.org/jira/browse/RATIS-122 > Project: Ratis > Issue Type: New Feature >Reporter: Tsz Wo Nicholas Sze >Assignee: Tsz Wo Nicholas Sze > Attachments: r122_20171017.patch, r122_20171017b.patch, > r122_20171024.patch, r122_20171025.patch, r122_20171026.patch, > r122_20171031.patch, r122_20171101.patch, r122_20171101b.patch, > r122_20171102.patch, r122_20171103.patch, r122_20171108.patch > > > I propose to add a new FileStore example. Below are the ideas: > - It uses Ratis to store files so that the files are replicated in a Raft > group. > - It is not a file system -- it only supports basic operations such as read, > write and delete but not ls, rename, etc. > - Its state machine stores the file data separated from the log in order to > reduce the log size. > - It can be served as a Ratis performance test. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (RATIS-122) Add a FileStore example
[
https://issues.apache.org/jira/browse/RATIS-122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16244884#comment-16244884
]
Hadoop QA commented on RATIS-122:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m
11s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 8 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
35s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
16s{color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
8s{color} | {color:red} ratis-common in master failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
6s{color} | {color:red} ratis-client in master failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
6s{color} | {color:red} ratis-server in master failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
6s{color} | {color:red} ratis-netty in master failed. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
58s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
6s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
8s{color} | {color:red} ratis-common in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
7s{color} | {color:red} ratis-client in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
10s{color} | {color:red} ratis-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
7s{color} | {color:red} ratis-netty in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
9s{color} | {color:red} ratis-examples in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
7s{color} | {color:red} ratis-common in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
5s{color} | {color:red} ratis-client in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
8s{color} | {color:red} ratis-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
6s{color} | {color:red} ratis-netty in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
8s{color} | {color:red} ratis-examples in the patch failed. {color} |
| {color:red}-1{color} | {color:red} cc {color} | {color:red} 0m 7s{color} |
{color:red} ratis-common in the patch failed. {color} |
| {color:red}-1{color} | {color:red} cc {color} | {color:red} 0m 5s{color} |
{color:red} ratis-client in the patch failed. {color} |
| {color:red}-1{color} | {color:red} cc {color} | {color:red} 0m 8s{color} |
{color:red} ratis-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} cc {color} | {color:red} 0m 6s{color} |
{color:red} ratis-netty in the patch failed. {color} |
| {color:red}-1{color} | {color:red} cc {color} | {color:red} 0m 8s{color} |
{color:red} ratis-examples in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 7s{color}
| {color:red} ratis-common in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 5s{color}
| {color:red} ratis-client in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 8s{color}
| {color:red} ratis-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 6s{color}
| {color:red} ratis-netty in the patch fa
[jira] [Commented] (RATIS-138) "mvn clean install" doesn't work.
[ https://issues.apache.org/jira/browse/RATIS-138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16244870#comment-16244870 ] Tsz Wo Nicholas Sze commented on RATIS-138: --- I just have reverted the commit in RATIS-129. > "mvn clean install" doesn't work. > - > > Key: RATIS-138 > URL: https://issues.apache.org/jira/browse/RATIS-138 > Project: Ratis > Issue Type: Bug >Reporter: Jitendra Nath Pandey > Attachments: RATIS-138.1.patch > > > mvn clean and mvn install work when performed in two different commands. > However, mvn clean install doesn't work. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (RATIS-129) Compile protobuf and shade if the shaded source directory is missing
[ https://issues.apache.org/jira/browse/RATIS-129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16244868#comment-16244868 ] Tsz Wo Nicholas Sze edited comment on RATIS-129 at 11/8/17 10:35 PM: - Reverted the commit. Resolving this as Not A Problem. was (Author: szetszwo): Resolving this as Not A Problem. > Compile protobuf and shade if the shaded source directory is missing > > > Key: RATIS-129 > URL: https://issues.apache.org/jira/browse/RATIS-129 > Project: Ratis > Issue Type: Improvement >Reporter: Tsz Wo Nicholas Sze >Assignee: Tsz Wo Nicholas Sze > Fix For: 0.2.0-alpha > > Attachments: r129_20171103.patch > > > Currently, we uses skipShade to activate protobuf compilation and shading. > It is better to check if the corresponding shaded source directory is missing. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (RATIS-129) Compile protobuf and shade if the shaded source directory is missing
[ https://issues.apache.org/jira/browse/RATIS-129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze resolved RATIS-129. --- Resolution: Not A Problem Resolving this as Not A Problem. > Compile protobuf and shade if the shaded source directory is missing > > > Key: RATIS-129 > URL: https://issues.apache.org/jira/browse/RATIS-129 > Project: Ratis > Issue Type: Improvement >Reporter: Tsz Wo Nicholas Sze >Assignee: Tsz Wo Nicholas Sze > Fix For: 0.2.0-alpha > > Attachments: r129_20171103.patch > > > Currently, we uses skipShade to activate protobuf compilation and shading. > It is better to check if the corresponding shaded source directory is missing. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Reopened] (RATIS-129) Compile protobuf and shade if the shaded source directory is missing
[ https://issues.apache.org/jira/browse/RATIS-129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze reopened RATIS-129: --- Reopen for reverting. > Compile protobuf and shade if the shaded source directory is missing > > > Key: RATIS-129 > URL: https://issues.apache.org/jira/browse/RATIS-129 > Project: Ratis > Issue Type: Improvement >Reporter: Tsz Wo Nicholas Sze >Assignee: Tsz Wo Nicholas Sze > Fix For: 0.2.0-alpha > > Attachments: r129_20171103.patch > > > Currently, we uses skipShade to activate protobuf compilation and shading. > It is better to check if the corresponding shaded source directory is missing. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (RATIS-138) "mvn clean install" doesn't work.
[ https://issues.apache.org/jira/browse/RATIS-138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16244865#comment-16244865 ] Tsz Wo Nicholas Sze commented on RATIS-138: --- Let's just revert RATIS-129 using git. I am happy to do so if it does not work. > "mvn clean install" doesn't work. > - > > Key: RATIS-138 > URL: https://issues.apache.org/jira/browse/RATIS-138 > Project: Ratis > Issue Type: Bug >Reporter: Jitendra Nath Pandey > Attachments: RATIS-138.1.patch > > > mvn clean and mvn install work when performed in two different commands. > However, mvn clean install doesn't work. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (RATIS-122) Add a FileStore example
[ https://issues.apache.org/jira/browse/RATIS-122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated RATIS-122: -- Attachment: r122_20171108.patch Thanks [~jnp]. Here is a new patch addressing your comments. r122_20171108.patch - Notice that TaskQueue is to guarantee the files will be written concurrently and the writes to each file are in-order. It seems useful so that I refactor it to a class in org.apache.ratis.util. - Notice also that WRITE should not happen in applyTransaction since startTransaction converts WRITE requests to WRITEHEADER requests. > Add a FileStore example > --- > > Key: RATIS-122 > URL: https://issues.apache.org/jira/browse/RATIS-122 > Project: Ratis > Issue Type: New Feature >Reporter: Tsz Wo Nicholas Sze >Assignee: Tsz Wo Nicholas Sze > Attachments: r122_20171017.patch, r122_20171017b.patch, > r122_20171024.patch, r122_20171025.patch, r122_20171026.patch, > r122_20171031.patch, r122_20171101.patch, r122_20171101b.patch, > r122_20171102.patch, r122_20171103.patch, r122_20171108.patch > > > I propose to add a new FileStore example. Below are the ideas: > - It uses Ratis to store files so that the files are replicated in a Raft > group. > - It is not a file system -- it only supports basic operations such as read, > write and delete but not ls, rename, etc. > - Its state machine stores the file data separated from the log in order to > reduce the log size. > - It can be served as a Ratis performance test. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (RATIS-138) "mvn clean install" doesn't work.
[
https://issues.apache.org/jira/browse/RATIS-138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16244841#comment-16244841
]
Hadoop QA commented on RATIS-138:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m
18s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
39s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
29s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
20s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
7s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
10s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 51s{color}
| {color:red} ratis-proto-shaded generated 100 new + 0 unchanged - 0 fixed =
100 total (was 0) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
3s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
48s{color} | {color:green} ratis-proto-shaded in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
16s{color} | {color:green} ratis-hadoop-shaded in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 10m
22s{color} | {color:green} root in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
32s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 48s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/ratis:date2017-11-08
|
| JIRA Issue | RATIS-138 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12896733/RATIS-138.1.patch |
| Optional Tests | asflicense javac javadoc unit xml compile |
| uname | Linux 8902dc9396d5 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-RATIS-Build/yetus-personality.sh
|
| git revision | master / 22b70e9 |
| Default Java | 1.8.0_151 |
| javac |
https://builds.apache.org/job/PreCommit-RATIS-Build/21/artifact/out/diff-compile-javac-ratis-proto-shaded.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-RATIS-Build/21/testReport/ |
| modules | C: ratis-proto-shaded ratis-hadoop-shaded . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-RATIS-Build/21/console |
| Powered by | Apache Yetus 0.5.0 http://yetus.apache.org |
This message was automatically generated.
> "mvn clean install" doesn't work.
> -
>
> Key: RATIS-138
> URL: https://issues.apache.org/jira/browse/RATIS-138
> Project: Ratis
> Issue Type: Bug
>Reporter: Jitendra Nath Pandey
> Attachments: RATIS-138.1.patch
>
>
> mvn clean and mvn install work when performed in two different commands.
> However, mvn clean install doesn't work.
--
This message was sent
[jira] [Commented] (RATIS-138) "mvn clean install" doesn't work.
[ https://issues.apache.org/jira/browse/RATIS-138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16244770#comment-16244770 ] Jitendra Nath Pandey commented on RATIS-138: cc [~elek], [~szetszwo] > "mvn clean install" doesn't work. > - > > Key: RATIS-138 > URL: https://issues.apache.org/jira/browse/RATIS-138 > Project: Ratis > Issue Type: Bug >Reporter: Jitendra Nath Pandey > Attachments: RATIS-138.1.patch > > > mvn clean and mvn install work when performed in two different commands. > However, mvn clean install doesn't work. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (RATIS-129) Compile protobuf and shade if the shaded source directory is missing
[ https://issues.apache.org/jira/browse/RATIS-129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16244769#comment-16244769 ] Jitendra Nath Pandey commented on RATIS-129: I have created RATIS-138, and attached a revert patch. > Compile protobuf and shade if the shaded source directory is missing > > > Key: RATIS-129 > URL: https://issues.apache.org/jira/browse/RATIS-129 > Project: Ratis > Issue Type: Improvement >Reporter: Tsz Wo Nicholas Sze >Assignee: Tsz Wo Nicholas Sze > Fix For: 0.2.0-alpha > > Attachments: r129_20171103.patch > > > Currently, we uses skipShade to activate protobuf compilation and shading. > It is better to check if the corresponding shaded source directory is missing. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (RATIS-138) "mvn clean install" doesn't work.
[ https://issues.apache.org/jira/browse/RATIS-138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jitendra Nath Pandey updated RATIS-138: --- Attachment: RATIS-138.1.patch > "mvn clean install" doesn't work. > - > > Key: RATIS-138 > URL: https://issues.apache.org/jira/browse/RATIS-138 > Project: Ratis > Issue Type: Bug >Reporter: Jitendra Nath Pandey > Attachments: RATIS-138.1.patch > > > mvn clean and mvn install work when performed in two different commands. > However, mvn clean install doesn't work. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (RATIS-138) "mvn clean install" doesn't work.
[ https://issues.apache.org/jira/browse/RATIS-138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16244767#comment-16244767 ] Jitendra Nath Pandey commented on RATIS-138: A revert of RATIS-129 fixes this problem. The patch is attached. > "mvn clean install" doesn't work. > - > > Key: RATIS-138 > URL: https://issues.apache.org/jira/browse/RATIS-138 > Project: Ratis > Issue Type: Bug >Reporter: Jitendra Nath Pandey > > mvn clean and mvn install work when performed in two different commands. > However, mvn clean install doesn't work. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (RATIS-129) Compile protobuf and shade if the shaded source directory is missing
[
https://issues.apache.org/jira/browse/RATIS-129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16244762#comment-16244762
]
Jitendra Nath Pandey commented on RATIS-129:
I was able to confirm that {{mvn clean install -DskipTests}} works by reverting
this.
I was expecting that RATIS-132 will undo this patch, however we can revert it
since RATIS-132 is already committed.
> Compile protobuf and shade if the shaded source directory is missing
>
>
> Key: RATIS-129
> URL: https://issues.apache.org/jira/browse/RATIS-129
> Project: Ratis
> Issue Type: Improvement
>Reporter: Tsz Wo Nicholas Sze
>Assignee: Tsz Wo Nicholas Sze
> Fix For: 0.2.0-alpha
>
> Attachments: r129_20171103.patch
>
>
> Currently, we uses skipShade to activate protobuf compilation and shading.
> It is better to check if the corresponding shaded source directory is missing.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (RATIS-95) Executable Jar for the ratis examples
[
https://issues.apache.org/jira/browse/RATIS-95?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Elek, Marton updated RATIS-95:
--
Attachment: RATIS-95.wip.patch
Preview version of the patch is uploaded. Stil I am thinking what is the best
place for the sh scripts (should I add it to the dist package or enough to move
it to the example/src/main/bin?)
> Executable Jar for the ratis examples
> -
>
> Key: RATIS-95
> URL: https://issues.apache.org/jira/browse/RATIS-95
> Project: Ratis
> Issue Type: Improvement
>Reporter: Elek, Marton
>Assignee: Elek, Marton
> Attachments: RATIS-95.wip.patch
>
>
> The current example project shows an example implementation of the base
> interfaces. I suggest to create simple CLI application for the test (just an
> additional class with main and argument parsing) to make it easier to
> demonstrate how a ratis cluster could be run.
> For example:
> {code}
> java -jar ratis-examples-uber.jar --port 2323 --id node2 --peers
> node3:localhost:4566,node1:localhost:3456
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (RATIS-135) Closed segments contribute to leaked memory as eviction policies are not honoured
[
https://issues.apache.org/jira/browse/RATIS-135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16244680#comment-16244680
]
Hadoop QA commented on RATIS-135:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 3m
52s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
11s{color} | {color:red} ratis-server in master failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
8s{color} | {color:red} ratis-server in master failed. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
8s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
8s{color} | {color:red} ratis-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
8s{color} | {color:red} ratis-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 8s{color}
| {color:red} ratis-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
8s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 8s{color}
| {color:red} ratis-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
4s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 5m 19s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/ratis:date2017-11-08
|
| JIRA Issue | RATIS-135 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12896716/RATIS-135.002.patch |
| Optional Tests | asflicense javac javadoc unit findbugs checkstyle
compile |
| uname | Linux 4eb15b734f50 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31
14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-RATIS-Build/yetus-personality.sh
|
| git revision | master / 22b70e9 |
| Default Java | 1.8.0_151 |
| mvninstall |
https://builds.apache.org/job/PreCommit-RATIS-Build/20/artifact/out/branch-mvninstall-ratis-server.txt
|
| compile |
https://builds.apache.org/job/PreCommit-RATIS-Build/20/artifact/out/branch-compile-ratis-server.txt
|
| mvninstall |
https://builds.apache.org/job/PreCommit-RATIS-Build/20/artifact/out/patch-mvninstall-ratis-server.txt
|
| compile |
https://builds.apache.org/job/PreCommit-RATIS-Build/20/artifact/out/patch-compile-ratis-server.txt
|
| javac |
https://builds.apache.org/job/PreCommit-RATIS-Build/20/artifact/out/patch-compile-ratis-server.txt
|
| unit |
https://builds.apache.org/job/PreCommit-RATIS-Build/20/artifact/out/patch-unit-ratis-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-RATIS-Build/20/testReport/ |
| modules | C: ratis-server U: ratis-server |
| Console output |
https://builds.apache.org/job/PreCommit-RATIS-Build/20/console |
| Powered by | Apache Yetus 0.5.0 http://yetus.apache.org |
This message was automatically generated.
> Closed segments contribute to leaked memory as eviction policies are not
> honoured
> -
>
> Key: RATIS-135
> URL: htt
[jira] [Commented] (RATIS-129) Compile protobuf and shade if the shaded source directory is missing
[ https://issues.apache.org/jira/browse/RATIS-129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16244626#comment-16244626 ] Elek, Marton commented on RATIS-129: RATIS-132 doesn't fix this problem. See the precommit message of RATIS-132: https://builds.apache.org/job/PreCommit-RATIS-Build/18/artifact/out/patch-compile-root.txt It's failed because the tag couldn't be used together with 'mvn clean package'. (Profile calculation happens before the 'clean', but the directory is removed during the clean). So IMHO this patch could be reverted and the build will be finally fixed. Maybe [~jingzhao] could confirm, if the problem -- reported in the comment of RATIS-102 -- no longer exists with RATIS-132 and reverted RATIS-129. > Compile protobuf and shade if the shaded source directory is missing > > > Key: RATIS-129 > URL: https://issues.apache.org/jira/browse/RATIS-129 > Project: Ratis > Issue Type: Improvement >Reporter: Tsz Wo Nicholas Sze >Assignee: Tsz Wo Nicholas Sze > Fix For: 0.2.0-alpha > > Attachments: r129_20171103.patch > > > Currently, we uses skipShade to activate protobuf compilation and shading. > It is better to check if the corresponding shaded source directory is missing. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (RATIS-135) Closed segments contribute to leaked memory as eviction policies are not honoured
[
https://issues.apache.org/jira/browse/RATIS-135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mukul Kumar Singh updated RATIS-135:
Attachment: RATIS-135.002.patch
patch v2 fixes the unit test failure.
> Closed segments contribute to leaked memory as eviction policies are not
> honoured
> -
>
> Key: RATIS-135
> URL: https://issues.apache.org/jira/browse/RATIS-135
> Project: Ratis
> Issue Type: Bug
>Reporter: Mukul Kumar Singh
>Assignee: Mukul Kumar Singh
> Attachments: RATIS-135.001.patch, RATIS-135.002.patch
>
>
> Closed segments leak memory as eviction does not remove these segments from
> closed segments list.
> {code}
> void evictCache(long[] followerIndices, long flushedIndex,
> long lastAppliedIndex) {
> List toEvict = evictionPolicy.evict(followerIndices,
> flushedIndex, lastAppliedIndex, closedSegments, maxCachedSegments);
> for (LogSegment s : toEvict) {
> s.evictCache();
> }
> {code}
> After the segments have been marked for eviction, they should be removed from
> queue as well.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (RATIS-77) "RaftServerProtocolService" and " GRpc.proto" inconsistent
[
https://issues.apache.org/jira/browse/RATIS-77?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16244522#comment-16244522
]
Jing Zhao commented on RATIS-77:
It's because if the incoming request is a stream, we expect response for each
chunk so that we can detect failure earlier.
> "RaftServerProtocolService" and " GRpc.proto" inconsistent
> --
>
> Key: RATIS-77
> URL: https://issues.apache.org/jira/browse/RATIS-77
> Project: Ratis
> Issue Type: Bug
>Reporter: kaiyangzhang
>Assignee: garvit
>
> *Maybe a bug, look at the comments in the code and GRpc.proto*
> {code:title=GRpc.proto|borderStyle=solid}
>..
>..
>//Executes a client-streaming call , return only one response.
>rpc installSnapshot(stream ratis.common.InstallSnapshotRequestProto)
> returns(ratis.common.InstallSnapshotReplyProto) {}
>...
>...
> {code}
> {code:title=RaftServerProtocolService.java |borderStyle=solid}
>
>
> @Override
> public StreamObserver installSnapshot(
> StreamObserver responseObserver) {
> return new StreamObserver() {
> @Override
> public void onNext(InstallSnapshotRequestProto request) {
> try {
> //receive a client-streaming call, return a response stream
> //code and GRpc.proto inconsistent
> final InstallSnapshotReplyProto reply =
> server.installSnapshot(request);
> responseObserver.onNext(reply);
> } catch (Throwable e) {
> LOG.info("{} got exception when handling installSnapshot {}: {}",
> id, request.getServerRequest(), e);
> responseObserver.onError(RaftGrpcUtil.wrapException(e));
> }
> }
>..
>..
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (RATIS-117) Add test for situation when old leader can/cannot commit log
[
https://issues.apache.org/jira/browse/RATIS-117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16244489#comment-16244489
]
Jing Zhao commented on RATIS-117:
-
The latest patch looks pretty good to me. Just some minors:
# Let's use "Assert.assertTrue" instead of Preconditions.assertTrue
{code}
158 Preconditions.assertTrue(logEntriesContains(followerLog, messages));
{code}
# Remove the following commented code.
{code}
174 //final RaftPeerId newLeaderId = waitForLeader(cluster).getId();
175 //// confirm the server with log is elected as new leader.
176 //
Preconditions.assertTrue(newLeaderId.equals(followerToSendLog.getId()));
177 //Thread.sleep(cluster.getMaxTimeout() + 100);
{code}
# The method {{assertLogEntriesNotMatch}}'s signature is not easy to follow. If
you want to check whether "any log entry containing the given simple msg should
follow certain rule", you can define a method like {{void
checkLogEntries(RaftLog log, List expectedMessages,
Predicate predicate)}}.
{code}
167 static void assertLogEntriesNotMatch(RaftLog log, TermIndex[] entries,
168 long startIndex, long expectedTerm, SimpleMessage...
expectedMessages) {
{code}
> Add test for situation when old leader can/cannot commit log
>
>
> Key: RATIS-117
> URL: https://issues.apache.org/jira/browse/RATIS-117
> Project: Ratis
> Issue Type: Test
>Reporter: Yubo Xu
>Assignee: Yubo Xu
>Priority: Minor
> Attachments: RATIS-117.001.patch, RATIS-117.002.patch,
> RATIS-117.003.patch
>
>
> As described in Question 7 of the quiz at
> [https://ramcloud.stanford.edu/~ongaro/userstudy/quizzes.html], an old leader
> can complete the commitment of an old log entry satisfying certain
> requirements. We need tests to confirm Ratis behaves correctly under such
> cases.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (RATIS-138) "mvn clean install" doesn't work.
Jitendra Nath Pandey created RATIS-138: -- Summary: "mvn clean install" doesn't work. Key: RATIS-138 URL: https://issues.apache.org/jira/browse/RATIS-138 Project: Ratis Issue Type: Bug Reporter: Jitendra Nath Pandey mvn clean and mvn install work when performed in two different commands. However, mvn clean install doesn't work. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (RATIS-122) Add a FileStore example
[ https://issues.apache.org/jira/browse/RATIS-122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16243496#comment-16243496 ] Jitendra Nath Pandey commented on RATIS-122: A few minor comments. * It might be better to make FileInfo abstract, because some basic operations are not supported, or will cause exceptions. * The tasks in writeQueue are being executed by multiple threads in the executor, therefore order is not guaranteed, the comment above the declaration could be misleading. * offerAndSubmit submits a task to the executor. And this task itself submits the real tasks to the user. It seems we are doing double scheduling. * In FileStoreStateMachine#applyTransaction, the request type WRITE is not handled. > Add a FileStore example > --- > > Key: RATIS-122 > URL: https://issues.apache.org/jira/browse/RATIS-122 > Project: Ratis > Issue Type: New Feature >Reporter: Tsz Wo Nicholas Sze >Assignee: Tsz Wo Nicholas Sze > Attachments: r122_20171017.patch, r122_20171017b.patch, > r122_20171024.patch, r122_20171025.patch, r122_20171026.patch, > r122_20171031.patch, r122_20171101.patch, r122_20171101b.patch, > r122_20171102.patch, r122_20171103.patch > > > I propose to add a new FileStore example. Below are the ideas: > - It uses Ratis to store files so that the files are replicated in a Raft > group. > - It is not a file system -- it only supports basic operations such as read, > write and delete but not ls, rename, etc. > - Its state machine stores the file data separated from the log in order to > reduce the log size. > - It can be served as a Ratis performance test. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
