[
https://issues.apache.org/jira/browse/ARTEMIS-508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Francesco Nigro updated ARTEMIS-508:
------------------------------------
Description:
https://github.com/franz1981/activemq-artemis/tree/34b7697a5d04e1979908f1153956daba21f5d86e/artemis-journal/src/main/java/org/apache/activemq/artemis/core/io/mapped
I've implemented in the package org.apache.activemq.artemis.core.io.mapped a
new memory mapped SequentialFile implementation to support fast write/read
operations for OSs that cannot use libaio or need RAM access performances on
standard Files.
The implementation is not thread-safe (it is needed?) and needs more buffer
checks (or a complex implementation). I'm using the Netty's PlatformDependent
class to perform bulk copy without safepoint's poll issues.
The implementation is simple but it's good as a proof of concept to compare it
against the others: i' ve added a coordinated-omission performance test to
measure the latency of a directWrite + OS jitter.
https://github.com/franz1981/activemq-artemis/blob/34b7697a5d04e1979908f1153956daba21f5d86e/artemis-journal/src/test/java/org/apache/activemq/artemis/core/io/aio/SequentialFileBench.java
The write tests show performances typical of memory mapped file: quoting from
Peter Lawrey "for burst of up to 10% of the main memory, it can sustain rates
of 1 - 3 GB/second written. e.g. A laptop with 8 GB of memory might handle
bursts of 800 MB at a rate of 1 GB per second. A server with 64 GB of memory
might handle a burst of 6.5 GB at a rate of 3 GB per second".
I want to improve the quality of the implementation by:
1- enforcing the original SequenceFile contract
2- replacing synchronized reads/writes with lock-free versions
3- replacing the EventExecutor with a Lock-Free (even wait-free) Array Queue +
EventLoop's poller
4- addressing all the False-Sharing issues around all the AtomicLong instances
used
5- reducing the garbage produced for the fast paths to 0
[6- expose try methods to allow direct flow control]
[7- replacing the Semaphore based rate limiter with a lock free one]
That's are all proposals, what do you think about it?
Regards
was:
https://github.com/franz1981/activemq-artemis/blob/34b7697a5d04e1979908f1153956daba21f5d86e/artemis-journal/src/test/java/org/apache/activemq/artemis/core/io/aio/SequentialFileBench.java
I've implemented in the package org.apache.activemq.artemis.core.io.mapped a
new memory mapped SequentialFile implementation to support fast write/read
operations for OSs that cannot use libaio or need RAM access performances on
standard Files.
The implementation is not thread-safe (it is needed?) and needs more buffer
checks (or a complex implementation). I'm using the Netty's PlatformDependent
class to perform bulk copy without safepoint's poll issues.
The implementation is simple but it's good as a proof of concept to compare it
against the others: i' ve added a coordinated-omission performance test to
measure the latency of a directWrite + OS jitter.
The write tests show performances typical of memory mapped file: quoting from
Peter Lawrey "for burst of up to 10% of the main memory, it can sustain rates
of 1 - 3 GB/second written. e.g. A laptop with 8 GB of memory might handle
bursts of 800 MB at a rate of 1 GB per second. A server with 64 GB of memory
might handle a burst of 6.5 GB at a rate of 3 GB per second".
I want to improve the quality of the implementation by:
1- enforcing the original SequenceFile contract
2- replacing synchronized reads/writes with lock-free versions
3- replacing the EventExecutor with a Lock-Free (even wait-free) Array Queue +
EventLoop's poller
4- addressing all the False-Sharing issues around all the AtomicLong instances
used
5- reducing the garbage produced for the fast paths to 0
[6- expose try methods to allow direct flow control]
[7- replacing the Semaphore based rate limiter with a lock free one]
That's are all proposals, what do you think about it?
Regards
> Sequential File Improvement + Performance Tests
> -----------------------------------------------
>
> Key: ARTEMIS-508
> URL: https://issues.apache.org/jira/browse/ARTEMIS-508
> Project: ActiveMQ Artemis
> Issue Type: Improvement
> Reporter: Francesco Nigro
> Priority: Critical
> Labels: feature, performance, test
>
> https://github.com/franz1981/activemq-artemis/tree/34b7697a5d04e1979908f1153956daba21f5d86e/artemis-journal/src/main/java/org/apache/activemq/artemis/core/io/mapped
> I've implemented in the package org.apache.activemq.artemis.core.io.mapped a
> new memory mapped SequentialFile implementation to support fast write/read
> operations for OSs that cannot use libaio or need RAM access performances on
> standard Files.
> The implementation is not thread-safe (it is needed?) and needs more buffer
> checks (or a complex implementation). I'm using the Netty's
> PlatformDependent class to perform bulk copy without safepoint's poll issues.
> The implementation is simple but it's good as a proof of concept to compare
> it against the others: i' ve added a coordinated-omission performance test to
> measure the latency of a directWrite + OS jitter.
> https://github.com/franz1981/activemq-artemis/blob/34b7697a5d04e1979908f1153956daba21f5d86e/artemis-journal/src/test/java/org/apache/activemq/artemis/core/io/aio/SequentialFileBench.java
> The write tests show performances typical of memory mapped file: quoting from
> Peter Lawrey "for burst of up to 10% of the main memory, it can sustain rates
> of 1 - 3 GB/second written. e.g. A laptop with 8 GB of memory might handle
> bursts of 800 MB at a rate of 1 GB per second. A server with 64 GB of memory
> might handle a burst of 6.5 GB at a rate of 3 GB per second".
> I want to improve the quality of the implementation by:
> 1- enforcing the original SequenceFile contract
> 2- replacing synchronized reads/writes with lock-free versions
> 3- replacing the EventExecutor with a Lock-Free (even wait-free) Array Queue
> + EventLoop's poller
> 4- addressing all the False-Sharing issues around all the AtomicLong
> instances used
> 5- reducing the garbage produced for the fast paths to 0
> [6- expose try methods to allow direct flow control]
> [7- replacing the Semaphore based rate limiter with a lock free one]
> That's are all proposals, what do you think about it?
> Regards
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
