[
https://issues.apache.org/jira/browse/ASTERIXDB-2540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chen Luo resolved ASTERIXDB-2540.
---------------------------------
Resolution: Implemented
> Optimize Performance Stability of Storage
> -----------------------------------------
>
> Key: ASTERIXDB-2540
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2540
> Project: Apache AsterixDB
> Issue Type: Improvement
> Components: STO - Storage
> Reporter: Chen Luo
> Assignee: Chen Luo
> Priority: Major
>
> This is one of a series of improvements to optimize the performance stability
> of our storage subsystem, which suffers from a number of problems. The end
> result is that there are periodically write stalls during data ingestion,
> even though the ingestion speed is relatively low.
> This improvement will deal with the following issues:
> 1. Bypass all queuing of disk writes during LSM flush and merge operations.
> Queuing (by BufferCache and IOManager) will cause serious problems to the
> fairness of disk writes. Thus, a small flush operation could be severely
> interfered with a large merge operation and would take a much longer time to
> finish.
> 2. Perform regular disk forces during flush and merge operations (16MB by
> default). This is very helpful to limit the I/O queue length of the file
> system and provides fairness to queries and other writers. This optimization
> has been implemented in most storage systems today, including Couchbase
> Server.
> 3. Optionally, add the support for rate limiting of disk writes to ensure the
> performance stability of queries. The user can configure the maximum disk
> write bandwidth for each dataset. This ensures that the system can provide
> stable performance for both queries and writes, even with large background
> merges.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
