[
https://issues.apache.org/jira/browse/HDDS-16870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrey Yarovoy reassigned HDDS-16870:
-------------------------------------
Assignee: Andrey Yarovoy
> EC: Support stateless positional read for EC keys
> -------------------------------------------------
>
> Key: HDDS-16870
> URL: https://issues.apache.org/jira/browse/HDDS-16870
> Project: Apache Ozone
> Issue Type: Improvement
> Components: EC Client
> Reporter: Andrey Yarovoy
> Assignee: Andrey Yarovoy
> Priority: Major
>
> h2. Problem
> HDDS-15424 added stateless positional read to \{{MultipartInputStream}}, but
> its
> constructor enables it only when all parts are \{{BlockInputStream}} or all
> are
> {\{StreamBlockInputStream}}. EC parts are \{{ECBlockInputStreamProxy}}, which
> does
> not implement \{{PartInputStream.readPositioned}} (the default throws
> {\{UnsupportedOperationException}}). EC keys therefore always use the
> {\{OzoneFSInputStream}} fallback: under \{{positionedReadLock}} it saves the
> position, seeks, reads and seeks back. As a result:
> * preads on one EC stream are serialized, and they also serialize with the
> stream's sequential reads;
> * every pread moves and restores the shared stream position, with the seek
> costs that come with it (see sub-task 1).
> h2. HDFS behavior
> {\{DFSStripedInputStream}} pread is not synchronized on the stream. It uses
> its
> own set of block readers (\{{preaderInfos}}) and reads (and, if degraded,
> decodes) only the requested range, so concurrent preads on one stream are safe
> and do not disturb the sequential read position.
> h2. Proposal
> * Implement \{{readPositioned}} on \{{ECBlockInputStreamProxy}} without
> touching
> the stream's sequential state:
> ** healthy: map the range to per-index spans and call \{{readPositioned}} on
> the
> per-index STANDALONE block streams, which are already stateless (in parallel
> with the sub-task 2 executor, or serially as a first step). Opening a
> per-index stream must become safe for concurrent callers;
> ** degraded: per-call buffers and decoder, reading and decoding only the range
> (sub-task 4);
> ** a failure in one pread falls back to a degraded read for that call; it does
> not switch the sequential reader's mode.
> * Admit EC parts in \{{MultipartInputStream}}, for example by letting each
> part
> report whether it supports stateless positional read instead of the
> {\{instanceof}} check.
> TDE keys also need \{{KeyInputStream}} to implement \{{PositionedReadable}},
> so
> that Hadoop's \{{CryptoInputStream}} can use its stateless pread path; that is
> HDDS-16396.
> h2. Compatibility
> Client only. No wire, configuration or data change.
> h2. Testing
> * Unit: \{{readPositioned}} returns the right bytes for every offset/length
> class,
> healthy and with 1 to parity-count missing indexes, and does not change
> {\{getPos()}}.
> * Unit: concurrent \{{readPositioned}} calls on one stream from several
> threads,
> interleaved with sequential reads, all return correct data.
> * Integration: extend the concurrent pread test of HDDS-16396 to EC keys.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
