[ https://issues.apache.org/jira/browse/IGNITE-15434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrey N. Gura updated IGNITE-15434: ------------------------------------ Fix Version/s: 3.0.0-alpha3 > Implement cursor for partitions > ------------------------------- > > Key: IGNITE-15434 > URL: https://issues.apache.org/jira/browse/IGNITE-15434 > Project: Ignite > Issue Type: Improvement > Reporter: Alexander Lapin > Assignee: Alexander Lapin > Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-alpha3 > > Time Spent: 5h > Remaining Estimate: 0h > > In order to unblock sql activities in required to have partition scanning > logic. Simple approach would be similar to meta storage range command: > cursors on the client side with corresponding raft commands that will manage > server side storage cursors. However given approach has some disadvantages, > for example, cursor's hasNext(), next(), close() etc are blocking by design > so it might lead (and as it's seen with Meta storage range will lead) to a > deadlock if response retrieval logic is blocked by synchronous get on hasNext > or similar. In order to overcome given issue we might use unblocking cursors > based on reactive flow model. It actually means that client side range/scan > should provide Publisher instance*.* > *Implementation details:* > Within InternalTable class new method was introduced: > {code:java} > /** > * Scans given partition, providing {@link Publisher<BinaryRow>} that > reflectively notifies about partition rows. > * @param p The partition. > * @param tx The transaction. > * @return {@link Publisher<BinaryRow>} t{@link Publisher<BinaryRow>} that > reflectively notifies about partition rows. > */ > @NotNull Publisher<BinaryRow> scan(int p, @Nullable Transaction tx); > {code} > Core publisher logic is implemented inside: > {code:java} > InternalTableImpl.PartitionScanPublisher.PartitionScanSubscription#PartitionScanSubscription > {code} > Please pay attention that inner batching logic is not implemented. > PartitionScanSubscription will request same amount of entries from Storage as > it was specified within subscription.request(). Seems that it doesn't worth > to implement it right now cause event reactive model wasn't approved yet. > Three new partition raft commands and corresponding handlers were added: > * ScanInitCommand > * ScanRetrieveBatchCommand > * ScanCloseCommand > All given commands are write ones, so it'll lead to full scan replication > over raft nodes. On the one hand such approach leads to transparent remapping > logic that eases sql quey processing logic in case of raft leadership > changes, on the other hand it requires more time for replication and more > space within raft log for such commands. In future such strong restrictions > might be be weakened. > -- This message was sent by Atlassian Jira (v8.3.4#803005)