[
https://issues.apache.org/jira/browse/IGNITE-15434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Lapin updated IGNITE-15434:
-------------------------------------
Description:
In order to unblock sql activities in required to have partition scanning
logic. Simple approach would be similar to meta storage range command: cursors
on the client side with corresponding raft commands that will manage server
side storage cursors. However given approach has some disadvantages, for
example, cursor's hasNext(), next(), close() etc are blocking by design so it
might lead (and as it's seen with Meta storage range will lead) to a deadlock
if response retrieval logic is blocked by synchronous get on hasNext or
similar. In order to overcome given issue we might use unblocking cursors based
on reactive flow model. It actually means that client side range/scan should
provide Publisher instance*.*
*Implementation details:*
Within InternalTable class new method was introduced:
{code:java}
/**
* Scans given partition, providing {@link Publisher<BinaryRow>} that
reflectively notifies about partition rows.
* @param p The partition.
* @param tx The transaction.
* @return {@link Publisher<BinaryRow>} t{@link Publisher<BinaryRow>} that
reflectively notifies about partition rows.
*/
@NotNull Publisher<BinaryRow> scan(int p, @Nullable Transaction tx);
{code}
Core publisher logic is implemented inside:
{code:java}
InternalTableImpl.PartitionScanPublisher.PartitionScanSubscription#PartitionScanSubscription
{code}
Please pay attention that inner batching logic is not implemented.
PartitionScanSubscription will request same amount of entries from Storage as
it was specified within subscription.request(). Seems that it doesn't worth to
implement it right now cause event reactive model wasn't approved yet.
Three new partition raft commands and corresponding handlers were added:
* ScanInitCommand
* ScanRetrieveBatchCommand
* ScanCloseCommand
All given commands are write ones, so it'll lead to full scan replication over
raft nodes. On the one hand such approach leads to transparent remapping logic
that eases sql quey processing logic in case of raft leadership changes, on the
other hand it requires more time for replication and more space within raft log
for such commands. In future such strong restrictions might be be weakened.
was:
In order to unblock sql activities in required to have partition scanning
logic. Simple approach would be similar to meta storage range command: cursors
on the client side with corresponding raft commands that will manage server
side storage cursors. However given approach has some disadvantages, for
example, cursor's hasNext(), next(), close() etc are blocking by design so it
might lead (and as it's seen with Meta storage range will lead) to a deadlock
if response retrieval logic is blocked by synchronous get on hasNext or
similar. In order to overcome given issue we might use unblocking cursors based
on reactive flow model. It actually means that client side range/scan should
provide Publisher instance*.*
*Implementation details:*
Within InternalTable class new method was introduced:
{code:java}
/**
* Scans given partition, providing {@link Publisher<BinaryRow>} that
reflectively notifies about partition rows.
* @param p The partition.
* @param tx The transaction.
* @return {@link Publisher<BinaryRow>} t{@link Publisher<BinaryRow>} that
reflectively notifies about partition rows.
*/
@NotNull Publisher<BinaryRow> scan(int p, @Nullable Transaction tx);
{code}
Core publisher logic is implemented inside:
{code:java}
InternalTableImpl.PartitionScanPublisher.PartitionScanSubscription#PartitionScanSubscription
{code}
Please pay attention that inner batching logic is not implemented.
PartitionScanSubscription will request same amount of entries from Storage as
it was specified within subscription.request(n). Seems that it doesn't worth to
implement it right now cause event reactive model wasn't approved yet.
Three new partition raft commands and corresponding handlers were added:
* ScanInitCommand
* ScanRetrieveBatchCommand
* ScanCloseCommand
All given commands are write ones, so it'll lead to full scan replication over
raft nodes. On the one hand such approach leads to transparent remapping logic
that eases sql quey processing logic in case of raft leadership changes, on the
other hand it requires more time for replication and more space within raft log
for such commands. In future such strong restrictions might be be weakened.
> Implement cursor for partitions
> -------------------------------
>
> Key: IGNITE-15434
> URL: https://issues.apache.org/jira/browse/IGNITE-15434
> Project: Ignite
> Issue Type: Improvement
> Reporter: Alexander Lapin
> Assignee: Alexander Lapin
> Priority: Major
> Labels: ignite-3
> Time Spent: 10m
> Remaining Estimate: 0h
>
> In order to unblock sql activities in required to have partition scanning
> logic. Simple approach would be similar to meta storage range command:
> cursors on the client side with corresponding raft commands that will manage
> server side storage cursors. However given approach has some disadvantages,
> for example, cursor's hasNext(), next(), close() etc are blocking by design
> so it might lead (and as it's seen with Meta storage range will lead) to a
> deadlock if response retrieval logic is blocked by synchronous get on hasNext
> or similar. In order to overcome given issue we might use unblocking cursors
> based on reactive flow model. It actually means that client side range/scan
> should provide Publisher instance*.*
> *Implementation details:*
> Within InternalTable class new method was introduced:
> {code:java}
> /**
> * Scans given partition, providing {@link Publisher<BinaryRow>} that
> reflectively notifies about partition rows.
> * @param p The partition.
> * @param tx The transaction.
> * @return {@link Publisher<BinaryRow>} t{@link Publisher<BinaryRow>} that
> reflectively notifies about partition rows.
> */
> @NotNull Publisher<BinaryRow> scan(int p, @Nullable Transaction tx);
> {code}
> Core publisher logic is implemented inside:
> {code:java}
> InternalTableImpl.PartitionScanPublisher.PartitionScanSubscription#PartitionScanSubscription
> {code}
> Please pay attention that inner batching logic is not implemented.
> PartitionScanSubscription will request same amount of entries from Storage as
> it was specified within subscription.request(). Seems that it doesn't worth
> to implement it right now cause event reactive model wasn't approved yet.
> Three new partition raft commands and corresponding handlers were added:
> * ScanInitCommand
> * ScanRetrieveBatchCommand
> * ScanCloseCommand
> All given commands are write ones, so it'll lead to full scan replication
> over raft nodes. On the one hand such approach leads to transparent remapping
> logic that eases sql quey processing logic in case of raft leadership
> changes, on the other hand it requires more time for replication and more
> space within raft log for such commands. In future such strong restrictions
> might be be weakened.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
