GitHub user pdu-mn1 opened a pull request:
https://github.com/apache/samza/pull/593
SAMZA-1763: Add async methods to Table API
Currently, Table API only has blocking/sync methods which limit the
throughput of remote tables. This change adds async methods to the API
to enable high throughput remote table accesses through usage of async
IO. The new methods are added to ReadableTable and ReadWriteTable. A
high level summary of the change is below:
- added async methods to table ReadableTable and ReadWriteTable.
- added async methods to TableRead/WriteFunction
- CompletableFuture is used for the async abstraction
- CachingTable are updated to support async methods
- added default impls for sync methods backed by async in table functions
- added helper class, Throttler/AsyncHelper to ease table development
- fixed existing test cases with table implementations
- added more thorough unit tests to RemoteTable CRUD methods
Additionally remove explicit check of config entries for remote table in
TestTableDescriptorsProvider since there is already a test case on
RemoteTableDescriptor.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/pdu-mn1/samza async-table-api-futures
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/samza/pull/593.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #593
----
commit 0fdc9f7982a0b241a78528471c9baa09ff02ad8a
Author: Peng Du <pdu@...>
Date: 2018-06-27T23:32:02Z
SAMZA-1763: Add async methods to Table API
Currently, Table API only has blocking/sync methods which limit the
throughput of remote tables. This change adds async methods to the API
to enable high throughput remote table accesses through usage of async
IO. The new methods are added to ReadableTable and ReadWriteTable. A
high level summary of the change is below:
- added async methods to table ReadableTable and ReadWriteTable.
- added async methods to TableRead/WriteFunction
- CompletableFuture is used for the async abstraction
- CachingTable are updated to support async methods
- added default impls for sync methods backed by async in table functions
- added helper class, Throttler/AsyncHelper to ease table development
- fixed existing test cases with table implementations
- added more thorough unit tests to RemoteTable CRUD methods
Additionally remove explicit check of config entries for remote table in
TestTableDescriptorsProvider since there is already a test case on
RemoteTableDescriptor.
----
---
