[jira] [Commented] (HBASE-13784) Add Async Client Table API
[ https://issues.apache.org/jira/browse/HBASE-13784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15266221#comment-15266221 ] Hudson commented on HBASE-13784: FAILURE: Integrated in HBase-Trunk_matrix #885 (See [https://builds.apache.org/job/HBase-Trunk_matrix/885/]) HBASE-15744 Port over small format/text improvements from HBASE-13784 (stack: rev 15631a76f5404728ee7f9683668038700f3cda4a) * hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/RpcClient.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/RpcRetryingCallerWithReadReplicas.java * hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/RpcRetryingCaller.java > Add Async Client Table API > -- > > Key: HBASE-13784 > URL: https://issues.apache.org/jira/browse/HBASE-13784 > Project: HBase > Issue Type: New Feature >Reporter: Jurriaan Mous >Assignee: Jurriaan Mous > Attachments: ClientScanner-class-diagram.png, HBASE-13784-v1.patch, > HBASE-13784-v2.patch, HBASE-13784-v3.patch, HBASE-13784-v4.patch, > HBASE-13784-v5.patch, HBASE-13784-v6.patch, HBASE-13784-v7.patch, > HBASE-13784.patch > > > With the introduction of the Async HBase RPC Client it is possible to create > an Async Table API and more. This issue is focussed on creating a first async > Table API so it is possible to do any non deprecated Table call in an async > way. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13784) Add Async Client Table API
[ https://issues.apache.org/jira/browse/HBASE-13784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15265728#comment-15265728 ] Jurriaan Mous commented on HBASE-13784: --- I want to finish the work on this patch. Currently it is not compatible with the latest master. I now want to break it up to several sub issues so the work gets committed more easily. It is possible to break up the patch in several subtasks like refactoring some of the classes so they can be reused, add the needed extra calls to the RPC Clients, remove RpcControllerFactory from all the scans, some smaller code enhancements etc. So expect some smaller issues referring to this issue in the coming time. The good thing is that some (or very maybe all) parts can be considered for the 1.x branch. > Add Async Client Table API > -- > > Key: HBASE-13784 > URL: https://issues.apache.org/jira/browse/HBASE-13784 > Project: HBase > Issue Type: New Feature >Reporter: Jurriaan Mous >Assignee: Jurriaan Mous > Attachments: ClientScanner-class-diagram.png, HBASE-13784-v1.patch, > HBASE-13784-v2.patch, HBASE-13784-v3.patch, HBASE-13784-v4.patch, > HBASE-13784-v5.patch, HBASE-13784-v6.patch, HBASE-13784-v7.patch, > HBASE-13784.patch > > > With the introduction of the Async HBase RPC Client it is possible to create > an Async Table API and more. This issue is focussed on creating a first async > Table API so it is possible to do any non deprecated Table call in an async > way. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13784) Add Async Client Table API
[ https://issues.apache.org/jira/browse/HBASE-13784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14697975#comment-14697975 ] stack commented on HBASE-13784: --- I owe you a review [~jurmous]? (Was out, sorry). Add Async Client Table API -- Key: HBASE-13784 URL: https://issues.apache.org/jira/browse/HBASE-13784 Project: HBase Issue Type: New Feature Reporter: Jurriaan Mous Assignee: Jurriaan Mous Attachments: ClientScanner-class-diagram.png, HBASE-13784-v1.patch, HBASE-13784-v2.patch, HBASE-13784-v3.patch, HBASE-13784-v4.patch, HBASE-13784-v5.patch, HBASE-13784-v6.patch, HBASE-13784-v7.patch, HBASE-13784.patch With the introduction of the Async HBase RPC Client it is possible to create an Async Table API and more. This issue is focussed on creating a first async Table API so it is possible to do any non deprecated Table call in an async way. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13784) Add Async Client Table API
[ https://issues.apache.org/jira/browse/HBASE-13784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14585796#comment-14585796 ] Jurriaan Mous commented on HBASE-13784: --- bq. When we thinking asynctable would show up? 1.3 or 2.0? It is too late for 1.2. It is likely too disruptive for 1.3.. though if all tests pass and asynctable is additive and doesn't significantly disturb blocking HTable implementation, it could come in for 1.3. (I put reply here) One thing that is not done with this patch is that we need to delete the sync RpcClient implementation. I am thinking to make a separate patch for that. (Currently there are some TODOs in the patch where implementations are missing) Many async internals will fail with that client handling the calls. Maybe AsyncRpcClient is now faster since it removes an abstraction layer. The current code does disturb a bit of the replica and the Scanner code. (ScannerCallable, ScannerCallableWithReplica is used within current code) So replacing the default RpcClient without fallback together with some changes internally in scans is maybe to disruptive for 1.3. Or maybe we can do some performance tests before really deciding. What is the current schedule for 2.0 and 1.3? I await the landing of HBASE-13893 before continuing the work on this patch. I need the changes to the tests so I can implement the Async Table tests. Add Async Client Table API -- Key: HBASE-13784 URL: https://issues.apache.org/jira/browse/HBASE-13784 Project: HBase Issue Type: New Feature Reporter: Jurriaan Mous Assignee: Jurriaan Mous Attachments: ClientScanner-class-diagram.png, HBASE-13784-v1.patch, HBASE-13784-v2.patch, HBASE-13784-v3.patch, HBASE-13784-v4.patch, HBASE-13784-v5.patch, HBASE-13784-v6.patch, HBASE-13784.patch With the introduction of the Async HBase RPC Client it is possible to create an Async Table API and more. This issue is focussed on creating a first async Table API so it is possible to do any non deprecated Table call in an async way. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13784) Add Async Client Table API
[ https://issues.apache.org/jira/browse/HBASE-13784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14586062#comment-14586062 ] Jurriaan Mous commented on HBASE-13784: --- Added HBASE-13902 to remove the Sync RpcClientImpl which is needed for this issue to land. Add Async Client Table API -- Key: HBASE-13784 URL: https://issues.apache.org/jira/browse/HBASE-13784 Project: HBase Issue Type: New Feature Reporter: Jurriaan Mous Assignee: Jurriaan Mous Attachments: ClientScanner-class-diagram.png, HBASE-13784-v1.patch, HBASE-13784-v2.patch, HBASE-13784-v3.patch, HBASE-13784-v4.patch, HBASE-13784-v5.patch, HBASE-13784-v6.patch, HBASE-13784.patch With the introduction of the Async HBase RPC Client it is possible to create an Async Table API and more. This issue is focussed on creating a first async Table API so it is possible to do any non deprecated Table call in an async way. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13784) Add Async Client Table API
[
https://issues.apache.org/jira/browse/HBASE-13784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14580350#comment-14580350
]
Jurriaan Mous commented on HBASE-13784:
---
{quote}
Ok. Yes, it would be nice if you could slot in your AsyncTable everywhere to
make sure it works in 'sync' mode blocking on the promise. Can you use junit
parameterize (http://junit.org/apidocs/org/junit/runners/Parameterized.html) so
tests run with current HTable implementation and then again with your Async'd
HTable Implementation?
{quote}
Somehow I didn't yet know of the Parameterized Junit tests. It seems like a
good solution for this problem. I will look into it.
bq. If you want feedback on current state of patch, add it to
reviews.apache.org and paste link here and I (or others) can you a bit of
feedback. Thanks.
Will do with the next patch.
Add Async Client Table API
--
Key: HBASE-13784
URL: https://issues.apache.org/jira/browse/HBASE-13784
Project: HBase
Issue Type: New Feature
Reporter: Jurriaan Mous
Assignee: Jurriaan Mous
Attachments: HBASE-13784-v1.patch, HBASE-13784-v2.patch,
HBASE-13784-v3.patch, HBASE-13784-v4.patch, HBASE-13784.patch
With the introduction of the Async HBase RPC Client it is possible to create
an Async Table API and more. This issue is focussed on creating a first async
Table API so it is possible to do any non deprecated Table call in an async
way.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13784) Add Async Client Table API
[ https://issues.apache.org/jira/browse/HBASE-13784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14580899#comment-14580899 ] stack commented on HBASE-13784: --- bq. Question: What is a good name for the PromiseKeepers? I learned it is a loaded term. Don't worry about it. It is generic. I just cited a specific instance of a once rising culturally 'interesting' use of the term. Add Async Client Table API -- Key: HBASE-13784 URL: https://issues.apache.org/jira/browse/HBASE-13784 Project: HBase Issue Type: New Feature Reporter: Jurriaan Mous Assignee: Jurriaan Mous Attachments: HBASE-13784-v1.patch, HBASE-13784-v2.patch, HBASE-13784-v3.patch, HBASE-13784-v4.patch, HBASE-13784-v5.patch, HBASE-13784.patch With the introduction of the Async HBase RPC Client it is possible to create an Async Table API and more. This issue is focussed on creating a first async Table API so it is possible to do any non deprecated Table call in an async way. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13784) Add Async Client Table API
[ https://issues.apache.org/jira/browse/HBASE-13784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14571911#comment-14571911 ] stack commented on HBASE-13784: --- bq. I am thinking of either making a clean Table implementation using AsyncTable internally or using the AsyncTable already within HTable itself. Ok. Yes, it would be nice if you could slot in your AsyncTable everywhere to make sure it works in 'sync' mode blocking on the promise. Can you use junit parameterize (http://junit.org/apidocs/org/junit/runners/Parameterized.html) so tests run with current HTable implementation and then again with your Async'd HTable Implementation? You could choose a few of the important client-side junits and have the work in the two modes? Looking at TestFromClientSide, probably the biggest client-side set of tests, it still has a bunch of deprecated 'new HTable' usage. I could help by doing cleanup on this test so it all went via Connection to get Table instances.. Would then have to jigger Connection and ConnectionFactory per parameterized run to return pure HTable for one run as it does now and then your fancy async'y version for the next run. If you make a list of junit test suites you'd like to parameterize and note obstacles to your getting your AsyncTable in under them, I can help with unblocking your obstacles. Thanks [~jurmous] Add Async Client Table API -- Key: HBASE-13784 URL: https://issues.apache.org/jira/browse/HBASE-13784 Project: HBase Issue Type: New Feature Reporter: Jurriaan Mous Assignee: Jurriaan Mous Attachments: HBASE-13784-v1.patch, HBASE-13784-v2.patch, HBASE-13784-v3.patch, HBASE-13784-v4.patch, HBASE-13784.patch With the introduction of the Async HBase RPC Client it is possible to create an Async Table API and more. This issue is focussed on creating a first async Table API so it is possible to do any non deprecated Table call in an async way. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13784) Add Async Client Table API
[ https://issues.apache.org/jira/browse/HBASE-13784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14571913#comment-14571913 ] stack commented on HBASE-13784: --- If you want feedback on current state of patch, add it to reviews.apache.org and paste link here and I (or others) can you a bit of feedback. Thanks. Add Async Client Table API -- Key: HBASE-13784 URL: https://issues.apache.org/jira/browse/HBASE-13784 Project: HBase Issue Type: New Feature Reporter: Jurriaan Mous Assignee: Jurriaan Mous Attachments: HBASE-13784-v1.patch, HBASE-13784-v2.patch, HBASE-13784-v3.patch, HBASE-13784-v4.patch, HBASE-13784.patch With the introduction of the Async HBase RPC Client it is possible to create an Async Table API and more. This issue is focussed on creating a first async Table API so it is possible to do any non deprecated Table call in an async way. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13784) Add Async Client Table API
[
https://issues.apache.org/jira/browse/HBASE-13784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14569166#comment-14569166
]
Jurriaan Mous commented on HBASE-13784:
---
[~stack] I am still a bit doubting. I am thinking of either making a clean
Table implementation using AsyncTable internally or using the AsyncTable
already within HTable itself. I would then call the async method within the
sync table call to fetch the promise. Something like the code below.
{code}
@Override
public Result get(final Get get) throws IOException {
try {
return asyncTable.get(get).get();
} catch (InterruptedException e) {
throw new InterruptedIOException(e.getMessage());
} catch (ExecutionException e) {
throw (IOException) e.getCause();
}
}
{code}
For a new Table implementation to work within the tests I have to make sure
Connection returns the new implementation and all relevant tests work through
fetching a table through Connection. This way I am certain the new Api is all
ok against all existing tests. It is too difficult to replicate all the Api
tests.
Add Async Client Table API
--
Key: HBASE-13784
URL: https://issues.apache.org/jira/browse/HBASE-13784
Project: HBase
Issue Type: New Feature
Reporter: Jurriaan Mous
Assignee: Jurriaan Mous
Attachments: HBASE-13784-v1.patch, HBASE-13784-v2.patch,
HBASE-13784-v3.patch, HBASE-13784-v4.patch, HBASE-13784.patch
With the introduction of the Async HBase RPC Client it is possible to create
an Async Table API and more. This issue is focussed on creating a first async
Table API so it is possible to do any non deprecated Table call in an async
way.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13784) Add Async Client Table API
[ https://issues.apache.org/jira/browse/HBASE-13784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14568143#comment-14568143 ] stack commented on HBASE-13784: --- bq. I am thinking of letting HTable use the AsyncTable methods where the promise is called directly. Sorry for being dumb but can you give example [~jurmous] ? Thanks. Add Async Client Table API -- Key: HBASE-13784 URL: https://issues.apache.org/jira/browse/HBASE-13784 Project: HBase Issue Type: New Feature Reporter: Jurriaan Mous Assignee: Jurriaan Mous Attachments: HBASE-13784-v1.patch, HBASE-13784-v2.patch, HBASE-13784-v3.patch, HBASE-13784.patch With the introduction of the Async HBase RPC Client it is possible to create an Async Table API and more. This issue is focussed on creating a first async Table API so it is possible to do any non deprecated Table call in an async way. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13784) Add Async Client Table API
[ https://issues.apache.org/jira/browse/HBASE-13784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14564548#comment-14564548 ] Jurriaan Mous commented on HBASE-13784: --- I sent out a mail to the mailinglist. Below my latest suggestion as I also added it to the mail. This version also includes a suggestion for the Coprocessor calls. I am currently proposing the following methods in the simpler AsyncTable interface: * exists(Get): ResponsePromiseBoolean * exists(ListGet): ResponsePromiseBoolean[] * get(Get): ResponsePromiseResult * get(ListGet): ResponsePromiseResult[] * mutate(Mutation): ResponsePromiseVoid - Instead of separate Put, Delete, Increment, IncrementValue and Append methods * checkAndMutate(byte[], byte[], byte[], CompareOp, byte[], Mutation): ResponsePromiseVoid - Will not accept Append and Increment * checkAndMutate(byte[], byte[], byte[], CompareOp, byte[], RowMutations): ResponsePromiseVoid - Will not accept Append and Increment * getScanner(Scan): ResponsePromiseVoid - AsyncResultScanner * coprocessorService(byte[]): AsyncCoprocessorRpcService * coprocessorService(ClassT extends Service, byte[] startkey, byte[] endkey, Batch.CallT,R callable): ResponsePromiseMapbyte[], R * batchCoprocessorService(Descriptors.MethodDescriptor, Message, byte[] startKey, byte[] endKey, R responsePrototype): ResponsePromiseMapbyte[], R To send multiple mutations you need to use a BufferedMutator which handles writeBuffers internally. I will look into making it possible to use it with Promises. Batch operations will be able to be done by a PromiseKeeper which is a promise to which other promises can be added. There will be a new AsyncResultScanner which handles incoming batches of result. It will not be possible to do next on it since this does not makes sense in an async context. There will be however a way to request a new batch with a promise. Add Async Client Table API -- Key: HBASE-13784 URL: https://issues.apache.org/jira/browse/HBASE-13784 Project: HBase Issue Type: New Feature Reporter: Jurriaan Mous Assignee: Jurriaan Mous Attachments: HBASE-13784-v1.patch, HBASE-13784.patch With the introduction of the Async HBase RPC Client it is possible to create an Async Table API and more. This issue is focussed on creating a first async Table API so it is possible to do any non deprecated Table call in an async way. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13784) Add Async Client Table API
[
https://issues.apache.org/jira/browse/HBASE-13784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14564485#comment-14564485
]
Jurriaan Mous commented on HBASE-13784:
---
bq. Well, we have timeouts and retries now (after you fix the 'stop' missed in
current patch?) Adding another timeout in RetryingResponsePromise would be on
top of these?
Well I think I will currently leave the logic alone and it can always be
addressed in a follow-up issue. It seems I was mistaken with forgetting
end-retry and missed that I copied some code of the current retry handler into
its failure handler. So currently the async retry handler works as the current
sync one.
{quote}
We need these?
mutate(ListMutation): ResponsePromiseVoid - Will not accept Append and
Increment because of nonce requirement.
mutate(RowMutations): ResponsePromiseVoid - Will not accept Append and
Increment
mutate(ListRowMutations): ResponsePromiseVoid - Will not accept Append and
Increment
Is it just because Append and Increment are not Mutations? Lets fix that rather
than do above?
{quote}
Append and Increment are mutations but I think it was chosen in the current
workflow to not handle them because they are nonce dependent and thus only
allowed to be sent one at a time to prevent problems.
I don't entirely like that there are 4 mutate methods. There are methods needed
to send multiple Mutations either grouped by row by RowMutation or just a list
of mutations to be able to write buffer them. (Since it is optimal to send
multiple mutations in one request)
Maybe we could remove this functionality completely from AsyncTable and ask
people to make an async BufferedMutator if they want to send multiple
mutations.
{quote}
(PromiseKeeping and changing scan... FYI, Scan has had a bunch of work done
since you were around last ... did you notice? It should be easier to fit it to
your new form that previous).
{quote}
I am not aware of all changes so will certainly take a deeper look into those
before trying to fit in an async workflow.
{quote}
Suggest you float message on dev list to get more input on your new API set.
{quote}
Will do!
Add Async Client Table API
--
Key: HBASE-13784
URL: https://issues.apache.org/jira/browse/HBASE-13784
Project: HBase
Issue Type: New Feature
Reporter: Jurriaan Mous
Assignee: Jurriaan Mous
Attachments: HBASE-13784-v1.patch, HBASE-13784.patch
With the introduction of the Async HBase RPC Client it is possible to create
an Async Table API and more. This issue is focussed on creating a first async
Table API so it is possible to do any non deprecated Table call in an async
way.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13784) Add Async Client Table API
[ https://issues.apache.org/jira/browse/HBASE-13784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14563727#comment-14563727 ] stack commented on HBASE-13784: --- bq. With the work I am doing I was trying to change as little as possible to the current behaviour. Fair enough. Suggestions for what might be improved upon given your now intimate knowledge of rpc appreciated as follow ons. bq. Should I add an overall timeout to the RetryingResponsePromise? (It seems I forgot to add a stop after max amount of retries has been reached in current patch) Well, we have timeouts and retries now (after you fix the 'stop' missed in current patch?) Adding another timeout in RetryingResponsePromise would be on top of these? Thanks for taking a look at [~louiscryan]'s work. bq. I would like to propose a bit different and simpler Api than is currently implemented in Table. No objection here. We need these? mutate(ListMutation): ResponsePromiseVoid - Will not accept Append and Increment because of nonce requirement. mutate(RowMutations): ResponsePromiseVoid - Will not accept Append and Increment mutate(ListRowMutations): ResponsePromiseVoid - Will not accept Append and Increment Is it just because Append and Increment are not Mutations? Lets fix that rather than do above? How we fix it so you don't need RowMutation and Mutation? Otherwise, all looks good (PromiseKeeping and changing scan... FYI, Scan has had a bunch of work done since you were around last ... did you notice? It should be easier to fit it to your new form that previous). Suggest you float message on dev list to get more input on your new API set. Nice work [~jurmous] Add Async Client Table API -- Key: HBASE-13784 URL: https://issues.apache.org/jira/browse/HBASE-13784 Project: HBase Issue Type: New Feature Reporter: Jurriaan Mous Assignee: Jurriaan Mous Attachments: HBASE-13784-v1.patch, HBASE-13784.patch With the introduction of the Async HBase RPC Client it is possible to create an Async Table API and more. This issue is focussed on creating a first async Table API so it is possible to do any non deprecated Table call in an async way. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13784) Add Async Client Table API
[
https://issues.apache.org/jira/browse/HBASE-13784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14562659#comment-14562659
]
Jurriaan Mous commented on HBASE-13784:
---
Thanks for your comments!
{quote}
This would be nice:
38 * TODO: this class is actually tied to one region, because most of the
paths make use of
39 * the regioninfo part of location when building requests. The only
reason it works for
40 * multi-region requests (e.g. batch) is that they happen to not use the
region parts.
41 * This could be done cleaner (e.g. having a generic parameter and 2
derived classes,
42 * RegionCallable and actual RegionServerCallable with ServerName.
...because as you note, otherwise the TableName is superfluous here:
{quote}
That comment was already in RegionServerCallable which I renamed to
AbstractRegionServerCallable to be able to make an Async version which shares
code. So that good suggestion falls outside of my work :)
{quote}
We need this method below?
protected void setLocation(final HRegionLocation location) {
It can't figure it out internally? Has to have it passed in?
We need this method below?
protected void setLocation(final HRegionLocation location) {
It can't figure it out internally? Has to have it passed in?
The sleep, throwable, getExceptionMessageAdditionalDetail , etc., methods were
there already?
Should this be set on construction?
{quote}
It was already there in the non abstract version of the class. Changing this
flow falls outside of the scope of the issue.
{quote}
abstract void setClientByServiceName(ServerName serviceName) throws IOException;
Get a new instance if you want to connect to a different service?
{quote}
Yes with the blocking variants it will set a Stub on prepare and with the async
variant it will set an AsyncRpcChannel internally. Those channels are currently
bound to the service due to the protocol. I left it up to the implementing
class to actually set the right typed properties.
{quote}
What is your understanding around how timeouts work now? Have you ported the
mess we have in old client where we have retries and timeouts ... with it
difficult to set an overall timeout on a connection attempt?
{quote}
With the work I am doing I was trying to change as little as possible to the
current behaviour. The current behaviour is implemented in the new
RetryingResponsePromise.
It is also possible to do a timer (With Netty HashedWheelTimer) on the
RetryingResponsePromise so it shuts down after some amount of time of waiting.
Should I add an overall timeout to the RetryingResponsePromise? (It seems I
forgot to add a stop after max amount of retries has been reached in current
patch)
{quote}
Louis Ryan This route would jibe w/ the PoC you are doing?
{quote}
I just now discovered [~louiscryan] his work. From what I can see is that Louis
is building a new Client and Server RPC client based on GRPC. He builds on top
of some abstractions I did while creating the AsyncRpcClient so is good to see
it being used further. It is nice to have a generalized implementation of the
RPC layer with HTTP2, Streams and things like extra GZip ChannelEncoders.
A new async client table interface would in principle not conflict with his
work although to make it as efficient as possible I am exposing some internals
of the async client like its internal promise and eventloop for promise
creation on top of Netty Thread pool. With the code that I see in the github
and GRPC async Calls it should be possible to add those constructs to his
implementation. I have already abstracted the needed methods to interfaces and
since his implementation is also Netty based it should be not difficult to add
the methods to his GrpcClientImpl and expose an AsyncRpcChannel implementation
to communicate with.
Add Async Client Table API
--
Key: HBASE-13784
URL: https://issues.apache.org/jira/browse/HBASE-13784
Project: HBase
Issue Type: New Feature
Reporter: Jurriaan Mous
Assignee: Jurriaan Mous
Attachments: HBASE-13784-v1.patch, HBASE-13784.patch
With the introduction of the Async HBase RPC Client it is possible to create
an Async Table API and more. This issue is focussed on creating a first async
Table API so it is possible to do any non deprecated Table call in an async
way.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13784) Add Async Client Table API
[ https://issues.apache.org/jira/browse/HBASE-13784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14562929#comment-14562929 ] Jurriaan Mous commented on HBASE-13784: --- I am looking into what and how to add actions to the AsyncTable. I would like to propose a bit different and simpler Api than is currently implemented in Table. All mutations go through mutate method and batch is eliminated. (I will ignore the coprocessor calls for now until the basic Api is clear) * exists(Get): ResponsePromiseBoolean * exists(ListGet): ResponsePromiseBoolean[] * get(Get): ResponsePromiseResult * get(ListGet): ResponsePromiseResult[] * mutate(Mutation): ResponsePromiseVoid - Instead of separate Put, Delete, Increment, IncrementValue and Append methods * mutate(ListMutation): ResponsePromiseVoid - Will not accept Append and Increment because of nonce requirement. * mutate(RowMutations): ResponsePromiseVoid - Will not accept Append and Increment * mutate(ListRowMutations): ResponsePromiseVoid - Will not accept Append and Increment * checkAndMutate(byte[], byte[], byte[], CompareOp, byte[], Mutation): ResponsePromiseVoid - Will not accept Append and Increment * checkAndMutate(byte[], byte[], byte[], CompareOp, byte[], RowMutations): ResponsePromiseVoid - Will not accept Append and Increment * getScanner(Scan): ResponsePromiseVoid - AsyncResultScanner Mutate with multiple mutations will check the writeBuffer and write packages to each region if its writeBuffer limit is reached or when there are no mutations left. Instead of the batch calls I would like to introduce a Promise that can manage multiple promises which I would like to call it PromiseKeeper. You add the promises of multiple calls to that PromiseKeeper and if all succeed that promise fires or if one fails that promise also fails. You can then get the results from an Array in the same order as you added the promises. This is also to replace the AsyncProcess. AsyncResultScanner. Am thinking of changing the workflow of the scanner for Async to better fit an Async workflow. Instead of next calls and an internal buffer I would like to create a nextBatch call and a way to set batch size. So this way the call and the result handling are combined. A promise on each next call makes no sense. What do you think? Add Async Client Table API -- Key: HBASE-13784 URL: https://issues.apache.org/jira/browse/HBASE-13784 Project: HBase Issue Type: New Feature Reporter: Jurriaan Mous Assignee: Jurriaan Mous Attachments: HBASE-13784-v1.patch, HBASE-13784.patch With the introduction of the Async HBase RPC Client it is possible to create an Async Table API and more. This issue is focussed on creating a first async Table API so it is possible to do any non deprecated Table call in an async way. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13784) Add Async Client Table API
[ https://issues.apache.org/jira/browse/HBASE-13784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14560795#comment-14560795 ] Jurriaan Mous commented on HBASE-13784: --- Forgot to mention: RemoteHTable does not have an implementation for asyncGet. It is quite a task to also make an async client for the REST gateway. This would strengthen the argument to make a separate AsyncTable interface with methods which returns Promises. But before doing so I like your opinions on the directions chosen. Add Async Client Table API -- Key: HBASE-13784 URL: https://issues.apache.org/jira/browse/HBASE-13784 Project: HBase Issue Type: New Feature Reporter: Jurriaan Mous Assignee: Jurriaan Mous Attachments: HBASE-13784.patch With the introduction of the Async HBase RPC Client it is possible to create an Async Table API and more. This issue is focussed on creating a first async Table API so it is possible to do any non deprecated Table call in an async way. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13784) Add Async Client Table API
[
https://issues.apache.org/jira/browse/HBASE-13784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14561385#comment-14561385
]
stack commented on HBASE-13784:
---
First, sweet!
High-level, I think adding a new AsyncTable Interface is a clean way of
bringing in this new functionality. It follows nicely the Table model: get one
as you need it and then close when done (lightweight). Alternatives, ones you
have already pondered, would be adding a marker Interface on Table -- say Async
-- but then you'd have to have methods named asyncGet in your Async Interface
because you can't override if only the return is different... That'd be ugly.
Not sure I understand all that is going on but here are a few remarks on the
patch:
This would be nice:
38 * TODO: this class is actually tied to one region, because most of the
paths make use of
39 * the regioninfo part of location when building requests. The
only reason it works for
40 * multi-region requests (e.g. batch) is that they happen to not
use the region parts.
41 * This could be done cleaner (e.g. having a generic parameter
and 2 derived classes,
42 * RegionCallable and actual RegionServerCallable with
ServerName.
...because as you note, otherwise the TableName is superfluous here:
public AbstractRegionServerCallable(Connection connection, TableName
tableName, byte[] row)
We need this method below?
protected void setLocation(final HRegionLocation location) {
It can't figure it out internally? Has to have it passed in?
The sleep, throwable, getExceptionMessageAdditionalDetail , etc., methods were
there already?
Should this be set on construction?
abstract void setClientByServiceName(ServerName serviceName) throws
IOException;
Get a new instance if you want to connect to a different service?
What is your understanding around how timeouts work now? Have you ported the
mess we have in old client where we have retries and timeouts ... with it
difficult to set an overall timeout on a connection attempt?
On your other comments:
+ I think it fine exposing netty internals; i.e. promise -- it well-known (You
going to hook up the listener on promise as over here
https://github.com/jurmous/etcd4j/blob/master/src/client/java/mousio/client/promises/ResponsePromise.java
-- smile)
+ Regards, It is in my opinion best to remove the blocking implementation
instead of engineering fake alternatives., and have them blocking run using
async and promise? That'd be fine.
[~sduskis] Any input on new AsyncTable Interface?
[~louiscryan] This route would jibe w/ the PoC you are doing?
Add Async Client Table API
--
Key: HBASE-13784
URL: https://issues.apache.org/jira/browse/HBASE-13784
Project: HBase
Issue Type: New Feature
Reporter: Jurriaan Mous
Assignee: Jurriaan Mous
Attachments: HBASE-13784-v1.patch, HBASE-13784.patch
With the introduction of the Async HBase RPC Client it is possible to create
an Async Table API and more. This issue is focussed on creating a first async
Table API so it is possible to do any non deprecated Table call in an async
way.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
