I resolved HBASE-3904 because there was no solution that everyone agreed on.
On Sat, Jul 9, 2011 at 12:48 PM, M. C. Srivas wrote:
> Its not clear from hbase-3904 what the issues are. If there's some code
> relying on isTableAvailable, that code is inherently broken.
>
> 1. isTa
Its not clear from hbase-3904 what the issues are. If there's some code
relying on isTableAvailable, that code is inherently broken.
1. isTableAvailable() is never reliable, because
(a) if it returns true, the table can disappear immediately after the
call finishes, or
(b) the tabl
Hi,
I want to get your opinion on what should be done for HBASE-3904 beyond
making createTable() synchronous.
Thanks
Comment for this issue is welcome.
Thanks
On Mon, Jun 20, 2011 at 5:38 PM, Ted Yu wrote:
> Can we wrap up 3904 with patch v5 and the following addition to javadoc of
> createTableAsync() ?
>
> If the client receives socket timeout exception, that implies master hasn't
> done populating .META. t
Here is code from HBaseClient:
private void handleTimeout(SocketTimeoutException e) throws
IOException {
if (shouldCloseConnection.get() || !running.get() ||
remoteId.rpcTimeout > 0) {
throw e;
}
BTW I think the following javadoc change is better than my
Shouldn't the ipc client ping be keeping the IPC alive?
On Mon, Jun 20, 2011 at 5:38 PM, Ted Yu wrote:
> Can we wrap up 3904 with patch v5 and the following addition to javadoc of
> createTableAsync() ?
>
> If the client receives socket timeout exception, that implies master hasn't
> done populat
Can we wrap up 3904 with patch v5 and the following addition to javadoc of
createTableAsync() ?
If the client receives socket timeout exception, that implies master hasn't
done populating .META. table with new region info.
Meaning HConnection.isTableAvailable() may return true prematurely.
On Mon
Well in that case you know how many regions you expect, we could catch
the socket exception and then check .META. until it's done or return
the call right away and keep checking .META.
J-D
On Mon, Jun 20, 2011 at 3:41 PM, Ted Yu wrote:
> Vidhyashankar did encounter socket timeout.
> I think even
Vidhyashankar did encounter socket timeout.
I think even with HBASE-4010 implemented, 70k region table creation would
still produce socket timeout.
On Mon, Jun 20, 2011 at 3:30 PM, Jean-Daniel Cryans wrote:
> Could be a socket timeout, was able to get one with just 200 regions,
> then calling isT
Could be a socket timeout, was able to get one with just 200 regions,
then calling isTableAvailable would trigger the case that was
described.
J-D
On Mon, Jun 20, 2011 at 3:28 PM, Ted Yu wrote:
> J-D:
> What's your assessment of why isTableAvailable() was inaccurate for a table
> with 70k region
J-D:
What's your assessment of why isTableAvailable() was inaccurate for a table
with 70k regions ?
Thanks
On Mon, Jun 20, 2011 at 3:01 PM, Jean-Daniel Cryans wrote:
> That's not how it works, whether the createTable call is sync or not
> the regions are first created in .META. before the call c
That's not how it works, whether the createTable call is sync or not
the regions are first created in .META. before the call can return.
And looking at that code, it seems it's highly inefficient. We should
not change the enable flag for every region, we should batch put all
the .META. rows and we
>From Vidhyashankar @ 20/May/11 05:53:
Hence there might be a case when all regions are indeed fully assigned in
META but it is just that the master is yet to populate META with the rest of
the regions.
The above means scanning .META. alone wouldn't guarantee that all the
regions of the table are
Can we simplify the following with this:
1st iteration of meta scanning, calculate numRegs = number of offline
regions + number of online regions (for underlying table)
subsequent iterations of meta scanning, if number of online regions (for
underlying table) reaches numRegs, break.
Cheers
On Mon
Why not the following pseudo-code for isTableAvailable:
scanMeta(table, Set online, Set offline) {
scan meta, inserting into the above sets any regions found for the
table in question
}
Set everSeenOnline = new Set();
while (true) {
Set online = new Set();
Set offline = new Set();
scanMet
I can update javadoc for createTableAsync based on patch v5.
How does that sound ?
Thanks
On Mon, Jun 20, 2011 at 10:55 AM, Jean-Daniel Cryans wrote:
> To me the inelegance of that solution outweighs its benefits i.e. I'd
> much rather have an imprecise isTableAvailable in the case the user
> go
To me the inelegance of that solution outweighs its benefits i.e. I'd
much rather have an imprecise isTableAvailable in the case the user
goes through createTableAsync (we could add a bit more doc) than
having the number of original regions stuck in the HTD.
J-D
On Mon, Jun 20, 2011 at 10:31 AM,
So passing initial region count through HTD is out of question ?
I think it requires less change than modifying HConnection.
Thanks
On Mon, Jun 20, 2011 at 10:27 AM, Stack wrote:
> Yeah, its not intuitive but failing your passing isTableAvailable the
> knowledge it needs, then its going to have
Yeah, its not intuitive but failing your passing isTableAvailable the
knowledge it needs, then its going to have to do heuristics to figure
table deploy. We can work on what these will look like -- region is
open but not split, no region opened in last N minutes or seconds --
but I think it will a
I put patch v5 on 3904.
When user calls HBaseAdmin.createTableAsync() to create the table,
HCM.isTableAvailable() is still vulnerable because currently it doesn't
known the desired number of regions.
We can add a new parameter to HCM.isTableAvailable() but
1. HConnection needs to be modified to acc
On Thu, Jun 16, 2011 at 5:43 PM, Ted Yu wrote:
> HBA.createTable() takes too long to execute. Client receives Socket timeout
> exception.
This is because the create is running in the master? Can we have
waiting happen client-side over in HBaseAdmin?
> Client calls HCM.isTableAvailable() which w
t; >
> > > Todd reported an issue which resulted in reverting my changes that
> caused
> > > HBA.createTable() to be synchronous.
> > >
> > > I want to see if we can reach consensus on how HBASE 3904 should be
> > handled.
> > >
> > > W
J-D
>> >>
>> >> On Thu, Jun 16, 2011 at 2:28 PM, Ted Yu wrote:
>> >> > Hi,
>> >> > J-D recently expressed the idea of changing behavior of
>> HBA.createTable()
>> >> to
>> >> > be consistent with its javado
d be synchronous.
> >> >
> >> > Todd reported an issue which resulted in reverting my changes that
> caused
> >> > HBA.createTable() to be synchronous.
> >> >
> >> > I want to see if we can reach consensus on how HBASE 3904 should be
> >> handled.
> >> >
> >> > We have 3 options:
> >> > 1. making HBA.createTable() to be synchronous in TRUNK
> >> > 2. pass initial region count in HTD
> >> > 3. add new method in HBA for synchronous table creation
> >> >
> >> > Thanks
> >> >
> >>
> >
>
;
>> > Todd reported an issue which resulted in reverting my changes that caused
>> > HBA.createTable() to be synchronous.
>> >
>> > I want to see if we can reach consensus on how HBASE 3904 should be
>> handled.
>> >
>> > We have 3 options:
>> > 1. making HBA.createTable() to be synchronous in TRUNK
>> > 2. pass initial region count in HTD
>> > 3. add new method in HBA for synchronous table creation
>> >
>> > Thanks
>> >
>>
>
teTable() to be synchronous.
> >
> > I want to see if we can reach consensus on how HBASE 3904 should be
> handled.
> >
> > We have 3 options:
> > 1. making HBA.createTable() to be synchronous in TRUNK
> > 2. pass initial region count in HTD
> > 3. add new method in HBA for synchronous table creation
> >
> > Thanks
> >
>
us.
>
> Todd reported an issue which resulted in reverting my changes that caused
> HBA.createTable() to be synchronous.
>
> I want to see if we can reach consensus on how HBASE 3904 should be handled.
>
> We have 3 options:
> 1. making HBA.createTable() to be synchronous in TRU
can reach consensus on how HBASE 3904 should be handled.
We have 3 options:
1. making HBA.createTable() to be synchronous in TRUNK
2. pass initial region count in HTD
3. add new method in HBA for synchronous table creation
Thanks
Hi,
I posted two solutions for HBASE-3904 and provided patch for one of them.
Please provide your comment.
Thanks
29 matches
Mail list logo
