Re: hbase replication : ERROR: Connection refused

[Ted]

thanks, that helped.

It showed it was doing a subsequent call via 127.0.0.1 so I double checked
all my configs to make sure I used the hostname/dns entries then it all
worked.

thanks.



On Tue, Aug 23, 2016 at 2:11 PM, Esteban Gutierrez 
wrote:

> Hello Ted,
>
> the first 2 commands interact with the HBase Master and the third command
> needs to get the splits from Meta, probably the exception is coming from
> there but its can you retry the commands but this time launching the hbase
> shell with the -d flag? e.g. "hbase shell -d" that should give us a better
> idea where that connection refused is coming from.
>
> cheers,
> esteban.
>
> --
> Cloudera, Inc.
>
>
> On Tue, Aug 23, 2016 at 1:46 PM, Ted  wrote:
>
> > Hi I'm running hbase 1.2.1 and I'm getting an error trying to setup
> > replication.
> >
> > I have 2 hbase systems running fine,
> > I can connect from both to each other via ssh and on all ports as there's
> > no firewall (this is just a test system).
> > I can connect from the source to the destination ZK on port 2181
> >
> > When I try to enable replication though I get "ERROR: Connection refused"
> > with no other helpful information anywhere.
> >
> > ---
> > hbase(main):002:0> create 'test', 'cf'
> > 0 row(s) in 2.5390 seconds
> >
> > => Hbase::Table - test
> > hbase(main):003:0> add_peer 'vm2', "vm2:2181:/hbase"
> > 0 row(s) in 0.1830 seconds
> >
> > hbase(main):004:0> enable_table_replication 'test'
> >
> > ERROR: Connection refused
> >
> > Here is some help for this command:
> > Enable a table's replication switch.
> >
> > Examples:
> >
> >   hbase> enable_table_replication 'table_name'
> >
> >
> > hbase(main):005:0>
> > ---
> >
> > There's no errors on the source logs, no log entries written at all
> during
> > the enable call.
> > On the destination logs there are also no errors.
> > If I do a "zk_dump" on the destination, I can see the source is actually
> > connected.
> >
> > No replication is taking place, and other than that "error: connection
> > refused", I see no error messages.
> >
> > Anyone with any ideas on what and where it's trying to connect to? and
> why
> > it's being refused?
> >
> > --
> > Ted.
> >
>



-- 
Ted.

Re: hbase replication : ERROR: Connection refused

[Esteban Gutierrez]

Hello Ted,

the first 2 commands interact with the HBase Master and the third command
needs to get the splits from Meta, probably the exception is coming from
there but its can you retry the commands but this time launching the hbase
shell with the -d flag? e.g. "hbase shell -d" that should give us a better
idea where that connection refused is coming from.

cheers,
esteban.

--
Cloudera, Inc.


On Tue, Aug 23, 2016 at 1:46 PM, Ted  wrote:

> Hi I'm running hbase 1.2.1 and I'm getting an error trying to setup
> replication.
>
> I have 2 hbase systems running fine,
> I can connect from both to each other via ssh and on all ports as there's
> no firewall (this is just a test system).
> I can connect from the source to the destination ZK on port 2181
>
> When I try to enable replication though I get "ERROR: Connection refused"
> with no other helpful information anywhere.
>
> ---
> hbase(main):002:0> create 'test', 'cf'
> 0 row(s) in 2.5390 seconds
>
> => Hbase::Table - test
> hbase(main):003:0> add_peer 'vm2', "vm2:2181:/hbase"
> 0 row(s) in 0.1830 seconds
>
> hbase(main):004:0> enable_table_replication 'test'
>
> ERROR: Connection refused
>
> Here is some help for this command:
> Enable a table's replication switch.
>
> Examples:
>
>   hbase> enable_table_replication 'table_name'
>
>
> hbase(main):005:0>
> ---
>
> There's no errors on the source logs, no log entries written at all during
> the enable call.
> On the destination logs there are also no errors.
> If I do a "zk_dump" on the destination, I can see the source is actually
> connected.
>
> No replication is taking place, and other than that "error: connection
> refused", I see no error messages.
>
> Anyone with any ideas on what and where it's trying to connect to? and why
> it's being refused?
>
> --
> Ted.
>

hbase replication : ERROR: Connection refused

[Ted]

Hi I'm running hbase 1.2.1 and I'm getting an error trying to setup
replication.

I have 2 hbase systems running fine,
I can connect from both to each other via ssh and on all ports as there's
no firewall (this is just a test system).
I can connect from the source to the destination ZK on port 2181

When I try to enable replication though I get "ERROR: Connection refused"
with no other helpful information anywhere.

---
hbase(main):002:0> create 'test', 'cf'
0 row(s) in 2.5390 seconds

=> Hbase::Table - test
hbase(main):003:0> add_peer 'vm2', "vm2:2181:/hbase"
0 row(s) in 0.1830 seconds

hbase(main):004:0> enable_table_replication 'test'

ERROR: Connection refused

Here is some help for this command:
Enable a table's replication switch.

Examples:

  hbase> enable_table_replication 'table_name'


hbase(main):005:0>
---

There's no errors on the source logs, no log entries written at all during
the enable call.
On the destination logs there are also no errors.
If I do a "zk_dump" on the destination, I can see the source is actually
connected.

No replication is taking place, and other than that "error: connection
refused", I see no error messages.

Anyone with any ideas on what and where it's trying to connect to? and why
it's being refused?

-- 
Ted.

Re: Hbase Row key lock

[iain wright]

HBase uses MVCC to deal with concurrent writes, this is a good explanation:

https://blogs.apache.org/hbase/entry/apache_hbase_internals_locking_and


-- 
Iain Wright

This email message is confidential, intended only for the recipient(s)
named above and may contain information that is privileged, exempt from
disclosure under applicable law. If you are not the intended recipient, do
not disclose or disseminate the message to anyone except the intended
recipient. If you have received this message in error, or are not the named
recipient(s), please immediately notify the sender by return email, and
delete all copies of this message.

On Tue, Aug 23, 2016 at 9:59 AM, Manjeet Singh 
wrote:

> Hi All,
> In Hbase current version what happen if two different client write the same
> rowkey at same time in hbase
> Assume if its kind of update.
> Does Rowkey lock Hbase server automatically deal with this?
>
>
> Thanks
> Manjeet
>
> On Wed, Aug 17, 2016 at 1:04 PM, Dima Spivak  wrote:
>
> > Row locks on the client side were deprecated in 0.94 (see HBASE-7341) and
> > removed in 0.96 (see HBASE-7315). As you note, they could lead to
> deadlocks
> > and also had problems when region moves or splits occurred.
> >
> > Is there a specific reason you're looking for this functionality,
> Manjeet?
> >
> > -Dima
> >
> > On Tuesday, August 16, 2016, Manjeet Singh 
> > wrote:
> >
> > > Hi All
> > >
> > > Can anyone help me about how and in which version of Hbase support
> Rowkey
> > > lock ?
> > > I have seen article about rowkey lock but it was about .94 version it
> > said
> > > that if row key not exist and any update request come and that rowkey
> not
> > > exist then in this case Hbase hold the lock for 60 sec.
> > >
> > > currently I am using Hbase 1.2.2 version
> > >
> > > Thanks
> > > Manjeet
> > >
> > >
> > >
> > > --
> > > luv all
> > >
> >
> >
> > --
> > -Dima
> >
>
>
>
> --
> luv all
>

Re: how to abort a table replication?

[Ted]

oh if you mean : ls /hbase/replication/peers
[]

is all it shows

It might be an error in the code though, the stacktrace says it's an
unexpected error although it's only a warning (it's a very short stack
trace) :

2016-08-23 10:30:57,444 INFO  [main.replicationSource,vm2-SendThread(
192.168.122.106:2182)] zookeeper.ClientCnxn: Opening socket connection to
server 192.168.122.106/192.168.122.106:2182. Will not attempt to
authenticate using SASL (unknown error)
2016-08-23 10:30:57,446 WARN  [main.replicationSource,vm2-SendThread(
192.168.122.106:2182)] zookeeper.ClientCnxn: Session 0x0 for server null,
unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
at
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
at
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)



On Tue, Aug 23, 2016 at 10:21 AM, Ted Yu  wrote:

> Please take a look at
> http://hbase.apache.org/book.html#trouble.tools.builtin.zkcli
>
>
>
> On Tue, Aug 23, 2016 at 10:07 AM, Ted  wrote:
>
> > the command out puts were as follows (note I originally ran them in
> > possibly the wrong order, I also tried re-adding the same id and removing
> > it again with no difference)
> >
> > hbase(main):003:0> list_peers
> >  PEER_ID CLUSTER_KEY STATE TABLE_CFS
> >  vm2 192.168.122.106:2182:/hbase ENABLED
> > 1 row(s) in 0.1250 seconds
> >
> > hbase(main):004:0> remove_peer 'vm2'
> > 0 row(s) in 0.0710 seconds
> >
> > hbase(main):005:0> list_peers
> >  PEER_ID CLUSTER_KEY STATE TABLE_CFS
> > 0 row(s) in 0.0080 seconds
> >
> > hbase(main):006:0> disable_table_replication 'test'
> > 0 row(s) in 0.0500 seconds
> > The replication swith of table 'test' successfully disabled
> >
> > hbase(main):007:0> list_peers
> >  PEER_ID CLUSTER_KEY STATE TABLE_CFS
> > 0 row(s) in 0.0230 seconds
> >
> > I'm not quite sure how to use "hbase zkcli", it's dropped me at the
> prompt,
> > but I don't see anything in the help which prints values or
> configuration.
> > Do you happen to know what I should run to see those values?
> >
> > thanks.
> >
> >
> >
> > On Tue, Aug 23, 2016 at 9:57 AM, Ted Yu  wrote:
> >
> > > Can you pastebin the output from the 3 commands, especially
> remove_peer ?
> > >
> > > Can you use 'hbase zkcli' to inspect / > > of zookeeper.znode.parent>/replication/peers ?
> > >
> > > Thanks
> > >
> > > On Tue, Aug 23, 2016 at 9:51 AM, Ted  wrote:
> > >
> > > > I'm using hbase 1.2.1 and I'm having problems aborting a table
> > > replication.
> > > >
> > > > I was testing replication and did an "add_peer" but I accidentally
> > > entered
> > > > the wrong port number.
> > > > The source server is now logging a bunch of
> "java.net.ConnectException:
> > > > Connection refused" errors which is understandable.
> > > >
> > > > I tried disable_peer, remove_peer, and disable_table_replication.
> > > >
> > > > it's removed the peer from list_peers, but I continue to get the
> errors
> > > in
> > > > the logs showing it's attempting to connect to the destination server
> > on
> > > > the wrong port number.
> > > >
> > > > Does anyone know how to actually stop that thread?
> > > > --
> > > > Ted.
> > > >
> > >
> >
> >
> >
> > --
> > Ted.
> >
>



-- 
Ted.

Re: how to abort a table replication?

[Ted Yu]

Please take a look at
http://hbase.apache.org/book.html#trouble.tools.builtin.zkcli



On Tue, Aug 23, 2016 at 10:07 AM, Ted  wrote:

> the command out puts were as follows (note I originally ran them in
> possibly the wrong order, I also tried re-adding the same id and removing
> it again with no difference)
>
> hbase(main):003:0> list_peers
>  PEER_ID CLUSTER_KEY STATE TABLE_CFS
>  vm2 192.168.122.106:2182:/hbase ENABLED
> 1 row(s) in 0.1250 seconds
>
> hbase(main):004:0> remove_peer 'vm2'
> 0 row(s) in 0.0710 seconds
>
> hbase(main):005:0> list_peers
>  PEER_ID CLUSTER_KEY STATE TABLE_CFS
> 0 row(s) in 0.0080 seconds
>
> hbase(main):006:0> disable_table_replication 'test'
> 0 row(s) in 0.0500 seconds
> The replication swith of table 'test' successfully disabled
>
> hbase(main):007:0> list_peers
>  PEER_ID CLUSTER_KEY STATE TABLE_CFS
> 0 row(s) in 0.0230 seconds
>
> I'm not quite sure how to use "hbase zkcli", it's dropped me at the prompt,
> but I don't see anything in the help which prints values or configuration.
> Do you happen to know what I should run to see those values?
>
> thanks.
>
>
>
> On Tue, Aug 23, 2016 at 9:57 AM, Ted Yu  wrote:
>
> > Can you pastebin the output from the 3 commands, especially remove_peer ?
> >
> > Can you use 'hbase zkcli' to inspect / > of zookeeper.znode.parent>/replication/peers ?
> >
> > Thanks
> >
> > On Tue, Aug 23, 2016 at 9:51 AM, Ted  wrote:
> >
> > > I'm using hbase 1.2.1 and I'm having problems aborting a table
> > replication.
> > >
> > > I was testing replication and did an "add_peer" but I accidentally
> > entered
> > > the wrong port number.
> > > The source server is now logging a bunch of "java.net.ConnectException:
> > > Connection refused" errors which is understandable.
> > >
> > > I tried disable_peer, remove_peer, and disable_table_replication.
> > >
> > > it's removed the peer from list_peers, but I continue to get the errors
> > in
> > > the logs showing it's attempting to connect to the destination server
> on
> > > the wrong port number.
> > >
> > > Does anyone know how to actually stop that thread?
> > > --
> > > Ted.
> > >
> >
>
>
>
> --
> Ted.
>

Re: how to abort a table replication?

[Ted]

oh whoops that only showed 2 of the commands, here's one showing all 3

hbase(main):011:0> disable_peer 'vm2'
0 row(s) in 0.1600 seconds

hbase(main):012:0> remove_peer 'vm2'
0 row(s) in 0.0560 seconds

hbase(main):013:0> disable_table_replication 'test'
0 row(s) in 0.0640 seconds
The replication swith of table 'test' successfully disabled


On Tue, Aug 23, 2016 at 10:07 AM, Ted  wrote:

> the command out puts were as follows (note I originally ran them in
> possibly the wrong order, I also tried re-adding the same id and removing
> it again with no difference)
>
> hbase(main):003:0> list_peers
>  PEER_ID CLUSTER_KEY STATE TABLE_CFS
>  vm2 192.168.122.106:2182:/hbase ENABLED
> 1 row(s) in 0.1250 seconds
>
> hbase(main):004:0> remove_peer 'vm2'
> 0 row(s) in 0.0710 seconds
>
> hbase(main):005:0> list_peers
>  PEER_ID CLUSTER_KEY STATE TABLE_CFS
> 0 row(s) in 0.0080 seconds
>
> hbase(main):006:0> disable_table_replication 'test'
> 0 row(s) in 0.0500 seconds
> The replication swith of table 'test' successfully disabled
>
> hbase(main):007:0> list_peers
>  PEER_ID CLUSTER_KEY STATE TABLE_CFS
> 0 row(s) in 0.0230 seconds
>
> I'm not quite sure how to use "hbase zkcli", it's dropped me at the
> prompt, but I don't see anything in the help which prints values or
> configuration. Do you happen to know what I should run to see those values?
>
> thanks.
>
>
>
> On Tue, Aug 23, 2016 at 9:57 AM, Ted Yu  wrote:
>
>> Can you pastebin the output from the 3 commands, especially remove_peer ?
>>
>> Can you use 'hbase zkcli' to inspect /> of zookeeper.znode.parent>/replication/peers ?
>>
>> Thanks
>>
>> On Tue, Aug 23, 2016 at 9:51 AM, Ted  wrote:
>>
>> > I'm using hbase 1.2.1 and I'm having problems aborting a table
>> replication.
>> >
>> > I was testing replication and did an "add_peer" but I accidentally
>> entered
>> > the wrong port number.
>> > The source server is now logging a bunch of "java.net.ConnectException:
>> > Connection refused" errors which is understandable.
>> >
>> > I tried disable_peer, remove_peer, and disable_table_replication.
>> >
>> > it's removed the peer from list_peers, but I continue to get the errors
>> in
>> > the logs showing it's attempting to connect to the destination server on
>> > the wrong port number.
>> >
>> > Does anyone know how to actually stop that thread?
>> > --
>> > Ted.
>> >
>>
>
>
>
> --
> Ted.
>



-- 
Ted.

Re: how to abort a table replication?

[Ted]

the command out puts were as follows (note I originally ran them in
possibly the wrong order, I also tried re-adding the same id and removing
it again with no difference)

hbase(main):003:0> list_peers
 PEER_ID CLUSTER_KEY STATE TABLE_CFS
 vm2 192.168.122.106:2182:/hbase ENABLED
1 row(s) in 0.1250 seconds

hbase(main):004:0> remove_peer 'vm2'
0 row(s) in 0.0710 seconds

hbase(main):005:0> list_peers
 PEER_ID CLUSTER_KEY STATE TABLE_CFS
0 row(s) in 0.0080 seconds

hbase(main):006:0> disable_table_replication 'test'
0 row(s) in 0.0500 seconds
The replication swith of table 'test' successfully disabled

hbase(main):007:0> list_peers
 PEER_ID CLUSTER_KEY STATE TABLE_CFS
0 row(s) in 0.0230 seconds

I'm not quite sure how to use "hbase zkcli", it's dropped me at the prompt,
but I don't see anything in the help which prints values or configuration.
Do you happen to know what I should run to see those values?

thanks.



On Tue, Aug 23, 2016 at 9:57 AM, Ted Yu  wrote:

> Can you pastebin the output from the 3 commands, especially remove_peer ?
>
> Can you use 'hbase zkcli' to inspect / of zookeeper.znode.parent>/replication/peers ?
>
> Thanks
>
> On Tue, Aug 23, 2016 at 9:51 AM, Ted  wrote:
>
> > I'm using hbase 1.2.1 and I'm having problems aborting a table
> replication.
> >
> > I was testing replication and did an "add_peer" but I accidentally
> entered
> > the wrong port number.
> > The source server is now logging a bunch of "java.net.ConnectException:
> > Connection refused" errors which is understandable.
> >
> > I tried disable_peer, remove_peer, and disable_table_replication.
> >
> > it's removed the peer from list_peers, but I continue to get the errors
> in
> > the logs showing it's attempting to connect to the destination server on
> > the wrong port number.
> >
> > Does anyone know how to actually stop that thread?
> > --
> > Ted.
> >
>



-- 
Ted.

Re: Hbase Row key lock

[Manjeet Singh]

Hi All,
In Hbase current version what happen if two different client write the same
rowkey at same time in hbase
Assume if its kind of update.
Does Rowkey lock Hbase server automatically deal with this?


Thanks
Manjeet

On Wed, Aug 17, 2016 at 1:04 PM, Dima Spivak  wrote:

> Row locks on the client side were deprecated in 0.94 (see HBASE-7341) and
> removed in 0.96 (see HBASE-7315). As you note, they could lead to deadlocks
> and also had problems when region moves or splits occurred.
>
> Is there a specific reason you're looking for this functionality, Manjeet?
>
> -Dima
>
> On Tuesday, August 16, 2016, Manjeet Singh 
> wrote:
>
> > Hi All
> >
> > Can anyone help me about how and in which version of Hbase support Rowkey
> > lock ?
> > I have seen article about rowkey lock but it was about .94 version it
> said
> > that if row key not exist and any update request come and that rowkey not
> > exist then in this case Hbase hold the lock for 60 sec.
> >
> > currently I am using Hbase 1.2.2 version
> >
> > Thanks
> > Manjeet
> >
> >
> >
> > --
> > luv all
> >
>
>
> --
> -Dima
>



-- 
luv all

how to abort a table replication?

[Ted]

I'm using hbase 1.2.1 and I'm having problems aborting a table replication.

I was testing replication and did an "add_peer" but I accidentally entered
the wrong port number.
The source server is now logging a bunch of "java.net.ConnectException:
Connection refused" errors which is understandable.

I tried disable_peer, remove_peer, and disable_table_replication.

it's removed the peer from list_peers, but I continue to get the errors in
the logs showing it's attempting to connect to the destination server on
the wrong port number.

Does anyone know how to actually stop that thread?
-- 
Ted.

Re: how to abort a table replication?

[Ted Yu]

Can you pastebin the output from the 3 commands, especially remove_peer ?

Can you use 'hbase zkcli' to inspect //replication/peers ?

Thanks

On Tue, Aug 23, 2016 at 9:51 AM, Ted  wrote:

> I'm using hbase 1.2.1 and I'm having problems aborting a table replication.
>
> I was testing replication and did an "add_peer" but I accidentally entered
> the wrong port number.
> The source server is now logging a bunch of "java.net.ConnectException:
> Connection refused" errors which is understandable.
>
> I tried disable_peer, remove_peer, and disable_table_replication.
>
> it's removed the peer from list_peers, but I continue to get the errors in
> the logs showing it's attempting to connect to the destination server on
> the wrong port number.
>
> Does anyone know how to actually stop that thread?
> --
> Ted.
>

Re: setMaxResultSize on Gets

[Bryan Beaudreault]

Great, thanks stack!

On Tue, Aug 23, 2016 at 12:54 AM Stack  wrote:

> On Mon, Aug 22, 2016 at 3:02 PM, Bryan Beaudreault <
> bbeaudrea...@hubspot.com
> > wrote:
>
> > In HBase 1.2.x and higher you can call setMaxResultSize on a Scan to
> limit
> > the impact of scans that are too aggressive, by bailing out at a certain
> > size response. The client side will nicely splice together all of the
> > isPartial responses to create a full one as well, pushing the danger to
> the
> > client side where it belongs.
> >
> > However, there doesn't appear to be any way to do similar on Gets,
> despite
> > Gets being backed by Scans under the covers.  I do notice
> > a setMaxResultsPerColumnFamily, but this seems much less straightforward
> > than setMaxResultSize.
> >
> > Was there a reason that Gets did not get the same treatment that Scans
> got?
> > We regularly see issues where someone has a large row and does a
> > too-aggressive Get against it. We're currently looking to build a wrapper
> > using setMaxResultsPerColumnFamily or adding support for setMaxResultSize
> > to Gets, but I want to make sure there wasn't a reason it was avoided in
> > the first place.
> >
>
> A simple case of Scan myopia/focus I'd say Bryan. Oversight. I don't see
> any reason why we shouldn't do Get in same way. In fact it came up recently
> in an internal conversation. The difficult part might the protocol going
> back and forth adding in an incremental. A Get is a Scan but only after it
> arrives at the server. I've not looked but hopefully it could be the same
> as Scan.
>
> St.Ack
>