Re: writing data to hbase from java code in eclipse

2011-09-21 Thread Ashish
Please check that you have created 'testtable' before running this program. thanks ashish On Thu, Sep 22, 2011 at 12:08 PM, vamshi krishna wrote: > Hi , > i am trying to cretae and write data to a table in hbase. For that i am > using hadoop-0.20.2 and hbase -0.90.4 single machine and pseudo dis

writing data to hbase from java code in eclipse

2011-09-21 Thread vamshi krishna
Hi , i am trying to cretae and write data to a table in hbase. For that i am using hadoop-0.20.2 and hbase -0.90.4 single machine and pseudo distributed mode. In eclipse helios, i write on e samll program import org.apache.hadoop.conf.Configuration; import org.apache.hadoop.hbase.HBaseConfiguration

RE: MR-Job custom property management?

2011-09-21 Thread Steinmaurer Thomas
Thanks for your suggestions! I think I go the route with a custom XML property file and using the -conf switch when starting the MR-job. Thanks again for your support. Much appreciated! Regards, Thomas -Original Message- From: Doug Meil [mailto:doug.m...@explorysmedical.com] Sent: Donne

Re: Relationship between family block size and cache blocks

2011-09-21 Thread Stack
On Wed, Sep 21, 2011 at 12:54 PM, Douglas Campbell wrote: > if a family's block size is smaller than some of the keyvalues in the family, > can those keyvalues be held in the cache? > A block will be size of the KV if the KV is > blocksize. IIRC, this KV that is > blocksize will go into the bloc

Re: Region Server does not shutdown gracefully

2011-09-21 Thread Stack
The bit about 1ms ping times sounds a bit good to me but Dan below says is better. St.Ack On Wed, Sep 21, 2011 at 6:21 PM, Daniel Washburn wrote: > > On Sep 21, 2011, at 1:51 PM, Bogdan Ghidireac wrote: > >>> HBase and HDFS are not made to span datacenters. >> >> Yes, I know. I think the correct

Re: Region Server does not shutdown gracefully

2011-09-21 Thread Daniel Washburn
On Sep 21, 2011, at 1:51 PM, Bogdan Ghidireac wrote: >> HBase and HDFS are not made to span datacenters. > > Yes, I know. I think the correct terminology is 3 availability zones > within the same region (ping between them is about 1ms) [snip] >> >> Is my assumption correct? Should I open a JI

Re: MR-Job custom property management?

2011-09-21 Thread Doug Meil
If you need to pass args to a mapper-task for example, using the Configuration object (as Stack suggested) is a common pattern. But just don't stick anything huge in there because that config is apparently copied repeatedly in the MR framework. On 9/21/11 12:09 PM, "Stack" wrote: >On Wed,

Re: Problem in implementing secondary indexes

2011-09-21 Thread Doug Meil
Additionally, see this... http://hbase.apache.org/book.html#secondary.indexes On 9/21/11 12:00 PM, "Stack" wrote: >Have you searched this lists' archives? A quick search on the net >seems to turn up relevant articles. Here is an older implementation >that you might study for ideas: >http

RE: Queries on Zookeeper failure and RegionServer restartup

2011-09-21 Thread Buttler, David
Hmm... Interesting perspective. From my point of view, if you add a node to an odd-numbered set, the reliability of the system will decrease. This is easiest to see between 1 and 2 nodes. For 2 nodes, if either node fails the whole system dies. P(A*B) < P(A) -- adding the second node can onl

Re: HBase HMaster/HServer 32bit/64bit issue

2011-09-21 Thread Matt Corgan
You'd probably have more of an issue with the amount of memory on 32 bit instance types. ~1.7gb. On Wed, Sep 21, 2011 at 12:45 PM, Ronen Itkin wrote: > It is a strange setup - I agree. > The thing is that is it running on AMAZON EC2 - and basically it all I have > right now. > > Does anyone el

Relationship between family block size and cache blocks

2011-09-21 Thread Douglas Campbell
Is there any relation between cache blocks and family blocks? Specifically, if a family's block size is smaller than some of the keyvalues in the family, can those keyvalues be held in the cache? Are the blocks in the cache designed to be the same size as the family's blocks? What is in a b

Re: HBase HMaster/HServer 32bit/64bit issue

2011-09-21 Thread Ronen Itkin
It is a strange setup - I agree. The thing is that is it running on AMAZON EC2 - and basically it all I have right now. Does anyone else has experienced different issues with that king of setup? On Wed, Sep 21, 2011 at 6:56 PM, Stack wrote: > On Wed, Sep 21, 2011 at 7:32 AM, Ronen Itkin wrote

major compaction time constantly growing

2011-09-21 Thread Oleg Ruchovets
Hi , Our environment hbase 90.2 (10 machine) We have 10 machine grid: master has 48G ram slaves machine has 16G ram. Region Server process has 4G ram Zookeeper process has 2G ram We have 4map/2reducer per machine We write from m/r job to hbase (2 jobs a day). 1) We run

Re: Region Server does not shutdown gracefully

2011-09-21 Thread Bogdan Ghidireac
> HBase and HDFS are not made to span datacenters. Yes, I know. I think the correct terminology is 3 availability zones within the same region (ping between them is about 1ms) > You could paste the thread dump This is the full stacktrace: Full thread dump Java HotSpot(TM) 64-Bit Server VM (20.1

Holes in our table.

2011-09-21 Thread Vidhyashankar Venkataraman
I pored over a few JIRAs and this looks like an issue many of you might have seen already. I am not sure. Do let me know if you guys have. We are currently having some problems with our cluster. I had pointed it out briefly in a mail titled "Unassigned holes in tables". We use a patched versio

Re: MR-Job custom property management?

2011-09-21 Thread Arun C Murthy
Also, look at the 'Tool' interface. On Sep 21, 2011, at 9:09 AM, Stack wrote: > On Wed, Sep 21, 2011 at 1:05 AM, Steinmaurer Thomas > wrote: >> when writing/executing a self-written MR-Job. How are you guys handle >> custom properties? As command-line arguments when starting the MR-job or >> via

Re: Queries on Zookeeper failure and RegionServer restartup

2011-09-21 Thread Joe Pallas
On Sep 20, 2011, at 8:37 AM, Buttler, David wrote: > Wait, you do realize that you have to have a majority of zookeeper nodes > alive for zookeeper to work, right? That means that you get lower > reliability with two nodes than one node: if either node goes down, zookeeper > will give up. Th

Re: MR-Job custom property management?

2011-09-21 Thread Stack
On Wed, Sep 21, 2011 at 1:05 AM, Steinmaurer Thomas wrote: > when writing/executing a self-written MR-Job. How are you guys handle > custom properties? As command-line arguments when starting the MR-job or > via a property file? > Command-line args works or passing w/ -D if your main is using Gen

Re: Region Server does not shutdown gracefully

2011-09-21 Thread Stack
On Wed, Sep 21, 2011 at 1:07 AM, Bogdan Ghidireac wrote: > I have an HBase 0.90.3 fleet with more than 100 hosts distributed > across 3 datacenters. > Yesterday, we experienced some network issues > for several minutes and the region servers from one data center lost > the connectivity with the na

Re: Problem in implementing secondary indexes

2011-09-21 Thread Stack
Have you searched this lists' archives? A quick search on the net seems to turn up relevant articles. Here is an older implementation that you might study for ideas: https://github.com/hbase-trx/hbase-transactional-tableindexed St.Ack On Wed, Sep 21, 2011 at 6:08 AM, Arsalan Bilal wrote: > I w

Re: HBase HMaster/HServer 32bit/64bit issue

2011-09-21 Thread Stack
On Wed, Sep 21, 2011 at 7:32 AM, Ronen Itkin wrote: > Is that a problem having HBase HMaster 32bit server working with 64bit > HRegion servers? > Can it cause problems? are there any known issues about this case? > That is a strange setup. If there are issues running this way, I cannot think wha

HBase HMaster/HServer 32bit/64bit issue

2011-09-21 Thread Ronen Itkin
Hi, Thanks for previous assistance! I have a question regarding HBase 32bit/64bit. Is that a problem having HBase HMaster 32bit server working with 64bit HRegion servers? Can it cause problems? are there any known issues about this case? Thank!! -- * Ronen Itkin* Taykey | www.taykey.com

Problem in implementing secondary indexes

2011-09-21 Thread Arsalan Bilal
I want to implement *secondary indexes *in* HBase-0.90.1-cdh3u0**. *I know that there is no native support available in Hbase. But can you tell me any other way to implement this? Can you refer any tutorial regarding this?* * -- Best Regards, Arsalan Bilal

RE: Hbase : Failed setting up proxy interface

2011-09-21 Thread Stuti Awasthi
Hi Guys, Issue is resolved, I exported poxy with username and password and it worked fine -Stuti From: Stuti Awasthi Sent: Wednesday, September 21, 2011 4:44 PM To: user@hbase.apache.org Subject: Hbase : Failed setting up proxy interface Hi, I have 3 nodes cluster. Each hostname resolve to IP co

Hbase : Failed setting up proxy interface

2011-09-21 Thread Stuti Awasthi
Hi, I have 3 nodes cluster. Each hostname resolve to IP correctly . Each machine is able to ping each other as well as passwordless SSH is also working fine. I wanted to start regionserver on all 3 machines so I updated conf/regionserver file with hostname of different machine. Firewall is off in

Re: Region server crash?

2011-09-21 Thread shanmuganathan.r
Hi Jack, I understood this concept. The DFS client is directly connected to the datanodes in hdfs and perform the read/write block data with communicate with the namenode by client protocol . Is there is no direct involvement of namenode in the HBase operation ? Thanks R.Shanm

Region Server does not shutdown gracefully

2011-09-21 Thread Bogdan Ghidireac
Hello, I have an HBase 0.90.3 fleet with more than 100 hosts distributed across 3 datacenters. Yesterday, we experienced some network issues for several minutes and the region servers from one data center lost the connectivity with the namenode. They started the shutdown sequence but about 20 host

Region Server does not shutdown gracefully

2011-09-21 Thread Bogdan Ghidireac
Hello, I have an HBase 0.90.3 fleet with more than 100 hosts distributed across 3 datacenters. Yesterday, we experienced some network issues for several minutes and the region servers from one data center lost the connectivity with the namenode. They started the shutdown sequence but about 20 host

MR-Job custom property management?

2011-09-21 Thread Steinmaurer Thomas
Hello, when writing/executing a self-written MR-Job. How are you guys handle custom properties? As command-line arguments when starting the MR-job or via a property file? Thanks! Regards, Thomas

Re: Region server crash?

2011-09-21 Thread Jack Levin
The master will detect that RS is down by periodically checking a zookeeper ( it will say in the master log, znode expired ). After, it will check to see if there is anything in /hbase/.logs directory for that region server, if something is found, master will replay the log records and 'push' them

Region server crash?

2011-09-21 Thread shanmuganathan.r
Hi All, I am running the HBase in fully distributed mode. I used the HBase 0.90.2 version . I have one doubt , that one is what will happen when the Region server is crashed ? If the one region is managed by one region server then what will happen after the crash of the region se