Re: Errors after major compaction

2011-07-05 Thread Eran Kutner
no. but I did run major compaction. As I explained initially, I disabled the table so I could change its TTL, then re-enabled it then ran major compaction so it would clean up the expired data due to the TTL change. -eran On Wed, Jul 6, 2011 at 02:43, Ted Yu wrote: > Eran: > You didn't run hb

Re: Seperating Application and Database Servers

2011-07-05 Thread James Ram
Hi, Thanks for your reply. We solved it. We had not started zookeeper earlier in the application server machine. When we started it, it is working fine. Thanks a lot again. With Regards, Jr. On Tue, Jul 5, 2011 at 6:53 PM, Ted Yu wrote: > Check your DNS. > localhost in the log below can also

How to apply multiple row filters in an efficient way?

2011-07-05 Thread Alt Control
Hello, My table holds stock information where the keys are in the format: -. In my mapreduce job I need to operate on a subset of that list, say 500-2000 stocks, out of a total of 7000~. Sometimes I also need to consider only rows after a certain date. Question is - how can I do that efficiently?

Re: hbase host dns ip and route for multi network interface card

2011-07-05 Thread Jameson Li
Hi, I think The problem is that: the /etc/hosts file is resolved the dns node3 to 192.168.1.15, but the hbase inner sometime uses the 192.168.1.13. When I use the command "ifdown eth0" on node3 and use stop-hbase.sh, there shows the message: 2011-07-06 10:25:50,683 DEBUG org.apache.hadoop.hbase.m

Re: Errors after major compaction

2011-07-05 Thread Ted Yu
Eran: You didn't run hbck during the enabling of gs_raw_events table, right ? I saw: 2011-06-29 16:43:50,395 DEBUG org.apache.hadoop.hbase.regionserver.CompactSplitThread: Compaction (major) requested for gs_raw_events,GSLoad_1308518553_168_WEB204,1308533970928.584dac5cc70d8682f71c4675a843c309. be

Re: WrongRegionException and inconsistent table found

2011-07-05 Thread Xu-Feng Mao
I forgot the version, we are using cdh3u0. Mao Xu-Feng 在 2011-7-6,0:59,Xu-Feng Mao 写道: We also check the master log, nothing interesting found. On Wed, Jul 6, 2011 at 12:58 AM, Xu-Feng Mao wrote: > Hi, > > We're running a hbase cluster including 37 regionservers. Today, we found > losts of W

Re: zookeeper connection issue - distributed mode

2011-07-05 Thread devushan...@gmail.com
Entry in the hbase-site.xml worked for me. thanks, devush On 05/07/2011 15:00, Florin P wrote: Hello! The property hbase.zookeeper.quorum is taken from hbase-site.xml on the HBase master machine. taken from Hbase master hbase-site.xml hbase.zookeeper.quorum

Re: WrongRegionException and inconsistent table found

2011-07-05 Thread Xu-Feng Mao
We also check the master log, nothing interesting found. On Wed, Jul 6, 2011 at 12:58 AM, Xu-Feng Mao wrote: > Hi, > > We're running a hbase cluster including 37 regionservers. Today, we found > losts of WrongRegionException when putting object into it. > > hbase hbck -details > reports that

WrongRegionException and inconsistent table found

2011-07-05 Thread Xu-Feng Mao
Hi, We're running a hbase cluster including 37 regionservers. Today, we found losts of WrongRegionException when putting object into it. hbase hbck -details reports that Chain of regions in table STable is broken; edges does not contain ztxrGmCwn-6BE32s3cX1TNeHU_I= ERROR: Found inconsist

Re: Errors after major compaction

2011-07-05 Thread Eran Kutner
Appreciate it, sorry I didn't get to it sooner. Had some crazy days :) -eran On Tue, Jul 5, 2011 at 17:19, Ted Yu wrote: > Eran: > I logged https://issues.apache.org/jira/browse/HBASE-4060 for you. > > On Mon, Jul 4, 2011 at 2:30 AM, Ted Yu wrote: > > > Thanks for the understanding. > > > >

Re: Errors after major compaction

2011-07-05 Thread Ted Yu
Eran: I logged https://issues.apache.org/jira/browse/HBASE-4060 for you. On Mon, Jul 4, 2011 at 2:30 AM, Ted Yu wrote: > Thanks for the understanding. > > Can you log a JIRA and put your ideas below in it ? > > > > On Jul 4, 2011, at 12:42 AM, Eran Kutner wrote: > > > Thanks for the explanation

Re: zookeeper connection issue - distributed mode

2011-07-05 Thread Sanel Zukan
Hi, On 60020 is usually region server; check hbase-default.xml or try to set 'hbase.regionserver' inside HBaseConfiguration object, if you are going to use it directly from application code. Regards, Sanel On Tue, Jul 5, 2011 at 4:00 PM, Florin P wrote: > Hello! > > The property hbase.zookeeper

Re: zookeeper connection issue - distributed mode

2011-07-05 Thread Florin P
Hello! The property hbase.zookeeper.quorum is taken from hbase-site.xml on the HBase master machine. taken from Hbase master hbase-site.xml hbase.zookeeper.quorum For me, it worked. Success, Florin --- On Tue, 7/5/11, d

Re: FW: Reg: The HLog that is created in region creation

2011-07-05 Thread Ted Yu
When hbase.hstore.blockingStoreFiles is reached in one Store, updates are blocked for this HRegion until a compaction is completed. Regards On Tue, Jul 5, 2011 at 12:09 AM, Ramkrishna S Vasudevan < ramakrish...@huawei.com> wrote: > > Hi all, > > Sorry Ted for not sending to the dev@ list. > > Fe

Re: hbck -fix

2011-07-05 Thread Edward Capriolo
On Mon, Jul 4, 2011 at 1:28 PM, Stack wrote: > On Sun, Jul 3, 2011 at 12:39 AM, Andrew Purtell > wrote: > > I've done exercises in the past like delete META on disk and recreate it > with the earlier set of utilities (add_table.rb). This always "worked for > me" when I've tried it. > > > > We ne

Re: Seperating Application and Database Servers

2011-07-05 Thread Ted Yu
Check your DNS. localhost in the log below can also mean that your hbase.zookeeper.quorum is carrying default value. Modify it to point to the real quorum. On Tue, Jul 5, 2011 at 1:27 AM, James Ram wrote: > Hi, > > We are running hadoop and hbase in a 9 machine cluster. We tried to put our > app

Re: client-side caching

2011-07-05 Thread Drew Dahlke
I've been down that road and did a memcache lookup within a mapper. My 9 node cluster managed to CPU and network spike the memcache server hard enough I practically DOS attacked it. I'd recommend taking your data (if it's not massive) and dumping it to a hadoop map file. Think of it as a file base

hbase host dns ip and route for multi network interface card

2011-07-05 Thread Jameson Li
Hi, when I start my hbase cluster, there are some error logs in the master-log: 2011-07-05 17:13:13,820 INFO org.apache.zookeeper.ClientCnxn: zookeeper.disableAutoWatchReset is false 2011-07-05 17:13:13,840 INFO org.apache.zookeeper.ClientCnxn: Attempting connection to server node3/192.168.1.15:2

Re: client-side caching

2011-07-05 Thread Claudio Martella
Totally understand. As a matter of fact I didn't mention my table is read-only or insert-only (no data is modified), so no real invalidation necessary here. I guess this means i should go for my own memcache on the client side. Thanks! On 7/5/11 11:39 AM, Ryan Rawson wrote: > Caching sounds easy

zookeeper connection issue - distributed mode

2011-07-05 Thread devush
Hi, i have following environment - hbase-0.90.1-cdh3u0 on ubuntu. I have following code for the distributed mode and i am calling this java code from remote client: HBaseConfiguration config = new HBaseConfiguration(); config.clear();

Re: client-side caching

2011-07-05 Thread Ryan Rawson
Caching sounds easy until you need to worry about invalidation. It's hard to build efficient and correct invalidation. On Jul 5, 2011 2:13 AM, "Claudio Martella" wrote: > I've seen that. But that's about caching on regionserver-side through > memcache. > > You still have the network roundtrip. I'd

how to check whether zookeeper quorum is working fine or not ?

2011-07-05 Thread praveenesh kumar
Hi, I have a 12-node hbase cluster setup with 3 nodes as a part of zookeeper quorum. I am able to run hbase shell, create tables.. and able to access tables in the shell. Now I am configuring pig to use hbase. While accessing records using pig, it is giving some Zookeeper exception and saying to c

Re: client-side caching

2011-07-05 Thread Claudio Martella
I've seen that. But that's about caching on regionserver-side through memcache. You still have the network roundtrip. I'd like to avoid at all contacting the regionserver, when possible. So I was guessing whether the hbase-client would have some caching embedded, otherwise I'll implement it throug

Partial replication

2011-07-05 Thread James Ram
Hi, Is it possible to replicate a particular dataset to another cluster instead of replicating the whole data? -- With Regards, Jr.

Seperating Application and Database Servers

2011-07-05 Thread James Ram
Hi, We are running hadoop and hbase in a 9 machine cluster. We tried to put our application server on a machine outside the HBase cluster, but we are getting the following error. Is there any way that we can do this? 11/07/05 11:26:41 WARN zookeeper.ClientCnxn: Session 0x0 for server null, unexpe

RE: FW: Reg: The HLog that is created in region creation

2011-07-05 Thread Ramkrishna S Vasudevan
Hi all, Sorry Ted for not sending to the dev@ list. Few more queries related to splitting 1. As per the flow when compaction is happening there may be few more store files created due to flushing. 2. Suppose initially 3 Store files have been selected for compaction at the end of compaction proc