Re: Slow scanning for PrefixFilter on EncodedBlocks

+1 for making PrefixFIlter seek instead of using a startRow explicitly. ./zahoor On Thu, Oct 18, 2012 at 4:05 AM, lars hofhansl wrote: > Oh yeah, I meant that one should always set the startrow as a matter of > practice - if possible - and never rely on the filter alone. > > > > ___

Re: crafting your key - scan vs. get

Neil, I've pointed you in the right direction. The rest of the exercise is left to the student. :-) While you used the comment about having fun, your question is boring. *^1 The fun part is for you now to play and see why I may have suggested the importance of column order. Sorry, but that r

RE: Checking major compaction

Hi Yes Kiran you can go thro the logs also. You will see some logs like 'Start major compaction for .. 'Compacting file 'Compacting file And finally 'Completed major/minor compaction.' I just don have some exact logs with me right now. But

Re: hbase deployment using VMs for data nodes and SAN for data storage

Lars, I think we need to clarify what we think of as a SAN. Its possible to have a SAN where the disks appear as attached storage, while the traditional view is that the disks are detached. There are some design considerations like cluster density where one would want to use a SAN like NetA

RE: Checking major compaction

>A simple test would be to just right some 10 rows I meant to say write some 10 rows.(not right) Regards Ram > -Original Message- > From: Ramkrishna.S.Vasudevan [mailto:ramkrishna.vasude...@huawei.com] > Sent: Thursday, October 18, 2012 2:05 PM > To: user@hbase.apache.org > Subject: RE: C

one RegionServer crashed and the whole cluster was blocked

Hi, All One of the RegionServer of our company’s cluster was crashed. At this time, I found: 1. All the RegionServer stopped handling the requests from the client side( requestsPerSecond=0 at the master-status UI page). 2. It takes about 12-15 minutes to recovery. 3. I have

Re: Comparison of hbase/hadoop with sql server

What is the difference between HBase and Hadoop+HBase? HBase runs on top of Hadoop components. Also, first answer us this question, before we answer yours: Will your "SQL Server" scale linearly as you add more machines? Can it easily scale horizontally and vertically? Seems to me like you're comp

RE: one RegionServer crashed and the whole cluster was blocked

> For 1, I knew the cluster began to split log and recover the data on > the > crashed RegionServer, will the recovery operation block all the > requests > from the client side? Ideally should not. But if your client was generating data for the regions that were dead at that time then client r

Re: Coprocessor end point vs MapReduce?

To echo what Mike said about KISS, would you use triggers for a large time-sensitive batch job in an RDBMS? It's possible, but probably not. Then you might want to think twice about using co-processors for such a purpose with HBase. On 10/17/12 9:50 PM, "Michael Segel" wrote: >Run your wee

Re: one RegionServer crashed and the whole cluster was blocked

Hi, Some stuff below: On Thu, Oct 18, 2012 at 1:30 PM, 张磊 wrote: > Hi, All > > One of the RegionServer of our company’s cluster was crashed. At this > time, I found: > > 1. All the RegionServer stopped handling the requests from the client > side( requestsPerSecond=0 at the master-statu

Re: ANN: HBase 0.94.2 is available for download

+1 on pushing to maven repo. Thanks. On Wed, Oct 17, 2012 at 1:49 PM, Ramkrishna.S.Vasudevan < ramkrishna.vasude...@huawei.com> wrote: > Thanks Jean for your update. > > Regards > Ram > > > -Original Message- > > From: Jean-Marc Spaggiari [mailto:jean-m...@spaggiari.org] > > Sent: Wednes

remote connection using HBase Java client

Hi, I have a standalone HBase 0.94.1 server running on my desktop. In the hbase-site.xml file, I just set hbase.rootdir. From my laptop, i want to connect to HBase server in my desktop. What should I change on my client and server HBase configuration files? Also, what should I change on /etc/

Re: ANN: HBase 0.94.2 is available for download

I'm on it. :) - Original Message - From: Amit Sela To: user@hbase.apache.org Cc: Sent: Thursday, October 18, 2012 8:25 AM Subject: Re: ANN: HBase 0.94.2 is available for download +1 on pushing to maven repo. Thanks. On Wed, Oct 17, 2012 at 1:49 PM, Ramkrishna.S.Vasudevan < ramkrishn

Using filters in REST/stargate returns 204 (No content)

I have a HBase Java client which has a couple of filters and just work fine, I get the expected result. Here is the code: HTable table = new HTable(conf, "apachelogs"); Scan scan = new Scan(); FilterList list = new FilterList(FilterList.Op

Re: Coprocessor end point vs MapReduce?

Doug, One thing that concerns me is that a lot of folks are gravitating to Coprocessors and may be using them for the wrong thing. Has anyone done any sort of research as to some of the limitations and negative impacts on using coprocessors? While I haven't really toyed with the idea of bulk

High IPC Latency

Hello, We are seeing slow times for read operations in our experiments. We are hoping that you guys can help us figure out what's going wrong. Here are some details: - We are running a read-only benchmark on our HBase cluster. - - There are 10 regionservers, each co-located with a datan

RE: High IPC Latency

Is it sustained for the same client hitting the same region server OR does it get better for the same client-RS combination when run for longer duration? Trying to eliminate Zookeeper from this. Thanks, Abhishek From: Yousuf Ahmad [mailto:myahm...@gmail.com] Sent: Thursday, October 18, 2012 11

Re: High IPC Latency

Also, what version of HBase/HDFS is this using? - Original Message - From: "Pamecha, Abhishek" To: "user@hbase.apache.org" Cc: Ivan Brondino ; Ricardo Vilaça Sent: Thursday, October 18, 2012 11:38 AM Subject: RE: High IPC Latency Is it sustained for the same client hitting the same

Re: Coprocessor end point vs MapReduce?

I agree with the concern and there isn't a ton of guidance on this area yet. On 10/18/12 2:01 PM, "Michael Segel" wrote: >Doug, > >One thing that concerns me is that a lot of folks are gravitating to >Coprocessors and may be using them for the wrong thing. >Has anyone done any sort of resea

Re: High IPC Latency

Hi, Thank you for your questions guys. We are using HBase 0.92 with HDFS 1.0.1. The experiment lasts 15 minutes. The measurements stabilize in the first two minutes of the run. The data is distributed almost evenly across the regionservers so each client hits most of them over the course of the

Re: Using filters in REST/stargate returns 204 (No content)

What does the HBase shell return if you try that scan programatically? On Thu, Oct 18, 2012 at 11:02 AM, Kumar, Suresh wrote: > > > I have a HBase Java client which has a couple of filters and just work > fine, I get the expected result. > > Here is the code: > > > > HTable table

Re: crafting your key - scan vs. get

Hi Neil, Mike summed it up well, as usual. :) Your choices of where to describe this "dimension" of your data (a one-to-many between users and events) are: - one row per event - one row per user, with events as columns - one row per user, with events as versions on a single cell The first tw

RE: Using filters in REST/stargate returns 204 (No content)

When I run the Java code, it returns the valid rows which match the regex. I base64encoded the qulaifier and family fields as well, still empty result. Suresh -Original Message- From: Andrew Purtell [mailto:apurt...@apache.org] Sent: Thursday, October 18, 2012 1:19 PM To: user@hbase.ap

Re: Unable to add co-processor to table through HBase api

Hi Folks, Still, i am unable to add the co-processors through HBase client api. This time i tried loading the coprocessor by providing the jar path along with parameters. But, it failed. I was able to add the same coprocessor to the table through HBase shell. I also dont see any logs regarding add

Re: error when open "hbase shell"

On Wed, Oct 17, 2012 at 2:42 AM, hua xiang wrote: > Hi, >when open hbase shell with "hdfs",there is an error. > > but "root" user can. >below is the error : > [root@hadoop2 ~]# su - hdfs > [hdfs@hadoop2 ~]$ id > uid=494(hdfs) gid=502(hadoop) groups=502(hadoop) > [hdfs@hadoop2 ~]$ hbase

Thrift Python client with regex

I am using Thrift (0.8.0) to get scan column values from a table. This code returns all the values. columns = ['mylog'] scanner = client.scannerOpen('apachelogs','', columns) result = client.scannerGet(scanner) while result: printRow(result[0]) result = client.scannerGet(scanner) p

Re: Thrift Python client with regex

We had the same question earlier. Unfortunately the documentation is wrong on this account; scannerOpen resolves to either a call to scan.addFamily or scan.addColumn, and neither directly supports regex matching. Regex pattern matching against colquals is definitely supported on the Java side, so

Re: WAL.Hlog vs. Hlog

On Thu, Oct 18, 2012 at 7:35 PM, Maoke wrote: > hi Stack and all, > > i noticed that the regionserver.Hlog is obsoleted by regionserver.wal.Hlog, > from version 0.20.6 to 0.90+. what is the major difference between the two, > in principle? what we should pay attention to when using the WAL.Hlog? >

Re: WAL.Hlog vs. Hlog

2012/10/19 Stack > On Thu, Oct 18, 2012 at 7:35 PM, Maoke wrote: > > hi Stack and all, > > > > i noticed that the regionserver.Hlog is obsoleted by > regionserver.wal.Hlog, > > from version 0.20.6 to 0.90+. what is the major difference between the > two, > > in principle? what we should pay atte

答复: hbase.client.scanner.timeout.period not being respected

Did you rebounce your server cluster ? Per HregionServer.java code : this.scannerLeaseTimeoutPeriod = conf.getInt(HConstants.HBASE_CLIENT_SCANNER_TIMEOUT_PERIOD, HConstants.DEFAULT_HBASE_CLIENT_SCANNER_TIMEOUT_PERIOD); seems this parameter is used by server side as well I am not an expe

RE: Coprocessor end point vs MapReduce?

A CP and Endpoints operates at a region level.. Any operation within one region we can perform using this.. I have seen in below use case that along with the delete there was a need for inserting data to some other table also.. Also this was kind of a periodic action.. I really doubt how the en

Re: Coprocessor end point vs MapReduce?

I might be little off here. If rows are moved to another table on weekly or daily basis, why not create per weekly or per day table. That way you need to copy and delete. Of course it will not work you are are selectively filtering between timestamps and clients have to have notion of multiple tabl

RE: High IPC Latency

Hi Yousuf > The client caches the locations of the regionservers, so after a couple > of > minutes of the experiment running, it wouldn't need to re-visit > ZooKeeper, > I believe. Correct me if I am wrong please. Yes you are right. Regards Ram > -Original Message- > From: Yousuf Ahmad [

RE: Unable to add co-processor to table through HBase api

>hAdmin.getTableDescriptor(Bytes.toBytes(tableName)).addCoprocessor(className, new Path("hdfs://hbasecluster/tmp/hbase_cdh4.jar"), Coprocessor.PRIORITY_USER,map); Anil, Don't you have to modify the table calling Admin API?? ! Not seeing that code here... -Anoop- __

Re: Unable to add co-processor to table through HBase api

Hi Anoop, Sorry, i am unable to understand what you mean by "have to modify the table calling Admin API??". Am i missing some other calls in my code? Thanks, Anil Gupta On Thu, Oct 18, 2012 at 9:43 PM, Anoop Sam John wrote: > > > >hAdmin.getTableDescriptor(Bytes.toBytes(tableName)).addCoproces

Re: High IPC Latency

Can you reproduce this against a single, local region server? Any chance that you can try with the just released 0.94.2? I would love to debug this. If would be a tremendous help if you had a little test program that reproduces this against a single server, so that I can see what is going on.

RE: Unable to add co-processor to table through HBase api

I can attach the code that I tried. Here as the HTD is getting modified we may need to call modifyTable(). My testclass did try this while doing creation of table itself. I will attach shortly. Regards Ram > -Original Message- > From: anil gupta [mailto:anilgupt...@gmail.com] > Sent: Fr

Re: Unable to add co-processor to table through HBase api

Hi Guys, Do you mean to say that i need to call the following method after the call to addCoprocessor method: public void *modifyTable*(byte[] tableName, HTableDescriptor htd)

RE: Unable to add co-processor to table through HBase api

Anil Yes the same. You got the HTD from the master to your client code and just added the CP into that Object. In order to reflect the change in the HBase cluster you need to call the modifyTable API with your changed HTD. Master will change the table. When you enable back the table, regio

RE: Unable to add co-processor to table through HBase api

Yes you are right. modifyTable has to be called. public class TestClass { private static HBaseTestingUtility UTIL = new HBaseTestingUtility(); @BeforeClass public static void setupBeforeClass() throws Exception { Configuration conf = UTIL.getConfiguration(); } @Before

Re: Unable to add co-processor to table through HBase api

Thanks a lot Guys. I really appreciate you help. I'll try this change in the morning and let you know the outcome. @Ram: Actually, i was trying to add the coprocessor to a per-existing table. I think yesterday you assumed that I am trying to add the coprocessor while creating the table. That's why

RE: Unable to add co-processor to table through HBase api

Ok Anil.. Not a problem.. My intention was to just see if the api was working during createtable so that it will help you. Regards Ram > -Original Message- > From: anil gupta [mailto:anilgupt...@gmail.com] > Sent: Friday, October 19, 2012 11:22 AM > To: user@hbase.apache.org > Subject: Re

Re: ANN: HBase 0.94.2 is available for download

+1 pushing to maven repo Thanks and Regards, S SYED ABDUL KATHER On Thu, Oct 18, 2012 at 10:32 PM, lars hofhansl wrote: > I'm on it. :) > > > > - Original Message - > From: Amit Sela > To: user@hbase.apache.org > Cc: > Sent: Thursday, October 18, 2012 8:25 AM > Su

回复: thrift usage

dear,hua i don't know 2012-10-19 Kind regards, Xu Jingang(徐金刚) Mobile: +86-18651865779 Email:jingan...@gmail.com 发件人：hua xiang 发送时间：2012-10-19 14:13 主题：thrift usage 收件人："user@hbase.apache.org" 抄送： HI, where is the interface of thrift, which is installed using rpm package. Bes