Re: How HBase perform per-column scan?

2013-03-10 Thread Ted Yu
Hi, Yun: Take a look at HBASE-5416 (Improve performance of scans with some kind of filters) which is in 0.94.5 release. In your case, you can use a filter which specifies column C as the essential family. Here I interpret column C as column family. Cheers On Sat, Mar 9, 2013 at 11:11 AM, yun

Re: How HBase perform per-column scan?

2013-03-10 Thread PG
Hi, Ted and Anoop, thanks for your notes. I am talking about column rather than column family, since physically column family should be able to perform efficiently (storage layer, CF's are stored separately). But columns of the same column family may be mixed physically, and that makes filters

Re: How HBase perform per-column scan?

2013-03-10 Thread Ted Yu
bq. physically column family should be able to perform efficiently (storage layer When you scan a row, data for different column families would be brought into memory (if you don't utilize HBASE-5416) Take a look at:

Re: How HBase perform per-column scan?

2013-03-10 Thread Anoop John
As per the above said, you will need a full table scan on that CF. As Ted said, consider having a look at your schema design. -Anoop- On Sun, Mar 10, 2013 at 8:10 PM, Ted Yu yuzhih...@gmail.com wrote: bq. physically column family should be able to perform efficiently (storage layer When

Re: Welcome our newest Committer Anoop

2013-03-10 Thread Ted Yu
Congratulations, Anoop. Keep up the good work. On Sun, Mar 10, 2013 at 9:42 AM, ramkrishna vasudevan ramkrishna.s.vasude...@gmail.com wrote: Hi All Pls welcome Anoop, our newest committer. Anoop's work in HBase has been great and he has helped lot of users in the mailing list. He has

Re: Welcome our newest Committer Anoop

2013-03-10 Thread Anil Gupta
Congrats Welcome Anoop!! Best Regards, Anil On Mar 10, 2013, at 9:58 AM, Ted Yu yuzhih...@gmail.com wrote: Congratulations, Anoop. Keep up the good work. On Sun, Mar 10, 2013 at 9:42 AM, ramkrishna vasudevan ramkrishna.s.vasude...@gmail.com wrote: Hi All Pls welcome Anoop, our

Re: Welcome our newest Committer Anoop

2013-03-10 Thread Jesse Yates
Great job Anoop! Keep up the good work. - Jesse Yates Sent from my iPhone On Mar 10, 2013, at 9:42 AM, ramkrishna vasudevan ramkrishna.s.vasude...@gmail.com wrote: Hi All Pls welcome Anoop, our newest committer. Anoop's work in HBase has been great and he has helped lot of users in the

can we use same column name for 2 different column families?

2013-03-10 Thread Ramasubramanian Narayanan
Hi, Is it fine to use same column name for 2 different column families? For example, In a table emp, can we have column name dob under column family F1 F2? Please let me know the impact of having like this if any... Note : I don't want to use dob1 or some other field name for the second

Re: can we use same column name for 2 different column families?

2013-03-10 Thread shashwat shriparv
Ya sure you can have that... as you always specify columnfamily to read out of hbase, i dont think there should be a problem in that. ∞ Shashwat Shriparv On Sun, Mar 10, 2013 at 11:41 PM, Ramasubramanian Narayanan ramasubramanian.naraya...@gmail.com wrote: Hi, Is it fine to use same

Re: RegionServers Crashing every hour in production env

2013-03-10 Thread Pablo Musa
That combo should be fine. Great!! If JVM is full GC'ing, the application is stopped. The below does not look like a full GC but that is a long pause in system time, enough to kill your zk session. Exactly. This pause is really making the zk expire the RS which shutsdown (logs in the end

Re: RegionServers Crashing every hour in production env

2013-03-10 Thread Sreepathi
Hi Stack/Ted/Pablo, Should we increase the hbase.rpc.timeout property to 5 minutes ( 30 ms ) ? Regards, - Sreepathi On Sun, Mar 10, 2013 at 11:59 AM, Pablo Musa pa...@psafe.com wrote: That combo should be fine. Great!! If JVM is full GC'ing, the application is stopped. The below

Re: unsubscribe request

2013-03-10 Thread Panshul Whisper
lol On Sun, Mar 10, 2013 at 7:29 PM, Dan Han dannahan2...@gmail.com wrote: I would like to unsubscribe now as the email is huge. Thanks. Best Wishes Dan Han -- Regards, Ouch Whisper 010101010101

Re: Welcome our newest Committer Anoop

2013-03-10 Thread lars hofhansl
Congrats Anoop. Welcome. (Don't break the tests :) ) From: ramkrishna vasudevan ramkrishna.s.vasude...@gmail.com To: d...@hbase.apache.org; user@hbase.apache.org Sent: Sunday, March 10, 2013 9:42 AM Subject: Welcome our newest Committer Anoop Hi All Pls

Re: unsubscribe request

2013-03-10 Thread lars hofhansl
No problem, Dan. From: Dan Han dannahan2...@gmail.com To: user@hbase.apache.org Sent: Sunday, March 10, 2013 12:36 PM Subject: Re: unsubscribe request Embarassing! It was supposed to send to another email address. Sorry about this. Best Wishes Dan Han On

Re: RegionServers Crashing every hour in production env

2013-03-10 Thread Pablo Musa
Hi Sreepathi, they say in the book (or the site), we could try it to see if it is really a timeout error or there is something more. But it is not recomended for production environments. I could give it a try if five minutes will ensure to us that the problem is the GC or elsewhere!! Anyway,

Re: RegionServers Crashing every hour in production env

2013-03-10 Thread Stack
You could increase your zookeeper session timeout to 5 minutes while you are figuring why these long pauses. http://hbase.apache.org/book.html#zookeeper.session.timeout Above, there is an outage for almost 5 minutes: We slept 225100ms instead of 3000ms, this is likely due to a long You have

Re: Welcome our newest Committer Anoop

2013-03-10 Thread Stack
Good on you Anoop! St.Ack On Sun, Mar 10, 2013 at 9:42 AM, ramkrishna vasudevan ramkrishna.s.vasude...@gmail.com wrote: Hi All Pls welcome Anoop, our newest committer. Anoop's work in HBase has been great and he has helped lot of users in the mailing list. He has contributed features

Re: Welcome our newest Committer Anoop

2013-03-10 Thread Andrew Purtell
Congratulations Anoop. Welcome! On Mon, Mar 11, 2013 at 12:42 AM, ramkrishna vasudevan ramkrishna.s.vasude...@gmail.com wrote: Hi All Pls welcome Anoop, our newest committer. Anoop's work in HBase has been great and he has helped lot of users in the mailing list. He has contributed

RE: How HBase perform per-column scan?

2013-03-10 Thread Liu, Raymond
Just curious, won't ROWCOL bloom filter works for this case? Best Regards, Raymond Liu As per the above said, you will need a full table scan on that CF. As Ted said, consider having a look at your schema design. -Anoop- On Sun, Mar 10, 2013 at 8:10 PM, Ted Yu yuzhih...@gmail.com

Re: RegionServers Crashing every hour in production env

2013-03-10 Thread Azuryy Yu
Hi Pablo, It'a terrible for a long minor GC. I don't think there are swaping from your vmstat log. but I just suggest you 1) add following JVM options: -XX:+DisableExplicitGC -XX:+UseCompressedOops -XX:GCTimeRatio=19 -XX:SoftRefLRUPolicyMSPerMB=0 -XX:SurvivorRatio=2 -XX:MaxTenuringThreshold=3

Re: RegionServers Crashing every hour in production env

2013-03-10 Thread Azuryy Yu
Pablo, another, what's your java version? On Mon, Mar 11, 2013 at 10:13 AM, Azuryy Yu azury...@gmail.com wrote: Hi Pablo, It'a terrible for a long minor GC. I don't think there are swaping from your vmstat log. but I just suggest you 1) add following JVM options: -XX:+DisableExplicitGC

Re: RegionServers Crashing every hour in production env

2013-03-10 Thread Andrew Purtell
Be careful with GC tuning, throwing changes at an application without analysis of what is going on with the heap is shooting in the dark. One particular good treatment of the subject is here: http://java.dzone.com/articles/how-tame-java-gc-pauses If you have made custom changes to blockcache or

RE: Welcome our newest Committer Anoop

2013-03-10 Thread Anoop Sam John
Thanks to all.. Hope to work more and more for HBase! -Anoop- From: Andrew Purtell [apurt...@apache.org] Sent: Monday, March 11, 2013 7:33 AM To: user@hbase.apache.org Subject: Re: Welcome our newest Committer Anoop Congratulations Anoop. Welcome! On

RE: Welcome our newest Committer Anoop

2013-03-10 Thread Uma Maheswara Rao G
Congrats Anoop. Great work! Regards, Uma From: ramkrishna vasudevan [ramkrishna.s.vasude...@gmail.com] Sent: Sunday, March 10, 2013 10:12 PM To: d...@hbase.apache.org; user@hbase.apache.org Subject: Welcome our newest Committer Anoop Hi All Pls welcome

RE: Welcome our newest Committer Anoop

2013-03-10 Thread rajeshbabu chintaguntla
Contratulations Anoop! From: Anoop Sam John [anoo...@huawei.com] Sent: Monday, March 11, 2013 9:00 AM To: user@hbase.apache.org Subject: RE: Welcome our newest Committer Anoop Thanks to all.. Hope to work more and more for HBase! -Anoop-

RE: How HBase perform per-column scan?

2013-03-10 Thread Anoop Sam John
ROWCOL bloom says whether for a given row (rowkey) a given column (qualifier) is present in an HFile or not. But for the user he dont know the rowkeys. He wants all the rows with column 'x' -Anoop- From: Liu, Raymond [raymond@intel.com] Sent:

RE: can we use same column name for 2 different column families?

2013-03-10 Thread Anoop Sam John
can we have column name dob under column family F1 F2? Just fine.. Go ahead.. :) -Anoop- From: Ramasubramanian Narayanan [ramasubramanian.naraya...@gmail.com] Sent: Sunday, March 10, 2013 11:41 PM To: user@hbase.apache.org Subject: can we use same column

Re: can we use same column name for 2 different column families?

2013-03-10 Thread ramkrishna vasudevan
As note of caution just don have two empty qualifiers in the same CF but u can still have empty qualifiers in diff CFs. Regards Ram On Mon, Mar 11, 2013 at 10:20 AM, Anoop Sam John anoo...@huawei.com wrote: can we have column name dob under column family F1 F2? Just fine.. Go ahead.. :)

RE: How HBase perform per-column scan?

2013-03-10 Thread Liu, Raymond
Hmm, I don't mean query bloom filter directly. I mean the storefilescanner will query rowcol bloomfilter to see is it need a seek or not. And I guess this will be performed on every row without need to specific a row keys? ROWCOL bloom says whether for a given row (rowkey) a given column