Re: does hbase master need to be a hadoop datanode as well?

2013-02-22 Thread anil gupta
On Feb 22, 2013 11:50 AM, "Jean-Marc Spaggiari" wrote: > > Just to add to Mohammad's advices, you should avoid to run ZK on the same > servers as you are running HBase. > > Reason is, if you are running in long GCs, ZK might miss the heartbeats and > thinks servers are down. So safer to run same s

Re: Problem In Understanding Compaction Process

2013-02-22 Thread Anty
Thanks Sergey In my use case. I want to directly analyze the underlying HFiles, So i can't tolerance duplicate data. Can you give me some pointers about how to make this procedure atomic? On Thu, Feb 21, 2013 at 6:07 AM, Sergey Shelukhin wrote: > There should be no duplicate records despite

Re: attributes - basic question

2013-02-22 Thread James Taylor
Same with us on Phoenix - we use the setAttribute on the client side and the getAttribute on the server side to pickup state on the Scan being executed. Works great. One thing to keep in mind, though: for a region observer coprocessor, the state you set on the client side will be sent to each r

RE: attributes - basic question

2013-02-22 Thread Anoop Sam John
We have used setAttribute() along with Scan which we are using in the CP. Ya it will work fine. Pls try with ur use case and if finding any issue pls report -Anoop- From: Toby Lazar [tla...@gmail.com] Sent: Saturday, February 23, 2013 4:07 AM To: user@hb

Re: issue with nutch-gora+hbase+zookeeper

2013-02-22 Thread kaveh minooie
In case anyone was wondering, the issue was resolved by copying the zoo.cfg in the hadoop conf directory (on classpath really) on the entire cluster. thanks On 02/22/2013 12:31 PM, kaveh minooie wrote: Hi everyone I am having this problem for couple of days now and would appreciate any idea

Re: Please welcome our newest committer: Sergey Shelukhin

2013-02-22 Thread ramkrishna vasudevan
Congrats Sergey. Regards Ram On Sat, Feb 23, 2013 at 7:17 AM, Marcos Ortiz wrote: > Congratulations, Sergey. > > > On 02/22/2013 04:39 PM, Ted Yu wrote: > >> Hi, >> Sergey has 51 issues under his name: >> * >> https://issues.apache.org/**jira/issues/?jql=project%20%** >> 3D%20HBASE%20AND%20assi

Re: Please welcome our newest committer: Sergey Shelukhin

2013-02-22 Thread Marcos Ortiz
Congratulations, Sergey. On 02/22/2013 04:39 PM, Ted Yu wrote: Hi, Sergey has 51 issues under his name: * https://issues.apache.org/jira/issues/?jql=project%20%3D%20HBASE%20AND%20assignee%20%3D%20sershe%20AND%20status%20in%20(Resolved%2C%20Closed) * * * * He was the driving force in finishing HB

Re: Please welcome our newest committer: Sergey Shelukhin

2013-02-22 Thread Jonathan Hsieh
Congrats Sergey, looking forware to more contributions. On Fri, Feb 22, 2013 at 1:39 PM, Ted Yu wrote: > Hi, > Sergey has 51 issues under his name: > * > > https://issues.apache.org/jira/issues/?jql=project%20%3D%20HBASE%20AND%20assignee%20%3D%20sershe%20AND%20status%20in%20(Resolved%2C%20Closed

Re: Please welcome our newest committer: Sergey Shelukhin

2013-02-22 Thread Sergey Shelukhin
Thanks guys! On Fri, Feb 22, 2013 at 4:24 PM, Jesse Yates wrote: > Great job Sergey! Welcome and keep up the good work. > --- > Jesse Yates > @jesse_yates > jyates.github.com > > > On Fri, Feb 22, 2013 at 4:20 PM, lars hofhansl wrote: > > > Congratulations, Sergey and welcome :)

Re: Please welcome our newest committer: Sergey Shelukhin

2013-02-22 Thread Jesse Yates
Great job Sergey! Welcome and keep up the good work. --- Jesse Yates @jesse_yates jyates.github.com On Fri, Feb 22, 2013 at 4:20 PM, lars hofhansl wrote: > Congratulations, Sergey and welcome :) > > > > > From: Ted Yu > To: d...@hbase.apache.or

Re: Please welcome our newest committer: Sergey Shelukhin

2013-02-22 Thread lars hofhansl
Congratulations, Sergey and welcome :) From: Ted Yu To: d...@hbase.apache.org; user@hbase.apache.org Sent: Friday, February 22, 2013 1:39 PM Subject: Please welcome our newest committer: Sergey Shelukhin Hi, Sergey has 51 issues under his name: * https://iss

Re: Please welcome our newest committer: Sergey Shelukhin

2013-02-22 Thread Elliott Clark
Welcome and Congrats. On Fri, Feb 22, 2013 at 2:15 PM, Gary Helmling wrote: > Congrats, Sergey! Great work! > > > On Fri, Feb 22, 2013 at 2:10 PM, Enis Söztutar wrote: > > > Congrats. Well deserved. > > > > > > On Fri, Feb 22, 2013 at 1:57 PM, Andrew Purtell > > wrote: > > > > > Congratulati

Question about object parameters to mappers

2013-02-22 Thread Gaetan Deputier
Hi Hbase users, (Using HBase 0.92.1 from Cloudera CDH4.1.1) I am currently writing a TableMapper which needs an object X defined outside my TableMapper class. I was wondering how can I pass this object X to my TableMapper ? I want to be sure that once the jobs are deployed on our cluster, X will

Re: attributes - basic question

2013-02-22 Thread Toby Lazar
Your last point was exactly what I was looking for. I am thinking about using attributes along with coprocessors to impose some application-level authorization constraints. For example, in a get, I can pass username and credential attributes and have the coprocessor filter results based on some r

Re: Please welcome our newest committer: Sergey Shelukhin

2013-02-22 Thread Gary Helmling
Congrats, Sergey! Great work! On Fri, Feb 22, 2013 at 2:10 PM, Enis Söztutar wrote: > Congrats. Well deserved. > > > On Fri, Feb 22, 2013 at 1:57 PM, Andrew Purtell > wrote: > > > Congratulations Sergey! > > > > > > On Fri, Feb 22, 2013 at 1:39 PM, Ted Yu wrote: > > > > > Hi, > > > Sergey ha

Re: does hbase master need to be a hadoop datanode as well?

2013-02-22 Thread Jean-Marc Spaggiari
Hi Harish, HBase still run ZooKeeper even in standalone mode: http://hbase.apache.org/book/standalone_dist.html So you still can face same kind of issues. But since it's not using Hadoop, there is more memory available for you HBase, which might reduce GCs "issues". JM 2013/2/22 Harish Krishnan

Re: Please welcome our newest committer: Sergey Shelukhin

2013-02-22 Thread Enis Söztutar
Congrats. Well deserved. On Fri, Feb 22, 2013 at 1:57 PM, Andrew Purtell wrote: > Congratulations Sergey! > > > On Fri, Feb 22, 2013 at 1:39 PM, Ted Yu wrote: > > > Hi, > > Sergey has 51 issues under his name: > > * > > > > > https://issues.apache.org/jira/issues/?jql=project%20%3D%20HBASE%20A

Re: Please welcome our newest committer: Sergey Shelukhin

2013-02-22 Thread Andrew Purtell
Congratulations Sergey! On Fri, Feb 22, 2013 at 1:39 PM, Ted Yu wrote: > Hi, > Sergey has 51 issues under his name: > * > > https://issues.apache.org/jira/issues/?jql=project%20%3D%20HBASE%20AND%20assignee%20%3D%20sershe%20AND%20status%20in%20(Resolved%2C%20Closed) > * > * > * > * He was the dr

Re: issue with nutch-gora+hbase+zookeeper

2013-02-22 Thread kaveh
all the ports are open. there is no iptable. -Original Message- From: alx...@aim.com Sent: Friday, February 22, 2013 1:02pm To: user@hbase.apache.org Subject: Re: issue with nutch-gora+hbase+zookeeper Turn off iptables and see if it resolves errors. You need to open all ports that hbase

Please welcome our newest committer: Sergey Shelukhin

2013-02-22 Thread Ted Yu
Hi, Sergey has 51 issues under his name: * https://issues.apache.org/jira/issues/?jql=project%20%3D%20HBASE%20AND%20assignee%20%3D%20sershe%20AND%20status%20in%20(Resolved%2C%20Closed) * * * * He was the driving force in finishing HBASE-5416 Improve performance of scans with some kind of filters. *

Re: does hbase master need to be a hadoop datanode as well?

2013-02-22 Thread Harish Krishnan
So in case of a stand alone set up, how come we are not experiencing those issues? Is it because of the limited data set that we are operating on? Thanks & Regards, Harish.T.K On Fri, Feb 22, 2013 at 1:19 PM, Harsh J wrote: > While this general guideline of not requiring a DN on HM is correct,

Re: attributes - basic question

2013-02-22 Thread Harsh J
The attributes are serialized along with the base operation request. There's perhaps no immediate client-side usage of this, it is used by the Mutation class to set a cluster ID in HBase's Replication context: http://svn.apache.org/viewvc/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hba

Re: does hbase master need to be a hadoop datanode as well?

2013-02-22 Thread Harsh J
While this general guideline of not requiring a DN on HM is correct, I just wanted to point out that the HM is hardly high on memory usage/requirements (as possibly stated below), as all it does is maintain the cluster and not serve nor hold any client-useful data or frequent requests. On Sat, Feb

Re: issue with nutch-gora+hbase+zookeeper

2013-02-22 Thread alxsss
Turn off iptables and see if it resolves errors. You need to open all ports that hbase and zookeeper need to connect each node. Alex. -Original Message- From: kaveh minooie To: user Sent: Fri, Feb 22, 2013 12:56 pm Subject: Re: issue with nutch-gora+hbase+zookeeper no here a

Re: issue with nutch-gora+hbase+zookeeper

2013-02-22 Thread kaveh minooie
no here are the logs, master: 2013-02-22 12:03:21,136 INFO org.apache.hadoop.hbase.master.LoadBalancer: Skipping load balancing. servers=10 regions=3 average=0.3 mostloaded=1 leastloaded=0 2013-02-22 12:03:21,141 DEBUG org.apache.hadoop.hbase.master.CatalogJanitor: Scanned 1 catalog row(s) an

Re: issue with nutch-gora+hbase+zookeeper

2013-02-22 Thread alxsss
Hi, Do you see any errors in HMaster and Regionserver logs? Alex. -Original Message- From: kaveh minooie To: user ; user ; user Sent: Fri, Feb 22, 2013 12:31 pm Subject: issue with nutch-gora+hbase+zookeeper Hi everyone I am having this problem for couple of days now and w

issue with nutch-gora+hbase+zookeeper

2013-02-22 Thread kaveh minooie
Hi everyone I am having this problem for couple of days now and would appreciate any idea or suggestion that any one might have. I am using nutch 2.x with hbase. due to nutch requirement I need to use older version of hbase (I am using 0.90.6 over hadoop 1.1.1 with 10 nodes with zookeepr 3.5.

Re: does hbase master need to be a hadoop datanode as well?

2013-02-22 Thread Mohammad Tariq
Yeah..Good point JM. Warm Regards, Tariq https://mtariq.jux.com/ cloudfront.blogspot.com On Sat, Feb 23, 2013 at 1:19 AM, Jean-Marc Spaggiari < jean-m...@spaggiari.org> wrote: > Just to add to Mohammad's advices, you should avoid to run ZK on the same > servers as you are running HBase. > > Rea

Re: does hbase master need to be a hadoop datanode as well?

2013-02-22 Thread Jean-Marc Spaggiari
Just to add to Mohammad's advices, you should avoid to run ZK on the same servers as you are running HBase. Reason is, if you are running in long GCs, ZK might miss the heartbeats and thinks servers are down. So safer to run same separately if you can. JM 2013/2/22 Mohammad Tariq > Hello Kaven

Re: does hbase master need to be a hadoop datanode as well?

2013-02-22 Thread Mohammad Tariq
Hello Kaven, It's not all necessary to run HM on server running DN. Actually it's not advisable to do so, as HM has high memory requirements and running other daemons on the same machine may lead you to problems. The basic cluster setup, which people normally use looks somewhat like this :

does hbase master need to be a hadoop datanode as well?

2013-02-22 Thread kaveh minooie
Hi everyone, I had a quick question. do I need to be running datanode on the server that is running hbase master or datanode should be running only on servers that are running as regionserver? thanks, -- Kaveh Minooie

attributes - basic question

2013-02-22 Thread Toby Lazar
What is the purpose of the getAttribute and setAttribute methods for classes implementing OperationWithAttributes? I don't see much documentation about them, haven't seen much discussion on this list about them, and am wondering what people use them for. Or, are they mostly just used internally?

Re: Co-Processor in scanning the HBase's Table

2013-02-22 Thread Anoop John
yes what Ted mentioned, u can try out.. -Anoop- On Fri, Feb 22, 2013 at 7:46 PM, Ted wrote: > Hbase-5416 should help your use case. > Cf2 would be the essential column family. > > This feature is available in 0.94.5 > > Cheers > > On Feb 22, 2013, at 5:55 AM, Farrokh Shahriari < > mohandes.zebe

Re: Co-Processor in scanning the HBase's Table

2013-02-22 Thread Ted
Hbase-5416 should help your use case. Cf2 would be the essential column family. This feature is available in 0.94.5 Cheers On Feb 22, 2013, at 5:55 AM, Farrokh Shahriari wrote: > @Anoop : Here is my situation: I have 2 columnFamilies(cf1,cf2),5-50 > qualifiers(they are variable),first I sho

Re: Co-Processor in scanning the HBase's Table

2013-02-22 Thread Farrokh Shahriari
@Anoop : Here is my situation: I have 2 columnFamilies(cf1,cf2),5-50 qualifiers(they are variable),first I should check the value of qualifiers in cf2,then if they equal to some values I'm going to get the values of cf1 by that rowkey. @Amita: thank you.I will check that too. Farrokh On Thu, Feb