Re: Exceeded limits on number of counters

[Michel Segel]

If you are using EMR you need to set this on startup you can't change it on the 
fly...

Sent from a remote device. Please excuse any typos...

Mike Segel

On Jul 2, 2013, at 12:24 PM, Jean-Marc Spaggiari  
wrote:

> Hi Glen,
> 
> You don't need to recompile to change this limit...
> 
> Take a look there:
> http://stackoverflow.com/questions/12140177/more-than-120-counters-in-hadoop
> 
> JM
> 
> 2013/7/2 Glen Arrowsmith :
>> Hi,
>> I'm getting an error on a map reduce task that use to work just fine for a 
>> few weeks.
>> 
>> Exceeded limits on number of counters - Counters=120 Limit=120
>> 
>> The full stderr output is at the bottom.
>> 
>> I'm using Amazon's Elastic MapReduce.
>> The following command starts the job
>> elastic-mapreduce --create --name REGISTER table to S3 v2" --num-instances 6 
>> --with-supported-products mapr-m5 --instance-type m1.xlarge --hive-script 
>> --arg s3://censored/dynamo-to-s3-v2.h --args 
>> -d,OUTPATH=s3://censored/out/,-d,INTABLE="REGISTER"
>> 
>> From what I've read you can't change the counter limit without recompiling.
>> 
>> Originally I had "fixed" this problem by upgrading from standard map reduce 
>> instances to mapr-m5 instances but that stopped working now for some reason.
>> 
>> Thanks very much in advance for your help
>> 
>> Glen Arrowsmith
>> Systems Architect
>> 
>> 
>> /mnt/var/lib/hadoop/steps/2/./hive-script:326: warning: Insecure world 
>> writable dir /home/hadoop/bin in PATH, mode 040757
>> Logging initialized using configuration in 
>> file:/home/hadoop/.versions/hive-0.8.1/conf/hive-log4j.properties
>> Hive history 
>> file=/mnt/var/lib/hive_081/tmp/history/hive_job_log_hadoop_201307020009_133883985.txt
>> OK
>> [snip]
>> Time taken: 0.389 seconds
>> OK
>> Time taken: 0.382 seconds
>> Total MapReduce jobs = 12
>> Launching Job 1 out of 12
>> Number of reduce tasks not specified. Defaulting to jobconf value of: 10
>> In order to change the average load for a reducer (in bytes):
>>  set hive.exec.reducers.bytes.per.reducer=
>> In order to limit the maximum number of reducers:
>>  set hive.exec.reducers.max=
>> In order to set a constant number of reducers:
>>  set mapred.reduce.tasks=
>> Starting Job = job_201307020007_0001, Tracking URL = 
>> http://ip-10-151-78-231.ec2.internal:9100/jobdetails.jsp?jobid=job_201307020007_0001
>> Kill Command = /opt/mapr/hadoop/hadoop-0.20.2/bin/../bin/hadoop job  
>> -Dmapred.job.tracker=maprfs:/// -kill job_201307020007_0001
>> Hadoop job information for Stage-12: number of mappers: 23; number of 
>> reducers: 10
>> 2013-07-02 00:09:30,325 Stage-12 map = 0%,  reduce = 0%
>> org.apache.hadoop.mapred.Counters$CountersExceededException: Error: Exceeded 
>> limits on number of counters - Counters=120 Limit=120
>>at 
>> org.apache.hadoop.mapred.Counters$Group.getCounterForName(Counters.java:318)
>>at 
>> org.apache.hadoop.mapred.Counters.findCounter(Counters.java:439)
>>at 
>> org.apache.hadoop.mapred.Counters.getCounter(Counters.java:503)
>>at 
>> org.apache.hadoop.hive.ql.exec.Operator.updateCounters(Operator.java:1150)
>>at 
>> org.apache.hadoop.hive.ql.exec.ExecDriver.updateCounters(ExecDriver.java:1281)
>>at 
>> org.apache.hadoop.hive.ql.exec.HadoopJobExecHelper.updateCounters(HadoopJobExecHelper.java:85)
>>at 
>> org.apache.hadoop.hive.ql.exec.HadoopJobExecHelper.progress(HadoopJobExecHelper.java:312)
>>at 
>> org.apache.hadoop.hive.ql.exec.HadoopJobExecHelper.progress(HadoopJobExecHelper.java:685)
>>at 
>> org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:494)
>>at 
>> org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:136)
>>at 
>> org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:133)
>>at 
>> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
>>at 
>> org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:47)
>> Ended Job = job_201307020007_0001 with exception 
>> 'org.apache.hadoop.mapred.Counters$CountersExceededException(Error: Exceeded 
>> limits on number of counters - Counters=120 Limit=120)'
>> FAILED: Execution Error, return code 1 from 
>> org.apache.hadoop.hive.ql.exec.MapRedTask
>> Command exiting with ret '255'
> 

Re: is hbase cluster support multi-instance?

[Michel Segel]

In theory or in practice. In practice, not a good idea and its not as straight 
forward as it seems.

Sent from a remote device. Please excuse any typos...

Mike Segel

On Jun 28, 2013, at 2:00 PM, rajeshbabu chintaguntla 
 wrote:

> you can start multiple instances of hbase by managing unique zookeeper quorum 
> and hdfs for each hbase instance. 
> 
> 
> Thanks and Regards,
> Rajeshbabu
> 
> From: ch huang [justlo...@gmail.com]
> Sent: Friday, June 28, 2013 10:05 AM
> To: user@hbase.apache.org
> Subject: is hbase cluster support multi-instance?
> 
> hi all:
> can hbase start more than one instance ,like mysql, if can ,how to manage
> these instances? ,thanks a lot

Re: Schema design for filters

[Michel Segel]

This doesn't make sense in that the OP wants schema less  structure, yet wants 
filtering on columns. The issue is that you do have a limited Schema, so Schema 
less is a misnomer.

In order to do filtering, you need to enforce object type within a column which 
requires a Schema to be enforced.

Again, this can be done in HBase.



Sent from a remote device. Please excuse any typos...

Mike Segel

On Jun 28, 2013, at 4:30 PM, Asaf Mesika  wrote:

> Yep. Other DBs like
> Mongo may have the stuff you need out of the box.
> Another option is to encode the whole class using Avro, and writing a
> filter on top of that.
> You basically use one column and store it there.
> Yes, you pay the penalty of loading your entire class and extract the
> fields you need to compare against, but I'm really not sure the other way
> is faster, taking into account the hint mechanism in Filter which is
> pinpointed thus grabs more bytes than it needs to.
> 
> Back what was said earlier: 1M rows- why not MySql?
> 
> On Friday, June 28, 2013, Otis Gospodnetic wrote:
> 
>> Hi,
>> 
>> I see.  Btw. isn't HBase for < 1M rows an overkill?
>> Note that Lucene is schemaless and both Solr and Elasticsearch can
>> detect field types, so in a way they are schemaless, too.
>> 
>> Otis
>> --
>> Performance Monitoring -- http://sematext.com/spm
>> 
>> 
>> 
>> On Fri, Jun 28, 2013 at 2:53 PM, Kristoffer Sjögren 
>> wrote:
>>> @Otis
>>> 
>>> HBase is a natural fit for my usecase because its schemaless. Im
>> building a
>>> configuration management system and there is no need for advanced
>>> filtering/querying capabilities, just basic predicate logic and
>> pagination
>>> that scales to < 1 million rows with reasonable performance.
>>> 
>>> Thanks for the tip!
>>> 
>>> 
>>> On Fri, Jun 28, 2013 at 8:34 PM, Otis Gospodnetic <
>>> otis.gospodne...@gmail.com> wrote:
>>> 
 Kristoffer,
 
 You could also consider using something other than HBase, something
 that supports "secondary indices", like anything that is Lucene based
 - Solr and ElasticSearch for example.  We recently compared how we
 aggregate data in HBase (see my signature) and how we would do it if
 we were to use Solr (or ElasticSearch), and so far things look better
 in Solr for our use case.  And our use case involves a lot of
 filtering, slicing and dicing. something to consider...
 
 Otis
 --
 Solr & ElasticSearch Support -- http://sematext.com/
 Performance Monitoring -- http://sematext.com/spm
 
 
 
 On Fri, Jun 28, 2013 at 5:24 AM, Kristoffer Sjögren 
 wrote:
> Interesting. Im actually building something similar.
> 
> A fullblown SQL implementation is bit overkill for my particular
>> usecase
> and the query API is the final piece to the puzzle. But ill definitely
 have
> a look for some inspiration.
> 
> Thanks!
> 
> 
> 
> On Fri, Jun 28, 2013 at 3:55 AM, James Taylor  wrote:
> 
>> Hi Kristoffer,
>> Have you had a look at Phoenix (
>> https://github.com/forcedotcom/phoenix
 )?
>> You could model your schema much like an O/R mapper and issue SQL
 queries
>> through Phoenix for your filtering.
>> 
>> James
>> @JamesPlusPlus
>> http://phoenix-hbase.blogspot.com
>> 
>> On Jun 27, 2013, at 4:39 PM, "Kristoffer Sjögren" 
>> wrote:
>> 
>>> Thanks for your help Mike. Much appreciated.
>>> 
>>> I dont store rows/columns in JSON format. The schema is exactly
>> that
 of a
>>> specific java class, where the rowkey is a unique object identifier
 with
>>> the class type encoded into it. Columns are the field names of the
 class
>>> and the values are that of the object instance.
>>> 
>>> Did think about coprocessors but the schema is discovered a runtime
 and I
>>> cant hard code it.
>>> 
>>> However, I still believe that filters might work. Had a look
>>> at SingleColumnValueFilter and this filter is be able to target
 specific
>>> column qualifiers with specific WritableByteArrayComparables.
>>> 
>>> But list comparators are still missing... So I guess the only way
>> is
 to
>>> write these comparators?
>>> 
>>> Do you follow my reasoning? Will it work?
>>> 
>>> 
>>> 
>>> 
>>> On Fri, Jun 28, 2013 at 12:58 AM, Michael Segel
>>> <

Re: How many column families in one table ?

[Michel Segel]

Short answer... As few as possible.

14 CF doesn't make too much sense.

Sent from a remote device. Please excuse any typos...

Mike Segel

On Jun 28, 2013, at 12:20 AM, Vimal Jain  wrote:

> Hi,
> How many column families should be there in an hbase table ? Is there any
> performance issue in read/write if we have more column families ?
> I have designed one table with around 14 column families in it with each
> having on average 6 qualifiers.
> Is it a good design ?
> 
> -- 
> Thanks and Regards,
> Vimal Jain

Re: Replication - ports/hosts

[Michel Segel]

What no ack from slave back to master???

Sent from a remote device. Please excuse any typos...

Mike Segel

On Jun 19, 2013, at 12:41 AM, Stack  wrote:

> On Mon, Jun 17, 2013 at 12:06 PM, Patrick Schless > wrote:
> 
>> Working on setting up HBase replication across a VPN tunnel, and following
>> the docs here: [1] (and here: [2]).
>> 
>> Two questions, regarding firewall allowances required:
>> 1) The docs say that the zookeeper clusters must be able to reach each
>> other. I don't see any docs on why this is (the high-level diagram doesn't
>> even show a ZK ensemble on the client side). Is it correct that the ZK
>> ensembles need to communicate with each other?
> Yes.  ZK ensemble is the gateway to the cluster whether connecting locally
> or remotely as replication does.
> 
> 
> 
>> 2) What ports are required to be open for inter-cluster Region Server
>> communication? Is it only the slave that needs to accept requests from the
>> master, or does the master also accept incoming requests from the slaves?
> 
> 
> Do you mean master 'cluster' in the above?  If so, yes, the master cluster
> sends data to the slave cluster (master RS to slave RSs).  The
> communication is one way; master to slave cluster.
> 
> If you are asking if hbase masters need to communicate, the answer is no.
> 
> St.Ack
> P.S. Doug, we need to add above to doc

Re: observer coprocessor question regarding puts

[Michel Segel]

Ok...

But then you are duplicating the data, so you will have to reconcile the two 
sets and there is a possibility that the data sets are out of sync.

I don't know your entire Schema, but if the row key is larger than the value, 
you may want to think about changing the Schema.


Sent from a remote device. Please excuse any typos...

Mike Segel

On Jun 13, 2013, at 9:34 PM, rob mancuso  wrote:

> Thx Mike, for the most part.
> 
> My key is substantially larger than my value, so I was thinking of leaving
> the cq->value stuff as is and just inverting the rowkey.
> 
> So the original table would have
> 
> [A, B, C] cf1:cq1 val1
> 
> And the secondary table would have
> 
> [C, B, A] cf1:cq1 val1
> On Jun 10, 2013 3:42 PM, "Michael Segel"  wrote:
> 
>> 
>> If I understand you ...
>> 
>> You have the row key = [A,B,C]
>> You want to create an inverted mapping of  Key [C] => {[A,B,C]}
>> 
>> That is to say that your inverted index would be all of the rows where the
>> value of C = x  .
>> And x is some value.
>> 
>> You should have to worry about column qualifiers just the values of A , B
>> and C.
>> 
>> In this case, the columns in your index will also be the values of the
>> tuples.
>> You really don't need C because you already have it, but then you'd need
>> to remember to add it to the pair (A, B) that you are storing.
>> I'd say waste the space and store (A,B,C) but that's just me.
>> 
>> 
>> Is that what you want to do?
>> 
>> -Mike
>> 
>> On Jun 9, 2013, at 12:16 PM, rob mancuso  wrote:
>> 
>>> Thx Anoop, I believe this is what I'm looking for.
>>> 
>>> Regarding my use case,  my rowkey is [A,B,C], but i also have a
>> requirement
>>> to access data by [C] only.  So I'm looking to use a post-put coprocessor
>>> to maintain one secondary index table where the rowkey starts with [C].
>> My
>>> cqs are numerics representing time and can be any number btw 1 and 3600
>> (ie
>>> seconds within an hour). Because I won't know the cq value for each
>>> incoming put (just the cf), I need something to deconstruct the put into
>> a
>>> list of cqs ...which I believe you've provided with getFamilyMap.
>>> 
>>> Thx again!
>>> On Jun 9, 2013 12:47 AM, "Anoop John"  wrote:
>>> 
 You want to have an index per every CF+CQ right?  You want to maintain
>> diff
 tables for diff columns?
 
 Put is having getFamilyMap method Map CF vs List KVs.  From this List of
 KVs you can get all the CQ names and values etc..
 
 -Anoop-
 
 On Sat, Jun 8, 2013 at 11:24 PM, rob mancuso 
>> wrote:
 
> Hi,
> 
> I'm looking to write a post-put observer coprocessor to maintain a
> secondary index.  Basically, my current rowkey design is a composite of
> A,B,C and I want to be able to also access data by C.  So all i'm
>> looking
> to do is invert the rowkey and apply it for all cf:cq values that come
 in.
> 
> My problem (i think), is that in all the good examples i've seen, they
 all
> deconstruct the Put by calling put.get(,)...implying they know
 the
> qualifier ahead of time.  I'm looking to specify the family and
>> generate
 a
> put to the secondary index table for all qualifiers ...not knowing or
> caring what the qualifier is.
> 
> Any pointers would be appreciated,
> Thx - Rob
> 
> Is there a way
>> 
>> 

Re: some problem about HBase index

[Michel Segel]

Huh?

Sorry but HBase *DOES* support composite keys.

The OP's question is can you create a row key that is comprised of both 
'student' and 'student_id'.

I'm not going in to more detail because this sounds like a homework problem.

Composite rowkey and composite index are two different things, yet still 
related because the row key has an implied index.


Sent from a remote device. Please excuse any typos...

Mike Segel

On Jun 13, 2013, at 5:22 AM, rajeshbabu chintaguntla 
 wrote:

> 
> HBase does not fully support composite keys.
> But you can use PrefixFilter and FuzzyRowFilter with some limitations if your 
> query involves part of key.
> http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/filter/PrefixFilter.html
> http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/filter/FuzzyRowFilter.html
> 
> Thanks,
> Rajeshbabu.
> 
> From: fx_bull [javac...@gmail.com]
> Sent: Thursday, June 13, 2013 1:46 PM
> To: user@hbase.apache.org
> Cc: d...@hbase.apache.org
> Subject: some problem about HBase index
> 
> hello
> 
> If I have a table called 'student',  I want to  use both of  student 
> number and student' name as  rowkey  to mark a student,
> 
>I want to verify can HBase support it  ,  and  if  it can, how should I do,
> 
>Thanks!

Re: Regarding Indexing columns in HBASE

[Michel Segel]

Quick and dirty...

Create an inverted table for each index
Then you can take the intersection of the result set(s) to get your list of 
rows for further filtering.

There is obviously more to this, but its the core idea...


Sent from a remote device. Please excuse any typos...

Mike Segel

On Jun 4, 2013, at 11:51 AM, Shahab Yunus  wrote:

> Just a quick thought, why don't you create different tables and duplicate
> data i.e. go for demoralization and data redundancy. Is your all read
> access patterns that would require 70 columns are incorporated into one
> application/client? Or it will be bunch of different clients/applications?
> If that is not the case then I think why not take advantage of more storage.
> 
> Regards,
> Shahab
> 
> 
> On Tue, Jun 4, 2013 at 12:43 PM, Ramasubramanian Narayanan <
> ramasubramanian.naraya...@gmail.com> wrote:
> 
>> Hi,
>> 
>> In a HBASE table, there are 200 columns and the read pattern for diffferent
>> systems invols 70 columns...
>> In the above case, we cannot have 70 columns in the rowkey which will not
>> be a good design...
>> 
>> Can you please suggest how to handle this problem?
>> Also can we do indexing in HBASE apart from rowkey? (something called
>> secondary index)
>> 
>> regards,
>> Rams
>> 

Re: querying hbase

[Michel Segel]

> Is there a benefit to restarting a regionserver in an OSGi container versus
> restarting a Java process?

Was that rhetorical?

Absolutely.
Think of a production environment where you are using HBase to serve data in 
real time.


Sent from a remote device. Please excuse any typos...

Mike Segel

On May 24, 2013, at 4:50 PM, Andrew Purtell  wrote:

> On Thu, May 23, 2013 at 5:10 PM, James Taylor wrote:
> 
>> Has there been any discussions on running the HBase server in an OSGi
>> container?
> 
> 
> I believe the only discussions have been on avoiding talk about coprocessor
> reloading, as it implies either a reimplementation of or taking on an OSGi
> runtime.
> 
> Is there a benefit to restarting a regionserver in an OSGi container versus
> restarting a Java process?
> 
> Or would that work otherwise like an update the coprocessor and filters in
> the container then trigger the embedded regionserver to do a quick close
> and reopen of the regions?
> 
> -- 
> Best regards,
> 
>   - Andy
> 
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)

Re: Any update for this issue HBASE-3529

[Michel Segel]

Boris Lublinsky and I published an article on something we did a few years ago. 
(It's on  InfoQ )

We did a small project of integrating Lucene and HBase for searching POI data.

It would probably do what you want. 

Sent from a remote device. Please excuse any typos...

Mike Segel

On May 30, 2013, at 9:35 PM, "dong.yajun"  wrote:

> hi Ted,
> 
> not yet right now about lily.
> 
> I would like to use hbase to store product reviews,  the system should
> support secondary index, full text search and faceting(Lucene) which
> support paging and sorting.
> 
> 
> On Fri, May 31, 2013 at 10:24 AM, Ted Yu  wrote:
> 
>> Jason is no longer working on this issue.
>> 
>> Can you tell us your use case ?
>> 
>> Have you looked at http://www.lilyproject.org/lily/index.html ?
>> 
>> Thanks
>> 
>> On Thu, May 30, 2013 at 7:06 PM, dong.yajun  wrote:
>> 
>>> Hello list,
>>> 
>>> Any one can give me some follow up information about this issue
>> HBASE-3529,
>>> I'm wondering it has more than 2 years no update.
>>> 
>>> Best,
>>> --
>>> *Rick Dong*
> 
> 
> 
> -- 
> *Ric Dong*

Re: EC2 Elastic MapReduce HBase install recommendations

[Michel Segel]

What I am saying is that by default, you get two mappers per node.
x4large can run HBase w more mapred slots, so you will want to tune the 
defaults based on machine size. Not just mapred, but also HBase stuff too. You 
need to do this on startup of EMR cluster though...

Sent from a remote device. Please excuse any typos...

Mike Segel

On May 9, 2013, at 2:39 AM, Pal Konyves  wrote:

> Principally I chose to use Amazon, because they are supposedly high
> performance, and what more important is: HBase is already set up if I chose
> it as an EMR Workflow. I wanted to save up the time setting up the cluster
> manually on EC2 instances.
> 
> Are you saying I will reach higher performance when I set up the HBase on
> the cluster manually, instead of the default Amazon HBase distribution? Or
> is it worth to tune the Amazon distribution with a bootstrap action? How
> long does it take, to set up the cluster with HDFS manually?
> 
> I will also try larger instance types.
> 
> 
> On Thu, May 9, 2013 at 6:47 AM, Michel Segel wrote:
> 
>> With respect to EMR, you can run HBase fairly easily.
>> You can't run MapR w HBase on EMR stick w Amazon's release.
>> 
>> And you can run it but you will want to know your tuning parameters up
>> front when you instantiate it.
>> 
>> 
>> 
>> Sent from a remote device. Please excuse any typos...
>> 
>> Mike Segel
>> 
>> On May 8, 2013, at 9:04 PM, Andrew Purtell  wrote:
>> 
>>> M7 is not Apache HBase, or any HBase. It is a proprietary NoSQL datastore
>>> with (I gather) an Apache HBase compatible Java API.
>>> 
>>> As for running HBase on EC2, we recently discussed some particulars, see
>>> the latter part of this thread: http://search-hadoop.com/m/rI1HpK90guwhere
>>> I hijack it. I wouldn't recommend launching HBase as part of an EMR flow
>>> unless you want to use it only for temporary random access storage, and
>> in
>>> which case use m2.2xlarge/m2.4xlarge instance types. Otherwise, set up a
>>> dedicated HBase backed storage service on high I/O instance types. The
>>> fundamental issue is IO performance on the EC2 platform is fair to poor.
>>> 
>>> I have also noticed a large difference in baseline block device latency
>> if
>>> using an old Amazon Linux AMI (< 2013) or the latest AMIs from this year.
>>> Use the new ones, they cut the latency long tail in half. There were some
>>> significant kernel level improvements I gather.
>>> 
>>> 
>>> On Wed, May 8, 2013 at 10:42 AM, Marcos Luis Ortiz Valmaseda <
>>> marcosluis2...@gmail.com> wrote:
>>> 
>>>> I think that you when you are talking about RMap, you are referring to
>>>> MapR´s distribution.
>>>> I think that MapR´s team released a very good version of its Hadoop
>>>> distribution focused on HBase called M7. You can see its overview here:
>>>> http://www.mapr.com/products/mapr-editions/m7-edition
>>>> 
>>>> But this release was under beta testing, and I see that it´s not
>> included
>>>> in the Amazon Marketplace yet:
>> https://aws.amazon.com/marketplace/seller-profile?id=802b0a25-877e-4b57-9007-a3fd284815a5
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 2013/5/7 Pal Konyves 
>>>> 
>>>>> Hi,
>>>>> 
>>>>> Has anyone got some recommendations about running HBase on EC2? I am
>>>>> testing it, and so far I am very disappointed with it. I did not change
>>>>> anything about the default 'Amazon distribution' installation. It has
>> one
>>>>> MasterNode and two slave nodes, and write performance is around 2500
>>>> small
>>>>> rows per sec at most, but I expected it to be way  better. Oh, and this
>>>> is
>>>>> with batch put operations with autocommit turned off, where each batch
>>>>> containes about 500-1000 rows... When I do it with autocommit, it does
>>>> not
>>>>> even reach the 1000 rows per sec.
>>>>> 
>>>>> Every nodes were m1.Large ones.
>>>>> 
>>>>> Any experiences, suggestions? Is it worth to try the RMap distribution
>>>>> instead of the amazon one?
>>>>> 
>>>>> Thanks,
>>>>> Pal
>>>> 
>>>> 
>>>> 
>>>> --
>>>> Marcos Ortiz Valmaseda
>>>> Product Manager at PDVSA
>>>> http://about.me/marcosortiz
>>> 
>>> 
>>> 
>>> --
>>> Best regards,
>>> 
>>>  - Andy
>>> 
>>> Problems worthy of attack prove their worth by hitting back. - Piet Hein
>>> (via Tom White)
>> 

Re: EC2 Elastic MapReduce HBase install recommendations

[Michel Segel]

With respect to EMR, you can run HBase fairly easily.
You can't run MapR w HBase on EMR stick w Amazon's release.

And you can run it but you will want to know your tuning parameters up front 
when you instantiate it.



Sent from a remote device. Please excuse any typos...

Mike Segel

On May 8, 2013, at 9:04 PM, Andrew Purtell  wrote:

> M7 is not Apache HBase, or any HBase. It is a proprietary NoSQL datastore
> with (I gather) an Apache HBase compatible Java API.
> 
> As for running HBase on EC2, we recently discussed some particulars, see
> the latter part of this thread: http://search-hadoop.com/m/rI1HpK90gu where
> I hijack it. I wouldn't recommend launching HBase as part of an EMR flow
> unless you want to use it only for temporary random access storage, and in
> which case use m2.2xlarge/m2.4xlarge instance types. Otherwise, set up a
> dedicated HBase backed storage service on high I/O instance types. The
> fundamental issue is IO performance on the EC2 platform is fair to poor.
> 
> I have also noticed a large difference in baseline block device latency if
> using an old Amazon Linux AMI (< 2013) or the latest AMIs from this year.
> Use the new ones, they cut the latency long tail in half. There were some
> significant kernel level improvements I gather.
> 
> 
> On Wed, May 8, 2013 at 10:42 AM, Marcos Luis Ortiz Valmaseda <
> marcosluis2...@gmail.com> wrote:
> 
>> I think that you when you are talking about RMap, you are referring to
>> MapR´s distribution.
>> I think that MapR´s team released a very good version of its Hadoop
>> distribution focused on HBase called M7. You can see its overview here:
>> http://www.mapr.com/products/mapr-editions/m7-edition
>> 
>> But this release was under beta testing, and I see that it´s not included
>> in the Amazon Marketplace yet:
>> 
>> https://aws.amazon.com/marketplace/seller-profile?id=802b0a25-877e-4b57-9007-a3fd284815a5
>> 
>> 
>> 
>> 
>> 2013/5/7 Pal Konyves 
>> 
>>> Hi,
>>> 
>>> Has anyone got some recommendations about running HBase on EC2? I am
>>> testing it, and so far I am very disappointed with it. I did not change
>>> anything about the default 'Amazon distribution' installation. It has one
>>> MasterNode and two slave nodes, and write performance is around 2500
>> small
>>> rows per sec at most, but I expected it to be way  better. Oh, and this
>> is
>>> with batch put operations with autocommit turned off, where each batch
>>> containes about 500-1000 rows... When I do it with autocommit, it does
>> not
>>> even reach the 1000 rows per sec.
>>> 
>>> Every nodes were m1.Large ones.
>>> 
>>> Any experiences, suggestions? Is it worth to try the RMap distribution
>>> instead of the amazon one?
>>> 
>>> Thanks,
>>> Pal
>> 
>> 
>> 
>> --
>> Marcos Ortiz Valmaseda
>> Product Manager at PDVSA
>> http://about.me/marcosortiz
> 
> 
> 
> -- 
> Best regards,
> 
>   - Andy
> 
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)

Re: Scala and Hbase, hbase-default.xml file seems to be for and old version of HBase (null)

[Michel Segel]

Isn't the defaults now embedded in the base jars?

Sent from a remote device. Please excuse any typos...

Mike Segel

On Apr 29, 2013, at 11:55 PM, Håvard Wahl Kongsgård 
 wrote:

> Nope.. the system is clean only CDH4 on it. And I can't find
> hbase-default.xml on the system.
> 
> However, I solved this issue my downloading
> 
> http://hbase_master:60010/conf, renaming it to hbase-default.xml and
> adding that to the classpath
> 
> So maybe a bug in CDH4.
> 
> On Mon, Apr 29, 2013 at 11:36 PM, Shahab Yunus  wrote:
>> This usually means that the hbase-default.xml could not be found on the
>> classpath (hbase-default is no longer included in conf folder anymore.) It
>> should be included in the classpath and it will have the correct version
>> which is populated when the jar is compiled/build. Have you verified that
>> there aren't any old hbase jars in your classpath (that did not have
>> hbase-default.xml packaged) that are being loaded first?
>> 
>> Regards,
>> Shahab
>> 
>> 
>> On Mon, Apr 29, 2013 at 1:04 PM, Håvard Wahl Kongsgård <
>> haavard.kongsga...@gmail.com> wrote:
>> 
>>> Hi, I get the following error with scala
>>> HBaseConfiguration.create()
>>> java.lang.RuntimeException: hbase-default.xml file seems to be for and
>>> old version of HBase (null), this version is 0.94.2-cdh4.2.1
>>> 
>>> the classpath is set path base on what I found with
>>> 
>>> ax | grep regionserver
>>> 
>>> -classpath
>>> /usr/lib/hbase/conf:/usr/lib/jvm/java-6-sun/lib/tools.jar:/usr/lib/hbase:/usr/lib/hbase/hbase-0.94.2-cdh4.2.1-security.jar:/usr/lib/hbase/hbase-0.94.2-cdh4.2.1-security-tests.jar:/usr/lib/hbase/hbase.jar:/usr/lib/hbase/lib/activation-1.1.jar:/usr/lib/hbase/lib/aopalliance-1.0.jar:/usr/lib/hbase/lib/asm-3.2.jar:/usr/lib/hbase/lib/avro-1.7.3.jar:/usr/lib/hbase/lib/commons-beanutils-1.7.0.jar:/usr/lib/hbase/lib/commons-beanutils-core-1.8.0.jar:/usr/lib/hbase/lib/commons-cli-1.2.jar:/usr/lib/hbase/lib/commons-codec-1.4.jar:/usr/lib/hbase/lib/commons-collections-3.2.1.jar:/usr/lib/hbase/lib/commons-configuration-1.6.jar:/usr/lib/hbase/lib/commons-daemon-1.0.3.jar:/usr/lib/hbase/lib/commons-digester-1.8.jar:/usr/lib/hbase/lib/commons-el-1.0.jar:/usr/lib/hbase/lib/commons-httpclient-3.1.jar:/usr/lib/hbase/lib/commons-io-2.1.jar:/usr/lib/hbase/lib/commons-lang-2.5.jar:/usr/lib/hbase/lib/commons-logging-1.1.1.jar:/usr/lib/hbase/lib/commons-net-3.1.jar:/usr/lib/hbase/lib/core-3.1.1.jar:/usr/lib/hbase/lib/gmbal-api-only-3.0.0-b023.jar:/usr/lib/hbase/lib/grizzly-framework-2.1.1.jar:/usr/lib/hbase/lib/grizzly-framework-2.1.1-tests.jar:/usr/lib/hbase/lib/grizzly-http-2.1.1.jar:/usr/lib/hbase/lib/grizzly-http-server-2.1.1.jar:/usr/lib/hbase/lib/grizzly-http-servlet-2.1.1.jar:/usr/lib/hbase/lib/grizzly-rcm-2.1.1.jar:/usr/lib/hbase/lib/guava-11.0.2.jar:/usr/lib/hbase/lib/guice-3.0.jar:/usr/lib/hbase/lib/guice-servlet-3.0.jar:/usr/lib/hbase/lib/high-scale-lib-1.1.1.jar:/usr/lib/hbase/lib/httpclient-4.1.3.jar:/usr/lib/hbase/lib/httpcore-4.1.3.jar:/usr/lib/hbase/lib/jackson-core-asl-1.8.8.jar:/usr/lib/hbase/lib/jackson-jaxrs-1.8.8.jar:/usr/lib/hbase/lib/jackson-mapper-asl-1.8.8.jar:/usr/lib/hbase/lib/jackson-xc-1.8.8.jar:/usr/lib/hbase/lib/jamon-runtime-2.3.1.jar:/usr/lib/hbase/lib/jasper-compiler-5.5.23.jar:/usr/lib/hbase/lib/jasper-runtime-5.5.23.jar:/usr/lib/hbase/lib/javax.inject-1.jar:/usr/lib/hbase/lib/javax.servlet-3.0.jar:/usr/lib/hbase/lib/jaxb-api-2.1.jar:/usr/lib/hbase/lib/jaxb-impl-2.2.3-1.jar:/usr/lib/hbase/lib/jersey-client-1.8.jar:/usr/lib/hbase/lib/jersey-core-1.8.jar:/usr/lib/hbase/lib/jersey-grizzly2-1.8.jar:/usr/lib/hbase/lib/jersey-guice-1.8.jar:/usr/lib/hbase/lib/jersey-json-1.8.jar:/usr/lib/hbase/lib/jersey-server-1.8.jar:/usr/lib/hbase/lib/jersey-test-framework-core-1.8.jar:/usr/lib/hbase/lib/jersey-test-framework-grizzly2-1.8.jar:/usr/lib/hbase/lib/jets3t-0.6.1.jar:/usr/lib/hbase/lib/jettison-1.1.jar:/usr/lib/hbase/lib/jetty-6.1.26.cloudera.2.jar:/usr/lib/hbase/lib/jetty-util-6.1.26.cloudera.2.jar:/usr/lib/hbase/lib/jruby-complete-1.6.5.jar:/usr/lib/hbase/lib/jsch-0.1.42.jar:/usr/lib/hbase/lib/jsp-2.1-6.1.14.jar:/usr/lib/hbase/lib/jsp-api-2.1-6.1.14.jar:/usr/lib/hbase/lib/jsp-api-2.1.jar:/usr/lib/hbase/lib/jsr305-1.3.9.jar:/usr/lib/hbase/lib/junit-4.10-HBASE-1.jar:/usr/lib/hbase/lib/kfs-0.3.jar:/usr/lib/hbase/lib/libthrift-0.9.0.jar:/usr/lib/hbase/lib/log4j-1.2.17.jar:/usr/lib/hbase/lib/management-api-3.0.0-b012.jar:/usr/lib/hbase/lib/metrics-core-2.1.2.jar:/usr/lib/hbase/lib/netty-3.2.4.Final.jar:/usr/lib/hbase/lib/paranamer-2.3.jar:/usr/lib/hbase/lib/protobuf-java-2.4.0a.jar:/usr/lib/hbase/lib/servlet-api-2.5-6.1.14.jar:/usr/lib/hbase/lib/servlet-api-2.5.jar:/usr/lib/hbase/lib/slf4j-api-1.6.1.jar:/usr/lib/hbase/lib/snappy-java-1.0.4.1.jar:/usr/lib/hbase/lib/stax-api-1.0.1.jar:/usr/lib/hbase/lib/xmlenc-0.52.jar:/usr/lib/hbase/lib/zookeeper.jar:/etc/hadoop/conf:/usr/lib/hadoop/lib/*:/usr/lib/hadoop/.//*:/usr/lib/hbase/hbase.jar:/usr/lib/hbase/lib/zookeeper.jar:/usr/lib/hbase/conf:/usr/lib/hadoop-hdf

Re: Schema Design Question

[Michel Segel]

I would have to agree. 
The use case doesn't make much sense for HBase and sounds a bit more like a 
problem for Hive.

The OP indicated that the data was disposable after a round of processing. 
IMHO Hive is a better fit.


Sent from a remote device. Please excuse any typos...

Mike Segel

On Apr 29, 2013, at 12:46 AM, Asaf Mesika  wrote:

> I actually don't see the benefit of saving the data into HBase if all you
> do is read per job id and purges it. Why not accumulate into HDFS per job
> id and then dump the file? The way I see it, HBase is good for querying
> parts of your data, even if it is only 10 rows. In your case your average
> is 1 billion, so streaming it from hdfs seems faster .
> 
> On Saturday, April 27, 2013, Enis Söztutar wrote:
> 
>> Hi,
>> 
>> Interesting use case. I think it depends on job many jobId's you expect to
>> have. If it is on the order of thousands, I would caution against going the
>> one table per jobid approach, since for every table, there is some master
>> overhead, as well as file structures in hdfs. If jobId's are managable,
>> going with separate tables makes sense if you want to efficiently delete
>> all the data related to a job.
>> 
>> Also pre-splitting will depend on expected number of jobIds / batchIds and
>> their ranges vs desired number of regions. You would want to keep number of
>> regions hosted by a single region server in the low tens, thus, your splits
>> can be across jobs or within jobs depending on cardinality. Can you share
>> some more?
>> 
>> Enis
>> 
>> 
>> On Fri, Apr 26, 2013 at 2:34 PM, Ted Yu >
>> wrote:
>> 
>>> My understanding of your use case is that data for different jobIds would
>>> be continuously loaded into the underlying table(s).
>>> 
>>> Looks like you can have one table per job. This way you drop the table
>>> after map reduce is complete. In the single table approach, you would
>>> delete many rows in the table which is not as fast as dropping the
>> separate
>>> table.
>>> 
>>> Cheers
>>> 
>>> On Sat, Apr 27, 2013 at 3:49 AM, Cameron Gandevia 
>>> 
 wrote:
>>> 
 Hi
 
 I am new to HBase, I have been trying to POC an application and have a
 design questions.
 
 Currently we have a single table with the following key design
 
 jobId_batchId_bundleId_uniquefileId
 
 This is an offline processing system so data would be bulk loaded into
 HBase via map/reduce jobs. We only need to support report generation
 queries using map/reduce over a batch (And possibly a single column
>>> filter)
 with the batchId as the start/end scan key. Once we have finished
 processing a job we are free to remove the data from HBase.
 
 We have varied workloads so a job could be made up of 10 rows, 100,000
>>> rows
 or 1 billion rows with the average falling somewhere around 10 million
 rows.
 
 My question is related to pre-splitting. If we have a billion rows all
>>> with
 the same batchId (Our map/reduce scan key) my understanding is we
>> should
 perform pre-splitting to create buckets hosted by different regions.
>> If a
 jobs workload can be so varied would it make sense to have a single
>> table
 containing all jobs? Or should we create 1 table per job and pre-split
>>> the
 table for the given workload? If we had separate table we could drop
>> them
 when no longer needed.
 
 If we didn't have a separate table per job how should we perform
>>> splitting?
 Should we choose our largest possible workload and split for that? even
 though 90% of our jobs would fall in the lower bound in terms of row
>>> count.
 Would we experience any issue purging jobs of varying sizes if
>> everything
 was in a single table?
 
 any advice would be greatly appreciated.
 
 Thanks
>> 

Re: How practical is it to add a timestamp oracle on Zookeeper

[Michel Segel]

Time is relative.
What does the timestamp mean? 

Sounds like a simple question, but its not. Is it the time your application 
says they wrote to HBase? Is it the time HBase first gets the row? Or is it the 
time that the row was written to the memstore? 

Each RS has its own clock in addition to your app server.


Sent from a remote device. Please excuse any typos...

Mike Segel

On Apr 16, 2013, at 7:14 AM, yun peng  wrote:

> Hi, All,
> I'd like to add a global timestamp oracle on Zookeep to assign globally
> unique timestamp for each Put/Get issued from HBase cluster. The reason I
> put it on Zookeeper is that each Put/Get needs to go through it and unique
> timestamp needs some global centralised facility to do it. But I am asking
> how practical is this scheme, like anyone used in practice?
> 
> Also, how difficulty is it to extend Zookeeper, or to inject code to the
> code path of HBase inside Zookeeper. I know HBase has Coprocessor on region
> server to let programmer to extend without recompiling HBase itself. Does
> Zk allow such extensibility? Thanks.
> 
> Regards
> Yun

Re: HBase random read performance

[Michel Segel]

Wouldn't do that... Changing block size is the last thing you want to do.

First question...

What is your key?

Second...
What is your record size that you are attempting to read.


Third...
Compare the 10k multiget versus 10k individual gets.

Fourth are your random keys sorted?
If not, try sorting them...

There are a lot of issues that can affect performance 



Sent from a remote device. Please excuse any typos...

Mike Segel

On Apr 15, 2013, at 3:17 AM, Anoop Sam John  wrote:

> Ankit
> I guess you might be having default HFile block size which is 
> 64KB.
> For random gets a lower value will be better. Try will some thing like 8KB 
> and check the latency?
> 
> Ya ofcourse blooms can help (if major compaction was not done at the time of 
> testing)
> 
> -Anoop-
> 
> From: Ankit Jain [ankitjainc...@gmail.com]
> Sent: Saturday, April 13, 2013 11:01 AM
> To: user@hbase.apache.org
> Subject: HBase random read performance
> 
> Hi All,
> 
> We are using HBase 0.94.5 and Hadoop 1.0.4.
> 
> We have HBase cluster of 5 nodes(5 regionservers and 1 master node). Each
> regionserver has 8 GB RAM.
> 
> We have loaded 25 millions records in HBase table, regions are pre-split
> into 16 regions and all the regions are equally loaded.
> 
> We are getting very low random read performance while performing multi get
> from HBase.
> 
> We are passing random 1 row-keys as input, while HBase is taking around
> 17 secs to return 1 records.
> 
> Please suggest some tuning to increase HBase read performance.
> 
> Thanks,
> Ankit Jain
> iLabs
> 
> 
> 
> --
> Thanks,
> Ankit Jain

Re: HBase Types: Explicit Null Support

[Michel Segel]

Silly question...
Null support. In a system where a column may or may not exist, how do you 
support null?

;-)

In terms of a key,  it's a primary key and can't be null.  


So what am I missing?


Sent from a remote device. Please excuse any typos...

Mike Segel

On Apr 1, 2013, at 10:26 PM, Nick Dimiduk  wrote:

> Furthermore, is is more important to support null values than squeeze all
> representations into minimum size (4-bytes for int32, &c.)?
> On Apr 1, 2013 4:41 PM, "Nick Dimiduk"  wrote:
> 
>> On Mon, Apr 1, 2013 at 4:31 PM, James Taylor wrote:
>> 
>>> From the SQL perspective, handling null is important.
>> 
>> 
>> From your perspective, it is critical to support NULLs, even at the
>> expense of fixed-width encodings at all or supporting representation of a
>> full range of values. That is, you'd rather be able to represent NULL than
>> -2^31?
>> 
>> On 04/01/2013 01:32 PM, Nick Dimiduk wrote:
>>> 
 Thanks for the thoughtful response (and code!).
 
 I'm thinking I will press forward with a base implementation that does
 not
 support nulls. The idea is to provide an extensible set of interfaces,
 so I
 think this will not box us into a corner later. That is, a mirroring
 package could be implemented that supports null values and accepts
 the relevant trade-offs.
 
 Thanks,
 Nick
 
 On Mon, Apr 1, 2013 at 12:26 PM, Matt Corgan 
 wrote:
 
 I spent some time this weekend extracting bits of our serialization
> code to
> a public github repo at 
> http://github.com/hotpads/**data-tools
> .
>  Contributions are welcome - i'm sure we all have this stuff laying
> around.
> 
> You can see I've bumped into the NULL problem in a few places:
> *
> 
> https://github.com/hotpads/**data-tools/blob/master/src/**
> main/java/com/hotpads/data/**primitive/lists/LongArrayList.**java
> *
> 
> https://github.com/hotpads/**data-tools/blob/master/src/**
> main/java/com/hotpads/data/**types/floats/DoubleByteTool.**java
> 
> Looking back, I think my latest opinion on the topic is to reject
> nullability as the rule since it can cause unexpected behavior and
> confusion.  It's cleaner to provide a wrapper class (so both
> LongArrayList
> plus NullableLongArrayList) that explicitly defines the behavior, and
> costs
> a little more in performance.  If the user can't find a pre-made wrapper
> class, it's not very difficult for each user to provide their own
> interpretation of null and check for it themselves.
> 
> If you reject nullability, the question becomes what to do in situations
> where you're implementing existing interfaces that accept nullable
> params.
>  The LongArrayList above implements List which requires an
> add(Long)
> method.  In the above implementation I chose to swap nulls with
> Long.MIN_VALUE, however I'm now thinking it best to force the user to
> make
> that swap and then throw IllegalArgumentException if they pass null.
> 
> 
> On Mon, Apr 1, 2013 at 11:41 AM, Doug Meil <
> doug.m...@explorysmedical.com
> 
>> wrote:
>> HmmmŠ good question.
>> 
>> I think that fixed width support is important for a great many rowkey
>> constructs cases, so I'd rather see something like losing MIN_VALUE and
>> keeping fixed width.
>> 
>> 
>> 
>> 
>> On 4/1/13 2:00 PM, "Nick Dimiduk"  wrote:
>> 
>> Heya,
>>> 
>>> Thinking about data types and serialization. I think null support is
>>> an
>>> important characteristic for the serialized representations,
>>> especially
>>> when considering the compound type. However, doing so in directly
>>> incompatible with fixed-width representations for numerics. For
>> instance,
> 
>> if we want to have a fixed-width signed long stored on 8-bytes, where
>>> do
>>> you put null? float and double types can cheat a little by folding
>>> negative
>>> and positive NaN's into a single representation (this isn't strictly
>>> correct!), leaving a place to represent null. In the long example
>>> case,
>>> the
>>> obvious choice is to reduce MAX_VALUE or increase MIN_VALUE by one.
>>> This
>>> will allocate an additional encoding which can be used for null. My
>>> experience working with scientific data, however, makes me wince at
>>> the
>>> idea.
>>> 
>>> The variable-width encodings have it a little easier. There's already
>>> enough going on that it's simpler to make room.
>>> 
>>> Remember, the final goal is to support order-preservi

Re: hbase time skew

[Michel Segel]

The actual time isn't an issue. It's that all of the nodes in the cluster have 
the same time...
Give or take a couple of ms.



Sent from a remote device. Please excuse any typos...

Mike Segel

On Mar 18, 2013, at 2:39 PM,  wrote:

> The problem is that in our case, the customer configures the NTP server and 
> it could be invalid. We're trying to cover user error cases, but on the other 
> hand we're trying to understand how big time skew hbase can handle...
> 
> Thanks,
> 
> YuLing
> 
> -Original Message-
> From: Kevin O'dell [mailto:kevin.od...@cloudera.com] 
> Sent: Saturday, March 16, 2013 5:49 AM
> To: user@hbase.apache.org
> Subject: Re: hbase time skew
> 
> I am going to agree with Michael on this one.  Don't change the clock skew, 
> fix it.
> 
> On Fri, Mar 15, 2013 at 9:56 PM, Michel Segel  
> wrote:
>> Create an ntp server local to the cluster? This will eliminate the skew in 
>> the first place.
>> 
>> Sent from a remote device. Please excuse any typos...
>> 
>> Mike Segel
>> 
>> On Mar 15, 2013, at 3:53 PM, Ted Yu  wrote:
>> 
>>> Are you using 0.94.x ?
>>> 
>>> If so, see the following:
>>> 
>>>   maxSkew = c.getLong("hbase.master.maxclockskew", 3);
>>>   warningSkew = c.getLong("hbase.master.warningclockskew", 1); 
>>> ./src/main/java/org/apache/hadoop/hbase/master/ServerManager.java
>>> 
>>> Cheers
>>> 
>>> On Fri, Mar 15, 2013 at 3:49 PM,  wrote:
>>> 
>>>> Hi,
>>>> 
>>>> We recently encountered the issue that HBase tables got into a state 
>>>> that was not disabled nor enabled. We found that the root cause was 
>>>> the linux clock skewed more than 5 hours. I googled and understood 
>>>> that hbase can only  handle about a couple of seconds time skew. We 
>>>> were wondering if there's any configuration in HBase that we can do 
>>>> so as to increase the number of seconds that hbase could handle?
>>>> 
>>>> Thanks very much,
>>>> 
>>>> YuLing
> 
> 
> 
> --
> Kevin O'Dell
> Customer Operations Engineer, Cloudera
> 

Re: HBase type support

[Michel Segel]

Andrew, 

I was aware of you employer, which I am pretty sure that they have already 
dealt with the issue of  exporting encryption software and probably hardware 
too.

Neither of us are lawyers and what I do know of dealing with the government 
bureaucracies, it's not always as simple of just filing the correct paperwork. 
(Sometimes it is, sometimes not so much, YMMV...)

Putting the hooks for encryption is probably a good idea. Shipping the 
encryption w the release or making it part of the official release, not so 
much. Sorry, I'm being a bit conservative here. 

IMHO I think fixing other issues would be of a higher priority, but that's just 
me;-)

Sent from a remote device. Please excuse any typos...

Mike Segel

On Mar 17, 2013, at 12:12 PM, Andrew Purtell  wrote:

>> This then leads to another question... suppose Apache does add encryption
> to Hadoop. While the Apache organization does have the proper paperwork in
> place, what then happens to Cloudera, Hortonworks, EMC, IBM, Intel, etc ?
> 
> Well I can't put that question aside since you've brought it up now
> twice and encryption feature candidates for Apache Hadoop and Apache HBase
> are something I have been working on. Its a valid question but since as you
> admit you don't know what you are talking about, perhaps stating uninformed
> opinions can be avoided. Only the latter is what I object to. I think the
> short answer is as an Apache contributor I'm concerned about the Apache
> product. Downstream repackagers can take whatever action needed including
> changes, since it is open source, or feedback about it representing a
> hardship. At this point I have heard nothing like that. I work for Intel
> and can say we are good with it.
> 
> On Sunday, March 17, 2013, Michael Segel wrote:
> 
>> Its not a question of FUD, but that certain types of encryption/decryption
>> code falls under the munitions act.
>> See: http://www.fas.org/irp/offdocs/eo_crypt_9611_memo.htm
>> 
>> Having said that, there is this:
>> http://www.bis.doc.gov/encryption/encfaqs6_17_02.html
>> 
>> In short, I don't as a habit export/import encryption technology so I am
>> not up to speed on the current state of the laws.
>> Which is why I have to question the current state of the US encryption
>> laws.
>> 
>> This then leads to another question... suppose Apache does add encryption
>> to Hadoop. While the Apache organization does have the proper paperwork in
>> place, what then happens to Cloudera, Hortonworks, EMC, IBM, Intel, etc ?
>> 
>> But lets put that question aside.
>> 
>> The point I was trying to make was that the core Sun JVM does support MD5
>> and SHA-1 out of the box, so that anyone running Hadoop and using the
>> 1.6_xx or the 1.7_xx versions of the JVM will have these packages.
>> 
>> Adding hooks that use these classes are a no brainer.  However, beyond
>> this... you tell me.
>> 
>> -Mike
>> 
>> On Mar 16, 2013, at 7:59 AM, Andrew Purtell  wrote:
>> 
>>> The ASF avails itself of an exception to crypto export which only
>> requires
>>> a bit of PMC housekeeping at release time. So "is not [ok]" is FUD. I
>>> humbly request we refrain from FUD here. See
>>> http://www.apache.org/dev/crypto.html. To the best of our knowledge we
>>> expect this to continue, though the ASF has not updated this policy yet
>> for
>>> recent regulation updates.
>>> 
>>> On Saturday, March 16, 2013, Michel Segel wrote:
>>> 
>>>> I also want to add that you could add MD5 and SHA-1, but I'd check on us
>>>> laws... I think these are ok, however other encryption/decryption code
>> is
>>>> not.
>>>> 
>>>> They are part of the std sun java libraries ...
>>>> 
>>>> Sent from a remote device. Please excuse any typos...
>>>> 
>>>> Mike Segel
>>>> 
>>>> On Mar 16, 2013, at 7:18 AM, Michel Segel 
>>>> wrote:
>>>> 
>>>>> Isn't that what you get through add on frameworks like TSDB and Kiji ?
>>>> Maybe not on the client side, but frameworks that extend HBase...
>>>>> 
>>>>> Sent from a remote device. Please excuse any typos...
>>>>> 
>>>>> Mike Segel
>>>>> 
>>>>> On Mar 16, 2013, at 12:45 AM, lars hofhansl  wrote:
>>>>> 
>>>>>> I think generally we should keep HBase a byte[] based key value store.
>>>>>> What we should add to HBase are tools that would allow client side
>> apps
&

Re: HBase type support

[Michel Segel]

If we look at TSDB, Kiji, Asynch HBase, it looks like extensions to HBase 
already exist.

I haven't looked at Salesforce,com's SQL interface, but I suspect that they too 
have some sort of framework where they have to enforce typing.


Sent from a remote device. Please excuse any typos...

Mike Segel

On Mar 17, 2013, at 10:01 PM, ramkrishna vasudevan 
 wrote:

> HBase shipping a generic framework for different interfaces is needed for
> ease of use for the users.  +1 on the idea.
> Getting out the correct result for float values, positive and negative
> integers had to be taken care by the users or by using some wrappers.
> 
> This will help to solve that problem to a great extent.
> 
> Regards
> Ram
> 
> On Sun, Mar 17, 2013 at 10:54 PM, Mohamed Ibrahim 
> wrote:
> 
>> I'm not a lawyer, but I think we're ok as long as it's in source code as
>> that is protected under freedom of speech in the US. See here (
>> http://en.wikipedia.org/wiki/Cryptography ) under Export Control, the part
>> related to Bernstein v. United States . I don't know about binaries like
>> deb, but I can tell that we download binaries for browsers every day and
>> they use encryption in lots of places. I believe if there's any real issues
>> it would have surfaced up by now.
>> 
>> As far as types in HBase, I think it is an excellent idea. I would suggest
>> to enable us to add a custom type, just like we can add our custom filters.
>> Some types that I had to code myself include CSV. There can be other custom
>> types that I need in the future, may be json, so the ability to add a
>> custom type might be a good feature.
>> 
>> Thanks,
>> Mohamed
>> 
>> 
>> On Sun, Mar 17, 2013 at 1:12 PM, Andrew Purtell 
>> wrote:
>> 
>>>> This then leads to another question... suppose Apache does add
>> encryption
>>> to Hadoop. While the Apache organization does have the proper paperwork
>> in
>>> place, what then happens to Cloudera, Hortonworks, EMC, IBM, Intel, etc ?
>>> 
>>> Well I can't put that question aside since you've brought it up now
>>> twice and encryption feature candidates for Apache Hadoop and Apache
>> HBase
>>> are something I have been working on. Its a valid question but since as
>> you
>>> admit you don't know what you are talking about, perhaps stating
>> uninformed
>>> opinions can be avoided. Only the latter is what I object to. I think the
>>> short answer is as an Apache contributor I'm concerned about the Apache
>>> product. Downstream repackagers can take whatever action needed including
>>> changes, since it is open source, or feedback about it representing a
>>> hardship. At this point I have heard nothing like that. I work for Intel
>>> and can say we are good with it.
>>> 
>>> On Sunday, March 17, 2013, Michael Segel wrote:
>>> 
>>>> Its not a question of FUD, but that certain types of
>>> encryption/decryption
>>>> code falls under the munitions act.
>>>> See: http://www.fas.org/irp/offdocs/eo_crypt_9611_memo.htm
>>>> 
>>>> Having said that, there is this:
>>>> http://www.bis.doc.gov/encryption/encfaqs6_17_02.html
>>>> 
>>>> In short, I don't as a habit export/import encryption technology so I
>> am
>>>> not up to speed on the current state of the laws.
>>>> Which is why I have to question the current state of the US encryption
>>>> laws.
>>>> 
>>>> This then leads to another question... suppose Apache does add
>> encryption
>>>> to Hadoop. While the Apache organization does have the proper paperwork
>>> in
>>>> place, what then happens to Cloudera, Hortonworks, EMC, IBM, Intel,
>> etc ?
>>>> 
>>>> But lets put that question aside.
>>>> 
>>>> The point I was trying to make was that the core Sun JVM does support
>> MD5
>>>> and SHA-1 out of the box, so that anyone running Hadoop and using the
>>>> 1.6_xx or the 1.7_xx versions of the JVM will have these packages.
>>>> 
>>>> Adding hooks that use these classes are a no brainer.  However, beyond
>>>> this... you tell me.
>>>> 
>>>> -Mike
>>>> 
>>>> On Mar 16, 2013, at 7:59 AM, Andrew Purtell 
>> wrote:
>>>> 
>>>>> The ASF avails itself of an exception to crypto export which only
>>>> requires
>>>>> a bit of PMC housekeeping at rel

Re: HBase type support

[Michel Segel]

I also want to add that you could add MD5 and SHA-1, but I'd check on us 
laws... I think these are ok, however other encryption/decryption code is not.

They are part of the std sun java libraries ...

Sent from a remote device. Please excuse any typos...

Mike Segel

On Mar 16, 2013, at 7:18 AM, Michel Segel  wrote:

> Isn't that what you get through add on frameworks like TSDB and Kiji ? Maybe 
> not on the client side, but frameworks that extend HBase...
> 
> Sent from a remote device. Please excuse any typos...
> 
> Mike Segel
> 
> On Mar 16, 2013, at 12:45 AM, lars hofhansl  wrote:
> 
>> I think generally we should keep HBase a byte[] based key value store.
>> What we should add to HBase are tools that would allow client side apps (or 
>> libraries) to built functionality on top of plain HBase.
>> 
>> Serialization that maintains a correct semantic sort order is important as a 
>> building block, so is code that can build up correctly serialized and 
>> sortable compound keys, as well as hashing algorithms.
>> 
>> Where I would draw the line is adding types to HBase itself. As long as one 
>> can write a client, or Filters, or Coprocessors with the tools provided by 
>> HBase we're good. Higher level functionality can then be built of on top of 
>> HBase.
>> 
>> 
>> For example, maybe we need to add better access API to the HBase WAL in 
>> order to have an external library implement idempotent transactions (which 
>> can be used to implement 2ndary indexes).
>> Maybe some other primitives have to be exposed in order to allow an external 
>> library to implement full transactions.
>> Or we might need a statistics framework (such as the one that Jesse is 
>> working on).
>> 
>> These are all building blocks that do not presume specific access patterns 
>> or clients, but can be used to implement them.
>> 
>> 
>> As usual, just my $0.02.
>> 
>> -- Lars
>> 
>> 
>> 
>> 
>> From: Nick Dimiduk 
>> To: user@hbase.apache.org 
>> Sent: Friday, March 15, 2013 10:57 AM
>> Subject: Re: HBase type support
>> 
>> I'm talking about MD5, SHA1, etc. It's something explicitly mentioned
>> in HBASE-7221.
>> 
>> On Fri, Mar 15, 2013 at 10:55 AM, James Taylor wrote:
>> 
>>> Hi Nick,
>>> What do you mean by "hashing algorithms"?
>>> Thanks,
>>> James
>>> 
>>> 
>>> On 03/15/2013 10:11 AM, Nick Dimiduk wrote:
>>> 
>>>> Hi David,
>>>> 
>>>> Native support for a handful of hashing algorithms has also been
>>>> discussed.
>>>> Do you think these should be supported directly, as opposed to using a
>>>> fixed-length String or fixed-length byte[]?
>>>> 
>>>> Thanks,
>>>> Nick
>>>> 
>>>> On Thu, Mar 14, 2013 at 9:51 AM, David Koch 
>>>> wrote:
>>>> 
>>>>  Hi Nick,
>>>>> 
>>>>> As an HBase user I would welcome this addition. In addition to the
>>>>> proposed
>>>>> list of datatypes A UUID/GUID type would also be nice to have.
>>>>> 
>>>>> Regards,
>>>>> 
>>>>> /David
>>>>> 
>>>>> 
>>>>> On Wed, Mar 13, 2013 at 5:42 PM, Nick Dimiduk 
>>>>> wrote:
>>>>> 
>>>>>  Hi all,
>>>>>> 
>>>>>> I'd like to draw your attention to HBASE-8089. The desire is to add type
>>>>>> support to HBase. There are two primary objectives: make the lives of
>>>>>> developers building on HBase easier, and facilitate better tools on top
>>>>>> 
>>>>> of
>>>>> 
>>>>>> HBase. Please chime in with any feature suggestions you think we've
>>>>>> 
>>>>> missed
>>>>> 
>>>>>> in initial conversations.
>>>>>> 
>>>>>> Thanks,
>>>>>> -n
>>>>>> 
>>>>>> [0]: 
>>>>>> https://issues.apache.org/**jira/browse/HBASE-8089<https://issues.apache.org/jira/browse/HBASE-8089>
>>>>>> 
>>>>>> 
> 

Re: HBase type support

[Michel Segel]

Isn't that what you get through add on frameworks like TSDB and Kiji ? Maybe 
not on the client side, but frameworks that extend HBase...

Sent from a remote device. Please excuse any typos...

Mike Segel

On Mar 16, 2013, at 12:45 AM, lars hofhansl  wrote:

> I think generally we should keep HBase a byte[] based key value store.
> What we should add to HBase are tools that would allow client side apps (or 
> libraries) to built functionality on top of plain HBase.
> 
> Serialization that maintains a correct semantic sort order is important as a 
> building block, so is code that can build up correctly serialized and 
> sortable compound keys, as well as hashing algorithms.
> 
> Where I would draw the line is adding types to HBase itself. As long as one 
> can write a client, or Filters, or Coprocessors with the tools provided by 
> HBase we're good. Higher level functionality can then be built of on top of 
> HBase.
> 
> 
> For example, maybe we need to add better access API to the HBase WAL in order 
> to have an external library implement idempotent transactions (which can be 
> used to implement 2ndary indexes).
> Maybe some other primitives have to be exposed in order to allow an external 
> library to implement full transactions.
> Or we might need a statistics framework (such as the one that Jesse is 
> working on).
> 
> These are all building blocks that do not presume specific access patterns or 
> clients, but can be used to implement them.
> 
> 
> As usual, just my $0.02.
> 
> -- Lars
> 
> 
> 
> 
> From: Nick Dimiduk 
> To: user@hbase.apache.org 
> Sent: Friday, March 15, 2013 10:57 AM
> Subject: Re: HBase type support
> 
> I'm talking about MD5, SHA1, etc. It's something explicitly mentioned
> in HBASE-7221.
> 
> On Fri, Mar 15, 2013 at 10:55 AM, James Taylor wrote:
> 
>> Hi Nick,
>> What do you mean by "hashing algorithms"?
>> Thanks,
>> James
>> 
>> 
>> On 03/15/2013 10:11 AM, Nick Dimiduk wrote:
>> 
>>> Hi David,
>>> 
>>> Native support for a handful of hashing algorithms has also been
>>> discussed.
>>> Do you think these should be supported directly, as opposed to using a
>>> fixed-length String or fixed-length byte[]?
>>> 
>>> Thanks,
>>> Nick
>>> 
>>> On Thu, Mar 14, 2013 at 9:51 AM, David Koch 
>>> wrote:
>>> 
>>>   Hi Nick,
 
 As an HBase user I would welcome this addition. In addition to the
 proposed
 list of datatypes A UUID/GUID type would also be nice to have.
 
 Regards,
 
 /David
 
 
 On Wed, Mar 13, 2013 at 5:42 PM, Nick Dimiduk 
 wrote:
 
   Hi all,
> 
> I'd like to draw your attention to HBASE-8089. The desire is to add type
> support to HBase. There are two primary objectives: make the lives of
> developers building on HBase easier, and facilitate better tools on top
> 
 of
 
> HBase. Please chime in with any feature suggestions you think we've
> 
 missed
 
> in initial conversations.
> 
> Thanks,
> -n
> 
> [0]: 
> https://issues.apache.org/**jira/browse/HBASE-8089
> 
> 

Re: hbase time skew

[Michel Segel]

Create an ntp server local to the cluster? This will eliminate the skew in the 
first place.

Sent from a remote device. Please excuse any typos...

Mike Segel

On Mar 15, 2013, at 3:53 PM, Ted Yu  wrote:

> Are you using 0.94.x ?
> 
> If so, see the following:
> 
>maxSkew = c.getLong("hbase.master.maxclockskew", 3);
>warningSkew = c.getLong("hbase.master.warningclockskew", 1);
> ./src/main/java/org/apache/hadoop/hbase/master/ServerManager.java
> 
> Cheers
> 
> On Fri, Mar 15, 2013 at 3:49 PM,  wrote:
> 
>> Hi,
>> 
>> We recently encountered the issue that HBase tables got into a state that
>> was not disabled nor enabled. We found that the root cause was the linux
>> clock skewed more than 5 hours. I googled and understood that hbase can
>> only  handle about a couple of seconds time skew. We were wondering if
>> there's any configuration in HBase that we can do so as to increase the
>> number of seconds that hbase could handle?
>> 
>> Thanks very much,
>> 
>> YuLing
>> 

Re: HBase type support

[Michel Segel]

Ok..

So how do you check types in a column when the column isn't defined in the 
Schema?


Sent from a remote device. Please excuse any typos...

Mike Segel

On Mar 15, 2013, at 10:06 AM, Nick Dimiduk  wrote:

> On Fri, Mar 15, 2013 at 5:25 AM, Michel Segel 
> wrote:
> 
>> You do realize that having to worry about one type is easier...
> 
> For HBase developers, that's true. The side-effect is that those worries
> are pushed out into users' applications. Think of the application developer
> who's accustomed to all the accoutrements provided by the Management System
> part of an RDBMS. They pick up HBase and have none of that. I think the
> motivations outlined in the attached document make a good case for bringing
> some of that burden out of users' applications.
> 
> A bit more freedom...
> 
> Support for raw byte[] doesn't go away. In this proposal, bytes remain the
> core plumbing of the system.
> 
> -n
> 
> On Mar 14, 2013, at 11:51 AM, David Koch  wrote:
>> 
>>> Hi Nick,
>>> 
>>> As an HBase user I would welcome this addition. In addition to the
>> proposed
>>> list of datatypes A UUID/GUID type would also be nice to have.
>>> 
>>> Regards,
>>> 
>>> /David
>>> 
>>> 
>>> On Wed, Mar 13, 2013 at 5:42 PM, Nick Dimiduk 
>> wrote:
>>> 
>>>> Hi all,
>>>> 
>>>> I'd like to draw your attention to HBASE-8089. The desire is to add type
>>>> support to HBase. There are two primary objectives: make the lives of
>>>> developers building on HBase easier, and facilitate better tools on top
>> of
>>>> HBase. Please chime in with any feature suggestions you think we've
>> missed
>>>> in initial conversations.
>>>> 
>>>> Thanks,
>>>> -n
>>>> 
>>>> [0]: https://issues.apache.org/jira/browse/HBASE-8089
>> 

Re: HBase type support

[Michel Segel]

You do realize that having to worry about one type is easier...

A bit more freedom...


Sent from a remote device. Please excuse any typos...

Mike Segel

On Mar 14, 2013, at 11:51 AM, David Koch  wrote:

> Hi Nick,
> 
> As an HBase user I would welcome this addition. In addition to the proposed
> list of datatypes A UUID/GUID type would also be nice to have.
> 
> Regards,
> 
> /David
> 
> 
> On Wed, Mar 13, 2013 at 5:42 PM, Nick Dimiduk  wrote:
> 
>> Hi all,
>> 
>> I'd like to draw your attention to HBASE-8089. The desire is to add type
>> support to HBase. There are two primary objectives: make the lives of
>> developers building on HBase easier, and facilitate better tools on top of
>> HBase. Please chime in with any feature suggestions you think we've missed
>> in initial conversations.
>> 
>> Thanks,
>> -n
>> 
>> [0]: https://issues.apache.org/jira/browse/HBASE-8089
>> 

Re: Eliminating duplicate values

[Michel Segel]

There are no duplicates.
Cells have versions, which are time stamped. You could set the number of 
versions to one... But I'd recommend sticking w the default 3.

Sent from a remote device. Please excuse any typos...

Mike Segel

On Mar 2, 2013, at 9:42 PM, Matt Corgan  wrote:

> I have a few use cases where I'd like to leverage HBase's high write
> throughput to blindly write lots of data even if most of it hasn't changed
> since the last write.  I want to retain MAX_VERSIONS=Integer.MAX_VALUE,
> however, I don't want to keep all the duplicate copies around forever.  At
> compaction time, I'd like the compactor to compare the values of cells with
> the same row/family/qualifier and only keep the *oldest* version of
> duplicates.  By keeping the oldest versions I can get a snapshot of a row
> at any historical time.
> 
> Lars, I think you said Salesforce retains many versions of cells - do you
> retain all the duplicates?
> 
> I'm guessing co-processors would be the solution and am looking for some
> pointers on the cleanest way to implement it or some code if anyone has
> already solved the problem.
> 
> I'm also wondering if people think it's a generic enough use case that
> HBase could support it natively, say, with a column family attribute
> DISCARD_NEWEST_DUPLICATE=true/false.  The cost would be higher CPU usage at
> compaction time because of all the value comparisons.
> 
> Thanks for any tips,
> Matt

Re: Hbase table with a nested entity

[Michel Segel]

Uhm you can easily do this in HBase a couple of different ways. :-)
Your option 2 is this solution he wants.
Or option 3, store the data into a map or some serializable object, (Single 
column)
Option 4, use Avro single column
Option 5, if data set is large enough, store it in a sequence file and use 
HBase as index.

You just need to free yourself from thinking only in terms of ER modeling.

Sent from a remote device. Please excuse any typos...

Mike Segel

On Feb 27, 2013, at 5:06 AM, yonghu  wrote:

> Hello Dastgiri,
> 
> I don't think HBase can support original nested schema which you want
> to define. But you can still store your data in HBase. I figured out
> several possible solutions:
> 
> 1. row_key: profileid + profilename + date, the column will be
> monthwiseProfileCount:uk and so on. However, this approach will cause
> data redundancy (profileid + profilename will repeatably appear), and
> the data which belongs to the same user are separated into different
> tuples.
> 
> 2. row_key: profileid + profilename. the column will be
> monthwiseProfileCount: date(e.g. 12/10/2010)/uk and so on. The benefit
> of this approach is that all the data belongs to the same user group
> together. However, as date is one part of column information. It will
> cause to create many columns if the value range of date is wide.
> 
> regards!
> 
> Yong
> 
> On Wed, Feb 27, 2013 at 5:38 AM, Dastagiri S Shaik
>  wrote:
>> Hi All,
>> 
>> I need to define a schema
>> 
>> profileid  (integer)
>> profilename (String)
>> monthwiseProfileCount (is having )
>>12/10/2010-->
>>uk:200
>>us:300
>>india:500
>> 
>>12/11/2010-->
>>uk:200
>>us:300
>>india:500
>> 
>> 
>> please help me.
>> 
>> Regards
>> Dastgiri
> 

Re: coprocessor enabled put very slow, help please~~~

[Michel Segel]

What happens when you have a poem like Mary had a little lamb?

Did you turn off the WAL on both table inserts, or just the index?

If you want to avoid processing duplicate docs... You could do this a couple of 
ways. The simplest way is to record the doc ID and a check sum for the doc. If 
the doc you are processing matches... You can simply do NOOP for the lines in 
the doc. (This isn't the fastest, but its easy.)
The other is to run a preprocess which removes duplicate doc from your 
directory and you then process the docs...

Third thing... Do a code review. Sloppy code will kill performance...

Sent from a remote device. Please excuse any typos...

Mike Segel

On Feb 20, 2013, at 5:26 AM, Prakash Kadel  wrote:

> michael, 
>   infact i dont care about latency bw doc write and index write.
> today i did some tests.
> turns out turning off WAL does speed up the writes by about a factor of 2.
> interestingly, enabling bloom filter did little to improve the checkandput.
> 
> earlier you mentioned
>>>>> The OP doesn't really get in to the use case, so we don't know why the
>>>> Check and Put in the M/R job.
>>>>> He should just be using put() and then a postPut().
> 
> 
> the main reason i use checkandput is to make sure the word count index doesnt 
> get duplicate increments when duplicate documents come in. additionally i 
> also need to dump dup free docs to hdfs for legacy system that we have in 
> place.
> is there some way to avoid chechandput?
> 
> 
> Sincerely,
> Prakash 
> 
> On Feb 20, 2013, at 10:00 PM, Michel Segel  wrote:
> 
>> I was suggesting removing the write to WAL on your write to the index table 
>> only.
>> 
>> The thing you have to realize that true low latency systems use databases as 
>> a sink. It's the end of the line so to speak.
>> 
>> So if you're worried about a small latency between the writing to your doc 
>> table, and then the write of your index.. You are designing the wrong system.
>> 
>> Consider that it takes some time t to write the base record and then to 
>> write the indexes.
>> For that period, you have a Schrödinger's cat problem as to if the row 
>> exists or not. Since HBase lacks transactions and ACID, trying to write a 
>> solution where you require the low latency... You are using the wrong tool.
> 

Re: coprocessor enabled put very slow, help please~~~

[Michel Segel]

I was suggesting removing the write to WAL on your write to the index table 
only.

The thing you have to realize that true low latency systems use databases as a 
sink. It's the end of the line so to speak.

So if you're worried about a small latency between the writing to your doc 
table, and then the write of your index.. You are designing the wrong system.

Consider that it takes some time t to write the base record and then to write 
the indexes.
For that period, you have a Schrödinger's cat problem as to if the row exists 
or not. Since HBase lacks transactions and ACID, trying to write a solution 
where you require the low latency... You are using the wrong tool.

Remember that HBase was designed as a distributed system for managing very 
large data sets. Your speed from using secondary indexes like an inverted table 
is in the read and not the write.

If you had append working, you could create an index if you could create a 
fixed sized key buffer. Or something down that path... Sorry, just thinking 
something out loud...

Sent from a remote device. Please excuse any typos...

Mike Segel

On Feb 19, 2013, at 1:53 PM, Asaf Mesika  wrote:

> 1. Try batching your increment calls to a List and use batch() to
> execute it. Should reduce RPC calls by 2 magnitudes.
> 2. Combine batching with scanning more words, thus aggregating your count
> for a certain word thus less Increment commands.
> 3. Enable Bloom Filters. Should speed up Increment by a factor of 2 at
> least.
> 4. Don't use keyValue.getValue(). It does a System.arraycopy behind the
> scenes. Use getBuffer() and getValueOffset() and getValueLength() and
> iterate on the existing array. Write your own Split without going into
> using String functions which goes through encoding (expensive). Just find
> your delimiter by byte comparison.
> 5. Enable BloomFilters on doc table. It should speed up the checkAndPut.
> 6. I wouldn't give up WAL. It ain't your bottleneck IMO.
> 
> On Monday, February 18, 2013, prakash kadel wrote:
> 
>> Thank you guys for your replies,
>> Michael,
>>   I think i didnt make it clear. Here is my use case,
>> 
>> I have text documents to insert in the hbase. (With possible duplicates)
>> Suppose i have a document as : " I am working. He is not working"
>> 
>> I want to insert this document to a table in hbase, say table "doc"
>> 
>> =doc table=
>> -
>> rowKey : doc_id
>> cf: doc_content
>> value: "I am working. He is not working"
>> 
>> Now, i to create another table that stores the word count, say "doc_idx"
>> 
>> doc_idx table
>> ---
>> rowKey : I, cf: count, value: 1
>> rowKey : am, cf: count, value: 1
>> rowKey : working, cf: count, value: 2
>> rowKey : He, cf: count, value: 1
>> rowKey : is, cf: count, value: 1
>> rowKey : not, cf: count, value: 1
>> 
>> My MR job code:
>> ==
>> 
>> if(doc.checkAndPut(rowKey, doc_content, "", null, putDoc)) {
>>for(String word : doc_content.split("\\s+")) {
>>   Increment inc = new Increment(Bytes.toBytes(word));
>>   inc.addColumn("count", "", 1);
>>}
>> }
>> 
>> Now, i wanted to do some experiments with coprocessors. So, i modified
>> the code as follows.
>> 
>> My MR job code:
>> ===
>> 
>> doc.checkAndPut(rowKey, doc_content, "", null, putDoc);
>> 
>> Coprocessor code:
>> ===
>> 
>>public void start(CoprocessorEnvironment env)  {
>>pool = new HTablePool(conf, 100);
>>}
>> 
>>public boolean postCheckAndPut(c,  row,  family, byte[] qualifier,
>> compareOp,   comparator,  put, result) {
>> 
>>if(!result) return true; // check if the put succeeded
>> 
>>HTableInterface table_idx = pool.getTable("doc_idx");
>> 
>>try {
>> 
>>for(KeyValue contentKV = put.get("doc_content",
>> "")) {
>>for(String word :
>> contentKV.getValue().split("\\s+")) {
>>Increment inc = new
>> Increment(Bytes.toBytes(word));
>>inc.addColumn("count", "", 1);
>>table_idx.increment(inc);
>>}
>>   }
>>} finally {
>>table_idx.close();
>>}
>>return true;
>>}
>> 
>>public void stop(env) {
>>pool.close();
>>}
>> 
>> I am a newbee to HBASE. I am not sure this is the way to do.
>> Given that, why is the cooprocessor enabled version much slower than
>> the one without?
>> 
>> 
>> Sincerely,
>> Prakash Kadel
>> 
>> 
>> On Mon, Feb 18, 2013 at 9:11 PM, Michael Segel
>> > wrote:
>>> 
>>> The  issue I was talking about was the use of a check and put.
>>> The OP wrote:
>>> each map inserts to doc table.(checkAndPut)
>>> regionobserver coprocessor does a postCheckAndPut and inserts some
>> rows to
>>> a index table.
>>> 
>>> My question is why does the OP use a checkAndP

Re: coprocessor enabled put very slow, help please~~~

[Michel Segel]

Why are you using an HTable Pool?
Why are you closing the table after each iteration through?

Try using 1 HTable object. Turn off WAL
Initiate in start()
Close in Stop()
Surround the use in a try / catch
If exception caught, re instantiate new HTable connection.

Maybe want to flush the connection after puts. 


Again not sure why you are using check and put on the base table. Your count 
could be off.

As an example look at poem/rhyme 'Marry had a little lamb'.
Then check your word count.

Sent from a remote device. Please excuse any typos...

Mike Segel

On Feb 18, 2013, at 7:21 AM, prakash kadel  wrote:

> Thank you guys for your replies,
> Michael,
>   I think i didnt make it clear. Here is my use case,
> 
> I have text documents to insert in the hbase. (With possible duplicates)
> Suppose i have a document as : " I am working. He is not working"
> 
> I want to insert this document to a table in hbase, say table "doc"
> 
> =doc table=
> -
> rowKey : doc_id
> cf: doc_content
> value: "I am working. He is not working"
> 
> Now, i to create another table that stores the word count, say "doc_idx"
> 
> doc_idx table
> ---
> rowKey : I, cf: count, value: 1
> rowKey : am, cf: count, value: 1
> rowKey : working, cf: count, value: 2
> rowKey : He, cf: count, value: 1
> rowKey : is, cf: count, value: 1
> rowKey : not, cf: count, value: 1
> 
> My MR job code:
> ==
> 
> if(doc.checkAndPut(rowKey, doc_content, "", null, putDoc)) {
>for(String word : doc_content.split("\\s+")) {
>   Increment inc = new Increment(Bytes.toBytes(word));
>   inc.addColumn("count", "", 1);
>}
> }
> 
> Now, i wanted to do some experiments with coprocessors. So, i modified
> the code as follows.
> 
> My MR job code:
> ===
> 
> doc.checkAndPut(rowKey, doc_content, "", null, putDoc);
> 
> Coprocessor code:
> ===
> 
>public void start(CoprocessorEnvironment env)  {
>pool = new HTablePool(conf, 100);
>}
> 
>public boolean postCheckAndPut(c,  row,  family, byte[] qualifier,
> compareOp, comparator,  put, result) {
> 
>if(!result) return true; // check if the put succeeded
> 
>HTableInterface table_idx = pool.getTable("doc_idx");
> 
>try {
> 
>for(KeyValue contentKV = put.get("doc_content", "")) {
>for(String word :
> contentKV.getValue().split("\\s+")) {
>Increment inc = new
> Increment(Bytes.toBytes(word));
>inc.addColumn("count", "", 1);
>table_idx.increment(inc);
>}
>   }
>} finally {
>table_idx.close();
>}
>return true;
>}
> 
>public void stop(env) {
>pool.close();
>}
> 
> I am a newbee to HBASE. I am not sure this is the way to do.
> Given that, why is the cooprocessor enabled version much slower than
> the one without?
> 
> 
> Sincerely,
> Prakash Kadel
> 
> 
> On Mon, Feb 18, 2013 at 9:11 PM, Michael Segel
>  wrote:
>> 
>> The  issue I was talking about was the use of a check and put.
>> The OP wrote:
>> each map inserts to doc table.(checkAndPut)
>> regionobserver coprocessor does a postCheckAndPut and inserts some rows 
>> to
>> a index table.
>> 
>> My question is why does the OP use a checkAndPut, and the RegionObserver's 
>> postChecAndPut?
>> 
>> 
>> Here's a good example... 
>> http://stackoverflow.com/questions/13404447/is-hbase-checkandput-latency-higher-than-simple-put
>> 
>> The OP doesn't really get in to the use case, so we don't know why the Check 
>> and Put in the M/R job.
>> He should just be using put() and then a postPut().
>> 
>> Another issue... since he's writing to  a different HTable... how? Does he 
>> create an HTable instance in the start() method of his RO object and then 
>> reference it later? Or does he create the instance of the HTable on the fly 
>> in each postCheckAndPut() ?
>> Without seeing his code, we don't know.
>> 
>> Note that this is synchronous set of writes. Your overall return from the 
>> M/R call to put will wait until the second row is inserted.
>> 
>> Interestingly enough, you may want to consider disabling the WAL on the 
>> write to the index.  You can always run a M/R job that rebuilds the index 
>> should something occur to the system where you might lose the data.  Indexes 
>> *ARE* expendable. ;-)
>> 
>> Does that explain it?
>> 
>> -Mike
>> 
>> On Feb 18, 2013, at 4:57 AM, yonghu  wrote:
>> 
>>> Hi, Michael
>>> 
>>> I don't quite understand what do you mean by "round trip back to the
>>> client". In my understanding, as the RegionServer and TaskTracker can
>>> be the same node, MR don't have to pull data into client and then
>>> process.  And you also mention the "unnecessary overhead", can you
>>> explain a little bit what operations or data processing can be seen as
>>> "unnecessary overhead".
>>> 
>>> Thanks
>>> 
>>>

Re: hbase-master-server slept

[Michel Segel]

First, 
In the world of Hadoop, if it ain't broke don't fix it, may not be the best 
advice.
HBase is still evolving at a good pace and you want to be closer to the latest 
releases.
CDH4 is stable so that I would agree that going to CDH4 would be best.

Second.
You are running this as a single machine within the VM.

What does the hardware look like?
Number of cores, physical or virtual ?
How much memory?

What type of disks and where are they?
( attached or SAN?)




Sent from a remote device. Please excuse any typos...

Mike Segel

On Feb 14, 2013, at 1:16 AM, So Hibino  wrote:

> Hi,
> 
> The VM was rebooted last night because of the host server memory error.
> So the dmesg was cleared at the time of the isuue.
> 
> The below dmesg is new and the issue has not been happened yet.
> 
> Thank you,
> So
> 
> On 2013/02/14 15:10, So Hibino wrote:
>> Hi,
>> 
>> I pasted dmseg and syslog on pastebin.com.
>> 
>> URL is below.
>> 
>> dmseg
>> http://pastebin.com/t3i3KRJ8
>> 
>> syslog
>> http://pastebin.com/LcBRYMyX
>> 
>> Any messages wasn't output for the time spanning the isuue.
>> 
>> thank you,
>> So
>> 
>> 
>> On 2013/02/13 9:31, Jean-Daniel Cryans [via Apache HBase] wrote:
>>> 
>>> 
>>> messages and syslog please, just for the time spanning the issue.
>>> 
>>> J-D
>>> 
>>> On Tue, Feb 12, 2013 at 4:10 PM, So Hibino  wrote:
 Hi,
 
> Can you post the dmesg and syslog spanning the time of the issue?
 Is the syslog you mean /var/log/message ?
 
 
 On 2013/02/13 4:00, Jean-Daniel Cryans [via Apache HBase] wrote:
> On Mon, Feb 11, 2013 at 8:06 PM, So Hibino <[hidden email] 
> > wrote:
> 
>> master log
> 
> Unfortunately there's not much in there apart from the weird sleep
> times. At this point I'd consider OS-level issues.
> 
>> We doesn't use EC2,but this server works with KVM.
> 
> Can you post the dmesg and syslog spanning the time of the issue?
> Please use pastebin.com as it can be really hard to read the logs in
> emails.
> 
> Thx,
> 
> J-D
> 
> 
> !
> -!
>>  --!
   ---
> If you reply to this email, your message will be added to the discussion 
> below:
> http://apache-hbase.679495.n3.nabble.com/hbase-master-server-slept-tp4038192p4038442.html
> To unsubscribe from hbase-master-server slept, click here 
> .
> NAML 
> 
 
 
 --
 H24.07より組織名が変わりました。
 
 NTTサービスエボリューション研究所
 日比野　壮　（So Hibino）
 
 〒239-0847 横須賀市光の丘1-1 (320C)
 TEL : 046-859-4582  FAX : 046-855-1282
 e-mail : hibino...@lab.ntt.co.jp
 
 
 
 
 
 --
 View this message in context: 
 http://apache-hbase.679495.n3.nabble.com/hbase-master-server-slept-tp4038192p4038459.html
 Sent from the HBase User mailing list archive at Nabble.com.
>>> 
>>> 
>>> 
>>> 
>>> ___
>>> If you reply to this email, your message will be added to the discussion 
>>> below:
>>> http://apache-hbase.679495.n3.nabble.com/hbase-master-server-slept-tp4038192p4038461.html
>>> 
>>> To unsubscribe from hbase-master-server slept, visit 
>>> http://apache-hbase.679495.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=4038192&code=aGliaW5vLnNvQGxhYi5udHQuY28uanB8NDAzODE5MnwtNjAxOTg5MzI0
> 
> 
> -- 
> H24.07より組織名が変わりました。
> 
> NTTサービスエボリューション研究所
> 日比野　壮　（So Hibino）
> 
> 〒239-0847 横須賀市光の丘1-1 (320C)
> TEL : 046-85

Re: restrict clients

[Michel Segel]

Not a simple answer.

You have to treat security like an onion.

You can restrict access to the cluster by limiting access to the cluster's 
subnet.
That's just the start.
You can then use IPTables to filter packets from hosts you don't want to have 
access.

Define what you want to accomplish and work from the network up.


Sent from a remote device. Please excuse any typos...

Mike Segel

On Feb 11, 2013, at 7:07 AM, Rita  wrote:

> Hi,
> 
> I am looking for more than an ACL. I want to control what clients can
> connect to the hbase cluster. Is that possible?
> 
> 
> On Fri, Feb 8, 2013 at 10:36 AM, Stas Maksimov  wrote:
> 
>> Hi Rita,
>> 
>> As far as I know ACL is on a user basis. Here's a link for you:
>> http://hbase.apache.org/book/hbase.accesscontrol.configuration.html
>> 
>> Thanks,
>> Stas
>> 
>> 
>> On 8 February 2013 15:20, Rita  wrote:
>> 
>>> Hi,
>>> 
>>> In an enterprise deployment, how can I restrict who can access the data?
>>> For example, I want only certain servers able to GET,PUT data everyone
>> else
>>> should be denied. Is this possible?
>>> 
>>> 
>>> 
>>> --
>>> --- Get your facts first, then you can distort them as you please.--
> 
> 
> 
> -- 
> --- Get your facts first, then you can distort them as you please.--

Re: how to model data based on "time bucket"

[Michel Segel]

Tough one in that if your events are keyed on time alone, you will hit a hot 
spot on write. Reads,not so much...

TSDB would be a good start ...

You may not need 'buckets' but just a time stamp  and set up a start and stop 
key values.

Sent from a remote device. Please excuse any typos...

Mike Segel

On Jan 28, 2013, at 7:06 AM, Oleg Ruchovets  wrote:

> Hi ,
> 
> I have such row data structure:
> 
> event_id | time
> =
> event1 | 10:07
> event2 | 10:10
> event3 | 10:12
> 
> event4 | 10:20
> event5 | 10:23
> event6 | 10:25
> 
> 
> Numbers of records is 50-100 million.
> 
> 
> Question:
> 
> I need to find group of events starting form eventX and enters to the time
> window bucket = T.
> 
> 
> For example: if T=7 munutes.
> Starting from event event1- {event1, event2 , event3} were detected durint
> 7 minutes.
> 
> Starting from event event2- {event2 , event3} were detected durint 7
> minutes.
> 
> Starting from event event4 - {event4, event5 , event6} were detected during
> 7 minutes.
> Is there a way to model the data in hbase to get?
> 
> Thanks

Re: Coprocessor / threading model

[Michel Segel]

There are a couple of different designs that you can use to perform the write 
to the secondary index.

I wouldn't call this an anti-pattern... (AP's comment)

Using htablepool wouldn't be my first choice, unless you are writing to a 
durable queue first which then uses the pool to write to the table.  This could 
work as part of a more general solution to handle indexing at a more general 
level. But that is a longer discussion.




Sent from a remote device. Please excuse any typos...

Mike Segel

On Jan 12, 2013, at 6:06 PM, Adrien Mogenet  wrote:

> Hi there,
> 
> I'm experiencing some issues with CP. I'm trying to implement an indexing
> solution (inspired by Annop's slides). In pre-put, I trigger another Put()
> in an external table (to build the secondary index). It works perfect for
> one client, but when I'm inserting data from 2 separate clients, I met
> issues with HTable object (the one used in pre-Put()), because it's not
> thread-safe. I decided to move on TablePool and that fixed my issue.
> 
> But if I increase the write-load (and concurrency) HBase is throwing a OOM
> exception because it can't create new native threads. Looking at HBase
> metrics "threads count", I see that roughly 3500 threads are created.
> 
> I'm looking for documentation about how CPs are working with threads :
> what/when should I protect against concurrency issues ? How may I solve my
> issue ?
> 
> Help is welcome :-)
> 
> -- 
> Adrien Mogenet
> 06.59.16.64.22
> http://www.mogenet.me

Re: HBase - Secondary Index

[Michel Segel]

Sorry, this makes no sense...

You are doing a range scan, I get that... 

Consider that in an inverted table as your index, each column would be your 
rowkey which will be in a sort order.

Extend get() to take in a range pair as parameters and limit the result set 
returned to those columns which fall within your range... 

Problem solved. Right?

The RPC and network traffic is kept to a minimum and you are still solving the 
underlying use case with cleaner code.

Just saying...


Sent from a remote device. Please excuse any typos...

Mike Segel

On Jan 8, 2013, at 6:30 PM, lars hofhansl  wrote:

> Different use cases.
> 
> 
> For global point queries you want exactly what you said below.
> For range scans across many rows you want Anoop's design. As usually it 
> depends.
> 
> 
> The tradeoff is bringing a lot of unnecessary data to the client vs having to 
> contact each region (or at least each region server).
> 
> 
> -- Lars
> 
> 
> 
> 
> From: Michael Segel 
> To: user@hbase.apache.org 
> Sent: Tuesday, January 8, 2013 6:33 AM
> Subject: Re: HBase - Secondary Index
> 
> So if you're using an inverted table / index why on earth are you doing it at 
> the region level? 
> 
> I've tried to explain this to others over 6 months ago and its not really a 
> good idea. 
> 
> You're over complicating this and you will end up creating performance 
> bottlenecks when your secondary index is completely orthogonal to your row 
> key. 
> 
> To give you an example... 
> 
> Suppose you're CCCIS and you have a large database of auto insurance claims 
> that you've acquired over the years from your Pathways product. 
> 
> Your primary key would be a combination of the Insurance Company's ID and 
> their internal claim ID for the individual claim. 
> Your row would be all of the data associated to that claim.
> 
> So now lets say you want to find the average cost to repair a front end 
> collision of an S80 Volvo. 
> The make and model of the car would be orthogonal to the initial key. This 
> means that the result set containing insurance records for Front End 
> collisions of S80 Volvos would be most likely evenly distributed across the 
> cluster's regions. 
> 
> If you used a series of inverted tables, you would be able to use a series of 
> get()s to get the result set from each index and then find their 
> intersections. (Note that you could also put them in sort order so that the 
> intersections would be fairly straight forward to find. 
> 
> Doing this at the region level isn't so simple. 
> 
> So I have to again ask why go through and over complicate things? 
> 
> Just saying... 
> 
> On Jan 7, 2013, at 7:49 AM, Anoop Sam John  wrote:
> 
>> Hi,
>> It is inverted index based on column(s) value(s)
>> It will be region wise indexing. Can work when some one knows the rowkey 
>> range or NOT.
>> 
>> -Anoop-
>> 
>> From: Mohit Anchlia [mohitanch...@gmail.com]
>> Sent: Monday, January 07, 2013 9:47 AM
>> To: user@hbase.apache.org
>> Subject: Re: HBase - Secondary Index
>> 
>> Hi Anoop,
>> 
>> Am I correct in understanding that this indexing mechanism is only
>> applicable when you know the row key? It's not an inverted index truly
>> based on the column value.
>> 
>> Mohit
>> On Sun, Jan 6, 2013 at 7:48 PM, Anoop Sam John  wrote:
>> 
>>> Hi Adrien
>>>  We are making the consistency btw the main table and
>>> index table and the roll back mentioned below etc using the CP hooks. The
>>> current hooks were not enough for those though..  I am in the process of
>>> trying to contribute those new hooks, core changes etc now...  Once all are
>>> done I will be able to explain in details..
>>> 
>>> -Anoop-
>>> 
>>> From: Adrien Mogenet [adrien.moge...@gmail.com]
>>> Sent: Monday, January 07, 2013 2:00 AM
>>> To: user@hbase.apache.org
>>> Subject: Re: HBase - Secondary Index
>>> 
>>> Nice topic, perhaps one of the most important for 2013 :-)
>>> I still don't get how you're ensuring consistency between index table and
>>> main table, without an external component (such as bookkeeper/zookeeper).
>>> What's the exact write path in your situation when inserting data ?
>>> (WAL/RegionObserver, pre/post put/WALedit...)
>>> 
>>> The underlying question is about how you're ensuring that WALEdit in Index
>>> and Main tables are perfectly sync'ed, and how you 're able to rollback in
>>> case of issue in both WAL ?
>>> 
>>> 
>>> On Fri, Dec 28, 2012 at 11:55 AM, Shengjie Min 
>>> wrote:
>>> 
> Yes as you say when the no of rows to be returned is becoming more and
 more the latency will be becoming more.  seeks within an HFile block is
 some what expensive op now. (Not much but still)  The new encoding
 prefix
 trie will be a huge bonus here. There the seeks will be flying.. [Ted
>>> also
 presented this in the Hadoop China]  Thanks to Matt... :)  I am trying to
 measure the scan 

Re: HBase - Secondary Index

[Michel Segel]

I suggest you think more about the problem... Whenever I hear someone talk 
about unnecessary RPC calls in a distributed system Well, I get skeptical. 
Especially in light of 10GBe.

Sloppy code is one thing. Being myopic is another. 

This is why I am asking for a more concrete use case. Lars makes a point that 
there is a wide variety of potential use cases. However trying to solve a bad 
use case with something that could have a negative impact on overall 
performance for other use cases... well, I'd like to avoid that if it were 
possible.

I think I can give you a couple of examples...

First if you had a use case where you really needed a distributed OLTP 
database, I'd say HBase wasn't the right tool and you should look at Informix's 
XPS engine,provided IBM still sells it... The point being 

Second, HBase sits on top of HDFS. Because of design issues in HDFS, like not 
having R/W access to files, there are limitations to HBase where we have to 
deal with things like compactions.  

I point this out because its a design constraint that impacts the solutions we 
design using HBase. 
In these two examples, we would want to question the use of HBase as part of 
the solution.  This is why I'm asking for a use case of where indexing at the 
region level makes sense. 

It sounds like the idea is to use the secondary index as a filter on a range 
scan. Using an inverted table, the columns are in sort order. So that it would 
be easier and lighter to fetch only the columns within the range of the query. 
Simple extension to using a secondary inverted table for your index.

Just saying...



Sent from a remote device. Please excuse any typos...

Mike Segel

On Jan 8, 2013, at 10:11 PM, ramkrishna vasudevan 
 wrote:

> As far as i can see its more related to using the coprocessor framework in
> this soln that helps us in a great way to avoid unnecessary RPC calls when
> we go with Region level indexing.
> 
> Regards
> Ram
> 
> On Wed, Jan 9, 2013 at 8:52 AM, Anoop Sam John  wrote:
> 
>> Totally agree with Lars.  The design came up as per our usage and data
>> distribution style etc.
>> Also the put performance we were not able to compromise. That is why the
>> region collocation based region based indexing design came :)
>> Also as we are having the indexing and index usage every thing happening
>> at server side, there is no need for any change in the client part
>> depending on what type of client u use. Java code or REST APIs or any
>> thing.  Also MR based parallel scans any thing can be comparably easy I
>> feel as there is absolutely no changes needed at client side.  :)
>> 
>> As Anil said there will be pros and cons for every way and which one suits
>> your usage, needs to be adopted. :)
>> 
>> -Anoop-
>> 
>> From: anil gupta [anilgupt...@gmail.com]
>> Sent: Wednesday, January 09, 2013 6:58 AM
>> To: user@hbase.apache.org; lars hofhansl
>> Subject: Re: HBase - Secondary Index
>> 
>> +1 on Lars comment.
>> 
>> Either the client gets the rowkey from secondary table and then gets the
>> real data from Primary Table. ** OR ** Send the request to all the RS(or
>> region) hosting a region of primary table.
>> 
>> Anoop is using the latter mechanism. Both the mechanism have their pros and
>> cons. IMO, there is no outright winner.
>> 
>> ~Anil Gupta
>> 
>> On Tue, Jan 8, 2013 at 4:30 PM, lars hofhansl  wrote:
>> 
>>> Different use cases.
>>> 
>>> 
>>> For global point queries you want exactly what you said below.
>>> For range scans across many rows you want Anoop's design. As usually it
>>> depends.
>>> 
>>> 
>>> The tradeoff is bringing a lot of unnecessary data to the client vs
>> having
>>> to contact each region (or at least each region server).
>>> 
>>> 
>>> -- Lars
>>> 
>>> 
>>> 
>>> 
>>> From: Michael Segel 
>>> To: user@hbase.apache.org
>>> Sent: Tuesday, January 8, 2013 6:33 AM
>>> Subject: Re: HBase - Secondary Index
>>> 
>>> So if you're using an inverted table / index why on earth are you doing
>> it
>>> at the region level?
>>> 
>>> I've tried to explain this to others over 6 months ago and its not really
>>> a good idea.
>>> 
>>> You're over complicating this and you will end up creating performance
>>> bottlenecks when your secondary index is completely orthogonal to your
>> row
>>> key.
>>> 
>>> To give you an example...
>>> 
>>> Suppose you're CCCIS and you have a large database of auto insurance
>>> claims that you've acquired over the years from your Pathways product.
>>> 
>>> Your primary key would be a combination of the Insurance Company's ID and
>>> their internal claim ID for the individual claim.
>>> Your row would be all of the data associated to that claim.
>>> 
>>> So now lets say you want to find the average cost to repair a front end
>>> collision of an S80 Volvo.
>>> The make and model of the car would be orthogonal to the initial key.
>> This
>>> means that the result set containing insurance records

Re: HBase - Secondary Index

[Michel Segel]

You haven't provided a use case...

You are defending a design without showing an example of how it its implemented.
Without a concrete example, it's impossible to determine if there is a design 
flaw in the initial design.

Sorry, but I don't think that enough thought has been done...
I'm trying to understand your reasoning, but without a concrete example... It's 
kind of hard.


Sent from a remote device. Please excuse any typos...

Mike Segel

On Jan 8, 2013, at 9:22 PM, Anoop Sam John  wrote:

> Totally agree with Lars.  The design came up as per our usage and data 
> distribution style etc.
> Also the put performance we were not able to compromise. That is why the 
> region collocation based region based indexing design came :)
> Also as we are having the indexing and index usage every thing happening at 
> server side, there is no need for any change in the client part depending on 
> what type of client u use. Java code or REST APIs or any thing.  Also MR 
> based parallel scans any thing can be comparably easy I feel as there is 
> absolutely no changes needed at client side.  :)
> 
> As Anil said there will be pros and cons for every way and which one suits 
> your usage, needs to be adopted. :)
> 
> -Anoop-
> 
> From: anil gupta [anilgupt...@gmail.com]
> Sent: Wednesday, January 09, 2013 6:58 AM
> To: user@hbase.apache.org; lars hofhansl
> Subject: Re: HBase - Secondary Index
> 
> +1 on Lars comment.
> 
> Either the client gets the rowkey from secondary table and then gets the
> real data from Primary Table. ** OR ** Send the request to all the RS(or
> region) hosting a region of primary table.
> 
> Anoop is using the latter mechanism. Both the mechanism have their pros and
> cons. IMO, there is no outright winner.
> 
> ~Anil Gupta
> 
> On Tue, Jan 8, 2013 at 4:30 PM, lars hofhansl  wrote:
> 
>> Different use cases.
>> 
>> 
>> For global point queries you want exactly what you said below.
>> For range scans across many rows you want Anoop's design. As usually it
>> depends.
> 

Re: HBase - Secondary Index

[Michel Segel]

Can you provide a use case?


Sent from a remote device. Please excuse any typos...

Mike Segel

On Jan 8, 2013, at 6:30 PM, lars hofhansl  wrote:

> Different use cases.
> 
> 
> For global point queries you want exactly what you said below.
> For range scans across many rows you want Anoop's design. As usually it 
> depends.
> 
> 
> The tradeoff is bringing a lot of unnecessary data to the client vs having to 
> contact each region (or at least each region server).
> 
> 
> -- Lars
> 
> 
> 
> 
> From: Michael Segel 
> To: user@hbase.apache.org 
> Sent: Tuesday, January 8, 2013 6:33 AM
> Subject: Re: HBase - Secondary Index
> 
> So if you're using an inverted table / index why on earth are you doing it at 
> the region level? 
> 
> I've tried to explain this to others over 6 months ago and its not really a 
> good idea. 
> 
> You're over complicating this and you will end up creating performance 
> bottlenecks when your secondary index is completely orthogonal to your row 
> key. 
> 
> To give you an example... 
> 
> Suppose you're CCCIS and you have a large database of auto insurance claims 
> that you've acquired over the years from your Pathways product. 
> 
> Your primary key would be a combination of the Insurance Company's ID and 
> their internal claim ID for the individual claim. 
> Your row would be all of the data associated to that claim.
> 
> So now lets say you want to find the average cost to repair a front end 
> collision of an S80 Volvo. 
> The make and model of the car would be orthogonal to the initial key. This 
> means that the result set containing insurance records for Front End 
> collisions of S80 Volvos would be most likely evenly distributed across the 
> cluster's regions. 
> 
> If you used a series of inverted tables, you would be able to use a series of 
> get()s to get the result set from each index and then find their 
> intersections. (Note that you could also put them in sort order so that the 
> intersections would be fairly straight forward to find. 
> 
> Doing this at the region level isn't so simple. 
> 
> So I have to again ask why go through and over complicate things? 
> 
> Just saying... 
> 
> On Jan 7, 2013, at 7:49 AM, Anoop Sam John  wrote:
> 
>> Hi,
>> It is inverted index based on column(s) value(s)
>> It will be region wise indexing. Can work when some one knows the rowkey 
>> range or NOT.
>> 
>> -Anoop-
>> 
>> From: Mohit Anchlia [mohitanch...@gmail.com]
>> Sent: Monday, January 07, 2013 9:47 AM
>> To: user@hbase.apache.org
>> Subject: Re: HBase - Secondary Index
>> 
>> Hi Anoop,
>> 
>> Am I correct in understanding that this indexing mechanism is only
>> applicable when you know the row key? It's not an inverted index truly
>> based on the column value.
>> 
>> Mohit
>> On Sun, Jan 6, 2013 at 7:48 PM, Anoop Sam John  wrote:
>> 
>>> Hi Adrien
>>>  We are making the consistency btw the main table and
>>> index table and the roll back mentioned below etc using the CP hooks. The
>>> current hooks were not enough for those though..  I am in the process of
>>> trying to contribute those new hooks, core changes etc now...  Once all are
>>> done I will be able to explain in details..
>>> 
>>> -Anoop-
>>> 
>>> From: Adrien Mogenet [adrien.moge...@gmail.com]
>>> Sent: Monday, January 07, 2013 2:00 AM
>>> To: user@hbase.apache.org
>>> Subject: Re: HBase - Secondary Index
>>> 
>>> Nice topic, perhaps one of the most important for 2013 :-)
>>> I still don't get how you're ensuring consistency between index table and
>>> main table, without an external component (such as bookkeeper/zookeeper).
>>> What's the exact write path in your situation when inserting data ?
>>> (WAL/RegionObserver, pre/post put/WALedit...)
>>> 
>>> The underlying question is about how you're ensuring that WALEdit in Index
>>> and Main tables are perfectly sync'ed, and how you 're able to rollback in
>>> case of issue in both WAL ?
>>> 
>>> 
>>> On Fri, Dec 28, 2012 at 11:55 AM, Shengjie Min 
>>> wrote:
>>> 
> Yes as you say when the no of rows to be returned is becoming more and
 more the latency will be becoming more.  seeks within an HFile block is
 some what expensive op now. (Not much but still)  The new encoding
 prefix
 trie will be a huge bonus here. There the seeks will be flying.. [Ted
>>> also
 presented this in the Hadoop China]  Thanks to Matt... :)  I am trying to
 measure the scan performance with this new encoding . Trying to >back
>>> port
 a simple patch for 94 version just for testing...   Yes when the no of
 results to be returned is more and more any index will become less
 performing as per my study  :)
 
 yes, you are right, I guess it's just a drawback of any index approach.
 Thanks for the explanation.
 
 Shengjie
 
 On 28 December 2012 04:14, Anoop Sam John  wrote:
 
>> Do 

Re: HBase - Secondary Index

[Michel Segel]

Can you provide a use case?


Sent from a remote device. Please excuse any typos...

Mike Segel

On Jan 8, 2013, at 6:30 PM, lars hofhansl  wrote:

> Different use cases.
> 
> 
> For global point queries you want exactly what you said below.
> For range scans across many rows you want Anoop's design. As usually it 
> depends.
> 
> 
> The tradeoff is bringing a lot of unnecessary data to the client vs having to 
> contact each region (or at least each region server).
> 
> 
> -- Lars
> 
> 
> 
> 
> From: Michael Segel 
> To: user@hbase.apache.org 
> Sent: Tuesday, January 8, 2013 6:33 AM
> Subject: Re: HBase - Secondary Index
> 
> So if you're using an inverted table / index why on earth are you doing it at 
> the region level? 
> 
> I've tried to explain this to others over 6 months ago and its not really a 
> good idea. 
> 
> You're over complicating this and you will end up creating performance 
> bottlenecks when your secondary index is completely orthogonal to your row 
> key. 
> 
> To give you an example... 
> 
> Suppose you're CCCIS and you have a large database of auto insurance claims 
> that you've acquired over the years from your Pathways product. 
> 
> Your primary key would be a combination of the Insurance Company's ID and 
> their internal claim ID for the individual claim. 
> Your row would be all of the data associated to that claim.
> 
> So now lets say you want to find the average cost to repair a front end 
> collision of an S80 Volvo. 
> The make and model of the car would be orthogonal to the initial key. This 
> means that the result set containing insurance records for Front End 
> collisions of S80 Volvos would be most likely evenly distributed across the 
> cluster's regions. 
> 
> If you used a series of inverted tables, you would be able to use a series of 
> get()s to get the result set from each index and then find their 
> intersections. (Note that you could also put them in sort order so that the 
> intersections would be fairly straight forward to find. 
> 
> Doing this at the region level isn't so simple. 
> 
> So I have to again ask why go through and over complicate things? 
> 
> Just saying... 
> 
> On Jan 7, 2013, at 7:49 AM, Anoop Sam John  wrote:
> 
>> Hi,
>> It is inverted index based on column(s) value(s)
>> It will be region wise indexing. Can work when some one knows the rowkey 
>> range or NOT.
>> 
>> -Anoop-
>> 
>> From: Mohit Anchlia [mohitanch...@gmail.com]
>> Sent: Monday, January 07, 2013 9:47 AM
>> To: user@hbase.apache.org
>> Subject: Re: HBase - Secondary Index
>> 
>> Hi Anoop,
>> 
>> Am I correct in understanding that this indexing mechanism is only
>> applicable when you know the row key? It's not an inverted index truly
>> based on the column value.
>> 
>> Mohit
>> On Sun, Jan 6, 2013 at 7:48 PM, Anoop Sam John  wrote:
>> 
>>> Hi Adrien
>>>  We are making the consistency btw the main table and
>>> index table and the roll back mentioned below etc using the CP hooks. The
>>> current hooks were not enough for those though..  I am in the process of
>>> trying to contribute those new hooks, core changes etc now...  Once all are
>>> done I will be able to explain in details..
>>> 
>>> -Anoop-
>>> 
>>> From: Adrien Mogenet [adrien.moge...@gmail.com]
>>> Sent: Monday, January 07, 2013 2:00 AM
>>> To: user@hbase.apache.org
>>> Subject: Re: HBase - Secondary Index
>>> 
>>> Nice topic, perhaps one of the most important for 2013 :-)
>>> I still don't get how you're ensuring consistency between index table and
>>> main table, without an external component (such as bookkeeper/zookeeper).
>>> What's the exact write path in your situation when inserting data ?
>>> (WAL/RegionObserver, pre/post put/WALedit...)
>>> 
>>> The underlying question is about how you're ensuring that WALEdit in Index
>>> and Main tables are perfectly sync'ed, and how you 're able to rollback in
>>> case of issue in both WAL ?
>>> 
>>> 
>>> On Fri, Dec 28, 2012 at 11:55 AM, Shengjie Min 
>>> wrote:
>>> 
> Yes as you say when the no of rows to be returned is becoming more and
 more the latency will be becoming more.  seeks within an HFile block is
 some what expensive op now. (Not much but still)  The new encoding
 prefix
 trie will be a huge bonus here. There the seeks will be flying.. [Ted
>>> also
 presented this in the Hadoop China]  Thanks to Matt... :)  I am trying to
 measure the scan performance with this new encoding . Trying to >back
>>> port
 a simple patch for 94 version just for testing...   Yes when the no of
 results to be returned is more and more any index will become less
 performing as per my study  :)
 
 yes, you are right, I guess it's just a drawback of any index approach.
 Thanks for the explanation.
 
 Shengjie
 
 On 28 December 2012 04:14, Anoop Sam John  wrote:
 
>> Do 

Re: Is it necessary to set MD5 on rowkey?

[Michel Segel]

This what wrote:
>> If you salt, you will have to do a *FULL* *TABLE* *SCAN* in order to
>> retrieve the row.
>> If you do something like a salt that uses only  a preset of N combinations,
>> you will have to do N get()s in order to fetch the row.
>> 

By definition the salt is a random number which is the first part of the one 
way crypt() function.
Using some modulo function is the second half of what I said. ;-)


Sent from a remote device. Please excuse any typos...

Mike Segel

On Dec 19, 2012, at 7:35 PM, Jean-Marc Spaggiari  
wrote:

> I have to disagree with the *FULL* *TABLE* *SCAN* in order to retrieve the 
> row.
> 
> If I know that I have one byte salting between 1 and 10, I will have
> to do 10 gets to get the row. And they will most probably all be on
> different RS, so it will not be more than 1 get per server. This will
> take almost the same time as doing a simple get.
> 
> I understand your point that salting is inducting some bad things, but
> on the other side, it's easy and can still be usefull. Hash will allow
> you a direct access with one call, but you still need to calculate the
> hash. So what's faster? Calculate the hash and do one call to one
> server? Or go directly with one call to multiple servers? It all
> depend on the way you access your data.
> 
> Personnaly, I'm using hash almost everwhere, but I still understand
> that some people might be able to use salting for their specific
> purposes.
> 
> JM
> 
> 2012/12/19, Michael Segel :
>> Ok,
>> 
>> Lets try this one more time...
>> 
>> If you salt, you will have to do a *FULL* *TABLE* *SCAN* in order to
>> retrieve the row.
>> If you do something like a salt that uses only  a preset of N combinations,
>> you will have to do N get()s in order to fetch the row.
>> 
>> This is bad. VERY BAD.
>> 
>> If you hash the row, you will get a consistent value each time you hash the
>> key.  If you use SHA-1, the odds of a collision are mathematically possible,
>> however highly improbable. So people have recommended that they append the
>> key to the hash to form the new key. Here, you might as well as truncate the
>> hash to just the most significant byte or two and the append the key. This
>> will give you enough of an even distribution that you can avoid hot
>> spotting.
> 

Re: HBase - Secondary Index

[Michel Segel]

Just a couple of questions...

First, since you don't have any natural secondary indices, you can create one 
from a couple of choices. Keeping it simple, you choose an inverted table as 
your index.

In doing so, you have one column containing all of the row ids for a given 
value.
This means that it is a simple get(). 

My question is that since you don't have any formal SQL syntax, how are you 
doing this all server side?


Sent from a remote device. Please excuse any typos...

Mike Segel

On Dec 18, 2012, at 2:28 AM, anil gupta  wrote:

> Hi Anoop,
> 
> Please find my reply inline.
> 
> Thanks,
> Anil Gupta
> 
> On Sun, Dec 16, 2012 at 8:02 PM, Anoop Sam John  wrote:
> 
>> Hi Anil
>>During the scan, there is no need to fetch any index data
>> to client side. So there is no need to create any scanner on the index
>> table at the client side. This happens at the server side.
> 
> 
>> 
>> For the Scan on the main table with condition on timestamp and customer
>> id, a scanner to be created with Filters. Yes like normal when there is no
>> secondary index. So this scan from the client will go through all the
>> regions in the main table.
> 
> 
> Anil: Do you mean that if the table is spread across 50 region servers in
> 60 node cluster then we need to send a scan request to all the 50 RS.
> Right? Doesn't it sounds expensive? IMHO you were not doing this in your
> solution. Your solution looked cleaner than this since you exactly knew
> which Node you need to go to for querying while using secondary index due
> to co-location(due to static begin part for secondary table rowkey) of
> region of primary table and secondary index table. My problem is little
> more complicated due to the constraints that: I cannot have a "static begin
> part" in the rowkey of my secondary table.
> 
> When it scans one particular region say (x,y] on the main table, using the
>> CP we can get the index table region object corresponding to this main
>> table region from the RS.  There is no issue in creating the static part of
>> the rowkey. You know 'x' is the region start key. Then at the server side
>> will create a scanner on the index region directly and here we can specify
>> the startkey. 'x' +  + ..  Using the results
>> from the index scan we will make reseek on the main region to the exact
>> rows where the data what we are interested in is available. So there wont
>> be a full region data scan happening.
> 
>> When in the cases where only timestamp is there but no customer id, it
>> will be simple again. Create a scanner on the main table with only one
>> filter. At the CP side the scanner on the index region will get created
>> with startkey as 'x' + ..When you create the scan
>> object and set startRow on that it need not be the full rowkey. It can be
>> part of the rowkey also. Yes like prefix.
>> 
>> Hope u got it now :)
> Anil: I hope now we are on same page. Thanks a lot for your valuable time
> to discuss this stuff.
> 
>> 
>> -Anoop-
>> 
>> From: anil gupta [anilgupt...@gmail.com]
>> Sent: Friday, December 14, 2012 11:31 PM
>> To: user@hbase.apache.org
>> Subject: Re: HBase - Secondary Index
>> 
>> On Fri, Dec 14, 2012 at 12:54 AM, Anoop Sam John 
>> wrote:
>> 
>>> Hi Anil,
>>> 
 1. In your presentation you mentioned that region of Primary Table and
>>> Region of Secondary Table are always located on the same region server.
>> How
>>> do you achieve it? By using the Primary table rowkey as prefix of  Rowkey
>>> of Secondary Table? Will your implementation work if the rowkey of
>> primary
>>> table cannot be used as prefix in rowkey of Secondary table( i have this
>>> limitation in my use case)?
>>> First all there will be same number of regions in both primary and index
>>> tables. All the start/stop keys of the regions also will be same.
>>> Suppose there are 2 regions on main table say for keys 0-10 and 10-20.
>>> Then we will create 2 regions in index table also with same key ranges.
>>> At the master balancing level it is easy to collocate these regions
>> seeing
>>> the start and end keys.
>>> When the selection of the rowkey that will be used in the index table is
>>> the key here.
>>> What we will do is all the rowkeys in the index table will be prefixed
>>> with the start key of the region/
>>> When an entry is added to the main table with rowkey as 5 it will go to
>>> the 1st region (0-10)
>>> Now there will be index region with range as 0-10.  We will select this
>>> region to store this index data.
>>> The row getting added into the index region for this entry will have a
>>> rowkey 0_x_5
>>> I am just using '_' as a seperator here just to show this. Actually we
>>> wont be having any seperator.
>>> So the rowkeys (in index region) will have a static begin part always.
>>> Will scan time also we know this part and so the startrow and endrow
>>> creation for the scan will be possible.. Note that we will store the
>> actual
>>> table row ke

Re: How to design a data warehouse in HBase?

[Michel Segel]

I don't know that I would recommend Impala at this stage in its development.
Sorry, it has a bit of growing up.

It's interesting, but no UDFs, right?

Sent from a remote device. Please excuse any typos...

Mike Segel

On Dec 13, 2012, at 4:42 PM, "Kevin O'dell"  wrote:

> Correct, Impala relies on the Hive Metastore.
> 
> On Thu, Dec 13, 2012 at 11:38 AM, Manoj Babu  wrote:
> 
>> Kevin,
>> 
>> Impala requires Hive right?
>> so to get the advantages of Impala do we need to go with Hive?
>> 
>> 
>> Cheers!
>> Manoj.
>> 
>> 
>> 
>> On Thu, Dec 13, 2012 at 9:03 PM, Mohammad Tariq 
>> wrote:
>> 
>>> Thank you so much for the clarification Kevin.
>>> 
>>> Regards,
>>>Mohammad Tariq
>>> 
>>> 
>>> 
>>> On Thu, Dec 13, 2012 at 9:00 PM, Kevin O'dell >>> wrote:
>>> 
 Mohammad,
 
  I am not sure you are thinking about Impala correctly.  It still uses
 HDFS so your data increasing over time is fine.  You are not going to
>>> need
 to tune for special CPU, Storage, or Network.  Typically with Impala
>> you
 are going to be bound at the disks as it functions off of data
>> locality.
 You can also use compression of Snappy, GZip, and BZip to help with
>> the
 amount of data you are storing.  You will not need to frequently update
 your hardware.
 
 On Thu, Dec 13, 2012 at 10:06 AM, Mohammad Tariq 
 wrote:
 
> Oh yes..Impala..good point by Kevin.
> 
> Kevin : Would it be appropriate if I say that I should go for Impala
>> if
 my
> data is not going to increase dramatically over time or if I have to
>>> work
> on only a subset of my BigData?Since Impala uses MPP, it may
> require specialized hardware tuned for CPU, storage and network
 performance
> for better results, which could become a problem if have to upgrade
>> the
> hardware frequently because of the growing data.
> 
> Regards,
>Mohammad Tariq
> 
> 
> 
> On Thu, Dec 13, 2012 at 8:17 PM, Kevin O'dell <
>>> kevin.od...@cloudera.com
>> wrote:
> 
>> To Mohammad's point.  You can use HBase for quick scans of the
>> data.
> Hive
>> for your longer running jobs.  Impala over the two for quick adhoc
>> searches.
>> 
>> On Thu, Dec 13, 2012 at 9:44 AM, Mohammad Tariq <
>> donta...@gmail.com>
>> wrote:
>> 
>>> I am not saying Hbase is not good. My point was to consider Hive
>> as
> well.
>>> Think about the approach keeping both the tools in mind and
>>> decide. I
>> just
>>> provided an option keeping in mind the available built-in Hive
> features.
>> I
>>> would like to add one more point here, you can map your Hbase
>>> tables
 to
>>> Hive.
>>> 
>>> Regards,
>>>Mohammad Tariq
>>> 
>>> 
>>> 
>>> On Thu, Dec 13, 2012 at 7:58 PM, bigdata <
>> bigdatab...@outlook.com>
>> wrote:
>>> 
 Hi, Tariq
 Thanks for your feedback. Actually, now we have two ways to
>> reach
 the
 target, by Hive and  by HBase.Could you tell me why HBase is
>> not
 good
>> for
 my requirements?Or what's the problem in my solution?
 Thanks.
 
> From: donta...@gmail.com
> Date: Thu, 13 Dec 2012 15:43:25 +0530
> Subject: Re: How to design a data warehouse in HBase?
> To: user@hbase.apache.org
> 
> Both have got different purposes. Normally people say that
>> Hive
 is
>>> slow,
> that's just because it uses MapReduce under the hood. And i'm
 sure
>> that
 if
> the data stored in HBase is very huge, nobody would write
> sequential
> programs for Get or Scan. Instead they will write MP jobs or
>> do
>>> something
> similar.
> 
> My point is that nothing can be 100% real time. Is that what
>>> you
>>> want?If
> that is the case I would never suggest Hadoop on the first
>>> place
 as
>>> it's
 a
> batch processing system and cannot be used like an OLTP
>> system,
>> unless
 you
> have thought of some additional stuff. Since you are talking
 about
> warehouse, I am assuming you are going to store and process
> gigantic
> amounts of data. That's the only reason I had suggested Hive.
> 
> The whole point is that everything is not a solution for
> everything.
>>> One
> size doesn't fit all. First, we need to analyze our
>> particular
 use
>>> case.
> The person, who says Hive is slow, might be correct. But only
>>> for
> his
> scenario.
> 
> HTH
> 
> Regards,
>Mohammad Tariq
> 
> 
> 
> On Thu, Dec 13, 2012 at 3:17 PM, bigdata <
 bigdatab...@outlook.com>
 wrote:
> 
>> Hi,
>> I've got the information that HIVE 's performance is too
>> low.
>>>

Re: How to design a data warehouse in HBase?

[Michel Segel]

You need to spend a bit of time on Schema design.
You need to flatten your Schema...
Implement some secondary indexing to improve join performance...

Depends on what you want to do... There are other options too...

Sent from a remote device. Please excuse any typos...

Mike Segel

On Dec 13, 2012, at 7:09 AM, lars hofhansl  wrote:

> For OLAP type queries you will generally be better off with a truly column 
> oriented database.
> You can probably shoehorn HBase into this, but it wasn't really designed with 
> raw scan performance along single columns in mind.
> 
> 
> 
> 
> From: bigdata 
> To: "user@hbase.apache.org"  
> Sent: Wednesday, December 12, 2012 9:57 PM
> Subject: How to design a data warehouse in HBase?
> 
> Dear all,
> We have a traditional star-model data warehouse in RDBMS, now we want to 
> transfer it to HBase. After study HBase, I learn that HBase is normally can 
> be query by rowkey.
> 1.full rowkey (fastest)2.rowkey filter (fast)3.column family/qualifier filter 
> (slow)
> How can I design the HBase tables to implement the warehouse functions, 
> like:1.Query by DimensionA2.Query by DimensionA and DimensionB3.Sum, count, 
> distinct ...
> From my opinion, I should create several HBase tables with all combinations 
> of different dimensions as the rowkey. This solution will lead to huge data 
> duplication. Is there any good suggestions to solve it?
> Thanks a lot!

Re: Put while move or split

[Michel Segel]

He could have meant that the region was unavailable for a longer duration.

My take is that you could have limited bandwidth, and a very large region on 
the move.
While I don't imagine that it could take that long to move a region, you never 
know how things are tuned and you could in theory hit a timeout.

I also have to wonder if HBase relies on the underlying HDFS to handle the move.

Thoughts?


Sent from a remote device. Please excuse any typos...

Mike Segel

On Dec 3, 2012, at 3:45 AM, ramkrishna vasudevan 
 wrote:

> Hi Jean
> 
> Incase of Puts and Scans if the split is in progress it will retry.
> Suppose before the client gets to know about the split, still if the put
> is directed with the parent region row key, internally HBase directs it to
> the correct daughter region.
> But i am not clear in what you mean by Region server down for 3 mins?  You
> mean during split your RS went down?
> 
> Regards
> Ram
> 
> 
> On Mon, Dec 3, 2012 at 8:23 AM, Jean-Marc Spaggiari > wrote:
> 
>> Ok. so if on the client side I'm "simply" using Puts, scans and gets
>> fron the HTableInterface, I don't have to worry about that since it
>> will retry.
>> 
>> But I still should consider catching exceptions when regions are not
>> available because the region server is down for less than 3 minutes,
>> right?
>> 
>> Thanks,
>> 
>> JM
>> 
>> 2012/12/2, Nicolas Liochon :
>>> It's not accessible, but it's more or less transparent (latency impact
>>> aside) for the end user: the hbase client will retry the operation.
>>> 
>>> On Sun, Dec 2, 2012 at 11:10 PM, Jean-Marc Spaggiari <
>>> jean-m...@spaggiari.org> wrote:
>>> 
 ?
>> 

Re: Why only check1-and-putMany and check1-and-deleteMany?

[Michel Segel]

Silly question...
Why?

I mean while it is technically feasible to do, can you come up with a use case 
and then show how it is better than the alternatives?



Sent from a remote device. Please excuse any typos...

Mike Segel

On Nov 29, 2012, at 12:44 PM, Jonathan Hsieh  wrote:

> Actually, I think we'd want to have repeated Conditions in to have multiple
> check predicates, and would want to add repeated Conditions to multi
> requests.
> 
> This would let us have arbitrary number of checks and arbitrary number of
> puts and deletes from an API point of view.
> 
> Jon.
> 
> On Mon, Nov 19, 2012 at 10:55 AM, Ted Yu  wrote:
> 
>> Looking at hbase-server/src/main/protobuf/Client.proto in trunk, I tried
>> the following formation to see if we can pass more than one qualifier:
>> 
>> Index: hbase-server/src/main/protobuf/Client.proto
>> ===
>> --- hbase-server/src/main/protobuf/Client.proto(revision 1411337)
>> +++ hbase-server/src/main/protobuf/Client.proto(working copy)
>> @@ -97,7 +97,7 @@
>> message Condition {
>>   required bytes row = 1;
>>   required bytes family = 2;
>> -  required bytes qualifier = 3;
>> +  repeated bytes qualifier = 3;
>>   required CompareType compareType = 4;
>>   required Comparator comparator = 5;
>> }
>> 
>> protoc is able to compile the above code.
>> This means we can potentially compare more than one qualifier in the check
>> step.
>> 
>> More discussion is needed to figure out the proper API which would be a
>> superset of what we have today:
>>  public boolean checkAndPut(final byte [] row,
>>  final byte [] family, final byte [] qualifier, final byte [] value,
>>  final Put put)
>> 
>> Cheers
>> 
>> On Sat, Nov 17, 2012 at 10:28 PM, Mike Spreitzer >> wrote:
>> 
>>> I am not looking at the trunk.  I am just a user looking at the interface
>>> documented on the web site and wondering why the atomic composite
>>> functionality is so limited.  It is not hard to imagine an expanded
>>> interface that allows checking an arbitrary set of columns and, if each
>>> has a value specified for it, doing an arbitrary set of Mutations.
>>> 
>>> Thanks,
>>> Mike
>>> 
>>> 
>>> 
>>> From:   Ted Yu 
>>> To: user@hbase.apache.org
>>> Date:   11/17/2012 11:44 PM
>>> Subject:Re: Why only check1-and-putMany and
>> check1-and-deleteMany?
>>> 
>>> 
>>> 
>>> Are you looking at trunk code ?
>>> 
>>> In trunk, take a look at HRegion.checkAndMutate() where you can specify
>>> ByteArrayComparable which does the comparison (matching).
>>> There is still the limit of single value:
>>>} else if (result.size() == 1 && !valueIsNull) {
>>>  KeyValue kv = result.get(0);
>>> 
>>> For the moment, only Put and Delete are supported:
>>> 
>>>if (!isPut && !(w instanceof Delete))
>>>  throw new DoNotRetryIOException("Action must be Put or Delete");
>>> If you have suggestion on how the check part can be more general, please
>>> comment.
>>> 
>>> Thanks
>>> 
>>> On Sat, Nov 17, 2012 at 7:30 PM, Mike Spreitzer 
>>> wrote:
>>> 
 If I understand correctly, an HTable has two composite atomic
>>> operations,
 checkAndPut and checkAndDelete.  The check part checks just a single
 column's value.  Why not allow checking more than one column?
 
 Why no checkAndAppend?
 
 The checkAndPut operation can put to multiple columns, right?  And the
 checkAndDelete operation can delete multiple columns, right?  Why not
>>> let
 them be mixed?  Why not checkAndMutate?
 
 Thanks!
 Mike
> 
> 
> 
> -- 
> // Jonathan Hsieh (shay)
> // Software Engineer, Cloudera
> // j...@cloudera.com

Re: Hbase MapReduce

[Michel Segel]

Do you think it would be a good idea to temper the use of CoProcessors?

This kind of reminds me of when people first started using stored procedures...


Sent from a remote device. Please excuse any typos...

Mike Segel

On Nov 24, 2012, at 11:46 AM, tom  wrote:

> Hi, but you do not need to us M/R. You could also use coprocessors.
> 
> See this site:
> https://blogs.apache.org/hbase/entry/coprocessor_introduction
> -> in the section "Endpoints"
> 
> An aggregation coprocessor ships with hbase that should match your 
> requirements.
> You just need to load it and eventually you can access it from HTable:
> 
> HTable.coprocessorExec(..) 
> 
> 
> Regards
> tom
> 
> Am 24.11.2012 18:32, schrieb Marcos Ortiz:
>> Regards, Dalia.
>> You have to use MapReduce for that.
>> In the HBase in Practice´s book, there are lot of great examples for this.
>> 
>> On 11/24/2012 12:15 PM, Dalia Sobhy wrote:
>>> Dear all,
>>> I wanted to ask a question..
>>> Do Hbase Aggregate Functions such as rowcount, getMax, get Average use 
>>> MapReduce to execute those functions?
>>> Thanks :D
> 

Re: more regionservers does not improve performance

[Michel Segel]

Here's a key statement:
Again, I am on grid which is used by many others, and the machines in my
cluster are not dedicated to my job. I am mainly looking at scalability
trends when running with various numbers of regionservers.

What do you notice when you monitor the cluster in ganglia?


Sent from a remote device. Please excuse any typos...

Mike Segel

On Oct 14, 2012, at 8:03 PM, lars hofhansl  wrote:

> Again, I am on grid which is used by many others, and the machines in my
> cluster are not dedicated to my job. I am mainly looking at scalability
> trends when running with various numbers of regionservers.

Re: Can we create rowkey with multiple columns

[Michel Segel]

The short answer is yes.
Longer answer... You could do this by creating a string w a character which 
delineates the separation of the fields.  Try using a pipe '|' character.
This would be generic and work in all instances, however if your key fields are 
fixed length, you could just concatenate them and then parse them on their set 
boundaries.

Sent from a remote device. Please excuse any typos...

Mike Segel

On Sep 23, 2012, at 6:06 AM, John Hancock  wrote:

> Ramasubramanian,
> 
> I think that you can if you know the maximum number of bytes the fields
> 2,3,5 are going to use.  This is easy if the fields are all primitive
> types.  Judging by the names of the columns you want to use for keys, it
> looks like long, long, boolean would work.
> 
> In your code use Bytes.toBytes() to convert the fields to byte arrays
> individually, then use Bytes.add() to glue the fields together to become
> your row key.
> 
> -John
> 
> On Sat, Sep 22, 2012 at 10:45 AM, Ramasubramanian Narayanan <
> ramasubramanian.naraya...@gmail.com> wrote:
> 
>> Hi,
>> 
>> Can we create rowkey with multiple columns? If so can u please share the
>> syntax of doing the same for the below scenario.
>> 
>> Table Name : TXNTABLE
>> Columns :
>> 1) Branch
>> 2) Account Number
>> 3) Txn Date
>> 4) Debit credit flag
>> 5) Txn Reference number.
>> 
>> In the above scenario I want to create a rowkey for the combination of
>> columns 2,3 & 5
>> 
>> regards,
>> Rams
>> 

Re: Problem with Hadoop and /etc/hosts file

[Michel Segel]

Just a hunch, w DNS, do you have your rDNS (reverse DNS lookup) set up 
correctly.

Sent from a remote device. Please excuse any typos...

Mike Segel

On Sep 15, 2012, at 8:04 PM, Alberto Cordioli  
wrote:

> This is the configuration I used till now...It works, but give the
> mentioned error (although the procedure seems to return correct
> results anyway.
> I think in /etc/hosts should be also the line
> 127.0.0.1 hostname
> 
> but in that case Hadoop does not start.
> 
> Alberto
> 
> On 14 September 2012 18:19, Shumin Wu  wrote:
>> Would that work for you?
>> 
>> 127.0.0.1localhost
>> 10.220.55.41  hostname
>> 
>> -Shumin
>> 
>> On Fri, Sep 14, 2012 at 6:18 AM, Alberto Cordioli <
>> cordioli.albe...@gmail.com> wrote:
>> 
>>> Hi,
>>> 
>>> I've successfully installed Apache HBase on a cluster with Hadoop.
>>> It works fine, but when I try to use Pig to load some data from an
>>> HBase table I get this error:
>>> 
>>> ERROR org.apache.hadoop.hbase.mapreduce.TableInputFormatBase - Cannot
>>> resolve the host name for /10.220.55.41 because of
>>> javax.naming.OperationNotSupportedException: DNS service refused
>>> [response code 5]; remaining name '41.55.220.10.in-addr.arpa'
>>> 
>>> Pig returns in any case the correct results (actually I don't know
>>> how), but I'd like to solve this issue.
>>> 
>>> I discovered that this error is due to a mistake in /etc/hosts
>>> configuration file. In fact, as reported in the documentation, I
>>> should add the line
>>> 127.0.0.1hostname
>>> (http://hbase.apache.org/book.html#os).
>>> 
>>> But if I add this entry my Hadoop cluster does not start since the
>>> datanote is bind to the local address instead to the hostname/IP
>>> address. For this reason in many tutorial it's suggested to remove
>>> such entry (e.g.
>>> 
>>> http://stackoverflow.com/questions/8872807/hadoop-datanodes-cannot-find-namenode
>>> ).
>>> 
>>> Basically if I add that line Hadoop won't work, but if I keep the file
>>> without the loopback address I get the above error.
>>> What can I do? Which is the right configuration?
>>> 
>>> 
>>> Thanks,
>>> Alberto
>>> 
>>> 
>>> 
>>> 
>>> --
>>> Alberto Cordioli
>>> 
> 
> 
> 
> -- 
> Alberto Cordioli
> 

Re: Problem with Hadoop and /etc/hosts file

[Michel Segel]

Just a hunch, w DNS, do you have your rDNS (reverse DNS lookup) set up 
correctly.

Sent from a remote device. Please excuse any typos...

Mike Segel

On Sep 15, 2012, at 8:04 PM, Alberto Cordioli  
wrote:

> This is the configuration I used till now...It works, but give the
> mentioned error (although the procedure seems to return correct
> results anyway.
> I think in /etc/hosts should be also the line
> 127.0.0.1 hostname
> 
> but in that case Hadoop does not start.
> 
> Alberto
> 
> On 14 September 2012 18:19, Shumin Wu  wrote:
>> Would that work for you?
>> 
>> 127.0.0.1localhost
>> 10.220.55.41  hostname
>> 
>> -Shumin
>> 
>> On Fri, Sep 14, 2012 at 6:18 AM, Alberto Cordioli <
>> cordioli.albe...@gmail.com> wrote:
>> 
>>> Hi,
>>> 
>>> I've successfully installed Apache HBase on a cluster with Hadoop.
>>> It works fine, but when I try to use Pig to load some data from an
>>> HBase table I get this error:
>>> 
>>> ERROR org.apache.hadoop.hbase.mapreduce.TableInputFormatBase - Cannot
>>> resolve the host name for /10.220.55.41 because of
>>> javax.naming.OperationNotSupportedException: DNS service refused
>>> [response code 5]; remaining name '41.55.220.10.in-addr.arpa'
>>> 
>>> Pig returns in any case the correct results (actually I don't know
>>> how), but I'd like to solve this issue.
>>> 
>>> I discovered that this error is due to a mistake in /etc/hosts
>>> configuration file. In fact, as reported in the documentation, I
>>> should add the line
>>> 127.0.0.1hostname
>>> (http://hbase.apache.org/book.html#os).
>>> 
>>> But if I add this entry my Hadoop cluster does not start since the
>>> datanote is bind to the local address instead to the hostname/IP
>>> address. For this reason in many tutorial it's suggested to remove
>>> such entry (e.g.
>>> 
>>> http://stackoverflow.com/questions/8872807/hadoop-datanodes-cannot-find-namenode
>>> ).
>>> 
>>> Basically if I add that line Hadoop won't work, but if I keep the file
>>> without the loopback address I get the above error.
>>> What can I do? Which is the right configuration?
>>> 
>>> 
>>> Thanks,
>>> Alberto
>>> 
>>> 
>>> 
>>> 
>>> --
>>> Alberto Cordioli
>>> 
> 
> 
> 
> -- 
> Alberto Cordioli
> 

Re: More rows or less rows and more columns

[Michel Segel]

Option c, depending on the use case, add a structure to you columns to store 
the data.
You may want to update this section


Sent from a remote device. Please excuse any typos...

Mike Segel

On Sep 10, 2012, at 12:30 PM, Harsh J  wrote:

> Hey Mohit,
> 
> See http://hbase.apache.org/book.html#schema.smackdown.rowscols
> 
> On Mon, Sep 10, 2012 at 10:56 PM, Mohit Anchlia  
> wrote:
>> Is there any recommendation on how many columns one should have per row. My
>> columns are < 200 bytes. This will help me to decide if I should shard my
>> rows with id + .
> 
> 
> 
> -- 
> Harsh J
> 

Re: adding data

[Michel Segel]

Ok, a couple of things

First, a contrarian piece of advice
Don't base your performance tuning on your initial load, but on your system at 
its steady state.
It's a simple concept that people forget and it can cause problems down the 
road

So we have two problems...

Rita w 13 billion rows and Hamed w 15,000 row inserts per second.

Both are distinct problems...

Rita, what constraints do you have? Have you thought about your schema? Have 
you thought about your region size? Have you tuned up HBase? How long do you 
have to load the data?
What is the growth and use of the data?
(these are pretty much the same for a DW,ODS, OLTP and NoSQL that and DBA would 
face.)

While you were already pointed to the bulk load, I thought you should also 
think about the other issues too.

Hamed, 
15k rows a second?

You have a slightly different problem. Rita asks about initial load. You have 
an issue with sustained input rate . 

You already see a problem with sequential keys... 
What is you planned access patterns? Size of the row, growth rate? Decay rate?
(do you even delete the data?)
Does the Schema make sense, or do you want to look at Asynchronous HBase?

Then there are other considerations...
Like your network and hardware...
What are you running on?
Memory, CPU, disk ... (ssd's?) 

A lot of unknown factors... So to help we're going to need more information

 
Sent from a remote device. Please excuse any typos...

Mike Segel

On Aug 4, 2012, at 2:44 AM, Hamed Ghavamnia  wrote:

> Hi,
> I'm facing a somewhat similar problem. I need to insert 15000 rows per
> second into hbase. I'm getting really bad results using the simple put
> api's (with multithreading). I've tried map/reduce integration as well. The
> problem seems to be the type of the row keys. My row keys have an
> incremental type, which makes hbase store them in the same region and
> therefore on the same node. I've tried changing my keys to a more random
> type, but still hbase stores them in the same region.
> Any solutions would be appreciated, some things which have crossed my mind:
> 1. To presplit my regions, but I'm not sure if the problem has anything to
> do with the regions.
> 2. Use the bulk load stated in you emails, but I don't where to start from.
> Do you have a link to a sample code which can be used?
> Any ideas?
> 
> On Sat, Aug 4, 2012 at 10:09 AM, anil gupta  wrote:
> 
>> Hi Rita,
>> 
>> HBase Bulk Loader is a viable solution for loading such huge data set. Even
>> if your import file has a separator other than tab you can use ImportTsv as
>> long as the separator is single character. If in case you want to put in
>> your business logic while writing the data to HBase then you can write your
>> own mapper class and use it with bulk loader. Hence, you can heavily
>> customize the bulk loader as per your needs.
>> These links might be helpful for you:
>> http://hbase.apache.org/book.html#arch.bulk.load
>> http://bigdatanoob.blogspot.com/2012/03/bulk-load-csv-file-into-hbase.html
>> 
>> HTH,
>> Anil Gupta
>> 
>> On Fri, Aug 3, 2012 at 9:54 PM, Bijeet Singh 
>> wrote:
>> 
>>> Well, if the file that you have contains TSV, you can directly use the
>>> ImportTSV utility of HBase to do a bulk load.
>>> More details about that can be found here :
>>> 
>>> http://hbase.apache.org/book/ops_mgt.html#importtsv
>>> 
>>> The other option for you is to run a MR job on the file that you have, to
>>> generate the HFiles, which you can later import
>>> to HBase using completebulkload.  HFiles are created using the
>>> HFileOutputFormat class.The output of Map should
>>> be Put or KeyValue. For Reduce you need to use configureIncrementalLoad
>>> which sets up reduce tasks.
>>> 
>>> Bijeet
>>> 
>>> 
>>> On Sat, Aug 4, 2012 at 8:13 AM, Rita  wrote:
>>> 
 I have a file which has 13 billion rows of key an value which I would
>>> like
 to place in Hbase. I was wondering if anyone has a good example to
>>> provide
 with mapreduce for some sort of work like this.
 
 
 tia
 
 
 --
 --- Get your facts first, then you can distort them as you please.--
 
>>> 
>> 
>> 
>> 
>> --
>> Thanks & Regards,
>> Anil Gupta
>> 

Re: Use of MD5 as row keys - is this safe?

[Michel Segel]

http://en.wikipedia.org/wiki/SHA-1

Check out the comparisons between the different SHA algos.

In theory a collision was found for SHA-1, but none found for SHA-2 does that 
mean that a collision doesn't exist? No, it means that it hasn't happened yet 
and the odds are that it won't be found. Possible? Yes, however, highly 
improbable. You have a better chance of winning the lotto...

The point was that if you are going to hash your key,then concatenate the 
initial key, you would be better off looking at the SHA-1 option. You have to 
consider a couple of factors...
1: availability of the algo. SHA-1 is in the standard java API and is readily 
available.
2: speed. Is SHA-1fast enough? Maybe, depending on your requirements. For most, 
I'll say probably.
3: Size of Key. SHA-1 is probably be smaller than having an MD-5 hash and the 
original key added.

Just food for thought...

Sent from a remote device. Please excuse any typos...

Mike Segel

On Jul 20, 2012, at 3:35 PM, Joe Pallas  wrote:

> 
> On Jul 20, 2012, at 12:16 PM, Michel Segel wrote:
> 
>> I don't believe that there has been any reports of collisions, but if. You 
>> are concerned you could use the SHA-1 for generating the hash. Relatively 
>> speaking, SHA-1is slower, but still fast enough for most applications.
> 
> Every hash function can have collisions, by definition.  If the correctness 
> of your design depends on collisions being impossible, rather than very rare, 
> then your design is faulty.
> 
> Cryptographic hash functions have the property that it is computationally 
> hard to create inputs that match a given output.  That doesn’t in itself make 
> cryptographic hash functions better than other hash functions for avoiding 
> hot-spotting.  (But it does usually make cryptographic hash functions more 
> expensive to compute than other hash functions.)
> 
> You may want to look at <http://www.strchr.com/hash_functions>  and 
> <http://programmers.stackexchange.com/questions/49550/which-hashing-algorithm-is-best-for-uniqueness-and-speed/145633#145633>.
> 
> Hope this helps,
> joe
> 
> 

Re: Use of MD5 as row keys - is this safe?

[Michel Segel]

I don't believe that there has been any reports of collisions, but if. You are 
concerned you could use the SHA-1 for generating the hash. Relatively speaking, 
SHA-1is slower, but still fast enough for most applications.

Don't know if it's speed relative to an MD5 and string cat, but it should yield 
a smaller key.

Sent from a remote device. Please excuse any typos...

Mike Segel

On Jul 20, 2012, at 11:31 AM, Damien Hardy  wrote:

> Le 20/07/2012 18:22, Jonathan Bishop a écrit :
>> Hi,
>> 
>> I know it is a commonly suggested to use an MD5 checksum to create a row
>> key from some other identifier, such as a string or long. This is usually
>> done to guard against hot-spotting and seems to work well.
>> 
>> My concern is that there no guard against collision when this is done - two
>> different strings or longs could produce the same row-key. Although this is
>> very unlikely, it is bothersome to consider this possibility for large
>> systems.
>> 
>> So what I usually do is concatenate the MD5 with the original identifier...
>> 
>> MD5(id) + id
>> 
>> which assures that the rowkey is both randomly distributed and unique.
>> 
>> Is this necessary, or is it the common practice to just use the MD5
>> checksum itself?
>> 
>> Thanks,
>> 
>> Jon
> 
> Hello Jonathan,
> 
> md5(id)+id is the good way to avoid hotspotting and insure uniqueness.
> 
> md5(id)[0]+id could be an other way to limit randomness of the rowid on
> 16 values
> You can now combine (with OR logic) 16 filters in a scanner (on for each
> letter available in md5 digest)
> it limits the balance on 16 potentials regions olso.
> 
> Cheers,
> 
> -- 
> Damien
> 

Re: Rowkey hashing to avoid hotspotting

[Michel Segel]

Reading hot spotting?
Hmmm there's a cache and I don't see any real use cases where you would have it 
occur naturally.

Sent from a remote device. Please excuse any typos...

Mike Segel

On Jul 17, 2012, at 10:53 AM, Alex Baranau  wrote:

> The most common reason for RS hotspotting during writing data in HBase is
> writing rows with monotonically increasing/decreasing row keys. E.g. if you
> put timestamp in the first part of your key, then you are likely to have
> monotonically increasing row keys. You can find more info about this issue
> and how to solve it here: [1] and also you may want to look at already
> implemented salting solution [2].
> 
> As for RS hotspotting during reading - it is hard to predict without
> knowing what it the most common data access patterns. E.g. putting model #
> in first part of a key may seem like a good distribution, but if your web
> site used mostly by Mercedes owners, the majority of the read load may be
> directed to just few regions. Again, salting can help a lot here.
> 
> +1 to what Cristofer said on other things, esp: use partial key scans were
> possible instead of filters and pre-split your table.
> 
> Alex Baranau
> --
> Sematext :: http://blog.sematext.com/ :: Hadoop - HBase - ElasticSearch -
> Solr
> 
> [1] http://bit.ly/HnKjbc
> [2] https://github.com/sematext/HBaseWD
> 
> On Tue, Jul 17, 2012 at 10:44 AM, AnandaVelMurugan Chandra Mohan <
> ananthu2...@gmail.com> wrote:
> 
>> Hi Cristofer,
>> 
>> Thanks for elaborate response!!!
>> 
>> I have no much information about production data as I work with partial
>> data. But based on discussion with my project partners, I have some answers
>> for you.
>> 
>> Number of model numbers and serial numbers will be finite. Not so many...
>> As far as I know,there is no predefined rule for model number or serial
>> number creation.
>> 
>> I have two access pattern. I count the number of rows for a specific model
>> number. I use rowkey filter for this. Also I filter the rows based on
>> model, serial number and some other columns. I scan the table with column
>> value filter for this case.
>> 
>> I will evaluate salting as you have explained.
>> 
>> Regards,
>> Anand.C
>> 
>> On Tue, Jul 17, 2012 at 12:30 AM, Cristofer Weber <
>> cristofer.we...@neogrid.com> wrote:
>> 
>>> Hi Anand,
>>> 
>>> As usual, the answer is that 'it depends'  :)
>>> 
>>> I think that the main question here is: why are you afraid that this
>> setup
>>> would lead to region server hotspotting? Is because you don't know how
>> your
>>> production data will seems?
>>> 
>>> Based on what you told about your rowkey, you will query mostly by
>>> providing model no. + serial no., but:
>>> 1 - How is your rowkey distribution? There are tons of different
>>> modelNumbers AND serialNumbers? Few modelNumbers and a lot of
>>> serialNumbers? Few of both?
>>> 2 - Putting modelNumber in front of your rowkey means that your data will
>>> be sorted by rowkey. So, what is the rule that determinates a modelNumber
>>> creation? Is it a sequential number that will be increased by time? If
>> so,
>>> are newer members accessed a lot more than older members? If not, what
>> will
>>> drive this number? Is it an encoding rule?
>>> 3 - Do you expect more write/read load over a few of these modelNumbers
>>> and/or serialNumbers? Will it be similar to a Pareto Distribution?
>>> Distributed over what?
>>> 
>>> Also, two other things got my attention here...
>>> 1 - Why are you filtering with regex? If your queries are over model no.
>> +
>>> serial no., why don't you just scan starting by your
>>> modelNumber+SerialNumber, and stoping on your next
>>> modelNumber+SerialNumber? Or is there another access pattern that doesn't
>>> apply to your composited rowkey?
>>> 2 - Why do you have to add a timestamp to ensure uniqueness?
>>> 
>>> Now, answering your question without more info about your data, you can
>>> apply hash in two ways:
>>> 1 - Generating a hash (MD5 is the most common as far as I read about) and
>>> using only this hash as your rowkey. Based on what you have told, this
>> way
>>> doesn't fit your needs, because you would not be able to do apply your
>>> filter anymore.
>>> 2 - Salting, by prefixing your current rowkey with a pinch of hash.
>> Notice
>>> that the hash portion must be your rowkey prefix to ensure a kind of
>>> balanced distribution over something (where something is your region
>>> servers). I'm working with a case that is a bit similar to yours, and
>> what
>>> I'm doing right now is calculating the hashValue of my rowkey and using a
>>> Java Formatter to create a hex string to prepend to my rowkey. Something
>>> like a String.format("%03x", hashValue)
>>> 
>>> In both cases, you still have to split your regions in advance, and it
>>> will be better to work your splitting before starting to feed your table
>>> with production data.
>>> 
>>> Also, you have to study the consequences that changing your rowkey will
>>> bring. It's not for free.
>>

Re: When node is down

[Michel Segel]

You don't notice it faster, it's the timeout. 
You can reduce the timeout, it's configurable. Default is 10 min.

There shouldn't be downtime of the cluster, just the node.

Note this is for Apache. MapR is different and someone from MapR should be able 
to provide details...

Sent from a remote device. Please excuse any typos...

Mike Segel

On Jun 22, 2012, at 8:41 AM, Tom Brown  wrote:

> Can it notice the node is down sooner? If that node is serving an active
> region (or if it's a datanode for an active region), that would be a
> potentially large amount of downtime.  With comodity hardware, and a large
> enough cluster, there will always be a machine or two being rebuilt...
> 
> Thanks!
> 
> -Tom
> 
> On Thursday, June 21, 2012, Michael Segel wrote:
> 
>> Assuming that you have an Apache release (Apache, HW, Cloudera) ...
>> (If MapR, replace the drive and you should be able to repair the cluster
>> from the console. Node doesn't go down. )
>> Node goes down.
>> 10 min later, cluster sees node down. Should then be able to replicate the
>> missing blocks.
>> 
>> Replace disk w new disk and rebuild file system.
>> Bring node up.
>> Rebalance cluster.
>> 
>> That should be pretty much it.
>> 
>> 
>> On Jun 21, 2012, at 10:17 PM, David Charle wrote:
>> 
>>> What is the best practice to remove a node and add the same node back for
>>> hbase/hadoop ?
>>> 
>>> Currently in our 10 node cluster; 2 nodes went down (bad disk, so node is
>>> down as its the root volume+data); need to replace the disk and add them
>>> back. Any quick suggestions or pointers to doc for the right procedure ?
>>> 
>>> --
>>> David
>> 
>> 

Re: Timestamp as a key good practice?

[Michel Segel]

You can't salt the key in the second table.
By salting the key, you lose the ability to do range scans, which is what you 
want to do.



Sent from a remote device. Please excuse any typos...

Mike Segel

On Jun 16, 2012, at 6:22 AM, Jean-Marc Spaggiari  
wrote:

> Thanks all for your comments and suggestions. Regarding the
> hotspotting I will try to salt the key in the 2nd table and see the
> results.
> 
> Yesterday I finished to install my 4 servers cluster with old machine.
> It's slow, but it's working. So I will do some testing.
> 
> You are recommending to modify the timestamp to be to the second or
> minute and have more entries per row. Is that because it's better to
> have more columns than rows? Or it's more because that will allow to
> have a more "squarred" pattern (lot of rows, lot of colums) which if
> more efficient?
> 
> JM
> 
> 2012/6/15, Michael Segel :
>> Thought about this a little bit more...
>> 
>> You will want two tables for a solution.
>> 
>> 1 Table is  Key: Unique ID
>>Column: FilePathValue: Full Path to file
>>Column: Last Update timeValue: timestamp
>> 
>> 2 Table is Key: Last Update time(The timestamp)
>>Column 1-N: Unique IDValue: Full Path to the
>> file
>> 
>> Now if you want to get fancy,  in Table 1, you could use the time stamp on
>> the column File Path to hold the last update time.
>> But its probably easier for you to start by keeping the data as a separate
>> column and ignore the Timestamps on the columns for now.
>> 
>> Note the following:
>> 
>> 1) I used the notation Column 1-N to reflect that for a given timestamp you
>> may or may not have multiple files that were updated. (You weren't specific
>> as to the scale)
>> This is a good example of HBase's column oriented approach where you may or
>> may not have a column. It doesn't matter. :-) You could also modify the
>> timestamp to be to the second or minute and have more entries per row. It
>> doesn't matter. You insert based on timestamp:columnName, value, so you will
>> add a column to this table.
>> 
>> 2) First prove that the logic works. You insert/update table 1 to capture
>> the ID of the file and its last update time.  You then delete the old
>> timestamp entry in table 2, then insert new entry in table 2.
>> 
>> 3) You store Table 2 in ascending order. Then when you want to find your
>> last 500 entries, you do a start scan at 0x000 and then limit the scan to
>> 500 rows. Note that you may or may not have multiple entries so as you walk
>> through the result set, you count the number of columns and stop when you
>> have 500 columns, regardless of the number of rows you've processed.
>> 
>> This should solve your problem and be pretty efficient.
>> You can then work out the Coprocessors and add it to the solution to be even
>> more efficient.
>> 
>> 
>> With respect to 'hot-spotting' , can't be helped. You could hash your unique
>> ID in table 1, this will reduce the potential of a hotspot as the table
>> splits.
>> On table 2, because you have temporal data and you want to efficiently scan
>> a small portion of the table based on size, you will always scan the first
>> bloc, however as data rolls off and compression occurs, you will probably
>> have to do some cleanup. I'm not sure how HBase  handles splits that no
>> longer contain data. When you compress an empty split, does it go away?
>> 
>> By switching to coprocessors, you now limit the update accessors to the
>> second table so you should still have pretty good performance.
>> 
>> You may also want to look at Asynchronous HBase, however I don't know how
>> well it will work with Coprocessors or if you want to perform async
>> operations in this specific use case.
>> 
>> Good luck, HTH...
>> 
>> -Mike
>> 
>> On Jun 14, 2012, at 1:47 PM, Jean-Marc Spaggiari wrote:
>> 
>>> Hi Michael,
>>> 
>>> For now this is more a proof of concept than a production application.
>>> And if it's working, it should be growing a lot and database at the
>>> end will easily be over 1B rows. each individual server will have to
>>> send it's own information to one centralized server which will insert
>>> that into a database. That's why it need to be very quick and that's
>>> why I'm looking in HBase's direction. I tried with some relational
>>> databases with 4M rows in the table but the insert time is to slow
>>> when I have to introduce entries in bulk. Also, the ability for HBase
>>> to keep only the cells with values will allow to save a lot on the
>>> disk space (futur projects).
>>> 
>>> I'm not yet used with HBase and there is still many things I need to
>>> undertsand but until I'm able to create a solution and test it, I will
>>> continue to read, learn and try that way. Then at then end I will be
>>> able to compare the 2 options I have (HBase or relational) and decide
>>> based on the results.
>>> 
>>> So yes, your reply helped because it's giving me a way to achieve 

Re: Full text search on HBase

[Michel Segel]

That's actually a bit out of date.

Sent from a remote device. Please excuse any typos...

Mike Segel

On Jun 6, 2012, at 8:56 AM, Otis Gospodnetic  wrote:

> https://issues.apache.org/jira/browse/HBASE-3529 
> 
>  
> Otis
> 
> Performance Monitoring for Solr / ElasticSearch / HBase - 
> http://sematext.com/spm 
> 
> 
> 
>> 
>> From: Jacques 
>> To: user@hbase.apache.org 
>> Sent: Tuesday, June 5, 2012 6:27 PM
>> Subject: Re: Full text search on HBase
>> 
>> You should check out the solbase deck by photo bucket from hbasecon.  Also
>> the work that jason ruthelgren did... Don't have jira offhand
>> On Jun 5, 2012 2:40 PM, "jack chrispoo"  wrote:
>> 
>>> Hi everyone,
>>> 
>>> Does anyone have experience with full text search on HBase? I was reading
>>> about hbasene, but last update was 2 years ago. I also read about lily, I
>>> was planning to try out lily. But before I try, it would be really nice if
>>> anyone can tell me what other options there are to do full text search on
>>> HBase, and if anyone has experience with lily please let me know how it
>>> performs.
>>> 
>>> Thanks
>>> jack
>>> 
>> 
>> 

Re: HBase (BigTable) many to many with students and courses

[Michel Segel]

Depends...
Try looking at a hierarchical model rather than a relational model...

One thing to remember is that joins are expensive in HBase.



Sent from a remote device. Please excuse any typos...

Mike Segel

On May 28, 2012, at 12:50 PM, Em  wrote:

> Hello list,
> 
> I have some time now to try out HBase and want to use it for a private
> project.
> 
> Questions like "How to I transfer one-to-many or many-to-many relations
> from my RDBMS's schema to HBase?" seem to be common.
> 
> I hope we can throw all the best practices that are out there in this
> thread.
> 
> As the wiki states:
> One should create two tables.
> One for students, another for courses.
> 
> Within the students' table, one should add one column per selected
> course with the course_id besides some columns for the student itself
> (name, birthday, sex etc.).
> 
> On the other hand one fills the courses table with one column per
> student_id besides some columns which describe the course itself (name,
> teacher, begin, end, year, location etc.).
> 
> So far, so good.
> 
> How do I access these tables efficiently?
> 
> A common case would be to show all courses per student.
> 
> To do so, one has to access the student-table and get all the student's
> courses-columns.
> Let's say their names are prefixed ids. One has to remove the prefix and
> then one accesses the courses-table to get all the courses and their
> metadata (name, teacher, location etc.).
> 
> How do I do this kind of operation efficiently?
> The naive and brute force approach seems to be using a Get-object per
> course and fetch the neccessary data.
> Another approach seems to be using the HTable-class and unleash the
> power of "multigets" by using the batch()-method.
> 
> All of the information above is theoretically, since I did not used it
> in code (I currently learn more about the fundamentals of HBase).
> 
> That's why I give the question to you: How do you do this kind of
> operation by using HBase?
> 
> Kind regards,
> Em
> 

Re: EndPoint Coprocessor could be dealocked?

[Michel Segel]

Sorry for the delay... Had a full day yesterday...

In a nut shell... Tough nut to crack.  I can give you a solution which you can 
probably enhance...

At the start, ignore coProcessors for now...

So what end up doing is the following.

General solution... N indexes..
Create a temp table in HBase. (1 column foo)

Assuming that you have a simple K,V index, so you just need to do a simple 
get() against the index to get the list of rows ...

For each index, fetch the rows.
For each row, write the rowid and then auto increment a counter in a column foo.

Then scan the table where foo's counter >= N. note that it should == N but just 
in case...

Now you have found multiple indexes.

Having said that... 
Again assuming your indexes are a simple K,V pair where V is a set of row ids...

Create a hash map of 
For each index:
 Get() row based on key
  For each rowid in row:
   If map.fetch(rowid) is null then add ( rowid, 1)
   Else increment the value in count;
  ;
;
For each rowid in map(rowid, count):
If count == number of indexes N
Then add rowid to result set.
;

Now just return the rows where you have it's rowid in the result set.

That you can do in a coprocessor...
  but you may have a memory issue... Depending on the number of rowid 
in your index.



does that help?


Sent from a remote device. Please excuse any typos...

Mike Segel

On May 14, 2012, at 8:20 AM, fding hbase  wrote:

> Hi Michel,
> 
> I indexed each column within a column family of a table, so we can query a
> row with specific column value.
> By multi-index I mean using multiple indexes at the same time on a single
> query. That looks like a SQL select
> with two *where* clauses of two indexed columns.
> 
> The row key of index table is made up of column value and row key of
> indexed table. For set intersection
> I used the utility class from Apache common-collections package
> CollectionUtils.intersection(). There's no
> assumption on sort order on indices. A scan with column value as startKey
> and column value+1 as endKey
> applied to index table will return all rows in indexed table with that
> column value.
> 
> For multi-index queries, previously I tried to use a scan for each index
> column and intersect of those
> result sets to get the rows that I want. But the query time is too long. So
> I decided to move the computation of
> intersection to server side and reduce the amount of data transferred.
> 
> Do you have any better idea?
> 
> On Mon, May 14, 2012 at 8:17 PM, Michel Segel 
> wrote:
> 
>> Need a little clarification...
>> 
>> You said that you need to do multi-index queries.
>> 
>> Did you mean to say multiple people running queries at the same time, or
>> did you mean you wanted to do multi-key indexes where the key is a
>> multi-key part.
>> 
>> Or did you mean that you really wanted to use multiple indexes at the same
>> time on a single query?
>> 
>> If its the latter, not really a good idea...
>> How do you handle the intersection of the two sets? (3 sets or more?)
>> Can you assume that the indexes are in sort order?
>> 
>> What happens when the results from the indexes exceed the amount of
>> allocated memory?
>> 
>> What I am suggesting you to do is to set aside the underpinnings of HBase
>> and look at the problem you are trying to solve in general terms.  Not an
>> easy one...
>> 
>> 
>> 
>> Sent from a remote device. Please excuse any typos...
>> 
>> Mike Segel
>> 
>> On May 14, 2012, at 4:35 AM, fding hbase  wrote:
>> 
>>> Hi all,
>>> 
>>> Is it possible to use table scanner (different from the host table
>> region)
>>> or
>>> execute coprocessor of another table, in the endpoint coprocessor?
>>> It looks like chaining coprocessors. But I found a possible deadlock!
>>> Can anyone help me with this?
>>> 
>>> In my testing environment I deployed the 0.92.0 version from CDH.
>>> I wrote an Endpoint coprocessor to do composite secondary index queries.
>>> The index is stored in another table and the index update is maintained
>>> by the client through a extended HTable. While a single index query
>>> works fine through Scanners of index table, soon after we realized
>>> we need to do multi-index queries at the same time.
>>> At first we tried to pull every row keys queried from a single index
>> table
>>> and do the merge (just set intersection) on the client,
>>> but that overruns the network bandwidth. So I proposed to try
>>> the endpoint coprocessor. The idea is to use coprocessors, one
>>> in master tabl

Re: hbase security

[Michel Segel]

Coprocessors are inside the engine... So they should be in place if you use the 
shell, or some other access method.

Sent from a remote device. Please excuse any typos...

Mike Segel

On May 15, 2012, at 6:11 AM, Rita  wrote:

> I am guessing I can´t use these features using shell, right?
> 
> 
> 
> On Tue, May 15, 2012 at 5:24 AM, Harsh J  wrote:
> 
>> HBase 0.92 has table-level security (among other goodies). Check out
>> this slide on what all it includes:
>> 
>> http://www.slideshare.net/ghelmling/new-hbase-features-coprocessors-and-security
>> 
>> There was also a good blog post earlier on how to set it up, but am
>> currently unable to locate it. I'll post back in case I find an
>> archive (or someone else may).
>> 
>> P.s. If you're making it to HBaseCon, you may not wanna miss
>> http://www.hbasecon.com/sessions/hbase-security-for-the-enterprise/
>> which also includes a tutorial (from Andrew).
>> 
>> On Tue, May 15, 2012 at 8:11 AM, Rita  wrote:
>>> Hello,
>>> 
>>> It seems for my hbase installation anyone can delete my tables. Is there
>> a
>>> way to prevent this? I would like to have only owner of Hmaster with
>> super
>>> authority.
>>> 
>>> tia
>>> 
>>> --
>>> --- Get your facts first, then you can distort them as you please.--
>> 
>> 
>> 
>> --
>> Harsh J
>> 
> 
> 
> 
> -- 
> --- Get your facts first, then you can distort them as you please.--

Re: EndPoint Coprocessor could be dealocked?

[Michel Segel]

Need a little clarification...

You said that you need to do multi-index queries.

Did you mean to say multiple people running queries at the same time, or did 
you mean you wanted to do multi-key indexes where the key is a multi-key part.

Or did you mean that you really wanted to use multiple indexes at the same time 
on a single query?

If its the latter, not really a good idea...
How do you handle the intersection of the two sets? (3 sets or more?)
Can you assume that the indexes are in sort order?

What happens when the results from the indexes exceed the amount of allocated 
memory?

What I am suggesting you to do is to set aside the underpinnings of HBase and 
look at the problem you are trying to solve in general terms.  Not an easy 
one...



Sent from a remote device. Please excuse any typos...

Mike Segel

On May 14, 2012, at 4:35 AM, fding hbase  wrote:

> Hi all,
> 
> Is it possible to use table scanner (different from the host table region)
> or
> execute coprocessor of another table, in the endpoint coprocessor?
> It looks like chaining coprocessors. But I found a possible deadlock!
> Can anyone help me with this?
> 
> In my testing environment I deployed the 0.92.0 version from CDH.
> I wrote an Endpoint coprocessor to do composite secondary index queries.
> The index is stored in another table and the index update is maintained
> by the client through a extended HTable. While a single index query
> works fine through Scanners of index table, soon after we realized
> we need to do multi-index queries at the same time.
> At first we tried to pull every row keys queried from a single index table
> and do the merge (just set intersection) on the client,
> but that overruns the network bandwidth. So I proposed to try
> the endpoint coprocessor. The idea is to use coprocessors, one
> in master table (the indexed table) and the other for each index table
> regions.
> Each master table region coprocessor instance invokes the index table
> coprocessor instances with its regioninfo (the startKey and endKey) and the
> scan,
> the index table region coprocessor instance scans and returns the row keys
> within the range of startKey and endKey passed in.
> 
> The cluster blocks sometimes in invoking the index table coprocessor. I
> traced
> into the code and found that when HConnection locates regions it will rpc
> to the same regionserver.
> 
> (After a while I found the index table coprocessor is equivalent to
> just a plain scan with filter, so I switched to scanners with filter, but
> the problem
> remains.)

Re: Occasional regionserver crashes following socket errors writing to HDFS

[Michel Segel]

Silly question...
Why are you using a reducer when working w HBase?

Second silly question... What is the max file size of your table that you are 
writing to?

Third silly question... How many regions are on each of your region servers

Fourth silly question ... There is this bandwidth setting... Default is 10MB... 
 Did you modify it?



Sent from a remote device. Please excuse any typos...

Mike Segel

On May 10, 2012, at 6:33 AM, Eran Kutner  wrote:

> Thanks Igal, but we already have that setting. These are the relevant
> setting from hdfs-site.xml :
>  
>dfs.datanode.max.xcievers
>65536
>  
>  
>dfs.datanode.handler.count
>10
>  
>  
>dfs.datanode.socket.write.timeout
>0
>  
> 
> Other ideas?
> 
> -eran
> 
> 
> 
> On Thu, May 10, 2012 at 12:25 PM, Igal Shilman  wrote:
> 
>> Hi Eran,
>> Do you have: dfs.datanode.socket.write.timeout set in hdfs-site.xml ?
>> (We have set this to zero in our cluster, which means waiting as long as
>> necessary for the write to complete)
>> 
>> Igal.
>> 
>> On Thu, May 10, 2012 at 11:17 AM, Eran Kutner  wrote:
>> 
>>> Hi,
>>> We're seeing occasional regionserver crashes during heavy write
>> operations
>>> to Hbase (at the reduce phase of large M/R jobs). I have increased the
>> file
>>> descriptors, HDFS xceivers, HDFS threads to the recommended settings and
>>> actually way above.
>>> 
>>> Here is an example of the HBase log (showing only errors):
>>> 
>>> 2012-05-10 03:34:54,291 WARN org.apache.hadoop.hdfs.DFSClient:
>>> DFSOutputStream ResponseProcessor exception  for block
>>> blk_-8928911185099340956_5189425java.io.IOException: Bad response 1 for
>>> block blk_-8928911185099340956_5189425 from datanode 10.1.104.6:50010
>>>   at
>>> 
>>> 
>> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$ResponseProcessor.run(DFSClient.java:
>>> 2986)
>>> 
>>> 2012-05-10 03:34:54,494 WARN org.apache.hadoop.hdfs.DFSClient:
>> DataStreamer
>>> Exception: java.io.InterruptedIOException: Interruped while waiting for
>> IO
>>> on channel java.nio.channels.SocketChannel[connected
>>> local=/10.1.104.9:59642remote=/
>>> 10.1.104.9:50010]. 0 millis timeout left.
>>>   at
>>> 
>>> 
>> org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:349)
>>>   at
>>> 
>>> 
>> org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157)
>>>   at
>>> 
>> org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:146)
>>>   at
>>> 
>> org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:107)
>>>   at
>> java.io.BufferedOutputStream.write(BufferedOutputStream.java:105)
>>>   at java.io.DataOutputStream.write(DataOutputStream.java:90)
>>>   at
>>> 
>>> 
>> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:
>>> 2848)
>>> 
>>> 2012-05-10 03:34:54,531 WARN org.apache.hadoop.hdfs.DFSClient: Error
>>> Recovery for block blk_-8928911185099340956_5189425 bad datanode[2]
>>> 10.1.104.6:50010
>>> 2012-05-10 03:34:54,531 WARN org.apache.hadoop.hdfs.DFSClient: Error
>>> Recovery for block blk_-8928911185099340956_5189425 in pipeline
>>> 10.1.104.9:50010, 10.1.104.8:50010, 10.1.104.6:50010: bad datanode
>>> 10.1.104.6:50010
>>> 2012-05-10 03:48:30,174 FATAL
>>> org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region
>> server
>>> serverName=hadoop1-s09.farm-ny.gigya.com,60020,1336476100422,
>>> load=(requests=15741, regions=789, usedHeap=6822, maxHeap=7983):
>>> regionserver:60020-0x2372c0e8a2f0008 regionserver:60020-0x2372c0e8a2f0008
>>> received expired from ZooKeeper, aborting
>>> org.apache.zookeeper.KeeperException$SessionExpiredException:
>>> KeeperErrorCode = Session expired
>>>   at
>>> 
>>> 
>> org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.connectionEvent(ZooKeeperWatcher.java:352)
>>>   at
>>> 
>>> 
>> org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.process(ZooKeeperWatcher.java:270)
>>>   at
>>> 
>>> 
>> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:531)
>>>   at
>>> org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:507)
>>> java.io.InterruptedIOException: Aborting compaction of store properties
>> in
>>> region
>>> 
>>> 
>> gs_users,611|QoCW/euBIKuMat/nRC5Xtw==,1334983658004.878522ea91f41cd76b903ea06ccd17f9.
>>> because user requested stop.
>>>   at
>>> org.apache.hadoop.hbase.regionserver.Store.compact(Store.java:998)
>>>   at
>>> org.apache.hadoop.hbase.regionserver.Store.compact(Store.java:779)
>>>   at
>>> 
>>> 
>> org.apache.hadoop.hbase.regionserver.HRegion.compactStores(HRegion.java:776)
>>>   at
>>> 
>>> 
>> org.apache.hadoop.hbase.regionserver.HRegion.compactStores(HRegion.java:721)
>>>   at
>>> 
>>> 
>> org.apache.hadoop.hbase.regionserver.CompactSplitThread.run(CompactSplitThread.java:81)
>>> 
>>> 
>>> This is from 10.1.104.9 (same machine running the region server that
>>> crashed):
>>> 2012-05-10 03:31:16,785 INFO
>>> org.apache.h

Re: Best Hbase Storage for PIG

[Michel Segel]

Ok...
5 machines...
Total cluster? Is that 5 DN?
Each machine 1quad core, 32gb ram, 7 x600GB not sure what types of drives. 


so let's assume 1control node running NN, JT, HM, ZK 
And 4 DN running DN,TT,RS.

We don't know your Schema, row size, or network. ( 10GBe, 1GBe, 100MBe?)

We also don't know if you've tuned GC implemented MSLABS ... Etc.

So 4 hours for 175Million rows? Could be ok.
Write your insert using a java M/R and see how long it takes.

Nor do we know how many. Slots you have on each box.
10k rows in a batch put() not really a good idea.
What's your region size?


Lots to think about before you can ask if you are doing the right thing, or if 
PIG is the bottleneck.


Sent from a remote device. Please excuse any typos...

Mike Segel

On Apr 26, 2012, at 7:09 AM, Rajgopal Vaithiyanathan  
wrote:

> My bad.
> 
> I had used cat /proc/cpuinfo | grep "processor"  | wc -l
> cat /proc/cpuinfo | grep “physical id” | sort | uniq | wc -l   => 4
> 
> so its 4 physical cores then!
> 
> and free -m gives me this.
> total   used   free sharedbuffers cached
> Mem: 32174  31382792  0123  27339
> -/+ buffers/cache:   3918  28256
> Swap:    24575  0  24575
> 
> 
> 
> On Thu, Apr 26, 2012 at 5:18 PM, Michel Segel 
> wrote:
> 
>> 32 cores w 32GB of Ram?
>> 
>> Pig isn't fast, but I have to question what you are using for hardware.
>> Who makes a 32 core box?
>> Assuming you mean 16 physical cores.
>> 
>> 7 drives? Not enough spindles for the number of cores.
>> 
>> Sent from a remote device. Please excuse any typos...
>> 
>> Mike Segel
>> 
>> On Apr 26, 2012, at 6:38 AM, Rajgopal Vaithiyanathan 
>> wrote:
>> 
>>> Hey all,
>>> 
>>> The default - HBaseStorage() takes hell lot of time for puts.
>>> 
>>> In a cluster of 5 machines, insertion of 175 Million records took 4Hours
>> 45
>>> minutes
>>> Question -  Is this good enough ?
>>> each machine has 32 cores and 32GB ram with 7*600GB harddisks. HBASE's
>> heap
>>> has been configured to 8GB.
>>> If the put speed is low, how can i improve them..?
>>> 
>>> I tried tweaking the TableOutputFormat by increasing the WriteBufferSize
>> to
>>> 24MB, and adding the multi put feature (by adding 10,000 puts in
>> ArrayList
>>> and putting it as a batch).  After doing this,  it started throwing
>>> 
>>> java.util.concurrent.ExecutionException: java.net.SocketTimeoutException:
>>> Call to slave1/172.21.208.176:60020 failed on socket timeout exception:
>>> java.net.SocketTimeoutException: 6 millis timeout while waiting for
>>> channel to be ready for read. ch :
>>> java.nio.channels.SocketChannel[connected
>>> local=/172.21.208.176:41135remote=slave1/
>>> 172.21.208.176:60020]
>>> 
>>> Which i assume is because, the clients took too long to put.
>>> 
>>> The detailed log is as follows from one of the reduce job is as follows.
>>> 
>>> I've 'censored' some of the details. which i assume is Okay.! :P
>>> 2012-04-23 20:07:12,815 INFO org.apache.hadoop.util.NativeCodeLoader:
>>> Loaded the native-hadoop library
>>> 2012-04-23 20:07:13,097 WARN
>>> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Source name ugi
>> already
>>> exists!
>>> 2012-04-23 20:07:13,787 INFO org.apache.zookeeper.ZooKeeper: Client
>>> environment:zookeeper.version=3.4.2-1221870, built on 12/21/2011 20:46
>> GMT
>>> 2012-04-23 20:07:13,787 INFO org.apache.zookeeper.ZooKeeper: Client
>>> environment:host.name=*.*
>>> 2012-04-23 20:07:13,787 INFO org.apache.zookeeper.ZooKeeper: Client
>>> environment:java.version=1.6.0_22
>>> 2012-04-23 20:07:13,787 INFO org.apache.zookeeper.ZooKeeper: Client
>>> environment:java.vendor=Sun Microsystems Inc.
>>> 2012-04-23 20:07:13,787 INFO org.apache.zookeeper.ZooKeeper: Client
>>> environment:java.home=/usr/lib/jvm/java-6-openjdk/jre
>>> 2012-04-23 20:07:13,787 INFO org.apache.zookeeper.ZooKeeper: Client
>>> environment:java.class.path=
>>> 2012-04-23 20:07:13,788 INFO org.apache.zookeeper.ZooKeeper: Client
>>> environment:java.library.path=**
>>> 2012-04-23 20:07:13,788 INFO org.apache.zookeeper.ZooKeeper: Client
>>> environment:java.io.tmpdir=***
>>> 2012-04-23 20:07:13,788 INFO org.apa

Re: Best Hbase Storage for PIG

[Michel Segel]

32 cores w 32GB of Ram?

Pig isn't fast, but I have to question what you are using for hardware.
Who makes a 32 core box?
Assuming you mean 16 physical cores.

7 drives? Not enough spindles for the number of cores.

Sent from a remote device. Please excuse any typos...

Mike Segel

On Apr 26, 2012, at 6:38 AM, Rajgopal Vaithiyanathan  
wrote:

> Hey all,
> 
> The default - HBaseStorage() takes hell lot of time for puts.
> 
> In a cluster of 5 machines, insertion of 175 Million records took 4Hours 45
> minutes
> Question -  Is this good enough ?
> each machine has 32 cores and 32GB ram with 7*600GB harddisks. HBASE's heap
> has been configured to 8GB.
> If the put speed is low, how can i improve them..?
> 
> I tried tweaking the TableOutputFormat by increasing the WriteBufferSize to
> 24MB, and adding the multi put feature (by adding 10,000 puts in ArrayList
> and putting it as a batch).  After doing this,  it started throwing
> 
> java.util.concurrent.ExecutionException: java.net.SocketTimeoutException:
> Call to slave1/172.21.208.176:60020 failed on socket timeout exception:
> java.net.SocketTimeoutException: 6 millis timeout while waiting for
> channel to be ready for read. ch :
> java.nio.channels.SocketChannel[connected
> local=/172.21.208.176:41135remote=slave1/
> 172.21.208.176:60020]
> 
> Which i assume is because, the clients took too long to put.
> 
> The detailed log is as follows from one of the reduce job is as follows.
> 
> I've 'censored' some of the details. which i assume is Okay.! :P
> 2012-04-23 20:07:12,815 INFO org.apache.hadoop.util.NativeCodeLoader:
> Loaded the native-hadoop library
> 2012-04-23 20:07:13,097 WARN
> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Source name ugi already
> exists!
> 2012-04-23 20:07:13,787 INFO org.apache.zookeeper.ZooKeeper: Client
> environment:zookeeper.version=3.4.2-1221870, built on 12/21/2011 20:46 GMT
> 2012-04-23 20:07:13,787 INFO org.apache.zookeeper.ZooKeeper: Client
> environment:host.name=*.*
> 2012-04-23 20:07:13,787 INFO org.apache.zookeeper.ZooKeeper: Client
> environment:java.version=1.6.0_22
> 2012-04-23 20:07:13,787 INFO org.apache.zookeeper.ZooKeeper: Client
> environment:java.vendor=Sun Microsystems Inc.
> 2012-04-23 20:07:13,787 INFO org.apache.zookeeper.ZooKeeper: Client
> environment:java.home=/usr/lib/jvm/java-6-openjdk/jre
> 2012-04-23 20:07:13,787 INFO org.apache.zookeeper.ZooKeeper: Client
> environment:java.class.path=
> 2012-04-23 20:07:13,788 INFO org.apache.zookeeper.ZooKeeper: Client
> environment:java.library.path=**
> 2012-04-23 20:07:13,788 INFO org.apache.zookeeper.ZooKeeper: Client
> environment:java.io.tmpdir=***
> 2012-04-23 20:07:13,788 INFO org.apache.zookeeper.ZooKeeper: Client
> environment:java.compiler=
> 2012-04-23 20:07:13,788 INFO org.apache.zookeeper.ZooKeeper: Client
> environment:os.name=Linux
> 2012-04-23 20:07:13,788 INFO org.apache.zookeeper.ZooKeeper: Client
> environment:os.arch=amd64
> 2012-04-23 20:07:13,788 INFO org.apache.zookeeper.ZooKeeper: Client
> environment:os.version=2.6.38-8-server
> 2012-04-23 20:07:13,788 INFO org.apache.zookeeper.ZooKeeper: Client
> environment:user.name=raj
> 
> 2012-04-23 20:07:13,788 INFO org.apache.zookeeper.ZooKeeper: Client
> environment:user.home=*
> 2012-04-23 20:07:13,788 INFO org.apache.zookeeper.ZooKeeper: Client
> environment:user.dir=**:
> 2012-04-23 20:07:13,790 INFO org.apache.zookeeper.ZooKeeper: Initiating
> client connection, connectString=master:2181 sessionTimeout=18
> watcher=hconnection
> 2012-04-23 20:07:13,822 INFO org.apache.zookeeper.ClientCnxn: Opening
> socket connection to server /172.21.208.180:2181
> 2012-04-23 20:07:13,823 INFO
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: The identifier of
> this process is 72909@slave1.slave1
> 2012-04-23 20:07:13,825 INFO org.apache.zookeeper.ClientCnxn: Socket
> connection established to master/172.21.208.180:2181, initiating session
> 2012-04-23 20:07:13,840 INFO org.apache.zookeeper.ClientCnxn: Session
> establishment complete on server master/172.21.208.180:2181, sessionid =
> 0x136dfa124e90015, negotiated timeout = 18
> 2012-04-23 20:07:14,129 INFO com.raj.OptimisedTableOutputFormat: Created
> table instance for index
> 2012-04-23 20:07:14,184 INFO org.apache.hadoop.util.ProcessTree: setsid
> exited with exit code 0
> 2012-04-23 20:07:14,205 INFO org.apache.hadoop.mapred.Task:  Using
> ResourceCalculatorPlugin :
> org.apache.hadoop.util.LinuxResourceCalculatorPlugin@4513e9fd
> 2012-04-23 20:08:49,852 WARN
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation:
> Failed all from
> region=index,,1335191775144.2e69ca9ad2a2d92699aa34b1dc37f1bb.,
> hostname=slave1, port=60020
> java.util.concurrent.ExecutionException: java.net.SocketTimeoutException:
> Call to slave1/172.21.208.176:60020 failed on socket timeout exception:
> java.net.SocketTim

Re: Problem to Insert the row that i was deleted

[Michel Segel]

Uhm... Not exactly Lars...
Just my $0.02 ...

While I don't disagree w Lars, I think the question you have to ask is why is 
the time stamp important?
Is it an element of the data or is it an artifact?
This kind of gets in to your Schema design and taking short cuts. You may want 
to instead create a data element or column containing the time stamp rather 
than rely on an HBase internal time stamp. 

Or you could increase the existing time stamp by 1 ns... ;-)
(Blame it on clock drift in your cluster? Of course we don't know the 
significance of the time stamp ... Or how often the row is un/re deleted... 
1000 times and you'd be off by a whole second.)

-Just saying... :-)


Sent from a remote device. Please excuse any typos...

Mike Segel

On Apr 25, 2012, at 1:14 AM, lars hofhansl  wrote:

> Your only chance is to run a major compaction on your table - that will get 
> rid of the delete marker. Then you can re-add the Put with the same TS.
> 
> -- Lars
> 
> ps. Rereading my email below... At some point I will learn to proof-read my 
> emails before I send them full of grammatical errors.
> 
> 
> - Original Message -
> From: Mahdi Negahi 
> To: Hbase 
> Cc: 
> Sent: Tuesday, April 24, 2012 10:46 PM
> Subject: RE: Problem to Insert the row that i was deleted
> 
> 
> 
> thanks for ur sharing 
> 
> so there is no solution for return back the row ( or cells/columns) ?
> 
> 
>> Date: Tue, 24 Apr 2012 22:39:49 -0700
>> From: lhofha...@yahoo.com
>> Subject: Re: Problem to Insert the row that i was deleted
>> To: user@hbase.apache.org
>> 
>> Rows (or rather cells/columns) are not actually deleted. Instead they are 
>> marked for deletion by a delete marker. The deleted cells are collected 
>> during the next major or minor comaction.
>> 
>> As long as the marker exist new Put (with thje same timestamp as the 
>> existing Put will affected by the delete marker.
>> The delete marker itself will exist until the next major compaction.
>> 
>> This might seems strange, but is actually an important feature of HBase as 
>> it allows operations to be executed in any order with the same end result.
>> 
>> -- Lars
>> 
>> 
>> From: Mahdi Negahi 
>> To: Hbase  
>> Sent: Tuesday, April 24, 2012 9:05 PM
>> Subject: Problem to Insert the row that i was deleted
>> 
>> 
>> 
>> 
>> 
>> I delete a row and I want to add the same row ( with same Timestamp ) to 
>> HBase but it is not added to the table. I know if I changed the timestamp it 
>> will added but it is necessary to add it with the same timestamp. 
>> 
>> please advice me where is my problem ?
>> 
>> regard
>> mahdi
> 

Re: Hbase Quality Of Service: large standarad deviation in insert time while inserting same type of rows in Hbase

[Michel Segel]

I guess Sesame Street isn't global... ;-) oh and of course I f'd the joke by 
saying Grover and not Oscar so it's my bad. :-(. [Google Oscar the groutch, and 
you'll understand the joke that I botched]

Its most likely GC and a mis tuned cluster.
The OP doesn't really get in to detail, except to say that his cluster is tiny. 
Yes, size does matter, regardless of those rumors to the contrary... 3 DN kinda 
small.  If he's splitting that often then his region size is too small, hot 
spotting and other things can impact performance however not in the way he 
described.

Also when you look at performance, look at reads, not writes. You can cache 
both and writes are less important than reads. (think about it.)

Since this type conversation keeps popping up, it would be a good topic for 
Strata in NY. (Not too subtle of a hint to those who are picking topics...) 
Good cluster design is important, more important than people think. 


Sent from a remote device. Please excuse any typos...

Mike Segel

On Apr 25, 2012, at 12:08 AM, Mikael Sitruk  wrote:

> 1. writes are not blocked during compaction
> 2. compaction cannot have a constant time since the files/regions are
> getting bigger
> 3. beside the GC pauses (which seems to be the best candidate here) on
> either the client or RS (what are your setting BTW, and data size per
> insert), did you presplit your regions or a split is occurring during the
> execution?
> 4. did you look at the logs? is there any operation that is taking too long
> there (in 0.92 you can configure and print any operation that will take
> long time)
> 
> 
> Regards
> Mikael.S
> 
> On Wed, Apr 25, 2012 at 4:58 AM, Michael Segel 
> wrote:
> 
>> Have you thought about Garbage Collection?
>> 
>> -Grover
>> 
>> Sent from my iPhone
>> 
>> On Apr 24, 2012, at 12:41 PM, "Skchaudhary"  wrote:
>> 
>>> 
>>> I have a cluster Hbase set-up. In that I have 3 Region Servers. There is
>> a
>>> table which has 27 Regions equally distributed among 3 Region servers--9
>>> regions per region server.
>>> 
>>> Region server 1 has ---region 1-9 Region server 2 has ---region 10-18
>> Region
>>> server 3 has ---region 19-27
>>> 
>>> Now when I start a program which inserts rows in region 1 and region 5
>> (both
>>> under Region Server-1) alternatively and on continuous basis, I see that
>> the
>>> insert time for each row is not constant or consistent---there is a lot
>> of
>>> variance or say standard deviation of insert time is quite large. Some
>> times
>>> it takes 2 ms to insert a row, sometimes 3 ms,sometimes 1000 ms and
>>> sometimes even > 3000 ms.Even though data size in rows is equal.
>>> 
>>> I understand that due to flushing and compaction of Regions the writes
>> are
>>> blocked---but then it should not be blocked for larger span of time and
>> the
>>> blockage time should be consistent for every flush/compaction (minor
>>> compaction).
>>> 
>>> All in all every time flush and compaction occurs it should take nearly
>> same
>>> time for each compaction and flush.
>>> 
>>> For our application we need a consistent quality of service and if not
>>> perfect atleast we need a well visible boundary lines--like for each row
>>> insert it will take some 0 to 10 ms and not more than 10 ms(just an
>> example)
>>> that even though minor compaction or flush occurs.
>>> 
>>> Is there any setting/configuration which I should try?
>>> 
>>> Any ideas of how to achieve it in Hbase.
>>> 
>>> Any help would be really appreciated.
>>> 
>>> Thanks in advance!!
>>> 
>>> --
>>> View this message in context:
>> http://old.nabble.com/Hbase-Quality-Of-Service%3A-large-standarad-deviation-in-insert-time-while-inserting-same-type-of-rows-in-Hbase-tp33740438p33740438.html
>>> Sent from the HBase User mailing list archive at Nabble.com.
>>> 
>> 

Re: HBase parallel scanner performance

[Michel Segel]

Narendra, 

Are you trying to solve a real problem, or is this a class project?

Your solution doesn't scale. It's a non starter. 130 seconds for each iteration 
times 1 million seconds is how long? 130 million seconds, which is ~36000 hours 
or over 4 years to complete.
(the numbers are rough but you get the idea...)

That's assuming that your table is static and doesn't change.

I didn't even ask if you were attempting any sort of server side filtering 
which would reduce the amount of data you send back to the client because it a 
moot point. 

Finer tuning is also moot.

So you insert a row in one table. You then do n^2 operations to pull out data.
The better solution is to insert data into 2 tables where you then have to do 
2n operations to get the same results. Thats per thread btw.  So if you were 
running 10 threads, you would have 10n^2  operations versus 20n operations to 
get the same result set.

A million row table... 1*10^13. Vs 2*10^6 

I don't believe I mentioned anything about HBase's internals and this solution 
works for any NoSQL database.


Sent from a remote device. Please excuse any typos...

Mike Segel

On Apr 19, 2012, at 7:03 AM, Narendra yadala  wrote:

> Hi Michel
> 
> Yes, that is exactly what I do in step 2. I am aware of the reason for the
> scanner timeout exceptions. It is the time between two consecutive
> invocations of the next call on a specific scanner object. I increased the
> scanner timeout to 10 min on the region server and still I keep seeing the
> timeouts. So I reduced my scanner cache to 128.
> 
> Full table scan takes 130 seconds and there are 2.2 million rows in the
> table as of now. Each row is around 2 KB in size. I measured time for the
> full table scan by issuing `count` command from the hbase shell.
> 
> I kind of understood the fix that you are specifying, but do I need to
> change the table structure to fix this problem? All I do is a n^2 operation
> and even that fails with 10 different types of exceptions. It is mildly
> annoying that I need to know all the low level storage details of HBase to
> do such a simple operation. And this is happening for just 14 parallel
> scanners. I am wondering what would happen when there are thousands of
> parallel scanners.
> 
> Please let me know if there is any configuration param change which would
> fix this issue.
> 
> Thanks a lot
> Narendra
> 
> On Thu, Apr 19, 2012 at 4:40 PM, Michel Segel 
> wrote:
> 
>> So in your step 2 you have the following:
>> FOREACH row IN TABLE alpha:
>>SELECT something
>>FROM TABLE alpha
>>WHERE alpha.url = row.url
>> 
>> Right?
>> And you are wondering why you are getting timeouts?
>> ...
>> ...
>> And how long does it take to do a full table scan? ;-)
>> (there's more, but that's the first thing you should see...)
>> 
>> Try creating a second table where you invert the URL and key pair such
>> that for each URL, you have a set of your alpha table's keys?
>> 
>> Then you have the following...
>> FOREACH row IN TABLE alpha:
>>  FETCH key-set FROM beta
>>  WHERE beta.rowkey = alpha.url
>> 
>> Note I use FETCH to signify that you should get a single row in response.
>> 
>> Does this make sense?
>> ( your second table is actually and index of the URL column in your first
>> table)
>> 
>> HTH
>> 
>> Sent from a remote device. Please excuse any typos...
>> 
>> Mike Segel
>> 
>> On Apr 19, 2012, at 5:43 AM, Narendra yadala 
>> wrote:
>> 
>>> I have an issue with my HBase cluster. We have a 4 node HBase/Hadoop
>> (4*32
>>> GB RAM and 4*6 TB disk space) cluster. We are using Cloudera distribution
>>> for maintaining our cluster. I have a single tweets table in which we
>> store
>>> the tweets, one tweet per row (it has millions of rows currently).
>>> 
>>> Now I try to run a Java batch (not a map reduce) which does the
>> following :
>>> 
>>>  1. Open a scanner over the tweet table and read the tweets one after
>>>  another. I set scanner caching to 128 rows as higher scanner caching is
>>>  leading to ScannerTimeoutExceptions. I scan over the first 10k rows
>> only.
>>>  2. For each tweet, extract URLs (linkcolfamily:urlvalue) that are there
>>>  in that tweet and open another scanner over the tweets table to see who
>>>  else shared that link. This involves getting rows having that URL from
>> the
>>>  entire table (not first 10k rows).
>>>  3. Do similar stuff as in step 2 for hashtags
>>>  (hashtagcolfamily:hashtagvalue).
>>>  4. Do

Re: HBase parallel scanner performance

[Michel Segel]

So in your step 2 you have the following:
FOREACH row IN TABLE alpha:
 SELECT something
 FROM TABLE alpha 
 WHERE alpha.url = row.url

Right?
And you are wondering why you are getting timeouts?
...
...
And how long does it take to do a full table scan? ;-)
(there's more, but that's the first thing you should see...)

Try creating a second table where you invert the URL and key pair such that for 
each URL, you have a set of your alpha table's keys?

Then you have the following...
FOREACH row IN TABLE alpha:
   FETCH key-set FROM beta 
   WHERE beta.rowkey = alpha.url

Note I use FETCH to signify that you should get a single row in response.

Does this make sense?
( your second table is actually and index of the URL column in your first table)

HTH 

Sent from a remote device. Please excuse any typos...

Mike Segel

On Apr 19, 2012, at 5:43 AM, Narendra yadala  wrote:

> I have an issue with my HBase cluster. We have a 4 node HBase/Hadoop (4*32
> GB RAM and 4*6 TB disk space) cluster. We are using Cloudera distribution
> for maintaining our cluster. I have a single tweets table in which we store
> the tweets, one tweet per row (it has millions of rows currently).
> 
> Now I try to run a Java batch (not a map reduce) which does the following :
> 
>   1. Open a scanner over the tweet table and read the tweets one after
>   another. I set scanner caching to 128 rows as higher scanner caching is
>   leading to ScannerTimeoutExceptions. I scan over the first 10k rows only.
>   2. For each tweet, extract URLs (linkcolfamily:urlvalue) that are there
>   in that tweet and open another scanner over the tweets table to see who
>   else shared that link. This involves getting rows having that URL from the
>   entire table (not first 10k rows).
>   3. Do similar stuff as in step 2 for hashtags
>   (hashtagcolfamily:hashtagvalue).
>   4. Do steps 1-3 in parallel for approximately 7-8 threads. This number
>   can be higher (thousands also) later.
> 
> 
> When I run this batch I got the GC issue which is specified here
> http://www.cloudera.com/blog/2011/02/avoiding-full-gcs-in-hbase-with-memstore-local-allocation-buffers-part-1/
> Then I tried to turn on the MSLAB feature and changed the GC settings by
> specifying  -XX:+UseParNewGC  and  -XX:+UseConcMarkSweepGC JVM flags.
> Even after doing this, I am running into all kinds of IOExceptions
> and SocketTimeoutExceptions.
> 
> This Java batch opens approximately 7*2 (14) scanners open at a point in
> time and still I am running into all kinds of troubles. I am wondering
> whether I can have thousands of parallel scanners with HBase when I need to
> scale.
> 
> It would be great to know whether I can open thousands/millions of scanners
> in parallel with HBase efficiently.
> 
> Thanks
> Narendra

Re: Storing extremely large size file

[Michel Segel]

Look, I don't want to be *that* guy, but just my $0.02 cents ...

I don't disagree that this topic doesn't come up all the time.  
But you have a couple of issues. What do you consider to be large?
1kb? 10? 100? >1MB?
(and then there's that lie that size doesn't matter... But let's not go 
there... ;-) 

Then there is the issue of region size, number of regions per RS...
(this question alone yields different answers from different people, meaning 
there isn't a single right answer or even a meaningful one.)

Then how about the heap size for the RS?
How about how much memory should have for your DNs...

You then Run in to some one saying that they read that they could build a DN 
with X hardware and yet this chapter now says you need to have Y and what's up 
with that...

Then you have the issue of alternatives like storing the blob in a sequence 
file while you store the index in HBase...


Just saying... :-)



Sent from a remote device. Please excuse any typos...

Mike Segel

On Apr 18, 2012, at 12:02 AM, lars hofhansl  wrote:

> I disagree. This comes up frequently and some basic guidelines should be 
> documented in the Reference Guide.
> If it is indeed not difficult than the section is the book will be short.
> 
> 
> 
> - Original Message -
> From: Michael Segel 
> To: "user@hbase.apache.org" 
> Cc: "user@hbase.apache.org" 
> Sent: Tuesday, April 17, 2012 3:43 PM
> Subject: Re: Storing extremely large size file
> 
> -1. It's a boring topic. 
> And it's one of those things that you either get it right or you end up 
> hiring a voodoo witch doctor to curse the author of the chapter...
> 
> I agree w Jack, it's not difficult just takes some planning and forethought.
> 
> Also reading lots of blogs... And some practice...
> 
> 
> Sent from my iPhone
> 
> On Apr 17, 2012, at 1:42 PM, "Dave Revell"  wrote:
> 
>> +1 Jack :)
>> 
>> On Tue, Apr 17, 2012 at 11:38 AM, Stack  wrote:
>> 
>>> On Tue, Apr 17, 2012 at 11:18 AM, Dave Revell 
>>> wrote:
 I think this is a popular topic that might deserve a section in The Book.
 
 By "this topic" I mean storing big binary chunks.
 
>>> 
>>> Get Jack Levin to write it (smile).
>>> 
>>> And make sure the values are compressed that you send over from the
>>> client
>>> 
>>> St.Ack
>>> 
> 
> 

Re: Is it possible to install two different Hbase versions in the same Cluster?

[Michel Segel]

Well, you could, however you run a greater risk of things breaking because you 
forgot to change a setting in the configuration file. You would have to change 
port listeners, location of config files, all sorts of things that you wouldn't 
have to change if you just segmented the nodes, different zk quorum and hmaster.

I mean the simple answer is sure, why not. But the longer answer is that you 
need to think more about what you want to do, why you want to do it, and what 
the least invasive way of doing it.

Sent from a remote device. Please excuse any typos...

Mike Segel

On Apr 16, 2012, at 6:39 AM, yonghu  wrote:

> Mike,
> 
> Can you explain why I can't put the RS on the same node?
> 
> Thanks!
> 
> Yong
> 
> On Mon, Apr 16, 2012 at 1:33 PM, Michel Segel  
> wrote:
>> Sure, just make sure you don't cross the configurations and don't put the RS 
>> on the same nodes.
>> 
>> 
>> Sent from a remote device. Please excuse any typos...
>> 
>> Mike Segel
>> 
>> On Apr 16, 2012, at 6:31 AM, yonghu  wrote:
>> 
>>> Hello,
>>> 
>>> I wonder if it's possible to install two different Hbase versions in
>>> the same cluster?
>>> 
>>> Thanks
>>> 
>>> Yong
>>> 
> 

Re: Is it possible to install two different Hbase versions in the same Cluster?

[Michel Segel]

Sure, just make sure you don't cross the configurations and don't put the RS on 
the same nodes.


Sent from a remote device. Please excuse any typos...

Mike Segel

On Apr 16, 2012, at 6:31 AM, yonghu  wrote:

> Hello,
> 
> I wonder if it's possible to install two different Hbase versions in
> the same cluster?
> 
> Thanks
> 
> Yong
> 

Re: efficient export w/o HDFS/copying

[Michel Segel]

Wouldn't that mean having the NAS attached to all of the nodes in the cluster?


Sent from a remote device. Please excuse any typos...

Mike Segel

On Mar 26, 2012, at 11:07 PM, Stack  wrote:

> On Mon, Mar 26, 2012 at 4:31 PM, Ted Tuttle  
> wrote:
>> Is there a method of exporting that skips the HDFS step?  We would
>> ideally like to export from HBase directly to an external filesystem
>> (e.g. our big slow NAS) skipping the HDFS step.
>> 
> 
> Do an OutputFormat that just writes files to your NAS and hook it up
> to the export tool in place of SequenceFileOutputFormat.  Set your new
> NASOutputFormat instead of SequenceFileOutputFormat here:
> http://hbase.apache.org/xref/org/apache/hadoop/hbase/mapreduce/Export.html#99
> (You'll probably have to override Exporter to do your customization
> copying bulk of createSubmittableJob into subclass)
> 
> St.Ack
> 

Re: Confirming a Bug

[Michel Segel]

Peter, that doesnt make sense.
 
I mean I believe you in what you are saying, but don't see how a VPN in would 
cause this variance in results.

Do you have any speculative execution turned on?

Are you counting just the numbers of rows in the result set, or are you using 
counters in the map reduce? (I'm assuming that you are running a map/reduce, 
and not just a simple connection and single threaded scan...).

I apologize if this had already been answered, I hadn't been following this too 
closely.

Sent from a remote device. Please excuse any typos...

Mike Segel

On Mar 22, 2012, at 8:01 PM, Peter Wolf  wrote:

> Hello again Lars and Lars,
> 
> Here is some additional information that may help you track this down.
> 
> I think this behavior has something to do with my VPN.  My servers are on the 
> Amazon Cloud and I normally run my client on my laptop via a VPN 
> (Tunnelblick: OS X 10.7.3; Tunnelblick 3.2.3 (build 2891.2932)).  This is 
> where I see the buggy behavior I describe.
> 
> However, when my Client is running on an EC2 machine, then I get different 
> behavior.  I can not prove that it is always correct, but in at least one 
> case my current code does not work on my laptop, but gets the correct number 
> of results on an EC2 machine.  Note that my scans are also much faster on the 
> EC2 machine.
> 
> I will do more tests to see if I can localize it further.
> 
> Hope this helps
> Thank you again
> Peter
> 
> 
> On 3/19/12 2:24 PM, Peter Wolf wrote:
>> Hello Lars and Lars,
>> 
>> Thank you for you help and attention.
>> 
>> I wrote a standalone test that exhibits the bug.
>> 
>> http://dl.dropbox.com/u/68001072/HBaseScanCacheBug.java
>> 
>> Here is the output.  It shows how the number of results and key value pairs 
>> varies as caching in changed, and families are included.  It shows the bug 
>> starting with 3 families and 5000 caching.  It also shows a new bug, where 
>> the query always fails with an IOException with 4 families.
>> 
>> CacheSize FamilyCount ResultCount KeyValueCount
>> 1000 1 1 10
>> 5000 1 1 10
> 

Re: hbase.cluster.distributed set to true but it says false

[Michel Segel]

1) your configuration is sub optimal so toss performance out the window. You 
appear to be on vms. Toss performance out the window.

2) sounds like you are picking up the default value and not your cluster's 
config.

Sent from a remote device. Please excuse any typos...

Mike Segel

On Mar 23, 2012, at 6:37 AM, Roberto Alonso  wrote:

> Hello,
> 
> I have been experimenting some problems with the fully distributed version.
> First of all I'll tell you my configuration:
> 
> I have 4 servers(server_{1,2,3,4}) with 6GB Ram and 2 cores. I installed
> hadoop in all of them, this is the configuration:
> - server_1 is namenode, datanode and secondarynamenode
> - server_2, server_3, server_4: datanodes
> The storage is aroung 500GB
> 
> On the other hand, I have installed hbase, and this is the configuration:
> - server_1: master and regionserver
> - server_2: zookeeper and regionserver
> - server_3 and server_4: regionserver
> 
> hbase-site.xml for each server looks like this:
>
>hbase.zookeeper.quorum
>server_2
>
>
>hbase.zookeeper.property.dataDir
>/hdfs/zookeeper
>
>
>dfs.support.append
>true
>
>
>hbase.rootdir
>hdfs://server_1:54310/hbase
>
>
>hbase.cluster.distributed
>true
> 
> 
> So I have some problems you may help me:
> 1º Insertion is slow. I have an alphanumeric row with two column families.
> It takes around 9 minutes to insert 20 rows, but this is more or less
> acceptable.
> 2º I have a map reduce job where I create a configuration:
>Configuration config = HBaseConfiguration.create();
>   and then I ask for config.get("hbase.cluster.distributed"); and It says
> "false", what do you think?
> 
> thanks a lot!

Re: multiple puts in reducer?

[Michel Segel]

There is nothing wrong in writing the output from a reducer to HBase.

The question you have to ask yourself is why are you using a reducer in the 
first place. ;-)

Look, you have a database. Why do you need a reducer?

It's a simple question... Right? ;-)

Look, I apologize for being cryptic. This is one of those philosophical design 
questions where you the developer/architect have to figure out the answer for 
yourself.  Maybe I should submit this as an HBaseconn topic for a presentation?

Sort of like how to do an efficient table join in HBase 

HTH
Sent from a remote device. Please excuse any typos...

Mike Segel

On Feb 28, 2012, at 11:16 PM, Jacques  wrote:

> I see nothing wrong with using the output of the reducer into hbase.   You
> just need to make sure duplicated operations wouldn't cause problems.  If
> using tableoutputformat, don't use random seeded keys.  If working straight
> against htable,  don't use increment.  We do this for some situations and
> either don't care about overwrites or use checkAndPut with a skip option in
> the application code.
> On Feb 28, 2012 9:40 AM, "Ben Snively"  wrote:
> 
>> Is there an assertion that you would never need to run a reducer when
>> writing to the DB?
>> 
>> It seems that there are cases when you would not need one, but the general
>> statement doesn't apply to all use cases.
>> 
>> If you were trying to process data where you may have two a map task (or
>> set of map tasks) output the same key,  you could have a case where you
>> need to reduce the data for that key prior to insert the result into hbase.
>> 
>> Am I missing something, but to me, that would be the deciding factor.  If
>> the key/values output in the map task are the exact values that need to be
>> inserted into HBase versus multiple values aggregated together and the
>> results put into the hbase entry?
>> 
>> Thanks,
>> Ben
>> 
>> 
>> On Tue, Feb 28, 2012 at 11:20 AM, Michael Segel
>> wrote:
>> 
>>> The better question is why would you need a reducer?
>>> 
>>> That's a bit cryptic, I understand, but you have to ask yourself when do
>>> you need to use a reducer when you are writing to a database... ;-)
>>> 
>>> 
>>> Sent from my iPhone
>>> 
>>> On Feb 28, 2012, at 10:14 AM, "T Vinod Gupta" 
>>> wrote:
>>> 
>>>> Mike,
>>>> I didn't understand - why would I not need reducer in hbase m/r? there
>>> can
>>>> be cases right.
>>>> My use case is very similar to Sujee's blog on frequency counting -
>>>> http://sujee.net/tech/articles/hadoop/hbase-map-reduce-freq-counter/
>>>> So in the reducer, I can do all the aggregations. Is there a better
>> way?
>>> I
>>>> can think of another way - to use increments in the map job itself. i
>>> have
>>>> to figure out if thats possible though.
>>>> 
>>>> thanks
>>>> 
>>>> On Tue, Feb 28, 2012 at 7:44 AM, Michel Segel <
>> michael_se...@hotmail.com
>>>> wrote:
>>>> 
>>>>> Yes you can do it.
>>>>> But why do you have a reducer when running a m/r job against HBase?
>>>>> 
>>>>> The trick in writing multiple rows... You do it independently of the
>>>>> output from the map() method.
>>>>> 
>>>>> 
>>>>> Sent from a remote device. Please excuse any typos...
>>>>> 
>>>>> Mike Segel
>>>>> 
>>>>> On Feb 28, 2012, at 8:34 AM, T Vinod Gupta 
>>> wrote:
>>>>> 
>>>>>> while doing map reduce on hbase tables, is it possible to do multiple
>>>>> puts
>>>>>> in the reducer? what i want is a way to be able to write multiple
>> rows.
>>>>> if
>>>>>> its not possible, then what are the other alternatives? i mean like
>>>>>> creating a wider table in that case.
>>>>>> 
>>>>>> thanks
>>>>> 
>>> 
>> 

Re: multiple puts in reducer?

[Michel Segel]

The assertion is that for most cases you shouldn't need one. That the rule of 
thumb is that you should have to defend your use of one. 

Reducers are expensive. Running multiple mappers in a job can be cheaper.

All I am saying is that you need to rethink your solution if you insist on 
using a reducer.



Sent from a remote device. Please excuse any typos...

Mike Segel

On Feb 28, 2012, at 11:40 AM, Ben Snively  wrote:

> Is there an assertion that you would never need to run a reducer when
> writing to the DB?
> 
> It seems that there are cases when you would not need one, but the general
> statement doesn't apply to all use cases.
> 
> If you were trying to process data where you may have two a map task (or
> set of map tasks) output the same key,  you could have a case where you
> need to reduce the data for that key prior to insert the result into hbase.
> 
> Am I missing something, but to me, that would be the deciding factor.  If
> the key/values output in the map task are the exact values that need to be
> inserted into HBase versus multiple values aggregated together and the
> results put into the hbase entry?
> 
> Thanks,
> Ben
> 
> 
> On Tue, Feb 28, 2012 at 11:20 AM, Michael Segel
> wrote:
> 
>> The better question is why would you need a reducer?
>> 
>> That's a bit cryptic, I understand, but you have to ask yourself when do
>> you need to use a reducer when you are writing to a database... ;-)
>> 
>> 
>> Sent from my iPhone
>> 
>> On Feb 28, 2012, at 10:14 AM, "T Vinod Gupta" 
>> wrote:
>> 
>>> Mike,
>>> I didn't understand - why would I not need reducer in hbase m/r? there
>> can
>>> be cases right.
>>> My use case is very similar to Sujee's blog on frequency counting -
>>> http://sujee.net/tech/articles/hadoop/hbase-map-reduce-freq-counter/
>>> So in the reducer, I can do all the aggregations. Is there a better way?
>> I
>>> can think of another way - to use increments in the map job itself. i
>> have
>>> to figure out if thats possible though.
>>> 
>>> thanks
>>> 
>>> On Tue, Feb 28, 2012 at 7:44 AM, Michel Segel >> wrote:
>>> 
>>>> Yes you can do it.
>>>> But why do you have a reducer when running a m/r job against HBase?
>>>> 
>>>> The trick in writing multiple rows... You do it independently of the
>>>> output from the map() method.
>>>> 
>>>> 
>>>> Sent from a remote device. Please excuse any typos...
>>>> 
>>>> Mike Segel
>>>> 
>>>> On Feb 28, 2012, at 8:34 AM, T Vinod Gupta 
>> wrote:
>>>> 
>>>>> while doing map reduce on hbase tables, is it possible to do multiple
>>>> puts
>>>>> in the reducer? what i want is a way to be able to write multiple rows.
>>>> if
>>>>> its not possible, then what are the other alternatives? i mean like
>>>>> creating a wider table in that case.
>>>>> 
>>>>> thanks
>>>> 
>> 

Re: multiple puts in reducer?

[Michel Segel]

Yes you can do it.
But why do you have a reducer when running a m/r job against HBase?

The trick in writing multiple rows... You do it independently of the output 
from the map() method.


Sent from a remote device. Please excuse any typos...

Mike Segel

On Feb 28, 2012, at 8:34 AM, T Vinod Gupta  wrote:

> while doing map reduce on hbase tables, is it possible to do multiple puts
> in the reducer? what i want is a way to be able to write multiple rows. if
> its not possible, then what are the other alternatives? i mean like
> creating a wider table in that case.
> 
> thanks

Re: Hbase vs MongoDB

[Michel Segel]

What's your thesis? 
Designing a comprehensive PMI system?
Something about NoSQL?

If you're doing something like Pacs, you really don't want to store the images 
in the database. So you end up with a hybrid...



Sent from a remote device. Please excuse any typos...

Mike Segel

On Feb 18, 2012, at 4:16 PM, Dalia Sobhy  wrote:

> 
> My usecase is developing a scalable medical App.
> This app has two main parts :1. Patient's Info2. Patient's Diagnosis, 
> Imaging...etc.3. How to link between them.
> I want to perform an API for developers to use...
> I want to use a NoSQL Database, because this is my thesis topic and I am 
> stuck in it. I only have a couple of months time period to accomplish it.
> So what I want is a guidance which better suits  my app, I will not perform a 
> complete app because I have time shortage.
> 
> 
>> Date: Sat, 18 Feb 2012 14:07:34 -0800
>> Subject: Re: Hbase vs MongoDB
>> From: whs...@gmail.com
>> To: user@hbase.apache.org
>> 
>> Can you please provide more information about your use case?  It would also
>> be helpful if you provided more information about specifically what you are
>> trying to assess.
>> 
>> While I don't have MongoDB specific experience, I'm sure some here do.
>> However, it is very difficult to be helpful when you ask such a generic
>> question.
>> 
>> thanks,
>> Jacques
>> 
>> 2012/2/18 Dalia Sobhy 
>> 
>>> 
>>> Could anyone help Hbase vs MongoDB??
> 

Re: help with schema

[Michel Segel]

Not easy to visualize...

Assuming your access path to the data is based on students, then you would 
serialize your college data as a column in the student's table.  

You need to forget your relational way of thinking.

You need to think not just in terms of data, but how you intend to use the 
data. If you have an orthogonal use case you will have to get creative... ;-)


Sent from a remote device. Please excuse any typos...

Mike Segel

On Feb 1, 2012, at 1:51 AM, francesco.tangari@gmail.com wrote:

> Suppose i have a relationship 1 to N, for example 
> Student , College. 
> Student Attributes: 
> Name,Surname,CollegeFKey,
> College attributes:
> CollegeKey,Other,Other.
> 
> Suppose that i have a program which read  students and Exams from a plain 
> text file.
> And  on this file i have duplicated Colleges and Duplicated Studens. 
> Like in denormalized tabels:
> CollegeId,Other,Other,Name,Surname,CollegeFkey.
> 1,x,y,Mike,M,1
> 1,x,y,R,P,1
> ...
> ...
> 
> you see i have to check in this case always that in my normalized db, i have 
> still not inserted in the Table College
> 2 times the key 1,   how can i solve this in Hbase ? i mean if i have 1000.. 
> tables and rows i dont want check
> for every Primary Key  and then for every Fk if it was inserted ok ?
> how can i solve that ? 
> 
> 
> 
> -- 
> francesco.tangari@gmail.com
> Inviato con Sparrow (http://www.sparrowmailapp.com/?sig)
> 

Re: PerformanceEvaluation results

[Michel Segel]

Tim,

Here's the problem in a nutshell, 
With respect to hardware, you have  5.4k rpms ? 6 drive and 8 cores?
Small slow drives, and still  a ratio less than one when you compare drives to 
spindles.

I appreciate that you want to maximize performance, but when it comes to 
tuning, you have to start before you get your hardware. 

 You are asking a question about tuning, but how can we answer if the numbers 
are ok?
Have you looked at your GCs and implemented mslabs? We don't know. Network 
configuration?

I mean that there's a lot missing and fine tuning a cluster is something you 
have to do on your own. I guess I could say your numbers look fine to me for 
that config... But honestly, it would be a swag.


Sent from a remote device. Please excuse any typos...

Mike Segel

On Feb 1, 2012, at 7:09 AM, Tim Robertson  wrote:

> Thanks Michael,
> 
> It's a small cluster, but is the hardware so bad?  We are particularly
> interested in relatively low load for random read write (2000
> transactions per second on <1k rows) but a decent full table scan
> speed, as we aim to mount Hive tables on HBase backed tables.
> 
> Regarding tuning... not exactly sure which you would be interested in
> seeing.  The config is all here:
> http://code.google.com/p/gbif-common-resources/source/browse/#svn%2Fcluster-puppet%2Fmodules%2Fhadoop%2Ftemplates
> 
> Cheers,
> Tim
> 
> 
> 
> On Wed, Feb 1, 2012 at 1:56 PM, Michael Segel  
> wrote:
>> No.
>> What tuning did you do?
>> Why such a small cluster?
>> 
>> Sorry, but when you start off with a bad hardware configuration, you can get 
>> Hadoop/HBase to work, but performance will always be sub-optimal.
>> 
>> 
>> 
>> Sent from my iPhone
>> 
>> On Feb 1, 2012, at 6:52 AM, "Tim Robertson"  
>> wrote:
>> 
>>> Hi all,
>>> 
>>> We have a 3 node cluster (CD3u2) with the following hardware:
>>> 
>>> RegionServers (+DN + TT)
>>>  CPU: 2x Intel(R) Xeon(R) CPU E5630 @ 2.53GHz (quad)
>>>  Disks: 6x250G SATA 5.4K
>>>  Memory: 24GB
>>> 
>>> Master (+ZK, JT, NN)
>>>  CPU: Intel(R) Xeon(R) CPU X3363 @ 2.83GHz, 2x6MB (quad)
>>>  Disks: 2x500G SATA 7.2K
>>>  Memory: 8GB
>>> 
>>> Memory wise, we have:
>>> Master:
>>>  NN: 1GB
>>>  JT: 1GB
>>>  HBase master: 6GB
>>>  ZK: 1GB
>>> RegionServers:
>>>  RegionServer: 6GB
>>>  TaskTracker: 1GB
>>>  11 Mappers @ 1GB each
>>>  7 Reducers @ 1GB each
>>> 
>>> HDFS was empty, and I ran randomWrite and scan both with number
>>> clients of 50 (seemed to spawn 500 Mappers though...)
>>> 
>>> randomWrite:
>>> 12/02/01 13:27:47 INFO mapred.JobClient: ROWS=52428500
>>> 12/02/01 13:27:47 INFO mapred.JobClient: ELAPSED_TIME=84504886
>>> 
>>> scan:
>>> 12/02/01 13:42:52 INFO mapred.JobClient: ROWS=52428500
>>> 12/02/01 13:42:52 INFO mapred.JobClient: ELAPSED_TIME=8158664
>>> 
>>> Would I be correct in thinking that this is way below what is to be
>>> expected of this hardware?
>>> We're setting up ganglia now to start debugging, but any suggestions
>>> on how to diagnose this would be greatly appreciated.
>>> 
>>> Thanks!
>>> Tim
> 

Re: No. of families

[Michel Segel]

Sorry, but you misunderstand.

Implementing a hierarchical model in any NoSQL database is trivial.  



Sent from a remote device. Please excuse any typos...

Mike Segel

On Dec 30, 2011, at 9:01 PM, Imran M Yousuf  wrote:

> Hi Michael,
> 
> I totally agree. Thats what we tried to implement in Smart CMS making
> it easy for clients to retrieve and persist data with more added
> facility. BTW, I forgot to mention that Smart CMS persists data in
> HBase.
> 
> Regards,
> 
> Imran
> 
> On Sat, Dec 31, 2011 at 8:56 AM, Michael Segel
>  wrote:
>> Hierarchical data doesn't necessarily has anything to do w column families. 
>> You can do a hierarchical model in a single column family.
>> It's pretty straight forward.
>> 
>> Sent from my iPhone
>> 
>> On Dec 30, 2011, at 6:34 PM, "Imran M Yousuf"  wrote:
>> 
>>> Hi,
>>> 
>>> Rather than addressing the issue of how many column family may be used and
>>> query performance on them, I would like to address the problem of
>>> hierarchical data.
>>> 
>>> We were facing an issue of storing hierarchical data in one of our
>>> applications and for solving that, and many other features, we turned
>>> developed Smart CMS - smart-cms.org If it sounds interesting to your
>>> problem let me know, we can then collaborate in more details.
>>> 
>>> Thank you,
>>> 
>>> Imran
>>> 
>>> On 28 Dec 2011 13:55, "Mohammad Tariq"  wrote:
>>> 
>>> Hello all,
>>> 
>>>   Having less no. of column families is advisable. It is feasible to
>>> have 2 or 3 sub column families within a single column family???I
>>> want to store xml data in Hbase and I have sub tags that may go down
>>> to 2 or 3 levels.
>>> 
>>> Regards,
>>>Mohammad Tariq
> 
> 
> 
> -- 
> Imran M Yousuf
> Entrepreneur & CEO
> Smart IT Engineering Ltd.
> Dhaka, Bangladesh
> Twitter: @imyousuf - http://twitter.com/imyousuf
> Blog: http://imyousuf-tech.blogs.smartitengineering.com/
> Mobile: +880-1711402557
> 

Re: Could an EC2 machine to 4 times slower than local dev workstation?

[Michel Segel]

Hi,
Yes the performance hit is normal.
Looks like you're seeing network latency on disk I/O.
Could also be a tuning issue. (differences in configurations...)

Not sure how much. CPU difference will impact performance, while disk I/O will 
really kill you.


Sent from a remote device. Please excuse any typos...

Mike Segel

On Dec 30, 2011, at 11:33 AM, Mark Kerzner  wrote:

> Thank you, Bryan,
> 
> that is very important and clear some cloudiness in my mind.
> 
> Sincerely,
> Mark
> 
> On Fri, Dec 30, 2011 at 10:54 AM, Bryan Beaudreault <
> bbeaudrea...@hubspot.com> wrote:
> 
>> We have also seen this in our testing, though we focused mainly on MR more
>> than HBase.
>> 
>> Keep in mind that EC2 Compute Units are defined as follows:
>> 
>> The amount of CPU that is allocated to a particular instance is expressed
>>> in terms of these EC2 Compute Units. We use several benchmarks and tests
>> to
>>> manage the consistency and predictability of the performance of an EC2
>>> Compute Unit. One EC2 Compute Unit provides the equivalent CPU capacity
>> of
>>> a 1.0-1.2 GHz 2007 Opteron or 2007 Xeon processor.
>> 
>> 
>> This does not even account for CPU contention that Amandeep mentioned,
>> which we have noticed at times as well.  Also, c1.mediums have a I/O
>> Performance rating of "Moderate."  I think this mainly refers to ethernet
>> speed, but it could refer to disk speed as well.
>> 
>> If your local workstation is a reasonably modern system, it is very
>> possible for you to see much better performance locally.  The difference
>> between 2.5 1.0 GHz 2007 processors (2.5 compute units) and a modern i5,
>> i7, or equivalent is huge not just in speed and number of cores, but
>> architecture, cache, etc.  In terms of HBase write speed, if you are
>> running on an SSD this could cause a substantial gap as well.
>> 
>> On Fri, Dec 30, 2011 at 12:38 AM, Amandeep Khurana 
>> wrote:
>> 
>>> Is your client program running on the same node? Given that c1.mediums
>> are
>>> on shared hosts, your neighbor might be overloading his VM, causing yours
>>> to starve.
>>> 
>>> On Fri, Dec 30, 2011 at 9:50 AM, Mark Kerzner 
>>> wrote:
>>> 
 Hi,
 
 I am running a small program to load about 1 million rows into HBase.
>> It
 takes 200 seconds on my dev machine, and 800 seconds on a c1.medium EC2
 machine. Both are running the same version of Ubuntu and the same
>> version
 of HBase. Everything is local on one machine in both cases.
 
 What could the difference between the two environments be? I did notice
 that my local machine has higher CPU loads:
 
 hbase 64%
 java (my app) 38%
 hdfs 20%
 
 whereas the EC2 machine
 hbase 47%
 java (my app) 23%
 hdfs 14%
 
 
 Sincerely,
 Mark
 
>>> 
>> 

Re: Data management strategy

[Michel Segel]

Richard,

Let's see if I understand what you want to do...

You have some data and you want to store it in some table A.
Some of the records/rows in this table have a limited life span of 3 days, 
others have a limited life span of 3 months. But both are the same records? By 
this I mean that both records contain the same type of data but there is some 
business logic that determines which record gets deleted.
( like purge all records that haven't been accessed in the last 3 days.)

If what I imagine is true, you can't use the standard TTL unless you know that 
after a set N hours or days the record will be deleted. Like all records will 
self destruct 30 days past creation.

The simplest solution would be to have a column that contains a, timestamp of 
last access and your application controls when this field gets updated. Then 
using cron, launch a job that scans the table and removes the rows which meet 
your delete criteria.

Since co-processors are new... Not yet in any of the commercial releases, I 
would suggest keeping the logic simple. You can always refactor your code to 
use Co-processors when you've had time to play with them.

Even with coprocessors because the data dies an arbitrary death, you will still 
have to purge the data yourself. Hence the cron job that marks the record for 
deletion and then does a major compaction on the table to really delete the 
rows...

Of course the standard caveats apply, assuming I really did understand what you 
wanted...

Oh and KISS is always the best practice... :-)

Sent from a remote device. Please excuse any typos...

Mike Segel

On Dec 21, 2011, at 12:03 PM, Richard Lawrence  wrote:

> Hi
> 
> I was wondering if I could seek some advance about data management in HBase?  
> I plan to use HBase to store data that has a  variable length lifespan, the 
> vast majority will be short but occasionally the data life time will be 
> significantly longer (3 days versus 3 months).  Once the lifespan is over I 
> need the data to be deleted at some point in the near future (within a few 
> day is fine).  I don’t think I can use standard TTL for this because that’s 
> fixed at a column family level.  Therefore, my plan was to run script every 
> few days that looks through external information for what needs to be kept 
> and then updates HBase in some way so that it can understand.  With the data 
> in HBase I can then use the standard TTL mechanism to clean up.
> 
> The two ways I can think of to let HBase know are:
> 
> Add a co-processor that updates timestamp on each read and then have my 
> process simply read the data.  I shied away from this because the 
> documentation indicated the co-processor can’t take row locks.  Does that 
> imply that it shouldn’t modify the underlying data.  For my use case the 
> timestamp doesn’t have to be perfect the keys are created in a such that the 
> underlying data is fixed at creation time.
> Add an extra column to each row that’s a cache flag and rewrite that at 
> various intervals so that the timestamp updates and prevents the TTL from 
> deleting it.
> 
> Are there other best practice alternatives?
> 
> Thanks
> 
> Richard
> 

Re: About performance issue of Hive/HBase vs Hive/HDFS

[Michel Segel]

Hey Bruce, 
There's a bit more... 
Setting up the region size to help minimize the number of regions,  then there 
is tuning the GC and also setting up mslabs.

You can check out Todd L.'s blog posts on Cloudera's website. I think he has 
both topics covered in blogs.

Doug may also have updated the HBase book too.

Sent from a remote device. Please excuse any typos...

Mike Segel

On Dec 21, 2011, at 7:32 AM, Bruce Bian  wrote:

> Hi Michel,
> Maybe I missed something, but that's what was said in those two posts and
> also the results I've got so far when I was doing my own tests.
> So as for tuning HBase, after ensuring data locality, using scanner caching
> and turning off block caching, what are other configs I should pay
> attention to, any tips?
> Yeah,I'm happy to give snappy a shot.
> 
> Regards,
> Bruce
> 
> On Wed, Dec 21, 2011 at 8:52 PM, Michel Segel 
> wrote:
> 
>> Ok... Just my random thoughts...
>> There definitely is overhead in HBase that doesn't exist when you are
>> doing direct access against a hive table. 4 to 5 times slower? I'd question
>> how you tuned your HBase.
>> 
>> Having said that, I would imagine that there are still some potential
>> improvements that could be done on hive to work better w HBase.
>> Also why LZO and not Snappy?
>> 
>> 
>> Sent from a remote device. Please excuse any typos...
>> 
>> Mike Segel
>> 
>> On Dec 21, 2011, at 1:14 AM, Bruce Bian  wrote:
>> 
>>> Hi there,
>>> After I read these two posts on the mailing list
>>> 
>> http://search-hadoop.com/m/nVaw59rFlY1/Performance+between+Hive+queries+vs.+Hive+over+HBase+queries&subj=Performance+between+Hive+queries+vs+Hive+over+HBase+queries
>>> 
>> http://search-hadoop.com/m/X1rzQ1QDSaf2/Hive%252BHBase+performance+is+much+poorer+than+Hive%252BHDFS&subj=Hive+HBase+performance+is+much+poorer+than+Hive+HDFS
>>> Seems like a 4~5X performance downgrade of Hive/HBase vs Hive/HDFS is
>>> expected due to hbase built another layer on top of HDFS. If this is the
>>> issue here, is it possible to bypass the HBase layer to read the HFiles
>>> stored on HDFS directly?
>>> Another possibility maybe the fact that for the same table, the storage
>> is
>>> much larger in HBase(around 5X in my test case, both uncompressed)than in
>>> Hive, as hbase stores each KV pair for one column which causes the key to
>>> be repeated several times. But after I tried compress the Hbase table
>> using
>>> LZO(now nearly the same as in hive uncompressed table), there's no
>>> performance gain for queries like select count(*) from xtable;
>>> Is there anyone working on this?Not sure whether I should put this post
>> to
>>> Hive's mailing list but there seems to be no progress on issues like
>>> https://issues.apache.org/jira/browse/HIVE-1231
>>> 
>>> Regards,
>>> Bruce
>> 

Re: About performance issue of Hive/HBase vs Hive/HDFS

[Michel Segel]

Ok... Just my random thoughts...
There definitely is overhead in HBase that doesn't exist when you are doing 
direct access against a hive table. 4 to 5 times slower? I'd question how you 
tuned your HBase.

Having said that, I would imagine that there are still some potential 
improvements that could be done on hive to work better w HBase.
Also why LZO and not Snappy?


Sent from a remote device. Please excuse any typos...

Mike Segel

On Dec 21, 2011, at 1:14 AM, Bruce Bian  wrote:

> Hi there,
> After I read these two posts on the mailing list
> http://search-hadoop.com/m/nVaw59rFlY1/Performance+between+Hive+queries+vs.+Hive+over+HBase+queries&subj=Performance+between+Hive+queries+vs+Hive+over+HBase+queries
> http://search-hadoop.com/m/X1rzQ1QDSaf2/Hive%252BHBase+performance+is+much+poorer+than+Hive%252BHDFS&subj=Hive+HBase+performance+is+much+poorer+than+Hive+HDFS
> Seems like a 4~5X performance downgrade of Hive/HBase vs Hive/HDFS is
> expected due to hbase built another layer on top of HDFS. If this is the
> issue here, is it possible to bypass the HBase layer to read the HFiles
> stored on HDFS directly?
> Another possibility maybe the fact that for the same table, the storage is
> much larger in HBase(around 5X in my test case, both uncompressed)than in
> Hive, as hbase stores each KV pair for one column which causes the key to
> be repeated several times. But after I tried compress the Hbase table using
> LZO(now nearly the same as in hive uncompressed table), there's no
> performance gain for queries like select count(*) from xtable;
> Is there anyone working on this?Not sure whether I should put this post to
> Hive's mailing list but there seems to be no progress on issues like
> https://issues.apache.org/jira/browse/HIVE-1231
> 
> Regards,
> Bruce

Re: Issue in using Hbase-explorer!

[Michel Segel]

HBase explorer is a third party tool, which has a lot of limitations.  YMMV
I think if you're having issues, you need to bug the author(s).



Sent from a remote device. Please excuse any typos...

Mike Segel

On Dec 16, 2011, at 4:16 AM, neuron005  wrote:

> 
> Hiii There
> My second post(I think I have more problems :P)
> I was trying to use  http://hbaseexplorer.wordpress.com/hbaseexplorer/
> hbase-explorer 
> I set it up. But whenever I try to check scan it shows.
> --
> Grails Runtime Exception
> Error Details
> Error 500: org.apache.hadoop.hbase.MasterNotRunningException:
> java.io.IOException: Could not read quorum servers from zoo.cfg
> Servlet: grails
> URI: /grails/hbaseSource/scan.dispatch
> Exception Message: Could not read quorum servers from zoo.cfg
> Caused by: org.apache.hadoop.hbase.MasterNotRunningException:
> java.io.IOException: Could not read quorum servers from zoo.cfg
> Class: HbaseSourceController
> At Line: [228]
> Code Snippet:
> Stack Trace
> 
> org.codehaus.groovy.runtime.InvokerInvocationException:
> org.apache.hadoop.hbase.MasterNotRunningException: java.io.IOException:
> Could not read quorum servers from zoo.cfg
>at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
>at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
>at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
>at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
>at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
>at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
>at
> org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:646)
>at
> org.apache.catalina.core.ApplicationDispatcher.processRequest(ApplicationDispatcher.java:436)
>at
> org.apache.catalina.core.ApplicationDispatcher.doForward(ApplicationDispatcher.java:374)
>at
> org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispatcher.java:302)
>at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
>at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
>at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
>at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
>at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
>at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
>at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
>at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
>at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
>at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
>at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)
>at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
>at
> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:567)
>at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
>at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
>at
> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:849)
>at
> org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583)
>at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:454)
>at java.lang.Thread.run(Thread.java:679)
> Caused by: org.apache.hadoop.hbase.MasterNotRunningException:
> java.io.IOException: Could not read quorum servers from zoo.cfg
>at
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getMaster(HConnectionManager.java:306)
>at org.apache.hadoop.hbase.client.HBaseAdmin.(HBaseAdmin.java:72)
>at com.nnapz.hbaseexplorer.HBaseClient.listTables(HBaseClient.java:74)
>at HbaseService.tableList(HbaseService.groovy:56)
>at HbaseService$tableList.call(Unknown Source)
>at HbaseSourceController$_closure9.doCall(HbaseSourceController.groovy:228)
>at HbaseSourceController$_closure9.doCall(HbaseSourceController.groovy)
>... 29 more
> Caused by: java.io.IOException: Could not read quorum servers from zoo.cfg
>at
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper.(ZooKeeperWrapper.java:81)
>at
> org.apache.hadoop.hbase.client.HCon

Re: Hadoop and Hbase compatibility

[Michel Segel]

You may want to consider making a set of symbolic links. So you would link 
hadoop.jar to the correct Hadoop version (whichever flavor you run.) and the 
link Hadoop.jar in HBase to that hadoop.jar in $HADOOP_HOME. (/usr/lib/hadoop 
in Cloudera's release as an example)

Sent from a remote device. Please excuse any typos...

Mike Segel

On Dec 15, 2011, at 1:04 PM, Joey Echeverria  wrote:

> For posterity, Mohammad fixed his problem with the following:
> 
> "I copied the hadoop-core-0.20.205.0.jar and
> commons-configuration-1.6.jar from hadoop folder to the hbase/lib
> folder, and after that Hbase was working fine."
> 
> On Thu, Dec 15, 2011 at 11:30 AM, Joey Echeverria  wrote:
>> (-hdfs-u...@hadoop.apache.org, +user@hbase.apache.org)
>> 
>> What error message are you seeing when you try to start HBase?
>> 
>> -Joey
>> 
>> On Fri, Dec 9, 2011 at 12:14 PM, Mohammad Tariq  wrote:
>>> Hi Joey,
>>>  Thanks a lot for the response. Hadoop is working fine in psuedo
>>> distribtued mode. I am able to use Hbase in standalone mode properly.
>>> 
>>> Regards,
>>> Mohammad Tariq
>>> 
>>> 
>>> 
>>> On Fri, Dec 9, 2011 at 5:59 PM, Joey Echeverria  wrote:
 Those versions should work fine together. Did you get Hadoop
 configured for psuedo distribtued mode correctly, or are you having
 trouble with both?
 
 -Joey
 
 On Fri, Dec 9, 2011 at 4:57 AM, Mohammad Tariq  wrote:
> Is there any specific combination of Hadoop and Hbase in order to use
> Hbase in atleast pseudo distributed mode???I am trying with
> Hadoop-0.20.205.0 and Hbase-0.90.4 but I am not able to set a pseudo
> distributed Hbase cluster..I have tried everything specified in the
> Hbase book under pseudo distributed mode section, but all in vain.
> Need some proper guidance badly..Or do I need some patch in order to
> make it work???
> 
> Regards,
> Mohammad Tariq
 
 
 
 --
 Joseph Echeverria
 Cloudera, Inc.
 443.305.9434
>> 
>> 
>> 
>> --
>> Joseph Echeverria
>> Cloudera, Inc.
>> 443.305.9434
> 
> 
> 
> -- 
> Joseph Echeverria
> Cloudera, Inc.
> 443.305.9434
> 

Re: Dealing with single point of failure

[Michel Segel]

Sorry to join late...
SPoF is a real problem if your planning to serve data realtime from your 
cluster.
( Yes you can do this w HBase ...)

Then, regardless of data loss, you have to bring up the cluster.
Down time can be significant enough to kill your business, depending on your 
use case.
Sure there are ways to make the NN more fault tolerant, but then you increase 
the complexity of your solution and still have to worry about automatic 
failover. 

MapR did a nice little trick that I would expect to show up in some fashion in 
Apache some time down the road. 

( my bet is that someone will be clever enough to reverse engineer this. )

Sent from a remote device. Please excuse any typos...

Mike Segel

On Dec 14, 2011, at 1:28 AM, "M. C. Srivas"  wrote:

> On Sat, Oct 29, 2011 at 1:34 PM, lars hofhansl  wrote:
> 
>> This is more of "theoretical problem" really.
>> Yahoo and others claim they lost far more data due to human error than any
>> HDFS problems (including Namenode failures).
>> 
> 
> Actually it is not theoretical at all.
> 
> SPOF  !=  data-loss.
> 
> Data-loss can occur even if you don't have any SPOF's.  Vice versa, many
> SPOF systems do not have data-loss (eg, a single Netapp).
> 
> SPOF == lack of high-availability.
> 
> Which is indeed the case with HDFS, even at Y!  For example, when a cluster
> is upgraded it becomes unavailable.
> 
> @Mark:
> the Avatar-node is not for the faint-hearted. AFAIK, only FB runs it.
> Konstantin Shvachko and co at eBay have a much better NN-SPOF solution in
> 0.22 that was just released. I recommend you try that.
> 
> 
> 
> 
> 
> 
> 
> 
> 
>> You can prevent data loss by having the namenode write the metadata to
>> another machine (via NFS or DRBD or if you have a SAN).
>> You'll still have an outage while switching over to a different machine,
>> but at least you won't lose any data.
>> 
>> 
>> Facebook has a partial solution (Avatarnode) and the HSFS folks are
>> working on a solution (which like Avatarnode mainly involves keeping
>> a hot copy of the Namenode so that failover is "instantaneous" - 1 or 2
>> minutes at most).
>> 
>> 
>> - Original Message -
>> From: Mark 
>> To: user@hbase.apache.org
>> Cc:
>> Sent: Saturday, October 29, 2011 11:46 AM
>> Subject: Dealing with single point of failure
>> 
>> How does one deal with the fact that HBase has a single point of failure..
>> namely the namenode. What steps can be taken to eliminate and/or minimize
>> the impact of a namenode failure? What can a situation where reliability is
>> of utmost importance should one choose an alternative technology.. ie
>> Cassandra?
>> 
>> Thanks
>> 
>> 

Re: getting row info without data

[Michel Segel]

Doesn't sound like it... He mentions column names...
sounds like he would be better off writing to two tables. One that stores only 
the column name and one that stores the data in each column.


Sent from a remote device. Please excuse any typos...

Mike Segel

On Nov 28, 2011, at 11:54 AM, Stack  wrote:

> On Mon, Nov 28, 2011 at 8:54 AM, Mikael Sitruk  
> wrote:
>> Hi
>> 
>> I would like to know if it is possible to retrieve the columns name and not
>> the whole content of rows.
>> The reason for such request is that the columns store high volumn of data
>> (2K) each (and i store 900 columns per key).
>> Retrieving the whole row and not the "Description/Metadata" of the row is
>> in this case very expensive.
>> 
> 
> Will 
> http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/filter/KeyOnlyFilter.html
> work for you?
> St.Ack
> 

Re: Multiple tables vs big fat table

[Michel Segel]

Mark,

I think you've gotten a bit more of an explanation...

The reason I say 'It depends...' is that there are arguments for either design.
If your log events are going to be accessed independently by type... Meaning 
that you're going to process only a single type of an event at a time, then it 
makes sense to separate the data.  Note I'm talking about your primary access 
path. 

At the same time, it was pointed out that if you're not going to be accessing 
the log events one at a time, you may actually want a hybrid approach where you 
keep your index in HBase but store your event logs in a sequence file.  

And again, it all depends on what you want to do with the data. That's why you 
can't always say ... 'if y then do x...' 

There are other issues too. How will the data end up sitting in the table? Sure 
his is more of an issue of schema/key design, but it will also have an impact 
on your systems performance.

In terms of get() performance HBase scales linearly. In terms of scans, it 
doesn't. 

So there's a lot to think about... 



Sent from a remote device. Please excuse any typos...

Mike Segel

On Nov 20, 2011, at 7:36 PM, Amandeep Khurana  wrote:

> Mark,
> 
> This is an interesting discussion and like Michel said - the answer to your
> question depends on what you are trying to achieve. However, here are the
> points that I would think about:
> 
> What are the access patters of the various buckets of data that you want to
> put in HBase? For instance, would the SearchLog and PageViewLog tables be
> access together all the time? Would they be primarily scanned or just
> random look ups. What are the cache requirements? Are both going to be
> equally read and written? Ideally, you want to store data with separate
> access patterns in separate tables.
> 
> Then, what kind of schema are you looking at. When I say schema, I mean
> keys and column families. Now, if you concatenate the three tables you
> mentioned and let's say your keys are prefixed with the type of data:
> 
> S
> P
> L
> 
> you will be using some servers more than others for different parts of the
> data. In theory, that should not happen but in most practical scenarios
> when splitting happens, regions tend to stick together. There are ways to
> work around that as well.
> 
> Like Lars said, it's okay to have multiple tables. But you don't want to
> end up 100s of tables. You ideally want to optimize for the number of
> tables depending on the access patterns.
> 
> Again, this discussion will be kind of abstract without a specific example.
> :)
> 
> -ak
> 
> 
> On Fri, Nov 18, 2011 at 1:29 PM, Mark  wrote:
> 
>> Is it better to have many smaller tables are one larger table? For example
>> if we wanted to store user action logs we could do either of the following:
>> 
>> Multiple tables:
>> - SearchLog
>> - PageViewLog
>> - LoginLog
>> 
>> or
>> 
>> One table:
>> - ActionLog where the key could be a concatenation of the action type ie
>> (search, pageview, login)
>> 
>> Any ideas? Are there any performance considerations on having multiple
>> smaller tables?
>> 
>> Thanks
>> 
>> 

Re: Schema design question - Hot Key concerns

[Michel Segel]

Hi,

OK...
First a caveat... I haven't seen your initial normalized schema, so take what I 
say with a grain of salt...

The problem you are trying to solve is one which can be solved better on an 
RDBMS platform and does not fit well in a NoSQL space.
 
Your scalability issue would probably be better solved with a redesign of your 
existing schema. Over twenty years ago Hyatt hotels moved their reservation 
system from the main frame to an unix box.  Back then those unix boxes were 
huge but thanks to Moore's law the system could probably fit on to a single box 
we use for a datanode.

The point I'm trying to make is that your problem is very similar to the same 
sort of problems you would face when writing a hotel reservation system.

Your problem is an OLTP problem which requires things like ACID compliance, 
transactions and different isolation levels...

Sent from a remote device. Please excuse any typos...

Mike Segel

On Nov 18, 2011, at 4:57 PM, Suraj Varma  wrote:

> Thanks for your response.
> 
> As always ... it is currently on an RDBMS having scalability concerns
>  and hence a nosql is being evaluated for this. :)
> 
> In the quora link posted, Todd mentions that "if the updates are in
> the 100 updates/sec range on the hot row" it may still be ok. I guess,
> that was one of the questions I had as in how hot is "hot"? If an app
> expects 100 updates/sec to a row ... perhaps that is not considered
> hot with HBase?
> 
> The only reason I'm even considering this is because the updates are
> to MemStore ... so, in my mind this is basically concurrently updating
> a SortedMap of SortedMaps etc in memory ... so, how drastic is the
> locking is there really?
> 
> Also - given the sparse column updates (i.e. mutually exclusive), I
> was wondering if that reduces locking in some way.
> 
> Michael ... on your #2 ... why would rows be locked when people are
> querying? Wouldn't they be querying by a specific timestamp ... so
> they would basically see a row as of that timestamp?
> 
> On #1, yes - I think that's the complexity moving this into app.
> 
> One other design thought we had was to have it as a tall table and
> have a "BOOKED_QTY" column that we do live rollups to get a
> SEATS_BOOKED which can then be used in lieu of SEATS_AVAILABLE (if we
> know the TOTAL ... that doesn't change for a SHOW_ID)
> 
> With columnar db, column level rollups are _supposed_ to be fast ...
> but again ... question is "how fast". Is there a general benchmark on
> this? (i.e. similar to the 100 updates/sec sort of rough ball park
> figure).
> 
> Are metrics collected by OpenTSDB like applications generally
> aggregated via MR jobs? Or are there live rollups happening when user
> specifies some criteria? If it is live, what sort of rollup times (say
> n ms per million rows, sort of ball park ...) can one expect?
> 
> 

Re: Multiple tables vs big fat table

[Michel Segel]

Mark, 
Simple answer ... it depends... ;-)

Longer answer...
What's your use case? What's your access pattern? Is the type of data, in this 
case evenly distributed in terms of size? 



Sent from a remote device. Please excuse any typos...

Mike Segel

On Nov 18, 2011, at 3:29 PM, Mark  wrote:

> Is it better to have many smaller tables are one larger table? For example if 
> we wanted to store user action logs we could do either of the following:
> 
> Multiple tables:
> - SearchLog
> - PageViewLog
> - LoginLog
> 
> or
> 
> One table:
>  - ActionLog where the key could be a concatenation of the action type ie 
> (search, pageview, login)
> 
> Any ideas? Are there any performance considerations on having multiple 
> smaller tables?
> 
> Thanks
> 
> 

Re: Connection to two difference versions

[Michel Segel]

Short answer no, not from java. You have a class loader problem which may make 
it difficult to instantiate a connection and HBase objects in the second 
instance.

Longer answer... Let me think about it... 

Of course having said that... I'm sure someone already has done this...

Sent from a remote device. Please excuse any typos...

Mike Segel

On Nov 17, 2011, at 4:24 AM, Sebastian Bauer  wrote:

> Hi. Is there any way to connect to two difference hbase versions?
> I try copytable mapreduce and i got:
> 
> 11/11/17 11:19:28 WARN client.HConnectionManager$HConnectionImplementation: 
> RemoteException connecting to RS
> org.apache.hadoop.ipc.RemoteException: 
> org.apache.hadoop.io.VersionMismatchException
>at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:881)
>at 
> org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:149)
>at $Proxy7.getProtocolVersion(Unknown Source)
>at 
> org.apache.hadoop.hbase.ipc.WritableRpcEngine.getProxy(WritableRpcEngine.java:182)
>at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:295)
>at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:272)
>at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:324)
>at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:228)
>at 
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1197)
> 
> -- 
> 
> Pozdrawiam
> Sebastian Bauer
> -
> http://tikecik.pl
> 
> 

Re: region size/count per regionserver

[Michel Segel]

The funny thing about tuning... What works for one situation may not work well 
for others.
Using the old recommendation of never exceeding 1000 R per RS, keeping it low 
around 100-200 and monitoring tables and changing the REgion Size on a table by 
table basis we are doing OK. 
( of course there are other nasty bugs that kill us... But that's a different 
thread...)

The point is that you need to decide what makes sense for you and what trade 
offs you can live with...

Just my two cents...

Sent from a remote device. Please excuse any typos...

Mike Segel

On Nov 2, 2011, at 9:10 PM, lars hofhansl  wrote:

> Do we know what would need to change in HBase in order to be able to manage 
> more regions per regionserver?
> With 20 regions per server, one would need 300G regions to just utilize 6T of 
> drive space.
> 
> 
> To utilize a regionserver/datanode with 24T drive space the region size would 
> be an insane 1T.
> 
> -- Lars
> 
> 
> From: Nicolas Spiegelberg 
> To: "user@hbase.apache.org" 
> Cc: Karthik Ranganathan ; Kannan Muthukkaruppan 
> 
> Sent: Tuesday, November 1, 2011 3:57 PM
> Subject: Re: region size/count per regionserver
> 
> Simple answer
> -
> 20 regions/server & <2000 regions/cluster is a good rule of thumb if you
> can't profile your workload yet.  You really want to ensure that
> 
> 1) You need to limits the regions/cluster so the master can have a
> reasonable startup time & can handle all the region state transitions via
> ZK.  Most bigger companies are running 2,000 in production and achieve
> reasonable startup times (< 2 minutes for region assignment on cold
> start).  If you want to test the scalability of that algorithm beyond what
> other companies need, admin beware.
> 2) The more regions/server you have, the faster that recovery can happen
> after RS death because you can currently parallelize recovery on a
> region-granularity.  Too many regions/server and #1 starts to be a problem.
> 
> 
> 
> Complicated answer
> --
> More information is optimize this formula.  Additional considerations:
> 
> 1) Are you IO-bound or CPU-bound
> 2) What is your grid topology like
> 3) What is your network hardware like
> 4) How many disks (not just size)
> 5) What is the data locality between RegionServer & DataNode
> 
> In the Facebook case, we have 5 racks with 20 nodes each.  Servers in the
> rack are connected by 1G Eth to a switch with a 10G uplink.  We are
> network bound.  Our saturation point is mostly commonly on the top-of-rack
> switch.  With 20 regions/server, we can roughly parallelize our
> distributed log splitting within a single rack on RS death (although 2
> regions do split off-rack).  This minimizes top-of-rack traffic and
> optimized our recovery time.  Even if you are CPU-bound, log splitting
> (hence recovery time) is an IO-bound operation.  A lot of our work on
> region assignment is about maximizing data locality, even on RS death, so
> we avoid top-of-rack saturation.
> 
> 
> On 11/1/11 10:54 AM, "Sujee Maniyam"  wrote:
> 
>> HI all,
>> My HBase cluster is 10 nodes, each node has 12core ,   48G RAM, 24TB disk,
>> 10GEthernet.
>> My region size is 1GB.
>> 
>> Any guidelines on how many regions can a RS  handle comfortably?
>> I vaguely remember reading some where to have no more than 1000 regions /
>> server; that comes to 1TB / server.  Seems pretty low for the current
>> hardware config.
>> 
>> Any rules of thumb?  experiences?
>> 
>> thanks
>> Sujee
>> 
>> http://sujee.net
> 

Re: Region has been OPENING for too long

[Michel Segel]

What did you set your max region size?



Sent from a remote device. Please excuse any typos...

Mike Segel

On Oct 31, 2011, at 5:07 AM, Matthew Tovbin  wrote:

> Ted,  thanks for such a rapid response.
> 
> You're right, we use hbase 0.90.3 from cdh3u1.
> 
> So, I suppose I need to make bulk loading in smaller bulks then. Any other
> suggestions?
> 
> 
> Best regards,
>Matthew Tovbin =)
> 
>> 
>> 
>> I assume you're using HBase 0.90.x where HBASE-4015 isn't available.
>> 
 5. And so on, till some of Slaves fail with "java.net.SocketException:
>> Too many open files".
>> Do you have some monitoring setup so that you can know the number of open
>> file handles ?
>> 
>> Cheers
>> 
>> On Sun, Oct 30, 2011 at 7:21 AM, Matthew Tovbin <[EMAIL PROTECTED]> wrote:
>> 
>>> Hi guys,
>>> 
>>>  I've bulkloaded a solid amount of data (650GB, ~14000 files) into Hbase
>>> (1master + 3regions) and now enabling the table results the
>>> following behavior on the cluster:
>>> 
>>>  1. Master says that opening started  -
>>>   "org.apache.hadoop.hbase.master.AssignmentManager: Handling
>>>  transition=RS_ZK_REGION_OPENING, server=slave..."
>>>  2. Slaves report about opening files in progress -
>>>  "org.apache.hadoop.hbase.regionserver.Store: loaded hdfs://"
>>>  3. Then after ~10 mins the following error occurs on hmaster -
>>>   "org.apache.hadoop.hbase.master.AssignmentManager: Regions in
> transition
>>>  timed out / Region has been OPENING for too long, reassigning
> region=..."
>>>  4. More slaves report about opening files in progress -
>>>  "org.apache.hadoop.hbase.regionserver.Store: loaded hdfs://"
>>>  5. And so on, till some of Slaves fail with "java.net.SocketException:
>>>  Too many open files".
>>> 
>>> 
>>> What I've done already to solve the issue (which DID NOT help though):
>>> 
>>>  1. Set 'ulimit -n 65536' for hbase user
>>>  2. Set hbase.hbasemaster.maxregionopen=360 (1 hour) in
> hbase-site.xml
>>> 
>>> 
>>> What else can I try?!
>>> 
>>> 
>>> Best regards,
>>>   Matthew Tovbin =)
>>> 

Re: A requirement to change time of the Hbase cluster.

[Michel Segel]

Maybe I'm missing something...

The purpose of using an ntp server is that your machines all have the same 
time. Also you would sync your ntp server clock to one of the global ntp 
servers so that you have an accurate clock for your network...

You shouldn't have to restart your cluster unless your clocks are all way off...

Sent from a remote device. Please excuse any typos...

Mike Segel

On Oct 25, 2011, at 5:14 AM, Gaojinchao  wrote:

> Perhaps we should. add a choice of supporting incremental meta-data. All the 
> timestamp is incremental, These data do not rely on the system time.
> 
> -邮件原件-
> 发件人: Gaojinchao [mailto:gaojinc...@huawei.com] 
> 发送时间: 2011年10月25日 12:33
> 收件人: user@hbase.apache.org
> 主题: A requirement to change time of the Hbase cluster.
> 
> Hi all,
> We have a requirement to change time of the Hbase cluster.
> The scene is the cluster changes the ntp server(my customer may do this),
> We are ready to do this:
> 1. stop the cluster
> 2. change the ntp server
> 3. start the cluster.
> But the cluster may move to one ntp server which system is slower.
> we find the meta data can't update becuase newly added record is covered by 
> old record and the cluster don't run normal.
> I have a way to deal with this situation. before we update the meta data ,we 
> can get it firstly and then compare the timestamp with the system time.
> if system time is lower than timestamp, updating metadata can use as 
> timestamp +1.

Re: Persist to HBase with JPA using HBql-JDBC-Driver (Examples)?

[Michel Segel]

ok...
There are two ways to use HBase. One is as part of a map reduce program. The 
other is as a NoSQL data store. 

If you are writing a m/r job, you really don't want to create any DAO layer or 
even think about a JPA layer. Sorry but the access API is too simple to require 
any JPA and you're not writing to multiple data sources so you really don't 
want to do any abstraction...
Keep It Simple.

If you are writing an app to access HBase outside of a map reduce job, most of 
the same arguments apply. HBase access patterns are too simple to require any 
abstraction. 


Sorry, 
JMHO
YMMV, so take what I say with a grain of salt.

Sent from a remote device. Please excuse any typos...

Mike Segel

On Oct 14, 2011, at 3:54 PM, Christian Schäfer  wrote:

> Hi St.Ack
> 
> yes I'm currently using Data Nucleus...and thought I missed that there is a 
> JDBC-Driver
> 
> 
> 
> 
> 
> Von: Stack 
> An: user@hbase.apache.org; Christian Schäfer 
> Gesendet: 18:35 Freitag, 14.Oktober 2011 
> Betreff: Re: Persist to HBase with JPA using HBql-JDBC-Driver (Examples)?
> 
> On Fri, Oct 14, 2011 at 1:26 PM, Christian Schäfer  
> wrote:
>> just saw that there is a JDBC-Driver for HBase.
>> 
>> So I wonder if it's possible to persist to HBase via JPA using that 
>> JDBC-Driver?
>> 
>> For now I still saw in the examples that there are Annotations included in 
>> HBql what seems not so far away from JPA-Annotations.
>> 
>> JPA-HBql-. Examples (I didnt find any) would be welcome :-)
>> 
> 
> I think you might need to temper your expectation a little.  I think
> that jdbc driver is for hbql -- least thats what it understands.
> Then I'm not sure of the state of hbql.
> 
> The above seems a little dodgy to be building a jpa bridge upon?  If
> you google jpa and hbase there seem to be a few possibilities
> (datanucleus for one).
> 
> St.Ack

Re: HBase Hush Application

[Michel Segel]

In your localhost file... Where do you have your machine name... Just a thought.

Sent from a remote device. Please excuse any typos...

Mike Segel

On Oct 2, 2011, at 1:17 PM, Mark  wrote:

> Just the good ol default hosts file
> 
> ##
> # Host Database
> #
> # localhost is used to configure the loopback interface
> # when the system is booting.  Do not change this entry.
> ##
> 
> 127.0.0.1   localhost
> 255.255.255.255 broadcasthost
> ::1 localhost
> fe80::1%lo0 localhost
> 
> 
> 
> On 10/2/11 11:04 AM, Zijad Purkovic wrote:
>> Can you show your hosts file, particularly info on localhost. Of
>> course omit any sensitive info like public IP's or hostnames?
>> 
>> On Sun, Oct 2, 2011 at 7:19 PM, Mark  wrote:
>>> I am trying to run the HBase URL application:
>>> https://github.com/larsgeorge/hbase-book on my local machine in
>>> psuedo-distributed mode using the Cloudera  CDH3 but I keep receiving the
>>> following error:
>>> 
>>> ...
>>> INFO [main] (ZooKeeper.java:373) - Initiating client connection,
>>> connectString=localhost:2181 sessionTimeout=18 watcher=hconnection
>>> INFO [main-SendThread()] (ClientCnxn.java:1041) - Opening socket connection
>>> to server localhost/0:0:0:0:0:0:0:1:2181
>>> INFO [main-SendThread(localhost:2181)] (ClientCnxn.java:949) - Socket
>>> connection established to localhost/0:0:0:0:0:0:0:1:2181, initiating session
>>> INFO [main-SendThread(localhost:2181)] (ClientCnxn.java:738) - Session
>>> establishment complete on server localhost/0:0:0:0:0:0:0:1:2181, sessionid =
>>> 0x132c59cf114, negotiated timeout = 4
>>> Exception in thread "main" java.lang.StringIndexOutOfBoundsException: String
>>> index out of range: -1
>>>at java.lang.String.substring(String.java:1937)
>>>at org.apache.hadoop.hbase.ServerName.parseHostname(ServerName.java:81)
>>>at org.apache.hadoop.hbase.ServerName.(ServerName.java:63)
>>>at
>>> org.apache.hadoop.hbase.MasterAddressTracker.getMasterAddress(MasterAddressTracker.java:62)
>>>at
>>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getMaster(HConnectionManager.java:568)
>>>at org.apache.hadoop.hbase.client.HBaseAdmin.(HBaseAdmin.java:94)
>>>at
>>> com.hbasebook.hush.schema.SchemaManager.process(SchemaManager.java:126)
>>>at com.hbasebook.hush.HushMain.main(HushMain.java:57)
>>> 
>>> 
>>> 
>>> When I type JPS I see /HMaster/ however I do not see any mention of
>>> /Zookeeper/. Is this to be expected? Is the above error I am receiving due
>>> to a misconfiguration of zookeeper or is it something completely unrelated?
>>> Is there something wrong with my hostname? Any ideas why I am receiving this
>>> error
>>> 
>>> Any help would be greatly appreciated. Thanks
>>> 
>>> 
>>> 
>> 
>> 
> 

Re: Writing MR-Job: Something like OracleReducer, JDBCReducer ...

[Michel Segel]

;> 
>>>>>>>>> 
>>>>>>>>> re:  "Store aggregated data in Oracle. "
>>>>>>>>> 
>>>>>>>>> To me, that sounds a like the "read-summary" example with
>>>>>>>> JDBC-Oracle
>>>>>>>> in
>>>>>>>>> the reduce step.
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> On 9/16/11 2:58 PM, "Chris Tarnas"  wrote:
>>>>>>>>> 
>>>>>>>>>> If only I could make NY in Nov :)
>>>>>>>>>> 
>>>>>>>>>> We extract out large numbers of DNA sequence reads from HBase,
>>>>>>>> run
>>>>>>>> them
>>>>>>>>>> through M/R pipelines to analyze and aggregate and then we load
>>>>>>>> the
>>>>>>>>>> results back in. Definitely specialized usage, but I could see
>>>>>>>> other
>>>>>>>>>> perfectly valid uses for reducers with HBase.
>>>>>>>>>> 
>>>>>>>>>> -chris
>>>>>>>>>> 
>>>>>>>>>> On Sep 16, 2011, at 11:43 AM, Michael Segel wrote:
>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> Sonal,
>>>>>>>>>>> 
>>>>>>>>>>> You do realize that HBase is a "database", right? ;-)
>>>>>>>>>>> 
>>>>>>>>>>> So again, why do you need a reducer?  ;-)
>>>>>>>>>>> 
>>>>>>>>>>> Using your example...
>>>>>>>>>>> "Again, there will be many cases where one may want a reducer,
>>>>>>>> say
>>>>>>>>>>> trying to count the occurrence of words in a particular
>>>>>>>> column."
>>>>>>>>>>> 
>>>>>>>>>>> You can do this one of two ways...
>>>>>>>>>>> 1) Dynamic Counters in Hadoop.
>>>>>>>>>>> 2) Use a temp table and auto increment the value in a column
>>>>>>>> which
>>>>>>>>>>> contains the word count.  (Fat row where rowkey is doc_id and
>>>>>>>> column is
>>>>>>>>>>> word or rowkey is doc_id|word)
>>>>>>>>>>> 
>>>>>>>>>>> I'm sorry but if you go through all of your examples of why
>>>>>>>> you
>>>>>>>> would
>>>>>>>>>>> want to use a reducer, you end up finding out that writing to
>>>>>>>> an
>>>>>>>> HBase
>>>>>>>>>>> table would be faster than a reduce job.
>>>>>>>>>>> (Again we haven't done an exhaustive search, but in all of the
>>>>>>>> HBase
>>>>>>>>>>> jobs we've run... no reducers were necessary.)
>>>>>>>>>>> 
>>>>>>>>>>> The point I'm trying to make is that you want to avoid using a
>>>>>>>> reducer
>>>>>>>>>>> whenever possible and if you think about your problem... you
>>>>>>>> can
>>>>>>>>>>> probably come up with a solution that avoids the reducer...
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> HTH
>>>>>>>>>>> 
>>>>>>>>>>> -Mike
>>>>>>>>>>> PS. I haven't looked at *all* of the potential use cases of
>>>>>>>> HBase
>>>>>>>> which
>>>>>>>>>>> is why I don't want to say you'll never need a reducer. I will
>>>>>>>> say
>>>>>>>> that
>>>>>>>>>>> based on what we've done at my client's site, we try very hard
>>>>>>>> to
>