Re: base running JVM6 and application server running JVM7

2012-06-19 Thread Dave Revell
You're likely to get a better response if you post your error messages and example code that reproduces the problem. :) Best, Dave On Tue, Jun 19, 2012 at 12:27 AM, Ben Cuthbert wrote: > All > > We have our application server running in Java 7 and hbase started and > running on Java 6 using the

Re: memstoreSizeMB=0

2012-06-13 Thread Dave Revell
Here's a good starting point: http://hbase.apache.org/book.html#store.memstore , I think you misunderstand what a memstore is. Good luck! -Dave On Wed, Jun 13, 2012 at 10:50 AM, ashdamle wrote: > > Hello. This is my first post here. I've setup hbase with 2 column families > 2GB heap size with

Re: Occasional regionserver crashes following socket errors writing to HDFS

2012-05-25 Thread Dave Revell
Have you verified that your nodes are not swapping? This has caused serious issues for many people, including me. Swapping can occur even if you have lots of available memory, for complicated reasons. Best, Dave On Thu, May 24, 2012 at 4:39 PM, Stack wrote: > On Thu, May 24, 2012 at 4:15 AM, E

Re: improve performance of a MapReduce job with HBase input

2012-05-25 Thread Dave Revell
Here's what I do: Scan scan = new Scan(...) scan.setCaching(5000); scan.setWhatever(...); TableMapReduceUtil.initTableMapperJob(tablename, scan, mapClass, mapOutKeyClass, mapOutValueClass, job); Does that help? -Dave On Fri, May 25, 2012 at 11:03 AM, Ey-Chih chow wrote: >

Re: Restrictions during compactions

2012-05-23 Thread Dave Revell
On Wed, May 23, 2012 at 6:15 AM, Takahiko Kawasaki < takahiko.kawas...@jibemobile.jp> wrote: > Hello, > > I'm a newbie and wondering whether or not there is any restriction during > HBase minor/major compactions. I read the online document but could not > find any explicit mention about restrictio

Re: HBase and MapReduce

2012-05-23 Thread Dave Revell
> > 1. HBase guarantees data locality of store files and Regionserver only if > it stays up for long. If there are too many region movements or the server > has been recycled recently, there is a high probability that store file > blocks are not local to the region server. But the getSplits comman

Re: EndPoint Coprocessor could be dealocked?

2012-05-16 Thread Dave Revell
Many people will probably try to use coprocessors as a way of implementing app logic on top of HBase without the headaches of writing a daemon. Sometimes client-side approaches are inadvisable; for example, there may be several client languages/runtimes and the app logic should not be reimplemented

Re: Splits and MapReduce

2012-05-15 Thread Dave Revell
On May 15, 2012, at 7:10 AM, Leon Mergen wrote: > Hello all, > > We are currently orienting on HBase as a possible way to store our log data > in a structured way, and I want to verify a few things I was not able to > find online. Specifically, what we are trying to achieve: > > * be able to quic

Re: MR job for creating splits

2012-05-14 Thread Dave Revell
Re: your question #1, you won't be able to pass information from mappers to reducers by using static variables. Since map tasks run in different JVM instances than reduce tasks, the value of the static variable will never be sent from the mapper JVM to the reducer JVM. It might work in standalone

Re: Manager Zookeeper Client Connections

2012-05-11 Thread Dave Revell
Hi Narenda, There are some shared static data structures inside the HBase client that are keyed on Configuration objects. Do you reuse the same Configuration object everytime you instantiate an HTable? By reusing the same Configuration objects you may cause ZooKeeper connections to be reused. I'm

Re: Occasional regionserver crashes following socket errors writing to HDFS

2012-05-10 Thread Dave Revell
Many mapreduce algorithms require a reduce phase (e.g. sorting)' > > Ok. So tell me why you would want to sort your input in to HBase and if > that's really a good thing? > Oops!... :-) > > > > > > > On May 10, 2012, at 12:31 PM, Dave Revell wrote: > &

Re: Occasional regionserver crashes following socket errors writing to HDFS

2012-05-10 Thread Dave Revell
This "you don't need a reducer" conversation is distracting from the real problem and is false. Many mapreduce algorithms require a reduce phase (e.g. sorting). The fact that the output is written to HBase or somewhere else is irrelevant. -Dave On Thu, May 10, 2012 at 6:26 AM, Michael Segel wrot

Re: Storing extremely large size file

2012-04-17 Thread Dave Revell
+1 Jack :) On Tue, Apr 17, 2012 at 11:38 AM, Stack wrote: > On Tue, Apr 17, 2012 at 11:18 AM, Dave Revell > wrote: > > I think this is a popular topic that might deserve a section in The Book. > > > > By "this topic" I mean storing big binary chunks. > &g

Re: Storing extremely large size file

2012-04-17 Thread Dave Revell
I think this is a popular topic that might deserve a section in The Book. By "this topic" I mean storing big binary chunks. -Dave On Tue, Apr 17, 2012 at 11:07 AM, kim young ill wrote: > i plan to move some data from relational-db to hbase, most of them are > binary with some hundreds KBs, woul

Re: merge regions?

2011-09-09 Thread Dave Revell
Bam! http://ofps.oreilly.com/titles/9781449396107/performance.html , search for "merging regions" Zamo! https://issues.apache.org/jira/browse/HBASE-1621 Cheers, Dave On Fri, Sep 9, 2011 at 9:54 AM, Geoff Hendrey wrote: > Hi - > > > > I think we're in a situation where we have too many regions (