Re: distributed cluster proble

2012-03-27 Thread Roberto Alonso
Hello, I put it in my .bashrc and resource it, and the problem is still there. I haven't seen to many documents in the web about this HBASE_CONF_DIR, something about PIG, but I am not using it. I have 4 servers, after putting HBASE_CONF_DIR, it only starts the first node, in the others there is

questions about splits in regions

2012-03-27 Thread Roberto Alonso CIPF
Hello All, I have some doubts about hbase that hopefully you can help me. My architecture is the next: I have 4 servers(server_{1,2,3,4}) with 6GB Ram and 2 cores. I installed hadoop in all of them, this is the configuration: - server_1 is namenode, datanode and secondarynamenode, jobtracker -

Re: Region server crashes

2012-03-27 Thread Lior Schachter
Thanks for the detailed answer. we are running 0.90.2, and the problem resolved after running major compaction manually. It seems that the problem was with client request waiting in the queue (so I don't understand why major compaction solved it...). Anyhow, I will try to apply the

serialize hbase data

2012-03-27 Thread Rita
Hello, I was wondering if there is an easy way to serialize hbase data so I can store it on a Unix filesystem. Since the data is unstructured I was thinking of creating a XML file which would represent it for each key and value. Any thoughts or ideas about this? -- --- Get your facts first,

Re: serialize hbase data

2012-03-27 Thread Stack
On Tue, Mar 27, 2012 at 5:10 AM, Rita rmorgan...@gmail.com wrote: Hello, I was wondering if there is an easy way to serialize hbase data so I can store it on a Unix filesystem. Since the data is unstructured I was thinking of creating a XML file which would represent it for each key and

ArrayIndexOutOfBoundsException in 0.90.7-SNAPSHOT

2012-03-27 Thread Daniel Lamberger
Hello, We recently migrated to 0.90.7-SNAPSHOT, and are encountering the above exception, which seems to fail various HBase operations. How it came to be: * We upgraded from 0.90.4 to 0.90.7, however not all slaves were restarted, i.e. we ran slaves from different versions for a couple of days.

Re: ArrayIndexOutOfBoundsException in 0.90.7-SNAPSHOT

2012-03-27 Thread Ted Yu
Index 20 corresponds to RS_ZK_REGION_FAILED_OPEN which was added by: HBASE-5490 Move the enum RS_ZK_REGION_FAILED_OPEN to the last of the enum list in 0.90 EventHandler (Ram) As of now, is there any server that is still running 0.90.4 ? Such server(s) wouldn't be able to

Still Seeing Old Data After a Delete

2012-03-27 Thread Shawn Quinn
Hello, In a couple of situations we were noticing some odd problems with old data appearing in the application, and I finally found a reproducible scenario. Here's what we're seeing in one basic case: 1. Using a scan in hbase shell one of our column cells (both the column name and value are

Re: Still Seeing Old Data After a Delete

2012-03-27 Thread lars hofhansl
Hey Shawn, how exactly did you delete the column? There are three types of delete markers: family, column, version. Your observation would be consistent with having used a version delete marker, which just marks are a specific version (the latest by default) for delete. Check out the HBase

Re: Still Seeing Old Data After a Delete

2012-03-27 Thread Shawn Quinn
Hi Lars, Thanks for the quick reply! In this case we we're doing a column delete like so: Delete delete = new Delete(rowKey); delete.deleteColumn(Bytes.toBytes(thing), Bytes.toBytes(value)); table.delete(delete); However, your response caused me to notice

Re: Still Seeing Old Data After a Delete

2012-03-27 Thread yonghu
Hi Shwan, My hbase-version is 0.92.0. I have to mention that in recently I noticed that the delete semantics between shell and Java api are different. In shell, if you delete one version, it will mask the versions whose timestamps are older than that version, it means that scan will not return

Re: Still Seeing Old Data After a Delete

2012-03-27 Thread lars hofhansl
Yes, this is quite confusing. Delete.deleteColumn should really be called deleteVersion or deleteColumnVersion. I wonder whether we could make a change like this in the 0.96 singularity release. -- Lars From: Shawn Quinn squ...@moxiegroup.com To:

Re: Migrating Tables with primary key and foreign key from mysql to Hbae

2012-03-27 Thread Jean-Daniel Cryans
Hi, HBase is not a relational database, it doesn't have foreign keys or constraints. I'd suggest you familiarize yourself with HBase by reading the refence manual[1] or buying the book[2]. Regards, J-D 1. http://hbase.apache.org/book/book.html 2. http://www.amazon.com/dp/1449396100 On Tue,

Starting Abnormally After Shutting Down For Some Time

2012-03-27 Thread Bing Li
Dear all, I got a weird problem when programming on the pseudo-distributed mode of HBase/Hadoop. The HBase/Hadoop were installed correctly. It also ran well with my Java code. However, if after shutting down the server for some time, for example, four or five days, I noticed that HBase/Hadoop

RE: questions about splits in regions

2012-03-27 Thread Agarwal, Saurabh
1. It is good idea to manage the region splits manually. For best practice, read http://hbase.apache.org/book.html - 2.8.2.7. Managed Splitting 2. default hbase mapreduce splitter create a map-tasks for each of the regions, read more details at http://hbase.apache.org/book.html#splitter

Re: Starting Abnormally After Shutting Down For Some Time

2012-03-27 Thread Jean-Daniel Cryans
Hi Bing, Two questions: - Can you look at the master log and see what's preventing the master from starting? - Did you change dfs.data.dir and dfs.name.dir in hdfs-site.xml? By default it writes to /tmp which can get cleaned up. J-D On Tue, Mar 27, 2012 at 12:52 PM, Bing Li lbl...@gmail.com

apache.hadoop.ipc.HBaseServer: (responseTooSlow)

2012-03-27 Thread Sindy
Hi, I'm using ycsb to evaluate Hbase. It worked well when loading data, but when do the all update operations ,the exception come out. Could you give any advices? Thanks a lot Hadoop 1.0.1 HBase 0.90.2 2012-03-27 22:04:06,605 WARN org.apache.hadoop.ipc.HBaseServer: (responseTooSlow):

Re: Starting Abnormally After Shutting Down For Some Time

2012-03-27 Thread Manish Bhoge
It says you have not started the hbase master. Once you restarted the system have you confirmed whether all hadoop daemons are running? sudo jps If you are using CDH package then you can automatically start the hadoop daemons on boot using reconfig package. Sent from my BlackBerry, pls excuse

Re: Migrating Tables with primary key and foreign key from mysql to Hbae

2012-03-27 Thread Neetu Ojha
Hi Jean, Thanks A lot for reply, I got your point about HBASE , let me give a little clear picture of what I am desiring from HBase , If any body is willing to migrate ones application on Hadoop and at teh same time migrate Mysql database to Hbase to get the advantage of the Hbase in that case

Re: Migrating Tables with primary key and foreign key from mysql to Hbae

2012-03-27 Thread Rohit Kelkar
Neetu, In my opinion it is a bad idea to copy paste a normalized schema from an RDBMS to a nosql database like hbase. Hbase encourages denormalization. Hbase does not support indexing out of the box like mysql/postgres/etc. So your retrieval time would be affected as the data size grows. For