Re: Non-sensical? Compaction Metrics Hbase UI

2014-08-11 Thread Geovanie Marquez
thanks for your response. Here I provide an example since I was looking into it. Like you mentioned it can be done but it takes some effort. I see this in the regionserver logs: 4:21:32.107 PM INFO org.apache.hadoop.hbase.regionserver.HRegion Starting compaction on d in region ,\x10\x00\x00\x00\x

Re: Non-sensical? Compaction Metrics Hbase UI

2014-08-11 Thread Bryan Beaudreault
Unfortunately I don't know an easy way. But I think you could figure this out by combining logs and the above. If you look at the logs on the server compacting, it should show the number of files being compacted, the total size, and the output file name. Then watch for the .tmp file growing to a

Re: Non-sensical? Compaction Metrics Hbase UI

2014-08-11 Thread Geovanie Marquez
nice, just took a look at this path in my system. I see the growth but that would only apply to major_compactions where all files are picked up and compacted because minor compactions won't be trackable this way without knowing which files were selected by the compaction algorithm. is there a meth

Re: Non-sensical? Compaction Metrics Hbase UI

2014-08-11 Thread Bryan Beaudreault
I've also seen that on 0.94.x. The compaction progress is almost never accurate. I've taken to doing: hdfs -du /hbase///.tmp to track the new hfile size, expecting it to be approximately the same size as total hfiles pre-compaction. On Mon, Aug 11, 2014 at 3:11 PM, Geovanie Marquez < geovanie

Re: Non-sensical? Compaction Metrics Hbase UI

2014-08-11 Thread Geovanie Marquez
*Attribute Name* *Value* HBase Version 0.98.1-cdh5.1.0, rUnknown HBase Compiled Sat Jul 12 08:20:49 PDT 2014, jenkins Hadoop Version 2.3.0-cdh5.1.0, r8e266e052e423af592871e2dfe09d54c03f6a0e8 Hadoop Compiled 2014-07-12T13:49Z, jenkins On Mon, Aug 11, 2014 at 12:56 PM, Ted Yu wrote: > Which relea

Re: Non-sensical? Compaction Metrics Hbase UI

2014-08-11 Thread Ted Yu
Which release of hbase are you using ? Thanks On Mon, Aug 11, 2014 at 9:49 AM, Geovanie Marquez < geovanie.marq...@gmail.com> wrote: > What is the meaning of the progress value below? > > > > I see the information broken down below in the following HBase UI. > > http://region-server-ip:60030/rs

Non-sensical? Compaction Metrics Hbase UI

2014-08-11 Thread Geovanie Marquez
What is the meaning of the progress value below? I see the information broken down below in the following HBase UI. http://region-server-ip:60030/rs-status?filter=general#regionCompactStats Region Name Num. ,`\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00,1398130030457.2dddb2b