Thanks a lot.... Jean & Bharath It really helping me to get clear and deeper view on how hbase perform its required operation...
Is that delete operation for old hfile happened by schedule? On Tue, Feb 25, 2014 at 8:17 PM, Bharath Vissapragada <bhara...@cloudera.com > wrote: > Hi, > > I believe CatalogJanitor is for a different purpose (cleaning meta post > splits). He should be looking at o.a.h.h.master.cleaner.* ! Anway files are > deleted. > > - Bharath > > > On Tue, Feb 25, 2014 at 5:43 PM, Jean-Marc Spaggiari < > jean-m...@spaggiari.org> wrote: > > > Hi Upendra, > > > > Olds files are moved by HBase later. Take a look at > > org.apache.hadoop.hbase.master.CatalogJanitor > > > > JM > > > > > > 2014-02-25 6:25 GMT-05:00 Upendra Yadav <upendra1...@gmail.com>: > > > > > Thanks... > > > > > > > > > On Tue, Feb 25, 2014 at 2:18 PM, Bharath Vissapragada < > > > bhara...@cloudera.com > > > > wrote: > > > > > > > In short, it needn't "download" files, that metadata is stored and > > > exposed > > > > in the API of a StoreFile and regionserver just calls that method! > > > > > > > > > > > > On Tue, Feb 25, 2014 at 2:16 PM, Bharath Vissapragada < > > > > bhara...@cloudera.com > > > > > wrote: > > > > > > > > > Not sure what you are asking but if you want to know the size of a > > > store > > > > > file, you can use this method, > > > > > > > > > > > > > > > > > > > > > > > > > https://github.com/apache/hbase/blob/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java#L1440 > > > > > > > > > > One such usage is here, > > > > > > > > > > > > > > > > > > > > > > > > > https://github.com/apache/hbase/blob/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/RatioBasedCompactionPolicy.java#L292 > > > > > > > > > > It gets the sizes of all the store files, I believe thats what you > > are > > > > > asking! Please correct me if I'm wrong, > > > > > > > > > > > > > > > On Tue, Feb 25, 2014 at 12:37 PM, Upendra Yadav < > > upendra1...@gmail.com > > > > >wrote: > > > > > > > > > >> Is hbase call delete request for old hfile(storeFile) file if > > > > minor/major > > > > >> compaction happen for them and generate new hfile and store it in > > > hdfs. > > > > >> > > > > >> for minor compaction how it will get size of every hfile. lets we > > have > > > > 20 > > > > >> no. of hfiles for a CF and minor compaction wants to perform. Will > > it > > > > >> download all hfile and decide compaction have to do or not. Or it > > have > > > > >> in-memory hfile's size information for all hfile. > > > > >> > > > > > > > > > > > > > > > > > > > > -- > > > > > Bharath Vissapragada > > > > > <http://www.cloudera.com> > > > > > > > > > > > > > > > > > > > > > -- > > > > Bharath Vissapragada > > > > <http://www.cloudera.com> > > > > > > > > > > > > > -- > Bharath Vissapragada > <http://www.cloudera.com> >
