Fwd: Bulk Import & Data Locality

2012-07-18 Thread Alex Baranau
he wrong way around. HBase has no clue as to where > >> blocks > >> >>> reside, nor does it know that the file system in fact uses separate > >> blocks. > >> >>> HBase stores files, HDFS does the block magic underneath the hood, > and > >&g

Re: Bulk Import & Data Locality

2012-07-18 Thread Alex Baranau
he wrong way around. HBase has no clue as to where > >> blocks > >> >>> reside, nor does it know that the file system in fact uses separate > >> blocks. > >> >>> HBase stores files, HDFS does the block magic underneath the hood, > and > >&g

Re: Bulk Import & Data Locality

2012-07-18 Thread Ben Kim
HBase stores files, HDFS does the block magic underneath the hood, and >> >>> transparent to HBase. >> >>> >> >>>> Also is it correct to say that if i set smaller data block size data >> >>>> locality gets worse, and if data block size gets

RES: Bulk Import & Data Locality

2012-07-18 Thread Cristofer Weber
2012 12:46 Para: hbase-u...@hadoop.apache.org; mapreduce-u...@hadoop.apache.org; hdfs-u...@hadoop.apache.org Assunto: Bulk Import & Data Locality Hello, As far as I understand Bulk Import functionality will not take into account the Data Locality question. MR job will create number of red

RES: Bulk Import & Data Locality

2012-07-18 Thread Cristofer Weber
ex Baranau [mailto:alex.barano...@gmail.com] Enviada em: quarta-feira, 18 de julho de 2012 12:46 Para: hbase-u...@hadoop.apache.org; mapreduce-u...@hadoop.apache.org; hdfs-u...@hadoop.apache.org Assunto: Bulk Import & Data Locality Hello, As far as I understand Bulk Import functionality will not take into acc

Bulk Import & Data Locality

2012-07-18 Thread Alex Baranau
Hello, As far as I understand Bulk Import functionality will not take into account the Data Locality question. MR job will create number of reducer tasks same as regions to write into, but it will not "advice" on which nodes to run these tasks. In that case Reducer task which writes HFiles of some