Re: M/R vs hbase problem in production

2011-08-14 Thread Lior Schachter
and in the gc.log of the region server we get CMS failures that cause full gc (that fails to free memory): 11867.254: [Full GC 11867.254: [CMS: 3712638K->3712638K(3712640K), 4.7779250 secs] 4032614K->4032392K(4057664K), [CMS Perm : 20062K->19883K(33548K)] icms_dc=100 , 4.7780440 secs] [Times: user

M/R vs hbase problem in production

2011-08-14 Thread Lior Schachter
Hi, cluster details: hbase 0.90.2. 10 machines. 1GB switch. use-case M/R job that inserts about 10 million rows to hbase in the reducer, followed by M/R that works with hdfs files. When the first job maps finish the second job maps starts and region server crushes. please note, that when running