We have hbase.hstore.compaction.kv.max to setBatch on compaction and default value is 10, which means we will write each 10 cells to writer. I think it can prevent using too much heap while compacting?
Thanks, Phil 2016-12-06 15:29 GMT+08:00 Guanghao Zhang <[email protected]>: > Now the scan context has size limit. And scan can break between cells. This > should help for this. What is the version of your cluster? > > 2016-12-06 13:35 GMT+08:00 聪聪 <[email protected]>: > > > I am glad to receive your reply！How can I find a big row quickly? If not > > ,when we proceed major compact,the regionserver dose not work all the > time . > > > > > > > > > > > > > > > > ------------------ 原始邮件 ------------------ > > 发件人: "Guanghao Zhang";<[email protected]>; > > 发送时间: 2016年12月6日(星期二) 中午12:13 > > 收件人: "user"<[email protected]>; > > > > 主题: Re: How to limit a single row being filled with millions of columus? > > > > > > > > There are a config hbase.table.max.rowsize but only used for user > get/scan. > > It will throw RowTooBigException when you scan a big row with millions of > > columns. But it doesn't work for compact. If i am not wrong, there are > not > > a way to prevent a single row being filled with millions of columns. > > > > 2016-12-06 11:52 GMT+08:00 聪聪 <[email protected]>: > > > > > Recently, I have a problem that confused me a long time. The problem is > > > that as we all know in HBase,there are millions of columns in a single > > row. > > > Full gc will happen when region proceeds major compact and it results > in > > > regionserver and hbase not working. Is there any good way to prevent a > > > single row being put(wrote) columns overmuch? > > > Hope to you soon! > > >
