[ https://issues.apache.org/jira/browse/HBASE-14436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14946152#comment-14946152 ]
Hudson commented on HBASE-14436: -------------------------------- FAILURE: Integrated in HBase-1.3-IT #215 (See [https://builds.apache.org/job/HBase-1.3-IT/215/]) HBASE-14436 HTableDescriptor#addCoprocessor will always make (stack: rev ab33a65a1adae33025a1034568397d6d67bf476a) * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionCoprocessorHost.java > HTableDescriptor#addCoprocessor will always make RegionCoprocessorHost create > new Configuration > ----------------------------------------------------------------------------------------------- > > Key: HBASE-14436 > URL: https://issues.apache.org/jira/browse/HBASE-14436 > Project: HBase > Issue Type: Improvement > Components: Coprocessors > Affects Versions: 1.2.1 > Reporter: Jianwei Cui > Assignee: stack > Priority: Minor > Fix For: 2.0.0, 1.2.0, 1.3.0, 1.0.3, 1.1.3, 0.98.16 > > Attachments: HBASE-14436-trunk-v1.patch, HBASE-14436-trunk-v2.patch > > > HTableDescriptor#addCoprocessor will set the coprocessor value as following > format: > {code} > public HTableDescriptor addCoprocessor(String className, Path jarFilePath, > int priority, final Map<String, String> kvs) > throws IOException { > ... > String value = ((jarFilePath == null)? "" : jarFilePath.toString()) + > "|" + className + "|" + Integer.toString(priority) + "|" + > kvString.toString(); > ... > } > {code} > If the 'jarFilePath' is null, the 'value' will always has the format > '|className|priority|' even if 'kvs' is null, which means no extra arguments > for the coprocessor. Then, in the server side, > RegionCoprocessorHost#getTableCoprocessorAttrsFromSchema will load the table > coprocessors as: > {code} > static List<TableCoprocessorAttribute> > getTableCoprocessorAttrsFromSchema(Configuration conf, > HTableDescriptor htd) { > ... > try { > cfgSpec = matcher.group(4); // => cfgSpec will be '|' for the > format '|className|priority|' > } catch (IndexOutOfBoundsException ex) { > // ignore > } > Configuration ourConf; > if (cfgSpec != null) { // => cfgSpec will be '|' for the format > '|className|priority|' > ourConf = new Configuration(false); > HBaseConfiguration.merge(ourConf, conf); > } > ... > } > {code} > The 'cfgSpec' will be '|' for the coprocessor formatted as > '|className|priority|', so that always create a new Configuration. > In our production, there are a lot of tables having table-level coprocessors, > so that the region server will create new Configurations for each region of > the table, this will consume a certain number of memory when we have many > such regions. > To fix the problem, we can make the HTableDescriptor not append the '|' if no > extra arguments for the coprocessor, or check the 'cfgSpec' more strictly in > server side which could avoid creating new Configurations for existed such > regions after the regions reopened. Discussions and suggestions are welcomed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)