[jira] [Updated] (HBASE-15686) unable to dynamically load a table coprocessor if it belongs in org.apache.hadoop
[ https://issues.apache.org/jira/browse/HBASE-15686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-15686: --- Attachment: 15686.v3.txt Renamed config as suggested > unable to dynamically load a table coprocessor if it belongs in > org.apache.hadoop > - > > Key: HBASE-15686 > URL: https://issues.apache.org/jira/browse/HBASE-15686 > Project: HBase > Issue Type: Bug > Components: Coprocessors >Affects Versions: 1.0.1 >Reporter: Sangjin Lee >Assignee: Ted Yu > Attachments: 15686.v2.txt, 15686.v3.txt, 15686.wip > > > As part of Hadoop's Timeline Service v.2 (YARN-2928), we're adding a table > coprocessor (YARN-4062). However, we're finding that the coprocessor cannot > be loaded dynamically. A relevant snippet for the exception: > {noformat} > java.io.IOException: Class > org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor > cannot be loaded > at > org.apache.hadoop.hbase.master.HMaster.sanityCheckTableDescriptor(HMaster.java:1329) > at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1269) > at > org.apache.hadoop.hbase.master.MasterRpcServices.createTable(MasterRpcServices.java:398) > at > org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:42436) > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2031) > at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107) > at > org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130) > at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107) > at java.lang.Thread.run(Thread.java:745) > Caused by: java.io.IOException: Class > org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor > cannot be loaded > at > org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.testTableCoprocessorAttrs(RegionCoprocessorHost.java:324) > at > org.apache.hadoop.hbase.master.HMaster.checkClassLoading(HMaster.java:1483) > at > org.apache.hadoop.hbase.master.HMaster.sanityCheckTableDescriptor(HMaster.java:1327) > ... 8 more > Caused by: java.lang.ClassNotFoundException: > org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor > at java.net.URLClassLoader$1.run(URLClassLoader.java:366) > at java.net.URLClassLoader$1.run(URLClassLoader.java:355) > at java.security.AccessController.doPrivileged(Native Method) > at java.net.URLClassLoader.findClass(URLClassLoader.java:354) > at java.lang.ClassLoader.loadClass(ClassLoader.java:425) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) > at java.lang.ClassLoader.loadClass(ClassLoader.java:358) > at > org.apache.hadoop.hbase.util.CoprocessorClassLoader.loadClass(CoprocessorClassLoader.java:275) > at > org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.testTableCoprocessorAttrs(RegionCoprocessorHost.java:322) > ... 10 more > {noformat} > We tracked it down to the fact that {{CoprocessorClassLoader}} regarding all > hadoop classes as exempt from loading from the coprocessor jar. Since our > coprocessor sits in the coprocessor jar, and yet the loading of this class is > delegated to the parent which does not have this jar, the classloading fails. > What would be nice is the ability to exclude certain classes from the exempt > classes so that they can be loaded via table coprocessor classloader. See > hadoop's {{ApplicationClassLoader}} for a similar feature. > Is there any other way to load this coprocessor at the table scope? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15686) unable to dynamically load a table coprocessor if it belongs in org.apache.hadoop
[ https://issues.apache.org/jira/browse/HBASE-15686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-15686: --- Attachment: 15686.v2.txt Patch v2 allows table coprocessor to specify excluded class prefixes (semi-colon delimited) through attribute. Please see if this formation is acceptable. Open to suggestions on attribute naming. > unable to dynamically load a table coprocessor if it belongs in > org.apache.hadoop > - > > Key: HBASE-15686 > URL: https://issues.apache.org/jira/browse/HBASE-15686 > Project: HBase > Issue Type: Bug > Components: Coprocessors >Affects Versions: 1.0.1 >Reporter: Sangjin Lee > Attachments: 15686.v2.txt, 15686.wip > > > As part of Hadoop's Timeline Service v.2 (YARN-2928), we're adding a table > coprocessor (YARN-4062). However, we're finding that the coprocessor cannot > be loaded dynamically. A relevant snippet for the exception: > {noformat} > java.io.IOException: Class > org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor > cannot be loaded > at > org.apache.hadoop.hbase.master.HMaster.sanityCheckTableDescriptor(HMaster.java:1329) > at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1269) > at > org.apache.hadoop.hbase.master.MasterRpcServices.createTable(MasterRpcServices.java:398) > at > org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:42436) > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2031) > at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107) > at > org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130) > at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107) > at java.lang.Thread.run(Thread.java:745) > Caused by: java.io.IOException: Class > org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor > cannot be loaded > at > org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.testTableCoprocessorAttrs(RegionCoprocessorHost.java:324) > at > org.apache.hadoop.hbase.master.HMaster.checkClassLoading(HMaster.java:1483) > at > org.apache.hadoop.hbase.master.HMaster.sanityCheckTableDescriptor(HMaster.java:1327) > ... 8 more > Caused by: java.lang.ClassNotFoundException: > org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor > at java.net.URLClassLoader$1.run(URLClassLoader.java:366) > at java.net.URLClassLoader$1.run(URLClassLoader.java:355) > at java.security.AccessController.doPrivileged(Native Method) > at java.net.URLClassLoader.findClass(URLClassLoader.java:354) > at java.lang.ClassLoader.loadClass(ClassLoader.java:425) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) > at java.lang.ClassLoader.loadClass(ClassLoader.java:358) > at > org.apache.hadoop.hbase.util.CoprocessorClassLoader.loadClass(CoprocessorClassLoader.java:275) > at > org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.testTableCoprocessorAttrs(RegionCoprocessorHost.java:322) > ... 10 more > {noformat} > We tracked it down to the fact that {{CoprocessorClassLoader}} regarding all > hadoop classes as exempt from loading from the coprocessor jar. Since our > coprocessor sits in the coprocessor jar, and yet the loading of this class is > delegated to the parent which does not have this jar, the classloading fails. > What would be nice is the ability to exclude certain classes from the exempt > classes so that they can be loaded via table coprocessor classloader. See > hadoop's {{ApplicationClassLoader}} for a similar feature. > Is there any other way to load this coprocessor at the table scope? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15686) unable to dynamically load a table coprocessor if it belongs in org.apache.hadoop
[ https://issues.apache.org/jira/browse/HBASE-15686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-15686: --- Attachment: 15686.wip I started coding something and then saw Jerry's comment. Parking the WIP patch here until we decide what to do next. > unable to dynamically load a table coprocessor if it belongs in > org.apache.hadoop > - > > Key: HBASE-15686 > URL: https://issues.apache.org/jira/browse/HBASE-15686 > Project: HBase > Issue Type: Bug > Components: Coprocessors >Affects Versions: 1.0.1 >Reporter: Sangjin Lee > Attachments: 15686.wip > > > As part of Hadoop's Timeline Service v.2 (YARN-2928), we're adding a table > coprocessor (YARN-4062). However, we're finding that the coprocessor cannot > be loaded dynamically. A relevant snippet for the exception: > {noformat} > java.io.IOException: Class > org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor > cannot be loaded > at > org.apache.hadoop.hbase.master.HMaster.sanityCheckTableDescriptor(HMaster.java:1329) > at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1269) > at > org.apache.hadoop.hbase.master.MasterRpcServices.createTable(MasterRpcServices.java:398) > at > org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:42436) > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2031) > at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107) > at > org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130) > at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107) > at java.lang.Thread.run(Thread.java:745) > Caused by: java.io.IOException: Class > org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor > cannot be loaded > at > org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.testTableCoprocessorAttrs(RegionCoprocessorHost.java:324) > at > org.apache.hadoop.hbase.master.HMaster.checkClassLoading(HMaster.java:1483) > at > org.apache.hadoop.hbase.master.HMaster.sanityCheckTableDescriptor(HMaster.java:1327) > ... 8 more > Caused by: java.lang.ClassNotFoundException: > org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor > at java.net.URLClassLoader$1.run(URLClassLoader.java:366) > at java.net.URLClassLoader$1.run(URLClassLoader.java:355) > at java.security.AccessController.doPrivileged(Native Method) > at java.net.URLClassLoader.findClass(URLClassLoader.java:354) > at java.lang.ClassLoader.loadClass(ClassLoader.java:425) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) > at java.lang.ClassLoader.loadClass(ClassLoader.java:358) > at > org.apache.hadoop.hbase.util.CoprocessorClassLoader.loadClass(CoprocessorClassLoader.java:275) > at > org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.testTableCoprocessorAttrs(RegionCoprocessorHost.java:322) > ... 10 more > {noformat} > We tracked it down to the fact that {{CoprocessorClassLoader}} regarding all > hadoop classes as exempt from loading from the coprocessor jar. Since our > coprocessor sits in the coprocessor jar, and yet the loading of this class is > delegated to the parent which does not have this jar, the classloading fails. > What would be nice is the ability to exclude certain classes from the exempt > classes so that they can be loaded via table coprocessor classloader. See > hadoop's {{ApplicationClassLoader}} for a similar feature. > Is there any other way to load this coprocessor at the table scope? -- This message was sent by Atlassian JIRA (v6.3.4#6332)