[jira] [Commented] (ACCUMULO-708) Modify ClassLoader to support different applications / multi-tenancy
[ https://issues.apache.org/jira/browse/ACCUMULO-708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13615801#comment-13615801 ] Dave Marion commented on ACCUMULO-708: -- does it make sense to move the two open subtasks for this feature to the next release (not 1.5.0) and close this issue. The feature is in the 1.5.0 codebase. The two open tickets are not critical to 1.5.0. Modify ClassLoader to support different applications / multi-tenancy Key: ACCUMULO-708 URL: https://issues.apache.org/jira/browse/ACCUMULO-708 Project: Accumulo Issue Type: Improvement Components: start Reporter: Dave Marion Assignee: Dave Marion Labels: classloader Fix For: 1.5.0 Attachments: ACCUMULO-708-1.patch, ACCUMULO-708-2.patch, ACCUMULO-708-3.patch, HelloWorld.jar Original Estimate: 24h Time Spent: 33h Remaining Estimate: 0h I'd like to expand the current classloader to support loading classes from HDFS and different application contexts. I'll be modifying the ticket as the idea matures. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ACCUMULO-708) Modify ClassLoader to support different applications / multi-tenancy
[ https://issues.apache.org/jira/browse/ACCUMULO-708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13615824#comment-13615824 ] Eric Newton commented on ACCUMULO-708: -- Yes, please move them to 1.6.0 or unassign them based on your own willingness to work on it for 1.6.0. :-) Modify ClassLoader to support different applications / multi-tenancy Key: ACCUMULO-708 URL: https://issues.apache.org/jira/browse/ACCUMULO-708 Project: Accumulo Issue Type: Improvement Components: start Reporter: Dave Marion Assignee: Dave Marion Labels: classloader Fix For: 1.5.0 Attachments: ACCUMULO-708-1.patch, ACCUMULO-708-2.patch, ACCUMULO-708-3.patch, HelloWorld.jar Original Estimate: 24h Time Spent: 33h Remaining Estimate: 0h I'd like to expand the current classloader to support loading classes from HDFS and different application contexts. I'll be modifying the ticket as the idea matures. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ACCUMULO-708) Modify ClassLoader to support different applications / multi-tenancy
[ https://issues.apache.org/jira/browse/ACCUMULO-708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13549769#comment-13549769 ] Hudson commented on ACCUMULO-708: - Integrated in Accumulo-Trunk #624 (See [https://builds.apache.org/job/Accumulo-Trunk/624/]) ACCUMULO-708 made per table classpaths work with mini accumulo cluster and added a test (Revision 1431474) Result = SUCCESS kturner : Files : * /accumulo/trunk/start/src/main/java/org/apache/accumulo/start/classloader/vfs/AccumuloVFSClassLoader.java * /accumulo/trunk/test/src/test/java/org/apache/accumulo/test/MiniAccumuloClusterTest.java * /accumulo/trunk/test/src/test/resources/FooFilter.jar Modify ClassLoader to support different applications / multi-tenancy Key: ACCUMULO-708 URL: https://issues.apache.org/jira/browse/ACCUMULO-708 Project: Accumulo Issue Type: Improvement Components: start Reporter: Dave Marion Assignee: Dave Marion Labels: classloader Fix For: 1.5.0 Attachments: ACCUMULO-708-1.patch, ACCUMULO-708-2.patch, ACCUMULO-708-3.patch, HelloWorld.jar Original Estimate: 24h Time Spent: 33h Remaining Estimate: 0h I'd like to expand the current classloader to support loading classes from HDFS and different application contexts. I'll be modifying the ticket as the idea matures. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ACCUMULO-708) Modify ClassLoader to support different applications / multi-tenancy
[ https://issues.apache.org/jira/browse/ACCUMULO-708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13546371#comment-13546371 ] Hudson commented on ACCUMULO-708: - Integrated in Accumulo-Trunk #613 (See [https://builds.apache.org/job/Accumulo-Trunk/613/]) ACCUMULO-708 get functional tests working again (Revision 1430013) ACCUMULO-708 get functional tests working again (Revision 1430012) Result = SUCCESS ecn : Files : * /accumulo/trunk/start/src/main/java/org/apache/accumulo/start/classloader/AccumuloClassLoader.java ecn : Files : * /accumulo/trunk/test/system/auto/TestUtils.py * /accumulo/trunk/test/system/auto/run.py Modify ClassLoader to support different applications / multi-tenancy Key: ACCUMULO-708 URL: https://issues.apache.org/jira/browse/ACCUMULO-708 Project: Accumulo Issue Type: Improvement Components: start Reporter: Dave Marion Assignee: Dave Marion Labels: classloader Fix For: 1.5.0 Attachments: ACCUMULO-708-1.patch, ACCUMULO-708-2.patch, ACCUMULO-708-3.patch, HelloWorld.jar Original Estimate: 24h Time Spent: 33h Remaining Estimate: 0h I'd like to expand the current classloader to support loading classes from HDFS and different application contexts. I'll be modifying the ticket as the idea matures. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ACCUMULO-708) Modify ClassLoader to support different applications / multi-tenancy
[ https://issues.apache.org/jira/browse/ACCUMULO-708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13543242#comment-13543242 ] Hudson commented on ACCUMULO-708: - Integrated in Accumulo-Trunk #603 (See [https://builds.apache.org/job/Accumulo-Trunk/603/]) ACCUMULO-708 ACCUMULO-866 ACCUMULO-869 ACCUMULO-870 many changes to accumulo-start (Revision 1428556) Result = SUCCESS kturner : Files : * /accumulo/trunk/bin/accumulo * /accumulo/trunk/core/src/main/java/org/apache/accumulo/core/conf/Property.java * /accumulo/trunk/core/src/main/java/org/apache/accumulo/core/master/thrift/TabletServerStatus.java * /accumulo/trunk/pom.xml * /accumulo/trunk/server/src/main/java/org/apache/accumulo/server/tabletserver/TabletServer.java * /accumulo/trunk/start/pom.xml * /accumulo/trunk/start/src/main/java/org/apache/accumulo/start/Main.java * /accumulo/trunk/start/src/main/java/org/apache/accumulo/start/classloader/AccumuloClassLoader.java * /accumulo/trunk/start/src/main/java/org/apache/accumulo/start/classloader/AccumuloFilesystemAlterationMonitor.java * /accumulo/trunk/start/src/main/java/org/apache/accumulo/start/classloader/vfs/AccumuloReloadingVFSClassLoader.java * /accumulo/trunk/start/src/main/java/org/apache/accumulo/start/classloader/vfs/AccumuloVFSClassLoader.java * /accumulo/trunk/start/src/main/java/org/apache/accumulo/start/classloader/vfs/ContextManager.java * /accumulo/trunk/start/src/main/java/org/apache/accumulo/start/classloader/vfs/providers * /accumulo/trunk/start/src/main/java/org/apache/accumulo/start/classloader/vfs/providers/HdfsFileObject.java * /accumulo/trunk/start/src/main/java/org/apache/accumulo/start/classloader/vfs/providers/HdfsFileSystemConfigBuilder.java * /accumulo/trunk/start/src/main/java/org/apache/accumulo/start/classloader/vfs/providers/HdfsReadOnlyFileContentInfoFactory.java * /accumulo/trunk/start/src/main/java/org/apache/accumulo/start/classloader/vfs/providers/HdfsReadOnlyRandomAccessContent.java * /accumulo/trunk/start/src/main/java/org/apache/accumulo/start/classloader/vfs/providers/ReadOnlyHdfsFileProvider.java * /accumulo/trunk/start/src/main/java/org/apache/accumulo/start/classloader/vfs/providers/ReadOnlyHdfsFileSystem.java * /accumulo/trunk/start/src/main/java/org/apache/commons/vfs2 * /accumulo/trunk/start/src/test/java/org/apache/accumulo/start/Test.java * /accumulo/trunk/start/src/test/java/org/apache/accumulo/start/classloader/vfs/AccumuloReloadingVFSClassLoaderTest.java * /accumulo/trunk/start/src/test/java/org/apache/accumulo/start/classloader/vfs/AccumuloVFSClassLoaderTest.java * /accumulo/trunk/start/src/test/java/org/apache/accumulo/start/classloader/vfs/ContextManagerTest.java * /accumulo/trunk/start/src/test/java/org/apache/accumulo/start/classloader/vfs/providers * /accumulo/trunk/start/src/test/java/org/apache/accumulo/start/classloader/vfs/providers/ReadOnlyHdfsFileProviderTest.java * /accumulo/trunk/start/src/test/java/org/apache/accumulo/start/classloader/vfs/providers/VfsClassLoaderTest.java * /accumulo/trunk/start/src/test/java/org/apache/accumulo/test/AccumuloDFSBase.java * /accumulo/trunk/start/src/test/java/org/apache/commons/vfs2 * /accumulo/trunk/start/src/test/resources/application1 * /accumulo/trunk/start/src/test/resources/default/conf/accumulo-site.xml * /accumulo/trunk/start/src/test/resources/disabled/conf/accumulo-site.xml Modify ClassLoader to support different applications / multi-tenancy Key: ACCUMULO-708 URL: https://issues.apache.org/jira/browse/ACCUMULO-708 Project: Accumulo Issue Type: Improvement Components: start Reporter: Dave Marion Assignee: Dave Marion Labels: classloader Fix For: 1.5.0 Attachments: ACCUMULO-708-1.patch, ACCUMULO-708-2.patch, ACCUMULO-708-3.patch, HelloWorld.jar Original Estimate: 24h Time Spent: 33h Remaining Estimate: 0h I'd like to expand the current classloader to support loading classes from HDFS and different application contexts. I'll be modifying the ticket as the idea matures. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ACCUMULO-708) Modify ClassLoader to support different applications / multi-tenancy
[ https://issues.apache.org/jira/browse/ACCUMULO-708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13543356#comment-13543356 ] Hudson commented on ACCUMULO-708: - Integrated in Accumulo-Trunk #605 (See [https://builds.apache.org/job/Accumulo-Trunk/605/]) ACCUMULO-708 added warning if nothing is added for a classpath entry and removed some screwy code I recently added (Revision 1428622) Result = SUCCESS kturner : Files : * /accumulo/trunk/start/src/main/java/org/apache/accumulo/start/classloader/AccumuloClassLoader.java Modify ClassLoader to support different applications / multi-tenancy Key: ACCUMULO-708 URL: https://issues.apache.org/jira/browse/ACCUMULO-708 Project: Accumulo Issue Type: Improvement Components: start Reporter: Dave Marion Assignee: Dave Marion Labels: classloader Fix For: 1.5.0 Attachments: ACCUMULO-708-1.patch, ACCUMULO-708-2.patch, ACCUMULO-708-3.patch, HelloWorld.jar Original Estimate: 24h Time Spent: 33h Remaining Estimate: 0h I'd like to expand the current classloader to support loading classes from HDFS and different application contexts. I'll be modifying the ticket as the idea matures. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ACCUMULO-708) Modify ClassLoader to support different applications / multi-tenancy
[ https://issues.apache.org/jira/browse/ACCUMULO-708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13502356#comment-13502356 ] Dave Marion commented on ACCUMULO-708: -- I think I'll have to set up the AccumuloVFSClassLoader in the failing tests using PowerMock like I did in my tests. Modify ClassLoader to support different applications / multi-tenancy Key: ACCUMULO-708 URL: https://issues.apache.org/jira/browse/ACCUMULO-708 Project: Accumulo Issue Type: Improvement Components: start Reporter: Dave Marion Assignee: Dave Marion Labels: classloader Fix For: 1.5.0 Attachments: ACCUMULO-708-1.patch, ACCUMULO-708-2.patch, ACCUMULO-708-3.patch, HelloWorld.jar Original Estimate: 24h Time Spent: 33h Remaining Estimate: 0h I'd like to expand the current classloader to support loading classes from HDFS and different application contexts. I'll be modifying the ticket as the idea matures. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ACCUMULO-708) Modify ClassLoader to support different applications / multi-tenancy
[ https://issues.apache.org/jira/browse/ACCUMULO-708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13502395#comment-13502395 ] Keith Turner commented on ACCUMULO-708: --- bq. I don't think that works. I checked out a clean copy of TRUNK, applied the patch, and uncommented the ACCUMULO_HOME system env setting in the pom.xml file. A couple of tests fail looking for the conf/accumulo-site.xml file. The conf/accumulo-site.xml file doesn't exist on a clean checkout. Good catch. If ACCUMULO_HOME is only needed by a unit test for the old classloader, and we are going to get rid of the old class loader, then maybe you could just drop that unit test before checking the code in. Just need to make sure that what that old unit test does is also done by the new unit test. Modify ClassLoader to support different applications / multi-tenancy Key: ACCUMULO-708 URL: https://issues.apache.org/jira/browse/ACCUMULO-708 Project: Accumulo Issue Type: Improvement Components: start Reporter: Dave Marion Assignee: Dave Marion Labels: classloader Fix For: 1.5.0 Attachments: ACCUMULO-708-1.patch, ACCUMULO-708-2.patch, ACCUMULO-708-3.patch, HelloWorld.jar Original Estimate: 24h Time Spent: 33h Remaining Estimate: 0h I'd like to expand the current classloader to support loading classes from HDFS and different application contexts. I'll be modifying the ticket as the idea matures. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ACCUMULO-708) Modify ClassLoader to support different applications / multi-tenancy
[ https://issues.apache.org/jira/browse/ACCUMULO-708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13502447#comment-13502447 ] Dave Marion commented on ACCUMULO-708: -- The above works and I believe it should only affect tests. I can put up another patch if you want. Modify ClassLoader to support different applications / multi-tenancy Key: ACCUMULO-708 URL: https://issues.apache.org/jira/browse/ACCUMULO-708 Project: Accumulo Issue Type: Improvement Components: start Reporter: Dave Marion Assignee: Dave Marion Labels: classloader Fix For: 1.5.0 Attachments: ACCUMULO-708-1.patch, ACCUMULO-708-2.patch, ACCUMULO-708-3.patch, HelloWorld.jar Original Estimate: 24h Time Spent: 33h Remaining Estimate: 0h I'd like to expand the current classloader to support loading classes from HDFS and different application contexts. I'll be modifying the ticket as the idea matures. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ACCUMULO-708) Modify ClassLoader to support different applications / multi-tenancy
[ https://issues.apache.org/jira/browse/ACCUMULO-708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13502531#comment-13502531 ] Hudson commented on ACCUMULO-708: - Integrated in Accumulo-Trunk #562 (See [https://builds.apache.org/job/Accumulo-Trunk/562/]) ACCUMULO-708 initial implementation of VFS class loader (Revision 1412398) Result = SUCCESS dlmarion : Files : * /accumulo/trunk/bin/accumulo * /accumulo/trunk/conf/examples/vfs-classloader * /accumulo/trunk/conf/examples/vfs-classloader/accumulo-site.xml * /accumulo/trunk/core/src/main/java/org/apache/accumulo/core/client/mock/MockInstanceOperations.java * /accumulo/trunk/core/src/main/java/org/apache/accumulo/core/conf/Property.java * /accumulo/trunk/core/src/main/java/org/apache/accumulo/core/file/BloomFilterLayer.java * /accumulo/trunk/core/src/main/java/org/apache/accumulo/core/iterators/AggregatingIterator.java * /accumulo/trunk/core/src/main/java/org/apache/accumulo/core/iterators/IteratorUtil.java * /accumulo/trunk/core/src/main/java/org/apache/accumulo/core/iterators/TypedValueCombiner.java * /accumulo/trunk/core/src/main/java/org/apache/accumulo/core/iterators/conf/ColumnToClassMapping.java * /accumulo/trunk/core/src/main/java/org/apache/accumulo/core/util/shell/commands/ClasspathCommand.java * /accumulo/trunk/core/src/main/java/org/apache/accumulo/core/util/shell/commands/ScanCommand.java * /accumulo/trunk/core/src/main/java/org/apache/accumulo/core/util/shell/commands/SetIterCommand.java * /accumulo/trunk/core/src/main/java/org/apache/accumulo/core/util/shell/commands/ShellPluginConfigurationCommand.java * /accumulo/trunk/core/src/test/resources/disabled * /accumulo/trunk/core/src/test/resources/disabled/conf * /accumulo/trunk/core/src/test/resources/disabled/conf/accumulo-site.xml * /accumulo/trunk/pom.xml * /accumulo/trunk/server/src/main/java/org/apache/accumulo/server/client/ClientServiceHandler.java * /accumulo/trunk/server/src/main/java/org/apache/accumulo/server/constraints/ConstraintLoader.java * /accumulo/trunk/server/src/main/java/org/apache/accumulo/server/master/Master.java * /accumulo/trunk/server/src/main/java/org/apache/accumulo/server/master/balancer/TableLoadBalancer.java * /accumulo/trunk/server/src/main/java/org/apache/accumulo/server/tabletserver/TabletServerResourceManager.java * /accumulo/trunk/server/src/main/java/org/apache/accumulo/server/test/functional/FunctionalTest.java * /accumulo/trunk/start/pom.xml * /accumulo/trunk/start/src/main/java/org/apache/accumulo/start/Main.java * /accumulo/trunk/start/src/main/java/org/apache/accumulo/start/classloader/vfs * /accumulo/trunk/start/src/main/java/org/apache/accumulo/start/classloader/vfs/AccumuloContextClassLoader.java * /accumulo/trunk/start/src/main/java/org/apache/accumulo/start/classloader/vfs/AccumuloReloadingVFSClassLoader.java * /accumulo/trunk/start/src/main/java/org/apache/accumulo/start/classloader/vfs/AccumuloVFSClassLoader.java * /accumulo/trunk/start/src/main/java/org/apache/commons * /accumulo/trunk/start/src/main/java/org/apache/commons/vfs2 * /accumulo/trunk/start/src/main/java/org/apache/commons/vfs2/provider * /accumulo/trunk/start/src/main/java/org/apache/commons/vfs2/provider/HdfsFileObject.java * /accumulo/trunk/start/src/main/java/org/apache/commons/vfs2/provider/HdfsFileSystemConfigBuilder.java * /accumulo/trunk/start/src/main/java/org/apache/commons/vfs2/provider/HdfsReadOnlyFileContentInfoFactory.java * /accumulo/trunk/start/src/main/java/org/apache/commons/vfs2/provider/HdfsReadOnlyRandomAccessContent.java * /accumulo/trunk/start/src/main/java/org/apache/commons/vfs2/provider/ReadOnlyHdfsFileProvider.java * /accumulo/trunk/start/src/main/java/org/apache/commons/vfs2/provider/ReadOnlyHdfsFileSystem.java * /accumulo/trunk/start/src/test/java/org/apache/accumulo/start/classloader * /accumulo/trunk/start/src/test/java/org/apache/accumulo/start/classloader/vfs * /accumulo/trunk/start/src/test/java/org/apache/accumulo/start/classloader/vfs/AccumuloContextClassLoaderTest.java * /accumulo/trunk/start/src/test/java/org/apache/accumulo/start/classloader/vfs/AccumuloReloadingVFSClassLoaderTest.java * /accumulo/trunk/start/src/test/java/org/apache/accumulo/start/classloader/vfs/AccumuloVFSClassLoaderTest.java * /accumulo/trunk/start/src/test/java/org/apache/accumulo/test * /accumulo/trunk/start/src/test/java/org/apache/accumulo/test/AccumuloDFSBase.java * /accumulo/trunk/start/src/test/java/org/apache/commons * /accumulo/trunk/start/src/test/java/org/apache/commons/vfs2 * /accumulo/trunk/start/src/test/java/org/apache/commons/vfs2/provider * /accumulo/trunk/start/src/test/java/org/apache/commons/vfs2/provider/ReadOnlyHdfsFileProviderTest.java * /accumulo/trunk/start/src/test/java/org/apache/commons/vfs2/provider/VfsClassLoaderTest.java * /accumulo/trunk/start/src/test/resources/HelloWorld.jar *
[jira] [Commented] (ACCUMULO-708) Modify ClassLoader to support different applications / multi-tenancy
[ https://issues.apache.org/jira/browse/ACCUMULO-708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13497667#comment-13497667 ] Keith Turner commented on ACCUMULO-708: --- Whats dfs.datanode.data.dir.perm being set to and what is your umask? Modify ClassLoader to support different applications / multi-tenancy Key: ACCUMULO-708 URL: https://issues.apache.org/jira/browse/ACCUMULO-708 Project: Accumulo Issue Type: Improvement Components: start Reporter: Dave Marion Assignee: Dave Marion Labels: classloader Fix For: 1.5.0 Attachments: ACCUMULO-708-1.patch Original Estimate: 24h Time Spent: 33h Remaining Estimate: 0h I'd like to expand the current classloader to support loading classes from HDFS and different application contexts. I'll be modifying the ticket as the idea matures. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ACCUMULO-708) Modify ClassLoader to support different applications / multi-tenancy
[ https://issues.apache.org/jira/browse/ACCUMULO-708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13497686#comment-13497686 ] Dave Marion commented on ACCUMULO-708: -- umask on cmd prompt yields: 0002 umask in code above is 022 newPermission in 755. Modify ClassLoader to support different applications / multi-tenancy Key: ACCUMULO-708 URL: https://issues.apache.org/jira/browse/ACCUMULO-708 Project: Accumulo Issue Type: Improvement Components: start Reporter: Dave Marion Assignee: Dave Marion Labels: classloader Fix For: 1.5.0 Attachments: ACCUMULO-708-1.patch Original Estimate: 24h Time Spent: 33h Remaining Estimate: 0h I'd like to expand the current classloader to support loading classes from HDFS and different application contexts. I'll be modifying the ticket as the idea matures. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ACCUMULO-708) Modify ClassLoader to support different applications / multi-tenancy
[ https://issues.apache.org/jira/browse/ACCUMULO-708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13496772#comment-13496772 ] Keith Turner commented on ACCUMULO-708: --- Still do not have the unit test running. It may be a configuration issue on my workstation, but its not the umask issue, still investigating. A few more comments on the code. * In AccumuloVFSClassLoader and AccumuloClassLoader the methods replaceEnvVars(), addUrl(), findAccumuloURLs(), etc. look very similar. Are these methods just copies? If not, what is the essence of the diffs? * Seems like unit test AccumuloContextClassLoaderTest would benefit from using two seprate jars. The same jar is loaded into two different context. Seems like it would be better to load two different jars with different classes. Then verify that each context contains only the expected classes and nothing else. * AccumuloReloadingVFSClassLoaderTest would also benefit from another jar. Test are needed for a new class coming into existence, a new version of an existing class coming into existence, and an existing class going out of existence. Moving forward on this we need to start thinking about having a table prop for configuring per table classloaders. If you would like help I can work on that after we get this checked in, just let me know. I was thinking of a scenario where we have the following two classloader contexts : * system class loader context : contains accumulo server and core jars and all deps in HDFS * table foo class loader context : contains user iterator jars in HDFS It seems like the two context above are siblings. Is this correct? Therefore the foo class loader will not have accumulo-core jars in itself or its parent class loaders. Will this cause problems when loading an iterator from the foo classloader? Modify ClassLoader to support different applications / multi-tenancy Key: ACCUMULO-708 URL: https://issues.apache.org/jira/browse/ACCUMULO-708 Project: Accumulo Issue Type: Improvement Components: start Reporter: Dave Marion Assignee: Dave Marion Labels: classloader Fix For: 1.5.0 Attachments: ACCUMULO-708-1.patch Original Estimate: 24h Time Spent: 33h Remaining Estimate: 0h I'd like to expand the current classloader to support loading classes from HDFS and different application contexts. I'll be modifying the ticket as the idea matures. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ACCUMULO-708) Modify ClassLoader to support different applications / multi-tenancy
[ https://issues.apache.org/jira/browse/ACCUMULO-708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13496781#comment-13496781 ] Josh Elser commented on ACCUMULO-708: - Dave, Keith, I did run into the same not-helpful Exception Keith posted above and traced it back to the same issue Dave linked to. After I realized AccumuloDFSBase.java:25 was sending the logging to /dev/null, I saw the actual error message, changed the default umask to 022 (755) as my system is configured and then was able to successfully run the tests via the command line. Modify ClassLoader to support different applications / multi-tenancy Key: ACCUMULO-708 URL: https://issues.apache.org/jira/browse/ACCUMULO-708 Project: Accumulo Issue Type: Improvement Components: start Reporter: Dave Marion Assignee: Dave Marion Labels: classloader Fix For: 1.5.0 Attachments: ACCUMULO-708-1.patch Original Estimate: 24h Time Spent: 33h Remaining Estimate: 0h I'd like to expand the current classloader to support loading classes from HDFS and different application contexts. I'll be modifying the ticket as the idea matures. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ACCUMULO-708) Modify ClassLoader to support different applications / multi-tenancy
[
https://issues.apache.org/jira/browse/ACCUMULO-708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13496790#comment-13496790
]
Keith Turner commented on ACCUMULO-708:
---
bq. I did run into the same not-helpful Exception Keith posted above and traced
it back to the same issue Dave linked to. After I realized
AccumuloDFSBase.java:25 was sending the logging to /dev/null
I think my umask is correct
{noformat}
$ umask
0022
{noformat}
AccumuloDFSBase.java line 25 is an import statement. Did you mean line 40 or
line 49?
Modify ClassLoader to support different applications / multi-tenancy
Key: ACCUMULO-708
URL: https://issues.apache.org/jira/browse/ACCUMULO-708
Project: Accumulo
Issue Type: Improvement
Components: start
Reporter: Dave Marion
Assignee: Dave Marion
Labels: classloader
Fix For: 1.5.0
Attachments: ACCUMULO-708-1.patch
Original Estimate: 24h
Time Spent: 33h
Remaining Estimate: 0h
I'd like to expand the current classloader to support loading classes from
HDFS and different application contexts. I'll be modifying the ticket as the
idea matures.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ACCUMULO-708) Modify ClassLoader to support different applications / multi-tenancy
[
https://issues.apache.org/jira/browse/ACCUMULO-708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13494455#comment-13494455
]
Keith Turner commented on ACCUMULO-708:
---
I applied the patch and have been unable to get the unit test to run. Keep
seeing something like the following
{noformat}
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.hdfs.MiniDFSCluster.startDataNodes(MiniDFSCluster.java:422)
at org.apache.hadoop.hdfs.MiniDFSCluster.init(MiniDFSCluster.java:280)
at
org.apache.accumulo.test.AccumuloDFSBase.clinit(AccumuloDFSBase.java:66)
... 21 more
{noformat}
bq. Where should this be documented? User guide, readme, etc?
Maybe add something to the users guide and add an example that shows off what
can be done.
Modify ClassLoader to support different applications / multi-tenancy
Key: ACCUMULO-708
URL: https://issues.apache.org/jira/browse/ACCUMULO-708
Project: Accumulo
Issue Type: Improvement
Components: start
Reporter: Dave Marion
Assignee: Dave Marion
Labels: classloader
Fix For: 1.5.0
Attachments: ACCUMULO-708-1.patch
Original Estimate: 24h
Time Spent: 33h
Remaining Estimate: 0h
I'd like to expand the current classloader to support loading classes from
HDFS and different application contexts. I'll be modifying the ticket as the
idea matures.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ACCUMULO-708) Modify ClassLoader to support different applications / multi-tenancy
[ https://issues.apache.org/jira/browse/ACCUMULO-708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13494491#comment-13494491 ] Dave Marion commented on ACCUMULO-708: -- I updated my copy and now the tests are failing for me also with a different error than what you are getting. Fun. Modify ClassLoader to support different applications / multi-tenancy Key: ACCUMULO-708 URL: https://issues.apache.org/jira/browse/ACCUMULO-708 Project: Accumulo Issue Type: Improvement Components: start Reporter: Dave Marion Assignee: Dave Marion Labels: classloader Fix For: 1.5.0 Attachments: ACCUMULO-708-1.patch Original Estimate: 24h Time Spent: 33h Remaining Estimate: 0h I'd like to expand the current classloader to support loading classes from HDFS and different application contexts. I'll be modifying the ticket as the idea matures. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ACCUMULO-708) Modify ClassLoader to support different applications / multi-tenancy
[ https://issues.apache.org/jira/browse/ACCUMULO-708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13486217#comment-13486217 ] Dave Marion commented on ACCUMULO-708: -- I contributed the HDFS provider to the Commons VFS project (VFS-442). Once accepted and released we should be able to remove this from our codebase. Modify ClassLoader to support different applications / multi-tenancy Key: ACCUMULO-708 URL: https://issues.apache.org/jira/browse/ACCUMULO-708 Project: Accumulo Issue Type: Improvement Components: start Reporter: Dave Marion Assignee: Dave Marion Labels: classloader Fix For: 1.5.0 Attachments: ACCUMULO-708-1.patch Original Estimate: 24h Time Spent: 33h Remaining Estimate: 0h I'd like to expand the current classloader to support loading classes from HDFS and different application contexts. I'll be modifying the ticket as the idea matures. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ACCUMULO-708) Modify ClassLoader to support different applications / multi-tenancy
[
https://issues.apache.org/jira/browse/ACCUMULO-708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13486226#comment-13486226
]
Josh Elser commented on ACCUMULO-708:
-
bq. I used /accumulo/classpath as an example, it made sense to me
Cool, I'm making the assumption that this will be useful to a large number of
people, and thus thinking forward to how users would want to use this. Using
`instance.dfs.dir`/{classpath,lib,whatever} would allow multiple separate
instances to be run over the same HDFS instance which could be very useful.
That should also guarantee you a unique HDFS directory.
Modify ClassLoader to support different applications / multi-tenancy
Key: ACCUMULO-708
URL: https://issues.apache.org/jira/browse/ACCUMULO-708
Project: Accumulo
Issue Type: Improvement
Components: start
Reporter: Dave Marion
Assignee: Dave Marion
Labels: classloader
Fix For: 1.5.0
Attachments: ACCUMULO-708-1.patch
Original Estimate: 24h
Time Spent: 33h
Remaining Estimate: 0h
I'd like to expand the current classloader to support loading classes from
HDFS and different application contexts. I'll be modifying the ticket as the
idea matures.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ACCUMULO-708) Modify ClassLoader to support different applications / multi-tenancy
[
https://issues.apache.org/jira/browse/ACCUMULO-708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13485755#comment-13485755
]
Dave Marion commented on ACCUMULO-708:
--
Steps I used to test locally:
{noformat}
mvn clean package -P assemble
tar zxf distro -C $ACCUMULO_HOME
hadoop fs -rmr /accumulo/classpath
hadoop fs -mkdir /accumulo/classpath
hadoop fs -moveFromLocal $ACCUMULO_HOME/lib/* /accumulo/classpath
hadoop fs -copyToLocal /accumulo/classpath/commons-vfs2-2.0.jar
$ACCUMULO_HOME/lib/.
hadoop fs -rmr /accumulo/classpath/commons-vfs2-2.0.jar
hadoop fs -copyToLocal /accumulo/classpath/accumulo-start-1.5.0-SNAPSHOT.jar
$ACCUMULO_HOME/lib/.
hadoop fs -rmr /accumulo/classpath/accumulo-start-1.5.0-SNAPSHOT.jar
#
# You will need to copy the following jar out of HDFS also if you plan to run
the old classloader.
#
hadoop fs -copyToLocal /accumulo/classpath/commons-jci-fam-1.0.jar
$ACCUMULO_HOME/lib/.
hadoop fs -rmr /accumulo/classpath/commons-jci-fam-1.0.jar
cp $ACCUMULO_HOME/conf/examples/512MB/native-standalone/* $ACCUMULO_HOME/conf/.
$ACCUMULO_HOME/bin/accumulo classpath
# Add to accumulo-site.xml
!-- VFS ClassLoader Settings --
property
nameclassloader.vfs.enabled/name
valuetrue/value
descriptionset to true to enable the vfs classloader. If not
enabled (default=false), then the old classloader is used/description
/property
$ACCUMULO_HOME/bin/accumulo classpath
#Then, add the default context classpath
property
nameclassloader.vfs.context.classpath.system/name
valuehdfs://localhost:8020/accumulo/classpath/value
descriptionlocation of the jars for the default (system)
context/description
/property
$ACCUMULO_HOME/bin/accumulo classpath
{noformat}
Modify ClassLoader to support different applications / multi-tenancy
Key: ACCUMULO-708
URL: https://issues.apache.org/jira/browse/ACCUMULO-708
Project: Accumulo
Issue Type: Improvement
Components: start
Reporter: Dave Marion
Assignee: Dave Marion
Labels: classloader
Fix For: 1.5.0
Attachments: ACCUMULO-708-1.patch
Original Estimate: 24h
Time Spent: 33h
Remaining Estimate: 0h
I'd like to expand the current classloader to support loading classes from
HDFS and different application contexts. I'll be modifying the ticket as the
idea matures.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ACCUMULO-708) Modify ClassLoader to support different applications / multi-tenancy
[ https://issues.apache.org/jira/browse/ACCUMULO-708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13485808#comment-13485808 ] Dave Marion commented on ACCUMULO-708: -- Thanks Josh. I used /accumulo/classpath as an example, it made sense to me. In reality though, it could be any location as long as it can be read by the OS user. I did not modify the assembly; by default 'classloader.vfs.enabled' is false and in this case the AccumuloClassLoader will be used (not the new AccumuloVFSClassLoader). In this default case, the jars will need to remain where they are now. I figure that this new classloader will need to be tested thoroughly (I expect a few issues will arise); so I think it makes sense to keep it experimental for now. Assuming all of the tests pass, Accumulo will still need to be modified to take advantage of the different contexts. Modify ClassLoader to support different applications / multi-tenancy Key: ACCUMULO-708 URL: https://issues.apache.org/jira/browse/ACCUMULO-708 Project: Accumulo Issue Type: Improvement Components: start Reporter: Dave Marion Assignee: Dave Marion Labels: classloader Fix For: 1.5.0 Attachments: ACCUMULO-708-1.patch Original Estimate: 24h Time Spent: 33h Remaining Estimate: 0h I'd like to expand the current classloader to support loading classes from HDFS and different application contexts. I'll be modifying the ticket as the idea matures. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ACCUMULO-708) Modify ClassLoader to support different applications / multi-tenancy
[ https://issues.apache.org/jira/browse/ACCUMULO-708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13478544#comment-13478544 ] Dave Marion commented on ACCUMULO-708: -- For those interested, I am done with unit tests. Will begin working on integration with Accumulo and local testing. Code is up at https://github.com/dlmarion/ACCUMULO-708 if you want to review. Modify ClassLoader to support different applications / multi-tenancy Key: ACCUMULO-708 URL: https://issues.apache.org/jira/browse/ACCUMULO-708 Project: Accumulo Issue Type: Improvement Components: start Reporter: Dave Marion Assignee: Dave Marion Labels: classloader Original Estimate: 24h Time Spent: 21h Remaining Estimate: 3h I'd like to expand the current classloader to support loading classes from HDFS and different application contexts. I'll be modifying the ticket as the idea matures. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
