Beware using the HDFS classloader in any Accumulo release that does not
contain commons-vfs-2.1 as a dependency.
Commons-vfs-2.0 has multiple known issues which prevent it from being
usable in the more basic sense.
Presently, there is only one release which contains the fix already:
Accumulo 1.7.2. The upcoming 1.6.6 and 1.8.0 releases will also have the
updated dependency.
Massimilian Mattetti wrote:
Hi Jim,
the approach of using namespace from HDFS looks promising. I need to
investigate a little on how it works but I guess I will take your advice.
Thank you.
Cheers,
Massimiliano
From: James Hughes <jn...@virginia.edu>
To: user@accumulo.apache.org
Date: 07/07/2016 08:28 PM
Subject: Re: java.lang.NoClassDefFoundError with fields of custom Filter
------------------------------------------------------------------------
Hi Massimiliano,
I'm a fan of producing uber jars for this kind of thing; we do that for
GeoMesa. There is one gotcha which can come up: if you have several uber
jars in lib/ext, they can collide in rather unexpected ways.
There are two options to call out:
First, Accumulo has support for loading jars from HDFS into namespaces.
With that, you could have various namespaces for different versions or
different collections of iterator projects. If you are sharing a dev
cloud with other projects or co-workers working on the same project that
can be helpful since it would avoid restarts, etc. Big thumbs-up for
this approach!
Second, rather than having an uber jar, you could build up zip files
with the various jars you need for your iterators and unzip them in
lib/ext. If you did that for multiple competing iterator projects, you'd
avoid duplication of code inside uber-jars. Also, you'd be able to see
if there are 8 versions of Log4J and Guava in lib/ext...;) It wouldn't
be as powerful as the namespace, but there's something nice about having
a low-tech approach.
Others will likely have varied experiences; I'm not sure if there's an
established 'best practice' here.
Cheers,
Jim
On Thu, Jul 7, 2016 at 12:56 PM, Massimilian Mattetti
<_massi...@il.ibm.com_ <mailto:massi...@il.ibm.com>> wrote:
Thanks for your prompt response, you are right Jim. There is a static
dependency to Log4J in my lexicoder. Adding the Log4J Jar to the
classpath solved the problem.
Would you suggest to use an Uber Jar to avoid this kind of problems?
Regards,
Massimiliano
From: James Hughes <_jnh5y@virginia.edu_ <mailto:jn...@virginia.edu>>
To: _user@accumulo.apache.org_ <mailto:user@accumulo.apache.org>
Date: 07/07/2016 06:25 PM
Subject: Re: java.lang.NoClassDefFoundError with fields of custom Filter
------------------------------------------------------------------------
Hi Massimilian,
As a quick note, your error says that it could not initialize class
accumulo.lexicoders.MyLexicoder. Did you provide all the dependencies
for your class on Accumulo's classpath?
That exception (or similar) can occur if there is a static block in your
MyLexicoder class which can't run properly.
Cheers,
Jim
On Thu, Jul 7, 2016 at 11:19 AM, Massimilian Mattetti
<_massi...@il.ibm.com_ <mailto:massi...@il.ibm.com>> wrote:
Hi,
I have implemented a custom filter and a custom lexicoder. Both this two
classes are packed in the same jar that has been deployed under the
directory $ACCUMULO_HOME/*lib*/*ext*of my Accumulo servers (version
1.7.1). The lexicoder is used by the filter to get the real object from
the accumulo value and test some conditions on it. When I tried to scan
the table applying this filter I got the following exception:
Caused by: java.lang.NoClassDefFoundError: Could not initialize class
accumulo.lexicoders.MyLexicoder
at accumulo.filters.MyFilter.<init>(MyFilter.java:24)
at sun.reflect.GeneratedConstructorAccessor9.newInstance(Unknown Source)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
at java.lang.Class.newInstance(Class.java:442)
at
org.apache.accumulo.core.iterators.IteratorUtil.loadIterators(IteratorUtil.java:261)
at
org.apache.accumulo.core.iterators.IteratorUtil.loadIterators(IteratorUtil.java:237)
at
org.apache.accumulo.core.iterators.IteratorUtil.loadIterators(IteratorUtil.java:218)
at
org.apache.accumulo.core.iterators.IteratorUtil.loadIterators(IteratorUtil.java:205)
at
org.apache.accumulo.tserver.tablet.ScanDataSource.createIterator(ScanDataSource.java:193)
at
org.apache.accumulo.tserver.tablet.ScanDataSource.iterator(ScanDataSource.java:127)
at
org.apache.accumulo.core.iterators.system.SourceSwitchingIterator.seek(SourceSwitchingIterator.java:180)
at org.apache.accumulo.tserver.tablet.Tablet.nextBatch(Tablet.java:880)
at org.apache.accumulo.tserver.tablet.Scanner.read(Scanner.java:98)
at org.apache.accumulo.tserver.scan.NextBatchTask.run(NextBatchTask.java:69)
at org.apache.htrace.wrappers.TraceRunnable.run(TraceRunnable.java:57)
... 4 more
I do not undestand how it is possible that the class loader is able to
find the filter and no the lexicoder. Am I missing something?
Thanks,
Massimiliano
