[
https://issues.apache.org/jira/browse/FLUME-2377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17503805#comment-17503805
]
Ralph Goers commented on FLUME-2377:
------------------------------------
I am all for limiting the dependencies in the core components. The only problem
may be that flume-ng-core includes some sources, sinks, and channels. At one
point I believe the FileChannel was using something from Hadoop. I don't know
if it still does.
> Don't automatically include Hadoop and HBase classpaths into Flume classpath
> ----------------------------------------------------------------------------
>
> Key: FLUME-2377
> URL: https://issues.apache.org/jira/browse/FLUME-2377
> Project: Flume
> Issue Type: Improvement
> Components: Shell, Technical Debt
> Reporter: Joshua Hyde
> Priority: Major
>
> HBase and Hadoop, for their own reasons, have Jersey 1.x libraries in their
> classpath. A custom source we wrote uses Jersey 2.x. For a number of reasons,
> the two versions of Jersey do not co-habitate on the same classpath nicely.
> As a result, we get stacktraces like the following if, for example, a
> developer has Hadoop or HBase installed on their system:
> {noformat}
> java.lang.NullPointerException
> at
> com.sun.jersey.core.provider.jaxb.AbstractJAXBProvider.setConfiguration(AbstractJAXBProvider.java:107)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.glassfish.hk2.utilities.reflection.ReflectionHelper.invoke(ReflectionHelper.java:1017)
> at org.jvnet.hk2.internal.ClazzCreator.methodMe(ClazzCreator.java:374)
> at org.jvnet.hk2.internal.ClazzCreator.create(ClazzCreator.java:427)
> at
> org.jvnet.hk2.internal.SystemDescriptor.create(SystemDescriptor.java:456)
> at
> org.jvnet.hk2.internal.PerLookupContext.findOrCreate(PerLookupContext.java:69)
> at org.jvnet.hk2.internal.Utilities.createService(Utilities.java:2488)
> at
> org.jvnet.hk2.internal.ServiceHandleImpl.getService(ServiceHandleImpl.java:98)
> at
> org.jvnet.hk2.internal.ServiceHandleImpl.getService(ServiceHandleImpl.java:87)
> at
> org.glassfish.jersey.internal.inject.ProviderToService.apply(ProviderToService.java:58)
> at
> org.glassfish.jersey.internal.inject.ProviderToService.apply(ProviderToService.java:54)
> at
> jersey.repackaged.com.google.common.collect.Iterators$8.transform(Iterators.java:860)
> at
> jersey.repackaged.com.google.common.collect.TransformedIterator.next(TransformedIterator.java:48)
> at java.util.AbstractCollection.addAll(AbstractCollection.java:333)
> at java.util.LinkedHashSet.<init>(LinkedHashSet.java:169)
> at
> jersey.repackaged.com.google.common.collect.Sets.newLinkedHashSet(Sets.java:292)
> at
> org.glassfish.jersey.internal.inject.Providers.getClasses(Providers.java:364)
> at
> org.glassfish.jersey.internal.inject.Providers.getProviders(Providers.java:186)
> at
> org.glassfish.jersey.message.internal.MessageBodyFactory.<init>(MessageBodyFactory.java:304)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> at
> org.glassfish.hk2.utilities.reflection.ReflectionHelper.makeMe(ReflectionHelper.java:1105)
> at org.jvnet.hk2.internal.ClazzCreator.createMe(ClazzCreator.java:291)
> at org.jvnet.hk2.internal.ClazzCreator.create(ClazzCreator.java:421)
> at
> org.jvnet.hk2.internal.SystemDescriptor.create(SystemDescriptor.java:456)
> at
> org.jvnet.hk2.internal.SingletonContext$1.compute(SingletonContext.java:114)
> at
> org.jvnet.hk2.internal.SingletonContext$1.compute(SingletonContext.java:102)
> at
> org.glassfish.hk2.utilities.cache.Cache$OriginThreadAwareFuture$1.call(Cache.java:97)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at
> org.glassfish.hk2.utilities.cache.Cache$OriginThreadAwareFuture.run(Cache.java:154)
> at org.glassfish.hk2.utilities.cache.Cache.compute(Cache.java:199)
> at
> org.jvnet.hk2.internal.SingletonContext.findOrCreate(SingletonContext.java:153)
> at org.jvnet.hk2.internal.Utilities.createService(Utilities.java:2488)
> at
> org.jvnet.hk2.internal.ServiceLocatorImpl.getService(ServiceLocatorImpl.java:629)
> at
> org.jvnet.hk2.internal.ServiceLocatorImpl.getService(ServiceLocatorImpl.java:614)
> at
> org.glassfish.jersey.server.model.ComponentModelValidator.<init>(ComponentModelValidator.java:97)
> at
> org.glassfish.jersey.server.ApplicationHandler.initialize(ApplicationHandler.java:457)
> at
> org.glassfish.jersey.server.ApplicationHandler.access$500(ApplicationHandler.java:162)
> at
> org.glassfish.jersey.server.ApplicationHandler$3.run(ApplicationHandler.java:322)
> at org.glassfish.jersey.internal.Errors$2.call(Errors.java:289)
> at org.glassfish.jersey.internal.Errors$2.call(Errors.java:286)
> at org.glassfish.jersey.internal.Errors.process(Errors.java:315)
> at org.glassfish.jersey.internal.Errors.process(Errors.java:297)
> at
> org.glassfish.jersey.internal.Errors.processWithException(Errors.java:286)
> at
> org.glassfish.jersey.server.ApplicationHandler.<init>(ApplicationHandler.java:319)
> at
> org.glassfish.jersey.server.ApplicationHandler.<init>(ApplicationHandler.java:272)
> at
> org.glassfish.jersey.server.ContainerFactory.createContainer(ContainerFactory.java:79)
> at
> org.glassfish.jersey.jetty.JettyHttpContainerFactory.createServer(JettyHttpContainerFactory.java:139)
> at com.me.CustomSource.start(CustomSource.java:186)
> at
> org.apache.flume.source.EventDrivenSourceRunner.start(EventDrivenSourceRunner.java:44)
> at
> org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:251)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at
> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:724)
> {noformat}
> The fact that {{flume-ng}} includes the Hadoop and HBase classpath
> automatically, even if we have no need for it, is a bit surprising and, if
> you don't know that the shell script is doing that, it can take a little bit
> of digging to figure it out.
> At the very least, I'd like the ability to tell {{flume-ng}}, "No, don't
> include the Hadoop classpath," and, "No, don't include the HBase classpath,",
> though if someone's willing to throw passivity to the wind, I'd be glad to
> just not have those external classpaths added to Flume's classpath to begin
> with.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
