Hi,
When I try the latest trunk (commit
da84afe1013cab896fb1fcc11ea6e8e6dba87f3c) I see no problem if I specify the
FQCN (org.apache.flume.source.MultiportSyslogTCPSource), but I see a
ClassNotFoundException when I try to specify the 'multiport_syslogtcp'
alias (stack trace below). I'm looking at SourceType.java in trunk, and I
do see the correct enum value defined:
[will@will-laptop apache-flume-ng]$ cat
flume-ng-configuration/src/main/java/org/apache/flume/conf/source/SourceType.java
| grep -i multi
* MultiportSyslogTCPSource
* @see org.apache.flume.source.MultiportSyslogTCPSource
MULTIPORT_SYSLOGTCP("org.apache.flume.source.MultiportSyslogTCPSource"),
...but when I look into the expanded tar dist that was created from 'mvn
install', I don't see that enum value, which is very strange:
tar xf flume-ng-dist/target/apache-flume-1.4.0-SNAPSHOT-dist.tar.gz
[will@will-laptop apache-flume-ng]$ cat
apache-flume-1.4.0-SNAPSHOT/flume-ng-configuration/src/main/java/org/apache/flume/conf/source/SourceType.java
| grep multi
[will@will-laptop apache-flume-ng]$
Here's the log trace:
28 Nov 2012 11:56:19,241 ERROR [conf-file-poller-0]
(org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run:204)
- Failed to load configuration data. Exception follows.
org.apache.flume.FlumeException: Unable to load source type:
multiport_syslogtcp, class: multiport_syslogtcp
at
org.apache.flume.source.DefaultSourceFactory.create(DefaultSourceFactory.java:93)
at
org.apache.flume.conf.properties.PropertiesFileConfigurationProvider.loadSources(PropertiesFileConfigurationProvider.java:319)
at
org.apache.flume.conf.properties.PropertiesFileConfigurationProvider.load(PropertiesFileConfigurationProvider.java:222)
at
org.apache.flume.conf.file.AbstractFileConfigurationProvider.doLoad(AbstractFileConfigurationProvider.java:123)
at
org.apache.flume.conf.file.AbstractFileConfigurationProvider.access$300(AbstractFileConfigurationProvider.java:38)
at
org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run(AbstractFileConfigurationProvider.java:202)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at
java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.ClassNotFoundException: multiport_syslogtcp
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:169)
at
org.apache.flume.source.DefaultSourceFactory.create(DefaultSourceFactory.java:91)
... 14 more
...and here's the config file I used that is equivalent to the original
config file:
a1.channels = c1
a1.sources = r1
a1.sinks = k1
a1.channels.c1.type = memory
a1.sources.r1.channels = c1
a1.sources.r1.type = multiport_syslogtcp
#a1.sources.r1.type = org.apache.flume.source.MultiportSyslogTCPSource
a1.sources.r1.ports = 10001 10002 10003
a1.sources.r1.host = localhost
a1.sources.r1.portHeader = port
a1.sinks.k1.channel = c1
a1.sinks.k1.type = file_roll
a1.sinks.k1.sink.directory = output
a1.sinks.k1.sink.rollInterval = -1
...and launched with this command:
in/flume-ng agent --conf conf --conf-file conf/flume.conf --name a1
Cheers,
Will
On Wed, Nov 28, 2012 at 1:22 AM, Alexander Alten-Lorenz <[email protected]
> wrote:
> The same in trunk:
>
> 28 Nov 2012 10:21:32,055 INFO [conf-file-poller-0]
> (org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty:912)
> - Added sinks: fileRoll Agent: agent1
> 28 Nov 2012 10:21:32,060 WARN [conf-file-poller-0]
> (org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.validateSources:571)
> - Removed src0 due to Could not create configuration!
>
> Also when I use the class name - same error.
>
> Thanks,
> Alex
>
> On Nov 28, 2012, at 4:50 AM, Roshan Naik <[email protected]> wrote:
>
> > Multiport syslog tcp source problem ..
> >
> > Here is the config I am using...
> >
> > agent1.channels = memoryChannel
> > agent1.channels.memoryChannel.type = memory
> >
> > agent1.sources = src0
> > agent1.sources.src0.type = multiport_syslogtcp
> > # agent1.sources.src0.type =
> > org.apache.flume.source.MultiportSyslogTCPSource
> > agent1.sources.src0.channels = memoryChannel
> > agent1.sources.src0.host = localhost
> > agent1.sources.src0.ports = 10001 10002 10003
> > agent1.sources.src0.portHeader = port
> >
> > agent1.sinks = fileRoll
> > agent1.sinks.fileRoll.type = file_roll
> > agent1.sinks.fileRoll.channel = memoryChannel
> > agent1.sinks.fileRoll.sink.directory = output
> > agent1.sinks.fileRoll.sink.rollInterval = -1
> >
> >
> > Here is the error...
> >
> > 2/11/27 19:46:24 INFO conf.FlumeConfiguration: Processing:fileRoll
> > 12/11/27 19:46:24 INFO conf.FlumeConfiguration: Processing:fileRoll
> > 12/11/27 19:46:24 INFO conf.FlumeConfiguration: Processing:fileRoll
> > 12/11/27 19:46:24 INFO conf.FlumeConfiguration: Processing:fileRoll
> > 12/11/27 19:46:24 INFO conf.FlumeConfiguration: Added sinks: fileRoll
> > Agent: agent1
> > *12/11/27 19:46:24 WARN conf.FlumeConfiguration: Removed src0 due to
> Could
> > not create configuration!*
> > 12/11/27 19:46:24 INFO conf.FlumeConfiguration: Post-validation flume
> > configuration contains configuration for agents: [agent1]
> > 12/11/27 19:46:24 INFO properties.PropertiesFileConfigurationProvider:
> > Creating channels
> > 12/11/27 19:46:24 INFO properties.PropertiesFileConfigurationProvider:
> > created channel memoryChannel
> > 12/11/27 19:46:24 INFO sink.DefaultSinkFactory: Creating instance of
> sink:
> > fileRoll, type: file_roll
> > 12/11/27 19:46:24 INFO nodemanager.DefaultLogicalNodeManager: Starting
> new
> > configuration:{ sourceRunners:{} sinkRunners:{fileRoll=SinkRunner: {
> > policy:org.apache.flume.sink.DefaultSinkProcessor@44a613f8counterGroup:{
> > name:null counters:{} } }}
> > channels:{memoryChannel=org.apache.flume.channel.MemoryChannel{name:
> > memoryChannel}} }
>
> --
> Alexander Alten-Lorenz
> http://mapredit.blogspot.com
> German Hadoop LinkedIn Group: http://goo.gl/N8pCF
>
>
