[
https://issues.apache.org/jira/browse/FLUME-1017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13246524#comment-13246524
]
Rafael Nunes commented on FLUME-1017:
-------------------------------------
Hi. Here are some points that I had to deal while trying to use a Syslog Source:
1. When specifying the source's type, use the full class name, as mentioned in
the [Getting
Started|https://cwiki.apache.org/confluence/display/FLUME/Getting+Started] wiki
page of Flume-NG. ie:
{code:title=web.conf|borderStyle=solid}
# Define a memory channel called ch1 on web agent
web.channels.ch1.type = memory
# Define an SyslogTCP source called accesslog on web agent and
# tell it to bind to 0.0.0.0:10514. Connect it to channel ch1.
web.sources.accesslog.type = org.apache.flume.source.SyslogTcpSource
web.sources.accesslog.host = 0.0.0.0
web.sources.accesslog.port = 10514
web.sources.accesslog.channels = ch1
# Define a logger sink that simply logs all events it receives
# and connect it to the other end of the same channel.
web.sinks.log-sink1.type = logger
web.sinks.log-sink1.channel = ch1
# Finally, now that we've defined all of our components, tell
# web which ones we want to activate.
web.sources = accesslog
web.sinks = log-sink1
web.channels = ch1
{code}
2. When using a RSYSLOG 4.6.3 (as in Fedora 14) to redirect remote messages to
Flume's Syslog Source, either TCP or UDP works great;
3. When using a SYSLOG 1.4.1 (as in RedHat 5.2), there are some issues:
a) SYSLOG 1.4.1 supports only UDP;
b) SYSLOG 1.4.1 only tries to connect on remote's host UDP port 514, so your
SyslogUPDSource have to have this port configured (or use any sort of port
redirection/translation).
c) The SyslogUtils.extractEvent method behaves different when used against
RSYSLOG UPD and SYSLOG UDP. In the excerpt of the SyslogUtils class bellow, I
try to explain what I saw until here:
{code:title=SyslogUtils.java|borderStyle=solid}
public static Event extractEvent(ChannelBuffer in)
throws IOException {
//omitted code
try {
while (!doneReading) {
// The line bellow generate a IndexOutOfBoundsException when using
// RSYSLOG, forcing the try/catch block execute the statement under
// the catch and then "return e". But when using against SYSLOG 1.4.1
// the Exception doesn't happen and the try block executes the
// "return null" line.
b = in.readByte();
switch (m) {
case START:
if (b == '<') {
m = Mode.PRIO;
} else {
m = Mode.ERR;
}
break;
case PRIO:
if (b == '>') {
m = Mode.DATA;
} else {
char ch = (char) b;
if (Character.isDigit(ch)) {
prio.append(ch); // stay in PRIO mode
} else {
m = Mode.ERR;
}
}
break;
case DATA:
// TCP syslog entries are separated by '\n'
if (b == '\n') {
e = buildEvent(prio, baos);
doneReading = true;
}
baos.write(b);
break;
case ERR:
if (b == '<') {
// check if its start of new event
m = Mode.PRIO;
}
// otherwise stay in Mode.ERR;
break;
}
}
return null;
} catch (IndexOutOfBoundsException eF) {
e = buildEvent(prio, baos);
}
return e;
}
{code}
Well, after removing the "return null" from the try block, I was able to get my
SyslogUDP Sources working with my RSYSLOG and SYSLOG servers.
Any comments?
Thanks for the good work on this project.
> syslog classes missing
> ----------------------
>
> Key: FLUME-1017
> URL: https://issues.apache.org/jira/browse/FLUME-1017
> Project: Flume
> Issue Type: Bug
> Components: Sinks+Sources
> Affects Versions: NG alpha 1
> Reporter: Alexander Lorenz-Alten
> Priority: Blocker
> Labels: sources, syslog
> Fix For: v1.2.0
>
> Attachments: flume.log, web.conf
>
>
> bin/flume-ng agent -n syslog-agent -f conf/flume-conf.properties.template
> Warning: No configuration directory set! Use --conf <dir> to override.
> 12/03/03 09:57:34 INFO lifecycle.LifecycleSupervisor: Starting lifecycle
> supervisor 1
> 12/03/03 09:57:34 INFO node.FlumeNode: Flume node starting - syslog-agent
> 12/03/03 09:57:34 INFO nodemanager.DefaultLogicalNodeManager: Node manager
> starting
> 12/03/03 09:57:34 INFO lifecycle.LifecycleSupervisor: Starting lifecycle
> supervisor 10
> 12/03/03 09:57:34 INFO properties.PropertiesFileConfigurationProvider:
> Configuration provider starting
> 12/03/03 09:57:34 INFO properties.PropertiesFileConfigurationProvider:
> Reloading configuration file:conf/flume-conf.properties.template
> 12/03/03 09:57:34 WARN properties.FlumeConfiguration: Agent configuration for
> 'exec-agent' source 'tail' has no valid channels. Removing.
> 12/03/03 09:57:34 WARN properties.FlumeConfiguration: Agent configuration for
> 'exec-agent': source 'ExecTail' does not have 'type' specified.
> 12/03/03 09:57:34 WARN properties.FlumeConfiguration: Agent configuration for
> 'exec-agent': source 'ExecTail' has some required attributes missing.
> Removing.
> 12/03/03 09:57:34 WARN properties.FlumeConfiguration: Agent configuration for
> 'exec-agent': source 'ExecTail' is not configured. Removing.
> 12/03/03 09:57:34 INFO properties.FlumeConfiguration: Post-validation flume
> configuration contains configuation for agents: [foo, exec-agent,
> syslog-agent]
> 12/03/03 09:57:34 ERROR properties.PropertiesFileConfigurationProvider:
> Failed to load configuration data. Exception follows.
> org.apache.flume.FlumeException: Unable to load source type: syslogTcp,
> class: syslogTcp
> at
> org.apache.flume.source.DefaultSourceFactory.create(DefaultSourceFactory.java:92)
> at
> org.apache.flume.conf.properties.PropertiesFileConfigurationProvider.loadSources(PropertiesFileConfigurationProvider.java:265)
> at
> org.apache.flume.conf.properties.PropertiesFileConfigurationProvider.load(PropertiesFileConfigurationProvider.java:213)
> at
> org.apache.flume.conf.file.AbstractFileConfigurationProvider.doLoad(AbstractFileConfigurationProvider.java:124)
> at
> org.apache.flume.conf.file.AbstractFileConfigurationProvider.access$300(AbstractFileConfigurationProvider.java:38)
> at
> org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run(AbstractFileConfigurationProvider.java:203)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
> at
> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: java.lang.ClassNotFoundException: syslogTcp
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:169)
> at
> org.apache.flume.source.DefaultSourceFactory.create(DefaultSourceFactory.java:90)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
