finetu created FLUME-3037:
-----------------------------
Summary: can't get events correctly while flume working with
python socket or telnet
Key: FLUME-3037
URL: https://issues.apache.org/jira/browse/FLUME-3037
Project: Flume
Issue Type: Question
Components: Sinks+Sources
Affects Versions: v1.7.0
Environment: ubuntu 14&java 1.8
centos &java 1.7
Reporter: finetu
I want to use flume for collecting logs in python scripts, so I follow user
guide to config flume with netcat source, then I use telnet and nc for test, it
works well.
config code :
a1.sources = r1
a1.sinks = k1
a1.channels = c1
# Describe/configure the source
a1.sources.r1.type = netcat
a1.sources.r1.bind = localhost
a1.sources.r1.port = 44444
a1.sinks.k1.type = logger
a1.channels.c1.type = memory
a1.channels.c1.capacity = 1000
a1.channels.c1.transactionCapacity = 100
a1.sources.r1.channels = c1
a1.sinks.k1.channel = c1
Then I use python to connect flume, and send some words to it like this:
import socket
def netcat(hostname, port):
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((hostname, port))
s.send("test words 1\n")
s.send("test words 2\n")
s.send("test words 3\n")
s.send("test words 4\n")
s.shutdown(socket.SHUT_WR)
s.close()
if __name__ == "__main__":
netcat("127.0.0.1",44444)
problem happens, flume can only receive 2 rows.
flume logs:
2016-12-28 16:44:32,248 (lifecycleSupervisor-1-0) [INFO -
org.apache.flume.source.NetcatSource.start(NetcatSource.java:169)] Created
serverSocket:sun.nio.ch.ServerSocketChannelImpl[/127.0.0.1:44444]
2016-12-28 16:44:41,814 (SinkRunner-PollingRunner-DefaultSinkProcessor) [INFO -
org.apache.flume.sink.LoggerSink.process(LoggerSink.java:95)] Event: {
headers:{} body: 74 65 73 74 20 77 6F 72 64 73 20 31 test words 1 }
2016-12-28 16:44:41,815 (SinkRunner-PollingRunner-DefaultSinkProcessor) [INFO -
org.apache.flume.sink.LoggerSink.process(LoggerSink.java:95)] Event: {
headers:{} body: 74 65 73 74 20 77 6F 72 64 73 20 32 test words 2 }
I got same result on both ubuntu&java1.8 and centos&java 1.7 and the same
result with telnet model in python.
Is there anything wrong with config or python scripts ?
I think the issue-2611 is similar to this one.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
