thanks gonzalo, it's running but still cant write on hdfs. the error is http://tempel.blankon.in/1527360
when Starting connection handler it always Checking file:/home/hdfs/adaw.conf for changes and i check on hdfs, there is no file On 16 November 2015 at 15:34, Gonzalo Herreros <[email protected]> wrote: > I see your problem now. You are using "memory-channel" and "memoryChannel" > to refer to the same thing. > Change the 3rd line to: > > hdfs-agent.channels= memoryChannel > > On 16 November 2015 at 08:31, zaenal rifai <[email protected]> wrote: > >> 1.Yes i get error, the error is " Agent configuration for hdfs-agent does >> not contain any valid channels. Marking it as invalid " >> here is for more about the error >> >> http://tempel.blankon.in/1527288 >> >> 2.No, i don't have enable the kerberos >> >> 3.i'm not sure about the problem, i tried another method sink and it >> still failed. >> >> can you give me some simple example for agent configuration to collect >> log and write on hdfs ? >> >> >> >> >> >> >> On 16 November 2015 at 14:47, Gonzalo Herreros <[email protected]> >> wrote: >> >>> I looks fine to me. Do you get any errors? >>> Do you have kerberos enabled? maybe is a security issue >>> Are you sure the problem is hdfs and not some netcat error? Try with >>> another sink to confirm that. >>> >>> Regards, >>> Gonzalo >>> >>> On 16 November 2015 at 07:35, zaenal rifai <[email protected]> >>> wrote: >>> >>>> Hello guys, i'm newbie on flume, i want to ask something about >>>> configuration on flume-agent which can write event on HDFS. >>>> >>>> for example which i can find on internet is like this. >>>> >>>> hdfs-agent.sources= netcat-collect >>>> hdfs-agent.sinks = hdfs-write >>>> hdfs-agent.channels= memory-channel >>>> hdfs-agent.sources.netcat-collect.type = netcat >>>> hdfs-agent.sources.netcat-collect.bind = 127.0.0.1 >>>> hdfs-agent.sources.netcat-collect.port = 11111 >>>> hdfs-agent.sinks.hdfs-write.type = hdfs >>>> hdfs-agent.sinks.hdfs-write.hdfs.path =/user/user/togatta/ >>>> hdfs-agent.sinks.hdfs-write.rollInterval = 10 >>>> hdfs-agent.sinks.hdfs-write.hdfs.writeFormat=Text >>>> hdfs-agent.sinks.hdfs-write.hdfs.fileType=DataStream >>>> hdfs-agent.channels.memoryChannel.type = memory >>>> hdfs-agent.channels.memoryChannel.capacity=10000 >>>> hdfs-agent.sources.netcat-collect.channels=memoryChannel >>>> hdfs-agent.sinks.hdfs-write.channel=memoryChannel >>>> >>>> this configuration will collect about log netcat event, but when i run >>>> it, it cant write on path hdfs which i had to config. So any suggest from >>>> all of you guys? >>>> >>> >>> >> >
