thanks gonzalo you are my man :D
On 16 November 2015 at 16:57, Gonzalo Herreros <[email protected]> wrote: > That is not on hdfs, it the home folder of the hdfs user on linux; it's > just checking your Flume configuration for changes, don't worry about it. > > Use the FileSink and see if it works so you can know if the issue is hdfs > related or not. > I think it's more likely your issue is syslog related. > > On 16 November 2015 at 08:47, zaenal rifai <[email protected]> wrote: > >> thanks gonzalo, it's running but still cant write on hdfs. >> >> the error is >> http://tempel.blankon.in/1527360 >> >> when Starting connection handler >> >> it always Checking file:/home/hdfs/adaw.conf for changes >> >> and i check on hdfs, there is no file >> >> On 16 November 2015 at 15:34, Gonzalo Herreros <[email protected]> >> wrote: >> >>> I see your problem now. You are using "memory-channel" and "memoryChannel" >>> to refer to the same thing. >>> Change the 3rd line to: >>> >>> hdfs-agent.channels= memoryChannel >>> >>> On 16 November 2015 at 08:31, zaenal rifai <[email protected]> >>> wrote: >>> >>>> 1.Yes i get error, the error is " Agent configuration for hdfs-agent >>>> does not contain any valid channels. Marking it as invalid " >>>> here is for more about the error >>>> >>>> http://tempel.blankon.in/1527288 >>>> >>>> 2.No, i don't have enable the kerberos >>>> >>>> 3.i'm not sure about the problem, i tried another method sink and it >>>> still failed. >>>> >>>> can you give me some simple example for agent configuration to collect >>>> log and write on hdfs ? >>>> >>>> >>>> >>>> >>>> >>>> >>>> On 16 November 2015 at 14:47, Gonzalo Herreros <[email protected]> >>>> wrote: >>>> >>>>> I looks fine to me. Do you get any errors? >>>>> Do you have kerberos enabled? maybe is a security issue >>>>> Are you sure the problem is hdfs and not some netcat error? Try with >>>>> another sink to confirm that. >>>>> >>>>> Regards, >>>>> Gonzalo >>>>> >>>>> On 16 November 2015 at 07:35, zaenal rifai <[email protected]> >>>>> wrote: >>>>> >>>>>> Hello guys, i'm newbie on flume, i want to ask something about >>>>>> configuration on flume-agent which can write event on HDFS. >>>>>> >>>>>> for example which i can find on internet is like this. >>>>>> >>>>>> hdfs-agent.sources= netcat-collect >>>>>> hdfs-agent.sinks = hdfs-write >>>>>> hdfs-agent.channels= memory-channel >>>>>> hdfs-agent.sources.netcat-collect.type = netcat >>>>>> hdfs-agent.sources.netcat-collect.bind = 127.0.0.1 >>>>>> hdfs-agent.sources.netcat-collect.port = 11111 >>>>>> hdfs-agent.sinks.hdfs-write.type = hdfs >>>>>> hdfs-agent.sinks.hdfs-write.hdfs.path =/user/user/togatta/ >>>>>> hdfs-agent.sinks.hdfs-write.rollInterval = 10 >>>>>> hdfs-agent.sinks.hdfs-write.hdfs.writeFormat=Text >>>>>> hdfs-agent.sinks.hdfs-write.hdfs.fileType=DataStream >>>>>> hdfs-agent.channels.memoryChannel.type = memory >>>>>> hdfs-agent.channels.memoryChannel.capacity=10000 >>>>>> hdfs-agent.sources.netcat-collect.channels=memoryChannel >>>>>> hdfs-agent.sinks.hdfs-write.channel=memoryChannel >>>>>> >>>>>> this configuration will collect about log netcat event, but when i >>>>>> run it, it cant write on path hdfs which i had to config. So any suggest >>>>>> from all of you guys? >>>>>> >>>>> >>>>> >>>> >>> >> >
