Re: NullPointerException : (Channel closed) on writeCheckpoint

2013-05-13 Thread Jay Vyas
Ahhh I had no batchSize parameter. So the fix was to add batchSize = 1 to my file and now its working. Anyone wants more details just ping me On Sun, May 12, 2013 at 7:13 PM, Jay Vyas jayunit...@gmail.com wrote: Hi ! Thanks --- Here's my config a1.sources = r1 a1.sinks = myFileSystem

Re: NullPointerException : (Channel closed) on writeCheckpoint

2013-05-13 Thread Jay Vyas
sure :) ... Maybe we could also commit the fix on our end to. would love a chance to dive into the flume code some more On Mon, May 13, 2013 at 11:39 AM, Brock Noland br...@cloudera.com wrote: Hmm that shouldn't cause an NPE. Could you find a JIRA for this? On Mon, May 13, 2013 at 10:11

Re: NullPointerException : (Channel closed) on writeCheckpoint

2013-05-13 Thread Brock Noland
I'd be highly surprised if not setting batchSize caused the NPE. Here is a good tuning article: https://blogs.apache.org/flume/entry/flume_performance_tuning_part_1 On Mon, May 13, 2013 at 10:58 AM, Jay Vyas jayunit...@gmail.com wrote: Alas - i cannot reproduce this bug anymore., maybe as a

Re: IOException with HDFS-Sink:flushOrSync

2013-05-13 Thread Rahul Ravindran
Pinging again since this has been happening a lot more frequently recently From: Rahul Ravindran rahu...@yahoo.com To: User-flume user@flume.apache.org Sent: Tuesday, May 7, 2013 8:42 AM Subject: IOException with HDFS-Sink:flushOrSync Hi,    We have noticed

Re: syslogTcp - is it possible to figure out senders IP (if not in message)?

2013-05-13 Thread Erik Steffl
That's for Flume agents, we are using rsyslog to send messages (so no Flume agents). However even if we used Flume agents I would still have the same question (for same reason, Flume agent can be misconfigured just like rsyslog can be misconfigured). Looking for something that would get

Re: IOException with HDFS-Sink:flushOrSync

2013-05-13 Thread Hari Shreedharan
What version of Hadoop are you using? Looks like you are getting hit by https://issues.apache.org/jira/browse/HADOOP-6762. Hari -- Hari Shreedharan On Monday, May 13, 2013 at 6:50 PM, Matt Wise wrote: So we've just had this happen twice to two different flume machines... we're using

Re: IOException with HDFS-Sink:flushOrSync

2013-05-13 Thread Rahul Ravindran
Thanks Hari for your help in this. Appreciate it. We will work towards upgrading to CDH 4.2.1 soon, and hopefully, this issue is resolved. ~Rahul. From: Hari Shreedharan hshreedha...@cloudera.com To: user@flume.apache.org user@flume.apache.org Sent: Monday,