Re: Flume secure communication

2013-03-12 Thread Inder Pall
As a cheap solution you can always setup a ssh tunnel through port forwarding to do this outside of flume...though it would need to be managed for timeouts/network errors Inder On Tuesday, March 12, 2013, Mike Percy wrote: > No network encryption support yet but there is a patch up at > https://

Re: Analysis of Data

2013-02-08 Thread Inder Pall
Another thought - for streaming analytics you'd need a system which scales so in retrospective how about something like a STORM SINK which internally can use FLUME again to write the processed event to a persistent SINK. - Inder On Fri, Feb 8, 2013 at 10:25 AM, Nitin Pawar wrote: > Hi Steve, > >

Re: Analysis of Data

2013-02-07 Thread Inder Pall
flume is a platform to get events to the right sink (HDFS, local-file, ) analytics is not something which falls in it's territory - Inder On Thu, Feb 7, 2013 at 3:22 PM, Surindhar wrote: > Hi, > > Does Flume supports Analysis of Data? > > Br, > > > -- - Inder "You are average of the 5 p

Re: Installation of full flume on window OS

2012-07-30 Thread Inder Pall
sspath}:${FLUME_CLASSPATH}" > else > FLUME_CLASSPATH="${opt_classpath}" > fi > fi > > if [ -z "${FLUME_HOME}" ]; then > FLUME_HOME=$(cd $(dirname $0)/..; pwd) > fi > > # prepend $FLUME_HOME/lib jars to the specified classpath (if any) &

Re: Installation of full flume on window OS

2012-07-30 Thread Inder Pall
PM, mardan Khan wrote: > HI Inder, > > Sorry for the previous email. Actually I past the snapshot directly into > editor which convert into some noisy text. Please see the snapshot as > attachment . > > Please let me know for any solution > > Thanks > > > > >

Re: Installation of full flume on window OS

2012-07-30 Thread Inder Pall
can you echo your CLASSPATH in the flume startup script. - inder On Mon, Jul 30, 2012 at 4:25 PM, mardan Khan wrote: > Hi Brock > > Thanks for reply. I have solved the problem of cygwin. Now i am getting > another error message: Could not find or load main class of > org.apache.flume.node.Appli

Re: [ANNOUNCE] Apache Flume 1.2.0 Released

2012-07-27 Thread Inder Pall
Mike, Good show as a release manager. Thanks, - Inder On Fri, Jul 27, 2012 at 11:26 AM, Mike Percy wrote: > Somehow it didn't come through via the JIRA query, but we also owe > recognition to Ralph Goers, who put significant work into creating the new > Flume project web site at http://flume.a

Re: [VOTE] GIT as primary source control for Apache Flume

2012-07-17 Thread Inder Pall
+1. Thanks, - Inder On Tue, Jul 17, 2012 at 12:15 PM, alo alt wrote: > +1 > > > On Jul 16, 2012, at 9:38 PM, Arvind Prabhakar wrote: > > > [X] +1 Accept git as the primary source control for Apache Flume. > > > > Regards, > > Arvind Prabhakar > > > > On Sun, Jul 15, 2012 at 12:22 PM, Hari Shree

Re: Is there an encrpted flume channel available?

2012-07-16 Thread Inder Pall
Why can't your encrypt the message before putting it to FLUME? That would the cleanest/quickest way of achieving it IMO...as you can control the keys(pub/private) - Inder On Mon, Jul 16, 2012 at 4:54 PM, Chandrashekhar Shaw wrote: > Can someone help me on this please?? > > I need this for securi

Re: Restarts without data loss

2012-07-10 Thread Inder Pall
our details and thoughts to this. > > Regards, > Arvind Prabhakar > > > On Tue, Jul 10, 2012 at 2:20 AM, Arvind Prabhakar wrote: > >> Hi Inder, >> On Mon, Jul 9, 2012 at 12:02 AM, Inder Pall wrote: >> >>> Arvind, >>> >>> to me this i

Re: Restarts without data loss

2012-07-09 Thread Inder Pall
Senthil, sorry to highjack this thread, but this caught my attention. How do you, if you do switching of channels i.e. mem to file dynamically. - inder On Mon, Jul 9, 2012 at 12:31 PM, Senthilvel Rangaswamy wrote: > We do use persistent channel when there is overflow. Using FileChannel for > r

Re: Restarts without data loss

2012-07-09 Thread Inder Pall
Arvind, to me this is an important use-case for frequent prod rollouts. How about thinking in the direction of supporting graceful shutdown for agents. I can't think of an elegant solution at the moment which will address all cases however what are thoughts regarding something on the lines -> 1.