Re: Flume integration

2016-11-21 Thread Ian Brooks
at 10:27, Ian Brooks <i.bro...@sensewhere.com[3]> wrote: *-Ian* Hi While I am following this discussion with interest, I am trying to comprehend any architectural benefit of a spark sink. Is there any feature in flume makes it more suitable to ingest stream data than sppark streami

Re: Flume integration

2016-11-21 Thread Ian Brooks
monetary damages arising from such loss, damage or destruction. On 13 July 2016 at 11:13, Ian Brooks <i.bro...@sensewhere.com[4]> wrote: Hi, I'm currently trying to implement a prototype Spark application that gets data from Flume and processes it. I'm using the pull based meth

Re: Flume integration

2016-11-20 Thread Ian Brooks
Hi Mich, Yes, i managed to resolve this one. The issue was because the way described in the docs doesn't work properly as in order for the Flume part to be notified you need to set the storageLevel on the PollingStream like JavaReceiverInputDStream flumeStream =

Flume integration

2016-07-13 Thread Ian Brooks
omething specific I should be doing to let the Flume server know the batch has been received and processed? *Ian Brooks*