Sounds good! Thanks!
Thanks, Hari
On Mon, Feb 9, 2015 at 6:42 PM, Ashish wrote:
> I did tried it, but was lost a bit on getting an Event from
> TransactionalEvent. I shall work out the details, meanwhile you can
> help me out with the Event part.
> thanks
> ashish
> On Tue, Feb 10, 2015 at 5
I did tried it, but was lost a bit on getting an Event from
TransactionalEvent. I shall work out the details, meanwhile you can
help me out with the Event part.
thanks
ashish
On Tue, Feb 10, 2015 at 5:56 AM, Hari Shreedharan
wrote:
> Correct - that would be pretty tricky. We could indeed modify
Correct - that would be pretty tricky. We could indeed modify the tool to take
a custom function to process each event - that would work. We must specify an
interface that the user must implement, say, FileChannelDataVerifier or
something. We then call this on each event.
Do you want to tak
I think I was not clear. I was taking about an offline tool which
would help us clean the Channel with valid Flume Event, with
invalid/malformed payload. What you described would work in Agent
flow. Given the current scenario, Charles would be keen to push the
backed up events to destination.
Here
It is not as easy, since the channel does not know what an event looks like or
why the transaction is being rolled back. That is something that is being
handled by the serializer and sink. We need to some how remove an event from
the channel if it the channel is broken. We might perhaps have to
Is it possible to extend File Channel Integrity tool to support
filtering out corrupt events? Something like once we get a record and
it's not corrupt, provide a placeholder function to validate Event
implemented by user. I still don't have too much insight into
FileChannel implementation.
On Tue,
Ok, thanks for the response. I created this ticket:
https://issues.apache.org/jira/browse/FLUME-2613
On Mon, Feb 2, 2015 at 11:19 AM, Hari Shreedharan wrote:
> Currently, no - there is no such tool, but this is a request that has come
> up time and again. Can you file a jira for this? If some
Currently, no - there is no such tool, but this is a request that has come up
time and again. Can you file a jira for this? If someone has time, they’d
probably pick it up
Thanks, Hari
On Mon, Feb 2, 2015 at 10:52 AM, Charles McLaughlin
wrote:
> Hello,
> We had a situation where one of our
Hello,
We had a situation where one of our Flume agents got stuck on a message due
to unexpected format. To get things moving again, I stopped the Flume
agent, moved the file backed channel data out of the way and re-started the
Flume agent. I'd like to pop the bad message from the queue data on