You could probably do this with a MorphlineInterceptor, e.g. via using the grok command in combination with the tryCatch command.
http://flume.apache.org/FlumeUserGuide.html#morphline-interceptor http://kitesdk.org/docs/current/kite-morphlines/index.html http://kitesdk.org/docs/current/kite-morphlines/morphlinesReferenceGuide.html#grok http://kitesdk.org/docs/current/kite-morphlines/morphlinesReferenceGuide.html#tryRules Wolfgang. On Feb 13, 2014, at 7:58 PM, Nikolaos Tsipas wrote: > Hello, > > We have a use case that requires the validation of headers on events received > by an avro source in order to consider an event as valid or invalid. If an > event is invalid then it should be routed to a different channel. > > We know how to route events based on the values of specific headers using > multiplexing. However, for the regex validation of headers flume doesn't seem > to provide any appropriate interceptors. > > For this reason, we are thinking to create a new interceptor that would allow > regex validation of headers and depending on the outcome a header would be > added (e.g. valid = true) > > Questions: > > * Does the above sound like a reasonable solution for what we want to achieve? > * What would be the best way to implement it in order to be beneficial for > the flume community? Extend the functionality of one of the existing > interceptors (e.g. RegexFilteringInterceptor) or provide a new one? > > Regards, > Nikolaos > > > > ---------------------------- > > http://www.bbc.co.uk > This e-mail (and any attachments) is confidential and may contain personal > views which are not the views of the BBC unless specifically stated. > If you have received it in error, please delete it from your system. > Do not use, copy or disclose the information in any way nor act in reliance > on it and notify the sender immediately. > Please note that the BBC monitors e-mails sent or received. > Further communication will signify your consent to this. > > ---------------------
