It could be useful for you this project [1] the side where the source is converted to avro using a custom parser.
[1] https://github.com/danieltardon/flume-ng-kafka-avro-sink On Fri, Aug 29, 2014 at 5:18 PM, Ed Judge <[email protected]> wrote: > From initial reading of what it does, it look promising. So, I could > ultimately leverage something like the readLine and toAvro commands > > Thanks, > Ed > > > On Aug 29, 2014, at 8:25 AM, Gaspar Muñoz <[email protected]> wrote: > > Hi Ed, > > maybe after read logs from your file, you can apply > org.apache.flume.sink.solr.morphline.MorphlineInterceptor$Builder > interceptor and then apply to Avro morphline. > > Is that what you are looking for? > > Regards. > > > 2014-08-29 13:43 GMT+02:00 Ed Judge <[email protected]>: >> >> I am looking for some good documentation that would explain how I can pull >> logs from a file and apply a certain schema to that log file format. I have >> gone threw the exercise of sending a file using a Avro source/sink with the >> flume_ng avro_client to a flume_ng agent but my understanding is that this >> is just using a "default Avro schema”. I want to be able build the schema a >> certain way. Assuming it is doable, is there anything out that documents >> this process? >> >> Thanks, >> Ed > > >
