NiFi-4181 + patch
Hey, I've been enjoying the new *Record processes, but I found CSVReading/Writing a bit much, with having to describe an Avro schema. While we do use Avro, it's a bit much to define a whole schema for the cases where we have 10-15+ columns of CSV data that we are typically trying to reduce down to a smaller, say 5 column Avro payload. Just being able to list the columns (when they aren't the first line) is helpful, I believe. So, I added a patch for CSVReading/Writing to work given a simple column list, if that helps anyone out! --Wes
Re: NiFi-4181 + patch
Wes, I'm reviewing this now, looking good! A cool feature of it, when used with "Treat First Line As Header" set to true, is that you can rename the columns/fields, so even if you have a header line you don't have to use it :) Regards, Matt On Wed, Jul 12, 2017 at 5:37 PM, Wes Lawrence wrote: > Hey, > > I've been enjoying the new *Record processes, but I found > CSVReading/Writing a bit much, with having to describe an Avro schema. > > While we do use Avro, it's a bit much to define a whole schema for the > cases where we have 10-15+ columns of CSV data that we are typically trying > to reduce down to a smaller, say 5 column Avro payload. Just being able to > list the columns (when they aren't the first line) is helpful, I believe. > > So, I added a patch for CSVReading/Writing to work given a simple column > list, if that helps anyone out! > > --Wes
Re: NiFi-4181 + patch
Ah, an unintended feature! That is useful, and I'll have to keep it in mind. Glad you're liking it, let me know if anything needs to be fixed up. --Wes On Wed, Jul 12, 2017 at 9:39 PM, Matt Burgess wrote: > Wes, > > I'm reviewing this now, looking good! A cool feature of it, when used > with "Treat First Line As Header" set to true, is that you can rename > the columns/fields, so even if you have a header line you don't have > to use it :) > > Regards, > Matt > > > On Wed, Jul 12, 2017 at 5:37 PM, Wes Lawrence > wrote: > > Hey, > > > > I've been enjoying the new *Record processes, but I found > > CSVReading/Writing a bit much, with having to describe an Avro schema. > > > > While we do use Avro, it's a bit much to define a whole schema for the > > cases where we have 10-15+ columns of CSV data that we are typically > trying > > to reduce down to a smaller, say 5 column Avro payload. Just being able > to > > list the columns (when they aren't the first line) is helpful, I believe. > > > > So, I added a patch for CSVReading/Writing to work given a simple column > > list, if that helps anyone out! > > > > --Wes >
