Re: Avro - Schema is good - Schema validation is bad

2015-12-25 Thread Stefán Baxter
Hi, I appreciate being able to turn it off, thank you. IMO scanning all the files in a directory structure sounds harmless until you start using directory pruning, on large structures, when this becomes a overhead. I really think this is an essential part of an overdue discussion on the "ground

Re: Avro - Schema is good - Schema validation is bad

2015-12-25 Thread Kamesh
For doing schema validation for Avro files, we are thinking of doing the following steps.Please provide your suggestions, before we implement this. - Schema validation feature will be configurable and user can provide whether he/she wants this feature or not by setting a configuration pro