Thinking some more there are 4 possible outcomes here ( https://github.com/geotools/geotools/blob/master/modules/unsupported/csv/src/main/java/org/geotools/data/csv/parse/CSVLatLonStrategy.java#L215 ): 1. All the fields are present and can be converted 2. A field is blank and should be encoded as NULL 3. The last field is blank and should be encoded as NULL (csvRecord.length < headers.length 4. The entire line is blank and a NULL feature should be returned or an Exception of some sort thrown (CSVRecord.length == 1 && csvRecord[0].isEmpty) Note: This is indistinguishable from a one attribute record that is empty
Similar issues occur in the other strategies. Does that make sense? I'm trying to find what other datastores do with invalid features. Ian On Sat, 14 Dec 2019 at 06:13, Andrea Aime <andrea.a...@geo-solutions.it> wrote: > On Fri, Dec 13, 2019 at 6:54 PM Ian Turton <ijtur...@gmail.com> wrote: > >> While fixing an issue with the GeoServer Importer module I stumbled >> across an issue with the CSV data store ( >> https://osgeo-org.atlassian.net/browse/GEOT-6471) basically, if >> you happen to have a blank line at the end of your file you end up with a >> feature that contains a set of null features. When I fixed this it broke >> another test testDataDoesNotContainAllFields in CSVLatLonState - this is >> why the parser trys to add nulls to "pad" out the record. >> >> I'm proposing to replace this with returning a NULL which will signal to >> the FeatureIterator that the stream has stopped as my feeling is that an >> invalid record should be an error and we should bail rather than hoping for >> the best and ploughing on, which is unlikely to be the best approach? See >> https://github.com/geotools/geotools/pull/2707 for the actual changes. >> It seems not to break any other modules. >> > > Hum... I think a possible way in between is treating the empty line as > such... one thing is a record (has at least one non whitespace char in it) > that > is missing some value... a whole different matter is a empty line, or one > filled with whitespace (which would show up as a record with just one blank > entry even after reading with the CSV library, I suppose?) > > Cheers > Andrea > > == GeoServer Professional Services from the experts! Visit > http://goo.gl/it488V for more information. == Ing. Andrea Aime @geowolf > Technical Lead GeoSolutions S.A.S. Via di Montramito 3/A 55054 Massarosa > (LU) phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549 > http://www.geo-solutions.it http://twitter.com/geosolutions_it > ------------------------------------------------------- *Con riferimento > alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - > Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni > circostanza inerente alla presente email (il suo contenuto, gli eventuali > allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i > destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per > errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le > sarei comunque grato se potesse darmene notizia. This email is intended > only for the person or entity to which it is addressed and may contain > information that is privileged, confidential or otherwise protected from > disclosure. We remind that - as provided by European Regulation 2016/679 > “GDPR” - copying, dissemination or use of this e-mail or the information > herein by anyone other than the intended recipient is prohibited. If you > have received this email by mistake, please notify us immediately by > telephone or e-mail.* > -- Ian Turton
_______________________________________________ GeoTools-Devel mailing list GeoTools-Devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geotools-devel
