Kasper,
I think this is somewhat close to another thing I am describing here:
https://github.com/svenvc/NeoCSV/issues/20
<https://github.com/svenvc/NeoCSV/issues/20>
The problem with extending NeoCSV endlessly is that some of the things
we need with "real-life" CSV files is the fact that they are not CSV
files at all, so it is hard to tell if it is a good idea to litter
NeoCSV with methods for edge cases that have literally to do with the
fact that part of our files are not CSV at all...
I somehow have the feeling some of what we need is a subclass of Stream
that knows about constraints like "a line-break is only the end of a
record if it is not part of a quoted field". So I am somewhat torn
between wanting that stuff in NeoCSV and not wanting to mix csv parsing
with handling of stupid ideas people have when exporting data in some
CSV-like file.
Maybe it is a good idea to collect a few concepts that have been
mentioned in these threads:
* Sometimes we want to skip lines (header, footer) without
interpreting their contents and without any side effects (not
CSV-compliant)
* Sometimes we want to ignore "additional data" after the end of a
defined number of columns (not CSV-compliant)
* Sometimes we need to know which line/column couldn't be parsed
correctly (related to CSV and non-CSV)
A plus would be if we could add the column name to the error message
like in "The column 'amount' in line 34 cannot be interpreted as
monetary amount" - but this is surely quite some work!
* Sometimes we want to interpret columns by the column names in the
header line (which may or may not be the first line of the file,
only the former being CSV-compliant)
Of course this all doesn't mean I am not a fan of NeoCSV. It is
well-written, works very well for "real" CSV and performs very well for
my use cases. Most of the things we are talking about here are problems
that arise when a CSV-file is not a CSV-file...
Joachim
Am 22.01.21 um 07:42 schrieb Kasper Osterbye:
As it happened, I ran into the exact same scenario as Joachim just the
other day,
that is, the external provider of my csv had added some new columns.
In my case
manifested itself in an error that an integer field was not an integer
(because new
columns were added in the middle).
Reading through this whole thread leaves me with the feeling that no
matter what Sven
adds, there is still a risk for error. Nevertheless, my suggestion
would be to add a
functionality to #skipHeaders, or make a sister method:
#assertAndSkipHeaders: numberOfColumns onFailDo: aBlock given the
actual number of headers
That would give me a way to handle the error up front.
This will only be interesting if your data has headers of cause.
Thanks for NeoCSV which I use all the time!
Best,
Kasper
--
-----------------------------------------------------------------------
Objektfabrik Joachim Tuchel mailto:jtuc...@objektfabrik.de
Fliederweg 1 http://www.objektfabrik.de
D-71640 Ludwigsburg http://joachimtuchel.wordpress.com
Telefon: +49 7141 56 10 86 0 Fax: +49 7141 56 10 86 1
