On 4/6/2011 2:02 PM, Kyle Banerjee wrote:
I'd go so far as to question the value of validating redundant data that
theoretically has meaning but which are never supposed to vary. The 4 and
the 5 simply repeat what is already known about the structure of the MARC
record. Choking on stuff like this is like having a web browser ask you want
to do with a page because it lacks a document type declaration.
Well, the problem is when the original Marc4J author took the spec at
it's word, and actually _acted upon_ the '4' and the '5', changing file
semantics if they were different, and throwing an exception if it was a
non-digit.
This actually happened, I'm not making this up! Took me a while to debug.
So do you think he got it wrong? How was he supposed to know he got it
wrong, he wrote to the spec and took it at it's word. Are you SURE there
aren't any Marc formats other than Marc21 out there that actually do use
these bytes with their intended meaning, instead of fixing them? How was
the Marc4J author supposed to be sure of that, or even guess it might be
the case, and know he'd be serving users better by ignoring the spec
here instead of following it? What documents instead of the actual
specifications should he have been looking at to determine that he ought
not to be taking those bytes at their words, but just ignoring them?
To realize that we have so much non-conformant data out there that we're
better off ignoring these bytes, is something you can really only learn
through experience -- and something you can then later realize you're
wrong on too:
Ie: I _thought_ I was writing only for Marc21, but then it turns out
I've got to accept records from Outer Weirdistan that are a kind of
legal Marc that actually uses those bytes for their intended meaning --
better go back and fix my entire software stack, involving various
proprietary and open source products from multiple sources, each of
which has undocumented behavior when it comes to these bytes, maybe they
follow the spec or maybe the follow Kyle's advice, but they don't tell
me. This is a mess.
Maybe this scenario is impossible, maybe there ARE and NEVER HAVE BEEN
any Marc variants that actually use leader bytes 20-22 in this way --
how can I determine that? I've just got to guess and hope for the
best. The point of specifications in the first place is for
inter-operability, so we know that if all software and data conforms to
the spec, then all software and data will interact in expected ways.
Once we start guessing at which parts of the spec we really ought to be
ignoring....
Again, I realize in the actual environment we've got, this is not a
luxury we have. But it's a fault, not a benefit, to have lots of
software everywhere behaving in non-compliant ways and creating invalid
(according to the spec!) data.
