Thank you for all the input, and I think I have resolved my particular issue. Battle won. War still raging.
Using the script suggested by Galen as an starting point, I wrote the following hack outputting integers denoting MARC records containing non-UTF-8 characters, but the script output nothing; all the data in all of my records was encoded as UTF-8: #!/usr/bin/perl # require use strict; use Encode; # initialize binmode STDIN, ":bytes"; $/ = "\035"; my $i = 0; # read STDIN while ( <> ) { # increment $i++; # check validity eval { my $utf8str = &Encode::is_utf8( $_, Encode::FB_CROAK ); }; # check for error if ( $@ ) { print "Record $i contains non-UTF-8 characters\n"; } } # done exit; Since all of the data in all of my records was UTF-8, then all of the leaders of all of the records need to have a value of "a" set in position #9 of the leader. So I wrote the following hack (circumventing MARC::Batch): #!/usr/bin/perl # require use strict; # initialize binmode STDIN, ":bytes"; binmode STDOUT, ":bytes"; $/ = "\035"; # loop through the input while ( <> ) { # do the work and output substr( $_, 9, 1 ) = "a"; print $_; } # done exit; I then fed the output of my fix routine to my indexing routing, and all of my problems seemed to go away. GIGO? I'm still not sure, but I think deep within MARC::Batch some sort of encoding is observed, honored, and output. And when the denoted encoding is not true and things like binmode( FILE, ":utf8" ) get called, output gets munged. Again, I'm not sure. It is almost exhausting. -- Eric Morgan University of Notre Dame