# New Ticket Created by Tobias Leich # Please include the string: [perl #123070] # in the subject line of all future correspondence about this issue. # <URL: https://rt.perl.org/Ticket/Display.html?id=123070 >
Taken from: http://irclog.perlgeek.de/perl6/2014-10-27#i_9574112 bronco_creek: I have a question about reading from a file. When I use, for example: for $in.lines -: $line {say $line} is see a "?" prepended to the first line. What's up with that? FROGGS: that sounds like a replacement character.. geekosaur: windows by any chance? a utf8 file with a byte order mark at the start (like windows text editors like to prepend), sent to a non-utf8 display, would do that FROGGS: bronco_creek: does that file start with a BOM? bronco_creek: Yes, Windows. FROGGS: bronco_creek: you've got notepad++ by any chance? bronco_creek: I'm using Padre. FROGGS: ahh, hmmm FROGGS: dunno if that allows you to save that file without a BOM bronco_creek: The file I'm trying to process is xml output from another program, but the leading "?" does not show up in Padre. timotimo: do you have some kind of hex editor? timotimo: now the question becomes timotimo: do we want to support byte order marks in rakudo? geekosaur: I think you'll need to; notepad in at least some versions of windows will insist on prepending a BOM geekosaur: some other text editors as well (and there's a handful of programs, including some from Microsoft, that depend on it!) geekosaur: even though you're not supposed to use a BOM with UTF-8 bronco_creek: I tried to strip the "?" with ~~ s/^\?// but that did not do the trick. geekosaur: yes, it's getting mapped to a replacement character on output; it's not actually a question mark timotimo: that's right timotimo: how about opening the file as binary and outputting the first few bytes as hexadecimals and looking at what exactly it is geekosaur: actually on display, not on output bronco_creek: Hmm. Drove me nut trying to figure out why all of my xml files were invalid when I tried to parse with XML or XML::Parser::Tiny timotimo: damn, that's annoying timotimo: very sorry to hear that, bronco_creek :(
