[RE: encoding(UTF16-LE) on Windows] Jan Dubois schrieb am 20.01.2011 um 12:45 (-0800): > On Thu, 20 Jan 2011, Michael Ludwig wrote: > > Erland Sommarskog schrieb am 20.01.2011 um 08:29 (-0000): > > > "Jan Dubois" (j...@activestate.com) writes: > > > > You need to stack the I/O layers in the right order. The :encoding() > > > > layer needs to come last (be at the bottom of the stack), *after* the > > > > :crlf layer adds the additional carriage returns. The way to pop the > > > > default :crlf layer is to start out with the :raw pseudo-layer: > > > > > > > > open(my $fh, ">:raw:encoding(UTF-16LE):crlf", $filename) or die $!; > > > > > > Certainly not anywhere close to intuitive. And the explanation is even > > > more muddy. "Needs to come last" - it is smack in the middle. "after > > > the :crlf layer" - it comes before. > > > > The explanation makes sense; so much so that I overlooked the fact that > > this is simply not how it works. Luckily, you were being vigilant. :-) > > Would you mind explaining how it is *not* working the way I > described it above?
Sorry - it works exactly the way you described above. I didn't read properly. I got confused by the uniform look of real and pseudo layers. The :raw pseudo layer is not a layer, but rather, as you write, an instruction to clear the stack, like this: :raw -> clear() :encoding(UTF-16LE) -> push( encoding(UTF-16LE) ) :crlf -> push( crlf ) I was *wrongly* thinking this, as if :raw were another layer, and not a clearing instruction: :raw -> push( raw ) # wrong! :encoding(UTF-16LE) -> push( encoding(UTF-16LE) ) :crlf -> push( crlf ) Regarding your explanation: > I realize that the fact that layers work as a "stack" may be > confusing, which is why I annotated "last" with "bottom of the stack". > Of course the one last on the stack is the first in the list of layers > passed to open() because stacks are LIFO (last in/first out): > > :raw - clears the existing :crlf layer from the stack > could have used :pop instead, but :raw is more robust > > :encoding(UTF-16LE) - pushes the :encoding layer to the stack. This makes > it the last layer on the stack (and also still the > first, for now). > > :crlf - pushes the :crlf layer on the stack. :encoding is > still the last layer, but :crlf is now the first. > > Now when you print a string to the filehandle, then it will be passed > to the top-most layer first (:crlf), which will s/\n/\r\n/g on the > string, and then passes it on to the next lower layer :encoding, which > will do the encoding, and when it reaches the bottom of the stack the > data is actually written to the filesystem. > > Files opened on Windows already have the :crlf layer pushed by default, > so you somehow need to get the :encoding layer *below* it. If > you have it on top, then the crlf substitution happens *after* the > encoding, leading to incorrect data. I think you've clarified it for all eternity. What would be the best place to add your explanation to the docs? http://perldoc.perl.org/functions/binmode.html http://perldoc.perl.org/functions/open.html http://perldoc.perl.org/perlunicode.html http://perldoc.perl.org/PerlIO.html Judging from existing content, I think PerlIO would be a good place for this addition. It already has a lot of great information. However, it starts going medias in res instead of first providing an overview and introducing the stack picture. This could be improved. On the downside, it is buried in the Modules Section. And the title [1] is just too technical and might scare novice readers away. Can you think of a better place for your user-friendly doc addition? You obviously know the docs far better than I do … :-) [1] PerlIO - On demand loader for PerlIO layers and root of PerlIO::* name space -- Michael Ludwig
