I've just been looking at this same problem as I see it occasionally in my logs. In my test case the warning occurs when writing plain HTML which I'm certain doesn't contain any UTF-16 surrogates. After some (too much!) poking around, I eventually realised that the reported surrogate is actually the size of the output (so in your example case, I'm guessing your output was DF98 == 57240 bytes). So that suggests to me that it's something to do with the FastCGI protocol header. However a cursory examination of the contents of the buffer passed to syswrite doesn't reveal this header to be present, and a quick look through the calling stack doesn't reveal any routine which might add it. The length of the buffer is, however, always the number which is later reported as a UTF-16 surrogate. So I conclude that the FastCGI protocol header is being put in place on a lower layer, out of my reach.
All this neatly explains the previously opaque distribution of the warning in my logs (output size isn't often in the UTF-16 reserved range), and why it was so difficult for me to find a reliable, simple test case (every time I tried to simplify the test case to isolate the cause, the warning went away because the simplified case wasn't in the right size range). Unfortunately, it also tells me that I'm not going to be able to fix it on this particular attack - I've run out of time so I'm going to have to put it aside (again) until I can make another chance to investigate further. Hope all this helps - I look forward to someone else solving it for me ;) Merlyn Kline -----Original Message----- From: Martin Ellison [mailto:] Sent: 12 May 2008 10:05 To: The elegant MVC web framework Subject: Re: [Catalyst] UTF-16 surrogate message when writing binary data(image) The error is specifically at /usr/lib/perl/5.8/IO/Handle.pm line 199, which is the second syswrite call in sub syswrite { @_ >= 2 && @_ <= 4 or croak 'usage: $io->syswrite(BUF [, LEN [, OFFSET]])'; if (defined($_[2])) { syswrite($_[0], $_[1], $_[2], $_[3] || 0); } else { syswrite($_[0], $_[1]); } } The error is in production, so I am adding some trace code to investigate further, but results will need to wait until I have pushed the code. 2008/5/10 Matt S Trout <[EMAIL PROTECTED]>: On Fri, May 09, 2008 at 02:58:41PM +0800, Martin Ellison wrote: > If I write binary data (a JPEG) using $c->res->body then I get all these > errors > > stderr: UTF-16 surrogate 0xdf98 at /usr/lib/perl/5.8/IO/Handle.pm line 199. > > My code looks like > > $c->res->content_type(q{image/jpeg}); > $c->res->header( 'Content-Disposition', q{inline} ); > $c->res->body($pic_image); > > Presumably, something is assuming that the output is Unicode text and trying > to interpret it accordingly. These error messages are all over the log, > making it difficult to read, besides any impact the situation may be having > on the output. > > Is there any way to fix this issue (eg something like binmode)? My best guess here is that $pic_image is, or looks like, a file handle, and so when Catalyst is sending the response it's doing so by reading from the filehandle and the error's turning up during $fh->read. It'd of course be easier for you to confirm this, since you have the copy of IO/Handle.pm and the line number - maybe you could look? A good way to check would be to loop reading $pic_image yourself and see if you get the same warning ... -- Matt S Trout Need help with your Catalyst or DBIx::Class project? Technical Director http://www.shadowcat.co.uk/catalyst/ Shadowcat Systems Ltd. Want a managed development or deployment platform? http://chainsawblues.vox.com/ http://www.shadowcat.co.uk/servers/ _______________________________________________ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/ -- Regards, Martin ([EMAIL PROTECTED]) IT: http://methodsupport.com Personal: http://thereisnoend.org
_______________________________________________ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/