On Dec 20, 2004, at 6:44 PM, David Wheeler wrote:
PS Somebody should drag autrijus into this.
I'll try to grab him on IRC in the morning...
I got him this morning. Here's the discussion:
09:50am] Theory: seen autrijus
[09:50am] purl: autrijus was last seen on #p5p 1 hour and 32 minutes ago, saying: rofl
[09:50am] autrijus: you seek me?
[09:50am] modred: you didn't even need to say his name three times
[09:50am] xantus: seek(0)
[09:50am] Theory: autrijus: I do!
[09:51am] autrijus: pray tell(), why?
[09:51am] Theory: autrijus: Are you on perl-qa?
[09:51am] autrijus: nay, I am not
[09:51am] Theory: autrijus: Let me find an archive link for a unicode & Test::Builder discussion there.
[09:51am] Theory: I have a feeling you'll already be familiar with the issue...
[09:51am] autrijus: http://www.nntp.perl.org/group/perâl.qa/3404
[09:52am] Theory: autrijus: D'oh!
[09:52am] Theory: You're way ahead of me.
[09:52am] Theory: autrijus: What do you think?
[09:52am] purl: I think Theory should try flossing more often!
[09:52am] autrijus: give me a test case that involves big5?
[09:52am] Theory vomits on purl
[09:53am] Theory: autrijus: Hrm. I'd have to dig up some Big5.
[09:53am] Theory: autrijus: The problem won't come up with Big5 unless you binmode Test::Builder's FHs to utf8.
[09:54am] Theory: autrijus: Schwern was suggesting that they just always be utf8, but I thought that'd break things when you used non-utf8 characters.
[09:54am] autrijus: if it's always in utf8 and you send random binary data there
[09:54am] autrijus: it could be unhappy.
[09:54am] Theory: autrijus: Exactly, and that's when Schwern threw up his hands in disgust.
[09:54am] Theory: seen Schwern
[09:54am] purl: Schwern was last seen on #perl 24 minutes ago, saying: crab:Â I've read "Naked"
[09:55am] Schwern left the chat room. (Ping timeout: 240 seconds)
[09:56am] Theory: autrijus: So it seems to me that there needs to be some way to tell Test::Builder what binmode to use on its file handles.
[09:56am] autrijus: so T::B uses its own fh
[09:56am] autrijus: apart from STDOUT
[09:56am] autrijus: and it does not inherid STDOUT's layers.
[09:56am] autrijus: is it that?
[09:56am] Theory: autrijus: Yes, it dupes STDOUT and STDERR, but the duping doesn't preserve binmode.
[09:56am] Theory: right
[09:57am] autrijus: so it seems to me that the right fix is for the duping to fix binmode.
[09:57am] Theory: It does the duping so it can know what it's outputting to STDOUT (and STDERR) as opposed to what the scripts are outputting.
[09:57am] Theory: autrijus: Me too, but I don't know if there's a way to detect the layer assigned to a file handle. Do you?
[09:57am] autrijus: I do.
[09:57am] uri: Theory: i bet dupping occurs at the system level so binmode (which is perl i/o level) is lost
[09:58am] autrijus: I don't want to go there.
[09:58am] Theory: uri: Good point.
[09:58am] autrijus thinks.
[09:58am] Theory: autrijus: Uh, why not? What is it?
[09:58am] uri: Theory: perl prolly calls dup() or variant to do it
[09:58am] Theory: uri: Yeah
[09:59am] uri: and just returns a plain handle. it should be smart about copying i/o flags
[09:59am] autrijus: you can use a layer to read io layers.
[09:59am] uri: but you can write a smart dup sub
[09:59am] autrijus: I don't think there is an api for that.
[09:59am] hachi: dup sub!
[09:59am] autrijus: and even if we write one, T::B could not ship it
[09:59am] autrijus: because it's, well, XS
[10:01am] Theory: autrijus: I was afraid of that.
[10:01am] autrijus: and sometimes it does not make sense.
[10:01am] Theory: autrijus: So the workaround is to have some way to tell Test::Builder what mode to use.
[10:01am] autrijus: yes.
[10:02am] autrijus: and in this regard I agree your binmode proposal.
[10:02am] Theory: autrijus: Okay.
[10:02am] Theory: pokes schwern
Ovid: Autrijus: what about using Ingy's Devel::Pointer to get down to that flag info? It's pure Perl and relatively cross-platform.
[10:11am] hachi: you can't actually twiddle the flag with that, I thought
[10:12am] Ovid: But you should be able to read them and create a new handle with appropriate flags, yes?
[10:12am] Theory: uri: I agree that it should be smarter about copying i/o flags.
[10:12am] Theory: Maybe I should mention it on #p5p
[10:12am] crab: what are we talking about again?
[10:12am] hachi: Ovid: I think the problem is that you can't actually touch the FH that is in question here... otherwise they would just do a binmode() on it
[10:13am] Ovid: Oh, I take it back. Devel::Pointer is Cozens' module. It's http://search.cpan.org/~ingy/Poinâter-0.10/ that I meant.
[10:13am] Ovid: Yes, but aren't the filehandles getting duped in T::B? If so, you can read the flags on the handles they're getting duped from.
[10:14am] Theory: ovid: Yes, the ideal solution would be for Test::B to detect the flags on the STDERR and STDOUT file handles and copy them to its duped versions.
[10:14am] Theory: ovid: Even better would be if Perl did it for you, as uri mentioned.
Theory: crab: http://www.nntp.perl.org/group/peârl.qa/3404
[10:20am] autrijus: Ovid: nono, it's the layers not the flags
[10:20am] autrijus: layers is not flags.
[10:20am] autrijus: very different.
[10:20am] Ovid: autrijus, I have no idea what that means
[10:21am] autrijus: ok. so say you have a ':via(something)' layer
[10:21am] autrijus: that layer can't reliably be restored into the ':via(something)' string.
[10:21am] autrijus: it's not a simple string
[10:21am] autrijus: like a file mode
[10:21am] autrijus: it's an object that got associated with the fh.
[10:22am] clkao: especially with via::dynamic
[10:22am] autrijus: especially with clkao's via::dynamic, yes.
[10:22am] Ovid: Aha, I'm reading perldoc -f open and (as usual) learning a few things.
[10:22am] autrijus: perldoc perliol as well
[10:23am] autrijus: also, each layer keeps state.
[10:23am] autrijus: so simply duping the layers may result in state clash
[10:24am] autrijus: so that's why perlio layers has a "Dup" method
[10:24am] autrijus: which should theoretically be used as part of dup()...
[10:24am] autrijus: weird. it says that open with "&" to dup will dup the layers.
[10:25am] Theory: Uh, so should open's duping be using that dupe?
[10:25am] Theory: autrijus: That doesn't seem to work.
[10:25am] autrijus: perlbug!
Anyway, you get the idea. I think what needs to happen is one of the following:
1. Perl gets smarter about duping file handles, so that the dupes get the same i/o layer settings as the handles they dupe.
2. Test::Builder somehow detects the i/o flags on STDERR and STDOUT and applies them to the duped file handles. I have no idea how to do this.
3. Test::Builder (and the modules that depend on it) provides some method for users to tell it what binmodes they're going to use.
I rather like the third suggestion, since the first two would require code like this:
#!/usr/bin/perl -w
use strict;
BEGIN { binmode STDERR, ":utf8"; binmode STDOUT ":utf8"; }
use Test::More;That is, I'd have to remember to set the binmode before loading Test::More. Better would be, IMO:
use Test::More tests => 5, binmode => ":utf8";
Is this do-able?
Regards,
David
