Re: [HACKERS] Careful PL/Perl Release Not Required

2011-02-11 Thread David E. Wheeler
On Feb 11, 2011, at 3:58 PM, Alex Hunsaker wrote: > You mean... we have been talking past each other this whole time? Well, since my second post, I think. I was wrong in the first one. > Olegs case _was_ a utf8 database. > From his original bug: > >>> Hi there, below is the problem, which I don

Re: [HACKERS] Careful PL/Perl Release Not Required

2011-02-11 Thread Alex Hunsaker
On Fri, Feb 11, 2011 at 16:42, David E. Wheeler wrote: > On Feb 11, 2011, at 12:57 PM, Alex Hunsaker wrote: > >> Yay! 1 > > Yes, this is all good. But it still seems to me that: > > * If your database is neither utf-8 no sql_ascii You mean... we have been talking past each other this whole time?

Re: [HACKERS] Careful PL/Perl Release Not Required

2011-02-11 Thread David E. Wheeler
On Feb 11, 2011, at 12:57 PM, Alex Hunsaker wrote: > Yay! 1 Yes, this is all good. But it still seems to me that: * If your database is neither utf-8 no sql_ascii * And your PL/Perl functions expect arguments that are byte soup * Once you upgrade to 9.1 they won't be * So you'll need to encode t

Re: [HACKERS] Careful PL/Perl Release Not Required

2011-02-11 Thread Alex Hunsaker
On Fri, Feb 11, 2011 at 11:07, David E. Wheeler wrote: > I don't understand where the bug is. If a string is encoded in utf-8 Perl > will not treat it as such unless the utf-8 flag is set. Ok so I think we agreed this is right: $ perl -E 'use URI::Escape; my $str = uri_unescape("%C3%A9"); say s

Re: [HACKERS] Careful PL/Perl Release Not Required

2011-02-11 Thread David E. Wheeler
On Feb 11, 2011, at 10:01 AM, Alex Hunsaker wrote: >> That *looks* like it is decoding the input string, which it is, but >> actually that will double utf8 encode your string. It does not seem to >> in this case because we are dealing with all ascii input. The trick >> here is its also telling per

Re: [HACKERS] Careful PL/Perl Release Not Required

2011-02-11 Thread David E. Wheeler
On Feb 11, 2011, at 9:34 AM, Andrew Dunstan wrote: > Is there an action item here? Is the documentation inadequate or inaccurate? > If so, please make a suggestion for improved wording. > > I certainly expect the change to be covered in the release notes. You can > check on the prominence given

Re: [HACKERS] Careful PL/Perl Release Not Required

2011-02-11 Thread David E. Wheeler
On Feb 11, 2011, at 9:44 AM, Alex Hunsaker wrote: > It is decoded... the input string "%C3%A9" actually is the _same_ > string utf-8, latin1 and SQL_ASCII decoded or not. Those are all ascii > characters. Calling utf8::decode("%C3%A9") is essentially a noop. No, it's not decoded. It doesn't matte

Re: [HACKERS] Careful PL/Perl Release Not Required

2011-02-11 Thread Alex Hunsaker
On Fri, Feb 11, 2011 at 10:44, Alex Hunsaker wrote: > On Fri, Feb 11, 2011 at 10:16, David E. Wheeler wrote: > That *looks* like it is decoding the input string, which it is, but > actually that will double utf8 encode your string. It does not seem to > in this case because we are dealing with a

Re: [HACKERS] Careful PL/Perl Release Not Required

2011-02-11 Thread Alex Hunsaker
On Fri, Feb 11, 2011 at 10:16, David E. Wheeler wrote: > On Feb 10, 2011, at 11:43 PM, Alex Hunsaker wrote: > Like I said, the terminology is awful. Yeah I use encode and decode to mean the same thing frequently :-(. >> In the the cited case he was passing "%C3%A9" to uri_unescape() and >> expe

Re: [HACKERS] Careful PL/Perl Release Not Required

2011-02-11 Thread Andrew Dunstan
On 02/11/2011 12:16 PM, David E. Wheeler wrote: [long discussion elided] Is there an action item here? Is the documentation inadequate or inaccurate? If so, please make a suggestion for improved wording. I certainly expect the change to be covered in the release notes. You can check on the

Re: [HACKERS] Careful PL/Perl Release Not Required

2011-02-11 Thread David E. Wheeler
On Feb 10, 2011, at 11:43 PM, Alex Hunsaker wrote: > I'd like to quibble with you over this point if I may. :-) > Per perldoc: JSON::XS > "utf8" flag disabled > When "utf8" is disabled (the default), then > "encode"/"decode" generate and expect Unicode strings ... > > So > - If you are

Re: [HACKERS] Careful PL/Perl Release Not Required

2011-02-10 Thread Alex Hunsaker
On Thu, Feb 10, 2011 at 21:53, David E. Wheeler wrote: > On Feb 10, 2011, at 5:28 PM, Alex Hunsaker wrote: >> The other thing that changed is non UTF-8 databases now also get >> character semantics. That is we convert from the database encoding >> into utf8 and visa versa on output. That probably

Re: [HACKERS] Careful PL/Perl Release Not Required

2011-02-10 Thread David E. Wheeler
On Feb 10, 2011, at 5:28 PM, Alex Hunsaker wrote: > Hrm? For UTF-8 databases, in practice, nothing should have changed-- > we already passed strings in as utf8. What I fixed was some corner > cases where some strings did not always have character semantics. See > The "Unicode Bug" and "Forcing Uni

Re: [HACKERS] Careful PL/Perl Release Not Required

2011-02-10 Thread Alex Hunsaker
On Thu, Feb 10, 2011 at 16:28, David E. Wheeler wrote: > Hackers, > > With regard to this (very welcome) commit: > >> commit 50d89d422f9c68a52a6964e5468e8eb4f90b1d95 >> Author: Andrew Dunstan >> Date:   Sun Feb 6 17:29:26 2011 -0500 >> >>     Force strings passed to and from plperl to be in UTF8

[HACKERS] Careful PL/Perl Release Not Required

2011-02-10 Thread David E. Wheeler
Hackers, With regard to this (very welcome) commit: > commit 50d89d422f9c68a52a6964e5468e8eb4f90b1d95 > Author: Andrew Dunstan > Date: Sun Feb 6 17:29:26 2011 -0500 > > Force strings passed to and from plperl to be in UTF8 encoding. > > String are converted to UTF8 on the way int