Re: On memory consumption - request for comments
On 02/23/2011 11:40 PM, Torsten Förtsch wrote: > On Thursday, February 24, 2011 07:45:29 Max Kanat-Alexander wrote: >> Hey Fred. So given the discussion that we've had on this, do you think >> that the next version of SizeLimit could change its Linux behavior to >> return the more appropriate rss size? > > yes Awesome, thank you. :-) -Max -- http://www.bugzillasource.com/ Competent, Friendly Bugzilla, Perl, and IT Services
Re: On memory consumption - request for comments
On Thursday, February 24, 2011 09:29:23 Max Kanat-Alexander wrote:
> On 02/23/2011 11:40 PM, Torsten Förtsch wrote:
> > On Thursday, February 24, 2011 07:45:29 Max Kanat-Alexander wrote:
> >> Hey Fred. So given the discussion that we've had on this, do you
> >> think that the next version of SizeLimit could change its Linux
> >> behavior to return the more appropriate rss size?
> >
> >
> >
> > yes
>
> Awesome, thank you. :-)
Max, could you please check if the following patch does what you want and try
it out in your environment?
Index: lib/Apache/SizeLimit/Core.pm
===
--- lib/Apache/SizeLimit/Core.pm(revision 1069512)
+++ lib/Apache/SizeLimit/Core.pm(revision 1069513)
@@ -117,9 +117,9 @@
sub _check_size {
my $class = shift;
-my ($size, $share) = $class->_platform_check_size();
+my ($size, $share, $unshared) = $class->_platform_check_size();
-return ($size, $share, $size - $share);
+return ($size, $share, defined $unshared ? $unshared : $size - $share);
}
sub _load {
@@ -176,7 +176,9 @@
return $class->_linux_size_check() unless $USE_SMAPS;
my $s = Linux::Smaps->new($$)->all;
-return ($s->size, $s->shared_clean + $s->shared_dirty);
+return ($s->size,
+ $s->shared_clean + $s->shared_dirty,
+ $s->private_clean + $s->private_dirty);
}
sub _linux_size_check {
Torsten Förtsch
--
Need professional modperl support? Hire me! (http://foertsch.name)
Like fantasy? http://kabatinte.net
Factoring out Apache::SizeLimit::Core?
Hey folks. Would it be possible to factor the memory-sizing code in Apache::SizeLimit::Core into a separate module (or is there already a separate module that does something similar)? I'd love a module that can just reliably tell me how much memory a process is using across all platforms. -Max -- http://www.bugzillasource.com/ Competent, Friendly Bugzilla, Perl, and IT Services
Re: On memory consumption - request for comments
On 02/24/2011 12:41 AM, Torsten Förtsch wrote: > Max, could you please check if the following patch does what you want and try > it out in your environment? Hey Torsten. Yes, I tested it and it works. Thank you so much! :-) -Max -- http://www.bugzillasource.com/ Competent, Friendly Bugzilla, Perl, and IT Services
Re: On memory consumption - request for comments
What's the rough ETA before this makes it to CPAN?
Working out if I want to add the patch to our build process or not...
On 24 Feb 2011, at 08:41, Torsten Förtsch wrote:
> On Thursday, February 24, 2011 09:29:23 Max Kanat-Alexander wrote:
>> On 02/23/2011 11:40 PM, Torsten Förtsch wrote:
>>> On Thursday, February 24, 2011 07:45:29 Max Kanat-Alexander wrote:
Hey Fred. So given the discussion that we've had on this, do you
think that the next version of SizeLimit could change its Linux
behavior to return the more appropriate rss size?
>>>
>>>
>>>
>>> yes
>>
>>Awesome, thank you. :-)
>
> Max, could you please check if the following patch does what you want and try
> it out in your environment?
>
> Index: lib/Apache/SizeLimit/Core.pm
> ===
> --- lib/Apache/SizeLimit/Core.pm(revision 1069512)
> +++ lib/Apache/SizeLimit/Core.pm(revision 1069513)
> @@ -117,9 +117,9 @@
> sub _check_size {
> my $class = shift;
>
> -my ($size, $share) = $class->_platform_check_size();
> +my ($size, $share, $unshared) = $class->_platform_check_size();
>
> -return ($size, $share, $size - $share);
> +return ($size, $share, defined $unshared ? $unshared : $size - $share);
> }
>
> sub _load {
> @@ -176,7 +176,9 @@
> return $class->_linux_size_check() unless $USE_SMAPS;
>
> my $s = Linux::Smaps->new($$)->all;
> -return ($s->size, $s->shared_clean + $s->shared_dirty);
> +return ($s->size,
> + $s->shared_clean + $s->shared_dirty,
> + $s->private_clean + $s->private_dirty);
> }
>
> sub _linux_size_check {
>
>
> Torsten Förtsch
>
> --
> Need professional modperl support? Hire me! (http://foertsch.name)
>
> Like fantasy? http://kabatinte.net
Re: [Mason] ANNOUNCE: Mason 2
oops is there any big player using Mason these days? On Tue, Feb 22, 2011 at 2:24 AM, Perrin Harkins wrote: > In case any of you Mason users on the mod_perl list aren't on the Mason > list... > > - Perrin > > -- Forwarded message -- > From: Jonathan Swartz > Date: Mon, Feb 21, 2011 at 11:16 AM > Subject: [Mason] ANNOUNCE: Mason 2 > To: Mason-Users List > > > I'm pleased to announce Mason 2, the first major version of Mason in ten > years: > > http://search.cpan.org/perldoc?Mason > > Mason 2 has been rearchitected and reimplemented from the ground up, > to take advantage of modern Perl techniques (Moose, Plack/PSGI) and to > correct long-standing feature and syntax inadequacies. Its new > foundations should allow its performance and flexibility to far exceed > Mason 1. > > Though little original code or documentation remains, Mason's core > philosophy is intact; it should still "feel like Mason" to existing > users. > > Major changes: > > * Name. The name is now Mason, instead of HTML::Mason. > > * Component classes. Each component is represented by its own (Moose) > class, rather than just an instance of a common class. This means that > components have their own namespaces, subroutines, methods, and > attributes, and can truly inherit from one other. See > http://search.cpan.org/perldoc?Mason::Manual::Components > > * Filters. A single powerful filter syntax and mechanism consolidates > three separate filter mechanisms from Mason 1 (filter blocks, > components with content, and escape flags). See > http://search.cpan.org/perldoc?Mason::Manual::Filters > > * Plugins. Moose roles are utilized to create a flexible plugin system > that can modify nearly every aspect of Mason's operation. Previously > core features such as caching can now be implemented in plugins. See > http://search.cpan.org/perldoc?Mason::Manual::Plugins > > * Web integration. Mason 1's bulky custom web handling code > (ApacheHandler, CGIHandler) has been replaced with a simple PSGI > handler and with plugins for web frameworks like Catalyst and Dancer. > The core Mason distribution is now completely web-agnostic. See > http://search.cpan.org/perldoc?Mason::Plugin::PSGIHandler > > * File naming. Mason now facilitates and enforces (in a customizable > way) standard file extensions for components: .m (top-level > components), .mi (internal components), and .pm (pure-perl > components). > > See http://search.cpan.org/perldoc?Mason::Manual::UpgradingFromMason1 > for a more detailed list of changes. > > Mason 2 is obviously still in alpha status, but it has a fair sized > test suite and I'm eager to start building web projects with it. I > hope you'll give it a try and let us know what you think! > > Best > Jon >
Re: On memory consumption - request for comments
On Thursday, February 24, 2011 11:27:42 Dave Hodgkinson wrote: > What's the rough ETA before this makes it to CPAN? end of April, perhaps. Torsten Förtsch -- Need professional modperl support? Hire me! (http://foertsch.name) Like fantasy? http://kabatinte.net
Re: [Mason] ANNOUNCE: Mason 2
Lovefilm big enough for you? On 24 Feb 2011, at 10:49, xiaolan wrote: > oops is there any big player using Mason these days? > > On Tue, Feb 22, 2011 at 2:24 AM, Perrin Harkins wrote: >> In case any of you Mason users on the mod_perl list aren't on the Mason >> list... >> >> - Perrin >> >> -- Forwarded message -- >> From: Jonathan Swartz >> Date: Mon, Feb 21, 2011 at 11:16 AM >> Subject: [Mason] ANNOUNCE: Mason 2 >> To: Mason-Users List >> >> >> I'm pleased to announce Mason 2, the first major version of Mason in ten >> years: >> >>http://search.cpan.org/perldoc?Mason >> >> Mason 2 has been rearchitected and reimplemented from the ground up, >> to take advantage of modern Perl techniques (Moose, Plack/PSGI) and to >> correct long-standing feature and syntax inadequacies. Its new >> foundations should allow its performance and flexibility to far exceed >> Mason 1. >> >> Though little original code or documentation remains, Mason's core >> philosophy is intact; it should still "feel like Mason" to existing >> users. >> >> Major changes: >> >> * Name. The name is now Mason, instead of HTML::Mason. >> >> * Component classes. Each component is represented by its own (Moose) >> class, rather than just an instance of a common class. This means that >> components have their own namespaces, subroutines, methods, and >> attributes, and can truly inherit from one other. See >> http://search.cpan.org/perldoc?Mason::Manual::Components >> >> * Filters. A single powerful filter syntax and mechanism consolidates >> three separate filter mechanisms from Mason 1 (filter blocks, >> components with content, and escape flags). See >> http://search.cpan.org/perldoc?Mason::Manual::Filters >> >> * Plugins. Moose roles are utilized to create a flexible plugin system >> that can modify nearly every aspect of Mason's operation. Previously >> core features such as caching can now be implemented in plugins. See >> http://search.cpan.org/perldoc?Mason::Manual::Plugins >> >> * Web integration. Mason 1's bulky custom web handling code >> (ApacheHandler, CGIHandler) has been replaced with a simple PSGI >> handler and with plugins for web frameworks like Catalyst and Dancer. >> The core Mason distribution is now completely web-agnostic. See >> http://search.cpan.org/perldoc?Mason::Plugin::PSGIHandler >> >> * File naming. Mason now facilitates and enforces (in a customizable >> way) standard file extensions for components: .m (top-level >> components), .mi (internal components), and .pm (pure-perl >> components). >> >> See http://search.cpan.org/perldoc?Mason::Manual::UpgradingFromMason1 >> for a more detailed list of changes. >> >> Mason 2 is obviously still in alpha status, but it has a fair sized >> test suite and I'm eager to start building web projects with it. I >> hope you'll give it a try and let us know what you think! >> >> Best >> Jon >>
Re: [Mason] ANNOUNCE: Mason 2
On Thu, Feb 24, 2011 at 5:49 AM, xiaolan wrote: > oops is there any big player using Mason these days? Amazon.com. And this list is probably dated, but I'm sure some of them still exist: http://masonhq.com/?MasonPoweredSites I work on some sites using Mason, including MoveOn.org, ONE.org, and credoaction.com. - Perrin
Re: [Mason] ANNOUNCE: Mason 2
Four years ago, Amazon.com was completing a huge conversion to a new site platform including Mason as the primary templating language. While they may be moving away from it in newer development, I'd have to guess a substantial part of the site still uses it. They still mention Mason in many current job postings. Jon On Feb 24, 2011, at 5:06 AM, Dave Hodgkinson wrote: > > Lovefilm big enough for you? > > On 24 Feb 2011, at 10:49, xiaolan wrote: > >> oops is there any big player using Mason these days? >> >> On Tue, Feb 22, 2011 at 2:24 AM, Perrin Harkins wrote: >>> In case any of you Mason users on the mod_perl list aren't on the Mason >>> list... >>> >>> - Perrin >>> >>> -- Forwarded message -- >>> From: Jonathan Swartz >>> Date: Mon, Feb 21, 2011 at 11:16 AM >>> Subject: [Mason] ANNOUNCE: Mason 2 >>> To: Mason-Users List >>> >>> >>> I'm pleased to announce Mason 2, the first major version of Mason in ten >>> years: >>> >>> http://search.cpan.org/perldoc?Mason >>> >>> Mason 2 has been rearchitected and reimplemented from the ground up, >>> to take advantage of modern Perl techniques (Moose, Plack/PSGI) and to >>> correct long-standing feature and syntax inadequacies. Its new >>> foundations should allow its performance and flexibility to far exceed >>> Mason 1. >>> >>> Though little original code or documentation remains, Mason's core >>> philosophy is intact; it should still "feel like Mason" to existing >>> users. >>> >>> Major changes: >>> >>> * Name. The name is now Mason, instead of HTML::Mason. >>> >>> * Component classes. Each component is represented by its own (Moose) >>> class, rather than just an instance of a common class. This means that >>> components have their own namespaces, subroutines, methods, and >>> attributes, and can truly inherit from one other. See >>> http://search.cpan.org/perldoc?Mason::Manual::Components >>> >>> * Filters. A single powerful filter syntax and mechanism consolidates >>> three separate filter mechanisms from Mason 1 (filter blocks, >>> components with content, and escape flags). See >>> http://search.cpan.org/perldoc?Mason::Manual::Filters >>> >>> * Plugins. Moose roles are utilized to create a flexible plugin system >>> that can modify nearly every aspect of Mason's operation. Previously >>> core features such as caching can now be implemented in plugins. See >>> http://search.cpan.org/perldoc?Mason::Manual::Plugins >>> >>> * Web integration. Mason 1's bulky custom web handling code >>> (ApacheHandler, CGIHandler) has been replaced with a simple PSGI >>> handler and with plugins for web frameworks like Catalyst and Dancer. >>> The core Mason distribution is now completely web-agnostic. See >>> http://search.cpan.org/perldoc?Mason::Plugin::PSGIHandler >>> >>> * File naming. Mason now facilitates and enforces (in a customizable >>> way) standard file extensions for components: .m (top-level >>> components), .mi (internal components), and .pm (pure-perl >>> components). >>> >>> See http://search.cpan.org/perldoc?Mason::Manual::UpgradingFromMason1 >>> for a more detailed list of changes. >>> >>> Mason 2 is obviously still in alpha status, but it has a fair sized >>> test suite and I'm eager to start building web projects with it. I >>> hope you'll give it a try and let us know what you think! >>> >>> Best >>> Jon >>> >
Re: [Mason] ANNOUNCE: Mason 2
Also, I realize now that "first major version in ten years" was a potentially misleading choice of words. Mason was actively developed until 2006, and has had maintenance releases since then. It's just that 1.00 was released ten years ago. On Feb 24, 2011, at 2:49 AM, xiaolan wrote: > oops is there any big player using Mason these days? > > On Tue, Feb 22, 2011 at 2:24 AM, Perrin Harkins wrote: >> In case any of you Mason users on the mod_perl list aren't on the Mason >> list... >> >> - Perrin >> >> -- Forwarded message -- >> From: Jonathan Swartz >> Date: Mon, Feb 21, 2011 at 11:16 AM >> Subject: [Mason] ANNOUNCE: Mason 2 >> To: Mason-Users List >> >> >> I'm pleased to announce Mason 2, the first major version of Mason in ten >> years: >> >>http://search.cpan.org/perldoc?Mason >> >> Mason 2 has been rearchitected and reimplemented from the ground up, >> to take advantage of modern Perl techniques (Moose, Plack/PSGI) and to >> correct long-standing feature and syntax inadequacies. Its new >> foundations should allow its performance and flexibility to far exceed >> Mason 1. >> >> Though little original code or documentation remains, Mason's core >> philosophy is intact; it should still "feel like Mason" to existing >> users. >> >> Major changes: >> >> * Name. The name is now Mason, instead of HTML::Mason. >> >> * Component classes. Each component is represented by its own (Moose) >> class, rather than just an instance of a common class. This means that >> components have their own namespaces, subroutines, methods, and >> attributes, and can truly inherit from one other. See >> http://search.cpan.org/perldoc?Mason::Manual::Components >> >> * Filters. A single powerful filter syntax and mechanism consolidates >> three separate filter mechanisms from Mason 1 (filter blocks, >> components with content, and escape flags). See >> http://search.cpan.org/perldoc?Mason::Manual::Filters >> >> * Plugins. Moose roles are utilized to create a flexible plugin system >> that can modify nearly every aspect of Mason's operation. Previously >> core features such as caching can now be implemented in plugins. See >> http://search.cpan.org/perldoc?Mason::Manual::Plugins >> >> * Web integration. Mason 1's bulky custom web handling code >> (ApacheHandler, CGIHandler) has been replaced with a simple PSGI >> handler and with plugins for web frameworks like Catalyst and Dancer. >> The core Mason distribution is now completely web-agnostic. See >> http://search.cpan.org/perldoc?Mason::Plugin::PSGIHandler >> >> * File naming. Mason now facilitates and enforces (in a customizable >> way) standard file extensions for components: .m (top-level >> components), .mi (internal components), and .pm (pure-perl >> components). >> >> See http://search.cpan.org/perldoc?Mason::Manual::UpgradingFromMason1 >> for a more detailed list of changes. >> >> Mason 2 is obviously still in alpha status, but it has a fair sized >> test suite and I'm eager to start building web projects with it. I >> hope you'll give it a try and let us know what you think! >> >> Best >> Jon >>
Re: Factoring out Apache::SizeLimit::Core?
Oh, maybe Proc::Processtable is what you want. - Perrin On Thu, Feb 24, 2011 at 8:46 AM, Perrin Harkins wrote: > Hi Max, > > I see a couple of things out there, like Memory::Usage. They don't > look as comprehensive though. It would be fine to take code from > SizeLimit if you want to put it into some other module. It should > probably be distributed outside of mod_perl to make it useful in the > way you're describing. > > - Perrin > > On Thu, Feb 24, 2011 at 3:42 AM, Max Kanat-Alexander > wrote: >> Hey folks. Would it be possible to factor the memory-sizing code in >> Apache::SizeLimit::Core into a separate module (or is there already a >> separate module that does something similar)? I'd love a module that can >> just reliably tell me how much memory a process is using across all >> platforms. >> >> -Max >> -- >> http://www.bugzillasource.com/ >> Competent, Friendly Bugzilla, Perl, and IT Services >> >
Re: Factoring out Apache::SizeLimit::Core?
Hi Max, I see a couple of things out there, like Memory::Usage. They don't look as comprehensive though. It would be fine to take code from SizeLimit if you want to put it into some other module. It should probably be distributed outside of mod_perl to make it useful in the way you're describing. - Perrin On Thu, Feb 24, 2011 at 3:42 AM, Max Kanat-Alexander wrote: > Hey folks. Would it be possible to factor the memory-sizing code in > Apache::SizeLimit::Core into a separate module (or is there already a > separate module that does something similar)? I'd love a module that can > just reliably tell me how much memory a process is using across all > platforms. > > -Max > -- > http://www.bugzillasource.com/ > Competent, Friendly Bugzilla, Perl, and IT Services >
Re: Apache::DBI and DBD::mysql ping not working
On Wed, Feb 23, 2011 at 4:07 PM, Daniel Manley wrote: > well, I think I figured this one out on my own. Though I've used Perl for a > number of years, I've never gotten to the XS file coding. and so after > running this under perl debug, I finally get the hint that maybe the mysql > ping() is done in C. And so it is. FYI, you should be able to define your own ping() and see it called. The Apache::DBI docs show how. - Perrin
Re: Factoring out Apache::SizeLimit::Core?
On 02/24/2011 05:47 AM, Perrin Harkins wrote: > Oh, maybe Proc::Processtable is what you want. Ah, no, it doesn't work on Win32 as far as I can tell. (Working on Win32 is one of my requirements.) SizeLimit is the only module I've been able to find that can reliably determine process size across platforms. -Max -- http://www.bugzillasource.com/ Competent, Friendly Bugzilla, Perl, and IT Services
Re: On memory consumption - request for comments
2011/2/24 Torsten Förtsch : > On Thursday, February 24, 2011 11:27:42 Dave Hodgkinson wrote: >> What's the rough ETA before this makes it to CPAN? > > end of April, perhaps. We can release this earlier than 2.0.6 if there are enough testers to give feedback.
Re: On memory consumption - request for comments
On 24 Feb 2011, at 15:44, Fred Moyer wrote: > 2011/2/24 Torsten Förtsch : >> On Thursday, February 24, 2011 11:27:42 Dave Hodgkinson wrote: >>> What's the rough ETA before this makes it to CPAN? >> >> end of April, perhaps. > > We can release this earlier than 2.0.6 if there are enough testers to > give feedback. I'll ask here.
Re: [Mason] ANNOUNCE: Mason 2
Jonathan Swartz wrote: Also, I realize now that "first major version in ten years" was a potentially misleading choice of words. Mason was actively developed until 2006, and has had maintenance releases since then. It's just that 1.00 was released ten years ago. Hi. I do not really know Mason (yet), but I just want to say that it is an /immense/ tribute to its developer(s) that a package that was developed more than 10 years ago is still in active use now, by many sites, and many very large sites. (The same goes for mod_perl of course). And when one compares the average of 2-3 minor releases per year over the last 5 years, with the monthly MS patches ... (http://www.microsoft.com/technet/security/bulletin/ms11-feb.mspx) This being said, I would like to report a small bug, I believe more of the web server than Mason, on the very reference page mentioned by Perrin : http://masonhq.com/code/history.html The last line of the "1.43 Dec 25, 2009" section shows an encoding problem. The name of Frédéric Brière seems to be encoded as UTF-8, but the page is rendered in iso-8859-1. The Content-type header returned by the server does not provide a character encoding (and neither does the html page itself) so the browser defaults to iso-8859-1. Since this is a "marketing" kind of page, it would look nicer if it appeared correctly. ;-) (Request-Line) GET /code/history.html HTTP/1.1 (Status-Line) HTTP/1.1 200 OK DateThu, 24 Feb 2011 21:55:59 GMT Server Apache/1.3.33 (Unix) mod_perl/1.29 Content-Typetext/html X-Cache MISS from www.masonhq.com Keep-Alive timeout=15, max=100 Connection Keep-Alive Transfer-Encoding chunked
CGI and character encoding
Hi.
I wonder if someone here can give me a clue as to where to look...
I am using
Apache/2.2.9 (Debian) DAV/2 SVN/1.5.1 mod_jk/1.2.26 PHP/5.2.6-1+lenny9 with Suhosin-Patch
mod_ssl/2.2.9 OpenSSL/0.9.8g mod_apreq2-20051231/2.6.0 mod_perl/2.0.4 Perl/v5.10.0
perl -MCGI -e 'print $CGI::VERSION'
3.52
A perl cgi-bin script running under mod_perl, receives posted form parameters from a form
defined as such :
http://www.w3.org/TR/html4/loose.dtd";>
...
...
(Note: the html page itself has been saved as UTF-8 by an UTF-8 aware editor)
When I retrieve the above hidden field using
my $chars = $cgi->param('de-utf8');
the variable $chars does contain the proper UTF-8 encoded *bytes* for the above string (in
other words, 2 bytes per character e.g.), but it arrives into the script /without/ the
perl "utf8" flag set.
If I then use this value to print to a filehandle opened as such :
open(FH,'>:utf8',"myfile");
print FH $chars,"\n";
It comes out of course as .. well, I cannot type this on my keyboard, but anyone aware of
double-encoding issues can imagine the "A-tilde Copyright A-tilde squiggle.. " result.
I can of course convert it, by using
$chars = Encode::decode('utf8',$cgi->param('de-utf8'));
but it is a p.i.t.a. and I would like to know if there is a way to retrieve the posted
value directly as UTF-8, and if yes what this depends on.
(I cannot find a setting for instance in the CGI.pm module documentation.)
Thanks.
André
P.S.
Unfortunately, when the browser (Firefox 3.5.3) is posting this data to the server, it is
posting it as something like
...
Content-Typemultipart/form-data;
boundary=---326972172326727
...
-326972172326727
Content-Disposition: form-data; name="de-utf8"
̟̊̚
-326972172326727
which means that there is no charset header to the parts either.
Re: CGI and character encoding
On 02/24/2011 04:31 PM, André Warnier wrote: I wonder if someone here can give me a clue as to where to look... The CGI.pm documentation talks about the -utf8 import flag which is probably what you're looking for. But it does caution not to use it for anything that needs to do file uploads. -- Michael Peters Plus Three, LP
Re: CGI and character encoding
Hi André,
There is a perlmonks post from a few years ago that explains one way
of automating this with CGI.pm. I've used this for several years now
without problems.
http://www.perlmonks.org/?node_id=651574
Just remember that decoding params is just one part of dealing with
utf-8. You need to worry about any data coming into or going out of
your app (reading files, retrieving from DB, send HTML out to the
browser, etc...). The following wiki book has some great information
on how to deal with utf-8 in your perl applications (and it also
includes the CGI.pm hack from Rhesa that I linked to above in the
perlmonks link).
http://en.wikibooks.org/wiki/Perl_Programming/Unicode_UTF-8
Cheers,
Cees Hek
On Fri, Feb 25, 2011 at 8:31 AM, André Warnier wrote:
> Hi.
>
> I wonder if someone here can give me a clue as to where to look...
>
> I am using
> Apache/2.2.9 (Debian) DAV/2 SVN/1.5.1 mod_jk/1.2.26 PHP/5.2.6-1+lenny9 with
> Suhosin-Patch mod_ssl/2.2.9 OpenSSL/0.9.8g mod_apreq2-20051231/2.6.0
> mod_perl/2.0.4 Perl/v5.10.0
>
> perl -MCGI -e 'print $CGI::VERSION'
> 3.52
>
> A perl cgi-bin script running under mod_perl, receives posted form
> parameters from a form defined as such :
>
> "http://www.w3.org/TR/html4/loose.dtd";>
>
>
>
>
>
> enctype="multipart/form-data" charset="UTF-8" method="POST">
> ...
>
> ...
>
> (Note: the html page itself has been saved as UTF-8 by an UTF-8 aware
> editor)
>
>
> When I retrieve the above hidden field using
>
> my $chars = $cgi->param('de-utf8');
>
> the variable $chars does contain the proper UTF-8 encoded *bytes* for the
> above string (in other words, 2 bytes per character e.g.), but it arrives
> into the script /without/ the perl "utf8" flag set.
>
> If I then use this value to print to a filehandle opened as such :
>
> open(FH,'>:utf8',"myfile");
> print FH $chars,"\n";
>
> It comes out of course as .. well, I cannot type this on my keyboard, but
> anyone aware of double-encoding issues can imagine the "A-tilde Copyright
> A-tilde squiggle.. " result.
>
> I can of course convert it, by using
>
> $chars = Encode::decode('utf8',$cgi->param('de-utf8'));
>
> but it is a p.i.t.a. and I would like to know if there is a way to retrieve
> the posted value directly as UTF-8, and if yes what this depends on.
> (I cannot find a setting for instance in the CGI.pm module documentation.)
>
>
> Thanks.
> André
>
> P.S.
> Unfortunately, when the browser (Firefox 3.5.3) is posting this data to the
> server, it is posting it as something like
>
> ...
> Content-Type multipart/form-data;
> boundary=---326972172326727
> ...
>
> -326972172326727
> Content-Disposition: form-data; name="de-utf8"
>
> ÄäÖöÜü
> -326972172326727
>
> which means that there is no charset header to the parts either.
>
RE: CGI and character encoding
FWIW, with CGI.pm I always iterate through the params and Encode::decode with
the appropriate encoding with an exception for anything binary. (file uploads
etc)
-Original Message-
From: André Warnier [mailto:a...@ice-sa.com]
Sent: Thursday, February 24, 2011 3:31 PM
To: mod_perl list
Subject: CGI and character encoding
Hi.
I wonder if someone here can give me a clue as to where to look...
I am using
Apache/2.2.9 (Debian) DAV/2 SVN/1.5.1 mod_jk/1.2.26 PHP/5.2.6-1+lenny9 with
Suhosin-Patch
mod_ssl/2.2.9 OpenSSL/0.9.8g mod_apreq2-20051231/2.6.0 mod_perl/2.0.4
Perl/v5.10.0
perl -MCGI -e 'print $CGI::VERSION'
3.52
A perl cgi-bin script running under mod_perl, receives posted form parameters
from a form
defined as such :
http://www.w3.org/TR/html4/loose.dtd";>
...
...
(Note: the html page itself has been saved as UTF-8 by an UTF-8 aware editor)
When I retrieve the above hidden field using
my $chars = $cgi->param('de-utf8');
the variable $chars does contain the proper UTF-8 encoded *bytes* for the above
string (in
other words, 2 bytes per character e.g.), but it arrives into the script
/without/ the
perl "utf8" flag set.
If I then use this value to print to a filehandle opened as such :
open(FH,'>:utf8',"myfile");
print FH $chars,"\n";
It comes out of course as .. well, I cannot type this on my keyboard, but
anyone aware of
double-encoding issues can imagine the "A-tilde Copyright A-tilde squiggle.. "
result.
I can of course convert it, by using
$chars = Encode::decode('utf8',$cgi->param('de-utf8'));
but it is a p.i.t.a. and I would like to know if there is a way to retrieve the
posted
value directly as UTF-8, and if yes what this depends on.
(I cannot find a setting for instance in the CGI.pm module documentation.)
Thanks.
André
P.S.
Unfortunately, when the browser (Firefox 3.5.3) is posting this data to the
server, it is
posting it as something like
...
Content-Typemultipart/form-data;
boundary=---326972172326727
...
-326972172326727
Content-Disposition: form-data; name="de-utf8"
̟̊̚
-326972172326727
which means that there is no charset header to the parts either.
Re: CGI and character encoding
Michael Peters wrote: On 02/24/2011 04:31 PM, André Warnier wrote: I wonder if someone here can give me a clue as to where to look... The CGI.pm documentation talks about the -utf8 import flag which is probably what you're looking for. But it does caution not to use it for anything that needs to do file uploads. Thanks. My workstation version of the CGI documentation is apparently outdated, and did not mention that "pragma". The CPAN version does. But yes, I will need file uploads too, and since there is no telling how exactly the -utf8 flag interferes with them, I think I'll stick with the p.i.t.a. method for now. I wonder why browsers do not put a charset parameter in the multipart/form-data parts.. It would seem like a logical and MIME-conformant thing to do.
Re: Apache::DBI and DBD::mysql ping not working
On Thu, 24 Feb 2011 03:28:19 +1100, Daniel Manley
wrote:
I've been digging around in my mod_perl-based apps and trying to figure
out why setting up the DB connections for pinging is still randomly
producing first-thing-in-the-morning "mysql server has gone away"
errors. I read up about the morning bug and such (adding ping() to
mysql.pm based on examples on the net) and still can't get it working.
I'm using apache 2.2.17 and mod_perl 2.0.4.
I would suggest looking into your mysql server "wait_timeout".
Calling $dbh->ping() on a handle is not enough to have
seamless recover from stale db connections. At least on MySQL.
Our code, simplifying, looks like:
- 8< ---
sub is_alive {
my ($dbh) = @_;
return if not $dbh;
if (not eval { $dbh->FETCH('Active') && $dbh->ping() }) {
warn "Found stale dbh handle: $dbh ($@)\n";
return;
}
return 1;
}
sub db_connect {
my $dbh = cached_dbh();
return $dbh if is_alive($dbh);
$dbh = new_db_connection();
return $dbh;
}
- 8< ---
--
Cosimo
[RELEASE CANDIDATE] Apache-SizeLimit-0.96 RC1
Please download, test, and report back. Changes include a change to the unshared size measurement. See the thread here for details - http://www.gossamer-threads.com/lists/modperl/modperl/102674 http://people.apache.org/~phred/Apache-SizeLimit-0.96-rc1.tar.gz MD5 (Apache-SizeLimit-0.96-rc1.tar.gz) = 4848428e3d9aa412c2aebed0462d4886 Changes: Require Apache::Test 1.36 [Fred] Unshared size is now interpreted as RSS - shared instead of VSIZE - shared on Linux [Torsten] Subtest 1 checks that Apache2::SizeLimit->_limits_are_exceeded() returns false without any limits. But if the test runs near the end of the test suite it may very well be that some other test has set a limit. [Torsten]
Re: CGI and character encoding
On 02/24/2011 03:31 PM, André Warnier wrote: > Hi. > > I wonder if someone here can give me a clue as to where to look... If you have a fairly recent CGI.pm, it will decode utf-8 properly for you (even avoiding double-decoding), but there are some caveats. In addition to what others have already said, If you are running under mod_perl (which obviously you are), CGI.pm adds a cleanup handler (via register_cleanup) which resets CGI.pm's global variables. One of the variables that gets reset is the PARAM_UTF8 variable (which the -utf8 import controls). Because of this, once the clenaup handler gets called, UTF-8 decoding gets turned off. You have to work around this by manually making sure $CGI::PARAM_UTF8 = 1 before calling CGI->new. Regards, Michael Schout
