Re: Module caching
Scott [EMAIL PROTECTED] writes: Hello all, I am working on a large modperl app, and one of the features of this is a plugin system that allows others to write and install modules. Everything is good as far as this goes, but the problem is updateing/deleting modules. It seems as though the code is cached until an apache restart (i.e code changes don't take effect, version numbers don't change). Is there a way to flush the INC hash of all the children programmatically, without a restart? I have looked at Apache::Reload and Apache::StatINC and tried to replicate the code, but it doesn't seem to be working. The best thing I happened to meet here: - apache compiled with mod_perl as DSO (for instance debian linux version) - graceful restart which is invisible for the clients but reloads perl interpreter when mod_perl is DSO
Re: each considered harmful?
Calling keys() (or values()) in void context is quite efficient.
Nice to now. So it seems the correct idiom for using each is:
keys %hash;
while( my($k,$v) = each %hash ) {
...
}
p.s. I've shown an example, so we can have this issue documented
together with other mod_perl gotchas. It's interesting that I don't
remember this issue being reported earlier.
Maybe it could be of some interest where I happened to get this
problem. I got it in my custom TransHandler implemented to reduce stat
calls and to obtain 'select from multiple directories' effect.
... (config file)
our %STATIC_FILES = (
'/img' = [ '/alternative/img', '/myapp/install/img' ],
'/css' = [ '/alternative/css', '/myapp/install/css' ],
...
)
... (transhandler file, simplified of course) ...
sub handler {
my $r = shift;
my $uri = $r-uri;
... detecting dynamic handlers ...
while( my($url, $dirs) = each %STATIC_FILES ) {
if( $uri =~ m{$url/(.*)$} ) {
foreach my $d (@$dirs) {
my $file = $d/$1;
if( -f $file ) {
$r-filename($file);
return OK;
}
}
}
}
}
Re: each considered harmful?
sub handler {
my $r = shift;
my $uri = $r-uri;
... detecting dynamic handlers ...
while( my($url, $dirs) = each %STATIC_FILES ) {
if( $uri =~ m{$url/(.*)$} ) {
foreach my $d (@$dirs) {
my $file = $d/$1;
if( -f $file ) {
$r-filename($file);
return OK;
}
}
}
}
}
ah, a real-world example. Just what we need. Care to write a short pod
section using this example, explaining the problem and the solution?
plain text or pod will do. we will add it to the coding chapter.
Hmm, let's use my english-like language...
The problem:
- the application static files are installed in /myapp/img,
/myapp/css, ...
- local site customization can be made by installing customized files
in /custom/img, /custom/css...
- in both cases they are accessed via /img/..., /css/... urls
The solution - we use custom transhandler to check whether the
customized version exists and use it if so.
--
( Marcin Kasperski | Working overtime sucks the spirit and motivation )
( http://www.mk.w.pl | out of a team. (Wells) )
()
( O CVS i zarzdzaniu wersjami: http://www.mk.w.pl/narzedzia/narzedzia_cvs )
Re: each considered harmful?
Does there exist some way to protect before this problem (some kind of auto-destructor, finally, whatever which would automatically rewind the hash internal iterator while leaving the context)? Not really a mod_perl problem, but you can read about the solution in the docs for each. There is a single iterator for each hash, shared by all each, keys, and values function calls in the program; it can be reset by reading all the elements from the hash, or by evaluating keys HASH or values HASH. I found this note before asking, believe me... But it seems to me that this solution is not satisfactory - calling 'keys' or 'values' is inefficient and destroys most gains from iterating over the hash using each... -- ( Marcin Kasperski | Communication takes place between people, documents ) ( http://www.mk.w.pl |are secondary. (Booch) ) () ( Porady dla twrcw serwisw WWW: http://www.mk.w.pl/porady/porady_www )
Re: mod_perl slower than expected?
Is there anything I may be missing about the general configuration or environment of mod_perl which may be causing this strange situation? Try profiling... Some time ago I found my modperl app to be fairly slow because I turned on use Carp (and was carping errors and warnings). After disabling carping it was almost 10 times faster. I have also found that SWIG generated interface to some C++ library is very inefficient in some places (especially destructing a lot of objects) and manually written XS module happened to be a few times faster (it was the case with a lot of small objects). I think there are also other similar potential problems... -- ( Marcin Kasperski | Most of the bad things that can happen to a project ) ( http://www.mk.w.pl | are the result of miscommunication. (Booch) ) () ( Porady dla programisty Oracle: http://www.mk.w.pl/porady/porady_oracle )
each considered harmful?
Hmm, probably well known but ... I have not met any direct warning of
this problem so far.
our %SOME_CONFIG = (
a = 1,
b = 2,
c = 3,
);
...
while (my($k,$v) = each %SOME_CONFIG) {
if( ... ) {
return; # or last, or throw exception
}
}
You probably see the problem - when this code is re-executed (next
request), the loop iterates just over 'the rest' of the hash.
Does there exist some way to protect before this problem (some kind of
auto-destructor, finally, whatever which would automatically rewind
the hash internal iterator while leaving the context)?
Re: stopping concurrent logins
Todd White [EMAIL PROTECTED] writes: i'm sure this is not a novel need, but i have failed to find or come up with just yet any (non-cookie) solution yet. i'm trying dearly to avoid cookies, but if that's the best or only way to do this, feel free to speak up. i'd love to hear from someone who has already tackled the problem of stopping concurrent web logins to a protected web space. realizing that ultimately people can share their username/password to a for-fee protected web site, we would at *least* like to avoid the possibility that two people could both be logged in at the same time from two different computers. the use of IP address doesn't seem adequate since many users come through a router/proxy running NAT. Hmm, as people are logged in, there must be some kind of session. So the only thing you need is to make sure that there can be at most one session for given user id - in fact, to invalidate any 'older' session(s) while logging in in the 'new' one. -- ( Marcin Kasperski | In any large change, 1/3 will think it is great, 1/3 ) ( http://www.mk.w.pl | will think it is stupid, and 1/3 will wait (Reed) ) () ( Porady dla programisty Oracle: http://www.mk.w.pl/porady/porady_oracle )
Re: Segmentation fault - is Error.pm the culprit?
Richard Clarke [EMAIL PROTECTED] writes: I'm using a subclass of Error.pm to throw exceptions when a DBI error happens. The SEGV only happens after I return SERVER_ERROR in response to the thrown and propogated error. I do not know whether it can be related to your problem but for me some cryptical problems disappeared when I replaced default Error.pm with the newer version from CPAN.
Re: [ANNOUNCE] mod_perl Developer's Cookbook available in Polish
Geoffrey Young [EMAIL PROTECTED] writes: for our native Polish speakers (as well as the archives), the Polish translation of the mod_perl Developer's Cookbook appears to be available and shipping. http://helion.pl/ksiazki/modpkp.htm Thanks! That's the great news for me (especially considering that 69z =~ 18$ and I can buy it nearby). -- ( Marcin Kasperski | Osoba jest omegalizacj ewolucji uniwersalnej na ) ( http://www.mk.w.pl | okrelonym odcinku etapowym (Teilhard de Chardin) ) () ( Wygeneruj dokumentacj: http://www.mk.w.pl/narzedzia/narzedzia_gendoc )
Configuration in module vs in apacheconfig...
{ yes, I saw the chapter in Stas Bekman's book, that's not exactly
what I discuss }
So, there is some mod_perl application. It has a lot of configurable
elements - from urls being used to constants predefining contents of
different drop-down boxes.
Currently the config information is spread between:
- apache config file included to httpd.conf (let's call it myapp.conf),
where different urls and directories are both mapped to handlers
and PerlSetVar'ed so the application can use them when needed,
log formats are defined etc
- separated application module (let's call it MyConfig.pm), where
the things like lookup contents, default field values, customizable
validation criteria or database passwords are kept
I would like to integrate those two.
After some initial consideration I found, that moving config from
MyConfig.pm to myapp.conf does not seem to be reasonable. Imagine
PerlSetVar'ing arrays and hashes which can refer one to another...
Currently I think about the opposite: putting everything configurable
and writing apache config in the style like:
Perl
use MyConfig.pm;
$PerlRequire = $MyConfig::InstallDir . '/startup.pl';
$PerlConfig .= ENDCONF;
Location $MyConfig::SomeUrl
$MyConfig::accessSettings
SetHandler perl-script
PerlHandler myapp::Handler::Some
Location
Location $MyConfig::OtherUrl
$MyConfig::accessSettings
SetHandler perl-script
PerlHandler myapp::Handler::Other
Location
ENDCONF
if($MyConfig::Debug) {
$LogLevel = 'debug';
$PerlConfig .= PerlWarn On\n;
} else {
$LogLevel = 'warn';
}
/Perl
(and more, and more, the above is just a sample to show what I am
thinking about, in fact most of the apache config would be done this
way)
The questions are:
1) As I checked, it seems that such a config would more-or-less
work. But I am a bit afraid of the restart semantics. How can
apache behave when graceful-ed or restart-ed having such a
configuration? Would it be same or different when mod_perl is
DSO'ed and when is statically linked?
2) I found that some Set*Handler directives are processed differently
when specified in myapp.conf and when specified in perl with
Apache-push_handlers (in particular the RestartHandler specified
with push_handlers was not executed while restart although it was
called on startup). Should I be afraid of anything similar after
following such a config change?
3) Do you think that this is the good/bad idea from any other reasons?
Re: [mp1] Partial success working on Tru64/DSO
(... revisiting old thread about my problems with compiling
apache/mod_perl as DSO on Tru64 unix)
My problem is still not solved but I get it up to the point where it
probably lies in some customary modules (which does not behave
correctly when unloaded/reloaded) and is not directly related to
apache/mod_perl.
As I got the minimal success (some simple mod_perl application
worked), I report here the way I compiled perl/apache/mod_perl in
that case. Maybe it will help someone in similar situation.
Below I cite the main parts of the script I used (with unnecessary
details omitted):
WORK_DIR=$HOME/tools_src
DIST_DIR=$HOME/DOWNLOAD
INST_DIR=$HOME/tools
PERL_INST_DIR=$INST_DIR/perl
APACHE_INST_DIR=$INST_DIR/apache
PERL_VERSION=perl-5.6.1
APACHE_VERSION=apache_1.3.27
MOD_PERL_VERSION=mod_perl-1.27
LIBAPREQ_VERSION=libapreq-1.0
DIGEST_MD5_VERSION=Digest-MD5-2.20
MIME_BASE64_VERSION=MIME-Base64-2.12
# ... cut a lot of other modules which are out-of-topic here
#
unpack() {
if [ -d $1 ]; then
rm -rf $1
fi
echo -n Unpacking $1...
gunzip -c $DIST_DIR/$1.tar.gz | tar xf -
echo done
}
#
if [ ! -d $WORK_DIR ]; then
mkdir -p $WORK_DIR
fi
if [ ! -d $INST_DIR ]; then
mkdir -p $INST_DIR
fi
#
cd $WORK_DIR
unpack $PERL_VERSION
cd $PERL_VERSION
# Some commments:
# -Uinstallusrbinperl in my case I use customary install directory
# -Uuseshrplib there is a makefile problem and compilation error
# when we define it (which we try to patch below, so please
# experiment if you like)
# -Uusemymalloc on 64bit platforms it is recommended according to some docs
# -Ubincompat5005 we don't want coredumps caused by name conflicts
# of malloc/free
# -Dusemultiplicity it helps a bit with DSO reloading
sh Configure -des -Dprefix=$PERL_INST_DIR -Uinstallusrbinperl \
-Uuseshrplib -Uusemymalloc -Ubincompat5005 \
-Dusemultiplicity
make
make test
make install
# don't forget it when installing outside defaults
PATH=$PERL_INST_DIR/bin:$PATH; export PATH
#
build_module() {
MOD=$1
cd $WORK_DIR
unpack $MOD
cd $WORK_DIR/$MOD
perl Makefile.PL $2 $3 $4 $5 $6 $7 $8 $9
make
make test
make install
}
build_module $DIGEST_MD5_VERSION
build_module $MIME_BASE64_VERSION
# ... cut others ...
#
unpack $APACHE_VERSION
unpack $MOD_PERL_VERSION
cd $WORK_DIR/$MOD_PERL_VERSION
perl Makefile.PL $PERL_MOD_DBG $PLTRACE \
APACHE_SRC=../$APACHE_VERSION/src \
APACHE_PREFIX=$APACHE_INST_DIR \
USE_APACI=1 \
USE_DSO=1 \
DO_HTTPD=1 \
ALL_HOOKS=1 \
EVERYTHING=1 \
APACI_ARGS=--enable-module=so,\
--prefix=$APACHE_INST_DIR,\
--enable-module=access,--enable-shared=access,\
--enable-module=actions,--enable-shared=actions,\
--enable-module=alias,--enable-shared=alias,\
--enable-module=asis,--enable-shared=asis,\
--enable-module=auth,--enable-shared=auth,\
--enable-module=auth_anon,--enable-shared=auth_anon,\
--enable-module=auth_db,--enable-shared=auth_db,\
--enable-module=auth_dbm,--enable-shared=auth_dbm,\
--enable-module=autoindex,--enable-shared=autoindex,\
--enable-module=cern_meta,--enable-shared=cern_meta,\
--enable-module=cgi,--enable-shared=cgi,\
--enable-module=digest,--enable-shared=digest,\
--enable-module=dir,--enable-shared=dir,\
--enable-module=env,--enable-shared=env,\
--enable-module=example,--enable-shared=example,\
--enable-module=expires,--enable-shared=expires,\
--enable-module=headers,--enable-shared=headers,\
--enable-module=imap,--enable-shared=imap,\
--enable-module=include,--enable-shared=include,\
--enable-module=info,--enable-shared=info,\
--enable-module=log_agent,--enable-shared=log_agent,\
--enable-module=log_config,--enable-shared=log_config,\
--enable-module=log_referer,--enable-shared=log_referer,\
--enable-module=mime,--enable-shared=mime,\
--enable-module=mime_magic,--enable-shared=mime_magic,\
--enable-module=mmap_static,--enable-shared=mmap_static,\
--enable-module=negotiation,--enable-shared=negotiation,\
--enable-module=proxy,--enable-shared=proxy,\
--enable-module=rewrite,--enable-shared=rewrite,\
--enable-module=setenvif,--enable-shared=setenvif,\
--enable-module=speling,--enable-shared=speling,\
--enable-module=status,--enable-shared=status,\
--enable-module=unique_id,--enable-shared=unique_id,\
--enable-module=userdir,--enable-shared=userdir,\
Re: [mp1] Still can not get working DSO configuration on Tru64
Stas Bekman [EMAIL PROTECTED] writes: Marcin Kasperski wrote: [...] It seems to me, they had problem with the '-Wl,-rpath' issue which I found to be minor and easily patched (by the way: maybe someone would want to correct it in distribution?). I wasn't following that thread, but if you send a patch that solves that problem, I'd be more than happy to commit it to the main distro. Hmm, so far I can only offer 'half-patch' and the problem explanation (I do not understand configure scripts well enough to guess how to correct them). The problem is that the -Wl,-rpath,/tools/perl/lib/5.6.1/alpha-dec_osf/CORE option (which, by the way, is returned by perl -V:ccdlflags when perl is compiled with -Duseshrplib) is not an option for 'ld' but for 'cc'. In fact it means 'dear cc, be so kind and pass to ld the option -rpath /tools/perl/lib/5.6.1/alpha-dec_osf/CORE). ld does not know '-Wl' and complains. As perl -V:ld returns 'ld', maybe this is the problem in perl, not in modperl, but I am not sure. I patched it just by replacing 'ld' with 'cc' in the Makefile generated by configure script - such correction caused link step to succeed (and 'unresolved PL_perl_destruct_level to be reported while apache startup' :-( ) Currently I plan to check the opposite correction (using -rpath ... and ld as link command), maybe this will change something. -- ( Marcin Kasperski | Working overtime sucks the spirit and motivation ) ( http://www.mk.w.pl | out of a team. (Wells) ) () ( Porady dla twrcw serwisw WWW: http://www.mk.w.pl/porady/porady_www )
Re: apache2 + mod_perl2 + dbd::oracle
Fabian Kreitner [EMAIL PROTECTED] writes: Hello all, I have problems connecting to an oracle db using mod_perl but cant figure out where the problem is. The script works fine from the bash but not through perl::registry. The script only sets the environment variables and then tries a connect. (...) !! ERROR: 12154 'Error while trying to retrieve text for error ORA-12154 (DBD ERROR: OCIServerAttach)' Either you removed message files from your Oracle installation (not very likely) or you failed to set ORACLE_HOME so that OCI libraries below DBD::Oracle notice this variable. Double check the way you set your environment.
Re: [mp1] Still can not get working DSO configuration on Tru64
The problem is that the -Wl,-rpath,/tools/perl/lib/5.6.1/alpha-dec_osf/CORE option (which, by the way, is returned by perl -V:ccdlflags when perl is compiled with -Duseshrplib) is not an option for 'ld' but for 'cc'. In fact it means 'dear cc, be so kind and pass to ld the option -rpath /tools/perl/lib/5.6.1/alpha-dec_osf/CORE). ld does not know '-Wl' and complains. As perl -V:ld returns 'ld', maybe this is the problem in perl, not in modperl, but I am not sure. I patched it just by replacing 'ld' with 'cc' in the Makefile generated by configure script - such correction caused link step to succeed (and 'unresolved PL_perl_destruct_level to be reported while apache startup' :-( ) Currently I plan to check the opposite correction (using -rpath .. and ld as link command), maybe this will change something. I tried it - I left ld as link command but replaced in Makefile -Wl,-rpath,/tools/... with -rpath /tools/... The results are exactly the same: link succeeded, PL_perl_destruct_level is unresolved while running apache. -- ( Marcin Kasperski | Communication takes place between people, documents ) ( http://www.mk.w.pl |are secondary. (Booch) ) () ( Sztuczki i kruczki w C++: http://www.mk.w.pl/porady/porady_cplusplus )
Re: [mp1] Still can not get working DSO configuration on Tru64
The results are exactly the same: link succeeded, PL_perl_destruct_level is unresolved while running apache. I found the workaround to avoid this effect: slightly patching apache build procedure so that the httpd binary is linked with perl libperl.so helped. The error disappears... now I get the coredump during the application startup. In case I manage to diagnose this core somehow, I will mention it here. -- ( Marcin Kasperski | A complex system designed from scratch never works) ( http://www.mk.w.pl |and cannot be patched to make it work. (Booch) ) () ( Porady dla programisty Oracle: http://www.mk.w.pl/porady/porady_oracle )
Re: [mp1] Still can not get working DSO configuration on Tru64
Marcin Kasperski [EMAIL PROTECTED] writes: The results are exactly the same: link succeeded, PL_perl_destruct_level is unresolved while running apache. I found the workaround to avoid this effect: slightly patching apache build procedure so that the httpd binary is linked with perl libperl.so helped. The error disappears... now I get the coredump during the application startup. In case I manage to diagnose this core somehow, I will mention it here. After recompiling perl, apache and modperl with debugging enabled I got the following backtrace. Does it reminds anything to anyone? I suspect something wrong in the fact, that perl_startup is called so late (during processing some PerlModule...) (Carp module mentioned there just happened to be the first PerlModule mentioned in my apache config) /apache-1.3.27, modperl-1.27/ Core file produced from executable 'httpd' Thread terminated at PC 0x3ffbff67de8 by signal SEGV (ladebug) where 0 0x3ffbff67de8 in S_new_he() hv.c:26 #1 0x3ffbff6b194 in Perl_share_hek(str=0x3feed70=@, len=1, hash=66) hv.c:1484 #2 0x3ffbff688c8 in Perl_hv_store(hv=0x140048cb0, key=0x3feed70=@, klen=1, val=0x140048d80, hash=66) hv.c:413 #3 0x3ffbff681f0 in Perl_hv_fetch(hv=0x140048cb0, key=0x3feed70=@, klen=1, lval=1) hv.c:210 #4 0x3ffbff03b7c in Perl_gv_fetchpv(nambeg=0x3feed70=@, add=1, sv_type=4) gv.c:669 #5 0x3ffbfefe56c in S_init_main_stash() perl.c:2523 #6 0x3ffbfef9c18 in S_parse_body(env=0x0, xsinit=0x3000180ecd8) perl.c:951 #7 0x3ffbfef9b0c in perl_parse(my_perl=0x140034040, xsinit=0x3000180ecd8, argc=3, argv=0x11fff9c48, env=0x0) perl.c:895 #8 0x3000180f58c in perl_startup(s=0x14001e860, p=0x14001e818) mod_perl.c:702 #9 0x30001817070 in perl_cmd_module(parms=0x11fffbe10, dummy=0x14001fdd8, arg=0x140020260=Carp) perl_config.c:582 #10 0x12001cee8 in invoke_cmd(cmd=0x30041842828, parms=0x11fffbe10, mconfig=0x14001fdd8, args=0x11fff9daf=) http_config.c:918 #11 0x12001d3ec in ap_handle_command(parms=0x11fffbe10, config=0x14001f250, l=0x11fff9da0=PerlModule Carp) http_config.c:1030 #12 0x12001d4f8 in ap_srm_command_loop(parms=0x11fffbe10, config=0x14001f250) http_config.c:1044 #13 0x12001db48 in ap_process_resource_config(s=0x14001e860, fname=0x1400112f0=/home/marcink/GAUSS/igoweb/src/tests/imf_efficiency/apache.conf, p=0x14001e818, ptemp=0x140338818) http_config.c:1332 #14 0x12001ea30 in ap_read_config(p=0x14001e818, ptemp=0x140338818, confname=0x1400112f0=/home/marcink/GAUSS/igoweb/src/tests/imf_efficiency/apache.conf) http_config.c:1616 #15 0x120010208 in standalone_main(argc=4, argv=0x11fffc018) http_main.c:5071 #16 0x120010ce8 in main(argc=4, argv=0x11fffc018) http_main.c:5456 #17 0x12000aad8 in __start(...) in /home/marcink/tools/apache/bin/httpd When I removed PerlModule directives and sticked only with PerlRequire, it occured that I must add 'use Apache;' on the beginning to use things like Apache-push_handlers (on normal installations it was not necessary, here I got ). And then I got similar coredump: Core file produced from executable 'httpd' Thread terminated at PC 0x3ffbff67de8 by signal SEGV (ladebug) where 0 0x3ffbff67de8 in S_new_he() hv.c:26 #1 0x3ffbff6b194 in Perl_share_hek(str=0x3feed70=@, len=1, hash=66) hv.c:1484 #2 0x3ffbff688c8 in Perl_hv_store(hv=0x140048cb0, key=0x3feed70=@, klen=1, val=0x140048d80, hash=66) hv.c:413 #3 0x3ffbff681f0 in Perl_hv_fetch(hv=0x140048cb0, key=0x3feed70=@, klen=1, lval=1) hv.c:210 #4 0x3ffbff03b7c in Perl_gv_fetchpv(nambeg=0x3feed70=@, add=1, sv_type=4) gv.c:669 #5 0x3ffbfefe56c in S_init_main_stash() perl.c:2523 #6 0x3ffbfef9c18 in S_parse_body(env=0x0, xsinit=0x3000180ecd8) perl.c:951 #7 0x3ffbfef9b0c in perl_parse(my_perl=0x140034040, xsinit=0x3000180ecd8, argc=3, argv=0x11fff9c48, env=0x0) perl.c:895 #8 0x3000180f58c in perl_startup(s=0x14001e860, p=0x14001e818) mod_perl.c:702 #9 0x30001817274 in perl_cmd_require(parms=0x11fffbe10, dummy=0x14001fdd8, arg=0x140020260=/home/marcink/src/tests/startup.pl) perl_config.c:613 #10 0x12001cee8 in invoke_cmd(cmd=0x30041842800, parms=0x11fffbe10, mconfig=0x14001fdd8, args=0x11fff9deb=) http_config.c:918 #11 0x12001d3ec in ap_handle_command(parms=0x11fffbe10, config=0x14001f250, l=0x11fff9da0=PerlRequire /home/marcink/src/tests/startup.pl) http_config.c:1030 #12 0x12001d4f8 in ap_srm_command_loop(parms=0x11fffbe10, config=0x14001f250) http_config.c:1044 #13 0x12001db48 in ap_process_resource_config(s=0x14001e860, fname=0x1400112f0=/home/marcink/src/tests/apache.conf, p=0x14001e818, ptemp=0x140235818) http_config.c:1332 #14 0x12001ea30 in ap_read_config(p=0x14001e818, ptemp=0x140235818, confname=0x1400112f0=/home/marcink/src/tests/efficiency/apache.conf) http_config.c:1616 #15 0x120010208 in standalone_main(argc=4, argv=0x11fffc018) http_main.c:5071 #16 0x120010ce8 in main(argc=4, argv=0x11fffc018) http_main.c:5456 #17 0x12000aad8 in __start(...) in /home/marcink/tools/apache/bin/httpd
Re: References for modperl usage in financial institutions?
Yes, please post your success stories here (...) I am more than happy being now able to add the new nice reference. Please, patch my English where necessary... Polish internet bank named Inteligo (http://www.inteligo.pl) recently migrated its transactional web service (the application used by the bank clients to make different kinds of payment orders, check account balances etc) from complicated Java-based solution to the modperl application. The application implements web frontend to the business services implemented by the main bank system and accessed via the bank middleware. It is worth mentioning that the application constitute the main access channel for the bank clients. After a few days of productional use the application is perceived to be much faster and lighter than the one previously used. Two words of warning: - inteligo 'informational' website (the pages visible under www.inteligo.pl) still use PHP and probably will continue to, - don't treat this as easy 'perl is faster than Java' claim, there was a lot of design and programming work behind the new application... Being a person who suggested using this technology and worked in a core development team I can admit that modperl fulfilled myp erformance expectations and allowed to develop complicated application fairly quickly. Thanks to all the people who developed this nice piece of software and its documentation and to everyone who answered my and my colleagues questions during the project. PS If only someone had some idea how to solve the DSO/Tru64 problem...
DSO/Tru64 (was Re: References for modperl usage in financial institutions?)
PS If only someone had some idea how to solve the DSO/Tru64 problem... We really need to find people on these platforms (True64, AIX, HP-UX, etc.) who can help to reproduce and resolve this kind of probs. If you know of those willing to help please ask them to subscribe on this list. I am using Tru64 - and if you have any suggestions of what to try, I will do it...
Re: [mp1] Still can not get working DSO configuration on Tru64
Stas Bekman [EMAIL PROTECTED] writes: Marcin Kasperski wrote: In short: I tried different compilation methods with two possible outcomes: a) apache and modperl compile succesfully but I get coredump while the application is starting (in all cases SEGVs, in some cases core's confused the debugger, in other I managed to notice __at_fork in backtrace) b) everything seem to compile succesfully but I get dlopen: /tools/apache/libexec/mod_perl.so: symbol PL_perl_destruct_level unresolved while the application is starting Maybe it is worth noting, that I happened to get b) also when I compiled perl with shared perl library (in contrary to suggestion found in modperl docs that building perl with shared library should help). 2 years ago, there was a similar struggle as you can see from this thread: http://marc.theaimsgroup.com/?t=9744223061r=1w=2 May be those participants are still using True64 and can give you some hints? It seems to me, they had problem with the '-Wl,-rpath' issue which I found to be minor and easily patched (by the way: maybe someone would want to correct it in distribution?). And all they wanted was to get statically linked modperl. This is the thing I already have. Maybe I will post a question to tru64-managers... -- ( Marcin Kasperski | (...) the only completion criterion for the Analysis ) ( http://www.mk.w.pl | and Design phases is - the date. (Martin) ) () ( Nie gub zgosze bdw: http://www.mk.w.pl/narzedzia/narzedzia_bugewid)
Re: [mp1] Still can not get working DSO configuration on Tru64
In short: I tried different compilation methods with two possible outcomes: a) apache and modperl compile succesfully but I get coredump while the application is starting (in all cases SEGVs, in some cases core's confused the debugger, in other I managed to notice __at_fork in backtrace) b) everything seem to compile succesfully but I get dlopen: /tools/apache/libexec/mod_perl.so: symbol PL_perl_destruct_level unresolved while the application is starting Maybe it is worth noting, that I happened to get b) also when I compiled perl with shared perl library (in contrary to suggestion found in modperl docs that building perl with shared library should help).
Re: [mp1] callback called exit (many times)
Am I right deducing, that I should attempt recompiling perl with -DEMERGENCY_SBRK? Try to, but this doesn't solve the original problem. You still have to try to narrow down the code that causes the problem. Using ab or a similar load tester might help to stress test. Using tools to prevent situations when the server runs out of memory is very important. See the online docs for more info. Using these tools should probably solve the problem altogether, without using any special compilation options. The memory problem itself is already diagnosed (some non-apache process allocated more than it was expected, therefore less memory remained for apache/modperl processes) and I will work on it. Nevertheless I'd like to improve apache behaviour in lack of memory conditions - if possible. Thanks for all the hints. -- ( Marcin Kasperski | In any large change, 1/3 will think it is great, 1/3 ) ( http://www.mk.w.pl | will think it is stupid, and 1/3 will wait (Reed) ) () ( Porady dla twrcw serwisw WWW: http://www.mk.w.pl/porady/porady_www )
[mp1] callback called exit (many times)
Recently I happened to get the error log flooded with the message callback called exit (repeated a couple of million times, probably by one process or a few processes). This was some side-effect of the lack of memory (earlier in the logs I found messages about allocation failures), which in turn was caused by some non-apache process allocating too much. Nevertheless, I am trying to find what happened to my apache/modperl process (processess?). After all, aborting some processes due to lack of memory is ok, but filling the whole disk with error log is not. It is possible that the problem is not caused by the modperl itself (for instance some of additional modules we implemented in both C and perl could go wild). But trying to understand the problem I would like to ask the following questions: a) Have anyone anywhere observed similar behaviour? b) When is the message 'callback called exit' printed - and can it be disabled? c) Do you have any idea of what should I pay attention to? -- ( Marcin Kasperski | Krlik to brzmi dumnie! Wszystko dla szympansa! ) ( http://www.mk.w.pl | Chomikiem jestem, wszystko mi wolno! (Mroek) ) () ( Sztuczki i kruczki w C++: http://www.mk.w.pl/porady/porady_cplusplus )
Re: [mp1] callback called exit (many times)
Stas Bekman [EMAIL PROTECTED] writes: Marcin Kasperski wrote: Recently I happened to get the error log flooded with the message callback called exit (repeated a couple of million times, probably by one process or a few processes). This was some side-effect of the lack of memory (...) Hope that the following somewhat helps: http://perl.apache.org/search/swish.cgi?query=callback+called+exitsbm=SecDsubmit=search Thank you for the pointer. Am I right deducing, that I should attempt recompiling perl with -DEMERGENCY_SBRK? The perl I use describes itself as below so probably this option is not turned on: $ perl -V Summary of my perl5 (revision 5.0 version 6 subversion 1) configuration: Platform: osname=dec_osf, osvers=5.1, archname=alpha-dec_osf uname='osf1 banach.softax.local v5.1 1885 alpha ' config_args='-des -Dprefix=/opt/igowwwroot/tools_test/perl -Uinstallusrbinperl -Duseshrplib=1 -Dusemymalloc -Ubincompat5005' hint=previous, useposix=true, d_sigaction=define usethreads=undef use5005threads=undef useithreads=undef usemultiplicity=undef useperlio=undef d_sfio=undef uselargefiles=define usesocks=undef use64bitint=define use64bitall=define uselongdouble=undef Compiler: cc='cc', ccflags ='-std -fprm d -ieee -D_INTRINSICS -I/usr/local/include -DLANGUAGE_C', optimize='-O4', cppflags='-std -fprm d -ieee -D_INTRINSICS -I/usr/local/include -DLANGUAGE_C' ccversion='V6.4-009', gccversion='', gccosandvers='' intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678 d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16 ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8 alignbytes=8, usemymalloc=y, prototype=define Linker and Libraries: ld='ld', ldflags =' -L/usr/local/lib' libpth=/usr/local/lib /usr/shlib /usr/ccs/lib /usr/lib/cmplrs/cc /usr/lib /var/shlib libs=-ldbm -ldb -lm -liconv -lutil perllibs=-lm -liconv -lutil libc=/usr/shlib/libc.so, so=so, useshrplib=false, libperl=libperl.a Dynamic Linking: dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags=' ' cccdlflags=' ', lddlflags='-shared -expect_unresolved * -O4 -msym -std -s -L/usr/local/lib' Characteristics of this binary (from libperl): Compile-time options: USE_64_BIT_INT USE_64_BIT_ALL USE_LARGE_FILES Built under dec_osf Compiled at Nov 21 2002 15:05:28 @INC: /opt/igowwwroot/tools/perl/lib/5.6.1/alpha-dec_osf /opt/igowwwroot/tools/perl/lib/5.6.1 /opt/igowwwroot/tools/perl/lib/site_perl/5.6.1/alpha-dec_osf /opt/igowwwroot/tools/perl/lib/site_perl/5.6.1 /opt/igowwwroot/tools/perl/lib/site_perl .
Re: References for modperl usage in financial institutions?
Thanks for all the people who sent me the references (both here and to private email). Maybe it would be good idea to add some such references to the 'sites running modperl' page. My business client after taking a look at http://perl.apache.org/outstanding/sites.html said 'nice, but there are mostly technical sites, almost no business...' I would also rename 'Technologies Extraordinare' link to something like 'Who is using modperl' but that is different story... -- ( Marcin Kasperski | In any large change, 1/3 will think it is great, 1/3 ) ( http://www.mk.w.pl | will think it is stupid, and 1/3 will wait (Reed) ) () ( Porady dla twrcw serwisw WWW: http://www.mk.w.pl/porady/porady_www )
Re: How do I force a 'Save Window?'
Bonus Points : Use Content-Disposition: attachment;filename=blabla and pre-set the name of the file on disk. I tried it hard (in my case I generate PDF files. No way. Internet Explorer ignored it. Finally I decided to perform redirect to URL ending with /blabla.pdf - then it worked more-or-less as expected. -- ( Marcin Kasperski | A reusable framework that is developed by itself will ) ( http://www.mk.w.pl |probably not be very reusable. (Martin)) () ( Porady dla twrcw serwisw WWW: http://www.mk.w.pl/porady/porady_www )
References for modperl usage in financial institutions?
I am looking for some examples of modperl being used in financial institutions (banks, brokers, ... but also large e-commerce). I spent a few hours searching the internet but it seems such a information is not easily available. I am interested in both official and unofficial information. Technical details are not necessary. Thanks in advance for any reply. -- ( Marcin Kasperski | Working overtime sucks the spirit and motivation ) ( http://www.mk.w.pl | out of a team. (Wells) ) () ( Dokument biznesowy w LaTeXu: http://www.mk.w.pl/porady/latex/mkofficial_cls)
Re: RFC: Template::YetAnother
Thomas Klausner [EMAIL PROTECTED] writes: Yes, this is a RFC for Yet Another Templating System. I know that there are a lot of those around. But this own might be different/interesting (at least I think/hope so...) (...) One note: while talking about templating systems for generating HTML, it can be inspiring to take a look at the Zope Page Templates (yes, I know, this is the python/zope world, not perl...). They found the nice idea of injecting loops/ifs etc into HTML in the way which does not spoil HTML markup. Their introduction is on http://www.zope.org/Documentation/Articles/ZPT1 and reference is in http://www.zope.org/Documentation/Books/ZopeBook/current/AppendixC.stx Here are some examples (copied from the introduction) title tal:content=here/titlePage Title/title The URL is span tal:replace=request/URLURL/span. table border=1 width=100% tr th#/ththId/ththMeta-Type/ththTitle/th /tr tr tal:repeat=item container/objectValues td tal:content=repeat/item/number#/td td tal:content=item/idId/td td tal:content=item/meta_typeMeta-Type/td td tal:content=item/titleTitle/td /tr /table -- ( Marcin Kasperski | Osoba jest omegalizacj ewolucji uniwersalnej na ) ( http://www.mk.w.pl | okrelonym odcinku etapowym (Teilhard de Chardin) ) () ( Wygeneruj dokumentacj: http://www.mk.w.pl/narzedzia/narzedzia_gendoc )
Re: How can I tell if a request was proxy-passed form an SSL server?
John Siracusa [EMAIL PROTECTED] writes: Assume I have a front-end (non-mod_perl) proxy server that supports both HTTP and HTTPS. I want to know, from within my Perl code on the back-end (HTTP-only) mod_perl server, if the current request was ProxyPass-ed to me based on an original HTTP or HTTPS request from the user. There doesn't appear to be a way to add headers to the proxy request on a per-virtual-host basis using mod_proxy, and I don't see any other headers that I can use to distinguish between an HTTP or HTTPS front-end server. Any ideas? What about the simple manual solution: frontend server proxies /some/url to /http/some/url on backend for HTTP and to /https/some/url on backend for HTTPS. Or something similar... -- ( Marcin Kasperski | Communication takes place between people, documents ) ( http://www.mk.w.pl |are secondary. (Booch) ) () ( Moje prywatne strony: http://www.kasperski.prv.pl )
Re: [mp-1 ?] Apache::Util::escape_html could handle single quotation
namely i had a bad feeling about this. we should not be implementing escape_html to begin with, the functionality should all be in apache. (...) The main point in this case: someone decided to escape (double quote) in escape_html. For HTML, ' (single quote) is practically equivalent (and can be used at the same places for the same purpose). I see no reason to differentiate handling of single and double quote while generating HTML pages. The reason to use escape_html vs HTML::Entities is clear - the speed. And at least in my case one needs to escape just , , , and ' - the characters which can spoil the way browser interprets HTML. -- ( Marcin Kasperski | Software is not released, it is allowed to escape.) ( http://www.mk.w.pl | ) () ( Moje prywatne strony: http://www.kasperski.prv.pl )
Re: [mp-1 ?] Apache::Util::escape_html could handle single quotation
some of us on modperl-dev had a discussion with Doug recently about expanding Apache::Util::escape_html() to do things like HTML::Entities (such as high-bit characters) and it was decided it was a bad idea. see: http://marc.theaimsgroup.com/?l=apache-modperl-cvsm=101708056429561w=2 and doug's reply: http://marc.theaimsgroup.com/?l=apache-modperl-devm=101708105030300w=2 One additional remark: while suggesting escaping of single quote, I'd vote against escaping national characters in escape HTML. For instance, I generate iso-8859-2 encoded Polish pages and I want my national characters to be left as-is while the text is escaped... To end the whole argumentation: I suggest escaping ' as this character is unsafe in HTML. Simultaneously, I suggest keeping advanced entities out of this utility function.
Re: [mp-1 ?] Apache::Util::escape_html could handle single quotation
Your patch seems to me to be partially wrong (you missed similar addition a few lines above, while calculating the destination size). Erm, yeah, so I see, now that you mention it. And this patch version seem to be correct (oh, maybe someone could consider also adding some test case to t/net/perl/util.pl but it does not seem to be very important) Nevertheless, I write here about the problem because I would really like having such a change in the mainstream modperl distribution. Keeping my own patched modperl distribution, integrating changes etc is a bit troublesome (organizationally). I'm think that, with mod_perl 2.0, mod_perl 1.x might not be high on maintainer's list of stuff to do, but Jim Winstead would probably accept a (proper!) patch and release libapreq-1.01. Hmm, should I do something to send it to him or is he reading this list? Regards (and thanks) Marcin -- ( Marcin Kasperski | Communication takes place between people, documents ) ( http://www.mk.w.pl |are secondary. (Booch) ) () ( Dokument biznesowy w LaTeXu: http://www.mk.w.pl/porady/latex/mkofficial_cls)
[mp-1 ?] Apache::Util::escape_html could handle single quotation
I use Apache::Util::escape_html to perform fast HTML-escaping of the
data before displaying it. Unfortunately, this function handles
, , and but does not handle ' (single quote) - which
can be escaped as apos;
It would be nice if apos was handled. After all, in HTML file it is
almost equivalent to double quote (it is only the matter of style
whether one uses a href=some url or a href='some url').
Moreover, there happen situation where on can not change quotation
model like in (adapted real example I happened to have trouble with):
a href=javascript:somefun('[%some_var%]')
(where some_var is a variable escaped with escape_html and unfortunately
it can happen to contain apostrophe mark)
It seems to me that it would suffice to slightly change the
function my_escape_html in src/modules/perl/Util.xs by
adding new else-if in both if sequences.
PS I do not know how the thing looks in modperl-2 but in case there is
similar problem, I would suggest similar solution.
--
( Marcin Kasperski | A reusable framework that is developed by itself will )
( http://www.mk.w.pl |probably not be very reusable. (Martin))
()
( Z kart bezpieczniej? http://www.mk.w.pl/artykuly/karty_niebezpieczenstwa )
Re: [mp-1 ?] Apache::Util::escape_html could handle single quotation
darren chamberlain [EMAIL PROTECTED] writes: * Marcin Kasperski [EMAIL PROTECTED] [2002-11-08 16:22]: I use Apache::Util::escape_html to perform fast HTML-escaping of the data before displaying it. Unfortunately, this function handles , , and but does not handle ' (single quote) - which can be escaped as apos; Hey, this is an easy one. Apply the attached patch to mod_perl-1.XX/src/modules/perl/Util.xs, and single quotes will be turned into apos; Your patch seems to me to be partially wrong (you missed similar addition a few lines above, while calculating the destination size). Nevertheless, I write here about the problem because I would really like having such a change in the mainstream modperl distribution. Keeping my own patched modperl distribution, integrating changes etc is a bit troublesome (organizationally).
Re: memory usage problem!
Plamen Stojanov [EMAIL PROTECTED] writes: I load 2Mb data from a database in perl hash and perl takes 15Mb memory. As I use this under mod_perl - perl never returns this memory to the OS. I must set a little number for MaxRequestsPerChild in order to restart perl interpreter not to eat a lot of memory. Is there any solution to avoid such memory usage? Simple: dont load this into memory. You can achieve this in multiple ways: - (if you really need to get this as hash) use DBM hash, mapping those resources to disk instead of memory - (if you can reconsider) use your database to sort the data according to your needs and just process it incrementally (for instance just scan record-after-record, do not keep the data in memory but just print them to the output socket) -- ( Marcin Kasperski | A reusable framework that is developed by itself will ) ( http://www.mk.w.pl |probably not be very reusable. (Martin)) () ( Dokument biznesowy w LaTeXu: http://www.mk.w.pl/porady/latex/mkofficial_cls)
How would you organize custom logging?
I am to provide some custom logging in my mod_perl application. Omitting some unnecessary details, I need to store some information after processing each request, putting this info into some central log (be it file or database) common for all apache processes and (if possible) common for multiple machines. I would like to ask for any suggestions/hints/patterns/... but there are also some specific qustions: 1) Apache somehow gathers notes from multiple processes into single access.log and error.log. Is it possible to reuse this functionality somehow for writing my custom information somewhere else? Considering such functionality is already present I would not like to reimplement logic protecting from simultaneous write from multiple processess... 2) Looking at the name, seems it is LogHandler where custom logging should take place. But does it really give real advantage over just logging on the end of PerlHandler? If so, how should I pass data between handlers - just stick to some perl global variable or use pnotes? 3) Are there any tools of 'gather logs from multiple machines to the central location' you would recommend? -- ( Marcin Kasperski | Communication takes place between people, documents ) ( http://www.mk.w.pl |are secondary. (Booch) ) () ( Nie gub zgosze bdw: http://www.mk.w.pl/narzedzia/narzedzia_bugewid)
Not a subroutine name... (was Re: problem with $r-push_handlers())
mod_perl_push_handlers: Not a subroutine name or CODE reference!
I have observed similar problem myself. I got the same error when I
wrote in my startup.pl
Apache-push_handlers(PerlChildInitHandler,
\MyApp::Main::on_child_init);
when I replaced it with
Apache-push_handlers(PerlChildInitHandler,
sub { MyApp::Main::on_child_init(); });
it works as expected.
It is probably worth noting, that in my opinion the first syntax
should work and there is something wrong in mod_perl code...
Is it something 'well known' or 'by design'?
Upgrading frontend apache - is it worth it?
I use well known configuration 'thin apacheproxy proxying fat mod_perl apache'. Currently I use apache1 and for now I do not plan upgrading mod_perl server to apache2. But I think about upgrading the frontend apache to apache2 - mainly because I hope that multithreaded proxy can be a bit more efficient and less memory consuming than the multiprocess proxy. So the question: is it really worth doing? Has anyone performed similar upgrade or compared proxying efficiency of apache1 with apache2? Can the analysis be influenced by using proxy to decode SSL too? Thanks in advance for your replies. -- ( Marcin Kasperski | For waterfall, milestones are commitments. For itera- ) ( http://www.mk.w.pl | tive development, they are decision points. (Martin) ) () ( Porady dla twrcw serwisw WWW: http://www.mk.w.pl/porady/porady_www )
Re: /usr/sbin/apachectl: line 198: 12577 Segmentation fault $HTTPD -t
Andrew G. Hammond [EMAIL PROTECTED] writes: I'm trying to use some modules I wrote through PerlModule in httpd.conf The modules are tested and correct. They appear to work when they're auto-loaded. However, I wish to load them on startup of the web server. When I add the following line to my httpd.conf: PerlModule Bounce::Handler and attempt to apachectl configtest I get: /usr/sbin/apachectl: line 198: 12577 Segmentation fault $HTTPD -t Any suggestions about what I might have done wrong? Modules loaded in startup.pl are initialized once in the parent process instead of being initialized in each of the child processess... The question is: which modules do you use. -- ( Marcin Kasperski | You have the right to peace, fun, and productive ) ( http://www.mk.w.pl |and enjoyable work. (Beck) ) () ( Sztuczki i kruczki w C++: http://www.mk.w.pl/porady/porady_cplusplus )
