Anyone know of specific problems with the malloc in Sun's libc?
W/ current apache and mod_perl, perl-5.6.1, Solaris-2.8. Been having problems with DBI dropping connections. One suggestion was to override the default of no and use the malloc that comes with perl-5.6.1. Anyone know of problems or any obvious symptoms? thanx. -- Steven Lembark 2930 W. Palmer Workhorse Computing Chicago, IL 60647 +1 800 762 1582
Re: Fast template system
Ryan Thompson wrote: Mark Maunder wrote to Ryan Thompson: Ryan Thompson wrote: There must be a faster way. I have thought about pre-compiling each HTML file into a Perl module, but there would have to be an automated (and secure) way to suck these in if the original file changes. Either that, or maybe someone has written a better parser. My code looks something like this, to give you an idea of what I need: Sure there are tons of good template systems out there. I think someone made a comment about writing a template system being a right of passage as a perl developer. But it's also more fun to do it yourself. :-) I guess you've tried compiling your regex with the o modifier? Yep, problem is there are several of them. I've done some work recently to simplify things, which might have a positive effect. Also, have you tried caching your HTML in global package variables instead of shared memory? I think it may be a bit faster than shared memory segments like Apache::Cache uses. (The first request for each child will be slower, but after they've each served once, they'll all be fast). Does your engine stat (access) the html file on disk for each request? You mentioned you're caching, but perhaps you're checking for changes to the file. Try to stat as My caching algorithm uses 2 levels: When an HTML file is requested, the instance of my template class checks in its memory cache. If it finds it there, great... everything is done within that server process. If it's not in the memory cache, it checks in a central MySQL cache database on the local machine. These requests are on the order of a few ms, thanks to an optimized query and Apache::DBI. NOT a big deal. If it's not in either cache, it takes it's lumps and goes to disk. If you're using a disk based table, in most cases, mysql would access the disk itself anyway. So whether you're getting the cached data from mysql or a file, it's still coming from disk. (yes mysql caches - especially if you're using InnoDB tables, but you're not gauranteed to save a disk access). Not sure how much html/content you have, but any chance you can stick it all in shared memory, or even better, give each child their own copy in a package global variable (like a hashref)? If it's under a meg (maybe even 2) you might be able to get away with that. In each cache, I use a TTL. (time() + $TTL), which is configurable, and usually set to something like 5 minutes in production, or 60 seconds during development/bug fixes. (And, for this kind of data, 5 minutes is pretty granular, as templates don't change very often.. but setting it any higher would, on average, have only a negligible improvement in performance at the risk of annoying developers :-). And, with debugging in my template module turned on, it has been observed that cache misses are VERY infrequent ( 0.1% of all requests). In fact, if I use this cache system and disable all parsing (i.e., just use it to include straight HTML into mod_perl apps), I can serve 150-200 requests/second on the same system. With my parsing regexps enabled, it drops to 50-60 requests/second. So, to me, it is clear where performance needs to be improved. :-) How about instead of having a cache expiry/TTL, you parse the HTML on the first request only and then always serve from the cache. To refresh the cache, you set a flag in shared memory. Then whenever a child is about to serve from cache, it just checks the flag in shared memory to see if it needs to refresh it's cache. That way you can 'push' out new code by just setting the flag and unsetting it once all running children have read from the cache. You'll also have every request served from the cache except the first. You'll also get the benifit of having each child serve live code for every request by keeping the flag set. That way your developers can code in realtime without a 60 second latency for new HTML to take effect when you want them to.
DirectoryAccelNoPass in mod_accel
Is there a way to specify an AccelNoPass directive (from mod_accel) that only affects a certain directory? For example, consider the following scenario: AccelPass /~user1/ http://127.0.0.1:8001/ AccelNoPass ~*\.gif$ ~*\.jpg$ AccelPass /~user2/ http://127.0.0.1:8002/ AccelNoPass ~*\.gif$ Someone might want to specify separate AccelNoPass settings for those two directories. It doesn't seem to work when I put it in Directory though; I get AccelNoPass not allowed here error. (I don't actually need this functionality at this point and I think it's an obscure case, but I felt it was worth pointing out.)
Re: DirectoryAccelNoPass in mod_accel
On Mon, 31 Dec 2001, Philip Mak wrote: Is there a way to specify an AccelNoPass directive (from mod_accel) that only affects a certain directory? For example, consider the following scenario: AccelPass /~user1/ http://127.0.0.1:8001/ AccelNoPass ~*\.gif$ ~*\.jpg$ AccelPass /~user2/ http://127.0.0.1:8002/ AccelNoPass ~*\.gif$ Someone might want to specify separate AccelNoPass settings for those two directories. It doesn't seem to work when I put it in Directory though; I get AccelNoPass not allowed here error. (I don't actually need this functionality at this point and I think it's an obscure case, but I felt it was worth pointing out.) No. Both AccelPass and AccelNoPass run in translation phase and sets or does not set 'accel-handler'. So if AccelNoPass could run in Location or Directory then it means that mod_accel needs to skip 'accel-handler' and found another one instead - mod_accel needs to run subrequest. I think it complicates processing and is not needed in many cases. Besides in your example case you can use such regexps: AccelNoPass ~*\.gif$ ~*^/~user1/.*\.jpg$ Igor Sysoev
Re: Fast template system
Yeah, TT is good, all right, especially with the ability to change the
start and end tags to something more compliant, like !-- and --.
Uh ... why aren't you syntax-validating the actual HTML output by your
application?
Also, there are multiple syntax-checkerts that allow you to specify
validation rules.
I'd gladly switch, if someone knows how to get it working on FreeBSD
-STABLE without breaking everything else :-)
Quit using FreeBSD ports. Compile from source. Your problem is almost
certainly due to FreeBSD's weird installation of Perl, specifically its
library paths.
The best solution I've found is to create a new user 'perl' and install
Perl and all libraries etc under that user's home directory.
My system:
nick@world ~uname -a
FreeBSD world.tonkinresolutions.com 4.4-PRERELEASE FreeBSD 4.4-PRERELEASE
#1: Sat Aug 11 18:17:44 PDT 2001
[...]
nick@world ~perl -v
This is perl, version 5.005_03 built for i386-freebsd
[...]
nick@world ~perl -MTemplate -e 'print $Template::VERSION,\n'
2.06
- nick
Nick Tonkin {|8^)
Re: Anyone know of specific problems with the malloc in Sun's libc?
On Sun, Dec 30, 2001 at 02:12:47PM -0600, Steven Lembark wrote: W/ current apache and mod_perl, perl-5.6.1, Solaris-2.8. Been having problems with DBI dropping connections. One suggestion was to override the default of no and use the malloc that comes with perl-5.6.1. Anyone know of problems or any obvious symptoms? I have not seen any problems with the Solaris malloc, in fact it seems that the VM behavior of our systems improved when we switched over. Consider setting the DBI_TRACE environment variable if you're having problems such as this. It will show you where your connections are going up and down. Does your DBD driver support pings? If not you should check out the Apache::DBI docs for info on how to roll your own.. If you feel like recompiling perl on solaris you might want to review the following options used to build perl with gcc 2.95.3. Seems to work well for us... CFLAGS='-pipe -mcpu=ultrasparc -mtune=ultrasparc -m32 -Wa,-xarch=v8plusa -O3 -ffast-math -fexpensive-optimizations' Configure -O -Dcc=gcc \ -Doptimize=@CFLAGS@ \ -Dlocincpth=@prefix@/include \ -Dloclibpth=@prefix@/lib \ [EMAIL PROTECTED] \ [EMAIL PROTECTED] \ -Ud_bincompat5005 \ -Ubincompat5005 \ -Uusemymalloc \ -Uinstallusrbinperl \ -Dlibperl=libperl.so \ -Duseshrplib=true \ -Dstatic_ext=Data/Dumper Fcntl IO POSIX Socket Sys/Hostname \ -Ui_db \ -Ui_ndbm \ -Ui_ndbm \ -des -- Paul Lindner[EMAIL PROTECTED] | | | | | | | | | | mod_perl Developer's Cookbook http://www.modperlcookbook.org Human Rights Declaration http://www.unhchr.ch/udhr/index.htm
ANNOUNCE: AxKit 1.5
AxKit 1.5 is now available for download from both AxKit.org, and soon CPAN mirrors worldwide. This is a major upgrade, so please use caution when upgrading live servers, as major internal parts of AxKit have changed with this release. Most significantly, a large amount of code has been migrated to C, and some parts of the code have been migrated to use libxml2, which you should install prior to installing AxKit for optimum performance. It is now possible if you have libxml2 installed to run AxKit entirely without XML::Parser, which will help for people who can't recompile Apache (to remove the included expat). There have been many performance improvements in this release, too many to mention. Perhaps the most significant though is that AxKit is now a first class Apache handler (although it still requires mod_perl), so you can do: SetHandler axkit, rather than have to use PerlHandler. This is a performance win, and will help if we ever need to move away from mod_perl to being a pure Apache module. For a full list of changes, please consult the axkit-users mailing list archives. Details of which are at http://axkit.org/mailinglist.xml Changes since AxKit 1.4_86 (beta): AxPoint added the ability to do bullet point transitions Fixed LibXSLT relative files with document() Fixed some core caching and delivery bugs Fixed MANIFEST missing files -- !-- Matt -- :-Get a smart net/:-
[ANNOUNCE] Cache::Mmap 0.04
file: $CPAN/authors/id/P/PM/PMH/Cache-Mmap-0.04.tar.gz size: 14113 bytes md5: 649bdb9816f4177c0eb81c077fd7614c Much to my dismay and embarrassment, the serious bug fix in version 0.03 introduced a serious bug of it's own. You'll be pleased to hear that I've added a test suite (currently only one file) to the distribution, so this sort of thing should happen less in the future. Again, I strongly advise all users to upgrade to this version. I've also removed all the older versions from CPAN, just in case. This version also does the file locking in XS, thus removing the assumptions about struct flock's layout. It also does its own mmap()ing in XS, removing the dependency on Malcom Beattie's Mmap.pm. This means that people don't need to figure out Malcom's slightly strange distribution name, and it compiles on 5.6 without any problems (I've tested it on 5.004_05 too). From the README: This module provides a shared cache, using a memory mapped file. Very useful for mod_perl applications. If routines are provided which interact with the underlying data, access to the cache is completely transparent, and the module handles all the details of refreshing cache contents, and updating underlying data, if necessary. -- Peter Haworth [EMAIL PROTECTED] I don't know what kinds of evil I've done in the past. I don't even want to know. I just want that evil to carry on having its good effects -- Damian Conway on his use of pack/unpack/vec
test, pls ignore
un/re subscribed to a different addy, THIS IS JUST A TEST!
Apache::Session getting DESTROYed in wrong order
Hey, I'm having problems with Apache::Session, the symptom is that none of my data is getting written to the database. It's not the nested-data problem, since I'm not using any nested data structures. After some investigation, I've discovered that the Apache::Session::Store::MySQL::DESTROY routine is getting called before the Apache::Session::MySQL::DESTROY routine, so when the latter is called it doesn't have any way to write to the database. I think Perl is supposed to guarantee that the outer object's DESTROY is called before the inner object's, but I think maybe this guarantee doesn't extend to the global destruction phase. So I'm wondering why the session is getting cleaned up in global destruction rather than refcount destruction. I've declared %session as a locally-scoped variable, so it should evaporate before global destruction, unless it's got circular data structures or something. Anyone know what might be going on? This is Apache::Session version 1.53. Note: this problem isn't related to mod_perl, but IIRC this list is the proper place for discussing Apache::Session. -Ken
cvs commit: modperl-2.0/t .cvsignore SMOKE.PL TEST.PL
stas01/12/31 00:45:13
Modified:.Makefile.PL
t.cvsignore
Removed: buildbugreport.pl
tSMOKE.PL TEST.PL
Log:
- start using the generate_script functions for auto-generating TEST,
REPORT and SMOKE utils
Revision ChangesPath
1.58 +11 -8 modperl-2.0/Makefile.PL
Index: Makefile.PL
===
RCS file: /home/cvs/modperl-2.0/Makefile.PL,v
retrieving revision 1.57
retrieving revision 1.58
diff -u -r1.57 -r1.58
--- Makefile.PL 25 Dec 2001 12:46:10 - 1.57
+++ Makefile.PL 31 Dec 2001 08:45:13 - 1.58
@@ -3,16 +3,19 @@
use warnings FATAL = 'all';
use lib qw(lib Apache-Test/lib);
-use Config;
-use Apache::Build ();
-use ModPerl::Code ();
-use ModPerl::MM ();
+use Config;
use File::Spec::Functions;
use DirHandle ();
+use Apache::Build ();
+use Apache::TestReportPerl ();
+use Apache::TestSmokePerl ();
+use Apache::TestRunPerl ();
use Apache::TestTrace;
use Apache::TestMM qw(test);
+use ModPerl::Code ();
+use ModPerl::MM ();
our $VERSION;
@@ -34,7 +37,6 @@
my @remote_makefile_dirs = Apache::Build::is_win32() ? () :
qw(docs/api/mod_perl-2.0);
-my @scripts = qw(t/TEST t/SMOKE);
configure();
ModPerl::MM::WriteMakefile(
@@ -103,8 +105,9 @@
#ModPerl::MM will use Apache::BuildConfig in subdir/Makefile.PL's
$build-save;
-for (@scripts) {
-Apache::TestMM::generate_script($_);
+for my $util (qw(Report Smoke Run)) {
+my $class = Apache::Test${util}Perl;
+$class-generate_script;
}
my $tables_dir = tables_dir($httpd_version);
@@ -167,7 +170,7 @@
sub clean_files {
my $path = $code-path;
-return [@{ $build-clean_files }, @scripts,
+return [@{ $build-clean_files },
xs/*.exp, xs/*.def,
map { $path/$_} @{ $code-clean_files }];
}
1.4 +1 -0 modperl-2.0/t/.cvsignore
Index: .cvsignore
===
RCS file: /home/cvs/modperl-2.0/t/.cvsignore,v
retrieving revision 1.3
retrieving revision 1.4
diff -u -r1.3 -r1.4
--- .cvsignore10 Dec 2001 07:44:53 - 1.3
+++ .cvsignore31 Dec 2001 08:45:13 - 1.4
@@ -1,5 +1,6 @@
TEST
SMOKE
+REPORT
.gdb-test-start
htdocs
logs
cvs commit: modperl-2.0/xs/maps apr_types.map
stas01/12/31 04:11:48
Modified:lib/Apache ParseSource.pm
xs/maps apr_types.map
Log:
apr sources adjustment: new enum apr_lockmech_e
Revision ChangesPath
1.33 +1 -1 modperl-2.0/lib/Apache/ParseSource.pm
Index: ParseSource.pm
===
RCS file: /home/cvs/modperl-2.0/lib/Apache/ParseSource.pm,v
retrieving revision 1.32
retrieving revision 1.33
diff -u -r1.32 -r1.33
--- ParseSource.pm7 Nov 2001 03:14:54 - 1.32
+++ ParseSource.pm31 Dec 2001 12:11:48 - 1.33
@@ -174,7 +174,7 @@
my %enums_wanted = (
Apache = { map { $_, 1 } qw(cmd_how input_mode filter_type) },
-APR = { map { $_, 1 } qw(apr_shutdown_how apr_read_type) },
+APR = { map { $_, 1 } qw(apr_shutdown_how apr_read_type apr_lockmech) },
);
my $defines_unwanted = join '|', qw{
1.11 +1 -0 modperl-2.0/xs/maps/apr_types.map
Index: apr_types.map
===
RCS file: /home/cvs/modperl-2.0/xs/maps/apr_types.map,v
retrieving revision 1.10
retrieving revision 1.11
diff -u -r1.10 -r1.11
--- apr_types.map 17 Sep 2001 00:38:07 - 1.10
+++ apr_types.map 31 Dec 2001 12:11:48 - 1.11
@@ -93,6 +93,7 @@
apr_os_lock_t| NOTIMPL
struct apr_lock_t| APR::Lock
apr_locktype_e | IV
+apr_lockmech_e | IV
apr_lockscope_e | IV
apr_readerwriter_e | IV
apr_lockmech_e_np| IV
cvs commit: modperl-2.0/xs/APR/Lock APR__Lock.h
stas01/12/31 04:12:54
Modified:xs/maps apr_functions.map
xs/APR/Lock APR__Lock.h
Log:
apr sources adjustment: new argument 'mech' added to apr_lock_create
Revision ChangesPath
1.27 +1 -1 modperl-2.0/xs/maps/apr_functions.map
Index: apr_functions.map
===
RCS file: /home/cvs/modperl-2.0/xs/maps/apr_functions.map,v
retrieving revision 1.26
retrieving revision 1.27
diff -u -r1.26 -r1.27
--- apr_functions.map 15 Dec 2001 23:51:13 - 1.26
+++ apr_functions.map 31 Dec 2001 12:12:54 - 1.27
@@ -164,7 +164,7 @@
MODULE=APR::Lock
!apr_lock_child_init
apr_lock_t *:apr_lock_create | mpxs_ | \
- SV *:CLASS, pool, type=0, scope=1, fname=lock.file | new
+ SV *:CLASS, pool, type=0, mech, scope=1, fname=lock.file | new
-apr_lock_destroy
apr_lock_DESTROY | | lock
apr_lock_data_get
1.2 +2 -1 modperl-2.0/xs/APR/Lock/APR__Lock.h
Index: APR__Lock.h
===
RCS file: /home/cvs/modperl-2.0/xs/APR/Lock/APR__Lock.h,v
retrieving revision 1.1
retrieving revision 1.2
diff -u -r1.1 -r1.2
--- APR__Lock.h 5 Mar 2001 03:53:28 - 1.1
+++ APR__Lock.h 31 Dec 2001 12:12:54 - 1.2
@@ -3,10 +3,11 @@
static MP_INLINE apr_lock_t *mpxs_apr_lock_create(pTHX_ SV *CLASS,
apr_pool_t *p,
apr_locktype_e type,
+ apr_lockmech_e mech,
apr_lockscope_e scope,
const char *fname)
{
apr_lock_t *retval=NULL;
-(void)apr_lock_create(retval, type, scope, fname, p);
+(void)apr_lock_create(retval, type, mech, scope, fname, p);
return retval;
}
cvs commit: modperl-2.0/xs/tables/current/ModPerl FunctionTable.pm
stas01/12/31 04:16:37
Modified:xs/tables/current/Apache ConstantsTable.pm FunctionTable.pm
StructureTable.pm
xs/tables/current/ModPerl FunctionTable.pm
Log:
sync to the latest httpd/apr
Revision ChangesPath
1.20 +10 -1 modperl-2.0/xs/tables/current/Apache/ConstantsTable.pm
Index: ConstantsTable.pm
===
RCS file: /home/cvs/modperl-2.0/xs/tables/current/Apache/ConstantsTable.pm,v
retrieving revision 1.19
retrieving revision 1.20
diff -u -r1.19 -r1.20
--- ConstantsTable.pm 15 Dec 2001 23:51:43 - 1.19
+++ ConstantsTable.pm 31 Dec 2001 12:16:37 - 1.20
@@ -2,7 +2,7 @@
# !!
# ! WARNING: generated by Apache::ParseSource/0.02
-# ! Sat Dec 15 15:39:22 2001
+# ! Mon Dec 31 20:02:32 2001
# ! do NOT edit, any changes will be lost !
# !!
@@ -221,6 +221,13 @@
'APR_POLLHUP',
'APR_POLLNVAL'
],
+'lockmech' = [
+ 'APR_LOCK_FCNTL',
+ 'APR_LOCK_FLOCK',
+ 'APR_LOCK_SYSVSEM',
+ 'APR_LOCK_PROC_PTHREAD',
+ 'APR_LOCK_DEFAULT'
+],
'limit' = [
'APR_LIMIT_CPU',
'APR_LIMIT_MEM',
@@ -347,6 +354,8 @@
'APR_ENETUNREACH',
'APR_EFTYPE',
'APR_EPIPE',
+ 'APR_EXDEV',
+ 'APR_ENOTEMPTY',
'APR_END'
],
'common' = [
1.29 +508 -60 modperl-2.0/xs/tables/current/Apache/FunctionTable.pm
Index: FunctionTable.pm
===
RCS file: /home/cvs/modperl-2.0/xs/tables/current/Apache/FunctionTable.pm,v
retrieving revision 1.28
retrieving revision 1.29
diff -u -r1.28 -r1.29
--- FunctionTable.pm 15 Dec 2001 23:51:43 - 1.28
+++ FunctionTable.pm 31 Dec 2001 12:16:37 - 1.29
@@ -2,7 +2,7 @@
# !!
# ! WARNING: generated by Apache::ParseSource/0.02
-# ! Sat Dec 15 15:39:32 2001
+# ! Mon Dec 31 19:58:28 2001
# ! do NOT edit, any changes will be lost !
# !!
@@ -347,6 +347,11 @@
},
{
'return_type' = 'int',
+'name' = 'ap_calc_scoreboard_size',
+'args' = []
+ },
+ {
+'return_type' = 'int',
'name' = 'ap_cfg_closefile',
'args' = [
{
@@ -555,6 +560,28 @@
},
{
'return_type' = 'void',
+'name' = 'ap_create_sb_handle',
+'args' = [
+ {
+'type' = 'void **',
+'name' = 'new_handle'
+ },
+ {
+'type' = 'apr_pool_t *',
+'name' = 'p'
+ },
+ {
+'type' = 'int',
+'name' = 'child_num'
+ },
+ {
+'type' = 'int',
+'name' = 'thread_num'
+ }
+]
+ },
+ {
+'return_type' = 'void',
'name' = 'ap_create_scoreboard',
'args' = [
{
@@ -2197,12 +2224,8 @@
'name' = 'ap_increment_counts',
'args' = [
{
-'type' = 'int',
-'name' = 'child_num'
- },
- {
-'type' = 'int',
-'name' = 'thread_num'
+'type' = 'void *',
+'name' = 'sbh'
},
{
'type' = 'request_rec *',
@@ -2236,6 +2259,11 @@
},
{
'return_type' = 'void',
+'name' = 'ap_init_scoreboard',
+'args' = []
+ },
+ {
+'return_type' = 'void',
'name' = 'ap_init_vhost_config',
'args' = [
{
@@ -3866,6 +3894,10 @@
{
'type' = 'int',
'name' = 'conn_id'
+ },
+ {
+'type' = 'void *',
+'name' = 'sbh'
}
]
},
@@ -3906,24 +3938,24 @@
'name' = 'level'
},
{
-'type' = 'apr_status_t status',
-'name' = 'arg3'
+'type' = 'apr_status_t',
+'name' = 'status'
},
{
'type' = 'const server_rec *',
'name' = 's'
},
{
-'type' = 'const request_rec *r',
-'name' = 'arg5'
+'type' = 'const request_rec *',
+'name' = 'r'
},
{
'type' = 'apr_pool_t *',
'name' = 'pool'
},
{
-'type' = 'const char *errstr',
-'name' = 'arg7'
+'type' = 'const char *',
+'name' = 'errstr'
}
]
},
@@ -4979,6 +5011,24 @@
'name' = 'ap_update_child_status',
'args' = [
{
+'type' = 'void *',
+'name' = 'sbh'
+ },
+ {
+'type' = 'int',
+'name' = 'status'
+ },
+ {
+
