Re: #perl SSI directive not recognised
Doug MacEachern wrote: > > I want Apache to switch into a > > resource-managed Project at startup, but to do so I need to find out the > > value of the "User" parameter in httpd.conf, and I can't find a way to do > > this - is there one? > > Apache::Server::uid returns the uid of the configured User. > you can access via: > Apache->server->uid at startup time > or > $r->server->uid at request time Brilliant - works a treat. Thanks :-) -- Alan Burlison -- $ head -1 /dev/bollocks initiate customer-directed OEMs
Re: #perl SSI directive not recognised
Doug MacEachern wrote:
> the #perl directive is disabled if modperl is built as dso, Makefile.PL
> prints a message about this. won't be an issue with 2.0 thanks to apr
> optional functions. but in 1.x, the modperl .so cannot resolve symbols
> from mod_include.so. at least, it can't on all platforms and can't if one
> were to disable LoadModule of mod_include.so
Thanks Doug. I'm using the Apache/perl/mod_perl that will ship as part of
Solaris 9, so I was a little concerned that we'd screwed something up :-)
>From your description, I'm guessing that the root cause of this that that
two dlopen'ed shared objects need to be able to cross-call each other,
correct? This might not be of much use if you already have a fix for 2.0,
but I have a little trick which makes this work on Solaris, at least. You
stick the following in the call to WriteMakefile in Makefile.PL
dynamic_lib => { OTHERLDFLAGS =>
'-h $(DLBASE).$(DLEXT) ' .
'-R\$$ORIGIN/.. $(INST_ARCHAUTODIR)/../OtherModule.so'
},
The '-h' establishes a location-independent name for the XSUB you are
currently building. The '-R' establishes a runpath to search for the
other .so that you want to link against, and the MyOtherModule.so bit says
where to find the other .so, relative to the obe you are currently
building. Here is a bit more info from a Makefile.PL where I actually
use this trick:
# The various .so files that comprise this module need to be able to
# cross-call each other, and therefore to prevent runtime linker errors it
is
# necessary to establish linker dependencies between the various .so
files.
#
# This causes several problems. The first of these is that perl .so files
are
# built in one directory (under ../blib in this case) and installed into
# another, so it is necessary to record the dependency using a path
relative to
# the dependent. This can be done with the $ORIGIN linker mechanism.
#
# The second problem is that it is necessary to specify the name of the
# dependee at link edit time in a manner that doesn't result in the
build-time
# path of the dependee being hard coded in to the dependent, as this would
# stop ld.so.1 performing its normal search process for the files. This
can't
# be done with the normal '--L/my/lib/path -lmylib' mechanism, because the
XSUB
# .so files aren't prefixed with 'lib'. To do this the -h linker flag is
used
# to explicitly set the SONAME in the dependee. This is then used as the
name
# of the dependent in the dependee rather than the full path by which it
was
# found at link edit time.
The appropriate bits of a 'dump -Lv of the resulting .so file are:
[INDEX] Tag Value
[1] NEEDED OtherModule.so <-- Dependency
[4] SONAME ThisModule.so <-- Name of this module
[5] RUNPATH $ORIGIN/.. <-- Where to search
[6] RPATH $ORIGIN/..
Hope this might help somebody sometime - it may be possible to pull a
similar stunt on other platforms.
Alan Burlison
#perl SSI directive not recognised
I'm sure this is a FAQ, but I can't find it anywhere, and the mail archives seem to be down. I am trying to get the following SSI directive to work: and I get the following error in the server log: unknown directive "perl" in parsed doc /var/apache/htdocs/index.html I know that I have SSI set up OK because if I replace the above with I see the date as expected. The /perl-status pages show that the MySSI module is loaded, and that the PerlSSI directive is enabled. I've carefully read through the example in the eagle book, and I can't spot what is wrong. Can someone spare me a clue? Thanks, -- Alan Burlison -- $ head -1 /dev/bollocks risk manage three-tier high-volume price performance
Re: Children dying
Stas Bekman wrote:
> > > No need for an apology :-) The trick is to build perl using the
> > > Solaris malloc (-Dusemymalloc as a flag to Configure), then apache,
> > > mod_perl and perl all agree on who manages memory.
> >
> > Might I suggest that this golden piece of information find it's way into the
> > guide? It's so rare to see a DEFINITIVE answer to one of the many ("YMMV!"
> > :-)exceptions to the vanilla mod_perl build process.
>
> The definitive answer is there for at least 2 years: "If in doubt compile
> statically", which covers Solaris as well. Why having a special case?
So what is the point of having DSO at all then?
The question was 'How do I build on Solaris with DSO?', the answer was
'Build perl to use the system malloc', I don't see what the problem with
that is.
--
Alan Burlison
--
$ head -1 /dev/bollocks
repurpose collaborative focus groups, going forwards
Re: Children dying
Andrew Ho wrote: > AB>Untrue. We ship mod_perl in Solaris 8 as a DSO, and it works fine. > > I apologize. Let me qualify my original statement. In general, you want to > compile mod_perl statically on Solaris 2.6 or 2.7 because in many > instances, it core dumps when built as a DSO. FWIW, my particular > experiences were with Perl 5.005_03 and 5.6.0, mod_perl 1.24 and 1.25, and > Apache 1.3.12, 1.3.14, 1.3.17, and 1.3.19 under Solaris 2.6 (both Sparc > and Intel) and 2.7 (Intel only). No need for an apology :-) The trick is to build perl using the Solaris malloc (-Dusemymalloc as a flag to Configure), then apache, mod_perl and perl all agree on who manages memory. > Humbly, And no need for that either! :-) Regards, -- Alan Burlison -- $ head -1 /dev/bollocks drive proximal infomediaries, going forwards
Re: Children dying
"Vasily S. Petrushin" wrote: > > Untrue. We ship mod_perl in Solaris 8 as a DSO, and it works fine. > > Fine, point us please to documentation how to do it. http://cpan.valueclick.com/authors/Doug_MacEachern/mod_perl-1.26.tar.gz Alan Burlison
Re: Children dying
Andrew Ho wrote: > A few other folks have given useful references on how to get stack traces, > as well as some other common causes of core dumps (compiling Apache with > its bundled expat is a big one). Here's another one--did you build > mod_perl on Solaris as a DSO? In general, you want to compile mod_perl > statically on Solaris because it will core otherwise. Untrue. We ship mod_perl in Solaris 8 as a DSO, and it works fine. Alan Burlison
Re: compiling troubles on Solaris 8
Geoffrey Young wrote: > > As an aside, Solaris 8 comes with prebuilt versions of Apache > > and mod_perl, > > does anyone familiar with HP-UX, AIX, or IRIX know whether this is true of > these platforms as well? > > Whether they are DSO mod_perl or not would also be helpful. On Solaris it is built as a DSO. -- Alan Burlison Solaris Kernel Development, Sun Microsystems
Re: compiling troubles on Solaris 8
Ged Haywood wrote: > Well OK, read "What Compiler Should Be Used to Build mod_perl?" in the > "install" section of Stas' new book if you want to use different compilers > - but don't say I didn't warn you! :) By all means, please feel free to buy our compiler. Alan Burlison
Re: compiling troubles on Solaris 8
> > > When I first started, it did not seem to be using gcc, so I renamed > > > /usr/ucb/cc to cc.default, and make /usr/ucb/cc a link to gcc. Please don't do that - switch it back and just make sure gcc is somewhere on your path. > Please note that the same compiler must be used to build Perl and mod_perl, Not so. gcc should work just fine. The problem is that when perl is configured and built, it saves the compiler name and flags in Config.pm. We build Solaris with the Workshop compilers, and therefore subsequent attempts to build modules (e.g. mod_perl) also try to use the same compiler. A workaround is to modify /usr/perl5/5.00503/sun4-solaris/Config.pm so it contains the correct information (flags etc) for gcc rather than Workshop. I've attached a sample Config.pm that should work with the copy of gcc that is shipped on the Solaris 8 companion cd. I suggest you move the standard Config.pm somewhere safe and use the attached file in its place. There are only 4 differences between it and the standard Config.pm. I'm working on something a little cleaner for Solaris 9, probably requiring you to set PERL5LIB to point perl to a gcc-ized version of Config.pm, rather than the standard one. As an aside, Solaris 8 comes with prebuilt versions of Apache and mod_perl, so unless you need the latest and greatest you might find it easiest to use the bundled versions - see 'man apache' for details. Alan Burlison Solaris Kernel Development, Sun Microsystems Config.pm.gz
Re: Problems building Bundle::Apache - libapreq-0.33
Stas Bekman wrote: > > Here is the output of 'make install' > > I don't think it logs the install of these headers. Please check the > created Makefile. Does it include the mod_perl.h in one of the targets? No. Alan Burlison
Re: Problems building Bundle::Apache - libapreq-0.33
Stas Bekman wrote: > Looks like an issue with APACHE_HEADER_INSTALL > http://perl.apache.org/guide/install.html#APACHE_HEADER_INSTALL > > Did you install mod_perl by yourself? Yes, as my attempt to build it as part of Bundle::Apache barfed. Here is the contents of my makepl_args.mod_perl: APACHE_SRC=/home1/software/apache/build/apache_1.3.20/src EVERYTHING=1 USE_APXS=1 WITH_APXS=/home1/web/apache_1.3.20/bin/apxs Here is the output of 'make install' $ make install (cd ./apaci && PERL5LIB=/home1/software/perl/cpan/build/mod_perl-1.26/lib /usr/ccs/bin/make) Files found in blib/arch: installing files in blib/lib into architecture dependent library tree Installing /usr/perl5/5.6.1/man/man3/Apache::PerlRunXS.3 Installing /usr/perl5/5.6.1/man/man3/Apache.3 Installing /usr/perl5/5.6.1/man/man3/Apache::Log.3 Installing /usr/perl5/5.6.1/man/man3/Apache::Constants.3 Installing /usr/perl5/5.6.1/man/man3/Apache::File.3 Installing /usr/perl5/5.6.1/man/man3/Apache::URI.3 Installing /usr/perl5/5.6.1/man/man3/Apache::Table.3 Installing /usr/perl5/5.6.1/man/man3/Apache::Util.3 Installing /usr/perl5/5.6.1/man/man3/Apache::Symbol.3 Installing /usr/perl5/5.6.1/man/man3/Apache::Leak.3 Installing /usr/perl5/5.6.1/man/man3/mod_perl_cvs.3 Installing /usr/perl5/5.6.1/man/man3/Apache::Registry.3 Installing /usr/perl5/5.6.1/man/man3/Apache::SizeLimit.3 Installing /usr/perl5/5.6.1/man/man3/cgi_to_mod_perl.3 Installing /usr/perl5/5.6.1/man/man3/Apache::Resource.3 Installing /usr/perl5/5.6.1/man/man3/Apache::PerlSections.3 Installing /usr/perl5/5.6.1/man/man3/Apache::PerlRun.3 Installing /usr/perl5/5.6.1/man/man3/Apache::Debug.3 Installing /usr/perl5/5.6.1/man/man3/Apache::Symdump.3 Installing /usr/perl5/5.6.1/man/man3/mod_perl_tuning.3 Installing /usr/perl5/5.6.1/man/man3/Apache::Status.3 Installing /usr/perl5/5.6.1/man/man3/Apache::RedirectLogFix.3 Installing /usr/perl5/5.6.1/man/man3/Apache::ExtUtils.3 Installing /usr/perl5/5.6.1/man/man3/Apache::Include.3 Installing /usr/perl5/5.6.1/man/man3/mod_perl_method_handlers.3 Installing /usr/perl5/5.6.1/man/man3/Apache::StatINC.3 Installing /usr/perl5/5.6.1/man/man3/Apache::test.3 Installing /usr/perl5/5.6.1/man/man3/Apache::RegistryLoader.3 Installing /usr/perl5/5.6.1/man/man3/Apache::httpd_conf.3 Installing /usr/perl5/5.6.1/man/man3/Apache::FakeRequest.3 Installing /usr/perl5/5.6.1/man/man3/mod_perl.3 Installing /usr/perl5/5.6.1/man/man3/mod_perl_traps.3 Installing /usr/perl5/5.6.1/man/man3/Apache::src.3 Installing /usr/perl5/5.6.1/man/man3/Apache::SIG.3 Installing /usr/perl5/5.6.1/man/man3/Bundle::Apache.3 Installing /usr/perl5/5.6.1/man/man3/Apache::Options.3 Writing /usr/perl5/site_perl/5.6.1/sun4-solaris-64int/auto/mod_perl/.packlist (cd ./apaci && /usr/ccs/bin/make install;) /home1/web/apache_1.3.20/bin/apxs -i -a -n perl libperl.so [activating module `perl' in /home1/web/apache_1.3.20/conf/httpd.conf] cp libperl.so /home1/web/apache_1.3.20/libexec/libperl.so chmod 755 /home1/web/apache_1.3.20/libexec/libperl.so cp /home1/web/apache_1.3.20/conf/httpd.conf /home1/web/apache_1.3.20/conf/httpd.conf.bak cp /home1/web/apache_1.3.20/conf/httpd.conf.new /home1/web/apache_1.3.20/conf/httpd.conf rm /home1/web/apache_1.3.20/conf/httpd.conf.new Appending installation info to /usr/perl5/5.6.1/lib/sun4-solaris-64int/perllocal.pod No header files installed. Alan Burlison
Re: Problems building Bundle::Apache - libapreq-0.33
Jens-Uwe Mager wrote: > Did you make install modperl yet? Yes, via APXS. mod_perl is installed and works just fine. I can see the missing mod_perl.h file in the modperl build directory, but it doesn't appear anywhere under site_perl - should mod_perl copy it somewhere during 'make install'? I'm using a slightly non-standard perl layout, so instead of being under /usr/local perl lives under /usr/perl5. From looking at where stuff generally goes, and the -I flags used when building libapreq I guess mod_perl.h should be somewhere under /usr/perl5/site_perl/5.6.1/sun4-solaris-64int/auto/Apache, but it isn't there. Alan Burlison
Problems building Bundle::Apache - libapreq-0.33
I'm trying to build Bundle::Apache and I get the following error when building libapreq (everthing else in the bundle builds fine): cc -c -I../c -I/usr/perl5/site_perl/5.6.1/sun4-solaris-64int/auto/Apache/include -I/usr/perl5/site_perl/5.6.1/sun4-solaris-64int/auto/Apache/include/include -I/usr/perl5/site_perl/5.6.1/sun4-solaris-64int/auto/Apache/include/regex -I/usr/perl5/site_perl/5.6.1/sun4-solaris-64int/auto/Apache/include/os/unix -I/home1/web/apache_1.3.20/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -xO3 -xdepend-DVERSION=\"0.33\" -DXS_VERSION=\"0.33\" -KPIC -I/usr/perl5/5.6.1/lib/sun4-solaris-64int/CORE Request.c "Request.xs", line 40: cannot find include file: "mod_perl.h" "Request.xs", line 86: syntax error before or at: SV "Request.xs", line 86: cannot recover from previous errors cc: acomp failed for Request.c The platform is Solaris, Perl 5.6.1 and Apache 1.3.20. Everything else is the latest available from CPAN. The INSTALL file is less than helpful, and a search of the modperl archives didn't find anything. Anyone have any suggestions? thanks, Alan Burlison
Re: Bitten by -D_FILE_OFFSET_BITS=64
"Paul G. Weiss" wrote:
> This has been fixed in a post 1.23 patch.
> Change the line
>
> $PERL_EXTRA_CFLAGS = "";
>
> to
>
> $PERL_EXTRA_CFLAGS = $] >= 5.006 ? $Config{ccflags} : "";
>
> and try again.
Note that this is not a complete fix. It will only work if you allow
mod_perl to build apache, as instructed in the original patch.
Unfortunately if you are using APXS this is not the case. In this case,
when you initially build Apache you need to do so with the flags
required to make it largefile aware. I think it might also be a good
idea to put a test in mod_perl's Makefile.PL to check that the
largefile-ness of both perl and apache agree when using APXS. On
Solaris you can do this by using nm and looking to see if the normal or
largefile versions of routines are in use, e.g. fstat or fsta64. I
don't know how portable to other platforms this is though.
--
Alan Burlison
Re: segfault in DSO mod_perl 1.23 (perl 5.6.0) with apache-1.3.12
Michael Poole wrote: > In hopes that this would fix the APXS build, I tried rebuilding with > that, but whenever the httpd tried to load the php3 or perl module, it > would die in pthread_mutex_lock. This was slightly odd, since php3 > would load fine when httpd was statically linked against mod_perl. > For the time being, I'll just roll back to using static mod_perl and > dynamic mod_ssl and php3. No, that's what you would expect. The problem is being caused because perl amd mod_perl are being built to be largefile aware, whereas by default Apache is not. By following Doug's instructions to let mod_perl build httpd you are building it with the same flags needed to make perl/modperl largefile aware, so Apache too ends up being largefile aware and all the bits then are in sync. If you use APXS you are building Apache seperately and with the standard config, i.e. not largefile aware, so it breaks. If you want to use APXS you have to build Apache to be largefile aware yourself. On Solaris at least the way to do this is to set CFLAGS='-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64' in your environment before invoking the Apache configure. -- Alan Burlison
Re: Bitten by -D_FILE_OFFSET_BITS=64
Ari Jolma wrote: > (This may be a general Perl question but since I have this problem > only with mod_perl I'm asking it here.) Perl's (5.6.0) Configure script > puts -D_FILE_OFFSET_BITS=64 into Config.pm's variable ccflags in > my machine (redhat 6.1 with linux 2.2.12). This causes for some > reason mod_perl.c to have stat structure with size 96 while it > has the size 88 in apache (and it is 88 usually). The result is > very poor behavior in make test. When I manually remove that > compiler directive everything works fine. My mod_perl is 1.23 > and apache is 1.3.12. I believe I've had this problem for long since > over the years :-) I've had this same problem occasionally but only > now investigated it thoroughly using gdb. Can anybody give an > explanation what is going on here? I was bitten my a similar problem on Solaris. Basically I built perl with the default configure settings on Solaris 8, which result in Perl being made largefile aware, which is A Good Thing. However, when I built Apache it was built without being largefile aware, which is A Very Bad Thing - at least if you want to use mod_perl, that is. This is because Perl and Apache don't agree on the size of off_t and other such things that change between the 32 and 64 bit file offset worlds. I fixed this by the simple expedient of making Apache largefile aware. On Solaris this was easy - I just set CFLAGS="-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64" in my environment before calling the Apache configure. Apache built with no errors, and as far as I can tell up until now it works just fine, including mod_perl. Obviously you are on a different platform, but perhaps building Apache with the Linux incantation needed to make it largefile aware will solve the problem for you? Alan Burlison
Re: Core dump: Solaris 2.7/mod_perl1.23/Apache1.3.12/perl5.6.0
Some more info: I've rebuilt everything with debug, and the SEGV is
happening in mod_perl.c in the following code:
#ifdef PERL_HEADER_PARSER
int PERL_HEADER_PARSER_HOOK(request_rec *r)
{
dSTATUS;
dPPDIR;
#ifdef PERL_INIT
PERL_CALLBACK("PerlInitHandler",
cld->PerlInitHandler);
#endif
PERL_CALLBACK("PerlHeaderParserHandler",
cld->PerlHeaderParserHandler);
return status;
}
#endif
specifically the macro dPPDIR. I notice that the request_rec pointer
passed in has one field that points to an invalid address, I don't know
if it is relevant:
parsed_uri= {
scheme = (nil)
hostinfo = (nil)
user = 0x174fa0 "/"
password = (nil)
hostname = (nil)
port_str = (nil)
path = 0x8000 ""
query = 0x175418 ""
fragment = 0x174e20 ""
hostent= 0x1755a0
port = 23U
is_initialized = 0
dns_looked_up = 1U
dns_resolved = 0
expanding out dPPDIR it refers to r->per_dir_config and &perl_module.
r->per_dir_config is a void*, and has the value 8, which sure doesn't
look like a valid address to me. perl_module on the other hand all
looks sane. I'm not sure where to look next - any suggestions?
--
Alan Burlison
Core dump: Solaris 2.7/mod_perl1.23/Apache1.3.12/perl5.6.0
This is a new one on me. mod_perl is built with APXS and perl is built with -DDL_UNLOAD_ALL_AT_EXIT, so it isn't coredumping during startup, but it cores the first time an access is made. Anyone have any suggestions? =>[1] perl_header_parser(0xd9480, 0xff16ca04, 0xa, 0xd9480, 0x2f707269, 0x2f617061), at 0xff124acc [2] run_method(0xd9480, 0x74, 0x8f764, 0x1, 0xda4e8, 0x4), at 0x32828 [3] process_request_internal(0xd9480, 0xff0c, 0xd9480, 0xd8440, 0xd9608, 0x40090), at 0x44d74 [4] ap_process_request(0x89000, 0xd9480, 0xd9480, 0xff0c, 0x0, 0x4), at 0x45090 [5] child_main(0x8f800, 0x8f800, 0xd9480, 0x0, 0x0, 0x0), at 0x3d0d0 [6] make_child(0x943d0, 0x0, 0x3916cee9, 0x943d0, 0x0, 0x0), at 0x3d2b8 [7] startup_children(0x8f988, 0x8d048, 0x8902c, 0x3916cee9, 0x0, 0x0), at 0x3d42c [8] standalone_main(0x8f800, 0x8f800, 0x8f824, 0xa912c, 0xb400, 0x8), at 0x3da64 [9] main(0x8f800, 0x8f800, 0x2, 0x88400, 0x0, 0x0), at 0x3e348 Alan Burlison
Re: mod_perl (DSO) dumping core with perl 5.6.0
"Paul G. Weiss" wrote: > > Sad to say still not working. I'd suggest building perl with -Uusemymalloc. Alan Burlison
Re: Modperl 1.22 and Perl 5.6.0
Doug MacEachern wrote: > i've benchmarked the two, it makes a HUGE difference under solaris, Perl's > malloc kicks the sh*t out of solaris system malloc. Perl's malloc is not > as much of a win under linux, i don't have the numbers handy though. The only potential gotcha with perl's malloc is that it doesn't work if you compile perl as a 64-bit app (-Duse64bitall), as it isn't 64 bit clean. I don't suppose there are may people trying to build a 63-bit Apache though... Alan Burlison
Re: A fix in sight: Apache::ASP: crash when placed in startup.pl (2)
Gerald Richter wrote: > > > Seems correct to me, although as I said before the patch should really > > go in DynaLoader - after all it is conceivable that perl embedders other > > than Apache could be hit by this problem. > > > Yes, I agree, but this will not before perl 5.006 and much people are still > using perl 5.004... This is unfortunately true. However, the next version of perl is imminent (ish) so it would be nice if the fix could be in it. Alan Burlison
Re: A fix in sight: Apache::ASP: crash when placed in startup.pl
Doug MacEachern wrote: > wow, *nice* catch!! Daniel, I can't thank you and Alan enough for your > efforts here. it's such a thorny problem to debug, the closest I came was > trying to prevent the dlclose of modperl's libperl.so, but had no idea why > that bandaid prevented the bleeding. I hadn't heard of ltrace, what a > spectacular looking tool. > I'd really like to get the workaround in 1.22, even if Apache and/or > DynaLoader is the "right" place, we don't have any control over the > release schedules of either and modperl-1.22 is already long overdue. You are welcome - it has always bugged me that I could never get APXS to work. Both Perl and Apache are included with Solaris 8 (http://www.sun.com/software/solaris/ea/linux.html), and the only way to add mod_perl to the preinstalled Apache/Perl is to use APXS. I've posted a bug against DynaLoader to P5P. I think the fix is probably to put an END handler in DynaLoader to unload the XS modules, but I'm not familiar enough with the perl interpreter cleanup processing to know if this is the best way. It might add some weight if you could add your comments to the bug preport in P5P - after all, the know who you are ;-) Alan Burlison
Re: A fix in sight: Apache::ASP: crash when placed in startup.pl (2)
Gerald Richter wrote: > Just one more thought: The mp_dso_unload, which Daniel has patched, is > registered as cleanup. So it should be called everytime the modperl > libraries is unloaded and only when modperl library is unloaded, we need to > unload the other libraries. So as far as I see this patch is all that is > neccessary. I am missing something? Seems correct to me, although as I said before the patch should really go in DynaLoader - after all it is conceivable that perl embedders other than Apache could be hit by this problem. Alan Burlison
Re: Why does Apache do this braindamaged dlclose/dlopen stuff?
Alan Burlison wrote: > > AB> from mod_perl -> perl libperl.so). Unfortunately the perl XS modules > > AB> loaded in during startup via dlopen are *not* unloaded, nor do they > > AB> succeed in locking the perl libperl.so into memory (you could construe > > AB> this as a linker bug). Then Apache reloads the mod_perl libperl.so, Actually, I retract that statement. It is *not* a linker bug. By explicitly adding a dependency between the XS .so modules and the perl libperl.so, the problem can be made to dissapear, as ld.so then knows that there is a dependency between the XS module and the perl libperl.so > > I think the linker is in error here for not adding the dependency on > > the library and that is what should be fixed... Nope, the *programmer* (or in this case, MakeMaker) is in error for not specifying the dependency when the XS .so module was built. Here's what one of our linker developers has to say about the subject: [You may be assuming that] > ii) the action of dlopen()'ing an object (say B.so) from an object > (say A.so) creats a binding between the two objects that insure > the latter cannot be deleted before the formed is deleted. > This isn't the case. ld.so.1 maintains bindings/refence counts > etc. between the objects it has control over - ie. a relocation > in one object that references another. The dlopen() family are > thought of as user code, and we do not try to interprete how > objects are bound that use these facilities. In fact we can > get in all sorts of issues if we do - a handle, just like a > function pointer, can be handed to other objects, thus who > knows what object *expects* another object to be in existance. i.e. if you decide to use dlopen/dlclose and you screw it up, then it is *your* fault, not the linkers. The fact that so many different systems show the same behaviour suggests that this is a common linker design decision. I actually think that the real fault is in DynaLoader. If you assume that the exiting of the perl interpreter is the same thing as the exit of the program, then it makes sense not to dlclose anything, as it will all be reclaimed on exit anyway. However, if you embed perl inside an application, the exit of the perl interpreter is certainly *not* the same thing as the program exiting, and DynaLoader should explicily dlclose all the files it has dlopened. I'm going to submit this as a bug to p5p. Alan Burlison
Re: Why does Apache do this braindamaged dlclose/dlopen stuff?
Gerald Richter wrote: > This works only if this dependencies are know at link time, but look at the > source of Dynloader. You can retrieve address of any (public)symbol inside a > library dynamicly at runtime. Now you have the entry address and can pass it > around. No linker will ever have a chance to know what you do in your > programm. As soon as you use such things (and Dynloader uses them), the > linker doesn't have chance! Nope, that's not how it works. Take a look at http://docs.sun.com:80/ab2/coll.45.10/LLM/@Ab2PageView/5121 *All* symbols in a shared library are known by ld.so Alan Burlison
Re: Why does Apache do this braindamaged dlclose/dlopen stuff?
Gerald Richter wrote: > That's the same on NT. It seems to occur on all OSs, so it won't help > anything to make the linker responsible, there are to much linkers... and I > am not sure if the linker can know under all circumstances which libraries > to unload. Yes it can. Its main job is to keep track and control the dependencies between libraries. It's just that sometimes thy don't do a particularly good job of it ;-) > I don't think we will convice the Apache people in the short term, that they > shouldn't unload the libraries (also we can discuss it with them). The only > practical solution I see, is the one to unload the libraries, as the patch > already does. The thing left to do is to port it to other OS's (like NT) > which does not have a function named dlclose (we need simply use another > name). This solution will work and if the Apache people one time decide to > not unload the modules, it won't hurt. I think they should be persuaded - this is a very insiduous bug and extremely hard to find. > P.S. Does you get any feedback from your post to p5p about the unload > function in Dynaloader? No. Nothing meaningful. Alan Burlison
Re: Why does Apache do this braindamaged dlclose/dlopen stuff?
Daniel Jacobowitz wrote: > > On Tue, Jan 18, 2000 at 08:03:42PM +, Alan Burlison wrote: > > The current fix is to forcibly unload the perl XS modules during the > > unload. However, on reflection I'm not at all sure this is the correct > > thing to do. Although you can unload the .so component of a perl > > module, you can't unload the .pm component, so just removing the .so > > part as in the current workaround is suspect at least. > > Remember - mod_perl is being unloaded. Perl going away. At this point > perl_destruct/perl_free have already been called, and thus the .pm > components are effectively unloaded. Ah yes, so they are. I still think dlclosing for no good reason sucks though. Alan Burlison
Re: Why does Apache do this braindamaged dlclose/dlopen stuff?
Vivek Khera wrote: > AB> To summarise: Apache dlclose's the mod_perl.so, which then results in > AB> the perl libperl.so being unloaded as well (there's a linker dependency > > Excellent summary... thanks! > > AB> from mod_perl -> perl libperl.so). Unfortunately the perl XS modules > AB> loaded in during startup via dlopen are *not* unloaded, nor do they > AB> succeed in locking the perl libperl.so into memory (you could construe > AB> this as a linker bug). Then Apache reloads the mod_perl libperl.so, > > I think the linker is in error here for not adding the dependency on > the library and that is what should be fixed... Don't worry, I already have a bug report open and someone from the linker group is having a look. I havn't been able to replicate the exact same problem in a minimal set of C files, but I'm working on it. I have a linker debug trace that shows the problem, though (on Solaris, use "LD_DEBUG=files,detail httpd -X" and look for the addresses that the perl libperl.so is mapped in at). However, other folks have reported the exact same problem on other OSs, eg Linux & BSD, so I think that in the short term we need to be realistic and find a way of not tickling what seems to be a common linker bug. Alan Burlison
Re: Why does Apache do this braindamaged dlclose/dlopen stuff?
Gerald Richter wrote: > Sorry, I missed that thread. I have posted this problem more then once here, > because it have beaten me and other often when using Embperl. The problem > there is often more hidden, because it doesn't SIGSEGV, it still works, but > some functionality (where Perl variables are tied to C variables) doesn't > work, so it's often not easy to detect. > > Unfortunably I never had the time to track this down enought to create a > real usefull patch (just a workaround, (PERL_STARTUP_DONE_CHECK), which will > cause the XS libraries only loaded after the second load of libperl.so; this > works for the startup, but not after a restart). To summarise: Apache dlclose's the mod_perl.so, which then results in the perl libperl.so being unloaded as well (there's a linker dependency from mod_perl -> perl libperl.so). Unfortunately the perl XS modules loaded in during startup via dlopen are *not* unloaded, nor do they succeed in locking the perl libperl.so into memory (you could construe this as a linker bug). Then Apache reloads the mod_perl libperl.so, which also results in the perl libperl.so being pulled back in, BUT AT A DIFFERENT ADDRESS! Result: The perl XS modules are linked to a 'ghost' of the originally loaded perl libperl.so. Result: all sorts of strangeness - coredumps, variables not being updated etc etc etc. The current fix is to forcibly unload the perl XS modules during the unload. However, on reflection I'm not at all sure this is the correct thing to do. Although you can unload the .so component of a perl module, you can't unload the .pm component, so just removing the .so part as in the current workaround is suspect at least. I think the correct fix is for the Apache core to avoid dlclosing anything it has dlopened in the first place. If new modules have been added to the config files, they should be dlopened, but any old ones should *not* be dlclosed, EVEN IF THEY ARE NO LONGER IN THE CONFIG FILE!!! I firmly believe this needs fixing in the Apache core, not by hacking around it in mod_perl. Alan Burlison
Why does Apache do this braindamaged dlclose/dlopen stuff?
Can someone please explain why Apache does all the dlclosing and dlopening of shared files on startup and a restart? I can think of no reson why this would ever be necessary - why on earth is it done? Alan Burlison
Re: DSO SEGV fix (was: A fix in sight: Apache::ASP: crash when placed in startup.pl)
"Frank D. Cringle" wrote: > Daniel Jacobowitz <[EMAIL PROTECTED]> writes: > > This worked! Can I get opinions on the attached patch? > > I am not competent to check whether the patch has any hidden problems, > but I hope Doug will be able to evaluate it and include it in the next > release. Your description of the sequence leading up to the crash > fits well with what I observed when trying to trace the problem with > gcc. > > It looks like Alan Burlison is on to a different dynamic linking > problem that also needs to be fixed. No, it's the exactly the same problem, and the fix works for me too. Alan Burlison
Apache 1.3.9 + mod_perl 1.21 + Solaris 2.7 dumps core
$ httpd -X Segmentation Fault(coredump) I thought this was the old 'apache & modperl use a different malloc' problem, so I recompiled Perl with -Uusemymalloc, along with all the other installed modules. No joy - it still core dumps. Here's the stack trace: (/package/SC5.0/bin/../WS5.0/bin/sparcv9/dbx) where =>[1] boot_Data__Dumper(0x4, 0xfef582a0, 0xfef62f54, 0x285814, 0xff0e7734, 0xfeff52f4), at 0xfefe4e0c [2] Perl_pp_entersub(0xfef62f1c, 0xfef582a0, 0xfef67734, 0x2857a8, 0x7, 0x33f170), at 0xfeef4dd4 [3] Perl_runops_standard(0xfef582a0, 0xfef676f8, 0x265a20, 0xfef62f10, 0xe6, 0xfef676f8), at 0xfeeeccb0 [4] perl_eval_sv(0xfef676f8, 0x0, 0xfef67200, 0x2, 0x2bc12c, 0xff1bdbc4), at 0xfeea33ac [5] perl_require_module(0x93eb0, 0x0, 0xff17597c, 0x0, 0xffbefdcc, 0x10), at 0xff13d754 [6] perl_cmd_module(0x93d20, 0x93db8, 0x93eb0, 0xff17597c, 0xffbefb04, 0xa), at 0xff1377a4 [7] invoke_cmd(0xff177e64, 0xffbefb04, 0x93db8, 0xffbedaab, 0xffbefb04, 0xff177e64), at 0x32790 [8] ap_handle_command(0x6f810, 0xff177e64, 0xffbedaa0, 0x258390, 0x0, 0x93200), at 0x32b7c [9] ap_srm_command_loop(0x2000, 0x93200, 0x93200, 0xffbefb04, 0x2f687474, 0x2f616363), at 0x32c50 [10] ap_process_resource_config(0x93b10, 0x893c4, 0x92640, 0x258250, 0x92668, 0x92640), at 0x32fd8 [11] ap_read_config(0x92640, 0x258250, 0x893c4, 0x8dab8, 0x0, 0x92640), at 0x337cc [12] standalone_main(0x8dc00, 0x8d800, 0x8dabc, 0x260b8c, 0xb400, 0x8), at 0x3c69c [13] main(0x8d800, 0x8dc00, 0x2, 0x86800, 0x0, 0x0), at 0x3d05c i.e. in the XS startup code for Data::Dumper (which *is* built with system malloc). Any clues or cuggestions, anyone? Here's the rest of my config information: My config: Apache 1.3.9 mod_perl 1.21 Solaris 2.7 $ cat makepl_args.modperl APACHE_SRC=/home2/web/build/apache_1.3.9/src EVERYTHING=1 USE_APXS=1 WITH_APXS=/home2/web/apache_1.3.9/bin/apxs $ perl -V Summary of my perl5 (5.0 patchlevel 5 subversion 3) configuration: Platform: osname=solaris, osvers=2.7, archname=sun4-solaris uname='sunos fubar 5.7 generic_106541-08 sun4u sparc sunw,ultra-30 ' hint=recommended, useposix=true, d_sigaction=define usethreads=undef useperlio=undef d_sfio=undef Compiler: cc='cc', optimize='-xO2', gccversion= cppflags='-I/usr/local/include' ccflags ='-I/usr/local/include' stdchar='char', d_stdstdio=define, usevfork=false intsize=4, longsize=4, ptrsize=4, doublesize=8 d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16 alignbytes=8, usemymalloc=n, prototype=define Linker and Libraries: ld='cc', ldflags =' -L/usr/local/lib' libpth=/usr/local/lib /lib /usr/lib /usr/ccs/lib libs=-lsocket -lnsl -ldl -lm -lc -lcrypt libc=/lib/libc.so, so=so, useshrplib=true, libperl=libperl.so Dynamic Linking: dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags=' -R /usr/local/lib/perl5/5.00503/sun4-solaris/CORE' cccdlflags='-KPIC', lddlflags='-G -L/usr/local/lib' Characteristics of this binary (from libperl): Built under solaris Compiled at Jan 12 2000 11:44:51 @INC: /usr/local/lib/perl5/5.00503/sun4-solaris /usr/local/lib/perl5/5.00503 /usr/local/lib/perl5/site_perl/5.005/sun4-solaris /usr/local/lib/perl5/site_perl/5.005 . Thanks, Alan Burlison
Re: Apache 1.3.9 + mod_perl 1.21 + Solaris 2.7 dumps core
"Frank D. Cringle" wrote: > Apache reloads itself during initial startup, to ensure that possible > configuration errors that might cause problems later (e.g. when > rotating logs) are caught as soon as possible. The combination of > this procedure with mod_perl and certain operating systems leads to > the symptoms that you are seeing. Typically, a SEGV is occurs when > accessing the boot routine of the first dynamically-loaded perl-XS > module. Close tracing will show that it happens the second time > through, during the trial reload, and that the symbol table used by > dlopen has been shot to hell. If you can find and fix this bug, you > will be awarded the mod_perl Nobel prize. I claim the prize, or at least a mention on the citation ;-) Your comment about dlopen set me thinking. There is a little-known feature on Solaris that allows you to debug the operation of ld.so.1. Here's a flavour: $ LD_DEBUG=basic ./httpd -X 06988: 06988: calling init: /usr/lib/libc.so.1 06988: 06988: 06988: calling init: /usr/lib/libmp.so.2 06988: 06988: 06988: calling init: /usr/lib/libnsl.so.1 06988: 06988: 06988: calling init: /usr/lib/libsocket.so.1 06988: 06988: 06988: transferring control: ./httpd 06988: 06988: 06988: cyclic objects for .init (before sorting) 06988: [2] /home1/software/perl/debug/lib/5.00503/sun4-solaris/CORE/libperl.so 06988: [1] /home2/web/apache_1.3.9/libexec/libperl.so 06988: 06988: cyclic objects for .init (after sorting) 06988: [2] /home1/software/perl/debug/lib/5.00503/sun4-solaris/CORE/libperl.so 06988: [1] /home2/web/apache_1.3.9/libexec/libperl.so 06988: 06988: calling init: /usr/lib/libgen.so.1 06988: 06988: 06988: calling init: /usr/lib/libcrypt_i.so.1 06988: 06988: 06988: calling init: /home1/software/perl/debug/lib/5.00503/sun4-solaris/CORE/libperl.so 06988: 06988: 06988: calling init: /home2/web/apache_1.3.9/libexec/libperl.so 06988: 06988: 06988: calling init: /home1/software/perl/debug/lib/5.00503/sun4-solaris/auto/Data/Dumper/Dumper.so 06988: 06988: 06988: calling init: /home1/software/perl/debug/lib/5.00503/sun4-solaris/auto/B/B.so 06988: Segmentation Fault(coredump) What that is saying is that the .init sections (pre-execution initialisation code) of the perl libperl.so and the libexec libperl.so have a cyclic dependency. I figured that sod's law meant that the order that ld.so.1 chose to pull the libraries in was the wrong order, so I tried the following to reverse the binding order: $ LD_PRELOAD='/home2/web/apache_1.3.9/libexec/libperl.so \ /home1/software/perl/debug/lib/5.00503/sun4-solaris/CORE/libperl.so' ./httpd $ hey presto! A working httpd. Obviously this is just a workaround - the real fix is to identify the cyclic dependencies and fix them properly. Alan Burlison
Re: Apache 1.3.9 + mod_perl 1.21 + Solaris 2.7 dumps core
"Frank D. Cringle" wrote: > Apache reloads itself during initial startup, to ensure that possible > configuration errors that might cause problems later (e.g. when > rotating logs) are caught as soon as possible. The combination of > this procedure with mod_perl and certain operating systems leads to > the symptoms that you are seeing. Typically, a SEGV is occurs when > accessing the boot routine of the first dynamically-loaded perl-XS > module. Close tracing will show that it happens the second time > through, during the trial reload, and that the symbol table used by > dlopen has been shot to hell. If you can find and fix this bug, you > will be awarded the mod_perl Nobel prize. Many thanks - that exactly matches my symptoms. I'll keep on digging, but don't hold your breath. Has anyone tried purify on this? Alan Burlison
Apache 1.3.9 + mod_perl 1.21 + Solaris 2.7 dumps core
$ httpd -X Segmentation Fault(coredump) I thought this was the old 'apache & modperl use a different malloc' problem, so I recompiled Perl with -Uusemymalloc, along with all the other installed modules. No joy - it still core dumps. Here's the stack trace: (/package/SC5.0/bin/../WS5.0/bin/sparcv9/dbx) where =>[1] boot_Data__Dumper(0x4, 0xfef582a0, 0xfef62f54, 0x285814, 0xff0e7734, 0xfeff52f4), at 0xfefe4e0c [2] Perl_pp_entersub(0xfef62f1c, 0xfef582a0, 0xfef67734, 0x2857a8, 0x7, 0x33f170), at 0xfeef4dd4 [3] Perl_runops_standard(0xfef582a0, 0xfef676f8, 0x265a20, 0xfef62f10, 0xe6, 0xfef676f8), at 0xfeeeccb0 [4] perl_eval_sv(0xfef676f8, 0x0, 0xfef67200, 0x2, 0x2bc12c, 0xff1bdbc4), at 0xfeea33ac [5] perl_require_module(0x93eb0, 0x0, 0xff17597c, 0x0, 0xffbefdcc, 0x10), at 0xff13d754 [6] perl_cmd_module(0x93d20, 0x93db8, 0x93eb0, 0xff17597c, 0xffbefb04, 0xa), at 0xff1377a4 [7] invoke_cmd(0xff177e64, 0xffbefb04, 0x93db8, 0xffbedaab, 0xffbefb04, 0xff177e64), at 0x32790 [8] ap_handle_command(0x6f810, 0xff177e64, 0xffbedaa0, 0x258390, 0x0, 0x93200), at 0x32b7c [9] ap_srm_command_loop(0x2000, 0x93200, 0x93200, 0xffbefb04, 0x2f687474, 0x2f616363), at 0x32c50 [10] ap_process_resource_config(0x93b10, 0x893c4, 0x92640, 0x258250, 0x92668, 0x92640), at 0x32fd8 [11] ap_read_config(0x92640, 0x258250, 0x893c4, 0x8dab8, 0x0, 0x92640), at 0x337cc [12] standalone_main(0x8dc00, 0x8d800, 0x8dabc, 0x260b8c, 0xb400, 0x8), at 0x3c69c [13] main(0x8d800, 0x8dc00, 0x2, 0x86800, 0x0, 0x0), at 0x3d05c i.e. in the XS startup code for Data::Dumper (which *is* built with system malloc). Any clues or cuggestions, anyone? Here's the rest of my config information: My config: Apache 1.3.9 mod_perl 1.21 Solaris 2.7 $ cat makepl_args.modperl APACHE_SRC=/home2/web/build/apache_1.3.9/src EVERYTHING=1 USE_APXS=1 WITH_APXS=/home2/web/apache_1.3.9/bin/apxs $ perl -V Summary of my perl5 (5.0 patchlevel 5 subversion 3) configuration: Platform: osname=solaris, osvers=2.7, archname=sun4-solaris uname='sunos fubar 5.7 generic_106541-08 sun4u sparc sunw,ultra-30 ' hint=recommended, useposix=true, d_sigaction=define usethreads=undef useperlio=undef d_sfio=undef Compiler: cc='cc', optimize='-xO2', gccversion= cppflags='-I/usr/local/include' ccflags ='-I/usr/local/include' stdchar='char', d_stdstdio=define, usevfork=false intsize=4, longsize=4, ptrsize=4, doublesize=8 d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16 alignbytes=8, usemymalloc=n, prototype=define Linker and Libraries: ld='cc', ldflags =' -L/usr/local/lib' libpth=/usr/local/lib /lib /usr/lib /usr/ccs/lib libs=-lsocket -lnsl -ldl -lm -lc -lcrypt libc=/lib/libc.so, so=so, useshrplib=true, libperl=libperl.so Dynamic Linking: dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags=' -R /usr/local/lib/perl5/5.00503/sun4-solaris/CORE' cccdlflags='-KPIC', lddlflags='-G -L/usr/local/lib' Characteristics of this binary (from libperl): Built under solaris Compiled at Jan 12 2000 11:44:51 @INC: /usr/local/lib/perl5/5.00503/sun4-solaris /usr/local/lib/perl5/5.00503 /usr/local/lib/perl5/site_perl/5.005/sun4-solaris /usr/local/lib/perl5/site_perl/5.005 . Thanks, Alan Burlison
