Re: Case of the Vanishing Symbol Tables
Are you using PerlFreshRestart? Same behavior, on or off. That's strange. With PerlFreshRestart on, it is supposed to clear things out when restarting, which seems consistent with what you're seeing. Are you sure there is no difference in that trace at all when you turn it on or off? You should be able to replace a PerlModule call with Perluse MyModule;/Perl in the same place in httpd.conf. Does that work for you? The Eagle book says to do that with earlier versions. This doesn't work either. They simply refuse to be loaded anywhere other than the startup.pl. What do you mean by refuse to be loaded? Here's the relevant quote from the Eagle (by way of http://www.modperl.com/): The only trick to this is that you must be careful that the PerlModule directive is called before any PerlPassThru directives appear. Otherwise Apache won't recognize the new directive and will abort with a configuration file syntax error. The other caveat is that PerlModule only works to bootstrap configuration directives in mod_perl versions 1.17 and higher. If you are using an earlier version, use this configuration section instead: Perl use Apache::PassThru (); /Perl
Re: Apache::SizeLimit Exit is Delayed
what about $r-headers_out-add(Connection = 'close'); Good idea! I'll put that into a future release. - Perrin
Re: Comparison of different caching schemes
IPC::ShareLite freezes/thaws the whole data structure, rather than just the hash element being accessed, IIRC, so is probably going to have extremely poor scaling characteristics. Worth adding to check, of course. No, it's probably not worth it. It would be worth adding IPC::Shareable though, because people never believe me when I tell them to use something else. Having some numbers would help. Another interesting option is mapping a MySQL table data structure directly to the data structure being stored. That could be useful as part of a comparison for storing non-complex data, i.e. a single scalar value. I'm not sure what a 'standard table' in MySQL is any more... Berkeley, MyISAM, ISAM... I guess we can try all these, but that's benchmarking the DB rather than the caching scheme, and we're not about to try every DB server we can find! No, of course not. It may be that the performance characteristics of these table types are well known already and I just don't follow the MySQL scene well enough to know. I thought maybe the default tables type (MyISAM?) which doesn't support transactions would have better speed for dirt simple storage like this. - Perrin
Re: Any good WebMail programs?
I can use a primer on researching WebMail programs http://users.ox.ac.uk/~mbeattie/wing/
Re: Defeating mod_perl Persistence
When using Mail::Sender only the first email is sent on my mod_perl server. When I investigated, I realised that the socket to the SMTP server was staying open after the completion of that first email (presumably mod_perl is responsible for this persistence). Is there any way to defeat the persistence on the socket while running my script under mod_perl, or do such scripts always need to be mod_cgi? You're probably storing it in a global so it never gets cleaned up. Don't do that. - Perrin
Re: Apache::SizeLimit Exit is Delayed
I'm using Apache::SizeLimit (on both Solaris and Linux) and getting the error_log entry: [Tue Dec 11 15:01:19 2001] (2520) Apache::SizeLimit httpd process too big, exiting at SIZE=44964 KB SHARE=10900 KB REQUESTS=389 LIFETIME=9505 seconds But the child process often doesn't exit right away. It may hang around for a couple of minutes or even an hour with the message occuring with every hit: Maybe this is because of a keep-alive connection. It's possible that Apache::exit() waits for all keep-alive activity to complete before exiting. Is there any way you can test it without keep-alive? - Perrin
Re: Case of the Vanishing Symbol Tables
All is well until you reach the end, where it starts reloading everything, and the namespace is now completely gone. Are you using PerlFreshRestart? I posted this once before and got blown off -- a pox on to those who tell me to check @INC or what not. No need to get testy. If this is causing a problem for your business, and free volunteer help is not working quickly enough for you, you can always contract Covalent to solve the problem. And don't tell me to just use `use` either -- they all load up Apache config directives, so PerlModule has to be called at some point so Apache recognizes the directives. You should be able to replace a PerlModule call with Perluse MyModule;/Perl in the same place in httpd.conf. Does that work for you? The Eagle book says to do that with earlier versions. - Perrin
Re: Comparison of different caching schemes
I sat down the other day and wrote a test script to try out various caching implementations. Very interesting. Looks like Cache::Mmap deserves more attention (and maybe a Cache::Cache subclass). Have I missed something obvious? Nothing much, but I'd like to see how these numbers vary with the size of the data being written. And it would be good if there were a way to ensure that the data had not been corrupted at the end of a run. Also, I'd like to see MLDBM + BerkeleyDB (not DB_File) with BerkeleyDB doing automatic locking, and IPC::MM, and IPC::Shareable, and IPC::ShareLite (though it doesn't serialize complex data by itself), and MySQL with standard tables. Of course I could just do them myself, since you were kind enough to provide code. - Perrin
Re: Re(2): Apache::SizeLimit Exit is Delayed
That was it. The child exited immediately when I hit the limit with KeepAlive Off. Now the question is: Is there a way to force an exit even with KeepAlive On? As Jay already pointed out, you usually don't want KeepAlive on with mod_perl. However, you could try changing the call $r-child_terminate() to Apache::exit(). If this seems to work better for you, let me know and I'll consider changing this in a future release of Apache::SizeLimit. - Perrin
Re: Re(2): Apache::SizeLimit Exit is Delayed
That was it. The child exited immediately when I hit the limit with KeepAlive Off. Now the question is: Is there a way to force an exit even with KeepAlive On? As Jay already pointed out, you usually don't want KeepAlive on with mod_perl. However, you could try changing the call $r-child_terminate() to Apache::exit(). If this seems to work better for you, let me know and I'll consider changing this in a future release of Apache::SizeLimit. - Perrin
Re: Comparison of different caching schemes
One important aspect missing from my tests is the actual concurrency testing. Oh, I guess I should have checked your code. I thought these were concurrent. That makes a huge difference. 2. Lock some part of cache for a request (Cache::Mmap buckets, MLDBM pages?) MLDBM::Sync locks the whole thing. - Perrin
Re: submit-data and chained handlers
e.g. $r-read($in,$r-header_in('Content-length'));
or $in=$r-content();
give's my handler the data, but unfotunately exclusive - so the data don't
reaches Apache::Registry and the cgi-script.
Any suggests?
Apache::RequestNotes.
- Perrin
Re: HTML forms and piplining templating systems
I'm curious about one thing: How easy is it to do fill-out forms with pipelining templating systems like AxKit? I'd think that in such cases the stylesheet would have to do so much work that it would become, in essence, like a CGI script. Can you explain what work you're talking about here? In general, form processing works great with a pipeline style. If you're displaying an empty form, there is nothing to do before running the template. If you're displaying a form with data filled in, you collect the data in the processing part of your code and then pass it to the template. What is it that's giving you trouble? As for which system to use, AxKit certainly has the broadest support for XML. If you're not enjoying working with XML you could try a system like Template Toolkit or HTML::Template that allow you to pass your data as normal perl data structures. Of course you can do that with Embperl too if you can be disciplined about keeping separate components for application logic vs. display. - Perrin
Re: Perl-status shows 1 Mb for File::Find!
I looked at Memory Usage at my perl-status page and was horrified: the biggest modules are: File::Find 1205208 bytes | 2597 OPs Convert::ASN1::parser1058185 bytes | 3069 OPs However, if I go inside File::Find, I can't figure out what takes up so much space: It could be internal structures holding the state. Are you pre-loading this module? It probably doesn't really use 1MB in each process. - Perrin
Re: Perl-status shows 1 Mb for File::Find!
I do preload the module. How do I find out what it really uses? In addition to Paul's advice, make sure you understand the output of your system's top command (or whatever you use to see process size). Usually there are multiple numbers, and some indicate real memory used while others indicate real + shared and virtual. There's info on this in the guide, and you can use modules like GTop or the size subs from Apache::SizeLimit to help figure it out. - Perrin
Re: User customisable website application?
I think the OpenInteract sample apps might do what you want. By the way, please put the text of your message in the body of the mail rather than as an attachment. - Perrin
Re: perl with java
I'm a java programmer. I don't know anything about perl. I just want to know whether it's possible to call servlets from perl scripts after validating some data provided by the user. There's no simple way to use both in the same request. Maybe a subrequest would work, or maybe you could use mod_perl for an earlier handler and then use Tomcat for the content handler. There are also ways to directly call Perl from Java and vice-versa. It's ill-advised though. Hybrid solutions always have overhead involved in going from one to the other and they tend to combine the worst of both worlds and complicate maintenance and development. I'd advise against trying to mix the two in a single request. If you really have to use both, try to find a way to get the validation work done in the perl program and then redirect to the servlet. - Perrin
Re: Oddity with redirects
Accessing the protected site blows up because the Apache object referent is undef (see below for examples). No doubt I've screwed up some part of the httpd.conf but cannot figure out what at this point... Without knowing anything much about the AuthCookie module, I would guess your problem has to do with confusing method handlers and regular handlers. if you prototype your handler with ($$) it expects to be called as a method handler and receive a class name in addition to the request object. See the documentation on method handlers in the mod_perl POD for more, or check the Eagle book. - Perrin
Re: Apache::Session using Frames
Just to let anyone who was wondering (and for the benefit of the archives), I ended up ditching sessions all together. Instead, I'm using Apache::AuthDBI to do authentication, and am making calls directly to my database server to maintain state. Its not the most pleasant way of maintaining state, but at least it works. Then this won't be much help to you anymore, but here are a couple of things that could have been happening: - Not getting the cookies sent from all frames. You may have a race condition involving when the cookie gets storeed and when other frames send their requests. - Not using the locking API correctly. You didn't post any code, but the locking options for Apache::Session sometimes trip people up and you may have had a misconfiguration of some kind. Anyway, there's nothing wrong with writing directly to the database. Apache::Session is just a convenient hash interface on top of that. - Perrin
Re: [RFC] Apache::CacheContent - Caching PerlFixupHandler
I would like to propose a new Apache module before I send it off to CPAN. The name chosen is Apache::CacheContent. This is very cool. I was planning to write one of these, and now I don't have to. Your implementation is short and interesting. I was planning to do it with a PerlFixupHandler and an Apache::Filter module to capture the output. While that approach wouldn't require the use of method handlers, I think yours may be easier for newbies because it doesn't require them to understand as many modules. The only real advantage of using Apache::Filter is that it would work well with existing Registry scripts. A couple of other C's for your R: A cache defines parameters that constitute a unique request. Your cache currently only handles the filename from the request as a parameter. It would be nice to also handle query args, POST data, and arbitrary headers like cookies or language choices. You could even support an optional request_keys method for handlers which would let people generate their own unique key based on their analysis of the request. Doing this would mean you would need to generate filenames based on the unique keys (probably by hashing, as in Cache::FileCache) and do an internal redirect to that file if available when someone sends a request that matches. Another thing that might be nice would be to store the TTL with the file rather than making the handler give it to you again each time. This is done in mod_proxy by putting an Expires header in the file and reading it before sending the file, but you could also store them in a dbm or something. Support for sending Expires headers automatically would also be useful. When I first thought about this problem, I wanted to do it the way Vignette StoryServer does: by having people link to the cached files directly and making the content generating code be the 404 handler for those files. That gives the best possible performance for cached files, since no PerlFixupHandler needs to run. The downside is that then you need an external process to go through and clean up expired files. It's also hard to handle complex cache criteria like query args. StoryServer does it by having really crazy generated file names and processing all the links to files on the way out so that they use the cached file names. Pretty ugly. I know you guys are pushing to get the book done, so don't feel pressured to address this stuff now. I think the current module looks more than good enough for an initial CPAN release. - Perrin
Re: how to make mod_perl actually run fast
It uses wildcard DNS and some other goodies to extend the mod_perl-server-via-proxy-from-static-server setup Why not just use name-based virtual hosts for the backend and avoid all the monkeying with DNS? - Perrin
[OT] Re: Vhosts + mod_perl
Now, how did I know I was going to get flamed?? :/ Anyway thanks for the example. Some manuals are too hard to read, you have to be a goddamn PhD to read some of them out there to untangle the mess of cross references and incomplete examples. :/ Some are, but this one isn't. This is also off-topic since these aren't mod_perl questions. But, with that e.g. that uses an IP address, from day to day I don't know what my IP address will be, can't I use: NameVirtualHost fred.trains.ath.cx VirtualHost fred ... foo /VirtualHost http://httpd.apache.org/docs/vhosts/name-based.html There are also tutorials on this subject linked from here: http://httpd.apache.org/docs/misc/tutorials.html - Perrin
Re: location directive
On Tue, 27 Nov 2001, [EMAIL PROTECTED] wrote: I have put mod_perl handler inside a virtual host section then inside the virtualhost section, i also put location directives to override overall modperl handler in some situations, with sethandler default-handler. for instance Alias /icons/ d:/Apache/icons/ location /icons/ SetHandler default-handler /location this works ok as soon as the uri is /icons/file but modperl handler intercept if the uri is just /icons/ which should otherwise show the directory index of the location. isn't it a sort of bug ? No. See if this post solves your problem: [EMAIL PROTECTED]">http://mathforum.org/epigone/modperl/relskoxpee/[EMAIL PROTECTED] - Perrin
RE: Apache::Session Problem -- Addendum
On Thu, 22 Nov 2001, Jonathan M. Hollin wrote:
My code now includes:
35: # Session handler...
36: my %session; undef my $session_id;
37: use Apache::Session::MySQL;
38: tie %session, 'Apache::Session::MySQL', $session_id,
39: { DataSource = 'dbi:mysql:sessions', UserName = 'db_user', Password
= 'secret' };
40: $session_id = session_id=$session{_session_id};;
I am using Apache-Session v1.54.
The above code generates the following error:
[Thu Nov 22 10:31:38 2001] [error] PerlRun: `Can't connect(
HASH(0xbdcf54)), no database driver specified and DBI_DSN env var not set at
E:/Apache/site/lib/Apache/Session/Lock/MySQL.pm line 36 (in cleanup)
Can't connect( HASH(0x47e2f14)), no database driver specified and DBI_DSN
env var not set at E:/Apache/site/lib/Apache/Session/Lock/MySQL.pm line 36'
You aren't following the documentation. perldoc Apache::Session::MySQL.
It says to pass in connection info for a lock connection.
- Perrin
Re: [OT] search.cpan.org
There's a simple solution: http://kobesearch.cpan.org/
Re: array's first element is empty
Not sure if this is mod_perl related, but i hope someone can help me anyway. When i do DBI queries from mod_perl handler and put all returned results in array then array's first element is empty, I wonder why? I don't like to shift off first element every time i return the result. Has anyone seen this before and what could cause this? Unless you see different behavior when running under CGI, this is not a mod_perl question. I suggest you ask it on http://perlmonks.org/ and post a code sample with your question. - Perrin
Re: Class data preloading modules
* initialize everything in the parent Apache, just make sure to create new DBI connections in the children. That's what I would do. It increases the amount of shared memory. Disconnect the DBI connection when you're done with it. Apache::DBI doesn't cache connections during startup, so this shouldn't be a problem. - Perrin
Re: PerlModule not updating %INC
I wonder if this has something to do with the multiple init thing
http://marc.theaimsgroup.com/?l=apache-modperlm=100510779912574w=2
It does. It's exactly the same bug.
if the interpreter is really being entirely broken down on each restart
(including the initial one that Apache does when it reparses httpd.conf)
%INC might be getting mucked up...
It shouldn't. I thought the interpreter didn't actually get broken down on
restart at all unless you have PerlFreshRestart on. However, I see that
Doug said this:
i think that using *Apache::ReadConfig elsewhere will still trigger the
double loading of all Perl{Require,Module}s, but i doubt the people
suffering from the double load problem are using that feature. so i'll
deal with that later.
David, are you using Apache::ReadConfig or anything that uses it?
- Perrin
Re: PerlModule not updating %INC
IIRC, I ran a test with only httpd.conf.default with only these additions PerlModule My::Foo then package My::Foo; warn initializing...; in lib/perl and I got 'initializing' on each restart. no Apache::ReadConfig going on here. And no PerlFreshRestart?
Re: Apache::Session Problem -- Addendum
I changed Apache::Session::DBI to Apache::Session::MySQL and tried
again.
What's the version number of your Apache::Session? It should be 1.54.
42: tie %session, 'Apache::Session::DBI',
43: {DataSource = dbi:$db_driver:sessions:$db_address};
With Apache::Session::MySQL, the docs say you should be passing the user
name and password.
- Perrin
Re: $r-set_handlers and $R-push_handlers
How can you specify a Location for these paramters, or can they be used only from within the current section? $r is a request object, so modifications only affect the current request. If you want to affect all requests, you need to do it in the configuration stage. - Perrin
Re: Fastcgi on win [Was: Re: Documentation patch for mod_perl?]
So mod_perl has a slight speed edge over fastcgi (which is overthrottled a little with four servers). Really? Maybe this is because multi-process handling isn't as fast on NT. Does it change much if you vary the number of servers? My goal is to give some kind of useful suggestion to people who need better performance on NT than mod_perl can currently give. - Perrin
Re: Linux Hello World Benchmarks - 11/19/2001
on 11/19/01 8:05 PM, Joshua Chamas at [EMAIL PROTECTED] wrote: It has been a while, but here's a new set of Hello World benchmarks! There was a recent announcement of HTML::Template::JIT, and Template Toolkit has an XS option now. Any chance you could put those into the next round? - Perrin
Re: Cookie authentication
I seem to recall at least one major Apache module having an option to use URL-based authentication instead of cookie-based... but I can't seem to find that from a cursory perusal of CPAN. Apache::ASP does this. - Perrin
Re: [Maybe OT] Modular design - calling pages like a subroutine with a twist.
In my opinion, trying to abstract that stuff away in a web application causes to more problems than it solves, especially where back buttons and bookmarks are concerned. We haven't found this to be the case. Our servers are sessionless, so bookmarks work fine. These are different (though related) concepts. The original poster was looking for a way to structure web programs without thinking about the breaks caused by the request model of HTTP, and that's what I was commenting on. You're talking about a way to preserve data across multiple page requests. I think it's easier to take a state machine approach, the way CGI::MxScreen or Apache::PageKit do. I don't think this works. The state machine can manage states going forward, but not backward. You can code a state machine that defines legal transitions from one state to another, and that could include stepping backward. There's no real concept of forward and backward in what I had in mind, just a collection of states and legal transitions between them. If you hit the back button, you're still okay as long as the form's data is stored in the URL or hidden fields rather than in a global session, i.e. going back will return you to the state you were in correctly. Most people instinctively code a state machine when they start using CGI, but they do it in the form of a bunch of statements like if $form_action eq 'save'. The frameworks I mentioned just pull it out and make it more explicit. If I understand your FormContext approach correctly, you are storing the state of the current application in URLs or hidden fields. This is what we used at eToys as well, and I think it's a pretty common solution. It's the only way to safely handle possibilities like multiple browser windows using the same application. There are some CPAN modules that help with this kind of thing, like CGI::EncryptForm. - Perrin
Re: Cookie authentication
Here we insert a session id on all requests, with Apache::Session whether the request is for a static or dynamic page and have a TransHandler to strip the session id and insert it into %ENV which seems to work for us. With this approach we don't necessarily need cookies, but verifying if a user is who the session was originally assigned to without a cookie is really impossible. At least to me with the limited amount of brain time I have put into it. Using some algorithm consisting of the end-users IP and some random number is OK until users behind the same NAT device try to steal each others session. Using the cookie is a way to verify that a user is the owner of the session id. Cookies are very easy to fake and modify on the client side. If you want to verify that a user is returning a session ID you sent him without modifying it you should use a MAC, like the ticket access stuff in the Eagle book. - Perrin
Re: Cookie authentication
Excuse my question if it seems dumb I'm not 100% on NAT and proxies, but the Eagle book says to 1 Choose a secret, 2 Select fields to be user for the MAC. It also suggests to use the remote IP address as one of those fields. 3 Compute the MAC via a MD5 hash and store in the clients browser. 4 On subsequent visits recompute the MAC and verify it matches the original stored MAC. How is this reliable in a situation where many similarly configured computers are behind a NAT/Proxy and one of the users try to steal someone else's session by getting their cookie/session_id info? Don't use the IP address in the cookie, just generate a unique ID of your own. I suggest using mod_unique_id. - Perrin
Re: [Maybe OT] Modular design - calling pages like a subroutine with a twist.
I'll try to show how Apache::ASP could help here. In Apache::ASP, scripts can be executed as subroutines, even with return values, and I think this goes to the heart of what you need here. The original e-mail was confusing, but I think what he's after is not so much the ability to call pages as subs but rather the ability to abstract away the fact that a sub might actually involve multiple user interactions (present a form, get a response, present another form, etc.) with breaks in actual execution. In other words, he wants to think of program execution in terms of a linear user session (as you would with a GUI app) rather than a series of separate requests. In my opinion, trying to abstract that stuff away in a web application causes to more problems than it solves, especially where back buttons and bookmarks are concerned. I think it's easier to take a state machine approach, the way CGI::MxScreen or Apache::PageKit do. (CGI::Application sort of does, but it doesn't capture the relationships of states to each other.) With Apache::ASP, I think people would generally embed the state transition logic in the pages, although this could probably be separated out if you were careful about it. - Perrin
Re: cgi-object not cacheable
If it was running under CGI, it would be compiling CGI.pm on each
request,
which I've seen take .3 seconds. Taking that long just to create the
new
CGI instance seems unusual. How did you time it? Are you using
Apache::DProf?
Wouldnt it be compiled at the use-statement ?
Yes, but when running under CGI (the protocol, not the module) that use
statement is executed every time.
I timed it using
module-internal loggingfunction which use time::hires.
That's usually pretty accurate, so I guess it really takes that long on your
system. Try Apache::Request! Or even one of the lighter CGI modules like
CGI_Lite.
in my case it means up to 4 connections per process, cause in fact it
is not one module but 2 (input and output) and each needs to handle 2
different
connections.
If you could reduce that, it would certainly help your application's
performance.
I hope to share databasehandles via IPC. One has to avoid that only
one process writes to a handle at the same time !!
IPC::Shareable, and most of the other options, use Storable to serialize
data structures. Storable can't serialize an open socket. You *CAN* share
sockets, but you'd have to write some custom C code to do it. You might
look at the Sybase thing that was posted here recently. (I haven't looked
at it yet, but it sounded interesting.)
if max. number is
reached - return 0. The calling application can then decide to print
an excuse due to the user 'cause we are so popular we cant server you
:)' or create and destroy a temporary handle to process the request.
Even with temporary handles, you have the possibility of all servers being
busy at once and thus using all 4 handles.
This would be something I would actually prefer to Apache::DBI, but I
dont know if its possible, but I'll try. Such a thing would be very
important, especially on slow servers with less ram, where Apache::DBI
opens to many connections in peak-times and leaves the system in a bad
condition ('to many open filehandles')
I still think you'd be better off just limiting the total number of servers
with MaxClients. Put a reverse proxy in front and you'll offload all the
work that doesn't require a database handle.
ps: just if one is interested: today I was very happy to wear a helmet
when I crashed with my bike ;)
I guess my mother was right about that. Keep your helmet on!
Glad you're not dead,
Perrin
Re: Doing Authorization using mod_perl from a programmers perspective
1. Can this be done (nicely) as a authentication/authorization handlier? Sure, or you could do it as part of another phase if it's easier for you. There are good exmples on CPAN or in the Eagle book. 2. Do most hosting companies allow authentication/authorization handlers? (Using HostRocket at the moment). Most hosting companies don't allow mod_perl. 3. What is the most appropriate session management system? I'm thinking of using cookies (client side) to store a session key, rather than resubmitting the password data. The server side stores this session key in the database. That sounds fine. There are examples of this in the book too. Make sure your session keys can't be forged. 4. How does the membership ID get passed to the next stage? It's typically stored in the session data. 5. What is the time to do additional access checking (for senior/admin users)? Impossible to answer. Totally depends on your setup. 6. What is a realistic time to expect all this to happen in? Also impossible to answer, since it depends on your developers and the full requirements. - Perrin
Re: Doing Authorization using mod_perl from a programmers perspective
Seemingly I can do Apache handlers though, so I *might* be okay. If you look at http://perl.apache.org/guide/, there's information on how to determine if you're really running mod_perl or not. If you can get a PerlHandler directive to work, you have mod_perl. I rather ambigously asked the question: How many milliseconds might this stage take, on a commerial hosting company's servers. Suppose without mod_perl this would be at least 300ms, or probably worse. It depends on what you're doing in that stage. In relative terms, programs that access databases usually run many times faster in mod_perl than with CGI. - Perrin
Re: [challenge] new mod_perl site
What about an additional requirement: The site should be validated sucessfully by validator.w3.org Maybe we should give the horse a chance to catch up to the cart... Once there is a design that people are satisfied with, you are more than welcome to submit patches to fix validation problems. - Perrin
Re: cgi-object not cacheable
One run of my script takes about 2 seconds. This includes a lot of database-queries, calculations and so on. about 0.3 seconds are used just for one command: $query=new CGI; That's really awfully slow. Are you positive it's running under mod_perl? Have you considered using Apache::Request instead of CGI.pm? This is not a problem of my persistent variables, cause this works with many other objects like db-handles (cant use Apache::DBI cause this keeps to many handles opened, so I need to cache and pool on my own), filehandles etc. Whoa, you can't use Apache::DBI but you can cache database handles yourself? That doesn't make any sense. What are you doing in your caching that's different from what Apache::DBI does? - Perrin
Re: [challenge] new mod_perl site
What about an additional requirement: The site should be validated sucessfully by validator.w3.org Maybe we should give the horse a chance to catch up to the cart... Once there is a design that people are satisfied with, you are more than welcome to submit patches to fix validation problems. - Perrin
Re: [OT] Data store options
Basically, I'm trying to understand when to use Cache::Cache, vs. Berkeley DB, and locking issues. (Perrin, I've been curious why at etoys you used Berkeley DB over other caching options, such as Cache::Cache). Cache::Cache didn't exist at the time. BerkeleyDB seemed easier than rolling our own file cache, but in retrospect I'm not sure it was the right way to go. We spent a fair amount of time figuring out how to work with BerkeleyDB effectively. 1) use Storable and write the files out myself. 2) use Cache::FileCache and have the work done (but can I traverse?) 3) use Berkeley DB (I understand the issues discussed in The Guide) If you do use BerkeleyDB, I suggest you just use the simple database-level lock. Otherwise, you have to think about deadlocks and I found the deadlock daemon that comes with it kind of difficult to use. So, what kind of questions and answers would help be weigh the options? If you have too many records, I suppose Cache::FileCache might eat up all your inodes. You can probably go pretty far with a modern OS before you hit a problem with this. I always wanted to benchmark Cache::FileCache against BerkeleyDB for different read/write loads and see how they do. I think FileCache would win on Linux. With regard to locking, IIRC, Cache::Cache doesn't lock, rather writes go to a temp file, then there's an atomic rename. Last in wins. If updates to a record are not based on previous content (such as a counter file) is there any reason this is not a perfectly good method -- as opposed to flock? That should be fine. - Perrin
Re: Prototype Mismatch
In my script a simple use Image::Magick; results in the following lines being added to my error.log everytime the script is called: Are you using PerlRun or Registry? Prototype mismatch: sub Apache::ROOT::shapeshifter::system::pm_files_2ecgi::Transparent vs ($;@) at E:/Apache/lib/Exporter.pm line 57. I'm not sure why this is complaining, but one thing you can do is not import this function. You can do a 'use Image::Magick ()' and call it with a full package name instead. - Perrin
Re: Documentation patch for mod_perl?
Hi, I am sending this after a suggestion that was given to me during in a perlmonks discussion. I was the one who suggested it. Why don't you take a shot at writing it? The modperl.pod seems like the logical place to patch. Then Stas could add your information to the guide as well. If you're not sure you've got all the facts right, post it to the list and people can check it. - Perrin
Re: Perl Script unable to stat
Hi, ScriptAlias is a CGI thing, not a mod_perl thing. If you're trying to run thing with Apache::Registry, you should use Alias instead. If you're trying to use CGI and need help with that, try posting your question on http://perlmonks.org/. - Perrin
Re: Are global variables truly global?
Will the different child processes created by Apache share the same variable? Or will each child create an additional instance? Each child has a separate interpreter and thus a separate instance of your global. If you need to co-ordinate data between processes, you might want to look at Cache::Cache or MLDBM::Sync. If you just need an exclusive lock, file locking is the simplest way to go. - Perrin
Re: mod_perl alias directories
1- Currently I have designated a /pm directory to contain my mod_perl codes, a collection of classes. I then have instances of these launched from non /pm directory. I did this so that I don't have to be back at cgi-bin perdicament where all my CGIs are in one directory. I never liked that style. But I'm still forced to put my base classes in /pm. Is there a way to instruct HTTP that perl code in any directory should be used as mod_perl similar to ExecCGI. Or perhaps for a better/cleaner approach, any perl code with some extension such as foo.cgi-mp or foo.cmp (Cgi-Mod-Perl). Look at the FilesMatch directive in the Apache documentation. You might also want Apache::Dispatch. 2- I currently have to restart my HTTP for changes to my mod_perl code take affect. Is there any way so I don't have to do this. Apache::StatINC or Apache::Reload. - Perrin
Re: Should i lock DB_File for read-only access?
If i run this code from command line or under mod_cgi, it work fine. But if it run repeatedly under mod_perl, it occasionally give me a database access timeout message (see code above), but error value $! still empty. What return value are you getting from tie? Incidentally, since this is a read-only situation, you can open the dbm once and keep the handle in a global. Then you save time by not needing to open it on each request. - Perrin
Re: Neo-Classical Transaction Processing (was Re: Excellent article...)
The mod_perl servers are the work horses, just like the custom servers. In a classical OLTP system, the customer servers are stateless, that is, if a server goes down, the TM/mod_proxy server routes around it. (The TM rollsback any transactions and restarts the entire request, which is interesting but irrelevant for this discussion.) If the work servers are fully loaded, you simply add more hardware. If all the servers are stateless, the system scales linearly, i.e. the number of servers is directly proportional to the number of users that can be served. The mod_perl servers in the eToys setup were essentially stateless. They had some local caching of user-specific data which made it desirable to go back to the same one if possible for the same user, but it would still work if the user switched to another server, since all state was stored in the central database. The trouble here should be obvious: sooner or later it becomes hard to scale the database. You can cache the read-only data, but the read/write data isn't so simple. Theoretically, the big players like Oracle and DB2 offer clustering solutions to deal with this, but they don't seem to get used very often. Other sites find ways to divide their traffic up (users 1 - n go to this database, n - m go to that one, etc.) However, you can usually scale up enough just by getting a bigger box to run your database on until you reach the reach the realm of Yahoo and Amazon, so this doesn't become an issue for most sites. If the whole multi-threaded server ever has to wait on a single shared resource (e.g. the garbage collector), all simultaneous requests in progress lose. This doesn't happen if the work servers are single threaded, i.e. shared nothing. You can easily model the resources required to process a request and no request has to wait on another, except to get CPU. But how can you actually make a shared nothing system for a commerce web site? They may not be sharing local memory, but you'll need read/write access to the same data, which means shared locking and waiting somewhere along the line. The front-end servers are different. They do no work. Their job is switching requests/responses. Some of the requests are going directly to the OS (e.g. get an icon) and others are going to the work servers. The front-end servers wait most of the time. Here I buy into the the asynchronous I/O model of serving multiple requests and not the lightweight process model. I do feel that servers built on non-blocking I/O would have been much more efficient on the eToys front end. I've seen how well thttpd can do. But since these boxes already scaled well beyond our needs with basic Apache/mod_proxy, we didn't bother to look for something more complicated. If I ran into problems with this now, I would probably look at Red Hat's Tux server. - Perrin
Re: namespace-troubles
Sorry if I came off sounding harsh. I spend a lot of my personal time answering newbie questions on this list, and it bothers me when people say that newbies don't get helped here. They generally get a lot of help here. If someone says it's in the guide and you can't find it, ask for a link to the relevant section. If you read that section and still don't get it, ask more questions. Sending people links to the proper docs in response to questions when possible is a simple matter of efficiency. - Perrin
Re: problem with Apache::SSIChain
LPorcano wrote: I am trying to set up a Apache::SSIChain to process the output from a cgi running under Apache::Registry I am using the following line in the conf file: PerlHandler Apache::OutputChain Apache::SSI Apache::Registry Shouldn't that be PerlHandler Apache::OutputChain Apache::SSIChain Apache::Registry instead?
Re: [OT] excellent modperl/etoys article by Perrin revisited
Perrin or others involved in the old eToys (or anyone in the new Toys) -- does anyone know if this is the same mod_perl technology you guys wrote? I'm not involved in that anymore, so I don't have first-hand information. However, a brief look at the site makes me think they are not using any of what we wrote. The URL structures and query args (i.e. the public API of the site) are all very different. I think it will make an interesting success story follow up if it is successful because it would also show how easy it was for the intellectual property written in mod_perl to be resold and reintegrated into another IT infrastructure which would make VCs happy (ie they would think more about being able to fund projects based on mod_perl if they know they could always resell the IP). You have to remember that what we wrote was a specific application, not a generic store or application server. That code would only be useful if you wanted a store that did everything (data model, navigation, order processing) exactly the way we did. I very much doubt KB is that similar. Also, the documentation was great for an in-house project but not enough for a packaged commercial app. There was a basic shared knowledge assumed in most of it about the business goals and environment. Keep in mind that we wrote very little infrastructure code - no more than I would expect a J2EE project this large to write. The infrastructure came from CPAN, Apache, etc. and is known to be very portable. I suspect that KB has a team who supports their site and they chose to use the technologies they already have in place. People hate change, and looking at a foreign codebase, possibly in a language you don't know, is daunting. It would be the same thing if someone walked into an ASP shop with a bundle of WebLogic code and said make it run, and work with our legacy systems, with these changes, by Christmas. And don't stop supporting our regular site. Oh, and we didn't hire any of the developers who wrote it or the business people who told them what to write, so you'll just have to read the source if you have questions that aren't in the documentation. Doesn't sound that appealing, does it? - Perrin
Re: When to use 'use' for accessing modules?
Steve Piner wrote:
Perrin Harkins wrote:
Chris Allen wrote:
[...]
Is $ENV{foo}='bar'; in startup.pl equivalent to PerlSetEnv foo bar
in httpd.conf?
Yes.
Are you sure? I experimented a few months ago, and found that
$ENV{foo}='bar'; would only last in each child until the first request
of the child completed.
You may be right. Depends on the setting of PerlSetupEnv, I think.
- Perrin
Re: [OT] P2EE Redux was: Excellent article on Apache/mod_perl at eToys
[EMAIL PROTECTED] wrote: I'm surprised that no one has, jokingly, suggested PEE. Sorry, couldn't resist. :) Neither could these people: http://pee.sourceforge.net/
Re: Excellent article on Apache/mod_perl at eToys
Matthew Kennedy wrote Is there a Perl equivalent to Struts? Mason seems to come close if you keep yourself disciplined somewhat. I mentioned a couple of tools in the article that are specifically aimed at MVC: OpenInteract, and Apache::PageKit. Actually, I think it's pretty easy to replace Struts. Think about what it offers. First, there's a set of taglibs for writing templates without embedding Java code. Template Toolkit blows it away in this area. (I once suggested to one of the authors of Struts that tags like logic:greaterThan parameter=number value=7 are a bit verbose. He told me to buy DreamWeaver.) It doesn't add much beyond advice in the model area. The controller is basically just a dispatch table which can easily be replaced with Apache::Dispatch or even a well-written httpd.conf. To be fair, it does some things to try and make forms easier to deal with. You'd need to get OpenInteract or PageKit to replace that stuff, or write some glue code for the form validators and such on CPAN. Of course you could use Mason, but most Mason users aren't aiming for a strict MVC separation and it's very easy to cheat when your templates are written in in-line perl. - Perrin
Re: Newbie having problems
John Michael wrote:
I am attempting first to ge them to run correctly under.
Apache::PerlRum
Be careful of PerlRum! It will give you a wicked hangover!
PerlRun is a quick way to port dirty scripts that you can't take the
time to fix. It's the slowest way to run anything in mod_perl, but if
your script uses lots of globals, imports functions all over the place,
etc. you may have no choice.
I had a routine in the beginning of the program to initialze my variable hash and
query param like so:
## initialize globals ##
foreach (keys %VAR){$VAR{$_} = ;}
I did this so that these variables would not propogate to other instances of the
script.
This was causing the error.
I removed it and added a this small sub:
sub clean_up {
undef %VAR;
Apache::exit();
}
Is this the correct way to do this or even necessary?
It should not be necessary. Unless you have delcared these to be in a
separate package, PerlRun will clean them up after every request. Are
you declaring a package in your script?
The Apache::exit() function is still causing an error, but the script runs and works
but is evidently not exiting gracefully.
Why are you needing to exit? Maybe this is something that you can
avoid, and improve performance.
Also, I am running the script in an eval{ } block in order to trap errors in the $@
variable.
IS this ok practice in mod-perl?
The entire script is in an eval block? You can do that, but if all you
want is a decent page when something fails, just put up your nice page
and set it up as the SERVER ERROR page in httpd.conf.
The scripts run, but sometimes through server errors.
I am getting the following errors.
[Tue Oct 23 06:57:27 2001] [error] PerlRun: `Not a CODE reference at
/home/usr1/digital/membersurl/perl/content_manager/handler.pl line
57.'
[Tue Oct 23 06:57:27 2001] [error] Can't locate object method uri
via package Apache::PerlRun at
/usr/lib/perl5/site_perl/5.6.0/i386-linux/Apache/PerlRun.pm line 212.
Things like this are usually the result of using libraries that don't
declare package names. Do you pull in a bunch of functions with
something like 'use mylib.pl' with no package names declared in
mylib.pl? There is more discussion about this in the guide. In fact,
all of this is in the guide.
- Perrin
Re: namespace-troubles
John Michael wrote: I don't know if this will help. I know very litlle about mod perl, but most of the people on this list don't seem to answer questions from beginners. Hmmm. You asked some questions that made it sound like you hadn't read the documentation, and you stil got 3 answers in 24 hours. One thing you should consider is that not everyone here is in the same time zone. I think you might try adding: use subs qw(main); in the top of your script. or instead of main() use main() That's probably not it. The difference between mail() and main() is that main() circumvents prototypes, as documented in the perlsub man page. If that doesn't work, you will probably need to make the lib a package and import it into your fetch.pl program. That probably *is* the answer, and a good programming practice for any perl project. Does anyone know of a news group that answers mod perl questions? There are lots of places that will answer mod_perl questions (http://perlmonks.org/ for example), but none as good as this list. - Perrin
Re: When to use 'use' for accessing modules?
Chris Allen wrote:
If site::products calls functions from site::customers, do I need
a 'use site::customers' in site::products, when I have already
done a 'use' in my startup.pl?
No, but I always do. It's good documentation, to remind you that if you
ever ran this code outside of mod_perl it would be necessary to have
that 'use' in place.
site::products contains the line:
@ISA=('site::base');
so that methods from site::base can be overridden in site::products.
Do I need a 'use site::base' in site::products for this to work
correctly?
Only if site::base has not been loaded already, but I always do it
anyway, for the same reason as above.
Is $ENV{foo}='bar'; in startup.pl equivalent to PerlSetEnv foo bar
in httpd.conf?
Yes.
Experience has shown that I *don't* need the 'use' statements anywhere
other than startup.pl - but I am not sure why, and would find some pointers
to a discussion of this very useful.
You don't need them because the modules they would load are already loaded.
I would also be interested to know that
if the 'use' statements *are* unnecessary, does including them add any extra
overhead of processing/memory??
There is a very fast check that happens once when the module containing
the 'use' is called. It checks to see if the used module is already
loaded (i.e. if it is in %INC). It also calls the used module's export
function, unless you pass an empty list:
use Foo ();
You should avoid importing symbols all over the place, as explained in
the Guide, but otherwise there is no significant overhead from having
use statements all over the place.
- Perrin
Re: Excellent article on Apache/mod_perl at eToys
Matt Sergeant wrote: OK, so what are we missing? Based on the comments I've seen here over the years, and some on Slashdot, the thing that seems to worry people the most is the lack of an obvious message queue API in Perl. I've seen plenty of implementations, but there isn't a plug-n-play CPAN module for this. Perhaps a port of JMS is in order. - Perrin
Re: Excellent article on Apache/mod_perl at eToys
Matthew Kennedy wrote: On Mon, 2001-10-22 at 21:27, Perrin Harkins wrote: It sounds like the limitation there is that you're interfacing with systems that can't notify you when something new happens. That's not Perl's fault. If you wrote your daemons in Java alpahabet soup, they'd still have to poll the pop3 server. Well, I'd more likely be using a standard javamail API within the context of a EJB server. There is a fairly rich set of call backs there which mean I generally don't have to poll/sleep etc. Also true of the java message service. I suppose the Java stuff does make it more obvious how to do that kind of thing. There are dozens of ways to have some perl code execute every time a message arrives at a mail server without polling (Mail::Filter, procmail, Perl pop3 server, maybe PerlMX, etc.), but there isn't any One True Perl Mail API. Pointing newbies in the right direction is the strength of standardized APIs, and it does seem to have helped Java gain acceptance. There are commercial JMS implementations that simply poll database tables, but most people don't seem to mind as long as that's all hidden behind the standard JMS API. So, maybe a version in Perl would be a useful thing. - Perrin
Re: [OT] P2EE Redux was: Excellent article on Apache/mod_perl at eToys
Stephen Adkins wrote: If no one suggests an appropriate list, I propose starting a p2ee group Can I just say that P2EE is a horrible, horrible name? It includes the Java version number (when is J3EE coming out?), as well as Sun's desperate attempt to make it sound like something you buy (Edition) rather than simply a collection of APIs. Something simple, like Perl Enterprise Project or Perl Enterprise APIs makes more sense to me. - Perrin
Re: Excellent article on Apache/mod_perl at eToys
on 10/22/01 11:13 AM, Matthew Kennedy at [EMAIL PROTECTED] wrote: Why was Berkeley DB chosen for caching when a RDBMS (mysql in this case) was already being used? It's faster and less resource-intensive for this kind of thing. We just wanted a really fast persistent hash, and didn't need SQL or relational concepts. Secondly, I've worked on a good-sized commerce site with mod_perl+HTML::Mason. One of the more dirty secrets is that the back-end of the site involves several standalone perl programs running as daemons. What's even worse is; most of them have to sit in poll/wait loops waiting on message from external systems (such as pop3 servers, ccvs etc.) It sounds like the limitation there is that you're interfacing with systems that can't notify you when something new happens. That's not Perl's fault. If you wrote your daemons in Java alpahabet soup, they'd still have to poll the pop3 server. For comparions, a nice aspect of j2ee applications IMHO is the application server tends to be more general. ie. the application server is not just the web server (as it is with mod_perl). I've found j2ee features such as message beans, queues and such especially useful for back-end work. For these reasons, I personally don't buy the argument that mod_perl makes an effective application server for a good sized task (your mileage will vary no doubt ;). I guess it all depends on what you think application server means. In our case, it was a server that ran code persistently, which supported an HTTP interface. It wasn't our front-end web server, and we could run anything we wanted to in it. Lots of people implement reliable queues in Perl. They generally use something like an RDBMS or Berkeley DB (which has a queue option) for the backing store. Many JMS implementations use an RDBMS for all the hard stuff as well. So again, I'm curious, what does the eToys.com site back-end stuff look like? The web site part that was discussed in the article ended at the Oracle database. There was a system built on Oracle's queuing technology that replicated orders to a backend system for processing. I believe the last revision of that was largely in PL/SQL. I don't know much about it, since there was an entirely different team working on it. - Perrin
Re: Excellent article on Apache/mod_perl at eToys
Thanks to all for the kind words. This article actually went up a little bit before it was supposed to, and there should be a revision going up soon with some grammatical fixes and a set of graphics to illustrate parts of it. I'll post a follow-up when that happens in case anyone wants to go and look at the pretty pictures. While we're on the subject, thanks to everyone who contributed to the many open source projects that we used. We couldn't have done it without you. - Perrin
Re: Excellent article on Apache/mod_perl at eToys
What I'd love to see is the avg spec and numbers of machines in each section. So how many proxy, mod_perl and search servers were required to give the phenomenal performance you managed to achieve. Well, this was a long time ago (I wrote the article over a year ago), and I don't remember exactly. The proxy machines were pretty basic, the search servers were heavy on CPU power, and the mod_perl servers were heavy on RAM. There were a lot of machines in the cluster, but I don't remember exactly how many and it changed over time. There were dozens of mod_perl servers when the cluster was at its biggest. Most of them were idle for the majority of the time, but they were all needed for the occasional peak load. I remember at one point I was feeling embarrassed about the number of machines and I told one of our sysadmins that it might have been a better strategy to get a big Sun box or two instead. He replied that a Sun box with equivalent power would have cost about 10 times as much as what we paid for our rackmounted Intel machines. After that, I didn't worry about it too much. - Perrin
Re: [OT] Re: Excellent article on Apache/mod_perl at eToys
Wasn't this seasonal rush at least partly caused by the so-called toywar (www.toywar.com) between eToys.com (the online retailer) and etoy.com (the art group)? There were a lot of DoS attacks (and some even uglier, nastier ones) in 1999 as a result of that. Most of that was dealt with through standard access control stuff, plus the throttling code that was mentioned in the article. The 2000 rush, which we built this new system for, was real customer traffic, although there's always some joker with a bot trying to buy all the PlayStation 2 units. - Perrin
Re: multiple rapid refreshes - how to handle them.
Is there a standard way of dealing with users who are on high bandwidth connections who hit refresh (hold down F5 in IE for example) many times on a page that generates alot of database activity? Try this: http://www.stonehenge.com/merlyn/LinuxMag/col17.html - Perrin
Re: cgi to mod_perl question
By making 'ThreadsPerChild 1' - the entire server runs as a single thread, mod_perl and any other requests (e.g. images) that come in - this is why the server slows to a snails pace. I've set up two instances now, one to handle all incoming requests, the other to serve as a backend proxy pass-thru for mod_perl requests. It seems to be working out fairly well. Ah, that makes sense. Good idea. People have reported that even with its limitations on NT, mod_perl is still a lot faster than CGI or ASP with PerlScript. - Perrin
Re: apache::dbi vs mysql relay
On http://www.firstworks.com/sqlrelay/programming/perldbd.html it says: For the duration of the session, the client stays connected to a database connection daemon. While one client is connected, no other client can connect. Care should be taken to minimize the length of a session. Doesn't seem good at all. Reminds me of the early days of mSQL (is it still working with one client at a time?) The documentation is confusing. My impression is that there is a pool of available database connections, as well as a pool of listener daemons. This statement seems to be saying that while connected you tie up one listener and one database connection. Their theory seems to be that connecting to their daemon is much faster than connecting to the database, and the actual database connections are kept open by the daemons. What I don't understand is why they separate the listener and database connection daemons if you always need one of each to do anything. In theory, using this thing could result in fewer database connections although it would impact performance at least a little. - Perrin
Re: Help with converting from plain CGI to mod_perl
Converting all the globals to Module variables has been a piece of cake, but I've run into one big issue. Depending on how you log into the system, the user will connect to different databases. When they do this, I store the Database Handle from DBI/DBD as $main::dbh. This variable has the scope of the length of processing the request and then should become undef. The best way to do this under mod_perl is to store it in $r-pnotes(). It's documented in perldoc Apache, as well as the guide and the eagle book. - Perrin
Re: search engine module?
Please make sure that it's possible to do a plain ordinary literal text string search. Nothing fancy, no case-folding, no automatic removal of puctuation, nothing like that. Just a literal string. Last night I tried to find perl -V on all the search engines mentioned on the mod_perl home page and they all failed in various interesting ways. The amazingly fast ht://Dig (http://www.htdig.org/) engine can do phrase searching, but I'm not certain how well it does with punctuation. - Perrin
Re: cgi to mod_perl question
It *seems* that the script can run once, but that the next time it is accessed, it cannot connect to the $server anylonger, it dies with a 'connection timed out' message ... What 'gotcha' is causing this? It sounds like this problem: http://mathforum.org/epigone/modperl/joidwendsmeld/3B30E553.AD3A7DB8@chamas. com In the message, Joshua suggests a workaround of running mod_perl single-threaded (which shouldn't make any difference, since mod_perl on Win32 is already single-threaded). - Perrin
Re: IPC::Shareable
I am trying to make IPC::Shareable work with my script, however I get this error: IPC::Shareable::SharedMem: shmget: Permission denied at /usr/local/lib/perl5/site_perl/5.005/IPC/Shareable.pm line 456 [Tue Oct 16 14:44:15 2001] [error] Could not create shared memory segment: Permission denied Does any one know what's up? In the time it takes you to debug it, you could switch your code to MLDBM::Sync and forget about all these annoying shared memory issues. It would probably be faster too. - Perrin
Re: IPC::Shareable
is there a delete method/operation any where? Could not find anything in the perldoc docs indicating a method like that! It supports the same interface as normal hashes, so you can delete keys in the same way. You may also want to read the MLDBM documentation if you haven't already. - Perrin
Re: cgi to mod_perl question
Unfortunately, I've now noticed that my server is unbearably slow - setting 'ThreadsPerChild 1' in my httpd.conf made my server slow to a snails pace (and it really is a pretty quick server on a T1 line dedicated to it, pretty much). That's odd. On Win32, mod_perl runs as a single blocking thread. I wouldn't expect changing that setting to affect anything. Are you sure you were running your script under Apache::Registry? (Maybe someone who actually uses Win32 could supply some help here?) Have I missed something here? Certainly this isn't the only way mod_perl can run if it needs to make IO::Socket::INET connections??? I do my mod_perl development on Linux, so I'm just repeating what others have posted before. I've never heard of an actual solution for this problem, just the workaround. Please help me out here - I'm truly just a Linux/*BSD guy trying to get a poor Win2K box w/ Apache to run the Perl scripts right ... You would certainly get better performance on Linux or BSD. If you have to use Win32 and need better performance, you may have to look at other options like ActiveState's PerlEx. - Perrin
Re: Variant of Monday-morning bug with Apache::DBI + DBD::Oracle
Apache::DBI's ping check worked fine, but when the dbh was ejected from the cache (and so went out of scope), something in the DESTROY stack was blocking, and holding the child up for 12m. I'm guesing the underlying DBD::Oracle code was trying to do a nice shutdown on the dbh, but obviously couldn't. Quick hack: Tweak Apache::DBI to keep the ejected dbh in scope, in a global @array or something, and perform a daily/weekly restart on apache. Are you loading the Oracle driver in the parent process (with startup.pl)? I think I remember this sometimes causing problems with re-connecting. Another solution is to have the child process exit if the ping fails. You get one failed request, but you clear out the messed up processes quickly and replace them with new ones that can connect safely. - Perrin
Re: search engine module?
I don't want to reinvent the wheel and I'm sure this has been done a zillion times, so does anyone know of a module in CPAN that I can use for this? Have you tried searching on http://search.cpan.org/? DBIx::FullTextSearch DBIxTextIndex Search::InvertedIndex Plus lots of other stuff like Glimpse and Swish which interface to C-based engines. - Perrin
Re: Problem with Apache-request
I need to find out why Apache-request is not being found and assigning a value. If I had to guess, I'd say your mod_perl is not installed correctly. I would rebuild Perl, apache, and mod_perl (static, not DSO) and see if that fixes the problem. If you want to troubleshoot it more before doing something that drastic, you could try some of the suggestions in the guide for verifying that you have a working mod_perl install. - Perrin
Re: Variant of Monday-morning bug with Apache::DBI + DBD::Oracle
PH Another solution is to have the child process exit if the ping PH fails. You get one failed request, but you clear out the messed PH up processes quickly and replace them with new ones that can PH connect safely. Yeah, good point. Although our poor little WAP service (for that is what was for) gets so few hits, they'll all be getting failed if we do that ; Okay, try never letting the connections go idle. Set up a cron to send in a request every minute or so, or whatever it takes to keep your Oracle connections active. - Perrin
Re: Problem with Apache-request
Try doing my $r=Apache::Request-new($r); You're confusing Apache::Request with Apache-request(), which is easy to do. Apache::Request is a different class, which is part of the libapreq distribution. Apache-request() returns the current request object (commonly called $r). - Perrin
Re: Off-topic - Apache Config - Load crises
Got a server which is getting hit really bad. Have to keep it up. You didn't really give us enough information to guess at what your problem is. You describe one symptom in this message: slow ping times. What else is wrong? Are pages loading slowly? Failing to load? What applications are you running? Are the problems with applications or static pages? Are you running mod_perl at all? How much memory is free? Are you swapping? How are you checking? What platform is this? Have you turned Off the top of my head, your MaxClients is set way too high for a mod_perl box. - Perrin
Re: PerlModule not updating %INC
We are using perl 5.6.0 for Apache 1.3/20, with mod_perl 1.26. Are you sure? There was a problem with %INC and PerlModule, but I thought it was fixed in 1.26. - Perrin
Re: PerlModule not updating %INC
At 18.07 -0400 10/11/2001, Perrin Harkins wrote: We are using perl 5.6.0 for Apache 1.3/20, with mod_perl 1.26. Are you sure? There was a problem with %INC and PerlModule, but I thought it was fixed in 1.26. - Perrin Indeed, like I said, I tested it by dumping %INC myself -- the modules are indeed missing when loaded with PerlModule. No, I meant are you sure you're running 1.26? Please doublecheck it, since this sounds so much like the bug from the previous release. - Perrin
Re: [OT] Off Topic?
Is there anything special about putting things, like, [OT] in the subject line, other than being informative? No, but it does allow people to use automatic mail filters to ignore OT messages. Is there a reference that provides a list of lists for apache.org that can be used to direct the message to the most appropriate list? The list you want may or may not be at apache.org. There is a list of resources at the end of The Guide: http://perl.apache.org/guide/help.html However, there is a much more complete list for Perl at http://lists.perl.org/. And don't forget the USENET groups. Also, there are a whole bunch of people at http://perlmonks.org/ who are positively chafing at the bit to answer tricky Perl questions for you. - Perrin
Re: apache::dbi vs mysql relay
Are there any benchmark comparisons between apache::dbi and mysql relay? I've never heard of this mysql relay before. A Google search found this: http://www.firstworks.com/sqlrelay.html Is that it? Looks interesting! We're planning on having four sql servers, one of them will do all of the writes to the db and the other three will only be used for reads from the db. The data in the db that is doing the writing will be constantly replicated across the other three machines. Which would be better to use to initiate a persistent database connection? The main concern we have is having to have two different database handles open for one script, one to the machine that does the writing and the other to do the reads. Can Apache::DBI detect this? Apache::DBI can certainly tell the difference between two database handles. It caches based on your connection parameters. For raw performance, you can't beat Apache::DBI. It simply does a hash lookup, and uses an already open connection if there is one. SQL Relay appears to need at least one socket connection to be created at the beginning of each request and torn down at the end. SQL Relay looks like it would scale further though, in situations where the number of open database connections becomes an issue. This is generally not a problem for small sites or sites running databases like Oracle on big iron, but it could be in your situation, using MySQL. Also, SQL Relay claims to have load balancing features between multiple replicated databases. With Apache::DBI, you would need to configure your mod_perl servers to point to specific databases. If you try this thing out, please report back to the list on how it works for you. I think a lot of people here might be interested. - Perrin
Re: Galleries/ModPerl/Images
Is there a reason NOT to put the images in a database? There are many. 1) You can't manipulate them with file-based tools any more. 2) Handling a request for an image consumes many more resources, since there is now application code and a database involved rather than just a simple static file request. 3) Most databases don't make dealing with BLOBs as simple as MySQL does, so your code becomes rather hard to port. Those are the biggies. Of course there are reasons to do it too, but every time I've put images/templates/files in a database I have lived to regret it. Storing the path seems to work better. - Perrin
Re: ANN/RFC: Apache::Session::Generate variants
Here is Apache::Session::Generate::* variants, which especially uses Apache standard C-modules. Apache::Session::Generate::ModUniqueId http://bulknews.net/lib/archives/Apache-Session-Generate-ModUniqueId-0.01.ta r.gz http://bulknews.net/lib/archives/Apache-Session-Generate-ModUniqueId-0.01.ht ml uses mod_unique_id for session id. Apache::Session::Generate::ModUsertrack http://bulknews.net/lib/archives/Apache-Session-Generate-ModUsertrack-0.01.t ar.gz http://bulknews.net/lib/archives/Apache-Session-Generate-ModUsertrack-0.01.h tml uses mod_usertrack's cookie for session id. This is great! Good work. One less piece to write when getting a new project started... - Perrin
Re: how to catch a killed task?
Perrin Harkins wrote: I believe the limiting done by BSD::Resource is pretty harsh and may actually be at the kernel level. I don't think you can catch the signal and deal with it yourself. What you should do is use Apache::SizeLimit to handle your size constraints, and just use BSD::Resource for extreme cases, i.e. out of control servers in tight loops. Thanks for your answer, but (I think) I need BSD::Resource because I allow users to write their own scripts (executed with reval from the Safe-Module) and they can write infinite loops or consume memory as they want - Apache::SizeLimit is not harsh enough for this ;-) Correct, which is why I said to use both. SizeLimit catches normal growth, and BSD::Resource catches emergencies. You set the limit for BSD::Resource higher than the one for SizeLimit. - Perrin
Re: POST and GET and getting multiple unsynced requests
I've just created a form like that just for testing and it worked fine, (form name=zbr method=POST action=/cgi-bin/printenv/zbr/?zbr=t) I think that's just a coincidence. IIRC, the spec doesn't require this to work, and it doesn't work in all browsers. The only real solution is to not do it. PATH_INFO was a good suggestion. I'd go with that if it can't be added to the POST data. - Perrin
Re: how to catch a killed task?
it just killed the task at soft limit without calling cleanup. what is wrong? I believe the limiting done by BSD::Resource is pretty harsh and may actually be at the kernel level. I don't think you can catch the signal and deal with it yourself. What you should do is use Apache::SizeLimit to handle your size constraints, and just use BSD::Resource for extreme cases, i.e. out of control servers in tight loops. or is there another possibility for cleaning up? probably there is a handler i can use which will be called when a program is finished, if so, how can i check if the program has ended as it shoulds or if it was killed by BSD::resource ? Put your cleanup code in a PerlChildExitHandler. Apache::SizeLimit will allow this to be called, since it uses the $r-child_terminate() call. BSD::Resource just does a harsh kill. - Perrin
Re: Setting Perl-stuff inside Perl section
@PerlModule = qw(Blah); # add others to the list Might as well just say use Blah;, I think. - Perrin
[OT] Re: What hourly rate to charge for programming?
Now take the amount you want to make and divide it by the number of hours you came up with above ($40,000 / 1,000). You get $40. That's your target hourly rate. And despite what they high-flying .com weanies were saying a year ago, that's going to be a nice living for a young guy unless you're smack in the middle of a high-cost area and can't bother to cook your own meals. Don't forget that self-employed people in the US must pay considerably more in social security, as well as covering the full cost of their own health insurance and other needed benefits. $40K as a consultant is much less spendable money than $40K as an employee. - Perrin
Re: Using DOM to build your output documents
* Jeffrey W. Baker ([EMAIL PROTECTED]) [011003 19:52]: I'd love to hear any other experiences with using the DOM to build output from scratch. I'd also like to hear people's opinions on XML::DOM (I think it stinks and I've replaced it with a C DOM implementation). I can't speak to XML::DOM, but I used XMLC in Java to do this and there were aspects that were pretty nifty. HTML_Tree is like XMLC, but less irritating, i.e. no lame compile cycle to deal with and somewhat more straightforward. There's also an XML version here: http://homepage.mac.com/pauljlucas/software/xml_tree/. I prefer a more traditional templating approach, but some people love this stuff. - Perrin
Re: CGI.pm params not being cleared?
It doesn't much matter whether you're using stacked handlers or not, or pushing vs. replacing, because the $r-child_terminate method seems to simply call the C exit(0) function under certain conditions (Win32, old version of apache, anything else?). Not much chance of any further action if that's happening. What is this Win32 you speak of? I actually don't think this is much of an issue since people running on Win32 probably don't use SizeLimit (with one process, it's not such an issue), and it wouldn't work if they did since it doesn't know how to check process size there. Of course if someone out there needs it on Win32 and wants to contribute code to make it work, I'll take the patch. - Perrin
Re: CGI.pm params not being cleared?
Alex Harper wrote: Aha! That's where the problem lies. I had recently added: PerlCleanupHandler +Apache::Sizelimit to my httpd.conf. I placed it there so the sizelimit would be enforced post-request. Removing the line fixed my problem. Whoa! This could be bad. Apache::SizeLimit calls $r-post_connection(). Is that replacing existing cleanup handlers? I thought post_connection()/register_cleanup() pushed handlers on the stack rather than replacing them. - Perrin
Re: CGI.pm params not being cleared?
Alex Harper wrote: I'm using a mod_perl 1.26 RPM I built using the RedHat RPM prep files and dropping in the 1.26 code. I believe it is built EVERYTHING=1. Apache::Status indicates all hooks are enabled, including: PerlStackedHandlers Enabled Okay. Would you mind putting some debugging statements in to find out what's being called? If you could put something in Apache::SizeLimit::exit_if_too_big() and something in CGI::_reset_globals(), maybe we can get to the bottom of this. - Perrin
Re: Restricting cpu time for mod_perl scripts? AND Reevaluating perl scripts under mod_perl.
I am currently using Apache::Resource to limit the maximum amount of ram the apache childs are allowed to use. It would be better to use Apache::SizeLimit for that. It's more flexible and allows requests to complete before shutting down the process. Apache::Resource is more useful as a safety measure to catch runaway processes. However, I can't really use PERL_RLIMIT_CPU because it is kind of pointless to kill every apache child that reaches this limit. I need a way to restrict cpu time on a per script or per run basis. But I am not sure if this is possible as far as I understand the mod_perl layout. Anybody did this? Or any suggestions? That isn't possible without some modification to apache. You would need to be able to check the CPU consumed in some asynchronous way as the request was processed, and there's no provision for that in the current apache process model. My second problem is related to this one. How can I add perlcode via httpd.conf (PerlModule, ...) which is reevaluated at EVERY request? All you need is a handler. Every request can run as many handlers as you configure in each stage. If you're asking how to assign a handler globally for every request, just use LocationMatch * or something similar. I am thinking about adding some own perl code which sets an alarm an checks every now an than how much cpu time the currently running perl script allready got I don't think you can do that. There are warnings in perlipc about how having a complex alarm handler can cause segfaults. Honestly, if all you're trying to do is keep coding mistakes from crashing your server with a tight loop, just set some reasonable limit with Apache::Resource. When there's no problem, apache children don't generally use much CPU at all. That means runaways really stand out and use up their allotted CPU limit quickly. Some other probleme here. How do I tell mod_perl to execute abc.pl inside the current scope but without saving the compiled code or any variables? You can't. You can remove the entries from the symbol table after running it (see Apache::PerlRun for an exmaple), but why do you want to do this? If you're just trying to save RAM, load the script at startup. If it's huge and you don't usually need it, tell the current process to exit when it's finished with this request. - Perrin
Re: Porting
We have an application written in Java using MVC which we would like to port to mySQL/Perl platform. We have used Struts,to create tags. Any inputs in this regards will be appreaciated. You will probably find that Struts tags can be ported nicely to Template Toolkit (http://template-toolkit.org). You might even be able to write a converter for your JSP pages. If you're interested in getting some object/relational mapping features and a basic MVC structure thrown in as well, you can check out OpenInteract (http://openinteract.org/). - Perrin
