Re: Memory explodes loading CSV into hash
On Sun, Apr 28, 2002 at 05:18:24PM +0200, Ernest Lergon wrote:
> Hi,
>
> in a mod_perl package I load a CSV file on apache startup into a simple
> hash as read-only class data to be shared by all childs.
>
> A loading routine reads the file line by line and uses one numeric field
> as hash entry (error checks etc. omitted):
>
> package Data;
>
> my $class_data = {};
>
> ReadFile ( 'data.txt', $class_data, 4 );
>
> sub ReadFile
> {
> my $filename = shift; # path string
> my $data = shift; # ref to hash
> my $ndx_field = shift; # field number
>
> my ( @record, $num_key );
> local $_;
>
> open ( INFILE, "<$filename" );
> while ( )
> {
> chomp;
> @record = split "\t";
> $num_key = $record[$ndx_field];
> $data->{$num_key} = [ @record ];
> }
> close ( INFILE );
> }
>
> sub new...
> creates an object for searching the data, last result, errors etc.
>
> sub find...
> method with something like:
> if exists $class_data->{$key} return...
> etc.
>
> Now I'm scared about the memory consumption:
>
> The CSV file has 14.000 records with 18 fields and a size of 2 MB
> (approx. 150 Bytes per record).
>
> Omitting the loading, top shows, that each httpd instance has 10 MB (all
> shared as it should be).
>
> But reading in the file explodes the memory to 36 MB (ok, shared as
> well)!
>
> So, how comes, that 2 MB data need 26 MB of memory, if it is stored as a
> hash?
>
> Reading perldebguts.pod I did not expect such an increasing:
>
> Description (avg.) CSV Perl
> 4 string fields (4 chars)16 bytes (32 bytes) 128 bytes
> 9 float fields (5 chars) 45 bytes (24 bytes) 216 bytes
> 5 string fields (rest) 89 bytes (32 bytes) 160 bytes
> the integer key (20 bytes) 20 bytes
> 150 bytes 524 bytes
>
> That will give 14.000 x 524 = approx. 7 MB, but not 26 MB !?
I tried this program in Perl (outside of modperl) and the memory
consumption is only 4.5MB:
#!/usr/bin/perl -w
$foo = {};
for ($i = 0; $i < 14000; $i++) {
$foo->{sprintf('%020d', $i)} = 'A'x150;
}
<>;
1;
So I suggest something else might be going on causing your memory
problems.
-jwb
Re: ASP Cookieless Sessions (WAS Re: Apache::ASP)
Secure sessions are hard work. You need to sit down and evaluate whether or not you
actually need *secure* sessions. If you decide that enough is at stake to really
tighten the screws, then read on.
The problem of the session ID in HTTP_REFERER is easy to tackle. You just need to
rewrite every URL on your secure site to be a redirect from a page that doesn't
include the session ID. For example:
http://cnn.com/
becomes
http://myhost.com/strip_session?url=http://cnn.com/
and the HTTP_REFERER at cnn.com doesn't include the session ID. Clearly the path
strip_session needs to not require a session ID.
-jwb
On 12/07/99, Joshua Chamas wrote:
> Serge Sozonoff wrote:
> >
> > Hi Joshua,
> >
> > I was wondering if you could give us a little update on the status of
> > Sessions w/o cookies for Apache::ASP?
> >
> > Have you started coding it already or is it at the bottom
> > of a big list of things to do?
> >
> > Many Thanks,
> >
> > Serge
> >
>
> I have been working with Remi Fasol on this, and what we
> have so far is a set up that uses the QueryString for
> keeping track of session ids, in addition to the cookie
> implementation that exists.
>
> The QueryString method represents nice backup to the existing
> cookie implementation for users that have cookies turned off,
> but functionally is not as nice because someone will get a
> new session id when reentering the site. So if a user starts
> browsing off site, and instead of hitting the back button,
> just reenters your URL in the location bar, they will start a
> new session. Remi commented that Amazon's sessions seem to
> have the same limitations.
>
> These cookieless sessions are implemented with a config setting
> SessionQueryStringID that if set, ASP will automatically parse
> out of the QueryString the session-id, and use that if there
> is no such session-id cookie. Then, whatever URL's you
> want the system to keep track of, the web developer uses the
>
> $Server->URL($url, { %params })
>
> API extension to create a URL that will have a session id
> built into it. This $Server->URL() method will be nice
> for just building URL's in general with or without the
> cookieless sessions, as it will joing the params hash
> into a nice ?query_string automatically.
>
> It reoccured to me just now (back from a sessions
> methods discussion a long time ago) that these query string
> cookies might show up in the referer logs of other sites if you
> have offsite links on your session id pages. I tried a workaround
> just now where a redirect program would handle offsite links, but
> the HTTP_REFERER is sticky to the last page visited, and I see
> no workaround to this security issue.
>
> What options are there anyone, for real cookieless sessions,
> without this security risk ??? We can't use IP authentication
> because of proxies/NAT, maybe an SSL cert, but not everyone has
> this, the UserAgent is not stratified enough to mean much,
> so that what, when we are trying to get past cookies here.
>
> -- Joshua
> _________
> Joshua Chamas Chamas Enterprises Inc.
> NodeWorks >> free web link monitoring Huntington Beach, CA USA
> http://www.nodeworks.com1-714-625-4051
--
Jeffrey Baker * [EMAIL PROTECTED]
Re: mod_perl Programmers demand is going up...
Stas Bekman wrote: > > Hi, folks > > You wouldn't beleive but I receive a great deal of mod_perl job offers. > Here is a partial quote of the most generic content of the offer: > > At any rate, we're finding it very hard to find mod_perl people and I > thought maybe you could refer me to someone you know. I can tell you > more about the client and the positions if you're interested. > > Which makes me thinking that we are too few and the demand is growing > (which is good for us :), but from the other side it's bad for mod_perl, > as when a project staff understands the power of mod_perl and want to use > it but have no experience with mod_perl and can hire none of mod_perl > programmers, there is no other choice left as to use some other probably > worse technology because they could dig up the required man power that has > the required knowledge in the other field. Stas, ever since you made that remark at the O'Reilly Conference, wishing that there was no such thing as money, I have known that you and I are of the same opinion regarding economics. Sadly, we seem to be stuck in this money-driven system for at least a little while more. There can be no such thing as a shortage of mod_perl programmers. When someone says that they can't find mod_perl people, what they really mean is that they can't find mod_perl people for the price they are willing to pay. We are expensive, you see. Frequently the prospective employer of a mod_perl person will decide that they simply cannot afford to go the mod_perl route. They instead use a different solution whose cost is lower. The second system isn't neccessarily inferior, it is simply the best system the client could afford. In light of the above, there is nothing you can do to relieve a shortage that doesn't exist. What you can do is lower the price of mod_perl skills by making them more widespread. This will allow more people to use mod_perl, and I think all of the people on this list would approve of that. I personally approve of anything which helps the worldwide flow of information. Now unfortunately a lot of people don't share our views on money. They want to maximize the monetary value of their mod_perl knowledge, which is all well and good as far as I care. The problem for these folks is that the more widespread basic mod_perl skills become, the harder it will be to charge extortionary prices for performing the simplest mod_perl tricks. In effect the price that mod_perl programmers can demand will become stratified: those with the most skills will still be very highly valued, but those with modest skills will see their wages decline. > Now the question is: if this is truth and you feel the same way, how do we > make the world know that there is a great demand for mod_perl programmers > and that people should learn mod_perl and not the stright-forward choice > like VB, ASP or other technology that drives the web? Most people are lazy. Since it is easier to get started with VB/ASP/Crapomatic-9000, most people are going to start there. The people with real initiative and drive will always find their way over to mod_perl or other advanced solutions. The only way to make more people use mod_perl is to make mod_perl easier for the lazy people. I think dougm's book and your upcoming book will do a lot to make mod_perl more accessible. -jwb -- Jeffrey W. Baker * [EMAIL PROTECTED] Critical Path, Inc. * we handle the world's email * www.cp.net 415.808.8807
Re: Apache::Sandwich - how can I put the right page title
Vivek Khera wrote: > > > "SM" == Shay Mandel <[EMAIL PROTECTED]> writes: > > SM> p.s. - the header file I'm getting is the same for all the pages, thus > SM> it does not resides in the same directory as the page itself. > > Conveniently, Netscape recognizes the html tag anywhere in the > document, not just in the section. I didn't test if it worked > for other browsers, though. That would not amount to valid HTML. According to the HTML 4.0 recommendation, "Every HTML document must have a TITLE element in the HEAD section." Further, the TITLE element is required. Every document must have exactly one TITLE element. http://www.w3.org/TR/REC-html40/struct/global.html#h-7.4.2 What the HTML spec does not require is for the HEAD section to come first. So if you must include the HEAD section at the end of the document, that would be fine. Experiment to ensure that this does not break user agents. -jwb -- Jeffrey W. Baker * [EMAIL PROTECTED] Critical Path, Inc. * we handle the world's email * www.cp.net 415.808.8807
Re: HTTP 1.0 / 1.1
Pouneh Mortazavi wrote: > > OK. so this is unrelated to the modperl mailing list. but its an > interesting question. If you know anywhere else I could post this, > please tell me. > > I'm trying to find out how many clients support HTTP 1.1 features such as > keepalives and domain-name based virtual hosts. > > Apparently netscape 3+,ie4+ all run HTTP 1.1. Which is good. This is 95% > of my user agents. > > When I look through the logs however, I see a *huge* amount of HTTP 1.0 > requests from what are claimed to be http 1.1 compliant browsers. I get 3 > times more HTTP1.0 requests than HTTP1.1 requests. I can understand that > some of these user agent strings may be rogue spiders/browsers > masquarading as netscape/ie ... but not that many! > > Can anyone explain, why this would be the case? I know that my apache > servers can implement HTTP protocol negotiation, but I definately do not > do that in my conf files... > > Has anyone else done any analysis on their webserver logs for http > protocols? I'm not terribly familiar with the Apache log format. Are you sure that the log records the protocol level in the request, or the protocol level of the response? The standard Apache config files include a rule to downgrade certain user agents to HTTP 1.0. Is this rule in your config file? Might an intervening proxy downgrade the request to 1.0? Regards, Jeffrey -- Jeffrey W. Baker * [EMAIL PROTECTED] Critical Path, Inc. * we handle the world's email * www.cp.net 415.808.8807
Re: installation problems
"Jesús Mª Lasso Sánchez" wrote: > > hi: > > I'm working with linux servers based on RedHat 6.0 with Apache 1.3.9. > > My problem is when compile the mod_perl module before to install Apache. More >exactly when i'm doing: perl Makefile.PL > > it produces this error when is creating the makefile: > > > + doing sanity check on compiler and options > ** A test compilation with your Makefile configuration > ** failed. This is most likely because your C compiler > ** is not ANSI. Apache requires an ANSI C Compiler, such > ** as gcc. The above error message from your compiler > ** will also provide a clue. > Aborting! > > > the OS is RedHat 6.0 and the c compiler is the packet "egcs". The problem just >happen when compile "mod_perl". I compiled Apache 1.3.9 without problems. Could you send the output of 'perl -V' and 'which `perl -V:cc`? Also what command line did you use for Makefile.PL? -jwb -- Jeffrey W. Baker * [EMAIL PROTECTED] Critical Path, Inc. * we handle the world's email * www.cp.net 415.808.8807
Where to start with mod_perl (was Re: Flaky software)
Glen Lee Edwards wrote: > > I'm brand new to the list, so I thought I'd better ask a question before > making another post: > > What is the specific purpose/slant of this list? Is it a developer's > list? A general list to help those running Mod Perl? Are only > programming geniuses allowed to post, or can the average joe ask dumb > questions? > > I've never run Mod Perl in a real life setting but like the concept. I > own several domains on commercial servers that I plan to eventually move > to my own server, so am trying to get acquainted the the best way to set > it up and administer it. > > If this isn't a list for those just learning Mod Perl, please point me in > the right direction. No this is just the place for beginners. First you should apply a minimum of effort by reading the guide at http://perl.apache.org/guide. Perhaps you might also want to buy the book at http://www.modperl.com/. After that you probably won't have any basic questions left :) But if you do, feel free to ask them here. -jwb -- Jeffrey W. Baker * [EMAIL PROTECTED] Critical Path, Inc. * we handle the world's email * www.cp.net 415.808.8807
Re: Duplicated emails from mod_perl list
"G.W. Haywood" wrote: > > On Mon, 22 Nov 1999, Andrei A. Voropaev wrote: > > > Probably this is off-topic. But it's about the list itself. Why do I > > get OLD emails from time to time. Today I got about 7 or 8 of > > those. I'm absolutely positive that I've seen them once before. Some > > of those emails are dated Nov. 16th > > > > This happened 2 or 3 times already so I decided to ask. Maybe there's > > some problem? Does anyone else gets them? > > > > Andrei > > Hi there, > > You aren't alone. Today I got 26 mails from the modperl list TWICE. > As I've had absolutely no response to the problem I posted on 3 Nov. > and a truckload of duplicated emails I'm starting to think about > unsubscribe modperl... Regarding your question of Nov 3, I don't see any cause for alarm. Are you seeing problems on actual code, or only in the test harness? All of my production mod_perl systems are fine, even though I've never bothered to look into why make test was failing. Are you planning to use Apache::src in your system? -jwb -- Jeffrey W. Baker * [EMAIL PROTECTED] Critical Path, Inc. * we handle the world's email * www.cp.net 415.808.8807
mod_perl on Apache 2.0
There was a short discussion a while ago about getting mod_perl working with Apache 2.0. Since Apache 2.0 can actually be built and run on a few platforms now, I think it is worth taking a lot at this for real. As far as I can tell, these are the broad things that need to be done: *Make all mod_perl functions reentrant *Remove assumptions about the process model *Remove request stages that may not exist any more (perl child init handler) *Make sure that all storage is being done within the request or connection record (needed because stage X might not have access to the same local storage as stage A). *Debug :) optionally: *Make mod_perl use APR. Also there is the additional headache of combing through the Apache:: classes fixing everything that isn't safe, like dependence on global storage. I'm assuming that Perl itself is reentrant, since it has been embedded in multithreaded environments before (IIS). Hopefully someone can comment on that. Discuss! -- Jeffrey W. Baker * [EMAIL PROTECTED] Critical Path, Inc. * we handle the world's email * www.cp.net 415.808.8807
Re: Perl Domination in CGI Programming
Stas Bekman wrote: [slashdot summary snipped] > Well, if the trend continues we ought to learn Java some day :) Ha! Well they are going to have to make JServ operable at some point if they want to replace Perl. Seriously, could they make their configuration process more opaque? I went to the JServ morning session in Monterey, and I was extremely disappointed. The whole first hour was spent on building and configuring the servlets. They seem to have several non-orthogonal configuration files that live in different places and have very hard to explain syntax and on and on and on. Blah. Even as a developer I want something that has a simple and straight-forward installation procedure. For the operations department it is even worse. JServ has a long way to go before it doesn't suck in that regard. And don't even get me started on their session handling and load balancing [sic]. -jwb -- Jeffrey W. Baker * [EMAIL PROTECTED] Critical Path, Inc. * we handle the world's email * www.cp.net 415.808.8807
Re: mod_perl 2.21 segmentation fault
> glibc 2.0.7 ^^^ Maybe you should consider upgrading to a non-experimental libc. Aside from that, I have no clue. I know dozens of people who use this same setup, and they all work out fine. Did you compile everything together yourself? What happense when you remove mod_perl or mod_ssl? Maybe you should wait for openSSL to actually be released. -jwb > apache 1.3.9 > mod_perl 2.21 > mod_ssl 2.2.4 > openssl 0.9.4 > CGI.pm 2.56 > Oracle 8.0.5 > DBD::Oracle 1.03 > DBI 1.13 > XML::Parser 2.27 > gcc 2.7.2.3 > linux kernel 2.2.13 > > -- > Radovan Semancik ([EMAIL PROTECTED]) > System Engineer, Business Global Systems a.s. >http://storm.alert.sk -- Jeffrey W. Baker * [EMAIL PROTECTED] Critical Path, Inc. * we handle the world's email * www.cp.net 415.808.8807
Re: Checking for valid dates
"Michael J. Miller" wrote: > > I did this using a JavaScript function that runs before the form is > submitted to validate the date locally (in the client, using JavaScript > Date objects) before submission. Has the side benefit of giving the > user real time feed back as well without a send back to the server > > Brgds, > > Mike. You still have to check it on the server side, unless you are willing to let bad data slip into your database or wherever. (Think user with JS turned off or simply submitting via LWP) -jwb -- Jeffrey W. Baker * [EMAIL PROTECTED] Critical Path, Inc. * we handle the world's email * www.cp.net 415.808.8807
Re: Failing to reconnect after Oracle "shutdown abort" (Apache::DBI)
Greg Stark wrote:
>
>*** From dbi-users - To unsubscribe, see the end of this message. ***
>*** DBI Home Page - http://www.symbolstone.org/technology/perl/DBI/ ***
>
> Tim Bunce <[EMAIL PROTECTED]> writes:
>
> > On Mon, Nov 01, 1999 at 09:01:48PM +, Tim Bunce wrote:
> > > Has anyone experienced a situation where a process (httpd for example)
> > > can't reconnect to Oracle after a "shutdown abort"?
> >
> > Thanks for your replies.
> >
> > The problem reported to me which prompted this email has actually
> > proven to be a user script error. The old database handle was being
> > reused, rather than the new one returned from a fresh call to
> > DBI->connect after the shutdown and restart.
>
> With a web server you always want to reuse the database handle as long as
> possible. That's true of any long-running application though.
>
> How is the application supposed to know it has been disconnected?
> Maybe the driver should (maybe optionally) automatically reconnect?
That's what the driver handle's ping method is for.
if (!$dbh->ping) { &reconnect; }
-jwb
--
Jeffrey W. Baker * [EMAIL PROTECTED]
Critical Path, Inc. * we handle the world's email * www.cp.net
415.808.8807
Re: Checking for valid dates
"Tubbs, Derric L" wrote: > > Is there a reasonably easy method to make sure that an entered date is > valid, I.E. is not 30 Jan 99? I am using Time::Local to convert a date > entered through an HTML form into the epoch offset (or whatever you call it) > and would like to make sure that valid dates are entered. Thanks Just because I'm curious, please explain what isn't valid about 30 Jan 99. ObPerl: The Date::Calc module might be what you need. -- Jeffrey W. Baker * [EMAIL PROTECTED] Critical Path, Inc. * we handle the world's email * www.cp.net 415.808.8807
Re: Runaway processes
Joshua Chamas wrote: > > Mike Dameron wrote: > > > > We have several scripts using Apache::ASP and DBI, which return > > information from an Oracle database. The problem is that if I run a > > report that taken a long time (not sure how long but over 30 minutes) > > and the connection either times out or I hit the stop button the oracle > > process which gets spawned does not die. I can see in the process list > > that the oracle process is still tied to a httpd process and continues > > to eat up cpu cycles. > > > > Why would you run a report from a web site that would take > 30+ minutes to finish ? Typical web connection timeouts > are a few minutes or less ? > > I think the real solution is to make your reports fast enough > to run from a web site, like run the real report every night, > and cache its results in the database for later that you can get > them from the browser instantly. Another workable solution is to make the reports in real time, but deliver them by email. Use the web interface to initiate the reporting process and send a confirmation. Cheers, Jeffrey -- Jeffrey W. Baker * [EMAIL PROTECTED] Critical Path, Inc. * we handle the world's email * www.cp.net 415.808.8807
Re: Apache::Session and File Upload (Was: Apache::Sessionhangs script)
Tobias Hoellrich wrote:
>
> Kip,
>
> I think I mentioned this yesterday in an private email message already. I
> had the same problem with Apache::Session 1.03 in my environment. For some
> obscure reason, my tied %session didn't go out of scope and was never
> "untied" at the end of one request.
There are no obscure reasons. If your hash isn't scoped at the request
level, then it is going to live forever.
> This results in a situation where Apache::Session keeps a readlock and the
> next time you try to use the same session it'll hang in IPC::Semaphore::op
> when it tries to acquire the readlock.
Actually, more than one process can hold the readlock at once. The
problem is that some process is holding a writelock indefinitely.
> I worked around this situation by explictly untie'ing the tied %session at
> the end of my request by using
>
> $r->register_cleanup(sub { untie %session });
>
> Since then the Semaphore problems have disappeared.
>
> Hope this helps
> Tobias
--
Jeffrey W. Baker * [EMAIL PROTECTED]
Critical Path, Inc. * we handle the world's email * www.cp.net
415.808.8807
Re: Apache::Session and File Upload (Was: Apache::Session hangs script)
Kip Cranford wrote:
>
> Again, I'm using mod_perl 1.21, apache 1.3.9, Apache::Session 1.03, on a
> RedHat 6 linux system with perl 5.005_03, and am using Netscape Comm.
> 4.51 as my browser.
>
> The problem now seems to be Apache::Session and file uploads. My
> handler is providing a simple file upload interface, and I'm using
> Apache::Session to keep track of filenames, content types, sizes, etc.
>
> Using a very simple script, in which I store only a single scalar
> variable in my session, and using the "multipart/form-data" encoding
> type on my form, I can get the script to hang every time. It _always_
> hangs in the same place in the "op" function:
>
> DB<1> IPC::Semaphore::op(/usr/lib/perl5/5.00503/IPC/Semaphore.pm:90):
> 90: croak 'Bad arg count' if @_ % 3;
> DB<1> IPC::Semaphore::op(/usr/lib/perl5/5.00503/IPC/Semaphore.pm:91):
> 91: my $data = pack("s*",@_);
> DB<1> IPC::Semaphore::op(/usr/lib/perl5/5.00503/IPC/Semaphore.pm:92):
> 92: semop($$self,$data);
The problem is that you are leaking session handles. For
Apache::Session to work, there must be zero references to the session
hash at the end of the request.
-jwb
--
Jeffrey W. Baker * [EMAIL PROTECTED]
Critical Path, Inc. * we handle the world's email * www.cp.net
415.808.8807
Re: eval'ed Apache::Session tie panics with POPSTACK?
Tobias Hoellrich wrote:
>
> I wanted to trap non-existant session-id's by bracketing the
>
> tie %{$href}, 'Apache::Session::DBI', $id,
> {
>DataSource => 'dbi:mysql:sessions',
>UserName => 'db_user',
>Password => 'db_passwd'
> };
>
> with an eval { }; block. Once I've done this and send a non-existant
> session-id I get the following in my error_log:
>
> panic: POPSTACK
> Callback called exit.
This is a bug in Perl.
> Without the eval around the tie the script dies with:
>
> [Tue Oct 19 16:55:09 1999] [error] Object does not exist in the data
> store at /usr/local2/lib/perl5/si
> te_perl/5.005/Apache/Session/DBIStore.pm line 192.
>
> So, do I have to look check the database myself for the session-id? Should
> I modify DBIStore.pm to not die, but do something else?
>
> Is this related to:
> http://x31.deja.com/getdoc.xp?AN=536753692&CONTEXT=940379169.1854472193&hitn
> um=1
Yes. Perhaps you should try upgrading to a newer Perl. You could also
modify DBIStore to not die(), but you will have to come up with some
other exception reporting mechanism.
Reagrds,
Jeffrey
> Cheers
> Tobias
--
Jeffrey W. Baker * [EMAIL PROTECTED]
Critical Path, Inc. * we handle the world's email * www.cp.net
415.808.8807
More on web application performance with DBI
Hi all, I have converted my text-only guide to web application performance using mod_perl and DBI into HTML. The guide now lives alongside my DBI examples page at http://www.saturn5.com/~jwb/dbi-performance.html . I have also conducted a silly benchmark to see how all of these optimization affect performance. Please remember that it is dangerous to extrapolate the results of a benchmark, especially one as rudimentary as this. With that said please consider the following data. Environment: DB Server: Oracle 8.0.6, Sun Ultra2, 2 CPUs, 2GB RAM, Sun A1000 disks App Server: Linux, PII 350, 128MB RAM, Apache 1.3.6, mod_perl 1.19 Benchmark Client: ApacheBench on same machine as application server Each benchmark consisted of a single request selecting one row from the database with a randomly selected primary key. The benchmark was run through 1000 requests with 10 simultaneous clients. The results were recorded using each level of optimization from my tutorial. Zero optimization: 41.67 requests/second Stage 1 (persistent connections): 140.17 requests/second Stage 2 (bound parameters): 139.20 requests/second Stage 3 (persistent statement handles): 251.13 requests/second It is interesting that the Stage 2 optimization didn't gain anything over Stage 1. I think this is because of the relative simplicity of my query, the small size of the test database (1000 rows), and the lack of other clients connecting to the database at the same time. In a real application, the cache thrashing that is caused by dynamic SQL statements would probably be detrimental to performance. In any case Stage 2 paves the way for Stage 3, which certainly does increase the request rate! So, check it out at http://www.saturn5.com/~jwb/dbi-performance.html Cheers, Jeffrey -- Jeffrey W. Baker * [EMAIL PROTECTED] Critical Path, Inc. * we handle the world's email * www.cp.net 415.808.8807
Re: Apache::Session. Why Apache? Why Session?
Dmitry Beransky wrote: > > I've been going through the code of Apache::Session trying to understand > how it works and how to use it when it occurred to me that the module > doesn't really measure up to it's name. I mean IMHO, the name is a bit > misleading. > > First. It looks like the module's code is completely self-contained and > doesn't depend on anything Apache. If my observations are true, why is it > in the Apache hierarchy (except possibly for historical reasons)? Yes, the name is purely historical. In reality, Apache::Session is a persistence framework with multiple interchangable backing stores. You still need to build your session management logic around it. I tried to implement actual session management in the original 0.1x Apache::Session series, but it was unworkable. Some people wanted expiration, some people didn't. Some wanted cookies, others wanted extended path information. I found it much more pleasing to simply implement the part that everyone needed (storage management) and leave the twidly little details up to the programmer. If you want an environment that gives you session handling for free, check out one of the many nice mod_perl toolkits, such as Embperl, Apache::ASP, and HTML::Mason (?). Embperl makes use of Apache::Session, but I don't know what implementation the others use. > Second. It doesn't really manage sessions. It provides implementations > for different models of persistent storage (memory, FS, DB, etc.). A > complete Apache-based session managing mechanism (with session > initialization, destruction, auto-expiration, etc.) still has to be coded > on top of it. > > Any comments? Shouldn't it be renamed? Possibly, but I don't know what the mechanics of moving a module are, and I can't think of a good name anyway. Perhaps Persistent::Hash? > Regards > Dmitry Beransky -jwb -- Jeffrey W. Baker * [EMAIL PROTECTED] Critical Path, Inc. * we handle the world's email * www.cp.net 415.808.8807
Re: please comment on new art for perl.apache.org
Alex Schmelkin wrote: > 1. What browsers are different percentages of your user populations using? > 2. What screen resolutions are different percentages of your user > populations using? > 3. What purpose will the site serve - simply providing information, or a > marketing/evangelical need? How important are each of these considerations? > 4. Who is the ultimate audience? - Who needs this information, and why? > 5. What does the audience already know, and what do they need to know? > 6. What is the feeling or mood to be? > 7. What are some of the positive attributes of the current sites as you see > them? What are some of the negative attributes? 1) and 2) are easy. All of them, and all of them. To design considering anything else is suicide. -jwb -- Jeffrey W. Baker * [EMAIL PROTECTED] Critical Path, Inc. * we handle the world's email * www.cp.net 415.808.8807
Re: DB connection pooling
Stefan Reitshamer wrote: > > Hello, > > Sorry if this is in a FAQ, but I dug through a lot of FAQs and archives and > couldn't find it: > > Is there any database connection pooling built into mod_perl, or DBI? Not really. You could take a run at DBD::Proxy. -jwb -- Jeffrey W. Baker * [EMAIL PROTECTED] Critical Path, Inc. * we handle the world's email * www.cp.net 415.808.8807
Re: Bug in libapreq makes form elements stick to an apache child
Ken Williams wrote:
> Are you sure? I'm still not sure you understand about defined(@list). It
> actually tests whether Perl has allocated memory for the structure, not whether
> the structure has ever been used or anything like that. And even the present
> behavior may change without notice.
Actually that isn't what defined does. defined is supposed to indicate
whether something has a real value or not. It is not and should not be
related to whether Perl has allocated memory for it, although one
implies the other. If no memory has been allocated, then the value is
certainly undefined. However, if memory has been allocated the value
may or may not be defined.
I understand that defined doesn't work on lists and hashes. I had
simply forgotten about that bit of trivia. defined most certainly does
work on list *members* and hash *entries*. For example, 'defined
$hash{key}' is definitely going to work. Same for 'defined $array[0]'.
> When you do "my @list = &func();" and &func returns no values, @list will be an
> empty list. And it will evaluate in a boolean context to 0 (the number of
> elements), which is false. You shouldn't really care whether Perl has
> allocated memory to it or not, because different versions of Perl will give you
> different results with that test.
I don't care whether Perl has allocate memory or not. All I care about
is whether or not there are any defined entries in the list, which I
think is most clearly expressed as 'if (defined $list[0])'. What is
more clear than that? 'if (@list)' could certainly work because an
empty list evaluates to 0 in a scalar context. But the code doesn't
look like what I am intending to do.
> Finally, what if &func returns (undef, 1) or even (undef)? If you use your
> "defined $list[0]" test, you'll get a false result here, when &func actually
> did return stuff.
Obviously I need to know the calling conventions and return values for
all of my functions :) In this case the function must either return an
empty list, or a list consisting of only defined scalar values.
> Of course, I don't know your specific application, but "defined @list" is one
> of the red flags of code.
Right. As I said I simply managed to let that bit of info slip my
mind. Perhaps I should use lint or some such.
-jwb
--
Jeffrey W. Baker * [EMAIL PROTECTED]
Critical Path, Inc. * we handle the world's email * www.cp.net
415.808.8807
Re: Bug in libapreq makes form elements stick to an apache child
Cliff Rayman wrote: > > `perldoc -f defined` yields a couple of sentences: > > You may also use C to check whether a subroutine exists, by > saying C without parentheses. On the other hand, use > of C upon aggregates (hashes and arrays) is not guaranteed to > produce intuitive results, and should probably be avoided. I hate it when that happens. > why not use: > > if(@foo_in) > > instead of: > > if (defined @foo_in) Yeah. I guess the reason I do the latter is b/c I want the code to reflect what I am actually trying to test. I don't really want to test the trueness of @foo, I want to test for it's existence. But in perl the operation is overloaded. Feh. I think I will test for defined $foo_in[0] instead. -- Jeffrey W. Baker * [EMAIL PROTECTED] Critical Path, Inc. * we handle the world's email * www.cp.net 415.808.8807
Re: Bug in libapreq makes form elements stick to an apache child
I should have mentioned in this bug report that the problem only occurs
when you call $r->param in a list context. Everything is fine in a
scalar context. For that reason I suspect this little bit of code:
sub param {
my $self = shift;
my($name, $value) = @_;
my $tab = $self->parms;
unless ($name) {
my %seen;
return wantarray ? grep { !$seen{$_}++ } keys %$tab : $tab;
}
if (defined $value) {
$tab->set($name, $value);
}
return wantarray ? ($tab->get($name)) : scalar $tab->get($name);
}
-jwb
Jeffrey Baker wrote:
>
> Hi,
>
> I haven't had time to really hunt this bug to its source, but here is
> the report anyway.
>
> Take as an example the URI /program?foo=bar&foo=baz&foo=quux
>
> In program, which uses libapreq, I have code which looks like this:
>
> my @foo_in = $r->param('foo');
>
> @foo_in should not be defined if there are no foo arguments in the query
> string. The problem is that once a foo argument has been seen, the
> @foo_in array is defined in that Apache child process for all time. The
> array will not be defined in any other Apache child.
>
> To reproduce the problem, use the handler I have included below and
> perform the following steps:
>
> 1) Visit /yoururl
> 2) Visit /yoururl?foo=bar&foo=baz
> 3) Visit /yoururl as many times as you have Apache child processes
>
> Here is the handler:
>
> package JWB;
> use Apache::Request;
>
> use strict;
>
> sub handler {
> my $r = Apache::Request->new(shift());
> $r->status(200);
> $r->content_type("text/plain");
> $r->send_http_header;
>
> my @foo_in = $r->param('foo');
>
> if (defined @foo_in) {
> print "foo_in is defined!\n";
> }
> else {
> print "foo_in is not defined!\n";
> }
>
> print "foo_in has this many entries: ",scalar(@foo_in);
>
> return 0;
> }
>
> 1;
>
> Regards,
> Jeffrey
Bug in libapreq makes form elements stick to an apache child
Hi,
I haven't had time to really hunt this bug to its source, but here is
the report anyway.
Take as an example the URI /program?foo=bar&foo=baz&foo=quux
In program, which uses libapreq, I have code which looks like this:
my @foo_in = $r->param('foo');
@foo_in should not be defined if there are no foo arguments in the query
string. The problem is that once a foo argument has been seen, the
@foo_in array is defined in that Apache child process for all time. The
array will not be defined in any other Apache child.
To reproduce the problem, use the handler I have included below and
perform the following steps:
1) Visit /yoururl
2) Visit /yoururl?foo=bar&foo=baz
3) Visit /yoururl as many times as you have Apache child processes
Here is the handler:
package JWB;
use Apache::Request;
use strict;
sub handler {
my $r = Apache::Request->new(shift());
$r->status(200);
$r->content_type("text/plain");
$r->send_http_header;
my @foo_in = $r->param('foo');
if (defined @foo_in) {
print "foo_in is defined!\n";
}
else {
print "foo_in is not defined!\n";
}
print "foo_in has this many entries: ",scalar(@foo_in);
return 0;
}
1;
Regards,
Jeffrey
--
Jeffrey W. Baker * [EMAIL PROTECTED]
Critical Path, Inc. * we handle the world's email * www.cp.net
415.808.8807
Re: Apache::Session hangs script
[EMAIL PROTECTED] wrote:
>
> > "Kip" == Kip Cranford writes:
>
> Kip> I'm using Apache/1.3.9 (Unix), mod_perl/1.21, Apache-Session-1.03, with
> Kip> MySQL 3.22.25. I'm using the the Apache::Session::DBI module with MySQL
> Kip> to provide some simple session management. I'm running the script as a
> Kip> PerlHandler...
>
> Kip>
>Apache::Session::SysVSemaphoreLocker::acquire_read_lock(/usr/lib/perl5/site_perl/5.005/Apache/Session/SysVSemaphoreLocker.pm:50):
> Kip> 50: $self->{sem}->op($self->{read_sem} + $self->{nsems}/2, 0,
> Yes, I know, I've posted it here before. For some people (are you
> using FreeBSD?) Apache::Session blocks forever after first usage. I've
> reported it to the list, but no one helped.
That's due to poor programming on your part. Read the archive.
-jwb
> I've ended up using dummy locking :-(
>
>
> --
> Alexander B. Povolotsky[ICQ 18277558]
> [2:5020/145][[EMAIL PROTECTED]]
--
Jeffrey W. Baker * [EMAIL PROTECTED]
Critical Path, Inc. * we handle the world's email * www.cp.net
415.808.8807
Re: Installation Problem with Apache::Session and Mason
[EMAIL PROTECTED] wrote:
>
> I'm having a problem setting up Apache::Session on
> one machine and not another, and I'm hoping that
> someone will recognize what I'm doing wrong.
>
> I'm using Mason to make the connection to
> the package when Apache starts up. On the problem
> server, the following happens on server start-up:
>
> starting httpd...
> Ambiguous use of read => resolved to "read" => at
> /usr/lib/perl5/site_perl/Apache/Session/SysVSemaphoreLocker.pm line
> 35.
> Ambiguous use of write => resolved to "write" => at
> /usr/lib/perl5/site_perl/Apache/Session/SysVSemaphoreLocker.pm line
> 35.
> Ambiguous use of {read} resolved to {"read"} at
> /usr/lib/perl5/site_perl/Apache/Session/SysVSemaphoreLocker.pm line
> 43.
> Ambiguous use of {write} resolved to {"write"} at
> /usr/lib/perl5/site_perl/Apache/Session/SysVSemaphoreLocker.pm line
> 44.
> Ambiguous use of {read} resolved to {"read"} at
> /usr/lib/perl5/site_perl/Apache/Session/SysVSemaphoreLocker.pm line
> 54.
> Ambiguous use of {write} resolved to {"write"} at
> /usr/lib/perl5/site_perl/Apache/Session/SysVSemaphoreLocker.pm line
> 61.
> Ambiguous use of {read} resolved to {"read"} at
> /usr/lib/perl5/site_perl/Apache/Session/SysVSemaphoreLocker.pm line
> 68.
> Ambiguous use of {write} resolved to {"write"} at
> /usr/lib/perl5/site_perl/Apache/Session/SysVSemaphoreLocker.pm line
> 74.
> Ambiguous use of {read} resolved to {"read"} at
> /usr/lib/perl5/site_perl/Apache/Session/SysVSemaphoreLocker.pm line
> 81.
> Ambiguous use of {read} resolved to {"read"} at
> /usr/lib/perl5/site_perl/Apache/Session/SysVSemaphoreLocker.pm line
> 85.
> Ambiguous use of {write} resolved to {"write"} at
> /usr/lib/perl5/site_perl/Apache/Session/SysVSemaphoreLocker.pm line
> 92.
> Ambiguous use of {write} resolved to {"write"} at
> /usr/lib/perl5/site_perl/Apache/Session/SysVSemaphoreLocker.pm line
> 96.
> Ambiguous use of {read} resolved to {"read"} at
> /usr/lib/perl5/site_perl/Apache/Session/SysVSemaphoreLocker.pm line
> 103.
> Ambiguous use of {write} resolved to {"write"} at
> /usr/lib/perl5/site_perl/Apache/Session/SysVSemaphoreLocker.pm line
> 106.
> Ambiguous use of {read} resolved to {"read"} at
> /usr/lib/perl5/site_perl/Apache/Session/SysVSemaphoreLocker.pm line
> 110.
> Ambiguous use of {write} resolved to {"write"} at
> /usr/lib/perl5/site_perl/Apache/Session/SysVSemaphoreLocker.pm line
> 111.
That is Perl being anal about the use of unquoted read and write as hash
keys. I should probably fix this but it isn't fatal.
> The server does start, however. But when I try to access
> a page that uses the session hash, the following appears
> in the httpd error log:
>
> [Thu Nov 4 11:17:21 1999] [error] Permission denied at
> /usr/lib/perl5/site_perl/Apache/Session/SysVSemaphoreLocker.pm line
> 64.
This is a different problem. At some point someone other than the user
that your httpd runs as created the semaphore block that you are tyring
to use. First, consult the ipcs and ipcrm man pages. Then, as root,
remove the semaphore block that is causing the problem and allow the
httpd to recreate it as the nobody user.
-jwb
> I've checked the permissions on the module itself, the
> directory where the session files are stored, and the
> permissions of the owner of the httpd process. I've
> had no problem on another server, also running Linux
> 2.0.35. All the corresponding PERL modules seem synched,
> as well as the Apache httpd binaries.
>
> Here is the script that is called (part of Mason's
> start up script called handler.pl):
>
> # This block of code can be enabled to create a session-hash that
> every
> # component can access. This is useful for maintaining state
> across
> # multiple requests. The Apache::Session module is required.
> #
> my %session;
> my $cookie = $r->header_in('Cookie');
> $cookie =~ s/SESSION_ID=(\w*)/$1/;
> tie %session, 'Apache::Session::File', $cookie, {'Directory' =>
> '/tmp/session'};
> $r->header_out("Set-Cookie" =>
> "SESSION_ID=$session{_session_id};") if ( !$cookie );
>
> # This creates a global called %session that is accessible in all
> components.
> # Feel free to rename this as needed.
> #
> local *HTML::Mason::Commands::session = \%session;
>
> $ah->handle_request($r);
>
> untie %HTML::Mason::Commands::session;
>
> This is Apache-Session-1.03 and HTML-Mason-0.72 on
> Apache 1.3.6.
>
> Thanks for any help/hints you can provide.
>
> Marc Kelly
> [EMAIL PROTECTED]
--
Jeffrey W. Baker * [EMAIL PROTECTED]
Critical Path, Inc. * we handle the world's email * www.cp.net
415.808.8807
Re: mod_perl on Apache 2.0
Matt Sergeant wrote:
>
> On Thu, 04 Nov 1999, Jeffrey Baker wrote:
> > I'm assuming that Perl itself is reentrant, since it has been embedded
> > in multithreaded environments before (IIS). Hopefully someone can
> > comment on that.
>
> This work was based on PERL_OBJECT support, which is currently only
> available on windows. It's a perl interpreter wrapped up in a C++ object so
> that the whole thing is reentrant. For 5.6 I believe a similar
> functionality but in C is being added (or is moving out of experimental
> mode) called MULTIPLICITY.
I believe -DMULTIPLICITY is already in 5.005_03. Check "man perlembed".
Now suppose we have more than one interpreter instance
running at the same time. This is feasible, but only if
you used the -DMULTIPLICITY flag when building Perl. By
default, that sets PL_perl_destruct_level to 1.
Let's give it a try:
#include
#include
/* we're going to embed two interpreters */
/* we're going to embed two interpreters */
#define SAY_HELLO "-e", "print qq(Hi, I'm $^X\n)"
int main(int argc, char **argv, char **env)
{
PerlInterpreter
*one_perl = perl_alloc(),
*two_perl = perl_alloc();
char *one_args[] = { "one_perl", SAY_HELLO };
char *two_args[] = { "two_perl", SAY_HELLO };
perl_construct(one_perl);
perl_construct(two_perl);
perl_parse(one_perl, NULL, 3, one_args, (char **)NULL);
perl_parse(two_perl, NULL, 3, two_args, (char **)NULL);
perl_run(one_perl);
perl_run(two_perl);
perl_destruct(one_perl);
perl_destruct(two_perl);
perl_free(one_perl);
perl_free(two_perl);
}
--
Jeffrey W. Baker * [EMAIL PROTECTED]
Critical Path, Inc. * we handle the world's email * www.cp.net
415.808.8807
