Re: Apache::Session - What goes in session?

[Ian Struble]

Not in the MS house that I am living in right now :^(

On Tue, 20 Aug 2002, Perrin Harkins wrote:

> Ian Struble wrote:
> > And just to throw one more wrench into the works.  You could load up only
> > the most popular data at startup and let the rest of the data get loaded
> > on a cache miss.  
> > 
> > That is one technique that we have used for some customer session
> > servers.  It allowed each server to start up in well under a minute
> > instead of in 15-30 minutes while pegging the DB.  The 15-30 minutes was
> > when we were dealing with ~5mil total entries and I would hate to see it
> > now that the size of the table has doubled.  Now we just need to do some
> > batch processing to determine what subset gets loaded at startup.
> 
> You could also just dump the whole thing into a Berkeley DB file every 
> now and then.
> 
> - Perrin
> 
> 
> 

Re: Apache::Session - What goes in session?

[Ian Struble]

And just to throw one more wrench into the works.  You could load up only
the most popular data at startup and let the rest of the data get loaded
on a cache miss.  

That is one technique that we have used for some customer session
servers.  It allowed each server to start up in well under a minute
instead of in 15-30 minutes while pegging the DB.  The 15-30 minutes was
when we were dealing with ~5mil total entries and I would hate to see it
now that the size of the table has doubled.  Now we just need to do some
batch processing to determine what subset gets loaded at startup.

Ian



On Tue, 20 Aug 2002 [EMAIL PROTECTED] wrote:

> I havent had much luck with that but we will look at it 
> again and see what we can get from it. We want to avoid 
> preloading all data per child direct from the database but 
> I wouldnt mind doing it on startup for the root process 
> and then copying it to each child.
> 
> J
> 
> 
> On Tue, 20 Aug 2002 16:39:45 -0500
>   [EMAIL PROTECTED] wrote:
> >Just to jump in here - as I understand it you can split a 
> >hash across 
> >multiple threads if you preload it before apache forks. 
> >So load it in your 
> >startup.pl and get it in memory prior to forking. It'll 
> >be part of the 
> >shared memory since you aren't writing to it. Or at least 
> >that's how I 
> >understand the theory to work anyway.
> >
> >Josh
> >
> >
> >
> >
> ><[EMAIL PROTECTED]>
> >08/20/2002 10:54 AM
> >
> >  
> > To: Tony Bowden <[EMAIL PROTECTED]>, md 
> ><[EMAIL PROTECTED]>
> > cc: Perrin Harkins <[EMAIL PROTECTED]>, 
> >[EMAIL PROTECTED]
> > Subject:Re: Apache::Session - What goes 
> >in session?
> >
> >
> >We do see some slowdown on our langauge translation db 
> >calls since they are so intensive. Moving to a 'per 
> >child' 
> >cache for each string as it came out of the db sped page 
> >loads up from 4.5 seconds to .6-1.0 seconds per page 
> >which 
> >is significant.
> >
> >Currently we are working on a 'per machine' cache so all 
> >children can benefit for each childs initial database 
> >read 
> >of the translated string, the differential between 
> >children is annoying in the 'per child cache' strategy.
> >
> >John-
> >
> >On Tue, 20 Aug 2002 16:33:07 +0100
> >   Tony Bowden <[EMAIL PROTECTED]> wrote:
> >>On Mon, Aug 19, 2002 at 06:54:01PM -0700, md wrote:
> >>> I can definitely get it all from the db, but that 
> >>>doesn't
> >>> seem very efficient.
> >>
> >>Don't worry about whether it *seems* efficient. Do it 
> >>right, and then
> >>worry about how to speed that up - if, and only if, it's 
> >>too slow.
> >>
> >>Premature optimisation is the root of all evil, and all 
> >>that ..
> >>
> >>At BlackStar the session was just a single hashed ID and 
> >>all other info
> >>was loaded from the database every time. We thought about 
> >>caching some
> >>info a few times, but always ran into problems with 
> >>replication.  In the
> >>end we discovered that fetching everything from the 
> >>database on every
> >>request wasn't noticeably slower than anything else we 
> >>could up with,
> >>and was a lot more flexible. Throwing more memory at the 
> >>database servers
> >>was usually quicker, cheaper and more effective than 
> >>micro-optimising
> >>our session vs caching strategy...
> >>
> >>Tony
> >
> >
> >
> >
> 
> 

Re: [OT] eToys Jingle (was: Where was that success story?)

[Ian Struble]

And further still into OT land, Israel is a pretty popular Hawaiian
artist.  Too bad he does not get play on the mainland.

Ian

On Wed, 6 Mar 2002, Tom Servo wrote:

> > What I really want to know is: what ever happened to that eToys jingle
> > that was on the commercials? It was almost as good as the site. My
> > children were all under 7 when the site folded, so those commercials and
> > that jingle REALLY pulled the heart strings. 
> > 
> 
> Heh, used to work there.   Song was:
> 
> "Somewhere Over the Rainbow/What a Wonderful World"
> by Israel Kamakawiwo'ole
> on the album "Facing Future"
> 
> Enjoy.
> 
> Brian Nilsen
> 
> 
> 
> 

Re: Win32 Proxy question

[Ian Struble]

How hard are you pounding it in the 'lab'?  I don't remember how hard I 
had to pound to break my win32 proxy(NT4,SP4 and Apache 1.3.9 or 11) but 
it wasn't all that hard.  You should be able to pound pretty hard with an 
LWP based pounder.

Ian

On Mon, 13 Nov 2000, siberian wrote:

> Under Win 2k Advanced Server using mod perl and mod proxy we get ok
> results in 'laboratory settings'. How that will translate in the real
> world is anyones guess, most likely poorly.
> 
> Thanks
> John- 
> 
> On Mon, 13 Nov 2000, Ian Struble wrote:
> 
> > You can still get alot out of a proxy if you have a win32 box doing 
> > heavyweight mod_perl stuff.  The only thing is that you need to have it 
> > on a different machine because mod_proxy doesn't hack it on a win32 
> > machine.  I'm sure that you could do it with something other that 
> > apache+mod_proxy if you wanted to keep it all on one machine.
> > 
> > Actually, I might have been having problems with mod_proxy because of 
> > problems in the tcp stack in SP4 that are now fixed in SP6.  So you 
> > might want to play around a little bit and see if it works.
> > 
> > Question for the list -- are we still limited to a single interpretter 
> > thread with mod_perl on win32?
> > 
> > Ian
> > 
> > 
> > On Mon, 13 Nov 2000, siberian wrote:
> > 
> > > I know I get a lot when I use a lightweight proxy in front of my modperl
> > > servers under UNIX but how about under Win32? Since it uses a different
> > > model does a
> > > reverse proxy really give you that warm and fuzzy feeling or does it just
> > > become another layer between the system and the user?
> > > 
> > > I am fairly ignorant of the way Win32 does its threading etc so I ask.
> > > 
> > > Thanks for any input
> > > 
> > > John Armstrong
> > > 
> > > 
> > 
> 
> 

Re: Win32 Proxy question

[Ian Struble]

You can still get alot out of a proxy if you have a win32 box doing 
heavyweight mod_perl stuff.  The only thing is that you need to have it 
on a different machine because mod_proxy doesn't hack it on a win32 
machine.  I'm sure that you could do it with something other that 
apache+mod_proxy if you wanted to keep it all on one machine.

Actually, I might have been having problems with mod_proxy because of 
problems in the tcp stack in SP4 that are now fixed in SP6.  So you 
might want to play around a little bit and see if it works.

Question for the list -- are we still limited to a single interpretter 
thread with mod_perl on win32?

Ian


On Mon, 13 Nov 2000, siberian wrote:

> I know I get a lot when I use a lightweight proxy in front of my modperl
> servers under UNIX but how about under Win32? Since it uses a different
> model does a
> reverse proxy really give you that warm and fuzzy feeling or does it just
> become another layer between the system and the user?
> 
> I am fairly ignorant of the way Win32 does its threading etc so I ask.
> 
> Thanks for any input
> 
> John Armstrong
> 
> 

Re: mod_perl rules! updated stats

[Ian Struble]

On Tue, 8 Aug 2000, Stas Bekman wrote:

> On 8 Aug 2000, Randal L. Schwartz wrote:
> 
> > > "Stas" == Stas Bekman <[EMAIL PROTECTED]> writes:
> > 
> > Stas> mod_perl rules :)
> > 
> > package Stonehenge::MyFixup;
> > 
> > ## PerlFixupHandler Stonehenge::MyFixup
> > 
> > use strict;
> > use vars qw($VERSION);
> > $VERSION = (qw$Revision: 1.6 $ )[-1];
> > 
> > use Apache::Constants qw(DECLINED);
> > 
> > sub handler {
> >   use Stonehenge::Reload; goto &handler if Stonehenge::Reload->reload_me;
> > 
> >   my $r = shift;
> > 
> > # [other stuff here deleted]
> > 
> >   if (not $r->proxyreq and $r->is_main) {
> > $r->header_out(X_mod_perl_rules =>
> >"mod_perl rules! Get it at http://perl.apache.org/");
> >   }
> > 
> >   return DECLINED;
> > }
> > 
> > 1;
> > 
> > :-)
> > 
> 
> Cool, if only people would look at the headers :)

Well I'm sure everyone here has once, twice, a thousand times.  I know 
that I would have gotten a laugh out of the headers had I seen that. :^)

Re: [OT] New element for CGI.pm (or StickyForms, etc.) (revised)

[Ian Struble]

I think that both the date_field and the time_field idea are good ones 
but how are you going to localize them?  The different formats for date 
and time might be a bit of a hassle.  But you might be able to have 
date_field return an array of 3 fields as is done with the check box 
constructor.  So you could use it like this and then arrange the date 
components however you want it:

  my ($year, $month, $day) = date_field(-name =>"end_date",
-value=>"2000-12-31")

  # I'm a farmer and I care most about the month...
  print "$month $day $year\n";


Ian

On Tue, 18 Jul 2000, Kenneth Lee wrote:

> Better still,
> 
>   print date_field(
> -name =>"expiry", 
> -value=>"2000-12-25");
> 
> so that if value is omitted, the current value will be used.
> But date format will be a great concern.
> 
> And how about time_field() also?
> 
> Sorry for the annoyance.
> 
> 
>  Original Message 
> Subject: [OT] New element for CGI.pm (or StickyForms, etc.)
> Date: Tue, 18 Jul 2000 17:45:21 +0800
> From: Kenneth Lee <[EMAIL PROTECTED]>
> To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
> 
> Hi all,
> 
> anyone thought of a 'date_field' element for CGI.pm (or StickyForms, etc.)?
> 
> this is my thought,
> 
>   use CGI qw/:standard/;
>   print date_field(
> -name =>"expiry", 
> -year =>2000, 
> -month=>12, 
> -day  =>25);
>   $expiry_date = param('expiry');
> 
> which is an elegant way to do
> 
>   use CGI qw/:standard/;
>   print textfield(-name=>'expiry_year', -value=>2000);
>   print popup_menu(-name=>'expiry_month', -values=>[1..12], -default=>12);
>   print popup_menu(-name=>'expiry_day', -values=>[1..31], -default=>25);
>   $expiry_date = sprintf "%d-%d-%d", param('expiry_year'), 
>   param('expiry_month'), param('expiry_day');
> 
> and the date validation can be done in query string parsing too.
> 
> Thanks for any input.
> Kenneth
> 

Re: Apache children hanging

[Ian Struble]

Someone just pointed out that this should probably go into the guide or 
FAQ somewhere.  Just a thought...

On Thu, 1 Jun 2000, Doug MacEachern wrote:

> > % gdb httpd $pid_of_spinning_process
> > % source modperl_x.xx/.gdbinit
> > % curinfo
> 
> oops, that should be:
> 
> % gdb httpd $pid_of_spinning_process
> (gdb) source modperl_x.xx/.gdbinit
> (gdb) curinfo
> 
> 
> 

Re: Apache children hanging

[Ian Struble]

Now if only I had known this two years ago...  Awsome tidbit though.  Thanks!

> you can find out which line of Perl code is triggering a spin, by
> attaching to the process with gdb;
> 
> % gdb httpd $pid_of_spinning_process
> % source modperl_x.xx/.gdbinit
> % curinfo
> 
> should show you the filename:line_number where Perl is stuck.
> 
> 

Re: Microsoft SQL 7.0 interface

[Ian Struble]

When you compile FreeTDS you will be better off using tds version 4.2 
unless you specifically need something in 7.0.

Here is the configure option you need for this:
--with-tdsver=VERSION

Ian


On Mon, 29 May 2000, Kee Hinckley wrote:

> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
> X-Spam-Rating: locus.apache.org 1.6.2 0/1000/N
> 
> At 5:45 PM -0500 5/26/00, Wang, Pin-Chieh wrote:
> >Any body knows how to access Microsoft SQL/on NT from Apache on Linux ?
> >Our data base is running on NT/SQL , but web server is running Apache/Linux
> 
> DBD::Sybase and FreeTDS will work (http://www.freetds.org/).
> 
> - -- 
> 
> Kee Hinckley - Somewhere Consulting Group - Cyberspace Architects(rm)
> 
> I'm not sure which upsets me more: that people are so unwilling to accept
> responsibility for their own actions, or that they are so eager to regulate
> everyone else's.
> 
> -BEGIN PGP SIGNATURE-
> Version: PGPfreeware 6.5.2 for non-commercial use 
> 
> iQA/AwUBOTH5aiZsPfdw+r2CEQKZ/ACdHfmeg1fmSfLlj5CiapCHXWF5vy8AoP7s
> nVY5U4aLCjUjbnib0uNRYSJ3
> =73+t
> -END PGP SIGNATURE-
> 

Re: [slightly OT] Problem with cookies

[Ian Struble]

On Thu, 6 Apr 2000, Perrin Harkins wrote:
> On Thu, 6 Apr 2000, Drew Taylor wrote:
> > I have a site which uses cookies for user tracking. If you go to
> > http://cloudstock.com/, the server is sending the cookie but the browser
> > is not accepting it ("warn before accepting cookie" is on). If I go to
> > http://www.cloudstock.com/ the cookie is sent & accepted. 
> 
> Does your Set-Cookie header include a path setting?  Some browsers require
> that.

If you don't have the 'path' set it may be defaulting to the directory of 
you request.  So either way(blank or /some/dir) you could have problems 
if you're not setting path=/.

-Ian

Re: Using network appliance Filer with modperl

[Ian Struble]

On Mon, 31 Jan 2000, Tim Bunce wrote:
> On Mon, Jan 31, 2000 at 11:16:23AM -0800, siberian wrote:
> > Hi All-
> > I am building a pretty in depth architecture for our new service
> > using ModPerl. I've done a lot of large scale/high traffic apps in modperl
> > before but never in conjunction with a network attached file server. I am
> > thinking that it would really make my life easy to have one central
> > repository of code, databases and sundry files that all the servers share
> > ( making it easy to swap out servers, add servers etc since its a central
> > file repository that everyone just hooks into ).
> > 
> > My question is : Has anyone experienced any 'gotchas' in putting perl code
> > that modperl handlers use on a Network Attached file server like a network
> > appliance box ( www.netapp.com )? I am assuming that there are no real
> > issues but before i go blow a ton of cash on this thing I wanted to be
> > sure that no one had found a problem.
> 
> And, just to be balanced, has anyone _not_ found any 'gotchas' and is
> enjoying life with a netapp or similar NFS file serving appliance?

I haven't really had any gotchas in terms of performance.  But you do 
have to plan things out if you are going to be working in a mixed 
NFS+CIFS environment because of permission issues.  Also I had a really 
hard time accessing a share with samba.  Supposedly that is fixed now 
but I have not had reason to test it.

Ian

OT: mod_proxy & socket error

[Ian Struble]

Hi all,

I have been getting the following error when I try to do some stress 
testing on a machine:

[error] [client 10.1.1.1] (55)No buffer space available: proxy: error
creating socket

Basically the box is just the front end proxy to a backend process that 
is running on another machine.  It is running FreeBSD 3.4-RELEASE.  I 
tweaked kern.ipc.maxsockbuf.  But since I am trying to fetch requests 
from a remote host increasing this setting did not help.  I am guessing 
that I am going to have to tweak some of the net.inet values to increase 
the buffers for the nic but I just can not seem to see the forest through 
the trees right now.  If anyone has already run into this or a similar 
problem in the past I would love to get the answser on a silver platter :^) 

Thanks and sorry for the slightly off topic post.  

Ian

Re: Apache locking up on WinNT

[Ian Struble]

With out going into too much detail I can tell you that you have probably
discovered for yourself that mod_perl on NT only has a single perl
interpreter thread.  Try putting a proxy in front of some backend mod_perl
procs so that when Joe super-slow-connection comes along he doesn't get to
tie up that thread.  But try to get the proxy on a unix box because when I
did this on an NT machine mod_proxy was a dog.  

If you are really stuck on NT you may want to mess around with 
ActiveState's PerlEx.  I have not done anything with it myself but I 
gather that it tries to do the same thing as mod_perl.  The only 
difference being that it perlforms alot better than mod_perl does on NT 
since it isn't crippled by the single interpreter issue.

Ian

On Mon, 17 Jan 2000, Matthew Robinson wrote:

> 
> I am currently in the process of transferring a database driven site from
> IIS to Apache on NT using mod_perl.  Apache seems to lock up after about
> 10-20 minutes and the only way to get things going again is to restart
> Apache (Apache is running from the console not as a service).
> 
> The site isn't particularly heavily loaded, currently handling a request
> every 5-10 seconds.
> 
> I have also noticed that on some occasions (after the lock up) the
> error.log contains an entry stating that one of my content handlers is not
> defined, the content handler works fine until this point.  I have checked
> the FAQ's etc and I am almost 100% certain that I don't have a problem with
> my namespace.
> 
> When the server locks up netstat lists a number of clients who have a
> TIME_WAIT status on port 8080 but these connections are not listed in
> /server-status.
> 
> I am using Apache/1.3.9 (Win32) with mod_perl/1.21 which I downloaded in
> December last year.  My worry is that there is a problem with Apache on NT
> and mod_perl, given that Apache on NT is multi-threaded.
> 
> Unfortunately, I am stuck in NT due to parts of the legacy system,
> otherwise I would move to Linux or FreeBSD.  If anyone can offer any
> suggestions I would be most grateful as the only alternative I have is to
> re-engineer the site in IIS.
> 
> If anyone has any suggestions, or would like further specific detail then
> please let me know.
> 
> Thanks
> 
> Matt
> 
> --
> Matthew RobinsonE: [EMAIL PROTECTED]
> Torrington Interactive Ltd  W: www.torrington.net
> 4 Printing House Yard   T: (44) 171 613 7200
> LONDON E2 7PR   F: (44) 171 613 7201
> 

Re: W32 + Apache::DBI ?

[Ian Struble]

> > Now all I have to do is rebuild perl with USE_THREADS.  I had someone else
> > play with getting mod_perl setup under NT and i don't think that they did
> > it quite right.  It seems like I only have one interpreter thread that is
> > being shared by all the different apache child threads.  Does this sound
> > like something that could happen if your perl binary didn't have threads
> > stuff compilled in?
> >
> 
> mod_perl under Win32 always just uses one interpreter, all perl requests are
> serialized. This has nothing to do with USE_THREADS. Nobody had made a
> threaded mod_perl so far and the thread support in perl 5.005 itself is
> experimetal (for example it has problems in regex's)
> 

*sigh*  Well at least I know now.  Thanks.

Ian

Re: W32 + Apache::DBI ?

[Ian Struble]

On Tue, 26 Oct 1999, Tim Bunce wrote:

> > 
> > Can't locate object method "trace_msg" via package "DBI" at
> > C:\Perl\site\5.00503\lib/DBI.pm line 311.
> > END failed--cleanup aborted.
> 
> That's a known bug that was fixed in DBI 1.10 (I believe).
> 

Just FYI, I had the problem in DBI 1.13.  It was the #define 
PERL_STARTUP_DONE_CHECK=1 that fixed my original problem with the 
start_mutex.

Ian

RE: W32 + Apache::DBI ?

[Ian Struble]

On Tue, 26 Oct 1999, Gerald Richter wrote:

> >
> > I have been digging around in the FAQ and archives for information about
> > people running mod_perl on a windows box and also using Apache::DBI, but
> > have come up with nothing.  I am under the impression that it is not
> > going to work becase whenever I try to load up the Apache::DBI module
> > apache starts up and then exists immediately leaving this in the
> > error log:
> >
> > [Mon Oct 25 15:06:11 1999] file .\main\http_main.c, line 5890, assertion
> > "start_mutex" failed
> >
> Build mod_perl with PERL_STARTUP_DONE_CHECK set (e.g. insert
> 
> #define PERL_STARTUP_DONE_CHECK 1
> 
> at the top of mod_perl.h or add it to the defines in MSVC++ Options dialog).

Thanks Gerald, this worked like a charm!  And thanks to everyone else for 
the input on this one.  It really helped ALOT!

Now all I have to do is rebuild perl with USE_THREADS.  I had someone else
play with getting mod_perl setup under NT and i don't think that they did
it quite right.  It seems like I only have one interpreter thread that is 
being shared by all the different apache child threads.  Does this sound 
like something that could happen if your perl binary didn't have threads 
stuff compilled in?

Ian

W32 + Apache::DBI ?

[Ian Struble]

Hi mod_perlers,

I have been digging around in the FAQ and archives for information about 
people running mod_perl on a windows box and also using Apache::DBI, but 
have come up with nothing.  I am under the impression that it is not 
going to work becase whenever I try to load up the Apache::DBI module 
apache starts up and then exists immediately leaving this in the error log:

[Mon Oct 25 15:06:11 1999] file .\main\http_main.c, line 5890, assertion 
"start_mutex" failed

Here is the section where that assertion is.  I have no idea what the 'Z' 
option is.  First time I have ever heard of it actually.

#ifdef WIN32
case 'Z':
exit_event = open_event(optarg);
APD2("child: opened process event %s", optarg);
cp = strchr(optarg, '_');
ap_assert(cp);
*cp = 0;
setup_signal_names(optarg);
start_mutex = ap_open_mutex(signal_name_prefix);
ap_assert(start_mutex);
child = 1;
break;

Does this mean that Apache::DBI is not thread safe and not going to be 
available to me in a Win32 environment?  And another followup question 
for someone who is as unlucky as me and doing stuff under NT,  is there 
only one interpreter thread available to all the 'child' threads?  It 
just seems like that based on the way that requests seem to queue up when 
a database request takes a little while to return.

Thanks for yur help,

Ian