Re: Using network appliance Filer with modperl
On Thu, 3 Feb 2000, Oleg Bartunov wrote:
[snipped]
> > > Stas, are you sure DESTROY executed when children died ?
> > > I'm using ApacheDBI and have problem with DESTROY when I use
> > > finish or disconnect methods.
> >
> > Of course! This works for me:
> > die.pl
> >
> > use MyRun;
> > print "Content-type: text/plain\n\n";
> > print "hi\n";
> > my $obj = new MyRun;
> > die "dying...";
> > print "End of program\n";
> >
> > MyRun.pm
> >
> > package MyRun;
> > sub new{ return bless {}, shift;}
> > DESTROY{ print STDERR "destructor was called\n";}
> > 1;
>
> Such simple DESTROY works for me also.
>
> >
> > The disconnect method shouldn't work as it gets overriden by Apache::DBI
> > with NOP. However finish() is supposed to work.
>
> I'm not sure it works:
>
> sub disconnect {
> my $self = shift;
> $self->sth_finish;
> warn "STH finished...\n";
> $self->{dbh}->disconnect;
> warn "DB disconnected...\n";
> }
>
> sub sth_finish {
> my $self = shift;
> foreach my $sth (keys %query) {
>$self->{$sth}->finish;
> }
> }
>
> sub DESTROY {
> my $self = shift;
> $self->disconnect;
> print STDERR "DB requests:PID:",$$,':', $self->total_db_requests(),"\n";
> }
> I never get debug messages in error log. But If I comment
> all calling of methods I got what I expected.
> sub disconnect {
> my $self = shift;
> # $self->sth_finish;
> warn "STH finished...\n";
> # $self->{dbh}->disconnect;
> warn "DB disconnected...\n";
> }
>
> sub DESTROY {
> my $self = shift;
> $self->disconnect;
> print STDERR "DB requests:PID:",$$,':', $self->total_db_requests(),"\n";
> }
>
> This behaivour doesn't depends whether or not I use ApacheDBI
So it's DBI. Did you try to add debug messages to DBI module? in finish
and disconnect? Actually in the appropriate DBD module of your db.
What are the results? Did you try to ask at the dbi-users mailing list?
Please summarize back to list or me, if you have solved it. This is an
important issue that has to be documented. Thanks!
> It seems DESTROY doesnt' executed when apache+mod_perl chidlren get
> killed :-)
It depends on how do you kill it. kill -9 is untrappable so you cannot do
a thing about it. If it's any other kill signal you can trap it with %SIG
and call the cleanup code. Does it help?
___
Stas Bekmanmailto:[EMAIL PROTECTED] http://www.stason.org/stas
Perl,CGI,Apache,Linux,Web,Java,PC http://www.stason.org/stas/TULARC
perl.apache.orgmodperl.sourcegarden.org perlmonth.comperl.org
single o-> + single o-+ = singlesheavenhttp://www.singlesheaven.com
Re: Using network appliance Filer with modperl
Hi, is I can step in here... ;-) At 15:11 2/3/00 +, Tim Bunce wrote: >> As a front-end we have 'cheap' PC's running Linux. The disks in the PC's >> are only used for the OS and temporary storage of logs, etc. >What level of web traffic are you handling 'from' the netapp? >E.g., how much traffic to the netapp is there when your web site >is getting peak traffic? As we put the maximum of RAM in our Linux boxes, in most cases we don't notice anything in the NetApp traffic when a site gets hit badly. For example, we host one of the Dutch National newspapers (http://www.nrc.nl) that way: because they come out with a daily edition around 4pm local time, traffic varies from about 300 Kbit/sec during the day to about 2Mbit/sec around the time the new update becomes available. However, we can't see anything special in the NetApp traffic graph at that time: it is all being served from the front-end server RAM. Since PC RAM is cheap, we can get a lot of mileage out of our NetApp. If we look at the total graph of NetApp traffic development of the past two years, that graph has only risen about 25% from the original average traffic. However, our web-traffic has quadrupled over that period, and the number of front-end servers now about 20 instead of the original 3. And the size of the NetApp has grown from 10 Gbyte to now about 45 Gbyte of diskspace. So I guess I would argue that maximum (relatively cheap) RAM in your front-end servers is much more important than the maximum NetApp bandwidth... Elizabeth Mattijsen Tel: 020-6005700Nieuwezijds Voorburgwal 68-70 Fax: 020-60018251012 SE AMSTERDAM Voor ernstige technische storingen zijn we buiten kantooruren bereikbaar: 06-29500176 of zie onze website. -- Web Ontwikkeling | Web Hosting | Web Onderhoud | Web Koppeling -- xxLINK, an Integra-Net company
Re: Using network appliance Filer with modperl
On Thu, Feb 03, 2000 at 01:01:43PM +0100, R. F. Lens wrote: > Hi, > > We've been running a modperl environment 'on' a NetApp since dec 1997 > and would't even dare to think about going back;). We've found no > gotcha's. Iff you can afford it I can really recommend it. The way we > use it is that we store all configs, libraries and sites on the netapp. > As a front-end we have 'cheap' PC's running Linux. The disks in the PC's > are only used for the OS and temporary storage of logs, etc. What level of web traffic are you handling 'from' the netapp? E.g., how much traffic to the netapp is there when your web site is getting peak traffic? Tim.
Re: Using network appliance Filer with modperl
Hi, We've been running a modperl environment 'on' a NetApp since dec 1997 and would't even dare to think about going back;). We've found no gotcha's. Iff you can afford it I can really recommend it. The way we use it is that we store all configs, libraries and sites on the netapp. As a front-end we have 'cheap' PC's running Linux. The disks in the PC's are only used for the OS and temporary storage of logs, etc. Ronald Tim Bunce wrote: > > On Mon, Jan 31, 2000 at 11:16:23AM -0800, siberian wrote: > > Hi All- > > I am building a pretty in depth architecture for our new service > > using ModPerl. I've done a lot of large scale/high traffic apps in modperl > > before but never in conjunction with a network attached file server. I am > > thinking that it would really make my life easy to have one central > > repository of code, databases and sundry files that all the servers share > > ( making it easy to swap out servers, add servers etc since its a central > > file repository that everyone just hooks into ). > > > > My question is : Has anyone experienced any 'gotchas' in putting perl code > > that modperl handlers use on a Network Attached file server like a network > > appliance box ( www.netapp.com )? I am assuming that there are no real > > issues but before i go blow a ton of cash on this thing I wanted to be > > sure that no one had found a problem. > > And, just to be balanced, has anyone _not_ found any 'gotchas' and is > enjoying life with a netapp or similar NFS file serving appliance? > > Tim. -- Ronald F. LensTel +31 (0)20 600 5700 xxLINK Internet Services Fax +31 (0)20 600 1825
Re: Using network appliance Filer with modperl
On Wed, 2 Feb 2000, Stas Bekman wrote:
> Date: Wed, 2 Feb 2000 22:03:38 +0200 (IST)
> From: Stas Bekman <[EMAIL PROTECTED]>
> To: Oleg Bartunov <[EMAIL PROTECTED]>
> Cc: mod_perl list <[EMAIL PROTECTED]>
> Subject: Re: Using network appliance Filer with modperl
>
> On Wed, 2 Feb 2000, Oleg Bartunov wrote:
>
> > On Wed, 2 Feb 2000, Stas Bekman wrote:
> >
> > > Date: Wed, 2 Feb 2000 20:21:20 +0200 (IST)
> > > From: Stas Bekman <[EMAIL PROTECTED]>
> > > To: Pascal Eeftinck <[EMAIL PROTECTED]>
> > > Cc: mod_perl list <[EMAIL PROTECTED]>
> > > Subject: Re: Using network appliance Filer with modperl
> > > # DESTROY makes all kinds of cleanups if the fuctions were interuppted
> > > # before their completion and haven't had a chance to make a clean up.
> > > ###
> > > sub DESTROY{
> > > my $self = shift;
> > >
> > > $self->sql_unlock_tables() if defined $self->{lock};
> > >
> > > $self->{sth}->finish if defined $self->{sth} and $self->{sth};
> > > $self->{dbh}->disconnect if defined $self->{dbh} and $self->{dbh};
> > >
> > > } # end of sub DESTROY
> >
> > Stas, are you sure DESTROY executed when children died ?
> > I'm using ApacheDBI and have problem with DESTROY when I use
> > finish or disconnect methods.
>
> Of course! This works for me:
> die.pl
>
> use MyRun;
> print "Content-type: text/plain\n\n";
> print "hi\n";
> my $obj = new MyRun;
> die "dying...";
> print "End of program\n";
>
> MyRun.pm
>
> package MyRun;
> sub new{ return bless {}, shift;}
> DESTROY{ print STDERR "destructor was called\n";}
> 1;
Such simple DESTROY works for me also.
>
> The disconnect method shouldn't work as it gets overriden by Apache::DBI
> with NOP. However finish() is supposed to work.
I'm not sure it works:
sub disconnect {
my $self = shift;
$self->sth_finish;
warn "STH finished...\n";
$self->{dbh}->disconnect;
warn "DB disconnected...\n";
}
sub sth_finish {
my $self = shift;
foreach my $sth (keys %query) {
$self->{$sth}->finish;
}
}
sub DESTROY {
my $self = shift;
$self->disconnect;
print STDERR "DB requests:PID:",$$,':', $self->total_db_requests(),"\n";
}
I never get debug messages in error log. But If I comment
all calling of methods I got what I expected.
sub disconnect {
my $self = shift;
# $self->sth_finish;
warn "STH finished...\n";
# $self->{dbh}->disconnect;
warn "DB disconnected...\n";
}
sub DESTROY {
my $self = shift;
$self->disconnect;
print STDERR "DB requests:PID:",$$,':', $self->total_db_requests(),"\n";
}
This behaivour doesn't depends whether or not I use ApacheDBI
Another question:
It seems DESTROY doesnt' executed when apache+mod_perl chidlren get killed :-)
Regards,
Oleg
>
> ___
> Stas Bekmanmailto:[EMAIL PROTECTED] http://www.stason.org/stas
> Perl,CGI,Apache,Linux,Web,Java,PC http://www.stason.org/stas/TULARC
> perl.apache.orgmodperl.sourcegarden.org perlmonth.comperl.org
> single o-> + single o-+ = singlesheavenhttp://www.singlesheaven.com
>
_
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: [EMAIL PROTECTED], http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83
Re: Using network appliance Filer with modperl
On Wed, Feb 02, 2000 at 08:21:20PM +0200, Stas Bekman wrote:
>
Oh, and while I'm being picky (nothing personal Stas!):
> $self->{sth}->finish if defined $self->{sth} and $self->{sth};
> $self->{dbh}->disconnect if defined $self->{dbh} and $self->{dbh};
There's no need to test for "defined" and "true" since undefined is false
and won't generate a warning when used in a boolean context. Thus just:
$self->{sth}->finish if $self->{sth};
$self->{dbh}->disconnect if $self->{dbh};
Tim:
Re: Using network appliance Filer with modperl
On Wed, 2 Feb 2000, Oleg Bartunov wrote:
> On Wed, 2 Feb 2000, Stas Bekman wrote:
>
> > Date: Wed, 2 Feb 2000 20:21:20 +0200 (IST)
> > From: Stas Bekman <[EMAIL PROTECTED]>
> > To: Pascal Eeftinck <[EMAIL PROTECTED]>
> > Cc: mod_perl list <[EMAIL PROTECTED]>
> > Subject: Re: Using network appliance Filer with modperl
> > # DESTROY makes all kinds of cleanups if the fuctions were interuppted
> > # before their completion and haven't had a chance to make a clean up.
> > ###
> > sub DESTROY{
> > my $self = shift;
> >
> > $self->sql_unlock_tables() if defined $self->{lock};
> >
> > $self->{sth}->finish if defined $self->{sth} and $self->{sth};
> > $self->{dbh}->disconnect if defined $self->{dbh} and $self->{dbh};
> >
> > } # end of sub DESTROY
>
> Stas, are you sure DESTROY executed when children died ?
> I'm using ApacheDBI and have problem with DESTROY when I use
> finish or disconnect methods.
Of course! This works for me:
die.pl
use MyRun;
print "Content-type: text/plain\n\n";
print "hi\n";
my $obj = new MyRun;
die "dying...";
print "End of program\n";
MyRun.pm
package MyRun;
sub new{ return bless {}, shift;}
DESTROY{ print STDERR "destructor was called\n";}
1;
The disconnect method shouldn't work as it gets overriden by Apache::DBI
with NOP. However finish() is supposed to work.
___
Stas Bekmanmailto:[EMAIL PROTECTED] http://www.stason.org/stas
Perl,CGI,Apache,Linux,Web,Java,PC http://www.stason.org/stas/TULARC
perl.apache.orgmodperl.sourcegarden.org perlmonth.comperl.org
single o-> + single o-+ = singlesheavenhttp://www.singlesheaven.com
Re: Using network appliance Filer with modperl
On Wed, Feb 02, 2000 at 08:21:20PM +0200, Stas Bekman wrote:
>
This thread has drifted way off topic, but anyway...
> my $do_sql = 'UNLOCK TABLES ';
> $self->{sth} = $self->{dbh}->prepare($do_sql);
> $self->{sth}->execute();
I'd express all that as:
$self->{dbh}->do("UNLOCK TABLES");
Tim.
Re: Using network appliance Filer with modperl
On Wed, 2 Feb 2000, Stas Bekman wrote:
> Date: Wed, 2 Feb 2000 20:21:20 +0200 (IST)
> From: Stas Bekman <[EMAIL PROTECTED]>
> To: Pascal Eeftinck <[EMAIL PROTECTED]>
> Cc: mod_perl list <[EMAIL PROTECTED]>
> Subject: Re: Using network appliance Filer with modperl
> # DESTROY makes all kinds of cleanups if the fuctions were interuppted
> # before their completion and haven't had a chance to make a clean up.
> ###
> sub DESTROY{
> my $self = shift;
>
> $self->sql_unlock_tables() if defined $self->{lock};
>
> $self->{sth}->finish if defined $self->{sth} and $self->{sth};
> $self->{dbh}->disconnect if defined $self->{dbh} and $self->{dbh};
>
> } # end of sub DESTROY
Stas, are you sure DESTROY executed when children died ?
I'm using ApacheDBI and have problem with DESTROY when I use
finish or disconnect methods.
Regards.
Oleg
>
>
>
>
> ___
> Stas Bekmanmailto:[EMAIL PROTECTED] http://www.stason.org/stas
> Perl,CGI,Apache,Linux,Web,Java,PC http://www.stason.org/stas/TULARC
> perl.apache.orgmodperl.sourcegarden.org perlmonth.comperl.org
> single o-> + single o-+ = singlesheavenhttp://www.singlesheaven.com
>
_
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: [EMAIL PROTECTED], http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83
Re: Using network appliance Filer with modperl
On Tue, 1 Feb 2000, Pascal Eeftinck wrote:
> At 11:25 1-2-2000 -0600, Leslie Mikesell wrote:
> >According to Elizabeth Mattijsen:
> >
> > > We have been using such a setup for over 2 years now. The only real issue
> > > we've found is not so much with mod_perl itself, but with MySQL. If you
> > > put your databases on the NetApp, either have a seperate central database
> > > server, or make damn sure you do not use the same database from two
> > > different front-end servers. We've seen database corruption that way
> > > (using Linux front-end servers with NFS 2).
> >
> >It is probably reasonable for MySQL to assume that only one server
> >is accessing the files at once since it has its own remote client
> >access protocol. Do you happen to know if there is a performance
> >difference for MySQL between local drives and a NetApp?
>
> All MySQL operations are atomic. You can always be sure that no two threads
> will be updating a table at the same time, for example. If you need to work
> on multiple tables while making sure data stays consistent among them, lock
> these tables before use.
>
> A single MySQL server will have no problems running with its databases
> mounted on an NFS store, although performance probably won't be very good.
> It shouldn't hurt too much for huge tables that you don't access that
> heavily, but those that you do use frequently had better be kept on a local
> disk. Of course you can just make symlinks in your var directory to the
> tables that you've stored on an NFS store somewhere.
>
> Multiple MySQL servers running on the same database stored on an NFS store
> is not something you should try at all. Threaded locking is unreliable by
> itself on Solaris and Linux if I read the MySQL docs correctly, and NFS
> locking is worse than that. You'd absolutely need a thread-safe flock()
> implementation that works over NFS as well - good luck with your quest. :)
>
>
> When you do use locking, be very very careful if you use Apache::DBI or
> similar persistent connections. MySQL threads keep tables locked until
> the thread ends (connection is closed) or the tables are unlocked. If your
> session die()'s while tables are locked, they will stay neatly locked as
> your connection won't be closed either This was a nasty one I bumped
> in to ...
Unless you write a clean code that worries to clean up no matter what
happens to the script. This is a snippet from my DB handling code. It's a
part of the bigger OO module, but notice the DESTROY subroutine that
worries to unlock the tables and disconnect if you aren't using
Apache::DBI (if you do it's just a NOP)
##
# lock the passed tables in the requested mode (READ|WRITE) and set
# internal flag to handle possible user abortions, so the tables will
# be unlocked thru the END{} block
#
# sql_lock_tables('table1','lockmode',..,'tableN','lockmode'
# lockmode = (READ | WRITE)
#
# _side_effect_ $self->{lock} = 'On';
#
##
sub sql_lock_tables{
my $self = shift;
my %modes = @_;
return unless %modes;
my $do_sql = 'LOCK TABLES ';
$do_sql .= join ",", map {"$_ $modes{$_}"} keys %modes;
$self->{sth} = $self->{dbh}->prepare($do_sql);
$self->{sth}->execute();
# Enough to set only one lock, unlock will remove them all
$self->{lock} = 'On';
} # end of sub sql_lock_tables
##
# unlock all tables, unset internal flag to handle possible user
# abortions, so the tables will be unlocked thru the END{} block
#
# sql_unlock_tables()
#
# _side_effect_: delete $self->{lock}
#
##
sub sql_unlock_tables{
my $self = shift;
my $do_sql = 'UNLOCK TABLES ';
$self->{sth} = $self->{dbh}->prepare($do_sql);
$self->{sth}->execute();
# Enough to set only one lock, unlock will remove them all
delete $self->{lock};
} # end of sub sql_unlock_tables
# DESTROY makes all kinds of cleanups if the fuctions were interuppted
# before their completion and haven't had a chance to make a clean up.
###
sub DESTROY{
my $self = shift;
$self->sql_unlock_tables() if defined $self->{lock};
$self->{sth}->finish if defined $self->{sth} and $self->{sth};
$self->{dbh}->disconnect if defined $self->{dbh} and $self->{dbh};
} # end of sub DESTROY
___
Stas Bekmanmailto:[EMAIL PROTECTED] http://www.stason.org/stas
Perl,CGI,Apache,Linux,Web,Java,PC http://www.stason.org/stas/TULARC
perl.apache.orgmodperl.sourcegarden.org perlmonth.comperl.org
single o-> + single o-+ = singlesheavenhttp://www.singlesheaven.com
Re: Using network appliance Filer with modperl
At 11:25 1-2-2000 -0600, Leslie Mikesell wrote: >According to Elizabeth Mattijsen: > > > We have been using such a setup for over 2 years now. The only real issue > > we've found is not so much with mod_perl itself, but with MySQL. If you > > put your databases on the NetApp, either have a seperate central database > > server, or make damn sure you do not use the same database from two > > different front-end servers. We've seen database corruption that way > > (using Linux front-end servers with NFS 2). > >It is probably reasonable for MySQL to assume that only one server >is accessing the files at once since it has its own remote client >access protocol. Do you happen to know if there is a performance >difference for MySQL between local drives and a NetApp? All MySQL operations are atomic. You can always be sure that no two threads will be updating a table at the same time, for example. If you need to work on multiple tables while making sure data stays consistent among them, lock these tables before use. A single MySQL server will have no problems running with its databases mounted on an NFS store, although performance probably won't be very good. It shouldn't hurt too much for huge tables that you don't access that heavily, but those that you do use frequently had better be kept on a local disk. Of course you can just make symlinks in your var directory to the tables that you've stored on an NFS store somewhere. Multiple MySQL servers running on the same database stored on an NFS store is not something you should try at all. Threaded locking is unreliable by itself on Solaris and Linux if I read the MySQL docs correctly, and NFS locking is worse than that. You'd absolutely need a thread-safe flock() implementation that works over NFS as well - good luck with your quest. :) When you do use locking, be very very careful if you use Apache::DBI or similar persistent connections. MySQL threads keep tables locked until the thread ends (connection is closed) or the tables are unlocked. If your session die()'s while tables are locked, they will stay neatly locked as your connection won't be closed either This was a nasty one I bumped in to ... Grtz, Pascal -- Pascal Eeftinck - arcade^planet.nl arcade^xs4all.nl - Perl is not a language, it's a way of life
Re: Using network appliance Filer with modperl
On Tue, Feb 01, 2000 at 10:29:30AM +0100, Elizabeth Mattijsen wrote: > At 11:16 1/31/00 -0800, siberian wrote: > >My question is : Has anyone experienced any 'gotchas' in putting perl code > >that modperl handlers use on a Network Attached file server like a network > >appliance box ( www.netapp.com )? I am assuming that there are no real > >issues but before i go blow a ton of cash on this thing I wanted to be > >sure that no one had found a problem. > > We have been using such a setup for over 2 years now. The only real issue > we've found is not so much with mod_perl itself, but with MySQL. If you > put your databases on the NetApp, either have a seperate central database > server, or make damn sure you do not use the same database from two > different front-end servers. We've seen database corruption that way > (using Linux front-end servers with NFS 2). The default linux MySQL has --skip-locking enabled by default so having two database servers access the same database tables would be very bad regardless of where the files lived. > With regards to fsck on large file systems: we've heard one horror stories > about that as well (with Xs4all here in Amsterdam). I recall having read > that they fixed the problem with the fsck taking very long on large > file-systems with the latest OnTap release. That's good news. Thanks. Tim.
Re: Using network appliance Filer with modperl
According to Elizabeth Mattijsen: > We have been using such a setup for over 2 years now. The only real issue > we've found is not so much with mod_perl itself, but with MySQL. If you > put your databases on the NetApp, either have a seperate central database > server, or make damn sure you do not use the same database from two > different front-end servers. We've seen database corruption that way > (using Linux front-end servers with NFS 2). It is probably reasonable for MySQL to assume that only one server is accessing the files at once since it has its own remote client access protocol. Do you happen to know if there is a performance difference for MySQL between local drives and a NetApp? Les Mikesell [EMAIL PROTECTED]
Re: Using network appliance Filer with modperl
Hi John, This is exactly the approach we use at boston.com, a large regional news site, serving roughly 3 million pages/day. For historical reasons, most of our web servers for serving static content are Netscape Enterprise servers, but we use modperl for all of our new development of dynamic sites (see http://mp3.boston.com/, the code for which we're planning on releasing as open source "real soon now".) We typically run the server root off of a local disk, to keep log traffic and other server accesses off the network. We have a shared document root that lives on the NetApp, all of the web servers read this. (The document root is mounted read/write, mostly because I didn't set it up originally. If I were setting this up from scratch, I'd probably try to have the web servers mount it readonly, with only the content upload hosts having write access.) We're running our perl include tree off of the NFS device also, which may or may not be a good idea, but it sure makes it easy to keep CPAN modules up to date. We've been extremely happy with the convenience and reliability of this setup. You *do* need to be aware of the differences in locking semantics across NFS devices. We're running mostly Solaris, and while I don't have any proof that it doesn't work, I have seen some strange conditions that lead me to believe that 'flock' doesn't always behave correctly across the NFS device. (Anyone who can provide proof otherwise, I'm all ears...) Atomic renames seem to behave as expected, so you can roll your own lockfile based mechanism if that's an issue, but we lately have taken to using MySQL to store shared data so that normal DB semantics can be used to coordinate distributed processes. Also, you need to worry a bit about OS versions of the hosts running your web servers -- binaries on the NFS device compiled for Solaris 7 aren't going to behave well if one of your web servers is running 2.51 (or even Linux!), for example. Good luck. On Mon, 31 Jan 2000, siberian wrote: > Hi All- > I am building a pretty in depth architecture for our new service > using ModPerl. I've done a lot of large scale/high traffic apps in modperl > before but never in conjunction with a network attached file server. I am > thinking that it would really make my life easy to have one central > repository of code, databases and sundry files that all the servers share > ( making it easy to swap out servers, add servers etc since its a central > file repository that everyone just hooks into ). > > My question is : Has anyone experienced any 'gotchas' in putting perl code > that modperl handlers use on a Network Attached file server like a network > appliance box ( www.netapp.com )? I am assuming that there are no real > issues but before i go blow a ton of cash on this thing I wanted to be > sure that no one had found a problem. > > Thanks for any pointers. > John Armstrong > =-=-=-=-=-=-=-=-=-=- My God! What have I done? -=-=-=-=-=-=-=-=-=-= Steve Reppucci [EMAIL PROTECTED] | Logical Choice Software http://logsoft.com/ | 508/958-0183 Be Open |
Re: Using network appliance Filer with modperl
At 11:16 1/31/00 -0800, siberian wrote: >My question is : Has anyone experienced any 'gotchas' in putting perl code >that modperl handlers use on a Network Attached file server like a network >appliance box ( www.netapp.com )? I am assuming that there are no real >issues but before i go blow a ton of cash on this thing I wanted to be >sure that no one had found a problem. We have been using such a setup for over 2 years now. The only real issue we've found is not so much with mod_perl itself, but with MySQL. If you put your databases on the NetApp, either have a seperate central database server, or make damn sure you do not use the same database from two different front-end servers. We've seen database corruption that way (using Linux front-end servers with NFS 2). With the regards of the single point of failure: only thing that failed so far was one fan, which could be replace without shutting the NetApp down. Also make sure that you go for the deal in which you get spare parts for just about everything, so that you can fix any hardware problems yourself very quickly. With regards to fsck on large file systems: we've heard one horror stories about that as well (with Xs4all here in Amsterdam). I recall having read that they fixed the problem with the fsck taking very long on large file-systems with the latest OnTap release. Elizabeth Mattijsen Tel: 020-6005700Nieuwezijds Voorburgwal 68-70 Fax: 020-60018251012 SE AMSTERDAM Voor ernstige technische storingen zijn we buiten kantooruren bereikbaar: 06-29500176 of zie onze website. -- Web Ontwikkeling | Web Hosting | Web Onderhoud | Web Koppeling -- xxLINK, an Integra-Net company
Re: Using network appliance Filer with modperl
According to Tim Bunce: > > > And, just to be balanced, has anyone _not_ found any 'gotchas' and is > > > enjoying life with a netapp or similar NFS file serving appliance? > > > > I haven't really had any gotchas in terms of performance. But you do > > have to plan things out if you are going to be working in a mixed > > NFS+CIFS environment because of permission issues. Also I had a really > > hard time accessing a share with samba. Supposedly that is fixed now > > but I have not had reason to test it. > > We wouldn't be using CIFS or samba. Just plan simple NFS file serving. I'm using one as a cvs repository and master copy that gets distributed via rsync to some other hosts but it isn't really serving the production hosts in real time yet. One thing I was hoping to do was to let it serve static content directly from my master image (you can get an http server also) but it keeps crashing when I try to give permission to a couple of subdirectories only and deny or issue a redirect on attempts to access anything else. This is probably my configuration error - I just haven't been sitting by a phone long enough to deal with a call to tech support recently... The thing doesn't seem especially fast at serving http but it doesn't slow down much with hundreds of concurrent requests either. Les Mikesell [EMAIL PROTECTED]
Re: Using network appliance Filer with modperl
siberian wrote: > > Hi All- > I am building a pretty in depth architecture for our new service > using ModPerl. I've done a lot of large scale/high traffic apps in modperl > before but never in conjunction with a network attached file server. I am > thinking that it would really make my life easy to have one central > repository of code, databases and sundry files that all the servers share > ( making it easy to swap out servers, add servers etc since its a central > file repository that everyone just hooks into ). Sounds like you are building in a single point of failure. If your NetApp goes down, everything goes down. Do you really have so much data that you can't easily synchronize the hard disks of several machines? If you do go with a NetApp, you need to think about it a lot. NFS has issues. For example, let's say that your NFS client software has a timeout period of 120 seconds. Let's also assume that each of your machines is serving 10 requests per second with 10 apache processes and is using half of its RAM. Suddenly, the NetApp fails. Horror of horrors! During your 120 seconds, 1200 requests queue up, which would require 1200 Apache processes to be blocked doing nothing. If your Apache is configured correctly, these people will wait forever for error messages. If Apache is not configured correctly, your machine will run out of swap and the kernel will panic. Neither of these is a good thing. Compare and contrast with a local disk failing on one server. The failure only affects a fraction of requests, and I/O errors on the local subsystem are usually quite quickly reported. The other thing to consider is that, if the NetApp goes down, it will be down for a long, long time. Their fsck isn't any faster than, for example, Linux's fsck. If the NetApp is very large, this could take hours. If you think the NetApp can't fail, I'll tell you that I've seen several failures in the real world. I've even seen them get so fucked up that they couldn't complete their fsck, after of course thrashing for 16 hours. Some of these machines had extremely demanding I/O loads, but others were simple file servers supporting a development organization. That's my take. -jwb > My question is : Has anyone experienced any 'gotchas' in putting perl code > that modperl handlers use on a Network Attached file server like a network > appliance box ( www.netapp.com )? I am assuming that there are no real > issues but before i go blow a ton of cash on this thing I wanted to be > sure that no one had found a problem. > > Thanks for any pointers. > John Armstrong
Re: Using network appliance Filer with modperl
On Mon, Jan 31, 2000 at 02:15:37PM -0800, Ian Struble wrote: > On Mon, 31 Jan 2000, Tim Bunce wrote: > > On Mon, Jan 31, 2000 at 11:16:23AM -0800, siberian wrote: > > > Hi All- > > > I am building a pretty in depth architecture for our new service > > > using ModPerl. I've done a lot of large scale/high traffic apps in modperl > > > before but never in conjunction with a network attached file server. I am > > > thinking that it would really make my life easy to have one central > > > repository of code, databases and sundry files that all the servers share > > > ( making it easy to swap out servers, add servers etc since its a central > > > file repository that everyone just hooks into ). > > > > > > My question is : Has anyone experienced any 'gotchas' in putting perl code > > > that modperl handlers use on a Network Attached file server like a network > > > appliance box ( www.netapp.com )? I am assuming that there are no real > > > issues but before i go blow a ton of cash on this thing I wanted to be > > > sure that no one had found a problem. > > > > And, just to be balanced, has anyone _not_ found any 'gotchas' and is > > enjoying life with a netapp or similar NFS file serving appliance? > > I haven't really had any gotchas in terms of performance. But you do > have to plan things out if you are going to be working in a mixed > NFS+CIFS environment because of permission issues. Also I had a really > hard time accessing a share with samba. Supposedly that is fixed now > but I have not had reason to test it. We wouldn't be using CIFS or samba. Just plan simple NFS file serving. Tim.
Re: Using network appliance Filer with modperl
On Mon, 31 Jan 2000, Tim Bunce wrote: > On Mon, Jan 31, 2000 at 11:16:23AM -0800, siberian wrote: > > Hi All- > > I am building a pretty in depth architecture for our new service > > using ModPerl. I've done a lot of large scale/high traffic apps in modperl > > before but never in conjunction with a network attached file server. I am > > thinking that it would really make my life easy to have one central > > repository of code, databases and sundry files that all the servers share > > ( making it easy to swap out servers, add servers etc since its a central > > file repository that everyone just hooks into ). > > > > My question is : Has anyone experienced any 'gotchas' in putting perl code > > that modperl handlers use on a Network Attached file server like a network > > appliance box ( www.netapp.com )? I am assuming that there are no real > > issues but before i go blow a ton of cash on this thing I wanted to be > > sure that no one had found a problem. > > And, just to be balanced, has anyone _not_ found any 'gotchas' and is > enjoying life with a netapp or similar NFS file serving appliance? I haven't really had any gotchas in terms of performance. But you do have to plan things out if you are going to be working in a mixed NFS+CIFS environment because of permission issues. Also I had a really hard time accessing a share with samba. Supposedly that is fixed now but I have not had reason to test it. Ian
Re: Using network appliance Filer with modperl
On Mon, Jan 31, 2000 at 11:16:23AM -0800, siberian wrote: > Hi All- > I am building a pretty in depth architecture for our new service > using ModPerl. I've done a lot of large scale/high traffic apps in modperl > before but never in conjunction with a network attached file server. I am > thinking that it would really make my life easy to have one central > repository of code, databases and sundry files that all the servers share > ( making it easy to swap out servers, add servers etc since its a central > file repository that everyone just hooks into ). > > My question is : Has anyone experienced any 'gotchas' in putting perl code > that modperl handlers use on a Network Attached file server like a network > appliance box ( www.netapp.com )? I am assuming that there are no real > issues but before i go blow a ton of cash on this thing I wanted to be > sure that no one had found a problem. And, just to be balanced, has anyone _not_ found any 'gotchas' and is enjoying life with a netapp or similar NFS file serving appliance? Tim.
Re: Using network appliance Filer with modperl
On Mon, 31 Jan 2000, Perrin Harkins wrote: > On Mon, 31 Jan 2000, siberian wrote: > > My question is : Has anyone experienced any 'gotchas' in putting perl code > > that modperl handlers use on a Network Attached file server like a network > > appliance box ( www.netapp.com )? > > Just make sure you have time synchronized between your > servers and the NetApp. Otherwise, things that use file > mod times (like Registry) will have strange behavior. and do everything you can to minimize nfs operations, cos if you run into locking issues you can hose every mod_perl host that's attached to your netapp. and only god knows what will happen when you reboot your netapp.
Re: Using network appliance Filer with modperl
siberian <[EMAIL PROTECTED]> wrote: >My question is : Has anyone experienced any 'gotchas' in putting perl code >that modperl handlers use on a Network Attached file server like a network >appliance box ( www.netapp.com )? I am assuming that there are no real >issues but before i go blow a ton of cash on this thing I wanted to be >sure that no one had found a problem. We don't have as large a setup as many places (we only have 3/4 Terabyte storage), but the biggest bottleneck I can see is network. The NetApps can handle multiple ethernets per IP interface and ATM, so with the proper networking, this wouldn't be too much of a problem. If the NetApp is serving both CIFS and NFS (to allow NT and Unix file serving), then make sure the NT setups are correct. It can have problems if mistakes are made in the NT area. Due to the performance issues and security, I would recommend a private switched network if going with ethernet. -- James Smith <[EMAIL PROTECTED]>, 409-862-3725 Texas A&M CIS Operating Systems Group, Unix Opinions are those of the author and not Texas A&M University.
Re: Using network appliance Filer with modperl
On Mon, 31 Jan 2000, siberian wrote: > My question is : Has anyone experienced any 'gotchas' in putting perl code > that modperl handlers use on a Network Attached file server like a network > appliance box ( www.netapp.com )? Just make sure you have time synchronized between your servers and the NetApp. Otherwise, things that use file mod times (like Registry) will have strange behavior. - Perrin
