Re: DESTROY not being called on stop/restart
On Tue, 2009-09-15 at 14:29 -0500, David Nicol wrote: I'm ignorant of internals specifics, but perl's full cleanup on exit needs to be called, and is called at an orderly shutdown. It doesn't get called at POSIX::exit, or when the process in which the interpreter is embedded exits without calling it. Presumably apache offers a hook for orderly shutdown and mod_perl takes advantage of that hook to run an orderly shutdown on the embedded interpreter. There is some danger of hanging on exit though, so there could be a good argument against not doing that by default. Graceful is supposed to do a stop/restart allowing requests to complete and so on, so it is easy to imagine that the hook might get run under a graceful but not under a stop, as stop is used to exit even when hung. I'm finding the same problem with a graceful-stop. It may be the nature of the code that I'm using, which does the following: To do a search: --- - check for the search in the cache - if result == $LOCK then go back to previous step - if no result - store $LOCK in cache - lock cache (using lexical object with DESTROY) - retrieve data from the DB - if data - store in cache - lock expires because of scope - DESTROY - if data loaded and stored to cache, then do nothing - else release lock - return data I'm testing it by: -- - locking DB search table with SQL - clearing the existing cache - start apache - make two requests for the same thing - first locks cache and tries to do query - second sees lock and waits - apachectl stop / restart / graceful-stop - apachectl start - retry one request This results in the last request hanging because the lock from the first request still hasn't been cleared. It may be because the code doesn't run to completion because the SQL query never receives a response, but I'd still have expected DESTROY to be called. The docs for mod_perl 1 imply that DESTROY blocks are called unless PERL_DESTRUCT_LEVEL is set to -1: http://perl.apache.org/docs/1.0/guide/troubleshooting.html#_warn__child_process_30388_did_not_exit__sending_another_SIGHUP This environment variable no longer seems to exist, and maybe behaviour has changed. Either way, it is a gotcha that should be documented (by somebody who understands what actually happens when) (... he says, neatly excluding himself :) For the record, the child exit handler is being called on stop/restart/graceful-stop and so handles this situation neatly. thanks Clint
mod_perl and Filter-Crypto
Hi everybody! I'm trying to use Filter-Crypto with mod_perl but with poor results. Everything works fine until I enable mod_perl. After this I'm getting errors like: Unrecognized character \\x1F at The line where it points is the begining of the encrypted part of file. Filter-Crypto change list (http://kobesearch.cpan.org/htdocs/Filter-Crypto/Changes.html) says, that some problems with mod_perl was solved a long time ago (Dec 2004) so I'm think there is a way to make mod_perl run soothly with Filter-Crypto. I'm using mod_perl 2.0.4, Active Perl 5.8.8 and Apache 2.2.8 under Windows. Regards, Jacek Pasternak
Re: Why people not using mod_perl
On Tue, 30 Nov 2004 22:38:11 + modperl[at]att.net wrote: 3) capacity/scalable mod_perl is very scalable --- I mean, one can properly config a single server to handle dynamic content for 200K daily unique IPs. PHP may end up with just 100K and servlet ends up at around 50K. I'm just curious, is this performance data still true in today? We have a new project building a website for a goverment which should handle lots of transportation data, servlet and modperl are two choices. So I googled and found this old message. http://www.gossamer-threads.com/lists/modperl/advocacy/75311
Ways to scale a mod_perl site
My algebra.com server serves about 77k pageviews and a little over a million objects requests per day (with half of it being served in just 4 hours). I peak out at 35 requests per second currently. I use mod_perl, mysql, and perlbal with everything running on one server. The server has a solid state disk to hold mysql data. I believe that it can handle 3x-5x more traffic all by itself. However, I am thinking of ways to scale up a mod_perl installation. 1) Use a load balancer like perlbal (I am already doing that) 2) Separate a MySQL database server from webservers. 3) Being enabled by item 2, add more webservers and balancers 4) Create a separate database for cookie data (Apache::Session objects) ??? -- not sure if good idea -- (next level) 5) Use a separate database handle for readonly database requests (SELECT), as opposed to INSERTS and UPDATEs. Use replication to access multiple slave servers for read only data, and only access the master for INSERT and UPDATE and DELETE. Any thoughts?
Re: Ways to scale a mod_perl site
On 09/16/2009 11:49 AM, Igor Chudov wrote: 1) Use a load balancer like perlbal (I am already doing that) A load balancer is good but so are proxies. If you can separate your application server from the server that servers static content then you'll get a boost even if they are on the same machine. 2) Separate a MySQL database server from webservers. This is probably the first and easiest thing you should do. 3) Being enabled by item 2, add more webservers and balancers 4) Create a separate database for cookie data (Apache::Session objects) ??? -- not sure if good idea -- I've never seen the need to do that. In fact, I would suggest you drop sessions altogether if you can. If you need any per-session information then put it in a cookie. If you need this information to be tamper-proof then you can create a hash of the cookie's data that you store as part of the cookie. If you can reduce the # of times that each request needs to actually hit the database you'll have big wins. 5) Use a separate database handle for readonly database requests (SELECT), as opposed to INSERTS and UPDATEs. Use replication to access multiple slave servers for read only data, and only access the master for INSERT and UPDATE and DELETE. Reducing DB usage is more important than this. Also, before you go down that road you should look at adding a caching layer to your application (memcached is a popular choice). -- Michael Peters Plus Three, LP
Re: Ways to scale a mod_perl site
On Wed, Sep 16, 2009 at 11:05 AM, Michael Peters mpet...@plusthree.comwrote: On 09/16/2009 11:49 AM, Igor Chudov wrote: 1) Use a load balancer like perlbal (I am already doing that) A load balancer is good but so are proxies. If you can separate your application server from the server that servers static content then you'll get a boost even if they are on the same machine. I have very little static content. Even images are generated. My site generates images of math formulae such as (x-1)/(x+1) on the fly., 2) Separate a MySQL database server from webservers. This is probably the first and easiest thing you should do. agreed 3) Being enabled by item 2, add more webservers and balancers 4) Create a separate database for cookie data (Apache::Session objects) ??? -- not sure if good idea -- I've never seen the need to do that. In fact, I would suggest you drop sessions altogether if you can. If you need any per-session information then put it in a cookie. If you need this information to be tamper-proof then you can create a hash of the cookie's data that you store as part of the cookie. If you can reduce the # of times that each request needs to actually hit the database you'll have big wins. I use sessions to keep users logged on. So the cookie is just an ID, and the sessions table stores data such as authenticated userid. I will double check, however, whether I give people sessions even if they are not logged in. Or maybe I can give them a cookie that will say I am not logged in, do not bother looking up my session. Hm 5) Use a separate database handle for readonly database requests (SELECT), as opposed to INSERTS and UPDATEs. Use replication to access multiple slave servers for read only data, and only access the master for INSERT and UPDATE and DELETE. Reducing DB usage is more important than this. Also, before you go down that road you should look at adding a caching layer to your application (memcached is a popular choice). It is not going to be that helpful due to dynamic content. (which is my site's advantage). But this may be useful for other applications. i
Re: Ways to scale a mod_perl site
On Wed, Sep 16, 2009 at 11:15 AM, C. J. L. wrote I would buy a fast server with 4 or more cpu cores and the SSD or SAS drives and run the backend db on a dedicated mysql instance. By the way, guys, the performance difference between a regular SATA drive and a fast SAS drive is comparatively small. The difference between a SAS drive and an SSD drive is tremendous. i
Re: Ways to scale a mod_perl site
Igor Chudov wrote: On Wed, Sep 16, 2009 at 11:05 AM, Michael Peters mpet...@plusthree.com mailto:mpet...@plusthree.com wrote: On 09/16/2009 11:49 AM, Igor Chudov wrote: 1) Use a load balancer like perlbal (I am already doing that) A load balancer is good but so are proxies. If you can separate your application server from the server that servers static content then you'll get a boost even if they are on the same machine. I have very little static content. Even images are generated. My site generates images of math formulae such as (x-1)/(x+1) on the fly., I can understand generating them on the fly for flexibility reasons, but I'd cache them, and serve them statically after that, rather than regenerate the images on every single request. You can accomplish that in the app itself, or just by throwing a caching proxy in front of it (maybe you're already doing this with perlbal) Adam
Re: Ways to scale a mod_perl site
On Wed, Sep 16, 2009 at 11:48 AM, Adam Prime adam.pr...@utoronto.ca wrote: Igor Chudov wrote I have very little static content. Even images are generated. My site generates images of math formulae such as (x-1)/(x+1) on the fly., I can understand generating them on the fly for flexibility reasons, but I'd cache them, and serve them statically after that, rather than regenerate the images on every single request. You can accomplish that in the app itself, or just by throwing a caching proxy in front of it (maybe you're already doing this with perlbal) I actually do cache generated pictures, I store them in a database table called 'bincache'. This way I do not have to compute and draw every image on the fly. If I have a picture in bincache, I serve it, and if I do not, I generate it and save it. That saves some CPU, but makes mysql work harder. i
Re: Ways to scale a mod_perl site
On 09/16/2009 12:13 PM, Brad Van Sickle wrote:
Can I get you to explain this a little more? I don't see how this could
be used for truly secure sites because I don't quite understand how
storing a hash in a plain text cookie would be secure.
If you need to store per-session data about a client that the client
shouldn't be able to see, then you just encrypt that data, base-64
encode it and then put it into a cookie.
If you don't care if the user sees that information you just want to
make sure that they don't change it then add an extra secure hash of
that information to the cookie itself and then verify it when you
receive it.
I like to use JSON for my cookie data because it's simple and fast, but
any serializer should work. Something like this:
use JSON qw(to_json from_json);
use Digest::MD5 qw(md5_hex);
use MIME::Base64::URLSafe qw(urlsafe_b64encode urlsafe_b64decode);
# to generate the cookie
my %data = ( foo = 1, bar = 2, baz = 'frob' );
$data{secure} = generate_data_hash(\%data);
my $cookie = urlsafe_b64encode(to_json(\%data));
print Cookie: $cookie\n;
# to process/validate the cookie
my $new_data = from_json(urlsafe_b64decode($cookie));
my $new_hash = delete $new_data-{secure};
if( $new_hash eq generate_data_hash($new_data) ) {
print Cookie is ok!\n;
} else {
print Cookie has been tampered with! Ignore.\n;
}
# very simple hash generation function
sub generate_data_hash {
my $data = shift;
my $secret = 'some configured secret';
return md5_hex($secret . join('|', map { $_ - $data-{$_} } keys
%$data));
}
Doing encryption and encoding on small bits of data (like cookies) in
memory will almost always be faster than having to hit the database
(especially if it's on another machine). But the biggest reason is that
it takes the load off the DB and puts it on the web machines which are
much easier to scale linearly.
I know a lot of true app servers (Websphere, etc..) store
this data in cached memory,
You could do the same with your session data, or even store it in a
shared resource like a BDB file. But unless it's available to all of
your web servers you're stuck with sticky sessions and that's a real
killer for performance/scalability.
--
Michael Peters
Plus Three, LP
Re: Ways to scale a mod_perl site
On 09/16/2009 12:48 PM, Adam Prime wrote: I have very little static content. Even images are generated. My site generates images of math formulae such as (x-1)/(x+1) on the fly., I can understand generating them on the fly for flexibility reasons, but I'd cache them, and serve them statically after that, rather than regenerate the images on every single request. Definitely good advice. Especially if your images are generated the same each time and never change. For instance, I don't think the image generated by the formula (x-1)/(x+1) would ever change (unless you changed your application code and in that case you can clear you cache). -- Michael Peters Plus Three, LP
Re: Ways to scale a mod_perl site
On 09/16/2009 01:02 PM, Igor Chudov wrote: I actually do cache generated pictures, I store them in a database table called 'bincache'. This way I do not have to compute and draw every image on the fly. If I have a picture in bincache, I serve it, and if I do not, I generate it and save it. That saves some CPU, but makes mysql work harder. Then don't put it in your database. A cache is not a permanent store and it's usage patterns will be different than a database. I'd either use a real cache like memcached or have your proxies cache them. In addition to that you can send the appropriate HTTP cache headers so that browsers themselves will never request that image again. Make the client machine do the caching. -- Michael Peters Plus Three, LP
Re: Ways to scale a mod_perl site
I'm curious... what is the hardware like on the one server? How many CPUs and RAM? Also, a few thoughts... - You do a 301 from algebra.com to www.algebra.com. That doesn't take much work from the server, but why not just serve up everything from the original location? - The algebra problem I just tried returned twelve separate images. What if, instead of serving gifs you displayed each stage of transformation of the equation using HTML and CSS? That would be rather tricky with things like root signs but I think it could be done - though a bit of work. I wish this site had been around when I was in high school. On Wed, Sep 16, 2009 at 11:48 AM, Adam Prime adam.pr...@utoronto.ca wrote: Igor Chudov wrote: On Wed, Sep 16, 2009 at 11:05 AM, Michael Peters mpet...@plusthree.commailto: mpet...@plusthree.com wrote: On 09/16/2009 11:49 AM, Igor Chudov wrote: 1) Use a load balancer like perlbal (I am already doing that) A load balancer is good but so are proxies. If you can separate your application server from the server that servers static content then you'll get a boost even if they are on the same machine. I have very little static content. Even images are generated. My site generates images of math formulae such as (x-1)/(x+1) on the fly., I can understand generating them on the fly for flexibility reasons, but I'd cache them, and serve them statically after that, rather than regenerate the images on every single request. You can accomplish that in the app itself, or just by throwing a caching proxy in front of it (maybe you're already doing this with perlbal) Adam
Re: Ways to scale a mod_perl site
Guys, I completely love this discussion about cookies. You have really enlightened me. I think that letting users store cookie info in a manner that is secure (involves both encryption and some form of authentication), instead of storing them in a table, could possibly result in a very substantial reduction of database use. The cookie is 1) Encrypted string that I want and 2) MD5 of that string with a secret code appended that the users do not know, which serves as a form of signing That should work. I will not change it now, but will do if I get 2x more traffic. That way I would need zero hits to the database to handle my users sessions. (I only retrieve account information when necessary) As far as I remember now, I do not store much more information in a session beyond username. (I hope that I am not wrong). So it should be easy. Even now, I make sure that I reset the cookie table only every several months. This way I would let users stay logged on forever. Thanks a lot. Igor
Re: Ways to scale a mod_perl site
On Wed, Sep 16, 2009 at 12:21 PM, Douglas Sims ratsb...@gmail.com wrote: I'm curious... what is the hardware like on the one server? How many CPUs and RAM? AMD Athlon quad core, running 32 bit Ubuntu Hardy. 16 GB of RAM. Algebra.Com data is stored on an SSD Also, a few thoughts... - You do a 301 from algebra.com to www.algebra.com. That doesn't take much work from the server, but why not just serve up everything from the original location? then I will have to serve algebra.com twice to all search engines. - The algebra problem I just tried returned twelve separate images. What if, instead of serving gifs you displayed each stage of transformation of the equation using HTML and CSS? That would be rather tricky with things like root signs but I think it could be done - though a bit of work. I rather like the way I do it, I let my site render images exactly how I want, as opposed to letting browsers do it. I wish this site had been around when I was in high school. thanks. I have some real math addicts on my site, who solved many thousands of problems and helped hundreds of kids. I am glad to serve them. i On Wed, Sep 16, 2009 at 11:48 AM, Adam Prime adam.pr...@utoronto.cawrote: Igor Chudov wrote: On Wed, Sep 16, 2009 at 11:05 AM, Michael Peters mpet...@plusthree.commailto: mpet...@plusthree.com wrote: On 09/16/2009 11:49 AM, Igor Chudov wrote: 1) Use a load balancer like perlbal (I am already doing that) A load balancer is good but so are proxies. If you can separate your application server from the server that servers static content then you'll get a boost even if they are on the same machine. I have very little static content. Even images are generated. My site generates images of math formulae such as (x-1)/(x+1) on the fly., I can understand generating them on the fly for flexibility reasons, but I'd cache them, and serve them statically after that, rather than regenerate the images on every single request. You can accomplish that in the app itself, or just by throwing a caching proxy in front of it (maybe you're already doing this with perlbal) Adam
Re: Ways to scale a mod_perl site
On Wed, Sep 16, 2009 at 11:49 AM, Igor Chudov ichu...@gmail.com wrote: Any thoughts? In addition to the good advice you're getting on the thread, here are some books you might find useful: - Practical mod_perl -- http://modperlbook.org/ -- is old, but has a lot of general architecture and tuning advice that really hasn't changed much since then. - High-Performance MySQL, the best book available on MySQL tuning. - Building Scalable Websites, which is about PHP sites, but has good food for thought. - Scalable Internet Architectures, a book that is more about general principles to apply to the problem. And, the most important piece of advice: Devel::NYTProf. Happy tuning, Perrin
Re: Ways to scale a mod_perl site
Perrin, thanks a lot. I bought all books recommended below. Should be a good read. I want to be ready when the need arises, and I do not want to do anything stupid in the meantime that would make me not scalable. Again, thank you. Igor On Wed, Sep 16, 2009 at 1:12 PM, Perrin Harkins phark...@gmail.com wrote: On Wed, Sep 16, 2009 at 11:49 AM, Igor Chudov ichu...@gmail.com wrote: Any thoughts? In addition to the good advice you're getting on the thread, here are some books you might find useful: - Practical mod_perl -- http://modperlbook.org/ -- is old, but has a lot of general architecture and tuning advice that really hasn't changed much since then. - High-Performance MySQL, the best book available on MySQL tuning. - Building Scalable Websites, which is about PHP sites, but has good food for thought. - Scalable Internet Architectures, a book that is more about general principles to apply to the problem. And, the most important piece of advice: Devel::NYTProf. Happy tuning, Perrin
Re: Why people not using mod_perl
On Wed, Sep 16, 2009 at 4:52 AM, Jeff Pang pa...@vfemail.net wrote: I'm just curious, is this performance data still true in today? We have a new project building a website for a goverment which should handle lots of transportation data, servlet and modperl are two choices. I don't know what the source of that data was. However, mod_perl is basically just Perl, and Perl is very fast. In most of the language benchmarks I've seen, Perl comes out a little ahead of PHP and somewhat behind Java. In real-world websites though, Perl often ends up being faster than Java because of slow Java web frameworks and the overly-abstract designs they encourage. You can certainly succeed at building large websites with either Perl or Java. I'd suggest you consider who will be doing the work and what the expenses will be. If you decide to use Java, go with open source. The commercial frameworks are slow and not worth the price. - Perrin
Re: Ways to scale a mod_perl site
Igor Chudov ichu...@gmail.com writes: My algebra.com server serves about 77k pageviews and a little over a million objects requests per day (with half of it being served in just 4 hours). I peak out at 35 requests per second currently. Some high-level advice: Profile everything you can to see where your bottlenecks are. If you don't have bottlenecks, simulate enough load that you do. I am frequently surprised by what turn out to be the slow parts of my code. At a high-level, you can use tools like top, vmstat, iostat, iotop, etc. to check whether it's CPU, memory, or disk space that you're running out of first. For CPU, you can use top to see which process is using most of your CPU, the database, app, or something else. Inside your app, you can use Perl's profiling tools to see which parts of your app need to be sped up. Hope this is helpful! Scott.
Re: Why people not using mod_perl
My site algebra.com is about 80,000 lines of mod_perl code. I wrote a relatively large framework, with many homegrown perl modules, about five years ago. It uses a database, image generation modules, a big mathematical engine that I wrote (that shows work, unlike popular third party packages), etc. All pages of my site are dynamic and it is very image heavy due to math formulae. I can say two things: 1) It is relatively fast, serving pages in 0.1 seconds or so 2) Despite the quantity of code, and its age, it is still very maintainable and understandable (to me). If I was to make a choice again, I would go with mod_perl again. With Perl, I can stand on the shoulders of giants like Lincoln etc, and use the brilliant stuff they provided to serve my users. i
Re: Why people not using mod_perl
--- On Thu, 9/17/09, Igor Chudov ichu...@gmail.com wrote: My site algebra.com is about 80,000 lines of mod_perl code. I wrote a relatively large framework, with many homegrown perl modules, about five years ago. It uses a database, image generation modules, a big mathematical engine that I wrote (that shows work, unlike popular third party packages), etc. All pages of my site are dynamic and it is very image heavy due to math formulae. I can say two things: 1) It is relatively fast, serving pages in 0.1 seconds or so 2) Despite the quantity of code, and its age, it is still very maintainable and understandable (to me). In that case, would you like to fix its mangled output? e.g. http://www.algebra.com/algebra/homework/divisibility/Prime_factorization_algorithm.wikipedia   (Redirected from Prime factorization algorithm) faster than O((1+ε)b) for all positive ε an integer M with 1 ≤ M ≤ N Pollard's p − 1 algorithm Section 4.5.4: Factoring into Primes, pp. 379–417. Chapter 5: Exponential Factoring Algorithms, pp. 191–226. Chapter 6: Subexponential Factoring Algorithms, pp. 227–284. Section 7.4: Elliptic curve method, pp. 301–313. Eric W. Weisstein, “RSA-640 Factored†v • d • e AKS · APR · Ballie–PSW · ECPP · Fermat · Lucas · Lucas–Lehmer · Lucas–Lehmer–Riesel · Proth's theorem · Pépin's · Solovay–Strassen · Miller–Rabin · Trial division Sieve of Atkin · Sieve of Eratosthenes · Sieve of Sundaram · Wheel factorization CFRAC · Dixon's · ECM · Euler's · Pollard's rho · P − 1 · P + 1 · QS · GNFS · SNFS · rational sieve · Fermat's · Shanks' square forms · Trial division · Shor's Ancient Egyptian multiplication · Aryabhata · Binary GCD · Chakravala · Euclidean · Extended Euclidean · integer relation algorithm · integer square root · Modular exponentiation · Schoof's · Shanks-Tonelli Looks like you've got utf8 and iso8859-1 messed up. Phil
Re: Why people not using mod_perl
This is a mod_perl list, so I would expect to see Perl championed pretty heavily, but Java, .net and there ilk are undoubtedly *the* choice for large web applications. I'd like to get into some discussion as to why almost all *large* sites choose these languages. I don't have any experience developing a large application in Java, although I do have a lot of experience working on the operations side of a large web application that is Java based. The reasons I generally hear for choosing Java over mod_perl are: 1) Speed - I don't buy this at all 2) Maintainability - I think this makes sense. Perl can be pretty easy to maintain if you stick a good framework around it, but you have to seek out that framework and YOU are responsible for adhereing to it. All of that is inherent in Java. It also helps that Java has OO built in. 3) Easier to package and build/move code - In my experience this is true. 4) Advantages to be gained from running on an actually application server - Also valid 5) Compatible enterprise class middleware - Also true, Java plugs into more truly enterprise level suff than Perl does. (security frameworks, etc... ) 6) Support A lot of the industry seems look at Perl as obsolete technology that has been replaced by *insert hot new technology of the week here* which is a total shame. I've worked with a lot of technologies and I think Perl is a great choice for small/medium websites and webapps, which is probably what most of us work on. But I'm very interested to know at what point (if any) a site/app grows too large or too complex for mod_perl and what defines that turning point. Could Amazon run on mod_perl for example? Phil Carmody wrote: --- On Thu, 9/17/09, Igor Chudov ichu...@gmail.com wrote: My site algebra.com is about 80,000 lines of mod_perl code. I wrote a relatively large framework, with many homegrown perl modules, about five years ago. It uses a database, image generation modules, a big mathematical engine that I wrote (that shows work, unlike popular third party packages), etc. All pages of my site are dynamic and it is very image heavy due to math formulae. I can say two things: 1) It is relatively fast, serving pages in 0.1 seconds or so 2) Despite the quantity of code, and its age, it is still very maintainable and understandable (to me). In that case, would you like to fix its mangled output? e.g. http://www.algebra.com/algebra/homework/divisibility/Prime_factorization_algorithm.wikipedia   (Redirected from Prime factorization algorithm) faster than O((1+ε)b) for all positive ε an integer M with 1 ≤ M ≤ N Pollard's p − 1 algorithm Section 4.5.4: Factoring into Primes, pp. 379–417. Chapter 5: Exponential Factoring Algorithms, pp. 191–226. Chapter 6: Subexponential Factoring Algorithms, pp. 227–284. Section 7.4: Elliptic curve method, pp. 301–313. Eric W. Weisstein, “RSA-640 Factored†v • d • e AKS · APR · Ballie–PSW · ECPP · Fermat · Lucas · Lucas–Lehmer · Lucas–Lehmer–Riesel · Proth's theorem · Pépin's · Solovay–Strassen · Miller–Rabin · Trial division Sieve of Atkin · Sieve of Eratosthenes · Sieve of Sundaram · Wheel factorization CFRAC · Dixon's · ECM · Euler's · Pollard's rho · P − 1 · P + 1 · QS · GNFS · SNFS · rational sieve · Fermat's · Shanks' square forms · Trial division · Shor's Ancient Egyptian multiplication · Aryabhata · Binary GCD · Chakravala · Euclidean · Extended Euclidean · integer relation algorithm · integer square root · Modular exponentiation · Schoof's · Shanks-Tonelli Looks like you've got utf8 and iso8859-1 messed up. Phil
Re: Why people not using mod_perl
On Thu, Sep 17, 2009 at 2:32 AM, Perrin Harkins phark...@gmail.com wrote: I don't know what the source of that data was. However, mod_perl is basically just Perl, and Perl is very fast. I think the more exact statement should be, mod_perl is compiled perl, mod_perl is very fast. But perl CGI...I must say it's very slow. Jenn.
Re: Why people not using mod_perl
On Wed, Sep 16, 2009 at 9:42 PM, Jenn G. practicalp...@gmail.com wrote: I think the more exact statement should be, mod_perl is compiled perl, mod_perl is very fast. But perl CGI...I must say it's very slow. Well, you can say CGI is slow, but Perl CGI is very fast compared to the alternatives. Have you ever tried Java CGI? Or PHP CGI? They're not fast. Also, I don't like to tell people that mod_perl is compiled because it's really no more compiled than any other perl script. If you want to be precise, you could say mod_perl is a persistent daemon for running perl code, just like servlets are a persistent daemon for running Java. - Perrin
Re: Why people not using mod_perl
On Thu, Sep 17, 2009 at 10:05 AM, Perrin Harkins phark...@gmail.com wrote: On Wed, Sep 16, 2009 at 9:42 PM, Jenn G. practicalp...@gmail.com wrote: I think the more exact statement should be, mod_perl is compiled perl, mod_perl is very fast. But perl CGI...I must say it's very slow. Well, you can say CGI is slow, but Perl CGI is very fast compared to the alternatives. Have you ever tried Java CGI? Or PHP CGI? They're not fast. but nobody run Java or PHP as CGI. the only thing I heard is somebody run php as fastcgi under lighttpd. Also, I don't like to tell people that mod_perl is compiled because it's really no more compiled than any other perl script. mod_perl loads and compiles perl scripts only once, but CGI loads and compiles them every time for each request. Am I right? thanks.
Re: Why people not using mod_perl
Doesn't Amazon run mod_perl/Mason? BTW, I agree with most of your points (would debate #4,5). I may substitute the phrase More convenient for Easier in #3. I would also add ... #7) How many engineers are available to hire that know or want to work with said technology? I built a great platform at eBay on mod_perl/Mason that handled eBay-size traffic; we ran 6 eBay sites on it. Now it is used for specialty e-commerce solutions like worldofgood.ebay.com, global.ebay.com (cross-border trade), dealfinder.ebay.com, etc. In fact, on the same hardware, the main eBay Java app would support ~6 threads per box; the mod_perl platform supported ~60 (prefork), significant CapEx and power savings (which adds up at a place like eBay). From: Brad Van Sickle bvs7...@gmail.com To: mod_perl list modperl@perl.apache.org Sent: Wednesday, September 16, 2009 3:31:30 PM Subject: Re: Why people not using mod_perl This is a mod_perl list, so I would expect to see Perl championed pretty heavily, but Java, .net and there ilk are undoubtedly *the* choice for large web applications. I'd like to get into some discussion as to why almost all *large* sites choose these languages. I don't have any experience developing a large application in Java, although I do have a lot of experience working on the operations side of a large web application that is Java based. The reasons I generally hear for choosing Java over mod_perl are: 1) Speed - I don't buy this at all 2) Maintainability - I think this makes sense. Perl can be pretty easy to maintain if you stick a good framework around it, but you have to seek out that framework and YOU are responsible for adhereing to it. All of that is inherent in Java. It also helps that Java has OO built in. 3) Easier to package and build/move code - In my experience this is true. 4) Advantages to be gained from running on an actually application server - Also valid 5) Compatible enterprise class middleware - Also true, Java plugs into more truly enterprise level suff than Perl does. (security frameworks, etc... ) 6) Support A lot of the industry seems look at Perl as obsolete technology that has been replaced by *insert hot new technology of the week here* which is a total shame. I've worked with a lot of technologies and I think Perl is a great choice for small/medium websites and webapps, which is probably what most of us work on. But I'm very interested to know at what point (if any) a site/app grows too large or too complex for mod_perl and what defines that turning point. Could Amazon run on mod_perl for example? Phil Carmody wrote: --- On Thu, 9/17/09, Igor Chudov ichu...@gmail.com wrote: My site algebra.com is about 80,000 lines of mod_perl code. I wrote a relatively large framework, with many homegrown perl modules, about five years ago. It uses a database, image generation modules, a big mathematical engine that I wrote (that shows work, unlike popular third party packages), etc. All pages of my site are dynamic and it is very image heavy due to math formulae. I can say two things: 1) It is relatively fast, serving pages in 0.1 seconds or so 2) Despite the quantity of code, and its age, it is still very maintainable and understandable (to me). In that case, would you like to fix its mangled output? e.g. http://www.algebra.com/algebra/homework/divisibility/Prime_factorization_algorithm.wikipedia   (Redirected from Prime factorization algorithm) faster than O((1+ε)b) for all positive ε an integer M with 1 ≤ M ≤ N Pollard's p − 1 algorithm Section 4.5.4: Factoring into Primes, pp. 379–417. Chapter 5: Exponential Factoring Algorithms, pp. 191–226. Chapter 6: Subexponential Factoring Algorithms, pp. 227–284. Section 7.4: Elliptic curve method, pp. 301–313. Eric W. Weisstein, “RSA-640 Factored†v • d • e AKS · APR · Ballie–PSW · ECPP · Fermat · Lucas · Lucas–Lehmer · Lucas–Lehmer–Riesel · Proth's theorem · Pépin's · Solovay–Strassen · Miller–Rabin · Trial division Sieve of Atkin · Sieve of Eratosthenes · Sieve of Sundaram · Wheel factorization CFRAC · Dixon's · ECM · Euler's · Pollard's rho · P − 1 · P + 1 · QS · GNFS · SNFS · rational sieve · Fermat's · Shanks' square forms · Trial division · Shor's Ancient Egyptian multiplication · Aryabhata · Binary GCD · Chakravala · Euclidean · Extended Euclidean · integer relation algorithm · integer square root · Modular exponentiation · Schoof's · Shanks-Tonelli Looks like you've got utf8 and iso8859-1 messed up. Phil
Re: Why people not using mod_perl
I would also add, in addition to the frameworks, the availability of tools such as Netbeans and Eclipse IDE's are unmatched in the perl domain. These IDE's provide many high-level conveniences for enterprise developers, most notably in the realm of SOA (such as graphical building of BPEL and CEP). After nearly 10 years building and maintaining a critical government system, we are sadly migrating away from mod_perl to a J2EE based solution due to the success and growth of our mod_perl-based system. mod_perl and MySQL has served as well when we were taking on medium-to-large loads...however, as we are growing to a distributed (multi-site, multi-node) system, with tie-ins to numerous internal and external business systems across the enterprise, with development partners working at distributed factories...tools such as Netbeans and it's tight integration with Glassfish, SVN, and Hudson make building at this level a lot more manageable. I found that mod_perl for large-scale web applications works great, and if necessary horizontal scaling is achievable to sustain even more load. However, when dealing with complex SOA architectures, and the management of business workflows...the framework support and tools to accomplish this just aren't there in perl. Add to this Jeff's comment on the availability of high caliber perl engineers...we are almost forced to make this decision. We will continue to use mod_perl for other uses, such as our custom SCM/ALM system we built over the years...but the main product is migrating. On Wed, Sep 16, 2009 at 10:47 PM, Jeff Nokes jeff_no...@yahoo.com wrote: Doesn't Amazon run mod_perl/Mason? BTW, I agree with most of your points (would debate #4,5). I may substitute the phrase More convenient for Easier in #3. I would also add ... #7) How many engineers are available to hire that know or want to work with said technology? I built a great platform at eBay on mod_perl/Mason that handled eBay-size traffic; we ran 6 eBay sites on it. Now it is used for specialty e-commerce solutions like worldofgood.ebay.com, global.ebay.com (cross-border trade), dealfinder.ebay.com, etc. In fact, on the same hardware, the main eBay Java app would support ~6 threads per box; the mod_perl platform supported ~60 (prefork), significant CapEx and power savings (which adds up at a place like eBay). -- *From:* Brad Van Sickle bvs7...@gmail.com *To:* mod_perl list modperl@perl.apache.org *Sent:* Wednesday, September 16, 2009 3:31:30 PM *Subject:* Re: Why people not using mod_perl This is a mod_perl list, so I would expect to see Perl championed pretty heavily, but Java, .net and there ilk are undoubtedly *the* choice for large web applications. I'd like to get into some discussion as to why almost all *large* sites choose these languages. I don't have any experience developing a large application in Java, although I do have a lot of experience working on the operations side of a large web application that is Java based. The reasons I generally hear for choosing Java over mod_perl are: 1) Speed - I don't buy this at all 2) Maintainability - I think this makes sense. Perl can be pretty easy to maintain if you stick a good framework around it, but you have to seek out that framework and YOU are responsible for adhereing to it. All of that is inherent in Java. It also helps that Java has OO built in. 3) Easier to package and build/move code - In my experience this is true. 4) Advantages to be gained from running on an actually application server - Also valid 5) Compatible enterprise class middleware - Also true, Java plugs into more truly enterprise level suff than Perl does. (security frameworks, etc... ) 6) Support A lot of the industry seems look at Perl as obsolete technology that has been replaced by *insert hot new technology of the week here* which is a total shame. I've worked with a lot of technologies and I think Perl is a great choice for small/medium websites and webapps, which is probably what most of us work on. But I'm very interested to know at what point (if any) a site/app grows too large or too complex for mod_perl and what defines that turning point. Could Amazon run on mod_perl for example? Phil Carmody wrote: --- On Thu, 9/17/09, Igor Chudov ichu...@gmail.com ichu...@gmail.com wrote: My site algebra.com is about 80,000 lines of mod_perl code. I wrote a relatively large framework, with many homegrown perl modules, about five years ago. It uses a database, image generation modules, a big mathematical engine that I wrote (that shows work, unlike popular third party packages), etc. All pages of my site are dynamic and it is very image heavy due to math formulae. I can say two things: 1) It is relatively fast, serving pages in 0.1 seconds or so 2) Despite the quantity of code, and its age, it is still very maintainable and understandable (to me).
Re: Why people not using mod_perl
You must have use my module Net::eBay, at some point, right? I wrote Net::eBay about 3 years ago. Igor On Wed, Sep 16, 2009 at 9:47 PM, Jeff Nokes jeff_no...@yahoo.com wrote: Doesn't Amazon run mod_perl/Mason? BTW, I agree with most of your points (would debate #4,5). I may substitute the phrase More convenient for Easier in #3. I would also add ... #7) How many engineers are available to hire that know or want to work with said technology? I built a great platform at eBay on mod_perl/Mason that handled eBay-size traffic; we ran 6 eBay sites on it. Now it is used for specialty e-commerce solutions like worldofgood.ebay.com, global.ebay.com (cross-border trade), dealfinder.ebay.com, etc. In fact, on the same hardware, the main eBay Java app would support ~6 threads per box; the mod_perl platform supported ~60 (prefork), significant CapEx and power savings (which adds up at a place like eBay). -- *From:* Brad Van Sickle bvs7...@gmail.com *To:* mod_perl list modperl@perl.apache.org *Sent:* Wednesday, September 16, 2009 3:31:30 PM *Subject:* Re: Why people not using mod_perl This is a mod_perl list, so I would expect to see Perl championed pretty heavily, but Java, .net and there ilk are undoubtedly *the* choice for large web applications. I'd like to get into some discussion as to why almost all *large* sites choose these languages. I don't have any experience developing a large application in Java, although I do have a lot of experience working on the operations side of a large web application that is Java based. The reasons I generally hear for choosing Java over mod_perl are: 1) Speed - I don't buy this at all 2) Maintainability - I think this makes sense. Perl can be pretty easy to maintain if you stick a good framework around it, but you have to seek out that framework and YOU are responsible for adhereing to it. All of that is inherent in Java. It also helps that Java has OO built in. 3) Easier to package and build/move code - In my experience this is true. 4) Advantages to be gained from running on an actually application server - Also valid 5) Compatible enterprise class middleware - Also true, Java plugs into more truly enterprise level suff than Perl does. (security frameworks, etc... ) 6) Support A lot of the industry seems look at Perl as obsolete technology that has been replaced by *insert hot new technology of the week here* which is a total shame. I've worked with a lot of technologies and I think Perl is a great choice for small/medium websites and webapps, which is probably what most of us work on. But I'm very interested to know at what point (if any) a site/app grows too large or too complex for mod_perl and what defines that turning point. Could Amazon run on mod_perl for example? Phil Carmody wrote: --- On Thu, 9/17/09, Igor Chudov ichu...@gmail.com ichu...@gmail.com wrote: My site algebra.com is about 80,000 lines of mod_perl code. I wrote a relatively large framework, with many homegrown perl modules, about five years ago. It uses a database, image generation modules, a big mathematical engine that I wrote (that shows work, unlike popular third party packages), etc. All pages of my site are dynamic and it is very image heavy due to math formulae. I can say two things: 1) It is relatively fast, serving pages in 0.1 seconds or so 2) Despite the quantity of code, and its age, it is still very maintainable and understandable (to me). In that case, would you like to fix its mangled output? e.g. http://www.algebra.com/algebra/homework/divisibility/Prime_factorization_algorithm.wikipedia   (Redirected from Prime factorization algorithm) faster than O((1+ε)b) for all positive ε an integer M with 1 ≤ M ≤ N Pollard's p − 1 algorithm Section 4.5.4: Factoring into Primes, pp. 379–417. Chapter 5: Exponential Factoring Algorithms, pp. 191–226. Chapter 6: Subexponential Factoring Algorithms, pp. 227–284. Section 7.4: Elliptic curve method, pp. 301–313. Eric W. Weisstein, “RSA-640 Factored†v • d • e AKS · APR · Ballie–PSW · ECPP · Fermat · Lucas · Lucas–Lehmer · Lucas–Lehmer–Riesel · Proth's theorem · Pépin's · Solovay–Strassen · Miller–Rabin · Trial division Sieve of Atkin · Sieve of Eratosthenes · Sieve of Sundaram · Wheel factorization CFRAC · Dixon's · ECM · Euler's · Pollard's rho · P − 1 · P + 1 · QS · GNFS · SNFS · rational sieve · Fermat's · Shanks' square forms · Trial division · Shor's Ancient Egyptian multiplication · Aryabhata · Binary GCD · Chakravala · Euclidean · Extended Euclidean · integer relation algorithm · integer square root · Modular exponentiation · Schoof's · Shanks-Tonelli Looks like you've got utf8 and iso8859-1 messed up. Phil
Re: Why people not using mod_perl
Well, actually Igor, we ended up writing eBay::API. We needed something that was able to extend many more web services that are internal-use only, that the public doesn't have access to. The fact that eBay web service data-types are probably the most complex out there, and they change often, we had to come up with a way to easily incorporate those changes by slurping up a giant WSDL, and auto-generating all the classes and data types, etc. But we do thank you for writing that. I knew of many API clients at the time that absolutely loved Net::eBay! In fact, I think at the time, the #2 API client (in listings) was perl-based, and using it. Cheers, - Jeff From: Igor Chudov ichu...@gmail.com To: Jeff Nokes jeff_no...@yahoo.com Cc: Brad Van Sickle bvs7...@gmail.com; mod_perl list modperl@perl.apache.org Sent: Wednesday, September 16, 2009 8:26:53 PM Subject: Re: Why people not using mod_perl You must have use my module Net::eBay, at some point, right? I wrote Net::eBay about 3 years ago. Igor On Wed, Sep 16, 2009 at 9:47 PM, Jeff Nokes jeff_no...@yahoo.com wrote: Doesn't Amazon run mod_perl/Mason? BTW, I agree with most of your points (would debate #4,5). I may substitute the phrase More convenient for Easier in #3. I would also add ... #7) How many engineers are available to hire that know or want to work with said technology? I built a great platform at eBay on mod_perl/Mason that handled eBay-size traffic; we ran 6 eBay sites on it. Now it is used for specialty e-commerce solutions like worldofgood.ebay.com, global.ebay.com (cross-border trade), dealfinder.ebay.com, etc. In fact, on the same hardware, the main eBay Java app would support ~6 threads per box; the mod_perl platform supported ~60 (prefork), significant CapEx and power savings (which adds up at a place like eBay). From: Brad Van Sickle bvs7...@gmail.com To: mod_perl list modperl@perl.apache.org Sent: Wednesday, September 16, 2009 3:31:30 PM Subject: Re: Why people not using mod_perl This is a mod_perl list, so I would expect to see Perl championed pretty heavily, but Java, .net and there ilk are undoubtedly *the* choice for large web applications. I'd like to get into some discussion as to why almost all *large* sites choose these languages. I don't have any experience developing a large application in Java, although I do have a lot of experience working on the operations side of a large web application that is Java based. The reasons I generally hear for choosing Java over mod_perl are: 1) Speed - I don't buy this at all 2) Maintainability - I think this makes sense. Perl can be pretty easy to maintain if you stick a good framework around it, but you have to seek out that framework and YOU are responsible for adhereing to it. All of that is inherent in Java. It also helps that Java has OO built in. 3) Easier to package and build/move code - In my experience this is true. 4) Advantages to be gained from running on an actually application server - Also valid 5) Compatible enterprise class middleware - Also true, Java plugs into more truly enterprise level suff than Perl does. (security frameworks, etc... ) 6) Support A lot of the industry seems look at Perl as obsolete technology that has been replaced by *insert hot new technology of the week here* which is a total shame. I've worked with a lot of technologies and I think Perl is a great choice for small/medium websites and webapps, which is probably what most of us work on. But I'm very interested to know at what point (if any) a site/app grows too large or too complex for mod_perl and what defines that turning point. Could Amazon run on mod_perl for example? Phil Carmody wrote: --- On Thu, 9/17/09, Igor Chudov ichu...@gmail.com wrote: My site algebra.com is about 80,000 lines of mod_perl code. I wrote a relatively large framework, with many homegrown perl modules, about five years ago. It uses a database, image generation modules, a big mathematical engine that I wrote (that shows work, unlike popular third party packages), etc. All pages of my site are dynamic and it is very image heavy due to math formulae. I can say two things: 1) It is relatively fast, serving pages in 0.1 seconds or so 2) Despite the quantity of code, and its age, it is still very maintainable and understandable (to me). In that case, would you like to fix its mangled output? e.g. http://www.algebra.com/algebra/homework/divisibility/Prime_factorization_algorithm.wikipedia   (Redirected from Prime factorization algorithm) faster than O((1+ε)b) for all positive ε an integer M with 1 ≤ M ≤ N Pollard's p − 1 algorithm Section 4.5.4: Factoring into Primes, pp. 379–417. Chapter 5: Exponential Factoring Algorithms, pp. 191–226. Chapter 6: Subexponential Factoring Algorithms, pp. 227–284. Section 7.4: Elliptic curve method, pp. 301–313. Eric
Re: Ways to scale a mod_perl site
How many servers? We have run the systems with about 500 million PV each day, with many squid boxes + 200 apache webservers + 200 mysql hosts. The applications were written with FastCGI. -Original Message- From: Igor Chudov Sent: Sep 16, 2009 11:49 AM To: Mod_Perl Subject: Ways to scale a mod_perl site My algebra.com server serves about 77k pageviews and a little over a million objects requests per day (with half of it being served in just 4 hours). I peak out at 35 requests per second currently. I use mod_perl, mysql, and perlbal with everything running on one server. The server has a solid state disk to hold mysql data. I believe that it can handle 3x-5x more traffic all by itself. However, I am thinking of ways to scale up a mod_perl installation. 1) Use a load balancer like perlbal (I am already doing that) 2) Separate a MySQL database server from webservers. 3) Being enabled by item 2, add more webservers and balancers 4) Create a separate database for cookie data (Apache::Session objects) ??? -- not sure if good idea -- (next level) 5) Use a separate database handle for readonly database requests (SELECT), as opposed to INSERTS and UPDATEs. Use replication to access multiple slave servers for read only data, and only access the master for INSERT and UPDATE and DELETE. Any thoughts?
