Smart Web Apps with IPC and threading
Since I started work on Apache::UploadMeter, I've started to look into the possibilities of creating smarter and more user-interactive webapps using IPC between webserver processes. Basically, the way this works is that you can open a magic URL at form submission/link-click time, which can be tied to a slave server, which can be instructed to provide any type of realtime feedback that you'd like - in the case of Apache::UploadMeter, the magic URL displays a snapshot of the upload process and reloads itself every X seconds, but form submissions can be done with this too. The problem is that under Apache 1.3, that wouldn't work well due to the fact that each child process is just that: a standalone process. And that's going to tie up a lot of servers and child processes, and basically make a big mess. However, under the Apache 2 threading MPM, this wouldn't seem to be a problem - especially if there is good shared resources between the threads. And mod_perl shouldn't be a problem either, as long as shared perl interpreters are used... So basically, I'd like to know if other people have any ideas that they'd be interested in sharing, and just basically see if we could look at some of the new ways of designing webapps like this. Looking forward to getting feedback, Issac PS. I posted to this list because I'd like to hear from mod_perl users - not just developers or advocates - but if this gets way OT, let's move this thread to the appropriate place.
HTTP file uploads with mod_accel
Has anyone been using mod_accel on a website that has HTTP file uploads? I'm having trouble getting file uploads to work with Internet Explorer 5.5, Netscape 4.7, or Opera 6 through mod_accel 1.0.10. If I access the backend Apache directly, it works. I can upload a 1491 byte file, but I can't upload a 13643 byte file (no matter which web browser I use). When I try to upload the 13643 byte file through mod_accel, the browser just keeps acting like it's loading the page and never finishes. I'm guessing there might be a buffering problem, but I'm not sure how to go about finding the cause of the problem and fixing it... any suggestions?
Re: HTTP file uploads with mod_accel
I use it with uploads and it all works fine. What I still haven't tested is the UPLOAD_HOOK functionality of Apache::Request under it, but I'll get around to that shortly. Issac Philip Mak wrote: Has anyone been using mod_accel on a website that has HTTP file uploads? I'm having trouble getting file uploads to work with Internet Explorer 5.5, Netscape 4.7, or Opera 6 through mod_accel 1.0.10. If I access the backend Apache directly, it works. I can upload a 1491 byte file, but I can't upload a 13643 byte file (no matter which web browser I use). When I try to upload the 13643 byte file through mod_accel, the browser just keeps acting like it's loading the page and never finishes. I'm guessing there might be a buffering problem, but I'm not sure how to go about finding the cause of the problem and fixing it... any suggestions?
Re: HTTP file uploads with mod_accel
On Sun, Jan 06, 2002 at 04:16:00PM +0200, Issac Goldstand wrote: I use it with uploads and it all works fine. What I still haven't tested is the UPLOAD_HOOK functionality of Apache::Request under it, but I'll get around to that shortly. Hmm, I wonder if it's dependent on the CPAN module used to parse the file uploads. I'm using MwfCGI.pm (distributed with mwForum), which is a somewhat modified version of CGI::Minimal.
Fixed (Re: HTTP file uploads with mod_accel)
Never mind, I'm an idiot. I just took a look at the error_log of my frontend and the problem became clear. [Sun Jan 6 09:42:04 2002] [error] [client 206.173.36.189] (13)Permission denied: accel: can't create tempfile /usr/local/apache/cache/tmpFtYxlf
make failure for Apache::Request
Hi, Here is my perl version info : stathy@server:~/.cpan/build/libapreq-0.33 perl -V Summary of my perl5 (revision 5.0 version 6 subversion 1) configuration: Platform: osname=linux, osvers=2.4.10-smp, archname=i686-linux-thread uname='linux app-server 2.4.10-smp #1 smp tue dec 11 09:35:51 cst 2001 i686 unknown ' config_args='' hint=recommended, useposix=true, d_sigaction=define usethreads=define use5005threads=define useithreads=undef usemultiplicity=undef useperlio=undef d_sfio=undef uselargefiles=define usesocks=undef use64bitint=undef use64bitall=undef uselongdouble=undef Compiler: cc='cc', ccflags ='-D_REENTRANT -fno-strict-aliasing -I/usr/local/include -I/home/stathy/loca l/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64', optimize='-O2 -march=i686', cppflags='-D_REENTRANT -fno-strict-aliasing -I/usr/local/include -I/home/sta thy/local/include' ccversion='', gccversion='2.95.3 20010315 (SuSE)', gccosandvers='' intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234 d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12 ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8 alignbytes=4, usemymalloc=n, prototype=define Linker and Libraries: ld='cc', ldflags ='-L/usr/local/lib -L/home/stathy/local/lib' libpth=/usr/local/lib /lib /usr/lib /home/stathy/local/lib libs=-lbind -lnsl -lndbm -lgdbm -ldb -ldl -lm -lpthread -lc -lcrypt -lutil perllibs=-lbind -lnsl -ldl -lm -lpthread -lc -lcrypt -lutil libc=, so=so, useshrplib=false, libperl=libperl.a Dynamic Linking: dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-rdynamic' cccdlflags='-fpic', lddlflags='-shared -L/usr/local/lib -L/home/stathy/local/lib' Characteristics of this binary (from libperl): Compile-time options: USE_THREADS USE_LARGE_FILES PERL_IMPLICIT_CONTEXT Built under linux Compiled at Dec 15 2001 11:06:44 @INC: /home/stathy/local/lib/perl5/5.6.1/i686-linux-thread /home/stathy/local/lib/perl5/5.6.1 /home/stathy/local/lib/site_perl/i686-linux-thread /home/stathy/local/lib/site_perl /home/stathy/local/lib/site_perl Here is the compiler command and error for 'request.c' : make[1]: Entering directory `/home/stathy/.cpan/build/libapreq-0.33/Request' cc -c -I../c -I/home/stathy/local/lib/site_perl/i686-linux-thread/auto/Apach e/include -I/home/stathy/local/lib/site_perl/i686-linux-thread/auto/Apache/i nclude/modules/perl -I/home/stathy/local/lib/site_perl/i686-linux-thread/aut o/Apache/include/include -I/home/stathy/local/lib/site_perl/i686-linux-threa d/auto/Apache/include/regex -I/home/stathy/local/lib/site_perl/i686-linux-th read/auto/Apache/include/os/unix -D_REENTRANT -fno-strict-aliasing -I/usr/lo cal/include -I/home/stathy/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_ BITS=64 -O2 -march=i686 -DVERSION=\0.33\ -DXS_VERSION=\0.33\ -fpic -I/ home/stathy/local/lib/perl5/5.6.1/i686-linux-thread/CORE Request.c Request.xs: In function `upload_hook': Request.xs:230: `thr' undeclared (first use in this function) Request.xs:230: (Each undeclared identifier is reported only once Request.xs:230: for each function it appears in.) make[1]: *** [Request.o] Error 1 make[1]: Leaving directory `/home/stathy/.cpan/build/libapreq-0.33/Request' make: *** [subdirs] Error 2 Any ideas would be appreciated : )
[ANNOUNCE] Apache::AppCluster 0.2
Hi all,
I'm about to post this module to CPAN. Please take a look and let me
know if you think this is appropriate for the Apache::* namespace and if
you have any problems with it ('make test' is quite comprehensive).
The module is available from:
http://www.swiftcamel.com/modules/Apache-AppCluster-0.02.tar.gz
~mark
Here is the readme:
Apache::AppCluster is a lightweight mod_perl RPC mechanism that allows
you to
use your mod_perl web servers as distributed application servers that
serve
multiple concurrent RPC requests to remote clients across a network. The
client
component has the ability to fire off multiple simultaneous requests to
multiple remote application servers and collect the responses
simultaneously.
This is similar to SOAP::Lite in that it is a web based RPC mechanism,
but
it has the advantage of being able to send/receive multiple concurrent
requests
to the same or different remote application servers and the
methods/functions
called on the remote servers may receive and return Perl data structures
of
arbritary complexity - entire objects can be flung back and forth with
ease.
Please see Apache::AppCluster::Client and Apache::AppCluster::Server
documentation for full details on server configuration (very easy) and
Client
usage (OO interface).
INSTALLATION:
Untar the distribution into a directory that will be readable by the
user
nobody. (i.e. dont use /root for installation). The test suite runs a
web server on port 8228 and this runs as user nobody.
As per usual do the following:
perl Makefile.PL
make
make test
make install
If you run into problems during the 'make test' stage, please email me
the
error log which is at: Server/t/error_log. Also include the last few
lines of 'make test' output.
APACHE CONFIG:
The documentation for Apache::AppCluster::Server contains everything
you'll
need to set up the server component. The only thing to keep in mind is
that
if you are going to be sending multiple concurrent requests from the
client
to an apache server, make sure the server is set up to handle the load.
Do this by setting MaxClients, StartServers, MinSpareServers and
MaxSpareServers.
If you're going to be hitting it with 20 concurrent requests, make sure
there
are 20 child servers standing by to handle your requests.
Re: Fixed (Re: HTTP file uploads with mod_accel)
On Sun, 6 Jan 2002, Philip Mak wrote: Never mind, I'm an idiot. I just took a look at the error_log of my frontend and the problem became clear. [Sun Jan 6 09:42:04 2002] [error] [client 206.173.36.189] (13)Permission denied: accel: can't create tempfile /usr/local/apache/cache/tmpFtYxlf My fault. I've just fixed it and in next release mod_accel would return 500 in this case. Igor Sysoev
Re: Fixed (Re: HTTP file uploads with mod_accel)
Igor, DOES mod_accel buffer the uploads as they come through? That feature would be critical for compatibility with libapreq's UPLOAD_HOOK, which I'm finding lots of nice uses for... Issac - Original Message - From: Igor Sysoev [EMAIL PROTECTED] To: Philip Mak [EMAIL PROTECTED] Cc: Issac Goldstand [EMAIL PROTECTED]; [EMAIL PROTECTED] Sent: Sunday, January 06, 2002 6:56 PM Subject: Re: Fixed (Re: HTTP file uploads with mod_accel) On Sun, 6 Jan 2002, Philip Mak wrote: Never mind, I'm an idiot. I just took a look at the error_log of my frontend and the problem became clear. [Sun Jan 6 09:42:04 2002] [error] [client 206.173.36.189] (13)Permission denied: accel: can't create tempfile /usr/local/apache/cache/tmpFtYxlf My fault. I've just fixed it and in next release mod_accel would return 500 in this case. Igor Sysoev
Re: Fixed (Re: HTTP file uploads with mod_accel)
On Sun, 6 Jan 2002, Issac Goldstand wrote: DOES mod_accel buffer the uploads as they come through? That feature would be critical for compatibility with libapreq's UPLOAD_HOOK, which I'm finding lots of nice uses for... Yes, mod_accel buffers completly uploads and starting from 1.0.8 broken uploads even do not go to backend. The are two reasons: 1. Backend should not wait slow client upload. 2. To implement simple fault-tolerance I need to collect whole upload. Although second reason does not require to collect whole upload before sending it to backend. There is workaround to use UPLOAD_HOOK on backend - you can use mod_proxy with mod_accel: ProxyPass /upload/ http://backend/upload/ ProxyPassReverse /upload/ http://backend/upload/ AccelPass / http://backend/ AccelNoPass/upload/ Igor Sysoev
take23 and what we're doing
take23.org has been very quiet. I think it'd be cool if everyone on this list posted a short piece about their current mod_perl project. It could be a module they're working on, a site they built, or even something as simple as how I used Apache::MP3 to let everyone in our house listen to our music collection. It'd be fun to see what everyone else is doing, and it'd really help give the site a lift. If the articles were RSSed and picked up by meerkat (www.oreillynet.com/meerkat) I think it'd do good for the world to see a constant flow of mod_perl activity. What do you say, Matt? Nat
Re: make failure for Apache::Request
Stathy G. Touloumis [EMAIL PROTECTED] writes: [...] usethreads=define use5005threads=define useithreads=undef [...] Request.xs: In function `upload_hook': Request.xs:230: `thr' undeclared (first use in this function) [...] Any ideas would be appreciated : ) Try experimenting with something like dTHX; somewhere above line 230, and read the section on threads in perlguts. If you can get apreq working with use5005threads, please submit a patch to [EMAIL PROTECTED]; This is one of the two outstanding issues that's holding up a new 1.0 release. The other one is a Mac OS-X solution, but nobody's volunteered one so far; or provided evidence of a successful build on that platform. Thanks in advance. -- Joe Schaefer
cvs commit: modperl-2.0/xs/Apache/Response Apache__Response.h
dougm 02/01/06 14:23:24 Modified:xs/Apache/Response Apache__Response.h Log: rcfg-wbucket is now a pointer Revision ChangesPath 1.5 +2 -2 modperl-2.0/xs/Apache/Response/Apache__Response.h Index: Apache__Response.h === RCS file: /home/cvs/modperl-2.0/xs/Apache/Response/Apache__Response.h,v retrieving revision 1.4 retrieving revision 1.5 diff -u -r1.4 -r1.5 --- Apache__Response.h13 Sep 2001 04:41:44 - 1.4 +++ Apache__Response.h6 Jan 2002 22:23:24 - 1.5 @@ -10,7 +10,7 @@ STRLEN len; \ const char *bodytext; \ modperl_cgi_header_parse(r, SvPV(sv,len), bodytext); \ -rcfg-wbucket.header_parse = 0; \ +rcfg-wbucket-header_parse = 0; \ } /* XXX: should only be part of Apache::compat */ @@ -23,5 +23,5 @@ r-content_type = apr_pstrdup(r-pool, type); } -rcfg-wbucket.header_parse = 0; /* turn off PerlOptions +ParseHeaders */ +rcfg-wbucket-header_parse = 0; /* turn off PerlOptions +ParseHeaders */ }
