Re: [ANNOUNCE] Apache::AppCluster 0.2
Gunther Birznieks wrote: Is this a lot different from PlRPC? Perhaps this should be a transport layer that is added to PlRPC rather than creating a brand-new service? Ideally it would be nice if it were architected with mod_perl and HTTP being the default mechanism of transport but that if you wanted to, you could make it into a standalone Perl daemon that forked by itself. The difference is that AppCluster allows you to call multiple remote methods on the server (or on multiple distributed servers) simultaneously. However, I wasn't aware of PlRPC and I really like the interface i.e. the way it creates a copy of the remote object locally and allows you to call methods on it as if it were just another object. I could not do this with AppCluster without sacrificing the concurrency it offers. At present you create a single appcluster object and then register the method calls you would like it to call, one by one. You then call a single method (with a timeout) that calls all registered remote methods simultaneously. (It uses non-blocking IO and so provides concurrency in a single thread of execution) re: allowing the server to be a standalone daemon that forks by itself. This project actually started exactly like that. I grabbed the pre-forking server example from the Cookbook and used that as the basis for the server. I found that performance was horrible though, because forking additional servers took too long and I also was using DBI and I missed Apache::DBI (and all the other great apache::* mods) too much. So I used a good solid server that offers a great Perl persistence engine. I'm not sure why anyone would want to roll their own server in Perl. If there is a reason, then I could change the server class to create a server abstraction layer of some kind. ~mark.
Re: [ANNOUNCE] Apache::AppCluster 0.2
On Wed, 9 Jan 2002, Mark Maunder wrote: The difference is that AppCluster allows you to call multiple remote methods on the server (or on multiple distributed servers) simultaneously. However, I wasn't aware of PlRPC and I really like the interface i.e. the way it creates a copy of the remote object locally and allows you to call methods on it as if it were just another object. would it require too much surgery and api change if you added the concurrency support to PlRPC?
Re: [ANNOUNCE] Apache::AppCluster 0.2
brian moseley wrote: On Wed, 9 Jan 2002, Mark Maunder wrote: The difference is that AppCluster allows you to call multiple remote methods on the server (or on multiple distributed servers) simultaneously. However, I wasn't aware of PlRPC and I really like the interface i.e. the way it creates a copy of the remote object locally and allows you to call methods on it as if it were just another object. would it require too much surgery and api change if you added the concurrency support to PlRPC? Well, I guess two methods could be added to the client object. One to add a concurrent request to be called (register_request()) and one to send all registered requests concurrently. I'm not the author though, so you'll have to chat to Jochen about that. The server and transport would have to be rewritten pretty much from scratch I think. The transport needs to be HTTP POST requests and responses. The server needs to be set up as a mod_perl handler that takes advantage of everything mod_perl has to offer. From my point of view, it's easier to duplicate Jochen's work in AppCluster by adding the same type of interface i.e. creating a copy of a remote object locally and calling methods on that object as per normal while the actual method call is submitted via a POST to a remote mod_perl app server. I dont really mind whether we incorporate this stuff into PlClient or AppCluster or both, but I do think that both the concurrency in AppCluster and tied object API in PlRPC are really useful and would be even better with the remote app server being mod_perl. An idea might be to incorporate both the AppCluster concurrency and PlRPC style api into an Apache::* module that gives us the best of both worlds with mod_perl performance (etc.) on the server side. (and then get rid of AppCluster since it will be redundant) Let me know if that sounds like a good idea and I'll start work on it. Perhaps we could call it Apache::PlRPC (now with added concurrency!) ~mark
Re: [ANNOUNCE] Apache::AppCluster 0.2
On Wed, 9 Jan 2002, Mark Maunder wrote: Well, I guess two methods could be added to the client object. One to add a concurrent request to be called (register_request()) and one to send all registered requests concurrently. I'm not the author though, so you'll have to chat to Jochen about that. couldn't you just subclass RPC::PlClient? The server and transport would have to be rewritten pretty much from scratch I think. The transport needs to be HTTP POST requests and responses. The server needs to be set up as a mod_perl handler that takes advantage of everything mod_perl has to offer. why needs? i'm sure lots of people would rather run a very lightweight non-http/apache-based server. I dont really mind whether we incorporate this stuff into PlClient or AppCluster or both, but I do think that both the concurrency in AppCluster and tied object API in PlRPC are really useful and would be even better with the remote app server being mod_perl. seems like the ideal api gives you the best functionality from both original apis and abstracts away the choice of transport and server. An idea might be to incorporate both the AppCluster concurrency and PlRPC style api into an Apache::* module that gives us the best of both worlds with mod_perl performance (etc.) on the server side. (and then get rid of AppCluster since it will be redundant) perhaps i misunderstand, but you're suggesting making the client an Apache module? why? i like the idea of being able to write client code that uses the same rpc api no matter whether i choose to use soap, xml-rpc, a more specific http post, plrpc's transport (whatever that might be), or whatever as the transport. not all of the drivers would have to support the same feature set (i think your mechanism supports arbitrarily deep data structures?). that rpc api is one of the things p5ee is looking for, i believe.
Re: [ANNOUNCE] Apache::AppCluster 0.2
brian moseley wrote: On Wed, 9 Jan 2002, Mark Maunder wrote: Well, I guess two methods could be added to the client object. One to add a concurrent request to be called (register_request()) and one to send all registered requests concurrently. I'm not the author though, so you'll have to chat to Jochen about that. couldn't you just subclass RPC::PlClient? The transport is different (HTTP/POST) and I dont think I can easily just drop in another (alternative) transport - I may as well rewrite. The server and transport would have to be rewritten pretty much from scratch I think. The transport needs to be HTTP POST requests and responses. The server needs to be set up as a mod_perl handler that takes advantage of everything mod_perl has to offer. why needs? i'm sure lots of people would rather run a very lightweight non-http/apache-based server. Agreed. Are there any more besides a standalone pure perl daemon and mod_perl/apache? I dont really mind whether we incorporate this stuff into PlClient or AppCluster or both, but I do think that both the concurrency in AppCluster and tied object API in PlRPC are really useful and would be even better with the remote app server being mod_perl. seems like the ideal api gives you the best functionality from both original apis and abstracts away the choice of transport and server. yeah - agreed. An idea might be to incorporate both the AppCluster concurrency and PlRPC style api into an Apache::* module that gives us the best of both worlds with mod_perl performance (etc.) on the server side. (and then get rid of AppCluster since it will be redundant) perhaps i misunderstand, but you're suggesting making the client an Apache module? why? Well, the server component (at present) is a mod_perl handler, and I wanted to bundle both together so I stuck in in the Apache namespace (pending any objections of course). Seems like RPC might make more sense if it becomes platform/server neutral, since Apache::* binds the server platform to mod_perl. i like the idea of being able to write client code that uses the same rpc api no matter whether i choose to use soap, xml-rpc, a more specific http post, plrpc's transport (whatever that might be), or whatever as the transport. not all of the drivers would have to support the same feature set (i think your mechanism supports arbitrarily deep data structures?). that rpc api is one of the things p5ee is looking for, i believe. It seems like you're asking a bit much here (the holy grail of RPC?). SOAP and xml-rpc are their own client/server system. Are we going to integrate this with both of them and support standalone perl daemons etc.? I considered writing a soap client that allows for concurrent requests, but I found SOAP::Lite to be slow under mod_perl so opted for rolling my own instead. Also SOAP is platform neutral and I'm not sure, but I think it wont allow for perl data structures as complex as Storable does. I think you should probably distinguish between transport and encoding. Transports could be http, https or a plain old socket (if we're talking to a perl daemon) and an encrypted socket connection using CBC. Encodings we've chatted about are Storable's freeze/thaw and SOAP's XML, and then xml-rpc (which I assume has it's own encoding of method name and params etc in xml - although I dont know it at all). I think having various transports is fine. But I'm not sure what the motivation is for varying the encodings, unless we want our client to talk to SOAP and xml-rpc servers, or vice versa. Perhaps integrating PlRPC and AppCluster's client API's and allowing for either standalone daemon or mod_perl server is a good start? We can use HTTP, HTTPS, direct sockets and encrypted sockets as the first transports. The client can have two modes - concurrent remote procedure calls, or creating a copy of the remote object PlRPC style. ~mark.
Re: [ANNOUNCE] Apache::AppCluster 0.2
Apache::AppCluster is now in CPAN and can be accessed at: http://search.cpan.org/search?dist=Apache-AppCluster This consists of a client and server module that can be used to develop mod_perl clustered web services. A client application can make multiple simultaneous API calls to a mod_perl server (or multiple servers) passing and receiving any perl reference or object as parameters. The server module runs the called module::method in an eval loop and handles errors gracefully while taking advantage of the persistence offered by mod_perl. The client has a bunch of useful methods and includes a timeout for the total time allowed for all remote calls to complete. Let me know if you have any suggestions or find this useful. (or find any bugs) ~mark *snip* The uploaded file Apache-AppCluster-0.02.tar.gz has entered CPAN as file: $CPAN/authors/id/M/MA/MAUNDER/Apache-AppCluster-0.02.tar.gz size: 23995 bytes md5: 0abff0a4a2aa053c9f8ae3c00dd86434 No action is required on your part Request entered by: MAUNDER (Mark D. Maunder) Request entered on: Sun, 06 Jan 2002 16:31:57 GMT Request completed: Sun, 06 Jan 2002 16:33:33 GMT
[ANNOUNCE] Apache::AppCluster 0.2
Hi all,
I'm about to post this module to CPAN. Please take a look and let me
know if you think this is appropriate for the Apache::* namespace and if
you have any problems with it ('make test' is quite comprehensive).
The module is available from:
http://www.swiftcamel.com/modules/Apache-AppCluster-0.02.tar.gz
~mark
Here is the readme:
Apache::AppCluster is a lightweight mod_perl RPC mechanism that allows
you to
use your mod_perl web servers as distributed application servers that
serve
multiple concurrent RPC requests to remote clients across a network. The
client
component has the ability to fire off multiple simultaneous requests to
multiple remote application servers and collect the responses
simultaneously.
This is similar to SOAP::Lite in that it is a web based RPC mechanism,
but
it has the advantage of being able to send/receive multiple concurrent
requests
to the same or different remote application servers and the
methods/functions
called on the remote servers may receive and return Perl data structures
of
arbritary complexity - entire objects can be flung back and forth with
ease.
Please see Apache::AppCluster::Client and Apache::AppCluster::Server
documentation for full details on server configuration (very easy) and
Client
usage (OO interface).
INSTALLATION:
Untar the distribution into a directory that will be readable by the
user
nobody. (i.e. dont use /root for installation). The test suite runs a
web server on port 8228 and this runs as user nobody.
As per usual do the following:
perl Makefile.PL
make
make test
make install
If you run into problems during the 'make test' stage, please email me
the
error log which is at: Server/t/error_log. Also include the last few
lines of 'make test' output.
APACHE CONFIG:
The documentation for Apache::AppCluster::Server contains everything
you'll
need to set up the server component. The only thing to keep in mind is
that
if you are going to be sending multiple concurrent requests from the
client
to an apache server, make sure the server is set up to handle the load.
Do this by setting MaxClients, StartServers, MinSpareServers and
MaxSpareServers.
If you're going to be hitting it with 20 concurrent requests, make sure
there
are 20 child servers standing by to handle your requests.
