XML::RPC
Hi, I am finally finding yes, there could be a reason for me to use this, I want to avoid SOAP, partly because of the license crap(fear), but also because I figure that XMLRPC will be easier to learn at first and likely will fit my small needs. But as I look at examples and various tools that support working with Perl they often are of a nature of a CGI and a standalone client. My needs aren't that small :) The only Apache modules I have found are with the SOAP::Lite module. So what do the you people recommend? I have found several, Frontier::RPC, there is a SOAP::Lite XML::RPC module, but it depends on SOAP::Lite which I find confusing, there is just RPC::XML on CPAN. Most important though, I of course want to run the server with mod_perl which is the only reason I consider this post to be somewhat on topic, I really want to use mod_perl with this and what group would be better to advise me on that :) What I want to be able to do is to expose some simple method calls, along with some hash data to a Java client. Say like a parser object from HTML::Parser. The client will be Java, but I get the idea that shouldn't really matter. Sorry if my terminology is off, I am swimming a bit.. Thanks, Eric http://www.kwinternet.com/eric (250) 655 - 9513 (PST Time Zone) Learn about the net, not the .Net!
Re: XML::RPC
I forgot one, http://sourceforge.net/projects/mod-xmlrpc/ But it isn't exactly well documented for a non C programmer. It looks like it might be an option. Eric http://www.kwinternet.com/eric (250) 655 - 9513 (PST Time Zone) Learn about the net, not the .Net!
Re: XML::RPC
On Sat, 20 Apr 2002, Eric Frazier wrote: > I am finally finding yes, there could be a reason for me to use this, I want > to avoid SOAP, partly because of the license crap(fear), but also because I > figure that XMLRPC will be easier to learn at first and likely will fit my > small needs. I think you should give SOAP another look. Check out O'Reilly's new SOAP book, written by the author of SOAP::Lite, and see if that doesn't change your mind. I'm no XMLRPC expert but the little I do know makes me think SOAP is likely to make it obsolete very soon. Also, I've found the SOAP::Lite module to be of excelent quality and the SOAP::Lite community to be very helpful. -sam
Re: XML::RPC
On Sun, Apr 21, 2002 at 03:16:53AM -0400, Sam Tregar wrote: > SOAP::Lite module to be of excelent quality and the SOAP::Lite community > to be very helpful. Apart from the obvious security bug, you mean? The one where it doesn't actually restrict what remote code can be run at all? If you are going to use SOAP, be *very* *very* aware of the security issues with any RPC-like mechanism, review any code you're going to use, thoroughly. And only then, decide whether it is actually the best solution. MBM -- Matthew Byng-Maddick <[EMAIL PROTECTED]> http://colondot.net/
Re: XML::RPC
On Sun, 21 Apr 2002 10:50:53 +0100 Matthew Byng-Maddick <[EMAIL PROTECTED]> wrote: : On Sun, Apr 21, 2002 at 03:16:53AM -0400, Sam Tregar wrote: : > SOAP::Lite module to be of excelent quality and the SOAP::Lite community : > to be very helpful. : : Apart from the obvious security bug, you mean? The one where it doesn't : actually restrict what remote code can be run at all? SOAP::Lite 0.55 was released some days ago, it addresses that issue according to http://www.soaplite.com/ -- fxn
Re: XML::RPC
On Sun, Apr 21, 2002 at 01:06:28PM +0200, F. Xavier Noria wrote: > On Sun, 21 Apr 2002 10:50:53 +0100 > Matthew Byng-Maddick <[EMAIL PROTECTED]> wrote: > : On Sun, Apr 21, 2002 at 03:16:53AM -0400, Sam Tregar wrote: > : > SOAP::Lite module to be of excelent quality and the SOAP::Lite community > : > to be very helpful. > : Apart from the obvious security bug, you mean? The one where it doesn't > : actually restrict what remote code can be run at all? > SOAP::Lite 0.55 was released some days ago, it addresses that issue > according to > http://www.soaplite.com/ I'm aware of this, but I can't stress the importance of reviewing such security-critical code. And the "excellent quality" of the code that was mentioned by Sam Tregar in his post. RPC often is a nightmare security-wise, the SOAP::Lite bug illustrates the problems perfectly. MBM -- Matthew Byng-Maddick <[EMAIL PROTECTED]> http://colondot.net/
Re: XML::RPC
Hi, The warning is nice, but it isn't that useful. Do you have an alternative? I am not looking to do great things, just get a Java client to be able to do a few method calls, and to accept a XML encoded hash of perl data. It is a fairly complex hash though. So I really don't know of another way to send this data. Perhaps if I get rid of the method calls and just use Registry, and then "hand" encode the XML hash with perl that would make you feel better? I still have to deal with auth and I don't see how a SOAP server is any less secure than the server itself is considering every request will be authenticated, no open server stuff, no requests that I don't know who they are coming from. But I am learning, so I am happy to learn more. It is just that your comments seem pretty general. Thanks, Eric At 11:46 AM 4/21/02 +0100, Matthew Byng-Maddick wrote: >On Sun, Apr 21, 2002 at 01:06:28PM +0200, F. Xavier Noria wrote: >> On Sun, 21 Apr 2002 10:50:53 +0100 >> Matthew Byng-Maddick <[EMAIL PROTECTED]> wrote: >> : On Sun, Apr 21, 2002 at 03:16:53AM -0400, Sam Tregar wrote: >> : > SOAP::Lite module to be of excelent quality and the SOAP::Lite community >> : > to be very helpful. >> : Apart from the obvious security bug, you mean? The one where it doesn't >> : actually restrict what remote code can be run at all? >> SOAP::Lite 0.55 was released some days ago, it addresses that issue >> according to >> http://www.soaplite.com/ > >I'm aware of this, but I can't stress the importance of reviewing such >security-critical code. And the "excellent quality" of the code that was >mentioned by Sam Tregar in his post. > >RPC often is a nightmare security-wise, the SOAP::Lite bug illustrates the >problems perfectly. > >MBM > >-- >Matthew Byng-Maddick <[EMAIL PROTECTED]> http://colondot.net/ > http://www.kwinternet.com/eric (250) 655 - 9513 (PST Time Zone) Learn about the net, not the .Net!
Re: XML::RPC
On Sat, Apr 20, 2002 at 11:49:24PM -0400, Eric Frazier wrote: > Hi, > > I am finally finding yes, there could be a reason for me to use this, I want > to avoid SOAP, partly because of the license crap(fear), but also because I > figure that XMLRPC will be easier to learn at first and likely will fit my > small needs. But as I look at examples and various tools that support > working with Perl they often are of a nature of a CGI and a standalone > client. My needs aren't that small :) The only Apache modules I have found > are with the SOAP::Lite module. > > So what do the you people recommend? I have found several, Frontier::RPC, > there is a SOAP::Lite XML::RPC module, but it depends on SOAP::Lite which I > find confusing, there is just RPC::XML on CPAN. Most important though, I of > course want to run the server with mod_perl which is the only reason I > consider this post to be somewhat on topic, I really want to use mod_perl > with this and what group would be better to advise me on that :) > > What I want to be able to do is to expose some simple method calls, along > with some hash data to a Java client. Say like a parser object from > HTML::Parser. The client will be Java, but I get the idea that shouldn't > really matter. Sorry if my terminology is off, I am swimming a bit.. I think you'll find RPC::XML to be a solidly engineered module. I've used it as a client and as a server to good effect. It includes stand-alone, CGI, and mod_perl based servers and a very nice client. If all you need is simple RPC calls I find that XML-RPC provides a much more straightforward environment. Even with my background in http based rpc systems it took me a while to pick up on the idiosyncrasies of SOAP. -- Paul Lindner[EMAIL PROTECTED] | | | | | | | | | | mod_perl Developer's Cookbook http://www.modperlcookbook.org/ Human Rights Declaration http://www.unhchr.ch/udhr/
Re: XML::RPC
Hi, Thanks, the thing that scared me about RPC::XML was the 0.37 version number :) I will give it a serious try before I go to something like SOAP. Within RPC::XML: Apache::RPC::Server - A subclass of RPC::XML::Server tuned for mod_perl I didn't see this until now. :) Getting warm fuzzy feeling now. Thanks again, Eric >I think you'll find RPC::XML to be a solidly engineered module. I've >used it as a client and as a server to good effect. It includes >stand-alone, CGI, and mod_perl based servers and a very nice client. > >If all you need is simple RPC calls I find that XML-RPC provides a >much more straightforward environment. Even with my background in >http based rpc systems it took me a while to pick up on the >idiosyncrasies of SOAP. > >-- >Paul Lindner[EMAIL PROTECTED] | | | | | | | | | | > >mod_perl Developer's Cookbook http://www.modperlcookbook.org/ > Human Rights Declaration http://www.unhchr.ch/udhr/ > http://www.kwinternet.com/eric (250) 655 - 9513 (PST Time Zone) Learn about the net, not the .Net!
Re: XML::RPC
> I think you'll find RPC::XML to be a solidly engineered module. I've > used it as a client and as a server to good effect. It includes > stand-alone, CGI, and mod_perl based servers and a very nice client. I will second the vote for RPC::XML. We use this module with great success under mod_perl. It's very easy to understand for the most part. There are a few little oddities, but I'm sure myself or Paul can get you around any obstacles you hit. -Fran
Re: XML::RPC
Ciao! As has been pointed out in the past, the VERSION of a module does not necessarily reveal the module's maturity or robustness. This is particularly true for the RPC::XML module. I am using it in two production environments, in both client and modperl-enabled server modes with excellent results. It's dependency on the XML::Parser module has been a NO-OP for my Solaris8 deployment platforms. I am pleased with its overall performance, ease of use and robustness, as well as the author's continued support. YMMV. Peace. > "Eric" == Eric Frazier <[EMAIL PROTECTED]> writes: Eric> Hi, Thanks, the thing that scared me about RPC::XML was the Eric> 0.37 version number :) Eric> I will give it a serious try before I go to something like Eric> SOAP. Eric> Within RPC::XML: Eric> Apache::RPC::Server - A subclass of RPC::XML::Server tuned Eric>for mod_perl Eric> I didn't see this until now. :) Getting warm fuzzy feeling Eric> now. Thanks again,
Suggestions on an XML-RPC Service using modperl?
Ciao! I would like some input on an intranet web service I am currently in the process of designing, the core of which will be modperl on UN*X. The service itself is to access a couple of back end data stores given parameters received in an XML-RPC request, then return the results in an XML-RPC formated response. The data from the back end sources will be loaded into memory at service initialization, for fast access. The data is small enough and memory plentiful enough to allow this. That's pretty much it in terms of the high level data flow. It has to be relatively fast, OTO 5+ requests/sec. as a relative volumetric. I'm going to use modperl due to the embedded perl interpreter characteristics it provides, allowing initialization overhead to be incurred at startup. I also wish to use an in-memory, read-only hash structure shared across all modperl processes for access to the cached back end data, rather than making expensive calls to these stores for each request. Again, throughput is critical. I would like your thoughts on the cache management concept of the service. I'm looking at MLDBM::Sync as the mechanism for managing the filesystem representation of the in-memory hash content. What to manage the in-memory structure itself in terms of accessing its content? Is a Tie structure too expensive? I want to end up with a single structure accessible to all of the modperl processes, loaded at service startup. This service will ultimately be registered within a UDDI/SOAP framework, FYI. However this will not be in the first incarnation of the service itself. Thoughts and comments welcome. Obviously this is an early brainstorm (more like a drizzle) but I hope to get a few stimulating comments from this most excellent resource, the list. Peace.
Re: Suggestions on an XML-RPC Service using modperl?
As far as the cacheing goes, we have had extremely good luck with IPC::ShareLite used to share info across mod_perl processes. --Jon R. Chip Turner wrote: > > "Bruce W. Hoylman" <[EMAIL PROTECTED]> writes: > > > Ciao! > > > > I would like some input on an intranet web service I am currently in the > > process of designing, the core of which will be modperl on UN*X. > > Excellent choice. This works quite well. Of course, like others on > this list, I might be a bit biased. > > > The service itself is to access a couple of back end data stores given > > parameters received in an XML-RPC request, then return the results in an > > XML-RPC formated response. The data from the back end sources will be > > loaded into memory at service initialization, for fast access. The data > > is small enough and memory plentiful enough to allow this. > > How often does the data change? How is it stored on the back end? > You may not need to cache anything if, say, you have a decent SQL > database on the backend. Caching never hurts, but it isn't always > necessary. The Cache::* modules may be of use for this, though, > should you still need it. You also might want to consider not sharing > the data in each process; the complexity gained vs the memory lost by > storing it in each process may be a workable tradeoff. I probably > would try it first with no cache, then a per-process on-demand cache, > then finally a shared cache, in that order. > > > That's pretty much it in terms of the high level data flow. It has to > > be relatively fast, OTO 5+ requests/sec. as a relative volumetric. > > This should be quite easy. I don't have the necessary setup handy to > benchmark it, but I imagine you can easily achieve performance at that > level using Frontier::RPC inside a mod_perl handler. We typically use > custom code for interfacing the handler, but IIRC the Frontier module > comes with a mod_perl handler that, if not enturely suitable, is > easily modified to your needs. > > > I'm going to use modperl due to the embedded perl interpreter > > characteristics it provides, allowing initialization overhead to be > > incurred at startup. I also wish to use an in-memory, read-only hash > > structure shared across all modperl processes for access to the cached > > back end data, rather than making expensive calls to these stores for > > each request. Again, throughput is critical. > > Five hits/second should be absolutely no problem. If you expect slow > clients, a mod_proxy in front of things (http://perl.apache.org/guide) > can help. > > > I would like your thoughts on the cache management concept of the > > service. I'm looking at MLDBM::Sync as the mechanism for managing the > > filesystem representation of the in-memory hash content. What to manage > > the in-memory structure itself in terms of accessing its content? Is a > > Tie structure too expensive? I want to end up with a single structure > > accessible to all of the modperl processes, loaded at service > > Chip > > -- > Chip Turner [EMAIL PROTECTED] > Red Hat Network
Re: Suggestions on an XML-RPC Service using modperl?
> I would like your thoughts on the cache management concept of the > service. I'm looking at MLDBM::Sync as the mechanism for managing the > filesystem representation of the in-memory hash content. What to manage > the in-memory structure itself in terms of accessing its content? MLDBM::Sync includes an option for using an in-memory cache in front of the dbm cache. This is fine for read-only data. > Is a Tie structure too expensive? Tied variables are slower than calling methods on objects, but if you have a class implemented as a TIE, you can always just call the STORE, FETCH, etc. methods directly. - Perrin
Re: Suggestions on an XML-RPC Service using modperl?
> Even then, I'd avoid disk-based cache systems, instead > preferring Cache::* if it must be shared, or just global variables if > it doesn't need to be. Cache::FileCache is disk-based, and it is the fastest of the Cache:: options for most data sets. There was a thread a little while back about data sharing that showed the top performers to be Cache::Mmap and IPC::MM. Cache::Cache and MLDBM::Sync should be more than fast enough for all but the most highly optimized systems. - Perrin
Re: Suggestions on an XML-RPC Service using modperl?
> As far as the cacheing goes, we have had extremely good luck with > IPC::ShareLite used to share info across mod_perl processes. IPC::ShareLite is not as fast as some of the other options, especially when dealing with a large data set. The disk-based options tend to be faster. - Perrin
Re: Suggestions on an XML-RPC Service using modperl?
Ciao!
I'm on the modper digest list so pardon the delayed response to your
input.
>>>>> "Aaron" == Aaron E Ross <[EMAIL PROTECTED]> writes:
Aaron> I've been working with a mod_perl based XML-RPC/SOAP
Aaron> service for a few months now and I thought I'd share some
Aaron> quick thoughts.
Thanks! I'll take 'um.
Aaron> As long as you are _sure_ that you won't be writing data,
Aaron> in-memory will be fast and easy to code. if you use
Aaron> objects you can pretty easily build an interface
Aaron> encouraging programmers to avoid modifying shared data.
Yes, I refuse to do *any* data arbitration between the localized
portions of the data (if any?) and the remote or official copies of it.
Copying it to a local store is meant to remove the access latency and
availability of the data, although I have not yet determined there to be
latency issues. There are however availability issues surround these
databases. I agree, an OO interface to it will facilitate its use and
certainly simplify any data format changes that are almost certain to
take place over the life of the service itself.
Aaron> Aim for what you may need later, 5+ will be easy as long as
Aaron> you have some memory.
I'm lobbying for excessive memory as we speak! Good point.
Aaron> I have found the Cache::Cache modules really easy to use,
Aaron> well written, documented and supported. MLDBM::Sync
Aaron> provides some locking, but if you really need concurrent
Aaron> access I would highly recommend BerkeleyDB,
Aaron> http://sleepycat.com/, nb this is _not_ DB_File.
Mr. Turner also mentioned the Cache::* modules for this. I'm not really
interested in locking the data in any way whatsoever. It's strictly
read-only so I see no reason to have to manage read locks. Therefore
concurrent access to an in-memory data store between all the modperl
processes should be no problem, correct?
Aaron> You haven't explained the cacheing logic at all.. perhaps
Aaron> you don't need a cache? maybe just an object that reads
Aaron> from the data stores on startup?
This is a good idea. It would eliminate the interim step of loading the
data onto the local machines prior to bringing it into memory. I'm
working on data access issues this week and want to be able to describe
the data better as well as gain query access to it. Then I should be
able to benchmark access and availability, my two primary concerns with
the data.
Aaron> If you do need a cacheing layer that updates on some event
Aaron> or expiration, remember to seperate the cacheing logic and
Aaron> the storage as much as possible to be able to tune and to
Aaron> scale up later on.
I'll keep this in mind.
Aaron> Why not write a simple object? I try to avoid tie's, b/c
Aaron> they are too "magical" for my taste, but i don't think
Aaron> there is any inherent overhead.
The object description of the data seems to be a good way to go. I
believe this will end up being the API I present to the handler for data
access. I agree.
Aaron> I would recommend using SOAP::Lite for both XML-RPC and
Aaron> SOAP. While the code is unreadable, the author is
Aaron> responsive and helpful, and the switch from XML-RPC to SOAP
Aaron> couldn't be easier.
You got the unreadable part right (c: Sorry. I've had occasion to use
Randy Ray's RPC-XML module in the past and it functioned very well. He
describes it as a reference implementation of the XML-RPC specification
and does not attest to its effeciency or speed. I'll do some comparison
shopping here with SOAP::Lite and see what comes out.
Aaron> Hope this helps, Aaron
Thanks for your insight into this matter. I really appreciate your
input.
Peace.
Re: Suggestions on an XML-RPC Service using modperl?
Ciao! > "Jon" == Jon Robison <[EMAIL PROTECTED]> writes: Jon> As far as the cacheing goes, we have had extremely good luck Jon> with IPC::ShareLite used to share info across mod_perl Jon> processes. Have you compared this module to the Cache::* modules? I believe the Cache::SharedMemoryCache module provides a shared memory implementation, for example. I have had a few suggestions offered surrounding the use of the Cache::* modules which is why I ask. Thanks for your thoughts. Peace.
Re: Suggestions on an XML-RPC Service using modperl?
Ciao! I'm on the digest for this list, so pardon my delay in responding to everyone's comments. > "Chip" == Chip Turner <[EMAIL PROTECTED]> writes: Chip> "Bruce W. Hoylman" <[EMAIL PROTECTED]> writes: >> Ciao! >> >> I would like some input on an intranet web service I am currently >> in the process of designing, the core of which will be modperl on >> UN*X. Chip> Excellent choice. This works quite well. Of course, like Chip> others on this list, I might be a bit biased. I'm sort of sneaking this in the back door. Much of the development surrounding these new objectives is centered in the J2EE architecture. I'm taking a 'don't say much' approach until it's working and cranking out the data. Then I'll be able to meet any advances on architecture choices. It's not that perl/Apache is taboo here, just not looked upon as an implementation solution of choice. However it won't be the first time I have faced these pundits. Chip> How often does the data change? How is it stored on the back Chip> end? You may not need to cache anything if, say, you have a Chip> decent SQL database on the backend. Caching never hurts, but Chip> it isn't always necessary. The Cache::* modules may be of use Chip> for this, though, should you still need it. You also might Chip> want to consider not sharing the data in each process; the Chip> complexity gained vs the memory lost by storing it in each Chip> process may be a workable tradeoff. I probably would try it Chip> first with no cache, then a per-process on-demand cache, then Chip> finally a shared cache, in that order. I still have to determine the dynamic characteristics of the data in question. This certainly needs to be taken into account in any cacheing or local storage of the data. The remote data is Oracle-ized in some 7.x database instances. When I speak of cacheing I'm referring to taking remote data sources and making the local for some period of time, after which they must be refreshed and localized again. I don't believe the service can withstand the cost of remote database access and still be as streamlined as the requirements specify. However I will be going through a series of benchmarks to see just which data management approach is appropriate. I also am keen to the unavailability of these external, remote data sources. They seem to be 'down' at the most inopportune moments. In summation, data throughput and availability are the two most compelling reasons for considering a localized, and I emphasize READ-ONLY capture of the remote stores (there are two). Chip> Five hits/second should be absolutely no problem. If you Chip> expect slow clients, a mod_proxy in front of things Chip> (http://perl.apache.org/guide) can help. This is a very interesting section of the guide. I recommend it to everyone simply for the sake knowing alternative configurations that include a modperl component. Chip> There are a lot of options, but really, I would hold off on Chip> deciding complicated caching schemes until you know what Chip> throughput you get without them. Even then, I'd avoid Chip> disk-based cache systems, instead preferring Cache::* if it Chip> must be shared, or just global variables if it doesn't need to Chip> be. This is a good point. I'm working on securing access to the data stores this week, so I should be able to measure throughput soon. Loading the in-memory stores direct from the data sources is a good idea, if necessary. This precludes any local storage of the data which precludes any refresh to disk logic, although I still will need to flush/fill memory on a regular basis. Thus I don't want to utilize per-process data stores, but rather a common shared memory represenetation that all the modperl processes can draw from Chip> Can you be more specific about what the data looks like, where Chip> it resides, and how expensive loading it is? I wouldn't worry Chip> about optimization yet, unless you know beyond the shadow of a Chip> doubt speed will be a problem. My hunch is you can do maybe Chip> 50-100 hits/second on decent Intel hardware via the Frontier Chip> modules, so I don't think performance will be a problem. This Chip> is unverified, though; I really need to benchmark it sometime. Chip> Maybe others have pushed Frontier to its speed limits? I'll be able to provide some numbers and the actual data representation later on this week. I agree on performance numbers. I'm getting awesome throuput now acording to ApacheBench, but that's only with my skeletal handler. As it I fill it out, I plan on benchmarking each significant piece of functionality for comparison purposes. Should be fun. Chip> Chip Thanks so much for your insightful comments. I appreciate the time you took to respond to my queries. Peace.
