> Yeah, it appears we're looking at two very different > beasts, and a solution > for one won't work on the other. I've pretty much only > ever used > infrequently updates caches, which puts a very different > spin on things > (including two layers, memory and filesystem, in some > cases), versus > frequently updated cacheing where the lifespan of a cached > item is measured > in minutes rather than days or weeks.
Well in most cases it's probably actually measured in hours to days (the exception is the few items being actively edited which probably do get updated every so many minutes to hours), but you're right, they're not the same animal. If I were only concerned about data cached in the scope of no less than days between updates, I'd be _much_ less concerned about the overhead involved in something like passing xml packets through a webservice to update a series of clustered applications. I still have some minor reservations about using cfhttp for it, but after the last couple emails even that's much less a concern for me now. I may even include the application sharing code in the TAPI open-source project once I get it all fleshed out. I've been contemplating adding some to the custom tags I use in Tapestry anyway, like the tag used for application cacheing. And the functions that support them... <cfmodule template="#request.tapi.cache()#" action="set" variable="application.blah.blah" objectdata="#mystruct#"> ... that tag handles all the locking and all the associative array (structure) work so that the variable attribute can contain spaces and punctuation, and it can be called from anywhere in the application because request.tapi.cache() uses request.tapi.getRelative() to create a relative path to the cache.cfm custom tag. I know a lot of people would just use <cf_cache blah blah> but I don't like using the custom tags directory for a handful of reasons. It's outside the application so if you move the app to another machine you have to remember to move all your custom tags and then there's the potential of there being a conflicting tag. Templates in the current directory can occlude the custom tag, so if there's a cache.cfm in the current directory that's suppposed to do something else, then you can't use <cf_cache > and you have to change it to <cfmodule name="cache"> and it may not be immediately obvious to other developers why you called it that way. Anyway, now I'm just rambling... :) > barneyb > --- > Barney Boisvert, Senior Development Engineer > AudienceCentral (formerly PIER System, Inc.) > [EMAIL PROTECTED] > voice : 360.671.8708 x12 > fax : 360.647.5351 > www.audiencecentral.com >> -----Original Message----- >> From: S. Isaac Dealey [mailto:[EMAIL PROTECTED] >> Sent: Wednesday, March 26, 2003 11:47 AM >> To: CF-Talk >> Subject: RE: J2EE session replication -- application >> replication? >> >> >> > I agree with the incremental regeneration of app vars. >> > Passing the whole >> > structure probably isn't smart, unless you need to. >> > I'm >> > not sure that >> > making each server recreate the data is a good idea, >> > though. I typically >> > put stuff in the app scope that is expensive to create >> > and >> > rarely changes. >> > Thus, passing around a large seralized packet isn't a >> > huge >> > deal, since it's >> > not going to happen all that often. And some >> > situations >> > (like content >> > caching) can probably go without synching, althuogh it >> > might be faster to >> > sync up, rather that regenerate on each server. >> >> Maybe I should elaborate a bit more. >> >> As an example, in my application there are a lot of >> objects drawn out of a >> view tap_Library, however, one of the columns in that >> view contains an xml >> packet. So because deserializing that xml packet is >> somewhat expensive, >> rather than deserializing it every time I want to get the >> object, >> I store it >> in an application variable >> application.myap.objects.F3UD3F-blah >> blah... you >> get the idea... it's a number or it's a UUID but it's >> stored in a >> structure >> there ... >> >> Now individual objects (these are content objects like >> Spectra has) may be >> updated "frequently" meaning daily or so or multiple >> times a day (you're >> working on a page and you make periodic edits throughout >> the day and you >> save changes any time you step away from the >> workstation)... but they're >> never going as frequent as the views for that object. You >> may >> open an object >> to look at it and then close it because it's not the one >> you needed, or >> someone else or a couple other people may open it in >> "read-only" >> mode at the >> same time, or people may view the "details" or fire other >> of its methods >> (add content, upload image), that don't update the object >> itself. >> >> It does use up some memory, but memory is less expensive >> than processors / >> processor speed, and certianly less expensive in terms of >> retreival than a >> combination cfquery / cfwddx|xmlparse, so there's a >> noticeable difference >> (believe me, on some pages it's the difference between 5 >> and 50 seconds to >> the end-user) when the data is cached. Which is why I've >> stuck with this >> cacheing routine for the past few years. >> >> My gut feeling about passing an xml packet through cfhttp >> to >> accomplish this >> is that it wouldn't speed anything up actually it could >> potentially slow >> things down. Lets say for instance you have 5 servers, >> and the object is >> being viewed on 2 and it gets updated on 1. The routine >> would pass the xml >> packet to all 4 other servers. One of them would have to >> deserialize the >> object and store it in the application scope. The other 3 >> may or >> may not. It >> could be wasted processor power and memory if they do >> since no >> one can tell >> if someone else is going to view the object on that >> server. Even if they >> don't deserialize the xml packet then you've spent what >> could in the long >> run turn out to be considerable bandwidth passing around >> those xml packets >> between servers when you have dozens or hundreds of >> people >> working with the >> content on 4 or 5 servers. >> >> > The "processing after CFFLUSH in OnRequestEnd" idea is >> > a >> > damn good one that I'd never though of. Being a FB >> > guy, >> > OnRequestEnd.cfm is usually reserved for debugigng >> > output >> > during development and such, but that's a >> > very good use for it. >> >> Thanks. :) Yea, FB sort of does it's own thing. >> >> Interrestingly enough (and I'm sure some folks on this >> list will think I'm >> smoking crack) in the prototype for the next version of >> Tapestry I've >> started working with a file-upload routine which uses >> OnRequestEnd.cfm to >> ensure that files being uploaded for new objects are >> properly associated >> with the object in the db and stored in the proper >> directory in the file >> system. The problem was that the way the application is >> designed, >> the files >> would be uploaded before the object was inserted into the >> db, so no >> relationship could be established at the time the object >> is uploaded >> (violation of foreign key constraint). >> >> So instead what I've done is I've established the >> object's new ID (UUID), >> uploaded the file and stored information about it in the >> request >> scope. Then >> in OnRequestEnd.cfm after the db record for the object >> has been >> inserted it >> moves those files from a user-specific upload directory >> to the appropriate >> directory for the object and inserts the related record >> in the database. >> There's a try-catch block around each file insert/move so >> that if it fails >> on one it doesn't kill the rest of the OnRequestEnd code, >> and >> then it scans >> the user-upload directory and attempts to delete any >> files that are left >> there for garbage collection. >> >> > barneyb >> >> > --- >> > Barney Boisvert, Senior Development Engineer >> > AudienceCentral (formerly PIER System, Inc.) >> > [EMAIL PROTECTED] >> > voice : 360.671.8708 x12 >> > fax : 360.647.5351 >> >> > www.audiencecentral.com >> >> >> -----Original Message----- >> >> From: S. Isaac Dealey [mailto:[EMAIL PROTECTED] >> >> Sent: Wednesday, March 26, 2003 10:45 AM >> >> To: CF-Talk >> >> Subject: RE: J2EE session replication -- application >> >> replication? >> >> >> >> >> >> > I know the use of an HTTP request was previously >> >> > marked >> >> > as >> >> > a poor idea, however I'm not sure I agree as I've >> >> > thought >> >> > this through. Here's a solution that I think would >> >> > work >> >> > quite well, assuming you encapsulate all your app >> >> > vars >> >> > in >> >> > a single object (which is a good idea anyway, IMO). >> >> > Of >> >> > course, I might be suggesting rebuilding JMS, which >> >> > I >> >> > know >> >> > exactly nothing about, but oh well. >> >> >> >> Yea, my one large project (Tapestry) does encapsulate >> >> all >> >> its application >> >> variables into its own structure, however, in this >> >> case >> >> it >> >> doesn't help much >> >> using what you've described below, and I'll explain >> >> why >> >> here in a moment. >> >> >> >> > - server 1 learns it needs to update it's app vars >> >> > (exactly how is app dependant). >> >> > - server 1 updates the app vars object however it >> >> > needs to. >> >> >> >> This much is already being done prior to clustering, >> >> so >> >> we're okay here. >> >> >> >> > - server 1 serializes the app vars object >> >> > (probably want XML, rather than the built in >> >> > binary >> >> > format). >> >> >> >> That's an idea... Although if the application is >> >> rather >> >> large >> >> (and caches a >> >> lot of data), serializing the structure into an xml >> >> packet >> >> whether it's wddx >> >> or otherwise could be rather time consuming. At some >> >> point, xml >> >> serialization then defeats the purpose of cacheing >> >> data >> >> in the application >> >> scope instead of going to the db server every time, so >> >> imho it's >> >> much better >> >> to just deal with the data that's been changed. It >> >> could >> >> be that you pass >> >> both an xml packet and a string representing the path >> >> to >> >> the data you're >> >> updating, such as >> >> >> >> <xml blah blah> ... </xml> >> >> application.myapp.someobject.stuff.3XLN7 >> >> >> >> and the clustered app receiving this messag then >> >> coppies >> >> the deserialized >> >> content of that xml packet into the indicated >> >> variable. >> >> Should take a lot >> >> less time than serializing the entire >> >> application.myapp >> >> structure. >> >> >> >> > - server 1 gets a list of cluster members (exactly >> >> > how >> >> > will be app/container/cluster controller >> >> > dependant). >> >> >> >> I'd go for a cachedafter query here. This allows me to >> >> not have to >> >> continually go to the db to get the list of server and >> >> use the same >> >> application sharing routine to flush the cachedafter >> >> timestamp and update >> >> the query to then add a new server to the list without >> >> having to >> >> do anything >> >> special like restarting the application or worse >> >> restarting the cf server. >> >> >> >> > - for each server, server 1 calls a secured web >> >> > service, >> >> > passing the serialized app vars object, which the >> >> > other servers then >> >> > deserialize and store in their app scope. >> >> >> >> I don't know that it needs to be a web service. I >> >> could >> >> do the same thing >> >> with CF5 and "secure" it by using a password or >> >> password-like >> >> algorithm, or >> >> better yet by only allowing other servers in the >> >> cluster >> >> to submit to the >> >> page receiving the data using the same server list >> >> that >> >> was used on the >> >> sending end and matching that list against the >> >> cgi.remote_addr >> >> variable. And >> >> what might actually be a best case scenario is to have >> >> each individual >> >> server in the cluster have a single receive-from and >> >> send-to address, such >> >> that the cluster is linked in a ring like a token-ring >> >> network or a >> >> web-ring. ;P The signal sent from the originating >> >> server >> >> is sent >> >> to only one >> >> machine which then receives it and serializes it in a >> >> queue using <cflock >> >> name="[myappname]_appshare"> with a long timeout limit >> >> in >> >> addition to any >> >> locking on the application scope -- this would just >> >> ensure that all the >> >> appshare commands are processed in sequential order. >> >> If a >> >> given machine in >> >> the ring is down or otherwise inaccessible, the >> >> machine >> >> attempting to send >> >> to it moves on to the next machine in the list when it >> >> doesn't get an "ok" >> >> response, so that any individual server in the ring >> >> won't >> >> break the entire >> >> mechanism. The ring itself ensures that the load >> >> created >> >> by the routine is >> >> distributed evenly amongst the servers and that >> >> refresh >> >> commands are >> >> processed in sequence. And this of course solves the >> >> problem described >> >> below. >> >> >> >> > The last piece of the puzzle would be making sure >> >> > more >> >> > than one server >> >> > doesn't start that process at any given time, but >> >> > that >> >> > shouldn't be a big >> >> > deal. Call another web service method after step >> >> > one >> >> > that >> >> > tells all servers >> >> > not to start updating, and handle collisions (if >> >> > two >> >> > servers start down >> >> > that new step at the same time) by alphabetic >> >> > ordering >> >> > of >> >> > server name or >> >> > something. >> >> >> >> > Feel free to tell me I'm a jackass, but that seems >> >> > to >> >> > me >> >> > like it'd work >> >> > well, and be very app independant, so it could be >> >> > reused >> >> > over and over. >> >> >> >> Not a jackass, it was much the same sort of mechanism >> >> I >> >> had in mind but >> >> wasn't entirely keen on using. Although now that I've >> >> read yours and put >> >> some more thought into it I'm a little less anxious >> >> about >> >> actually >> >> implementing it. >> >> >> >> I'm still concerned about sending an xml packet >> >> between >> >> the machines and I >> >> think I have a solution for that as well. All servers >> >> in >> >> the cluster >> >> retrieve their cache data initially from the same >> >> source >> >> (a single db or a >> >> clustered db, but it's all the same data in either >> >> case). >> >> That >> >> data from the >> >> db is then cached in the application scope. And in the >> >> case of my >> >> application in many instances it's already choosing to >> >> simply delete data >> >> from the application scope rather than writing over >> >> it, >> >> which then forces >> >> the application to go back to the db when the data >> >> doesn't exist in the >> >> application scope. So the simplest solution (without >> >> going to JMS) is to >> >> send a one-line command to each machine in the ring >> >> with >> >> two variables >> >> indicating the originating server and the application >> >> data to refresh. The >> >> receiving server deletes the variable, sends the "ok" >> >> response >> >> back and then >> >> sends the same command on to the next server in the >> >> list >> >> and waits for the >> >> ok back. If it doesn't receive the ok, it moves on to >> >> the >> >> next >> >> server in the >> >> ring until it receives an ok or reaches the >> >> originating >> >> server. >> >> >> >> And best of all, to prevent this process from slowing >> >> the >> >> users down, you >> >> run the routine from the originating server using >> >> request >> >> scope >> >> data in the >> >> OnRequestEnd.cfm template after a <cfflush> tag. Only >> >> the >> >> first request >> >> occurs on a user requested page (and that only after >> >> the >> >> page has been >> >> delivered) -- remaining requests occur in the "ether" >> >> between the machines >> >> and so no one is affected (speed of delivery) by those >> >> transactions. And >> >> since the data's being deleted and then automatically >> >> recreated >> >> from source >> >> on demand, you don't have to worry about anyone having >> >> cache data that's >> >> "out of sequence". >> >> >> >> s. isaac dealey 954-776-0046 >> >> >> >> new epoch http://www.turnkey.to >> >> >> >> lead architect, tapestry cms >> >> http://products.turnkey.to >> >> >> >> tapestry api is opensource >> >> http://www.turnkey.to/tapi >> >> >> >> certified advanced coldfusion 5 developer >> >> http://www.macromedia.com/v1/handlers/index.cfm?ID=218 >> >> 16 >> >> >> >> >> > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ >> > ~~~ >> > ~~~~~~~~~~~| >> > Archives: >> > http://www.houseoffusion.com/cf_lists/index.cfm?forumid >> > =4 >> > Subscription: >> > http://www.houseoffusion.com/cf_lists/index. >> > cfm?method=subscribe&forumid=4 >> > FAQ: http://www.thenetprofits.co.uk/coldfusion/faq >> > Get the mailserver that powers this list at >> > http://www.coolfusion.com >> >> > Unsubscribe: > http://www.houseoffusion.com/cf_lists/uns >> ubscribe.cfm?user=633.558.4 > s. isaac dealey 954-776-0046 > new epoch http://www.turnkey.to > lead architect, tapestry cms http://products.turnkey.to > tapestry api is opensource http://www.turnkey.to/tapi > certified advanced coldfusion 5 developer > http://www.macromedia.com/v1/handlers/index.cfm?ID=21816 > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > ~~~~~~~~~~~| > Archives: > http://www.houseoffusion.com/cf_lists/index.cfm?forumid=4 > Subscription: http://www.houseoffusion.com/cf_lists/index. > cfm?method=subscribe&forumid=4 > FAQ: http://www.thenetprofits.co.uk/coldfusion/faq > Your ad could be here. Monies from ads go to support these > lists and provide more resources for the community. > http://www.fusionauthority.com/ads.cfm > Unsubscribe: http://www.houseoffusion.com/cf_lists/uns > ubscribe.cfm?user=633.558.4 s. isaac dealey 954-776-0046 new epoch http://www.turnkey.to lead architect, tapestry cms http://products.turnkey.to tapestry api is opensource http://www.turnkey.to/tapi certified advanced coldfusion 5 developer http://www.macromedia.com/v1/handlers/index.cfm?ID=21816 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~| Archives: http://www.houseoffusion.com/cf_lists/index.cfm?forumid=4 Subscription: http://www.houseoffusion.com/cf_lists/index.cfm?method=subscribe&forumid=4 FAQ: http://www.thenetprofits.co.uk/coldfusion/faq Signup for the Fusion Authority news alert and keep up with the latest news in ColdFusion and related topics. http://www.fusionauthority.com/signup.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
