Re: [Radiant] Re: Multiple Server Deployment for Radiant

[William Ross]

On 27 Jun 2011, at 04:56, Shanison wrote:

 Hi William,
 
 Thank you very much for your detailed explanation and suggestion. This
 is very helpful. For my case, actually it is for railover, so I think
 I have to use rSync or write an extension that money patch the asset
 manager, where upon any files uploaded to the server, I can do a
 synchronization.
 
 This is a bit trouble though. We have also thought of storing all
 assets in the db instead. Synchronization of db is much easier and
 efficient. But this means that we have to money patch the asset
 manager again to make it store in db instead of file system.

That would probably be easier to code: it's just a different paperclip storage 
module. See eg.


http://patshaughnessy.net/2009/4/14/database-storage-for-paperclip-rewritten-to-use-a-single-table

but it doesn't feel like a very appealing solution. You would miss out on a lot 
of server-side optimisation: delivering static files is always the way to go if 
you can.

It should be fairly easy to trigger a script from the asset manager. Again, 
it's not our code but the behaviour of Paperclip that you need to change. Have 
a look at the flush_writes and flush_deletes methods in 
Paperclip::Storage::Filesystem. If I was doing this I think I'd add 
:after_storage and :after deletion callbacks.

best,

will

Re: [Radiant] Re: Multiple Server Deployment for Radiant

[Shanison]

Hi WIlliam,

Thanks a lot for the reference. Yes, static file would perform better 
compares to storing large BLOB in db. If i have to use the database method, 
i should think of a way of generating the files out from the db and serves 
as a cache and have to think of invalidating the cache if the files are 
updated.

It is a pity that there were not call back that we can hook up to do some 
file management after uploading or deleting. Have to alias the methods to 
achieve the purpose. 


Regards,
Shanison

[Radiant] Re: Multiple Server Deployment for Radiant

[Shanison]

Hi William,

Thank you very much for your detailed explanation and suggestion. This
is very helpful. For my case, actually it is for railover, so I think
I have to use rSync or write an extension that money patch the asset
manager, where upon any files uploaded to the server, I can do a
synchronization.

This is a bit trouble though. We have also thought of storing all
assets in the db instead. Synchronization of db is much easier and
efficient. But this means that we have to money patch the asset
manager again to make it store in db instead of file system.

Thanks,
Shanison

On Jun 20, 3:14 am, William Ross w...@spanner.org wrote:
 On 19 Jun 2011, at 16:54,Shanisonwrote:

  Hi all,

  I have one Radiant website but deployed to a few servers, and one of
  the server S1 will be used to serve static file content and serve as a
  reverse proxy and redirect dynamic page request back to the other
  servers S2, S3, S4. So my question is that when admin login to the
  admin page, it can be from any of the servers, S2 or S3, S4. If he
  uploads files e.g. images to S2, how can the rest of servers serves
  the files? If a request comes in to display a page and is served at
  S3, that images used in the page won't be displayed as it is not in
  the same server.

  Is there a way to synchronize it? Thank you.

 I've done this different ways in the past and the I think the right one would 
 depend on your reason for spreading across several servers in the first place:

 * If it's for load-sharing, and you can think of one of the servers as the 
 master, you may find it's enough just to redirect all requests for /admin and 
 /assets to the master.

 * if it's for redundancy and failover, you will need some scripting behind 
 radiant to move files around. Again, the method depends on your situation: 
 you can do it with rsync or an active push from the receiving server. It 
 would be easy to write an extension that triggered the mirroring script upon 
 image update, but note that you'd have to duplicate all the thumbnail sizes 
 as well as the original image.

 * If it's geographical, or the other two solutions don't appeal, you could 
 just use a CDN with uplift, or the built-in S3 support we get from paperclip.

 I think the most successful strategy I've used was to set up both database 
 servers (this was a rather paranoid and very busy site) as simple static-file 
 asset servers, and to direct all requests for static files from those two. 
 There were some rather basic rsync scripts to mirror uploaded assets. I 
 figured that if the database went down then the rest didn't really matter 
 anyway.

 If anyone knows a standard (cross-platform) way that we could meet this 
 requirement from within radiant's asset manager I'd be very happy to write it 
 in.

 For what it's worth, I don't deploy across multiple servers any more, apart 
 from a separate database host. More recently I've found that one decent 
 server with a long cache life is more efficient than several with only brief 
 caches (as they must be, because you're only ever clearing the cache on one 
 of them).

 best,

 will