As a lot of you are already aware I am the guy behind feed-squirrel.com<http://www.feed-squirrel.com>, and I need a little help with a problem I am having.
Currently the site is aggregating ~500 RSS feeds, but checking these feeds is growing to be a pain in the butt. Having to get CF to check each of these feeds regulary (ideally every 15 minutes) is more difficult than it sounds. At the moment, the system is flip-flopping between two styles of RSS interrogation depending on which appears to be working best at the time. There are: 1) One big <CFLOOP> that goes through the feeds sequentially checking them and pulling in new content where necessary 2) Firing a load of threads off using an Async Event Gateway to individually check feeds and do their thing. Both of the above methods use the same function at their core. However, there are problems with these 1) takes about 12 minutes to run, which has caused CF to be unreliable in the past, and also caused HostMySite to raise it as a concern. 2) works beautifully when it's working, but every few hours, the gateway appears to just stop working for no good reason. I cannot be using an unreliable method of import. So, does anyone have any ideas on how else I might be able to import this data in a way that's relatively fast, and reliable? I do have a rudimentary pinging system implemented, but very few of the publishers know this exists. ANY help much appreciated... Regards Neil Middleton Visit feed-squirrel.com <http://www.feed-squirrel.com> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~| Message: http://www.houseoffusion.com/lists.cfm/link=i:4:238094 Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4 Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4 Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4 Donations & Support: http://www.houseoffusion.com/tiny.cfm/54
