RE: Serving Files via cfcontent - how much processing is required
Mark: >> otherwise I would say use mod_xsendfile I have sort of gone down the IIS road but have found a company that has an Apache emulator ISAPI that is willing to add that functionality to their product. http://www.helicontech.com/ape/ I will see what they come up with. Dave: >> A symlink is just a pointer. >> http://en.wikipedia.org/wiki/NTFS_symbolic_link I was indeed misunderstanding your original post and its intent. This may have real possibilities and I will have to look further into testing the practical application of generating and deleting them programmatically. I still favor the mod_x-sendfile methodology available in Apache which puts the burden on the correct layer, the web server, to do the heavy lifting and will have to see if it can be added to IIS. William: >> I would highly recommend using a Content Delivery Network like Mosso Files to store your files Over the past 3 years I have looked at various Content delivery services, although not at Mosso, and all have been a magnitude more expensive at the volume we run over our current methodology and with no way to control costs. In this particular endeavor we have more time than money to spend. The intent was always break even. And while not perfect, close enough is better than all others have done in this particular niche market (Halo Custom Edition). Gentlemen, Thank you for your responses they have all helped and given me much to think about and digest. Thanks again. Dennis Powers UXB Internet - A Website Design & Hosting Company P.O. Box 6028 Wolcott, CT 06716 203-879-2844 http://www.uxbinternet.com ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology-Michael-Dinowitz/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:334659 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: Serving Files via cfcontent - how much processing is required
Sent from my iPhone On Jun 17, 2010, at 4:49 PM, Dave Watts wrote: > >> In other applications I routinely use cfcontent to serve protected >> files on >> extranet applications however the traffic ( 10-20 files/day) is >> nowhere near >> as rigorous as will be required here with 12,000 per day of 40Meg >> average >> per file. I am considering serving the files now through cfcontent >> via HTTP >> instead of FTP for a couple of reasons. 1) because most Internet >> Security >> programs block FTP and we have to help people (mostly kids) open >> the port >> and 2) to prevent direct linking to the files because this endeavor >> is >> funded by ads on the website (and my wallet). >> >> I know the most efficient way to serve this quantity and size of >> files is >> via ftp but what I don't know is what is required by the various CF >> engines >> AdobeCF, OpenBD, Bluedragon, Ralio to serve up the same via HTTP. >> Will our >> new server hardware handle that kind of HTTP file traffic (I >> suspect so), >> will OpenBD/Tomcat be up to the task or will I need a different >> CFML engine? >> Essentially what I need to know is what it would take to routinely >> serve >> that many/size files through the CF engine. I don't want to go down >> this >> road and find that people are having problems downloading because >> the CF >> engine / Web server can't keep up. Has anyone had experience with >> this and >> can you offer some advice? > > My advice for you would be to avoid using CFCONTENT for this, as it's > really not designed for this. Each request using CFCONTENT will use > one of your threads, I think, and there are decent alternatives. > > The alternative I'd recommend would be the use of temporary symlinks. > When someone is granted access to a file, you'd create a symlink for > that file, pointing to a web-accessible location, then let the user > download it via HTTP. Sometime after, you'd delete the symlink. > Exactly how long after, I'm not sure - that would be a balance of > convenience to the user (in case they don't download it immediately) > versus the potential for abuse (the user provides the link to someone > else). > > Dave Watts, CTO, Fig Leaf Software > http://www.figleaf.com/ > http://training.figleaf.com/ > > Fig Leaf Software is a Veteran-Owned Small Business (VOSB) on > GSA Schedule, and provides the highest caliber vendor-authorized > instruction at our training centers, online, or onsi > I would highly recommend using a Content Delivery Network like Mosso Files to store your files, then you can symlink to them by passing headers as a re-direct. This way you don't have the overhead of distribution, hardware and requests... You pay a small fee per month for transfer and storage. Do a cost analysis, including your time and sys admin time in upgrades to handle all of this badwidth. Check out the CDN writeup on openboxitsolutions.com for some real life examples and price savings.. Feel free to reach out to me, as well for more information. I highly recommend not hosting static delivery especially to the extent you're looking at without weighing CDN options. William Attwood Systems Engineer Software Engineer 801-859-2987 > ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology-Michael-Dinowitz/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:334634 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: Serving Files via cfcontent - how much processing is required
> > Well, creating a symlink is a very easy thing, and it doesn't take > > nearly as long as serving a file > > Maybe I am misunderstanding this concept. Could you provide a loose example. A symlink is just a pointer. When you create one, you're allowing access to the file from two different locations on the filesystem. You're not copying the file. If you have a 2KB and a 2GB file, creating symlinks to both would take the same amount of time. Your CF script could do this (via CFEXECUTE), present the link to the user, then exit and let the user download the file via HTTP. http://en.wikipedia.org/wiki/NTFS_symbolic_link Dave Watts, CTO, Fig Leaf Software http://www.figleaf.com/ http://training.figleaf.com/ Fig Leaf Software is a Veteran-Owned Small Business (VOSB) on GSA Schedule, and provides the highest caliber vendor-authorized instruction at our training centers, online, or onsite. ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology-Michael-Dinowitz/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:334633 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
RE: Serving Files via cfcontent - how much processing is required
>> Well, creating a symlink is a very easy thing, and it doesn't take >> nearly as long as serving a file Maybe I am misunderstanding this concept. Could you provide a loose example. Thanks. ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology-Michael-Dinowitz/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:334631 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: Serving Files via cfcontent - how much processing is required
> This was another thought as well but the issue there is moving those files > back and forth programmatically with CF. I thought that it would use as much > processor/threads as actually serving it up. Well, creating a symlink is a very easy thing, and it doesn't take nearly as long as serving a file. > I see PHP sites do it but I am not sure if they have other ways to handle > the actual transfer of the file via the web server with a re-direct utility > or something. > > After I sent the email I remembered that the CoolFusion guys had an > authenticator ISAPI filter for IIS which essentially runs a CF script before > allowing access to the file through the web server. I just don't know if it > will work on 64bit IIS7 and am waiting for their reply via email. Are they still in business? The mod_xsendfile Apache module looks like a good way to handle this, as Mark mentioned. I'm not aware of any port of that to IIS though. Dave Watts, CTO, Fig Leaf Software http://www.figleaf.com/ http://training.figleaf.com/ Fig Leaf Software is a Veteran-Owned Small Business (VOSB) on GSA Schedule, and provides the highest caliber vendor-authorized instruction at our training centers, online, or onsite ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology-Michael-Dinowitz/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:334630 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
RE: Serving Files via cfcontent - how much processing is required
>> My advice for you would be to avoid using CFCONTENT for this, as it's >> really not designed for this. That was my feeling as well. I have always tried to stay away from much file manipulation with the CF engine since, as you say, it wasn't designed for it. >> The alternative I'd recommend would be the use of temporary symlinks. This was another thought as well but the issue there is moving those files back and forth programmatically with CF. I thought that it would use as much processor/threads as actually serving it up. I see PHP sites do it but I am not sure if they have other ways to handle the actual transfer of the file via the web server with a re-direct utility or something. After I sent the email I remembered that the CoolFusion guys had an authenticator ISAPI filter for IIS which essentially runs a CF script before allowing access to the file through the web server. I just don't know if it will work on 64bit IIS7 and am waiting for their reply via email. I am looking for any and all ideas and opinions on this subject since I have yet to find a full solution. Thanks! Dennis Powers UXB Internet - A Website Design & Hosting Company P.O. Box 6028 Wolcott, CT 06716 203-879-2844 http://www.uxbinternet.com ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology-Michael-Dinowitz/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:334629 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: Serving Files via cfcontent - how much processing is required
Doh, I just realised you're not running apache, otherwise I would say use mod_xsendfile: http://tn123.ath.cx/mod_xsendfile/ Maybe there is something similar for IIS? Mark On Fri, Jun 18, 2010 at 8:49 AM, Dave Watts wrote: > > > In other applications I routinely use cfcontent to serve protected files > on > > extranet applications however the traffic ( 10-20 files/day) is nowhere > near > > as rigorous as will be required here with 12,000 per day of 40Meg average > > per file. I am considering serving the files now through cfcontent via > HTTP > > instead of FTP for a couple of reasons. 1) because most Internet Security > > programs block FTP and we have to help people (mostly kids) open the port > > and 2) to prevent direct linking to the files because this endeavor is > > funded by ads on the website (and my wallet). > > > > I know the most efficient way to serve this quantity and size of files is > > via ftp but what I don't know is what is required by the various CF > engines > > AdobeCF, OpenBD, Bluedragon, Ralio to serve up the same via HTTP. Will > our > > new server hardware handle that kind of HTTP file traffic (I suspect so), > > will OpenBD/Tomcat be up to the task or will I need a different CFML > engine? > > Essentially what I need to know is what it would take to routinely serve > > that many/size files through the CF engine. I don't want to go down this > > road and find that people are having problems downloading because the CF > > engine / Web server can't keep up. Has anyone had experience with this > and > > can you offer some advice? > > My advice for you would be to avoid using CFCONTENT for this, as it's > really not designed for this. Each request using CFCONTENT will use > one of your threads, I think, and there are decent alternatives. > > The alternative I'd recommend would be the use of temporary symlinks. > When someone is granted access to a file, you'd create a symlink for > that file, pointing to a web-accessible location, then let the user > download it via HTTP. Sometime after, you'd delete the symlink. > Exactly how long after, I'm not sure - that would be a balance of > convenience to the user (in case they don't download it immediately) > versus the potential for abuse (the user provides the link to someone > else). > > Dave Watts, CTO, Fig Leaf Software > http://www.figleaf.com/ > http://training.figleaf.com/ > > Fig Leaf Software is a Veteran-Owned Small Business (VOSB) on > GSA Schedule, and provides the highest caliber vendor-authorized > instruction at our training centers, online, or onsi > > ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology-Michael-Dinowitz/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:334628 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
Re: Serving Files via cfcontent - how much processing is required
> In other applications I routinely use cfcontent to serve protected files on > extranet applications however the traffic ( 10-20 files/day) is nowhere near > as rigorous as will be required here with 12,000 per day of 40Meg average > per file. I am considering serving the files now through cfcontent via HTTP > instead of FTP for a couple of reasons. 1) because most Internet Security > programs block FTP and we have to help people (mostly kids) open the port > and 2) to prevent direct linking to the files because this endeavor is > funded by ads on the website (and my wallet). > > I know the most efficient way to serve this quantity and size of files is > via ftp but what I don't know is what is required by the various CF engines > AdobeCF, OpenBD, Bluedragon, Ralio to serve up the same via HTTP. Will our > new server hardware handle that kind of HTTP file traffic (I suspect so), > will OpenBD/Tomcat be up to the task or will I need a different CFML engine? > Essentially what I need to know is what it would take to routinely serve > that many/size files through the CF engine. I don't want to go down this > road and find that people are having problems downloading because the CF > engine / Web server can't keep up. Has anyone had experience with this and > can you offer some advice? My advice for you would be to avoid using CFCONTENT for this, as it's really not designed for this. Each request using CFCONTENT will use one of your threads, I think, and there are decent alternatives. The alternative I'd recommend would be the use of temporary symlinks. When someone is granted access to a file, you'd create a symlink for that file, pointing to a web-accessible location, then let the user download it via HTTP. Sometime after, you'd delete the symlink. Exactly how long after, I'm not sure - that would be a balance of convenience to the user (in case they don't download it immediately) versus the potential for abuse (the user provides the link to someone else). Dave Watts, CTO, Fig Leaf Software http://www.figleaf.com/ http://training.figleaf.com/ Fig Leaf Software is a Veteran-Owned Small Business (VOSB) on GSA Schedule, and provides the highest caliber vendor-authorized instruction at our training centers, online, or onsi ~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology-Michael-Dinowitz/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:334627 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
