Re: [Tutor] File storage vs DB storage.
Thanks, gentlemen. Indeed, using directories may be a better bet when there are many files involved. At least, this has been my feeling, and i gather the same from your replies. I'll be using Lighttpd + FastCGI + Python + MySQL (maybe PostGRE here) on Win. XP SP2. Any known caveats of this combo before i plunge? Thanks again Lloyd On 4/10/07, Alan Gauld [EMAIL PROTECTED] wrote: OkaMthembo [EMAIL PROTECTED] wrote Ok, to get to the point: when building a web app that should store an retrieve photos, what would be more efficient between the app storing the files on a local hard disk, or saving blobs/ binary data to a database? Someone toldme that database storage should be avoided as it would cause a botleneck, I wouldn't say a bottleneck but it can cause problems by making the database itself very big. That can cause problems for some operations, including admin tasks. Some databases work better with blobs than others. whereas i dont want to imagine the nightmare of managing tons of files sitting in a folder. I use Windows XP. Naming issues are usually the biggest problem. It will work best if you use a numrerical naming convention and manage all of the meta data in the database, along with the filename. The advantage of using the file system is that you can use different folders to help organise the files - folders for month created or submitter or whatever. Also you can use the facilities of the OS to create encrypted folders, secure folders, compressed folders, move the folders onto other drives as the capacity expands, do incremental backups etc etc. HTH, -- Alan Gauld Author of the Learn to Program web site http://www.freenetpages.co.uk/hp/alan.gauld ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor -- The Stupidry Foundry ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] File storage vs DB storage.
OkaMthembo [EMAIL PROTECTED] wrote I'll be using Lighttpd + FastCGI + Python + MySQL on Win. XP SP2. Any known caveats of this combo before i plunge? That should be fine I think. Alan G. ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] File storage vs DB storage.
OkaMthembo schrieb: Indeed, using directories may be a better bet when there are many files involved. At least, this has been my feeling, and i gather the same from your replies. I suggest that you take a look at a database abstraction layer or ORM like SQLAlchemy [1], that makes handling the data layer much easier and also allows you to map different data sources (e.g. file system/database) onto the same data object very easily. For example, there is a recipe in the SQLObject docs [2] on how to store images in the filesystem but access them transparently via object properties (aka dynamic attributes). This recipe could be easily applied to SQLAlchemy mapper objects as well. Chris [1] http://sqlalchemy.org [2] http://sqlobject.org/SQLObject.html#adding-magic-attributes-properties ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
[Tutor] File storage vs DB storage.
Hello all, Im a newbie and am currently going through the Python Tutorial for the second time. At work, i use C#, but what a delight Python is! Ok, to get to the point: when building a web app that should store an retrieve photos, what would be more efficient between the app storing the files on a local hard disk, or saving blobs/ binary data to a database? Someone toldme that database storage should be avoided as it would cause a botleneck, whereas i dont want to imagine the nightmare of managing tons of files sitting in a folder. I use Windows XP. Cheers, -- The Stupidry Foundry ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] File storage vs DB storage.
Hi, Managaing a lot of files is not a nightmare if you use/build the right tools. Why not saving filenames in the DB and then just using the ordinary web server to deliver them? Getting the blob out of the DB and then serving it through the web app is going to be a lot slower than letting, say, Apache do it. I wrote a system that kept 15000 files in a directory tree. You just let your software deal with them Hugo ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] File storage vs DB storage.
OkaMthembo [EMAIL PROTECTED] wrote Ok, to get to the point: when building a web app that should store an retrieve photos, what would be more efficient between the app storing the files on a local hard disk, or saving blobs/ binary data to a database? Someone toldme that database storage should be avoided as it would cause a botleneck, I wouldn't say a bottleneck but it can cause problems by making the database itself very big. That can cause problems for some operations, including admin tasks. Some databases work better with blobs than others. whereas i dont want to imagine the nightmare of managing tons of files sitting in a folder. I use Windows XP. Naming issues are usually the biggest problem. It will work best if you use a numrerical naming convention and manage all of the meta data in the database, along with the filename. The advantage of using the file system is that you can use different folders to help organise the files - folders for month created or submitter or whatever. Also you can use the facilities of the OS to create encrypted folders, secure folders, compressed folders, move the folders onto other drives as the capacity expands, do incremental backups etc etc. HTH, -- Alan Gauld Author of the Learn to Program web site http://www.freenetpages.co.uk/hp/alan.gauld ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
