what Robert suggest, is something exactly what the big-big photos gallery out there used to store their image. Distributing the image into many servers. But i am not very sure, how they store the image location information inside the database.
But i like the idea of using imageUId to determine where to store the image. I think, storing image in a bucket with 100 images each can make the image more manageable.. thanks for that idea.. On 7/11/08, Robert Cummings <[EMAIL PROTECTED]> wrote: > On Thu, 2008-07-10 at 15:46 +0800, paragasu wrote: >> i am planning to create a web photo gallery. i know there is a lot >> available out there, >> but i really want to create my own. the problem is not about creating >> the photo gallery. >> i want it to be scalable. >> >> the plan is saving the image metadata in the database and keep the >> original files >> in a folder. while it work perfectly, i afraid how many files 1 >> directory can keep. >> in 1 year, there going to be more than 1000 photo uploaded or more. >> sure it bring problem >> to maintain thus files. >> >> when i see big2 website out there like friendster, photobucket. the >> url for the image change >> like >> pic-1.domain.com/images/filename.jpg >> pic-2.domain.com/images/filename.jpg >> etc. they seems to put the image in different subdomain. Just wan't to >> ask anyone who >> did create a big-big web gallery like this. How did you do it? > > Those guys are probably having their images hit over a million times a > day. They also likely have millions of images. They are using multiple > servers to load balance the image serving. Whether this also is used to > distribute the images themselves is another question. > > In Linux a single directory can easily handle hundreds of thousands of > files. If you expect there to be million in short time, you can always > bucket the images. Bucketing involves generating some kind of ID that > can be used to generate bucket directories. For instance if you decide > you want 1000 buckets, you can an unique integer ID for images in the > database and modulo the uid by 1000 to determine the bucket directory. > This means if you have 100,000 images then there would be 1000 > directories and 100 images in each. As you grow to 1 million images you > would approach 1000 images in 1000 directories. Server offloading can > work similarly. And you can combine approaches. For instance if you have > 10 servers to dedicate to images then you can do the following: > > $serverId = $imageUid % 10; > $bucketId = $imageUid % 1000; > > $url = "http://image{$serverId}.myDomain.com"; > ."/{$bucketId}/{$imageFile}"; > > So for the first 20 images you would have: > > http://image1.myDomain.com/1/aaaaaa.jpg > http://image2.myDomain.com/2/aaaaab.jpg > http://image3.myDomain.com/3/aaaaac.jpg > http://image4.myDomain.com/4/aaaaad.jpg > http://image5.myDomain.com/5/aaaaae.jpg > http://image6.myDomain.com/6/aaaaaf.jpg > http://image7.myDomain.com/7/aaaaag.jpg > http://image8.myDomain.com/8/aaaaah.jpg > http://image9.myDomain.com/9/aaaaai.jpg > http://image10.myDomain.com/10/aaaaaj.jpg > http://image1.myDomain.com/11/aaaaak.jpg > http://image2.myDomain.com/12/aaaaal.jpg > http://image3.myDomain.com/13/aaaaam.jpg > http://image4.myDomain.com/14/aaaaan.jpg > http://image5.myDomain.com/15/aaaaao.jpg > http://image6.myDomain.com/16/aaaaap.jpg > http://image7.myDomain.com/17/aaaaaq.jpg > http://image8.myDomain.com/18/aaaaar.jpg > http://image9.myDomain.com/19/aaaaas.jpg > http://image10.myDomain.com/20/aaaaat.jpg > > Of course, if you have multiple servers, you need some mechanism on the > backend to migrate uploaded images to the appropriate server. > > Hope that helps. > > Cheers, > Rob. > -- > http://www.interjinn.com > Application and Templating Framework for PHP > >
