Rick is dead on correct, I call I chunking blob data.. There is an article here on a simple implementation:
http://www.dreamwerx.net/phpforum/?id=1 I've had hundreds of thousands of files in this type of storage before with no issues. On Tue, 3 Jul 2007, Rick James wrote: > I gave up on putting large blobs in Mysql -- too many limits around 16MB. > > Instead I broke blobs into pieces, inserting them with a sequence number. > > Added benefit: Does not clog up replication while huge single-insert is > being copied over network and reexecuted on slaves. > > > -----Original Message----- > > From: Paul McCullagh [mailto:[EMAIL PROTECTED] > > Sent: Wednesday, June 27, 2007 2:57 AM > > To: Ann W. Harrison > > Cc: MySQL List; MySQL Internal > > Subject: Re: Blob data > > > > Hi Ann, > > > > Currently, the thoughts on how to make the BLOB references secure go > > like this: > > > > The BLOB reference consists of 2 components: The first component is > > basically an index used to find the BLOB on the server. The second > > component is a random number generated when the BLOB is created. > > > > The random number acts as an "authorization code", and is checked > > when the BLOB is requested. So if the authorization code supplied in > > the BLOB reference does not match the code stored by the server for > > that BLOB, then the BLOB is not returned. > > > > If the authorization code is a 4-byte number, then the chances of > > getting the correct code for any particular BLOB is 1 in 4 billion. > > This makes it practically impossible to "discover" a BLOB by > > generating BLOB references and requesting them from the server. > > > > However, it does mean that once you have a valid BLOB reference it > > remains valid until the BLOB is deleted. So you can pass it > > around to > > your friends, or post it on the internet if you like. > > > > In order to prevent this (it will depend on the site, as to whether > > this is required), it would be possible to add a dynamic > > component to > > the BLOB reference which has a certain lifetime (for example, it > > expires after a certain amount of time, or when a database > > session is > > closed). > > > > Such a component would have to be added to the BLOB reference URL by > > the storage engine on the fly. So, as the SELECT result is being > > generated, the dynamic component is added to the BLOB references > > returned in the rowset. > > > > Security of the BLOB streaming stuff is one of the major issues, so > > further comments, questions and ideas are welcome! > > > > Best regards, > > > > Paul > > > > On Jun 26, 2007, at 4:36 PM, Ann W. Harrison wrote: > > > > > Paul McCullagh wrote: > > >> > > >> It will also be possible to store the BLOBs "out-of-row". In this > > >> case, only a BLOB reference is stored in the row. The > > reference is > > >> basically a URL which can be used to retrieve the data. So when > > >> you do an SQL SELECT which includes a BLOB column, the resulting > > >> rowset does not contain the data, just the BLOB reference (URL). > > > > > > How does this work with access privileges? Can you just send random > > > numbers in the URL until you start seeing blob data? > > > > > > Best regards, > > > > > > > > > Ann > > > > > > -- > > MySQL Internals Mailing List > > For list archives: http://lists.mysql.com/internals > > To unsubscribe: > > http://lists.mysql.com/[EMAIL PROTECTED] > > > > > > > -- > MySQL General Mailing List > For list archives: http://lists.mysql.com/mysql > To unsubscribe: http://lists.mysql.com/[EMAIL PROTECTED] > -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe: http://lists.mysql.com/[EMAIL PROTECTED]
