That's not a bad idea at all and I'll check it out. However, since the data is written from a client, I can only do arbitrary chopping without separating them in a sensible manner. Maybe I don't need it though, as I could use it for setting up a pageing system in memory. Thanks!!!
2007/3/18, Teg <[EMAIL PROTECTED]>:
Hello Dimitris, If I was going to do this, I'd chop up the binary file in some manageable length like, say 1 meg, insert each chunk as a blob,including an index record for each chunk so, you can select them in order and re-assemble them piece by piece as you enumerate through the records. In that way you never have to hold the entire 200 meg file in memory and you get some kind of random access. Basically you're using SQlite as a file system and each record becomes a "cluster". I think that's very doable as far as storage is concerned. Don't know about the locking part. Since the bottleneck is the disk drive, I'd probably use a single worker and a queue to serialize access to the DB. C Saturday, March 17, 2007, 7:06:46 PM, you wrote: DPS> Hi all, DPS> I want to do the following; save a set of 100-200 (or even more) binary DPS> files into a single DB file. The binary files are of 100-200 (or a bit DPS> more :-) ) MB each. My requirements are: DPS> 1) Many clients should be able to connect to the database to save their DPS> file. Clients are actually client programs that calculate the binary DPS> file. So the db server must be able to handle the concurrency of requests. DPS> 2) The file would be portable and movable (i.e. copy-paste will do, no DPS> special arrangement to move around) DPS> Ideally I would like to provide client programs with a stream to read DPS> and write files. I guess the files should be stored as blob records in a DPS> single table within the database. So my question is whether all this is DPS> possible, since I am not very familiar with SQLite (I have been DPS> redirected here). Just to straighten things, I know this is not the DPS> orthodox use of a DBMS, i.e. I should store my nice scientific data in DPS> tables and define good relations and suff. I really do believe in this DPS> scheme. However at this point, I have to provide evidence that such an DPS> anorthodox solution is also feasible (not to mention that I have to DPS> compare access performance with flat binary files :-/ ). DPS> TIA DPS> -- ds DPS> ----------------------------------------------------------------------------- DPS> To unsubscribe, send email to [EMAIL PROTECTED] DPS> ----------------------------------------------------------------------------- -- Best regards, Teg mailto:[EMAIL PROTECTED] ----------------------------------------------------------------------------- To unsubscribe, send email to [EMAIL PROTECTED] -----------------------------------------------------------------------------
