On 5/22/06, Dennis Jenkins <[EMAIL PROTECTED]> wrote:
Brannon King wrote: > The benefits I'm trying to get out of sqlite are the data queries. I > collect a large, sparse 2D array from hardware. The hardware device is > giving me a few GB of data data at 200MB/s. Future hardware versions > will be four times that fast and give me terabytes of data. After I > have the data, I then have to go through and make calculations on > sub-boxes of that data. (I'll post some more about that in a different > response.) I was trying to avoid coding my own > sparce-matrix-file-stream-mess that I would have to do if I didn't > have a nice DB engine. I think sqlite will work. I think it will be > fast enough. I'll have some nice RAID controllers on the production > machines with 48-256MB caches.Hello Brannon, I am simply curious. This sounds like an amazing engineering challenge. If it is not a secret, can you describe what this data represents and how it will be used?
Me too! The two largest database setups I've worked with: the total of all the call records for a major phone company, and the cumulative records of most of the drugs bought in the United States, don't add up to as much as this. What are you sampling at 1/200th of a microsecond?
