On Tue, Jun 28, 2005 at 04:15:15PM -0400, Patrick Dunnigan wrote:
> I am currently using SQLite to process @ 400 million records (and climbing)
> a day by reading files, importing them into SQLite, and summarizing. The
> summed data goes into Oracle. This is a production application that is very
Patrick Dunnigan wrote:
> I am currently using SQLite to process @ 400 million records (and climbing)
> a day by reading files, importing them into SQLite, and summarizing. The
Wow! How much memory those 400 milion of records is using ?
Do you use Sqlite 2 or 3 version ?
I was told that SQlite 3
: "Lorenzo Jorquera" <[EMAIL PROTECTED]>
To:
Sent: Monday, June 27, 2005 4:55 PM
Subject: [sqlite] using sqlite as a temporary database to process lots of
data
Hi,
My company is starting a project in wich we will have to process
large amounts of data and save the results to a d
> My company is starting a project in wich we will have to process
> large amounts of data and save the results to a db2 database.
> Because this data process needs to be very fast, the idea is to load
> the data in memory and apply the transformations in C++. I think that
> sqlite inmemory datab
On Tue, 28 Jun 2005, Rasmus Christian Kaae wrote:
>If you are running Windows you may use a similar memory-mapped file (see the
>Win32API for details).
>
>
>Does anyone know which is fastest -- Using a memory-mapped file (or
>/dev/shm) in comparison with sqlite's internal memory mapped tree?
Hav
If you are running Windows you may use a similar memory-mapped file (see the
Win32API for details).
Does anyone know which is fastest -- Using a memory-mapped file (or /dev/shm) in
comparison with sqlite's internal memory mapped tree?
--
Best regards / Med venlig hilsen
Rasmus Christian Kaae - [
If you are using Linux, you can use the /dev/shm. This is a memory
resident file system. I use this and find it about 200 times faster
than writing to disk. The only problem is that this is erased when the
server reboots. I hope this is of some use :)
Ben
Lorenzo Jorquera wrote:
Hi,
My
Hi,
My company is starting a project in wich we will have to process
large amounts of data and save the results to a db2 database.
Because this data process needs to be very fast, the idea is to load
the data in memory and apply the transformations in C++. I think that
sqlite inmemory databases c
8 matches
Mail list logo
