Maksim Yevmenkin wrote:
>
>>>so, just with plain ascii file i get four times the speed i get with
>>>sqlite. note that my c program will scale linearly with the size of
>>>dataset (just like i see with sqlite).
>>>
>>>
>> With anything related to computers, there are always tradeoffs - most
>>commonly power for complexity, and flexibility for speed. Your C
>>program *should* be faster than anything SQLite can do - it's simpler
>>and more specific to the problem you're trying to solve. On the flip
>>side, it'll never do anything other than what it already does - it can
>>never be used to solve any other problem.
>>
>>
>
>what you say is correct, but four (!) times performance increase?
>please, you have got to admit that something is not right here.
>
>
>
I have to agree with Thomas, your expectations are too high. If I'd be
using a relational database and I could get within a factor of 4 of what
I can concoct in C I'd declare victory. Relational databases are often
far from speedy, even on simple queries. You pay for:
* variable record formats
* integrity checks
* duplicate storage of keys in the BTree
* duplicate storage of keys in multiple tables
* the ACID property, even if you're not using it in your samples
* the ability to perform queries in a flexible way
* .....
If your database is simple you may be better off performance wise by
rolling your own solution, or using another database. MySQL is pretty
fast if you run it using ISAM tables, but you pay with data corruption
if the DB or system crashes.
If your queries generally produce a sizeable percentage of the records
stored you might as well do a sequential scan over a file, if written
with care, performance will be completely I/O bound. Use the 'mmap'
system call or equivalent to map the DB into memory, and you can read
your DB using pointer arithmetic, and use 'memmove' for updates.
Gé
