Re: [sqlite] How to decrease IO usage

2013-01-22 Thread Eric Grange
Then, do you know a way to optimize my usage ? I suppose you have a pattern to which database are accessed at any point in time? If so rather than opening and closing databases directly, you could pool them, so that if an operation is needed on a recently opened database, it will still be o

Re: [sqlite] How to decrease IO usage

2013-01-11 Thread Simon Slavin
On 10 Jan 2013, at 2:42pm, Bebel wrote: > I include all the operation made by my action (so open/close are included). > > I create one base per user, so I prefer not to merge database to make one > bigger. Okay. This means that your enquiry is not the standard one of "faster operations at al

Re: [sqlite] How to decrease IO usage

2013-01-11 Thread Bebel
Hi Simon, I include all the operation made by my action (so open/close are included). I create one base per user, so I prefer not to merge database to make one bigger. Thanks for your help. -- View this message in context: http://sqlite.1065341.n5.nabble.com/How-to-decrease-IO-usage-tp66474p

Re: [sqlite] How to decrease IO usage

2013-01-11 Thread Bebel
Hi Igor, Well it's a interesting solution, and I already explored this possibility. Now, it will be too expansive and complicated for us to migrate data from sqlite to nosql solution. We choose sqlite about 5 years ago, when nosql was not an industrial solution. Sqlite was the best choice. Now w

Re: [sqlite] How to decrease IO usage

2013-01-11 Thread Bebel
Hi Michael, Thanks for your feed back, but as far as I understand the pragma cache size, it's only increase the cache size during the session. But the cache is deleted when the database is close. In my case, I only make short transaction on the database, like open, read and immediately close. I c

Re: [sqlite] How to decrease IO usage

2013-01-10 Thread Simon Slavin
On 9 Jan 2013, at 10:59pm, Bebel wrote: > Now my problem is that my limitation for increase my latency is on the > amount of IO ops perform by the databases. In fact, sqlite make a lot of > random IO on my disks (many hundred) and this increase my disk latency. > > I made a test on a empty data

Re: [sqlite] How to decrease IO usage

2013-01-10 Thread Igor Tandetnik
On 1/9/2013 5:59 PM, Bebel wrote: I'm working with a lot of sqlite base (many million), but they are very small, 2000 entries for the bigger. In this base, I create 6 tables, but I mostly work on just one of them. My structure are very simple, I stored varchar and blob. Looks like you are usin

Re: [sqlite] How to decrease IO usage

2013-01-10 Thread Michael Black
@sqlite.org Subject: [sqlite] How to decrease IO usage Hi there, Let me explain you my problem. I'm working with a lot of sqlite base (many million), but they are very small, 2000 entries for the bigger. In this base, I create 6 tables, but I mostly work on just one of them. My structure are

[sqlite] How to decrease IO usage

2013-01-10 Thread Bebel
Hi there, Let me explain you my problem. I'm working with a lot of sqlite base (many million), but they are very small, 2000 entries for the bigger. In this base, I create 6 tables, but I mostly work on just one of them. My structure are very simple, I stored varchar and blob. My amount of inse