[sqlite] Sparse Data Sets

2007-06-28 Thread Carl Jacobs
I'm planning to use SQLite in an application that has a lot of rows (1M+) with a moderate number of columns (about 50) with the data being somewhat sparsely populated - ie each row may only have data in a primary column and 5 other columns. Will I get a performance hit - in terms of the size of

Re: [sqlite] Re: concers about database size

2006-03-16 Thread Carl Jacobs
Daniel > The most common scenarios: > - get a single marker from subset of individuals > - get a subset of markers from a single individual > - get a subset of markers from a subset of individuals Sounds like this might define your database. Each individual has 500,000 markers, but maybe they

Re: [sqlite] sqlite performance with sizeable tables

2006-03-11 Thread Carl Jacobs
Nicolas, From: <[EMAIL PROTECTED]> > On the other hand, I tried to make better use of the cache: if I run my 1M inserts in 10 transactions of 100,000, things get a bit slower than 100 transactions of 10,000 inserts. > I tried one transaction of 1,000,000 inserts and the test app hangs at 100% cpu

Re: [sqlite] "not an error"

2006-03-09 Thread Carl Jacobs
stale error_code. It's a workaround, I know, but it works. Regards, Carl. - Original Message ----- From: "Carl Jacobs" <[EMAIL PROTECTED]> To: Sent: Thursday, March 09, 2006 5:50 PM Subject: [sqlite] "not an error" > Hello, > > I have the following proce

[sqlite] "not an error"

2006-03-08 Thread Carl Jacobs
Hello, I have the following process (roughly) on a 3.xx database: == Start of process == 1) sqlite_prepare (an insert statement which is then used many times) 2) loop to insert data (thousands of iterations) sqlite_reset / sqlite_bindxxx / sqlite_step 3) 'ATTACH DATABASE secondary' do some

Re: [sqlite] ANN: SQLiteSpy 1.5.1 released

2006-02-03 Thread Carl Jacobs
Whether to use an external DLL or have the functionality embedded? I have a both-ways compromise. I store a copy of the DLL inside the app as a resource. Then if the DLL doesn't extst in the app dir I copy the DLL out, but this also allows me to update the DLL to something more recent by replacing

Re: [sqlite] Auto Increment?

2006-01-31 Thread Carl Jacobs
> Quoting Dennis Cote ([EMAIL PROTECTED]): > > Doesn't this mean that SQLite only supports 2^63 rows with autoincrement? > > That means you can insert one row per millisecond for 29 million years. Well actually, not quite. The website states that the database size is limited to 2^41 bytes. http://

Re: [sqlite] Decimal separator

2006-01-31 Thread Carl Jacobs
> > All would be fine but look at this : > > > > create table test( > > price double, > > amount double default 0 > > ); > > > > insert into test(price) values("12,0"); > > > > amount now = 0.0 The world seems to have settled on using Arabic numerals 0, 1, 2 ... 9. I think we should think about s

Re: [sqlite] Compile SQLite

2006-01-30 Thread Carl Jacobs
Merijn sqlite.dll and the console utility were the first two programs that I ever compiled on VS2005. VS is a bit of an "experience" compared to what I'm used to - Delphi - so I just followed the wiki in very dumb step-by-step manner. You don't need to download any files, as the sqlite.def file is

Re: [sqlite] Slow sqlite3_step - more details.

2006-01-17 Thread Carl Jacobs
> What language and/or wrapper library are you using to drive SQLite? > > Also, how large are the records in your database? You said earlier that > they have 45 fields, but not what type of data was stored in any of > them. Since you are doing string searches, I assume that at least one > contains

[sqlite] Slow sqlite3_step - more details.

2006-01-16 Thread Carl Jacobs
Hello, sorry for the lengthy email. I feel like I'm missing something, but I don't know what - is there anyone who could at least help me with a direction to search in. I'm using Windows XP Sp2, SQLite3.DLL V3.2.7. One of the tests which follows was repeated with bleeding edge SQLite3.DLL V3.3.1

Re: [sqlite] Fw: Slow sqlite_step

2006-01-15 Thread Carl Jacobs
Eric, Thanks for the quick response. > Sounds to me like the matching rows are located near the beginning of > the table's physical storage, so you get them quickly, and then the next > sqlite3_step() has to scan the rest of the table. An idea worth testing, but using the official sqlite3 console

[sqlite] Fw: Slow sqlite_step

2006-01-15 Thread Carl Jacobs
Hello, I'm a new user to this group as I haven't had any issues with sqlite till now. I tried searching to see if this question has been asked before, but couldn't find anything. I'm using Windows. I have a table with ~45 colums and ~17000 records. I do a search for some stuff using 'SELECT * FR