Re: [sqlite] add new column to table

2005-01-10 Thread Bert Verhees
Op maandag 10 januari 2005 23:56, schreef Lloyd Thomas: > Thanks. That is going OK but I am having a problem with updating the new > column with the info I need. It seems to update with the same entry from my > users table to all rows. > > UPDATE call_data SET caller_name = (SELECT firstname ||

Re: [sqlite] excessive malloc() calls

2005-01-10 Thread John Richard Moser
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 ~From what I can tell people are just in shock and awe that checking 3000 tables each holding several years of data for a company (again: several years of data for 3000 different companies) calls malloc() several million times. Interesting enough,

Re: [sqlite] $ in table and column names

2005-01-10 Thread Doug Currie
On Jan 10, 2005, at 6:00 PM, D. Richard Hipp wrote: > A user has reported a bug saying that SQLite does > not allow the '$' in the middle of indentifiers > (without quoting). The bug reports says that > statements like this: > >CREATE TABLE ex$1( col$abc INTEGER ); > > are legal and work

Re: [sqlite] $ in table and column names

2005-01-10 Thread Randall Randall
On Jan 10, 2005, at 6:00 PM, D. Richard Hipp wrote: A user has reported a bug saying that SQLite does not allow the '$' in the middle of indentifiers (without quoting). The bug reports says that statements like this: CREATE TABLE ex$1( col$abc INTEGER ); are legal and work fine in other

Re: [sqlite] excessive malloc() calls

2005-01-10 Thread D. Richard Hipp
Steve Frierdich wrote: I have been noticing all the email messages about excessive malloc calls. Is there a serious bug in Sqlite about malloc being called excessively causing memory leaks in sqlite version 3? And if there is, is there a way to fix it the source code? There are no memory leaks

Re: [sqlite] excessive malloc() calls

2005-01-10 Thread Kurt Welgehausen
> SELECT x, y, z FROM table1 ORDER BY 2; > SELECT x, y, z FROM table1 ORDER BY y; > ... > This is a standard SQL thing, apparently. It surprised me too > when I first found out about it (and had to fix SQLite to do it.) Using an integer is generally deprecated. It was left in the std to

Re: [sqlite] add new column to table

2005-01-10 Thread Lloyd Thomas
Thanks. That is going OK but I am having a problem with updating the new column with the info I need. It seems to update with the same entry from my users table to all rows. UPDATE call_data SET caller_name = (SELECT firstname || surname AS 'caller_name' FROM users WHERE extn_no = (SELECT

Re: [sqlite] excessive malloc() calls

2005-01-10 Thread Steve Frierdich
I have been noticing all the email messages about excessive malloc calls. Is there a serious bug in Sqlite about malloc being called excessively causing memory leaks in sqlite version 3? And if there is, is there a way to fix it the source code? Thank Steve D. Richard Hipp wrote: Andrew

Re: [sqlite] excessive malloc() calls

2005-01-10 Thread Will Leshner
On Mon, 10 Jan 2005 17:07:11 -0500, D. Richard Hipp <[EMAIL PROTECTED]> wrote: > This is a standard SQL thing, apparently. It surprised me too > when I first found out about it (and had to fix SQLite to do it.) Very interesting. Thanks.

Re: [sqlite] excessive malloc() calls

2005-01-10 Thread Matt_Arrington
Tom, Yes I agree. It does make a lot of assumptions. I would not consider using that code permanently without knowing exactly the way the system worked, and then I'd add error checking code that would detect misuse and/or overflow at run time. We use this type of allocation scheme for real time

Re: [sqlite] excessive malloc() calls

2005-01-10 Thread D. Richard Hipp
Will Leshner wrote: On Mon, 10 Jan 2005 13:49:21 -0800, [EMAIL PROTECTED] By the way, "ORDER BY 1" I believe to mean order by column id = 1. So it's not really a meaningless statement. Is that true? I don't get the from the documentation, but I'm probably just looking at it wrong. Yes. You

Re: [sqlite] excessive malloc() calls

2005-01-10 Thread Matt_Arrington
Derrell Thanks for that! I did not have the primary key set up right. (I'm an SQL newbie) which forced me to use "order by." By the way, "ORDER BY 1" I believe to mean order by column id = 1. So it's not really a meaningless statement. I think any "order by clause" is going to cause a per row

Re: [sqlite] excessive malloc() calls

2005-01-10 Thread D. Richard Hipp
Andrew Shakinovsky wrote: I have noticed with SQLite (at least this was true with 2.8x, not sure about 3x) that if you try to use an ORDER BY with a table that doesn't have an index on the field you are ORDERing by, it will do the entire sort (presumably just the keys) in memory. This will cause

Re: [sqlite] excessive malloc() calls

2005-01-10 Thread D. Richard Hipp
[EMAIL PROTECTED] wrote: > > The "order by" clause I think is what causes the memory allocation of > "aType" for each row. I think you are right. The aType is cached when data is coming out of the database file so the allocation only occurs once. But when data is coming out of the sorter, the

Re: [sqlite] excessive malloc() calls

2005-01-10 Thread Will Leshner
On Mon, 10 Jan 2005 13:49:21 -0800, [EMAIL PROTECTED] > By the way, "ORDER BY 1" I believe to mean order by column id = 1. So it's > not really a meaningless statement. Is that true? I don't get the from the documentation, but I'm probably just looking at it wrong.

RE: [sqlite] excessive malloc() calls

2005-01-10 Thread Andrew Shakinovsky
I have noticed with SQLite (at least this was true with 2.8x, not sure about 3x) that if you try to use an ORDER BY with a table that doesn't have an index on the field you are ORDERing by, it will do the entire sort (presumably just the keys) in memory. This will cause you to run out of memory if

Re: [sqlite] excessive malloc() calls

2005-01-10 Thread Will Leshner
On Mon, 10 Jan 2005 13:43:20 -0800, [EMAIL PROTECTED] > This does point out however, that Win32 users who need to use ORDER BY on a > large table are paying a hefty price. Is that true even if they are using ORDER BY properly?

Re: [sqlite] excessive malloc() calls

2005-01-10 Thread Matt_Arrington
I'm just doing the select statement I mentioned in my first message. SELECT * FROM 'SPY' ORDER BY 1 ASC; "1" is the column id which in my case I thought was a primary integer key. The "order by" clause I think is what causes the memory allocation of "aType" for each row. I did not create the

Re: [sqlite] excessive malloc() calls

2005-01-10 Thread Tom
I am sorry, typing too fast I had made typos which made my message not understandable. So here it is again: I would say it is one of the most dangerous code snippets I have seen in a while. The code makes a lot of assumptions but they are NOT explicitly stated. It may work under some strictly

Re: [sqlite] V2.8 or V3.0 for embedded devices?

2005-01-10 Thread Randy J. Ray
Christian Smith wrote: For non-text data, SQLite 3.x has more efficient storage, as binary data is stored in binary format and not text format. I've heard figures such as 25% savings on disk space bandied about, but YMMV. The storage is not binary compatible with SQLite 2.8.x files, and SQLite 3

Re: [sqlite] add new column to table

2005-01-10 Thread Lloyd Thomas
Thanks Paul, I have used that example before to recreate a table. Can I use the same thing to recreate a table and populate a new column with data from a select query. The table I wish to recreate has an ID number in it and I wish to use this to get the forename and surname from

[sqlite] Question about EXCEPT

2005-01-10 Thread Keith Herold
Since EXCEPT returns the left side of a query, once duplicates are deleted from the select on the right, how might one get the !Left AND !Right rows, i.e., the rows which are not in *both* the left and right selects. I had thought EXCEPT did this, but it doesn't, and I was wondering if SQlite had

Re: [sqlite] speed

2005-01-10 Thread Ulrik Petersen
Hi Brandon, Brandon Whalen wrote: I'm currently trying to use sqlite to manage a database in a c program. I have 4 tables where each table has at most 6 columns. I've found that if I use a select statement and run that statement through a callback function that I get incredibly slow response

Re: [sqlite] excessive malloc() calls

2005-01-10 Thread Derrell . Lipman
[EMAIL PROTECTED] writes: > I switched my application over to SQLite3 and did some performance > profiling and found that the majority of the processing time spent is > making calls to malloc(). > > sqlite3_step() is the function that is making all the excessive calls, one > call per row fetched.

Re: [sqlite] speed

2005-01-10 Thread D. Richard Hipp
Brandon Whalen wrote: > I've found that if I use a select statement and run that statement > through a callback function that I get incredibly slow response > times. > > I've tested my sql statement on the command line and its rather > fast > The command-line shell is using exactly the same

Re: [sqlite] excessive malloc() calls

2005-01-10 Thread D. Richard Hipp
[EMAIL PROTECTED] wrote: I switched my application over to SQLite3 and did some performance profiling and found that the majority of the processing time spent is making calls to malloc(). It sounds like you have a bad malloc() implementation. What OS and compiler are you using. Malloc() on linux

Re: [sqlite] excessive malloc() calls

2005-01-10 Thread Matt_Arrington
>It would seem really useful to be able to compare the performance of >two or more companies. Having each of them in a separate table means to >do any kind of comparison you need a join. It would seem lots better >to have a single table with a column for the company name. Eliminates >LOTS of work

RE: [sqlite] excessive malloc() calls

2005-01-10 Thread Sandy Ganz
Did some hashing code which used pooling and preallocate of block and it made the hash way (technical term) faster in insert and delete operations. As well as much less overhead for small hash object. Basically kept it's own pool of hash items. For SQLite might be harder to implement since things

Re: [sqlite] excessive malloc() calls

2005-01-10 Thread Andrew Piskorski
On Mon, Jan 10, 2005 at 09:51:50AM -0800, [EMAIL PROTECTED] wrote: > sqlite3_step() is the function that is making all the excessive calls, one > call per row fetched. > about 3000 of these tables. One complete scan of all 3000 companies will > make roughly 5.5 million malloc() calls! That

Re: [sqlite] V2.8 or V3.0 for embedded devices?

2005-01-10 Thread chorlya
There are some changes in CVS that introduced a few SQLITE_OMIT_* macros that are supposed to exclude some features at compile time. Since you're on embedded system, I guess you'll find quite a few features that you can omit, hence reduce memory footprint. I don't think there's any documentation

[sqlite] speed

2005-01-10 Thread Brandon Whalen
I'm currently trying to use sqlite to manage a database in a c program. I have 4 tables where each table has at most 6 columns. I've found that if I use a select statement and run that statement through a callback function that I get incredibly slow response times. I've found that the

[sqlite] excessive malloc() calls

2005-01-10 Thread Matt_Arrington
I switched my application over to SQLite3 and did some performance profiling and found that the majority of the processing time spent is making calls to malloc(). sqlite3_step() is the function that is making all the excessive calls, one call per row fetched. The program is a stock scanning /

[sqlite] add new column to table

2005-01-10 Thread Lloyd Thomas
I wish to create a new column in a table and add data, which is queried from another table.What is the best way? Lloyd

Re: [sqlite] V2.8 or V3.0 for embedded devices?

2005-01-10 Thread Christian Smith
On Mon, 10 Jan 2005, Markus Oliver Junginger wrote: >We are developing applications that also run on PocketPC devices; Sqlite >2.8.x is used here as the database. > >For the next release of our software the question is if we should switch >to SQLite 3. What's the experts' recommendation? The

[sqlite] V2.8 or V3.0 for embedded devices?

2005-01-10 Thread Markus Oliver Junginger
We are developing applications that also run on PocketPC devices; Sqlite 2.8.x is used here as the database. For the next release of our software the question is if we should switch to SQLite 3. What's the experts' recommendation? The main concern is probably memory consumption, which should