Re: [sqlite] finding pairwise similarity

2012-01-14 Thread Jean-Christophe Deschamps
where `similarity` is a measure of overlap of `t` between each pair of `c` and is calculated as the number of `t` common between a given pair divided by the total number of unique `t` in the pair. Just like Igor, I take "unique `t` in pair" to mean "distinct `t` in pair". I want the resul

Re: [sqlite] Some pieces of the puzzle are coming together

2012-01-14 Thread John Elrick
On Fri, Jan 13, 2012 at 5:13 PM, Max Vlasov wrote: > John, > I read your previous post and unfortunately, your conversation with Richard > didn't reveal much details. I worked with scenario similar tou yours > (Delphi + statically linked sqlite) for several years and during this > period an advan

Re: [sqlite] Some pieces of the puzzle are coming together

2012-01-14 Thread John Elrick
On Sat, Jan 14, 2012 at 10:42 AM, Dan Kennedy wrote: > On 01/14/2012 04:49 AM, John Elrick wrote: > >> >> So, obviously the problem is that _malloc is being called a much larger >> number of times by 3.7.9 in this particular circumstance -- roughly 70 >> times as often. _memcpy demonstrates roug

Re: [sqlite] Some pieces of the puzzle are coming together

2012-01-14 Thread John Elrick
On Fri, Jan 13, 2012 at 5:58 PM, Richard Hipp wrote: > On Fri, Jan 13, 2012 at 4:49 PM, John Elrick >wrote: > > > > > 3.6.17 > > > > Procedure % TimeTimeCalls > > _sqlite3_step 58.4% 1.795,052 > > _memcpy 22.8% 0.691,342,9

Re: [sqlite] Some pieces of the puzzle are coming together

2012-01-14 Thread John Elrick
On Fri, Jan 13, 2012 at 5:20 PM, Pavel Ivanov wrote: > > At this point I've exhausted my avenues of research. Does anyone have > any > > further ideas as to what may be going on which could cause this increase > in > > memory management activity? Or any suggestions of where to look next? > > II

[sqlite] sqlite3_set_auxdata & invalid pointer

2012-01-14 Thread gwenn
Hello, I am trying to add custom aggregation function support in a golang driver (scalar functions are ok). While testing, I got this: *** glibc detected *** ./6.out: realloc(): invalid pointer: 0x02daa1c5 *** === Backtrace: = /lib/x86_64-linux-gnu/libc.so.6(+0x72656)[0x2b9a7b5d

Re: [sqlite] finding pairwise similarity

2012-01-14 Thread Igor Tandetnik
Mr. Puneet Kishor wrote: > I have two tables like so > >CREATE TABLE c (c_no INTEGER PRIMARY KEY, c_name TEXT); >CREATE TABLE t (t_no INTEGER PRIMARY KEY, t_name TEXT); >CREATE TABLE c_t (c_no INTEGER, t_no INTEGER); > > Every row in `c` has one or more `t`. I want, in a single SQL,

Re: [sqlite] SSD with TRIM

2012-01-14 Thread Max Vlasov
On Sat, Jan 14, 2012 at 11:52 PM, Simon Slavin wrote: > > On 14 Jan 2012, at 7:24pm, Max Vlasov wrote: > > > On Sat, Jan 14, 2012 at 6:12 AM, Simon Slavin > wrote: > > > >> Fast. Fasty fast. Speed is high. INSERT, UPDATE, and DELETE all > >> significantly faster. SELECT is a bit faster, but

Re: [sqlite] SSD with TRIM

2012-01-14 Thread Simon Slavin
On 14 Jan 2012, at 7:24pm, Max Vlasov wrote: > On Sat, Jan 14, 2012 at 6:12 AM, Simon Slavin wrote: > >> Fast. Fasty fast. Speed is high. INSERT, UPDATE, and DELETE all >> significantly faster. SELECT is a bit faster, but there's less difference. > > Simon, very interesting. Can you make s

Re: [sqlite] SSD with TRIM

2012-01-14 Thread Max Vlasov
On Sat, Jan 14, 2012 at 6:12 AM, Simon Slavin wrote: > > Fast. Fasty fast. Speed is high. INSERT, UPDATE, and DELETE all > significantly faster. SELECT is a bit faster, but there's less difference. > > Simon, very interesting. Can you make some tests related to internal fragmentation? As an a

[sqlite] finding pairwise similarity

2012-01-14 Thread Mr. Puneet Kishor
I have two tables like so CREATE TABLE c (c_no INTEGER PRIMARY KEY, c_name TEXT); CREATE TABLE t (t_no INTEGER PRIMARY KEY, t_name TEXT); CREATE TABLE c_t (c_no INTEGER, t_no INTEGER); Every row in `c` has one or more `t`. I want, in a single SQL, the following source_c, targ

Re: [sqlite] Some pieces of the puzzle are coming together

2012-01-14 Thread Dan Kennedy
On 01/14/2012 04:49 AM, John Elrick wrote: So, obviously the problem is that _malloc is being called a much larger number of times by 3.7.9 in this particular circumstance -- roughly 70 times as often. _memcpy demonstrates roughly 6 times as many calls I ran a test with a much smaller dataset

Re: [sqlite] Slow commits

2012-01-14 Thread Dimiter 'malkia' Stanev
Hi Normand, Just for sanity, check whether Windows is not creating previous versions for you. Right click on the file in Explorer -> Properties -> Previous Versions. Also check whether Caching is enabled on your HDD: Control Panel -> Device Manager -> Disk Drives -> (Click on your HDD) The

[sqlite] Bug: "Invalid statement in fillWindow()" from Explain query plan using subquery

2012-01-14 Thread Luke Stevens
Using SQLite 3.7.4 on Android. To reproduce: create table S (id string, name string); create table F (id string, sid string); explain query plan select (select count(*) from F where F.sid=S.id) as FCount, S.name from S; Result: Empty cursor, with log showing: SqliteCursor.cpp Invali

Re: [sqlite] Using non-aggregate columns in group by (with an additional issue)

2012-01-14 Thread David Garfield
A very minor change on your query is all that is needed. select max(x.timestamp),x.value,x.person from table as x, (select person,max(value) as maxvalue from table group by person order by maxvalue desc) as y where x.person = y.person and x.value = y.maxvalue group by x.person, x.value I don't