Re: [sqlite] DEvelopment Error

[Jay A. Kreibich]

On Mon, Jul 25, 2011 at 09:13:49PM -0400, Daniel Spain scratched on the wall:

> i most recently began making a text based gaming engine and am having
> trouble. i downloaded the SQLITE3 package and did: implib sqlite3.lib
> sqlite3.dll then copied sqlite3.lib to my project directory. in my 
> programs main healer i did : #include "sqlite3ext.h" and linked the
> LIB file.

  You want to use "sqlite3.h" in applications.
  
  The "sqlite3ext.h" file is for building library extensions

   -j


-- 
Jay A. Kreibich < J A Y  @  K R E I B I.C H >

"Intelligence is like underwear: it is important that you have it,
 but showing it to the wrong people has the tendency to make them
 feel uncomfortable." -- Angela Johnson
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] DEvelopment Error

[Daniel Spain]

i most recently began making a text based gaming engine and am having trouble. 
i downloaded the SQLITE3 package and did: implib sqlite3.lib sqlite3.dll then 
copied sqlite3.lib to my project directory. in my programs main healer i did : 
#include "sqlite3ext.h" and linked the LIB file. then ran a test routine. 
voidTestBed(void){ sqlite3 *database; int rc;  rc = 
sqlite3_open("TEST.DB",); printf("\nRC=%d\r\r",rc);} when i 
compile i get the following: Error E4251 SQLMUD.C 847: Undefined symbol 
'sqlite3_api' in function TestBed. i even tried it this way: sqlite3_open_v2 
then to just make sure the library wasnt compiling wrong i linked the sqlite3.c 
directly into the program and got the same result. Thanks for any help anyone 
can provide. i'm using Borland C++ 5.5.1
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Code First Support?

[Simon Slavin]

On 26 Jul 2011, at 12:02am, Ryan Killian wrote:

> Im very new to system.data.sqlite, and was wondering if there's any way to
> get code-first support working for EF 4.1 yet? Ive seen posts saying its not
> supported, but all are dated prior to April.

Okay, after a quick google this turns out to be two ways of referring to the 
same thing.





Knock yourselves out.

Simon.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] Code First Support?

[Ryan Killian]

Hello,

Im very new to system.data.sqlite, and was wondering if there's any way to
get code-first support working for EF 4.1 yet? Ive seen posts saying its not
supported, but all are dated prior to April.

Thanks!
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] SQLite3.dll for Win 64

[Shane Harrelson]

The makefile builds lemon.exe from lemon.c as part of the build process, so
make sure it was generated correctly as well.

"lemon -x" should print a version number.
"lemon -?" should print an error and help message.

Again, HTH.
-Shane


On Mon, Jul 25, 2011 at 5:51 PM, Shane Harrelson
wrote:

>
> The TK_* identifiers are all defined in parse.h which is generated by
> lemon.exe from parse.y.
> parse.h is "included" in the amalgamation file, sqlite3.c.
> Try deleting your parse.c and parse.h and sqlite3.c and re-running your
> make.
> Check that parse.h was generated correctly and subsequently concatenated
> into sqlite3.c correctly.
>
> HTH.
> -Shane
>
>
>
> On Mon, Jul 25, 2011 at 3:49 PM, Everton Vieira wrote:
>
>> So, now i'm stock on this errors:
>>
>> sqlite3.c
>> sqlite3.c(40972) : warning C4244: '=' : conversion from '__int64' to
>> 'int',
>> poss
>> ible loss of data
>> sqlite3.c(57259) : error C2065: 'TK_REGISTER' : undeclared identifier
>> sqlite3.c(57266) : error C2065: 'TK_UMINUS' : undeclared identifier
>> sqlite3.c(57266) : error C2065: 'TK_INTEGER' : undeclared identifier
>> sqlite3.c(57266) : error C2065: 'TK_FLOAT' : undeclared identifier
>> sqlite3.c(57273) : error C2065: 'TK_STRING' : undeclared identifier
>> sqlite3.c(57273) : error C2065: 'TK_FLOAT' : undeclared identifier
>> sqlite3.c(57273) : error C2065: 'TK_INTEGER' : undeclared identifier
>> sqlite3.c(57282) : error C2065: 'TK_FLOAT' : undeclared identifier
>> sqlite3.c(57284) : error C2065: 'TK_INTEGER' : undeclared identifier
>> sqlite3.c(57284) : error C2065: 'TK_FLOAT' : undeclared identifier
>> sqlite3.c(57293) : error C2065: 'TK_UMINUS' : undeclared identifier
>> sqlite3.c(57307) : error C2065: 'TK_NULL' : undeclared identifier
>> sqlite3.c(57312) : error C2065: 'TK_BLOB' : undeclared identifier
>> sqlite3.c(61966) : error C2065: 'TK_VARIABLE' : undeclared identifier
>> sqlite3.c(63498) : error C2196: case value '0' already used
>> sqlite3.c(63752) : error C2196: case value '0' already used
>> sqlite3.c(63822) : error C2196: case value '0' already used
>> sqlite3.c(63823) : error C2196: case value '0' already used
>> sqlite3.c(63824) : error C2196: case value '0' already used
>> sqlite3.c(63825) : error C2196: case value '0' already used
>> sqlite3.c(63826) : error C2196: case value '0' already used
>> sqlite3.c(63847) : error C2196: case value '0' already used
>> sqlite3.c(63848) : error C2196: case value '0' already used
>> sqlite3.c(63849) : error C2196: case value '0' already used
>> sqlite3.c(63870) : error C2196: case value '0' already used
>> sqlite3.c(63871) : error C2196: case value '0' already used
>> sqlite3.c(63872) : error C2196: case value '0' already used
>> sqlite3.c(64070) : error C2196: case value '0' already used
>> sqlite3.c(64071) : error C2196: case value '0' already used
>> sqlite3.c(64072) : error C2196: case value '0' already used
>> sqlite3.c(64073) : error C2196: case value '0' already used
>> sqlite3.c(64191) : error C2196: case value '0' already used
>> sqlite3.c(64214) : error C2196: case value '0' already used
>> sqlite3.c(64238) : error C2196: case value '0' already used
>> sqlite3.c(64254) : error C2196: case value '0' already used
>> sqlite3.c(64272) : error C2196: case value '0' already used
>> sqlite3.c(64354) : error C2196: case value '0' already used
>> sqlite3.c(64355) : error C2196: case value '0' already used
>> sqlite3.c(64356) : error C2196: case value '0' already used
>> sqlite3.c(64357) : error C2196: case value '0' already used
>> sqlite3.c(64358) : error C2196: case value '0' already used
>> sqlite3.c(64359) : error C2196: case value '0' already used
>> sqlite3.c(64410) : error C2196: case value '0' already used
>> sqlite3.c(64411) : error C2196: case value '0' already used
>> sqlite3.c(64412) : error C2196: case value '0' already used
>> sqlite3.c(64413) : error C2196: case value '0' already used
>> sqlite3.c(64546) : error C2196: case value '0' already used
>> sqlite3.c(64547) : error C2196: case value '0' already used
>> sqlite3.c(64588) : error C2196: case value '0' already used
>> sqlite3.c(64605) : error C2196: case value '0' already used
>> sqlite3.c(64654) : error C2196: case value '0' already used
>> sqlite3.c(64666) : error C2196: case value '0' already used
>> sqlite3.c(69909) : error C2065: 'TK_COLUMN' : undeclared identifier
>> sqlite3.c(69911) : error C2065: 'TK_AS' : undeclared identifier
>> sqlite3.c(70079) : error C2065: 'TK_DELETE' : undeclared identifier
>> sqlite3.c(70082) : error C2065: 'TK_INSERT' : undeclared identifier
>> sqlite3.c(70186) : error C2065: 'TK_STRING' : undeclared identifier
>> sqlite3.c(70231) : error C2065: 'TK_TRIGGER' : undeclared identifier
>> sqlite3.c(70231) : error C2065: 'TK_COLUMN' : undeclared identifier
>> sqlite3.c(70255) : error C2065: 'TK_COLUMN' : undeclared identifier
>> sqlite3.c(70327) : error C2065: 'TK_ID' : undeclared identifier
>> sqlite3.c(70327) : error C2051: case 

Re: [sqlite] SQLite3.dll for Win 64

[Shane Harrelson]

The TK_* identifiers are all defined in parse.h which is generated by
lemon.exe from parse.y.
parse.h is "included" in the amalgamation file, sqlite3.c.
Try deleting your parse.c and parse.h and sqlite3.c and re-running your
make.
Check that parse.h was generated correctly and subsequently concatenated
into sqlite3.c correctly.

HTH.
-Shane



On Mon, Jul 25, 2011 at 3:49 PM, Everton Vieira  wrote:

> So, now i'm stock on this errors:
>
> sqlite3.c
> sqlite3.c(40972) : warning C4244: '=' : conversion from '__int64' to 'int',
> poss
> ible loss of data
> sqlite3.c(57259) : error C2065: 'TK_REGISTER' : undeclared identifier
> sqlite3.c(57266) : error C2065: 'TK_UMINUS' : undeclared identifier
> sqlite3.c(57266) : error C2065: 'TK_INTEGER' : undeclared identifier
> sqlite3.c(57266) : error C2065: 'TK_FLOAT' : undeclared identifier
> sqlite3.c(57273) : error C2065: 'TK_STRING' : undeclared identifier
> sqlite3.c(57273) : error C2065: 'TK_FLOAT' : undeclared identifier
> sqlite3.c(57273) : error C2065: 'TK_INTEGER' : undeclared identifier
> sqlite3.c(57282) : error C2065: 'TK_FLOAT' : undeclared identifier
> sqlite3.c(57284) : error C2065: 'TK_INTEGER' : undeclared identifier
> sqlite3.c(57284) : error C2065: 'TK_FLOAT' : undeclared identifier
> sqlite3.c(57293) : error C2065: 'TK_UMINUS' : undeclared identifier
> sqlite3.c(57307) : error C2065: 'TK_NULL' : undeclared identifier
> sqlite3.c(57312) : error C2065: 'TK_BLOB' : undeclared identifier
> sqlite3.c(61966) : error C2065: 'TK_VARIABLE' : undeclared identifier
> sqlite3.c(63498) : error C2196: case value '0' already used
> sqlite3.c(63752) : error C2196: case value '0' already used
> sqlite3.c(63822) : error C2196: case value '0' already used
> sqlite3.c(63823) : error C2196: case value '0' already used
> sqlite3.c(63824) : error C2196: case value '0' already used
> sqlite3.c(63825) : error C2196: case value '0' already used
> sqlite3.c(63826) : error C2196: case value '0' already used
> sqlite3.c(63847) : error C2196: case value '0' already used
> sqlite3.c(63848) : error C2196: case value '0' already used
> sqlite3.c(63849) : error C2196: case value '0' already used
> sqlite3.c(63870) : error C2196: case value '0' already used
> sqlite3.c(63871) : error C2196: case value '0' already used
> sqlite3.c(63872) : error C2196: case value '0' already used
> sqlite3.c(64070) : error C2196: case value '0' already used
> sqlite3.c(64071) : error C2196: case value '0' already used
> sqlite3.c(64072) : error C2196: case value '0' already used
> sqlite3.c(64073) : error C2196: case value '0' already used
> sqlite3.c(64191) : error C2196: case value '0' already used
> sqlite3.c(64214) : error C2196: case value '0' already used
> sqlite3.c(64238) : error C2196: case value '0' already used
> sqlite3.c(64254) : error C2196: case value '0' already used
> sqlite3.c(64272) : error C2196: case value '0' already used
> sqlite3.c(64354) : error C2196: case value '0' already used
> sqlite3.c(64355) : error C2196: case value '0' already used
> sqlite3.c(64356) : error C2196: case value '0' already used
> sqlite3.c(64357) : error C2196: case value '0' already used
> sqlite3.c(64358) : error C2196: case value '0' already used
> sqlite3.c(64359) : error C2196: case value '0' already used
> sqlite3.c(64410) : error C2196: case value '0' already used
> sqlite3.c(64411) : error C2196: case value '0' already used
> sqlite3.c(64412) : error C2196: case value '0' already used
> sqlite3.c(64413) : error C2196: case value '0' already used
> sqlite3.c(64546) : error C2196: case value '0' already used
> sqlite3.c(64547) : error C2196: case value '0' already used
> sqlite3.c(64588) : error C2196: case value '0' already used
> sqlite3.c(64605) : error C2196: case value '0' already used
> sqlite3.c(64654) : error C2196: case value '0' already used
> sqlite3.c(64666) : error C2196: case value '0' already used
> sqlite3.c(69909) : error C2065: 'TK_COLUMN' : undeclared identifier
> sqlite3.c(69911) : error C2065: 'TK_AS' : undeclared identifier
> sqlite3.c(70079) : error C2065: 'TK_DELETE' : undeclared identifier
> sqlite3.c(70082) : error C2065: 'TK_INSERT' : undeclared identifier
> sqlite3.c(70186) : error C2065: 'TK_STRING' : undeclared identifier
> sqlite3.c(70231) : error C2065: 'TK_TRIGGER' : undeclared identifier
> sqlite3.c(70231) : error C2065: 'TK_COLUMN' : undeclared identifier
> sqlite3.c(70255) : error C2065: 'TK_COLUMN' : undeclared identifier
> sqlite3.c(70327) : error C2065: 'TK_ID' : undeclared identifier
> sqlite3.c(70327) : error C2051: case expression not constant
> sqlite3.c(70334) : error C2065: 'TK_DOT' : undeclared identifier
> sqlite3.c(70334) : error C2051: case expression not constant
> sqlite3.c(70342) : error C2065: 'TK_ID' : undeclared identifier
> sqlite3.c(70357) : error C2065: 'TK_CONST_FUNC' : undeclared identifier
> sqlite3.c(70357) : error C2051: case expression not constant
> sqlite3.c(70358) : error C2065: 'TK_FUNCTION' : undeclared 

Re: [sqlite] SQLite3.dll for Win 64

[Everton Vieira]

Probably is even possible, but much better would be to have this dll especially
for the community.

Anyone have done this dll before?

2011/7/25 Robert Myers 

> Check your compiler documentation. You should be able to link C .obj
> files and call C functions without too much hassle. The biggest issue
> will be getting all the functions declared in a Pascal readable format.
>
> On 7/25/2011 3:10 PM, Everton Vieira wrote:
> > The software that uses this dll is made in pascal so i can't include c
> files
> > in the project.
> >
> > 2011/7/25 Teg 
> >
> >> Hello Everton,
> >>
> >> EV> Don't have any easy way to make this dll? There's a lot of 64bits
> >> systems
> >> EV> out there that will need this dll.
> >>
> >> I just compile it into my program and don't bother with DLL's. I find
> >> that DLL's in general make my software less reliable so, I avoid them.
> >>
> >> Static linking all the way
> >>
> >> C
> >>
> >>
> >>
> >>
> >>
> >> ___
> >> sqlite-users mailing list
> >> sqlite-users@sqlite.org
> >> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
> >>
> >
> >
>
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>



-- 
Everton Vieira.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] [FTS3] Understanding the Flow of data through the tokenizer

[Abhinav Upadhyay]

On Mon, Jul 25, 2011 at 9:54 AM, Dan Kennedy  wrote:
> On 07/24/2011 08:16 PM, Abhinav Upadhyay wrote:
>> Hi,
>>
>> I am trying to write my own custom tokenizer to filter stopwords apart
>> from doing normalization and stemming. I have gone through the
>> comments in fts3_tokenizer.h and also read the implementation of the
>> simple tokenizer. While overall I am able to understand what I need to
>> do to implement this tokenizer, but I still cannot visualize how the
>> FTS engine calls the tokenizer and what data in what form it passes to
>> it.
>>
>> Does the FTS engine pass the complete document data to the tokenizer
>> or it passes some chunks of data, or individual words ? I need to
>> understand this part because the next function needs to set the
>> offsets accordingly. By just going through the code of the simple
>> tokenizer I could not completely comprehend it (it would have been
>> better if I could debug it).
>>
>> By the next functio I mean this: int (*xNext)(
>>      sqlite3_tokenizer_cursor *pCursor,   /* Tokenizer cursor */
>>      const char **ppToken, int *pnBytes,  /* OUT: Normalized text for token 
>> */
>>      int *piStartOffset,  /* OUT: Byte offset of token in input buffer */
>>      int *piEndOffset,    /* OUT: Byte offset of end of token in input 
>> buffer */
>>      int *piPosition      /* OUT: Number of tokens returned before this one 
>> */
>>    );
>> };
>>
>> It would be better if you could explain what is the role of these
>> parameters: piEndOffset , piStartOffset ?
>
> Each time xNext() returns SQLITE_OK to return a new token, xNext()
> should set:
>
>   *piStartOffset to the number of bytes in the input buffer before
>   start of the token being returned,
>
>   *piEndOffset to *piStartOffset plus the number of bytes in the
>   token text, and
>
>   *piPosition to the number of tokens that occur in the input buffer
>   before the token being returned.

Thanks for the explanation. I was able to correct my implementation :-)
.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] SQLite3.dll for Win 64

[Robert Myers]

Check your compiler documentation. You should be able to link C .obj
files and call C functions without too much hassle. The biggest issue
will be getting all the functions declared in a Pascal readable format.

On 7/25/2011 3:10 PM, Everton Vieira wrote:
> The software that uses this dll is made in pascal so i can't include c files
> in the project.
>
> 2011/7/25 Teg 
>
>> Hello Everton,
>>
>> EV> Don't have any easy way to make this dll? There's a lot of 64bits
>> systems
>> EV> out there that will need this dll.
>>
>> I just compile it into my program and don't bother with DLL's. I find
>> that DLL's in general make my software less reliable so, I avoid them.
>>
>> Static linking all the way
>>
>> C
>>
>>
>>
>>
>>
>> ___
>> sqlite-users mailing list
>> sqlite-users@sqlite.org
>> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>>
>
>

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] SQLite3.dll for Win 64

[Everton Vieira]

The software that uses this dll is made in pascal so i can't include c files
in the project.

2011/7/25 Teg 

> Hello Everton,
>
> EV> Don't have any easy way to make this dll? There's a lot of 64bits
> systems
> EV> out there that will need this dll.
>
> I just compile it into my program and don't bother with DLL's. I find
> that DLL's in general make my software less reliable so, I avoid them.
>
> Static linking all the way
>
> C
>
>
>
>
>
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>



-- 
Everton Vieira.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] SQLite3.dll for Win 64

[Teg]

Hello Everton,

EV> Don't have any easy way to make this dll? There's a lot of 64bits systems
EV> out there that will need this dll.

I just compile it into my program and don't bother with DLL's. I find
that DLL's in general make my software less reliable so, I avoid them.

Static linking all the way

C





___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] SQLite3.dll for Win 64

[Everton Vieira]

So, now i'm stock on this errors:

sqlite3.c
sqlite3.c(40972) : warning C4244: '=' : conversion from '__int64' to 'int',
poss
ible loss of data
sqlite3.c(57259) : error C2065: 'TK_REGISTER' : undeclared identifier
sqlite3.c(57266) : error C2065: 'TK_UMINUS' : undeclared identifier
sqlite3.c(57266) : error C2065: 'TK_INTEGER' : undeclared identifier
sqlite3.c(57266) : error C2065: 'TK_FLOAT' : undeclared identifier
sqlite3.c(57273) : error C2065: 'TK_STRING' : undeclared identifier
sqlite3.c(57273) : error C2065: 'TK_FLOAT' : undeclared identifier
sqlite3.c(57273) : error C2065: 'TK_INTEGER' : undeclared identifier
sqlite3.c(57282) : error C2065: 'TK_FLOAT' : undeclared identifier
sqlite3.c(57284) : error C2065: 'TK_INTEGER' : undeclared identifier
sqlite3.c(57284) : error C2065: 'TK_FLOAT' : undeclared identifier
sqlite3.c(57293) : error C2065: 'TK_UMINUS' : undeclared identifier
sqlite3.c(57307) : error C2065: 'TK_NULL' : undeclared identifier
sqlite3.c(57312) : error C2065: 'TK_BLOB' : undeclared identifier
sqlite3.c(61966) : error C2065: 'TK_VARIABLE' : undeclared identifier
sqlite3.c(63498) : error C2196: case value '0' already used
sqlite3.c(63752) : error C2196: case value '0' already used
sqlite3.c(63822) : error C2196: case value '0' already used
sqlite3.c(63823) : error C2196: case value '0' already used
sqlite3.c(63824) : error C2196: case value '0' already used
sqlite3.c(63825) : error C2196: case value '0' already used
sqlite3.c(63826) : error C2196: case value '0' already used
sqlite3.c(63847) : error C2196: case value '0' already used
sqlite3.c(63848) : error C2196: case value '0' already used
sqlite3.c(63849) : error C2196: case value '0' already used
sqlite3.c(63870) : error C2196: case value '0' already used
sqlite3.c(63871) : error C2196: case value '0' already used
sqlite3.c(63872) : error C2196: case value '0' already used
sqlite3.c(64070) : error C2196: case value '0' already used
sqlite3.c(64071) : error C2196: case value '0' already used
sqlite3.c(64072) : error C2196: case value '0' already used
sqlite3.c(64073) : error C2196: case value '0' already used
sqlite3.c(64191) : error C2196: case value '0' already used
sqlite3.c(64214) : error C2196: case value '0' already used
sqlite3.c(64238) : error C2196: case value '0' already used
sqlite3.c(64254) : error C2196: case value '0' already used
sqlite3.c(64272) : error C2196: case value '0' already used
sqlite3.c(64354) : error C2196: case value '0' already used
sqlite3.c(64355) : error C2196: case value '0' already used
sqlite3.c(64356) : error C2196: case value '0' already used
sqlite3.c(64357) : error C2196: case value '0' already used
sqlite3.c(64358) : error C2196: case value '0' already used
sqlite3.c(64359) : error C2196: case value '0' already used
sqlite3.c(64410) : error C2196: case value '0' already used
sqlite3.c(64411) : error C2196: case value '0' already used
sqlite3.c(64412) : error C2196: case value '0' already used
sqlite3.c(64413) : error C2196: case value '0' already used
sqlite3.c(64546) : error C2196: case value '0' already used
sqlite3.c(64547) : error C2196: case value '0' already used
sqlite3.c(64588) : error C2196: case value '0' already used
sqlite3.c(64605) : error C2196: case value '0' already used
sqlite3.c(64654) : error C2196: case value '0' already used
sqlite3.c(64666) : error C2196: case value '0' already used
sqlite3.c(69909) : error C2065: 'TK_COLUMN' : undeclared identifier
sqlite3.c(69911) : error C2065: 'TK_AS' : undeclared identifier
sqlite3.c(70079) : error C2065: 'TK_DELETE' : undeclared identifier
sqlite3.c(70082) : error C2065: 'TK_INSERT' : undeclared identifier
sqlite3.c(70186) : error C2065: 'TK_STRING' : undeclared identifier
sqlite3.c(70231) : error C2065: 'TK_TRIGGER' : undeclared identifier
sqlite3.c(70231) : error C2065: 'TK_COLUMN' : undeclared identifier
sqlite3.c(70255) : error C2065: 'TK_COLUMN' : undeclared identifier
sqlite3.c(70327) : error C2065: 'TK_ID' : undeclared identifier
sqlite3.c(70327) : error C2051: case expression not constant
sqlite3.c(70334) : error C2065: 'TK_DOT' : undeclared identifier
sqlite3.c(70334) : error C2051: case expression not constant
sqlite3.c(70342) : error C2065: 'TK_ID' : undeclared identifier
sqlite3.c(70357) : error C2065: 'TK_CONST_FUNC' : undeclared identifier
sqlite3.c(70357) : error C2051: case expression not constant
sqlite3.c(70358) : error C2065: 'TK_FUNCTION' : undeclared identifier
sqlite3.c(70358) : error C2051: case expression not constant
sqlite3.c(70394) : error C2065: 'TK_NULL' : undeclared identifier
sqlite3.c(70412) : error C2065: 'TK_AGG_FUNCTION' : undeclared identifier
sqlite3.c(70424) : error C2065: 'TK_SELECT' : undeclared identifier
sqlite3.c(70424) : error C2051: case expression not constant
sqlite3.c(70425) : error C2065: 'TK_EXISTS' : undeclared identifier
sqlite3.c(70425) : error C2051: case expression not constant
sqlite3.c(70427) : error C2065: 'TK_IN' : undeclared identifier
sqlite3.c(70427) : error C2051: case 

Re: [sqlite] Reducing compiled size

[Baruch Burstein]

For the record, I know that the documentation says the OMIT_* flags don't
work with the regular amalgamation, but I have in fact used many of them and
they seem to work. Two that only partially worked (a bug?) are
SQLITE_OMIT_AUTOVACUUM and SQLITE_OMIT_AUTHORIZATION (I am not sure about
that second one, as I am writing this from memory. I think it was that one)
They gave a warning when compiling sqlite3.c and an error when linking it to
my program.

On Mon, Jul 25, 2011 at 6:16 PM, Dave Hope  wrote:

> Hi Richard,
>
> I'm using gcc version 4.4.3 (Ubuntu 4.4.3-4ubuntu5). Also tried with
> VC9 which yields very similar sizes.
>
> $ gcc main.c sqlite3.c -lpthread -ldl
> $ size a.out
>   textdata bss dec hex filename
>  51447536881168  519331   7eca3 a.out
>
> It looks like this could have been because I wasn't optimizing for size:
>
> $ gcc main.c sqlite3.c -lpthread -ldl -Os
> $ size a.out
>   textdata bss dec hex filename
>  32648735881000  331075   50d43 a.out
>
> Thanks for pointing out what I missed,
>
> Dave
>
> On Mon, Jul 25, 2011 at 4:07 PM, Richard Hipp  wrote:
> > On Mon, Jul 25, 2011 at 10:39 AM, Dave Hope  wrote:
> >
> >> Hi all,
> >>
> >> I read on the sqlite website that the compiled size can be reduced to
> >> 275KiB ( http://www.sqlite.org/different.html#small ), however I'm
> >> struggling to get it much below 500 KiB using the SQLITE_OMIT_*
> >> options. I'm producing my own amalgamation, which I believe you can
> >> use the OMIT options with:
> >>
> >
> > So I type:
> >
> >gcc -Os -c sqlite3.c; size sqlite3.o
> >
> > And on Linux with gcc 4.1.0 I get 333802 bytes.  This is without any OMIT
> > options.  And you cannot get it below 500KB?  What compiler are you
> using?
> >
> >
> > --
> > D. Richard Hipp
> > d...@sqlite.org
> > ___
> > sqlite-users mailing list
> > sqlite-users@sqlite.org
> > http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
> >
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] New madIS v1.3 release

[Eleytherios Stamatogiannakis]

madIS is an extensible relational database system built upon the SQLite
database and with extensions written in Python (via APSW SQLite
wrapper). Its is developed at:

http://madis.googlecode.com

Due to madIS’ SQLite core, the database format of madIS is exactly the
same as SQLite’s one. This means that all SQLite database files are 
directly usable with madIS.

In usage, madIS, feels like a lightweight personal Hive+Hadoop 
programming environment, without the distributed processing capabilities 
of Hadoop. Nevertheless due to its low overhead while running on a 
single computer (compared to Hadoop), madIS can easily handle tens of 
millions of rows on a single desktop/laptop computer.

In version 1.3 of madIS:

  - A great deal of testing has been done on Linux, Windows and Mac OSX.
  - madIS now only depends on APSW on all these systems, so it is easier 
to be installed
  - XMLPARSE was added. XMLPARSE processes its input in a streaming 
fashion, and has been tested with very large (~20+ GB) XML source files 
without problems.
  - JPACK functions were added. Jpacks are now the preferable way to 
store a set of values into a single tuple. For easy viewing and 
exporting of the jpacks, their format was based on the JSON format.
  - Heavy testing under Windows and Mac OSX. CLIPBOARD and CLIPOUT 
virtual tables work under all OSes.
  - CLIPOUT and CLIPBOARD, have been tested with Excel, Libre/Open 
Office Calc, and iWork Numbers.
  - Functions that return tables, can easily be coded now, by using 
Python's generators (yield)
  - A lot of completions (via TAB) have been added to mterm. Mterm's 
completion engine can automatically complete, tables, column names, 
table index names and database filenames in attach database.

In detail:

MTERM changes:

While using madIS's terminal (mterm), mterm completes (via TAB) column 
names, tables names, etc. of the opened and attached databases.

Also by default mterm colours column separators and if more than 4 
columns are returned, mterm "tags" the columns with numbers:

mterm> select * from deficit;
[1|1 | People's Republic of China [3|272.500 |2010
[1|2 | Japan [3|166.500 |2010
[1|3 | Germany [3|162.300 |2010
--- Column names ---
[1|Rank [2|Country [3|CAB [4|Year

  Note: In mterm the column number tags are coloured red in above example


XMLPARSE:

If i wished to retrieve the date and author of madIS project's Source 
Changes ATOM feed:

mterm> select * from
(XMLPARSE 
'tt'
select * from file('http://code.google.com/feeds/p/madis/hgchanges/basic')
) limit 3;
2011-07-25T14:07:07Z|est...@servum
2011-07-25T14:04:09Z|est...@servum
2011-07-22T14:08:11Z|est...@servum
--- Column names ---
[1|updated [2|author_name
Query executed in 0 min. 0 sec 543 msec
mterm>

In above query, XMLPARSE is used in an "inverted form" which is easier 
to write, when chaining virtual tables.


JPACKS:

Frequently, the need to store multiple values into a tuple arises while 
processing data. Previously in madIS a lot of formats were used to store 
all these multiple values (space separated, comma separated, tab 
separated). Now JPACKs are the recommended way to store multiple values.

JPACKs are based on the JSON format, with the exception that a JPACK of 
a single value is itself. Some examples are presented below:

mterm> select jpack('a');
a

mterm> select jpack('a', 'b');
["a","b"]

mterm> select jsplit(jpack('a','b','c'));
a|b|c

mterm> select jsplitv(jpack('a','b','c'));
a
b
c

mterm> select jgroup(c1) from (select 'a' as c1 union select 'b');
["a","b"]


CLIPBOARD and CLIPOUT:

If a selection of data has been made in the web browser or a spreadsheet 
program then by executing the following in mterm we could access the 
clipboard data as a table:

mterm> select * from CLIPBOARD();
Rank↓ |Country↓ |CAB (billion US dollars)↓ |Year↓
1 | People's Republic of China |272.500 |2010
2 | Japan |166.500 |2010
3 | Germany |162.300 |2010
--- Column names ---
[1|C1 [2|C2 [3|C3 [4|C4
Query executed in 0 min. 0 sec 204 msec

* For above data i selected the top 3 rows while browsing 
http://en.wikipedia.org/wiki/List_of_sovereign_states_by_current_account_balance
 
.

Similarly if i wished to process and send the data to a spreadsheet 
program then i could execute the following in mterm:

mterm> CLIPOUT select * from deficit;
1
--- Column names ---
[1|return_value
Query executed in 0 min. 0 sec 111 msec

and paste in a spreadsheet program.

Both CLIPBOARD and CLIPOUT are virtual tables. CLIPOUT is used in an 
"inverted form" which is easier to write, when chaining queries.

-- Lefteris
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] SELECT query first run is VERY slow

[Max Vlasov]

On Mon, Jul 25, 2011 at 5:45 PM, Григорий Григоренко  wrote:
>
> 1) SQLITE has to read about _half of index_ before it can use it (and 
> understand there are no records matching query).
>
> If cache is enough to hold 50 Mb then on subsuquent queries sqlite process is 
> not reading at all.
>

Please, post your query. To understand whether sqlite reads too much
or not it's better to know what exactly you want to select.


> 2) SQLITE is reading abnormally slowly during this first-time running query  
> (waiting for something a lot?).
>

Is this with the recreated index or still the one that was created
during the lifetime of your program?

Max
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] SQLite3.dll for Win 64

[Everton Vieira]

ops! I've run on cmd

2011/7/25 John Drescher 

> On Mon, Jul 25, 2011 at 10:52 AM, Everton Vieira 
> wrote:
> > So far so good but return an error:
> >
> > C:\Dattna\Applications\Installs\sqlite\sqlite-src-3070701>nmake -f
> > Makefile.msc sqlite3.dll
> >
> > Microsoft (R) Program Maintenance Utility Version 10.00.30319.01
> > Copyright (C) Microsoft Corporation.  All rights reserved.
> >cl.exe -O2 -Femkkeywordhash.exe  -DSQLITE_ENABLE_FTS3=1
> > -DSQLITE_ENABLE_
> > RTREE=1 -DSQLITE_ENABLE_COLUMN_METADATA=1 -DSQLITE_MAX_TRIGGER_DEPTH=100
> > .\tool
> > \mkkeywordhash.c
> > Microsoft (R) C/C++ Optimizing Compiler Version 16.00.30319.01 for x64
> > Copyright (C) Microsoft Corporation.  All rights reserved.
> > mkkeywordhash.c
> > .\tool\mkkeywordhash.c(6) : fatal error C1034: stdio.h: no include path
> set
> > NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual
> Studio
> > 10.0
> > \VC\bin\amd64\cl.exe"' : return code '0x2'
> > Stop.
> > Anyone knows how can I include this path that is missing?
> >
>
> Did you run that from a "Visual Studio 2010 command prompt" ?
>
> John
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>



-- 
Everton Vieira.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] SELECT query first run is VERY slow

[Black, Michael (IS)]

You need to normalize your "kind" value.



.pragma cache_size=15000;

drop index idxlog_kind_computer;

create table kind(id integer,kind text);
insert into kind values(1,'debug');
insert into kind values(2,'error');
insert into kind values(3,'info');
insert into kind values(4,'timing');
insert into kind values(5,'warn');
update log set kind=1 where kind='debug';
update log set kind=2 where kind='error';
update log set kind=3 where kind='info';
update log set kind=4 where kind='timing';
update log set kind=5 where kind='warn';
create index idxlog_kind_computer ON log(kind,computer);



Then see how long your first query takes.



Michael D. Black

Senior Scientist

NG Information Systems

Advanced Analytics Directorate




From: sqlite-users-boun...@sqlite.org [sqlite-users-boun...@sqlite.org] on 
behalf of Григорий Григоренко [grigore...@mail.ru]
Sent: Monday, July 25, 2011 8:45 AM
To: sqlite-users@sqlite.org
Subject: EXT :Re: [sqlite]SELECT query first run is VERY slow

I think I narrowed the problem a bit.  Guys, hope I'm not bothering you too 
much :)


I've calculated size of index (it is index on log (kind,computer) ) of its own: 
dropped index, run VACUUM and re-created index.

Database file increased by 105 Mb (and sqlite3 process counter shows that there 
were ~105 Mb written to disk).


This means that index on log(kind, computer) takes 105 Mb of database file (and 
whole size of database is 1259 Mb).


Now, I'm running query which is using this index (and is not returning any 
data) and monitor that sqlite3 process reads ~50 Mb.


So there are  two major problems here.

1) SQLITE has to read about _half of index_ before it can use it (and 
understand there are no records matching query).

If cache is enough to hold 50 Mb then on subsuquent queries sqlite process is 
not reading at all.

2) SQLITE is reading abnormally slowly during this first-time running query  
(waiting for something a lot?).

During index creation I monitored sqlite3 process and it was consuming CPU at 
~20% rate and it's doing I/O at ~10 Mb per second rate.
That's what I call "normal load"!














___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Reducing compiled size

[Dave Hope]

Hi Richard,

I'm using gcc version 4.4.3 (Ubuntu 4.4.3-4ubuntu5). Also tried with
VC9 which yields very similar sizes.

$ gcc main.c sqlite3.c -lpthread -ldl
$ size a.out
   textdata bss dec hex filename
 51447536881168  519331   7eca3 a.out

It looks like this could have been because I wasn't optimizing for size:

$ gcc main.c sqlite3.c -lpthread -ldl -Os
$ size a.out
   textdata bss dec hex filename
 32648735881000  331075   50d43 a.out

Thanks for pointing out what I missed,

Dave

On Mon, Jul 25, 2011 at 4:07 PM, Richard Hipp  wrote:
> On Mon, Jul 25, 2011 at 10:39 AM, Dave Hope  wrote:
>
>> Hi all,
>>
>> I read on the sqlite website that the compiled size can be reduced to
>> 275KiB ( http://www.sqlite.org/different.html#small ), however I'm
>> struggling to get it much below 500 KiB using the SQLITE_OMIT_*
>> options. I'm producing my own amalgamation, which I believe you can
>> use the OMIT options with:
>>
>
> So I type:
>
>    gcc -Os -c sqlite3.c; size sqlite3.o
>
> And on Linux with gcc 4.1.0 I get 333802 bytes.  This is without any OMIT
> options.  And you cannot get it below 500KB?  What compiler are you using?
>
>
> --
> D. Richard Hipp
> d...@sqlite.org
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Reducing compiled size

[Richard Hipp]

On Mon, Jul 25, 2011 at 10:39 AM, Dave Hope  wrote:

> Hi all,
>
> I read on the sqlite website that the compiled size can be reduced to
> 275KiB ( http://www.sqlite.org/different.html#small ), however I'm
> struggling to get it much below 500 KiB using the SQLITE_OMIT_*
> options. I'm producing my own amalgamation, which I believe you can
> use the OMIT options with:
>

So I type:

gcc -Os -c sqlite3.c; size sqlite3.o

And on Linux with gcc 4.1.0 I get 333802 bytes.  This is without any OMIT
options.  And you cannot get it below 500KB?  What compiler are you using?


-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Reducing compiled size

[Dave Hope]

Thanks Stephan

Sorry, you're quite right. I was actually using OPTS="" but omitted it
when I pasted it into my e-mail.

Thanks

Dave

On Mon, Jul 25, 2011 at 3:55 PM, Stephan Beal  wrote:
> On Mon, Jul 25, 2011 at 4:51 PM, Dave Hope  wrote:
>
>> Hi Stephan,
>>
>> I initially added these to the Makefile, but looking at the mailing
>> list suggested otherwise:
>>
>> http://www.mail-archive.com/sqlite-users@sqlite.org/msg37813.html
>
>
> That post says to use:
>
> make "OPTS=-..."
>
> So it seems that OPTS= is missing.
>
> --
> - stephan beal
> http://wanderinghorse.net/home/stephan/
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Reducing compiled size

[Stephan Beal]

On Mon, Jul 25, 2011 at 4:51 PM, Dave Hope  wrote:

> Hi Stephan,
>
> I initially added these to the Makefile, but looking at the mailing
> list suggested otherwise:
>
> http://www.mail-archive.com/sqlite-users@sqlite.org/msg37813.html


That post says to use:

make "OPTS=-..."

So it seems that OPTS= is missing.

-- 
- stephan beal
http://wanderinghorse.net/home/stephan/
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] SQLite3.dll for Win 64

[John Drescher]

On Mon, Jul 25, 2011 at 10:52 AM, Everton Vieira  wrote:
> So far so good but return an error:
>
> C:\Dattna\Applications\Installs\sqlite\sqlite-src-3070701>nmake -f
> Makefile.msc sqlite3.dll
>
> Microsoft (R) Program Maintenance Utility Version 10.00.30319.01
> Copyright (C) Microsoft Corporation.  All rights reserved.
>        cl.exe -O2 -Femkkeywordhash.exe  -DSQLITE_ENABLE_FTS3=1
> -DSQLITE_ENABLE_
> RTREE=1 -DSQLITE_ENABLE_COLUMN_METADATA=1 -DSQLITE_MAX_TRIGGER_DEPTH=100
> .\tool
> \mkkeywordhash.c
> Microsoft (R) C/C++ Optimizing Compiler Version 16.00.30319.01 for x64
> Copyright (C) Microsoft Corporation.  All rights reserved.
> mkkeywordhash.c
> .\tool\mkkeywordhash.c(6) : fatal error C1034: stdio.h: no include path set
> NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual Studio
> 10.0
> \VC\bin\amd64\cl.exe"' : return code '0x2'
> Stop.
> Anyone knows how can I include this path that is missing?
>

Did you run that from a "Visual Studio 2010 command prompt" ?

John
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] SQLite3.dll for Win 64

[Everton Vieira]

So far so good but return an error:

C:\Dattna\Applications\Installs\sqlite\sqlite-src-3070701>nmake -f
Makefile.msc sqlite3.dll

Microsoft (R) Program Maintenance Utility Version 10.00.30319.01
Copyright (C) Microsoft Corporation.  All rights reserved.
cl.exe -O2 -Femkkeywordhash.exe  -DSQLITE_ENABLE_FTS3=1
-DSQLITE_ENABLE_
RTREE=1 -DSQLITE_ENABLE_COLUMN_METADATA=1 -DSQLITE_MAX_TRIGGER_DEPTH=100
.\tool
\mkkeywordhash.c
Microsoft (R) C/C++ Optimizing Compiler Version 16.00.30319.01 for x64
Copyright (C) Microsoft Corporation.  All rights reserved.
mkkeywordhash.c
.\tool\mkkeywordhash.c(6) : fatal error C1034: stdio.h: no include path set
NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual Studio
10.0
\VC\bin\amd64\cl.exe"' : return code '0x2'
Stop.
Anyone knows how can I include this path that is missing?

Many Thanks.
Everton
2011/7/22 Richard Hipp 

> On Fri, Jul 22, 2011 at 7:30 AM, Everton Vieira 
> wrote:
>
> > anyone knows how to make this
> > dll with the source?
> >
>
> nmake -f Makefile.msc sqlite3.dll
>
>
> --
> D. Richard Hipp
> d...@sqlite.org
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>



-- 
Everton Vieira.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Reducing compiled size

[Dave Hope]

Hi Stephan,

I initially added these to the Makefile, but looking at the mailing
list suggested otherwise:

http://www.mail-archive.com/sqlite-users@sqlite.org/msg37813.html

Thanks

Dave

On Mon, Jul 25, 2011 at 3:46 PM, Stephan Beal  wrote:
> On Mon, Jul 25, 2011 at 4:39 PM, Dave Hope  wrote:
>
>> make sqlite3.c "-DSQLITE_OMIT_ALTERTABLE -DSQLITE_OMIT_ANALYZE
>>
>
> Shouldn't that be:
>
> make sqlite3.c CPPFLAGS="..."
>
> ?
>
> Make doesn't understand -D (no, that's not a smiley).
>
> --
> - stephan beal
> http://wanderinghorse.net/home/stephan/
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Reducing compiled size

[Stephan Beal]

On Mon, Jul 25, 2011 at 4:39 PM, Dave Hope  wrote:

> make sqlite3.c "-DSQLITE_OMIT_ALTERTABLE -DSQLITE_OMIT_ANALYZE
>

Shouldn't that be:

make sqlite3.c CPPFLAGS="..."

?

Make doesn't understand -D (no, that's not a smiley).

-- 
- stephan beal
http://wanderinghorse.net/home/stephan/
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] SELECT query first run is VERY slow

[Luuk]

On 25-07-2011 15:45, Григорий Григоренко wrote:
> I think I narrowed the problem a bit.  Guys, hope I'm not bothering you too 
> much :)
> 
> 
> I've calculated size of index (it is index on log (kind,computer) ) of its 
> own: dropped index, run VACUUM and re-created index.


I thought that, if you have an index on a,b,c,d
than you should not have an index on a,b,c too
because if you use those 3 field in the where-clause, use can be made of
the 4-field index

Therefore i think you can do:
DROP INDEX idxlog_kind;
DROP INDEX idxlog_kind_computer;
DROP INDEX idxlog_kind_computer_process;


It drops your db-size, and a 'normal' select will make use of the
idxlog_kind_computer_process_who index.


C:\TEMP>sqlite3 2011-07-24.dblite   0run.cmd

C:\TEMP>sqlite3 2011-07-24.dblite   0http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] Reducing compiled size

[Dave Hope]

Hi all,

I read on the sqlite website that the compiled size can be reduced to
275KiB ( http://www.sqlite.org/different.html#small ), however I'm
struggling to get it much below 500 KiB using the SQLITE_OMIT_*
options. I'm producing my own amalgamation, which I believe you can
use the OMIT options with:

"Special versions of the SQLite amalgamation that do work with a
predetermined set of SQLITE_OMIT_* options can be generated." (
http://www.sqlite.org/compile.html#omitfeatures ).

Here's what I'm doing to compile:

./configure CFLAGS="-DSQLITE_OMIT_ALTERTABLE -DSQLITE_OMIT_ANALYZE
-DSQLITE_OMIT_ATTACH -DSQLITE_OMIT_AUTHORIZATION
-DSQLITE_OMIT_AUTOINCREMENT -DSQLITE_OMIT_AUTOMATIC_INDEX
-DSQLITE_OMIT_AUTORESET -DSQLITE_OMIT_AUTOVACUUM
-DSQLITE_OMIT_BETWEEN_OPTIMIZATION -DSQLITE_OMIT_BLOB_LITERAL
-DSQLITE_OMIT_BTREECOUNT -DSQLITE_OMIT_BUILTIN_TEST -DSQLITE_OMIT_CAST
-DSQLITE_OMIT_CHECK -DSQLITE_OMIT_COMPILEOPTION_DIAGS
-DSQLITE_OMIT_COMPLETE -DSQLITE_OMIT_DATETIME_FUNCS
-DSQLITE_OMIT_DEPRECATED -DSQLITE_OMIT_EXPLAIN
-DSQLITE_OMIT_FLAG_PRAGMAS -DSQLITE_OMIT_FLOATING_POINT
-DSQLITE_OMIT_FOREIGN_KEY -DSQLITE_OMIT_GET_TABLE
-DSQLITE_OMIT_INCRBLOB -DSQLITE_OMIT_INTEGRITY_CHECK
-DSQLITE_OMIT_LIKE_OPTIMIZATION -DSQLITE_OMIT_LOAD_EXTENSION
-DSQLITE_OMIT_LOCALTIME -DSQLITE_OMIT_MEMORYDB
-DSQLITE_OMIT_OR_OPTIMIZATION -DSQLITE_OMIT_PAGER_PRAGMAS
-DSQLITE_OMIT_PRAGMA -DSQLITE_OMIT_PROGRESS_CALLBACK
-DSQLITE_OMIT_QUICKBALANCE -DSQLITE_OMIT_REINDEX
-DSQLITE_OMIT_SCHEMA_PRAGMAS -DSQLITE_OMIT_SCHEMA_VERSION_PRAGMAS
-DSQLITE_OMIT_SHARED_CACHE -DSQLITE_OMIT_TCL_VARIABLE
-DSQLITE_OMIT_TEMPDB -DSQLITE_OMIT_TRACE -DSQLITE_OMIT_TRIGGER
-DSQLITE_OMIT_TRUNCATE_OPTIMIZATION -DSQLITE_OMIT_UTF16
-DSQLITE_OMIT_VACUUM -DSQLITE_OMIT_VIEW -DSQLITE_OMIT_VIRTUALTABLE
-DSQLITE_OMIT_WAL -DSQLITE_OMIT_WSD -DSQLITE_OMIT_XFER_OPT"

make sqlite3.c "-DSQLITE_OMIT_ALTERTABLE -DSQLITE_OMIT_ANALYZE
-DSQLITE_OMIT_ATTACH -DSQLITE_OMIT_AUTHORIZATION
-DSQLITE_OMIT_AUTOINCREMENT -DSQLITE_OMIT_AUTOMATIC_INDEX
-DSQLITE_OMIT_AUTORESET -DSQLITE_OMIT_AUTOVACUUM
-DSQLITE_OMIT_BETWEEN_OPTIMIZATION -DSQLITE_OMIT_BLOB_LITERAL
-DSQLITE_OMIT_BTREECOUNT -DSQLITE_OMIT_BUILTIN_TEST -DSQLITE_OMIT_CAST
-DSQLITE_OMIT_CHECK -DSQLITE_OMIT_COMPILEOPTION_DIAGS
-DSQLITE_OMIT_COMPLETE -DSQLITE_OMIT_DATETIME_FUNCS
-DSQLITE_OMIT_DEPRECATED -DSQLITE_OMIT_EXPLAIN
-DSQLITE_OMIT_FLAG_PRAGMAS -DSQLITE_OMIT_FLOATING_POINT
-DSQLITE_OMIT_FOREIGN_KEY -DSQLITE_OMIT_GET_TABLE
-DSQLITE_OMIT_INCRBLOB -DSQLITE_OMIT_INTEGRITY_CHECK
-DSQLITE_OMIT_LIKE_OPTIMIZATION -DSQLITE_OMIT_LOAD_EXTENSION
-DSQLITE_OMIT_LOCALTIME -DSQLITE_OMIT_MEMORYDB
-DSQLITE_OMIT_OR_OPTIMIZATION -DSQLITE_OMIT_PAGER_PRAGMAS
-DSQLITE_OMIT_PRAGMA -DSQLITE_OMIT_PROGRESS_CALLBACK
-DSQLITE_OMIT_QUICKBALANCE -DSQLITE_OMIT_REINDEX
-DSQLITE_OMIT_SCHEMA_PRAGMAS -DSQLITE_OMIT_SCHEMA_VERSION_PRAGMAS
-DSQLITE_OMIT_SHARED_CACHE -DSQLITE_OMIT_TCL_VARIABLE
-DSQLITE_OMIT_TEMPDB -DSQLITE_OMIT_TRACE -DSQLITE_OMIT_TRIGGER
-DSQLITE_OMIT_TRUNCATE_OPTIMIZATION -DSQLITE_OMIT_UTF16
-DSQLITE_OMIT_VACUUM -DSQLITE_OMIT_VIEW -DSQLITE_OMIT_VIRTUALTABLE
-DSQLITE_OMIT_WAL -DSQLITE_OMIT_WSD -DSQLITE_OMIT_XFER_OPT"

Any suggestions as to where I'm going wrong?

Thanks

Dave
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] SELECT query first run is VERY slow

[Григорий Григоренко]

I think I narrowed the problem a bit.  Guys, hope I'm not bothering you too 
much :)


I've calculated size of index (it is index on log (kind,computer) ) of its own: 
dropped index, run VACUUM and re-created index.

Database file increased by 105 Mb (and sqlite3 process counter shows that there 
were ~105 Mb written to disk).


This means that index on log(kind, computer) takes 105 Mb of database file (and 
whole size of database is 1259 Mb).


Now, I'm running query which is using this index (and is not returning any 
data) and monitor that sqlite3 process reads ~50 Mb. 


So there are  two major problems here.

1) SQLITE has to read about _half of index_ before it can use it (and 
understand there are no records matching query).

If cache is enough to hold 50 Mb then on subsuquent queries sqlite process is 
not reading at all. 

2) SQLITE is reading abnormally slowly during this first-time running query  
(waiting for something a lot?).

During index creation I monitored sqlite3 process and it was consuming CPU at 
~20% rate and it's doing I/O at ~10 Mb per second rate.
That's what I call "normal load"! 














___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] EXT :Re: SELECT query first run is VERY slow

[Григорий Григоренко]

> 
> #1 I take it your query is CPU bound the first time?

Hmm, not sure. CPU was still under 1 % usage.

> 
> #2 Can you show us the query planner please?
> #3 Can you show us the query planner minus the "INDEXED BY"?

Sure. Executing:

SELECT * FROM log INDEXED BY idxlog_kind_computer  WHERE kind = 'info' AND 
computer=1 and id > 7070636 LIMIT 100;

Query plan:

0|0|0|SEARCH TABLE log USING INDEX idxlog_kind_computer (kind=? AND computer=?) 
(~3 rows)

Executing:

SELECT * FROM log WHERE kind = 'info' AND computer=1 and id > 7070636 LIMIT 100;

Query plan:

0|0|0|SEARCH TABLE log USING INDEX idxlog_kind_computer_process_who (kind=? AND 
computer=?) (~3 rows)


Query planner has selected idxlog_kind_computer_process_who index.



> 
> 

> #4 Can you show us sqlite_stat1?

Actually, there isn't any. I've not ran ANALYZE on this database. 
Since I'm using INDEXED BY there's no need in query planner decisions.

> 
> #5 Can you show us your tables now?
> 

You mean db schema?

// table with computer, process and who values
CREATE TABLE item(name text,value text);
CREATE UNIQUE INDEX idxitem_name_value ON item(name,value);

// log-table
CREATE TABLE log(id integer primary key autoincrement,msg text,created_at 
int,kind,computer,process,who);

CREATE INDEX idxlog_created_at ON log(created_at);
CREATE INDEX idxlog_kind ON log(kind);
CREATE INDEX idxlog_kind_computer ON log(kind,computer);
CREATE INDEX idxlog_kind_computer_process ON log(kind,computer,process);
CREATE INDEX idxlog_kind_computer_process_who ON log(kind,computer,process,who);

Since I've normalized data selecting (in real-life application) looks like:
SELECT ... kind,msg,c.value as computer,p.value as process,w.value as 
who,created_at,id FROM log ...
 LEFT JOIN item c ON c.rowid=log.computer 
 LEFT JOIN item p ON p.rowid=log.process 
 LEFT JOIN item w ON w.rowid=log.who 
...



> #6 What happens if you do "pragma cache_size=15000"?  I'd make cache_size = 
> (readbytes/8192)*1.1 at least.

(readbytes/8192)*1.1 gives us ~ 6700 pages. Tried "pragma cache_size=15000;" 
and nothing changed. 

Sqlite3 still has read ~ 50 Mb and spent 27 seconds on first query run.



> 
> #7 Care to post your database again?
> 

http://dl.dropbox.com/u/2168777/db2.rar

> 
> 
> It still seems to me this should run faster the first time unless you have a 
> really slow disk system or sqlite is doing something silly (which I doubt).
> 
> Perhaps the cache being to small is hurting things.

I'm not  sure cache settings are important.

You see, I've tried running query with different cache_size. Even with 
cache_size=100. 
Tried cache_size from 100 to 25000 and monitored memory usage of console 
process to make sure
it was using only the amount of memory given by cache.

Timing is _always_ ~ 1 second on subsequent (not first) runs. You can try it 
yourself with attached data.







> 
> 
> 
> Michael D. Black
> 
> Senior Scientist
> 
> NG Information Systems
> 
> Advanced Analytics Directorate
> 
> 
> 
> 
> From: sqlite-users-boun...@sqlite.org [sqlite-users-boun...@sqlite.org] on 
> behalf of Григорий Григоренко [grigore...@mail.ru]
> Sent: Monday, July 25, 2011 3:30 AM
> To: sqlite-users
> Subject: EXT :Re: [sqlite]SELECT query first run is VERY slow
> 
> Okay, here are some steps I've done.
> 
> 1) normalize db;
> 
> I've created single table (item). "computer","process" and "who" fields in 
> log table became rowid integers pointing to this table.
> "kind" is still a short string.
> 
> 2) give up covering indexes;
> 
> For those not aware (http://www.sqlite.org/queryplanner.html , 
> 1.7 Covering Indices) it's an index that has additional columns at it's end ( 
> being selected by query).
> Covering index eliminate the need to read data from db records. But it 
> increases size of index and size of database.
> 
> 3) use INDEXED BY to suggest index to query planner.
> 
> Since indexes are created exactly for specific query.
> 
> 
> Software was working during weekend.
> 
> Yesterday's database has 6 mln records and it's only 1.1 Gb in size.
> 
> Comparing to 4 mln records and 3.5 Gb size before.
> 
> Now, the long running query took 27 seconds and it has read 50 Mb from 
> database (compare to 2 minutes and 307 Mb before).
> 
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
> 
> 
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] System.DllNotFoundException: SQLite.Interop.DLL

[Grant Dunoon]

Thanks for persisting with me on this, I recompiled it as your email below
and it now runs in both windows and linux. I have not tried the Debug
version again, so the problem first time around could have been finger
trouble.

On a side note it look like I'm also getting a memory leak in linux. Will
keep investigating and open a new thread if I find anything.

Thanks again.
Grant


On 25 July 2011 05:47, Joe Mistachkin  wrote:

> Very odd, with the "UseSqliteStandard" property enabled, that error message
> should not be possible.
>
> How are you compiling the SQLite.Interop project?
>
> When you compiled the System.Data.SQLite assembly, did you use the real
> MSBuild (i.e. not XBuild)?
>
> The command line I gave you would have produced a "Debug" build, did you
> grab
> the assembly from that directory?
>
> For a "Release" build, you can use the following command line (all on one
> line):
>
>MSBuild System.Data.SQLite.2008.csproj /t:Rebuild
> /p:Configuration=Release
> /p:UseInteropDll=false /p:UseSqliteStandard=true
>
> One potential problem I thought of the other day was the casing of the file
> extension on the DLLs.  I checked in a change to make the file extensions
> for
> the DLLs all lower case.
>
> Another possible issue is (although, this was Windows specific):
>
>https://bugzilla.novell.com/show_bug.cgi?id=636915
>
> --
> Joe Mistachkin
>
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>



-- 
Grant Dunoon
www.TryBooking.com
Tel: +61 (0) 402 993 808
Tel: +61 (3) 9012 3460
Skype (W): grant.dunoon
Email: gr...@dunoon.com.au
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] EXT :Re: SELECT query first run is VERY slow

[Black, Michael (IS)]

An 8X speedup is a pretty good achievement...congrats...



#1 I take it your query is CPU bound the first time?

#2 Can you show us the query planner please?

#3 Can you show us the query planner minus the "INDEXED BY"?

#4 Can you show us sqlite_stat1?

#5 Can you show us your tables now?

#6 What happens if you do "pragma cache_size=15000"?  I'd make cache_size = 
(readbytes/8192)*1.1 at least.

#7 Care to post your database again?



It still seems to me this should run faster the first time unless you have a 
really slow disk system or sqlite is doing something silly (which I doubt).

Perhaps the cache being to small is hurting things.



Michael D. Black

Senior Scientist

NG Information Systems

Advanced Analytics Directorate




From: sqlite-users-boun...@sqlite.org [sqlite-users-boun...@sqlite.org] on 
behalf of Григорий Григоренко [grigore...@mail.ru]
Sent: Monday, July 25, 2011 3:30 AM
To: sqlite-users
Subject: EXT :Re: [sqlite]SELECT query first run is VERY slow

Okay, here are some steps I've done.

1) normalize db;

I've created single table (item). "computer","process" and "who" fields in log 
table became rowid integers pointing to this table.
"kind" is still a short string.

2) give up covering indexes;

For those not aware (http://www.sqlite.org/queryplanner.html , 
1.7 Covering Indices) it's an index that has additional columns at it's end ( 
being selected by query).
Covering index eliminate the need to read data from db records. But it 
increases size of index and size of database.

3) use INDEXED BY to suggest index to query planner.

Since indexes are created exactly for specific query.


Software was working during weekend.

Yesterday's database has 6 mln records and it's only 1.1 Gb in size.

Comparing to 4 mln records and 3.5 Gb size before.

Now, the long running query took 27 seconds and it has read 50 Mb from database 
(compare to 2 minutes and 307 Mb before).

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] System.Data.SQLite: commands not persisting on db.

[Simon Slavin]

On 25 Jul 2011, at 8:26am, Mattia wrote:

> I've tried doing transactions (committing and disposing them after each one)
> and yes, I've closed the database after doing my operations, with no luck at
> all.

You understand that you don't do this:

BEGIN TRANSACTION
insert 1
END TRANSACTION
BEGIN TRANSACTION
insert 2
END TRANSACTION
BEGIN TRANSACTION
insert 3
END TRANSACTION

but instead you do this:

BEGIN TRANSACTION
insert 1
insert 2
insert 3
END TRANSACTION

right ?

Simon.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] SELECT query first run is VERY slow

[Григорий Григоренко]

Okay, here are some steps I've done.

1) normalize db;

I've created single table (item). "computer","process" and "who" fields in log 
table became rowid integers pointing to this table. 
"kind" is still a short string.

2) give up covering indexes;

For those not aware (http://www.sqlite.org/queryplanner.html , 1.7 Covering 
Indices) it's an index that has additional columns at it's end ( being selected 
by query).
Covering index eliminate the need to read data from db records. But it 
increases size of index and size of database.

3) use INDEXED BY to suggest index to query planner.

Since indexes are created exactly for specific query.


Software was working during weekend. 

Yesterday's database has 6 mln records and it's only 1.1 Gb in size.

Comparing to 4 mln records and 3.5 Gb size before.

Now, the long running query took 27 seconds and it has read 50 Mb from database 
(compare to 2 minutes and 307 Mb before).

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] System.Data.SQLite: commands not persisting on db.

[Mattia]

Hello, sorry for my late reply but I've been away the past week.
I've tried doing transactions (committing and disposing them after each one)
and yes, I've closed the database after doing my operations, with no luck at
all.

Mat.

2011/7/15 Simon Slavin 

>
> On 15 Jul 2011, at 4:11pm, Jim Morris wrote:
>
> > On 7/15/2011 1:26 AM, Mattia wrote:
> >> - deleting the old database (data.db) and renaming new.db with the
> >> correct name (new.db becomes data.db).
> > After this step, as a test, reopen the connection to new.db and ensure
> > that the data is there.
> >
> > Are you sure a commit is done before closing connection?
>
> I don't think Mattia has declared any transactions at all, so each change
> is done as its own transaction.  But your other point is valid:
>
> Mattia: you are actually performing some instruction to close the database,
> right ?  You're not just letting your application exit ?
>
> Simon.
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>



-- 
-mat
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users