[sqlite] [bug]DSQLITE_OMIT_DISKIO link failure

[weiyang wang]

Hi,

With checking the source code build, i could see DSQLITE_OMIT_DISKIO does
not work properly.

the problem is that symbols, which are related to DSQLITE_OMIT_DISKIO in
btree.c and attach.c, are not properly tied to DSQLITE_OMIT_DISKIO. so
caused linker errors.

best regards,
wang

Re: [sqlite] Re: sqlite_omit_xx build failure

[weiyang wang]

thanks.

that is correct. i have just found this issue 5 mins ago. i am re-doing the
compiling now.

and .

it works.

thanks. have a nice day

wang



On 6/12/07, Dan Kennedy <[EMAIL PROTECTED]> wrote:


On Tue, 2007-06-12 at 09:56 +0200, weiyang wang wrote:
> Hi Dr.H,
> Yes, I did a clean build in a clean directory( (top)/bld ). and i had
run
> 'make clean' before i run the 'make'.
>
> i am wondering whether the lemon tool configuration in my environment is
> well done or not.
>
> thanks in advance.
>
> wang
>
> On 6/11/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote:
> >
> > "weiyang wang" <[EMAIL PROTECTED]> wrote:
> > > hi,
> > >
> > > i am trying to get a smaller sqlite lib by adding sqlite_omit_xx
> > defines.
> > > but it failed at the compiler time.
> > >
> > > steps:
> > > 1, in 'Makefile.in <http://makefile.in/>', omit macro is added:
> > >
> > >   TCC += -DSQLITE_OMIT_ALTERTABLE=1

Do this as well:

  OPTS += -DSQLITE_OMIT_ALTERTABLE=1

The key is that the same set of -DSQLITE_OMIT_ options have
to be passed to "lemon" as the C compiler.

Dan.

> > >
> > > 2, run '(top)/configure' and 'make' in cygwin,
> > >
> > > 3, i got the fowllowing error:
> > >
> >
> > Did you do this from a clean directory?  Did you
> > do "make clean" before doing "make"?
> > --
> > D. Richard Hipp <[EMAIL PROTECTED]>
> >
> >
> >
-
> >
> > To unsubscribe, send email to [EMAIL PROTECTED]
> >
> >
-
> >
> >



-
To unsubscribe, send email to [EMAIL PROTECTED]

-

Re: [sqlite] Re: sqlite_omit_xx build failure

[weiyang wang]

Hi Dr.H,
Yes, I did a clean build in a clean directory( (top)/bld ). and i had run
'make clean' before i run the 'make'.

i am wondering whether the lemon tool configuration in my environment is
well done or not.

thanks in advance.

wang

On 6/11/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote:


"weiyang wang" <[EMAIL PROTECTED]> wrote:
> hi,
>
> i am trying to get a smaller sqlite lib by adding sqlite_omit_xx
defines.
> but it failed at the compiler time.
>
> steps:
> 1, in 'Makefile.in <http://makefile.in/>', omit macro is added:
>
>   TCC += -DSQLITE_OMIT_ALTERTABLE=1
>
> 2, run '(top)/configure' and 'make' in cygwin,
>
> 3, i got the fowllowing error:
>

Did you do this from a clean directory?  Did you
do "make clean" before doing "make"?
--
D. Richard Hipp <[EMAIL PROTECTED]>


-

To unsubscribe, send email to [EMAIL PROTECTED]

-

[sqlite] sqlite_omit_xx build failure

[weiyang wang]

hi,

i am trying to get a smaller sqlite lib by adding sqlite_omit_xx defines.
but it failed at the compiler time.

steps:
1, in 'Makefile.in', omit macro is added:

 TCC += -DSQLITE_OMIT_ALTERTABLE=1

2, run '(top)/configure' and 'make' in cygwin,

3, i got the fowllowing error:

./.libs/libsqlite3.a(parse.o): In function `sqlite3Parser':
/home/ww1/sqlite-3.3.17/bld/parse.y:1066: undefined reference to
`_sqlite3AlterB
eginAddColumn'
/home/ww1/sqlite-3.3.17/bld/parse.y:1063: undefined reference to
`_sqlite3AlterF
inishAddColumn'
/home/ww1/sqlite-3.3.17/bld/parse.y:1060: undefined reference to
`_sqlite3AlterR
enameTable'

it seems that parse.y is not stripped with the omit define
(SQLITE_OMIT_ALTERTABLE) so that parse.c still contains the unwanted
symbols.

it is said that the omit defines must also be fed to lemon tool to gererate
the correct parse.c, but i failed to find out where it is.

appreciation will be given for any hint.

wang

Re: [sqlite] PRAGMA cache_size = 0

[weiyang wang]

correct my former message, the smallest is 10, (10 pages in ram)

On 6/11/07, B V, Phanisekhar <[EMAIL PROTECTED]> wrote:


What happens if I set the cache_size to 0? Will I be able to do any of
update/delete/insert/select operations?

When I set the cache_size to 0, is there any freeing up of memory by
sqlite?



Regards,

Phani

Re: [sqlite] PRAGMA cache_size = 0

[weiyang wang]

the smallest cache_size is 512k, i guess.

On 6/11/07, B V, Phanisekhar <[EMAIL PROTECTED]> wrote:


What happens if I set the cache_size to 0? Will I be able to do any of
update/delete/insert/select operations?

When I set the cache_size to 0, is there any freeing up of memory by
sqlite?



Regards,

Phani

Re: [sqlite] sqlite3_temp_directory in main.c

[weiyang wang]

by checking the codes, i could see temp_store_directory is used to overide
the platform-specific default temp directory in run time.
That is, if temp_store_directory is 0 (the default value of
temp_store_directory), then OS layer (os_win.x and os_unix.x) will provide
the OS default temp directory. (for example, */var/tmp, /usr/tmp for unix*).


if temp_store_directory is configured by pragma, then the
configured  temp_store_directory will be used instead of OS default one.

the following code is from os_win.c :: sqlite3WinTempFileName()

 if( sqlite3_temp_directory )
 {
  ... ...
 }else if( isNT() )
 {
  ... ...
 }



On 6/7/07, Tom Briggs <[EMAIL PROTECTED]> wrote:



  I can see your point, I guess, though I can't say that it seems like
a major issue to me.

  Just out of curiosity, why aren't the defaults derived in os_win.x
and os_unix.c sufficient?

> -Original Message-
> From: weiyang wang [mailto:[EMAIL PROTECTED]
> Sent: Wednesday, June 06, 2007 11:17 AM
> To: sqlite-users@sqlite.org
> Subject: Re: [sqlite] sqlite3_temp_directory in main.c
>
> the concern here is to configure the default temp directory in library
> compile time, on the OS porting layer.
> in current version, the default value of
> sqlite3_temp_directory is hardcoded
> as 0 in main.c for all platforms.
> would it be better to make it configrable for different
> platforms and use
> PRAGMA to overide the defult in runtime.
>
> i am looking forward to your opiniions.
>
> thanks again.
>
> wang
>
> On 6/6/07, Tom Briggs <[EMAIL PROTECTED]> wrote:
> >
> >
> >   Why not just use PRAGMA temp_store_directory, as the comments
> > directly above that line suggest?
> >
> > > -Original Message-
> > > From: weiyang wang [mailto:[EMAIL PROTECTED]
> > > Sent: Wednesday, June 06, 2007 7:09 AM
> > > To: sqlite-users
> > > Subject: [sqlite] sqlite3_temp_directory in main.c
> > >
> > > hi,
> > >
> > > i found the following line in main.c
> > >
> > > char *sqlite3_temp_directory = 0;
> > >
> > > which seems to remove the possibilities that the customer
> > > platform could
> > > specify sqlite3_temp_directory.
> > >
> > > i suggest that this line is removed.
> > >
> > > any comments?
> > >
> > > thanks in advance.
> > >
> > > wang
> > >
> >
> >
> >
> --
> ---
> > To unsubscribe, send email to [EMAIL PROTECTED]
> >
> >
> --
> ---
> >
> >
>


-
To unsubscribe, send email to [EMAIL PROTECTED]

-

Re: [sqlite] Re: progress_handler: progress in percentage?

[weiyang wang]

thanks igor.
i will find other ways to achieve the similar.


On 6/6/07, Igor Tandetnik <[EMAIL PROTECTED]> wrote:


weiyang wang  wrote:
> i am looking into the sqlite3_progress_handler() and i am wondering
> the possiblity of representing progress in percentage.

It's practically impossible: there is no way, in general, to know
beforehand how long it will take to complete the query.

Igor Tandetnik



-
To unsubscribe, send email to [EMAIL PROTECTED]

-

Re: [sqlite] sqlite3_temp_directory in main.c

[weiyang wang]

the concern here is to configure the default temp directory in library
compile time, on the OS porting layer.
in current version, the default value of sqlite3_temp_directory is hardcoded
as 0 in main.c for all platforms.
would it be better to make it configrable for different platforms and use
PRAGMA to overide the defult in runtime.

i am looking forward to your opiniions.

thanks again.

wang

On 6/6/07, Tom Briggs <[EMAIL PROTECTED]> wrote:



  Why not just use PRAGMA temp_store_directory, as the comments
directly above that line suggest?

> -Original Message-
> From: weiyang wang [mailto:[EMAIL PROTECTED]
> Sent: Wednesday, June 06, 2007 7:09 AM
> To: sqlite-users
> Subject: [sqlite] sqlite3_temp_directory in main.c
>
> hi,
>
> i found the following line in main.c
>
> char *sqlite3_temp_directory = 0;
>
> which seems to remove the possibilities that the customer
> platform could
> specify sqlite3_temp_directory.
>
> i suggest that this line is removed.
>
> any comments?
>
> thanks in advance.
>
> wang
>


-
To unsubscribe, send email to [EMAIL PROTECTED]

-

[sqlite] progress_handler: progress in percentage?

[weiyang wang]

hi,

i am looking into the sqlite3_progress_handler() and i am wondering the
possiblity of representing progress in percentage.

the period of progressing update is based on number of  virtual machine
opcodes, is there anyway that we can estimate the total amount of opcodes of
one query during the preparation stage so that the progress status could be
shown in percentage.

thanks in advance.

[sqlite] sqlite3_temp_directory in main.c

[weiyang wang]

hi,

i found the following line in main.c

char *sqlite3_temp_directory = 0;

which seems to remove the possibilities that the customer platform could
specify sqlite3_temp_directory.

i suggest that this line is removed.

any comments?

thanks in advance.

wang

Re: [sqlite] file structure issue

[weiyang wang]

Nuno and Vivien thanks for your replies. i will try that.

wang


On 5/23/07, Nuno Lucas <[EMAIL PROTECTED]> wrote:


On 5/23/07, weiyang wang <[EMAIL PROTECTED]> wrote:
> does anyone know how can i get the source codes with the early file
> structure? (65 seperate files)? thanks in advance.

Download the complete source code [1] and run "./configure" on it (you
need a POSIX build system, like linux, cygwin or MSYS).

After that is done, you should have all preprocessed files generated,
so you can just copy them to where you want.


Regards,
~Nuno Lucas

[1] http://www.sqlite.org/sqlite-3.3.17.tar.gz

>
> wang
>


-
To unsubscribe, send email to [EMAIL PROTECTED]

-

[sqlite] file structure issue

[weiyang wang]

hi,

i can see the source file structure has been changed from 65 (sqliteint.h,
os_common.h,. )files before to only 2 files (sqlite3.h)now.
and this change bring me a problem when i try to integrate sqlite with other
platforms.

in my sqlite integration, i have to put my porting layer os_xxx into a
seperate 'cpp' file and to include related defines (defined in
sqliteint.hand os_common.h in early file structure).

and due to compiler issues, i cant include sqlite3.c in my 'cpp' file. i am
trapped here.

does anyone know how can i get the source codes with the early file
structure? (65 seperate files)? thanks in advance.

wang

[sqlite] file structure-can i get the source codes with original structure?

[weiyang wang]

hi,

i can see the source file structure has been changed from 65 files before to
only 2 files now.
and this change bring me a problem when i try to integrate sqlite with other
platforms.

in my sqlite integration, i have to put my porting layer os_xxx into a
seperate 'cpp' file and to include related defines (defined in
sqliteint.hand os_common.h in early file structure).

and due to compiler issues, i cant include sqlite3.c in my 'cpp' file. i am
trapped here.

does anyone know how can i get the source codes with the early file
structure? (65 seperate files)? thanks in advance.

wang

[sqlite] temp file directory

[weiyang wang]

hi,
i found the way to open a temp file is different from other files (.db files
or journal files)

open temp file: open with the relative path. and then point to the fullpath
while open other files: get the fullpath first, and then open.

codes are cited as below:


in pager.c
sqlite3pager_open()

 if( zFilename && zFilename[0] ){
   {
*  zFullPathname = sqlite3OsFullPathname(zFilename);
*  if( zFullPathname ){
*rc = sqlite3OsOpenReadWrite(zFullPathname, , );
*  }
   }
 }else{
*rc = sqlite3pager_opentemp(zTemp, );
*zFilename = zTemp;
*zFullPathname = sqlite3OsFullPathname(zFilename);
*if( rc==SQLITE_OK ){
 tempFile = 1;
   }

can anyone explain what is the concern here? this disorder affected my
porting layer.

thanks

wang

[sqlite] memory footprint and leakage_2

[weiyang wang]

hi,

i realized i missed some codes in sqlite3GenericRealloc() for the
calculating.
after modifying my codes, the testing results are:

the highest water mark of memory usage is 16684 bytes, and 0 memory leakage.


the same configuratiions as described in the early.

16684bytes, is it the right figure?


thanks

wang

[sqlite] memory footprint and leakage

[weiyang wang]

hi,

i am evaluating sqlite performance for mobile platform now.

i found something discussable about memory footprint and leakage.

the highest water line of memeory usage is 44611bytes (by malloc() ), and
there seems a large memory leakage (34017bytes) in my testing scenario.
'cache_size' is set to the smallest, '1', but i dont know why it still use
so much. for low-profile mobile platforms, such amount seems too much.

can somebody tell me something about this?


-the way i measure the memroy usage
in os_common.h

sqlite3GenericMalloc(int n){
 char *p = (char *)malloc(n+8);
 ...
 ...
 if( p ){
   *(int *)p = n;
 }
 currentMemoryConsumption += n;
 if (currentMemoryConsumption > maxMemoryConsuption)
 {
   maxMemoryConsuption = currentMemoryConsumption;
 }
 return (void *)p;
}

void sqlite3GenericFree(void *p){
 assert(p);
 currentMemoryConsumption -= *((int*) ((char*)p - 8));
 ...
 ...
}

--testing
scenario--
the testing is done with windows filesystem,
cache_size=1; page_size = 1k (default),

the database being operated on contains 250 enties, no index:

contacts.db size: 7,168 bytes


CREATE TABLE contacts (name text,phone int);

A.H.|1439
Abel|1654
Absalonsgade,|1658
...
...

the queries are:
select number from contacts where rowid = 1
select number from contacts where rowid = 125
select number from contacts where rowid = 64
INSERT INTO contacts values('kim','')
INSERT INTO contacts values('wang','')
update contacts set phone='34' where name = 'kim'
update contacts set phone='71' where name = 'wang'
update contacts set phone='' where name = 'Eskildsgade,'
update contacts set phone='' where name = 'Bustrupgade,'
begin
INSERT INTO contacts VALUES('abc','49')
INSERT INTO contacts VALUES('klaus','41')
update contacts set phone='1657' where name = 'Eskildsgade,'
update contacts set phone='1737' where name = 'Bustrupgade,'
delete from contacts where name = 'abc'
delete from contacts where name = 'klaus'
commit
delete from contacts where name = 'kim'
delete from contacts WHERE name = 'wang'
delete from contacts where name = 'Eskildsgade,'
delete from contacts WHERE name = 'Bustrupgade,'

Re: [sqlite] journal-off assert

[weiyang wang]

Martin, thanks for your message.

i found out that the problem is actually caused by the missing update for
offset in my porting layer. i thought the two operations: 'seek' and 'write'
work in a strict coupled way, but they are not. so i need update my
structure that stores the offset values each time after the writing or
reading operations.

best regards,

wang


On 8/24/06, Martin Pfeifle <[EMAIL PROTECTED]> wrote:


hi,
i got the same error when I ported sqlite to an operating system using a
proprietary file system.
The reason was that our file system did not support sparse files. i.e.
the fstat-command returned the wrong file-size.

Maybe you should independently of SQLite try to
* create a file,
* write data to it
* do an fseek over the end of the file
* write data to the file
and then determine the size with fstat.
In my case that was not working.
It was a bug of the underlying file system and not of SQLite...
best Martin



- Ursprüngliche Mail 
Von: weiyang wang <[EMAIL PROTECTED]>
An: sqlite-users@sqlite.org
Gesendet: Donnerstag, den 24. August 2006, 09:52:37 Uhr
Betreff: [sqlite] journal-off assert


hi,

i am trying to integrate sqlite into mobile platform, and porting layer,
like 'win_os.c' , have been created for that purpose.

the integrated sqlite works well on our mobile platform with the default
settings, but error come out when i try to tune the performance with
'pragma
cache_size = 1'.

what i have done is:

/* create a new database*/
   sqlite3_open("contacts.db", );

/*tune the cache_size for m */
sqlite3_exec(contactsdb,"pragma default_cache_size = 1" , callback, 0,
);

/*create a table */
   sqlite3_exec(contactsdb,"create table IF NOT EXISTS contacts (name
text,phone int)" , callback, 0, );

   /*insert values into the table, totally 250 entries*/
  for ( i< 250)
{
  sqlite3_exec(("insert xxx,xxx");
}


the symptom is :

after the size of the database increased to 3K, the prgram can not insert
entries into the table anymore. the error returned is ' The database disk
image is malformed'

i have checked the porting layer i have made, the read, write, and offset
operations seems correct.

then, i recompile the sqlite codes with SQLITE_DEBUG and add
'  sqlite3_exec(contactsdb,"PRAGMA vdbe_trace=ON" , callback, 0,
);',
an assetion stalled the execution:   "assert( pPager->journalOff==jSz );"
in
pager.c

the stacked vaules at the broken time is :
pPager->journalOff= 1544
jSz= 1543
file size of contacts.db-journal is 1543 (checked from windows)

i guess the problems are related to file offset, reading, and writing. or
related to bugs of SQlite, i am not quite sure.

so, is there anyone can help me out?


best regards,

yang

[sqlite] journal-off assert

[weiyang wang]

hi,

i am trying to integrate sqlite into mobile platform, and porting layer,
like 'win_os.c' , have been created for that purpose.

the integrated sqlite works well on our mobile platform with the default
settings, but error come out when i try to tune the performance with 'pragma
cache_size = 1'.

what i have done is:

/* create a new database*/
   sqlite3_open("contacts.db", );

/*tune the cache_size for m */
sqlite3_exec(contactsdb,"pragma default_cache_size = 1" , callback, 0,
);

/*create a table */
   sqlite3_exec(contactsdb,"create table IF NOT EXISTS contacts (name
text,phone int)" , callback, 0, );

   /*insert values into the table, totally 250 entries*/
  for ( i< 250)
 {
  sqlite3_exec(("insert xxx,xxx");
 }


the symptom is :

after the size of the database increased to 3K, the prgram can not insert
entries into the table anymore. the error returned is ' The database disk
image is malformed'

i have checked the porting layer i have made, the read, write, and offset
operations seems correct.

then, i recompile the sqlite codes with SQLITE_DEBUG and add
'  sqlite3_exec(contactsdb,"PRAGMA vdbe_trace=ON" , callback, 0,
);',
an assetion stalled the execution:   "assert( pPager->journalOff==jSz );" in
pager.c

the stacked vaules at the broken time is :
pPager->journalOff= 1544
jSz= 1543
file size of contacts.db-journal is 1543 (checked from windows)

i guess the problems are related to file offset, reading, and writing. or
related to bugs of SQlite, i am not quite sure.

so, is there anyone can help me out?


best regards,

yang