Re: [sqlite] porting sqlite on ARM cortex M7

[Mateusz Wajchęprzełóż]

https://www.sqlite.org/custombuild.html
https://www.sqlite.org/vfs.html

śr., 26 cze 2019 o 08:34  napisał(a):

> Dear SQLite Team,
>
>
>
> I am very new to SQLite and want to use SQLite in my project which is based
> on ARM Cortex M7 core. Do we have any examples available for porting SQLite
> to cortex M series microcontroller??
>
>
>
> Regards
>
> Vivek Aggarwal
>
> Firmware Team Lead
>
> NGX Technologies Pvt Ltd
>
>
>
> ___
> sqlite-users mailing list
> sqlite-users@mailinglists.sqlite.org
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] porting sqlite on ARM cortex M7

[Jens Alfke]

> On Jun 25, 2019, at 11:34 PM, vi...@ngxtech.com wrote:
> 
> I am very new to SQLite and want to use SQLite in my project which is based
> on ARM Cortex M7 core. Do we have any examples available for porting SQLite
> to cortex M series microcontroller??

It’s written in highly portable C, so just compile it using your toolchain and 
OS, whatever that is (mbed OS, embedded Linux, whatever.)

The only case where you might have difficulties is if your embedded platform 
doesn’t have a filesystem. In that case I believe you’d need to use SQLite’s 
virtual-table API to map it to your persistent storage.

—Jens
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Porting SQLite: Stack error

[Richard Hipp]

Try commenting-out the three lines and
https://www.sqlite.org/src/artifact/e384cb73f?ln=77-79 and see if that
helps.  Let us know either way.

On 7/13/17, LukasOsswald  wrote:
> Hello,
>
> I am trying to port SQLite3 to an embedded system using an ARM926 running a
> real time os. The VFS is based on the example OneFile VFS created by the
> SQLite Developers.
>
>
> The following compiler flags are used:
> 'SQLITE_INT64_TYPE=long long',
> 'SQLITE_UINT64_TYPE=unsigned long long',
> 'SQLITE_TEMP_STORE=3',
> 'SQLITE_THREADSAFE=0',
> 'SQLITE_OS_OTHER=1',
> 'SQLITE_BYTEORDER=0',
> 'SQLITE_UNTESTABLE=1 ',
> 'SQLITE_DEFAULT_MEMSTATUS=1',
> 'SQLITE_OMIT_WAL=1',
> 'SQLITE_OMIT_DEPRECATED=1',
> 'SQLITE_OMIT_SHARED_CACHE=1 ',
> 'SQLITE_OMIT_DATETIME_FUNCS=1',
> 'SQLITE_OMIT_LOCALTIME=1',
> 'SQLITE_OMIT_LOAD_EXTENSION=1',
> 'SQLITE_OMIT_AUTHORIZATION=1 ',
> 'SQLITE_OMIT_MEMORYDB=1 ',
> 'SQLITE_OMIT_TEMPDB=1',
> 'NDEBUG',
> 'I_WANT_NO_DEPRECATED',
>
> After successfully opening the database, the following SQL Statement is
> executed.
>
>sqlite3_exec(db, "CREATE TABLE Person(id INTEGER, name VARCHAR(255))",
> NULL, 0, &zErrMsg);
>
> While debugging the code I run into an "end of frame" error caused by the
> yy_reduce() function of the Parser.(after some successful uses of this
> function). To be precise: The stack is destroyed after pushing data.
>
> - The maximum Stacksize is configured to 32 KByte. Is this enought?
> - Is there any further configuration needed to make it work?
> - Are the compile-time options compatible with each other ?
> - Are there any suggestion how this issue can be resolved ?
>
> Thank you in advance!
>
>
>
>
>
>
>
> --
> View this message in context:
> http://sqlite.1065341.n5.nabble.com/Porting-SQLite-Stack-error-tp96609.html
> Sent from the SQLite mailing list archive at Nabble.com.
> ___
> sqlite-users mailing list
> sqlite-users@mailinglists.sqlite.org
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>


-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Porting SQLite to another operating system (not supported out of the box)

[J Decker]

this contains a very basic vfs implementation; kinda uses internal file
abstraction stuff so a lot can be stripped out and replaced with
fopen/fread/etc...

https://code.google.com/p/c-system-abstraction-component-gui/source/browse/src/sqlite/sqlite_interface.c

maybe it's more cryptic than I think

On Thu, Feb 12, 2015 at 11:52 AM, Roger Binns  wrote:

> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> On 02/10/2015 07:03 PM, Jay Kreibich wrote:
> > ... VFS is unlikely to make the cut. ... similar things about the
> > xBestIndex() and xFilter() functions
>
> I haven't read the book, but one thing that may help is not using C
> for these.  I think it is easier to understand VFS & BestIndex etc
> using a higher level language, and use those to prototype, and then
> drop down to C once you understand the problem.
>
> As an example, my APSW wrapper (Python) lets you write a VFS
> "inheriting" from an existing one.  You only need to override the
> methods you care about, rather than having to implement everything
> from scratch.  This is how hard it is to do a vfs that xors the data
> read and written:
>
>   http://rogerbinns.github.io/apsw/example.html#example-vfs
>
> This is a vtable with blank BestIndex implementation:
>
>   http://rogerbinns.github.io/apsw/example.html#example-vtable
>
> I believe the wrappers for other languages are similar.  Of course for
> a book you'd have to pick a language/wrapper and hope it is acceptable
> for the audience.
>
> Roger
>
> -BEGIN PGP SIGNATURE-
> Version: GnuPG v1
>
> iEYEARECAAYFAlTdBHMACgkQmOOfHg372QRvYgCgg0Y1/Scvo+SGwOvPLkq3zBCF
> nb0Anj2XQydblsSTFd0szBZ1afuLto8q
> =mvfo
> -END PGP SIGNATURE-
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Porting SQLite to another operating system (not supported out of the box)

[Roger Binns]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 02/10/2015 07:03 PM, Jay Kreibich wrote:
> ... VFS is unlikely to make the cut. ... similar things about the
> xBestIndex() and xFilter() functions

I haven't read the book, but one thing that may help is not using C
for these.  I think it is easier to understand VFS & BestIndex etc
using a higher level language, and use those to prototype, and then
drop down to C once you understand the problem.

As an example, my APSW wrapper (Python) lets you write a VFS
"inheriting" from an existing one.  You only need to override the
methods you care about, rather than having to implement everything
from scratch.  This is how hard it is to do a vfs that xors the data
read and written:

  http://rogerbinns.github.io/apsw/example.html#example-vfs

This is a vtable with blank BestIndex implementation:

  http://rogerbinns.github.io/apsw/example.html#example-vtable

I believe the wrappers for other languages are similar.  Of course for
a book you'd have to pick a language/wrapper and hope it is acceptable
for the audience.

Roger

-BEGIN PGP SIGNATURE-
Version: GnuPG v1

iEYEARECAAYFAlTdBHMACgkQmOOfHg372QRvYgCgg0Y1/Scvo+SGwOvPLkq3zBCF
nb0Anj2XQydblsSTFd0szBZ1afuLto8q
=mvfo
-END PGP SIGNATURE-
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Porting SQLite to another operating system (not supported out of the box)

[Jay Kreibich]

On Feb 10, 2015, at 11:21 AM, Peter Aronson  wrote:

> You could add VFS creation if you ever do a revised edition (along with a 
> virtual table example that actually used xBestIndex and xFilter…)

Given that the book is over four years old and covers to the end of SQLIte3 
3.6.x, there are a lot of things that would need to go into a revised edition… 
including a lot more examples of everything, according to reviews.  We greatly 
underestimated the number of SQLite developers that were touching SQL for the 
first time, and I would have never guessed people would have considered yet 
another SQL lesson to be so important, given that there are a million books and 
a bazillion websites on learning SQL basics.  You can literally find books on 
“SQL For Dummies” (Allen Taylor) to “SQL For Smarties” (Joe Celko), and 
everything in-between.  That last book (or books, actually) is awesome, BTW, 
and the “Advanced SQL Programming” one should be on the shelf of every database 
programmer doing anything more advanced than an address book.

Regardless, if we do a second edition (and at this point that’s an extremely, 
exceptionally big “if”), VFS is unlikely to make the cut.  Consider that out of 
the thousands of SQLite applications and billions of installed databases, there 
are likely less than 100 production VFS modules in the whole world.  Spending a 
lot of time and pages, driving up the cost of the book, covering an extremely 
advanced and obscure topic is a poor trade-off (every page averages about a day 
to write/edit/prep, and adds about $0.10 to the price of the book).  If you 
need that level of integration and detail, working in the guts of the I/O and 
locking system, you should likely hand the task to a systems engineer that is 
familiar with the problem domain and isn’t afraid of looking through a few 
headers and examples to figure it all out.  It’s advanced, custom stuff that 
is, almost by definition, not textbook work.  It is the kind of work that 
requires digging through nitty-gritty code, documentation, and examples from 
both SQLite and your environment.  This is the kind of thing that’s learned 
from years of experience, not by reading it in a book.

That isn’t meant to be a criticism of the original poster— there is a huge 
difference between asking if anyone knows where to start looking, and asking 
for detailed step-by-step instructions.  In fact, if we did decide to put some 
information about VFS modules in a book, it would likely be a discussion of how 
the structures and APIs fit together, what they’re used for, and the types of 
things that can be done with them— exactly the kind of info you need to get 
started, but not much beyond that.  After all, what goes in those functions is 
going to be extremely dependent on the environment the VFS is trying to use.

I might say similar things about the xBestIndex() and xFilter() functions.  
While the APIs and how they are used is a tad confusing, their purpose and 
function should be reasonably straight forward to someone comfortable with 
relational data management and design.  While the book attempts to cover how 
the APIs are meant to perform their tasks (and has a six page discussion on 
their purpose and use), actually writing such a function is extremely dependent 
on understanding the virtual table being design— and the data in it. I feel it 
is something that just needs to be done by a skilled engineer, with a lot of 
detailed knowledge about the problem that’s trying to be solved.  Again, there 
aren’t any real textbook examples here; yes, I could write a contrived example, 
but if they didn’t understand from a general description, a single specific 
example is unlikely to help anyone in their specific case.  At the end of the 
day, both functions are an optimizations anyways.  You can write a functional 
virtual table without them, it might just run a tad slower.  If you really need 
that last bit of performance, you need an engineering that knows how to get 
detailed work done, and can verify the correctness of the system when they’re 
done— even if it involves a little of banging heads against walls, and cursing 
at the debugger.  That’s where the magic happens anyways.


 -j


--  
Jay A. Kreibich < J A Y @ K R E I B I.C H >

"Intelligence is like underwear: it is important that you have it, but showing 
it to the wrong people has the tendency to make them feel uncomfortable." -- 
Angela Johnson





___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Porting SQLite to another operating system (not supported out of the box)

[Dan Kennedy]

On 02/11/2015 12:31 AM, Simon Slavin wrote:

On 10 Feb 2015, at 5:01pm, Clemens Ladisch  wrote:


Janke, Julian wrote:

In my opinion, this means, we must ""only"" write a VFS implementation for
our target platform.

What file API is there?

It looks like a standard POXIS implementation.  My guess is that the best place 
to start would be



including the examples linked in section 2.3.  The OP should come back if he 
has more specific questions after that.


A simple VFS for systems that support a basic set of posix primitives is 
here:


  http://www.sqlite.org/src/artifact/69b2085076654

The code is quite easy to follow too.

Dan.







Simon.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users


___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Porting SQLite to another operating system (not supported out of the box)

[Richard Hipp]

On 2/10/15, Janke, Julian  wrote:
> >
> I know there are two implementations for unix and windows systems in
> os_unix.c and os_win.c,
> but these are very large files, which are not very helpful for a first
> rough
> understanding.
>

See also: https://www.sqlite.org/src/artifact/0396f220561f
-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Porting SQLite to another operating system (not supported out of the box)

[Simon Slavin]

On 10 Feb 2015, at 5:01pm, Clemens Ladisch  wrote:

> Janke, Julian wrote:
>> In my opinion, this means, we must ""only"" write a VFS implementation for
>> our target platform.
> 
> What file API is there?

It looks like a standard POXIS implementation.  My guess is that the best place 
to start would be



including the examples linked in section 2.3.  The OP should come back if he 
has more specific questions after that.

Simon.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Porting SQLite to another operating system (not supported out of the box)

[Peter Aronson]

You could add VFS creation if you ever do a revised edition (along with a 
virtual table example that actually used xBestIndex and xFilter...).


On Tuesday, February 10, 2015 9:58 AM, Jay Kreibich  wrote:
 

>
>
>
>
>No, it does not.  Using SQLite covers Virtual Tables in great detail, but not 
>VFS systems.  They’re somewhat unusual, after all.
>
>I assume you’ve read the VFS docs: http://www.sqlite.org/vfs.html
>
>From there, my suggestion is to look closely at the code for the VFS structure 
>(http://www.sqlite.org/c3ref/vfs.html) and the IO Methods structure 
>(http://www.sqlite.org/c3ref/io_methods.html).  Both those structs contain a 
>bunch of function pointers for which you need to provide code.
>
>-j
>
>
>
>On Feb 10, 2015, at 10:31 AM, Stephan Beal  wrote:
>
>> On Tue, Feb 10, 2015 at 5:27 PM, Janke, Julian 
>> wrote:
>> 
>>> So my question is, if there is any additional information,  how to write a
>>> VFS?
>>> Or does anyone have a clue how to start best?
>>> 
>> 
>> This book:
>> 
>> http://www.amazon.de/Using-SQLite-Jay-Kreibich-ebook/dp/B008IGK5QM/
>> 
>> resp.
>> 
>> http://shop.oreilly.com/product/9780596521196.do
>> 
>> covers VFS creation in detail with a step-by-step example.
>> 
>> -- 
>> - stephan beal
>> http://wanderinghorse.net/home/stephan/
>> http://gplus.to/sgbeal
>> "Freedom is sloppy. But since tyranny's the only guaranteed byproduct of
>> those who insist on a perfect world, freedom will have to do." -- Bigby Wolf
>> ___
>> sqlite-users mailing list
>> sqlite-users@sqlite.org
>> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>
>--  
>Jay A. Kreibich < J A Y @ K R E I B I.C H >
>
>"Intelligence is like underwear: it is important that you have it, but showing 
>it to the wrong people has the tendency to make them feel uncomfortable." -- 
>Angela Johnson
>
>
>
>
>
>
>___
>sqlite-users mailing list
>sqlite-users@sqlite.org
>http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>
>
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Porting SQLite to another operating system (not supported out of the box)

[Clemens Ladisch]

Janke, Julian wrote:
> In my opinion, this means, we must ""only"" write a VFS implementation for
> our target platform.

What file API is there?


Regards,
Clemens
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Porting SQLite to another operating system (not supported out of the box)

[Stephan Beal]

On Tue, Feb 10, 2015 at 5:58 PM, Jay Kreibich  wrote:

> No, it does not.  Using SQLite covers Virtual Tables in great detail, but
> not VFS systems.  They’re somewhat unusual, after all.
>

My apologies - i mixed my terminology there!

-- 
- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
"Freedom is sloppy. But since tyranny's the only guaranteed byproduct of
those who insist on a perfect world, freedom will have to do." -- Bigby Wolf
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Porting SQLite to another operating system (not supported out of the box)

[Jay Kreibich]

No, it does not.  Using SQLite covers Virtual Tables in great detail, but not 
VFS systems.  They’re somewhat unusual, after all.

I assume you’ve read the VFS docs: http://www.sqlite.org/vfs.html

>From there, my suggestion is to look closely at the code for the VFS structure 
>(http://www.sqlite.org/c3ref/vfs.html) and the IO Methods structure 
>(http://www.sqlite.org/c3ref/io_methods.html).  Both those structs contain a 
>bunch of function pointers for which you need to provide code.

 -j



On Feb 10, 2015, at 10:31 AM, Stephan Beal  wrote:

> On Tue, Feb 10, 2015 at 5:27 PM, Janke, Julian 
> wrote:
> 
>> So my question is, if there is any additional information,  how to write a
>> VFS?
>> Or does anyone have a clue how to start best?
>> 
> 
> This book:
> 
> http://www.amazon.de/Using-SQLite-Jay-Kreibich-ebook/dp/B008IGK5QM/
> 
> resp.
> 
> http://shop.oreilly.com/product/9780596521196.do
> 
> covers VFS creation in detail with a step-by-step example.
> 
> -- 
> - stephan beal
> http://wanderinghorse.net/home/stephan/
> http://gplus.to/sgbeal
> "Freedom is sloppy. But since tyranny's the only guaranteed byproduct of
> those who insist on a perfect world, freedom will have to do." -- Bigby Wolf
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

--  
Jay A. Kreibich < J A Y @ K R E I B I.C H >

"Intelligence is like underwear: it is important that you have it, but showing 
it to the wrong people has the tendency to make them feel uncomfortable." -- 
Angela Johnson





___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Porting SQLite to another operating system (not supported out of the box)

[Stephan Beal]

On Tue, Feb 10, 2015 at 5:27 PM, Janke, Julian 
wrote:

> So my question is, if there is any additional information,  how to write a
> VFS?
> Or does anyone have a clue how to start best?
>

This book:

http://www.amazon.de/Using-SQLite-Jay-Kreibich-ebook/dp/B008IGK5QM/

resp.

http://shop.oreilly.com/product/9780596521196.do

covers VFS creation in detail with a step-by-step example.

-- 
- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
"Freedom is sloppy. But since tyranny's the only guaranteed byproduct of
those who insist on a perfect world, freedom will have to do." -- Bigby Wolf
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Porting SQLite to Nurit OS ver

[Stephan Beal]

On Wed, Jul 2, 2014 at 2:41 PM, Clemens Ladisch  wrote:

> > |Prototype causes non-standard conversion from `int' to
> `sqlite3_int64'
>
> This is just an overzealous and absurdly useless warning.
> Try to reduce the warning level.
>

AFAIK, any conversion from a less precise type to a more precise type is
does not constitute a warning. Only when precision "could" be lost
(int64==>int, or signedness differences) is a warning in order. Maybe
sqlite3_int64 on that platform is of a type smaller than int, or is
unsigned? That'd be weird, but i've seen weirder (8-byte booleans!).

> warning 572: Potentially dangerous pointer cast:
> > |sizeof `struct FuncDef' ("../sqlite/sqlite3.c",L8668/C24) differs
> from
> > |sizeof `struct FuncDef[8]' and
> > |accessing an lvalue through this pointer alias may violate
> assumptions
> > |in the ANSI C Language Reference used by the optimizer.
> > |(See (X3.159-1989 p.39,L18) on object access for more details.)
>
> I'm quite sure that this interpretation of the standard is plain wrong,
>

+1, and unless i'm misunderstanding the problem (haven't got the code on
this machine and am too lazy to google it up), it's simple to deduce that
this is a compiler bug: in C, for any array of type X, any element of that
array is guaranteed to be exactly sizeof(X). C guarantees this, or
conventional array traversal (via ptr++ resp. ++ptr) could not work.

-- 
- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
"Freedom is sloppy. But since tyranny's the only guaranteed byproduct of
those who insist on a perfect world, freedom will have to do." -- Bigby Wolf
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Porting SQLite to Nurit OS ver

[Clemens Ladisch]

Paulo Roberto wrote:
> I'm trying to port SQLite (3.08.04.03) to the Nurit 8400 POS device that
> has an ARM 7 processor and is running Nurit OS 7.50.13a
>
> The error appears when the compiler is building the sqlite.c file (the same
> flags are used).

> |Prototype causes non-standard conversion from `int' to `sqlite3_int64'

This is just an overzealous and absurdly useless warning.
Try to reduce the warning level.

> warning 572: Potentially dangerous pointer cast:
> |sizeof `struct FuncDef' ("../sqlite/sqlite3.c",L8668/C24) differs from
> |sizeof `struct FuncDef[8]' and
> |accessing an lvalue through this pointer alias may violate assumptions
> |in the ANSI C Language Reference used by the optimizer.
> |(See (X3.159-1989 p.39,L18) on object access for more details.)

I'm quite sure that this interpretation of the standard is plain wrong,
but I don't have that particular edition.  (If I were suspicious, I'd
assume that this is why they specified the page number instead of the
section.)

> [color] Max attempts exhausted at 2 with 12 regs in contention. All registers 
> spilled!!
> ...

Anyway, that compiler is just buggy.  You could try to reduce or disable
optimizations, but I wouldn't trust the compiler output anyway.

If possible, use gcc.


Regards,
Clemens
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Porting SQLite to Nurit OS ver

[Kees Nuyt]

On Wed, 2 Jul 2014 12:02:01 +0100, Simon Slavin 
wrote:

>
> Thank you for your precise and useful description of what is happening.
>
> Unfortunately this list does not allow attachments, but can you paste
> a copy of the final error, the one about the register map,
> to one of your posts ?  Make sure we can see both the full
> error text and which line of code it's complaining about.

The error report was included, you just have to scroll down a bit more.

-- 
Groet, Cordialement, Pozdrawiam, Regards,

Kees Nuyt

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Porting SQLite to Nurit OS ver

[Simon Slavin]

On 1 Jul 2014, at 10:32pm, Paulo Roberto  wrote:

> There are a lot of warnings due to data type conversion and in the end an
> error relative to register map that I didn't understand.
> 
> I would appreciate any help to solve this issue and build it correctly.
> 
> The compiler used is an hcarm 4.5a.
> The building process output is below in italic and the error output in bold.

Thank you for your precise and useful description of what is happening.

Unfortunately this list does not allow attachments, but can you paste a copy of 
the final error, the one about the register map, to one of your posts ?  Make 
sure we can see both the full error text and which line of code it's 
complaining about.

The warnings about type conversion you can ignore.  Because SQLite's source 
code has to be suitable for many different compilers, it's possible to get rid 
of warnings from every compiler that may be used and they each enforce type 
conversion differently.  But we treat errors as more serious than warnings and 
may be able to figure out what's causing your register map error.

Simon.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Porting SQLite to plain C RTOS

[mm.w]

http://pubs.opengroup.org/onlinepubs/7908799/xsh/systime.h.html

http://pubs.opengroup.org/onlinepubs/7908799/xsh/flockfile.html

http://pubs.opengroup.org/onlinepubs/009695399/functions/fcntl.html

// IEEE Std 1003.1, 2004 Edition @!!= linux



On Tue, May 13, 2014 at 7:55 AM, Artur Deterer wrote:

> Hello SQLite Developers,
>
>
>
> my company would like to use sqlite in our embedded systems for which we
> use
> a plain C proprietary operating system.
>
> Memory allocation and deallocation within this system is realized through
> standard malloc/free functions. At this point of
>
> system development we are trying to adapt SQLite to our platform and  face
> some difficulties:
>
>
>
> Before I tell you which errors occur I explain to you the work steps I
> walked through.
>
> 1)As far as I know I have to tell sqlite that it should not
> work
> on the implemented operating systems but on a
>
> custom platform and this is to be done setting the macro
> SQL_OTHER_OS to 1. (in my code, line 30: SQL_OTHER_OS=1;)
>
> I did this within the first lines of sqlite3.c file. and at
> line 9870 I changed the value of SQL_OTHER_OS to 1 too.
>
> 2)With our single-threaded task scheduler we also have to tell
> sqlite to not allow multithreading operations
>
> as I did changing the value of SQL_THREADSAFE to 0 in line
> 7081.
>
> 3)On your web pages you wrote that a user has to implement the
> functions sqlite3_os_init() and sqlite3_os_end()
>
> in case that he wants sqlite to run on a custom system but
> I
> do not know which implementation is needed or whether
>
> an own implementation is needed because our os memory
> operations are based on malloc and free.
>
> 4)As I read your manual to port SQLite to a new operating
> system, I realized that an own implementation of
>
> sqlite3_vfs is necessary. I took
>  test_demovfs.c as
> skeleton for a ownOSVFS implementation. Within our os
>
> we have an own implementation of FAT32 file system
> functions
> which I used for substituting the file operation
>
> functions offered in test_demovfs.
>
> 5)I set the function sqlite3_initialize() within sqlite3.c in
> comments to implement this function within uTaskerVFS.c.
>
> This file also includes the sqlite3 header file and is
> appended to this email.
>
>
>
> As I walked through these steps I am not confident whether my
> implementation
> is correct or misses any point (e.g. sqlite3_os_init()).
>
> The main reason I write to you is that our cross-compiler (ARM GCC) for an
> ARM Cortex M4 system throws compiler errors
>
> according to sqlite. They tell me that references to several c structures
> (e.g. timeval, flock) are missing. These are used within a
>
> code part that asks whether the current platform is Linux (if macro: line
> 23528 ) what obviously is not correct.
>
> I think that if the macro SQL_OTHER_OS set to 1 the code should not enable
> the code part  designed for linux os.
>
>
>
> Have you an idea of solving the porting difficulties I described in this
> email ?
>
> It would be great ,too, if you could tell me whether I am wrong with my
> implementation.
>
> Can you please give me a more detailed instruction or maybe an example
>
> integration of sqlite into a custom os?
>
>
>
> Kind regards
>
> Deterer
>
>
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>
>
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Porting Sqlite to MQX Operating system

[JCharavda]

Geoff,
Did you manage to get sqlite working with MQX?
-- Jay




--
View this message in context: 
http://sqlite.1065341.n5.nabble.com/Porting-Sqlite-to-MQX-Operating-system-tp16685p69428.html
Sent from the SQLite mailing list archive at Nabble.com.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Porting Sqlite to MQX Operating system

[Richard Hipp]

On Thu, Apr 25, 2013 at 7:38 AM, arun.pradeep  wrote:

>
> Hi Geoff
>
>   I am currently working on porting of SQLITE to MQX RTOS in MPC5125
> Platform.
>
> I am trying to rewrite the sqlite_os_init() for MQX and I have taken Win32
> VFS implementation as the reference.
>

Have you considered using the (much simpler) test_demovfs.c VFS as a
reference implementation?



>
> But I am not able to map all of the Win32 file system calls with that of
> MQX
> functions.
>
> Please advise me on the list of file system functions that can be
> implemented as part of VFS for MQX.
>
>
>
> Regards,
> Arun
>
>
> GeoffW wrote:
> >
> > hello
> >
> > I thought I had better update this and confess to my sins. Good job no
> one
> > is reading this thread as it is an embarassingly stupid mistake :rules:
> >
> > I cant believe I did this and then took so long to spot it.
> >
> > int sqlite3_os_init(void)
> > {
> > static sqlite3_vfs mqxVfs = {
> > 1, /* iVersion */
> > 0,   /* szOsFile ->ARRRGGGH <*/
> > MAX_PATH,  /* mxPathname */
> > etc 
> >
> > I started creating the ported file from osWin.c, I had quite a struggle
> > getting it to compile initially using CodeWarrior, so I was making
> several
> > temporary hacks to allow it to compile, which I then revisited later to
> > correct. I had replaced the sizeof(winFile) with a zero, and then forgot
> > to change it to sizeof(mqxFile).
> > This caused a fun crash down in the bowels of the paging code.
> >
> > Once I corrected this one liner it fixed the crash and sqlite is
> basically
> > up and running now. I still have a fair bit of work left to implement
> some
> > interface functions which are just dummy stubs at present, but I am
> making
> > progress now.
> >
> > Regards Geoff
> >
> >
> >
> >
> >
>
> --
> View this message in context:
> http://old.nabble.com/Porting-Sqlite-to-MQX-Operating-system-tp27792715p35333159.html
> Sent from the SQLite mailing list archive at Nabble.com.
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>



-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Porting Sqlite to MQX Operating system

[arun.pradeep]

Hi Geoff

  I am currently working on porting of SQLITE to MQX RTOS in MPC5125
Platform. 

I am trying to rewrite the sqlite_os_init() for MQX and I have taken Win32
VFS implementation as the reference.

But I am not able to map all of the Win32 file system calls with that of MQX
functions.

Please advise me on the list of file system functions that can be
implemented as part of VFS for MQX.



Regards,
Arun 


GeoffW wrote:
> 
> hello 
> 
> I thought I had better update this and confess to my sins. Good job no one
> is reading this thread as it is an embarassingly stupid mistake :rules:
> 
> I cant believe I did this and then took so long to spot it.
> 
> int sqlite3_os_init(void)
> {
> static sqlite3_vfs mqxVfs = {
> 1, /* iVersion */
> 0,   /* szOsFile ->ARRRGGGH <*/
> MAX_PATH,  /* mxPathname */
> etc 
> 
> I started creating the ported file from osWin.c, I had quite a struggle
> getting it to compile initially using CodeWarrior, so I was making several
> temporary hacks to allow it to compile, which I then revisited later to
> correct. I had replaced the sizeof(winFile) with a zero, and then forgot
> to change it to sizeof(mqxFile).
> This caused a fun crash down in the bowels of the paging code.
> 
> Once I corrected this one liner it fixed the crash and sqlite is basically
> up and running now. I still have a fair bit of work left to implement some
> interface functions which are just dummy stubs at present, but I am making
> progress now.
> 
> Regards Geoff
> 
> 
> 
> 
> 

-- 
View this message in context: 
http://old.nabble.com/Porting-Sqlite-to-MQX-Operating-system-tp27792715p35333161.html
Sent from the SQLite mailing list archive at Nabble.com.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Porting Sqlite to MQX Operating system

[arun.pradeep]

Hi Geoff

  I am currently working on porting of SQLITE to MQX RTOS in MPC5125
Platform. 

I am trying to rewrite the sqlite_os_init() for MQX and I have taken Win32
VFS implementation as the reference.

But I am not able to map all of the Win32 file system calls with that of MQX
functions.

Please advise me on the list of file system functions that can be
implemented as part of VFS for MQX.



Regards,
Arun 


GeoffW wrote:
> 
> hello 
> 
> I thought I had better update this and confess to my sins. Good job no one
> is reading this thread as it is an embarassingly stupid mistake :rules:
> 
> I cant believe I did this and then took so long to spot it.
> 
> int sqlite3_os_init(void)
> {
> static sqlite3_vfs mqxVfs = {
> 1, /* iVersion */
> 0,   /* szOsFile ->ARRRGGGH <*/
> MAX_PATH,  /* mxPathname */
> etc 
> 
> I started creating the ported file from osWin.c, I had quite a struggle
> getting it to compile initially using CodeWarrior, so I was making several
> temporary hacks to allow it to compile, which I then revisited later to
> correct. I had replaced the sizeof(winFile) with a zero, and then forgot
> to change it to sizeof(mqxFile).
> This caused a fun crash down in the bowels of the paging code.
> 
> Once I corrected this one liner it fixed the crash and sqlite is basically
> up and running now. I still have a fair bit of work left to implement some
> interface functions which are just dummy stubs at present, but I am making
> progress now.
> 
> Regards Geoff
> 
> 
> 
> 
> 

-- 
View this message in context: 
http://old.nabble.com/Porting-Sqlite-to-MQX-Operating-system-tp27792715p35333159.html
Sent from the SQLite mailing list archive at Nabble.com.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Porting a simple logon script to SQLite3 from MySQL

[David Garfield]

C Lindgren writes:
>   $sql=$db->exec("INSERT INTO users(ID,username,password)
>   VALUES 
> ('0','".$username."','".$password."')");

In MYSQL, null and 0 are magic for a INTEGER PRIMARY KEY
AUTO_INCREMENT column.

In SQLite, only null is magic for a INTEGER PRIMARY KEY column.

Use NULL instead of '0' in both platforms.

--David Garfield

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Porting a simple logon script to SQLite3 from MySQL

[C Lindgren]

Quoting Stephan Beal :


On Wed, Oct 5, 2011 at 6:56 PM, C Lindgren  wrote:


if (isset ($_post ['submit'] )) {



Aside from this use of POST being a huge security hole, $_post is spelled
wrong: it whould be $_POST (or $_REQUEST if you want to treat GET/POST the
same).



   $sql=$db->exec("INSERT INTO
users(ID,username,password)
   VALUES
('0','".$username."','".$**password."')");



If it's not clear why that is a huge security hole, google for "sql
injection attack" and then read up on PDO::prepare() for how to avoid that
problem:

http://php.net/manual/en/pdo.prepare.php

--
- stephan beal
http://wanderinghorse.net/home/stephan/
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users



Thanks...

So, I can eliminate the $_POST block of code and replace it with  
PDO::prepare() then execute it with PDOStatement::execute() ?


I'm sure I'll have additional questions after I rewrite that code block.
Even though this is in an intranet environment it's nice to know!

Thanks again for your straight answer.

--
C Lindgren


___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Porting a simple logon script to SQLite3 from MySQL

[Simon Slavin]

On 5 Oct 2011, at 5:56pm, C Lindgren wrote:

> I'm trying to port a simple logon script that was originally for MySQL to 
> SQLite3. 

MySQL uses connections to a server with a password.  SQLite accesses a file on 
a hard disk.  Make sure your web app (probably apache or httpd) has enough 
access to the database file to be able to open it with readwrite permissions.  
Do this by temporarily lowering the permissions all the way.

Simon.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Porting a simple logon script to SQLite3 from MySQL

[Stephan Beal]

On Wed, Oct 5, 2011 at 6:56 PM, C Lindgren  wrote:

> if (isset ($_post ['submit'] )) {


Aside from this use of POST being a huge security hole, $_post is spelled
wrong: it whould be $_POST (or $_REQUEST if you want to treat GET/POST the
same).


>$sql=$db->exec("INSERT INTO
> users(ID,username,password)
>VALUES
> ('0','".$username."','".$**password."')");
>

If it's not clear why that is a huge security hole, google for "sql
injection attack" and then read up on PDO::prepare() for how to avoid that
problem:

http://php.net/manual/en/pdo.prepare.php

-- 
- stephan beal
http://wanderinghorse.net/home/stephan/
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Porting Sqlite to MQX OS: Question 3

[thedigitalsean]

Geoff,

I know this is an old post, but I am investigating designs for a project
that might benefit from Sqlite support on MQX.  In your initial post you
mentioned posting your results.  Were you ever able to get this working?

Thanks,

Sean
-- 
View this message in context: 
http://old.nabble.com/Porting-Sqlite-to-MQX-OS%3A-Question-3-tp27985475p31090638.html
Sent from the SQLite mailing list archive at Nabble.com.

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Porting SQLite 3.7.2 to vxWorks 6.7

[Scott A Mintz]

Thanks.  I took a guess that it was fixed in 3.7.3 and manually downloaded 
the source.  Of course hitting refresh in my browser brought up the 
correct page.  PEBKAC.

-Scott

sqlite-users-boun...@sqlite.org wrote on 10/11/2010 04:04:32 PM:

> See:
> 
> http://www.sqlite.org/src/info/d1ed743b6e
> 
> for changes for SQLITE_OMIT_WAL.
> 
> 3.7.3 should be available from the downloads page:
> 
> http://www.sqlite.org/download.html
> 
> as of October 8, 2010.  You might need to refresh your browser cache.
> 
> Direct link to the amalgamation is:
> 
> http://www.sqlite.org/sqlite-amalgamation-3_7_3.zip
> 
> HTH.
> -Shane
> 
> On Mon, Oct 11, 2010 at 12:11 PM, Scott A Mintz  rockwell.com> wrote:
> > I think there is a bug in 3.7.2 when you define SQLITE_OMIT_WAL. 
 There
> > are references to pagerPagecount() that cannot be resolved because 
that
> > function is defined inside of a #ifndef SQLITE_OMIT_WAL.
> >
> > I also keep seeing references to 3.7.3 in this list.  But the SQLite
> > download page only has links for 3.7.2.  Where can I download 3.7.3?
> >
> > -Scott
> >
> > sqlite-users-boun...@sqlite.org wrote on 10/09/2010 07:38:24 AM:
> >
> >> Hello Scott,
> >>
> >> I did some searching with these symbols, they should be in
> > 'libos.a'.
> >>
> >> rtpLib.o - rtpVerifyAndLock
> >>
> >> pgMgrLib.o - pgMgrPageFree, pgMgrPageAllocAt
> >>
> >> you should modify kernel components with your VIP project.
> >>
> >> Best Regards,
> >>
> >> Huang ZhiHua
> >>
> >> 2010/10/9 Scott A Mintz 
> >>
> >> > Thank you.  Those changes (modified slightly for 3.7.2) allowed me 
to
> >> > create a DKM project that compiles sqlite3.c to libSQLite3.a.
> >> >
> >> > However, when I link my main VIP project, I get the following
> > unresolved
> >> > errors:
> >> > dld: warning: Undefined symbol 'rtpVerifyAndLock' in file
> > 'partialImage.o'
> >> > dld: warning: Undefined symbol 'pgMgrPageFree' in file
> > 'partialImage.o'
> >> > dld: warning: Undefined symbol 'pgMgrPageAllocAt' in file
> > 'partialImage.o'
> >> >
> >> > Those are not directly used by SQLite.  But I have a feeling that 
one
> > or
> >> > more of the file I/O, semaphore, locking, or memory library system
> > calls
> >> > are...
> >> >
> >> > -Scott
> >> >
> >> > sqlite-users-boun...@sqlite.org wrote on 10/08/2010 07:38:45 AM:
> >> >
> >> > > Hello Scott,
> >> > >
> >> > > Below is my patch on the latest SQLite 3.7.3. Please notice 
that
> > I
> >> > only
> >> > > verify it with GCC 4.1.2 compiler in VxWorks 6.6/6.7/6.8(I have 
not
> >> > verify
> >> > > it with my real target machine yet).
> >> > >
> >> > > *** sqlite3.c.orig2010-10-08 10:42:22.0 +0800
> >> > > --- sqlite3.c2010-10-08 19:24:18.390625000 +0800
> >> > > ***
> >> > > *** 17,22 
> >> > > --- 17,26 
> >> > >   ** language. The code for the "sqlite3" command-line shell is 
also
> > in
> >> > a
> >> > >   ** separate file. This file contains only code for the core 
SQLite
> >> > > library.
> >> > >   */
> >> > > + #if defined(OS_VXWORKS)
> >> > > + #include 
> >> > > + #endif /* OS_VXWORKS */
> >> > > +
> >> > >   #define SQLITE_CORE 1
> >> > >   #define SQLITE_AMALGAMATION 1
> >> > >   #ifndef SQLITE_PRIVATE
> >> > > ***
> >> > > *** 22795,22801 
> >> > > --- 22799,22811 
> >> > >   #include 
> >> > >   #include 
> >> > >   #include 
> >> > > +
> >> > > + #if defined(OS_VXWORKS) && defined(_WRS_KERNEL)
> >> > > + #include 
> >> > > + #else
> >> > >   #include 
> >> > > + #endif /* OS_VXWORKS */
> >> > > +
> >> > >   #include 
> >> > >   #include 
> >> > >
> >> > > ***
> >> > > *** 24945,24951 
> >> > > --- 24955,24965 
> >> > >   /*
> >> > >** Close a file.
> >> > >*/
> >> > > + #if (OS_VXWORKS < 600)
> >> > >   static int semClose(sqlite3_file *id) {
> >> > > + #else
> >> > > + static int semClose_native(sqlite3_file *id) {
> >> > > + #endif
> >> > > if( id ){
> >> > >   unixFile *pFile = (unixFile*)id;
> >> > >   semUnlock(id, NO_LOCK);
> >> > > ***
> >> > > *** 25581,25587 
> >> > > --- 25595,25607 
> >> > >   }
> >> > >   return -1;
> >> > > }
> >> > > +
> >> > > + #if defined(OS_VXWORKS) && defined(_WRS_KERNEL)
> >> > > +   got = write(id->h, (char *)pBuf, cnt);
> >> > > + #else
> >> > > got = write(id->h, pBuf, cnt);
> >> > > + #endif /* OS_VXWORKS */
> >> > > +
> >> > >   #endif
> >> > > TIMER_END;
> >> > > if( got<0 ){
> >> > > ***
> >> > > *** 26762,26768 
> >> > > --- 26782,26792 
> >> > > semIoFinder,  /* Finder function name */
> >> > > semIoMethods, /* sqlite3_io_methods object name 
*/
> >> > > 1,/* shared memory is disabled */
> >> > > + #if (OS_VXWORKS < 600)
> >> > > semClose, /* xClose method */
> >> > > + #else
> >> > > +   semClose_native,  /* xClose method */
> >> > > + #endif
> >> > > semLock,  /* xLock method 

Re: [sqlite] Porting SQLite 3.7.2 to vxWorks 6.7

[Shane Harrelson]

See:

http://www.sqlite.org/src/info/d1ed743b6e

for changes for SQLITE_OMIT_WAL.

3.7.3 should be available from the downloads page:

http://www.sqlite.org/download.html

as of October 8, 2010.  You might need to refresh your browser cache.

Direct link to the amalgamation is:

http://www.sqlite.org/sqlite-amalgamation-3_7_3.zip

HTH.
-Shane

On Mon, Oct 11, 2010 at 12:11 PM, Scott A Mintz  wrote:
> I think there is a bug in 3.7.2 when you define SQLITE_OMIT_WAL.  There
> are references to pagerPagecount() that cannot be resolved because that
> function is defined inside of a #ifndef SQLITE_OMIT_WAL.
>
> I also keep seeing references to 3.7.3 in this list.  But the SQLite
> download page only has links for 3.7.2.  Where can I download 3.7.3?
>
> -Scott
>
> sqlite-users-boun...@sqlite.org wrote on 10/09/2010 07:38:24 AM:
>
>> Hello Scott,
>>
>>     I did some searching with these symbols, they should be in
> 'libos.a'.
>>
>> rtpLib.o - rtpVerifyAndLock
>>
>> pgMgrLib.o - pgMgrPageFree, pgMgrPageAllocAt
>>
>>     you should modify kernel components with your VIP project.
>>
>> Best Regards,
>>
>> Huang ZhiHua
>>
>> 2010/10/9 Scott A Mintz 
>>
>> > Thank you.  Those changes (modified slightly for 3.7.2) allowed me to
>> > create a DKM project that compiles sqlite3.c to libSQLite3.a.
>> >
>> > However, when I link my main VIP project, I get the following
> unresolved
>> > errors:
>> > dld: warning: Undefined symbol 'rtpVerifyAndLock' in file
> 'partialImage.o'
>> > dld: warning: Undefined symbol 'pgMgrPageFree' in file
> 'partialImage.o'
>> > dld: warning: Undefined symbol 'pgMgrPageAllocAt' in file
> 'partialImage.o'
>> >
>> > Those are not directly used by SQLite.  But I have a feeling that one
> or
>> > more of the file I/O, semaphore, locking, or memory library system
> calls
>> > are...
>> >
>> > -Scott
>> >
>> > sqlite-users-boun...@sqlite.org wrote on 10/08/2010 07:38:45 AM:
>> >
>> > > Hello Scott,
>> > >
>> > >     Below is my patch on the latest SQLite 3.7.3. Please notice that
> I
>> > only
>> > > verify it with GCC 4.1.2 compiler in VxWorks 6.6/6.7/6.8(I have not
>> > verify
>> > > it with my real target machine yet).
>> > >
>> > > *** sqlite3.c.orig    2010-10-08 10:42:22.0 +0800
>> > > --- sqlite3.c    2010-10-08 19:24:18.390625000 +0800
>> > > ***
>> > > *** 17,22 
>> > > --- 17,26 
>> > >   ** language. The code for the "sqlite3" command-line shell is also
> in
>> > a
>> > >   ** separate file. This file contains only code for the core SQLite
>> > > library.
>> > >   */
>> > > + #if defined(OS_VXWORKS)
>> > > + #include 
>> > > + #endif /* OS_VXWORKS */
>> > > +
>> > >   #define SQLITE_CORE 1
>> > >   #define SQLITE_AMALGAMATION 1
>> > >   #ifndef SQLITE_PRIVATE
>> > > ***
>> > > *** 22795,22801 
>> > > --- 22799,22811 
>> > >   #include 
>> > >   #include 
>> > >   #include 
>> > > +
>> > > + #if defined(OS_VXWORKS) && defined(_WRS_KERNEL)
>> > > + #include 
>> > > + #else
>> > >   #include 
>> > > + #endif /* OS_VXWORKS */
>> > > +
>> > >   #include 
>> > >   #include 
>> > >
>> > > ***
>> > > *** 24945,24951 
>> > > --- 24955,24965 
>> > >   /*
>> > >    ** Close a file.
>> > >    */
>> > > + #if (OS_VXWORKS < 600)
>> > >   static int semClose(sqlite3_file *id) {
>> > > + #else
>> > > + static int semClose_native(sqlite3_file *id) {
>> > > + #endif
>> > >     if( id ){
>> > >       unixFile *pFile = (unixFile*)id;
>> > >       semUnlock(id, NO_LOCK);
>> > > ***
>> > > *** 25581,25587 
>> > > --- 25595,25607 
>> > >       }
>> > >       return -1;
>> > >     }
>> > > +
>> > > + #if defined(OS_VXWORKS) && defined(_WRS_KERNEL)
>> > > +   got = write(id->h, (char *)pBuf, cnt);
>> > > + #else
>> > >     got = write(id->h, pBuf, cnt);
>> > > + #endif /* OS_VXWORKS */
>> > > +
>> > >   #endif
>> > >     TIMER_END;
>> > >     if( got<0 ){
>> > > ***
>> > > *** 26762,26768 
>> > > --- 26782,26792 
>> > >     semIoFinder,              /* Finder function name */
>> > >     semIoMethods,             /* sqlite3_io_methods object name */
>> > >     1,                        /* shared memory is disabled */
>> > > + #if (OS_VXWORKS < 600)
>> > >     semClose,                 /* xClose method */
>> > > + #else
>> > > +   semClose_native,          /* xClose method */
>> > > + #endif
>> > >     semLock,                  /* xLock method */
>> > >     semUnlock,                /* xUnlock method */
>> > >     semCheckReservedLock      /* xCheckReservedLock method */
>> > > ***
>> > > *** 27517,27523 
>> > >     noLock = eType!=SQLITE_OPEN_MAIN_DB;
>> > >
>> > >
>> > > ! #if defined(__APPLE__) || SQLITE_ENABLE_LOCKING_STYLE
>> > >     struct statfs fsInfo;
>> > >     if( fstatfs(fd, &fsInfo) == -1 ){
>> > >       ((unixFile*)pFile)->lastErrno = errno;
>> > > --- 27541,27547 
>> > >     noLock = eType!=SQLITE_OPEN_MAIN_DB;
>> > >
>> > >
>> > > ! #if (defined(__APPLE__) || SQLITE_ENABLE_LO

Re: [sqlite] Porting SQLite 3.7.2 to vxWorks 6.7

[Scott A Mintz]

I think there is a bug in 3.7.2 when you define SQLITE_OMIT_WAL.  There 
are references to pagerPagecount() that cannot be resolved because that 
function is defined inside of a #ifndef SQLITE_OMIT_WAL.

I also keep seeing references to 3.7.3 in this list.  But the SQLite 
download page only has links for 3.7.2.  Where can I download 3.7.3?

-Scott

sqlite-users-boun...@sqlite.org wrote on 10/09/2010 07:38:24 AM:

> Hello Scott,
> 
> I did some searching with these symbols, they should be in 
'libos.a'.
> 
> rtpLib.o - rtpVerifyAndLock
> 
> pgMgrLib.o - pgMgrPageFree, pgMgrPageAllocAt
> 
> you should modify kernel components with your VIP project.
> 
> Best Regards,
> 
> Huang ZhiHua
> 
> 2010/10/9 Scott A Mintz 
> 
> > Thank you.  Those changes (modified slightly for 3.7.2) allowed me to
> > create a DKM project that compiles sqlite3.c to libSQLite3.a.
> >
> > However, when I link my main VIP project, I get the following 
unresolved
> > errors:
> > dld: warning: Undefined symbol 'rtpVerifyAndLock' in file 
'partialImage.o'
> > dld: warning: Undefined symbol 'pgMgrPageFree' in file 
'partialImage.o'
> > dld: warning: Undefined symbol 'pgMgrPageAllocAt' in file 
'partialImage.o'
> >
> > Those are not directly used by SQLite.  But I have a feeling that one 
or
> > more of the file I/O, semaphore, locking, or memory library system 
calls
> > are...
> >
> > -Scott
> >
> > sqlite-users-boun...@sqlite.org wrote on 10/08/2010 07:38:45 AM:
> >
> > > Hello Scott,
> > >
> > > Below is my patch on the latest SQLite 3.7.3. Please notice that 
I
> > only
> > > verify it with GCC 4.1.2 compiler in VxWorks 6.6/6.7/6.8(I have not
> > verify
> > > it with my real target machine yet).
> > >
> > > *** sqlite3.c.orig2010-10-08 10:42:22.0 +0800
> > > --- sqlite3.c2010-10-08 19:24:18.390625000 +0800
> > > ***
> > > *** 17,22 
> > > --- 17,26 
> > >   ** language. The code for the "sqlite3" command-line shell is also 
in
> > a
> > >   ** separate file. This file contains only code for the core SQLite
> > > library.
> > >   */
> > > + #if defined(OS_VXWORKS)
> > > + #include 
> > > + #endif /* OS_VXWORKS */
> > > +
> > >   #define SQLITE_CORE 1
> > >   #define SQLITE_AMALGAMATION 1
> > >   #ifndef SQLITE_PRIVATE
> > > ***
> > > *** 22795,22801 
> > > --- 22799,22811 
> > >   #include 
> > >   #include 
> > >   #include 
> > > +
> > > + #if defined(OS_VXWORKS) && defined(_WRS_KERNEL)
> > > + #include 
> > > + #else
> > >   #include 
> > > + #endif /* OS_VXWORKS */
> > > +
> > >   #include 
> > >   #include 
> > >
> > > ***
> > > *** 24945,24951 
> > > --- 24955,24965 
> > >   /*
> > >** Close a file.
> > >*/
> > > + #if (OS_VXWORKS < 600)
> > >   static int semClose(sqlite3_file *id) {
> > > + #else
> > > + static int semClose_native(sqlite3_file *id) {
> > > + #endif
> > > if( id ){
> > >   unixFile *pFile = (unixFile*)id;
> > >   semUnlock(id, NO_LOCK);
> > > ***
> > > *** 25581,25587 
> > > --- 25595,25607 
> > >   }
> > >   return -1;
> > > }
> > > +
> > > + #if defined(OS_VXWORKS) && defined(_WRS_KERNEL)
> > > +   got = write(id->h, (char *)pBuf, cnt);
> > > + #else
> > > got = write(id->h, pBuf, cnt);
> > > + #endif /* OS_VXWORKS */
> > > +
> > >   #endif
> > > TIMER_END;
> > > if( got<0 ){
> > > ***
> > > *** 26762,26768 
> > > --- 26782,26792 
> > > semIoFinder,  /* Finder function name */
> > > semIoMethods, /* sqlite3_io_methods object name */
> > > 1,/* shared memory is disabled */
> > > + #if (OS_VXWORKS < 600)
> > > semClose, /* xClose method */
> > > + #else
> > > +   semClose_native,  /* xClose method */
> > > + #endif
> > > semLock,  /* xLock method */
> > > semUnlock,/* xUnlock method */
> > > semCheckReservedLock  /* xCheckReservedLock method */
> > > ***
> > > *** 27517,27523 
> > > noLock = eType!=SQLITE_OPEN_MAIN_DB;
> > >
> > >
> > > ! #if defined(__APPLE__) || SQLITE_ENABLE_LOCKING_STYLE
> > > struct statfs fsInfo;
> > > if( fstatfs(fd, &fsInfo) == -1 ){
> > >   ((unixFile*)pFile)->lastErrno = errno;
> > > --- 27541,27547 
> > > noLock = eType!=SQLITE_OPEN_MAIN_DB;
> > >
> > >
> > > ! #if (defined(__APPLE__) || SQLITE_ENABLE_LOCKING_STYLE) &&
> > > !defined(OS_VXWORKS)
> > > struct statfs fsInfo;
> > > if( fstatfs(fd, &fsInfo) == -1 ){
> > >   ((unixFile*)pFile)->lastErrno = errno;
> > > ***
> > > *** 27530,27536 
> > > }
> > >   #endif
> > >
> > > ! #if SQLITE_ENABLE_LOCKING_STYLE
> > >   #if SQLITE_PREFER_PROXY_LOCKING
> > > isAutoProxy = 1;
> > >   #endif
> > > --- 27554,27560 
> > > }
> > >   #endif
> > >
> > > ! #if SQLITE_ENABLE_LOCKING_STYLE && !OS_VXWORKS
> > >   #if SQLITE_PREFER_PROXY_LOCKING
> > > isAutoProxy

Re: [sqlite] Porting SQLite 3.7.2 to vxWorks 6.7

[Scott A Mintz]

I am a little reluctant to modify our kernel configuration.  Primarily out 
of ignorance because I know very little about vxWorks. 

Do you happen to know if these functions are used by the new WAL logic? 
I'm assuming by the names that they could be related to memory mapped 
files.  What switches would I set to disable WAL and use the old 
mechanism?

-Scott

sqlite-users-boun...@sqlite.org wrote on 10/09/2010 07:38:24 AM:

> Hello Scott,
> 
> I did some searching with these symbols, they should be in 
'libos.a'.
> 
> rtpLib.o - rtpVerifyAndLock
> 
> pgMgrLib.o - pgMgrPageFree, pgMgrPageAllocAt
> 
> you should modify kernel components with your VIP project.
> 
> Best Regards,
> 
> Huang ZhiHua
> 
> 2010/10/9 Scott A Mintz 
> 
> > Thank you.  Those changes (modified slightly for 3.7.2) allowed me to
> > create a DKM project that compiles sqlite3.c to libSQLite3.a.
> >
> > However, when I link my main VIP project, I get the following 
unresolved
> > errors:
> > dld: warning: Undefined symbol 'rtpVerifyAndLock' in file 
'partialImage.o'
> > dld: warning: Undefined symbol 'pgMgrPageFree' in file 
'partialImage.o'
> > dld: warning: Undefined symbol 'pgMgrPageAllocAt' in file 
'partialImage.o'
> >
> > Those are not directly used by SQLite.  But I have a feeling that one 
or
> > more of the file I/O, semaphore, locking, or memory library system 
calls
> > are...
> >
> > -Scott
> >
> > sqlite-users-boun...@sqlite.org wrote on 10/08/2010 07:38:45 AM:
> >
> > > Hello Scott,
> > >
> > > Below is my patch on the latest SQLite 3.7.3. Please notice that 
I
> > only
> > > verify it with GCC 4.1.2 compiler in VxWorks 6.6/6.7/6.8(I have not
> > verify
> > > it with my real target machine yet).
> > >
> > > *** sqlite3.c.orig2010-10-08 10:42:22.0 +0800
> > > --- sqlite3.c2010-10-08 19:24:18.390625000 +0800
> > > ***
> > > *** 17,22 
> > > --- 17,26 
> > >   ** language. The code for the "sqlite3" command-line shell is also 
in
> > a
> > >   ** separate file. This file contains only code for the core SQLite
> > > library.
> > >   */
> > > + #if defined(OS_VXWORKS)
> > > + #include 
> > > + #endif /* OS_VXWORKS */
> > > +
> > >   #define SQLITE_CORE 1
> > >   #define SQLITE_AMALGAMATION 1
> > >   #ifndef SQLITE_PRIVATE
> > > ***
> > > *** 22795,22801 
> > > --- 22799,22811 
> > >   #include 
> > >   #include 
> > >   #include 
> > > +
> > > + #if defined(OS_VXWORKS) && defined(_WRS_KERNEL)
> > > + #include 
> > > + #else
> > >   #include 
> > > + #endif /* OS_VXWORKS */
> > > +
> > >   #include 
> > >   #include 
> > >
> > > ***
> > > *** 24945,24951 
> > > --- 24955,24965 
> > >   /*
> > >** Close a file.
> > >*/
> > > + #if (OS_VXWORKS < 600)
> > >   static int semClose(sqlite3_file *id) {
> > > + #else
> > > + static int semClose_native(sqlite3_file *id) {
> > > + #endif
> > > if( id ){
> > >   unixFile *pFile = (unixFile*)id;
> > >   semUnlock(id, NO_LOCK);
> > > ***
> > > *** 25581,25587 
> > > --- 25595,25607 
> > >   }
> > >   return -1;
> > > }
> > > +
> > > + #if defined(OS_VXWORKS) && defined(_WRS_KERNEL)
> > > +   got = write(id->h, (char *)pBuf, cnt);
> > > + #else
> > > got = write(id->h, pBuf, cnt);
> > > + #endif /* OS_VXWORKS */
> > > +
> > >   #endif
> > > TIMER_END;
> > > if( got<0 ){
> > > ***
> > > *** 26762,26768 
> > > --- 26782,26792 
> > > semIoFinder,  /* Finder function name */
> > > semIoMethods, /* sqlite3_io_methods object name */
> > > 1,/* shared memory is disabled */
> > > + #if (OS_VXWORKS < 600)
> > > semClose, /* xClose method */
> > > + #else
> > > +   semClose_native,  /* xClose method */
> > > + #endif
> > > semLock,  /* xLock method */
> > > semUnlock,/* xUnlock method */
> > > semCheckReservedLock  /* xCheckReservedLock method */
> > > ***
> > > *** 27517,27523 
> > > noLock = eType!=SQLITE_OPEN_MAIN_DB;
> > >
> > >
> > > ! #if defined(__APPLE__) || SQLITE_ENABLE_LOCKING_STYLE
> > > struct statfs fsInfo;
> > > if( fstatfs(fd, &fsInfo) == -1 ){
> > >   ((unixFile*)pFile)->lastErrno = errno;
> > > --- 27541,27547 
> > > noLock = eType!=SQLITE_OPEN_MAIN_DB;
> > >
> > >
> > > ! #if (defined(__APPLE__) || SQLITE_ENABLE_LOCKING_STYLE) &&
> > > !defined(OS_VXWORKS)
> > > struct statfs fsInfo;
> > > if( fstatfs(fd, &fsInfo) == -1 ){
> > >   ((unixFile*)pFile)->lastErrno = errno;
> > > ***
> > > *** 27530,27536 
> > > }
> > >   #endif
> > >
> > > ! #if SQLITE_ENABLE_LOCKING_STYLE
> > >   #if SQLITE_PREFER_PROXY_LOCKING
> > > isAutoProxy = 1;
> > >   #endif
> > > --- 27554,27560 
> > > }
> > >   #endif
> > >
> > > ! #if SQLITE_ENABLE_LOCKING_STYLE && !OS_VXWORKS
> > >   #if SQLITE_PREFER_PROXY_LOCKING
> > > 

Re: [sqlite] Porting SQLite 3.7.2 to vxWorks 6.7

[ZhiHua Huang]

Hello Scott,

I did some searching with these symbols, they should be in 'libos.a'.

rtpLib.o - rtpVerifyAndLock

pgMgrLib.o - pgMgrPageFree, pgMgrPageAllocAt

you should modify kernel components with your VIP project.

Best Regards,

Huang ZhiHua

2010/10/9 Scott A Mintz 

> Thank you.  Those changes (modified slightly for 3.7.2) allowed me to
> create a DKM project that compiles sqlite3.c to libSQLite3.a.
>
> However, when I link my main VIP project, I get the following unresolved
> errors:
> dld: warning: Undefined symbol 'rtpVerifyAndLock' in file 'partialImage.o'
> dld: warning: Undefined symbol 'pgMgrPageFree' in file 'partialImage.o'
> dld: warning: Undefined symbol 'pgMgrPageAllocAt' in file 'partialImage.o'
>
> Those are not directly used by SQLite.  But I have a feeling that one or
> more of the file I/O, semaphore, locking, or memory library system calls
> are...
>
> -Scott
>
> sqlite-users-boun...@sqlite.org wrote on 10/08/2010 07:38:45 AM:
>
> > Hello Scott,
> >
> > Below is my patch on the latest SQLite 3.7.3. Please notice that I
> only
> > verify it with GCC 4.1.2 compiler in VxWorks 6.6/6.7/6.8(I have not
> verify
> > it with my real target machine yet).
> >
> > *** sqlite3.c.orig2010-10-08 10:42:22.0 +0800
> > --- sqlite3.c2010-10-08 19:24:18.390625000 +0800
> > ***
> > *** 17,22 
> > --- 17,26 
> >   ** language. The code for the "sqlite3" command-line shell is also in
> a
> >   ** separate file. This file contains only code for the core SQLite
> > library.
> >   */
> > + #if defined(OS_VXWORKS)
> > + #include 
> > + #endif /* OS_VXWORKS */
> > +
> >   #define SQLITE_CORE 1
> >   #define SQLITE_AMALGAMATION 1
> >   #ifndef SQLITE_PRIVATE
> > ***
> > *** 22795,22801 
> > --- 22799,22811 
> >   #include 
> >   #include 
> >   #include 
> > +
> > + #if defined(OS_VXWORKS) && defined(_WRS_KERNEL)
> > + #include 
> > + #else
> >   #include 
> > + #endif /* OS_VXWORKS */
> > +
> >   #include 
> >   #include 
> >
> > ***
> > *** 24945,24951 
> > --- 24955,24965 
> >   /*
> >** Close a file.
> >*/
> > + #if (OS_VXWORKS < 600)
> >   static int semClose(sqlite3_file *id) {
> > + #else
> > + static int semClose_native(sqlite3_file *id) {
> > + #endif
> > if( id ){
> >   unixFile *pFile = (unixFile*)id;
> >   semUnlock(id, NO_LOCK);
> > ***
> > *** 25581,25587 
> > --- 25595,25607 
> >   }
> >   return -1;
> > }
> > +
> > + #if defined(OS_VXWORKS) && defined(_WRS_KERNEL)
> > +   got = write(id->h, (char *)pBuf, cnt);
> > + #else
> > got = write(id->h, pBuf, cnt);
> > + #endif /* OS_VXWORKS */
> > +
> >   #endif
> > TIMER_END;
> > if( got<0 ){
> > ***
> > *** 26762,26768 
> > --- 26782,26792 
> > semIoFinder,  /* Finder function name */
> > semIoMethods, /* sqlite3_io_methods object name */
> > 1,/* shared memory is disabled */
> > + #if (OS_VXWORKS < 600)
> > semClose, /* xClose method */
> > + #else
> > +   semClose_native,  /* xClose method */
> > + #endif
> > semLock,  /* xLock method */
> > semUnlock,/* xUnlock method */
> > semCheckReservedLock  /* xCheckReservedLock method */
> > ***
> > *** 27517,27523 
> > noLock = eType!=SQLITE_OPEN_MAIN_DB;
> >
> >
> > ! #if defined(__APPLE__) || SQLITE_ENABLE_LOCKING_STYLE
> > struct statfs fsInfo;
> > if( fstatfs(fd, &fsInfo) == -1 ){
> >   ((unixFile*)pFile)->lastErrno = errno;
> > --- 27541,27547 
> > noLock = eType!=SQLITE_OPEN_MAIN_DB;
> >
> >
> > ! #if (defined(__APPLE__) || SQLITE_ENABLE_LOCKING_STYLE) &&
> > !defined(OS_VXWORKS)
> > struct statfs fsInfo;
> > if( fstatfs(fd, &fsInfo) == -1 ){
> >   ((unixFile*)pFile)->lastErrno = errno;
> > ***
> > *** 27530,27536 
> > }
> >   #endif
> >
> > ! #if SQLITE_ENABLE_LOCKING_STYLE
> >   #if SQLITE_PREFER_PROXY_LOCKING
> > isAutoProxy = 1;
> >   #endif
> > --- 27554,27560 
> > }
> >   #endif
> >
> > ! #if SQLITE_ENABLE_LOCKING_STYLE && !OS_VXWORKS
> >   #if SQLITE_PREFER_PROXY_LOCKING
> > isAutoProxy = 1;
> >   #endif
> > ***
> > *** 27793,27799 
> > ** tests repeatable.
> > */
> > memset(zBuf, 0, nBuf);
> > ! #if !defined(SQLITE_TEST)
> > {
> >   int pid, fd;
> >   fd = open("/dev/urandom", O_RDONLY);
> > --- 27817,27823 
> > ** tests repeatable.
> > */
> > memset(zBuf, 0, nBuf);
> > ! #if !defined(SQLITE_TEST) && !defined(OS_VXWORKS)
> > {
> >   int pid, fd;
> >   fd = open("/dev/urandom", O_RDONLY);
> >
> > I'v used definitions as below:
> >
> > EXTRA_DEFINE+=-DOS_VXWORKS_660=660
> > EXTRA_DEFINE+=-DOS_VXWORKS_670=670
> > EXTRA_DEFINE+=-DOS_VXWORKS_680=680
> > EXTRA_DEFINE+=-DOS_VXWORKS=OS_VXWORKS_670
> > EXTRA_

Re: [sqlite] Porting SQLite 3.7.2 to vxWorks 6.7

[Scott A Mintz]

Thank you.  Those changes (modified slightly for 3.7.2) allowed me to 
create a DKM project that compiles sqlite3.c to libSQLite3.a.

However, when I link my main VIP project, I get the following unresolved 
errors:
dld: warning: Undefined symbol 'rtpVerifyAndLock' in file 'partialImage.o'
dld: warning: Undefined symbol 'pgMgrPageFree' in file 'partialImage.o'
dld: warning: Undefined symbol 'pgMgrPageAllocAt' in file 'partialImage.o'

Those are not directly used by SQLite.  But I have a feeling that one or 
more of the file I/O, semaphore, locking, or memory library system calls 
are...

-Scott

sqlite-users-boun...@sqlite.org wrote on 10/08/2010 07:38:45 AM:

> Hello Scott,
> 
> Below is my patch on the latest SQLite 3.7.3. Please notice that I 
only
> verify it with GCC 4.1.2 compiler in VxWorks 6.6/6.7/6.8(I have not 
verify
> it with my real target machine yet).
> 
> *** sqlite3.c.orig2010-10-08 10:42:22.0 +0800
> --- sqlite3.c2010-10-08 19:24:18.390625000 +0800
> ***
> *** 17,22 
> --- 17,26 
>   ** language. The code for the "sqlite3" command-line shell is also in 
a
>   ** separate file. This file contains only code for the core SQLite
> library.
>   */
> + #if defined(OS_VXWORKS)
> + #include 
> + #endif /* OS_VXWORKS */
> +
>   #define SQLITE_CORE 1
>   #define SQLITE_AMALGAMATION 1
>   #ifndef SQLITE_PRIVATE
> ***
> *** 22795,22801 
> --- 22799,22811 
>   #include 
>   #include 
>   #include 
> +
> + #if defined(OS_VXWORKS) && defined(_WRS_KERNEL)
> + #include 
> + #else
>   #include 
> + #endif /* OS_VXWORKS */
> +
>   #include 
>   #include 
> 
> ***
> *** 24945,24951 
> --- 24955,24965 
>   /*
>** Close a file.
>*/
> + #if (OS_VXWORKS < 600)
>   static int semClose(sqlite3_file *id) {
> + #else
> + static int semClose_native(sqlite3_file *id) {
> + #endif
> if( id ){
>   unixFile *pFile = (unixFile*)id;
>   semUnlock(id, NO_LOCK);
> ***
> *** 25581,25587 
> --- 25595,25607 
>   }
>   return -1;
> }
> +
> + #if defined(OS_VXWORKS) && defined(_WRS_KERNEL)
> +   got = write(id->h, (char *)pBuf, cnt);
> + #else
> got = write(id->h, pBuf, cnt);
> + #endif /* OS_VXWORKS */
> +
>   #endif
> TIMER_END;
> if( got<0 ){
> ***
> *** 26762,26768 
> --- 26782,26792 
> semIoFinder,  /* Finder function name */
> semIoMethods, /* sqlite3_io_methods object name */
> 1,/* shared memory is disabled */
> + #if (OS_VXWORKS < 600)
> semClose, /* xClose method */
> + #else
> +   semClose_native,  /* xClose method */
> + #endif
> semLock,  /* xLock method */
> semUnlock,/* xUnlock method */
> semCheckReservedLock  /* xCheckReservedLock method */
> ***
> *** 27517,27523 
> noLock = eType!=SQLITE_OPEN_MAIN_DB;
> 
> 
> ! #if defined(__APPLE__) || SQLITE_ENABLE_LOCKING_STYLE
> struct statfs fsInfo;
> if( fstatfs(fd, &fsInfo) == -1 ){
>   ((unixFile*)pFile)->lastErrno = errno;
> --- 27541,27547 
> noLock = eType!=SQLITE_OPEN_MAIN_DB;
> 
> 
> ! #if (defined(__APPLE__) || SQLITE_ENABLE_LOCKING_STYLE) &&
> !defined(OS_VXWORKS)
> struct statfs fsInfo;
> if( fstatfs(fd, &fsInfo) == -1 ){
>   ((unixFile*)pFile)->lastErrno = errno;
> ***
> *** 27530,27536 
> }
>   #endif
> 
> ! #if SQLITE_ENABLE_LOCKING_STYLE
>   #if SQLITE_PREFER_PROXY_LOCKING
> isAutoProxy = 1;
>   #endif
> --- 27554,27560 
> }
>   #endif
> 
> ! #if SQLITE_ENABLE_LOCKING_STYLE && !OS_VXWORKS
>   #if SQLITE_PREFER_PROXY_LOCKING
> isAutoProxy = 1;
>   #endif
> ***
> *** 27793,27799 
> ** tests repeatable.
> */
> memset(zBuf, 0, nBuf);
> ! #if !defined(SQLITE_TEST)
> {
>   int pid, fd;
>   fd = open("/dev/urandom", O_RDONLY);
> --- 27817,27823 
> ** tests repeatable.
> */
> memset(zBuf, 0, nBuf);
> ! #if !defined(SQLITE_TEST) && !defined(OS_VXWORKS)
> {
>   int pid, fd;
>   fd = open("/dev/urandom", O_RDONLY);
> 
> I'v used definitions as below:
> 
> EXTRA_DEFINE+=-DOS_VXWORKS_660=660
> EXTRA_DEFINE+=-DOS_VXWORKS_670=670
> EXTRA_DEFINE+=-DOS_VXWORKS_680=680
> EXTRA_DEFINE+=-DOS_VXWORKS=OS_VXWORKS_670
> EXTRA_DEFINE+=-DSQLITE_HOMEGROWN_RECURSIVE_MUTEX
> EXTRA_DEFINE+=-DSQLITE_ENABLE_LOCKING_STYLE=1
> EXTRA_DEFINE+=-DSQLITE_OMIT_LOAD_EXTENSION
> 
> 
> Best Regards,
> 
> Huang Zhihua
> 
> 2010/10/8 Scott A Mintz 
> 
> > I searched through all the mail archives to see if I could find 
someone
> > that has ported SQLite to vxWorks in kernel mode.  Apparently, there 
are a
> > few folks attempting it.  And fewer succeeding at it.
> >
> > I found an article published by ZhiHua Huang where he describes the 
mods
> > he made to port SQLite 3.6.23.1 to vxWorks 6.5
> >

Re: [sqlite] Porting SQLite 3.7.2 to vxWorks 6.7

[ZhiHua Huang]

Hello Scott,

Below is my patch on the latest SQLite 3.7.3. Please notice that I only
verify it with GCC 4.1.2 compiler in VxWorks 6.6/6.7/6.8(I have not verify
it with my real target machine yet).

*** sqlite3.c.orig2010-10-08 10:42:22.0 +0800
--- sqlite3.c2010-10-08 19:24:18.390625000 +0800
***
*** 17,22 
--- 17,26 
  ** language. The code for the "sqlite3" command-line shell is also in a
  ** separate file. This file contains only code for the core SQLite
library.
  */
+ #if defined(OS_VXWORKS)
+ #include 
+ #endif /* OS_VXWORKS */
+
  #define SQLITE_CORE 1
  #define SQLITE_AMALGAMATION 1
  #ifndef SQLITE_PRIVATE
***
*** 22795,22801 
--- 22799,22811 
  #include 
  #include 
  #include 
+
+ #if defined(OS_VXWORKS) && defined(_WRS_KERNEL)
+ #include 
+ #else
  #include 
+ #endif /* OS_VXWORKS */
+
  #include 
  #include 

***
*** 24945,24951 
--- 24955,24965 
  /*
   ** Close a file.
   */
+ #if (OS_VXWORKS < 600)
  static int semClose(sqlite3_file *id) {
+ #else
+ static int semClose_native(sqlite3_file *id) {
+ #endif
if( id ){
  unixFile *pFile = (unixFile*)id;
  semUnlock(id, NO_LOCK);
***
*** 25581,25587 
--- 25595,25607 
  }
  return -1;
}
+
+ #if defined(OS_VXWORKS) && defined(_WRS_KERNEL)
+   got = write(id->h, (char *)pBuf, cnt);
+ #else
got = write(id->h, pBuf, cnt);
+ #endif /* OS_VXWORKS */
+
  #endif
TIMER_END;
if( got<0 ){
***
*** 26762,26768 
--- 26782,26792 
semIoFinder,  /* Finder function name */
semIoMethods, /* sqlite3_io_methods object name */
1,/* shared memory is disabled */
+ #if (OS_VXWORKS < 600)
semClose, /* xClose method */
+ #else
+   semClose_native,  /* xClose method */
+ #endif
semLock,  /* xLock method */
semUnlock,/* xUnlock method */
semCheckReservedLock  /* xCheckReservedLock method */
***
*** 27517,27523 
noLock = eType!=SQLITE_OPEN_MAIN_DB;


! #if defined(__APPLE__) || SQLITE_ENABLE_LOCKING_STYLE
struct statfs fsInfo;
if( fstatfs(fd, &fsInfo) == -1 ){
  ((unixFile*)pFile)->lastErrno = errno;
--- 27541,27547 
noLock = eType!=SQLITE_OPEN_MAIN_DB;


! #if (defined(__APPLE__) || SQLITE_ENABLE_LOCKING_STYLE) &&
!defined(OS_VXWORKS)
struct statfs fsInfo;
if( fstatfs(fd, &fsInfo) == -1 ){
  ((unixFile*)pFile)->lastErrno = errno;
***
*** 27530,27536 
}
  #endif

! #if SQLITE_ENABLE_LOCKING_STYLE
  #if SQLITE_PREFER_PROXY_LOCKING
isAutoProxy = 1;
  #endif
--- 27554,27560 
}
  #endif

! #if SQLITE_ENABLE_LOCKING_STYLE && !OS_VXWORKS
  #if SQLITE_PREFER_PROXY_LOCKING
isAutoProxy = 1;
  #endif
***
*** 27793,27799 
** tests repeatable.
*/
memset(zBuf, 0, nBuf);
! #if !defined(SQLITE_TEST)
{
  int pid, fd;
  fd = open("/dev/urandom", O_RDONLY);
--- 27817,27823 
** tests repeatable.
*/
memset(zBuf, 0, nBuf);
! #if !defined(SQLITE_TEST) && !defined(OS_VXWORKS)
{
  int pid, fd;
  fd = open("/dev/urandom", O_RDONLY);

I'v used definitions as below:

EXTRA_DEFINE+=-DOS_VXWORKS_660=660
EXTRA_DEFINE+=-DOS_VXWORKS_670=670
EXTRA_DEFINE+=-DOS_VXWORKS_680=680
EXTRA_DEFINE+=-DOS_VXWORKS=OS_VXWORKS_670
EXTRA_DEFINE+=-DSQLITE_HOMEGROWN_RECURSIVE_MUTEX
EXTRA_DEFINE+=-DSQLITE_ENABLE_LOCKING_STYLE=1
EXTRA_DEFINE+=-DSQLITE_OMIT_LOAD_EXTENSION


Best Regards,

Huang Zhihua

2010/10/8 Scott A Mintz 

> I searched through all the mail archives to see if I could find someone
> that has ported SQLite to vxWorks in kernel mode.  Apparently, there are a
> few folks attempting it.  And fewer succeeding at it.
>
> I found an article published by ZhiHua Huang where he describes the mods
> he made to port SQLite 3.6.23.1 to vxWorks 6.5
> http://www.mail-archive.com/sqlite-users@sqlite.org/msg51531.html
>
> Using that as a starting point, I modified my files but I get the
> following errors:
> "C:/CCViews/Mintz_NetlinxUCS_L7x/NetLinxUCS/SQLite3_LIB/sqlite3.c", line
> 27262: error (dcc:1633): parse error  near 'struct'
> "C:/CCViews/Mintz_NetlinxUCS_L7x/NetLinxUCS/SQLite3_LIB/sqlite3.c", line
> 27262: error (dcc:1206): syntax error
> "C:/CCViews/Mintz_NetlinxUCS_L7x/NetLinxUCS/SQLite3_LIB/sqlite3.c", line
> 27262: fatal error (dcc:1340): can't recover from earlier errors
>
> This is the code it's complaining about.  The "struct statfs fsInfo;" line
> is line 27262.
>
> #ifdef FD_CLOEXEC
>  fcntl(fd, F_SETFD, fcntl(fd, F_GETFD, 0) | FD_CLOEXEC);
> #endif
>
>  noLock = eType!=SQLITE_OPEN_MAIN_DB;
>
>
> #if defined(__APPLE__) || SQLITE_ENABLE_LOCKING_STYLE
>  struct statfs fsInfo;
>  if( fstatfs(fd, &fsInfo) == -1 ){
>((unixFile*)pFile)->lastErrno = errno;
>if( dirfd>=0 ) close(dirfd); /* silently leak if fa

Re: [sqlite] Porting Sqlite to MQX OS: Question 3

[Pavel Ivanov]

If task 1 will have only one connection to customer.db and task 2 will
have only one connection to fault.db then it looks like you don't need
any locking indeed. And most part of mutexes is not necessary too,
although probably you'll need ones that protect some global variables
(can't say how many of these is there).

Pavel

On Mon, Mar 22, 2010 at 8:09 AM, GeoffW  wrote:
>
> Hello
>
> I am still working away at this porting exercise, I have a couple of general
> questions please that I would value some comments on.
>
> The absence of a file locking mechanism in MQX means the best I can do is a
> "dot lock" type locking strategy. I have implemented this now, but what is
> concerning me is the amount of extra file opening and closing it is creating
> in addition to the actual accessing of the database. In my embedded system I
> am using NOR flash which has a limited write cycle life. I dont really have
> a good feel as to what amount of extra flash write overhead I am causing, is
> there an easy way to roughly estimate this ?
>
> If this really is an issue then I have no option other than to have no file
> locking. This might be acceptable in our typical application, I just need to
> make sure I understand the limitations.
>
> Lets say for the sake of an example, I have an Application made up of 10
> tasks. Task 1 and only task 1 can access a customer.db database file. Task 2
> and only task 2 can access say a fault logging database fault.db.
>
> Are these 2 statement correct for the above situation ?
>
> 1) This will work fine with no file locking.
>
> 2) I must make sure that I have implemented the mutex system to allow me to
> define SQLITE_THREADSAFE.
>
> Just wanted to sanity check my understanding of the concepts here.
>
> Thanks Geoff
>
>
>
>
>
>
>
>
>
> --
> View this message in context: 
> http://old.nabble.com/Porting-Sqlite-to-MQX-OS%3A-Question-3-tp27985475p27985475.html
> Sent from the SQLite mailing list archive at Nabble.com.
>
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Porting Sqlite to MQX OS: Question 2

[GeoffW]

Hello Dan


>Do the tasks share a heap? And do you have multiple connections
>to the database (multiple calls to sqlite3_open() or sqlite3_open_v2().

I would like to keep the implementation as general purpose as possible so we
can use sqlite in the future in differient Applications. Therefore I will
say I need to have multiple connections to the database. Each of the tasks
in the Application, have their own task stack, but mallocs() for example
will all take blocks from a common memory pool. Not too sure of how this
effects my locking strategy implementation.

To be honest, despite having read the locking bits of osUnix.c numerous
times, I am still pretty much lost.

Regards Geoff









Dan Kennedy-4 wrote:
> 
> 
> On Mar 16, 2010, at 5:22 AM, GeoffW wrote:
> 
>>
>> Hello Dan
>>
>> Thanks for your useful input. To answer your questions.
>>
>>> Do you have any file-locking primitives provided by the OS?
>> There are no file locking OS Primitives at all that I can use.
>>
>>> Do you have clients connecting to the database from multiple
>>> processes? Or only multiple threads within the same process?
>>
>> My Application doesnt have a concept of processes and threads as you  
>> might
>> find in Windows.
>> Think of it as a  small data collection embedded Application. The  
>> platform
>> has essentially 1 fixed Application running, made up of say 10  
>> different
>> concurrent tasks.
> 
> Do the tasks share a heap? And do you have multiple connections
> to the database (multiple calls to sqlite3_open() or sqlite3_open_v2().
> 
> If you only have one connection, then you don't need any locking. Have
> xCheckReservedLock() set its result variable to 0 for all calls.
> 
> If all tasks share a heap, maybe you can implement locking in-memory
> using global variables in the VFS layer. Or, if you strictly use
> shared-cache mode, you will not need any locking.
> 
> Dan.
> 
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
> 
> 

-- 
View this message in context: 
http://old.nabble.com/Porting-Sqlite-to-MQX-OS%3A-Question-2-tp27874124p27918689.html
Sent from the SQLite mailing list archive at Nabble.com.

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Porting Sqlite to MQX OS: Question 2

[Dan Kennedy]

On Mar 16, 2010, at 5:22 AM, GeoffW wrote:

>
> Hello Dan
>
> Thanks for your useful input. To answer your questions.
>
>> Do you have any file-locking primitives provided by the OS?
> There are no file locking OS Primitives at all that I can use.
>
>> Do you have clients connecting to the database from multiple
>> processes? Or only multiple threads within the same process?
>
> My Application doesnt have a concept of processes and threads as you  
> might
> find in Windows.
> Think of it as a  small data collection embedded Application. The  
> platform
> has essentially 1 fixed Application running, made up of say 10  
> different
> concurrent tasks.

Do the tasks share a heap? And do you have multiple connections
to the database (multiple calls to sqlite3_open() or sqlite3_open_v2().

If you only have one connection, then you don't need any locking. Have
xCheckReservedLock() set its result variable to 0 for all calls.

If all tasks share a heap, maybe you can implement locking in-memory
using global variables in the VFS layer. Or, if you strictly use
shared-cache mode, you will not need any locking.

Dan.

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Porting Sqlite to MQX OS: Question 2

[GeoffW]

Hello Dan

Thanks for your useful input. To answer your questions.

>Do you have any file-locking primitives provided by the OS?
There are no file locking OS Primitives at all that I can use.

>Do you have clients connecting to the database from multiple
>processes? Or only multiple threads within the same process?

My Application doesnt have a concept of processes and threads as you might
find in Windows.
Think of it as a  small data collection embedded Application. The platform
has essentially 1 fixed Application running, made up of say 10 different
concurrent tasks. I would like to structure the program  so that any of the
tasks can safely access the database. The frequency of reading or writing to
the database will be fairly low, so performance isnt a prime consideration.
However integrity of the database is vital, I dont want to risk corruption
due to a broken or misunderstood file locking implementation.

Your message on first reading seemed to imply that the internal lockstate
variable  (NONE ,PENDING ,RESERVED, EXCLUSIVE etc) that Sqlite keeps on each
db file access is enough to make the locking safe, without needing an OS
lock/unlock primitive? I suspect I am mis-interpreting your comments here
though ?

Thanks for any further clarification on how to implement this.

Regards Geoff





Dan Kennedy-4 wrote:
> 
> 
> On Mar 15, 2010, at 10:04 PM, GeoffW wrote:
> 
>>
>> Hi Kees
>>
>> Thanks for the response and giving me a clue to look at dot locking,  
>> I had
>> missed the dot lock mechanism as I had concentrated mainly on the  
>> osWin.c
>> file. I cant find any documentation other than the source code on  
>> how this
>> low level file locking stuff is implemented.
>>
>> I am currently reading through the code to see if I can figure it  
>> out. The
>> unix.c module is horrendously complicated so I am struggling to make  
>> sense
>> of it.
> 
> It is more complicated. But the comments in it are often better than
> those in os_win.c. See the comments above unixLock(), at or around line
> 1208 of os_unix.c. Also here:
> 
>http://www.sqlite.org/lockingv3.html
> 
>> In simple terms I was thinking that when the db file is opened for  
>> writing,
>> I could use the filename with ".lock" appended, and open that new  
>> dummy
>> file, which is then used to determine if I have a lock on the original
>> database file or not ?
> 
> You can implement locking that way. But it would be a last
> resort. And is the sort of trick that could be prone to race
> conditions.
> 
> Do you have any file-locking primitives provided by the OS?
> 
> Do you have clients connecting to the database from multiple
> processes? Or only multiple threads within the same process?
> 
>> As the code says for dot locking there is really only a 2 state lock,
>> exclusive or no lock. I got lost by the complexity of the code, for  
>> vxworks
>> it seems to keep linked lists of filenames, not sure why I would  
>> need to do
>> that, also it does lots of other stuff that I didnt really  
>> understand for
>> example updating the file timestamp on the ".lock" file.
> 
> 2 locking states are all SQLite needs to work. An exclusive lock
> for writers and a shared lock for readers. The other locks are
> only required to increase concurrencly between readers and the
> single writer.
> 
> If you handle RESERVED and PENDING locks in the same way as EXCLUSIVE
> locks, things will work fine, just you lose a little concurrency.
> 
> 
> 
> 
>> I was hoping Dr Hipp could manage to find a few minutes to write  
>> some notes
>> explaining what needs to be implemented re: dot locking and what can  
>> be
>> simplified and ignored.
>>
>> Thanks if anyone can fill in some details here.
>>
>> Geoff
>>
>>
>>
>> Kees Nuyt wrote:
>>>
>>> On Fri, 12 Mar 2010 02:22:31 -0800 (PST), GeoffW
>>>  wrote:
>>>

 Hi

 Now I have got a half ported version of Sqlite up and running on  
 MQX I
>> cant
 put off any longer the question of file locking and how I tackle  
 that ?
>> For
 the moment i had made the xLock() and xUnlock() the same as in  
 osWin.c
 version but with the Windows lock() and unlock() function calls  
 commented
 out.

 The problem I have is that MQX's file system is just ANSI C, and  
 it lacks
 any lock/unlock functions.

 As Sqlite has been ported to so many diverse operating systems I  
 am hoping
 that this question of what to do for an ANSI only File system has  
 already
 been hit and a workaround defined ?

 I am pretty hazy on this topic of file locking so I would  
 appreciate some
 basic discussion and guidance. It might help if I explain my  
 application a
 little. It is an embedded system where we are just running one
>> application,
 but it is multithreaded. For my initial porting work I have made
 SQLITE_THREADSAFE = 0 (so I probably dont even need a file lock in  
 the
 configuration I have at prese

Re: [sqlite] Porting Sqlite to MQX OS: Question 2

[Dan Kennedy]

On Mar 15, 2010, at 10:04 PM, GeoffW wrote:

>
> Hi Kees
>
> Thanks for the response and giving me a clue to look at dot locking,  
> I had
> missed the dot lock mechanism as I had concentrated mainly on the  
> osWin.c
> file. I cant find any documentation other than the source code on  
> how this
> low level file locking stuff is implemented.
>
> I am currently reading through the code to see if I can figure it  
> out. The
> unix.c module is horrendously complicated so I am struggling to make  
> sense
> of it.

It is more complicated. But the comments in it are often better than
those in os_win.c. See the comments above unixLock(), at or around line
1208 of os_unix.c. Also here:

   http://www.sqlite.org/lockingv3.html

> In simple terms I was thinking that when the db file is opened for  
> writing,
> I could use the filename with ".lock" appended, and open that new  
> dummy
> file, which is then used to determine if I have a lock on the original
> database file or not ?

You can implement locking that way. But it would be a last
resort. And is the sort of trick that could be prone to race
conditions.

Do you have any file-locking primitives provided by the OS?

Do you have clients connecting to the database from multiple
processes? Or only multiple threads within the same process?

> As the code says for dot locking there is really only a 2 state lock,
> exclusive or no lock. I got lost by the complexity of the code, for  
> vxworks
> it seems to keep linked lists of filenames, not sure why I would  
> need to do
> that, also it does lots of other stuff that I didnt really  
> understand for
> example updating the file timestamp on the ".lock" file.

2 locking states are all SQLite needs to work. An exclusive lock
for writers and a shared lock for readers. The other locks are
only required to increase concurrencly between readers and the
single writer.

If you handle RESERVED and PENDING locks in the same way as EXCLUSIVE
locks, things will work fine, just you lose a little concurrency.




> I was hoping Dr Hipp could manage to find a few minutes to write  
> some notes
> explaining what needs to be implemented re: dot locking and what can  
> be
> simplified and ignored.
>
> Thanks if anyone can fill in some details here.
>
> Geoff
>
>
>
> Kees Nuyt wrote:
>>
>> On Fri, 12 Mar 2010 02:22:31 -0800 (PST), GeoffW
>>  wrote:
>>
>>>
>>> Hi
>>>
>>> Now I have got a half ported version of Sqlite up and running on  
>>> MQX I
> cant
>>> put off any longer the question of file locking and how I tackle  
>>> that ?
> For
>>> the moment i had made the xLock() and xUnlock() the same as in  
>>> osWin.c
>>> version but with the Windows lock() and unlock() function calls  
>>> commented
>>> out.
>>>
>>> The problem I have is that MQX's file system is just ANSI C, and  
>>> it lacks
>>> any lock/unlock functions.
>>>
>>> As Sqlite has been ported to so many diverse operating systems I  
>>> am hoping
>>> that this question of what to do for an ANSI only File system has  
>>> already
>>> been hit and a workaround defined ?
>>>
>>> I am pretty hazy on this topic of file locking so I would  
>>> appreciate some
>>> basic discussion and guidance. It might help if I explain my  
>>> application a
>>> little. It is an embedded system where we are just running one
> application,
>>> but it is multithreaded. For my initial porting work I have made
>>> SQLITE_THREADSAFE = 0 (so I probably dont even need a file lock in  
>>> the
>>> configuration I have at present ?)
>>> However I would of course like to make SQLITE_THREADSAFE = 1 so that
>>> multiple tasks can safely access a database.
>>
>> (Disclaimer: this is not my specialism, I hope it helps
>> nevertheless)
>>
>> SQLITE_THREADSAFE has effect on threads within the same
>> process (=task), especially when they use the same
>> connection. You need file locks for synchronisation between
>> different processes. If the filesystem doesn't provide them,
>> you could use the dot lockfile paradigm, which seems to be
>> associated with SQLITE_ENABLE_LOCKING_STYLE.
>>
>>> Thanks for any guidance
>>>
>>> Regards Geoff
>> -- 
>>  (  Kees Nuyt
>>  )
>> c[_]
>> ___
>> sqlite-users mailing list
>> sqlite-users@sqlite.org
>> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>>
>>
>
> -- 
> View this message in context: 
> http://old.nabble.com/Porting-Sqlite-to-MQX-OS%3A-Question-2-tp27874124p27905578.html
> Sent from the SQLite mailing list archive at Nabble.com.
>
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Porting Sqlite to MQX OS: Question 2

[GeoffW]

Hi Kees

Thanks for the response and giving me a clue to look at dot locking, I had
missed the dot lock mechanism as I had concentrated mainly on the osWin.c
file. I cant find any documentation other than the source code on how this
low level file locking stuff is implemented.

I am currently reading through the code to see if I can figure it out. The
unix.c module is horrendously complicated so I am struggling to make sense
of it. 

In simple terms I was thinking that when the db file is opened for writing,
I could use the filename with ".lock" appended, and open that new dummy
file, which is then used to determine if I have a lock on the original
database file or not ?
As the code says for dot locking there is really only a 2 state lock,
exclusive or no lock. I got lost by the complexity of the code, for vxworks
it seems to keep linked lists of filenames, not sure why I would need to do
that, also it does lots of other stuff that I didnt really understand for
example updating the file timestamp on the ".lock" file.

I was hoping Dr Hipp could manage to find a few minutes to write some notes
explaining what needs to be implemented re: dot locking and what can be
simplified and ignored.

Thanks if anyone can fill in some details here.

Geoff



Kees Nuyt wrote:
> 
> On Fri, 12 Mar 2010 02:22:31 -0800 (PST), GeoffW
>  wrote:
> 
>>
>>Hi
>>
>>Now I have got a half ported version of Sqlite up and running on MQX I
cant
>>put off any longer the question of file locking and how I tackle that ?
For
>>the moment i had made the xLock() and xUnlock() the same as in osWin.c
>>version but with the Windows lock() and unlock() function calls commented
>>out.
>>
>>The problem I have is that MQX's file system is just ANSI C, and it lacks
>>any lock/unlock functions.
>>
>>As Sqlite has been ported to so many diverse operating systems I am hoping
>>that this question of what to do for an ANSI only File system has already
>>been hit and a workaround defined ?
>>
>>I am pretty hazy on this topic of file locking so I would appreciate some
>>basic discussion and guidance. It might help if I explain my application a
>>little. It is an embedded system where we are just running one
application,
>>but it is multithreaded. For my initial porting work I have made
>>SQLITE_THREADSAFE = 0 (so I probably dont even need a file lock in the
>>configuration I have at present ?)
>>However I would of course like to make SQLITE_THREADSAFE = 1 so that
>>multiple tasks can safely access a database.
> 
> (Disclaimer: this is not my specialism, I hope it helps
> nevertheless)
> 
> SQLITE_THREADSAFE has effect on threads within the same
> process (=task), especially when they use the same
> connection. You need file locks for synchronisation between
> different processes. If the filesystem doesn't provide them,
> you could use the dot lockfile paradigm, which seems to be
> associated with SQLITE_ENABLE_LOCKING_STYLE.
> 
>>Thanks for any guidance
>>
>>Regards Geoff
> -- 
>   (  Kees Nuyt
>   )
> c[_]
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
> 
> 

-- 
View this message in context: 
http://old.nabble.com/Porting-Sqlite-to-MQX-OS%3A-Question-2-tp27874124p27905578.html
Sent from the SQLite mailing list archive at Nabble.com.

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Porting Sqlite to MQX OS: Question 2

[Kees Nuyt]

On Fri, 12 Mar 2010 02:22:31 -0800 (PST), GeoffW
 wrote:

>
>Hi
>
>Now I have got a half ported version of Sqlite up and running on MQX I cant
>put off any longer the question of file locking and how I tackle that ? For
>the moment i had made the xLock() and xUnlock() the same as in osWin.c
>version but with the Windows lock() and unlock() function calls commented
>out.
>
>The problem I have is that MQX's file system is just ANSI C, and it lacks
>any lock/unlock functions.
>
>As Sqlite has been ported to so many diverse operating systems I am hoping
>that this question of what to do for an ANSI only File system has already
>been hit and a workaround defined ?
>
>I am pretty hazy on this topic of file locking so I would appreciate some
>basic discussion and guidance. It might help if I explain my application a
>little. It is an embedded system where we are just running one application,
>but it is multithreaded. For my initial porting work I have made
>SQLITE_THREADSAFE = 0 (so I probably dont even need a file lock in the
>configuration I have at present ?)
>However I would of course like to make SQLITE_THREADSAFE = 1 so that
>multiple tasks can safely access a database.

(Disclaimer: this is not my specialism, I hope it helps
nevertheless)

SQLITE_THREADSAFE has effect on threads within the same
process (=task), especially when they use the same
connection. You need file locks for synchronisation between
different processes. If the filesystem doesn't provide them,
you could use the dot lockfile paradigm, which seems to be
associated with SQLITE_ENABLE_LOCKING_STYLE.

>Thanks for any guidance
>
>Regards Geoff
-- 
  (  Kees Nuyt
  )
c[_]
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Porting Sqlite to MQX Operating system

[GeoffW]

hello 

I thought I had better update this and confess to my sins. Good job no one
is reading this thread as it is an embarassingly stupid mistake :rules:

I cant believe I did this and then took so long to spot it.

int sqlite3_os_init(void)
{
static sqlite3_vfs mqxVfs = {
1, /* iVersion */
0,   /* szOsFile ->ARRRGGGH <*/
MAX_PATH,  /* mxPathname */
etc 

I started creating the ported file from osWin.c, I had quite a struggle
getting it to compile initially using CodeWarrior, so I was making several
temporary hacks to allow it to compile, which I then revisited later to
correct. I had replaced the sizeof(winFile) with a zero, and then forgot to
change it to sizeof(mqxFile).
This caused a fun crash down in the bowels of the paging code.

Once I corrected this one liner it fixed the crash and sqlite is basically
up and running now. I still have a fair bit of work left to implement some
interface functions which are just dummy stubs at present, but I am making
progress now.

Regards Geoff




-- 
View this message in context: 
http://old.nabble.com/Porting-Sqlite-to-MQX-Operating-system-tp27792715p27874110.html
Sent from the SQLite mailing list archive at Nabble.com.

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Porting Sqlite to MQX Operating system

[GeoffW]

Hi 

I wasted another entire day on this without really being able to grasp what
the problem is.

The call sequence is for the sqlite3OsClose() function

Working Windows version

pager_unlock() pPager->jfpd where pMethods =0
releaseAllSavepoints() pPager->sjfpd where pMethods =0
above 2 lines repeat 3 times 

then

sqlite3PagerClose() pPager->fd where pMethods  <> 0  hence a real file close
of the test.db file occurs


MQX faulty version

pager_unlock() pPager->jfpd where pMethods = 0
pager_unlock() pPager->jfpd where pMethods = 0

releaseAllSavepoints() pPager->sjfpd where pMethods <> 0

This call to releaseAllSavepoints has the pMethods pointer pointing at the
sqlite3_io_methods mqxIoMethod structure, so a  call to mqxClose() occurs
and a the db file is erroneously closed.

Any ideas on this would be greatly appreciated. Thanks

Geoff







GeoffW wrote:
> 
> hello Dan 
> 
> Thanks for the very speedy reply. I have just been experimenting with the
> working Windows version.
> 
> I only get one hit of the winOpen() function and that is with a filename
> parameter of the full path to my test.db file
> 
> So maybe it is not a temporary journal file or maybe I am misunderstanding
> what you a saying ?
> 
> 
> The only other thing I have noticed since my initial post is that even for
> the working Windows version I am getting 7 calls to the sqlite3OsClose()
> function.  The actual close of the file is prevented by the line if(
> pId->pMethods ). As that is NULL, it doesnt close the file except for the
> very last 7th time it is called where pMethods is not NULL.   I didnt
> expect it to do this on the windows version, seems odd to me.
> 
> On MQX the 2nd call to sqlite3OsClose() has pId->pMethods != NULL so an
> actual close occurs.
> 
> Geoff
> 
> 
> 
> 
> 
> Dan Kennedy-4 wrote:
>> 
>>>
>>> http://old.nabble.com/file/p27792715/stack.jpg
>>>
>>> I know its a long shot but does Dr Hipp or anyone have any possible  
>>> ideas
>>> what could give rise to a premature file close ?  Sorry my message  
>>> was so
>>> long. Thanks for any ideas
>> 
>> SQLite thinks it is closing the statement journal here, not
>> the db file. This file should have been opened with the pathname
>> argument to the xOpen call set to NULL.
>> 
>> Dan.
>> 
>> ___
>> sqlite-users mailing list
>> sqlite-users@sqlite.org
>> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>> 
>> 
> 
> 

-- 
View this message in context: 
http://old.nabble.com/Porting-Sqlite-to-MQX-Operating-system-tp27792715p27823835.html
Sent from the SQLite mailing list archive at Nabble.com.

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Porting Sqlite to MQX Operating system

[GeoffW]

hello Dan 

Thanks for the very speedy reply. I have just been experimenting with the
working Windows version.

I only get one hit of the winOpen() function and that is with a filename
parameter of the full path to my test.db file

So maybe it is not a temporary journal file or maybe I am misunderstanding
what you a saying ?


The only other thing I have noticed since my initial post is that even for
the working Windows version I am getting 7 calls to the sqlite3OsClose()
function.  The actual close of the file is prevented by the line if(
pId->pMethods ). As that is NULL, it doesnt close the file except for the
very last 7th time it is called where pMethods is not NULL.   I didnt expect
it to do this on the windows version, seems odd to me.

On MQX the 2nd call to sqlite3OsClose() has pId->pMethods != NULL so an
actual close occurs.

Geoff





Dan Kennedy-4 wrote:
> 
>>
>> http://old.nabble.com/file/p27792715/stack.jpg
>>
>> I know its a long shot but does Dr Hipp or anyone have any possible  
>> ideas
>> what could give rise to a premature file close ?  Sorry my message  
>> was so
>> long. Thanks for any ideas
> 
> SQLite thinks it is closing the statement journal here, not
> the db file. This file should have been opened with the pathname
> argument to the xOpen call set to NULL.
> 
> Dan.
> 
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
> 
> 

-- 
View this message in context: 
http://old.nabble.com/Porting-Sqlite-to-MQX-Operating-system-tp27792715p27793173.html
Sent from the SQLite mailing list archive at Nabble.com.

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Porting Sqlite to MQX Operating system

[Dan Kennedy]

>
> http://old.nabble.com/file/p27792715/stack.jpg
>
> I know its a long shot but does Dr Hipp or anyone have any possible  
> ideas
> what could give rise to a premature file close ?  Sorry my message  
> was so
> long. Thanks for any ideas

SQLite thinks it is closing the statement journal here, not
the db file. This file should have been opened with the pathname
argument to the xOpen call set to NULL.

Dan.

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Porting into a microcontroller, minimum requirements

[Dennis Jenkins]

Jay A. Kreibich wrote:
> On Wed, May 07, 2008 at 10:25:49PM -0400, Andrew Cunningham scratched on the 
> wall:
>
>   
>> "I have doubts that you will be able to get SQLite to work on anything
>> less than a 32-bit processor.
>> D. Richard Hipp"
>> 
>
>   
>> I was under the impression as long as the processor had enough room to
>> hold the program (and RAM) it would work.
>> 
>
>   Yeah, but you have to compile it first...
>
>   
>> The difference I would have
>> expected would simply be speed of execution, but eventually getting
>> there.  With simple inserts/queries and the speed of sqlite, I thought
>> it should be okay.
>> 
>
>   I suspect the issue has to do with 64-bit integer support.  SQLite does
>   a lot with native 64-bit integer values (e.g. "unsigned long long int").
>   Most 32-bit processors have instructions that can deal with 64 bit
>   values as single, whole values, even if they require a lot of
>   slight-of-hand behind the scenes.  Even if the processor doesn't have
>   explicit instructions, the compiler can often fake it by using the 
>   overflow bits and a lot of byte shuffling.
>
>   I suppose it would be possible to play the same games with a 16 bit
>   (or even 8) processor, but I'm guessing the compilers for most of
>   these smaller chips don't support long long ints.  Even if they do,
>   it is going to be pretty slow and inflate the code side.
>
> -j
>
>   

I once thought about trying to compile it with "cc65" to run on an 
Apple IIe, for fun.  I then calculated that the Apple II doesn't have 
enough RAM in a flat address space.  I then decided that I'd rather go 
outside and enjoy the sunshine. :)

However... I still wonder if it would be possible.  Maybe if the 
compile could automatically split the code into sections that could work 
like "overlays" did back in the Borland C/Pascal days on MS-DOS.  But I 
suspect that memory access would be a problem

I could also compile Sqlite into MIPS R2000, and write a small 
emulator to run on the Apple (that transparently handled bank-switching 
and paging RAM).  That might work.  Slowly...


-- 
CONFIDENTIALITY NOTICE
This electronic mail and the information contained herein are intended for the 
named recipient only.  It may contain confidential, proprietary and/or 
privileged information.  If you have received this electronic mail in error, 
please do not read any text other than the text of this notice and do not open 
any attachments. Also, please immediately notify the sender by replying to this 
electronic mail or by collect call to (262) 796-0925. After notifying the 
sender as described above, please delete this electronic mail message 
immediately and purge the item from the deleted items folder (or the 
equivalent) of your electronic mail system. Thank you.

--

The above email disclaimer is required by our legal department.  However, if 
you would like a more humorous disclaimer, check here: 
http://www-users.cs.york.ac.uk/susan/joke/disclaim.htm


___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Porting into a microcontroller, minimum requirements

[Jay A. Kreibich]

On Wed, May 07, 2008 at 10:25:49PM -0400, Andrew Cunningham scratched on the 
wall:

> "I have doubts that you will be able to get SQLite to work on anything
> less than a 32-bit processor.
> D. Richard Hipp"

> I was under the impression as long as the processor had enough room to
> hold the program (and RAM) it would work.

  Yeah, but you have to compile it first...

> The difference I would have
> expected would simply be speed of execution, but eventually getting
> there.  With simple inserts/queries and the speed of sqlite, I thought
> it should be okay.

  I suspect the issue has to do with 64-bit integer support.  SQLite does
  a lot with native 64-bit integer values (e.g. "unsigned long long int").
  Most 32-bit processors have instructions that can deal with 64 bit
  values as single, whole values, even if they require a lot of
  slight-of-hand behind the scenes.  Even if the processor doesn't have
  explicit instructions, the compiler can often fake it by using the 
  overflow bits and a lot of byte shuffling.

  I suppose it would be possible to play the same games with a 16 bit
  (or even 8) processor, but I'm guessing the compilers for most of
  these smaller chips don't support long long ints.  Even if they do,
  it is going to be pretty slow and inflate the code side.

-j

-- 
Jay A. Kreibich < J A Y  @  K R E I B I.C H >

"'People who live in bamboo houses should not throw pandas.' Jesus said that."
   - "The Ninja", www.AskANinja.com, "Special Delivery 10: Pop!Tech 2006"
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Porting into a microcontroller, minimum requirements

[Andrew Cunningham]

*snip*

"I have doubts that you will be able to get SQLite to work on anything
less than a 32-bit processor.
D. Richard Hipp"

I was under the impression as long as the processor had enough room to
hold the program (and RAM) it would work.  The difference I would have
expected would simply be speed of execution, but eventually getting
there.  With simple inserts/queries and the speed of sqlite, I thought
it should be okay.

If I went with a 32 bit processor, would 512KB of flash and 32KB of
RAM (same 2GB storage) be enough assuming I didn't perform huge
queries?  If I build without any OS, are there extra libraries that
would be required?

Andy
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Porting into a microcontroller, minimum requirements

[Eduardo Morras]

At 02:43 07/05/2008, you wrote:

>Hi,
>
>I was wondering if anyone has any basic guide lines on embedding SQLite into
>a microcontroller.  For example, I am considering using an 8/16 bit processor
>with 1 MB flash, 1 MB SRAM and 2 GB data storage (SD card).  Has anyone ported
>this before to an embedded system without an OS?

I have ported it to a 32bit. Take a look at contiki, it's a nano 
(femto) OS that works very well, is very portable and easy to add 
sqlite to it (in 32 bit world)

>Thanks,
>Andy

L


One reason that life is complex is that it has a real part and an 
imaginary part
-Andrew Koenig  

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Porting into a microcontroller, minimum requirements

[D. Richard Hipp]

On May 6, 2008, at 8:43 PM, Andrew Cunningham wrote:

> Hi,
>
> I was wondering if anyone has any basic guide lines on embedding  
> SQLite into
> a microcontroller.  For example, I am considering using an 8/16 bit  
> processor
> with 1 MB flash, 1 MB SRAM and 2 GB data storage (SD card).  Has  
> anyone ported
> this before to an embedded system without an OS?
>

I have doubts that you will be able to get SQLite to work on anything  
less than a 32-bit processor.

D. Richard Hipp
[EMAIL PROTECTED]



___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] porting sqlite3 to embeded os-----lock question

[allen . zhang]

thank you for your reply.
I have make it run on our embeded os and make some simple tests. it runs 
ok.
great!!





"Nuno Lucas" <[EMAIL PROTECTED]> 
2007-05-12 07:03
Please respond to
sqlite-users@sqlite.org


To
sqlite-users@sqlite.org
cc

Subject
Re: [sqlite] porting sqlite3 to embeded os-lock question






On 5/9/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> 
wrote:
> I am porting sqlite3 to the embeded os,such as threadx,nucleus,ect..
> I am writing the file such as os_threadx.c,os_nucleus.c according to the
> os_win.c,os_unix.c.
> I have read the os_win.c and find that there is a switcher OS_WINCE in 
the
> struct winFile.
> Is this mean the windows platform don't need the function such as share
> memory(CreateFileMappingW,MapViewOfFile) inside the OS_WINCE 
swither?

The shared memory is needed on WinCE to implement the LockFile
functions (which don't exist on WinCE).
If you don't need to do locking (because only your app will access the
db), you don't need the shared memory thing.

>
> whether I should realize the share memory lock function in the embeded 
os?
> I have find there is not a direct way similar to the windows share 
memory
> and the interface funcitons.
> It seems difficult to simulate the share memory funciton and it's lock
> function in my embeded os.
> Does it mean I must realize it. or the porting will fail.

It's up to you to know what level of compatibility you need. I would
guess that for you embedded os you don't need the locking part, so can
safely replace it with dummy functions that always succeed.

>
> another question:
> There is also a little difficult to realize the
> sqlite3WinThreadSpecificData function to get the thread information,
> Is this also must realize ?

If you use threads, then that would depend on your use of sqlite.


Regards,
~Nuno Lucas

>
> thanks a lot.
> allen.zhang
>
>

-
To unsubscribe, send email to [EMAIL PROTECTED]
-

Re: [sqlite] porting sqlite3 to embeded os-----lock question

[Nuno Lucas]

On 5/9/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote:

I am porting sqlite3 to the embeded os,such as threadx,nucleus,ect..
I am writing the file such as os_threadx.c,os_nucleus.c according to the
os_win.c,os_unix.c.
I have read the os_win.c and find that there is a switcher OS_WINCE in the
struct winFile.
Is this mean the windows platform don't need the function such as share
memory(CreateFileMappingW,MapViewOfFile) inside the OS_WINCE swither?


The shared memory is needed on WinCE to implement the LockFile
functions (which don't exist on WinCE).
If you don't need to do locking (because only your app will access the
db), you don't need the shared memory thing.



whether I should realize the share memory lock function in the embeded os?
I have find there is not a direct way similar to the windows share memory
and the interface funcitons.
It seems difficult to simulate the share memory funciton and it's lock
function in my embeded os.
Does it mean I must realize it. or the porting will fail.


It's up to you to know what level of compatibility you need. I would
guess that for you embedded os you don't need the locking part, so can
safely replace it with dummy functions that always succeed.



another question:
There is also a little difficult to realize the
sqlite3WinThreadSpecificData function to get the thread information,
Is this also must realize ?


If you use threads, then that would depend on your use of sqlite.


Regards,
~Nuno Lucas



thanks a lot.
allen.zhang




-
To unsubscribe, send email to [EMAIL PROTECTED]
-

[sqlite] Re : [sqlite] Porting sqlite3 library problems....

[Fred a]

Hello,
 
Thank you for your help ...
But i found the problem... On my platform seeking in file seem buggy in certain 
case ...
So after a moment my sqlite3 header disappear from db file.
 
Thank you, D. Richard Hipp, the new way for port sqlite3 is very good ... 
Sqlite3 is a very good library ...
 
Best regards,
 
Fred

- Message d'origine 
De : [EMAIL PROTECTED]
À : sqlite-users@sqlite.org; Fred a <[EMAIL PROTECTED]>
Envoyé le : Dimanche, 21 Mai 2006, 8h09mn 57s
Objet : Re: [sqlite] Porting sqlite3 library problems


Fred a <[EMAIL PROTECTED]> wrote:
> Hello all,
>  
> I port successfully sqlite3 library on other platform...
> But i have some problems...
>  
> When i try to insert some records, if for example i insert 
> 80: good; if i try to insert 81 records the database is corrupted

This suggests to me that your port is not as
successful as you think it is.

So what is the "other platform" that you are
porting to?
--
D. Richard Hipp   <[EMAIL PROTECTED]>

Re: [sqlite] Porting sqlite3 library problems....

[drh]

Fred a <[EMAIL PROTECTED]> wrote:
> Hello all,
>  
> I port successfully sqlite3 library on other platform...
> But i have some problems...
>  
> When i try to insert some records, if for example i insert 
> 80: good; if i try to insert 81 records the database is corrupted

This suggests to me that your port is not as
successful as you think it is.

So what is the "other platform" that you are
porting to?
--
D. Richard Hipp   <[EMAIL PROTECTED]>

Re: [sqlite] Porting sqlite3 library problems....

[Jay Sprenkle]

On 5/21/06, Fred a <[EMAIL PROTECTED]> wrote:

Hello all,

I port successfully sqlite3 library on other platform...
But i have some problems...

When i try to insert some records, if for example i insert 80: good; if i try 
to insert 81 records the database is corrupted
I don t understand because if make a SELECT query with my library on a big 
table created on pc with sqlite3 win32 : it work fine.


Can you post a test program?
Have you checked for hardware problems?

Re: [sqlite] Porting SQL to run on a proprietary operating system

[Axel Mammes]

I gave up on this port. Far more complicated than what I initially thought.
Guess I'll have to implement stupid flat files manually. :((


On 12/30/05, Jay Sprenkle <[EMAIL PROTECTED]> wrote:
> Yes, most people load the data to persistant memory when shutting
> down, or periodically
> to save a snapshot, and load it when booting up. It's fast but if your 
> hardware
> can't detect a power failure and write the data with what power
> remains in the power supply
> capacitors you risk losing some data in case of power failure between 
> snapshots.
>
> On 12/30/05, Axel Mammes <[EMAIL PROTECTED]> wrote:
> > If I use memory database, the content will be zeroed when I reboot the
> > equipment. Only memory that is allocated for the filesystem is
> > persistant between power cycles.
> >
>

Re: [sqlite] Porting SQL to run on a proprietary operating system

[Jay Sprenkle]

Yes, most people load the data to persistant memory when shutting
down, or periodically
to save a snapshot, and load it when booting up. It's fast but if your hardware
can't detect a power failure and write the data with what power
remains in the power supply
capacitors you risk losing some data in case of power failure between snapshots.

On 12/30/05, Axel Mammes <[EMAIL PROTECTED]> wrote:
> If I use memory database, the content will be zeroed when I reboot the
> equipment. Only memory that is allocated for the filesystem is
> persistant between power cycles.
>

Re: [sqlite] Porting SQL to run on a proprietary operating system

[Axel Mammes]

If I use memory database, the content will be zeroed when I reboot the
equipment. Only memory that is allocated for the filesystem is
persistant between power cycles.

On 12/29/05, Jay Sprenkle <[EMAIL PROTECTED]> wrote:
> If you use ":memory:" as your database name and it will keep the tables in 
> RAM.
> The only drawback there is I believe you can't share data between threads. 
> Since
> you only have one thread that might be just what you need.
>
> On 12/29/05, Axel Mammes (gmail) <[EMAIL PROTECTED]> wrote:
> > Flash will only be used for seldom changed tables or config parameters.
> >
> > Otherwise I am limited to the battery backed up RAM-based file system.
> >
> >
> >
> > -Original Message-
> > From: Jay Sprenkle [mailto:[EMAIL PROTECTED]
> > Sent: Jueves, 29 de Diciembre de 2005 12:10 p.m.
> > To: [EMAIL PROTECTED]
> > Subject: Re: [sqlite] Porting SQL to run on a proprietary operating system
> >
> > I did a project using sqlite under web server CGI. It worked very well and
> > was pretty light on required resources. My guess is this will work for you
> > as well but you'll need to research some changes to the database engine to
> > work against flash drives. I believe several people use in ram databases
> > and occasionally flush them to flash storage to prevent 'wearing out' the
> > flash prematurely.
> >
> > On 12/29/05, Axel Mammes (gmail) <[EMAIL PROTECTED]> wrote:
> > > I write software for electronic funds transfer terminals. For this project
> > > in particular I am using a Verifone Vx570 terminal (www.verifone.com).
> > >
> > > The platform consists in a 32 bit ARM9 processor with 4-32 MB battery
> > backed
> > > up RAM and 4-32 flash. The operating system is called Verix. It supports
> > > multi-tasking, but multithreading is rather limited. For example a file
> > > handle opened in one thread cannot be used by another thread. The compiler
> > I
> > > will be using is ARM Real View Compiler 2.01.
> > >
> > > My app will have approximately 80 tables, so I want to add ACID
> > transaction
> > > support, fast indexed lookups, compressed tables and SQL query support. I
> > > don´t want to use standard flat files, since I will end up writing a lot
> > > more code. I already made that mistake in a previous project, trying to
> > > reinvent the wheel.
> > >
> > > I will probably have to create a single threaded DB Engine task and use
> > > pipes to send messages between tasks, since shared memory is not supported
> > > either. I want to keep only one instance of the engine loaded to save
> > > memory.
> > >
> > > Is this the correct approach or should I just dynamically link the sqlite
> > > engine to all my tasks and forget about pipes to make my life easier?
> > >
> > > Any ideas/suggestions? Has anyone here written a port like this? What
> > about
> > > the memory limitations? Is this too strict?
> > >
> > > Regards
> > > Axel
> > >
> > >
> >
> >
> > --
> > ---
> > The Castles of Dereth Calendar: a tour of the art and architecture of
> > Asheron's Call
> > http://www.lulu.com/content/77264
> >
> >
>
>
> --
> ---
> The Castles of Dereth Calendar: a tour of the art and architecture of
> Asheron's Call
> http://www.lulu.com/content/77264
>

Re: [sqlite] Porting SQL to run on a proprietary operating system

[Jay Sprenkle]

If you use ":memory:" as your database name and it will keep the tables in RAM.
The only drawback there is I believe you can't share data between threads. Since
you only have one thread that might be just what you need.

On 12/29/05, Axel Mammes (gmail) <[EMAIL PROTECTED]> wrote:
> Flash will only be used for seldom changed tables or config parameters.
>
> Otherwise I am limited to the battery backed up RAM-based file system.
>
>
>
> -Original Message-
> From: Jay Sprenkle [mailto:[EMAIL PROTECTED]
> Sent: Jueves, 29 de Diciembre de 2005 12:10 p.m.
> To: [EMAIL PROTECTED]
> Subject: Re: [sqlite] Porting SQL to run on a proprietary operating system
>
> I did a project using sqlite under web server CGI. It worked very well and
> was pretty light on required resources. My guess is this will work for you
> as well but you'll need to research some changes to the database engine to
> work against flash drives. I believe several people use in ram databases
> and occasionally flush them to flash storage to prevent 'wearing out' the
> flash prematurely.
>
> On 12/29/05, Axel Mammes (gmail) <[EMAIL PROTECTED]> wrote:
> > I write software for electronic funds transfer terminals. For this project
> > in particular I am using a Verifone Vx570 terminal (www.verifone.com).
> >
> > The platform consists in a 32 bit ARM9 processor with 4-32 MB battery
> backed
> > up RAM and 4-32 flash. The operating system is called Verix. It supports
> > multi-tasking, but multithreading is rather limited. For example a file
> > handle opened in one thread cannot be used by another thread. The compiler
> I
> > will be using is ARM Real View Compiler 2.01.
> >
> > My app will have approximately 80 tables, so I want to add ACID
> transaction
> > support, fast indexed lookups, compressed tables and SQL query support. I
> > don´t want to use standard flat files, since I will end up writing a lot
> > more code. I already made that mistake in a previous project, trying to
> > reinvent the wheel.
> >
> > I will probably have to create a single threaded DB Engine task and use
> > pipes to send messages between tasks, since shared memory is not supported
> > either. I want to keep only one instance of the engine loaded to save
> > memory.
> >
> > Is this the correct approach or should I just dynamically link the sqlite
> > engine to all my tasks and forget about pipes to make my life easier?
> >
> > Any ideas/suggestions? Has anyone here written a port like this? What
> about
> > the memory limitations? Is this too strict?
> >
> > Regards
> > Axel
> >
> >
>
>
> --
> ---
> The Castles of Dereth Calendar: a tour of the art and architecture of
> Asheron's Call
> http://www.lulu.com/content/77264
>
>


--
---
The Castles of Dereth Calendar: a tour of the art and architecture of
Asheron's Call
http://www.lulu.com/content/77264

RE: [sqlite] Porting SQL to run on a proprietary operating system

[Axel Mammes \(gmail\)]

Flash will only be used for seldom changed tables or config parameters.

Otherwise I am limited to the battery backed up RAM-based file system.



-Original Message-
From: Jay Sprenkle [mailto:[EMAIL PROTECTED] 
Sent: Jueves, 29 de Diciembre de 2005 12:10 p.m.
To: [EMAIL PROTECTED]
Subject: Re: [sqlite] Porting SQL to run on a proprietary operating system

I did a project using sqlite under web server CGI. It worked very well and
was pretty light on required resources. My guess is this will work for you
as well but you'll need to research some changes to the database engine to
work against flash drives. I believe several people use in ram databases
and occasionally flush them to flash storage to prevent 'wearing out' the
flash prematurely.

On 12/29/05, Axel Mammes (gmail) <[EMAIL PROTECTED]> wrote:
> I write software for electronic funds transfer terminals. For this project
> in particular I am using a Verifone Vx570 terminal (www.verifone.com).
>
> The platform consists in a 32 bit ARM9 processor with 4-32 MB battery
backed
> up RAM and 4-32 flash. The operating system is called Verix. It supports
> multi-tasking, but multithreading is rather limited. For example a file
> handle opened in one thread cannot be used by another thread. The compiler
I
> will be using is ARM Real View Compiler 2.01.
>
> My app will have approximately 80 tables, so I want to add ACID
transaction
> support, fast indexed lookups, compressed tables and SQL query support. I
> don´t want to use standard flat files, since I will end up writing a lot
> more code. I already made that mistake in a previous project, trying to
> reinvent the wheel.
>
> I will probably have to create a single threaded DB Engine task and use
> pipes to send messages between tasks, since shared memory is not supported
> either. I want to keep only one instance of the engine loaded to save
> memory.
>
> Is this the correct approach or should I just dynamically link the sqlite
> engine to all my tasks and forget about pipes to make my life easier?
>
> Any ideas/suggestions? Has anyone here written a port like this? What
about
> the memory limitations? Is this too strict?
>
> Regards
> Axel
>
>


--
---
The Castles of Dereth Calendar: a tour of the art and architecture of
Asheron's Call
http://www.lulu.com/content/77264

Re: [sqlite] Porting...

[Christian Smith]

On Wed, 14 Sep 2005, Jay Siegel wrote:

>I'm on my second day of porting SQLite to an embedded
>environment.  I'm not sure if this is the appropriate
>place to put these comments about the experience.
>I've seen some type of "ticket" mechanism in CVS but I
>don't know if these comments raise to the level of a
>"bug", etc.
>
>Anyway, three things:
>
>1. There are many instances where (char *) to(unsigned
>char *) implied casts are done (and vice versa).
>These should really be fixed so fewer compiler
>warnings are generated.  Perhaps I'm the only one who
>likes my builds to have no warnings...


Patches are always welcome, especially patches to make the build cleaner
but which otherwise don't affect functionality, as these are generally
easier to review.


>
>2. There is an implied availability of ctype, stdargs,
>stdio, etc.  All of these #includes and references to
>the routines used within them should be moved to the
>OS file.  It is not always the case that these
>routines are available and in fact, are not available
>on my platform.


SQLite is intended to be targeted at ANSI C based platforms. ctype.h,
stdarg.h and stdio.h are all C89 files, and it is reasonable to expect
these files are available. Where they are not, the platform is seriously
deficient in what it provides, and in fact varargs are not even usable
without stdarg.h. Would you expect SQLite to be viable if your platform
were 8-bit? Probably not. So don't expect it to be viable without change
if you haven't even got a reasonable ANSI C environment.

Christian

-- 
/"\
\ /ASCII RIBBON CAMPAIGN - AGAINST HTML MAIL
 X   - AGAINST MS ATTACHMENTS
/ \

Re: [sqlite] Porting...

[D. Richard Hipp]

On Wed, 2005-09-14 at 11:55 -0700, Jay Siegel wrote:
> I turned on almost all of the OMIT definitions
> since size is very important on my embedded platform. 
> This causes a few problems with definitions in
> parse.c.  This is documented somewhere and I expected
> it.  It shouldn't be necessary to hand-code out the
> parse.c code for triggers, etc. though.  It would be
> much easier if, based on the various OMIT definitions,
> stub routines would be defined that force a parse
> error, etc.

Pass the appropriate -D options into lemon when
you are generating parse.c and no hand editing
is required.

-- 
D. Richard Hipp <[EMAIL PROTECTED]>

Re: [sqlite] Porting...

[Jay Sprenkle]

On 9/14/05, Jay Siegel <[EMAIL PROTECTED]> wrote:
> 
> The porting effort has now turned to a testing
> effort...



Don't forget there are a lot of regression test cases provided with the 
source.
 
---
The Castles of Dereth Calendar: a tour of the art and architecture of 
Asheron's Call
http://www.lulu.com/content/77264

Re: [sqlite] Porting SQLite to platform with no 64-bit integer support

[F.W.A. van Leeuwen]

Never mind, the native compiler DOES support 64-bit integers :-)

Re: [sqlite] Porting help please :-)

[Gé Weijers]

David,

It looks like LockRecord does what you'd want to get this port done. If
a single-threaded port is all you want you can probably use the logic in
 the os_win.c file.

The difference in the Win32 and Posix models are the following:

Win32: locks are held by file handles
Posix: locks are held by processes

Win32: different threads can have their own locks if they have private
file handles
Posix: all threads have to coordinate their locks, and delay closing
file handles until no more locks are held


I try to avoid Posix locks whenever I can.

Good luck,

Gé

David Pitcher wrote:

> Thanks for responding Gé
> 
> There are the usual options of buffered and unbuffered file i/o in
> AmigaOS. A given file can be opened either in a shared mode, or an
> exclusive mode by a process, but it can also be locked first and then
> opened. It cannot be locked individually for a given pthread running in
> that process.
> 
> __For coarse grain locking there is Lock( filename, mode )__
> 
> Lock( ) allows you to lock a given file by name, and you get a lock
> descriptor. That lock descriptor can either be SHARED_LOCK ( ACCESS_READ
> ) or EXCLUSIVE_LOCK ( ACCESS_WRITE ), indicating more of the intention
> of the descriptor rather than causing any limitation on the outcome of a
> Read or Write operation on any resulting file descriptor. In any case
> the lock descriptor has to be held while its in use.
> 
> You can then access the file descriptor from that lock descriptor safe
> in the knowledge that it is yours.
> 
> The problem being that it locks the whole file, rather than individual
> records/ranges inside it. This is how I currently implement it to
> prevent two different processes both running on the same database from
> overwriting each other.
> 
> It works to a fashion, but its not sophisticated like I think it should
> be - and reading your reply, it could be.
> 
> __For fine grain lock there is LockRecord( ... ) and LockRecords( ... )__
> 
> LockRecord( ... ) allows you to lock a given range to yourself ( which I
> didn't actually realise I had as an option to be honest until I went
> back to the developer documentation for the OS just now ) in either
> EXCLUSIVE or SHARED mode.
> 
> LockRecords( ... ) allows you to lock multiple ranges to yourself at
> once ( kind of like a lockset ).
> 
> Both of those have optional timeouts, and IMMED suboptions to either
> wait for a lock or return straight away if its already locked.
> 
> Now Im not totally sure whether or not the lock record option treats
> individual threads as different potential lock holders, thats something
> Ill try in a minute ( and it isn't clear from the documentation either
> ). But it certainly locks it system wide.
> 
> So following your reply, it indicates that I should follow the Win32
> implementation pattern? Then if I can confirm if threads are treated as
> individual lockholders or not I can use the AmigaOS pthread.library to
> multi-thread it. If not, it stays single threaded but at least
> potentially atomic system wide - which should be enough to make it a
> decent enough port.
> 
> Dave.

Re: [sqlite] Porting help please :-)

[David Pitcher]

Thanks for responding Gé
There are the usual options of buffered and unbuffered file i/o in AmigaOS. 
A given file can be opened either in a shared mode, or an exclusive mode by 
a process, but it can also be locked first and then opened. It cannot be 
locked individually for a given pthread running in that process.

__For coarse grain locking there is Lock( filename, mode )__
Lock( ) allows you to lock a given file by name, and you get a lock 
descriptor. That lock descriptor can either be SHARED_LOCK ( ACCESS_READ ) 
or EXCLUSIVE_LOCK ( ACCESS_WRITE ), indicating more of the intention of the 
descriptor rather than causing any limitation on the outcome of a Read or 
Write operation on any resulting file descriptor. In any case the lock 
descriptor has to be held while its in use.

You can then access the file descriptor from that lock descriptor safe in 
the knowledge that it is yours.

The problem being that it locks the whole file, rather than individual 
records/ranges inside it. This is how I currently implement it to prevent 
two different processes both running on the same database from overwriting 
each other.

It works to a fashion, but its not sophisticated like I think it should be - 
and reading your reply, it could be.

__For fine grain lock there is LockRecord( ... ) and LockRecords( ... )__
LockRecord( ... ) allows you to lock a given range to yourself ( which I 
didn't actually realise I had as an option to be honest until I went back to 
the developer documentation for the OS just now ) in either EXCLUSIVE or 
SHARED mode.

LockRecords( ... ) allows you to lock multiple ranges to yourself at once 
( kind of like a lockset ).

Both of those have optional timeouts, and IMMED suboptions to either wait 
for a lock or return straight away if its already locked.

Now Im not totally sure whether or not the lock record option treats 
individual threads as different potential lock holders, thats something Ill 
try in a minute ( and it isn't clear from the documentation either ). But it 
certainly locks it system wide.

So following your reply, it indicates that I should follow the Win32 
implementation pattern? Then if I can confirm if threads are treated as 
individual lockholders or not I can use the AmigaOS pthread.library to 
multi-thread it. If not, it stays single threaded but at least potentially 
atomic system wide - which should be enough to make it a decent enough port.

Dave. 

Re: [sqlite] Porting help please :-)

[Gé Weijers]

David Pitcher wrote:

> The claim that it works over a networked drive has made me 
> start to think that operating system specific locking mechanisms are totally 
> the wrong way to go, and that I have misunderstood the code.

OS locking mechanisms are supposed to work over a network.

>[...] or should I just sit down with a pen, ignore the calls to the
Windows API in
> the win32 one and try and work it out from the unix implementation?

The NT branch of the Win32 implementation does the same thing as the
Unix one. It does not have to work around the silly Posix lock semantics
so it's simpler. I'd start there.

Gé

Re: [sqlite] Porting help please :-)

[Gé Weijers]

Dave,

The os_unix.c locking code is extremely convoluted because of the rather
insane Posix locking semantics, which tend to make writing threaded code
excessively complicated, e.g. if two threads are accessing the same file
and one thread closes its file handle the other thread loses all its locks.

I don't know anything about AmigaOS, but if two threads can open the
same file using two file handles, and can independently lock and unlock
ranges in the file then I'd take a closer look at os_win.c to determine
the logic. The semantics are not all that hard. For Windows there are
two case:

o Pre-NT locks: you can only have write locks
o NT and later: you can have read and write locks

Sqlite implements a randomized method of simulating read locks for
Pre-NT operating systems (Win95/Win98/WinME). It's explained well in
os_win.c

An explanation on how AmigaOS locking works would be useful.

Gé


David Pitcher wrote:
> there
> 
> Ive just recently ported ( in the nastiest dirtiest way so far ) sqlite onto 
> another platform - that is AmigaOS version 4.0 ( 
> powerpc 
> ):http://www.os4depot.net/index.php?function=showfile&file=library/misc/sqlite3.tar.gz
> 
> However I had to put a health warning on it because in order to get it to 
> work in the limited spare time I have had so far I had to remove all the 
> locking code in os_amiga.c ( ripped straight from the os_unix.c file with a 
> few changes to stop a few clashes in naming of #defined' variables with OS 
> provided ones ) else any operation would just fail saying the database was 
> locked ( which yes probably is something wrong in the C runtime library 
> emulation ).
> 
> So in order to restore sane locking I wrote an AmigaOS specific version of 
> the os_amiga.c file, but using the Operating System locks built in. Now 
> looking at the windows implementation it does seem to call operating system 
> methods for locking files exclusive or shared, but specifically on a section 
> of the file. The claim that it works over a networked drive has made me 
> start to think that operating system specific locking mechanisms are totally 
> the wrong way to go, and that I have misunderstood the code.
> 
> Excuse me if this sounds terribly 'newbie'.
> 
> So, a question, should I NOT be using operating system specific locks ( 
> indeed my attempts to get it to work once I used OS functions to lock the 
> file failed to work in a sqlite friendly way ) and instead be just setting 
> various offsets in the file to certain bitpatterns to indicate to potential 
> sharers what the state is? My implementation is the only one that attempts 
> to store a lock descriptor in struct OsFile also.
> 
> I have the feeling Ive missed a porting document or something. Anybody can 
> set me straight as to whether or not I should be modifying byte offsets in 
> the file, using OS provided locking systems ( and their semantics ) or 
> should I just sit down with a pen, ignore the calls to the Windows API in 
> the win32 one and try and work it out from the unix implementation?
> 
> Regards
> 
> Dave. 
>