Re: [sqlite] NUL handling bugs (was Re: c-api document suggestion)

2011-09-26 Thread David Garfield
Yes, but an API does not have to permit memory areas that large, so can restrict itself to int, if it chooses. It isn't necessarily a good choice, but it isn't necessarily wrong either. Especially if int is properly defined (which, of course, it isn't in one of your examples). A simple example

Re: [sqlite] NUL handling bugs (was Re: c-api document suggestion)

2011-09-26 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 09/26/2011 03:31 PM, David Garfield wrote: > And I suppose the alternative to fixing them in SQLite is to > document them. Ticket created where the team can give their official response and people can be pointed the next time this comes up:

Re: [sqlite] NUL handling bugs (was Re: c-api document suggestion)

2011-09-26 Thread Jan Hudec
On Mon, Sep 26, 2011 at 12:44:08 -0400, David Garfield wrote: > Using int instead of size_t A) unnaturally limits data length, > possibly to 64K and B) doesn't well describe the data. Beyond that, > it isn't necessarily a bug. The language needed size_t to provide a > type guaranteed to be

Re: [sqlite] NUL handling bugs (was Re: c-api document suggestion)

2011-09-26 Thread David Garfield
Yes, I am referring to functions in the SQL, not functions in the API. And I suppose the alternative to fixing them in SQLite is to document them. I suppose if it were to be fixed, it could break people depending on the bad behavior. On the other hand, the existing behavior means that some

Re: [sqlite] NUL handling bugs (was Re: c-api document suggestion)

2011-09-26 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 09/26/2011 01:13 PM, David Garfield wrote: > In documented one case > (prepare), NULs are string terminators even with the count, and there > is an extra optimization you may apply. All this works (I think). Agreed, ugly but documented. BTW in

Re: [sqlite] NUL handling bugs (was Re: c-api document suggestion)

2011-09-26 Thread David Garfield
OK Roger, lets start over. The API has functions that deal with strings. They take either a NUL terminated buffer with a length specified as negative, or a buffer and length in which NULs are not special. In documented one case (prepare), NULs are string terminators even with the count, and

Re: [sqlite] NUL handling bugs (was Re: c-api document suggestion)

2011-09-26 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 09/26/2011 09:44 AM, David Garfield wrote: > As the implementer of an API, you must use one form or the other, or > hideously duplicate code. That is philosophy. > It is TRIVIAL to convert from the > NUL-terminated form to the counted form, so

Re: [sqlite] NUL handling bugs (was Re: c-api document suggestion)

2011-09-26 Thread David Garfield
Roger Binns writes: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > On 09/23/2011 05:51 PM, David Garfield wrote: > >> SQLite's API supports both (mostly). Internally, you must use one or > >> the other (or hideously duplicate code), > > Not really. If your own code only uses NUL

Re: [sqlite] NUL handling bugs (was Re: c-api document suggestion)

2011-09-24 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 09/23/2011 05:51 PM, David Garfield wrote: >> SQLite's API supports both (mostly). Internally, you must use one or >> the other (or hideously duplicate code), Not really. If your own code only uses NUL termination then use that form of APIs. If

Re: [sqlite] NUL handling bugs (was Re: c-api document suggestion)

2011-09-23 Thread David Garfield
Roger Binns writes: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > On 09/23/2011 08:47 AM, David Garfield wrote: > > But that is the point. Strings are generally defined in two ways. > > Either: > > > > 1) a pointer, and count every byte up to but not including a NUL. > > > > 2) a

Re: [sqlite] NUL handling bugs (was Re: c-api document suggestion)

2011-09-23 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 09/23/2011 08:47 AM, David Garfield wrote: > But that is the point. Strings are generally defined in two ways. > Either: > > 1) a pointer, and count every byte up to but not including a NUL. > > 2) a pointer and a length, and count every byte in

Re: [sqlite] NUL handling bugs (was Re: c-api document suggestion)

2011-09-23 Thread David Garfield
But that is the point. Strings are generally defined in two ways. Either: 1) a pointer, and count every byte up to but not including a NUL. 2) a pointer and a length, and count every byte in the specified length. If you have a specified length, the length matters, and NULs do not. NUL is a

Re: [sqlite] NUL handling bugs (was Re: c-api document suggestion)

2011-09-23 Thread Richard Hipp
On Thu, Sep 22, 2011 at 7:53 PM, David Garfield < garfi...@irving.iisd.sra.com> wrote: > As far as I am concerned, this is a SERIOUS bug in sqlite. SQLite does whatever you ask it to do. It makes no attempt to enforce good string hygiene. If you hand it well-formed strings, it gives the

Re: [sqlite] NUL handling bugs (was Re: c-api document suggestion)

2011-09-22 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 09/22/2011 04:53 PM, David Garfield wrote: > As far as I am concerned, this is a SERIOUS bug in sqlite. It gets worse. Many of those string functions will also happily operate on blobs where it makes no sense. (A bunch of bytes without an

[sqlite] NUL handling bugs (was Re: c-api document suggestion)

2011-09-22 Thread David Garfield
As far as I am concerned, this is a SERIOUS bug in sqlite. When you have counted strings, which is all sqlite has (except for some API functions), NULs have no special meaning. This is fairly easily seen in the command line interface, where the code ignores the length and uses strlen forms.