Re: [sqlite] Feature request: hash function for fast indices on text and blob fields

[Roger Binns]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 03/06/2011 05:22 AM, Alexey Pechnikov wrote:
> I think one or more hash functions may be
> included into core SQLite.

(Argggh.  You posted the same message to multiple mailing lists.  That is
annoying.)

Why can't you use the SQLite API in order to register the functions for your
databases?

Every now and then people request that some favourite function special to
them be part of the core.  Rarely does anyone show that it would be broadly
useful to the however many billion instances of SQLite around the world.
And even rarer still does anyone show how the existing SQLite mechanisms to
add functions are problematic.

For something to be added to the core it means the code has to be compatible
with the SQLite license (public domain, on file contributor agreement), has
to be implemented by the SQLite team (fit in with the existing coding style,
error handling, test infrastructure), has to be included in the testing
(remember there is 100% MCDC coverage) and has to be maintained in SQLite 3
until it is end of lifed.  This is a lot of work to inflict on the team.

The murmur hash is good example of why it would be a bad idea to include the
SQLite core.  There are many different versions (1, 2, 2A, 2-32, 2-64, 3)
and it currently works on binary blobs.  To use it for strings would require
defining encoding and byte ordering.

You can create an extension that includes a whole bunch of hash functions.
Make it available to the world by getting it listed at the end of
http://sqlite.org/contrib where you can see a good 'extension-functions.c'
to follow.  If the team see the extension being downloaded a lot then that
is a good case for migrating it into the core.

Roger
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk1z0UUACgkQmOOfHg372QRTcwCguCl0ZZCDnYHSdK9hyRucdEE2
aiEAoMRGLPH8RCuD/mOnW2levNbr/QyF
=D5kz
-END PGP SIGNATURE-
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Feature request: hash function for fast indices on text and blob fields

[Alexey Pechnikov]

Yes, I have my extension for this
http://sqlite.mobigroup.ru/dir?name=ext/murmurhash

But I think the feature will be very helpful as core function.

2011/3/6 Enrico Thierbach :
>
> On 06.03.2011, at 14:22, Alexey Pechnikov wrote:
>
>> Sqlite does not has ability to compress indices. There is huge
>> performance problem with populating a database with indexed
>> text or blob fields. But we can emulate indices compression
>> by using index on hashed field (and populate the hash field by
>> trigger as example). I think one or more hash functions may be
>> included into core SQLite.
>>
>> Note: I use the public domain licensed murmur hash algorithm
>> (as extension) but this need custom build of SQLite or external
>> extension.
>>
>> Does anybody has the same problem?
>>
>
> Hey Alexey,
>
> couldn't you just define any hash function you like using SQLite's interface 
> as outlined here?
>
> http://www.sqlite.org/c3ref/create_function.html
>
> /eno
>
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>



-- 
Best regards, Alexey Pechnikov.
http://pechnikov.tel/
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Feature request: hash function for fast indices on text and blob fields

[Enrico Thierbach]

On 06.03.2011, at 14:22, Alexey Pechnikov wrote:

> Sqlite does not has ability to compress indices. There is huge
> performance problem with populating a database with indexed
> text or blob fields. But we can emulate indices compression
> by using index on hashed field (and populate the hash field by
> trigger as example). I think one or more hash functions may be
> included into core SQLite.
> 
> Note: I use the public domain licensed murmur hash algorithm
> (as extension) but this need custom build of SQLite or external
> extension.
> 
> Does anybody has the same problem?
> 

Hey Alexey,

couldn't you just define any hash function you like using SQLite's interface as 
outlined here?

http://www.sqlite.org/c3ref/create_function.html

/eno

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] Feature request: hash function for fast indices on text and blob fields

[Alexey Pechnikov]

Sqlite does not has ability to compress indices. There is huge
performance problem with populating a database with indexed
text or blob fields. But we can emulate indices compression
by using index on hashed field (and populate the hash field by
trigger as example). I think one or more hash functions may be
included into core SQLite.

Note: I use the public domain licensed murmur hash algorithm
(as extension) but this need custom build of SQLite or external
extension.

Does anybody has the same problem?

-- 
Best regards, Alexey Pechnikov.
http://pechnikov.tel/
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users