This is really cool, thanks for sharing! I wonder though, is the compression done per field? I read the source but I couldn't figure it out quickly (not really used to the sqlite codebase). What are the compression ratios you achieve?
Wout. On Mon, Sep 24, 2018 at 3:58 PM Domingo Alvarez Duarte <mingo...@gmail.com> wrote: > Hello ! > > After looking at how compression is implemented in fts3 and wanting the > same for fts5 I managed to get a working implementation that I'm sharing > here with the same license as sqlite3 in hope it can be useful to others > and maybe be added to sqlite3. > > Cheers ! > > > Here is on implementation of optional compression and min_word_size for > columns in fts5: > > === > > create virtual table if not exists docs_fts using fts5( > doc_fname unindexed, doc_data compressed, > compress=compress, uncompress=uncompress, > tokenize = 'unicode61 min_word_size=3' > ); > > === > > https://gist.github.com/mingodad/7fdec8eebdde70ee388db60855760c72 > > > And here is an implementation of optional compression for columns in > fts3/4: > > === > > create virtual table if not exists docs_fts using fts4( > doc_fname, doc_data, > tokenize = 'unicode61', > notindexed=doc_fname, notcompressed=doc_fname, > compress=compress, uncompress=uncompress > ); > > === > > https://gist.github.com/mingodad/2f05cd1280d58f93f89133b2a2011a4d > > _______________________________________________ > sqlite-users mailing list > sqlite-users@mailinglists.sqlite.org > http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users > _______________________________________________ sqlite-users mailing list sqlite-users@mailinglists.sqlite.org http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
