Re: [sqlite] (Info) Shox96 Compression as SQLite UDF

[Jens Alfke]

> On Feb 28, 2019, at 3:14 PM, Wout Mertens  wrote:
> 
> It seems to me that it could be worthwhile to do transparent compression on
> JSON strings, perhaps using a differently trained dictionary…

Regular LZ/zip type compression already does a good job on JSON, even fairly 
short bits. Not just because of repeated key strings, but because there are a 
number of very common byte sequences like
},{":"","{{}}"}"},"},"
(those are literal quotes)

Shox96 is an interesting idea, but it seems like a real disadvantage that it 
only compresses alphanumeric characters, fails on non-ASCII, and is biased 
toward English.

—Jens
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] (Info) Shox96 Compression as SQLite UDF

[Wout Mertens]

Wonderful!

Things I wonder:

* would it be possible to set up columns in such a way that the compression
is transparent, that is, existing queries remain unchanged?
* How does it fare on JSON strings? I notice that double-quotes are short,
but array and object delimiters are 11/12 bits?

It seems to me that it could be worthwhile to do transparent compression on
JSON strings, perhaps using a differently trained dictionary…

Wout.


On Wed, Feb 27, 2019 at 12:26 PM Arun - Siara Logics (cc) 
wrote:

> Shox96 is a compression technique for Short Strings. It can achieve upto
> 65% compression. This technique is available for compressing text columns
> in SQLite as loadable extension in the repository
> https://github.com/siara-cc/Shox96_Sqlite_UDF.
>
> Output screenshot:
> https://github.com/siara-cc/Shox96_Sqlite_UDF/blob/master/output.png?raw=true
> To find out more about Shox96 click: https://github.com/siara-cc/Shox96
> To find out how Shox96 works click:
> https://github.com/siara-cc/Shox96/blob/master/Shox96_Article_0_2_0.pdf?raw=true
>
> P.S.: The compressor and decompressor are built for short strings using
> less memory suitable for constrained environments such as Arduino Uno and
> ESP8266. So may not be as fast as Zip or GZip.
>
> Related projects:
> Sqlite3 Library for ESP32
> https://github.com/siara-cc/esp32_arduino_sqlite3_lib
> Sqlite3 Library for ESP8266
> https://github.com/siara-cc/esp_arduino_sqlite3_lib
> Sqlite3 Library for ESP-IDF
> https://github.com/siara-cc/esp32-idf-sqlite3
> Storing compressed Shox96 text content in Arduino Flash Memory
> https://github.com/siara-cc/Shox96_Arduino_Progmem_lib
> Shox96 Compression Library for Arduino
> https://github.com/siara-cc/Shox96_Arduino_lib
>
> ___
> sqlite-users mailing list
> sqlite-users@mailinglists.sqlite.org
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

RE: [sqlite] info

[Reid Thompson]

Reid Thompson wrote:
> info

apologies - i was trying to get the mailing list to give me a listing of
available commands.  Trying to suspend my subscription while out of the
office.

reid

Re: [sqlite] Info for DEFERRED, IMMEDIATE, and EXCLUSIVE transactions

[Tito Ciuro]

Thanks a lot Dr. Hipp.
Regards,
-- Tito
On 19 oct 2004, at 13:51, D. Richard Hipp wrote:
Tito Ciuro wrote:
Version 3.0.8 adds support for DEFERRED, IMMEDIATE, and EXCLUSIVE 
transactions. Where can I find more info about that?
http://www.sqlite.org/lang.html#transaction

Re: [sqlite] Info for DEFERRED, IMMEDIATE, and EXCLUSIVE transactions

[D. Richard Hipp]

Tito Ciuro wrote:
Version 3.0.8 adds support for DEFERRED, IMMEDIATE, and EXCLUSIVE 
transactions. Where can I find more info about that?

http://www.sqlite.org/lang.html#transaction
--
D. Richard Hipp -- [EMAIL PROTECTED] -- 704.948.4565