Writing binary data to SQLite

2020-10-22 Thread Clonk
There is apparently already a proposal by @snej for openArray[byte] / openArray[char] <-> string conversion :

Writing binary data to SQLite

2020-10-22 Thread Clonk
I think the problem that prevents db_sqlite fril handlind binary data correctly is the usage of `column_text` everywhere without checking `column_type` \- on SQLITE_BLOB it should use instead of `column_blob`; the API doesn't have to change as the result of column_blob can be stored into a strin

Writing binary data to SQLite

2020-10-21 Thread leorize
We can always extend db_sqlite :p There's ndb which is db_sqlite but with prepared statements: And @Araq's ormin is pretty cool too:

Writing binary data to SQLite

2020-10-21 Thread sky_khan
Prepared queries can be many times faster depending on use case. So, I think they should have been part of stdlib but I guess its too late for v1. Here is what I meant by "diving into sqlite3.nim" import sqlite3, db_sqlite, strutils, strformat let dbname = ":memory:" le

Writing binary data to SQLite

2020-10-21 Thread Clonk
So I've done more tests. With preparedStatement you can actually insert binary data but I haven't been able to retrieve it (same issue of cstring data getting truncated on select when using either getValue or getRow). tiny_sqlite is able to do this with the minor issue that you have store your

Writing binary data to SQLite

2020-10-20 Thread Clonk
Interesting, I'll check it out !

Writing binary data to SQLite

2020-10-20 Thread GULPF
You might want to take a look at my SQLite library, which represents blobs using `seq[byte]`:

Writing binary data to SQLite

2020-10-20 Thread sky_khan
AFAIK, you can insert blob values only by converting them to hexadecimal using sql, unless you're using prepared statements. Last time I checked db_sqlite.nim in stdlib had no support for prepared statements. You should dive into low level wrapper sqlite3.nim or find another library which suppor

Writing binary data to SQLite

2020-10-20 Thread Clonk
Thanks for your answer but I'm not sure if I understand correctly your answer :D. Copying the data into a StringStream works. y problem lies when I have a character 0 in binary data with string (I assume other character could reproduce wierd phenomenon like that such as n). Using toHex / parse

Writing binary data to SQLite

2020-10-20 Thread Clonk
I have the same problem when I use binary data obtained from msgpack. >From the SQLite perspective, I ended up with error varying results (my input >data is a pseudo-random seq of float) from a truncated string to an exception >`dbError unrecognized token`.

Writing binary data to SQLite

2020-10-20 Thread enthus1ast
something like this could work: var data: seq[float64] = @[0.1, 0.2] var strBuf = newString(data.sizeof()) copyMem(addr strBuf[0], addr data, data.sizeof()) ## .. ## Write strBuf to your sqlite blob. ## .. # get your data back var data2:

Writing binary data to SQLite

2020-10-20 Thread enthus1ast
The issue with this approach is, that it makes the sqlite database not portable between systems of different byte endianess.

Writing binary data to SQLite

2020-10-20 Thread Clonk
Hello, I have data `seq[float64]` that I wish to write in a SQLite database in a binary format. So I tried to do it this way : let db = open(dbName, "", "", "") db.exec(sql"DROP TABLE IF EXISTS test") let createTableStr = sql"""CREATE TABLE test( id INTEG