Re: [sqlite] Loading a database from memory

[Glenn McCord]

On Wed, Jul 20, 2011 at 3:41 AM, st@gz <stephen@gmail.com> wrote:
> Implement sqlite-vfs with pure memory block.
>
>
> spmemvfs-0.3.tar.gz
>
>
>
> http://code.google.com/p/sphivedb/downloads/list

What is the license of spmemvfs? There is nothing stating it in the
source code, but the website http://code.google.com/p/sphivedb/ seems
to imply GPL v2, unless that is only referring to sphivedb.

Thanks.


>
>
>
> -- Original --
> From:  "Simon Slavin"<slav...@bigfraud.org>;
> Date:  Tue, Jul 19, 2011 10:10 PM
> To:  "General Discussion of SQLite Database"<sqlite-users@sqlite.org>;
>
> Subject:  Re: [sqlite] Loading a database from memory
>
>
>
> On 19 Jul 2011, at 3:04pm, Pavel Ivanov wrote:
>
>>> So you can copy any block of memory you have a handle for into that, use 
>>> SQLite to manipulate the data while it's in memory
>>
>> Simon, could you elaborate what you meant by that. To my knowledge you
>> can't just copy any block of memory into SQLite and make it treat this
>> memory block as database. Did you meant something else?
>
> Apparently I misremembered.  I thought you could use the sqlite3 handle and 
> find a pointer to the start of a block of memory the data was stored in.  But 
> now I see no way to do this.  Sorry about the misinformation.
>
> Simon.
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Loading a database from memory

[st@gz]

Implement sqlite-vfs with pure memory block.


spmemvfs-0.3.tar.gz  



http://code.google.com/p/sphivedb/downloads/list


 
-- Original --
From:  "Simon Slavin"<slav...@bigfraud.org>;
Date:  Tue, Jul 19, 2011 10:10 PM
To:  "General Discussion of SQLite Database"<sqlite-users@sqlite.org>; 

Subject:  Re: [sqlite] Loading a database from memory

 

On 19 Jul 2011, at 3:04pm, Pavel Ivanov wrote:

>> So you can copy any block of memory you have a handle for into that, use 
>> SQLite to manipulate the data while it's in memory
> 
> Simon, could you elaborate what you meant by that. To my knowledge you
> can't just copy any block of memory into SQLite and make it treat this
> memory block as database. Did you meant something else?

Apparently I misremembered.  I thought you could use the sqlite3 handle and 
find a pointer to the start of a block of memory the data was stored in.  But 
now I see no way to do this.  Sorry about the misinformation.

Simon.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Loading a database from memory

[Simon Slavin]

On 19 Jul 2011, at 3:04pm, Pavel Ivanov wrote:

>> So you can copy any block of memory you have a handle for into that, use 
>> SQLite to manipulate the data while it's in memory
> 
> Simon, could you elaborate what you meant by that. To my knowledge you
> can't just copy any block of memory into SQLite and make it treat this
> memory block as database. Did you meant something else?

Apparently I misremembered.  I thought you could use the sqlite3 handle and 
find a pointer to the start of a block of memory the data was stored in.  But 
now I see no way to do this.  Sorry about the misinformation.

Simon.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Loading a database from memory

[Pavel Ivanov]

> So you can copy any block of memory you have a handle for into that, use 
> SQLite to manipulate the data while it's in memory

Simon, could you elaborate what you meant by that. To my knowledge you
can't just copy any block of memory into SQLite and make it treat this
memory block as database. Did you meant something else?


Pavel


On Tue, Jul 19, 2011 at 9:54 AM, Simon Slavin  wrote:
>
> On 19 Jul 2011, at 5:55am, Glenn McCord wrote:
>
>> I've just had a quick look at the backup API, and it seems to make use
>> of filenames and sqlite databases, which is not exactly what I'm
>> after.
>
> SQLite has a pseudo-filename of ':memory:' which refers to a database kept 
> entirely in memory.  So you can copy any block of memory you have a handle 
> for into that, use SQLite to manipulate the data while it's in memory, then 
> use the backupAPI again to copy the results back to some memory you then 
> write to your MPEG4 file.
>
> But I think in the meantime others in the thread have come up with better 
> suggestions for you.
>
> Simon.
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Loading a database from memory

[Simon Slavin]

On 19 Jul 2011, at 5:55am, Glenn McCord wrote:

> I've just had a quick look at the backup API, and it seems to make use
> of filenames and sqlite databases, which is not exactly what I'm
> after.

SQLite has a pseudo-filename of ':memory:' which refers to a database kept 
entirely in memory.  So you can copy any block of memory you have a handle for 
into that, use SQLite to manipulate the data while it's in memory, then use the 
backupAPI again to copy the results back to some memory you then write to your 
MPEG4 file.

But I think in the meantime others in the thread have come up with better 
suggestions for you.

Simon.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Loading a database from memory

[Pavel Ivanov]

> I'm starting to get the impression that the only way for me to do this
> is to make my own vfs layer...
> http://stackoverflow.com/questions/3839158/using-sqlite-with-stdiostream

Yes, this is the best way you can do it. Other option is to take that
data, save it to some temporary file and then open that file as a
database.

> Would sqlite's test_onefile.c example code be the most appropriate
> starting point? http://www.sqlite.org/src/doc/trunk/src/test_onefile.c

I didn't look into implementation details there but generally it looks
like a good starting point.


Pavel


On Tue, Jul 19, 2011 at 12:17 AM, Glenn McCord  wrote:
> Hi. I've been trying to find out the best way of loading an sqlite
> database from memory. Basically I'd like to be able to save a small
> sqlite database as meta data to an audio file, i.e. an mpeg4
> container. I'd then be able read, modify then write it back to the
> file.
>
> I'm starting to get the impression that the only way for me to do this
> is to make my own vfs layer...
> http://stackoverflow.com/questions/3839158/using-sqlite-with-stdiostream
>
> Would sqlite's test_onefile.c example code be the most appropriate
> starting point? http://www.sqlite.org/src/doc/trunk/src/test_onefile.c
>
> If someone could offer ways of achieving this, or could at least
> clarify that a vfs is the only way to go, then I'd appreciate the
> feedback.
>
> Thanks.
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Loading a database from memory

[Giuseppe Luciano]

> What I need is the ability to read in an array of bytes, (read from
> meta data of an mpeg4, say, via fstream or some other equivalent),
> that represents the raw data of an sqlite database.
> 
> I'm trying to avoid having to parse in that data, save it to a
> separate file first, then load it using a mysql open() call.
> 
> I'll admit that I only had a quick read, so may have missed something.
> 
> Thanks

Is not easier save the db at the end of mp4 file? I don't know the mp4 file 
format, but I beleave that thare's a field that says the size of file, so 
db_size = filesize-mp4_filesize and db_start_offset=mp4_filesize.
  
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Loading a database from memory

[Glenn McCord]

On Tue, Jul 19, 2011 at 4:39 PM, Glenn McCord  wrote:
> On Tue, Jul 19, 2011 at 4:31 PM, Simon Slavin  wrote:
>>
>> On 19 Jul 2011, at 5:17am, Glenn McCord wrote:
>>
>>> Hi. I've been trying to find out the best way of loading an sqlite
>>> database from memory. Basically I'd like to be able to save a small
>>> sqlite database as meta data to an audio file, i.e. an mpeg4
>>> container. I'd then be able read, modify then write it back to the
>>> file.
>>
>> You can use the SQLite backup API to transfer an entire database from or to 
>> memory:
>>
>> http://www.sqlite.org/backup.html
>>

I've just had a quick look at the backup API, and it seems to make use
of filenames and sqlite databases, which is not exactly what I'm
after.
What I need is the ability to read in an array of bytes, (read from
meta data of an mpeg4, say, via fstream or some other equivalent),
that represents the raw data of an sqlite database.

I'm trying to avoid having to parse in that data, save it to a
separate file first, then load it using a mysql open() call.

I'll admit that I only had a quick read, so may have missed something.

Thanks


>> However, I'm not sure of the wisdom of saving data in this way.  The SQLite 
>> file format is not simple or easy to inspect, and uses up a lot of space 
>> because it has to hold indexing information.  Would it not make more sense 
>> to encode your data as XML or JSON and save this (or a .zipped version of 
>> this) in your MPEG4 instead ?
>>
>
> I was thinking the same thing, but some of the guys around the office
> are pretty keen on the idea of using sqlite. I'm just doing the
> required research and weighing up the options. Using JSON seems like a
> good idea.
>
> Thanks for the link.
>
>> Simon.
>> ___
>> sqlite-users mailing list
>> sqlite-users@sqlite.org
>> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>>
>
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Loading a database from memory

[Glenn McCord]

On Tue, Jul 19, 2011 at 4:31 PM, Simon Slavin  wrote:
>
> On 19 Jul 2011, at 5:17am, Glenn McCord wrote:
>
>> Hi. I've been trying to find out the best way of loading an sqlite
>> database from memory. Basically I'd like to be able to save a small
>> sqlite database as meta data to an audio file, i.e. an mpeg4
>> container. I'd then be able read, modify then write it back to the
>> file.
>
> You can use the SQLite backup API to transfer an entire database from or to 
> memory:
>
> http://www.sqlite.org/backup.html
>
> However, I'm not sure of the wisdom of saving data in this way.  The SQLite 
> file format is not simple or easy to inspect, and uses up a lot of space 
> because it has to hold indexing information.  Would it not make more sense to 
> encode your data as XML or JSON and save this (or a .zipped version of this) 
> in your MPEG4 instead ?
>

I was thinking the same thing, but some of the guys around the office
are pretty keen on the idea of using sqlite. I'm just doing the
required research and weighing up the options. Using JSON seems like a
good idea.

Thanks for the link.

> Simon.
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Loading a database from memory

[Simon Slavin]

On 19 Jul 2011, at 5:17am, Glenn McCord wrote:

> Hi. I've been trying to find out the best way of loading an sqlite
> database from memory. Basically I'd like to be able to save a small
> sqlite database as meta data to an audio file, i.e. an mpeg4
> container. I'd then be able read, modify then write it back to the
> file.

You can use the SQLite backup API to transfer an entire database from or to 
memory:

http://www.sqlite.org/backup.html

However, I'm not sure of the wisdom of saving data in this way.  The SQLite 
file format is not simple or easy to inspect, and uses up a lot of space 
because it has to hold indexing information.  Would it not make more sense to 
encode your data as XML or JSON and save this (or a .zipped version of this) in 
your MPEG4 instead ?

Simon.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users