On Jul 1, 2010, at 4:35 AM, Samuel GARCIA wrote:
Hi list,
I am new user of sqlalchemy. I play with python and MySQL for several years
now and I discover sqlalchemy 1 year ago, it change my life.
So thank you for this fantastic work.
I develop a project of analysing big experimental dataset of intra/extra
cellular recordings : OpenElectrophy.
I highly use database for this, schema are really simple but table and row
are really heavy because I use BLOB to store big raw data signal.
I usually play with 1Go to 300Go databases.
Playing with SQL is very util but problems are coming because signals get
bigger and bigger.
After a walk on google, a solution could come with streaming Binary field.
In short to be able to take a field chunk by chunk.
PBXT engine for mysql seems to umplement it: http://www.blobstreaming.org/
Postgree SQL seems to natively support sort of streaming.
I don't known for SQLite.
It seems that there is not an official way to stream big binary fields in
SQL world. Correct me if I am wrong I am a newbie in this fields.
At the end my questions :
1 - Does sqlalchemy plan to implement a unified layer for playing with
stream-able field ?
I really can't figure out the amount of work to achieve that, sorry if it is
naive.
2 - Could you give me somes tips to deal with streaming with theses backend :
MySQL, SQLite, postgree
its not something DBAPI supports, and to my knowledge only cx_oracle has an
actual streamable blob type - im not aware of one for MySQL, PG, etc. So
there are no plans for cross-DBAPI support of streaming blobs in SQLAlchemy
right now since DBAPI doesn't provide it for us (we have some support for
cx_oracle's feature).My advice for now would be that storing BLOBs are
probably not a good idea for streams of data larger than a few megs - I'd use
the filesystem instead.
--
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalch...@googlegroups.com.
To unsubscribe from this group, send email to
sqlalchemy+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en.
