Well, just to add: if I run my application (using psycopg2) with
stream_results=True, I get some weird errors also; even on create_all()
or drop_all() commands, which I think are pretty basic. Or is it mixing
the commands and not executing in order?
On 04/29/2013 01:55 PM, Michael Bayer wrote:
Well seems like the "stream_results" feature must be extremely
unpopular because I can confirm this behavior even using
connection-local execution options, in 0.7 also. This feature is
tested so will have to see exactly what's up with those tests.
On Apr 29, 2013, at 9:49 AM, Pau Tallada <tall...@pic.es
<mailto:tall...@pic.es>> wrote:
Hi,
I have some very big datasets that we want to process in batches of
reasonable size.
In order to do that, we enable the 'stream_results' execution option
(available in Postgresql) and we use the fetchmany method to retrieve
the records in batches.
With those records we must build a numpy array, but its constructor
complains about the RowProxy wrapper.
So I tried using the cursor directly to retrieve the selected rows,
but the cursor skips the first one if 'stream_results' is enabled.
Let me explain with an example:
engine = create_engine(db_url, execution_options = {'stream_results'
: True})
rs = engine.execute("SELECT * FROM table")
print len(rs.fetchall())
rs = engine.execute("SELECT * FROM table")
print len(rs.cursor.fetchall())
The first one displays *N*, while the second one displays *N-1*.
Any clues? :/
Thank you in advance!
Pau.
--
----------------------------------
Pau Tallada Crespí
Dep. d'Astrofísica i Cosmologia
Port d'Informació Científica (PIC)
Tel: +34 93 586 8233
----------------------------------
--
You received this message because you are subscribed to the Google
Groups "sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to sqlalchemy+unsubscr...@googlegroups.com
<mailto:sqlalchemy+unsubscr...@googlegroups.com>.
To post to this group, send email to sqlalchemy@googlegroups.com
<mailto:sqlalchemy@googlegroups.com>.
Visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
--
You received this message because you are subscribed to the Google
Groups "sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to sqlalchemy+unsubscr...@googlegroups.com.
To post to this group, send email to sqlalchemy@googlegroups.com.
Visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
--
You received this message because you are subscribed to the Google Groups
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to sqlalchemy+unsubscr...@googlegroups.com.
To post to this group, send email to sqlalchemy@googlegroups.com.
Visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
