Well stream_results is only valid with a SELECT statement. To use PG server
side cursors as much as possible, what you need to do is do create_engine()
with server_side_cursors=True which will take effect for any SELECT statements,
but not any CREATE/DROP/INSERT/UPDATE/DELETE. This is a PG specific feature.
Also the bug seems like it might be limited to the fetchall() and fetchmany()
methods. If I get rows back with fetchone() it appears things might be OK but
am still testing.
On Apr 29, 2013, at 12:59 PM, Richard Gerd Kuesters <rich...@humantech.com.br>
wrote:
> Well, just to add: if I run my application (using psycopg2) with
> stream_results=True, I get some weird errors also; even on create_all() or
> drop_all() commands, which I think are pretty basic. Or is it mixing the
> commands and not executing in order?
>
>
> On 04/29/2013 01:55 PM, Michael Bayer wrote:
>> Well seems like the "stream_results" feature must be extremely unpopular
>> because I can confirm this behavior even using connection-local execution
>> options, in 0.7 also. This feature is tested so will have to see exactly
>> what's up with those tests.
>>
>>
>> On Apr 29, 2013, at 9:49 AM, Pau Tallada <tall...@pic.es> wrote:
>>
>>> Hi,
>>>
>>> I have some very big datasets that we want to process in batches of
>>> reasonable size.
>>> In order to do that, we enable the 'stream_results' execution option
>>> (available in Postgresql) and we use the fetchmany method to retrieve the
>>> records in batches.
>>> With those records we must build a numpy array, but its constructor
>>> complains about the RowProxy wrapper.
>>>
>>> So I tried using the cursor directly to retrieve the selected rows, but the
>>> cursor skips the first one if 'stream_results' is enabled.
>>>
>>> Let me explain with an example:
>>>
>>> engine = create_engine(db_url, execution_options = {'stream_results' :
>>> True})
>>>
>>> rs = engine.execute("SELECT * FROM table")
>>> print len(rs.fetchall())
>>>
>>> rs = engine.execute("SELECT * FROM table")
>>> print len(rs.cursor.fetchall())
>>>
>>> The first one displays N, while the second one displays N-1.
>>>
>>> Any clues? :/
>>>
>>> Thank you in advance!
>>>
>>> Pau.
>>>
>>> --
>>> ----------------------------------
>>> Pau Tallada Crespí
>>> Dep. d'Astrofísica i Cosmologia
>>> Port d'Informació Científica (PIC)
>>> Tel: +34 93 586 8233
>>> ----------------------------------
>>>
>>>
>>> --
>>> You received this message because you are subscribed to the Google Groups
>>> "sqlalchemy" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an
>>> email to sqlalchemy+unsubscr...@googlegroups.com.
>>> To post to this group, send email to sqlalchemy@googlegroups.com.
>>> Visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
>>> For more options, visit https://groups.google.com/groups/opt_out.
>>>
>>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "sqlalchemy" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to sqlalchemy+unsubscr...@googlegroups.com.
>> To post to this group, send email to sqlalchemy@googlegroups.com.
>> Visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>>
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "sqlalchemy" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sqlalchemy+unsubscr...@googlegroups.com.
> To post to this group, send email to sqlalchemy@googlegroups.com.
> Visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
--
You received this message because you are subscribed to the Google Groups
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to sqlalchemy+unsubscr...@googlegroups.com.
To post to this group, send email to sqlalchemy@googlegroups.com.
Visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
