[sqlalchemy] Re: server_side_cursors
OK, ive added ticket #514 to remind me to evaluate/test the patch
out..will try to commit by tomorrow.
On Mar 18, 2007, at 3:16 PM, Steve Huffman wrote:
Here's a patch that add's a DelayedMetaResultProxy(ResultProxy) class.
In the case of fetchone and fetchmany, it overwrites itself with the
ResultProxy's version after the first call. Actually, fetchone calls
the super class's initially and simply sets the metadata before
returning. I would have done that for all of the affected functions
with a decorator, but the close() call in the others causes trouble.
On 3/17/07, Michael Bayer [EMAIL PROTECTED] wrote:
no its a psycopg thing, its like this:
# server side cursor (giving it a name makes it server side.
psycopg2 devs think thats a good API.)
cursor = connection.cursor(x)
# execute. cursor now has pending results.
cursor.execute(select * from table)
SQLAlchemy's result wrapper, ResultProxy, then calls:
metadata = cursor.metadata
in order to get information about the columns in the result set. but
*this fails*, because we are using server side cursors, and
metadata apparently does not know to fetch its info from the
server. doesnt happen on my machine. not sure if this changes due
to psycopg2 versions, PG setup, or what. if we can determine its a
psycopg2 version issue, then everyone can just upgrade.
anyway what psycopg wants you to do is:
row = cursor.fetchone()
metadata = cursor.metadata
then it works. a tweaked out ResultProxy is needed to handle this
extra complexity in this case.
if youre saying we shouldnt use metadata at all, that doesnt work
because ResultProxy is globally used for all results regardless of
their source, textual, column-based, and otherwise. also the
databases will often not give you back the same column name as what
you gave it (case conventions, etc) and in some cases we dont even
have a column name to start with (like select some_function()).
On Mar 17, 2007, at 8:41 PM, Steve Huffman wrote:
I may be missing something fundamental here, but why doesn't it
already know the metadata since I defined the columns in which I'm
interested?
thing_table = sa.Table(thing, md, sa.Column('id', sa.Integer,
primary_key = True))
On 3/17/07, Michael Bayer [EMAIL PROTECTED] wrote:
the cursor metadata often cannot be read until fetchone() is
called
first. the current result set implementation we have doesnt call
fetchone() before it tries to get the metadata, and normally it
shouldnt (since the result set doesnt even know if its the result
of a
select/insert/whatever). id like an alternate result set class
to go
into effect when PG/server side cursors/select is used to do
this, i
think someone was supposed to send a patch. its hard for me to
develop since my version of PG 8.1 doesnt seem to reproduce the
issue.
On Mar 17, 8:14 pm, [EMAIL PROTECTED]
[EMAIL PROTECTED] wrote:
I was excited to see the server_side_cursors option that was added
recently.
I saw the reports of it not working with autoload = True, but I've
been having trouble getting it to work at all.
When attempting to select a row using:
t2.select().execute().fetchone()
I get:
INFO sqlalchemy.engine.base.Engine.0x..d0 SELECT thing.id FROM
thing
INFO sqlalchemy.engine.base.Engine.0x..d0 {}
Traceback (most recent call last): File stdin, line 1, in ?
File build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py,
line
811, in __repr__
File build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py,
line
671, in _get_col
File build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py,
line
659, in _convert_key
sqlalchemy.exceptions.NoSuchColumnError: Could not locate
column in
row for column '0'
This query runs fine without server_side_cursors = True
Any suggestions?
postgres_delay_metadata.patch
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---
[sqlalchemy] Re: server_side_cursors
the cursor metadata often cannot be read until fetchone() is called
first. the current result set implementation we have doesnt call
fetchone() before it tries to get the metadata, and normally it
shouldnt (since the result set doesnt even know if its the result of a
select/insert/whatever). id like an alternate result set class to go
into effect when PG/server side cursors/select is used to do this, i
think someone was supposed to send a patch. its hard for me to
develop since my version of PG 8.1 doesnt seem to reproduce the issue.
On Mar 17, 8:14 pm, [EMAIL PROTECTED]
[EMAIL PROTECTED] wrote:
I was excited to see the server_side_cursors option that was added
recently.
I saw the reports of it not working with autoload = True, but I've
been having trouble getting it to work at all.
When attempting to select a row using:
t2.select().execute().fetchone()
I get:
INFO sqlalchemy.engine.base.Engine.0x..d0 SELECT thing.id FROM thing
INFO sqlalchemy.engine.base.Engine.0x..d0 {}
Traceback (most recent call last): File stdin, line 1, in ?
File build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py, line
811, in __repr__
File build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py, line
671, in _get_col
File build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py, line
659, in _convert_key
sqlalchemy.exceptions.NoSuchColumnError: Could not locate column in
row for column '0'
This query runs fine without server_side_cursors = True
Any suggestions?
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---
[sqlalchemy] Re: server_side_cursors
I may be missing something fundamental here, but why doesn't it
already know the metadata since I defined the columns in which I'm
interested?
thing_table = sa.Table(thing, md, sa.Column('id', sa.Integer,
primary_key = True))
On 3/17/07, Michael Bayer [EMAIL PROTECTED] wrote:
the cursor metadata often cannot be read until fetchone() is called
first. the current result set implementation we have doesnt call
fetchone() before it tries to get the metadata, and normally it
shouldnt (since the result set doesnt even know if its the result of a
select/insert/whatever). id like an alternate result set class to go
into effect when PG/server side cursors/select is used to do this, i
think someone was supposed to send a patch. its hard for me to
develop since my version of PG 8.1 doesnt seem to reproduce the issue.
On Mar 17, 8:14 pm, [EMAIL PROTECTED]
[EMAIL PROTECTED] wrote:
I was excited to see the server_side_cursors option that was added
recently.
I saw the reports of it not working with autoload = True, but I've
been having trouble getting it to work at all.
When attempting to select a row using:
t2.select().execute().fetchone()
I get:
INFO sqlalchemy.engine.base.Engine.0x..d0 SELECT thing.id FROM thing
INFO sqlalchemy.engine.base.Engine.0x..d0 {}
Traceback (most recent call last): File stdin, line 1, in ?
File build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py, line
811, in __repr__
File build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py, line
671, in _get_col
File build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py, line
659, in _convert_key
sqlalchemy.exceptions.NoSuchColumnError: Could not locate column in
row for column '0'
This query runs fine without server_side_cursors = True
Any suggestions?
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---
[sqlalchemy] Re: server_side_cursors
no its a psycopg thing, its like this:
# server side cursor (giving it a name makes it server side.
psycopg2 devs think thats a good API.)
cursor = connection.cursor(x)
# execute. cursor now has pending results.
cursor.execute(select * from table)
SQLAlchemy's result wrapper, ResultProxy, then calls:
metadata = cursor.metadata
in order to get information about the columns in the result set. but
*this fails*, because we are using server side cursors, and
metadata apparently does not know to fetch its info from the
server. doesnt happen on my machine. not sure if this changes due
to psycopg2 versions, PG setup, or what. if we can determine its a
psycopg2 version issue, then everyone can just upgrade.
anyway what psycopg wants you to do is:
row = cursor.fetchone()
metadata = cursor.metadata
then it works. a tweaked out ResultProxy is needed to handle this
extra complexity in this case.
if youre saying we shouldnt use metadata at all, that doesnt work
because ResultProxy is globally used for all results regardless of
their source, textual, column-based, and otherwise. also the
databases will often not give you back the same column name as what
you gave it (case conventions, etc) and in some cases we dont even
have a column name to start with (like select some_function()).
On Mar 17, 2007, at 8:41 PM, Steve Huffman wrote:
I may be missing something fundamental here, but why doesn't it
already know the metadata since I defined the columns in which I'm
interested?
thing_table = sa.Table(thing, md, sa.Column('id', sa.Integer,
primary_key = True))
On 3/17/07, Michael Bayer [EMAIL PROTECTED] wrote:
the cursor metadata often cannot be read until fetchone() is called
first. the current result set implementation we have doesnt call
fetchone() before it tries to get the metadata, and normally it
shouldnt (since the result set doesnt even know if its the result
of a
select/insert/whatever). id like an alternate result set class
to go
into effect when PG/server side cursors/select is used to do this, i
think someone was supposed to send a patch. its hard for me to
develop since my version of PG 8.1 doesnt seem to reproduce the
issue.
On Mar 17, 8:14 pm, [EMAIL PROTECTED]
[EMAIL PROTECTED] wrote:
I was excited to see the server_side_cursors option that was added
recently.
I saw the reports of it not working with autoload = True, but I've
been having trouble getting it to work at all.
When attempting to select a row using:
t2.select().execute().fetchone()
I get:
INFO sqlalchemy.engine.base.Engine.0x..d0 SELECT thing.id FROM thing
INFO sqlalchemy.engine.base.Engine.0x..d0 {}
Traceback (most recent call last): File stdin, line 1, in ?
File build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py,
line
811, in __repr__
File build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py,
line
671, in _get_col
File build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py,
line
659, in _convert_key
sqlalchemy.exceptions.NoSuchColumnError: Could not locate column in
row for column '0'
This query runs fine without server_side_cursors = True
Any suggestions?
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---
[sqlalchemy] Re: server_side_cursors
sorry, i meant cursor.description, not cursor.metadata. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: server_side_cursors
SQLAlchemy's result wrapper, ResultProxy, then calls:
metadata = cursor.metadata
My question was why doesn't ResultProxy use the sqlalchemy metadata I
defined when I defined the sqlalchemy Table?
to psycopg2 versions, PG setup, or what. if we can determine its a
psycopg2 version issue, then everyone can just upgrade.
which version are you using?
On Mar 17, 2007, at 8:41 PM, Steve Huffman wrote:
I may be missing something fundamental here, but why doesn't it
already know the metadata since I defined the columns in which I'm
interested?
thing_table = sa.Table(thing, md, sa.Column('id', sa.Integer,
primary_key = True))
On 3/17/07, Michael Bayer [EMAIL PROTECTED] wrote:
the cursor metadata often cannot be read until fetchone() is called
first. the current result set implementation we have doesnt call
fetchone() before it tries to get the metadata, and normally it
shouldnt (since the result set doesnt even know if its the result
of a
select/insert/whatever). id like an alternate result set class
to go
into effect when PG/server side cursors/select is used to do this, i
think someone was supposed to send a patch. its hard for me to
develop since my version of PG 8.1 doesnt seem to reproduce the
issue.
On Mar 17, 8:14 pm, [EMAIL PROTECTED]
[EMAIL PROTECTED] wrote:
I was excited to see the server_side_cursors option that was added
recently.
I saw the reports of it not working with autoload = True, but I've
been having trouble getting it to work at all.
When attempting to select a row using:
t2.select().execute().fetchone()
I get:
INFO sqlalchemy.engine.base.Engine.0x..d0 SELECT thing.id FROM thing
INFO sqlalchemy.engine.base.Engine.0x..d0 {}
Traceback (most recent call last): File stdin, line 1, in ?
File build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py,
line
811, in __repr__
File build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py,
line
671, in _get_col
File build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py,
line
659, in _convert_key
sqlalchemy.exceptions.NoSuchColumnError: Could not locate column in
row for column '0'
This query runs fine without server_side_cursors = True
Any suggestions?
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---
