Re: [DB-SIG] Improved support for prepared SQL statements

2014-12-21 Thread Tony Locke
Mike, I agree. You quote: "The first time you pass a statement to the cursor's execute() method, it prepares the statement. For subsequent invocations of execute(), the preparation phase is skipped if the statement is the same.” This is basically what pg8000 does but I've found you need a couple

Re: [DB-SIG] Improved support for prepared SQL statements

2014-12-21 Thread Michael Bayer
M.-A. Lemburg wrote: > On 21.12.2014 20:56, Michael Bayer wrote: >> M.-A. Lemburg wrote: >> >>> Whether or not a cursor is client or server side depends on many >>> things and is generally database backend specific. MS SQL Server >>> for example will default to server side cursors and revert

Re: [DB-SIG] Improved support for prepared SQL statements

2014-12-21 Thread M.-A. Lemburg
On 21.12.2014 20:56, Michael Bayer wrote: > > > M.-A. Lemburg wrote: > >> >> Whether or not a cursor is client or server side depends on many >> things and is generally database backend specific. MS SQL Server >> for example will default to server side cursors and revert to >> client side for s

Re: [DB-SIG] Improved support for prepared SQL statements

2014-12-21 Thread M.-A. Lemburg
On 21.12.2014 20:46, Michael Bayer wrote: > M.-A. Lemburg wrote: >>> However, getting back to the issue of prepared statements, I will note that >>> specific to MySQL, MySQL’s own DBAPI driver, MySQL-connector-Python, does in >>> fact tie the prepared statement to the cursor: >>> http://dev.mysql.

Re: [DB-SIG] Improved support for prepared SQL statements

2014-12-21 Thread Michael Bayer
M.-A. Lemburg wrote: > > Whether or not a cursor is client or server side depends on many > things and is generally database backend specific. MS SQL Server > for example will default to server side cursors and revert to > client side for some special cases: > > http://msdn.microsoft.com/en-u

Re: [DB-SIG] Improved support for prepared SQL statements

2014-12-21 Thread Michael Bayer
M.-A. Lemburg wrote: > > Just to clarify: both MS SQL Server and IBM DB2 use ODBC as their > native database interface C API. > > They have both extended it to include some special features which > are not part of the ODBC standard, but in way that's compatible > with ODBC. Just look at the s

Re: [DB-SIG] Improved support for prepared SQL statements

2014-12-21 Thread M.-A. Lemburg
On 19.12.2014 00:12, Michael Bayer wrote: > M.-A. Lemburg wrote: >> I think there's a misunderstanding here. A named cursor refers to >> a database cursor pointing into a result set. Once that result >> set has been created and the cursor points to it, you cannot >> run another .execute() against

Re: [DB-SIG] Improved support for prepared SQL statements

2014-12-21 Thread M.-A. Lemburg
On 21.12.2014 19:40, Michael Bayer wrote: > > > M.-A. Lemburg wrote: > >> >> My experience is from working with ODBC and ODBC drivers. >> IBM DB2 and MS SQL Server use ODBC as their native database >> interface API. In ODBC, cursors are called "statements" and >> you have two modes of operation

Re: [DB-SIG] Improved support for prepared SQL statements

2014-12-21 Thread Michael Bayer
M.-A. Lemburg wrote: > > My experience is from working with ODBC and ODBC drivers. > IBM DB2 and MS SQL Server use ODBC as their native database > interface API. In ODBC, cursors are called "statements" and > you have two modes of operation: > > a) direct execution, which sends the SQL straig

Re: [DB-SIG] Improved support for prepared SQL statements

2014-12-20 Thread INADA Naoki
On Thu, Dec 18, 2014 at 7:39 PM, M.-A. Lemburg wrote: > On 17.12.2014 19:13, INADA Naoki wrote: >> As I said before, prepared statement is normally bound to connection. >> So `.prepare()` method should be connection's method, not cursor's. >> >> prepared = conn.prepare("SELECT ?+?") >> ... >> cur

Re: [DB-SIG] Improved support for prepared SQL statements

2014-12-20 Thread INADA Naoki
> > In my experience, caching of prepared statements in modern use is not a major > source of performance gain unless your application overall is already > blindingly fast, more than Python typically provides. Consider that a site > like reddit.com serves six billion page views a month from Po

Re: [DB-SIG] Improved support for prepared SQL statements

2014-12-20 Thread INADA Naoki
On Thu, Dec 18, 2014 at 9:33 PM, M.-A. Lemburg wrote: > On 18.12.2014 12:27, INADA Naoki wrote: >> On Thu, Dec 18, 2014 at 7:39 PM, M.-A. Lemburg wrote: >>> On 17.12.2014 19:13, INADA Naoki wrote: As I said before, prepared statement is normally bound to connection. So `.prepare()` meth

Re: [DB-SIG] Improved support for prepared SQL statements

2014-12-20 Thread SF Markus Elfring
> Can someone please take up this task and prove to me with benchmarks that > this effort will be worthwhile > (or at least show me what I’m going to have to put into my documentation as > to when these new APIs are appropriate) ? Some database software supports the processing of prepared SQL s

Re: [DB-SIG] Improved support for prepared SQL statements

2014-12-18 Thread Michael Bayer
M.-A. Lemburg wrote: > On 18.12.2014 23:19, Michael Bayer wrote: >>> On Dec 18, 2014, at 3:57 PM, Michael Bayer wrote: >>> On Dec 18, 2014, at 3:39 PM, M.-A. Lemburg wrote: > That would make the entire feature a non-starter for me. SQLAlchemy > doesn’t hold cursors ope

Re: [DB-SIG] Improved support for prepared SQL statements

2014-12-18 Thread M.-A. Lemburg
On 18.12.2014 23:19, Michael Bayer wrote: > >> On Dec 18, 2014, at 3:57 PM, Michael Bayer wrote: >> >>> >>> On Dec 18, 2014, at 3:39 PM, M.-A. Lemburg wrote: >>> >>> That would make the entire feature a non-starter for me.SQLAlchemy doesn’t hold cursors open beyond a single statem

Re: [DB-SIG] Improved support for prepared SQL statements

2014-12-18 Thread Michael Bayer
> On Dec 18, 2014, at 3:57 PM, Michael Bayer wrote: > >> >> On Dec 18, 2014, at 3:39 PM, M.-A. Lemburg wrote: >> >> >>> That would make the entire feature a non-starter for me.SQLAlchemy >>> doesn’t hold cursors open beyond a single statement.My users would very >>> much want a pre

Re: [DB-SIG] Improved support for prepared SQL statements

2014-12-18 Thread Michael Bayer
> On Dec 18, 2014, at 4:36 PM, M.-A. Lemburg wrote: > >> >> Which kinds of errors are you referring to, if the statement has not been >> invoked, I would imagine this refers only to syntactical errors? What kind >> of application contains SQL that may have syntactical errors that only >> be

Re: [DB-SIG] Improved support for prepared SQL statements

2014-12-18 Thread M.-A. Lemburg
On 18.12.2014 21:57, Michael Bayer wrote: > >> On Dec 18, 2014, at 3:39 PM, M.-A. Lemburg wrote: >> >> >>> That would make the entire feature a non-starter for me.SQLAlchemy >>> doesn’t hold cursors open beyond a single statement.My users would very >>> much want a prepared-statement-pe

Re: [DB-SIG] Improved support for prepared SQL statements

2014-12-18 Thread Michael Bayer
> On Dec 18, 2014, at 3:39 PM, M.-A. Lemburg wrote: > > >> That would make the entire feature a non-starter for me.SQLAlchemy >> doesn’t hold cursors open beyond a single statement.My users would very >> much want a prepared-statement-per-transaction object. > > Perhaps you ought to

Re: [DB-SIG] Improved support for prepared SQL statements

2014-12-18 Thread M.-A. Lemburg
On 18.12.2014 21:23, Michael Bayer wrote: > Basically I want there to be very good reason for this feature to be an > explicit part of the DBAPI because it is going to give me personally a lot of > extra headaches when people start asking for it in the name of “performance”, > especially since t

Re: [DB-SIG] Improved support for prepared SQL statements

2014-12-18 Thread M.-A. Lemburg
On 18.12.2014 16:59, Michael Bayer wrote: > >> On Dec 18, 2014, at 5:39 AM, M.-A. Lemburg wrote: >> >> On 17.12.2014 19:13, INADA Naoki wrote: >>> As I said before, prepared statement is normally bound to connection. >>> So `.prepare()` method should be connection's method, not cursor's. >>> >>>

Re: [DB-SIG] Improved support for prepared SQL statements

2014-12-18 Thread Michael Bayer
> On Dec 18, 2014, at 2:00 PM, SF Markus Elfring > wrote: > >> Can someone please take up this task and prove to me with benchmarks that >> this effort will be worthwhile >> (or at least show me what I’m going to have to put into my documentation as >> to when these new APIs are appropriate)

Re: [DB-SIG] Improved support for prepared SQL statements

2014-12-18 Thread Vernon D. Cole
1) There is already an established usual practice (or quasi-standard) for prepared statements: >>> sql_stmt = "some sql string" >>> cursr.prepare(sql_stmt) >>> cursr.execute(sql_stmt, args) # note: the statement string must be >>> identical Any change in sql_stmt undoes the prepared status and th

Re: [DB-SIG] Improved support for prepared SQL statements

2014-12-18 Thread Michael Bayer
> On Dec 18, 2014, at 5:39 AM, M.-A. Lemburg wrote: > > On 17.12.2014 19:13, INADA Naoki wrote: >> As I said before, prepared statement is normally bound to connection. >> So `.prepare()` method should be connection's method, not cursor's. >> >> prepared = conn.prepare("SELECT ?+?") >> ... >> c

Re: [DB-SIG] Improved support for prepared SQL statements

2014-12-18 Thread Michael Bayer
> On Dec 7, 2014, at 4:06 PM, SF Markus Elfring > wrote: > > I imagine that it will be more efficient occasionally to offer also a > base class like "prepared_statement" so that the parameter specification > does not need to be parsed for every passed command. > I suggest to improve correspondi

Re: [DB-SIG] Improved support for prepared SQL statements

2014-12-18 Thread Michael Bayer
> On Dec 18, 2014, at 2:10 AM, SF Markus Elfring > wrote: > I am looking for higher service levels without following the software design directions from object-relational managers like SQLAlchemy and SQLObject. >> >> If you would like a very mature and well proven SQL abstractio

Re: [DB-SIG] Improved support for prepared SQL statements

2014-12-18 Thread M.-A. Lemburg
On 18.12.2014 12:27, INADA Naoki wrote: > On Thu, Dec 18, 2014 at 7:39 PM, M.-A. Lemburg wrote: >> On 17.12.2014 19:13, INADA Naoki wrote: >>> As I said before, prepared statement is normally bound to connection. >>> So `.prepare()` method should be connection's method, not cursor's. >>> >>> prepa

Re: [DB-SIG] Improved support for prepared SQL statements

2014-12-18 Thread M.-A. Lemburg
On 17.12.2014 19:13, INADA Naoki wrote: > As I said before, prepared statement is normally bound to connection. > So `.prepare()` method should be connection's method, not cursor's. > > prepared = conn.prepare("SELECT ?+?") > ... > cur = conn.cursor() > cur.execute(prepared, (1, 2)) > cur.fetchone

Re: [DB-SIG] Improved support for prepared SQL statements

2014-12-18 Thread SF Markus Elfring
>> Can the prepare() method return a handle for a SQL statement? > > The proposal says: > > """ > Return values are not defined. > """ How do you think about to change the application programming interface here? Would a class like "prepared_statement" be more useful at this place? >> Will pre

Re: [DB-SIG] Improved support for prepared SQL statements

2014-12-18 Thread INADA Naoki
As I said before, prepared statement is normally bound to connection. So `.prepare()` method should be connection's method, not cursor's. prepared = conn.prepare("SELECT ?+?") ... cur = conn.cursor() cur.execute(prepared, (1, 2)) cur.fetchone() # returns (3,) On Thu, Dec 18, 2014 at 2:54 AM, M

Re: [DB-SIG] Improved support for prepared SQL statements

2014-12-18 Thread SF Markus Elfring
> Please explain what you want in more detail, so we can check > whether there's room for improvement :-) Can the SQL update statement be encapsulated by a dedicated Python class for example? > Note that the DB-API doesn't require implementing connection > or cursor pools. It only provides ways

Re: [DB-SIG] Improved support for prepared SQL statements

2014-12-18 Thread Tony Locke
On the question of extending the DB-API to accommodate prepared statements, from my narrow point of view as a maintainer of pg8000 it's unnecessary, as pg8000 uses prepared statements for everything anyway. In later versions of Postgres, the query plan is potentially updated every time a prepared s

Re: [DB-SIG] Improved support for prepared SQL statements

2014-12-18 Thread SF Markus Elfring
>>> I am looking for higher service levels without following the software >>> design directions from object-relational managers like SQLAlchemy >>> and SQLObject. > > If you would like a very mature and well proven SQL abstraction layer > that does not provide any design directions whatsoever (not

Re: [DB-SIG] Improved support for prepared SQL statements

2014-12-18 Thread SF Markus Elfring
> How about: > > """ > The prepared operation is only available until the next call to > one of the .execute*() methods or another call to the .prepare() > method using a different command string than the one used for > preparing the previous operation. > """ Can the prepare() method return a han

Re: [DB-SIG] Improved support for prepared SQL statements

2014-12-17 Thread M.-A. Lemburg
On 17.12.2014 20:37, Tony Locke wrote: > On the question of extending the DB-API to accommodate prepared > statements, from my narrow point of view as a maintainer of pg8000 > it's unnecessary, as pg8000 uses prepared statements for everything > anyway. In later versions of Postgres, the query plan

Re: [DB-SIG] Improved support for prepared SQL statements

2014-12-17 Thread M.-A. Lemburg
On 17.12.2014 18:54, Michael Bayer wrote: > >> On Dec 17, 2014, at 10:41 AM, M.-A. Lemburg wrote: >> >> >> Another possibility is to write a layer on top of the DB-API >> to abstract the underlying queries away from the application >> and only have the layer provide dedicated methods for the >> t

Re: [DB-SIG] Improved support for prepared SQL statements

2014-12-17 Thread Michael Bayer
> On Dec 17, 2014, at 10:41 AM, M.-A. Lemburg wrote: > > > Another possibility is to write a layer on top of the DB-API > to abstract the underlying queries away from the application > and only have the layer provide dedicated methods for the > things the application needs, such as query method

Re: [DB-SIG] Improved support for prepared SQL statements

2014-12-17 Thread M.-A. Lemburg
On 17.12.2014 16:18, SF Markus Elfring wrote: >> Please explain what you want in more detail, so we can check >> whether there's room for improvement :-) > > Can the SQL update statement be encapsulated by a dedicated Python > class for example? Sure. You could write a class that only supports up

Re: [DB-SIG] Improved support for prepared SQL statements

2014-12-17 Thread M.-A. Lemburg
On 17.12.2014 15:38, SF Markus Elfring wrote: >>> Can the prepare() method return a handle for a SQL statement? >> >> The proposal says: >> >> """ >> Return values are not defined. >> """ > > How do you think about to change the application programming > interface here? > > Would a class like "pr

Re: [DB-SIG] Improved support for prepared SQL statements

2014-12-17 Thread M.-A. Lemburg
On 17.12.2014 14:50, SF Markus Elfring wrote: >> How about: >> >> """ >> The prepared operation is only available until the next call to >> one of the .execute*() methods or another call to the .prepare() >> method using a different command string than the one used for >> preparing the previous ope

Re: [DB-SIG] Improved support for prepared SQL statements

2014-12-17 Thread M.-A. Lemburg
On 09.12.2014 13:24, Daniele Varrazzo wrote: > On Tue, Dec 9, 2014 at 9:40 AM, M.-A. Lemburg wrote: >> On 07.12.2014 22:06, SF Markus Elfring wrote: >>> Hello, >>> >>> An interface for parameterised SQL statements (working with >>> placeholders) is provided by the execute() method from the Cursor

Re: [DB-SIG] Improved support for prepared SQL statements

2014-12-09 Thread Daniele Varrazzo
On Tue, Dec 9, 2014 at 9:40 AM, M.-A. Lemburg wrote: > On 07.12.2014 22:06, SF Markus Elfring wrote: >> Hello, >> >> An interface for parameterised SQL statements (working with >> placeholders) is provided by the execute() method from the Cursor class >> at the moment. >> https://docs.python.org/3

Re: [DB-SIG] Improved support for prepared SQL statements

2014-12-09 Thread M.-A. Lemburg
On 07.12.2014 22:06, SF Markus Elfring wrote: > Hello, > > An interface for parameterised SQL statements (working with > placeholders) is provided by the execute() method from the Cursor class > at the moment. > https://docs.python.org/3/library/sqlite3.html#sqlite3.Cursor.execute > > I assume th

[DB-SIG] Improved support for prepared SQL statements

2014-12-09 Thread SF Markus Elfring
Hello, An interface for parameterised SQL statements (working with placeholders) is provided by the execute() method from the Cursor class at the moment. https://docs.python.org/3/library/sqlite3.html#sqlite3.Cursor.execute I assume that the "SQL Statement Object" from the SQLite C interface is r