[Rails-core] Re: Request for comments: prepared statements in ActiveRecord

2007-08-23 Thread Roderick van Domburg
Hi Hongli, > > Very interesting to see you working on this project. I work a lot for > > Enterprises and it makes a lot of difference when it comes to Oracle > > with prepared statements and data binding. In case of loops etc. we > > had find the performance difference to be in multiples. > > > I

[Rails-core] Re: Request for comments: prepared statements in ActiveRecord

2007-08-18 Thread Hongli Lai
On Aug 17, 6:21 pm, Deepak Kumar <[EMAIL PROTECTED]> wrote: > Very interesting to see you working on this project. I work a lot for > Enterprises and it makes a lot of difference when it comes to Oracle > with prepared statements and data binding. In case of loops etc. we > had find the performanc

[Rails-core] Re: Request for comments: prepared statements in ActiveRecord

2007-08-17 Thread Deepak Kumar
Very interesting to see you working on this project. I work a lot for Enterprises and it makes a lot of difference when it comes to Oracle with prepared statements and data binding. In case of loops etc. we had find the performance difference to be in multiples. I was just wondering how far this

[Rails-core] Re: Request for comments: prepared statements in ActiveRecord

2007-06-25 Thread Brian Hartin
Thank you so much for working on this feature. Prepared statement/ bind variable support is critical for Oracle projects, especially in the (evil) enterprise world. I know that Rails isn't targeted for large enterprise apps, but that doesn't mean it isn't targeted at _enterprises_. Lots of smal

[Rails-core] Re: Request for comments: prepared statements in ActiveRecord

2007-05-18 Thread Hongli Lai
On May 17, 9:55 pm, Luca Mearelli <[EMAIL PROTECTED]> wrote: > aahh I won't be at the conf... any way to meet in a virtual space? > perhaps the #railsconf irc channel? Yes, I'd be delighted to. :) I hope we can solve this together. --~--~-~--~~~---~--~~ You recei

[Rails-core] Re: Request for comments: prepared statements in ActiveRecord

2007-05-18 Thread Hongli Lai
On May 18, 12:58 am, "Mislav Marohnić" <[EMAIL PROTECTED]> wrote: > It would be good to know what is this treshold. For some reason I'm unable to reproduce that problem now. Perhaps I was mistaken. > It is possible that there is a performance boost on raw queries, but that > it's not visible be

[Rails-core] Re: Request for comments: prepared statements in ActiveRecord

2007-05-17 Thread Mislav Marohnić
On 5/17/07, Hongli Lai <[EMAIL PROTECTED]> wrote: > > > I tested this with MySQL. I can't have unlimited number of prepared > statements. After a certain threshold, queries will fail. It would be good to know what is this treshold. ... according to benchmarks, caching > prepared statements doesn

[Rails-core] Re: Request for comments: prepared statements in ActiveRecord

2007-05-17 Thread Hongli Lai
On May 16, 1:04 pm, "Mislav Marohnić" <[EMAIL PROTECTED]> wrote: > The former refers to your intention to drop prepared statements after each > query. The latter is what would happen if you didn't cleanup these prepared > statements - you'd suddenly have hundreds of them in the database. I think >

[Rails-core] Re: Request for comments: prepared statements in ActiveRecord

2007-05-17 Thread Luca Mearelli
Jeremy Kemper wrote: > On 5/15/07, Michael A. Schoen <[EMAIL PROTECTED]> wrote: >>Though I like what was suggested (by you I think) back in Feb, around >>the idea of SqlString < String. Has anyone looked into that further? > > I wish :) > I actually started this morning to look into it ;) >>>Ar

[Rails-core] Re: Request for comments: prepared statements in ActiveRecord

2007-05-17 Thread Jeremy Kemper
On 5/15/07, Michael A. Schoen <[EMAIL PROTECTED]> wrote: > Jeremy Kemper wrote: > > I'd start with using bound variables first, though. They're always > > beneficial and are a natural stepping stone to prepared statements > > which can be added later for those queries which may benefit (many do >

[Rails-core] Re: Request for comments: prepared statements in ActiveRecord

2007-05-17 Thread Michael A. Schoen
Luca Mearelli wrote: > Hongli Lai wrote: > In any case using real bind variable would represent a further > optimization step, as it'd allow a fine tuning of which bind variables > vs literal values use in each query. Agreed. In my tests the _client_ performance impact of using real bind varia

[Rails-core] Re: Request for comments: prepared statements in ActiveRecord

2007-05-16 Thread Mislav Marohnić
On 5/16/07, Hongli Lai <[EMAIL PROTECTED]> wrote: > > > On May 16, 5:42 am, Sam Smoot <[EMAIL PROTECTED]> wrote: > > So I guess my question is: Is flushing the statements really necessary > > or are you just working under an assumption the database (Postgres/ > > MySQL) will blindly allow it to gro

[Rails-core] Re: Request for comments: prepared statements in ActiveRecord

2007-05-15 Thread Luca Mearelli
Hongli Lai wrote: > On May 16, 5:49 am, "Michael A. Schoen" <[EMAIL PROTECTED]> wrote: > >>I'd have to disagree with you there. Rails actually performs very well >>with Oracle. A change made a while back to default to "similar" >>cursor_sharing resulted in much of the benefit of bind variables. >

[Rails-core] Re: Request for comments: prepared statements in ActiveRecord

2007-05-15 Thread Hongli Lai
On May 16, 5:42 am, Sam Smoot <[EMAIL PROTECTED]> wrote: > So I guess my question is: Is flushing the statements really necessary > or are you just working under an assumption the database (Postgres/ > MySQL) will blindly allow it to grow to unreasonable levels? I'm sorry, I do not understand wha

[Rails-core] Re: Request for comments: prepared statements in ActiveRecord

2007-05-15 Thread Hongli Lai
On May 16, 5:49 am, "Michael A. Schoen" <[EMAIL PROTECTED]> wrote: > I'd have to disagree with you there. Rails actually performs very well > with Oracle. A change made a while back to default to "similar" > cursor_sharing resulted in much of the benefit of bind variables. > "performs badly" is de

[Rails-core] Re: Request for comments: prepared statements in ActiveRecord

2007-05-15 Thread Hongli Lai
On May 16, 1:11 am, "Jeremy Kemper" <[EMAIL PROTECTED]> wrote: > I like the gist of your idea and approach. Thanks for working on it. > > I'd start with using bound variables first, though. They're always > beneficial and are a natural stepping stone to prepared statements > which can be added lat

[Rails-core] Re: Request for comments: prepared statements in ActiveRecord

2007-05-15 Thread Michael A. Schoen
Jeremy Kemper wrote: > I'd start with using bound variables first, though. They're always > beneficial and are a natural stepping stone to prepared statements > which can be added later for those queries which may benefit (many do > not). From my tests w/ Oracle, all the benefit was in the use o

[Rails-core] Re: Request for comments: prepared statements in ActiveRecord

2007-05-15 Thread Michael A. Schoen
Hongli Lai wrote: > Oracle support. Oracle lives by prepared statements, even if those > prepared statements aren't reused, as someone has pointed out. One can > argue that few Rails developers use Oracle, but I can also argue that > that's because Rails performs badly on Oracle. I'd have to disa

[Rails-core] Re: Request for comments: prepared statements in ActiveRecord

2007-05-15 Thread Sam Smoot
Just wondering, but is there any advantage to trying to manually manage the execution plan cache? I know MSSQL-Server for example caches the execution-plan of every parameterized Query (anything executed with the sp_executesql stored-procedure), which means the majority of ADO and ADO.NET based ap

[Rails-core] Re: Request for comments: prepared statements in ActiveRecord

2007-05-15 Thread Jeremy Kemper
On 5/14/07, Hongli Lai <[EMAIL PROTECTED]> wrote: > I'm working on long-awaited support for prepared statements in > ActiveRecord. Before I finish my work, I would like to know whether > the Ruby on Rails core team accepts my design, and whether they will > review my patch (once it is finished) se

[Rails-core] Re: Request for comments: prepared statements in ActiveRecord

2007-05-15 Thread Hongli Lai
On May 15, 11:55 pm, Courtenay <[EMAIL PROTECTED]> wrote: > On 5/14/07, Hongli Lai <[EMAIL PROTECTED]> wrote: > I know this is tangential, but is there any good reason for storing > images in the database? > > I can think of a few, but they're not very good; > - security (but that can be handled

[Rails-core] Re: Request for comments: prepared statements in ActiveRecord

2007-05-15 Thread Courtenay
On 5/14/07, Hongli Lai <[EMAIL PROTECTED]> wrote: > > On May 14, 9:07 pm, "Mislav Marohnić" <[EMAIL PROTECTED]> > wrote: > > Doesn't that negate the whole idea of having prepared statements in the > > database in the first place? I don't think Rails core team would accept the > > patch that slows d

[Rails-core] Re: Request for comments: prepared statements in ActiveRecord

2007-05-15 Thread Hongli Lai
On May 15, 11:17 am, Hongli Lai <[EMAIL PROTECTED]> wrote: > Here's an update. I noticed that using prepared statements will indeed > make MySQL queries a bit slower. The unit tests in an unmodified Rails > edge source tree took 18 seconds. After adding prepared statements, > they took 28 seconds.

[Rails-core] Re: Request for comments: prepared statements in ActiveRecord

2007-05-15 Thread Hongli Lai
Here's an update. I noticed that using prepared statements will indeed make MySQL queries a bit slower. The unit tests in an unmodified Rails edge source tree took 18 seconds. After adding prepared statements, they took 28 seconds. I've been working on optimizing this since yesterday evening, and

[Rails-core] Re: Request for comments: prepared statements in ActiveRecord

2007-05-14 Thread Michael A. Schoen
Alexey Verkhovsky wrote: >> and they would see a noticable gain only when these prepared statements >> were reused for multiple queries. > It's the same with Oracle. Prepared statement is merely a way to avoid > hitting SQL parser and query optimizer. I've done some testing w/ Oracle, and a very

[Rails-core] Re: Request for comments: prepared statements in ActiveRecord

2007-05-14 Thread Mislav Marohnić
On 5/14/07, Hongli Lai <[EMAIL PROTECTED]> wrote: > > > Are there any other discussion points left? Yes. In your place I would contact Zach to get an update on his progress. If he really did some refactoring like he said, it is most likely that he made some statement builder class himself. Pickin

[Rails-core] Re: Request for comments: prepared statements in ActiveRecord

2007-05-14 Thread Hongli Lai
On May 14, 10:35 pm, "Alexey Verkhovsky" <[EMAIL PROTECTED]> wrote: > On 5/14/07, Mislav Marohnić <[EMAIL PROTECTED]> wrote: > > > On 5/14/07, Alexey Verkhovsky <[EMAIL PROTECTED]> wrote: > > I'd dare to say that Oracle isn't exactly very popular in Rails world. > > It is in my corner of the woods

[Rails-core] Re: Request for comments: prepared statements in ActiveRecord

2007-05-14 Thread Alexey Verkhovsky
On 5/14/07, Mislav Marohnić <[EMAIL PROTECTED]> wrote: > On 5/14/07, Alexey Verkhovsky <[EMAIL PROTECTED]> wrote: > I'd dare to say that Oracle isn't exactly very popular in Rails world. It is in my corner of the woods. Let me not mention that evil E-word again :) > and they would see a noticable

[Rails-core] Re: Request for comments: prepared statements in ActiveRecord

2007-05-14 Thread Mislav Marohnić
On 5/14/07, Alexey Verkhovsky <[EMAIL PROTECTED]> wrote: > > > From past experiences, I would expect to see a noticeable, if not > huge, gain from caching prepared statements. At least, with Oracle. I'd dare to say that Oracle isn't exactly very popular in Rails world. The majority uses Postgres

[Rails-core] Re: Request for comments: prepared statements in ActiveRecord

2007-05-14 Thread Alexey Verkhovsky
On 5/14/07, Hongli Lai <[EMAIL PROTECTED]> wrote: > And I'm not sure that using prepared statements (and immediately > deleting them) really makes things slower. Why would it be? It'll be somewhat slower than just throwing a SQL statement at the database because it adds extra synchronous communic

[Rails-core] Re: Request for comments: prepared statements in ActiveRecord

2007-05-14 Thread Hongli Lai
On May 14, 9:07 pm, "Mislav Marohnić" <[EMAIL PROTECTED]> wrote: > Doesn't that negate the whole idea of having prepared statements in the > database in the first place? I don't think Rails core team would accept the > patch that slows down ActiveRecord just for the sake of using a database > feat

[Rails-core] Re: Request for comments: prepared statements in ActiveRecord

2007-05-14 Thread Mislav Marohnić
On 5/14/07, Hongli Lai <[EMAIL PROTECTED]> wrote: > > > The current implementation immediately removes prepared statements > after using them once Doesn't that negate the whole idea of having prepared statements in the database in the first place? I don't think Rails core team would accept the pa

[Rails-core] Re: Request for comments: prepared statements in ActiveRecord

2007-05-14 Thread Hongli Lai
On May 14, 8:25 pm, "Mislav Marohnić" <[EMAIL PROTECTED]> wrote: > So, every query ever issued will become a prepared statement. Isn't this > going to lead to unnecessary overhead? I mean, there are *a lot* of various > SELECT statements generated by even a simple Rails application. Does it > real

[Rails-core] Re: Request for comments: prepared statements in ActiveRecord

2007-05-14 Thread Mislav Marohnić
On 5/14/07, Hongli Lai <[EMAIL PROTECTED]> wrote: > > > The use of prepared statements in MySQL/PostgreSQL will > completely replace the argument escaping stuff. So, every query ever issued will become a prepared statement. Isn't this going to lead to unnecessary overhead? I mean, there are *a lo

[Rails-core] Re: Request for comments: prepared statements in ActiveRecord

2007-05-14 Thread Hongli Lai
On May 14, 7:15 pm, "Mislav Marohnić" <[EMAIL PROTECTED]> wrote: > Nicely done! But am I to understand StatementBuilder code is going to > replace (cleanup) all the string hacks currently involved in building SQL > queries? Will the usage of prepared statements in MySQL/PostgreSQL become > implici

[Rails-core] Re: Request for comments: prepared statements in ActiveRecord

2007-05-14 Thread Mislav Marohnić
On 5/14/07, Hongli Lai <[EMAIL PROTECTED]> wrote: > > > I have documented the design at: http://izumi.plan99.net/blog/?p=35 Nicely done! But am I to understand StatementBuilder code is going to replace (cleanup) all the string hacks currently involved in building SQL queries? Will the usage of pr