Re: Using EXISTS instead of IN for subqueries
On Tue, Mar 26, 2013 at 5:47 PM, Petite Abeillewrote: > > On Mar 26, 2013, at 10:03 PM, Alex Gaynor wrote: > > > For what it's worth, SQL2011 does define OFFSET, finally. > Just checked and it appears this has been implemented with SQL Server 2012. A quick check of the other database with Django backends seems to show that all of them have a beta or stable release with support for a simple limit/offset syntax, with many supporting the standard defined syntax. This might make the generic approach as simple as adding an DatabaseOperatoins.limit_offset_sql() abstraction. Regards, Michael Manfre -- You received this message because you are subscribed to the Google Groups "Django developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at http://groups.google.com/group/django-developers?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Re: Proposal for allowing dynamic site based on domain name WITHOUT changing settings.SITE_ID.
I've just read the previous discussion and I now have the same conclusion as yours - the optional request argument doesn't provide much the user can't do easily anyway. They can always make a custom function to grab the site based on the request. The value of Site.objects.get_current() as you previously mentioned is in allowing code with no access to the request object to fetch the current site. Therefore, it wouldn't make sense to add dynamic multitenancy without also adding the ability for Site.objects.get_current() to return the current, dynamic, site; however this solution requires storing the request in the thread state, which introduces another set of complications... -- You received this message because you are subscribed to the Google Groups "Django developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at http://groups.google.com/group/django-developers?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Re: Using EXISTS instead of IN for subqueries
On Mar 26, 2013, at 10:03 PM, Alex Gaynorwrote: > For what it's worth, SQL2011 does define OFFSET, finally. Perhaps worthwhile mentioning as well : "Do not try to implement a scrolling window using LIMIT and OFFSET. Doing so will become sluggish as the user scrolls down toward the bottom of the list." -- Scrolling Cursor, What Not To Do http://www.sqlite.org/cvstrac/wiki?p=ScrollingCursor Just saying... -- You received this message because you are subscribed to the Google Groups "Django developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at http://groups.google.com/group/django-developers?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Re: Using EXISTS instead of IN for subqueries
For what it's worth, SQL2011 does define OFFSET, finally. Alex On Tue, Mar 26, 2013 at 5:00 PM, Petite Abeillewrote: > > On Mar 26, 2013, at 4:19 PM, Michael Manfre wrote: > > > Maybe someday the non-standard LIMIT/OFFSET keywords will get added to > the > > standard (I truly hope this happens) so Oracle, MSSQL, DB2, and Informix > > could share SQL with postgres and mysql without needing to mangle it. > > FWIW, Oracle 12c is rumored to support OFFSET/FETCH FIRST: > > • Top-N now using Row limiting clause eg. “OFFSET 10 ROWS FETCH FIRST 10 > ROWS ONLY”. Similar to mySQL syntax. > > http://www.oracle-base.com/blog/2012/10/06/oracle-openworld-2012-day-5/ > > > -- > You received this message because you are subscribed to the Google Groups > "Django developers" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to django-developers+unsubscr...@googlegroups.com. > To post to this group, send email to django-developers@googlegroups.com. > Visit this group at http://groups.google.com/group/django-developers?hl=en > . > For more options, visit https://groups.google.com/groups/opt_out. > > > -- "I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) "The people's good is the highest law." -- Cicero -- You received this message because you are subscribed to the Google Groups "Django developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at http://groups.google.com/group/django-developers?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Re: Proposal for allowing dynamic site based on domain name WITHOUT changing settings.SITE_ID.
On 03/25/2013 09:25 PM, meric wrote: > Previous discussion: > https://groups.google.com/forum/?fromgroups=#!searchin/django-developers/dynamic$20sites/django-developers/QSXLGSxy7Vk/TxgiJzz5nd8J > https://code.djangoproject.com/ticket/16983 > https://code.djangoproject.com/ticket/4438 If you're interested in this area, you should review https://code.djangoproject.com/ticket/15089 and https://groups.google.com/forum/?fromgroups=#!topic/django-developers/4jw79p76uUo - as far as I'm aware, that ticket/thread represent the farthest anyone's gotten towards a specific proposal for handling dynamic multitenancy with contrib.sites. > My proposal allows getting the current site based on request.get_host(), > without changing settings.SITE_ID, and maintaining backwards compatibility. > > Required code changes: http://dpaste.com/1035045/ > > I propose adding an optional `request` argument to > Site.objects.get_current(), so that: > > `Site.objects.get_current()` returns current `Site` based on > settings.SITE_ID > `Site.objects.get_current(request)` returns current `Site` based on > request.get_host() > > Both conditional branches will be cached in SITE_CACHE. > > The current changes I've proposed means if there is no site with domain > == request.get_host(), it won't be able to get the current site. An > alternative would be to fall back onto using settings.SITE_ID. > > What do you think? So it seems that basically this proposal just provides a way for user code to get a Site instance by request host; it doesn't address the issue of Django internally getting the current site using settings.SITE_ID only (since even with this patch, Django would never internally pass the request and get the new behavior). If that's the extent of the benefit (a new way for user code to get the current site), then I don't see the reason to complicate the behavior of the existing functions with new optional arguments; rather just add a new and separate Site.objects.get_current_for_request or similar. That makes sense to me as a first step. Carl -- You received this message because you are subscribed to the Google Groups "Django developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at http://groups.google.com/group/django-developers?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Re: Using EXISTS instead of IN for subqueries
On 2013-03-26 15:54, Michael Manfre wrote: > On Tue, Mar 26, 2013 at 12:40 PM, Anssi Kääriäinen > deal with limit/offset. A generic approach would be nice to have, > but I can't imagine a generic way that would let me generate the > "SELECT ... FROM (SELECT ROW_NUMBER() OVER (...)) WHERE ..." > monstrosity with lots of column aliasing that I currently > construct. The closest I've come is an ugly nested query using TOP (the TSQL analog to LIMIT, but as mentioned earlier, there's no OFFSET counterpart), and inverting the sort conditions: -- want sorted by "a asc, b desc, c asc" -- assuming LIMIT=10, OFFSET=20 select * from ( select top 10 -- LIMIT * from ( select top 30 -- LIMIT+OFFSET=10+20 * from tbl order by a asc, b desc, c asc ) top_half order by a desc, b asc, c desc -- note inversion ) reversed_top_half order by a asc, b desc, c asc It's been a while since I've done it, so it Works™, but (1) there's the inevitable fence-posting error I'd have to verify, (2) it involves sorting, reverse-sorting, then re-sorting (not exactly the speediest operation), and (3) it's hideous. It doesn't seem to require the column-aliasing you mention, and it is a fairly generic approach, but I can't say I recommend it :-) -tkc -- You received this message because you are subscribed to the Google Groups "Django developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at http://groups.google.com/group/django-developers?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Re: URL dispatcher fallthrough?
+1 here! It might also be a great feature to run the 404 through before sending that email out. A replacement and/or complement to IGNORABLE_404_URLS Val On Tue, Mar 26, 2013 at 1:25 PM, Loic Bistuerwrote: > +1 for me. > > Having a catchall view under a single URL pattern is a tightly coupled > system, more so than allowing independent views to "test the water". > > Django core relies on middleware hacks because the URL dispatcher is missing > this very feature. Having this in core would allow a cleaner implementation > of the fallback mechanisms in contrib.flatpages and contrib.redirects. > > -- > Loic > > On Mar 19, 2013, at 11:18 PM, Adrian Holovaty wrote: > > I'd rather not add this to the framework, as it's already possible > with a "wrapper" view (as others have suggested). And on a more > theoretical level, it introduces more coupling between URL patterns > and views. > > > -- > You received this message because you are subscribed to the Google Groups > "Django developers" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to django-developers+unsubscr...@googlegroups.com. > To post to this group, send email to django-developers@googlegroups.com. > Visit this group at http://groups.google.com/group/django-developers?hl=en. > For more options, visit https://groups.google.com/groups/opt_out. > > -- You received this message because you are subscribed to the Google Groups "Django developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at http://groups.google.com/group/django-developers?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Re: URL dispatcher fallthrough?
+1 for me. Having a catchall view under a single URL pattern is a tightly coupled system, more so than allowing independent views to "test the water". Django core relies on middleware hacks because the URL dispatcher is missing this very feature. Having this in core would allow a cleaner implementation of the fallback mechanisms in contrib.flatpages and contrib.redirects. -- Loic On Mar 19, 2013, at 11:18 PM, Adrian Holovatywrote: > I'd rather not add this to the framework, as it's already possible > with a "wrapper" view (as others have suggested). And on a more > theoretical level, it introduces more coupling between URL patterns > and views. -- You received this message because you are subscribed to the Google Groups "Django developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at http://groups.google.com/group/django-developers?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Re: Using EXISTS instead of IN for subqueries
On Tuesday, March 26, 2013 5:19:52 PM UTC+2, Michael Manfre wrote: > > > > On Tue, Mar 26, 2013 at 10:23 AM, Tim Chase >> wrote: > >> EXISTS also has some nice features >> like the ability to do testing against multiple columns, i.e., you >> can't do something like >> >> select * >> from tbl1 >> where (col1, col2) in ( >>select a, b >>from tbl2 >>) >> >> but that's a simple EXISTS query. >> > > Agreed, EXISTS is more flexible and at least to me, often easier to read, > format, and maintain. Assuming this is implemented with a database feature, > I'll most likely enable the EXISTS change for django-mssql. My main > objections to a blanket change is to not have the specific behaviors of one > database dictate how every other database must behave. Some one needs to be > the voice of 3rd party database backends and it appears for the time being, > I'm wearing that hat. > One of the main reasons for the change is that EXISTS allows for queries that are impossible with IN. In addition EXISTS semantics regarding NULLs is wanted. And, at least PostgreSQL performs better with NOT EXISTS than NOT IN. Granted, the better performance on PostgreSQL is perhaps the most important reason for me, but this change is not only about that. The main reason for this thread was to find out if there are some databases where performance of EXISTS is worse than IN. The DB feature approach seems good because it allows investigating performance characteristics one DB at time. > >> > > MSSQL is a 2nd-class citizen in the Django world, so I'm +1 >> > >> > Reasoning like that helps to keep it in its place. >> >> MSSQL's lack of certain core features is what does that. >> OFFSET/LIMIT provided by pretty much every other DB vendor? Use >> baroque hacks to get the same functionality. > > > The non "baroque hack" way of doing limit offset is with nested selects. > Maybe someday the non-standard LIMIT/OFFSET keywords will get added to the > standard (I truly hope this happens) so Oracle, MSSQL, DB2, and Informix > could share SQL with postgres and mysql without needing to mangle it. > Hmmh, this means Oracle, MSSQL, DB2 and Informix are doing more or less the same thing for limit/offset support? If so, then having a more generic approach to this problem than having a custom compiler per backend might be worth it... BTW there is already something like LIMIT and OFFSET in SQL 2008 standard. The syntax is different than LIMIT/OFFSET, and supported only by some vendors... See https://en.wikipedia.org/wiki/Select_%28SQL%29#FETCH_FIRST_clause - Anssi -- You received this message because you are subscribed to the Google Groups "Django developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at http://groups.google.com/group/django-developers?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Re: Using EXISTS instead of IN for subqueries
On Tue, Mar 26, 2013 at 10:23 AM, Tim Chasewrote: > > In SQL 2008r2, the optimizer is usually smart enough to end up with > > the same execution plan for IN and EXISTS queries. Historically, > > EXISTS was usually the faster operation for SQL Server and if > > memory serves it had to deal with its ability to bail out of the > > EXISTS query sooner compared to the IN query. > > I'd have to go back and re-test 2008r2, as my testing was on 2005 > (and earlier). But my testing directly contradicts your > "Historically..." bit, as I DISTINCTly (bad SQL pun intended) remember > being surprised precisely because of what you say: EXISTS should be > able to optimize and bail early. My "Historically" comment is based upon memory of observed behaviors from many years ago and could even be an ingrained memory dating back to SQL Server 7. I'll concede that your testing is probably more accurate than my memory of observed SQL optimizer behaviors from upwards of a decade ago. The execution plan is a result of the specific query and schema, so it's possible that we're both correct and both wrong at the same time. > EXISTS also has some nice features > like the ability to do testing against multiple columns, i.e., you > can't do something like > > select * > from tbl1 > where (col1, col2) in ( >select a, b >from tbl2 >) > > but that's a simple EXISTS query. > Agreed, EXISTS is more flexible and at least to me, often easier to read, format, and maintain. Assuming this is implemented with a database feature, I'll most likely enable the EXISTS change for django-mssql. My main objections to a blanket change is to not have the specific behaviors of one database dictate how every other database must behave. Some one needs to be the voice of 3rd party database backends and it appears for the time being, I'm wearing that hat. > > > MSSQL is a 2nd-class citizen in the Django world, so I'm +1 > > > > Reasoning like that helps to keep it in its place. > > MSSQL's lack of certain core features is what does that. > OFFSET/LIMIT provided by pretty much every other DB vendor? Use > baroque hacks to get the same functionality. The non "baroque hack" way of doing limit offset is with nested selects. Maybe someday the non-standard LIMIT/OFFSET keywords will get added to the standard (I truly hope this happens) so Oracle, MSSQL, DB2, and Informix could share SQL with postgres and mysql without needing to mangle it. Regards, Michael Manfre -- You received this message because you are subscribed to the Google Groups "Django developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at http://groups.google.com/group/django-developers?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Re: Using EXISTS instead of IN for subqueries
On 2013-03-25 22:16, Michael Manfre wrote: > On Monday, March 25, 2013 6:58:12 AM UTC-4, Tim Chase wrote: > > I can only speak for testing IN-vs-EXISTS speed on MSSQLServer at > > $OLD_JOB, but there it's usually about the same, occasionally > > with IN winning out. > > In SQL 2008r2, the optimizer is usually smart enough to end up with > the same execution plan for IN and EXISTS queries. Historically, > EXISTS was usually the faster operation for SQL Server and if > memory serves it had to deal with its ability to bail out of the > EXISTS query sooner compared to the IN query. I'd have to go back and re-test 2008r2, as my testing was on 2005 (and earlier). But my testing directly contradicts your "Historically..." bit, as I DISTINCTly (bad SQL pun intended) remember being surprised precisely because of what you say: EXISTS should be able to optimize and bail early. EXISTS also has some nice features like the ability to do testing against multiple columns, i.e., you can't do something like select * from tbl1 where (col1, col2) in ( select a, b from tbl2 ) but that's a simple EXISTS query. > > MSSQL is a 2nd-class citizen in the Django world, so I'm +1 > > Reasoning like that helps to keep it in its place. MSSQL's lack of certain core features is what does that. OFFSET/LIMIT provided by pretty much every other DB vendor? Use baroque hacks to get the same functionality. I seem to recall other issues, though v2008 seems to have addressed many of them (min date = 1900-01-01 which was an issue when dealing with historical/genealogical data; seems to be better in 2008) Either way, if EXISTS in MSSQL is now faster than IN, it just is one more tally in the "plus" column for why this might be a good idea (modulo implementation complexities). -tkc -- You received this message because you are subscribed to the Google Groups "Django developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at http://groups.google.com/group/django-developers?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Re: URL dispatcher fallthrough?
On Tue, Mar 26, 2013 at 10:44 AM, mericwrote: > Thinking through Adrian's post and mine above, it appears to be a trade > off between coupling in the framework and increased responsibility for user > views code. Personally I would opt for the former, because IMHO the idea of > a framework is to reduce responsibility of user code and promoting same > sets of design patterns for common problems (rather than each similar > program having different adhoc ways to deal with the same problems I.e > rather than having different ways people would implement a catch all view, > have views send the same signal if the given arguments were not > interpretable.) first before anything else, otherwise what else can a > framework provide?. Even though I suggested the use of a catch-all view for handling multiple objects in the beginning of the thread, I now see the value of moving this to urls.py. For more complex websites and scenarios, some patterns may emerge that lead to such a number of possibilities that at some point, from a design perspective, you start wondering if you're not packing too much routing into the views (with a catch-all view) when they should be handled by the URL routing module. Previously -1, now +1 on this idea. Cheers, AT -- You received this message because you are subscribed to the Google Groups "Django developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at http://groups.google.com/group/django-developers?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Re: What can I do to get feedback on my pull request?
Adrian's one of the BDFLs (I'm the other), so if he says "no" that stands. Usually, we operate the way most open source projects do: "rough consensus and working code". You've got the second ("working code") so you need to work on the first, I think. Usually a strong consensus is enough to override a committer's veto, even a BDFL veto, so if you can show that this is a strong enough need, Adrian might drop his veto. [For more on how we make decisions, see https://docs.djangoproject.com/en/1.5/internals/contributing/bugs-and-features/#how-we-make-decisions] Of course, I'm putting words in Adrian mouth, so this is all just a guess. It's possible that he's really against this, in which case you're basically SOL. Adrian, if you're still following this thread... maybe want to weigh in on how we might convince you to drop that veto? Jacob On Tue, Mar 26, 2013 at 8:17 AM, mericwrote: > Is the 'vote' a majority or a consensus must be achieved? > > -- > You received this message because you are subscribed to the Google Groups > "Django developers" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to django-developers+unsubscr...@googlegroups.com. > To post to this group, send email to django-developers@googlegroups.com. > Visit this group at http://groups.google.com/group/django-developers?hl=en. > For more options, visit https://groups.google.com/groups/opt_out. > > -- You received this message because you are subscribed to the Google Groups "Django developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at http://groups.google.com/group/django-developers?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Re: What can I do to get feedback on my pull request?
Is the 'vote' a majority or a consensus must be achieved? -- You received this message because you are subscribed to the Google Groups "Django developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at http://groups.google.com/group/django-developers?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Re: What can I do to get feedback on my pull request?
On Mon, Mar 25, 2013 at 10:50 PM, mericwrote: > What action, if any, do you suggest I take now? If it's gonna get in, you/we need to talk Adrian into dropping his -1 vote. Jacob -- You received this message because you are subscribed to the Google Groups "Django developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at http://groups.google.com/group/django-developers?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Re: Using EXISTS instead of IN for subqueries
On 26 maalis, 07:16, Michael Manfrewrote: > On Monday, March 25, 2013 6:58:12 AM UTC-4, Tim Chase wrote: > > > I can only speak for testing IN-vs-EXISTS speed on MSSQLServer at > > $OLD_JOB, but there it's usually about the same, occasionally with IN > > winning out. > > In SQL 2008r2, the optimizer is usually smart enough to end up with the > same execution plan for IN and EXISTS queries. Historically, EXISTS was > usually the faster operation for SQL Server and if memory serves it had to > deal with its ability to bail out of the EXISTS query sooner compared to > the IN query. > > MSSQL is a 2nd-class citizen in the Django world, so I'm +1 > > > > Reasoning like that helps to keep it in its place. > > Anssi, > > Any chance of adding a new database feature to flip the behavior of __in to > either IN or EXISTS? Sounds like this change of logical and documented > behavior is being made specifically because of failings with Postgresql. > The feature would also help satisfy the deprecation cycle normally used for > changes to documented behaviors. Sub-queries are more likely to expose > database specific issues with the SQL provided by Django (normally when > used with aggregates or slicing). Adding the database feature might save > every other backend from having to potentially jump > through unnecessary hoops (mangling more SQL). Yes, this is a possibility and I think I will take this path. This adds a bit of complexity to the code but not much. There is already support for both IN and EXISTS subqueries in the ORM (as of couple of days ago). In addition I will change the __in lookup to exclude NULL values from the inner query so that EXISTS and IN give the same results. By default PostgreSQL and SQLite will get EXISTS as preferred subquery type. For other databases it will be easy to flip the feature flag if it seems EXISTS is better than IN for the DB. - Anssi -- You received this message because you are subscribed to the Google Groups "Django developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at http://groups.google.com/group/django-developers?hl=en. For more options, visit https://groups.google.com/groups/opt_out.