Re: Expensive queryset cloning
On Mar 17, 3:11 am, Alexander Schepanovski wrote: > Can you find that patch and post somewhere? > If not still thanks for this idea. Unfortunately, no. Gone with my old laptop. - Anssi -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Default project layout / directory structure
If we're talking about the lowest common denominator and keeping things simple, this is what I think we'd have: myapp1/ myapp2/ myproject/ -django.wsgi.sample -manage.py -management/ --commands/ ---myproject_mycommand.py.sample -media/ --myproject/ ---readme.txt -models.py -settings.py -static/ --myproject/ -templates/ --myproject/ -templatetags/ --myproject_tags.py.sample -urls.py -vendor/ --readme.txt -views.py otherapp1/ otherapp2/ Basically it's what we have now plus three things, samples of common optional files (which could include a link to the relevant docs page so people can get started right away), suggestions for naming conventions (use the app name as a prefix to avoid clashes with other apps) and a vendor folder for 3rd party reusable apps: - a sample wsgi wrapper - a sample management command showing that management commands should be prefixed with the app name to avoid clashes with other apps - a readme.txt reminding people that the media folder is for uploaded content (not static content) and suggesting that they use an `upload_to` value of `app_name/model_name/field_name` on their model file and image fields as a starting point - a static folder showing that static content should be placed in a folder named after the app to avoid clashes with other apps - a templates folder showing that templates should be placed in a folder named after the app to avoid clashes with other apps - a sample template tag library showing that template tag libraries should be prefixed with the app name to prevent clashes with other apps and suffixed with "_tags" to prevent quirky import issues (e.g. myproject_tags.py, myproject_form_tags.py) - a vendor folder that is automatically added to the python path (at the beginning, or after the project folder) and a readme.txt explaining that 3rd party reusable apps or specific versions of apps that exist elsewhere on the system should be placed here and imported with `from otherapp... import...` Another thing to point out is that a "project" is just an "app" with a settings.py file, a media folder, and perhaps a vendor folder. My big thing is that we should drive home the importance of decoupling or loosely coupling your apps. Unless you are 100% certain that you will ALWAYS want to bundle an app with your project AND and that you will NEVER want to use an app without your project, your other apps should exist as separate modules that can be placed anywhere on your python path. If Django were to automatically add the vendor folder to the python path as the second folder (after the folder containing your settings.py file), it would provide a standard way for users to include specific versions of 3rd party apps or their own reusable apps that may or may not be installed elsewhere on the hosting environment. For example, bundling a specific version of Django to be used even if another version of Django is already installed. I've recently decoupled a bunch of apps that were initially too tightly coupled to a common library that I use in most projects. As the apps grew and as we started working on more projects, it became troublesome to have to bundle this increasingly large library of apps when only a handful would be used by a project, and there were also times when I'd have liked to make use of just one in a project that a 3rd party had source code access to without giving them access to everything. I wish that I'd learned sooner to always try and build smaller, simpler apps with more limited and specific scope that were decoupled or more loosely coupled to other apps so that they could be more easily reused. Anything that Django can do to make this point more obvious would be a good thing. Cheers. Tai. On Mar 17, 2:21 pm, Simon Litchfield wrote: > On Mar 15, 9:46 am, Russell Keith-Magee > wrote: > > > On Fri, Mar 11, 2011 at 1:14 PM, Simon Litchfield wrote: > > > Who votes we should come up with a django-blessed 'official' default > > > project layout / directory structure? > > > Sure -- no disagreement that it would be good to have some common > > ground with regards to project layout. All we need now is to agree > > what that blessed project layout should be. :-) > > Lowest common denominator is what we're after here I think. Eg -- > > manage.py > --conf > settings.py > settings_dev.py > --lib > other_app1 > other_app2 > --my_app1 > --my_app2 > --media > --templates > --templatetags (project tags, non-app specific, if you have any) > --views.py (project views, non-app specific, if you have any) > --urls.py (project urls, non-app specific) > > Or maybe this -- > > manage.py > --conf > settings.py > settings_dev.py > (wsgi files) > --lib > other_app1 > other_app2 > --project > my_app1 > my_app2 > templatetags (project tags, non-app specific, if you have any) > views.py (project views, non-app specific, if you have any) > urls.py (project urls, non-app specific) > --media > css > img >
Re: Default project layout / directory structure
On Mar 15, 9:46 am, Russell Keith-Magee wrote: > On Fri, Mar 11, 2011 at 1:14 PM, Simon Litchfield wrote: > > Who votes we should come up with a django-blessed 'official' default > > project layout / directory structure? > > Sure -- no disagreement that it would be good to have some common > ground with regards to project layout. All we need now is to agree > what that blessed project layout should be. :-) Lowest common denominator is what we're after here I think. Eg -- manage.py --conf settings.py settings_dev.py --lib other_app1 other_app2 --my_app1 --my_app2 --media --templates --templatetags (project tags, non-app specific, if you have any) --views.py (project views, non-app specific, if you have any) --urls.py (project urls, non-app specific) Or maybe this -- manage.py --conf settings.py settings_dev.py (wsgi files) --lib other_app1 other_app2 --project my_app1 my_app2 templatetags (project tags, non-app specific, if you have any) views.py (project views, non-app specific, if you have any) urls.py (project urls, non-app specific) --media css img js uploads --templates This way we keep our pythons separate from our chickens. Also to suggest a preferred media structure or not- at least a centralised uploads / var media folder would be good practice I think, at least from provisioning and permissions etc point of view. Thoughts? These are just a few suggestions to get the conversation started! -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Expensive queryset cloning
> I had a patch for this problem somewhere, but can't find it now. > Basically it added inplace() method to queryset, and after that no > cloning of the inner query class would happen. The outer QuerySet > would still be cloned, but that is relatively cheap. This was to > prevent usage of the old reference to the QuerySet accidentally.* This > was done by keeping a "usage" count both in the inner query instance > and in the outer QuerySet instance. Can you find that patch and post somewhere? If not still thanks for this idea. -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Default project layout / directory structure
I am not currently involved in Django Development, but as someone new to Django I can endorse increased standardization & documentation in the area of project management & structuring. I have struggled to figure out the right set up for our project(s). I have also tried to grapple with coupling django projects, DRY, & pluggablity with a workable git setup (one repo, or many? submodules?), along with setting up virtualenv along with pip & fabric for a large application consisting of multiple projects. I realize this isn't the forum for my project structure as a django user (I've posted on that in django-users), but grappling with project management has been puzzling for me, being new to django, and wanted to provide my perspective to developers and encourage this thread. If more guidelines exist, more people may find themselves doing things in standard ways rather than coming up with unique structures because manage.py startproject doesn't quite fit their needs. Ben On Mar 14, 5:46 pm, Russell Keith-Magee wrote: > On Fri, Mar 11, 2011 at 1:14 PM, Simon Litchfield wrote: > > Who votes we should come up with a django-blessed 'official' default > > project layout / directory structure? > > Sure -- no disagreement that it would be good to have some common > ground with regards to project layout. All we need now is to agree > what that blessed project layout should be. :-) > > And there's the rub. There are a bunch of people with different ideas. > For example, take one idea in your list: virtualenv and pip. For many, > this won't be even slightly controversial. But for others, it will be > a complete show-stopper, because there is many a site that enforces a > 'package manager only' approach to system adminstration. > > Yours, > Russ Magee %-) -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Composite primary keys
On Mar 16, 2011, at 9:13 AM, Carl Meyer wrote: > I'm not expressing an opinion one way or another on composite primary > key syntax, but I don't agree here that a Django model "field" must > map one-to-one to a database column. That's fair, but a composite index lacks some of the characteristics of a field (assignability, for example). Most DBs don't have functions that explicitly iterate over indexes, so such a thing isn't really readable, either. It might be appealing to have a models.Index base class that represents an index on a table, and have db_index=True be a shortcut to creating one. That might be more machinery than we want just for composite primary keys though. -- -- Christophe Pettus x...@thebuild.com -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Composite primary keys
On Mar 16, 11:43 am, Christophe Pettus wrote: > On Mar 16, 2011, at 2:24 AM, Johannes Dollinger wrote: > > > I would be nice if support for composite primary keys would be implemented > > as a special case of general composite fields. > > It's appealing, but the reality is that no existing back-end actually has > such an animal as a composite field. In all of these cases, what we're > really creating is a composite index on a set of standard fields. > Introducing a more powerful index-creation syntax into Django isn't a bad > idea, but we shouldn't call it a "field" if it is not. I'm not expressing an opinion one way or another on composite primary key syntax, but I don't agree here that a Django model "field" must map one-to-one to a database column. It already does not, in the case of ManyToManyField, and at some point I would like to introduce (irrespective of composite primary keys) a more general ORM abstraction for composite fields (i.e. model Fields that map to more than one database column) as a path to cleaning up the implementation of GenericForeignKey. Carl -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Composite primary keys
On Mar 16, 2011, at 2:24 AM, Johannes Dollinger wrote: > I would be nice if support for composite primary keys would be implemented as > a special case of general composite fields. It's appealing, but the reality is that no existing back-end actually has such an animal as a composite field. In all of these cases, what we're really creating is a composite index on a set of standard fields. Introducing a more powerful index-creation syntax into Django isn't a bad idea, but we shouldn't call it a "field" if it is not. -- -- Christophe Pettus x...@thebuild.com -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Expensive queryset cloning
On Mar 16, 10:14 am, Thomas Guettler wrote: > Hi Alexander, > > I have seen this in my app, too. It still runs fast enough. But > I guess the django code could be optimized. > I had a patch for this problem somewhere, but can't find it now. Basically it added inplace() method to queryset, and after that no cloning of the inner query class would happen. The outer QuerySet would still be cloned, but that is relatively cheap. This was to prevent usage of the old reference to the QuerySet accidentally.* This was done by keeping a "usage" count both in the inner query instance and in the outer QuerySet instance. - Anssi (*) qs.filter(pk=1) qs.filter(foo=bar) would be an error but qs = qs.filter(pk=1) qs.filter(foo=bar) would be ok. -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Composite primary keys
I would be nice if support for composite primary keys would be implemented as a special case of general composite fields. There would be no need for new Meta options: class Foo(Model): x = models.FloatField() y = models.FloatField() coords = models.CompositeField((x, y), db_index=True) a = models.ForeignKey(A) b = models.ForeignKey(B) pair = models.CompositeField((a, b), primary_key=True) A CompositeField descriptor would then return a namedtuple of its values and would support queries: filter(coords__x=42) filter(coords=(1,2)) Adding the individual fields may be optional, e.g, CompositeField((FloatField(), FloatField()), db_index=True). This has been proposed before: http://groups.google.com/group/django-developers/browse_thread/thread/32f861c8bd5366a5 __ Johannes -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Expensive queryset cloning
Hi Alexander,
I have seen this in my app, too. It still runs fast enough. But
I guess the django code could be optimized.
Thomas
On 15.03.2011 01:49, Alexander Schepanovski wrote:
> I was optimizing my django app and ran into this. My app was spending
> too much time cloning querysets. I looked into code but didn't find
> any simple way to make it faster. But this is not needed actually. In
> most use cases "a parent" of a clone is thrown out. So usually one
> just need to mutate queryset not clone it.
>
> For example:
> Item.objects.filter(category=12).exclude(visible=False).order_by('-
> date')[:20]
> clones 4 times and 0 is needed. Using Q objects we can reduce it to 3
> (still too much) and complicate the code.
>
> Even
> Item.objects.get(pk=2)
> clones 2 times.
>
> Personally, I would like all querysets mutate not clone by default.
> And when one need a clone just make it explicitly.
>
--
Thomas Guettler, http://www.thomas-guettler.de/
E-Mail: guettli (*) thomas-guettler + de
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.
Re: Expensive queryset cloning
> We haven't yet, but had been planning on exploring a way to mutate the > existing class in most circumstances, but haven't > dug into it too much yet. I have and I use this monkey patch for now: https://gist.github.com/872145 -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Wrong error message when user having is_staff=False tries to login to admin
At the risk of bike-shedding this to death - if the current behavior included the correct message (user can't access the admin) - would we seriously consider a ticket asking to replace it with the current "misleading/more secure" message, for security's sake? On Wed, 16 Mar 2011 06:11:18 +0200, Tai Lee wrote: I also don't think it should be considered a security vulnerability to reveal that an authenticated user does not have permission to access the admin (or any other) app. If the credentials are valid and they authenticate against the defined authentication backends, then we should assume that we are talking to a trusted authenticated user and we should present error messages that are at the very least not misleading. Assuming that any authenticated user might be an attacker who has brute forced a password and presenting obscure error messages to authenticated users is not helping anybody. Cheers. Tai. On Mar 16, 3:41 am, "Brian O'Connor" wrote: 2011/3/15 Juan Pablo Martínez > The admin is not "one more app" is (if I may) the app with more weight > on most sites. Someone who has access to the admin has access to most > or all information. There is no "one more app. " This has nothing to do with the argument here. The account in question, as already stated many times, has no access to the admin site. That's the whole point of this discussion. Carelessness or neglect of a click in the admin should't call into> question the admin with the justification "that does not happen > again." This has to do with deliberately misleading users. I've been stuck by this at least once in my django career, and artemy has too. People make mistakes, it happens. > I think if everyone is going to fix "contrib" to your needs the > contrib lost all independence. I especially don't understand this statement. The whole point of django-developers is to discuss development of django, and by extension (because there are no other lists, as far as I'm aware) the contrib modules. Everyone comes here to help make the project better, to help fit their needs. That's the whole point, as far as I'm concerned. A reasonable suggestion was made, in which a few people came back and said that by doing this improvement, it would open a security issue. Myself, and others have stated that in fact, this would not be a security issue, and have provided examples. At this point, I'll absolutely never forget to check the is_staff flag purely because I've been following this discussion. What I don't understand is why there is such a huge opposition to the change. -- Brian O'Connor -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
