Re: Having a MongoDB connector for Django

['Tom Evans' via Django developers (Contributions to Django itself)]

On Fri, Sep 15, 2017 at 6:44 PM, Nes Dis  wrote:
> I would like to thank everyone for their valuable comments. Simultaneously I
> would like to comment on some conceptions regarding using MongoDB. Its not
> accurate to state that relational joins cannot happen in MongoDB. It can be
> done at the application level. LEFT JOIN and INNER JOIN. A detailed
> description of this is available.

I don't think anyone has said that this cannot be done, but instead
that it is not desirable to do so.

A django ORM adapter that supports Mongo would be used by people who
would not understand that, whilst you can do relational things using
mongo, it will not perform well.

In an earlier email you said:

> 2) Use a completely new wrapper to deal with non-rel DBs
> That's a bit ridiculous, because now I have to reinvent Django ORM,
> contrib packages and everything under the sun.

This is precisely the problem. If a mongodb connector was provided for
Django's ORM, people would expect to be able to use it with all these
packages that are designed to work with relational databases, and will
be annoyed when the project is slow once real amounts of data are
added.

If you want relational data, use a relational database, and use an
object relational mapping API to access it. If you want document data,
use a document store, and use an object document mapping API to access
it (PyMongo provides an ODM).

If you want both, choose one as your primary data store (I prefer
relational), all changes go through there. On modifying the primary
store, update the secondary store, according to your business
requirements (eg, instantly, batched ETL, daily, etc)

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFHbX1K%2BfhRL7%3DJRmBXk_3aAyK_WYD7c44zn00gZGjJthBfr-A%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Having a MongoDB connector for Django

['Tom Evans' via Django developers (Contributions to Django itself)]

Short answer: always use the appropriate tool

Relational databases and document stores have different uses and
purposes. Using a document store like a relational database (eg, with
an ORM (emphasis on the R)) is a bad idea, and using a relational
database as a document store is similarly foolish.

Work out what questions you want to ask of your data, then structure
the data in a way that allows you to query it efficiently.

If the format desired is a document store, I wouldn't attempt to
shoehorn that in to an ORM wrapper, I'd use something like mongothon.

Cheers

Tom

On Fri, Sep 8, 2017 at 8:50 AM, Nes Dis  wrote:
> Hello
>
> I am wondering what is the state of the art on Django having a backend
> connector for MongoDB database backend. There are a few solutions out there
> but they don't work as expected.
>
> A possible solution for this is to have a connector which translates SQL
> queries created in Django, into MongoDB queries.
>
> I would like to hear the expert opinion from the esteemed members of this
> group on this concept.
>
> A working solution for this can be found here: djongo. (Django + Mongo =
> Djongo) The project is hosted on github.
>
> Regards
> Nes Dis
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/b0ce04d1-62cb-4765-b850-06c4a5b0607f%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFHbX1%2BNpZuWrAgcMwkcJQ4Xg4TrM700JRsYV_AnW05_9L3joA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Automatic prefetching in querysets

['Tom Evans' via Django developers (Contributions to Django itself)]

Is this opt-{in,out} considered to be a global flag, meant to be
toggled on or off depending on whether it is an "expert" working on
the project or not?

I don't think that would be a good idea, almost all of our projects
have a mix of skill levels, and people move from team to team on a
regular basis. I don't think we have a project that has only been
worked on by senior developers or only worked on by junior developers.

What I think would be exceptionally useful would be to emit loud
warnings when developing (DEBUG=True or runserver) whenever this code
can detect that a prefetch MAY be applicable. A lot of our junior
developers are not as aware about DB structures and SQL performance
(particularly now that they don't design schemas and write DB
queries), so warnings as part of their normal development help trigger
teaching moments. Magically (sometimes) making things faster doesn't
teach them anything.

Turning this feature on/off per app is scary. If we're dealing with
models from AppA, we will get auto pre-fetching, but if we work with
models from AppB, we do not? If we're dealing with those models in the
same module, we will have different behaviour depending on which model
is being used? Please no.

I also think that it might be handy to specify
"qs.prefetch_related(auto=True)", or "with qs.auto_prefetch():", which
would then trigger the newly proposed behaviour. It's like "I want you
to prefetch all the things I use, but I don't know what just yet".
Having said that, I'd also like that to spit out a warning (again, dev
only) that specified what was actually prefetched, because why waste a
DB query in production to determine what we know whilst developing?

Cheers

Tom



On Wed, Aug 16, 2017 at 5:28 PM, Brice Parent  wrote:
> Le 16/08/17 à 14:07, Adam Johnson a écrit :
>>
>> I wouldn't want is to give free optimisations to non-ORM pros
>
>
> Something like 99% of django projects are by non-ORM pros, it can be easy to
> forget that on a mailing list of experts.
>
> I don't count myself as an ORM or SQL expert at all, but I've worked on at
> least 2 projects for which they had contracted such experts to optimise
> their projects once they reached some limits. Those experts only worked for
> them for a limited period of time, and if we changed this behaviour by
> default, I'm 100% sure they wouldn't understanding why their website was
> getting slower with the new release. And they would probably only see the
> performance issue on staging servers if they have some, or in production
> with a full database.
> I admit this can be a great thing for new comers and even small sites. But
> remember that, with or without this functionality, their website will need
> to be optimised in many ways when they start to have more users, and
> removing this 1+N common pitfall (or hiding the problem, as it is not
> completely solved as it's not the more efficient fix) will not be the only
> point they should look at.
> I think the solution for this would simply be to have it as opt-out, not to
> harm any already-working and optimised websites and allow to maintain a
> stable behaviour for 3rd party apps, but make it opt-in to new users just by
> pre-setting some setting in the settings.py file generated by "django-admin
> startproject". It could be a simple boolean setting or a list of apps for
> which we want the auto-fetch feature to be activated on. With this, new
> projects would be optimised unless they want to optimise manually, existing
> projects would still work without decreasing performance or creating memory
> problems in any case. Best of both worlds.
> And for existing non-optimised but existing websites, I prefer the
> status-quo than risking to create problems that might occur on production
> website.
> We could also, as proposed elsewhere in this thread, warn the developers (so
> only when settings.debug is set) when this kind of 1+N queries are executed
> that it could be way more efficient, either by activating the auto-fetch
> functionality or by manually prefetch related data.
>
> - Brice
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/dc483976-f7d0-3aee-4f9e-76f515e5c761%40brice.xyz.
>
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to th

Re: Time based one time password and django ?

['Tom Evans' via Django developers (Contributions to Django itself)]

Also django-two-factor, which builds on django-otp and provides all
extra bits you might need, eg setup views, QR code generation for
device registration, login wizards etc supporting HOTP/TOTP, static
tokens, Yubikey and SMS.

https://markusholtermann.eu/2016/09/2-factor-authentication-in-django/

Cheers

Tom

On Mon, Jan 16, 2017 at 4:38 PM, Alexander Dutton
 wrote:
> There's also , which is fairly easy
> to integrate into a Django project.
>
> As a long-term user of (but rare contributor to) Django I'd say the ease
> of using one of a number of third-party solutions points to keeping it
> out of core.
>
> Relatedly, integrating custom auth (i.e. not just username/password)
> would be easier if the Django admin site deferred to LOGIN_URL by
> default instead of presenting its own login form. An example issue is
> when one uses some sort of web server SSO module with
> RemoteUserMiddleware, and then the admin site presents asks for a
> username and password for already-authenticated non-staff (who have no
> local credentials).
>
> Yours,
>
> Alex
>
>
>
> On 16/01/17 16:28, Gavin Wahl wrote:
>> I have a project that implements TOTP and U2F as a third-party
>> package: https://github.com/gavinwahl/django-u2f
>>
>> On Sunday, January 15, 2017 at 3:47:56 AM UTC-7, ludovic coues wrote:
>>
>> Hello,
>>
>> After reading the recent thread on authentification in django, I
>> wondered about the chance of getting a 2-step auth mechanism in
>> django.contrib.
>>
>> Time based one time password, or TOTP, is now part of the RFC 6238.
>> For those who don't know it, it use a shared secret and current time
>> to produce 6 digit number. That number change every 30 seconds and is
>> used to confirm login after entering a correct username and password.
>>
>> As far as I can tell, there is no such thing present in django
>> currently. But I don't know if it's because nobody have done the  work
>> or if there are reason to not include 2-step solution in django.
>>
>> --
>>
>> Cordialement, Coues Ludovic
>> +336 148 743 42
>>
>> --
>> You received this message because you are subscribed to the Google
>> Groups "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send
>> an email to django-developers+unsubscr...@googlegroups.com
>> .
>> To post to this group, send email to django-developers@googlegroups.com
>> .
>> Visit this group at https://groups.google.com/group/django-developers.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/django-developers/7a3b3837-5c24-4984-abb8-d68d9ce31459%40googlegroups.com
>> .
>> For more options, visit https://groups.google.com/d/optout.
>
> --
> You received this message because you are subscribed to the Google Groups 
> "Django developers  (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/fb94cbfa-5987-4aa7-e74a-6fa53ce05cce%40alexdutton.co.uk.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFHbX1J02Kz62Th38x_USngaBQ-tKOd3DJH0OUyp4FFyMC879Q%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Django 2.0 Python version support (Python 3.6+ only?)

['Tom Evans' via Django developers (Contributions to Django itself)]

On Thu, Jan 5, 2017 at 8:10 PM, Asif Saifuddin  wrote:
> Hi,
>
> django 2.0 will be released in december 2017 and ubuntu 18.04 will be
> released in april 2018 which will default atleast 3.6, so I think this
> should also be taken as consideration while deciding.

I know supporting endless versions of python is not desirable, but
please bear in mind that some of us are in situations where what the
latest release of Ubunutu is not really relevant.

Our organisation uses CentOS 6, which is not EOL until the end of
2020. In CentOS 6, the stock version of python is python 2.6; we go
through special measures (EPEL) to get that up to python 2.7. If we
wanted to make the move to Python 3, we'd be talking about Python 3.4,
again through EPEL.

We have an infrastructure team responsible for provisioning servers,
and it is on their schedule that OS upgrades occur - it is not easy
for us as developers to argue that this team should spend significant
resource to upgrade to a later OS version or to roll custom python
RPMs.

Updating all of our codebase to Python 3 is going to be a pain for us,
it is hard to argue a business need with "Everything stays exactly the
same but is slightly more secure and easier to maintain".
Realistically, when we move to Python 3, it will be because the
supported Django LTS requires it. If we also have to jump through lots
of hoops to get the very latest Python 3 release, it won't make it
easier to argue, it will mean we are more likely to postpone it and
keep using old django versions, particularly on internal intranet
sites.

If there is a way that Python 3.4 support can be maintained without
significant detriment or penalty, this would be greatly appreciated by
those of us running more conservative enterprise distributions.

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFHbX1L%2BxkpR%3DJ6K%2BPTvD6OZT34ZPDgw62HXnUmi5BUrPOm5%2Bg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: #27485 new New feature Allow ALLOWED_HOSTS to accept an IP-range / wildcard

['Tom Evans' via Django developers (Contributions to Django itself)]

On Sat, Nov 19, 2016 at 1:01 AM, Florian Apolloner
 wrote:
> On Thursday, November 17, 2016 at 5:07:07 PM UTC+1, Tom Evans wrote:
>>
>> Or:
>>   from socket import gethostname, gethostbyname
>>   ALLOWED_HOSTS = [ gethostname(), gethostbyname(gethostname()), ]
>
>
> That a) adds your hostname and b) (assuming you properly configured your
> system) 127.0.0.1  -- so as long as they are using 192.* to access the site,
> this does not help.

Our servers are configured such that "localhost" resolves to
127.0.0.1, and the hostname resolves to the local IP of the server.

I don't think our servers are in any way misconfigured, or configured
in a "special" manner - my laptop is configured in precisely the same
manner out of the box.

The offered solution works correctly on all of our development and
production servers, and also on our developers local machines running
various versions of Linux.

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFHbX1K_1dCLrMQm4cy0u1i1cnEzLJV%2Bb_1-p9n58ERV7%3Dghvg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: #27485 new New feature Allow ALLOWED_HOSTS to accept an IP-range / wildcard

['Tom Evans' via Django developers (Contributions to Django itself)]

On Wed, Nov 16, 2016 at 10:29 PM, Thomas Turner  wrote:
> Hi
>
> The other day I raise a ticket to do with ALLOWED_HOSTS and it was suggested
> that I post on this group before creating a patch
> The ticket is https://code.djangoproject.com/ticket/27485
>
> The problem
>
> Now that Django 1.10.3 forces ALLOWED_HOSTS on with debug I have a problem.
> The problem is that when I developing I use an ip-address range of
> 192.168.1.*.
> The Ip-address can change depending on the computer I use.
> so I required the ability to add wildcard partly in an hostname.
> ie
> ALLOWED_HOSTS = ['192.168.1.*', '.mydomain.com']

Or:
  from socket import gethostname, gethostbyname
  ALLOWED_HOSTS = [ gethostname(), gethostbyname(gethostname()), ]

?

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFHbX1LndZK4DR7Lvqd3YeOBJoT33PpnGW-Vr84mF%3Ds1zdTQFQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Adding an some warnings, checks, or exceptions for urls

['Tom Evans' via Django developers (Contributions to Django itself)]

On Mon, Apr 25, 2016 at 1:00 PM, Robert Roskam  wrote:
> I realize this is a really, really obvious and easily fixed newbie error.
> But here it is:
>
> urlpatterns = [
> url(r'^sitemap/',
> TemplateView.as_view(template_name='brochure/sitemap.html'),
> name="sitemap"),
> # More urls
> ]
>
> Sitemap should end with dollar sign.

The problem is that the url should *usually* end with a dollar sign.
There are valid use cases for a url not starting with a caret or
ending with a dollar.

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFHbX1JcWdZbdLs3HzD-z671tdDbXO2TU%3DQLf_UvjjRbySGYEw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Provide a way to pass kwargs when initializing the storage class

['Tom Evans' via Django developers (Contributions to Django itself)]

On Sat, Nov 7, 2015 at 11:58 AM, Raphaël Barrois
 wrote:
> Hello,
>
>
> The core of the proposed solution seems quite interesting; however, it also 
> introduces a new configuration format for backends.
>
> Caches and databases use a dict with a "BACKEND" key and an "OPTIONS" dict 
> for kwargs to pass to the backend.
> Likewise, entries in the TEMPLATES list are dicts with a "BACKEND" key and 
> various options.
>
>
> Do you think that the new setting should match these options instead of the 
> proposed two-tuples?

I like explicit:

STORAGE = {
'default': {
'ENGINE': 'path.to.foo.Storage',
'OPTIONS': { ..}
},
}

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFHbX1JGG%2BJJStg%2BRK9ZZzXRi0zyi4QUjSLqQqNqoh1KGU2XnQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: #25582: Add a way to build URLs with query strings

['Tom Evans' via Django developers (Contributions to Django itself)]

On Fri, Oct 23, 2015 at 12:59 PM, guettli  wrote:
> API proposal:
>
> Add a new kwarg to reverse():
>
>reverse(..., get=None)
>
>
> Example: reverse('my_view_name', kwargs={'pk': '1'}, get=dict(param='value')

Would 'get' be a dict or a querydict? (URL parameters can be repeated,
dict keys cannot.)

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFHbX1LmPAtgX1jykxYb-qx1eVn6ig4LLM96w0s0LNuomr7iTg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Making max_length argument optional

['Tom Evans' via Django developers (Contributions to Django itself)]

On Tue, Sep 22, 2015 at 1:49 AM, Podrigal, Aron
 wrote:
> Different schemas?? Schema will always be different for each database
> backend according to its datatypes. I really don't understand what your
> concern is. In any case your free to specify a max_length=N where it will be
> the same for all backends.

This change would allow a django site running on postgres that you
could not dump the data from and reimport in to mysql without data
loss. It would make it easy to write non-portable apps without
thinking.

>
> While this makes sense, the use of TextField is also for the purpose of
> having rendered a text widget for ModelForms. So we should allow a
> max_length of None for both.
>

(I'm replying to two emails from you in one, hope that is alright)

No, TextField is used to designate an arbitrary length text field.
Having a widget.Textarea as the default input for this field is just a
default, it is perfectly correct to override the defaults either for
CharField or TextField to provide the desired widget type.

One should not be using models.TextField because a widget.Textarea is
desired for forms, instead it is to tell the database backend that the
column for this field should be an arbitrary length text type.

If the problem that is to be solved is that there is no easy way to
specify an arbitrary length text field that uses an input, this can be
solved with a very simple class:

class TextFieldWithInput(models.TextField):
def formfield(self, **kwargs):
defaults = {'widget': forms.TextInput}
defaults.update(kwargs)
return super(TextfieldWithInput, self).formfield(**defaults)

The main argument seems to be "Its a pain to think about the size of
my data when defining my database tables"; if we aren't thinking about
it then, then when do we think about it? Its kind of important..

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFHbX1LbVfR4LdaqfF2S266dxhYRKA5B_dvbBA17c720spJKeA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Making max_length argument optional

['Tom Evans' via Django developers (Contributions to Django itself)]

On Mon, Sep 21, 2015 at 5:21 PM, Paulo Maciel
 wrote:
> +1 max_length optional
>

I'm slightly worried from a DB point of view. Whilst blessed
PostgreSQL supports variable length fields seemingly without issues,
other DB engines (MySQL, for instance), have significant performance
issues using a TEXT field versus a VARCHAR field - particularly on
indexed fields, it must do a row read to retrieve data, rather than
use an indexed value.

Remember that VARCHAR columns can be fully indexed, whilst TEXT
columns can only have the first N (specified by user) characters
indexed, and reading the data will always require a row read, even if
the text is shorter than the indexed size.

In the ticket, it states that

"""
If I read the oracle docs correctly, specifying a length is mandatory
for VARCHAR2, and the maximum is 4000 bytes. For MySQL, it's 65K
bytes, for PostgreSQL and SQLite it's very large.
"""

For MySQL, it is 64k (65,535 bytes is *not* 65k!), and that is not the
whole story - 64k is the maximum record size - all your columns must
fit within that limit, not just one column in the record.

http://dev.mysql.com/doc/refman/5.7/en/column-count-limit.html

I'm slightly concerned as a MySQL/MariaDB user that this will lead to
"reusable" apps that are not performant on MySQL. Given that we
(generally) let Django define the database tables based upon models,
it seems much more important that the definition is correct and high
performing in all supported backends than to avoid choosing what
database columns you require in the tables - after all, that is what
the TextField is for, to allow the programmer to specify "this is an
arbitrary length text field".

I'm all for DB engines that support arbitrary length varchar fields to
use that when you specify a TextField. I'm very not keen on having
MySQL represent the majority of char fields as TEXT because no-one
could be bothered to specify the appropriate length.

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFHbX1KnWTfw6Hk18Buu3hZOiHVB2RW4JNk_PFdaO_QoLK6CmA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Get current user in model signal pre_save

['Tom Evans' via Django developers (Contributions to Django itself)]

On Wed, Sep 9, 2015 at 4:11 PM, Xavier Palacín Ayuso
 wrote:
> I want to collects current user in model signal pre_save, to prevent remove
> super user permission to current super users.

Hi Xavier. This mailing list is for discussing the development of
django itself, not for discussing how to develop in django.

For help with developing in django, please use django-users mailing
list (and be patient - just because no-one has answered you in 6 hours
does not mean you should kick it up to django-developers).

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFHbX1LmaqFniJr5gLMkfcewasCm6EW8nZjDpnMHan9jrSdndg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: request for API review of streaming responses additions

['Tom Evans' via Django developers (Contributions to Django itself)]

On Mon, Sep 7, 2015 at 3:04 PM, Aymeric Augustin
 wrote:
> 2015-09-07 10:00 GMT+02:00 Yann Fouillat :
>> I agree, do you know what tools could I use to emulate 3G ?
>
> As far as I know, the canonical tools are:
>
> - on Linux, netem:
> http://www.linuxfoundation.org/collaborate/workgroups/networking/netem
> - on OS X, Network Link Conditioner:
> https://developer.apple.com/library/ios/documentation/NetworkingInternetWeb/Conceptual/NetworkingOverview/WhyNetworkingIsHard/WhyNetworkingIsHard.html
>

As well as these tools, there is similar functionality built in to the
chrom(e|ium) browser.

Inspect the page, toggle "Device mode" (click the phone icon next to
"Elements" tab), and options to throttle the network (with various
presets) will appear on the page.

I don't know a way to throttle network in chrome without also toggling
device emulation however, so the page will look different (within a
viewport).

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFHbX1L70nEAhUSOovkjLBH0CyRxABx1yUCvcVwAz5k45DDJvg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Feature: Support a javascript template language on the server

[Tom Evans]

On Sat, May 30, 2015 at 5:52 PM, Emil Stenström  wrote:
> Hi,
>
> This is the second feature proposal as part of my general drive for getting
> Django to work better for javascript heavy sites.
>
> Support a javascript template language on the server
> 
>
> The multiple template engine work has made it possible to switch out Django
> Templates with another engine. One of the most powerful things this enables
> is to use a template language that is executable both on the server and
> client.
>
> A typical use-case for this could be that you have a list of tweets on your
> site. When the user first loads the page you send the full HTML for all the
> rendered tweets over. When there's a new tweet you would like to just add
> that one tweet to the list, without re-rendering the whole page. Now you
> have a couple of options:
>
> 1. Reimplement the Django template code from your site in javascript. This
> is harder the more complex the template your have. Also risks bugs when you
> change the server-side template but forget the client-side one.
>
> 2. Use Pjax, render everything on the server and send the result to the
> page. This sends unnecessary data over the wire, and requires that you
> figure out how to append the data in the correct location.
>
> 3. Send the server-side templates to the browser, and then re-render on the
> client when new data arrives. This uses the least data over the wire, and
> does not require that you keep track of mappings, just update the data and
> re-render everything (React.js's virtual DOM can make this really fast if
> you need it to be).
>
> Option 3 opens up lots of interesting use-cases for Django, that previously
> was only possible for javascript-based frameworks.

ISTM that django already provides all these "options" (you are clearly
slanting the view that "options" 1 and 2 are rubbish and only option 3
is a the valid choice..). Pluggable template engines landed in 1.8,
read here for extensive details:

https://myks.org/en/multiple-template-engines-for-django/

With pluggable template engines, is there anything else required in
Django core to plug in whatever template language your project
requires?

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFHbX1JeebcCzYWnESY7wEFFWpQ6WFJPJ7u4XW%3DDAcV_Um_19w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: An easier path to upgrade from one LTS release to another

[Tom Evans]

On Thu, May 7, 2015 at 9:04 AM, Abdullah Esmail
 wrote:
> Hello,
> I apologize if this has been discussed already. I searched the topics, but
> didn't find anything.
> First, I'd like to thank all of you for the truly amazing framework. I've
> been using django since 1.0 and it made my life much more easier and
> enjoyable and _stable_.
>
> The reason why I love django is that stability is a high priority,
> backward-compatibility is well maintained and clear, crystal-clear and
> extra-detailed documentation, the "batteries-included" aspect, based on
> python, and the steady and stable evolution of the framework. Everything is
> well put together in an almost-perfect package. Thank you.
>
> To my main issue, have you explored the possibility of creating a direct
> path from one LTS release to the next?
> I think the "official" way of doing this now is to go through all releases
> in-between (1.4 -> 1.5 -> 1.6 -> 1.7 -> 1.8).
> It would be really great if there was a direct LTS-to-LTS path (1.4 -> 1.8).
>
> I'm not sure if the new system check framework can play a role here to make
> this easier than before.
> This might add a whole new layer of complexity and extra work, but I believe
> it's worth it.
> If it makes any difference, I don't mind helping with this and actually
> bringing this feature to life.
> I'm not that familiar with the internals of django, but I'm willing to learn
> and get my hands dirty.
>
> I guess I should ask, is there a good reason why there is no direct upgrade
> path between LTS releases?
> Do you guys think it's not worth the extra effort and the current way is an
> acceptable trade-off?
>
> Thank you,

What would this feature look like? In order to go from one LTS to the
next LTS, you will need to do various things in turn, in order to
reflect the reality of changing 4 (or however many) minor versions one
at one time.

The easiest way to do that is to step through each of the intermediate
versions release notes in turn, applying the changes for that release
before moving on to the next one. I don't see what advantage collating
them all in to one document would have, given that you will still need
to do each set of steps in turn.

Cheers

Tom

PS: We are following this approach at $JOB - stick to LTS until the
next LTS and then upgrade via each minor release. Or I guess "will
be", 1.8 being only the second LTS ;)

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFHbX1JSurjb11HKhjVUa%2BHCOr7B6Yw1Kjh7Pi_S17Ua7m1Jsw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Pass exception to error handlers

[Tom Evans]

On Wed, Apr 22, 2015 at 3:05 PM, Tim Graham  wrote:
> I have some concerns from a security standpoint. For example, some exception
> messages are definitely not meant to be displayed to end users and may leak
> server implementation details. For example:

This is saying you can't have a gun because you might shoot yourself
in the foot, but then how do you shoot the bear?

The error handler is under the developers control, so what they choose
to do with the exception is their business. The default implementation
need not show anything more than is currently available, but it could
be replaced with something that does what the developer needs, and it
would be their responsibility that they keep their toes, so to speak.

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFHbX1J-uiNoP9OuHqX3ekPj77k%3DmbEbWsevRnakmYevBQu5EQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Two phase cleaning of models/forms

[Tom Evans]

On Wed, Mar 25, 2015 at 9:37 AM, Thomas Güttler  wrote:
> Up to now the validation/cleaning of a attribute can not access the
> value of an other attribute.
>
> The work around is to override the clean() method of the form/model.
>
> A team mate and I talked about it, and we asked ourself, if
> a two phase cleaning would help here.
>
> Our idea:
>
>  - The first clean phase works like the current implementation
>
>  - The second clean phase is optional and does nothing except the user
> implements it.
>
>  - In the second clean phase your are allowed to peek into
>the results of the first clean phase.
>The clean method for the second phase would need to get the other values
> passed in.

How is this different from the existing behaviour of clean_FOO(),
followed by clean()? Is that not "clean phase 1" and "clean phase 2"?

On Wed, Mar 25, 2015 at 2:14 PM, Preston Timmons
 wrote:
> There are times when I've definitely wanted this feature. Particularly, when
> multiple fields on a form have this type of constraint. Putting all the
> logic in the clean method gets convoluted.

You can place each multi field constraint to be tested in to its own
method and call them from clean() if it gets too unwieldy.

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFHbX1LMrvA4zE2yt%3DhLV-SW%2B6dzP8paUDZR2qfWc%3Dr2-TAcuA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: A general way to batch SQL queries in Django

[Tom Evans]

On Fri, Feb 27, 2015 at 3:19 PM, aRkadeFR  wrote:
> What do you mean by a single roundtrip?

He means asking the database server to consider multiple queries
(concurrently?), and return data once all of them are available. In
pseudo code:

people, jobs, cities = DB.fetch_multi(
Person.objects.all(),
Job.objects.all(),
City.objects.all())

MySQL's C API supports executing multiple SQL statements in a single
round trip, the data sets are made available in turn to the client API

http://dev.mysql.com/doc/refman/5.7/en/c-api-multiple-queries.html

however, each statement is executed in turn, not concurrently, so the
only speed up you would have is that there is only one query to parse,
and a few small packets less sent to the database server... The
database connection is not torn down between requests to the database.

The only way I could see it having any non-marginal effect is if you
have high latency to your database server. But then you have a lot of
problems.

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFHbX1%2BMgfwHL%3DL8hjgRMyf%2B2-Rap9L01PF9cxERFqU6c8uHdA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Testing email settings

[Tom Evans]

On Fri, Feb 27, 2015 at 9:01 AM, aRkadeFR  wrote:
> Hello,
>
> I've written couple of time similar command for my project too.
>
> But if I take a step back, these commands (for my projects) are
> only here to test that my SMTP settings are well setup. Thus,
> the test sending email is quite unnecessary, I would like a check
> that connects to the SMTP server (if the emails settings are
> setup else do nothing) when the application starts.
> (I don't know if there's something like saying to a SMTP server:
> am I allowed here? without sending an actual email)
>
> Have a good day

In Simple Mail Transfer Protocol terms, it definitely is (EHLO, MAIL
FROM, RCPT TO, then disconnect without sending DATA). But SMTP is not
the only mail backend, and smptlib does not expose that level of
connection detail - it will merely raise a different exception type if
any of those commands do not return 250 status.

Hey, that time spent reading RFC 821 at uni just paid off ;)

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFHbX1J0tS-QCCW6xJBuH0QuutPtV8S0m3zZo3VznR9YCTud3g%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Settings: lists or tuples?

[Tom Evans]

On Thu, Jan 22, 2015 at 12:32 PM, Andreas Kahnert
 wrote:
> Hi again,
> Well, I can acknoledge that your reasons for list (beginner friendly) are as
> good as my reasons for tuples (seems to be more logical choice for things
> that are static). To say it in other words, my idea was simply: Use tuples
> and the programmer will know that these arn't ment to be altered at runtime.

It is incorrect to say that the reason for lists is "beginner
friendliness". It may well make things friendlier for beginners, but
what it definitely helps with is for more advanced deployment
configuration and for modifying settings from base or default settings
files.

Secondly, settings are *only* altered at run time. The module pointed
at by DJANGO_SETTINGS_MODULE is imported, its code is run, and the
settings object is produced. From this point on, yes, settings should
not be modified, but that tells nothing about how they can be modified
within the settings module during runtime but before being frozen in
the settings object.

>From my POV therefore, it is doubly wrong to have tuples and say this
is because the values are meant to be immutable, because it does not
reflect reality.

For instance:

  from project.default_settings import *
  DEBUG=True
  INSTALLED_APPS.append('django_debug_toolbar')
  LOGGING['loggers']['homepage'] = { 'handlers': 'logfile', 'loglevel': DEBUG }

All of this modification is perfectly valid, and this is why (imo) the
default should be list instead of tuples.

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFHbX1Ja3SOQyskMDbykmyguJRQvcnLtRatHn2qhOfYgWnqCVQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Proposal of new feature: handler to HttpResponseNotAllowed (status 405)

[Tom Evans]

On Fri, Oct 3, 2014 at 10:17 PM, Poorbahrdew .poorbahrdew
 wrote:
> Hi guys,
>
> I know, this is a quite old thread, but recently I had the same issue with
> these decorators for the same reason: how to handle 405 responses in user
> friendly way?
>
> I have found a simple solution which doesn't require to write your own
> middleware and I want to share it with people facing the same problem in the
> future :).

In my original response last year, I was trying to explain that this
is handled by middleware, not that you have to handle it with your own
custom middleware. I then listed out the error handlers that django
provides, noting that handler500 will handle *all uncaught
exceptions*.

Therefore, if you want to do something special with a particular
exception, like a custom exception like HttpMethodNotAllowed, you can
catch this with a custom handler500 and handle it there.

No patching of views required.

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFHbX1KKrPkorj-zB3OBD80A4bvR%3DWAxWwyc3W9%3DbGye7WFO-w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Please don't kill the ecosystem

[Tom Evans]

On Tue, Sep 2, 2014 at 2:53 PM, Aymeric Augustin
 wrote:
> 2014-09-02 15:33 GMT+02:00 Tom Evans :
>
>> this story was scored
>> at 8 points, it took a junior developer much longer than 8 points and
>> wasn't finished in a single sprint - and 1.3->1.4 was *easy*
>
>
> I don't know how much a point or a sprint is worth in this context :-/

Sorry, context is important! "A sprint" ~ 9 work days, points vary
wildly from sprint to sprint unfortunately - 12 is about a sprints
worth of work.

> 

Good steps.

>
> Did your junior developer follow a similar process? If he did, what took
> them
> so long?
>
> What I want to understand is how much of an advantage I have when it comes
> to
> upgrading Django. What do I know that isn't written down and makes it so
> hard
> for others?

I think it was more distraction by topics he had not come across. We
set him off by saying "look at the release notes, go thru each change
in turn, see if we are affected and what we need to fix it".

The problem then was that he was new. He hadn't had to deal too much
with TZ support, so adding TZ support to our project meant a day of
learning about how TZ work in UNIX.

Similarly, changing the project layout is a good thing, but then he
spent a day learning about package layouts - not a bad thing by any
means, but in the 1.3 -> 1.4 release there are 75 similarly complex
enhancements/deprecations/incompatibilities - it just takes some time
to go through if you have not come across that topic before, eg,
clickjacking.

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFHbX1%2B%2Bzdz0GK2jvJJ5tH1zLAi3agc3JNFhN4nWEXBBaLPATw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Please don't kill the ecosystem

[Tom Evans]

On Mon, Sep 1, 2014 at 9:45 PM, Pkl  wrote:
>
> Hello,
>
> I once was once lured to an ideal of long-term stability and
> retrocompatibility, by nice docs like this one :
> https://docs.djangoproject.com/en/dev/misc/api-stability/
>
> But for some years, stuffs have actually been getting worse and worse, with
> each django release bringing its little crowd of nightmares.
>

Our solution is to not chase releases. Django 1.4 is marked as "Long
Term Stable", all our projects use Django 1.4 and no more; if/when a
newer stable release is proposed we will consider moving our projects
to that. We do then have some problems with 3rd party apps that do not
support 1.4..

There was a thread a few weeks ago that discussed (perhaps
tangentially) what the next LTS release branch will be, where it was
brought up that whatever release it is, the API will actually have to
be stable(!), probably ruling out 1.7. So, LTS will probably be 1.4
for some time yet.

On Tue, Sep 2, 2014 at 1:48 PM, Aymeric Augustin
 wrote:
> I'm not sure I understand completely where your frustration comes from. The
> tone of your email seems disproportionate with the effort needed to replace
> all instances of "django.conf.urls.defaults" with "django.conf.urls". That
> takes about ten minutes, if you count the time to locate the information in
> the release notes and make the commit.

It's not really, though is it? You encounter the problem because
you've upgraded django version; this will not be the only thing that
is broken, and so fixing this one thing will not enable you to run
your test suites.. it can take several iterations before you know that
it is fixed.

If you track django development and have used django for several
years, then these changes are relatively easy. If you are new to using
django, it is not so obvious and easy; as an example, we had an old
site running on django 1.3 that we wanted to move to 1.4 so that we
are using the same version of django throughout; this story was scored
at 8 points, it took a junior developer much longer than 8 points and
wasn't finished in a single sprint - and 1.3->1.4 was *easy*! We
prefer to spend our points on our own features (and senior developers
are expensive :)


Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFHbX1%2BW2E0UO4kZZBwGHf%3DbA4%2BH4-u95T3jtGkUVX03ZOzQuQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Improvement to objects.get_or_create and objects.update_or_create

[Tom Evans]

On Wed, Aug 27, 2014 at 10:19 PM, Benjamin Scherrey
 wrote:
> I don't believe the functionality is backwards incompatible at all unless
> I'm missing something. The new behavior of automatically selecting the
> optimal search field (prioritized by pk first then by any discovered field
> marked as unique) would only occur if the 'default' parameter was None.

So, if I passed in kwargs={'pk': 7, 'foo': 'bar', 'wibble': 'quuz'},
with "foo" and "wibble" being non-indexed fields, then the query that
would be run by get_or_create to determine whether the item exists
would differ in your new version.

This means it is not backwards compatible, as if I passed in those
arguments to get_or_create(), then I expect one of three things to
happen:

  1) the object returned to have those values
  2) an object is newly created that has those values
  3) an error occurs because an object could not be created that has
those values.

The proposed change breaks that contract, eg it would simply return
whatever item has the pk 7, regardless of 'foo' or 'wibble'
attributes.

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFHbX1J-EL%3DPV2x9dZevZzcaWG%2BE30O18nf3gFrLQKjFoK1O9A%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: The low down on the "Unsettings" project

[Tom Evans]

Google "django-development unsettings". It's been discussed at length on list.

Cheers

Tom

On Fri, Jun 20, 2014 at 8:06 PM, Andy Baker  wrote:
> This sounds really interesting. Is there anything about this in writing? I'm
> not a fan of listening to audio on tech subjects (must... skim... read...).
> Not sure if anyone else shares this prejudice but do post here if there is a
> follow-up blog post or similar.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/8997f769-6905-4e11-8a40-80b6a23d63ae%40googlegroups.com.
>
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFHbX1Jw%3DAJafEoB7aFg6%2BsuhsGq2Xc9qkeAAcPZ06r3qWtfNA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Terms for database replication

[Tom Evans]

On Thu, Jun 5, 2014 at 8:07 PM, Justin Holmes  wrote:
> OK.  How about "canon" and "replica" ?

I think we all get sent to the naughty step if we continue to discuss
this, so I will just say that "master" is being used as an adjective
in this context, it is the master database, or the master connection,
or indeed master branch. "Master" as in slavery is when it is used as
a noun. A few dictionary definition of the adjective "master":

"of or pertaining to a master from which copies are made: master film;
master matrix; master record;master tape."
"Being an original from which copies are made."
"an original from which copies can be made; especially :  a master
recording (as a magnetic tape)"

No other word is suitable because this word universally encapsulates
this concept, not just in IT, but in all aspects of all trades that
make copies of things.

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFHbX1J5mV-PD%2BecOhEn-3yhd2ay7VzSohQUy6Kw1-R8OsnE7w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Terms for database replication

[Tom Evans]

On Thu, Jun 5, 2014 at 5:08 PM, Justin Holmes  wrote:
> I don't want to devolve completely into an etymological circlejerk here, but
> my sense is that "master" in the VCS sense is like "master key," rather than
> describing the interpersonal relationship of involuntary servitude.
>

And in databases, it means the connection on which you can "master"
records. Words have many meanings.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFHbX1LbZACH7MPdmz7_M_x1pvhjH-d%2BjV%3Dya4S%2BVvTTGPheMA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Terms for database replication

[Tom Evans]

Please revert this change as soon as possible.

If the project has become so PC sensitive that the word "slave" is no
longer permitted to be uttered, then "replica" is an alternate term,
but "primary" is not.

Have you ever set up "primary-primary replication"? No, neither have
I. Master-master replication is common, please do not take it upon
yourselves to re-program our vocabulary.

I have a primary master, I do not have a primary primary!

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFHbX1%2BUa81AruHH7mRK9SyQCABQrprPDd7sdTRrNN0_DpNPOg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Why not Single Table Inheritance?

[Tom Evans]

On Thu, May 15, 2014 at 4:11 PM, Christian Schmitt
 wrote:
> This is already merged.
>
> https://docs.djangoproject.com/en/1.6/topics/db/models/#multi-table-inheritance
>

MTI is not STI, nor is it polymorphic.

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFHbX1%2BUzp1VpY7JNK4%2BCV%3Djr8Ave7Gt7jbtW6JMs9B_KVacsw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Proposal: update_fields shortcut

[Tom Evans]

On Wed, Mar 5, 2014 at 2:05 PM, Paulo Poiati  wrote:
> Hello everyone,
>
> While developing standard web applications one thing I frequently do is
> updating just one field of a model instance and save it after that. The best
> way to do that today (if you want to avoid the update in all the fields) is
> with the code bellow.
>
> product.name = 'Name changed again'
> product.save(update_fields=['name'])
>
>
> I want to propose a shortcut for this kind of operation. I came up with two
> solutions.
>
> Solution One
>
> product.save(update_fields={'name': 'Name changed again'})
>
>
> Solution Two (My preferred)
>
> product.update_fields(name='Name changed again')
>
>

Solution Three (existing)

Product.objects.filter(pk=product.pk).update(name='Name changed again')

?

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFHbX1%2Bxf77_vMPK881gaQuE9EKf6uyKiZHEMS%2BMPev3hWzQNw%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: GSOC 2014 Project Proposal

[Tom Evans]

On Sat, Feb 22, 2014 at 5:31 PM, Devashish Badlani  wrote:
> Sir,
>
>
> Sample projects with the updated Django 1.6.2,use of latest modules in each
> of them and an helpful documentaion ,would certainly enhance the value of
> DjangoBook is what I feel
>

How would this work? The book currently admonishes readers that:

"""
The community edition of The Django Book is in transition. While the
book mentions Django version 1.4 in places, the vast majority of the
book is for Django version 1.0
"""

So you will write sample projects aimed at 1.6.2 for each chapter in a
book written for 1.0? This does not seem wise.

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFHbX1JKaQGjj8DjyKHqQeC0MyEs4AYwUu7X0PSTnLkN_CsLcQ%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Django ORM support for NoSql databases

[Tom Evans]

On Tue, Dec 17, 2013 at 4:50 PM, Chris Wilson  wrote:
> Hi all,
>
>
> On Tue, 17 Dec 2013, Tom Evans wrote:
>
>> On Tue, Dec 17, 2013 at 3:35 PM, parisrocks
>>  wrote:
>>>
>>> Waiting for official Django ORM support for NoSql databases.
>>
>>
>> MongoDB (and the vast majority of NoSQL databases) are not relational
>> databases, they are document oriented databases, they store documents of an
>> unspecified type and allow you to retrieve documents by id, or perhaps other
>> attributes of the document.
>>
>> At no point with MongoDB can you have relationships between two different
>> documents that can be followed or queried. Therefore, I'm quite confused
>> what you want from an "ORM" interface to a NoSQL database. Each document has
>> no relationships, there is nothing for an "ORM" to do...
>
>
> I think it would be more helpful to say that it IS possible to store the ID
> of a document in a field of another, of course, and that means that
> documents CAN have relationships:

I really don't. Yes, some NoSQL stores can do semi-relational things
(like RethinkDB), and you can, as you demonstrate, store raw ids in
the document and use application logic to "make joins", but I think it
clouds the issue and makes people think that NoSQL datastores can
somehow store their relational data in it and perform relational
queries on it, if only those darn developers would write it.

It is nonsensical. The 'R' in the ORM is important; if you cant
perform equivalent relational queries in the same manner as all the
relational databases currently supported by the ORM, then what is the
purpose of attempting to shoe-horn it into the same API?

Relational and document databases serve different purposes, if you are
attempting to do relational logic in a document database, then IMO
you're starting out from an invalid position. Nothing prevents you
from using both concurrently, of course.

I guess the one thing that could be done would be to write a django
API interacting with all the different NoSQL engines, but not
attempting to mimic the ORM. This would give a "django nosql"
solution, without trying to force a square peg into a round hole.
Three downsides to this:
  1) SQL backends all generate SQL (duh) which means there is a lot of
shared code in the ORM.
  2) Different stores python adapters can have very different
interfaces and features, making it hard to define a base feature set.
  3) It's a document database, there is very little involved in
looking up and retrieving a document, it's easier just to use the
specific adapter for the store you want directly.

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFHbX1LNLwknbVXy_Amo1qaQ%3DUxrBmB7RBrbWJ%2BADE9VCuxYsg%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Django ORM support for NoSql databases

[Tom Evans]

On Tue, Dec 17, 2013 at 3:35 PM, parisrocks
 wrote:
> Hi Django Users,
> I tried Django recently and really liked the simplistic approach for
> building sites.
> But there's no official support for NoSQL databases like Cassandra or
> MongoDB, there's a great community of NoSQL users waiting for an official
> ORM support from Django like me. I would say Django with NoSQL ORM support
> could actually make it more popular compared to its rivals Ruby on Rails or
> even Java for that reason.
>
> Waiting for official Django ORM support for NoSql databases.
>
> Thanks
>

ORM stands for Object-Relational Mapper. It is a tool for expressing
relational databases - databases in which objects are related to other
objects - as a way of presenting an object oriented view of that data.

MongoDB (and the vast majority of NoSQL databases) are not relational
databases, they are document oriented databases, they store documents
of an unspecified type and allow you to retrieve documents by id, or
perhaps other attributes of the document.

At no point with MongoDB can you have relationships between two
different documents that can be followed or queried.

Therefore, I'm quite confused what you want from an "ORM" interface to
a NoSQL database. Each document has no relationships, there is nothing
for an "ORM" to do...

NoSQL databases can already be used within django as document stores,
simply install the appropriate python adapter and go nuts.

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFHbX1KVvSJ_B49KwZE3eWz51t37fg88KY0yPpnc1BT5tkv4Hw%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: mod_python support

[Tom Evans]

On Mon, Nov 25, 2013 at 5:09 PM, Ramiro Morales  wrote:
> Now that specs like WSGI exist there is no point in going back one decade
> implementing interfaces to cater for one particular web server integration
> implementation.

Is that any worse than where we are now, using an interface that only
allows integration with apps written in one language? I'd hoped that
integrating web apps in to web servers had been solved by FCGI, but
along came python...

Cheers

Tom

PS - Not criticising any decision here, we use wsgi/mod_wsgi for our
python apps. it's just a pain to have to use something else for the
wordpress sites, something else for the perl sites (only have one of
these left now, thank $DEITY!)

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFHbX1Jqz7T3eGxojesq58vKTxUEFYUW%3DPdQNCYw6d3BBM_i1g%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Add strutctured settings module to django 1.7?

[Tom Evans]

On Tue, Oct 1, 2013 at 6:42 PM, Tino de Bruijn  wrote:
> To all of you who don't see benefit in putting certain (definitely not all!)
> settings in the environment, I would like to say: try it. It will finally
> make sense that you set DJANGO_SETTINGS_MODULE in the first place. I never
> understood why anybody would want that until this :D.
>
> I was sceptical first as well, but tried out Heroku and therefore this
> method. I have since converted all projects to a structure where environment
> specific variables are stored, well, in the environment. Still my settings
> is a module, where prod.py and dev.py inherit from common.py, but that is
> mainly because certain apps are not necessary on either dev of production
> servers. So that is all checked into the repository. I don't see any
> necessity to store those environment vars in a repository, as they contain
> login keys and static files buckets etc.

Tracking what login keys are assigned to a specific instance of an
application is the entire purpose of configuration management. The
reason we keep code in VCS is so that we can be assured what is
deployed is coherent and consistent, the same is true for
configuration, it must be known and verifiable.

The entire basis of my position is that configuration is essential
code which must be tracked and be verifiable with an audit history of
changes. If you don't feel that to be the case, you are unlikely to
see the benefit in your configuration.

The second benefit is in consistency. Both approaches take the same
overall method, one file contains secrets/host specific configuration
for this server, the second contains generic configuration, and the
two are combined. With the environment variables way, where is your
host specific configuration? It could be in .login, in a gunicorn
launch script, in a shell script - anywhere. How is the configuration
defined? Are you "setenv foo bar" because this host is Solaris and
you're running csh, or is this an "export foo=bar". Perhaps they are
in your supervisord configuration.

Compare to using a python file - you have full power of the python
interpreter to do whatever you need. Everything is in one language,
the same language as your code. All configuration will apply
regardless of how you start the server, whether it is supervisord
launching a wsgi process or you running runserver. There is no
requirement for this file to belong to the same repository as the
code, or even be in a repository.

There is absolutely nothing that using environment variables as a
source of configuration gives you that you cannot do using a python
configuration file and it makes your configuration management messy
and chaotic.

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFHbX1LTavB%2BreZDONOAmq2OZvtsEhcCc5q5ymn3JAisy3iALg%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Add strutctured settings module to django 1.7?

[Tom Evans]

On Thu, Sep 26, 2013 at 2:21 PM, Andres Osinski
 wrote:
> Honestly, I find the idea of *not* using a settings file to be inconvenient,
> and the notion of using environment variables for doing anything aside from
> pointing to a settings file to be pretty disgusting.

+1

The idea that before you start working on a project you need to infer
and set the 50 or so environment variables required is anathema to me.

The most important concern is that these settings are not part of your
code, they are part of your configuration. Consistency of
configuration is one of the most important things in DevOps, and so
IMHO the most important thing in your project is tracking and managing
the changes in your configuration.

To me, that means one thing. Configuration files live in VCS
repositories - this is the only way to track changes to your
configuration, to validate that there are no unexpected changes to the
file.

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
For more options, visit https://groups.google.com/groups/opt_out.

Re: get_cache and multiple caches

[Tom Evans]

On Mon, Sep 23, 2013 at 5:55 AM, Anssi Kääriäinen
 wrote:
> On 09/20/2013 06:29 PM, Tom Evans wrote:
>>
>> On Fri, Sep 20, 2013 at 4:13 PM, Florian Apolloner
>>  wrote:
>>>>
>>>> It seems more sensible to hook something that has the lifetime of the
>>>> request to the request, rather than stick it in TLS, keyed to the
>>>> thread serving the request.
>>>
>>>
>>> Jupp, sadly I don't see a sensible way around thread local storage here
>>> :(
>>>
>> All good points, I just have this mental "HMM" when I see TLS as the
>> solution for anything. TLS is already used for things like the
>> language code of the current request, which avoids you having to pass
>> around a state object or passing down lang_code all down the stack,
>> but means that things like URL reversing and resolving benchmarks are
>> slow (O(n²)) with USE_I18N on.
>
>
> Huh? What is the n here? And why would passing the lang_code down the stack
> help?
>
>  - Anssi

n is the number of URLs.

In LocaleRegexProvider get_language() is called each time the reqexp
is requested - not just each time the regexp is compiled.
LocaleRegexProvider being the base class of RegexURLPattern and
RegexURLResolver

https://github.com/django/django/blob/master/django/core/urlresolvers.py#L160

Each time get_language() is called you do a TLS get. If you have a
large number of URLs, get_language() is called once for each URL
pattern that is checked against the current URL. If you have thousands
of URLs, and the current URL does not match (or matches one of the
last ones listed), then you have thousands of unnecessary TLS fetches
per URL resolve.

The fetches are unnecessary since the layer above could look up the
current language once, and pass that down. That pattern would not
allow the magic for regex() to be hidden behind a @property, and so
instead expensive TLS lookups occur inside regex() instead of inside
the caller of regex().

See also "URL dispatcher slow?" from last October, and:
  http://mindref.blogspot.co.uk/2012/10/python-web-routing-benchmark.html

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
For more options, visit https://groups.google.com/groups/opt_out.

Re: get_cache and multiple caches

[Tom Evans]

On Fri, Sep 20, 2013 at 4:13 PM, Florian Apolloner
 wrote:
>> It seems more sensible to hook something that has the lifetime of the
>> request to the request, rather than stick it in TLS, keyed to the
>> thread serving the request.
>
>
> Jupp, sadly I don't see a sensible way around thread local storage here :(
>

All good points, I just have this mental "HMM" when I see TLS as the
solution for anything. TLS is already used for things like the
language code of the current request, which avoids you having to pass
around a state object or passing down lang_code all down the stack,
but means that things like URL reversing and resolving benchmarks are
slow (O(n²)) with USE_I18N on.

The problem with tying everything to the request is that you end up
with code that only works with requests or request-like-objects. The
problem with not tying everything to the request is that things get
slower. :(

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
For more options, visit https://groups.google.com/groups/opt_out.

Re: get_cache and multiple caches

[Tom Evans]

On Fri, Sep 20, 2013 at 3:10 PM, Florian Apolloner
 wrote:
> The main issue here isn't recreating the objects on demand, but the impact
> they have, eg a new memcached connection. Now imagine a complex system where
> each part issues get_cache('something') to get the cache

On the other hand each call to get_cache('foo') now requires access to
TLS. TLS is slw. Going through something slow to get to something
that is supposed to be a speed up...

> and you'll end up
> with a few connections per request instead of one.

Would it be better to leave the API and semantics of get_cache alone
and provide a new way of accessing caches through the request object,
leaving them cached on the request object for the duration of the
request, and thus avoiding the need of TLS.

The reason to use TLS is because requests are commonly served from
threaded servers, and cache clients may not be thread safe, so a
desire to stop different requests from simultaneously accessing the
same cache client. Requests aren't shared between threads, and so a
per request cache would be inherently thread safe.

It seems more sensible to hook something that has the lifetime of the
request to the request, rather than stick it in TLS, keyed to the
thread serving the request.

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
For more options, visit https://groups.google.com/groups/opt_out.

Re: get_cache and multiple caches

[Tom Evans]

On Wed, Sep 18, 2013 at 12:29 PM, Curtis Maloney
 wrote:
> I started working on a CacheManager for dealing with thread local cache
> instances, as was suggested on IRC by more than one person.
>

The problem that Florian identified was that recreating cache
instances each time get_cache() was called could be costly. Before you
go too far down the thread local route, could you verify that
retrieving cache objects from a thread local cache is in any way
faster than simply recreating them as demanded.

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Support Negative Indexing on QuerySets

[Tom Evans]

On Thu, Aug 1, 2013 at 10:44 AM, Loic Bistuer  wrote:
> On Aug 1, 2013, at 4:05 PM, Tom Evans  wrote:
>
>> qs = ...
>> print len(qs)
>> print qs[0]
>> print qs[-1]
>> print qs[0]
>>
>> How many queries for this?
>
> Just one and "qs[-1]" will return the last element of the cached result.
>
> I'm not trying to be pedantic, I'm just pointing out that a queryset becomes 
> a different beast once it has been evaluated; it's basically a simple list of 
> cached result.

Yes you are right, I was mistaken in thinking that indexing would
evaluate the queryset if not evaluated - I think it should tbh,
equating qs[10] to either qs[10:11].get() or object_cache[10] does not
seem right, but impossible to change existing accepted behaviour.

I do not like that the behaviour of qs[10] changes whether the qs is
evaluated or not, or that iterating through a queryset by index could
be woefully slow unless the developer explicitly evaluates the
queryset beforehand - and if you're a developer who thinks that it is
right to iterate through a queryset using an index, you probably would
not be aware of the need to evaluate it first.

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Support Negative Indexing on QuerySets

[Tom Evans]

On Wed, Jul 31, 2013 at 7:39 PM, Loic Bistuer  wrote:
> In your example "print qs[0]" evaluates a *clone* of "qs", not "qs" itself.
>
> Therefore "qs[0]; qs[-1]; qs[0]" triggers 3 queries, just like "qs[0]; qs[0]; 
> qs[0]" would.
>

Fine, be pedantic:

qs = ...
print len(qs)
print qs[0]
print qs[0]

This is one query.

qs = ...
print len(qs)
print qs[0]
print qs[-1]
print qs[0]

How many queries for this?

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Support Negative Indexing on QuerySets

[Tom Evans]

On Tue, Jul 30, 2013 at 11:04 PM, Wim Lewis  wrote:
>
> On 30 Jul 2013, at 2:06 PM, Florian Apolloner wrote:
>> How do you think such support would look like? For negative indices you'd 
>> have to know the size of the resultset to be able to do "limit somthing 
>> offset length-your_negative_index" -- this doesn't seem to make any sense 
>> for an ORM. You can always do list(qs)]:-1] though…
>
> It seems like the first comment in the ticket answers that question. Django 
> would reverse the sense of the query's ordering clause and use a simple LIMIT.
>

What would it do if the query had already been evaluated? IE, how many
queries does this run?

  qs = Model.objects.filter(…).order_by(…)
  print qs[0]
  print qs[-1]
  print qs[0]

If it is as simple as reversing the order by and negating the index,
then the programmer themselves can do this, there is no need for
magic. Having code that looks like it is a simple operation, but is in
fact changing the logic of when queries are evaluated is evil in my
opinion.

-1

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Deprecate FCGI support in Django 1.7

[Tom Evans]

On Sun, Jul 21, 2013 at 8:28 AM, Some Developer
 wrote:
> On 21/07/2013 05:08, Curtis Maloney wrote:
>>
>> I wasn't aware there was a particular performance issue, but I'll
>> certainly keep it in mind.
>
>
> Take a look at this:
>
> http://www.peterbe.com/plog/fcgi-vs-gunicorn-vs-uwsgi
>
> I've probably already said it but if you want some help with this project
> I'd be interested.

Did you notice that the first thing that anyone does when benchmarking
HTTP servers is that they reduce the work being done by the server so
that it is trivial. That link explains it best, if you throw away
everything that Django provides and simply use it to render a static
string, then uWSGI is faster than gunicorn, which is faster than FCGI.
If you leave any of it enabled however, they all benchmark at
precisely the same speed.

So, if your django website does not use sessions nor databases nor
templates, then sure, you get a huge performance benefit from fiddling
around with serving mechanisms.

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Proposal of new feature: handler to HttpResponseNotAllowed (status 405)

[Tom Evans]

On Mon, Jul 8, 2013 at 10:01 AM, Jorge C. Leitão
 wrote:
> Django allows users to define handlers to some exceptions, most notably
> http404, server error (status 500) and permission denied (status 403).
>

In fact, django already allows you to install handlers to process any
kind of unhandled exception, by installing middleware with a
process_exception() method that handles exceptions of that type.

> I here propose this feature to be extended to HttpResponseNotAllowed (status
> 405).
>

This is a HttpResponse subclass, not an exception. handle404 does not
get involved if you return a HttpResponseNotFound response from your
view, only if you raise a http.Http404 exception. Basically:

Middleware can handle any unhandled exception before the other
exception handlers are involved.
handle403 handles uncaught django.core.exceptions.PermissionDenied exceptions.
handle404 handles uncaught django.http.Http404 exceptions.
handle500 handles all other uncaught exceptions.

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Perception of attitude in tickets

[Tom Evans]

On Mon, May 13, 2013 at 4:10 PM, Jacob Kaplan-Moss  wrote:
> Hi Tom --
>
> It really sucks that when I say "if you have feedback please send it
> over here", you hear "I'm not listening".
>
> I'm sorry, but I don't have the mental bandwidth to follow 20,000
> individual tickets. It's impossible. I just fucking can't do it.
> Believe me, I've tried, and failed, many times. I'm sorry I'm such a
> slacker.

I don't think anyone is asking you to do this. This ticket in question
wasn't lacking bandwidth from committers, it was visited many times by
committers, who each time summarily dismissed the ticket - "We're not
doing this because x years ago, God said thus". There was enough
mental bandwidth for it to be covered 4 times over.

I'm aware that I'm simplifying a complex scenario by reducing it to a
single case, but this is the kind of response I've seen all over
tickets, emails and so on.

You're not the only person who has time constraints, each of has a
choice of what we work on in our spare time. When I read these sorts
of tickets, perfectly valid feature requests knocked down for
precisely no reason, why should I waste my time trying to jump through
these arbitrary hoops? The short answer is I don't, I work on my own
projects.

What I read in the OP was someone who felt in a similar situation.

>
> I *do* have the bandwidth to follow a single mailing list. If you want
> my attention, that's how you get it.
>
> If you really want to help, if you really want to get a positive
> outcome from this, then how about you give me a hand and follow (part
> of) Trac for me? Watch some tickets, and if/when they get stalled
> bring them here.

Ho hum, so a lack of engagement with parts of the community is my fault is it?

I have to stay current with Django to be competent at my job, which
means following these mailing lists. I also benefit from Django, so I
feel compelled to help out as I can, which I do by contributing to
users@.

When it comes to open source work, you can't compel people to work on
what you want them to work on, people will work on what they want to
work on. If you make it so that it is hard for them to work on it,
they won't work on your project. If you look objectively at both mine
and the OPs posts, ignore the bits that make you angry, maybe you will
see that you're losing potential contributors. Maybe you won't, I'll
go back to lurking.

This is your project, so how you structure it and accept feedback is
entirely under your control. You could set something up to mail
developers@ iff a ticket is re-opened after being closed if you want
the discussion on here and not on tickets. At the moment, you invite
people to make comments on tickets and then ignore them.

The problem that I thought this thread was discussing was "Why do lots
of people feel there is an engagement issue". If only I had known that
it was about finding mugs to do triage..

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Perception of attitude in tickets

[Tom Evans]

On Mon, May 13, 2013 at 3:51 AM, Russell Keith-Magee
 wrote:
> My apologies if I wasn't clear - that wasn't what I was saying at all. What
> I meant is that we can't institute a process like "Every core developer must
> spend 4 hours per week triaging tickets or they will lose their core
> developer status". This would be a completely reasonable course of action if
> you were a paid employee -- your employer is just telling you what you have
> to do to get paid -- but that dynamic doesn't exist in a volunteer project.
> In a volunteer project the only reason the "hard" stuff gets done is because
> people volunteer to do it.
>
> However, in this case, Jacob *did* give a detailed explanation:
>
> "This seems like a needless function; it's already possible to just
> re-look-up the object from the database."
>
> It was rejected because the need wasn't clear. Simon then reopened the
> ticket, and gave a detailed use case, to which Jacob responded:
>
> "I'm really not convinced by Simon's use case -- adding "reload()" only
> saves you a single line of code. Let's do our best to keep Django as svelte
> as possible."
>
> What more detail should Jacob have provided? The feature isn't that complex.
> It's not like he's got an opportunity to present a PhD thesis in relational
> algebra. It's a simple feature, which has been rejected because in Jacob's
> opinion, it can be achieved in other ways.
>
> Jacob didn't explicitly call for a discussion on the mailing list. Perhaps
> he should have. However, when the ticket was reopened for the second time,
> James Bennett (ubernostrum) pointed at project policy to have these
> discussions on the mailing list.
>
> What should James have done instead?
>

Perhaps the issue is that there is a feeling that no-one is listening
to the community? This particular issue was shot down because a core
dev didn't like the style of the change. They felt that adding a
function to be explicit about reloading an object is wrong since it
bloats django.

This was decided 5 years ago, communicated in a single line. Any
attempt by the community to say "wait a minute, I'd really like this"
gets shot down and told to bring it here, or worse. 3 days ago, the
response was "If you want it, you have to make it happen."

Perhaps this wasn't clear, that was a member of your community trying
to make it happen. There have been several attempts over the past 5
years by people trying to make it happen. Each time someone has tried
to make it happen, after the initial attempt, the ticket has been
re-closed "BFDL already said no, just go away".

In this case, happy days, a few hours of discussion on the ML, and
this ticket is now accepted. You need to understand that not all of
your users are comfortable or capable of advocating on mailing lists,
but are happy to contribute to tickets. With this ticket, 5 years of
asking for this feature on the ticket was pointless, and this is what
the OP is railing against. You gave people a way to contribute, but
then ignore them. Perhaps "ML or GTFO" if not the right approach for
attracting contributors.

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [GSoC 2013] Revamping validation functionality proposal

[Tom Evans]

On Fri, Apr 26, 2013 at 8:54 AM, Christopher Medrela
 wrote:
> Thank you for your feedback. I really appreciate every comment because that
> let me improve my proposal.
>
> 1. First of all, I noticed that the license of django-secure is copyright.
> Google forces us to release code under one of approved license [1], so a
> question to Carl Meyer: do you mind changing license?
>

The license is BSD 3-clause:

https://github.com/carljm/django-secure/blob/master/LICENSE.txt

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Ticket 20147 - deprecate and replace request.META

[Tom Evans]

On Mon, Apr 8, 2013 at 9:02 PM, Luke Plant  wrote:
> Hi all,
>
> This is already the subject of a ticket, but I didn't get a response
> yet. Basically, the idea is replace things like:
>
>   request.META['HTTP_ACCEPT']
>
> with
>
>   request.HEADERS['Accept']
>
> request.META should be deprecated and replaced with request._META,
> because it is just an implementation detail, and a really bizarre one at
> that, full of cruft from a previous generation of web applications (CGI)
> that should not be exposed in our API.
>
> Anything else needed from META should also be replaced with a sensible API.

request.META is not just headers. What happens to the things that are
arbitrarily decided to be "not needed"? Who decides that HTTP_HOST is
important, but SERVER_PROTOCOL is not?

>
> This might seem to be a big change simply for the sake of a clean API,
> but I'm more and more motivated by these thoughts:
>
> * Web development is hard enough as it is. "Explain this to a newbie
> without getting embarrassed" is a good test.
>
> * A general philosophy of pro-actively keeping things clean and sensible
> is necessary to avoid being overrun with madness long term.
>
> There is also the advantage of a *much* cleaner repr(request),
> especially in a development environment, because you don't have all the
> other junk from os.environ.

The basis of web development was CGI. From CGI came everything.
Therefore, almost all webservers - Apache, nginx - almost all
frameworks - RoR, PHP, Django - that deal with the web will have
similar or equivalent environment hashes like this.
The convention for headers (uppercase header name, s/-/_/g, prepend
'HTTP_') is similarly ubiquitous.

Your argument is that this structure is confusing to the absolute
newbie who has never programmed a web application in his life. My
counter argument is that changing it would be confusing to anyone who
has ever programmed a web application in the last 20 years.

>
> The biggest problem is what to do with our test Client and
> RequestFactory, which allow specifying headers as keyword arguments
> using the CGI style of naming e.g.:
>
>   self.client.get('/path', HTTP_X_REQUESTED_WITH='XMLHttpRequest')
>
> Since keyword arguments can't have "-" in them, this is harder to
> replace. We could either leave it as it is, or add a new 'headers'
> keyword argument to these methods, deprecating **extra.

This seems a problem with the API to Client and RequestFactory, not a
problem with request.META

>
> The silliness has infected other places, like SECURE_PROXY_SSL_HEADER
> which follows the same CGI convention (and in each case the docs have to
> note how to do it correctly!). In this case we can detect people using
> the old style 'HTTP_' and raise a deprecation warning, and allow the
> sensible way.

Again, to me that seems like the flaw is in the code handling
SECURE_PROXY_SSL_HEADER. Yes, that code does need to look up the
header name in request.META, it does not need to be specified in that
format.

>
> We would probably also need to add a few methods/attributes to
> HttpRequest to proxy the few things you need from request.META that are
> not headers, like REMOTE_ADDRESS and REMOTE_USER
>
> Is anyone strongly opposed to this? If not, in Aymeric's spirit of
> decisiveness, I'll push it forward.

Changing this just in Django seems wrong - in fact it says that the
only thing of relevance in request.META is headers.

One man's cruft is another man's dinner. Everything in request.META is
of interest to someone, somewhere. When I'm debugging a failed
request, anything in request.META may be of interest.

If the intention is to make header handling easier for beginners to
understand, it might make more sense to normalize how one accesses and
sets headers in django at the same time. Having code like this:

   response.HEADERS['Wibble'] = request.HEADERS['Wibble']

makes more sense than:

  response['Wibble'] = request.HEADERS['Wibble']

I'd be -1 against dropping request.META, -1 on changing anything in
repr(request), +1 on adding re(quest|sponse).HEADERS that reads from
META/writes to _headers.

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: URL dispatcher fallthrough?

[Tom Evans]

On Mon, Mar 18, 2013 at 3:23 PM, julianb  wrote:
> Hi,
>
> imagine the following use case:
>
> You build an online store where you have sorted products into several
> categories and maybe associated an occasion. Now you want to build URLs. So
> the URL schema that all of the store's owners agree on is:
>
> //
> //
> //
>

Really straying into user talk now...

If you have a heterogeneous hierarchy of objects that have slugs, and
you wish to look up arbitrary objects by slug, perhaps just go the
whole hog, add a SluggedItem model with a generic foreign key, and a
view that fetches items from their slugs and dispatches to the
appropriate view.

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Documenting lazy() and memoize()

[Tom Evans]

On Mon, Mar 11, 2013 at 2:50 PM, Tom Evans  wrote:
> Hi all
>
> Someone just asked on users@ "How do I reverse a URL inside
> settings.py". I gave a solution (eventually; got it wrong the first
> time!) using two functions from django.utils.functional, lazy() and
> memoize().
>
> Neither of these two functions are documented. and so aren't part of
> the API - really I shouldn't have been suggesting them.
>
> Is this deliberate? They seem perfectly useful and correct
> implementations of lazy() and memoize(), and for places like this,
> very useful.
>
> If it is not deliberate, I can whip up some documentation so that they
> can become blessèd parts of the API.
>
> Cheers
>
> Tom

Any thoughts on this please?

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Documenting lazy() and memoize()

[Tom Evans]

Hi all

Someone just asked on users@ "How do I reverse a URL inside
settings.py". I gave a solution (eventually; got it wrong the first
time!) using two functions from django.utils.functional, lazy() and
memoize().

Neither of these two functions are documented. and so aren't part of
the API - really I shouldn't have been suggesting them.

Is this deliberate? They seem perfectly useful and correct
implementations of lazy() and memoize(), and for places like this,
very useful.

If it is not deliberate, I can whip up some documentation so that they
can become blessèd parts of the API.

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: postgresql range types v2

[Tom Evans]

On Thu, Jan 17, 2013 at 10:08 PM, Russell Keith-Magee
 wrote:
> The other thing I'd suggest is to look at others doing similar work. For
> example, Zachary Voase has been working on extensions to Django's PostgreSQL
> backend to support a whole lot of extra PostgreSQL features[1]. From the
> look of it, he hasn't got to range types yet, so what you've done here could
> probably be added as part of that project. PostgreSQL has tons of great
> features, and it would be great to be able to expose them in a clean way to
> Django developers; better yet would be to expose them all as a single
> extension library.
>
> [1] https://github.com/zacharyvoase/django-postgres
>

We use the django-postgres-netfields library [1] extensively at $JOB,
which allows you to store v4/v6 IP addresses and netmasks as native
types in the database, and perform IP-like queries on them, eg:

Foo.objects.filter(mask__net_contains_or_equals=request['REMOTE_ADDR'])

Perhaps worth looking at as well?

Cheers

Tom

[1] https://github.com/adamcik/django-postgresql-netfields

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Django 1.1 is not installable

[Tom Evans]

On Wed, Dec 12, 2012 at 3:56 PM, Michael Elsdörfer  wrote:
> $ pip install django==1.1

If you mean "The most recent point release in the 1.1 family", then
that is "Django>1.1,<1.2"*.
If you mean 1.1.1, then that is "Django==1.1.1"

Cheers

Tom

* If you are using pypi, then "Django<1.2" will do the trick. However,
if you are running your own cheeseshop, you may have imported 1.0.x
and 1.2.x but no 1.1.x - being explicit about the versions you want
avoids confusion.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: [ANNOUNCE] Security releases (Django 1.3.5, Django 1.4.3, Django 1.5 beta 2)

[Tom Evans]

On Mon, Dec 10, 2012 at 10:38 PM, James Bennett  wrote:
> Django 1.3.5, Django 1.4.3 and Django 1.5 beta 2 have just been issued
> in response to security issues.
>
> Details are available here:
>
> https://www.djangoproject.com/weblog/2012/dec/10/security/
>

Is the second part of the fix in any way optional? For instance, one
of my sites is a SAML/SSO identity provider - people log into it in
order to be granted access to other sites in the federation, and it is
routine for the "next" parameter during authentication to refer to any
site in the federation, not just the local domain.

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Yet another __ne not equal discussion

[Tom Evans]

On Fri, Nov 30, 2012 at 2:51 PM, Marek Brzóska  wrote:
>
>
>
> 2012/11/30 Tom Evans 
>>
>> On Fri, Nov 30, 2012 at 1:29 PM, Marek Brzóska 
>> wrote:
>> > Has the matter been completely put away?
>> >
>> > I would like to bring it up again.
>> >
>> > I have Articles and Categories. An Article belongs to Categories:
>> >
>> > class Category(model):
>> >   pass
>> > class Article(model):
>> >   category = ManyToManyField(Category)
>> >   status = CharField(max_length=10)
>> >
>> > Now I want all categories with articles that have status different than
>> > "archived".
>> >
>> > I cannot do this with django's ORM!
>> >
>> > Category.objects.filter(~Q(article_status="archived"))
>>
>> What precisely is wrong with:
>>
>> Category.objects.exclude(article_status='archived')
>
> It excludes all categories that have at least one archived article.
>
> And I want categories that have at least one NOT archived article.
>
> Example: I have one category: politics. I have two articles in this
> category: "Vote Obama!" which archived and "U.S wars" which is not archived.
> Category.objects.exclude(article_status='archived')
> will show no categories, while I want my only category to show, because
> there is one not archived article, "U.S. wars".

That's a very different question:

Category.objects.exclude(article__status='archived').annotate(num_articles=Count('article')).filter(num_articles__gt=0)

Still answerable via the ORM.

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Yet another __ne not equal discussion

[Tom Evans]

On Fri, Nov 30, 2012 at 1:29 PM, Marek Brzóska  wrote:
> Has the matter been completely put away?
>
> I would like to bring it up again.
>
> I have Articles and Categories. An Article belongs to Categories:
>
> class Category(model):
>   pass
> class Article(model):
>   category = ManyToManyField(Category)
>   status = CharField(max_length=10)
>
> Now I want all categories with articles that have status different than
> "archived".
>
> I cannot do this with django's ORM!
>
> Category.objects.filter(~Q(article_status="archived"))

What precisely is wrong with:

Category.objects.exclude(article_status='archived')

>
> will result in a query like this (simplified for readibility):
> SELECT * FROM
> category LEFT OUTER JOIN articlecategory ON category.id =
> articlecategory.category_id
> WHERE NOT
> "category"."id" IN (
>   SELECT U1."category_id"
>   FROM "articlecategory" U1 INNER JOIN "article" U2 ON (U1."article_id" =
> U2."id")
>   WHERE
>   (
>   U2."status" = 'Archived'
>   AND U1."category_id" IS NOT NULL
>)
> )
>
> The key problem here is that the ~Q() part negates whole big part of SQL
> query, and instead of getting all categories with at least one not archived
> article, I get all categories that have no archived articles! You must
> agree, that it is not very unlikely that someone wants to query for sth like
> this. I cannot use Q(__lt)|Q(__gt) becouse this is not integer. If status
> was enumerated, I could select all values that are different than
> "archived". But in this case I cannot do anything (except custom sql which
> seems like overkill to such simple need).
>
> Adding not equal operator (say __ne) would allow to do this really simply:
> Category.objects.filter(article_status__ne="archived")
>
> potentially resulting with query like this:
> SELECT * FROM
> category LEFT OUTER JOIN articlecategory ON category.id =
> articlecategory.category_id
> WHERE
> "category"."id" IN (
>   SELECT U1."category_id"
>   FROM "articlecategory" U1 INNER JOIN "article" U2 ON (U1."article_id" =
> U2."id")
>   WHERE
>   (
>   U2."status" != 'Archived'
>   AND U1."category_id" IS NOT NULL
>)
> )
>
> My question is why deny user such obvious usage of ORM?
>
> And btw, "not_in" would be really usefull too, argument the same, but I want
> to exclude three statuses ("archived", "deleted" and "draft").

Category.objects.exclude(article_status__in=[...])


Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Django and South

[Tom Evans]

On Tue, Nov 20, 2012 at 4:29 AM, Лебедев Илья  wrote:
> I've realized that django has no migration tool in development and that
> looks weird for me. Everybody use South, but django neither includes it in
> contrib nor uses it in development.
> What's the big deal? Can someone please explain.

https://groups.google.com/group/django-developers/browse_thread/thread/e8eede37f396e59e?hl=en-GB&noredirect=true

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Feature Request: Support for abstract models in ModelForm

[Tom Evans]

On Tue, Nov 13, 2012 at 7:04 PM, Rohit Banga  wrote:
> I have some abstract models and I have created several concrete
> implementations of each of these models. The usecase is create multiple sets
> of tables across each having the same set of fields. When I use abstract
> models for modelforms it works fine until I get a unique key in the picture.
> Details of the problem I faced are detailed in this django ticket.
> https://code.djangoproject.com/ticket/19271
>
> Working version of code: (Works just by chance - Apparently it is not
> supported)
> https://github.com/iamrohitbanga/django_ticket_19271/tree/code_working
>
> Non-Working verison of the code (unique key in abstract model)
> https://github.com/iamrohitbanga/django_ticket_19271/tree/code_not_working
>
> Is it possible to support this feature in a future release?
>
> Thanks
> Rohit
>

Can you explain what an "abstract ModelForm" is for? ModelForms are
used to accept and validate browser input and serialise that data to a
Model, storing or updating that data in a database. Abstract models do
not correspond to database tables, so what is the use of this?

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: A.objects.getdefault

[Tom Evans]

On Mon, Oct 15, 2012 at 1:42 PM, Anssi Kääriäinen
 wrote:
> On 10/15/2012 03:13 PM, Ole Laursen wrote:
>
> On Friday, October 12, 2012 3:35:53 PM UTC+2, Chris Wilson wrote:
>>
>> I'm strongly in favour of a simple, obvious way to do the common thing,
>> which is to return None if the object doesn't exist, instead of throwing
>> an exception. My preferred method names would be .nget(), .get_or_none()
>> or .first().
>
>
> IMHO the fact that we have all these inventive solutions is ample proof that
> people do come across this problem and find it annoying. I just found out
> today that there's a somewhat related .latest().
>
> Anyway, without a sponsor with commit access I guess this is dead, so I'll
> shut up now. Thank you for your time.
>
> For the record, I still do like this idea, specifically .get_or_none().
>
> It seems there is significant support for this idea. I do think this method
> is Pythonic. There are cases where no match from get() isn't exceptional,
> and in such cases try-except just feels wrong.
>
> The counter argument is that this is API bloat, and this will set an example
> for more API bloat.
>
> In the end this is a decision with almost no technical considerations and a
> lot of "good taste" considerations. So, this seems like BDFL area.
>
> If there are no signs from BDFL that .get_or_none() is acceptable, then lets
> bury this one. If it is acceptable, then I am willing to do all the work to
> get this committed.
>
>  - Anssi

Where would this method live? On the queryset/manager? I'm not
convinced it should be there, since it's a utility method.

My personal ideal would be a new django.shortcuts.get_object_or_none
shortcut function, as I can see using this as an analogue to
get_object_or_404, but I have a feeling shortcuts is not intended to
grow.

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: URL dispatcher slow?

[Tom Evans]

On Thu, Oct 11, 2012 at 10:58 PM, Łukasz Rekucki  wrote:
> On 11 October 2012 10:20, Russell Keith-Magee  wrote:
>>
>> And don't just say "Why are Django's URL resolvers slow?". Do some
>> profiling, and come back with an analysis of where the time is being
>> spent and/or wasted.
>
> FWIW, here's a link to a cProfile result for the mentioned
> benchmark[1] on Django 1.4.1 and CPython 2.7.3. A quick look shows
> that we're calling get_language() 1.5mln times (read: for every
> pattern), so that's definitely going to slow down things.
>
> I'll try running the same with 1.3 and maybe 1.4 without the locale
> mixin and post if I find anything interesting.
>
> [1]: https://gist.github.com/3875701
>
>

It definitely doesn't help; ideally language code should be calculated
once per call to resolve, and the same value used throughout.
Hard-coding a value of 'en' for language code in
LocaleRegexProvider.regex gives an idea of what gains would be:

Stock django:

static[0] msecrps  tcalls  funcs
django1583   6318 143 69

static[-1]msecrps  tcalls  funcs
django   208794791934 70


Hard coded 'en' in LocaleRegexProvider.regex:

static[0] msecrps  tcalls  funcs
django1424   7023 133 67

static[-1]msecrps  tcalls  funcs
django4972   2011 929 68

static[0] represents tests matching the first URL in the urlconf,
static[-1] represents tests matching the approximately 100th URL in
the urlconf.

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: URL dispatcher slow?

[Tom Evans]

On Thu, Oct 11, 2012 at 2:53 PM, Tom Evans  wrote:
> On Wed, Oct 10, 2012 at 7:52 AM, Moonlight  wrote:
>> Here is an article comparing various URL dispatchers:
>>
>> http://mindref.blogspot.com/2012/10/python-web-routing-benchmark.html
>>
>> What cause django URL dispatcher that much... slow?
>>
>
> Now that I've looked in detail at the test, it is because the test is
> nonsensical. Each time it tests the URLs, it constructs a fresh WSGI
> application. Each fresh application has to compile each URL in the
> urlconf before using it. It then destroys the application, and starts
> another one up.
>
> In a normal django application, you do not start from fresh on each
> request. This explains why the performance degradation from
> "static[0]" to "static[-1] " - which is the difference between testing
> the first url in the urlconf and the last url in the urlconf - is so
> pathological, each time it is run Django is recompiling each and every
> URL regexp in the urlconf (as well as _all_ other work django does at
> server start).
>
> This is testing something, but it is not testing django as a web
> application, or as Django is intended to be run.
>
> Cheers
>
> Tom

I was wrong, the WSGI app is initialized correctly just once.
Something pathological is happening with this test case, the
performance of the resolver looks decidedly dicey as number of
patterns grows.

Something for the weekend…

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: URL dispatcher slow?

[Tom Evans]

On Wed, Oct 10, 2012 at 7:52 AM, Moonlight  wrote:
> Here is an article comparing various URL dispatchers:
>
> http://mindref.blogspot.com/2012/10/python-web-routing-benchmark.html
>
> What cause django URL dispatcher that much... slow?
>

Now that I've looked in detail at the test, it is because the test is
nonsensical. Each time it tests the URLs, it constructs a fresh WSGI
application. Each fresh application has to compile each URL in the
urlconf before using it. It then destroys the application, and starts
another one up.

In a normal django application, you do not start from fresh on each
request. This explains why the performance degradation from
"static[0]" to "static[-1] " - which is the difference between testing
the first url in the urlconf and the last url in the urlconf - is so
pathological, each time it is run Django is recompiling each and every
URL regexp in the urlconf (as well as _all_ other work django does at
server start).

This is testing something, but it is not testing django as a web
application, or as Django is intended to be run.

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: URL dispatcher slow?

[Tom Evans]

On Thu, Oct 11, 2012 at 7:02 AM, Yo-Yo Ma  wrote:
> Why does every conversation about Django's performance met with "GTFO we 
> don't care"? (that was a rhetorical question :). I'd venture to guess that 
> most "It's fast enough for me!" responses are predicated on experiences that 
> can be likened to personal blog development, rather than large scale, 10+ 
> server deployments (e.g., Disqus, et al).

At $JOB, we used to write our websites in C++. No, really, we used to
write all our web apps as part of custom apache modules that used our
own C++ framework. If I compared our C++ url routing code against any
of those python stacks tested, it would absolutely murder them. Yet we
don't write new websites in C++ - why?

The expensive parts of a website are not routing requests, it is DB
queries and waiting to write responses to clients. When you look at it
like that, the speed of "grunt" parts of your framework is simply not
relevant. Django's template system is considerably faster than our
XSLT templates we used in C++ (that doesn't paint XSLT in a good
light).

So, the benchmarks are interesting. They tell us which stacks are
fully featured, and which stacks are very lightweight. Apart from
that, they don't tell us much at all - is Django's template engine
slow, or is it about right for the work it does? This benchmark
doesn't tell us that, it only says it is slower than a bare bones
template engine, which is unsurprising, and shouldn't be a cause for
concern.

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Possible session key creation regression in 1.4

[Tom Evans]

On Fri, May 18, 2012 at 1:38 PM, Aymeric Augustin
 wrote:
> Le 18 mai 2012 à 11:51, Tom Evans  a écrit :
>
>> On Wed, May 16, 2012 at 4:38 PM, Aymeric Augustin
>>  wrote:
>>> 2012/5/16 Tom Evans :
>>>> So, is the session key being available part of the API, or is relying
>>>> on the session key existing incorrect?
>>>
>>> Hi Tom,
>>>
>>> Accessing the session key before saving the session is incorrect.
>>>
>>
>> Accessing the session key before saving the session is incorrect, but
>> there is nothing in the session API to determine if a session is
>> saved.
>
> if session.session_key == None:
>    ...
>
>> I don't see a good way to support 1.3 and 1.4 or in 1.3 to work around the 
>> bug that this fixes
>
> If I understand correctly, you're writing a pluggable app and you want to 
> make it compatible with Django 1.3 and 1.4.
>
> You didn't know this bug existed in 1.3, so I don't understand your sudden 
> urge to work around it in your application. Just keep your current code as is 
> and if your users complain, show them that it's a bug in Django and tell them 
> that they should upgrade.

That may be fine for open source code, it's not appropriate here. I
need to support my features in Django 1.2 and upwards. We have seen
strange things where the session id that we extract is not correct,
and this bug, which I was not aware of, explains that issue. Therefore
I do need to work around it, now that I am aware of it.

Saying to my users, "oh, and now I don't support the version of Django
you use any more, drop all your other projects and upgrade now" is not
acceptable.

>
>> without explicitly saving the session
>> object each time prior to accessing the session key, which is not a
>> particularly clever way of doing things. This API needs to be looked at.
>
> The following sounds perfectly simple and reasonable to me:
>
> if session.session_key == None:
>    session.save()
>
> The if clause will match only under Django 1.4, since under 1.3 session_key 
> isn't None when it should. So you support automatically both versions.

Grrr - session_key should never be None. This should be part of the
explicit contract of the session API - if I'm looking at the
session_key, I want to know what is OR WILL BE the session id. If the
session needs to be saved in order to generate the key, it should be
saved by the backend.

Or to put it another way; my code is never interested in a None
session_key. A valid session never has a key of None. Therefore I now
have to put code in to explicitly ensure that I have a valid
session_key

The bug that was fixed was that accessing session_key before the
session was saved could potentially give you a different session_key
than the one which eventually ends up being used. The fix was to make
the session_key property return None when unsaved - changing the API -
rather than at that point determining what session_key should be used,
which would have kept the same API.

Due to the API changing, and Django not merging bug fixes back through
branches, I now know that I have an issue with 1.3 and below clients
that must be accounted for, and that I must ensure the session is
saved before accessing it, as this ensures in both 1.3 and 1.4 that
the session.session_key is the key for the users session.

This means that each and every place I access session_key in code that
must work in 1.3 and 1.4, I must put in place code like this:

  # We cannot access session.session_key before the session is saved
  # Django provides no way of determining if the session has been saved
  # so we must always save it here. However, there is a race in the
  # session creation that may cause it to fail on key collision.
  while True:
try:
  request.session.save()
except IntegrityError:
  # In <=1.4, session.save() does not generate a new session key
  # if it conflicts, therefore we must do it for Django.
  if django.VERSION[0] < 2 and django.VERSION[1] < 5:
# There is no API to request a new session key without churning
# the session. Use internal API.
request.session._session_key = request.session._get_new_session_key()
  continue
else:
  break
  session_identifier = SSOIdentifier(identifier=request.session.session_key)

Yeah, that API totally rocks. Totally better than determining what the
session_key is when someone wants to know what the session_key is and
we haven't saved the session yet.

>
> I'm sorry, but it's hard to read your criticism in a constructive way when 
> the solution is obvious and doesn't need any changes in Django's APIs.
>
> You raised several related points in your next email; could you create 
> tickets on Tra

Re: Possible session key creation regression in 1.4

[Tom Evans]

On Fri, May 18, 2012 at 10:51 AM, Tom Evans  wrote:
> On Wed, May 16, 2012 at 4:38 PM, Aymeric Augustin
>  wrote:
>> 2012/5/16 Tom Evans :
>>> So, is the session key being available part of the API, or is relying
>>> on the session key existing incorrect?
>>
>> Hi Tom,
>>
>> Accessing the session key before saving the session is incorrect.
>>
>
> Accessing the session key before saving the session is incorrect, but
> there is nothing in the session API to determine if a session is
> saved.
>
> Accessing the session key is a documented feature of sessions.
>
> I don't see a good way to support 1.3 and 1.4, or in 1.3 to work
> around the bug that this fixes, without explicitly saving the session
> object each time prior to accessing the session key, which is not a
> particularly clever way of doing things. This API needs to be looked
> at.
>

Further to this, there is a potential race condition calling
session.save() on a unsaved session using the DB backend.

When the session key is generated, _get_new_session_key() is called,
which generates a session key until one is found that doesn't exist in
the backend store. save() then tries one time to store this session in
the database. If two sessions are being saved simultaneously, and
generate the same (unused) session id, then one of the session saves
will fail.

This behaviour contrasts with create(), which will keep cycling
session ids until it manages to persist one in the backend.

So to sum up:

You can't look at session.session_key without the session having being saved.
You can't determine whether a session has been saved or not.
Saving a session has a race condition, failing with an IntegrityError
in case of key collision.

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Possible session key creation regression in 1.4

[Tom Evans]

On Wed, May 16, 2012 at 4:38 PM, Aymeric Augustin
 wrote:
> 2012/5/16 Tom Evans :
>> So, is the session key being available part of the API, or is relying
>> on the session key existing incorrect?
>
> Hi Tom,
>
> Accessing the session key before saving the session is incorrect.
>

Accessing the session key before saving the session is incorrect, but
there is nothing in the session API to determine if a session is
saved.

Accessing the session key is a documented feature of sessions.

I don't see a good way to support 1.3 and 1.4, or in 1.3 to work
around the bug that this fixes, without explicitly saving the session
object each time prior to accessing the session key, which is not a
particularly clever way of doing things. This API needs to be looked
at.

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Exception Middleware not run [was Re: Subclass handle_uncaught_exception]

[Tom Evans]

On Thu, May 17, 2012 at 9:51 AM, rorycl  wrote:
> Hi Tom
>
> Thanks for your email.
>
> On May 16, 4:14 pm, Tom Evans  wrote:
>> On Wed, May 16, 2012 at 12:32 PM, Rory Campbell-Lange
>>
>>  wrote:
>
>> > I have a view raising a specific exception I would like to catch and
>> > provide an HttpResponse based on this exception.
>>
>> If it is one view raising one specific exception, why aren't you
>> catching it in that view? Exception middleware is used to provide a
>> more generic error page, or for advanced error handling, or to handle
>> errors from a number of views. If it is just for one view, one
>> exception, it is more coherent to handle the error in the view.
>
> As noted before in this thread I have a lazy rowgetter that only
> triggers an exception
>  in the template. In other words, the except clause in my view isn't
> triggered.
>
>    try:
>        rota = self.model.xmlapi_get_location (**kwargs)
>    except DBException, e:
>        code, message = [epart.strip() for epart in e.msg.split(':')]
>        return HttpResponse(message, status=code,
>                            content_type="text/plain")
>
>> > My middleware is as follows -- its simply a sketch at present:
>>
>> >    class ExceptCatcher(object):
>> >        def process_exception(self, request, exception):
>> >            HttpResponse("Error encountered")
>>
>> Is this code verbatim? If so, you aren't returning the newly generated
>> HttpResponse, just creating it.
>
> I'm simply trying to trigger the exception handler in ipdb and that is
> not happening
> using either either the standard middleware or
> decorator_from_middleware
> methodologies. I guess this might be because control has passed from
> the view?
> decorators.py:handle_uncaught_exception _is_ called, but I am unable
> to interject my
> ExceptCatcher with it.
>
>> These sorts of questions should be on django-users really.
>
> Sorry if this is the wrong place to post. However there are a number
> of people on the
> web reporting that they are unable to trigger process_exception. Hence
> my query here.
>
>

Ah, you are returning a TemplateResponse? This changes things.

TemplateResponses are rendered outside of the view. Exception
middleware processes exceptions raised by the view, which this is not
- the view successfully completes and returns a valid HttpResponse
object.

I expect this may be a bug, or at least an unintended consequence.

This is all handled in django.core.handlers.base.BaseHandler. The view
is called here:

https://github.com/django/django/blob/master/django/core/handlers/base.py#L109

and if that raises an exception, Django will try to use the middleware
classes that have handle_exception() methods to generate a response.

If the returned response from the view is a template response, it is
rendered here:

https://github.com/django/django/blob/master/django/core/handlers/base.py#L133

if that raises an exception that isnt http.Http404,
exceptions.PermissionDenied or SystemExit, then it will use the
handle_uncaught_exception to handle the exception and return a
response.

I think this would need a new feature to fix. If a TemplateResponse
fails to render and throws an exception, you would want any middleware
that handles this to be passed both the exception and the
TemplateResponse that caused the exception, not just the exception
object.

I think you should raise a feature request ticket for this.

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Possible session key creation regression in 1.4

[Tom Evans]

On Wed, May 16, 2012 at 4:38 PM, Aymeric Augustin
 wrote:
> 2012/5/16 Tom Evans :
>> So, is the session key being available part of the API, or is relying
>> on the session key existing incorrect?
>
> Hi Tom,
>
> Accessing the session key before saving the session is incorrect.
>
> Previously, it used to return something, but that something could be
> random data instead of the actual session key. Now it returns None,
> making the error more obvious.
>
> For more information, see https://code.djangoproject.com/ticket/11555
>
> Best regards,
>

Hi Aymeric

Looking at the 1.3 code for sessions, I can understand why this
changed. If the generated key collides with a pre-existing session,
new session keys are generated until it does not, and the session key
is changed from the initial one.

However, I do not understand the logic behind the change that fixes this issue.

When we create a session object for a user who does not have a session
id, it seems logical to me that at that point we are creating the
session - the user does not have a session, all users have sessions,
this user must be created a session.

Instead, the session is not created until after the request has been
finished, I presume in order to only create a session when something
is written to it, and to avoid saving the session twice in the initial
request.

Even with this optimization, if I access the session_key of a session
object, it should be apparent that I want the session id of the
current session. If that means that the session must be saved in order
to determine what that session id is, then that is what should happen.
I believe in POLA, and to me it is astonishing that a valid session
does not have an session id when queried, even temporarily.

I find it strange this change was neither documented nor mentioned in
the release notes. In the ticket, you said:

"From the user's point of view, returning None instead of a wrong
session key is probably better, although it might break code that
relies on getting a string." (actually you return None instead of a
correct session id, except when that session id collides with an
existing one)

and

"That is a rather big change, so I'd like to hear from someone else
before working on it."

You had clearly grasped that this was a change that could easily break
peoples code. I'd like to know why that information was not relayed
into the release notes, or anything about the volatility of session
keys added to the docs.

Even though I don't like the API, if it had been in the relnotes, I
wouldn't have spent half a day hunting through the relnotes, docs and
finally diffs to determine what had changed and produce a test case, I
would simply have been able to update the places that refer to session
id to ensure that they existed before use.

Cheers

Tom

PS: I sometimes am accused of coming of as personally confrontational
or critical. This is _absolutely_ not what I'm trying to say here!
Something like this should have been included in the relnotes, so
perhaps a better process can be used in future to ensure that changes
like this are properly documented. To Aymeric's credit, he had
realised when he changed this that it could/would affect people, his
knowledge should have been included in the relnotes!

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Exception Middleware not run [was Re: Subclass handle_uncaught_exception]

[Tom Evans]

On Wed, May 16, 2012 at 12:32 PM, Rory Campbell-Lange
 wrote:
> I'm obviously whistling in the breeze here, but I'd be very grateful for
> some help. I've provided some more info below.
>
> I have a view raising a specific exception I would like to catch and
> provide an HttpResponse based on this exception.

If it is one view raising one specific exception, why aren't you
catching it in that view? Exception middleware is used to provide a
more generic error page, or for advanced error handling, or to handle
errors from a number of views. If it is just for one view, one
exception, it is more coherent to handle the error in the view.

>
> I cannot trigger the exception middleware either as standard middleware
> or using decorator_from_middleware.
>
> My middleware is as follows -- its simply a sketch at present:
>
>    class ExceptCatcher(object):
>        def process_exception(self, request, exception):
>            HttpResponse("Error encountered")

Is this code verbatim? If so, you aren't returning the newly generated
HttpResponse, just creating it.

These sorts of questions should be on django-users really.

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Possible session key creation regression in 1.4

[Tom Evans]

Hi all

One of our sites was recently upgraded to 1.4 and now some previously
working code is failing.

The code tries to access and utilize the session id from the user, but
if it tries to do this on the request the session is created on, the
session id is None. The equivalent session object in 1.3 always
presents a valid session id.

1.3.1:

>>> import django
>>> from django.http import HttpRequest
>>> from django.conf import settings
>>> from django.utils.importlib import import_module
>>>
>>> engine = import_module(settings.SESSION_ENGINE)
>>> req = HttpRequest()
>>> req.session = engine.SessionStore(None)
>>> print django.VERSION
(1, 3, 1, 'final', 0)
>>> print req.session.session_key
a8b409a6b29ac5a59022350445f8bfa4


1.4:

>>> import django
>>> from django.http import HttpRequest
>>> from django.conf import settings
>>> from django.utils.importlib import import_module
>>>
>>> engine = import_module(settings.SESSION_ENGINE)
>>> req = HttpRequest()
>>> req.session = engine.SessionStore(None)
>>> print django.VERSION
(1, 4, 0, 'final', 0)
>>> print req.session.session_key
None


This is using the DB backend, but perusing the changes to
contrib.session, I think all backends will be similarly affected.

So, is the session key being available part of the API, or is relying
on the session key existing incorrect?

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Will django escaping ever consider context of javascript and CSS?

[Tom Evans]

On Fri, Apr 27, 2012 at 11:58 PM, Voulnet  wrote:
> Hello provides great protection from XSS by escaping output to
> webpages, but it only does it in HTML context. XSS can be executed
> when user input is inserted into javascript or CSS, which have
> different context and rules than HTML, so HTML context escaping
> doesn't help/protect.
>
> Are there any remote chance of django escaping extending to other
> contexts beside HTML?
>

There is the built-in escapejs filter that works fine for making user
generated content safe.

https://docs.djangoproject.com/en/1.4/ref/templates/builtins/#escapejs

Although it would be nice to be able to alter the escaping algorithm
in use, so that one can do things like this:

{% block head %}


{% autoescape javascript %}
var user = '{{ user.name }}';
{% endautoescape %}

{% endblock %}

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Allowing formsets to contribute kwargs to forms they create

[Tom Evans]

On Tue, Apr 24, 2012 at 3:23 PM, Andrew Ingram  wrote:
> I've had the need for this quite frequently, Whenever I'm using a
> custom form class with modelformset_factory I almost always end up
> having to define a custom formset class that does nothing except pass
> an extra kwargs (usually user) to each form.
>
> It looks like your approach only adds this facility to BaseFormSet, I
> think the formset factories will need to hook into it as well.
>
> Regards,
> Andrew Ingram
>

Yes, good point. I've only anticipated the cases where you specify
kwargs when instantiating the formset, and where you specify kwargs by
calling BaseFormSet's init method from a subclass. The same technique
could/should be available from the factory, I'll upload a new patch.

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Allowing formsets to contribute kwargs to forms they create

[Tom Evans]

Hi all

I raised a ticket about this new feature, with patch:

https://code.djangoproject.com/ticket/18166

I'd quite like to quickly either complete this or dump the idea.

Opinions?

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: auth.user refactor: the profile aproach

[Tom Evans]

On Tue, Apr 10, 2012 at 6:04 PM, Alex Ogier  wrote:
> Hi Tom,
>
> The best rounded description with pros and cons is Solution 2a on
> https://code.djangoproject.com/wiki/ContribAuthImprovements
>
> You are correct that I am primarily thinking of pluggable authentication
> when I think of this new user model. The reason is that there is nothing
> stopping you from continuing to place app data outside the user model as has
> been standard for a while now. For example, there is nothing stopping you
> from using the following pattern in you app's view:
>
> if request.user.is_authenticated():
>     try:
>     prefs = GalleryPref.objects.get(user=request.user)
>     except GalleryPref.DoesNotExist:
>     prefs = None
>
> That is, unless you have a reason that your particular data should be
> eagerly loaded on every request there is no reason to require it on the
> user. In fact app developers are incentivized to keep their data separate in
> order to remain compatible with the default user.
>
> The solution isn't perfect, it does in fact provide some barriers to this
> pattern. The Gallery app must explicitly choose to foreign key to
> settings.USER_MODEL, and once they do *changing* which model the setting
> points to requires a migration of your table. These are both real issues,
> but I don't think that user model bloat will be because there is a
> straightforward way to work around it if it does prove to be an issue for
> any particular project.
>
> The only thing this proposal kills is magic proxying back from user
> attributes. If you really wanted to, you could roll your own proxy
> attributes to app fields, after all you control the entire user class.
>
> Best,
> Alex Ogier
>

Hi Alex, thanks for that. I think that a lot of this information
should end up in the revised docs, showing patterns on how to
(correctly) use the new features, when to add fields to user and when
not to, etc. and we should be fine.

I wasn't that interested in the magic proxying. Eventually, all magic
dies and has to be removed, so it is better to just be explicit about
what is happening.

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: auth.user refactor: the profile aproach

[Tom Evans]

On Tue, Apr 10, 2012 at 3:13 PM, Ian Lewis  wrote:
> Hi,
>
> I'm not getting why you *have* to add fields to the User model to store data
> pertaining to the user. There is nothing in the proposal for pluggable user
> models that says you can never have a seperate model with a foreign key to
> the user model. It just means that you can define your user model the way
> you want it to be.

That is perfectly fine. The problem comes when there is a simple
system to add fields to the user model, people will use it to add
fields to the user model in their pluggable apps, for 'simplicity' and
'ease of use'.

> Why can't third party apps have a model with a foreign key to the user table
> with the pluggable models approach? I imagine you are right that every app
> and it's brother adding fields to the user model is not realistic but I
> don't think that anyone has proposed that. Certainly not me.

The proposed solution as decided by BDFL diktat is 2a from [1]. I quote:

Split off as much as possible of auth.User into orthogonal mixins that
can be reused.
Modify auth.User to inherit these mixins. Care must be taken to ensure
that the database expression of the new User model is identical to the
old User model, to ensure backwards compatibility.
Unrelated and third-party apps can indicate that they depend on
various orthogonal mixins. For example, contrib.admin can specify that
it works with auth.User out of the box, and with any model
implementing PermissionsMixin if you supply your own login forms.

At the moment, you cannot change the user model, so we do not have
issues relating to third party apps changing the user model. With the
proposed solution, you would be able to change the user model, so we
may have issues.

It's also enlightening to read the code from Alex's Django branch,
which is an initial implementation of option 2a.

> The thing I
> want to be able to is define user models suitable for my project. Third
> party apps adding their own fields wasn't proposed by anyone AFAIK, nor was
> specifically requiring that you add them yourself. Some might require that
> your user has something like an 'email' field because that would be a common
> field across apps but app specific data can easily go on a seperate model
> included with the app that simply has a FK to user. You can then only fetch
> that data on requests that need it.
>
> I'm sorry but doing a JOIN every request is a BAD idea. You will run into
> problems there quickly and have no way out of it besides ditching auth
> completely (and thus all the thirdparty apps you use that depend on it).

I completely disagree, but I'm not here to try and convince people how
to design their databases. A JOIN every request will not end the
world. Besides, it is far more likely to be a separate query than a
JOIN, and would only happen on views that required that data.

More to the point, what basis are you making this claim on? People
love to pipe up "JOINs are slow and evil", but have you actually
analysed the cost compared to monolithic tables?

> Assuming the user table and profile tables are small is awfully short
> sighted.

To be fair, I was slightly ambiguous with my use of the word 'small'. I said:

>> Queries against monolithic tables are much slower than a few queries on much
>> smaller tables.

Here 'small' means fewer columns, not less tuples.

However, assuming tables will be small is precisely what you have just
done - "we must not have JOINs, they are evil, but it doesn't matter
because the user table will only have the columns I desire". I agree
that it is a short sighted position, if you do not prevent the table
becoming monolithic ;)


Cheers

Tom

[1] https://code.djangoproject.com/wiki/ContribAuthImprovements

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: auth.user refactor: the profile aproach

[Tom Evans]

On Fri, Apr 6, 2012 at 7:31 PM, Alex Ogier  wrote:
> Tai, read https://gist.github.com/2289395 for a summary of many reasons why
> I think profiles are a bad idea, and unifying multiple profiles is an even
> worse idea.
>
> Best,
> Alex Ogier

Hi Alex

Is https://gist.github.com/2289395 the complete proposal for what is
to be implemented? It seems more of a point by point rebuttal of
another proposal.

Is the approved solution to have mixins which contribute to a user
class? Are pluggable apps expected to provide mixins that contribute
to the user model?

Does this proposal fix the current issues with the user model in new projects?

My biggest concerns with this approach:

1) Monolithic user tables.

Adding apps that want their own user storage with this system requires
new columns in your user model. This is not the best approach. Foreign
keys to separate user data tables per app is much more appropriate.

Monolithic tables kill performance. As an example, a C++ ORM I have
used had this approach, and it was common to come across a user table
with 100+ columns, including some massive fields (unbounded text
fields, blobs). On most page requests, only a few columns were looked
at, but every page paid the cost of retrieving massive user objects.

The most common complaint against profiles is that they are 'slow' as
you have to join to many tables, or issue multiple queries. Queries
against monolithic tables are much slower than a few queries on much
smaller tables.

2) Constant flux on my user model as a site develops

Adding new apps which require user storage would require schema
change. If you have lots of users, this is a real pain. When adding an
app profile, all that is required is a new table to be created, which
would not lock a critical table like 'user' for a long period.


I've no objection to allowing project managers more control over the
user model, but I don't think we should encourage the majority of
pluggable apps to pollute the user model - in fact, probably only apps
dealing with AAA.

Eg, a photo gallery app may want to store the preferred thumbnail size
and whether to open images in a new window. These do not belong on the
user model, but on a photo gallery app profile. Most users may not
have preferences, and for those that do, it is only relevant to access
that info on a photo gallery page.

A twitter auth app may want to store an oauth token. This should
belong on the user model, as it is used for AAA, and may be accessed
on any request.

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Proposal: upgrading the choices machinery for Django

[Tom Evans]

2012/4/4 Łukasz Langa :
> Wiadomość napisana przez Tom Evans w dniu 4 kwi 2012, o godz. 18:40:
>
>> The class definition grates. When we say things like:
>>
>>  class Gender(Choices):
>>    male = Choice("male")
>>
>> That says to me that Gender.male is mutable. Ick.
>
> Thanks for your feedback. Do model and form field definitions say that as 
> well? It's true that I could have block users from mutating the value but 
> "we're all consenting adults here", right?

Yes, they do. If I have a model with a field named 'address', I expect
that different instances of that model may have different values for
that attribute.

If on the other hand, I have a class with an attribute 'MALE', then I
expect that value to be immutable.

>
>> There is no easy way by inspecting the code to see what choice a value
>> of 7 equates to any-more.
>
> True, the same goes for spotting errors with manual numbering when there are 
> many values.

That makes no sense. If you want to see what value a choice has with
the proposed system, you have to work out precisely what ids will be
automagically assigned to a particular choice.

If you want to spot errors with the current system, you simply look
for choices with the same number. You don't have to work out what
number a choice is, it is explicitly there.

>
>> Do the choices start from 0 or do they start
>> from 1, as inferred by the examples (Why? What is wrong with 0?).
>> Repetition isn't good, but ambiguity is worse.
>
> It's a matter of aesthetics and as such it's totally subjective. I made the 
> values start from 1 so that the first Choice.Group can have value 0 and not 
> -1 which looks ugly.
>
>> This grouping system just seems destined for data loss/confusion. If I
>> want to split a group in two, the enums in the new group change
>> values! That is not a good approach, and was not necessary with the
>> old system.
>
> I can't see how they have to.

So, If I take your examples for groups, and annotate it with the
automagic assigned id:

> class License(Choices):
> COPYLEFT = Choices.Group(0)
> gpl_any = Choice("GPL, any")# id: 1
> gpl2 = Choice("GPL 2")  # id: 2
> gpl3 = Choice("GPL 3")  # id: 2
> lgpl = Choice("LGPL")   # id: 3
> agpl = Choice("Affero GPL") # id: 4
>
> PUBLIC_DOMAIN = Choices.Group(100)
> bsd = Choice("BSD") # id: 101
> public_domain = Choice("Public domain") # id: 102
>
> OSS = Choices.Group(200)
> apache2 = Choice("Apache 2")# id: 201
> mozilla = Choice("MPL") # id: 202

and now I decide that I want BSD licenses as a separate group to
public domain licenses, as they are not the same thing.

> class License(Choices):
> COPYLEFT = Choices.Group(0)
> gpl_any = Choice("GPL, any")# id: 1
> gpl2 = Choice("GPL 2")  # id: 2
> gpl3 = Choice("GPL 3")  # id: 2
> lgpl = Choice("LGPL")   # id: 3
> agpl = Choice("Affero GPL") # id: 4
>
> PUBLIC_DOMAIN = Choices.Group(100)
> public_domain = Choice("Public domain") # id: 101
>
> BSD = Choices.Group(150)
> bsd = Choice("BSD") # id: 151
>
> OSS = Choices.Group(200)
> apache2 = Choice("Apache 2")# id: 201
> mozilla = Choice("MPL") # id: 202

public_domain has changed from 102 => 101
bsd has changed from 101 => 151

These ids only change because ids in this system are handed out
automagically in order of appearance in the class, and grouping is
handled by order of appearance in the class.

>> If I had a vote, I'd be strongly -1 on
>> any proposal with this sort of grouping, it seems dangerous and wrong.
>
> Can you elaborate on what is dangerous about them?

So as above. Your counter will be "well, in those cases you need to
explicitly set an id on those choices". This relies on the person
making the changes realizing that there is magic happening, and take
that into account. That is dangerous compared to the current system,
where I would trust even a novice developer to change and move around
options.

This is because the current system is explicitly clear in what is happening.

>
>> Finally, the proposal seems to concentrate solely on choices as an
>> enum. The proposal completely ignores that choices can be almost any
>> kind of value, e

Re: Proposal: upgrading the choices machinery for Django

[Tom Evans]

2012/4/3 Łukasz Langa :
> A nice HTML rendering of this proposal is available at:
> http://lukasz.langa.pl/2/upgrading-choices-machinery-django/

Disclaimer: all my code currently looks like 'Way 3'.

In general, I like it. There are a few things I don't like in it:

The class definition grates. When we say things like:

  class Gender(Choices):
male = Choice("male")

That says to me that Gender.male is mutable. Ick.


There is no easy way by inspecting the code to see what choice a value
of 7 equates to any-more. Do the choices start from 0 or do they start
from 1, as inferred by the examples (Why? What is wrong with 0?).
Repetition isn't good, but ambiguity is worse.


I don't like the grouping at all. Grouping under the old system was
clean and simple, it didn't require magic arguments, groups didn't
require that all options are contiguous in a single range.

This grouping system just seems destined for data loss/confusion. If I
want to split a group in two, the enums in the new group change
values! That is not a good approach, and was not necessary with the
old system.

This grouping system cannot logically arrange the options into groups,
since it relies on them being attributes of the class, and so you need
to invent a way of grouping. If I had a vote, I'd be strongly -1 on
any proposal with this sort of grouping, it seems dangerous and wrong.


Finally, the proposal seems to concentrate solely on choices as an
enum. The proposal completely ignores that choices can be almost any
kind of value, eg:

MALE="m"
FEMALE="f"
UNKNOWN="?"
GENDER_CHOICES = (
(MALE, "Male"),
(FEMALE, "Female"),
(UNKNOWN, "Unknown"),
  )

this would be an entirely appropriate choices for a CharField.

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: [GSoC 2012] Enhanced contrib.auth

[Tom Evans]

On Fri, Mar 30, 2012 at 10:39 AM, Stratos Moros  wrote:
> You can read the proposal nicely formatted here:
> https://gist.github.com/8dd9fb27127b44d4e789

Hi Stratos

It's a long proposal, so this is a brain dump of bits that I find
interesting/worrisome.

I'm sure you've followed the recent threads on the topic. The (wildly)
different solutions garnered from those long threads are all listed on
this wiki page:

  https://code.djangoproject.com/wiki/ContribAuthImprovements

I don't think this proposal ties in with any of them? Your proposal
involves multiple user models, whilst none of them do.

Login:

Where have auth backends gone in your plan? Why do user objects have a
login method, login should be distinct from user objects, otherwise
login is coupled to a user object, and you cannot log in to the same
user using different authentication techniques.

It is common these days to provide multiple ways of authenticating to
your users, if I authenticate by smartcard, user/password, Facebook
auth, SAML federation, or a "remember me" signed auth cookie, I should
still get the same user object. The choice of authentication is
irrelevant.

More to the point, it should be *my* choice. Forcing authentication
into the user model removes that choice, or requires us to have N user
models per user, one per auth method.


Authentication mixins:

This goes to the above point; if I have to mixin an authentication
class to my user object (adding the required login() + others
methods), it means I can only have one authentication mechanism for a
particular user model.


Deprecating user profiles:

The purpose of user profiles is to provide a place for a pluggable app
to store its own information about a user. Adding a pluggable app
should not mean having to add fields to your user model, but with no
user profiles, that's what is suggested.

Once you have a significant number of pluggable apps, you could have a
User model with a crazy number of fields - I can envisage scenarios
where you have over 100 fields. This makes doing anything with the
user model slower. Furthermore, storage for each field is now required
for every user, whilst with a pluggable profile, it would only exist
if the user utilizes that app.

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Tagging 1.4 django release in Subversion

[Tom Evans]

On Sun, Mar 25, 2012 at 12:02 PM, Florian Apolloner
 wrote:
> Hi,
>
> it's not tagged yet on purpose.
>
> Cheers,
> Florian
>

Out of interest, is there any documentation of the release process?
I'm also intrigued how you have a release tarball before you have
tagged the release!

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: auth.User refactor: reboot

[Tom Evans]

On Tue, Mar 20, 2012 at 6:02 PM, Donald Stufft  wrote:
> So you fix the problem for the people who agree that username should be
> longer, and that email
> should be longer, but not the people who feel that email should be longer
> but username is fine?
>
> Those settings do not feel any cleaner to me personally than monkey patching
> the models.
>

The difference is that new projects will have sane defaults for the
size of an email address, and support using an email address, or a
token that looks like an email address (facebook), as a username.

Monkey patching the models is a process that has to be repeated again,
and again, and again, and again, and again, and again.

So, with all bugs, people tend only to care if it affects you. My
company has hundreds of different websites, all of which SSO
authenticate against a central site.
Most of our newer websites are Django based. Each time we make a site,
the DB structure is created that simply cannot handle a good
percentage of our users - 18% of them have email addresses that do not
fit within the django limits.
So, we're not dumb, we can fix this - and we have for all our existing
sites. The problem is that when a new site is created, that developer
has to explicitly remember to fix the broken django default behaviour.

Guess what happens when they do not remember to do it? I get a defect
report, which I then have to co-ordinate with the client, co-ordinate
with the developer, fix the site, inform the customer.

It is most frustrating to have to continue to fix the same bug, over
and over again, with no sign that the project responsible for the bug
will ever fix the problem. Using a setting pragmatically fixes the
problem, but offends people's sensibilities.

Even with pluggable auth, the proposal is to leave the stock user
model exactly as it is now - ie broken. If using a setting can fix
this pragmatically going forward, then why not?

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: auth.User refactor: reboot

[Tom Evans]

On Tue, Mar 20, 2012 at 5:41 PM, Donald Stufft  wrote:
> What Alex said. If it was _just_ the username then you'd have a good
> argument for
> a setting like that. However there's username, email, some people want to
> add
> fields, some want to get rid of, or combine fields. Ideally any change would
> work
> for as many of those use cases without turning into a giant set of boolean
> flags
> that drastically alter the DB schema.
>

No-one is talking about a 'giant set of boolean flags', or drastically
altering the DB schema.

The email field needs to accept email addresses of up to 254 octets.
The username field should be able to hold common fields used as a
username, eg emails, and hence needs to be longer.

That is one boolean flag. No more. If you want more flexibility on the
model, you would use the proposed pluggable auth models.

What the much maligned setting gives us is a way to provision new
sites and migrate existing sites that care about that situation in to
a position where users with long email addresses can sign up to a
django site. Score.

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: auth.User refactor: reboot

[Tom Evans]

On Tue, Mar 20, 2012 at 5:17 PM, Alex Ogier  wrote:
>> Would something like the following alleviate that problem?
>>
>> class User(models.Model):
>>    if settings.USE_LONG_USER_FIELDS:
>>        username = models.CharField(max_length=255, unique=True, ...)
>>    else:
>>        username = models.CharField(max_length=30, unique=True, ...)
>>    ...
>
> Maybe, but there's a lot more than username length that people want to
> change. Adding piecemeal settings to manage all this is putting wallpaper
> over gaping holes. It's not a good long-term solution.
>

There is a long list of new features people would like to have with
auth Models. All I would like is that as well as these wonderful new
features, we also pay some attention to fixing bugs. Am I the only
person who thinks it is strange to completely re-do auth and NOT fix
the underlying problems?

You can add the same settings to whatever long term solution is deemed
appropriate, as the current plan for that is to have the stock
pluggable User model have the same length fields as the existing one.

Adding piecemeal settings is not good, however this solution does
solve the email and username length issues for:

* Users who read relnotes and want to fix their models
* Users who make new projects

It also doesn't affect:

* Users who don't read relnotes
* Users who read relnotes but don't want to fix their models

None of the other proposed solutions immediately fix the default
username and email length issues for new projects.

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: auth.User refactor: reboot

[Tom Evans]

On Tue, Mar 20, 2012 at 1:37 PM, Russell Keith-Magee
 wrote:
>
> On 20/03/2012, at 8:38 PM, Tom Evans wrote:
>>
>>>  * It's completely backwards compatible.
>>>
>>> If you've got an existing app with normal ForeignKeys to auth.User,
>>> the app will continue to work, without any migrations, as long as
>>> the rest of your project uses auth.User. It will also co-exist with
>>> any configurable app that is configured to use auth.User as the
>>> user model. It will only fall down if you have a configurable app that
>>> uses a different User model (i.e., you can't use a new feature without
>>> ensuring all the parts you need support the new feature).
>>
>> So, what do I do when 3rd party app1 wants a BizarroUser model, and
>> 3rd party app2 wants WeirdUser model? Using the two together as a
>> mix-in would cause problems with any name/definition clash.
>
> Absolutely. That's why I've kept talking about the need for a strong 
> interface contract, and the fact that contrib.admin is in a strong position 
> to set a minimal User interface contract.
>
>> User profiles solve the issue of app specific data in a better way
>> than specifying additional fields on a a base user object,
>> particularly as the number of apps increases. Whilst there is an
>> additional cost in joining to the user profile table, typically in
>> app1 you only need app1.UserProfile, and not appN.UserProfile.
>
> Go back and read my posts -- I've repeatedly said that I prefer UserProfile 
> as a pattern for *app specific* data. The only role that I see pluggable user 
> models solving is the issue of username and email length/uniqueness, and/or 
> the tracking of additional data required for the authentication process. 
> Anything else app specific should be kept in a UserProfile (or UserProfiles, 
> or some other model related 1-1 with User).

(I'm not manually fixing up the line length in your emails any more,
so sorry to others about the quoted line lengths)

Yes, I'm aware of your stance on this. I was merely pointing out that
with this scheme you will get 'pluggable' apps that insist on their
own models, ie they are no longer pluggable. It's not a big deal.

> My proposal for this would be to treat the migration process the same as 
> we've done for any other feature that we've added to Django -- make it an 
> opt-in change with a documented migration path.
>
> For example, when we introduced localization, we set the default to USE_L10N 
> = False, and made the default behaviors unchanged from the older unlocalized 
> behavior. However, if you opted in to USE_L10N = True, you got all the nice 
> new localization features, and possibly a couple of migration headaches. 
> However, because it was an explicit opt-in, you're on the lookout for things 
> that might have changed or broken.
>
> Similarly, I would argue that we if we include this change, we have to ship a 
> concrete User model that is *unchanged* from it's current definition. We then 
> have three options:
>
>  1) Ship an updated User model (say, SimpleUser) with corrected email 
> max_length (and any other changes that pop up). We document the process for 
> opting in to using SimpleUser, including the ALTER TABLE statements that are 
> required to modify any existing databases.
>
>  2) Ship a range of sample User models representing common User model 
> patterns (e.g., using email as login, username but no email, etc) and provide 
> the migration path for each.
>
>  3) Punt on the entire issue, document the limitations with the built in User 
> model, and let the community manage User models; document what people need to 
> do in order to switch, but leave migration as an open question that people 
> providing new User models need to answer.
>
> Personally, I'd be in favor of option (3), mostly because what the last 6 
> years has taught us is that no matter what we pick as field names/sizes, 
> *someone* will be unhappy. However, I won't get too bent out of shape if 
> Django ends up shipping at least 1 new concrete User model -- at the very 
> least, it's a good way to prove we can eat our own dogfood.

Right. So the conclusion to this is "Ship broken, allow people to fix it".

>
> If we introduce 1-N new User classes, we could also take the opportunity to 
> make one of the new User models the default user if you deploy auth using an 
> App definition, but User the default otherwise. This means that any new 
> projects would get the new SimpleUser class, but existing projects would get 
> the older User class.
>
> The key point here is that we're not forcing every Dj

Re: auth.User refactor: reboot

[Tom Evans]

On Sat, Mar 17, 2012 at 5:59 AM, Russell Keith-Magee
 wrote:
> Ok - I've been keeping quiet on this; partially due to the same tone issues 
> that you've described, but also because I don't have a huge amount of spare 
> time at the moment, and I don't want to be the person who makes a bunch of 
> promises (expressed or implied) to work on something and then can't deliver 
> on those promises.

Hi Russell - could you try to configure your mailer such that it will
word wrap at 78 characters. Trying to re-format such a large reply is
tricky, so I've chopped most of your email rather than manually word
wrapping it myself.

>  * It's completely backwards compatible.
>
> If you've got an existing app with normal ForeignKeys to auth.User,
> the app will continue to work, without any migrations, as long as
> the rest of your project uses auth.User. It will also co-exist with
> any configurable app that is configured to use auth.User as the
> user model. It will only fall down if you have a configurable app that
> uses a different User model (i.e., you can't use a new feature without
> ensuring all the parts you need support the new feature).

So, what do I do when 3rd party app1 wants a BizarroUser model, and
3rd party app2 wants WeirdUser model? Using the two together as a
mix-in would cause problems with any name/definition clash.

User profiles solve the issue of app specific data in a better way
than specifying additional fields on a a base user object,
particularly as the number of apps increases. Whilst there is an
additional cost in joining to the user profile table, typically in
app1 you only need app1.UserProfile, and not appN.UserProfile.

Profile settings shared amongst many apps are 'site specific' and live
on the 'blessed' site user profile denoted by AUTH_PROFILE_MODULE, and
accessed via User.get_profile(), which caches the profile. To me,
anyone wishing to add fields to User should be adding them to their
site profile.

Just while we're talking about profiles, obviously the additional join
to access the profile has costs, but sometimes the cost can appear
higher than it actually is. If you have 20 distinct profile classes,
one for each pluggable app to store it's data, and you only require
access to one of them - which is likely, each app will only know about
its own profile - then the cost of the join may be similar to the cost
of searching and fetching a much larger user record, if the fields on
the profiles instead lived on the User object.

>
>  * It solves the immediate problem ...
>
> As I see it, the immediate problem is that developers want to
> be able to modify the base requirements of auth.User. There
> may well be people who want to completely change contrib.auth,
> but for the moment, the 90% case can be solved by modifying
> max_length or setting unique=True on the email field, and/or
> removing the username field. The rest of auth.User is fine, at least for now.

I would disagree with this. The immediate problem from my POV is that
a user who takes the latest release of django, installs the stock auth
app has an email field which cannot hold valid email addresses, and a
username field that cannot hold email addresses. It's unclear that
your proposal changes this at all.

One issue that you have not mentioned at all is schema changes with
pluggable user models. If your average user is going to specify what
fields his User model has in it, what processes will be put in place
to allow the user to move from their current auth_user table
structure?

What about fixing the stock model definition (which will still have an
email field, I assume)? Since schema changes have not been mentioned,
I'm inferring that this would not change, and the 'fixed' version of
django, by default, would still disallow a wide range of valid email
addresses.


I'm not -1 on pluggable auth models, I just think it is orthogonal to
the real issue. A 'fix' which doesn't allow longer email addresses by
default in d.c.a is not a fix as far as I am concerned.

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: auth.User refactor: reboot

[Tom Evans]

On Fri, Mar 16, 2012 at 4:53 PM, Jacob Kaplan-Moss  wrote:
> Hi folks --
> […]

I'm not in favour of pluggable user models, as for me, they solve the
wrong problem. A pluggable user model has to be set up by the project
developer, whilst the attributes an app may need are specified solely
by the app developer.

If a project developer decides to add a 3rd party app to his project,
where do the user preferences for that app live? Does the user model
automatically get expanded with the extra fields required by that app?
It all seems icky to me.

To my mind, User + app specific user profiles are the correct
approach, plus fixing the current minor issues with d.c.a., and
providing tools and documentation to allow users to manage that
change.

Put another way, what does a pluggable user model get us? What is the
big selling point, apart from being able to specify arbitrary columns
to appear in auth_user rather than myapp_userprofile.

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: authentication by email

[Tom Evans]

On Thu, Mar 15, 2012 at 4:49 PM, Carl Meyer  wrote:
> Hi Daniel,
>
> On 03/15/2012 09:24 AM, Daniel Sokolowski wrote:
>> Why can we not just increase the length limit on the username field?,
>> Why can't we just throw a validation error if data entered is to long
>> and the schema has not been updated? I think the answer yes we can and
>> easily.
>
> I don't mean to pick on you specifically, but to me this is an excellent
> example of a casual assertion of something we can "easily" do that
> doesn't hold up under real examination (for instance, if you tried to
> write the patch to actually do it).
>

Why would anyone waste their time working on a patch that would be
instantly rejected, as has been made patently clear throughout.

The solution to these bugs is easy, it is just being ignored because
it is 'too hard' to make these kinds of changes. Instead wishy-washy
ways of completely rewriting authentication are dreamt up, planned,
and then dropped. This is how I would fix it:

1) Fix the email length limit bug, and mark the field as a db index,
but hide this by a settings hack, called CONTRIB_AUTH_EMAIL_FIX. If
the settings hack is not set, raise a PendingDeprecationWarning

2) Add a management command 'auth_NN_inspectdb', NN being the version
number, obviously. This command introspects the auth_user table, and
determines if it needs changing, and prints out whether the table
needs updating, which could include verbatim SQL statements for their
engine. To cope with multiple databases, it takes a --database arg.

3) Add a management command 'auth_NN_updatedb'. This command
introspects the auth_user table, and if it supports the engine,
updates the DDL.To cope with mutliple databases, it takes a --database
arg.

4) Add the following release notes:
The contrib.auth.models.User model has increased the length of the
email field to 254 characters, in order to allow all legal email
addresses to be stored. In addition, the email field has been marked
as a db index, in order to more easily allow email authentication.

These changes are not made automatically by Django, sine they involve
altering the structure of your database. Instead, you must enable this
fix with by adding the settings.CONTRIB_AUTH_EMAIL_FIX. In addtion,
there are two tools included to assist you inspect your database
structure and migrate to the new structure.




If you do not want to make these changes, you can instead keep using
1.4 behaviour by not adding settings.CONTRIB_AUTH_EMAIL_FIX=True. You
will receive a PendingDeprecationWarning when starting up, as this fix
will soon become default.

5) Since this scares you all so much, bump version to 2.0

6) Once released, reverse the hack in trunk so that
settings.CONTRIB_AUTH_NO_EMAIL_FIX gets you 1.4 behaviour, and warns a
DeprecationWarning.

7) Add the following relnotes to 2.1

The contrib.auth.models.User schema changes originally introduced in
2.0 are now the default. If you still do not wish to migrate your
database structure, you can keep using the 1.4 behaviour for now, by
adding settings.CONTRIB_AUTH_NO_EMAIL_FIX=True to settings.py



If you do not either set settings.CONTRIB_AUTH_NO_EMAIL_FIX or update
your database schema, then you will have problems when you encounter a
user with an email address longer than 30 characters. The exact
problem you will encounter depends upon your database:



I feel the amount of work involved here for your average django end
user is minimal, certainly compared to things like adding csrf tokens
throughout.

It has been made abundantly clear that this approach would not be
considered - in particular, using settings hacks to pragmatically
achieve change - so why bother pushing it any further? I love a clean
design as much as the next person, but not fixing bugs that exclude
users on grounds of 'aesthetics' is pedantic and wrong.

> How do you propose to "throw a validation error" if "the schema has not
> been updated"? How do we know whether it's been updated? Are you
> proposing that Django should introspect the users database table every
> time it starts up in order to check the length of the username field?
> Where would you put this introspection check? Have you considered the
> effects this would have on every user of contrib.auth (both those who do
> and don't run the schema migration)? And the effect on Django
> development (needing to run the tests with both an "old" and "new" table
> to ensure that the backwards-compatibility handling is tested?)

This is a bug, and fixing it will affect lots of Django users. I'm fed
up of this excuse being used to avoid fixing it.

You seem to take the view that change is too hard, and therefore we
should not do this until we have a magical solution that is all things
for all men.

Personally, I take the view that every single django site out there is
broken for 'maria.hernandez.gonza...@lar.longcorpname.com', has been
for a long time, and sweet FA has been done to fix it. That view says
fix it, fix it now, 

Re: authentication by email

[Tom Evans]

2012/3/9 Łukasz Rekucki :
> On 9 March 2012 17:46, Tom Evans  wrote:
>>
>> Lets look at one isolated aspect. The User email field in d.c.auth is
>> too short. Emails can be up to 248 characters long, and d.c.auth only
>> allows 75.
>
> The latest RFC[1] actually specifies this as 256 *octets* with max of
> 64 octets for the local part and 255 octets for the domain name. So
> 248 *characters* would actually be incorrect and all the tedious and
> error prone fixing of every Django instance would just get wasted.
>

Sorry, 248 was a typo. If you look at my earlier reply in this thread,
I had correctly stated the maximum length of an email address as 254
*characters*.

If you check two paragraphs later in the RFC that you linked to, you
would see confirmation of this:

http://tools.ietf.org/html/rfc5321#section-4.5.3.1.3

You may also need to remind yourself what the definition of a path is.

As all email headers are either 7 or 8 bit encodings, describing 254
octets as 254 characters is perfectly valid.

None of this changes that django keeps shipping with bad defaults.
This was brought up for 1.1, 1.2, 1.3 and now 1.4.

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: authentication by email

[Tom Evans]

On Fri, Mar 9, 2012 at 3:52 PM, Luke Plant  wrote:
> On 09/03/12 14:49, Tom Evans wrote:
>
>>> Yes, since no one needs it. Okay no one isn't true, but no one (for true
>>> this time) who needed it stepped up and said "I'll implement it and see that
>>> it ends up in trunk"
>>>
>>
>> I'm sorry, that completely mis-characterises the situation. Lots of
>> people want this, but every single time this has been brought up since
>> I started using django (1.0), a core dev or BFDL has explicitly ruled
>> it out, saying "we cannot do this, requires schema migration, we'll
>> look at it for the next version".
>
> This is not true. There have been times when we have said we cannot
> consider it right now, and that a solid proposal should be brought up at
> the calls for proposals for 1.1/1.2/1.3 etc., or at other times when the
> core developers can look at it.
>
> And every time, the proposal has either been missing or simply not been
> adequate - for example, the GSoC 2010 and 2011 proposals regarding this.
> Russell gave detailed feedback on why these solutions were not adequate
> [1], and at other times has provided feedback on possible strategies
> [2]. And the people who went away to work on the problem have gone
> silent or admitted defeat. One presumes they fixed their immediate
> problem in a way that would not be a general solution, and moved on, and
> that is perfectly understandable. A adequate solution to this is very
> hard, and probably requires solving several other big problems first
> (like at least one of lazy foreign keys/virtual models/database migrations).
>
> It isn't, however, impossible. Our definition of 'need' is that someone
> is sufficiently motivated to overcome the obstacles, and contribute a
> solution that works for other people as well as themselves.
>
> Regards,
>
> Luke
>
>
> [1] http://goo.gl/swTpr
>
> [2] http://goo.gl/7p1JN
>

I disagree. There are small problems with the User model that have not
been fixed. Every proposal you've listed does not address these
issues, but instead looks to find ways of replacing the base user
object, which is a complex and difficult project.

There is a reason why these are the proposals. Every time someone
proposes fixing these minor bugs, the cry from on high is that
instead, d.c.auth will be overhauled to allow for more customization
by the project, immediately increasing the scope and complexity. But
this fabled d.c.auth2 never appears.

In fact, your reply alludes to this. We're talking about why a field
is too short to contain valid values, and you suggest that the
solution inevitably will involve "lazy foreign keys/virtual
models/database migrations" - it's changing one line, and adding a
comment to relnotes!

We may have to agree to disagree on how easy some of these things are to fix.

Lets look at one isolated aspect. The User email field in d.c.auth is
too short. Emails can be up to 248 characters long, and d.c.auth only
allows 75.

Clearly, this is a bug, is wrong and should be fixed. How can we fix it?

We can alter the length of the field in the model definition. What
problems does this bring?

Users who do not manually update their database will not be able to
store longer length emails, and will get 'strange effects' when they
try - either silent truncation, or database integrity errors. How can
we mitigate this for users who refuse to update?

Provide a compat shim that resets the email length to that in 1.3, and
immediately mark it PendingDeprecation. Users who read the relnotes
and don't want to change their DB can use the shim, and have a clear
expectation of when they must have updated their database.

I still think these bug fixes are paralysed by a fear of change, and a
desire to engineer a solution that is every thing to every man. Lets
just fix these glaring bugs…

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: authentication by email

[Tom Evans]

On Fri, Mar 9, 2012 at 3:36 PM, Luke Plant  wrote:
> What you are really saying is this: being pragmatic means that we
> prioritise *your* immediate need above the need to keep the code and the
> docs maintainable, and above the need to maintain compatibility with
> existing installations.
>
> There are a million-and-one hacks we would add if we took that approach,
> and Django would have a million-and-one more bugs, or possibly much
> worse - if you add just 10 boolean switches like the one you suggested
> in an earlier email, you've got over 1000 combinations to test and
> debug. The only way to avoid that in a framework like Django is well
> defined APIs for components. No-one has brought such a thing to the
> table and seen it through, that's why it hasn't come in.
>
> I believe that all the problems you have can be solved already:
>
>  - you can write your own authentication backend to authenticate
>   by email
>
>  - you can write your own login and account creation views
>
>  - you can even override admin views by inserting items into
>   urls.py
>

Even with these things, you are limited by the length of the username
field, the length of the email field, and the arbitrary restrictions
on what consists a username.

The length restrictions could be fixed straight away, and a note added
to the next relnotes informing users that the field has changed in
size. If they do not update their DB structure, then depending on the
database used, they will either get errors when a field value is
longer than the old limit, or silent truncation.

It's not ideal, but it doesn't cause any major problems for anyone who
reads the relnotes, and it will have to be done at some point anyway.
Even if Django did have schema migrations, the same behaviour would
occur for anyone who read the relnotes and did not apply the
migration, and many users with 'interesting' DB structures would not
be able to simply apply schema migrations either, so they would have
to do it by hand anyway.

It feels to me that only fear of change and how to manage that change
is what is stopping d.c.auth from being fixed.

>
> Yes, these are all more-or-less hacky. And they have problems. For
> instance, what if you have two accounts with the same email address? (I
> have that in some of my Django projects, BTW). The current code allows
> it, so we can't change anything that would make it impossible, at either
> the DB level or the Python level. This is the kind of issue that means
> you can't just use the existing models.
>

Good for you! I don't see that as an issue. If you are using email
address as a unique name for identifying a user, we already have a
column for that, it's called username, and it has some of the right
semantics (unique, indexed field). It is the arbitrary length and
content restrictions that stop this from happening right now.

I don't see why wanting to use email address as the unique identifier
and username for a user requires any modification to the database,
apart from (as I keep repeating) removing the arbitrary length and
content restrictions on username.

> A full and correct solution that didn't introduce dozens of new bugs is
> quite hard - and I'm talking about just the code required to fix this
> one feature for you, without doing anything more generally useful.
>
> That is why we're not going to put hacks like this in core - we would
> have to support these hacks for at least 3 versions if we did. We are
> interested in *removing* existing hacks from our code base, and this is
> vital if Django is going to see increases in features and functionality.
> We are not interested in adding more hacks.
>
> We also will refuse to litter our documentation with things like: "if
> you've got this setting, it does this, otherwise you get this other
> behaviour, and you've got to watch out for all these bugs because this
> is a gross hack".
>

I agree with this 100%. d.c.auth needs to be fixed, not have many
different settings that alter it's behaviour.

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: authentication by email

[Tom Evans]

On Fri, Mar 9, 2012 at 8:13 AM, Florian Apolloner  wrote:
>> Should these things really take five years? What happened to pragmatic?
>
>
> Yes, since no one needs it. Okay no one isn't true, but no one (for true
> this time) who needed it stepped up and said "I'll implement it and see that
> it ends up in trunk"
>

I'm sorry, that completely mis-characterises the situation. Lots of
people want this, but every single time this has been brought up since
I started using django (1.0), a core dev or BFDL has explicitly ruled
it out, saying "we cannot do this, requires schema migration, we'll
look at it for the next version".

At this point, the discussion is stopped. Once a proposal has the
smack down from a BFDL, what point is there trying to get it
integrated?

The next version comes, has it been addressed? No. Will it be
addressed in 1.4? No. Will it be addressed in 1.5? Possibly, but I
doubt it.

At $JOB we've given up waiting for some trivial changes, and have
simply patched django to not have these silly, arbitrary and incorrect
requirements¹ on email address and username.

We live in the real world, we have to be able to handle email
addresses that are of a valid length. We cannot tell a paying customer
"I'm sorry, your email address is too long, do you have one under 75
characters?"

It's a really invalid position to take, knowing something is broken
and incorrect, but not fixing it because 'change is hard'.

Cheers

Tom

¹ Here is my list of things that are broken with d.c.auth:

1) Length and character restrictions on username field. If I want to
use emails address as usernames, or usernames with spaces, the choices
made at lawrence.com 3+ years ago should not restrict me.
2) Email address fields must be large enough to hold valid email
addresses - that's 254 characters.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Error while importing URLconf 'permitprint.urls': day is out of range for month

[Tom Evans]

On Wed, Feb 29, 2012 at 3:10 PM, furby  wrote:
> A site I built is breaking due to the leap year day, Feb. 29th.  The
> error can be seen here:
>
> www.permitprint.com
>
> Any help resolving this would be appreciated.  I can't track down the
> exception past the traceback.

Do not post questions about using django to django-developers, this
list is for developing django itself. Especially don't post questions
to both lists at the same time.

I'll answer your django-users@ email.

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Revisiting multiline tags

[Tom Evans]

On Fri, Feb 24, 2012 at 3:12 PM, Alex Gaynor  wrote:
> Folks, you seem to have missed Russell's point.  Even if 100 people +1 this,
> it's meaningless.  That's a tiny fraction of this mailing list's readership,
> much less of the Django community at large.  Django is the way it is
> because, first and foremost, of taste.  If you'd like to make an argument as
> to *why* it's useful, that's useful, but we don't take polls.

I think all the arguments have been stated, and quite well.

So, how many people have to disagree with a BDFL before something will
be reconsidered? I do understand what the 'D' in BDFL stands for…

Perhaps it would be better for the reasons for rejecting this change
to be made clear. Russell alludes to this being an aesthetic decision.
I am no aesthete, so for those of us not so gifted, can you explain
why having multiline capable tags would ruin the beauty of django?

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Newline stripping in templates: the dnl way

[Tom Evans]

On Fri, Feb 24, 2012 at 2:10 PM, Tobia  wrote:
> Hi all,
> regarding issue #2594 "Template system should handle whitespace
> better" explaining the need for some kind of (optional) whitespace
> stripping from the output of templates, I've been looking at the
> proposed solutions and compared them to other template and macro
> engines.
>
> In particular, a historical but still widely used and general-purpose
> macro engine is m4. Its own feature for controlled newline stripping
> is the "dnl" reserved word, "Discard to Next Line." It works by
> "chomping" all input until and including the next newline.
>
> For example, to call a macro foo() without copying over the newline
> that appears after the macro call in the template, a m4 programmer
> would write:
>
> foo('bar`, `whatever')dnl
>
> An equivalent feature in Django templates would enable template
> developers to strip newlines from specific lines, while keeping
> backwards compatibility with existing templates.
>
> So if the general idea is well-accepted, I propose the "{#" token. The
> example from the issue would become:
>
> 
> {% for item in items %}{#
>    {{ item }}
> {% endfor %}{#
> 
>
> It is already a reserved token in the template system, so it's not
> going to break anything. The existing comment syntax {# ... #} is
> already restricted to single-line comments, so there are no multi-line
> comments that would break. Plus, I'd wager it could be implemented
> quite efficiently.
>
> What do you think?
>
> Tobia
>

I'd be strongly -1 on anything that makes template language look more like m4!

Having said that, I can see this being useful. We use templates to
render most kinds of output. When we render text emails, the newlines
and indentation are particularly important, as everything gets
displayed 'as-is' by the client, and having a template that is
readable and also renders correctly is tricky.

Generally where this happens is where we open a block level tag, and
would then normally have a newline for readability, but cannot due to
us not wanting a newline to appear. This could be addressed by having
a different open/close tag for tags which chomp the preceeding/next
character if it is a newline. Eg:

In your folder are:
{^ for item in folder ^}
{{ item }},
{^ endfor ^}

This would render as "In your folder are: item1,item2,item3,"

Changing some of the carets:

In your folder are:
{% for item in folder ^}
{{ item }},
{^ endfor %}

This would render as "In your folder are:\nitem1,item2,item3,\n"

I'm not 100% convinced though.

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Improving docs for User.is_authenticated()

[Tom Evans]

Hi all

I don't like this function that much. It doesn't actually check
whether users are authenticated - which is to say, they have presented
credentials which we have accepted and authorized them to use to the
site. Instead it always returns true. is_not_anonymous_user() may be a
better name.

User.is_authenticated() is documented like so:

"""
is_authenticated()
Always returns True. This is a way to tell if the user has been
authenticated. This does not imply any permissions, and doesn't check
if the user is active - it only indicates that the user has provided a
valid username and password.


This is misleading, as it doesn't actually indicate that the user has
provided a valid username and password, since it always returns True.

There can be many ways that a user authenticates without having to
provide username and password, and User objects not automatically
instantiated by the auth middleware (eg, manually looking up a user)
haven't been authenticated at all.

Eg, this code:

def myview(request):
  user = User.objects.all().order_by('?')[0]
  user.is_authenticated()

At no point has that user object been authenticated, or supplied valid creds.

Obviously, this function cannot change in behaviour or name, so I
suggest altering the docs, dropping the clause about indicating that
the user has provided username and password to make it clearer what
this method does.

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: How to create a new command in django-admin.py

[Tom Evans]

On Thu, Feb 23, 2012 at 1:22 PM, Johan  wrote:
> Hi
>
> I know how to create a new command in manage.py, this link explains it
> nicely : 
> https://docs.djangoproject.com/en/dev/howto/custom-management-commands/#command-objects
> . But how do I create new command which would be available in django-
> admin.py or in manage.py BUT without having to add the application to
> the INSTALLED_APPS tuple?
>
> Regards
> Johan

This should be on django-users, unless you are proposing a new way of
doing this.

The answer to your question is "you can't", btw.

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: auth.User usernames

[Tom Evans]

On Sat, Feb 18, 2012 at 4:25 AM, Tai Lee  wrote:
> It's not that hard to just set up a OneToOneField back to User, and
> use signals to automatically create a User when you create your own
> User/Profile. Then you can still make use of 3rd party apps that rely
> on contrib.auth or contrib.sessions, and also make use of groups from
> contrib.auth, etc.
>
> Cheers.
> Tai.
>

Once you are aware of the issue, it's easier to change the code so
that it DTRT in the first place. Any other solution has notable
downsides, like the username field not being what the user enters to
log in with, the email field not being correct.

Eg, with your fix authenticating/storing longer emails on a
UserProfile object may work, but it's unlikely that third party code
which sends emails will be aware that the correct email address to use
is user.get_profile().real_email_field rather than user.email.

Those sorts of bugs are much harder to find/fix than the actual bugs,
which are that both email and username fields are too short, and the
username field has arbitrary restrictions on what it can contain.

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: auth.User usernames

[Tom Evans]

On Wed, Feb 15, 2012 at 10:37 PM, Donald Stufft  wrote:
> I know this has been discussed before, but I wanted to bring it up again in
> light of the oncoming Djnago 1.4 beta.
>
> Can we increase the length of the username field in auth.User?
> I think that a max_length of 75 (to match the default EmailField) would allow 
> developers
> a lot more breathing room with regards to what they use as the value for 
> username.

75 isn't large enough these days for either email or username. We run
a patched version of django for some time that has changed both these
fields to 255 characters in order to accommodate the needs of our
users. See RFC 3696.

Schema migration concerns aside, it is a pity that Django continues to
ship with a non conforming understanding of what an email may be.
Personally, I believe this is a nonsense concern - if you state in the
relnotes that a field has changed size, and that you must update your
database schemas, **and people don't do it**, the only issue that can
occur is that when a user attempts to use a longer email address or
username the failure is an IntegrityError instead of a form error. Big
whoop, should have read the relnotes.

Each django "minor"* version release that I have updated through has
included equally disruptive changes, I don't see why the prevarication
over this one.

I understand that the timing isn't great for this right now, but this
is not an unknown or new problem. Another major release slips by
without a fix for this. mañana, mañana…

Cheers

Tom


* As in '1.1 -> 1.2', not as in 'insignificant version'

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.