Re: New additions to django.contrib?

[Russell Keith-Magee]

On Sat, Mar 2, 2013 at 11:38 AM, Per-Olof Åstrand wrote:

> First, I would like to give all my compliments to the persons that made
> Django what it is. Although my background is in scientific computing since
> 25 years back (using python, numpy, scipy, among other things), I came into
> contact with Django 0.91 in 2006 when one of my students used it in a
> project. I have used it myself since 2009 in a private genealogy project
> which is alive at webfactional.com (running django trunk) since a year
> ago.
>
> I can perhaps spend a couple of hours per week on this hobby project, so I
> have realized that I have a quite different view on what should be included
> in django.contrib than what seems to be the present view on the django
> mailing list. Rather than slimming django core, I would like to see more
> "best practice" apps included in django.contrib. I (and I think/hope many
> with me) simply do not have the time to browse the jungle of third-party
> apps to test, compare and find out what might work for my purpose. I would
> like to see that some of these decisions are made for me by more
> experienced users/developers, if possible. When I see comments on that
> admin should be developed separately, I disagree. At least I would like to
> see a discussion early in each release cycle to find out if there are any
> third-party apps "worthy of" being included in django.contrib, and perhaps
> one can be included in each release cycle.
>
> I see another perspective on this from my work. For our python software, I
> allow python, numpy, scipy and a few more packages. To include more
> packages is a big deal for a huge number of reasons (project complexity,
> learning curve, maintenance, etc). On the other hand, the packages included
> in the project should be exploited fully. I think Django face the same
> problem in many environments; it is a big deal if a particular third-party
> app can be included or not.
>
> A suggestion is django-debug-toolbar. I think that is the kind of app new
> users should learn how to use relatively early in a learning process. Also
> if included django, a lot of best practice programming in views and
> templates can be documented in a better way if django-debug-toolbar is
> available. For me, this is a no-brainer if the app authors are interested,
> license conditions is fulfilled, core developer capacity is available, etc..
>
> A second suggestion is django-mptt. It is a text-book approach that has
> many use-cases, and I cannot see that there is a need for more than one
> standard implementation (although there are also other ways to implement
> trees which have their use-cases).
>
> I realize I represent a user group that is not very visible, with only
> occasional comments on django-users and in tickets, but I hope my opinions
> are welcome anyway.
>

Hi Per-Olof,

Thanks for your feedback.

I'll completely agree that django-debug-toolbar is a very useful package. I
haven't had occasion to use django-mptt myself, but I've heard many people
speak well of it. I'm sure it would be easy to come up with a dozen or more
packages that are similarly useful and high quality.

However, I don't believe that adding these packages to django.contrib will
provide any benefit at all.

Adding a package to core won't make the code better. It will be the same
code, just in a different repository.

Adding a package to core won't increase the number of people working on it,
either. The Django core team is already stretched fairly thinly -- adding
more responsibilities must result in either parts of the core will get less
attention, or we need to add more people to manage the core. However,
someone who is a good maintainer of a third-party package may not be the
best person to add to core, so all we would be doing is giving them access
to a specific part of the contrib tree… which means that once again,
nothing has changed -- we've just changed the repository holding the code.

The best you can hope for is that by putting some sort of "official"
moniker on it, this will attract more attention and thus more contributors.
However, this also means that unofficial projects will find it harder to
get contributors, regardless of how good they might be. In some cases, this
will be a good thing -- nobody benefits from having a dozen social auth
integration libraries, for example. But in other areas -- especially when
there are multiple technical approaches -- encouraging a monoculture isn't
a good idea.

There's another side effect of putting more packages in contrib -- you slow
the rate of release of the contrib packages to the rate of release of
Django itself.

Consider, for example, localflavor. The Australian localflavor hasn't
needed an update for several years -- there have been no new states added,
and phone number formats haven't changed. The Indonesian and Italian
localflavors have both required changes due to changes in provincial
naming. However, if the change to 

Re: Django 1.4.3 Error Login

[Russell Keith-Magee]

On Sat, Mar 2, 2013 at 12:50 PM, Ana Molf  wrote:

> This function work under dev server but not ander prod
> Technical server contact said me that there are a problem because at SSH
>
>
Hi Ana,

You've posted this message to django-developers -- the mailing list for
people wanting to discuss the development of Django itself. This sort of
question about Django usage should be posted to django-users.

When you do post to django-users, you're going to need to provide a lot
more detail if you want a helpful response. Specifically, you need to
provide the errors that you're seeing, and the details of your production
platform. Saying "This doesn't work in production" tells us nothing about
the errors you're getting, or the differences between your development and
production environments.

Yours,
Russ Magee %-)

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Django 1.4.3 Error Login

[Ana Molf]

This function work under dev server but not ander prod 
Technical server contact said me that there are a problem because at SSH

[28/Feb/2013 23:58:04] "GET /login/ HTTP/1.1" 200 15710
[28/Feb/2013 23:58:12] "POST /login/ HTTP/1.1" 500 6965

def login_test(request):
state = "Ingreso de usuarios registrados:"
current_date=datetime.datetime.now()
#ua=request.META('HTTP_USER_AGENT', 'unknown')
ip=request.META['REMOTE_ADDR']
#user=request.user
username = password = ''
if request.POST:
username = request.POST['username']
password = request.POST['password']
user = authenticate(username=username, password=password)
if user is not None and user.is_active:
login(request, user)
state = "You're successfully logged in!"
return render(request, 
'_xxx.html',{'username':username, 'password':password})
else:
state = "Su usuario ha sido desactivado!. Contacte al administrador 
del sitio para recuperar sus credenciales"
return render_to_response('xx.html',locals())
else:
state = "Su usuario o contraseña son incorrectos! Contacte al 
administrador del sitio"
return render_to_response('x.html',{'state':state, 'username': 
username,'current_date': current_date, 'ip': ip})

At dev when submit can reach app ok
At prod when submit reply is 500.html

Server: Apache, debian

 at html

Any comment would be great

Thanks

Ana


-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

New additions to django.contrib?

[Per-Olof Åstrand]

First, I would like to give all my compliments to the persons that made 
Django what it is. Although my background is in scientific computing since 
25 years back (using python, numpy, scipy, among other things), I came into 
contact with Django 0.91 in 2006 when one of my students used it in a 
project. I have used it myself since 2009 in a private genealogy project 
which is alive at webfactional.com (running django trunk) since a year ago.

I can perhaps spend a couple of hours per week on this hobby project, so I 
have realized that I have a quite different view on what should be included 
in django.contrib than what seems to be the present view on the django 
mailing list. Rather than slimming django core, I would like to see more 
"best practice" apps included in django.contrib. I (and I think/hope many 
with me) simply do not have the time to browse the jungle of third-party 
apps to test, compare and find out what might work for my purpose. I would 
like to see that some of these decisions are made for me by more 
experienced users/developers, if possible. When I see comments on that 
admin should be developed separately, I disagree. At least I would like to 
see a discussion early in each release cycle to find out if there are any 
third-party apps "worthy of" being included in django.contrib, and perhaps 
one can be included in each release cycle.

I see another perspective on this from my work. For our python software, I 
allow python, numpy, scipy and a few more packages. To include more 
packages is a big deal for a huge number of reasons (project complexity, 
learning curve, maintenance, etc). On the other hand, the packages included 
in the project should be exploited fully. I think Django face the same 
problem in many environments; it is a big deal if a particular third-party 
app can be included or not.

A suggestion is django-debug-toolbar. I think that is the kind of app new 
users should learn how to use relatively early in a learning process. Also 
if included django, a lot of best practice programming in views and 
templates can be documented in a better way if django-debug-toolbar is 
available. For me, this is a no-brainer if the app authors are interested, 
license conditions is fulfilled, core developer capacity is available, etc..

A second suggestion is django-mptt. It is a text-book approach that has 
many use-cases, and I cannot see that there is a need for more than one 
standard implementation (although there are also other ways to implement 
trees which have their use-cases).

I realize I represent a user group that is not very visible, with only 
occasional comments on django-users and in tickets, but I hope my opinions 
are welcome anyway.

Best regards,
Per-Olof

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Transforming django-admin.py to a shell script

[Łukasz Rekucki]

On 1 March 2013 22:38, Alon Nisser  wrote:

> at least from my windows exprience with Django (yes, I know this isn't a
> common use case, but still) the current django-admin.py and manage.py do
> need python preface to run right (while inside a virtualenv)


I'm not 100% sure if the Windows Python Launcher[1] works with virtualenv,
but it most likely works with venv support in Python 3.3. It should also
allow you to run Python scripts without the .py suffix

Anyways, what kind of "shell" you had in mind? Batch ? Powershell ? Bash ?.

[1]: http://blog.python.org/2011/07/python-launcher-for-windows_11.html

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Transforming django-admin.py to a shell script

[Alon Nisser]

at least from my windows exprience with Django (yes, I know this isn't a 
common use case, but still) the current django-admin.py and manage.py do 
need python preface to run right (while inside a virtualenv) - but maybe 
I'm wrong, at least from my peak in Django setup.py It doesn't utilize 
`entrypoints` so I guess this is why on some platforms the basic shell 
usage fails. anyway wouldn't it be more future proof to start enabling the 
option to use Django-admin and manage as shell script. the current .py 
version would continue to work (even if in a somewhat broken way) but 
slowly the tutorials would move to simpler shell script without the .py 
ending.

On Friday, March 1, 2013 8:40:20 PM UTC+2, Alex Ogier wrote:
>
> On Fri, Mar 1, 2013 at 1:25 PM, Alon Nisser  
> wrote: 
> > and then it could be called as `django-admin somecommend` instead of 
> `python 
> > django-admin.py somecommand`. 
> > since python (using setuptools entry points) makes making a python 
> script 
> > into a shell script quite easy I guess this has been Discussed before 
> but I 
> > didn't find the discussion and the explanation why doesn't Django team 
> deem 
> > this path worthy. 
> > I think most python web frameworks use some kind of a shell script (or 
> using 
> > a customized paster/gearbox to provide this functionality) except for 
> > Django. 
>
> django-admin.py is already an executable script. So you don't need 
> `python django-admin.py somecommand`, just `django-admin.py 
> somecommand` should suffice. The .py suffix is a bit ugly, to be sure, 
> but it's not worth breaking every single introductory tutorial on the 
> web to fix. Besides, it does have some merit as it is consistent with 
> `./manage.py somecommand`, which is the recommended way to run pretty 
> much every command except "startproject". 
>
> Best, 
> Alex Ogier 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: deprecation of AUTH_PROFILE_MODULE

[Florian Apolloner]

Hi,

On Friday, March 1, 2013 9:00:28 PM UTC+1, Luke Sneeringer wrote:
>
> I'd be interested in actually doing this if folks on the list think it's a 
> good idea.
>

Doing it outside of Django core if fine, inside core is -0 to -1 from me.

Cheers,
Florian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: deprecation of AUTH_PROFILE_MODULE

[Luke Sneeringer]

I do think that there's a pretty common "alternate use case" which we could 
support more easily. That's the use case where someone wants a User model that 
is either:

 * exactly like the Django User model, but with e-mail address as the unique 
identifying field instead of username
 * exactly like the above bullet, but with additional fields

The new 1.5 user customization makes adding fields to the user model a very 
trivial task, but it's still a lot of code to make e-mail addresses serve the 
function of usernames. You need to write the model, define USERNAME_FIELD, 
REQUIRED_FIELDS, and all the methods, which basically work almost identically 
to the auth.User reference implementation. It's also not very DRY.

I was considering writing an implementation of this, which I think would see a 
lot of use, and putting it up on PyPI, but it seems like something that would 
be even better if offered as part of Django itself.

Here is, I think, a way we could do it without breaking backwards compatibility:

 * Make an EmailBasedUser class which functions identically to User, except 
that email is the USERNAME_FIELD.
 * Make auth.User subclass EmailBasedUser, add `username` and set it as 
USERNAME_FIELD. User would still be AUTH_USER_MODEL.
 * Provide instructions in the documentation saying, "If you only want this one 
change, set AUTH_USER_MODEL to django.contrib.auth.EmailBasedUser

What is driving me here is that I think that this is a very common use case -- 
"I just want email addresses to function as usernames", and Django 1.5 still 
seems to require a lot of code to do that. It seems like it should be a quick 
change -- "just change this one setting" -- and it's not.

I'd be interested in actually doing this if folks on the list think it's a good 
idea.

Best Regards,
Luke Sneeringer

On Feb 27, 2013, at 4:47 PM, Russell Keith-Magee  
wrote:

> 
> 
> On Thu, Feb 28, 2013 at 4:24 AM, Alper Çuğun  wrote:
> 
> I looked into this today with 1.5 hitting and working on a project and this 
> seemed relevant.
> 
> On Monday, 2 May 2011 19:22:53 UTC+2, Carl Meyer wrote:
> 1. A specification of the minimal useful interface for a User (perhaps
> in the form of an abstract base model?) 
> 
> A User should not require a username. Usernames are a hideous remnant that 
> are impossible to justify on consumer facing websites. Authentication on 
> e-mail should be built-in if not the default.
>  
> A user *doesn't* require a username. It requires a unique identifying field. 
> However, by default, for historical reasons, the Django's built-in User model 
> has a username field. The whole point of the new feature is that specifying 
> an alternative is now a simple activity.
> 4. A migration path that meets our backwards-compatibility standards.
> 
> I saw the configurable User model that just landed in 1.5 but for a 
> non-library developer this is not very useful. It looks like a very large 
> amount of code for a rather small function.
> 
> It's not clear what you're suggesting here. Django has a built in User model. 
> That hasn't changed. You can now specify your own User model, and reuse large 
> parts of Django's built in parts if that is helpful in your circumstances. 
> What exactly is your concern here?
>  
> Furthermore: “If you intend to set AUTH_USER_MODEL, you should set it before 
> running manage.py syncdb for the first time.” is somewhat prohibitive to 
> existing projects. “you may need to look into using a migration tool like 
> South to ease the transition.” is not really a solution unless it is obvious 
> what changes should be made.
> 
> Well, this is a situation where we can't provide documentation, because 
> everyone's specific circumstances will be different. The best advice for 
> custom user models is "set it before you run syncdb", which is what we've 
> said. If you've already run syncdb, you'll need to migrate your models… but 
> exactly what that entails will entirely depend on what your own project has 
> done.
>  
> Also I don't understand: “This example illustrates how most of the components 
> work together, but is not intended to be copied directly into projects for 
> production use.” Code that is ready for production is the point of the 
> documentation for me. If this isn't, I think it's a documentation bug.
> 
> The point of that comment is that you shouldn't treat the documentation as a 
> cargo-cult, just copying and pasting. What is there will *work*. However, in 
> a real application, you need to pay attention to error messages, validation 
> conditions and so on.  
> 
> Yours,
> Russ Magee %-)
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group 

Re: Transforming django-admin.py to a shell script

[Alex Ogier]

On Fri, Mar 1, 2013 at 1:25 PM, Alon Nisser  wrote:
> and then it could be called as `django-admin somecommend` instead of `python
> django-admin.py somecommand`.
> since python (using setuptools entry points) makes making a python script
> into a shell script quite easy I guess this has been Discussed before but I
> didn't find the discussion and the explanation why doesn't Django team deem
> this path worthy.
> I think most python web frameworks use some kind of a shell script (or using
> a customized paster/gearbox to provide this functionality) except for
> Django.

django-admin.py is already an executable script. So you don't need
`python django-admin.py somecommand`, just `django-admin.py
somecommand` should suffice. The .py suffix is a bit ugly, to be sure,
but it's not worth breaking every single introductory tutorial on the
web to fix. Besides, it does have some merit as it is consistent with
`./manage.py somecommand`, which is the recommended way to run pretty
much every command except "startproject".

Best,
Alex Ogier

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Transforming django-admin.py to a shell script

[Alon Nisser]

and then it could be called as `django-admin somecommend` instead of 
`python django-admin.py somecommand`. 
since python (using setuptools entry points) makes making a python script 
into a shell script quite easy I guess this has been Discussed before but I 
didn't find the discussion and the explanation why doesn't Django team deem 
this path worthy.
I think most python web frameworks use some kind of a shell script (or 
using a customized paster/gearbox to provide this functionality) except for 
Django.

thanks for the clarification

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Switch to database-level autocommit

[Stephan Jaensch]

 

> I'd like to improve transactions handling in Django. The first step is the 
> current emulation of autocommit with database-level autocommit.
>
[...] 

> I don't see much point in providing an option to turn autocommit off, 
> because starting a transaction is a much more explicit way to achieve the 
> same effect. We usually don't provide "backwards compatibility with bugs".
>
> Yay or nay?
>

A definite yay from me!

Cheers,
Stephan 

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Switch to database-level autocommit

[Anssi Kääriäinen]

On 1 maalis, 14:48, Aymeric Augustin
 wrote:

> Yay or nay?

+1.

I discussed this on IRC with Aymeric. I tried to find cases where this
breaks currently working code, and I was mostly unsuccessful. It seems
there could be such cases (doing raw SQL writes, commit at end, and no
ORM writes, all this outside transaction management), but it seems
such cases are rare, and worked more by luck than knowledge of how
Django's tx management works. Explicit transactions for such cases
would be good anyways.

This will be a backwards incompatible change. The current behaviour is
explicitly documented, and the new behaviour is clearly different in
some cases. However, finding cases where one actually wants the
current behaviour (ORM writes commit, raw SQL writes do not, and any
query will start a transaction) is hard. Doing a full deprecation path
for this will be ugly, so just changing this without deprecation
period seems good to me.

All in all, this might cause some pain to very small portion of users
when upgrading to 1.6. Those users will almost certainly be happy
after they have upgraded their code. Otherwise this will be a big win
for everybody.

 - Anssi

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Moving from PIL to Pillow

[Alex Clark]

On 2013-02-27 22:00:17 +, Alex Ogier said:


On Wed, Feb 27, 2013 at 4:42 PM, Aymeric Augustin
 wrote:


On 27 févr. 2013, at 21:40, Marijonas Petrauskas 
 wrote:



Why isn't Pillow the recommended Django image library yet? PIL has
been unmaintained for almost 3 years and has a number of annoying bugs
(e.g. fails to open some valid JPEG files, maybe has some security
issues as well). Pillow, on the other hand, is a backwards-compatible
community-maintained fork, which has most of those issues fixed and
will even support Python 3 soon.

I think this change would involve (1) running the test suite with
Pillow, (2) updating the documentation and (3) updating the ImageField
warning shown when PIL is not installed. Looks like an easy change.
What do you think?



Yes, we'll have to choose a Python 3-compatible replacement for PIL.

I've converted your email into a ticket: 
https://code.djangoproject.com/ticket/19934


I assume that Pillow and PIL are API-compatible. Django could fallback to PIL
if Pillow isn't installed, to preserve backwards compatibility.

--
Aymeric



This seems like pretty much a no-brainer. PIL is a notoriously
backwards python package and more or less abandonware, and Pillow is a
well-supported fork.

As regards API-compatibility, Pillow is so much compatible that it
even installs to the same locations. Unless you manually check for it,
you shouldn't even notice that you are getting Pillow libraries
instead of PIL libraries. Which means that supporting pillow *should*
be as easy as Marijonas suggests, unless there actually are real
regressions.



There should be very few regressions because prior to Pillow 2.0 (which 
is unreleased, and will support Python 2.6 - Python 3.3), we've made 
very few, if any, image code fixes. (And for whatever it's worth, 
whenever such fixes were considered, we attempted to track them with 
the "upstream" PIL).


In any event, if you do find a regression please report it here:

- https://github.com/python-imaging/Pillow/issues

Peer reviewed pull requests for such issues will be merged and released 
promptly :-)




Just install it instead of or in addition to PIL when
testing,



I would recommend removing PIL first, as I think we've seen some issues 
related to both PIL & Pillow being installed simultaneously. (And both 
distributions provide the same packages/modules, similar to the way 
"distribute" provides the "setuptools" package.)




and update any documentation that refers to PIL and
ImageFields to recommend Pillow instead.



+1. The only thing Pillow could not provide was the distribution name 
"PIL" on PyPI. So you must "pip install Pillow" instead of PIL. 
Everything else e.g. "from PIL import Image" should be the same.



Alex Clark, Pillow (PIL fork) author




Best,
Alex Ogier



--
Alex Clark · http://about.me/alex.clark


--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Switch to database-level autocommit

[Jacob Kaplan-Moss]

Hey Aymeric -

Yes, I think this is the right thing to do: +1.

Jacob

On Fri, Mar 1, 2013 at 4:48 AM, Aymeric Augustin
 wrote:
> Hello,
>
> I'd like to improve transactions handling in Django. The first step is the
> current emulation of autocommit with database-level autocommit.
>
> ** Rationale **
>
> PEP249 says that any SQL query opens a transaction and that it's the
> application's job to commit (or rollback) changes.  This model is also
> required by the SQL standard. But it isn't very developer-friendly.
>
> To alleviate the pain, Django commits after each ORM write.  Unfortunately,
> this means that each read opens a transaction, that eventually gets
> committed after the next write, or rolled back at the end of the query. Such
> transactions are useless and don't come for free. Relying on them to enforce
> integrity is extremely fragile — what if an external library starts writing
> to a log table in the middle of one of these implicit transactions? The term
> "footgun" comes to mind.
>
> Database authors have reached the same conclusion, and most databases
> supported by Django use autocommit by default, ignoring the SQL standard. On
> PostgreSQL and SQLite, this is the only mode available.
>
> As a consequence, to implement the behavior mandated by PEP 249, the Python
> libraries (psycopg2, sqlite3, etc.) automatically start transactions. And
> then Django automatically commits them. This is not only wasteful, but also
> buggy. It's the root cause of "idle in transaction" connections on
> PostgreSQL. It's also sometimes poorly implemented: for instance, executing
> "SAVEPOINT …" on SQLite commits implicitly. (It's arguably a bug in the
> design of the sqlite3 module. The Python bug tracker suggests it's going to
> be documented.)
>
> Basically, Django intends to provide autocommit by default. Rather than
> fight the database adapter that itselfs fights the database, I propose to
> simply turn autocommit on, and stop implicitly starting and committing
> transactions. Explicit is better than implicit.
>
> ** Implementation **
>
> All databases supported by Django provide an API to turn autocommit on:
>
> - http://initd.org/psycopg/docs/connection.html#connection.autocommit
> -
> http://docs.python.org/2/library/sqlite3#sqlite3.Connection.isolation_level
> - http://mysql-python.sourceforge.net/MySQLdb.html => conn.autocommit()
> -
> http://cx-oracle.sourceforge.net/html/connection.html#Connection.autocommit
>
> This obviously has far-reaching consequences on transaction handling in
> Django, but the current APIs should still work. (Fixing them is part 2 of
> the plan.) The general idea is that Django will explicitly start a
> transaction when entering transaction management.
>
> This will obviously impact maintainers of backend for other databases, but
> if it works on Oracle (which doesn't have autocommit — it's emulated in OCI)
> and on PostgreSQL (which enforces autocommit), I hope it can work anywhere.
>
> ** Backwards-compatibility **
>
> Roughly, I'd classify Django users in four groups:
> 1 - "Transactions, how do they work?"
> 2 - "Django autocommits, doesn't it?"
> 3 - "I'm using TransactionMiddleware!"
> 4 - "I'm managing my transactions."
>
> Groups 1 and 2 won't see the difference. There won't be any difference for
> group 3. Group 4 may be impacted by the change, but I believe most people in
> this category have autocommit turned on already — or would like to, if
> they're on MySQL — and will understand the change.
>
> I don't see much point in providing an option to turn autocommit off,
> because starting a transaction is a much more explicit way to achieve the
> same effect. We usually don't provide "backwards compatibility with bugs".
>
> Yay or nay?
>
> --
> Aymeric.
>
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers?hl=en.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Switch to database-level autocommit

[Christophe Pettus]

On Mar 1, 2013, at 4:48 AM, Aymeric Augustin wrote:
> Yay or nay?

+1.
--
-- Christophe Pettus
   x...@thebuild.com

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Switch to database-level autocommit

[Alex Gaynor]

+1 from me. Here's a patch to add autocommit to MySQL:
https://github.com/django/django/pull/857

FWIW: any sort of scheme where a transaction is kept open all request
(including a transaction that only ever reads), will cause you serious pain
if you're trying to do migrations on MySQL with traffic.

Alex


On Fri, Mar 1, 2013 at 7:07 AM, Carl Meyer  wrote:

> On 03/01/2013 05:48 AM, Aymeric Augustin wrote:
> > Basically, Django intends to provide autocommit by default. Rather than
> > fight the database adapter that itselfs fights the database, I propose
> > to simply turn autocommit on, and stop implicitly starting and
> > committing transactions. Explicit is better than implicit.
>
> +1.
>
> Carl
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers?hl=en
> .
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>


-- 
"I disapprove of what you say, but I will defend to the death your right to
say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
"The people's good is the highest law." -- Cicero

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Switch to database-level autocommit

[Carl Meyer]

On 03/01/2013 05:48 AM, Aymeric Augustin wrote:
> Basically, Django intends to provide autocommit by default. Rather than
> fight the database adapter that itselfs fights the database, I propose
> to simply turn autocommit on, and stop implicitly starting and
> committing transactions. Explicit is better than implicit.

+1.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Switch to database-level autocommit

[Aymeric Augustin]

On 1 mars 2013, at 14:38, Javier Guerra Giraldez  wrote:

> On Fri, Mar 1, 2013 at 7:48 AM, Aymeric Augustin
>  wrote:
>> To alleviate the pain, Django commits after each ORM write.
> 
> just curious:  why is it so?   i can't think of any reason not to tie
> transaction to the request/response cycle by default. (ie what
> TransactionMiddleware does).

TransactionMiddleware makes efficient connection pooling nearly
impossible.

> this is the default on a few other frameworks i use, and has already
> saved my behinds more than once.


I agree that it's a good default for small-to-medium sites.

This is somewhere in a later part of my plan :)

-- 
Aymeric.



-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Switch to database-level autocommit

[Javier Guerra Giraldez]

On Fri, Mar 1, 2013 at 7:48 AM, Aymeric Augustin
 wrote:
> To alleviate the pain, Django commits after each ORM write.

just curious:  why is it so?   i can't think of any reason not to tie
transaction to the request/response cycle by default. (ie what
TransactionMiddleware does).

this is the default on a few other frameworks i use, and has already
saved my behinds more than once.


-- 
Javier

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Switch to database-level autocommit

[Florian Apolloner]

Yay (+1) and preferably ASAP to get it tested by users right now.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Switch to database-level autocommit

[Donald Stufft]

On Friday, March 1, 2013 at 7:48 AM, Aymeric Augustin wrote:
> Hello,
>  
> I'd like to improve transactions handling in Django. The first step is the 
> current emulation of autocommit with database-level autocommit.
>  
> ** Rationale **
>  
> PEP249 says that any SQL query opens a transaction and that it's the 
> application's job to commit (or rollback) changes.  This model is also 
> required by the SQL standard. But it isn't very developer-friendly.
>  
> To alleviate the pain, Django commits after each ORM write.  Unfortunately, 
> this means that each read opens a transaction, that eventually gets committed 
> after the next write, or rolled back at the end of the query. Such 
> transactions are useless and don't come for free. Relying on them to enforce 
> integrity is extremely fragile — what if an external library starts writing 
> to a log table in the middle of one of these implicit transactions? The term 
> "footgun" comes to mind.
>  
> Database authors have reached the same conclusion, and most databases 
> supported by Django use autocommit by default, ignoring the SQL standard. On 
> PostgreSQL and SQLite, this is the only mode available.
>  
> As a consequence, to implement the behavior mandated by PEP 249, the Python 
> libraries (psycopg2, sqlite3, etc.) automatically start transactions. And 
> then Django automatically commits them. This is not only wasteful, but also 
> buggy. It's the root cause of "idle in transaction" connections on 
> PostgreSQL. It's also sometimes poorly implemented: for instance, executing 
> "SAVEPOINT …" on SQLite commits implicitly. (It's arguably a bug in the 
> design of the sqlite3 module. The Python bug tracker suggests it's going to 
> be documented.)
>  
> Basically, Django intends to provide autocommit by default. Rather than fight 
> the database adapter that itselfs fights the database, I propose to simply 
> turn autocommit on, and stop implicitly starting and committing transactions. 
> Explicit is better than implicit.
>  
> ** Implementation **
>  
> All databases supported by Django provide an API to turn autocommit on:
>  
> - http://initd.org/psycopg/docs/connection.html#connection.autocommit
> - http://docs.python.org/2/library/sqlite3#sqlite3.Connection.isolation_level
> - http://mysql-python.sourceforge.net/MySQLdb.html => conn.autocommit()
> - http://cx-oracle.sourceforge.net/html/connection.html#Connection.autocommit
>  
> This obviously has far-reaching consequences on transaction handling in 
> Django, but the current APIs should still work. (Fixing them is part 2 of the 
> plan.) The general idea is that Django will explicitly start a transaction 
> when entering transaction management.
>  
> This will obviously impact maintainers of backend for other databases, but if 
> it works on Oracle (which doesn't have autocommit — it's emulated in OCI) and 
> on PostgreSQL (which enforces autocommit), I hope it can work anywhere.
>  
> ** Backwards-compatibility **
>  
> Roughly, I'd classify Django users in four groups:
> 1 - "Transactions, how do they work?"
> 2 - "Django autocommits, doesn't it?"
> 3 - "I'm using TransactionMiddleware!"
> 4 - "I'm managing my transactions."
>  
> Groups 1 and 2 won't see the difference. There won't be any difference for 
> group 3. Group 4 may be impacted by the change, but I believe most people in 
> this category have autocommit turned on already — or would like to, if 
> they're on MySQL — and will understand the change.
>  
> I don't see much point in providing an option to turn autocommit off, because 
> starting a transaction is a much more explicit way to achieve the same 
> effect. We usually don't provide "backwards compatibility with bugs".
>  
> Yay or nay?
+1  
>  
> --  
> Aymeric.
>  
>  
>  
> --  
> You received this message because you are subscribed to the Google Groups 
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com 
> (mailto:django-developers+unsubscr...@googlegroups.com).
> To post to this group, send email to django-developers@googlegroups.com 
> (mailto:django-developers@googlegroups.com).
> Visit this group at http://groups.google.com/group/django-developers?hl=en.
> For more options, visit https://groups.google.com/groups/opt_out.
>   
>   

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Switch to database-level autocommit

[Aymeric Augustin]

Hello,

I'd like to improve transactions handling in Django. The first step is the 
current emulation of autocommit with database-level autocommit.

** Rationale **

PEP249 says that any SQL query opens a transaction and that it's the 
application's job to commit (or rollback) changes.  This model is also required 
by the SQL standard. But it isn't very developer-friendly.

To alleviate the pain, Django commits after each ORM write.  Unfortunately, 
this means that each read opens a transaction, that eventually gets committed 
after the next write, or rolled back at the end of the query. Such transactions 
are useless and don't come for free. Relying on them to enforce integrity is 
extremely fragile — what if an external library starts writing to a log table 
in the middle of one of these implicit transactions? The term "footgun" comes 
to mind.

Database authors have reached the same conclusion, and most databases supported 
by Django use autocommit by default, ignoring the SQL standard. On PostgreSQL 
and SQLite, this is the only mode available.

As a consequence, to implement the behavior mandated by PEP 249, the Python 
libraries (psycopg2, sqlite3, etc.) automatically start transactions. And then 
Django automatically commits them. This is not only wasteful, but also buggy. 
It's the root cause of "idle in transaction" connections on PostgreSQL. It's 
also sometimes poorly implemented: for instance, executing "SAVEPOINT …" on 
SQLite commits implicitly. (It's arguably a bug in the design of the sqlite3 
module. The Python bug tracker suggests it's going to be documented.)

Basically, Django intends to provide autocommit by default. Rather than fight 
the database adapter that itselfs fights the database, I propose to simply turn 
autocommit on, and stop implicitly starting and committing transactions. 
Explicit is better than implicit.

** Implementation **

All databases supported by Django provide an API to turn autocommit on:

- http://initd.org/psycopg/docs/connection.html#connection.autocommit
- http://docs.python.org/2/library/sqlite3#sqlite3.Connection.isolation_level
- http://mysql-python.sourceforge.net/MySQLdb.html => conn.autocommit()
- http://cx-oracle.sourceforge.net/html/connection.html#Connection.autocommit

This obviously has far-reaching consequences on transaction handling in Django, 
but the current APIs should still work. (Fixing them is part 2 of the plan.) 
The general idea is that Django will explicitly start a transaction when 
entering transaction management.

This will obviously impact maintainers of backend for other databases, but if 
it works on Oracle (which doesn't have autocommit — it's emulated in OCI) and 
on PostgreSQL (which enforces autocommit), I hope it can work anywhere.

** Backwards-compatibility **

Roughly, I'd classify Django users in four groups:
1 - "Transactions, how do they work?"
2 - "Django autocommits, doesn't it?"
3 - "I'm using TransactionMiddleware!"
4 - "I'm managing my transactions."

Groups 1 and 2 won't see the difference. There won't be any difference for 
group 3. Group 4 may be impacted by the change, but I believe most people in 
this category have autocommit turned on already — or would like to, if they're 
on MySQL — and will understand the change.

I don't see much point in providing an option to turn autocommit off, because 
starting a transaction is a much more explicit way to achieve the same effect. 
We usually don't provide "backwards compatibility with bugs".

Yay or nay?

-- 
Aymeric.



-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: first() and last(), earliest() and latest()

[Camilo Nova]

+1 for first and last

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.