Re: Widening participation (Thoughts from DjangoCon)

[Jeremy Dunck]

An alternative that might work well is to triage tickets to mentors, so
that a list of tickets with willing mentors is available.  It would feel
less judge-y than "easy pickings" and also broaden the pool of tickets that
could be worked by a newcomer.  Of course it hinges on a willing pool of
mentors and making the time needed to do the work.


On Fri, Oct 26, 2018 at 2:43 PM Ian Foote  wrote:

> Hi Carlton,
>
> I've had similar thoughts sitting in the back of my mind for at least a
> couple of months, so thank you for sharing this. I agree that finding
> tickets is one of the big problems here, both for new contributors and for
> sprint leaders. At Pycon UK I took on the role of sprint leader along with
> Adam Johnson and directing people to appropriate tickets was a definite
> difficulty. I was also unaware of the django core mentorship list and will
> be joining that soon. I'm willing to spend some time mentoring a small
> number of people, life permitting.
>
> Ian
>
> On Fri, 26 Oct 2018 at 14:44, Carlton Gibson 
> wrote:
>
>> Hi All.
>>
>> OK, so last week I was at DjangoCon US in San Diego. (Thank you if you
>> organised that! Hi! if we met and chatted.)
>> I gave a talk ("Your web framework needs you!") inspired by the
>> discussion on the
>> 
>> DSF list and the proposal to dissolve Django Core
>> . (Can’t see the DSF list? Join
>> the DSF
>> 
>> .)
>> I was asking for more participation in general, and participation that is
>> more representative of the wider Django community in particular.
>>
>> There was lots of good input from many people, including (but not, at
>> all, limited to) representatives of groups such Pyladies, DjangoGirls, and
>> so on.
>>
>>
>> The recurring themes seem to me to fit into three categories:
>>
>>1. The importance of *mentoring*.
>>2. The difficulty of *finding tickets*.
>>3. The importance of *sprints*.
>>
>> The rest here is a summary of that. Hopefully it’s useful.
>>
>> Mentoring
>>
>> For whatever reasons, the exiting *Contributing How-To*
>>  doesn’t
>> lead to contributions from a demographic that matches the wider Django
>> Community.
>>
>> The point that came up again and again about this was that *mentoring*
>> is one of the best (perhaps the best?) tool in helping to change this.
>>
>> Django Core Mentorship
>>
>> We don’t have an official mentoring programme but we do have the 
>> django-core-mentorship
>> list .
>>
>> This must be about the best-kept secret in the Django world: it’s gets ≈0
>> traffic, but I told everybody at DjangoCon about it, and that they should
>> use it.
>>
>> If you are not on django-core-mentorship, and you’re willing to help
>> prospective contributors, please sign-up. I’m hoping we can drive some
>> traffic to it.
>>
>> Maybe there’s call for something more formal, but at least until DCM is
>> actually being used, that seems (to me) like something we can postpone.
>>
>> Finding Tickets
>>
>> The next thing was that there’s not enough guidance on what to work on.
>>
>> The guidance is to look for *Easy Pickings*. There are ≈1300 accepted
>> open tickets in TRAC. 13 of these are marked *Easy Pickings*.
>>
>> That’s not enough. I think we’re too tight with it (or need another
>> grade).
>>
>> There are *many* tickets which aren’t super hard: I put it that, most of
>> our community solve harder problems every day *using Django* than most
>> tickets require.
>>
>> Yes, they still require time, love, energy, etc — and maybe some
>> mentoring — but it’s not primary research, in the main.
>>
>> I talked to people who had (at the conference) got the test suite running
>> and such, but been overawed by the (for want of a better phrase) *sheer
>> face* of issue tracker.
>>
>> We would do well to invite people better here. (I don’t have instant
>> solutions.)
>>
>> Sprints
>>
>> I’m not historically a Django-sprinter. (I have too many children for
>> that TBH, but they’re getting older…)
>>
>> I always thought it was for a hard-core to work on hard issues.
>>
>> Shows what I know… 🙂
>>
>> It was strongly impressed upon me that the real benefit of sprints is
>> being able to give new contributors the support they need to make their
>> first (or second or…) contribution.
>>
>> In particular, groups such as Pyladies can organise a sprint event with
>> the specific goal of helping members of the community get across the
>> barriers to contributing. This can reach numbers that otherwise simply
>> aren’t possible. (So wow. Basically.)
>>
>> Sprints & Mentoring
>>
>> Obviously having mentors at sprints is a key component.
>>
>> But even if you (or I) can’t attend a sprint, maybe we can (sometimes) be
>> 

Re: TransactionManagementError is raised when autocommit …

[Jeremy Dunck]

You can use atomic just over the section that causes the error.  The issue
is that different db engines have different semantics under error during
transaction. Rolling back to the last savepoint (as atomic does when
nested) recovers the ability to complete the remainder of the transaction.
With a savepoint to roll back to, in some DBs, the full transaction is lost.
On Mar 14, 2016 11:09 AM, "Tore Lundqvist"  wrote:

> I recently upgrade Django 1.7->1.8 and ended up needing to patch
> set_rollback()  and comment out the "exception if not in an atomic block"
> part. The problem was that sometimes database errors like IntegrityError
> occurred because multiple workers was trying to create the same data but
> that's ok, the code captures the error and does a get instead of a create
> and moves on. A rollback was not an option here as that wound roll back a
> lot of other things that was ok. Way do you need to be in a atomic block to
> use this function? Is the filosofi of Djangos transaction handling that an
> exception from the database should always be handled with a rollback no
> matter what?
>
> Cheers
> Tore
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/a8daaa98-f66a-40e5-bb4d-c0d4ad647f70%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAM0i3f4i5qi%3DHH%2B4kU8Kh%3D2kTgprCUNHQEWW4EmvYwpNV4hRKQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: TransactionManagementError is raised when autocommit …

[Jeremy Dunck]

I've had this scenario before - you have two interleaving units of work
(through composition of code from different sources or concerns).  You want
progress recorded for one unit of work, but perhaps not the other.  Without
django, you'd have two open connections.  In my experience the simplest way
to accommodate this is to have two DB aliases pointed at the same DB.  Set
one to be a test mirror of the other.  Use different aliased connections
for the two concerns.  Then you can use atomic (as suggested and typical).

On Fri, Mar 4, 2016 at 2:11 PM, Tore Lundqvist  wrote:

> I don't what all updates to be in one commit each so I can't wrap just the
> update with a atomic block I would need to have it over a bigger chuck of
> code. That chunk might call a subrutin that also needs a commit and if I
> wrap that update in a atomic block that atomic block is nested and results
> in a save point which is useless.
>
> Den fredag 4 mars 2016 kl. 22:46:30 UTC+1 skrev Aymeric Augustin:
>>
>> If you do what Simon and I suggest, you should get the result you just
>> described. If you don’t, please explain what happens.
>>
>> Within a transaction — and disabling autocommit means you’re within a
>> transaction — transaction.atomic() uses savepoints.
>>
>> Note that in Django 1.6, after set_autocommit(False), you couldn’t use
>> transaction.atomic(). That was fixed in 1.8 (I think).
>>
>> --
>> Aymeric.
>>
>> On 04 Mar 2016, at 21:21, Tore Lundqvist  wrote:
>>
>> Hi, Simon
>>
>> No, I would need to wrap everything i a atomic block to start the
>> transaction and it's only when that outermost atomic block exits that I
>> actually get a commit the nested ones just makes save point.
>>
>> /Tore
>>
>> Den fredag 4 mars 2016 kl. 21:09:17 UTC+1 skrev charettes:
>>>
>>> Hi Tore,
>>>
>>> Is there a reason you can't simply wrap your updates in a
>>> `transaction.atomic()` blocks?
>>>
>>> You should be able to avoid deadlocks and control exactly when data is
>>> written to disk
>>> with this construct.
>>>
>>> Simon
>>>
>>> Le vendredi 4 mars 2016 15:02:45 UTC-5, Tore Lundqvist a écrit :

 Reply to comments in ticket:
 https://code.djangoproject.com/ticket/26323#comment:10

 Hi,

 @Aagustin: I get your point but in the code I'm working on there is a
 lot of transaction.commit(), not to handle transactions but to manage when
 data is written to disk and to avoid deadlocks. Running in autocommit mode
 does not work, its slow and sometimes the commits is used to save in a
 known good state. So I disable autocommit with
 transaction.set_autocommit(False) and run the code with explicit commits in
 it and than turn autocommit on again.

 The documentation for set_autocommit says "Once you turn autocommit
 off, you get the default behavior of your database adapter, and Django
 won’t help you." That is what I want and thats way I think that
 the TransactionManagementError should not de raise if your not using atomic
 blocks.

>>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to django-developers+unsubscr...@googlegroups.com.
>> To post to this group, send email to django-d...@googlegroups.com.
>> Visit this group at https://groups.google.com/group/django-developers.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/django-developers/a6077f33-1113-4767-828c-8b2c0c77bd78%40googlegroups.com
>> 
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>>
>> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/79611d30-21e0-45bc-8ab7-8754a39db4fc%40googlegroups.com
> 
> .
>
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at h

Re: Testing Signals

[Jeremy Dunck]

I suspect you have 2 different definitions of the signal under different
import paths.  Ensure your python path doesn't have overlapping
directories, and that your signal imports refer to the same sys.modules key
(e.g. some_app.signals.foo all over, not proj.some_app.signals.foo and
some_app.signals.foo in various places).

This might help troubleshoot:
https://gist.github.com/jdunck/857091





On Fri, Aug 14, 2015 at 2:39 PM, Grant Means  wrote:

> I posted this in Django users but didn't get any reply. I hope you don't
> mind me asking here. I does involve Django internals.
>
> Simply put, I have an app that has a model that's consumed by other
> "pluggable" apps. This model will send certain signals during certain
> events that the pluggable apps can consume and respond to.
>
> I'd like to test that my model indeed sends the expected signal during
> specific actions. I wrote a test that connects to the signal in setUp() and
> uses a local listener function to set a class attribute to True if the
> handler is called. You can see the relevant code here:
>
> https://dpaste.de/mAGw
>
> This works fine if I test the module directly using ./manage.py test 
> test_models.py. However if I run test  or simply test the signals
> don't appear to connect. Using PyCharm I stepped through the code and found
> that when I call .connect() in my TestCase, and step into Signal.connect()
> I can see all of the expected receivers on `self.receivers`.
>
> However if I step into the Signal.send() method when the signal is fired,
> none of the expected receivers are in place. Again, this works if I test
> the module directly just not if I use `test ` or `test`.
>
> Does anyone have any guidance on how I could get this to work? Thanks!
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/6d8a3d9b-b04b-4a08-93c4-8363834e41b2%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAM0i3f71E2wrKuQeu2sDjrX36rUy7kvF_-iOb2JiDu%2BwFDvs6w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Multiple template engines for Django - week 13

[Jeremy Dunck]

If getting proper support for other template backends would only delay the
1.8 release timeline by a couple weeks, I think that is preferable to a
generalized 1.8 backend which only include DTL until 1.9.

What do others think?


On Sat, Jan 3, 2015 at 3:23 PM, Aymeric Augustin <
aymeric.augus...@polytechnique.org> wrote:

> Hello,
>
> Here’s the thirteenth update (good thing I learnt all these ordinals when
> I was a kid!)
> https://myks.org/fr/multiple-template-engines-for-django/#2015-01-04
>
> At this point my main concern is the 1.8 feature freeze scheduled in one
> week.
> Dealing properly with some of the items I listed in my report for week 11
> may
> require changes falling in the "new features” bucket and therefore being
> postponed to Django 1.9. At worst, I’ll have to document that some features
> only work with the Django template language in Django 1.8. That’s sad but
> not
> worth delaying the entire project until early 2016.
>
> --
> Aymeric.
>
>
>
> > On 27 déc. 2014, at 23:59, Aymeric Augustin <
> aymeric.augus...@polytechnique.org> wrote:
> >
> > Hello,
> >
> > My twelfth update is online:
> > https://myks.org/en/multiple-template-engines-for-django/#2014-12-28
> >
> > It comes with a question — should I drop `select_template` from the
> backend API? I think I should. Follow the link above for details.
> >
> > Looking forward to your feedback,
> >
> > --
> > Aymeric.
> >
> >
> >
> >> On 20 déc. 2014, at 23:57, Aymeric Augustin <
> aymeric.augus...@polytechnique.org> wrote:
> >>
> >> Hello,
> >>
> >> I haven’t written to this mailing-list for three weeks because nothing
> warranted your immediate attention.
> >>
> >> Now the code is in reasonably good shape. Feel free to have a look:
> >> https://github.com/aaugustin/django/commits/multiple-template-engines
> >>
> >> I plan to merge this branch within a few days.
> >>
> >> Updates for weeks 9 to 11 are available at the usual location:
> >> https://github.com/aaugustin/django/commits/multiple-template-engines
> >>
> >> --
> >> Aymeric.
> >>
> >>
> >>
> >>> On 30 nov. 2014, at 10:37, Aymeric Augustin <
> aymeric.augus...@polytechnique.org> wrote:
> >>>
> >>> Hello,
> >>>
> >>> Here’s my eight update — this is getting repetitive :-)
> >>> https://myks.org/en/multiple-template-engines-for-django/#2014-11-30
> >>>
> >>> --
> >>> Aymeric.
> >>>
> >>>
> >>>
>  On 23 nov. 2014, at 00:02, Aymeric Augustin <
> aymeric.augus...@polytechnique.org> wrote:
> 
>  Hello,
> 
>  I published my seventh update:
>  https://myks.org/en/multiple-template-engines-for-django/#2014-11-23
> 
>  I have another pull request ready for review:
>  https://github.com/django/django/pull/3605
> 
>  Let me know if you have any questions!
> 
>  --
>  Aymeric.
> 
> 
> 
> > On 16 nov. 2014, at 09:19, Aymeric Augustin <
> aymeric.augus...@polytechnique.org> wrote:
> >
> > Hello,
> >
> > Here’s my sixth update:
> > https://myks.org/en/multiple-template-engines-for-django/#2014-11-16
> >
> > I have a first pull request ready for review:
> > https://github.com/django/django/pull/3555
> >
> > Whenever possible, I’ll merge changes that make sense regardless of
> the Multiple Template Engines Project in small batches, in order to
> minimize the size of the final pull request.
> >
> > --
> > Aymeric.
> >
> >
> >
> >> On 9 nov. 2014, at 22:05, Aymeric Augustin <
> aymeric.augus...@polytechnique.org> wrote:
> >>
> >> Hello,
> >>
> >> Here’s my fifth update:
> >>
> https://myks.org/en/multiple-template-engines-for-django/#2014-11-09
> >>
> >> The first step of my project, as outlined in my original proposal,
> is complete.
> >>
> >> --
> >> Aymeric.
> >>
> >>
> >>
> >>> On 1 nov. 2014, at 23:30, Aymeric Augustin <
> aymeric.augus...@polytechnique.org> wrote:
> >>>
> >>> Hello,
> >>>
> >>> I’m happy to annonce that the DEP is ready for public review!
> >>>
> >>> Direct link, HTML:
> >>> https://myks.org/en/multiple-template-engines-for-django/dep/
> >>>
> >>> Direct link, ReStructured Text:
> >>>
> https://raw.githubusercontent.com/aaugustin/mtefd/master/multiple-template-engines.rst
> >>>
> >>> (I’m not reproducing it here because I don’t think email is a very
> good medium for communicating 50kB of markup.)
> >>>
> >>> I’ve written a bit more about what “ready” means:
> >>>
> https://myks.org/en/multiple-template-engines-for-django/#2014-11-01
> >>>
> >>> Of course, your regularly scheduled update will also be available
> (in half an hour):
> >>>
> https://myks.org/en/multiple-template-engines-for-django/#2014-11-02
> >>>
> >>> I’m looking forward to your feedback!
> >>>
> >>> --
> >>> Aymeric.
> >>>
> >>>
> >>>
>  On 26 oct. 2014, at 22:36, Aymeric Augustin <
> aymeric.augus...@polyt

Re: status of 1.8 release blockers

[Jeremy Dunck]

Thank you, Tim, for shepherding this.
On Jan 3, 2015 8:09 AM, "Tim Graham"  wrote:

> Here is the fourth update with a week to go until alpha. At this time, I
> am thinking we'll have the feature freeze on Monday, January 12 as planned,
> but perhaps issue the actual alpha release a couple days later just to give
> some time for some extra polish in case any large patches are committed
> next weekend.
>
> #23861 -
>  Fields referenced in
> migrations cannot be (re)moved, even if migrations have been squashed
> 
> #23891  - Revise deprecation
> / removal timeline for IPAddressField
>  (resolved)
> Owner: Tim
> Now: I committed a patch for the second ticket and will polish the patch
> for the first issue.
> Last time: I've implemented the system check solution I proposed and am
> waiting for review and/or concerns with this approach.
>
> After completing the issue above, I'll prioritize any issues here:
>
> https://code.djangoproject.com/query?status=assigned&status=new&keywords=~1.8-alpha
>
> Resolved since last time:
>
> #22340 -
>  Legacy Table Creation
> Methods Not Properly Deprecated
> 
> Owner: Tim
> Now: Resolved this by removing the deprecation as discussed in the ticket.
> Last time: I took a closer look at this and don't think we need to do this
> deprecation at all as it doesn't seem to offer any benefits. See the ticket
> for details.
>
> #23745 - Migrations migrate is slow
> 
> Owner: Claude/Markus
> Now: Patch committed; thanks to Markus and Claude.
> Last time: I committed the test that Markus wrote and Claude incorporated
> a fix from Markus so the branch appears about ready for merge. Markus will
> give it another review.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/b196f095-530c-45ec-9b21-82fcd1c06ed8%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAM0i3f5KyOKN_z_OExai9ExkkThPQL3570XW0w6E6EJsO1MNQA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Storage engine aliases?

[Jeremy Dunck]

Aymeric,
  That's an interesting idea I hadn't considered -- perhaps storages could
also then be marked trusted and untrusted, and processing/display of those
files could take it into consideration.  I agree the security requirements
are different.

Florian,
  I agree that changing APIs would cause confusion.  It may be possible to
provide an upgrade path as we did for DATABASES and CACHES, though.

  The benefit I'm shooting for is to make media handling more modular and
clear, rather than seeming a special case.

  As for instances, yes, I was using that as a shorthand -- even so, it
could be a dict with common interface (**kwargs) handed to the storage
backend.  FileSystemStorage takes location=None, base_url=None, s3boto
takes acl=None, bucket=None, **settings.  I doubt there will be much common
interface, but to me this supports the argument that aliases would be
useful -- right now it is hard to construct a storage backend dynamically
without coding to a specific store.  connections['default'] is a powerful
abstraction.

This could be a migration path:

FILE_STORAGES = {} -> expands to
  'static': {
'class': 'django.contrib.staticfiles.storage.StaticFilesStorage',
'trusted': True, # based on Aymeric's feedback
'OPTIONS': {
   'location': settings.STATIC_ROOT,
   'base_url': settings.STATIC_URL
},
  },
  'media': {
'class': 'django.core.files.storage.FileSystemStorage',
'trusted': False,
'OPTIONS': {
  'location': settings.MEDIA_ROOT,
  'base_url': settings.MEDIA_URL
}
  }
}



On Mon, Sep 29, 2014 at 2:26 PM, Aymeric Augustin <
aymeric.augus...@polytechnique.org> wrote:

> Hi Jeremy,
>
> That could be useful for any website that gets some of its assets from the
> code (JS, CSS), others from a CDN (eg. product photos), others from another
> CDN (eg. tutorial illustrations), etc. However we’d have to make sure it
> beats the common solution of having a model instance for each file and a
> method that generates the URL.
>
> Even though the documentation has improved a lot, static files still tend
> to confuse beginners. On one hand, making them less of a special case could
> help. On the other hand changing the APIs will create confusion again.
>
> Finally, static and media (user-uploaded) file have different
> requirements, especially in terms of security. I think it’s useful to keep
> the concepts separate, even if they ultimately depend on the same APIs —
> basically the Storage base class that defines the usual file APIs plus an
> URL.
>
> --
> Aymeric.
>
>
> On 29 sept. 2014, at 22:46, Jeremy Dunck  wrote:
>
> Right now, I think that static/media handling is fairly confused in the
> documentation, and a bit confused in the code itself.
>
> We have a few special-cases floating around:
>
>default_storage (needed for legacy before storage backends)
>staticfiles_storage (needed for collectstatic/handling)
>{% static %} handles mapping relative URLs to absolute URLs, but does
> not allow using a storage engine other than {% STATIC_URL %}
> or staticfiles_storage.
>
> I was surprised to find that
>django.contrib.staticfiles.templatetags
>and
>django.templatetags.static
> both have implementations of {% static %} with slightly different
> semantics (one based on STATIC_URL, one based on staticfiles_storage).
>
> It seems to me that it might be useful to introduce aliases (as in CACHES
> and DATABASES) in order to allow referring to storage engines in a
> less-coupled way.
>
> {% static %} could then take a storage engine alias, and the special-case
> of repo-static file handling and user-uploaded file handling could mostly
> go away.
>
> MEDIA_URL/MEDIA_ROOT and STATIC_URL/STATIC_ROOT as special cases could
> (after a deprecation process) go away in favor of these storage aliases.
>
> The syntax I'll hazard as a proposal could be:
>
> FILE_STORES = {
>'static': ,
>'dynamic': ,
>
>'alias1': 'alias2' # possibly map aliases to satisfy app-required
> aliases.
> }
>
> {% file relative-url storage-alias %}
>
> Would do people think of this idea?
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
&

Storage engine aliases?

[Jeremy Dunck]

Right now, I think that static/media handling is fairly confused in the
documentation, and a bit confused in the code itself.

We have a few special-cases floating around:

   default_storage (needed for legacy before storage backends)
   staticfiles_storage (needed for collectstatic/handling)
   {% static %} handles mapping relative URLs to absolute URLs, but does
not allow using a storage engine other than {% STATIC_URL %}
or staticfiles_storage.

I was surprised to find that
   django.contrib.staticfiles.templatetags
   and
   django.templatetags.static
both have implementations of {% static %} with slightly different semantics
(one based on STATIC_URL, one based on staticfiles_storage).

It seems to me that it might be useful to introduce aliases (as in CACHES
and DATABASES) in order to allow referring to storage engines in a
less-coupled way.

{% static %} could then take a storage engine alias, and the special-case
of repo-static file handling and user-uploaded file handling could mostly
go away.

MEDIA_URL/MEDIA_ROOT and STATIC_URL/STATIC_ROOT as special cases could
(after a deprecation process) go away in favor of these storage aliases.

The syntax I'll hazard as a proposal could be:

FILE_STORES = {
   'static': ,
   'dynamic': ,
   
   'alias1': 'alias2' # possibly map aliases to satisfy app-required
aliases.
}

{% file relative-url storage-alias %}

Would do people think of this idea?

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAM0i3f6sMwgFxAVC4P6U9Z97zAd%2B%3DmDV837KbYZmBx3ycuu%2BMg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Two proposals for the Django Code of Conduct.

[Jeremy Dunck]

As someone affected by an issue that would fall under the proposed change
[1], I still support an explicit guideline about external behavior
influencing internal acceptance.  The safety of all members is more
important than the risk of misapplication of the rule.

[1]
http://doubleunion.tumblr.com/post/77929475144/how-not-to-support-women-in-tech
On Sep 9, 2014 11:47 AM, "Alex Gaynor"  wrote:

> When Jacob and I originally drafted the CoC, we specifically included an
> enumeration of some disallowed behaviors on the recommendation of the Ada
> Initiative -- it was their view that the list helped to minimize rules
> lawyering, whereby someone attempts to explain how they could not have
> known their behavior was disallowed.
>
> On reflection, I completely agree with their reasoning and am very glad we
> included it.
>
> Alex
>
>
> On Tue, Sep 9, 2014 at 11:43 AM, barbara.shaurette <
> barbara.shaure...@gmail.com> wrote:
>
>> As someone who has been the target of harassment at conferences (I've
>> been lucky, only minor incidents for me, although the same can't be said
>> for other of my female colleagues), I prefer explicit over implicit. If
>> someone is a harasser outside the community, I won't feel safe encountering
>> them in a conference setting either.
>>
>> That said, I trust the discretion of the core team and the DSF membership
>> and I'll be interested to see how they decide on the matter.
>>
>>  --
>> You received this message because you are subscribed to the Google Groups
>> "Django developers" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to django-developers+unsubscr...@googlegroups.com.
>> To post to this group, send email to django-developers@googlegroups.com.
>> Visit this group at http://groups.google.com/group/django-developers.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/django-developers/9f89cb24-f572-444d-9723-386eb93e495a%40googlegroups.com
>> 
>> .
>>
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
>
> --
> "I disapprove of what you say, but I will defend to the death your right
> to say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
> "The people's good is the highest law." -- Cicero
> GPG Key fingerprint: 125F 5C67 DFE9 4084
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/CAFRnB2XfgpLy-zrb%2BSF%3D2Cz9KzErEWANYmtwmDCLn1xb8wS8yQ%40mail.gmail.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAM0i3f5yRYBy6qGO1R2x9QOANi_xRDCTVLBuZ7qZL2%3DwMjduPQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Would AssertMaxQueries (similar to AssertNumQueries) be a useful addition

[Jeremy Dunck]

I use this method on my own test subclasses, and I find it useful as a
tripwire: a cause for review and consideration, more than a hard error. Did
the number of queries go up on this change? Is that reasonable or a
mistake? Have we blown the perf budget so we should refactor? Or maybe the
number should just be raised to something that seems like a reasonable next
tripwire.

These limits aren't flaky or false positives, so they seem useful supports
given a reasonable test suite otherwise.
On Aug 17, 2014 8:20 AM, "Florian Apolloner"  wrote:

> I am not so convinced, what would you put in as the upper limit? While
> preventing n+1, it still requires you to know what n in this testcase is
> and changing n can lead to funny errors. Currently we are documenting
> (hopefully) how those query counts come together, so it's clear what's
> happening when it breaks…
>
> On Sunday, August 17, 2014 4:39:17 PM UTC+2, James Bennett wrote:
>>
>> I like the idea -- if nothing else, it would provide an easy way to have
>> a test suite discover N+1 problems, especially in templates.
>>
>> Not sure about the feasibility, though; would you be willing to work up a
>> rough implementation of it?
>>
>  --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/41676693-480d-4dfc-82a6-377da0f54057%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAM0i3f7nC_hVDwL2MojcQn3Qhex8TGPZnYwdYh%3DTH%2BcWN1yZ6w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

narrow writes (as 3rd-party library)

[Jeremy Dunck]

I'm attempting to implement narrow writes (that is, writing only fields
which have changed).

I would be able to do this as a 3rd-party Mixin library if some changes
were made to Model.save_base.

  1)  returned whether the row was created or updated, e.g. if .save_base
returned the `updated` value which it uses to send the .post_save signal.
  2) raise something finer-grained than DatabaseError when force_update is
used (so that I could then try a (wide) insert-or-update.  The new
exception could inherit from DatabaseError for backwards-compat.

Any objections to me making these changes?

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAM0i3f5tnHVeHeNA5cqXOzTyN-S6YbjBRtOZm7o0-noLSHiQdQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: django-firebird backend: migrations and NULL fields

[Jeremy Dunck]

How about adding a flag to Operations? implied_null, perhaps.
On May 14, 2014 7:52 AM, "Andrew Godwin"  wrote:

> Hi,
>
> That's currently the only approach I'm afraid - there's an open issue
> (raised by Shai Berger I believe) that column_sql should be broken down
> into more component pieces so it can be more easily changed, but that's a
> difficult task to do and not something I want to go and change completely
> as we're trying to release 1.7.
>
> What we could do is change those NULL and NOT NULL strings to be defined
> on the schema editor class itself, and then you could set sql_null_suffix =
> "" or similar - do you think that would be a reasonable approach?
>
> Andrew
>
>
> On Wed, May 14, 2014 at 7:05 AM, maxi  wrote:
>
>> Hi,
>>
>> I'm trying to implement the new schema migration of django 1.7 for
>> django-firebird backend.
>> In column_sql method, when the field can be null, in firebird, is not
>> necessary add the NULL keyword. So to change this behavior I need override
>> the entire column_sql method just to change one line:
>>
>>
>> if null:
>> sql += " NULL"<-- Here
>> else:
>> sql += " NOT NULL"
>>
>> I just need:
>> if not null:
>> sql += " NOT NULL"
>>
>>
>> Is the correct approach rewrite the column_sql method? Are there other
>> approach?
>>
>> Regards.
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Django developers" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to django-developers+unsubscr...@googlegroups.com.
>> To post to this group, send email to django-developers@googlegroups.com.
>> Visit this group at http://groups.google.com/group/django-developers.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/django-developers/e9f2-a5bf-4e13-b7c1-0a08abd18a4a%40googlegroups.com
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>  --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/CAFwN1upTtLEmGjz5f2vUBV6kEFDqdiyRyz4GhM3i3WbSyot0aw%40mail.gmail.com
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAM0i3f6Jw6Bs8FEeTmU7wVwA3SQKwcruKqtTReCa_-GrxbZmww%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Proposal for prepared statements API

[Jeremy Dunck]

On the None -> IS NULL issue, I presume there are, for any given use case,
not that many argument permutations of None and not None passed.  I suggest
that the PreparedStatement abstraction map to multiple actual prepared
statements, one for each None/not None permutation.  Then when executing,
you know the value, and you know to use the (None, not None) statement if
given (a=None, b=1) for example.

(This sort of mapping of object to different statements may be needed for
fallback on backends that don't support prepareds, as well.)

As for deferring get_prep_lookup until execution, do we lose any pruning or
other optimization opportunities for normal QuerySet usage if we go that
path?





On Tue, Mar 25, 2014 at 7:22 AM, Anssi Kääriäinen
wrote:

> On Tuesday, March 25, 2014 2:53:42 PM UTC+2, Curtis Maloney wrote:
>>
>> ps = MyModel.objects.filter(foo__lt=Param('a').prepare()
>>
>> The result is now a callable that accepts one parameter - "a".  To invoke
>> the query:
>>
>> results = ps(a=1000)
>>
>>
>> Clearly it's early days yet - I've written no code.  And akaariai has
>> pointed out already there's some corners cases which won't work well with
>> existing behaviours (e.g. foo=None being silently translated to
>> foo__isnull=True), but it's best to get this idea under wider public
>> scrutiny earlier, rather than later.
>>
>
> I like this style of prepared statements. It is explicit, and
> implementation should be achievable without too much added code complexity.
> I prefer ps.execute(a=1000) personally, but the exact syntax isn't that
> important at this stage.
>
> There will be a couple of corner cases that will be hard to solve. The
> problems are around value preparation during .filter() calls and how
> certain special values are dealt with. Quickly thinking the value
> preparation (basically get_prep_lookup() call) shouldn't be that much of a
> problem - it is currently done during .filter() calls, but it should be
> possible to defer it to execution time.
>
> The foo=None case is something that likely can't be solved. The problem
> here is that foo=None translates to WHERE foo IS NULL, while foo=1
> translates to WHERE foo = 1. These are syntactically different queries, and
> thus single prepared statement can't handle both of these. There are also
> cases where isnull=True/False require different join promotion depending if
> True or False is supplied. These again can't be handled.
>
> I am OK for just documenting these corner cases. They aren't something
> that should happen too often. The implementation for prepared statements is
> relatively straightforward (generate SQL, prepare it once, execute using
> given values), but if the corner case needs to be handled the
> implementation will be more complex, likely so much more complex that
> nobody will implement this feature.
>
> In short: +1 for implementing this with documentation of the corner cases.
>
>  - Anssi
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/3123fda6-d7b3-46d3-82ec-28ed3003e837%40googlegroups.com
> .
>
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAM0i3f4JVV4%2B7fCV4%3Dj_Pn%2BqRAicVNp1YNBKYRFhEVtVQTsZRw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Callable LazyObject?

[Jeremy Dunck]

(It's a change in the DB code, not the template, but your point stands.)

I think I agree with you that changing LazyObject is risky when there's a
workaround and it would be difficult to address the risk of
callable(LazyObject()) becoming true.

OK, thanks for feedback.


On Thu, Mar 6, 2014 at 8:12 AM, Luke Plant  wrote:

> On 05/03/14 23:05, Jeremy Dunck wrote:
>
> > if ...
> > elif isinstance(value, LazyObject):
> >   pass
> > elif callable(value):
> >   ...
>
> My gut instinct is that if Django's template code has to be patched and
> special cased to accommodate this change, there will be lots of other
> code that needs to be patched too.
>
> LazyObject is already pretty hairy without trying to get it to support
> more things.
>
> Also, it seems that normally there should be other solutions. In many
> cases, instead of:
>
>   User = SimpleLazyObject(lambda: get_user_model())
>
> you could use:
>
>   User = lambda **kwargs: get_user_model()(**kwargs)
>
> ...assuming that the only thing you need 'User' to do is to produce User
> instances when you call it. (If you need it to be an actual class for
> some reason, then this won't work - but in most cases I'd suggest that
> the consuming code shouldn't need an actual class).
>
> Regards,
>
> Luke
>
>
> --
> "God demonstrates his love towards us in this, that while we were
> still sinners, Christ died for us." (Romans 5:8)
>
> Luke Plant || http://lukeplant.me.uk/
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/53189E5F.7010306%40cantab.net
> .
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAM0i3f4Li3Bqsz6sd1d3LURZzeOH_gQ05gK4XUdF_7GsdAL%2B8A%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Callable LazyObject?

[Jeremy Dunck]

I recently had a need for a LazyObject which was callable.  The __call__
meta method isn't forwarded to _wrapped, so it's an error to call, even if
the underlying _wrapped does support it.

In my case, was trying to do the following:

User = SimpleLazyObject(lambda: get_user_model())

User()...

I patched LazyObject to make it happy:

def _a_call_(self, *args, **kwargs):
if self._wrapped is empty:
self._setup()
return self._wrapped(*args, **kwargs)

LazyObject.__call__ = _a_call_.__get__(User)

Unfortunately, I then ran into a related problem -- by making LazyObject
callable, anyone passing a lazy user, e.g. request.user, to the ORM would
fall into the callable(value) branch:
https://github.com/django/django/blob/b77f26313cddbfde20dcf2661e9bd35458c2d1bd/django/db/models/sql/query.py#L1043

That then caused a new User to be constructed rather than using the
request.user.pk.

I see that callable criteria are deprecated there, but was wondering how
people would feel about adding a branch before that:

if ...
elif isinstance(value, LazyObject):
  pass
elif callable(value):
  ...

That would allow request.user to still be used (as it no doubt widely is)
even after LazyObject grows a callable (if its underlying object has one).

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAM0i3f7hshHzW9XsuRkJaT%2BaVEMetTKkzD7nUiDqMF9%2BzAR4rw%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Benchmarking 1.5 vs 1.6

[Jeremy Dunck]

It may be useful to place a comment where @wraps was removed, lest some later 
do-gooder "fix" it.

On Sep 17, 2013, at 12:06 AM, Marc Tamlyn  wrote:

> Can't say I'm hugely worried about perfect tracebacks and introspection with 
> internal kind of functions here if it affects performance. The patch looks 
> good. I'd add a comment to say why we're not using @wraps in that context 
> though so someone doesn't go back and add it.
> 
> On 17 Sep 2013 07:57, "Anssi Kääriäinen"  wrote:
>> I did some investigation to query_iterator, and most of the slowdown is 
>> caused by functools.wraps. When used in nested context @wraps seems to be 
>> really slow. For example, try this with and without @wraps: 
>> http://pastebin.com/5ejAKuKd
>> 
>> Should we care about this slowdown? Can we do without @wraps? My 
>> understanding is that @wraps gives you __name__, __module__ and __doc__ 
>> which are easy to do manually. In addition it gives introspection the right 
>> args and kwargs for the wrapped function (so that IDEs see it correctly). 
>> This will be a lot harder to do manually, and doing so will likely result in 
>> similar slowdown.
>> 
>> The biggest hit case is if somebody is looping over a raw SQL queryset with 
>> .fetchone(). This use case will be severely punished. Even .execute(); 
>> .fetchmany() has a 1.5x slowdown (the benchmark is this one: 
>> https://github.com/django/djangobench/commit/94fc28d99f95c65d26d0fad8af44e46c49282220
>>  - simple SELECT + fetchall() for the ten returned rows.
>> 
>> I have created a patch that removes the usage of @wraps and makes 
>> connection.wrap_database_errors a cached_property. The cached_property 
>> should be safe as the context manager created by wrap_database_errors is 
>> stateless. This patch gets rid of the slowdown (1.00-1.05x slower in 
>> repeated runs for raw_sql benchmark). With just removed @wraps the slowdown 
>> is 1.1x. The patch is here: 
>> https://github.com/akaariai/django/commit/eafc373c16350abe51c565c8aefbe36cabf5219e
>> 
>> Should I apply the patch, and should I apply it to stable/1.6.x, too? 
>> Removal of @wraps is really low risk, usage of @cached_property is also low 
>> risk, but not as low as removal of @wraps.
>> 
>>  - Anssi
>> 
>> On Saturday, September 14, 2013 9:21:43 PM UTC+3, Anssi Kääriäinen wrote:
>>> 
>>> I just ran djangobench comparing 1.5 to 1.6. The biggest thing seems to be 
>>> form_create bechmark, the average is 15x slower. But for some reason the 
>>> minimum runtime is just 1.16x slower. This seems a bit strange. This might 
>>> be worth more investigation.
>>> 
>>> Otherwise there doens't seem to be anything alarming going on. Most ORM 
>>> operations are slightly faster (likely due to __deepcopy__ removal).
>>> 
>>> There are a couple of individual benchmarks worth mentioning... url_reverse 
>>> and query_iterator are slower (1.15x and 1.12x respectively). The 
>>> default_middleware and qs_filter_chaining benchmarks have gained a lot 
>>> (2.0x and 3.1x respectively). model_save_existing/model_save_new are faster 
>>> (2x+), but model_creation is slightly slower. I suspect there is something 
>>> going on with connection creation or transactions for model_creation, after 
>>> all model_creation is a convenience method for model.__init__ + 
>>> model_save_new which both aren't slower.
>>> 
>>> What else? url_reverse and template compliation under i18n slightly slower. 
>>> For the rest of benchmarks, see below.
>>> 
>>> Running 'default_middleware' benchmark ...
>>> Min: 0.001260 -> 0.000554: 2.2755x faster
>>> Avg: 0.001389 -> 0.000681: 2.0394x faster
>>> Significant (t=8.222155)
>>> Stddev: 0.00037 -> 0.00048: 1.2843x larger (N = 50)
>>> 
>>> Running 'form_clean' benchmark ...
>>> Min: 0.42 -> 0.42: no change
>>> Avg: 0.49 -> 0.43: 1.1341x faster
>>> Not significant
>>> Stddev: 0.4 -> 0.1: 6.6718x smaller (N = 50)
>>> 
>>> Running 'form_create' benchmark ...
>>> Min: 0.85 -> 0.99: 1.1657x slower
>>> Avg: 0.88 -> 0.001396: 15.8410x slower
>>> Not significant
>>> Stddev: 0.1 -> 0.00916: 783.3314x larger (N = 50)
>>> 
>>> Running 'l10n_render' benchmark ...
>>> Min: 0.011702 -> 0.014151: 1.2093x slower
>>> Avg: 0.012200 -> 0.014846: 1.2169x slower
>>> Significant (t=-4.824893)
>>> Stddev: 0.00233 -> 0.00310: 1.3311x larger (N = 50)
>>> 
>>> Running 'model_creation' benchmark ...
>>> Min: 0.000311 -> 0.000400: 1.2860x slower
>>> Avg: 0.000336 -> 0.000432: 1.2840x slower
>>> Significant (t=-2.799691)
>>> Stddev: 0.00015 -> 0.00019: 1.2751x larger (N = 50)
>>> 
>>> Running 'model_delete' benchmark ...
>>> Min: 0.000407 -> 0.000468: 1.1500x slower
>>> Avg: 0.000423 -> 0.000484: 1.1440x slower
>>> Significant (t=-7.131460)
>>> Stddev: 0.5 -> 0.4: 1.1807x smaller (N = 50)
>>> 
>>> Running 'model_save_existing' benchmark ...
>>> Min: 0.062923 -> 0.021829: 2.8826x faster
>>> Avg: 0.063206 -> 0.022001: 2.8729x faster
>>> Significant (t=793.026302)
>>> Stddev: 0.00034 -> 0.0001

Re: Not calling things twice in templates

[Jeremy Dunck]

I've had this issue and have used {% with %} or moved the my.bonnet() call
into the view/context.I agree, not ideal, but I was never moved to make
it better in general.

If you're suggesting a general caching layer in the template such that a
given expression is only called once in the course of a template render
(including inheritance, includes, etc.) then I think you would need a way
of whitelisting which expressions were without side effects (similar to
is_safe w/ autoescaping).  It seems like a fair bit of book-keeping, but it
could clearly be added in a backwards-compatible way.




On Sun, Jun 2, 2013 at 2:36 PM, Daniele Procida  wrote:

> The for ... empty pattern in templates is common and useful: <
> https://docs.djangoproject.com/en/dev/ref/templates/builtins/#for-empty>
>
> But this is another common pattern:
>
> {% if my_bonnet.bees %}
> 
> {% for bee in my_bonnet.bees %}
> {{ bee }}
> ...
>
> In other words, we have to check first for items in the iterable before
> creating a  or whatever other furniture to put them in.
>
> The problem here is that my_bonnet.bees gets asked for twice, and it could
> be my.bonnet(), some very expensive function.
>
> One solution is to put everything inside:
>
> {% with my_bonnet.bees as bees %}
>
> but now:
>
> * we've used up a precious level of indentation
> * we've introduced a new variable in the templates to worry about
> * it just feels a bit fussy
>
> Is this enough of an issue to make it worthwhile implementing some other
> approach in the template system?
>
> Daniele
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers?hl=en
> .
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: django.utils.simplejson + stdlib json

[Jeremy Dunck]

After reading that ticket, I'm not sure what Alex and I are
disagreeing on.  I was suggesting using django.utils.simplejson
internally until it's removed, so that at least people could be
consistent with django itself.  That way, DjangoJSONEncoder would be a
simplejson.JSONEncoder (if simplejson is installed) until we remove
the shim.

I agree w/ deprecating utils.simplejson, and I agree that leaky shims
are a fundamental problem, but it does seem to me that the state of
play could be better by not using simplejson and json both in the same
codebase.  DjangoJSONEncoder(json.JSONEncoder) but
django.utils.simplejson is simplejson if it's available.

Anyway, since we've already deprecated utils.simplejson, since we've
already done the damage here, and since doing a release to fix this is
probably not going to help that many people (they've either already
fixed their backwards-incompat or aren't planning on going to 1.5 any
time soon)... maybe I agree we should leave it alone.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: django.utils.simplejson + stdlib json

[Jeremy Dunck]

Yes, sorry, am mobile right now. Thanks Mr Ogier. :)

On Apr 11, 2013, at 6:44 PM, Alex Ogier  wrote:

> I think what he is saying is that many third-party libraries call the
> equivalent of django.utils.simplejson.dumps("...",
> cls=DjangoJSONEncoder) which, despite django.utils.simplejson only
> being in a pending deprecation state, is broken.
> 
> On Thu, Apr 11, 2013 at 9:22 PM, Alex Gaynor  wrote:
>> When doing what? What do I need to do to trigger this?
>> 
>> Alex
>> 
>> On Apr 11, 2013 6:16 PM, "Jeremy Dunck"  wrote:
>>> 
>>> If a user of django has simplejson installed, django itself will use both
>>> the stdlib and simplejson.
>>> 
>>> On Apr 11, 2013, at 5:54 PM, Alex Gaynor  wrote:
>>> 
>>> I basically agree with what Bob said on the ticket, it's unclear to me
>>> from your email how this manifests, other than trying to use something from
>>> the standard library with simplejson, which is obviously wrong.
>>> 
>>> Alex
>>> 
>>> 
>>> On Thu, Apr 11, 2013 at 5:51 PM, Jeremy Dunck  wrote:
>>>> 
>>>> I've just seen a documented example of this breaking things in the wild.
>>>> 
>>>> https://github.com/simplejson/simplejson/issues/37
>>>> 
>>>> I think we broke backwards-compat here - django 1.5.1. plus sentry
>>>> 5.4.5 dies because django's own DjangoJSONEncoder depends on stdlib
>>>> json, but sentry (and lots of things) use django.utils.simplejson,
>>>> which uses simplejson if available.
>>>> 
>>>> Unfortunately, this is incompatible - we kept utils.simplejson as a
>>>> compat shim, but I think until it goes away, we should keep using it
>>>> internally as well.
>>>> 
>>>> Alternatives?
>>>> 
>>>> --
>>>> You received this message because you are subscribed to the Google Groups
>>>> "Django developers" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send an
>>>> email to django-developers+unsubscr...@googlegroups.com.
>>>> To post to this group, send email to django-developers@googlegroups.com.
>>>> Visit this group at
>>>> http://groups.google.com/group/django-developers?hl=en.
>>>> For more options, visit https://groups.google.com/groups/opt_out.
>>> 
>>> 
>>> 
>>> --
>>> "I disapprove of what you say, but I will defend to the death your right
>>> to say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
>>> "The people's good is the highest law." -- Cicero
>>> GPG Key fingerprint: 125F 5C67 DFE9 4084
>>> 
>>> --
>>> You received this message because you are subscribed to the Google Groups
>>> "Django developers" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an
>>> email to django-developers+unsubscr...@googlegroups.com.
>>> To post to this group, send email to django-developers@googlegroups.com.
>>> Visit this group at
>>> http://groups.google.com/group/django-developers?hl=en.
>>> For more options, visit https://groups.google.com/groups/opt_out.
>>> 
>>> 
>>> 
>>> --
>>> You received this message because you are subscribed to the Google Groups
>>> "Django developers" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an
>>> email to django-developers+unsubscr...@googlegroups.com.
>>> To post to this group, send email to django-developers@googlegroups.com.
>>> Visit this group at
>>> http://groups.google.com/group/django-developers?hl=en.
>>> For more options, visit https://groups.google.com/groups/opt_out.
>> 
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Django developers" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to django-developers+unsubscr...@googlegroups.com.
>> To post to this group, send email to django-developers@googlegroups.com.
>> Visit this group at http://groups.google.com/group/django-developers?hl=en.
>> For more options, visit https://groups.google.com/groups/opt_out.
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers?hl=en.
> For more options, visit https://groups.google.com/groups/opt_out.
> 
> 

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: django.utils.simplejson + stdlib json

[Jeremy Dunck]

If a user of django has simplejson installed, django itself will use both the 
stdlib and simplejson. 

On Apr 11, 2013, at 5:54 PM, Alex Gaynor  wrote:

> I basically agree with what Bob said on the ticket, it's unclear to me from 
> your email how this manifests, other than trying to use something from the 
> standard library with simplejson, which is obviously wrong.
> 
> Alex
> 
> 
> On Thu, Apr 11, 2013 at 5:51 PM, Jeremy Dunck  wrote:
>> I've just seen a documented example of this breaking things in the wild.
>> 
>> https://github.com/simplejson/simplejson/issues/37
>> 
>> I think we broke backwards-compat here - django 1.5.1. plus sentry
>> 5.4.5 dies because django's own DjangoJSONEncoder depends on stdlib
>> json, but sentry (and lots of things) use django.utils.simplejson,
>> which uses simplejson if available.
>> 
>> Unfortunately, this is incompatible - we kept utils.simplejson as a
>> compat shim, but I think until it goes away, we should keep using it
>> internally as well.
>> 
>> Alternatives?
>> 
>> --
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-developers+unsubscr...@googlegroups.com.
>> To post to this group, send email to django-developers@googlegroups.com.
>> Visit this group at http://groups.google.com/group/django-developers?hl=en.
>> For more options, visit https://groups.google.com/groups/opt_out.
> 
> 
> 
> -- 
> "I disapprove of what you say, but I will defend to the death your right to 
> say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
> "The people's good is the highest law." -- Cicero
> GPG Key fingerprint: 125F 5C67 DFE9 4084
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers?hl=en.
> For more options, visit https://groups.google.com/groups/opt_out.
>  
>  

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

django.utils.simplejson + stdlib json

[Jeremy Dunck]

I've just seen a documented example of this breaking things in the wild.

https://github.com/simplejson/simplejson/issues/37

I think we broke backwards-compat here - django 1.5.1. plus sentry
5.4.5 dies because django's own DjangoJSONEncoder depends on stdlib
json, but sentry (and lots of things) use django.utils.simplejson,
which uses simplejson if available.

Unfortunately, this is incompatible - we kept utils.simplejson as a
compat shim, but I think until it goes away, we should keep using it
internally as well.

Alternatives?

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: "Design Decision Needed" is gone. Good riddance.

[Jeremy Dunck]

"easy" is for people new to contributing django or to open source in
general.  Your view of "easy" may be different than our intended
meaning. :)

On Mon, Apr 8, 2013 at 2:58 PM, Lennart Regebro  wrote:
> On Mon, Apr 8, 2013 at 4:57 PM, Jacob Kaplan-Moss  wrote:
>> Thanks, please let me know if you have questions!
>
> Nope, but a suggestion: Mark new tickets as "easy pickings" by
> default. During the sprint I didn't find any ticket that wasn't either
> more than a year old (often indeed flagged as DDN) or easy. :-)
>
> //Lennart
>
> --
> You received this message because you are subscribed to the Google Groups 
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers?hl=en.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Django Core mentorship list

[Jeremy Dunck]

Ahem:
[1] https://groups.google.com/forum/?fromgroups#!forum/django-core-mentorship

:)


On Wed, Apr 3, 2013 at 11:44 AM, Jeremy Dunck  wrote:
> Hey all,
>   I've just created django-core-mentorship[1] with founding members
> including Carl Meyer, Jacob Kaplan-Moss, Simon Charette, and Russell
> Keith-Magee.
>
>   Modeled after pythonmentors.com, the intention is to help more
> people make the leap from using django to contributing to it.  It is a
> complement to django-developers, where most members are already quite
> experienced with django and development in general.
>
>   django-core-mentorship will consist just of people interested in
> learning to contribute, or members of django-core who are interested
> in helping those people succeed.  We'll have a Code along the lines of
> pythonmentors as well, but for now, I hope this email will suffice.
>
>   If you ever see conversation on this list where you feel a new
> contributor is being turned away (or turned off) by the existing
> django-developers discussion, please direct them to
> django-core-mentorship.
>
>   If you've considered contributing to Django but haven't felt
> welcome, this list is for you.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Django Core mentorship list

[Jeremy Dunck]

Hey all,
  I've just created django-core-mentorship[1] with founding members
including Carl Meyer, Jacob Kaplan-Moss, Simon Charette, and Russell
Keith-Magee.

  Modeled after pythonmentors.com, the intention is to help more
people make the leap from using django to contributing to it.  It is a
complement to django-developers, where most members are already quite
experienced with django and development in general.

  django-core-mentorship will consist just of people interested in
learning to contribute, or members of django-core who are interested
in helping those people succeed.  We'll have a Code along the lines of
pythonmentors as well, but for now, I hope this email will suffice.

  If you ever see conversation on this list where you feel a new
contributor is being turned away (or turned off) by the existing
django-developers discussion, please direct them to
django-core-mentorship.

  If you've considered contributing to Django but haven't felt
welcome, this list is for you.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: why does django.db.signals map to django.core.signals

[Jeremy Dunck]

I'm not sure what you mean by "maps to".

The basic signal framework is in django.dispatcher. If you're
implementing https://code.djangoproject.com/ticket/11398 then you
probably want to import: from django.dispatcher import Signal.

django.db imports django.core.signals because it is a *consumer* of
the core signals request_finished and request_started.

You probably want to add to django.db.models.signals rather than
django.db in any case.
https://github.com/django/django/blob/master/django/db/models/signals.py



On Wed, Mar 27, 2013 at 4:06 PM, Joseph Curtin  wrote:
> Hey all,
>
>I'm working on adding that pre_syncdb signal, I've come across a rother
> peculiar detail. When I import django.db.signals, it maps to
> django.core.signals. Am I missing something here? I know for example,
> django.conf generates the settings import. Can someone point me to tho logic
> of these path edits?
>
> Cheers,
> -Joseph Curtin
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers?hl=en.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Persistent connections, take 2

[Jeremy Dunck]

It sounds like we need a way to tell the worker that we are done sending 
requests to it so that the worker can do cleanup (of which db conn close is one 
task). This mirrors the previous request_finished "coupling" to 
requests_finished. 

(OS?) Signal? Sentinel queue/socket/named pipe + background thread?


On Mar 18, 2013, at 7:36 AM, Aymeric Augustin 
 wrote:

> On 28 févr. 2013, at 00:12, Aymeric Augustin 
>  wrote:
> 
>> I'm just wondering if 10 minutes is a good default value for CONN_MAX_AGE.
> 
> 
> Since I committed the patch, I discovered that persistent connections don't 
> interact well with runserver.
> 
> By default, the development server creates a new thread for each request it 
> handles. Not only does this negate the effect of persistent connections 
> (they're thread-local), but it also keeps connections open until they're 
> garbage collected, making it easy to exceed the maximum number of available 
> connections.
> 
> Florian independently discovered the same problem with gunicorn + gevent, 
> because the gevent worker runs each request in its own "thread" (greenlet).
> 
> 
> This raises the following questions:
> 
> 1) Do we want to enable persistent connections in production by default?
>a) yes
>b) yes, through a deprecation path
>c) no
> 
> 2) Assuming the answer to 1) is yes, can we fix runserver?
>a) close connections at the end of the dev server request handler
>=> creates tight coupling between the dev server and the ORM :(
>b) disable persistent connections when DEBUG  = True
>=> badly targeted: some people may use runserver with DEBUG = False, 
> or want persistent connections when DEBUG = True
>c) ???
> 
> 
> The lazy solution is to disable persistent connections by default, document 
> the problem with servers that run each request in its own thread, and 
> recommend to set `DATABASES['default']['CONN_MAX_AGE'] = 0 if settings.DEBUG 
> else 600`.
> 
> Besides, I've observed that performance is often less of a concern that 
> backwards-compatibility and ease of use; from this perspective, disabling 
> persistent connections would be appropriate.
> 
> 
> What do you think?
> 
> -- 
> Aymeric.
> 
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers?hl=en.
> For more options, visit https://groups.google.com/groups/opt_out.
> 
> 

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Switch to database-level autocommit

[Jeremy Dunck]

I, for one, would prefer that we not recommend TransactionMiddleware
so highly.  Now that I'm doing active development with Rails, it's
quite clear to me that a major difference in the ORM layers are which
DB they grew up with -- the django orm is tuned best to postgres,
while working passably on mysql, while the rails orm works well on
mysql (such as it can) and worse on postgres.

But in particular, when I had a site with moderate (median?) traffic
and slow-ish requests (200ms-2s), TransactionMiddleware was absolutely
a disaster on mysql because it handled locking very poorly and TM
makes transactions much longer than they typically need to be.
Postgres handles xact locking much better, so this pain is not felt.
So: I would prefer to make it clear that postgres is the best choice,
though mysql works well, but specifically warning against TM on mysql.


On Tue, Mar 5, 2013 at 6:03 AM, Chris Wilson  wrote:
> Hi Lennart,
>
>
> On Tue, 5 Mar 2013, Lennart Regebro wrote:
>
>> I do agree that 99.9% of the sites committing on the end of the request is
>> the right thing to do, and do think that this should be the default,
>> although I realize that debate is already lost.
>
>
> In a perfect academic world where there are never any errors, that approach
> is probably academically correct. In my opinion it makes correct error
> handling extremely difficult, because database errors (unique violations,
> concurrent modifications, etc) are thrown far away from the code that knows
> how to deal with them (the code that caused them in the first place).
>
> Cheers, Chris.
> --
> Aptivate | http://www.aptivate.org | Phone: +44 1223 967 838
> Future Business, Cam City FC, Milton Rd, Cambridge, CB4 1UY, UK
>
> Aptivate is a not-for-profit company registered in England and Wales
> with company number 04980791.
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers?hl=en.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Switch to database-level autocommit

[Jeremy Dunck]

I'm not sure what you're referring to here - integrity, uniqueness,
and locking are handled at the individual query level - transactions
just cause locks to be held (if needed) until commit or rollback.  Can
you give a concrete example of an exception being raised at commit
time?


On Tue, Mar 5, 2013 at 6:03 AM, Chris Wilson  wrote:
> Hi Lennart,
>
>
> On Tue, 5 Mar 2013, Lennart Regebro wrote:
>
>> I do agree that 99.9% of the sites committing on the end of the request is
>> the right thing to do, and do think that this should be the default,
>> although I realize that debate is already lost.
>
>
> In a perfect academic world where there are never any errors, that approach
> is probably academically correct. In my opinion it makes correct error
> handling extremely difficult, because database errors (unique violations,
> concurrent modifications, etc) are thrown far away from the code that knows
> how to deal with them (the code that caused them in the first place).
>
> Cheers, Chris.
> --
> Aptivate | http://www.aptivate.org | Phone: +44 1223 967 838
> Future Business, Cam City FC, Milton Rd, Cambridge, CB4 1UY, UK
>
> Aptivate is a not-for-profit company registered in England and Wales
> with company number 04980791.
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers?hl=en.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Switch to database-level autocommit

[Jeremy Dunck]

On Mon, Mar 4, 2013 at 1:34 AM, Aymeric Augustin
 wrote:
> On 4 mars 2013, at 01:07, Shai Berger  wrote:
...
>> The use of savepoints in Django apps so far has been very little, as far as 
>> I know. One point
>> I'm unsure of is the interaction of savepoints with cursors, since querysets 
>> are lazy; so the scenario
>> I'm worrirf about is, generally speaking,
>>
>> @atomic
>> def main():
>>   for obj in query_with_more_than_100_objects:
>>   try:
>>   handle(obj)
>>   except Bad:
>>   pass
>>
>> @atomic
>> def handle(obj):
>>   if good(obj):
>>   mark_good(obj)
>>   obj.save()
>>   else:
>>   raise Bad(obj)
>>
>> Will the next (database) fetch after an exception is raised get the right 
>> objects?
>>
>> My reading of the Postgresql documentation is that it will do the right 
>> thing, not so sure about
>> the other backends.
>
> Django currently doesn't support server side cursors — at least on 
> PostgreSQL, most likely on
> other databases too. Accessing the first object loads the entire queryset 
> instantly.

Actually, the queryset API tries pretty hard to avoid this:
https://github.com/django/django/blob/2cd0edaa477b327024e4007c8eaf46646dcd0f21/django/db/models/query.py#L954

It is true that by default, both psycopg and mysqldb do load the full
resultset despite using the use of .fetchmany rather than .fetchall
https://github.com/django/django/blob/0c82b1dfc48b4870e8fbcfb782ae02cdca821e1f/django/db/models/sql/compiler.py#L765
But this is not django's fault, and I have always wished more drivers
would handle streaming results better.

In any case, as far as I know, the above code (mixing 2 queries with
partial result returns on a single connection) has never worked
properly, and I think it will stay equally broken in the new world.  I
think this could be fairly easily (manually) tested by setting
GET_ITERATOR_CHUNK_SIZE  to an absurdly low size. Which is to say, +1
from me.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Full time job opportunity in SF Bay Area - Python/Django developer

[Jeremy Dunck]

Please do not send recruitment emails to this mailing list.  This is
your one warning.  Repeating will result in you being banned from the
list.

On Fri, Jan 25, 2013 at 12:47 PM,   wrote:
> We are looking for a Full stack engineer with strong python and django
> development skills. Our client is a startup company in San Mateo founded in
> 2009 by a group of MIT engineers. They are looking for talented developers
> to join their team and work on various projects of interest, from their
> unique user interfaces to their custom-designed hardware systems. If you
> want to learn more, please email me at ali...@sqasolution.com
>
> http://renttesters.com/careers/full-stack-software-engineer/
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to
> django-developers+unsubscr...@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers?hl=en.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Django Admin Revamp - Any updates?

[Jeremy Dunck]

For the record: It's bad timing for Idan.  He lives in Tel Aviv which
is currently receiving intermittent rocket attacks.  He may be a bit
slow to respond.  ;)

Let's wish him and his family safety and the luxury of worrying about
django's admin in good time.

On Thu, Nov 15, 2012 at 7:23 AM, Victor Hooi  wrote:
> Hi,
>
> I'm guessing there aren't any updates on this? Lol.
>
> Idan - you mentioned you'd like to get thoughts on what we hope to achieve
> in a new admin - basically, what is the purpose of Django's contrib.admin -
> is that right?
>
> Is there some place that people can brainstorm or contribute their thoughts
> on this? Should somebody make a wiki page for collecting all this?
>
> Cheers,
> Victor
>
>
> On Friday, 18 May 2012 20:18:59 UTC+10, patrickk wrote:
>>
>> I agree with Idan. We mainly did Grappelli because of the look & feel (and
>> then added some functionality like autocompletes).
>>
>> However, it just doesn´t make sense to simply "beautify" the existing
>> admin-interface. Rethinking the functionality, adding flexibility, being
>> able to customize ... these are all necessary steps IMHO, but I´m still
>> missing a clear approach/roadmap on how/when this should happen.
>>
>> I planned to give a talk on djangocon.eu about how to improve the
>> admin-interface. Unfortunately, I had to step back from that idea because of
>> some customer-related projects. Still, the thoughts are there and I´m happy
>> to discuss this issue anytime. This discussion could start with defining the
>> so-called "trusted editor" – what does he/she knows resp. needs to know when
>> dealing with the admin-interface? What are the consequences (e.g., does an
>> editor care about an app-list, does he even know what it is)? What
>> working-groups do we need (python, html/css, js, ...)? How can we publish
>> the process/discussion? And much more ... let´s start ... but how?
>>
>> best,
>> patrick
>>
>>
>> Am Montag, 30. April 2012 23:41:14 UTC+2 schrieb Idan Gazit:
>>>
>>>
>>>
>>> On Monday, April 30, 2012 at 10:16 AM, Brett H wrote:
>>>
>>> > Increasing the flexibility for development and integration is more
>>> > important than trying to 2nd guess where we are going to be in 5 years
>>> > time.
>>>
>>> Fair enough, but that sort of flexibility is available now. Nothing is
>>> preventing you from starting your 3rd-party admin app today.
>>>
>>>
>>>
>>> The issue is Django's officially-blessed, officially-documented admin.
>>> I'm not sure it's better to have admin in contrib (or contrib at all, but
>>> that's a separate ball of wax). I have a feeling that this issue will
>>> probably be addressed once again now that we're on github.
>>>
>>> All the same, if there's going to be an official django admin, I'd like
>>> it to give some thought to the issues I've raised. I have no problem (read:
>>> would love) to draw upon existing solutions for an admin revamp, but right
>>> now I don't have a fitness function to guide my decisions, and I think that
>>> is necessary. Not a spec, just an attempt to step back and think about what
>>> the admin should do.
>>>
>>> -I
>>>
>>>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/django-developers/-/CTFFiNcb9KMJ.
>
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-developers?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Class based views: A standard hook for http-method-independent code

[Jeremy Dunck]

+1, this looks like a good change anyway and doesn't smell to me.

On Fri, Nov 16, 2012 at 6:09 AM, Daniel Sokolowski
 wrote:
> I like this approach.
>
> From: George Hickman
> Sent: Thursday, November 15, 2012 7:27 AM
> To: django-developers@googlegroups.com
> Subject: Re: Class based views: A standard hook for http-method-independent
> code
>
> I have a slightly different proposal, one where we can avoid the extra hook
> but hopefully cover everyone's use cases too.
>
> https://github.com/ghickman/django/commit/85ac39a481074c25af1ed72a7a12e62ff5425e54
>
> I've personally never liked the setting of args, kwargs & request from
> within dispatch since it seems like it's feature creep of the dispatch
> method. However I'm also in the same boat as many of the other posters here
> in needing to do permissions related checks before dispatch is called.
>
> With my suggestion above you would be able to put your pre-dispatch code in
> a subclasses overridden dispatch before calling super while also depending
> on args, kwargs & request on self.
>
> On Thursday, November 15, 2012 4:18:34 AM UTC, Aaron Merriam wrote:
>>
>> If the super call changes any data then by the time you've run whatever
>> code comes after the super call, the changes have already occured.
>>
>>
>> If you wait to call super before running your own code, then request,
>> args, and kwargs are not available on the request, so anything that depends
>> on them being there (such as self.get_object()) will not work, so it must be
>> re-implemented,
>> Otherwise you have to set request, args, kwargs yourself which does not
>> feel very DRY.
>>
>> For me, the entire reason I would like this change, is so that I can do
>> something before dispatch that uses self.request/args/kwargs.  Everything I
>> want can be accomplished within dispatch, but not as cleanly, or as DRY as
>> if this method hook existed.
>>
>> On Wednesday, November 14, 2012 6:49:06 AM UTC-7, Daniel Sokolowski wrote:
>>>
>>> Can you elaborate the nasty side effects you are thinking of? I can’t
>>> think of any that that the base views do to warrant your statement.
>>>
>>> From: Aaron Merriam
>>> Sent: Friday, November 09, 2012 3:12 PM
>>> To: django-d...@googlegroups.com
>>> Subject: Re: Class based views: A standard hook for
>>> http-method-independent code
>>>
>>> That pattern has nasty side-effects.  It can be used in some cases but it
>>> fails in most.
>>>
>>> On Friday, November 9, 2012 8:28:47 AM UTC-7, Daniel Sokolowski wrote:

 I’ve done the below in the past, the only issue with that is if you have
 side effects in parent’s dispatch you don’t want executed but you would 
 also
 run that risk if you had an initialize() method work flow; in the end I 
 find
 dispatch() is enough in my experience.

 def dispatch(self, request, *args, **kwargs):
 parent_dispatch_return = super(Class, self).dispatch(request, *args,
 **kwargs)
 ...my code based on values based on the super call...
 return parent_dispatch_return

 From: Jordan Hagan
 Sent: Friday, November 09, 2012 12:37 AM
 To: django-d...@googlegroups.com
 Subject: Re: Class based views: A standard hook for
 http-method-independent code

 Hey Russ,

 The main point of concern which I think you may be missing is that
 self.kwargs and self.args are set inside dispatch, so using other mixins
 that require self.kwargs and self.args to be set (most do) will not work,
 without doing:

 def dispatch(self, request, *args, **kwargs):
 self.args = args;
 self.kwargs = kwargs
 self.init()
 return super(Class, self).dispatch(request, *args, **kwargs)

 Which isn't very tidy, to me having self.args and self.kwargs be set
 twice (once in my overridden dispatch method, and once in the original
 dispatch) feels wrong. I can't give you a good reason for it, it just feels
 bad every time I do it. The only way to work around this is to override
 dispatch without calling the original, and essentially duplicate the
 original dispatch method with an init call added in.

 Cheers,
 Jordan

 On Fri, Nov 9, 2012 at 6:25 PM, Russell Keith-Magee
  wrote:
>
>
>
> On Fri, Nov 9, 2012 at 1:05 PM, Aaron Merriam 
> wrote:
>>
>> Without setting request, args, and kwargs on on the view instance
>> (which is done during the base dispatch view), anything in the view that
>> assumes these values are present cannot run.
>>
>> Most of my views end up with functions which retrieve an object and
>> then do some basic validation to ensure that a user has permissions, or 
>> that
>> the object is valid in some fashion, or that some set of conditions is 
>> met
>> prior to allowing any method call to happen.
>>
>> I have found that without this init method, the vast majority of my
>> views end up 

Call for use cases of metrics in django core

[Jeremy Dunck]

If you use/monitor/graph metrics (the idea, not Coda's library, but that
would be good, too), I'd like to hear from you.

What sort of metrics, under what implementation, would be useful to you
operationally?

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Feature request: collectstatic shouldn't recopy files that already exist in destination

[Jeremy Dunck]

Would it be reasonable to have a backend-specific hook to determine a 
fingerprint, where that could be mtime or md5 or whathaveyou as long as 
equality (or maybe ordering) works?



On Oct 8, 2012, at 10:23 AM, Alex Ogier  wrote:

> On Mon, Oct 8, 2012 at 1:06 PM, ptone  wrote:
> While git may be common, and your problem not unique - this is still a 
> condition of your dev environment rendering modification dates invalid. There 
> might be other situations where this is the case (I've run into scripts that 
> muck with modification dates based on camera/jpeg metadata).
> 
> So after some further discussion on IRC - it was determined that md5, while 
> somewhat common, was far from a standard, and was likely not to be available 
> as remote call for network based storage backends. And so the final 
> resolution is to wontfix the ticket.
> 
> In the end - this lack of a universal fingerprint is just a limitation of our 
> storage tools.
> 
> -Preston
> 
> Is there a reason this fingerprint must be universal? If you're dealing with 
> a backend like S3, where network latency and expensive writes are a problem, 
> but md5 is a builtin remote call (available on any GET), why not just do an 
> md5 sum in the _save() method? Basically, just buffer the File object you 
> receive, take an md5 in python, and then make a decision whether to upload or 
> not. In the common case of reading from local disk and writing to S3, this is 
> a big win, and doesn't require cooperation from any other backends, or 
> standardizing on md5 as a fingerprint method.
> 
> Best,
> Alex Ogier 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to 
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at 
> http://groups.google.com/group/django-developers?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

A bit of Django history

[Jeremy Dunck]

I was searching around for a different old blog post and found this one:
http://jacobian.org/writing/why-django/

It made me smile - we've come a ways since then. :)

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Results of testing votizen's py2.6/7 codebase against django master (319e1355190)

[Jeremy Dunck]

On Tue, Sep 18, 2012 at 11:23 AM, Jeremy Dunck  wrote:
...
> And the last one, I hesitate to raise because it's likely to be
> specific to my machine, but... our (django-nose-based) test runner
> hangs after completing the suite but before tearing down.  Using
> dtruss I can see it's hanging on select/kevent, and using gdb I see
> it's in _mysql_ConnectionObject_query.  I'm unsure where to get debug
> symbols for either mac python or mysqldb (or at least, I haven't spent
> the time to get there).  I mention it despite it seeming
> environment-specific because it's possibly hanging due to the
> unicode/py3 changes.  I just don't know.
>
> #0  0x7fff9867aaf2 in read ()
> #1  0x0001105fe8ba in vio_read ()
> #2  0x00011057fbb1 in my_real_read ()
> #3  0x00011057f778 in my_net_read ()
> #4  0x000110572cb2 in cli_safe_read ()
> #5  0x00011057705d in cli_read_query_result ()
> #6  0x00011057608e in mysql_real_query ()
> #7  0x000110563d14 in _mysql_ConnectionObject_query ()
> #8  0x00010f738d77 in PyEval_EvalFrameEx ()

Florian and Anssi helped troubleshoot this -
https://code.djangoproject.com/ticket/18984

I am using multi-db with some TEST_MIRROR'd aliases, each of which has
a pending transaction.  The flush at the end of TestCase locks forever
waiting for a table lock, essentially deadlocking.  Rolling back
pending transactions just before flush releases the related locks.

I consider this a release blocker because many multidb apps would
likely hit this.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Results of testing votizen's py2.6/7 codebase against django master (319e1355190)

[Jeremy Dunck]

In response to the call to test py2.x on django's current master, I
ran the test suite for Votizen.  Our codebase is currently running
with django 1.4.x.

Our codebase is ~100kloc, ~32kloc of which is tests.  We use abstract
model inheritance a good bit, but no concrete inheritance.  No i18n or
multi-tz.

In use:
  python2.6/2.7
  python-memcache
  tastypie
  MySQLdb
  django-celery/celery
  django-imagekit
  django-classy-tags

Results (so far):
  1) Our custom test runner broke: we used to force DB connection
feature detection (there was a bug in TEST_MIRROR'd feature detection)
so we could do some juggling; forced detection is now gone and is
implicit; this seems to remove the need to even do the underlying hack
for us.

  2) Revealed a DB alias which was missing a TEST_MIRROR declaration
for us.  It used to work, but we were wrong and just lucky.

  3) tastypie broken by a const moving in the django tree; fixed in
https://github.com/toastdriven/django-tastypie/commit/71a1f0a161a7f3d549bce9b07984212ee4a8c81a

  4) after upgrading tastypie master
(27d17b7db7afd6c81da24f64f5b4562b59134582), a bunch of failures like:
...
return self.dispatch('list', request, **kwargs)
  vi +176  upvote/api/tastypie.py  # dispatch
response = self._dispatch(request_type, request, **kwargs)
...
  vi +172  upvote/api/tastypie.py  # _dispatch
request, **kwargs)
  vi +474  
/Users/jdunck/.virtualenvs/votizen-1.5/lib/python2.7/site-packages/tastypie/resources.py
 # dispatch
response = method(request, **kwargs)
  vi +1130 
/Users/jdunck/.virtualenvs/votizen-1.5/lib/python2.7/site-packages/tastypie/resources.py
 # get_list
paginator = self._meta.paginator_class(request.GET,
sorted_objects, resource_uri=self.get_resource_uri(),
limit=self._meta.limit, max_limit=self._meta.max_limit,
collection_name=self._meta.collection_name)
TypeError: get_resource_uri() takes exactly 2 arguments (1 given)

-- it turns out our subclass .get_resource_uri *required*
bundle_or_obj, while tastypie internally does bundle_or_obj=None.  It
happened to work on 0.9.11; switching ours to also be optional fixed.

  5) We have some URLNode subclasses and as such had to have a
compatibility layer for future.url and furture.URLNode moving to
django.template.

  6) We had some test failures where
TransactionalTestCase.assertContains used to work (text was coerced by
smart_str prior to d1452f60974da6f0e54ff9ad7a03d2c115675d10).  While
we could argue that only text vars should be passed to
assert(Not)Contains, I think the removal of smart_str(text) here was
likely unintentional.  Post-py3ization, I think it should probably be
put back as force_text(text, response._charset).
I raised a ticket and made a related patch/pull req:
https://code.djangoproject.com/ticket/18980


I have a couple remaining issues which I've run out of time to work
through just now:

upvote.api.tests.test_tastypie:VoterResourceTestCase.test_X
ValueError: astimezone() cannot be applied to a naive datetime
(tastypie master + dj 1.5?)

upvote.candidates.tests.views.test_profile_home:CandidatesProfileTest.test_friends
EmptyPage: That page contains no results
(almost certainly an internal problem)

And the last one, I hesitate to raise because it's likely to be
specific to my machine, but... our (django-nose-based) test runner
hangs after completing the suite but before tearing down.  Using
dtruss I can see it's hanging on select/kevent, and using gdb I see
it's in _mysql_ConnectionObject_query.  I'm unsure where to get debug
symbols for either mac python or mysqldb (or at least, I haven't spent
the time to get there).  I mention it despite it seeming
environment-specific because it's possibly hanging due to the
unicode/py3 changes.  I just don't know.

#0  0x7fff9867aaf2 in read ()
#1  0x0001105fe8ba in vio_read ()
#2  0x00011057fbb1 in my_real_read ()
#3  0x00011057f778 in my_net_read ()
#4  0x000110572cb2 in cli_safe_read ()
#5  0x00011057705d in cli_read_query_result ()
#6  0x00011057608e in mysql_real_query ()
#7  0x000110563d14 in _mysql_ConnectionObject_query ()
#8  0x00010f738d77 in PyEval_EvalFrameEx ()

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Testing multidb with TEST_MIRROR

[Jeremy Dunck]

On Thu, Sep 6, 2012 at 12:16 PM, Anssi Kääriäinen
 wrote:
> On 3 syys, 07:40, Anssi Kääriäinen  wrote:
>> I would like to make the TransactionTestCase faster. Currently when
>> running Django's test suite, for every test ran you will truncate
>> around 1000 tables, then create around 4000 objects (permissions +
>> content types). Likely you will write to one or two tables in the
>> test, and then do the truncate/recreate dance again. There is room for
>> improvement. Track which tables have been modified and refresh only
>> those tables.
>
> I now have a pretty good WIP approach of tracking changes in testing.
> The changes can be found from here: [https://github.com/akaariai/
> django/tree/fast_tests_merged]. The approach relies on existing
> signals + a new "model_changed" signal, which is used to signal
> bulk_create and .update() changes, and could be used for raw SQL
> changes, too.
>
> The results are promising:
>   - PostgreSQL with selenium tests, master 33m4s, patched: 9m51s
>   - SQLite, no selenium tests, master: 7m35s, patched: 3m6s.

Well that is pretty damn awesome.  It also doesn't fix my problem.
Let's ignore my problem for now, though.  I'd like to look at the
diffs.  I doubt this is the real diff:
https://github.com/akaariai/django/compare/master...fast_tests_merged

What's the base branch for the fast_tests_merged comparison?

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Testing multidb with TEST_MIRROR

[Jeremy Dunck]

I have been working on a master/slave router library [1] and have run
into some trouble testing a client application of it.

The issue is that TestCase (as designed) holds test db writes in a
transaction, but the read slave connection (which is to the same DB
under TEST_MIRROR) does not have visibility to the changes (at least
in READ COMMITTED isolation).

For example, if I save a model instance to the master and then attempt
to read from the slave, it won't be there because the test transaction
is still pending.

Has anyone else had success testing master/slave with Django?  I think
a solution probably belongs in core.

Options I can see to address this:
   1) change TEST_MIRROR'd aliases to use the same underlying DB
connection while under test
   2) switch all tests to TransactionTestCase
   3) switch isolation visibility under test
   4) give up hitting replicas under test (that is, have an IS_TEST
branch in the router).

Of these ideas, the 1st is the most appealing to me - it'd mean some
jiggery to not repeatedly close/teardown the DB, but otherwise likely
would be transparent.

2) would have a very bad performance impact.
3) would potentially have bad interactions with some applications.
4) would be a doc change but we'd perhaps need a standard way to
indicate the test context - and also the icky feeling of branching
under test in the app code.

If you have other ideas on how to address the problem, I'm happy to hear them.

Would anyone be opposed to me making TEST_MIRROR aliases use the same
underlying conn?

[1] https://github.com/votizen/django-pindb

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: #16455 PostGIS 2.0 support

[Jeremy Dunck]

That sounds good to me, though I'm not a committer on the tree as far
as I know.   Anssi?


On Fri, Aug 10, 2012 at 8:12 AM, Flavio Curella
 wrote:
> Just to clarify, the plan is:
>
> 1) Merge in patch v8 (2d index for 2-dimensional fields, _nd index for 3D
> and up)
> 2) Open a new ticket for the new 3D operators.
>
> Let me know if that sounds good, or if you can think of any way of improving
> the patch.
>
> Thanks,
> Flavio.
>
> On Thursday, August 9, 2012 5:00:43 PM UTC-5, jdunck wrote:
>>
>> On Thu, Aug 9, 2012 at 2:42 PM, Flavio Curella 
>> wrote:
>> > From the benchmark, my understanding is that, on PostGIS 2.0:
>> >
>> > 1) for the 2D case, the best index is 'gist (columname)'
>> > 2) for the 3D case, the best index is 'gist (columname
>> > gist_geometry_ops_nd)' only when using specific 3D operators (ie: '&&&')
>> >
>> > I think we should create the index for 2D case, just as we did on
>> > PostGIS
>> > 1.5.
>> >
>> > Re: the 3D case, 2) brings the question: should we add these new
>> > operators?
>> > Is it out of scope for the ticket?
>>
>> I think we clearly should, but landing basic support now and adding
>> new operators later is also useful.
>>
>> Thanks for doing this work, by the way. :)
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/django-developers/-/yDzsjw77NzYJ.
>
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-developers?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: #16455 PostGIS 2.0 support

[Jeremy Dunck]

On Thu, Aug 9, 2012 at 2:42 PM, Flavio Curella  wrote:
> From the benchmark, my understanding is that, on PostGIS 2.0:
>
> 1) for the 2D case, the best index is 'gist (columname)'
> 2) for the 3D case, the best index is 'gist (columname
> gist_geometry_ops_nd)' only when using specific 3D operators (ie: '&&&')
>
> I think we should create the index for 2D case, just as we did on PostGIS
> 1.5.
>
> Re: the 3D case, 2) brings the question: should we add these new operators?
> Is it out of scope for the ticket?

I think we clearly should, but landing basic support now and adding
new operators later is also useful.

Thanks for doing this work, by the way. :)

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: #16455 PostGIS 2.0 support

[Jeremy Dunck]

I'm fairly familiar with the django gis code, but I haven't been
following postgis 2.0.  Could you point me towards some discussion of
the index changes from 1.5 to 2.0?  On the face of it, I'm doubtful
that they've removed a useful form of indexing with no
replacement/alternative.

It's hard for me to assess the patch w/o some more context re. the
upstream postgis changes.

On Wed, Aug 8, 2012 at 8:12 AM, Flavio Curella  wrote:
> Hi,
>
> I've made some work on https://code.djangoproject.com/ticket/16455 and I
> think I'm close to have it ready for checking.
>
> Can I have a quick review from the committers? I'm mostly interested in
> feedback about:
>
> * the v8 patch installs multidimensional index for columns with more than 2
> dimensions. From the PostGIS docs, seems like this kind of indexes are slow,
> and I can't fiind any advantages to use them. Patch v7 doesn't install the
> indexes. Any preference about which patch should we use?
> * testing the patch: do already existing tests cover? if not, where's the
> most appropriate place where to put the new tests?
> * documentation.
>
> Thanks,
> Flavio.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/django-developers/-/Bao8-qgH7fwJ.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-developers?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Upcoming sprint to update Django's tutorial

[Jeremy Dunck]

Hey all, just a quick reminder - we're running this docs sprint today,
starting now.

If you'd like to participant remotely, join #django-sprint.

If you are a core contrib and might -1 our changes, I'd love to hear
feedback. :)

I'll put up a page with some useful links soon, and it will be linked
from the Sprints page: https://code.djangoproject.com/wiki/Sprints

On Tue, Jul 17, 2012 at 1:27 PM, Marijonas Petrauskas  wrote:
> +1!
>
> Few other thoughts -
> It would also be nice to have best practices regarding version control
> system file layout (eg. local settings pattern). Besides, I think that
> tutorial (and overall Django) recommendation to use absolute paths for
> project-relative locations (templates, static dirs) in settings file is a
> bit sub-optimal, particularly when project is stored in a shared repository.
> Most of us end up using os.path.join(os.path.dirname(__file__... trick, so
> why not make it 'recommended'? Current 'official' approach is to make a copy
> of settings.py file for each installation. However, usually only a small
> part of the directives vary from installation to installation, and copying
> common directives is against DRY.
>
>
> Have fun,
> Marijonas
>
>
> On Tue, Jul 17, 2012 at 8:44 PM, Luke Granger-Brown 
> wrote:
>>
>> I think it would be good to make a new documentation page suggesting
>> various best practices - speaking from my point of view, it was hard for me
>> to figure out what I *should* be doing, because everyone was doing it
>> differently and each method had their own pros and cons. I ended up using a
>> mishmash and was in a nightmare situation where nothing actually worked.
>>
>> Obviously this page would need to be kept up to date - maybe even just
>> "some people said these things about Django best practices, read these
>> blogposts for information" would be fine - just some starting pointers. I
>> know especially on Windows its not that uncommon to entirely neglect
>> pip/virtualenv.
>>
>> On Jul 17, 2012 8:31 PM, "Alex Ogier"  wrote:
>>>
>>> On Tue, Jul 17, 2012 at 3:07 PM, Jeremy Dunck  wrote:
>>>>
>>>> I was wondering if people would be opposed to an opinionated tutorial?
>>>>  For example: you should use virtualenv and pip, south, should handle
>>>> requirements this way, should prefer factories over fixtures, should
>>>> have this project directory layout, etc.
>>>>
>>>> I could go either way - my preferred approach isn't right in all
>>>> cases, and it might seem a distraction to the absolute beginner or a
>>>> person who has their own opinions.
>>>
>>>
>>> I think we should shy away from teaching "best practices" when they are
>>> external to Django. Pointing people at other useful projects in an aside may
>>> be useful, but making pip, virtualenv and south part of the mainline
>>> tutorial is a bad idea for two reasons: (1) For people who are already
>>> versed in python and/or web development best practices, it takes away from
>>> what they want to learn: the core features of Django that differentiate it
>>> from other frameworks. (2) For people who are brand new to programming
>>> and/or python, it blurs boundaries and confuses them about what is really
>>> important. A new programmer has no way to distinguish between "manage.py
>>> startproject tutorial" and "pip install south". One is a core feature of
>>> Django development, the other is a third-party Python tool to download a
>>> third-party dependency.
>>>
>>> People can and do write blog posts all the time that go something like,
>>> "How to install Django on Ubuntu 12.04" that give a series of six commands
>>> to paste into a console. There's always the danger that they are incorrect
>>> or misguided, but on the whole they are more likely to be relevant for
>>> setting up a sane Django environment on some specific operating system than
>>> we can be in a general tutorial. They are also dated and appropriately
>>> transient: a blog post from 2008 can be forgiven for missing some latest
>>> best practices, whereas a tutorial enshrined in Django's official
>>> documentation cannot.
>>>
>>> Best,
>>> Alex Ogier
>>>
>>> --
>>> You received this message because you are subscribed to the Google Groups
>>> "Django developers" group.
>>> To post to this group, send email to django-developers@googlegroups.com.
>

Re: pre_init/post_init vs. performance

[Jeremy Dunck]

Can I get a review of the branch?  25% performance improvement of
Model.__init__ in stock django seems worth landing. :)


On Wed, Jul 18, 2012 at 3:48 AM, Jeremy Dunck  wrote:
> On Sun, Jul 15, 2012 at 11:07 AM, Jeremy Dunck  wrote:
>>
>> That is indeed what I meant, but I think Anssi's probably right that
>> this belongs in contribute_to_class instead of a new adhoc thing in
>> ModelBase.__new__.
>>
>> I'll take a swing at making this change.
>>
>> Opened related ticket: https://code.djangoproject.com/ticket/18627
>
>
> OK, I have a very rough cut w/ all tests passing here:
> https://github.com/jdunck/django/tree/18627_pre_init
>
> I poked in a flag, sys.JDUNCK_NEW, which can be turned on/off for old
> and new implementation - this is poked in at django.__init__
>
> To test performance, using tests/test_sqlite modified with:
>
> INSTALLED_APPS = [
> 'regressiontests.model_fields'
> ]
>
> and
> DATABASES['default']['name'] = ':memory:'
>
> Testing the pre_init hook:
>
> timeit.Timer("""
> for i in names:
> models.Person(name=i)
> """, """
> from django.core.management import call_command
> from regressiontests.model_fields import models
> call_command('syncdb')
> names = [str(i) for i in range(1000)]
> """).timeit(1000)
>
> Under the existing signal usage: 24.609718084335327
> Under the new approach avoiding signal usage: 18.074234008789062
>
> I'd expect post_init to be a smaller gain since the destination
> function has more overhead, but still a gain.
>
> Note that the pre_init and post_init signals are still fired -- it's
> just that django as-shipped won't use any listeners on them, so the
> Model.__init__ performance improves.
>
> In some cases, object construction currently is larger than query time
> - a simple 'select x from y limit 1000' would have similar overhead to
> this.  Also, note that this gain would be seen on all models in any
> project using GFK or ImageField but no other pre_init or post_init
> hooks - not just those models which have the fields.  With this
> switch, only the models with the hooking fields have the related
> overhead.
>
> Feedback welcome.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: More efficient m2m assignment SQL

[Jeremy Dunck]

On Thu, Jul 26, 2012 at 11:26 PM, Anssi Kääriäinen
 wrote:
> On 27 heinä, 08:15, Jeremy Dunck  wrote:
...
>> I would have expected something along the lines of :
>>
>> DELETE FROM `api_voter_districts` WHERE `voter_id` = 1
>>
>> But instead it's 2 queries:
>>
>> SELECT `api_voter_districts`.`id`, `api_voter_districts`.`voter_id`,
>> `api_voter_districts`.`district_id` FROM `api_voter_districts` WHERE
>> `api_voter_districts`.`voter_id` = 1
>>
>> DELETE FROM `api_voter_districts` WHERE `id` IN (2, 3, 4, 5, 6,...)
...
> To me it seems there is no reason to do two queries in delete if the
> following conditions hold:
>   - there are no signals sent for the deleted objects
>   - the delete does not cascade further (actually you don't need to
> collect even in this case - you can do the cascade query completely in
> the DB without fetching the IDs first).
>
> For automatic M2M tables the above conditions always hold, but it is
> not guaranteed if you are using the "through" argument.

Ah, yes, hmm.  Actually, it *is* possible to get signals attached to
the through.

from django.db.models import signals
signals.pre_delete.connect(handler, sender=Voter.districts.through)

So I don't think we can even take that shortcut on just the auto m2m
tables.  As the "signals maintainer", let me just say: damn, they
cause a lot of trouble. :-)

OK, it would be easy enough to add a signal method to see if there are
signals attached for a given sender (model in this case).

I'll work up a patch for the simplified case of auto-intermediates and
no receivers for pre/post delete and see where that gets us in
performance.

Do we already have machinery in place for "does this model have the
potential to cascade"?  I see Collector uses
._meta.get_all_related_objects, but that seems pretty heavy for the
simple need to branch depending on "yes or no".

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

More efficient m2m assignment SQL

[Jeremy Dunck]

I have 2 models:

class District(Model):
  pass

class Voter(Model):
   districts = ManyToManyField(District)

Sometimes I want to reset the m2m list:

voter.districts = other_districts

I noticed that this uses ManyRelatedManager._clear_items, which, among
other things, boils down to:

voter.districts.through._default_manager.filter(voter_id=voter.pk).delete()

I would have expected something along the lines of :

DELETE FROM `api_voter_districts` WHERE `voter_id` = 1

But instead it's 2 queries:

SELECT `api_voter_districts`.`id`, `api_voter_districts`.`voter_id`,
`api_voter_districts`.`district_id` FROM `api_voter_districts` WHERE
`api_voter_districts`.`voter_id` = 1

DELETE FROM `api_voter_districts` WHERE `id` IN (2, 3, 4, 5, 6,...)

Was it intentional to take 2 queries?  That is, is there a use case
where avoiding the single DELETE query is preferable?

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: pre_init/post_init vs. performance

[Jeremy Dunck]

On Sun, Jul 15, 2012 at 11:07 AM, Jeremy Dunck  wrote:
>
> That is indeed what I meant, but I think Anssi's probably right that
> this belongs in contribute_to_class instead of a new adhoc thing in
> ModelBase.__new__.
>
> I'll take a swing at making this change.
>
> Opened related ticket: https://code.djangoproject.com/ticket/18627


OK, I have a very rough cut w/ all tests passing here:
https://github.com/jdunck/django/tree/18627_pre_init

I poked in a flag, sys.JDUNCK_NEW, which can be turned on/off for old
and new implementation - this is poked in at django.__init__

To test performance, using tests/test_sqlite modified with:

INSTALLED_APPS = [
'regressiontests.model_fields'
]

and
DATABASES['default']['name'] = ':memory:'

Testing the pre_init hook:

timeit.Timer("""
for i in names:
models.Person(name=i)
""", """
from django.core.management import call_command
from regressiontests.model_fields import models
call_command('syncdb')
names = [str(i) for i in range(1000)]
""").timeit(1000)

Under the existing signal usage: 24.609718084335327
Under the new approach avoiding signal usage: 18.074234008789062

I'd expect post_init to be a smaller gain since the destination
function has more overhead, but still a gain.

Note that the pre_init and post_init signals are still fired -- it's
just that django as-shipped won't use any listeners on them, so the
Model.__init__ performance improves.

In some cases, object construction currently is larger than query time
- a simple 'select x from y limit 1000' would have similar overhead to
this.  Also, note that this gain would be seen on all models in any
project using GFK or ImageField but no other pre_init or post_init
hooks - not just those models which have the fields.  With this
switch, only the models with the hooking fields have the related
overhead.

Feedback welcome.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Git questions

[Jeremy Dunck]

On Fri, Jun 8, 2012 at 7:57 AM, Anssi Kääriäinen
 wrote:
> On 8 kesä, 17:28, Luke Plant  wrote:
>> First, thanks so much to Aymeric and Anssi and others for the
>> contribution guidelines, they're very helpful.
>>
>> I've got some questions that are due to my ignorance of git (I have
>> managed to avoid it as something I need in daily use, I still think it's
>> got a brain-damaged UI...)
>>
>> In this section:
>>
>> https://docs.djangoproject.com/en/dev/internals/contributing/committi...
>>
>> it's written you can use this:
>>
>>   git push --dry-run upstream master
>>
>> to check outgoing changes. However for me the output of that command is
>> a short and very unhelpful message, something like this:
>>
>> To g...@github.com:django/django.git
>>45d4331..2d5f9e4  master -> master
>
> The idea is you do next "git log" to see what are the actual changes.
>
>> The alternative for checking outgoing changes that I've found is using log:
>>
>>   git log -p upstream/master..master
>>
>> However, I've found this doesn't work as I expect sometimes, because
>> somehow after a pull, the branch pointer for 'remotes/upstream/master'
>> has not been updated to where I expect it to be (the last commit pulled
>> from upstream), but is left where it was. I've observed this several
>> times. If I do 'git fetch upstream', rather than 'git pull upstream
>> master', then the pointers always update, but I thought the whole point
>> of doing 'pull' was 'pull=fetch then merge'.  Am I doing something wrong?

Actually, I think what is going on here is that git-pull does a
fetch-and-merge of whatever the branch is set to track upstream.  This
may not be the branch you hope/expect it to be.

"
   When a local branch is started off a remote-tracking branch,
git sets up the branch so that git
   pull will appropriately merge from the remote-tracking branch.
This behavior may be changed via the
   global branch.autosetupmerge configuration flag. That setting
can be overridden by using the
   --track and --no-track options, and changed later using git
branch --set-upstream.
"

If you create a local branch from another local branch, I believe the
new branch gets the same upstream branch that the original local
branch had.

To see what branch your local is tracking (if any):
$ git branch --list -vv

For example, my output there includes:
  master  76d5daa [upstream/master] Changed `manage.py
shell`'s help text to reflect that it can invoke bpython.
  virtual_signals c8dc85d [votizen/virtual_signals: ahead 1] Clarify args

If you find your local is on the wrong branch, you can set it like so:

git branch --set-upstream  

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Django git guidelines

[Jeremy Dunck]

I noticed this hasn't made it to master yet.  Could it?  I'm running
sprints and there's a bit of confusion on how to contribute to git.

Cheers,
  Jeremy

On Thu, Jun 7, 2012 at 9:53 AM, Aymeric Augustin
 wrote:
> On 6 juin 2012, at 21:09, Anssi Kääriäinen wrote:
>> I am sure there is still a lot to polish. I might have tried to change
>> too big portion of the docs in one go. Still, I would like to commit
>> what I have tomorrow, so that the sprinters at djangocon have the
>> possibility to use the guidelines and begin the polishing work.
>> Alternatively, if it is possible to build the docs and make them
>> available somewhere for the sprinters, I could wait for reviews from
>> the sprinters, then commit the code early next week. This would make
>> for a really good review for the guidelines.
>>
>> The work is available from:
>> https://github.com/akaariai/django/tree/django_git_guidelines
>> or for compare view:
>> https://github.com/akaariai/django/compare/django_git_guidelines
>
> Hello Anssi,
>
> As discussed on IRC, I reviewed your patch, copy-edited it a bit slightly and 
> committed it.
>
> This can definitely still be polished, but we had to start somewhere, and 
> it's done. We'll adjust it as necessary.
>
> Many thanks for your work!
>
> --
> Aymeric.
>
> --
> You received this message because you are subscribed to the Google Groups 
> "Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to 
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at 
> http://groups.google.com/group/django-developers?hl=en.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Upcoming sprint to update Django's tutorial

[Jeremy Dunck]

Thanks for the pointers to existing tickets.

I was wondering if people would be opposed to an opinionated tutorial?
 For example: you should use virtualenv and pip, south, should handle
requirements this way, should prefer factories over fixtures, should
have this project directory layout, etc.

I could go either way - my preferred approach isn't right in all
cases, and it might seem a distraction to the absolute beginner or a
person who has their own opinions.


On Mon, Jul 16, 2012 at 11:15 PM, Aymeric Augustin
 wrote:
> 2012/7/16 Jeremy Dunck :
>> On August 4, PyLadies SF will collaborate with Django sprinters in San
>> Francisco to improve the Django tutorial.
>
> Hi Jeremy,
>
> This is an excellent idea! Some people have worked on new tutorials
> but the patches weren't reviewed and committed. Groups of sprinters
> have a much better chance of completing them.
>
> Specifically, there are two drafts for new tutorials (#16671:
> "reusable apps" and #16779: "contributing") in Trac. It'd be nice to
> complete, proof-read and commit them.
>
> The tickets related to the tutorials contain a few more ideas:
> https://code.djangoproject.com/query?status=!closed&component=Documentation&summary=~tuto
>
> Best regards,
>
> --
> Aymeric.
>
> --
> You received this message because you are subscribed to the Google Groups 
> "Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to 
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at 
> http://groups.google.com/group/django-developers?hl=en.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Upcoming sprint to update Django's tutorial

[Jeremy Dunck]

On August 4, PyLadies SF will collaborate with Django sprinters in San
Francisco to improve the Django tutorial.

I expect around 30 people will attend, including Alex Gaynor and Karen
Rustad ( 
http://pyvideo.org/video/713/improving-documentation-with-beginners-mind-o
).

We'll run a prep session on Aug 1 for people who are new to
contributing - the idea is to help get machines set up, give some feel
for the task, etc.

On Aug 4, I suspect the work will go as:

Identify the audience/reader archetypes
poke holes in the existing tutorial given the archetypes
group people around the holes they want to fill
collaborate on a new outline
split off for specific prose writing
then final edit pass (or 2) to fix editorial voice/congruence problems

If you would like more information or disagree with the general
direction, please speak up.

Cheers,
  Jeremy

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: pre_init/post_init vs. performance

[Jeremy Dunck]

On Sun, Jul 15, 2012 at 9:51 AM, Andy McCurdy  wrote:
>
> On Jul 14, 2012, at 6:37 PM, Russell Keith-Magee wrote:
>
>>
>> My only concern is:
>>
>>> We could store which fields have these hooks upon ModelBase.__new__
>>> construction and so skip most fields and overhead in __init__.
>>
>> I'm not sure if I'm misreading your intentions here, but just to be
>> sure -- I'd like to avoid explicitly naming certain field types as
>> "special" in ModelBase if at all possible. If we explicitly name
>> certain fields in ModelBase, then it means that third-parties with the
>> same requirement won't be able to exploit the same API entry point. In
>> the case of GenericForeignKey specifically, it also means that we
>> would be introducing a dependency (even if it was name-only) on
>> contrib into core -- which is something we've been trying to move away
>> from.
>
>
>
> I think Jeremy means doing something like this in ModelBase.__new__:
>
>
> meta.fields_needing_init = [field for field in fields if hasattr(field, 
> 'model_pre_init')]
>
>
> And in ModelBase.__init__:
>
>
> for field in self._meta.fields_needing_init:
> field.model_pre_init(*args, **kwargs)
>
>
> This way, in ModelBase.__init__, only the fields that actually need to be 
> initialized have this method called.
>

That is indeed what I meant, but I think Anssi's probably right that
this belongs in contribute_to_class instead of a new adhoc thing in
ModelBase.__new__.

I'll take a swing at making this change.

Opened related ticket: https://code.djangoproject.com/ticket/18627

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

pre_init/post_init vs. performance

[Jeremy Dunck]

I was poking around in our (Votizen's) use of signals and thinking
about making some tooling so that signal usage was a bit more
transparent.

In doing so, I noticed that GenericForeignKey hooks the model pre_init
signal.  It does that because GFK needs a chance to munge kwargs from
the GFK field name to the content_id and object_id fields.

Signal dispatch with zero receivers is much much faster than with 1 or
more receivers, and pre_init is fired quite a lot.  Pretty much every
decent sized project I've worked on has used GFK, so I imagine this is
a large sap on performance overall.

Similarly, I just noticed that ImageField does a similar thing,
setting dimension values in post_init.

I think we could get a significant performance win if, instead of
using pre_init/post_init here, we changed Model.__init__ to call a
per-field hook that could munge the Model.__init__ kwargs.

SomeField.model_pre_init(instance, args, kwargs)
  or maybe for symmetry with contribute_to_class:
SomeField.contribute_to_instance(instance, args, kwargs)

We could store which fields have these hooks upon ModelBase.__new__
construction and so skip most fields and overhead in __init__.

I'm happy to make a pull request but checking here to see if there's
any pushback.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: XSS and string interpolation

[Jeremy Dunck]

On Jun 28, 2012, at 6:57 AM, Luke Plant  wrote:

> Hi all,
> 
> 2) Any better name than 'html_fragment'?
> 

I like the general approach, but I miss the security-minded namse of "escape" 
and "mark safe".   Maybe "safe_html_fragment" or "make_safe_html_fragment"? 
Getting annoyingly long, I know. 

(Apologies if this feels bike shedding)

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Announcement: twice-monthly Django sprints in San Francisco

[Jeremy Dunck]

I've just opened signup for our 2nd sprint - Saturday, June 16th, from
10a to 6p:

http://sfdjangosprint2.eventbrite.com/

Please sign up if you plan to attend -- it's free of course.


On Tue, May 22, 2012 at 2:30 PM, Jeremy Dunck  wrote:
> My company, Votizen, now has an office in San Francisco, just a block
> from the 4th St Caltrain station.  We have room to host sprints -
> easily 25 people, though we can grow if there is demand.  We have a
> professional kitchen, great coffee gear, and maybe a kegerator soon.
> There are many great places to eat within an easy walk.
>
> We will begin hosting day-long, twice-monthly sprints for Django on
> the 1st and 3rd Saturdays of each month.  We hope that by making it a
> comfortable length and a regular occurrence, we can grow a good core
> of contribution.
>
> Note: this first sprint will be on Sunday, June 3rd due to a prior
> commitment - sorry about that. :-)
>
> Since we need to manage space and setup demands in order to make this
> a regular event, we'll use eventbrite for free tickets.
>
> If your company would like to sponsor food & beverage on a per-event
> basis, that would be appreciated.  Please contact me off-list for
> logistics.
>
> So, if you are in the bay area and would like to contribute,
> collaborate, mentor or be mentored, you are invited to attend - please
> sign up:
>  http://sfdjangosprint.eventbrite.com/

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

San Francisco sprint info

[Jeremy Dunck]

We (Votizen) hosted a 1-day sprint yesterday from 10a to 6p.  I had
capped attendance at 30 just for logistical reasons - there's room for
more but I wanted to gauge interest.

24 people signed up, and I'd guess that about 20 people total were
there for at least part of the day.  I think we peaked at about 16
people at once.  I think this is a really great turnout, but I look
forward to building it up.

Julien Phalip was available to take patches - thanks, Julien!

I worked on making signals work with class inheritance (a handler
connected to Some(Model) does not consistently receive calls for
SubclassOf(Some)).  This is an old problem made worse with the
addition of abstract and proxy models.  I've intended to work on it
for over a year, so it's great to finally get some focus on it.  The
branch so far has a proof of concept with tests passing - now we need
some discussion on backwards-compatibility issues and, if we can come
to some conclusion, I'll work on improving performance of the updated
implementation.
  https://github.com/votizen/django/tree/virtual_signals
  https://code.djangoproject.com/query?status=!closed&keywords=~signals

The attendees collectively worked on admin, documentation, signals,
mezzanine, and probably some other things I didn't see.  :)

As previously noted[1] we plan to host these sprints in San Francisco
twice per month - 1st and 3rd weekends of the month (mostly Saturdays)
- at the same location (Votizen, 292 Townsend) and time (10a - 6p).

We put up a job whiteboard which listed Votizen (contact me), Burning
Man (brianber...@burningman.com), and Priceonomics
(o...@priceonomics.com).

We covered drinks at a net cost of about $2 per attendee.  If you or
your organization would like to sponsor food or beverage, I'll make
sure you're publicly noted and thanked.

To all who came, thank you for contributing to the core and the
community.  To all who didn't, please consider joining us next time.

I'll be putting up a signup form for the next one shortly.

Cheers,
  Jeremy

[1]
https://www.djangoproject.com/weblog/2012/may/24/sf-sprints/

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: #18094: signals, model inheritance, and proxy models

[Jeremy Dunck]

On Fri, Apr 20, 2012 at 9:01 AM, Anssi Kääriäinen
 wrote:
> On Apr 12, 10:27 pm, Anssi Kääriäinen  wrote:
>> > So perhaps we do need the signal inheritance behavior to be opt-in when
>> > connecting the signal handler. I think I'd like to see a deprecation
>> > path so that eventually the inheritance behavior is the default, and you
>> > have to opt out of it, as I think in most cases it's the desired behavior.
>>
>> Unfortunately this seems to be the only backwards compatible way
>> forward. I don't know how to do the technical details... from
>> __future__ import pre_save? :)
>
> Could we force the caller to define the wanted signal inheritance mode
> when .connect() is called? The inherit mode must be one of True,False
> or None. Default of None means no inheritance (as now) but it will be
> deprecated.

I took a swing at a similar implementation here:
https://github.com/votizen/django/tree/fe0f92b0f50bcd47742638ff3b8ff94af83ff9c5
(Tracking branch: https://github.com/votizen/django/tree/virtual_signals )

I haven't tried to address performance yet, but I've read all of
https://code.djangoproject.com/ticket/16679 and think it's possible to
keep sane overhead.

> Another problem is that delete fires the same signal multiple times in
> the inheritance chain. We can't remove the parent signal for backwards
> compatibility reasons, nor can we fire them as then inheriting
> listeners will see the delete signals multiple times.

I think I solved this my the simple expedient of seen_receivers here:
https://github.com/votizen/django/blob/270666cc701dff1eb1a8e5e649634aee5980b737/django/dispatch/dispatcher.py#L249
Basically we can group up the receivers so that they are fired at most once.

...
> Another option is to continue as currently: model signals aren't
> handled coherently, and there is no signal inheritance. Django users
> have managed to survive thus far with the current signals
> implementation, so maybe it doesn't need fixing?

I'm less of a fan of signals than I used to be, but I think this line
of reasoning is wrong - mostly people don't notice when their signal
handlers are doing the wrong thing.  The lack of visibility makes it
difficult.  I have definitely seen bugs caused by this misbehavior.

(I'm not in favor of removing signals entirely, for what it's worth -
I still don't see a better way in some cases - but I do think more
visibility would be useful.)

> Any opinions on the above transition phase  idea? Other ideas? Is the
> "define signal inheritance on connect" the wanted behavior in 1.7 or
> should it be something else?

I think this is the right approach - but I'm not sure why you said
"1.7" here.  You mean 1.5 or done with the deprecation path by 1.7?

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Non-default managers in related field lookups

[Jeremy Dunck]

On Fri, Jun 1, 2012 at 2:26 AM, Russell Keith-Magee
 wrote:
> On Fri, Jun 1, 2012 at 8:53 AM, Jeremy Dunck  wrote:
...
>> Candidate.context('site') would return a manager depending on the
>> requested context.
>> (or perhaps Candidate.objects.using(context='site')... or
>> Candidate.objects.context('site'))
>>
>> # get an instance while assigning ._state.context == 'site', similar
>> to ._state.db
>> candidate_with_unpublished_races = Candidate.context('site')
...
> Is this really something that's going to be able to be managed at the
> routing level? It seems to me that the decision about exactly which
> set of objects you want returned needs much finer control that a
> router method will allow.
>
...
> Here's a counter-proposal:
>
...
>  * race.candidates.all() would use _default_manager
>
>  * race.candidates('objects_all').all() would use the objects_all manager
>
>  * race.candidates('objects').all() would also use _default_manager
> (but explicitly)


OK, the problem I'm trying to address is "stickiness" of purpose.
Unless we have well-known names, the admin app couldn't know to use
Main.related_name(context_name).all on its inline lookups.

The thought was that

class EventAdmin(Admin):
   def queryset(self):
  return self.model.objects('admin')

would result in an event instance whose person_set descriptor would
continue returning objects as useful for admin, without needing to
know to call event.person_set('admin') and so on.  I think well-known
names would lead to composability problems between apps.

I felt that putting it in the router nicely paralleled db_for_read in
the sense that the instance.save method writes to the ._state.db from
which the instance was received, or asks db_for_write if no such DB is
known.  This continues the idea of db aliases, where the alias name is
abstract and the behavior of that name is centralized into the router.

In your objection, yes, I'm saying that a race gotten through
Race.objects.context('admin') (or Race.object('admin') if you prefer
to spell it that way) would have a .candidates which returns for the
'admin' context, whatever that means, unless told otherwise.  The
implicit contract right now is that it returns 'default'.If you
wanted candidates outside of the 'admin' subset, you probably didn't
want to get a race through that context.

In any case, I understand your concern that the router approach might
not be flexible enough for general purposes, but I have a specific
purpose of needing "sticky" behavior in related lookups in a similar
manner to ._state.db does currently.

If you can see what I'm driving at, can you see a way to integrate
addressing the "sticky" need with your approach?

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Non-default managers in related field lookups

[Jeremy Dunck]

It feels to me that each place that ._default_manager is mentioned
here is a misfeature:
https://github.com/django/django/blob/2cd516002d43cdc09741618f0a0db047ee6d78fd/django/db/models/fields/related.py

As an example, we (Votizen) currently have some default managers which
show subsets of all objects (published, for example).

class Candidate(...):
   races = ManyToMany(Race, related_name='candidates')

   objects = CandidateWithPublishedRacesManager()
   objects_all = Manager()

class Race(...):
   objects = PublishedRaceManager()
   objects_all = Manager()

The intention here is that we show the blessed subset by default
(mostly for the site), show more for admin, shell, scripts, etc.

The trouble is this asymmetry:

candidate_with_unpublished_races = Candidate.objects_all.all()[0]

candidate_with_unpublished_races.races.all() !=
Race.objects.filter(candidates=candidate_with_unpublished_races)

This affects admin (because while we can control
CandidateAdmin.queryset, we can't control Candidate.races).  But it
also comes as a surprise fairly often in general.  Note that no error
is raised, you just process a subset of the data you meant to,
generally.

(We have considered instead adding manager methods for the site:
Candidate.objects.for_site()..., but you see this has the same problem
- assuming the default manager doesn't call that chained query
method.)


OK, for a solution, here's an API I think would remove the asymmetry:


Candidate.context('site') would return a manager depending on the
requested context.
(or perhaps Candidate.objects.using(context='site')... or
Candidate.objects.context('site'))

# get an instance while assigning ._state.context == 'site', similar
to ._state.db
candidate_with_unpublished_races = Candidate.context('site')

# the router grows a method to help decide what to do on a per-model basis.
class DatabaseRouter(object):
   ...# as now except, for example:
   def manager_for_context(self, context, model_class):
 # returns a manager
 # e.g.
 return getattr(model_class, "objects_%s" % context)
 # could also by dynamically constructed.

(It seems to me the router is the natural place to have
project-specific customization of what happens with managers, though
perhaps some community norms would be needed

And then perhaps models can have defaults, as they do DBs:

class FooModel(Model):
  ...
  class Meta:
  context = "site"

class FooAdmin(Admin):
  model = FooModel

  class Meta:
 context = "admin"

For backwards-compatibility, any model without a context specified in
meta gets "default", and the default database manager always resolves
to model_class._default_manager.

For app composition, there is still value in the concept of a default
manager, though I'm not entirely clear how this fits together.

Perhaps Frob.objects would resolve to some manager based on the default context,
unless .objects had been explicitly assigned to be a vanilla manager
(much as a queryset resolves the db via the router unless .using(db=)
has been used).

Note, I'm not entirely happy with this, but it gives something to start from.

Thoughts?

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Announcement: twice-monthly Django sprints in San Francisco

[Jeremy Dunck]

My company, Votizen, now has an office in San Francisco, just a block
from the 4th St Caltrain station.  We have room to host sprints -
easily 25 people, though we can grow if there is demand.  We have a
professional kitchen, great coffee gear, and maybe a kegerator soon.
There are many great places to eat within an easy walk.

We will begin hosting day-long, twice-monthly sprints for Django on
the 1st and 3rd Saturdays of each month.  We hope that by making it a
comfortable length and a regular occurrence, we can grow a good core
of contribution.

Note: this first sprint will be on Sunday, June 3rd due to a prior
commitment - sorry about that. :-)

Since we need to manage space and setup demands in order to make this
a regular event, we'll use eventbrite for free tickets.

If your company would like to sponsor food & beverage on a per-event
basis, that would be appreciated.  Please contact me off-list for
logistics.

So, if you are in the bay area and would like to contribute,
collaborate, mentor or be mentored, you are invited to attend - please
sign up:
  http://sfdjangosprint.eventbrite.com/

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Application init inconsistent

[Jeremy Dunck]

On Tue, May 8, 2012 at 9:33 AM, Alex Ogier  wrote:
> My guess is that Django is doing some normalization on the name you are
> importing. It does this to prevent double imports, for example importing
> 'projectname.appname' and 'appname' which would otherwise be considered
> separate modules even if they come from the same source.
>
> Just a theory, but it explains the behavior you are seeing if you are
> importing a different name when you do the native import.

This is correct, but it's generally a bad idea to have overlapping
import paths due to this double-import problem.

I wrote this wart remover some time ago - it might help spot the problem:
https://gist.github.com/857091

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: django db library doesn't handle quoted table/field names

[Jeremy Dunck]

On Thu, Apr 12, 2012 at 3:06 PM, Craig Lucas  wrote:
> i have developed a database for a client that uses camel casing in the
> database. we are now trying to get a models.py file to mimic the
> database structure and i have run into a few bugs with regards to
> using quoted table names.
>
> The first issue is when you specify a foreign key field it takes the
> table name + the field name to generate a unique constraint name. This
> logic does not take into consideration that the field name has quotes
> around it so it fails when executing the sql to create the foreign key
> constraint. This wouldnt be so much of a problem if you could specify
> the name of the foreign key (which I use through the related_name
> parameter) and the engine would actually use it as the constraint
> name.

Did the db_column attribute not work for this?  You shouldn't need to
do literal quotes in the value you pass to db_column - django
generally quotes field names in all cases.
https://docs.djangoproject.com/en/1.4/ref/models/fields/#db-column

> The next problem is somewhat similar. The django toolkit tries to
> automatically create indexes on foreign key fields but doesnt take
> into consideration that the table has quotes around it OR that the
> table is in a schema.
> Note that I override the db_table to force it to be in a schema.
>
> In the django code in creation.py:
>
> def get_index_sql(index_name, opclass=''):
>                return (style.SQL_KEYWORD('CREATE INDEX') + ' ' +
>
> style.SQL_TABLE(qn(truncate_name(index_name,self.connection.ops.max_name_length(.replace('"','').replace('.',
> '_') + ' ' +
>                        style.SQL_KEYWORD('ON') + ' ' +
>                        style.SQL_TABLE(qn(db_table)) + ' ' +
>                        "(%s%s)" % (style.SQL_FIELD(qn(f.column)),
> opclass) +
>                        "%s;" % tablespace_sql)
>
> I have added the ".replace('"','').replace('.', '_')" part to format
> the name properly to account for customized table names.

You're right that there isn't support for schema, though there is for
db_table, db_column and db_tablespace.  I think db_schema fits fine,
as long as the semantics make sense - I know schema is a common
concept, but do different backends have a similar notion?  Can foreign
keys to schemas be done, for example?

...
>   ApplicationKey = models.AutoField(primary_key=True, db_column =
> '"ApplicationKey"')

For what it's worth, I would have expected this to work:
ApplicationKey = models.AutoField(primary_key=True, db_column =
'ApplicationKey')

And to be more pythonic:
application_key = models.AutoField(primary_key=True, db_column='ApplicationKey')

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: #18094: signals, model inheritance, and proxy models

[Jeremy Dunck]

On Thu, Apr 12, 2012 at 9:31 AM, Carl Meyer  wrote:
> Hi all,
>
> There's a discussion ongoing on ticket #18094
> (https://code.djangoproject.com/ticket/18094) that has enough potential
> back-compat implications that it seems worth getting feedback here.

Small note, I'll try to respond to the whole thing later.  This ticket
exists on the topic, too:
https://code.djangoproject.com/ticket/9318

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Looking for some insight into a tiny piece of the django codebase

[Jeremy Dunck]

On Mon, Mar 19, 2012 at 11:17 PM, Tim Diggins  wrote:
> Hi -
>
> I'm wondering what the lines at the end of django/db/models/
> deletion.py do:
>
>        for model, instances in self.data.iteritems():
>            for instance in instances:
>                setattr(instance, model._meta.pk.attname, None)

A PK of None indicates to Django that the model hasn't been saved.  In
this case, it's possible that you'll be holding references to these
objects and will later call .save on them.  In that case, the save
should be an insert (with a new ID) rather than inserting the previous
ID.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Pretty Django model instaces updating

[Jeremy Dunck]

On Thu, Mar 1, 2012 at 9:28 PM, Carl Meyer  wrote:

> The "only_fields" kwarg suggestion comes from ticket 4102, and it is
> primarily intended as a performance optimization for large models by
> only sending the values of certain fields to the database at all,
> something that your proposed code does not do.

It is also useful in the case where you have fields on the model which
have authoritatively changed, but have other local values which might
be stale.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Mobile Login????

[Jeremy Dunck]

You've mailed the django-developers list which is for development *of
Django itself*.  The proper mailing list about *using* Django is
django-users.  Please send this message to django-users unless you're
proposing a change to Django.

http://groups.google.com/group/django-users



On Mon, Feb 27, 2012 at 11:24 AM, relihkcin  wrote:
> Hey, has anyone developed a mobile login? i'm having issues. When i
> load the login page into a  for my mobile app. It wont work. But
> if i redirect it to webbrowser. it works. I'm still a couple months
> new with django. Any help would be appriacted. thanks
>
> --
> You received this message because you are subscribed to the Google Groups 
> "Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to 
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at 
> http://groups.google.com/group/django-developers?hl=en.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Testing multidb without inconsistency

[Jeremy Dunck]

On Sat, Feb 25, 2012 at 2:13 AM, Anssi Kääriäinen
 wrote:
> On Feb 25, 9:24 am, Jeremy Dunck  wrote:
>> I have a master and a replica.  In test, the TEST_MIRROR on the
>> replica points to master.  I make writes to master, then read from
>> replica, then assert some numbers.
>>
...
> I think you really should use a transactional test case. The reason is
> that before you do a commit, your writes really should not be visible.
> Hiding this in testing could lead to hiding bugs.

Of course that is the case, but here the transaction is artificial -
part of the runner, not of the code under test.  All that using
TransactionTestCase does is side-step the pending transaction, not
model the code under test more accurately.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Testing multidb without inconsistency

[Jeremy Dunck]

I have a master and a replica.  In test, the TEST_MIRROR on the
replica points to master.  I make writes to master, then read from
replica, then assert some numbers.

def test_stuff(self):
Foo.objects.using('master').create()
number_received = do_expensive_computations()
# Down in some guts somewhere else, a query is called like so:
# Foo.objects.using('replica').count()

And that number bubbles up eventually
assertEqual(number_received, 1)

And this fails because the master has written, but it's in a pending
transaction (enforced by TestCase as a perf optimization).

I'm not trying to test the consistency here, I'm just trying to test
that that code over there is correctly aggregating previous data.

I understand the usefulness of the TestCase perf, and I could use
TransactionTestCase, but again, I'm not trying to test transactions.
This will affect a bunch of my code (i.e. many of my tests would have
to be TransactionTestCase).

I'd like the TEST_MIRROR'd replica to have visibility to master
writes.  Right now, a new connection is opened for each alias.  It
seems to me that it would be appropriate to share the connection on
TEST_MIRROR'd DBs.  I can see that this would cause trouble if you
*wanted* to test against the visibility window, though.

What do people think might be a reasonable solution?

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Caching back-refernces on one-to-one fields

[Jeremy Dunck]

On Thu, Jan 19, 2012 at 5:14 PM, Adrian Holovaty  wrote:
> On Wed, Jan 18, 2012 at 1:07 PM, Shai Berger  wrote:
>> I have a small improvement to suggest for one-to-one fields: Make them cache
>> back-references on related objects. That is, assume
>
> Yes! Good improvement. And we should do the same thing for one-to-many
> fields (ForeignKeys):
>
...
> john = Author.objects.get(name='john')
> books = list(john.book_set.all())
>
> # Does a database query, but should be smart enough to simply return john.
> books[0].author
> """
>
> I'm pretty sure there's a long-standing ticket for this, but I'm not
> sure which one it is. Shai, does your solution approach this in a way
> that can solve the issue for ForeignKeys as well?

This one seems much more likely to be a breaking change - it isn't
that unusual to do something like:

special_book = Book.objects.get(...)
special_book.fiddly_bit = True
# note no save

for author in Author.objects.all():
for book in author.books.all():
   if book.fiddly_bit:
   book.related_books.add(special_book)

The semantics of this change under a new caching layer.

If the intention is that it's really the same object (and not a new
instance of the same db object retrieved from a caching layer) then
this introduces actions at a distance.

This is ye olde Django ticket:
https://code.djangoproject.com/ticket/17

And for context: ;-)
http://blogs.tedneward.com/2006/06/26/The+Vietnam+Of+Computer+Science.aspx

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

First-run experience w/ psycopg2 and django 1.3.x

[Jeremy Dunck]

There's a known problem w/ latest pg (2.4.2) and we decided not to
backport the fix to 1.3.x because there's a workaround.

https://code.djangoproject.com/ticket/16250

All well and good, but someone coming to Django w/ postgres will have
a bad first-run experience because the docs don't mention this issue.

I think this should be mentioned here:
https://docs.djangoproject.com/en/1.3/topics/install/#database-installation
https://docs.djangoproject.com/en/1.3/ref/settings/#std:setting-DATABASE-ENGINE
https://docs.djangoproject.com/en/1.3/intro/tutorial01/

Where else might it be useful?

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Caching back-refernces on one-to-one fields

[Jeremy Dunck]

On Wed, Jan 18, 2012 at 11:07 AM, Shai Berger  wrote:
> Do you see a reason why I should not post a ticket?

+1

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Proposal: Logout user when they change their password.

[Jeremy Dunck]

On Sun, Jan 8, 2012 at 2:57 PM, Arnoud van Heuvelen
 wrote:
...
> 3) Save the password hash (or part of it) in the session and compare
> it against our data. If the hash is not the same, the user needs to be
> logged out. This wouldn't change the database, but the downside is
> that this causes overhead on every request.

+1.

1) We need a way to handle existing sessions when upgrading to the new
Django w/ this support.  I think the most natural support for the
validation is
auth.get_user:
https://code.djangoproject.com/browser/django/trunk/django/contrib/auth/__init__.py?rev=16539#L94
to check the hash before returning.

2) I don't think the user should be logged out of their own session
when changing the password, and so we'd need to update the active
session on its way back to the user.

This is messier - we'd need to detect when the password changed
(User.set_password called?) and update the related request session, if
any.  That may be best accomplished through a signal fired when the
password is changed and handled in session.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Sprint in San Francisco

[Jeremy Dunck]

Hello all,
  My apologies for the silence on the sprint plans - holidays and all that.

  I'm back to looking for space for the sprint - Bitbucket has only
recently moved into their new space and are still settling in.  Thanks
to them for the initial offer.

  In light of that, I'd like to push the dates to Jan 21-22 -
hopefully the last push back.

  If you have a location in the city, available Jan 21-22, able to
host 20-50 developers in support of Django's 1.4 release, please let
me know.  All that is needed is chairs, desks, and decent internet
connection for the guests.

  Thanks, and Happy New Year. :-)


On Fri, Dec 9, 2011 at 3:18 PM, Gabriel Hurley  wrote:
> I'll be there as well! Either set of dates works for me.
>
>     - Gabriel
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/django-developers/-/hCK2OwboH40J.
>
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-developers?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Django sprint

[Jeremy Dunck]

Hi Ryan,
  Actually, I'm still looking to firm up plans.  I should have sent
info earlier - I'm sorry about that.

  I'm now hoping to run the sprint Jan 21-22.  It may be at
Bitbucket's office, or it may not.  I'll follow up with more specifics
on this tomorrow if at all possible.


On Mon, Jan 2, 2012 at 8:51 PM, Ryan Hiebert  wrote:
> Have the details of the sprint at BitBucket / Atlassian been confirmed? If 
> so, I'm wanting to participate, and am looking for where to RSVP, etc.
>
> --
> You received this message because you are subscribed to the Google Groups 
> "Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to 
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at 
> http://groups.google.com/group/django-developers?hl=en.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: DoS using POST via hash algorithm collision

[Jeremy Dunck]

On Thu, Dec 29, 2011 at 12:10 PM, Paul McMillan  wrote:
...
>> That seems like a simpler workaround than arch upgrade or replacing
>> dict implementation.
>
> This problem has nothing to do with slowloris.
>
> Replacing dict implementation prevents an attacker from producing keys
> which are intentionally n^2 hard for dictionary operations.

Sure, I understand these are 2 different attack vectors.  I just meant
that putting a proxy in front is a general solution that isn't
invasive to app code.  It seems that this crafted-hash-collision
vector doesn't have a clean answer like that.  There are workarounds,
but they may not apply to particular codebases.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: DoS using POST via hash algorithm collision

[Jeremy Dunck]

On Thu, Dec 29, 2011 at 8:19 AM, Christophe Pettus  wrote:
...
> It's an interesting result, but I'm not sure how much to be worried about it 
> in the field.  A SlowLoris or similar attack would seem to be far more 
> effective and less implementation-dependent.

Slow Loris can be avoided by putting a proxy capable of buffering
requests until completion between the app server and the web, right?
That seems like a simpler workaround than arch upgrade or replacing
dict implementation.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: SOPA bill question

[Jeremy Dunck]

On Mon, Dec 26, 2011 at 4:33 PM, Etienne Robillard  wrote:
> What thus you Django developers all think of this proposed law and how could
> this may affect open source communities such as Django and Python
> to develop novel apps specifically for creating user-generated content?

This feels off-topic to me; the breadth of SOPA is far beyond UGC, far
beyond Django, and Django is useful for many kinds of sites that do
not involve UGC.

I, for one, am against it, but would prefer to discuss it in a proper forum.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Multi-TZ should be in release notes

[Jeremy Dunck]

I suppose I should read the whole doc and not just ctrl-F before reporting 
omissions.  :)



On Dec 25, 2011, at 10:03 PM, Karen Tracey  wrote:

> On Mon, Dec 26, 2011 at 12:58 AM, Jeremy Dunck  wrote:
> I think Multi-TZ should get soms love in the release notes.
> 
> https://code.djangoproject.com/changeset/17106
> 
> https://docs.djangoproject.com/en/dev/releases/1.4-alpha-1/
> 
> Was it an oversight?
> 
> 
> It's here:
> 
> https://docs.djangoproject.com/en/dev/releases/1.4-alpha-1/#support-for-time-zones
> 
> Karen
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to 
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at 
> http://groups.google.com/group/django-developers?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Multi-TZ should be in release notes

[Jeremy Dunck]

I think Multi-TZ should get soms love in the release notes.

https://code.djangoproject.com/changeset/17106

https://docs.djangoproject.com/en/dev/releases/1.4-alpha-1/

Was it an oversight?

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Python 3 port - all tests now pass on 2.5.4, 2.6.2, 2.7.2 and 3.2.2 with the same codebase

[Jeremy Dunck]

On Mon, Dec 5, 2011 at 7:28 AM, Carl Meyer  wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> On 12/05/2011 08:10 AM, Vinay Sajip wrote:
>> Not too bad for a first complete pass. Over two and a half hours to
>> run, and that' s running from /dev/shm (IIUC effectively RAMdisk) and
>> with fsync = off. All else set to default values. Time to investigate
>> Jeremy's link to Frank Wiles' post ...
>
> I think first investigating any difference in performance between trunk
> and your py3k branch makes more sense than heading straight to more
> Postgres performance tweaks.

I agree.

Carl, to be clear, if py3 pg is slower, but py2 pg is still in line w/
baseline, would that block merge to trunk?  It's quite possible the
problem lies in psycopg, but I think merging (with caveat emptor on
the py3/pg combination) would still be good.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Python 3 port - all tests now pass on 2.5.4, 2.6.2, 2.7.2 and 3.2.2 with the same codebase

[Jeremy Dunck]

On Sun, Dec 4, 2011 at 5:10 PM, Vinay Sajip  wrote:
>
> On Dec 5, 12:57 am, Anssi Kääriäinen  wrote:
>
>> There is one easy thing you can do for testing, set fsync to off in
>> postgresql.conf. The file is probably at /etc/postgresql/9.1/main/
>> postgresql.conf. Then restart the server and tests should be way
>> faster. Note that if your machine crashes, you will likely lose all
>> data. But as this is testing server that should not matter.
>
> I've already done the fsync = off, but it didn't make that much
> difference :-(

See also:
http://www.revsys.com/writings/postgresql-performance.html

Postgresql ships with some not-great default settings.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Sprint in San Francisco

[Jeremy Dunck]

On Sat, Dec 3, 2011 at 1:27 PM, Brodie Rao  wrote:
> Bitbucket/Atlassian would love to host a sprint. We're about to move
> from the Mission to our own office building in SoMa in a couple of
> weeks. We'll have plenty of space for anyone who wants to attend.

Thanks for the offer, Brodie.  I'll write offlist for some details
about the space/timing.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Sprint in San Francisco

[Jeremy Dunck]

On Sat, Dec 3, 2011 at 7:59 PM, Russell Keith-Magee
 wrote:
> On Saturday, December 3, 2011, Jeremy Dunck  wrote:
>> Hey all,
>>  With the 1.4 release coming up, I thought it'd be a good time to
>> schedule a sprint to get in any ponies or help shake out bugs.
>
> Sounds like a great idea to me!
>
> Since you're the DSF secretary, you already know that the DSF will help out
> any way we can, but for the benefit of anyone else contemplating helping out
> by organizing a sprint: If there is anything the DSF can do to make a sprint
> event better (eg paying for catering, helping with any equipment rental
> costs, or possibly even helping with travel costs to get a core developer in
> the room) don't hesitate to ask.

I believe Paul McMillan plans to attend.  I am honestly not sure what
other core are in the SF area or if one would like to visit.  Julien
Phalip will be arriving here in early January - he's said he might be
able to attend Jan 7-8 but would definitely attend Jan 14-15.  Since
we're still in the preliminary planning, I'm inclined to change it.

Votizen (my new employer) plans to pick up the tab for food & drink.
With Bitbucket's offer to provide hosting space, I think we're
starting to be in good shape.

Brodie, does it make any difference to move from Jan 7-8 to Jan 14-15
so that Julien can attend?

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Sprint in San Francisco

[Jeremy Dunck]

Hey all,
  With the 1.4 release coming up, I thought it'd be a good time to
schedule a sprint to get in any ponies or help shake out bugs.

  I'm in San Francisco these days, which I hear is an OK place to try
to host a sprint.  Given the holidays coming up shortly, I was
planning for just after that, probably January 7 - 8.  I don't see
major conferences scheduled around that - are there any specific
objections to those dates?

  Have we thought about timeline on feature freeze, etc?

  If anyone in the SF area would like to host the sprint, that would
be most welcome - I'm just getting started with the plan.  I'm
coordinating w/ David Cramer who orgs the SF meetup.

  I assume meeting in the city proper would be better for attendance,
but we could also probably go to Hacker Dojo in Mountain View.  Do
people feel strongly about this?

  Cheers,
Jeremy

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Improving test data experience

[Jeremy Dunck]

On Wed, Nov 30, 2011 at 2:26 PM, Adrian Holovaty  wrote:

> Just to be clear, how would you get the "diff" of what's changed?
> Would it automatically change the fixture files after you close the
> shell session? Or would that be up to you?

Yes, I was thinking of catching SystemExit, KeyboardInterrupt or maybe
using atexit and dumping at that time.
We're all using source control here, right? :-)

I guess another thing we could do is provide a function that could be
called to finalize the fixtures.

./manage.py testshell
>>> # hackety hack
>>> from django import finalize_fixtures
>>> finalize_fixtures()
^C

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Improving test data experience

[Jeremy Dunck]

On Wed, Nov 30, 2011 at 11:30 PM, Aymeric Augustin
 wrote:
...
> In my current job we're using https://github.com/rbarrois/factory_boy, we've 
> dropped all but the most trivial fixtures (sites, languages, an admin user), 
> and we haven't looked back.

Nifty, thanks.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Improving test data experience

[Jeremy Dunck]

Hi all,
  I've got a codebase with a fair bit of fixtures and have been
experiencing some pain around it.  One of the pains is migrating the
format of fixtures when a schema change occurs.

  I posted a thread to south-users to discuss the idea of adding a
feature to aid applying migrations to fixture data:
  
http://groups.google.com/group/south-users/browse_thread/thread/e358d13a55545082

  Evgeny responded that perhaps scripting generation of the fixture
was a good approach, and Carl responded that he thinks that generating
the needed test data in the TestCase itself is preferable.  Both
approaches avoid the immediate problem of wishing I could migrate
fixture data, because generating the data under their approaches
avoids having fixtures as an authoritative source of data.

  That surprised me a bit since I generally have viewed fixtures as
things to set up once.  Have others on the list tried these approaches
(or other approaches) and have any thoughts you'd like to share on it?

   In any case, I think it would be good to do one or both of the following:

   1) expand on the testing guide to present fixtures as one option
for test data and point out the options to script fixture generation
or avoiding fixtures in favor of TestCase-called generation,
discussing tradeoffs in the approaches
   2) add a django-admin testshell command which would act like
testserver - loading fixtures into a test DB and dropping you into the
shell to munge it as needed; this would be paired with a south feature
for applying migration fixtures.

  Feedback, please?

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Clearing cache between tests (#11505)

[Jeremy Dunck]

On Sun, Oct 30, 2011 at 4:19 PM, Jeremy Dunck  wrote:
> On Sun, Oct 30, 2011 at 8:48 AM, Jim Dalton  wrote:
> ...
>> Anyhow, I honestly agree with you and Aymeric that a simple cache.clear() is 
>> fine, but I thought the CLEAR_BETWEEN_TESTS flag was a good way to answer 
>> any objections that we should be cautious about clearing the cache. If there 
>> are no such objections, let's just do a simple cache.clear() then. It's 
>> literally a single line of code in the test runner, along with a test and a 
>> note in the docs. I can do that work as soon as some kind of consensus is 
>> reached.
>>
>> Thanks for your feedback.
>
> I locally have a test-runner class that calls .clear() on setUp.
> Works for me.  But we run CI, so test against prod.  I needed to point
> to different CACHES under test to avoid flushing prod cache 20 times
> per day.
>
> Another approach that can work is to have the test runner generate a
> unique key prefix for each test run.  Then the keys will all be misses
> on later runs.  If invading the available key length isn't acceptable,
> then the test runner could also have a custom KEY_FUNCTION to hash the
> resulting key.  A downside is that this can create significant
> pressure on the cache.  I think that is less bad than clearing cache,
> though, since that is effectively infinite cache pressure.
>

And now I'm reminded that we talked about all this before:
https://groups.google.com/forum/#!topic/django-developers/zlaPsP13dUY/discussion

Sorry for not really adding anything here. :-/

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Clearing cache between tests (#11505)

[Jeremy Dunck]

On Sun, Oct 30, 2011 at 8:48 AM, Jim Dalton  wrote:
...
> Anyhow, I honestly agree with you and Aymeric that a simple cache.clear() is 
> fine, but I thought the CLEAR_BETWEEN_TESTS flag was a good way to answer any 
> objections that we should be cautious about clearing the cache. If there are 
> no such objections, let's just do a simple cache.clear() then. It's literally 
> a single line of code in the test runner, along with a test and a note in the 
> docs. I can do that work as soon as some kind of consensus is reached.
>
> Thanks for your feedback.

I locally have a test-runner class that calls .clear() on setUp.
Works for me.  But we run CI, so test against prod.  I needed to point
to different CACHES under test to avoid flushing prod cache 20 times
per day.

Another approach that can work is to have the test runner generate a
unique key prefix for each test run.  Then the keys will all be misses
on later runs.  If invading the available key length isn't acceptable,
then the test runner could also have a custom KEY_FUNCTION to hash the
resulting key.  A downside is that this can create significant
pressure on the cache.  I think that is less bad than clearing cache,
though, since that is effectively infinite cache pressure.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Cleaning up manage.py and import paths

[Jeremy Dunck]

On Mon, Oct 10, 2011 at 3:05 PM, Carl Meyer  wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
...
> 2. People write code that imports things inconsistently (sometimes with
> the project prefix, sometimes without it), and then when they go to
> deploy (without manage.py or setup_environ), their imports break and
> they don't understand why. This comes up frequently in #django. In order
> to fix it they either have to make all their imports consistent, or add
> both the inner and outer directories to sys.path/PYTHONPATH in their
> deployment setup. Inconsistent imports should fail fast, not when you
> move to production.

If anyone is interested in this specific problem, I've written a
script to highlight places your sys.path allows aliasing of modules:
https://gist.github.com/857091

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: #17028 - diveintopython.org links now dead

[Jeremy Dunck]

How about:
http://web.archive.org/web/20110726001211/http://diveintopython.org/
?

On Wed, Oct 12, 2011 at 2:37 AM, rikuthero...@gmail.com
 wrote:
> Hello,
>
> As described on the ticket [1], it seems that diveintopython.org is dead, so
> I changed every occurence for  the mirror
> http://diveintopython.nfshost.com/, that it seems to be the most updated.
> The changes are on the patch attached in the ticket [2].
>
> Best regards
>
> [1] https://code.djangoproject.com/ticket/17028
> [2]
> https://code.djangoproject.com/attachment/ticket/17028/diveintopython_mirror_doc.patch
>
> --
> Pablo Recio Quijano
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-developers?hl=en.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Shouldn't views.generic.create_update.lookup_object use the default manager instead of .objects?

[Jeremy Dunck]

It is indeed a bug.

On Fri, Oct 7, 2011 at 10:19 AM, Guilherme Salgado
 wrote:
> Hi there,
>
> That function uses .objects on whatever model is passed to it by
> update_object() (or delete_object()), and I can't think of a reason to
> use that instead of the default manager (although my knowledge of
> Django internals is rather minimal), so I thought I'd ask here first
> before filing a bug.
>
> Cheers,
> Guilherme
>
> --
> You received this message because you are subscribed to the Google Groups 
> "Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to 
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at 
> http://groups.google.com/group/django-developers?hl=en.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: We're a group of students at UC Berkeley looking to contribute to Django

[Jeremy Dunck]

On Wed, Sep 28, 2011 at 3:36 AM, Paul McMillan  wrote:
> I'm Paul, one of the core devs for Django. I'm curious which class
> you're enrolled in and what your requirements are. I'm always excited
> to see new people looking to get involved in the project, and I happen
> to be VERY local.

Hi Paul & Berkeley people.  I'm newly in the SF area.

If you want to meet up for a sprint some time, that would be sweet.

Paul, are you active in any local Python/Django user groups?

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: confusing things in Trac UI (was: Design decision for #1625...)

[Jeremy Dunck]

On Tue, Sep 20, 2011 at 2:16 PM, Carl Meyer  wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> On Tuesday, September 20, 2011 1:26:22 PM UTC-6, jdunck wrote:
>> Well, I meant to mark accepted as endorsement of the patch, but that
>> made me owner.
>

>> I can't own it since I'm not a core committer.
>
> Hmm, I wasn't aware of this policy. We don't seem to make real
> consistent use of the "owned by" field, but as far as I was aware it was
> just a way to signal that you were working on this ticket so someone
> else with interest would check with you before diving in and doing a
> bunch of work. I thought anybody could make use of this field if they
> planned to push the ticket forward and wanted to signal that.

Well, I mostly meant that I think it's RFC (pending running the full
suite) and because of that, I shouldn't be the owner.  I think the OP
hoped I'd check it in, and I was trying to make my intentions clear in
the thread.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Design decision for #1625 - Adding traceback to return value from send_robust when error occurs

[Jeremy Dunck]

On Tue, Sep 20, 2011 at 12:09 PM, Jim D.  wrote:
> Awesome, great suggestions both. That's a cleaner API and the implementation
> itself is even slightly cleaner.
> I updated the patch and uploaded it to the ticket. If you or anyone else
> wants to review it and ideally move it forward, that would be excellent.
> Thanks!

Well, I meant to mark accepted as endorsement of the patch, but that
made me owner.

I can't own it since I'm not a core committer.

Setting it back to nobody set the status to "new".

Hopefully a core will agree this is a good approach.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Design decision for #1625 - Adding traceback to return value from send_robust when error occurs

[Jeremy Dunck]

On Tue, Sep 20, 2011 at 9:00 AM, Jim D.  wrote:
> https://code.djangoproject.com/ticket/16245

I've looked at the patch and it seems good to me.

I have a suggestion:

Rather than (receiver, err, traceback), why not (receiver, exc_info)
where exc_info is the triple returned by sys.exc_info()?  There's no
harm in changing the meaning of the 2nd tuple element if people are
opting into the change.

Similarly, rather than append_traceback, why not call it
exc_info=True?  This parallels the logging module's use of the kwarg:
http://docs.python.org/library/logging.html#logging.Logger.debug

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Semantics when calling select_related repeatedly

[Jeremy Dunck]

Currently, calling select_related causes the QS to forget previously
added fields.  Also, it seems that depth calls are not forgotten.

For example, cribbing from the tests here:
https://code.djangoproject.com/browser/django/trunk/tests/modeltests/select_related/tests.py#L129

If this:
 Species.objects.select_related('genus__family')
were replaced with
 Species.objects.select_related('genus__family').select_related('genus')
then the query count would go up.

That may be intuitive when written all at once, but consider when
'genus__family' is set by a default manager, and later some queryset
is constructed with select_related('genus').

Additionally, if it were:
  Species.objects.select_related(depth=1).select_related('genus__family')
I'm not sure what should happen.

It seems to me that calling .select_related should be additive, and if
you didn't want a previously select_related in effect, you should have
a way to reset it.  The .order_by() analog isn't available since
.select_related() means "select all related to max depth of 5".   So
we'd need a .select_related(depth=0) or similar.

So, each call would resolve to a list of fields.  depth=1 would be
shorthand for finding all the fields at depth  of 1 and adding it to
the set of previously .select_related fields.

Do people agree this is desirable?  I can see that this would be
backwards-incompatible since some people might be knowingly and
intentionally narrowing the previous calls with a shorter/shallower
call.  So then perhaps a "from __future__" sort of mechanism,
.select_related(additive=True) or
QuerySet(additive_select_related=True) or similar.

Or, if .select_related should not be additive, I'd like to see 1) docs
improved in this area and/or 2) a warning issued when a previous
.select_related was overridden.

Thoughts?

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Just curious, why is admin.site.register([Model], [ModelAdmin]) not done within ModelAdmin?

[Jeremy Dunck]

On Sun, Sep 11, 2011 at 11:56 AM, Joshua Russo  wrote:
> I've wondered this for a while, but is there a reason why the call to
> admin.site.register(([Model], [ModelAdmin]) is not done within the init of
> ModelAdmin?

You can run multiple AdminSite instances mounted at different URLs.
admin.site is just a default instance of AdminSite for ease of use.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Race condition in get_or_create

[Jeremy Dunck]

We've been running in prod without trouble under 'read committed' for
about a week, though not under heavy load -- it's a fairly new site.

I'm not sure how much assurance I can offer at higher load, sorry.

On Fri, Sep 2, 2011 at 11:42 AM, Ian Clelland  wrote:
>
>
> On Fri, Sep 2, 2011 at 11:22 AM, Jeremy Dunck  wrote:
>>
>> On Fri, Sep 2, 2011 at 11:04 AM, Ian Clelland  wrote:
>> > I'm seeing errors which I believe are due to a race condition in
>> > django.db.models.query.get_or_create, on a fairly high traffic site. Our
>> > production servers are running Django 1.2.5, but I don't see any changes
>> > in
>> > the code in trunk that would affect this. (I'm totally willing to
>> > construct
>> > a test case against trunk, but I'm posting this here in case it's
>> > already a
>> > recognized bug, or an error on my part)
>> > If two requests make the same call to get_or_create(), at roughly the
>> > same
>> > time, with a database server in REPEATABLE_READ isolation level, then I
>> > believe that it's possible for the following sequence of events to
>> > occur:
>>
>> I suspect you're using MySQL.  Am I right?  We just switched a mysql
>> site from repeatable read (the default) to read committed (postgres's
>> default, hence a better-tested path in django) for this very reason.
>
> We are. 5.0.51a, I believe.
> READ COMMITTED would definitely solve this -- that seems to be the main
> point behind #13906, but there seems to be some resistance there. Have you
> encountered any other issues from making that switch?
>
> Regards,
> Ian Clelland
> 
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-developers?hl=en.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Race condition in get_or_create

[Jeremy Dunck]

On Fri, Sep 2, 2011 at 11:04 AM, Ian Clelland  wrote:
> I'm seeing errors which I believe are due to a race condition in
> django.db.models.query.get_or_create, on a fairly high traffic site. Our
> production servers are running Django 1.2.5, but I don't see any changes in
> the code in trunk that would affect this. (I'm totally willing to construct
> a test case against trunk, but I'm posting this here in case it's already a
> recognized bug, or an error on my part)
> If two requests make the same call to get_or_create(), at roughly the same
> time, with a database server in REPEATABLE_READ isolation level, then I
> believe that it's possible for the following sequence of events to occur:

I suspect you're using MySQL.  Am I right?  We just switched a mysql
site from repeatable read (the default) to read committed (postgres's
default, hence a better-tested path in django) for this very reason.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.