Re: Ability to use method based routing

[Cal Leeming [Simplicity Media Ltd]]

Just had a look at django-nap, nice concept but it sadly seems to have
suffered the same over-engineering fate as the others (DRF, Piston,
Tastypie etc). Happy to discuss why in a separate thread, but probably
off topic for this one.

At present I'm just looking for proposals to introduce request
inspection support into the views, prior to them being inspected by
process_view.

Some options off the top of my head;

* Create a new method_dispatch() callback object, then check if your
view is an instance/base of that class and pass the request over
accordingly.
* Modify url() to support methods (but from what I can tell, this
would break a lot of functionality, such as url reversing).

This is, of course, assuming that a core developer agrees that this
functionality should be added at all. I'd be happy to give further
reasoning/clarification if no one sees the benefit of adding such a
feature.

Cal


On Fri, Jul 18, 2014 at 1:55 PM, Curtis Maloney
 wrote:
> Sounds like you're heading for a cleaner version of the Publisher pattern in
> django-nap...?
>
>
> On 18 July 2014 07:34, Cal Leeming [Simplicity Media Ltd]
>  wrote:
>>
>> Hello,
>>
>> Currently implementing a method dispatcher (e.g. a single URL that
>> goes to different views based on HTTP method) is not feasible because
>> it will break decorators.
>>
>> For example:
>> https://djangosnippets.org/snippets/2041/
>>
>> In theory you could use middleware to store the request object inside
>> the view callback using process_view, or use frame/locals inspection
>> on __getattr__ from your dispatcher, but these are all very nasty
>> hacks.
>>
>> It would be nice if Django either had a built in way to do method
>> based dispatching, or if base.py:get_response() was modified to pass
>> over the request object somehow to the callback (although that doesn't
>> feel particular clean either).
>>
>> The use case for this would be a REST API, in theory I could move the
>> routing functionality into a view and decorate those methods, but then
>> the decorators aren't really in the right place and routing logic is
>> moved away from urls.py, which also feels unclean.
>>
>> I'm not sure what my proposal would be at this stage, if anyone else
>> is interested lets get a dialog going,
>>
>> Cal
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Django developers" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to django-developers+unsubscr...@googlegroups.com.
>> To post to this group, send email to django-developers@googlegroups.com.
>> Visit this group at http://groups.google.com/group/django-developers.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/django-developers/CAHKQagE4RLvhLeJg%2BZPctd7FLv799mQnOgKFQEb2dTEfryheng%40mail.gmail.com.
>> For more options, visit https://groups.google.com/d/optout.
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/CAG_XiSDfZOXvSfODHOXS3bdRoZvNLTw3py-yZF%2Bf17dCM9ephw%40mail.gmail.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAHKQagHw%3Dyx5hhqUsNSRn%2BrW4sTeRLUiOYsUiyifx1wVeWOHWQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Ability to use method based routing

[Cal Leeming [Simplicity Media Ltd]]

Hello,

Currently implementing a method dispatcher (e.g. a single URL that
goes to different views based on HTTP method) is not feasible because
it will break decorators.

For example:
https://djangosnippets.org/snippets/2041/

In theory you could use middleware to store the request object inside
the view callback using process_view, or use frame/locals inspection
on __getattr__ from your dispatcher, but these are all very nasty
hacks.

It would be nice if Django either had a built in way to do method
based dispatching, or if base.py:get_response() was modified to pass
over the request object somehow to the callback (although that doesn't
feel particular clean either).

The use case for this would be a REST API, in theory I could move the
routing functionality into a view and decorate those methods, but then
the decorators aren't really in the right place and routing logic is
moved away from urls.py, which also feels unclean.

I'm not sure what my proposal would be at this stage, if anyone else
is interested lets get a dialog going,

Cal

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAHKQagE4RLvhLeJg%2BZPctd7FLv799mQnOgKFQEb2dTEfryheng%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Terms for database replication

[Cal Leeming [Simplicity Media Ltd]]

Yup, the BDFL is still strong in this one ;)

Cal


On Thu, Jun 5, 2014 at 6:50 PM, Alex Gaynor  wrote:

> Hi everybody.
>
> The Django core developers have made our decision on the terminology we're
> going to use; I'd ask that you stop using django-developers to debate this
> further.
>
> Alex
>
>
> On Thu, Jun 5, 2014 at 10:43 AM, Unai Zalakain 
> wrote:
>
>> Greetings!
>>
>>
>>  I saw that someone suggested "leader" and "follower" - I haven't thought
>>> through whether I find this more palatable.
>>>
>>
>> Well, as an individualist I am, I find those terms quite uninviting too.
>>  Hoping to downplay it a bit, what about BDSM terms "Dominant" and
>> "Submissive", "Dom" and "Sub" or "Top" and "Bottom"?
>>
>> --
>> unai
>>
>
>
>
> --
> "I disapprove of what you say, but I will defend to the death your right
> to say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
> "The people's good is the highest law." -- Cicero
> GPG Key fingerprint: 125F 5C67 DFE9 4084
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/CAFRnB2WQZt-u7wsU2KN5Oiq2hjWqXwmc80ME1wrHgLKN%3D1PL3Q%40mail.gmail.com
> 
> .
>
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAHKQagHG%3DAC-k2x9Eevg9yV8k8W%3Dmf9ScRZbgmDSnD8HmoU_bA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Terms for database replication

[Cal Leeming [Simplicity Media Ltd]]

For once, I'm going to +1 you Tom.

Cal


On Thu, Jun 5, 2014 at 2:52 PM, Tom Evans  wrote:

> Please revert this change as soon as possible.
>
> If the project has become so PC sensitive that the word "slave" is no
> longer permitted to be uttered, then "replica" is an alternate term,
> but "primary" is not.
>
> Have you ever set up "primary-primary replication"? No, neither have
> I. Master-master replication is common, please do not take it upon
> yourselves to re-program our vocabulary.
>
> I have a primary master, I do not have a primary primary!
>
> Cheers
>
> Tom
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/CAFHbX1%2BUa81AruHH7mRK9SyQCABQrprPDd7sdTRrNN0_DpNPOg%40mail.gmail.com
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAHKQagGa1WE8yH2xpf0DrKv9i_m9%2B6akcDTHexeUZmq%3DQZHUuA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: That's enough.

[Cal Leeming [Simplicity Media Ltd]]

I'd like to thank you both for acknowledging this post and offering to do
something about it.

I'll certainly follow up on this with a more detailed post about specifics,
I owe the community that much.

Cal


On Wed, May 28, 2014 at 3:34 AM, Russell Keith-Magee <
russ...@keith-magee.com> wrote:

> Hi Cal,
>
> I'd like to reiterate what Jacob has said - some change in communities is
> inevitable, but we'd aspire to those changes being positive for the most
> part. I'm saddened that your experience of those changes in the Django
> community hasn't been positive.
>
> I'd also like to make the same offer Jacob has made -- if you feel like
> going into specifics, either in public or in private, please get in touch.
> I'd be only too happy to discuss the problems as you see them, and if
> there's something we (i.e., the core team as stewards of the community) can
> do to improve things, I'd be very happy to explore those options with you.
>
> Yours,
> Russ Magee %-)
>
>
>
> On Wed, May 28, 2014 at 10:23 AM, Jacob Kaplan-Moss wrote:
>
>> I'm sorry you feel that way, Cal; your contributions have been
>> appreciated, and I've personally appreciated having you around. Thanks for
>> all you've done.
>>
>> If you ever feel up to sharing with me more specifics, so perhaps we can
>> try to change things to be more welcoming to contributions, well, you know
>> how to reach me.
>>
>> Good luck,
>>
>> Jacob
>>
>>
>> On Tue, May 27, 2014 at 7:49 PM, Cal Leeming [Simplicity Media Ltd] <
>> cal.leem...@simplicitymedialtd.co.uk> wrote:
>>
>>> I remember the first day that I discovered Django after moving from CI
>>> (PHP) back in 2009, it opened me up to a whole new world of design
>>> principles and ideas that I hadn't even considered before.
>>>
>>> The community was absolutely fantastic, I took huge pleasure in helping
>>> other people with their problems and got my first real taste for open
>>> source contribution.
>>>
>>> Over the years I perfected my techniques and design methodologies,
>>> sharing what I learnt as I went along whilst spreading the word of Django,
>>> having a really swell time.
>>>
>>> I've selflessly defended the core maintainers when criticized with lack
>>> of innovation or particular handling of a ticket, holding many of the core
>>> maintainers in the highest regard, despite their sometimes awkward bedside
>>> manner.
>>>
>>> Five years later, and I can barely recognise the project and community
>>> that I once loved. I've seen several events of genuine arguments being shot
>>> down with no solid logic, questionable design methodologies, lengthy
>>> debates taking place over subjects that shouldn't even be a problem to
>>> begin with, contradictory arguments made by several core maintainers, and a
>>> previously smooth contribution process reduced to a political nightmare.
>>>
>>> Should you care what I have to say? Not really, I'm not a core
>>> maintainer, just another programmer.
>>>
>>> Perhaps this post will encourage others to speak up, perhaps it will be
>>> met with hostility, perhaps many of you will disagree with me, or perhaps
>>> it's just me.
>>>
>>> Am I going to continue to use Django? Probably, until someone makes a
>>> better alternative.
>>>
>>> Cal
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Django developers" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to django-developers+unsubscr...@googlegroups.com.
>>> To post to this group, send email to django-developers@googlegroups.com.
>>> Visit this group at http://groups.google.com/group/django-developers.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/django-developers/CAHKQagGWFiik4aYXtYzRW%2BGOJNQX1rJnAPzo4O1vo3h4%3DHK_bw%40mail.gmail.com<https://groups.google.com/d/msgid/django-developers/CAHKQagGWFiik4aYXtYzRW%2BGOJNQX1rJnAPzo4O1vo3h4%3DHK_bw%40mail.gmail.com?utm_medium=email&utm_source=footer>
>>> .
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>>  --
>> You received this message because you are subscribed to the Google Groups
>> "Django developers" group.
>> To unsubscribe from this group and stop receiving emails from it, 

That's enough.

[Cal Leeming [Simplicity Media Ltd]]

I remember the first day that I discovered Django after moving from CI
(PHP) back in 2009, it opened me up to a whole new world of design
principles and ideas that I hadn't even considered before.

The community was absolutely fantastic, I took huge pleasure in helping
other people with their problems and got my first real taste for open
source contribution.

Over the years I perfected my techniques and design methodologies, sharing
what I learnt as I went along whilst spreading the word of Django, having a
really swell time.

I've selflessly defended the core maintainers when criticized with lack of
innovation or particular handling of a ticket, holding many of the core
maintainers in the highest regard, despite their sometimes awkward bedside
manner.

Five years later, and I can barely recognise the project and community that
I once loved. I've seen several events of genuine arguments being shot down
with no solid logic, questionable design methodologies, lengthy debates
taking place over subjects that shouldn't even be a problem to begin with,
contradictory arguments made by several core maintainers, and a previously
smooth contribution process reduced to a political nightmare.

Should you care what I have to say? Not really, I'm not a core maintainer,
just another programmer.

Perhaps this post will encourage others to speak up, perhaps it will be met
with hostility, perhaps many of you will disagree with me, or perhaps it's
just me.

Am I going to continue to use Django? Probably, until someone makes a
better alternative.

Cal

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAHKQagGWFiik4aYXtYzRW%2BGOJNQX1rJnAPzo4O1vo3h4%3DHK_bw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Great Wall of DEP

[Cal Leeming [Simplicity Media Ltd]]

I have to agree that the decentralized nature of the mailing list would
probably stand the test of time a bit better, plus the ability to in-line
reply makes larger discussions easier on the eyes.

In reference to OPs original comments;
Enhancement proposals are a great idea, but ultimately they will suffer
with the exact same problem as other PRs, waiting on core devs. Unless a
core dev has a particular interest in that specific EP, then you're pretty
much waiting on a core dev to have "spare time" where they haven't got
anything else they'd rather be doing, as well as not being paid to do it.
That's not a dig at the core devs in any way, it's just a well established
behaviour of open source contributors.

Other projects have got around this issue by getting sponsorship and having
an experienced full time person(s) keeping on top of things, leaving the
rest of the core team to focus on getting the code done. Another
alternative is to set a threshold on proposals, meaning that it has to be
either accepted or rejected after a certain number of days since the last
edit (by means of a vote count or key person decision).  Really it comes
down to two things.. keeping it simple enough for users to contribute
easily, whilst enforcing a certain level of quality assurance, which can
only be done by a trusted (and hopefully experienced) person. Personally I
don't think Django has found the sweet spot for this yet.

Cal


On Thu, May 8, 2014 at 1:49 PM, Łukasz Rekucki  wrote:

> Hi folks,
>
> Discussing DEPs on Github inside Pull Request comments seems like a
> really bad idea, because it excludes a whole bunch of people that are
> subscribed to this list (I always thought this was the sole purpose of
> this list).
>
> The best thing about PEPs is that they always get posted to python-dev
> in *full text* and the discussion happens there. The email thread then
> gets recorded inside the PEP so that 10 years later you can easily
> read the whole discussion. Github comments don't have this feature,
> really.
>
> Throwing in my 2 cents,
> Łukasz
>
>
> On 8 May 2014 14:21, Florian Apolloner  wrote:
> > Hi,
> >
> >
> > On Thursday, May 8, 2014 2:13:52 PM UTC+2, Carl wrote:
> >>
> >> Just noticed this message, and the DEP PRs are still open a week later.
> >>
> >>
> >> Can someone shuffle this along, please?
> >
> >
> > We are in the final stages of 1.7, I personally would rather focus on
> that
> > first.
> >
> > --
> > You received this message because you are subscribed to the Google Groups
> > "Django developers" group.
> > To unsubscribe from this group and stop receiving emails from it, send an
> > email to django-developers+unsubscr...@googlegroups.com.
> > To post to this group, send email to django-developers@googlegroups.com.
> > Visit this group at http://groups.google.com/group/django-developers.
> > To view this discussion on the web visit
> >
> https://groups.google.com/d/msgid/django-developers/1d33bd04-a509-4c34-8bfe-2e4df31a3add%40googlegroups.com
> .
> >
> > For more options, visit https://groups.google.com/d/optout.
>
>
>
> --
> Łukasz Rekucki
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/CAEZs-EL_x6PB9jYe_OWDfQUYvV%3DH5jA52EBCeFM50xcY%2BLRMTA%40mail.gmail.com
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAHKQagHFk-Pm5V5D%3D26n1HL-Ahq3zD%2BvCG2JVP1mfk2Qf41S5Q%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Docs relating to splitting test files

[Cal Leeming [Simplicity Media Ltd]]

Thanks for letting me know, my eyes probably glazed over that paragraph,
which is my own fault for not reading the docs more carefully XD

Sorry for wasting time

Cal


On Fri, May 2, 2014 at 9:52 PM, Chris Beaven  wrote:

> On Saturday, May 3, 2014 6:10:39 AM UTC+12, Cal Leeming [Simplicity Media
> Ltd] wrote:
>>
>> This approach seems to work fine, but doesn't appear to be mentioned
>> anywhere in the Django docs (or at least, I couldn't see it).
>>
>> Would this be a valid candidate for a docs patch?
>>
>
> Hi Cal,
>
> The 1.6 release notes document the change:
> https://docs.djangoproject.com/en/dev/releases/1.6/#discovery-of-tests-in-any-test-module
> and there's also the following paragraph about it here:
> https://docs.djangoproject.com/en/1.6/topics/testing/overview/#writing-tests
>
> When you run your tests, the default behavior of the test utility is to
>> find all the test cases (that is, subclasses of unittest.TestCase) in any
>> file whose name begins with test, automatically build a test suite out of
>> those test cases, and run that suite.
>
>
> We're assuming knowledge of Python's new standard discovery of tests, but
> I guess it wouldn't hurt to explain it a bit more explicitly...
>
>>  --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/0d0a6104-f4c7-4e6d-914a-dd0a6ba18c92%40googlegroups.com<https://groups.google.com/d/msgid/django-developers/0d0a6104-f4c7-4e6d-914a-dd0a6ba18c92%40googlegroups.com?utm_medium=email&utm_source=footer>
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAHKQagG8bcgigij%2Bduk5cDOje%2BiGHyPJ_pjhun4EDEA%3DZ2450w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Docs relating to splitting test files

[Cal Leeming [Simplicity Media Ltd]]

Hello,

Please see the following discussion;
http://stackoverflow.com/questions/5160688/organizing-django-unit-tests/20932450#20932450

This approach seems to work fine, but doesn't appear to be mentioned
anywhere in the Django docs (or at least, I couldn't see it).

Would this be a valid candidate for a docs patch?

Cal

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAHKQagHbSsG3_U7o89Wocc%3DsT-JNpSoBM4GXGvhijujGcvd3jQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Add support for get_or_none?

[Cal Leeming [Simplicity Media Ltd]]

Okay great, I'll send a docs patch in next week (assuming no one shows any
disagreement), and hopefully that'll be the end of it!

Thanks to all for your feedback.

Cal


On Fri, Mar 28, 2014 at 10:04 PM, Aymeric Augustin <
aymeric.augus...@polytechnique.org> wrote:

> On 28 mars 2014, at 22:28, Cal Leeming [Simplicity Media Ltd] <
> cal.leem...@simplicitymedialtd.co.uk> wrote:
>
> > So is that a "no" to the docs patch proposal?
>
> It isn't. Like I said, I really don't care. If someone wants to commit
> something, that's fine.
>
> --
> Aymeric.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/F8FA7A7F-2A4C-4745-9CD9-E8D329ED39F8%40polytechnique.org
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAHKQagHByFK69uJfPvCPpf87wwoVwykX-bL9nEA7KQgoC3m%3D6w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Add support for get_or_none?

[Cal Leeming [Simplicity Media Ltd]]

So is that a "no" to the docs patch proposal?

Cal


On Fri, Mar 28, 2014 at 8:52 PM, Aymeric Augustin <
aymeric.augus...@polytechnique.org> wrote:

> Hello,
>
> I'm not sure this thread is going anywhere and I don't care either way.
> If you've been reading up to this point...
>
> ... either you have too much spare time. Rather than rehashing this debate,
> may I suggest triaging some tickets? That would really help!
>
>
> https://docs.djangoproject.com/en/dev/internals/contributing/triaging-tickets/
>
> https://code.djangoproject.com/query?status=!closed&stage=Unreviewed&desc=1&order=changetime
>
> ... or you really want this feature badly. I'll just leave this here:
>
> from django.db.models.query import QuerySet
> from django.db.models.manager import Manager
>
> def queryset_get_or_none(self, *args, **kwargs):
> try:
> return self.get(*args, **kwargs)
> except self.model.DoesNotExist:
> pass
>
> QuerySet.get_or_none = queryset_get_or_none
>
> def manager_get_or_none(self, *args, **kwargs):
> return self.get_queryset().get_or_none(*args, **kwargs)
>
> Manager.get_or_none = manager_get_or_none
>
> (That kind of monkey-patching isn't the worst because even if
> Django grows a method with that name, it will behave the same.)
>
> --
> Aymeric.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/9AE24931-1E65-44C4-808E-AFD232B697C7%40polytechnique.org
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAHKQagHAWXK3MZh152SBywh8gY6wF7Cb4o230qZoavqdvOw%2BTQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Add support for get_or_none?

[Cal Leeming [Simplicity Media Ltd]]

On Thu, Mar 27, 2014 at 1:10 AM, Russell Keith-Magee <
russ...@keith-magee.com> wrote:

>
> On Wed, Mar 26, 2014 at 8:19 PM, Cal Leeming [Simplicity Media Ltd] <
> cal.leem...@simplicitymedialtd.co.uk> wrote:
>
>>
>>
>>
>> On Tue, Mar 25, 2014 at 11:40 PM, Russell Keith-Magee <
>> russ...@keith-magee.com> wrote:
>>
>>>
>>> On Thu, Mar 20, 2014 at 9:31 PM, Florian Apolloner <
>>> f.apollo...@gmail.com> wrote:
>>>
>>>> On Thursday, March 20, 2014 2:01:25 PM UTC+1, Cal Leeming [Simplicity
>>>> Media Ltd] wrote:
>>>>>
>>>>> I'll give it a couple more days for a BDFL to gives the thumbs up/down.
>>>>>
>>>>
>>>> Well, we don't have BDFLs anymore and Shai already said he is -0 on it,
>>>> count me in with a relatively strong -0 too. I'd be a bit more open to it
>>>> if you could a better name. That said I generally agree with Shai about the
>>>> validation, eg this should be handled by the database constraints already
>>>> -- otherwise selecting via get doesn't make much sense imo.
>>>>
>>>
>>> Count me as a -0 as well. I simply don't see the problem with catching
>>> exceptions. Changing the name doesn't modify my objections.
>>>
>>
>> Then why did first() [1] get added? The only difference is that first()
>> adds a reference to [0], but it still saves the equal amount of code and is
>> still there for the same reasons of convenience, no?
>>
>
> first() was primarily added as an analog for latest().
>
> Personally, I see a significant difference between first()/latest() and
> get_or_none().
>

Personally, I would not class this as being significantly different enough
to justify get_or_none() being less worthy.


>
> first()/latest() is a specific, common query that has been optimized: list
> all, order by X, give me the first one.
>

> get_or_none() is a second version of an existing query: get(), but with a
> different return value. To me, this is duplication of an API, not a
> different query.
>

I do see where you are coming from with the "duplication of an existing
API" argument, though in my personal opinion I would argue it's a common
enough use case to justify inclusion.


>
> In my ideal world, the get(default=None) approach would be what we would
> do; but, as others have pointed out, default is a valid column name, so
> this option isn't available to us.
>

I would be a strong -1 on using any keyword argument on the existing .get()
to specify the default, whatever the name.


> We already have a shortcut for get_object_or_404; a matching
> get_object_or_none makes sense to me, and puts the API where it make sense
> to me - as a shortcut for someone who is repeating the "catch DoesNotExist"
> pattern regularly and wants an easier way.
>

For me, using shortcuts doesn't feel nice on the eyes at all, and I would
continue to use the mixin rather than shortcuts, though this is a
difference of opinion in taste.

Another alternative would be to add GetOrNoneMixin to the official docs,
similar to dictfetchall() [1].

If the argument of "duplication of an existing API" is going to be the nail
in the coffin, my next proposal would be for the aforementioned docs patch
instead.

[1]
https://docs.djangoproject.com/en/dev/topics/db/sql/#executing-custom-sql-directly


>
> Yours,
> Russ Magee %-)
>
>  --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/CAJxq84_281FRq4eJ6mxjBfWREi_50naTg%2B6mcvLBKSozeQwjYw%40mail.gmail.com<https://groups.google.com/d/msgid/django-developers/CAJxq84_281FRq4eJ6mxjBfWREi_50naTg%2B6mcvLBKSozeQwjYw%40mail.gmail.com?utm_medium=email&utm_source=footer>
> .
>
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAHKQagFEo%2Bn-rYdUzwwB5oyXjw04T6f3K4vWLDFysjB5P%3DaTpA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Add support for get_or_none?

[Cal Leeming [Simplicity Media Ltd]]

On Tue, Mar 25, 2014 at 11:40 PM, Russell Keith-Magee <
russ...@keith-magee.com> wrote:

>
> On Thu, Mar 20, 2014 at 9:31 PM, Florian Apolloner 
> wrote:
>
>> On Thursday, March 20, 2014 2:01:25 PM UTC+1, Cal Leeming [Simplicity
>> Media Ltd] wrote:
>>>
>>> I'll give it a couple more days for a BDFL to gives the thumbs up/down.
>>>
>>
>> Well, we don't have BDFLs anymore and Shai already said he is -0 on it,
>> count me in with a relatively strong -0 too. I'd be a bit more open to it
>> if you could a better name. That said I generally agree with Shai about the
>> validation, eg this should be handled by the database constraints already
>> -- otherwise selecting via get doesn't make much sense imo.
>>
>
> Count me as a -0 as well. I simply don't see the problem with catching
> exceptions. Changing the name doesn't modify my objections.
>

Then why did first() [1] get added? The only difference is that first()
adds a reference to [0], but it still saves the equal amount of code and is
still there for the same reasons of convenience, no?

[1]
https://docs.djangoproject.com/en/dev/ref/models/querysets/#django.db.models.query.QuerySet.first


>
> I'm probably a true neutral 0 on a django.shortcuts method mirroring
> get_object_or_404 (which, BTW, is what the way-back-in-2007 thread was
> proposing). I still don't see the point, but at least it's in a shortcuts
> module, and clearly labelled as such, not a method on a manager duplicating
> existing functionality.
>
> Yours,
> Russ Magee %-)
>
>  --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/CAJxq84_rYfpfSGicAKFDrLdtTKW9q1tr%2BM03xXx_qRCi4usTEw%40mail.gmail.com<https://groups.google.com/d/msgid/django-developers/CAJxq84_rYfpfSGicAKFDrLdtTKW9q1tr%2BM03xXx_qRCi4usTEw%40mail.gmail.com?utm_medium=email&utm_source=footer>
> .
>
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAHKQagHBH%2BCH2w%2BKtbbRPcjn6EW9HKnbEqDd5pvPf2pD-fPBiA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Add support for get_or_none?

[Cal Leeming [Simplicity Media Ltd]]

That's actually not a bad idea at all, much better than get_or_none().

Cal


On Tue, Mar 25, 2014 at 7:20 AM, Cheng Chi  wrote:

> What about name it fetch()?
>
>
> On Friday, March 14, 2014 3:45:31 AM UTC+11, Cal Leeming [Simplicity Media
> Ltd] wrote:
>>
>> Seems this issue was brought up several years ago, though the thread was
>> later hijacked for other functionality and get_or_none fizzled out.
>> https://groups.google.com/forum/#!topic/django-developers/Saa5nbzqQ2Q
>>
>> In Django 1.6 there were convenience methods added for .first(), for the
>> same principle of not having to catch an IndexError (or in this case, a
>> DoesNotExist error);
>> https://docs.djangoproject.com/en/dev/ref/models/
>> querysets/#django.db.models.query.QuerySet.first
>>
>> This seems to be wanted by several users, as seen here;
>> http://stackoverflow.com/questions/1512059/django-get-
>> an-object-form-the-db-or-none-if-nothing-matches
>>
>> Seems to be quite an easy fix, just needs a proper patch.
>>
>> Any thoughts?
>>
>> Cal
>>
>  --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/815dc10d-0e4d-4db9-b42a-e2adf9760786%40googlegroups.com<https://groups.google.com/d/msgid/django-developers/815dc10d-0e4d-4db9-b42a-e2adf9760786%40googlegroups.com?utm_medium=email&utm_source=footer>
> .
>
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAHKQagFSP9CCjT8yzexiHi3dKDuJL1jYCcYwk-eGPM2hCz2Q%2BQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Add support for get_or_none?

[Cal Leeming [Simplicity Media Ltd]]

I'll give it a couple more days for a BDFL to gives the thumbs up/down.

If no reply, then I'll make a patch and open a ticket.

Cal


On Mon, Mar 17, 2014 at 9:17 PM, Adam Mesha  wrote:

> And the documentation suggests that if you do have a field named defaults
> and you want to use it with get_or_create, you should use defaults__exact.
> I don't see why a similar solution wouldn't work for other queryset methods.
>
>
> 2014-03-17 11:20 GMT+02:00 Gwildor Sok :
>
> Actually, at the moment you can't have a column named "defaults" either if
>> you want to use your model with the current get_or_create function, so
>> naming a keyword argument like that is not that uncommon.
>>
>>
>> On Sunday, March 16, 2014 6:39:47 PM UTC+1, Cal Leeming [Simplicity Media
>> Ltd] wrote:
>>
>>> Still waiting for other people to chime in on this thread, but so far
>>> I'm not seeing any argument against the explained logic for having
>>> `.get_or_none()`.
>>>
>>> Comments appear to be more regarding how not to do it, rather than not
>>> doing it at all.
>>>
>>> Ultimately this either needs someone to find a flaw in my logic (have I
>>> missed anything?), or needs a BDFL decision.
>>>
>>> If all this needs to get approved is a patch, then I'll be happy to do
>>> so.
>>>
>>> Cal
>>>
>>>
>>>
>>>
>>> On Sat, Mar 15, 2014 at 5:06 PM, Shai Berger  wrote:
>>>
>>>> There is a family of names that would be valid -- names that cannot be
>>>> used to
>>>> name fields -- and that is names that begin with dunder.
>>>>
>>>> I would like to see neither get(__default=x) nor first(__only=True) --
>>>> I think
>>>> that's quite ugly -- I just want to remind us that technically, the
>>>> option
>>>> exists.
>>>>
>>>> On Friday 14 March 2014 12:40:25 Michael Manfre wrote:
>>>> > Good point. I forgot that some people would do that.
>>>> >
>>>> >
>>>> > On Fri, Mar 14, 2014 at 11:52 AM, Florian Apolloner
>>>> >
>>>> > wrote:
>>>>
>>>> > > On Friday, March 14, 2014 4:50:49 PM UTC+1, Michael Manfre wrote:
>>>> > >> On Fri, Mar 14, 2014 at 11:15 AM, Cal Leeming [Simplicity Media
>>>> Ltd] <
>>>> > >>
>>>> > >> cal.l...@simplicitymedialtd.co.uk> wrote:
>>>> > >>>> .get(or=None) (of some description) would be my preference, but
>>>> even
>>>> > >>>> that is ugly and confuses the existing API with "special"
>>>> keywords that
>>>> > >>>> aren't actually a filter.
>>>> > >>>
>>>> > >>> I would be strong -1 on having a special keyword.
>>>> > >>
>>>> > >> Even if the special keyword is 'default'? .get(..., default=None)
>>>> is a
>>>> > >> common python pattern that fits well with this usage.
>>>> > >
>>>> > > Yes, especially 'default' -- which is a perfectly valid name for a
>>>> table
>>>> > > column.
>>>> > >
>>>> > > --
>>>> > > You received this message because you are subscribed to the Google
>>>> Groups
>>>> > > "Django developers" group.
>>>> > > To unsubscribe from this group and stop receiving emails from it,
>>>> send an
>>>> > > email to django-develop...@googlegroups.com.
>>>> > > To post to this group, send email to django-d...@googlegroups.com.
>>>>
>>>> > > Visit this group at http://groups.google.com/
>>>> group/django-developers.
>>>> > > To view this discussion on the web visit
>>>> > > https://groups.google.com/d/msgid/django-developers/
>>>> 4aa7d1e3-4fb3-429e-a95
>>>> > > a-6e52cc9b511a%40googlegroups.com<https://groups.google.com/
>>>> d/msgid/django
>>>> > > -developers/4aa7d1e3-4fb3-429e-a95a-6e52cc9b511a%40googl
>>>> egroups.com?utm_me
>>>> > > dium=email&utm_source=footer> .
>>>> > >
>>>> > > For more options, visit https://groups.google.com/d/optout.
>>>>
>>>> --
>>>> You received this 

Re: Add support for get_or_none?

[Cal Leeming [Simplicity Media Ltd]]

Still waiting for other people to chime in on this thread, but so far I'm
not seeing any argument against the explained logic for having
`.get_or_none()`.

Comments appear to be more regarding how not to do it, rather than not
doing it at all.

Ultimately this either needs someone to find a flaw in my logic (have I
missed anything?), or needs a BDFL decision.

If all this needs to get approved is a patch, then I'll be happy to do so.

Cal




On Sat, Mar 15, 2014 at 5:06 PM, Shai Berger  wrote:

> There is a family of names that would be valid -- names that cannot be
> used to
> name fields -- and that is names that begin with dunder.
>
> I would like to see neither get(__default=x) nor first(__only=True) -- I
> think
> that's quite ugly -- I just want to remind us that technically, the option
> exists.
>
> On Friday 14 March 2014 12:40:25 Michael Manfre wrote:
> > Good point. I forgot that some people would do that.
> >
> >
> > On Fri, Mar 14, 2014 at 11:52 AM, Florian Apolloner
> >
> > wrote:
> > > On Friday, March 14, 2014 4:50:49 PM UTC+1, Michael Manfre wrote:
> > >> On Fri, Mar 14, 2014 at 11:15 AM, Cal Leeming [Simplicity Media Ltd] <
> > >>
> > >> cal.l...@simplicitymedialtd.co.uk> wrote:
> > >>>> .get(or=None) (of some description) would be my preference, but even
> > >>>> that is ugly and confuses the existing API with "special" keywords
> that
> > >>>> aren't actually a filter.
> > >>>
> > >>> I would be strong -1 on having a special keyword.
> > >>
> > >> Even if the special keyword is 'default'? .get(..., default=None) is a
> > >> common python pattern that fits well with this usage.
> > >
> > > Yes, especially 'default' -- which is a perfectly valid name for a
> table
> > > column.
> > >
> > > --
> > > You received this message because you are subscribed to the Google
> Groups
> > > "Django developers" group.
> > > To unsubscribe from this group and stop receiving emails from it, send
> an
> > > email to django-developers+unsubscr...@googlegroups.com.
> > > To post to this group, send email to
> django-developers@googlegroups.com.
> > > Visit this group at http://groups.google.com/group/django-developers.
> > > To view this discussion on the web visit
> > >
> https://groups.google.com/d/msgid/django-developers/4aa7d1e3-4fb3-429e-a95
> > > a-6e52cc9b511a%40googlegroups.com<
> https://groups.google.com/d/msgid/django
> > > -developers/4aa7d1e3-4fb3-429e-a95a-6e52cc9b511a%
> 40googlegroups.com?utm_me
> > > dium=email&utm_source=footer> .
> > >
> > > For more options, visit https://groups.google.com/d/optout.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/3039581.NUMVpPu4sL%40deblack
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAHKQagEX0RkofK_J_HyWgSyX8VE9OeaBcGT6cr%2B%3DsPiJJ5fciA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Add support for get_or_none?

[Cal Leeming [Simplicity Media Ltd]]

On Fri, Mar 14, 2014 at 3:08 PM, Josh Smeaton wrote:

> Shai has changed my mind. Most uses of the get or none pattern that I've
> used could be replaced by .first(), since it's unlikely there will be
> multiple objects with the kind of query you'd be using with a get.
>

The keyword here is `unlikely`. As mentioned in the other email, using
`first()` on the assumption that there is only one item, without doing an
assertion check, can lead to unexpected behaviour.

The use case for `first()` is not the same as the use case for
`get_or_none()`, but the principle of why they should be there is the same.


> I really dislike the get_or_create shortcut syntax, and I don't think a
> good name for get_or_none exists for the manager method.
>

I would agree the name `get_or_none` does feel a little odd, but still
struggling to think of an alternative.


>
> .get(or=None) (of some description) would be my preference, but even that
> is ugly and confuses the existing API with "special" keywords that aren't
> actually a filter.
>

I would be strong -1 on having a special keyword.


>
> So, I take back my +1.
>
> Josh
>
>
> On Saturday, 15 March 2014 02:01:52 UTC+11, Cal Leeming [Simplicity Media
> Ltd] wrote:
>
>> At present, I'd propose implementing it on the manager - the same as
>> `.get()`
>>
>> I would agree the naming convention does seem out of place, but at the
>> same time, no suitable alternative jumps to mind straight away (any
>> suggestions??)
>>
>> Cal
>>
>> On Thu, Mar 13, 2014 at 9:34 PM, Josh Smeaton wrote:
>>
>>> +1 on get_or_none. It seems to be a pattern that comes up quite a lot in
>>> user code, and I know I've had use for it lots of times. Cal, are you
>>> thinking of having a loose function get_or_none(qs, args, kwargs), or
>>> implementing it directly on the manager? I think it'd make sense to
>>> implement on the manager, but the name doesn't "fit" with the other methods
>>> available, so perhaps it'd be better to match it up with get_or_create as a
>>> simple shortcut.
>>>
>>> Josh
>>>
>>>
>>> On Friday, 14 March 2014 04:48:16 UTC+11, Cal Leeming [Simplicity Media
>>> Ltd] wrote:
>>>
>>>> Sorry yes, you're quite right.
>>>>
>>>> To be clear - the proposed solution in this thread is to make
>>>> `.get_or_none()` work exactly the same as `.get()`, with the only exception
>>>> that None is returned in place of DoesNotExist. All other logic (args,
>>>> kwargs, exceptions etc) stay exactly the same.
>>>>
>>>> Cal
>>>>
>>>>
>>>> On Thu, Mar 13, 2014 at 5:41 PM, Sean Bleier  wrote:
>>>>
>>>>> I like the implementation referenced in the SO thread, but I would
>>>>> just point out that `.get_or_none()` should accept both *args and 
>>>>> **kwargs.
>>>>> `Q` objects can be passed in as arguments to `.get()` and `.filter()`, so
>>>>> it's only natural to allow that for `.get_or_none()`.
>>>>>
>>>>>
>>>>> On Thu, Mar 13, 2014 at 10:26 AM, Cal Leeming [Simplicity Media Ltd] <
>>>>> cal.l...@simplicitymedialtd.co.uk> wrote:
>>>>>
>>>>>> Just read through all those threads/tickets, here's my conclusion.
>>>>>>
>>>>>> #2659 was rejected 8 years ago [1] on the basis that it's a "feature
>>>>>> creep", and that it "doesn't offer anything revolutionary". However the
>>>>>> same could be said for .first() and .last(), yet those were accepted.
>>>>>>
>>>>>> #11352 was rejected by luke plant 2 years ago [4] based on the
>>>>>> suggested implementation in that ticket, which is not the same
>>>>>> implementation as what I'm proposing this time round. The design of
>>>>>> `get_object_or_none` being added into shortcuts is not a good approach, 
>>>>>> and
>>>>>> was right to be rejected.
>>>>>>
>>>>>> #17546 was rejected 2 years ago [3] on the basis that #2659 and
>>>>>> #11352 were rejected, both of which I've addressed above.
>>>>>>
>>>>>> First argument - `first()` and `.last()` have been added, yet the
>>>>>> principle behind why they were added is the same as `.get_or_none()`.
>>>>>> Second arg

Re: Add support for get_or_none?

[Cal Leeming [Simplicity Media Ltd]]

On Fri, Mar 14, 2014 at 12:41 PM, Shai Berger  wrote:

> On Thursday 13 March 2014 14:34:18 Josh Smeaton wrote:
> > +1 on get_or_none. It seems to be a pattern that comes up quite a lot in
> > user code, and I know I've had use for it lots of times.
>
> Since 1.6, you should just be using first(). Compared to the
> try-get-except-DoesNotExist-return-None pattern, it is only missing the
> validation that there is, indeed, at most one objet matching the criteria.
> In
> a large majority of the cases I've seen, that validation is nice to have
> if it
> comes for free, but not worth a special effort, because it is taken care
> of by
> database constraints (in a significant part, the criteria just select by
> pk).
>

Picking the first item available (on the assumption that it is the only
item, without doing an assertion check) can lead to unexpected behaviour in
an application. On that basis, I would argue there is a valid use case
separate from `first()`


>
> I would be happy to have a validating first(),


Adding `MultipleObjectsReturned()` validation into `first()` probably isn't
going to fly for two reasons. The concept of first()/last() would break if
you could not have multiple objects, and changing would break backwards
compatibility significantly. As such I would be -1 for changing `first()`


> but the current proposition, as
> far as I understand, just overlaps existing API almost completely. I
> wouldn't
> veto it, but a strong -0 from me.
>
> Shai.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/1857272.IFIdcI8OYn%40deblack
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAHKQagG0gqOrvY%2BN_ZuU3KAC2LGZpnf3jXfR1G_Smh1BpCMXLw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Add support for get_or_none?

[Cal Leeming [Simplicity Media Ltd]]

At present, I'd propose implementing it on the manager - the same as
`.get()`

I would agree the naming convention does seem out of place, but at the same
time, no suitable alternative jumps to mind straight away (any
suggestions??)

Cal

On Thu, Mar 13, 2014 at 9:34 PM, Josh Smeaton wrote:

> +1 on get_or_none. It seems to be a pattern that comes up quite a lot in
> user code, and I know I've had use for it lots of times. Cal, are you
> thinking of having a loose function get_or_none(qs, args, kwargs), or
> implementing it directly on the manager? I think it'd make sense to
> implement on the manager, but the name doesn't "fit" with the other methods
> available, so perhaps it'd be better to match it up with get_or_create as a
> simple shortcut.
>
> Josh
>
>
> On Friday, 14 March 2014 04:48:16 UTC+11, Cal Leeming [Simplicity Media
> Ltd] wrote:
>
>> Sorry yes, you're quite right.
>>
>> To be clear - the proposed solution in this thread is to make
>> `.get_or_none()` work exactly the same as `.get()`, with the only exception
>> that None is returned in place of DoesNotExist. All other logic (args,
>> kwargs, exceptions etc) stay exactly the same.
>>
>> Cal
>>
>>
>> On Thu, Mar 13, 2014 at 5:41 PM, Sean Bleier  wrote:
>>
>>> I like the implementation referenced in the SO thread, but I would just
>>> point out that `.get_or_none()` should accept both *args and **kwargs. `Q`
>>> objects can be passed in as arguments to `.get()` and `.filter()`, so it's
>>> only natural to allow that for `.get_or_none()`.
>>>
>>>
>>> On Thu, Mar 13, 2014 at 10:26 AM, Cal Leeming [Simplicity Media Ltd] <
>>> cal.l...@simplicitymedialtd.co.uk> wrote:
>>>
>>>> Just read through all those threads/tickets, here's my conclusion.
>>>>
>>>> #2659 was rejected 8 years ago [1] on the basis that it's a "feature
>>>> creep", and that it "doesn't offer anything revolutionary". However the
>>>> same could be said for .first() and .last(), yet those were accepted.
>>>>
>>>> #11352 was rejected by luke plant 2 years ago [4] based on the
>>>> suggested implementation in that ticket, which is not the same
>>>> implementation as what I'm proposing this time round. The design of
>>>> `get_object_or_none` being added into shortcuts is not a good approach, and
>>>> was right to be rejected.
>>>>
>>>> #17546 was rejected 2 years ago [3] on the basis that #2659 and #11352
>>>> were rejected, both of which I've addressed above.
>>>>
>>>> First argument - `first()` and `.last()` have been added, yet the
>>>> principle behind why they were added is the same as `.get_or_none()`.
>>>> Second argument - The implementation being suggested in this thread is
>>>> not the same as what has been suggested in the three rejected tickets.
>>>> Third argument - Thread [2] had mostly positive feedback, but there was
>>>> no BDFL decision specifically on `get_or_none`.
>>>>
>>>> If I'm missing something here, please let me know.
>>>>
>>>> Cal
>>>>
>>>> [1] https://code.djangoproject.com/ticket/2659
>>>> [2] https://groups.google.com/forum/?fromgroups=#!searchin/
>>>> django-developers/get_default/django-developers/3RwDxWKPZ_A/
>>>> mPtAlQ2b0DwJ
>>>> [3] https://code.djangoproject.com/ticket/17546
>>>> [4] https://code.djangoproject.com/ticket/11352
>>>>
>>>>
>>>> On Thu, Mar 13, 2014 at 5:05 PM, Shai Berger wrote:
>>>>
>>>>> On Thursday 13 March 2014 18:45:31 Cal Leeming [Simplicity Media Ltd]
>>>>> wrote:
>>>>> > Seems this issue was brought up several years ago, though the thread
>>>>> was
>>>>> > later hijacked for other functionality and get_or_none fizzled out.
>>>>> > https://groups.google.com/forum/#!topic/django-
>>>>> developers/Saa5nbzqQ2Q
>>>>> >
>>>>> > In Django 1.6 there were convenience methods added for .first(), for
>>>>> the
>>>>> > same principle of not having to catch an IndexError (or in this
>>>>> case, a
>>>>> > DoesNotExist error);
>>>>> > https://docs.djangoproject.com/en/dev/ref/models/
>>>>> querysets/#django.db.model
>>>>> > s.query.QuerySet.first
>>>&g

Re: Add support for get_or_none?

[Cal Leeming [Simplicity Media Ltd]]

Sorry yes, you're quite right.

To be clear - the proposed solution in this thread is to make
`.get_or_none()` work exactly the same as `.get()`, with the only exception
that None is returned in place of DoesNotExist. All other logic (args,
kwargs, exceptions etc) stay exactly the same.

Cal


On Thu, Mar 13, 2014 at 5:41 PM, Sean Bleier  wrote:

> I like the implementation referenced in the SO thread, but I would just
> point out that `.get_or_none()` should accept both *args and **kwargs. `Q`
> objects can be passed in as arguments to `.get()` and `.filter()`, so it's
> only natural to allow that for `.get_or_none()`.
>
>
> On Thu, Mar 13, 2014 at 10:26 AM, Cal Leeming [Simplicity Media Ltd] <
> cal.leem...@simplicitymedialtd.co.uk> wrote:
>
>> Just read through all those threads/tickets, here's my conclusion.
>>
>> #2659 was rejected 8 years ago [1] on the basis that it's a "feature
>> creep", and that it "doesn't offer anything revolutionary". However the
>> same could be said for .first() and .last(), yet those were accepted.
>>
>> #11352 was rejected by luke plant 2 years ago [4] based on the suggested
>> implementation in that ticket, which is not the same implementation as what
>> I'm proposing this time round. The design of `get_object_or_none` being
>> added into shortcuts is not a good approach, and was right to be rejected.
>>
>> #17546 was rejected 2 years ago [3] on the basis that #2659 and #11352
>> were rejected, both of which I've addressed above.
>>
>> First argument - `first()` and `.last()` have been added, yet the
>> principle behind why they were added is the same as `.get_or_none()`.
>> Second argument - The implementation being suggested in this thread is
>> not the same as what has been suggested in the three rejected tickets.
>> Third argument - Thread [2] had mostly positive feedback, but there was
>> no BDFL decision specifically on `get_or_none`.
>>
>> If I'm missing something here, please let me know.
>>
>> Cal
>>
>> [1] https://code.djangoproject.com/ticket/2659
>> [2]
>> https://groups.google.com/forum/?fromgroups=#!searchin/django-developers/get_default/django-developers/3RwDxWKPZ_A/mPtAlQ2b0DwJ
>> [3] https://code.djangoproject.com/ticket/17546
>> [4] https://code.djangoproject.com/ticket/11352
>>
>>
>> On Thu, Mar 13, 2014 at 5:05 PM, Shai Berger  wrote:
>>
>>> On Thursday 13 March 2014 18:45:31 Cal Leeming [Simplicity Media Ltd]
>>> wrote:
>>> > Seems this issue was brought up several years ago, though the thread
>>> was
>>> > later hijacked for other functionality and get_or_none fizzled out.
>>> > https://groups.google.com/forum/#!topic/django-developers/Saa5nbzqQ2Q
>>> >
>>> > In Django 1.6 there were convenience methods added for .first(), for
>>> the
>>> > same principle of not having to catch an IndexError (or in this case, a
>>> > DoesNotExist error);
>>> >
>>> https://docs.djangoproject.com/en/dev/ref/models/querysets/#django.db.model
>>> > s.query.QuerySet.first
>>> >
>>> > This seems to be wanted by several users, as seen here;
>>> >
>>> http://stackoverflow.com/questions/1512059/django-get-an-object-form-the-db
>>> > -or-none-if-nothing-matches
>>> >
>>> > Seems to be quite an easy fix, just needs a proper patch.
>>> >
>>> > Any thoughts?
>>> >
>>> You linked the wrong thread.
>>>
>>>
>>> https://groups.google.com/forum/?fromgroups=#!searchin/django-developers/get_default/django-developers/3RwDxWKPZ_A/mPtAlQ2b0DwJ
>>>
>>>
>>> https://groups.google.com/forum/?fromgroups=#!searchin/django-developers/first%28%29/django-developers/iaOIvwzUhx4/x5wKtl7Bh2sJ
>>>
>>> I was (and still am) for a get_or_none() that raises an exception when
>>> it finds multiple objects, but we were overruled.
>>>
>>> Shai.
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Django developers" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to django-developers+unsubscr...@googlegroups.com.
>>> To post to this group, send email to django-developers@googlegroups.com.
>>> Visit this group at http://groups.google.com/group/django-developers.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/django-develop

Re: Add support for get_or_none?

[Cal Leeming [Simplicity Media Ltd]]

Just read through all those threads/tickets, here's my conclusion.

#2659 was rejected 8 years ago [1] on the basis that it's a "feature
creep", and that it "doesn't offer anything revolutionary". However the
same could be said for .first() and .last(), yet those were accepted.

#11352 was rejected by luke plant 2 years ago [4] based on the suggested
implementation in that ticket, which is not the same implementation as what
I'm proposing this time round. The design of `get_object_or_none` being
added into shortcuts is not a good approach, and was right to be rejected.

#17546 was rejected 2 years ago [3] on the basis that #2659 and #11352 were
rejected, both of which I've addressed above.

First argument - `first()` and `.last()` have been added, yet the principle
behind why they were added is the same as `.get_or_none()`.
Second argument - The implementation being suggested in this thread is not
the same as what has been suggested in the three rejected tickets.
Third argument - Thread [2] had mostly positive feedback, but there was no
BDFL decision specifically on `get_or_none`.

If I'm missing something here, please let me know.

Cal

[1] https://code.djangoproject.com/ticket/2659
[2]
https://groups.google.com/forum/?fromgroups=#!searchin/django-developers/get_default/django-developers/3RwDxWKPZ_A/mPtAlQ2b0DwJ
[3] https://code.djangoproject.com/ticket/17546
[4] https://code.djangoproject.com/ticket/11352


On Thu, Mar 13, 2014 at 5:05 PM, Shai Berger  wrote:

> On Thursday 13 March 2014 18:45:31 Cal Leeming [Simplicity Media Ltd]
> wrote:
> > Seems this issue was brought up several years ago, though the thread was
> > later hijacked for other functionality and get_or_none fizzled out.
> > https://groups.google.com/forum/#!topic/django-developers/Saa5nbzqQ2Q
> >
> > In Django 1.6 there were convenience methods added for .first(), for the
> > same principle of not having to catch an IndexError (or in this case, a
> > DoesNotExist error);
> >
> https://docs.djangoproject.com/en/dev/ref/models/querysets/#django.db.model
> > s.query.QuerySet.first
> >
> > This seems to be wanted by several users, as seen here;
> >
> http://stackoverflow.com/questions/1512059/django-get-an-object-form-the-db
> > -or-none-if-nothing-matches
> >
> > Seems to be quite an easy fix, just needs a proper patch.
> >
> > Any thoughts?
> >
> You linked the wrong thread.
>
>
> https://groups.google.com/forum/?fromgroups=#!searchin/django-developers/get_default/django-developers/3RwDxWKPZ_A/mPtAlQ2b0DwJ
>
>
> https://groups.google.com/forum/?fromgroups=#!searchin/django-developers/first%28%29/django-developers/iaOIvwzUhx4/x5wKtl7Bh2sJ
>
> I was (and still am) for a get_or_none() that raises an exception when
> it finds multiple objects, but we were overruled.
>
> Shai.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/201403131905.09028.shai%40platonix.com
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAHKQagFCyR2GGcY%2BV%2BGzdR%3DKi3P9%2BTVbT4BzVD_bDoJBN1w6Qw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Add support for get_or_none?

[Cal Leeming [Simplicity Media Ltd]]

I should just mention, the accepted answer in the SO thread is not the fix
I'm proposing.

I'm thinking something like;
http://stackoverflow.com/a/2021833/1267398

Cal


On Thu, Mar 13, 2014 at 4:45 PM, Cal Leeming [Simplicity Media Ltd] <
cal.leem...@simplicitymedialtd.co.uk> wrote:

> Seems this issue was brought up several years ago, though the thread was
> later hijacked for other functionality and get_or_none fizzled out.
> https://groups.google.com/forum/#!topic/django-developers/Saa5nbzqQ2Q
>
> In Django 1.6 there were convenience methods added for .first(), for the
> same principle of not having to catch an IndexError (or in this case, a
> DoesNotExist error);
>
> https://docs.djangoproject.com/en/dev/ref/models/querysets/#django.db.models.query.QuerySet.first
>
> This seems to be wanted by several users, as seen here;
>
> http://stackoverflow.com/questions/1512059/django-get-an-object-form-the-db-or-none-if-nothing-matches
>
> Seems to be quite an easy fix, just needs a proper patch.
>
> Any thoughts?
>
> Cal
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAHKQagED58k46A83FEee8eGOpX_nWD6PLHxCDvp-LWMrGG5jrA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Add support for get_or_none?

[Cal Leeming [Simplicity Media Ltd]]

Seems this issue was brought up several years ago, though the thread was
later hijacked for other functionality and get_or_none fizzled out.
https://groups.google.com/forum/#!topic/django-developers/Saa5nbzqQ2Q

In Django 1.6 there were convenience methods added for .first(), for the
same principle of not having to catch an IndexError (or in this case, a
DoesNotExist error);
https://docs.djangoproject.com/en/dev/ref/models/querysets/#django.db.models.query.QuerySet.first

This seems to be wanted by several users, as seen here;
http://stackoverflow.com/questions/1512059/django-get-an-object-form-the-db-or-none-if-nothing-matches

Seems to be quite an easy fix, just needs a proper patch.

Any thoughts?

Cal

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAHKQagEqGR%3DuPNhki1%3D%3DXjA7pT79ur-HJmGiaW%2B6ZdFdMXjt%3Dw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Backwards compatibility and field validation

[Cal Leeming [Simplicity Media Ltd]]

It seems there is a split reaction on whether or not models should assume
data is clean, and comes down to design decision.

I think that a docs patch explaining this entire concept to new users would
be sufficient for now (and I'm more than happy to spend time writing it).

Would any of the core devs object to such a patch?

Cal


On Fri, Oct 18, 2013 at 8:19 PM, Marc Tamlyn  wrote:

> I think Anssi summed up my feelings on this very well, in particular with
> the issues with update(). Similarly create() would also bypass as it does
> not call save() AFAIK.
>
> Given these commonly used approaches do not work, and that the name of the
> method - save() - implies only saving to the database - I'm -1 on save()
> calling clean().
>
> That said, there is an interesting option to explore in ModelForms as to
> how to push more if the validation to the database where that's
> appropriate. This would increase the performance of write-heavy sites, and
> also enforce better data integrity. At the moment all cleaning is done
> before the instance is saved, and the assumption is that the instance will
> pass db validation. As far as I know, we don't currently have any code
> which detects problems there (dB integrity errors) and ties them back to
> the fields or instances which have caused the issue. This is obviously
> backend-specific code, and is far from straightforward. I think some of the
> pending ValidationError changes could make it possible.
>
> Anssi and Andrew - do you think the dB can actually tell is enough
> information to get this right? I guess if not an option would be to fall
> back to the current query-creating code to work out why the save failed.
>
> Most of this is throwing thoughts around, apologies if they're illogical…
>
> Marc
> On 18 Oct 2013 15:18, "Cal Leeming [Simplicity Media Ltd]" <
> cal.leem...@simplicitymedialtd.co.uk> wrote:
>
>> Sorry please ignore my last email, my email client went a bit weird and
>> sent the draft whilst I was still editing/thinking. Here is the proper
>> version;
>>
>> This is yet another reason why I don't think it would be reasonable to
>> expect field validation within the model.
>>
>> If the validations were done using DB constraints (as per Anssi's reply)
>> then this would be a reasonable expectation, however as mentioned before
>> changes in the field data can come from multiple places and it is not safe
>> to assume that any data in the model is ever going to be valid.
>>
>> I think if the developer understands why this is important, they would be
>> adding a lot of code to handle these different scenarios anyway, so I don't
>> think removing the need to call full_clean() manually would be of much
>> benefit.
>>
>> However that being said, I would be in favour of adding a call which
>> allows you to validate changed and/or all fields, making it easier to
>> handle these different validation scenarios. (i.e. being able to easily
>> detect if it is old data that has caused validation error, or new data).
>>
>> Cal
>>
>>
>>
>>
>> On Fri, Oct 18, 2013 at 3:09 PM, Cal Leeming [Simplicity Media Ltd] <
>> cal.leem...@simplicitymedialtd.co.uk> wrote:
>>
>>> This is yet another reason why I don't think it would be reasonable to
>>> expect field validation within the model.
>>>
>>> I also think that introducing such a check would not only lure the user
>>> into a false sense of security
>>>
>>> If the validations were done using DB constraints (as per Anssi's reply)
>>> then this would be a reasonable expectation, however as mentioned before
>>> changes in the field data can come from multiple places and it is not safe
>>> to assume that any data in the model is ever going to be valid.
>>>
>>> Given the performance concerns and the minimal amount of benefit that
>>> validating on save() would give,
>>>
>>>
>>> On Fri, Oct 18, 2013 at 2:42 PM, Florian Apolloner <
>>> f.apollo...@gmail.com> wrote:
>>>
>>>>
>>>>
>>>> On Friday, October 18, 2013 4:28:21 AM UTC+2, Karen Tracey wrote:
>>>>>
>>>>> Wasn't there also concern for double validation performed during form
>>>>> clean and then model instance save?
>>>>>
>>>>
>>>> Yes, technically we would probably have to track the validation state
>>>> per field and also track changes to it etc… :(
>>>>
>>>> --
>>>> You received this message becaus

Re: Backwards compatibility and field validation

[Cal Leeming [Simplicity Media Ltd]]

Sorry please ignore my last email, my email client went a bit weird and
sent the draft whilst I was still editing/thinking. Here is the proper
version;

This is yet another reason why I don't think it would be reasonable to
expect field validation within the model.

If the validations were done using DB constraints (as per Anssi's reply)
then this would be a reasonable expectation, however as mentioned before
changes in the field data can come from multiple places and it is not safe
to assume that any data in the model is ever going to be valid.

I think if the developer understands why this is important, they would be
adding a lot of code to handle these different scenarios anyway, so I don't
think removing the need to call full_clean() manually would be of much
benefit.

However that being said, I would be in favour of adding a call which allows
you to validate changed and/or all fields, making it easier to handle these
different validation scenarios. (i.e. being able to easily detect if it is
old data that has caused validation error, or new data).

Cal




On Fri, Oct 18, 2013 at 3:09 PM, Cal Leeming [Simplicity Media Ltd] <
cal.leem...@simplicitymedialtd.co.uk> wrote:

> This is yet another reason why I don't think it would be reasonable to
> expect field validation within the model.
>
> I also think that introducing such a check would not only lure the user
> into a false sense of security
>
> If the validations were done using DB constraints (as per Anssi's reply)
> then this would be a reasonable expectation, however as mentioned before
> changes in the field data can come from multiple places and it is not safe
> to assume that any data in the model is ever going to be valid.
>
> Given the performance concerns and the minimal amount of benefit that
> validating on save() would give,
>
>
> On Fri, Oct 18, 2013 at 2:42 PM, Florian Apolloner 
> wrote:
>
>>
>>
>> On Friday, October 18, 2013 4:28:21 AM UTC+2, Karen Tracey wrote:
>>>
>>> Wasn't there also concern for double validation performed during form
>>> clean and then model instance save?
>>>
>>
>> Yes, technically we would probably have to track the validation state per
>> field and also track changes to it etc… :(
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Django developers" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to django-developers+unsubscr...@googlegroups.com.
>> To post to this group, send email to django-developers@googlegroups.com.
>> Visit this group at http://groups.google.com/group/django-developers.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/django-developers/31648978-7105-422f-ad62-68943e324a04%40googlegroups.com
>> .
>>
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAHKQagFULsc%2BHRKxnR2i_4KZJjLNuPD8aHnbwnf1GjhzduopVg%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Backwards compatibility and field validation

[Cal Leeming [Simplicity Media Ltd]]

This is yet another reason why I don't think it would be reasonable to
expect field validation within the model.

I also think that introducing such a check would not only lure the user
into a false sense of security

If the validations were done using DB constraints (as per Anssi's reply)
then this would be a reasonable expectation, however as mentioned before
changes in the field data can come from multiple places and it is not safe
to assume that any data in the model is ever going to be valid.

Given the performance concerns and the minimal amount of benefit that
validating on save() would give,


On Fri, Oct 18, 2013 at 2:42 PM, Florian Apolloner wrote:

>
>
> On Friday, October 18, 2013 4:28:21 AM UTC+2, Karen Tracey wrote:
>>
>> Wasn't there also concern for double validation performed during form
>> clean and then model instance save?
>>
>
> Yes, technically we would probably have to track the validation state per
> field and also track changes to it etc… :(
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/31648978-7105-422f-ad62-68943e324a04%40googlegroups.com
> .
>
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAHKQagEy%2B3F_0tCkeTBJ-QT4BOwrZqyGzwFsZx84i58HJyxrZw%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Backwards compatibility and field validation

[Cal Leeming [Simplicity Media Ltd]]

Thank you for the detailed reply, much appreciated.

I think the problem isn't just storing model validations with migrations,
because it's probably good practice for any developer to assume that field
data may be invalid (i.e. manual field/table changes, unknown validation
bugs from previous releases, buggy migration scripts etc). And a
for/model.save() loop would only work if auto-repair was feasible, in which
case the developer has to decide how to handle validation failures at
certain points in the code. The typical scenarios you'd handle when
encountering invalid data would be to ignore, repair or error - depending
on where you are within the code.

The approaches that come to mind are;

1) save(validate=True)
This still feels superfluous because we can just call full_clean() before
save().

2) full_clean() then save()
This works but you have to manually check the errors to see if it was old
or new data that causes validation

3) save(validate_only_changes=True) OR
full_clean(validate_only_changes=True)
Feels a bit better as it allows me to handle background jobs gracefully,
say a cron that is failing on a record that it doesn't know/need at that
point of execution.

4) CharField(validate_old=True)
This would not allow context specific handling (i.e. we want to enforce
validate old if we have a user request because we can ask them for updated
info, but do not enforce validate when inside a background job which does
not have the necessary context to request repair)

>From what I can tell, 3 would be a good approach based on the logic of
making it easier for the developer to decide how the failed validation
should be handled at certain points in the code.

Do you think that `save(validate_only_changes=True)` would be a good
approach (worthy of a patch), or am I over-engineering this problem?

Cal


On Thu, Oct 17, 2013 at 12:08 AM, Russell Keith-Magee <
russ...@keith-magee.com> wrote:

>
> On Wed, Oct 16, 2013 at 12:15 AM, Cal Leeming [Simplicity Media Ltd] <
> cal.leem...@simplicitymedialtd.co.uk> wrote:
>
>> Sorry I should have made myself a little more clear.
>>
>> When I said "invalidating data inside your models" I was referring to any
>> previous data that was saved when the validator was set to a minimum length
>> of 4 characters. By then changing the validator to be 5 characters, you are
>> effectively breaking save() on any data that was saved before that point,
>> and the ability to fix the model data may not be available in that specific
>> save() context. For example, a cron job that updates a "last_checked" field
>> on the user would not necessarily have data relevant to changes in the
>> users phone number (which could have its min/max length changed at any
>> point in the validators life time).
>>
>> Hope that makes more sense
>>
>> Your analysis is correct, but you're possibly over thinking this a bit.
>
> Consider a world in which model validation *was* on by default. You write
> some models with a char field, and you insert some data. All valid, all
> saved successfully. Now you change your models and add a validation
> condition. You've just exposed yourself to exactly the same situation,
> without feature-level backwards compatibility ever being part of the
> picture.
>
> What this highlights is that migration needs to be part of the whole
> solution. At the very least, as part of adding default model validation,
> we'd need to document the fact that all existing data must be assumed to be
> potentially invalid, and provide an easy way to force validation.
> Unfortunately, there's not sure I see any easy way to do this other than a
> "for model in database: model.save()" loop.
>
> Yours,
> Russ Magee %-)
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/CAJxq848-%3DwSerY4HT9zAcff9YVtjQLm7Xf6EmA_JhF3MX7xFkA%40mail.gmail.com
> .
>
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAHKQagFR811sPZPyvKgqDUndJx-rAVFHWT6O8N4kpZFBKxsXZg%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Backwards compatibility and field validation

[Cal Leeming [Simplicity Media Ltd]]

I'd considered having a `validate` attribute on save(), but this feels
unnecessary because we can already have model.full_clean(), having
`validate` seems superfluous.

One idea I'd had was to only validate fields which had changed. The reason
for this is because we cannot assume data in the model from previous save()
will validate correctly on this save() (i.e. in the event of validation
rules changing). However the concept of only validating changed fields
seems confusing and likely to sting a lot of people.

Ultimately I think code based validation cannot be enforced at the model
layer for these reasons, with the exception of IntegrityError raised from
db constraints (which in my opinion is completely different to validations
in code).

I'd like to hear what a core dev has to say on the matter, at the very
least I think this is worthy of a docs patch to explain this more clearly
to users.

Cal


On Wed, Oct 16, 2013 at 5:20 AM, poswald  wrote:

> One problem with it as you've found is that you sometimes do want control
> over it on a per-model or even per-instance (per-save) basis. One way to
> deal with this is to create something like the following that you can
> extend in your models:
>
> class ValidatedModel(models.Model):
> def save(self, validate=True, *args, **kwargs):
> if validate:
> self.full_clean()
> super(ValidatedModel, self).save(*args, **kwargs)
>
> class Meta:
> abstract = True
>
>
> This helps you if you're trying it follow the 'fat model' philosophy and
> centralize the validation logic down out of the forms and into the model
> layer. It might end up validating twice if used in a model form though, I'm
> not sure. I personally wouldn't be against having a setting to change in
> the project-wide default behavior in a future Django version, but I think
> the ability to override it is important.
>
>
>
>
> On Wednesday, October 16, 2013 1:15:04 AM UTC+9, Cal Leeming [Simplicity
> Media Ltd] wrote:
>
>> Sorry I should have made myself a little more clear.
>>
>> When I said "invalidating data inside your models" I was referring to any
>> previous data that was saved when the validator was set to a minimum length
>> of 4 characters. By then changing the validator to be 5 characters, you are
>> effectively breaking save() on any data that was saved before that point,
>> and the ability to fix the model data may not be available in that specific
>> save() context. For example, a cron job that updates a "last_checked" field
>> on the user would not necessarily have data relevant to changes in the
>> users phone number (which could have its min/max length changed at any
>> point in the validators life time).
>>
>> Hope that makes more sense
>>
>> Cal
>>
>>
>> On Tue, Oct 15, 2013 at 5:12 PM, Cal Leeming [Simplicity Media Ltd] <
>> cal.l...@**simplicitymedialtd.co.uk> wrote:
>>
>>> I've been thinking about this a little bit more, and it seems that the
>>> design approach of validating on save() is not entirely correct.
>>>
>>> For example, say you have a validator that ensures a field must be at
>>> least 4 characters long, and a few months later the validation is then
>>> changed to be 5 characters long. This means any save() call on that field
>>> (which may include a modification to a field which is unrelated to the
>>> change being made) will then stop that save from happening. By applying the
>>> validation at the model level, you are then potentially invalidating data
>>> inside your models. This is fine for forms, because it can be handled at
>>> the user side (by returning a form error saying X field is incorrect), but
>>> would cause problems at the model layer.
>>>
>>> I think by design it would be very wrong to validate the model fields at
>>> the point of save(), despite seeming like it would be the correct thing to
>>> do.
>>>
>>> Could someone please verify if I have come to the correct conclusion
>>> here?
>>>
>>> Cal
>>>
>>>
>>> On Tue, Oct 15, 2013 at 4:43 PM, Cal Leeming [Simplicity Media Ltd] <
>>> cal.l...@**simplicitymedialtd.co.uk> wrote:
>>>
>>>> Hello,
>>>>
>>>> I am trying to understand why field validators (model.full_clean()) are
>>>> not called for model.save()
>>>>
>>>> I've spent about an hour reviewing all the discussions/replies on this
>>>> subject;
>>>> http://www.xormedia.com/**django-model-validatio

Re: Backwards compatibility and field validation

[Cal Leeming [Simplicity Media Ltd]]

Sorry I should have made myself a little more clear.

When I said "invalidating data inside your models" I was referring to any
previous data that was saved when the validator was set to a minimum length
of 4 characters. By then changing the validator to be 5 characters, you are
effectively breaking save() on any data that was saved before that point,
and the ability to fix the model data may not be available in that specific
save() context. For example, a cron job that updates a "last_checked" field
on the user would not necessarily have data relevant to changes in the
users phone number (which could have its min/max length changed at any
point in the validators life time).

Hope that makes more sense

Cal


On Tue, Oct 15, 2013 at 5:12 PM, Cal Leeming [Simplicity Media Ltd] <
cal.leem...@simplicitymedialtd.co.uk> wrote:

> I've been thinking about this a little bit more, and it seems that the
> design approach of validating on save() is not entirely correct.
>
> For example, say you have a validator that ensures a field must be at
> least 4 characters long, and a few months later the validation is then
> changed to be 5 characters long. This means any save() call on that field
> (which may include a modification to a field which is unrelated to the
> change being made) will then stop that save from happening. By applying the
> validation at the model level, you are then potentially invalidating data
> inside your models. This is fine for forms, because it can be handled at
> the user side (by returning a form error saying X field is incorrect), but
> would cause problems at the model layer.
>
> I think by design it would be very wrong to validate the model fields at
> the point of save(), despite seeming like it would be the correct thing to
> do.
>
> Could someone please verify if I have come to the correct conclusion here?
>
> Cal
>
>
> On Tue, Oct 15, 2013 at 4:43 PM, Cal Leeming [Simplicity Media Ltd] <
> cal.leem...@simplicitymedialtd.co.uk> wrote:
>
>> Hello,
>>
>> I am trying to understand why field validators (model.full_clean()) are
>> not called for model.save()
>>
>> I've spent about an hour reviewing all the discussions/replies on this
>> subject;
>> http://www.xormedia.com/django-model-validation-on-save/
>>
>> http://stackoverflow.com/questions/4441539/why-doesnt-djangos-model-save-call-full-clean
>> https://code.djangoproject.com/ticket/13100
>> https://groups.google.com/forum/#!topic/django-developers/uIhzSwWHj4c
>>
>> From what I can tell this was rejected on the basis of breaking backwards
>> compatibility.
>>
>> It is unclear whether this would have been desired behavior had backwards
>> compatibility not been an issue.
>>
>> It also seems to me that it would be possible to maintain backwards
>> compatibility by using settings flags to determine the default behavior,
>> albeit at the cost of increased complexity.
>>
>> So I have four questions;
>>
>> 1) Without taking backwards compatibility into consideration, is it
>> reasonable to expect field validation automatically upon calling
>> model.save()
>> 2) At what point would Django devs be willing to break backwards
>> compatibility? Would this be considered on 2.x, or is it life time?
>> 3) Could backwards compatibility not be maintained by means of a flag in
>> settings? For example, settings.MODEL_VALIDATE_ON_SAVE=True would cause the
>> validation to be handled using the new approach?
>> 4) Assuming this cannot/will not be implemented, what would be the most
>> suitable workaround? Are there any risks in calling full_clean() without
>> ever using ModelForm? Should ModelForm always be used instead of using a
>> model directly? etc.
>>
>> Any comments/feedback would be much appreciated, I am also happy to spend
>> time updating docs to reflect responses in this thread.
>>
>> Thanks
>>
>> Cal
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAHKQagEXvMe0Te%2B8KtEGTR44pEXDBWkH7rFT%3DVjYogukeY6QKA%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Backwards compatibility and field validation

[Cal Leeming [Simplicity Media Ltd]]

I've been thinking about this a little bit more, and it seems that the
design approach of validating on save() is not entirely correct.

For example, say you have a validator that ensures a field must be at least
4 characters long, and a few months later the validation is then changed to
be 5 characters long. This means any save() call on that field (which may
include a modification to a field which is unrelated to the change being
made) will then stop that save from happening. By applying the validation
at the model level, you are then potentially invalidating data inside your
models. This is fine for forms, because it can be handled at the user side
(by returning a form error saying X field is incorrect), but would cause
problems at the model layer.

I think by design it would be very wrong to validate the model fields at
the point of save(), despite seeming like it would be the correct thing to
do.

Could someone please verify if I have come to the correct conclusion here?

Cal


On Tue, Oct 15, 2013 at 4:43 PM, Cal Leeming [Simplicity Media Ltd] <
cal.leem...@simplicitymedialtd.co.uk> wrote:

> Hello,
>
> I am trying to understand why field validators (model.full_clean()) are
> not called for model.save()
>
> I've spent about an hour reviewing all the discussions/replies on this
> subject;
> http://www.xormedia.com/django-model-validation-on-save/
>
> http://stackoverflow.com/questions/4441539/why-doesnt-djangos-model-save-call-full-clean
> https://code.djangoproject.com/ticket/13100
> https://groups.google.com/forum/#!topic/django-developers/uIhzSwWHj4c
>
> From what I can tell this was rejected on the basis of breaking backwards
> compatibility.
>
> It is unclear whether this would have been desired behavior had backwards
> compatibility not been an issue.
>
> It also seems to me that it would be possible to maintain backwards
> compatibility by using settings flags to determine the default behavior,
> albeit at the cost of increased complexity.
>
> So I have four questions;
>
> 1) Without taking backwards compatibility into consideration, is it
> reasonable to expect field validation automatically upon calling
> model.save()
> 2) At what point would Django devs be willing to break backwards
> compatibility? Would this be considered on 2.x, or is it life time?
> 3) Could backwards compatibility not be maintained by means of a flag in
> settings? For example, settings.MODEL_VALIDATE_ON_SAVE=True would cause the
> validation to be handled using the new approach?
> 4) Assuming this cannot/will not be implemented, what would be the most
> suitable workaround? Are there any risks in calling full_clean() without
> ever using ModelForm? Should ModelForm always be used instead of using a
> model directly? etc.
>
> Any comments/feedback would be much appreciated, I am also happy to spend
> time updating docs to reflect responses in this thread.
>
> Thanks
>
> Cal
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAHKQagFmejyyT2K8PdHx87b5s62Wcxq_u-63F%3DQK7_4w%3DxZQDQ%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Backwards compatibility and field validation

[Cal Leeming [Simplicity Media Ltd]]

Hello,

I am trying to understand why field validators (model.full_clean()) are not
called for model.save()

I've spent about an hour reviewing all the discussions/replies on this
subject;
http://www.xormedia.com/django-model-validation-on-save/
http://stackoverflow.com/questions/4441539/why-doesnt-djangos-model-save-call-full-clean
https://code.djangoproject.com/ticket/13100
https://groups.google.com/forum/#!topic/django-developers/uIhzSwWHj4c

>From what I can tell this was rejected on the basis of breaking backwards
compatibility.

It is unclear whether this would have been desired behavior had backwards
compatibility not been an issue.

It also seems to me that it would be possible to maintain backwards
compatibility by using settings flags to determine the default behavior,
albeit at the cost of increased complexity.

So I have four questions;

1) Without taking backwards compatibility into consideration, is it
reasonable to expect field validation automatically upon calling
model.save()
2) At what point would Django devs be willing to break backwards
compatibility? Would this be considered on 2.x, or is it life time?
3) Could backwards compatibility not be maintained by means of a flag in
settings? For example, settings.MODEL_VALIDATE_ON_SAVE=True would cause the
validation to be handled using the new approach?
4) Assuming this cannot/will not be implemented, what would be the most
suitable workaround? Are there any risks in calling full_clean() without
ever using ModelForm? Should ModelForm always be used instead of using a
model directly? etc.

Any comments/feedback would be much appreciated, I am also happy to spend
time updating docs to reflect responses in this thread.

Thanks

Cal

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAHKQagHu%2BLV%3DLNq%3DYaKc2PsF20qWniMkPhAiP9ZrOVVrMmVGkg%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: AbstractUser to get more abstract

[Cal Leeming [Simplicity Media Ltd]]

Although I cannot comment on the specific of how this could be done better,
I do at least concur that implementing a custom AbstractUser could be done
in a much better way!!

Cal


On Thu, Sep 12, 2013 at 9:44 PM, Abdulaziz Alfoudari <
aziz.alfoud...@gmail.com> wrote:

> This is a continuation of my post on 
> stackoverflow
> .
>
> With the introduction of Django 1.5, it was possible to create a custom
> User model which is flexible enough to have any user profile the developer
> wants created. However, looking at a very common problem which is using the
> email as the primary user identifier instead of username, the solution
> requires copying most of Django's internal definition of AbstractUser and
> that is only to remove the username field.
>
> A better solution in my opinion is make AbstractUser even more abstract by
> removing username field, and allowing the developer to explicitly specify
> the field to be used as the user identifier. This will require a tiny extra
> work for those that use the current default behavior, but it will also
> greatly reduce the work needed for the very common problem of using email
> as the user identifier.
>
> Please share your thoughts and opinions on this.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Proposal: remove triage stage someday/maybe

[Cal Leeming [Simplicity Media Ltd]]

+1

On Sun, Aug 18, 2013 at 3:57 PM, Tim Graham  wrote:

> I think the state is useful because it provides a way to filter out
> tickets that are not immediately actionable (for example, there are several
> tickets that suggest schema migrations). When migrations land in core, we
> can then "accept" these tickets.
>
>
> On Sunday, August 18, 2013 4:10:14 AM UTC-4, Wim Feijen wrote:
>>
>> Hi,
>>
>> When triaging tickets I came across the triage stage Someday/Maybe. The
>> docs say I shouldn't use it. :) I think they are right.
>>
>> For me triaging tickets is all about making a decision. I make a choice
>> between either: "yes, good idea", or "no, we definitely don't want that".
>> Someday/maybe seems like not a good choice here, because it leaves things
>> in the uncertain.
>>
>> Therefore, I'd like to propose to remove this triage stage, like we did
>> with design decision needed.
>>
>> Currently, there are 47 tickets marked as someday/maybe. Looking over
>> them, it seems to me that:
>> 1. most can be marked as Accepted. A solution may or may not be hard to
>> find but we accept that django could improve here.
>> 2. some should actually be marked as won't fix but weren't out of
>> hospitality.
>>
>> What is your opinion?
>>
>> Wim
>>
>  --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Specifying lists/tuples in models - wtf?

[Cal Leeming [Simplicity Media Ltd]]

Hi Noah,

Sorry yes I remember discussing this last time now, it wasn't until I hit
send that I remembered about 5 seconds after.

I think I'm having a bit of a stupid day - my apologies.

Cal

On Wed, Jun 5, 2013 at 8:57 PM, Noah Kantrowitz  wrote:

>
> On Jun 5, 2013, at 12:56 PM, Cal Leeming [Simplicity Media Ltd] wrote:
>
> > Hello,
> >
> > The following;
> > class Meta:
> > ordering = ('hostname')
> >
> > Results in;
> > amber.reseller: "ordering" refers to "h", a field that doesn't exist.
> > amber.reseller: "ordering" refers to "o", a field that doesn't exist.
> > amber.reseller: "ordering" refers to "s", a field that doesn't exist.
> > amber.reseller: "ordering" refers to "t", a field that doesn't exist.
> > amber.reseller: "ordering" refers to "n", a field that doesn't exist.
> > amber.reseller: "ordering" refers to "a", a field that doesn't exist.
> > amber.reseller: "ordering" refers to "m", a field that doesn't exist.
> > amber.reseller: "ordering" refers to "e", a field that doesn't exist.
> >
> > To fix this, I have to use;
> > class Meta:
> > ordering = ('hostname', )
> >
> > However in python cli;
> >
> > salt1 foxx # python
> > Python 2.6.6 (r266:84292, Dec 26 2010, 22:31:48)
> > [GCC 4.4.5] on linux2
> > Type "help", "copyright", "credits" or "license" for more information.
> > >>> ['hostname']
> > ['hostname']
> > >>> lol = ['hostname']
> > >>> lol[0]
> > 'hostname'
> > >>> lol[1]
> > Traceback (most recent call last):
> >   File "", line 1, in 
> > IndexError: list index out of range
> >
> > Could anyone please explain why Django is not treating this list/tuple
> properly?
>
> (x) isn't a tuple for the same reason (x+1)*4 isn't a tuple. (x,) is a
> tuple of one item.
>
> --Noah
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers?hl=en
> .
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Specifying lists/tuples in models - wtf?

[Cal Leeming [Simplicity Media Ltd]]

Okay, please ignore the below, it was because I used a tuple rather than a
list in my test.

>>> lol = ('hostname')
>>> print lol[0]
h

Cal

On Wed, Jun 5, 2013 at 8:56 PM, Cal Leeming [Simplicity Media Ltd] <
cal.leem...@simplicitymedialtd.co.uk> wrote:

> Hello,
>
> The following;
> class Meta:
> ordering = ('hostname')
>
> Results in;
> amber.reseller: "ordering" refers to "h", a field that doesn't exist.
> amber.reseller: "ordering" refers to "o", a field that doesn't exist.
> amber.reseller: "ordering" refers to "s", a field that doesn't exist.
> amber.reseller: "ordering" refers to "t", a field that doesn't exist.
> amber.reseller: "ordering" refers to "n", a field that doesn't exist.
> amber.reseller: "ordering" refers to "a", a field that doesn't exist.
> amber.reseller: "ordering" refers to "m", a field that doesn't exist.
> amber.reseller: "ordering" refers to "e", a field that doesn't exist.
>
> To fix this, I have to use;
> class Meta:
> ordering = ('hostname', )
>
> However in python cli;
>
> salt1 foxx # python
> Python 2.6.6 (r266:84292, Dec 26 2010, 22:31:48)
> [GCC 4.4.5] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
> >>> ['hostname']
> ['hostname']
> >>> lol = ['hostname']
> >>> lol[0]
> 'hostname'
> >>> lol[1]
> Traceback (most recent call last):
>   File "", line 1, in 
> IndexError: list index out of range
>
> Could anyone please explain why Django is not treating this list/tuple
> properly?
>
> Cal
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Specifying lists/tuples in models - wtf?

[Cal Leeming [Simplicity Media Ltd]]

Hello,

The following;
class Meta:
ordering = ('hostname')

Results in;
amber.reseller: "ordering" refers to "h", a field that doesn't exist.
amber.reseller: "ordering" refers to "o", a field that doesn't exist.
amber.reseller: "ordering" refers to "s", a field that doesn't exist.
amber.reseller: "ordering" refers to "t", a field that doesn't exist.
amber.reseller: "ordering" refers to "n", a field that doesn't exist.
amber.reseller: "ordering" refers to "a", a field that doesn't exist.
amber.reseller: "ordering" refers to "m", a field that doesn't exist.
amber.reseller: "ordering" refers to "e", a field that doesn't exist.

To fix this, I have to use;
class Meta:
ordering = ('hostname', )

However in python cli;

salt1 foxx # python
Python 2.6.6 (r266:84292, Dec 26 2010, 22:31:48)
[GCC 4.4.5] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> ['hostname']
['hostname']
>>> lol = ['hostname']
>>> lol[0]
'hostname'
>>> lol[1]
Traceback (most recent call last):
  File "", line 1, in 
IndexError: list index out of range

Could anyone please explain why Django is not treating this list/tuple
properly?

Cal

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Perception of attitude in tickets

[Cal Leeming [Simplicity Media Ltd]]

I was going to mention this before, but wasn't sure how to word it.

Russell has hit the spot though, giving the user a more personal
experience, not just automated (or manual) copy-pasta.

Cal

On Thu, May 16, 2013 at 1:29 PM, Russell Keith-Magee <
russ...@keith-magee.com> wrote:

>
> On Thu, May 16, 2013 at 7:16 PM, Luke Plant  wrote:
>
>> On 15/05/13 19:36, ptone wrote:
>>
>> > I wonder if a slightly more concise version of this should be added to
>> > the triaging docs instead of a wiki page (fine place to draft it
>> though).
>> >
>> >
>> https://docs.djangoproject.com/en/dev/internals/contributing/triaging-tickets/#closing-tickets
>> >
>> > I feel that the wiki pages aren't very discoverable, and unless we are
>> > talking about patching trac to include this, such a comment won't carry
>> > the official weight of being in the project docs.
>>
>> That's a good idea. The purpose of this wiki page is to make it easy for
>> triagers to link to - I personally hate having to go and find the right
>> page in the docs, but I can remember to do "DevelopersMailingList"
>> instead of "developers mailing list".
>>
>> Patching Trac sounds like a really good idea to me. While I completely
> appreciate the intent of these wiki messages, the way those messages are
> "deployed" at the present strikes me as something that could easily be
> interpreted as rude. I'd like to see a much more "humane" usage of these
> messages, so that new users to Trac don't feel like they're interaction
> with Django as a project isn't a mechanical rejection.
>
> Yours,
> Russ Magee %-)
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers?hl=en
> .
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Perception of attitude in tickets

[Cal Leeming [Simplicity Media Ltd]]

The fact that one of the core founders of Django has responded to say
"Consider it done", is probably about as official as it can get.

I have gone ahead and written up a section explaining how the 5-for-1
system works, any amendments or additions would be welcomed.

https://code.djangoproject.com/wiki/UsingTheMailingList#a5-for-1

The more people that know about this, the better.. if everyone could try
and mention it in passing conversations on IRC, or on stale
tickets/discussions etc.

Hope this helps

Cal

On Thu, May 16, 2013 at 1:56 AM, Jacob Kaplan-Moss wrote:

> On Tue, May 14, 2013 at 5:41 PM, Cal Leeming [Simplicity Media Ltd]
>  wrote:
> > * Make the 5-for-1 (or 10-for-1) system official, not many people seem to
> > realise this exists. This will give incentive to core devs to spend a bit
> > longer on a ticket, maybe even throwing in a pleasentry or two
> (optional). I
> > often found that if I assisted with other tickets and showed myself to be
> > proactive on the ML, then my tickets would usually get the attention of a
> > core dev faster and/or with more detailed response.
>
> Consider it done - I'll take any and all 5:1 requests personally, and
> I'll shame uh... *encourage* other core devs to do the same.
>
> What do you think we should do to make it more "official"? Add it to
> the contribution docs? Put it on the code.d.c somewhere?
>
> Jacob
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers?hl=en
> .
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Perception of attitude in tickets

[Cal Leeming [Simplicity Media Ltd]]

Hello all,

I just spent around 90 minutes reading through everyones comments (word for
word), and writing up a reply offering my two cents.

First off - a few years back someone introduced a 5-for-1 system where if
you triaged five other tickets, you could request for a core dev to give
detailed attention to any ticket of your choice. I think this is an
EXCELLENT way of giving something back to the community, keeping the ticket
queue down, and getting the results that you need.

>From core devs point of view - Posting a discussion to django-developers
and raising a ticket in Trac are not for the lighthearted. If you want
something in the core, you better have a good argument for it and be
prepared to follow it through to the bitter end. If you simply post a
suggestion without justification, then you can't expect the core dev to
sign it off, they are not mind readers. It's also generally accept that
although the core devs are polite, they will give you their brutally honest
truth which can sometimes be misinterpreted as rudeness. The core devs are
not here to be friendly, they are here to keep Django alive.

>From users point of view - One of the responses from the core developers
was, look if you want this feature, help us make it happen. But the
majority of users are not familiar with the internals of how Django works,
and would find it difficult to contribute the quality of code (if at all)
required for inclusion into the core. Sometimes people want to help and
contribute, but the process can be confusing and engaging with the core
devs in a discussion can often be a frustrating thing for those who are new
to the world of open source contributions.

However, Tom raises a very good point when he said "why should I waste my
time trying to jump through these arbitrary hoops? The short answer is I
don't, I work on my own
projects". Personally, I have become extremely frustrated at some of the
decisions and lack of attention being shown on some of these tickets in the
past. But it could also be argued that I should have stood my ground and
put forward a solid argument.

Yo-Yo Ma raises an important point of "The burden of proof is on the
originator of an idea", and this is in-line with my above comments. Shai
also raises a good point, that core devs time is scarce and often would
like some other people to give tickets a flick over before going back to
it.. core devs do not have time to give each ticket their full attention.
Imho, if you want the extra time, you have to earn it (either in
contributing, or putting forward a solid argument). Look at it like this,
for every 5 minutes you put in, you can get back roughly 1 minute from a
core dev (YMMV).

I think really this all comes down to a lack of understand about the Django
etiquette, and users feeling like the core devs are not really giving them
the attention they expected.

Therefore, I propose the following recommendations;

* Make the 5-for-1 (or 10-for-1) system official, not many people seem to
realise this exists. This will give incentive to core devs to spend a bit
longer on a ticket, maybe even throwing in a pleasentry or two (optional).
I often found that if I assisted with other tickets and showed myself to be
proactive on the ML, then my tickets would usually get the attention of a
core dev faster and/or with more detailed response.

* Explain what wontfix means to users, and what they can do to change it
(without having to read page after page of contribution instructions).
Perhaps some sort of automated response that explains all this, and points
the user to the mailing list. Until Kirby explained what wontfix meant a
few posts back, I had assumed it meant "this is not acknowledged".

To summarize my thoughts in a simple bullet point list;

* Core devs are not here to be friendly, expect polite yet brutal honesty,
with no pleasantries.
* If you want 5 minutes of core dev time, spend 1 hour triaging tickets and
link with proof (5-for-1).
* Trac could be so much better
* Explain what wontfix means to users, and that they need to present a
solid argument with proof to get it looked at again

Cal

On Fri, May 10, 2013 at 6:00 PM, Simon  wrote:

> Hi,
>
> When I started using Python a couple of months ago, a quick Google for
> frameworks turned up a lot of results for Django so I decided to give it a
> spin.
>
> I'd like to give some feedback on my experience to date. There are a lot
> of features I really love, some that are a little quirky and some that are
> downright inflexible. None of this will be news - it's the same for every
> framework. That said, I started to have doubts when I was attempting to
> find solutions/workarounds to the problems I encountered.
>
> Today was the 5th or 6th time that I've ended up at the ticket system and
> seen people saying "This would really help me" and a core developer saying
> "I don't see the need" (rather arbitrarily IMHO) and closing as wontfix.
> This is invariably followed by people asking for reconsi

Re: Relative path support for TEMPLATE_DIRS and others in settings.py (django ticket 694)

[Cal Leeming [Simplicity Media Ltd]]

Also the stupid thing as well, we are already using 'templatetags' and
'filters'.. I just had no idea it supported 'appname/templates' as well.

*doh*

Cal

On Tue, Jan 1, 2013 at 8:47 PM, Cal Leeming [Simplicity Media Ltd] <
cal.leem...@simplicitymedialtd.co.uk> wrote:

> Just realised why this has been so heavily debated, and just discovered
> something new.
>
> https://docs.djangoproject.com/en/dev/ref/templates/api/#loader-types
> django.template.loaders.app_directories.Loader
>
> If 'myproject.polls' is placed into INSTALLED_APPS then Django will look
> at '/path/to/myproject/polls/templates/' for the templates.
>
> I didn't even know about app_directories loader, and using it sounds much
> cleaner than having to use relative paths.
>
> In regards to the other two vars (media root etc), as stated before, those
> should really be deployment specific and not relative, this is a devops
> problem not a code problem.
>
> As such, I'm now -9000 on supporting relative paths.
>
> Thanks for explaining this Florian - makes a lot more sense now.
>
> Cal
>
> On Tue, Jan 1, 2013 at 7:36 PM, Florian Apolloner 
> wrote:
>
>> Hi,
>>
>>
>> On Tuesday, January 1, 2013 7:48:59 PM UTC+1, Cal Leeming [Simplicity
>> Media Ltd] wrote:
>>>
>>> Rather than saying "spend 30 seconds thinking about it", could you
>>> perhaps spend 30 seconds explaining why using relative paths for
>>> TEMPLATE_DIRS would be considered a bad thing to do?
>>
>>
>> I don't think that per se it would be a bad thing to do, but then you
>> could also just dump the project name (from startproject) into
>> INSTALLED_APPS and have the same effect (plus you can have project
>> templatetags and filters, yay :)) as Aymeric already tried to explain. I
>> guess your point for relative TEMPLATE_DIRS is only that you can have
>> PROJECT_DIR + templates which is already fullfiller by the app
>> template-loader in a much cleaner way imo.
>>
>> I hope that clears things up a bit.
>>
>> Cheers,
>> Florian
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Django developers" group.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msg/django-developers/-/xuVeOmwI_8AJ.
>>
>> To post to this group, send email to django-developers@googlegroups.com.
>> To unsubscribe from this group, send email to
>> django-developers+unsubscr...@googlegroups.com.
>> For more options, visit this group at
>> http://groups.google.com/group/django-developers?hl=en.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Relative path support for TEMPLATE_DIRS and others in settings.py (django ticket 694)

[Cal Leeming [Simplicity Media Ltd]]

Just realised why this has been so heavily debated, and just discovered
something new.

https://docs.djangoproject.com/en/dev/ref/templates/api/#loader-types
django.template.loaders.app_directories.Loader

If 'myproject.polls' is placed into INSTALLED_APPS then Django will look at
'/path/to/myproject/polls/templates/' for the templates.

I didn't even know about app_directories loader, and using it sounds much
cleaner than having to use relative paths.

In regards to the other two vars (media root etc), as stated before, those
should really be deployment specific and not relative, this is a devops
problem not a code problem.

As such, I'm now -9000 on supporting relative paths.

Thanks for explaining this Florian - makes a lot more sense now.

Cal

On Tue, Jan 1, 2013 at 7:36 PM, Florian Apolloner wrote:

> Hi,
>
>
> On Tuesday, January 1, 2013 7:48:59 PM UTC+1, Cal Leeming [Simplicity
> Media Ltd] wrote:
>>
>> Rather than saying "spend 30 seconds thinking about it", could you
>> perhaps spend 30 seconds explaining why using relative paths for
>> TEMPLATE_DIRS would be considered a bad thing to do?
>
>
> I don't think that per se it would be a bad thing to do, but then you
> could also just dump the project name (from startproject) into
> INSTALLED_APPS and have the same effect (plus you can have project
> templatetags and filters, yay :)) as Aymeric already tried to explain. I
> guess your point for relative TEMPLATE_DIRS is only that you can have
> PROJECT_DIR + templates which is already fullfiller by the app
> template-loader in a much cleaner way imo.
>
> I hope that clears things up a bit.
>
> Cheers,
> Florian
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/django-developers/-/xuVeOmwI_8AJ.
>
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-developers?hl=en.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Relative path support for TEMPLATE_DIRS and others in settings.py (django ticket 694)

[Cal Leeming [Simplicity Media Ltd]]

Just re-read my post, and realised it may have come across a bit loaded
(was replying very quickly)

No insult was intended, and it's a genuine question

Cal

On Tue, Jan 1, 2013 at 6:48 PM, Cal Leeming [Simplicity Media Ltd] <
cal.leem...@simplicitymedialtd.co.uk> wrote:

> For the record, the only time I'd suggested using relative paths is for
> 'TEMPLATE_DIRS' only, I do not use the other two.
>
> Rather than saying "spend 30 seconds thinking about it", could you perhaps
> spend 30 seconds explaining why using relative paths for TEMPLATE_DIRS
> would be considered a bad thing to do?
>
> Your post makes sense for STATIC_ROOT and MEDIA_ROOT, but it makes no
> sense for TEMPLATE_DIRS, imo.
>
> Cal
>
>
> On Tue, Jan 1, 2013 at 6:28 PM, Aymeric Augustin <
> aymeric.augus...@polytechnique.org> wrote:
>
>> A modern Django project is a collection of apps. Files are looked up
>> under conventional paths within apps. Modules (especially the settings
>> module) can live anywhere on $PYTHONPATH. Actually, there's not such thing
>> as a project root.
>>
>> For instance, instead of using TEMPLATE_DIRS, project-wide templates can
>> go into an "myproject" application. You need one anyway as soon as you
>> write a custom template tag or filter.
>>
>> There are two special cases that don't fit into apps: STATIC_ROOT and
>> MEDIA_ROOT. (Technically, there's ALLOWED_INCLUDE_ROOTS too, but it isn't
>> commonly used.)
>>
>> Static files are collected from the apps into STATIC_ROOT that is then
>> served by the web server. To avoid accidentally leaking Python code over
>> the web, it's a good idea to keep this directory outside of your source
>> tree.
>>
>> Media files are written by the application server into MEDIA_ROOT. It's a
>> good idea to put them on a separate partition (DoS by filling up / isn't
>> fun), or at least in a directory outside of your source tree, which must be
>> read-only for the web server.
>>
>> For local development, it's certainly fine to store the code in
>> /home/myproject, compile the static files in /home/myproject/static, and
>> uploading the media files in /home/myproject/media, and it's convenient to
>> express that with relative paths. But it's hardly a best practice in
>> production — at least, not until you've spent 30 seconds thinking about it.
>>
>> Django leaves it up to the developer to structure the settings for
>> different environments. This means we cannot provide a "development only"
>> settings template.
>>
>> For these reasons, I'm against codifying PROJECT_ROOT and relative paths
>> in Django's default settings files. No amount of "+1 because I'm doing it
>> already" or blog posts will make it a best practice in production.
>>
>> And developers will keep doing it until sysadmins tell them not to.
>>
>> --
>> Aymeric.
>>
>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Django developers" group.
>> To post to this group, send email to django-developers@googlegroups.com.
>> To unsubscribe from this group, send email to
>> django-developers+unsubscr...@googlegroups.com.
>> For more options, visit this group at
>> http://groups.google.com/group/django-developers?hl=en.
>>
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Relative path support for TEMPLATE_DIRS and others in settings.py (django ticket 694)

[Cal Leeming [Simplicity Media Ltd]]

For the record, the only time I'd suggested using relative paths is for
'TEMPLATE_DIRS' only, I do not use the other two.

Rather than saying "spend 30 seconds thinking about it", could you perhaps
spend 30 seconds explaining why using relative paths for TEMPLATE_DIRS
would be considered a bad thing to do?

Your post makes sense for STATIC_ROOT and MEDIA_ROOT, but it makes no sense
for TEMPLATE_DIRS, imo.

Cal

On Tue, Jan 1, 2013 at 6:28 PM, Aymeric Augustin <
aymeric.augus...@polytechnique.org> wrote:

> A modern Django project is a collection of apps. Files are looked up under
> conventional paths within apps. Modules (especially the settings module)
> can live anywhere on $PYTHONPATH. Actually, there's not such thing as a
> project root.
>
> For instance, instead of using TEMPLATE_DIRS, project-wide templates can
> go into an "myproject" application. You need one anyway as soon as you
> write a custom template tag or filter.
>
> There are two special cases that don't fit into apps: STATIC_ROOT and
> MEDIA_ROOT. (Technically, there's ALLOWED_INCLUDE_ROOTS too, but it isn't
> commonly used.)
>
> Static files are collected from the apps into STATIC_ROOT that is then
> served by the web server. To avoid accidentally leaking Python code over
> the web, it's a good idea to keep this directory outside of your source
> tree.
>
> Media files are written by the application server into MEDIA_ROOT. It's a
> good idea to put them on a separate partition (DoS by filling up / isn't
> fun), or at least in a directory outside of your source tree, which must be
> read-only for the web server.
>
> For local development, it's certainly fine to store the code in
> /home/myproject, compile the static files in /home/myproject/static, and
> uploading the media files in /home/myproject/media, and it's convenient to
> express that with relative paths. But it's hardly a best practice in
> production — at least, not until you've spent 30 seconds thinking about it.
>
> Django leaves it up to the developer to structure the settings for
> different environments. This means we cannot provide a "development only"
> settings template.
>
> For these reasons, I'm against codifying PROJECT_ROOT and relative paths
> in Django's default settings files. No amount of "+1 because I'm doing it
> already" or blog posts will make it a best practice in production.
>
> And developers will keep doing it until sysadmins tell them not to.
>
> --
> Aymeric.
>
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-developers?hl=en.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Relative path support for TEMPLATE_DIRS and others in settings.py (django ticket 694)

[Cal Leeming [Simplicity Media Ltd]]

On Sun, Dec 30, 2012 at 11:01 PM, Luke Plant  wrote:

> On 29/12/12 04:08, Cal Leeming [Simplicity Media Ltd] wrote:
>
> > Could we not have something like this in the settings.py, which in turn
> > enabled the code pasted above?
> > TEMPLATE_PATH_RELATIVE=True
>
> For consistency, we'd need STATICFILES_PATH_RELATIVE,
> STATIC_ROOT_PATH_RELATIVE, MEDIA_ROOT_PATH_RELATIVE etc. which is
> craziness. Having just one extra setting is a big deal.
>
> There are use cases for all of these being absolute paths (or at least
> some of them), and use cases for all of them being relative. We've
> already chosen absolute paths, and you can generate absolute from
> relative using os.path.realpath etc.
>
> The only option I can is whether we put that snippet of code (e.g.
> PROJECT_ROOT=os.path.realpath(os.path.dirname(__file__)) ) into the
> settings file generated when starting a new project.
>

+1


>
> Luke
>
>
> --
> "If we could just get everyone to close their eyes and visualise
> world peace for an hour, imagine how serene and quiet it would be
> until the looting started" -- Anon
>
> Luke Plant || http://lukeplant.me.uk/
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-developers?hl=en.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Relative path support for TEMPLATE_DIRS and others in settings.py (django ticket 694)

[Cal Leeming [Simplicity Media Ltd]]

Since the day I started using Django, I have always used a relative path
for TEMPLATES_DIR.

import os
CURRENT_DIR = os.path.realpath(os.path.dirname(__file__))
TEMPLATE_DIRS  =  "%s/templates/" % ( CURRENT_DIR, )

Imho, the idea of having to hard code your PROJECT_ROOT is ludicrous.

I understand the arguments being made about relative paths on settings, but
I am yet to come across a project where the above code has not worked.

Could we not have something like this in the settings.py, which in turn
enabled the code pasted above?
TEMPLATE_PATH_RELATIVE=True

Cal

On Sat, Dec 29, 2012 at 2:07 AM, Luke Plant  wrote:

>
>
> On 28/12/12 16:42, Daniel Sokolowski wrote:
> > PROJECT_ROOT is what I have been using myself and seen it done by others
> > so to me it makes sense to introduce an official setting for this
> purpose.
>
> PROJECT_ROOT would still need to be defined inside the settings.py
> module, using the os.path.dir(__file__) etc. dance, because it can't be
> defined within django code (which doesn't know where your project
> lives), and a utility function to do it isn't worth it (it's only one
> line, and you don't want your settings file to be doing imports to
> Django code).
>
> So I agree with the original WONTFIX here.
>
> I also think that app directories for the template loader is a better
> solution, and making a 'project' app if necessary answers the objection.
>
> However, I do see a case for putting the PROJECT_ROOT code into the
> settings.py generated by creating a new template, and updating the help
> text for TEMPLATE_DIRS and STATICFILES_DIRS to mention it.
>
>
> Luke
>
>
> --
> The fashion wears out more apparel than the man.
> -- William Shakespeare
>
> Luke Plant || http://lukeplant.me.uk/
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-developers?hl=en.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Automatic deployment tool

[Cal Leeming [Simplicity Media Ltd]]

On Sun, Dec 9, 2012 at 11:40 PM, Jonathan Slenders <
jonathan.slend...@gmail.com> wrote:

> Thanks for your feedback, Cal,
>
> You're right about the documentation, some very useful parts aren't even
> documented at all. (There are comments in the code, but there are some
> little things you wouldn't know from the readme.)
>
> During the last years I also put most of the effort in experimenting and
> getting something really useful. It took a while before I got there, and it
> didn't make sense to document features which were to be refactored every
> now and then. However, now I feel its quite stable. I should start making
> better documentation, and a lot of usage examples. I also cannot deny that
> the learning curve may be a little bit steeper then Fabric in the very
> beginning, but soon you will see the advantages.
>


> If only I were as good in selling a project as I can code. :)
>

I know that feeling


>
> Anyway, I hope this can also improve automatic deployment of Django
> applications for other people.
>
> Cheers,
> Jonathan
>
>
> Le lundi 10 décembre 2012 00:15:58 UTC+1, Cal Leeming [Simplicity Media
> Ltd] a écrit :
>>
>> Hi Jonathan,
>>
>> Just from a very brief point of view.. my eyes started to glaze over
>> whilst looking at the github README, and even more so when I looked at the
>> code.
>>
>> Even if this was the best thing since sliced bread, the documentation in
>> its current state leaves me with the feeling of "why do I want to use this".
>>
>> I think what would benefit this project massively is good/easy to read
>> documentation, with a simple overview section explaining common uses, what
>> makes it better than alternatives, etc.. maybe via readthedocs..?
>>
>> Statements such as "It's as declarative as possible.." sound impressive,
>> but don't really give me much insight into what this is, and why I'd want
>> to use it.
>>
>> Hope this helps!
>>
>> Cal
>>
>> On Sun, Dec 9, 2012 at 3:30 PM, Jonathan Slenders 
>> wrote:
>>
>>> Hi Everyone,
>>>
>>> In the past there have been some discussionh about how to deploy Django
>>> web applications through SSH. How to use Fabric or other tools, and whether
>>> we should provide or maybe force users to deploy applications according to
>>> a certain conventions.
>>>
>>> Back then, maybe already more than a year ago, I said that I was working
>>> on my own deployment tool. [1] Something that could be used instead of
>>> Fabric. It's a tool which could probably help a lot of you, although it can
>>> take a while to adopt. The core consists of high quality code. I took me a
>>> while before I felt confident enough for releasing this, and it has been
>>> refactored more then any project I did before. Things still can be
>>> improved, but it's ready to share with you.
>>>
>>> Key features are:
>>>
>>>- Interactive execution of remote commands. Locally, they will
>>>appear in a pseudo terminal (created with openpty), so that even editors
>>>like Vim or Emacs works fine when you run them on the remote end. You can
>>>easy start an interactive shell on any host as well.
>>>- Reusability of all deployment code is a key point. It's as
>>>declarative as possible, but without loosing Python's power to express
>>>everything as dynamic as you'd like to. Deployment code is hierarchically
>>>structured, with inheritance where possible.
>>>- Parallel execution is easy when enabled, while keeping interaction
>>>with these remote processes possible through pseudoterminals. Every
>>>parallel task gets his own terminal, either a new xterm or gnome-terminal
>>>window, a tmux pane, or whatever you'd like to.
>>>- Logging of your deployments. New loggers are easily pluggable into
>>>the system.
>>>
>>>
>>> So, enjoy!
>>>
>>> So, what does it have to do with Django? I have a setup-definition of
>>> what we use for Django deployment [2]. However, I suppose that quite a lot
>>> of people aren't using uwsgi like us. So, I'd like to know what the most
>>> common use cases of Django deployment are. If I can cover most cases, it's
>>> very easy for end-users to pick one, override what they don't like, and
>>> enjoy the full power of this deployment system.
>>>
>>> For instance, to demonstrate the

Re: Automatic deployment tool

[Cal Leeming [Simplicity Media Ltd]]

Hi Jonathan,

Just from a very brief point of view.. my eyes started to glaze over whilst
looking at the github README, and even more so when I looked at the code.

Even if this was the best thing since sliced bread, the documentation in
its current state leaves me with the feeling of "why do I want to use this".

I think what would benefit this project massively is good/easy to read
documentation, with a simple overview section explaining common uses, what
makes it better than alternatives, etc.. maybe via readthedocs..?

Statements such as "It's as declarative as possible.." sound impressive,
but don't really give me much insight into what this is, and why I'd want
to use it.

Hope this helps!

Cal

On Sun, Dec 9, 2012 at 3:30 PM, Jonathan Slenders <
jonathan.slend...@gmail.com> wrote:

> Hi Everyone,
>
> In the past there have been some discussionh about how to deploy Django
> web applications through SSH. How to use Fabric or other tools, and whether
> we should provide or maybe force users to deploy applications according to
> a certain conventions.
>
> Back then, maybe already more than a year ago, I said that I was working
> on my own deployment tool. [1] Something that could be used instead of
> Fabric. It's a tool which could probably help a lot of you, although it can
> take a while to adopt. The core consists of high quality code. I took me a
> while before I felt confident enough for releasing this, and it has been
> refactored more then any project I did before. Things still can be
> improved, but it's ready to share with you.
>
> Key features are:
>
>- Interactive execution of remote commands. Locally, they will appear
>in a pseudo terminal (created with openpty), so that even editors like Vim
>or Emacs works fine when you run them on the remote end. You can easy start
>an interactive shell on any host as well.
>- Reusability of all deployment code is a key point. It's as
>declarative as possible, but without loosing Python's power to express
>everything as dynamic as you'd like to. Deployment code is hierarchically
>structured, with inheritance where possible.
>- Parallel execution is easy when enabled, while keeping interaction
>with these remote processes possible through pseudoterminals. Every
>parallel task gets his own terminal, either a new xterm or gnome-terminal
>window, a tmux pane, or whatever you'd like to.
>- Logging of your deployments. New loggers are easily pluggable into
>the system.
>
>
> So, enjoy!
>
> So, what does it have to do with Django? I have a setup-definition of what
> we use for Django deployment [2]. However, I suppose that quite a lot of
> people aren't using uwsgi like us. So, I'd like to know what the most
> common use cases of Django deployment are. If I can cover most cases, it's
> very easy for end-users to pick one, override what they don't like, and
> enjoy the full power of this deployment system.
>
> For instance, to demonstrate the power. If we want to connect to a Django
> shell_plus of our Mobile Vikings production system, we type in the
> interactive shell:
>
> > mobile_vikings django shell_plus
>
> This will call the shell_plus function of our django setup, it will ask on
> which host the shell needs to be started, and immediately fire an
> interactive shell_plus of the remote server in your terminal.
>
> [1] https://github.com/jonathanslenders/python-deployer
> [2]
> https://github.com/citylive/citylive-operations/blob/master/deployment/deployer/contrib/services/django.py
>
> I'll publish one of these days on pypi.
>
> All feedback is welcome. For bugs/feature requests on things which arn't
> Django related, please go to the github.
>
> Cheers,
> Jonathan
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/django-developers/-/k4RS_9Kmn9cJ.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-developers?hl=en.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: #18557: get_or_create() causes a race condition with MySQL

[Cal Leeming [Simplicity Media Ltd]]

Curious - I didn't know about that.

At some point I will have to build a unit test for all these approaches,
and see which comes out best - prob won't have time to sort this out for a
few weeks though.

Cal

On Tue, Sep 4, 2012 at 3:53 PM, lucky  wrote:

> *E) Use locking with SELECT query*
> According to
> http://dev.mysql.com/doc/refman/5.5/en/innodb-consistent-read.html
>
>  If you want to see the “freshest” state of the database, use either the
> READ COMMITTED isolation level or a locking read:
>  SELECT * FROM t LOCK IN SHARE MODE;
>
> It locks the SELECT query until all concurrent transactions will commit
> requested rows. They are conventional conflict resolution mechanisms for
> databases.
>
>
>
> On Sunday, August 12, 2012 9:41:15 PM UTC+6, Cal Leeming [Simplicity Media
> Ltd] wrote:
>
>> Based on all the responses given so far, here are the options available.
>>
>> Further feedback would be much appreciated.
>>
>> *A) Use READ COMMITTED as a global/my.cnf:*
>>
>> Last time we tried read committed isolation levels, it caused various PHP
>> applications to break for an unknown reason - as it was in a production
>> environment we had to instantly revert to save downtime, and after it was
>> reverted the problem went away. Sadly no time time was put into finding the
>> exact cause of why this broke.
>>
>> This also broke PHP applications to which we did not have any source code
>> access, and caused some deadlocking problems - again, due to lack of source
>> code we were unable to determine the root cause of the problem.
>>
>> In general, it seems that READ COMMITTED may break apps that execute
>> database queries in a certain way.
>>
>>
>> *B) Use "SET TRANSACTION ISOLATION LEVEL READ COMMITTED" before every
>> transaction (or apply to the session somehow).*
>>
>> It is not clear how this would integrate nicely into the ORM. Is there a
>> cleaner way of ensuring a transaction isolation level is set to read
>> commited, other than having to call the following before every transaction?
>>
>> >>> connection.cursor().execute('S**ET TRANSACTION ISOLATION LEVEL READ
>> COMMITTED')
>>
>> You could apply the above to the session but, as above, I'm not sure how
>> you'd ensure every db session had this query executed, other than doing it
>> manually (which again, seems ugly).
>>
>> >>> connection.cursor().execute('S**ET SESSION TRANSACTION ISOLATION
>> LEVEL READ COMMITTED')
>>
>> Should there perhaps be an additional ticket that raises the need to have
>> a decorator that does this for us, or a settings.py attribute of some sort?
>>
>>
>> *C) Execute a commit before each get_or_create() call*
>>
>> It is worth noting that committing the transaction prior to calling
>> get_or_create() has given our apps a 100% success rate in preventing this
>> race condition, where as previously the app wouldn't even last 60 seconds
>> without throwing an IntegrityError exception.
>>
>> So although this approach is not fool proof (as you detailed in your
>> earlier threads), it has been close enough to prevent the problem from
>> happening in our use case.
>>
>>
>> *D) Use database level auto commit*
>>
>> Karen - could you clarify further on this, as I might have misunderstood.
>>
>> Looking at the docs, MySQL states that autocommit for transactions are
>> enabled by default
>>
>> http://dev.mysql.com/doc/**refman/5.0/en/commit.html<http://dev.mysql.com/doc/refman/5.0/en/commit.html>
>>
>> I couldn't find any other mentioning of 'autocommit' in the MySQL docs
>> - so I'm not sure this would have any impact?
>>
>>
>> On Sat, Aug 11, 2012 at 11:29 PM, Karen Tracey  wrote:
>>
>>> On Thu, Aug 9, 2012 at 5:58 PM, Cal Leeming [Simplicity Media Ltd] <
>>> cal.l...@**simplicitymedialtd.co.uk> wrote:
>>>
>>>> Sorry, please ignore that last message, I see now that you
>>>> were referring to this:
>>>>
>>>> https://docs.djangoproject.**com/en/dev/ref/databases/#**
>>>> autocommit-mode<https://docs.djangoproject.com/en/dev/ref/databases/#autocommit-mode>
>>>>
>>>>
>>>> So essentially, the official documentation would state that to resolve
>>>> this problem, you would use the following for your db settings:
>>>>
>>>> 'OPTIONS': {
>>>> 'autocommit': True,
>>>> }
>>

Re: #18557: get_or_create() causes a race condition with MySQL

[Cal Leeming [Simplicity Media Ltd]]

On Sun, Aug 12, 2012 at 4:41 PM, Cal Leeming [Simplicity Media Ltd] <
cal.leem...@simplicitymedialtd.co.uk> wrote:

> Based on all the responses given so far, here are the options available.
>
> Further feedback would be much appreciated.
>
> *A) Use READ COMMITTED as a global/my.cnf:*
>
> Last time we tried read committed isolation levels, it caused various PHP
> applications to break for an unknown reason - as it was in a production
> environment we had to instantly revert to save downtime, and after it was
> reverted the problem went away. Sadly no time time was put into finding the
> exact cause of why this broke.
>
> This also broke PHP applications to which we did not have any source code
> access, and caused some deadlocking problems - again, due to lack of source
> code we were unable to determine the root cause of the problem.
>
> In general, it seems that READ COMMITTED may break apps that execute
> database queries in a certain way.
>
>
> *B) Use "SET TRANSACTION ISOLATION LEVEL READ COMMITTED" before every
> transaction (or apply to the session somehow).*
>
> It is not clear how this would integrate nicely into the ORM. Is there a
> cleaner way of ensuring a transaction isolation level is set to read
> commited, other than having to call the following before every transaction?
>
> >>> connection.cursor().execute('SET TRANSACTION ISOLATION LEVEL READ
> COMMITTED')
>
> You could apply the above to the session but, as above, I'm not sure how
> you'd ensure every db session had this query executed, other than doing it
> manually (which again, seems ugly).
>
> >>> connection.cursor().execute('SET SESSION TRANSACTION ISOLATION LEVEL
> READ COMMITTED')
>
> Should there perhaps be an additional ticket that raises the need to have
> a decorator that does this for us, or a settings.py attribute of some sort?
>

Quote from: https://code.djangoproject.com/ticket/14026

> You certainly can do this already with the OPTIONS key on the Database
dictionary, see  http://docs.djangoproject.com/en/dev/ref/settings/#options
>  To set ISOLATION LEVEL use something like {"init_command": "SET SESSION
TRANSACTION ISOLATION LEVEL $DESIRED_ISOLATION_LEVEL"}


>
>
> *C) Execute a commit before each get_or_create() call*
>
> It is worth noting that committing the transaction prior to calling
> get_or_create() has given our apps a 100% success rate in preventing this
> race condition, where as previously the app wouldn't even last 60 seconds
> without throwing an IntegrityError exception.
>
> So although this approach is not fool proof (as you detailed in your
> earlier threads), it has been close enough to prevent the problem from
> happening in our use case.
>
>
> *D) Use database level auto commit*
>
> Karen - could you clarify further on this, as I might have misunderstood.
>
> Looking at the docs, MySQL states that autocommit for transactions are
> enabled by default
>
> http://dev.mysql.com/doc/refman/5.0/en/commit.html
>
> I couldn't find any other mentioning of 'autocommit' in the MySQL docs
> - so I'm not sure this would have any impact?
>
>
> On Sat, Aug 11, 2012 at 11:29 PM, Karen Tracey  wrote:
>
>> On Thu, Aug 9, 2012 at 5:58 PM, Cal Leeming [Simplicity Media Ltd] <
>> cal.leem...@simplicitymedialtd.co.uk> wrote:
>>
>>> Sorry, please ignore that last message, I see now that you
>>> were referring to this:
>>>
>>> https://docs.djangoproject.com/en/dev/ref/databases/#autocommit-mode
>>>
>>> So essentially, the official documentation would state that to resolve
>>> this problem, you would use the following for your db settings:
>>>
>>> 'OPTIONS': {
>>> 'autocommit': True,
>>> }
>>>
>>> Is that correct?
>>>
>>
>> No...that syntax is pulled out of a PostgreSQL doc note and I don't think
>> it would work with MySQL, though I am not entirely sure of that.
>>
>> Also I am not sure I would recommend a global DB level setting for this
>> -- you're dispensing with any transactions at that point, which may well be
>> inappropriate for an app that is having trouble with get_or_create. It's
>> very hard for Django to give explicit instructions for what is best to do
>> "in general" since it all depends on the needs of the application with
>> respect to transactions. I would say in general I'd recommend "read
>> committed" isolation level vs. database-level autocommit, but the ticket
>> noted that read committed "can bre

Re: #18557: get_or_create() causes a race condition with MySQL

[Cal Leeming [Simplicity Media Ltd]]

Based on all the responses given so far, here are the options available.

Further feedback would be much appreciated.

*A) Use READ COMMITTED as a global/my.cnf:*

Last time we tried read committed isolation levels, it caused various PHP
applications to break for an unknown reason - as it was in a production
environment we had to instantly revert to save downtime, and after it was
reverted the problem went away. Sadly no time time was put into finding the
exact cause of why this broke.

This also broke PHP applications to which we did not have any source code
access, and caused some deadlocking problems - again, due to lack of source
code we were unable to determine the root cause of the problem.

In general, it seems that READ COMMITTED may break apps that execute
database queries in a certain way.


*B) Use "SET TRANSACTION ISOLATION LEVEL READ COMMITTED" before every
transaction (or apply to the session somehow).*

It is not clear how this would integrate nicely into the ORM. Is there a
cleaner way of ensuring a transaction isolation level is set to read
commited, other than having to call the following before every transaction?

>>> connection.cursor().execute('SET TRANSACTION ISOLATION LEVEL READ
COMMITTED')

You could apply the above to the session but, as above, I'm not sure how
you'd ensure every db session had this query executed, other than doing it
manually (which again, seems ugly).

>>> connection.cursor().execute('SET SESSION TRANSACTION ISOLATION LEVEL
READ COMMITTED')

Should there perhaps be an additional ticket that raises the need to have a
decorator that does this for us, or a settings.py attribute of some sort?


*C) Execute a commit before each get_or_create() call*

It is worth noting that committing the transaction prior to calling
get_or_create() has given our apps a 100% success rate in preventing this
race condition, where as previously the app wouldn't even last 60 seconds
without throwing an IntegrityError exception.

So although this approach is not fool proof (as you detailed in your
earlier threads), it has been close enough to prevent the problem from
happening in our use case.


*D) Use database level auto commit*

Karen - could you clarify further on this, as I might have misunderstood.

Looking at the docs, MySQL states that autocommit for transactions are
enabled by default

http://dev.mysql.com/doc/refman/5.0/en/commit.html

I couldn't find any other mentioning of 'autocommit' in the MySQL docs - so
I'm not sure this would have any impact?


On Sat, Aug 11, 2012 at 11:29 PM, Karen Tracey  wrote:

> On Thu, Aug 9, 2012 at 5:58 PM, Cal Leeming [Simplicity Media Ltd] <
> cal.leem...@simplicitymedialtd.co.uk> wrote:
>
>> Sorry, please ignore that last message, I see now that you
>> were referring to this:
>>
>> https://docs.djangoproject.com/en/dev/ref/databases/#autocommit-mode
>>
>> So essentially, the official documentation would state that to resolve
>> this problem, you would use the following for your db settings:
>>
>> 'OPTIONS': {
>> 'autocommit': True,
>> }
>>
>> Is that correct?
>>
>
> No...that syntax is pulled out of a PostgreSQL doc note and I don't think
> it would work with MySQL, though I am not entirely sure of that.
>
> Also I am not sure I would recommend a global DB level setting for this --
> you're dispensing with any transactions at that point, which may well be
> inappropriate for an app that is having trouble with get_or_create. It's
> very hard for Django to give explicit instructions for what is best to do
> "in general" since it all depends on the needs of the application with
> respect to transactions. I would say in general I'd recommend "read
> committed" isolation level vs. database-level autocommit, but the ticket
> noted that read committed "can break legacy apps" (why, I'm not sure, and
> it doesn't explain), so for the sake of completeness I mentioned that
> database level autocommit would also fix the race condition that exists in
> get_or_create.
>
> I don't believe the doc can give a blanket "do this and your code will
> work" statement here. It can say Django's own code in get_or_create
> requires either that the transaction isolation level be set to "read
> committed" or DB level autocommit be used. Whether that is best done for
> the project globally via an init_command or only for certain requests via
> explicit cursor commands (see
> http://groups.google.com/group/django-users/msg/55fa3724d2754013) depends
> on the project itself and what else it is doing besides calling
> get_or_create.
>
> Karen
>
>  --
> You received this message because you ar

Re: #18557: get_or_create() causes a race condition with MySQL

[Cal Leeming [Simplicity Media Ltd]]

Sorry, please ignore that last message, I see now that you
were referring to this:

https://docs.djangoproject.com/en/dev/ref/databases/#autocommit-mode

So essentially, the official documentation would state that to resolve this
problem, you would use the following for your db settings:

'OPTIONS': {
'autocommit': True,
}

Is that correct?

Cal

On Thu, Aug 9, 2012 at 10:56 PM, Cal Leeming [Simplicity Media Ltd] <
cal.leem...@simplicitymedialtd.co.uk> wrote:

> Thanks for the detailed explanation on this.
>
> Am I correct in assuming you are referring to the following decorator as
> being the fix for the problem?
>
> @transaction.autocommit
> def viewfunc(request):
> pass
>
>
> On Thu, Aug 9, 2012 at 3:55 AM, Karen Tracey  wrote:
>
>> On Wed, Aug 8, 2012 at 9:25 AM, Cal Leeming [Simplicity Media Ltd] <
>> cal.leem...@simplicitymedialtd.co.uk> wrote:
>>
>>> I'm not entirely sure that suggesting every query needs to be committed
>>> is the right way forward either, given that you only need to commit once
>>> before get_or_create() is called to prevent the issue.
>>>
>>> Could you expand on why you feel every query would need to be committed?
>>>
>>
>> get_or_create does:
>>
>> 1 - get
>> 2 - if get fails due to DoesNotExist, create
>> 3 - if create fails due to IntegrityError, get again
>>
>> The problem with repeatable read transaction level is that step 3 can
>> never return something other than what was found (or not) in step 1. If
>> some other thread, between the times at which 1 and 2 happen, creates what
>> this thread is attempting to get_or_create, then the get_or_create is going
>> to fail, because the get will still find nothing in step 3, causing
>> get_or_create to re-raise the IntegrityError from step 2.
>>
>> Issuing a commit before calling get_or_create doesn't fix this race
>> condition that exists within the get_or_create code itself. Such a pre-call
>> commit is only going to be helpful if the current transaction has already
>> done a read that has fixed what is going to be returned by the DB in step
>> 1. If that has happened, then committing before calling get_or_create will
>> reduce the likelihood of hitting the race condition, since step 1 may then
>> read a row that has been created subsequent to the last read by this thread
>> but before this thread gets to step 1 in get_or_create. Essentially the
>> pre-commit narrows the race condition window in this case. (If no read has
>> yet been done by the transaction to already set in stone what this
>> transaction will read in step 1, then committing before calling
>> get_or_create does not help at all.)
>>
>> Regardless of what's done before the call into get_or_create, there's
>> still a small window of opportunity, between steps 1 and 2, for
>> get_or_create to fail. get_or_create requires either DB level autocommit or
>> read committed transaction isolation level in order for it to be free of
>> this race condition. Recommending anything else in Django's docs would be
>> misleading.
>>
>>
>> Karen
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Django developers" group.
>> To post to this group, send email to django-developers@googlegroups.com.
>> To unsubscribe from this group, send email to
>> django-developers+unsubscr...@googlegroups.com.
>> For more options, visit this group at
>> http://groups.google.com/group/django-developers?hl=en.
>>
>
>
>
> --
>
> *Cal Leeming
> Technical Support | Simplicity Media Ltd
> **US *310-362-7070 | *UK *02476 100401 | *Direct *02476 100402
>
> *Available 24 hours a day, 7 days a week.*
>
>


-- 

*Cal Leeming
Technical Support | Simplicity Media Ltd
**US *310-362-7070 | *UK *02476 100401 | *Direct *02476 100402

*Available 24 hours a day, 7 days a week.*

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: #18557: get_or_create() causes a race condition with MySQL

[Cal Leeming [Simplicity Media Ltd]]

Thanks for the detailed explanation on this.

Am I correct in assuming you are referring to the following decorator as
being the fix for the problem?

@transaction.autocommit
def viewfunc(request):
pass


On Thu, Aug 9, 2012 at 3:55 AM, Karen Tracey  wrote:

> On Wed, Aug 8, 2012 at 9:25 AM, Cal Leeming [Simplicity Media Ltd] <
> cal.leem...@simplicitymedialtd.co.uk> wrote:
>
>> I'm not entirely sure that suggesting every query needs to be committed
>> is the right way forward either, given that you only need to commit once
>> before get_or_create() is called to prevent the issue.
>>
>> Could you expand on why you feel every query would need to be committed?
>>
>
> get_or_create does:
>
> 1 - get
> 2 - if get fails due to DoesNotExist, create
> 3 - if create fails due to IntegrityError, get again
>
> The problem with repeatable read transaction level is that step 3 can
> never return something other than what was found (or not) in step 1. If
> some other thread, between the times at which 1 and 2 happen, creates what
> this thread is attempting to get_or_create, then the get_or_create is going
> to fail, because the get will still find nothing in step 3, causing
> get_or_create to re-raise the IntegrityError from step 2.
>
> Issuing a commit before calling get_or_create doesn't fix this race
> condition that exists within the get_or_create code itself. Such a pre-call
> commit is only going to be helpful if the current transaction has already
> done a read that has fixed what is going to be returned by the DB in step
> 1. If that has happened, then committing before calling get_or_create will
> reduce the likelihood of hitting the race condition, since step 1 may then
> read a row that has been created subsequent to the last read by this thread
> but before this thread gets to step 1 in get_or_create. Essentially the
> pre-commit narrows the race condition window in this case. (If no read has
> yet been done by the transaction to already set in stone what this
> transaction will read in step 1, then committing before calling
> get_or_create does not help at all.)
>
> Regardless of what's done before the call into get_or_create, there's
> still a small window of opportunity, between steps 1 and 2, for
> get_or_create to fail. get_or_create requires either DB level autocommit or
> read committed transaction isolation level in order for it to be free of
> this race condition. Recommending anything else in Django's docs would be
> misleading.
>
>
> Karen
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-developers?hl=en.
>



-- 

*Cal Leeming
Technical Support | Simplicity Media Ltd
**US *310-362-7070 | *UK *02476 100401 | *Direct *02476 100402

*Available 24 hours a day, 7 days a week.*

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: #18557: get_or_create() causes a race condition with MySQL

[Cal Leeming [Simplicity Media Ltd]]

I'm not entirely sure that suggesting every query needs to be committed is
the right way forward either, given that you only need to commit once
before get_or_create() is called to prevent the issue.

Could you expand on why you feel every query would need to be committed?

Cal

On Wed, Aug 8, 2012 at 1:29 PM, Karen Tracey  wrote:

> On Wed, Aug 8, 2012 at 8:13 AM, Cal Leeming [Simplicity Media Ltd] <
> cal.leem...@simplicitymedialtd.co.uk> wrote:
>
>> Does anyone else have any input at this stage??
>>
>> It seems to me that the most appropriate way forward is a documentation
>> update which explains that get_or_create() is not atomically safe across
>> all databases, and may need a commit() before hand, or a read isolation
>> change (with warnings about both).
>>
>>
> The docs should note that READ COMMITTED isolation level, or
> database-level autocommit of every query, is required for get_or_create()
> to work properly under MySQL. Recommending an alternative of a commit
> beforehand strikes me as wrong.
>
> Karen
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-developers?hl=en.
>



-- 

*Cal Leeming
Technical Support | Simplicity Media Ltd
**US *310-362-7070 | *UK *02476 100401 | *Direct *02476 100402

*Available 24 hours a day, 7 days a week.*

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: #18557: get_or_create() causes a race condition with MySQL

[Cal Leeming [Simplicity Media Ltd]]

Does anyone else have any input at this stage??

It seems to me that the most appropriate way forward is a documentation
update which explains that get_or_create() is not atomically safe across
all databases, and may need a commit() before hand, or a read isolation
change (with warnings about both).

On Mon, Jul 16, 2012 at 10:06 PM, Anssi Kääriäinen
wrote:

> On 16 heinä, 23:43, Ian Kelly  wrote:
> > On Mon, Jul 16, 2012 at 2:18 PM, Cal Leeming [Simplicity Media Ltd]
> >
> >  wrote:
> > > Okay - anyone else want to throw their thoughts at this?
> >
> > > Also - messing with the isolation levels on MySQL is really not a great
> > > idea.. it can cause all sorts of unexpected behavior.
> >
> > Just a thought -- I don't have MySQL handy to test on, but what if the
> > savepoint occurred before the first get instead of before the create?
> > Would rolling back the savepoint then release the row lock and allow
> > the second get to read the row?
> >
> > If this works but performance is an issue, then perhaps we could
> > dynamically order the operations according to the known isolation
> > level?
>
> Unfortunately this doesn't work. There is another ticket where this
> was discussed. At first it seems to work, but it turns out that it
> works only if the get_or_create is called as the first thing in the
> query. See https://code.djangoproject.com/ticket/13906#comment:29 and
> https://code.djangoproject.com/ticket/13906#comment:30 for details.
> Basically MySQL in repeatable read isolation works weird in this case.
>
>  - Anssi
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-developers?hl=en.
>
>


-- 

*Cal Leeming
Technical Support | Simplicity Media Ltd
**US *310-362-7070 | *UK *02476 100401 | *Direct *02476 100402

*Available 24 hours a day, 7 days a week.*

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: #18557: get_or_create() causes a race condition with MySQL

[Cal Leeming [Simplicity Media Ltd]]

Okay - anyone else want to throw their thoughts at this?

Also - messing with the isolation levels on MySQL is really not a great
idea.. it can cause all sorts of unexpected behavior.

Cal

On Mon, Jul 16, 2012 at 7:28 PM, Anssi Kääriäinen
wrote:

> On 16 heinä, 14:00, "Cal Leeming [Simplicity Media Ltd]"
>  wrote:
> > Hi guys,
> >
> > Okay so, this has been marked as wontfix in its current approach.
> >
> > The problem exists purely because of the way MySQL transactions and
> indexes
> > work - if you create a row that matches a unique index, it won't show up
> as
> > a row until you commit (which is correct), but if you try and insert
> > another row that matches this same unique index it will say the key
> already
> > exists (this is to prevent unique constraint race conditions within
> > transactions).
> >
> > This really isn't a bug in the code, but more that developers need to
> > understand how the database works, and take the appropriate actions (in
> > this case, it's that you need to commit() the transaction to guarantee
> safe
> > usage of get_or_create()).
> >
> > As aaugustin brought up, we can't automatically commit as this would
> break
> > transaction control, and there is little point in adding a 'commit=True'
> > argument onto get_or_create() because you could just do this yourself.
> >
> > Therefore, I'm proposing a documentation update that highlights and
> > explains this in a clear and simple way - rather than any sort of code
> > change. Would this be an appropriate way forward?
>
> I have tried different ways of solving this issue. To me it seems that
> unless one does a commit in get_or_create or changes the default
> isolation level used by MySQL then there is no cure for this issue.
> And this issue seems to pop out regularly. I don't the above mentioned
> approaches as good solutions.
>
> I would like to see the possibility to use different isolation levels
> if the user chooses so. This would be a good solution in my opinion:
> use READ COMMITTED isolation level if you are having problems with
> get_or_create. Allowing use of different isolation levels has other
> advantages, too. The use of custom isolation level should come with a
> big warning of "you should know what you are doing, or prepare for
> trouble".
>
> So, lets document the current behavior for now, and check if we can do
> something to the isolation level issue later on.
>
>  - Anssi
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-developers?hl=en.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

[Django] #18557: get_or_create() causes a race condition with MySQL

[Cal Leeming [Simplicity Media Ltd]]

Hi guys,

Okay so, this has been marked as wontfix in its current approach.

The problem exists purely because of the way MySQL transactions and indexes
work - if you create a row that matches a unique index, it won't show up as
a row until you commit (which is correct), but if you try and insert
another row that matches this same unique index it will say the key already
exists (this is to prevent unique constraint race conditions within
transactions).

This really isn't a bug in the code, but more that developers need to
understand how the database works, and take the appropriate actions (in
this case, it's that you need to commit() the transaction to guarantee safe
usage of get_or_create()).

As aaugustin brought up, we can't automatically commit as this would break
transaction control, and there is little point in adding a 'commit=True'
argument onto get_or_create() because you could just do this yourself.

Therefore, I'm proposing a documentation update that highlights and
explains this in a clear and simple way - rather than any sort of code
change. Would this be an appropriate way forward?

Cal

-- Forwarded message --
From: Django 
Date: Thu, Jul 12, 2012 at 10:25 PM
Subject: Re: [Django] #18557: get_or_create() causes a race condition with
MySQL
To:
Cc: django-upda...@googlegroups.com


#18557: get_or_create() causes a race condition with MySQL
-+-
 Reporter:  foxwhisper   |Owner:  nobody
 Type:  Uncategorized|   Status:  closed
Component:  Database layer   |  Version:  1.4
  (models, ORM)  |   Resolution:  wontfix
 Severity:  Normal   | Triage Stage:
 Keywords:   |  Unreviewed
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by aaugustin):

 * status:  reopened => closed
 * resolution:   => wontfix


Comment:

 Replying to [ticket:18557 foxwhisper]:
 > Is there any chance this patch would make it into the core?
 [[BR]]

 To be honest, as is, there is no chance for this code to make it into
 Django:
 - it isn't a patch, just a chunk of code,
 - it's specific to MySQL but it appears that it should go into
 django.db.models,
 - there's no explanation of why you're using this technique and why it
 works, and it isn't strikingly obvious either,
 - it messes transaction control,
 - no tests, no docs,
 - the code itself very far from Django's coding standards (`print`, `raise
 Exception`, etc.)

 Regarding transaction control, I see you've put the responsibility on the
 developer. It's hand-vawing, and I don't think we can put the issue aside
 like this. Transaction control is one of the more complicated parts of
 Django and I don't expect more than 0.1% of the framework's users to
 understand it. (I don't understand it completely myself.)

 I acknowledge that the problem exists, but this specific piece of code
 doesn't look like an appropriate solution, at least not in its current
 state.

 The way to move forward on a ticket that received a wontfix by a core
 developer is to bring up the issue on django-developers. Thanks!

--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Django-nonrel patches

[Cal Leeming [Simplicity Media Ltd]]

Hi Daniel,

Thanks for giving some feedback on this.

I completely agree that one of its biggest downfalls is that it tries to
treat MongoDB as a relational store, and I think this is what I meant by it
just didn't "feel right".

Also +1 on the comments made about it feeling hacky, and I suspect that if
Mongo ever made it into the Django core, it would be redesigned
and re-implemented from scratch.

I'd love to see Mongo in the Django core one day, but not as django-nonrel.

Cal

On Sun, Jun 24, 2012 at 5:47 AM, Daniel Greenfeld  wrote:

>
>
> We evaluated django-nonrel for use in projects and looked again at
> django-nonrel for our talk at DjangoCon Europe. To summarize our
> findings and opinions:
>
> 1. django-nonrel is stuck on Django 1.3, which has some security
> implications.
> 2. django-nonrel is unsupported. It switched maintainers and the
> current maintainer is not working on it.
> 3. [pydanny opinion warning] django-nonrel wasn't adopted in Django
> core because it lacked adequate documentation and tests.
> 4. [pydanny opinion warning] django-nonrel treats MongoDB as a
> relational store, which it most certainly is not.
> 5. [pydanny opinion warning] django-nonrel smells like a giant hack
> done by very well intentioned people..
>
> Hope that helps,
>
> --
> 'Knowledge is Power'
> Daniel Greenfeld
> http://pydanny.com
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-developers?hl=en.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Django-nonrel patches

[Cal Leeming [Simplicity Media Ltd]]

Hi Andres,

Afaik, there's currently some compatibility issues with Django 1.4 - so
it's not currently stable.

Also, in my own personal opinion - after having a chance to use the mongo
models with Django, in my personal opinion, it just didn't "feel right".
Not entirely sure how to explain what that means, but for whatever reason
the approach used just didn't feel like it was the best approach that could
be used.

Would be interesting to hear what conclusions other people came too,
especially any core devs involved in the ORM.

Cal

On Sat, Jun 23, 2012 at 5:30 AM, Andres Douglas wrote:

> Hey All,
>
> I've been trying to use django with mongo and it seems like django-nonrel
> is still the best option out there. The only sad fact is that it's still
> not part of the official django. Looking for an answer as to why that was
> the case, I came across this thread. Are tests and docs all that are
> missing? Can someone provide pointers to what might be expected and
> considered acceptable?
>
> It seems like a lot of the hard work has already been done figuring out
> what needed to change and the only thing left shouldn't be as hard -
> hopefully these are not famous last words.
>
> On Sunday, January 8, 2012 12:45:08 PM UTC-5, Jonas H. wrote:
>>
>> On Sun 08 Jan 2012 05:39:02 PM CET, Emil Stenström wrote:
>> >
>> >
>> > On Thursday, December 8, 2011 10:31:39 PM UTC+1, jo...@lophus.orgwrote:
>> >>
>> >> 1.) I didn't write the code, I'm just submitting the patches in their
>> >> current state
>> >>
>> > Jonas: Do you have information on who wrote the code that you
>> submitted? I
>> > guess a good way forward would be to find those people, and have them
>> > contribute a brief explanation of what problem their patch was trying to
>> > solve. For a "solution" to be accepted, you need to know the problem.
>> >
>>
>> Waldemar Kornewald wrote most of the code but he's pretty busy these
>> days.
>>
>  --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/django-developers/-/PmkY0RQA248J.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-developers?hl=en.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: GitHub migration done!

[Cal Leeming [Simplicity Media Ltd]]

Apologies if this question has already been answered or seems silly but -
is there a reason Mercurial is needed?? Can contributors not just switch to
using git?

i.e. if we have deprecated SVN, then why isn't Mercurial also being
deprecated??

Cal

On Wed, May 2, 2012 at 2:38 PM, Florian Apolloner wrote:

>
>
> On Wednesday, May 2, 2012 3:37:07 PM UTC+2, Florian Apolloner wrote:
>>
>> Far from easy, last time (or actually times) I tried to use it it broke
>> in many horrible ways :( [And either the link isn't listing an up2date
>> project or it's really dead since 3 years]
>>
>
> Ignore the dead status, I had to many github tabs open :/  -- it's
> apperently still maintained actively
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/django-developers/-/kaED8cf1TXMJ.
>
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-developers?hl=en.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: GitHub migration done!

[Cal Leeming [Simplicity Media Ltd]]

Amazing news!! Great job, Adrian.

Cal

On Sat, Apr 28, 2012 at 4:08 AM, Adrian Holovaty wrote:

> On Fri, Apr 27, 2012 at 11:50 AM, Adrian Holovaty 
> wrote:
> > We're going to do the migration to GitHub today. This means we'll no
> > longer be committing code to our Subversion repository. Committers,
> > please hold off on making commits until the migration is done.
>
> OK, it's live!
>
> https://github.com/django/django
>
> A few quick points (though this definitely deserves a more in-depth
> writeup):
>
> * I renamed the old (mirror) repository to
> https://github.com/django/django-old. We had talked about deleting it
> outright to avoid confusion, but I realized at the last minute that
> doing so would have deleted all the pull requests. I didn't want to
> throw all of that out, and I think we can figure out a way to use
> those pull requests in the new repository.
>
> * I only migrated trunk (now called "master"), rather than including
> all of the branches. This was the result of a bunch of discussion on
> IRC with Brian R., et al. The thinking is that it kept the migration a
> lot cleaner/simpler, and we can always add branches later. Of course,
> we'll need to create the latest release branches. Otherwise, we can
> consider the SVN branches on an individual basis.
>
> * As expected, all forks of the old repository are now broken. Can
> somebody volunteer to write some documentation on how to upgrade your
> old fork to use the new upstream repo (rebase? simple patch?)?
>
> * We're going to keep the Subversion repository around indefinitely,
> but it'll no longer be updated.
>
> * We're going to keep using Trac for tickets, but pull requests on
> GitHub are also welcome.
>
> * Clearly there are lots of bits of process that need to be updated
> now, from the django-updates mailing list to our "contributing"
> documentation, etc. We'll take care of all of that in the coming days,
> and we should all expect some degree of confusion and unsettlement in
> the community. That's totally fine, and we'll get through it. :-)
>
> * Finally, big thanks to the folks on IRC today who helped me through
> the process and contributed good ideas.
>
> I'm planning to write up a blog post on how the process went, for the
> benefit of the five open-source projects still using Subversion.
>
> Adrian
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-developers?hl=en.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Django is a serious framework, really

[Cal Leeming [Simplicity Media Ltd]]

Yo Jason, I'm really sorry for you, and imma let you finish, but Django is
one of the best frameworks of all time.

(I knew I'd get a chance to use that meme one day!)

On Wed, Apr 11, 2012 at 7:37 PM, Erik Stein  wrote:

>
> -- erik
>
> Am 11.04.2012 um 14:54 schrieb Russell Keith-Magee:
>
> > On Wednesday, 11 April 2012 at 8:10 PM, Jason Ma wrote:
>
> [snip]
>
>
> Erik Stein
> Programmierung, Grafik
>Oranienstr. 32   10999 Berlin, Germany
>fon +49 30 69201880   fax +49 30 692018809
>email e...@classlibrary.net
>
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-developers?hl=en.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: MySQL connection pooling - preferred method??

[Cal Leeming [Simplicity Media Ltd]]

On Fri, Feb 17, 2012 at 9:58 PM, Florian Apolloner wrote:

>
>
> On Friday, February 17, 2012 10:11:57 PM UTC+1, Cal Leeming [Simplicity
> Media Ltd] wrote:
>>
>> # Apparently this will stop many connections to MySQL
>> from django.core import signals
>> from django.db import close_connection
>> signals.request_finished.**disconnect(close_connection)
>>
>
> This approach has quite a few issues on it's own, eg for postgres if the
> transaction is broken all following requests will raise a 500. You have to
> at least reset the connection state to something useable again.
>

Could you elaborate on this a bit more? And would this affect MySQL?


>
> I'd love to see this as a 'settings.py' option, does anyone else think
>> this would be a good idea?? Something like 'persistent' : True.. maybe?
>>
>
> -1, we already have enough of them ;)
>

Hmm - what about a documentation update, so at least people in the future
don't have to go trawling through tons of mailing lists to find this.


>
> Cheers,
> Florian
>
>  --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/django-developers/-/pBSx93aPffIJ.
>
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-developers?hl=en.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: MySQL connection pooling - preferred method??

[Cal Leeming [Simplicity Media Ltd]]

Hi all,

Wwe actually put a patch into production about 2 weeks ago, which seems to
have reduced the connection count, whilst being stable and not having any
inconsistency problems.

# Apparently this will stop many connections to MySQL
from django.core import signals
from django.db import close_connection
signals.request_finished.disconnect(close_connection)

Although it's not connection pooling, it does stop the original problem of
lots of connections to the db.

I'd love to see this as a 'settings.py' option, does anyone else think this
would be a good idea?? Something like 'persistent' : True.. maybe?

Cal



On Sat, Jan 28, 2012 at 5:52 PM, Cal Leeming [Simplicity Media Ltd] <
cal.leem...@simplicitymedialtd.co.uk> wrote:

> Hi Russ,
>
> Thanks very much for the reply. I guess ultimately my question was "do any
> of the connection pooling solutions have an impact on the
> stability/performance of Django, or more importantly, cause any problems
> with the ORM?"
>
> I had very little luck in finding write ups by anyone on this, and it
> seems to be a question often asked.
>
> So I'm going to put time aside to test them all myself, then do a full
> write up about it - I'll reply back to the thread once done.
>
> Cheers
>
> Cal
>
>
> On Sat, Jan 28, 2012 at 2:15 AM, Russell Keith-Magee <
> russ...@keith-magee.com> wrote:
>
>> Hi Cal,
>>
>> I'm not exactly sure what it is you're looking for.
>>
>> The position of the core team has been fairly clear -- there are third
>> party connection pooling tools that handle connection pooling very
>> well.
>>
>> The recommendation of the core team is that you should use these tools.
>>
>> The alternative is to try an engineer a solution into Django's DB
>> connection stack. This solution would inevitably be less stable than
>> one that originates from a project whose sole purpose is implementing
>> a connection pool.
>>
>> If you're looking for a recommendation for a connection pooler for
>> MySQL, that's another matter. Unfortunately, I can't be much help
>> here; I don't keep on top of developments in the MySQL world, so I
>> can't comment with any authority.
>>
>> Yours,
>> Russ Magee %-)
>>
>> On Thu, Jan 26, 2012 at 5:01 AM, Cal Leeming [Simplicity Media Ltd]
>>  wrote:
>> > Damn - no thoughts on this from anyone?
>> >
>> >
>> > On Wed, Jan 25, 2012 at 12:11 AM, Cal Leeming [Simplicity Media Ltd]
>> >  wrote:
>> >>
>> >> Hi all,
>> >>
>> >> After spending about 30 minutes looking through old tickets, long
>> >> discussion threads and various blogs, I'm still not clear on the MySQL
>> >> connection pooling topic.
>> >>
>> >> To quote Russ: "the capability already exists in third party tools, and
>> >> they're in a position to do a much better job at it than us because
>> it's
>> >> their sole focus" [3]
>> >>
>> >> Could a core dev (or anyone else with experience on this) clarify which
>> >> approach is recommended, on the following conditions:
>> >>
>> >> * Safety (should not cause any strangeness with query cache or ORM)
>> >> * Performance (should avoid causing Django to open a new database
>> >> connection on every request)
>> >>
>> >> I found various ways to accomplish this, one of which was to use
>> >> SQLalchemy[1], another was to stop Django from closing the database
>> >> connection after each query[2].
>> >>
>> >> I'm hoping this thread will also serve as a final answer for anyone
>> else
>> >> looking for clarification.
>> >>
>> >> Many thanks
>> >>
>> >> Cal
>> >>
>> >>
>> >> [1]
>> http://menendez.com/blog/mysql-connection-pooling-django-and-sqlalchemy/
>> >>
>> >> [2]
>> http://stackoverflow.com/questions/1125504/django-persistent-database-connection
>> >>
>> >> [3]
>> http://groups.google.com/group/django-developers/browse_thread/thread/6f1e9c6e81aff1de/bf34e546e4217277?lnk=gst&q=mysql+pooling#bf34e546e4217277
>> >>
>> >>
>> >
>> > --
>> > You received this message because you are subscribed to the Google
>> Groups
>> > "Django developers" group.
>> > To post to this group, send email to django-developers@googlegroups.com
&g

Re: MySQL connection pooling - preferred method??

[Cal Leeming [Simplicity Media Ltd]]

Hi Russ,

Thanks very much for the reply. I guess ultimately my question was "do any
of the connection pooling solutions have an impact on the
stability/performance of Django, or more importantly, cause any problems
with the ORM?"

I had very little luck in finding write ups by anyone on this, and it seems
to be a question often asked.

So I'm going to put time aside to test them all myself, then do a full
write up about it - I'll reply back to the thread once done.

Cheers

Cal

On Sat, Jan 28, 2012 at 2:15 AM, Russell Keith-Magee <
russ...@keith-magee.com> wrote:

> Hi Cal,
>
> I'm not exactly sure what it is you're looking for.
>
> The position of the core team has been fairly clear -- there are third
> party connection pooling tools that handle connection pooling very
> well.
>
> The recommendation of the core team is that you should use these tools.
>
> The alternative is to try an engineer a solution into Django's DB
> connection stack. This solution would inevitably be less stable than
> one that originates from a project whose sole purpose is implementing
> a connection pool.
>
> If you're looking for a recommendation for a connection pooler for
> MySQL, that's another matter. Unfortunately, I can't be much help
> here; I don't keep on top of developments in the MySQL world, so I
> can't comment with any authority.
>
> Yours,
> Russ Magee %-)
>
> On Thu, Jan 26, 2012 at 5:01 AM, Cal Leeming [Simplicity Media Ltd]
>  wrote:
> > Damn - no thoughts on this from anyone?
> >
> >
> > On Wed, Jan 25, 2012 at 12:11 AM, Cal Leeming [Simplicity Media Ltd]
> >  wrote:
> >>
> >> Hi all,
> >>
> >> After spending about 30 minutes looking through old tickets, long
> >> discussion threads and various blogs, I'm still not clear on the MySQL
> >> connection pooling topic.
> >>
> >> To quote Russ: "the capability already exists in third party tools, and
> >> they're in a position to do a much better job at it than us because it's
> >> their sole focus" [3]
> >>
> >> Could a core dev (or anyone else with experience on this) clarify which
> >> approach is recommended, on the following conditions:
> >>
> >> * Safety (should not cause any strangeness with query cache or ORM)
> >> * Performance (should avoid causing Django to open a new database
> >> connection on every request)
> >>
> >> I found various ways to accomplish this, one of which was to use
> >> SQLalchemy[1], another was to stop Django from closing the database
> >> connection after each query[2].
> >>
> >> I'm hoping this thread will also serve as a final answer for anyone else
> >> looking for clarification.
> >>
> >> Many thanks
> >>
> >> Cal
> >>
> >>
> >> [1]
> http://menendez.com/blog/mysql-connection-pooling-django-and-sqlalchemy/
> >>
> >> [2]
> http://stackoverflow.com/questions/1125504/django-persistent-database-connection
> >>
> >> [3]
> http://groups.google.com/group/django-developers/browse_thread/thread/6f1e9c6e81aff1de/bf34e546e4217277?lnk=gst&q=mysql+pooling#bf34e546e4217277
> >>
> >>
> >
> > --
> > You received this message because you are subscribed to the Google Groups
> > "Django developers" group.
> > To post to this group, send email to django-developers@googlegroups.com.
> > To unsubscribe from this group, send email to
> > django-developers+unsubscr...@googlegroups.com.
> > For more options, visit this group at
> > http://groups.google.com/group/django-developers?hl=en.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-developers?hl=en.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: MySQL connection pooling - preferred method??

[Cal Leeming [Simplicity Media Ltd]]

Damn - no thoughts on this from anyone?

On Wed, Jan 25, 2012 at 12:11 AM, Cal Leeming [Simplicity Media Ltd] <
cal.leem...@simplicitymedialtd.co.uk> wrote:

> Hi all,
>
> After spending about 30 minutes looking through old tickets, long
> discussion threads and various blogs, I'm still not clear on the MySQL
> connection pooling topic.
>
> To quote Russ: "the capability already exists in third party tools, and
> they're in a position to do a much better job at it than us because it's
> their sole focus" [3]
>
> Could a core dev (or anyone else with experience on this) clarify which
> approach is recommended, on the following conditions:
>
> * Safety (should not cause any strangeness with query cache or ORM)
> * Performance (should avoid causing Django to open a new database
> connection on every request)
>
> I found various ways to accomplish this, one of which was to use
> SQLalchemy[1], another was to stop Django from closing the database
> connection after each query[2].
>
> I'm hoping this thread will also serve as a final answer for anyone else
> looking for clarification.
>
> Many thanks
>
> Cal
>
> [1]
> http://menendez.com/blog/mysql-connection-pooling-django-and-sqlalchemy/
> [2]
> http://stackoverflow.com/questions/1125504/django-persistent-database-connection
> [3]
> http://groups.google.com/group/django-developers/browse_thread/thread/6f1e9c6e81aff1de/bf34e546e4217277?lnk=gst&q=mysql+pooling#bf34e546e4217277
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

MySQL connection pooling - preferred method??

[Cal Leeming [Simplicity Media Ltd]]

Hi all,

After spending about 30 minutes looking through old tickets, long
discussion threads and various blogs, I'm still not clear on the MySQL
connection pooling topic.

To quote Russ: "the capability already exists in third party tools, and
they're in a position to do a much better job at it than us because it's
their sole focus" [3]

Could a core dev (or anyone else with experience on this) clarify which
approach is recommended, on the following conditions:

* Safety (should not cause any strangeness with query cache or ORM)
* Performance (should avoid causing Django to open a new database
connection on every request)

I found various ways to accomplish this, one of which was to use
SQLalchemy[1], another was to stop Django from closing the database
connection after each query[2].

I'm hoping this thread will also serve as a final answer for anyone else
looking for clarification.

Many thanks

Cal

[1] http://menendez.com/blog/mysql-connection-pooling-django-and-sqlalchemy/
[2]
http://stackoverflow.com/questions/1125504/django-persistent-database-connection
[3]
http://groups.google.com/group/django-developers/browse_thread/thread/6f1e9c6e81aff1de/bf34e546e4217277?lnk=gst&q=mysql+pooling#bf34e546e4217277

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: DoS using POST via hash algorithm collision

[Cal Leeming [Simplicity Media Ltd]]

Hi Luciano,

Curious, I was unaware of any such DoS vulnerability - makes for very
interesting reading.

Thanks for sharing this with the list - may be worth sending to
django-users as well.

Cal

On Thu, Dec 29, 2011 at 2:26 AM, Luciano Pacheco  wrote:

> Hi all,
>
> Have you guys seen this?
> http://www.ocert.org/advisories/ocert-2011-003.html
>
> PDF with some more explanation:
> http://www.nruns.com/_downloads/advisory28122011.pdf
>
> Regards,
> --
> Luciano Pacheco
> blog.lucmult.com.br
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-developers?hl=en.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Django error page - MemoryError

[Cal Leeming [Simplicity Media Ltd]]

Anyone else have any thoughts on if I should submit this for consideration
into the core?

Cal

On Tue, Dec 20, 2011 at 8:16 PM, Cal Leeming [Simplicity Media Ltd] <
cal.leem...@simplicitymedialtd.co.uk> wrote:

> Comments below, apologies for the email signature on previous emails!
>
> On Tue, Dec 20, 2011 at 7:56 PM, Paul McMillan  wrote:
>
>> > Place a try/catch for MemoryError on the exception handler to send back
>> a
>> > simple exception traceback to the browser.
>>
>> Yes, this makes sense, as long as we are sure the memory error is
>> raised by Django code, not user code.
>>
>
> Agreed, this would be wrapped around get_traceback_html() most likely.
>
>
>>
>> > Include a configuration settings option to limit the maximum payload it
>> will
>> > send back to the browser per variable (i.e. maybe 500kb per stack
>> frame, or
>> > 2kb per variable etc)
>>
>> I think we should select some reasonable limits for these, and
>> hardcode them, rather than adding a setting. Users who are debugging
>> the entire contents of multi-megabyte variable values on the html
>> debug page are doing it wrong.
>>
>
> I completely agree.
>
> I'm wondering if it would be better to have the variable commented out
> with a small message if it's above X amount, or whether it should display
> up to X amount, then display a small message explaining why it's been
> clipped.
>
>
>>
>> -Paul
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Django developers" group.
>> To post to this group, send email to django-developers@googlegroups.com.
>> To unsubscribe from this group, send email to
>> django-developers+unsubscr...@googlegroups.com.
>> For more options, visit this group at
>> http://groups.google.com/group/django-developers?hl=en.
>>
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Django error page - MemoryError

[Cal Leeming [Simplicity Media Ltd]]

Comments below, apologies for the email signature on previous emails!

On Tue, Dec 20, 2011 at 7:56 PM, Paul McMillan  wrote:

> > Place a try/catch for MemoryError on the exception handler to send back a
> > simple exception traceback to the browser.
>
> Yes, this makes sense, as long as we are sure the memory error is
> raised by Django code, not user code.
>

Agreed, this would be wrapped around get_traceback_html() most likely.


>
> > Include a configuration settings option to limit the maximum payload it
> will
> > send back to the browser per variable (i.e. maybe 500kb per stack frame,
> or
> > 2kb per variable etc)
>
> I think we should select some reasonable limits for these, and
> hardcode them, rather than adding a setting. Users who are debugging
> the entire contents of multi-megabyte variable values on the html
> debug page are doing it wrong.
>

I completely agree.

I'm wondering if it would be better to have the variable commented out with
a small message if it's above X amount, or whether it should display up to
X amount, then display a small message explaining why it's been clipped.


>
> -Paul
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-developers?hl=en.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Django error page - MemoryError

[Cal Leeming [Simplicity Media Ltd]]

Hi Alex,

Please note, I am already using a try/catch block on MemoryError, and this
does indeed resolve the problem.

I think at the very least, we should attempt to generate the text
exception, and if it fails due to a particular circumstance, then it will
just fall back to doing whatever it originally does.

I agree that it won't always 100% work (although I'm yet to see it fail in
my tests), but generating the text based exception has a fairly small
footprint, so it should work the majority of the time.

Cal

On Tue, Dec 20, 2011 at 8:01 PM, Alex Gaynor  wrote:

>
>
> On Tue, Dec 20, 2011 at 1:56 PM, Paul McMillan  wrote:
>
>> > Place a try/catch for MemoryError on the exception handler to send back
>> a
>> > simple exception traceback to the browser.
>>
>> Yes, this makes sense, as long as we are sure the memory error is
>> raised by Django code, not user code.
>>
>> > Include a configuration settings option to limit the maximum payload it
>> will
>> > send back to the browser per variable (i.e. maybe 500kb per stack
>> frame, or
>> > 2kb per variable etc)
>>
>> I think we should select some reasonable limits for these, and
>> hardcode them, rather than adding a setting. Users who are debugging
>> the entire contents of multi-megabyte variable values on the html
>> debug page are doing it wrong.
>>
>> -Paul
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Django developers" group.
>> To post to this group, send email to django-developers@googlegroups.com.
>> To unsubscribe from this group, send email to
>> django-developers+unsubscr...@googlegroups.com.
>> For more options, visit this group at
>> http://groups.google.com/group/django-developers?hl=en.
>>
>>
> Once you get a real memory error (i.e. one indicating you're out of
> memory, not just some operation wouldn't work "" * sys.maxint for
> example), doing *any* allocation might fail, so any limits you try to put
> down won't work in all circumstances.
>
> Alex
>
> --
> "I disapprove of what you say, but I will defend to the death your right
> to say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
> "The people's good is the highest law." -- Cicero
>
>  --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-developers?hl=en.
>



-- 

-- 

*Cal Leeming*
Technical Director | www.simplicitymedialtd.co.uk
Registered company number 7143564

Office: +44 (0)2476 980798 | Cell: +44 (0)7534 971120

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Django error page - MemoryError

[Cal Leeming [Simplicity Media Ltd]]

Hey,

So, we have a few clients who use Django for processing large amounts of
data in a single query.

If an exception is raised in development, the get_traceback_html() method
fails with a MemoryError, and in the event that it doesn't, you end up with
huge variable data print outs making the debug page somewhat tedious to use.

Temporarily, I've put a try/catch on MemoryError to generate a text based
email, which at least tells us what originally happened, albeit with the
useful variable information.

With all this in mind, I would like to see if anyone else thinks the
following might be a good idea


   - Place a try/catch for MemoryError on the exception handler to send
   back a simple exception traceback to the browser.

   - Include a configuration settings option to limit the maximum payload
   it will send back to the browser per variable (i.e. maybe 500kb per stack
   frame, or 2kb per variable etc)


I'd be happy to provide a patch for this, assuming it has any chance of
getting into the core.

Any thoughts/feedback would be appreciated.

Cal

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Small problem with HttpResponseRedirect

[Cal Leeming [Simplicity Media Ltd]]

After reading all the comments, I now completely agree that Django is doing
the right thing and it falls onto the user to do sanity checking.

Always helps to have another set of eyes, so thank you to everyone who took
time to post their thoughts!

Cal

On Tue, Dec 6, 2011 at 7:54 AM, Paul McMillan  wrote:

> As Ian said, Django does the right thing here according to my tests
> too, and generates the absolute URIs required by RFC 2616. If you've
> figured out some way to actually get location headers that are
> noncompliant, that would be a bug, but the handling of // is correct.
>
> -Paul
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-developers?hl=en.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Small problem with HttpResponseRedirect

[Cal Leeming [Simplicity Media Ltd]]

Not sure if this should have a bug ticket raised or not.. wanted to get
core devs thoughts.


_redir = "//your/path/with/an/extra/slash/for/whatever/reason"
HttpResponseRedirect(_redir)
returns "Location: http://your/path/with/an/extra/slash/for/whatever/reason";

_redir = "/your/path/with/no/extra/slash"
HttpResponseRedirect(_redir)
returns "Location: /your/path/with/no/extra/slash"

It seems that if the URL has an extra slash on the start of it, redirection
doesn't work properly.

Should this be classed as bad user sanity checking, or
HttpResponseRedirect() not being flexible enough?

I personally feel it's not being flexible enough, but would be good to hear
your thoughts.

Cal

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Django memcache key generation

[Cal Leeming [Simplicity Media Ltd]]

Thanks for the reply Russ.

I think ultimately it comes down to the fact that it was difficult to
figure out what was happening, where as the actual fix itself (as you
mentioned below) is a simple 2 or 3 line code change.

Now that I've thought about it more, I totally agree that my original
suggestion is not the right way forward.

However, I think what would be beneficial is a section within the
documentation that explicitly states that the key being specified, is not
the key that will end up in memcache. Something like:

"Be advised, the cached key will not be accessible from other non-django
webapps accessing memcache directly. This is due to the key being mangled
to include support for features such as 'version' and KEY_PREFIX. You can
either prepend the appropriate string yourself (explanation goes here), or
replace the method that generates they key to avoid it being mangled
(explanation goes here)".

If you'd be happy for this to be included in the docs, I'll go ahead and
submit a patch.

Cal


On Sat, Nov 19, 2011 at 10:44 AM, Russell Keith-Magee <
russ...@keith-magee.com> wrote:

> On Sat, Nov 19, 2011 at 4:08 PM, Cal Leeming [Simplicity Media Ltd]
>  wrote:
> > Hi,
> > Since the release of 1.3, Django has changed the way it generates the
> > memcache key name.
> > If I was to do:
> > cache.set('hello', 'world', 300)
> > It would actually store the result as ":1:hello". Obviously, this is
> because
> > the version number is stored, along with the key prefix.
> > But, this inherently breaks legacy applications which share memcache
> values
> > between other webapps (e.g. php/memcache-python direct etc). It was also
> > incredibly frustrating to try and work out why on earth Django was seeing
> > the key, and another application was not.
>
> The release notes mention the fact that there were changes to caching;
> it should perhaps be a little more explicit about the fact that keys
> would not be compatible between 1.2 and 1.3.
>
> [1] https://docs.djangoproject.com/en/dev/releases/1.3/#caching-changes
>
> > Therefore, I would like to recommend either one of two things (a -
> being my
> > personal preference):
> > a) If the version number is the 'default' and there is no KEY PREFIX,
> then
> > it won't attempt to prepend the key with ":1:", instead keeping it in its
> > original format.
>
> -1 from me. One simple behavior is better than a behavior that
> changes, even if those changes are predictable.
>
> > OR
> > b) Make it easier for users to disable this feature via config (I
> understand
> > there is an explanation of how to use 'make_key', but it's not a very
> > intuitive or friendly out-of-the-box approach)
>
> Can you give any suggestions on how to make this easier? At the
> moment, you need to:
>
>  1) Define a 2 line function
>
>  2) Put the path to that function in a setting.
>
> This process is documented, along with an example of a cache key function.
>
> I'm all for making things simpler when they can be, but in this case,
> I'm actually at a bit of a loss as to how this could be any simpler.
> Suggestions welcome.
>
> Yours,
> Russ Magee %-)
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-developers?hl=en.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: get_traceback_html vulnerable to infinite loop if called twice

[Cal Leeming [Simplicity Media Ltd]]

Hi Dennis,

I would hazard to say that ExceptionReporter/get_traceback_html() is not
public API (it's not listed anywhere in documentation) - and as such it
would be the user implementation of this which is the bug, not
the get_traceback_html method itself (i.e. you'd need to prevent this
condition from happening prior to get_traceback_html being called). I could
be wrong on this though!

On a related side note, I don't believe django-crashlog is a sane approach
to use, not only is it going to fill up your database with huge blobs of
data, but what happens if the INSERT fails or the database is unreachable?
I'd recommend switching to something like ReportBug() (
http://djangosnippets.org/snippets/2214/ ) - which is mail based, New Relic
(http://www.newrelic.com) or using a better approach to store the
tracebacks (maybe serialize to disk, use MongoDB etc). Also remember, that
when storing tracebacks, you need to make sure you implement DoS protection
into the Middleware, otherwise bad people can do bad things very easily
:) Any further discussion on that subject though should be moved over to
django-users.

Cal


On Thu, Nov 17, 2011 at 5:50 PM, dlukeparker wrote:

> Hi all,
>
> I would like to know if the behavior I am seeing is to be expected, or
> should be reported as a bug.
>
> I made a modification to the crash log middleware
> http://code.google.com/p/django-crashlog/
> to get it to save the lovely html error report that is available when
> settings.DEBUG == True.
>
> Sometimes, if settings.DEBUG is set to true and an exception happens
> my debug server process goes into a loop until memory is exhausted and
> it crashes. I have debugged the problem to the piece of code that
> loops-till-dead. My modified crashlog code collects the html traceback
> and persists it as expected, then the exception processing continues,
> eventually bubbling up to the point where the debug view is processed.
>
> In the get_traceback_html method of the ExceptionReporter class in
> django/views/debug.py the following lines are the location of the
> (presumably) infinite loop that occurs when the method is called for
> the second time for the same exception stack:
>
>frames = self.get_traceback_frames()
>for i, frame in enumerate(frames):
>if 'vars' in frame:
>frame['vars'] = [(k, force_escape(pprint(v))) for k, v
> in frame['vars']]
>frames[i] = frame
>
> I can and will eliminate the problem by omitting the debug html call
> if settings.DEBUG is true, but I would like to know, am I breaking
> some design constraint by causing this to be called twice, or should I
> report it as a bug?
>
> Thanks,
>
> dlp
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-developers?hl=en.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Django memcache key generation

[Cal Leeming [Simplicity Media Ltd]]

Hi,

Since the release of 1.3, Django has changed the way it generates the
memcache key name.

If I was to do:
cache.set('hello', 'world', 300)

It would actually store the result as ":1:hello". Obviously, this is
because the version number is stored, along with the key prefix.

But, this inherently breaks legacy applications which share memcache values
between other webapps (e.g. php/memcache-python direct etc). It was also
incredibly frustrating to try and work out why on earth Django was seeing
the key, and another application was not.

Therefore, I would like to recommend either one of two things (a - being my
personal preference):

a) If the version number is the 'default' and there is no KEY PREFIX, then
it won't attempt to prepend the key with ":1:", instead keeping it in its
original format.

OR

b) Make it easier for users to disable this feature via config (I
understand there is an explanation of how to use 'make_key', but it's not a
very intuitive or friendly out-of-the-box approach)

It'd be interesting to hear what the core devs have to say on this, and
anyone elses thoughts.

Cal

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Feature request prop for django db cache extention

[Cal Leeming [Simplicity Media Ltd]]

Hi Russell,

Thanks for the detailed response. It looks like both those solutions you
pasted have already dealt with this functionality - so thank you for
bringing these up.

Although it'd be really nice to see this in the core one day, is it worth me
pursuing to try and have something like this looked at again, or would it be
fighting a lost battle??

Cal

On Sat, Oct 15, 2011 at 3:52 AM, Russell Keith-Magee <
russ...@keith-magee.com> wrote:

> On Saturday, October 15, 2011, Cal Leeming [Simplicity Media Ltd] <
> cal.leem...@simplicitymedialtd.co.uk> wrote:
> > Hey all,
> > Today, we had a client getting around 600k webapp requests per hour (avg
> 6k/min), and had to do some emergency perf hotfixes (CodeIgniter+PHP)
> > In the end, we monkey-patched the code so raw SQL statement was used to
> generate a cache key, and we performed a lookup on that. It was absolutely
> great at preventing snowballing of things like tmp tables causing connection
> backlog - and resolved the problem completely.
> > Anyway, it got me thinking that it would be nice if Django could do this
> out of the box. Sure, the Django query cache works great, but when you have
> a large cluster of servers, and large queries causing tmp tables, something
> like this would be perfect.
> > Something like a DATABASE_CACHE='memcache://127.0.0.1:11211'
> > Does anyone have any thoughts about doing a feature request for this? If
> feedback is positive, I'll put in a feature request and put some time aside
> to write the patch.
>
> It sounds to me like you've just rediscovered Django ticket #17.
>
> We've made the decision that this sort of caching would fundamentally
> change the design contract of the ORM, and there isn't a single obvious way
> to solve the problem that we could provide as an optional behavior (cache
> setting is fine; it's cache invalidation where problems arise).
>
> However, the good news is that it can be handled as an external extension
> to the ORM. There are a number of external model caching libraries that
> address this problem; for example Johnny Cache and Cache Machine.
>
> Yours,
> Russ Magee %-)
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-developers?hl=en.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Feature request prop for django db cache extention

[Cal Leeming [Simplicity Media Ltd]]

Hey all,

Today, we had a client getting around 600k webapp requests per hour (avg
6k/min), and had to do some emergency perf hotfixes (CodeIgniter+PHP)

In the end, we monkey-patched the code so raw SQL statement was used to
generate a cache key, and we performed a lookup on that. It was absolutely
great at preventing snowballing of things like tmp tables causing connection
backlog - and resolved the problem completely.

Anyway, it got me thinking that it would be nice if Django could do this out
of the box. Sure, the Django query cache works great, but when you have a
large cluster of servers, and large queries causing tmp tables, something
like this would be perfect.

Something like a DATABASE_CACHE='memcache://127.0.0.1:11211'

Does anyone have any thoughts about doing a feature request for this? If
feedback is positive, I'll put in a feature request and put some time aside
to write the patch.

Cal

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: #10863 - Email full stack traces (feedback please!)

[Cal Leeming [Simplicity Media Ltd]]

That's pretty nice - something like this would be a nice additional feature.

Any thoughts core devs??

On Wed, Oct 5, 2011 at 8:26 PM, Patryk Zawadzki wrote:

> On Wed, Oct 5, 2011 at 12:40 PM, Cal Leeming [Simplicity Media Ltd]
>  wrote:
> > Could I bring the latest comment on the following ticket to the attention
> of
> > you guys:
> > https://code.djangoproject.com/ticket/10863#comment:17
> > Any feedback would be much appreciated.
>
> I'm not a core dev but since you're asking for feedback…
>
> Instead of emailing people with HTML I'd like to see a _useful_ stack
> trace in text format instead. I suggest you take a look at a debugging
> tool I wrote some time ago, it manages to get useful info without
> resorting to any rich formatting (in fact it only uses the logging
> library):
>
> https://github.com/patrys/great-justice
>
> --
> Patryk Zawadzki
> I solve problems.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-developers?hl=en.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

#10863 - Email full stack traces (feedback please!)

[Cal Leeming [Simplicity Media Ltd]]

Could I bring the latest comment on the following ticket to the attention of
you guys:

https://code.djangoproject.com/ticket/10863#comment:17

Any feedback would be much appreciated.

Thanks

Cal

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Curious (bug?) with db filter __in

[Cal Leeming [Simplicity Media Ltd]]

Nice - didn't know about that, will check that out too.

Thanks for the feedback on this everyone!

Cal

On Mon, Oct 3, 2011 at 8:15 PM, Ian Kelly  wrote:

> Have a look at the DatabaseOperations.max_in_list_size method, but it
> sounds like it's not going to be a simple constant for MySQL.
> On Oct 3, 2011 1:01 PM, "Cal Leeming [Simplicity Media Ltd]" <
> cal.leem...@simplicitymedialtd.co.uk> wrote:
> > Ahh - max_allowed_packet pops up its ugly head again - it
> > wouldn't surprise me if this was the case.
> >
> > Thank you for testing this against Postgre - I will post the test results
> > against MySQL tonight/tomorrow.
> >
> > Cal
> >
> > On Mon, Oct 3, 2011 at 7:58 PM, Jacob Kaplan-Moss  >wrote:
> >
> >> On Mon, Oct 3, 2011 at 1:31 PM, Cal Leeming [Simplicity Media Ltd]
> >>  wrote:
> >> > So, came up against a strange thing today.
> >> > Database backend is MySQL 5.5 (Percona variant)
> >> >
> >> > If I attempt to do an __in query with a list containing 50 thousand
> >> entries,
> >> > the query wouldn't fail, but would instead return no results. If I
> split
> >> it
> >> > up into chunks (say 100) - it would work fine.
> >> > For example:
> >> > _list = ['user1', 'user2'] # imagine this list has exactly 50 thousand
> >> > entries
> >> > Members.objects.filter(username__in = _list) # no results
> >> > Members.objects.filter(username__in = _list[:100]) # 100 results
> >> > I can provide exact further info, but this was just a preliminary
> email
> >> to
> >> > see if this was expected behavior - or actually a bug??
> >>
> >> I'm guessing it's MySQL being "special" again. I tried IN queries with
> >> 100k arguments on postgres and it worked fine. The MySQL documentation
> >> states that "The number of values in the IN list is only limited by
> >> the max_allowed_packet value."
> >> (
> >>
> http://dev.mysql.com/doc/refman/5.5/en/comparison-operators.html#function_in
> >> ),
> >> so perhaps investigate this max_allowed_packet setting?
> >>
> >> Jacob
> >>
> >> --
> >> You received this message because you are subscribed to the Google
> Groups
> >> "Django developers" group.
> >> To post to this group, send email to django-developers@googlegroups.com
> .
> >> To unsubscribe from this group, send email to
> >> django-developers+unsubscr...@googlegroups.com.
> >> For more options, visit this group at
> >> http://groups.google.com/group/django-developers?hl=en.
> >>
> >>
> >
> > --
> > You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> > To post to this group, send email to django-developers@googlegroups.com.
> > To unsubscribe from this group, send email to
> django-developers+unsubscr...@googlegroups.com.
> > For more options, visit this group at
> http://groups.google.com/group/django-developers?hl=en.
> >
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-developers?hl=en.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Curious (bug?) with db filter __in

[Cal Leeming [Simplicity Media Ltd]]

Ahh - max_allowed_packet pops up its ugly head again - it
wouldn't surprise me if this was the case.

Thank you for testing this against Postgre - I will post the test results
against MySQL tonight/tomorrow.

Cal

On Mon, Oct 3, 2011 at 7:58 PM, Jacob Kaplan-Moss wrote:

> On Mon, Oct 3, 2011 at 1:31 PM, Cal Leeming [Simplicity Media Ltd]
>  wrote:
> > So, came up against a strange thing today.
> > Database backend is MySQL 5.5 (Percona variant)
> >
> > If I attempt to do an __in query with a list containing 50 thousand
> entries,
> > the query wouldn't fail, but would instead return no results. If I split
> it
> > up into chunks (say 100) - it would work fine.
> > For example:
> > _list = ['user1', 'user2'] # imagine this list has exactly 50 thousand
> > entries
> > Members.objects.filter(username__in = _list) # no results
> > Members.objects.filter(username__in = _list[:100]) # 100 results
> > I can provide exact further info, but this was just a preliminary email
> to
> > see if this was expected behavior - or actually a bug??
>
> I'm guessing it's MySQL being "special" again. I tried IN queries with
> 100k arguments on postgres and it worked fine. The MySQL documentation
> states that "The number of values in the IN list is only limited by
> the max_allowed_packet value."
> (
> http://dev.mysql.com/doc/refman/5.5/en/comparison-operators.html#function_in
> ),
> so perhaps investigate this max_allowed_packet setting?
>
> Jacob
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-developers?hl=en.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Curious (bug?) with db filter __in

[Cal Leeming [Simplicity Media Ltd]]

Ja - that was going to be my next step - bit stretched for time at the
moment - so was just a quick email to see if anyone had any thoughts.

I think even if it was a problem with MySQL - an error should have been
raised somewhere.

Either way - I'll build a test for it and let you know the results.

Thx

Cal

On Mon, Oct 3, 2011 at 7:52 PM, Christophe Pettus  wrote:

>
> On Oct 3, 2011, at 11:31 AM, Cal Leeming [Simplicity Media Ltd] wrote:
>
> > I can provide exact further info, but this was just a preliminary email
> to see if this was expected behavior - or actually a bug??
>
> You might try capturing the generated SQL and running it on a command line
> against the DB to see if the problem is in MySQL, or in the Django backend.
>
> --
> -- Christophe Pettus
>   x...@thebuild.com
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-developers?hl=en.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Curious (bug?) with db filter __in

[Cal Leeming [Simplicity Media Ltd]]

So, came up against a strange thing today.

Database backend is MySQL 5.5 (Percona variant)

If I attempt to do an __in query with a list containing 50 thousand entries,
the query wouldn't fail, but would instead return no results. If I split it
up into chunks (say 100) - it would work fine.

For example:
_list = ['user1', 'user2'] # imagine this list has exactly 50 thousand
entries
Members.objects.filter(username__in = _list) # no results
Members.objects.filter(username__in = _list[:100]) # 100 results

I can provide exact further info, but this was just a preliminary email to
see if this was expected behavior - or actually a bug??

Thanks

Cal

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Revisiting proxied SSL headers

[Cal Leeming [Simplicity Media Ltd]]

On Mon, Sep 26, 2011 at 5:39 PM, Carl Meyer  wrote:

> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> On 09/26/2011 06:16 AM, Cal Leeming [Simplicity Media Ltd] wrote:
> > Unless you can guarantee that all web application servers/load balancers
> > are going to correctly handle the header out of the box (i.e.
> > inject/strip where necessary), then there's no way this could be
> > "securely" introduced.
>
> The proposal is not to do anything automatically or by default, but to
> require the user to explicitly set what header to look for, with
> documentation warning them that they should only do this if they know
> that the header is always set by their proxy.
>

Ah - in that case (is it has to be configured by the user first, and doesn't
take any assumptions) then that would be much better). I thought you meant a
patch to make this "just work - out of the box".


>
> I agree with Luke and Paul that support for this should be in core - as
> they've discussed, the status quo is in itself a security problem. And
> while fixing it with a custom middleware is certainly possible, it
> requires monkey-patching the request.is_secure() method. Anytime we have
> to tell a sizable percentage of our users "you really should add this
> custom code to your project that monkeypatches Django core" it's a
> pretty strong signal that core should provide better hooks instead.
>
> > The reason I say per case basis, is because we've had to implement this
> > same middleware ourselves into multiple clients, all of which had to be
> > slightly different due to the handling of the SSL header at the load
> > balancer.
>
> Can you be more specific here? The only implementation differences that
> I can imagine would be which header is checked, and which value is
> expected to indicate SSL vs non-SSL. The proposed hook (currently
> implemented in django-secure) already accounts for this variation by
> requiring the user to explicitly specify the header and value that
> should indicate an SSL request.
>

Yeah - the thing you mentioned above about forcing the user to configure the
exact header tackles this point.

The problem I had previously, was that some load balancers don't forcibly
send or strip certain SSL offloading headers (such as X-Forwarded-Proto),
which could result in the user being able to trick the web application into
thinking it's in HTTPS, when it's not. But this isn't an issue if it's
"configured on demand".


>
> Carl
> -BEGIN PGP SIGNATURE-
> Version: GnuPG v1.4.10 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>
> iEYEARECAAYFAk6AqrMACgkQ8W4rlRKtE2dxmwCfRf+gXWLvm/Bup6r0ySJ0qVVU
> luMAmwd5XT98WU/kIWLhbUy8NA3aPy7K
> =AS+A
> -END PGP SIGNATURE-
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-developers?hl=en.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: get_field_display: Django needs a template filter to allow get_field_display for a dynamic field name

[Cal Leeming [Simplicity Media Ltd]]

Ivan,

I completely agree that it would be useful to have something like this, as I
have some up against this *exact* same problem in the past.

However, when I raised it as an issue on IRC - the only response I got was
"stop putting your application logic into the templates" lol.

So, although I'm +1 on this, I suspect there may be others who would
strongly disagree with having it as part of the core.

Cal

On Mon, Sep 26, 2011 at 1:01 PM, Ivan Kharlamov wrote:

>
>
> There is often a need to write model-independent Django templates. By
> now, the only field name-dependent attribute in Django templates is
> get_field_display, because the particular field names must be
> hardcoded for each model.
>
> What I propose is a universal filter which should provide human-
> readable value for a field of any type.
>
> For example, like this:
>
> 
>
>{% for item in query %}
>
>
>
>{% for field in fields %}
>
>{{item|human_readable:field}}
>
>{% endfor %}
>
>
>
>{% endfor %}
>
> 
>
> I'm talking about something like this, but secure against exposing
> _meta attributes:
>
> def human_readable(value, arg):
>if hasattr(value, 'get_' + str(arg) + '_display'):
>return getattr(value, 'get_%s_display' % arg)()
>elif hasattr(value, str(arg)):
>if callable(getattr(value, str(arg))):
>return getattr(value, arg)()
>else:
>return getattr(value, arg)
>else:
>try:
>return value[arg]
>except KeyError:
>return settings.TEMPLATE_STRING_IF_INVALID
>
> Here's the corresponding ticket:
> https://code.djangoproject.com/ticket/16034
>
> Best regards,
> Ivan Kharlamov
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-developers?hl=en.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Revisiting proxied SSL headers

[Cal Leeming [Simplicity Media Ltd]]

Just my two cents worth, but I think something like this is such a 'per case
basis', that it probably shouldn't be included in the core.

Unless you can guarantee that all web application servers/load balancers are
going to correctly handle the header out of the box (i.e. inject/strip where
necessary), then there's no way this could be "securely" introduced.

The reason I say per case basis, is because we've had to implement this same
middleware ourselves into multiple clients, all of which had to be slightly
different due to the handling of the SSL header at the load balancer.

Cal

On Mon, Sep 26, 2011 at 1:02 PM, Luke Plant  wrote:

> On 26/09/11 12:45, Tom Evans wrote:
> > On Sat, Sep 24, 2011 at 9:28 PM, Luke Plant 
> wrote:
> >>
> >> I'm happy to be proved wrong, of course. Apache is very popular, though,
> >> so if its hard in Apache, it could be said to be hard full stop.
> >>
> >
> >   RequestHeader unset X-Forwarded-Protocol
> >
> > Not precisely what I'd call hard.
>
> I am indeed happy to have been proved wrong :-) ... if slightly
> embarrassed...
>
> I suppose we should check that this definitely works in conjunction with
> mod_proxy and whichever module it is that sets X-Forwarded-Protocol/Ssl.
>
> > I suppose it is analogous to DB routers. Django doesn't provide
> > routers to handle the common ways to scale a database, but they are
> > simple enough to write for your specific setup. There is a simple way
> > to add your own fixups to requests, and it works, so we don't need to
> > burden the core or contrib with it.
>
> Given the security problems of getting HttpRequest.is_secure() wrong
> either way, and the common solution to this particular problem, I think
> it is better to have support in the core for this.
>
> Luke
>
> --
> "I regret I wasn't born with opposable toes." (Calvin and Hobbes)
>
> Luke Plant || http://lukeplant.me.uk/
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-developers?hl=en.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Python 3 and you

[Cal Leeming [Simplicity Media Ltd]]

Can I ask, have the django core team already accepted that Django will
eventually be a 3.x framework, or will it be un-officially forked?

Personally - I'd love to see people ride the 2.x train until its last dying
breath, but that's just me ;)

Cal

2011/9/14 Ákos Péter Horváth 

> Another vote to python3 :-)
>
> Really, I started to port that with a recursive 2to3. It is not too far
> from good working. There are no big magic things, altough I think a py2 and
> py3 support isn't possible from a common source tree. Some deep core
> improvement is needed too, mostly on the unicode line, but nothing really
> hard.
>
> Anybody knows somebody who started a django/py3 port already? We should
> unify our efforts.
>
> If core team doesn't like the py3 idea, we should start with a patch which
> follows always the latest devel version. I strongly anticipate a project
> fork (see what happened with the plone & bluebream).
>
> 2to3 (3to2), etc are good start for an initial port, but I think they
> aren't enough to do this on day-to-day base. Their conversion algo isn't
> enough, some handwork is needed.
>
> thank you
>
> MaXX
>
>
> On Wed, Sep 14, 2011 at 5:55 PM, Daniel Lindsley wrote:
>
>> Jannis,
>>
>>
>>   I wasn't trying to suggest we leave anyone behind, far from it. I
>> was suggesting move the code to Python 3 now, while there's less code
>> there (than some future date) but using 3to2[1] to help others on
>> Python 2.X. Since Django still supports 2.5, it's possible that this
>> isn't even an option, as I don't know if 3to2 can translate back that
>> far reliably. Simply getting the question out there for others to mull
>> over.
>>
>>
>> Daniel
>>
>>
>>
>> On Sep 14, 10:36 am, Jannis Leidel  wrote:
>> > Daniel,
>> >
>> > >   "You have my sword." I want to see this happen & would love to be a
>> > > part of it.
>> >
>> > Huzzah!
>> >
>> > > A couple questions:
>> >
>> > > * How should patches be provided? Trac? BitBucket?
>> >
>> > For now via Trac, that's why we've moved the changes into a SVN branch.
>> > Unless anyone has a better idea I could create a Trac component "Python
>> 3"
>> > so we can track the tickets easily.
>> >
>> > > * Where should feedback go? This mailing list? Somewhere else?
>> >
>> > Feedback should go here, on the developers mailing list, to get as many
>> > eyes on it as possible.
>> >
>> > > * This is further off, but once we have a ported Django, how do get
>> > > the community (specifically pluggable apps) onboard? I'm assuming the
>> > > docs are meant to do this but wondering if there's anything else we
>> > > can be doing (like perhaps a Django-specific 2to3 (extension?) to
>> > > cover common Django conventions).
>> >
>> > Very good question, I'm uncertain as to how the "helpers" I mentioned
>> > will look like in the end. Whether they will be part of Django (e.g.
>> > a management command to run 2to3 on an app) or if we "only" provide the
>> > necessary compatibility library (e.g. "six") so that 3rd party app
>> > authors would still keep writing apps with Python 2 but would allow
>> > their apps to be translated to Python 3 automatically. Documenting ways
>> > of how to write a setup.py to do the conversion during install time
>> > is *in* the scope of what we need to provide, IMO. Whether we need
>> > Django-specific 2to3 fixers isn't clear at this time as the porting
>> > has only just begun.
>> >
>> > > * Do we have a target date? I know this is hard with a volunteer-only
>> > > effort, but if we setup some sort of timeline, we'd at least have a
>> > > metric & something to shoot/push for.
>> >
>> > One assumption of the strategy I outlined was the fact that Django is
>> > as close to 3.X as possible. Django 1.4 will require Python 2.5 or
>> > higher, but I'm not sure how quick we can do the jump to 2.6, which
>> > is recommended by the Python porting docs [1].
>> >
>> > >   Finally, a philosophical question on approach: Should we really be
>> > > doing 2to3 (leaving the Django codebase in Python 2.X for a long time)
>> > > or would it be better to port Django over to Python 3 & use 3to2 for
>> > > existing Python 2.X installs? I confess I don't know much about the
>> > > current state of 3to2 (nor how most other Python libraries are
>> > > handling the transition). But I do know Django will continue to grow
>> > > over time & I worry that, at some point in the future we'll be making
>> > > more even more work for someone else to do the 3-only work.
>> >
>> > I personally haven't ported a 2.X library completely to 3.X yet, so I
>> > can also only guess. But from what I've seen in the community I'm afraid
>> > of a "clean cut" port because it has a high risk of leaving many
>> projects
>> > and apps behind. In that sense it seems more sensible to me to see the
>> > port to Python 3 just as another step of our Python version deprecation
>> > policy, which we at some point take with a complete conversion.
>> Basically
>> > a "burn bridges as soon as everyone is safe" approa

Re: please reopen ticket 15567

[Cal Leeming [Simplicity Media Ltd]]

+1, if the user/pass is entered, that user is entitled so know what its own
permissions are.

The error should give "You have insufficient access to this page" or
something like that.

Cal

On Tue, Sep 13, 2011 at 6:12 PM, Florian Apolloner wrote:

> -1, This would leak information about the users (But I am sure that's
> discussed at length in the other threads)
>
>  --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/django-developers/-/5iy7pazGNGkJ.
>
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-developers?hl=en.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Proposal for a new templatetag definition syntax

[Cal Leeming [Simplicity Media Ltd]]

I may have misunderstood this thread slightly but, should templatetags
follow the same structure as *args, **kwargs??

i.e.

*args
{% create_title "hello world" 800 800 %}

**kwargs
{% create_title title="helloworld" width=800 height=800 %}

mixture
{% create_title "helloworld" width=800 height=800 %}

Apologies if I have completely missed the point of this thread.

Cal

On Mon, Sep 12, 2011 at 10:39 AM, Jonas H.  wrote:

> On 09/12/2011 10:47 AM, lucky wrote:
>
>> Hello, Alex,
>>
>> I agree with Wim Feijen. The ability to implement a arbitrary syntax
>> for tag parameters is too abstract problem and this causes an increase
>> in the complexity of the solution. Tag delelopers are forced to define
>> a grammar and deal with lexing and parsing. End users must to learn
>> and remember a custom syntax for each of tags. [...]
>>
>>
>> We should not be afraid of the python-like grammar for tag
>> paramenters. There is not principal difference beetween custom {%
>> mytag src limit 1 offset 10 %} and pythonic {% mytag src limit=1
>> offset=10 %}. The restrict syntax makes the solution simpler.
>>
>
> +1
>
> Template tags should be forced to have a simple syntax. Users should not
> have to look up the syntax each time.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To post to this group, send email to 
> django-developers@**googlegroups.com
> .
> To unsubscribe from this group, send email to
> django-developers+unsubscribe@**googlegroups.com
> .
> For more options, visit this group at http://groups.google.com/**
> group/django-developers?hl=en
> .
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: runfcgi umask unusable without daemonize=false.. why?

[Cal Leeming [Simplicity Media Ltd]]

Hey Roberto,

Yeah I wanted to use uWSGI for this client, but the fastcgi.py file had been
originally monkey patched to integrate with the client application (yeah, a
total pain in the ass), which meant we *had* to use the manage.py runfcgi
stuff - otherwise it meant re-writing a chunk of stuff. :/

Cal

On Fri, Sep 9, 2011 at 3:24 PM, Roberto De Ioris  wrote:

>
> Il giorno 09/set/2011, alle ore 16:05, Cal Leeming [Simplicity Media Ltd]
> ha scritto:
>
> > Hi,
> >
> > Is there any reason why umask is unusable without daemonize=false?
> >
> > This means I can't manage processes within supervisorctl (when having to
> use fastcgi due to client not being able to have uwsgi).
>
> Not related to your specific question, but remember that uWSGI can natively
> speaks fastcgi (both in static and dynamic way)
>
> static config:
> uwsgi --fastcgi-socket  ...
>
> dynamic (apache passing the socket as fd 0)
> uwsgi --protocol=fastcgi ...
>
>
> --
> Roberto De Ioris
> http://unbit.it
> JID: robe...@jabber.unbit.it
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-developers?hl=en.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

runfcgi umask unusable without daemonize=false.. why?

[Cal Leeming [Simplicity Media Ltd]]

Hi,

Is there any reason why umask is unusable without daemonize=false?

This means I can't manage processes within supervisorctl (when having to use
fastcgi due to client not being able to have uwsgi).

For example:

srwxr-xr-x 1 tt tt  0 Sep  9 15:00 fastcgi

That means that nginx (www-data) is unable to write to the socket.

Same discussion here:
http://stackoverflow.com/questions/3431029/socket-permissions-when-running-django-with-fastcgi

Cal

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Possible documentation (or code) bug related to DEBUG+INTERNAL_IPS

[Cal Leeming [Simplicity Media Ltd]]

Hi,

Documentation states that IPs within 'INTERNAL_IPS' will:
 - See debug comments, when DEBUG is True.

To my eyes, that means that unless the IP is in there, they won't see the
debug comments/page.

However, the debug page is still shown to the user when DEBUG is on, even if
they are not in the internal_ips.

Either this is a bug in the code (this is bad), or the documentation needs
to be amended to be clearer (unless I have misread / skipped something??).

Cal

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: PHP-inspired user-friendly in-browser DJango install

[Cal Leeming [Simplicity Media Ltd]]

On Tue, Sep 6, 2011 at 6:29 PM, Alec Taylor  wrote:

> Good morning,
>
> I have been using Drupal for a while now, but am slowly moving toward
> DJango.
>
> I am currently working on quite an ambitious project, which I will be
> releasing under a permissive license (New BSD or better).
>
> I would like my project to be as accessible as possible. To this end,
> I would like people without any Python or DJango knowledge to be able
> to quickly setup my project.
>

Uh.. what? Django != Drupal.

Can you please explain a bit more about what you want the end result to be.

Thanks


>
> Here is the web-interface functionality I would like to present to the
> user:
>
> 
>
> 
> 
> 
> 
> 
> 
> [Install]
>
> Can DJango be made do work this way? — If so, do you have some example
> code or an existing project which does this?
>
> Thanks for all suggestions,
>
> Alec Taylor
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-developers?hl=en.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: What is Django?

[Cal Leeming [Simplicity Media Ltd]]

On Wed, Aug 24, 2011 at 11:04 AM, Tom Evans wrote:

> On Tue, Aug 23, 2011 at 6:34 PM, Cal Leeming [Simplicity Media Ltd]
>  wrote:
> >
> > +1 on this idea :)
>
> I don't think Russell is looking for votes on whether to do it, he's
> looking for someone to actually do it :)
>

Lol, yeah I guessed that already. *adds into todo list*


>
> Cheers
>
> Tom
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-developers?hl=en.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: What is Django?

[Cal Leeming [Simplicity Media Ltd]]

On Tue, Aug 23, 2011 at 4:54 PM, Russell Keith-Magee <
russ...@keith-magee.com> wrote:

> Hi Julian,
>
> If your intention here was to suggest something that you think
> Django's website should cover, then this *is* the right forum.
> However, my reponse would be to ask what is wrong with the overview
> that is already there? The front page of Django's website gives a 1
> paragraph summary, with a link to a longer overview page:
>
> https://docs.djangoproject.com/en/1.3/intro/overview/
>
> We've also kicked around the idea of writing up some "Django for
> non-techs" guides -- the sort of thing you might give to a manager or
> boss to convince them that Django is worth adopting. We don't have
> anything to show for this, but if you (or anyone else) is interested
> in volunteering and has skills in copywrighting and design, this could
> be a valuable contribution.
>

+1 on this idea :) Maybe some pages which show the amount of code needed in
other languages, vs Django. Or the amount of steps necessary to perform a
task, vs Django. In clear, easy to see format.


>
> Yours,
> Russ Magee %-)
>
> On Tue, Aug 23, 2011 at 7:55 PM, Karen Tracey  wrote:
> > Please ask general questions about Django on django-users. The topic of
> this
> > list is the development of Django itself.
> >
> > Karen
> >
> > --
> > You received this message because you are subscribed to the Google Groups
> > "Django developers" group.
> > To post to this group, send email to django-developers@googlegroups.com.
> > To unsubscribe from this group, send email to
> > django-developers+unsubscr...@googlegroups.com.
> > For more options, visit this group at
> > http://groups.google.com/group/django-developers?hl=en.
> >
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-developers?hl=en.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: django-irc-logs.com

[Cal Leeming [Simplicity Media Ltd]]

On Mon, Jul 25, 2011 at 1:31 PM, Jannis Leidel  wrote:

> On 25.07.2011, at 13:25, Bernhard Essl wrote:
>
> > Dear django devs,
> >
> > unfortunately the site where the #django IRC logs used to be archived
> > has been down for quite a while. I've tried to find out what happened
> > to that site but failed.
> > Because I think the IRC logs are an important source of information
> > for all django users I re-built the site.
> > I bought webspace and a domain for two years, too (
> > http://django-irc-logs.com/ ). The layout isn't very sexy but I just
> > thought it would be a good thing if the content was online anyway. The
> > site's basic features are searching and browsing the logs.
> >
> > I hope you like my weekend project. I'd appreciate your ideas and
> comments.
>
> Hi Bernd,
>
> thanks for your efforts, in fact I've been working on a new edition of
> the previous IRC logger myself and would love to join forces here. I got
> the database dumps of the old logger from Brian


Excellent, this was going to be my next question.

If I can be of any assistance for performance tuning or advice on db/se,
then feel free to give me a shout.

, so we should be able to
> get something more complete online. I suggest we put the site on
> djangoproject.com, too. I'll try to clean up my code this week and put it
> online.
>
> Jannis
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-developers?hl=en.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: django-irc-logs.com

[Cal Leeming [Simplicity Media Ltd]]

Also, on a side note, it may be that django-developers isn't the right place
for this discussion.

Might be a good idea to do a post on django-users also, as I'd imagine that
list would benefit more from this information!

Cal

On Mon, Jul 25, 2011 at 1:21 PM, Cal Leeming [Simplicity Media Ltd] <
cal.leem...@simplicitymedialtd.co.uk> wrote:

> Cute :)
>
> A few questions:
>
>- What back end do you use for the search engine?
>
>- Have you tested this system once it contains 5+ million rows? (you
>might be interested to register for the webcast on handling large amounts 
> of
>data in Django -
>
> https://spreadsheets.google.com/a/foxwhisper.co.uk/spreadsheet/viewform?hl=en_US&formkey=dENPX3BMUUFMbi1CbElwV3BvOEdkNmc6MQ#gid=0
> )
>
>- I like the hover over colors for the individual nicknames :) It'd be
>nice if you could click on a line of text from a user, and it highlights 
> all
>the other messages for that user on the page.
>
>- Can you make it highlight the searched 'keyword' in the results (
>http://django-irc-logs.com/search/?q=hello )
>
>
> Haven't seen the code but end result is great for weekend project though,
> good work! :)
>
> Cal
>
>
>
> On Mon, Jul 25, 2011 at 12:25 PM, Bernhard Essl wrote:
>
>> Dear django devs,
>>
>> unfortunately the site where the #django IRC logs used to be archived
>> has been down for quite a while. I've tried to find out what happened
>> to that site but failed.
>> Because I think the IRC logs are an important source of information
>> for all django users I re-built the site.
>> I bought webspace and a domain for two years, too (
>> http://django-irc-logs.com/ ). The layout isn't very sexy but I just
>> thought it would be a good thing if the content was online anyway. The
>> site's basic features are searching and browsing the logs.
>>
>> I hope you like my weekend project. I'd appreciate your ideas and
>> comments.
>>
>> Yours, Bernd
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Django developers" group.
>> To post to this group, send email to django-developers@googlegroups.com.
>> To unsubscribe from this group, send email to
>> django-developers+unsubscr...@googlegroups.com.
>> For more options, visit this group at
>> http://groups.google.com/group/django-developers?hl=en.
>>
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: django-irc-logs.com

[Cal Leeming [Simplicity Media Ltd]]

Cute :)

A few questions:

   - What back end do you use for the search engine?

   - Have you tested this system once it contains 5+ million rows? (you
   might be interested to register for the webcast on handling large amounts of
   data in Django -
   
https://spreadsheets.google.com/a/foxwhisper.co.uk/spreadsheet/viewform?hl=en_US&formkey=dENPX3BMUUFMbi1CbElwV3BvOEdkNmc6MQ#gid=0
)

   - I like the hover over colors for the individual nicknames :) It'd be
   nice if you could click on a line of text from a user, and it highlights all
   the other messages for that user on the page.

   - Can you make it highlight the searched 'keyword' in the results (
   http://django-irc-logs.com/search/?q=hello )


Haven't seen the code but end result is great for weekend project though,
good work! :)

Cal



On Mon, Jul 25, 2011 at 12:25 PM, Bernhard Essl wrote:

> Dear django devs,
>
> unfortunately the site where the #django IRC logs used to be archived
> has been down for quite a while. I've tried to find out what happened
> to that site but failed.
> Because I think the IRC logs are an important source of information
> for all django users I re-built the site.
> I bought webspace and a domain for two years, too (
> http://django-irc-logs.com/ ). The layout isn't very sexy but I just
> thought it would be a good thing if the content was online anyway. The
> site's basic features are searching and browsing the logs.
>
> I hope you like my weekend project. I'd appreciate your ideas and comments.
>
> Yours, Bernd
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-developers?hl=en.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Storing IP address as integer within database to remove need for full text search

[Cal Leeming [Simplicity Media Ltd]]

On Tue, Jul 19, 2011 at 1:02 PM, Daniel Swarbrick <
daniel.swarbr...@gmail.com> wrote:

> The snippet seems to have been removed (returns "page not found").


Wtf? :X

http://djangosnippets.org/snippets/

Looks like they have suffered some sort of data loss.. I'm seeing only
snippets from 2 weeks ago, then 1 from an hour ago lol.

I've reposted:

http://djangosnippets.org/snippets/2493/

And as luck would have it, it's given me the EXACT same ID... wtf? More than
a coincidence? lol.


> I
> was curious to have a look at how you were handling this. For sure,
> Postgres has native support for ipv4 and ipv6 address storage. AFAIK,
> MySQL does not... although could store an ipv4 address in a 32-bit
> unsigned int field. I don't know of any DB engines that support 128
> bit ints however (for ipv6).
>

Yeah, it's storing using an unsigned int, but there's no way the same logic
could be applied to ipv6.

If I was to continue using MySQL for ipv6 storage, I'd probably create a
table with a column for each byte, convert to an int, and apply a unique
index to them all.


>
> Django 1.4 alpha already uses Postgres' 'inet' field type for
> IPAddressField and GenericIPAddressField. Other DB backends use
> varchar(15) and varchar(39) respectively - which probably leads to
> some interest sorting side effects.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-developers?hl=en.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.