Re: Recognising Contributions

[Tom Forbes]

Perhaps we could do this as part of a Sphinx plugin? Right now each entry
in the release notes is only implicitly tied to the pull request that adds
it.

If we could add some kind of pull request ID marker to the release note
entries we could create an inline link to the PR (which might be very
useful by itself) as well as using it derive a list of contributors for
each release.

Tom

On Wed, 30 Jun 2021 at 09:16, Carlton Gibson 
wrote:

> Hi David,
>
> Thanks for this. Yes.
>
> Let's assume the 2020-2021 time filter is in place.
>
> Mariusz recently picked up James' PR to add the list of Core Contributors
> (back) to the website, which is/was part of the DEP 10 governance changes.
> https://github.com/django/djangoproject.com/pull/1099
>
> The hope is that the DSF Board will approve that in their next meeting,
> and we can get it live. With hindsight we perhaps could have moved quicker
> but, the idea was to move on from there to recognise current and new
> contributors on a more ongoing basis too.
>
> So... my hope was to probably do something per-major release — so 3.2,
> 4.0, 4.1, etc. (Maybe we could do it every month but...)
>
> * Who were the contributors?
> * Who were the new contributors?(Special callout)
> * Who was on the Triage and Review team? ('cause it ain't just code)
> * And, can we identify other folks to call out...? (T&R team was an
> attempt to capture participation here.)
>
> I think Simon's github-to-sqlite tool is a good candidate.
> Some others I've collected whilst this has been bubbling on the low-ring:
>
> * Katie McLaughlin provided some git log pointers
> https://glasnt.com/blog/script-o-hatrack/
> * See also https://github.com/LABHR/octohatrack
> * GitHub built this based on Simon's ideas:
> https://octo.github.com/projects/flat-data
> * "A git query language" https://github.com/filhodanuvem/gitql
> * "git quick stats" https://github.com/arzzen/git-quick-stats
>
> I think there's plenty of tooling there to show how to get the info we
> want.
> At a guess it's a couple of evenings exploring, and then pulling it into a
> report.
>
> I think if we were to do something along these lines, starting a new
> tradition, for Django 4.0 in December, that would be really great.
>
> I'm not sure as yet on the exact format to present all that.
> The blog post for the _Final_ versions could say more without too much
> difficulty.
> (e.g. https://www.djangoproject.com/weblog/2021/apr/06/django-32-released/
> )
>
>
> Kind Regards,
>
> Carlton
>
>
>
>
>
> On Tuesday, 29 June 2021 at 21:35:16 UTC+2 smi...@gmail.com wrote:
>
>> Hi all,
>>
>> I've had this sat in my drafts for a while. Rather than let it sit on the
>> shelf any longer I thought it better to share.
>>
>> I've been thinking about recognising contributions recently. The main
>> issue with the notes here is that it focuses on code rather than
>> contributions to the wider Django ecosystem. However, if there are
>> improvements that we could make here I think we should explore those, and
>> maybe some of them could be used more widely.
>>
>> Here are a few ideas of how contributions could be recognise following a
>> peer review of other projects. Some are better than others, some are easier
>> to implement than others. Hopefully something to prompt some discussion.
>> What do folk think? How would you feel if you were recognised in one of
>> these ways?
>>
>> - Add Python style `contributed by` in the release notes. I'm not so sure
>> about adding the ticket number (in fact I think I saw Nick Pope point to
>> something today that says we don't ref tickets?). [1]
>>
>> - For the headline features add names to the blog post [2]. Could also
>> add link to their blog / website /Twitter (less sure about this second
>> part).
>>
>> - The blog post (or another page) to include a long list of names of
>> everyone who contributeted a commit in that release. I think it's fine if
>> this is long, can probably use Simon W's GitHub-to-sqlite repo for this so
>> it is sustainable. [3]
>>
>> - For the headline features make a series of Twitter posts highlighting
>> them and acknowledge contributors. I'm thinking something along the lines
>> of what Adam Johnson did for the 3.2 release but include names & thank
>> yous.
>>
>> - A rust style thanks page [4] (but **not** the all time list, I don't
>> think that's helpful and it's on GitHub anyway).
>>
>> - A go style contributor summit. (I don't think this is feasible, even
>> remotely. But I'll put it out there!). A slight variation on this could be
>> folk who have contributed could apply for different coloured conference
>> passes/lanyards.  [5]
>>
>> Kind Regards
>>
>> David
>>
>> [1]https://docs.python.org/3/whatsnew/3.8.html
>> [2]https://www.djangoproject.com/weblog/2021/apr/06/django-32-released/
>> [3]https://github.com/dogsheep/github-to-sqlite
>> [4]https://thanks.rust-lang.org/
>> [5]https://blog.golang.org/contributors-summit-2019
>>
> --
> You received this message because you are subscri

Re: Adding ability to choose AutoField type (signed vs unsigned)

[Tom Forbes]

I don't think we need this anymore, at least not by default. The default 64
bit range is probably enough for the time being. We could switch the
default to be `PositiveBigIntegerField` instead of `BigIntegerField` but
I'm not sure if that's sensible.

An explicit `id = PositiveBigIntegerField(...)` workaround would be fine to
be honest, for those that need it.

On Tue, Apr 6, 2021 at 1:12 PM Caio Ariede  wrote:

> Hello folks,
>
> Now that we’ve completed these:
>
> https://code.djangoproject.com/ticket/31007
> https://code.djangoproject.com/ticket/30987
>
> I’m wondering if https://code.djangoproject.com/ticket/56 is still
> something we want to solve.
>
> I feel like the workaround would be to explicitly define ``id =
> models.PositiveBigIntegerField(primary_key=True)``
>
> In the other hand, we could also add ``PositiveBigAutoField`` to make
> things easier now that we have the ability to define the
> ``default_auto_field`` per app.
>
> Thoughts?
>
> — Caio
> On 12 Apr 2020 15:01 -0300, Jure Erznožnik ,
> wrote:
>
> I would like to try again:
> I believe having a general setting for autofields could cause all
> sorts of issues. As illustrated, I immediately thought that some
> people might want to use that for migrating to UUID fields for that (I
> read about using them somewhere about when I started using Django 5
> years ago). The point is that an option like that immediately sparks
> imagination.
>
> However, the problem being attempted at here is only migrating from
> signed to unsigned integer. A very minor change most deployments would
> not even be affected by.
>
> That one could be handled less aggressively by:
>
> OTOH, JUST FOR the unsigned vs signed integer PK, it should be relatively
> easy to modify makemigrations code detect existing migrations for the
> model, detect that the only difference is IntegerField vs
> PositiveIntegerField and in this case skip generating the migration (for
> migrating the field to an unsigned one). There could also be a flag to the
> management command specifying to force generating said migrations.
>
>
> I believe this would be unintrusive enough allowing everyone to
> migrate their models at will. Since most of users won't even notice
> the difference, a solution like this could be preferable even if it
> introduces one additional special case in the code.
> OTOH, the proposed setting DOES provide more flexibility (to introduce
> more varied options for the PK field), but a necessity for a
> biginteger or guid or anything else, for that matter, is already
> catered for in the API - by specifying the super-special pk manually
> for the affected table.
>
> I sure hope I'm not just derailing the discussion with my uninformed
> ideas, but perhaps the minor change could be handled in a simpler
> manner like proposed above.
>
> LP,
> Jure
>
> On Sat, Apr 11, 2020 at 12:21 AM Caio Ariede 
> wrote:
>
>
> I believe that it is already possible to trigger new migrations in a
> third-party app by changing its AppConfig.label, for example. Please
> correct me if I’m wrong.
>
> From that perspective, it think it wouldn’t be wrong if a migration is
> created after someone changes its AppConfig’s default_auto_field. The extra
> migrations could be handled manually using MIGRATION_MODULES.
>
> I feel like adding such option to AppConfig would give a pretty good
> flexibility, but I’m not sure it dismisses the use of
> settings.DEFAULT_AUTO_FIELD. Specially if one desires to keep an old
> default behavior.
>
> --
> Caio
>
>
>
> On Apr 10, 2020, at 04:29, Nick Pope  wrote:
>
> People can also choose to set this for a third-party app by subclassing
> the AppConfig, but as you say, they'd then be forced to handle migrations
> manually - is this even avoidable? I'm not sure how this would look for
> moving to a new default though.
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/3FE8A93E-1A7F-436D-87CC-3D87A6C16801%40gmail.com
> .
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/CAJ%3D9zicjm4nWYRD9FM0EcMVFbWtj8wPHB084P%2BC_5x9wLgrgPQ%40mail.gmail.com
> .
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To

Re: Django's build: Try Earthly?

[Tom Forbes]

Hey!
I've recently been trying to improve our local/CI experience

so this is quite a nice and timely message!

I really like the look of Earthly and it has some uses at $WORK that I'm
excited about, but for Django the most annoying thing is the large build
matrix

rather than a complex dockerfile
. I
really dislike docker-compose for this, but I can't see anything in Earthly
that would help manage or reduce this yaml-spaghetti?

Tom

On Wed, Mar 17, 2021 at 8:10 PM Vlad A. Ionescu  wrote:

> Hi Django devs,
>
> Nick Pope directed me to this list. Wondering if anyone is interested in
> trying https://github.com/earthly/earthly for Django's build.
>
> Earthly could help with reproducing CI failures locally (via containers),
> testing against multiple versions of Python in parallel, or for migrating
> between CI vendors.
>
> It works well on top of Jenkins and GH Actions.
>
> Happy to put together a demo PR if this is of interest. Let me know!
>
> Cheers,
> Vlad.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/CAO_OUmM%2BMFU1nPMzFaEsGcD4rh6tFh-wQ3T_hK4GH5-4%2BCxgxQ%40mail.gmail.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFNZOJM1SEpJiJdsq0AFmOefZFRBPmaDXZseE3e4J7QALw%2BsKw%40mail.gmail.com.

Re: Help with ticket #28426

[Tom Forbes]

Personally I don’t think we should prompt the user for anything. If the user 
gives the username and password in the url (user:pass@host) then we can use 
that, otherwise we just throw an error. 

A more complicated solution involving making a request, detecting a 401, 
prompting the user and retrying the request is a bit overkill and potentially a 
(unlikely) breaking change.

Tom

> On 4 Mar 2021, at 17:55, 'Adam Johnson' via Django developers (Contributions 
> to Django itself)  wrote:
> 
> 
> I also think this is feature creep and if it's a complicated change it's not 
> worth it.
> 
>> On Thu, 4 Mar 2021 at 18:36, Tim Graham  wrote:
>> I'd like to see what your code looks like so far. Personally, this is 
>> sounding a lot more complicated than I imagined when I accepted the ticket. 
>> I doubt this is a highly requested feature that couldn't be solved another 
>> way (e.g. downloading the template file without Django), and It's not clear 
>> to me that adding logic to Django is worth it.
>> 
>>> On Thursday, March 4, 2021 at 11:01:41 AM UTC-5 benc...@gmail.com wrote:
>>> Thanks for the fast reply
>>> 
>>> I found these test cases but I am not sure how to extend them with basic 
>>> auth, because I don't know if the LiveServerTestCase is capable of doing 
>>> basic auth. As I see currently there is no testcase checking basic auth 
>>> here and it has to be checked by hand.
>>> 
>>> I am sorry but could you elaborate on this please because I couldn't find 
>>> these functions.
>>> 
>>> "first add view (in admin_scripts/urls.py is fine) to first check for basic 
>>> auth credentials and then pass off to `serve` (as the existing route 
>>> already does). That should give us the reproduce (or we can closed as 
>>> fixed)."
>>> 
>>> Best Regards,
>>>Bence
>>> 
>>> Carlton Gibson  ezt írta (időpont: 2021. márc. 4., 
>>> Cs, 11:15):
 Hi Bence, welcome! 
 
 There are already a couple of tests in place to check the remove fetching: 
 
 https://github.com/django/django/blob/05bbff82638731a6abfed2fe0ae06a4d429cb32f/tests/admin_scripts/tests.py#L2047-L2072
 
 Without changing the command code I'd first add view (in 
 admin_scripts/urls.py is fine) to first check for basic auth credentials 
 and then pass off to `serve` (as the existing route already does). That 
 should give us the reproduce (or we can closed as fixed). 
 
 Then I think it's easier if you open a PR with your suggested change after 
 that (but more-or-less sounds plausible without looking in depth.) 
 
 Hopefully that gets you going? 
 
 Kind Regards,
 
 Carlton
 
>>> 
 -- 
 You received this message because you are subscribed to the Google Groups 
 "Django developers (Contributions to Django itself)" group.
 To unsubscribe from this group and stop receiving emails from it, send an 
 email to django-develop...@googlegroups.com.
 To view this discussion on the web visit 
 https://groups.google.com/d/msgid/django-developers/66771c44-5b9f-4c29-b92a-91cd3092e016n%40googlegroups.com.
>> 
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-developers+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-developers/814187f8-dc2b-47e1-b91d-76d0ab78241an%40googlegroups.com.
> 
> 
> -- 
> Adam
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/CAMyDDM31cTcecyx%2BD%3DEP4nFD2qmW1f%3DOFrQfL7D2j-k9d-cT0w%40mail.gmail.com.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/076043F9-46D1-4B6A-A6F0-9034F87EBD5D%40tomforb.es.

Re: Invitation to participate in a survey about Django

[Tom Forbes]

I receive a few of these kinds of emails privately (I’m assuming they
scrape my email from git history). On one hand I think it might be
appropriate to post something like this to the developers mailing list if
it was specifically targeted to Django, and it’s definitely good to help
with research.

On the other hand, like many of the other emails I receive this looks to be
an impersonal and auto-generated one (the bold really doesn’t help) with a
completely vague call to action: your requirement to “understand how
technical debt is managed” means nothing to me and doesn’t entice me to
spend time answering 18+ questions on a Google form for you.

The “what’s in it for me” section is also completely meaningless - how will
answering this help me maintain the quality of Django? So to me this is
basically spam and I don’t think it’s appropriate to post here. I’m
replying publicly in case anyone else has any opinions on if this
constitutes as (academic) spam or not.

In the hopes of being constructive I’d suggest perhaps a paragraph
explaining who you are, what the actual concrete work is that you are
doing, how our contributions can help further that and the outcomes you
want to achieve. Because right now you could replace startlingly few words
to turn this into an email about people’s thoughts on the new Pepsi flavor.

Tom

On Mon, 1 Mar 2021 at 23:17, Tan, J.  wrote:

> Dear Django contributor,
>
> We are doing research on understanding how developers manage a special
> kind of Technical Debt in *Python.*
>
> We kindly ask 15-20 minutes of your time to fill out our survey. To help
> you decide whether to fill it in, we clarify two points.
>
> “Why should I answer this survey?”
>
> Your participation is essential for us to correctly understand how
> developers manage Technical Debt.
>
> “What is in it for me?”
>
> Your valuable contributions to *Django* are part of the information we
> analyzed for this study. Thus, if you help us further by answering
> this survey, there are two immediate benefits:
>
>- you help to improve the efficiency of maintaining the quality of
>*Django*.
>- the results will be used to propose recommendations to manage
>technical debt and create tool support.
>
> Here is the link to the survey
> 
> .
>
> Thank you for your time and attention.
>
> Kind regards,
> Jie Tan, Daniel Feitosa and Paris Avgeriou
>
> Software Engineering and Architecture group 
> Faculty of Science and Engineering
> University of Groningen, the Netherlands
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/CAJvbMuPzdbmzUtv1qC5SXdRG7yM5R3Gq70VOQivJFY69Jxz-PA%40mail.gmail.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFNZOJNMA8KdBWE5uOrdKeHbWmtVGesVsUc1eXOn35kinmu92g%40mail.gmail.com.

Re: About the ORM icontains operator's disadvantage on PostgreSQL performance and query results.

[Tom Forbes]

Thank you for the clarification! I think the biggest argument for this
change is the fact that uppercasing Unicode can cause incorrect results to
be returned.

Given that we now have much better support for custom index types, perhaps
we should change this? We need a custom expression index anyway, so it
might not be a huge ask to say “now you should use a gin index”?

Tom

On Sun, 28 Feb 2021 at 09:28, Tom Forbes  wrote:

> Thank you for the clarification!
>
> On Sun, 28 Feb 2021 at 09:23, Mesut Öncel  wrote:
>
>> Yes, 7 years ago I reviewed the ticket for the updated code, but there
>> are problems there.https: //code.djangoproject.com/ticket/32485 The
>> ticket I opened has the necessary details. If you want more details, I can
>> make the necessary explanations.
>>
>> Tom Forbes , 28 Şub 2021 Paz, 12:36 tarihinde şunu yazdı:
>>
>>> Unless I’m missing something, the ticket you linked was last updated 7
>>> years ago with two now-broken images. Which ticket did you open with the
>>> examples?
>>>
>>> Tom
>>>
>>> On Sun, 28 Feb 2021 at 08:31, Mesut Öncel 
>>> wrote:
>>>
>>>> The reason I opened all the problems in a single ticket was to show the
>>>> disadvantages of the "UPPER" function used in the "icontains" operator.I
>>>> think that instead of using "UPPER (column) like" used in the Django ORM
>>>> filter method, the "icontains" operator should use the "ilike" operator and
>>>> this should be changed in the source code I specified in the ticket. From
>>>> the query examples I added to the ticket, you can see that the query "UPPER
>>>> (column) like" generated by the "icontains" operator has a performance
>>>> problem and produces incorrect results. If you use "ilike" instead of
>>>> "UPPER (column)" in the icontains operator, the 2 problems I mentioned will
>>>> be solved.
>>>>
>>>> Tom Forbes , 28 Şub 2021 Paz, 01:17 tarihinde şunu
>>>> yazdı:
>>>>
>>>>> Hey,
>>>>> I think it would be best if you opened a new ticket explaining your
>>>>> issue and the performance problems you’ve found (especially around Turkish
>>>>> characters). I’m not entirely clear what you wish to be changed: you wish
>>>>> for an expression index to be created automatically?
>>>>>
>>>>> Tom
>>>>>
>>>>> On Sat, 27 Feb 2021 at 21:13, Mesut Öncel 
>>>>> wrote:
>>>>>
>>>>>> Hi,
>>>>>> I opened the ticket https://code.djangoproject.com/ticket/3575. In
>>>>>> this ticket, I wanted to prove the effect of UPPER function used in
>>>>>> icontains operator on indexes and query result. UPPER function causes
>>>>>> performance problems regarding indexing in PostgreSQL. Another issue is
>>>>>> that it does not give correct results especially for Turkish characters.
>>>>>>
>>>>>> Database administrators do not want to add individual indexes in this
>>>>>> regard. Because adding more than one index to the same column in full 
>>>>>> text
>>>>>> search will negatively affect cost and performance.
>>>>>>
>>>>>> An expression index should be created to prevent performance loss in
>>>>>> database queries to be used outside the framework. However, the existence
>>>>>> of such an index should be notified to everyone who uses this database, 
>>>>>> and
>>>>>> the use of "UPPER (name) ilike '% AA%" instead of "ilike" should be
>>>>>> encouraged. This will cause PostgreSQL to block the "ilike" operator.
>>>>>>
>>>>>> If you give the necessary approval in this regard, I want to open a
>>>>>> pull request for the field specified on the ticket.
>>>>>>
>>>>>> Thanks.
>>>>>>
>>>>>> --
>>>>>> You received this message because you are subscribed to the Google
>>>>>> Groups "Django developers (Contributions to Django itself)" group.
>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>> send an email to django-developers+unsubscr...@googlegroups.com.
>>>>>> To view this discussion on the web visit
>>>

Re: About the ORM icontains operator's disadvantage on PostgreSQL performance and query results.

[Tom Forbes]

Thank you for the clarification!

On Sun, 28 Feb 2021 at 09:23, Mesut Öncel  wrote:

> Yes, 7 years ago I reviewed the ticket for the updated code, but there are
> problems there.https: //code.djangoproject.com/ticket/32485 The ticket I
> opened has the necessary details. If you want more details, I can make the
> necessary explanations.
>
> Tom Forbes , 28 Şub 2021 Paz, 12:36 tarihinde şunu yazdı:
>
>> Unless I’m missing something, the ticket you linked was last updated 7
>> years ago with two now-broken images. Which ticket did you open with the
>> examples?
>>
>> Tom
>>
>> On Sun, 28 Feb 2021 at 08:31, Mesut Öncel  wrote:
>>
>>> The reason I opened all the problems in a single ticket was to show the
>>> disadvantages of the "UPPER" function used in the "icontains" operator.I
>>> think that instead of using "UPPER (column) like" used in the Django ORM
>>> filter method, the "icontains" operator should use the "ilike" operator and
>>> this should be changed in the source code I specified in the ticket. From
>>> the query examples I added to the ticket, you can see that the query "UPPER
>>> (column) like" generated by the "icontains" operator has a performance
>>> problem and produces incorrect results. If you use "ilike" instead of
>>> "UPPER (column)" in the icontains operator, the 2 problems I mentioned will
>>> be solved.
>>>
>>> Tom Forbes , 28 Şub 2021 Paz, 01:17 tarihinde şunu
>>> yazdı:
>>>
>>>> Hey,
>>>> I think it would be best if you opened a new ticket explaining your
>>>> issue and the performance problems you’ve found (especially around Turkish
>>>> characters). I’m not entirely clear what you wish to be changed: you wish
>>>> for an expression index to be created automatically?
>>>>
>>>> Tom
>>>>
>>>> On Sat, 27 Feb 2021 at 21:13, Mesut Öncel 
>>>> wrote:
>>>>
>>>>> Hi,
>>>>> I opened the ticket https://code.djangoproject.com/ticket/3575. In
>>>>> this ticket, I wanted to prove the effect of UPPER function used in
>>>>> icontains operator on indexes and query result. UPPER function causes
>>>>> performance problems regarding indexing in PostgreSQL. Another issue is
>>>>> that it does not give correct results especially for Turkish characters.
>>>>>
>>>>> Database administrators do not want to add individual indexes in this
>>>>> regard. Because adding more than one index to the same column in full text
>>>>> search will negatively affect cost and performance.
>>>>>
>>>>> An expression index should be created to prevent performance loss in
>>>>> database queries to be used outside the framework. However, the existence
>>>>> of such an index should be notified to everyone who uses this database, 
>>>>> and
>>>>> the use of "UPPER (name) ilike '% AA%" instead of "ilike" should be
>>>>> encouraged. This will cause PostgreSQL to block the "ilike" operator.
>>>>>
>>>>> If you give the necessary approval in this regard, I want to open a
>>>>> pull request for the field specified on the ticket.
>>>>>
>>>>> Thanks.
>>>>>
>>>>> --
>>>>> You received this message because you are subscribed to the Google
>>>>> Groups "Django developers (Contributions to Django itself)" group.
>>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>>> an email to django-developers+unsubscr...@googlegroups.com.
>>>>> To view this discussion on the web visit
>>>>> https://groups.google.com/d/msgid/django-developers/ba167e5c-49dd-4d96-a2c4-855082639d4en%40googlegroups.com
>>>>> <https://groups.google.com/d/msgid/django-developers/ba167e5c-49dd-4d96-a2c4-855082639d4en%40googlegroups.com?utm_medium=email&utm_source=footer>
>>>>> .
>>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google
>>>> Groups "Django developers (Contributions to Django itself)" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to django-developers+unsubscr...@googlegroups.com.
>>>>
>>> To view this discussion on the 

Re: About the ORM icontains operator's disadvantage on PostgreSQL performance and query results.

[Tom Forbes]

Unless I’m missing something, the ticket you linked was last updated 7
years ago with two now-broken images. Which ticket did you open with the
examples?

Tom

On Sun, 28 Feb 2021 at 08:31, Mesut Öncel  wrote:

> The reason I opened all the problems in a single ticket was to show the
> disadvantages of the "UPPER" function used in the "icontains" operator.I
> think that instead of using "UPPER (column) like" used in the Django ORM
> filter method, the "icontains" operator should use the "ilike" operator and
> this should be changed in the source code I specified in the ticket. From
> the query examples I added to the ticket, you can see that the query "UPPER
> (column) like" generated by the "icontains" operator has a performance
> problem and produces incorrect results. If you use "ilike" instead of
> "UPPER (column)" in the icontains operator, the 2 problems I mentioned will
> be solved.
>
> Tom Forbes , 28 Şub 2021 Paz, 01:17 tarihinde şunu yazdı:
>
>> Hey,
>> I think it would be best if you opened a new ticket explaining your issue
>> and the performance problems you’ve found (especially around Turkish
>> characters). I’m not entirely clear what you wish to be changed: you wish
>> for an expression index to be created automatically?
>>
>> Tom
>>
>> On Sat, 27 Feb 2021 at 21:13, Mesut Öncel  wrote:
>>
>>> Hi,
>>> I opened the ticket https://code.djangoproject.com/ticket/3575. In this
>>> ticket, I wanted to prove the effect of UPPER function used in icontains
>>> operator on indexes and query result. UPPER function causes performance
>>> problems regarding indexing in PostgreSQL. Another issue is that it does
>>> not give correct results especially for Turkish characters.
>>>
>>> Database administrators do not want to add individual indexes in this
>>> regard. Because adding more than one index to the same column in full text
>>> search will negatively affect cost and performance.
>>>
>>> An expression index should be created to prevent performance loss in
>>> database queries to be used outside the framework. However, the existence
>>> of such an index should be notified to everyone who uses this database, and
>>> the use of "UPPER (name) ilike '% AA%" instead of "ilike" should be
>>> encouraged. This will cause PostgreSQL to block the "ilike" operator.
>>>
>>> If you give the necessary approval in this regard, I want to open a pull
>>> request for the field specified on the ticket.
>>>
>>> Thanks.
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Django developers (Contributions to Django itself)" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to django-developers+unsubscr...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/django-developers/ba167e5c-49dd-4d96-a2c4-855082639d4en%40googlegroups.com
>>> <https://groups.google.com/d/msgid/django-developers/ba167e5c-49dd-4d96-a2c4-855082639d4en%40googlegroups.com?utm_medium=email&utm_source=footer>
>>> .
>>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to django-developers+unsubscr...@googlegroups.com.
>>
> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/django-developers/CAFNZOJOr7N%3DT_7Jwj5fqv2ec3UuoVqrjCUwC_C9s_3-sckHbwQ%40mail.gmail.com
>> <https://groups.google.com/d/msgid/django-developers/CAFNZOJOr7N%3DT_7Jwj5fqv2ec3UuoVqrjCUwC_C9s_3-sckHbwQ%40mail.gmail.com?utm_medium=email&utm_source=footer>
>> .
>>
>
>
> --
> İyi çalışmalar. Saygılarımla.
>
> *Mesut Öncel*
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/CAACgSnQE4KJyKNU7hi1REQFXCVyo07QhBw6F2QUxUP5m%2BK06qQ%40mail.gmail.com
> <https://groups.google.com/d/msgid/django-developers/CAACgSnQE4KJyKNU7hi1REQFXCVyo07QhBw6F2QUxUP5m%2BK06qQ%40mail.gmail.com?utm_medium=email&utm_source=footer>
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFNZOJOBzXOTSsE-i1cudhGgZ84_tV30%2B40kp%3D0JQnw7xwDnmA%40mail.gmail.com.

Re: About the ORM icontains operator's disadvantage on PostgreSQL performance and query results.

[Tom Forbes]

Hey,
I think it would be best if you opened a new ticket explaining your issue
and the performance problems you’ve found (especially around Turkish
characters). I’m not entirely clear what you wish to be changed: you wish
for an expression index to be created automatically?

Tom

On Sat, 27 Feb 2021 at 21:13, Mesut Öncel  wrote:

> Hi,
> I opened the ticket https://code.djangoproject.com/ticket/3575. In this
> ticket, I wanted to prove the effect of UPPER function used in icontains
> operator on indexes and query result. UPPER function causes performance
> problems regarding indexing in PostgreSQL. Another issue is that it does
> not give correct results especially for Turkish characters.
>
> Database administrators do not want to add individual indexes in this
> regard. Because adding more than one index to the same column in full text
> search will negatively affect cost and performance.
>
> An expression index should be created to prevent performance loss in
> database queries to be used outside the framework. However, the existence
> of such an index should be notified to everyone who uses this database, and
> the use of "UPPER (name) ilike '% AA%" instead of "ilike" should be
> encouraged. This will cause PostgreSQL to block the "ilike" operator.
>
> If you give the necessary approval in this regard, I want to open a pull
> request for the field specified on the ticket.
>
> Thanks.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/ba167e5c-49dd-4d96-a2c4-855082639d4en%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFNZOJOr7N%3DT_7Jwj5fqv2ec3UuoVqrjCUwC_C9s_3-sckHbwQ%40mail.gmail.com.

Re: Add a stable and documented setting to add files to the runserver watcher

[Tom Forbes]

 Hey Diptesh!
It was always my intention to document and expose the autoreloader signals
to third party applications, that way ariadne can configure watches on the
graphql files it knows about. I think this is preferable to having users
having to configure this directly in their projects. We kept the API as
private initially as I wasn’t sure how stable it was going to be, but there
haven’t been any major changes required so I think it’s time to open it up
and mark it as something packages can build upon, if nobody else disagrees?

Regarding a setting - I’m of two minds about this. It’s a good “escape
hatch” for packages that don’t use this signal yet, but it’s also not
particularly… great. Not sure.

Tom

On 8 Feb 2021 at 02:16:45, Diptesh Choudhuri 
wrote:

> I am working on a graphQL project which uses django and ariadne
> . This package supports
> 
> loading your graphQL schema from external **.graphql *files. While
> developing the schema, I continually change these files and have to restart
> my *runserver *every time I do so, since the changes are not watched by
> default. This obviously gets quickly tiring since designing a GraphQL
> schema takes a lot of iterations.
>
> There is a somewhat hacky fix
>  to this (here's
> 
> an implementation that I used in a project), but the problem with it is
> that it does not work at all for extensions like *django_extension's
> runserver *command. Additionally, it requires you to have to start an
> app, since it is defined in the *AppConfig, *something which might be
> inconvenient to some people.
>
> I propose we add a *AUTORELOADER_ADD_FILES *variable to the *settings.py *file
> which should be a Linux glob (eg: *AUTORELOADER_ADD_FILES = **BASE_DIR /
> "**/*.graphql"*).
>
> If approved, I would like to make a PR for the same.
>
> Regards
> Diptesh Choudhuri
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/0164ea90-33a2-4ee5-b6cd-df508b6ab013n%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFNZOJMCxT4%2BGTNJqJbzr0_1pPTdtv-urNJqWR%2B%3DcJmQP0nEZw%40mail.gmail.com.

Re: Increase default integer keys to 64 bits

[Tom Forbes]

Better than that: 3.2 is the first step to changing the default. See 
“customising the type of primary keys” in the release notes 
(https://docs.djangoproject.com/en/dev/releases/3.2/).

“GENERATED BY DEFAULT AS IDENTITY” would be a nice improvement though.

Tom

> On 29 Jan 2021, at 01:58, Curtis Maloney  wrote:
> 
> 
> I recall a discussion some time about about adding a setting to control which 
> field to use for the default PK... seems there's some overlap here.
> 
> --
> C
> 
> 
>> On Fri, 29 Jan 2021, at 12:42, Christophe Pettus wrote:
>> tl;dr: Introduce new field types to handle auto-incremented ID fields, 
>> change the PostgreSQL backend to use the preferred syntax
>> 
>> --
>> 
>> One of the most common issues my company runs into on Django sites is that 
>> models.AutoField defaults to a 32-bit integer (int32).  2^31-1 possible 
>> entries is just not that many anymore, and by the time the developers 
>> realize this and need to move to a 64 bit integer key, it's too late to do 
>> so conveniently, because expanding the field is very painful (in PostgreSQL, 
>> at least).
>> 
>> While models.AutoBigField exists, it's barely mentioned in examples, and is 
>> often overlooked.
>> 
>> Changing AutoField to 64 bits would result in all kinds of breakage; at 
>> best, a lot of very unplanned and expensive migrations.
>> 
>> My proposal is:
>> 
>> 1. Create two new field types to represent auto-incrementing primary keys.  
>> I'd suggest IdentityField and SmallIdentityField for int64 and int32, 
>> respectively.
>> 
>> 2. Change all examples to use SerialField instead of AutoField.
>> 
>> 3. As a side note, switch the PostgreSQL backend to use the standard 
>> "GENERATED BY DEFAULT AS IDENTITY" syntax.  This became available in 
>> PostgreSQL version 10, but the previous version (9.6) reaches EOL in 
>> November 2021.
>> 
>> 4. At some point in the future, deprecate AutoField and AutoBigField.
>> 
>> This would result in new projects getting 64 bit primary keys by default.  I 
>> think that's a positive.  For small tables, the size difference hardly 
>> matters; for big tables, we have saved a major foot-gun of either integer 
>> exhaustion or a very expensive data migration problem.
>> 
>> --
>> 
>> Comments?
>> 
>> --
>> -- Christophe Pettus
>>x...@thebuild.com
>> 
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers  (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-developers+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-developers/F4A50AB6-8CC1-4B6E-BD1B-B865F53A645A%40thebuild.com.
>> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/50ec245f-21a4-4ab9-ac51-55be2c969310%40www.fastmail.com.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/DCEFF85C-0C69-4C63-8E50-B7A4D36ADA1E%40tomforb.es.

Re: Snowflake db backend

[Tom Forbes]

> Honestly, I don't think we'd release this as a third party package.  We have 
> had to code around a lot of core db specific code to get Snow SQL to work

This is good feedback that is valuable for us to hear. Even if you don’t 
release it as a fully maintained package just understanding the pain points you 
encountered while developing this is great feedback that can help us improve.

Would you be willing to share specific code snippets here, or just general 
areas that you found challenging while developing this backend?

> That's what has made the process harder than it should be to support a 
> non-core db.  It creates second class system support in favor of the core dbs 
> that are code and tested against.  Remove all of the db backends and make 
> them all third party and you'd see a lot more db support.

I can see a world where even core Django database backends are much more plugin 
oriented and could definitely bring some advantages. But that’s a non-trivial 
task with a lot of constraints especially for a fairly community driven project 
like Django.

Tom

> On 22 Jan 2021, at 22:34, Scott Fought  wrote:
> 
> Snowflake (at this point) isn't as fast as PostgreSQL for smaller 
> transactions.  PostgreSQL couldn't handle the performance we needed to drive 
> our apps and we really didn't want to support a hybrid db app.  We give up a 
> little speed for the convenience of having everything in one system.  At a 
> recent Snowflake event, they stated that 30% of their users were using 
> Snowflake for application backends.  They are also working to improve their 
> execution time on smaller transactions.
> 
> Honestly, I don't think we'd release this as a third party package.  We have 
> had to code around a lot of core db specific code to get Snow SQL to work.  
> If there was a hard API for DB backends my response would be different.  It 
> would only make sense for us if this backend were part of standard Django 
> testing and support consideration.
> 
> "Merging the backend straight to core would impose the requirement that all 
> changes to Django core consider Snowflake"
> That's what has made the process harder than it should be to support a 
> non-core db.  It creates second class system support in favor of the core dbs 
> that are code and tested against.  Remove all of the db backends and make 
> them all third party and you'd see a lot more db support.
> 
> Scott
> 
>> On Friday, January 22, 2021 at 5:22:24 PM UTC-5 Adam Johnson wrote:
>> Yes we would favour a third party backend. Merging the backend straight to 
>> core would impose the requirement that all changes to Django core consider 
>> Snowflake, which is a fairly steep cost when there's only one organization 
>> using it (at current). If you release as a third party package and show some 
>> adoption, it could then be argued that it's worth merging.
>> 
>> Additionally Scott - would love to see Disney appear on this page, perhaps 
>> the "Platinum" section would feel comfortable: 
>> https://www.djangoproject.com/fundraising/ ;)
>> 
>>> On Fri, 22 Jan 2021 at 21:57, Tom Forbes  wrote:
>>> I think this should definitely be released as a third party package, and if 
>>> there is enough community interest it might be considered for inclusion. We 
>>> could definitely update the docs to link to the package though.
>>> 
>>> On a side note, is Snowflake fast enough for general purpose web apps? When 
>>> we evaluated it the performance characteristics where similar to Redshift, 
>>> optimised for large analytical aggregation rather than fast, smaller result 
>>> sets typically seen with Django apps.
>>> 
>>> Tom
>>> 
>>>>> On 22 Jan 2021, at 21:49, Scott Fought  wrote:
>>>>> 
>>>> 
>>>> We have written a Snowflake DB backend that we use internally in several 
>>>> projects and are considering releasing it for public use.  My preference 
>>>> would be to incorporate it as a core backend so the nuances of Snowflake 
>>>> SQL are considered when changes are made to the core system.  Is this 
>>>> something we should pursue?
>>>> 
>>>> 
>>>> 
>>>> Thanks,
>>>> 
>>>> 
>>>> 
>>>> Scott
>>>> 
>>>> -- 
>>>> You received this message because you are subscribed to the Google Groups 
>>>> "Django developers (Contributions to Django itself)" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send an 
>&

Re: Snowflake db backend

[Tom Forbes]

I think this should definitely be released as a third party package, and if 
there is enough community interest it might be considered for inclusion. We 
could definitely update the docs to link to the package though.

On a side note, is Snowflake fast enough for general purpose web apps? When we 
evaluated it the performance characteristics where similar to Redshift, 
optimised for large analytical aggregation rather than fast, smaller result 
sets typically seen with Django apps.

Tom

> On 22 Jan 2021, at 21:49, Scott Fought  wrote:
> 
> 
> We have written a Snowflake DB backend that we use internally in several 
> projects and are considering releasing it for public use.  My preference 
> would be to incorporate it as a core backend so the nuances of Snowflake SQL 
> are considered when changes are made to the core system.  Is this something 
> we should pursue?
> 
> 
> 
> Thanks,
> 
> 
> 
> Scott
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/3e5034a8-6c75-4f28-9830-2c87b671ae86n%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/08448193-19F7-4AA8-A112-ED6802DC7FBF%40tomforb.es.

Re: Polymorphics relationship

[Tom Forbes]

Hey, when proposing a feature it helps to give an example and clearly state 
what you wish to add, then we can give a much better opinion.

Tom

> On 21 Jan 2021, at 20:41, Jonnathan Carrasco  
> wrote:
> 
> I have inspect the code base of django core and database, and we need to add 
> this feature to framework (if doest not exists) for more efficiency database 
> queries and workflow. I have been programming with django 3 month ago, and 
> I'd like to contrib for this feature. So what do you think about it?
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/1a622a77-1fa0-442c-9fd6-357b1de2n%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/4EED0389-DDD9-46B5-982A-E94F598B1FBF%40tomforb.es.

Re: Technical Board Decision Needed: Admin append_slash behaviour.

[Tom Forbes]

I agree with this especially around the point about timing attacks. 

I don’t believe potentially being able to infer the names of models in a very, 
very noisy way (thousands of requests) gives anyone leverage in a system or 
even any particularly sensitive information at all. Maybe in a really extreme 
scenario involving other issues within an app like some form of blind 
class-based direct object access vulnerability, but that’s really stretching 
the imagination.

If I had to pick I would say 1, but it still leaves open timing attacks as I’d 
expect the permission check path would take an order of magnitude longer than 
the fast “404 but not really” path. So it begs the question “do we really want 
to?”

Tom

> On 6 Jan 2021, at 13:20, James Bennett  wrote:
> 
> I'm going to be the contrarian here and at least ask whether the right answer 
> is "don't do any of these options".
> 
> To see why, consider a hypothetical world where we do one of the above 
> options, and a site sets whatever config is necessary to disable APPEND_SLASH 
> behavior in the admin.
> 
> The very next thing we're going to get is a report to the security address 
> saying there's a detectable timing difference between what we do to hide a 
> real app/model from a user with admin-access-but-not-to-that-model/app, and 
> what we do when that app/model genuinely doesn't exist, and in order to 
> properly hide apps/models could we please address that?
> 
> (and if we do end up taking one of the above options, please consider this my 
> report of this vulnerability, as I can pretty much guarantee that 404'ing a 
> nonexistent app/model will be faster than finding it, resolving a URL into 
> it, and doing the associated permission check)
> 
> And this will never end. For a user who already has been granted admin 
> access, there likely *always* will be some detectable difference between the 
> admin's behavior with respect to a model/app that isn't present versus one 
> that is present but for which the user is not authorized. In an app with the 
> complexity of the admin, there are just too many possible 
> behavioral-difference vectors that people will be able to come up with, and 
> we'll never be able to close them all reliably.
> 
> So what I'd really like us to do here is pause and consider what kinds of 
> guarantees we can really commit to. Some are obvious and good and we already 
> do or try to do them -- you shouldn't be able to access/modify/delete things 
> you haven't been granted access to, you shouldn't be able to escalate your 
> own privileges, you shouldn't be able to XSS the admin, etc.
> 
> But it's not at all clear to me that "the existence or nonexistence of 
> apps/models to which they don't have access is undetectable to an admin user" 
> is one of the guarantees the admin should make. I understand the rationale, 
> but I'm not sure the threat it wants to be securing against is one we 
> reasonably *can* secure against.
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/CAL13Cg_1QRhht3eSD29AJ25Cbkp%2BfbCxyOi7GdhYpfzcEkmDbQ%40mail.gmail.com.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/827E088B-21AE-4B02-8BB7-6A6065414045%40tomforb.es.

Re: GitHub Actions

[Tom Forbes]

Sorry for bumping such an old thread, but I've created a github actions POC 
and written it up here: 
https://forum.djangoproject.com/t/improving-the-contribution-experience-with-github-actions/5964

On Monday, 11 November 2019 at 08:11:57 UTC Johannes Maron wrote:

> Hi Mariusz, 
>
> Cool, let me know if you have any questions about the config file that I 
> wrote. I am happy to help :)
>
> Regarding the output, yes I can feel you. However I don’t know if we 
> really need to run tests at verbosity level 2 on CI. One can always do that 
> online or enable it if need be.
>
> At list of tests that failed in a human readable output format would be 
> nice. GitHub supports annotation on checks, but it's not really convenient 
> yet to be used.
>
> Using it for linters is a good start. We could also do the docs build and 
> limit that one to be only build if the docs actually changed. You can 
> trigger builds based on file condition 
> ,
>  
> which is pretty neat.
>
> Something that we also haven’t explored are scheduled builds. Those could 
> be helpful to do regression tests on especially on dependencies.
>
> Anyhow, love to see where this is going.
> -Joe
> On 11. Nov 2019, 15:53 +0900, Mariusz Felisiak , 
> wrote:
>
>
> I would suggest to open a PR to GitHub now and to get the Fellows 
>> involved. Ask them how to best integrate a trial without disturbing their 
>> workflow. What do you guys think?
>>
>>
> I'm going to play with GitHub actions in this week (or in the next one) 
> but I'm still not convinced. It's fine to use them for linters, but for 
> more complicated jobs it generates output that is less readable, IMO. You 
> don't get a formatted table but a raw output, so you have to scroll down 
> ~20k lines to check what went wrong. This can be really frustrating 
> especially with more then one failure. Nevertheless I'm going to play with 
> it.
>
> Best
> Mariusz
>
>
> --
> You received this message because you are subscribed to a topic in the 
> Google Groups "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this topic, visit 
> https://groups.google.com/d/topic/django-developers/xDCmZPLTOQQ/unsubscribe
> .
> To unsubscribe from this group and all its topics, send an email to 
> django-develop...@googlegroups.com.
>
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/6692d50f-b02d-4824-aa12-be22a20c3f0f%40googlegroups.com
>  
> 
> .
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/0c944a34-6c73-457b-a860-a8b355589113n%40googlegroups.com.

Re: Add decorator django.utils.functional.cached_classproperty

[Tom Forbes]

I’m slightly skeptical - a cached class property seems slightly iffy and could 
be achieved with just a @functools.cache’d classmethod instead? 

Is there a specific use case where a cached, class _property_ would be the best 
way? And perhaps we need to care about concurrent access here, more than with 
our other cache implementations.

However, this is a really small change, so maybe it’s OK.

Tom

> On 30 Dec 2020, at 14:46, Boris Krause  wrote:
> Hello,
> I propose new feature - decorator cached_classproperty.
> Django has a lot of methods for caching, like cached_page and cached_property.
> Сaching improves response times.
> For me it was useful.
> What do you think about it?
> https://github.com/django/django/pull/13813
> Thanks!:)
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/fe13f5c9-8ef5-44eb-bfc4-731865a9123an%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/4E510911-08BE-4B94-8BD8-97B8391709C0%40tomforb.es.

Re: Integrating migrations with the check framework

[Tom Forbes]

 Thanks for all the input here!

I didn’t know about django-migration-linter, and it does seem quite
interesting. It works by looking at the complete set of SQL statements that
would be run, and running regular expressions/string comparisons on it (
https://github.com/3YOURMIND/django-migration-linter/blob/master/django_migration_linter/sql_analyser/base.py#L59-L62).
I don’t hate this idea, but I think that having direct access to the
migration objects themselves could greatly simplify the implementation.

Django-pg-zero-downtime-migrations also looks interesting and is more
in-line with what I was imagining (
https://github.com/tbicr/django-pg-zero-downtime-migrations/blob/master/django_zero_downtime_migrations/backends/postgres/schema.py)
- it overrides the schema editor to detect unsafe migrations, and provides
a nice way to abort migrations that take locks for too long. However I feel
that the implementation could be greatly simplified with some support for
Django - you shouldn’t really need to override the schema editor, Django
should pass you a list of operations and ask “is this safe?”. The results
can be reported via the checks framework which allows application owners
can decide to treat these warnings as errors or silence them as needed. It
also brings those warnings front-and-centre while developing, rather than
having it fail when running “migrate” later on in the deployment process.

Django-safemigrate is very related and something I wanted to cover in a
later message. I feel that some of this functionality could (or even
should) be integrated with Django, as migrations in “real” apps are usually
a two step process: you run the schema migrations, which might add a
nullable column, deploy your app, then run data migrations to populate the
column. At least bringing the concept of a pre-deploy and a post-deploy
migration to Django would be pretty useful I think.

I’d say the open questions so far are:

   1. Does it make sense to add a specific system check registration
   function for migrations? If not, how do we expose this to users wishing to
   write system checks?
   2. Are we comfortable with creating a public API for some of the
   migrations internals?
   3. Should we (or can we?) make the distinction between pre-deploy and
   post-deploy migrations?


Another consideration with all this is that for a non-trivial amount of
applications all this doesn’t really matter. If you’ve got an internal app
that is used during business hours almost none of this applies - you can
get away with downtime and “dangerous” migrations, and similarly if you are
starting a new project you don’t want Django to start complaining about
adding a non-null field to a table that has no rows in a project that isn’t
even deployed yet. This is why I think the MVP of this would just be to
allow users to write their own business-specific checks for migrations and
allow third party apps to more easily integrate with the migration
framework, rather than anything more complicated.

Tom

On 23 Dec 2020 at 17:32:23, Ryan Hiebert  wrote:

> This is an interesting discussion that is separate from, but related to,
> django-safemigrate, which we use to separate which migrations may run
> before or after deployment. I intend follow along with this discussion, to
> see if there are reasonable ways to identify when these zero-downtime
> operations should be run compared with deployments.
>
> https://github.com/aspiredu/django-safemigrate
>
> On Wed, Dec 23, 2020 at 10:01 AM Paveł Tyślacki 
> wrote:
>
>> Want to share lib I work previously for migration safety with postgres
>> some time ago:
>> https://github.com/tbicr/django-pg-zero-downtime-migrations there are
>> also checker and replacement of unsafe mirations with safe replacement.
>>
>> ср, 23 дек. 2020 г. в 12:18, Adam Johnson :
>>
>>> Hi Tom,
>>>
>>> I love this idea and would like to help developers write safe migrations
>>> too.
>>>
>>> You didn't mention this pre-existing project:
>>> https://github.com/3YOURMIND/django-migration-linter . It might be
>>> worth checking how that works.
>>>
>>> I'm not sure what any public API could look like here. In
>>> django-linear-migrations (
>>> https://github.com/adamchainz/django-linear-migrations ) I tapped a bit
>>> into the MigrationLoader class, which is undocumented but fairly stable.
>>> Perhaps we could just document a few more of the migration internals?
>>>
>>> On Tue, 22 Dec 2020 at 22:58, Tom Forbes  wrote:
>>>
>>>> Hey,
>>>> A fairly common problem with large and/or highly trafficked web
>>>> applications is migration safety: you want your migrations to be
>>>> non-locking and instantaneous. The specifics are very database dependent
&

Integrating migrations with the check framework

[Tom Forbes]

Hey,
A fairly common problem with large and/or highly trafficked web
applications is migration safety: you want your migrations to be
non-locking and instantaneous. The specifics are very database dependent
but in Postgres simply altering the nullability of a column can incur
serious downtime depending on the size of the table in question. The same
thing goes for adding a new non-nullable column to a table or even just
creating an index without the `CONCURRENTLY` option.

Code review is one way of catching these issues but an automated solution
that runs as part of the test suite would be the preferred solution. I
don’t think using some kind of linter for this is a good idea as it doesn’t
have the context to decide if it _is_ a safe migration (i.e removing `NOT
NULL` from a column in the same migration that creates the column is safe).
A linter also doesn’t know which migrations are pending and which have been
applied.

Rails has the pretty nice strong_migrations Gem (
https://github.com/ankane/strong_migrations) which attempts to detect
dangerous migrations. Unless I’m missing something right now it seems quite
hard to add automated checks like this to Django migrations that can
inspect the full set of “pending migration operations". We have the
`pre_migrate` signal that takes a “plan” argument with the scary sounding
caveat that the passed object has no public API, but using a signal feels
kind of wrong. It should be part of a system check - i.e a warning that
says “your app may die if this migration is applied”.

I’d like to make this simpler to implement on a per-project basis (or in a
third party app) but I’m not sure how. It should be a system check but the
only suitable hook we have is the generic `register()` callback that takes
a set of `app_configs`, and I’d expect going from that to a pending set of
operations would be non trivial. A user-facing interface for this might be
a function that is called for every specific type of migration operation
with the operation itself and a list of all operations that would be
applied of “migrate” was run.

```
@some_migration_callback_decorator(AlterField)
def check_null(operation, pending_operations):
If operation.is_going_to_not_null:
if not table_being_created(operation.table_name,
pending_operations):
return Warning(f’Setting {operation.field_name} to NOT NULL may
cause downtime’)
```

I’m not sure if Django should ship migration warnings itself but having the
ability for projects to enforce arbitrary constraints on their migrations
might be an interesting feature?

Slightly related to https://code.djangoproject.com/ticket/31700, where we
want to add more contextual/coloured outputs for dangerous operations.

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFNZOJNoi583wUoQp%2BRBB_%2B3i_vqSPRbVNwhiU_OG7V2eL8x3g%40mail.gmail.com.

Re: Changing the discovery method for management commands

[Tom Forbes]

 I think it would be quite simple to add support for setuptools entrypoints
into the discovery code (using Poetry, or with equivalent setup.py/setup.cfg
statements):

[tool.poetry.plugins.django]
“setup_pytest" = “pytest_django.commands.setup_pytest”

Despite it being easy I’m not sure there is a compelling enough reason to
add this is. Can you  elaborate on exactly what `setup_pytest` would do? I
see use cases mainly around commands that modify files in your Django
project after installation, can you think of any other situations where
this would be useful?

Tom

On 19 Dec 2020 at 13:27:57, Diptesh Choudhuri 
wrote:

> As you said, most cases. The remaining cases have absolutely no way of
> defining management commands. One example is *pytest-django *which
> doesn't provide a django app but could benefit with a mangement command 
> *python
> manage.py setup_pytest*.
>
> On Saturday, December 19, 2020 at 2:11:32 PM UTC+5:30 Adam Johnson wrote:
>
>> Why? I don't see an impetus to avoid creating a Django app. In most use
>> cases there are related models or other Django bits to go with a management
>> command.
>>
>> On Sat, 19 Dec 2020 at 01:48, Diptesh Choudhuri 
>> wrote:
>>
>>> As of now, if you need to create a management command, it is necessary
>>> to create a file *app_name/management/commands/my_command.py, *and then
>>> add *app_name *to *INSTALLED_APPS *in *settings.py. *This prevents
>>> non-django packages from defining their own management commands, because it
>>> explicitly requires them to create a django app which just adds a bunch of
>>> unnecessary files to their source code.
>>>
>>> I propose we overhaul the existing management command discovery system
>>> so that it is easier to write management commands. Also I suggest we keep
>>> the default discoverer in place so as to maintain backwards compatibility.
>>>
>>> All of this will require documentation and I am ready to make a PR for
>>> that too. Please tell me if the idea is feasible, and I will get to work on
>>> it ASAP.
>>>
>>> Best
>>> Diptesh Choudhuri
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Django developers (Contributions to Django itself)" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to django-develop...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/django-developers/a7f9bf60-da49-404b-ac70-192220149059n%40googlegroups.com
>>> 
>>> .
>>>
>>
>>
>> --
>> Adam
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/8d85fcfa-8d1a-4e03-90e1-a00e07f6b6can%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFNZOJNfOPV1aHdohxCtruDo0wN%2BFZhF5229CEYsYRk0qB1C_g%40mail.gmail.com.

Re: App label with dot

[Tom Forbes]

Yes, a more friendly error message would be far better here! User-facing 
runtime exceptions like unpacking errors are never good.

This should be quite a simple change to make, I’d suggest opening a trac ticket 
first and we can go from there. 

Tom

> On 18 Dec 2020, at 16:38, Federico Capoano  wrote:
> 
> 
> Thanks Tim, would it be useful to have a more user friendly error message?
> 
> Il ven 18 dic 2020, 11:08 Tim Graham  ha scritto:
>> app_label "should be a valid Python identifier." Dots aren't allowed.
>> 
>> https://docs.djangoproject.com/en/stable/ref/applications/#django.apps.AppConfig.label
>> 
>>> On Friday, December 18, 2020 at 10:47:45 AM UTC-5 Federico Capoano wrote:
>>> Defining the app_label of an AppConfig with a dot, like "myproject.subapp" 
>>> is allowed, but raises an exception in the migration framework (too many 
>>> values to unpack).
>>> 
>>> Did anybody notice this?
>>> 
>>> Is it supposed to be allowed or not at all?
>>> 
>>> Best regards
>>> Federico Capoano
>> 
>> -- 
>> You received this message because you are subscribed to a topic in the 
>> Google Groups "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this topic, visit 
>> https://groups.google.com/d/topic/django-developers/3cbMviLrSzI/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email to 
>> django-developers+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-developers/134214d9-e3c1-4f74-9920-f0a62d4d675bn%40googlegroups.com.
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/CAERYH6XvS3_z-akSZw_xy8FRHbdZoxeOYN3D_fVyRJ%3D7B9zUFA%40mail.gmail.com.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/98CF92C9-1E9D-4F56-9D4E-A6F78737002E%40tomforb.es.

Re: Replace the default django test runner to a more robust testing framework (like pytest)

[Tom Forbes]

I would be very much in favour of this. The developer experience of pytest is 
far better than the unittest module, the tests look more Pythonic and there is 
a lot more of an ecosystem we can tap into.

There was some recent work done about adding timings to the Djanfo test suite 
as part of the multi-process test runner. This was way more difficult than it 
should have been.

Unittest, along with the logging module, are very old and archaic stdlib 
modules that are ok-ish, but definitely hobbled by legacy interfaces and a 
“Java inspired” pre-pythonic code style.

I think it’s safe to say that most of the Python ecosystem has settled on 
pytest, and if we go along with that as well then we can reduce our maintenance 
burden whilst making our users happier.

Tom

> On 15 Dec 2020, at 13:52, Diptesh Choudhuri  
> wrote:
> 
> Though the django default runner gets the job done, I think its high time to 
> leverage the power of dedicated frameworks for testing django apps. I will 
> take up the case of pytest here because other frameworks like nose are not 
> widely used.
> 
> For everyone who has used pytest for testing in django, I think we will all 
> agree that pytest is much better suited to this task. 
> The passing tests look better.
> Failing tests are much more verbose with where the error actually occurred.
> Parameterizing the test cases using the default runner requires a third party 
> library and looks difficult (I will be honest, I haven't used this package 
> myself at all). 
> Injecting runtime conditions into the tests is quite difficult, if not 
> impossible, and requires a lot of hacky workarounds.
> I am not sure about this, but the default runner also doesn't cache results 
> from previously run tests, making retesting unchanged tests much slower.
> This might sound non-trivial but typing pytest is much more easier than 
> typing python manage.py test into the command line.
> IDEs like VSCode can discover tests written for pytest but not the ones 
> written using the default runner. This might not sound a big deal, but it is 
> infinitely more easier to run tests using the GUI that these IDEs have. They 
> provide real-time feedback and show exactly which tests have passed and what 
> haven't in the code source itself. Here is a picture showing what I mean.
> The directory structure created by python manage.py startapp my_app creates a 
> tests.py file. This goes against testing best practices which say that all 
> tests should be aggregated at one place (as opposed to inside individual 
> apps).
> A single tests.py file encourages new, inexperienced developers to write all 
> their tests in a single file. This, yet again, goes against testing best 
> practices which put a lot of emphasis on modularization of tests. Models 
> should be tested in test_models.py, views in test_views.py etc. 
> Though unittest's setUp and tearDown work fine, I personally find that 
> pytest's fixture system provides a bit more freedom. I might want to put all 
> my test methods in one class, but I might not want to run the setUp and 
> tearDown for every test method. (Let me know if this point is a bit unclear.)
> pytest and pytest-django are quire mature frameworks and don't need a lot of 
> changes. We could make it easier to setup tests right out of the box. For 
> example, django-admin startproject my_project could create a pytest.ini right 
> out of the box in the project root. 
> 
> Most of my focus would be on rewriting and adding documentation. Even though 
> widely used, pytest and pytest-django are pretty difficult to get started 
> with for a newbie. I intend to fix this by adding a lot of examples to the 
> docs (maybe even make a separate repository for this).
> 
> NOTE: I intend to work on this project for GSoC 2021. I am just scouting 
> beforehand to see how the community receives it.
> 
> Please let me know what you think of this idea. Any critiques will be welcome.
> 
> Best,
> Diptesh
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/8c5e0baa-5104-4876-a936-8792b3c8d5b8n%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/FDB4BE6D-226A-4B23-8A75-27267EC500D3%40tomforb.es.

Re: Primary Key AutoField -> UUID Field

[Tom Forbes]

Uri, I would definitely suggest using an actual UUID rather than rolling your 
own ad-hoc UUID implementation, even if your database does not support a native 
UUID type.

Brian, there is some work ongoing that should make this possible. We will add a 
setting to control the type of the auto-created primary key type. Initially it 
will be restricted to AutoField subclasses only, but the intent is to make 
UUIDFields work with this as well. 

The first bit of work is here: https://github.com/django/django/pull/13179

Tom

> On 14 Nov 2020, at 16:54, אורי  wrote:
> 
> 
> Hi Brian,
> 
> In Speedy Net we use a randomly generated string of digits, either 15 digits 
> or 20 digits, as a primary key in all our models. We never use 
> auto-incrementing integers as primary keys. You can take a look at our 
> implementation, especially class BaseModel which is the base class for all 
> our models, and class TimeStampedModel (most of our models inherit from 
> TimeStampedModel):
> 
> https://github.com/speedy-net/speedy-net/blob/master/speedy/core/base/models.py
> 
> We also defined managers with querysets in which bulk_create() and delete() 
> are disabled.
> 
> Uri Rodberg, Speedy Net.
> אורי
> u...@speedy.net
> 
> 
>> On Sat, Nov 14, 2020 at 6:32 PM Brian Carter  
>> wrote:
>> Feature Request:
>> 
>> I like to use the UUIDField as the primary key in my models, and I think it 
>> would be nice to have django expose a setting in the settings.py to override 
>> the default primary key on models. 
>> 
>> Currently, if I don’t specify a field as the primary key, Django 
>> automatically uses an AutoField (auto-incrementing integer). I’d like to be 
>> able to automatically use a UUID field, but then still be overridden in a 
>> specific model if I specify a different primary key.
>> 
>> Brian Carter
>> brianrcarte...@gmail.com
>> 
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers  (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-developers+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-developers/EAF4690E-C862-4C68-A7DD-C2979F19BC3C%40gmail.com.
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/CABD5YeFqeRfd6%2BMuMdzsagvjKdNu-iEDND-omuePX9TEw9TGTw%40mail.gmail.com.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/833A8A9F-14A2-4C37-8780-778F983C4B9D%40tomforb.es.

Re: Fail test runs on unraisable exceptions?

[Tom Forbes]

Would we need to run “gc.collect” after each test to make it deterministic?

I’m all for this change - I think it’s a sensible thing to do.

Tom

> On 30 Oct 2020, at 16:30, Adam Johnson  wrote:
> 
> 
> -Werror transforms warnings into exceptions. This is what happens with the 
> asyncio case yes - but the exception is unraisable so python prints the stack 
> trace and carries on as if no problem occurred.
> 
> Focussing only on the asyncio case may be counter productive. They are 
> warnings by default and it requires a warning filter to convert them to 
> exceptions, which are then unraisable.
> 
> It’s easy to create unraisable exceptions through other mechanisms, such as 
> in __del__ methods, but in practice thankfully rare. But I think in general 
> any unraisable exception represents something you would want to know about in 
> your test suite - just like a normal raisable exception.
> 
>> On Fri, 30 Oct 2020 at 13:38, Carlton Gibson  
>> wrote:
>> 
>> 
>>> On 30 Oct 2020, at 14:20, Adam Johnson  wrote:
>>> 
>>> My proposal is to put this into the test runner for Django users
>> 
>> OK, I misunderstood. Gotcha. 
>> 
>> >  … I'm not sure if failing the test run would always make sense.
>> 
>> Always no, for the reasons given. Always awaiting every coroutine can be 
>> hard, and not always (often not 🤔) significant. 
>> 
>> Let me rephrase, why isn’t python -Werror (or a more nuanced version of 
>> that) sufficient? 
>> 
>> 
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-developers+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-developers/9AD9DB42-5B0D-41CA-995F-2DF4CD92783A%40gmail.com.
> -- 
> Adam
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/CAMyDDM1cDG0vE_EYiLJ_5ih7%3D4Zh4Keofk3oQRx%2BFLFqEhM-ig%40mail.gmail.com.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/1D214EE7-B48E-438C-8036-89E375AEAF85%40tomforb.es.

Re: Welcome email

[Tom Forbes]

 The export finally finished after a whole day running.

The gzip compressed mbox:
https://www.icloud.com/iclouddrive/09XBHCr5L5BNcOBLob_HA2x6g
<https://www.icloud.com/iclouddrive/09XBHCr5L5BNcOBLob_HA2x6g#django-messages>
The entire raw output from the tool:
https://www.icloud.com/iclouddrive/0LY_ccOgG6VjfV3INexGMXwZQ
<https://www.icloud.com/iclouddrive/0LY_ccOgG6VjfV3INexGMXwZQ#raw_django_messages>

We’ve got 56,282 total messages. I think the script from Discourse I linked
above would be the perfect way to import these as it creates the users on
demand and isn’t limited by the external Discourse API. I’ll leave that to
anyone who has access and is willing (Andrew?).

Tom

On 29 Oct 2020 at 16:43:07, Tom Forbes  wrote:

> It actually might be simpler than I suggested. Discourse has a script to
> import an mbox:
> https://github.com/discourse/discourse/blob/master/script/import_scripts/mbox.rb
>
> I’m running the box importer script now and it appears to work fine. While
> some people might have historical mbox files, it might be better to get the
> source of truth from Google Groups?
>
> Tom
>
> On 29 Oct 2020 at 16:11:04, Andrew Godwin  wrote:
>
>> I'd be more than happy to assist a trial of moving things to the forum;
>> we've had it running for over a year now, and I feel it's a much easier way
>> to run a community.
>>
>> Among other things, we can:
>> - Move posts to the right forum when they post in the wrong one (rather
>> than emailing back and saying "please post again over here")
>> - Delete people's personal information when they accidentally post it
>> - Remove CoC-violating posts and not leave any history
>>
>> Either I can try the import out if someone pops the right permissions
>> over to me, or I'm happy to supervise enough forum API access for you to
>> try it, Tom.
>>
>> Andrew
>>
>> On Thursday, October 29, 2020 at 9:23:45 AM UTC-6 carlton...@gmail.com
>> wrote:
>>
>>> I don’t have any controls here. I’m pretty sure Florian would be the
>>> most likely candidate. (No doubt it’s Jacob.)
>>>
>>>
>>> --
>> You received this message because you are subscribed to the Google Groups
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to django-developers+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/django-developers/44e041d6-9397-4a73-87a0-e4643c034db0n%40googlegroups.com
>> <https://groups.google.com/d/msgid/django-developers/44e041d6-9397-4a73-87a0-e4643c034db0n%40googlegroups.com?utm_medium=email&utm_source=footer>
>> .
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFNZOJOAVFTNe%2BdJBjoJ8MWChswmLAZiGzqRPmPKYuVwuMqqiw%40mail.gmail.com.

Re: Welcome email

[Tom Forbes]

 It actually might be simpler than I suggested. Discourse has a script to
import an mbox:
https://github.com/discourse/discourse/blob/master/script/import_scripts/mbox.rb

I’m running the box importer script now and it appears to work fine. While
some people might have historical mbox files, it might be better to get the
source of truth from Google Groups?

Tom

On 29 Oct 2020 at 16:11:04, Andrew Godwin  wrote:

> I'd be more than happy to assist a trial of moving things to the forum;
> we've had it running for over a year now, and I feel it's a much easier way
> to run a community.
>
> Among other things, we can:
> - Move posts to the right forum when they post in the wrong one (rather
> than emailing back and saying "please post again over here")
> - Delete people's personal information when they accidentally post it
> - Remove CoC-violating posts and not leave any history
>
> Either I can try the import out if someone pops the right permissions over
> to me, or I'm happy to supervise enough forum API access for you to try it,
> Tom.
>
> Andrew
>
> On Thursday, October 29, 2020 at 9:23:45 AM UTC-6 carlton...@gmail.com
> wrote:
>
>> I don’t have any controls here. I’m pretty sure Florian would be the most
>> likely candidate. (No doubt it’s Jacob.)
>>
>>
>> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/44e041d6-9397-4a73-87a0-e4643c034db0n%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFNZOJMRmFZrTsWAJdtoZLaTh8fz2NHpdnE%3DFV-sknb7MDtohg%40mail.gmail.com.

Re: Welcome email

[Tom Forbes]

Amazing, thank you!

If I get access I’ll create a specific section and try importing the latest 100 
topics. If you could try seeing if we can grab the messages from Google Takeout 
that would be fantastic, otherwise I’ll try the linked tool.

I’m actually not sure who owns this group. Who created it originally? This 
might be lost in the sands of time, but it might be good to find out if we can 
as this group might disappear if their account is suspended/deleted for 
whatever reason.

Tom

> On 29 Oct 2020, at 15:14, Carlton Gibson  wrote:
> 
> Hey Tom. 
> 
> I pinged the Forum Powers That Be™ to see about permissions for you. 
> 
>> On 29 Oct 2020, at 16:09, Tom Forbes  wrote:
>> 
>> One alternative to the scraper might be using Google Takeout 🥡  
>> (https://takeout.google.com/), but it’s unclear if that would export the 
>> messages themselves. And it would require the actual owner of the group to 
>> do that with their account.
> 
> Difficulty of getting the messages out was what led me to wonder if someone 
> had an MBOX somewhere with the whole thing in… (But I didn’t try anything.) 
> 
> I suspect if we were to map the top 50-100 posters we’d more than cover the 
> lion’s share of the history, mapping messages to authors.
> 
> If you’re keen to try a PoC, that would be amazing. 
> Thanks. 
> C. 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/5ECC0881-62D0-4683-93E9-953D560D6085%40gmail.com.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/44BA0A69-BA7E-4BBE-AC8D-6EAC6930A380%40tomforb.es.

Re: Welcome email

[Tom Forbes]

One alternative to the scraper might be using Google Takeout 🥡  
(https://takeout.google.com/), but it’s unclear if that would export the 
messages themselves. And it would require the actual owner of the group to do 
that with their account.

Tom

> On 29 Oct 2020, at 15:05, Tom Forbes  wrote:
> 
> I think a google groups to discourse import is more than possible and would 
> be good. I was quite skeptical about discourse at first, but it’s really 
> quite awesome and definitely a set up from google groups.
> 
> After a brief bit of searching I can see there is a tool to export all 
> messages from a google group[1]. Discourse has a pretty good API but lacks 
> the ability to create messages as another user using an admin API key, so we 
> can do the import one of two ways. For each groups message we:
> 
> 1. Take the email and create a pseudo-user with a specific email address 
> (t...@tomforb.es.groups.import), then post as that user
> 2. Attempt to look up if the user already exists in Discourse, create an API 
> token for them and post the message as that user.
> 
> Either way we get a central, searchable place for these messages. The first 
> is far simpler but we loose attribution for users registered on both places. 
> The second required adding a specific API key to existing users, which might 
> trigger some confusing emails.
> 
> I have some time free in the next few days and I’d love to do a proof of 
> concept to see if it’s feasible, but I’d need elevated permissions on the 
> forum.
> 
> Tom
> 
> 1. https://github.com/icy/google-group-crawler
> 
>>> On 29 Oct 2020, at 10:54, Carlton Gibson  wrote:
>>> 
>> 
>> Hi Jure. Thanks for the follow-up. 
>> 
>> I'm not sure there are SO MANY misplaced emails but bracketing that…
>> 
>> I spoke to the Ops team. The group IS moderated BUT there's no way to 
>> communicate with someone that they've sent to the wrong address, so unless 
>> it's actual spam (vs usage question wrong place — I hope that distinction 
>> makes sense) it's let through. (Adam's quick-fire "You've found the wrong 
>> list..." being more friendly than the silent abyss.) 
>> 
>> So that's the "serious issue with the moderation queue" you speak of. 
>> (We COULD MAYBE change policy there but…)
>> 
>> 
>> PAUSE FOR BREATH
>> 
>> 
>> We could also decamp on mass to the forum (forum.djangoproject.com)
>> 
>> That has Topics. Someone posts in Internals > Mentorship with a usage 
>> question and moderators just move it to Using Django. 
>> Very simple. 
>> 
>> The UI is also great. 
>> The notifications are customizable.
>> 
>> It's Open Source. 
>> 
>> I'll mention one more... 
>> 
>> When you're typing a post, it searches the history to show you similar 
>> posts. 
>> 
>> The only downside would be loosing the archive here. 
>> 
>> I search that. I wonder how many others do? (Some definitely) 
>> 
>> Could we bit that bullet? 
>> 
>> Could we export the data from here and move it? 
>> https://support.google.com/groups/answer/9975859?hl=en
>> Anyone happen to have the full list history already?
>> 
>> I wonder what an import/export would look like. 
>> 
>> 
>> AND BREATHE
>> 
>> (😀)
>> 
>> Kind Regards,
>> 
>> Carlton
>> 
>> 
>> 
>> 
>> 
>>> On Tuesday, 27 October 2020 at 06:40:02 UTC+1 jure.er...@gmail.com wrote:
>>> I think this thread was abandoned, but wrongfully so. Frequency of 
>>> unrelated posts is increasing lately.
>>> 
>>> Are there any serious issues with the moderation queue Google provides?
>>> 
>>> Reminder: https://support.google.com/groups/answer/2466386?hl=en
>>> 
>>> 
>>> LP,
>>> Jure
>>> 
>>>> On Saturday, 25 July 2020 at 00:58:26 UTC+2 m...@kye.id.au wrote:
>>>> This issue is exacerbated by similarly unaware / uncaring mailing list 
>>>> users responding to support requests.
>>>> 
>>>> What is the best path forward here?
>>>> 
>>>>>> On 10 Jul 2020, at 3:09 pm, Jure Erznožnik  wrote:
>>>>>> 
>>>>> 
>>>> 
>>>>> I too volunteer for the screening job. 
>>>>> 
>>>>> LP,
>>>>> Jure
>>>>> 
>>>>> On 09/07/2020 16:10, Peter Inglesby wrote:
>>>>>> Hi folks,
>>>>>> 
>

Re: Welcome email

[Tom Forbes]

I think a google groups to discourse import is more than possible and would be 
good. I was quite skeptical about discourse at first, but it’s really quite 
awesome and definitely a set up from google groups.

After a brief bit of searching I can see there is a tool to export all messages 
from a google group[1]. Discourse has a pretty good API but lacks the ability 
to create messages as another user using an admin API key, so we can do the 
import one of two ways. For each groups message we:

1. Take the email and create a pseudo-user with a specific email address 
(t...@tomforb.es.groups.import), then post as that user
2. Attempt to look up if the user already exists in Discourse, create an API 
token for them and post the message as that user.

Either way we get a central, searchable place for these messages. The first is 
far simpler but we loose attribution for users registered on both places. The 
second required adding a specific API key to existing users, which might 
trigger some confusing emails.

I have some time free in the next few days and I’d love to do a proof of 
concept to see if it’s feasible, but I’d need elevated permissions on the forum.

Tom

1. https://github.com/icy/google-group-crawler

> On 29 Oct 2020, at 10:54, Carlton Gibson  wrote:
> 
> 
> Hi Jure. Thanks for the follow-up. 
> 
> I'm not sure there are SO MANY misplaced emails but bracketing that…
> 
> I spoke to the Ops team. The group IS moderated BUT there's no way to 
> communicate with someone that they've sent to the wrong address, so unless 
> it's actual spam (vs usage question wrong place — I hope that distinction 
> makes sense) it's let through. (Adam's quick-fire "You've found the wrong 
> list..." being more friendly than the silent abyss.) 
> 
> So that's the "serious issue with the moderation queue" you speak of. 
> (We COULD MAYBE change policy there but…)
> 
> 
> PAUSE FOR BREATH
> 
> 
> We could also decamp on mass to the forum (forum.djangoproject.com)
> 
> That has Topics. Someone posts in Internals > Mentorship with a usage 
> question and moderators just move it to Using Django. 
> Very simple. 
> 
> The UI is also great. 
> The notifications are customizable.
> 
> It's Open Source. 
> 
> I'll mention one more... 
> 
> When you're typing a post, it searches the history to show you similar posts. 
> 
> The only downside would be loosing the archive here. 
> 
> I search that. I wonder how many others do? (Some definitely) 
> 
> Could we bit that bullet? 
> 
> Could we export the data from here and move it? 
> https://support.google.com/groups/answer/9975859?hl=en
> Anyone happen to have the full list history already?
> 
> I wonder what an import/export would look like. 
> 
> 
> AND BREATHE
> 
> (😀)
> 
> Kind Regards,
> 
> Carlton
> 
> 
> 
> 
> 
>> On Tuesday, 27 October 2020 at 06:40:02 UTC+1 jure.er...@gmail.com wrote:
>> I think this thread was abandoned, but wrongfully so. Frequency of unrelated 
>> posts is increasing lately.
>> 
>> Are there any serious issues with the moderation queue Google provides?
>> 
>> Reminder: https://support.google.com/groups/answer/2466386?hl=en
>> 
>> 
>> LP,
>> Jure
>> 
>>> On Saturday, 25 July 2020 at 00:58:26 UTC+2 m...@kye.id.au wrote:
>>> This issue is exacerbated by similarly unaware / uncaring mailing list 
>>> users responding to support requests.
>>> 
>>> What is the best path forward here?
>>> 
> On 10 Jul 2020, at 3:09 pm, Jure Erznožnik  wrote:
> 
 
>>> 
 I too volunteer for the screening job. 
 
 LP,
 Jure
 
 On 09/07/2020 16:10, Peter Inglesby wrote:
> Hi folks,
> 
> Is there any moderation for posts from new users?  It can be enabled, and 
> I'd be willing to be part of a team that filters posts from new users.
> 
> All the best,
> 
> Peter.
> 
> On Thu, 9 Jul 2020 at 13:59, Adam Johnson  wrote:
>> I think that's a good improvement, not bikeshedding :)
>> 
>> On Thu, 9 Jul 2020 at 13:17, Shai Berger  wrote:
>>> Sorry for the bike-shedding, but I think the text should drop the
>>> "using Django" language. The people who come here with these questions
>>> clearly think of themselves as developers, not users.
>>> 
>>> IMO It should go something like,
>>> 
>>> Welcome to django-developers, the mailing list for discussion
>>> of the development of Django itself.
>>> 
>>> This mailing list is not for support developing apps and
>>> websites with Django. For support, please follow the "Getting
>>> Help" page: https://docs.djangoproject.com/en/3.0/faq/help/ .
>>> This page will direct you to the django-users mailing list or
>>> other resources, so you can find people who are willing to
>>> support you, and to ask your question in a way that makes it
>>> easy for them to answer.
>>> 
>>> Thanks,
>>> 
>>> The Django Community
>>>

Re: Instance Based Management?

[Tom Forbes]

I think what Mathew really wants is support for table partitioning. You can get 
this right now with this library[1] for postgres. I’m not sure if this makes 
sense to add to core, however support is quite broad (MYSQL, MariaDB, Postgres 
and Oracle).

1. 
https://django-postgres-extra.readthedocs.io/en/master/table_partitioning.html

Tom

> On 27 Oct 2020, at 05:30, Jure Erznožnik  wrote:
> 
> 
> Hi Matthew,
> 
> I think you found the wrong mailing list for this question. Might I suggest 
> you try django-us...@googlegroups.com? The question seems better suited there.
> 
> That said, I don't know why you wouldn't want to use foreign keys in this 
> scenario, but Django does support a thing called content types for what you 
> seem to be suggesting. There's a section on that page called "Generic 
> relations". 
> 
> Have a look.
> 
> LP,
> Jure
> 
> On 27. 10. 20 01:21, Matthew Amstutz wrote:
>> 
>> Hello, I was wondering about instance based management. If I'm wrong, please 
>> tell me. 
>> 
>> When we have users and user generated content in a large database, query 
>> times are increased significantly. Why is there no instance based manager 
>> (like the models.Manager()) that basically generates a table for each user 
>> and queries ONLY that table? Would that not just flatten the database 
>> instead of increasing it's size? For example, if we have 1,000,000 users all 
>> of which generate at least 10 posts per day and one of the users only 
>> generates 5 in the span of 10 days, unless we have a many to many field or 
>> something to hold those five posts, the query time to find their posts would 
>> be ridiculous.
>> 
>> So if we have a table generated for each user that holds arbitrary 
>> connections to anything they generate, it would in theory cut query times 
>> significantly. Why is there no feature like this? Again, if I'm wrong please 
>> tell me but the amount of tables doesn't matter and instead the data they 
>> hold does so, in my understanding, 1,000,000,000 posts will always be the 
>> size of 1,000,000,000 posts no matter their organization.
>> 
>> I've got ideas on implementation and even asyncronous supports as well as 
>> customization but I have no idea how to bring this up to the django 
>> developers and I'm not even sure it would work (though, no matter how hard I 
>> try, I can't see anything wrong with it). 
>> 
>> Let me know your input and if there's a way I can ask the django devs about 
>> this and possibly even suggest a few things pertaining to it. I'd like to 
>> help make django the best it can be and if this works and we can implement 
>> it, django will be very fast with user generated content.
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-developers+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-developers/7e36ded7-2f3d-43c2-881c-cbc75c80b5c2n%40googlegroups.com.
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/944bfa26-a0bf-69bb-f76a-c0654910eb20%40gmail.com.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/BA1C3DF2-3BD1-473F-BA01-016ACA251D5D%40tomforb.es.

Re: Logging in from one browser logs me out from other browsers (after any change in PBKDF2PasswordHasher.iterations)

[Tom Forbes]

You might have a point regarding the frequency of bumping the PBKDF iteration 
setting. Is bumping it 5 times in 13 months really required? On the other hand 
you might want to consider staying on the LTS releases and avoid issues such as 
this, and the issue you’re describing is quite niche.

However, I would say that based on your previous posts to this mailing lists 
around authentication that you are definitely in need of some form of federated 
login/SSO for your several web properties. That would certainly alleviate this 
issue and some of the other problems you’ve reported here.

Tom 

> On 3 Sep 2020, at 09:47, ⁨אורי⁩ <⁨u...@speedy.net⁩> wrote:
> 
> Hi,
> 
> On Thu, Sep 3, 2020 at 11:23 AM Shai Berger  > wrote:
> 
> Please be aware that this is a security issue. The passwords are
> encrypted as protection for the case that they fall into the hands of
> an attacker, but for this protection to be effective, it must stay hard
> and costly to brute-force them. The number of iterations is enlarged in
> order to keep this cost up with the improvements of available hardware.
> If you intend to keep a user's password un-updated for many years, it's
> almost as bad as keeping it in plaintext -- in 10-15 years, we'd expect
> the number of iterations in current Django to be grossly insufficient.
> 
> I don't intend to keep the settings of now for 10-15 years. But since I 
> launched Speedy Net in Django 1.11 in production 13 months ago, I upgraded to 
> 2.0, 2.1, 2.2, 3.0 and now 3.1. These are 5 major version upgrades in 13 
> months. I don't see a reason why the number of iterations should have changed 
> 5 times in 13 months. Even if I would upgrade Django every 8 months, I prefer 
> to keep the number of iterations and change it every 2-3 years, if this logs 
> out users. I'm not sure if I'll write a blog post, but you can see our patch 
> on GitHub:
> 
> https://github.com/speedy-net/speedy-net/blob/master/speedy/core/patches/session_patches.py
>  
> 
> 
> I wish I knew about this issue before and then I would have patched something 
> like this before, before causing this to change 5 times in production.
> 
> אורי.
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/CABD5YeFp_9btTbguvBDyUxCaaYcX4VD9thsddp7hdRqVL%2BJnuw%40mail.gmail.com
>  
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/C8F40769-AC5B-41FE-8A17-BDACCC39CE2C%40tomforb.es.

Re: Making max_length argument optional

[Tom Forbes]

I’m not a fan of implicit max_lengths. Is having to add a keyword argument to a 
model field really that much of a burden? And we also would likely never be 
able to change the default without headaches.

> On 12 Aug 2020, at 13:19, t...@carrick.eu  wrote:
> 
> I'd like to revive this discussion and try to come to a consensus as it's 
> something I find myself wishing for (for Postgres in particular).
> 
> My suggestion, after reading through everything:
> 
> Give CharField a default max_length that is consistent across all vendors. It 
> doesn't really matter what the number is other than that it should be large 
> enough to be useful but small enough to work everywhere. I think 100 or 255 
> are both fine options.
> 
> If you set max_length=None explicitly, on Postgres this will use an unlimited 
> varchar, on everything else will raise an exception on migrate.
> 
> Any thoughts?
> 
> Cheers,
> Tom
> 
> On Saturday, March 5, 2016 at 2:13:14 PM UTC+1 Shai Berger wrote:
> On Saturday 05 March 2016 02:24:17 Loïc Bistuer wrote: 
> > I’m not too keen on a contrib.pg  field. CharField is 
> > the base class of many 
> > fields, a __init__ kwarg approach such as max_length=None allows us to 
> > reach those as well. 
> > 
> 
> That's a good point; Can we enable max_length=None in a mixin? 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/dfaaa9d3-dff3-46a3-899f-dd7f4eddfe87n%40googlegroups.com
>  
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/468B12C0-2864-4221-9985-044F340E56E1%40tomforb.es.

Re: Django 4.0 sessions

[Tom Forbes]

> Is it possible to change the sessions deprecation Django version so that 
> users will have about 2 or 3 years to convert their sessions?


Any session (or signature) that uses sha1 will no longer be valid in Django 
4.0. Keeping insecure hash algorithms around for another 2 to 3 years at the 
framework level, especially when used for sessions, is probably not a good idea.

As a side note Uri, it’s good practice to keep your messages concise when 
posting. The very long first paragraph in your message containing 4 or 5 
questions is hard to parse and respond to, and not all of it appears relevant.

Tom

> On 9 Aug 2020, at 13:37, אורי  wrote:
> 
> 
> Django developers,
> 
> I'm using Django 3.0 with sessions on my websites, Speedy Net and Speedy 
> Match. I understand that sessions as they are now (in Django 3.0) will be 
> changed and removed in Django 4.0. I would like to know how will it affect 
> the users of my websites? Will it log out any user who didn't visit the site 
> from Django 3.1 to Django 3.2, or will it also log out any user who didn't 
> login again to the website using Django 3.1 and 3.2? I use persistent 
> sessions and cookies for ~30 years, and I expect sessions to keep working 
> when I upgrade Django. I think I can handle logging out users who didn't 
> visit the site for ~16 months, but I can't handle users who visited the site 
> as authenticated users but didn't login again in more than ~16 months - this 
> will mean that when I upgrade (eventually) to Django 4.0, most of my users 
> will suddenly be logged out (who should be logged in). Is there a way to 
> overcome this? Are sessions objects created by the website other than when 
> users log in? This deprecation can cause me not to want to upgrade Django to 
> 4.0, which is a shame. I upgraded all versions of Django from 1.8 to 3.0, and 
> I'm about to upgrade to 3.1 as well. I also had a problem when I upgraded to 
> Django 2.1, which affected my users, and there was a bug for about 6 weeks on 
> my website, because of introducing SESSION_COOKIE_SAMESITE with the default 
> 'Lax' in Django 2.1 
> [https://stackoverflow.com/questions/59298548/set-cookie-is-not-working-in-chrome-with-two-websites].
>  Due to this bug my website didn't work properly for about 6 weeks. And I 
> don't want to cause more problems when I upgrade Django to 4.0. Actually I 
> would like the window to be wider - for example, log out users who didn't 
> visit the website for 2 or 3 years, and anyway convert their sessions 
> automatically without forcing them to login again (if they visited my 
> websites during this time). Will sessions be converted automatically or can I 
> cause them to be converted automatically to the new format and hashing 
> algorithm, while using Django 3.1 and 3.2, before I upgrade Django to 4.0?
> 
> By the way, both my websites are configured to log in and log out users 
> together - if they log in or log out from one website, they should be 
> automatically logged in or logged out from the other domain too.
> 
> Is it possible to change the sessions deprecation Django version so that 
> users will have about 2 or 3 years to convert their sessions?
> 
> Thanks,
> 
> אורי
> (Uri)
> u...@speedy.net
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/CABD5YeHY_%3DBaZyi0L%2Bp0tFmrJsFZp3V9n3JfR%3D925-440RnsRA%40mail.gmail.com.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/92E6F300-E63C-4C21-8BF0-F4309FD7E0CF%40tomforb.es.

Re: contrib.auth.Group comment

[Tom Forbes]

I think this makes sense - I would create a ticket in Trac.

On Wednesday, 17 June 2020 at 10:55:44 UTC+1 mike.d...@gmail.com wrote:

> Auth Groups are fabulous for designing access control into a project. 
> However, success in such design relies heavily on either user memory or 
> documentation. 
>
> In my case it has to be documentation. 
>
> To that end I believe the contrib.auth.Group model needs a comment field. 
> That would let business administrators document what each group or role 
> might be responsible for and then system admins could assign correct 
> permissions accordingly.
>
> The change required in contrib.auth.models.py is a new line inserted in 
> the Group class between "name" and "permissions" as follows ...
>
> comment = models.TextField(_("comment"), default="", blank=True)
>
> I see this as a security improvement because human error in assigning 
> group permissions can make security holes.
>
> Cheers
>
> Mike
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/5f8675ba-a879-4ac9-aa99-b23370422f77n%40googlegroups.com.

Re: Defaulting to use BigAutoField in models

[Tom Forbes]

I have an initial MR for phase 1 here: 
https://github.com/django/django/pull/13179. It's lacking docs but I think 
the rest of the points are covered.

It still needs docs, but a few observations/thoughts:

`UUIDField` is not useful at all without a `default` being set. It's likely 
not safe to add a `uuid.uuid4` default to the field, so documentation will 
have to show creating a `UUIDField` subclass. This might cause big 
headaches with migrations though, if the field is moved around.

M2MFields are still possibly an issue - while you might want UUIDFields for 
all models you also might want plain `BigAutoField's` for M2M through 
tables for efficiency. 

This error message[1] is not helpful and is more likely to crop up now. Is 
it worth adding a ticket to somehow improve the message?

If I change the `MODEL_DEFAULT_PK` setting to `django.db.models.UUIDField` 
and run a test on Postgres I end up with a weird situation:

from django.contrib.sites.models import Site
Site._meta.pk

Site.objects.values_list("pk")


The "UUIDField.to_python" method is also never invoked. I'm not clear if 
this is something to do with the Django test suite or indicative of a 
deeper problem. 

1. https://code.djangoproject.com/ticket/30526

On Sunday, 28 June 2020 at 14:27:28 UTC+1 Tom Forbes wrote:

> I spent some time last week experimenting with my patch, the biggest issue 
> I could find was that under some situations we add explicit casts to array 
> fields in Postgres based on the *expected* PK field type (i.e `WHERE 
> [1,2,3]::int @> [1,2,3]::bigint`) which causes the query to be invalid. The 
> casts all look redundant however working out when they are redundant is 
> pretty complex, so I'm not sure if this approach has any legs.
>
> > Regarding third party apps a solution could be to allow a per-app config 
> option or encourage them to explicitly choose which primary key they use 
> for their model but I'm not a big fan of baking logic in the migration 
> framework to treat some fields in a special way.
>
> Auto-generated M2M tables are an issue here, we would need to add some 
> option to ManyToManyField() to allow passing in a PK field type. But I like 
> the approach of just saying "if you ship migrations, be explicit about your 
> primary keys" as it seems pretty simple.
>
> Overall I think your two phase plan seems like a good approach 🙌 - lets 
> do it?
>
> On Thursday, 18 June 2020 at 15:18:35 UTC+1 charettes wrote:
>
>> +1 from me as well but I don't think we should add any logic to the 
>> migration framework to handle the transition.
>>
>> I think the release plan should be something along the following
>>
>> Phase 1:
>> - New projects are generated with MODEL_DEFAULT_PK = 
>> 'django.db.models.BigAutoField'
>> - A system check or warning is emitted when the setting is not defined 
>> and MODEL_DEFAULT_PK default to 'django.db.models.AutoField' during the 
>> transition period. The warning should warn that the value will default to 
>> 'django.db.models.AutoField' in a future version of Django.
>> - An upgrade path is mentioned in the docs to mention that explicit ` id 
>> = AutoField()` must be assigned to preserve previous behaviour and avoid 
>> migrations. This one of the thing projects such as django-codemod could 
>> really help with[0]
>>
>> Phase 2:
>> - settings.MODEL_DEFAULT_PK now defaults to 
>> 'django.db.models.BigAutoField'
>> - System check/warning about undefined MODEL_DEFAULT_PK setting is 
>> removed.
>> - Let the migration framework generate migrations just like it normally 
>> would for projects that didn't follow the documented migration path.
>> - Optionally add a system check that warns about usage of `AutoField` 
>> because of its possible overflow.
>>
>> > I think we should restrict the setting between normal and big auto 
>> fields only. Allowing UUID's would be changing the type, with the potential 
>> for havoc with code incompalities throughout django. It's also not possible 
>> to migrate tables over to the new type.
>>
>> I'm not sure I agree here. For folks that are setting a up a new project 
>> starting with UUIDField primary keys can be useful and if we're adding a 
>> setting for this purpose I think we should it make it as flexible as 
>> possible.
>>
>>
>> [0] https://github.com/browniebroke/django-codemod
>>
>> Le jeudi 11 juin 2020 11:28:59 UTC-4, Tom Forbes a écrit :
>>>
>>> I’d like to re-propose switching Django to use BigAutoField’s rather 
>>> than the current AutoField. This has been p

Re: Defaulting to use BigAutoField in models

[Tom Forbes]

I spent some time last week experimenting with my patch, the biggest issue 
I could find was that under some situations we add explicit casts to array 
fields in Postgres based on the *expected* PK field type (i.e `WHERE 
[1,2,3]::int @> [1,2,3]::bigint`) which causes the query to be invalid. The 
casts all look redundant however working out when they are redundant is 
pretty complex, so I'm not sure if this approach has any legs.

> Regarding third party apps a solution could be to allow a per-app config 
option or encourage them to explicitly choose which primary key they use 
for their model but I'm not a big fan of baking logic in the migration 
framework to treat some fields in a special way.

Auto-generated M2M tables are an issue here, we would need to add some 
option to ManyToManyField() to allow passing in a PK field type. But I like 
the approach of just saying "if you ship migrations, be explicit about your 
primary keys" as it seems pretty simple.

Overall I think your two phase plan seems like a good approach 🙌 - lets do 
it?

On Thursday, 18 June 2020 at 15:18:35 UTC+1 charettes wrote:

> +1 from me as well but I don't think we should add any logic to the 
> migration framework to handle the transition.
>
> I think the release plan should be something along the following
>
> Phase 1:
> - New projects are generated with MODEL_DEFAULT_PK = 
> 'django.db.models.BigAutoField'
> - A system check or warning is emitted when the setting is not defined and 
> MODEL_DEFAULT_PK default to 'django.db.models.AutoField' during the 
> transition period. The warning should warn that the value will default to 
> 'django.db.models.AutoField' in a future version of Django.
> - An upgrade path is mentioned in the docs to mention that explicit ` id = 
> AutoField()` must be assigned to preserve previous behaviour and avoid 
> migrations. This one of the thing projects such as django-codemod could 
> really help with[0]
>
> Phase 2:
> - settings.MODEL_DEFAULT_PK now defaults to 'django.db.models.BigAutoField'
> - System check/warning about undefined MODEL_DEFAULT_PK setting is removed.
> - Let the migration framework generate migrations just like it normally 
> would for projects that didn't follow the documented migration path.
> - Optionally add a system check that warns about usage of `AutoField` 
> because of its possible overflow.
>
> > I think we should restrict the setting between normal and big auto 
> fields only. Allowing UUID's would be changing the type, with the potential 
> for havoc with code incompalities throughout django. It's also not possible 
> to migrate tables over to the new type.
>
> I'm not sure I agree here. For folks that are setting a up a new project 
> starting with UUIDField primary keys can be useful and if we're adding a 
> setting for this purpose I think we should it make it as flexible as 
> possible.
>
>
> [0] https://github.com/browniebroke/django-codemod
>
> Le jeudi 11 juin 2020 11:28:59 UTC-4, Tom Forbes a écrit :
>>
>> I’d like to re-propose switching Django to use BigAutoField’s rather than 
>> the current AutoField. This has been proposed[1] before (and a MR made[2]) 
>> but it was closed due to implementation issues and not much else has 
>> happened since then.
>>
>> As many of you are aware the max value a standard AutoField can hold is 
>> 2,147,483,647 (2.1 billion) which sounds like more than you can ever need. 
>> But it’s often not, and you only find out at the worst possible time - out 
>> of the blue at night and during a period of rapid growth. The process for 
>> fixing this issue also becomes a lot harder as your data grows - when 
>> you’ve hit the limit you’re looking at a multi-hour `ALTER TABLE` on 
>> Postgres during which writes and reads are blocked, or a risky operation to 
>> create a new table with the correct primary key and copy old data over in 
>> batches. Basically if you’ve experienced this before you wouldn’t wish it 
>> on your worst enemy.
>>
>> I’m proposing that we add a `MODELS_PRIMARY_KEY` (name improvements 
>> welcome!) setting that _defaults_ to `BigAutoField`, with prominent 
>> docs/release notes saying that to preserve the existing behaviour this 
>> should be set to `AutoField`. I think this the only realistic way we can 
>> implement this for new projects in a way that ensures it will be used 
>> meaningfully and not forgotten about until it’s too late.
>>
>> Rails managed to do this migration somewhat painlessly due the big 
>> differences between Rails and Django models. As Django migrations are 
>> derived from the current model state so there’s no

Re: Making startproject's settings more 12-factor-y

[Tom Forbes]

A small incremental change with some Django helpers and showing their use in 
the default settings.py is a great step forward, I would be +1 on that.

Tom

> On 25 Jun 2020, at 18:52, Kit La Touche  wrote:
> 
> 
> Wow, `distutils.util.strtobool` is great to know about!
> 
> So, can we refocus this conversation? This is starting to look like previous 
> conversations on this topic, which pull in a lot of possibilities but don't 
> lead to a change. How do we go about generating a DEP or other 
> consensus-building tool on what we want here?
> 
> It seems to me this conversation has historically gotten stuck by trying to 
> bite off a bigger bite. Therefore, I would recommend a minimal change that 
> gestures towards the direction we want to explore.
> 
> Personally, I think that at minimum providing Django-builtin "get from env"  
> helpers would be great; beyond that, I'd love to have them be included around 
> `DEBUG` and `SECRET_KEY` with the current values as defaults, so they're 
> optional. Once we see how this gets used, we can see about passing it a file 
> instead of `os.environ`, or borrowing other ideas from any of the various 
> supporting projects that have been suggested.
> 
> It's clear that different people have different use-cases and different 
> needs, but regardless, I think that it's clear also that including values 
> like DEBUG and SECRET_KEY as hard coded values in settings by default does 
> not point people towards good practices. What "good practices" are is likely 
> to differ in each person's case, but I think that suggesting one option 
> (again, my vote is "look in the environment") will at least help newer devs 
> understand that this is a topic they should learn more about.
> 
> Thanks,
> 
> --Kit
> 
>> On Thu, Jun 25, 2020 at 11:16 AM Javier Buzzi  wrote:
>> Hi Bobby, yes, thank you, this looks around the line of what i would like us 
>> to implement in Django. 
>> 
>> Side note: i saw this config('DEBUG', default=False, cast=bool) and thought 
>> "there is NO WAY that works", that led me to from distutils.util import 
>> strtobool, absolute mind blown! Thanks!
>> 
>> -Buzzi
>> 
>>> On Thursday, June 25, 2020 at 1:03:19 PM UTC-4, Bobby Mozumder wrote:
>>> There’s also python-decouple that I use that I haven’t seen mentioned in 
>>> this thread. It lets you set specific environment variables in a separate 
>>> .env file or INI file: https://github.com/henriquebastos/python-decouple
>>> 
>>> -bobby
>>> 
 On Jun 25, 2020, at 4:47 AM, Javier Buzzi  wrote:
 
 Hey Tom, cool project haven't heard of it, looks to me more inline with 
 validating and converting user input from an api/form. I could really see 
 myself using this in my personal projects, however this looks like we'd be 
 going back to the class based configuration that im trying to avoid. 
 Nonetheless thank you for the share!
 
 - Buzzi
 
> On Thursday, June 25, 2020 at 4:34:11 AM UTC-4, Tom Carrick wrote:
> Javier, I just wanted to point out another option for configuration: 
> pydantic - it offers a very slick and intuitive interface for settings 
> management across environments, seamless handing of environment variables 
> by using type hints, and so on. I wouldn't recommend it for anything 
> other than large sites with complex configurations, but it does work well 
> for those, once you grapple with how to integrate it with django's 
> settings so they're all exposed as `settings.FOO`, and so on.
> 
> I don't think I would want to integrate anything like this into Django 
> proper, but it might deserve a mention in the documentation.
> 
> Tom
> 
>> On Wed, 24 Jun 2020 at 23:52, Javier Buzzi  wrote:
>> This makes sense, I have a project that has a lot of settings files that 
>> get activated depending on the value of DJANGO_SETTINGS_MODULE. The 
>> solution i outlined above takes your reservations under consideration, 
>> if you want to use it, great, if not also great -- its a supplement not 
>> a requirement.
>> 
>> - Buzzi
>> 
>>> On Wednesday, June 24, 2020 at 5:24:35 PM UTC-4, Dan Davis wrote:
>>>  tMost of the world is not as seamless as heroku.  My DevOps won't give 
>>> me any more than a handful of environment variables.  I wanted 
>>> something like DATABASE_URL, but all I have is DJANGO_LOG_DIR and 
>>> DJANGO_SETTINGS_MODULE, and so I need many, many settings files. I 
>>> think that happens a lot, and maybe a common pattern.
>>> 
>>> From a 12factor perspective, I would like to get it down to local 
>>> settings (development) and production settings - yet for a lot of 
>>> users, DevOps is not really supporting a full PaaS-like experience any 
>>> way.
>>> 
>>> So - all of this has to be optional, which seems to rule out making it 
>>> part of the starting project template.  For sure, I've got my personal 
>>>

Re: The blacklist / master issue

[Tom Forbes]

As an international framework I think we should make our interface as language 
and culturally agnostic as possible. ‘Allow’ and ‘Deny’ are simply semantically 
clearer than ‘white’ and ‘black’. That alone is a convincing argument for me.

> On 19 Jun 2020, at 13:55, Alexander Lyabah  wrote:
> 
> 
> 
> Django in international framework, not US-framework. You should not change 
> variable names just because meaning of some words have been changed in US 
> recently. Those words have been used in source-code for years, and nobody put 
> racism in those word when this framework was founded and nobody puts any 
> racism in when one is using for creation something big and meaningful.
> 
> What I'm encourage you to do, is to thing farther than what is going on right 
> now.
> 
> If Django Foundation really want to help in this revolution - add a banner on 
> that landing page. Feel free to choose
> 
> https://eji.org/
> https://org2.salsalabs.com/o/6857/p/salsa/donation/common/public/
> 
> And this kind of contribution will work much better.
> 
> Thank you, for this opportunity to share my opinion.
> 
>> On Monday, June 15, 2020 at 7:28:23 PM UTC+3, Tom Carrick wrote:
>> This ticket was closed wontfix as requiring a discussion here.
>> 
>> David Smith mentioned this Tox issue stating it had been closed, but to me 
>> it seems like it hasn't been closed (maybe there's something I can't see) 
>> and apparently a PR would be accepted to add aliases at the least (this is 
>> more recent than the comment on the Django ticket).
>> 
>> My impetus to bring this up mostly comes from reading this ZDNet article - 
>> it seems like Google have already made moves in this direction and GitHub is 
>> also planning to. Usually Django is somewhere near the front for these types 
>> of changes.
>> 
>> I'm leaning towards renaming the master branch and wherever else we use that 
>> terminology, but I'm less sure about black/whitelist, though right now it 
>> seems more positive than negative. Most arguments against use some kind of 
>> etymological argument, but I don't think debates about historical terms are 
>> as interesting as how they affect people in the here and now.
>> 
>> I don't think there is an easy answer here, and I open this can of worms 
>> somewhat reluctantly. I do think Luke is correct that we should be concerned 
>> with our credibility if we wrongly change this, but I'm also worried about 
>> our credibility if we don't.
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/1c9178a3-cb80-428c-bacb-e8904695f6b6o%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/FB71B5CC-7224-4C17-8C9A-E4B263B4D0C8%40tomforb.es.

Re: Defaulting to use BigAutoField in models

[Tom Forbes]

I think we should re-open ticket 31007[1] as there was a consensus in 2017 
during the previous attempt at this[2] and things have not really changed - a 
32bit primary key is still not a sensible default given the calamity that can 
occur when you hit the end of it’s range. 

The sticky point is the implementation - how do we migrate new projects whilst 
minimising disruption to existing projects (who either won’t ever hit the 
limit, or have too much data to migrate wholesale). I think special casing the 
migrations framework is an avenue to explore, so I created this today and to my 
surprise it seems to work quite well: 
https://github.com/orf/django/commit/0a335f208cee1376c25eff55c6f866de122c7839 
<https://github.com/orf/django/commit/0a335f208cee1376c25eff55c6f866de122c7839>.

The obvious effect of this is that for old projects the Django model thinks its 
primary key is 64 bit while the underlying table is only 32 bit. I can’t think 
of much practical impact of this approach: 
- Python 3 doesn’t distinguish between 32/64 bit numbers so code will continue 
to work as-is. 
- Third party migrations that ship with 32 bit columns won’t be impacted, and 
it’s up to them to create migrations if they want to. 
- Adding `id = BigAutoField(primary_key=True)` to a model will create an 
explicit AlterField migration, when projects are able to migrate.

Am I missing something?

> I haven't thought this through, but maybe it's possible to restrict the 
> change to inside AutoField, rather than create a new field class. If its 
> db_parameters() method / db_type() could receive enough context from the 
> migration history, it could potentially determine its own database type.

I’m really not sure how we would do that: wouldn’t it require a pretty big 
refactor? It would definitely be cleaner.

1. https://code.djangoproject.com/ticket/31007 
<https://code.djangoproject.com/ticket/31007>
2. 
https://groups.google.com/forum/#!msg/django-developers/imBJwRrtJkk/P4g0Y87lAgAJ
 
<https://groups.google.com/forum/#!msg/django-developers/imBJwRrtJkk/P4g0Y87lAgAJ>


> On 12 Jun 2020, at 12:11, Adam Johnson  wrote:
> 
> I haven't thought this through, but maybe it's possible to restrict the 
> change to inside AutoField, rather than create a new field class. If its 
> db_parameters() method / db_type() could receive enough context from the 
> migration history, it could potentially determine its own database type. For 
> those who want to fixed field sizes, BigAutoField and SmallAutoField could be 
> left, and MediumAutoField added.
> 
> (Whilst poking around I think I spotted a small cleanup: 
> https://code.djangoproject.com/ticket/31698 
> <https://code.djangoproject.com/ticket/31698> )
> 
> On Fri, 12 Jun 2020 at 11:40, Tom Forbes  <mailto:t...@tomforb.es>> wrote:
>> I think we should restrict the setting between normal and big auto fields 
>> only. Allowing UUID's would be changing the type, with the potential for 
>> havoc with code incompalities throughout django. It's also not possible to 
>> migrate tables over to the new type.
> 
> That’s a really good point, I got a bit swept up with the idea to realise 
> that it would break a lot of things if applied generally!
> 
>> The autodetector knows if a model is new. It could be that during one 
>> version Django outputs BigAutoField for fields added in CreateModel only.
> 
> We could make every automatic PK become a BigAutoField but include some logic 
> in the migrations framework to ignore changes between an auto_created 
> AutoField -> BigAutoField? This would ignore all existing models in your 
> migration history, but all completely new models would receive BigAutoField 
> PKs from the start. Users who explicitly add a `BigAutoField` to a model with 
> a previously created `AutoField` would see a migration created as the 
> `auto_created` flag is not set. It would also play nicely with foreign keys 
> which also need to be 8 bytes.
> 
> This would take care of third party apps with migrations and not require a 
> setting but the downside is that model state will be slightly different from 
> the database. All models would have `BigAutoField`’s whereas the database 
> would only have an `AutoField`. This feels slightly iffy to me even though I 
> cannot think of a case where it would have any practical effect.
> 
>> On 11 Jun 2020, at 19:22, Adam Johnson > <mailto:m...@adamj.eu>> wrote:
>> 
>> Big +1 on solving this from me.
>> 
>> - The setting would take any dotted path to a class, or a single class name 
>> for a build in field. This would potentially solve [3], and could be useful 
>> to people who want to default to other fields like UUIDs (or a custom 
>> BigAutoField) for whatever reason

Re: Proposal - Warn user when creating or applying a delete migration?

[Tom Forbes]

I think opening a migration file in the users $EDITOR could be a good flag to 
add, but I would be wary of making it a default behaviour. I can see it being 
useful in some situations but annoying in others.

I made a ticket for changing the output: 
https://code.djangoproject.com/ticket/31700

> On 12 Jun 2020, at 13:01, Adam Johnson  wrote:
> 
> That output change would be useful, but not accessible. Perhaps we should 
> also use "+" for creative and "-" for destructive operations?
> 
> We could also open the generated migration file in the user's $EDITOR, if the 
> terminal is interactive. This would be a nice usability improvement for use 
> of --empty or other situations where the user knows they need to edit the 
> migration too.
> 
> On Fri, 12 Jun 2020 at 12:56, Tom Forbes  <mailto:t...@tomforb.es>> wrote:
> Fully agree, I would be against adding a y/n prompt. 
> 
> However we could make the `makemigrations` output more noticeable if it 
> contains destructive operations? We could make the this line[1] output red 
> text on destructive operations, yellow on maybe destructive (like altering a 
> type), and green on safe operations?
> 
> 1. 
> https://github.com/django/django/blob/master/django/core/management/commands/makemigrations.py#L216
>  
> <https://github.com/django/django/blob/master/django/core/management/commands/makemigrations.py#L216>
> 
>> On 12 Jun 2020, at 12:52, Adam Johnson > <mailto:m...@adamj.eu>> wrote:
>> 
>> 'makemigrations' is a code generator, not a "do it right all the time" 
>> wizard. There are many situations where it doesn't do what the user 
>> intended. Users are ultimately responsible for the contents of migration 
>> files and should read them before applying them, and also use sqlmigrate to 
>> see what the underlying SQL is.
>> 
>> I think our "workflow" section could make this clearer: 
>> https://docs.djangoproject.com/en/3.0/topics/migrations/#workflow 
>> <https://docs.djangoproject.com/en/3.0/topics/migrations/#workflow>
>> 
>> I am unsure that adding warnings or errors will force users to pay more 
>> attention. Often users run without warnings displayed.
>> 
>> On Fri, 12 Jun 2020 at 12:41, Jason Johns > <mailto:jjohns98...@gmail.com>> wrote:
>> Should Django output a warning and/or require a prompt when a DeleteModel or 
>> RemoveField are to be executed when applying migrations?
>> 
>> Over at the pythondev slack group, a user wanted to rename a model to 
>> another name, and wasn't aware of  the `db_table` attribute in Model Meta.  
>> So a new model was created, the old one removed.  Then a migration was made 
>> and applied... only to delete the data on the staging environment.  This 
>> user was also unaware of the typical three step migration process 
>> recommended for removing a model or field, and made the statement that there 
>> should have been a prompt during migrate that data loss was a possibility.
>> 
>> This got me thinking.  Googling around seems to suggest this experience of 
>> running a delete migration is a somewhat common footgun because the 
>> implications are not as explicit as Python warnings and exceptions.  As a 
>> result, it might be beneficial for migrate to have some tweaks to attempt to 
>> reduce surprise.
>> 
>> How about this:
>> Migrate checks if any executions of migrations.DeleteModel or 
>> migrations.RemoveField are included in the migrations being applied
>> If so, output a warning message about a table being dropped or column being 
>> deleted and that data loss will occur
>> Output a yes/no input prompt asking if the user has made a data migration or 
>> is aware and acknowledges the risk
>> 
>> Is this something that would fit in with the overall philosophy of Django?  
>> To be honest, I'm undecided about this, but this may be my own biased 
>> feelings that this should be implicit in the developer's mind when 
>> considering removing columns and tables.  But then again, I have been burned 
>> by this in a toy project.
>> 
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-developers+unsubscr...@googlegroups.com 
>> <mailto:django-developers+unsubscr...@googlegroups.com>.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-developers/e4af0410-7eab-4431

Re: Proposal - Warn user when creating or applying a delete migration?

[Tom Forbes]

Fully agree, I would be against adding a y/n prompt. 

However we could make the `makemigrations` output more noticeable if it 
contains destructive operations? We could make the this line[1] output red text 
on destructive operations, yellow on maybe destructive (like altering a type), 
and green on safe operations?

1. 
https://github.com/django/django/blob/master/django/core/management/commands/makemigrations.py#L216
 


> On 12 Jun 2020, at 12:52, Adam Johnson  wrote:
> 
> 'makemigrations' is a code generator, not a "do it right all the time" 
> wizard. There are many situations where it doesn't do what the user intended. 
> Users are ultimately responsible for the contents of migration files and 
> should read them before applying them, and also use sqlmigrate to see what 
> the underlying SQL is.
> 
> I think our "workflow" section could make this clearer: 
> https://docs.djangoproject.com/en/3.0/topics/migrations/#workflow 
> 
> 
> I am unsure that adding warnings or errors will force users to pay more 
> attention. Often users run without warnings displayed.
> 
> On Fri, 12 Jun 2020 at 12:41, Jason Johns  > wrote:
> Should Django output a warning and/or require a prompt when a DeleteModel or 
> RemoveField are to be executed when applying migrations?
> 
> Over at the pythondev slack group, a user wanted to rename a model to another 
> name, and wasn't aware of  the `db_table` attribute in Model Meta.  So a new 
> model was created, the old one removed.  Then a migration was made and 
> applied... only to delete the data on the staging environment.  This user was 
> also unaware of the typical three step migration process recommended for 
> removing a model or field, and made the statement that there should have been 
> a prompt during migrate that data loss was a possibility.
> 
> This got me thinking.  Googling around seems to suggest this experience of 
> running a delete migration is a somewhat common footgun because the 
> implications are not as explicit as Python warnings and exceptions.  As a 
> result, it might be beneficial for migrate to have some tweaks to attempt to 
> reduce surprise.
> 
> How about this:
> Migrate checks if any executions of migrations.DeleteModel or 
> migrations.RemoveField are included in the migrations being applied
> If so, output a warning message about a table being dropped or column being 
> deleted and that data loss will occur
> Output a yes/no input prompt asking if the user has made a data migration or 
> is aware and acknowledges the risk
> 
> Is this something that would fit in with the overall philosophy of Django?  
> To be honest, I'm undecided about this, but this may be my own biased 
> feelings that this should be implicit in the developer's mind when 
> considering removing columns and tables.  But then again, I have been burned 
> by this in a toy project.
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/e4af0410-7eab-4431-8c56-5094d9498b92o%40googlegroups.com
>  
> .
> 
> 
> -- 
> Adam
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/CAMyDDM0ROmXKTaZ7fuw7cvXLLRk8AbrWw1EhcQMssOofpYGKGA%40mail.gmail.com
>  
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/69C34E37-ADE5-4B56-8439-5F8CC962017C%40tomforb.es.

Re: Defaulting to use BigAutoField in models

[Tom Forbes]

> I think we should restrict the setting between normal and big auto fields 
> only. Allowing UUID's would be changing the type, with the potential for 
> havoc with code incompalities throughout django. It's also not possible to 
> migrate tables over to the new type.

That’s a really good point, I got a bit swept up with the idea to realise that 
it would break a lot of things if applied generally!

> The autodetector knows if a model is new. It could be that during one version 
> Django outputs BigAutoField for fields added in CreateModel only.

We could make every automatic PK become a BigAutoField but include some logic 
in the migrations framework to ignore changes between an auto_created AutoField 
-> BigAutoField? This would ignore all existing models in your migration 
history, but all completely new models would receive BigAutoField PKs from the 
start. Users who explicitly add a `BigAutoField` to a model with a previously 
created `AutoField` would see a migration created as the `auto_created` flag is 
not set. It would also play nicely with foreign keys which also need to be 8 
bytes.

This would take care of third party apps with migrations and not require a 
setting but the downside is that model state will be slightly different from 
the database. All models would have `BigAutoField`’s whereas the database would 
only have an `AutoField`. This feels slightly iffy to me even though I cannot 
think of a case where it would have any practical effect.

> On 11 Jun 2020, at 19:22, Adam Johnson  wrote:
> 
> Big +1 on solving this from me.
> 
> - The setting would take any dotted path to a class, or a single class name 
> for a build in field. This would potentially solve [3], and could be useful 
> to people who want to default to other fields like UUIDs (or a custom 
> BigAutoField) for whatever reason
> 
> I think we should restrict the setting between normal and big auto fields 
> only. Allowing UUID's would be changing the type, with the potential for 
> havoc with code incompalities throughout django. It's also not possible to 
> migrate tables over to the new type.
> 
> What do you think the solution is for third-party apps? They ship their own 
> migrations and can't really be tied to project state.
> 
> As Django migrations are derived from the current model state so there’s no 
> way I can think of to express “make this auto-generated field a BigAutoField 
> only if this model is new”.
> 
> The autodetector knows if a model is new. It could be that during one version 
> Django outputs BigAutoField for fields added in CreateModel only.
> 
> On Thu, 11 Jun 2020 at 16:28, Tom Forbes  <mailto:t...@tomforb.es>> wrote:
> I’d like to re-propose switching Django to use BigAutoField’s rather than the 
> current AutoField. This has been proposed[1] before (and a MR made[2]) but it 
> was closed due to implementation issues and not much else has happened since 
> then.
> 
> As many of you are aware the max value a standard AutoField can hold is 
> 2,147,483,647 (2.1 billion) which sounds like more than you can ever need. 
> But it’s often not, and you only find out at the worst possible time - out of 
> the blue at night and during a period of rapid growth. The process for fixing 
> this issue also becomes a lot harder as your data grows - when you’ve hit the 
> limit you’re looking at a multi-hour `ALTER TABLE` on Postgres during which 
> writes and reads are blocked, or a risky operation to create a new table with 
> the correct primary key and copy old data over in batches. Basically if 
> you’ve experienced this before you wouldn’t wish it on your worst enemy.
> 
> I’m proposing that we add a `MODELS_PRIMARY_KEY` (name improvements welcome!) 
> setting that _defaults_ to `BigAutoField`, with prominent docs/release notes 
> saying that to preserve the existing behaviour this should be set to 
> `AutoField`. I think this the only realistic way we can implement this for 
> new projects in a way that ensures it will be used meaningfully and not 
> forgotten about until it’s too late.
> 
> Rails managed to do this migration somewhat painlessly due the big 
> differences between Rails and Django models. As Django migrations are derived 
> from the current model state so there’s no way I can think of to express 
> “make this auto-generated field a BigAutoField only if this model is new”. 
> The way I see it is that a global setting is very easy to toggle and there is 
> little chance of missing the large numbers of migrations that would be 
> generated during the Django update. Smaller applications could apply the 
> migrations with little issue and larger applications would be able to opt-out 
> (as well as be reminded that this is a problem they could face!).
> 
> Some spec

Defaulting to use BigAutoField in models

[Tom Forbes]

I’d like to re-propose switching Django to use BigAutoField’s rather than the 
current AutoField. This has been proposed[1] before (and a MR made[2]) but it 
was closed due to implementation issues and not much else has happened since 
then.

As many of you are aware the max value a standard AutoField can hold is 
2,147,483,647 (2.1 billion) which sounds like more than you can ever need. But 
it’s often not, and you only find out at the worst possible time - out of the 
blue at night and during a period of rapid growth. The process for fixing this 
issue also becomes a lot harder as your data grows - when you’ve hit the limit 
you’re looking at a multi-hour `ALTER TABLE` on Postgres during which writes 
and reads are blocked, or a risky operation to create a new table with the 
correct primary key and copy old data over in batches. Basically if you’ve 
experienced this before you wouldn’t wish it on your worst enemy.

I’m proposing that we add a `MODELS_PRIMARY_KEY` (name improvements welcome!) 
setting that _defaults_ to `BigAutoField`, with prominent docs/release notes 
saying that to preserve the existing behaviour this should be set to 
`AutoField`. I think this the only realistic way we can implement this for new 
projects in a way that ensures it will be used meaningfully and not forgotten 
about until it’s too late.

Rails managed to do this migration somewhat painlessly due the big differences 
between Rails and Django models. As Django migrations are derived from the 
current model state so there’s no way I can think of to express “make this 
auto-generated field a BigAutoField only if this model is new”. The way I see 
it is that a global setting is very easy to toggle and there is little chance 
of missing the large numbers of migrations that would be generated during the 
Django update. Smaller applications could apply the migrations with little 
issue and larger applications would be able to opt-out (as well as be reminded 
that this is a problem they could face!).

Some specifics:
- The setting would take any dotted path to a class, or a single class name for 
a build in field. This would potentially solve [3], and could be useful to 
people who want to default to other fields like UUIDs (or a custom 
BigAutoField) for whatever reason
- The setting would also be used for GenericForeignKeys, which right now are 
backed by a PositiveIntegerField and so suffer from the same AutoField 
limitations

Any thoughts on this?

Tom

1. https://groups.google.com/d/msg/django-developers/imBJwRrtJkk/P4g0Y87lAgAJ 


2. https://github.com/django/django/pull/8924/ 


3. https://code.djangoproject.com/ticket/56 




-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/435EC704-3EF6-4EF4-BF85-175AE29C01F5%40tomforb.es.

Re: timesince 'depth' parameter

[Tom Forbes]

I like this idea. However while `depth=1` makes sense for shorter time periods 
where the difference between `4 hours` and `4 hours and 3 minutes ago` is not 
required to be displayed, when you get a value that is over a year ago it loses 
a lot of granularity. The difference between `1 year` and `1 year and 11 
months` is much more important.

In this case you probably want `depth=2` (“1 year, 6 months ago”), but only for 
values over a year ago. Given that, maybe a simple depth flag isn’t enough? I 
think what we really want here is a `compact=True` flag.

> On 22 May 2020, at 16:59, Toby Such  wrote:
> 
> Hi all!
> 
> I've been a little frustrated with the timesince filter that comes with 
> default Django. By default it always comes with 2 parts e.g. 4 hours and 3 
> minutes ago but I would much rather have just the 4 hours. In my own projects 
> I've simply created my own version of the timesince filter and accompanying 
> utils function that powers it that just has the code which adds the second 
> part removed. However I wonder why this isn't standard as I can't think of a 
> single website that displays time since as multiple parts. 
> 
> I'm proposing adding a 'depth' parameter to the timesince function which 
> would allow the developer to set how many parts to render. At depth one it 
> would be "3 weeks". 2 depth: "3 weeks 2 days" (default behaviour), 3 depth: 
> "3 weeks, 2 days, 4 hours ago" etc.
> 
> Does anyone have any thoughts related to this?
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/97fbcab8-3f0f-4277-9296-fdcff7fcdab5%40googlegroups.com
>  
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/DCC0D41F-A34F-4FFD-B8CC-39988EBE357E%40tomforb.es.

Re: Disabling .delete() in querysets in Django

[Tom Forbes]

I don’t think a Django-wide setting to disable Queryset.delete() is 
appropriate. As you said, it’s easy enough to configure for the specific models 
that may (or may not!) benefit from this. 

A Django-wide setting like this would also break just about everything that 
calls “delete()” on a queryset.

Tom

> On 20 May 2020, at 13:57, אורי  wrote:
> 
> 
> Django developers,
> 
> I recently disabled .delete() in querysets in all managers in my project, 
> after encountering a bug in a bulk delete queryset in my tests 
> [https://code.djangoproject.com/ticket/31600]. Actually it was quite simple 
> to disable .delete() in querysets, but I had to define my own QuerySet class:
> 
> class QuerySet(models.query.QuerySet):
> def delete(self):
> raise NotImplementedError("delete is not implemented.")
> 
> I also disabled bulk_create() in managers for similar reasons. And I thought, 
> maybe you can introduce optional settings in which if it's set, will disable 
> delete() and bulk_create() in managers and querysets? I think this may be 
> useful, since the delete() and bulk_create() methods have problems - they 
> don't call the model's methods and in some cases (such as in my case) related 
> objects from other models don't get deleted (with delete()) and models don't 
> get validated (with bulk_create()).
> 
> I also call each model's self.full_clean() method before saving, which may 
> also be a setting in Django. I don't want invalid objects to be saved to the 
> database.
> 
> If you think this is a good idea and want me to implement it, I can try to 
> submit a PR.
> 
> By the way, I checked and I found out that I can still delete objects from 
> our admin interface, so it probably doesn't use the delete() queryset.
> 
> Thanks,
> Uri.
> 
> אורי
> u...@speedy.net
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/CABD5YeGBinT0uXyCdmw6%3DARAN5URFWk-Sh%3D1wvb8Fbfuj8c_pQ%40mail.gmail.com.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/2553E532-E331-41E8-A700-7D5AE5BB572B%40tomforb.es.

Re: Removing url() ?

[Tom Forbes]

An idea for a Django equivalent to “rails generate” has been floating around my 
mind for quite a while. Automated codemods are not only useful for fixing 
deprecations but also many common, repetitive tasks that might involve 
modifying multiple files or boilerplate (creating a simple model, adding a 
route, etc).

libCST looks perfect for this, thanks for sharing.

Tom

> On 10 May 2020, at 14:12, Bruno Alla  wrote:
> 
> 
> I've started working on a tool provides a set of codemodders for Django:
> 
> https://github.com/browniebroke/django-codemod
> 
> It's based on Instagram's libCST (work with Concrete Syntax Tree). This is 
> very much a work in progress, but I think it could help with these kind of 
> deprecations that require changing a lot of files. 
> 
> I'm not really satisfied by the API of the CLI, I would prefer to get 
> something like pyupgrade, where you specify your min & max Django version, 
> but I guess it's better than nothing.
> 
> It's very much a work in progress and feedback is welcome!
> 
>> On Saturday, 9 May 2020 17:13:51 UTC+1, Adam Johnson wrote:
>> Automated upgrade tooling would be great. Developers end up building their 
>> own anyway.
>> 
>> One more tool we could copy is pyupgrade ( 
>> https://pypi.org/project/pyupgrade/ ). It's based upon an extension to 
>> 'tokenize' by the author (Anthony Sottile) that allows roundtripping (which 
>> I guess would be useful being merged to the stdlib). But it also looks 
>> possibly painful to work with.
>> 
>> I've tried Bowler and it has a nice interface, allowing git-add-patch 
>> presentation through its idiff() command. But it doesn't have many examples 
>> in its documentation, which made me give up. It's also based on lib2to3, 
>> through a separated copy called fissix.
>> 
>> codemod's README lists Python 2 support only :(
>> 
>> regexes for find/replace, maybe with a sed command, would be the lowest 
>> common denominator. Combined with "git add --patch", then "git checkout ." 
>> that's basically what codemod does (afaiu).
>> 
>>> On Sat, 9 May 2020 at 13:32, Aymeric Augustin 
>>>  wrote:
>>> Hello,
>>> 
>>> When proposing deprecations that require numerous but simple code changes, 
>>> providing an automated upgrade path could reduce frustrations about 
>>> "useless churn".
>>> 
>>> Unfortunately, I don't know a great way to do that on Python code. Here are 
>>> the options I'm aware of:
>>> 
>>> - Writing 2to3 fixers is painful, in my experience. Also, I don't think 
>>> that 2to3 will survive the deprecation of lib2to3; removal is scheduled for 
>>> Python 3.12.
>>> - Bowler seems designed to make it less painful, but is also written on top 
>>> of lib2to3, so not a long term solution.
>>> - codemod gives up fully automated refactoring (which is at odds with 
>>> Python's dynamic nature) and attempts "computer-aided refactoring".
>>> - Most likely IDEs like PyCharm can rename an import throughout a project.
>>> 
>>> If I had to do something in this area, I'd try codemod, probably.
>>> 
>>> Of course, it will still be boring to perform the same change across N 
>>> projects for no direct project-level upside, but at least it may require 
>>> less time and less focus.
>>> 
>>> This is relevant for the thread about renaming request.GET/POST/etc. If we 
>>> proceed with that change, we're bound to have the exact same discussion 
>>> about removing the original names.
>>> 
>>> Best regards,
>>> 
>>> -- 
>>> Aymeric.
>>> 
>>> 
>>> 
 On 5 May 2020, at 17:41, Collin Anderson  wrote:
 
 Hi All,
 
 Are we _sure_ we want to completely get rid of url()?
 
 Yesterday, url() was given a RemovedInDjango40Warning [PR]. The removal 
 was approved as part of the new path() syntax back in 2017 [DEP 0201].
 
 Is this really worth it? It's only a few lines of code to keep backward 
 compatibility, and it seems to me it would take almost no work to maintain 
 that compatibility shim compared to the countless programmer hours needed 
 to upgrade their code, including many unpaid programmers working on open 
 source projects. I'll personally need to do a find/replace for url() to 
 re_path() across my ~20 projects, and update all of the imports. Isn't 
 this useless code churn?
 
 Yes, there's an advantage to having one and only one way to do it, so I 
 agree we should encourage people to use re_path() instead of url(), but 
 shouldn't we also make it as easy as possible to upgrade django? Is 
 getting rid of url() really worth the cost?
 
 Yes, the removal is still ~3+ years away, but that's a question of _when_ 
 not _if_.
 
 If we want, we _could_ deprecate url() without giving it an actual removal 
 date [Compatibility Discussion] and leave the compatibility shim around 
 longer, if not indefinitely.
 
 Thanks,
 Collin
 
 
 [PR] https://github.com/django/django/pull/12855
 
 [DEP 0201] 
 https:/

Re: Generate JWTs with Django

[Tom Forbes]

Thank you for the fantastic summary James. You’re spot on about the various 
implementation issues that plague JWT libraries.

While I think it’s an OK technology if you use a very explicit subset and you 
know what you’re doing (see https://github.com/google/jws) it is indeed a bit 
of a minefield and I’m not sure we should be promoting it.

That being said, are signed cookies really a suitable replacement for JWTs? 
Right off the bat there is a size limit that’s shared with other cookies on 
your domain, and you might not want to (or be able to) encode the data in a 
cookie to begin with.

Maybe we can update the docs to show how you you would might use some of the 
signing primitives instead of JWTs, but this also sounds a bit dangerous 🤷‍♂️ 

>> On 27 Apr 2020, at 03:53, James Bennett  wrote:
> On Sun, Apr 26, 2020 at 8:46 AM Adam Johnson  wrote:
>> 
>> James, I too would like to know your criticisms! I've always understood that 
>> they aren't much different to signed cookies, but I haven't looked too 
>> deeply at them.
> 
> Well, people asked. So.
> 
> The short summary is: JWT is over-complex, puts too much power in the
> attacker's hands, has too many configuration knobs, and makes poor
> cryptographic choices. This is why we see vulnerabilities in JWT
> libraries and JWT-using systems again and again and again.
> 
> And even if you get every single bit of it right, in the ideal perfect
> world with the ideal perfect implementation, the payoff is you've put
> in a ton of work to get something that already existed: signed
> cookies, for the use case of session identification, or any of several
> better token or token-like systems -- everything from PASETO to just
> timestamped HMAC'd values -- for the use case of inter-service
> communication.
> 
> The longer version goes more like this...
> 
> JWT is a more complex thing than many people appreciate. In fact, it's
> at least *five* things, each specified in its own RFC, plus some more
> you have to know about and implement if you want any hope of getting
> the actually-sorta-secure version. And right off the bat, that's
> worrying: the more different things you have to implement, and the
> more places you have to look to find out what and how to implement,
> the more opportunities there are to make mistakes.
> 
> This complexity comes from the sheer number of different options JWT
> tries to support, which in turn is an anti-pattern. JWTs may be
> signed, or they may not. They may be encrypted, or they may not. There
> are multiple different options for how to sign, how to encrypt, how to
> manage and specify keys... in a well-designed system there would be
> far fewer options. Ideally, there'd be only one option, and the
> solution for a vulnerability being found in it would be to increment
> the version of the underlying spec, and change to something
> better.
> 
> Anyway. In JWT, signature and encryption schemes are effectively
> negotiable, which is yet another anti-pattern: you are putting
> enormous power in the hands of attackers to negotiate you down to a
> bad, or nonexistent, cipher/algorithm for encryption or signing. TLS/SSL
> learned this lesson the hard way; JWT has chosen not to learn it at
> all. Worse, JWT embeds the negotiation about how to handle the token
> into the token itself. This is just asking for trouble, and in fact
> trouble has routinely occurred. As a somewhat sarcastic author of my
> acquaintance put it:
> 
>> It is extraordinarily easy to screw up JWT. JWT is a JSON format
>> where you have to parse and interpret a JSON document to figure out
>> how to decrypt and authenticate a JSON document. It has revived bugs
>> we thought long dead, like “repurposing asymmetric public keys as
>> symmetric private keys”.
> 
> (from: https://latacora.micro.blog/a-childs-garden/)
> 
> More succinctly: JWTs inherently and unavoidably violate the
> Cryptographic Doom Principle
> (https://moxie.org/blog/the-cryptographic-doom-principle/). Worse,
> JWTs put full control of the violation in the hands of the attacker,
> who -- thanks to the high level of configurability JWT offers -- is
> free to find the set of options most likely to compromise you, and
> foist them on you.
> 
> That quoted line above about revived long-dead bugs is no joke,
> incidentally. Here's one from a few years back:
> 
> https://auth0.com/blog/critical-vulnerabilities-in-json-web-token-libraries/
> 
> Five different libraries, written in three different languages, all
> had the same bug. I once had the misfortune of arguing with someone
> who attributed this to the libraries being built by bad programmers,
> or in bad languages. But I personally tend to agree with a line from
> this talk:
> 
> https://www.okta.com/video/oktane19-cryptographic-wrong-answers/
> 
>> Now you can say look those are all implementation vulnerabilities,
>> it's not actually JWTs fault, I disagree. If I can find the same
>> vulnerability in five popular libraries, then maybe the spec d

Re: Google Groups contingency plan

[Tom Forbes]

Actually, after taking a look I found that there is actually a new version of 
Groups that is in beta 
(https://support.google.com/a/answer/9687393?hl=en&ref_topic=14869 
<https://support.google.com/a/answer/9687393?hl=en&ref_topic=14869>). It seems 
they are more tightly integrating it with G-Suite, and targeting it towards 
schools and businesses rather than public mailing lists/usenet archives.

Some features are potentially being deprecated 
(https://support.google.com/a/answer/9687393?hl=en&ref_topic=14869#deprecated 
<https://support.google.com/a/answer/9687393?hl=en&ref_topic=14869#deprecated>),
 but I don’t think we use any of those.

I’m glad that Groups isn’t going to die, and I guess perhaps it makes this 
thread redundant, but the lack of an easy export is troubling and unexpected. 
Perhaps with the new release it might be possible without resorting to 
screen-scraping the public index pages?

Tom

> On 26 Apr 2020, at 21:14, Adam Johnson  wrote:
> 
>  I think python.org <http://python.org/> and Django are managed by different 
> (legal) entities so it doesn't make sense for Django mailing lists to use the 
> domain name python.org <http://python.org/>
> 
> I think it's fine to share with them, if possible.  As Tom, says other 
> projects like scikit do.
> 
> 
> Tom - you didn't mention backing up the archives. I had a search and it seems 
> there's no official tool, but there is a script here: 
> https://geek.co.il/2018/05/16/migrating-google-groups-archives-between-accounts
>  
> <https://geek.co.il/2018/05/16/migrating-google-groups-archives-between-accounts>
>  . I think having a trial run ofthis would be a good first step to knowing 
> where we're at with preserving the data.
> 
> 
> On Sun, 26 Apr 2020 at 17:45, Tom Forbes  <mailto:t...@tomforb.es>> wrote:
>> I think python.org <http://python.org/> and Django are managed by different 
>> (legal) entities so it doesn't make sense for Django mailing lists to use 
>> the domain name python.org <http://python.org/>
> That’s a good point, it was just an idea. However running mailserver can be 
> annoying and there are economies of scale when it comes to security, spam 
> filtering and ease of administration. We could potentially have a django.com 
> <http://django.com/> mailing list address that points at a mailsever managed 
> by python.org <http://python.org/>, and other projects like scikit have their 
> mailing lists hosted by them.
> 
>> Actually as far as I see it, we only need mailing lists and don't need any 
>> web interface features we get from Google Groups
> 
> We certainly do need it, if only for historic searching. User management and 
> other niceties are great to have as well.
> 
>> why do we have to worry about it now
> 
> I mean, it’s a contingency plan. More of a contingency discussion actually. 
> The point is to discuss future events before they happen. If the result is 
> “it’s not worth thinking about now, let's decide when/if we get there” then 
> that’s valid outcome.
> 
>> On 26 Apr 2020, at 17:34, ⁨אורי⁩ <⁨u...@speedy.net 
>> <mailto:u...@speedy.net>⁩> wrote:
>> 
>> 
>> On Sun, Apr 26, 2020 at 7:14 PM Tom Forbes > <mailto:t...@tomforb.es>> wrote:
>> Hello,
>> Given the recent worldwide situation I’ve found myself thinking a lot about 
>> contingency plans recently. I wanted to raise a question about our reliance 
>> on Google Groups and if we had any contingency plans if this service was 
>> shut down?
>> 
>> To put it politely, it's pretty obvious that Google Groups is in 
>> “maintenance mode” and there is little, if any, active development. 
>> Alongside this:
>> 1. Google has apparently broken core functionality like advanced searching, 
>> and left it broken for years at a time 
>> (https://en.wikipedia.org/wiki/Google_Groups#Google_Groups 
>> <https://en.wikipedia.org/wiki/Google_Groups#Google_Groups>)
>> 2. Google itself has shut down down maintenance-mode services before even if 
>> they have an active user base (see Google Reader)
>> 3. Google Groups clearly doesn’t bring any revenue to Google at all, and is 
>> probably a legal liability.
>> 
>> Given this, if we have not already we should maybe consider what we would do 
>> if it was announced tomorrow that Google Groups was shutting down in 3 to 6 
>> months?
>> 
>> Perhaps we’ve already considered this but I couldn’t find any discussions 
>> about this (and it’s hard to search for threads that reference “Google 
>> Groups” as every thread contains that string!). Some food f

Re: Google Groups contingency plan

[Tom Forbes]

> I think python.org <http://python.org/> and Django are managed by different 
> (legal) entities so it doesn't make sense for Django mailing lists to use the 
> domain name python.org <http://python.org/>
That’s a good point, it was just an idea. However running mailserver can be 
annoying and there are economies of scale when it comes to security, spam 
filtering and ease of administration. We could potentially have a django.com 
mailing list address that points at a mailsever managed by python.org 
<http://python.org/>, and other projects like scikit have their mailing lists 
hosted by them.

> Actually as far as I see it, we only need mailing lists and don't need any 
> web interface features we get from Google Groups

We certainly do need it, if only for historic searching. User management and 
other niceties are great to have as well.

> why do we have to worry about it now

I mean, it’s a contingency plan. More of a contingency discussion actually. The 
point is to discuss future events before they happen. If the result is “it’s 
not worth thinking about now, let's decide when/if we get there” then that’s 
valid outcome.

> On 26 Apr 2020, at 17:34, ⁨אורי⁩ <⁨u...@speedy.net⁩> wrote:
> 
> 
> On Sun, Apr 26, 2020 at 7:14 PM Tom Forbes  <mailto:t...@tomforb.es>> wrote:
> Hello,
> Given the recent worldwide situation I’ve found myself thinking a lot about 
> contingency plans recently. I wanted to raise a question about our reliance 
> on Google Groups and if we had any contingency plans if this service was shut 
> down?
> 
> To put it politely, it's pretty obvious that Google Groups is in “maintenance 
> mode” and there is little, if any, active development. Alongside this:
> 1. Google has apparently broken core functionality like advanced searching, 
> and left it broken for years at a time 
> (https://en.wikipedia.org/wiki/Google_Groups#Google_Groups 
> <https://en.wikipedia.org/wiki/Google_Groups#Google_Groups>)
> 2. Google itself has shut down down maintenance-mode services before even if 
> they have an active user base (see Google Reader)
> 3. Google Groups clearly doesn’t bring any revenue to Google at all, and is 
> probably a legal liability.
> 
> Given this, if we have not already we should maybe consider what we would do 
> if it was announced tomorrow that Google Groups was shutting down in 3 to 6 
> months?
> 
> Perhaps we’ve already considered this but I couldn’t find any discussions 
> about this (and it’s hard to search for threads that reference “Google 
> Groups” as every thread contains that string!). Some food for thought: the 
> Python development mailing lists (i.e 
> https://mail.python.org/archives/list/python-...@python.org/ 
> <https://mail.python.org/archives/list/python-...@python.org/>) are browsable 
> through a service called HyperKitty (https://gitlab.com/mailman/hyperkitty 
> <https://gitlab.com/mailman/hyperkitty>), which is built with Django. We 
> could potentially ask for a mailing list on the python.org 
> <http://python.org/> mail server or we could host HyperKitty and MailMan 
> ourselves?
> 
> 
> 
> I think python.org <http://python.org/> and Django are managed by different 
> (legal) entities so it doesn't make sense for Django mailing lists to use the 
> domain name python.org <http://python.org/>. Actually as far as I see it, we 
> only need mailing lists and don't need any web interface features we get from 
> Google Groups. Personally my email is powered by Google which is like Gmail 
> and I assign labels per mailing list, so if I want to search I can just type 
> text and the label name and I can see any messages since I joined the mailing 
> lists (I never delete messages). I think if, if Google will close Google 
> Groups they will give us at least a few months notice and then we can discuss 
> it, why do we have to worry about it now? But just in case, I think running a 
> (free software & open source) mailing list software on a specific server will 
> do, something similar to what python.org <http://python.org/> is doing but 
> not using their domain name (unless Django becomes parts of the python.org 
> <http://python.org/> legal entity).
> 
> By the way, is Django a commercial entity? Why does it use the .com domain 
> suffix? Python uses .org, but doesn't Django use .org too? From reading the 
> website I see that the Django Software Foundation is non-profit.
> 
> --
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+

Google Groups contingency plan

[Tom Forbes]

Hello,
Given the recent worldwide situation I’ve found myself thinking a lot about 
contingency plans recently. I wanted to raise a question about our reliance on 
Google Groups and if we had any contingency plans if this service was shut down?

To put it politely, it's pretty obvious that Google Groups is in “maintenance 
mode” and there is little, if any, active development. Alongside this:
1. Google has apparently broken core functionality like advanced searching, and 
left it broken for years at a time 
(https://en.wikipedia.org/wiki/Google_Groups#Google_Groups 
)
2. Google itself has shut down down maintenance-mode services before even if 
they have an active user base (see Google Reader)
3. Google Groups clearly doesn’t bring any revenue to Google at all, and is 
probably a legal liability.

Given this, if we have not already we should maybe consider what we would do if 
it was announced tomorrow that Google Groups was shutting down in 3 to 6 months?

Perhaps we’ve already considered this but I couldn’t find any discussions about 
this (and it’s hard to search for threads that reference “Google Groups” as 
every thread contains that string!). Some food for thought: the Python 
development mailing lists (i.e 
https://mail.python.org/archives/list/python-...@python.org/ 
) are browsable 
through a service called HyperKitty (https://gitlab.com/mailman/hyperkitty 
), which is built with Django. We could 
potentially ask for a mailing list on the python.org  mail 
server or we could host HyperKitty and MailMan ourselves?

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/2715F232-322F-4C3D-BEE7-9C63C2B0ECFB%40tomforb.es.


signature.asc
Description: Message signed with OpenPGP

Re: Remove automatic date-naming of migrations (00XX_auto_YYYMMDD)

[Tom Forbes]

I would be in favour of Adam’s proposed changes. Adding more context to them 
and stripping the appended timestamp would make them more user friendly.

> As a developer I would like to know who generated the code. If a migration is 
> auto-generated, I would like to know that. I checked and auto-generated 
> migrations in my project have a comment such as "# Generated by Django 2.1.15 
> on 2020-01-21 15:31". 

A couple of things:
1. Your VCS will tell you who generated the code
2. The name of the file including “auto” in it doesn’t mean that it wasn’t 
completely edited by a developer afterwards, making the “auto” useless in 
knowing who generated the code

A common operation when looking at a list of migrations is to try and find a 
specific one. I think this is a much more common occurrence than looking at the 
list of migrations to know how many have been (possibly) generates by Django.

This would make the common case more convenient and simpler, which is good.

Tom

>> On 25 Apr 2020, at 17:45, אורי  wrote:
> 
> 
> 
>> On Sat, Apr 25, 2020 at 5:48 PM Adam Johnson  wrote:
>> Re: Uri: 
>>> If the file name is not like ‘auto’ name with the current date + time, it 
>>> looks like a migration which was written by a developer.
>> 
>> I think making a distinction between "auto generated" and "hand written" 
>> migrations is a bad idea. Django's makemigrations is a code generator, but 
>> as a developer you're still responsible for the code. The autodetector isn't 
>> perfect, and perhaps never can be. Even "simple" cases like adding a through 
>> table to a ManyToManyField are autodetected "incorrectly" ( a real migration 
>> needs SeparateDatabaseAndState 
>> https://docs.djangoproject.com/en/3.0/howto/writing-migrations/#changing-a-manytomanyfield-to-use-a-through-model
>>  ).
>> 
>> I fear marking "auto generated" migrations differently would just encourage 
>> (more) lax use of migrations files without reading their content, which 
>> invites more risk for data loss and anger with Django for not being perfect.
> As a developer I would like to know who generated the code. If a migration is 
> auto-generated, I would like to know that. I checked and auto-generated 
> migrations in my project have a comment such as "# Generated by Django 2.1.15 
> on 2020-01-21 15:31". In most cases auto-generated migrations are good and 
> don't need to be edited by a developer. So, I would prefer the migration file 
> names to be auto-generated too (‘auto’ name with the current date + time).
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/CABD5YeFvWJSotjqhB_L%3Dt9i%3Dx3%3DRt%2Bh-S_gYCAgTV3C8RFMjwg%40mail.gmail.com.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CC4B31C7-EC63-4531-BF69-3672B9909133%40tomforb.es.

Re: Deadlock bug in logging? Reproducible case

[Tom Forbes]

Just to confirm - are you running into this issue while using the Django 
development server in production, or does this occur with Gunicorn/uwsgi as 
well?

Tom

> On 23 Apr 2020, at 12:40, Brian Tiemann  wrote:
> 
> It also happens in 3.6.9, which is my prod environment.
> 
> FWIW, by way of background context, this is not a heavily used app, not 
> enough so to warrant logging to a custom buffered consumer or remote agent. 
> It's just a small office using it. But the main home view's UI has 5 AJAX 
> calls that cause the site to lock up on a regular basis, such that I have to 
> have a watchdog in place to bounce Apache whenever it happens.
> 
> It's not the logging itself that's the issue; none of the AJAX views log 
> anything. The app only actually logs maybe 100 lines per day. My 
> investigation however turned up that if I commented out LOGGING altogether, 
> it worked fine. Further trimming down of the codebase showed that I could 
> remove all the loggers and it would still happen as long as any handlers were 
> defined. So that's how I set up the demo. The logging itself isn't important, 
> in other words; it's that the whole app is wedging just because I happen to 
> have a vanilla LOGGING setup configured.
> 
> 
> 
> On Thursday, April 23, 2020 at 7:02:11 AM UTC-4, Brian Tiemann wrote:
> Hi — Thanks for looking into this. Adam, I skipped over details like the 
> Python version because I noted them in the readme on my gitlab demo. But I 
> was using Python 3.7.2 for the record.
> 
> I used "deadlock" because my digging into this had previously turned up 
> https://bugs.python.org/issue6721  , 
> suggesting this was to do with thread locking. But I'm not qualified to make 
> that call for sure, so I don't want to throw red herrings into the mix. If 
> "standstill" is a better term for this that's what I'll use.
> 
> Thanks again.
> 
> 
> On Wednesday, April 22, 2020 at 1:23:31 PM UTC-4, Brian Tiemann wrote:
> Hi all,
> 
> I was directed here after getting corroboration on #django and elsewhere.
> 
> I have what appears to be a bug in which a Django app can deadlock if you hit 
> it with a lot (>3) of AJAX requests within a short time (i.e. all triggered 
> in parallel from a single HTML page). I have a reproducible case here:
> 
> https://github.com/data-graham/wedge 
> 
> I'd really appreciate it if someone could take a look and let me know whether 
> I'm doing something wrong, or if there's something systemic going on. Thanks 
> much!
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/9a5bc078-66b9-4e09-97c2-8340b8882c02%40googlegroups.com
>  
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/9B0CF9F5-DB99-4D82-BAFC-4D9E481BCFD7%40tomforb.es.

Re: GSoC Proposal to extend the parallel test runner to Windows and macOS and to support Oracle parallel test running

[Tom Forbes]

You’re both most likely correct, but I think it’s good to get some numbers and 
compare implementation complexity. 

It’s also worth considering that often as a project grows in complexity the 
time it takes to do a full test run grows with it, so perhaps the overhead 
would be in a fairly constant (and reasonable) percentage of overall runtime.

Or maybe not! I just think we should benchmark the naive implementation along 
with the others.

Tom

>> On 30 Mar 2020, at 19:41, charettes  wrote:
> 
> I agree with Adam here, it not just about running the SQL itself; the current 
> migration framework require models to be (re-)rerendered before a lot of 
> schema operations which slows down dramatically as project grows.
> 
> The dump/restore approach will undeniably be faster than running migrations 
> for each test process.
> 
> Simon
> 
> Le lundi 30 mars 2020 14:25:05 UTC-4, Adam Johnson a écrit :
>> 
>>> I just finished rewriting it to re-run migrations and setup in each worker. 
>>> I didn't bench mark it so I'll do that again later, but as a gut reaction 
>>> it isn't significantly slower; but of course the numbers will tell the real 
>>> story.
>> 
>> I'm a little concerned about rerunning migrations in each worker in case 
>> they are not deterministic (for some maybe slightly odd projects).
>> 
>> Also I'd expect in many projects a dump and restore will be faster than 
>> running migrations.
>> 
>>> On Mon, 30 Mar 2020 at 13:13, Ahmad A. Hussein  wrote:
>>> You're definitely right!
>>> 
>>> I just finished rewriting it to re-run migrations and setup in each worker. 
>>> I didn't bench mark it so I'll do that again later, but as a gut reaction 
>>> it isn't significantly slower; but of course the numbers will tell the real 
>>> story.
>>> 
>>> Passing the dump string through shared memory or converting in-memory to on 
>>> disk seems way too overkill now. The performance drop probably won't be 
>>> significant especially since this is SQLite.
>>> 
>>> I'll update you here with the exact benchmarks later. I'm busy now with 
>>> school work, but I'll get back to it again later today.
>>> 
>>> It is definitely exciting to find out!
>>> 
>>> Regards,
>>> Ahmad
>>> 
>>>> On Mon, Mar 30, 2020 at 11:46 AM Tom Forbes  wrote:
>>>> Awesome! However there is always a tradeoff between speed and code 
>>>> complexity - it might not be worth a more complex system involving shared 
>>>> memory or copying SQLite files in other ways if it only saves ~5 seconds 
>>>> at the start of a test run compared to re-running the migrations in each 
>>>> worker. I guess this might be unique to SQLite where things are generally 
>>>> faster.
>>>> 
>>>> Just something to keep in mind, I’m excited to see the results though!
>>>> 
>>>> Tom
>>>> 
>>>>> On 30 Mar 2020, at 10:34, Ahmad A. Hussein  wrote:
>>>>> 
>>>>>  I did stumble upon shared memory when I originally wanted to pass the 
>>>>> database connection itself into shared memory. Your idea of passing the 
>>>>> query string is MUCH more reasonable, can be done using shared memory, 
>>>>> and will ultimately be much more cleanly written.
>>>>> 
>>>>> i'll definitely test out if it has significant performance boost and see 
>>>>> how it compares to other methods.
>>>>> 
>>>>> I did consider copying the SQLite database file itself. That's 
>>>>> essentially what's happening with VACUUM INTO. It's a SQLite command that 
>>>>> lets you clone a source database whether in-memory or on disk into a 
>>>>> database file. I got the parallel test runner to work with these database 
>>>>> files, creating a database file for each worker. I think there's a 
>>>>> performance boost to have if we make the databases for workers in-memory, 
>>>>> especially if the test suite and tables grow in number; however, we also 
>>>>> need to consider the time it takes to make the databases in-memory.
>>>>> 
>>>>> Re-running migrations and recreating the SQLite database from scratch for 
>>>>> every worker might be a time-sink, but it could actually prove quicker 
>>>>> than other methods. I'll try it out as well and see.
>>>>> 

Re: GSoC Proposal to extend the parallel test runner to Windows and macOS and to support Oracle parallel test running

[Tom Forbes]

Awesome! However there is always a tradeoff between speed and code complexity - 
it might not be worth a more complex system involving shared memory or copying 
SQLite files in other ways if it only saves ~5 seconds at the start of a test 
run compared to re-running the migrations in each worker. I guess this might be 
unique to SQLite where things are generally faster.

Just something to keep in mind, I’m excited to see the results though!

Tom

> On 30 Mar 2020, at 10:34, Ahmad A. Hussein  wrote:
> 
>  I did stumble upon shared memory when I originally wanted to pass the 
> database connection itself into shared memory. Your idea of passing the query 
> string is MUCH more reasonable, can be done using shared memory, and will 
> ultimately be much more cleanly written.
> 
> i'll definitely test out if it has significant performance boost and see how 
> it compares to other methods.
> 
> I did consider copying the SQLite database file itself. That's essentially 
> what's happening with VACUUM INTO. It's a SQLite command that lets you clone 
> a source database whether in-memory or on disk into a database file. I got 
> the parallel test runner to work with these database files, creating a 
> database file for each worker. I think there's a performance boost to have if 
> we make the databases for workers in-memory, especially if the test suite and 
> tables grow in number; however, we also need to consider the time it takes to 
> make the databases in-memory.
> 
> Re-running migrations and recreating the SQLite database from scratch for 
> every worker might be a time-sink, but it could actually prove quicker than 
> other methods. I'll try it out as well and see.
> 
> Thank you for your suggestions
> 
> Regards,
> Ahmad  
> 
> On Mon, Mar 30, 2020 at 9:39 AM Tom Forbes  <mailto:t...@tomforb.es>> wrote:
> There is an interesting addition to the standard library in Python 3.8: 
> multiprocessing.shared_memory 
> (https://docs.python.org/3/library/multiprocessing.shared_memory.html 
> <https://docs.python.org/3/library/multiprocessing.shared_memory.html>). It’s 
> only on 3.8 but it might be worth seeing if that has a performance 
> improvement over passing a large string into the workers init process.
> 
> Have you also considered simply re-running the migrations and creating the 
> SQLite database from scratch in each worker, or simply copying the SQLite 
> database file itself?
> 
>> On 30 Mar 2020, at 08:13, Ahmad A. Hussein > <mailto:ahmadahusse...@gmail.com>> wrote:
>> 
>> 
>> Thank you Adam for the amazing pointer. That's exactly the solution I was 
>> looking for. 
>> 
>> I didn't even know VACUUM INTO was a thing in SQLite3 (quite nifty). Both 
>> sources you listed can definitely copy memory-to-memory efficiently. I had 
>> also found two useful APIs in the standard library's documentation: 
>> iterdump() and backup(). The latter is an attempt at recreating the online 
>> backup API you linked.
>> 
>> I did face an issue in my implementation though. Memory-to-memory copying is 
>> impossible under the current conditions.
>> 
>> We can't copy memory-to-memory and retain the resultant databases into our 
>> spawned workers since spawned child processes don't share an environment 
>> with parent processes.
>> 
>> There are two possible workarounds:
>>  1. To copy our in-memory database into 
>> an on disk database using VACUUM INTO and subsequently restore it into 
>> in-memory during worker initialization
>>  2. Pass a string query resulting from 
>> calling Connection.iterdump() into worker initialization and call that to 
>> restore copies of the database into in-memory for every child process.
>> 
>> I'm testing out both now to see if there are any major differences. There 
>> might be a performance difference between them. I'm also checking if there 
>> are any unexpected errors that might arise.
>> 
>> Regards,
>> Ahmad
>> 
>> On Sun, Mar 29, 2020 at 5:56 PM Adam Johnson > <mailto:m...@adamj.eu>> wrote:
>> Currently what I'm figuring out is getting a SQL dump to change SQLite's 
>> cloning method and implementing an Oracle cloning method. I'm searching 
>> through Django's documentation and internal code to see if there is a 
>> ready-made SQL dump method for SQLite and if not I'll search for it in a 
>> third-party library, and if I still don't find it, I'll start thinking about 
>> a ground-up implementation using the O

Re: GSoC Proposal to extend the parallel test runner to Windows and macOS and to support Oracle parallel test running

[Tom Forbes]

There is an interesting addition to the standard library in Python 3.8: 
multiprocessing.shared_memory 
(https://docs.python.org/3/library/multiprocessing.shared_memory.html). It’s 
only on 3.8 but it might be worth seeing if that has a performance improvement 
over passing a large string into the workers init process.

Have you also considered simply re-running the migrations and creating the 
SQLite database from scratch in each worker, or simply copying the SQLite 
database file itself?

>> On 30 Mar 2020, at 08:13, Ahmad A. Hussein  wrote:
> 
> Thank you Adam for the amazing pointer. That's exactly the solution I was 
> looking for. 
> 
> I didn't even know VACUUM INTO was a thing in SQLite3 (quite nifty). Both 
> sources you listed can definitely copy memory-to-memory efficiently. I had 
> also found two useful APIs in the standard library's documentation: 
> iterdump() and backup(). The latter is an attempt at recreating the online 
> backup API you linked.
> 
> I did face an issue in my implementation though. Memory-to-memory copying is 
> impossible under the current conditions.
> 
> We can't copy memory-to-memory and retain the resultant databases into our 
> spawned workers since spawned child processes don't share an environment with 
> parent processes.
> 
> There are two possible workarounds:
>  1. To copy our in-memory database into 
> an on disk database using VACUUM INTO and subsequently restore it into 
> in-memory during worker initialization
>  2. Pass a string query resulting from 
> calling Connection.iterdump() into worker initialization and call that to 
> restore copies of the database into in-memory for every child process.
> 
> I'm testing out both now to see if there are any major differences. There 
> might be a performance difference between them. I'm also checking if there 
> are any unexpected errors that might arise.
> 
> Regards,
> Ahmad
> 
> On Sun, Mar 29, 2020 at 5:56 PM Adam Johnson  wrote:
>>> Currently what I'm figuring out is getting a SQL dump to change SQLite's 
>>> cloning method and implementing an Oracle cloning method. I'm searching 
>>> through Django's documentation and internal code to see if there is a 
>>> ready-made SQL dump method for SQLite and if not I'll search for it in a 
>>> third-party library, and if I still don't find it, I'll start thinking 
>>> about a ground-up implementation using the ORM.
>> 
>> SQLite has an online backup API and the "VACUUM INTO" command both of which 
>> can efficiently clone databases: https://sqlite.org/backup.html . I think 
>> they can even copy memory-to-memory with the right arguments.
>> 
>>> On Sat, 28 Mar 2020 at 06:59, Ahmad A. Hussein  
>>> wrote:
>>> Apologies for the late response. I've had to attend to personal matters 
>>> concerning the current crisis we're all facing. All is well though
>>> 
>>> I should have posted a more detailed post from the get-go. I apologize for 
>>> the lack of clarity as well.
>>> 
>>> Last week, I initially did exactly what you suggested. I called 
>>> django.setup() in each child process during worker initialization. This 
>>> fixed app registry issues but like you said it isn't enough for testing. 
>>> Test apps and test models were missing and caused tons of errors. Later, I 
>>> read through runtests.py and saw the setup method there; it was exactly 
>>> what I needed since it searched for setup the correct template backend, 
>>> searched for test apps and added them to installed apps, and called 
>>> django.setup(). I passed that method along with its initial arguments into 
>>> the runner so I could call it during worker initialization. That fixed all 
>>> errors related to state initialization. I do wonder though if any 
>>> meaningful time could be saved if we use a cache for setup instead of 
>>> having to call the setup for each process.
>>> 
>>> The last glaring hole was correct database connections. I had a naming 
>>> mismatch with Postgres and I ended up fixing that through prepending 
>>> "test_" in each cloned database's name during worker initialization. In 
>>> case of the start method being fork, we can safely ignore that step and 
>>> it'll work fine. 
>>> 
>>> Currently what I'm figuring out is getting a SQL dump to change SQLite's 
>>> cloning method and implementing an Oracle cloning method. I'm searching 
>>> through Django's documentation and internal code to see if there is a 
>>> ready-made SQL dump method for SQLite and if not I'll search for it in a 
>>> third-party library, and if I still don't find it, I'll start thinking 
>>> about a ground-up implementation using the ORM. 
>>> 
>>> As for progress on the Oracle cloning method, I'm consulting Oracle 
>>> documentation right now to see if anything has changed in the last 5 years. 
>>> If I don't find anything interesting, I'll start toying around with Shai 
>>> Berger's ideas to see what works and what's performance-costly.
>

Re: GSoC proposal research - type hinting

[Tom Forbes]

As an avid user of type hints I think it would be fantastic. I believe with the 
upcoming release we only target Python versions that support type hinting, 
which was one of the issues holding us back before.

I would recommend choosing a specific part of Django and investigating adding 
type hints. Django is quite big, and I think most of the value for type hints 
comes from user-facing components rather than some internals.

I would start looking at:
1. The ORM
2. Class based views
3. The Request/Response objects

And see if you think it’s viable to add a decent amount of type hints to any of 
these. There is some “magic” in each of these which might be hard to express 
with type hints, but it would be valuable to know how far we could get.

Tom

> On 27 Mar 2020, at 23:07, Kacper Szmigiel  wrote:
> 
> 
> Hello everyone!
> 
> My name is Kacper, I'm 22 y.o. CS student from Lodz, Poland. I've started 
> researching some nice feature for my GSoC proposal, and I have a few 
> questions to you, dear Community :)
> Unfortunately, due to this whole mess caused by covid-19 I had like no time 
> to sink into Django codebase and figure some things out by myself (none of 
> polish Universities were ready to switch to e-learning). I have a few years 
> of experience in using Django for my commercial projects, but I had no 
> occasion to contribute to the framework yet.
> 
> What I would like to implement is type hinting, as suggested here: 
> https://code.djangoproject.com/ticket/29299
> 
> So, my first question is: what do you think about this one? Does it make 
> sense at all? I suppose such feature could be really useful for many of 
> developers 
> (https://www.willmcgugan.com/blog/tech/post/adding-type-hints-to-the-django-orm/).
> 
> As I mentioned, I had no time to read the code and understand how Django 
> works. Do you have any tips on where should I start my technical research in 
> reference to this feature? I know I have not much time to prepare my 
> proposal, but I will do my best during this weekend and send it to get some 
> feedback.
> 
> Thanks in advance, any tips and tricks welcome!
> 
> Kindest regards,
> Kacper Szmigiel
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/61592cd2-87b5-4d6b-a2f8-52a3dbf3fa21%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/FAFB6CCC-0A8E-4272-AE8D-F808B285981E%40tomforb.es.

Re: GSoC Mentors

[Tom Forbes]

Happy to help any way I can (very time constrained in the near future, but 
that should ease off). 

I think sharing the burden as a group is a great idea.

On Wednesday, 25 March 2020 08:43:49 UTC, Carlton Gibson wrote:
>
> Hi all. 
>
> We're reaching the end of the application period for GSoC. 
>
> In order to know how many students we might accept, we need to know how 
> many prospective mentors we have. 
>
> This falls into two kinds of job: 
>
> 1. General project management help: communicating with students to help 
> them set a schedule and a rhythm, and make sure they're able to make 
> progress. (The hope is they're self motivated but...) 
>
> 2. More technical input. For this I'm thinking particularly about the 
> migrations framwork (Markus, Simon, Shai, Andrew, ...) but *experienced 
> hands* with knowledge of the internals: your input would be invaluable 
> here. 
>
> The forum seems to be working well, and it's a good format for this kind 
> of thing, so I want to aim to focus the discussion there. 
>
> I'd like to share the mentoring as a group. We'd need to assign specific 
> mentors but I don't see why it can't be a group thing. 
>
> If you could hang out a bit on the forum (mainly for task 1) and/or offer 
> technical input, which you might be doing already/anyway (for task 2) then 
> that would be a super contribution. 
>
> Please let me know if you'd be willing to help mentor. 
>
> Thanks
>
> Kind Regards,
>
> Carlton
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/447df6b5-5bf2-45a2-985d-59a239c285b2%40googlegroups.com.

Re: GSoC Proposal to extend the parallel test runner to Windows and macOS and to support Oracle parallel test running

[Tom Forbes]

Ahh, interesting! I think I can help mentoring, I’ll leave a reply in the 
official thread 💪

> On 26 Mar 2020, at 11:40, Carlton Gibson  wrote:
> 
> Hi Tom.
> 
> (I was wondering if you might fancy mentoring on this project? 🙂)
> 
> On Thursday, 26 March 2020 12:35:34 UTC+1, Tom Forbes wrote:
> > incompatible in macOS due to a recent update
> 
> Can you elaborate on this a bit? How has a recent update made forking 
> incompatible on MacOS?
> 
> macOS has deprecated the fork launch method, to be removed in a future 
> version.
> 
> Python 3.8+ then swaps the default launch method to spawn, so on Python 3.8+ 
> we no longer have parallel test running on the mac.
> 
> (For now, fork still actually works, so you can `set_start_method()` to work 
> around this, but...)
> 
> Kind Regards,
> 
> Carlton
> 
> 
> --
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com 
> <mailto:django-developers+unsubscr...@googlegroups.com>.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/e5a1464f-553c-4990-8dd2-6c4086e329c4%40googlegroups.com
>  
> <https://groups.google.com/d/msgid/django-developers/e5a1464f-553c-4990-8dd2-6c4086e329c4%40googlegroups.com?utm_medium=email&utm_source=footer>.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/6BEF04AA-F616-46E9-85F8-BD8CF8C38533%40tomforb.es.


signature.asc
Description: Message signed with OpenPGP

Re: GSoC Proposal to extend the parallel test runner to Windows and macOS and to support Oracle parallel test running

[Tom Forbes]

> incompatible in macOS due to a recent update

Can you elaborate on this a bit? How has a recent update made forking 
incompatible on MacOS?

Tom

> On 23 Mar 2020, at 19:22, Ahmad A. Hussein  wrote:
> 
> Django's parallel test runner works through forking processes, making it 
> incompatible on Windows by default and incompatible in macOS due to a recent 
> update. Windows and macOS both support spawn and have it enabled by default. 
> Databases are cloned for each worker.
> 
> To switch from fork to spawn, state setup will be handled by spawned 
> processes instead of through inheritance via fork. Worker’s connections to 
> databases can still be done through get_test_db_clone_settings which changes 
> the names of databases assigned to a specific worker_id; however, SQLite's 
> cloning method is incompatible with spawn.  
> 
> 
> SQLite’s cloning method relies on it both being in-memory and fork as when we 
> fork the main process, we retain SQLite's in-memory database in the child 
> process. The solution is to get a SQL dump of the database and throw it into 
> the target cloned databases. This is also the established standard in 
> handling MySQL’s cloning process.
> 
> Both Postgresql's and MySQL's cloning methods are independent of fork or 
> spawn and won't require any modification.
> 
> Oracle has not been implemented in the parallel test runner originally even 
> on Linux and I propose to extend support to Oracle as well in my proposal. I 
> want to confirm if there is significant support behind this as a feature or 
> not before I commit to writing a specification, but as a summary it is 
> definitely possible as the work collaborated on by Aymeric Augustin and Shai 
> Berger show that cloning CAN be done through multiple ideas. The reason why 
> it's a headache is that Oracle does not support separate databases under a 
> single user- unlike our other supported databases, so we can't clone 
> databases without introducing another user. Some methods may also need to be 
> rewritten to accommodate for the Oracle backend, but that isn't an issue.
> 
> I've glossed over writing out a schedule or a more detailed abstract as I I'm 
> mainly posting here to see if there is indeed support for the Oracle proposal 
> and to make sure I am not missing any details in regards to adapting the 
> current parallel test runner to work through spawn. Let me know what you 
> think.
> 
> Regards,
> Ahmad
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/317f67c6-4b23-483f-ada5-9bdbb45d0997%40googlegroups.com
>  
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/322804F5-F44F-4EF8-AF33-C460EC509915%40tomforb.es.

Re: New Merger nomination.

[Tom Forbes]

Thanks for that James! Out of interest what is the exact implementation of 
the public vote? Is that performed at a meeting, via the mailing list or 
via the same/similar software used to elect the DSF board?

Tom

On Friday, 13 March 2020 20:25:58 UTC, James Bennett wrote:
>
> So I guess it's worth walking through how to do this. 
>
> The first step would be a member of the Technical Board deciding 
> Mariusz' suggestion is a good one, and nominating Claude to be a 
> Merger, putting the question to the full Technical Board for voting: 
> "Shall Claude be appointed a Merger?" 
>
> The Technical Board would have one week to cast their votes (+1, 0, or 
> -1) on the question. Those votes must be cast publicly. 
>
> If the result of that vote is 3 or higher, Claude would be appointed a 
> Merger, though the appointment would be a temporary one because of the 
> pending Technical Board election (an outgoing Technical Board can only 
> make temporary appointments under DEP 10). Once the election is over, 
> the new Technical Board could make the appointment permanent. 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/a8835cc9-56d2-41d0-a6a9-23d3afc7a8b8%40googlegroups.com.

Re: [Probably BUG] set_password and check_password accept values other than string as parameters

[Tom Forbes]

There is an argument to be made for not using force_bytes, but there is 
also a semantic difference between “force” and “to”. I wouldn’t think it 
would be possible to change force_bytes to throw an exception without 
serious compatibility issues - str() is how lazy objects are resolved IIRC, 
and there are numerous other places it is called with arguments that need 
to be cast to a string.

The behavior of the make_password method is quite surprising to be honest, 
and maybe the advantages of being able to pass any object into the method 
is entirely academic because nobody passes anything but strings on purpose. 
On the other hand this is the first post I can remember about this. 

On Thursday, 12 March 2020 19:55:46 UTC, Dawid Czeluśniak wrote:
>
> Tom,
>
> I believe that I found the root cause of this issue. Let's take a closer 
> look at force_bytes 
> <https://github.com/django/django/blob/master/django/utils/encoding.py#L82> 
> function from Django and to_bytes 
> <https://github.com/pallets/werkzeug/blob/master/src/werkzeug/_compat.py#L195>
>  
> function from Werkzeug (just for comparison). There is one *fundamental* 
> difference 
> between these: force_bytes from Django fallbacks to string representation 
> of the object whereas to_bytes from Werkzeug raises TypeError if it can't 
> cast the value to bytes. That's why when calling make_password with 
> object that can't be "easily" cast to bytes returns a generated hash:
>
> In [1]: from django.contrib.auth.hashers import make_password
>
> In [2]: make_password([1, 2, 3])
> Out[2]: 
> 'pbkdf2_sha256$18$Y9azq0uVWSoh$E7O13LefzP1I4MylXfFYKDkgCnZen6tf+FAblnBYssQ='
>
> whereas generate_password_hash from Werkzeug raises TypeError:
>
> In [16]: from werkzeug.security import generate_password_hash
>
> In [17]: generate_password_hash([1, 2, 3])
> TypeError: Expected bytes
>
> IMHO throwing an exception is a more reasonable approach.
>
> Anyway, thank you for your help :)
> Dawid
>
> On Thursday, 12 March 2020 19:17:15 UTC+1, Tom Forbes wrote:
>>
>> In this context it means that you shouldn’t encrypt, hash or otherwise 
>> manipulate the password before passing it into the method. 
>>
>> Django, many other packages and Python itself will accept objects that 
>> can be coerced into a string (via __str__) rather than throw an exception. 
>> We’re all consenting adults here - if you want to pass a non-string object 
>> into “make_password” then that’s up to you.
>>
>> The question really is if this is a common enough mistake to warrant a 
>> guard against strange input. I’d say no, however a small change to the 
>> documentation might be in order.
>>
>> Tom
>>
>> On 12 Mar 2020, at 17:41, Dawid Czeluśniak  wrote:
>>
>> Adam,
>>
>> If it's perfectly fine to pass almost any not-None object to 
>> make_password function and it returns correctly generated hash then why 
>> does the documentation say:
>>
>> make_password(password, salt=None, hasher='default')
>>> Creates a hashed password in the format used by this application. It 
>>> takes one mandatory argument: the password in *plain-text*.
>>
>>  
>>
>> https://docs.djangoproject.com/en/3.0/topics/auth/passwords/#django.contrib.auth.hashers.make_password
>>
>> What does "plain-text" mean there?
>>
>> Thanks,
>> Dawid
>>
>>
>>
>> On Thursday, 12 March 2020 18:18:59 UTC+1, Adam Johnson wrote:
>>>
>>> User provided passwords are validated already: 
>>> https://docs.djangoproject.com/en/3.0/topics/auth/passwords/#module-django.contrib.auth.password_validation
>>>
>>> When using set_password directly, you as the programmer are responsible 
>>> for ensuring the value you use for password is valid. Normally this means 
>>> calling the functions detailed in "Integrating validation" beforehand.
>>>
>>> -- 
>>> Adam
>>>
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-d...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-developers/bda75e4c-2dae-42ef-91f3-c3054031c800%40googlegroups.com
>>  
>> <https://groups.google.com/d/msgid/django-developers/bda75e4c-2dae-42ef-91f3-c3054031c800%40googlegroups.com?utm_medium=email&utm_source=footer>
>> .
>>
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/c36ce153-4464-4fac-b620-a2c9a2d80e2f%40googlegroups.com.

Re: New Merger nomination.

[Tom Forbes]

+1 from me! Seems like a no-brainer to make Claude a merger, his past and 
present contributions have been fantastic.

On Friday, 13 March 2020 19:30:19 UTC, Mariusz Felisiak wrote:
>
> I nominate Claude Paroz to a new MERGER for 8 years of contributions to 
> Django, including triaging tickets, reviewing PRs, and serving as the 
> Django translations manager. Claude is one of the most  active contributor 
> with outstanding impact on Django. He was also awarded the 2017 Malcolm 
> Tredinnick Memorial Prize (
> https://www.djangoproject.com/weblog/2018/jan/22/2017-malcolm-tredinnick-prize-claude-paroz/
> ).
>
> Best,
> Mariusz Felisiak
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/3a6bda3a-5d97-4883-aaa8-ef42562d5559%40googlegroups.com.

Re: [Probably BUG] set_password and check_password accept values other than string as parameters

[Tom Forbes]

In this context it means that you shouldn’t encrypt, hash or otherwise 
manipulate the password before passing it into the method. 

Django, many other packages and Python itself will accept objects that can be 
coerced into a string (via __str__) rather than throw an exception. We’re all 
consenting adults here - if you want to pass a non-string object into 
“make_password” then that’s up to you.

The question really is if this is a common enough mistake to warrant a guard 
against strange input. I’d say no, however a small change to the documentation 
might be in order.

Tom

> On 12 Mar 2020, at 17:41, Dawid Czeluśniak  wrote:
> 
> Adam,
> 
> If it's perfectly fine to pass almost any not-None object to make_password 
> function and it returns correctly generated hash then why does the 
> documentation say:
> 
> make_password(password, salt=None, hasher='default')
> Creates a hashed password in the format used by this application. It takes 
> one mandatory argument: the password in plain-text.
>  
> https://docs.djangoproject.com/en/3.0/topics/auth/passwords/#django.contrib.auth.hashers.make_password
>  
> 
> 
> What does "plain-text" mean there?
> 
> Thanks,
> Dawid
> 
> 
> 
> On Thursday, 12 March 2020 18:18:59 UTC+1, Adam Johnson wrote:
> User provided passwords are validated already: 
> https://docs.djangoproject.com/en/3.0/topics/auth/passwords/#module-django.contrib.auth.password_validation
>  
> 
> 
> When using set_password directly, you as the programmer are responsible for 
> ensuring the value you use for password is valid. Normally this means calling 
> the functions detailed in "Integrating validation" beforehand.
> 
> -- 
> Adam
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/bda75e4c-2dae-42ef-91f3-c3054031c800%40googlegroups.com
>  
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/0E23455A-7EED-4EE5-AA90-8763BA68E5D7%40tomforb.es.

Re: Use "raise from" where appropriate, all over the codebase

[Tom Forbes]

I agree with this change from a correctness standpoint but I would like to make 
the point that chained exceptions might be slightly annoying when displayed via 
console output, as you see the inner exception first and have to scroll up to 
see the exception you actually have to handle.

Tom

> On 18 Jan 2020, at 09:55, Ram Rachum  wrote:
> 
> 
> Hi guys,
> 
> I recently made a big ticket/PR to Django, and Shai Berger told me I should 
> first talk about it in this mailing list.
> 
> This is the ticket: https://code.djangoproject.com/ticket/31177 and its PR: 
> https://github.com/django/django/pull/12339
> 
> It's a generalization of this ticket that I opened and wrote a patch for a 
> few days ago: https://code.djangoproject.com/ticket/31166 It was discussed 
> and merged.
> 
> Basically, I think that in any place where we catch an exception, and then 
> wrap it with our own exception for whatever reason, we should use the "raise 
> new_exception from old_exception" syntax rather than just "raise 
> new_exception".
> 
> This means that when Python displays the stacktrace (and when we display it 
> in the debug page,) this will make it have a text of "this exception is the 
> direct result of" instead of "During handling of the above exception, another 
> exception occurred". This is more accurate, because the latter basically 
> means "there was an error in the process of error-handling," which is 
> absolutely not the case for us, and could mislead a user.
> 
> Note that we already started transitioning to "raise from", slowly:
>  - Commit by Mariusz Felisiak: 
> https://github.com/django/django/commit/84dcd1624784c2239f96a97b3151145a85dfbbe3
>  - Commit by Diederik van der Boor: 
> https://github.com/django/django/commit/25f21bd2376603c8e233a0a0e5a726a0fdfdd33e
>  - Commit by Thomas Allison: 
> https://github.com/django/django/commit/3e8b7333904f1ab6aa18eeb508659256f3644816
> 
> What do you think? 
> 
> 
> Thanks,
> Ram.
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/25086cdc-24ab-4f0a-bdb9-9756551ac170%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/74BA6A05-D971-47EF-A0B6-4EF1149E83E6%40tomforb.es.

Re: declarative settings

[Tom Forbes]

My two cents: JSON isn’t great as a configuration language - It’s annoyingly 
picky about some things. YAML or TOML are “better” (for some definition of 
better) choices for this domain, in my option. However, Django is historically 
quite hesitant about including third party packages and I think including a 
TOML or YAML parser even as an optional dependency might not be the way forward.

So rather than just “allowing people to use JSON files for settings” it would 
be very interesting to explore what a pluggable settings backend would look 
like. It seems that work in this area would be best spent on a general 
abstraction rather than a specific one. There was some discussion around this 
idea in the GSOC thread and I’m sure it’s come up before. Personally I think 
configuration management is an area of Django that is quite underdeveloped 
compared to other frameworks.

> On 30 Dec 2019, at 22:46, Christian González  
> wrote:
> 
> Hello,
> 
> I recently "finished" my first really working version of GDAPS, my
> Generic Django Application Plugin System. It's noway perfect, but does
> what it should: providing pluggable apps for an app framework, including
> a more or less flexible frontend with each django app.
> 
> I had much struggle with it, and one of the lessons I learned was
> Django's setup system, and how it deals with loading apps. Unfortunately
> Django can't load/unload apps on the fly, so it is necessary to restart
> Django whenever a new GDAPS app is installed via pip.
> 
> But: I want to resurrect an old theme again which would, in a way,
> improve some of the loading problems I encountered. Django's settings
> are code. Which is, in fact, a very good thing, as it makes it extremely
> flexible and adaptable to different setups. But, as discussed with the
> SECRET_KEY here, some of the settings _have_ to be coded very
> complicated, and it makes some things like per-app-settings extremely
> uncomfortable.
> 
> What if - and please don't kill me instantly - yes, I am a newcomer, and
> not a good programmer maybe - but some things are viewed better from
> "outside" - what if Django settings could be "declarative"?
> 
> So instead of Python code like
> 
> INSTALLED_APPS = [
> 'django.contrib.admin',
> 'django.contrib.auth',
> 'django.contrib.contenttypes'
> ]
> 
> This would be in an e.g. JSON file
> 
> {
> 
> "INSTALLED_APPS": [
> "django.contrib.admin",
> "django.contrib.auth",
> "django.contrib.contenttypes"
> ] ,
> ROOT_URLCONF: "fooproject.urls"
> }
> 
> Django's settings.py would look different: It would load that
> settings.json file and set the appropriate values into local code - so
> this wouldn't make much difference.
> 
> Except 2 things:
> 
> 1. Apps could have (default) settings, and they could be merged MUCH
> easier. Things like namespaced classes that are overwriting values like
> DRF/graphene does, would be completely unnecessary. The main
> settings.json file could be the "last word" in the process of settings,
> so anything an app would suggest could be overrided in the main file.
> 
> 2. Installed apps could be managed much more comfortable. Adding an app
> could be done by a script (JSON editing is easy. Editing code
> (=settings.py) is error prone and uncomfortable). I have a Django
> command script ATM for that, but just because I add a line into
> settings.py to add some additional apps to the list.
> 
> This even could be done with backwards compatibility, because Django
> would keep it's settings.py file optionally:
> 
> * read json settings (if they exist), use them
> * load settings.py which allows to override them again (using some
> special code tricks like dynamic loading, environments etc.)
> 
> Please tell me what you think about that.
> 
> Christian
> 
> 
> --
> Dr. Christian González
> https://nerdocs.at
> 
> --
> You received this message because you are subscribed to the Google Groups 
> "Django developers  (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/3047a7b6-3fa0-d574-4bb6-7842b7aed44a%40nerdocs.at.
> 

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/D7000C61-69C0-4E59-9DB8-4BB1413B0C98%40tomforb.es.


signature.asc
Description: Message signed with OpenPGP

Re: Forms submitted by bots

[Tom Forbes]

IMO this is outside of the scope of this mailing list. There is no simple 
answer to this problem, nor a general solution. There is however a wealth of 
information on this topic that you can find with a few Google searches.

If there is something we can do to Django to make it easier to integrate 3rd 
party bot detection services then that’s something we could discuss, but it’s 
not sensible to include any attempts detection inside Django itself.

One of the better ways of zero-friction verification is Google’s recapture 
service - they have a hidden zero-interaction version that uses google’s 
historical data on that users browsing habits to detect if they are a bot. 

Tom

> On 14 Dec 2019, at 13:29, אורי  wrote:
> 
> 
> Hi,
> 
> I'm looking for a solution which will be zero time wasting for human users, 
> not even clicking on the recaptcha's button, but on the other hand will 
> prevent bots from submitting the form. I'm not sure how this can be 
> implemented technically. But right now the "no bots" field I added prevents 
> all the bots from submitting the forms on our websites. The only disadvantage 
> of it is that it wastes about ~30 seconds of human users who have to type 
> this number for the form to be submitted successfully.
> 
> We also have a registration form but luckily it was not abused (yet) by bots. 
> All the other forms on our website (except registration, login and contact) 
> can only be submitted by logged in users anyway. And if the user is logged in 
> - he is not a bot.
> 
> I have another website which uses MediaWiki and there it was flooded by bots 
> who also signed up for the website and created accounts and then pages. I 
> guess these bots are MediaWiki-specific bots.
> 
> אורי
> u...@speedy.net
> 
> 
>> On Sat, Dec 14, 2019 at 1:41 PM Adam Johnson  wrote:
>> Preventing bot submissions is a bit of an arms race. Django could add some 
>> protection but if many Django sites use it then bot scripts might be adapted 
>> to workaround it.
>> 
>> I've had success using django-recaptcha in the past: 
>> https://github.com/praekelt/django-recaptcha . django-honeypot looks good 
>> too.
>> 
>> I think adding such protection would be a major project - research plus 
>> implementation. I think it could make a good Google Summer of Code project 
>> and will suggest it to the other DSF members.
>> 
>> ‪On Fri, 13 Dec 2019 at 04:02, ‫אורי‬‎  wrote:‬
>>> Thank you. How do I use django-honeypot and how does it work? How does it 
>>> affect our human users and how does it affect the bots? I didn't understand 
>>> from their website.
>>> 
>>> I added a new feature request https://code.djangoproject.com/ticket/31085
>>> אורי
>>> u...@speedy.net
>>> 
>>> 
 On Fri, Dec 13, 2019 at 5:50 AM Kye Russell  wrote:
 This is more of a support question, but: 
 https://github.com/jamesturk/django-honeypot will thwart the majority of 
 (naive) automation attempts. 
 
 
> On 13 December 2019 at 10:42:54 am, אורי (u...@speedy.net) wrote:
> 
> Django developers,
> 
> After releasing Speedy Net to production I received lots of spam to our 
> contact forms [https://en.speedy.net/contact/ & 
> https://en.speedymatch.com/contact/]. I found out that all of these spam 
> messages were produced by bots. I had to add a new "no bots" field to 
> this form, where I just ask users to type a specific number and validate 
> it in the form. Since I added this field I didn't receive any more spam 
> from the contact forms. I know Django is using CSRF cookie directives, 
> but isn't it possible to prevent bots from submitting forms? I would like 
> to remove the "no bots" field from this form as it is wasting time of our 
> users who want to contact us. But I don't want to receive messages from 
> bots. Is there another way to prevent bots from submitting forms?
> 
> אורי
> u...@speedy.net
> --
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/CABD5YeGj%2BFdsrmq%3D_Yai3bJHDSG_5Q1tmXSHLSQv4YexgomZxQ%40mail.gmail.com.
 
 -- 
 You received this message because you are subscribed to the Google Groups 
 "Django developers (Contributions to Django itself)" group.
 To unsubscribe from this group and stop receiving emails from it, send an 
 email to django-developers+unsubscr...@googlegroups.com.
 To view this discussion on the web visit 
 https://groups.google.com/d/msgid/django-developers/CANK-ykkZSVhyDA4nNg0JKudbK9zdyXGVews48MN0pmAT47fb_A%40mail.gmail.com.
>>> 
>>> -- 
>>> You received this message because you are subscribed to the Google Groups 
>>> "Dja

Re: Deprecate HttpRequest.is_ajax

[Tom Forbes]

n decorators have been proposed and refactors.
>>> 
>>> It seems to me that if there is no reliable way of determining it from the 
>>> back-end side, then in the end it will be a convention between the front 
>>> and the back. This could be a GET parameter, a "ClientWants: JSONOrNothing" 
>>> header, or whatever convention you like to make, but not rely on a 
>>> convention which seems to be fading out.
>>> 
>>> You can ensure the actual convention by setting the header manually (as 
>>> stated in the is_ajax doc) - as you do with the CSRF token. Another 
>>> convention could be better (i.e. "accepts json")
>>> 
>>> So far, is what the discussion went through to my understanding :)
>>> 
>>> 
>>> I'm currently handling this with custom headers and the caller (the 
>>> browser) tells the server what kind of outputs it can handle in different 
>>> types of output.
>>> 
>>> The server then performs the branching at certain code points, specifically 
>>> the ones mentioned above. DRF allows me to choose the appropriate renderer. 
>>> Though I should mention here, that the data is already serialised at that 
>>> point: sometimes this creates issues for renderers that might desire more 
>>> information to do their work. Just mentioning that render stages need to be 
>>> accounted for too. This may not be a problem for core Django as it doesn't 
>>> have stages.
>>> 
>>> Again, sorry, but still hoping this helped in some way.
>>> 
>>> 
>>> 
>>> 
>>> LP,
>>> Jure
>>> 
>>> 
>>> 
>>> On 19/11/2019 01:06, Matemática A3K wrote:
>>>> 
>>>> I agree with Adam that it should be deprecated with no replacement.
>>>> 
>>>> The content negotiation is something that should be in but not as a 
>>>> replacement of it, as a general improvement.
>>>> 
>>>> I think there shouldn't be a replacement because "is_ajax" asks whether it 
>>>> came from a ""regular"" browser instead of jQuery and with the content 
>>>> negotiation you ask if the requester accepts a type - which can also lead 
>>>> to errors because the client may also accept other types (no example 
>>>> coming to my mind), and if so, it will lead to undesired behavior.
>>>> 
>>>> The right approach would be making AJAX requests request JSON output 
>>>> explicitly, by using a dedicated endpoint or by appending something that 
>>>> manifests their intention - like in 
>>>> https://docs.djangoproject.com/en/2.2/topics/class-based-views/mixins/#more-than-just-html
>>>>  
>>>> <https://docs.djangoproject.com/en/2.2/topics/class-based-views/mixins/#more-than-just-html>
>>>>  is done with a get parameter. Not decide the response type by where it 
>>>> came from as it is unreliable as stated before, it provides convenience in 
>>>> some use cases but can lead to errors.
>>>> 
>>>> Seems better to me to refactor the view code so you can write a different 
>>>> view for Ajax requests that returns a JSON without code duplication.
>>>> 
>>>> As a shortcut, something like "For simple AJAX endpoints wrap your view 
>>>> with (something like) a "jsonview" decorator which will check the accept 
>>>> header (with something like Claude's code), return the appropriate error 
>>>> code if not, set the response type accordingly, and you should return a 
>>>> dict of strings (you have to take care of the serialization, i.e with 
>>>> https://docs.djangoproject.com/en/2.2/topics/serialization/#serialization-formats-json
>>>>  
>>>> <https://docs.djangoproject.com/en/2.2/topics/serialization/#serialization-formats-json>).
>>>> 
>>>> On Mon, Nov 18, 2019 at 3:28 PM Tom Forbes >>> <mailto:t...@tomforb.es>> wrote:
>>>> What I meant by that is it’s not an approach that scales well to lots of 
>>>> views. It might be better to have separate endpoints to return JSON (e.g 
>>>> adding a /json suffix), and in the past this has made services I’ve worked 
>>>> on a lot more maintainable and easy to understand. But it’s not as quick 
>>>> to do as `if request.is_ajax()` and requires a bit more upfront work. If 
>>>> you find you need to do this a lot then maybe something more structured 
&

Re: Deprecate HttpRequest.is_ajax

[Tom Forbes]

What I meant by that is it’s not an approach that scales well to lots of views. 
It might be better to have separate endpoints to return JSON (e.g adding a 
/json suffix), and in the past this has made services I’ve worked on a lot more 
maintainable and easy to understand. But it’s not as quick to do as `if 
request.is_ajax()` and requires a bit more upfront work. If you find you need 
to do this a lot then maybe something more structured like Django Rest 
Framework will be a better choice, which also handles content negotiation 
really well (it can produce XML, CSV, JSON, etc etc).

> On 18 Nov 2019, at 15:18, Matthew Pava  wrote:
> 
> “In my opinion there are not many good reasons to have to change behaviour if 
> a request is made via XHR. I think the most common usage is to have a single 
> view that returns a JSON response or a HTML response depending on if XHR is 
> used (https://github.com/search?l=Python&q=request.is_ajax&type=Code 
> <https://github.com/search?l=Python&q=request.is_ajax&type=Code>), which 
> isn’t great and isn’t reliable.”
>  
> I do this. What would the best way to handle this? Perhaps the proper 
> practice should be documented when it is deprecated?
>  
> From: django-developers@googlegroups.com 
> [mailto:django-developers@googlegroups.com] On Behalf Of Tom Forbes
> Sent: Saturday, November 16, 2019 10:16 AM
> To: django-developers@googlegroups.com
> Subject: Re: Deprecate HttpRequest.is_ajax
>  
> I would agree. Flask has done the same:
>  
> DeprecationWarning: Request.is_xhr is deprecated. Given that the 
> X-Requested-With header is not a part of any spec, it is not reliable
>  
> In my opinion there are not many good reasons to have to change behaviour if 
> a request is made via XHR. I think the most common usage is to have a single 
> view that returns a JSON response or a HTML response depending on if XHR is 
> used (https://github.com/search?l=Python&q=request.is_ajax&type=Code 
> <https://github.com/search?l=Python&q=request.is_ajax&type=Code>), which 
> isn’t great and isn’t reliable.
>  
> 
> 
> On 16 Nov 2019, at 16:08, Adam Johnson mailto:m...@adamj.eu>> 
> wrote:
>  
> Django's HttpRequest.is_ajax method determines whether the request was made 
> with the JS API XMLHttpRequest 
> https://docs.djangoproject.com/en/2.2/ref/request-response/#django.http.HttpRequest.is_ajax
>  
> <https://docs.djangoproject.com/en/2.2/ref/request-response/#django.http.HttpRequest.is_ajax>
>  . It does so by checking the X-Requested-With header.
>  
> The new way of making "AJAX" requests from the browser is the JavaScript 
> fetch() API : https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API 
> <https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API> .
>  
> I think the  is_ajax() documentation is at least a little misleading in 
> pretending XMLHttpRequest is the only JS API. There also aren't any special 
> headers set by fetch() so it's not possible to detect its requests.
>  
> I propose deprecating is_ajax() with no replacement.
>  
> Thoughts?
> 
> -- 
> Adam
>  
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com 
> <mailto:django-developers+unsubscr...@googlegroups.com>.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/CAMyDDM0i-p0ZxBj-fSheGs-2pMXH7K7Oka%3DCjy1YXx-emBu3mw%40mail.gmail.com
>  
> <https://groups.google.com/d/msgid/django-developers/CAMyDDM0i-p0ZxBj-fSheGs-2pMXH7K7Oka%3DCjy1YXx-emBu3mw%40mail.gmail.com?utm_medium=email&utm_source=footer>.
>  
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com 
> <mailto:django-developers+unsubscr...@googlegroups.com>.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/84DCD242-69A8-4B8D-9EB6-243312B5F77F%40tomforb.es
>  
> <https://groups.google.com/d/msgid/django-developers/84DCD242-69A8-4B8D-9EB6-243312B5F77F%40tomforb.es?utm_medium=email&utm_source=footer>.
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, se

Re: Deprecate HttpRequest.is_ajax

[Tom Forbes]

I think this is a good starting point. What do we think about adding a 
“accepts_json” helper of some kind? It seems that the vast, vast majority of 
usages I can find of “is_ajax” is to return a JSON response, which I feel could 
be served nicely with a helper. There are also, annoyingly, two different JSON 
content types that are sent (the correct application/json and the incorrect 
text/json), which a helper could take care of. 


> On 17 Nov 2019, at 12:56, Claude Paroz  wrote:
> 
> I'm afraid that implementing a whole content negociation framework is a bit 
> ambitious, unless someone has much time to devote to that.
> 
> We could start smaller, similar to django-accept-header. I quickly sketched 
> what it could look like in:
> https://github.com/django/django/compare/master...claudep:accept_header?expand=1
> 
> Claude
> 
> Le dimanche 17 novembre 2019 12:32:50 UTC+1, Asif Saif Uddin a écrit :
> any plan with 
> https://github.com/django/deps/blob/master/draft/content-negotiation.rst 
>  
> one?
>  
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/7aff16cc-aeff-4b5e-b402-e4d587bc9315%40googlegroups.com
>  
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/22260C6C-702C-40F3-A847-38C0A69D3C43%40tomforb.es.

Re: Deprecate HttpRequest.is_ajax

[Tom Forbes]

I would agree. Flask has done the same:

DeprecationWarning: Request.is_xhr is deprecated. Given that the 
X-Requested-With header is not a part of any spec, it is not reliable

In my opinion there are not many good reasons to have to change behaviour if a 
request is made via XHR. I think the most common usage is to have a single view 
that returns a JSON response or a HTML response depending on if XHR is used 
(https://github.com/search?l=Python&q=request.is_ajax&type=Code 
), which isn’t 
great and isn’t reliable.


> On 16 Nov 2019, at 16:08, Adam Johnson  wrote:
> 
> Django's HttpRequest.is_ajax method determines whether the request was made 
> with the JS API XMLHttpRequest 
> https://docs.djangoproject.com/en/2.2/ref/request-response/#django.http.HttpRequest.is_ajax
>  
> 
>  . It does so by checking the X-Requested-With header.
> 
> The new way of making "AJAX" requests from the browser is the JavaScript 
> fetch() API : https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API 
>  .
> 
> I think the  is_ajax() documentation is at least a little misleading in 
> pretending XMLHttpRequest is the only JS API. There also aren't any special 
> headers set by fetch() so it's not possible to detect its requests.
> 
> I propose deprecating is_ajax() with no replacement.
> 
> Thoughts?
> 
> -- 
> Adam
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/CAMyDDM0i-p0ZxBj-fSheGs-2pMXH7K7Oka%3DCjy1YXx-emBu3mw%40mail.gmail.com
>  
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/84DCD242-69A8-4B8D-9EB6-243312B5F77F%40tomforb.es.

Re: GitHub Actions

[Tom Forbes]

No there isn’t. Right now Docker-box is optimized for local development and 
exists outside of the main repository, and for now I’d suggest not trying to 
integrate it. That can be some future work though!

Regarding services vs apt, docker services is the only realistic way to do it. 
We cannot rely on all versions of every database we run to be packaged, 
installable and configurable in Ubuntu in a timely + simple manner, and it will 
quickly become a mess (different configurations between different versions for 
example). 

Just supplying a matrix of database versions that pull and start stock database 
images via “services” is simple and effective. This is basically how docker-box 
works. Postgis is annoying, but we publish a docker image as part of docker-box 
that we can use.

Tom

>> On 9 Nov 2019, at 08:40, Johannes Hoppe  wrote:
> 
> > Then why not use all the work already done in django-docker-box there? Then 
> > this would be "Run Django CI with Docker in the Azure cloud via Github 
> > Actions instead of Jenkins via the Jenkins-Github integration (plugin?)”
> 
> @Matematica that sounds promising is there a PR for that, that I have missed? 
> I’d love to not reinvent the wheel :)
>> On 9. Nov 2019, 07:21 +0900, django-developers@googlegroups.com, wrote:
>> 
>> Then why not use all the work already done in django-docker-box there? Then 
>> this would be "Run Django CI with Docker in the Azure cloud via Github 
>> Actions instead of Jenkins via the Jenkins-Github integration (plugin?)"
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/fdcbfc00-977c-4587-a823-8e806cddf280%40Spark.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/17BD40A2-6A1B-4B44-8F49-11F2520F9898%40tomforb.es.

Re: GitHub Actions

[Tom Forbes]

> 2. You can build dependent builds, or build stages, but you will need to keep 
> everything in one file. (This gets messy without anchors & co).
> 3. You can give your jobs proper names and it works wonderful with a matrix 
> too.
> 4. You can add lots of OS dependent steps (or other dependencies) to avoid 
> duplication.
> 5. Fast failing is defined per job. So if a job inside a matrix fails, only 
> runners in the same matrix will be killed. Further more, jobs that depend on 
> this job will not be started. However, other will. (Very powerful stuff, love 
> it.)
> 
> I put all my work and results into a branch. Here's what I implemented:
> 1. flake8, isort
> 2. sqlite3 all OSxPythonv versions
> 3. PostgreSQL
> 4. PostGIS
> 5. Selenium (headless) FF & Chrome (flaky as ever :/ )
> 
> Here is the config file:
> https://github.com/codingjoe/django/blob/30962/flake8/.github/workflows/ci.yml
>  
> <https://github.com/codingjoe/django/blob/30962/flake8/.github/workflows/ci.yml>
> 
> The output is here:
> https://github.com/codingjoe/django/commit/7a6fe4d7956caa6fb9a1ffe190310f48df031190/checks?check_suite_id=300072254
>  
> <https://github.com/codingjoe/django/commit/7a6fe4d7956caa6fb9a1ffe190310f48df031190/checks?check_suite_id=300072254>
> 
> I use sphinx and libenchant on GH-actions in other projects, but I need to 
> check out the sphinx is build in our current CI setup first.
> I'll try to draft a solution for that too.
> 
> So far I am pretty happy. I haven't encountered anything that isn't possible 
> yet. Concurrency is also great.
> Regarding speed, I haven't looked much into caching yet, and I am not sure 
> it's a big time saver, since Django doesn't have many dependencies.
> 
> On Thursday, November 7, 2019 at 1:44:00 PM UTC+9, Johannes Hoppe wrote:
> @Florian, dependent builds or build stages are possible, seealso: 
> https://github.com/codingjoe/django/commit/eeefc83a85ba5e91b98c4e29fb9b20896612cc8c/checks?check_suite_id=299641652
>  
> <https://github.com/codingjoe/django/commit/eeefc83a85ba5e91b98c4e29fb9b20896612cc8c/checks?check_suite_id=299641652>
> 
> --
> Johannes Hoppe
> 
> www.johanneshoppe.com <http://www.johanneshoppe.com/>
> 
> Want to chat? Let's get a coffee!
> https://calendly.com/codingjoe/call <https://calendly.com/codingjoe/call>
> 
> Lennéstr. 19
> 14469 Potsdam
> 
> USt-IdNr.: DE284754038
> On 7. Nov 2019, 10:16 +0900, Tom Forbes  <mailto:t...@tomforb.es>>, wrote:
>> Maybe this discussion is slightly off topic, and at the risk of derailing 
>> things I’d like to put out my view on this.
>> 
>> There is more to it than just “using Github’s computing power”, just as 
>> there is more to using AWS than “using Amazon’s computing power”. That’s 
>> only a small part of it: it’s also the ecosystem, the integrations, the 
>> maintenance cost, the *ease of use*, the support, etc.
>> 
>> Django’s “core business” is Django. Time and resources that are spent not 
>> focusing on improving that should be reduced. Keeping Jenkins alive and 
>> stable seems like a waste to me. Something similar could be said about 
>> flake8 vs black.
>> 
>> Jenkins is actually a really interesting and unique piece of technology, far 
>> more interesting than most people realize.  But who cares because we don’t 
>> need interesting. We need to run “pip install flake8 && flake8” on every 
>> commit and show the results in Github. That’s a solved problem thanks to 
>> Travis, CircleCI, actions or even Gitlab-CI with 0 effort from us.
>> 
>> So IMO it shouldn’t matter at all if Jenkins is open or closed source. Time 
>> is, as always, our fiat and the expenditure of it is what counts. To that 
>> end I would honestly avoid self hosting runners, there are some easier 
>> alternatives we can try first to support Oracle.
>> 
>> Anyway, I will make the flake8 merge request tomorrow and we can see where 
>> we go from there.
>> 
>>> On 7 Nov 2019, at 01:10, Matemática A3K >> <mailto:matematica@gmail.com>> wrote:
>>> 
>>> 
>>> 
>>> 
>>> On Wed, Nov 6, 2019 at 3:51 AM Carlton Gibson >> <mailto:carlton.gib...@gmail.com>> wrote:
>>> Hey Shai. 
>>> 
>>> On Wednesday, 6 November 2019 08:43:21 UTC+1, Shai Berger wrote:
>>> 
>>> Is there benefit enough in GitHub Actions (over Jenkins) to justify a
>>> move from an open-source based solution?
>>> 
>>> I don't think we have to move away entirely but it would be good to bring 
>>> in s

Re: GitHub Actions

[Tom Forbes]

Maybe this discussion is slightly off topic, and at the risk of derailing 
things I’d like to put out my view on this.

There is more to it than just “using Github’s computing power”, just as there 
is more to using AWS than “using Amazon’s computing power”. That’s only a small 
part of it: it’s also the ecosystem, the integrations, the maintenance cost, 
the *ease of use*, the support, etc.

Django’s “core business” is Django. Time and resources that are spent not 
focusing on improving that should be reduced. Keeping Jenkins alive and stable 
seems like a waste to me. Something similar could be said about flake8 vs black.

Jenkins is actually a really interesting and unique piece of technology, far 
more interesting than most people realize.  But who cares because we don’t need 
interesting. We need to run “pip install flake8 && flake8” on every commit and 
show the results in Github. That’s a solved problem thanks to Travis, CircleCI, 
actions or even Gitlab-CI with 0 effort from us.

So IMO it shouldn’t matter at all if Jenkins is open or closed source. Time is, 
as always, our fiat and the expenditure of it is what counts. To that end I 
would honestly avoid self hosting runners, there are some easier alternatives 
we can try first to support Oracle.

Anyway, I will make the flake8 merge request tomorrow and we can see where we 
go from there.

> On 7 Nov 2019, at 01:10, Matemática A3K  wrote:
> 
> 
> 
> 
>> On Wed, Nov 6, 2019 at 3:51 AM Carlton Gibson  
>> wrote:
>> Hey Shai. 
>> 
>>> On Wednesday, 6 November 2019 08:43:21 UTC+1, Shai Berger wrote:
>>> 
>>> Is there benefit enough in GitHub Actions (over Jenkins) to justify a 
>>> move from an open-source based solution? 
>> 
>> I don't think we have to move away entirely but it would be good to bring in 
>> something else... (or at least try it...) 
>> 
>> These are the top on my mind reasons for wanting to: 
>> 
>> * Mariusz spends Quite a lot of time™ maintaining Jenkins and all the job 
>> definitions etc. This doesn't go away with another builder but if we can 
>> move to declarative config file in the repo, then that could become shared 
>> work. (Jenkins has this these days no...? But we don't...) 
>> * I'd really like to try GitHub actions Windows builds. Maybe we could get 
>> Jenkins to behave better but currently we have a failure on every force 
>> push. 
>> * Maybe we can stage runs: i.e. do the lint, and some basic builds first. Do 
>> one Python against each DB before running then all. And so on, to save some 
>> trees. (Again maybe we might be able to do this with Jenkins but it seems 
>> more likely to actually happen if we give GitHub Actions a trial.)
>> * I think we're running up against capacity for the sponsored space, so 
>> builds slow down. If we can spread the load we should get faster CI. 
>> 
>> Kind Regards,
>> 
>> Carlton
> 
> I also share Shai's concerns. Thinking a bit about it, it is about depending 
> on other's resources for the workflow. Nothing in Django changes, as all the 
> tests are the same and can be ran with tox - and all the source code is the 
> same - the difference would be using Github's computing power instead of 
> Jenkin's for leveraging resources (i.e. manpower) which are not infinite.
> 
> It would be ideal - to me - to have the entire workflow depending on FOSS 
> solutions. I guess this was one of the reasons that Jenkins was chosen 
> instead, say, Travis, but if the health of Django improves with this, the 
> overall impact to the the community will be better than staying being users 
> of a project which we don't contribute (at least to my knowledge): almost 
> nothing changes to Jenkins - well, it starts loosing users over Github, I 
> think the same may happen with Gitlab - and Django gets an improvement 
> because is already dependent on Github.
> 
> It would be also an alarm to Jenkins in the way that it needs to catch up 
> with others, if Github Actions end up providing the same functionality with a 
> lower setup and maintainability effort, the might migrate there.
> 
> When you use others' servers like in SaaS, there is no way but trusting the 
> other to "behave as expected", even using the AGPL. As long as it a conscious 
> decision, seems good to me :)
> 
>> 
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-developers+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-developers/e509afbc-21ea-4b3b-9e23-18718594fa49%40googlegroups.com.
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com.
> To vi

Re: GitHub Actions

[Tom Forbes]

Here it is: 
https://github.com/orf/django-github-actions/tree/master/.github/workflows 
<https://github.com/orf/django-github-actions/tree/master/.github/workflows>

There are two actions I’ve added here: a lint step and a matrix of sqlite tests 
(Windows, MacOS and Ubuntu * py36 and py37). It’s all vanilla Django on the 
latest master, however I did remove the pylibmc dependency for now as that’s 
slightly tricky to include (and there are no 3.8 wheels available).

I’ve got four example PR’s for you:

1. Linting failures: https://github.com/orf/django-github-actions/pull/1 
<https://github.com/orf/django-github-actions/pull/1>
2. A failing test: https://github.com/orf/django-github-actions/pull/2 
<https://github.com/orf/django-github-actions/pull/2>
3. Nothing failing: https://github.com/orf/django-github-actions/pull/3 
<https://github.com/orf/django-github-actions/pull/3>
4. Modifying the CI to test on Python 3.8: 
https://github.com/orf/django-github-actions/pull/4 
<https://github.com/orf/django-github-actions/pull/4>

I’ve had some experience with actions but all this took me about half an hour 
(on very poor airport WiFi). It’s really simple and really effective. The devil 
is obviously in the details but based on my work with docker-box we can make 
good progress with our test matrix really fast. And we can do it step-by-step.

The usage limits are documented here: 
https://help.github.com/en/github/automating-your-workflow-with-github-actions/about-github-actions#usage-limits
 
<https://help.github.com/en/github/automating-your-workflow-with-github-actions/about-github-actions#usage-limits>.
 I’m not sure how they apply to organisations (maybe just “free”?), but I think 
Github would allow us an increased limit.

What do we think?

Tom


> On 6 Nov 2019, at 13:58, Tom Forbes  wrote:
> 
> Yes there will be. Now Github has added caching I think we are good to go.
> 
> I will send a link here with the longer running on my fork and we can look at 
> starting there. Once (if?) that’s merged then we can enable the “allow forks 
> to run actions” option and we can iteratively add more tests as merge 
> requests.
> 
> Regarding Jenkins: it’s a beast that’s often unreliable. We save on effort 
> there, get a more reliable CI tool and have our CI files versioned alongside 
> our code.
> 
>  I’m not sure open vs closed source comes into play here, and it should be 
> mentioned that Github actions (the steps and the runner) is almost completely 
> open source. And believe me when I say the source is a lot easier to 
> follow/debug than the reams of often incomprehensible code a Jenkins plugin 
> typically has!
> 
> Tom
> 
>> On 6 Nov 2019, at 12:56, Preeti Sharma  wrote:
>> 
>> 
>> Right, but there has to be some changes done i think.
>> 
>> On Wed 6 Nov, 2019, 6:05 PM Florian Apolloner, > <mailto:f.apollo...@gmail.com>> wrote:
>> 
>> 
>> On Wednesday, November 6, 2019 at 8:43:21 AM UTC+1, Shai Berger wrote:
>> Is there benefit enough in GitHub Actions (over Jenkins) to justify a 
>> move from an open-source based solution? 
>> 
>> Yes, less server costs (even if sponsored). Less things to maintain for us 
>> (Jenkins is a beast). Better and more reliable integration with github. 
>> 
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-developers+unsubscr...@googlegroups.com 
>> <mailto:django-developers+unsubscr...@googlegroups.com>.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-developers/d34795bc-7490-43b0-bdcb-8a3023939c9b%40googlegroups.com
>>  
>> <https://groups.google.com/d/msgid/django-developers/d34795bc-7490-43b0-bdcb-8a3023939c9b%40googlegroups.com?utm_medium=email&utm_source=footer>.
>> 
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-developers+unsubscr...@googlegroups.com 
>> <mailto:django-developers+unsubscr...@googlegroups.com>.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-developers/CAMzw02X%2BqLKO6hKKh6bT0JzVBUubeO4AaW4o3YQAXnCpxcJvYw%40mail.gmail.com
>>  
>> <https://groups.google.com/d/msgid/django-developers/CAMzw02X%2BqLKO6hKKh6bT0JzVBUubeO4AaW4o3YQAXnCpxcJvYw%40mail.gmail.com?utm_medium=email&utm_source=footer>.

-- 
You received this m

Re: GitHub Actions

[Tom Forbes]

Yes there will be. Now Github has added caching I think we are good to go.

I will send a link here with the longer running on my fork and we can look at 
starting there. Once (if?) that’s merged then we can enable the “allow forks to 
run actions” option and we can iteratively add more tests as merge requests.

Regarding Jenkins: it’s a beast that’s often unreliable. We save on effort 
there, get a more reliable CI tool and have our CI files versioned alongside 
our code.

 I’m not sure open vs closed source comes into play here, and it should be 
mentioned that Github actions (the steps and the runner) is almost completely 
open source. And believe me when I say the source is a lot easier to 
follow/debug than the reams of often incomprehensible code a Jenkins plugin 
typically has!

Tom

> On 6 Nov 2019, at 12:56, Preeti Sharma  wrote:
> 
> 
> Right, but there has to be some changes done i think.
> 
>> On Wed 6 Nov, 2019, 6:05 PM Florian Apolloner,  wrote:
>> 
>> 
>>> On Wednesday, November 6, 2019 at 8:43:21 AM UTC+1, Shai Berger wrote:
>>> Is there benefit enough in GitHub Actions (over Jenkins) to justify a 
>>> move from an open-source based solution? 
>> 
>> Yes, less server costs (even if sponsored). Less things to maintain for us 
>> (Jenkins is a beast). Better and more reliable integration with github. 
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-developers+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-developers/d34795bc-7490-43b0-bdcb-8a3023939c9b%40googlegroups.com.
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/CAMzw02X%2BqLKO6hKKh6bT0JzVBUubeO4AaW4o3YQAXnCpxcJvYw%40mail.gmail.com.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/1C0ECC83-D6F9-4F7E-8B00-A6A00C783DC1%40tomforb.es.

Re: GitHub Actions

[Tom Forbes]


Yep! My life has been unexpectedly busy the last few weeks so I’ve made no 
progress on it, but my idea is to prototype it in that repo before making a MR 
to Django itself. 

Starting with limiting would be easiest, but I actually don’t think it would be 
particularly hard to have the full suite running on actions.

There are a few open questions:

1. How do we support Oracle?
2. Can we (or do we want to?) unify it somehow with docker-box?
3. Will the standard github actions concurrency handle the number of builds we 
put through it in a reasonable time?

I’ve been experimenting with Github actions in a few repositories and I’m very 
impressed with their flexibility. It’s built to be much more than just a CI 
tool so if we wanted to automate anything about the repository we can do it 
with Actions: 

- workflow tags on merge requests, synchronized with Trac?
- messages to first time contributors, thanking them and giving them some 
advice on common issues new time contributors run into? (I really think we 
should do this)
- a comment with a link to the Django ticket if one is not already present?
- close MRs if the trac ticket is closed?

Side note: I’m also increasingly concerned about the lack of delivery on some 
Google groups messages. I had not been sent the first message from Johannes 
(it’s not in spam or any other place), and would have missed it if Felix has 
not replied. 

Tom

>> On 31 Oct 2019, at 09:49, Florian Apolloner  wrote:
> 
> I think Tom Forbes started working on it already, you might wanna sync up 
> with him: https://github.com/orf/django-github-actions
> 
>> On Thursday, October 31, 2019 at 10:40:47 AM UTC+1, Johannes Hoppe wrote:
>> I would love to give Selenium a try. O god I hate it when selenium tests 
>> just randomly fail and I can't restart them.
>> I think the ability to just restart actions as a user is going to be a major 
>> improvement to the current workflow.
>> 
>>> On Thursday, October 31, 2019 at 6:36:24 PM UTC+9, Florian Apolloner wrote:
>>> Hi,
>>> 
>>> yes I had the same thought yesterday. I think trying with linters first 
>>> should be an easy low hanging fruit (and to get a feeling for it). Tests in 
>>> general might get a bit harder.
>>> 
>>> Cheers,
>>> Florian
>>> 
>>>> On Thursday, October 31, 2019 at 10:30:10 AM UTC+1, Johannes Hoppe wrote:
>>>> Hey there,
>>>> 
>>>> I saw that GitHub actions are already enabled for the Django organization.
>>>> Since general availability was announced starting Nov. 13. by GitHub, I 
>>>> wonder, should we use it?
>>>> 
>>>> I know we have a very complicated build setup, but maybe it makes sense to 
>>>> move some checks over to GitHub actions.
>>>> All the linters and docs build should be easy. Things like Oracle, might 
>>>> be a bit more difficult, but not impossible I guess.
>>>> 
>>>> Anyhow, I would be interested on your thoughts.
>>>> 
>>>> Cheers
>>>> -Joe
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/07a4594d-d2db-4ae2-a121-1d1bda3e649e%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/FE9D758A-F4EB-47A3-9964-FCDA36663387%40tomforb.es.

Re: Removing old branches from the Django Git repository.

[Tom Forbes]

I second this, there are a few other branches in that list (successful or not) 
that have historic value to Django. Is there a pressing need to delete them, 
other than spring cleaning?

I guess maybe it’s sentimental value, and nobody would ever check them out, but 
still...

Tom

> On 10 Oct 2019, at 20:15, Adam Johnson  wrote:
> 
> 
> Can we leave magic-removal as a tag since it’s such a pivotal point in 
> djangos history? xD
> 
>> On Thu, 10 Oct 2019 at 19:09, Mariusz Felisiak  
>> wrote:
>> Hi y'all,
>> 
>> We're going to remove some old branches from the Django Git repository 
>> on 1st November 2019:
>> 9 old branches related with Google SOC projects:
>> soc2009/admin-ui
>> soc2009/http-wsgi-improvements
>> soc2009/i18n-improvements
>> soc2009/model-validation
>> soc2009/multidb
>> soc2009/test-improvements
>> soc2010/app-loading
>> soc2010/query-refactor
>> soc2010/test-refactor
>> 17 attic branches:
>> attic/boulder-oracle-sprint
>> attic/full-history
>> attic/generic-auth
>> attic/gis
>> attic/i18n
>> attic/magic-removal
>> attic/multi-auth
>> attic/multiple-db-support
>> attic/new-admin
>> attic/newforms-admin
>> attic/per-object-permissions
>> attic/queryset-refactor
>> attic/schema-evolution
>> attic/schema-evolution-ng
>> attic/search-api
>> attic/sqlalchemy
>> attic/unicode
>> Best,
>> Mariusz
>> 
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-developers+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-developers/9837cb41-66bc-40f2-8296-75f0ad173ee3%40googlegroups.com.
> -- 
> Adam
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/CAMyDDM2kqTztbjzBg%2BOtTCO1M-5TxpaguH60BNv3m5TUe2T-dw%40mail.gmail.com.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/07142C38-7A56-4F8E-979A-A2574DB40026%40tomforb.es.

Re: Explore integrating django-docker-box in some way?

[Tom Forbes]

I did experiment with it. The main problem is that the image we use requires a 
bunch of system libraries (see 
https://github.com/django/django-docker-box/blob/master/Dockerfile#L4 
<https://github.com/django/django-docker-box/blob/master/Dockerfile#L4>). I 
think a few where missing our outdated in Alpine, but if that's changed then 
I'm not against trying it again.

Tom

> On 8 Oct 2019, at 11:58, Nick Sarbicki  wrote:
> 
> Just out of interest, Tom, have you tried using Alpine as the base? Or is 
> there a reason to avoid it?
> 
> I usually find once you introduce all the dependencies for django it doesn't 
> make a huge difference but it might shave some of the weight off if we're 
> worried about image size.
> 
> 
> - Nick
> 
> 
> On Tue, Oct 8, 2019 at 11:52 AM Josh Smeaton  <mailto:josh.smea...@gmail.com>> wrote:
> I was going to archive the repo, but it seems I don't have the necessary 
> permissions. Carlton, do you?
> 
> On Tuesday, 8 October 2019 21:45:22 UTC+11, Adam Johnson wrote:
> +1 to archiving django-box
> 
> On Tue, 8 Oct 2019 at 11:01, Tom Forbes > wrote:
> Thank you for the kind words Bruno! I'm glad it's helped you, if you have any 
> suggestions for improvements then please open an issue on the repo or post a 
> message here, I know it's not perfect. I would have replied earlier but this 
> message didn't get delivered to me.
> 
> I'm biased, but I'd be +1 on archiving the old django-box. It's served us 
> well, but unless someone is willing to spend some time updating it then it's 
> going to confuse new users.
> 
> On Friday, 4 October 2019 12:40:02 UTC+1, Bruno Alla wrote:
> Just a note that as a newish contributor to Django, this django-docker-box is 
> fantastic, it makes things much easier to setup.
> 
> Thank you to everyone involved!
> 
> On Wednesday, 5 December 2018 00:02:27 UTC, Tom Forbes wrote:
> To have this completely working at sprints without having everyone building 
> their own local images we would need to have the Jenkins server use docker in 
> some capacity. This would also require an official django account on Docker 
> hub.
> 
> The pattern I’m using right now is that on every build we pull the 
> django-ci:latest image (from my personal account). Docker uses this image as 
> a cache automatically (preventing rebuilds). On any successful master build 
> we push the new image to docker-hub, so subsequent builds can utilise it.
> 
> Then anyone wanting to speed up their bootstrapping can do docker-compose 
> pull and automatically have the latest image available for running right 
> away. We can make this smaller, for sure, but we can also suggest people 
> download this beforehand (i.e at their hotel).
> 
> I don’t know how feasible this is but it’s also very easy to run a caching 
> docker mirror (docker run -p 5000:5000 registry). Organizers could run this 
> at large events and configuring docker to use a local mirror on the network 
> is a one-line change for atendees.
> 
> 
> 
> 
> 
> On 4 December 2018 at 23:52:42, Josh Smeaton (josh@gmail.com <>) wrote:
> 
>> Size of the image could definitely be a concern, especially at sprints where 
>> wifi speeds aren't always optimal. The django-box image is significantly 
>> larger though so it'd still be a net win. There are also optimisations that 
>> can be made to the image for reducing size over time, so I'd fully expect it 
>> to come down. I've spent a little bit of time trying to optimise a $work$ 
>> python docker file, I'll provide what I've got as an issue to possibly look 
>> at.
>> 
>> I see that the ticket has been accepted and I think that's a great step 
>> forward. I'd also like to hear from the infrastructure team what their 
>> thoughts on using docker over customised build environments would be.
>> 
>> Florian, Tim, Markus .. any thoughts? (Apologies, I've missed some, this 
>> list of names is from memory).
>> 
>> On Wednesday, 5 December 2018 10:39:16 UTC+11, Tom Forbes wrote:
>> Thank you for the reply Josh. I didn’t anticipate any suggestions for 
>> including this inside core but off the back of your suggestion I’ve made a 
>> ticket here: https://code.djangoproject.com/ticket/30010 
>> <https://code.djangoproject.com/ticket/30010>.
>> 
>> I don’t think it should be complex at all to include this inside Django - 
>> it’s four or five new files at most. Hopefully this should improve the 
>> experience at sprints, however the current Dockerfile weighs in at 650+mb so 
>> the problem may switch from ‘it is hard

Re: Explore integrating django-docker-box in some way?

[Tom Forbes]

Thank you for the kind words Bruno! I'm glad it's helped you, if you have 
any suggestions for improvements then please open an issue on the repo or 
post a message here, I know it's not perfect. I would have replied earlier 
but this message didn't get delivered to me.

I'm biased, but I'd be +1 on archiving the old django-box. It's served us 
well, but unless someone is willing to spend some time updating it then 
it's going to confuse new users.

On Friday, 4 October 2019 12:40:02 UTC+1, Bruno Alla wrote:
>
> Just a note that as a newish contributor to Django, this django-docker-box 
> is fantastic, it makes things much easier to setup. 
>
> Thank you to everyone involved!
>
> On Wednesday, 5 December 2018 00:02:27 UTC, Tom Forbes wrote:
>>
>> To have this completely working at sprints without having everyone 
>> building their own local images we would need to have the Jenkins server 
>> use docker in some capacity. This would also require an official django 
>> account on Docker hub.
>>
>> The pattern I’m using right now is that on every build we pull the 
>> django-ci:latest image (from my personal account). Docker uses this 
>> image as a cache automatically (preventing rebuilds). On any successful 
>> master build we push the new image to docker-hub, so subsequent builds can 
>> utilise it.
>>
>> Then anyone wanting to speed up their bootstrapping can do docker-compose 
>> pull and automatically have the latest image available for running right 
>> away. We can make this smaller, for sure, but we can also suggest people 
>> download this beforehand (i.e at their hotel).
>>
>> I don’t know how feasible this is but it’s also very easy to run a 
>> caching docker mirror (docker run -p 5000:5000 registry). Organizers 
>> could run this at large events and configuring docker to use a local mirror 
>> on the network is a one-line change for atendees.
>>
>>
>>
>>
>> On 4 December 2018 at 23:52:42, Josh Smeaton (josh@gmail.com) wrote:
>>
>> Size of the image could definitely be a concern, especially at sprints 
>> where wifi speeds aren't always optimal. The django-box image is 
>> significantly larger though so it'd still be a net win. There are also 
>> optimisations that can be made to the image for reducing size over time, so 
>> I'd fully expect it to come down. I've spent a little bit of time trying to 
>> optimise a $work$ python docker file, I'll provide what I've got as an 
>> issue to possibly look at. 
>>
>> I see that the ticket has been accepted and I think that's a great step 
>> forward. I'd also like to hear from the infrastructure team what their 
>> thoughts on using docker over customised build environments would be.
>>
>> Florian, Tim, Markus .. any thoughts? (Apologies, I've missed some, this 
>> list of names is from memory).
>>
>> On Wednesday, 5 December 2018 10:39:16 UTC+11, Tom Forbes wrote: 
>>>
>>> Thank you for the reply Josh. I didn’t anticipate any suggestions for 
>>> including this inside core but off the back of your suggestion I’ve made a 
>>> ticket here: https://code.djangoproject.com/ticket/30010.
>>>
>>> I don’t think it should be complex at all to include this inside Django 
>>> - it’s four or five new files at most. Hopefully this should improve the 
>>> experience at sprints, however the current Dockerfile weighs in at 650+mb 
>>> so the problem may switch from ‘it is hard to set up an environment’ to ‘it 
>>> is hard to download one’!
>>>
>>>
>>>
>>> On 5 November 2018 at 23:02:30, Josh Smeaton (josh.s...@gmail.com) 
>>> wrote:
>>>
>>> I'm sorry I haven't had the time to review or contribute yet, but I 
>>> think it'll be a very useful project - especially for new contributors that 
>>> might have a little docker experience. The current vagrant solution is 
>>> heavy, does not work properly on windows and some linuxes, and isn't that 
>>> easy to maintain or deploy. I'd be in favour of adding the docker files 
>>> directly to django/django to minimise setup burden (DJANGO_PATH), and 
>>> improving the contributing docs to show users how to test using docker. 
>>>
>>> One of the hardest things I found at sprints was getting development 
>>> environments setup to effectively contribute - even using the docker-box 
>>> project which I understand quite well. Anything we can do to improve that 
>>> situation will be v

Re: Feature idea: bulk_associate: Add ManyToMany relationships in bulk

[Tom Forbes]

Hey David,
I like this idea, while I don’t think the use case is common there have been a 
few times where I’ve needed this and got around it by creating/modifying the 
through model in bulk. Having a method that does this would be good IMO.

Unless anyone has strong opinions against this then can you make a ticket? 

Tom

>> On 30 Sep 2019, at 02:14, David Foster  wrote:
> 
> Here is another API variation I might suggest:
> 
> M1.m2_set.add_pairs(*[(m1, m2), ...], assert_no_collisions=False)
> # --- OR ---
> M1.m2_set.add_pair_ids(*[(m1_id, m2_id), ...], assert_no_collisions=False)
> 
> This has the advantages of being more similar to the existing add() API and 
> not requiring a special function import.
> 
> For bulk_disassociate() the analogous API would be:
> 
> M1.m2_set.remove_pairs(*[(m1, m2), ...])
> # --- OR ---
> M1.m2_set.remove_pair_ids(*[(m1_id, m2_id), ...])
> 
> - David
> 
>> On Thursday, September 26, 2019 at 10:45:45 AM UTC-7, David Foster wrote:
>> Given the following example model:
>> 
>> class M1(models.Model):
>> m2_set = models.ManyToManyField('M2')
>> 
>> It is already possible to associate one M1 with many M2s with a single DB 
>> query:
>> 
>> m1.m2_set.add(*m2s)
>> 
>> However it's more difficult to associate many M1s with many M2s, 
>> particularly if you want to skip associations that already exist.
>> 
>> # NOTE: Does NOT skip associations that already exist!
>> m1_and_m2_id_tuples = [(m1_id, m2_id), ...]
>> M1_M2 = M1.m2_set.through
>> M1_M2.objects.bulk_create([
>> M1_M2(m1_id=m1_id, m2_id=m2_id)
>> for (m1_id, m2_id) in
>> m1_and_m2_id_tuples
>> ])
>> 
>> What if we could do something like the following instead:
>> 
>> bulk_associate(M1.m2_set, [(m1, m2), ...])
>> # --- OR ---
>> bulk_associate_ids(M1.m2_set, [(m1_id, m2_id), ...])
>> 
>> I propose to write and add a bulk_associate() method to Django. I also 
>> propose to add a paired bulk_disassociate() method.
>> 
>> 1. Does this sound like a good idea in general?
>> 
>> 
>> In more detail, I propose adding the specific APIs, importable from 
>> django.db:
>> 
>> M1 = TypeVar('M1', bound=Model)  # M1 extends Model
>> M2 = TypeVar('M2', bound=Model)  # M2 extends Model
>> 
>> def bulk_associate(
>> M1_m2_set: ManyToManyDescriptor,
>> m1_m2_tuples: 'List[Tuple[M1, M2]]',
>> *, assert_no_collisions: bool=True) -> None:
>> """
>> Creates many (M1, M2) associations with O(1) database queries.
>> 
>> If any requested associations already exist, then they will be left 
>> alone.
>> 
>> If you assert that none of the requested associations already exist,
>> you can pass assert_no_collisions=True to save 1 database query.
>> """
>> pass
>> 
>> def bulk_associate_ids(
>> M1_m2_set: ManyToManyDescriptor,
>> m1_m2_id_tuples: 'List[Tuple[Any, Any]]',
>> *, assert_no_collisions: bool=True) -> None:
>> pass
>> 
>> If assert_no_collisions is False then (1 filter) query and (1 bulk_create) 
>> query will be performed.
>> If assert_no_collisions is True then only (1 bulk_create) will be performed.
>> 
>> def bulk_disassociate(
>> M1_m2_set: ManyToManyDescriptor,
>> m1_m2_tuples: 'List[Tuple[M1, M2]]') -> None:
>> """
>> Deletes many (M1, M2) associations with O(1) database queries.
>> """
>> pass
>> 
>> def bulk_disassociate_ids(
>> M1_m2_set: ManyToManyDescriptor,
>> m1_m2_id_tuples: 'List[Tuple[Any, Any]]') -> None:
>> pass
>> 
>> The database connection corresponding to the M1_M2 through-table will be 
>> used.
>> 
>> 2. Any comments on the specific API or capabilities?
>> 
>> 
>> If this sounds good I'd be happy to open an item on Trac and submit a PR.
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/49b45d05-836f-4512-91b8-8e4dbb55f6a4%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/A1F23375-5004-40D1-9B77-775E337CB53F%40tomforb.es.

Re: Redis cache support in core

[Tom Forbes]

That's not necessarily a blocker, and Memcached is not simple to run on Windows 
either - there are no official prebuilt binaries, as far as I'm aware.

> On 5 Sep 2019, at 14:29, Matthew Pava  wrote:
> 
> I’d just like to point out that Redis support on Windows is limited at best. 
> All other technologies that Django uses, as far as I can recall, do support 
> Windows.
> 
> From: django-developers@googlegroups.com 
> [mailto:django-developers@googlegroups.com] On Behalf Of Jacob Rief
> Sent: Thursday, September 5, 2019 1:33 AM
> To: django-developers@googlegroups.com
> Subject: Re: Redis cache support in core
> 
> I'm also in favor on having it as part of the core, since memcache is also 
> supported.
> 
> One of the nice features Redis provides, is the possibility to invalidate one 
> or more cached object by using a wildcard key.
> It namely is the method delete_pattern() added by django-redis-cache to the 
> given Django caching backend. That
> (or a similar method) then should be part of the other Django caching 
> backends as well, such as the dummy cache or in-memory cache.
> 
> Jacob
> --
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/CAJ_HXxo9jpr-%3DFmLXcYwwR6tdn-NVDfrEoo-qYnB-vQwKv9O%3DA%40mail.gmail.com
>  
> .
> 
> --
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/b9357775f9684aa888091bce83f2852b%40iss2.ISS.LOCAL
>  
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/10675445-9101-4F5A-8151-A105B939E482%40tomforb.es.


signature.asc
Description: Message signed with OpenPGP

Re: ipdb throws AttributeError in autoreload.py with 2.2.2

[Tom Forbes]

IPython seems to override `__main__`, which I certainly did not expect: 
https://github.com/ipython/ipython/blob/257e4c45563598810b0a771de286004617510fc1/IPython/core/debugger.py#L238
 

 and perhaps 
https://github.com/ipython/ipython/blob/7b42de99c651de35f487adea3f57824ad97bcd74/IPython/testing/globalipapp.py#L115
 
.

I’ve made a ticket here: https://code.djangoproject.com/ticket/30588#ticket 
, and a PR to fix it: 
https://github.com/django/django/pull/11507 


> , and I know the autoreload has been an area of some frustration lately and 
> as such I'm not sure of the best approach for handling this case.

I do feel responsible for the frustration, sorry! My advice to anyone that 
hears a little voice in their head asking “how hard can building an autoloader 
be?”: ignore it. That being said, I think we’ve ironed out the worst of the 
issues, despite this one being pretty bad :/

> On 23 Jun 2019, at 16:46, Jason Johns  wrote:
> 
> Hello,
> 
> About two weeks ago, work updated the base docker image to use django 2.2.2 
> and I noticed an issue right off the bat with using ipdb 0.12 and ipython 
> 7.5.0 (which are also defaults for the docker image).  I really didn't have 
> the time to dig more, and got along well enough with pdb in the meantime.  
> This weekend, I had some free time to poke around, and these are my findings:
> 
> 
> In a DRF serializer helper method, I import ipdb and set a breakpoint.  The 
> API call executes and stops here.  After anywhere from 1 to 3 seconds pause 
> at the breakpoint, I then see the following output from ipdb> and the 
> runserver process exits.
> > /orm/service/api/serializers/helpers/facet_generator.py(62)__make_facet()
>  61 import ipdb; ipdb.set_trace()
> ---> 62 log.info(f'Facet field: {facet_field}')
>  63 log.info(f'Facet values: {facet_values}')
> 
> ipdb> Traceback (most recent call last):
>   File "/orm/manage.py", line 15, in 
> execute_from_command_line(sys.argv)
>   File 
> "/usr/local/lib/python3.7/site-packages/django/core/management/__init__.py", 
> line 381, in execute_from_command_line
> utility.execute()
>   File 
> "/usr/local/lib/python3.7/site-packages/django/core/management/__init__.py", 
> line 375, in execute
> self.fetch_command(subcommand).run_from_argv(self.argv)
>   File 
> "/usr/local/lib/python3.7/site-packages/django/core/management/base.py", line 
> 323, in run_from_argv
> self.execute(*args, **cmd_options)
>   File 
> "/usr/local/lib/python3.7/site-packages/django/core/management/commands/runserver.py",
>  line 60, in execute
> super().execute(*args, **options)
>   File 
> "/usr/local/lib/python3.7/site-packages/django/core/management/base.py", line 
> 364, in execute
> output = self.handle(*args, **options)
>   File 
> "/usr/local/lib/python3.7/site-packages/django/core/management/commands/runserver.py",
>  line 95, in handle
> self.run(**options)
>   File 
> "/usr/local/lib/python3.7/site-packages/django/core/management/commands/runserver.py",
>  line 102, in run
> autoreload.run_with_reloader(self.inner_run, **options)
>   File "/usr/local/lib/python3.7/site-packages/django/utils/autoreload.py", 
> line 585, in run_with_reloader
> start_django(reloader, main_func, *args, **kwargs)
>   File "/usr/local/lib/python3.7/site-packages/django/utils/autoreload.py", 
> line 570, in start_django
> reloader.run(django_main_thread)
>   File "/usr/local/lib/python3.7/site-packages/django/utils/autoreload.py", 
> line 288, in run
> self.run_loop()
>   File "/usr/local/lib/python3.7/site-packages/django/utils/autoreload.py", 
> line 294, in run_loop
> next(ticker)
>   File "/usr/local/lib/python3.7/site-packages/django/utils/autoreload.py", 
> line 334, in tick
> for filepath, mtime in self.snapshot_files():
>   File "/usr/local/lib/python3.7/site-packages/django/utils/autoreload.py", 
> line 350, in snapshot_files
> for file in self.watched_files():
>   File "/usr/local/lib/python3.7/site-packages/django/utils/autoreload.py", 
> line 249, in watched_files
> yield from iter_all_python_module_files()
>   File "/usr/local/lib/python3.7/site-packages/django/utils/autoreload.py", 
> line 103, in iter_all_python_module_files
> return iter_modules_and_files(modules, frozenset(_error_files))
>   File "/usr/local/lib/python3.7/site-packages/django/utils/autoreload.py", 
> line 120, in iter_modules_and_files
> sys_file_paths.append(module.__file__)
> AttributeError: module '__main__' has no attribute '__file__'
> 
> If you suspect this is an IPython bug, please report it at:
> https://github.com/ipython/ipython/issues
> or sen

Re: Django 2.2 and the watchman reloader

[Tom Forbes]

Just an update to this: I've made a WIP PR to add watchdog support here:
https://github.com/django/django/pull/11480

I'd like to disable it on Windows and MacOS, and there are some tricky
issues to resolve around how events are triggered (watchdog uses a number
of threads internally, and we need to ensure only the main thread triggers
the file changed notification), but otherwise it seems to be working OK
locally.

On Mon, Feb 25, 2019 at 10:22 PM Tom Forbes  wrote:

> I have a small PR here to remove the “watchman unavailable” message,
> whilst keeping the one that specifies which reloader we are using:
> https://github.com/django/django/pull/11025.
>
>
>
>
> On 21 February 2019 at 20:52:29, Claude Paroz (cla...@2xlibre.net) wrote:
>
> Le jeudi 21 février 2019 21:43:43 UTC+1, Tom Forbes a écrit :
>>
>> Hey Claude,
>> Thanks for your feedback on the feature, I fully agree with you. I think
>> we should remove that warning message about the missing package. I will
>> make a PR to do that.
>>
>
> I'm not completely sure it's a good idea to entirely remove the message.
> Maybe just telling the used reloader would be fine.
>
>
>> Regarding creating another reloader: it should not be that difficult to
>> do at all since we have all the other pieces in place. In theory it's just
>> implementing a class with single generator method.
>>
>> If people agree I would like to use the 'watchdog' package for this
>> rather than the pyinotify library as it would be quicker to implement, a
>> lot nicer to work with and is easier to test.
>>
>
> ++1, watchdog is better maintained, looks like pyinotify is dead. Tell me
> if you need help, even if you seems a lot more knowledgeable than me on the
> subject.
>
> Thanks.
>
> Claude
>
>
>> On Thu, 21 Feb 2019, 19:56 Claude Paroz,  wrote:
>>
>>> Le jeudi 21 février 2019 19:48:31 UTC+1, Dan Davis a écrit :
>>>>
>>>> Claude,
>>>>
>>>> I've tested a Django based application on 2.2b1 without watchman on
>>>> Windows, it does tell you about watchman, but it doesn't fail to run.
>>>> Apparently, it falls back to the old way of reloading.   Is this not the
>>>> behavior on Debian/Ubuntu?
>>>>
>>>
>>> Yes it is. I would say this is still a slight regression in two ways:
>>>
>>> - no messages told you the reload method was not optimal before. So now
>>> people will try to "fix" their system, more than before.
>>> - for Debian-based systems, you could improve the reloading performance
>>> by installing system or pip packages in 30 seconds. Now you have to spend
>>> 30 minutes to search how watchman can be installed and to compile the
>>> package (+ you have to care yourself for any security issue).
>>>
>>> Claude
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Django developers (Contributions to Django itself)" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to django-develop...@googlegroups.com.
>>> To post to this group, send email to django-d...@googlegroups.com.
>>> Visit this group at https://groups.google.com/group/django-developers.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/django-developers/b059553a-e25e-4d90-beed-bf7e0f797305%40googlegroups.com
>>> <https://groups.google.com/d/msgid/django-developers/b059553a-e25e-4d90-beed-bf7e0f797305%40googlegroups.com?utm_medium=email&utm_source=footer>
>>> .
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/6de70f64-9bbb-4017-bd10-c85e94ae08d0%40googlegroups.com
> <https://groups.google.com/d/msgid/django-developers/6de70f64-9bbb-4017-bd10-c85e94ae08d0%40googlegroups.com?utm_medium=email&utm_source=footer>
> .
> For more options, visit https://groups.google.com/d/optout.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFNZOJMLobYmyXQh2vV93%3DP8OCSmKi29-BM9Z1FPiw-YM7FXkw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: ERROR: The included URLConf 'website.urls' does not appear to have patterns in it. If u see valid patterns in the file then there is issue of Circular import.

[Tom Forbes]

As Adam states before, in the first reply to your original message, this 
mailing list is for the internal development of Django and not for getting help 
with your Django projects.

There are numerous other places to ask, check his email for a few, and a lot of 
people (myself included) willing to help you there. 

Tom

> On 28 May 2019, at 17:44, Madhur Kabra  wrote:
> 
> Thanks for your answer, but this isn't working. Could you help me with 
> another solution
> 
>> On Monday, May 27, 2019 at 10:06:37 PM UTC+5:30, rajan khadka wrote:
>> in this
>> from django.urls import URLPattern, url
>> from . import views
>> 
>> urlpatterns = [
>> url('', views.index, name=index),
>> ]
>> can u replace with this
>> from django.urls import path
>> from . import views
>> 
>> urlpatterns = [
>> path('', views.index, name=index),
>> ]
>> 
>> 
>>> On Mon, May 27, 2019 at 8:32 PM Madhur Kabra  wrote:
>>> I am getting the url conf error, I  have attached the relevant files.
>>> Thanks for the assistance
>>> -- 
>>> You received this message because you are subscribed to the Google Groups 
>>> "Django developers (Contributions to Django itself)" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an 
>>> email to django-d...@googlegroups.com.
>>> To post to this group, send email to django-d...@googlegroups.com.
>>> Visit this group at https://groups.google.com/group/django-developers.
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/django-developers/452a42ef-70be-4f8f-b963-223909507a48%40googlegroups.com.
>>> For more options, visit https://groups.google.com/d/optout.
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/933ad758-2c1d-452b-bfd9-f8549ddfb847%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/DEF03CF8-1B44-4F11-B972-04F12527C3FA%40tomforb.es.
For more options, visit https://groups.google.com/d/optout.

Re: DEP 0008 (Formatting using Black) is accepted

[Tom Forbes]

There has been a particularly healthy debate on the mailing list, I think it 
was clear that a consensus was not going to be reached. Are there any points 
you wanted to bring up there that where not already made on the mailing list 
thread?

Tom

> On 11 May 2019, at 17:30, Michael Martinez  
> wrote:
> 
> I'm disappointed to learn that this was merged without a healthy debate on 
> the deps repo. 
> 
>> On Friday, May 10, 2019 at 8:53:44 PM UTC-5, Andrew Godwin wrote:
>> One quick clarification - when I said "stable (1.0)" release, I in fact 
>> meant the first release that the Black project officially marks as stable.
>> 
>> Black doesn't use versioning that would result in a stable release being 
>> called 1.0, as far as I know, given they are on 19.3b0 right now!
>> 
>> Andrew
>> 
>>> On Friday, May 10, 2019 at 6:46:37 PM UTC-7, Andrew Godwin wrote:
>>> Hi everyone,
>>> 
>>> The Technical Board has voted on DEP 0008 
>>> (https://github.com/django/deps/blob/master/accepted/0008-black.rst), after 
>>> the extensive discussion here and the updates done as a result.
>>> 
>>> The board voted in favour, and so I've moved the DEP into the accepted 
>>> directory.
>>> 
>>> One of the key changes to the proposal was to wait until Black got to a 
>>> stable (1.0) release, so it will sit in "accepted" until that happens, at 
>>> which point we will do the implementation (including the big reformat 
>>> itself) and then move the DEP to "final".
>>> 
>>> There's no hard timeline on when Black will hit a stable release, but it's 
>>> very close; there are a few changes we'd like to make sure are included 
>>> when we reformat Django, such as keeping lists with trailing commas on 
>>> multiple lines (https://github.com/python/black/pull/826). That's why we're 
>>> waiting for the stable release rather than reformatting now.
>>> 
>>> Thanks to everyone who chimed in on the discussion - this was not an easy 
>>> decision, and the feedback helped a lot to work out what was best for us.
>>> 
>>> I also want to personally thank Herman Schistad, for prompting this 
>>> discussion, and Aymeric Augustin, for his work on writing and updating the 
>>> DEP!
>>> 
>>> Yours in auto-formatting,
>>> Andrew
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/8f6f5de0-f489-4271-9c2e-af3b0a97f365%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/B3F49C9A-FAE8-41BA-A9EE-06222826B54F%40tomforb.es.
For more options, visit https://groups.google.com/d/optout.

Re: injecting settings

[Tom Forbes]

I think what you are describing is a hard problem(tm) with no clear and generic 
way to solve it. Django doesn’t have the concept of a dependency tree in apps, 
and while this is annoying in some cases there isn’t much we can do to solve it.

If I understand your proposition, it would inevitably involve adding some kind 
of inter-app dependency information to Django? And in your given example about 
overwriting rather than merging dictionaries: there is no safe way to do this. 
What if merging the two dicts produces an invalid setting?

While I definitely feel settings can be improved (environments!), I’m not sure 
if what you are proposing has much benefit outside of your specific use case. 
However perhaps you could share some more code to make your idea a bit clearer.

FYI, setuptools entrypoints provides a really nice generic plugin discovery 
system that you might be able to leverage if you are not already. Pytest uses 
it to discover plugins without configuration, you can check their documentation 
for examples.

Tom

> On 7 May 2019, at 22:55, Christian González  
> wrote:
> 
> 
> Sorry about writing, and not testing myself before:
> 
>> 2. Set a default value at import time:
>> 
>> # apps.py
>> 
>> from django.apps import AppConfig
>> from django.conf import settings
>> 
>> class MyAppConfig(AppConfig):
>> name = 'my_app'
>> verbose_name = "..."
>> 
>> def ready(self):
>> if not hasattr(settings, 'MY_SETTING'):
>> settings.MY_SETTING = 'my_default'
> This works.
> 
> I have an app that has the following code in Config.ready():
> 
> settings.WEBPACK_LOADER.update(
> {
> "foobar": {
> "STATS_FILE": os.path.join(
> settings.BASE_DIR,
> "{}/frontend/webpack-stats.json".format(
> os.path.abspath(os.path.dirname(__file__))
> ),
> )
> }
> }
> )
> And the app webpack_loader is located *after* this app in INSTALLED apps. 
> Within webpack-loader, the settings dict WEBPACK_LOADER inclusive "foobar" is 
> available. So this is kind of "solved" for me - but - it's a bit of a hack.
> 
> I'd really appreciate a "standard" approach from Dajngo...
> 
> Thanks for your patience, ;-)
> 
> Christian
> 
> -- 
> Dr. Christian González
> https://nerdocs.at
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/3727cd68-3ed6-1366-2277-0815ae90da5c%40nerdocs.at.
> For more options, visit https://groups.google.com/d/optout.
> 

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/6407D4E6-FD52-4F1F-AAC8-86C15A9CBBCC%40tomforb.es.
For more options, visit https://groups.google.com/d/optout.

Re: About query strings in urls when using rest api

[Tom Forbes]

This mailing list is for the development of Django itself, not for support 
using Django. Please use the django-users mailing list for that, or IRC #django 
on freenode, or a site like Stack Overflow.

Also, please never add private credentials in query strings like that. Use a 
POST request.

> On 3 May 2019, at 15:05, Surajeet Das  wrote:
> 
> I am actually developing an android app which will fetch data using an api .  
> I am trying to pass multiple parameters in the url using query strings, but I 
> am not able concatenate those parameters in the url. 
> eg: U/userapi/?email=abc&pass=bcd
> I trying to achieve the above example.
> How do I take email and password parameter together in the url ?
> Help me out on that.
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/2025cb43-6023-48b4-9141-664307933084%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/8B88AF3B-74B5-4016-AEFD-904460A8EB16%40tomforb.es.
For more options, visit https://groups.google.com/d/optout.

Re: A different approach for the auto-reloader

[Tom Forbes]

Hey Ramiro,
This sounds like a pretty awesome idea in general. It would also solve one long 
standing issue where the http socket is closed and re-opened during reloading, 
leading to pageviews after code changes failing.

I’m trying to think of some edge cases where we need to be careful, and there 
are three I can think of:
1. It’s really hard to know where a request is going at this level, so we could 
end up with a page view triggering 20 static files requests very quickly, which 
would lead to stating all files 20x redundantly.

2. How would we handle (potentially ridiculous) things like code imported from 
a thread during app ready? In general wouldn’t we need some kind of thread to 
poll for new imports to watch after Django has been started?

3. Are there any workflows that would be disrupted by this? I can’t think of 
any that I use, but it’s possible that people have come to rely on the existing 
“does this throw an exception” in some cases, and triggering a request only to 
get this could be annoying?

Overall though this feels like a good idea. Stat based reloaders are a lot 
simpler to deal with than platform specific fs monitoring. Ask thanks for the 
kind words about my refactor, I’m glad the abstractions work on something like 
this.

Tom

> On 24 Apr 2019, at 05:33, Ramiro Morales  wrote:
> 
> Hi all,
> 
> I had a stab at a somewhat simpler development server automatic reloading 
> strategy 
> https://github.com/django/django/compare/master...ramiro:synch-reloader
> 
> Intention is to test how an implementation of a design by Gary Bernhardt 
> would look. The best written description I could find is this:
> 
> https://github.com/devlocker/tychus/issues/3
> 
> Gary also had posted some tweets (this is how I got interested in the topic) 
> which seems to have been deleted since then.
> 
> Main idea is: Actual checking of changes on the filesystem for modules under 
> monitoring isn't performed in a loop or by depending on a OS kernel feature 
> but per-HTTP request by a front-end proxy process which is in charge of 
> restarting the 'upstream' web server process (in our case a dumbed-down 
> runserver dev server) only when it detects there have been changes. 
> 
> Been meaning to try this for some time. It would  have been much harder 
> before Tom Forbes' work on refactoring and cleaning up the reloading code for 
> Django 2.2. IMHO Tom's code is so very well thought that for example I just 
> had to lightly subclass StatReload to implement this totally different 
> strategy.
> 
> Current form of the code is a new experimental 'serverrun' (for lack of a 
> better name) added to the Django code base whose command line UI mimics 100% 
> the runserver one. 
> 
> It copies code from a few places of our code base: The runserver command, the 
> WSGI app hosting code, etc.
> 
> I decided to implement this as a new built-in command for now a) to ease 
> experimentation and b) because it needs some minor changes to the 'runserver' 
> command to handle cosmetic details (logging). If the idea is accepted (read 
> further below for reasons in favor of this) then maybe we can switch 
> runserver to this code. Or if the idea isn't deemed appropate for Django core 
> them I might implement it as an standalone  django app/project.
> 
> If the idea of a smarter stat()-based FS status monitor like this gets 
> actually tested and validated in the field (i.e. by users with big source 
> code trees) it could allow us to possibly stop needing to depend on all of:
> 
> * watchman
> * pyinotify
> * watchdog
> (and removing our support code for them from the Django code base).
> 
> Also, this would mean:
> 
> * Setup simplification for final users (no third party Python libraries or 
> system daemon to install)
> * Better cross-platform portability for Django (we go back to piggy-backing 
> stat() from the stdlib as our only way yo trigger code reloading).
> 
> Additionally, as the reloading is performed fully (by restarting the whole 
> HTTP server) and is triggered from another process (the transparent http 
> proxy one) we can drop some contortions we currently need to make:
> 
> - Having to wait for the app registry stabilization
> - Avoiding race conditions with the url resolver
> 
> I suspect there could be power efficiency advantages too as:
> 
> * The scanning for changes is triggered by HTTP requests which should be less 
> frequent than periodically every N seconds.
> * If the developer modifies more than one file before switching to the 
> browser there is need of only one FS scan to cater for all these changes, 
> which is performed just in time for the first HTTP request so the code 
> executed to render/serve it is 100% accurate in 

Re: Proposal to format Django using black

[Tom Forbes]

> so there is no way to know who is nodding along with which arguments.

That somewhat impedes the point of posting this to get consensus, and if you 
ascribe to that view then all mailing list discussions about project wide 
changes are somewhat useless, as there could be a silent majority completely 
against the idea being proposed?

There is a majority of people who have expressed opinions in this thread in 
favor of black, so he’s not necessarily trying to influence the discussion as 
much as pointing out a fact.

That’s not to say consensus has been reached though, judging by the number of 
replies here there are a lot of strong views either way.

Tom

> On 25 Apr 2019, at 09:59, Tobias Kunze  wrote:
> 
>> On 19-04-25 08:24:34, Herman S wrote:
>> From what I gather there is a clear majority favoring Black, […]
> 
> Please don't resort to influencing the discussion by way of presenting a
> majority opinion like that. People on django-dev are generally good at
> not repeating points that have been made already, or cluttering an
> ongoing discussion with a lot of "+1", so there is no way to know who is
> nodding along with which arguments.
> 
> Tobias
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers  (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/20190425085906.mol7ok72yw2u7f6p%40cordelia.localdomain.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/DFCA73FE-5D66-4C1D-B91F-690DD76B71F1%40tomforb.es.
For more options, visit https://groups.google.com/d/optout.

Re: Proposal to format Django using black

[Tom Forbes]

I would be +1 for Black. I think it makes a lot of sense for a project like 
Django, and it does seem that a non trivial amount of both contributor and 
reviewer time is spent on formatting fixes.

The choice of double quotes by default used to annoy me, but after using black 
for a while I think that the benefits outweigh the downsides.

One thing we have not considered here is that after running black on Django a 
huge portion of our outstanding merge requests will have conflicts, some of 
which might be tricky to rebase. I’m not sure there is much we can do about 
that though.

Tom

> On 17 Apr 2019, at 09:26, Tobias Kunze  wrote:
> 
> Hi Dan,
> 
>> On 19-04-16 20:33:29, Dan Davis wrote:
>> +1 isort
>> -1 black
>> 
>> I think that codestyle checkers are better, because you teach yourself
>> proper style for python.
> 
> I appreciate this argument, but: As Django community our primary concern
> in this discussion has to be the impact black would have on the Django
> code base and the Django development process – educating our
> contributors cannot be a primary concern, compared to making
> contributions as easy as possible.
> 
> Tobias
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers  (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/20190417082648.miicsigxjwoevmkl%40cordelia.localdomain.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/AFFEB466-A328-4DA6-8845-F2DFB55C1ED1%40tomforb.es.
For more options, visit https://groups.google.com/d/optout.

Re: Why does ModelForm do validation and not Model

[Tom Forbes]

The idea is that you generally always have to do extensive validation when
accepting user input through a form. These validations could require
additional database queries or other somewhat expensive lookups (especially
with validate unique).

However if you are loading data from a trusted source, e.g:

for row in your_csv_file:
instance = Model(**row)
instance.save()

Then there is no need to call that potentially slow full_clean(). There is
not much value in slowing down all .save()’s needlessly - the developer
should know when it’s appropriate to run validations and can run
full_clean() when needed.




On 16 April 2019 at 21:42:24, Will Gordon (wpg4...@gmail.com) wrote:

So the validation is cheaper when performed by ModelForm, as opposed to the
Model?
--
You received this message because you are subscribed to the Google Groups
"Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-developers/8d929b8e-b0e9-4a88-b796-26f00266f729%40googlegroups.com

.
For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFNZOJMUPSptux9yeSVtcHLi2HR80YfzgXmhMqnWQ_eEMYqqdA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Add to QuerySet method `one` that acts as `first` but without ordering

[Tom Forbes]

I don’t think we should add a method like this for a few reasons. Firstly 
without an order by in SQL the order of rows is undefined, in practice myql 
orders rows but Postgres returns a different order per transaction. This will 
be confusing to users who don’t understand this and come to implicitly rely on 
first() being stable.

Secondly if you are filtering on an indexed column the overhead will be next to 
none. Is this not the case?

And lastly, changing this would be major backwards incompatible change.

Tom

> On 13 Apr 2019, at 21:58, alTus  wrote:
> 
> I've posted the source code if `first`. You can see there that if qs is not 
> ordered then ordering by pk is added.
> It's the main point of this issue btw.
> 
> суббота, 13 апреля 2019 г., 22:53:52 UTC+3 пользователь Florian Apolloner 
> написал:
>> 
>> qs.order_by().first() should do what you want and is not worth adding .one()
>> 
>> Cheers,
>> Florian
>> 
>>> On Saturday, April 13, 2019 at 5:48:29 PM UTC+2, alTus wrote:
>>> Hello.
>>> 
>>> Please consider if that proposal can be useful not only for me :)
>>> 
>>> `QuerySet.first` is quite useful shortcut but its drawback is that it 
>>> involves ordering so some DB queries become expensive.
>>> But quite often (and very often while debugging) there's a need just to get 
>>> one object that satisfy some filters.
>>> 
>>> Current `first` implementation to compare with:
>>> 
>>> def first(self):
>>> """Return the first object of a query or None if no match is 
>>> found."""
>>> for obj in (self if self.ordered else self.order_by('pk'))[:1]:
>>> return obj
>>> 
>>> Proposal (as an example of implementation):
>>> 
>>> def one(self):
>>> """Return one random object of a query or None if no match is 
>>> found."""
>>> for obj in self.order_by()[:1]:
>>> return obj
>>> 
>>> That would be short, simple and wouldn't require any imports in client code.
>>> 
>>> Thank you.
>>> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/fc6e71e5-a9b5-4cb5-9283-de1130ff0b2e%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/B83882F9-E932-4258-93D7-100A3D4A933D%40tomforb.es.
For more options, visit https://groups.google.com/d/optout.

Re: Request to reconsider ticket #27910: using an Enum class in model Field choices

[Tom Forbes]

I really like this implementation, it seems really clean and simple.

I would suggest that it might be worth seeing it’s simple to make the
display text optional, I think in the general case capitalising the key
name would be acceptable. Users can always override it if required or if
they need translation support.

Even if we decide to make the display names required in all cases, it’s a
great improvement over the current way of defining choices IMO.

Tom




On 13 April 2019 at 10:33:39, Shai Berger (s...@platonix.com) wrote:

Back to this issue for the DjangoCon EU sprints...

James' objection about the inclusion testing needed for validation
seems moot, because validation can be done by conversion to the enum
type (as noted by Tom).

Raphael's warning about the woes of translation are already handled by
the suggested implementation -- because it isn't the enum value that
we make translatable, but rather an attribute on it.

I should point that the suggested implementation uses IntEnum and
StrEnum. The Python documentation recommends against using these,
except in the case that one needs to account for compatibility with an
existing protocol -- which, I submit to you, is exactly our case (the
"protocol" being the types available for database fields).

As a reminder, the relevant links are:

https://code.djangoproject.com/ticket/27910
https://github.com/django/django/compare/master...shaib:ticket_27910?expand=1

Thanks,
Shai.

-- 
You received this message because you are subscribed to the Google Groups
"Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-developers/20190413113323.1342ed7d.shai%40platonix.com.

For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFNZOJMMh6nH8VdAozeVnLbQXanqB5b864Ro1g%2BGm2N6hWM%3DgQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Official Django Docker Container Deprecated

[Tom Forbes]

I think the point we are trying to make is that it’s fundamentally not a
good thing to try and distribute a one-size fits all docker image for a
specific framework.

For reference here is one you can use yourself:

FROM python:3
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY . .
CMD [ "gunicorn", "my.app" ]

If someone is unable to make an equivalent Dockerfile then they will be
really confused when they realise that they need to customise it, because
few projects are as simple as that.

You should also likely not embed Apache inside your app container - it’s
kind of missing the general idea of Docker.

To re-iterate: The Django project had no hand in creating the ‘official’
image. The Docker project retired the original Django image for reasons
that are clearly explained here , and
those reasons still hold today.

Tom




On 28 February 2019 at 12:56:33, Alexander Lyabah (a.lya...@checkio.org)
wrote:

I can make a version for production use (in a week or two), for your
critics.

For example, based on Appache wsgi.

PS: maybe it is also worth to make a docker image for testing changes in
Django source?

On Wednesday, February 27, 2019 at 4:31:17 PM UTC+2, Jamesie Pic wrote:
>
> > most people currently lean towards a microservice architecture and
> therefore towards flask.
> "according to the 2018 JetBrains Developer Survey" and some people.
> Why start a project with flask in 2019 instead of Quart which or
> Starlette is another question that I suppose is out of the scope of
> this mailing list.
>
> Anyway, the point of Docker is to build your own image that supports
> both development and production given different runtime parameters.
> The agile practice with docker is to build your immutable image in CI,
> test it, deploy it to staging, have on-click deployment to production.
>
> The security and best practice documentation from docker are indeed a
> lot to grasp, and beginners will most of the time start making
> insecure (running as root) and inefficient (fat) images. Therefore for
> their security Django might want to document making a docker file,
> perhaps based on the alpine image that's the most lightweight.
>
> --
> ∞
>
--
You received this message because you are subscribed to the Google Groups
"Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-developers/f07ad32e-e74f-4cd3-945a-ed92692c2209%40googlegroups.com

.
For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFNZOJP5k%3Dsn8VkdwseuVqnMC6X40y349urNMRFZfD7FEypvbg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Official Django Docker Container Deprecated

[Tom Forbes]

There never was an official Django image, it was an "official docker" 
Django image that they maintained. The page image page explains why it was 
deprecated: https://hub.docker.com/_/django

For most usages of this image, it was already not bringing in django from 
> this image, but actually from your project's requirements.txt, so the 
> only "value" being added here was the pre-installing of mysql-client, 
> postgresql-client, and sqlite3 for various uses of the django framework.
>

I fully agree with this, it made little sense as an image at all.

On Tuesday, 26 February 2019 07:39:34 UTC, Alexander Lyabah wrote:
>
> Hi guys,
>
> I found out that official django container is deprecated. Why you don't 
> want to support it?
>
> When you search "django" in Python subreddit 
> https://monosnap.com/file/R3uAqdrcrtxjCyKgHhhe7B8lO3up5q
>
> you see "Flask has overtaken Django according to the 2018 JetBrains 
> Developer Survey" 
> https://www.reddit.com/r/Python/comments/ao5dml/flask_has_overtaken_django_according_to_the_2018/
>
> and the main point in this discussion that it is hard to start from 0 to 
> real project.
>
> That why I was surprised that you don't have a simple docker container 
> with configured django with apache or/and django with nginx . It can save 
> so much time for people who just want to launch a simple pet project with 3 
> visitors per week.
>
> Thank you, I love Django and wish you be better and bigger, have a great 
> week.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/db603927-bed9-4734-8e52-a0500c9d69e0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Django 2.2 and the watchman reloader

[Tom Forbes]

I have a small PR here to remove the “watchman unavailable” message, whilst
keeping the one that specifies which reloader we are using:
https://github.com/django/django/pull/11025.




On 21 February 2019 at 20:52:29, Claude Paroz (cla...@2xlibre.net) wrote:

Le jeudi 21 février 2019 21:43:43 UTC+1, Tom Forbes a écrit :
>
> Hey Claude,
> Thanks for your feedback on the feature, I fully agree with you. I think
> we should remove that warning message about the missing package. I will
> make a PR to do that.
>

I'm not completely sure it's a good idea to entirely remove the message.
Maybe just telling the used reloader would be fine.


> Regarding creating another reloader: it should not be that difficult to do
> at all since we have all the other pieces in place. In theory it's just
> implementing a class with single generator method.
>
> If people agree I would like to use the 'watchdog' package for this rather
> than the pyinotify library as it would be quicker to implement, a lot nicer
> to work with and is easier to test.
>

++1, watchdog is better maintained, looks like pyinotify is dead. Tell me
if you need help, even if you seems a lot more knowledgeable than me on the
subject.

Thanks.

Claude


> On Thu, 21 Feb 2019, 19:56 Claude Paroz,  wrote:
>
>> Le jeudi 21 février 2019 19:48:31 UTC+1, Dan Davis a écrit :
>>>
>>> Claude,
>>>
>>> I've tested a Django based application on 2.2b1 without watchman on
>>> Windows, it does tell you about watchman, but it doesn't fail to run.
>>> Apparently, it falls back to the old way of reloading.   Is this not the
>>> behavior on Debian/Ubuntu?
>>>
>>
>> Yes it is. I would say this is still a slight regression in two ways:
>>
>> - no messages told you the reload method was not optimal before. So now
>> people will try to "fix" their system, more than before.
>> - for Debian-based systems, you could improve the reloading performance
>> by installing system or pip packages in 30 seconds. Now you have to spend
>> 30 minutes to search how watchman can be installed and to compile the
>> package (+ you have to care yourself for any security issue).
>>
>> Claude
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to django-develop...@googlegroups.com.
>> To post to this group, send email to django-d...@googlegroups.com.
>> Visit this group at https://groups.google.com/group/django-developers.
>> To view this discussion on the web visit https://groups.google.com/d/
>> msgid/django-developers/b059553a-e25e-4d90-beed-
>> bf7e0f797305%40googlegroups.com
>> <https://groups.google.com/d/msgid/django-developers/b059553a-e25e-4d90-beed-bf7e0f797305%40googlegroups.com?utm_medium=email&utm_source=footer>
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
> --
You received this message because you are subscribed to the Google Groups
"Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-developers/6de70f64-9bbb-4017-bd10-c85e94ae08d0%40googlegroups.com
<https://groups.google.com/d/msgid/django-developers/6de70f64-9bbb-4017-bd10-c85e94ae08d0%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFNZOJOOELOoSEGGx9LrMryrYFhqm%2Bzd_PPZ-ztarjiZyR%2Bvnw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Use CDN for djangoproject.com

[Tom Forbes]

Awesome work! For my location (Lisbon, Portugal) it takes about 130ms to
retrieve the HTML for a docs page (
https://django-docs.global.ssl.fastly.net/en/2.1/intro/reusable-apps/ to be
specific). The same page on docs.djangoproject.com responds in 800–900ms.




On 24 February 2019 at 14:35:55, Tobias McNulty (tob...@caktusgroup.com)
wrote:

Hi Tom,

Thanks for your message. I think we'll end up with Fastly since it would be
free, but I'm waiting to see their sponsorship contract. CloudFront would
work too but I don't know of any such open source sponsorship options with
AWS.

I will say wildcard purging looks a bit simpler in CloudFront, but your
idea purging the whole cache only for non-dev builds could work (provided
we have a lower cache timeout or a single wildcard purge condition set up
for the dev builds, I guess).

Feel free to test and post any feedback about Fastly prior to the potential
transition here: https://django-docs.global.ssl.fastly.net/en/2.1/ (this is
set up on a free dev account, so no custom SSL)

For the sake of comparison I'm working on getting a distribution set up for
CloudFront too, but it won't be so simple to test (without a DNS or server
configuration change) since I don't think CloudFront supports passing a
custom Host header to the origin like Fastly does (i.e., you'll probably
need to edit /etc/hosts).

Cheers,
Tobias


On Sat, Feb 23, 2019, 7:15 PM Tom Forbes  wrote:

> Sorry, I did not completely grok your message. I would be in favour of
> just invalidating the whole cache if needed, it seems the simplest
> solution. Invalidating most of the cache on every non-dev deploy would also
> be OK I think.
>
> On Sun, 24 Feb 2019, 00:10 Tom Forbes,  wrote:
>
>> Which CDN are we going to use? Fastly has awesome sub 100ms global
>> invalidation which we can trigger on every deploy, and cloudflare has
>> something similar.
>>
>> On Sun, 24 Feb 2019, 00:00 Tobias McNulty, 
>> wrote:
>>
>>> Hi all,
>>>
>>> An implementation question has come up regarding cache lifetime (see this
>>> PR <https://github.com/django/djangoproject.com/pull/870>). Right now,
>>> the whole site (including docs) has the site-wide Django cache enabled
>>> <https://github.com/django/djangoproject.com/blob/master/djangoproject/settings/prod.py#L43-L45>,
>>> with a timeout of 5 minutes
>>> <https://github.com/django/djangoproject.com/blob/master/djangoproject/settings/common.py#L27>.
>>> A couple docs views (search_suggestions
>>> <https://github.com/django/djangoproject.com/blob/master/docs/views.py#L248-L250>
>>> and search_description
>>> <https://github.com/django/djangoproject.com/blob/master/docs/views.py#L272-L274>)
>>> views have longer timeouts set (to 1 hour and 1 week, respectively).
>>>
>>> Once released, the vast majority of Django docs won't change much,
>>> except for the release notes section and any (likely minor) related updates
>>> to the docs themselves. To get the most benefit out of a CDN, it would
>>> obviously be desirable to set the timeout to something greater than 5
>>> minutes.
>>>
>>> At the same time, there are moments when a quick update to the docs *is*
>>> desired, and waiting an hour or more for any cached pages to expire may
>>> cause significant confusion, for example, in conjunction with a security
>>> release for which stubbed (non-final) release notes may have already been
>>> pushed out and cached.
>>>
>>> I see two main options at this point (which could even be combined):
>>>
>>> 1) Invalidate the whole cache (or at least some key release notes URLs)
>>> any time there's a docs build that has changes. It would be pretty easy to
>>> piggyback off of the existing business logic for avoiding a rebuild
>>> <https://github.com/django/djangoproject.com/blob/master/docs/management/commands/update_docs.py#L86-L93>
>>> if the git checkout hasn't changed (in the update_docs management command).
>>> 2) Pick subsections of the docs (e.g., for anything matching
>>> '///releases/*' and perhaps the development docs) that would
>>> keep a shorter cache timeout of 5-10 minutes. All URLs not specifically
>>> requiring this special treatment would get a longer timeout, perhaps
>>> somewhere between 1 and 24 hours.
>>>
>>> So, some questions for the list:
>>>
>>> * Are there sections of the docs besides '///releases/'
>>> and '//dev/' that might update frequently and merit some combination
>>> of invalidation and/or a short

Re: Use CDN for djangoproject.com

[Tom Forbes]

Sorry, I did not completely grok your message. I would be in favour of just
invalidating the whole cache if needed, it seems the simplest solution.
Invalidating most of the cache on every non-dev deploy would also be OK I
think.

On Sun, 24 Feb 2019, 00:10 Tom Forbes,  wrote:

> Which CDN are we going to use? Fastly has awesome sub 100ms global
> invalidation which we can trigger on every deploy, and cloudflare has
> something similar.
>
> On Sun, 24 Feb 2019, 00:00 Tobias McNulty,  wrote:
>
>> Hi all,
>>
>> An implementation question has come up regarding cache lifetime (see this
>> PR <https://github.com/django/djangoproject.com/pull/870>). Right now,
>> the whole site (including docs) has the site-wide Django cache enabled
>> <https://github.com/django/djangoproject.com/blob/master/djangoproject/settings/prod.py#L43-L45>,
>> with a timeout of 5 minutes
>> <https://github.com/django/djangoproject.com/blob/master/djangoproject/settings/common.py#L27>.
>> A couple docs views (search_suggestions
>> <https://github.com/django/djangoproject.com/blob/master/docs/views.py#L248-L250>
>> and search_description
>> <https://github.com/django/djangoproject.com/blob/master/docs/views.py#L272-L274>)
>> views have longer timeouts set (to 1 hour and 1 week, respectively).
>>
>> Once released, the vast majority of Django docs won't change much, except
>> for the release notes section and any (likely minor) related updates to the
>> docs themselves. To get the most benefit out of a CDN, it would obviously
>> be desirable to set the timeout to something greater than 5 minutes.
>>
>> At the same time, there are moments when a quick update to the docs *is*
>> desired, and waiting an hour or more for any cached pages to expire may
>> cause significant confusion, for example, in conjunction with a security
>> release for which stubbed (non-final) release notes may have already been
>> pushed out and cached.
>>
>> I see two main options at this point (which could even be combined):
>>
>> 1) Invalidate the whole cache (or at least some key release notes URLs)
>> any time there's a docs build that has changes. It would be pretty easy to
>> piggyback off of the existing business logic for avoiding a rebuild
>> <https://github.com/django/djangoproject.com/blob/master/docs/management/commands/update_docs.py#L86-L93>
>> if the git checkout hasn't changed (in the update_docs management command).
>> 2) Pick subsections of the docs (e.g., for anything matching
>> '///releases/*' and perhaps the development docs) that would
>> keep a shorter cache timeout of 5-10 minutes. All URLs not specifically
>> requiring this special treatment would get a longer timeout, perhaps
>> somewhere between 1 and 24 hours.
>>
>> So, some questions for the list:
>>
>> * Are there sections of the docs besides '///releases/'
>> and '//dev/' that might update frequently and merit some combination
>> of invalidation and/or a shorter cache time? And what's a good cache
>> timeout for such pages?
>> * How long are we comfortable waiting for *other* (not frequently
>> updated) pages to timeout, in the event they do change?
>>
>> Tobias
>>
>> On Fri, Feb 15, 2019 at 7:13 AM Tobias McNulty 
>> wrote:
>>
>>> Thanks for sharing the results.
>>>
>>> I did manage to get a domain set up with working SSL, in case you want
>>> to use it: https://django-docs.global.ssl.fastly.net/en/2.1/
>>>
>>> Tobias
>>>
>>> On Thu, Feb 14, 2019, 11:49 PM Cheng C >>
>>>> Thanks for the test site, Tobias.
>>>>
>>>> Tested from Melbourne, Australia:
>>>>
>>>> https://docs.djangoproject.com/en/2.1/
>>>> Average Ping: 268ms
>>>>  Browser: 22 requests, 238KB transferred, Finish: 2.72s,
>>>> DOMContentLoaded: 1.37s, Load: 1.68s
>>>>
>>>> https://docs.djangoproject.com.global.prod.fastly.net/en/2.1/
>>>> Average Ping: 28ms
>>>>  Browser: 22 requests, 240KB transferred, Finish: 947ms,
>>>> DOMContentLoaded: 627ms, Load: 910ms
>>>>
>>>> Tested on Chrome with "Disable cache" checked, and no render issue was
>>>> found.
>>>>
>>>> On Friday, February 15, 2019 at 2:09:09 PM UTC+11, Tobias McNulty wrote:
>>>>>
>>>>> Adam, is there another provider you would recommend instead, that does
>>>>> not require changing DNS providers? FWIW, python.org 

Re: Use CDN for djangoproject.com

[Tom Forbes]

e this a try and let me know if you
>>>> see any issues:
>>>> https://docs.djangoproject.com.global.prod.fastly.net/en/2.1/ (Not for
>>>> permanent use, obviously; you'll get a cert warning, and some pages will
>>>> redirect you back to https://docs.djangoproject.com.)
>>>>
>>>> To keep this thread from getting too noisy, you can find me (tobias1)
>>>> in #django-dev on FreeNode.
>>>>
>>>> Cheers,
>>>> Tobias
>>>>
>>>> On Thu, Feb 14, 2019 at 8:26 AM Adam Johnson  wrote:
>>>>
>>>>> I have not had great experience with Fastly in the past and would
>>>>> avoid them. They run an old fork of Varnish which is not fun to configure.
>>>>>
>>>>> On Thu, 14 Feb 2019 at 11:16, Josh Smeaton 
>>>>> wrote:
>>>>>
>>>>>> Cloudflare have many SSL options, including fully encrypted and
>>>>>> authenticated comms all the way through (terminate and reconnect).
>>>>>> Typically done by having a “hidden” origin domain that also hosts a
>>>>>> certificate. I’m unsure if it’s possible to have both origin and front
>>>>>> hosting the same name so that DNS alone can decide to hit cdn or origin.
>>>>>>
>>>>>> Anyway, it seems weird to me to dismiss a CDN offhand “because
>>>>>> security”. Especially considering the size of the providers and the
>>>>>> expertise their teams have.
>>>>>>
>>>>>> Cloudflare (fastly, cloudfront, whatever) aren’t some “random TLS”
>>>>>> providers. I would probably go as far to say that putting a CDN in front 
>>>>>> of
>>>>>> both the docs and the release packages would likely be a net improvement 
>>>>>> in
>>>>>> security for users.
>>>>>>
>>>>>> On Thu, 14 Feb 2019 at 21:58, Tom Forbes  wrote:
>>>>>>
>>>>>>> That makes sense, but in this case we are only talking about
>>>>>>> potentially yielding control of the docs subdomain which is not used to
>>>>>>> serve sensitive build artefacts?
>>>>>>>
>>>>>>> Another option is fastly.com, who support other large open source
>>>>>>> projects for free. They essentially give you geographically distributed
>>>>>>> HAProxy instances and you have a lot more control over them. I believe
>>>>>>> several large Linux distributions use them to serve cached apt packages.
>>>>>>>
>>>>>>> Regarding TLS termination, unfortunately any CDN we use will likely
>>>>>>> need to do this for the whole domain to get any benefit. The Django docs
>>>>>>> are text/html heavy with very few, if any, images. So the real speed
>>>>>>> benefit will have to come from serving that, which requires TLS 
>>>>>>> termination
>>>>>>> (and therefore interception) at their end.
>>>>>>>
>>>>>>> On Thu, 14 Feb 2019, 06:32 Markus Holtermann, <
>>>>>>> in...@markusholtermann.eu> wrote:
>>>>>>>
>>>>>>>> Hi all
>>>>>>>>
>>>>>>>> to elaborate on what Tobias said: we deliberately have the
>>>>>>>> infrastructure spread across multiple service providers: DNS registry,
>>>>>>>> nameservers, hosting, TLS certificate authority, … None of them have 
>>>>>>>> access
>>>>>>>> to everything. The reason is that we offer the download of the release
>>>>>>>> artifacts from the djangoproject.com website. And we would like to
>>>>>>>> ensure that the TLS termination happens by us and not some random 
>>>>>>>> service
>>>>>>>> provider. After all, Django is used by enterprises that do have some
>>>>>>>> restrictions on where you're allowed to download software from.
>>>>>>>>
>>>>>>>> By handing over DNS to some CDN provider, we loose the ability to
>>>>>>>> ensure that happens.
>>>>>>>>
>>>>>>>> That said, if there's a CDN that works as a reverse proxy and
>>>>>>>> doesn't require us to hand over

Re: Django 2.2 and the watchman reloader

[Tom Forbes]

Hey Claude,
Thanks for your feedback on the feature, I fully agree with you. I think we
should remove that warning message about the missing package. I will make a
PR to do that.

Regarding creating another reloader: it should not be that difficult to do
at all since we have all the other pieces in place. In theory it's just
implementing a class with single generator method.

If people agree I would like to use the 'watchdog' package for this rather
than the pyinotify library as it would be quicker to implement, a lot nicer
to work with and is easier to test.

On Thu, 21 Feb 2019, 19:56 Claude Paroz,  wrote:

> Le jeudi 21 février 2019 19:48:31 UTC+1, Dan Davis a écrit :
>>
>> Claude,
>>
>> I've tested a Django based application on 2.2b1 without watchman on
>> Windows, it does tell you about watchman, but it doesn't fail to run.
>> Apparently, it falls back to the old way of reloading.   Is this not the
>> behavior on Debian/Ubuntu?
>>
>
> Yes it is. I would say this is still a slight regression in two ways:
>
> - no messages told you the reload method was not optimal before. So now
> people will try to "fix" their system, more than before.
> - for Debian-based systems, you could improve the reloading performance by
> installing system or pip packages in 30 seconds. Now you have to spend 30
> minutes to search how watchman can be installed and to compile the package
> (+ you have to care yourself for any security issue).
>
> Claude
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/b059553a-e25e-4d90-beed-bf7e0f797305%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFNZOJMK_MCbKByWkVKC7QY8_As30%2BnKA4rR1c9UNqWLhUX60g%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.