Re: A Django Async Roadmap

[charettes]

AFAICT the performance hit is minimal unless you are doing something
slow when logging warnings (e.g. -Walways with a slow I/O py.warnings 
handler).

The Python/Django instrumentation simply adds a value to Model._state
and look it up on attribute accesses.

Cheers,
Simon

Le lundi 20 août 2018 19:34:11 UTC-4, Curtis Maloney a écrit :
>
> In general this sounds like a tremendously useful tool... I'm caused to 
> wonder, however... what, if any, are the performance impacts? 
>
> -- 
> Curtis 
>
>
>
> On 08/21/2018 08:10 AM, charettes wrote: 
> > Regarding the lazy loading of deferred fields and foreign keys 
> > I wanted to mention I've been working on a third-party application 
> > that allows overriding the default behavior[0]. 
> > 
> > The project works by tainting objects retrieved from "sealed" 
> > querysets and having fields descriptors lookup whether or not 
> > the object is "sealed" on attribute access and warn the developer 
> > about it if it's the case. Warnings can be elevated to errors 
> > using `filterwarnings` when deemed appropriated (e.g. CI, staging). 
> > 
> > It has been an useful tool to assist in figuring out where 
> > `select_related()` and `prefetch_related()` should be used 
> > to adjust complex projects database interactions. 
> > 
> > I assume a similar pattern could be used to mark objects retrieved 
> > from `QuerySet.__aiter__` to prevent non-async queries from being 
> > performed on attribute accesses; on `Model._state.async = True` 
> > field descriptors would error out. 
> > 
> > 
> > Cheers, 
> > Simon 
> > 
> > P.-S. 
> > 
> > While the project might look complex most of the code takes care 
> > of the delicate tasks of replacing fields descriptors once models 
> > are configured which could be significantly simplified if it was 
> > part of Django core. 
> > 
> > [0] https://github.com/charettes/django-seal 
> > Le samedi 9 juin 2018 02:30:59 UTC-4, Josh Smeaton a écrit : 
> > 
> > I think most of what you've laid out sounds great and that pursuing 
> > async Django is in the projects best interests. The sync to async 
> > and async to sync wrappers that have come out of channels give me 
> > much more confidence that this project is doable in a reasonable 
> > amount of time with backwards compatibility being preserved. 
> > 
> > The only real concern I have at the moment is around your comments 
> > regarding on demand foreign key traversal. If existing code running 
> > synchronously is going to be impacted, that's going to be very 
> > difficult for a lot of people. If it's only asynchronous traversal 
> > that'll have issues, then I have no real concern, as on demand 
> > foreign key fetching is usually a bug anyway. 
> > 
> > Having a brief read through the psycopg asynchronous docs[0], it 
> > looks like a number of features will be impossible or troublesome to 
> > use, like transactions, executemany, and named cursors (.iterator() 
> > with server side cursors). We'd also need to investigate how 
> > pgbouncer would work in async mode, as most large sites using 
> > postgres are also using pgbouncer. I would expect support can only 
> > further improve, especially if there is a driver like django 
> > pushing. Fallback would just be to run inside a thread pool though, 
> > so it's not a blocker for the rest of the proposal. 
> > 
> > Very exciting times ahead! 
> > 
> > [0] http://initd.org/psycopg/docs/advanced.html#asynchronous-support 
> >  
> > 
> > On Monday, 4 June 2018 23:18:23 UTC+10, Andrew Godwin wrote: 
> > 
> > Hello everyone, 
> > 
> > For a while now I have been working on potential plans for 
> > making Django async-capable, and I finally have a plan I am 
> > reasonably happy with and which I think we can actually do. 
> > 
> > This proposed roadmap, in its great length, is here: 
> > 
> > https://www.aeracode.org/2018/06/04/django-async-roadmap/ 
> >  
> > 
> > I'd like to invite discussion on this potential plan - 
> including: 
> > 
> >   - Do we think async is worth going after? Note that this is 
> > just async HTTP capability, not WebSockets (that would remain in 
> > Channels) 
> > 
> >   - Can we do this in a reasonable timeframe? If not, is there a 
> > way around that? 
> > 
> >   - Are the proposed modifications to how Django runs sensible? 
> > 
> >   - How should we fund this? 
> > 
> > There's many more potential questions, and I really would love 
> > feedback on this. I'm personally pretty convinced that we can 
> > and should do this, but this is a decision we cannot take 
> > lightly, and I would love to hear what you have to say. 
> > 
> > Andrew 

Re: A Django Async Roadmap

[Curtis Maloney]

In general this sounds like a tremendously useful tool... I'm caused to 
wonder, however... what, if any, are the performance impacts?


--
Curtis



On 08/21/2018 08:10 AM, charettes wrote:

Regarding the lazy loading of deferred fields and foreign keys
I wanted to mention I've been working on a third-party application
that allows overriding the default behavior[0].

The project works by tainting objects retrieved from "sealed"
querysets and having fields descriptors lookup whether or not
the object is "sealed" on attribute access and warn the developer
about it if it's the case. Warnings can be elevated to errors
using `filterwarnings` when deemed appropriated (e.g. CI, staging).

It has been an useful tool to assist in figuring out where
`select_related()` and `prefetch_related()` should be used
to adjust complex projects database interactions.

I assume a similar pattern could be used to mark objects retrieved
from `QuerySet.__aiter__` to prevent non-async queries from being
performed on attribute accesses; on `Model._state.async = True`
field descriptors would error out.


Cheers,
Simon

P.-S.

While the project might look complex most of the code takes care
of the delicate tasks of replacing fields descriptors once models
are configured which could be significantly simplified if it was
part of Django core.

[0] https://github.com/charettes/django-seal
Le samedi 9 juin 2018 02:30:59 UTC-4, Josh Smeaton a écrit :

I think most of what you've laid out sounds great and that pursuing
async Django is in the projects best interests. The sync to async
and async to sync wrappers that have come out of channels give me
much more confidence that this project is doable in a reasonable
amount of time with backwards compatibility being preserved.

The only real concern I have at the moment is around your comments
regarding on demand foreign key traversal. If existing code running
synchronously is going to be impacted, that's going to be very
difficult for a lot of people. If it's only asynchronous traversal
that'll have issues, then I have no real concern, as on demand
foreign key fetching is usually a bug anyway.

Having a brief read through the psycopg asynchronous docs[0], it
looks like a number of features will be impossible or troublesome to
use, like transactions, executemany, and named cursors (.iterator()
with server side cursors). We'd also need to investigate how
pgbouncer would work in async mode, as most large sites using
postgres are also using pgbouncer. I would expect support can only
further improve, especially if there is a driver like django
pushing. Fallback would just be to run inside a thread pool though,
so it's not a blocker for the rest of the proposal.

Very exciting times ahead!

[0] http://initd.org/psycopg/docs/advanced.html#asynchronous-support


On Monday, 4 June 2018 23:18:23 UTC+10, Andrew Godwin wrote:

Hello everyone,

For a while now I have been working on potential plans for
making Django async-capable, and I finally have a plan I am
reasonably happy with and which I think we can actually do.

This proposed roadmap, in its great length, is here:

https://www.aeracode.org/2018/06/04/django-async-roadmap/


I'd like to invite discussion on this potential plan - including:

  - Do we think async is worth going after? Note that this is
just async HTTP capability, not WebSockets (that would remain in
Channels)

  - Can we do this in a reasonable timeframe? If not, is there a
way around that?

  - Are the proposed modifications to how Django runs sensible?

  - How should we fund this?

There's many more potential questions, and I really would love
feedback on this. I'm personally pretty convinced that we can
and should do this, but this is a decision we cannot take
lightly, and I would love to hear what you have to say.

Andrew

--
You received this message because you are subscribed to the Google 
Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to django-developers+unsubscr...@googlegroups.com 
.
To post to this group, send email to django-developers@googlegroups.com 
.

Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/0b3b92b9-eee7-4909-82d6-c0138e8b5760%40googlegroups.com 

Re: A Django Async Roadmap

[charettes]

Regarding the lazy loading of deferred fields and foreign keys
I wanted to mention I've been working on a third-party application
that allows overriding the default behavior[0].

The project works by tainting objects retrieved from "sealed"
querysets and having fields descriptors lookup whether or not
the object is "sealed" on attribute access and warn the developer
about it if it's the case. Warnings can be elevated to errors
using `filterwarnings` when deemed appropriated (e.g. CI, staging).

It has been an useful tool to assist in figuring out where
`select_related()` and `prefetch_related()` should be used
to adjust complex projects database interactions.

I assume a similar pattern could be used to mark objects retrieved
from `QuerySet.__aiter__` to prevent non-async queries from being
performed on attribute accesses; on `Model._state.async = True`
field descriptors would error out.


Cheers,
Simon

P.-S.

While the project might look complex most of the code takes care
of the delicate tasks of replacing fields descriptors once models
are configured which could be significantly simplified if it was
part of Django core.

[0] https://github.com/charettes/django-seal
Le samedi 9 juin 2018 02:30:59 UTC-4, Josh Smeaton a écrit :
>
> I think most of what you've laid out sounds great and that pursuing async 
> Django is in the projects best interests. The sync to async and async to 
> sync wrappers that have come out of channels give me much more confidence 
> that this project is doable in a reasonable amount of time with backwards 
> compatibility being preserved.
>
> The only real concern I have at the moment is around your comments 
> regarding on demand foreign key traversal. If existing code running 
> synchronously is going to be impacted, that's going to be very difficult 
> for a lot of people. If it's only asynchronous traversal that'll have 
> issues, then I have no real concern, as on demand foreign key fetching is 
> usually a bug anyway.
>
> Having a brief read through the psycopg asynchronous docs[0], it looks 
> like a number of features will be impossible or troublesome to use, like 
> transactions, executemany, and named cursors (.iterator() with server side 
> cursors). We'd also need to investigate how pgbouncer would work in async 
> mode, as most large sites using postgres are also using pgbouncer. I would 
> expect support can only further improve, especially if there is a driver 
> like django pushing. Fallback would just be to run inside a thread pool 
> though, so it's not a blocker for the rest of the proposal.
>
> Very exciting times ahead!
>
> [0] http://initd.org/psycopg/docs/advanced.html#asynchronous-support
>
> On Monday, 4 June 2018 23:18:23 UTC+10, Andrew Godwin wrote:
>>
>> Hello everyone,
>>
>> For a while now I have been working on potential plans for making Django 
>> async-capable, and I finally have a plan I am reasonably happy with and 
>> which I think we can actually do.
>>
>> This proposed roadmap, in its great length, is here:
>>
>> https://www.aeracode.org/2018/06/04/django-async-roadmap/
>>
>> I'd like to invite discussion on this potential plan - including:
>>
>>  - Do we think async is worth going after? Note that this is just async 
>> HTTP capability, not WebSockets (that would remain in Channels)
>>
>>  - Can we do this in a reasonable timeframe? If not, is there a way 
>> around that?
>>
>>  - Are the proposed modifications to how Django runs sensible?
>>
>>  - How should we fund this?
>>
>> There's many more potential questions, and I really would love feedback 
>> on this. I'm personally pretty convinced that we can and should do this, 
>> but this is a decision we cannot take lightly, and I would love to hear 
>> what you have to say.
>>
>> Andrew
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/0b3b92b9-eee7-4909-82d6-c0138e8b5760%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Model default modelform

[Adam Johnson]

Not sure that's what's being suggested here James?

But I'm -1 on this because it's adding more coupling between models and
forms.

Also Jamesie, can't you just subclass in your ModelAdmin to
replace get_form / View classes to replace get_form_class and achieve the
same thing? As far as I understand the only thing your suggestion changes
is that the forms used in views and admin can be changed at once, which is
not necessarily desirable as both are intended for different users.

On Mon, 20 Aug 2018 at 15:38, James Bennett  wrote:

> I'd be -1 on anything that encourages people to use ModelForm with all
> fields included by default; that's asking for mass-assignment security
> holes.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/CAL13Cg--AGfXz4uLtAbC0miGCrQE9M1o6_cbC114UULBVhOPJg%40mail.gmail.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>


-- 
Adam

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAMyDDM0wXj1%2B-jk61f-MHarZjMdXpYmPptxf4cC0SXZD1WvOqA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Model default modelform

[James Bennett]

I'd be -1 on anything that encourages people to use ModelForm with all
fields included by default; that's asking for mass-assignment security
holes.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAL13Cg--AGfXz4uLtAbC0miGCrQE9M1o6_cbC114UULBVhOPJg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Model default modelform

[Jamesie Pic]

Hi all,

Currently when you want a model to be edited from a custom modelform, you
need to make use of that new modelform manually in your create/update views
and admin.

Would it be possible to add a new overridable method in the model to
generate a default modelform ?

Then, default create/update views (including admin's) could detect if a
model has such a method and call it to get the modelform instead of going
through the default factory.

For example:

class YourModel(models.Model):
def get_form_class(self):
return YourModelForm

Thanks in advance for your feedback

Have a great day

-- 
∞

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAC6Op1_e-065up%2Bs5mz7d%3D%3Dcxmz7w5coE0cMswfUpgOh3eRRjw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Help wanted testing proposal for the Migration Graph algorithm

[Carlton Gibson]

Making the script less noisy here I commented out the actual work.
(So it didn't function as a test.) 
(No comment. )

Corrected version:

```
import sys

from django.db import connection
from django.db.migrations.loader import MigrationLoader

loader = MigrationLoader(connection)

backwards = loader.graph.root_nodes()
forwards = loader.graph.leaf_nodes()

print('Calculating backward plans:')
for root in backwards:
loader = MigrationLoader(connection)
sys.stdout.write('.')
plan = loader.graph.backwards_plan(root)
#print(plan)
sys.stdout.write('\n')
print('Calculating forward plans:')
for leaf in forwards:
loader = MigrationLoader(connection)
sys.stdout.write('.')
plan = loader.graph.forwards_plan(leaf)
# print(plan)

sys.stdout.write('\nRun Done\n\n')

```

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/981a90ea-fa4a-4c6f-8e78-198d5a3751a1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: HTML5 and XHTML5 documents

[Nils Fredrik Gjerull]

Den 20. aug. 2018 11:32, skrev Nils Fredrik Gjerull:
> XML
> materialization of HTML5, and there is also the SGML-inspired version.

I intended to write XML serialization :)

-- 
Nils Fredrik Gjerull
-
"Ministry of Eternal Affairs"
Computer Department
( Not an official title :) )

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/4b2bfede-fa4f-7995-2392-a08b6cbedd8e%40gjerull.net.
For more options, visit https://groups.google.com/d/optout.

Re: HTML5 and XHTML5 documents

[Nils Fredrik Gjerull]

Den 17. aug. 2018 22:07, skrev James Bennett:
>
> If you're basing your understanding on browser support, you're not
> doing XML/XHTML. You're doing "a thing that looks like XHTML and works
> in my browser".

Webstandards has and probably always will be defined by browser support.
That's how the web works. I am sure you know it. XHTML5 is a XML
materialization of HTML5, and there is also the SGML-inspired version.
It is possible have valid xml markup without a schema, and it is
possible to make a schema more or less tolerant. The reason there is no
official schema for the XML version is the same as the reason there is
no official SGML DTD. Browser support is a moving target. Any official
schema also need to take custom elements into consideration. So a schema
can be strict about all the wrapping tags and header tags, but it need
to be more "free-form" when it comes to what kind of tags can be inside
the body.

It not not only work in "my browser" it work for every major browser out
there. Which is the "de facto" definition of when browser technology can
be used.

Regards

-- 
Nils Fredrik Gjerull
-
"Ministry of Eternal Affairs"
Computer Department
( Not an official title :) )


-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/7ee1f4b0-42ed-40ac-c0d2-a63df5f6a62b%40gjerull.net.
For more options, visit https://groups.google.com/d/optout.