Re: Adding a middleware to match cookies

[Jeff Willette]

Carl, thanks for the reply. 

Wy would this not help the efficiency of the downstream caches? Is it 
because the request has already passed through them with the cookies 
intact? and when it comes back through the response they have no way to 
know they have been stripped?

On Saturday, January 7, 2017 at 12:02:30 PM UTC+9, Carl Meyer wrote:
>
> Hi Jeff, 
>
> On 01/06/2017 06:21 PM, Jeff Willette wrote: 
> > I understand that calling is_authenticated on a user will require the 
> > session to be accessed and the vary by cookie header to be in the 
> > response, but if I understand how caching systems work then this will 
> > cause all cookies in the request to be taken into account, correct? 
>
> Yes. HTTP doesn't provide any way to say "vary only on this cookie, not 
> the others." Be nice if it did! 
>
> > What if there was an optional middleware early in the request 
> > processing that matched cookies based on a regex in settings and then 
> > modified the header to only include the matched cookies? 
> > 
> > That way...the unauthed users request will vary by cookies, but we 
> > would have removed all inconsequential cookies so all unauthed users 
> > will have the same set of cookies (likely none), and authed users 
> > will have (sessionid) or whatever else you wish to match and everyone 
> > will be happily cached correctly. 
> > 
> > Is there a hole in my thinking anywhere? Would this work as I 
> > expect? 
>
> I think it could work, yeah. It won't help the efficiency of any other 
> downstream HTTP caches, but they would still be safe (not serve anyone 
> the wrong response). And you should be able to help efficiency of 
> Django's own cache this way, if you strip cookies that Django / your 
> code doesn't care about before the request ever reaches the caching 
> middleware. Try it and experiment! 
>
> Carl 
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/b2b46018-f3e0-45c4-9161-cd68ecc9a1ea%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Django 2.0 Python version support (Python 3.6+ only?)

[James Bennett]

On Fri, Jan 6, 2017 at 7:03 PM Tim Graham  wrote:

> I don't know if matters to anyone, but I guess as long as we support
> Python 3.4 we can't do the type hinting project (PEP 484) since that's new
>
> The typing module also exists standalone on PyPI and thus is pip
> installable for Python 3 < 3.5.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAL13Cg_5F5MY54Dcc_58w0DcaX5k%2BJwZ_NtfeWJicqi1fc1Z0w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Django 2.0 Python version support (Python 3.6+ only?)

[Tim Graham]

I don't know if matters to anyone, but I guess as long as we support Python 
3.4 we can't do the type hinting project (PEP 484) since that's new in 3.5?

On Friday, January 6, 2017 at 12:08:07 PM UTC-5, Tim Graham wrote:
>
> Tom, I'm not following how Python 3.4 support in Django 2.0 will benefit 
> you if you want to stick to LTS versions of Django? I think either you or I 
> have a misunderstanding somewhere. I'll try to recap:
>
> Django 1.11 is the next LTS. It's supported until April 2020 and supports 
> Python 3.4. The next LTS, Django 2.2 is due in April 2019, after the Python 
> 3.4 end of life in March 2019 -- so I don't think we can justify supporting 
> Python 3.4 in that LTS unless we decide to base our supported Python policy 
> on CentOS rather than Python's own support lifecycle. Florian and Daniele 
> are proposing supporting Python 3.4 for Django 2.0 and/or 2.1. Those 
> versions have security support ending in April 2019 and December 2019, 
> respectively. If you want to use Python 3.4 and maximize the time you can 
> receive Django security updates, stick with 1.11 LTS (April 2020).
>
> Look at these tables if you're in doubt:
> https://www.djangoproject.com/download/#supported-versions
> https://docs.python.org/devguide/#status-of-python-branches
>
> On Friday, January 6, 2017 at 11:22:17 AM UTC-5, Tom Evans wrote:
>>
>> On Thu, Jan 5, 2017 at 8:10 PM, Asif Saifuddin  wrote: 
>> > Hi, 
>> > 
>> > django 2.0 will be released in december 2017 and ubuntu 18.04 will be 
>> > released in april 2018 which will default atleast 3.6, so I think this 
>> > should also be taken as consideration while deciding. 
>>
>> I know supporting endless versions of python is not desirable, but 
>> please bear in mind that some of us are in situations where what the 
>> latest release of Ubunutu is not really relevant. 
>>
>> Our organisation uses CentOS 6, which is not EOL until the end of 
>> 2020. In CentOS 6, the stock version of python is python 2.6; we go 
>> through special measures (EPEL) to get that up to python 2.7. If we 
>> wanted to make the move to Python 3, we'd be talking about Python 3.4, 
>> again through EPEL. 
>>
>> We have an infrastructure team responsible for provisioning servers, 
>> and it is on their schedule that OS upgrades occur - it is not easy 
>> for us as developers to argue that this team should spend significant 
>> resource to upgrade to a later OS version or to roll custom python 
>> RPMs. 
>>
>> Updating all of our codebase to Python 3 is going to be a pain for us, 
>> it is hard to argue a business need with "Everything stays exactly the 
>> same but is slightly more secure and easier to maintain". 
>> Realistically, when we move to Python 3, it will be because the 
>> supported Django LTS requires it. If we also have to jump through lots 
>> of hoops to get the very latest Python 3 release, it won't make it 
>> easier to argue, it will mean we are more likely to postpone it and 
>> keep using old django versions, particularly on internal intranet 
>> sites. 
>>
>> If there is a way that Python 3.4 support can be maintained without 
>> significant detriment or penalty, this would be greatly appreciated by 
>> those of us running more conservative enterprise distributions. 
>>
>> Cheers 
>>
>> Tom 
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/680d3d55-a9d4-46d9-a587-c49649f24cbf%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Adding a middleware to match cookies

[Carl Meyer]

Hi Jeff,

On 01/06/2017 06:21 PM, Jeff Willette wrote:
> I understand that calling is_authenticated on a user will require the
> session to be accessed and the vary by cookie header to be in the
> response, but if I understand how caching systems work then this will
> cause all cookies in the request to be taken into account, correct?

Yes. HTTP doesn't provide any way to say "vary only on this cookie, not
the others." Be nice if it did!

> What if there was an optional middleware early in the request
> processing that matched cookies based on a regex in settings and then
> modified the header to only include the matched cookies?
> 
> That way...the unauthed users request will vary by cookies, but we
> would have removed all inconsequential cookies so all unauthed users
> will have the same set of cookies (likely none), and authed users
> will have (sessionid) or whatever else you wish to match and everyone
> will be happily cached correctly.
> 
> Is there a hole in my thinking anywhere? Would this work as I
> expect?

I think it could work, yeah. It won't help the efficiency of any other
downstream HTTP caches, but they would still be safe (not serve anyone
the wrong response). And you should be able to help efficiency of
Django's own cache this way, if you strip cookies that Django / your
code doesn't care about before the request ever reaches the caching
middleware. Try it and experiment!

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/65281bd7-a2f6-d428-9743-683714c83057%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: optional middleware to eliminate cookies in request header

[Tim Graham]

It is a repost, let's use the first post: 
https://groups.google.com/d/msg/django-developers/4kwZP8Rq5IU/rQi5GgI6DgAJ

All first time posters go through a moderation queue.

On Friday, January 6, 2017 at 9:50:56 PM UTC-5, Jeff Willette wrote:
>
> I just posted this and I didn't see it pop up in the group, so I am sorry 
> if this is a repost. 
>
> I recently submitted a bad fix (
> https://code.djangoproject.com/ticket/27686#comment:6). I made a mistake 
> in thinking about how the caching system works but I think I have another 
> solution to the problem. 
>
> The solution in the ticket is good, but I would prefer to avoid another 
> request to get authed content and I would like to keep most of the logic in 
> django and continue to use the is_authenticated method. 
>
> What if there was a middleware early on in the request processing that 
> would match the cookie header by a regex in settings. If it did not find 
> one of the matches than it would eliminate the cookie header. This way the 
> vary by cookie header will still be sent, but inconsequential cookies will 
> be removed so that al unauthed users would have the same page cached 
> (varied by cookie with no cookies) and authed users would have theirs 
> cached correctly as well. 
>
> Is there anything wrong with my thinking about how this would work? Do you 
> think this is a good idea?
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/6f67f844-7135-4c96-904a-d0447144835f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Adding a middleware to match cookies

[Jeff Willette]

I recently proposed a bad fix (https://code.djangoproject.com/ticket/27686) but 
I think the problem still remains and I might have a way arpund it.

I understand that calling is_authenticated on a user will require the session 
to be accessed and the vary by cookie header to be in the response, but if I 
understand how caching systems work then this will cause all cookies in the 
request to be taken into account, correct?


The idea in the ticket about using ajax requests is good, but i would prefer to 
keep the page differences in the django logic and avoid an extra request to the 
server on every page load.

What if there was an optional middleware early in the request processing that 
matched cookies based on a regex in settings and then modified the header to 
only include the matched cookies?

That way...the unauthed users request will vary by cookies, but we would have 
removed all inconsequential cookies so all unauthed users will have the same 
set of cookies (likely none), and authed users will have (sessionid) or 
whatever else you wish to match and everyone will be happily cached correctly.

Is there a hole in my thinking anywhere? Would this work as I expect?

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/422d07a9-daac-4b3d-9f85-5cdd338fd8a4%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

optional middleware to eliminate cookies in request header

[Jeff Willette]

I just posted this and I didn't see it pop up in the group, so I am sorry 
if this is a repost. 

I recently submitted a bad fix 
(https://code.djangoproject.com/ticket/27686#comment:6). I made a mistake 
in thinking about how the caching system works but I think I have another 
solution to the problem. 

The solution in the ticket is good, but I would prefer to avoid another 
request to get authed content and I would like to keep most of the logic in 
django and continue to use the is_authenticated method. 

What if there was a middleware early on in the request processing that 
would match the cookie header by a regex in settings. If it did not find 
one of the matches than it would eliminate the cookie header. This way the 
vary by cookie header will still be sent, but inconsequential cookies will 
be removed so that al unauthed users would have the same page cached 
(varied by cookie with no cookies) and authed users would have theirs 
cached correctly as well. 

Is there anything wrong with my thinking about how this would work? Do you 
think this is a good idea?

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/4fb1ca99-6144-40aa-b840-df514cefdcc2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Use of HTML autofocus attribute in admin (#27692)

[Karen Tracey]

On Fri, Jan 6, 2017 at 12:48 PM, Tim Graham  wrote:

> Do you think we should remove autofocus from the search box to remedy this
> complaint (and possibly go back to using JavaScript for that) or are there
> any better solutions here?


I personally find the existing auto-focus behavior more annoying than
helpful, as it seems I'm much more likely to want to navigate by cursor or
page down than search when I land on an admin list page, and it forces me
to first get the focus away from the search box before I can do what I
want. So I wouldn't mind if auto-focus went away entirely.

Having it not only grab the focus but scroll back to to the top of the page
would be extraordinarily annoying, let's please avoid doing that.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CACS9radGBOdtFs6YbOgyMX%3DPL%3DqtK6kVYM-7bNUYAU2Z33RwEg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Use of HTML autofocus attribute in admin (#27692)

[Adam Johnson]

I think we should remove it and maybe add the Javascript back, this is 
going to affect many peoples' workflows, some of whom are in my 
organization.

On Friday, January 6, 2017 at 5:48:12 PM UTC, Tim Graham wrote:
>
> As part of removing inline JavaScript in the admin [0], snippets such as 
> document.getElementById("id_old_password").focus() were replaced with the 
> HTML5 autofocus attribute.
>
> This attribute has a possibly undesired behavior [1]: "*Warning*: this 
> attribute will force a page scroll to the field with the autofocus attribute 
> set, even within an iframe. So: know your audience, know the position of 
> the form."
>
> This attribute could be problematic on the change list's admin's search 
> box: "When coming back to the list screen from the edit screen, the 
> browser normally remembers your scroll position and restores it. This is 
> desirable behavior if you want to work through a long lists of objects. 
> However, when you include a search field in the list screen, it has 
> autofocus enabled, which prevents the scroll position from being restored. 
> This is annoying and unhelpful." [2]
>
> Do you think we should remove autofocus from the search box to remedy this 
> complaint (and possibly go back to using JavaScript for that) or are there 
> any better solutions here?
>
> [0] 
> https://github.com/django/django/commit/d638cdc42acec608c1967f44af6be32a477c239f
> [1] http://www.wufoo.com/html5/attributes/02-autofocus.html
> [2] https://code.djangoproject.com/ticket/27692
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/71125cb8-ba1a-4cde-99bc-e7360b2547dc%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Use of HTML autofocus attribute in admin (#27692)

[Tim Graham]

As part of removing inline JavaScript in the admin [0], snippets such as 
document.getElementById("id_old_password").focus() were replaced with the 
HTML5 autofocus attribute.

This attribute has a possibly undesired behavior [1]: "*Warning*: this 
attribute will force a page scroll to the field with the autofocus attribute 
set, even within an iframe. So: know your audience, know the position of 
the form."

This attribute could be problematic on the change list's admin's search 
box: "When coming back to the list screen from the edit screen, the browser 
normally remembers your scroll position and restores it. This is desirable 
behavior if you want to work through a long lists of objects. However, when 
you include a search field in the list screen, it has autofocus enabled, 
which prevents the scroll position from being restored. This is annoying 
and unhelpful." [2]

Do you think we should remove autofocus from the search box to remedy this 
complaint (and possibly go back to using JavaScript for that) or are there 
any better solutions here?

[0] 
https://github.com/django/django/commit/d638cdc42acec608c1967f44af6be32a477c239f
[1] http://www.wufoo.com/html5/attributes/02-autofocus.html
[2] https://code.djangoproject.com/ticket/27692

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/20d6d2bc-5e56-4531-91a3-9c4e01550296%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Django 2.0 Python version support (Python 3.6+ only?)

[Tim Graham]

Tom, I'm not following how Python 3.4 support in Django 2.0 will benefit 
you if you want to stick to LTS versions of Django? I think either you or I 
have a misunderstanding somewhere. I'll try to recap:

Django 1.11 is the next LTS. It's supported until April 2020 and supports 
Python 3.4. The next LTS, Django 2.2 is due in April 2019, after the Python 
3.4 end of life in March 2019 -- so I don't think we can justify supporting 
Python 3.4 in that LTS unless we decide to base our supported Python policy 
on CentOS rather than Python's own support lifecycle. Florian and Daniele 
are proposing supporting Python 3.4 for Django 2.0 and/or 2.1. Those 
versions have security support ending in April 2019 and December 2019, 
respectively. If you want to use Python 3.4 and maximize the time you can 
receive Django security updates, stick with 1.11 LTS (April 2020).

Look at these tables if you're in doubt:
https://www.djangoproject.com/download/#supported-versions
https://docs.python.org/devguide/#status-of-python-branches

On Friday, January 6, 2017 at 11:22:17 AM UTC-5, Tom Evans wrote:
>
> On Thu, Jan 5, 2017 at 8:10 PM, Asif Saifuddin  > wrote: 
> > Hi, 
> > 
> > django 2.0 will be released in december 2017 and ubuntu 18.04 will be 
> > released in april 2018 which will default atleast 3.6, so I think this 
> > should also be taken as consideration while deciding. 
>
> I know supporting endless versions of python is not desirable, but 
> please bear in mind that some of us are in situations where what the 
> latest release of Ubunutu is not really relevant. 
>
> Our organisation uses CentOS 6, which is not EOL until the end of 
> 2020. In CentOS 6, the stock version of python is python 2.6; we go 
> through special measures (EPEL) to get that up to python 2.7. If we 
> wanted to make the move to Python 3, we'd be talking about Python 3.4, 
> again through EPEL. 
>
> We have an infrastructure team responsible for provisioning servers, 
> and it is on their schedule that OS upgrades occur - it is not easy 
> for us as developers to argue that this team should spend significant 
> resource to upgrade to a later OS version or to roll custom python 
> RPMs. 
>
> Updating all of our codebase to Python 3 is going to be a pain for us, 
> it is hard to argue a business need with "Everything stays exactly the 
> same but is slightly more secure and easier to maintain". 
> Realistically, when we move to Python 3, it will be because the 
> supported Django LTS requires it. If we also have to jump through lots 
> of hoops to get the very latest Python 3 release, it won't make it 
> easier to argue, it will mean we are more likely to postpone it and 
> keep using old django versions, particularly on internal intranet 
> sites. 
>
> If there is a way that Python 3.4 support can be maintained without 
> significant detriment or penalty, this would be greatly appreciated by 
> those of us running more conservative enterprise distributions. 
>
> Cheers 
>
> Tom 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/e28e848c-4739-4d1d-aea7-82646bd68433%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Django 2.0 Python version support (Python 3.6+ only?)

['Tom Evans' via Django developers (Contributions to Django itself)]

On Thu, Jan 5, 2017 at 8:10 PM, Asif Saifuddin  wrote:
> Hi,
>
> django 2.0 will be released in december 2017 and ubuntu 18.04 will be
> released in april 2018 which will default atleast 3.6, so I think this
> should also be taken as consideration while deciding.

I know supporting endless versions of python is not desirable, but
please bear in mind that some of us are in situations where what the
latest release of Ubunutu is not really relevant.

Our organisation uses CentOS 6, which is not EOL until the end of
2020. In CentOS 6, the stock version of python is python 2.6; we go
through special measures (EPEL) to get that up to python 2.7. If we
wanted to make the move to Python 3, we'd be talking about Python 3.4,
again through EPEL.

We have an infrastructure team responsible for provisioning servers,
and it is on their schedule that OS upgrades occur - it is not easy
for us as developers to argue that this team should spend significant
resource to upgrade to a later OS version or to roll custom python
RPMs.

Updating all of our codebase to Python 3 is going to be a pain for us,
it is hard to argue a business need with "Everything stays exactly the
same but is slightly more secure and easier to maintain".
Realistically, when we move to Python 3, it will be because the
supported Django LTS requires it. If we also have to jump through lots
of hoops to get the very latest Python 3 release, it won't make it
easier to argue, it will mean we are more likely to postpone it and
keep using old django versions, particularly on internal intranet
sites.

If there is a way that Python 3.4 support can be maintained without
significant detriment or penalty, this would be greatly appreciated by
those of us running more conservative enterprise distributions.

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFHbX1L%2BxkpR%3DJ6K%2BPTvD6OZT34ZPDgw62HXnUmi5BUrPOm5%2Bg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Changing {% include %} to strip a trailing newline

[Florian Apolloner]

On Friday, January 6, 2017 at 3:13:27 PM UTC+1, Tim Graham wrote:
>
> I'd like to merge the current patch that removes the trailing newline in 
> the attrs.html template, then revisit the issue of {% include %} stripping 
> the trailing newline for the next version of Django so we don't need to 
> rush a decision about that.
>

Sounds good! 

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/74b14848-248e-4684-bcbd-f699b8377a81%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Changing {% include %} to strip a trailing newline

[Tim Graham]

I'd like to merge the current patch that removes the trailing newline in 
the attrs.html template, then revisit the issue of {% include %} stripping 
the trailing newline for the next version of Django so we don't need to 
rush a decision about that.

On Thursday, January 5, 2017 at 10:05:30 AM UTC-5, Alasdair Nicol wrote:
>
> Would adding the future include tag to the builtins in the template 
> OPTIONS work? If it did, then you'd only have to make the change to the 
> TEMPLATES setting instead of every template.
>
> OPTIONS={
> 'builtins': ['django.template.future_include'],  # put new include 
> in it's own module so that we don't add anything else in future.
> }
>
> I'm not familiar with that code, so I don't know whether that would 
> override the old include tag. 
>
> Cheers,
> Alasdair
>
> On Thursday, 5 January 2017 14:49:58 UTC, Tim Graham wrote:
>>
>> Hi Simon, I mentioned that idea offhand in an earlier mail, however, I 
>> feel that it would be really disruptive to force all projects to go through 
>> that deprecation and require adding {% load include from future %} to every 
>> template that uses the tag to silence warnings (and then remove those lines 
>> sometime after the deprecation ends). (More disruptive than just making a 
>> backwards-incompatible change that's likely to affect a (hopefully small) 
>> subset of users.) I guess I'm open to it if others think that's the way to 
>> go though.
>>
>> On Thursday, January 5, 2017 at 9:20:36 AM UTC-5, charettes wrote:
>>>
>>> Hey Tim,
>>>
>>> I just wanted to mention that another approach for a transition plan 
>>> could be
>>> to use {% load include from future %} like we did with the {% url %} tag.
>>>
>>> That doesn't answer the question of whether or not we should make the 
>>> change
>>> but I think it offers a better upgrade path than a global template 
>>> option or
>>> a keep_trailing_newline kwarg.
>>>
>>> Simon
>>>
>>> Le mercredi 4 janvier 2017 14:58:42 UTC-5, Tim Graham a écrit :

 Shortly after template widget rendering was merged, an issue about 
 extra newlines appearing in the rendered output was reported [0]:

 For example, from django-money:

 UIC-Franc
 \n\n
 US Dollar
 \n\n

 The newlines aren't useful and they break assertions like this:

 US Dollar in form.as_p

 --- end report---

 The reporter suggested removing the trailing newline in the attrs.html 
 template but Adam Johnson reported: "POSIX states that all text files 
 should end with a newline, and some tools break when operating on text 
 files missing the final newline. That's why git has the warning \ No 
 newline at end of file and Github has a warning symbol for the missing 
 newline."

 I suggested that perhaps {% include %} could do .rstrip('\n') on 
 whatever it renders.

 Preston pointed out that Jinja2 does something similar:

 http://jinja.pocoo.org/docs/dev/templates/#whitespace-control

- a single trailing newline is stripped if present
- other whitespace (spaces, tabs, newlines etc.) is returned 
unchanged

 I noted that the issue of {% include %} adding extra newlines was 
 raised in #9198 [1] but Malcolm said, "For backwards compatibility reasons 
 we cannot change the default behaviour now (there will be templates that 
 rely on the newline being inserted)." I was skeptical this would be a 
 burdensome change.

 Carl replied:  "It seems quite likely to me that there are templates in 
 the wild relying on preservation of the final newline. People render 
 preformatted text (e.g. text emails) using DTL. I probably have some text 
 email templates in older projects myself that would break with that 
 change. 

 We could add a new option to {% include %}, though." Adam also said, 
 "I too have used DTL for text emails that would break under that 
 behaviour. 
 New option to include sounds good to me."


 Me again: In the long run, having {% include %} remove the trailing 
 newline seems like a more sensible default. For example, I wouldn't expect 
 this code to have a newline inserted between it:


 {% include "foo.txt" %}
 {% include "bar.txt" %}
  

 An option for {% include %} seems superfluous given that if you were 
 writing


 {% include "foo.txt" %}{% include "bar.txt" %}
  

 now you can write the first thing which is much more intuitive.


 How about a keep_trailing_newline TEMPLATES option for backwards 
 compatibility for those who don't want to adapt their templates for the 
 new 
 behavior? Jinja2 has that option.

 Carl replied: An engine option may be better than an option to {% 
 include %}, though it doesn't allow us to ensure that we strip the 
 newline in the specific case of attrs.

>>

Re: Django 2.0 Python version support (Python 3.6+ only?)

[Tim Graham]

So you all want to do a one time exception to our guidance of of "Typically 
we will support a Python version up to and including the first Django LTS 
release whose security support ends after security support for that version 
of Python ends." and support Python 3.4 for how long? Or revise the 
guidance?

On Friday, January 6, 2017 at 8:16:02 AM UTC-5, Daniele Procida wrote:
>
> On Fri, Jan 6, 2017, Florian Apolloner > 
> wrote: 
>
> >In the end (in my experience), people are using Django everywhere and 
> part 
> >of the usage also comes from the fact that it's not that hard to deploy 
> for 
> >sysadmins since python is available anywhere; compiling a new Python + 
> >infrastructure around it is something else again and requires a lot of 
> >change requests in some companies. 
>
> In practical terms it makes a big difference. Remember <
> https://groups.google.com/forum/#!topic/django-developers/qCjfOu-FPxQ/discussion>,
>  
> that prompted a change of LTS policy? 
>
> *Anything* that makes the transition easier is to be welcomed, and that 
> doesn't just mean technically easier, it also means easier to think about 
> and to talk about to project managers and clients and web project owners. 
>
> Reassurance in time of change counts for a great deal. When someone gets 
> to spend a day or two basking in the glory of a top item on Hacker News 
> because he wrote a "Don't go to Python 3" article, there is clearly some 
> reassuring to be done. 
>
> If the technical cost of supporting 3.4 in Django 2.0 is not too high, I 
> feel it would be valuable to have it. 
>
> The actual technical justification for keeping it may be weak, but 
> barriers to adoption are not always technical ones anyway, and my 
> preference would be to keep them as low as possible. 
>
> Daniele 
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/4061bff0-5f24-49e9-83bf-7171662543c3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Django 2.0 Python version support (Python 3.6+ only?)

[Aymeric Augustin]

Hello,

I agreed with Florian and Daniele. Python 3.4 will be supported until March 
2019, giving it over 1 year of overlap with Django 2.0, including the entire 
mainstream support period.

I don’t expect supporting Python 3.4 to be a burden or dropping it to allow 
large gains. The language-level differences with Python 3.5 and 3.6 are 
minimal, unless I missed things that matter for Django, that is, allow us to 
remove problematic code.

It’s a different story from Python 2.4 vs. 2.5 vs. 2.6: back then 
`except Exception as exc` would break older Pythons until someone reported it 
and we didn’t have CI. Good times.

Perhaps we could update the support policy to say: Django X.Y will support 
Python versions that are under security support until the end of mainstream 
support for Django X.Y. The difference with the current policy is “mainstream 
support” instead of “extended support”.

Without changing the official policy, I think it would be nice to be more 
lenient with supported Python 3.x versions in the version of Django than drops 
Python 2. This isn’t a technical argument, it’s a marketing / developer 
relations argument. That doesn’t make it invalid :-)

And even if we keep support for Python 3.4 we’ll still be make our lives 
incredibly easier by dropping support for Python 2.

Best regards,

-- 
Aymeric.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/54E5855F-191B-49CD-9319-759CBD1BF81E%40polytechnique.org.
For more options, visit https://groups.google.com/d/optout.

Re: Django 2.0 Python version support (Python 3.6+ only?)

[Daniele Procida]

On Fri, Jan 6, 2017, Florian Apolloner  wrote:

>In the end (in my experience), people are using Django everywhere and part 
>of the usage also comes from the fact that it's not that hard to deploy for 
>sysadmins since python is available anywhere; compiling a new Python + 
>infrastructure around it is something else again and requires a lot of 
>change requests in some companies.

In practical terms it makes a big difference. Remember 
,
 that prompted a change of LTS policy?

*Anything* that makes the transition easier is to be welcomed, and that doesn't 
just mean technically easier, it also means easier to think about and to talk 
about to project managers and clients and web project owners. 

Reassurance in time of change counts for a great deal. When someone gets to 
spend a day or two basking in the glory of a top item on Hacker News because he 
wrote a "Don't go to Python 3" article, there is clearly some reassuring to be 
done.

If the technical cost of supporting 3.4 in Django 2.0 is not too high, I feel 
it would be valuable to have it.

The actual technical justification for keeping it may be weak, but barriers to 
adoption are not always technical ones anyway, and my preference would be to 
keep them as low as possible.

Daniele

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/20170106131548.2143269717%40mail.wservices.ch.
For more options, visit https://groups.google.com/d/optout.

Re: Django 2.0 Python version support (Python 3.6+ only?)

[Florian Apolloner]

"EDIT://" Oh, and while it is true that it is a bit more work for us to 
support multiple python versions, I've never seen it that bad. Installing 
more CI runners which have the matching python versions does not hurt that 
much either. Actually it might nowadays even be easier on CentOS than on 
Ubuntu given that the deadsnakes repo is somewhat dead nowadays if I 
understood that correctly.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/b815468d-345f-43ad-a392-ff93b19bf1b6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Django 2.0 Python version support (Python 3.6+ only?)

[Florian Apolloner]

After thinking a bit more: Are there any concrete reasons to drop 3.4/3.5 
aside from new features? Sure, security is an issue, but looking at the 
issues with cookie parsing we would have been better off by immediately 
fixing ourself instead of waiting for python (same goes for XML). So in the 
end, while I usually advocate for security and up2date systems, I do not 
think we gain much by dropping python 3.4 and/or 3.5.

I cannot speak for everyone else, but I do have the feeling that Django 
will get somewhat annoying to deploy for me (Don't get me wrong, I am 
getting paid to do that, but other people are not in such a lucky situation 
and I'd still like to see Django grow in enterprise environments). We are 
on RedHat 7, which I'll certainly stay on for a while. I can probably fetch 
newer python versions from IUS, but then I'll have to recompile mod_wsgi 
etc… Given that RedHat does not package Django at all, I am running from 
the upstream releases, which is perfectly fine for me -- but it would be 
great if I can use somewhat newer Django versions too.

In the end (in my experience), people are using Django everywhere and part 
of the usage also comes from the fact that it's not that hard to deploy for 
sysadmins since python is available anywhere; compiling a new Python + 
infrastructure around it is something else again and requires a lot of 
change requests in some companies.

No matter how we decide, I'd like to see python 3.4 supported on Django 2.0 
to ensure that people that wanna upgrade can at least try the first 
py3-only version without having to upgrade their systems (Ubuntu Trusty is 
still on 3.4 and still supported). And then maybe try to get some feedback 
from some companies and the versions of python they are using on Django 2.0.

Cheers,
Florian


On Tuesday, December 27, 2016 at 4:12:57 PM UTC+1, Tim Graham wrote:
>
> When I drafted the 1.11 release notes in May, I wrote, "The next major 
> release, Django 2.0, will only support Python 3.5+."
>
> Our Python version support policy is "Typically, we will support a Python 
> version up to and including the first Django LTS release whose security 
> support ends after security support for that version of Python ends."
>
> Python 3.5's EOL is September 2020 which I think is sufficiently close to 
> Django 1.11's EOL of April 2020 that we could say Django 2.0 is Python 
> 3.6+. The alternative is not to drop Python 3.5 compatibility until Django 
> 2.2 LTS which is supported until April 2022. I don't see much advantage to 
> that. Any objections?
>
> p.s. There is already a ticket suggesting to take advantage of a Python 
> 3.6 feature:
> https://code.djangoproject.com/ticket/27635* - *django.utils.crypto 
> should use secrets on Python 3.6+
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/6f7124e6-e00a-4cf6-9fca-91ee1c8dd91c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Working towards a simpler GCBV implementation?

[Florian Apolloner]

On Friday, January 6, 2017 at 11:11:37 AM UTC+1, Alex Scott wrote:
>
> If most everyone agrees that Tom's Vanilla Views are superior than what's 
> currently in core, maybe the path forward is to integrate into 2.0?
>

2.0 is not going to be as backwards incompatible as you might, it is mainly 
2.0 because we drop py2.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/2c578e8d-7025-4f35-83ee-64c102bc6427%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Template handling of undefined variables

[Adam Johnson]

>
> I apologise if adding the tests has made it harder to improve the
> behaviour of the tag.


I don't think you have anything to apologise for! More tests is always
better. This has clarified the current behaviour 👌

On 6 January 2017 at 10:15, Alasdair Nicol  wrote:

> Hi,
>
> On Thursday, 5 January 2017 17:51:39 UTC, Tim Martin wrote:
>>
>>
>>
>> On Thursday, 5 January 2017 04:15:31 UTC, Carl Meyer wrote:
>>>
>>> Hi Tim,
>>>
>>> On 01/04/2017 03:39 PM, Tim Martin wrote:
>>>
>>> > 1) There are test cases where we have templates that should treat "x
>>> >is y" as True where x and y are both undefined.
>>>
>>> Really? Is this an accidental side effect of some historical change, or
>>> really the intent of the test? That behavior seems buggy to me; more
>>> likely to be a bug-magnet than useful.
>>>
>>
>> There are a couple of test cases in test_if.py:
>> test_if_is_both_variables_missing and test_if_is_not_both_variables_missing.
>> These are verifying this specific case, so they haven't been introduced by
>> accident. I haven't got as far as figuring out whether it was a fully
>> thought through decision or whether someone just created these cases for
>> completeness.
>>
>
> I added the tests in ticket 26479 (see comment 5 onwards) [1]. I believe I
> added them for consistency with == and != operators, which have similar
> tests [2], [3] (trickier to spot because they are numbered). I apologise if
> adding the tests has made it harder to improve the behaviour of the tag.
>
> cheers,
> Alasdair
>
> [1]: https://code.djangoproject.com/ticket/26479#comment:5
> [2]: https://github.com/django/django/blob/master/tests/
> template_tests/syntax_tests/test_if.py#L85
> [3]: https://github.com/django/django/blob/master/tests/
> template_tests/syntax_tests/test_if.py#L112
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/django-developers/96f764fa-25c6-4104-9b6b-
> 8226e3b6bd8f%40googlegroups.com
> 
> .
>
> For more options, visit https://groups.google.com/d/optout.
>



-- 
Adam

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAMyDDM1_EWy8PTCbvQF1qU4vPAWtALpCJzFm7-dfQtU2JuR_Fw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Template handling of undefined variables

[Alasdair Nicol]

Hi,

On Thursday, 5 January 2017 17:51:39 UTC, Tim Martin wrote:
>
>
>
> On Thursday, 5 January 2017 04:15:31 UTC, Carl Meyer wrote:
>>
>> Hi Tim, 
>>
>> On 01/04/2017 03:39 PM, Tim Martin wrote: 
>>  
>> > 1) There are test cases where we have templates that should treat "x 
>> >is y" as True where x and y are both undefined. 
>>
>> Really? Is this an accidental side effect of some historical change, or 
>> really the intent of the test? That behavior seems buggy to me; more 
>> likely to be a bug-magnet than useful. 
>>
>
> There are a couple of test cases in test_if.py: 
> test_if_is_both_variables_missing and 
> test_if_is_not_both_variables_missing. These are verifying this specific 
> case, so they haven't been introduced by accident. I haven't got as far as 
> figuring out whether it was a fully thought through decision or whether 
> someone just created these cases for completeness.
>

I added the tests in ticket 26479 (see comment 5 onwards) [1]. I believe I 
added them for consistency with == and != operators, which have similar 
tests [2], [3] (trickier to spot because they are numbered). I apologise if 
adding the tests has made it harder to improve the behaviour of the tag. 

cheers,
Alasdair

[1]: https://code.djangoproject.com/ticket/26479#comment:5
[2]: 
https://github.com/django/django/blob/master/tests/template_tests/syntax_tests/test_if.py#L85
[3]: 
https://github.com/django/django/blob/master/tests/template_tests/syntax_tests/test_if.py#L112

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/96f764fa-25c6-4104-9b6b-8226e3b6bd8f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Working towards a simpler GCBV implementation?

[Alex Scott]

If most everyone agrees that Tom's Vanilla Views are superior than what's 
currently in core, maybe the path forward is to integrate into 2.0?

On Wednesday, January 4, 2017 at 3:00:03 PM UTC-5, Adam Johnson wrote:
>
> Fair enough, there are other community resources for this.
>
> The PR adding a pointer in the docs to ccbv.co.uk in 
> https://github.com/django/django/pull/7785 is a good idea that came from 
> this thread though.
>
> On 4 January 2017 at 13:08, Tim Graham > 
> wrote:
>
>> We typically shy away from endorsing third-party libraries in the Django 
>> docs. I don't think it makes much sense to stay something to the effect of 
>> "The built-in views are too complex so we recommend using other library 
>> instead."
>>
>>
>> On Wednesday, January 4, 2017 at 7:30:54 AM UTC-5, Asif Saifuddin wrote:
>>>
>>> Hi,
>>>
>>> I will update the doc pointing vanilla views as simpler alternative 
>>> implementation.
>>>
>>> Thanks
>>>
>>> On Tuesday, January 3, 2017 at 7:20:24 PM UTC+6, Adam Johnson wrote:

 I think this is probably too disruptive a change for Django core, 
 especially after so long with the current GCBV implementations - it would 
 require all users to rewrite their CBV's. Possibly the documentation could 
 recommend django-vanilla-views?

 On 3 January 2017 at 13:02, Asif Saifuddin  wrote:

> Hi,
>
> I have started work on https://github.com/django/django/pull/7783 for 
> converting django built in generic views to django-vanilla-views.
>
> I would like to hear what you think before I proceed for more.
>
> Thanks,
>
> Asif
>
> On Friday, September 16, 2016 at 1:37:36 AM UTC+6, Asif Saifuddin 
> wrote:
>>
>> Hi Tom,
>>
>> I am basically +1 to see this change in the django core. The package 
>> is 3 years old and should be tested enough. If you/other core team 
>> members 
>> thinks that now is a good time to include it to core and deprecation of 
>> older API, then I will be willing to work and send PR for this.
>>
>> Looking for others opinions.
>>
>> Thanks,
>>
>> Asif
>>
>> On Thursday, October 3, 2013 at 3:09:58 PM UTC+6, Tom Christie wrote:
>>>
>>> Hi folks,
>>>
>>> I recently released an alternative implementation of Django's 
>>> existing class based views.  The intention was to mirror the *exact* 
>>> same 
>>> set of functionality that we currently provide, but simplify the 
>>> implementation and API.  It's nothing cleverer or grander than a clean 
>>> re-write, but the end result is *significantly* less complex.  The 
>>> class 
>>> hierarchy is trivial, the API is much smaller, and the flow control is 
>>> much 
>>> more obvious.
>>>
>>> I won't go into any more here, as there's plenty of detail in the 
>>> documentation:
>>>
>>> http://django-vanilla-views.org
>>>
>>> There's also useful context in the related blog post, here:
>>>
>>> http://dabapps.com/blog/fixing-djangos-generic-class-based-views/
>>>
>>> The difficult thing here really is that there's no obvious approach 
>>> to introducing something like this in a backwards compatible way.
>>>
>>> It might be that we could take an incremental approach using the 
>>> standard deprecation process, but given the nature of the GCBVs it'd 
>>> likely 
>>> be pretty awkward.
>>> There might be some bits of deprecation process we simply wouldn't 
>>> be able to deal with in any sensible way, and we'd certainly end up 
>>> with a 
>>> seriously gnarly implementation during the deprecation period.
>>>
>>> I'd be interested in getting some opinions from folks on the 
>>> following:
>>>
>>> * If a simpler GCBV implementation along the lines of 
>>> django-vanilla-views is something we think we should working towards.
>>> * What approaches we might be able to take to dealing with backwards 
>>> compatibility if we did want to do so.
>>>
>>> Thanks for your time.
>>>
>>>   Tom
>>>
>>> -- 
> You received this message because you are subscribed to the Google 
> Groups "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send 
> an email to django-develop...@googlegroups.com.
> To post to this group, send email to django-d...@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/5792872e-157c-4ba6-87c1-bf0f5a07d981%40googlegroups.com
>  
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>
>

Re: GeoDjango OffdbRasterField

[Daniel Wiesmann]

I am not sure what the policy is for components with such strong
restrictions. So I guess this is a question back to the group:

What are requirements for components within Django with regard to system
architecture? Are components required to work on distributed systems?

If there is no general answer to this question, please advise on this
specific case.

On 29-12-2016 20:07, Piero Toffanin wrote:
> Your analysis is spot on, PostGIS references the path to off-db rasters
> using an absolute path, which needs to be resolved on the PostGIS server.
> 
> I guess a developer could map a folder (via samba for example) on the
> PostGIS server to match that of the Django server so that both servers
> share the same path structure. There would probably be some problems
> with this approach as well, especially when mixing different operating
> systems, or synchronization issues depending on the type of network
> mapping used.
> 
> If this addition is too problematic to be added in core, I could just
> isolate the code and release it as a plugin, with some documentation
> about the limits of the approach.
> 
> On Thursday, December 29, 2016 at 2:35:59 PM UTC-5, Daniel Wiesmann wrote:
> 
> This is an interesting idea and approach, thanks Piero for the
> suggestion and the proposed solution.
> 
> I agree with Piero and Adam, it would be a great addition to the
> raster field. Especially for large volumes of data (not only for
> individual large files, but also for many small files). Raster data
> can eat up a lot of storage quickly, and delegating that to an
> off-db storage is useful.
> 
> I have been looking at the out-of-db feature for PostGIS rasters,
> but I am not sure if it can be used in the Django context. The
> problem I see is the storage location.
> 
> If I understand it right, then the raster files need to be on the
> same system as the PostGIS instance, i.e. they need to be available
> on the filesystem of the postgis server. Like that, PostGIS can
> access the files from within its raster operations (such as
> intersections, getting pixel values etc), and it will work as if the
> data was stored in the db directly.
> 
> In that case however, the out-of-db field would only work if Django
> is running on the same server as the PostGIS instance. So I think it
> would fail for more distributed systems, where you have multiple
> application instances, and a remote PostGIS, which might have
> replicas etc.
> 
> In your experience Piero, is that correct? How are the out-of-db
> raster files stored by PostGIS?
> 
> I have never used the out-of-db raster option so I am not sure if I
> understood the storage mechanism correclty.
> 
> Coincidently, I recently had a similar problem, and I have been
> testing a version of a raster field which is a subclass of the
> regular Django FileField. In this approach, the storage can be any
> Django compatible storage (including remote object storages such as
> S3). The downside is that PostGIS will no longer recognize the data
> as rasters, so lookups and PostGIS internal functions will not be
> available.
> 
> 
> This is still quite experimental (it uses a patched Django with an
> extension of the GDALRaster) and is slightly hacky, but does convert
> the remote rasters into GDALRaster instances when opened.
> 
> 
> https://github.com/geodesign/django-raster/compare/raster_file_field
> 
> 
> 
> An ideal solution would be a mixture of the two, but I am not sure
> if PostGIS can handle remote storages.
> 
> 
> On Tuesday, December 20, 2016 at 4:33:21 PM UTC, Tim Graham wrote:
> 
> Daniel Wiesmann did all the work for RasterField. I'm not sure
> if he follows this list but you can find his email address in
> the Django commit longs and mail him to ask for his input.
> 
> https://github.com/yellowcap
> 
> On Tuesday, December 20, 2016 at 11:27:25 AM UTC-5, Piero
> Toffanin wrote:
> 
> It's stored on the file system. This is to improve
> performance when storing large geospatial datasets.
> 
> This would only work on PostGIS.
> 
> On Thursday, December 15, 2016 at 3:11:37 PM UTC-5, Adam
> Johnson wrote:
> 
> I can't say I'm that familiar with GeoDjango, but that
> does sound like a useful feature. Where does the data
> get stored if not in the DB? And does this feature exist
> on any of the other database backends that GeoDjango
> supports?
> 
> On 14 December 2016 at 18:40, Piero Toffanin
>  wrote:
> 
> Hello,
> 
> Not sure this is the right place to post this, if
> not, could so