Re: Enforcing a max size for form field values read into memory (review/determination of next steps needed)

[Tim Graham]

A new ticket asks how to elegantly handle this in the admin actions where 
it's easy to post more than 1000 values using the "select all" button. My 
feeling is that applications shouldn't change their behavior based on this 
setting, but a consensus on how to proceed here might be useful for 
documenting best practices as other apps run into similar troubles.

https://code.djangoproject.com/ticket/26810

On Saturday, May 7, 2016 at 10:57:55 AM UTC-4, Tim Graham wrote:
>
> After discussion on the PR, we concluded that the reasons that the 
> request.body check was removed weren't valid, so this check is reinstated. 
> I believe I've addressed all Tom Christie's concerns by now. If anyone else 
> would like to take a look, now is the time. Thanks!
>
> https://github.com/django/django/pull/6447
>
> On Tuesday, April 26, 2016 at 6:47:09 PM UTC-4, Tim Graham wrote:
>>
>> It seems there was request.body checking in previous iterations of the PR 
>> but it was dropped for reasons that aren't entirely clear to me:
>> https://github.com/django/django/pull/3852#discussion_r35350372
>>
>> On Wednesday, April 20, 2016 at 9:30:48 PM UTC-4, Cristiano Coelho wrote:
>>>
>>> Hi,
>>>
>>> In particular I'm interested in this new setting: 
>>> DATA_UPLOAD_MAX_MEMORY_SIZE 
>>> [1]
>>> that only seems to be checked against mutlparts [2] and url encoded[3] 
>>> request bodies.
>>>
>>> It could be good that this setting is also checked against other types 
>>> where request.body is read directly, as you can still get the 
>>> content-length from the body right? Please correct me if I'm wrong, but 
>>> when in already django code all body data is always loaded in memory except 
>>> for files and multi-part uploads which are streamed.
>>> So JSON, XML or even plain text post requests could benefit from the 
>>> DATA_UPLOAD_MAX_MEMORY_SIZE setting and it could be very convenient for 
>>> example if an attacker sends a huge json, the python (at least 2.7) 
>>> json.loads call usually crashes with an out of memory error when the string 
>>> is too big while still creating a huge RAM spike.
>>>
>>>
>>> [1] 
>>> https://github.com/django/django/pull/6447/files#diff-ba8335f5987fcd81d41c28cd1879a9bfR291
>>> [2] 
>>> https://github.com/django/django/pull/6447/files#diff-ba8335f5987fcd81d41c28cd1879a9bfR291
>>> [3] 
>>> https://github.com/django/django/pull/6447/files#diff-0eb6c5000a61126731553169fddb306eR294
>>>
>>>
>>> El martes, 19 de abril de 2016, 13:06:27 (UTC-3), Tom Christie escribió:

 > If you are using django-rest-framework, how would the fields counter 
 work?. It would be a shame if only multi part and urlencoded uploads would 
 have the benefit of these checks, while still allowing json, xml and 
 others 
 still be "exploited".
 Note I didn't really read the code changes completely so I'm talking 
 with almost no knowledge on the proposed change.

 They wouldn't be respected by anything other than multi-part or 
 urlencoded requests.
 Tim's correct in noting that accessing `request.body` or 
 `request.stream` won't apply these checks (which is for example, what REST 
 framework does).

 Even so I think this is probably a reasonable approach. We could add 
 support for respecting these settings in REST framework too, once they 
 exist.(Although I think we'd have need to have a stricter consideration of 
 backwards compat wrt. folks POSTing large amounts of JSON data)



-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/cf03948e-bc7b-40ae-935f-51341fbdfba5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Future of the development server's auto-reloading

[Tim Graham]

A pull request is proposed to add a new setting to allow specifying a 
custom reloader:

https://github.com/django/django/pull/6719

Is this something anyone else would find useful and does it seems like we 
could continue support that option even if autoreloading is refactored 
based on the ideas here?

On Saturday, June 4, 2016 at 8:37:25 PM UTC-4, Elf Sternberg wrote:
>
>
>
> On Saturday, August 8, 2015 at 2:53:32 PM UTC-7, Aymeric Augustin wrote:
>>
>> Hello,
>>
>> While writing some horrific code [1] related to the development server’s 
>> auto-reloading, I realized that its design is fundamentally flawed and I 
>> think that we should reconsider it.
>>
>> The current implementation walks through the list of all imported Python 
>> modules, attempts to figure out which file they come from and builds a list 
>> of these files. Then it watches all these files for changes by checking if 
>> their mtime changed every second or, since 1.7, with inotify on Linux. I 
>> tried to do it with kqueue on BSD (read: OS X) but I failed. This stuff is 
>> hard.
>>
>> In addition to the source of imported Python modules, the autoreloader 
>> watches .mo files in order to catch translation changes and source files 
>> for modules that failed to import — it extracts them from exception 
>> tracebacks. This shows the limits of basing the list on imported Python 
>> modules. Even with such hacks, the auto-reloader will never handle all 
>> cases. Two examples:
>>
>> - It doesn’t survive a syntax error in the settings module. I have 
>> reasons to believe that this would be extremely messy to fix.
>> - If a module reads a configuration file on disk at startup and caches 
>> it, the auto-reloader doesn’t detect changes to that file.
>>
>> So we’re stuck with cases where the development server doesn’t reload. I 
>> can’t say they’re edge cases; who never made a typo in a settings file? 
>> People who run tutorials report that it’s a very common cause of 
>> frustration for beginners.
>>
>> It's also wasteful. There’s little need to watch every file of every 
>> dependency in the virtualenv for changes.
>>
>> I think it would be much better to remove the auto-reloading logic from 
>> Django and have an external process watch the current working directory for 
>> changes. (When virtualenv is considered appropriate for people with exactly 
>> 0 days of Django experience under their belt, I think we can require a 
>> dependency for the development server’s auto-reloading.)
>>
>> The behavior would be much easier to describe: if you change something in 
>> your source tree, it reloads automatically; if you change something 
>> somewhere else, you must reload manually. It would also be quite easy to 
>> customize with a standard include / exclude mechanism to select which files 
>> should be watched.
>>
>> We aren’t in the business of writing production web servers, neither are 
>> we in the business of writing a filesystem watcher :-) Watchman [2] appears 
>> to be a robust solution. However I would prefer a pure-Python 
>> implementation that could easily be installed in a virtualenv. Does someone 
>> have experience in this area?
>>
>> I’m not making a concrete proposal as I haven’t studied the details. It 
>> this point I’d just like to hear what you think of the general concept.
>>
>> Thanks!
>>
>> -- 
>> Aymeric.
>>
>> [1] https://github.com/django/django/pull/5106
>> [2] https://github.com/facebook/watchman
>> [3] 
>> http://tutorial.djangogirls.org/en/django_installation/index.html#virtual-environment
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/114738d3-48fc-4901-bc6c-868fb1ff6e2d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Google Summer of Code Updates - Class based indexes

[Tim Graham]

Support for more index types is planned. This is only part 1 of the work. 
Please see https://gist.github.com/akki/7fd50505928dac58dc350e6cb186a404 
for the timeline.

On Monday, June 27, 2016 at 5:02:06 PM UTC-4, thinkwel...@gmail.com wrote:
>
> Will it be possible to create indexes other than btree? In reviewing the 
> code it doesn't look like it but I hope I'm wrong. It'd be awesome to 
> support the many more specific index options without using run_sql.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/005780c8-d7e0-445d-90f6-1530d4688c3f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Google Summer of Code Updates - Class based indexes

[thinkwelldesigns]

Will it be possible to create indexes other than btree? In reviewing the 
code it doesn't look like it but I hope I'm wrong. It'd be awesome to 
support the many more specific index options without using run_sql.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/3574d906-4ef0-4dd8-a2ea-8b70d470cfc7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Google Summer of Code Updates - Class based indexes

[akki]

This week's work on previous PR:

   - Change in signature of Index class
   - Write tests for the Index class
   - Address all issues on PR

The previous PR has been merged. Now indexes can be added using the AddIndex 
migration 
.


The next checkpoint is to add support for using Index classes from model's 
Meta class. Corresponding ticket - #26808 
. The code is almost ready, I 
will be sending a pull request soon.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/287c3128-5752-43bc-a68e-97ecf1ef1468%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Django Model Fields a repr_output property, to include a field in the string representation of an object

[Tim Graham]

A new model field option doesn't seem necessary. I think a cleaner solution 
would be something like a decorator that takes a list of fields, e.g.

@repr_fields('first_name', 'last_name')
class Person(...):
 ...

This doesn't need to live in Django itself though.

On Thursday, June 23, 2016 at 7:30:50 PM UTC-4, Ben Friedland wrote:
>
> Has a feature like this ever been considered? 
>
> If a model has no __unicode__, __str__ or __repr__ representation, then 
> maybe it could devise a string representation by collecting fields which 
> have this value set to True. 
>
> Example:
>
> Without the feature:
>
> class Person(models.Model):
>  first_name = models.CharField(max_length=50)
>  last_name = models.CharField(max_length=50)
>
> >>> person = Person(first_name='Ben', last_name='Friedland')
> >>> print person
> # fairly useless object representation
>
>
> This feature would work something like: 
>
> class Person(models.Model):
>  first_name = models.CharField(max_length=50, *repr_output=True*)
>  last_name = models.CharField(max_length=50, *repr_output=True*)
>
> >>> person = Person(first_name='Ben', last_name='Friedland')
> >>> print person
># includes fields 
> specified via repr_output=True
>
> If this would be useful I'd be happy to formally create an issue and even 
> implement the feature. 
>
> Thanks!
>
> Ben Friedland
> www.bugben.com
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/b018a812-c773-4766-97fe-f15969592fb2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: change commit message format to present tense?

[Daniele Procida]

On Mon, Jun 27, 2016, Reinout van Rees  wrote:

>Op 26-06-16 om 05:31 schreef Kevin Christopher Henry:
>> If anyone's put off by the hectoring tone of the imperative mood, it
>> might be better to think of it as the indicative mood. That is:
>>
>> (This will) "add password validation to prevent the usage of...".
>>
>> rather than
>>
>> (You must) "add password validation to prevent the usage of..."!
>
>"It might be better to think of it as...": it is exactly this extra 
>thinkwork that everyone reading the messages has to do. We write it once 
>and read it many times: what should we optimize for?
>In our source code, the answer is clearly that you should optimize for 
>readability.
>Why is it suddenly different for commit messages?

I'm inclined to agree with Reinout that a descriptive message makes more sense 
and is easier to understand than one written in the form of an instruction.

Why would a log record be written as an imperative?

"Updated gnools" tells you what has happened (and even "Updates gnools" is a 
label saying what the thing it's attached to will do).

"Update gnools" would make sense as the label for an interface button, but not 
really as a record in a log.

Apart from consistency with Git's own messages, I don't really see the 
advantage.

Daniele

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/20160627115539.95936983%40mail.wservices.ch.
For more options, visit https://groups.google.com/d/optout.

Re: Adding a database-agnostic JSONField into Django

[Brad Jasper]

Current maintainer of https://github.com/bradjasper/django-jsonfield here
and I would definitely support this.

-Brad

On Thu, Jun 23, 2016 at 9:28 AM, Aymeric Augustin <
aymeric.augus...@polytechnique.org> wrote:

> Hello,
>
> On 23 Jun 2016, at 15:40, Tim Graham  wrote:
> > Marc said, "I'm happy for JSONField to be made a core field on the
> condition that it's underlying support is more than a text blob on all our
> main databases. It sounds like this will soon be the case.”
>
>
> In order to discourage people from using JSONField on databases where it
> cannot be implemented efficiently i.e. not with a text blob, the best
> solution may be to add it to Django and raise a warning when it’s misused.
>
> I’m not sure we can do that with a check, though, as we can’t introspect
> database routing logic (without adding a disproportionate amount of
> complexity). It would have to be a runtime warning.
>
> --
> Aymeric.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAAmjBRJKDDT%2BAdDhbGqATChS%2BF6POK9X0fcQmQFNq33Zc7c29A%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: change commit message format to present tense?

[Raphael Hertzog]

Hi,

Le lundi 27 juin 2016, Reinout van Rees a écrit :
> > (You must) "add password validation to prevent the usage of..."!
> 
> "It might be better to think of it as...": it is exactly this extra
> thinkwork that everyone reading the messages has to do. We write it once and
> read it many times: what should we optimize for?
> In our source code, the answer is clearly that you should optimize for
> readability.
> Why is it suddenly different for commit messages?

The first persons reading the commit messages are the persons
reviewing the pull requests and for them, it is quite natural to see
the commit message as "(If merged, this will) fix foo".

Even for someone reading the commits after they have been merged,
when they are clearly history, it's not particularly complicated
to read them as "(This commit did) fix foo".

While this looks like some extra thinkwork, I don't think it's
particularly problematic, and it's defininetely not on the scale
of unreadable code. Whatever the tense used, if the commit message
has the required information, the reader will figure it out.

I have no special preference, but this thread pointed me towards some
widely agreed convention and I'm thankful for that since I tend to always
hesitate when writing a commit message. Now I will stick to the git
convention.

Cheers,
-- 
Raphaël Hertzog ◈ Writer/Consultant ◈ Debian Developer

Discover the Debian Administrator's Handbook:
→ http://debian-handbook.info/get/

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/20160627102828.GA22531%40home.ouaza.com.
For more options, visit https://groups.google.com/d/optout.

Re: change commit message format to present tense?

[Riccardo Magliocchetti]

Il 27/06/2016 11:49, Reinout van Rees ha scritto:

Op 26-06-16 om 05:31 schreef Kevin Christopher Henry:

If anyone's put off by the hectoring tone of the imperative mood, it
might be better to think of it as the indicative mood. That is:

(This will) "add password validation to prevent the usage of...".

rather than

(You must) "add password validation to prevent the usage of..."!


"It might be better to think of it as...": it is exactly this extra thinkwork
that everyone reading the messages has to do. We write it once and read it many
times: what should we optimize for?
In our source code, the answer is clearly that you should optimize for 
readability.
Why is it suddenly different for commit messages?


I think you are overthinking here :) just look at django commit history vs the 
git or the linux one. Which commit message would you like to when chasing a bug?


They have a far better culture at that to what in my experience have found in 
the python ecosystem. So let's just copy that :)


--
Riccardo Magliocchetti
@rmistaken

--
You received this message because you are subscribed to the Google Groups "Django 
developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/828d95a7-8a3d-df99-33c0-b27ef8d0add9%40gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: change commit message format to present tense?

[Reinout van Rees]

Op 26-06-16 om 05:31 schreef Kevin Christopher Henry:

If anyone's put off by the hectoring tone of the imperative mood, it
might be better to think of it as the indicative mood. That is:

(This will) "add password validation to prevent the usage of...".

rather than

(You must) "add password validation to prevent the usage of..."!


"It might be better to think of it as...": it is exactly this extra 
thinkwork that everyone reading the messages has to do. We write it once 
and read it many times: what should we optimize for?
In our source code, the answer is clearly that you should optimize for 
readability.

Why is it suddenly different for commit messages?

Git will probably be replaced by an even better tool in 5 years' time 
(just like svn before that and cvs before): will we go back to the 
original proper English then?



In English they're usually expressed the same way, but not so in other
languages. Anecdotally, I saw someone comment that in Portuguese the two
are different and that the indicative, but not the imperative, would
make sense for a commit message. Reinout, I'm curious if that
distinction would make a difference in your native language?


In Dutch, the imperative and indicative are different.
But that would be better solved by using the indicative also in English, 
right?


- "Adds password validation to prevent..."

The changelog will now look just like installation instructions:

- Install django using `pip install django`.

- Add a database.

- Run `python manage.py `





Reinout

--
Reinout van Rees  http://reinout.vanrees.org/
rein...@vanrees.org   http://www.nelen-schuurmans.nl/
"Learning history by destroying artifacts is a time-honored atrocity"

--
You received this message because you are subscribed to the Google Groups "Django 
developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/nkqsr9%24oh2%241%40ger.gmane.org.
For more options, visit https://groups.google.com/d/optout.