Re: [Django] #34968: MultiPartParser silent large header fields size failures

[Django]

#34968: MultiPartParser silent large header fields size failures
-+-
 Reporter:  Standa Opichal   |Owner:  Standa
 Type:   |  Opichal
  Cleanup/optimization   |   Status:  closed
Component:  HTTP handling|  Version:  4.2
 Severity:  Normal   |   Resolution:  fixed
 Keywords:   | Triage Stage:  Ready for
 |  checkin
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by Mariusz Felisiak ):

 * status:  assigned => closed
 * resolution:   => fixed


Comment:

 In [changeset:"1c6e8ec4ed6d9c374161eda965160e4782c7d71e" 1c6e8ec]:
 {{{
 #!CommitTicketReference repository=""
 revision="1c6e8ec4ed6d9c374161eda965160e4782c7d71e"
 Fixed #34968 -- Made multipart parsing of headers raise an error on too
 long headers.

 This also allow customizing the maximum size of headers via
 MAX_TOTAL_HEADER_SIZE.
 }}}

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/0107018c012ac350-bf6bbe65-2f19-4458-bb0f-c8eed95f6eae-00%40eu-central-1.amazonses.com.

Re: [Django] #34968: MultiPartParser silent large header fields size failures

[Django]

#34968: MultiPartParser silent large header fields size failures
-+-
 Reporter:  Standa Opichal   |Owner:  Standa
 Type:   |  Opichal
  Cleanup/optimization   |   Status:  assigned
Component:  HTTP handling|  Version:  4.2
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:  Ready for
 |  checkin
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by Mariusz Felisiak):

 * needs_better_patch:  1 => 0
 * stage:  Accepted => Ready for checkin


-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/0107018c00950489-f5474d7e-e4da-4ece-b4c2-d45d91840212-00%40eu-central-1.amazonses.com.

Re: [Django] #34968: MultiPartParser silent large header fields size failures

[Django]

#34968: MultiPartParser silent large header fields size failures
-+-
 Reporter:  Standa Opichal   |Owner:  Standa
 Type:   |  Opichal
  Cleanup/optimization   |   Status:  assigned
Component:  HTTP handling|  Version:  4.2
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:  Accepted
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  1
Easy pickings:  0|UI/UX:  0
-+-

Comment (by Standa Opichal):

 > Can it create a DoS vector attack?

 If the limit is changed to be higher the amount of memory necessary to
 parse each message part is going to double and it would also extend the
 time to process as it tries to start with 1024 and doubles the
 `header_end` lookahead chunk every time it doesn't find any.

 The PR has been modified to stay on previous 1024 bytes with a module
 level constant so the change by itself doesn't pose a threat.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/0107018bccdc730a-11065f45-0c5f-4349-bd5a-4d2023019456-00%40eu-central-1.amazonses.com.

Re: [Django] #34968: MultiPartParser silent large header fields size failures

[Django]

#34968: MultiPartParser silent large header fields size failures
-+-
 Reporter:  Standa Opichal   |Owner:  Standa
 Type:   |  Opichal
  Cleanup/optimization   |   Status:  assigned
Component:  HTTP handling|  Version:  4.2
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:  Accepted
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  1
Easy pickings:  0|UI/UX:  0
-+-
Changes (by Mariusz Felisiak):

 * owner:  nobody => Standa Opichal
 * needs_better_patch:  0 => 1
 * type:  Bug => Cleanup/optimization
 * status:  new => assigned
 * stage:  Unreviewed => Accepted


-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/0107018bc8a83c64-6375bc57-8eaa-43de-8509-c65c9b4afa7f-00%40eu-central-1.amazonses.com.

Re: [Django] #34968: MultiPartParser silent large header fields size failures

[Django]

#34968: MultiPartParser silent large header fields size failures
---+--
 Reporter:  opichals   |Owner:  nobody
 Type:  Bug|   Status:  new
Component:  HTTP handling  |  Version:  4.2
 Severity:  Normal |   Resolution:
 Keywords: | Triage Stage:  Unreviewed
Has patch:  1  |  Needs documentation:  0
  Needs tests:  0  |  Patch needs improvement:  0
Easy pickings:  0  |UI/UX:  0
---+--

Comment (by opichals):

 PR updated

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/0107018bc881942f-9bf50221-599e-4878-a82a-528c6759b558-00%40eu-central-1.amazonses.com.

Re: [Django] #34968: MultiPartParser silent large header fields size failures

[Django]

#34968: MultiPartParser silent large header fields size failures
---+--
 Reporter:  opichals   |Owner:  nobody
 Type:  Bug|   Status:  new
Component:  HTTP handling  |  Version:  4.2
 Severity:  Normal |   Resolution:
 Keywords: | Triage Stage:  Unreviewed
Has patch:  1  |  Needs documentation:  0
  Needs tests:  0  |  Patch needs improvement:  0
Easy pickings:  0  |UI/UX:  0
---+--

Comment (by opichals):

 > I wonder how niche your use case is as it has worked this way since the
 beginning (d725cc9734272f867d41f7236235c28b3931a1b2).

 Indeed. We have seen it in production where our client had tried to upload
 files using Postman which includes also the unicode version of Content-
 Disposition filename which was more than 240 characters long effectively
 doubling the size of the header line itself which made it fail:
 {{{
 Content-Disposition: form-data; name="content"; filename="test.txt"
 filename*=UTF-8'test.txt'
 }}}

 > Maybe we could use a module constant for this 樂 e.g.
 django.http.multipartparser.MAX_HTTP_HEADER_LENGTH and set it initially to
 1024.

 Of course, going to adjust the PR.

 The name you're proposing seems like it could be confused with a single
 header line length limit.
 What about `django.http.multipartparser.MAX_TOTAL_HEADER_SIZE` (taken from
 https://github.com/openstack-archive/deb-python-
 eventlet/blob/master/eventlet/wsgi.py and also
 https://support.oracle.com/knowledge/Middleware/2302288_1.html)?

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/0107018bc87c0d18-3a1f490a-f2e1-4d77-9572-ab951c524d80-00%40eu-central-1.amazonses.com.

Re: [Django] #34968: MultiPartParser silent large header fields size failures

[Django]

#34968: MultiPartParser silent large header fields size failures
---+--
 Reporter:  opichals   |Owner:  nobody
 Type:  Bug|   Status:  new
Component:  HTTP handling  |  Version:  4.2
 Severity:  Normal |   Resolution:
 Keywords: | Triage Stage:  Unreviewed
Has patch:  1  |  Needs documentation:  0
  Needs tests:  0  |  Patch needs improvement:  0
Easy pickings:  0  |UI/UX:  0
---+--

Comment (by Mariusz Felisiak):

 Thanks for the report. I wonder how niche your use case is as it has
 worked this way since the beginning
 (d725cc9734272f867d41f7236235c28b3931a1b2). Can it create a DoS vector
 attack? Maybe we could use a module constant for this 樂 e.g.
 `django.http.multipartparser.MAX_HTTP_HEADER_LENGTH` and set it initially
 to 1024.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/0107018bc87312f2-982e6c46-7546-493c-93f3-e28c93129e44-00%40eu-central-1.amazonses.com.

Re: [Django] #34968: MultiPartParser silent large header fields size failures

[Django]

#34968: MultiPartParser silent large header fields size failures
---+--
 Reporter:  opichals   |Owner:  nobody
 Type:  Bug|   Status:  new
Component:  HTTP handling  |  Version:  4.2
 Severity:  Normal |   Resolution:
 Keywords: | Triage Stage:  Unreviewed
Has patch:  1  |  Needs documentation:  0
  Needs tests:  0  |  Patch needs improvement:  0
Easy pickings:  0  |UI/UX:  0
---+--
Description changed by opichals:

Old description:

> The `MultiPartParser` silently ignores parts of which the http header
> fields exceed 1024 bytes.
>
> This is caused by the 1024 value being hardcoded here
> https://github.com/django/django/blob/main/django/http/multipartparser.py#L743
>
> Here is a common http header fields limits across popular web servers
> (from https://stackoverflow.com/a/60623751/2448773):
>  * Apache - 8K
>  * Nginx - 4K-8K
>  * IIS - 8K-16K
>  * Tomcat - 8K – 48K
>  * Node (<13) - 8K; (>13) - 16K
>
> Also reported at https://stackoverflow.com/questions/70572148/django-
> silently-discarding-uploaded-files-with-long-paths

New description:

 The `MultiPartParser` silently ignores parts of which the http header
 fields exceed 1024 bytes. This causes file uploads to 'ignore' the
 attached file without receiving any type of error or exception.

 This is caused by the 1024 value being hardcoded here
 https://github.com/django/django/blob/main/django/http/multipartparser.py#L743

 Here is a common http header fields limits across popular web servers
 (from https://stackoverflow.com/a/60623751/2448773):
  * Apache - 8K
  * Nginx - 4K-8K
  * IIS - 8K-16K
  * Tomcat - 8K – 48K
  * Node (<13) - 8K; (>13) - 16K

 Also reported at https://stackoverflow.com/questions/70572148/django-
 silently-discarding-uploaded-files-with-long-paths

--

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/0107018bc871692c-5c24b21e-b693-4249-b38c-7845d7288b86-00%40eu-central-1.amazonses.com.

Re: [Django] #34968: MultiPartParser silent large header fields size failures

[Django]

#34968: MultiPartParser silent large header fields size failures
---+--
 Reporter:  opichals   |Owner:  nobody
 Type:  Bug|   Status:  new
Component:  HTTP handling  |  Version:  4.2
 Severity:  Normal |   Resolution:
 Keywords: | Triage Stage:  Unreviewed
Has patch:  1  |  Needs documentation:  0
  Needs tests:  0  |  Patch needs improvement:  0
Easy pickings:  0  |UI/UX:  0
---+--
Description changed by opichals:

Old description:

> The `MultiPartParser` silently ignores parts of which the http header
> fields exceed 1024 bytes.
>
> This is caused by the 1024 value being hardcoded here
> https://github.com/django/django/blob/main/django/http/multipartparser.py#L743
>
> Also reported at https://stackoverflow.com/questions/70572148/django-
> silently-discarding-uploaded-files-with-long-paths

New description:

 The `MultiPartParser` silently ignores parts of which the http header
 fields exceed 1024 bytes.

 This is caused by the 1024 value being hardcoded here
 https://github.com/django/django/blob/main/django/http/multipartparser.py#L743

 Here is a common http header fields limits across popular web servers
 (from https://stackoverflow.com/a/60623751/2448773):
  * Apache - 8K
  * Nginx - 4K-8K
  * IIS - 8K-16K
  * Tomcat - 8K – 48K
  * Node (<13) - 8K; (>13) - 16K

 Also reported at https://stackoverflow.com/questions/70572148/django-
 silently-discarding-uploaded-files-with-long-paths

--

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/0107018bc86dc64d-02057485-2b9a-4799-a534-b8914f1a5a40-00%40eu-central-1.amazonses.com.