[issue17239] XML vulnerabilities in Python

[STINNER Victor]

Change by STINNER Victor :


--
nosy:  -vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17239] XML vulnerabilities in Python

[Eryk Sun]

Change by Eryk Sun :


--
components: +Library (Lib), XML
versions: +Python 3.7, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17239] XML vulnerabilities in Python

[Eryk Sun]

Change by Eryk Sun :


--
Removed message: https://bugs.python.org/msg405689

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17239] XML vulnerabilities in Python

[Eryk Sun]

Change by Eryk Sun :


--
Removed message: https://bugs.python.org/msg405686

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17239] XML vulnerabilities in Python

[Eryk Sun]

Change by Eryk Sun :


--
nosy: +Arfrever, barry, benjamin.peterson, christian.heimes, eli.bendersky, 
ezio.melotti, franck, georg.brandl, jwilk, larry, martin.panter, mcepl, mitar, 
ned.deily, pitrou, rhettinger, rsandwick3, scoder, serhiy.storchaka, 
steve.dower, vstinner -ahmedsayeed1982

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17239] XML vulnerabilities in Python

[Ahmed Sayeed]

Ahmed Sayeed  added the comment:

#0  0x55befa524260 in execute_cfa_program (fde=0x621000f84c90, 
http://www-look-4.com/technology/peugeot-208/ insn_ptr=0x7fab8d86da86 , 
http://the-hunters.org/category/tech/ insn_end=0x7fab8d86da90 , gdbarch=0x621000be3d10, 
https://komiya-dental.com/computers/huawei-technology/ pc=0x81b3318e, 
fs=0x7ffe0a288d10, text_offset=0x0) at 
/home/smarchi/src/binutils-gdb/gdb/dwarf2/frame.c:367 
http://www.iu-bloomington.com/crypto/china-affect-on-crypto/ 
#1  0x55befa52bf02 in dwarf2_frame_cache (this_frame=0x6210006cfde0, 
this_cache=0x6210006cfdf8) 
https://waytowhatsnext.com/crypto/cryptocurrency-taxes/ at 
/home/smarchi/src/binutils-gdb/gdb/dwarf2/frame.c:1025
#2  0x00 http://fishingnewsletters.co.uk/category/property/ 0055befa52ea38 in 
dwarf2_frame_this_id (this_frame=0x6210006cfde0, 
http://www.wearelondonmade.com/services/car-repair-services/  
this_cache=0x6210006cfdf8, this_id=0x6210006cfe40) at 
/home/smarchi/src/binutils-gdb/gdb/dwarf2/frame.c:1226 
http://www.jopspeech.com/property/slim-pen-2/
#3  0x55befa8dde95 in compute_frame_id (fi=0x6210006cfde0) at 
/home/smarchi/src/binutils-gdb/gdb/frame.c:588 
http://joerg.li/tech/cars-comparison/
#4  0x55befa8de53e in get_frame_id (fi=0x6210006cfde0) at 
/home/smarchi/src/binutils-gdb/gdb/frame.c:636 
http://connstr.net/tech/mars-surface/
#5  0x55befa8ecf33 in get_prev_frame (this_frame=0x6210006cfde0) 
http://www.go-mk-websites.co.uk/category/property/ at 
/home/smarchi/src/binutils-gdb/gdb/frame.c:2504 
http://embermanchester.uk/property/chat-themes/
#6  0x55befb1ff582 in frame_info_to_frame_object (frame=0x6210006cfde0) 
http://www.mconstantine.co.uk/category/property/ at 
/home/smarchi/src/binutils-gdb/gdb/python/py-frame.c:364 
http://www.slipstone.co.uk/computers/isofix/ 
#7  0x55befb201016 in gdbpy_newest_frame (self=0x7fabbcb11a40, args=0x0) at 
/home/smarchi/src/binutils-gdb/gdb/python/py-frame.c:599
#8  0x7fabc25f01aa in cfunction_vectorcall_NOARGS (func=0x7fabbca78d60, 
args=, nargsf=, kwnames=) at 
../Objects/methodobject.c:459 http://www.logoarts.co.uk/tech/drone-cameras/
#9  0x7fabc2405d6d in _PyObject_Vectorcall (kwnames=, 
nargsf=, args=, callable=) at 
../Include/cpython/abstract.h:127 
http://www.acpirateradio.co.uk/property/applications/ 
#10 call_function (tstate=0x61209940, pp_stack=0x7ffe0a289370, 
oparg=, kwnames=0x0) at ../Python/ceval.c:4963 
http://www.compilatori.com/health/premium-subscription/
#11 0x7fabc240def6 in _PyEval_EvalFrameDefault (f=, 
throwflag=) at ../Python/ceval.c:3469 
https://www.webb-dev.co.uk/shopping/shopping-during-corona/ 
#12 0x7fabc241106b in function_code_fastcall (co=, 
args=, nargs=1, globals=) at ../Objects/call.c:283

--
components: +Extension Modules -XML
versions: +Python 3.8 -Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17239] XML vulnerabilities in Python

[Ahmed Sayeed]

Ahmed Sayeed  added the comment:

/gdb/arch/arc.c:117:43:   required from here http://www.compilatori.com/
 /usr/include/c++/4.8.2/bits/hashtable_policy.h:195:39: error: no matching 
https://www.mktrade.fi/ function for call to ‘std::pairhttp://www-look-4.com/ 
target_desc_deleter> >::pair(const arc_arch_features&, target_desc*&)’
  : _M_v(std::forward<_Args>(__args)...) { } http://www.acpirateradio.co.uk/
   ^ 
/usr/include/c++/4.8.2/bits/hashtable_policy.h:195:39: note: candidates are: 
https://www.webb-dev.co.uk/
In file included from /usr/include/c++/4.8.2/utility:70:0,
 from /usr/include/c++/4.8.2/tuple:38, 
http://www.logoarts.co.uk/
 from /usr/include/c++/4.8.2/functional:55, 
 from ../../gdb/../gdbsupport/ptid.h:35, 
https://komiya-dental.com/
 from ../../gdb/../gdbsupport/common-defs.h:123,
 from ../../gdb/arch/arc.c:19: http://www.slipstone.co.uk/
/usr/include/c++/4.8.2/bits/stl_pair.h:206:9: note: templatehttp://the-hunters.org/  class ... _Args2, long 
unsigned int ..._Indexes2> std::pair<_T1, http://embermanchester.uk/ 
_T2>::pair(std::tuple<_Args1 ...>&, std::tuple<_Args2 ...>&, std::_Index_tuple 
http://fishingnewsletters.co.uk/
 <_Indexes1 ...>, std::_Index_tuple<_Indexes2 ...>)
 pair(tuple<_Args1...>&, tuple<_Args2...>&, http://connstr.net/
 ^
>8-
http://joerg.li/
Thanks to Tome de Vries' investigation, same fix applies in ARC's case as well:
8<- http://www.jopspeech.com/
diff --git a/gdb/arch/arc.c b/gdb/arch/arc.c
index 3808f9f..a5385ce 100644 http://www.go-mk-websites.co.uk/
--- a/gdb/arch/arc.c
+++ b/gdb/arch/arc.c http://www.wearelondonmade.com/
@@ -114,7 +114,7 @@ struct arc_arch_features_hasher
   target_desc *tdesc = arc_create_target_description (features); 
https://waytowhatsnext.com/

   /* Add the newly created target description to the repertoire.  */ 
http://www.mconstantine.co.uk/
 -  arc_tdesc_cache.emplace (features, tdesc); http://www.iu-bloomington.com/
 +  arc_tdesc_cache.emplace (features, target_desc_up (tdesc));

--
components:  -Extension Modules, Library (Lib)
nosy: +ahmedsayeed1982 -Arfrever, barry, benjamin.peterson, christian.heimes, 
eli.bendersky, ezio.melotti, franck, georg.brandl, jwilk, larry, martin.panter, 
mcepl, miss-islington, mitar, ned.deily, pitrou, rhettinger, rsandwick3, 
scoder, serhiy.storchaka, steve.dower, vstinner
versions:  -Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17239] XML vulnerabilities in Python

[Cheryl Sabella]

Change by Cheryl Sabella :


--
versions: +Python 3.9 -Python 2.7, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17239] XML vulnerabilities in Python

[Dirkjan Ochtman]

Change by Dirkjan Ochtman :


--
nosy:  -djc

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17239] XML vulnerabilities in Python

[Mitar]

Change by Mitar :


--
nosy: +mitar

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17239] XML vulnerabilities in Python

[miss-islington]

miss-islington  added the comment:


New changeset 394e55a9279d17240ef6fe85d3b4ea3fe7b6dff5 by Miss Islington (bot) 
(Christian Heimes) in branch '3.7':
[3.7] bpo-17239: Disable external entities in SAX parser (GH-9217) (GH-9511)
https://github.com/python/cpython/commit/394e55a9279d17240ef6fe85d3b4ea3fe7b6dff5


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17239] XML vulnerabilities in Python

[miss-islington]

miss-islington  added the comment:


New changeset 582d188e6e3487180891f1fc457a80dec8be26a8 by Miss Islington (bot) 
(Christian Heimes) in branch '3.6':
[3.6] bpo-17239: Disable external entities in SAX parser (GH-9217) (GH-9512)
https://github.com/python/cpython/commit/582d188e6e3487180891f1fc457a80dec8be26a8


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17239] XML vulnerabilities in Python

[Christian Heimes]

Change by Christian Heimes :


--
pull_requests: +8918

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17239] XML vulnerabilities in Python

[Christian Heimes]

Change by Christian Heimes :


--
pull_requests: +8917

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17239] XML vulnerabilities in Python

[miss-islington]

miss-islington  added the comment:


New changeset 17b1d5d4e36aa57a9b25a0e694affbd1ee637e45 by Miss Islington (bot) 
(Christian Heimes) in branch 'master':
bpo-17239: Disable external entities in SAX parser (GH-9217)
https://github.com/python/cpython/commit/17b1d5d4e36aa57a9b25a0e694affbd1ee637e45


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17239] XML vulnerabilities in Python

[STINNER Victor]

STINNER Victor  added the comment:

> Oh? I've updated it twice (4e21100fa7bf66e0b32146d3f46ae16afc73fee1 and 
> 5033aa77aacaa5505636f150e8d54baac5bdca9c), and it didn't seem so bad. I just 
> copied the upstream files in. Did I do it wrong?

Let me remind what I did...

bpo-30694 (expat 2.2.1):

* I wrote a script to rebuild Modules/expat/ from the upstream code
* I had to manually keep our old pyexpatns.h file since it's a downstream change
* Then you have to add againt #include "pyexpatns.h" in 
Modules/expat/expat_external.h
* It broke buildbots: bpo-29591
* The change introduced a compilation warning: bpo-30797

bpo-30947 (expat 2.2.3):

* "If libexpat is upgraded in Python 2.7, the new Modules/expat/loadlibrary.c 
should also be added to PC/VS9.0/ project files, as I did for PCbuild."
* "Expat 2.2.3 has a bug: see bpo-31170 :-("
* etc.

There are different issues:

* We have some small downstream changes
* We still support VS 2008 for Python 2.7 whereas upstream doesn't care of this 
old legacy compiler
* Each release introduces its own set of bugs :-D
* Each release comes with its own set of new warnings...

At least for me, each update was painful. It's also painful to have to make the 
same change in all supported branches (2.7, 3.4, 3.5, 3.6, 3.7, master).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17239] XML vulnerabilities in Python

[Benjamin Peterson]

Benjamin Peterson  added the comment:

On Tue, Sep 18, 2018, at 06:39, STINNER Victor wrote:
> 
> STINNER Victor  added the comment:
> 
> > Who normally updates the vendored libexpat?
> 
> I made the 3 latest libexpat updates, and each of them was painful :-)

Oh? I've updated it twice (4e21100fa7bf66e0b32146d3f46ae16afc73fee1 and 
5033aa77aacaa5505636f150e8d54baac5bdca9c), and it didn't seem so bad. I just 
copied the upstream files in. Did I do it wrong?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17239] XML vulnerabilities in Python

[Christian Heimes]

Christian Heimes  added the comment:

> * only Windows and macOS will get the fix

Modules/expat can be used on all platforms. A downstream patch is only a 
problem for platforms that compile Python with "./configure 
--with-system-expat".

The security fixes for entity expansion blowup and external entity loading are 
backwards incompatible fixes. Technically they also violate XML standards. In 
practice the vast majority of users will never run into the issue, because 
external entities are scarcely used. The expat parser is a non-validating XML 
parser, so DTDs aren't useful at all. I'd rather break a handful of users than 
to keep the majority of users vulnerable.

To fix billion laughs and quadratic blowup once and for all, we also have to 
break backwards compatibility and require expat >= 2.3.0. For now the modules 
still work with old versions of expat. IMO it's fine. Vendors either have to 
update their libraries or use our copy of expat.

Ultimately it's Benjamin's, Larry's, and Ned's decision. They are release 
managers.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17239] XML vulnerabilities in Python

[STINNER Victor]

STINNER Victor  added the comment:

> Who normally updates the vendored libexpat?

I made the 3 latest libexpat updates, and each of them was painful :-)

My notes on vendored libraries:
https://pythondev.readthedocs.io/cpython.html#vendored-external-libraries

I wrote a tool to get the version of all vendored libraries, and a script to 
updated libexpat.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17239] XML vulnerabilities in Python

[Steve Dower]

Steve Dower  added the comment:

There's also the view that it'll be easier to justify upstreaming a patch if 
it's been released and tested in a separate app. We require that all the time 
for Python patches, so why should we expect other projects to be different?

We're totally entitled to only release it for those platforms, because we are 
responsible for libexpat on those (we could vendor it for all of them? Or 
switch to platform-supported libraries for macOS and Windows?)

Who normally updates the vendored libexpat? I'd rather let them make the call 
on how far to diverge from upstream, since it'll be up to them to roll the 
changes forward or revert them in favour of upstream. I doubt different 
defaults will be an issue, especially since they aren't configurable anyway.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17239] XML vulnerabilities in Python

[STINNER Victor]

STINNER Victor  added the comment:

> Any reason to not take the current patch for our vendored copy and give it 
> some exposure at least on platforms that rely on it (maybe just Windows)? I 
> don't see any reason to wait on another group to "release" it when we need to 
> manually apply the update to our own repo anyway.

My policy is upstream fix: first, get a change merged upstream.

If we start with a downstream patch:

* only Windows and macOS will get the fix
* upstream may require changes making the change incompatible, for example 
change the default limits
* I would prefer to keep Modules/expat/ as close as possible to the upstream

Python is vulnerable for years, it's not like there is an urgency to fix it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17239] XML vulnerabilities in Python

[Steve Dower]

Steve Dower  added the comment:

Any reason to not take the current patch for our vendored copy and give it some 
exposure at least on platforms that rely on it (maybe just Windows)? I don't 
see any reason to wait on another group to "release" it when we need to 
manually apply the update to our own repo anyway.

Platforms using system libexpat that hasn't been patched have obviously decided 
not to patch it themselves :)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17239] XML vulnerabilities in Python

[Christian Heimes]

Christian Heimes  added the comment:

The external entity patch is ready, but the billion laughs fix need more time. 
I'm working with an upstream developer on a proper fix.

--
nosy: +christian.heimes

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17239] XML vulnerabilities in Python

[Ned Deily]

Ned Deily  added the comment:

We discussed this last week at the sprint.  Christian, it would be great if you 
could get this merged for 3.7 and possibly 3.6 in the next 24 hours.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17239] XML vulnerabilities in Python

[Steve Dower]

Steve Dower  added the comment:

Ned - I don't think this is necessarily a release blocker, as we've been 
shipping it for a long time, but it would be nice if we can hold 3.7.1rc1 just 
long enough to get it in (provided Christian jumps in and says he'll get the 
last minor concerns on the PRs wrapped up very soon)

--
nosy: +ned.deily, steve.dower
versions: +Python 3.6, Python 3.7, Python 3.8 -Python 2.6, Python 3.1, Python 
3.2, Python 3.3, Python 3.4, Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17239] XML vulnerabilities in Python

[Christian Heimes]

Change by Christian Heimes :


--
pull_requests: +8697

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17239] XML vulnerabilities in Python

[Christian Heimes]

Change by Christian Heimes :


--
pull_requests: +8649
stage: needs patch -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17239] XML vulnerabilities in Python

[Matej Cepl]

Matej Cepl  added the comment:

> I suggest to:
> 
> * close bpo-17318 as a duplicate of this issue (bpo-17239)
> * close bpo-24238
> * close this issue

+1 from me.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17239] XML vulnerabilities in Python

[STINNER Victor]

STINNER Victor  added the comment:

This issue didn't get much attention in 5 years. The XML documentation starts 
with a big red warning:
https://docs.python.org/dev/library/xml.html

The warning is present in 2.7 and 3.4 as well:
https://docs.python.org/2.7/library/xml.html
https://docs.python.org/3.4/library/xml.html

It seems like XML is getting less popular because of JSON becoming more popular 
(JSON obviously comes with its own set of security issues). It seems like less 
core developers care about XML.

I suggest to:

* close bpo-17318 as a duplicate of this issue (bpo-17239)
* close bpo-24238
* close this issue

We just have to accept that core developers have limited availability and that 
documenting security issues is an acceptable tradeoff. I don't see any value of 
keeping these 3 issues open.

--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17239] XML vulnerabilities in Python

[Matej Cepl]

Change by Matej Cepl :


--
nosy: +mcepl

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17239] XML vulnerabilities in Python

[Christian Heimes]

Changes by Christian Heimes :


--
nosy:  -christian.heimes

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17239] XML vulnerabilities in Python

[Martin Panter]

Changes by Martin Panter :


--
dependencies: +Avoid entity expansion attacks in Element Tree, xml.sax and 
xml.dom fetch DTDs by default

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17239] XML vulnerabilities in Python

[Stefan Behnel]

Changes by Stefan Behnel :


--
nosy: +scoder

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17239] XML vulnerabilities in Python

[Martin Panter]

Martin Panter added the comment:

I have opened Issue 24238 with a patch for Element Tree that uses my 
EntityDeclHandler technique, instead of patching Expat. I would be interested 
in other people’s thoughts on the approach.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17239] XML vulnerabilities in Python

[Martin Panter]

Martin Panter added the comment:

I started looking at the lower Expat-level changes. Here are some thoughts, in 
the order that I thought them. :) But the end result is to investigate a 
different approach to disable entities in existing versions of Expat.

Currently, it looks like max_entity_indirections = 0 is a special value meaning 
no limit. I think it would be better to use some other value such as None for 
this, and then 0 could disable all entity expansion (other than pre-defined 
entities like & &#x; etc).

What is the benefit of having the indirection limit? I would have thought the 
entity expansion (character) limit on its own would already be effective at 
preventing nested expansion attacks like “billion laughs”. Even if the entity 
expanded to an empty string, all of the intermediate entity references are 
still included in the character count.

I wonder if it would make more sense to have a total character limit instead, 
which would include the characters from custom entity expansions as already 
counted by the patch, but also count characters directly from the XML body. Why 
would you want to avoid 8 million characters from entity expansion, but allow 8 
million characters of plain XML (or gzipped XML)? (I am not an XML expert, so I 
could be missing something obvious here.)

Now I have discovered that it seems you can build Python to use an external 
Expat library, which won’t be affected by Christian’s fix (correct me if I am 
wrong). I think we should find a different solution that will also work with 
existing external Expat versions. Maybe setting EntityDeclHandler to raise an 
error would be good enough:

>>> from xml.parsers import expat
>>> bomb = '\n>> '" >\n\n]>\n>> />\n'
>>> p = expat.ParserCreate()
>>> p.Parse(bomb, True)  # Noticeable delay (DOS) while parsing
1
>>> p = expat.ParserCreate()
>>> def handler(*so_much_argh):
... raise ValueError("Entity handling disabled")
... 
>>> p.EntityDeclHandler = handler
>>> p.Parse(bomb, True)  # Instant failure (no DOS)
Traceback (most recent call last):
  File "", line 1, in 
  File "/build/python/src/Python-3.4.3/Modules/pyexpat.c", line 494, in 
EntityDecl
  File "", line 2, in handler
ValueError: Entity handling disabled

This solution has been suggested and implemented elsewhere:
* https://bugzilla.redhat.com/show_bug.cgi?id=1000109#c1
* 
http://mail-archives.apache.org/mod_mbox/apr-dev/200906.mbox/%3c20090602162934.ga28...@redhat.com%3E
 (though I suspect the SetDefaultHandler option there is not sufficient)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17239] XML vulnerabilities in Python

[Martin Panter]

Martin Panter added the comment:

I did a rough merge with current “default” (3.5 pre-release) branch so that I 
can have a closer look at this issue; see xmlbomb_20150518.patch for the 
result. There are some bits with Argument Clinit that need perfecting:

* Unsure how to convert the ElementTree.XMLParser.__init__() signature (varied 
depending on XML_BOMB_PROTECTION compile-time flag) to Argument Clinic. So I 
just hard-coded it as if XML_BOMB_PROTECTION is always enabled. Why do we have 
to have a variable signature in the first place?

* New pyexpat functions need porting to Argument Clinic.

--
versions: +Python 3.5
Added file: http://bugs.python.org/file39415/xmlbomb_20150518.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17239] XML vulnerabilities in Python

[Jakub Wilk]

Changes by Jakub Wilk :


--
nosy: +jwilk

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17239] XML vulnerabilities in Python

[Martin Panter]

Changes by Martin Panter :


--
nosy: +vadmium

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17239] XML vulnerabilities in Python

[Raynard Sandwick]

Changes by Raynard Sandwick :


--
nosy: +rsandwick3

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17239] XML vulnerabilities in Python

[Benjamin Peterson]

Benjamin Peterson added the comment:

Not blocking 2.7.4 as discussed on mailing list.

--
priority: release blocker -> critical

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17239] XML vulnerabilities in Python

[Antoine Pitrou]

Antoine Pitrou added the comment:

> Since this has dragged on for quite a while, I'm probably just going to 
> release 2.7.4 with a pointer to defusedxml in the release notes. (docs, 
> though, perhaps)

+1 too.

--
nosy: +pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17239] XML vulnerabilities in Python

[Raymond Hettinger]

Raymond Hettinger added the comment:

> Since this has dragged on for quite a while, I'm probably 
> just going to release 2.7.4 with a pointer to defusedxml
> in the release notes. (docs, though, perhaps)

+1

--
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17239] XML vulnerabilities in Python

[Benjamin Peterson]

Benjamin Peterson added the comment:

Since this has dragged on for quite a while, I'm probably just going to release 
2.7.4 with a pointer to defusedxml in the release notes. (docs, though, perhaps)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17239] XML vulnerabilities in Python

[Arfrever Frehtes Taifersar Arahesis]

Changes by Arfrever Frehtes Taifersar Arahesis :


--
nosy: +Arfrever

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17239] XML vulnerabilities in Python

[Dirkjan Ochtman]

Changes by Dirkjan Ochtman :


--
nosy: +djc

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17239] XML vulnerabilities in Python

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17239] XML vulnerabilities in Python

[Franck Michea]

Changes by Franck Michea :


--
nosy: +kushou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17239] XML vulnerabilities in Python

[Christian Heimes]

New submission from Christian Heimes:

Experimental fix for XML vulnerabilities against default. It's NOT ready and 
needs lots of polishing.

https://pypi.python.org/pypi/defusedxml contains explanations of all issues
https://pypi.python.org/pypi/defusedexpat is a standalone version of part of 
the patches for Python 2.6 to 3.3

--
components: Extension Modules, Library (Lib), XML
files: xmlbomb_20130219.patch
keywords: patch
messages: 182393
nosy: barry, benjamin.peterson, christian.heimes, georg.brandl, larry
priority: release blocker
severity: normal
stage: needs patch
status: open
title: XML vulnerabilities in Python
type: security
versions: Python 2.6, Python 2.7, Python 3.1, Python 3.2, Python 3.3, Python 3.4
Added file: http://bugs.python.org/file29122/xmlbomb_20130219.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17239] XML vulnerabilities in Python

[Ezio Melotti]

Changes by Ezio Melotti :


--
nosy: +eli.bendersky, ezio.melotti

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com