[issue32072] Issues with binary plists

[Ned Deily]

Change by Ned Deily :


--
Removed message: https://bugs.python.org/msg342093

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32072] Issues with binary plists

[Ned Deily]

Ned Deily  added the comment:


New changeset c59731d92dc73111d224876f1caa064097aad786 by larryhastings (Serhiy 
Storchaka) in branch '3.4':
[3.4] bpo-32072: Fix issues with binary plists. (GH-4455) (#4658)
https://github.com/python/cpython/commit/c59731d92dc73111d224876f1caa064097aad786


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32072] Issues with binary plists

[Ronald Oussoren]

Ronald Oussoren  added the comment:

@larry: plists are Apple's equivalent to Windows INI files ;-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32072] Issues with binary plists

[Larry Hastings]

Larry Hastings  added the comment:

Thank you for the fix!  I just wish I knew what plists were ;-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32072] Issues with binary plists

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Thanks Larry.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32072] Issues with binary plists

[Larry Hastings]

Larry Hastings  added the comment:


New changeset 43f014d3f12468edf61046f0612edc7660042fd5 by larryhastings (Serhiy 
Storchaka) in branch '3.5':
[3.5] bpo-32072: Fix issues with binary plists. (GH-4455) (#4656)
https://github.com/python/cpython/commit/43f014d3f12468edf61046f0612edc7660042fd5


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32072] Issues with binary plists

[Larry Hastings]

Larry Hastings  added the comment:


New changeset c59731d92dc73111d224876f1caa064097aad786 by larryhastings (Serhiy 
Storchaka) in branch '3.4':
[3.4] bpo-32072: Fix issues with binary plists. (GH-4455) (#4658)
https://github.com/python/cpython/commit/c59731d92dc73111d224876f1caa064097aad786


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32072] Issues with binary plists

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

For example:

a = []
for i in range(22):
a = [a, a]

b = plistlib.dumps(a, fmt=plistlib.FMT_BINARY)

The result is 130 bytes long on patched plistlib. But plistlib.dumps(b) will 
expand to a structure consuming almost a gigabyte of memory on unpatched 
plistlib. Increasing the level of nesting by one will duplicate memory 
consumption, so it is easy to consume all available memory on any computer.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32072] Issues with binary plists

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:


New changeset 8cd31082fba88cf0064590fd3d55b6c1c964f11c by Serhiy Storchaka 
(Miss Islington (bot)) in branch '3.6':
bpo-32072: Fix issues with binary plists. (GH-4455) (#4654)
https://github.com/python/cpython/commit/8cd31082fba88cf0064590fd3d55b6c1c964f11c


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32072] Issues with binary plists

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
pull_requests: +4570

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32072] Issues with binary plists

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
pull_requests: +4568

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32072] Issues with binary plists

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
nosy: +larry

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32072] Issues with binary plists

[Roundup Robot]

Change by Roundup Robot :


--
pull_requests: +4566

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32072] Issues with binary plists

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:


New changeset a897aeeef647259a938a36cb5eb6680c86021c6a by Serhiy Storchaka in 
branch 'master':
bpo-32072: Fix issues with binary plists. (#4455)
https://github.com/python/cpython/commit/a897aeeef647259a938a36cb5eb6680c86021c6a


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32072] Issues with binary plists

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Ronald, could you please make a review? I want to merge this before 3.7.0a3.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32072] Issues with binary plists

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
keywords: +patch
pull_requests: +4393
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32072] Issues with binary plists

[Serhiy Storchaka]

New submission from Serhiy Storchaka :

plistlib creates a new objects when read references instead of using
already read object.

As result it doesn't preserve identity:

>>> import plistlib
>>> a = [['spam']]*2
>>> a[0] is a[1]
True
>>> b = plistlib.loads(plistlib.dumps(a, fmt=plistlib.FMT_BINARY))
>>> b == a
True
>>> b[0] is b[1]
False

And plistlib.loads() is vulnerable to plists containing cyclic
references (as was exposed in issue31897). For example,
plistlib.loads(b'bplist00\xa1\x00\x08\x00\x00\x00\x00\x00\x00\x01\x01\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x0a')
could return a list containing itself, but it is failed with
RecursionError.

plistlib.dumps() preserves reference in the output, but it saves
redundant copies. For example plistlib.dumps([[]]*5,
fmt=plistlib.FMT_BINARY) saves a list containing 5 identical empty
lists, but it saves an empty list 5 times, and only the last copy is
used. The other 4 copies are not referenced and just spent the file
volume and the space of reference numbers. Saving
['spam']*100]*100]*100]*100]*100 will result in a multigigabyte,
while less than a kilobyte would be enough for saving it. Loading
properly saved ['spam']*100]*100]*100]*100]*100 withe the current
plistlib.loads() will cause consuming many gigabytes of memory.

1. The issues with plistlib.dumps() are:
1a) Inefficient saving data with references. This is minor resource usage issue.
1b) Impossibility to save a data with cyclic references. This is a
lack of a feature.

2. The issues with plistlib.loads() are:
2a) Inefficient loading data with references. This can be not just a
resource usage issue, but a security issue. Loading an malicious input
data smaller than 100 byte ([[[...]*2]*2]*2) can cause consuming many
gigabytes of memory.
2b) Impossibility to load a data with cyclic references. This is a
lack of a feature, but can be lesser security issue. Small malicious
input can cause RecursionError. If the recursion limit is set high and
you are unlucky it can cause a stack overflow.

Security issues affect you only when you load plists from untrusted sources.

Adding the proper support of references could be considered a new
feature, but taking to account security issues it should be backported
up to 3.4 when the support of binary plists was added.

--
assignee: serhiy.storchaka
components: Library (Lib)
messages: 306493
nosy: ned.deily, ronaldoussoren, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Issues with binary plists
type: security
versions: Python 3.4, Python 3.5, Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com