[issue26707] plistlib fails to parse bplist with 0x80 UID values
John Lehr added the comment: Based on the format specification pointed to by Serhiy, perhaps this a better patch, correcting size from previous patch submission and treating: 706,707c706,708 < # tokenH == 0x80 is documented as 'UID' and appears to be used for < # keyed-archiving, not in plists. --- > elif tokenH == 0x80: # UID > s = self._get_size(tokenL + 1) > return int.from_bytes(self._fp.read(s), 'big') I have compared output with OS X plutil and plistlib.load() with this patch and the values are identical for UID fields. -- Added file: http://bugs.python.org/file42395/plistlib_uid.diff ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26707> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26707] plistlib fails to parse bplist with 0x80 UID values
John Lehr added the comment: I’m sorry, but the files in which I detected the problem cannot be circulated. I will try to create a test account on Snapchat and generate some test data, but I can’t do this anytime soon. > On Apr 7, 2016, at 12:51 AM, SilentGhost <rep...@bugs.python.org> wrote: > > > SilentGhost added the comment: > > Here is the version of the patch suitable for the Rietveld. John, could you > perhaps provide an example file that uses UID values? > > Also, the code is identical to handling of 0x50 token, perhaps it could be > incorporated into it. > > -- > nosy: +SilentGhost > stage: -> patch review > versions: +Python 3.6 > Added file: http://bugs.python.org/file42390/issue26707.diff > > ___ > Python tracker <rep...@bugs.python.org> > <http://bugs.python.org/issue26707> > ___ -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26707> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26707] plistlib fails to parse bplist with 0x80 UID values
John Lehr added the comment: I’m glad you found it in the Apple specification. I looked, but missed it. I would absolutely defer to you on your assessment of the decoding. > On Apr 7, 2016, at 1:46 AM, Serhiy Storchaka <rep...@bugs.python.org> wrote: > > > Serhiy Storchaka added the comment: > > UID is rather int than bytes. And I would use a special UID type. > > According to Apple's sources [1], the size of UID data is tokenL+1, not > self._get_size(tokenL). > > [1] http://www.opensource.apple.com/source/CF/CF-1153.18/CFBinaryPList.c > > -- > > ___ > Python tracker <rep...@bugs.python.org> > <http://bugs.python.org/issue26707> > ___ -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26707> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26707] plistlib fails to parse bplist with 0x80 UID values
New submission from John Lehr:
libplist raises an invalid file exception on loading properly formed binary
plists containing UID (0x80) values. The binary files were tested for form
with plutil.
Comments at line 706 state the value is defined but not in use in plists, and
the object is not handled. However, I find them frequently in bplists, e.g.,
iOS Snapchat application files. I have attached a proposed patch that I have
tested on these files and can now successfully parse them with the _read_object
method in the _BinaryPlistParser class.
My proposed patch is pasted below for others consideration while waiting for
the issue to be resolved.
706,707c706,708
< # tokenH == 0x80 is documented as 'UID' and appears to be used for
< # keyed-archiving, not in plists.
---
> elif tokenH == 0x80: #UID
> s = self._get_size(tokenL)
> return self._fp.read(s).decode('ascii')
Thanks for your consideration.
--
components: Library (Lib)
files: plistlib_uid.diff
keywords: patch
messages: 262974
nosy: slo.sleuth
priority: normal
severity: normal
status: open
title: plistlib fails to parse bplist with 0x80 UID values
type: crash
versions: Python 3.5
Added file: http://bugs.python.org/file42388/plistlib_uid.diff
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue26707>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21538] plistlib unable to load iOS7 Safari History.plist
John Lehr added the comment: Serhiy, I applied the patch and can confirm this is a fix (three previous failing loads of History.plists from three different devices now load successfully). Thank you, I would not likely have identified the issue myself. On Mon, May 19, 2014 at 11:08 PM, Serhiy Storchaka rep...@bugs.python.orgwrote: Changes by Serhiy Storchaka storch...@gmail.com: Removed file: http://bugs.python.org/file35299/plistlib_read_refs.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21538 ___ -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21538 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21538] plistlib unable to load iOS7 Safari History.plist
John Lehr added the comment: Patch plistlib_read_refs_2.patch corrects load error for iOS Safari History.plist. -- resolution: - fixed status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21538 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21538] plistlib unable to load iOS7 Safari History.plist
New submission from John Lehr:
plistlib has key error when attempting to load the iOS7
var/mobile/Applications/com.apple.mobilesafari/Library/Safari/History.plist.
No detected issues with other iOS7 plists.
In [8]: with
open('cases/vitale_test_iphone/unback/AppDomain-com.apple.mobilesafari/Library/Safari/History.plist',
'rb') as f:
...: plist = plistlib.lo
plistlib.load plistlib.loads
...: plist = plistlib.load(f)
...:
---
KeyError Traceback (most recent call last)
ipython-input-8-7824ac03848c in module()
1 with
open('cases/vitale_test_iphone/unback/AppDomain-com.apple.mobilesafari/Library/Safari/History.plist',
'rb') as f:
2 plist = plistlib.load(f)
3
/usr/lib/python3.4/plistlib.py in load(fp, fmt, use_builtin_types, dict_type)
993 p = _FORMATS[fmt]['parser'](use_builtin_types=use_builtin_types)
994
-- 995 return p.parse(fp)
996
997
/usr/lib/python3.4/plistlib.py in parse(self, fp)
620 ) = struct.unpack('6xBBQQQ', trailer)
621 self._fp.seek(offset_table_offset)
-- 622 offset_format = '' + _BINARY_FORMAT[offset_size] *
num_objects
623 self._ref_format = _BINARY_FORMAT[self._ref_size]
624 self._object_offsets = struct.unpack(
KeyError: 3
--
messages: 218821
nosy: slo.sleuth
priority: normal
severity: normal
status: open
title: plistlib unable to load iOS7 Safari History.plist
type: crash
versions: Python 3.4
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue21538
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21538] plistlib unable to load iOS7 Safari History.plist
John Lehr added the comment: I have shared the file for which I have permission, but neither of two History.plists in my possession will load, both with the same traceback. Thank you. On Mon, May 19, 2014 at 4:18 PM, Ned Deily rep...@bugs.python.org wrote: Ned Deily added the comment: Can you attach a zipped version of the failing plist or at least identify what data in it is causing the exception? Presumably that plist is user-specific, i.e. contains browser history. -- nosy: +ned.deily ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21538 ___ -- Added file: http://bugs.python.org/file35295/History.plist.7z ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21538 ___ History.plist.7z Description: application/7z-compressed ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21538] plistlib unable to load iOS7 Safari History.plist
John Lehr added the comment: Ned, I understand. I did get permission to post that history, and as I feared, the sanitized History.plist I'm attaching now doesn't have the loading issue that the first did. Possibly a URL character is throwing the exception? Can you suggest where I can put a print statement in the library or some other way to post to get you meaningful debug information? I don't have any experience with pdb, and a verbose python execution is pretty lengthy. As I said before, I have two more extensive History.plist files from two different devices that won't load. On Mon, May 19, 2014 at 5:51 PM, Ned Deily rep...@bugs.python.org wrote: Ned Deily added the comment: John, a quick look at the file shows that it contains browsing history information that might be personally identifiable. I don't think that is appropriate to store on a public web site like this one so I've deleted the file. Is there some way to provide a simpler file with non-personal history? -- nosy: +ned.deily ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21538 ___ -- Added file: http://bugs.python.org/file35298/History.plist ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21538 ___ History.plist Description: Binary data ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
