[issue22468] Tarfile using fstat on GZip file object

[Martin Panter]

Martin Panter added the comment:

Hoping my clarification in the documentation is enough to call this fixed

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed
versions: +Python 2.7, Python 3.6 -Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22468] Tarfile using fstat on GZip file object

[Roundup Robot]

Roundup Robot added the comment:

New changeset 94a94deaf06a by Martin Panter in branch '3.5':
Issues #22468, #21996, #22208: Clarify gettarinfo() and TarInfo usage
https://hg.python.org/cpython/rev/94a94deaf06a

New changeset e66c476b25ec by Martin Panter in branch 'default':
Issue #22468: Merge gettarinfo() doc from 3.5
https://hg.python.org/cpython/rev/e66c476b25ec

New changeset 9d5217aaea13 by Martin Panter in branch '2.7':
Issues #22468, #21996, #22208: Clarify gettarinfo() and TarInfo usage
https://hg.python.org/cpython/rev/9d5217aaea13

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22468] Tarfile using fstat on GZip file object

[Martin Panter]

Martin Panter added the comment:

I am posting a documentation patch which I hope should clarify that objects 
like GzipFile won’t work automatically with gettarinfo(). It also has other 
modifications to address Issue 21996 (name must be text) and help with Issue 
22208 (clarify non-OS files won’t work).

--
assignee:  -> docs@python
components: +Documentation
keywords: +patch
nosy: +docs@python
stage:  -> patch review
versions: +Python 3.5
Added file: http://bugs.python.org/file39136/gettarinfo.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22468] Tarfile using fstat on GZip file object

[Martin Panter]

Martin Panter added the comment:

I think a warning in the documentation might be helpful.

However a special check in the code doesn’t seem right. Would you check for 
LZMAFile and BZ2File as well? Some of the other attributes (modification time, 
owner, etc) may be useful even for a GzipFile, and the programmer can just 
overwrite the file size attribute if necessary.

--
nosy: +vadmium

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22468] Tarfile using fstat on GZip file object

[Mark Lawrence]

Mark Lawrence added the comment:

msg227328 states "it's not a really common scenario" but I believe we must 
still allow for it, what do others think?

--
nosy: +BreamoreBoy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22468] Tarfile using fstat on GZip file object

[Ned Deily]

Changes by Ned Deily :


--
nosy: +lars.gustaebel

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22468] Tarfile using fstat on GZip file object

[Bart Olsthoorn]

New submission from Bart Olsthoorn:

CPython tarfile `gettarinfo` method uses fstat to determine the size of a file 
(using its fileobject). When that file object is actually created with 
Gzip.open (so a GZipfile), it will get the compressed size of the file. The 
addfile method will then continue to read the uncompressed data of the gzipped 
file, but will read too few bytes, resulting in a tar of incomplete files.

I suggest checking the file object class before using fstat to determine the 
size, and raise a warning if it's a gzip file.

To clarify, this only happens when adding a GZip file object to tar. I know 
that it's not a really common scenario, and the problem is really that GZip 
file size can only properly be determined by uncompressing and reading it 
entirely, but I think it's nice to not fail without warning.

So this is an example that is failing:
```
import tarfile
c = io.BytesIO()
with tarfile.open(mode='w', fileobj=c) as tar:
  for textfile in ['1.txt.gz', '2.txt.gz']:
with gzip.open(textfile) as f:
  tarinfo = tar.gettarinfo(fileobj=f)
  tar.addfile(tarinfo=tarinfo, fileobj=f)
  data = c.getvalue()
return data
```

Instead this reads the proper filesize and writes the files to a tar:
```
import tarfile
c = io.BytesIO()
with tarfile.open(mode='w', fileobj=c) as tar:
  for textfile in ['1.txt.gz', '2.txt.gz']:
with gzip.open(textfile) as f:
  buff = f.read()
  tarinfo = tarfile.TarInfo(name=f.name)
  tarinfo.size = len(buff)
  tar.addfile(tarinfo=tarinfo, fileobj=io.BytesIO(buff))
  data = c.getvalue()
return data
```

--
messages: 227328
nosy: bartolsthoorn
priority: normal
severity: normal
status: open
title: Tarfile using fstat on GZip file object
type: behavior
versions: Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com