[issue32054] Creating RPM on Python 2 works, but Python 3 fails because of sys.implementation.cache_tag

[Steve Dower]

Steve Dower  added the comment:

Distutils is now deprecated (see PEP 632) and all tagged issues are being 
closed. From now until removal, only release blocking issues will be considered 
for distutils.

If this issue does not relate to distutils, please remove the component and 
reopen it. If you believe it still requires a fix, most likely the issue should 
be re-reported at https://github.com/pypa/setuptools

--
nosy: +steve.dower
resolution:  -> out of date
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32054] Creating RPM on Python 2 works, but Python 3 fails because of sys.implementation.cache_tag

[Pedro]

New submission from Pedro :

I am trying to create an RPM on SLES12 SP2. (I think that corresponds to 
OpenSUSE 42.2.) This is my setup.py file, nothing special:

import setuptools
setuptools.setup(name='MyApp',
 version='1.2.3',
 options={'bdist_rpm': {'post_install': 'post_install.sh',
'post_uninstall': 
'post_uninstall.sh'}})

Running `python setup.py bdist_rpm` with Python 2 works. However, running 
`python3 setup.py bdist_rpm` outputs the following:

running bdist_rpm
running egg_info
writing top-level names to MyApp.egg-info/top_level.txt

...

byte-compiling 
/home/pedro/MyApp/build/bdist.linux-x86_64/rpm/BUILDROOT/MyApp-1.2.3-1.x86_64/usr/lib/python3.4/site-packages/MyApp/my_file.py
 to my_file.cpython-34.pyc

...

Processing files: MyApp-1.2.3-1.noarch
error: File not found: 
/home/pedro/MyApp/build/bdist.linux-x86_64/rpm/BUILDROOT/MyApp-1.2.3-1.x86_64/usr/lib/python3.4/site-packages/MyApp/my_file.py

RPM build errors:
File not found: 
/home/pedro/MyApp/build/bdist.linux-x86_64/rpm/BUILDROOT/MyApp-1.2.3-1.x86_64/usr/lib/python3.4/site-packages/MyApp/my_file.py
error: command 'rpmbuild' failed with exit status 1

The problem appears to be that setuptools is generating a file that ends in 
`.cpython-34.pyc`, and later looks for a file without the `.cpython-34` but 
can't find it.

The RPM generation process on Python 3 goes through 
`distutils.util.byte_compile()`, which in turn calls 
`importlib.util.cache_from_source(path)`, where `path` is the file to be 
byte-compiled. `cache_from_source()` injects the value of 
`sys.implementation.cache_tag` (which is equal to 'cpython-34' on SLES12 SP2) 
into the filename of the compiled file. This attribute of `sys` does not exist 
in Python 2. So it looks like `setuptools` alters the filename with that tag 
during byte compilation, but later forgets that it modified the filename and 
fails because it's looking for the original name.

I ended up working around this by patching `distutils.file_util.write_file` to 
eliminate the .pyc entries from INSTALLED_FILES:

orig_bytecode_var = os.environ.get('PYTHONDONTWRITEBYTECODE', '')
os.environ['PYTHONDONTWRITEBYTECODE'] = '1'
orig_write_file = distutils.file_util.write_file

def my_patch(*args, **kwargs):
new_args = list(args)
if args[0] == 'INSTALLED_FILES':
new_args[1] = [fname for fname in args[1] if fname[-4:] not in 
('.pyc', '.pyo')]
orig_write_file(*new_args, **kwargs)
distutils.file_util.write_file = my_patch

setuptools.setup(name='MyApp',
 version='1.2.3',
 options={'bdist_rpm': {'post_install': 'post_install.sh',
'post_uninstall': 
'post_uninstall.sh'}})

os.environ['PYTHONDONTWRITEBYTECODE'] = orig_bytecode_var
distutils.file_util.write_file = orig_write_file


But I don't think I should have to do anything special to build an RPM on 
Python 3 vs. Python 2. Is this expected behavior?

--
components: Distutils
messages: 306400
nosy: dstufft, eric.araujo, pgacv2
priority: normal
severity: normal
status: open
title: Creating RPM on Python 2 works, but Python 3 fails because of 
sys.implementation.cache_tag
type: behavior
versions: Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com