[issue38705] venv creation on macOS Catalina is failing

[Nikhil Benesch]

Nikhil Benesch  added the comment:

So it looks like /usr/bin/python3 is just a shim that immediately execs the 
real Python binary:


/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/bin/python3

This binary is identical to the one that ends up in the venv.

Problematically, this binary is dynamically linked, and links the "Python3" 
shared library using a relative path:

$ otool -L 
/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/bin/python3
/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/bin/python3:
@executable_path/../Python3 (compatibility version 3.8.0, current 
version 3.8.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current 
version 1292.0.0)

So obviously copying that binary to a different path breaks the 
@executable_path-relative link. Ah, well. Definitely seems like Apple's bug. 
Not sure there is anything that even could be done on the CPython side.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38705] venv creation on macOS Catalina is failing

[Nikhil Benesch]

Nikhil Benesch  added the comment:

Oops, sorry, I forgot to actually include the correct command in the 
workaround. Here it is:

$ /usr/bin/python3 -c 'import venv; venv.create("venv2", symlinks=True);'

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38705] venv creation on macOS Catalina is failing

[Nikhil Benesch]

Nikhil Benesch  added the comment:

This issue continues to exist with the system Python on macOS. Here is how to 
reproduce on macOS Catalina 10.15.7 and Xcode 12.1.

$ /usr/bin/python3 --version
Python 3.8.2

$ /usr/bin/python3 -c 'import venv; venv.create("venv");'

$ venv/bin/python3
dyld: Library not loaded: @executable_path/../Python3
  Referenced from: /Users/benesch/venv/bin/python3
  Reason: image not found
Abort trap: 6

Weird, isn't it? The thing that gets copied into the venv is just a totally 
different file than /usr/bin/python3:

$ ls -lah venv/bin/python3
-rwxr-xr-x 1 benesch staff 148K Oct 26 23:16 venv/bin/python3

$ ls -lah /usr/bin/python3
-rwxr-xr-x 1 root wheel 31K Sep 21 20:29 /usr/bin/python3

I assume this is Apple's bug, but I'll be damned if I have to file another 
report into the blackhole that is radar. If someone needs a quick workaround, 
just create your venvs with symlinks, which works just fine:

$ /usr/bin/python3 -c 'import venv; venv.create("venv2");'
$ venv2/bin/python3
Python 3.8.2 (default, Sep 24 2020, 19:37:08) 
[Clang 12.0.0 (clang-1200.0.32.21)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> 

Incidentally this is why `/usr/bin/python3 -m venv` works just fine, since it 
defaults to `symlinks=True` on macOS.

--
nosy: +benesch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38705] venv creation on macOS Catalina is failing

[Ned Deily]

Change by Ned Deily :


--
resolution:  -> third party
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38705] venv creation on macOS Catalina is failing

[Den Delimarsky]

Den Delimarsky  added the comment:

@ned.deily actually you bring up a very good point, I did not notice that it 
was using the default Catalina Python 3 install. I've chatted with 
@brett.cannon about this earlier and it seemed like the issue was in that the 
Python version I was using is no longer supported, with the fix likely being in 
one of the later branches. I've updated my installation from python.org and 
everything works as expected now. Let me know if you want me to close this 
issue, and thank you for your insights!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38705] venv creation on macOS Catalina is failing

[Ned Deily]

Ned Deily  added the comment:

Thanks for the report but without a supplying a reproducible test case it's 
really difficult to say what issue you might be seeing. FWIW, I was not able to 
reproduce a failure but I was just guessing what your project code really does. 
 Also be aware that you appear to be using the Apple-supple Python 3.7.3 that 
is included with Apple's Xcode 11 Command Line Tools for 10.15 Catalina and is 
not part of a standard macOS installation.  I believe that Python3 is there 
only to support other parts of the Command Line Tools and it is not recommended 
that you use it for your own projects.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38705] venv creation on macOS Catalina is failing

[Den Delimarsky]

Den Delimarsky  added the comment:

Should also mention that this is working as expected in Python 3.8.0

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38705] venv creation on macOS Catalina is failing

[Den Delimarsky]

New submission from Den Delimarsky :

I have a project, where I am working inside a virtual environment (on macOS 
Catalina). Inside said project, I am attempting to create a new virtual 
environment with:

venv.create(virtual_environment_directory, with_pip=True)

This, however, errors out:

  File 
"/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/runpy.py",
 line 193, in _run_module_as_main
"__main__", mod_spec)
  File 
"/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/runpy.py",
 line 85, in _run_code
exec(code, run_globals)
  File "/Users/user1/Documents/repos/adg/src/adg/__main__.py", line 35, in 

CommandProcessor.validate(args, current_os)
  File "/Users/user1/Documents/repos/adg/src/adg/helpers/commandprocessor.py", 
line 21, in validate
LibraryProcessor.process_libraries(command.library, command.platform, 
command.out)
  File "/Users/user1/Documents/repos/adg/src/adg/helpers/coreutil.py", line 79, 
in process_libraries
LibraryInstaller.create_environment()
  File "/Users/user1/Documents/repos/adg/src/adg/helpers/coreutil.py", line 40, 
in create_environment
venv.create(virtual_environment_directory, with_pip=True)
  File 
"/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/venv/__init__.py",
 line 363, in create
builder.create(env_dir)
  File 
"/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/venv/__init__.py",
 line 68, in create
self._setup_pip(context)
  File 
"/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/venv/__init__.py",
 line 261, in _setup_pip
subprocess.check_output(cmd, stderr=subprocess.STDOUT)
  File 
"/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/subprocess.py",
 line 395, in check_output
**kwargs).stdout
  File 
"/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/subprocess.py",
 line 487, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command 
'['/Users/user1/Documents//repos/adg/src/dtemp/bin/python3', '-Im', 
'ensurepip', '--upgrade', '--default-pip']' died with .

Seems like attempting to run this both from inside and outside the virtual 
environment results in the error.

--
components: macOS
messages: 356061
nosy: dend, ned.deily, ronaldoussoren
priority: normal
severity: normal
status: open
title: venv creation on macOS Catalina is failing
type: behavior
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com