Re: [Pythonmac-SIG] Problems building a binary from cpython that can be installed in /Users/{USER}/Library

[Christopher Barker]

I would very highly recommend that you not reinvent the wheel here.

But if you really don't like the way Py2App or PyInstaller does it, then I
wouldn't try to build the entire "normal" mac distribution anyway, and
start at a lower level. Maybe look at homebrew for ideas?

BTW -- there's a third option: conda and "conda constructor" -- might be
worth a look.

-CHB




On Sun, Nov 14, 2021 at 7:57 AM Barry Scott  wrote:

>
>
> On 10 Nov 2021, at 20:28, Paul Romer  wrote:
>
> I am new to this sig. I hope it is appropriate to pose the following
> detailed question to its members. Let me know if I have misunderstood.
>
> I'll provide a bit of background, specify the steps I took, then describe
> the two issues I'm facing on which I would welcome advice.
>
> =
>
> *Background: *
> I am trying to create a self-contained Python application that a user
> could download and run on MacOS that does not have Python 3 installed. I
> am aware that PyInstaller and py2app are alternative paths for creating
> self-contained apps and am pursuing them in parallel.
>
> The issue I want to describe here arose as part of experiments with the
> process for creating a binary for a recent version of Python (3.9.8) that
> can be installed under the user's home directory.
>
> /Users/{USER}/Library.
>
> According to the information in the cpython repository, it seems that it
> should be possible to build an installer for a full version of Python that
> can be installed someplace on a Mac other than the default location:
> /Library. In particular, one should be able to do this using
>
> build-installer.py
>
> in the
>
> Mac/BuildScript
>
> directory.
>
> *=*
>
> *Steps I Took:*
>
> In the cpython directory I ran:
>
> > ./configure --enable-framework=/Users/{USER}/Library/Frameworks \
> --enable-optimizations
>
>
> Why do you think you need to build CPYTHON at all?
>
> I use py3app with the python.org downloaded versions of python.
>
>
> I confirmed that the Makefile included the line:
>
> CONFIG_ARGS= '--enable-framework=/Users/{USER}/Library/Frameworks'
> '--enable-optimizations'
>
> (I also tried replacing {USER} with the actual user on my system, 'admin'.
> This yielded the same results.)
>
> I put the external files OpenSSL, tcl/tck, ... (details below) that are
> required into the
>
> .../Universal/other-sources
>
> directory. I deleted my install of Python3.9 and backed-up and deleted all
> the files from
>
> /usr/local/bin
>
> Then from
>
> /Mac/BuildScript
>
> I ran
>
> > python3 build-installer.py \
> --universal-archs=intel-64 \
> --dep-target=11
>
> Along the way, there were several warnings, but no errors. The end result
> is a dmg file:
>
> python-3.9.8+-macos12-2021-11-10.dmg
>
> =
>
> *Issues:*
>
> 1. I am not sure how to convert the dmg into a "macOS flat installer
> package" and would welcome any advice on this point.
>
>
>
> You do not need to. All the user needs to do it drag the .app
> folder into (typically) /Applications.
>
>
> 2. But in any case, I do not think that an installer based on this dmg
> will allow an install in the Library of the user. When I inspect the dmg
> file, I find that the post-install scripts include references to the usual
> file location in /Library. For example, the script that runs after
> installing the framework has the lines:
>
> PYVER="3.9"
> FWK="/Library/Frameworks/Python.framework/Versions/3.9"
>
>
>  What post-install script? That's not something that py2app does.
>
>
> The script that runs after installing the files for pip has the lines:
>
> PYVER="3.9"
> PYMAJOR="3"
> FWK="/Library/Frameworks/Python.framework/Versions/${PYVER}"
> RELFWKBIN="../../..${FWK}/bin"
>
> This suggests that I have misunderstood the instructions. So I would
> appreciate any advice on how to succeed on this path, or suggestions about
> alternative paths that might work. In particular, the alternative that
> might be feasible would be to use the work from
>
> https://github.com/codrsquad/portable-python
>
> This is a recent project, not the one with the same name that was
> abandoned that you'll find here:
>
> https://portablepython.com/
>
>
> For Barry's Emacs I do 2 steps.
>
> 1. Build the .app bundle.
> 2. Build .dmg that has a bunch of files including the .app bundle.
>
> There may be a bit too much complexity but have look at how I build the
> .app bundle in
>
> https://github.com/barry-scott/BarrysEmacs/blob/master/Editor/PyQtBEmacs/build-macosx.sh
> 
> and
>
> https://github.com/barry-scott/BarrysEmacs/blob/master/Editor/PyQtBEmacs/brand.setup-macosx.py
>
> I'm using PyQt not Tk.
>
> Barry
>
>
>
>
> Thanks for reading through all this!
>
>
> =
>

Re: [Pythonmac-SIG] Problems building a binary from cpython that can be installed in /Users/{USER}/Library

[Barry Scott]

> On 10 Nov 2021, at 20:28, Paul Romer  wrote:
> 
> I am new to this sig. I hope it is appropriate to pose the following detailed 
> question to its members. Let me know if I have misunderstood. 
> 
> I'll provide a bit of background, specify the steps I took, then describe the 
> two issues I'm facing on which I would welcome advice. 
> 
> =
> 
> Background: 
> I am trying to create a self-contained Python application that a user could 
> download and run on MacOS that does not have Python 3 installed. I am aware 
> that PyInstaller and py2app are alternative paths for creating self-contained 
> apps and am pursuing them in parallel. 
> 
> The issue I want to describe here arose as part of experiments with the 
> process for creating a binary for a recent version of Python (3.9.8) that can 
> be installed under the user's home directory. 
> 
> /Users/{USER}/Library. 
> 
> According to the information in the cpython repository, it seems that it 
> should be possible to build an installer for a full version of Python that 
> can be installed someplace on a Mac other than the default location: 
> /Library. In particular, one should be able to do this using 
> 
> build-installer.py 
> 
> in the
> 
> Mac/BuildScript 
> 
> directory. 
> 
> =
> 
> Steps I Took:
> 
> In the cpython directory I ran: 
> 
> > ./configure --enable-framework=/Users/{USER}/Library/Frameworks \
> --enable-optimizations

Why do you think you need to build CPYTHON at all?

I use py3app with the python.org  downloaded versions of 
python.

> 
> I confirmed that the Makefile included the line: 
> 
> CONFIG_ARGS= '--enable-framework=/Users/{USER}/Library/Frameworks' 
> '--enable-optimizations'
> 
> (I also tried replacing {USER} with the actual user on my system, 'admin'. 
> This yielded the same results.)  
> 
> I put the external files OpenSSL, tcl/tck, ... (details below) that are 
> required into the 
> 
> .../Universal/other-sources 
> 
> directory. I deleted my install of Python3.9 and backed-up and deleted all 
> the files from 
> 
> /usr/local/bin 
> 
> Then from 
> 
> /Mac/BuildScript 
> 
> I ran 
> 
> > python3 build-installer.py \
> --universal-archs=intel-64 \
> --dep-target=11
> 
> Along the way, there were several warnings, but no errors. The end result is 
> a dmg file: 
> 
> python-3.9.8+-macos12-2021-11-10.dmg
> 
> =
> 
> Issues:
> 
> 1. I am not sure how to convert the dmg into a "macOS flat installer package" 
> and would welcome any advice on this point. 


You do not need to. All the user needs to do it drag the .app folder 
into (typically) /Applications.

> 
> 2. But in any case, I do not think that an installer based on this dmg will 
> allow an install in the Library of the user. When I inspect the dmg file, I 
> find that the post-install scripts include references to the usual file 
> location in /Library. For example, the script that runs after installing the 
> framework has the lines: 
> 
> PYVER="3.9"
> FWK="/Library/Frameworks/Python.framework/Versions/3.9"

 What post-install script? That's not something that py2app does.

> 
> The script that runs after installing the files for pip has the lines: 
> 
> PYVER="3.9"
> PYMAJOR="3"
> FWK="/Library/Frameworks/Python.framework/Versions/${PYVER}"
> RELFWKBIN="../../..${FWK}/bin"
> 
> This suggests that I have misunderstood the instructions. So I would 
> appreciate any advice on how to succeed on this path, or suggestions about 
> alternative paths that might work. In particular, the alternative that might 
> be feasible would be to use the work from 
> 
> https://github.com/codrsquad/portable-python 
> 
> 
> This is a recent project, not the one with the same name that was abandoned 
> that you'll find here:
> 
> https://portablepython.com/ 
For Barry's Emacs I do 2 steps.

1. Build the .app bundle.
2. Build .dmg that has a bunch of files including the .app bundle.

There may be a bit too much complexity but have look at how I build the .app 
bundle in
https://github.com/barry-scott/BarrysEmacs/blob/master/Editor/PyQtBEmacs/build-macosx.sh
 

and
https://github.com/barry-scott/BarrysEmacs/blob/master/Editor/PyQtBEmacs/brand.setup-macosx.py
 


I'm using PyQt not Tk.

Barry


> 
> 
> Thanks for reading through all this! 
> 
> 
> =
> 
> PS: Contents of my .../Universal/other-sources directory: 
> 
> -rw-r--r--   1 admin  staff367573 Nov 10 00:53 
> ncurses-5.9-20120616-patch.sh.bz2
> -rw-r--r--   1 admin  staff   

[Pythonmac-SIG] ANN: PyObjC 8.0

[Ronald Oussoren via Pythonmac-SIG]

A new macOS release means a new major release of PyObjC. I'm happy to announce 
that PyObjC 8 is available on PyPI with full support for macOS 12, including 
APIs introduced in that release.

I've also been working on performance and code quality of the core bridge. 
There are significant speedups of the core bridge in PyObjC 8.0, and I have 
more ideas to reduce the performance difference between regular Python methods 
and calling Objective-C APIs.

The release is a bit later than I would have liked, in part because I've been 
worrying about the covid-19 situation in Holland. Because of that I've spend 
some more time away from the computer.

A full change log is included below.

Backward incompatible changes
In PyObjC 7 and earlier it was possible to leave out "output" arguments when 
calling a method with a Python implementation:

class MyClass(NSObject):
@objc.typedSelector(b"@@:o^@")
def methodWithOutput_(self, a):
return 1, 2

o = MyClass.alloc().init()
print(o.methodWithOutput_())
This no longer works, it is always necessary to pass in all arguments, which 
was already true for methods implemented in Objective-C. That is:

print(o.methodWithOutput_(None))
This change both simplifies the PyObjC code base and was required to cleanly 
implement vectorcall support (see the section on performance below).

Removed bindings for InterfaceBuilderKit. This was a way to integrate with 
InterfaceBuilder in old versions of Xcode, but support for that was dropped 
before the release of Mac OS X 10.8.

Bindings for the Objective-C runtime API related to "associated objects" is now 
enabled unconditionally. This will cause problems when running or building on 
macOS 10.6 or earlier.

It is no longer possible to deploy to macOS 10.7 or earlier when you attempt to 
create a formal protocol. Protocol creation already failed on those platform 
due to lack of the required runtime API, and that will now result in a crash 
because PyObjC no longer checks for availability of that runtime API.

GH-371: Remove manual bindings for a number of old CoreGraphics APIs

The following functions are no longer available:

CGDataProviderCreate
CGDataProviderCreateDirectAccess
These functions were removed as a public API in macOS 10.8, but were still 
available through PyObjC through old backward compatibility code. That code has 
now been removed.

For compatibility with Python's socket APIs functions that return a "struct 
sockaddr" (either by reference or as a function result) will now encode the IP 
address as a string and not a bytes object.

The (undocumented) API in pyobjc-api.h (used in some framework bindings to 
integratie with pyobjc-core) has changed in an incompatible way, in particular 
the API for "caller" functions now mostly mirrors the vectorcall convention.

Adding a method with a double underscore name will now raise an exception at 
class definition time instead of silently not creating the Objective-C method.

 class AClass (NSObject):
 ...

 def __foo_bar__(self, a, b, c):
 pass

 MethodNamesClass.__foo_bar__ = __foo_bar__
Before PyObjC 8 this would add a __foo_bar__ selector to the Python 
representation of the class without adding a selector to the Objective-C class.

Use objc.python_method to mark this as a python-only function.

Upcoming incompatible changes
The module PyObjCTools.Signals is deprecated and will be removed in PyObjC 9.

objc.initFrameworkWrapper and objc.parseBridgeSupport are deprecated and will 
be removed in PyObjC 9.

These functions implement support for ".bridgesupport" XML files, something 
that PyObjC hasn't used itself in a number of releases (in part because system 
versions of those files are at best incomplete).

Performance
Most performance changes use features introduced in Python 3.9, performance in 
older Python versions is unchanged except for the effects of general cleanup.

Implement the "vectorcall" protocol for objc.function, objc.WeakRef, 
objc.selector, objc.IMP, objc.python_method.

This reduces the interpreter overhead for calling instances of these objects.

Implement Py_TPFLAGS_METHOD_DESCRIPTOR for objc.selector, objc.python_method.

Use vectorcall in the method stub that forwards Objective-C calls to Python.

Convert internal calls into Python to the vectorcall protocol (pyobjc-core)

Add more optimized vectorcall implementation to :class:objc.function, objc.IMP 
and objc.selector for simpler callables.

"Simpler" methods are those with a small number of plain arguments, although 
the definition of what's simple will evolve over time.

Generic Implementation Quality
GH-391: Fix some spelling errors found by the codespell 
 tool.

The codespell tool is also run as part of pre-commit hooks.

GH-296: use clang-format for Objective-C code

The Objective-C code for the various extensions has been reformatted using 
clang-format, and this enforced by a pre-commit hook.

GH-374: Use pyupgrade to modernize the 

[Pythonmac-SIG] Problems building a binary from cpython that can be installed in /Users/{USER}/Library

[Paul Romer]

I am new to this sig. I hope it is appropriate to pose the following
detailed question to its members. Let me know if I have misunderstood.

I'll provide a bit of background, specify the steps I took, then describe
the two issues I'm facing on which I would welcome advice.

=

*Background: *
I am trying to create a self-contained Python application that a user could
download and run on MacOS that does not have Python 3 installed. I am aware
that PyInstaller and py2app are alternative paths for creating
self-contained apps and am pursuing them in parallel.

The issue I want to describe here arose as part of experiments with the
process for creating a binary for a recent version of Python (3.9.8) that
can be installed under the user's home directory.

/Users/{USER}/Library.

According to the information in the cpython repository, it seems that it
should be possible to build an installer for a full version of Python that
can be installed someplace on a Mac other than the default location:
/Library. In particular, one should be able to do this using

build-installer.py

in the

Mac/BuildScript

directory.

*=*

*Steps I Took:*

In the cpython directory I ran:

> ./configure --enable-framework=/Users/{USER}/Library/Frameworks \
--enable-optimizations

I confirmed that the Makefile included the line:

CONFIG_ARGS= '--enable-framework=/Users/{USER}/Library/Frameworks'
'--enable-optimizations'

(I also tried replacing {USER} with the actual user on my system, 'admin'.
This yielded the same results.)

I put the external files OpenSSL, tcl/tck, ... (details below) that are
required into the

.../Universal/other-sources

directory. I deleted my install of Python3.9 and backed-up and deleted all
the files from

/usr/local/bin

Then from

/Mac/BuildScript

I ran

> python3 build-installer.py \
--universal-archs=intel-64 \
--dep-target=11

Along the way, there were several warnings, but no errors. The end result
is a dmg file:

python-3.9.8+-macos12-2021-11-10.dmg

=

*Issues:*

1. I am not sure how to convert the dmg into a "macOS flat installer
package" and would welcome any advice on this point.

2. But in any case, I do not think that an installer based on this dmg will
allow an install in the Library of the user. When I inspect the dmg file, I
find that the post-install scripts include references to the usual file
location in /Library. For example, the script that runs after installing
the framework has the lines:

PYVER="3.9"
FWK="/Library/Frameworks/Python.framework/Versions/3.9"

The script that runs after installing the files for pip has the lines:

PYVER="3.9"
PYMAJOR="3"
FWK="/Library/Frameworks/Python.framework/Versions/${PYVER}"
RELFWKBIN="../../..${FWK}/bin"

This suggests that I have misunderstood the instructions. So I would
appreciate any advice on how to succeed on this path, or suggestions about
alternative paths that might work. In particular, the alternative that
might be feasible would be to use the work from

https://github.com/codrsquad/portable-python

This is a recent project, not the one with the same name that was abandoned
that you'll find here:

https://portablepython.com/


Thanks for reading through all this!


=

PS: Contents of my .../Universal/other-sources directory:

-rw-r--r--   1 admin  staff367573 Nov 10 00:53
ncurses-5.9-20120616-patch.sh.bz2
-rw-r--r--   1 admin  staff   2826473 Nov 10 00:53 ncurses-5.9.tar.gz
-rw-r--r--   1 admin  staff   9834044 Nov  9 22:19 openssl-1.1.1l.tar.gz
-rw-rw-rw-@  1 admin  staff   2956627 Nov 10 01:06
sqlite-autoconf-3350500.tar.gz
-rw-rw-rw-@  1 admin  staff  10259009 Nov  9 22:55 tcl8.6.11-src.tar.gz
-rw-rw-rw-@  1 admin  staff   4496914 Nov 10 00:37 tk8.6.11-src.tar.gz
-rw-rw-rw-@  1 admin  staff   1490665 Nov 10 00:45 xz-5.2.3.tar.gz

=


Paul Romer
p...@paulromer.net | paulromer.net | @paulromer

___
Pythonmac-SIG maillist  -  Pythonmac-SIG@python.org
https://mail.python.org/mailman/listinfo/pythonmac-sig
unsubscribe: https://mail.python.org/mailman/options/Pythonmac-SIG