Re: [Pythonmac-SIG] py2app questions

2020-12-16 Thread Timothy M. Shead


> On Dec 15, 2020, at 10:17 AM, Christopher Barker  wrote:
> 
> On Mon, Dec 14, 2020 at 11:28 PM Timothy M. Shead 
> mailto:t...@shead-custom-design.com>> wrote:
> > How was Python build?
> 
> The Python executable and all dependencies are installed using Conda:
> 
> Python 3.8.5 (default, Sep  4 2020, 02:22:02) 
> [Clang 10.0.0 ] :: Anaconda, Inc. on darwin
> 
> Note that the conda build of Python on OS-X is a "plain *nix" build. which 
> may be the source of your issues. I'm not sure what the difference is other 
> than the app bundle wrapper (pythonw), but I wouldn't be supposed if there's 
> a difference that will break py2app.

I didn’t know there was a difference.  Thanks for the heads-up!

Cheers,
Tim


___
Pythonmac-SIG maillist  -  Pythonmac-SIG@python.org
https://mail.python.org/mailman/listinfo/pythonmac-sig
unsubscribe: https://mail.python.org/mailman/options/Pythonmac-SIG


Re: [Pythonmac-SIG] py2app questions

2020-12-15 Thread Christopher Barker
On Mon, Dec 14, 2020 at 11:28 PM Timothy M. Shead <
t...@shead-custom-design.com> wrote:

> > How was Python build?
>
> The Python executable and all dependencies are installed using Conda:
>
> Python 3.8.5 (default, Sep  4 2020, 02:22:02)
> [Clang 10.0.0 ] :: Anaconda, Inc. on darwin
>

Note that the conda build of Python on OS-X is a "plain *nix" build. which
may be the source of your issues. I'm not sure what the difference is other
than the app bundle wrapper (pythonw), but I wouldn't be supposed if
there's a difference that will break py2app.

HTH,
 -CHB



> >> And when I look at the main executable, the second rpath looks
> questionable:
> >>
> >>$ otool -l dist/hello.app/Contents/MacOS/hello
> >>…
> >>Load command 16
> >>  cmd LC_RPATH
> >>  cmdsize 32
> >> path @loader_path/../lib (offset 12)
> >>Load command 17
> >>  cmd LC_RPATH
> >>  cmdsize 48
> >> path @loader_path/../../../../../ (offset 12)
> >
> > Was this file created by py2app, or did you change it afterwards?  The
> stub executable in current releases of py2app should not contain LC_RPATH
> entries at all.
>
> This is the executable built by py2app, with no meddling from me:
>
> $ rm -rf build dist
> $ python setup.py py2app
> $ otool -l dist/hello.app/Contents/MacOS/hello
>
> >> What is the right approach to address this?  Manually copy the missing
> .dylib files into dist/hello.app/Contents/lib?  I’m too new to know what to
> expect from py2app, but I’m surprised that it would be necessary for
> something as ubiquitous as zlib?
> >
> > Py2app, or rather the macholib library used by py2app, does not process
> @rpath correctly. The code is older than the introduction of this feature
> and is not easily adjusted for it because some information needed for
> correctly dealing with @rpath and @loader_path is lost before the code that
> should use the information is active.  I have rewriting that code on my too
> long todo list.
> >
> > Note that this works for me, with the Python.org installation of Python.
>
> Many thanks,
> Tim
>
> ___
> Pythonmac-SIG maillist  -  Pythonmac-SIG@python.org
> https://mail.python.org/mailman/listinfo/pythonmac-sig
> unsubscribe: https://mail.python.org/mailman/options/Pythonmac-SIG
>


-- 
Christopher Barker, PhD

Python Language Consulting
  - Teaching
  - Scientific Software Development
  - Desktop GUI and Web Development
  - wxPython, numpy, scipy, Cython
___
Pythonmac-SIG maillist  -  Pythonmac-SIG@python.org
https://mail.python.org/mailman/listinfo/pythonmac-sig
unsubscribe: https://mail.python.org/mailman/options/Pythonmac-SIG


Re: [Pythonmac-SIG] py2app questions

2020-12-14 Thread Timothy M. Shead


> On Dec 14, 2020, at 1:21 AM, Ronald Oussoren  wrote:
> 
> How was Python build?

The Python executable and all dependencies are installed using Conda:

Python 3.8.5 (default, Sep  4 2020, 02:22:02) 
[Clang 10.0.0 ] :: Anaconda, Inc. on darwin


>> And when I look at the main executable, the second rpath looks questionable:
>> 
>>$ otool -l dist/hello.app/Contents/MacOS/hello
>>…
>>Load command 16
>>  cmd LC_RPATH
>>  cmdsize 32
>> path @loader_path/../lib (offset 12)
>>Load command 17
>>  cmd LC_RPATH
>>  cmdsize 48
>> path @loader_path/../../../../../ (offset 12)
> 
> Was this file created by py2app, or did you change it afterwards?  The stub 
> executable in current releases of py2app should not contain LC_RPATH entries 
> at all.

This is the executable built by py2app, with no meddling from me:

$ rm -rf build dist
$ python setup.py py2app
$ otool -l dist/hello.app/Contents/MacOS/hello

>> What is the right approach to address this?  Manually copy the missing 
>> .dylib files into dist/hello.app/Contents/lib?  I’m too new to know what to 
>> expect from py2app, but I’m surprised that it would be necessary for 
>> something as ubiquitous as zlib?
> 
> Py2app, or rather the macholib library used by py2app, does not process 
> @rpath correctly. The code is older than the introduction of this feature and 
> is not easily adjusted for it because some information needed for correctly 
> dealing with @rpath and @loader_path is lost before the code that should use 
> the information is active.  I have rewriting that code on my too long todo 
> list.
> 
> Note that this works for me, with the Python.org installation of Python.

Many thanks,
Tim

___
Pythonmac-SIG maillist  -  Pythonmac-SIG@python.org
https://mail.python.org/mailman/listinfo/pythonmac-sig
unsubscribe: https://mail.python.org/mailman/options/Pythonmac-SIG


Re: [Pythonmac-SIG] py2app questions

2020-12-14 Thread Ronald Oussoren via Pythonmac-SIG


> On 13 Dec 2020, at 04:53, Timothy M. Shead  
> wrote:
> 
> Many thanks for py2app, of all the packaging apps I’ve tried, it’s been the 
> most usable by far; that said, I have questions.  Using conda, python 3.8.5, 
> and py2app 0.22 on MacOS 10.15.7 with the following test app:
> 
># hello.py
>print(“Hello, World!”)
> 
> And the following default setup.py:
> 
>from setuptools import setup
> 
>APP = ["hello.py”]
>DATA_FILES = []
>OPTIONS = {}
> 
>setup(
>app=APP,
>data_files=DATA_FILES,
>options={"py2app": OPTIONS},
>setup_requires=["py2app”],
>)
> 
> Everything builds and runs fine:
> 
>$ python setup.py py2app
>$ dist/hello.app/Contents/MacOS/hello
>Hello, World!
> 
> However, upon closer inspection, I see that there are a couple of libs that 
> are being loaded from outside the bundle:
> 
>$ DYLD_PRINT_LIBRARIES=1 dist/hello.app/Contents/MacOS/hello
>...
>/Users/tshead/miniconda3/envs/flow/lib/libz.1.dylib
>/Users/tshead/miniconda3/envs/flow/lib/libffi.7.dylib
> 
> Focusing on libz, I see that it’s being loaded from a library that is part of 
> the bundle:
> 
>$ otool -L 
> dist/hello.app/Contents/Resources/lib/python3.8/lib-dynload/zlib.so
>dist/hello.app/Contents/Resources/lib/python3.8/lib-dynload/zlib.so:
>   @rpath/libz.1.dylib (compatibility version 1.0.0, current version 
> 1.2.11)
>…

How was Python build? 

> 
> And when I look at the main executable, the second rpath looks questionable:
> 
>$ otool -l dist/hello.app/Contents/MacOS/hello
>…
>Load command 16
>  cmd LC_RPATH
>  cmdsize 32
> path @loader_path/../lib (offset 12)
>Load command 17
>  cmd LC_RPATH
>  cmdsize 48
> path @loader_path/../../../../../ (offset 12)

Was this file created by py2app, or did you change it afterwards?  The stub 
executable in current releases of py2app should not contain LC_RPATH entries at 
all.

> 
> What is the right approach to address this?  Manually copy the missing .dylib 
> files into dist/hello.app/Contents/lib?  I’m too new to know what to expect 
> from py2app, but I’m surprised that it would be necessary for something as 
> ubiquitous as zlib?

Py2app, or rather the macholib library used by py2app, does not process @rpath 
correctly. The code is older than the introduction of this feature and is not 
easily adjusted for it because some information needed for correctly dealing 
with @rpath and @loader_path is lost before the code that should use the 
information is active.  I have rewriting that code on my too long todo list.

Note that this works for me, with the Python.org  
installation of Python.

Ronald
—

Twitter / micro.blog: @ronaldoussoren
Blog: https://blog.ronaldoussoren.net/
> 
> Cheers,
> Tim
> 
> ___
> Pythonmac-SIG maillist  -  Pythonmac-SIG@python.org
> https://mail.python.org/mailman/listinfo/pythonmac-sig
> unsubscribe: https://mail.python.org/mailman/options/Pythonmac-SIG

___
Pythonmac-SIG maillist  -  Pythonmac-SIG@python.org
https://mail.python.org/mailman/listinfo/pythonmac-sig
unsubscribe: https://mail.python.org/mailman/options/Pythonmac-SIG


[Pythonmac-SIG] py2app questions

2020-12-13 Thread Timothy M. Shead
Many thanks for py2app, of all the packaging apps I’ve tried, it’s been the 
most usable by far; that said, I have questions.  Using conda, python 3.8.5, 
and py2app 0.22 on MacOS 10.15.7 with the following test app:

# hello.py
print(“Hello, World!”)

And the following default setup.py:

from setuptools import setup

APP = ["hello.py”]
DATA_FILES = []
OPTIONS = {}

setup(
app=APP,
data_files=DATA_FILES,
options={"py2app": OPTIONS},
setup_requires=["py2app”],
)

Everything builds and runs fine:

$ python setup.py py2app
$ dist/hello.app/Contents/MacOS/hello
Hello, World!

However, upon closer inspection, I see that there are a couple of libs that are 
being loaded from outside the bundle:

$ DYLD_PRINT_LIBRARIES=1 dist/hello.app/Contents/MacOS/hello
...
/Users/tshead/miniconda3/envs/flow/lib/libz.1.dylib
/Users/tshead/miniconda3/envs/flow/lib/libffi.7.dylib

Focusing on libz, I see that it’s being loaded from a library that is part of 
the bundle:

$ otool -L 
dist/hello.app/Contents/Resources/lib/python3.8/lib-dynload/zlib.so
dist/hello.app/Contents/Resources/lib/python3.8/lib-dynload/zlib.so:
@rpath/libz.1.dylib (compatibility version 1.0.0, current version 
1.2.11)
…

And when I look at the main executable, the second rpath looks questionable:

$ otool -l dist/hello.app/Contents/MacOS/hello
…
Load command 16
  cmd LC_RPATH
  cmdsize 32
 path @loader_path/../lib (offset 12)
Load command 17
  cmd LC_RPATH
  cmdsize 48
 path @loader_path/../../../../../ (offset 12)

What is the right approach to address this?  Manually copy the missing .dylib 
files into dist/hello.app/Contents/lib?  I’m too new to know what to expect 
from py2app, but I’m surprised that it would be necessary for something as 
ubiquitous as zlib?

Cheers,
Tim

___
Pythonmac-SIG maillist  -  Pythonmac-SIG@python.org
https://mail.python.org/mailman/listinfo/pythonmac-sig
unsubscribe: https://mail.python.org/mailman/options/Pythonmac-SIG


[Pythonmac-SIG] py2app questions

2016-09-25 Thread Barry Scott
I’m a happy user of py2app and thanks to all the work that has gone into py2app.

I’m using py2app to package a couple of apps that use PyQt and pytz.

I found that the resulting app  has the .dylib files in the python35.zip.
Is that correct? I had assumed that .dylib files need to be in the .app
as files, is there a trick to run them out of the .zip file?

I have been copying in the PyQt .dylib, plugins etc into the .app with
a script that adds these in Contents/Frameworks etc after fixing up the RPATHs.

For pytz to work in a py2app .app pkg_resources needs to work and it
does not. Is this a known issue? I worked around it with a stub pkg_resources
package that reached into the python35.zip and pulled out the zoneinfo files.

It seems that py2app will package up all the files in a package, not just the
.py files. Is that the algorithm that is used?

Barry

___
Pythonmac-SIG maillist  -  Pythonmac-SIG@python.org
https://mail.python.org/mailman/listinfo/pythonmac-sig
unsubscribe: https://mail.python.org/mailman/options/Pythonmac-SIG


Re: [Pythonmac-SIG] py2app questions..

2013-06-25 Thread Ronald Oussoren

On 25 Jun, 2013, at 16:26, Chris Barker - NOAA Federal  
wrote:

> Ronald,
> 
> Did you mean this to go to the list? I've forwarded it on...

I did probably use the wrong reply button :-(

> 
> Anyway, thanks for the notes -- I'll keep all this in mind when I
> build that app again.
> 
>>> I've learned a few things from this.
>>> 
>>> 1) this is odd:
>>> right near the top, I see:
>>>prescript.py
>>>imports: Image
>>> that's the prescript for the PIL recipe -- I"m not using PIL, it
>>> seems it's runnign nthe recipe, and using what the recipe imported --
>>> not good.
>> 
>> Are you sure about not using PIL (if even indirectly)?
> 
> Now that i think about it again, I think MPL makes some optional use
> of PIL -- that's probably it.

Working with the dependency graph is currently too hard for humans, amongst 
other because there is too much detail. A number of people have started to use 
modulegraph outside of py2app and based on interactions on modulegraph's issue 
tracker I have some ideas to improve the situation.

In the long run there probably should be a GUI tool for py2app that makes this 
a lot easier, but that's not going to happen anytime soon. 

Ronald

___
Pythonmac-SIG maillist  -  Pythonmac-SIG@python.org
http://mail.python.org/mailman/listinfo/pythonmac-sig
unsubscribe: http://mail.python.org/mailman/options/Pythonmac-SIG


Re: [Pythonmac-SIG] py2app questions..

2013-06-25 Thread Chris Barker - NOAA Federal
Ronald,

Did you mean this to go to the list? I've forwarded it on...

Anyway, thanks for the notes -- I'll keep all this in mind when I
build that app again.

>> I've learned a few things from this.
>>
>> 1) this is odd:
>>  right near the top, I see:
>> prescript.py
>> imports: Image
>>  that's the prescript for the PIL recipe -- I"m not using PIL, it
>> seems it's runnign nthe recipe, and using what the recipe imported --
>> not good.
>
> Are you sure about not using PIL (if even indirectly)?

Now that i think about it again, I think MPL makes some optional use
of PIL -- that's probably it.

>> An idea how to tweak the MPL recipe to only pull in the backend we
>> need? is there a way to pass options to he recipes?
>
> I hope Guido doesn't mind that I've borrowed his time machine. Support for 
> this was added in py2app 0.7:
>
> - Smarter matplotlib recipe, it is now possible to specify which backends 
> should
>   be included. Issue #44, reported by Adam Kovics.

this is nice -- thanks!

>> 4) Ipython is getting pulled in my Matplotlib too:

> That should be fixable by excluding iPython.

I'll give that a shot.

thanks,
 -Chris


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Pythonmac-SIG maillist  -  Pythonmac-SIG@python.org
http://mail.python.org/mailman/listinfo/pythonmac-sig
unsubscribe: http://mail.python.org/mailman/options/Pythonmac-SIG


Re: [Pythonmac-SIG] py2app questions..

2013-04-15 Thread Chris Barker - NOAA Federal
Ronald et al,

As matplotlib seems to be responsible for bringing eveything else in,
I'm trying a different tack:

1) I've removed many of the recipes from my p2app distribution,
notably teh matplotlib one. So now the only recipe getting run is wx.

2) I know that matplotlib isn't going to work right, as there are
issues with matplotlib data, etc. So I've tried to remove it by adding
it to "excludes". However, it's getting included anyway, in the
site_packages zip file. (but doesn't work...)

But moving on...

I"ve added ['matplotlib'] to "packages".

Now I get it included, in full, in Resources/lib/python2.7, and non
longer in the site_packages zip, and the app works fine.

the matplotlib package is 64MB though, so it would be nice to trim it
down, but this isn't bad, and the total package is now 134MB, rather
than the 210MB I did have.

So I suggest that the matplotlib recipe should:

- remove matplotlib from the site_packages zip

- install the matplotlib package in it's entirety (though maybe strip
out the *.py files, and things like tests, etc.)

- not run matplotlib through modulegraph -- whatever the user is
actually using (numpy, etc) is probably imported elsewhere anyway.
(though maybe including numpy would make some sense...)

I hope this helps others in the future in any case.

-Chris








On Fri, Apr 12, 2013 at 1:47 PM, Chris Barker - NOAA Federal
 wrote:
> OK,
>
> I've run py2app with --xref. I've enclosed the html output.
>
> Again, this app uses:
>
>> wxPython
>> numpy
>> matplotlib
>
> (and a few other stdlib packages...)
>
> I've learned a few things from this.
>
> 1) this is odd:
>   right near the top, I see:
>  prescript.py
>  imports: Image
>   that's the prescript for the PIL recipe -- I"m not using PIL, it
> seems it's runnign nthe recipe, and using what the recipe imported --
> not good.
>
>
> 2) do I need all of Carbon?
> Carbon
> imports: warnings
> imported by: Carbon.AE Carbon.Appearance Carbon.AppleEvents
> Carbon.CarbonEvents Carbon.ControlAccessor Carbon.Controls Carbon.Ctl
> Carbon.Dialogs Carbon.Dlg Carbon.Dragconst Carbon.Events Carbon.Evt
> Carbon.File Carbon.Files Carbon.Menu Carbon.Qd Carbon.QuickDraw
> Carbon.Res Carbon.TextEdit Carbon.Win Carbon.Windows EasyDialogs
> aepack aetools macostools macresource plistlib
>
> I'm guessing py2ap uses plistlib, or something...
>
> 3) Matplotlib has multipel back-ends -- that's why I'm getting TK
> stuff (and from there, Carbon?):
>FileDialog
>imports: Dialog Tkinter fnmatch os
>imported by: matplotlib.backends.backend_tkagg
>
> An idea how to tweak the MPL recipe to only pull in the backend we
> need? is there a way to pass options to he recipes?
>
> 4) Ipython is getting pulled in my Matplotlib too:
>
> IPython
> imports:
> 
> IPython.zmq matplotlib.sphinxext.ipython_directive
>
> which explains the zmq.so
>
> ( and a lot of other stuff -- iPython pulls in a lot )
>
> 5) It looks like Image is getting pulled in a couple places:
> ...
>  docutils.parsers.rst.directives.images docutils.writers.html4css1
> prescript.py scipy.misc.pilutil
>
> docutils, prescript, and scipy (not sure why scipy is pulled in yet, I
> removed that dependency...)
>
> 6)OpenGL
> imported by: IPython.lib.inputhookglut 
>
> so that's getting pulled in by IPython, which is pulled in by MPL... ( I 
> think )
> (wow! OPenGL sure has a lot of modules!)
>
> 7) bsddb seems to be recusively importing itself...
>
> bsddb
> imports: UserDict _bsddb bsddb.dbutils collections os warnings weakref
> imported by: bsddb.db bsddb.dbutils dbhash
>
> bsddb.db
> imports: _bsddb bsddb
> imported by: bsddb.dbutils
>
> bsddb.dbutils
> imports: bsddb bsddb.db time
> imported by: bsddb
>
> 
>
> OK, I'm getting bored. A lot of this is really caused by Matplotlib --
> which make some sense, it's been designed to be a developers tool (or
> at least to be used interactively), with all sorts of options,
> determined at run-time.
>
> Is there a way to turn off a recipe other than removing ot from my
> py2app install?
>
> This is making me think that the MPL recipe should pull in the MPL
> package, but not scan it for dependencies...
>
> Still not sure about PIL -- but I wouldn't be surprised if MPL is
> involved somehow!
>
> -Chris
>
>
>
> --
>
> Christopher Barker, Ph.D.
> Oceanographer
>
> Emergency Response Division
> NOAA/NOS/OR&R(206) 526-6959   voice
> 7600 Sand Point Way NE   (206) 526-6329   fax
> Seattle, WA  98115   (206) 526-6317   main reception
>
> chris.bar...@noaa.gov



-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Pythonmac-SIG maillist  -  Pythonmac-SIG@python.org
http://mail.python.org/mailman/listinfo/pythonmac-sig
unsubscribe: http://mail.python.org/mailman/opt

Re: [Pythonmac-SIG] py2app questions..

2013-04-12 Thread Chris Barker - NOAA Federal
On Thu, Apr 11, 2013 at 10:50 PM, Ronald Oussoren
 wrote:
> On 11 Apr, 2013, at 23:50, Chris Barker - NOAA Federal 
>  wrote:

>> *** using recipe: virtualenv ***
>> *** using recipe: sip ***
...

>> It looks like the recipes for ALL of the packages I have installed are
>> being run -- a bug perhaps?
>
> That's not a bug, all recipes get called on every build and each one checks 
> if it actually has to do something (mostly by checking for the presence of 
> modules in the dependency graph).

hmm -- maybe a re-write of that message to something like "checking
recipe: something".

But anyway, as far as I can tell, I'm getting all the packages there
is a recipe for that I have installed. Or certainly a bunch of extras.

I'll poke into the dependency graph and see if I can see what's going on.

-Chris




-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Pythonmac-SIG maillist  -  Pythonmac-SIG@python.org
http://mail.python.org/mailman/listinfo/pythonmac-sig
unsubscribe: http://mail.python.org/mailman/options/Pythonmac-SIG


Re: [Pythonmac-SIG] py2app questions..

2013-04-11 Thread Ronald Oussoren

On 10 Apr, 2013, at 20:19, Chris Barker - NOAA Federal  
wrote:

> First: Ronald, thanks for keeping py2app up to date!
> 
> The good news: I just ran a recent py2app on an older app of mine, and
> it all worked out of the box.
> 
> However, the resulting bundle is HUGE. A lot of this is inevitable,
> I'm using a universal build, and some big packages, but there seems to
> be some room for improvement:
> 
> The app  in question is using:
> 
> wxPython
> numpy
> a tiny bit of scipy
> matplotlib
> 
> So it's going to be big!
> 
> Numpy, scipy and matplotlib all have a lot of intertwining modules,
> many of which get imported whether you need them or not, so probably
> the best one can do is include them all (more on that later)
> 
> However, I'm getting a couple packages that I have no idea why:
> 
> OpenGL

This I don't know.

> email

This will get better with py2app 0.8. In 0.7 the entire email package
gets included when some module has a dependency on a module from the
email package because email uses __import__ to provide compatibility aliases. 
In 0.8 I've taught modulegraph about those aliases and therefore only include
the bits of the email package that are actually used.

> zmq

I'v seen other reports that zmq gets pulled in without being used
(in the context of py2app build problems, some app wouldn't be packaged
correctly until pyzmq got removed). I haven't had time yet to see 
why zmq confuses py2app.

> 
> hand-removing them from the bundle doesn't break anything.

Py2app detects dependencies by looking for import statements in the
module bytecode. That doesn't mean the import is actually used (for example
because it is optionally imported on some other platform).

The --xref and --graph options for py2app can be used to emit a module
dependency graph that might show what's going on here.

> 
> I'm wondering if their recipes are getting triggered by accident
> somehow -- how can I tell how/why a particular package got included?
> 
> I'm also getting a number of *.so files that I don't understand --
> some of the larger ones are:
> 
> _bsddb.so
> _sqlite3.so
> _imaging.so  (is that PIL?)
> 
> there are a LOT more, but it's had to know what they are from just the
> names, and most are pretty small anyway  (except the wx ones that I
> need and expect to be big...)
> 
> I understand that it's a goal of py2app to work "out of the box"
> without needing to hand-tweak a lot to get it to work. This is a
> worthy goal, and the recipes approach works great. However, it would
> be nice if there was an alternate approach that made it easier to
> build a more optimized package. One idea:
> 
> Py2app (and all the other stand-alone builders I've looked at) figure
> out want to include from source code analysis. however, as it's pretty
> common for packages to "import" stuff that may not be used a
> particular app, you can get a lot of stuff you dontt need. For
> instance, I'm pretty sure that PIL import tkInter. These imports are
> often wrapped in an if clause or inside a function that may never get
> called, but source code analysis isn't going to find all of those, so
> the same thing to do is include it all -- resulting in bloated
> packages.

That's basicly correct. The recipe system does make it possible to tweak
the dependency graph, and that's (as an example) used in a recipe for
pydoc to remove the import dependency from pydoc on Tkinter as you don't
want to pull in Tkinter unless it is used by other parts of the app.
 
> 
> I propose an alternative -- analysis of the app at run-time: The user
> would run the app (maybe a test suite), then we'd take a look at
> sys.modules and see everything that was actually loaded. This would
> miss anything that wasn't exercised by the code you ran, but for most
> cases, a test suite would bring in everything (comprehensive test
> suites are hard to come by, but all it would need to do is test enough
> to import every package it might use -- that's not a heavy lift).
> 
> This seems so easy -- am I missing something???

Writing support code for this in py2app is easy enough, but that does
require having a comprehensive enough test suite for apps and writing
those is much harder (especially for GUI apps, which is where py2app is
used the most)

> 
> To support this, py2app would need a way to bypass the source code
> analysis, and instead load a data file with the list of modules that
> need to be included. Actually, as it sometimes takes a while to scan
> teh code, it would be nice of py2app could optinally dump the results
> of teh source code analysis, and be abel to re-load it later, rather
> than needed to re-run.
> 
> 1) Am I missing something as to why the run-time analysis wouldn't work ?
> 
> 2) how hard would it be to patch py2app to load a module list, rather
> than scan the source?

Fairly easy, py2app uses modulegraph to build a module dependency graph
and then extracts the list of modules and extensions from that. 

But, I'm not sure it that would be 

[Pythonmac-SIG] py2app questions..

2013-04-10 Thread Chris Barker - NOAA Federal
First: Ronald, thanks for keeping py2app up to date!

The good news: I just ran a recent py2app on an older app of mine, and
it all worked out of the box.

However, the resulting bundle is HUGE. A lot of this is inevitable,
I'm using a universal build, and some big packages, but there seems to
be some room for improvement:

The app  in question is using:

wxPython
numpy
a tiny bit of scipy
matplotlib

So it's going to be big!

Numpy, scipy and matplotlib all have a lot of intertwining modules,
many of which get imported whether you need them or not, so probably
the best one can do is include them all (more on that later)

However, I'm getting a couple packages that I have no idea why:

OpenGL
email
zmq

hand-removing them from the bundle doesn't break anything.

I'm wondering if their recipes are getting triggered by accident
somehow -- how can I tell how/why a particular package got included?

I'm also getting a number of *.so files that I don't understand --
some of the larger ones are:

_bsddb.so
_sqlite3.so
_imaging.so  (is that PIL?)

there are a LOT more, but it's had to know what they are from just the
names, and most are pretty small anyway  (except the wx ones that I
need and expect to be big...)

I understand that it's a goal of py2app to work "out of the box"
without needing to hand-tweak a lot to get it to work. This is a
worthy goal, and the recipes approach works great. However, it would
be nice if there was an alternate approach that made it easier to
build a more optimized package. One idea:

Py2app (and all the other stand-alone builders I've looked at) figure
out want to include from source code analysis. however, as it's pretty
common for packages to "import" stuff that may not be used a
particular app, you can get a lot of stuff you dontt need. For
instance, I'm pretty sure that PIL import tkInter. These imports are
often wrapped in an if clause or inside a function that may never get
called, but source code analysis isn't going to find all of those, so
the same thing to do is include it all -- resulting in bloated
packages.

I propose an alternative -- analysis of the app at run-time: The user
would run the app (maybe a test suite), then we'd take a look at
sys.modules and see everything that was actually loaded. This would
miss anything that wasn't exercised by the code you ran, but for most
cases, a test suite would bring in everything (comprehensive test
suites are hard to come by, but all it would need to do is test enough
to import every package it might use -- that's not a heavy lift).

This seems so easy -- am I missing something???

To support this, py2app would need a way to bypass the source code
analysis, and instead load a data file with the list of modules that
need to be included. Actually, as it sometimes takes a while to scan
teh code, it would be nice of py2app could optinally dump the results
of teh source code analysis, and be abel to re-load it later, rather
than needed to re-run.

1) Am I missing something as to why the run-time analysis wouldn't work ?

2) how hard would it be to patch py2app to load a module list, rather
than scan the source?

Thoughts??

-Chris






-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Pythonmac-SIG maillist  -  Pythonmac-SIG@python.org
http://mail.python.org/mailman/listinfo/pythonmac-sig
unsubscribe: http://mail.python.org/mailman/options/Pythonmac-SIG