Phillip J. Eby schrieb: >At 04:33 PM 11/9/2005 -0800, Guido van Rossum wrote: > > >>On 11/9/05, Phillip J. Eby <[EMAIL PROTECTED]> wrote: >> >> >>>By the way, while we're on this subject, can we make the optimization >>>options be part of the compile() interface? Right now the distutils has to >>>actually exec another Python process whenever you want to compile >>>code with >>>a different optimization level than what's currently in effect, whereas if >>>it could pass the desired level to compile(), this wouldn't be necessary. >>> >>> >>Makes sense to me; we need a patch of course. >> >> > >But before we can do that, it's not clear to me if it should be part of the >existing "flags" argument, or whether it should be separate. Similarly, >whether it's just going to be a level or an optimization bitmask in its own >right might be relevant too. > >For the current use case, obviously, a level argument suffices, with 'None' >meaning "whatever the command-line level was" for backward >compatibility. And I guess we could go with that for now easily enough, >I'd just like to know whether any of the AST or optimization mavens had >anything they were planning in the immediate future that might affect how >the API addition should be structured. > > > I'm using a totally different approach for the above problem. I have implemented two functions in the sys module, that make the startup flags accessible at runtime. This also solves some other problems I had, as you will see in the examples below:
The first function makes most of the flags readable (I have ommited the flags, that are documented as deprecated in the code): sys.getrunflag(name) -> integer Return one of the interpreter run flags. Possible names are 'Optimize', 'Verbose', 'Interactive', 'IgnoreEnvironment', 'Debug', 'DivisionWarning', 'NoSite', 'NoZipImport', 'UseClassExceptions', 'Unicode', 'Frozen', 'Tabcheck'. getrunflag('Optimize') for example returns the current value of Py_OptimizeFlag. The second function makes a few flags writable: sys.setrunflag(name, value) -> integer Set an interpreter run flag. The only flags that can be changed at runtime are Py_VerboseFlag ('Verbose') and Py_OptimizeFlag ('Optimize'). Returns the previous value of the flag. As you can see, I have also introduced the new flag Py_NoZipImport that can be activated with -Z at startup. This bypasses the activation of zipimport and is very handy, if you edit modules stored in the filesystem, that are normally imported from a zip archive and you want to test your modifications. With this flag, there is no need to delete, rename or update the zip archive or to modify sys.path to ensure that your changed modules are imported from the filesystem and not from the zip archive. And here are a few usable examples for the new functions: 1.) You have an application, that does a huge amount of imports and some of them are mysterious, so you want to track them in verbose mode. You could start python with -v or -vv, but then you get hundreds or thousands of lines of output. Instead, you can do the following: import sys import ... import ... oldval = sys.setrunflag('Verbose', 1) # -v, use 2 for -vv import ... import ... sys.setrunflag('Verbose', oldval) import ... import ... Now, you get only verbose messages for the imports that you want to track. 2.) You need to generate optimized byte code (without assertions and docstrings) from a source code, no matter how the interpreter was started: import sys ... source = ... oldval = sys.setrunflag('Optimize', 2) # -OO, use 1 for -O bytecode = compile(source, ...) sys.setrunflag('Optimize', oldval) ... 3.) You have to build a command line for the running application (e.g. for registration in the registry) and need to check, if you are running a script or a frozen executable (this assumes, that your freeze tool sets the Py_FrozenFlag): import sys ... if sys.getrunflag('Frozen'): commandline = sys.executable else: commandline = '%s %s' % (sys.executable, sys.argv[0]) ... NOTE: My own freeze tool sib.py, which is part of the VendorID package (www.riverbankcomputing.co.uk/vendorid) doesn't set the Py_FrozenFlag yet. I will provide an update soon. ---- And now back to the original subject: I have done nearly the same changes, that Osvaldo provided with his patch and I would highly appreciate if this patch goes into the next release. The main reason why I changed the import behavior was pythonservice.exe from the win32 extensions. pythonservice.exe imports the module that contains the service class, but because pythonservice.exe doesn't run in optimized mode, it will only import a .py or a .pyc file, not a .pyo file. Because we always generate bytecode with -OO at distribution time, we either had to change the behavior of pythonservice.exe or change the import behavior of Python. It is essential for us to remove assertions and docstrings in our commercial Python applications at distribution time, because assertions are meaningful only at development time and docstrings may contain implementation details, that our customers should never see (this makes reverse engineering a little bit harder, but not impossible). Another important reason is, that the number of files in the standard library is reduced dramatically. I can have .py and .pyc files in lib/pythonX.Y containing assertions and docstrings and put optimized code in lib/pythonXY.zip. Then, if I need docstrings, I just bypass zipimport as described above with -Z. On the customer site, we only need to install the zip archive (our customers are not developers, just end users; they may not even recognize that we use Python for application development). Guido, if it was intentional to separate slightly different generated bytecode into different files and if you have good reasons for doing this, why have I never seen a .pyoo file :-) For instance, nobody would give the output of a C compiler a different extension when different compiler flags are used. I would appreciate to see the generation of .pyo files completely removed in the next release. Just create them as .pyc files, no matter if -O or -OO is used or not. At runtime, Python should just ignore (jump over?) assert statements when started with -O or ignore assert statements and docstrings when started with -OO if they are in the .pyc file. ---- There are two reasons, why I haven't started a discussion about those issues earlier on this list: 1.) I have done those changes (and a lot of other minor and major changes) in Python 2.3.x at a time when Python 2.4.x came up and we still use Python 2.3.5. I just wanted to wait until I have the next release of our Python runtime environment (RTE) ready that will contain the most recent Python version. 2.) In the last months, my time was limited, because I first had to stabilize the current RTE (developers and customers were waiting on it). A few notes about this Python runtime environment: The RTE that I maintain contains everything (mainly, but not only Python) that is needed to run our applications on the customer site. The applications are distributed as frozen binaries either containing the whole application or only the frozen main script together with an additional zip archive holding the application specific modules and packages. Tools like py2exe or cx_Freeze follow a different approach: they always package a kind of runtime environment together with the application. There is nothing bad with this approach, but if you provide more than one application, you waste resources and it may be harder to maintain (bugfixes, updates, etc.). Our RTE currently runs on AIX, HP-UX, IRIX, Windows and Linux, SunOS/Solaris will follow in the near future. It gets installed together with the application(s) on the customer site if it is not already there in the appropriate version. It is vendor specific (more in the sense of a provider, not necessarily in the sense of a seller) and shouldn't interfere with any other software already installed on the target site. This RTE is the result of years of experience of commercial application development with Python (we started with Python-1.3.x a long long time ago). My guideline is: Application developers and/or customers should not worry about the requirements needed to get the applications running on the target site. Developers should spend all their time for application development and customers can expect a complete installation with only a very small set of system requirements/dependencies. In general, I would like to discuss those things that increase Python's usability especially in a commercial environment and things that make Python more consistent across platforms (e.g. unifying the filesystem layout on Windows and UNIX/Linux), but I don't know if this is the right mailing list. Ulli _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com