Re: calling extension's autoconf/make from distutils
On Sep 20, 10:43 pm, Gary Jefferson <[EMAIL PROTECTED]>
wrote:
> I've got a python extension that comes with its own standard autoconf/
> automake system, and I can "python setup.py build" just fine with it
> as long as I have previously done "./configure" in that directory.
>
> However, 'python setup.py bdist_rpm' can't hope to have done './
> configure' first, as it untars and tries to build the extension there.
>
> Is there a hook for bdist_rpm (and friends) that will allow me to
> insert a './configure' in the build process, sometime before it tries
> to build the extension?
>
> Thanks,
> Gary
I ended up simply subclassing 'Extension' and having it os.system('./
configure') before proceeding.
This isn't perfect, as it does the './configure' everytime, but it
works.
Gary
--
http://mail.python.org/mailman/listinfo/python-list
calling extension's autoconf/make from distutils
I've got a python extension that comes with its own standard autoconf/ automake system, and I can "python setup.py build" just fine with it as long as I have previously done "./configure" in that directory. However, 'python setup.py bdist_rpm' can't hope to have done './ configure' first, as it untars and tries to build the extension there. Is there a hook for bdist_rpm (and friends) that will allow me to insert a './configure' in the build process, sometime before it tries to build the extension? Thanks, Gary -- http://mail.python.org/mailman/listinfo/python-list
Re: distutils, extensions, and missing headers
On Sep 20, 12:08 pm, Robert Kern <[EMAIL PROTECTED]> wrote: > Gary Jefferson wrote: > > On Sep 20, 1:22 am, Robert Kern <[EMAIL PROTECTED]> wrote: > >> Use the "headers" keyword to setup() to list theheaderfiles you want > >> installed. > > > I've tried "headers=['header1.h', 'header2.h']" in setup() as well > > as in Extension(), and neither seem to get the files into the bdist or > > bdist_rpm tarball. Am I doing it wrong? I can't seem to find any > > examples that use this via google. > > You might need an accurate relative path. I'm assuming that you don't actually > keep header1.h and header2.h in the top-level directory next to the setup.py. I think I've got it, just needed to do a MANIFEST.in file as in http://groups.google.com/group/comp.lang.python/browse_thread/thread/64ed59544c4c047e/e329ad06c50cff46?lnk=gst&q=distutils+header&rnum=1#e329ad06c50cff46 . Apparently, the headers arg doesn't affect sdist or bdist. With that, the headers get included in the tarballs, and bdist_rpm no longer complains about them. Unfortunately, it now complains about missing .o files. These .o's are built automatically via 'setup.py build', but not via 'setup.py bdist_rpm', for reasons I can't fathom at the moment. But if I figure it out, I'll post here. Gary -- http://mail.python.org/mailman/listinfo/python-list
Re: distutils, extensions, and missing headers
Robert, thanks for the help! On Sep 20, 1:22 am, Robert Kern <[EMAIL PROTECTED]> wrote: > > Use the "headers" keyword to setup() to list the header files you want > installed. I've tried "headers=['header1.h', 'header2.h']" in setup() as well as in Extension(), and neither seem to get the files into the bdist or bdist_rpm tarball. Am I doing it wrong? I can't seem to find any examples that use this via google. > For other files, it depends on where you need them to go. If you want the data > files to be inside the package, you should use the "package_data" keyword. It > was introduced in Python 2.4, so if you need to support pre-2.4 Pythons, there > are recipes floating around to do so more nicely. > > http://docs.python.org/dist/node12.html > > For other things (and hopefully, you can live with package data), use > "data_files": > > http://docs.python.org/dist/node13.html I also tried using data_files to get the headers included, but can't seem to get that to work either. No errors are reported for either method. Thanks, Gary -- http://mail.python.org/mailman/listinfo/python-list
distutils, extensions, and missing headers
My setup.py (with extension) seems to work great for build and install, but for bdist_rpm, compilation of the extension fails because some of the headers needed to build the extension aren't in the bdist tarball. I've tried adding a 'depends=[]' to the Extension definition with these header files present, but they still don't get put in the tarball. What's the key to getting headers or other [non-python] files included in a bdist? Thanks, Gary -- http://mail.python.org/mailman/listinfo/python-list
Re: selective logger disable/enable
Gary Jefferson wrote: > So maybe I don't have all this figured out quite as well as I thought. > What I really want to do is set an environment variable, MYDEBUG, which > contains a list of wildcarded logger names, such as "a.*.c a.d" (which > becomes ['a.*.c', 'a.d'], and then selectively crank the loglevel up to > DEBUG for those that match. > > In order to do that, I think I need something kind of like filters, but > for logger name... I'm not seeing how to do this, even after playing > with variations of test15, 18, and 21. > > What I do have working at the moment is passing a list of non-wildcard > logger names, i.e., doing the wildcard expansion manually such as > "['a.b.c', 'a.c.c', 'a.f.c', 'a.d']. Is there anyway to automate this > dynamically? > > BTW, I do understand that 'a.d' is essentially equivalent to 'a.d.*', > but I'm dealing with a hierarchy that doesn't always tidy up like that. > For example, I have top.network.server.http, top.network.server.smtp, > top.network.client.http, and top.network.client.smtp. Obviously, if I > only want server or client DEBUG msgs, this is easy. And sometimes > that's exactly what I want. Other times, I want only smtp DEBUG msgs, > and the hierarchy won't help me get that (unless I break it for just > getting client or server msgs), etc. So I would really like to figure > out how to do 'a.*.c'. > > Any ideas? > > Thanks again, > Gary Okay, I think I'm back on track again: I can do regex comparisons against the name field of the LogRecord instance handed to me by filter(). Gary -- http://mail.python.org/mailman/listinfo/python-list
Re: logging module and doctest
Peter Otten wrote: > Peter Otten wrote: > > > Gary Jefferson wrote: > > > >> I've written a logging.filter and would like to use doctest on it > >> (using a StreamHandler for stdout), but this doesn't seem possible. > >> Output from the logger seems to disappear (running the doctest strings > >> through the interpreter as-is yields expected results). I assume this > >> is because doctest does something with logging. > > > > It redirects stdout to a StringIO subclass to capture the output. > > > >> Is there any way to make these work together? > > > > Using the StreamHandler with something like > > > > class WrapStdOut(object): > > def __getattr__(self, name): > > return getattr(sys.stdout, name) > > > > instead of sys.stdout directly should work. > > Or create the StreamHandler inside your doctest where sys.stdout is already > redirected. > > Peter I was creating the StreamHandler inside doctest. Turns out, however, that I hadn't imported 'sys', and so it didn't work. Its a little strange that this didn't result in an error, but there you have it. Thanks for the tip that led me to discover this. Gary -- http://mail.python.org/mailman/listinfo/python-list
Re: selective logger disable/enable
So maybe I don't have all this figured out quite as well as I thought. What I really want to do is set an environment variable, MYDEBUG, which contains a list of wildcarded logger names, such as "a.*.c a.d" (which becomes ['a.*.c', 'a.d'], and then selectively crank the loglevel up to DEBUG for those that match. In order to do that, I think I need something kind of like filters, but for logger name... I'm not seeing how to do this, even after playing with variations of test15, 18, and 21. What I do have working at the moment is passing a list of non-wildcard logger names, i.e., doing the wildcard expansion manually such as "['a.b.c', 'a.c.c', 'a.f.c', 'a.d']. Is there anyway to automate this dynamically? BTW, I do understand that 'a.d' is essentially equivalent to 'a.d.*', but I'm dealing with a hierarchy that doesn't always tidy up like that. For example, I have top.network.server.http, top.network.server.smtp, top.network.client.http, and top.network.client.smtp. Obviously, if I only want server or client DEBUG msgs, this is easy. And sometimes that's exactly what I want. Other times, I want only smtp DEBUG msgs, and the hierarchy won't help me get that (unless I break it for just getting client or server msgs), etc. So I would really like to figure out how to do 'a.*.c'. Any ideas? Thanks again, Gary On Jan 23, 3:01 am, "Vinay Sajip" <[EMAIL PROTECTED]> wrote: > Glad the tests/examples (log_testxx.py) helped. When I get a > chance, I will try to work some of them into the docs... > > Best regards, > > Vinay Sajip -- http://mail.python.org/mailman/listinfo/python-list
logging module and doctest
I've written a logging.filter and would like to use doctest on it (using a StreamHandler for stdout), but this doesn't seem possible. Output from the logger seems to disappear (running the doctest strings through the interpreter as-is yields expected results). I assume this is because doctest does something with logging. Is there any way to make these work together? Gary -- http://mail.python.org/mailman/listinfo/python-list
Re: selective logger disable/enable
Vinay Sajip wrote: > > BTW I would also advise reading PEP-282 to understand more about the > logging approach. You've been most helpful, Vinay. The PEP section on Filters states that I can do what I've been trying to do with filters, but doesn't provide enough information to do it (or, at least, I'm too thick to guess at how it would work by looking at the API and PEP and trying a dozen different ways). Luckily, the examples you point to from your original package do provide enough info; log_test15.py held the key. I still feel like it would be more intuitive if filters were inherited down the hierarchy instead of having to go through the extra steps of getting at the root handler first, but I'm sure there are good reasons for not doing this. One more question, is there any way to get the list of all named loggers (from the Manager, perhaps)? Or... maybe I don't need this, either, as MatchFilter (log_test18.py) seems to do what I was thinking I need the list of logger names for... most excellent. Thanks, Gary BTW, the python logging module is one of the best readily available loggers I've come across in any language. -- http://mail.python.org/mailman/listinfo/python-list
Re: selective logger disable/enable
Vinay Sajip wrote:
>
> I don't know enough about your target environment and application to
> necessarily give you the best advice, but I'll make some general
> comments which I hope are useful. You seem to be thinking that loggers
> are binary - i.e. you turn them on or off. But they can be controlled
> more finely than that; you should be able to get the effect that you
> want by using the different logging levels available judiciously, as
> well as using the fact that loggers inhabit a named hierarchy. Note
> that loggers, by default, inherit the level of the first ancestor
> logger which has an explicitly set level (by ancestor, I mean in the
> name hierarchy). The root logger's default level is WARNING, so by
> default all loggers will work at this level. [N.B. You can also set
> levels for individual handlers, e.g. to ensure that only CRITICAL
> conditions are emailed to a specified email address using SMTPHandler,
> or that ERROR conditions and above are written to file but not to
> console.]
>
> So, for your networking scenario, let's suppose you do the following:
> Have all network loggers live in the hierarchy namespace below
> "network" (e.g. "network", "network.tcp", "network.http" etc.). By
> default, all of these will only log events of severity WARNING and
> above. Also, suppose you log events in your application code, which are
> sometimes but not always of interest, at level DEBUG or level INFO.
> Then, these events will never show up in the logging output, since they
> are below WARNING in severity. Subsequently, if you want to turn on
> logging verbosity for the network code only, you can arrange, via a
> command-line switch or environment variable or configuration file, to
> do
>
> logging.getLogger("network").setLevel(logging.DEBUG)
>
> whereupon you will start seeing events from the networking code at
> severity DEBUG and INFO. This will affect all loggers in the "network"
> hierarchy whose levels you have not explicitly set (so that they will
> get the effective level of the first ancestor which has a level
> explicitly set - the logger named "network").
>
> If all you are interested in is turning on verbosity based on different
> event severities (levels), you should not need to use or set Filters.
> Filters are for use only when levels don't meet your use case
> requirements.
>
> Best regards,
>
> Vinay Sajip
Vinay, okay, I think what you described will work out for me -- thank
you very much for the explanation.
I am still a bit confused about Filters, though. It seems they are a
bit of an anomoly in the hierarchical view of loggers that the API
supports elsewhere, i.e., filters don't seem to inherit... Or am I
missing something again? Here's a quick example:
import logging
log1 = logging.getLogger("top")
log2 = logging.getLogger("top.network")
log3 = logging.getLogger("top.network.tcp")
log4 = logging.getLogger("top.network.http")
log5 = logging.getLogger("top.config")
log6 = logging.getLogger("top.config.file")
logging.basicConfig(level=logging.DEBUG,
format='%(asctime)s %(levelname)s %(message)s')
filter = logging.Filter("top.network")
log1.addFilter(filter) # only affects log1, do this for each of log2-7
too?
log1.debug("I'm top")
log2.debug("I'm top.network")
log3.debug("I'm top.network.tcp")
log4.debug("I'm top.network.http")
log5.debug("I'm top.config")
log6.debug("I'm top.config.file")
This is only for the binary case (and I think if I ignore the binary
case and filters altogether as you suggested), but it really would be
nice to be able to squelch /all/ output from loggers that belong to
certain parts of the namespace by using a filter as above (which I
can't get to work).
Perhaps if I set up a basicConfig with a loglevel of nothing, I can get
this to approximate squelching of everything but that which I
explicitly setLevel (which does inherit properly).
In other words, the addFilter/removeFilter part of the API seems rather
impotent if it can't be inherited in the logging namespaces. In fact,
I can't really figure out a use case where I could possibly want to use
it without it inheriting. Obviously I'm missing something. I'm sure
I've consumed more attention that I deserve already in this thread,
but, do you have any pointers which can enlighten me as to how to
effectively use addFilter/removeFilter?
many thanks,
Gary
--
http://mail.python.org/mailman/listinfo/python-list
Re: selective logger disable/enable
Vinay Sajip wrote: > > The documentation for Logger - see > > http://docs.python.org/lib/node406.html > > - shows that there are addFilter() and removeFilter() methods on the > Logger class which you can use to add or remove filters from individual > Logger instances. From the above page (entitled "14.5.1 Logger > Objects"): > > addFilter(filt) > Adds the specified filter filt to this logger. > > removeFilter(filt) > Removes the specified filter filt from this logger. > > The parent section of Section 14.5.1, which is Section 14.5, was the > first search result when I just searched Google for "python logging". Thanks for the reply Vinay. I had been reading those docs prior to posting, but addFilter/removeFilter only work on individual logger instances and handlers. I want to globally enable/disable multiple logger instances, based on their namespaces (see the example I provided). I cannot find a way to do this. I can certainly call addFilter on every instance of every logger throughout the source code, but this is more than inconvenient -- it breaks when new modules are added with their own loggers in their own namespaces (until fixed by manually adding addFilter() to those instances). e.g., suppose you have a source tree with a several components that do network access, and several components that do other things like saving state to disk, accessing the filesystem, etc. And suppose you find a bug that you think is in the networking code. It would be nice to be able to GLOBALLY enable just the loggers that belong in the networking namespace, and disable all others (similar to how Filters are supposed to work for individual loggers). And then to be able to do this with any component, by providing, for example, a command line switch or environment variable. Otherwise, the poor programmer is forced to go and edit every module in the source tree to selectively turn on/off their respecitve loggers. Or am I missing something really obvious about how this is done with the logging module? thanks, Gary -- http://mail.python.org/mailman/listinfo/python-list
selective logger disable/enable
Suppose I have 3 modules that belong to a project, 'A', 'A.a' and 'B',
and each module has its own logger, created with:
module1logger = logging.getLogger('project.A')
and
module2logger = logging.getLogger('project.A.a')
and
module3logger = logging.getLogger('project.B')
And I want to selectively enable/disable these, per module (for
example, I only want logging from A and A.a, or just from B, etc). It
seems like logging.Filter is supposed to let me do this, but I can't
see how to apply it globally. Is there really no other way that to add
a addFilter(filter) call to each of these loggers individually?
logging.basicConfig gets inherited by all loggers, but it doesn't seem
capable of giving a Filter to all loggers. Is there any way to do this?
--
http://mail.python.org/mailman/listinfo/python-list
