Re: Odd behavior with imp.reload and logging
On Thu, Sep 22, 2011 at 3:54 PM, Andrew Berg wrote: > The main program is an IRC bot, which could potentially be in use by > many people in several channels on a network. As it is, the bot can only > connect to one server, but it could probably be set up to connect to any > number of networks. Making a number of quick fixes or changes to one > module could be very irritating to users if the bot has to terminate > each time, especially if those users don't know or care about that > specific module. > Startup time is an issue because it must connect to a network before it > can take any input. Also, many disconnects/reconnects could easily cause > problems (like the network refusing the connection as a DoS prevention > measure). Playing with networking and the desire to reload without restarting? I think Pike may be a good choice for you. It has a C-like syntax but Python-like guts; you use braces to delimit blocks of code, but arrays and mappings and such are first-class objects that you can pass around and use. It's a favorite of mine, but quite off-topic for this mailing list; I'd be happy to help you get started with it. Chris Angelico -- http://mail.python.org/mailman/listinfo/python-list
Re: Environment variables not visible from Python
hello,
On Thu, Sep 22, 2011 at 06:12:01AM +, Steven D'Aprano wrote:
> I don't understand why some environment variables are not visible from
> Python.
>
> [steve@wow-wow ~]$ echo $LINES $COLUMNS $TERM
> 30 140 xterm
> [steve@wow-wow ~]$ python2.6
> Python 2.6.6 (r266:84292, Dec 21 2010, 18:12:50)
> [GCC 4.1.2 20070925 (Red Hat 4.1.2-27)] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
> >>> import os
> >>> (os.getenv('LINES'), os.getenv('COLUMNS'), os.getenv('TERM'))
> (None, None, 'xterm')
I think TERM is inherited from parent shell, but LINES and
COLUMNS are re-created every child shell. IMHO it's normally,
cause TERM will not change in child, but another variables should
changed...
Look at this:
airween@sebulba:~$ export LINES COLUMNS TERM
airween@sebulba:~$ python2.6
Python 2.6.6 (r266:84292, Mar 25 2011, 19:24:58)
[GCC 4.5.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> (os.getenv('LINES'), os.getenv('COLUMNS'), os.getenv('TERM'))
('65', '210', 'rxvt-256color')
a.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Environment variables not visible from Python
On Thu, Sep 22, 2011 at 11:42 AM, Steven D'Aprano
wrote:
> I don't understand why some environment variables are not visible from
> Python.
>
> [steve@wow-wow ~]$ echo $LINES $COLUMNS $TERM
> 30 140 xterm
> [steve@wow-wow ~]$ python2.6
> Python 2.6.6 (r266:84292, Dec 21 2010, 18:12:50)
> [GCC 4.1.2 20070925 (Red Hat 4.1.2-27)] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
import os
(os.getenv('LINES'), os.getenv('COLUMNS'), os.getenv('TERM'))
> (None, None, 'xterm')
>
>
I have this:
$ python
Python 2.6.6 (r266:84292, Sep 15 2010, 16:22:56)
[GCC 4.4.5] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.getenv('LINES')
>>>
$ python -S
Python 2.6.6 (r266:84292, Sep 15 2010, 16:22:56)
[GCC 4.4.5] on linux2
>>> import os
>>> os.getenv('LINES')
'35'
>>>
I hope it helps narrow things down somewhat.
--
regards,
kushal
--
http://mail.python.org/mailman/listinfo/python-list
Environment variables not visible from Python
I don't understand why some environment variables are not visible from
Python.
[steve@wow-wow ~]$ echo $LINES $COLUMNS $TERM
30 140 xterm
[steve@wow-wow ~]$ python2.6
Python 2.6.6 (r266:84292, Dec 21 2010, 18:12:50)
[GCC 4.1.2 20070925 (Red Hat 4.1.2-27)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> (os.getenv('LINES'), os.getenv('COLUMNS'), os.getenv('TERM'))
(None, None, 'xterm')
--
Steven
--
http://mail.python.org/mailman/listinfo/python-list
Re: with statements does not delete the object
y...@zioup.com writes: > Is this the expected behaviour: You can learn the expected behaviour for ‘with’ in the documentation http://docs.python.org/reference/compound_stmts.html#the-with-statement>. > with mylib.token() as t: > do_something > > dir() > > In the last dir(), after the with "loop" is finished, t still shows up... I > expected it to be unreferenced by then. What gives you that expectation, when it's not the case for any of ‘if’, ‘for’, ‘while’, ‘try’, and so on? The set of statements which introduce a new scope is small, and ‘with’ is not one of them. -- \ “No smoothen the lion.” —lion cage, zoo, Czech Republic | `\ | _o__) | Ben Finney -- http://mail.python.org/mailman/listinfo/python-list
Re: Odd behavior with imp.reload and logging
On 2011.09.22 12:09 AM, Chris Angelico wrote: > On-the-fly reloading of modules isn't really one of Python's > strengths. Everyone who asks about it seems to be doing rapid > development/debugging and wanting to save on startup time (as opposed > to, say, running a server and updating code in it while it's active > and serving clients), so the question becomes: Which is more of a > problem, startup delay or the risk that it's not the same as a clean > start? Python doesn't guarantee that your debugging session is going > to be useful - if you reload that module and weird things happen, it > could be because of reload(), not because of a module bug. The main program is an IRC bot, which could potentially be in use by many people in several channels on a network. As it is, the bot can only connect to one server, but it could probably be set up to connect to any number of networks. Making a number of quick fixes or changes to one module could be very irritating to users if the bot has to terminate each time, especially if those users don't know or care about that specific module. Startup time is an issue because it must connect to a network before it can take any input. Also, many disconnects/reconnects could easily cause problems (like the network refusing the connection as a DoS prevention measure). I'm not tied to any particular solution, and it's quite possible I'm missing something since I am still a beginner. -- CPython 3.2.2 | Windows NT 6.1.7601.17640 | Thunderbird 6.0.2 -- http://mail.python.org/mailman/listinfo/python-list
with statements does not delete the object
Is this the expected behaviour: with mylib.token() as t: do_something dir() In the last dir(), after the with "loop" is finished, t still shows up... I expected it to be unreferenced by then. -- Yves. http://www.SollerS.ca/ http://ipv6.SollerS.ca http://blog.zioup.org/ -- http://mail.python.org/mailman/listinfo/python-list
Re: Odd behavior with imp.reload and logging
On Thu, Sep 22, 2011 at 2:47 PM, Andrew Berg wrote: > This makes me wonder what else stays around after a reload and what side > effects there are, though. I would really like to purge everything from > the previous import. The main program has no dependence on the module > whatsoever. > On-the-fly reloading of modules isn't really one of Python's strengths. Everyone who asks about it seems to be doing rapid development/debugging and wanting to save on startup time (as opposed to, say, running a server and updating code in it while it's active and serving clients), so the question becomes: Which is more of a problem, startup delay or the risk that it's not the same as a clean start? Python doesn't guarantee that your debugging session is going to be useful - if you reload that module and weird things happen, it could be because of reload(), not because of a module bug. Ranting Rick will probably expect me to mention Pike here, but I won't. Muahahahaha. oh. I just did. Oh well! ChrisA -- http://mail.python.org/mailman/listinfo/python-list
Re: Python deadlock using subprocess.popen and communicate
On Sep 21, 8:58 pm, Roy Smith wrote: > In article , > Chris Rebert wrote: > > > Popen.poll(): > > Check if child process has terminated. Set **and return** > > returncode attribute. > > [Direct quote from the docs; emphasis added] > > Doh. I read right past that and didn't see it. Thanks for the > correction. I removed the loop and changed it to a single communicate now, still get a deadlock. All threads just stop, even threads that have nothing to do with the ongoing subprocesses. I specifically created a thread that dumps the stack trace of all threads so I can try and view the deadlocks, but that stops responding at the same time as the deadlock. This wasn't happening before so not sure why it changed suddenly. -- http://mail.python.org/mailman/listinfo/python-list
Re: Odd behavior with imp.reload and logging
On 2011.09.21 11:22 PM, Steven D'Aprano wrote: > You could > try something like this (untested): That works. Thanks! This makes me wonder what else stays around after a reload and what side effects there are, though. I would really like to purge everything from the previous import. The main program has no dependence on the module whatsoever. -- CPython 3.2.2 | Windows NT 6.1.7601.17640 | Thunderbird 6.0.2 -- http://mail.python.org/mailman/listinfo/python-list
Re: Odd behavior with imp.reload and logging
On Wed, 21 Sep 2011 20:53:04 -0500, Andrew Berg wrote:
> When using a logger in a module and then using imp.reload to reload the
> module, logger messages are repeated in direct proportion to the number
> of times the modules was loaded. That is, on the first import, the
> message is written once, but on the second run, each message is written
> twice, three times on the third run, and so on.
[...]
> What causes this, and how can I fix it (or at least work around it)? Due
> to the nature of the program, it's much more convenient to reload a
> module than to restart the entire program (especially when testing).
You unconditionally add a handler every time you reload the module,
regardless of whether or not the old handler is still there. You could
try something like this (untested):
import logging
test_logger = logging.getLogger(__name__)
if not test_logger.handlers:
console_handler = logging.StreamHandler()
console_formatter = logging.Formatter(
'{asctime} - {module} - {funcName} - line {lineno} - '
'{levelname} - {message}', style='{'
)
console_handler.setFormatter(console_formatter)
test_logger.addHandler(console_handler)
test_logger.setLevel(logging.DEBUG)
test_logger.info('Test info')
--
Steven
--
http://mail.python.org/mailman/listinfo/python-list
Re: Python deadlock using subprocess.popen and communicate
In article , Chris Rebert wrote: > Popen.poll(): > Check if child process has terminated. Set **and return** > returncode attribute. > [Direct quote from the docs; emphasis added] Doh. I read right past that and didn't see it. Thanks for the correction. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python deadlock using subprocess.popen and communicate
On Wed, Sep 21, 2011 at 8:32 PM, Roy Smith wrote: > My reading of the docs > (http://docs.python.org/release/2.6.7/library/subprocess.html#popen-objec > ts) says that Popen.poll() doesn't return a value, it sets the object's > return code attribute, which you can then interrogate. Popen.poll(): Check if child process has terminated. Set **and return** returncode attribute. [Direct quote from the docs; emphasis added] > More than that, your loop logic looks amazingly complicated for what's > basically a simple thing. I suggest something along the lines of: > > # assuming process.returncode is initially None > while process.returncode is None: > out, err = process.communicate() > Log(out) > > I haven't tested that, but I think (from reading the docs) that's the > right idea. There is no point in communicate()-ing multiple times; communicate() reads until EOF and only returns after subprocess termination! Cheers, Chris -- http://mail.python.org/mailman/listinfo/python-list
Re: Python deadlock using subprocess.popen and communicate
On Sep 21, 8:33 pm, Chris Rebert wrote: > On Wed, Sep 21, 2011 at 8:09 PM, Atherun wrote: > > This is on windows with python 2.6. > > I can't seem to remove a possibility of a deadlock in one of my > > scripts at the moment. Its not a constant deadlock but it appears > > from time to time. The code is below: > > > try: > > > process = > > subprocess.Popen(cmd,stdout=subprocess.PIPE,stderr=subprocess.STDOUT) > > > NotDone = True > > data = [] > > while NotDone: > > NotDone = False > > > result = process.poll() > > if result == None: > > NotDone = True > > if NotDone: > > out, err = process.communicate() > > Log(out) > > > I was able to get the program to write out the stack trace for my > > threads and it is usually deadlocked on sys.stdout.read or > > _internal_poll of subproceess. > > > I've even tried the above, with using "with > > tempfile.NamedTemporaryFiles() as buff:" and writing to the file, but > > it still deadlocks. In the past I work around this by running fewer > > processes asynchronously but I would really like to get this solved so > > I don't have to wait to see if it'll be caused with any changes I > > make. > > > Any tips would be appreciated. > > Your polling loop is completely pointless. The code can be simplified to: > > process = > subprocess.Popen(cmd,stdout=subprocess.PIPE,stderr=subprocess.STDOUT) > out, err = process.communicate() > log(out) > > Depending on the particular program you're running in a subprocess, > you may need to take heed of the Note in the communicate() docs: > "Note: The data read is buffered in memory, so *do not use this > method* if the data size is large or unlimited." [Emphasis added] > > Cheers, > Chris > --http://rebertia.com I'm pretty sure thats the problem, this is a generic catch all function for running subprocesses. It can be anything to a simple command to a complex command with a ton of output. I'm looking for a better solution to handle the case of running subprocesses that have an undetermined amount of output. (i.e. p4 edit's or tools that have an excessive amount of logging) I'll definitely simplify the loop, thanks for the tip on that part. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python deadlock using subprocess.popen and communicate
In article <098f3d78-85f5-44e7-ba72-f2270a24d...@o9g2000vbo.googlegroups.com>, Atherun wrote: > This is on windows with python 2.6. > I can't seem to remove a possibility of a deadlock in one of my > scripts at the moment. Its not a constant deadlock but it appears > from time to time. The code is below: > > try: > > process = > subprocess.Popen(cmd,stdout=subprocess.PIPE,stderr=subprocess.STDOUT) > > NotDone = True > data = [] > while NotDone: > NotDone = False > > result = process.poll() > if result == None: > NotDone = True > if NotDone: > out, err = process.communicate() > Log(out) > > I was able to get the program to write out the stack trace for my > threads and it is usually deadlocked on sys.stdout.read or > _internal_poll of subproceess. > > I've even tried the above, with using "with > tempfile.NamedTemporaryFiles() as buff:" and writing to the file, but > it still deadlocks. In the past I work around this by running fewer > processes asynchronously but I would really like to get this solved so > I don't have to wait to see if it'll be caused with any changes I > make. > > Any tips would be appreciated. My reading of the docs (http://docs.python.org/release/2.6.7/library/subprocess.html#popen-objec ts) says that Popen.poll() doesn't return a value, it sets the object's return code attribute, which you can then interrogate. More than that, your loop logic looks amazingly complicated for what's basically a simple thing. I suggest something along the lines of: # assuming process.returncode is initially None while process.returncode is None: out, err = process.communicate() Log(out) I haven't tested that, but I think (from reading the docs) that's the right idea. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python deadlock using subprocess.popen and communicate
On Wed, Sep 21, 2011 at 8:09 PM, Atherun wrote: > This is on windows with python 2.6. > I can't seem to remove a possibility of a deadlock in one of my > scripts at the moment. Its not a constant deadlock but it appears > from time to time. The code is below: > > try: > > process = > subprocess.Popen(cmd,stdout=subprocess.PIPE,stderr=subprocess.STDOUT) > > NotDone = True > data = [] > while NotDone: > NotDone = False > > result = process.poll() > if result == None: > NotDone = True > if NotDone: > out, err = process.communicate() > Log(out) > > I was able to get the program to write out the stack trace for my > threads and it is usually deadlocked on sys.stdout.read or > _internal_poll of subproceess. > > I've even tried the above, with using "with > tempfile.NamedTemporaryFiles() as buff:" and writing to the file, but > it still deadlocks. In the past I work around this by running fewer > processes asynchronously but I would really like to get this solved so > I don't have to wait to see if it'll be caused with any changes I > make. > > Any tips would be appreciated. Your polling loop is completely pointless. The code can be simplified to: process = subprocess.Popen(cmd,stdout=subprocess.PIPE,stderr=subprocess.STDOUT) out, err = process.communicate() log(out) Depending on the particular program you're running in a subprocess, you may need to take heed of the Note in the communicate() docs: "Note: The data read is buffered in memory, so *do not use this method* if the data size is large or unlimited." [Emphasis added] Cheers, Chris -- http://rebertia.com -- http://mail.python.org/mailman/listinfo/python-list
Python deadlock using subprocess.popen and communicate
This is on windows with python 2.6. I can't seem to remove a possibility of a deadlock in one of my scripts at the moment. Its not a constant deadlock but it appears from time to time. The code is below: try: process = subprocess.Popen(cmd,stdout=subprocess.PIPE,stderr=subprocess.STDOUT) NotDone = True data = [] while NotDone: NotDone = False result = process.poll() if result == None: NotDone = True if NotDone: out, err = process.communicate() Log(out) I was able to get the program to write out the stack trace for my threads and it is usually deadlocked on sys.stdout.read or _internal_poll of subproceess. I've even tried the above, with using "with tempfile.NamedTemporaryFiles() as buff:" and writing to the file, but it still deadlocks. In the past I work around this by running fewer processes asynchronously but I would really like to get this solved so I don't have to wait to see if it'll be caused with any changes I make. Any tips would be appreciated. -- http://mail.python.org/mailman/listinfo/python-list
Re: Odd behavior with imp.reload and logging
On Thu, Sep 22, 2011 at 12:44 PM, Andrew Berg wrote: > The reload isn't controlled by the module, but I have no problem > clearing out any loggers at the beginning. I'm thinking more along the lines of closing them in the old module before firing imp.reload() - maybe have a function in the module that cleans itself up and then reloads itself. And yes, I had been thinking of closing the handler, but I don't know the logging module much. ChrisA -- http://mail.python.org/mailman/listinfo/python-list
Re: Odd behavior with imp.reload and logging
On 2011.09.21 08:57 PM, Chris Angelico wrote: > Unfortunately, Python doesn't really like modules to be reloaded. Are > you able to explicitly close the logger before reloading? The reload isn't controlled by the module, but I have no problem clearing out any loggers at the beginning. I'm looking through the logging docs, but I don't see what you mean by explicitly closing the logger. Handler.close() doesn't help. -- CPython 3.2.2 | Windows NT 6.1.7601.17640 | Thunderbird 6.0.2 -- http://mail.python.org/mailman/listinfo/python-list
Re: Odd behavior with imp.reload and logging
On Thu, Sep 22, 2011 at 11:53 AM, Andrew Berg wrote: > What causes this, and how can I fix it (or at least work around it)? Due > to the nature of the program, it's much more convenient to reload a > module than to restart the entire program (especially when testing). > Unfortunately, Python doesn't really like modules to be reloaded. Are you able to explicitly close the logger before reloading? ChrisA -- http://mail.python.org/mailman/listinfo/python-list
Odd behavior with imp.reload and logging
When using a logger in a module and then using imp.reload to reload the
module, logger messages are repeated in direct proportion to the number
of times the modules was loaded. That is, on the first import, the
message is written once, but on the second run, each message is written
twice, three times on the third run, and so on.
With this code:
> import logging
>
> test_logger = logging.getLogger(__name__)
> console_handler = logging.StreamHandler()
> console_formatter = logging.Formatter('{asctime} - {module} - {funcName} -
> line {lineno} - {levelname} - {message}', style='{')
> console_handler.setFormatter(console_formatter)
> test_logger.addHandler(console_handler)
> test_logger.setLevel(logging.DEBUG)
>
> test_logger.info('Test info')
I get this in the interpreter:
> Python 3.2.2 (default, Sep 4 2011, 09:07:29) [MSC v.1500 64 bit (AMD64)] on
> win32
> Type "help", "copyright", "credits" or "license" for more information.
import os, imp
import test2
> 2011-09-21 20:51:02,421 - test2 - - line 11 - INFO - Test info
imp.reload(test2)
> 2011-09-21 20:51:10,665 - test2 - - line 11 - INFO - Test info
> 2011-09-21 20:51:10,665 - test2 - - line 11 - INFO - Test info
>
What causes this, and how can I fix it (or at least work around it)? Due
to the nature of the program, it's much more convenient to reload a
module than to restart the entire program (especially when testing).
--
CPython 3.2.2 | Windows NT 6.1.7601.17640 | Thunderbird 6.0.2
PGP/GPG Public Key ID: 0xF88E034060A78FCB
--
http://mail.python.org/mailman/listinfo/python-list
[ANNC] pynguin-0.12 (fixes problems running on Windows)
Pynguin is a python-based turtle graphics application. It combines an editor, interactive interpreter, and graphics display area. It is meant to be an easy environment for introducing some programming concepts to beginning programmers. http://pynguin.googlecode.com/ This release fixes problems which prevented the program from working properly on Windows systems. Pynguin is tested with Python 2.7.1 and PyQt 4.8.3 and will use Pygments syntax highlighting if available. Pynguin is released under GPLv3. Changes in pynguin-0.12: Important fixes - Fixed menu items and dialogs not working on Windows - Fixed error on Windows when trying to write backup files Pynguin API - bgcolor() - raise exception if using color component outside of 0-255 - make util.nudge_color() API match util.choose_color() Canvas - allow setting background color - fix custom svg avatars to allow any size avatar Integrated Editor - comment / uncomment line / region Integrated Console - added command to clear history - clears line before writing commands selected in menu Examples - added fractals example file -- http://mail.python.org/mailman/listinfo/python-list
Re: About MAKE_FUNCTION opcode in Python 3
On 21 September 2011 04:09, Terry Reedy wrote: > I agree that doc should be updated for 3.x. Please suggest a new working if > you can, as well as copying the source snippet above. Add me terry.reedy as > nosy. Done: http://bugs.python.org/issue13026 -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: Operator commutativity
Steven D'Aprano wrote: After playing around with various combinations of C1, C2, D1 and D2, it seems to me that the rule is: If the right-hand argument is a subclass of the left-hand argument, AND also defines __radd__ directly rather than inheriting it, then its __radd__ method is called before the left-hand argument's __add__ method. which strikes me as a strangely specific and not very useful rule. I suspect it might be an accident of implementation rather than a deliberate feature. Update to rule: If the right-hand argument is a subclass of the left-hand argument AND a __radd__ is defined anywhere between the left-hand argument's class up to and including the right-hand argument's class, then __radd__ is called, otherwise the left-hand arguments __add__ is called. And it makes perfect sense -- a + b is, after all, an __add__ function; the only reason to call __radd__ instead is if it has been defined for a subclass, and the only reason to define it is because it needs something different from a + b that a doesn't know about. Probably not a clear explanation -- maybe somebody else can dress it up a bit. ~Ethan~ -- http://mail.python.org/mailman/listinfo/python-list
Re: Where can I find a lexical spec of python?
On 21/09/11 20:01, Shaofei Cheng wrote: > Yes, I'm using this document now but I was wondering if there is a formal > spec for lexical grammar? It looks like some part of the doc > "http://docs.python.org/py3k/reference/grammar.html"; is missing. > We can find some replacement in lexical_analysis.html but it seems this > document is write for a python user instead of a guy trying to implement > python. Yes, I understood alright what you were asking. As I said, I don't believe there is any more formal documentation of the Python language than what you have already seen. However, I am of the opinion that it gives you all the information you need, and that a complete formal grammar, while perhaps, for you, easier to interpret, would not add any information. Thomas -- http://mail.python.org/mailman/listinfo/python-list
Re:Re: Where can I find a lexical spec of python?
Yes, I'm using this document now but I was wondering if there is a formal spec for lexical grammar? It looks like some part of the doc "http://docs.python.org/py3k/reference/grammar.html"; is missing. We can find some replacement in lexical_analysis.html but it seems this document is write for a python user instead of a guy trying to implement python. 在 2011-09-22 00:55:45，"Thomas Jollans" 写道： >On 21/09/11 18:33, 程劭非 wrote: >> Thanks Thomas. >> I've read the document >> http://docs.python.org/py3k/reference/lexical_analysis.html >> >> but I worried it might leak some language features like "tab magic". >> >> For I'm working on a parser with JavaScript I need a more strictly defined >> spec. >> >> Currently I have a highlighter here >> ->http://shaofei.name/python/PyHighlighter.html >> (Also the lexer http://shaofei.name/python/PyLexer.html) >> >> As you can see, I just make its behavior align with CPython, but I'm not >> sure what the real python lexical grammar is like. >> >> Does anyone know if there is a lexical grammar spec like other >> languages(e.g. http://bclary.com/2004/11/07/#annex-a)? > >I believe the language documentation on docs.python.org is all the >documentation of the language there is. It may not be completely formal, >and in parts it concentrates not on the actual rules but on the original >implementation, but, as far as I can tell, it tells you everything you >need to know to write a new parser for the Python language, without any >ambiguity. > >You appear to be anxious about implementing the indentation mechanism >correctly. The language documentation describes a behaviour precisely. >What is the problem? > >Thomas > >> >> Please help me. Thanks a lot. >> 在 2011-09-21 19:41:33，"Thomas Jollans" 写道： >>> On 21/09/11 11:44, 程劭非 wrote: Hi, everyone, I've found there was several tokens used in python's grammar(http://docs.python.org/reference/grammar.html) but I didn't see their definition anywhere. The tokens listed here: >>> >>> They should be documented in >>> http://docs.python.org/py3k/reference/lexical_analysis.html - though >>> apparently not using these exact terms. >>> NEWLINE >>> Trivial: U+000A >>> ENDMARKER >>> End of file. >>> NAME >>> documented as "identifier" in 2.3 >>> INDENT DEDENT >>> Documented in 2.1.8. >>> NUMBER >>> Documented in 2.4.3 - 2.4.6 >>> STRING >>> Documented in 2.4.2 >>> I've got some infomations from the source code(http://svn.python.org/projects/python/trunk/Parser/tokenizer.c) but I'm not sure which feature is only for this specified implementaion. (I saw tabstop could be modified with comments using "tab-width:", ":tabstop=", ":ts=" or "set tabsize=", is this feature really in spec?) >>> >>> That sounds like a legacy feature that is no longer used. Somebody >>> familiar with the early history of Python might be able to shed more >>> light on the situation. It is inconsisten with the spec (section 2.1.8): >>> >>> """ >>> Indentation is rejected as inconsistent if a source file mixes tabs and >>> spaces in a way that makes the meaning dependent on the worth of a tab >>> in spaces; a TabError is raised in that case. >>> """ >>> >>> - Thomas >>> -- >>> http://mail.python.org/mailman/listinfo/python-list >> > -- http://mail.python.org/mailman/listinfo/python-list
Re: Operator commutativity
Mark Dickinson wrote: On Sep 21, 2:07 am, Steven D'Aprano wrote: After playing around with various combinations of C1, C2, D1 and D2, it seems to me that the rule is: If the right-hand argument is a subclass of the left-hand argument, AND also defines __radd__ directly rather than inheriting it, then its __radd__ method is called before the left-hand argument's __add__ method. which strikes me as a strangely specific and not very useful rule. I suspect it might be an accident of implementation rather than a deliberate feature. I'm 99.9% sure it's deliberate rather than an accident of implementation. See the note in the docs at: http://docs.python.org/reference/datamodel.html#emulating-numeric-types Support that you're subclassing int, (class MyInt(int): ...) and you want to intercept additions of the form 3 + MyInt(4) (perhaps because you want MyInt to be 'contagious', so that an arithmetic operation that combines an int and a MyInt returns a MyInt). How would you achieve this without this rule? I think Steven's objection was in not calling subclass.__radd__ when __radd__ is inherited rather than directly overridden. Interestingly enough, the following works as expected (at least, as I expected ;)... class C1(object): def __add__(self, other): print "C1.__add__(%r, %r)" % (self, other) class C2(C1): def __radd__(self, other): print "C2.__radd__(%r, %r)" % (self, other) class C3(C2): pass C1() + C2() # --> C2.__radd__(<...C2...>, <...C1...) C1() + C3() # --> C2.__radd__(<...C3...>, <...C1...>) C2() + C3() # --> C1.__add__(<...C2...>, <...C3...>) ~Ethan~ -- http://mail.python.org/mailman/listinfo/python-list
Re: Operator commutativity
On Sep 21, 2:07 am, Steven D'Aprano wrote: > After playing around with various combinations of C1, C2, D1 and D2, it > seems to me that the rule is: > > If the right-hand argument is a subclass of the left-hand argument, AND also > defines __radd__ directly rather than inheriting it, then its __radd__ > method is called before the left-hand argument's __add__ method. > > which strikes me as a strangely specific and not very useful rule. I suspect > it might be an accident of implementation rather than a deliberate feature. I'm 99.9% sure it's deliberate rather than an accident of implementation. See the note in the docs at: http://docs.python.org/reference/datamodel.html#emulating-numeric-types Support that you're subclassing int, (class MyInt(int): ...) and you want to intercept additions of the form 3 + MyInt(4) (perhaps because you want MyInt to be 'contagious', so that an arithmetic operation that combines an int and a MyInt returns a MyInt). How would you achieve this without this rule? -- Mark -- http://mail.python.org/mailman/listinfo/python-list
Re: Re: Where can I find a lexical spec of python?
On Wed, Sep 21, 2011 at 9:33 AM, 程劭非 wrote: > Thanks Thomas. > I've read the document > http://docs.python.org/py3k/reference/lexical_analysis.html > > but I worried it might leak some language features like "tab magic". > > For I'm working on a parser with JavaScript I need a more strictly defined > spec. > > Currently I have a highlighter here > ->http://shaofei.name/python/PyHighlighter.html > (Also the lexer http://shaofei.name/python/PyLexer.html) > > As you can see, I just make its behavior align with CPython, but I'm not sure > what the real python lexical grammar is like. > > Does anyone know if there is a lexical grammar spec like other languages(e.g. > http://bclary.com/2004/11/07/#annex-a)? Closest thing would be http://docs.python.org/py3k/reference/grammar.html Cheers, Chris -- http://mail.python.org/mailman/listinfo/python-list
Re: Where can I find a lexical spec of python?
On 21/09/11 18:33, 程劭非 wrote: > Thanks Thomas. > I've read the document > http://docs.python.org/py3k/reference/lexical_analysis.html > > but I worried it might leak some language features like "tab magic". > > For I'm working on a parser with JavaScript I need a more strictly defined > spec. > > Currently I have a highlighter here > ->http://shaofei.name/python/PyHighlighter.html > (Also the lexer http://shaofei.name/python/PyLexer.html) > > As you can see, I just make its behavior align with CPython, but I'm not sure > what the real python lexical grammar is like. > > Does anyone know if there is a lexical grammar spec like other languages(e.g. > http://bclary.com/2004/11/07/#annex-a)? I believe the language documentation on docs.python.org is all the documentation of the language there is. It may not be completely formal, and in parts it concentrates not on the actual rules but on the original implementation, but, as far as I can tell, it tells you everything you need to know to write a new parser for the Python language, without any ambiguity. You appear to be anxious about implementing the indentation mechanism correctly. The language documentation describes a behaviour precisely. What is the problem? Thomas > > Please help me. Thanks a lot. > 在 2011-09-21 19:41:33，"Thomas Jollans" 写道： >> On 21/09/11 11:44, 程劭非 wrote: >>> Hi, everyone, >>> I've found there was several tokens used in python's >>> grammar(http://docs.python.org/reference/grammar.html) but I didn't see >>> their definition anywhere. The tokens listed here: >> >> They should be documented in >> http://docs.python.org/py3k/reference/lexical_analysis.html - though >> apparently not using these exact terms. >> >>> NEWLINE >> Trivial: U+000A >> >>> ENDMARKER >> End of file. >> >>> NAME >> documented as "identifier" in 2.3 >> >>> INDENT >>> DEDENT >> Documented in 2.1.8. >> >>> NUMBER >> Documented in 2.4.3 - 2.4.6 >> >>> STRING >> Documented in 2.4.2 >> >>> I've got some infomations from the source >>> code(http://svn.python.org/projects/python/trunk/Parser/tokenizer.c) but >>> I'm not sure which feature is only for this specified implementaion. (I >>> saw tabstop could be modified with comments using "tab-width:", >>> ":tabstop=", ":ts=" or "set tabsize=", is this feature really in spec?) >> >> That sounds like a legacy feature that is no longer used. Somebody >> familiar with the early history of Python might be able to shed more >> light on the situation. It is inconsisten with the spec (section 2.1.8): >> >> """ >> Indentation is rejected as inconsistent if a source file mixes tabs and >> spaces in a way that makes the meaning dependent on the worth of a tab >> in spaces; a TabError is raised in that case. >> """ >> >> - Thomas >> -- >> http://mail.python.org/mailman/listinfo/python-list > -- http://mail.python.org/mailman/listinfo/python-list
Re:Re: Where can I find a lexical spec of python?
Thanks Thomas. I've read the document http://docs.python.org/py3k/reference/lexical_analysis.html but I worried it might leak some language features like "tab magic". For I'm working on a parser with JavaScript I need a more strictly defined spec. Currently I have a highlighter here ->http://shaofei.name/python/PyHighlighter.html (Also the lexer http://shaofei.name/python/PyLexer.html) As you can see, I just make its behavior align with CPython, but I'm not sure what the real python lexical grammar is like. Does anyone know if there is a lexical grammar spec like other languages(e.g. http://bclary.com/2004/11/07/#annex-a)? Please help me. Thanks a lot. 在 2011-09-21 19:41:33，"Thomas Jollans" 写道： >On 21/09/11 11:44, 程劭非 wrote: >> Hi, everyone, >> I've found there was several tokens used in python's >> grammar(http://docs.python.org/reference/grammar.html) but I didn't see >> their definition anywhere. The tokens listed here: > >They should be documented in >http://docs.python.org/py3k/reference/lexical_analysis.html - though >apparently not using these exact terms. > >> NEWLINE >Trivial: U+000A > >> ENDMARKER >End of file. > >> NAME >documented as "identifier" in 2.3 > >> INDENT >> DEDENT >Documented in 2.1.8. > >> NUMBER >Documented in 2.4.3 - 2.4.6 > >> STRING >Documented in 2.4.2 > >> I've got some infomations from the source >> code(http://svn.python.org/projects/python/trunk/Parser/tokenizer.c) but >> I'm not sure which feature is only for this specified implementaion. (I >> saw tabstop could be modified with comments using "tab-width:", >> ":tabstop=", ":ts=" or "set tabsize=", is this feature really in spec?) > >That sounds like a legacy feature that is no longer used. Somebody >familiar with the early history of Python might be able to shed more >light on the situation. It is inconsisten with the spec (section 2.1.8): > >""" >Indentation is rejected as inconsistent if a source file mixes tabs and >spaces in a way that makes the meaning dependent on the worth of a tab >in spaces; a TabError is raised in that case. >""" > >- Thomas >-- >http://mail.python.org/mailman/listinfo/python-list -- http://mail.python.org/mailman/listinfo/python-list
Re: Graphing
On 2011-09-21, Frank Ruiz wrote: > I am looking to plot some data points related to system metrics. > Benchmarking, etc. Can someone give some recommendations on a good > way to graph these datapoints in python. I started looking into > matplotlib, however was interested in others experiences. I've always liked gnuplot-py, but I was a gnuplot user for 10+ years before I started using Python. If you don't already know how to use gnuplot, then perhaps gnuplot-py might not seem as intuitive. http://gnuplot-py.sourceforge.net/ -- Grant Edwards grant.b.edwardsYow! Hmmm ... an arrogant at bouquet with a subtle gmail.comsuggestion of POLYVINYL CHLORIDE ... -- http://mail.python.org/mailman/listinfo/python-list
Re: Graphing
On Wed, 21 Sep 2011 08:44:13 -0700, Frank Ruiz wrote: > I am looking to plot some data points related to system metrics. > Benchmarking, etc. Can someone give some recommendations on a good way > to graph these datapoints in python. I started looking into > matplotlib, however was interested in others experiences. I like biggles. Reasonably straightforward, well defaulted. Simple plots require only simple code. biggles.sourceforge.net -- To email me, substitute nowhere->spamcop, invalid->net. -- http://mail.python.org/mailman/listinfo/python-list
Graphing
I am looking to plot some data points related to system metrics. Benchmarking, etc. Can someone give some recommendations on a good way to graph these datapoints in python. I started looking into matplotlib, however was interested in others experiences. -- http://mail.python.org/mailman/listinfo/python-list
Wing IDE 4.0.4 released
Hi, Wingware has released version 4.0.4 of Wing IDE, an integrated development environment designed specifically for the Python programming language. Wing IDE is a cross-platform Python IDE that provides a professional code editor with vi, emacs, and other key bindings, auto-completion, call tips, refactoring, a powerful graphical debugger, version control, unit testing, search, and many other features. **Changes in Version 4.0.4** This is a maintenance release with the following changes: * Shell history operates on whole blocks and filters by entered prefix * Auto-editing (must be enabled with Editor > Auto-Editing preferences): * Auto-enter closing quotes, comments, and parentheses, braces, etc * Tab through auto-entered invocation arguments * Apply quote, comment character, or parenthesis to selection * Use dynamic analysis for goto-definition in the editor, when available * Support ## comments and new block syntax in Mako files * Allow scrolling editor one page below the last line * Refactoring can move symbols to a new file * PyLint panel improvements, including support for version 0.23+ * Commands to copy editor and project file names to clipboard * About 70 other minor features and bug fixes included vi mode improvements See the change log for details. **New Features in Version 4.0** Version 4.0 adds the following new major features: * Refactoring -- Rename/move symbols, extract to function/method, and introduce variable * Find Uses -- Find all points of use of a symbol * Diff/Merge -- Graphical file and repository comparison and merge * Django Support -- Debug Django templates, run Django unit tests, and more * matplotlib Support -- Maintains live-updating plots in shell and debugger * Simplified Licensing -- Includes all OSes and adds Support+Upgrades subscriptions Complete change log: http://wingware.com/pub/wingide/4.0.4/CHANGELOG.txt Details on licensing changes: http://wingware.com/news/2011-02-16 **About Wing IDE** Wing IDE is an integrated development environment designed specifically for the Python programming language. It provides powerful editing, testing, and debugging features that help reduce development and debugging time, cut down on coding errors, and make it easier to understand and navigate Python code. Wing IDE can be used to develop Python code for web, GUI, and embedded scripting applications. Wing IDE is available in three product levels: Wing IDE Professional is the full-featured Python IDE, Wing IDE Personal offers a reduced feature set at a low price, and Wing IDE 101 is a free simplified version designed for teaching beginning programming courses with Python. Version 4.0 of Wing IDE Professional includes the following major features: * Professional quality code editor with vi, emacs, and other keyboard personalities * Code intelligence for Python: Auto-completion, call tips, find uses, goto-definition, error indicators, refactoring, smart indent and rewrapping, and source navigation * Advanced multi-threaded debugger with graphical UI, command line interaction, conditional breakpoints, data value tooltips over code, watch tool, and externally launched and remote debugging * Powerful search and replace options including keyboard driven and graphical UIs, multi-file, wild card, and regular expression search and replace * Version control integration for Subversion, CVS, Bazaar, git, Mercurial, and Perforce * Integrated unit testing with unittest, nose, and doctest frameworks * Django support: Debugs Django templates, provides project setup tools, and runs Django unit tests * Many other features including project manager, bookmarks, code snippets, diff/merge tool, OS command integration, indentation manager, PyLint integration, and perspectives * Extremely configurable and may be extended with Python scripts * Extensive product documentation and How-Tos for Django, matplotlib, Plone, wxPython, PyQt, mod_wsgi, Autodesk Maya, and many other frameworks Please refer to http://wingware.com/wingide/features for a detailed listing of features by product level. System requirements are Windows 2000 or later, OS X 10.3.9 or later (requires X11 Server), or a recent Linux system (either 32 or 64 bit). Wing IDE supports Python versions 2.0.x through 3.2.x and Stackless Python. For more information, see the http://wingware.com/ **Downloads** Wing IDE Professional and Wing IDE Personal are commercial software and require a license to run. A free trial can be obtained directly from the product when launched. Wing IDE Pro -- Full-featured product: http://wingware.com/downloads/wingide/4.0 Wing IDE Personal -- A simplified IDE: http://wingware.com/downloads/wingide-personal/4.0 Wing IDE 101 -- For teaching with Python: http://wingware.com/downloads/wingide-101/4.0 **Purchasing and Upgrading** Wing 4.0 requires an upgrade for Wing IDE 2.x and 3.x users at a cost of 1/2 the full product pricing. Upgrade a lice
Re: Where can I find a lexical spec of python?
On 21/09/11 11:44, 程劭非 wrote: > Hi, everyone, > I've found there was several tokens used in python's > grammar(http://docs.python.org/reference/grammar.html) but I didn't see > their definition anywhere. The tokens listed here: They should be documented in http://docs.python.org/py3k/reference/lexical_analysis.html - though apparently not using these exact terms. > NEWLINE Trivial: U+000A > ENDMARKER End of file. > NAME documented as "identifier" in 2.3 > INDENT > DEDENT Documented in 2.1.8. > NUMBER Documented in 2.4.3 - 2.4.6 > STRING Documented in 2.4.2 > I've got some infomations from the source > code(http://svn.python.org/projects/python/trunk/Parser/tokenizer.c) but > I'm not sure which feature is only for this specified implementaion. (I > saw tabstop could be modified with comments using "tab-width:", > ":tabstop=", ":ts=" or "set tabsize=", is this feature really in spec?) That sounds like a legacy feature that is no longer used. Somebody familiar with the early history of Python might be able to shed more light on the situation. It is inconsisten with the spec (section 2.1.8): """ Indentation is rejected as inconsistent if a source file mixes tabs and spaces in a way that makes the meaning dependent on the worth of a tab in spaces; a TabError is raised in that case. """ - Thomas -- http://mail.python.org/mailman/listinfo/python-list
Where can I find a lexical spec of python?
Hi, everyone, I've found there was several tokens used in python's grammar(http://docs.python.org/reference/grammar.html) but I didn't see their definition anywhere. The tokens listed here: NEWLINE ENDMARKER NAME INDENT DEDENT NUMBER STRING I've got some infomations from the source code(http://svn.python.org/projects/python/trunk/Parser/tokenizer.c) but I'm not sure which feature is only for this specified implementaion. (I saw tabstop could be modified with comments using "tab-width:", ":tabstop=", ":ts=" or "set tabsize=", is this feature really in spec?)-- http://mail.python.org/mailman/listinfo/python-list
Re: Killing threads (was Re: Cancel or timeout a long running regular expression)
On Wed, Sep 21, 2011 at 07:41:50AM +0200, Martin v. Loewis wrote: > > Is it just that nobody's implemented it, or is there a good reason for > > avoiding offering this sort of thing? > > I've been considering to implement killing threads several times for the > last 15 years (I think about it once every year), and every time > I give up because it's too complex and just not implementable. > > To start with, a simple flag in the thread won't do any good. I don't agree. Now if you had written that it wouldn't solve all problem, I could understand that. But I have been in circumstances where a simple flag in the thread implementation would have been helpfull. > It will > not cancel blocking system calls, so people will complain that the > threads they meant to cancel continue to run forever. Instead, you > have to use some facility to interrupt blocking system calls. You > then have to convince callers of those blocking system calls not to > retry when they see that the first attempt to call it was interrupted. > And so on. But this is no longer an implementation problem but a use problem. If someone gets an IOError for writing on a closed pipe and he cathes the exception and retries the write in a loop, then this a problem of the author of this loop, not of exceptions. So if one thread throws an exception to an other thread for instance to indicate a timeout for the latter and the latter catches that exception and tries again what it was doing in a loop, that is entirely the problem of the author of that loop and not of the abilty of one thread throwing an exception in an other. Unless of course there may be a lot of such problematic loops within the internal python code. -- Antoon Pardon -- http://mail.python.org/mailman/listinfo/python-list
Tunneling using paramiko to establish tunnel with lab-router
Hello Friends,
I am looking to implement tunneling in python and using "Paramiko" for
this..
Sending theTunneling script using Paramiko -Python.
It's facing some prob, hangs on running the script..output message it says
".. Missing Handlers". Plz let me know the corrections to be done in this
script to making tunneling successful (I hv hard-coded the jump server IP
address 10.64.116.34 and password - "Cisco123" in the script to test it
once). It's urgent, plz giv me solution ASAP!!
Thanks n regards,
Vinay
import select
import SocketServer
import sys
import paramiko
from threading import Thread
import time
class ForwardServer(SocketServer.ThreadingTCPServer):
daemon_threads = True
allow_reuse_address = True
class Handler (SocketServer.BaseRequestHandler):
def handle(self):
try:
chan =self.ssh_transport.open_channel('direct-tcpip',
(self.chain_host,self.chain_port), self.request.getpeername())
except Exception, e:
print('Incoming request to %s:%d failed: %s' %
(self.chain_host,self.chain_port, repr(e)))
return
if chan is None:
print('Incoming request to %s:%d was rejected by the SSH server.' %
(self.chain_host,self.chain_port))
return
print('Connected! Tunnel open %r -> %r -> %r' %
(self.request.getpeername(), chan.getpeername(), (self.chain_host,
self.chain_port)))
while True:
r, w, x = select.select([self.request, chan],[], [])
if self.request in r:
data = self.request.recv(1024)
if len(data) ==0:
break
chan.send(data)
if chan in r:
data = chan.recv(1024)
if len(data)== 0:
break
self.request.send(data)
chan.close()
self.request.close()
print('Tunnel closed from %r' % (self.request.getpeername(),))
class Tunnel():
def __init__(self,ip='10.64.116.34'):
self.c = paramiko.SSHClient()
self.c.load_system_host_keys()
self.c.set_missing_host_key_policy(paramiko.AutoAddPolicy())
self.c.connect(ip, username='root', password='Cisco123')
self.trans = self.c.get_transport()
print "Inside Tunnel __init__"
def startTunnel(self):
class SubHandler(Handler):
chain_host = '127.0.0.1'
chain_port = 30001
ssh_transport = self.c.get_transport()
print "Inside startTunnel function Subhandler"
def ThreadTunnel():
self.t = ForwardServer(('127.0.0.1', 30001), SubHandler)
self.t.serve_forever()
print "Inside ThreadTunnel-function "
Thread(target=ThreadTunnel).start()
print "printing after Thread start "
def stopTunnel(self):
print "Before stopping tunnel "
t.shutdown()
self.trans.close()
self.c.close()
self.t.socket.close()
if __name__=="__main__":
t= Tunnel()
t.startTunnel()--
http://mail.python.org/mailman/listinfo/python-list
