ANN: Omaha Python Users Group Meeting, July 12 2007 @ 7pm
The second Thursday of the month is quickly approaching. What: Omaha Python Users Group Meeting When: July 12, 2007 - 7pm Why: Topics: + Lightning Talks + discuss/vote on a change in venue + more... Refreshments: + Pizza and Pop will be served. Please make sure and mail the list with toppings and flavor requests for the meeting. Where:Reboot The User 13416 A Street Omaha, NE 68144 map: http://tinyurl.com/2lzv55 More Information on the group is available @ http://www.OmahaPython.org/ -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations.html
itools 0.16.2 released
itools is a Python library, it groups a number of packages into a single
meta-package for easier development and deployment:
itools.catalogitools.i18n itools.tmx
itools.cmsitools.ical itools.uri
itools.csvitools.odf itools.vfs
itools.datatypes itools.pdf itools.web
itools.gettextitools.rest itools.workflow
itools.handlers itools.rss itools.xhtml
itools.html itools.schemas itools.xliff
itools.http itools.stl itools.xml
The big news in this release is the License upgrade; now itools is
available under the terms and conditions of the GPL version 3. Check
the LICENSE.txt file.
Also, a bunch of bugs have been fixed. Some of them have triggered
small backwards incompatible changes; check the UPGRADE-0.16.2 file for
the details.
The package that has changed the most is itools.stl:
- Now STL returns by default the stream of events, not the serialized
byte string. The new parameter mode replaces the boolean parameter
html, it has four modes: 'events' (default), 'xml', 'xhtml' and
'html'.
- Now it is possible to insert a sequence of XML events into ${...}
expressions.
- New public function set_prefix that rewrites the relative URLs of
the given XML events. Its prototype is stl(events, prefix).
- The main function stl accepts a generator or a Parser instance
for the events parameter (not only a list). It also accepts a byte
string for the prefix parameter (not only a Path instance).
- Fix when stl:repeat and stl:if expressions are together in the
same element.
Other changes worth to mention are:
- Reorganize the RELEASE and UPDATE files. The old ones are kept on
the doc folder.
- [itools.catalog] Fix catalog.search(), without parameters it
returns all indexed documents.
- [itools.cms] Various minor fixes for the user interface (Wiki, web
pages view, search form, folder's browse view, login form, user's
edit account form and CSS).
- [itools.handlers] Fix has_handler(path) for special paths, like
has_handler('.').
- [itools.i18n] Rewrite the accept module. Much simpler, now language
negotiation behaves as defined by RFC2616. And accept objects are
correctly serialized.
- [itools.pdf] Various RML fixes.
- [itools.web] The new method context.get_accept_language replaces
context.request.accept_language
- [itools.xhtml] Escape the text nodes when serializing to HTML.
Credits:
- Hervé Cauwelier worked on itools.cms;
- J. David Ibáñez fixed bugs;
- Henry Obein fixed bugs;
- Sylvain Taverne helped fixing bugs;
Resources
-
Download
http://download.ikaaro.org/itools/itools-0.16.2.tar.gz
Home
http://www.ikaaro.org/itools
Mailing list
http://mail.ikaaro.org/mailman/listinfo/itools
Bug Tracker
http://bugs.ikaaro.org/
--
J. David Ibáñez
Itaapy http://www.itaapy.com Tel +33 (0)1 42 23 67 45
9 rue Darwin, 75018 Paris Fax +33 (0)1 53 28 27 88
--
http://mail.python.org/mailman/listinfo/python-announce-list
Support the Python Software Foundation:
http://www.python.org/psf/donations.html
python-cjson 1.0.3x6 released - fixed segfault
This is an enhanced version of python-cjson, the _fast_ JSON encoder/decoder for python. Bugfix: - Fixed segmentation fault when decoding specially crafted unicode strings. I recommend you to upgrade python-cjson, since a critical bug has been fixed. All existing and new unit tests are passed with python 2.3.5, 2.4.3 and 2.5.1 without problems. Windows installer for Python 2.5.1 included. Download, examples and more information: http://python.cx.hu/python-cjson -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations.html
bbfreeze 0.95.2
Hi all, I've just uploaded bbfreeze 0.95.2 to python's cheeseshop. bbfreeze creates standalone executables from python scripts. It's similar in functionality to py2exe or cx_Freeze. This release fixes issues with python installation, where the zlib module is called zlibmodule.so (e.g. fedora core 7). If you previously got zipimport.ZipImportError: can't decompress data; zlib not available when trying to run the frozen executables, try version 0.95.2. *NEW* support for egg files: bbfreeze scans zipped egg files and now includes whole egg files/directories in the distribution. Programs using setuptools' pkg_resources module will now work. It offers the following features: easy installation bbfreeze can be installed with setuptools' easy_install command. zip/egg file import tracking bbfreeze tracks imports from zip files. multiple script freezing bbfreeze can freeze multiple scripts at once. python interpreter included bbfreeze will create an extra executable named 'py', which might be used like the python executable itself. bbfreeze works on windows and UNIX-like operating systems. It currently does not work on OS X. bbfreeze has been tested with python 2.4 and 2.5. bbfreeze will not work with python versions prior to 2.3 as it uses the zipimport feature introduced with python 2.3. Links cheese shop entry: http://cheeseshop.python.org/pypi/ bbfreeze/http://cheeseshop.python.org/pypi/bbfreeze/ homepage: http://systemexit.de/bbfreeze/ mercurial repository: http://systemexit.de/repo/bbfreeze Regards, - Ralf -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations.html
Re: storing pickles in sql data base
On Wed, 11 Jul 2007 17:14:43 -0500, Chris Mellon wrote: [pickle] Protocol 0 (the default) is a text protocol, it's safe to store in a text field or write to a text file. It's not really a text protocol it's more a binary protocol that uses just the ASCII range of byte values. You have to write and read the text files in binary mode or they break if taken across platform boundaries because of the different line endings in Linux and Windows for instance. Ciao, Marc 'BlackJack' Rintsch -- http://mail.python.org/mailman/listinfo/python-list
Anonymous Classes
Hello. I have a library (SQLObject) that stores data as class variables. I would like to set a class variable in a certain context, specific to a certain instance of an object. This would require some sort of anonymous class. I have attempted to use the following code to set the connection string: | class SQLStorage: |def __init__(self, c, debug = False): |config = StorageConfiguration(c) | |connection = sqlobject.connectionForURI(config.databaseString) |if debug: |connection.debug = True | |# I don't know whether this is right. My belief is that we can |# subclass each table and use _connection on them. |class newDatum(DatumTable): |_connection = connection | |class newMetadatum(MetadatumTable): |_connection = connection | |class newChecksum(ChecksumTable): |_connection = connection | |self.__SQLDatum = newDatum |self.__SQLMetadatum = newMetadatum |self.__SQLChecksum = newChecksum This does not work; Python complains that the classes already exist when SQLObject is instantiated for a second time. This has led me to try instantiating a subclass using DatumTable.__class__ and friends, but this requires setting a class name, which, again, limits me to a single instance. I *could* have a counter that appends a number to each class name, but that's a fairly ugly solution; is it possible to create an anonymous class in Python? Thanks, Lachlan. -- http://mail.python.org/mailman/listinfo/python-list
PyCF_DONT_IMPLY_DEDENT
I found this flag in codeop.py PyCF_DONT_IMPLY_DEDENT = 0x200 # Matches pythonrun.h It is apperantly a parser flag used in the PARSER_FLAGS define in pythonrun.c. Questions: what does it actually mean and why is this implementation detail exposed to Python code. I tried to change the value and it caused severe problems. When it is a constant anyway why does it have to be passed to the compile function in codeop.py as a variable? Thx -- http://mail.python.org/mailman/listinfo/python-list
Re: PEP 3107 and stronger typing (note: probably a newbie question)
[EMAIL PROTECTED] (Alex Martelli) writes: If what you wonder about, and the theory mentioned by Clemmer and detailed by the AQF, are both true, then this may help explain why some programmers are fiercely innovative why other, equally intelligent ones, prefer to stick with some plodding, innovation-killing process that only works well for repetitive tasks: the latter group may be the ones who dread errors, and therefore miss the making mistakes, experiencing failures, and learning from them that is how we improve. The idea of designing languages with more and more support for ensuring program correctness is to put the established, repetitive processes into the computer where it belongs, freeing the programmer to be innovative while still providing high assurance of that the program will be right the first time. And some of the most innovative work in software is going into this area today. Also, taking a learn-from-mistakes approach is fine and dandy if the consequences of the mistakes stay contained to those who make them. It's completely different if the consequences are imposed on other people who weren't part of the process. Vast amounts of software today (and I mean the stuff that clpy denizens write for random web servers or desktop apps, not just scary stuff like nuclear reactor code) has the potential to screw people who had nothing to do with the development process. It's unreassuring to hear the the developers say oh cool, we learned from the mistake when that happens. So, it's irresponsible to deliberately choose development processes that externalize risks onto outsiders that way. -- http://mail.python.org/mailman/listinfo/python-list
Re: bool behavior in Python 3000?
Steven D'Aprano schrieb: On Wed, 11 Jul 2007 08:04:33 +0200, Stargaming wrote: No, I think Bjoern just wanted to point out that all those binary boolean operators already work *perfectly*. You just have to emphasize that you're doing boolean algebra there, using `bool()`. Explicit is better than implicit. So we should always write things explicitly like: if bool(bool(some_condition) is True) is True: first_line = str(some_string).split(str(\n))[int(0)] n = int(int(len(list(some_list))) + int(1)) elif bool(bool(some_condition) is False) is True: f = float(math.sin(float(6.0)/float(math.pi))) instead of the less explicit code. I'll try to remember that, thank you for the advice. You're missing like 400 bool(...) is True constructs there! Fatal error, recursion depth reached. Aww! -- http://mail.python.org/mailman/listinfo/python-list
Re: Anonymous Classes
Lachlan Gunn a écrit :
Hello.
I have a library (SQLObject) that stores data as class variables. I would
like to set a class variable in a certain context, specific to a certain
instance of an object. This would require some sort of anonymous class. I
have attempted to use the following code to set the connection string:
| class SQLStorage:
You probably want:
class SQLStorage(object):
|def __init__(self, c, debug = False):
|config = StorageConfiguration(c)
|
|connection = sqlobject.connectionForURI(config.databaseString)
|if debug:
|connection.debug = True
|
|# I don't know whether this is right. My belief is that we can
|# subclass each table and use _connection on them.
|class newDatum(DatumTable):
|_connection = connection
|
|class newMetadatum(MetadatumTable):
|_connection = connection
|
|class newChecksum(ChecksumTable):
|_connection = connection
|
|self.__SQLDatum = newDatum
|self.__SQLMetadatum = newMetadatum
|self.__SQLChecksum = newChecksum
Are you sure you need name mangling here ?
This does not work; Python complains that the classes already exist when
SQLObject is instantiated for a second time. This has led me to try
instantiating a subclass using DatumTable.__class__ and friends, but this
requires setting a class name, which, again, limits me to a single
instance. I *could* have a counter that appends a number to each class
name, but that's a fairly ugly solution; is it possible to create an
anonymous class in Python?
Yes. Using type() :
Help on class type in module __builtin__:
class type(object)
| type(object) - the object's type
| type(name, bases, dict) - a new type
class Toto(object):
... pass
...
class Tutu(object):
... def __init__(self, what):
... self._toto = type('Autototo', (Toto,), dict(_what=what))
...
t = Tutu(42)
t._toto
class '__main__.Autototo'
t._toto._what
42
t._toto()
__main__.Autototo object at 0xb7c3806c
t2 = Tutu(1138)
t2._toto
class '__main__.Autototo'
t2._toto._what
1138
--
http://mail.python.org/mailman/listinfo/python-list
Re: Tuple vs List: Whats the difference?
In article [EMAIL PROTECTED], Shafik [EMAIL PROTECTED] wrote: Hello folks, I am an experienced programmer, but very new to python (2 days). I wanted to ask: what exactly is the difference between a tuple and a list? I'm sure there are some, but I can't seem to find a situation where I can use one but not the other. Lists and tuples are both sequences, and can be used interchangeably in many situations. The big difference is that lists are mutable (can be modified), and tuples are immutable (cannot be modified). Lists also have a richer API (for example the index and count methods). If you are going to need to change the contents of the sequence (appending, removing, altering a value, etc), then use a list. If you require the sequence to be immutable (such as being used as the key in a dictionary), then use a tuple. In general, tuples should be a little more efficient than lists. However, altering a list is more efficient than creating a new tuple, so always prefer tuples does not necessarily lead to a faster overall program. Unless performance is critical, I wouldn't worry about this factor. IMHO, a more important reason to prefer tuples is that since they are immutable, you don't have to worry about side effects. If you call a function and pass it a list, you have no guarantee that the list won't be changed during the function call. With tuples, any attempt to change the tuple will raise an exception. If it is important to the caller that the sequence remain unchanged, then a tuple is a safer representation for that sequence. Dave -- http://mail.python.org/mailman/listinfo/python-list
Re: atexit, sys.exit, sys.exitfunc, reaching end of source code
On 12 jul, 01:32, Wojciech Mu a [EMAIL PROTECTED] wrote: [EMAIL PROTECTED] wrote: I am playing with theatexitmodule but I don't find a way to see the difference between a script calling sys.exit(returncode) and the interpreting arriving at the end of the source code file. This has a semantic difference for my applications. Is there a way to determine in an exithandler (that is registered usingatexit.register) how I exited? Actually sys.exit raises exception SystemExit, but if interpreter reaches end of script exception is not raised. Try something like this: if __name__ == '__main__': exit_code_for_exithandler = None try: #... sys.exit(5) pass #... except SystemExit, e: exit_code_for_exithandler = e.code print sys.exit called else: print end of script w. I am getting quite confused with this SystemExit. If I register a custom sys.excepthook then it is never invoked when I do sys.exit(.) -- http://mail.python.org/mailman/listinfo/python-list
Re: Anonymous Classes
Thanks for your help. After trying your suggestion for a while, I eventually realised that SQLObject was doing something funny in its metaclass that was causing the issue, which I got around by mangling the class names with a counter. -- http://mail.python.org/mailman/listinfo/python-list
web page text extractor
Hello, For a project, I need to develop a corpus of online news stories. I'm looking for an application that, given the url of a web page, copies the rendered text of the web page (not the source HTNL text), opens a text editor (Notepad), and displays the copied text for the user to examine and save into a text file. Graphics and sidebars to be ignored. The examples I have come across are much too complex for me to customize for this simple job. Can anyone lead me to the right direction? Thanks, gk -- http://mail.python.org/mailman/listinfo/python-list
pytz giving incorrect offset and timezone
Hi All, Using pytz, I am facing a problem with Asia/Calcutta, described below. Asia/Calcutta is actually IST, which is GMT + 5:30. But while using pytz, it was recognized as HMT (GMT + 5:53). While I digged into the oslan database, I see the following: # Zone NAME GMTOFF RULES FORMAT [UNTIL] Zone Asia/Calcutta 5:53:28 - LMT 1880 # Kolkata 5:53:20 - HMT 1941 Oct # Howrah Mean Time? 6:30 - BURT 1942 May 15 # Burma Time 5:30 - IST 1942 Sep 5:30 1:00 IST 1945 Oct 15 5:30 - IST Searching in this group, I saw a similar problem posted at http://groups.google.co.in/group/comp.lang.python/browse_thread/thread/55496e85797ac890 without any solutions. I mailed to Stuart and also posted it at the launchpad of pytz, but did not get any response. Unable to know how to proceed further. Any suggestion will be of vital help. thanks Sanjay -- http://mail.python.org/mailman/listinfo/python-list
Samodzielny program pod linuksem
Witam. Zastanawiam się w jaki sposób mogę stworzyć samodzielny program pod linuksem napisany w pythonie, aby można go było uruchomić bez instalacji bibliotek pythona, chodzi mi o coś takiego jak py2exe pod Windowsa. Pozdrawiam JP -- http://mail.python.org/mailman/listinfo/python-list
Re: PEP 3107 and stronger typing (note: probably a newbie question)
Donn Cave [EMAIL PROTECTED] writes: I've wondered if programmers might differ a lot in how much they dread errors, or how they react to different kinds of errors. For example, do you feel a pang of remorse when your program dies with a traceback - I mean, what a horrible way to die? I'm reminded of the time I found out that a program I had worked on had crashed due to a coding bug. It was the control software for an ATM switch. I had moved on from that job a year or so earlier, but I found out about the crash because it took out vast swaths of data communications for the whole US northeast corridor for 2+ days (almost like the extended power outage of 2003) and it was on the front page of the New York Times. The first thing I thought of was that a certain subroutine I had rewritten was the culprit. I got on the phone with a guy I had worked with to ask what the situation was, and I was very relieved to find out that the error was in a part of the code that I hadn't been anywhere near. That program was a mess of spaghetti C code but even more carefully written code keeps crashing the same way. It was one of the incidents that now has me interested in the quest for type-safe languages with serious optimizing compilers, that will allow us to finally trash every line of C code currently in existence ;-). -- http://mail.python.org/mailman/listinfo/python-list
Re: Samodzielny program pod linuksem
stainboy napisał(a): Witam. Zastanawiam się w jaki sposób mogę stworzyć samodzielny program pod linuksem napisany w pythonie, aby można go było uruchomić bez instalacji bibliotek pythona, chodzi mi o coś takiego jak py2exe pod Windowsa. I think you tried to post on pl.comp.lang.python so I'd just set FUT. -- Jarek Zgoda Skype: jzgoda | GTalk: [EMAIL PROTECTED] | voice: +48228430101 We read Knuth so you don't have to. (Tim Peters) -- http://mail.python.org/mailman/listinfo/python-list
Re: Should I use Python for these programs?
CC [EMAIL PROTECTED] wrote: 8 --- Basically, it's almost impossible to use ethernet on other than PCs, on the official LAN. I will have to look into this further though, as higher data rate stuff is a problem with serial. Although I could do 10Mbps RS-422 I suppose. This kind of implies USB into the PC - Haven't seen a 16550 driven much above 1Mb/s As you have a DSP in the system think about USB (with the chips you mentioned) into a DSP then the DSP's serial port converted to RS-485 for every signal (clock, data in and out and sync) and then everything is native DSP on the remote device. That gives you multiple channels of better than Audio response, full duplex. And since your eyes are slower than your ears, that should be good enough... - Hendrik -- http://mail.python.org/mailman/listinfo/python-list
Launch One Application From Another
Hello Guys, I'm looking for the best method to launch one Python application from another? Is something like sub process going to be my best bet? Or is there something more specific for launching one python app from another? Thanks guys, Rob -- http://mail.python.org/mailman/listinfo/python-list
Re: Try a different type of debugger.
Thanks for making this terrific program. Can you add a setting for screen size? (perhaps desired Width and Height in pixels)? I have my screen set at 1024x768 (the web browser window is of course a little bit smaller) and I can't see all the program's rendering at the far right. -Mike On 7/11/07, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote: This one does not need you to set any breakpoints. It records the entire run. Handy, if you don't know where to start. Run the program once and after that all the runtime data is available to you. Which means you can jump to any point in the run and verify the code against runtime data. The user interface is the python code itself viewed inside a Firefox browser. Scroll to a line you're interested in and click the variable. It requires Firefox and Python 2.5. http://codeinvestigator.googlepages.com/main -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations.html -- http://mail.python.org/mailman/listinfo/python-list
Re: 2**2**2**2**2 wrong? Bug?
Gabriel Genellina [EMAIL PROTECTED] wrote: En Wed, 11 Jul 2007 16:39:17 -0300, Paul McGuire [EMAIL PROTECTED] escribió: As was pointed out earlier, left-associativity with exponentiation is of little value, since (2**3)**2 is the same as 2**(3*2) or 2**6=64. Just for curiosity: This helps to find the answer to the problem Which is the largest number that can be written with only 3 digits? Some people stop at 999, others try 99**9 and 9**99, and the winner is 9**9**9, or: Actually I think 9**9E9 is bigger! -- Nick Craig-Wood [EMAIL PROTECTED] -- http://www.craig-wood.com/nick -- http://mail.python.org/mailman/listinfo/python-list
Re: os.wait() losing child?
Jason Zheng [EMAIL PROTECTED] wrote: Nick Craig-Wood wrote: The problem you are having is you are letting Popen do half the job and doing the other half yourself. Except that I never wanted Popen to do any thread management for me to begin with. Popen class has advertised itself as a replacement for os.popen, popen2, popen4, and etc., and IMHO it should leave the clean-up to the users, or at least leave it as an option. Here is a way which works, done completely with Popen. Polling the subprocesses is slightly less efficient than using os.wait() but does work. In practice you want to do this anyway to see if your children exceed their time limits etc. I think your polling way works; it seems there no other way around this problem other than polling or extending Popen class. I think polling is probably the right way of doing it... Internally subprocess uses os.waitpid(pid) just waiting for its own specific pids. IMHO this is the right way of doing it other than os.wait() which waits for any pids. os.wait() can reap children that you weren't expecting (say some library uses os.system())... -- Nick Craig-Wood [EMAIL PROTECTED] -- http://www.craig-wood.com/nick -- http://mail.python.org/mailman/listinfo/python-list
variable naming query
I'ma a newbie python user and would like clarification on variable naming conventions. What is the difference between self.myvariable self._myvariable self.__myvariable and when should I use each of them? -- http://mail.python.org/mailman/listinfo/python-list
Re: profiling a C++ python extension
Hi! It doesn't suffice to compile the python interpreter with -pg, as the module is loaded via dlopen. I solved the problem for my case compiling an executable with embedded python and the module itself. I would wish, that there would be an easier way. Best regards, Michael -- http://mail.python.org/mailman/listinfo/python-list
Re: variable naming query
On Thu, 12 Jul 2007 10:33:03 -, loial [EMAIL PROTECTED] wrote: I'ma a newbie python user and would like clarification on variable naming conventions. What is the difference between self.myvariable This is the convention for public attributes. self._myvariable This is the convention for private attributes. self.__myvariable This causes the name to be mangled in an inconvenient way by the runtime. You probably /don't/ want to name your variables like this, since the consequence is primarily that the result is harder to use. Jean-Paul -- http://mail.python.org/mailman/listinfo/python-list
Re: Rats! vararg assignments don't work
On May 30, 7:29 am, Sion Arrowsmith [EMAIL PROTECTED]
wrote:
samwyse [EMAIL PROTECTED] wrote:
samwysewrote:
I thought that I'd try this:
first, *rest = arglist
Needless to say, it didn't work.
[ ... ]
My use-case is (roughtly) this:
first, *rest = f.readline().split()
return dispatch_table{first}(*rest)
first, rest = f.readline().split(None, 1)
return dispatch_table{first}(*rest.split())
Hey, I like that! Thanks!
--
http://mail.python.org/mailman/listinfo/python-list
Re: PEP 3107 and stronger typing (note: probably a newbie question)
Paul Rubin http://[EMAIL PROTECTED] writes: The idea of designing languages with more and more support for ensuring program correctness is to put the established, repetitive processes into the computer where it belongs, freeing the programmer to be innovative while still providing high assurance of that the program will be right the first time. This seems to make the dangerous assumption that the programmer has the correct program in mind, and needs only to transfer it correctly to the computer. I would warrant that anyone who understands exactly how a program should work before writing it, and makes no design mistakes before coming up with a program that works, is not designing a program of any interest. Also, taking a learn-from-mistakes approach is fine and dandy if the consequences of the mistakes stay contained to those who make them. It's completely different if the consequences are imposed on other people who weren't part of the process. Certainly. Which is why the trend continues toward developing programs such that mistakes of all kinds cause early, obvious failure -- where they have a much better chance of being caught and fixed *before* innocent hands get ahold of them. -- \We have to go forth and crush every world view that doesn't | `\ believe in tolerance and free speech. -- David Brin | _o__) | Ben Finney -- http://mail.python.org/mailman/listinfo/python-list
Re: pytz giving incorrect offset and timezone
On Jul 12, 11:47 am, Sanjay [EMAIL PROTECTED] wrote: Hi All, Using pytz, I am facing a problem with Asia/Calcutta, described below. Asia/Calcutta is actually IST, which is GMT + 5:30. But while using pytz, it was recognized as HMT (GMT + 5:53). While I digged into the oslan database, I see the following: # Zone NAME GMTOFF RULES FORMAT [UNTIL] Zone Asia/Calcutta 5:53:28 - LMT 1880 # Kolkata 5:53:20 - HMT 1941 Oct # Howrah Mean Time? 6:30 - BURT 1942 May 15 # Burma Time 5:30 - IST 1942 Sep 5:30 1:00 IST 1945 Oct 15 5:30 - IST Searching in this group, I saw a similar problem posted athttp://groups.google.co.in/group/comp.lang.python/browse_thread/threa... without any solutions. I mailed to Stuart and also posted it at the launchpad of pytz, but did not get any response. Unable to know how to proceed further. Any suggestion will be of vital help. thanks Sanjay I don't use pytz myself that often so I can't be sure, but I don't think it's a bug in pytz. The problem seems to be that the timezone has changed for the location. Now, without a date as reference, pytz can't know what timezone to use when constructing the tzinfo; you might want a date from the 1800's. When you're constructing the datetime with the tzinfo argument, you're saying: use this timezone as the local timezone. datetime_new (the constructor in C) never calls the tzinfo to verify that the timezone is still valid, it just uses it. On the other hand: When you construct a datetime with datetime.now() and pass a timezone, datetime_now (again, in C) calls the method fromutz() on the tzinfo object. Now the pytz tzinfo object has a reference by which to choose the current timezone for the location, and that's why it's correct when you use datetime.now() but not for a manual construction. A workaround (or maybe the proper way to do it) is to construct the datetime without a tzinfo set, and then use the localize() method on the tzinfo object, this will give you the correct result. tz = pytz.timezone(Asia/Calcutta) mydate = datetime.datetime(2007, 2, 18, 15, 35) print tz.localize(mydate) 2007-02-18 15:35:00+05:30 -- http://mail.python.org/mailman/listinfo/python-list
Re: PEP 3107 and stronger typing (note: probably a newbie question)
John Nagle [EMAIL PROTECTED] wrote: I've worked in big mainframe shops, where an operating system crash caused everything to suddenly freeze, red lights came on all over the building, and a klaxon sounded. I've worked for aerospace companies, where one speaks of defects, not bugs, and there are people around whose job involves getting in high performance aircraft and flying with the software written there. I've worked with car-sized robot vehicles, ones big enough to injure people. This gives one a stronger feeling about wanting to eliminate software defects early. Does this actually cause you to make less goofy mistakes? - I find that no matter how careful I try to be, I still f***k up. not so much in the big things, its the little things that get me... And the errors seem to come in runs - almost like a bad hair day. - Hendrik -- http://mail.python.org/mailman/listinfo/python-list
Re: Tuple vs List: Whats the difference?
Shafik [EMAIL PROTECTED] wrote: Hello folks, I am an experienced programmer, but very new to python (2 days). I wanted to ask: what exactly is the difference between a tuple and a list? I'm sure there are some, but I can't seem to find a situation where I can use one but not the other. Welcome to Python. From a practical point of view, use lists - you will get exceptions if you use a list where a tuple is required, such as a key in a dict, and other more obscure cases. The philosophical differences - tuple as struct vs list as collection argument, makes no practical difference - you can use a list in the same structured way if you want, and lists have more nice methods as they are not immutable as tuples are. Google this Group for Index to see some flame wars. hth - Hendrik -- http://mail.python.org/mailman/listinfo/python-list
Re: allow scripts to use .pth files?
On Jul 8, 3:53 pm, John Machin [EMAIL PROTECTED] wrote: I got the impression that the OP was suggesting that the interpreter look in the directory in which it found the script. [...] I got the impression that the problem was that the package was not only not on sys.path but also not in the same directory as the script that wanted to import it. Otherwise the OP's script.p?h file need only contain .\n (or the path to the directory in which it resided!!), and I doubt that he was proposing something so silly. And as I'm sure you realize, those two impression slightly contradict each other. Anyway, a small modification to my second approach would also work in the case looking for packages in a directory that's located somewhere relative to the one where the script was found: if __name__ == '__main__': import sys, os.path base = sys.path[0] for d in 'lib', 'modules': d2 = os.path.join(base, d) if os.path.isdir(d2): sys.path.append(d2) base = os.path.join(base, '..') for d in 'modules', 'lib': d2 = os.path.join(base, d) if os.path.isdir(d2): sys.path.append(d2) # for debugging print repr(sys.path) (BTW, this is such a fun script to type. My fingers keep typing 'os.path' where I mean 'sys.path' and vice versa.) -- http://mail.python.org/mailman/listinfo/python-list
Re: Portable general timestamp format, not 2038-limited
Ilya Zakharevich [EMAIL PROTECTED] writes: *This* was my question; and with kitchentop book, one cannot expect to find such an answer. If interested, Wiki looks like a good place to look it up. I don't know what Wiki[0] has to do with it, but just to check: URL:http://c2.com/cgi/wiki?WhatIsNorthOfTheNorthPole Nope, can't find any existing page about it. [0] Left unqualified, referring to a site named Wiki refers to *the* Wiki, at URL:http://c2.com/cgi/wiki. If you mean some other subsequent Wiki site, you'll need to be more specific. -- \Consider the daffodil. And while you're doing that, I'll be | `\ over here, looking through your stuff. -- Jack Handey | _o__) | Ben Finney -- http://mail.python.org/mailman/listinfo/python-list
Re: Tuple vs List: Whats the difference?
Dave Baum [EMAIL PROTECTED] writes: In article [EMAIL PROTECTED], Shafik [EMAIL PROTECTED] wrote: what exactly is the difference between a tuple and a list? I'm sure there are some, but I can't seem to find a situation where I can use one but not the other. Lists and tuples are both sequences, and can be used interchangeably in many situations. The big difference is that lists are mutable (can be modified), and tuples are immutable (cannot be modified). Lists also have a richer API (for example the index and count methods). This common misconception falls foul of only considering the implementation details. The choice of when to use one or the other is more one of intent: A tuple is best for heterogeneous sequences, a list is best for homogeneous sequences. Here's a better explanation: URL:http://jtauber.com/blog/2006/04/15/python_tuples_are_not_just_constant_lists -- \ No matter how cynical I get, I can't seem to keep up. -- | `\ Lily Tomlin | _o__) | Ben Finney -- http://mail.python.org/mailman/listinfo/python-list
Re: Re-raising exceptions with modified message
On Jul 8, 8:50 am, Christoph Zwerschke [EMAIL PROTECTED] wrote:
Did you run this?
With Py 2.5 I get a syntax error, and with Py 2.5 I get:
new.__class__ = old.__class__
TypeError: __class__ must be set to a class
-- Chris
Damn, I'd have sworn I ran the final copy that I posted, but
apparently I did manage to have a typo creep in as I was prettifying
the code. You need to lose the '()' in the definition of Empty. (I'd
orignally had it subclass Exception but discovered that it wasn't
needed.)
class Empty: pass
I can't figure out the other complaint, though, as old.__class_ should
be a class. I guess I need to upgrade; I am using PythonWin 2.4.3
(#69, Apr 11 2006, 15:32:42) [MSC v.1310 32 bit (Intel)] on win32.
(Of course, at work they're still stuck on 2.4.2.) Printing
type(old.__class__) gives me type 'classobj'; maybe using
setattr(new, '__class__', old.__class__)
instead of the assignment would work, or maybe it's a bug/feature
introduced in 2.5. (Trying this code:
class Empty(old.__class__): pass
brings us back to the TypeError: function takes exactly 5 arguments
(0 given) message that we're trying to avoid.)
Anyway, running the corrected version under 2.4.X gives me this:
Traceback (most recent call last):
File C:\Python24\Lib\site-packages\pythonwin\pywin\framework
\scriptutils.py, line 310, in RunScript
exec codeObject in __main__.__dict__
File C:\Documents and Settings\dentos\Desktop\scripting
\modify_message.py, line 19, in ?
test(lambda: unicode('\xe4'))
File C:\Documents and Settings\dentos\Desktop\scripting
\modify_message.py, line 16, in test
raise modify_message(e, lambda: str(e) + , sorry!)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe4 in position
0: ordinal not in range(128), sorry!
--
http://mail.python.org/mailman/listinfo/python-list
Re: allow scripts to use .pth files?
On Jul 12, 9:55 pm, samwyse [EMAIL PROTECTED] wrote: On Jul 8, 3:53 pm, John Machin [EMAIL PROTECTED] wrote: I got the impression that the OP was suggesting that the interpreter look in the directory in which it found the script. [...] I got the impression that the problem was that the package was not only not on sys.path but also not in the same directory as the script that wanted to import it. Otherwise the OP's script.p?h file need only contain .\n (or the path to the directory in which it resided!!), and I doubt that he was proposing something so silly. And as I'm sure you realize, those two impression slightly contradict each other. Your sureness is misplaced. Please explain. -- http://mail.python.org/mailman/listinfo/python-list
Re: allow scripts to use .pth files?
On Jul 8, 3:53 pm, John Machin [EMAIL PROTECTED] wrote: I'm curious whether you think that the OP's use of .pth was a typo, and whether you have read this: http://docs.python.org/lib/module-site.html I've read it, but not recently; the syntax of the .pyh files was in the back of my head. I had forgotten about the sitecustomize module, though. Unfortunately for the OP, while the documentation states, After these path manipulations, an attempt is made to import a module named sitecustomize, the import is apparently done *before* the path to the script is prepended to sys.path. (My name isn't Luke, so I don't feel the need to read the source.) I'm guessing that the OP's real question is, How does one do site customizations when one doesn't have write access to the site directories? Lots of programs can be installed and run from ~/bin, but locating ~/lib can be hard, at least for arbitrary values of bin and lib. At this point, someone usually tells me to write a PEP; perhaps the OP would like to try his hand? In keeping with the spirit of the site customizations, I'd specify that all .pyh files get imported, not just one matching the name of the script. (This is more robust in the face of hard-links, packages consisting of multiple scripts, etc.) I'd also suggest an attempt to import a module named mycustomize after the path to the invoked script is prepended to sys.path. -- http://mail.python.org/mailman/listinfo/python-list
Re: allow scripts to use .pth files?
On Jul 12, 7:20 am, John Machin [EMAIL PROTECTED] wrote: On Jul 12, 9:55 pm, samwyse [EMAIL PROTECTED] wrote: On Jul 8, 3:53 pm, John Machin [EMAIL PROTECTED] wrote: I got the impression that the OP was suggesting that the interpreter look in the directory in which it found the script. [...] I got the impression that the problem was that the package was not only not on sys.path but also not in the same directory as the script that wanted to import it. Otherwise the OP's script.p?h file need only contain .\n (or the path to the directory in which it resided!!), and I doubt that he was proposing something so silly. And as I'm sure you realize, those two impression slightly contradict each other. Your sureness is misplaced. Please explain. Oops, you're right. See my other post. -- http://mail.python.org/mailman/listinfo/python-list
Re: bool behavior in Python 3000?
Alan Isaac skrev: Since it is seemingly ignored in most of the comments on this thread, I just want to remind that PEP 285 http://www.python.org/dev/peps/pep-0285/ says this: In an ideal world, bool might be better implemented as a separate integer type that knows how to perform mixed-mode arithmetic. I mentioned Python 3000 since that is an opportunity for an ideal world. You forgot to quote this bit: 4) Should we strive to eliminate non-Boolean operations on bools in the future, through suitable warnings, so that for example True+1 would eventually (in Python 3000) be illegal? = No. There's a small but vocal minority that would prefer to see textbook bools that don't support arithmetic operations at all, but most reviewers agree with me that bools should always allow arithmetic operations. Nis -- http://mail.python.org/mailman/listinfo/python-list
Re: 2**2**2**2**2 wrong? Bug?
Gabriel Genellina [EMAIL PROTECTED] writes: Which is the largest number that can be written with only 3 digits? Some people stop at 999, others try 99**9 and 9**99, and the winner is 9**9**9 Since exponentiation is (I think?) the highest-order function we understand for writing *only* digits on the page, then yes, your answer is right, if no other mathematical notation is allowed. Since others seem to be reading only 3 digits as 3 digits plus other well-defined mathematical notation, here's the best answer given those criteria: URL:http://www.scottaaronson.com/writings/bignumbers.html -- \ I stayed up all night playing poker with tarot cards. I got a | `\ full house and four people died. -- Steven Wright | _o__) | Ben Finney -- http://mail.python.org/mailman/listinfo/python-list
Re: variable naming query
loial [EMAIL PROTECTED] writes: What is the difference between self.myvariable Indicates to the reader that the attribute 'myvariable' is available for use as part of the interface of the object. Prefer this style unless you have good reason in a particular case to do otherwise. self._myvariable Indicates to the reader that the attribute '_myvariable' should not be used as part of the interface to the object. self.__myvariable Indicates to the reader that the attribute '__myvariable' is not available by that name outside the object, and name mangling is automatically done to discourage its use from outside the object. and when should I use each of them? Use each of them to indicate the above conditions where appropriate. Note that none of them will change the nature of the attribute, and Python will allow use of any of them by the correct name. There is no such thing as limited-access attributes in Python; we rely on the maxim that we're all consenting adults here. If an attribute exists in the current scope, it is available for any use regardless of what name you give it. -- \ I wish there was a knob on the TV to turn up the intelligence. | `\ There's a knob called 'brightness' but it doesn't work. -- | _o__) Eugene P. Gallagher | Ben Finney -- http://mail.python.org/mailman/listinfo/python-list
Re: bool behavior in Python 3000?
Terry Reedy [EMAIL PROTECTED] writes: In Guido's opinion (and mine, but his counts 100x), the positive benefits of the current implementation are greater than the net positive benefits of a 'pure' type. See http://www.python.org/dev/peps/pep-0285/ I assume you're referring to: 6) Should bool inherit from int? = Yes. In an ideal world, bool might be better implemented as a separate integer type that knows how to perform mixed-mode arithmetic. However, inheriting bool from int eases the implementation enormously [...further explanation...] I accept Guido's explanation in the PEP, that the implementation is made much easier, as an explanation of why bool inherits from int. I haven't seen people here expressing that they want the opposite. To my mind the more fundamental issue is this one: 4) Should we strive to eliminate non-Boolean operations on bools in the future, through suitable warnings, so that for example True+1 would eventually (in Python 3000) be illegal? = No. There's a small but vocal minority that would prefer to see textbook bools that don't support arithmetic operations at all, but most reviewers agree with me that bools should always allow arithmetic operations. Frustratingly, unlike the above point about inheritance, the PEP gives no explanation of why the answer to this is No. All we get is most reviewers agree, with no explanation of *why*. So, I'm left with the points already made in this thread as to why the answer should be yes, and no source online for an official explanation of the no. -- \ I don't like country music, but I don't mean to denigrate | `\ those who do. And for the people who like country music, | _o__) denigrate means 'put down'. -- Bob Newhart | Ben Finney -- http://mail.python.org/mailman/listinfo/python-list
Re: Screen Scraping Question
On Jul 11, 10:31 pm, Dan Stromberg - Datallegro [EMAIL PROTECTED] wrote: On Wed, 11 Jul 2007 12:45:21 +, jeffbg123 wrote: Hey, I am trying to make a bot for a flash game using python. However I am having some trouble with a screen scraping strategy. Is there an accepted way to compare a full screenshot with the image that I want to locate? It is a math based game, so I just have to check what number, 1-9, appears in the center of the flash game. Is there an easier method to do this? Thanks -Jeff That sounds like an OCR problem, unless the numbers are so consistently rendered that you can just store copies of all 9 (I'd pick a non-lossy format) and compare. The numbers are always rendered the same. So I don't know if OCR is a necessary step. Also, what if I just got the data from the packets? Any disadvantages to that? Any good python packet capturing libraries? Thanks -- http://mail.python.org/mailman/listinfo/python-list
Re: New release: Crunchy 0.9
On Jul 10, 12:37 am, André [EMAIL PROTECTED] wrote: Crunchy 0.9 has been released. It is available athttp://code.google.com/p/crunchy What is Crunchy? Crunchy is a an application that transforms html Python tutorials into interactive session viewed within a browser. We are not aware of any other application (in any language) similar to Crunchy. Currently Crunchy has only been fully tested with Firefox; we know that some browsers simply don't work with it. Crunchy should work with all operating systems - it has been tested fairly extensively on Linux, Windows and Mac OS. What is new in this release? Crunchy has been rewritten from scratch from the previous version (0.8.2), to use a custom plugin architecture. This makes easier to extend and add new functionality. Rather than list the differences with the old release, it is easier to list the essential features of this new version. 1. Crunchy can work best with specially marked-up html tutorials. However, it can now work with any html tutorials - including the official Python tutorial on the python.org site. Html pages can be loaded locally or remotely from anywhere on the Internet. Crunchy uses a combination of Elementtree and BeautifulSoup to process html pages. Non W3C-compliant pages can be handled, but the visual appearance is not guaranteed to reproduce that normally seen using a browser. 2. Crunchy can insert a number of Python interpreters inside a web page. In the default mode, it does that whenever it encounters an html pre element which is assumed to contain some Python code. These interpreters can either share a common environment (e.g. modules imported in one of them are known in the other) or be isolated one from another. 3. Crunchy adds automatic css styling to the Python code - you can look at the official Python tutorial using your browser (all Python code in blue) and compare with what Crunchy displays to give you a good idea. 4. Instead of inserting an interpreter, Crunchy can insert a code editor that can be used to modify the Python code on the page and execute it. The editor can be toggled to become a fairly decent syntax aware editor that can save and load files. 5. Crunchy has a doctest feature where the code inside the pre is taken to be the result of an interpreter session and the user has to write the code so as to make the interpreter session valid; this is useful in a teaching environment. Messages from the Crunchy's doctest are friendlier for Python beginners than the usual tracebacks. 6. Crunchy has a small graphics library that can be imported, either inside an editor or an interpreter, to produce simple graphics (even animations!) inside the browser. 7. For the user that needs better quality graphics, Crunchy supports programs (such as matplotlib) that can create image files; by executing the code, the image produced is loaded inside the browser window. In this capacity, Crunchy could be used as a front end for libraries such as matplotlib. 8. Crunchy supports code execution of files as separate processes, making it suitable to launch gui based application from the browser window. 9. Crunchy's interpreter has an interactive help feature like many python-aware IDEs. 10. Crunchy includes a fairly comprehensive tutorial on its own use, as well as a reference for tutorial writers that want to make their tutorials crunchy-friendlier. 11. As a security feature, crunchy strips all pre-existing javascript code from an html page before displaying it inside the browser window. Bug reports, comments and suggestions are always welcome. André Roberge, for the Crunchy team. Fanmail: http://paddy3118.blogspot.com/2007/07/ive-been-crunched.html - Paddy -- http://mail.python.org/mailman/listinfo/python-list
Re: os.wait() losing child?
Nick Craig-Wood [EMAIL PROTECTED] writes: I think your polling way works; it seems there no other way around this problem other than polling or extending Popen class. I think polling is probably the right way of doing it... It requires the program to wake up every 0.1s to poll for freshly exited subprocesses. That doesn't consume excess CPU cycles, but it does prevent the kernel from swapping it out when there is nothing to do. Sleeping in os.wait allows the operating system to know exactly what the process is waiting for, and to move it out of the way until those conditions are met. (Pedants would also notice that polling introduces on average 0.1/2 seconds delay between the subprocess dying and the parent reaping it.) In general, a program that waits for something should do so in a single call to the OS. OP's usage of os.wait was exactly correct. Fortunately the problem can be worked around by hanging on to Popen instances until they are reaped. If all of them are kept referenced when os.wait is called, they will never end up in the _active list because the list is only populated in Popen.__del__. Internally subprocess uses os.waitpid(pid) just waiting for its own specific pids. IMHO this is the right way of doing it other than os.wait() which waits for any pids. os.wait() can reap children that you weren't expecting (say some library uses os.system())... system calls waitpid immediately after the fork. This can still be a problem for applications that call wait in a dedicated thread, but the program can always ignore the processes it doesn't know anything about. -- http://mail.python.org/mailman/listinfo/python-list
Re: os.wait() losing child?
Jason Zheng [EMAIL PROTECTED] writes:
greg wrote:
Jason Zheng wrote:
Hate to reply to my own thread, but this is the working program
that can demonstrate what I posted earlier:
I've figured out what's going on. The Popen class has a
__del__ method which does a non-blocking wait of its own.
So you need to keep the Popen instance for each subprocess
alive until your wait call has cleaned it up.
The following version seems to work okay.
It still doesn't work on my machine. I took a closer look at the Popen
class, and I think the problem is that the __init__ method always
calls a method _cleanup, which polls every existing Popen
instance.
Actually, it's not that bad. _cleanup only polls the instances that
are no longer referenced by user code, but still running. If you hang
on to Popen instances, they won't be added to _active, and __init__
won't reap them (_active is only populated from Popen.__del__).
This version is a trivial modification of your code to that effect.
Does it work for you?
#!/usr/bin/python
import os
from subprocess import Popen
pids = {}
counts = [0,0,0]
for i in xrange(3):
p = Popen('sleep 1', shell=True, cwd='/home', stdout=file(os.devnull,'w'))
pids[p.pid] = p, i
print Starting child process %d (%d) % (i,p.pid)
while (True):
pid, ignored = os.wait()
try:
p, i = pids[pid]
except KeyError:
# not one of ours
continue
del pids[pid]
counts[i] += 1
#terminate if count10
if (counts[i]==10):
print Child Process %d terminated. % i
if reduce(lambda x,y: x and (y=10), counts):
break
continue
print Child Process %d terminated, restarting % i
p = Popen('sleep 1', shell=True, cwd='/home', stdout=file(os.devnull,'w'))
pids[p.pid] = p, i
--
http://mail.python.org/mailman/listinfo/python-list
Re: lists and dictionaries
On 11 jul, 21:08, Ladislav Andel [EMAIL PROTECTED] wrote:
Hi,
I have a list of dictionaries.
e.g.
[{'index': 0, 'transport': 'udp', 'service_domain': 'dp0.example.com'},
{'index': 1, 'transport': 'udp', 'service_domain': 'dp1.example.com'},
{'index': 0, 'transport': 'tcp', 'service_domain': 'dp0.example.com'},
{'index': 1, 'transport': 'tcp', 'service_domain': 'dp1.example.com'}]
how could I make a new list of dictionaries which would look like:
[{'transports': ['udp','tcp'], 'service_domain': 'dp0.example.com'},
{'transports': ['udp','tcp'], 'service_domain': 'dp1.example.com'}]
Could you help me, please?
doms = {}
for entry in oldList:
doms.setdefault(entry['service_domain'],
[]).append(entry['transport'])
newList = [{'transports': t, 'service_domain': d} for d,t in
doms.iteritems()]
--
http://mail.python.org/mailman/listinfo/python-list
Re: Portable general timestamp format, not 2038-limited
Ben Finney wrote: [0] Left unqualified, referring to a site named Wiki refers to *the* Wiki, at URL:http://c2.com/cgi/wiki. If you mean some other subsequent Wiki site, you'll need to be more specific. I'm betting they meant http://en.wikipedia.org/wiki/Main_Page or one of its non-English siblings. -- Lew -- http://mail.python.org/mailman/listinfo/python-list
Re: web page text extractor
2007/7/12, kublai [EMAIL PROTECTED]:
For a project, I need to develop a corpus of online news stories. I'm
looking for an application that, given the url of a web page, copies
the rendered text of the web page (not the source HTNL text), opens a
text editor (Notepad), and displays the copied text for the user to
examine and save into a text file. Graphics and sidebars to be
ignored. The examples I have come across are much too complex for me
to customize for this simple job. Can anyone lead me to the right
direction?
def textonly(url):
# Get the HTML source on url and give only the main text
f = urllib2.urlopen(url)
text = f.read()
r = re.compile('\[^\\]*\')
newtext = r.sub('',text)
while newtext != text:
text = newtext
newtext = r.sub('',text)
return text
--
Andre Engels, [EMAIL PROTECTED]
ICQ: 6260644 -- Skype: a_engels
--
http://mail.python.org/mailman/listinfo/python-list
Re: web page text extractor
2007/7/12, Andre Engels [EMAIL PROTECTED]:
I forgot to include
import urllib2, re
here
def textonly(url):
# Get the HTML source on url and give only the main text
f = urllib2.urlopen(url)
text = f.read()
r = re.compile('\[^\\]*\')
newtext = r.sub('',text)
while newtext != text:
text = newtext
newtext = r.sub('',text)
return text
--
Andre Engels, [EMAIL PROTECTED]
ICQ: 6260644 -- Skype: a_engels
--
http://mail.python.org/mailman/listinfo/python-list
Re: lists and dictionaries
On 12 jul, 04:49, anethema [EMAIL PROTECTED] wrote:
li = [ {'index': 0, 'transport': 'udp', 'service_domain':
'dp0.example.com'},
{'index': 1, 'transport': 'udp', 'service_domain':
'dp1.example.com'},
{'index': 0, 'transport': 'tcp', 'service_domain':
'dp0.example.com'},
{'index': 1, 'transport': 'tcp', 'service_domain':
'dp1.example.com'}]
I like this solution:
[{ 'transports': [d['transport'] for d in li if
d['service_domain'] == dom],
'service_domain': dom,
} for dom in set(d2['service_domain'] for d2 in li)]
merely because it takes one line. Humorously enough, it appears to be
twice as efficient,
Correct me if I´m wrong, that is a O(n**2) solution, to O(n) problem.
--
http://mail.python.org/mailman/listinfo/python-list
Re: web page text extractor
On 2007-07-12 04:42:25 -0500, kublai [EMAIL PROTECTED] said: For a project, I need to develop a corpus of online news stories. I'm looking for an application that, given the url of a web page, copies the rendered text of the web page (not the source HTNL text), opens a text editor (Notepad), and displays the copied text for the user to examine and save into a text file. Graphics and sidebars to be ignored. The examples I have come across are much too complex for me to customize for this simple job. Can anyone lead me to the right direction? You may find BeautifulSoup or templatemaker to be of assistance: http://www.crummy.com/software/BeautifulSoup/ http://www.holovaty.com/blog/archive/2007/07/06/0128 -- http://mail.python.org/mailman/listinfo/python-list
Lists in classes
Hello, Learning python from a c++ background. Very confused about this: class jeremy: list=[] def additem(self): self.list.append(hi) return temp = jeremy() temp.additem() temp.additem() print temp.list temp2 = jeremy() print temp2.list == The output gives: ['hi','hi'] ['hi','hi'] Why does adding items to one instance produce items in a separate instance? Doesn't each instance of jeremy have its' own list? Many thanks for clearing up this newbie confusion. Jeremy. -- http://mail.python.org/mailman/listinfo/python-list
Re: Lists in classes
On 12 jul, 17:23, Jeremy Lynch [EMAIL PROTECTED] wrote: Hello, Learning python from a c++ background. Very confused about this: class jeremy: list=[] def additem(self): self.list.append(hi) return temp = jeremy() temp.additem() temp.additem() print temp.list temp2 = jeremy() print temp2.list == The output gives: ['hi','hi'] ['hi','hi'] Why does adding items to one instance produce items in a separate instance? Doesn't each instance of jeremy have its' own list? You've defined list (very bad choice for a name), as a class variable. To declare instance variable you should have written: class jeremy: -- http://mail.python.org/mailman/listinfo/python-list
Re: Lists in classes
On Jul 12, 10:23 am, Jeremy Lynch [EMAIL PROTECTED] wrote: Hello, Learning python from a c++ background. Very confused about this: class jeremy: list=[] def additem(self): self.list.append(hi) return temp = jeremy() temp.additem() temp.additem() print temp.list temp2 = jeremy() print temp2.list == The output gives: ['hi','hi'] ['hi','hi'] Why does adding items to one instance produce items in a separate instance? Doesn't each instance of jeremy have its' own list? Many thanks for clearing up this newbie confusion. Jeremy. The reason it works like that is that your variable list isn't an instance variable per se. Instead, you should have it like this: code class jeremy: def __init__(self): self.lst=[] def additem(self): self.lst.append(hi) return /code Now it works as expected. It's some kind of scope issue, but I can't explain it adequately. Mike -- http://mail.python.org/mailman/listinfo/python-list
Re: Lists in classes
On 12 jul, 17:23, Jeremy Lynch [EMAIL PROTECTED] wrote: Hello, Learning python from a c++ background. Very confused about this: class jeremy: list=[] You've defined list (very bad choice of a name, BTW), as a class variable. To declare is as instance variable you have to prepend it with self. -- http://mail.python.org/mailman/listinfo/python-list
Re: Lists in classes
On Jul 12, 6:23 pm, Jeremy Lynch [EMAIL PROTECTED] wrote: Hello, Learning python from a c++ background. Very confused about this: class jeremy: list=[] def additem(self): self.list.append(hi) return temp = jeremy() temp.additem() temp.additem() print temp.list temp2 = jeremy() print temp2.list == The output gives: ['hi','hi'] ['hi','hi'] Why does adding items to one instance produce items in a separate instance? Doesn't each instance of jeremy have its' own list? Many thanks for clearing up this newbie confusion. Jeremy. You are defining the list in the class context and so it becomes a class field/member. For defining instance members you need to always prefix those with self (this) in the contexts it is available (f.e. in the instance method context). bests, ./alex -- .w( the_mindstorm )p. -- http://mail.python.org/mailman/listinfo/python-list
Re: Lists in classes
Jeremy Lynch wrote: Hello, Learning python from a c++ background. Very confused about this: class jeremy: list=[] def additem(self): self.list.append(hi) return temp = jeremy() temp.additem() temp.additem() print temp.list temp2 = jeremy() print temp2.list == The output gives: ['hi','hi'] ['hi','hi'] Why does adding items to one instance produce items in a separate instance? Doesn't each instance of jeremy have its' own list? Many thanks for clearing up this newbie confusion. Jeremy. In this code, list (bad name) is a class attribute and all therefor in all instances, the list attribute is reference to the class attribute unless otherwise assigned, as in __init__. For instance, try: temp = jeremy() temp.additem() temp.additem() print temp.list temp2 = jeremy() temp2.list = [1,2,3] print temp.list, temp2.list, jeremy.list And see which ones look the same (same reference) or look different. James -- http://mail.python.org/mailman/listinfo/python-list
Re: Lists in classes
Bart Ogryczak wrote: On 12 jul, 17:23, Jeremy Lynch [EMAIL PROTECTED] wrote: Hello, Learning python from a c++ background. Very confused about this: class jeremy: list=[] You've defined list (very bad choice of a name, BTW), as a class variable. To declare is as instance variable you have to prepend it with self. Ouch! 'self' is *not* a reserved ord in python, it doesn't do anything. So just popping 'self' in front of something doesn't bind it to an instance. Here is how it works: class Jeremy(object): # you better inherit from 'object' at all times classlist = [] # class variable def __init__(self): # constructor self.instancelist = [] # instance variable def add_item(self, item): self.instancelist.append(item) 'self' is the customary name for the first argument of any instance method, which is always implicitly passed when you call it. I think it translates to C++'s 'this' keyword, but I may be wrong. Simply put: The first argument in an instance-method definition (be it called 'self' or otherwise) refers to the current instance. Note however that you don't explicitly pass the instance to the method, that is done automatically: j = Jeremy() # Jeremy.__init__ is called at this moment, btw j.add_item(hi) # See? 'item' is the first arg you actually pass I found this a bit confusing at first, too, but it's actually very clean, I think. /W -- http://mail.python.org/mailman/listinfo/python-list
Re: web page text extractor
On Jul 12, 5:24 pm, Andre Engels [EMAIL PROTECTED] wrote:
2007/7/12, Andre Engels [EMAIL PROTECTED]:
I forgot to include
import urllib2, re
here
def textonly(url):
# Get the HTML source on url and give only the main text
f = urllib2.urlopen(url)
text = f.read()
r = re.compile('\[^\\]*\')
newtext = r.sub('',text)
while newtext != text:
text = newtext
newtext = r.sub('',text)
return text
--
Andre Engels, [EMAIL PROTECTED]
ICQ: 6260644 -- Skype: a_engels
Andre I think that unfortunately your solution will not ignore inlined
scripting, inlined styling, etc.
On the otherside, I don't think there are many solutions available,
other than the Lynx approach somebody
has already suggested.
bests,
./alex
--
.w( the_mindstorm )p.
--
http://mail.python.org/mailman/listinfo/python-list
Re: web page text extractor
On Jul 12, 10:22 pm, Jon Rosebaugh [EMAIL PROTECTED] wrote: On 2007-07-12 04:42:25 -0500, kublai [EMAIL PROTECTED] said: For a project, I need to develop a corpus of online news stories. I'm looking for an application that, given the url of a web page, copies the rendered text of the web page (not the source HTNL text), opens a text editor (Notepad), and displays the copied text for the user to examine and save into a text file. Graphics and sidebars to be ignored. The examples I have come across are much too complex for me to customize for this simple job. Can anyone lead me to the right direction? You may find BeautifulSoup or templatemaker to be of assistance: http://www.crummy.com/software/BeautifulSoup/http://www.holovaty.com/blog/archive/2007/07/06/0128 Thanks all for your suggestions. I will try first the Lynx solution. Cheers, gk -- http://mail.python.org/mailman/listinfo/python-list
Re: Lists in classes
Thanks for all the replies, very impressive. Got it now. Jeremy. On Jul 12, 4:23 pm, Jeremy Lynch [EMAIL PROTECTED] wrote: Hello, Learning python from a c++ background. Very confused about this: class jeremy: list=[] def additem(self): self.list.append(hi) return temp = jeremy() temp.additem() temp.additem() print temp.list temp2 = jeremy() print temp2.list == The output gives: ['hi','hi'] ['hi','hi'] Why does adding items to one instance produce items in a separate instance? Doesn't each instance of jeremy have its' own list? Many thanks for clearing up this newbie confusion. Jeremy. -- http://mail.python.org/mailman/listinfo/python-list
Re: Re-raising exceptions with modified message
On Jul 12, 6:31 am, samwyse [EMAIL PROTECTED] wrote: On Jul 8, 8:50 am, Christoph Zwerschke [EMAIL PROTECTED] wrote: With Py 2.5 I get: new.__class__ = old.__class__ TypeError: __class__ must be set to a class Hmmm, under Python 2.4.X, printing repr(old.__class__) gives me this: class exceptions.UnicodeDecodeError at 0x00A24F00 while under 2.5.X, I get this: type 'exceptions.UnicodeDecodeError' So, let's try sub-classing the type: def modify_message(old, f): class Empty: pass new = Empty() print old.__class__ =, repr(old.__class__) print Empty =, repr(Empty) new.__class__ = Empty class Excpt(old.__class__): pass print Excpt =, repr(Excpt) print Excpt.__class__ =, repr(Excpt.__class__) new.__class__ = Excpt new.__dict__ = old.__dict__.copy() new.__str__ = f return new Nope, that gives us the same message: old.__class__ = type 'exceptions.UnicodeDecodeError' Empty = class __main__.Empty at 0x00AB0AB0 Excpt = class '__main__.Excpt' Excpt.__class__ = type 'type' Traceback (most recent call last): [...] TypeError: __class__ must be set to a class Excpt ceratinly appears to be a class. Does anyone smarter than me know what's going on here? -- http://mail.python.org/mailman/listinfo/python-list
Re: os.wait() losing child?
Hrvoje Niksic wrote: greg wrote: Actually, it's not that bad. _cleanup only polls the instances that are no longer referenced by user code, but still running. If you hang on to Popen instances, they won't be added to _active, and __init__ won't reap them (_active is only populated from Popen.__del__). Perhaps that's the difference between Python 2.4 and 2.5. In 2.4, Popen's __init__ always appends self to _active: def __init__(...): _cleanup() ... self._execute_child(...) ... _active.append(self) This version is a trivial modification of your code to that effect. Does it work for you? Nope it still doesn't work. I'm running python 2.4.4, tho. $ python test.py Starting child process 0 (26497) Starting child process 1 (26498) Starting child process 2 (26499) Child Process 2 terminated, restarting Child Process 2 terminated, restarting Child Process 2 terminated, restarting Child Process 2 terminated, restarting Child Process 2 terminated, restarting Child Process 2 terminated, restarting Child Process 2 terminated, restarting Child Process 2 terminated, restarting Child Process 2 terminated, restarting Child Process 2 terminated. Traceback (most recent call last): File test.py, line 15, in ? pid, ignored = os.wait() OSError: [Errno 10] No child processes -- http://mail.python.org/mailman/listinfo/python-list
MP3 AVI DIVX MPEG SOFT = www.GEGEREKA.com
Millions files for everyone. Music, movies, soft and other media. Check it now!! http://www.gegereka.com -- http://mail.python.org/mailman/listinfo/python-list
Re: storing pickles in sql data base
On 12 Jul 2007 06:00:59 GMT, Marc 'BlackJack' Rintsch [EMAIL PROTECTED] wrote: On Wed, 11 Jul 2007 17:14:43 -0500, Chris Mellon wrote: [pickle] Protocol 0 (the default) is a text protocol, it's safe to store in a text field or write to a text file. It's not really a text protocol it's more a binary protocol that uses just the ASCII range of byte values. You have to write and read the text files in binary mode or they break if taken across platform boundaries because of the different line endings in Linux and Windows for instance. It's fine as long as you use universal line endings mode. -- http://mail.python.org/mailman/listinfo/python-list
Re: Lists in classes
Thanks for clearing up the other incorrect answers! In true Python fashion, I would also remind everyone of the *documentation* - in particular the Python tutorial. These are very elementary mistakes to be making - even worse as part of attempted answers. The Python tutorial is at http://docs.python.org/tut/tut.html In particular see the section on classes: http://docs.python.org/tut/node11.html Note however that the tutorial doesn't show the current best practice of subclassing your classes from object, i.e. class MyClass(object): #...etc... -Jason On Jul 12, 11:51 am, Wildemar Wildenburger [EMAIL PROTECTED] wrote: Bart Ogryczak wrote: On 12 jul, 17:23, Jeremy Lynch [EMAIL PROTECTED] wrote: Hello, Learning python from a c++ background. Very confused about this: class jeremy: list=[] You've defined list (very bad choice of a name, BTW), as a class variable. To declare is as instance variable you have to prepend it with self. Ouch! 'self' is *not* a reserved ord in python, it doesn't do anything. So just popping 'self' in front of something doesn't bind it to an instance. Here is how it works: class Jeremy(object): # you better inherit from 'object' at all times classlist = [] # class variable def __init__(self): # constructor self.instancelist = [] # instance variable def add_item(self, item): self.instancelist.append(item) 'self' is the customary name for the first argument of any instance method, which is always implicitly passed when you call it. I think it translates to C++'s 'this' keyword, but I may be wrong. Simply put: The first argument in an instance-method definition (be it called 'self' or otherwise) refers to the current instance. Note however that you don't explicitly pass the instance to the method, that is done automatically: j = Jeremy() # Jeremy.__init__ is called at this moment, btw j.add_item(hi) # See? 'item' is the first arg you actually pass I found this a bit confusing at first, too, but it's actually very clean, I think. /W -- http://mail.python.org/mailman/listinfo/python-list
Re: PEP 3107 and stronger typing (note: probably a newbie question)
In article [EMAIL PROTECTED], Steve Holden [EMAIL PROTECTED] wrote: Donn Cave wrote: Someday we will look at variables like we look at goto. How very functional. I believe some people naturally think in terms of state transformations and some in terms of functional evaluation. I am pretty firmly in the former camp myself, so variables are a natural repository for state to me. Don't worry - there will be a state transformation monad for you! Nature or nurture? it would be interesting to see some identical twin studies on novice programmers. Since few of us were exposed first to strictly functional programming, though, you have to wonder. In its day, goto was of course very well loved. Donn Cave, [EMAIL PROTECTED] -- http://mail.python.org/mailman/listinfo/python-list
Re: PEP 3107 and stronger typing (note: probably a newbie question)
In article [EMAIL PROTECTED], Ben Finney [EMAIL PROTECTED] wrote: Paul Rubin http://[EMAIL PROTECTED] writes: The idea of designing languages with more and more support for ensuring program correctness is to put the established, repetitive processes into the computer where it belongs, freeing the programmer to be innovative while still providing high assurance of that the program will be right the first time. This seems to make the dangerous assumption that the programmer has the correct program in mind, and needs only to transfer it correctly to the computer. I would warrant that anyone who understands exactly how a program should work before writing it, and makes no design mistakes before coming up with a program that works, is not designing a program of any interest. I don't get it. Either you think that the above mentioned support for program correctness locks program development into a frozen stasis at its original conception, in which case you don't seem to have read or believed the whole paragraph and haven't been reading much else in this thread. Certainly up to you, but you wouldn't be in a very good position to be drawing weird inferences as above. Or you see original conception of the program as so inherently suspect, that random errors introduced during implementation can reasonably be seen as helpful, which would be an interesting but unusual point of view. Donn Cave, [EMAIL PROTECTED] -- http://mail.python.org/mailman/listinfo/python-list
Re: Tuple vs List: Whats the difference?
Thanks to all of you, Im sure I'll pick your brains some more later down the line. --Shafik -- http://mail.python.org/mailman/listinfo/python-list
Re: web page text extractor
kublai wrote: For a project, I need to develop a corpus of online news stories. I'm looking for an application that, given the url of a web page, copies the rendered text of the web page (not the source HTNL text), opens a text editor (Notepad), and displays the copied text for the user to examine and save into a text file. Graphics and sidebars to be ignored. The examples I have come across are much too complex for me to customize for this simple job. Can anyone lead me to the right direction? Super-simplistic: import lxml.etree as et parser = et.HTMLParser() tree = et.parse(http://the/page.html;, parser) print tree.xpath(string(/html/body)) http://codespeak.net/lxml/ You may want to use the incredibly versatile lxml.html.clean module first to remove any annoying content. It's not released yet but available in a branch: http://codespeak.net/svn/lxml/branch/html/ Stefan -- http://mail.python.org/mailman/listinfo/python-list
Re: Problem with building extension in Python
On Jul 5, 4:33 am, Gabriel Genellina [EMAIL PROTECTED] wrote:
En Thu, 05 Jul 2007 05:08:58 -0300, [EMAIL PROTECTED]
escribió:
I have already install Microsoft visual studio .NET 2003 and MinGw,
when I try to build a extension:
Is this for Python 2.4? I think you can't compile Python 2.5 with VS2003.
python my_extension_setup.py build ( or install ) , I get an error:
LINK : fatal error LNK1141: failure during build of exports file
error: command 'C:\Program Files\Microsoft Visual Studio .NET
2003\Vc7\bin\link.exe' failed with exit status 1141.What shoud I
do???
Almost surely you got a previous error, making link to fail. Try to
correct *that* error.
--
Gabriel Genellina
Microsoft Visual Studio .NET 2003 Version 7.1 is the correct
version to build extensions for Python 2.5. There is also a Microsoft
Visual Studio .NET 2003 Version 7.0 as well, which is not acceptable
for building extensions. The first release (version 7.0) was purchased
by many, and is relatively easy to find. The following release
(version 7.1) was a minor revision and is very hard to find, but it is
what's necessary for building Python extensions. I hope future Python
releases are built using a more standard compiler.
I'm posting in this thread because I am running into the same
problem as the original poster, linker error 1141. Here's a summary of
what I've gotten to work properly so far:
Python 2.5, Pyrex, and Microsoft Visual Studio 2003 .NET (version 7.1)
are successfully installed
I have successfully built Pyrex extensions using Distutils
I have successfully imported and ran Pyrex extensions in my Python
code
The problems occur when I try to include C libraries in my Pyrex
code. Specifically, I need some trig functions found in C's math.h
header files. My .pyx file is as follows:
#
# Use C math functions
#
def cFunctionTester(float theta):
import sys
# Import the native C functions we need
cdef extern from math.h:
double cos(double theta)
double sin(double theta)
result[0] = cos(theta)
result[1] = sin(theta)
return 5
I try to build the extension by running my setup.py file from the
command prompt. My setup.py file is presented below:
from distutils.core import setup
from distutils.extension import Extension
from Pyrex.Distutils import build_ext
setup(
name = PyrexGuide,
ext_modules=[
Extension(cTest, [cTest.pyx], libraries = [])
],
cmdclass = {'build_ext': build_ext}
)
I run this code through the command prompt by entering setup.py
install The following error appears:
C:\Python25\Lib\site-packages\Pyrex\Distutilssetup.py install
running install
running build
running build_ext
building 'cTest' extension
c:\Program Files\Microsoft Visual Studio .NET 2003\Vc7\bin\cl.exe /c /
nologo /Ox
/MD /W3 /GX /DNDEBUG -IC:\Python25\include -IC:\Python25\PC /
TccTest.c /Fobuild
\temp.win32-2.5\Release\cTest.obj
cTest.c
c:\Program Files\Microsoft Visual Studio .NET 2003\Vc7\bin\link.exe /
DLL /nologo
/INCREMENTAL:NO /LIBPATH:C:\Python25\libs /LIBPATH:C:
\Python25\PCBuild /EXPORT:
initcTest build\temp.win32-2.5\Release\cTest.obj /OUT:build
\lib.win32-2.5\cTest.
pyd /IMPLIB:build\temp.win32-2.5\Release\cTest.lib
LINK : error LNK2001: unresolved external symbol initcTest
build\temp.win32-2.5\Release\cTest.lib : fatal error LNK1120: 1
unresolved exter
nals
LINK : fatal error LNK1141: failure during build of exports file
error: command 'c:\Program Files\Microsoft Visual Studio .NET
2003\Vc7\bin\link
.exe' failed with exit status 1141
C:\Python25\Lib\site-packages\Pyrex\Distutils
I do not know what the problem is -- perhaps the linker cannot locate
math.h? I've tried copying the math.h file to a whole bunch of other
directories, but there's still no luck. Any help anyone can provide
would be greatly appreciated.
Joe Stoffa
[EMAIL PROTECTED]
--
http://mail.python.org/mailman/listinfo/python-list
Re: PEP 3107 and stronger typing (note: probably a newbie question)
Donn Cave [EMAIL PROTECTED] writes: Don't worry - there will be a state transformation monad for you! Nature or nurture? it would be interesting to see some identical twin studies on novice programmers. Since few of us were exposed first to strictly functional programming, though, you have to wonder. In its day, goto was of course very well loved. Guy Steele used to describe functional programming -- the evaluation of lambda-calculus without side effects -- as separation of Church and state, a highly desirable situation ;-). (For non-FP nerds, the above is a pun referring to Alonzo Church, who invented lambda calculus in the 1920's or so). -- http://mail.python.org/mailman/listinfo/python-list
Re: PEP 3107 and stronger typing (note: probably a newbie question)
On 7/12/07, Donn Cave [EMAIL PROTECTED] wrote: In article [EMAIL PROTECTED], Ben Finney [EMAIL PROTECTED] wrote: Paul Rubin http://[EMAIL PROTECTED] writes: The idea of designing languages with more and more support for ensuring program correctness is to put the established, repetitive processes into the computer where it belongs, freeing the programmer to be innovative while still providing high assurance of that the program will be right the first time. This seems to make the dangerous assumption that the programmer has the correct program in mind, and needs only to transfer it correctly to the computer. I would warrant that anyone who understands exactly how a program should work before writing it, and makes no design mistakes before coming up with a program that works, is not designing a program of any interest. I don't get it. Either you think that the above mentioned support for program correctness locks program development into a frozen stasis at its original conception, in which case you don't seem to have read or believed the whole paragraph and haven't been reading much else in this thread. Certainly up to you, but you wouldn't be in a very good position to be drawing weird inferences as above. Or you see original conception of the program as so inherently suspect, that random errors introduced during implementation can reasonably be seen as helpful, which would be an interesting but unusual point of view. You can't prove a program to be correct, in the sense that it's proven to do what it's supposed to do and only what it's supposed to do. You can prove type-correctness, and the debate is really over the extent that a type-correct program is also behavior-correct. Personally, I remain unconvinced, but am open to evidence - I've only heard anecdotes which are readily discounted by other anecdotes. I absolutely do not believe that pure functional style programming, where we shun variables, is the best model for computing, either now or in the future. -- http://mail.python.org/mailman/listinfo/python-list
Re: os.wait() losing child?
Hrvoje Niksic [EMAIL PROTECTED] wrote: Nick Craig-Wood [EMAIL PROTECTED] writes: I think your polling way works; it seems there no other way around this problem other than polling or extending Popen class. I think polling is probably the right way of doing it... It requires the program to wake up every 0.1s to poll for freshly exited subprocesses. That doesn't consume excess CPU cycles, but it does prevent the kernel from swapping it out when there is nothing to do. Sleeping in os.wait allows the operating system to know exactly what the process is waiting for, and to move it out of the way until those conditions are met. (Pedants would also notice that polling introduces on average 0.1/2 seconds delay between the subprocess dying and the parent reaping it.) Sure! You could get rid of this by sleeping until a SIGCHLD arrived maybe. In general, a program that waits for something should do so in a single call to the OS. OP's usage of os.wait was exactly correct. Disagree for the reason below. Internally subprocess uses os.waitpid(pid) just waiting for its own specific pids. IMHO this is the right way of doing it other than os.wait() which waits for any pids. os.wait() can reap children that you weren't expecting (say some library uses os.system())... system calls waitpid immediately after the fork. os.system probably wasn't the best example, but you take my point I think! This can still be a problem for applications that call wait in a dedicated thread, but the program can always ignore the processes it doesn't know anything about. Ignoring them isn't good enough because it means that the bit of code which was waiting for that process to die with os.getpid() will never get called, causing a deadlock in that bit of code. What is really required is a select() like interface to wait which takes more than one pid. I don't think there is such a thing though, so polling is your next best option. -- Nick Craig-Wood [EMAIL PROTECTED] -- http://www.craig-wood.com/nick -- http://mail.python.org/mailman/listinfo/python-list
Re: Problem with Python's robots.txt file parser in module robotparser
In article [EMAIL PROTECTED], John Nagle [EMAIL PROTECTED] wrote: Nikita the Spider wrote: Hi John, Are you sure you're not confusing your sites? The robots.txt file at www.ibm.com contains the double slashed path. The robots.txt file at ibm.com is different and contains this which would explain why you think all URLs are denied: User-agent: * Disallow: / Ah, that's it. The problem is that ibm.com redirects to http://www.ibm.com;, but but ibm.com/robots.txt does not redirect. For comparison, try microsoft.com/robots.txt, which does redirect. Strange thing for them to do, isn't it? Especially with two such different robots.txt files. -- Philip http://NikitaTheSpider.com/ Whole-site HTML validation, link checking and more -- http://mail.python.org/mailman/listinfo/python-list
Re: asyncore and OOB data
In article [EMAIL PROTECTED], Steve Holden [EMAIL PROTECTED] writes: Douglas Wells wrote: Third, the TCP protocol, which you have selected via the SOCK_STREAM option doesn't support OOB at all, so there's no way that you can even send OOB data in the manner that you are expecting. Wrong. The URGENT pointer indicates that the data begins with out-of-band data. You are correct, however, in stating that the FTP protocol doesn't support or implement out-of-band data transmissions. Well, first I have to admit that I just looked at the wiki article on out-of-band (http://en.wikipedia.org/wiki/Out-of-band) and it also makes a statement that TCP urgent data is OOB. But I disagree with that interpretation. So, let me acknowledge that there are dissenting opinions, and explain my assertion in terms of the OP's problem of wanting a separate data stream (or at least that was my interpretation). First, I fully acknowledge that the concept of TCP's URGENT pointer itself is out-of-band, but I don't know of any way to associate data, in the form of an identifiable set of octets, with that concept. As supporting evidence I offer the following: - There is no way for the sender and the receiver to agree on the *start* of an urgent data set. The end point is clearly identified by the urgent pointer, but the start is time dependent and is not identified in the TCP communication stream. Thus, the sender could not express the concept of sending a particular data value, e.g. This string, to the receiver reliably. - If receivers are highly responsive (meaning that the receiver is always able to process incoming data without having to discard any of it), there will be no difference in the behavior of two receivers, one of which properly implements the concept of the urgent pointer, and the other of which totally ignores the concept of urgency. - There appears to be no way to actually read any OOB data from a Berkeley socket API. (As I noted my system, at least, returns an error in this case.) Let's switch to the positive, however. What TCP does have is the concept of urgent data. Urgent data is data that the sender believes is so important that the receiver should discard intermediate data, if necessary, in order to process the urgent data -- but the urgent data is sent *in-band*. No, technically the urgent data is out-of-band, isn't it? There is no requirement to ditch the other data in the packet, so urgent data can be interspersed without loss of regular in-band data if required. This is the same point that I argued above. I now acknowledge that some people think of the TCP urgent concept as OOB. I don't; but this is a matter of definition and almost certainly not an issue for coherent argument. In the case of FTP, the concept of urgent data is used to allow the user side to interrupt the data transfer, which it does by sending an ABOR command. The underlying problem is that the server might be blocked on a receive on the data connection and not listening for commands on the control connection. So the user side is required to perform some special action that includes sending urgent data, which the server can then process specially. This problem arises primarily in systems without either threading or asynchronous I/O APIs. Your use of asyncore should alleviate this problem. The thing that may be confusing you is that for whatever reason the designers of the Berkeley socket API (which is what you are using when running under POSIX/Linux/MS Windows), chose to use the OOB flags (e.g., MSG_OOB) to indicate the *transmission* of OOB data -- but it's not really OOB data. In fact, when I attempt to receive using the MSG_OOB flag using the OS interfaces on my system, I get back an error (EINVAL - Invalid argument). I would expect that asyncore would report this error somehow or other. You can, of course, expect what you like, but experience is the best guide to what will actually happen. Agreed, but having acknowledged that I had never used asyncore, I was trying to express the hope that the designers would not purposely discard a error condition without notifying the user, particularly for a reliable transfer method such as TCP. Steve - dmw -- . Douglas Wells . Connection Technologies . . Internet: -sp9804- -at - contek.com- . -- http://mail.python.org/mailman/listinfo/python-list
Re: asyncore and OOB data
In article [EMAIL PROTECTED], billiejoex [EMAIL PROTECTED] writes: Douglas Wells wrote: Second, when I look at the FTP specification, I don't find the concept of OOB anywhere. So, it's not clear what OOB data would mean in terms of the defined FTP commands in any case. Steve Holde wrote: You are correct, however, in stating that the FTP protocol doesn't support or implement out-of-band data transmissions. Wait a sec. RFC959 [1], on chapter 4.1.3 says: [ ... ] I believe that the TCP urgent flag, activated by using socket.MSG_OOB, should be set when client must send the Sync signal (RFC854 [2] talks about it). I think that you do not find references of OOB in RFC959 (FTP) just because it is specified into RFC854 (Telnet). The point that I was trying to make, which Steve made more clearly that I did in the referenced paragraph, is that FTP doesn't support OOB data transmission, not that RFP doesn't require the use of OOB information (in the form of the Telnet Synch signal). Note that FTP limits the commands that may be issues while a data transfer in progress. The RFC isn't totally explicit, but the general notion is that issuing a data transfer command (such as STOR or RETR) is not allowed while another data transfer is in progress. - dmw -- . Douglas Wells . Connection Technologies . . Internet: -sp9804- -at - contek.com- . -- http://mail.python.org/mailman/listinfo/python-list
Re: lists and dictionaries
Thank you to all of you guys.
It's exactly I was looking for.
You provide scant information for this task. For example, is the
new list ordered or unordered? Can the list corresponding to the
'transports' key contain duplicates?
unordered is fine.. I will be storing it in DB from that list.
It should not happen to get 'transports' duplicates in my application.
It's for testing domains where are running SIP servers.
Lada
Bart Ogryczak wrote:
On 12 jul, 04:49, anethema [EMAIL PROTECTED] wrote:
li = [ {'index': 0, 'transport': 'udp', 'service_domain':
'dp0.example.com'},
{'index': 1, 'transport': 'udp', 'service_domain':
'dp1.example.com'},
{'index': 0, 'transport': 'tcp', 'service_domain':
'dp0.example.com'},
{'index': 1, 'transport': 'tcp', 'service_domain':
'dp1.example.com'}]
I like this solution:
[{ 'transports': [d['transport'] for d in li if
d['service_domain'] == dom],
'service_domain': dom,
} for dom in set(d2['service_domain'] for d2 in li)]
merely because it takes one line. Humorously enough, it appears to be
twice as efficient,
Correct me if I´m wrong, that is a O(n**2) solution, to O(n) problem.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Lists in classes
Alex Popescu a écrit : (snip) You are defining the list in the class context and so it becomes a class field/member. 'attribute' is the pythonic term. -- http://mail.python.org/mailman/listinfo/python-list
Re: lists and dictionaries
Thank you to all of you guys.
It's exactly I was looking for.
Lada
Bart Ogryczak wrote:
On 12 jul, 04:49, anethema [EMAIL PROTECTED] wrote:
li = [ {'index': 0, 'transport': 'udp', 'service_domain':
'dp0.example.com'},
{'index': 1, 'transport': 'udp', 'service_domain':
'dp1.example.com'},
{'index': 0, 'transport': 'tcp', 'service_domain':
'dp0.example.com'},
{'index': 1, 'transport': 'tcp', 'service_domain':
'dp1.example.com'}]
I like this solution:
[{ 'transports': [d['transport'] for d in li if
d['service_domain'] == dom],
'service_domain': dom,
} for dom in set(d2['service_domain'] for d2 in li)]
merely because it takes one line. Humorously enough, it appears to be
twice as efficient,
Correct me if I´m wrong, that is a O(n**2) solution, to O(n) problem.
--
http://mail.python.org/mailman/listinfo/python-list
Re: PEP 3107 and stronger typing (note: probably a newbie question)
Chris Mellon [EMAIL PROTECTED] writes: You can't prove a program to be correct, in the sense that it's proven to do what it's supposed to do and only what it's supposed to do. You can prove type-correctness, and the debate is really over the extent that a type-correct program is also behavior-correct. But every mathematical theorem corresponds to a type, so if you can formalize an argument that your code has a certain behavior, then a type checker can statically verify it. There are starting to be programming languages with embedded proof assistants, like Concoqtion (google for it). Of course you can only prove formal properties of programs, which says nothing about whether the application is doing the right thing for what the user needs. However, you're still way ahead of the game if you have a machine-checked mathematical proof that (say) your multi-threaded program never deadlocks or clobbers data through race conditions, instead of merely a bunch of test runs in which you didn't observe deadlock or clobbered data. Similarly for things like floating-point arithmetic, Intel and AMD now use formal verification on their designs, apparently unlike in the days of the notorious original Pentium FDIV implementation, which passed billions of test vectors and then shipped with an error. Larger applications like the Javacard bytecode interpreter have been certified for various properties and pretty soon we may start seeing certified compilers and OS kernels. Things have come a long way since the 1970's. Personally, I remain unconvinced, but am open to evidence - I've only heard anecdotes which are readily discounted by other anecdotes. I absolutely do not believe that pure functional style programming, where we shun variables, is the best model for computing, either now or in the future. I wonder if you looked at the Tim Sweeney presentation that I linked to a few times upthread. -- http://mail.python.org/mailman/listinfo/python-list
Class decorators do not inherit properly
I have a class that does MCMC sampling (Python 2.5) that uses decorators -- one in particular called _add_to_post that appends the output of the decorated method to a class attribute. However, when I subclass this base class, the decorator no longer works: Traceback (most recent call last): File /Users/chris/Projects/CMR/closed.py, line 132, in module class M0(MetropolisHastings): File /Users/chris/Projects/CMR/closed.py, line 173, in M0 @_add_to_post NameError: name '_add_to_post' is not defined yet, when I look at the dict of the subclass (here called M0), I see the decorator method: In [5]: dir(M0) Out[5]: ['__call__', '__doc__', '__init__', '__module__', '_add_to_post', ... I dont see what the problem is here -- perhaps someone could shed some light. I thought it might be the underscore preceding the name, but I tried getting rid of it and that did not help. Thanks. -- http://mail.python.org/mailman/listinfo/python-list
Re: profiling a C++ python extension
rasmus [EMAIL PROTECTED] wrote: I have used gprof to profile stand alone C++ programs. I am also aware of pure python profilers. However, is there a way to get profile information on my C++ functions when they are compiled in a shared library (python extension module) and called from python. Any suggestions or tips would be helpful. You didn't say what platform you're using. If you're using linux and have root access, oprofile should be able to do what you need. -M- -- http://mail.python.org/mailman/listinfo/python-list
Re: os.wait() losing child?
Jason Zheng [EMAIL PROTECTED] wrote: Hrvoje Niksic wrote: Actually, it's not that bad. _cleanup only polls the instances that are no longer referenced by user code, but still running. If you hang on to Popen instances, they won't be added to _active, and __init__ won't reap them (_active is only populated from Popen.__del__). Perhaps that's the difference between Python 2.4 and 2.5. In 2.4, Popen's __init__ always appends self to _active: Yes, that changed between 2.4 and 2.5. Note that if you take a copy of 2.5's subprocess.py, it ought to work fine with 2.4. -M- -- http://mail.python.org/mailman/listinfo/python-list
Re: web page text extractor
On Jul 13, 2:19 am, Stefan Behnel [EMAIL PROTECTED] wrote: kublai wrote: For a project, I need to develop a corpus of online news stories. I'm looking for an application that, given the url of a web page, copies the rendered text of the web page (not the source HTNL text), opens a text editor (Notepad), and displays the copied text for the user to examine and save into a text file. Graphics and sidebars to be ignored. The examples I have come across are much too complex for me to customize for this simple job. Can anyone lead me to the right direction? Super-simplistic: import lxml.etree as et parser = et.HTMLParser() tree = et.parse(http://the/page.html;, parser) print tree.xpath(string(/html/body)) http://codespeak.net/lxml/ You may want to use the incredibly versatile lxml.html.clean module first to remove any annoying content. It's not released yet but available in a branch: http://codespeak.net/svn/lxml/branch/html/ Stefan Hi, Stefan, This looks very interesting. I will look into this first thing tonight. Gotta hit some golf bugs, I mean, balls first. It's a beautiful afternoon here in Edmonton. Cheers, gk -- http://mail.python.org/mailman/listinfo/python-list
inspite of proper package structure, no module named xyz error.
hello all, can some one suggest me the best solution for a package? I have a package folder called idm and there are the following files in there. __init__.py which is right now empty. mainwindow.py which has a single class called MainWindow which is an MDI Parent frame from wxpython. student.py which is a child frame which gets called from the main menu. exam.py, another child frame. now when I try to run mainwindow.py file with the import idmstatement, I get the following error. no module named idm. what could be the problem? I have imported all the necessary modules in mainwindow. kindly help. regards, Krishnakant. -- http://mail.python.org/mailman/listinfo/python-list
Re: bash-style pipes in python?
On Wed, 11 Jul 2007 20:55:48 -0700, faulkner wrote:
On Jul 11, 8:56 pm, Dan Stromberg - Datallegro
[EMAIL PROTECTED] wrote:
I'm constantly flipping back and forth between bash and python.
Sometimes, I'll start a program in one, and end up recoding in the
other, or including a bunch of python inside my bash scripts, or snippets
of bash in my python.
But what if python had more of the power of bash-style pipes? I might not
need to flip back and forth so much. I could code almost entirely in python.
The kind of thing I do over and over in bash looks like:
#!/usr/bin/env bash
# exit on errors, like python. Exit on undefind variables, like
python.
set -eu
# give the true/false value of the last false command in a pipeline
# not the true/false value of the lat command in the pipeline - like
# nothing I've seen
set -o pipefail
# save output in output, but only echo it to the screen if the
command fails
if ! output=$(foo | bar 21)
then
echo $0: foo | bar failed 12
echo $output 12
exit 1
fi
Sometimes I use $PIPESTATUS too, but not that much.
I'm aware that python has a variety of pipe handling support in its
standard library.
But is there a similarly-simple way already, in python, of hooking the
stdout of
process foo to the stdin of process bar, saving the stdout and errors from
both
in a variable, and still having convenient access to process exit values?
Would it be possible to overload | (pipe) in python to have the same
behavior as in
bash?
I could deal with slightly more cumbersome syntax, like:
(stdout, stderrs, exit_status) = proc('foo') | proc('bar')
...if the basic semantics were there.
How about it? Has someone already done this?
class P(subprocess.Popen):
def __or__(self, otherp):
otherp.stdin.write(self.stdout.read())
otherp.stdin.close()
return otherp
def __init__(self, cmd, *a, **kw):
for s in ['out', 'in', 'err']: kw.setdefault('std' + s, -1)
subprocess.Popen.__init__(self, cmd.split(), *a, **kw)
print (P('cat /etc/fstab') | P('grep x')).stdout.read()
of course, you don't need to overload __init__ at all, and you can
return otherp.stdout.read() instead of otherp, and you can make
__gt__, __lt__ read and write files. unfortunately, you can't really
fudge , , |, or any of the more useful pipes, but you can make
more extensive use of __or__:
class Pipe:
def __or__(self, other):
if isinstance(other, Pipe): return ...
elif isinstance(other, P): return ...
def __init__(self, pipe_type): ...
k = Pipe(foo)
m = Pipe(bar)
P() |k| P()
P() |m| P()
This is quite cool.
I have to ask though - isn't this going to read everything from foo until
foo closes its stdout, and then write that result to bar - rather than
doing it a block at a time like a true pipe?
I continued thinking about this last night after I sent my post, and
started wondering if it might be possible to come up with something where
you could freely intermix bash pipes and python generators, by faking the
pipes using generators and some form of concurrent function composition -
perhaps threads or subprocesses for the concurrency. Of course, that
imposes some extra I/O, CPU and context switch overhead, since you'd have
python shuttling data from foo to bar all the time instead of foo sending
data directly to bar, but the ability to mix python and pipes might be
worth it.
Even better might be to have a choice between intermixable and fast.
Comments anyone?
--
http://mail.python.org/mailman/listinfo/python-list
DBXML and the Xerces DOM
I downloaded Berkeley DB XML and put together a test script to load a
large 250mb XML file into the database. Storage and retrieval works
great, but I need DOM access to the file without loading it all into a
string and re-parsing it. In the c++ api (they used swig to expose c+
+ to python) it says there is a XmlDocument::getContentAsDOM() method,
but when I try to access it via python I just get an attribute error.
I looked in the source and the method isn't mapped at all! Has anyone
had this work, ever?
from dbxml import *
mgr = XmlManager()
container = mgr.openContainer(test.dbxml)
document = container.getDocument('test1')
dom = document.getContentAsDOM()
Traceback (most recent call last):
File look.py, line 10, in ?
dom = document.getContentAsDOM()
File /usr/lib/python2.4/site-packages/dbxml.py, line 277, in
lambda
__getattr__ = lambda self, name: _swig_getattr(self, XmlDocument,
name)
File /usr/lib/python2.4/site-packages/dbxml.py, line 28, in
_swig_getattr
raise AttributeError,name
AttributeError: getContentAsDOM
--
http://mail.python.org/mailman/listinfo/python-list
Best way to add values from keys
I have a data structure that looks like this:
dates = {}; last_song = [] #; year = {}; week = {}; date = {};
artist = []; song = {}
#dates = {'2007': {'25': {'06/23/07': {'aerosmith': [{'sweet
emotion': 1}, {'dream on': 2}],
# 'Metallica': [{'Fade to
Black': 1}, {'Master of Puppets': 1}]},
# 'last_song': [Master of Puppets', 'Fade to Black',
'sweet emotion']}}}
What is the most effecient way to go through a week and determine how
many times each song was played?
--
http://mail.python.org/mailman/listinfo/python-list
Re: bool behavior in Python 3000?
Alan Isaac wrote: Bjoern Schliessmann wrote: Is there any type named bool in standard Python? type(True) type 'bool' Thanks anyway, but I remembered it shortly after sending. Thus the cancel (seems to have failed a bit). Regards, Björn -- BOFH excuse #384: it's an ID-10-T error -- http://mail.python.org/mailman/listinfo/python-list
wx.App console window [Windows]
Hi All, I'm looking for a way to hide console window created by wx.App class. examplary code below: import wx class Gui(wx.App): def __init__(self, *pargs, **kwargs): wx.App.__init__(self, *pargs, **kwargs) if __name__ == __main__: app = Gui() app.MainLoop() produces a blank console window (than dies). What shall I do to make wx.App run without raising console window? thanks in advance -- http://mail.python.org/mailman/listinfo/python-list
Re: bool behavior in Python 3000?
Steven D'Aprano wrote: It seems to me that you deliberately misunderstood him. I know for sure I didn't. Why else would you type-cast the integers 2 and 1 to bools to supposedly demonstrate that there's nothing wrong with operations between bools returning ints? Kindly excuse me bothering You. Björn -- BOFH excuse #419: Repeated reboots of the system failed to solve problem -- http://mail.python.org/mailman/listinfo/python-list
Re: os.wait() losing child?
Nick Craig-Wood wrote: Sure! You could get rid of this by sleeping until a SIGCHLD arrived maybe. Yah, I could also just dump Popen class and use fork(). But then what's the point of having an abstraction layer any more? This can still be a problem for applications that call wait in a dedicated thread, but the program can always ignore the processes it doesn't know anything about. Ignoring them isn't good enough because it means that the bit of code which was waiting for that process to die with os.getpid() will never get called, causing a deadlock in that bit of code. Are you talking about something like os.waitpid(os.getpid())? If the process has completed and de-zombified by another os.wait() call, I thought it would just throw an exception; it won't cause a deadlock by hanging the process. ~Jason -- http://mail.python.org/mailman/listinfo/python-list
Re: os.wait() losing child?
Nick Craig-Wood [EMAIL PROTECTED] writes: This can still be a problem for applications that call wait in a dedicated thread, but the program can always ignore the processes it doesn't know anything about. Ignoring them isn't good enough because it means that the bit of code which was waiting for that process to die with os.getpid() will never get called, causing a deadlock in that bit of code. It won't deadlock, it will get an ECHILD or equivalent error because it's waiting for a PID that doesn't correspond to a running child process. I agree that this can be a problem if and when you use libraries that can call system. (In that case sleeping for SIGCHLD is probably a good solution.) What is really required is a select() like interface to wait which takes more than one pid. I don't think there is such a thing though, so polling is your next best option. Except for the problems outlined in my previous message. And the fact that polling becomes very expensive (O(n) per check) once the number of processes becomes large. Unless one knows that a library can and does call system, wait is the preferred solution. -- http://mail.python.org/mailman/listinfo/python-list
How to create new files?
Hi, I'm trying to create a Python equivalent of the C++ ifstream class, with slight behavior changes. Basically, I want to have a filestream object that will allow you to overload the '' and '' operators to stream out and stream in data, respectively. So far this is what I have: class filestream: def __init__( self, filename ): self.m_file = open( filename, rwb ) # def __del__( self ): # self.m_file.close() def __lshift__( self, data ): self.m_file.write( data ) def __rshift__( self, data ): self.m_file.read( data ) So far, I've found that unlike with the C++ version of fopen(), the Python 'open()' call does not create the file for you when opened using the mode 'w'. I get an exception saying that the file doesn't exist. I expected it would create the file for me. Is there a way to make open() create the file if it doesn't exist, or perhaps there's another function I can use to create the file? I read the python docs, I wasn't able to find a solution. Also, you might notice that my self.m_file.read() function is wrong, according to the python docs at least. read() takes the number of bytes to read, however I was not able to find a C++ equivalent of sizeof() in Python. If I wanted to read in a 1 byte, 2 byte, or 4 byte value from data into python I have no idea how I would do this. Any help is greatly appreciated. Thanks. -- http://mail.python.org/mailman/listinfo/python-list
New guy help with setup
I just downloaded 2.5 and read the readme file. It says Before you can build Python, you must first confiigure itStart by running the script ./configure. I can't find ./configure I am running Windows XP SP2 btw Python.exe gives me a black window Pythonw.exe does nothing (I assume this is a windows interface) thanks meg99 -- http://mail.python.org/mailman/listinfo/python-list
Re: bool behavior in Python 3000?
Steven D'Aprano [EMAIL PROTECTED] writes: Expressions like (i == j) used to return 0 and 1, and it was to avoid breaking hacks like the above that bools were implemented as a subclass of int, not because being able to write the above was a specific feature requested. In the hypothetical bright new world of Python with bools that are actually bools, the above are good cases for explicit being better than implicit: int(x b) * f(x) -1 ** int(i == j) It makes more sense to explicitly cast bools to ints when you want to do integer arithmetic on them, rather than explicitly casting bools to bools to do boolean arithmetic! I feel strongly enough about this that I believe that being able to write (x b) * f(x) is a DISADVANTAGE -- it gives me a real WTF moment to look at the code. Just because it looks funny to you doesn't mean it is a hack. It turns out that many mathemtical formulas can be written more clearly using this notation (see e.g. at knuth et al's concrete mathematics), and in mathematics limited and often ambiguous ad hoc notation that is neatly subsumed by this scheme is widely established. And I don't think adding int is an improvement: ``(x b) * f(x)`` is plenty clear (not easily misread as something else and, I believe, not even particularly difficult to figure out even for mediocre python programmers) but adding padding just obscures formula structure. Of course it doesn't in the above examples with less than half a dozen terms, but IMO for something longer the int-litter, even it may be soothing the psyches of the anally retentive, is counterproductive. 'as -- http://mail.python.org/mailman/listinfo/python-list
Re: New guy help with setup
meg99 wrote: I just downloaded 2.5 and read the readme file. It says Before you can build Python, you must first confiigure itStart by running the script ./configure. I can't find ./configure I am running Windows XP SP2 You downloaded the wrong file. You want the Windows Installer: http://python.org/ftp/python/2.5.1/python-2.5.1.msi HTH Paul -- pkm ~ http://paulmcnett.com -- http://mail.python.org/mailman/listinfo/python-list
Re: New guy help with setup
On Jul 12, 4:24 pm, Paul McNett [EMAIL PROTECTED] wrote: meg99 wrote: I just downloaded 2.5 and read the readme file. It says Before you can build Python, you must first confiigure itStart by running the script ./configure. I can't find ./configure I am running Windows XP SP2 You downloaded the wrong file. You want the Windows Installer:http://python.org/ftp/python/2.5.1/python-2.5.1.msi HTH Paul -- pkm ~http://paulmcnett.com My applogies - I did download and install 2.5.1 meg99 -- http://mail.python.org/mailman/listinfo/python-list
