Does Pygoogle allows for advanced search options?
Hello All, Can anyone help me with the Pygoogle: from pygoogle import pygoogle word = u'something' request_word = word.encode('utf-8') request = ('%s+site:.edu' % request_word) g = pygoogle(request) g.get_result_count() Now, I realized that domain restriction works (site:.edu etc.), but I would like to be able to control for language too. Is that possible with the Pygoogle? If not, how can I make that happen? Thanks! PM -- http://mail.python.org/mailman/listinfo/python-list
Re: How to run an EXE, with argument, capture output value
On 2010-11-18, Tim Harig wrote: > On 2010-11-18, noydb wrote: >> I have an executable that I want to run within python code. The exe >> requires an input text file, the user to click a 'compute' button, and >> then the exe calculates several output values, one of which I want to >> capture into a variable. Can I use Python to supply the input file, >> execute the exe and capture the output value, like such that the exe Sorry, I missed the second part, it's time for me to go to bed. >> really doesn't need to be 'seen'? Or, would the user still have to >> click the 'compute' button? > > Python 3.1.2 (r312:79147, Oct 9 2010, 00:16:06) > [GCC 4.4.4] on linux2 > Type "help", "copyright", "credits" or "license" for more information. > >>> import subprocess > >>> pig = subprocess.Popen(["/usr/games/pig"], stdin=subprocess.PIPE) > >>> result = pig.communicate(input=b"This is sample text.\n") > Isthay isway amplesay exttay. > >>> With capturing the output, it looks like: >>> pig = subprocess.Popen(["/usr/games/pig"], stdin=subprocess.PIPE, >>> stdout=subprocess.PIPE) >>> result = pig.communicate(input=b"This is sample text.\n")[0] >>> result b'Isthay isway amplesay exttay.\n' >>> You can also get the return code if you need it: >>> pig.returncode 0 -- http://mail.python.org/mailman/listinfo/python-list
Re: String multi-replace
Steven D'Aprano: the original file is 139MB (that's the typical size for it). Eliminating diacritics is just a little toping on the cake; the processing is something else. Thanks anyway for your suggestion, SxN PS Perhaps I should have mention that I'm on Python 2.7 -- http://mail.python.org/mailman/listinfo/python-list
Re: How to run an EXE, with argument, capture output value
On 2010-11-18, noydb wrote: > I have an executable that I want to run within python code. The exe > requires an input text file, the user to click a 'compute' button, and > then the exe calculates several output values, one of which I want to > capture into a variable. Can I use Python to supply the input file, > execute the exe and capture the output value, like such that the exe > really doesn't need to be 'seen'? Or, would the user still have to > click the 'compute' button? > > Any code snippets or guidance would be very much appreciated. I have > found that > > import os > os.system('C:\xTool\stats_hall.exe') > > will run the exe. And, maybe these execl/execle/execlp/etc functions > might be what I need for adding in the argument, but documentation > seems to indicate that these do not return output. ?? If you are not already, I would highly suggest using Python3 with the subprocess module: http://docs.python.org/py3k/library/subprocess.html It puts everything in one place and supercedes the exec* functions which where a PITA. You can 95% of what you need simply using subprocess.Popen(). There are several examples from this group in the past few days; but, the process looks something like this: Python 3.1.2 (r312:79147, Oct 9 2010, 00:16:06) [GCC 4.4.4] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import subprocess >>> pig = subprocess.Popen(["/usr/games/pig"], stdin=subprocess.PIPE) >>> result = pig.communicate(input=b"This is sample text.\n") Isthay isway amplesay exttay. >>> -- http://mail.python.org/mailman/listinfo/python-list
Re: String multi-replace
Thanks for your answers. Benjamin Kaplan: of course dict is a type... silly me! I'll blame it on the time (it's midnight here). Chris Rebert: I'll have a look. Thank you both, SxN -- http://mail.python.org/mailman/listinfo/python-list
Re: String multi-replace
On Wed, 17 Nov 2010 20:21:06 -0800, Sorin Schwimmer wrote: > Hi All, > > I have to eliminate diacritics in a fairly large file. What's "fairly large"? Large to you is probably not large to your computer. Anything less than a few dozen megabytes is small enough to be read entirely into memory. > Inspired by http://code.activestate.com/recipes/81330/, I came up with > the following code: If all you are doing is replacing single characters, then there's no need for the 80lb sledgehammer of regular expressions when all you need is a delicate tack hammer. Instead of this: * read the file as bytes * search for pairs of bytes like chr(195)+chr(130) using a regex * replace them with single bytes like 'A' do this: * read the file as a Unicode * search for characters like  * replace them with single characters like A using unicode.translate() (or str.translate() in Python 3.x) The only gotcha is that you need to know (or guess) the encoding to read the file correctly. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Some syntactic sugar proposals
On Wed, 17 Nov 2010 16:31:40 +, Mark Wooding wrote: > But I don't think that's the big problem with this proposal. The real > problem is that it completely changes the evaluation rule for the > conditional expression. (The evaluation rule is already pretty screwy: > Python is consistently left-to-right -- except here.) Not quite... >>> 1+2*3 7 >>> (1+2)*3 9 But other than that, I agree with your analysis for why Python should not be changed to allow: t = foo() as v if pred(v) else default_value Not everything needs to be a one liner. If you need this, do it the old- fashioned way: t = foo() if not pred(t): t = default_value -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: String multi-replace
On Wed, Nov 17, 2010 at 8:21 PM, Sorin Schwimmer wrote: > Hi All, > > I have to eliminate diacritics in a fairly large file. > > Inspired by http://code.activestate.com/recipes/81330/, I came up with the > following code: > > #! /usr/bin/env python > > import re > > nodia={chr(196)+chr(130):'A', # mamaliga > chr(195)+chr(130):'A', # A^ > chr(195)+chr(142):'I', # I^ > chr(195)+chr(150):'O', # OE > chr(195)+chr(156):'U', # UE > chr(195)+chr(139):'A', # AE > chr(197)+chr(158):'S', > chr(197)+chr(162):'T', > chr(196)+chr(131):'a', # mamaliga > chr(195)+chr(162):'a', # a^ > chr(195)+chr(174):'i', # i^ > chr(195)+chr(182):'o', # oe > chr(195)+chr(188):'u', # ue > chr(195)+chr(164):'a', # ae > chr(197)+chr(159):'s', > chr(197)+chr(163):'t' > } > name="R\xc3\xa2\xc5\x9fca" > > regex = re.compile("(%s)" % "|".join(map(re.escape, nodia.keys( > print regex.sub(lambda mo: dict[mo.string[mo.start():mo.end()]], name) Have you considered using string.maketrans() and str.translate() instead? It's simpler and likely faster than generating+using regexes like that. http://docs.python.org/library/string.html#string.maketrans Cheers, Chris -- Cue someone quoting Zawinski. http://blog.rebertia.com -- http://mail.python.org/mailman/listinfo/python-list
Re: Help in language development
Hello Pedro Igor, On Tue, 16 Nov 2010 12:51:07 -0800, pedro igor sampaio avelino wrote: > Hello, my name Pedro Igor, I am a student and develop applications in > python for 1 year. I enrolled in the group to contribute in developing > this wonderful language that helps me both in day-to-day, but I'm going > through some difficulties because they do not know where to start, can > someone please give me some steps so that I can contribute at once > development for all, where do I start, if the documentation or a book. I > know that most people have more important issues to address in other > posts but assistance does not cost anything. If you are interested in helping others, you can start here, on this newsgroup or mailing list. When other people ask questions, try to give the answer yourself. Or you could go to the Python wiki and contribute: http://wiki.python.org/moin/ Or go to the bug tracker, and look for bugs you can work on. Be aware though that the standard for having patches accepted is quite high. http://bugs.python.org/ -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: simple(?) Python C module question
On Wed, Nov 17, 2010 at 7:42 PM, Terry Reedy wrote: > On 11/17/2010 7:25 PM, Mark Crispin wrote: > > Have you looked at ctypes? It's not suitable for all libraries, but >>> it can often obviate the need to write any C code: >>> http://docs.python.org/release/2.6.6/library/ctypes.html#module-ctypes >>> >> >> Hmm. I don't think that it helps, especially as I don't really want to >> make the consumers of this module know anything about c-client or its >> calling conventions. >> > > For the record, a C library wrapper written in Python with cytpes is nearly > indistinguishable to users from an equivalent wrapper written in C. However, > given that you are more comfortable with C than Python and have gotten the > info you need, stick with that. > ...except when you go to recompile a C extension module, if a dependency has changed its API, the C compiler will at least output a warning at the relevant spot. ctypes code is likely to be rather less friendly about it - the word "segfault" comes to mind. I'd say that the 3 main options for C <-> Python interop are: 1) ctypes (nice for light use and if you only need python, not that well type checked, probably the best bet if you're targeting python implementations other than CPython alone) 2) swig (nice if you plan to expose your C API to lots of languages) 3) cython (nice if you only need python, and want to more or less freely intermix normal python with C symbols using a very python-like syntax and with speed near that of C) -- http://mail.python.org/mailman/listinfo/python-list
Re: String multi-replace
On Wed, Nov 17, 2010 at 11:21 PM, Sorin Schwimmer wrote: > Hi All, > > I have to eliminate diacritics in a fairly large file. > > Inspired by http://code.activestate.com/recipes/81330/, I came up with the > following code: > > #! /usr/bin/env python > > import re > > nodia={chr(196)+chr(130):'A', # mamaliga > chr(195)+chr(130):'A', # A^ > chr(195)+chr(142):'I', # I^ > chr(195)+chr(150):'O', # OE > chr(195)+chr(156):'U', # UE > chr(195)+chr(139):'A', # AE > chr(197)+chr(158):'S', > chr(197)+chr(162):'T', > chr(196)+chr(131):'a', # mamaliga > chr(195)+chr(162):'a', # a^ > chr(195)+chr(174):'i', # i^ > chr(195)+chr(182):'o', # oe > chr(195)+chr(188):'u', # ue > chr(195)+chr(164):'a', # ae > chr(197)+chr(159):'s', > chr(197)+chr(163):'t' > } > name="R\xc3\xa2\xc5\x9fca" > > regex = re.compile("(%s)" % "|".join(map(re.escape, nodia.keys( > print regex.sub(lambda mo: dict[mo.string[mo.start():mo.end()]], name) > > But it won't work; I end up with: > > Traceback (most recent call last): > File "multirep.py", line 25, in > print regex.sub(lambda mo: dict[mo.string[mo.start():mo.end()]], name) > File "multirep.py", line 25, in > print regex.sub(lambda mo: dict[mo.string[mo.start():mo.end()]], name) > TypeError: 'type' object is not subscriptable > > What am I doing wrong? > > Thanks for your advice, > SxN > dict is a type, not a dict. Your dict is called nodia. I'm guess that's what you meant to use. > > -- > http://mail.python.org/mailman/listinfo/python-list > -- http://mail.python.org/mailman/listinfo/python-list
String multi-replace
Hi All, I have to eliminate diacritics in a fairly large file. Inspired by http://code.activestate.com/recipes/81330/, I came up with the following code: #! /usr/bin/env python import re nodia={chr(196)+chr(130):'A', # mamaliga chr(195)+chr(130):'A', # A^ chr(195)+chr(142):'I', # I^ chr(195)+chr(150):'O', # OE chr(195)+chr(156):'U', # UE chr(195)+chr(139):'A', # AE chr(197)+chr(158):'S', chr(197)+chr(162):'T', chr(196)+chr(131):'a', # mamaliga chr(195)+chr(162):'a', # a^ chr(195)+chr(174):'i', # i^ chr(195)+chr(182):'o', # oe chr(195)+chr(188):'u', # ue chr(195)+chr(164):'a', # ae chr(197)+chr(159):'s', chr(197)+chr(163):'t' } name="R\xc3\xa2\xc5\x9fca" regex = re.compile("(%s)" % "|".join(map(re.escape, nodia.keys( print regex.sub(lambda mo: dict[mo.string[mo.start():mo.end()]], name) But it won't work; I end up with: Traceback (most recent call last): File "multirep.py", line 25, in print regex.sub(lambda mo: dict[mo.string[mo.start():mo.end()]], name) File "multirep.py", line 25, in print regex.sub(lambda mo: dict[mo.string[mo.start():mo.end()]], name) TypeError: 'type' object is not subscriptable What am I doing wrong? Thanks for your advice, SxN -- http://mail.python.org/mailman/listinfo/python-list
Re: Using Python and Connecting to MySQL remotely WITHOUT MySQL installed on local computer
On 11/17/2010 10:19 PM, Tim Harig wrote: > On 2010-11-18, Steve Holden wrote: >> On 11/17/2010 7:21 PM, Tim Harig wrote: >>> On 2010-11-18, dave wrote: http://sourceforge.net/projects/mysql-python/ Using this package, WITHOUT having MySQL installed on my Mac OS X, how can I use python to connect to a remote MySQL server? All of the tutorials mention having to download MySQL! >>> >>> You don't have to install all of MySQL, just the client libraries. I would >>> assume that almost every MySQL connector uses these libraries; but, you >>> might see if the MySQL-ODBC connector will work without them. It is a long >>> shot. Your last option would be to recreate your own connector without >>> using the MySQL client libraries. I am not really sure what the purpose of >>> reinventing this wheel would be. >> >> I believe that the coming trend is to implement the MySQL client >> protocol directly in Python, thereby obviating the need for any MySQL >> client installation on the machine hosting the Python code. > > One of the advantages to using the MySQL supplied library is that if the > line protocol changes the connector automatically inherits the work already > done by MySQL for the price of a stable API (yes, I am aware that the MySQL > API *has* changed through the years). That could be very relevant in the > near future as the probject settles its stable forks. > >> The pymysql project at http://code.google.com/p/pymysql/ is one such >> solution, aimed at satisfying Python 3 users without the need to port >> existing low-level client code. > > That still looks like alpha code. Would you be willing to make a statement > as to its stability? No, that was purely an example. I am, however, using MySQL Connector/Python https://launchpad.net/myconnpy in a series of commercial Python 3 classes. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon 2011 Atlanta March 9-17 http://us.pycon.org/ See Python Video! http://python.mirocommunity.org/ Holden Web LLC http://www.holdenweb.com/ -- http://mail.python.org/mailman/listinfo/python-list
How to run an EXE, with argument, capture output value
Hello All, I would appreciate some guidance on this. I'm a newbe, sorry if I sound dumb - I kind of am on this stuff! I have an executable that I want to run within python code. The exe requires an input text file, the user to click a 'compute' button, and then the exe calculates several output values, one of which I want to capture into a variable. Can I use Python to supply the input file, execute the exe and capture the output value, like such that the exe really doesn't need to be 'seen'? Or, would the user still have to click the 'compute' button? Any code snippets or guidance would be very much appreciated. I have found that import os os.system('C:\xTool\stats_hall.exe') will run the exe. And, maybe these execl/execle/execlp/etc functions might be what I need for adding in the argument, but documentation seems to indicate that these do not return output. ?? Thanks much. -- http://mail.python.org/mailman/listinfo/python-list
[ANN]: 'tren' Cross-Platform Batch Renaming Tool, Version 1.239 Released
'tren' Version 1.239 is now released and available for download at: http://www.tundraware.com/Software/tren The last public release was 1.217. - What's New In This Release? --- This release fixes several critical bugs and adds a number of new features. Existing users are urged to upgrade at their earliest opportunity: FIXED: Backups weren't being improperly named. FIXED: Changed TREN envvar processing to properly parse quoting. FIXED: Each renaming target must either be something that exists or a wildcard that expands to something that exists. Formerly, nonexistent renaming targets were silently ignored. NEW: Debug now shows the incremental change to each file name as each renaming request is applied left-to-right. Use -dq to see this more clearly. NEW: The environment variable TRENINCL is now supported. This allows you to specify a path to search when looking for include files. NEW: Implemented /NAMESOFAR/ renaming token. NEW: Added -e type case conversion option - new kind of renaming request. c - Capitalize l - Lower s - Swap case t - Title -> Char's followiing non-alpha are capitalized u - Upper NEW: Added -T option to allow user to "target" a substring of the full filename for renaming OTHER: Documentation additions and updates including clarification of old features, description of new features, and improved table-of-contents for PDF and PS document formats. What Is 'tren'? -- 'tren' is a general purpose file and directory renaming tool. Unlike commands like 'mv', 'tren' is particularly well suited for renaming *batches* of files and/or directories with a single command line invocation. 'tren' eliminates the tedium of having to script simpler tools to provide higher-level renaming capabilities. 'tren' is also adept at renaming only *part of an existing file or directory name* either based on a literal string or a regular expression pattern. You can replace any single, group, or all instances of a given string in a file or directory name. 'tren' implements the idea of a *renaming token*. These are special names you can embed in your renaming requests that represent things like the file's original name, its length, date of creation, and so on. There are even renaming tokens that will substitute the content of any environment variable or the results of running a program from a shell back into the new file name. 'tren' can automatically generate *sequences* of file names based on their dates, lengths, times within a given date, and so on. In fact, sequences can be generated on the basis of any of the file's 'stat' information. Sequence "numbers" can be ascending or descending and the count can start at any initial value. Counting can take place in one of several internally defined counting "alphabets" (decimal, hex, octal, alpha, etc.) OR you can define your own counting alphabet. This allows you to create sequences in any base (2 or higher please :) using any symbol set for the count. 'tren' is written in pure Python and requires Python version 2.6.x or later. It is known to run on various Unix-like variants (FreeBSD, Linux, MacOS X) as well as Windows. It will also take advantage of 'win32all' Python extensions on a Windows system, if they are present. - Complete details of all fixes, changes, and new features can be found in the WHATSNEW.txt and documentation files included in the distribution. A FreeBSD port has been submitted as well. -- http://mail.python.org/mailman/listinfo/python-list
Re: simple(?) Python C module question
On 11/17/2010 7:25 PM, Mark Crispin wrote: Have you looked at ctypes? It's not suitable for all libraries, but it can often obviate the need to write any C code: http://docs.python.org/release/2.6.6/library/ctypes.html#module-ctypes Hmm. I don't think that it helps, especially as I don't really want to make the consumers of this module know anything about c-client or its calling conventions. For the record, a C library wrapper written in Python with cytpes is nearly indistinguishable to users from an equivalent wrapper written in C. However, given that you are more comfortable with C than Python and have gotten the info you need, stick with that. -- Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
Re: Using Python and Connecting to MySQL remotely WITHOUT MySQL installed on local computer
On 2010-11-18, Steve Holden wrote: > On 11/17/2010 7:21 PM, Tim Harig wrote: >> On 2010-11-18, dave wrote: >>> http://sourceforge.net/projects/mysql-python/ >>> >>> Using this package, WITHOUT having MySQL installed on my Mac OS X, how >>> can I use python to connect to a remote MySQL server? >>> >>> All of the tutorials mention having to download MySQL! >> >> You don't have to install all of MySQL, just the client libraries. I would >> assume that almost every MySQL connector uses these libraries; but, you >> might see if the MySQL-ODBC connector will work without them. It is a long >> shot. Your last option would be to recreate your own connector without >> using the MySQL client libraries. I am not really sure what the purpose of >> reinventing this wheel would be. > > I believe that the coming trend is to implement the MySQL client > protocol directly in Python, thereby obviating the need for any MySQL > client installation on the machine hosting the Python code. One of the advantages to using the MySQL supplied library is that if the line protocol changes the connector automatically inherits the work already done by MySQL for the price of a stable API (yes, I am aware that the MySQL API *has* changed through the years). That could be very relevant in the near future as the probject settles its stable forks. > The pymysql project at http://code.google.com/p/pymysql/ is one such > solution, aimed at satisfying Python 3 users without the need to port > existing low-level client code. That still looks like alpha code. Would you be willing to make a statement as to its stability? -- http://mail.python.org/mailman/listinfo/python-list
Re: returning results from function
On 11Nov2010 15:29, Chris Rebert wrote: | > On Nov 11, 2010, at 1:54 PM, Chris Rebert wrote: | >> On Thu, Nov 11, 2010 at 1:16 PM, Neil Berg wrote: | >> time_y = ncfile.variables['time_y'][:] # (time,int) [yrs] | >> time_m = ncfile.variables['time_m'][:] # (time,int) [mnths] | >> time_d = ncfile.variables['time_d'][:] # (time,int) [days] | >> time_h = ncfile.variables['time_h'][:] # (time,float) [hrs] | >> ntim =len(time_h) | >> for tim_idx in range(0,ntim): | >> local_date = utc_to_local(time_y[tim_idx],time_m[tim_idx],time_d[tim_idx],int(time_h[tim_idx])) | >> ***Here is where I'd like to see the returned values so I can create new arrays that store them * | | Add: | print(local_date) | Or if you want to be fancy: | print("%.4d-%.2d-%.2d %.2d" % local_date) It's worth noting that if you know the number of return arguments you can do this: locyr, locmm, locdd, lochh = itc_to_local(...) and then easily use the individual values. Cheers, -- Cameron Simpson DoD#743 http://www.cskk.ezoshosting.com/cs/ Admit Nothing. Blame Everyone. Be Bitter. - Matt Hopkins, DoD #1197, -- http://mail.python.org/mailman/listinfo/python-list
Re: Using Python and Connecting to MySQL remotely WITHOUT MySQL installed on local computer
On 11/17/2010 7:21 PM, Tim Harig wrote: > On 2010-11-18, dave wrote: >> http://sourceforge.net/projects/mysql-python/ >> >> Using this package, WITHOUT having MySQL installed on my Mac OS X, how >> can I use python to connect to a remote MySQL server? >> >> All of the tutorials mention having to download MySQL! > > You don't have to install all of MySQL, just the client libraries. I would > assume that almost every MySQL connector uses these libraries; but, you > might see if the MySQL-ODBC connector will work without them. It is a long > shot. Your last option would be to recreate your own connector without > using the MySQL client libraries. I am not really sure what the purpose of > reinventing this wheel would be. I believe that the coming trend is to implement the MySQL client protocol directly in Python, thereby obviating the need for any MySQL client installation on the machine hosting the Python code. The pymysql project at http://code.google.com/p/pymysql/ is one such solution, aimed at satisfying Python 3 users without the need to port existing low-level client code. As an irrelevance might I also add that the trend for built-in extension modules is to require a reference implementation in Python to ease the task of those wishing to port the language and get as much functionality (albeit at some performance in cost) available as early in the porting cycle as possible. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon 2011 Atlanta March 9-17 http://us.pycon.org/ See Python Video! http://python.mirocommunity.org/ Holden Web LLC http://www.holdenweb.com/ -- http://mail.python.org/mailman/listinfo/python-list
Re: How to read such file and sumarize the data?
On 11/17/2010 7:51 PM, Terry Reedy wrote: > On 11/17/2010 6:10 PM, Steve Holden wrote: > >> $ cat data.py >> lines = open("data.txt").readlines() > > Since you iterate through the file just once, there is no reason I can > think of to make a complete in-memory copy. That would be a problem with > a multi-gigabyte log file ;=). In 3.x at least, open files are line > iterators and one would just need > You are indeed perfectly correct. Thank you. Probably old-ingrained habits showing through. Open files have been line iterators since 2.2, I believe. regards Steve > lines = open("data.txt") > >> from collections import defaultdict >> c = defaultdict(int) >> for line in lines: >> ls = line.split() >> if len(ls)> 3 and ls[3].startswith("NCPU="): >> amt = int(ls[3][5:]) >> c[ls[0]] += amt >> for key, value in c.items(): >> print key, ":", value >> >> >> $ python data.py >> xyz : 4 >> tanhoi : 1 >> sabril : 6 >> >> regards >> Steve > > -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon 2011 Atlanta March 9-17 http://us.pycon.org/ See Python Video! http://python.mirocommunity.org/ Holden Web LLC http://www.holdenweb.com/ -- http://mail.python.org/mailman/listinfo/python-list
Re: Program, Application, and Software
On 18/11/2010 00:28, Ben Finney wrote: Alexander Kapps writes: On 17.11.2010 19:38, Boštjan Mejak wrote: What is the difference between a program, an application, and software? Alexander's guide is good. Some notes from a native speaker of English: Program: A sequence of one or more instructions (even 'print "hello"' is a valid Python program) Some like to sub-divide “script” as a type of program; I don't see the point, and refer to such things as programs. [snip] I'd probably say that a "script" is a program which is normally not interactive: you just set it up, start it, and let it do its work (a "batch script", for example). It's also written in a language primarily designed for convenience rather than speed (Want to manipulate large chunks of text? Fine! :-)). -- http://mail.python.org/mailman/listinfo/python-list
Re: How to read such file and sumarize the data?
On 11/17/2010 6:10 PM, Steve Holden wrote: $ cat data.py lines = open("data.txt").readlines() Since you iterate through the file just once, there is no reason I can think of to make a complete in-memory copy. That would be a problem with a multi-gigabyte log file ;=). In 3.x at least, open files are line iterators and one would just need lines = open("data.txt") from collections import defaultdict c = defaultdict(int) for line in lines: ls = line.split() if len(ls)> 3 and ls[3].startswith("NCPU="): amt = int(ls[3][5:]) c[ls[0]] += amt for key, value in c.items(): print key, ":", value $ python data.py xyz : 4 tanhoi : 1 sabril : 6 regards Steve -- Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
Re: Program, Application, and Software
On 11/17/2010 05:10 PM, Alexander Kapps wrote: On 17.11.2010 19:38, Boštjan Mejak wrote: What is the difference between a program, an application, and software? Software: The parts of a computer that you *can't* kick. Programmer: the part that usually gets kicked... -tkc -- http://mail.python.org/mailman/listinfo/python-list
Re: simple(?) Python C module question
On Thu, 18 Nov 2010, Mark Wooding posted: [snip] Whoo-hoo! That's exactly what I was looking for. If we ever meet in person, I owe you a beer, sir. And by that I mean real beer (from what we call a "microbrew"), not Budweiser... :) -- Mark -- http://panda.com/mrc Democracy is two wolves and a sheep deciding what to eat for lunch. Liberty is a well-armed sheep contesting the vote. -- http://mail.python.org/mailman/listinfo/python-list
Re: Program, Application, and Software
Alexander Kapps writes: > Application: Usually a large(er), complex program I'd say that an `application' is specifically a program intended for direct human use. Other things are servers, daemons and utilities. But I might just be weird. -- [mdw] -- http://mail.python.org/mailman/listinfo/python-list
Re: simple(?) Python C module question
Mark Crispin writes: > I have a Python module written in C that interfaces with an external C > library. Basically, the project is to make it possible to use that > library from Python scripts. If you know who I am, you can guess > which library. :) You have your very own Wikipedia page, so others probably needn't guess. > However, I now need to write a method that creates what the library calls a > "stream", and I need method calls to work on that stream. > > The obvious way to do this in any other OO language is to have an > object that holds the stream from the library in an instance variable > (which actually will be constant for that object instance), and has > various object methods for operating on that stream. Yeah, that's pretty much the way it works in Python, though it's a bit more tedious. (I suppose it's too late to suggest using Cython now. It makes this sort of thing much less unpleasant.) > I assume that the object methods are defined by a PyMethodDef table, > just as they are for the module. But how do I: > [1] define the object You need two things here: a C structure to represent the innards of your custom Python type, and a Python type block (a PyTypeObject) providing everything the Python interpreter needs to know about it. The C structure can be pretty much whatever you like, as long as it begins with the macro PyObject_HEAD: typedef struct thingy { PyObject_HEAD /* your stuff here; maybe: */ stream *s; } thingy; The PyTypeObject is big and boring to fill in but you can probably leave most of it null. The structure is described in the Python API manual, but I think you'll need to fill in: * a name for the type (qualified by your module's name; tp_name); * the object size (sizeof(thingy); tp_basicsize); * a deallocation function (tp_dealloc); * some flags (you probably want Py_TPFLAGS_DEFAULT and maybe Py_TPFLAGS_BASETYPE if you don't mind people making subclasses; tp_flags); * a string to be shown to the user if he asks for help about your type (tp_doc); * a pointer to a methods table (tp_methods); * a pointer to a table of attribute getters and setters (if you want to expose stuff as attributes rather than methods; tp_getset); * an allocator (probably PyType_GenericAlloc; tp_alloc); and * a function to construct a new instance (to be called when Python tries to construct an object by calling the type -- leave it null if you construct instances in some other way; tp_new). You might also want to implement the tp_str function to provide useful information about your object's state (e.g., for debugging scripts). There are some standard protocols which might be useful to implement; it doesn't sound like your streams would benefit from behaving like numbers or sequences, but maybe they might be iterable. You attach methods on the type by mentioning them in the PyMethodDef table you set in tp_methods; they get a pointer to the recipient object (a `thingy' as above) so they can find the underlying stream if they want. Getters and setters are pretty similar, but have a simpler interface because they don't need to mess with argument parsing.. > [2] create an instance of the object with the stream and methods If you want Python programs to be able to make streams by saying stream = mumble.stream(...) or whatever then you'll need to implement tp_new: this is pretty much like a standard method, except it gets a pointer to a PyTypeObject to tell it what kind of type to make. (The constructor may be called to construct a subclass of your type.) Parse the arguments and check that they're plausible; then make a new skeleton instance by t = (thingy *)ty->tp_alloc(ty, 0); (where ty is the type pointer you received), fill in your bits, and return (PyObject *)t. Otherwise, well, you go through the same procedure with tp_alloc, only you know which type you want statically. > [3] hook the object's destruction to a library stream-close function This is the tp_dealloc function. It should free up any of your resources, and then say obj->ob_type->tp_free(obj); Python will have arranged for this function to exist if you left tp_free as a null pointer. > Python does NOT need to look at the stream in any way. Ideally, the > object is just a blob that only has method calls. That's what you get anyway, if you use the C API. If you want Python programs to be able to poke about inside your objects, you have to let them explicitly, and that means writing code. > This ought to be simple, and not even require me to know much Python > since basically the task is just this module and a few very basic > Python scripts to use it. Other people will be writing the real > scripts. It doesn't require you to know any Python at all. It /does/ require a certain familiarity with the implementation, though. > Of course, I could just have the open method return the stream pointe
Re: Program, Application, and Software
Alexander Kapps writes: > On 17.11.2010 19:38, Boštjan Mejak wrote: > > What is the difference between a program, an application, and > > software? Alexander's guide is good. Some notes from a native speaker of English: > Program: A sequence of one or more instructions (even 'print "hello"' > is a valid Python program) Some like to sub-divide “script” as a type of program; I don't see the point, and refer to such things as programs. > Application: Usually a large(er), complex program An application might also be several programs working together, along with the non-program software they use to do their jobs: e.g. “LibreOffice is an application consisting of many programs, data sets, libraries, images, document templates, and other software”. > Software: The parts of a computer that you *can't* kick. It's worth noting a common error of non-native English users: “software” is an uncountable noun, like “hardware”. This is unlike “program” and “application”, both of which are countable. So it makes no more sense to speak of “a software” than “a clothing”; both are wrong. If you want to refer to countable items, “a software work” is acceptable if you want to refer generically to a work of digitally-stored information. Otherwise be more precise: a software work might be “a document”, “an image”, “an audio recording”, “a program”, “a database”, et cetera. Hope that helps. -- \ “It's my belief we developed language because of our deep inner | `\ need to complain.” —Jane Wagner, via Lily Tomlin | _o__) | Ben Finney -- http://mail.python.org/mailman/listinfo/python-list
Re: simple(?) Python C module question
On Wed, 17 Nov 2010, Grant Edwards posted: Hey, it's the IMAP guy! Get 'im! Busted! :p Alright, here's the full story. As may be obvious to some, the module is to be a Python interface into c-client. What may not be obvious is that this is for QA automation. The consumers of this module will know Python but not necessarily C and certainly not c-client. Right now the module is called cclient but that will certainly change. So, I have methods such as cclient.create that in my module look like: static PyObject *cclient_create(PyObject *self, PyObject *args) { char *mailbox; PyObject *ret = NULL; if(PyArg_ParseTuple(args, "s", &mailbox)) ret = Py_BuildValue("i", mail_create(NULL, mailbox)); return ret; } and indeed within Python cclient.create("testbox") creates "testbox" as expected. A bunch of other methods are the same. Now, however, I need cclient.open, and expect to do something like: static PyObject *cclient_create(PyObject *self, PyObject *args) { char *mailbox; MAILSTREAM *stream; PyObject *ret = NULL; if(PyArg_ParseTuple(args, "s", &mailbox) && (stream = mail_open(NULL, mailbox, 0))) { ret = ???; // set ret to a "cclient stream" object that has "stream" as // an instance variable, and a method table, presumably via // PyMethodDef, for this "cclient stream" object. } return ret; } So, if in Python, I do something like: stream = cclient.open("testbox") print stream.uid(1) I expect that to call an method like static PyObject *cclient_create(PyObject *self, PyObject *args) { long sequence; MAILSTREAM *stream = ???; // get stream from self PyObject *ret = NULL; if(PyArg_ParseTuple(args, "l", &sequence)) ret = Py_BuildValue("i", mail_uid(stream, sequence)); return ret; } So, basically, what I'm missing are the two "???" things above, plus how to make this "cclient stream" object call mail_close(stream). In SmallTalk or Objective C, the method table would be defined as part of a factory object that has a "new" factory method that in my case would take "stream" as an argument and set it as the instance method. Then to get the stream, I'd do something like stream = [self getStream]; Have you looked at ctypes? It's not suitable for all libraries, but it can often obviate the need to write any C code: http://docs.python.org/release/2.6.6/library/ctypes.html#module-ctypes Hmm. I don't think that it helps, especially as I don't really want to make the consumers of this module know anything about c-client or its calling conventions. Thanks for any clues... -- Mark -- http://panda.com/mrc Democracy is two wolves and a sheep deciding what to eat for lunch. Liberty is a well-armed sheep contesting the vote. -- http://mail.python.org/mailman/listinfo/python-list
Re: simple(?) Python C module question
On Wed, Nov 17, 2010 at 3:35 PM, Grant Edwards wrote: > On 2010-11-17, Mark Crispin wrote: > > Hey, it's the IMAP guy! Get 'im! > >> I have a Python module written in C that interfaces with an external >> C library. Basically, the project is to make it possible to use that >> library from Python scripts. If you know who I am, you can guess >> which library. :) > > Have you looked at ctypes? It's not suitable for all libraries, but > it can often obviate the need to write any C code: > > http://docs.python.org/release/2.6.6/library/ctypes.html#module-ctypes This. You may also want to check out [0] and [1], which use ctypes and generally make short work of writing C bindings. For a more complete example, see [2]. Geremy Condra [0]: http://code.activestate.com/recipes/576731-c-function-decorator/ [1]: http://code.activestate.com/recipes/576734-c-struct-decorator/ [2]: http://gitorious.org/evpy -- http://mail.python.org/mailman/listinfo/python-list
Re: Using Python and Connecting to MySQL remotely WITHOUT MySQL installed on local computer
On 2010-11-18, dave wrote: > http://sourceforge.net/projects/mysql-python/ > > Using this package, WITHOUT having MySQL installed on my Mac OS X, how > can I use python to connect to a remote MySQL server? > > All of the tutorials mention having to download MySQL! You don't have to install all of MySQL, just the client libraries. I would assume that almost every MySQL connector uses these libraries; but, you might see if the MySQL-ODBC connector will work without them. It is a long shot. Your last option would be to recreate your own connector without using the MySQL client libraries. I am not really sure what the purpose of reinventing this wheel would be. -- http://mail.python.org/mailman/listinfo/python-list
Using Python and Connecting to MySQL remotely WITHOUT MySQL installed on local computer
http://sourceforge.net/projects/mysql-python/ Using this package, WITHOUT having MySQL installed on my Mac OS X, how can I use python to connect to a remote MySQL server? All of the tutorials mention having to download MySQL! -- http://mail.python.org/mailman/listinfo/python-list
Re: Is Unladen Swallow dead?
On Nov 17, 2010, at 5:09 PM, John Nagle wrote: > On 11/17/2010 12:49 PM, John Ladasky wrote: >> On Nov 16, 2:30 pm, laspi wrote: >>> Is Unladen Swallow dead? >> >> No, it's just resting. > > For those who don't get that, The Monty Python reference: > "http://www.mtholyoke.edu/~ebarnes/python/dead-parrot.htm"; A link to the source material: http://www.youtube.com/user/montypython?blend=1&ob=4#p/c/6FD5A97331C1B802/0/npjOSLCR2hE -- Cheers Philip -- http://mail.python.org/mailman/listinfo/python-list
Re: simple(?) Python C module question
On Wed, Nov 17, 2010 at 3:18 PM, Mark Crispin wrote: > I have a Python module written in C that interfaces with an external C > library. Basically, the project is to make it possible to use that library > from Python scripts. If you know who I am, you can guess which library. :) > > I have gotten as far as writing the module, and I can call methods in the > module that call the library and do the intended thing. So far, so good. > > However, I now need to write a method that creates what the library calls a > "stream", and I need method calls to work on that stream. > > The obvious way to do this in any other OO language is to have an object > that holds the stream from the library in an instance variable (which > actually will be constant for that object instance), and has various object > methods for operating on that stream. > > I assume that the object methods are defined by a PyMethodDef table, just as > they are for the module. But how do I: > [1] define the object > [2] create an instance of the object with the stream and methods > [3] hook the object's destruction to a library stream-close function > > Python does NOT need to look at the stream in any way. Ideally, the object > is just a blob that only has method calls. > Of course, I could just have the open method return the stream pointer as a > big int, and have module methods that take the stream pointer as their first > argument, just as in C code. If I did that, the project would have been > done by now. But the result wouldn't be very OO or Pythonish; and more to > the point other people will have to use it. I hate when people inflict > quick, dirty, stupid, ugly, lazy programmer abominations on me; and so I > feel obligated to do it right rather than inflict an abomination on > others... :) > > Thanks in advance for any pointers and/or help. Definitely not a direct answer to your questions, but you might consider using SWIG (http://www.swig.org/Doc1.3/Python.html ); haven't used it myself, but I've generally heard good things about it. Ironically, from the docs, it seems the C modules it generates approximately use your "abomination" approach (except the pointer is at least stored as an opaque object rather than an int); however, importantly, it also generates a .py module with appropriate nice wrapper classes to wrap the function calls as much more natural method calls. So, directly using your "abomination" approach and then manually writing a wrapper class in Python itself is also another option. Cheers, Chris -- http://blog.rebertia.com -- http://mail.python.org/mailman/listinfo/python-list
first attempts with pybluez on Linux fail as well
Hi, Wanted to write a first simple example with pybluez and offer a serial connection service with a given name. What I tried (being inspired by http://people.csail.mit.edu/albert/bluez-intro/x290.html ) is: server_sock=bluetooth.BluetoothSocket( bluetooth.RFCOMM ) port = bluetooth.PORT_ANY # original example used get_available_port() # buy this seems obsolete server_sock.bind(("",port)) server_sock.listen(1) print "listening on port %d" % port loc_host, loc_port = server_sock.getsockname() print "HOST", loc_host, "PORT??", loc_port uuid = "1e0ca4ea-299d-4335-93eb-27fcfe7fa848" bluetooth.advertise_service( server_sock, "FooBar Service", uuid ) Unde Linux the file does not produce an error message, but When I use another PC to look for services I do NOT see the service "FooBar Service" What might I be doing wrong? Are there any special requirements on the buetooth dongle / bluetooth stack to make abov example work? Thanks a lot for your help. -- http://mail.python.org/mailman/listinfo/python-list
Re: How to read such file and sumarize the data?
On Wed, 17 Nov 2010 13:45:58 -0800, huisky wrote: > Say I have following log file, which records the code usage. I want to > read this file and do the summarize how much total CPU time consumed for > each user. > Two points you should think about: - I don't think you can extract CPU time from this log: you can get the process elapsed time and the number of CPUs each run has used, but you can't calculate CPU time from those values since you don't know how the process spent waiting for i/o etc. - is the first (numeric) part of the first field on the line a process id? If it is, you can match start and stop messages on the value of the first field provided that this value can never be shared by two processes that are both running. If you can get simultaneous duplicates, then you're out of luck because you'll never be able to match up start and stop lines. > Is Python able to do so or say easy to achieve this?, anybody can give > me some hints, appricate very much! > Sure. There are two approaches possible: - sort the log on the first two fields and then process it with Python knowing that start and stop lines will be adjacent - use the first field as the key to an array and put the start time and CPU count in that element. When a matching stop line is found you, retrieve the array element, calculate and output or total the usage figure for that run and delete the array element. -- martin@ | Martin Gregorie gregorie. | Essex, UK org | -- http://mail.python.org/mailman/listinfo/python-list
Re: Is Unladen Swallow dead?
On Wed, 17 Nov 2010 23:51:01 +0100, Alexander Kapps wrote: > On 17.11.2010 23:09, John Nagle wrote: >> On 11/17/2010 12:49 PM, John Ladasky wrote: >>> On Nov 16, 2:30 pm, laspi wrote: Is Unladen Swallow dead? >>> >>> No, it's just resting. >> >> For those who don't get that, The Monty Python reference: >> "http://www.mtholyoke.edu/~ebarnes/python/dead-parrot.htm"; > > Thank you John for making my light enough Wallet even lighter, now I > have to go and buy the original English version. Seems the German > translation sucks (misses a lot) and my copy lacks the original dub. > While you're at it, pick up the video of "Monty Python and the Holy Grail". the project name, Unladen Swallow, is a reference to the film. -- martin@ | Martin Gregorie gregorie. | Essex, UK org | -- http://mail.python.org/mailman/listinfo/python-list
Re: simple(?) Python C module question
On 2010-11-17, Mark Crispin wrote: Hey, it's the IMAP guy! Get 'im! > I have a Python module written in C that interfaces with an external > C library. Basically, the project is to make it possible to use that > library from Python scripts. If you know who I am, you can guess > which library. :) Have you looked at ctypes? It's not suitable for all libraries, but it can often obviate the need to write any C code: http://docs.python.org/release/2.6.6/library/ctypes.html#module-ctypes -- Grant Edwards grant.b.edwardsYow! I wonder if I should at put myself in ESCROW!! gmail.com -- http://mail.python.org/mailman/listinfo/python-list
Re: Is Unladen Swallow dead?
On 11/17/10 4:51 PM, Alexander Kapps wrote: On 17.11.2010 23:09, John Nagle wrote: On 11/17/2010 12:49 PM, John Ladasky wrote: On Nov 16, 2:30 pm, laspi wrote: Is Unladen Swallow dead? No, it's just resting. For those who don't get that, The Monty Python reference: "http://www.mtholyoke.edu/~ebarnes/python/dead-parrot.htm"; Thank you John for making my light enough Wallet even lighter, now I have to go and buy the original English version. Seems the German translation sucks (misses a lot) and my copy lacks the original dub. They're all (legitimately) on Youtube now. http://www.youtube.com/watch?v=npjOSLCR2hE -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: How to read such file and sumarize the data?
On 17/11/2010 22:49, Tim Harig wrote: On 2010-11-17, MRAB wrote: When a user starts, save the info in the first dict, and when a user finishes, calculate the elapsed time and add it to the total for that user. Perhaps you know more about the structure of this data. It seems to me that a user might have more then a single job(?) running at once. I therefore made the assumption that each start must be matched to its coorisponding stop. I did make certain assumptions. It's up to the OP to adapt it to the actual problem accordingly. :-) -- http://mail.python.org/mailman/listinfo/python-list
simple(?) Python C module question
This is something that ought to be simple, but going through the documentation hasn't come up with the answer. Hopefully someone can answer it faster than I can figure it out from the documentation. I am using Python 2.6 for a project. I do not have a choice in the matter, so telling me to use Python 3.1 is not helpful. [I already wasted a while learning that Python 2.x and Python 3.x are very different, and documentation for the one is lies for the other] I know relatively little about Python, but am experienced in other OO languages. The OO jargon I'll use in this question is as is used in SmallTalk or perhaps Objective-C (just in case Python uses other jargon). I have a Python module written in C that interfaces with an external C library. Basically, the project is to make it possible to use that library from Python scripts. If you know who I am, you can guess which library. :) I have gotten as far as writing the module, and I can call methods in the module that call the library and do the intended thing. So far, so good. However, I now need to write a method that creates what the library calls a "stream", and I need method calls to work on that stream. The obvious way to do this in any other OO language is to have an object that holds the stream from the library in an instance variable (which actually will be constant for that object instance), and has various object methods for operating on that stream. I assume that the object methods are defined by a PyMethodDef table, just as they are for the module. But how do I: [1] define the object [2] create an instance of the object with the stream and methods [3] hook the object's destruction to a library stream-close function Python does NOT need to look at the stream in any way. Ideally, the object is just a blob that only has method calls. This ought to be simple, and not even require me to know much Python since basically the task is just this module and a few very basic Python scripts to use it. Other people will be writing the real scripts. Of course, I could just have the open method return the stream pointer as a big int, and have module methods that take the stream pointer as their first argument, just as in C code. If I did that, the project would have been done by now. But the result wouldn't be very OO or Pythonish; and more to the point other people will have to use it. I hate when people inflict quick, dirty, stupid, ugly, lazy programmer abominations on me; and so I feel obligated to do it right rather than inflict an abomination on others... :) Thanks in advance for any pointers and/or help. -- Mark -- http://panda.com/mrc Democracy is two wolves and a sheep deciding what to eat for lunch. Liberty is a well-armed sheep contesting the vote. -- http://mail.python.org/mailman/listinfo/python-list
Re: strange subprocess behavior when calling ps
On 16Nov2010 20:18, Ned Deily wrote: | In article | <55f26d5c-aba9-4892-9e2c-1caa9988e...@v23g2000vbi.googlegroups.com>, | Roger Davis wrote: | > I am running 2.6.6 under MacOS 10.6.4 on a MacBook Pro Intel. I have | > appended the code below. I am running both commands directly in a | > Terminal window running tcsh. | | See "man compat". What you are seeing is the difference between ps(1) | output in "legacy" mode, attempting to duplicate the old, non-POSIX | behavior from 10.3 days, or "unix2003" mode. Terminal login sessions | are normally automatically started with the COMMAND_MODE environment | variable set: | | $ echo $COMMAND_MODE | unix2003 | | Adding an "env={"COMMAND_MODE": "unix2003"}" argument to your subprocess | Popen call should do the trick. For an added datum, my machine (MacOSX 10.6.5 Intel Macbook Air) says "legacy" instead of "unix2003". -- Cameron Simpson DoD#743 http://www.cskk.ezoshosting.com/cs/ I understand a fury in your words, but not your words. - William Shakespeare -- http://mail.python.org/mailman/listinfo/python-list
Re: Raw Unicode docstring
On 17.11.2010 06:14, John Machin wrote: On Nov 17, 9:34 am, Alexander Kapps wrote: >>> ur"Scheißt\nderBär\nim Wald?" Nicht ohne eine Genehmigung von der Umwelt Erhaltung Abteilung. The typical response around here is "Ja, aber nur wenn er Klopapier dabei hat." :-D -- http://mail.python.org/mailman/listinfo/python-list
Re: Program, Application, and Software
On 17.11.2010 19:38, Boštjan Mejak wrote: What is the difference between a program, an application, and software? Program: A sequence of one or more instructions (even 'print "hello"' is a valid Python program) Application: Usually a large(er), complex program Software: The parts of a computer that you *can't* kick. -- http://mail.python.org/mailman/listinfo/python-list
Re: How to read such file and sumarize the data?
On 11/17/2010 4:45 PM, huisky wrote: > Say I have following log file, which records the code usage. > I want to read this file and do the summarize how much total CPU time > consumed for each user. > Is Python able to do so or say easy to achieve this?, anybody can give > me some hints, appricate very much! > > > Example log file. > ** I'm assuming the following (unquoted) data is in file "data.txt": > LSTC license server version 224 started at Sun Dec 6 18:56:48 2009 > using configuration file /usr/local/lstc/server_data > xyz 15...@trofast3.marin.ntnu.no LS-DYNA_971 NCPU=1 started Sun Dec 6 > 18:57:40 > 15...@trofast3.marin.ntnu.no completed Sun Dec 6 19:42:55 > xyz 15...@trofast3.marin.ntnu.no LS-DYNA_971 NCPU=2 started Sun Dec 6 > 20:17:02 > 15...@trofast3.marin.ntnu.no completed Sun Dec 6 20:26:03 > xyz 18...@trofast2.marin.ntnu.no LS-DYNA_971 NCPU=1 started Sun Dec 6 > 21:01:17 > 18...@trofast2.marin.ntnu.no completed Sun Dec 6 21:01:28 > tanhoi 5...@iimt-tanhoi-w.ivt.ntnu.no LS-DYNA_971 NCPU=1 started Mon > Dec 7 09:31:00 > 5...@iimt-tanhoi-w.ivt.ntnu.no presumed dead Mon Dec 7 10:36:48 > sabril 18...@trofast2.marin.ntnu.no LS-DYNA_971 NCPU=2 started Mon > Dec 7 13:14:47 > 18...@trofast2.marin.ntnu.no completed Mon Dec 7 13:24:07 > sabril 18...@trofast2.marin.ntnu.no LS-DYNA_971 NCPU=2 started Mon > Dec 7 14:21:34 > sabril 18...@trofast2.marin.ntnu.no LS-DYNA_971 NCPU=2 started Mon > Dec 7 14:28:42 > 18...@trofast2.marin.ntnu.no killed Mon Dec 7 14:31:48 > 18...@trofast2.marin.ntnu.no killed Mon Dec 7 14:32:06 The line wrapping being wrong shouldn't affect the logic. $ cat data.py lines = open("data.txt").readlines() from collections import defaultdict c = defaultdict(int) for line in lines: ls = line.split() if len(ls) > 3 and ls[3].startswith("NCPU="): amt = int(ls[3][5:]) c[ls[0]] += amt for key, value in c.items(): print key, ":", value $ python data.py xyz : 4 tanhoi : 1 sabril : 6 regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon 2011 Atlanta March 9-17 http://us.pycon.org/ See Python Video! http://python.mirocommunity.org/ Holden Web LLC http://www.holdenweb.com/ -- http://mail.python.org/mailman/listinfo/python-list
Re: Is Unladen Swallow dead?
On 17.11.2010 23:09, John Nagle wrote: On 11/17/2010 12:49 PM, John Ladasky wrote: On Nov 16, 2:30 pm, laspi wrote: Is Unladen Swallow dead? No, it's just resting. For those who don't get that, The Monty Python reference: "http://www.mtholyoke.edu/~ebarnes/python/dead-parrot.htm"; Thank you John for making my light enough Wallet even lighter, now I have to go and buy the original English version. Seems the German translation sucks (misses a lot) and my copy lacks the original dub. Damned. :-) -- http://mail.python.org/mailman/listinfo/python-list
Re: How to read such file and sumarize the data?
On 2010-11-17, MRAB wrote: > When a user starts, save the info in the first dict, and when a user > finishes, calculate the elapsed time and add it to the total for that > user. Perhaps you know more about the structure of this data. It seems to me that a user might have more then a single job(?) running at once. I therefore made the assumption that each start must be matched to its coorisponding stop. -- http://mail.python.org/mailman/listinfo/python-list
Re: How to read such file and sumarize the data?
On 2010-11-17, huisky wrote: > I want to read this file and do the summarize how much total CPU time > consumed for each user. > Is Python able to do so or say easy to achieve this?, anybody can give > me some hints, appricate very much! The question is, is the information you want available in the data. > Example log file. > ** > LSTC license server version 224 started at Sun Dec 6 18:56:48 2009 > using configuration file /usr/local/lstc/server_data > xyz 15...@trofast3.marin.ntnu.no LS-DYNA_971 NCPU=1 started Sun Dec 6 > 18:57:40 > 15...@trofast3.marin.ntnu.no completed Sun Dec 6 19:42:55 > xyz 15...@trofast3.marin.ntnu.no LS-DYNA_971 NCPU=2 started Sun Dec 6 > 20:17:02 > 15...@trofast3.marin.ntnu.no completed Sun Dec 6 20:26:03 > xyz 18...@trofast2.marin.ntnu.no LS-DYNA_971 NCPU=1 started Sun Dec 6 > 21:01:17 > 18...@trofast2.marin.ntnu.no completed Sun Dec 6 21:01:28 > tanhoi 5...@iimt-tanhoi-w.ivt.ntnu.no LS-DYNA_971 NCPU=1 started Mon > Dec 7 09:31:00 > 5...@iimt-tanhoi-w.ivt.ntnu.no presumed dead Mon Dec 7 10:36:48 > sabril 18...@trofast2.marin.ntnu.no LS-DYNA_971 NCPU=2 started Mon > Dec 7 13:14:47 > 18...@trofast2.marin.ntnu.no completed Mon Dec 7 13:24:07 > sabril 18...@trofast2.marin.ntnu.no LS-DYNA_971 NCPU=2 started Mon > Dec 7 14:21:34 > sabril 18...@trofast2.marin.ntnu.no LS-DYNA_971 NCPU=2 started Mon > Dec 7 14:28:42 > 18...@trofast2.marin.ntnu.no killed Mon Dec 7 14:31:48 > 18...@trofast2.marin.ntnu.no killed Mon Dec 7 14:32:06 I see starts, completes, kills, and presumed deads. The question is can the starts be matched to the completes and kills either from the numbers before @ or from a combination of the address and NCPU. You will need to figure out whether or not you want to count the presumed deads in your calculations. Assuming that the starts and stops can be corrilated, it is a simple matter of finding the pairs and using the datetime module to find the difference in time between them. -- http://mail.python.org/mailman/listinfo/python-list
Re: How to read such file and sumarize the data?
On 17/11/2010 21:45, huisky wrote: Say I have following log file, which records the code usage. I want to read this file and do the summarize how much total CPU time consumed for each user. Is Python able to do so or say easy to achieve this?, anybody can give me some hints, appricate very much! Example log file. ** LSTC license server version 224 started at Sun Dec 6 18:56:48 2009 using configuration file /usr/local/lstc/server_data xyz 15...@trofast3.marin.ntnu.no LS-DYNA_971 NCPU=1 started Sun Dec 6 18:57:40 15...@trofast3.marin.ntnu.no completed Sun Dec 6 19:42:55 xyz 15...@trofast3.marin.ntnu.no LS-DYNA_971 NCPU=2 started Sun Dec 6 20:17:02 15...@trofast3.marin.ntnu.no completed Sun Dec 6 20:26:03 xyz 18...@trofast2.marin.ntnu.no LS-DYNA_971 NCPU=1 started Sun Dec 6 21:01:17 18...@trofast2.marin.ntnu.no completed Sun Dec 6 21:01:28 tanhoi 5...@iimt-tanhoi-w.ivt.ntnu.no LS-DYNA_971 NCPU=1 started Mon Dec 7 09:31:00 5...@iimt-tanhoi-w.ivt.ntnu.no presumed dead Mon Dec 7 10:36:48 sabril 18...@trofast2.marin.ntnu.no LS-DYNA_971 NCPU=2 started Mon Dec 7 13:14:47 18...@trofast2.marin.ntnu.no completed Mon Dec 7 13:24:07 sabril 18...@trofast2.marin.ntnu.no LS-DYNA_971 NCPU=2 started Mon Dec 7 14:21:34 sabril 18...@trofast2.marin.ntnu.no LS-DYNA_971 NCPU=2 started Mon Dec 7 14:28:42 18...@trofast2.marin.ntnu.no killed Mon Dec 7 14:31:48 18...@trofast2.marin.ntnu.no killed Mon Dec 7 14:32:06 Here's how I would probably do it: Use 2 dicts, one for the start time of each user, and the other for the total elapsed time of each user. For each line extract the user name, date/time and whether the user is starting, finishing, etc. When a user starts, save the info in the first dict, and when a user finishes, calculate the elapsed time and add it to the total for that user. The date/time can be parsed by the time or datetime module. -- http://mail.python.org/mailman/listinfo/python-list
Re: Is Unladen Swallow dead?
On 11/17/2010 12:49 PM, John Ladasky wrote: On Nov 16, 2:30 pm, laspi wrote: Is Unladen Swallow dead? No, it's just resting. For those who don't get that, The Monty Python reference: "http://www.mtholyoke.edu/~ebarnes/python/dead-parrot.htm"; Owner: Oh yes, the, uh, the Norwegian Blue...What's,uh...What's wrong with it? Mr. Praline: I'll tell you what's wrong with it, my lad. 'E's dead, that's what's wrong with it! Owner: No, no, 'e's uh,...he's resting. Mr. Praline: Look, matey, I know a dead parrot when I see one, and I'm looking at one right now. Owner: No no he's not dead, he's, he's restin'! Remarkable bird, the Norwegian Blue, idn'it, ay? Beautiful plumage! Mr. Praline: The plumage don't enter into it. It's stone dead. Owner: Nononono, no, no! 'E's resting! Mr. Praline: All right then, if he's restin', I'll wake him up! (shouting at the cage) 'Ello, Mister Polly Parrot! I've got a lovely fresh cuttle fish for you if you show... (owner hits the cage) Owner: There, he moved! Mr. Praline: No, he didn't, that was you hitting the cage! Owner: I never!! Mr. Praline: Yes, you did! Owner: I never, never did anything... Mr. Praline: (yelling and hitting the cage repeatedly) 'ELLO POLLY! Testing! Testing! Testing! Testing! This is your nine o'clock alarm call! (Takes parrot out of the cage and thumps its head on the counter. Throws it up in the air and watches it plummet to the floor.) Mr. Praline: Now that's what I call a dead parrot. (There's more, but you get the idea.) John Nagle -- http://mail.python.org/mailman/listinfo/python-list
How to read such file and sumarize the data?
Say I have following log file, which records the code usage. I want to read this file and do the summarize how much total CPU time consumed for each user. Is Python able to do so or say easy to achieve this?, anybody can give me some hints, appricate very much! Example log file. ** LSTC license server version 224 started at Sun Dec 6 18:56:48 2009 using configuration file /usr/local/lstc/server_data xyz 15...@trofast3.marin.ntnu.no LS-DYNA_971 NCPU=1 started Sun Dec 6 18:57:40 15...@trofast3.marin.ntnu.no completed Sun Dec 6 19:42:55 xyz 15...@trofast3.marin.ntnu.no LS-DYNA_971 NCPU=2 started Sun Dec 6 20:17:02 15...@trofast3.marin.ntnu.no completed Sun Dec 6 20:26:03 xyz 18...@trofast2.marin.ntnu.no LS-DYNA_971 NCPU=1 started Sun Dec 6 21:01:17 18...@trofast2.marin.ntnu.no completed Sun Dec 6 21:01:28 tanhoi 5...@iimt-tanhoi-w.ivt.ntnu.no LS-DYNA_971 NCPU=1 started Mon Dec 7 09:31:00 5...@iimt-tanhoi-w.ivt.ntnu.no presumed dead Mon Dec 7 10:36:48 sabril 18...@trofast2.marin.ntnu.no LS-DYNA_971 NCPU=2 started Mon Dec 7 13:14:47 18...@trofast2.marin.ntnu.no completed Mon Dec 7 13:24:07 sabril 18...@trofast2.marin.ntnu.no LS-DYNA_971 NCPU=2 started Mon Dec 7 14:21:34 sabril 18...@trofast2.marin.ntnu.no LS-DYNA_971 NCPU=2 started Mon Dec 7 14:28:42 18...@trofast2.marin.ntnu.no killed Mon Dec 7 14:31:48 18...@trofast2.marin.ntnu.no killed Mon Dec 7 14:32:06 -- http://mail.python.org/mailman/listinfo/python-list
Re: strange subprocess behavior when calling ps
Thanks for the clarification on exceptions, Chris! Roger -- http://mail.python.org/mailman/listinfo/python-list
Re: Is Unladen Swallow dead?
On Nov 16, 2:30 pm, laspi wrote: >Is Unladen Swallow dead? No, it's just resting. -- http://mail.python.org/mailman/listinfo/python-list
Re: strange subprocess behavior when calling ps
On Wed, Nov 17, 2010 at 11:29 AM, Roger Davis wrote: >> Completely off topic but I think the try clause could be rewritten that way: >> ... >> Don't use bare except clause, you're masking syntax errors for instance, >> which will be flagged as 'unexpected error in generation ...". >> In a more general manner, if something unexpected happens it's better to >> just let the exception raise uncought. If you want to handle some >> errors, meaning you're kindof expecting them then add a explicit clause >> (like you did with KeyboardInterrupt). >> >> JM >> >> PS : "except Exception :" will catch most of the exceptions (all >> inheriting from that class). It's better than using a bare "except :" >> clause. (Exception won't catch SyntaxError) > > Thanks for the suggestion JM, it is off-topic and, although I will > first just say that the exception mechanism is *not* one of the > reasons I use Python (and stop there with regard to the whole > exception mechanism and various usage strategies in general), I do > have a few specific questions about a couple of your statements if you > don't mind following up. > > First, inserting a syntax error into my existing code does not hide a > SyntaxError exception as you have stated: > % ./pid.py > File "./pid.py", line 14 > if pslines not good Python talky-talk > ^ > SyntaxError: invalid syntax > > It appears that the interpreter is catching the syntax error before > the code is even executed. Now try: # junk.py @#$%^& gibberish @#$%^ # main.py import junk You'll get a run-time SyntaxError. > Finally, and this does not apply to your comments in particular, in > researching around about exception handling I often see the usage > > except Exception, e: > > suggested, but can't for the life of me figure out what the heck the > ', e' part does. Can anybody explain what this means and why it might > be desirable (or not)? That's how `except Exception as e` used to be spelled. The exception object will be bound to the variable `e`, allowing you to inspect it further. For example, one might use that form of `except` with an I/O-related exception to be able to check its `errno` attribute in order to handle the error properly. If you're just catching a generic Exception, said form of `except` is less useful. Cheers, Chris -- http://blog.rebertia.com -- http://mail.python.org/mailman/listinfo/python-list
Re: strange subprocess behavior when calling ps
On Nov 16, 11:19 pm, Ned Deily wrote: > Interesting. It appears that OS X 10.6 takes into account the ... Thanks very much for your thorough explanation, Ned! I think I've got what I need now. Roger -- http://mail.python.org/mailman/listinfo/python-list
Re: Help: Guide needed in trying to delete/flush the content of a fifo file.
Thanks Mrab ... the way of how to use the pipe for the producer as well as the consumer is a bit confusing to me. As i am using the subprocess for both the producer and the consumer as above. Can you please explain me further using a bit of pseudocode. Thanks for ur concern. In Nov 17, 12:13 am, MRAB wrote: > On 16/11/2010 06:52, Ton wrote: > > > > > On Nov 16, 1:47 am, MRAB wrote: > >> On 15/11/2010 11:03, Ton wrote: > > >>> On Nov 14, 11:55 pm, MRAB wrote: > On 14/11/2010 14:48, ton ph wrote:> Hi python geeks, > > I have problem which i have been trying to find out for the past > > some days, i have a device which feeds info to my fifo continuosly, and > > a thread of mine reads the > > fifo continuosly. Now when i change a parameter in the device, it sends > > me different values. Now my problem is that , i want to get rid of the > > contents of my previous info which > > is present in my buffer of the fifo.So i want to flush the fifo content > > when my device starts sending different info i am implementing > > writing and reading the fifo using two different threads. > > Please someone guide me solving my problem. > > I highly apologise everyone in case my post is not so clear... > > Thanks everyone in advance. > > When the info changes, the thread which is putting items into the fifo > could flush it just by getting items from it (using a non-blocking get) > until it's empty. > > >>> Hi Mrab, > >>> Thanks for your immediate reply , can you please guide me with any > >>> tool or library function which i can flush my fifo content .. i use > >>> os.mkfifo() to make the fifo. Or is there any other way i could do > >>> this ... > >>> Thanks > > >> Ah, you're using pipes; I thought you might've been using a queue. I > >> don't know of a way of flushing a pipe. > > >> I wonder whether it's a good idea for the producer to keep filling the > >> fifo, because the consumer know how 'old' the info is when it arrives, > >> and there's your problem of discarding stale info. Perhaps the producer > >> should wait until the consumer has sent an acknowledgement before > >> sending more info. > > > Thanks Mrab, > > the problem now is that the producer continously dumps the > > information and since i am implementing the producer using the > > subprocess as like this > > > cmd = ['my files'] > > proc = subprocess.Popen(cmd, shell=True, bufsize=0, > > stdout=subprocess.PIPE, stderr=subprocess.STDOUT ) > > > while flag == True: > > line = proc.stdout.read() > > > Now what i want is to synchronise the problem of actual info the > > device is sending and the proper reading as the device is sending and > > not about reading the previous old info in the fifo... hope i have > > made clearer my prob. even i have tried proc.stdout.flush(), to flush > > the previous old info in the fifo. Thnks > > Use two pipes, one each way. The consumer could use send a message to > the producer requesting an update, which the producer could send back > via the other pipe. Thanks marab for your immediate reply. But how actually i go for using pipes in the producer and consumer is a bit doubtful . can you please explain with a pseudocode ... as in my case both producer and consumer are executed using the subprocess as my previous post Thanks -- http://mail.python.org/mailman/listinfo/python-list
Re: strange subprocess behavior when calling ps
> Completely off topic but I think the try clause could be rewritten that way: > ... > Don't use bare except clause, you're masking syntax errors for instance, > which will be flagged as 'unexpected error in generation ...". > In a more general manner, if something unexpected happens it's better to > just let the exception raise uncought. If you want to handle some > errors, meaning you're kindof expecting them then add a explicit clause > (like you did with KeyboardInterrupt). > > JM > > PS : "except Exception :" will catch most of the exceptions (all > inheriting from that class). It's better than using a bare "except :" > clause. (Exception won't catch SyntaxError) Thanks for the suggestion JM, it is off-topic and, although I will first just say that the exception mechanism is *not* one of the reasons I use Python (and stop there with regard to the whole exception mechanism and various usage strategies in general), I do have a few specific questions about a couple of your statements if you don't mind following up. First, inserting a syntax error into my existing code does not hide a SyntaxError exception as you have stated: % cat pid.py #!/usr/bin/python import os import sys import subprocess def main(): psargs= ["/bin/ps", "-e"] try: ps= subprocess.Popen(psargs, stdout=subprocess.PIPE, close_fds=True) psout= ps.communicate()[0] pslines= psout.splitlines() if pslines not good Python talky-talk for line in pslines: print "%s" % line except KeyboardInterrupt: print "Keyboard interrupt received -- terminating." sys.stdout.flush() sys.exit(-1) except: print "%s: unexpected error in generation of system process list" % prognm sys.stdout.flush() sys.exit(-1) main() % ./pid.py File "./pid.py", line 14 if pslines not good Python talky-talk ^ SyntaxError: invalid syntax It appears that the interpreter is catching the syntax error before the code is even executed. Second, python.org's exception hierarchy documentation (Section 6.1 at http://docs.python.org/library/exceptions.html) shows all exception types except SystemExit, KeyboardInterrupt and GeneratorExit as being descendants of Exception. This includes SyntaxError, a child of StandardError which is itself a child of Exception. So, if I say 'except Exception:' then isn't that clause going to process any child exception type of Exception (including SyntaxError) that I haven't already covered in a separate except clause? (Except of course that my interpreter doesn't seem to treat a syntax error as any kind of exception at all!) Finally, and this does not apply to your comments in particular, in researching around about exception handling I often see the usage except Exception, e: suggested, but can't for the life of me figure out what the heck the ', e' part does. Can anybody explain what this means and why it might be desirable (or not)? Thanks! -- http://mail.python.org/mailman/listinfo/python-list
Re: Program, Application, and Software
On Wed, Nov 17, 2010 at 10:38 AM, Boštjan Mejak wrote: > What is the difference between a program, an application, and software? (1) This has nothing at all to do with (wx)Python specifically. (2) Did you try consulting Wikipedia?: http://en.wikipedia.org/wiki/Computer_program http://en.wikipedia.org/wiki/Application_software http://en.wikipedia.org/wiki/Computer_software Or did you find the articles somehow lacking or something? Regards, Chris -- http://mail.python.org/mailman/listinfo/python-list
Re: Pickle in a POST/GET request give EOFError
On Wed, 2010-11-17 at 15:44 +0100, Romaric DEFAUX wrote: > After entirely rewrite my code to not use Web service but socket (a real > client/server program) I finally found the problem... And it's not > linked to the POST or GET method... > It's because of that : > g.db.escape_string(fields['hosted_web_site']) > (escape_string is the function in MySQLdb library) > It escapes the simple quote of the pickled object, and break it... > It's good to know, NEVER escape a pickled object :) If you are worried about the binary-ness of the pickle string you can base64 encode it (using the base64 module). -- Adam Tauno Williams -- http://mail.python.org/mailman/listinfo/python-list
Program, Application, and Software
What is the difference between a program, an application, and software? -- http://mail.python.org/mailman/listinfo/python-list
Re: Raw Unicode docstring
On 17/11/2010 16:32, RJB wrote: On Nov 16, 1:56 pm, Boštjan Mejak wrote: Hello, how does one write a raw unicode docstring? If I have backslashes in the docstring, I must tuck an 'r' in front of it, like this: r"""This is a raw docstring.""" If I have foreign letters in the docstring, I must tuck a 'u' in front of it, like this: u"""This is a Unicode docstring.""" What if I have foreign characters *and* backslashes in my docstring? How to write the docstring then? ru"""My raw unicode docstring.""" or ur"""My unicode docstring.""" Please answer my question, although it may sound like a noobish one. Thanks. Check out http://cse.csusb.edu/dick/samples/python.syntax.html#stringprefix which lists alternate string prefixes. Does any bodyy know if "ur" and "UR" mean the same thing? I've tried them. It looks like the answer is yes. -- http://mail.python.org/mailman/listinfo/python-list
Re: Raw Unicode docstring
On 11/17/2010 11:32 AM, RJB wrote: > On Nov 16, 1:56 pm, Boštjan Mejak wrote: >> Hello, >> >> how does one write a raw unicode docstring? If I have backslashes in >> the docstring, I must tuck an 'r' in front of it, like this: >> r"""This is a raw docstring.""" >> >> If I have foreign letters in the docstring, I must tuck a 'u' in front >> of it, like this: >> u"""This is a Unicode docstring.""" >> >> What if I have foreign characters *and* backslashes in my docstring? >> How to write the docstring then? >> ru"""My raw unicode docstring.""" >> or >> ur"""My unicode docstring.""" >> >> Please answer my question, although it may sound like a noobish one. Thanks. > > Check out > http://cse.csusb.edu/dick/samples/python.syntax.html#stringprefix > which lists alternate string prefixes. > > Does any bodyy know if "ur" and "UR" mean the same thing? Yes, they do. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon 2011 Atlanta March 9-17 http://us.pycon.org/ See Python Video! http://python.mirocommunity.org/ Holden Web LLC http://www.holdenweb.com/ -- http://mail.python.org/mailman/listinfo/python-list
Sqlalchemy access to Firefox's places.sqlite
Howdy, Lazy, wanting to access Firefox's places.sqlite via Sqlalchemy. How about this: Sqlite Manager -> places.sqlite -> Export Wizard -> table name sqlite_manager this produces file sqlite_manager.sql which looks like: BEGIN TRANSACTION; INSERT INTO "sqlite_master" VALUES('table','moz_bookmarks','moz_bookmarks', 2,'CREATE TABLE moz_bookmarks (id INTEGER PRIMARY KEY,type INTEGER, fk INTEGER DEFAULT NULL, parent INTEGER, position INTEGER, title LONGVARCHAR, keyword_id INTEGER, folder_type TEXT, dateAdded INTEGER, lastModified INTEGER)'); INSERT INTO "sqlite_master" VALUES('index','moz_bookmarks_itemindex', 'moz_bookmarks',3,'CREATE INDEX moz_bookmarks_itemindex ON moz_bookmarks (fk, type)'); ... Is there an easy way to go from this sql to Sqlalchemy code? Thanks, Kent -- http://mail.python.org/mailman/listinfo/python-list
Re: Some syntactic sugar proposals
Christopher writes: > i don't like magic names. what about: > > t = foo() as v if pred(v) else default_value This is an improvement on `it'; anaphorics are useful in their place, but they don't seem to fit well with Python. But I don't think that's the big problem with this proposal. The real problem is that it completely changes the evaluation rule for the conditional expression. (The evaluation rule is already pretty screwy: Python is consistently left-to-right -- except here.) Evaluating a conditional expression starts in the middle, by evaluating the condition. If the condition is true, then it evaluates the consequent (to the left); otherwise it evaluates the alternative (to the right). Screwy, but tolerable. The proposal is to evaluate the /consequent/, stash it somewhere, evaluate the condition, and then either output the consequent which we evaluated earlier or the alternative which we must evaluate now. Of course, the implementation must be able to tell which of these evaluation rules to apply. The marker to look for is either `it' (original anaphoric proposal -- this is the real reason why `it' should be a reserved word: its presence radically alters the evaluation rule, so it ought to be a clear syntactic marker) or `as' (as suggested above). Elsewhere in the language, `as' is pretty consistent in what it does: it provides a name for a thing described elsewhere (a `with' context object, or an imported thing) -- but nothing else. It certainly doesn't suggest a change in the way anything else is evaluated. 1/x if x != 0 else None works for any numeric x; it'd be really surprising to me if 1/x as hunoz if x != 0 else None didn't. -1 on this one. -- [mdw] -- http://mail.python.org/mailman/listinfo/python-list
Re: Raw Unicode docstring
On Nov 16, 1:56 pm, Boštjan Mejak wrote: > Hello, > > how does one write a raw unicode docstring? If I have backslashes in > the docstring, I must tuck an 'r' in front of it, like this: > r"""This is a raw docstring.""" > > If I have foreign letters in the docstring, I must tuck a 'u' in front > of it, like this: > u"""This is a Unicode docstring.""" > > What if I have foreign characters *and* backslashes in my docstring? > How to write the docstring then? > ru"""My raw unicode docstring.""" > or > ur"""My unicode docstring.""" > > Please answer my question, although it may sound like a noobish one. Thanks. Check out http://cse.csusb.edu/dick/samples/python.syntax.html#stringprefix which lists alternate string prefixes. Does any bodyy know if "ur" and "UR" mean the same thing? -- http://mail.python.org/mailman/listinfo/python-list
pySerial Vs 2 Serial Ports
Dear all, I Have two modules communicating by Bluetooth. Basically I want to rotate two vector using the accelerometer data. I read the data using pyserial like this: ### ser1 = serial.Serial(port='COM6',baudrate=19200) if ser1.isOpen(): print "Comm 1 is open" ser2 = serial.Serial(port='COM43',baudrate=19200) if ser2.isOpen(): print "Comm 2 is open" ### And then retirieve the raw data in a while loop like this: while True: line1 = ser2.readline() line2 = ser1.readline() ... My problem is the following... when I read from only one module, all goes smooth, but when i try to read from the two modules, the second one that i read retrieves the data with a huge delay. In terms of debug, I tried to run two independent scripts, each one opening and retrieving data from one COM port and there isn't any delay, only when i read the two COM's in the same script i get the delay. Any Clue? Thanks in Advance. Best Regards. -- http://mail.python.org/mailman/listinfo/python-list
Re: how to use socket to get packet which destination ip is not local?
Hans writes: > I tried socket bind to 0.0.0.0, but it only binds to any local ip, not > any ip which may not be local. therefore the socket cannot get that > dhcp offer packet even I can use wireshark to see that packet did come > to this pc. You must use a raw socket for this. Raw sockets are fiddly, not very portable, and require privilege. Sorry. -- [mdw] -- http://mail.python.org/mailman/listinfo/python-list
Re: How can I catch segmentation fault in python?
Just checked what is valgrind, sounds promising. Let me try that. Thanks a lot, Justin. On Nov 17, 9:47 am, Wolfgang Rohdewald wrote: > On Mittwoch 17 November 2010, Wolfgang Rohdewald wrote: > > > On Mittwoch 17 November 2010, justin wrote: > > > But the problem is that the code is not mine, and it takes > > > over a day for me to get the point where the segmentation > > > fault occurred. Plus, it seems that the point is not > > > deterministic > > > > Still, I think I should at least try to figure out exactly > > > at which point the segmentation fault occurs, and think > > > where to go from there according to your kind advice. > > > try valgrind > > hit the send button too fast... > > even if the segmentation fault only happens after a long time > valgrind might find problems much sooner, and fixing them > might remove the segmentation fault. > > -- > Wolfgang -- http://mail.python.org/mailman/listinfo/python-list
Re: How can I catch segmentation fault in python?
On Mittwoch 17 November 2010, Wolfgang Rohdewald wrote: > On Mittwoch 17 November 2010, justin wrote: > > But the problem is that the code is not mine, and it takes > > over a day for me to get the point where the segmentation > > fault occurred. Plus, it seems that the point is not > > deterministic > > > > Still, I think I should at least try to figure out exactly > > at which point the segmentation fault occurs, and think > > where to go from there according to your kind advice. > > try valgrind hit the send button too fast... even if the segmentation fault only happens after a long time valgrind might find problems much sooner, and fixing them might remove the segmentation fault. -- Wolfgang -- http://mail.python.org/mailman/listinfo/python-list
Oracle jdbc sql select for update in python
I want to do a select from...for update in python, update the selected row and then commit; However cannot seem to get it to work...the update statement seems to be waiting because the row is locked. Presumably oracle thinks the update is another transaction. How can I get this to work? What statements do I need to ensure Oracle treats this as one transaction? -- http://mail.python.org/mailman/listinfo/python-list
Re: How can I catch segmentation fault in python?
On Mittwoch 17 November 2010, justin wrote: > But the problem is that the code is not mine, and it takes > over a day for me to get the point where the segmentation > fault occurred. Plus, it seems that the point is not > deterministic > > Still, I think I should at least try to figure out exactly at > which point the segmentation fault occurs, and think where to > go from there according to your kind advice. try valgrind -- mit freundlichen Grüssen with my best greetings Wolfgang Rohdewald dipl. Informatik Ing. ETH Rohdewald Systemberatung Karauschenstieg 4 D 21640 Horneburg Tel.: 04163 826 819 Fax: 04163 826 828 Internet: http://www.rohdewald.de -- http://mail.python.org/mailman/listinfo/python-list
Re: Some syntactic sugar proposals
On Wed, 17 Nov 2010 10:18:51 -0500 Mel wrote: > Christopher wrote: > > >> ? Of course we can write it as > >> t = foo() if pred(foo()) else default_value > >> but here we have 2 foo() calls instead of one. Why can't we write > >> just something like this: > >> t = foo() if pred(it) else default_value > >> where "it" means "foo() value"? > > > > i don't like magic names. what about: > > > > t = foo() as v if pred(v) else default_value > > !! so: assignment inside an expression. > I like the idea of having an "as ... if" construct, though. :) /W -- To reach me via email, replace INVALID with the country code of my home country. But if you spam me, I'll be one sour Kraut. -- http://mail.python.org/mailman/listinfo/python-list
Re: Some syntactic sugar proposals
Christopher wrote: >> ? Of course we can write it as >> t = foo() if pred(foo()) else default_value >> but here we have 2 foo() calls instead of one. Why can't we write just >> something like this: >> t = foo() if pred(it) else default_value >> where "it" means "foo() value"? > > i don't like magic names. what about: > > t = foo() as v if pred(v) else default_value !! so: assignment inside an expression. Mel. -- http://mail.python.org/mailman/listinfo/python-list
Re: Pickle in a POST/GET request give EOFError
Le 16/11/2010 17:47, Romaric DEFAUX a écrit : Hi everybody ! First time I write to this mailing list :) I started writing in python last week, that's probably why I can't understand the following problem... I create a list called web_site_list. This list contain dictionaries called web_site. And some values in this dictionaries are list too. I do that in a function and I return this : return pickle.dumps(web_site_list) This is working fine :) If I do : print "%s" % pickle.loads(system.get_web_site_list()) I've got the right stuffs. For example it returns : [{'documentroot_size': '120', 'servername': '---default---', 'client': 'undefined', 'documentroot': '/var/www/', 'client_contact': 'undefined', 'serveralias': []}] I send this to a web service. I send it like that : #I put it in params def system_updateweb_site(server, login, password): params = {} params['login'] = login params['password'] = password params['action'] = 'updateweb_site' params['servername'] = get_servername() params['hosted_web_site'] = get_web_site_list() return call_system_ws(server, params) #Here's how I send it (I tried in GET and POST) def call_system_ws(host, params): query_string = urllib.urlencode(params) #GET # f = urllib.urlopen("http://%s/ws?%s"; % (host, query_string)) #POST f = urllib.urlopen("http://%s/ws"; % (host), query_string) result = f.readline().strip() if result == 'ERROR': msg = f.readline().strip() return (False, msg) return (True, result) On the server side : if action == 'updateweb_site': if not (fields.has_key('servername') and fields.has_key('hosted_web_site')): raise WSError('missing parameter : servername or hosted_web_site') log ('ERROR : missing parameter : servername or hosted_web_site') else: servername=g.db.escape_string(fields['servername']) hosted_web_site=g.db.escape_string(fields['hosted_web_site']) output = systemserver.updateweb_site(cursor, servername, hosted_web_site) In systemserver.py : def updateweb_site(cursor, host, hosted_web_site): web_site_list = pickle.loads(hosted_web_site) return "%s" % (web_site_list) I catch this error :* *: args = () message = '' Why ? If I just print hosted_web_site, I get this on my web page : (lp0\n(dp1\nS\'documentroot_size\'\np2\nS\'120\'\np3\nsS\'servername\'\np4\nS\'default\'\np5\nsS\'client\'\np6\nS\'undefined\'\np7\nsS\'documentroot\'\np8\nS\'/var/www/\'\np9\nsS\'client_contact\'\np10\ng7\nsS\'serveralias\'\np11\n(lp12\nsa. It's the "pickled view" of [{'documentroot_size': '120', 'servername': '---default---', 'client': 'undefined', 'documentroot': '/var/www/', 'client_contact': 'undefined', 'serveralias': []}] Can someone help me please ? I spend my afternoon to google to try to find a solution... Thanks in advance !!! Romaric Defaux After entirely rewrite my code to not use Web service but socket (a real client/server program) I finally found the problem... And it's not linked to the POST or GET method... It's because of that : g.db.escape_string(fields['hosted_web_site']) (escape_string is the function in MySQLdb library) It escapes the simple quote of the pickled object, and break it... It's good to know, NEVER escape a pickled object :) Romaric Defaux smime.p7s Description: S/MIME Cryptographic Signature -- http://mail.python.org/mailman/listinfo/python-list
Re: How can I catch segmentation fault in python?
On Nov 17, 1:06 am, John Nagle wrote: > On 11/16/2010 10:15 PM, swapnil wrote: > > > > > > > On Nov 17, 10:26 am, justin wrote: > >> Hi all, > > >> I am calling a program written in C inside Python using ctypes, > >> and it seems that sometimes the program in C crashes while it's being > >> used in Python. > >> Even under the circumstances, I want to get the Python program going > >> by handling the segmentation fault. > > >> I've already searched the Internet, but couldn't get the right answer > >> to catch them. > >> Could any of you please let me know how to deal with this and catch > >> the segmentation fault in Python? > > >> Thanks, > >> Justin. > > > Segmentation fault isn't exactly an exception that you can catch. It > > usually means something has gone horribly wrong, like dereferencing > > invalid pointer, trying to access memory out of process's range. Since > > if you run out of memory Python simply raises MemoryError exception, > > which you can catch. So that is not the case for segmentation fault. > > Either fix the program so it doesn't crash,or run the offending > module and the C code in a subprocess. > > John Nagle Thanks guys for this fast replies, Turns out that it is a fact that segmentation fault is not what I can deal with in programming level. But the problem is that the code is not mine, and it takes over a day for me to get the point where the segmentation fault occurred. Plus, it seems that the point is not deterministic Still, I think I should at least try to figure out exactly at which point the segmentation fault occurs, and think where to go from there according to your kind advice. Again many thanks, Justin. -- http://mail.python.org/mailman/listinfo/python-list
Re: Some syntactic sugar proposals
> ? Of course we can write it as > t = foo() if pred(foo()) else default_value > but here we have 2 foo() calls instead of one. Why can't we write just > something like this: > t = foo() if pred(it) else default_value > where "it" means "foo() value"? i don't like magic names. what about: t = foo() as v if pred(v) else default_value -- http://mail.python.org/mailman/listinfo/python-list
Re: Cannot Remove File: Device or resource busy
Am 17.11.2010 09:16, schrieb Brett Bowman: > Good ideas, but I've tried them already: > -No del command, or replacing it with a set-to-null, neither solve my file > access problem. > -PdfFileReader has no close() function, and causes an error. Weird, but > true. > -pdf_handle.close() on the other hand, fails to solve the problem. You have to close the file explicitly, for example with a proper with block. with open(outputFile, "rb") as fh: pdf_pypdf = PdfFileReader(fh) # process the PDF file here Christian -- http://mail.python.org/mailman/listinfo/python-list
Re: urllib2 error
On Wed, Nov 17, 2010 at 5:31 PM, Kushal Kumaran wrote: > On Wed, Nov 17, 2010 at 5:18 PM, asit wrote: >> I have this piece of code >> >> import urllib2 >> >> proc_url = 'http://www.nse-india.com/content/historical/EQUITIES/2001/ >> JAN/cm01JAN2001bhav.csv.zip' >> print 'processing', proc_url >> req = urllib2.Request(proc_url) >> res = urllib2.urlopen(req) >> >> when i run this...following error comes >> >> Traceback (most recent call last): >> >> HTTPError: HTTP Error 403: Forbidden >> > > The web server is probably trying to forbid downloads using automated > tools by examining the user agent header (downloading using wget also > fails, for example). You can work around that by setting the > User-Agent header to the same value as a browser that works. > > The urllib2 documentation covers how to set headers in requests. > > -- > regards, > kushal > -- Regards, Asit Kumar Dhal TCS, Mumbai blog: http://asitdhal.blogspot.com/ Thanx...it worked -- http://mail.python.org/mailman/listinfo/python-list
Re: urllib2 error
On Wed, Nov 17, 2010 at 5:18 PM, asit wrote: > I have this piece of code > > import urllib2 > > proc_url = 'http://www.nse-india.com/content/historical/EQUITIES/2001/ > JAN/cm01JAN2001bhav.csv.zip' > print 'processing', proc_url > req = urllib2.Request(proc_url) > res = urllib2.urlopen(req) > > when i run this...following error comes > > Traceback (most recent call last): > > HTTPError: HTTP Error 403: Forbidden > The web server is probably trying to forbid downloads using automated tools by examining the user agent header (downloading using wget also fails, for example). You can work around that by setting the User-Agent header to the same value as a browser that works. The urllib2 documentation covers how to set headers in requests. -- regards, kushal -- http://mail.python.org/mailman/listinfo/python-list
urllib2 error
I have this piece of code import urllib2 proc_url = 'http://www.nse-india.com/content/historical/EQUITIES/2001/ JAN/cm01JAN2001bhav.csv.zip' print 'processing', proc_url req = urllib2.Request(proc_url) res = urllib2.urlopen(req) when i run this...following error comes Traceback (most recent call last): File "C:/Python26/reqnse.py", line 92, in res = urllib2.urlopen(req) File "C:\Python26\lib\urllib2.py", line 124, in urlopen return _opener.open(url, data, timeout) File "C:\Python26\lib\urllib2.py", line 395, in open response = meth(req, response) File "C:\Python26\lib\urllib2.py", line 508, in http_response 'http', request, response, code, msg, hdrs) File "C:\Python26\lib\urllib2.py", line 433, in error return self._call_chain(*args) File "C:\Python26\lib\urllib2.py", line 367, in _call_chain result = func(*args) File "C:\Python26\lib\urllib2.py", line 516, in http_error_default raise HTTPError(req.get_full_url(), code, msg, hdrs, fp) HTTPError: HTTP Error 403: Forbidden would anyone help me why ?? -- http://mail.python.org/mailman/listinfo/python-list
Re: strange subprocess behavior when calling ps
Roger Davis wrote: Hi all, [snip] Roger Davis # code follows #!/usr/bin/python import sys import subprocess def main(): psargs= ["/bin/ps", "-e"] try: ps= subprocess.Popen(psargs, stdout=subprocess.PIPE, close_fds=True) psout= ps.communicate()[0] pslines= psout.splitlines() for line in pslines: print "%s" % line except KeyboardInterrupt: print "Keyboard interrupt received -- terminating." sys.stdout.flush() sys.exit(-1) except: print "%s: unexpected error in generation of system process list" % prognm sys.stdout.flush() sys.exit(-1) main() Completely off topic but I think the try clause could be rewritten that way: try: ps= subprocess.Popen(psargs, stdout=subprocess.PIPE, close_fds=True) psout= ps.communicate()[0] pslines= psout.splitlines() for line in pslines: print "%s" % line except KeyboardInterrupt: print "Keyboard interrupt received -- terminating." finally: sys.stdout.flush() Don't use bare except clause, you're masking syntax errors for instance, which will be flagged as 'unexpected error in generation ...". In a more general manner, if something unexpected happens it's better to just let the exception raise uncought. If you want to handle some errors, meaning you're kindof expecting them then add a explicit clause (like you did with KeyboardInterrupt). JM PS : "except Exception :" will catch most of the exceptions (all inheriting from that class). It's better than using a bare "except :" clause. (Exception won't catch SyntaxError) -- http://mail.python.org/mailman/listinfo/python-list
In-process interpreters
Hi, Please refer my post on Python-ideas list http://mail.python.org/pipermail/python-ideas/2010-November/008666.html and provide feedback if any. -- http://mail.python.org/mailman/listinfo/python-list
Re: strange subprocess behavior when calling ps
In article <61496525-afab-4d19-a7e9-e61fb46e0...@n30g2000vbb.googlegroups.com>, Roger Davis wrote: > First, I *still* don't quite understand why this happens with my 2.6.6 > interpreter but not my 2.6.1, and why another of the respondents to > this thread (Chris) could not duplicate this problem with his own > 2.6.6 environment. Is there something defective with my Python > environment having to do with the 2.6.6 upgrade I installed directly > from python.org? (Chris used Fink, apparently.) If so, how can I clean > it up? I do have an easier time believing that this is a mangled > installation issue now that the problem has essentially been reduced > to the handling of an environment variable which I can well imagine > could be easily garbled somewhere in a version mismatch scenario. Interesting. It appears that OS X 10.6 takes into account the deployment target of the executable. Prior to 2.7, all recent python.org installed Pythons have been built with MACOSX_DEPLOYMENT_TARGET = 10.3, allowing the same build to work on all releases from 10.3.9 on. So, when you launch a script with the python.org interpreter (with a 10.3 deployment target), OS X automatically sets COMMAND_MODE to 'legacy'. The Apple-supplied Python 2.6.1 in OS X 10.6 is built with a deployment target of 10.6, so it gets a COMMAND_MODE of 'unix2003'. And, as far as I can tell, OS X 10.5 honors COMMAND_MODE but does not automatically set it or tailor it like 10.6 does. Live and learn! > Second, I do need to make this work on multiple platforms, primarily > Linux in addition to the Mac but also with a potential variety of > Python releases on both MacOS and Linux. I'm not sure if Linux will > object to this COMMAND_MODE business, I guess I'll just have to try > and see what happens. Any thoughts on that? As far as I know, COMMAND_MODE has no special meaning on other platforms so it should be ignored anywhere but on OS X. If you want to be extra cautious, you could test for 'sys.platform == 'darwin'. > Finally, to split hairs a bit concerning your suggested solution, I > have a question about the additional env={'COMMAND_MODE': 'unix2003'} > argument. This works for me, but my reading of the subprocess > documentation leads me to believe that this will actually wipe out the > entire environment that would otherwise be inherited from the parent > and replace it with that single setting. I don't think this has any > ill effects here with regard to ps, at least none I can detect at the > moment, but wouldn't a perhaps safer solution be to do > > os.environ['COMMAND_MODE']= 'unix2003' > > prior to the Popen() to instead augment/correct the existing > environment which will then later be inherited by the child, assuming > no explicit env= optional argument is used? This also works for me, > and I think I'm more inclined to go this route unless you can think of > a good reason not to do so. That analysis seems correct. So it comes down to whether there are any environment variable settings the user could make that would positively or negatively affect the operation of your script. -- Ned Deily, n...@acm.org -- http://mail.python.org/mailman/listinfo/python-list
Re: import site fails - Please Help
On Nov 15, 1:46 pm, Helmut Jarausch wrote: > Hi, I'm completely puzzled and I hope someone > can shed some light on it. > > After cloning a running system, booting the new machine from a rescue CD, > chroot to the new root partition, I get the following strange error > from python upon startup > > python -v> import site failed > > > st= os.stat(path) > stat() argument 1 must be encoded string without NULL bytes, not str > > So, what's broken? How can I get out what 'path' contained at this > moment? > > Many thanks for your help, > Helmut. Perhaps its not able to locate the libraries. Try using http://docs.python.org/using/cmdline.html#envvar-PYTHONPATH and http://docs.python.org/using/cmdline.html#envvar-PYTHONHOME -- http://mail.python.org/mailman/listinfo/python-list
Re: MATLAB to Python?
MATLABdude writes: > Hi! > > Can you, please, try to help me with Python? I try to convert a MATLAB > program to Python. > > Here are the MATLAB codes: > http://pastebin.com/eJetYizv > http://pastebin.com/0eXTVfyN > > Here is my Python code: > http://pastebin.com/jCPdLHx7 > > What is wrong with my Python code? The program doesn't produce quite > the same lambdas as the MATLAB program does. My code probably has lots > of errors because I'm just learning the basics about Python. > > Thanks for all the help! I don't understand what's going on in your code, but line 81 you have: xx = range(-kappa, h, kappa+1) I guess that the step is supposed to be h, so you should write: xx = range(-kappa, kappa+1, h) -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: MATLAB to Python?
On Nov 17, 12:04 am, MATLABdude wrote: > Hi! > > Can you, please, try to help me with Python? I try to convert a MATLAB > program to Python. > > Here are the MATLAB > codes:http://pastebin.com/eJetYizvhttp://pastebin.com/0eXTVfyN > > Here is my Python code:http://pastebin.com/jCPdLHx7 > > What is wrong with my Python code? The program doesn't produce quite > the same lambdas as the MATLAB program does. My code probably has lots > of errors because I'm just learning the basics about Python. > > Thanks for all the help! I was about to say it was a pretty good looking piece of code, and probably if anything is wrong it's a typo somewhere. But then my eye caught something at the last minute: xx = range(-kappa, h, kappa+1) Looks like you're treating range arguments as (start,step,stop), same order as in Matlab. In Python it's (start,stop,step), so you should be using this: xx = range(-kappa,kappa+1,h) Other than that, you might jus have to pepper your scripts with prints and disps to catch the exact point where the discrepancy occurs. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: MATLAB to Python?
MATLABdude wrote: > Can you, please, try to help me with Python? I try to convert a MATLAB > program to Python. > > Here are the MATLAB codes: > http://pastebin.com/eJetYizv > http://pastebin.com/0eXTVfyN > > Here is my Python code: > http://pastebin.com/jCPdLHx7 That is too much code for me, in a problem domain I'm not familiar with. > What is wrong with my Python code? The program doesn't produce quite > the same lambdas as the MATLAB program does. My code probably has lots > of errors because I'm just learning the basics about Python. If no-one fixes it for you I suggest that you break the code in smaller functions (at most ten lines) that you can test separately. That should make it easier to spot the misbehaving parts of your calculation. If you come back then with one of these, the matlab equivalent, sample input and expected output a significantly larger number of people can and may be willing to help. Peter -- http://mail.python.org/mailman/listinfo/python-list
Re: Cannot Remove File: Device or resource busy
Good ideas, but I've tried them already: -No del command, or replacing it with a set-to-null, neither solve my file access problem. -PdfFileReader has no close() function, and causes an error. Weird, but true. -pdf_handle.close() on the other hand, fails to solve the problem. On Tue, Nov 16, 2010 at 11:25 PM, Dennis Lee Bieber wrote: > On Tue, 16 Nov 2010 17:37:10 -0800, Brett Bowman > declaimed the following in gmane.comp.python.general: > > > > > And then I test the result: > > try: > > pdf_handle = open(outputFile, "rb") > > pdf_pypdf = PdfFileReader(pdf_handle) > > del pdf_pypdf > > del pdf_handle > > except Exception, e: > > return "Unable to open file: %s with error: %s" % (outputFile, > > str(e)) > > > You seem enamored of "del", which is something I've only used for > special purposes, and even then rarely -- binding a null object to the > name is just as effective for most uses. > >While the common Python does garbage collect objects when the > reference count goes to zero, there is no real guarantee of this. > >I'd replace that >del pdf_handle > whit >pdf_handle.close() > > -- >Wulfraed Dennis Lee Bieber AF6VN >wlfr...@ix.netcom.comHTTP://wlfraed.home.netcom.com/ > > -- > http://mail.python.org/mailman/listinfo/python-list > -- http://mail.python.org/mailman/listinfo/python-list
MATLAB to Python?
Hi! Can you, please, try to help me with Python? I try to convert a MATLAB program to Python. Here are the MATLAB codes: http://pastebin.com/eJetYizv http://pastebin.com/0eXTVfyN Here is my Python code: http://pastebin.com/jCPdLHx7 What is wrong with my Python code? The program doesn't produce quite the same lambdas as the MATLAB program does. My code probably has lots of errors because I'm just learning the basics about Python. Thanks for all the help! -- http://mail.python.org/mailman/listinfo/python-list