Re: [Tutor] Google App Engine
On Tue, Apr 08, 2008 at 10:35:37AM -0700, Dinesh B Vadhia wrote: Hi! Google announced an app server that allows pure Python developed applications/services to use their infrastructure. This maybe of use to many on this list. Further details can be found at: http://appengine.google.com/ The SDK include a modified Python 2.5.2 and Django 0.96.1, WebOb 0.9 and PyYAML 3.05. As an aside, does anyone here have experience of WebOb and specifically is it a mini web framework (like webpy)? Cheers Dinesh Hi, how safe is it to just run the dev server, as I didn't get one of the prerelease accounts either. I'm running one in a XEN sandbox on Debian Etch using the Nginxi HTTP server to proxy domain:80 to localhost:8080 Regards HC ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Google App Engine
H.C. v. Stockhausen wrote: On Tue, Apr 08, 2008 at 10:35:37AM -0700, Dinesh B Vadhia wrote: Hi! Google announced an app server how safe is it to just run the dev server, as I didn't get one of the prerelease accounts either. If by 'safe' you mean 'secure', I don't really know but I guess it is probably pretty safe. The dev server is based on BaseHTTPServer and other elements of the Python std lib. If by 'safe' you mean 'robust', then no. I wouldn't use the dev server for a production server: - The dev server is single-threaded - it only serves one request at a time. - The datastore is written in Python so it will probably not match the performance of PostgreSQL or the native Google Apps datastore. - The default datastore stores all objects in memory so it will not scale well. - User login is stubbed out There is a google group where you might get a more detailed answer: http://groups.google.com/group/google-appengine Kent ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Tutor Digest, Vol 50, Issue 9
Hello :-) Can someone please explain to me ho can I find out how many elements are there in one record of a list? The problem is as follows: I have a txt file from which I read data into Python. The file looks something like this: 01 bla bla bla 23,15 2345,67 02 alb alb 2,4 890,1 03 bal bla alb lab 567,12345 87,45 I need to be able to discriminate the string parts from the numeric ones. Since the number of words in the file can vary, I have to be able to find out when they are finished and when the floats come in mystring[0]- always integer mystring[1]- string (word) mystring[1-X]- last string (word) mystring[X+1]- always float mystring[X+2]- always float it would have been nice if I could find out the total number of the fields in one list record so that I could then adress them via a variable. ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] socket / over network
in case it helps here is a very basic example:
import MySQLdb, glob, os, re, shutil
from ftplib import FTP
a=file_to_fetch
ftp=FTP('ftp_server')
ftp.login('user_name','password')
try:
aa=ftp.nlst(a)
b='/home/a'
bb=os.path.basename(aa[0])
e=os.path.basename(b)
c=open(b, 'wb')
ftp.retrbinary('RETR '+aa[0], c.write)
c.close()
well u just copied some pieces of my own code to maybe help you get started
with ftp as you maybe don't know that you have to open a file for writing
then write into it the stream from ftp retrieve cmd.
On Mon, Apr 7, 2008 at 7:16 AM, Alan Gauld [EMAIL PROTECTED]
wrote:
Nathan McBride [EMAIL PROTECTED] wrote
Going off of wha tyou said, if I choose to use ftp, is there a way i
could do everything from within python including the server to get
the
files? Is there like a ftp module for python to help in the passing
of
the files between the computers?
Yes, there is an ftp module in the standard library.
Alan G
___
Tutor maillist - Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor
___
Tutor maillist - Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Tutor Digest, Vol 50, Issue 9
Hi Gloom, You should give a look at the method split (of the string objects) and int. The first is used do break a string into smaller pieces and the other to convert a string to an int object, raising an exception when it is not possible. On Wed, Apr 9, 2008 at 9:29 AM, Gloom Demon [EMAIL PROTECTED] wrote: Hello :-) Can someone please explain to me ho can I find out how many elements are there in one record of a list? The problem is as follows: I have a txt file from which I read data into Python. The file looks something like this: 01 bla bla bla 23,15 2345,67 02 alb alb 2,4 890,1 03 bal bla alb lab 567,12345 87,45 I need to be able to discriminate the string parts from the numeric ones. Since the number of words in the file can vary, I have to be able to find out when they are finished and when the floats come in mystring[0]- always integer mystring[1]- string (word) mystring[1-X]- last string (word) mystring[X+1]- always float mystring[X+2]- always float it would have been nice if I could find out the total number of the fields in one list record so that I could then adress them via a variable. ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor -- Ruivaldo Neto ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Tutor Digest, Vol 50, Issue 9
Gloom Demon wrote: Hello :-) Can someone please explain to me ho can I find out how many elements are there in one record of a list? The len() function gives the length of a list. I have a txt file from which I read data into Python. The file looks something like this: 01 bla bla bla 23,15 2345,67 02 alb alb 2,4 890,1 03 bal bla alb lab 567,12345 87,45 I need to be able to discriminate the string parts from the numeric ones. Since the number of words in the file can vary, I have to be able to find out when they are finished and when the floats come in You can also use slice indexing with negative numbers to index from the end: In [50]: data = '''01 bla bla bla 23,15 2345,67 : 02 alb alb 2,4 890,1 : 03 bal bla alb lab 567,12345 87,45 : '''.splitlines() In [51]: for line in data: : line = line.split() # Break the line at whitespace : print len(line) # Number of elements in the line : print line[1:-2] : print line[-2:] : print : : 6 ['bla', 'bla', 'bla'] ['23,15', '2345,67'] 5 ['alb', 'alb'] ['2,4', '890,1'] 7 ['bal', 'bla', 'alb', 'lab'] ['567,12345', '87,45'] Negative indices index from the end of the list, so line[1:-2] gives you the elements from line[1] up to but not including line[-2] which is the next-to-last element. Kent ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Tutor Digest, Vol 50, Issue 9
On Wed, Apr 9, 2008 at 12:59 PM, rui [EMAIL PROTECTED] wrote:
Hi Gloom,
You should give a look at the method split (of the string objects) and
int.
The first is used do break a string into smaller pieces and the other to
convert a string to an int object, raising an exception when it is not
possible.
On Wed, Apr 9, 2008 at 9:29 AM, Gloom Demon [EMAIL PROTECTED] wrote:
Hello :-)
Can someone please explain to me ho can I find out how many elements are
there in one record of a list?
The problem is as follows:
I have a txt file from which I read data into Python.
The file looks something like this:
01 bla bla bla 23,15 2345,67
02 alb alb 2,4 890,1
03 bal bla alb lab 567,12345 87,45
I need to be able to discriminate the string
parts from the numeric ones.
Since the number of words in the file can vary, I have to be able to find
out when they are finished
and when the floats come in
mystring[0]- always integer
mystring[1]- string (word)
mystring[1-X]- last string (word)
mystring[X+1]- always float
mystring[X+2]- always float
it would have been nice if I could find out the total number of the
fields in one list record so that I could then adress them via a variable.
___
Tutor maillist - Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor
--
Ruivaldo Neto
___
Tutor maillist - Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor
my guess would be something like this :
a=open('/home/some_file.txt')
for line in a:
tmp_list_1=line.split()
num_floats=0
num_strings=0
for i in tmp_list_1:
if type(i) == int:
num_floats=num_floats+1
else:
num_strings=num_strings+1
if you explain more you case maybe i can get more ideas - hope this helps
___
Tutor maillist - Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Tutor Digest, Vol 50, Issue 9
import re
items = []
for line in open('data.txt'):
items.append(re.sub('\n', '', line).split(' '))
- Original Message -
From: Gloom Demon [EMAIL PROTECTED]
To: tutor@python.org
Date: Wed, 9 Apr 2008 15:29:35 +0300
Subject: Re: [Tutor] Tutor Digest, Vol 50, Issue 9
Hello :-)
Can someone please explain to me ho can I find out how many elements are
there in one record of a list?
The problem is as follows:
I have a txt file from which I read data into Python.
The file looks something like this:
01 bla bla bla 23,15 2345,67
02 alb alb 2,4 890,1
03 bal bla alb lab 567,12345 87,45
I need to be able to discriminate the string parts from the numeric ones.
Since the number of words in the file can vary, I have to be able to find out
when they are finished
and when the floats come in
mystring[0]- always integer
mystring[1]- string (word)
mystring[1-X]- last string (word)
mystring[X+1]- always float
mystring[X+2]- always float
it would have been nice if I could find out the total number of the fields in
one list record so that I could then adress them via a variable.
___
Tutor maillist - Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Tutor Digest, Vol 50, Issue 9
Kepala Pening wrote:
import re
items = []
for line in open('data.txt'):
items.append(re.sub('\n', '', line).split(' '))
Hmm. So much to say about so little code!
- the re.sub() is not needed - the split() will remove the trailing newline:
In [53]: 'a b\n'.split()
Out[53]: ['a', 'b']
- you don't need re to replace a fixed character, you can use str.replace():
In [55]: 'a b\n'.replace('\n', '')
Out[55]: 'a b'
- If you just want to strip the trailing newline you can use strip() or
rstrip(), with or without args, depending on how strict you want to be:
In [56]: 'a b\n'.strip()
Out[56]: 'a b'
- It's not clear that the OP wants a list of lines, but if so, a list
comprehension is much more succinct:
items = [ line.split() for line in open('data.txt') ]
would do the job just fine.
Kent
___
Tutor maillist - Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Google App Engine
On Wed, Apr 09, 2008 at 08:04:03AM -0400, Kent Johnson wrote: H.C. v. Stockhausen wrote: On Tue, Apr 08, 2008 at 10:35:37AM -0700, Dinesh B Vadhia wrote: Hi! Google announced an app server how safe is it to just run the dev server, as I didn't get one of the prerelease accounts either. If by 'safe' you mean 'secure', I don't really know but I guess it is probably pretty safe. The dev server is based on BaseHTTPServer and other elements of the Python std lib. If by 'safe' you mean 'robust', then no. I wouldn't use the dev server for a production server: - The dev server is single-threaded - it only serves one request at a time. - The datastore is written in Python so it will probably not match the performance of PostgreSQL or the native Google Apps datastore. - The default datastore stores all objects in memory so it will not scale well. - User login is stubbed out Thanks for your reply, Kent. I woulnd't expect the dev server to scale too well either. Security is what I was mainly concerned with. Just in case, I have resricted access to the development console at /_ah/admin through HTTP basic authentication. Thanks, HC ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
[Tutor] Process that starts processes
I have been searching for a while but I can't seem to find anything that will do this, so... In my python program I am starting a process using subprocess.Popen. This is working fine, but the process I am starting starts several other processes. Is there any way (using subprocess or a different module) to control the processes the original creates (by control I mean feed them input, capture output, and kill them). I hope that is enough information (its pretty much all I have). Thanks. ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
[Tutor] Doubts about Pylint
I'd never used Pylint until yesterday, when I discovered that Ulipad had a Pylint plugin that enabled me to run Pylint on scripts within Ulipad. But I'm wondering about some of the results. I noticed that it was complaining that my variable names violated convention. Here's an image of running Pylint on a script with only 7 lines: http://www.rcblue.com/Misc/PylintInUlipad.jpg Since when is 'az' a bad variable name? And 'AZ' is OK? And I tried Pylint on an official Python module file, calendar.py. Look what I got when all the checkboxes are checked!!: http://www.rcblue.com/Ulipad/PyLintOnCalendar.py.jpg Comments? Dick Moores UliPad The Python Editor: http://code.google.com/p/ulipad/ ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Doubts about Pylint
On Wed, Apr 9, 2008 at 11:43 AM, Dick Moores [EMAIL PROTECTED] wrote: Comments? Since we started using code profilers and checkers like pyLint etc., we've had a motto: This is a guide. It is not the gospel. Take from pylint what you think helps and ignore the rest. It's just a tool and you can choose how to use it. That is, unless you want to actually change pylint. I'm sure there's opportunity to do that, as well, if you are so inclined. All that said, your az example seems a little silly on pylint's part. :) /alex ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Doubts about Pylint
On Wed, Apr 9, 2008 at 12:43 PM, Dick Moores [EMAIL PROTECTED] wrote: I'd never used Pylint until yesterday, when I discovered that Ulipad had a Pylint plugin that enabled me to run Pylint on scripts within Ulipad. But I'm wondering about some of the results. I noticed that it was complaining that my variable names violated convention. Here's an image of running Pylint on a script with only 7 lines: http://www.rcblue.com/Misc/PylintInUlipad.jpg Since when is 'az' a bad variable name? And 'AZ' is OK? Note: I've never used pylint before, so this is all speculation based on a bit of reading of the pylint web page[1] and PEP 8 [2]. In your code snippet, az is a global variable. Pylint has a regular expression that determines if a global variable name matches your coding convention. By default, that regex is (([A-Z_][A-Z1-9_]*)|(__.*__))$. At a guess, that's because pylint's author believes the only global variables you should have are psuedo-constants, and that they should have all uppercase names. That seems reasonable to me, if a bit strict. That particular check does not line up with the PEP 8 coding conventions, which just suggest that global variables follow the same naming rules as functions. I haven't gone through pylint in a lot of detail, but it looks like most of the other regular expressions are designed to default to the PEP 8 coding style conventions, or something close to them. If your coding conventions are different from the defaults pylint assumes, you'll probably need to do some setup. And I tried Pylint on an official Python module file, calendar.py. Look what I got when all the checkboxes are checked!!: http://www.rcblue.com/Ulipad/PyLintOnCalendar.py.jpg Comments? Most of that looks like valid complaints about calendar.py. The one exception that jumps out at me is the warning about a relative import from __future__. At lot of the other warnings probably depend on the context. For instance, the single-character variable names are ugly, but if they're used inside a loop or a list comprehension they are probably fine. Other than the fact that it's a long list, did *you* have any comments? You present this list like it's a bad thing, but it seems to me that pylint is doing exactly what it should. Do you think that there's something wrong with pylint? Are you just surprised that calendar.py doesn't adhere to pylint's coding guidelines? 1: http://www.logilab.org/card/pylintfeatures 2: http://www.python.org/dev/peps/pep-0008/ ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Doubts about Pylint
At 09:59 AM 4/9/2008, Alex Ezell wrote: On Wed, Apr 9, 2008 at 11:43 AM, Dick Moores [EMAIL PROTECTED] wrote: Comments? Since we started using code profilers and checkers like pyLint etc., we've had a motto: This is a guide. It is not the gospel. Take from pylint what you think helps and ignore the rest. It's just a tool and you can choose how to use it. Your advice is well-taken. But 2 points/puzzlements. 1. Why does Pylint advocate variable names be in all caps? I thought I should reserve all caps for indicating a constant.. 2. Why is the code in many official Python modules (in Python25\Lib) so sloppy by Pylint standards? See those many warnings and errors in http://www.rcblue.com/Ulipad/PyLintOnCalendar.py.jpg That is, unless you want to actually change pylint. I'm sure there's opportunity to do that, as well, if you are so inclined. All that said, your az example seems a little silly on pylint's part. :) Dick Moores UliPad The Python Editor: http://code.google.com/p/ulipad/ ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Doubts about Pylint
On Wed, Apr 9, 2008 at 9:43 AM, Dick Moores [EMAIL PROTECTED] wrote: I'd never used Pylint until yesterday ... Since when is 'az' a bad variable name? And 'AZ' is OK? ... Comments? I understand that Pylint settings and output are *very* customizable. I seem to remember talk about a PEP['Style Guilde'] config but I don't know if that ever happened. ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Doubts about Pylint
At 10:14 AM 4/9/2008, Jerry Hill wrote: Other than the fact that it's a long list, did *you* have any comments? You present this list like it's a bad thing, but it seems to me that pylint is doing exactly what it should. Do you think that there's something wrong with pylint? Are you just surprised that calendar.py doesn't adhere to pylint's coding guidelines? Yes, I didn't make that clear. See my previous post, which seems to have crossed yours. 1: http://www.logilab.org/card/pylintfeatures 2: http://www.python.org/dev/peps/pep-0008/ Thanks for these links. Knew about #2, but not #1. Dick UliPad The Python Editor: http://code.google.com/p/ulipad/ ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Google App Engine
I watched the Campfire videos. Very interesting. Big drawback: no support for join queries. Reasons given seemed pretty weak. How would one migrate an existing app that has hundreds of (in some cases) involved joins? The only way I can see is to write a bunch of queries and then join them in the python code. Seems ugly. Would be nice also to see support for GWT (and pyjamas). -- Bob Gailer 919-636-4239 Chapel Hill, NC ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Google App Engine
bob gailer wrote: I watched the Campfire videos. Very interesting. Big drawback: no support for join queries. Reasons given seemed pretty weak. Because the underlying datastore (BigTable) doesn't support them? I'm not sure but I think this is a key to the scalability of the data store. How would one migrate an existing app that has hundreds of (in some cases) involved joins? The only way I can see is to write a bunch of queries and then join them in the python code. Seems ugly. Yes, I think so too. My guess is that if you started from scratch with GAE you would structure the data differently. Or maybe GAE just isn't suitable for that kind of app. Would be nice also to see support for GWT (and pyjamas). What is the barrier to using pyjamas? Kent ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
[Tutor] List comprehensions
Here is a for loop operating on a list of string items: data = [string 1, string 2, string 3, string 4, string 5, string 6, string 7, string 8, string 9, string 10, string 11] result = for item in data: result = item + \n print result I want to replace the for loop with a List Comrehension (or whatever) to improve performance (as the data list will be 10,000]. At each stage of the for loop I want to print the result ie. [print (item + \n) for item in data] But, this doesn't work as the inclusion of the print causes an invalid syntax error. Any thoughts? Dinesh ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] List comprehensions
On Wed, Apr 9, 2008 at 7:12 AM, Dinesh B Vadhia [EMAIL PROTECTED] wrote: I want to replace the for loop with a List Comrehension (or whatever) to improve performance (as the data list will be 10,000]. At each stage of the for loop I want to print the result ie. List comprehensions are for building lists, not consuming them. If you want to do something with every element in a list, other than building a new list with it, you should be using a for loop. [print (item + \n) for item in data] But, this doesn't work as the inclusion of the print causes an invalid syntax error. Any thoughts? Don't do this. Perhaps you could share a small piece of code that you think is too slow, and ask for advice in speeding it up? If you're not sure which small pieces of code are too slow, you need to profile your application to find out. See the documentation for python's profile module. If you don't have enough code written to profile, then it's probably too early to be doing these optimizations. -- Jerry ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] List comprehensions
On Wed, Apr 9, 2008 at 7:44 PM, Jerry Hill [EMAIL PROTECTED] wrote: On Wed, Apr 9, 2008 at 7:12 AM, Dinesh B Vadhia [EMAIL PROTECTED] wrote: I want to replace the for loop with a List Comrehension (or whatever) to improve performance (as the data list will be 10,000]. At each stage of the for loop I want to print the result ie. List comprehensions are for building lists, not consuming them. If you want to do something with every element in a list, other than building a new list with it, you should be using a for loop. [print (item + \n) for item in data] But, this doesn't work as the inclusion of the print causes an invalid syntax error. Any thoughts? Don't do this. Perhaps you could share a small piece of code that you think is too slow, and ask for advice in speeding it up? If you're not sure which small pieces of code are too slow, you need to profile your application to find out. See the documentation for python's profile module. If you don't have enough code written to profile, then it's probably too early to be doing these optimizations. -- Jerry ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor if you explain the source of the list, what do you want to change in it, what destination will it take, i m sure the guys here will help a lot. ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] List comprehensions
Sorry, let's start again. Here is a for loop operating on a list of string items: data = [string 1, string 2, string 3, string 4, string 5, string 6, string 7, string 8, string 9, string 10, string 11] result = for item in data: result = some operation on item print result I want to replace the for loop with another structure to improve performance (as the data list will contain 10,000 string items]. At each iteration of the for loop the result is printed (in fact, the result is sent from the server to a browser one result line at a time) The for loop will be called continuously and this is another reason to look for a potentially better structure preferably a built-in. Hope this makes sense! Thank-you. Dinesh - Original Message - From: Kent Johnson To: Dinesh B Vadhia Cc: tutor@python.org Sent: Wednesday, April 09, 2008 12:40 PM Subject: Re: [Tutor] List comprehensions Dinesh B Vadhia wrote: Here is a for loop operating on a list of string items: data = [string 1, string 2, string 3, string 4, string 5, string 6, string 7, string 8, string 9, string 10, string 11] result = for item in data: result = item + \n print result I'm not sure what your goal is here. Do you mean to be accumulating all the values in data into result? Your sample code does not do that. I want to replace the for loop with a List Comrehension (or whatever) to improve performance (as the data list will be 10,000]. At each stage of the for loop I want to print the result ie. [print (item + \n) for item in data] But, this doesn't work as the inclusion of the print causes an invalid syntax error. You can't include a statement in a list comprehension. Anyway the time taken to print will swamp any advantage you get from the list comp. If you just want to print the items, a simple loop will do it: for item in data: print item + '\n' Note this will double-space the output since print already adds a newline. If you want to create a string with all the items with following newlines, the classic way to do this is to build a list and then join it. To do it with the print included, try result = [] for item in data: newItem = item + '\n' print newItem result.append(newItem) result = ''.join(result) Kent ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] List comprehensions
i can't think of anything but a loop here UNLESS you take the list from its source one element at a time, process it then print the result. example of this would be : list comes in from standard input. list comes from a database list is read from a file. so again where the list comes from is important. if its standard input then you program will be easy won't use any memory i guess. import sys # cgi cgitb if going web data = sys.stdin.readline() do what ever you like with data print data On Wed, Apr 9, 2008 at 8:15 PM, Dinesh B Vadhia [EMAIL PROTECTED] wrote: Sorry, let's start again. Here is a for loop operating on a list of string items: data = [string 1, string 2, string 3, string 4, string 5, string 6, string 7, string 8, string 9, string 10, string 11] result = for item in data: result = some operation on item print result I want to replace the for loop with another structure to improve performance (as the data list will contain 10,000 string items]. At each iteration of the for loop the result is printed (in fact, the result is sent from the server to a browser one result line at a time) The for loop will be called continuously and this is another reason to look for a potentially better structure preferably a built-in. Hope this makes sense! Thank-you. Dinesh - Original Message - *From:* Kent Johnson [EMAIL PROTECTED] *To:* Dinesh B Vadhia [EMAIL PROTECTED] *Cc:* tutor@python.org *Sent:* Wednesday, April 09, 2008 12:40 PM *Subject:* Re: [Tutor] List comprehensions Dinesh B Vadhia wrote: Here is a for loop operating on a list of string items: data = [string 1, string 2, string 3, string 4, string 5, string 6, string 7, string 8, string 9, string 10, string 11] result = for item in data: result = item + \n print result I'm not sure what your goal is here. Do you mean to be accumulating all the values in data into result? Your sample code does not do that. I want to replace the for loop with a List Comrehension (or whatever) to improve performance (as the data list will be 10,000]. At each stage of the for loop I want to print the result ie. [print (item + \n) for item in data] But, this doesn't work as the inclusion of the print causes an invalid syntax error. You can't include a statement in a list comprehension. Anyway the time taken to print will swamp any advantage you get from the list comp. If you just want to print the items, a simple loop will do it: for item in data: print item + '\n' Note this will double-space the output since print already adds a newline. If you want to create a string with all the items with following newlines, the classic way to do this is to build a list and then join it. To do it with the print included, try result = [] for item in data: newItem = item + '\n' print newItem result.append(newItem) result = ''.join(result) Kent ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] List comprehensions
Dinesh B Vadhia wrote: Here is a for loop operating on a list of string items: data = [string 1, string 2, string 3, string 4, string 5, string 6, string 7, string 8, string 9, string 10, string 11] result = for item in data: result = item + \n print result I'm not sure what your goal is here. Do you mean to be accumulating all the values in data into result? Your sample code does not do that. I want to replace the for loop with a List Comrehension (or whatever) to improve performance (as the data list will be 10,000]. At each stage of the for loop I want to print the result ie. [print (item + \n) for item in data] But, this doesn't work as the inclusion of the print causes an invalid syntax error. You can't include a statement in a list comprehension. Anyway the time taken to print will swamp any advantage you get from the list comp. If you just want to print the items, a simple loop will do it: for item in data: print item + '\n' Note this will double-space the output since print already adds a newline. If you want to create a string with all the items with following newlines, the classic way to do this is to build a list and then join it. To do it with the print included, try result = [] for item in data: newItem = item + '\n' print newItem result.append(newItem) result = ''.join(result) Kent ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] List comprehensions
Dinesh B Vadhia wrote: Here is a for loop operating on a list of string items: data = [string 1, string 2, string 3, string 4, string 5, string 6, string 7, string 8, string 9, string 10, string 11] result = for item in data: result = some operation on item print result I want to replace the for loop with another structure to improve performance (as the data list will contain 10,000 string items]. At each iteration of the for loop the result is printed (in fact, the result is sent from the server to a browser one result line at a time) Any savings you have from optimizing this loop will be completely swamped by the network time. Why do you think this is a bottleneck? You could use [ sys.stdout.write(some operation on item) for item in data ] but I consider this bad style and I seriously doubt you will see any difference in performance. The for loop will be called continuously and this is another reason to look for a potentially better structure preferably a built-in. What do you mean 'called continuously'? Kent ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] List comprehensions
On Wed, Apr 9, 2008 at 4:15 PM, Dinesh B Vadhia [EMAIL PROTECTED] wrote: Sorry, let's start again. This version really isn't any more helpful than the first one. I know you corrected the sample code, but you haven't addressed any of the fundamental questions that Kent or I asked. I want to replace the for loop with another structure to improve performance (as the data list will contain 10,000 string items]. At each iteration of the for loop the result is printed (in fact, the result is sent from the server to a browser one result line at a time) Are you looking for a different data structure to hold your list of strings, or are you looking for a replacement for the for loop? How long does it take to generate your list? How long does each iteration of your for loop take? What are acceptable times for each of these? You need to know these things before you can seriously optimize anything. If building up the list takes a long time and you only use it once, consider using a generator instead of building the whole list and then processing it. This spreads out the time to create the list and operates on each piece of data as soon as it's available. I don't think you're going to find a replacement for a for loop that is inherently faster. You keep talking about list comprehensions, but I don't see how a list comprehension is even appropriate for the problem you're describing. The for loop will be called continuously and this is another reason to look for a potentially better structure preferably a built-in. Again, it would be helpful to discuss actual bits of code, so we can see if there are places you can gain some performance. It's hard to optimize psuedocode, because there are sometimes very minor changes you can make which affect performance quite a bit. For instance, attribute lookup in python is relatively slow. If you can hoist any attribute lookups out of your loop, you will get some performance increases. Also, you should mention what version of python you're using and what platform it's running on. -- Jerry ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] List comprehensions
On Wed, Apr 9, 2008 at 4:48 PM, Kent Johnson [EMAIL PROTECTED] wrote: You could use [ sys.stdout.write(some operation on item) for item in data ] but I consider this bad style and I seriously doubt you will see any difference in performance. This really isn't a good idea. It will take just as long as the for loop, plus it builds a list with the return code for each call to sys.stdout.write() call (which is probably None). Then after building the list with 10,000 entries of None, it throws it away. That can't be good for performance. -- Jerry ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Process that starts processes
John Chandler [EMAIL PROTECTED] wrote working fine, but the process I am starting starts several other processes. Is there any way (using subprocess or a different module) to control the processes the original creates (by control I mean feed them input, capture output, and kill them). Its tricky but if you know their names and they are only produced from your master you could use 'ps' and 'grep' to get the pids. You can then attach to the pids. But its very messy. It is probably more reliable to replace your top level process with Python code, but that depends on how complex that code is... Thee may be other ways to get the pids using lower level proc type calls but you still have the problem of identifying which pids you are interested in! All I can think of at this time of night! Alan G. ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Doubts about Pylint
Dick Moores [EMAIL PROTECTED] wrote Since when is 'az' a bad variable name? And 'AZ' is OK? When it is a constant. pyLint sees that you are assigning a numeroc literal and so thinks that this may be a definition of a constant value. If you disd someting like A = 8 az = A It may well be happy since A is a constant and the variable is being assigned the constant rather than the literal. All lint tools are by tradition very exacting, even the C version which pyLint is modelled on is notorious for throwing irrelevant errors - like not checking the return value of a printf() function (printf returns the number of characters printed but virtually nobody ever checks that!) As another poster said treat it in the spirit it is intended, a tool to highlight *possible* causes for concern not definite causes. -- Alan Gauld Author of the Learn to Program web site http://www.freenetpages.co.uk/hp/alan.gauld ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
[Tutor] Copy script
I have a folder of 150,000 pdf files and I need to copy 20,000 of them to another folder. The ones I need to copy are in a .txt file. Attached is a sample of the input .txt file and a dos directory of the folder containing the files. I'm not sure of the best way to accomplish this. Thank you in advance for your help. Sample input file: 12347424 12347425 12347426 12347427 Sample dir on folder: 03/20/2008 09:21 AM 145,257 00011479 [7ac0c741-9d2f-4a1d-9dbf-8de27ca0abb4].pdf 03/20/2008 09:21 AM 154,655 00011600 [cd752a5a-388a-4bad-9a52-fb3711f685b8].pdf 03/20/2008 09:21 AM 145,556 00015234 [4344f5ff-fa58-4e20-bf27-697c71a81fbc].pdf 03/20/2008 09:21 AM 152,785 12347424 [44cc0d43-a80d-4415-8e92-b6a4f62986b4].pdf 03/20/2008 09:21 AM 145,551 12347425 [0ea7b60a-3631-4f64-91fa-6e385296f18f].pdf Volume in drive C has no label. Volume Serial Number is 0873-E099 Directory of C:\Documents and Settings\test 04/09/2008 03:57 PMDIR . 04/09/2008 03:57 PMDIR .. 03/20/2008 09:21 AM 145,647 0737 [f05d4dcc-8c0e-4660-b9f7-83708dfc782e].pdf 03/20/2008 09:21 AM 144,945 1237 [b951dfd8-af0e-4243-9951-0cf63b7f5f1a].pdf 03/20/2008 09:21 AM 145,984 1491 [7a9d5843-55d7-4a37-9903-a210ff538d12].pdf 03/20/2008 09:21 AM 145,644 1777 [dd9b13cc-f088-4957-a2df-a9dac2959e81].pdf 03/20/2008 09:21 AM 145,730 1995 [cb12e1ef-074a-4b41-abfc-6aa2048d5008].pdf 03/20/2008 09:21 AM 149,319 2609 [dc64fabf-fe2f-47bf-90a4-f1eef43586d6].pdf 03/20/2008 09:21 AM 145,994 2789 [b7242dd2-cfa9-435c-a8f7-05d9d500dbcc].pdf 03/20/2008 09:21 AM 145,788 4169 [9ed9f002-12e2-4a7a-952e-2d2a5a44b775].pdf 03/20/2008 09:21 AM 152,189 4247 [18b05f77-d429-4231-8e99-541e07613e3a].pdf 03/20/2008 09:21 AM 161,951 8623 [decd3cb9-c6bf-4d17-9bdd-cb1d63023aeb].pdf 03/20/2008 09:21 AM 147,243 00010957 [1ae26ed1-8114-49ab-9eeb-40f636ffd257].pdf 03/20/2008 09:21 AM 145,257 00011479 [7ac0c741-9d2f-4a1d-9dbf-8de27ca0abb4].pdf 03/20/2008 09:21 AM 154,655 00011600 [cd752a5a-388a-4bad-9a52-fb3711f685b8].pdf 03/20/2008 09:21 AM 145,556 00015234 [4344f5ff-fa58-4e20-bf27-697c71a81fbc].pdf 03/20/2008 09:21 AM 152,785 12347424 [44cc0d43-a80d-4415-8e92-b6a4f62986b4].pdf 03/20/2008 09:21 AM 145,551 12347425 [0ea7b60a-3631-4f64-91fa-6e385296f18f].pdf 04/09/2008 03:57 PM 0 dir.txt 17 File(s) 2,374,238 bytes 2 Dir(s) 39,714,271,232 bytes free 12347424 12347425 12347426 12347427 12347428 12347429 12347430 12347431 12347432 12347433 12347434 12347435 12347437 12347438 12347439 12347440 12347442 12347443 ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Copy script
Que Prime [EMAIL PROTECTED] wrote I have a folder of 150,000 pdf files and I need to copy 20,000 of them to another folder. The ones I need to copy are in a .txt file. Sounds straightforward but... Attached is a sample of the input .txt file and a dos directory of the folder containing the files. I'm not sure of the best way to accomplish this. Thank you in advance for your help. Sample input file: 12347424 12347425 Sample dir on folder: 03/20/2008 09:21 AM 145,257 00011479 [7ac0c741-9d2f-4a1d-9dbf-8de27ca0abb4].pdf I don;t see how the input file relates to the pdf files? Which part of the pdf file does the input numbers refer to? However it may be that the DOS FOR command with the /F option may be adapted to do what you want. It iterates over the content of a file performing a command per item in the file... Try Help FOR to get more info and try experimenting... DOS is a braindead beast but does sometimes have the facilities needed... The XP DOS commands are considerably more powerful than the old DOS set. Some of them even have a Unix like backtick facility! -- Alan Gauld Author of the Learn to Program web site http://www.freenetpages.co.uk/hp/alan.gauld ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Copy script
Alan Gauld wrote: "Que Prime" [EMAIL PROTECTED] wrote I have a folder of 150,000 pdf files and I need to copy 20,000 of them to another folder. The ones I need to copy are in a .txt file. Sounds straightforward but... Attached is a sample of the input .txt file and a dos directory of the folder containing the files. I'm not sure of the best way to accomplish this. Thank you in advance for your help. Sample input file: 12347424 12347425 Sample dir on folder: 03/20/2008 09:21 AM 145,257 00011479 [7ac0c741-9d2f-4a1d-9dbf-8de27ca0abb4].pdf I don;t see how the input file relates to the pdf files? Which part of the pdf file does the input numbers refer to? There was a time in the good old days that programmers knew about writing specifications. Nowadays specification by example seems to be the defacto standard. Sigh. I can guess that given 03/20/2008 09:21 AM 152,785 12347424 [44cc0d43-a80d-4415-8e92-b6a4f62986b4].pdf 12347424 [44cc0d43-a80d-4415-8e92-b6a4f62986b4].pdf is the file name. I happen to notice that 12347424 appears in the input file and in the assumed file name. So the specification *might be* choose files where characters 5-12 of the name match the eight characters of a line of the input file. Wouldn't it be nice if the asker of the question were to say "yes that is correct" or "no and here is the correct specification". It would have been even nicer if he/she had given the specification the first time. Would have saved everyone time and helped ensure a quick and correct answer. -- Bob Gailer 919-636-4239 Chapel Hill, NC ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Copy script
Que Prime wrote: I have a folder of 150,000 pdf files and I need to copy 20,000 of them to another folder. The ones I need to copy are in a .txt file. Attached is a sample of the input .txt file and a dos directory of the folder containing the files. I'm not sure of the best way to accomplish this. So a sample filename is 12347425 [0ea7b60a-3631-4f64-91fa-6e385296f18f].pdf and you would want to copy this because 12347425 is in the txt file? If that is correct understanding, I think I would - read the txt file and put all the numbers in a set. - iterate through the file names in the source dir - isolate the initial number part of the file name, strip the leading 0's - check if the initial number is in the set - if so, then copy the file Are all the files in one directory? If so I guess you will have to have enough memory to hold all the file names (as well as the set of numbers) and probably a bit of patience! I don't know a way to generate the sequence without creating the full list. Kent Thank you in advance for your help. Sample input file: 12347424 12347425 12347426 12347427 Sample dir on folder: 03/20/2008 09:21 AM 145,257 00011479 [7ac0c741-9d2f-4a1d-9dbf-8de27ca0abb4].pdf 03/20/2008 09:21 AM 154,655 00011600 [cd752a5a-388a-4bad-9a52-fb3711f685b8].pdf 03/20/2008 09:21 AM 145,556 00015234 [4344f5ff-fa58-4e20-bf27-697c71a81fbc].pdf 03/20/2008 09:21 AM 152,785 12347424 [44cc0d43-a80d-4415-8e92-b6a4f62986b4].pdf 03/20/2008 09:21 AM 145,551 12347425 [0ea7b60a-3631-4f64-91fa-6e385296f18f].pdf ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Copy script
Alan Gauld wrote:
Que Prime [EMAIL PROTECTED] wrote
I have a folder of 150,000 pdf files and I need to copy 20,000 of
them to
another folder. The ones I need to copy are in a .txt file.
Assuming the text file is a list of those to be copied,
wouldn't a simple bash script do the trick?
(don't know about windows, but this would work on Mac or Linux,
and, I assume there must be a way to do this with a windows command
line script of some sort.)
#!/bin/bash
cd /path/to/dir/with/files/and/list
llistofiles=$(cat your .txt file here)
echo Copying files...
for each i in $listofiles
do
cp $i /path/to/other/folder/$i
echo All done...b'bye...
exit
Or, perhaps with tcl
#!/usr/bin/wish
set listofiles [read your txt file here]
puts Copying files...
foreach a {$listofiles} {
file copy $a /path/to/target/dir
}
puts All done...b'bye...
exit
I imagine python could do something quite similar,
but confess I am just lurking on this list and have barely begun
to learn python.
/tony
--
Anthony Baldwin
http://www.BaldwinLinguas.com
Translation Interpreting
http://www.TransProCalc.org
Free translation project mgmt software
http://www.LinguasOS.org
Linux for Translators
___
Tutor maillist - Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Doubts about Pylint
At 03:15 PM 4/9/2008, Alan Gauld wrote: Dick Moores [EMAIL PROTECTED] wrote Since when is 'az' a bad variable name? And 'AZ' is OK? When it is a constant. pyLint sees that you are assigning a numeric literal and so thinks that this may be a definition of a constant value. If you did something like A = 8 az = A It may well be happy since A is a constant and the variable is being assigned the constant rather than the literal. Thanks, Alan, but I tried your A = 8 az = A and got the same complaint about az. Dick UliPad The Python Editor: http://code.google.com/p/ulipad/ ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
