Re: [Tutor] Beginner question (variables, namespaces...) (fwd)
-- Forwarded message -- Date: Fri, 7 Apr 2006 21:05:33 -0600 From: Jesse <[EMAIL PROTECTED]> To: Danny Yoo <[EMAIL PROTECTED]> Subject: Re: [Tutor] Beginner question (variables, namespaces...) I tried redefining the "higher-order" variables as functions, but it didn't quite work. Here's a simplified example: var1 = 2 def timestwo(x): return x*2 var2 = timestwo(var1) print var1, var2 var1 = 3 print var1, var2 This results in the output: 2, 4 3,4 ..which is not what I'm aiming for. Maybe I'll have to follow Bob's advice and just store all of the variable assignments in a function, and then call the function every time I change one of the variables (based on user input). I could still leave the higher-order variables as functions as per your advice, but that alone doesn't seem to do the trick. Unless I've missed something. Jesse ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
[Tutor] Watch and control access to an executable
Hi Tutors!
I have a problem that I've solved using Python, and I want to know if
I've done a good job :-)
Here's the problem:
At my work we have a Windows network. On a network drive lives a program
that a couple people need to access (there's a shortcut on each user's
Desktop to the executable). The problem is, only *one* person at a time
should run the program. For various reasons, it's possible for
information to get corrupted if two people are using the program at the
same time.
Here's the old solution to the problem:
Pick-up the phone.
Dial Dave's extension.
Dave this is Bill, I'm going to use the program.
Use the program.
Call Dave again and tell him I'm done using the program.
And Dave does the same for me.
Putting it mildly - this is a less than ideal way to run software ;-)
Now here's my solution:
1. Create a program let's call it 'myProg' that spawns 'otherProg'.
2. 'myProg' will utilize a config file.
3. When 'myProg' is started it looks in the config file to see if
'otherProg' is running.
4. If 'otherProg' is not running, start it and write to the config file
that 'otherProg' is running (also write who is running it).
5. 'myProg' continues to run on the user's computer, continuously
checking the PID (of 'otherProg') to see if the process is still alive.
6. When we don't find the PID anymore, write to the config file that
'otherProg' isn't running.
7. Shutdown 'myProg'.
Now in step #3 above - if 'myProg' reads the config file and finds that
the 'otherProg' is currently running, the user is warned that 'The
program is currently in use by !' And
'otherProg' is not started.
Couple of other things.
1. I'm taking 'myProg' and creating a single-file executable using py2exe.
2. 'myProg.exe' and the config file live on the network in the same
directory as 'otherProg'.
3. User's have a shortcut on the Desktop to 'myProg.exe' instead of
'otherProg'.
BTW, I've never stopped to consider if there was a simple 'Windows
networking / permissions' type solution to the problem. I just went
straight to Python :-)
Here's my code. I'd appreciate any critiques!
Thanks,
Bill
import time
import os
import sys
import getpass
from ConfigParser import ConfigParser
import win32pdhutil
import win32con
import win32api
CONFIG_FILE = 'WatchProc.ini'
# Is there a better way to deal with this
# default config file data?
defaultConfigData = \
"""
[Process]
proc =
[Current User]
user =
[Executable]
exe =
[Process Status]
running =
[Shutdown Status]
ok =
"""
class WatchProc:
def __init__(self):
self.config = Config()
self.user = getpass.getuser()
self.checkConfig()
def checkConfig(self):
"""
Check the config file and see if a process is listed.
If nothing is listed throw an error, else do
checkStatus().
"""
proc = self.config.getConfig('Process', 'proc')
if proc == '':
self.configFileError()
else:
self.checkStatus()
def checkStatus(self):
"""
Check the config file to see if the process is running
or not. If running throw an error, else start the app.
"""
status = self.config.getConfig('Process Status', 'running')
if status == 'True': # App is in use.
self.usageError()
elif status == 'False': # App not in use.
self.startApp()
else: # Config file is not setup properly.
self.configFileError()
def startApp(self):
"""
Write the user's name to the config file.
Start the executable.
Then monitor the process.
"""
self.config.setConfig('Current User', 'user', self.user)
self.startExe()
time.sleep(1)
self.monitorProcess()
def monitorProcess(self):
"""
Get the process name from the config file then continuously
check to see if the process is running. When the process
dies, call cleanup().
"""
procname = self.config.getConfig('Process', 'proc')
self.config.setConfig('Shutdown Status', 'ok', False)
CHECK = True
while CHECK:
try:
pid = \
win32pdhutil.FindPerformanceAttributesByName(procname)
time.sleep(.5)
except: # App has stopped running.
CHECK = False
self.cleanup()
def startExe(self):
"""
Grab the name of the executable to start.
Write to the config file that we're running.
Spawn.
"""
exe = self.config.getConfig('Executable', 'exe')
self.config.setConfig('Process Status', 'running', True)
os.spawnv(os.P_NOWAIT, exe, [])
def cleanup(self):
"""
Set the config file to the proper values (for a clean
shutdown) and then exit.
"""
self.config.setConfig('Shutdown Status', 'ok',
Re: [Tutor] Unittest not running all test cases
Kent: > -Original Message- > From: Kent Johnson [mailto:[EMAIL PROTECTED] > Sent: Friday, April 07, 2006 3:59 PM > To: Carroll, Barry > Subject: Re: [Tutor] Unittest not running all test cases > > Carroll, Barry wrote: > > Kent: > > > > I just rechecked my class names, and there are no duplicates. I was > > careful to preface all of the classes in testsymc39 with 'C39...', and > > used 'S25...' in testsym25. Likewise, the classes in each module have > > names that describe the type of data being tested, so names are unique > > within each module as well. > > > > Is there a maximum length for class names? Some of my class names are > > pretty long, and only differ in the last few characters. I'd be > > surprised if that were the case in Python, but I'm short on other ideas. > > I don't know of any limits on class names. They are just strings in > dictionaries after all. > > Try running your tests in verbose mode, you should at least be able to > figure out which ones aren't running. Try > > unittest.main(argv=['', '-v']) > > Kent > I tried your suggestion. Lo and behold, all the test cases ran! So I tried the terse mode again. All the test cases STILL ran! So the problem has vanished without a trace. Or a good reason. Having a problem just disappear like that bothers me almost more than the original problem. (It's also a little embarrassing.) =8^( Anyway, thanks for your help. Regards, Barry [EMAIL PROTECTED] 541-302-1107 We who cut mere stones must always be envisioning cathedrals. -Quarry worker's creed ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Beginner question (variables, namespaces...)
> > buy_containers = roundup(need/container) > > > Yeah, rather than code these as explicit variables, I'd strongly recommend > rewriting each of these as functions. Concretely: > > def buy_containers(stock, weekly_quota, container): > return roundup(need(stock, weekly_quota) - extra(stock, weekly_quota)) Gaaa. I don't know where the heck that function came from. *grin* Let me try that again: # def buy_containers(stock, weekly_quota, container): return roundup(need(stock, weekly_quota) / container) # Sorry; my eyes must have been wandering when I was writing that code. ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Beginner question (variables, namespaces...)
Jesse wrote:
> Why is it that when one variable is assigned a value in terms of
> another variable, assigning a new value to the first doesn't change
> the value of the second? This is giving me a huge headache, since I
> have a bunch of variables defined in terms of one another, and I want
> to be able to dynamically update them (I did not include the
> definitions of the functions overstock and roundup, but they work):
>
> stock = float(raw_input("Enter stock (in terms of portions: "))
> container = float(raw_input("Enter portions per container: "))
> price_per_container = float(raw_input("Enter price per container: "))
> weekly_quota = float(raw_input("Enter quota (in terms of portions): "))
> extra = overstock(stock, weekly_quota) # overstock returns 0 if the
> first argument is less than the second; otherwise it returns the
>
> difference between the first argument and the second.
> need = weekly_quota - extra
> buy_containers = roundup(need/container) # roundup rounds a
> non-integer to the next highest integer
> buy_portions = buy_containers * container
> leftover = buy_portions - need
> cost = price_per_container * buy_containers
>
> I would like to write a function that will update the values of the
> above variables given an increase in only the stock variable.
> Otherwise I'd have to repeat a bunch of code...:(
def recalculate():
global extra, need, buy_containers, buy_portions, leftover, cost
extra = overstock(stock, weekly_quota)
need = weekly_quota - extra
buy_containers = roundup(need/container)
buy_portions = buy_containers * container
leftover = buy_portions - need
cost = price_per_container * buy_containers
That's all there is to it.
___
Tutor maillist - Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Beginner question (variables, namespaces...)
On Fri, 7 Apr 2006, Jesse wrote:
> Why is it that when one variable is assigned a value in terms of another
> variable, assigning a new value to the first doesn't change the value of
> the second?
Hi Jesse,
If you have a "variable" that depends on the values of other parameters,
that's just begging to be represented as a function. Let's look at some
of those computations:
> stock = float(raw_input("Enter stock (in terms of portions: "))
> container = float(raw_input("Enter portions per container: "))
> price_per_container = float(raw_input("Enter price per container: "))
> weekly_quota = float(raw_input("Enter quota (in terms of portions): "))
> extra = overstock(stock, weekly_quota)
> need = weekly_quota - extra
> buy_containers = roundup(need/container) # roundup rounds a non-integer to
> the next highest integer
> buy_portions = buy_containers * container
> leftover = buy_portions - need
> cost = price_per_container * buy_containers
Yeah, rather than code these as explicit variables, I'd strongly recommend
rewriting each of these as functions. Concretely:
def extra(stock, weekly_quota):
return overstock(stock, weekly_quota)
def need(stock, weekly_quota):
return weekly_quota - extra(stock, weekly_quota)
def buy_containers(stock, weekly_quota, container):
return roundup(need(stock, weekly_quota) - extra(stock, weekly_quota))
...
It's a little ugly, but this coding explicitely defines the dependencies
between the inputs into our system and the expected outputs. When we look
at buy_containers(), we can easily see that any change in the stock,
weekly_quote, or container parameters may have some profound affect on the
output to buy_containers().
That dependency between input and output is what we try to capture when we
write a function. So if you recode your assignment statements as function
definitions, you should be better able to handle changes to your input.
If we think of the names we use for these concepts, there's something
ironic that functions behave better on varying data than variables. Oh
well.
As an advanced aside: the "equations" that you're writing into Python,
unfortunately, aren't treated as real math equations, but as separate,
independent assignment statements. I think I know what you want, but
Python doesn't provide it out of the box. What I think you want is called
"constraint" programming. Such systems do exist. For example:
http://www.logilab.org/projects/constraint
I can't vouch for the maturity of logilab's "constraint" module; I haven't
played with it yet.
But if you're interested, you may want to look at the treatment of simple
constraint systems in the classic textbook "Structure and Interpretation
of Computer Programs":
http://mitpress.mit.edu/sicp/full-text/book/book-Z-H-22.html#%_sec_3.3.5
It should be very possible to adapt the material there into Python,
although it might take a fair bit of work if you're not familiar with
Scheme.
Good luck to you!
___
Tutor maillist - Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor
[Tutor] Beginner question (variables, namespaces...)
Why is it that when one variable is assigned a value in terms of another variable, assigning a new value to the first doesn't change the value of the second? This is giving me a huge headache, since I have a bunch of variables defined in terms of one another, and I want to be able to dynamically update them (I did not include the definitions of the functions overstock and roundup, but they work):
stock = float(raw_input("Enter stock (in terms of portions: "))container = float(raw_input("Enter portions per container: "))price_per_container = float(raw_input("Enter price per container: "))
weekly_quota = float(raw_input("Enter quota (in terms of portions): "))extra = overstock(stock, weekly_quota) # overstock returns 0 if the first argument is less than the second; otherwise it returns the
difference between the first argument and the second.need = weekly_quota - extrabuy_containers = roundup(need/container) # roundup rounds a non-integer to the next highest integer
buy_portions = buy_containers * containerleftover = buy_portions - needcost = price_per_container * buy_containers
I would like to write a function that will update the values of the above variables given an increase in only the stock variable. Otherwise I'd have to repeat a bunch of code...:(
Jesse
___
Tutor maillist - Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Python performance resources & resouce usage hints
Well, thanks very much Kent, Hugo and Danny. I went with the "never-ending blocking queues" and sentinel data approach. When running tests with a continual stream of packets being received 3ms apart, CPU usage peaked at 15%, was usually around 7-9%, and when deployed the packets will separated by seconds rather than milliseconds. Thanks for the assistance, I've now overcome my fear of blocking I/O :). Regards, Lia, Clarke On 4/8/06, Liam Clarke <[EMAIL PROTECTED]> wrote: > Thanks very much all. :) I'll have a crack this afternoon and let you know. > > Kent - the increase in the queue size for the socket server is to > allow for any delay in processing packets; it has a default queue size > of 5 and then it starts rejecting packets; more of a safety policy > when reducing CPU usage than a direct attempt to reduce CPU usage. > > Once again, thanks for the input! > > On 4/8/06, Danny Yoo <[EMAIL PROTECTED]> wrote: > > > > > > On Fri, 7 Apr 2006, Kent Johnson wrote: > > > > > Hugo Gonz�lez Monteverde wrote: > > > > You are not using the optional timeout and blocking which 'get' > > > > provides (!) > > > > > > > > Try setting it and see your CPU usage go down. This will implement > > > > blocking, and the queue will be used as soon as data is there. Set > > > > block > > > > to True and a timeout if you need to use it (looks like you only need > > > > blocking) > > > > > > > > while True: > > > > try: > > > > data = self.queue.get(True) > > > > self.DAO.send_data(data) > > > > > > I think he will need the timeout too, otherwise the shutdown flag will > > > only be checked when there is work which is probably not what he wants. > > > > > > This could be fixed: when setting the 'shutdown' flag, also push a > > sentinel piece of data into the queue. That'll wake the thread back up, > > and that'll give it the opportunity to process a shutdown. > > > > The thread here: > > > > http://mail.python.org/pipermail/tutor/2006-January/044557.html > > > > has some more examples of this. > > > > > > Hope this helps! > > > > > > > > ___ > > Tutor maillist - Tutor@python.org > > http://mail.python.org/mailman/listinfo/tutor > > > > > > > ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Unittest not running all test cases
Kent: I just rechecked my class names, and there are no duplicates. I was careful to preface all of the classes in testsymc39 with 'C39...', and used 'S25...' in testsym25. Likewise, the classes in each module have names that describe the type of data being tested, so names are unique within each module as well. Is there a maximum length for class names? Some of my class names are pretty long, and only differ in the last few characters. I'd be surprised if that were the case in Python, but I'm short on other ideas. Thanks for your help. Regards, Barry [EMAIL PROTECTED] 541-302-1107 We who cut mere stones must always be envisioning cathedrals. -Quarry worker's creed > -Original Message- > Date: Fri, 07 Apr 2006 15:42:15 -0400 > From: Kent Johnson <[EMAIL PROTECTED]> > Subject: Re: [Tutor] Unittest not running all test cases > Cc: tutor@python.org > Message-ID: <[EMAIL PROTECTED]> > Content-Type: text/plain; charset=ISO-8859-1; format=flowed > <> > > I don't think there is a limit like this. I wonder, do you have any name > collisions? For example if testsymc39 and testsym25 both contain class > TestSym then only the second one will run because it will shadow the > name from the first module. > > If this is the case you need a more sophisticated way of accumulating > tests. You might want to look into using nose or one of the other > test-discovery frameworks listed here: > http://pycheesecake.org/wiki/PythonTestingToolsTaxonomy#UnitTestingTools > > Kent ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Python performance resources & resouce usage hints
Thanks very much all. :) I'll have a crack this afternoon and let you know. Kent - the increase in the queue size for the socket server is to allow for any delay in processing packets; it has a default queue size of 5 and then it starts rejecting packets; more of a safety policy when reducing CPU usage than a direct attempt to reduce CPU usage. Once again, thanks for the input! On 4/8/06, Danny Yoo <[EMAIL PROTECTED]> wrote: > > > On Fri, 7 Apr 2006, Kent Johnson wrote: > > > Hugo Gonz�lez Monteverde wrote: > > > You are not using the optional timeout and blocking which 'get' provides > > > (!) > > > > > > Try setting it and see your CPU usage go down. This will implement > > > blocking, and the queue will be used as soon as data is there. Set > > > block > > > to True and a timeout if you need to use it (looks like you only need > > > blocking) > > > > > > while True: > > > try: > > > data = self.queue.get(True) > > > self.DAO.send_data(data) > > > > I think he will need the timeout too, otherwise the shutdown flag will > > only be checked when there is work which is probably not what he wants. > > > This could be fixed: when setting the 'shutdown' flag, also push a > sentinel piece of data into the queue. That'll wake the thread back up, > and that'll give it the opportunity to process a shutdown. > > The thread here: > > http://mail.python.org/pipermail/tutor/2006-January/044557.html > > has some more examples of this. > > > Hope this helps! > > > > ___ > Tutor maillist - Tutor@python.org > http://mail.python.org/mailman/listinfo/tutor > > > ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] string formatting question
On 7 Apr 2006, [EMAIL PROTECTED] wrote:
> Sorry I didn't make my question clearer. Bascially I
> want to replace this line:
>
> address="64.41.134.60"/>
>
> With:
>
> address="64.41.134.60"/>
>
> So the regex grouping are that I want to keep
> portNumber= and tcpORudp= and replace the values.
> Which will be varibles in my code.
>
> The question is more on the string formatting in the
> replace. How do use two %s in one statement?
>
> i.e.: re.sub('\1 %s \2 %s' % var1 % var2, line)
You could write it simply like that:
Python> s = ''
Python> re.sub('".*?"','"%s"',s,2)
''
Python> re.sub('".*?"','"%s"',s,2) % ('1000', 'TCP')
''
Or you could exploit the fact that you can use a function instead of a
simply string as substitution; in that function you can do really
complicated things.
Karl
--
Please do *not* send copies of replies to me.
I read the list
___
Tutor maillist - Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Unittest not running all test cases
Carroll, Barry wrote: > Greetings: > > I'm not certain this is the right forum for this question. If not, please point me to the correct one. > > I am using the unittest module to test a package our team is writing. > I presently have three modules of test cases and a top level module to run the entire suite. The hierarchy looks like this: > testsymgen top level > testsymc39 61 test cases > testsym2544 test cases > testsymc93 0 test cases (so far) > > testsymgen is a very simple file. Here it is: > > import unittest > from testsymc39 import * > from testsym25 import * > from testsymc93 import * > > > if __name__=='__main__': > unittest.main( ) > > Each of the sub-modules runs correctly, but when I run testsymgen, only 99 test cases are executed. Can anyone tell me why this should happen. Is there some sort of limit on the number of test cases that can be run in a batch? I don't think there is a limit like this. I wonder, do you have any name collisions? For example if testsymc39 and testsym25 both contain class TestSym then only the second one will run because it will shadow the name from the first module. If this is the case you need a more sophisticated way of accumulating tests. You might want to look into using nose or one of the other test-discovery frameworks listed here: http://pycheesecake.org/wiki/PythonTestingToolsTaxonomy#UnitTestingTools Kent ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
[Tutor] Unittest not running all test cases
Greetings: I'm not certain this is the right forum for this question. If not, please point me to the correct one. I am using the unittest module to test a package our team is writing. I presently have three modules of test cases and a top level module to run the entire suite. The hierarchy looks like this: testsymgen top level testsymc39 61 test cases testsym2544 test cases testsymc93 0 test cases (so far) testsymgen is a very simple file. Here it is: > import unittest from testsymc39 import * from testsym25 import * from testsymc93 import * if __name__=='__main__': unittest.main( ) > Each of the sub-modules runs correctly, but when I run testsymgen, only 99 test cases are executed. Can anyone tell me why this should happen. Is there some sort of limit on the number of test cases that can be run in a batch? Thanks in advance for your help. Regards, Barry Regards, Barry [EMAIL PROTECTED] 541-302-1107 We who cut mere stones must always be envisioning cathedrals. -Quarry worker's creed ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Python performance resources & resouce usage hints
On Fri, 7 Apr 2006, Kent Johnson wrote: > Hugo Gonz�lez Monteverde wrote: > > You are not using the optional timeout and blocking which 'get' provides (!) > > > > Try setting it and see your CPU usage go down. This will implement > > blocking, and the queue will be used as soon as data is there. Set > > block > > to True and a timeout if you need to use it (looks like you only need > > blocking) > > > > while True: > > try: > > data = self.queue.get(True) > > self.DAO.send_data(data) > > I think he will need the timeout too, otherwise the shutdown flag will > only be checked when there is work which is probably not what he wants. This could be fixed: when setting the 'shutdown' flag, also push a sentinel piece of data into the queue. That'll wake the thread back up, and that'll give it the opportunity to process a shutdown. The thread here: http://mail.python.org/pipermail/tutor/2006-January/044557.html has some more examples of this. Hope this helps! ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] string formatting question
Jerome Jabson wrote:
> Hi Kent,
>
> Sorry I didn't make my question clearer. Bascially I
> want to replace this line:
>
> address="64.41.134.60"/>
>
> With:
>
> address="64.41.134.60"/>
>
> So the regex grouping are that I want to keep
> portNumber= and tcpORudp= and replace the values.
> Which will be varibles in my code.
>
> The question is more on the string formatting in the
> replace. How do use two %s in one statement?
>
> i.e.: re.sub('\1 %s \2 %s' % var1 % var2, line)
Ok, actually now I can reread your original question and it makes sense :-)
I think I'm having an off day for answering questions. I'm glad Alan is
with it :-) he gave the answer you need.
Kent
___
Tutor maillist - Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] string formatting question
Hi Kent,
Sorry I didn't make my question clearer. Bascially I
want to replace this line:
With:
So the regex grouping are that I want to keep
portNumber= and tcpORudp= and replace the values.
Which will be varibles in my code.
The question is more on the string formatting in the
replace. How do use two %s in one statement?
i.e.: re.sub('\1 %s \2 %s' % var1 % var2, line)
Thanks again!
> Hello,
>
> I'm trying to replace some strings in a line of
text,
> using some regex functions. My question is: If
there's
> more then one regex grouping I want to replace in
one
> line of a file, how can I use the String Formatting
> operator (%s) in two places?
Hi Jerome,
I don't understand your question. Can you give a
complete example of
the
line from the file and the new line you want to
create?
>
> Here's the line it matches in the file:
>
> address="64.41.134.60"/>
>
> Here's the regex:
> m_sock = re.compile('(portNumber=)"\d+"
> (tcpORudp=)"[A-Z]+"')
You have put parentheses around fixed strings, so your
groups will
always be the same. Is that what you want?
>
> My replace should look like this:
> \1 "112" \2 "TCP"
> (obviously "112" and "TCP" would be varibles)
This looks like you want to make the string
portNumber= 112 tcpORudp= TCP
but that doesn't have any variable text from the
original string so I
think I must not understand.
Kent
>
> My problem now is how do I construct the replace
> statement?
> twork = m_sock.sub('\1 %s \2 %s', % port_num %
proto,
> twork)
>
> But of course this does not work! :-( Is there a
> better way to do this? Or am I just doing this all
> wrong?
>
> Thanks in advance!
>
__
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com
___
Tutor maillist - Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] function caller
> Hi, Hi, > Can someone explain me function and caller relationship when passing > arguments? There is an explanation of this in vitually any tutorial on Python. Haver you read any of these? Have you any experience in other languages that we can relate an explanation too? For example, you could read the Modules and Functions topic in my tutor. If you still don't undeerstand come back with a specific question and we can try to help some more. Alan G Author of the learn to program web tutor http://www.freenetpages.co.uk/hp/alan.gauld ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] string formatting question
> My problem now is how do I construct the replace
> statement?
> twork = m_sock.sub('\1 %s \2 %s', % port_num % proto,
> twork)
The format operator takes a tuple:
twork = m_sock.sub('\1 %s \2 %s' % (port_num, proto), twork)
So I removed the comma after the string, used a single percent
operator and I put the two variables in a tuple
HTH,
Alan G
Author of the learn to program web tutor
http://www.freenetpages.co.uk/hp/alan.gauld
___
Tutor maillist - Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Python performance resources & resouce usage hints
Hugo González Monteverde wrote: > You are not using the optional timeout and blocking which 'get' provides (!) > > Try setting it and see your CPU usage go down. This will implement > blocking, and the queue will be used as soon as data is there.Set > block > to True and a timeout if you need to use it (looks like you only need > blocking) > > while True: > try: > data = self.queue.get(True) > self.DAO.send_data(data) I think he will need the timeout too, otherwise the shutdown flag will only be checked when there is work which is probably not what he wants. I agree, this is a good solution. Kent ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] string formatting question
Jerome Jabson wrote:
> Hello,
>
> I'm trying to replace some strings in a line of text,
> using some regex functions. My question is: If there's
> more then one regex grouping I want to replace in one
> line of a file, how can I use the String Formatting
> operator (%s) in two places?
Hi Jerome,
I don't understand your question. Can you give a complete example of the
line from the file and the new line you want to create?
>
> Here's the line it matches in the file:
>
> address="64.41.134.60"/>
>
> Here's the regex:
> m_sock = re.compile('(portNumber=)"\d+"
> (tcpORudp=)"[A-Z]+"')
You have put parentheses around fixed strings, so your groups will
always be the same. Is that what you want?
>
> My replace should look like this:
> \1 "112" \2 "TCP"
> (obviously "112" and "TCP" would be varibles)
This looks like you want to make the string
portNumber= 112 tcpORudp= TCP
but that doesn't have any variable text from the original string so I
think I must not understand.
Kent
>
> My problem now is how do I construct the replace
> statement?
> twork = m_sock.sub('\1 %s \2 %s', % port_num % proto,
> twork)
>
> But of course this does not work! :-( Is there a
> better way to do this? Or am I just doing this all
> wrong?
>
> Thanks in advance!
>
> __
> Do You Yahoo!?
> Tired of spam? Yahoo! Mail has the best spam protection around
> http://mail.yahoo.com
> ___
> Tutor maillist - Tutor@python.org
> http://mail.python.org/mailman/listinfo/tutor
>
>
___
Tutor maillist - Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor
[Tutor] string formatting question
Hello,
I'm trying to replace some strings in a line of text,
using some regex functions. My question is: If there's
more then one regex grouping I want to replace in one
line of a file, how can I use the String Formatting
operator (%s) in two places?
Here's the line it matches in the file:
Here's the regex:
m_sock = re.compile('(portNumber=)"\d+"
(tcpORudp=)"[A-Z]+"')
My replace should look like this:
\1 "112" \2 "TCP"
(obviously "112" and "TCP" would be varibles)
My problem now is how do I construct the replace
statement?
twork = m_sock.sub('\1 %s \2 %s', % port_num % proto,
twork)
But of course this does not work! :-( Is there a
better way to do this? Or am I just doing this all
wrong?
Thanks in advance!
__
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com
___
Tutor maillist - Tutor@python.org
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Quick question,
Carlos Benevides wrote: > Hi, > > Can someone tell me what the r before the expression means, as in > "re.compile(r'(\.exe|\.zip|\.pif|\.scr|\.ps|\.pdf|\.ppt)$')"? It is not specific to regular expressions, it is called a raw string. http://docs.python.org/tut/node5.html ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] function caller
josip wrote: > Can someone explain me function and caller relationship when passing > arguments? Hi... what do you mean? Is there something specific you're not getting if you take a look at: http://www.ibiblio.org/obp/thinkCSpy/chap03.htm http://docs.python.org/tut/node6.html#SECTION00660 We'd be glad to help with that, but we need the doubt to be more specific. Hugo ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Python performance resources & resouce usage hints
Liam Clarke wrote: > Each thread's run() method basically looks like this - > > while True: > try: > data = self.queue.get(False) > self.DAO.send_data(data) > except Empty: > if self.shutdown: > print "\DAO closing" > return > continue > > Hi Liam, I checked the response by Kent. I completely agree with him in that the problem is the endless polling you have. However, you do not have to implement a sleep in your loop; the main issue may be that you are not using the facilities of the 'get' method in the queue. From the queue docs: get([block[, timeout]]) Remove and return an item from the queue. If optional args block is true and timeout is None (the default), block if necessary until an item is available. If timeout is a positive number, it blocks at most timeout seconds and raises the Empty exception if no item was available within that time. Otherwise (block is false), return an item if one is immediately available, else raise the Empty exception (timeout is ignored in that case). You are not using the optional timeout and blocking which 'get' provides (!) Try setting it and see your CPU usage go down. This will implement blocking, and the queue will be used as soon as data is there. Set block to True and a timeout if you need to use it (looks like you only need blocking) while True: try: data = self.queue.get(True) self.DAO.send_data(data) Hope that helps, Hugo ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Quick question,
On Fri, April 7, 2006 8:23 pm, Carlos Benevides wrote: > Hi, > > Can someone tell me what the r before the expression means, as in > "re.compile(r'(\.exe|\.zip|\.pif|\.scr|\.ps|\.pdf|\.ppt)$')"? r is the way of making a string "raw". This means that escape characters will not be interpreted. Here is an example that should make it clear. >>> foo = "This is \t me" >>> print foo This is me >>> foo = r"This is \t me" >>> print foo This is \t me >>> Since slashes are used often for writing regexps, it's useful to make the strings raw. Peace. -- -NI ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Python performance resources & resouce usage hints
Liam Clarke wrote: > Hi, > > I've developed what would be my largest Python app to date. And, going > from the crude Windows Task Manager, it's trying to use as much CPU > time as it can get when it's idle. > > This, no doub,t is due to my design. > > I have an UDP socket server, a packet cruncher, and a DAO. > Each resides in it's own thread, and each thread is connected to the > next by a Queue. > > So it's server ---> cruncher --> DAO. > > Each thread's run() method basically looks like this - > > while True: > try: > data = self.queue.get(False) > self.DAO.send_data(data) > except Empty: > if self.shutdown: > print "\DAO closing" > return > continue > > > i.e. each thread is looping endlessly until data arrives via the > queue. I can't believe it chews the CPU time the way it does, but I > suppose it's easy to do so. Yes, it's easy. You have basically told the thread to check the queue as often as possible. So it is running as fast as it can, checking the queue and handling whatever comes its way. > > My query to the list is twofold - > > First, if anyone knows of any websites with articles on Python > threading optimisation/pitfalls websites, I'd be greatly appreciative. There's not much. This might help: http://linuxgazette.net/107/pai.html This book is excellent for teaching some of the tools that are used for communication and synchronization between threads: http://greenteapress.com/semaphores/ There are many threading related recipes in the Python Cookbook, both online and printed. http://aspn.activestate.com/ASPN/Cookbook/Python > Okay, the 2nd piece of advice I'm seeking, is what would be the most > efficient path here? > My initial thoughts are along three lines: > > Either: > > A) Increase the queue size of the socket servers I don't see how that would help. > B) Use timer threads to 'pulse' my threads. That's a lot of additional complexity. > A) Increase the queue size of the socket servers > B) Use blocking queues > > Or: > > A) Use blocking queues with a timeout > B) Use the socket servers to "wake" processing threads These are all better. > > As Python doesn't have sleeping threads etc, the third option is my > current measure of last resort, as there'll be some substantial > rejigging, and I'm not overly experienced with threads. Use time.sleep() to sleep a thread. The simplest fix is to add a time.sleep() into your loops. Then the thread will check the queue, process any work, and sleep for a little while. (This is called a busy loop.) The disadvantage of this solution is that there is a trade off between CPU usage and responsiveness. If the sleep is very short, the thread will be very responsive but it will still run a lot and use CPU when idling. If you make the timeout long, the idle CPU will be very low but responsiveness will be poor. If you can live with a response delay of 0.05 or 0.1 second, try that, it should cut the CPU usage dramatically. Even a 0.01 delay might make a big difference. A better fix is to use blocking queues or other blocking events. In this approach, a thread will block until there is something to do, then wake up, do its work and go back to sleep. The hitch here is you need to find another way to signal the thread to exit. One possibility is just to mark them as daemon threads, then they will exit when your app exits. This is a very simple solution if you don't have any cleanup actions you want to do in the threads. Another possibility might be to send a "quit" message in the queue. When the thread sees the special quit message, it forwards it to the next thread and exits. If neither of these work then you could use a queue.get() with a timeout so you check the done flag periodically. Kent ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
[Tutor] Quick question,
Hi, Can someone tell me what the r before the expression means, as in "re.compile(r'(\.exe|\.zip|\.pif|\.scr|\.ps|\.pdf|\.ppt)$')"? I've seen regular expressions both ways, and I've seen them work with or without the r. Just wondering what it does, or if it makes a difference. Thanks. ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
[Tutor] function caller
Hi, Can someone explain me function and caller relationship when passing arguments? Thanks How low will we go? Check out Yahoo! Messengers low PC-to-Phone call rates.___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
[Tutor] Python performance resources & resouce usage hints
Hi, I've developed what would be my largest Python app to date. And, going from the crude Windows Task Manager, it's trying to use as much CPU time as it can get when it's idle. This, no doub,t is due to my design. I have an UDP socket server, a packet cruncher, and a DAO. Each resides in it's own thread, and each thread is connected to the next by a Queue. So it's server ---> cruncher --> DAO. Each thread's run() method basically looks like this - while True: try: data = self.queue.get(False) self.DAO.send_data(data) except Empty: if self.shutdown: print "\DAO closing" return continue i.e. each thread is looping endlessly until data arrives via the queue. I can't believe it chews the CPU time the way it does, but I suppose it's easy to do so. My query to the list is twofold - First, if anyone knows of any websites with articles on Python threading optimisation/pitfalls websites, I'd be greatly appreciative. I've been reading the Performance tips on the official wiki, so if there's anything similar I'd be keen to look at it. Okay, the 2nd piece of advice I'm seeking, is what would be the most efficient path here? My initial thoughts are along three lines: Either: A) Increase the queue size of the socket servers B) Use timer threads to 'pulse' my threads. Or: A) Increase the queue size of the socket servers B) Use blocking queues Or: A) Use blocking queues with a timeout B) Use the socket servers to "wake" processing threads As Python doesn't have sleeping threads etc, the third option is my current measure of last resort, as there'll be some substantial rejigging, and I'm not overly experienced with threads. On the wiki http://wiki.python.org/moin/PythonSpeed/PerformanceTips#periodic, I read that "There is a function in the sys module, setcheckinterval, which you can call to tell the interpreter how often to perform these periodic checks." Should I even think about that? I'm not after performance so much as less utilisation of system resources... Much thanks for any guidance offered. Regards, Liam Clarke ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
