[Tutor] use of variables in re.sub
Helo, I need to do a substitution in a regular expression that depends on a variable. To simplify I want to be able to do this substitution: text2='XX. AA YY. DD' re.sub(ur\. (AA|BB|ÇÇ,(.) \g1,text2) this would give as a result: 'XX(.) AA YY. DD' Which is exactly what I want. But when I try to substitute AA|BB|CC for a variable I do not know how to make it work. I wanted to use re.compile. I tryed: NP= uAA|BB|ÇÇ reNP=re.compile(ur'%s'%NP) But if I try: reNP.match(text2) And I do not know how to use the variables in there. Should I use re.compile or is there something else that I should be using Thank you Eva Bofias ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] associating two objects without ORM and processing a text file
neubyr wrote: I am not sure how to save an object in memory to a file before exiting the program. Any examples or related documentation links would be really helpful. I am guessing it would be using some kind of before teardown method, but not sure about it. Any help? Look at the pickle or shelve modules. http://www.doughellmann.com/PyMOTW/pickle/index.html http://www.doughellmann.com/PyMOTW/shelve/index.html This email is confidential and subject to important disclaimers and conditions including on offers for the purchase or sale of securities, accuracy and completeness of information, viruses, confidentiality, legal privilege, and legal entity disclaimers, available at http://www.jpmorgan.com/pages/disclosures/email. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] associating two objects without ORM and processing a text file
On 02/14/2013 12:35 PM, Prasad, Ramit wrote: neubyr wrote: I am not sure how to save an object in memory to a file before exiting the program. Any examples or related documentation links would be really helpful. I am guessing it would be using some kind of before teardown method, but not sure about it. Any help? Look at the pickle or shelve modules. http://www.doughellmann.com/PyMOTW/pickle/index.html http://www.doughellmann.com/PyMOTW/shelve/index.html You miss the point. The OP wants to make sure the text file is saved no matter how the program happens to exit. He's not asking how to format the file. -- DaveA ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
[Tutor] Newbie Here -- Averaging Adding Madness Over a Given (x) Range?!?!
Hello all, This is my first post here. I have tried to get answers from StackOverflow, but I realized quickly that I am too green for that environment. As such, I have purchased Beginning Python (2nd edition, Hetland) and also the $29.00 course available from learnpythonthehardway(dot)com. I have been reading fervently, and have enjoyed python -- very much. I can do all the basic printing, math, substitutions, etc. Although, I am stuck when trying to combine all the new skills I have been learning over the past few weeks. Anyway, I was hoping to get some help with something NON-HOMEWORK related. (I swear.) I have a task that I have generalized due to the nature of what I am trying to do -- and it's need to remain confidential. My end goal as described on SO was: Calculating and Plotting the Average of every (X) items in a list of (Y) total, but for now I am only stuck on the actual addition, and/or averaging items -- in a serial sense, based on the relation to the previous number, average of numbers, etc being acted on. Not the actual plotting. (Plotting is pretty EZ.) Essentially: 1. I have a list of numbers that already exist in a file. I generate this file by parsing info from logs. 2. Each line contains an integer on it (corresponding to the number of milliseconds that it takes to complete a certain repeated task). 3. There are over a million entries in this file, one per line; at any given time it can be just a few thousand, or more than a million. Example: --- 173 1685 1152 253 1623 Eventually what I'll need to do is: 1. Index the file and/or count the lines, as to identify each line's positional relevance so that it can average any range of numbers that are sequential; one to one another. 2. Calculate the difference between any given (x) range. In order to be able to ask the program to average every 5, 10, 100, 100, or 10,000 etc. -- until completion. This includes the need to dealing with stray remainders at the end of the file that aren't divisible by that initial requested range. (ie: average some file with 3,245 entries by 100 -- not excluding the remaining 45 entries, in order to represent the remainder.) So, looking above, transaction #1 took 173 milliseconds, while transaction #2 took 1685 milliseconds. Based on this, I need to figure out how to do two things: 1. Calculate the difference of each transaction, related to the one before it AND record/capture the difference. (An array, list, dictionary -- I don't care.) 2. Starting with the very first line/entry, count the first (x number) and average (x). I can obtain a Happy medium for what the gradient/delta is between sets of 100 over the course of the aggregate. ie: --- Entries 1-100 = (eventualPlottedAvgTotalA) Entries 101-200 = (eventualPlottedAvgTotalB) Entries 201-300 = (eventualPlottedAvgTotalC) Entries 301-400 = (eventualPlottedAvgTotalD) From what I can tell, I don't need to indefinitely store the values, only pass them as they are processed (in order) to the plotter. I have tried the following example to sum a range of 5 entries from the above list of 5 (which works), but I don't know how to dynamically pass the 5 at a time until completion, all the while retaining the calculated averages which will ultimately be passed to pyplot at a later time/date. What I have been able to figure out thus far is below. ex: Python 2.7.3 (default, Jul 24 2012, 10:05:38) [GCC 4.7.0 20120507 (Red Hat 4.7.0-5)] on linux2 Type help, copyright, credits or license for more information. plottedTotalA = ['173', '1685', '1152', '253', '1623'] sum(float(t) for t in plottedTotalA) 4886.0 I received 2 answers from SO, but was unable to fully capture what they were trying to tell me. Unfortunately, I might need a baby-step / Barney-style mentor who is willing to guide me on this. I hope this makes sense to someone out there, and thank you in advance for any help that you can provide. I apologize in advance for being so thick if its uber-EZ. -- Mike ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] associating two objects without ORM and processing a text file
Dave Angel wrote:
On 02/14/2013 12:35 PM, Prasad, Ramit wrote:
neubyr wrote:
I am not sure how to save an object in memory to a file before exiting the
program. Any examples or
related documentation links would be really helpful. I am guessing it
would be using some kind of
before teardown method, but not sure about it. Any help?
Look at the pickle or shelve modules.
http://www.doughellmann.com/PyMOTW/pickle/index.html
http://www.doughellmann.com/PyMOTW/shelve/index.html
You miss the point. The OP wants to make sure the text file is saved no
matter how the program happens to exit. He's not asking how to format
the file.
Hmm. Good point Dave, I did miss that point.
My knee jerk response is a try/finally block, but I am sure there
are better ways.
# UNTESTED
stored_data = {}
try:
stored_data = load_data()
while True:
#do program
except Exception:
raise # reraise exception to keep trace and still
# propogate error for attention
finally:
store_data(stored_data) # Save data since we are exiting
# (intentionally or not).
~Ramit
This email is confidential and subject to important disclaimers and
conditions including on offers for the purchase or sale of
securities, accuracy and completeness of information, viruses,
confidentiality, legal privilege, and legal entity disclaimers,
available at http://www.jpmorgan.com/pages/disclosures/email.
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Newbie Here -- Averaging Adding Madness Over a Given (x) Range?!?!
On 02/14/2013 03:55 PM, Michael McConachie wrote: Hello all, This is my first post here. I have tried to get answers from StackOverflow, but I realized quickly that I am too green for that environment. As such, I have purchased Beginning Python (2nd edition, Hetland) and also the $29.00 course available from learnpythonthehardway(dot)com. I have been reading fervently, and have enjoyed python -- very much. I can do all the basic printing, math, substitutions, etc. Although, I am stuck when trying to combine all the new skills I have been learning over the past few weeks. Anyway, I was hoping to get some help with something NON-HOMEWORK related. (I swear.) I have a task that I have generalized due to the nature of what I am trying to do -- and it's need to remain confidential. My end goal as described on SO was: Calculating and Plotting the Average of every (X) items in a list of (Y) total, but for now I am only stuck on the actual addition, and/or averaging items -- in a serial sense, based on the relation to the previous number, average of numbers, etc being acted on. Not the actual plotting. (Plotting is pretty EZ.) If you're stuck on the addition, why give us all the other parts? Your problem statement is very confused, and you don't show much actual code. Essentially: 1. I have a list of numbers that already exist in a file. I generate this file by parsing info from logs. 2. Each line contains an integer on it (corresponding to the number of milliseconds that it takes to complete a certain repeated task). 3. There are over a million entries in this file, one per line; at any given time it can be just a few thousand, or more than a million. Example: --- 173 1685 1152 253 1623 So write a loop that reads this file into a list of ints, converting each line. Then you can tell us you've got a list of about a million ints. Eventually what I'll need to do is: 1. Index the file and/or count the lines, as to identify each line's positional relevance so that it can average any range of numbers that are sequential; one to one another. 2. Calculate the difference between any given (x) range. In order to be able to ask the program to average every 5, 10, 100, 100, or 10,000 etc. -- until completion. This includes the need to dealing with stray remainders at the end of the file that aren't divisible by that initial requested range. (ie: average some file with 3,245 entries by 100 -- not excluding the remaining 45 entries, in order to represent the remainder.) So, looking above, transaction #1 took 173 milliseconds, while transaction #2 took 1685 milliseconds. Based on this, I need to figure out how to do two things: 1. Calculate the difference of each transaction, related to the one before it AND record/capture the difference. (An array, list, dictionary -- I don't care.) What difference, what transaction, related how? 2. Starting with the very first line/entry, count the first (x number) and average (x). I can obtain a Happy medium for what the gradient/delta is between sets of 100 over the course of the aggregate. What's an x-number? What, what, which, who ? ie: --- Entries 1-100 = (eventualPlottedAvgTotalA) Entries 101-200 = (eventualPlottedAvgTotalB) Entries 201-300 = (eventualPlottedAvgTotalC) Entries 301-400 = (eventualPlottedAvgTotalD) From what I can tell, I don't need to indefinitely store the values, only pass them as they are processed (in order) to the plotter. I have tried the following example to sum a range of 5 entries from the above list of 5 (which works), but I don't know how to dynamically pass the 5 at a time until completion, all the while retaining the calculated averages which will ultimately be passed to pyplot at a later time/date. What I have been able to figure out thus far is below. ex: Python 2.7.3 (default, Jul 24 2012, 10:05:38) [GCC 4.7.0 20120507 (Red Hat 4.7.0-5)] on linux2 Type help, copyright, credits or license for more information. plottedTotalA = ['173', '1685', '1152', '253', '1623'] sum(float(t) for t in plottedTotalA) 4886.0 I received 2 answers from SO, but was unable to fully capture what they were trying to tell me. Unfortunately, I might need a baby-step / Barney-style mentor who is willing to guide me on this. I hope this makes sense to someone out there, and thank you in advance for any help that you can provide. I apologize in advance for being so thick if its uber-EZ. If you want to make a sublist out of the first 2 items in a list, you can use a slice (notice the colon): allvalues = [ 173, 1685, 1152, 263, 1623, 19 ] firsttwo = allvalues[0:2] To get the 3rd such sublist, use othertwo = allvalues[4:2] If you've made such a list, you can readily use sum directly on it: mysum = sum(othertwo) -- DaveA ___ Tutor
[Tutor] Python needs your help -- trademark under attack
Hello all, The Python Software Foundation is the organisation which protects and manages the boring bits of keeping a big open source project alive: the legal and contractual parts, funding for projects, trademarks and copyrights. If you are based in Europe, or know somebody who uses Python in Europe, the PSF needs your help. There is a company in the UK who has applied to trademark the name Python and are claiming the *exclusive* right to use the word Python for software, servers, and web services over the entire European Union. You can read more about this here: http://pyfound.blogspot.com/2013/02/python-trademark-at-risk-in-europe-we.html If you have documentation of European user groups, trade associations, books, conferences, scans of job advertisements for Python programmers, software that uses some variation of Python in the name, etc. your evidence will be helpful in defeating this attempted grab of the Python name. You can also testify to the fact that when you read or hear of the name Python in relation to computers and the Internet, you think of Python the programming language. Thank you. -- Steven ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] associating two objects without ORM and processing a text file
On 02/14/2013 04:33 PM, Prasad, Ramit wrote:
Dave Angel wrote:
On 02/14/2013 12:35 PM, Prasad, Ramit wrote:
neubyr wrote:
I am not sure how to save an object in memory to a file before exiting the
program. Any examples or
related documentation links would be really helpful. I am guessing it would be
using some kind of
before teardown method, but not sure about it. Any help?
Look at the pickle or shelve modules.
http://www.doughellmann.com/PyMOTW/pickle/index.html
http://www.doughellmann.com/PyMOTW/shelve/index.html
You miss the point. The OP wants to make sure the text file is saved no
matter how the program happens to exit. He's not asking how to format
the file.
Hmm. Good point Dave, I did miss that point.
My knee jerk response is a try/finally block, but I am sure there
are better ways.
# UNTESTED
stored_data = {}
try:
stored_data = load_data()
while True:
#do program
except Exception:
raise # reraise exception to keep trace and still
# propogate error for attention
finally:
store_data(stored_data) # Save data since we are exiting
# (intentionally or not).
That would be my reaction as well. I would, however make it conditional
on some changes having been made. That way if this program run only
made queries, the effort and risk of saving can be avoided.
The other thing I'd recommend is to store the data in an alternate file,
and only delete the original when the alternate is ready to rename.
That way, you can't readily get into trouble if something crashes while
saving.
--
DaveA
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Newbie Here -- Averaging Adding Madness Over a Given (x) Range?!?!
On 15/02/13 07:55, Michael McConachie wrote: Essentially: 1. I have a list of numbers that already exist in a file. I generate this file by parsing info from logs. 2. Each line contains an integer on it (corresponding to the number of milliseconds that it takes to complete a certain repeated task). 3. There are over a million entries in this file, one per line; at any given time it can be just a few thousand, or more than a million. Example: --- 173 1685 1152 253 1623 A million entries sounds like a lot to you or me, but to your computer, it's not. When you start talking tens or hundreds of millions, that's possibly a lot. Do you know how to read those numbers into a Python list? Here is the baby step way to do so: data = [] # Start with an empty list. f = open(filename) # Obviously you have to use the actual file name. for line in f: # Read the file one line at a time. num = int(line) # Convert each line into an integer (whole number) data.append(num) # and append it to the end of the list. f.close() # Close the file when done. Here's a more concise way to do it: with open(filename) as f: data = [int(line) for line in f] Once you have that list of numbers, you can sum the whole lot: sum(data) or just a range of the items: sum(data[:100]) # The first 100 items. sum(data[100:200]) # The second 100 items. sum(data[-50:]) # The last 50 items. sum(data[1000:]) # Item 1001 to the end. (See below.) sum(data[5:99:3]) # Every third item, starting at index 5 and ending at index 98. This is called slicing, and it is perhaps the most powerful and useful technique that Python gives you for dealing with lists. The rules though are not necessarily the most intuitive though. A slice is either a pair of numbers separated with a colon, inside the square brackets: data[start:end] or a triple: data[start:end:step] Any of these three numbers can be left out. The default values are: start=0 end=length of the sequence being sliced step=1 They can also be negative. If start or end are negative, they are interpreted as from the end rather than from the beginning. Item positions are counted from 0, which will be very familiar to C programmers. The start index is included in the slice, the end position is excluded. The model that you should think of is to imagine the sequence of items labelled with their index, starting from zero, and with a vertical line *between* each position. Here is a sequence of 26 items, showing the index in the first line and the value in the second: |0|1|2|3|4|5|6|7|8|9| ... |25| |a|b|c|d|e|f|g|h|i|j| ... |z | When you take a slice, the items are always cut at the left. So, if the above is called letters, we have: letters[0:4] # returns abcd letters[2:8] # returns cdefgh letters[2:8:2] # returns ceg letters[-3:] # returns xyz Eventually what I'll need to do is: 1. Index the file and/or count the lines, as to identify each line's positional relevance so that it can average any range of numbers that are sequential; one to one another. No need. Python already does that, automatically, when you read the data into a list. 2. Calculate the difference between any given (x) range. In order to be able to ask the program to average every 5, 10, 100, 100, or 10,000 etc. -- until completion. This includes the need to dealing with stray remainders at the end of the file that aren't divisible by that initial requested range. I don't quite understand you here. First you say difference, then you say average. Can you show a sample of data, say, 10 values, and the sorts of typical calculations you want to perform, with the answers you expect to get? For example, here's 10 numbers: 103, 104, 105, 109, 111, 112, 115, 120, 123, 128 Here are the running averages of 3 values: (103+104+105)/3 (104+105+109)/3 (105+109+111)/3 (109+111+112)/3 (111+112+115)/3 (112+115+120)/3 (115+120+123)/3 (120+123+128)/3 Is that what you mean? If so, then Python can deal with this trivially, using slicing. With your data stored in list data, as above, I can say: for i in range(0, len(data)-3): # Stop 3 from the end. print sum(data[i:i+3]) to print the running sums taking three items at a time. The rest of your post just confuses me. Until you explain exactly what calculations you are trying to perform, I can't tell you how to perform them :-) -- Steven ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Newbie Here -- Averaging Adding Madness Over a Given (x) Range?!?!
On 2/14/2013 3:55 PM, Michael McConachie wrote: [snip] I agree with dave angel - the specification is far from clear. please clarify. perhaps a simple example that goes from input to desired output. -- Bob Gailer 919-636-4239 Chapel Hill NC ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] use of variables in re.sub
On 14 February 2013 12:57, Eva Bofias eva.bof...@gmail.com wrote: Helo, Hi, I need to do a substitution in a regular expression that depends on a variable. To simplify I want to be able to do this substitution: text2='XX. AA YY. DD' re.sub(ur\. (AA|BB|ÇÇ,(.) \g1,text2) There is a missing bracket in that expression. did you mean ur\. (AA|BB|ÇÇ)? this would give as a result: 'XX(.) AA YY. DD' Which is exactly what I want. But when I try to substitute AA|BB|CC for a variable I do not know how to make it work. How about this? import re text2='XX. AA YY. DD' re.sub(ur\. (AA|BB|ÇÇ),r(.) \g1,text2) # ¡Note the r raw string! 'XX(.) AA YY. DD' Or have I misunderstood? Oscar ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] associating two objects without ORM and processing a text file
On Wed, Feb 13, 2013 at 1:55 PM, Alan Gauld alan.ga...@btinternet.comwrote: On 13/02/13 19:14, neubyr wrote: I am not sure how to save an object in memory to a file before exiting the program. Any examples or related documentation links would be really helpful. I am guessing it would be using some kind of before teardown method, but not sure about it. Any help? If using class methods or standalone functions just call them explicitly at the start and end of your program. If you want to be sure it gets called use a try/finally try: Book.loadBooks(filename) # load the data # do your program stuff finally: Book.saveBooks(filename) # save the data That ensures that even if there is an exception the data will always be saved. Thanks Alan! - N ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] associating two objects without ORM and processing a text file
On Thu, Feb 14, 2013 at 4:05 PM, Dave Angel da...@davea.name wrote:
On 02/14/2013 04:33 PM, Prasad, Ramit wrote:
Dave Angel wrote:
On 02/14/2013 12:35 PM, Prasad, Ramit wrote:
neubyr wrote:
I am not sure how to save an object in memory to a file before exiting
the program. Any examples or
related documentation links would be really helpful. I am guessing it
would be using some kind of
before teardown method, but not sure about it. Any help?
Look at the pickle or shelve modules.
http://www.doughellmann.com/**PyMOTW/pickle/index.htmlhttp://www.doughellmann.com/PyMOTW/pickle/index.html
http://www.doughellmann.com/**PyMOTW/shelve/index.htmlhttp://www.doughellmann.com/PyMOTW/shelve/index.html
You miss the point. The OP wants to make sure the text file is saved no
matter how the program happens to exit. He's not asking how to format
the file.
Hmm. Good point Dave, I did miss that point.
My knee jerk response is a try/finally block, but I am sure there
are better ways.
# UNTESTED
stored_data = {}
try:
stored_data = load_data()
while True:
#do program
except Exception:
raise # reraise exception to keep trace and still
# propogate error for attention
finally:
store_data(stored_data) # Save data since we are exiting
# (intentionally or not).
That would be my reaction as well. I would, however make it conditional
on some changes having been made. That way if this program run only made
queries, the effort and risk of saving can be avoided.
The other thing I'd recommend is to store the data in an alternate file,
and only delete the original when the alternate is ready to rename. That
way, you can't readily get into trouble if something crashes while saving.
Thanks Ramit and Dave!
I haven't had chance to code/learn further, but I will give play with this
soon.
I do have a doubt regarding this - e.g. how would I implement this if my
program/application is web based. For example, loading the text file during
web server start and stop.
Hope to try it out soon!
- N
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] associating two objects without ORM and processing a text file
On Thu, Feb 14, 2013 at 4:33 PM, Prasad, Ramit
ramit.pra...@jpmorgan.com wrote:
My knee jerk response is a try/finally block, but I am sure there
are better ways.
The atexit.register decorator hooks sys.exitfunc:
http://docs.python.org/2/library/atexit
Registered atexit functions are run early during interpreter
finalization. Signal handling is still enabled (e.g. SIGINT), and you
can still import. In the standard lib, logging and multiprocessing use
atexit.
In a C extension there's also Py_AtExit for registering a C function:
http://docs.python.org/2/c-api/sys.html#Py_AtExit
Calling the exitfuncs is the last task in Py_Finalize, after
everything has been torn down, so they should not use the C-API.
Example:
import atexit
from tempfile import NamedTemporaryFile
from subprocess import Popen, PIPE
from ctypes import CDLL, pythonapi
@atexit.register
def f():
print shutdown: atexit
# register a C function
with NamedTemporaryFile() as so:
p = Popen(['gcc', '-xc', '-shared', '-fPIC', '-o',
so.name, '-'], stdin=PIPE, stdout=so)
p.communicate('''#include stdio.h
void fc(void) {printf(shutdown: Py_AtExit\\n);}''')
fc = CDLL(so.name).fc # keep reference
pythonapi.Py_AtExit(fc)
try:
raise RuntimeError
finally:
print shutdown: finally
Output:
shutdown: finally
Traceback (most recent call last):
File atexit_example.py, line 20, in module
raise RuntimeError
RuntimeError
shutdown: atexit
shutdown: Py_AtExit
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor
