Re: [Tutor] The Best Way to go About with Self Modifying Code/Code Generation?
Aaron Tp wrote: Hey all, I have a question on using self-modifying code/code generation in Python; namely how to do it. I know others have already said not to do this, and to be honest I was going to say that same thing, but I have changed my mind. Buggrit, this is Python, where we are all consenting adults and you're allowed to shoot yourself in the foot if you want. Writing self-modifying code actually covers a large range of techniques, ranging from the simple and straight-forward use of conventional factory functions, to the completely crazy self-modifying source code. There's a lot to learn. Rather than try to cover it all, I'll sketch out the various things you should learn about before you can make self-modifying code. Obviously basic Python skills, how to read and write files, import modules, etc. That's the boring part, and I'm going to assume you know all that. If you don't, do a few tutorials and then come back. Still here? Cool! On to the hard part. But first, beware: HERE BE DRAGONS As fans of the British fantasy author Terry Pratchett all know, there are two types of dragons, swamp dragons and Draco nobilis, the noble dragon. Swamp, or common, dragons are small, mostly harmless, with a very finicky digestive system and a regrettable tendency to blow up, but with a little bit of care and attention that can be occasionally useful. The noble dragon, on the other hand, are huge, cunning, cruel, malicious, and anything but harmless. They don't so much blow up as silently fly down out of the sky and vapourize you with white-hot flame. They are best avoided by anyone other than experts. Actually, even experts should avoid them. Writing self-modifying code is like dealing with dragons. And like dealing with dragons, you should start off with the fairly harmless variety before moving up to the more tricky type. So start with this simple example of self-modifying code: http://code.activestate.com/recipes/68429-ring-buffer/ This is probably as harmless and simple as self-modifying code can get: you have an instance which dynamically changes it's own class as it goes, so as to change its behaviour. Other languages might choke on this, but for Python, this counts as trivial. If this example confuses you, if it's too hard for you to understand how it works, then you're not ready to try dragon taming. Learn some more Python first before you get into self-modifying code. We can help you with that. Other simple techniques to start with include: - dynamic code generation with factory functions and closures - shadowing globals, built-ins or modules with your own code - rebinding methods, functions or classes at runtime - monkey-patching builtin functions - functions or methods which rebind *themselves* at runtime Feel free to ask for help on any of these. Or google on key phrases such as monkey-patching, shadowing builtins, factory function, closure. So far, this all counts as the easy, conservative side of self-modifying code. Factory functions are even staid and boring. Shadowing is sometimes useful. Even monkey-patching, which is what those craz-ee Ruby hackers do, is almost respectable. Respectable like drinking two bottles of gin before going fox-hunting -- everybody knows you shouldn't do it, but if you survive the first time, you start to think nothing could possibly go wrong. Be warned that easy is relative -- these can actually be fairly advanced techniques, and are tricky to get right and difficult to debug when they go wrong. And things *will* go wrong. Remember the warnings from the creator of the Unix operating system, Brian Kernighan: Everyone knows that debugging is twice as hard as writing a program in the first place. So if you’re as clever as you can be when you write it, how will you ever debug it? Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it. Okay. You've got your handgun, you're ready to shoot your own foot. Once you've learned to tame the swamp dragons of runtime rebinding, monkey-patching and dynamic code generation, you're ready to move up to the next, slightly more radical, stage: eval and exec. Here are some links to get you started: http://lucumr.pocoo.org/2011/2/1/exec-in-python/ http://nedbatchelder.com/blog/201206/eval_really_is_dangerous.html http://code.activestate.com/recipes/500261-named-tuples/ Still got all your toes? Excellent. Now you *might* be ready to go after the *really* big dragons: - AST manipulation http://www.dalkescientific.com/writings/diary/archive/2010/02/22/instrumenting_the_ast.html http://docs.python.org/library/ast.html - byte code hacks http://nedbatchelder.com/blog/200804/wicked_hack_python_bytecode_tracing.html http://wiki.python.org/moin/ByteplayDoc http://code.activestate.com/recipes/498242/
Re: [Tutor] The Best Way to go About with Self Modifying Code/Code Generation?
Thank you all for your responses. I don't know if I have the capability to even complete such a project, or if I possess the necessary will to start, but at least I understand it a bit more than when I was starting. --Aaron ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Seeking help with reading and writing files in Python
* Aristotle aristo...@triad.rr.com [2012-06-26 14:28]: The sequence of events that I am using now is to open a program that will gather gallons used from user, call a module to make calculations, use the info from the module to write info to a file named FinalProjectBill.txt. After that is done, I open another program that will read the info written to FinalProjectBill.txt I guess my confusion about the instructions is, if I should read from one file FinalProjectBill.txt first how does data get written to that file to begin with? Based on this part, you have already answered your own question. Specifically, you get your input file, FinalProjectBill.txt, by collecting data from the user, doing some calculations, and outputting to the file. I'm not sure I see the problem. :-( -- David Rock da...@graniteweb.com pgpgnpfBiWpdl.pgp Description: PGP signature ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Python and boot sequences
* Dave Wilder d.wil...@f5.com [2012-06-28 12:54]: Can a Python script be written that has the ability to stop a Linux device in the middle of a boot when a certain sequence occurs and then perform an action? For example, when I boot my switch (w/ Linux OS 2.7.3), I want to stop the boot when I see a series of asterisks. When I see this, I need to hit the ESCSHIFT9 sequence. Then, I need to select a menu option (e.g. 1 to run a test or Q to quit and continue w/ the boot). Another example would be when doing a PXE boot, selecting the image to load when prompted during the reboot. Probably not. The issue is that you are dealing with the bootstrap process of a system. Typically, this is not a place where any semblance of external user interaction exists. The examples you show represent compiled-in options that are more likely part of the initial ramdisk, BIOS-level code for a specific device, or something similar. The PXE boot menu example is usually something that's being presented by an external server (via cobbler, or a similar product). There isn't an OS at that point that would likely be able to supply a python environment to use. I think we would need to better understand exactly what you are trying to build, but you are probably looking for something that would be a much lower level than python. -- David Rock da...@graniteweb.com pgp7NxdORAGka.pgp Description: PGP signature ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] The Best Way to go About with Self Modifying Code/Code Generation?
Steven D'Aprano, 08.07.2012 15:48: Hey all, I have a question on using self-modifying code/code generation in Python; namely how to do it. I know others have already said not to do this, and to be honest I was going to say that same thing, but I have changed my mind. Buggrit, this is Python, where we are all consenting adults and you're allowed to shoot yourself in the foot if you want. :) You're right. Saying don't do that is easy, but giving people the freedom to shoot themselves in the foot sometimes has accidental side effects that end up pushing our world another bit forward. (Though usually not, but that's ok) Stefan ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Seeking help with reading and writing files in Python
On 26/06/12 19:28, Aristotle wrote: The sequence of events that I am using now is to open a program that will gather gallons used from user, call a module to make calculations, use the info from the module to write info to a file named FinalProjectBill.txt. ... From Instructor: You should read from one file (FinalProjectBill.txt) and then write to one (BranleyFinal.txt) with the results as we are learning file i/o (input/output) - ... You should not write to your input file. My question is why do I need two text files FinalProjectBill.txt and BranleyJamesFinal.txt ? I didn't read this properly the first time around. The answer to your question is that you don't. It is a design decision. But, if you write back to the input file it makes things more complex because you may have to read/write from the same file at the same time(*). This means moving the file cursor back and forth between the point where you are reading and the end of the file. There are functions to do this but it adds a lot of extra overhead and is easy to get wrong. It is much simpler to read the data from one file and write the output to another and let Python take care of moving the cursor automatically. (*)Of course you may choose to read all the input data in, close the file then overwrite it with your output. That's easy but sometimes the input file is too big or the processing time too limited to do that. Or you may need to preserve the data for use in subsequent processing. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
[Tutor] Mapping ID's for corresponding values in different Columns
Hi-- My current input looks like the following: FILE1.csv PERSON_IDPERSON_NAME 1 Jen 2 Mike 3 Jim 4 5 Jane 6 Joe 7 Jake FILE2.csv PERSON_ID PERSON_NAME Jim Mike Jane Todd Jen _ I want to fill into the PERSON_ID column of FILE2.csv the corresponding ID's associated with those names as identified in FILE1.csv. At first I imported the csv module and was using the csv.Reader, but then it seemed simple enough just to write something like: for line in file2: print(line) giving me the following output: PERSON_ID, PERSON_NAME , Jim , Mike , Jane , Todd , Jen I think I understand the issue at a conceptual level, but not quite sure how to fully implement it: a) I want to build a dictionary to create keys, such that each number in file1 corresponds to a unique string in column B of file1. b) then write a for loop like the following: for person_name in file2: if person_name.file2 == person_name.file1: person_id.file2 == person_id.file1 c) write into file2 the changes to person_id's... But it's pretty difficult for me to get past this stage. Am I on the right track? And more importantly, how could I learn how to actually implement this in smaller stages? Thanks so much. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Mapping ID's for corresponding values in different Columns (UPDATE)
I thought it made sense to read the two columns in File1 in as a dictionary
(where the key is actually the name, so that we can search on it later),
and the column of interest in File2 as a list. Finding the common values
then is just:
for item in file2_list:
for line in file1_dict:
if item == line:
print item
Creating a new dictionary and filling it in with the common values gives us
something like:
new_dict = {}
for item in file2_list:
for line in file1_dict:
if item == line:
new_dict[item] = line
I'm not quite sure where to go from here, though. I want:
a) to edit the new_dict such that we get the proper key and value
combination, instead of what it is now which is just the proper values.
b) Once we have the correct dictionary with the proper keys and their
values, how do I then output it into the form (???):
File2.csv
PERSON_IDPERSON_NAME
key1 Jim
key100 Mike
key3 Jane
key989 Todd
etc... etc...
On Sun, Jul 8, 2012 at 2:47 PM, Fred G bayespoker...@gmail.com wrote:
Hi--
My current input looks like the following:
FILE1.csv
PERSON_IDPERSON_NAME
1 Jen
2 Mike
3 Jim
4
5 Jane
6 Joe
7 Jake
FILE2.csv
PERSON_ID PERSON_NAME
Jim
Mike
Jane
Todd
Jen
_
I want to fill into the PERSON_ID column of FILE2.csv the corresponding
ID's associated with those names as identified in FILE1.csv.
At first I imported the csv module and was using the csv.Reader, but then
it seemed simple enough just to write something like:
for line in file2:
print(line)
giving me the following output:
PERSON_ID, PERSON_NAME
, Jim
, Mike
, Jane
, Todd
, Jen
I think I understand the issue at a conceptual level, but not quite sure
how to fully implement it:
a) I want to build a dictionary to create keys, such that each number in
file1 corresponds to a unique string in column B of file1.
b) then write a for loop like the following:
for person_name in file2:
if person_name.file2 == person_name.file1:
person_id.file2 == person_id.file1
c) write into file2 the changes to person_id's...
But it's pretty difficult for me to get past this stage. Am I on the right
track? And more importantly, how could I learn how to actually implement
this in smaller stages?
Thanks so much.
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Mapping ID's for corresponding values in different Columns (UPDATE)
On 9 July 2012 00:50, Fred G bayespoker...@gmail.com wrote: I thought it made sense to read the two columns in File1 in as a dictionary (where the key is actually the name, so that we can search on it later), and yes... the column of interest in File2 as a list. Finding the common values then is just: for item in file2_list: for line in file1_dict: if item == line: print item No. Once you have a dictionary from which you can look up id's from names, you want to simply read through the lines in file 2, and for each line, you want to pick up the name from that line and look up the corresponding id using the dict. Then, having looked up the number, you want to then write out a line to a new version of File2 that will replace File2 once done. That's more or less it. Exactly how you handle the old/new File2 issue is up to you. Either write a new file and rename the files once done (safer since you can keep the old file as a backup), or read the entire file into memory and then process and write the whole thing back out to the same file (less safe -- what happens if something goes wrong while you're writing out the data to the file?) Walter ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Python and boot sequences
The switch is in a state where user process can be run once the boot sequence has been halted (the aforementioned ESCSHIFT9). However the time between when the reboot is kicked off and when the asterisks appear (and the boot must be stopped), there is no shell Environment. I may be trying to hammer a nail in with a wrench. So I'm going to do a little more digging into the switch code and get to a point where a shell script can be run as Alan suggested. Then I'll be back. Thanks all for the replies. Dave -Original Message- From: tutor-bounces+d.wilder=f5@python.org [mailto:tutor-bounces+d.wilder=f5@python.org] On Behalf Of Alan Gauld Sent: Wednesday, July 04, 2012 1:07 PM To: tutor@python.org Subject: Re: [Tutor] Python and boot sequences On 28/06/12 13:54, Dave Wilder wrote: Can a Python script be written that has the ability to stop a Linux device in the middle of a boot when a certain sequence occurs and then perform an action? It depends... For example, when I boot my switch (w/ Linux OS 2.7.3), I want to stop the boot when I see a series of asterisks. When I see this, I need to hit the ESCSHIFT9 Could you do it with a bash script? If so you could do it with Python. But I've no idea whether your switch will be in a state to run user processes at that point in its boot sequence. If you don't have a shell environment its going to be a whole lot harder. doing a PXE boot, ??? -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
[Tutor] Reading a csv of coordinates, trying to write a csv of bearings.
I'm Not looking for an absolute solution, but perhaps some insight
into some snippets of code, or
suggestions of where I should seek out answers to this issue.
Or where I've gone wrong below.
NOTE: currently this 'code' below reads my file and writes a file, but
what it's doing in the middle isn't what it's supposed to do (I know
that!) I just modified what I had to at least read/write, now I need
to add the atan2() stuff.
I'm currently reading 'learning python' (Lutz) for 2.6 (not 3.0!)
I must use 2.6 because that is what esri's ArcGIS 10.0 uses.
I'm Doing (1/2 way through) the non-programmers tutorial for python
2.6, hoping to 'graduate' to the
http://docs.python.org/release/2.6.8/tutorial/index.html once I get
through the non-programmers tutorial.
Objectives: read a CSV file of coordinates, and generate a
new CSV file of bearings.
Provided: A valid CSV file with coordinates (every other line blank,
if that matters)
Intended result: A valid CSV file with bearings
Logic?: one function, accepting a filename for the input, and a filename
for the output (i.e. Coord_to_bearing(infile, outfile))
Caveats:
A bearing is declination from north (meaning the angle between the
direction and north).
2 points define a line (y = mx + b). (obviously)
Supposedly atan2() (from import math) will be in the code
http://en.wikipedia.org/wiki/Atan2
Sample input:
0,1,1
1,2,2
2,3,3
3,4,3
4,1268144.125,226540.2969
5,1268463.75,226260.1563
Sample output:
OID1,OID2,ANGLE
0,1,45.0
1,2,45.0
2,3,90.0
3,4,?
4,5,?
Since I am a beginner, I re-used some code from a previous task, and
kept lots of it
commented out to use to help with future coding.
Partial Code written in pursuit of this goal (with limited success)
it's messy, and incorrect, but it's a start, maybe hurting more than
helping.
'''
The code herin was initially written to find a nmea sample file
('nmea_sample.txt) within the same folder as this .py file.
It has been modified to read the giveN CSV file of coordinates, and
generate a new CSV file of bearings.
Python will try to find the first parameter (ie, coordinates.csv) in
the same folder in which the .py file is saved.
The second parameter is the output that will be saved in the same
folder automatically.
Put another way... The new .csv file will be saved to the same folder
in which .py file is saved.
I could use user input but trying to keep it simple.
1 Define a function that will be run (he name of the file, in the
working directory so that Python
knows what file to find, and where.
2. open the file
3. line' = f.readline() establishes 'line' and reads/returns one line from f's
file, up to the end of the line.
4. read the file line by line (while loop)
5. assigning 'data' as splitting the line by commas (',')
6. Extract information by line
7. Calculating the Bearings #this step is missing
8. Creating a new CSV file.
# I left lots of comments and 'old' code in so that I could use this
as a learning experience.
'''
import math #importing math so that atan2 is available
#1 Establishing the filenames
def coord2bearing(coordFile,csvFile): # defining a function that
will be used later, a return is required at the end when function is
done.
# these two parameters will be
filled when calling this function in the python Shell.
import csv #2importing the csv module that has the necessary tools
I need to read/create a csv file.
csv_writer = csv.writer(open(csvFile, 'wb'), delimiter = ',') #
not sure if I need the ',' in here yet?.
#
creating a csv file by using csv module's tool called 'writer' and
'open';
#
'csv_writer' is just a variable name that could be anything
#
defining 'csv_writer' as an object that has attributes and tools
#
Important Note: Using 'wb' instead of 'w' caused the csv file to have
line breaks.
myfile = open (coordFile) # open the original sample data file by
using the open function (coordinates2.csv) (original coordinates.csv
had blank lines)
filename = 'coordinates.csv'
#2 The 'open()' command opens the extablished filename.
f = open(filename)
#3 applying 'line' and the f.readline statement
line = f.readline() #3
#4 read the file line by line in a 'while' loop.
OID = 0 # setting the OID varible to zero
while (line): #...while there is a line to go to...
#5 assigning 'data' as splitting the line by commas (',')
data = line.split(',') #not sure yet how the .csv is read, if
it 'sees' a comma or not...?
[Tutor] using dynamic import statements
Here is what I want to do: I have a bunch of modules to import. instead of duplicating a lot of code for each import, I want to do something like this: importList = [ sys, os, imp, stat, re, webbrowser, Image, StringIO, shutil, datetime ] for object in importList: try: if debug == ON: print Importing module %s % (object) exec( import + object) except ImportError as error: print %s %s requires the Python %s library. % ( appName, str(appVersion), object ) print An error occurred when attempting to load the library. The error was '%s'. % ( error ) exit() Everything appears to run okay, however, when the first piece of code that relies upon one of these imported modules is executed, I get an error: Traceback (most recent call last): File a.py, line 122, in module imp.load_module(object,fp,pathName,description) File ./Modules/functions.py, line 133, in module def special_match(strg, search=re.compile(r'[^a-zA-Z0-9\.\ \-\#\$\*\@\!\%\^\]').search): NameError: name 're' is not defined Is is possible to do what I am trying to do, or am I doing something wrong? Thanks ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
