Returning a value from code string
Say I have a database containing chunks of Python code. I already have a
way to easily load, edit, and save them. What is slightly baffling to me
is how I can effectively pass this code some arguments, run it, and
somehow get a return value.
(Acutally, in the process of writing this post, I figured out a pretty
good way of doing it. Here's the rest of my post and the solution I came
up with. Enjoy my thought process!)
Right now I'm thinking of just defining a standard function name, and
using that as the entry/exit point for the DB-stored code. Is this
reasonable? Is there some more Pythonic way?
I'll probably compile the code and store the bytecode to the database
(in addition to the plain text) for faster execution time, as well. It
shouldn't care whether the stored code is a string or a code object.
What if a stored chunk of code doesn't need to return anything? It would
be more convenient to just execute the code without the standard
function. Also, it'd be nice to wrap this in a class:
# module test.py
class Codeobj:
def __init__(self, code=None):
if code: self.code = code
else: self.code = ""
def __call__(self, args=None, **kwargs):
# We can pass in a name/value dictionary
if args: kwargs.update(args)
exec self.code
# We don't need a standard function if we're not returning
# anything
if locals().has_key('std_func_name'):
return std_func_name(**kwargs)
if __name__ == "__main__":
# Load code from the DB
double = """\
def std_func_name(n):
return n*2"""
addthree = """\
def std_func_name(n):
return n+3"""
noreturn = "print 'arg = %s' % kwargs['arg']"
a = Codeobj(double)
b = Codeobj(addthree)
c = Codeobj(noreturn)
# Calling with name/value dictionary
print a(args={'n':5})
# Calling with specific named argument
print b(n=4)
c(arg='foo')
# EOF
$ python test.py
10
7
arg = foo
If I wanted to simplify the 'noreturn' example (well, if there were more
lines of code this might make it simpler), I could have defined it like
this:
noreturn = """\
locals().update(kwargs)
print 'arg = %s' % arg"""
-Kirk McDonald
--
http://mail.python.org/mailman/listinfo/python-list
Re: Returning a value from code string
Steven D'Aprano wrote: > On Fri, 27 Jan 2006 20:33:53 -0800, Kirk McDonald wrote: > > >> Say I have a database containing chunks of Python code. I already >> have a way to easily load, edit, and save them. > > > Why? I am implementing an Everything Engine-like system (see everything2.com and everydevel.com) in Python. (The original is in Perl, and by all accounts is some pretty ugly code.) The Everything idiom is that everything in the engine is a 'node.' A node is a discrete unit that can be loaded from and saved to the database. A node can be asked to display itself in the page, it has an owner (and users are themselves nodes), and so on. One kind of node is a Superdoc, which is a document that mixes HTML with code. (I have simply implemented these as mod_python PSP documents, which is all hunky-dory.) Superdocs comprise essentially all of the important bits of the site. Another kind of node (I'm still deciding whether to call them Codenodes or Opcodes or maybe Pynodes) is a chunk of code that can be asked to run itself, and which can be edited, on the fly, from within the website. Thus, one can both alter the functionality of the site, and add functionality, from the site itself (so long as you have the user permissions to do so). -Kirk McDonald -- http://mail.python.org/mailman/listinfo/python-list
Re: Returning a value from code string
Fredrik Lundh wrote:
> executing the code in a custom namespace is a lot cleaner:
>
> ns = {}
> ns["os"] = os # insert "preimported" values
> ns["app"] = app # insert "preloaded" objects
> exec self.code in ns
> try:
> func = ns["std_func_name"]
> except KeyError:
> pass
> else:
> func(**kwargs)
>
> instead of using a standard function, you can let the code
> objects talk to the application via a preloaded object or an
> interface module. e.g. instead of
>
> def std_func_name(app, args):
> app.something()
>
> your code object could simply become
>
> app.something()
>
These code objects should all return an instance of a certain class (or
one of its descendants). In another branch of this topic, I described
roughly what this is for. So, without going into too much detail, I'll
just repeat that this is for a website backend. Whenever an HTML form is
submitted, a hidden "op" element (short for operation) in the form is
checked. This value will send the rest of the form data on to the code
object it represents. (If you've read my other post: These code objects
are nodes, the value for the "op" element is the ID number of one of
them, or possibly one of a few mnemonics for some of the more important
ops.)
So, taking another swipe at it with your suggestions:
def __call__(self, args=None, **kwargs):
ns = {}
ns.update(kwargs)
ns["args"] = args
ns["retval"] = None
exec self.code in ns
try:
func = ns["std_func_name"]
except KeyError:
return ns["retval"]
else:
# Already passed kwargs into the namespace
return func()
Then, a code object can either return a value by using whatever happens
to be in 'retval' when it's done executing, or by defining a
'std_func_name' function and using its return value.
So the interface for these Codenodes needs to be able to accept
arguments as a name/value dictionary corresponding to the elements in
the HTML form (the 'args' argument in my example above). This dictionary
is passed on to the code object. The kwargs are a few important
variables (not supplied by the form) that all Codenodes need to know,
such as the current user and maybe a database cursor.
Each Codenode knows how to deal with the input from its associated form.
(Or forms! There is, for example, a generalized "commit" Codenode that
knows how to update the state of *any* node if a form follows a certain
template.) When it is done, it needs to then return whichever node
should be displayed next. If the operation being attempted violates the
user's permissions, it might return the "access denied" node. (Actually,
in that case it will raise an AccessDenied exception and expect
something down the line to deal with it, so I can define what the
"access denied" node is in a central place, but you get the idea.) If
the operation resulted in a new node being created, it might return the
new node. &c.
Sorry if I tend to ramble, heh. :-)
-Kirk McDonald
--
http://mail.python.org/mailman/listinfo/python-list
Re: MYSql, CGI web page search code not working
Fred wrote:
> No matter what I type in the form text box (or even if I leave it
> blank) I get all the records.
Try this:
#!/usr/local/bin/python
print "Content-Type: text/html\n"
import MySQLdb
import cgi
db=MySQLdb.connect(host = 'localhost', db = 'phone')
cursor=db.cursor()
cursor.execute("Select * from phone where name=%s order by name", (name,))
result = cursor.fetchall()
for record in result:
print ''
print record[0]
print '--'
print record[1]
print '--'
print record[2]
print '--'
print record[3]
print ''
(Assuming the name of your text field is "name".)
--
http://mail.python.org/mailman/listinfo/python-list
Re: MYSql, CGI web page search code not working
Fred wrote:
> Yeah, I already tried that (except you have a , after name.
>
> Your code produces the same error:
>
> NameError: name 'name' is not defined
>
> I know I am close!! Just missing some small thing...
>
Oh, duh. I forgot something:
#!/usr/local/bin/python
print "Content-Type: text/html\n"
import MySQLdb
import cgi
form = cgi.FieldStorage()
db=MySQLdb.connect(host = 'localhost', db = 'phone')
cursor=db.cursor()
cursor.execute("Select * from phone where name=%s order by name",
(form['name'].value,))
result = cursor.fetchall()
for record in result:
print ''
print record[0]
print '--'
print record[1]
print '--'
print record[2]
print '--'
print record[3]
print ''
The comma is intentional: the MySQLdb wants the argument(s) as a tuple.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Returning a value from code string
Max wrote: > Kirk McDonald wrote: > >> Another kind of node (I'm still deciding >> whether to call them Codenodes or Opcodes or maybe Pynodes) is a chunk >> of code that can be asked to run itself, and which can be edited, on >> the fly, from within the website. Thus, one can both alter the >> functionality of the site, and add functionality, from the site itself >> (so long as you have the user permissions to do so). >> > > As Steven said, "U R pwn3d". 1f you d0n't sp3a|< l33t (if you don't > speak leet), that means you are screaming "hack me, use me to launch > attacks on other computers, and than attack my computer". Unless you > have some revolutionary ideas in code-security analysis. In which case > you can a lot more money than from implementing Everything2 in python. > > --Max Heavens! Normal users can't edit code! They won't even see it! I'm not a *total* moron. The only thing users will be able to enter is some simplified HTML. This is a convenience feature for the (trusted) admins of the site. There are some simple permission features built into the API. Every database-altering API call takes the current user as an argument, and if they're not allowed, it tells them to get bent. Everything2 does this more or less the same way, and they've had few issues in the seven or so years they've been operating. -Kirk McDonald -- http://mail.python.org/mailman/listinfo/python-list
Re: MYSql, CGI web page search code not working
Fred wrote:
> OK one more... how would I do a "LIKE" instead of a = in this code?
>
> cursor.execute("Select * from phone where name=%s order by name",
> (form['name'].value,))
>
> Right off I think:
>
> cursor.execute("Select * from phone where name like %%s% order by
> name",
> (form['name'].value,))
>
> But it blows up...
>
This should work:
cursor.execute("Select * from phone where name like %s order by name",
('%'+form['name'].value+'%',))
-Kirk McDonald
--
http://mail.python.org/mailman/listinfo/python-list
Re: MYSql, CGI web page search code not working
Dennis Lee Bieber wrote: > On Sat, 28 Jan 2006 10:14:44 -0800, Kirk McDonald <[EMAIL PROTECTED]> > declaimed the following in comp.lang.python: > > > >>The comma is intentional: the MySQLdb wants the argument(s) as a tuple. > > > The DB-API wants tuples... But my last perusal of the MySQLdb Python > code showed that it would work with naked singletons... Ah! So it does. However, I still pass 'em as a tuple as a matter of course, since it's documented that way. *shrug* (Also, it saves that many keystrokes if I need to add arguments.) -Kirk McDonald -- http://mail.python.org/mailman/listinfo/python-list
Re: String Manipulation Help!
Dave wrote:
> OK, I'm stumped.
>
> I'm trying to find newline characters (\n, specifically) that are NOT
> in comments.
>
> So, for example (where "<-" = a newline character):
> ==
> 1: <-
> 2: /*<-
> 3: --<-
> 4: comment<-
> 5: --<-
> 6: */<-
> 7: <-
> 8: CODE CODE CODE<-
> 9: <-
> ==
[snip]
Well, I'm sure there is some regex that'll do it, but here's a stupid
iterative solution:
def newlines(s):
nl = []
inComment = False
for i in xrange(len(s)):
if s[i:i+2] == '/*':
inComment = True
if s[i:i+2] == '*/':
inComment = False
if inComment: continue
if s[i] == '\n':
nl.append(i)
return tuple(nl)
Your example returns:
(0, 64, 65, 80, 81)
This probably isn't as fast as a regex, but at least it works.
-Kirk McDonald
--
http://mail.python.org/mailman/listinfo/python-list
Re: Decoupling the version of the file from the name of the module.
[EMAIL PROTECTED] wrote: > I'm a newbie experimenting with Python. I want to incrementally develop > a module called 'circle'. The problem is now that the file name is used > for two purposes. To keep track of the version number and as the name > for the module. So when I develop the first version of my file I have > to call it circle_a.py. The name of the module then automatically > becomes circle_a. But when I develop the next increment and call my > file circle_b.py the module name changes as well. > > Basically I want to decouple the version of my file from the name of > the module. > > Is there a *simple* way out of this dilemma. > I would recommend just naming the file circle.py, and defining something like a variable named __version__ or maybe __revision__ at the top of the module. Then you can, I don't know, back up your old versions to other filenames or something. Or, if you really want to do this right, you could install Subversion. :-) -Kirk McDonald -- http://mail.python.org/mailman/listinfo/python-list
Re: Decoupling the version of the file from the name of the module.
[EMAIL PROTECTED] wrote: > Now suppose I have make a new version with __version__ = 1.1. What > shall I call this file and (I don't want to overwrite the old file if I > need to go back to it) how do I import it from the shell. Your advice > sounds nice, but I would appreciate if you could give me (or point me > to) a simple example. > > Thanks > Before you make a new version, rename circle.py to, e.g., circle-1.0.py, and then create the new version as circle.py. Then you can access the new version just like you accessed the old. If you make yet another new version, then rename the current circle.py as circle-1.1.py, and lather, rinse, repeat. However, I'd still look into a version control system like Subversion. It can do all of this for you. -Kirk McDonald -- http://mail.python.org/mailman/listinfo/python-list
Re: Returning a value from code string
Peter Hansen wrote: > In the specific case in question, Kirk's first post gave little or no > hint about how much he knew about security issues and, by asking "why?", > with a single word Steven hoped to learn enough to say things like "oh, > there's a package designed to do that which you can just plug in" or > "oh, it's clear you are totally new to issues of network security and > are about to put your foot right in it, so here's some background to > save you from yourself", or whatever... Heh heh. For what it's worth, I did have a perfectly good reason for trying to do what might otherwise be a suicidally insane idea. On the other hand, I've been re-evaluating my needs, and I'll probably be hardcoding the more important bits of code that I was going to store in the database (in that pulling these things from the database whenever I need to use them is remarkably slow compared to letting mod_python cache them, and these particular functions are unlikely to ever change). If I do need a way to have dynamic code in the database, I've added a thing that just reflects POST data back to the database-stored PSP document in which the form was found (if the form wants it to), letting the PSP document deal with it. That should be adequate. In short, I learned something about exec and namespaces that I'm not actually going to use. Hooray! -Kirk McDonald -- http://mail.python.org/mailman/listinfo/python-list
Re: Storing lines from a text file
[EMAIL PROTECTED] wrote: > so this: > a, b, c, d, e =f.readlines() > > ..this will put the first line in a, second in b, etc? How do I > accomplish this when I'm not sure how many lines there are going to be > every time? Thanks. > With a list: http://python.org/doc/2.4.2/tut/node5.html#SECTION005140000 -Kirk McDonald -- http://mail.python.org/mailman/listinfo/python-list
Re: Storing lines from a text file
[EMAIL PROTECTED] wrote:
> With what kind of list? I don't see how I can do it with a list unless
> I create one indefinate list and use the objects in the indefinate list
> for the names of the lists to hold the lines of text. Is that how you
> are suggesting that I do it?
>
You're thinking too hard. Say you want to read in all the lines from the
file object f and just print them out one at a time:
lines = f.getlines()
for line in lines:
print line
Simple.
-Kirk McDonald
--
http://mail.python.org/mailman/listinfo/python-list
Re: Storing lines from a text file
[EMAIL PROTECTED] wrote:
> I keep getting an error when I try to use what you said Mr. McDonald. I
> think I've got something wrong, but take a look if you can.
>
> log = open('C:\log_0.txt')
> lines = log.getlines()
> for line in lines:
> print line
>
> When I debug it the error I get is the following:
>AttributeError: 'file' object has no attribute 'getlines'
>
D'oh! That's because the method is readlines(). Stupid brain:
log = open('C:\log_0.txt')
lines = log.readlines()
for line in lines:
print line
-Kirk McDonald
--
http://mail.python.org/mailman/listinfo/python-list
URL Character Decoding
If you have a link such as, e.g.: Main menu! The space will be translated to the character code '%20' when you later retrieve the GET data. Not knowing if there was a library function that would convert these back to their actual characters, I've written the following: import re def sub_func(m): return chr(int(m.group()[1:], 16)) def parse_title(title): p = re.compile(r'%[0-9][0-9]') return re.sub(p, sub_func, title) (I know I could probably use a lambda function instead of sub_func, but I come to Python via C++ and am still not entirely used to them. This is clearer to me, at least.) I guess what I'm asking is: Is there a library function (in Python or mod_python) that knows how to do this? Or, failing that, is there a different regex I could use to get rid of the substitution function? -Kirk McDonald -- http://mail.python.org/mailman/listinfo/python-list
Re: URL Character Decoding
Kirk McDonald wrote: > If you have a link such as, e.g.: > > Main menu! > > The space will be translated to the character code '%20' when you later > retrieve the GET data. Not knowing if there was a library function that > would convert these back to their actual characters, I've written the > following: > > import re > > def sub_func(m): > return chr(int(m.group()[1:], 16)) > > def parse_title(title): > p = re.compile(r'%[0-9][0-9]') > return re.sub(p, sub_func, title) > > (I know I could probably use a lambda function instead of sub_func, but > I come to Python via C++ and am still not entirely used to them. This is > clearer to me, at least.) > > I guess what I'm asking is: Is there a library function (in Python or > mod_python) that knows how to do this? Or, failing that, is there a > different regex I could use to get rid of the substitution function? > > -Kirk McDonald Actually, I just noticed this doesn't really work at all. The URL character codes are in hex, so not only does the regex not match what it should, but sub_func fails miserably. See why I wanted a library function? -Kirk McDonald -- http://mail.python.org/mailman/listinfo/python-list
Re: URL Character Decoding
Kirk McDonald wrote: > Actually, I just noticed this doesn't really work at all. The URL > character codes are in hex, so not only does the regex not match what it > should, but sub_func fails miserably. See why I wanted a library function? > > -Kirk McDonald Not to keep talking to myself, but looks like sub_func works fine, and the regex just needs to be r'%[0-9a-fA-F][0-9a-fA-F]'. But even so. -Kirk McDonald -- http://mail.python.org/mailman/listinfo/python-list
Re: URL Character Decoding
Paul McGuire wrote:
> "Kirk McDonald" <[EMAIL PROTECTED]> wrote in message
> news:[EMAIL PROTECTED]
>
>>If you have a link such as, e.g.:
>>
>>Main menu!
>>
>>The space will be translated to the character code '%20' when you later
>>retrieve the GET data.
>>
>>I guess what I'm asking is: Is there a library function (in Python or
>>mod_python) that knows how to do this? Or, failing that, is there a
>>different regex I could use to get rid of the substitution function?
>>
>>-Kirk McDonald
>
>
>
>>>>import urllib
>>>>urllib.quote("index.py?title=Main Menu")
>
> 'index.py%3Ftitle%3DMain%20Menu'
>
>>>>urllib.unquote("index.py%3Ftitle%3DMain%20Menu")
>
> 'index.py?title=Main Menu'
>
>
Perfect! Thanks.
-Kirk McDonald
--
http://mail.python.org/mailman/listinfo/python-list
Introspection with classes
Me again! Same project as before, if you're playing along at home.
The Everything Engine model (which I am blatantly copying in my project)
is that everything in the engine is a 'node,' and that every node can be
stored to and read from the database. Since this is the backend for a
website, minimizing hits on the database is a worthy goal. Thus, it is
paramount that reading in any given subclass of the base 'Node' class
take (optimally) one query. (A simple, yet fairly simple-minded design
-- and my first stab at the problem -- would be to have each subclass
read in its own data after calling its parent's -- or parents'! -- read
method(s). This has the disadvantage of cluttering a lot of code with
database queries and making it quite difficult to create new subclasses.)
Here's a rough sketch of what I'm doing at the moment. I've invested
enough time into it at this point that I've totally lost track of
whether it's a good idea, and I would please like someone to tell me.
Every node has certain attributes. In this outline, let's say these are
the node's unique id number, its title, and the id number of its owner.
class Node:
dbInfo = { 'title': ['node', 'title',""]
'owner_id' : ['node', 'owner_id', None] }
def __init__(self):
self.default()
def default(self):
self.node_id = None
for k, v in self.dbInfo.items():
setattr(self, k, v[2])
def read(self, db, node_id):
# construct a database query from the table/column info in
# dbInfo
def commit(self, db):
# write the current values to the db
def insert(self, db):
# insert a new node using the current values
def nuke(self, db):
# remove the node from the db, then:
self.default()
So, simple enough. (The real version has signifigantly more stuff going
on, as well as a permission system, but I have elided these for the sake
of clarity.) By doing that dance with dbInfo, I can easily subclass Node.
Say we want a new node that holds some text. We just need to add a new
attribute like so:
class Document(Node):
dbInfo = {}
dbInfo.update(Node.dbInfo)
dbInfo['content'] = ['docs', 'text', ""]
... and that's it. That's the whole Document class. (Well, the real
version has a render method that takes the content, parses it in
interesting ways, and writes it to the client, but that's simple, too.)
The shortness of this class is why this strikes me as a good idea.
The other advantage is I can make use of multiple inheritance. First,
say we have a class that just holds a dictionary:
class DictNode(Node):
dbInfo = {}
dbInfo.update(Node.dbInfo)
dbInfo['stuff'] = ['dictnode', 'dict', {}]
(Assume a mechanism is provided to automatically pickle the dictionary,
which there is.)
Now say I want a nodetype that's just a document with a dictionary
attached. This is almost exactly how the User class is implemented: the
document holds the user's "homenode," which is just a brief bio or
whatever the user wants to put in there, and the dictionary holds the
user's settings. They snap together like Legos. It looks just like this:
class User(Document, DictNode):
dbInfo = {}
dbInfo.update(Document.dbInfo)
dbInfo.update(DictNode.dbInfo)
If the User class wanted its own attributes (say, a password), they
would just get added to the end.
This is my third stab at the problem (the first was mentioned above, the
second was basically the same as this but dbInfo was an instance
variable instead of a class variable which, uh, was sort of dumb).
Coming from C++, I'm still wrapping my brain around Python's
introspection abilties. Can I generalize this more? Am I missing some
idiom or language feature that would benefit me? Where's my time machine?
-Kirk McDonald
--
http://mail.python.org/mailman/listinfo/python-list
Re: OO conventions
Daniel Nogradi wrote:
> I'm relatively new to object oriented programming, so get confused
> about its usage once in a while. Suppose there is a class Image that
> has a number of methods, rotate, open, verify, read, close, etc. Then
> to use this class my natural guess would be to have something like
>
> image = Image( )
> image.read( "myfile.jpg" )
> image.rotate( )
> image.close( )
>
> But now it turns out that the PIL module uses this as
>
> image = Image.open( "myfile.jpg" )
> image.verify( )
> image.rotate( )
> image.close( )
>
> Perhaps the real Image class of PIL doesn't have these methods
> exactly, but doesn't matter, my point is the way it works. Is it
> normal that instead of first creating an instance of a class, it
> starts right away with one its methods? I of course understand that
> the PIL people simply made a choice that their module works this way,
> period, but I'm just wondering if it wouldn't have been more "logical"
> to do it along the way of my first example.
>
> I guess it's just a matter of convention or how the programmer feels
> like, but there are no conventions of this type? Which would be more
> pythonic? Or I shouldn't worry and it's totally up to the developer?
If I were coding a class like that, I'd probably do it like this:
image = Image("myfile.jpg")
image.rotate()
image.close()
And, in addition, there would be an image.open() method. In the above,
__init__ just calls it on its own when the filename is specified.
(Though, of course, I'm not at all familiar with the PIL module; there
might be more stuff going on behind the scenes with the way they do it.)
-Kirk McDonald
--
http://mail.python.org/mailman/listinfo/python-list
Re: cgi - secure sessions
[EMAIL PROTECTED] wrote: > Hey, > > I was just wondering if / how would it be possible to create secure > sessions for a website using Python CGI... I thought of using cookies, > and things looked promising for a while; I could login through a form > which pointed to a cgi script which created sent the user cookies, but > I found that when a script to detect the cookies was run through a > server side include line in the html, it couldn't get any cookies, but > it would work fine when run directly through the browser (which is > useless to me). > > If anybody could help with this it would be great. Python is the only > programming language that I'm relatively comfortable in at the moment, > so using the usual PHP or Javascript just isn't an option for me > unfortunately. > > GazaM > For what it's worth, mod_python supports sessions: http://www.modpython.org/live/current/doc-html/pyapi-sess.html I've been playing with them recently, and they seem to work. :-) -Kirk McDonald -- http://mail.python.org/mailman/listinfo/python-list
Re: cgi - secure sessions
GazaM wrote: > What I have is a cgi script run through a server > side include line in the html, which looks for the session cookie, if > it is present will say 'logged in as "user"' and if the cookie isn't > there will display a login form. Now, the big showstopper here is that, > because session cookies are stored in http headers sent by the client > (afaik) the cgi script can't get any, because the http headers are > passed onto the html file and any cgi scripts inside don't get > anything... is there a workaround possible? Python has a built-in Cookie module: http://www.python.org/doc/2.4.2/lib/module-Cookie.html It may simplify matters. -Kirk McDonald -- http://mail.python.org/mailman/listinfo/python-list
Re: nDimensional sparse histogram in python.
KraftDiner wrote: > The dictionary is sorted by the point key. > Is there a way to sort the dictionary by the value? > Seems to me this goes against the purpose of a dictionary but > thats what I need to do.. > Well, it's probably not all that efficient, but it is simple code: sortedList = [(v, k) for k, v in self.histo.items()] sortedList.sort() Should do it... -Kirk McDonald -- http://mail.python.org/mailman/listinfo/python-list
Re: nDimensional sparse histogram in python.
KraftDiner wrote:
> Ok so this is nice.. Just one thing.. When you try to get a value from
> a dictionary
> and it isn't found in the dictionary things go bad...
>
> Take this for example:
>
> class histogram(object):
> def __init__(self):
> self.histo = {}
>
> def update(self, point):
> if self.histo.get(point) != None:
> self.histo[point] = self.histo[point] + 1
> else:
> self.histo[point] = 1
>
> def get(self, point):
> return self.histo[point]
>
>
> hist = histogram()
> hist.update((0,0,0))
> hist.update((0,0,1))
> hist.update((0,0,1))
> hist.get((0,0,0))
> hist.get((0,0,1))
> hist.get((0,0,2))
>
> spews out this error:
>
> Traceback (most recent call last):
> File "histogram.py", line 21, in ?
> hist.get((0,0,2))
> File "histogram.py", line 12, in get
> return self.histo[point]
> KeyError: (0, 0, 2)
>
Try this:
class histogram(object):
def __init__(self):
self.histo = {}
def update(self, point):
self.histo[point] = self.histo.get(point, 0) + 1
def get(self, point):
return self.histo.get(point, 0)
dict.get's default return value is your friend.
-Kirk McDonald
--
http://mail.python.org/mailman/listinfo/python-list
Re: nDimensional sparse histogram in python.
KraftDiner wrote: > Many thanks everyone. > > One last quick question... > The dictionary, I believe, is sorted by the key. > Is there a way to sort it by the value? > Say I wanted to put out a list of the frequency sorted by highest to > lowest? > The dictionary itself is actually unordered; a C++ std::map this ain't. To sort its elements, you need to build a list of the items and sort that, e.g.: items = [(v, k) for k, v in self.histo.items()] items.sort() This will give you a list of (value, key) tuples sorted by value. -Kirk McDonald -- http://mail.python.org/mailman/listinfo/python-list
Re: Automatic class attribute
Franck PEREZ wrote: > Hello all, > > Considering the following code : > > class C(object): >...: observers = [] >...: >...: @classmethod >...: def showObservers(cls): >...: print cls.observers > > class D1(C): >...: observers = [] #could it be moved in C ? > > class D2(C): >...: observers = [] #could it be moved in C ? > > I want each child class of C to have it's own "observers" class attribute. > > The code I provided works... but I'd like to avoid typing "observers = > []" in each child class. > > Is it possible to define something in C which would basically mean : > "for each child class, automatically bind a new list attribute called > observers" ? > > Are metaclasses a way ? Is it possible to avoid them ? > Thanks a lot, > Franck By an astounding coincidence, I was just working on a similar problem. Metaclasses can do this, no problem: class M(type): def __init__(cls, name, bases, dict): cls.observers = [] def showObservers(cls): print cls.observers class C(object): __metaclass__ = M class D1(C): pass class D2(C): pass -Kirk McDonald -- http://mail.python.org/mailman/listinfo/python-list
Re: Global variables, Classes, inheritance
DaveM wrote:
> Although I've programmed for fun - on and off - since the mid 70's, I'm
> definitely an OO (and specifically Python) beginner.
>
> My first question is about global variables. Are they, as I'm starting to
> suspect, a sin against God or just best avoided? Having got my current
> application working using them, I'm not sure whether I want to refactor it,
> but equally, I'd like to write the best code I can.
Python globals are actually remarkably safe, at least compared to some
other languages. The key is that, when you import a module, its globals
stay in its namespace. I've got nothing against them.
>
> Secondly (and this is related), if you have multiple instances of the same
> class, but need them all to have access to a dictionary created at run-time,
> is there a class way to this without calling the creation method multiple
> times? The best thought I've come up with is for the class instance to check
> its name and only if it's the first one do the dictionary creation, but I
> haven't tried it yet.
A class attribute would do this easily:
class Foo(object):
d = {}
a = Foo()
b = Foo()
Foo.d = { 'a' : 'apple' }
print a.d
print b.d
That will print out {'a':'apple'} twice.
>
> My third difficulty is with variables in a class. What's the difference
> between the following?:
>
> class Foo:
> i = 12345
> ...
>
i is a class attribute. It is bound to the class (which is, itself, an
object). You can access i both from the class itself as well as from an
instance of the class, i.e., Foo.i and Foo().i will both work.
> class Foo:
> self.i = 12345
> ...
>
This does not work. There is no name 'self' bound to class Foo.
> class Foo:
> def __init__(self):
> self.i = 12345
> ...
>
This binds the name 'i' to all instances of class Foo; if this were C++,
we'd call self.i a member variable.
> class Foo:
> def __init(self):
> i = 12345
> ...
>
(You forgot the trailing __, but whatever. :-) This binds the name 'i'
to the local namespace of the __init__ function. It is just like any
other local variable in a function.
Namespaces in Python are really great. It is worth reading up on
globals() and locals() if you don't get them yet.
-Kirk McDonald
--
http://mail.python.org/mailman/listinfo/python-list
Re: python's library support
Sean wrote: > I am a newbie in python, and I have a feeling that python provides less > library support than perl www.cpan.org This seems a big discussion > topic. > > I want to know if there is extensive algorithm library support in > python. I know there is a pretty neat module in perl to implement graph > theory. Is there a graph theory module in python? > > Thanks, > Python's standard library is actually quite extensive; it's one of the language's greatest strengths: http://www.python.org/doc/2.4.2/lib/lib.html However, it is perhaps true that it is lacking in the raw algorithm department. On the subject of graphs, a quick Google search brings up this nice-looking library: https://networkx.lanl.gov/ For computationally intensive array manipulation, there's always good ol' numarray: http://www.stsci.edu/resources/software_hardware/numarray The standard library can do quite a lot, and if it fails you, there's more than likely a third-party library available. -Kirk McDonald -- http://mail.python.org/mailman/listinfo/python-list
Re: UnboundMethodType and MethodType
Schüle Daniel wrote: > Hello all, > > >>> class Q: > ... def bar(self): > ... pass > ... > >>> import types > >>> types.UnboundMethodType is types.MethodType > True > >>> > >>> type(Q.bar) > > >>> > >>> q = Q() > >>> type(q.bar) > > >>> > >>> type(q.bar) is types.UnboundMethodType > True > >>> q.bar > > > >>> > > I think is not very consistent > notice q.bar is bounded although type(q.bar) > says it's types.UnboundedMethodType > what do you think? > > Regard, Daniel > I think it's perfectly consistent: >>> class B(object): ... def bar(self): pass ... >>> B.bar >>> type(B.bar) >>> b = B() >>> b.bar > >>> type(b.bar) >>> id(B.bar) -1211888788 >>> id(b.bar) -1211888788 It's the same function, whether it's bound or not. Thus, it should always have the same type. It's simply called in different ways. You can just as easily say: >>> B.bar(b) As: >>> b.bar() -Kirk McDonald -- http://mail.python.org/mailman/listinfo/python-list
Re: UnboundMethodType and MethodType
Scott David Daniels wrote: > To elaborate on this, once 'id' is called, you drop the reference. > This allows quite surprising things like: > >>> id(7**8) == id(8**7) > True > >>> a, b = 7**8, 8**7 > >>> id(a) == id(b) # this time there are other references to a and b > False > > If you wanted to test the original code for identity match: > >>> B.bar is B().bar > False > is the appropriate test (the 'is' test holds the identities through > the comparison). > > By the by, this is tricky stuff, nobody should expect to understand > it thoroughly without both study and testing. > > --Scott David Daniels > [EMAIL PROTECTED] You know what? That makes perfect sense. Thank you. -Kirk McDonald -- http://mail.python.org/mailman/listinfo/python-list
Re: calculating on matrix indices
Brian Blais wrote: > Hello, > > In my attempt to learn python, migrating from matlab, I have the > following problem. Here is what I want to do, (with the wrong syntax): > > from numpy import * > > t=arange(0,20,.1) > x=zeros(len(t),'f') > > idx=(t>5) > tau=5 > x[idx]=exp(-t[idx]/tau) # <---this line is wrong (gives a TypeError) > > #-- > > what is the best way to replace the wrong line with something that > works: replace all of the values of x at the indices idx with > exp(-t/tau) for values of t at indices idx? > > I do this all the time in matlab scripts, but I don't know that the > pythonic preferred method is. > > > > thanks, > > bb > > You're specifying the type of x but not of t. You need to change the line where you assign t to: t = arange(0, 20, .1, 'f') I'm not sure why it doesn't figure that own on its own (since it obviously does hold floats), but this does cause it to work. -Kirk McDonald -- http://mail.python.org/mailman/listinfo/python-list
Re: embedding python in HTML
John Salerno wrote: > bruno at modulix wrote: > >> You've got to understand that Python is *not* a 'ServerPage' language >> (-> php, asp, jsp etc) in itself. Your server can now run python, fine, >> but *how* ? CGI ? FastCGI ? mod_python ? other ? (hint: it's probably >> just plain old CGI...) > > > So does that mean I need to have something further on the server? Or is > this something I can do on my end? How do I find out what I need? If you really want to use Python as a server page language, mod_python has support for Python Server Pages via its PSP handler: Python Server Pages: http://modpython.org/live/current/doc-html/pyapi-psp.html PSP handler: http://modpython.org/live/current/doc-html/hand-psp.html This of course means your server needs to have mod_python installed and configured. (Consult your server administrator.) However, I've always found PSP to be somewhat fiddly, and mixing any serious code with the HTML text is hardly pretty. A more common (and bare-metal) approach is CGI. In CGI, a request for a page runs a script, the output of which is the HTML page. I think this only requires that the server has Python installed, which you have said is the case. Python has signifigant standard library support for writing CGI. You should examine Python's standard cgi module: http://python.org/doc/2.4.2/lib/module-cgi.html That page also has some nice examples to get you started. And maybe its Cookie module, if you ever feel like messing with cookies: http://python.org/doc/2.4.2/lib/module-Cookie.html Slightly less bare-metal is using mod_python directly (rather than via its PSP module). This is probably preferable to plain CGI if mod_python is available, as it caches scripts as long as they are not changed. This is faster than reading them off the disk every time. By and large, mod_python's API replaces (or at least wraps) the standard library's CGI support if you go this route. Again, this is only available if your server has mod_python installed, which may or may not be the case. -- http://mail.python.org/mailman/listinfo/python-list
Re: 2-dimensional data structures
anthonyberet wrote: > Thanks for the advice (to everyone in the thread). > I think I will go with nested lists. > However, I am running into a conceptual problem. > My approach will be firstly to remove all the impossible digits for a > square by searching the row and column for other occurances. > > However, I wondering how to approach the search of the nine regions of > the grid. I am thinking of producing another nested list, again 9x9 to > store the contents of each region, and to update this after each pass > through -and update of- the main grid (row and column). > > I am not sure how to most efficiently identify which region any given > square on the grid is actually in - any thoughts, for those that have > done this? - I don't want a massive list of IF conditionals if I can > avoid it. > When I wrote my Sudoku solver (a terribly inefficient and naive recursive algorithm that I originally wrote in C++ and was the first thing I ever wrote in Python), I used numarray. It allows two-dimensional slicing: def inBlock(x, y, val): blockX = x/size * size blockY = y/size * size return val in sudoku[blockX:blockX+size, blockY:blockY+size] 'size' in this example is a global variable equal to the size of the puzzle, so 3 for a normal puzzle. 'sudoku' is a global two-dimensional array simply holding the values in the puzzle. (There are any number of things in this can be improved, such as using the // floor division operator rather than /, not using global variables, and so on. What can I say? It was a straight conversion from C++. I hardly knew what "Pythonic" meant.) -Kirk McDonald -- http://mail.python.org/mailman/listinfo/python-list
Re: setting file permissions on a web server
Daniel Nogradi wrote:
> I have next to zero experience with windows but as far as I know
> windows doesn't have file permissions at all (anyone, please correct
> me if I'm wrong :)) so in windows land it doesn't make any sense to
> "change file permissions". Even if it has some sort of a notion of
> file permissions I wouldn't know how that gets translated into unix.
This is getting a little off-topic, I admit, but the NT-derived versions
of Windows do indeed have file permissions. The Windows-equivalent of
chmod (and chown) is cacls ("Change Access Control Lists"):
http://www.ss64.com/nt/cacls.html
In essence, each file has an owner and a list of other users and groups,
who may each have "Full control" (meaning they can change permissions),
read-only, write-only, read-write, or no access.
Windows 95 and its descendants don't have permissions any more than DOS
does. (Except with respect to Windows network file-sharing.)
(Heh, I checked just before posting this and someone beat me to it.
Here's my post anyway.) :-)
-Kirk McDonald
--
http://mail.python.org/mailman/listinfo/python-list
TypeCheck vs IsInstance in C API
I'm examining the C API, and I have a question. http://docs.python.org/api/object.html There are two functions that appear to do nearly same thing, and I just want to be certain I'm not missing something. First is PyObject_IsInstance: int PyObject_IsInstance(PyObject *inst, PyObject *cls); Returns 1 if inst is an instance of the class cls or a subclass of cls, or 0 if not... Second is PyObject_TypeCheck: int PyObject_TypeCheck(PyObject *o, PyTypeObject *type); Return true if the object o is of type type or a subtype of type... Now, I can see that IsInstance can take a tuple as the second argument and check the type of the first argument against every item in the tuple. I also see that TypeCheck was added in version 2.2. Why was it added? Its functionality already seems covered by IsInstance. Is it a new-style vs. old-style class thing? -Kirk McDonald -- http://mail.python.org/mailman/listinfo/python-list
Re: Question on try/except
Dan wrote: > While perusing sre.py in the standard library, I came upon this snippet > of code in the _compile() function definition: > > try: > p = sre_compile.compile(pattern, flags) > except error, v: > raise error, v # invalid expression > > Is there some particular use in catching an exception and immediately > re-raising it? Why catch it at all? > > /Dan > All I can think of is that it changes the traceback to point to the re-raise and not the original raise. -Kirk McDonald -- http://mail.python.org/mailman/listinfo/python-list
Re: what is the keyword "is" for?
daniel wrote: > I'm so confused by the keyword "is" and "==" equal sign, it seems they > could be exchanged in some contexts, but not in others, what's the > difference between them in terms of comparation? > > thanks... > > daniel > 'is' compares object identity. == compares values. >>> a = [1, 2, 3] >>> b = [1, 2, 3] >>> a is b False >>> a == b True In this example, a and b refer to two separate lists that happen to hold the same values. Thus, 'is' returns False, and == returns True. On the other hand: >>> c = d = [4, 5, 6] >>> c is d True >>> c == d True Here, c and d refer to the same list. Therefore, 'is' returns true (they both refer to the same object), as does == (an object is always equal to itself, unless you overload the equality check in a weird way). The distinction can easily be seen if we try to mutate these lists: >>> a.append(4) >>> a is b False >>> a == b False >>> c.append(7) >>> c is d True >>> c == d True When we mutate a, b is not affected. They are two different lists, and changing 'a' makes it so they are no longer equal. When we mutate c, d IS affected; they refer to the same list. You can easily confuse yourself if you ever talk about applying 'is' to (for example) integers. Python may re-use certain small integers when you might not expect it to; this is done in the interests of efficiency. If you only compare the /values/ of numbers (with ==), then you will never notice this. >>> a = 1 >>> b = 1 >>> c = 100 >>> d = 100 >>> a is b True >>> c is d False -Kirk McDonald -- http://mail.python.org/mailman/listinfo/python-list
API functions not working as expected
... for reasons that are obvious in retrospect. Specifically, I am talking about the PyNumber_InPlace* family of functions. For example, the docs for InPlaceAdd say: PyObject* PyNumber_InPlaceAdd(PyObject *o1, PyObject *o2) Return value: New reference. Returns the result of adding o1 and o2, or NULL on failure. The operation is done in-place when o1 supports it. This is the equivalent of the Python statement "o1 += o2". But, of course, numbers are immutable. None of them support in-place addition. This is not the same as o1 += o2, as o1 is not actually changed when using this function. Am I missing something here? Is there, in fact, no point to these InPlace* functions? -Kirk McDonald -- http://mail.python.org/mailman/listinfo/python-list
Turning a callback function into a generator
Let's say I have a function that takes a callback function as a parameter, and uses it to describe an iteration: def func(callback): for i in [1, 2, 3, 4, 5]: callback(i) For the sake of argument, assume the iteration is something more interesting than this which relies on the callback mechanism. The function is an existing interface, and I cannot change it. I want to somehow, in some way, provide an iteration interface to this function. Thoughts? -Kirk McDonald -- http://mail.python.org/mailman/listinfo/python-list
Re: Turning a callback function into a generator
[EMAIL PROTECTED] wrote: > Peter Otten wrote: > >>Kirk McDonald wrote: >> >> >>>Let's say I have a function that takes a callback function as a >>>parameter, and uses it to describe an iteration: >>> >>>def func(callback): >>> for i in [1, 2, 3, 4, 5]: >>> callback(i) >>> > > > Which object is immutable? the callback or the function? If its the > callback then > > def func(callback): >for i in numbers: > yield callback(i) > > If the function is immutable, then its a bit harder. The callback has > to be able to do the processing. You can't use an iterator here > because call stack gets in the way. You could store the information > being passed to the callback in a list, then iterate over the list > afterwards. Or you could have the callback be able to handle all of > the work at once. > > What do you intend to use this for? Python has a lot of options and > you may not be using the best one for the problem > It is the function that is immutable. I am writing a library for the D programming language that is not totally unlike Boost.Python: http://dsource.org/projects/pyd/wiki It is still in the fairly early stages, although the basic function and class wrapping do work. This particular functionality is required to wrap D's basic iteration protocol, the opApply function: http://www.digitalmars.com/d/statement.html#foreach opApply works using a callback, as described. Because D does not (yet) have anything like Python's "yield", wrapping it with Python's iteration interface is turning out to be dreadfully annoying. -Kirk McDonald -- http://mail.python.org/mailman/listinfo/python-list
Re: Turning a callback function into a generator
Alex Martelli wrote: > Lawrence D'Oliveiro <[EMAIL PROTECTED]> wrote: > > >>In article <[EMAIL PROTECTED]>, >> Kirk McDonald <[EMAIL PROTECTED]> wrote: >> >> >>>I want to somehow, in some way, provide an iteration interface to this >>>function. Thoughts? >> >>Run it in a separate thread/process? > > > Sounds best to me. Specifically, given (e.g.) something like the OP's > > def func(callback): > for i in [1, 2, 3, 4, 5]: > callback(i) > > we might have a wrapper such as: > > import thread > import Queue > > def wrap_cb_into_gen(func): > q = Queue.Queue(1) # maximum size of 1 > def callback(item): > q.put(item) > all_done_sentinel = object() > def thread_skeleton(): > func(callback) > q.put(all_done_sentinel) > thread.start_new_thread(thread_skeleton, ()) > while True: > item = q.get() > if item is all_done_sentinel: break > yield item > > > Of course, there are lighter-weight options than a length-1 Queue for > the purpose of synchronizing these two threads, but I always tend to > prefer Queue (for its simplicity, generality, solidity) to other > synchronization structures and architectures, unless there is some very > strong reason to do otherwise in a specific case. Also, I normally use > module threading rather than the lower-level module thread -- here, > clearly, either one will do just fine. > > > Alex Threads are probably a weak point of mine. I have little experience with them, and so I am inclined to paranoia while using them. What implications does this have for "func"? In my case, it is a function in an extension module. Does it have to be thread safe? If it isn't, what limitations does this impose on the use of this wrapper? All that aside, I read the previous threads but it took until just now to understand how this works. :-) I'll probably have a go at implementing this in D, as that will be more convenient for the library... -Kirk McDonald -- http://mail.python.org/mailman/listinfo/python-list
Re: Augument assignment versus regular assignment
nagy wrote: > I do the following. First create lists x,y,z. Then add an element to x > using the augumented assignment operator. This causes all the other > lists to be changed also. > But if I use the assignment x=x+[4] instead of using the augumented > assignment, the y and z lists do not change. > Why is that? > This does not happen when I work with integer data type for example, as > shown below. > > Thanks for your help > Nagy > > x=y=z=[] In this example, the '[]' creates a new list object. x, y, and z are all set to reference that object. x+=[2] This does an "in-place" operation on that list, modifying (or "mutating") the object directly. x > > [2] > y > > [2] > z > > [2] > x=x+[4] This creates a new list that is the concatenation of the list created above (the list [2]) with a new list (the list [4]). This brand new list is bound to the name 'x'. The names 'y' and 'z' are left unchanged. That is, they still point to the original list. x > > [2, 4] > y > > [2] > z > > [2] > a=b=4 This binds the names 'a' and 'b' to the integer object 4. b > > 4 > a+=2 This attempts to mutate the integer object 4, by adding 2 to it. However, numbers in Python are immutable, and so the in-place operation fails. Thus, it creates a new integer object equal to 6 (actually, CPython keeps a cache of certain smaller integer objects and reuses them, but this does not matter in practice). This new integer object is bound to the name 'a'. The name 'b' remains bound to the original 4 object. a > > 6 > b > > 4 > -- http://mail.python.org/mailman/listinfo/python-list
Re: Augument assignment versus regular assignment
nagy wrote: > Thanks, Kirk. > I considered the += as only a shorthand notation for the assignment > operator. > Since for lists + is simply a concatetation, I am not sure it x=x+[2] > is creating a brand > new list. Could you refer me to any documentation on this? Yes: http://docs.python.org/ref/augassign.html "An augmented assignment expression like x += 1 can be rewritten as x = x + 1 to achieve a similar, but not exactly equal effect. In the augmented version, x is only evaluated once. Also, when possible, the actual operation is performed in-place, meaning that rather than creating a new object and assigning that to the target, the old object is modified instead." This behavior is only logical. Consider: >>> x = [2] >>> y = x + [4] After these operations, we have two lists: x (the list [2]) and y (the list [2, 4]). This is because the expression "x + [4]" creates a new list. We then bind this new list to the name 'y', and leave the name 'x' alone. If we then say this: >>> x = x + [6] We are doing much the same operation. We are creating a new list (the list [2, 6]), and binding it to the name 'x'. The list [2], previously bound to 'x', is no longer bound to anything, so Python frees it. The augmented assignment, as I went over previously, attempts to modify the list object directly. Any names bound to the object (or any other objects that reference the object) will see the changes. -Kirk McDonald -- http://mail.python.org/mailman/listinfo/python-list
Re: Augument assignment versus regular assignment
Frank Millman wrote: > nagy wrote: > >>Thanks, Kirk. >>I considered the += as only a shorthand notation for the assignment >>operator. >>Since for lists + is simply a concatetation, I am not sure it x=x+[2] >>is creating a brand >>new list. Could you refer me to any documentation on this? >>Thanks, >>Nagy > > > My habit is to check the id. > > x = [1,2] id(x) > > -1209327188 > x += [4] x > > [1,2,4] > id(x) > > -1209327188 > x = x + [6] x > > [1,2,4,6] > id(x) > > -1209334664 > > So it looks as if x += [] modifies the list in place, while x = x + [] > creates a new list. > > I am not sure if this is 100% guaranteed, It is. This is true for any mutable type. > as I have noticed in the past > that id's can be reused under certain circumstances. Perhaps one of the > resident gurus can comment. > -- http://mail.python.org/mailman/listinfo/python-list
