Combing Medusa's Hair... (Design Pattern)
This is an idea I got thinking about COM objects, and getting some support from Mark Hammond, Python's Win32 wizard. The goal is to have a host language (not Python) instantiate an object that runs against the Python interpreter, which lives as its own process. The VMs have various ways of implementing this. Mono isn't that different right? In this design pattern, you have something like a dry cleaner's, where people submit jobs at the counter, and go away right away with a ticket (Python returns -- but keeps running). When they come back is more up to them. Work has been done in the meantime (or not, if the queue is backed up). So Python needs a queue in the front, to accept these job orders, a facility for issuing tickets, and a way to catalog what tasks are completed in some urn or receptacle (given this is Python, we might call it a "holy grail"). The host process has a method from querying the Python object as to whether such-and-such a job is complete or not. More primitively, it could just check an output bin (folder) for the expected file (perhaps each "hair" creates a PDF in roughly 1 to 4 seconds). The reason "Medusa" is useful wordplay is it reminds us of the asynchronous server embedded in early Zope. How Medusa relates to Twisted is lore for others to recount. However, given we're spawning strands, hairs or threads each time a host process calls into our object, we're looking at a multi-snaked environment, so the image could hardly be more apt. The act of "combing" suggests some synchronization / communication between threads, but that's not a requirement. More on tap here: http://mail.python.org/pipermail/edu-sig/2010-December/010141.html Kirby -- http://mail.python.org/mailman/listinfo/python-list
win32 design pattern: COM localserver?
Greetings gurus.
I'm faced with the challenge of having a single-threaded proprietary
Win32
language wanting to do multiple calls against a Python COM object.
Something like...
loObj = CREATEOBJECT("map_maker")
loObj.report( params, filename)
do while .True.
yadda yadda
enddo
RETURN
I.e. the client program continues execution while loObj churns away
somewhere doing work in Python, say accessing a MySQL database
and building a map, embedding it in a PDF.
Am I right that the COM object will need to "serve forever" i.e. be
built around some non-terminating loop? Will the client's call then
spawn a process or thread that continues until finished?
All it has to do is leave a file on disk somewhere, then die. But
the client can't be kept on the hook until that's all done.
I've had good luck doing an InProcess COM object the generates
a PDF from ReportLab. But this "stay alive while returning control
to the client" challenge looks like it might be a job for sockets?
Gee I hope not.
Quick links to favorite blog articles will give me a sense of the
ballpark... Any clues welcome. Yes, I have the Win32 book from
O'Reilly.
Kirby
PS: my thanks to Ethan Furman for performing maintenance and
upgrades on his dbf module on PyPI. The FoxPro gods thank you.
http://pypi.python.org/pypi/dbf/0.88.16
PPS: some thoughts about IronPython filed here, after trying to
catch up some, sorry about my formatting, here too.
http://mail.python.org/pipermail/edu-sig/2010-December/010138.html
--
http://mail.python.org/mailman/listinfo/python-list
Re: DBF (VFP) to XLS (Excel) in pure Python
On Dec 1, 10:28 pm, "kirby.ur...@gmail.com" wrote: > Playing around with arcane tools to read those pesky DBF files (with > memo fields), like floating wine barrels cast off the sinking VFP > ship. > Although it's true I don't know that I'm getting "in memory" DBF reads, the bulk of the time in the code I just shared is in the "writing to XLS" inner loop. Looking to a different solution for writing, perhaps a DLL available for $ (one time). Been reading more about VFP, Silverlight and HTML5. Python seems in a good position. All caught up on IronPython now too. Kirby -- http://mail.python.org/mailman/listinfo/python-list
DBF (VFP) to XLS (Excel) in pure Python
Playing around with arcane tools to read those pesky DBF files (with
memo fields), like floating wine barrels cast off the sinking VFP
ship.
http://pypi.python.org/pypi/dbf
http://packages.python.org/dbf/dbf-module.html
Me threading on comp.lang.python:
http://bit.ly/e547Za
xlwt and xlrt for reading and writing Excel data, without needing
Excel, are also making Python attractive
Here's a code fragment giving the idea:
from os import path
from time import time
import dbf
def test2():
thepath = path.normpath(<< file path >>)
thefile = "shiptemp.dbf"
pathfile = path.join(thepath, thefile)
thedbf = dbf.VfpTable(pathfile)
header_style = easyxf('font: name Arial, bold True, height 200;')
book = Workbook()
sheet1 = book.add_sheet('Sheet 1')
for (i, name) in enumerate(thedbf.field_names):
sheet1.write(0, i, name, header_style)
for (i, thecol) in enumerate(thedbf.structure()):
name, thetype = str(thecol).split()
thelen, thedec = 0,0
if "(" in thetype:
if "," in thetype:
thelen, thedec = [int(k) for k in
thetype[2:-1].split(",")]
else:
thelen, thedec = int(thetype[2:-1]), 0
thetype = thetype[0]
if thetype == "M":
thelen = 100
elif thelen == 0:
thelen = 1
colwidth = max(len(name), int(thelen))
# print colwidth
sheet1.col(i).width = colwidth * 310
thedbf.top()
messages = []
for row in range(1, len(thedbf)):
record = thedbf.next()
for col in range(len(thedbf.field_names)):
try:
sheet1.row(row).write(col, record[col])
except Exception:
pass
book.save("C:\\Documents and Settings\\HP_Administrator\\My
Documents\\Visual FoxPro Projects\\shiptemp.xls")
thedbf.close()
if __name__ == "__main__":
start = time()
test2()
end = time()
print end - start
At the moment I'm unable to get the 1500 record x 156 column table
(with memos) to stay in
memory even with the disk file closed, which I understand is a feature
of this package
http://pypi.python.org/pypi/dbf/
>>> thetable = dbf.VfpTable(thefile)
>>> thetable[1]
<< data dump >>
>>> thetable.close(keep_table=True, keep_memos=True)
Traceback (most recent call last):
File "", line 1, in
thetable.close(keep_table=True, keep_memos=True)
File "C:/Python26/Lib/site-packages\dbf\tables.py", line 1141, in
close
for record in yo:
File "C:/Python26/Lib/site-packages\dbf\tables.py", line 676, in
next
record = yo._table[yo._index]
File "C:/Python26/Lib/site-packages\dbf\tables.py", line 817, in
__getitem__
return yo._table[yo._index[value]]
File "C:/Python26/Lib/site-packages\dbf\tables.py", line 655, in
__getitem__
yo._meta.dfd.seek(location)
AttributeError: 'NoneType' object has no attribute 'seek'
Same error with:
>>> thetable = dbf.VfpTable(thefile)
>>> thetable.close(keep_memos=True)
Python 2.6 / Win7
Kirby
--
http://mail.python.org/mailman/listinfo/python-list
Re: Packages at Python.org
> http://packages.python.org/dbf/ > > So how *do* you get source code from such a web place? I'm not > finding > a tar ball or installer. Sorry if I'm missing something obvious, like > a link > to Sourceforge. > Thanks to very quick replies with pointers to http://pypi.python.org/pypi/dbf/ instead of to packages.python.org/dbf So this packages.python.org subdomain is perhaps unaware of its pypi.python.org competition, which has a brighter and more open look and feel? What's the story there I wonder? Quite an archeology (reminds me of the Vaults of Parnassus). Thanks again, for restoring me to productivity (was dead in the water for a minute there). Kirby -- http://mail.python.org/mailman/listinfo/python-list
Packages at Python.org
With Microsoft abandoning Visual FoxPro come 2015, we have 100K developers jumping ship (rough guess), perhaps to dot NET, but not necessarily.** This page is potentially getting a lot of hits (I'm not privy to the analytics): http://packages.python.org/dbf/ So how *do* you get source code from such a web place? I'm not finding a tar ball or installer. Sorry if I'm missing something obvious, like a link to Sourceforge. Kirby ** Unconfirmed rumors about IronPython leave me blog searching this afternoon. Still part of Codeplex? -- http://mail.python.org/mailman/listinfo/python-list
Re: weird pickle behavior in Python 3.1.2 + Eclipse 3.5.2
On Jun 4, 9:47 am, Peter Otten <__pete...@web.de> wrote:
> I can provoke the error in "naked" Python 3 by changing the
> Example.__module__ attribute:
>
> Python 3.1.1+ (r311:74480, Nov 2 2009, 15:45:00)
> [GCC 4.4.1] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
> >>> import pickle
> >>> class Example:
>
> ... pass
> ...
> >>> pickle.dumps(Example())
>
> b'\x80\x03c__main__\nExample\nq\x00)\x81q\x01}q\x02b.'
> >>> Example.__module__ = "builtins"
> >>> pickle.dumps(Example())
>
> Traceback (most recent call last):
> File "", line 1, in
> File "/usr/lib/python3.1/pickle.py", line 1358, in dumps
> Pickler(f, protocol, fix_imports=fix_imports).dump(obj)
> _pickle.PicklingError: Can't pickle : attribute lookup
> builtins.Example failed
>
> What's the value of __module__ when you run your code in Eclipse?
>
> Peter
Thank you for replying.
Here's from Eclipse console:
>>> Example.__module__
'builtins'
>>> __name__
'builtins'
Duplicating your result in naked Python:
Python 3.1.2 (r312:79149, Mar 21 2010, 00:41:52) [MSC v.1500 32 bit
(Intel)] on win32
Type "copyright", "credits" or "license()" for more information.
>>> class Example:
pass
>>> import pickle
>>> Example.__module__
'__main__'
>>> f = open('testpickle.pkl','wb')
>>> obj = Example()
>>> obj
<__main__.Example object at 0x02A26690>
>>> pickle.dump(obj, f)
>>> Example.__module__ = 'builtins'
>>> obj2 = Example()
>>> pickle.dump(obj2, f)
Traceback (most recent call last):
File "", line 1, in
pickle.dump(obj2, f)
File "C:\Python31\lib\pickle.py", line 1354, in dump
Pickler(file, protocol, fix_imports=fix_imports).dump(obj)
_pickle.PicklingError: Can't pickle : attribute
lookup builtins.Example failed
So what if I'm in an Eclipse pydev console and
change the Example.__module__ to '__main__'
>>> import sys; print('%s %s' % (sys.executable or sys.platform, sys.version))
C:\Python31\python.exe 3.1.2 (r312:79149, Mar 21 2010, 00:41:52) [MSC
v.1500 32 bit (Intel)]
>>> import pickle
>>> class Example:
... pass
...
>>> Example.__module__
'builtins'
>>> Example.__module__ = '__main__'
>>> obj = Example()
>>> obj
<__main__.Example object at 0x029E8FD0>
>>> f = open('testpickle.pkl','wb')
>>> pickle.dump(obj, f)
Traceback (most recent call last):
File "", line 1, in
File "C:\Python31\lib\pickle.py", line 1354, in dump
Pickler(file, protocol, fix_imports=fix_imports).dump(obj)
_pickle.PicklingError: Can't pickle :
attribute lookup __main__.Example failed
>>>
Dang.
Any insights?
Kirby
--
http://mail.python.org/mailman/listinfo/python-list
weird pickle behavior in Python 3.1.2 + Eclipse 3.5.2
Here we are in an Eclipse pydev console, running Python 3.1.2. For
the most part, everything is working great.
However...
>>> import sys; print('%s %s' % (sys.executable or sys.platform, sys.version))
C:\Python31\python.exe 3.1.2 (r312:79149, Mar 21 2010, 00:41:52) [MSC
v.1500 32 bit (Intel)]
>>> import pickle
>>> class Example:
... def __init__(self):
... self.name = "Hello"
... def __repr__(self):
... return "an Example object named {}".format(self.name)
...
...
>>> obj = Example()
>>> obj
an Example object named Hello
Note that I'm opening in binary, like I'm supposed to with this
latest protocol:
>>> f = open("testpickle.pkl",'wb')
Should be able to do this, no problemo:
>>> pickle.dump(obj, f)
Traceback (most recent call last):
File "", line 1, in
File "C:\Python31\lib\pickle.py", line 1354, in dump
Pickler(file, protocol, fix_imports=fix_imports).dump(obj)
_pickle.PicklingError: Can't pickle : attribute
lookup builtins.Example failed
The above works fine in "naked Python" 3.1.2 by the way.
So this could be a problem with Eclipse / Pydev and/or
user error. What am I missing?
Just normal data structures work:
>>> test = [1,2,3]
>>> pickle.dump(test,f)
>>> f.close()
>>>
Any other Eclipse users out there who can at least duplicate this
weirdness?
Kirby Urner
in Portland "Keep Portland Weird" Oregon
--
http://mail.python.org/mailman/listinfo/python-list
FWD: Lore about Division
Alan: 2^(1/2) Kirby: Also pow(2, 0.5) in Python. Alan: Gee, I wonder if pow(2, (1/2)) also works in Python? Kirby: It does if you remember integer division in earlier versions, such that one needed: from __future__ import division or, simply made it 1./2 or 1/2. or something, to force a floating point answer. In 3.x, division is natively floating, with // to force an integer output. [ EDITOR'S NOTE: not quite right, may be floating type output with integer value ] This is Python 3.1, right out of the box: >>> pow(2, 1/2) 1.4142135623730951 Alan: That reminds me of FORTRAN http://en.wikipedia.org/wiki/Fortran_language_features#Implicit_and_explicit_typing "Unless specified otherwise, all variables starting with letters I, J, K, L, M and N are default INTEGER, and all others are default REAL; other data types must be explicitly declared. This is known as implicit typing and is a heritage of early FORTRAN days." Kirby: I think the original intent was something like "closure" i.e. int / int should give int, whereas float / float or float / int or int / float should coerce float answer, and that is how it was done. However it was later realized, by Guido and others, that in source code, when you see a / b, you really have no way of knowing what's going through there, i.e. a and b might name anything, so it gets hard to debug if the behavior is so dependent on 1 versus 3 use cases. Better to just have a consistent float output, no matter what. Then use // if you want to force an integer value output, even if you input floats to get floats back: >>> 1.//2. 0.0 >>> 1/2 0.5 >>> 3.//2. 1.0 It's quite basic to a language, how it deals with division, so Guido had to devise a scheme that would let the snake "shed its skin". Already, the brand name (Python) was in his favor, as that's what Pythons do (shed skin). A lot of basic changes piled up, including a more complete move to a Unicode basis for source files. If you're going to break backward compatibility, might as well do a lot of breaking all at once. As of this writing, we're on the other end of it, having made the leap from 2.x to 3.x. The backward incompatibility gap has been crossed. However, a number of important libraries and other assets still need to bring up the rear, so a new PEP (Python Enhancement Proposal) has been accepted, to freeze Python for awhile, in terms of its core syntax and semantics. Give the library people some breathing room. This seems a good plan. [ original context: http://groups.yahoo.com/group/synergeo/ ] -- http://mail.python.org/mailman/listinfo/python-list
pythonic chew toyz (continued)
On Sun, May 3, 2009 at 5:52 PM, kirby wrote: > From: "kirby.ur...@gmail.com" > Date: Jan 14, 2:18 pm > Subject: pythonic chew toyz (a column, by K. Urner) v.1 n.1 > To: comp.lang.python > > > Selena spotted me right away as "the O'Reilly spy", me agreeing she > had an exceptionally good memory, maybe we could recruit her? Our > banter traces to the first meeting on OS Bridge and how I introduced > myself [0], not yet realizing Allison Randal was in the audience (way > more an insider, as to how many Ls in Randall).[1] > Selena kindly welcomed us into her home for an OS Bridge meeting awhile back, where we tackled the daunting task of sorting through a great many interesting proposals, mine barely squeaking by (yes, helps to have a self interested party at the table, I don't deny it). Audrey was a strong advocate for Allison Randal's getting to speak, but of course that's a shoe in. I'm more the maverick, not having invented any virtual machines yet, just a lot of verbiage mostly, with source code sprinkled in (syntactic sugar we call it). Selena is a Postgres jock many of you may already know. http://www.chesnok.com/daily/ Take a look for news on the new Sun - Oracle merger (yes we know, MySQL). Audrey is with our Legion of Tech especially in the Rails community (which Kaplan-Moss said some nice things about). Together these two are anchoring OS Bridge, our new Open Source conference in Oregon, the Pacific Northwest always in the mood for some big geeky festival, with lots of fun themes, like FOSS on the Farm. http://opensourcebridge.org/sessions/240 > Last night in the Roman room (CubeSpace) I proposed she could emcee > the panel we're planning (maybe?), after the new Mayor (Sam Adams) > kicks it off with a keynote -- all in the proposal stage, looking over > her shoulder. > Sam got rather famous after I posted this or was it before I forget, but a Wild West town like ours is supposed to go overboard into various forms of burlesque, player piano optional, maybe a few head turning cross-dressers (truly, PDX is proud of its heritage, think our airport is pretty spiffy, plus Ikea, other destinations, take the Max, Go By Train, very green). > I was also there for the __metaclasses__ discussion by John Melesky, > and for work, most of that same day (this is PDX CubeSpace, a lot of > us rent office space). Jason talked about the __missing__ rib (in > dict).[2] > > As we were getting settled, they talked about OSCON, perking my > interest. Whereas some take the view we're somehow competing with San > Jose (where some of my friends work, or worked in the case of Maxtor > and IBM), my model is more Cirque du Soleil, given where "geeks" come > from (something about chickens).[3] > I haven't decided about OSCON yet, would love to go, love the show, but this retarded economy has me showing my bank every move, no suitcases of cash like before (just kidding). Seriously, that O'Reilly School of Technology guy was like "we're gonna miss the free beer" referring to sometimes generous Alaska airlines people, maybe Frontier I don't know, giving passengers an up front taste for our offerings, this being not only a FOSS capital, per CSM 2005, but a Beer Capital as well (funny how these go together hand in glove -- no tap at Free Geek though, whereas at Cubespace it's like 50% foam). Anyway, I've heard many wonderful things about San Jose and will say this first hand: the Silicon Valley is not our enemy, nor is the Silicon Hills (Austin), so there, you have it in writing. > Five OSCONs running in parallel in any given summer would make more > sense than just one, but then O'Reilly isn't quite as big as McGraw- > Hill (BYTE), yet. Anyway, Portland is happy to share the glory. OS > Bridge is yet another bulb on the breadboard (aka grid), getting > brighter by the day. > > Speaking of BYTE, I also learned last night that Eliza, our geek > therapist, is alive and well in Python, as a part of the Natural > Language Toolkit or NLTK. And she's got company. Some of you may > recall Hugh Kenner's hilarious investigation of chat bots, then very > new, in a way back issue (1980s).[4] > I tried to get Nat Bobbitt interested in this but he said "no more comms" perhaps being strategic, time will tell. I admit to not following through either, making Sun Tzu and Eliza have conversational intercourse, a great high school project, and now we have a Watcher between Pycons -- Vern Ceder, tasked with scoping out talent, Pythonistas too young for a Pycon maybe (Hyatts pretty racy, lots of alcoholic beverages), but still able to exhibit in a poster session, perhaps with a teacher as stand in, proud for the whole school (Python Teacher of the Year?) -- we'll have s
pythonic chew toyz (a column, by K. Urner) v.1 n.1
Selena spotted me right away as "the O'Reilly spy", me agreeing she had an exceptionally good memory, maybe we could recruit her? Our banter traces to the first meeting on OS Bridge and how I introduced myself [0], not yet realizing Allison Randal was in the audience (way more an insider, as to how many Ls in Randall).[1] Last night in the Roman room (CubeSpace) I proposed she could emcee the panel we're planning (maybe?), after the new Mayor (Sam Adams) kicks it off with a keynote -- all in the proposal stage, looking over her shoulder. I was also there for the __metaclasses__ discussion by John Melesky, and for work, most of that same day (this is PDX CubeSpace, a lot of us rent office space). Jason talked about the __missing__ rib (in dict).[2] As we were getting settled, they talked about OSCON, perking my interest. Whereas some take the view we're somehow competing with San Jose (where some of my friends work, or worked in the case of Maxtor and IBM), my model is more Cirque du Soleil, given where "geeks" come from (something about chickens).[3] Five OSCONs running in parallel in any given summer would make more sense than just one, but then O'Reilly isn't quite as big as McGraw- Hill (BYTE), yet. Anyway, Portland is happy to share the glory. OS Bridge is yet another bulb on the breadboard (aka grid), getting brighter by the day. Speaking of BYTE, I also learned last night that Eliza, our geek therapist, is alive and well in Python, as a part of the Natural Language Toolkit or NLTK. And she's got company. Some of you may recall Hugh Kenner's hilarious investigation of chat bots, then very new, in a way back issue (1980s).[4] Hugh, by the way, was a leading 20th century James Joyce scholar and fan of R. Buckminster Fuller (see 'The Pound Era'), Bucky a champion for me as well (hence my rbf.py -- named after the guy, for use with VPython, POV-Ray, x3d etc., more on edu-sig). I'll catch up again later, maybe from Chicago, maybe before.[5] [0] http://mail.python.org/pipermail/portland/2008-November/000512.html [1] http://radar.oreilly.com/allison/ [2] http://mybizmo.blogspot.com/2009/01/ppug-2009113.html [3] http://worldgame.blogspot.com/2007/06/pro-python-propaganda.html [4] http://mathforum.org/kb/message.jspa?messageID=5256703&tstart=0 [5] http://showmedo.com/videos/video?name=1010050&fromSeriesID=101 [ 2nd edition: reformatted, fixed endnotes ] << with thanks to verizon wireless >> -- http://mail.python.org/mailman/listinfo/python-list
pythonic chew toyz (a column, by K. Urner) v.1 n.1
Selena spotted me right away as "the O'Reilly spy", me agreeing she had an exceptionally good memory, maybe we could recruit her? Our banter traces to the first meeting on OS Bridge and how I introduced myself [0], not yet realizing Allison Randal was in the audience (way more an insider, as to how many Ls).[1] Last night in the Roman room (CubeSpace) I proposed she could emcee the panel we're planning (maybe?), after the new Mayor (Sam Adams) kicks it off with a keynote -- all in the proposal stage, looking over her shoulder. I was also there for the __metaclasses__ discussion by John Melesky, and for work, most of that same day (this is PDX CubeSpace, a lot of us rent office space). Jason talked about the __missing__ rib (in dict). As we were getting settled, they talked about OSCON, perking my interest. Whereas some take the view we're somehow competing with San Jose (where a lot of my friends work), my model is more Cirque du Soleil, given where "geeks" come from (something about chickens). Five OSCONs running in parallel in any given summer would make more sense than just one, but then O'Reilly isn't quite as big as McGraw- Hill (BYTE), yet. Anyway, Portland is happy to share the glory. OS Bridge another bulb on the breadboard (aka grid), getting brighter by the day. Speaking of BYTE, I also learned last night that Eliza, our geek therapist, is alive and well in Python, as a part of the Natural Language Toolkit or NLTK. And she's got company. Some of you may recall Hugh Kenner's hilarious investigation of chat bots, then very new, in a way back issue (1980s).[3] Hugh, by the way, was a leading 20th century James Joyce scholar and fan of R. Buckminster Fuller (see 'The Pound Era'), Bucky a champion for me as well (hence my rbf.py -- named after the guy, for use with VPython, POV-Ray, x3d etc., more on edu-sig). I'll catch up again later, maybe from Chicago, maybe before.[4] [1] http://mail.python.org/pipermail/portland/2008-November/000512.html [2] http://mybizmo.blogspot.com/2009/01/ppug-2009113.html [3] http://mathforum.org/kb/message.jspa?messageID=5256703&tstart=0 [4] http://showmedo.com/videos/video?name=1010050&fromSeriesID=101 -- http://mail.python.org/mailman/listinfo/python-list
