Re: Completion of The Python Papers Volume 4 Issue 1

[Maurice Ling]

My apologies.

It should be Volume 4 Issue 1 instead of Issue 4 Volume 1.

Thank you Scott for pointing this out.

Regards
Maurice

On May 6, 10:08 pm, "mauricel...@acm.org" 
wrote:
> Welcome to Issue 4 Volume 1 of The Python Papers. This marks our 4th
> year in business. It is my pleasure to announce 4 improvements made to
> The Python Papers Anthology.
>
> Firstly, we had created a class of editorial members in our team – the
> Editorial Reviewers (ER). This is in addition to the existing Editors-
> in-Chief (EIC) and Associate Editors (AE). In the words of our revised
> editorial policy, “ER is a tier of editors deemed to have in-depth
> expertise knowledge in specialized areas. As members of the editorial
> board, ERs are accorded editorial status but are generally not
> involved in the strategic and routine operations of the periodicals
> although their
> expert opinions may be sought at the discretion of EIC.” To this
> aspect, we are glad to have Jacob Kaplan-Moss (President, Django
> Software Foundation; Partner, Revolution Systems, LLC; Co-BDFL,
> Django) as our first ER. Welcome aboard.
>
> Secondly, we had adopted a fast publication scheme. In the past,
> accepted manuscripts will only be published on the release date in the
> month of April, August and December each year. With the use of Open
> Journal System, we are able to publish manuscripts as they are being
> accepted. This means that the release date in the month of April,
> August and December is now effectively our closing date for the issue.
> For example, manuscripts accepted after April 2009 will be routed to
> immediate publication in Volume 4 Issue 2. This is to being
> information faster to our readers, rather than waiting for a few
> months.
>
> Thirdly, we announce a set of manuscript templates for our authors and
> contributors for their manuscript preparation. As such, we had
> revamped our website (http://www.pythonpapers.org) to reflect these
> changes.
>
> Lastly, we are glad to announce the launch of our 2nd publication
> under The Python Papers Anthology – The Python Papers Source Codes
> (ISSN 1836-621X). TPPSC, for short, is a collection of software and
> source codes, usually associated with papers published in The Python
> Papers and The Python Papers Monograph. These codes are refereed for
> originality, accuracy, completeness, and lasting value. TPPSC will
> act as a developers codebook of high-quality, peer-reviewed codes.
> TPPSC is an annual publication using the same fast publication scheme.
>
> We thank all our readers and contributors for their continued support.
>
> Happy reading.
>
> Maurice Ling
> Co-EIC, TPPA

--
http://mail.python.org/mailman/listinfo/python-list

Re: Re; finding euclidean distance,better code?

[Maurice LING]

Hendrik van Rooyen wrote:
> On Saturday 29 March 2008 03:09:46 Steven D'Aprano wrote:
>> On Fri, 28 Mar 2008 16:59:59 +0100, Robert Bossy wrote:
>>> Gabriel Genellina wrote:
 That's what I said in another paragraph. "sum of coordinates" is using
 a different distance definition; it's the way you measure distance in a
 city with square blocks. I don't know if the distance itself has a
 name, but
>>> I think it is called Manhattan distance in reference of the walking
>>> distance from one point to another in this city.
>> You know, there are other cities than Manhattan. Some of them even have
>> streets and blocks.
> 
> Sorry about having to dispel your illusions, but -
> 
> In Printed Circuit Board Layout jargon, the 'manhattan distance' is
> the sum of the distances along the orthogonal axes between
> two points on the board that should be connected.
> 
> The sum of all such distances is an idealised minimum for the
> total track length on a double sided board, given that it were 
> possible to lay all tracks with segments connected by vias,
> making strictly increasing progress in the desired direction,
> by laying x direction tracks on the one, and y direction tracks 
> on the other side of the board without having to "backtrack" 
> - i.e. having to "dodge around" obstacles, thereby adding 
> "overshooting" segments of track.
> 
> (A via is a through plated hole that connects copper traces or 
> tracks on opposite sides of the board) 
> 
> So I have met the beast, but I have no concept of its origin,
> other than the mind numbing regularity of the layout of the
> suburb of the city after which it seems to be named -
> For all I know 'manhatten' could be a native american word that
> means "net".
> 
> Have you noticed that when people say "Sorry.but" they are 
> not normally sorry at all?
> 
> :-)
> 
> -  Hendrik
> 

Manhatten distance is also known as "taxicab distance" or "city block 
distance" - the average distance of getting from point A to point B in a 
city with roads laid out as grids.

maurice
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Multiple Submits in HTML Forms - Cherrypy

[Maurice LING]

Carsten Haese wrote:
> On Tue, 2008-03-18 at 07:14 +0000, Maurice LING wrote:
>> Hi,
>>
>> Assuming that I have this code for Cherrypy 3
>>
>> class Welcome:
>>  def index(self):
>>  return """
>>  
>>  
>>  
>>  """
>>  index.exposed = True
>>
>> How should I write "btn_handler" so that it will perform different 
>> actions when different button is pressed?
> 
> Something like this would do it:
> 
>  def btn_handler(self, AddBtn=None, EditBtn=None):
>  if AddBtn:
>  return "You pressed Add!"
>  if EditBtn:
>  return "You pressed Edit!"
> 
> Alternatively you could use a kwargs dictionary and test it for the
> presence of the "AddBtn" or "EditBtn" keys.
> 

Thank you Carsten
maurice
-- 
http://mail.python.org/mailman/listinfo/python-list

Multiple Submits in HTML Forms - Cherrypy

[Maurice LING]

Hi,

Assuming that I have this code for Cherrypy 3

class Welcome:
 def index(self):
 return """



"""
 index.exposed = True

How should I write "btn_handler" so that it will perform different 
actions when different button is pressed?

Thanks in advance

Cheers
maurice
-- 
http://mail.python.org/mailman/listinfo/python-list

The Python Papers is looking for additional Associate Editors

[Maurice LING]

"The Python Papers" (http://pythonpapers.org), ISSN 1834-3147, is an 
online e-journal, covering articles on Python in the community, industry 
and academia. We were established in the second half of 2006 and 
launched our first issue in November 2006. Since then, we have released 
3 more issues.

Recently, we have also initiated a monograph series, which will be known 
as "The Python Papers Monograph Series" 
(http://pythonpapers.org/monograph.html) with its own ISSN, but managed 
by "The Python Papers" Editorial Committee.

Due to this new initiative, we are looking for an additional member to 
join the team as an Associate Editor from February 2008.

The main roles of an Associate Editor includes:

1. Organizing and chairing reviews for submissions
2. Reviewing submissions
3. Answering queries regarding the periodicals
4. Assisting the Editor-in-Chief in tasks as required

The current workload is estimated to be about 3 hours per week.

We invite committed individuals with a good knowledge of Python 
programming to join our team.

Please send in your CV, a statement of intention, and answer the 
following questions to us at [EMAIL PROTECTED] before December 20, 
2007.

1. An account of your programming experiences using your projects for 
illustrations
2. An account of your publication record and experiences with 
peer-review process
3. A brief description of your expertise in computer science
4. A brief account of your understanding about The Python Papers
5. How will you propose to improve The Python Papers

Please direct any queries to [EMAIL PROTECTED]

Thank you
Maurice Ling
Associate Editor, The Python Papers


-- 
http://mail.python.org/mailman/listinfo/python-list

The Python Papers is looking for additional Associate Editors

[maurice ling]

"The Python Papers" (http://pythonpapers.org), ISSN
1834-3147, is an online e-journal, covering articles
on Python in the community, industry and academia. We
were established in the second half of 2006 and
launched our first issue in November 2006. Since then,
we have released 3 more issues.

Recently, we have also initiated a monograph series,
which will be known as "The Python Papers Monograph
Series" (http://pythonpapers.org/monograph.html) with
its own ISSN, but managed by "The Python Papers"
Editorial Committee.

Due to this new initiative, we are looking for an
additional member to join the team as an Associate
Editor from February 2008.

The main roles of an Associate Editor includes:

1. Organizing and chairing reviews for submissions
2. Reviewing submissions
3. Answering queries regarding the periodicals
4. Assisting the Editor-in-Chief in tasks as required

The current workload is estimated to be about 3 hours
per week.

We invite committed individuals with a good knowledge
of Python programming to join our team.

Please send in your CV and a statement of intention to
us at [EMAIL PROTECTED] before December 20,
2007. Please direct any queries to
[EMAIL PROTECTED]

Thank you
Maurice Ling
Associate Editor, The Python Papers
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Joining open source python projects

[Maurice LING]

Giampaolo Rodola' wrote:
> Hi there,
> I don't know if such a thing has been already discussed, in which case
> I'm sorry.
> I was wondering if there's a place for python open source projects
> that need help.
> It thought it would be very nice having a place where developers could
> submit "help requests" for their projects and let the users view them
> and eventually join them if they want to.
> 
> Does someone knows if such a service already exist or not?

You might want to tell us what you like to do?
What your experiences/skillset are?
How much time you can commit?
What do you want to learn?
etc etc

Alternatively look in Sourceforge.com, there is never enough hands.

maurice
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: SOAPpy port reuse

[Maurice LING]

Diez B. Roggisch wrote:
> Maurice LING schrieb:
>> Hi,
>>
>> I have a problem:
>>
>> 1. Assuming that my application is a SOAP server that uses SOAPpy,
>> 2. I am given port 35021 for use.
>>
>> What I normally do (simply) is:
>>
>> functionlist = []
>> import SOAPpy
>> server = SOAPpy.SOAPServer((, 35021))
>> for func in functionlist: server.registerFunction(func)
>> server.serve_forever()
>>
>> My question is: How can I shutdown this server and reuse port 35021 
>> when my functionlist changes?
>>
>> Currently, after killing the python process which runs this SOAP 
>> server, the port (35021 in this case) cannot be re-used, as though it 
>> is still phantom-ly bounded to some process (which should have been 
>> killed).
> 
> It shouldn't be that way. Either you still have some process lying 
> around hogging the port. Or the OS needs a while to re-enable the port 
> for allocation. That happened to me quite a few times.
> 
> Shutting down gracefully might speed up things I guess.
> 

I am under the impression that SOAPpy.SOAPServer.serve_forever() is an 
"endless" loop. I had been suggested to see if there is a method of 
SOAPpy.SOAPServer (which I can call through a wrapper function in 
functionlist) that can enable me to gracefully shutdown the server.

Any advice?

Thanks
Maurice
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Book: Python Power!: The Comprehensive Guide

[Maurice LING]

John Salerno wrote:
> Anyone know anything about this book? I've read a few intro Python books 
> already, but I'm always interested in reading more to reinforce the 
> language. No reviews on Amazon yet so I'm not sure if it's good or not.
> 
> Thanks.


A cursory glance while standing in the bookshop suggest that I should 
give it a closer read - there are some interesting stuffs that caught my 
eyes but I only have time for a quick flip then.

maurice
-- 
http://mail.python.org/mailman/listinfo/python-list

SOAPpy port reuse

[Maurice LING]

Hi,

I have a problem:

1. Assuming that my application is a SOAP server that uses SOAPpy,
2. I am given port 35021 for use.

What I normally do (simply) is:

functionlist = []
import SOAPpy
server = SOAPpy.SOAPServer((, 35021))
for func in functionlist: server.registerFunction(func)
server.serve_forever()

My question is: How can I shutdown this server and reuse port 35021 when 
my functionlist changes?

Currently, after killing the python process which runs this SOAP server, 
the port (35021 in this case) cannot be re-used, as though it is still 
phantom-ly bounded to some process (which should have been killed).

Thanks in advance.

Cheers
maurice
-- 
http://mail.python.org/mailman/listinfo/python-list

The Python Papers Monograph Series (TPPM): Call for Proposals

[Maurice LING]

The editorial committee of The Python Papers (ISSN 1834-3147) calls for 
proposals for The Python Papers Monograph Series.

Potential authors should contact the editors at [EMAIL PROTECTED] 
at an early stage of preparation.

We await your indications of interests.

Regards
Maurice Ling
Associate Editor, The Python Papers


What is The Python Papers Monograph Series?
===
The Python Papers Monograph Series (ISSN under application) is a 
sub-publication of The Python Papers (ISSN 1834-3147). This monograph 
series provides a refereed format for publication of monograph-length 
reports including dissertations, conference proceedings, case studies, 
advanced-level lectures, and similar material of theoretical or 
empirical importance. It does so quickly,  informally and at a high 
level, where the Python programming language is an integral aspect.

In some cases, the timeliness of a manuscript is more important than its 
form, which may be unfinished or unpolished. Hence, it is possible that 
proofs may be outlined with an intention to publish elsewhere at a later 
date.

Refereeing is done by members of the The Python Papers' editorial board 
and other authorities in the topic concerned and may be acknowledged in 
name in the published form. Scientific quality is the over-riding 
criterion for refereeing.

Suggestions for publication, in the form of outlines and representative 
samples, are invited by the editorial board for assessment. Potential 
authors should contact the editors at [EMAIL PROTECTED]

Publication in this monograph series is intended as a service to the 
Python users and scientific community at large. Work in this series is 
licensed under the Creative Commons 2.5 license subject to Attribution,
Non-Commercial and Share-Alike conditions. The full legal code may be 
found at http://creativecommons.org/licenses/byncsa/2.1/au/. Once 
published and copyrighted, they can be documented and discussed in the 
scientific literature.
=
-- 
http://mail.python.org/mailman/listinfo/python-list

The Python Papers Monograph Series (TPPM): Call for Proposals

[Maurice LING]

The editorial committee of The Python Papers (ISSN 1834-3147) calls for 
proposals for The Python Papers Monograph Series.

Potential authors should contact the editors at [EMAIL PROTECTED] 
at an early stage of preparation.

We await your indications of interests.

Regards
Maurice Ling
Associate Editor, The Python Papers


What is The Python Papers Monograph Series?
===
The Python Papers Monograph Series (ISSN under application) is a 
sub-publication of The Python Papers (ISSN 1834-3147). This monograph 
series provides a refereed format for publication of monograph-length 
reports including dissertations, conference proceedings, case studies, 
advanced-level lectures, and similar material of theoretical or 
empirical importance. It does so quickly,  informally and at a high 
level, where the Python programming language is an integral aspect.

In some cases, the timeliness of a manuscript is more important than its 
form, which may be unfinished or unpolished. Hence, it is possible that 
proofs may be outlined with an intention to publish elsewhere at a later 
date.

Refereeing is done by members of the The Python Papers' editorial board 
and other authorities in the topic concerned and may be acknowledged in 
name in the published form. Scientific quality is the over-riding 
criterion for refereeing.

Suggestions for publication, in the form of outlines and representative 
samples, are invited by the editorial board for assessment. Potential 
authors should contact the editors at [EMAIL PROTECTED]

Publication in this monograph series is intended as a service to the 
Python users and scientific community at large. Work in this series is 
licensed under the Creative Commons 2.5 license subject to Attribution,
Non-Commercial and Share-Alike conditions. The full legal code may be 
found at http://creativecommons.org/licenses/byncsa/2.1/au/. Once 
published and copyrighted, they can be documented and discussed in the 
scientific literature.
=
-- 
http://mail.python.org/mailman/listinfo/python-list

[ANN] The Python Papers volume 2 issue 3

[Maurice LING]

On behalf of the editorial board of The Python Papers (ISSN 1834-3147), 
we are pleased to announce the release of our latest edition, Volume 2 
Issue 3.

This is an important milestone for all of us as the next issue will mark 
our first year in "press".

Volume 2 Issue 3 can be found at 
http://archive.pythonpapers.org/ThePythonPapersVolume2Issue3.pdf and 
http://www.scribd.com/doc/272054/ThePythonPapersVolume2Issue3

Table of Contents:
Letter from the editor  Pages 3

*Industry Articles*
Python User Group Highlights: Introducing BrisPyPages 5-6
The Barriers Women Face in Tech Communities Pages 7-13
Python Testimonials Pages 14-17
The Longest Common Substring and Sentence Modification  Pages 18-27
Python Events   Page 44

*Peer Reviewed Section*
Pyphant - A Python Framework for Modelling Reusable Information 
Processing TasksPages 28-43


We will like to thank the community for their contributions and support.

Regards
Maurice Ling
Associate Editor
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: why should I learn python

[Maurice LING]

windandwaves wrote:
> Can someone tell me why I should learn python?  I am a webdeveloper,
> but I often see Python mentioned and I am curious to find out what I
> am missing out on.
> 
> Thank you
> 
> Nicolaas
> 

The last edition of The Python Papers (volume 2 issue 2, 
http://archive.pythonpapers.org/ThePythonPapersVolume2Issue2.pdf) has an 
article on Django, a web framework. It may help you see what you are 
missing out on.

maurice
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Database intensive application

[Maurice LING]

Rohit wrote:
> I am a novice. I want to know whether Python can be used to develop
> client/server database and web applications like .NET. Which is the
> best book/source to learn Python?
> 

There are database access libraries for almost every RDBMS.
Web applications using Django, CherryPy, TurboGears and many other tools
Best book(s): Dive into Python, Learning Python

maurice
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: ANN: YahooQuote 0.1.0

[Maurice LING]

Hi David,

I can't download from your link 
http://www.freenet.org.nz/python/yahooquote/YahooQuote-0.1.tar.gz

Cheers
maurice

DavidM wrote:
> Hi,
> 
> After a 3-year break from working with it, I've just released version 0.1.0
> (the first formally packaged release) of YahooQuote. As the name implies,
> it's a stockmarket prices fetcher.
> 
> Features:
>  - easily fetch stock price quotes and histories
>  - fetch bulk histories across one or more of the major stockmarket
>indexes
>  - fast efficient automatic caching via a MetaKit database
>  - pleasant object-oriented interface
>  - thorough epydoc-generated API doco
>  - simple examples
> 
> This supersedes the unpackaged YahooQuote module of 2004, and brings
> YahooQuote up to date with the present Yahoo Finance web interface.
> 
> http://www.freenet.org.nz/python/yahooquote
> 
> License: LGPL
> 
> Enjoy
> 
> Cheers
> David
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Who uses Python?

[Maurice LING]

[EMAIL PROTECTED] wrote:
> i use it for text mining, processing large text corpora for scientific
> purposes. i'm also working on some neat data mining tools written in
> python (called orange, in case someone's interested)
> 

Hi,

I am very interested with your use of Python. I am into text mining as 
well. Perhaps we can communicate more on this...

I am also interested in your data mining tools, orange... Care to give 
more details.

Thanks in advance
Maurice
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Who uses Python?

[Maurice LING]

walterbyrd wrote:
> I mean other than sysadmins, programmers, and web-site developers?
> 
> I have heard of some DBAs who use a lot of python.
> 
> I suppose some scientists. I think python is used in bioinformatics. I
> think some math and physics people use python.
> 
> I suppose some people use python to learn "programming" in general.
> Python would do well as a teaching language.
> 
> I would think that python would be a good language for data analysis.
> 
> Anything else? Finance? Web-analytics? SEO? Digital art?
> 

I use Python for my bioinformatics work, mainly in text analysis and 
transcriptome analysis.

ML
-- 
http://mail.python.org/mailman/listinfo/python-list

[ANN] The Python Papers Volume 2 Issue 2

[Maurice LING]

Hi everyone,

After some delays yesterday, Volume 2 Issue 2 of The Python Papers had 
been officially released today. Download it from www.pythonpapers.org

This issue marks a major landmark in our publication. We present a 
number of industry articles. These include "Python in Education" and 
"MPD WebAMP", as well as a great insight into Python in Germany, a 
wrap-up of PyCon 2007, a preview of EuroPython 2007 and a look at some 
great videos prepared by primary school students. Our peer-reviewed 
section reproduces two selected papers which were originally presented 
at the Open Source Developer's Conference 2006 (Melbourne, Australia).

Thank you everyone for all your support.

Cheers
Maurice Ling
(Associate Editor)
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: db.commit() to take effect

[Maurice LING]

John Machin wrote:
> progman wrote:
> 
>>I was testing the python+mysql
>>
>>here are my sample codes:
>>
>>import MySQLdb
>>from pprint import pprint
>>db = MySQLdb.connect(host="localhost", user="root", passwd="password",
>>db="database")
>>cursor = db.cursor()
>>cursor.execute('update promo set date=100")
>>
>>
>>
>>
>>i was expecting the cursor.execute will update my db immediately.
>>it wasn't. not until i  run db.commit(), then only i see the value
>>changes.
>>
>>it that the right way to update db?
> 
> 
> Short answer: yes
> 
> Longer answer: In most non-trivial db apps, to carry out a given
> real-world transaction, you will need to do multiple updates/inserts,
> and you want them all to happen, or none to happen. The database would
> be inconsistent if your app or server crashed half-way through. Imagine
> you are transferring some money to someone else's bank account. The
> money gets deducted from your account but not added to theirs -- not
> good. [A real banking system would not be that simple, but you should
> get the general idea.]  So you do db.commit() after the last
> insert/update.
> 
> HTH,
> John
> 

Adding to that, PEP 249 specified that if a DBMS has auto-commit 
feature, it should be set to "off", implying that cursor.execute() and 
cursor.executemany() methods do not commit the database. Database commit 
is defined as .commit() method in connect object.

Cheers
ML
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: The Python Papers Edition One

[Maurice LING]

Jerry Hill wrote:
> On 11/25/06, Jerry Hill <[EMAIL PROTECTED]> wrote:
> 
>> On 23 Nov 2006 15:09:11 -0800, [EMAIL PROTECTED]
>> <[EMAIL PROTECTED]> wrote:
>> > Yes, it's true that you can't resell copies of The Python Papers for
>> > personal profits, but you may derive from it, reproduce and propagate
>> > it. You're quite right to point it out.
>>
>> My problem with this is that I can't use anything in your publication
>> when working on commercial software.  If I were to derive code from
>> something in the Python Papers, my understanding is that I would be
>> obligated to release it under a Creative Commons license.  In fact,
>> even if all I do is read an article and then incorporate concepts from
>> it in my code, my understanding is that I may be creating a derivative
>> work.
>>
>> Since the code that I write for work belongs to my employer, and may
>> someday be sold, I need to be careful about the licensing issues. They
>> might not be very happy with me if I wrote software for them that
>> ended up being encumbered with a license they didn't like.  I don't
>> think there's much of a problem with Issue 1, since I don't think I'd
>> end up copying the code in the 'Python Coding Idioms' article, and
>> MontyLingua is GPL software and thus has its own licensing issues for
>> commercial software.  Still, if there was a useful article on, say,
>> database or web interfaces, I would have to avoid it.  If I have to
>> avoid using any articles that might actually be useful in my
>> professional life, I feel like I should probably just avoid the
>> journal all together.  If I've misrepresented the relevant copyright
>> issues, I'd be happy for someone to correct me.
>>

As Steven mentioned -- anything you can read is copyrighted. The 
difference is whether is the copyright effective or enforceable. What do 
I mean by this? Without copyright, there will not be plagarism. Ask 
yourself this question, can you copy William Shakespeare's MacBeth and 
submit it as a literary work for a Master of Literary Arts degree? I 
believe the candidate will be expelled from university. William 
Shakespeare's MacBeth is still copyrighted work but not "enforceable" 
because it is pre-1900's work and the author had been dead for more than 
50 years. Similarly, works in public domain are still copyrighted -- 
academically, using work in public domain without attribution (giving 
credits in the form of citations) is still plagarism.

This means that everything you had read since the days of "ABC..." are 
copyrighted. That includes all codes you've seen in colleges etc etc. I 
am afraid that to avoid copyright altogether, as far as your work is 
concerned, you might have to seclude yourself in some pacific islands 
and re-discover mathematics and computer science all over again from 1 + 
1 = 2, and 2 + 1 = 3, and so on. Even so, patents will still get you at 
the end.

In copyright, there is fair use. There is no way of avoiding it totally 
-- how many ways are there to write a list comprehension?

Copyright just says attribute credits when you use someone else's work 
within the limits of fair use; otherwise you might have to pay for it in 
the form of a licence, subject to the copyright owner. I believe you've 
done all these in college when writing your essays.

I believe in most cases, a simple declaration like "This function is a 
re-implementation (or adaptation) of that found in " will suffice.

Have you not read "The Python Cookbook", in book form or from the 
website? How do you attribute credits when you are using the codes?

Cheers
maurice
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Will GPL Java eat into Python marketshare?

[Maurice LING]

Paul Boddie wrote:

> Maurice LING wrote:
> 
>>Say given cytoscape.jar, I'll like to be able to do this:
>>
>> >>> from cytoscape javaimport cytoscape
>> >>> c = cytoscape()
>>
>>And the tighest way I see that this can be done is for Python VM to
>>execute Java bytecodes like Python bytecodes. That is, Python VM
>>executes Java bytecodes directly and not through object mapping which I
>>think is that JPyPe is doing.
> 
> 
> This kind of thing is what I wanted to do with my javaclass experiment,
> but I lost motivation/momentum after getting only some of the things to
> work. Translating Java bytecodes isn't hard for most of the instruction
> set, although exception handling was awkward at the time (before Python
> 2.5's new-style Exception class), and I'd have to think through the
> interpackage referencing problem again (circular references are not
> typically a problem in the Java package hierarchy). In the end, I
> couldn't justify spending the time in order to use certain pieces of
> software that weren't so compelling after all.
> 
>

I take a simplistic view that Java bytecodes is all that is to be dealt 
with. And the book "Programming for the Java Virtual Machine" by Joshua 
Engel did demonstrated in principle that it is possible to program in 
Java bytecodes. There are also other researchware which tried to compile 
other languages into Java bytecodes. I am not sure if exception handling 
etc are dealt with before bytecode level. I also envision that the core 
implementation of JVM is a big switch statement, just like in Python VM 
(the bytecode executor).

Will it then be the case of adding the set of Java bytecodes into Python 
bytecodes and implementing Java bytecode operations? Or am I just 
fooling myself here?

ML
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Will GPL Java eat into Python marketshare?

[Maurice LING]

Stephen Eilert wrote:

> Maurice LING escreveu:
> 
> 
>>>I once wrote a partial JVM in Modula-3 (strictly a researchware
>>>effort), so I can imagine it being done technically.  But why?
>>>
>>>The big problem with Java-and-Python is not the VMs underneath.  It is
>>>the fact that Java has layers upon layers upon layers of idiosyncratic
>>>libraries and idioms.  When you write bindings to that world (even if
>>>the bindings are generated automagically), you have to *think* in
>>>those same layers.  The Python-oriented developer suddenly has to use
>>>a dozen imports in order to do things already done better in
>>>Pythonesque libraries.
>>>
>>
>>The main use I can see is to be able to incorporate Java applications
>>into Python. For example, I am using Cytoscape (www.cytoscape.org) which
>>is in Java. I do hope that I can control Cytoscape from Python and
>>manipulate its objects from Python.
>>
>>Say given cytoscape.jar, I'll like to be able to do this:
>>
>> >>> from cytoscape javaimport cytoscape
>> >>> c = cytoscape()
>>
>>And the tighest way I see that this can be done is for Python VM to
>>execute Java bytecodes like Python bytecodes. That is, Python VM
>>executes Java bytecodes directly and not through object mapping which I
>>think is that JPyPe is doing.
>>
>>I must say that this is part of even a fluffier dream that one day, I
>>can take any applications and play around with it in Python. Currently,
>>my collaborators wrote in Perl and Java, so it is not easy for me to use
>>their work in my work.
>>
>>ML
> 
> 
> What is wrong with the other way around and Jython?
> 
Nothing wrong with the other way round - JVM executing *Python bytecodes*.

Cytoscape has a plugin with enables one to bring up Jython interpreter 
but it is way too slow - make sure you start to load it up before lunch 
if you want to execute a few lines of codes after lunch.

ML
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Will GPL Java eat into Python marketshare?

[Maurice LING]

> 
> I once wrote a partial JVM in Modula-3 (strictly a researchware
> effort), so I can imagine it being done technically.  But why?
> 
> The big problem with Java-and-Python is not the VMs underneath.  It is
> the fact that Java has layers upon layers upon layers of idiosyncratic
> libraries and idioms.  When you write bindings to that world (even if
> the bindings are generated automagically), you have to *think* in
> those same layers.  The Python-oriented developer suddenly has to use
> a dozen imports in order to do things already done better in
> Pythonesque libraries.
> 

The main use I can see is to be able to incorporate Java applications 
into Python. For example, I am using Cytoscape (www.cytoscape.org) which 
is in Java. I do hope that I can control Cytoscape from Python and 
manipulate its objects from Python.

Say given cytoscape.jar, I'll like to be able to do this:

 >>> from cytoscape javaimport cytoscape
 >>> c = cytoscape()

And the tighest way I see that this can be done is for Python VM to 
execute Java bytecodes like Python bytecodes. That is, Python VM 
executes Java bytecodes directly and not through object mapping which I 
think is that JPyPe is doing.

I must say that this is part of even a fluffier dream that one day, I 
can take any applications and play around with it in Python. Currently, 
my collaborators wrote in Perl and Java, so it is not easy for me to use 
their work in my work.

ML
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Will GPL Java eat into Python marketshare?

[Maurice LING]

Dennis Lee Bieber wrote:

> On Wed, 15 Nov 2006 22:41:19 GMT, Maurice LING <[EMAIL PROTECTED]>
> declaimed the following in comp.lang.python:
> 
> 
>>I'm hoping for a more optimistic outcome that this may open a 
>>possibility for tigher interoperability between java programs and python 
>>programs. That is, run java class files or java codes natively on python 
>>VM. Is this still a blue sky dream?
>>
> 
>   Most unlikely to happen... I don't really see anyone going to the
> effort to change the javac back-end to target a totally different
> runtime engine. 

I admit that it is very very unlikely. I guess it is just a wild dream 
of mine to run Java bytecodes and Python bytecodes on Python VM. I do 
have a wild vision that we can import java libraries (as jar files) into 
CPython.


ML
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python deployment options.

[Maurice LING]

Cameron Laird wrote:

> In article <[EMAIL PROTECTED]>,
> Richard Charts <[EMAIL PROTECTED]> wrote:
>   .
>   .
>   .
> 
>>Well on a Win machine, probably.
>>Almost every Linux machine you come across will have (most likely a
>>fairly recent build of) python.  For Macs, I'm not so sure but it's
>>probably closer to Linux than Win.
>>
> 
> 
> Recent releases of Mac OS build in Python.

Python has been part of Mac OSX since OSX 10.2 (maybe even earlier but 
10.2 is the earliest I've used).

ML
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Will GPL Java eat into Python marketshare?

[Maurice LING]

walterbyrd wrote:
> Some think it will.
> 
> Up untill now, Java has never been standard across different versions
> of Linux and Unix. Some think that is one reason that some developers
> have avoided Java in favor of Python. Now that Java has been GPL'd that
> might change.
> 
> IMO: it won't make much difference. But I don't really know.
> 

I'm hoping for a more optimistic outcome that this may open a 
possibility for tigher interoperability between java programs and python 
programs. That is, run java class files or java codes natively on python 
VM. Is this still a blue sky dream?

maurice
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Easy to use distributed system?

[Maurice LING]

Jim Jones wrote:
> I am looking for a system in Python that will easily allow me to distribute 
> processes across multiple systems?So, if I have a function 'foo', I'd 
> like to be able to call something along the lines of
> 
> distribute(foo(x))
> 
> And have the system figure out which node is available for process, and then 
> have the results returned in some sort of callback fashion.
> 
> Any insight is greatly appreciated.
> 

Globus toolkit
pyGlobus, python interface to Globus 
(http://dsd.lbl.gov/gtg/projects/pyGlobus/)
pyLinda (http://www-users.cs.york.ac.uk/aw/pylinda/)

Cheers
maurice
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: state of SOAP and python?

[Maurice LING]

Hi,

I've been using SOAPpy for a number of my work. Looks good.
maurice

Mark Harrison wrote:
> So I'm investigating doing some SOAP work... Any concensus on
> what the best python libraries are for doing this?
> 
> Too bad, xmlrpc is choking on our long longs. :-(
> 
> Many TIA,
> Mark
> 
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: When Python *Eggs* better than Python *distutils*?? What's Eggs?

[Maurice LING]

Xavier Morel wrote:

> [EMAIL PROTECTED] wrote:
> 
>> So basically Python Eggs precompiles and compresses
>> binaries for you so you just have to load it to run
>> your app?
>>
> 
> Nah, Eggs is a packaging system, what you don't have to do is 
> compile/configure, because the packaging does that for you. It also 
> handles things like pre-required packages (the packages that this 
> package uses), even though that doesn't always work atm.
> 
> If you ever used a debian-based linux system, think of Eggs as a 
> Python-specific apt-get.
> 
> The most advanced kind of language-specific installer atm is the Ruby 
> Gems packaging and distribution system (http://docs.rubygems.org/), go 
> check it for more insight of what Eggs wants to be (and I hope it'll 
> evolve to be as good as gems) (and the install software should be named 
> eggs or hatch, not easy_install, ugly).

I think this is one very specific area that one BIG community-wide 
concerted effort is better than a few smaller ones. Is there any chance 
of better coordination in this area?

maurice
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Visual Python, really "Visual"?

[Maurice LING]

Tolga wrote:
> After a very rapid entrance into the Python, I have immediately looked
> for a good IDE. Komodo and Wing IDE look very good and I think they are
> enough. But now, I am searching for a Pyhton environment which should
> look like Delphi / Kylix, Borland's C++ builder or Allegro Common Lisp.
> I have found a plug-in named "Visual Python" and this name naturally
> maked me happy. But is it really "Visual" and does it provide a WYSIWYG
> rapid development environment? Has it drag'n drop buttons, checkboxes,
> radio buttons, forms et cetera?
> 

Please correct me if I am wrong. It seems that most of the python GUI 
builders require or is based on wxPython or others and not tkinter. I am 
using Mac OSX with Fink, so it is a real hassle for me to build wxPython 
and install it (haven't had much luck over the last 4-6 tries).

So, is there any python GUI builders based on tkinter? Personally, I 
prefer to drag and design a screen rather than programming it abstractly.

Thanks
maurice
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Is there anything that pickle + copy_reg cannot serialize?

[Maurice LING]

>>> Since copy_reg lets you specify arbitrary code to serialize arbitrary
>>> objects, you shouldn't run into any single object that you cannot
>>> serialize to a pickle.
>>
>>
>> [snip - example of pickling code objects]
>>
>>
>> I cannot understand 2 things, which I seek assistance for:
>> 1. Is code object the only thing can cannot be pickled (less facing
>> recursion limits)?
> 
> 
> No.  There are lots of objects that cannot be pickled by default.  Any 
> extension type which does not explicitly support it cannot be pickled.  
> Generators cannot be pickled.  Method descriptors can't be pickled.  Et 
> cetera.

Thank Jean-Paul.

Sorry for not specifying clearly enough. Given that copy_reg lets you 
specify arbitrary code to serialize arbitrary objects, of which some are 
taken care of by pickle, in the set of possible Python types,

 >>> import types
 >>> dir(types)
['BooleanType', 'BufferType', 'BuiltinFunctionType', 
'BuiltinMethodType', 'ClassType', 'CodeType', 'ComplexType', 
'DictProxyType', 'DictType', 'DictionaryType', 'EllipsisType', 
'FileType', 'FloatType', 'FrameType', 'FunctionType', 'GeneratorType', 
'InstanceType', 'IntType', 'LambdaType', 'ListType', 'LongType', 
'MethodType', 'ModuleType', 'NoneType', 'NotImplementedType', 
'ObjectType', 'SliceType', 'StringType', 'StringTypes', 'TracebackType', 
'TupleType', 'TypeType', 'UnboundMethodType', 'UnicodeType', 
'XRangeType', '__builtins__', '__doc__', '__file__', '__name__']

What types cannot be serialized by pickle (besides CodeType) and 
requires handling using copy_reg?

Thanks
Maurice


> 
>> 2. In the above example, how copy_reg works with pickle?
> 
> 
> Any time pickle thinks it has found something it cannot pickle, it asks 
> the copy_reg module for some help.  The above example basically teaches 
> the copy_reg module how to give the pickle module the help it needs for 
> code objects.
> 
> Jean-Paul
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Is there anything that pickle + copy_reg cannot serialize?

[Maurice LING]

> Since copy_reg lets you specify arbitrary code to serialize arbitrary 
> objects, you shouldn't run into any single object that you cannot 
> serialize to a pickle.

In http://www.effbot.org/librarybook/pickle.htm, it specifically 
mentions that code objects cannot be pickled and require the use of 
copy_reg, as follows:

import copy_reg
import pickle, marshal, types

#
# register a pickle handler for code objects

def code_unpickler(data):
 return marshal.loads(data)

def code_pickler(code):
 return code_unpickler, (marshal.dumps(code),)

copy_reg.pickle(types.CodeType, code_pickler, code_unpickler)

#
# try it out

CODE = """
print "suppose he's got a pointed stick"
"""

code = compile(CODE, "", "exec")

exec code
exec pickle.loads(pickle.dumps(code))


I cannot understand 2 things, which I seek assistance for:
1. Is code object the only thing can cannot be pickled (less facing 
recursion limits)?
2. In the above example, how copy_reg works with pickle?

Thanks and Cheers
Maurice

> 
> However, both pickle implementations are recursive, so you will be 
> limited by the amount of memory you can allocate for your stack.  By 
> default, this will limit you to something like object graphs 333 edges 
> deep or so (if I'm counting stack frames correctly).  Note that this 
> does not mean you cannot serialize more than 333 objects at a time, 
> merely that if it takes 333 or more steps to go from the first object to 
> any other object in the graph (using the traversal order pickle uses), 
> the pickling will fail.  You can raise this limit, to a point, with 
> sys.setrecursionlimit().
> 
> Jean-Paul
-- 
http://mail.python.org/mailman/listinfo/python-list

Is there anything that pickle + copy_reg cannot serialize?

[Maurice LING]

Hi,

I need to look into serialization for python objects, including codes, 
recursive types etc etc. Currently, I have no idea exactly what needs to 
be serialized, so my scope is to be as wide as possible.

I understand that marshal is extended by pickle to serialize class 
instances, shared elements, and recursive data structures 
(http://www.effbot.org/librarybook/pickle.htm) but cannot handle code 
types. pickle can be used together with copy_reg and marshal to 
serialize code types as well 
(http://www.effbot.org/librarybook/copy-reg.htm).

So my question will be, are there anything that pickle/copy_reg/marshal 
combination cannot serialize? If so, what are the workarounds?

Thanks
Maurice
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: New Python book

[Maurice LING]

I had the opportunity to glance through the book in Borders yesterday. 
On the whole, I think it is well covered and is very readable. Perhaps I 
was looking for a specific aspect, and I find that threads did not get 
enough attention. Looking at the index pages, the topics on threads 
(about 4-5 pages) is mainly found in the context of GUI programming.

maurice


Dick Moores wrote:

> (Sorry, my previous post should not have had "Tutor" in the subject 
> header.)
> 
> Magnus Lie Hetland's new book, _Beginning Python: From Novice to
> Professional_ was published by Apress on Sept. 26 (in the U.S.). My copy
> arrived in the mail a couple of days ago. Very much worth a look, IMHO.
> But what do the experts here think?
> 
> 
> 
> Dick Moores
> [EMAIL PROTECTED]
> 
> 

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Do thread die?

[Maurice LING]

Frithiof Andreas Jensen wrote:

> "Maurice LING" <[EMAIL PROTECTED]> wrote in message
> news:[EMAIL PROTECTED]
> 
> 
>>I do have another dumb question which is OT here. Say aFunc method
>>instantiates a SOAP server that serves forever, will it prevent bFunc
>>from running as a separate thread?
> 
> 
> If the SOAP server thread never sleeps or block, it will effectively stop
> everything else in your program by eating all the CPU time available. If it
> does some IO and other OS functions, probably not because it is likely to
> block on those - I do not know SOAPpy in detail, but it being a socket-based
> server it should end up in a select loop somewhere. i.e. block when no work
> is available. which is what you want.
> 
> 
>>For example,
>>
>>class myClass4:
>> def repeat(self, s): return s+s
>> def aFunc(self, a):
>> import SOAPpy
>> serv = SOAPpy.SOAPServer((a[0], a[1]))
>> serv.registerFunction(repeat)
>> serv.serve_forever()
>> def bFunc(self, b): pass
>> def runAll(self, a, b):
>> threading.Thread(target=self.aFunc, args = (a)).start()
>> threading.Thread(target=self.bFunc, args = (b)).start()
>>
>>if __name__=='__main__': myClass4().runAll(['localhost', 8000], 'hi')
>>
>>Will the 2nd thread (bFunc) ever run since the 1st thread is running
>>forever? Intuitively, I think that both threads will run but I just want
>>to be doubly sure, because some of my program logic depends on the 2nd
>>thread running while the 1st thread acts as a SOAP server or something.
> 
> 
> Both should run independently, sharing the CPU-time available for your
> application. Remember "main" is a thread  too, so you will want "main" to
> hang around while your threads are running and you will want "main" to block
> on something also, thread.join(), time.sleep(), command line parser e.t.c.
> whatever is natural.
> 
> 

Somehow I cannot reconcile your replies because I am essentially asking
the same thing and expanding on the original question with an example of
what I am trying to do, but the replies seems contradictory. Do you mind
to explain a bit more?

thanks
Maurice

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: plateform info.

[Maurice LING]

Mikael Olofsson wrote:

> Monu Agrawal wrote:
> 
>> Hi I want to know whether the program is being run on windows or on
>> Xnix. Is there any variable or method which tells me that it's windows?
> 
> 
> Will this help?
> 
>  >>> import sys
>  >>> sys.platform
> 'win32'
> 
> There is also the platform module, that can give you a lot more 
> information about the your platform. Try help(platform).
> 
> /MiO

Adding onto this (which may be of need), but if this seems to be a
hijack, my apologies..

Is there a way of getting the path to the site-packages directory?
Considering that Mac OSX with Fink installs python libraries in
/sw/lib/python, Mac OSX itself has python libraries in
/System/Library/Frameworks/Python.framework/Versions//lib, etc
etc...

maurice
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Do thread die?

[Maurice LING]

Frithiof Andreas Jensen wrote:

> "Maurice LING" <[EMAIL PROTECTED]> wrote in message
> news:[EMAIL PROTECTED]
> 
> 
>>I do have another dumb question which is OT here. Say aFunc method
>>instantiates a SOAP server that serves forever, will it prevent bFunc
>>from running as a separate thread?
> 
> 
> If the SOAP server thread never sleeps or block, it will effectively stop
> everything else in your program by eating all the CPU time available. If it
> does some IO and other OS functions, probably not because it is likely to
> block on those - I do not know SOAPpy in detail, but it being a socket-based
> server it should end up in a select loop somewhere. i.e. block when no work
> is available. which is what you want.
> 
> 
>>For example,
>>
>>class myClass4:
>> def repeat(self, s): return s+s
>> def aFunc(self, a):
>> import SOAPpy
>> serv = SOAPpy.SOAPServer((a[0], a[1]))
>> serv.registerFunction(repeat)
>> serv.serve_forever()
>> def bFunc(self, b): pass
>> def runAll(self, a, b):
>> threading.Thread(target=self.aFunc, args = (a)).start()
>> threading.Thread(target=self.bFunc, args = (b)).start()
>>
>>if __name__=='__main__': myClass4().runAll(['localhost', 8000], 'hi')
>>
>>Will the 2nd thread (bFunc) ever run since the 1st thread is running
>>forever? Intuitively, I think that both threads will run but I just want
>>to be doubly sure, because some of my program logic depends on the 2nd
>>thread running while the 1st thread acts as a SOAP server or something.
> 
> 
> Both should run independently, sharing the CPU-time available for your
> application. Remember "main" is a thread  too, so you will want "main" to
> hang around while your threads are running and you will want "main" to block
> on something also, thread.join(), time.sleep(), command line parser e.t.c.
> whatever is natural.
> 
> 

Somehow I cannot reconcile your replies because I am essentially asking 
the same thing and expanding on the original question with an example of 
what I am trying to do, but the replies seems contradictory. Do you mind 
to explain a bit more?

thanks
Maurice
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: plateform info.

[Maurice LING]

Mikael Olofsson wrote:
> Monu Agrawal wrote:
> 
>> Hi I want to know whether the program is being run on windows or on
>> Xnix. Is there any variable or method which tells me that it's windows?
> 
> 
> Will this help?
> 
>  >>> import sys
>  >>> sys.platform
> 'win32'
> 
> There is also the platform module, that can give you a lot more 
> information about the your platform. Try help(platform).
> 
> /MiO

Adding onto this (which may be of need), but if this seems to be a 
hijack, my apologies..

Is there a way of getting the path to the site-packages directory? 
Considering that Mac OSX with Fink installs python libraries in 
/sw/lib/python, Mac OSX itself has python libraries in 
/System/Library/Frameworks/Python.framework/Versions//lib, etc 
etc...

maurice
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: reading files with error

[Maurice Ling]

[EMAIL PROTECTED] wrote:


I have written a program to do something similar.  My strategy is:
* use os.read() to read 512 bytes at a time
* when os.read fails, seek to the next multiple of 512 bytes
  and write '\0' * 512 to the output
I notice this code doesn't deal properly with short reads, but in my experience
they never happen (because the disk error makes an entire block unreadable, and
a block is never less than 512 bytes)

I use this code on a unix-style system.

def dd(src, target, bs=512):
   src = os.open(src, os.O_RDONLY)
   if os.path.exists(target):
   target = os.open(target, os.O_WRONLY | os.O_APPEND, 0600)
   existing = os.lseek(target, 0, SEEK_END)
   else:
   existing = 0
   target = os.open(target, os.O_WRONLY | os.O_CREAT, 0600)

   total = os.lseek(src, 0, SEEK_END) / bs
   os.lseek(src, existing, SEEK_SET)
   os.lseek(target, existing, SEEK_SET)

   if existing: print "starting at", existing
   i = existing / bs
   f = 0
   lastrem = -1

   last = start = time.time()
   while 1:
   try:
   block = os.read(src, bs)
   except os.error, detail:
   if detail.errno == errno.EIO:
   block = "\0" * bs
   os.lseek(src, (i+1) * bs, SEEK_SET)
   f = f + 1
   else:
   raise  
   if block == "": break


   i = i + 1
   os.write(target, block)

   now = time.time()
   if i % 1000 or now - last < 1: continue
   last = now

   frac = i * 1. / total
   rem = int((now-start) * (1-frac) / frac)
   if rem < 60 or abs(rem - lastrem) > .5:
   rm, rs = divmod(rem, 60)
   lastrem = rem
   spd = i * 512. / (now - start) / 1024 / 1024
   sys.stderr.write("%8d %8d %8d %3.1f%% %6d:%02d %6.1fMB/s\r"
   % (i, f, i-f, i * 100. / total, rm, rs, spd))
   sys.stderr.write("\n")
 


Sorry but what are SEEK_END and SEEK_SET?

Maurice

--
Maurice Han Tong LING, BSc(Hons)(Biochem), AdvDipComp, CPT, SSN, FIFA, 
MASBMB, MAMBIS, MACM

Doctor of Philosophy (Science) Candidate, The University of Melbourne
mobile: +61 4 22781753, +65 96669233
mailing address: Department of Zoology, The University of Melbourne
 Royal Parade, Parkville, Victoria 3010, Australia
residence: 9/41 Dover Street, Flemington, Victoria 3031, Australia
resume: http://maurice.vodien.com/maurice_resume.pdf
www: http://www.geocities.com/beldin79/

begin:vcard
fn:Maurice Ling
n:Ling;Maurice
org:The University of Melbourne;Department of Zoology
adr:;;Gate 12, Genetics Lane;Parkville;Victoria;3010;Australia
email;internet:[EMAIL PROTECTED]
title:Ph.D. Candidate
tel;cell:+61 4 22781753
x-mozilla-html:FALSE
url:http://www.geocities.com/beldin79/
version:2.1
end:vcard

-- 
http://mail.python.org/mailman/listinfo/python-list

reading files with error

[Maurice LING]

Hi,

I'm trying to read some files (video files) that seems to have some 
errors in it. Basically, I cannot copy it out of discs as that gives me 
an error message but I can still play the file using a media player like 
VLC or QuickTime. I understand that copying a file will also invoke 
checking routines as well, and I am guessing that the file may have some 
parity-bit error or something.

Am I able to use Python to force read the entire file (full length)? 
That is, do not check the read for errors. I know that this is insideous 
in many uses but in media files, it may only result in a skipped frame 
or something. What I've done is something like this:

 >>> f = open('/Volumes/NEW/gameday/morning.dat', 'rb')
 >>> data = f.read()
 >>> o = open('/Users/mauriceling/Desktop/o.dat', 'wb')
 >>> f.close()
 >>> o.write(data)
 >>> o.close()

What I've noticed is this:
1. sometimes it (Python) only reads to roughly the point of initial copy 
error (I try to take note of how far drag-and-drop copying proceeds 
before it fails)
2. sometimes it is able to read pass the drag-and-drop copying 
fail-point but may or may not be full length.

What determines if Python is able to make it pass drag-and-drop copying 
fail-point?

Is there anyway to do what I want, force read full length?

Thanks and cheers
Maurice
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Do thread die?

[Maurice LING]

> 
>> My current implementation may be ugly. I have a class
>>
>> class myThread(threading.Thread):
>> def __init__(self, func):
>> self.func = func
>> threading.Thread.__init__(self)
>> def run(self):
>> print '%s function running' % self.func
>> self.func()
>>
>> which is used in
>>
>> class myClass: #using myThread
>> def __init__(self): pass
>> def aFunc(self): pass
>> def bFunc(self): pass
>> def runAll(self):
>> myThread(self.aFunc).start()
>> myThread(self.bFunc).start()
> 
> 
> 
> There is a school of thought that says that a derived object should 
> never be altered such that it cannot be used as its parent could. I 
> happen to agree largely with this. Now, your MyThread implementation is 
> _reducing_ the functionality of its ancestor Thread object such that you 
> could not use a myThread in place of a Thread. I believe that you should 
> not be subclassing Thread to do this. Your myClass doesn't need it 
> anyway. Look at this modified myClass:
> 
> class myClass2:
> def aFunc(self): pass
> def bFunc(self): pass
> def runAll(self):
> threading.Thread(target=self.aFunc).start()
> threading.Thread(target=self.bFunc).start()
> 
Thanks everyone. Furthering that, is the following legal?

class myClass3:
 def aFunc(self, a): pass
 def bFunc(self, b): pass
 def runAll(self, a, b):
 threading.Thread(target=self.aFunc, args = (a)).start()
 threading.Thread(target=self.bFunc, args = (b)).start()

I do have another dumb question which is OT here. Say aFunc method 
instantiates a SOAP server that serves forever, will it prevent bFunc 
from running as a separate thread?

For example,

class myClass4:
 def repeat(self, s): return s+s
 def aFunc(self, a):
 import SOAPpy
 serv = SOAPpy.SOAPServer((a[0], a[1]))
 serv.registerFunction(repeat)
 serv.serve_forever()
 def bFunc(self, b): pass
 def runAll(self, a, b):
 threading.Thread(target=self.aFunc, args = (a)).start()
 threading.Thread(target=self.bFunc, args = (b)).start()

if __name__=='__main__': myClass4().runAll(['localhost', 8000], 'hi')

Will the 2nd thread (bFunc) ever run since the 1st thread is running 
forever? Intuitively, I think that both threads will run but I just want 
to be doubly sure, because some of my program logic depends on the 2nd 
thread running while the 1st thread acts as a SOAP server or something.

Thanks and Cheers
Maurice
-- 
http://mail.python.org/mailman/listinfo/python-list

Do thread die?

[Maurice LING]

Hi,

I just have a simple question about threads. My classes inherits from 
threading.Thread class. I am calling threading.Thread.run() method to 
spawn a few threads to parallel some parts of my program. No thread 
re-use, pooling, joining ... just plainly spawn a thread, run a routine.

So, at the end of run(), what happens to the thread? Just die?

While I am on it, can threading.Thread.run() accept any parameters?

My current implementation may be ugly. I have a class

class myThread(threading.Thread):
 def __init__(self, func):
 self.func = func
 threading.Thread.__init__(self)
 def run(self):
 print '%s function running' % self.func
 self.func()

which is used in

class myClass: #using myThread
 def __init__(self): pass
 def aFunc(self): pass
 def bFunc(self): pass
 def runAll(self):
 myThread(self.aFunc).start()
 myThread(self.bFunc).start()

if __name__=='__main__': myClass().runAll()

Is this a good way?

Thanks and cheers
Maurice
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Calling ftp commands from python

[Maurice LING]

Thierry Lam wrote:
> Is it possible to run an ftp command to connect to some remote computer
> on the network.
> 
> For example, if I want to retrieve some data from
> \\remcomputer\datafiles on the network and copy it to my local
> computer, how do I do it in python on the Unix side?
> 
> I don't want to use mount since I don't have permission.
> 
> Thanks
> Thierry
> 

I use ftplib in the standard libraries.

from ftplib import FTP

def grab_geneontology():
 """Function to download gene ontology file."""
 ftp = FTP('ftp.geneontology.org')
 ftp.login()
 ftp.cwd('/pub/go/ontology')
 ftp.retrbinary('retr gene_ontology.obo', 
open('gdata/gene_ontology', 'wb').write)
 ftp.quit()

if __name__ == '__main__': grab_geneontology()

maurice
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Standalone applications ?

[Maurice LING]

henne wrote:
>  > I should have added that my platform is Linux.
> 
> http://davidf.sjsoft.com/mirrors/mcmillan-inc/install1.html
> 
> Squeeze works on Linux too.

It tells me that the installer files are not found.
M
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Java RMI-like services in Python

[Maurice LING]

Thank you
Harald and Alan,

Pyro seems to be used widely in this area but PyLinda is much cleaner 
and simpler to use.

However, given that the object or class to be executed by execution 
machines are not known at development time, I wish to seek you advice on 
remote class loading.

Can an object (containing methods) be loaded into PyLinda's TupleSpace? 
Sorry, this may be a dumb question as lists are technically objects. 
Just want to be doubly sure.

Thanks and cheers
maurice
-- 
http://mail.python.org/mailman/listinfo/python-list

Java RMI-like services in Python

[Maurice LING]

Hi,

I am wondering if Python has services or frameworks that does the same 
as Java RMI?

What I am seeking is to do "pseudo-clustering". That is, a server will 
contains a program to control what is needed for execution. This will be 
pretty much like process management. Call this controller, 
HeadControl.py. What HeadControl.py can do is to expose a SOAP interface.

Each client machine will have Executor.py. So when Executor.py is 
executed, it will connect to HeadControl through SOAP. HeadControl will 
then spawn a thread to represent the connection.

Now, this part is possible in Java, but I am not sure if it is possible 
in Python. Executor will find out from HeadControl what it can do and 
downloads the corresponding objects and data from HeadControl and 
executes it and sends results back to HeadControl.

2 scenarios that this might work. Firstly, Executor can be part of a 
client program. In this case, temporary files may be written to the 
client's HDD. Of course, due to security, this method may not be 
desirable. Secondly, Executor may be made into a web applet kinda thing. 
In this case, writing to HDD may be impossible.

I know something like this had been achieved in Java 
(http://www-128.ibm.com/developerworks/java/library/j-super.html) but 
wondering if it is possible in Python. Is so, how?

Thanks.

Cheers
Maurice
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Connecting to Firebird database using Kinterbasdb+Python

[Maurice LING]

Mathias Waack wrote:
> Maurice LING wrote:
> 
> 
>>I've been using FB1.5 and access the database using Kinterbasdb +
>>Python. My connection is established using kinterbasdb.connect() method
>>and the parameters host, dns, database, user, password are all defaulted
>>to 'None'.
>>
>>On my own machine running Mac OSX 10.3, I can connect using the following:
>>host = 'localhost'
>>database = ''
>>user = ''
>>password = ''
>>
>>At the same time, I can also connect if I set host=None on my machine.
>>
>>However, I cannot use 'localhost' on a shared Linux machine (not
>>allowed. Don't ask why, system admin's mandate. And the Linux machine is
>>without inetd). So when I set host=None, I get this error:
> 
> 
> Just for my understanding: if you start your script on the machine hosting
> the DB you're able to connect. If you start it on a different machine, you
> are not able to connect without giving a hostname? Whats wrong with
> offering the the name (or IP-address) of your MacOS-box instead of
> 'localhost' to the script?
> 

Hi Mathias,

What I am trying to do is "port" a workable program from my own machine 
(Mac OSX) to a larger machine (Linux). So, the DB and the program are 
also on the same Linux machine.

On the Linux machine, I cannot use localhost, so I set host parameter in 
kinterbasdb.connect() method to None, which is the default kinterbasdb 
uses. On Linux machine, I get the error as posted. On my Mac OSX 
machine, initiating host parameter to 'localhost' or None has no 
difference, it works.

Thanks and cheers
Maurice
-- 
http://mail.python.org/mailman/listinfo/python-list

Connecting to Firebird database using Kinterbasdb+Python

[Maurice LING]

Hi,

I've been using FB1.5 and access the database using Kinterbasdb + 
Python. My connection is established using kinterbasdb.connect() method 
and the parameters host, dns, database, user, password are all defaulted 
to 'None'.

On my own machine running Mac OSX 10.3, I can connect using the following:
host = 'localhost'
database = ''
user = ''
password = ''

At the same time, I can also connect if I set host=None on my machine.

However, I cannot use 'localhost' on a shared Linux machine (not 
allowed. Don't ask why, system admin's mandate. And the Linux machine is 
without inetd). So when I set host=None, I get this error:

Connecting to Muscopedia Database
Connection: 
localhost:/mnt/disk/home/mling/muscorian/BioDatabases/muscopedia.fdb:mouse:mouse
 
<-- default
None <-- actual host (with 'print self.dbhost' where 
kinterbasdb.connect(host = self.dbhost,.)
OperationalError
(-902, 'isc_attach_database: Unable to complete network request to host 
"localhost".. Failed to establish a connection.. Connection refused. ')
   File "muscorian.py", line 641, in main
 result = mosys.run(command)

   File "muscorian.py", line 597, in run
 elif (command[0] == 'linkmuscopedia'): 
self.commandLinkMuscopediaDB(command)

   File "muscorian.py", line 129, in commandLinkMuscopediaDB
 self.muscopedia = PubMedGrabber.FBUtilities(str(command[1]))

   File "abcrawl/PubMedGrabber.py", line 340, in __init__
 user = self.dbuser, password = self.dbpwd, charset='UNICODE_FSS')

   File "/usr/lib/python2.2/site-packages/kinterbasdb/__init__.py", line 
470, in connect
 return Connection(*args, **keywords_args)

   File "/usr/lib/python2.2/site-packages/kinterbasdb/__init__.py", line 
608, in __init__
 self._C_con = _k.attach_db(dsn, dpb, dialect)

Any ideas or solutions?

Thanks in advance.

Cheers
maurice
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Migrating from Windows to OS X

[Maurice LING]

[EMAIL PROTECTED] wrote:
> Hello, fellow programmers!
> 
> I am sitting in front of a nice new PowerBook portable which has OS
> 10.4 installed.  The Python.org web site says that Apple has shipped OS
> 10.4 with Python 2.3.5 installed.  How exactly do I access this?  I
> have searched through the Applications and Libraries folders.  I found
> the site-packages directory, but nothing other than that.
> 
> Thanks for your help,
> John Ladasky
> 

If I recall correctly, the actual Python installation is in 
/System/Library/Frameworks/Python.framework but the site-package is 
symlinked to /Library/Python. Do not attempt to touch that. This 
installation can be accessed in Terminal by typing "pythonw".

Are you or will you be using package managers such as darwinports or Fink?

maurice
-- 
http://mail.python.org/mailman/listinfo/python-list

which ports to use for SOAP?

[Maurice LING]

Hi,

I'm writing something that specifies the use of SOAP. One requirement 
that fumbles me is the port number(s) to use. Is there any way to find 
out which ports are not used by the system? I've looked in the library 
reference and doesn't seems to get anything.

Is there a problem with say 2 programs using the same ports?

Thanks in advance.

Maurice
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: UML to Python/Java code generation

[Maurice LING]

Thanks James. I've googled a few times and it is always tools that 
generates UML from Python codes that comes up in the top hits, which 
brings me to wonder if there is something that does the opposite.

maurice

James wrote:
>>Is there any UML tools that is able to take UML and generate Python codes?
> 
> 
> Dia2code generates Python from UML.
> Boa Constructor generates UML from Python.
> PyUt.
> Object Domain's UML Tool (Commercial)
> 
> You need to use Google.
> 
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Generating .pyo from .py

[Maurice LING]

say your code is "test.py" and you usually run it as "python test.py", 
then the "-O" option will generate .pyo files.
That is "python -O test.py"

Cheers maurice

skn wrote:
> Hello.,
> 
> Does the python compiler provide an option to generate a .pyo(optimized byte
> code file) from a .py (source file)?
> 
> For generating .pyc I know that I only have to pass the source file name as
> an argument to py_compile.py.
> But for generating .pyo, I could not find anything.
> 
> Any clues/help will be greatly appreciated.
> 
> With best regards,
> skn
> 
> 
-- 
http://mail.python.org/mailman/listinfo/python-list

UML to Python/Java code generation

[Maurice LING]

Hi,

Is there any UML tools that is able to take UML and generate Python codes?

Cheers
Maurice
-- 
http://mail.python.org/mailman/listinfo/python-list

provide 3rd party lib or not... philosophical check

[Maurice LING]

Hi,

Just a philosophical check here. When a program is distributed, is it 
more appropriate to provide as much of the required 3rd party libraries, 
like SOAPpy, PLY etc etc, in the distribution itself or it is the 
installer's onus to get that part done?

Cheers
Maurice
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: something like CPAN, PPMs?

[Maurice LING]

deelan wrote:
> Maurice LING wrote:
> 
>> Hi Alex,
>>
>> I am actually working on something like that as an academic project. 
>> At this stage, at least for the purpose of my scope, it will not be as 
>> extensive as CPAN but a set of mechanisms for the same effect for Python.
> 
> 
> don't foget to keep an eye on python's eggs:
> 
> <http://peak.telecommunity.com/DevCenter/PythonEggs>
> 
> and related blog posts:
> 
> <http://dirtsimple.org/2005/05/eggs-get-closer-to-hatching.html>
> <http://dirtsimple.org/2005/05/dirt-simple-download-and-install-cpan.html>
> <http://dirtsimple.org/2005/05/easyinstall-new-era-in-python-package.html>
> 
> HTH.
> 

Thanks,

I think PythonEggs will have increasing effect in the future as Python 
is moving more towards enterprise development, as what we hope to see. 
Although it is stated in their website that using PythonEggs does not 
automatically confer platform-independence, it might still be a good way 
for enterprises, especially on the reverse engineering issue.

My approach now is to work out a prototype system that is made as a 3rd 
party installation process for my software. My software requires tools 
like PLY, SciPy etc etc. My idea is that by doing "python setup.py 
install" to install my stuffs, a function in setup.py calls a system (in 
my software, call it centipyde) and installs the pre-requisites as 
required. "Centipyde" can then be developed into a better management tool.

As I've posted in many mailing lists, I've intended this to be an 
academic project (any other contradictory decisions made will be 
announced). However, I do welcome opinions.

Cheers
Maurice
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: something like CPAN, PPMs?

[Maurice LING]

Hi Alex,

I am actually working on something like that as an academic project. At 
this stage, at least for the purpose of my scope, it will not be as 
extensive as CPAN but a set of mechanisms for the same effect for Python.

maurice

Alex Gittens wrote:
> I'm new to Python from Perl, and loving it. Has there ever been any
> discussion of creating a similar resource as CPAN for Python, or a
> similar distribution method as PPMs? Seems like it would make a great
> language even better.
> 
> Alex
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Melbourne (Australia) Python User Group

[Maurice LING]

Hi Anthony,

count me in...

Cheers
Maurice

Anthony wrote:
> Hi all,
> 
> I'm interested in starting a Python user group in Melbourne,
> Australia. So far there seems to have been a lot of interest from
> various parties, but for whatever reasons it's fizzled out. So I've
> decided that if there's going to be a user group, I'll have to start
> it myself (hopefully without the meetup.com albatross).
> 
> If you're in Melbourne, you're a python programmer and you're
> interested in meeting up, drop me an email and let me know. I'll
> probably also contact the Linux user groups in the area, but if you
> know of any other groups that would be interested, please let me know.
> If you want to forward this email on to other people or groups, feel
> free to do so.
> 
> Thanks,
> 
> Anthony
> 

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Is Python suitable for a huge, enterprise size app?

[Maurice LING]

Peter Hansen wrote:

> Maurice LING wrote:
> 
>> It makes big difference (legally) to if the codes are there and 
>> someone sees it, to if the codes are locked in some packaged or zipped 
>> form and someone reverse-engineer it. It is legally as different as if 
>> you drop money on the ground and I pick it up, to pick-pocketing you 
>> and take the money.
>>
>> Nobody seems to be able to understand this simple logic.
> 
> 
> So you're saying that reverse engineering Java bytecode is illegal, 
> while doing the same with Python bytecode is not?  Or something like 
> that?  (And you're a lawyer, right?  Because if you're not, and you're 
> not citing your sources, why is it we should put any value in these 
> comments about what is (legally) true?)
> 
> -Peter

What I'm saying is reverse engineering anything is illegal unless 
allowed by the laws of the state, be it  bytecodes or 
compiled executables, but if the original source codes are there, you 
can see it.

To put it sexually and crudely (to get the idea across), if a female 
strips and parade in front of me, I'm not violating any law to open my 
eyes and look at it (whether morally or religiously right is a total 
different matter) but it is criminal for me to grab any moving female, 
strip her and look at her naked. Can see the point?

maurice
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Is Python suitable for a huge, enterprise size app?

[Maurice LING]

Peter Hansen wrote:

> Maurice LING wrote:
> 
>> It makes big difference (legally) to if the codes are there and 
>> someone sees it, to if the codes are locked in some packaged or zipped 
>> form and someone reverse-engineer it. It is legally as different as if 
>> you drop money on the ground and I pick it up, to pick-pocketing you 
>> and take the money.
>>
>> Nobody seems to be able to understand this simple logic.
> 
> 
> So you're saying that reverse engineering Java bytecode is illegal, 
> while doing the same with Python bytecode is not?  Or something like 
> that?  (And you're a lawyer, right?  Because if you're not, and you're 
> not citing your sources, why is it we should put any value in these 
> comments about what is (legally) true?)
> 
> -Peter

I am not saying reverse engineering Python bytecodes is legal while 
reverse engineering Java bytecodes is illegal. Please do not put words 
into my mouth, I am just saying "codes locked in some packaged or zipped 
form", reverse engineering is illegal unless specifically approved, be 
it Python bytecodes, Java bytecodes, Smalltalk, compiled executables etc 
etc... AND THIS HAD BEEN DISCUSSED IN OTHER THREADS, like in "bytecode 
non-backcompatibility" in April 2005 of python-list.

I am not a lawyer. Based on what I've read and seen, I won't try it. 
Below is an example of such articles by a law firm, citating case laws. 
Please google the term "reverse engineering computer sources" and you 
will find heaps of materials. You might like to test the depth of your 
pockets in courtroom, be my guest


 From http://www.jenkins-ip.com/serv/serv_6.htm

COMPUTER PROGRAMS

In the case of computer programs, the EU directive states (11) that the 
ideas and principles underlying a program are not protected by 
copyright, and that (12) logic, algorithms and programming languages may 
to some extent comprise ideas and principles.

Analysis of the function of a program (but not decompilation (13))is 
permitted under Article 5.3, if it is carried out by a licensed user in 
the normal use of the program.

Reverse engineering is allowed under Article 6, but only for the single 
purpose of producing an interoperable program (rather than a competing 
program).

For this purpose, in addition to reverse engineering itself (i.e. 
producing a high level version of the code) subsequent forward 
engineering to produce the interoperable program is permitted.

However, the reverse engineer has to cross a host of formidable barriers 
before he can make use of this right;

1. It must be indispensable to reverse engineer to obtain the 
necessary information.
2. The reverse engineering has to be by a licensee or authorised user.
3. The necessary information must not already have been readily 
available to those people.
4. Only the parts of the program necessary for interoperability 
(i.e. the interfaces) can be reproduced.
5. The information generated by the reverse engineering cannot be 
used for anything other than achieving interoperability of an 
independently created program.
6. The information cannot be passed on to others except where 
necessary for this purpose.
7. The information obtained cannot be used to make a competing 
program (rather than just an interoperable one).
8. The "legitimate interests" of the copyright owner or "normal 
exploitation" of the program must not be prejudice.

Thus, far from creating a general right to reverse engineer, these 
provisions create only the smallest of openings for the reverse 
engineer; they are intended for use only to defeat locked, confidential, 
proprietary interfaces.


(11) Council Directive 96/9/EC 11 March 1996; enacted in the UK as S.I. 
1997 No. 3032 13th recital
(12) 14th recital
(13) See, for example, Copyright, Designs and Patent Act 1988, Section 
29(4)

  SEGA ENTERPRISES LTD v. ACCOLADE INC 977 F.2D 1510 (9TH CIR. 1992)

This US software copyright case concerned Sega's video game console and 
cartridges. The cartridges had a 20-25 byte code segment which was 
interrogated by the console, as a security measure.

Accolade disassembled the code which was common to three different Sega 
games cartridges, to find the security segment, and included it in 
competing games cartridges.

The Ninth Circuit held this disassembly to be a permitted "fair use" of 
the copyright in the games programs.

ATARI v. NINTENDO 975 F.2D 872 (FED. CIR. 1992)

This US software copyright case concerned Nintendo’s NES video game 
console and cartridges. The cartridges contained a microprocessor, and 
program code, and was interrogated by the console microprocessor, as a 
security measure, like the Sega system. The security was potentially a 
two-way process, with the console checking for a valid cartridge and the 
potential for the cartridge

Re: Is Python suitable for a huge, enterprise size app?

[Maurice LING]

> 
>> You can
>>see Python going down the sewer pipes, right on their faces.  Two,
>>security.  "This python sounds pretty interesting.  Tell me about the
>>security.  How can we prevent people from stealing our source code,
>>which we just spent millions developing?  ...  Hmm, trust the developers
>>out there not to peek?  Oh, sure, let's use it."  
> 
> 
> Just like Java, which is so easy to reverse-engineer...
> 
> 
It makes big difference (legally) to if the codes are there and someone 
sees it, to if the codes are locked in some packaged or zipped form and 
someone reverse-engineer it. It is legally as different as if you drop 
money on the ground and I pick it up, to pick-pocketing you and take the 
money.

Nobody seems to be able to understand this simple logic.

Yes, Java class files can be reverse-engineered quite easily but the act 
of doing that is criminal, unless performed under specified Council 
Directives (in EU) or under any statutory law in specific countries. But 
the act of looking at the codes if it is there is not criminal, just 
like reading a book in bookstore. If anyone can program in binary today, 
no codes will be safe with them anyway...

maurice
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: hard memory limits

[Maurice LING]

Bengt Richter wrote:

> On Sat, 07 May 2005 14:03:34 +1000, Maurice LING <[EMAIL PROTECTED]> wrote:
> 
> 
>>John Machin wrote:
>>
>>>On Sat, 07 May 2005 02:29:48 GMT, [EMAIL PROTECTED] (Bengt Richter) wrote:
>>>
>>>
>>>
>>>>On Sat, 07 May 2005 11:08:31 +1000, Maurice LING <[EMAIL PROTECTED]> wrote:
>>>>
>>>>
>>>>>It doesn't seems to help. I'm thinking that it might be a SOAPpy 
>>>>>problem. The allocation fails when I grab a list of more than 150k 
>>>>>elements through SOAP but allocating a 1 million element list is fine in 
>>>>>python.
>>>>>
>>>>>Now I have a performance problem...
>>>>>
>>>>>Say I have 3 lists (20K elements, 1G elements, and 0 elements), call 
>>>>>them 'a', 'b', and 'c'. I want to filter all that is in 'b' but not in 
>>>>>'a' into 'c'...
>>>>>
>>>>>
>>>>>
>>>>>>>>a = range(1, 10, 5)
>>>>>>>>b = range(0, 100)
>>>>>>>>c = []
>>>>>>>>for i in b:
>>>>>
>>>>>... if i not in a: c.append(i)
>>>>>...
>>>>>
>>>>>This takes forever to complete. Is there anyway to optimize this?
>>>>>
>>>>
>>>>Checking whether something is in a list may average checking equality with
>>>>each element in half the list. Checking for membership in a set should
>>>>be much faster for any significant size set/list. I.e., just changing to
>>>>
>>>>a = set(range(1, 10, 5))
>>>>
>>>>should help. I assume those aren't examples of your real data ;-)
>>>>You must have a lot of memory if you are keeping 1G elements there and
>>>>copying a significant portion of them. Do you need to do this file-to-file,
>>>>keeping a in memory? Perhaps page-file thrashing is part of the time 
>>>>problem?
>>>
>>>
>>>Since when was 100 == 1G??
>>>
>>>Maurice, is this mucking about with 1M or 1G lists in the same
>>>exercise as the "vm_malloc fails when allocating a 20K-element list"
>>>problem? Again, it might be a good idea if you gave us a little bit
>>>more detail. You haven't even posted the actual *PYTHON* error message
>>>and stack trace that you got from the original problem. In fact,
>>>there's a possible interpretation that the (system?) malloc merely
>>>prints the vm_malloc message and staggers on somehow ...
>>>
>>>Regards,
>>>John
>>
>>This is the exact error message:
>>
>>*** malloc: vm_allocate(size=9203712) failed (error code=3)
>>*** malloc[489]: error: Can't allocate region
>>
>>Nothing else. No stack trace, NOTHING.
>>
> 
> 1. Can you post minimal exact code that produces the above exact error 
> message?
> 2. Will you? ;-)
> 
> Regards,
> Bengt Richter

I've re-tried the minimal code mimicking the error in interactive mode 
and got this:

 >>> from SOAPpy import WSDL
 >>> serv = 
WSDL.Proxy('http://eutils.ncbi.nlm.nih.gov/entrez/eutils/soap/v1.1/eutils.wsdl' 
)
 >>> result = serv.run_eSearch(db='pubmed', term='mouse', retmax=50)
*** malloc: vm_allocate(size=9121792) failed (error code=3)
*** malloc[901]: error: Can't allocate region
Traceback (most recent call last):
   File "", line 1, in ?
   File "/sw/lib/python2.3/site-packages/SOAPpy/Client.py", line 453, in 
__call__
 return self.__r_call(*args, **kw)
   File "/sw/lib/python2.3/site-packages/SOAPpy/Client.py", line 475, in 
__r_call
 self.__hd, self.__ma)
   File "/sw/lib/python2.3/site-packages/SOAPpy/Client.py", line 347, in 
__call
 config = self.config)
   File "/sw/lib/python2.3/site-packages/SOAPpy/Client.py", line 212, in 
call
 data = r.getfile().read(message_len)
   File "/sw/lib/python2.3/socket.py", line 301, in read
 data = self._sock.recv(recv_size)
MemoryError
 >>>


When changed retmax to 15, it works nicely.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: hard memory limits

[Maurice LING]

John Machin wrote:
> On Sat, 07 May 2005 02:29:48 GMT, [EMAIL PROTECTED] (Bengt Richter) wrote:
> 
> 
>>On Sat, 07 May 2005 11:08:31 +1000, Maurice LING <[EMAIL PROTECTED]> wrote:
>>
>>>It doesn't seems to help. I'm thinking that it might be a SOAPpy 
>>>problem. The allocation fails when I grab a list of more than 150k 
>>>elements through SOAP but allocating a 1 million element list is fine in 
>>>python.
>>>
>>>Now I have a performance problem...
>>>
>>>Say I have 3 lists (20K elements, 1G elements, and 0 elements), call 
>>>them 'a', 'b', and 'c'. I want to filter all that is in 'b' but not in 
>>>'a' into 'c'...
>>>
>>>
>>>>>>a = range(1, 10, 5)
>>>>>>b = range(0, 100)
>>>>>>c = []
>>>>>>for i in b:
>>>
>>>... if i not in a: c.append(i)
>>>...
>>>
>>>This takes forever to complete. Is there anyway to optimize this?
>>>
>>
>>Checking whether something is in a list may average checking equality with
>>each element in half the list. Checking for membership in a set should
>>be much faster for any significant size set/list. I.e., just changing to
>>
>> a = set(range(1, 10, 5))
>>
>>should help. I assume those aren't examples of your real data ;-)
>>You must have a lot of memory if you are keeping 1G elements there and
>>copying a significant portion of them. Do you need to do this file-to-file,
>>keeping a in memory? Perhaps page-file thrashing is part of the time problem?
> 
> 
> Since when was 100 == 1G??
> 
> Maurice, is this mucking about with 1M or 1G lists in the same
> exercise as the "vm_malloc fails when allocating a 20K-element list"
> problem? Again, it might be a good idea if you gave us a little bit
> more detail. You haven't even posted the actual *PYTHON* error message
> and stack trace that you got from the original problem. In fact,
> there's a possible interpretation that the (system?) malloc merely
> prints the vm_malloc message and staggers on somehow ...
> 
> Regards,
> John

This is the exact error message:

*** malloc: vm_allocate(size=9203712) failed (error code=3)
*** malloc[489]: error: Can't allocate region

Nothing else. No stack trace, NOTHING.

maurice
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: hard memory limits

[Maurice LING]

James Stroud wrote:

> Sorry Maurice, apparently in bash its "ulimit" (no n). I don't use bash, so I 
> don't know all of the differences offhand. Try that.
> 
> James

Thanks guys,

It doesn't seems to help. I'm thinking that it might be a SOAPpy 
problem. The allocation fails when I grab a list of more than 150k 
elements through SOAP but allocating a 1 million element list is fine in 
python.

Now I have a performance problem...

Say I have 3 lists (20K elements, 1G elements, and 0 elements), call 
them 'a', 'b', and 'c'. I want to filter all that is in 'b' but not in 
'a' into 'c'...

 >>> a = range(1, 10, 5)
 >>> b = range(0, 100)
 >>> c = []
 >>> for i in b:
... if i not in a: c.append(i)
...

This takes forever to complete. Is there anyway to optimize this?

Thanks in advance

Cheers
Maurice
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: hard memory limits

[Maurice LING]

Hi everyone,

thanks for your help.

Yes, I'm using Mac OSX 1.3 with 256MB Ram. Each element in the list is a 
float. The list is actually a retrieved results of document IDs from 
SOAP interface. And Mac OSX does not have 'unlimit' command as shown,

Maurice-Lings-Computer:~ mauriceling$ unlimit
-bash: unlimit: command not found
Maurice-Lings-Computer:~ mauriceling$ which unlimit
Maurice-Lings-Computer:~ mauriceling$ sh unlimit
unlimit: unlimit: No such file or directory
Maurice-Lings-Computer:~ mauriceling$


Cheers
Maurice
-- 
http://mail.python.org/mailman/listinfo/python-list

hard memory limits

[Maurice LING]

Hi,

I think I've hit a system limit in python when I try to construct a list 
of 200,000 elements. My error is

malloc: vm_allocate (size = 2400256) failed..

Just wondering is this specific to my system or what? Will adding more 
RAM helps in this case?

Thanks and cheers
Maurice
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: So many things that need to be decided....

[Maurice LING]

Hi,

It seems that you are just starting on Python, same as my situation a 
year ago. After almost going down the same route as you do, I tell 
myself just start on something. You can decide on everything but by 
the time you've decided, everything changes again.

Initially, I started on Java then came to Python. Ok, now you've decided 
on Python, GUI library and Editor isn't an issue. Despite all the 
negativities with PyDev with Eclipse, I am still using it.

I really don't see why you should lay everything in stone before you 
start. Things may or may not fall into place. Worry about it when it 
worries you.

"do not distress yourself with imaginings" -desiderata

Good luck.

Cheers
maurice

Heather Stovold wrote:
> Wow - deciding to program in python sure requires a lot of decisions!
> 
> So far:
> 
> I've decided on python for the programming language.
> I've decided on wxpython for the GUI
> I've decided on DrPython for the Editor
> 
> I still need to decide on a database   I've really only used Access,
> and my SQL skills aren't that great.  It would also need to be free
> Any suggestions??
> 
> Am I going to need a program/package for a report writer?  or that might
> depend on the database I use?
> 
> I've never had to make so many decisions to program in my life, because I've
> generally only had a couple of options, or those options were decided by
> other people.
> 
> I really appreciate the help this newsgroup has provided - both for my
> previous post - and for the reading of other posts I've done.   I'm sure
> I'll have a million more questions soon!
> 
> Thanks for the help!
> 
> Heather
> 
> 
> 
> 
-- 
http://mail.python.org/mailman/listinfo/python-list

"implementation" of PEP 262 as an academic project

[Maurice LING]

Hi all,
I reckoned that if I'm on this work, I might as well make it into an 
academic engineering-type project for my pass degree. Hence, I am 
sending this posting out to everyone to inform of my intentions. Despite 
the possible interests in this work, academic requires a 'personal' 
project with defined work.

Of course, I do hope that comments and suggestions continue to flow like 
milk and honey. If in event I do decide against making it into an 
academic project or when it had cease to be academically-focused (i.e. 
submitted and passed my thesis), I will have the onus to inform everyone 
again through comp.lang.python (python-list) and catalog-sig mailing list.

Thank you everyone for your kind understanding.
Cheers
Maurice

Hi,
I've just read PEP 262 last night and finds that it does more or less 
describes what I have in mind. However, I am not sure if there is every 
need for such a descriptive database file or something slimmer, like 
Fink's .info files will suffice.

An example of Fink's .info file is:

Package: g77
Version: 3.4.1
Revision: 1
BuildDependsOnly: true
Source: mirror:gnu:gcc/gcc-%v/gcc-%v.tar.bz2
Source-MD5: 31b459062499f9f68d451db9cbf3205c
NoSourceDirectory: True
ConfigureParams: --enable-languages=f77 --infodir='${prefix}/share/info' 
--libexecdir='${prefix}/lib' --disable-shared
#BuildDepends: dejagnu
PatchScript: <<
 #!/bin/sh
 cd gcc-%v/gcc
 mv Makefile.in Makefile.in.orig
 sed 's|$(ALL_CPPFLAGS) $(INCLUDES)|$(INCLUDES) $(ALL_CPPFLAGS)|g' < 
Makefile.in.orig > Makefile.in
<<
CompileScript: <<
 #!/bin/sh
 mkdir darwin
 cd darwin
 ../gcc-%v/configure %c
 make CFLAGS='-O' LIBCFLAGS='-g -O2' LIBCXXFLAGS='-g -O2 
-fno-implicit-templates' profiledbootstrap
#cd gcc; make check-g77
<<
InstallScript: <<
 #!/bin/sh
 cd darwin
 make install prefix=%i
 cd %i/bin
 /bin/rm -f gcc gccbug cpp gcov powerpc-apple*
 ln -s %p/bin/g77 f77
 darwinvers=`/usr/bin/uname -v | cut -f1 -d":" | awk '{print $4}'`
 gccvers=`%i/bin/g77 -dumpversion | head -1 | cut -f4 -d" "`
 ln -s 
%p/lib/gcc/powerpc-apple-darwin${darwinvers}/${gccvers}/include/g2c.h 
%i/include/g2c.h
 /bin/rm -rf %i/share/locale %i/man
 /bin/rm -f %i/lib/charset.alias
 /bin/rm -f %i/share/info/gcc* %i/share/info/cpp*
 /bin/mv -f %i/lib/libiberty.a %i/lib/libiberty-g77.a
<<
License: GPL
DocFiles: gcc-%v/gcc/f/ChangeLog gcc-%v/COPYING gcc-%v/COPYING.LIB
Description: GNU Fortran compiler
DescDetail: <<
 g77 consists of several components:

 1) The g77 command itself.
 2) The libg2c run-time library.  This library contains the
machine code needed to support capabilities of the Fortran
language that are not directly provided by the machine code
generated by the g77 compilation phase.
 3) The compiler itself, internally named f771.
f771 does not generate machine code directly --
it generates assembly code, leaving the conversion to
actual machine code to an assembler, usually named as.
 g77 supports some fortran90 features, like automatic arrays,
 free source form, and DO WHILE.
<<
DescPort: <<
 Installs g77 from the FSF gcc distribution.
 This version does not install in /usr. It contains it's own cc1 and
 libgcc.a installed in %p.
 libiberty.a moved to libiberty-g77.a to avoid conflict with ddd.
<<
DescUsage: <<
 If you get unresolved symbol '_saveFP', add -lcc_dynamic when linking.
 Does not support -framework argument, to link frameworks use -Wl flag
 (for example, to link vecLib use "-Wl,-framework -Wl,vecLib").
 No man page, use "info g77".
<<
Homepage: http://gcc.gnu.org/onlinedocs/g77/
Maintainer: Jeffrey Whitaker <[EMAIL PROTECTED]>
 

Implementing the API specified in PEP 262 is desirable.
What I am thinking is this,
1. when user specify a package to install, the package's .info file will 
be looked up in 'pkginfo' directory (in PEP 262, it is the INSTALLDB 
directory that holds all these .info files).

2. to the system, only 3 things are crucial: where to get the package? 
what packages the package needs? how to install? These 3 things are the 
real critical parts of .info file, the rest are information and metadata.

3. from the dependencies, the system now creates a tree of dependencies. 
Can all dependencies be satisfied, i.e. are there any required packages 
that are not in standard library and there is no .info file for?

4. dependencies are satisfied (install the packages) from terminal leaf 
nodes on the dependency tree to the root node (which is the one the user 
wants to install)

5. appropriate entries are made in appropriate files (i.e. 
pkg-install.log) to signify which packages are installed.

6. satisfy the files needed for API requirements of PEP 262.
Please tell me what you think...
Cheers
Maurice
--
http://mail.python.org/mailman/listinfo/python-list

Re: Python site-packages and import

[Maurice LING]

Peter Saffrey wrote:
(apologies for starting a new thread - Google can't retrieve the other
message for some reason)
Yes, /usr/lib/python/site-packages is in sys.path. This series of
commands should explain what I mean: I've put the Python ID3 module in
a sub-directory of site-packages as an illustration.
[EMAIL PROTECTED]:~$ ls /usr/lib/python2.3/site-packages/
ID3.py  ID3.pyc  ID3.pyo  README  apt_inst.so  apt_pkg.so  apt_proxy 
bsddb3  debconf.py  debconf.pyc  debconf.pyo  id3  pj  twisted

[EMAIL PROTECTED]:~$ ls /usr/lib/python2.3/site-packages/apt_proxy/
__init__.py   __init__.pyo  apt_proxy.pyc  apt_proxy_conf.py  
apt_proxy_conf.pyo  apt_proxytap.pyc  memleak.py   memleak.pyo 
misc.pyc  packages.py   packages.pyo
__init__.pyc  apt_proxy.py  apt_proxy.pyo  apt_proxy_conf.pyc 
apt_proxytap.py apt_proxytap.pyo  memleak.pyc  misc.py 
misc.pyo  packages.pyc  twisted_compat

[EMAIL PROTECTED]:~$ ls /usr/lib/python2.3/site-packages/id3   
ID3.py  ID3.pyc  ID3.pyo

[EMAIL PROTECTED]:~$ python
Python 2.3.5 (#2, Feb  9 2005, 00:38:15) 
[GCC 3.3.5 (Debian 1:3.3.5-8)] on linux2
Type "help", "copyright", "credits" or "license" for more information.

import sys
sys.path
['', '/usr/lib/python23.zip', '/usr/lib/python2.3',
'/usr/lib/python2.3/plat-linux2', '/usr/lib/python2.3/lib-tk',
'/usr/lib/python2.3/lib-dynload',
'/usr/local/lib/python2.3/site-packages',
'/usr/lib/python2.3/site-packages']
import apt_proxy.memleak
import id3.ID3
Traceback (most recent call last):
  File "", line 1, in ?
ImportError: No module named id3.ID3

Peter
your site-package/id3 lacks a __init__.py file. '__init__.py' file can 
be just an empty file but it needs to be there for python to be 
considered as a package.

Alternatively, you can use 'from id3 import ID3' instead of 'import id3.ID3'
Cheers
Maurice
--
http://mail.python.org/mailman/listinfo/python-list

Re: New Python website

[Maurice LING]

lpe wrote:
Hi maurice
thanks for your interest, that surely looks interesting (and promising)
I had no experience with any of the packages you mentioned, but it may
well be usefull.
Please email me with more details of what you had in mind.
Hi,
I've just read PEP 262 last night and finds that it does more or less 
describes what I have in mind. However, I am not sure if there is every 
need for such a descriptive database file or something slimmer, like 
Fink's .info files will suffice.

An example of Fink's .info file is:

Package: g77
Version: 3.4.1
Revision: 1
BuildDependsOnly: true
Source: mirror:gnu:gcc/gcc-%v/gcc-%v.tar.bz2
Source-MD5: 31b459062499f9f68d451db9cbf3205c
NoSourceDirectory: True
ConfigureParams: --enable-languages=f77 --infodir='${prefix}/share/info' 
--libexecdir='${prefix}/lib' --disable-shared
#BuildDepends: dejagnu
PatchScript: <<
 #!/bin/sh
 cd gcc-%v/gcc
 mv Makefile.in Makefile.in.orig
 sed 's|$(ALL_CPPFLAGS) $(INCLUDES)|$(INCLUDES) $(ALL_CPPFLAGS)|g' < 
Makefile.in.orig > Makefile.in
<<
CompileScript: <<
 #!/bin/sh
 mkdir darwin
 cd darwin
 ../gcc-%v/configure %c
 make CFLAGS='-O' LIBCFLAGS='-g -O2' LIBCXXFLAGS='-g -O2 
-fno-implicit-templates' profiledbootstrap
#cd gcc; make check-g77
<<
InstallScript: <<
 #!/bin/sh
 cd darwin
 make install prefix=%i
 cd %i/bin
 /bin/rm -f gcc gccbug cpp gcov powerpc-apple*
 ln -s %p/bin/g77 f77
 darwinvers=`/usr/bin/uname -v | cut -f1 -d":" | awk '{print $4}'`
 gccvers=`%i/bin/g77 -dumpversion | head -1 | cut -f4 -d" "`
 ln -s 
%p/lib/gcc/powerpc-apple-darwin${darwinvers}/${gccvers}/include/g2c.h 
%i/include/g2c.h
 /bin/rm -rf %i/share/locale %i/man
 /bin/rm -f %i/lib/charset.alias
 /bin/rm -f %i/share/info/gcc* %i/share/info/cpp*
 /bin/mv -f %i/lib/libiberty.a %i/lib/libiberty-g77.a
<<
License: GPL
DocFiles: gcc-%v/gcc/f/ChangeLog gcc-%v/COPYING gcc-%v/COPYING.LIB
Description: GNU Fortran compiler
DescDetail: <<
 g77 consists of several components:

 1) The g77 command itself.
 2) The libg2c run-time library.  This library contains the
machine code needed to support capabilities of the Fortran
language that are not directly provided by the machine code
generated by the g77 compilation phase.
 3) The compiler itself, internally named f771.
f771 does not generate machine code directly --
it generates assembly code, leaving the conversion to
actual machine code to an assembler, usually named as.
 g77 supports some fortran90 features, like automatic arrays,
 free source form, and DO WHILE.
<<
DescPort: <<
 Installs g77 from the FSF gcc distribution.
 This version does not install in /usr. It contains it's own cc1 and
 libgcc.a installed in %p.
 libiberty.a moved to libiberty-g77.a to avoid conflict with ddd.
<<
DescUsage: <<
 If you get unresolved symbol '_saveFP', add -lcc_dynamic when linking.
 Does not support -framework argument, to link frameworks use -Wl flag
 (for example, to link vecLib use "-Wl,-framework -Wl,vecLib").
 No man page, use "info g77".
<<
Homepage: http://gcc.gnu.org/onlinedocs/g77/
Maintainer: Jeffrey Whitaker <[EMAIL PROTECTED]>

Implementing the API specified in PEP 262 is desirable.
What I am thinking is this,
1. when user specify a package to install, the package's .info file will 
be looked up in 'pkginfo' directory (in PEP 262, it is the INSTALLDB 
directory that holds all these .info files).

2. to the system, only 3 things are crucial: where to get the package? 
what packages the package needs? how to install? These 3 things are the 
real critical parts of .info file, the rest are information and metadata.

3. from the dependencies, the system now creates a tree of dependencies. 
Can all dependencies be satisfied, i.e. are there any required packages 
that are not in standard library and there is no .info file for?

4. dependencies are satisfied (install the packages) from terminal leaf 
nodes on the dependency tree to the root node (which is the one the user 
wants to install)

5. appropriate entries are made in appropriate files (i.e. 
pkg-install.log) to signify which packages are installed.

6. satisfy the files needed for API requirements of PEP 262.
Please tell me what you think...
Cheers
Maurice
--
http://mail.python.org/mailman/listinfo/python-list

Re: New Python website

[Maurice LING]

Michael Soulier wrote:
On 28 Apr 2005 17:45:02 -0700, lpe <[EMAIL PROTECTED]> wrote:
http://www.pycode.com
I was kinda suprised when I could not find any good sites with 3rd
party modules (other than the Vaults of Parnassus, where you must host
files elsewhere), so I decided to write one myself :)
It is brand new and might still be buggy, but hopefully it will be
usefull to some people.  Feel free to join and upload any of your code.
thanks

Something wrong with PyPi?
Mike
I think it is quite clear when he says "I could not find any good sites 
with 3rd party modules (other than the Vaults of Parnassus, where you 
must host files elsewhere)", suggesting that he is looking for a site 
whereby 3rd party modules can be hosted, rather than a site telling you 
where 3rd party modules are hosted.

maurice
--
http://mail.python.org/mailman/listinfo/python-list

Re: New Python website

[Maurice LING]

lpe wrote:
http://www.pycode.com
I was kinda suprised when I could not find any good sites with 3rd
party modules (other than the Vaults of Parnassus, where you must host
files elsewhere), so I decided to write one myself :)
It is brand new and might still be buggy, but hopefully it will be
usefull to some people.  Feel free to join and upload any of your code.
thanks
Hi,
Just yesterday, I was frustrated waiting for replies from Catelog-SIG 
about the possibilities of a package management system that works like 
Fink (fink.sf.net). Basically, the problems I see is that C extension 
modules aren't portable across major Python revisions and there is no 
easy way to maintain the installed packages in site-packages directory. 
My scenario is that a system admin has to maintain 50 different 
libraries and their dependencies...

So I've decided to write something myself... and call it 'centipyde'.
Modelled against Fink and Darwinports (darwinports.opendarwin.org) 
(obviously I'm a Mac user), information (in text file) are in a folder 
(centipyde/pkginfo/) as .info files. Each package will have a .info file 
which tells the system (centipyde) where to download the source tar (or 
zip) and how to install the package, as well as the dependecies, 
maintaining the installed packages etc etc. No difference from other 
package managers (a goal)...

It is still very rough at this moment, so please bear with me. 
Basically, the user will have to cvs checkout the system and just run it.

I've checked it into Sourceforge, under IB-DWB project (which is 
abandoned) as 'centipyde'. But for some reason, I still can't view it in 
 ViewCVS yet. Anyway, the directory layout is

../centipyde
../centipyde/centipyde.py
../centipyde/pgkinfo
../centipyde/pgkinfo/ply15.info
ply15.info contains the following text (pretty much modelled against Fink):
package=ply15
maintainer=.
dependencies=.
downloadurl=http://systems.cs.uchicago.edu/ply/ply-1.5.tar.gz
prebuildscript=tar zxvf ply-1.5.tar.gz
sourcedir=ply-1.5
buildscript=python setup.py build
installscript=sudo python setup.py install
centipyde.py is the following:
=
"""
Author: Maurice H.T. Ling <[EMAIL PROTECTED]>
Copyright (c) 2005 Maurice H.T. Ling
Date created : 28th April 2005
"""
PKGINFOPATH = 'pkginfo'
INSTALL_LOG = 'install.log'
import os
import string
import sys
def install_package(package_name):
f = open(os.getcwd() + os.sep + PKGINFOPATH + os.sep + package_name 
+ '.info', 'r')
install_info = {}
for line in f.readlines():
line = string.split(line, '=')
if line[1][-1] == os.linesep:
install_info[line[0]] = string.strip(line[1][:-1])
else: install_info[line[0]] = string.strip(line[1])
f.close()
print "Package Installation Information: " + str(install_info)

os.system('curl -O ' + str(install_info['downloadurl']))
preinstall = []
preinstall = string.split(install_info['prebuildscript'], ';')
for cmd in preinstall: os.system(cmd)
cwd = os.getcwd()
print cwd
os.chdir(os.path.join(os.getcwd(), install_info['sourcedir']))
print os.getcwd()
buildscript = []
buildscript = string.split(install_info['buildscript'], ';')
for cmd in buildscript: os.system(cmd)
installscript = []
installscript = string.split(install_info['installscript'], ';')
for cmd in installscript: os.system(cmd)
if sys.argv[1] == 'install':
install_package(sys.argv[2])
=
When I run "python centipyde.py install ply15", PLY1.5 gets downloaded 
from David Beazley's website, uncompressed and installed into the 
site-package as shown here:

znichols-maurice:~/MyProjects/ib-dwb/centipyde mauriceling$ ls -alltotal 8
drwxr-xr-x   5 mauricel  mauricel   170 28 Apr 17:37 .
drwxr-xr-x  10 mauricel  mauricel   340 28 Apr 16:21 ..
drwxr-xr-x   5 mauricel  mauricel   170 28 Apr 17:33 CVS
-rw-r--r--   1 mauricel  mauricel  1385 28 Apr 23:47 centipyde.py
drwxr-xr-x   4 mauricel  mauricel   136 28 Apr 17:36 pkginfo
znichols-maurice:~/MyProjects/ib-dwb/centipyde mauriceling$ sudo python 
centipyde.py install ply15
Package Installation Information: {'maintainer': '.', 'sourcedir': 
'ply-1.5', 'package': 'ply15', 'downloadurl': 
'http://systems.cs.uchicago.edu/ply/ply-1.5.tar.gz', 'installscript': 
'sudo python setup.py install', 'dependencies': '.', 'buildscript': 
'python setup.py build', 'prebuildscript': 'tar zxvf ply-1.5.tar.gz'}
  % Total% Received % Xferd  Average Speed  Time 
  Curr.
 Dload  Upload TotalCurrent  Left 
  Speed
100 69278  100 692780 0   7746  0  0:00:08  0:00:08  0:00:00 
31811
ply-1.5/
ply-1.5/doc/
ply-1.5/doc/ply.html
ply-1.5/CHANGES
ply-1.5/COPYING

. [snipped] .
ply-1.5/test/yacc_uprec.exp
ply-1.5/test/yacc_uprec.py
/sw/lib/python2.3/distutils/dist.py:213: UserWarning: 'licence' 
distribution option is deprecated; use 'license'

Re: Python site-packages and import

[Maurice LING]

Peter Saffrey wrote:
I'm trying to write a python service, with an executable in
/usr/local/bin, but modules that are kept in a sub-directory of
/usr/lib/python/site-packages.
Using apt-proxy as my template, I've put the modules in
/usr/lib/python/site-packages/mymodules and tried to import them with
import mymodules.module1. However, this doesn't seem to be working,
even though something very similar works for apt-proxy. I thought this
might be something to do with sys.path, but I can't work out why it's
different for my modules and those used by apt-proxy.
To be fair, it is past midnight, and I'm probably just being stupid.
Would anybody care to point out where? :)
Peter
your site-packages directory must be seen in python. try
import sys
sys.path
and see if your site-packages directory is listed?
maurice
--
http://mail.python.org/mailman/listinfo/python-list

Re: Can .py be complied?

[Maurice LING]

steve.leach wrote:
python -o foo.exe foo.py
at the command line, and get an executable, without any further effort.

Hence making the resulting program useless to users of most operating 
systems.
In close sourced development, which most corporates may prefer, yes, the 
resulting program is useless to users of most operating systems.

In open sourced developement, it is still a good feature to have. At 
least for distribution to end users or as trial.

To end users, they don't care, as long as they can click and run the 
program they need.

To developers, if the codes are close source, nothing can be done anyway 
even if you have the codes, licensing agreements and contracts usually 
forbids everything. If the codes are open source, you will get the codes 
 anyway and do according to the limits of the licence.

maurice
--
http://mail.python.org/mailman/listinfo/python-list

bytecode non-backcompatibility

[Maurice LING]

Martin v. Löwis wrote:
Maurice LING wrote:
I've emailed to catelog-sig mailing list and is still waiting to hear
something. Currently, I have no idea of the structure of PyPI. I hope I
can hear from them soon and generate some starting points...

Posting questions is not the only way to find answers. The source code
of PyPI is available: sf.net/projects/pypi.
Regards,
Martin

I've browsed the source codes of PyPI in sf.net, nothing pops up as very 
useful to me... I reckoned that sitting in here or hoping for replies 
from catelog-sig isn't going to help much.

On that, I've prototyped a very simple proof-of-concept of what I have 
in mind, regarding a Fink-like tool for Python's 3rd party libraries 
management. Please bear in mind that this is a proof-of-concept 
prototype, really bare bones. It can be found in 'centipyde' module in 
sf.net/projects/ib-dwb. In case the CVS hadn't updated by the time 
someone reads this, the directory layout is this:

../centipyde
../centipyde/centipyde.py
../centipyde/pgkinfo
../centipyde/pgkinfo/ply15.info
ply15.info contains the following text (pretty much modelled against Fink):
package=ply15
maintainer=.
dependencies=.
downloadurl=http://systems.cs.uchicago.edu/ply/ply-1.5.tar.gz
prebuildscript=tar zxvf ply-1.5.tar.gz
sourcedir=ply-1.5
buildscript=python setup.py build
installscript=sudo python setup.py install
centipyde.py is the following:
=
"""
Author: Maurice H.T. Ling <[EMAIL PROTECTED]>
Copyright (c) 2005 Maurice H.T. Ling
Date created : 28th April 2005
"""
PKGINFOPATH = 'pkginfo'
INSTALL_LOG = 'install.log'
import os
import string
import sys
def install_package(package_name):
f = open(os.getcwd() + os.sep + PKGINFOPATH + os.sep + package_name 
+ '.info', 'r')
install_info = {}
for line in f.readlines():
line = string.split(line, '=')
if line[1][-1] == os.linesep:
install_info[line[0]] = string.strip(line[1][:-1])
else: install_info[line[0]] = string.strip(line[1])
f.close()
print "Package Installation Information: " + str(install_info)

os.system('curl -O ' + str(install_info['downloadurl']))
preinstall = []
preinstall = string.split(install_info['prebuildscript'], ';')
for cmd in preinstall: os.system(cmd)
cwd = os.getcwd()
print cwd
os.chdir(os.path.join(os.getcwd(), install_info['sourcedir']))
print os.getcwd()
buildscript = []
buildscript = string.split(install_info['buildscript'], ';')
for cmd in buildscript: os.system(cmd)
installscript = []
installscript = string.split(install_info['installscript'], ';')
for cmd in installscript: os.system(cmd)
if sys.argv[1] == 'install':
install_package(sys.argv[2])
=
When I run "python centipyde.py install ply15", PLY1.5 gets downloaded 
from David Beazley's website, uncompressed and installed into the 
site-package.

All comments and code donations are welcomed.
Cheers
Maurice
--
http://mail.python.org/mailman/listinfo/python-list

Re: bytecode non-backcompatibility

[Maurice LING]

Martin v. Löwis wrote:
[automatically install a set of packages]
What do you think?

That would certainly be possible. Contributions are welcome.
Regards,
Martin
I've emailed to catelog-sig mailing list and is still waiting to hear 
something. Currently, I have no idea of the structure of PyPI. I hope I 
can hear from them soon and generate some starting points...

Maurice
--
http://mail.python.org/mailman/listinfo/python-list

Re: bytecode non-backcompatibility

[Maurice LING]

Terry Reedy wrote:
I doubt anyone disputes that upgrades are more hassle than we would like. 
My main point was that freezing CPython's technology is not the solution. 
Any other upgrade helper that you can contribute will be welcome.

Terry J. Reedy

So there is no way of releasing a close-source application written in 
Python?

Forget about reverse engineering or decompiling, things like EU Council 
Directive, Berne Convention , and legislation of individual countries 
are to be used and argued for handling what decompilation is allowed or 
not allowed. It does make a difference between  looking at the codes 
when it is given and performing an act of decompilation to look at the 
codes. As mentioned in one of my postings, it may be analogous to the 
difference between you dropping some cash on the ground and I take it, 
and I performing the act of pickpocketing and take your money.

I am not advocating close source in any sense and I believe the choice 
of open or close source distribution is individual's and corporate's rights.

Cheers
Maurice
--
http://mail.python.org/mailman/listinfo/python-list

Re: bytecode non-backcompatibility

[Maurice LING]

Martin v. Löwis wrote:
Maurice LING wrote:
technicalities are wrong but situation remains unchanged.

For C modules, it is very likely that new versions of Python
will continue to break the ABI, by changing the layout of
structures.
The most straight-forward way to deal with it as a sysadmin
or user is to install multiple versions of Python on a single
machine. If Fink considers python2.4 as a replacement for
python2.3, then this is a flaw in Fink. In Debian, there is
a python package, which currently depends on python2.3.
Sometime in the future, it will depend on python2.4. Users
which update will then get python2.4, however, python2.3
will remain installed and usable, with all the extension
modules that were installed for it.
Regards,
Martin
Fink does not consider python2.4 to be a replacement for python2.3. In 
fact, you can install python2.2, 2.3 and 2.4 in the same machine with 
Fink. It will maintain 3 sets of libraries as /sw/lib/python2.2, 
/sw/lib/python2.3 and /sw/lib/python2.4. The chore is that when say Fink 
installs python2.4, all the libraries in /sw/lib/python2.3/site-packages 
have to be re-installed into /sw/lib/python2.4/site-packages, one by 
one. There is no simple way of doing that... which makes any system 
admin needing to re-install 50 3rd party libraries into 
/sw/lib/python2.4/site-packages a big task, as well as satisfying the 
necessary dependencies.

So if C extension API (or whatever that is really called) is stable, the 
system admin can just copy all of /sw/lib/python2.3/site-packages into 
/sw/lib/python2.4/site-packages and it should work. From what you've 
said, it seems that this isn't possible. So my alternative solution is 
that PyPI have a mechanism to maintain what had been installed in the 
site-package directory and to download the libraries and install into 
the new site-package directory...

What do you think?
Cheers
maurice
--
http://mail.python.org/mailman/listinfo/python-list

Re: bytecode non-backcompatibility

[Maurice LING]

Mike Meyer wrote:
Maurice LING <[EMAIL PROTECTED]> writes:

The idea of having to release the program or library as source files
does ring alarms in many executives in corporate world. Less freezing
the modules (which renders it platform dependent) or using Jython
(which is not possible when C extensions are involved), there is no
middle grounds in CPython for distribution without source codes.

You're right - and byte codes *as they exist now* aren't an acceptable
middle ground, either. The problem isn't that the interpreter might
change (that can be defeated by distributing the interpreter with the
bytecode files), but that the byte code can be decompiled to get your
Python source back. See http://www.crazy-compilers.com/decompyle/ > for an example.
I know that bytecodes can be decompiled back to python source. Similarly 
Java class files can be decompiled back to Java source files quite 
easily. IANAL, however, I do think that reverse engineering can be seen 
as a purposeful act in the eyes of law. I remembered that EU Council 
Directive 9/250/ECC does explicits the allowable scope of decompilation 
(for interoperability, preserving expression of ideas, achieving 
performance of objectives).

For companies, I think it is touchy to ask permission to release the 
source codes even though they may be alright with releasing the same 
Java JAR file into the community.

We do know that regardless of whatever methods of compilation there may 
be, if some genius is willing to spend sufficient time to crack a 
program. Even if he has to read the binary codes, it will still break. 
But that is a wilfull act rather than just looking at the source codes 
when it is distributed with it. It's just like picking up your money 
from the ground if you had dropped it as compared to pickpocketing you 
as compared to hold you at knife point and robbing you.. Same end 
result but it is viewed differently in the eyes of law and morals.


Selling byte codes to someone who's worried about shipping source is
selling them snake oil. I think it's unprofessional, at best.
--
http://mail.python.org/mailman/listinfo/python-list

Re: bytecode non-backcompatibility

[Maurice LING]

What can be done in PYAN is to encourage all 3rd party library
developers to centralize their libraries in it, which I think all will
gladly respond. All that is needed to be  deposited in PYAN is a
description file, like the simplest setup.py file. All that is needed
in this description file is
1. where to download the source codes (e.g. CVS)?
2. version number?
3. standard stuffs (optional) like authors, descriptions, copyright?

To copy all of what CPAN does, you need to know what other modules it
depends on, so you can automatically download those as well.
Yes, dependencies are necessary to be declared. The maintainers of each 
package should list the dependencies as well to make it work.


Python can then have a built-in mechanism to read the description file
and download the source codes and do the standard "sudo python
setup.py install" to install the library into site-package.

I don't like this - it would make Python depend on sudo being
available. I'd rather it not do that, and let each systems
administrator issue the command according to *their* security policy.
If you are installing packages into your home directory, then sudo is 
not needed. But if you are installing it for everybody's use, then it is 
necessary. Fink runs using superuser privileges.


I think something like this can be set up for Python quite easily. I
recall some time back, there's a long discussion about setting up
Python's version of CPAN but I can't recall the contents of the
discussions.

It's not clear it's easy. It's *very* clear it won't happen until
someone steps up to do it. The latter is a problem. 

I've posted my method in catelog-sig list. And is willing to offer some 
time on it... if that helps... But to get it working, I think it will 
need 5-10x more time than I can put into it...

Cheers
maurice
--
http://mail.python.org/mailman/listinfo/python-list

Re: bytecode non-backcompatibility

[Maurice LING]

The same *can* be said for some decade-old .py files.  Certainly, most 
everything written for 2.0 onward still works.  The same will also be true 
for .pyc files as long as you run them with their corresponding binary and 
as long as the binary stills run.

However, .pyc files are private, temporary caches created by a CPython 
interpreter for its own use and tied to its current implementation 
technology.  They are not intended to be a public distribution form and 
indeed cannot be a means to run Python programs on other interpreters. 
Guido has stated that he want the freedom to change or even replace parts 
of the internal implementation as he sees fit. (2.5 will probably get a new 
compiler, with the difference mostly transparent to users.)

One of the principles of OOP is separation of interface from 
implementation.  User of a class should only use the public interface and 
not depend on private implementation.  One reason is so implementation can 
be changed even radically as long as interface is kept constant.  The 
Python language is interface.  CPython bytecodes are implementation.

From a technical perspective, I can accept that .pyc files are private 
and temporary. To a certain extend, it does helps development cycle. 
Every time I amend my source codes, I just run it without having to 
consider or needing to re-compile the source files.

The idea of having to release the program or library as source files 
does ring alarms in many executives in corporate world. Less freezing 
the modules (which renders it platform dependent) or using Jython (which 
is not possible when C extensions are involved), there is no middle 
grounds in CPython for distribution without source codes.

Every now and then, there will be new threads in this list about people 
asking the means and possibilities of releasing a module/program without 
having to release the source code (there's another new thread on it 
today, "Can .py be compiled?")...


What I do have resources (time and energy) for is to work with the 
maintainers of PyPI to implement the package maintenance system I've 
described..

Good.  I agree that we need more along that line.
I've posted my idea on catelog-sig mailing list. Hope it get picked up...
Cheers
Maurice
--
http://mail.python.org/mailman/listinfo/python-list

Re: Python internal design

[Maurice LING]

Emre Turkay wrote:
Hi Folks,
I am designing a tool, in which there are dynamic types and
variables with these types. In this respect, it is more like an
interpreted language design.
I wonder how these issues are implemented in Python are there any
documents or articles about it, which I can read and get an idea.
Thanks,
emre
Hi,
I've been trying to look that up as well, in CoRR, ACM digital library 
but not much luck. I've got only peripheral hits using "python virtual 
machine" as search term. If you are looking for something at the level 
of "python language specification" and "python virtual machine 
specification", I am unable to find anything yet.

Please tell me if you managed to find something useful.
Cheers
Maurice
--
http://mail.python.org/mailman/listinfo/python-list

Re: Can .py be complied?

[Maurice LING]

monkey wrote:
Hi all, I am new to programming, already have a glace on introduction of
c++, java and finally decided on python. But I found that the .py file is
just like the source file, how can I make a program without revealing its
source? (may be my question is a little bit stupid)

It is generally not very easy or straight-forward. The developers of 
CPython had generally intended that the source codes be the actual 
distribution, and is not likely to change. (see the thread on "bytecode 
non-backcompatibility")

For now, you can use pyfreeze to snap the application, which is to 
bundle your application to a python interpreter (bootstrapping) as a 
package but this will not create a portable application. You can only 
run the application on the native system that it is frozen on. For 
example, if i freeze my application on Mac OSX, I won't be able to run 
that on MS Windows. Freezing bootstraps the system's python onto the 
application.

If your application does not use any C modules, you can try to use 
Jython instead. Program in python but use jythonc to convert it into 
Java source files and package it into Java JAR files, then you will only 
need to release the JAR files without needing to release your codes.

Cheers
Maurice
--
http://mail.python.org/mailman/listinfo/python-list

Re: bytecode non-backcompatibility

[Maurice LING]

One difference between Java and Python is this: Java bytecodes are, as I 
understand it, part of the Java language definition.  CPython bytecodes are 
intentionally not part of the language at all.  Except maybe fore PyPy, 
other implementations do not use them.  Jython translates Python source to 
Java bytecodes.  Pyrex translates augmented Python source to C, to be 
compiled to native machine code.  Ironman translates, I presume, to .NET 
common language.  PyParrot (don't know if it has an official name) 
translates to Parrot bytecodes.  Viper translated to OCamel.

If you want an implementation with frozen bytecodes, you are free to make 
one ;-)

Terry J. Reedy

So there are currently 7 implementations or variations of the Python 
language and you are suggesting I make another one? If I am to do that, 
I will use CPython 2.4.1 and call it a implementation that maintains the 
current set of bytecodes (newer versions can have more bytecodes and 
deprecated bytecodes are still left there for backward compatibility) 
and C API interface. Till now I still do not know what is so exceedingly 
prohibitive to do that? Why should I create such a direct fork? 
Personally I do not have the resources to maintain this fork.

The Pacman (binary executable) that I played on a 286 a decade ago still 
works well on my brother's pentium III system, so from the point of a 
user, it is backward compatible. The same can't be said for .pyc files 
or .so files (I may be wrong with .so files here).

What I do have resources (time and energy) for is to work with the 
maintainers of PyPI to implement the package maintenance system I've 
described..

maurice
--
http://mail.python.org/mailman/listinfo/python-list

Re: bytecode non-backcompatibility

[Maurice LING]

Terry Reedy wrote:
"Maurice LING" <[EMAIL PROTECTED]> wrote in message 
news:[EMAIL PROTECTED]


Now I understand that Python bytecodes are only dealing with pure python 
source codes.

Then stop blaming (machine-independent) CPython 'bytecodes' for any 
problems you have with compiled-to-machine-language C extensions, which 
have nothing to do with bytecodes.  Bytecodes also have nothing directly to 
do with source-code compatibility. 


technicalities are wrong but situation remains unchanged.
maurice
--
http://mail.python.org/mailman/listinfo/python-list

Re: bytecode non-backcompatibility

[Maurice LING]

2. The current compilation scheme (compiling to bytecode as and when
it is needed) works well for scripting purposes but is less desirable
in commercial settings. Less distribution happens when it is used
purely for scripting purposes, such as system maintenance or tuning.

The solution with the current situation depends on what you mean by
"commercial settings".
Commerically, I will mean distributing the program without the codes. 
The best way under the current method is to freeze the application. In 
Java, you can just distribute the JAR files or CLASS files. I can 
imagine there will be some level of touchiness when source codes are 
distributed...


3. Using Python in commercial settings will usually require
distribution of resulting software and it is may or may not be
desirable to distribute source codes as well. Unless the application
is frozen, distributing source code is a must.

People have recommended that you distribute the Python interpreter
with commercial applications. This makes installing them much easier
for the end user. It also means the byte code and the interpreter will
always match.

4. One advantage that Java platform has is that it does not require
the release of source files and still promotes platform-independence.

Java is only "platform-independent" to the degree that the jvm and
libraries you need for your application are already available.  A
quick google finds applications that require jvm versions - so I
suspect that you can't always run Java code built for a new version of
the jvm on older jvms. So you have to make sure the target platform
has a recent enough implementation of the jvm installed. You may have
problems if you try using a different groups jvm as well. I haven't
looked into that in a number of years.
So you can't just ship java bytecodes to someone expecting they'll be
able to run it out of the box. They may well need to update their java
environment, or install a second one (I recall one time having four
Java products that took in total three different jvms to run. Bleah.)
This isn't really different from Python. In both cases, the end user
has to have a suitable version of the bytecode interpreter and
libraries installed. Java is a little better in that they provide
backwards compatability.
I think backward compatibility is good enough. A Java JAR file compiled 
under for JVM 1.2 can still run in JVM 1.4. I don't think anyone will or 
can expect forward compatibility, that will require godly powers...

According to Steve Holden's reply, Python C API is only backward 
compatible in minor versions but not major versions. The issue is, why 
not major versions as well unless there's something exceedingly prohibitive?


5. Unstable bytecodes makes updating to a newer version of Python very
tedious and risk breaking old scripts, if they uses C modules.

Unstable bytecodes have nothing to do with these problems. 

The current CPython installation process puts the python command and
the libraries for different versions in different directories. This
allows you to have multiple versions installed so you can keep old
scripts working with the old version until you've had time to test
them. It also makes life much easier on the developers, as they can
have a development version installed on the machine at the same time
as they have a production version without breaking the old scripts. It
also means you have to reinstall all your modules on the new
installation - which is what makes the update process tedious for me.
Now, this could be mitigated by having Python libraries installed in
the same location for all versions. You could fix all the bytecode
files by running compileall.py as a script. Sometimes, a module won't
work properly on a new version of Python, and will have to be
updated. You'll have to find those by trial and error and fix them as
you find them. You'll also have to recompile all the C modules, which
will break running scripts on the old interpreter.
Under the currewnt system, old scripts that are run by the new
interpreter will break if all the modules they need aren't installed
yet. Once they're installed, the scripts should work.
If you have scripts that use C modules that aren't installed in the
standard Python search path, they may well break on the new
interpreter until you recompile them. Having both versions of the
interpreter available makes this tolerable.
The cure I proposed seems worse than disease. If you've got a better
solution, I'm sure we'd be interested in hearing it.
Perhaps this is another call for Python version of CPAN (CPyAN or PYAN). 
 It can be modelled after Fink or Darwinports.

If I remembered correctly, Fink uses apt-get and curl.
What can be done in PYAN is to encourage all 3rd party library 
developers to centralize their libraries in it, which I think all will 
gladly respond. All that is needed to be  deposited in PYAN is a 
description file, like the simplest setup.py file. All that is needed in 
this description file

Re: bytecode non-backcompatibility

[Maurice LING]

All you have to do is convince the developers to declare the set of
bytecodes fixed, and they'd be stable. I don't think that will be
easy, as it removes one of the methods used to improve the performance
of Python. Since rebuilding the bytecode files is trivial (just invoke
compileall.py in the library as a script), this isn't really a
problem. The only thing that breaks are modules that muck about inside
the bytecode. I don't know how much bytecode improvements have sped
things up, but I do know that if I ever used a module that mucked
around with byte codes, I wasn't aware of it - so this is a tradeoff
I'm more than happy to make.
Thanks Mike,
My arguments to the developers will be:
1. Python had gone from a purely scripting language to a general purpose 
programming language.

2. The current compilation scheme (compiling to bytecode as and when it 
is needed) works well for scripting purposes but is less desirable in 
commercial settings. Less distribution happens when it is used purely 
for scripting purposes, such as system maintenance or tuning.

3. Using Python in commercial settings will usually require distribution 
of resulting software and it is may or may not be desirable to 
distribute source codes as well. Unless the application is frozen, 
distributing source code is a must.

4. One advantage that Java platform has is that it does not require the 
release of source files and still promotes platform-independence.

5. Unstable bytecodes makes updating to a newer version of Python very 
tedious and risk breaking old scripts, if they uses C modules.

6. Unstable bytecodes also makes research work into Python, such as, 
just-in-time compilation and just-in-time specialization unfavourable as 
they may only be applicable to a specific version of Python. There is 
much less chance of getting a project grant than if the same project is 
applied for Java (stable bytecodes).

What other point are there?
I may be chopped by saying this but by having a stable set of bytecodes, 
we may lose a means of optimization. But we may gain more from filling 
the need for commerical distribution of applications writing in Python 
and ease of upgrading...

At current stage, every time a new version of Python is installed in my 
server or system, I have to test and ensure the needed libraries are 
there... which may be a horror in corporate settings. Couple that with 
messy dependencies of the libraries to be installed. I remembered the 
time I was trying to install Eric 3, the dependencies makes me want to 
give up... I have to install Qt, then pyQt, then something else, then 
Eric3. Imagine you need 10 of those libraries... which may happen... I 
am not yet masochistic enough to take pleasures in this...

I also hope some python developers are reading this thread as well.. 
 Call this a desperate plea from some of us..

Cheers
Maurice
--
http://mail.python.org/mailman/listinfo/python-list

Re: bytecode non-backcompatibility

[Maurice LING]

It is a nuisance. It's more of a nuisance when third part modules with
'c' components are compiled for the new version of python (a windows
specific problem).
I did find that the last upgrade forced me to evaluate which extensions
I actually used. The answer was 'a few less than I thought'. It became
a good opportunity to spring clean my 'site-packages' folder.

I find this part of the story a nuisance, C components in 3rd party 
modules... What are the C components compiled into? What are actually 
"so" files?

I am using Fink to maintain my Python installation and Fink maintains 
the 3rd party libraries in /sw/lib/python2.3/site-packages. Imagine the 
trouble if I do a "fink selfupdate" and "fink update-all" and finds that 
my Python is replaced by a newer version, say Python 2.4. Then all my 
3rd party libraries in /sw/lib/python2.3/site-packages are un-usable... 
And I will have to re-download and re-install all the libraries again.

Yes, I do agree that it is a nice time to clean out "site-packages" but 
imagine the work of a system admin who has to re-install 50 packages... 
I do believe that we can use some help here, if possible... Perhaps 
someone can enlighten us on the technicalities of "so" files and/or C 
bindings in Python...

Now I understand that Python bytecodes are only dealing with pure python 
source codes. However, the same question lies, how can it (set of 
bytecodes) be made stable, like Java bytecodes, which are pretty stable? 
 Perhaps a better question will be, what techniques Java VM designers 
use that enables Java class files (and JAR files) to be stable and 
usable across versions that is lacking in Python?

Thanks.
Cheers
Maurice
--
http://mail.python.org/mailman/listinfo/python-list

bytecode non-backcompatibility

[Maurice LING]

Hi,
I've been using Python for about 2 years now, for my honours project and 
now my postgrad project. I must say that I am loving it more and more 
now. From my knowledge, Python bytecodes are not back-compatible. I must 
say that my technical background isn't strong enough but is there any 
good reason for not being back-compatible in bytecodes?

My problem is not about pure python modules or libraries but the problem 
is with 3rd party libraries with C bindings (not python pure). It means 
that with every upgrade of python, I have to reinstall all my 3rd party 
libraries which can be quite a bit of work...

I do hope this problem will be sorted out some day.
Cheers
Maurice
--
http://mail.python.org/mailman/listinfo/python-list

Re: python LEX

[Maurice LING]

jozo wrote:
I have to work on python lexical definition in Lex. I spent lots of my
time to find regular expresions written for Lex of Python language but
nothing.
Can somebody help me?
I nEED hELP 


Jython has a Python language lexer written using JavaCC.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Embedding & Extending Python &other scripting languages

[Maurice LING]

Hi Tommy,
After reading what you've written, it is still very vague for me. Is it 
a program that reads a specification and outputs the corresponding codes 
in the langauge you want?

Cheers
Maurice
Tommy Nordgren wrote:
I'm interested in doing a rather ambitious project concerning compiler
construction tools. 
My tools will parse specification files containing for example lalr
parser specifications.
The specifications will contain embedded semantic actions which i want to
allow writing in any object-oriented language.
The specifications will name a scripting language and a target language 
module . Then my tool will create an embedded interpreter for the 
scripting language, which will load a target language module written in 
the scripting language, and use it to generate code in the target 
language.

I want to provide my system as open source.
Now my basic problem is how to find what scripting languages are 
installed on a users system, and generate the necessary makefiles 
automatically. I will be using swig for interface creation.
I want to support at least perl and python, with target language support 
for at least java and c++ initially.

I probably want to support codegenerators written in ruby and tcl as 
well.

Any help would be appreciated.
I prefer email replies to <[EMAIL PROTECTED]>
Stockholm, April 21, 2005
Tommy Nordgren
--
http://mail.python.org/mailman/listinfo/python-list

Using Jython in Ant Build Process

[Maurice LING]

Hi,
I am looking for a way to use Jython in Ant build process. I have some 
pure Python scripts (not using any C extensions) that I'll like to 
incorporate into Java using Jython. I heard that this can be done but 
you can I set up Ant to do this? Sorry, I'm no expert with Ant.

By the way, I'm using Eclipse.
Thanks again.
Cheers
Maurice
--
http://mail.python.org/mailman/listinfo/python-list

Re: new to mac OS10

[Maurice LING]

Robert Kern wrote:
3. Apple-installed Python's command line tools are symlinked from 
/usr/bin to /System/Library/Frameworks/Python.framework but the OSX 
installer for Python 2.4.1 places the commandline tools in 
/usr/local/bin and symlinked to /Library/Frameworks/Python.framework. 
So it seems to me that Python 2.4.1 (installed using OSX installer for 
Python 2.4.1) is not a precise replacement of Apple-installed Python...

Bingo. It's not intended to be one.
Hi Robert,
Once I've tried to install Python 2.4.1 through the installer Bob 
Ippolito built. I've already for Fink installed Python 2.3.5 in my 
system. So,

/usr/bin/python -> /sw/bin/python
and the newly installed Python 2.4.1 created /usr/local/bin/python -> 
/Library/Frameworks/Python.framework/python

Does /usr/bin/python takes precedence over /usr/local/bin/python?
Somehow it rings an alarm in me...
Cheers
Maurice
--
http://mail.python.org/mailman/listinfo/python-list

Re: new to mac OS10

[Maurice LING]

Hi Thomas,
It seems that you've cleanly killed the Apple-installed Python, which 
isn't too bad a thing after all. What I can suggest you do is this... 
Copy the entire /System/Library/Frameworks/Python.framework directory 
from someone and drop it into your system (same place of course). I will 
not suggest installing Python 2.4.1 until the Apple-installed Python is 
sorted out. My reasons being,

1. I have no idea what Mac OSX uses Python for. Although symlink may get 
you through most of the time but I cannot be sure that none of OSX's 
stuffs are hardcoded to use the Python in 
/System/Library/Frameworks/Python.framework.

2. Installing another Python may or may not be a precise remedy. It may 
just worsen things and I won't want to bet on that.

3. Apple-installed Python's command line tools are symlinked from 
/usr/bin to /System/Library/Frameworks/Python.framework but the OSX 
installer for Python 2.4.1 places the commandline tools in 
/usr/local/bin and symlinked to /Library/Frameworks/Python.framework. So 
it seems to me that Python 2.4.1 (installed using OSX installer for 
Python 2.4.1) is not a precise replacement of Apple-installed Python...

For me, my setup is this
ls -all /usr/bin/python*
lrwxr-xr-x  1 root  wheel   14  4 Apr 08:40 /usr/bin/python -> 
/sw/bin/python
lrwxr-xr-x  1 root  wheel   72  1 Apr  1976 /usr/bin/python2.3 -> 
../../System/Library/Frameworks/Python.framework/Versions/2.3/bin/python
lrwxr-xr-x  1 root  wheel   10  1 Apr  1976 /usr/bin/pythonw -> pythonw2.3
-rwxr-xr-x  1 root  wheel  122  9 Jun  2003 /usr/bin/pythonw2.3

/usr/bin/pythonw2.3 and /usr/bin/python2.3 points to the same thing, 
/System/Library/Frameworks/Python.framework/Versions/2.3/bin/python. And 
I have Fink's Python in /sw/bin/python.

So now, for me, if I need to use the Apple-installed Python, my command 
line is "pythonw" while "python" gives me Fink's Python as shown below:

~ mauriceling$ pythonw
Python 2.3 (#1, Sep 13 2003, 00:49:11)
[GCC 3.3 20030304 (Apple Computer, Inc. build 1495)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path
['', 
'/System/Library/Frameworks/Python.framework/Versions/2.3/lib/python23.zip', 
'/System/Library/Frameworks/Python.framework/Versions/2.3/lib/python2.3', 
'/System/Library/Frameworks/Python.framework/Versions/2.3/lib/python2.3/plat-darwin', 
'/System/Library/Frameworks/Python.framework/Versions/2.3/lib/python2.3/plat-mac', 
'/System/Library/Frameworks/Python.framework/Versions/2.3/lib/python2.3/plat-mac/lib-scriptpackages', 
'/System/Library/Frameworks/Python.framework/Versions/2.3/lib/python2.3/lib-tk', 
'/System/Library/Frameworks/Python.framework/Versions/2.3/lib/python2.3/lib-dynload', 
'/System/Library/Frameworks/Python.framework/Versions/2.3/lib/python2.3/site-packages']
>>>

~ mauriceling$ python
Python 2.3.5 (#1, Apr  6 2005, 13:01:10)
[GCC 3.3 20030304 (Apple Computer, Inc. build 1671)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path
['', '/sw/lib/python23.zip', '/sw/lib/python2.3', 
'/sw/lib/python2.3/plat-darwin', '/sw/lib/python2.3/plat-mac', 
'/sw/lib/python2.3/plat-mac/lib-scriptpackages', 
'/sw/lib/python2.3/lib-tk', '/sw/lib/python2.3/lib-dynload', 
'/sw/lib/python2.3/site-packages', 
'/sw/lib/python2.3/site-packages/Numeric', 
'/sw/lib/python2.3/site-packages/libsbml', 
'/sw/lib/python2.3/site-packages/gtk-2.0']
>>>

There is no problem with having multiple versions and copies of Pythons 
in your system. The problems are:

1. get all the symlinks correct
2. know when to use what and how to use
3. means that you may have to install the same 3rd party library x-times 
for x-numbers of Pythons you might have... quite a pain...

Cheers
Maurice
--
http://mail.python.org/mailman/listinfo/python-list

Re: RE Engine error with sub()

[Maurice LING]

Solved it. Instead of modifying Replacer class, I've made another class 
which initiates a list of Replacer objects from a list of substitution 
rule files. And then iterates through the list of Replacer objects and 
calls upon their own substitute() method. It seems to work.

Thanks for all your advices.
Cheers
Maurice
Maurice LING wrote:
Hi all,
I think I might have a workaround to this problem but have no idea how 
to work it through. I hope that someone can kindly help me out because I 
do not quite understand the mechanics of the _make_regex() method in the 
original codes...

My idea is, instead of having one UserDict, have a list of UserDicts. So 
 a large unprocessable replacement rule set is split into multiple 
smaller files, with each file read into a UserDict and it is made into a 
 RE matcher. Then iterative matching using a list of REs.

In short, the current precedure is
1 dictionary, 1 RE, 1 RE matcher... to match inputs
My workaround is to change it to
list of dictionaries, list of REs, list of RE matcher... iterative 
matching of inputs.

Can someone kindly help me out here?
Thanks in advance.
Cheers,
Maurice
--
http://mail.python.org/mailman/listinfo/python-list

Re: RE Engine error with sub()

[Maurice LING]

Hi all,
I think I might have a workaround to this problem but have no idea how 
to work it through. I hope that someone can kindly help me out because I 
do not quite understand the mechanics of the _make_regex() method in the 
original codes...

My idea is, instead of having one UserDict, have a list of UserDicts. So 
 a large unprocessable replacement rule set is split into multiple 
smaller files, with each file read into a UserDict and it is made into a 
 RE matcher. Then iterative matching using a list of REs.

In short, the current precedure is
1 dictionary, 1 RE, 1 RE matcher... to match inputs
My workaround is to change it to
list of dictionaries, list of REs, list of RE matcher... iterative 
matching of inputs.

Can someone kindly help me out here?
Thanks in advance.
Cheers,
Maurice
Maurice LING wrote:
Hi,
I have the following codes:
from __future__ import nested_scopes
import re
from UserDict import UserDict
class Replacer(UserDict):
"""
An all-in-one multiple string substitution class. This class was 
contributed by Xavier
Defrang to the ASPN Python Cookbook 
(http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/81330)
and [EMAIL PROTECTED]

Copyright: The methods _make_regex(), __call__() and substitute() 
were the work of Xavier Defrang,
__init__() was the work of [EMAIL PROTECTED], all others were 
the work of Maurice Ling"""

def __init__(self, dict = None, file = None):
"""Constructor. It calls for the compilation of regular 
expressions from either
a dictionary object or a replacement rule file.

@param dict: dictionary object containing replacement rules with 
the string to be
replaced as keys.
@param file: file name of replacement rule file
"""
self.re = None
self.regex = None
if file == None:
UserDict.__init__(self, dict)
self._make_regex()
else:
UserDict.__init__(self, self.readDictionaryFile(file))
self._make_regex()

def cleanDictionaryFile(self, file):
"""
Method to clean up the replacement rule dictionary file and 
write the cleaned
file as the same name as the original file."""
import os
dict = self.readDictionaryFile(file)
f = open(file, 'w')
for key in dict.keys(): f.write(str(key) + '=' + str(dict[key]) 
+ os.linesep)
f.close()

def readDictionaryFile(self, file):
"""
Method to parse a replacement rule file (file) into a dictionary 
for regular
expression processing. Each rule in the rule file is in the form:
=
"""
import string
import os
f = open(file, 'r')
data = f.readlines()
f.close()
dict = {}
for rule in data:
rule = rule.split('=')
if rule[1][-1] == os.linesep: rule[1] = rule[1][:-1]
dict[str(rule[0])] = str(rule[1])
print '%s replacement rule(s) read from %s' % 
(str(len(dict.keys())), str(file))
return dict

def _make_regex(self):
""" Build a regular expression object based on the keys of the 
current dictionary """
self.re = "(%s)" % "|".join(map(re.escape, self.keys()))
self.regex = re.compile(self.re)

def __call__(self, mo):
""" This handler will be invoked for each regex match """
# Count substitutions
self.count += 1 # Look-up string
return self[mo.string[mo.start():mo.end()]]
def substitute(self, text):
""" Translate text, returns the modified text. """
# Reset substitution counter
self.count = 0
# Process text
#return self._make_regex().sub(self, text)
return self.regex.sub(self, text)
def rmBracketDuplicate(self, text):
"""Removes the bracketed text in occurrences of ' 
()'"""
regex = re.compile(r'(\w+)\s*(\(\1\))')
return regex.sub(r'\1', text)

def substituteMultiple(self, text):
"""Similar to substitute() method except that this method loops 
round the same text
multiple times until no more substitutions can be made or when 
it had looped
10 times. This is to pre-ampt for cases of recursive 
abbreviations."""
count = 1 # to get into the loop
run = 0 # counter for number of runs thru the text
while count > 0 and run < 10:
count = 0
text = self.rmBracketDuplicate(self.substitute(text))
count = count + self.count
run = run + 1
print "Pass

Re: RE Engine error with sub()

[Maurice LING]

Hi Dennis,
Dennis Benzinger wrote:
Maurice LING schrieb:
Hi,
I have the following codes:
from __future__ import nested_scopes
 > [...]
Are you still using Python 2.1?
In every later version you don't need the
"from __future__ import nested_scopes" line.
So, if you are using Python 2.1 I strongly recommend
upgrading to Python 2.4.1.
I am using Python 2.3.5, installed using Fink. That is the latest 
version Fink has to offer.

[...]
It all works well for rule count up to 800+ but when my replacement 
rules swells up to 1800+, it gives me a runtime error that says 
"Internal error in regular expression engine"... traceable to "return 
self.regex.sub(self, text)" in substitute() method.
[...]

I didn't read your code, but this sounds like you have a problem with 
the regular expression engine being recursive in Python versions < 2.4.
Try again using Python 2.4 or later (i.e. Python 2.4.1). The new regular 
expression engine is not recursive anymore.

Apparently this problem had been reported in Bugs item #857676 
(http://mail.python.org/pipermail/python-bugs-list/2003-December/021473.html) 
and (http://www.lehuen.com/nicolas/index.php/Pytst/2005/04). Bugs item 
#857676 is consistent with my problem as in it works with smaller lists 
(~1000) but not much bigger than that. The problem seems to lie in the 
fact that the SRE engine works on a 16-bit opcode...

Cheers
Maurice
--
http://mail.python.org/mailman/listinfo/python-list

RE Engine error with sub()

[Maurice LING]

Hi,
I have the following codes:
from __future__ import nested_scopes
import re
from UserDict import UserDict
class Replacer(UserDict):
"""
An all-in-one multiple string substitution class. This class was 
contributed by Xavier
Defrang to the ASPN Python Cookbook 
(http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/81330)
and [EMAIL PROTECTED]

Copyright: The methods _make_regex(), __call__() and substitute() 
were the work of Xavier Defrang,
__init__() was the work of [EMAIL PROTECTED], all others were 
the work of Maurice Ling"""

def __init__(self, dict = None, file = None):
"""Constructor. It calls for the compilation of regular 
expressions from either
a dictionary object or a replacement rule file.

@param dict: dictionary object containing replacement rules 
with the string to be
replaced as keys.
@param file: file name of replacement rule file
"""
self.re = None
self.regex = None
if file == None:
UserDict.__init__(self, dict)
self._make_regex()
else:
UserDict.__init__(self, self.readDictionaryFile(file))
self._make_regex()

def cleanDictionaryFile(self, file):
"""
Method to clean up the replacement rule dictionary file and 
write the cleaned
file as the same name as the original file."""
import os
dict = self.readDictionaryFile(file)
f = open(file, 'w')
for key in dict.keys(): f.write(str(key) + '=' + str(dict[key]) 
+ os.linesep)
f.close()

def readDictionaryFile(self, file):
"""
Method to parse a replacement rule file (file) into a 
dictionary for regular
expression processing. Each rule in the rule file is in the form:
=
"""
import string
import os
f = open(file, 'r')
data = f.readlines()
f.close()
dict = {}
for rule in data:
rule = rule.split('=')
if rule[1][-1] == os.linesep: rule[1] = rule[1][:-1]
dict[str(rule[0])] = str(rule[1])
print '%s replacement rule(s) read from %s' % 
(str(len(dict.keys())), str(file))
return dict

def _make_regex(self):
""" Build a regular expression object based on the keys of the 
current dictionary """
self.re = "(%s)" % "|".join(map(re.escape, self.keys()))
self.regex = re.compile(self.re)

def __call__(self, mo):
""" This handler will be invoked for each regex match """
# Count substitutions
self.count += 1 # Look-up string
return self[mo.string[mo.start():mo.end()]]
def substitute(self, text):
""" Translate text, returns the modified text. """
# Reset substitution counter
self.count = 0
# Process text
#return self._make_regex().sub(self, text)
return self.regex.sub(self, text)
def rmBracketDuplicate(self, text):
"""Removes the bracketed text in occurrences of ' 
()'"""
regex = re.compile(r'(\w+)\s*(\(\1\))')
return regex.sub(r'\1', text)

def substituteMultiple(self, text):
"""Similar to substitute() method except that this method loops 
round the same text
multiple times until no more substitutions can be made or when 
it had looped
10 times. This is to pre-ampt for cases of recursive 
abbreviations."""
count = 1 # to get into the loop
run = 0 # counter for number of runs thru the text
while count > 0 and run < 10:
count = 0
text = self.rmBracketDuplicate(self.substitute(text))
count = count + self.count
run = run + 1
print "Pass %d: Changed %d things(s)" % (run, count)
return text


Normally I will use the following to instantiate my module:
replace = Replacer('', 'rule.mdf')
rule.mdf is in the format of "=\n"

Then using replace.substituteMultiple('') to carry out multiple 
replacements.

It all works well for rule count up to 800+ but when my replacement 
rules swells up to 1800+, it gives me a runtime error that says 
"Internal error in regular expression engine"... traceable to "return 
self.regex.sub(self, text)" in substitute() method.

Any ideas or workarounds?
Thanks in advance.
Cheers,
Maurice
--
http://mail.python.org/mailman/listinfo/python-list

Re: new to mac OS10

[Maurice LING]

I was actually trying to update to the newest python version, and I had 
read something saying it would conflict with the old version, so I went 
through and deleted all the folders that had "python" in the name =]. 
clever of me, huh?  now I can't make either the new or the old work. 
Once again, any help would be terrific.

THN
Ok... I am going to assume a few things and will need you to check a few 
things... please tell me if my assumptions are wrong...

assumption 1: you are afraid of "python overload' in Mac because you 
want the latest version of Python but Apple doesn't seems to maintain 
the upgrade of the Apple-installed Python.

assumption 2: you are installing the latest version of Python through Fink.
I need you to check the following things and tell me the results:
1. go to /usr/bin and do a "ls -all python*"
2. check for presence of /System/Library/Framework/Python.framework
3. do a "which python"
4. do a "which pythonw"
5. execute "python"
6. execute "pythonw"
i need to assess the damage done :P
Cheers
Maurice
--
http://mail.python.org/mailman/listinfo/python-list

Re: new to mac OS10

[Maurice LING]

Thomas Nelson wrote:
I'm on a mac OS X (10.3.8), and I seem to have accidentally destroyed 
the default python installation.  How should I put it on?  Do I need to 
use the unix version?  any help would be greatly appreciated.

THN
Hi Thomas,
I'm using OSX 10.3.8 as well. Just wondering, how did you "destroy" it? 
What I am thinking is, it may not be as destroyed as you think it might 
have...

cheers
maurice
--
http://mail.python.org/mailman/listinfo/python-list