C extensions and memory leakage

[Sheldon]

Hi,

I have a program that I have extended with a C function. When the
program is run once or twice everything is ok but in a loop of 5 to 12
iterations, the memory runs out and the program crashes.
Now I have gone through this program thoroughly to check if all arrays
have been deallocated prior to exiting and they have, but still the
problem exists.

Now I am wondering if the problem is in Python and the wrapper? Does
anybody have any idea or experience with this? I am running on
Mandrake10 using python 2.3. I am not exactly sure which C wrapper I am
using as I have copied it from another person.

thanks in advance,
/Sheldon

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: dict.reserve and other tricks

[bearophileHUGS]

Thank you for the answers Terry Reedy and Klaas.

> Since you are writing extensions, you can create a built-in subclass of
> dict to experiment with.  I presume the 2.5 default dict should be a model.

That way it's doable, but I think it's of limited use too; I'd like to
remove elements from arbitrary (normal) dicts.


Klaas:

> Well, you can reduce the memory usage to virtually nothing by using a
> generator expression rather than list comprehension.

Are you sure? I don't think so. Can you show a little example?
As you know this can't work:

def filterdict(pred, indict):
  todel = (k for k,v in indict.iteritems() if not pred(k,v))
  for key in todel:
del indict[key]

d = dict.fromkeys(xrange(8), 0)
print d
filterdict(lambda k,v: k & 1, d)
print d

(You can remove elements from the dict and put  them into a list, and
then put back the elements into the dict, this is probably the only way
I know of *theoretically* keeping the memory used about  the same with
Python. In practice the less memory consuming version may be the
filterdict I have shown).


> arbitrary python code can be executed by __hash__
> and deleting (DECREF) python objects.

I am starting to understand. I'll think more about this.

Thank you,
bye,
bearophile

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: problem with defining a global class instance

[[EMAIL PROTECTED]]

sharath B N wrote:
> hi,
> i am sort of newbie to python. I am trying to do a super Market
> simulation with OOP in python. I have problems with using a class
> instance as global...
> def generate (... ,,...)
>
> " in this function i define the global variables "
> global  stock,stockManager, manager etc.
>
>
> class Manager
> ...
> ...
> ...
> def  create_stockManager(..)
> """ this is a method in class manager"""
> stockManager = StockManager( name)
> stockManager.create_Stock(..)
>
>
> now this gives an attribute error sayin  stockManager has no
> attribute create_Stock
>
> if i create the StockManager instance in the generate func
> itself...then this problem doesnt comebut i need it this way for
> the program to make sense..
> can somebody help me
> thnks
> Sharath

Python uses the concept of namespaces, the keyword 'global' binds a
local variable to the global namespace, take as example:

# this amount is in the global namespace
amount = 98

def get_amount_local():
# this amount is local in this function
amount = 23
return amount

def get_amount_global():
# this global is taken from the gloabl namespace
global amount
return amount

print get_amount_local() # so this will print 23
print get_amount_global() # so this will print 98

see  http://docs.python.org/ref/assignment.html

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: popen(1-4) as a seperate process

[Astan Chee]

Fredrik Lundh wrote:
> Astan Chee wrote:
>
>   
>> Yes, that is true. But everytime I run a os.popen() it executes as a 
>> child process of the current running one. How do I launch as a seperate 
>> process?
>> 
>
> what's your definition of "separate process", and how is that different 
> from a child process?  (all processes created by a process start out as 
> child processes).
>
>   
>> Note that I dont care about whatever happens to the seperate process 
>> itself.
>> 
>
> so why are you using os.popen() if you're not interested in talking to 
> it?  maybe os.system("command&") would be more appropriate ?
>
> 
>
>   
My appologies,
What I meant by seperate process is that the spawned process is not the 
child of the current running process or the spawned process does not 
have a parent (if that is possible).
popen not a good idea? Im not sure myself frankyl.
What would be a good method to do this? I've tried your 
os.system("command&") and the spawned process is still the child of the 
current running process (which is not what I want to happen).
Thanks again and sorry for the confusion.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: [wxPython] wxFrame don't have Bind attribute ??

[Franz Steinhaeusler]

On 29 Oct 2006 22:31:09 -0800, "Jia Lu" <[EMAIL PROTECTED]> wrote:

>Hi all
> I am using wxPy 2.6.3.2-2, But when run an application with self.Bind
>, I got an error that there is no Bind.
>
> How can I fix it. thanx

Is it really an instance of wx.Frame? 
What do you get, if you make a "print self" statement?
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: popen(1-4) as a seperate process

[Fredrik Lundh]

Astan Chee wrote:

> Yes, that is true. But everytime I run a os.popen() it executes as a 
> child process of the current running one. How do I launch as a seperate 
> process?

what's your definition of "separate process", and how is that different 
from a child process?  (all processes created by a process start out as 
child processes).

> Note that I dont care about whatever happens to the seperate process 
> itself.

so why are you using os.popen() if you're not interested in talking to 
it?  maybe os.system("command&") would be more appropriate ?



-- 
http://mail.python.org/mailman/listinfo/python-list

Re: problem with defining a global class instance

[Fredrik Lundh]

sharath B N wrote:

> def generate (... ,,...)
> 
> " in this function i define the global variables "
> global  stock,stockManager, manager etc.
> 
> 
> class Manager
> ...
> ...
> ...
> def  create_stockManager(..)
> """ this is a method in class manager"""
> stockManager = StockManager( name)
> stockManager.create_Stock(..)
> 
> now this gives an attribute error sayin  stockManager has no
> attribute create_Stock

so where is it?



-- 
http://mail.python.org/mailman/listinfo/python-list

problem with defining a global class instance

[sharath B N]

hi,
i am sort of newbie to python. I am trying to do a super Market
simulation with OOP in python. I have problems with using a class
instance as global...
def generate (... ,,...)

" in this function i define the global variables "
global  stock,stockManager, manager etc.


class Manager
...
...
...
def  create_stockManager(..)
""" this is a method in class manager"""
stockManager = StockManager( name)
stockManager.create_Stock(..)


now this gives an attribute error sayin  stockManager has no
attribute create_Stock

if i create the StockManager instance in the generate func
itself...then this problem doesnt comebut i need it this way for
the program to make sense..
can somebody help me
thnks
Sharath
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: how to print pdf with python on a inkjet printer.

[Tim Roberts]

"krishnakant Mane" <[EMAIL PROTECTED]> wrote:
>
>hello all.
>I am developing an ncurses based python application that will require
>to create pdf reports for printing.
>I am not using py--qt or wx python.
>it is a consol based ui application and I need to make a pdf report
>and also send it to a lazer or ink jet printer.
>is it possible to do so with python?

Making the PDF is easy.  Go get ReportLab from www.reportlab.org.  I
consider it the best Python PDF solution.

Printing is more complicated, because you actually need something that can
RENDER the PDF.  The most common such renderer is the Acrobat Reader, and
you can actually call the Acrobat Reader from a command line with a
parameter that tells it to print the file automatically.  The only
disadvantage is that using that method will only print to the "default"
printer.

>or is it that I will have to use the wxpython library asuming that
>there is a print dialog which can open up the list of printers?

Even if you got the list of printers, what would you do with it?
-- 
Tim Roberts, [EMAIL PROTECTED]
Providenza & Boekelheide, Inc.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python v PHP: fair comparison?

[Fredrik Lundh]

[EMAIL PROTECTED] wrote:

> Huh?  I've been doing paid python web work since 2000.  I'm fairly sure
> that Yahoo! groups predates that by a while

http://en.wikipedia.org/wiki/EGroups



-- 
http://mail.python.org/mailman/listinfo/python-list

Re: determining the bounds of a tuple returned from a database

[ronrsr]

very sorry, that was my error - len(result[0]) and len(result[1]) both
return 1 --

i think I'm misunderstanding what len() does - to me they appear to
have 2 or 3 elements, or at least be composed of a string of some
length.

I guess len() isn't the function i'm looking for then.  How do I tell
how many strings, or how many bytes are in each dimension?  so that I
can iterate through them.

Each entry in that list is a keyword - Alternately, is there any fast
way to parse that into a sorted list of distinct keywords.

very sorry for the error.

bests,

-rsr-

Fredrik Lundh wrote:
> ronrsr wrote:
>
> > it looks like the  len() function is the one I want.
> >
> > for:   len(result) - i get 248,
> >
> > but for len(result[0]) or len(result[1]) i always get 0.
>
> that's a bit surprising, because both items are tuples that contain
> exactly one item:
>
> >> (('Agricultural subsidies; Foreign aid',), ('Agriculture; Sustainable
> >> Agriculture - Support; Organic Agriculture; Pesticides, US, Childhood
> >> Development, Birth Defects; Toxic Chemicals',),
> 
> 

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: popen(1-4) as a seperate process

[Astan Chee]

Fredrik Lundh wrote:

Astan Chee wrote:

  
Im trying to popen (or more specifically os.popen4() ) from wxPython. 
I've read the documentation on popen and it says I can do a popen as a 
seperate process or popen not as a child process



where does it say that?  afaik, the whole point of the popen API is to 
run an external process and use pipes to communicate with it.




  
Yes, that is true. But everytime I run a os.popen() it executes as a 
child process of the current running one. How do I launch as a seperate 
process?
Note that I dont care about whatever happens to the seperate process 
itself.

Thanks
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: popen(1-4) as a seperate process

[Fredrik Lundh]

Astan Chee wrote:

> Im trying to popen (or more specifically os.popen4() ) from wxPython. 
> I've read the documentation on popen and it says I can do a popen as a 
> seperate process or popen not as a child process

where does it say that?  afaik, the whole point of the popen API is to 
run an external process and use pipes to communicate with it.



-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How fuzzy is get_close_matches() in difflib?

[John Henry]

I encountered a case where I am trying to match "HIDESST1" and
"HIDESCT1" against ["HIDEDST1", "HIDEDCT1", "HIDEDCT2", "HIDEDCT3"]

Well, they both hit "HIDEDST1" as the first match which is not exactly
the result I was looking for.  I don't understand why "HIDESCT1" would
not hit "HIDEDCT1" as a first choice.

Steven D'Aprano wrote:
> On Thu, 16 Nov 2006 20:19:50 -0800, John Henry wrote:
>
> > I did try them and I am impressed.  It helped me found a lot of useful
> > info.   I just want to get a feel as to what constitutes a "match".
>
> The source code has lots of comments, but they don't explain the basic
> algorithm (at least not in the difflib.py supplied with Python 2.3).
>
> There is no single diff algorithm, but I believe that the basic idea is to
> look for insertions and/or deletions of strings. If you want more
> detail, google "diff". Once you have a list of differences, the closest
> match is the search string with the fewest differences.
>
> As for getting a feel of what constitutes a match, I really can't make any
> better suggestion than just try lots of examples with the interactive
> Python shell.
> 
> 
> 
> -- 
> Steven D'Aprano

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: os.lisdir, gets unicode, returns unicode... USUALLY?!?!?

[Fredrik Lundh]

gabor wrote:

> get an Unicode-exception, as everywhere else. you see, exceptions are 
> ok, i can deal with them.

> p.s: one additional note. if you code expects os.listdir to return 
> unicode, that usually means that all your code uses unicode strings. 
> which in turn means, that those filenames will somehow later interact 
> with unicode strings. which means that that byte-string-filename will 
> probably get auto-converted to unicode at a later point, and that 
> auto-conversion will VERY probably fail

it will raise an exception, most likely.  didn't you just say that 
exceptions were ok?



-- 
http://mail.python.org/mailman/listinfo/python-list

Re: determining the bounds of a tuple returned from a database

[Fredrik Lundh]

ronrsr wrote:

> it looks like the  len() function is the one I want.
> 
> for:   len(result) - i get 248,
> 
> but for len(result[0]) or len(result[1]) i always get 0.

that's a bit surprising, because both items are tuples that contain 
exactly one item:

>> (('Agricultural subsidies; Foreign aid',), ('Agriculture; Sustainable
>> Agriculture - Support; Organic Agriculture; Pesticides, US, Childhood
>> Development, Birth Defects; Toxic Chemicals',),



-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PIL - "despeckle" image ?

[Paul McGuire]

On Nov 16, 10:58 pm, [EMAIL PROTECTED] wrote:
> Hi - I have some images which I would like to remove specks from using
> the PIL. I would like to be able to say that if a pixel is in a blob of
> less than n contiguous pixels then all pixels in that blob should be
> removed.
>
> The images are 8 bit images with only 0 and 255 values.
>
> I can think of quite crude ways of doing this but I'm sure there are
> some smart ways of doing it - would anyone be able to point me in the
> right direction ?
>
> Lastly in case it matters the reason I'm doing this is that the images
> are scanned images of sheets of papers on which sample signatures have
> been placed. Typically the signatures are much smaller than the sheets
> of paper so I would like to 'autocrop' the images to remove the white
> space around the signature. At some point in the process the images
> having picked up small islands of pixels (dust on scanner or paper ?)
> which are nowhere near the signature but which are causing my autocrop
> efforts to go awry.
>

What if you break up your sheet into zones of some arbitrary size (1/2"
say), and compute a pixel density?  (Density isn't really required,
since all zones are the same size - raw pixel count per zone will do
just as well.)  Then locate the signature based on the zones with
density above threshold X (determine X by trial and error).  Lastly,
clip within these zones if this is still necessary.

-- Paul

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: determining the bounds of a tuple returned from a database

[ronrsr]

it looks like the  len() function is the one I want.

for:   len(result) - i get 248,

but for len(result[0]) or len(result[1]) i always get 0.


ronrsr wrote:
> here is my result.
> How do I determine the number of tuples in this array, returned from a
> mysql database?
>
> How do I determine the number of characters or entry in each tuple?
>
> thanks very much for your assistance,
>
> -rsr-
>
>
> (('Agricultural subsidies; Foreign aid',), ('Agriculture; Sustainable
> Agriculture - Support; Organic Agriculture; Pesticides, US, Childhood
> Development, Birth Defects; Toxic Chemicals',), ('Antibiotics,
> Animals',), ('Agricultural Subsidies, Global Trade',), ('Agricultural
> Subsidies',), ('Biodiversity',), ('Citizen Activism',), ('Community
> Gardens',), ('Cooperatives',), ('Dieting',), ('Agriculture, Cotton',),
> ('Agriculture, Global Trade',), ('Pesticides, Monsanto',),
> ('Agriculture, Seed',), ('Coffee, Hunger',), ('Pollution, Water,
> Feedlots',), ('Food Prices',), ('Agriculture, Workers',), ('Animal
> Feed, Corn, Pesticides',), ('Aquaculture',), ('Chemical Warfare',),
> ('Compost',), ('Debt',), ('Consumerism',), ('Fear',), ('Pesticides, US,
> Childhood Development, Birth Defects',), ('Corporate Reform,
> Personhood (Dem. Book)',), ('Corporate Reform,  Personhood, Farming
> (Dem. Book)',), ('Crime Rates, Legislation, Education',), ('Debt,
> Credit Cards',), ('Democracy',), ('Population, World',), ('Income',),
> ('Democracy, Corporate Personhood, Porter Township (Dem. Book)',),
> ('Disaster Relief',), ('Dwellings, Slums',), ('Economics, Mexico',),
> ('Economy, Local',), ('Education, Protests',), ('Endangered Habitat,
> Rainforest',), ('Endangered Species',), ('Endangered Species,
> Extinction',), ('antibiotics, livestock',), ('Pesticides, Water',),
> ('Environment, Environmentalist',), ('Food, Hunger, Agriculture, Aid,
> World, Development',), ('Agriculture, Cotton Trade',), ('Agriculture,
> Cotton, Africa',), ('Environment, Energy',), ('Fair Trade (Dem.
> Book)',), ('Farmland, Sprawl',), ('Fast Food, Globalization,
> Mapping',), ('depression, mental illness, mood disorders',), ('Economic
> Democracy, Corporate Personhood',), ('Brazil, citizen activism, hope,
> inspiration, labor issues',), ('citizen activism, advice, hope',),
> ('Pharmaceuticals, Medicine, Drugs',), ('Community Investing',),
> ('Environment, Consumer Waste Reduction, Consumer Behavior and
> Taxes',), ('Hunger, US, Poverty',), ('FERTILITY, Women',),
> ('Corporatism, Environment',), ('Economic Democracy, Corporate
> Farming',), ('Economic Democracy, Inspiration',), ('FEAR
> (Fearlessness)',), ('Federal budget, military spending, foreign
> cultural exchange programs',), ('Fish Farming',), ('Fish population,
> commercial fishing industry',), ('GMO, BT Cotton',), ('Food borne
> Illness, Water',), ('Food Charity',), ('Food Charity, Urban Farming',),
> ('Food Consumption, Luxury',), ('Food Cost, international',), ('Food
> Disposal, Waste',), ('Food stamps, eligibility for',), ('Food,
> expenditure on',), ('Foreign Aid',), ('Genetics, Food, Agriculture,
> International',), ('Global Policy, Bush Administration',), ('Global
> Trade, food',), ('Global Trade, free trade, clothing',),
> ('Globalization',), ('GMO, SACTO Protests',), ('GMO, Wheat, trade',),
> ('Happiness and income',), ('Health Care - Spending',), ('Hope',),
> ('Hope, inspirational quote',), ('Human Cloning',), ('Hunger',),
> ('Hunger, income',), ('IMF, Argentina',), ('IMF, failings',),
> ('Inequality',), ('INEQUALITY, GLOBALIZATION, POVERTY',), ('Inequality,
> Globalization, Latin America',), ('Inspiration',), ('Junk Food',),
> ('Junk Food, Soft Drinks',), ('Junk Food ,Children, Education,
> Media',), ('Junk Food, Education, Children, Nutrition',), ('Junk Food,
> Global, Coke',), ('Junk Food, McDonald\x92s',), ('Junk Food,
> McDonald\x92s, Global',), ('Junk Food, Soft Drinks, Water',), ('Junk
> Food, Education, Nutrition, Childhood Illness',), ('Junk food,
> Nutrition, Schools',), ('Labor',), ('Labor Issues, Agriculture,
> Sustainable Agriculture Support',), ('Labor Issues, Employee
> Ownership',), ('Land Reform, Brazil, Infant mortality',),
> ('Livestock',), ('Meat',), ('Meat, Brazil, Soybeans, Deforestation',),
> ('Meat, concentration',), ('Meat, Recalls, Contamination',), ('Meat,
> Recalls',), ('Media',), ('Microloans',), ('Military Spending',),
> ('Nutrition Programs, School Meals, Snapple',), ('Nutrition,
> Finland',), ('Obesity',), ('Obesity, Childhood',), ('Obesity,
> Africa',), ('Organic Food',), ('Obesity, Global',), ('Obesity, related
> illnesses',), ('Obesity, related illnesses, children',), ('Oil',),
> ('Organic Agriculture',), ('Organic Agriculture, Growth',), ('Organic
> Agriculture, Nutrition',), ('Organic Agriculture, Productivity',),
> ('Organic food prices',), ('Community',), ('Corporate Concentration',),
> ('Community Gardens, Urban Gardens',), ('Pollution, Air, CO2
> emissions',), ('Population, Water',), ('Poverty, Aid',), ('Poverty,
> World Bank, Hunger',), ('O

Re: python and accessibility issues.

[Paddy]

krishnakant Mane wrote:

> this question is streight forward, short and sweet.
> I am developing some applications and being blind myself, expect that
> including me, all other blind people must be able to use it.
> so my question is,
> is wx python complying with Microsoft Active Accessibility or MSAA for short?
> I will use wx python or any other gui library that is having support
> for MSAA built in.
> thanking all.
> Krishnakant.
wxpython is based on wxwindows which has a statement on accessability
here: http://www.wxwindows.org/docs/access.htm

That page has links to their forums which might be able to provide you
with more information.

- Paddy.

-- 
http://mail.python.org/mailman/listinfo/python-list

determining the bounds of a tuple returned from a database

[ronrsr]

here is my result.
How do I determine the number of tuples in this array, returned from a
mysql database?

How do I determine the number of characters or entry in each tuple?

thanks very much for your assistance,

-rsr-


(('Agricultural subsidies; Foreign aid',), ('Agriculture; Sustainable
Agriculture - Support; Organic Agriculture; Pesticides, US, Childhood
Development, Birth Defects; Toxic Chemicals',), ('Antibiotics,
Animals',), ('Agricultural Subsidies, Global Trade',), ('Agricultural
Subsidies',), ('Biodiversity',), ('Citizen Activism',), ('Community
Gardens',), ('Cooperatives',), ('Dieting',), ('Agriculture, Cotton',),
('Agriculture, Global Trade',), ('Pesticides, Monsanto',),
('Agriculture, Seed',), ('Coffee, Hunger',), ('Pollution, Water,
Feedlots',), ('Food Prices',), ('Agriculture, Workers',), ('Animal
Feed, Corn, Pesticides',), ('Aquaculture',), ('Chemical Warfare',),
('Compost',), ('Debt',), ('Consumerism',), ('Fear',), ('Pesticides, US,
Childhood Development, Birth Defects',), ('Corporate Reform,
Personhood (Dem. Book)',), ('Corporate Reform,  Personhood, Farming
(Dem. Book)',), ('Crime Rates, Legislation, Education',), ('Debt,
Credit Cards',), ('Democracy',), ('Population, World',), ('Income',),
('Democracy, Corporate Personhood, Porter Township (Dem. Book)',),
('Disaster Relief',), ('Dwellings, Slums',), ('Economics, Mexico',),
('Economy, Local',), ('Education, Protests',), ('Endangered Habitat,
Rainforest',), ('Endangered Species',), ('Endangered Species,
Extinction',), ('antibiotics, livestock',), ('Pesticides, Water',),
('Environment, Environmentalist',), ('Food, Hunger, Agriculture, Aid,
World, Development',), ('Agriculture, Cotton Trade',), ('Agriculture,
Cotton, Africa',), ('Environment, Energy',), ('Fair Trade (Dem.
Book)',), ('Farmland, Sprawl',), ('Fast Food, Globalization,
Mapping',), ('depression, mental illness, mood disorders',), ('Economic
Democracy, Corporate Personhood',), ('Brazil, citizen activism, hope,
inspiration, labor issues',), ('citizen activism, advice, hope',),
('Pharmaceuticals, Medicine, Drugs',), ('Community Investing',),
('Environment, Consumer Waste Reduction, Consumer Behavior and
Taxes',), ('Hunger, US, Poverty',), ('FERTILITY, Women',),
('Corporatism, Environment',), ('Economic Democracy, Corporate
Farming',), ('Economic Democracy, Inspiration',), ('FEAR
(Fearlessness)',), ('Federal budget, military spending, foreign
cultural exchange programs',), ('Fish Farming',), ('Fish population,
commercial fishing industry',), ('GMO, BT Cotton',), ('Food borne
Illness, Water',), ('Food Charity',), ('Food Charity, Urban Farming',),
('Food Consumption, Luxury',), ('Food Cost, international',), ('Food
Disposal, Waste',), ('Food stamps, eligibility for',), ('Food,
expenditure on',), ('Foreign Aid',), ('Genetics, Food, Agriculture,
International',), ('Global Policy, Bush Administration',), ('Global
Trade, food',), ('Global Trade, free trade, clothing',),
('Globalization',), ('GMO, SACTO Protests',), ('GMO, Wheat, trade',),
('Happiness and income',), ('Health Care - Spending',), ('Hope',),
('Hope, inspirational quote',), ('Human Cloning',), ('Hunger',),
('Hunger, income',), ('IMF, Argentina',), ('IMF, failings',),
('Inequality',), ('INEQUALITY, GLOBALIZATION, POVERTY',), ('Inequality,
Globalization, Latin America',), ('Inspiration',), ('Junk Food',),
('Junk Food, Soft Drinks',), ('Junk Food ,Children, Education,
Media',), ('Junk Food, Education, Children, Nutrition',), ('Junk Food,
Global, Coke',), ('Junk Food, McDonald\x92s',), ('Junk Food,
McDonald\x92s, Global',), ('Junk Food, Soft Drinks, Water',), ('Junk
Food, Education, Nutrition, Childhood Illness',), ('Junk food,
Nutrition, Schools',), ('Labor',), ('Labor Issues, Agriculture,
Sustainable Agriculture Support',), ('Labor Issues, Employee
Ownership',), ('Land Reform, Brazil, Infant mortality',),
('Livestock',), ('Meat',), ('Meat, Brazil, Soybeans, Deforestation',),
('Meat, concentration',), ('Meat, Recalls, Contamination',), ('Meat,
Recalls',), ('Media',), ('Microloans',), ('Military Spending',),
('Nutrition Programs, School Meals, Snapple',), ('Nutrition,
Finland',), ('Obesity',), ('Obesity, Childhood',), ('Obesity,
Africa',), ('Organic Food',), ('Obesity, Global',), ('Obesity, related
illnesses',), ('Obesity, related illnesses, children',), ('Oil',),
('Organic Agriculture',), ('Organic Agriculture, Growth',), ('Organic
Agriculture, Nutrition',), ('Organic Agriculture, Productivity',),
('Organic food prices',), ('Community',), ('Corporate Concentration',),
('Community Gardens, Urban Gardens',), ('Pollution, Air, CO2
emissions',), ('Population, Water',), ('Poverty, Aid',), ('Poverty,
World Bank, Hunger',), ('Organic Produce',), ('Organic Produce,
Farmer\x92s Markets, CSA Community Initiatives (Dem. Book)',),
('Pakistan, Education, Poverty, Terrorism',), ('Pesticides',),
('Pollution, US',), ('Population, Women',), ('Poverty, Labor Issues',),
('t2',), ('Agriculture, Subsidies, Farming, Ranching, Government,
Livestock, 

Re: os.lisdir, gets unicode, returns unicode... USUALLY?!?!?

[Martin v. Löwis]

gabor schrieb:
> i also recommend this approach.
> 
> also, raising an exception goes well with the principle of the least
> surprise imho.

Are you saying you wouldn't have been surprised if that had been
the behavior? How would you deal with that exception in your code?

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Heap Memory

[Thinker]

Gregory Pi鎑ro wrote:
> This computer it was running on has 2GB of RAM and 6GB of virtual
> memory so I really doubt I had used up all of that memory.  I didn't
> watch it the whole time so I can't be sure though.  Any ideas what
> could have been going on there?
>
> -Greg
>   
What is your OS? Maybe you should show out the memory usage of your python
process. In FreeBSD, you should set evironment variable
'MALLOC_OPTIONS=P' to
print out usage of malloc()。Maybe you can find a way, in your system,
to print out usage of the
heap.

-- 
Thinker Li - [EMAIL PROTECTED] [EMAIL PROTECTED]
http://heaven.branda.to/~thinker/GinGin_CGI.py

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: os.lisdir, gets unicode, returns unicode... USUALLY?!?!?

[Martin v. Löwis]

Jean-Paul Calderone schrieb:
>> How would you propose listdir should behave?
> 
> Umm, just a wild guess, but how about raising an exception which includes
> the name of the file which could not be decoded?

There may be multiple of these, of course, but I assume that you want
it to report the first one it encounters?

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: cPickle problems

[Klaas]

Jeff  Poole wrote:
> Good idea.  Well, I did that, and I found out that the object causing
> problems is a ParseResults object (a class from PyParsing) and that the
> __getstate__ member is in fact an empty string ('').  I'm not sure
> where this leaves me...  The PyParsing code clearly never creates such

Sounds like ParseResults is not intended to be pickable.

-Mike

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: dict.reserve and other tricks

[Klaas]

[EMAIL PROTECTED] wrote:
> I have started doing practice creating C extensions for CPython, so
> here are two ideas I have had, possibly useless.
>
> If you keep adding elements to a CPython dict/set, it periodically
> rebuilds itself. So maybe dict.reserve(n) and a set.reserve(n) methods
> may help, reserving enough (empty) memory for about n *distinct* keys
> the programmer wants to add to the dict/set in a short future. I have
> seen that the the C API of the dicts doesn't allow this, and I don't
> know if this can be implemented modifying the dicts a bit. Do you think
> this may be useful?

It has been proposed before and rejected.  How often is dict creation a
bottleneck in python apps?  I'd guess not often.  Do you know of any
examples

Optimize space use also isn't terribly compelling, as a "tight" dict
can be created from a "loose" dict d using dict(d).

<>
> Most of the times such functions are good enough, but sometimes the
> dicts are big, so to reduce memory used I remove keys in place:
>
> def filterdict(pred, indict):
>   todel = [k for k,v in indict.iteritems() if not pred(k,v)]
>   for key in todel:
> del indict[key]

<>
> But doing the same thing while iterating on the dict may be faster and
> use even less memory.

Well, you can reduce the memory usage to virtually nothing by using a
generator expression rather than list comprehension.

> This iteration&deletion capability is probably not safe enough to be
> used inside Python programs, but a compiled C module with a function
> that works like that filterdict (and deletes while iterating) may be
> created, and its use is safe from Python programs.

<>

> >The dictionary p should not be mutated during iteration. It is safe (since 
> >Python 2.1) to modify the values of the keys as you iterate over the 
> >dictionary, but only so long as the set of keys does not change.<
>
> Do you think it may be useful to create to create such C filterdict
> function that deletes while iterating? (To create such function it
> probably has to bypass the CPython dict API).

Such a beast would be fiendish to write, I think.  Remember, arbitrary
python code can be executed by __hash__ and deleting (DECREF) python
objects.

-Mike

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Heap Memory

[Gregory Piñero]

On 11/16/06, Thinker <[EMAIL PROTECTED]> wrote:
> What is your OS? Maybe you should show out the memory usage of your python
> process. In FreeBSD, you should set evironment variable
> 'MALLOC_OPTIONS=P' to
> print out usage of malloc()。Maybe you can find a way, in your system,
> to print out usage of the
> heap.
>
It is Windows XP.  Is there a simliar method for it?  I am curious.

-Greg
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How fuzzy is get_close_matches() in difflib?

[Steven D'Aprano]

On Thu, 16 Nov 2006 20:19:50 -0800, John Henry wrote:

> I did try them and I am impressed.  It helped me found a lot of useful
> info.   I just want to get a feel as to what constitutes a "match".

The source code has lots of comments, but they don't explain the basic
algorithm (at least not in the difflib.py supplied with Python 2.3).

There is no single diff algorithm, but I believe that the basic idea is to
look for insertions and/or deletions of strings. If you want more
detail, google "diff". Once you have a list of differences, the closest
match is the search string with the fewest differences.

As for getting a feel of what constitutes a match, I really can't make any
better suggestion than just try lots of examples with the interactive
Python shell.



-- 
Steven D'Aprano 

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Press button to load data

[jim-on-linux]

Without being able to run the code my question is 
where is the  id  in the lambda defined? 


On Thursday 16 November 2006 22:31, jim wrote:
> Thanks for your help, but now I have a another
> problem so here is my code again
> when I run this it prints  id>
>
> from Tkinter import *
> import shelve
> from tkMessageBox import showerror
>
> shelvename = shelve.open('class-shelve2')
> cat = (' Name ', ' Account # ', ' Amount Due ',
> ' Date Due ')
>
> def NameFields(top):
> name1 = Label(None, text=cat[0],
> relief=RIDGE, width=20, fg='blue', bg='white',
> font=('bold',15))
> name2 = Label(None, text=cat[1],
> relief=RIDGE, width=15, fg='blue', bg='white',
> font=('bold',15))
> name3 = Label(None, text=cat[2],
> relief=RIDGE, width=15, fg='blue', bg='white',
> font=('bold',15))
> name4 = Label(None, text=cat[3],
> relief=RIDGE, width=15, fg='blue', bg='white',
> font=('bold',15))
> name1.grid(row=0, column=0, sticky=NSEW)
> name2.grid(row=0, column=1, sticky=NSEW)
> name3.grid(row=0, column=2, sticky=NSEW)
> name4.grid(row=0, column=3, sticky=NSEW)
> top.columnconfigure(0, weight=1)
> top.columnconfigure(1, weight=1)
> top.columnconfigure(2, weight=1)
> top.columnconfigure(3, weight=1)
>
>
> def DisplayBills(top):
> c=0
> x = []
> global bill
> for bill in shelvename:
> global funcs
> bill1 = Button(None, text=
> shelvename[bill].name,
> font=('bold',10),command=(lambda x = id:
> fetchRecord(x)))
>
> bill2 = Label(None, text=
> shelvename[bill].account, relief=RIDGE,
> font=('bold',10))
> bill3 = Label(None, text=
> shelvename[bill].paymentDue, relief=RIDGE,
> font=('bold',10), fg='red') bill4 = Label(None,
> text= shelvename[bill].dateDue, relief=RIDGE,
> font=('bold',10))
> bill1.grid(row=c, column=0,
> sticky=NSEW) bill2.grid(row=c,column=1,
> sticky=NSEW) bill3.grid(row=c,column=2,
> sticky=NSEW) bill4.grid(row=c,column=3,
> sticky=NSEW) c = c + 1
> return bill
>
> def fetchRecord(x):
> print x
>
>
>
> top = Tk()
>
> DisplayBills(top), NameFields(top)
>
> mainloop()
>
> jim-on-linux wrote:
> > Just from a glance my thoughts are to
> > start with one file and build on it. Make
> > a class of it so you can loop it to use
> > it over for each record.
> >
> >
> > You wrote that the info was in a file on
> > the hd. If it is in a file on the hd, use the
> > open()
> > function, read from the file, only one record
> > and write the data to a list.
> >
> > You can incorporate the
> > button option,
> >
> > "command = CallSomeFunction",
> >
> > to call a function that builds a window,
> > and loads the data into labels or
> > entry boxes.
> > If you are going to modify
> > the data, entry boxes allow you to
> > modify it and save it back to a
> > file.
> >
> > Also, when using the open() function,
> > close it after you get the data you need.
> > otherwise you may experience
> > unexpected problems.
> >
> > client = open('client', 'r')
> > client.read() (readline()) (readlines())
> > client.close()
> >
> > jim-on-linux
> >
> > http//:www.inqvista.com
-- 
http://mail.python.org/mailman/listinfo/python-list

PIL - "despeckle" image ?

[shearichard]

Hi - I have some images which I would like to remove specks from using
the PIL. I would like to be able to say that if a pixel is in a blob of
less than n contiguous pixels then all pixels in that blob should be
removed.

The images are 8 bit images with only 0 and 255 values.

I can think of quite crude ways of doing this but I'm sure there are
some smart ways of doing it - would anyone be able to point me in the
right direction ?

Lastly in case it matters the reason I'm doing this is that the images
are scanned images of sheets of papers on which sample signatures have
been placed. Typically the signatures are much smaller than the sheets
of paper so I would like to 'autocrop' the images to remove the white
space around the signature. At some point in the process the images
having picked up small islands of pixels (dust on scanner or paper ?)
which are nowhere near the signature but which are causing my autocrop
efforts to go awry.

Thanks

Richard.

-- 
http://mail.python.org/mailman/listinfo/python-list

popen(1-4) as a seperate process

[Astan Chee]

Hi,
Im trying to popen (or more specifically os.popen4() ) from wxPython. 
I've read the documentation on popen and it says I can do a popen as a 
seperate process or popen not as a child process but it doesnt say how.
Can anyone help?
Thanks
-- 
http://mail.python.org/mailman/listinfo/python-list

Weekly Python Patch/Bug Summary

[Kurt B. Kaiser]

Patch / Bug Summary
___

Patches :  416 open (-14) /  3463 closed (+16) /  3879 total ( +2)
Bugs:  930 open ( +8) /  6333 closed (+17) /  7263 total (+25)
RFE :  244 open ( -1) /   244 closed ( +3) /   488 total ( +2)

New / Reopened Patches
__

tkSimpleDialog freezes when apply raises exception  (2006-11-11)
   http://python.org/sf/1594554  opened by  Hirokazu Yamamoto

Iterating closed StringIO.StringIO  (2005-11-18)
   http://python.org/sf/1359365  reopened by  doerwalter

Cross compiling patches for MINGW  (2006-11-16)
   http://python.org/sf/1597850  opened by  Han-Wen Nienhuys

Patches Closed
__

`in` for classic object causes segfault  (2006-11-07)
   http://python.org/sf/1591996  closed by  loewis

askyesnocancel helper for tkMessageBox  (2005-11-08)
   http://python.org/sf/1351744  closed by  loewis

PyErr_CheckSignals returns -1 on error, not 1  (2006-11-07)
   http://python.org/sf/1592072  closed by  gbrandl

make pty.fork() allocate a controlling tty  (2003-11-08)
   http://python.org/sf/838546  closed by  loewis

Add missing elide argument to Text.search  (2006-11-07)
   http://python.org/sf/1592250  closed by  loewis

mailbox: use fsync() to ensure data is really on disk  (2006-06-29)
   http://python.org/sf/1514544  closed by  akuchling

mailbox (Maildir): avoid losing messages on name clash  (2006-06-29)
   http://python.org/sf/1514543  closed by  akuchling

Fix struct.pack on 64-bit archs (broken on 2.*)  (2004-10-02)
   http://python.org/sf/1038854  closed by  loewis

Cross building python for mingw32  (2003-11-13)
   http://python.org/sf/841454  closed by  loewis

httplib: allowing stream-type body part in requests  (2004-11-12)
   http://python.org/sf/1065257  closed by  loewis

support whence argument for GzipFile.seek (bug #1316069)  (2005-11-12)
   http://python.org/sf/1355023  closed by  loewis

fix for 1067728: Better handling of float arguments to seek  (2004-11-17)
   http://python.org/sf/1067760  closed by  loewis

ftplib transfer problem with certain servers  (2005-11-17)
   http://python.org/sf/1359217  closed by  loewis

bdist_rpm still can't handle dashes in versions  (2005-11-18)
   http://python.org/sf/1360200  closed by  loewis

Fix the vc8 solution files  (2006-08-19)
   http://python.org/sf/1542946  closed by  krisvale

Practical ctypes example  (2006-09-15)
   http://python.org/sf/1559219  closed by  theller

New / Reopened Bugs
___

Unfortunate naming of variable in heapq example  (2006-11-08)
CLOSED http://python.org/sf/1592533  opened by  Martin Thorsen Ranang

gettext has problems with .mo files that use non-ASCII chars  (2006-11-08)
CLOSED http://python.org/sf/1592627  opened by  Russell Phillips

replace groups doesn't work in this special case  (2006-11-06)
   http://python.org/sf/1591319  reopened by  tomek74

readline problem on ia64-unknown-linux-gnu  (2006-11-08)
   http://python.org/sf/1593035  opened by  Kate Minola

No IDLE in Windows  (2006-11-09)
CLOSED http://python.org/sf/1593384  opened by  A_V_I

No IDLE in Windows  (2006-11-09)
CLOSED http://python.org/sf/1593407  opened by  A_V_I

No IDLE in Windows  (2006-11-09)
CLOSED http://python.org/sf/1593442  opened by  A_V_I

site-packages isn't created before install_egg_info  (2006-09-28)
CLOSED http://python.org/sf/1566719  reopened by  loewis

Modules/unicodedata.c contains C++-style comment  (2006-11-09)
CLOSED http://python.org/sf/1593525  opened by  Mike Kent

No IDLE in Windows  (2006-11-09)
CLOSED http://python.org/sf/1593634  opened by  A_V_I

poor urllib error handling  (2006-11-09)
   http://python.org/sf/1593751  opened by  Guido van Rossum

small problem with description  (2006-11-09)
CLOSED http://python.org/sf/1593829  opened by  Atlas

Word should be changed on page 3.6.1  (2006-11-11)
CLOSED http://python.org/sf/1594742  opened by  jikanter

Make docu for dict.update more clear  (2006-11-11)
CLOSED http://python.org/sf/1594758  opened by  Christoph Zwerschke

make install fails, various modules do not work  (2006-11-11)
CLOSED http://python.org/sf/1594809  opened by  Evan

doctest simple usage recipe is misleading  (2006-11-12)
   http://python.org/sf/1594966  opened by  Ken Rimey

smtplib.SMTP.sendmail() does not provide transparency  (2006-11-12)
CLOSED http://python.org/sf/1595045  opened by  Avi Kivity

texinfo library documentation fails to build  (2006-11-12)
   http://python.org/sf/1595164  opened by  Mark Diekhans

User-agent header added by an opener is "frozen"  (2006-11-13)
   http://python.org/sf/1595365  opened by  Björn Steinbrink

parser module bug for nested try...except statements  (2006-11-13)
CLOSED http://python.org/sf/1595594  opened by  Kay Schluehr

SocketServer allow_reuse_address checked in constructor  (2006-11-13)
   http://python.org/sf/1595742  opened by  Peter Parente

read() in windows

Re: How fuzzy is get_close_matches() in difflib?

[John Henry]

I did try them and I am impressed.  It helped me found a lot of useful
info.   I just want to get a feel as to what constitutes a "match".


Steven D'Aprano wrote:
> On Thu, 16 Nov 2006 16:40:49 -0800, John Henry wrote:
>
> > I am just wondering what's with get_close_matches() in difflib.  What's
> > the magic?   How fuzzy do I need to get in order to get a match?
>
>
> Why don't you try it and see?
>
> >>> from difflib import get_close_matches
> >>> get_close_matches("appel", ["ape", "apple", "peach", "puppy"])
> ['apple', 'ape']
> >>> import keyword as _keyword
> >>> get_close_matches("wheel", _keyword.kwlist)
> ['while']
> >>> get_close_matches("apple", _keyword.kwlist)
> []
> >>> get_close_matches("accept", _keyword.kwlist)
> ['except']
>
>
> Those example, by the way, come from here:
> 
> >>> help(get_close_matches)
> 
> 
> 
> 
> -- 
> Steven

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Printing/writing an integer to a file

[Ben Finney]

"Steven D'Aprano" <[EMAIL PROTECTED]> writes:

> On Fri, 17 Nov 2006 14:18:12 +1100, Ben Finney wrote:
>
> > The function call syntax doesn't allow a space between the
> > function name and the opening parenthesis.
>
> Are you sure?

Are you kidding? Do you think I'd post to this newsgroup if I wasn't
completely sure about what I wrote?

> I thought it was allowed but not recommended because it is
> hard to read.
>
> >>> len ([1, 2, 3])
> 3
>
>
> Seems to work for me.

I hope that answers your question then :-)

-- 
 \"If you continue running Windows, your system may become |
  `\ unstable."  -- Microsoft, Windows 95 BSOD message |
_o__)  |
Ben Finney

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Printing/writing an integer to a file

[Steven D'Aprano]

On Fri, 17 Nov 2006 14:18:12 +1100, Ben Finney wrote:

> The function call syntax doesn't allow a space between the function
> name and the opening parenthesis.

Are you sure? I thought it was allowed but not recommended because it is
hard to read.

>>> len ([1, 2, 3])
3


Seems to work for me.

And if you think that's bad, check out this:

>>> '+'.join   (   [  'a'  ,  'b'  ]  )
'a+b'

How to write legal but unreadable code in Python.

-- 
Steven D'Aprano 

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Printing/writing an integer to a file

[Ben Finney]

PS <[EMAIL PROTECTED]> writes:

> Friends,

Please, don't send message bodies to public discussion forums in any
format other than plain text. HTML email in particular has wildly
differing implementations and renderings.

> I am new to python and did search on the web on how to achieve this:
> ( I am trying to append the line numbers to all the lines of a file for now)
> Thanks!!
> =
> import os, sys
> fileName = os.path.join("C:", "temp", "x1.txt")
> fileobject = open(fileName, 'r')
> outputDir = "C://temp//"
> linenumber = 0
> fileName1 = outputDir + " x2.txt"

You've used 'os.path.join' corretly above; you should use that any
time you want to compose a file path.

output_dir = "C://temp//"
out_file_name = os.path.join(output_dir, "x2.txt")

> fileobject1 = open(fileName1, 'w')
> while (1):
>     L = fileobject.readline()
>     if L=="":
>     print "**Done"
>     break

A file object is iterable, returning a line of text each time. This
means you don't need to manually loop and extract each line; you can
write this loop as:

out_file = open(out_file_name, 'w')
for in_line in in_file:
# process the line

>     linenumber += 1
>     fileobject1.write (ln)

The function call syntax doesn't allow a space between the function
name and the opening parenthesis.

You are referencing 'ln' without ever defining it. Presumably you mean
to use 'linenumber'.

  linenumber += 1
  out_file.write(linenumber)

>     fileobject1.write(":: "+ L)
>    
> fileobject1.close()
> =

Wrapping all this advice up::

import os
import sys

in_file_name = os.path.join("C:", "temp", "x1.txt")
in_file = open(in_file_name, 'r')
out_dir = "C://temp//"
out_file_name = os.path.join(out_dir, "x2.txt")
out_file = open(out_file_name, 'w')

linenumber = 0
for in_line in in_file:
linenumber += 1
out_file.write(linenumber)
out_file.write(":: " + in_line)

out_file.close()
in_file.close()
print "**Done"

An improvement would be to use string formatting to construct the
output line:

linenumber = 0
for in_line in in_file:
linenumber += 1
out_line = "%d:: %s" % (linenumber, in_line)
out_file.write(out_line)

This also means you're one easy step away from padding the line number
to a minimum width::

out_line = "%06d:: %s" % (linenumber, in_line)

-- 
 \ "I must say that I find television very educational. The minute |
  `\   somebody turns it on, I go to the library and read a book."  -- |
_o__) Groucho Marx |
Ben Finney

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: dict.reserve and other tricks

[Terry Reedy]

<[EMAIL PROTECTED]> wrote in message 
news:[EMAIL PROTECTED]
> If you keep adding elements to a CPython dict/set, it periodically
> rebuilds itself. So maybe dict.reserve(n) and a set.reserve(n) methods
> may help, reserving enough (empty) memory for about n *distinct* keys
> the programmer wants to add to the dict/set in a short future. I have
> seen that the the C API of the dicts doesn't allow this, and I don't
> know if this can be implemented modifying the dicts a bit. Do you think
> this may be useful?

Ideas for user-controlled 'performance tuning' have been proposed on the 
development list.  GVR has rejected such as more trouble than they are 
worth.  Instead, effort has gone into making the implementation as good as 
he and others know how.

Since you are writing extensions, you can create a built-in subclass of 
dict to experiment with.  I presume the 2.5 default dict should be a model.

Terry Jan Reedy



-- 
http://mail.python.org/mailman/listinfo/python-list

Re: cPickle problems

[Jeff Poole]

Good idea.  Well, I did that, and I found out that the object causing
problems is a ParseResults object (a class from PyParsing) and that the
__getstate__ member is in fact an empty string ('').  I'm not sure
where this leaves me...  The PyParsing code clearly never creates such
a member and my code never creates it.  In fact, searching all the code
involved (not including what is in /usr/lib/python2.4 ), shows no use
of __getstate__ at all (or even the string "getstate").

Ok, I figured it out.  ParseResults has the following member:

def __getattr__( self, name ):
if name not in self.__slots__:
if self.__tokdict.has_key( name ):
if name not in self.__accumNames:
return self.__tokdict[name][-1][0]
else:
return ParseResults([ v[0] for v in
self.__tokdict[name] ])
else:
return ""
return None

So when something tries to retrieve .__getstate__, it returns an empty
string.  If I tell it to raise an AttributeException instead, then I
get this message:

TypeError: a class that defines __slots__ without defining __getstate__
cannot be pickled

I think I need to talk to the author of PyParsing about this one...
Thanks for your help!


Jeff


> Someway, self.__getstate__ is a string, not a method...
> Since it fails inside a python module, you could print some debug
> information to see what happens, like repr(self), type(self), 
> repr(getstate)...
>
>
>
> --
> Gabriel Genellina
> Softlab SRL
>
> __
> Correo Yahoo!
> Espacio para todos tus mensajes, antivirus y antispam ¡gratis!
> ¡Abrí tu cuenta ya! - http://correo.yahoo.com.ar

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Printing/writing an integer to a file

[Gabriel Genellina]

At Thursday 16/11/2006 22:33, PS wrote:



Friends,

I am new to python and did search on the web on how to achieve this:
( I am trying to append the line numbers to all the lines of a file for now)


But you forget to say what went wrong... Next time, post the error 
message including the full traceback.

Using my crystal ball I can see a few problems here:


import os, sys

fileName = os.path.join("C:", "temp", "x1.txt")
fileobject = open(fileName, 'r')
outputDir = "C://temp//"
linenumber = 0
fileName1 = outputDir + " x2.txt"


I see that you already know how to build properly a file name using 
os.path.join - well, some lazyness is ok...



fileobject1 = open(fileName1, 'w')
while (1):
L = fileobject.readline()
if L=="":
print "**Done"
break


Usually that's written as:

for L in fileobject:
... do something ...
else:
print "** Done"

for is used to iterate over some sequence (or iterator), and a file 
acts as its own iterator, yielding one line at a time.

The else clause -optional- is executed when nothing remains to iterate.


linenumber += 1
fileobject1.write (ln)
fileobject1.write(":: "+ L)


I think here is your problem. You want the line number (printed as 
text), followed by two :, a space, and the original line contents. I'd use:


fileobject1.write("%d:: %s" % (linenumber, L))


fileobject1.close()
=


Enough for a good start. Happy pythoning!


--
Gabriel Genellina
Softlab SRL 


__
Correo Yahoo!
Espacio para todos tus mensajes, antivirus y antispam ¡gratis! 
¡Abrí tu cuenta ya! - http://correo.yahoo.com.ar
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How fuzzy is get_close_matches() in difflib?

[Steven D'Aprano]

On Thu, 16 Nov 2006 16:40:49 -0800, John Henry wrote:

> I am just wondering what's with get_close_matches() in difflib.  What's
> the magic?   How fuzzy do I need to get in order to get a match?


Why don't you try it and see?

>>> from difflib import get_close_matches
>>> get_close_matches("appel", ["ape", "apple", "peach", "puppy"])
['apple', 'ape']
>>> import keyword as _keyword
>>> get_close_matches("wheel", _keyword.kwlist)
['while']
>>> get_close_matches("apple", _keyword.kwlist)
[]
>>> get_close_matches("accept", _keyword.kwlist)
['except']


Those example, by the way, come from here:

>>> help(get_close_matches)




-- 
Steven

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Writing to Registry

[Gabriel Genellina]

At Thursday 16/11/2006 21:39, Samantha wrote:


> http://docs.python.org/lib/module--winreg.html#l2h-5827
Thanks for the link.


Notice that surely you already have the full Python documentation, it 
comes with the standard distribution.



--
Gabriel Genellina
Softlab SRL 


__
Correo Yahoo!
Espacio para todos tus mensajes, antivirus y antispam ¡gratis! 
¡Abrí tu cuenta ya! - http://correo.yahoo.com.ar
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: cPickle problems

[Gabriel Genellina]

At Thursday 16/11/2006 21:48, Jeff  Poole wrote:


  File "/usr/lib/python2.4/pickle.py", line 313, in save
rv = reduce(self.proto)
  File "/usr/lib/python2.4/copy_reg.py", line 83, in _reduce_ex
dict = getstate()
TypeError: 'str' object is not callable


Someway, self.__getstate__ is a string, not a method...
Since it fails inside a python module, you could print some debug 
information to see what happens, like repr(self), type(self), repr(getstate)...




--
Gabriel Genellina
Softlab SRL 


__
Correo Yahoo!
Espacio para todos tus mensajes, antivirus y antispam ¡gratis! 
¡Abrí tu cuenta ya! - http://correo.yahoo.com.ar
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python v PHP: fair comparison?

[Luis M. González]

[EMAIL PROTECTED] ha escrito:

> Luis M. González wrote:
> > Cameron Laird ha escrito:
> > > Perhaps it's timely to clarify the "newer" above:  Guido
> > > made Python public in '89-90, and Rasmus showed PHP to
> > > others in '94-95.
> >
> > OK. But since when has python been considered a viable alternative for
> > web development?
> > As a generalp purpose language, it's older.
> > But as a web development language, it's olnly when people started to
> > look for the "rails killer" and many python alternatives started to
> > come up (although Django has been in development for a long time before
> > all this hype).
>
> Huh?  I've been doing paid python web work since 2000.  I'm fairly sure
> that Yahoo! groups predates that by a while, and I know that
> mod_python/httpdapy goes back at least to 1998 (and Python CGI predates
> that significantly).

Do not forget the subject of this thread.
Nobody is criticizing python here. I am a "believer"!
We are talking about important details that matter when choosing a
language, and specially when choosing one for web development.
I'm sure Yahoo, Google and other companies can afford having
knowledgeable people, hardware and resources to use python they way
they like.
The problem is (or was) for mere mortals trying to create a web site on
shared hostings.

We all know that mod_python has its issues, and still is not a common
or affordable feature in the vast mayority of web hosts. And prior to
that, you only had cgi or other rather unknown solutions.
mod_php had a clear advantage here. Perhaps not as a language but as an
ubiquitous and cheap alternative.

In the last months (and I said "months" not years) the situation
improved a lot, but still it is a fair to make a comparison.
My opinion is that python, as a language, is by far a much better
alternative for any use.
But php, as a platform, still has a many advantages, being availability
the main one.

Luis

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Writing to Registry

[Mark Elston]

Note:  this is untested (since I don't like screwing around
in the registry...)

Have you tried using REG_DWORD?

Mark
* Samantha wrote (on 11/16/2006 3:48 PM):
> I am working with this recipes:
> http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/66011
> 
> The problem I am having is setting a hex value.
> This line ---   SetValueEx(aKey,"MyNewKey",0, REG_SZ, 
> r"c:\winnt\explorer.exe")
> I want something Like --   SetValueEx(aKey,"MyNewSize",0, REG_SZ, 120 or 
> some value)
> Help would be appreciated.
> S
> 
> 
-- 
http://mail.python.org/mailman/listinfo/python-list

Printing/writing an integer to a file

[PS]

Friends,

I am new to python and did search on the web on how to achieve this:
( I am trying to append the line numbers to all the lines of a file for now)

Thanks!!

=
import os, sys

fileName = os.path.join("C:", "temp", "x1.txt")
fileobject = open(fileName, 'r')
outputDir = "C://temp//"
linenumber = 0
fileName1 = outputDir + "x2.txt"
fileobject1 = open(fileName1, 'w')
while (1):
   L = fileobject.readline()
   if L=="":
   print "**Done"
   break
   linenumber += 1
   fileobject1.write(ln)
   fileobject1.write(":: "+ L)

fileobject1.close()
=

--
Regards,
Prem
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: cPickle problems

[Jeff Poole]

To clutter this up with yet another message, this is what happens if I
use pickle instead of cPickle.  Same error, but it has more of a stack
trace so someone more advanced than myself might be able to pick out
what is going amiss.

Traceback (most recent call last):
  File "./generateTools.py", line 50, in ?
proj.findVHDLfiles("vhd")
  File "/cygdrive/c/fpga/tools/FPGAProject.py", line 631, in
findVHDLfiles
cpickle.dump(filenameToClassDic,pickleFile,PICKLE_FILE_PROTOCOL)
  File "/usr/lib/python2.4/pickle.py", line 1382, in dump
Pickler(file, protocol, bin).dump(obj)
  File "/usr/lib/python2.4/pickle.py", line 231, in dump
self.save(obj)
  File "/usr/lib/python2.4/pickle.py", line 293, in save
f(self, obj) # Call unbound method with explicit self
  File "/usr/lib/python2.4/pickle.py", line 663, in save_dict
self._batch_setitems(obj.iteritems())
  File "/usr/lib/python2.4/pickle.py", line 677, in _batch_setitems
save(v)
  File "/usr/lib/python2.4/pickle.py", line 338, in save
self.save_reduce(obj=obj, *rv)
  File "/usr/lib/python2.4/pickle.py", line 433, in save_reduce
save(state)
  File "/usr/lib/python2.4/pickle.py", line 293, in save
f(self, obj) # Call unbound method with explicit self
  File "/usr/lib/python2.4/pickle.py", line 663, in save_dict
self._batch_setitems(obj.iteritems())
  File "/usr/lib/python2.4/pickle.py", line 677, in _batch_setitems
save(v)
  File "/usr/lib/python2.4/pickle.py", line 293, in save
f(self, obj) # Call unbound method with explicit self
  File "/usr/lib/python2.4/pickle.py", line 614, in save_list
self._batch_appends(iter(obj))
  File "/usr/lib/python2.4/pickle.py", line 629, in _batch_appends
save(x)
  File "/usr/lib/python2.4/pickle.py", line 293, in save
f(self, obj) # Call unbound method with explicit self
  File "/usr/lib/python2.4/pickle.py", line 576, in save_tuple
save(element)
  File "/usr/lib/python2.4/pickle.py", line 338, in save
self.save_reduce(obj=obj, *rv)
  File "/usr/lib/python2.4/pickle.py", line 433, in save_reduce
save(state)
  File "/usr/lib/python2.4/pickle.py", line 293, in save
f(self, obj) # Call unbound method with explicit self
  File "/usr/lib/python2.4/pickle.py", line 663, in save_dict
self._batch_setitems(obj.iteritems())
  File "/usr/lib/python2.4/pickle.py", line 677, in _batch_setitems
save(v)
  File "/usr/lib/python2.4/pickle.py", line 313, in save
rv = reduce(self.proto)
  File "/usr/lib/python2.4/copy_reg.py", line 83, in _reduce_ex
dict = getstate()
TypeError: 'str' object is not callable


Jeff  Poole wrote:
> Oh, and I suppose I should provide some version information:
>
> $ python
> Python 2.4.3 (#1, May 18 2006, 07:40:45)
> [GCC 3.3.3 (cygwin special)] on cygwin
>
>
> Jeff  Poole wrote:
> > This is going to be a pretty vague message because it involves a large
> > block of code I'd rather avoid posting.  Basically, I've been pickling
> > a dictionary of instances of a class I've created (which contains
> > references to other instances of other classes).  At some point in the
> > last few weeks, pickling has stopped working with the following error:
> >
> > Traceback (most recent call last):
> >   File "./generateTools.py", line 50, in ?
> > proj.findVHDLfiles("vhd")
> >   File "/cygdrive/c/fpga/tools/FPGAProject.py", line 630, in
> > findVHDLfiles
> > cpickle.dump(filenameToClassDic,pickleFile,PICKLE_FILE_PROTOCOL)
> >   File "/usr/lib/python2.4/copy_reg.py", line 83, in _reduce_ex
> > dict = getstate()
> > TypeError: 'str' object is not callable
> >
> >
> > Does anyone have any idea why this might be failing in this way?  It's
> > driving me crazy.  If need be I can probably provide the code to the
> > class, though it's about 1000 lines, most of which is PyParsing code.
> > I appreciate any help you guys can provide.
> > 
> > 
> > Jeff

-- 
http://mail.python.org/mailman/listinfo/python-list

dict.reserve and other tricks

[bearophileHUGS]

I have started doing practice creating C extensions for CPython, so
here are two ideas I have had, possibly useless.

If you keep adding elements to a CPython dict/set, it periodically
rebuilds itself. So maybe dict.reserve(n) and a set.reserve(n) methods
may help, reserving enough (empty) memory for about n *distinct* keys
the programmer wants to add to the dict/set in a short future. I have
seen that the the C API of the dicts doesn't allow this, and I don't
know if this can be implemented modifying the dicts a bit. Do you think
this may be useful?

---

Sometime I need to remove some elements from dicts/sets. If I know a
rule that tells what elements have to be removed I can use code like
this:

import itertools

def filterset(predicate, inset):
  return set(itertools.ifilter(predicate, inset))

print filterset(lambda x: x & 1, set(range(10)))

Output:
set([1, 3, 9, 5, 7])

And a similar one to filter dicts that works with a predicate(key,
val).
Most of the times such functions are good enough, but sometimes the
dicts are big, so to reduce memory used I remove keys in place:

def filterdict(pred, indict):
  todel = [k for k,v in indict.iteritems() if not pred(k,v)]
  for key in todel:
del indict[key]

d = dict.fromkeys(xrange(8), 0)
print d
filterdict(lambda k,v: k & 1, d)
print d

# Output:
# {0: 0, 1: 0, 2: 0, 3: 0, 4: 0, 5: 0, 6: 0, 7: 0}
# {1: 0, 3: 0, 5: 0, 7: 0}


But doing the same thing while iterating on the dict may be faster and
use even less memory.
This iteration&deletion capability is probably not safe enough to be
used inside Python programs, but a compiled C module with a function
that works like that filterdict (and deletes while iterating) may be
created, and its use is safe from Python programs.

The C++ STL API of maps allows to delete items while iterating on the
map (but probably it has to be done only when necessary, because it may
be a source for bugs):

typedef map isMap;
typedef isMap::value_type isValType;
typedef isMap::iterator isMapItor;

void main() {
  isMap m;

  ... // items added to m

  for(isMapItor itr = m.begin(); itr != m.end();) {
if(itr->second == "somestring")
  m.erase(itr++);
else
  ++itr;
  }


The Python/C API, 7.4.1 Dictionary Objects, says that's not possible
with CPython dicts:

>The dictionary p should not be mutated during iteration. It is safe (since 
>Python 2.1) to modify the values of the keys as you iterate over the 
>dictionary, but only so long as the set of keys does not change.<

Do you think it may be useful to create to create such C filterdict
function that deletes while iterating? (To create such function it
probably has to bypass the CPython dict API).

Bye,
bearophile

-- 
http://mail.python.org/mailman/listinfo/python-list

How fuzzy is get_close_matches() in difflib?

[John Henry]

I am just wondering what's with get_close_matches() in difflib.  What's
the magic?   How fuzzy do I need to get in order to get a match?

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Writing to Registry

[Samantha]

Thanks for the link.
S
"Gabriel Genellina" <[EMAIL PROTECTED]> wrote in message 
news:[EMAIL PROTECTED]
> At Thursday 16/11/2006 20:48, Samantha wrote:
>
>>I am working with this recipes:
>>http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/66011
>>
>>The problem I am having is setting a hex value.
>>This line ---   SetValueEx(aKey,"MyNewKey",0, REG_SZ,
>>r"c:\winnt\explorer.exe")
>>I want something Like --   SetValueEx(aKey,"MyNewSize",0, REG_SZ, 120 or
>>some value)
>
> Usually when you don't know how to use a certain function, you begin by 
> reading the documentation for it:
> http://docs.python.org/lib/module--winreg.html#l2h-5827
>
>
> -- 
> Gabriel Genellina
> Softlab SRL
> __
> Correo Yahoo!
> Espacio para todos tus mensajes, antivirus y antispam ¡gratis! ¡Abrí tu 
> cuenta ya! - http://correo.yahoo.com.ar 


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Secure Python

[Paul Boddie]

timmy wrote:
> Paul Boddie wrote:
> > Diez B. Roggisch wrote:

[Re-adding material...]

> >>At least to me - and I presume pretty much everybody except you in this
> >>thread - this means that he is interested in executing arbitrary pieces of
> >>python code inside the interpreter, which comes from e.g. players who
> >>customize their in-game behavior of their avatars.

Here is where the issue of sandboxing is indirectly introduced into the
discussion.

> >>Now how exactly does linux (or any other resource limiting technique on any
> >>OS) help here - killing the whole game server surely isn't a desirable
> >>solution when one player goes berserk, might it be intentionally or not.
>
> > And this is where the hot topics collide: people want performant
> > multitasking with lots of shared state (the global interpreter lock
> > controversy) together with sandboxing so that the individual threads
> > can't access most of that shared state (the restricted execution
> > controversy).
>
> i'm not talking about sandboxing, that's a whole different kettle of
> fish. i'm talking about resource managment options you can set in for
> instance, the linux kernel.

Yes, I know. I was merely covering related concepts of relevance
introduced earlier in the discussion (see above). In any case, if you
only have control over resource limits on execution contexts at the
operating system process level, yet your server architecture is
entirely based on a single process with many (micro)threads, then
you've got to consider the problem of restricting their
behaviour/consumption using the facilities available to you, most
likely by considering other server architectures. This, I argue, is
perfectly reasonable in order to solve both issues being discussed
using operating system facilities.

> you can limit the cpu and memory a process uses while still allowing it
> the same access it would have outside of a sandbox. that way if any
> clever monkeys try to dos you they merely consume their alloted quota.

Indeed.

Paul

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: cPickle problems

[Jeff Poole]

Oh, and I suppose I should provide some version information:

$ python
Python 2.4.3 (#1, May 18 2006, 07:40:45)
[GCC 3.3.3 (cygwin special)] on cygwin


Jeff  Poole wrote:
> This is going to be a pretty vague message because it involves a large
> block of code I'd rather avoid posting.  Basically, I've been pickling
> a dictionary of instances of a class I've created (which contains
> references to other instances of other classes).  At some point in the
> last few weeks, pickling has stopped working with the following error:
>
> Traceback (most recent call last):
>   File "./generateTools.py", line 50, in ?
> proj.findVHDLfiles("vhd")
>   File "/cygdrive/c/fpga/tools/FPGAProject.py", line 630, in
> findVHDLfiles
> cpickle.dump(filenameToClassDic,pickleFile,PICKLE_FILE_PROTOCOL)
>   File "/usr/lib/python2.4/copy_reg.py", line 83, in _reduce_ex
> dict = getstate()
> TypeError: 'str' object is not callable
>
>
> Does anyone have any idea why this might be failing in this way?  It's
> driving me crazy.  If need be I can probably provide the code to the
> class, though it's about 1000 lines, most of which is PyParsing code.
> I appreciate any help you guys can provide.
> 
> 
> Jeff

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: os.lisdir, gets unicode, returns unicode... USUALLY?!?!?

[gabor]

Jean-Paul Calderone wrote:
> On Fri, 17 Nov 2006 00:31:06 +0100, "\"Martin v. Löwis\"" 
> <[EMAIL PROTECTED]> wrote:
>> gabor schrieb:
 All this code will typically work just fine with the current behavior,
 so people typically don't see any problem.

>>>
>>> i am sorry, but it will not work. actually this is exactly what i did,
>>> and it did not work. it dies in the os.path.join call, where file_name
>>> is converted into unicode. and python uses 'ascii' as the charset in
>>> such cases. but, because listdir already failed to decode the file_name
>>> with the filesystem-encoding, it usually also fails when tried with
>>> 'ascii'.
>>
>> Ah, right. So yes, it will typically fail immediately - just as you
>> wanted it to do, anyway; the advantage with this failure is that you
>> can also find out what specific file name is causing the problem
>> (whereas when listdir failed completely, you could not easily find
>> out the cause of the failure).
>>
>> How would you propose listdir should behave?
> 
> Umm, just a wild guess, but how about raising an exception which includes
> the name of the file which could not be decoded?
> 

i also recommend this approach.

also, raising an exception goes well with the principle of the least 
surprise imho.

gabor
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Writing to Registry

[Gabriel Genellina]

At Thursday 16/11/2006 20:48, Samantha wrote:


I am working with this recipes:
http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/66011

The problem I am having is setting a hex value.
This line ---   SetValueEx(aKey,"MyNewKey",0, REG_SZ,
r"c:\winnt\explorer.exe")
I want something Like --   SetValueEx(aKey,"MyNewSize",0, REG_SZ, 120 or
some value)


Usually when you don't know how to use a certain function, you begin 
by reading the documentation for it:

http://docs.python.org/lib/module--winreg.html#l2h-5827


--
Gabriel Genellina
Softlab SRL 


__
Correo Yahoo!
Espacio para todos tus mensajes, antivirus y antispam ¡gratis! 
¡Abrí tu cuenta ya! - http://correo.yahoo.com.ar
-- 
http://mail.python.org/mailman/listinfo/python-list

cPickle problems

[Jeff Poole]

This is going to be a pretty vague message because it involves a large
block of code I'd rather avoid posting.  Basically, I've been pickling
a dictionary of instances of a class I've created (which contains
references to other instances of other classes).  At some point in the
last few weeks, pickling has stopped working with the following error:

Traceback (most recent call last):
  File "./generateTools.py", line 50, in ?
proj.findVHDLfiles("vhd")
  File "/cygdrive/c/fpga/tools/FPGAProject.py", line 630, in
findVHDLfiles
cpickle.dump(filenameToClassDic,pickleFile,PICKLE_FILE_PROTOCOL)
  File "/usr/lib/python2.4/copy_reg.py", line 83, in _reduce_ex
dict = getstate()
TypeError: 'str' object is not callable


Does anyone have any idea why this might be failing in this way?  It's
driving me crazy.  If need be I can probably provide the code to the
class, though it's about 1000 lines, most of which is PyParsing code.
I appreciate any help you guys can provide.


Jeff

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Secure Python

[timmy]

Paul Boddie wrote:
> Diez B. Roggisch wrote:
> 
> 
> [Multiplayer game servers]
> 
> 
>>Now how exactly does linux (or any other resource limiting technique on any
>>OS) help here - killing the whole game server surely isn't a desirable
>>solution when one player goes berserk, might it be intentionally or not.
> 

> And this is where the hot topics collide: people want performant
> multitasking with lots of shared state (the global interpreter lock
> controversy) together with sandboxing so that the individual threads
> can't access most of that shared state (the restricted execution
> controversy). 

i'm not talking about sandboxing, that's a whole different kettle of 
fish. i'm talking about resource managment options you can set in for 
instance, the linux kernel.
you can limit the cpu and memory a process uses while still allowing it 
the same access it would have outside of a sandbox. that way if any 
clever monkeys try to dos you they merely consume their alloted quota.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Secure Python

[timmy]

Diez B. Roggisch wrote:
>>as posted before, linux kernel limit.
>>
>>then you and your users can go as crazy as you want and you won't take
>>out your system.
>>
>>maybe you should think a little more before going on the attack like that.
> 
> 
> You should maybe read a little bit more when making bold statements about
> the feasibility of a sandboxed _PYTHON_. The OP wrote:
> 

there is nothing preventing you putting limits on the resources each 
process uses, on just about any modern day OS

> At least to me - and I presume pretty much everybody except you in this
> thread - 

Oh no i understand perfectly what he wants, i merely suggest a simple OS 
based solution.

this means that he is interested in executing arbitrary pieces of
> python code inside the interpreter, which comes from e.g. players who
> customize their in-game behavior of their avatars. 
> 
> Now how exactly does linux (or any other resource limiting technique on any
> OS) help here - killing the whole game server surely isn't a desirable
> solution when one player goes berserk, might it be intentionally or not.

resource managment does not kill anything it merely prevents one process 
running away and consuming the whole server. this is EXACTLY what he is 
afraid of.
if he intends on running arbitrary code then i suggest he spawns each 
one as a seperate thread with a spefic name and merely set limits on all 
  processes named X. that way he can run any whacky code he wants safely 
inside those processes without fear of any one of them crashing the 
server. I know it can be done under any of the nix's, I'm not sure how 
to do so under windows, but it could probably be done.

> 

> It is a recurring and pretty much understandable request on c.l.py to be
> able to do so - sometimes it arises in the disguise of killable threads.
> But unfortunately the solution doesn't seem to be as simple as one would
> wish.

i can understand people wanting an application based cross platform 
solution to this, but i'm yet to see anything practicle hence i suggest 
and OS based solution.

> 
> Diez
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: accessing fortran modules from python

[sam]

thanks guys,

i'll follow this up more in a couple of weeks when i know what i need
to do better.

sam

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: os.lisdir, gets unicode, returns unicode... USUALLY?!?!?

[Jean-Paul Calderone]

On Fri, 17 Nov 2006 00:31:06 +0100, "\"Martin v. Löwis\"" <[EMAIL PROTECTED]> 
wrote:
>gabor schrieb:
>>> All this code will typically work just fine with the current behavior,
>>> so people typically don't see any problem.
>>>
>>
>> i am sorry, but it will not work. actually this is exactly what i did,
>> and it did not work. it dies in the os.path.join call, where file_name
>> is converted into unicode. and python uses 'ascii' as the charset in
>> such cases. but, because listdir already failed to decode the file_name
>> with the filesystem-encoding, it usually also fails when tried with
>> 'ascii'.
>
>Ah, right. So yes, it will typically fail immediately - just as you
>wanted it to do, anyway; the advantage with this failure is that you
>can also find out what specific file name is causing the problem
>(whereas when listdir failed completely, you could not easily find
> out the cause of the failure).
>
>How would you propose listdir should behave?

Umm, just a wild guess, but how about raising an exception which includes
the name of the file which could not be decoded?

Jean-Paul
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pyj file extension

[Paul Boddie]

Diez B. Roggisch wrote:
>
> http://www.crazy-compilers.com/decompyle/
>
> But only up to python 2.3. No idea if that is sufficient & if there is
> anything newer out there.

Usual service message: sources available from here...

http://packages.debian.org/unstable/source/decompyle

Paul

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to make a python file to a DLL?

[Larry Bates]

many_years_after wrote:
> Any solution?
> 
> Thanks.
> 
You can certainly make python program/function into a COM object
which can be called from other languages quite easily and I think
is the more preferred method in newer software.  py2exe has
examples of creating COM objects.

-Larry
-- 
http://mail.python.org/mailman/listinfo/python-list

Writing to Registry

[Samantha]

I am working with this recipes:
http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/66011

The problem I am having is setting a hex value.
This line ---   SetValueEx(aKey,"MyNewKey",0, REG_SZ, 
r"c:\winnt\explorer.exe")
I want something Like --   SetValueEx(aKey,"MyNewSize",0, REG_SZ, 120 or 
some value)
Help would be appreciated.
S


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: About alternatives to Matlab

[Robert Kern]

Matimus wrote:
> There is also Scilab. I've only used it a tiny bit but for the task it
> worked well. I do know that it has a somewhat restrictive license. It
> is open source, but you aren't allowed to modify and redistribute the
> source. I get the feeling that some people avoid Scilab but I'm not
> sure why. If anybody knows the reason I'd be happy to hear it.

I think you just stated it.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: deciding what is a dir (folder) in a zip file

[Gabriel Genellina]

At Thursday 16/11/2006 10:48, Jim wrote:


I'm trying to read .zip files and drop from the listing those files
that are directories.  I'm using the zipfile module.

Googling has increased my suspicion that
it is not just a matter of seeing if the file name ends in '/' and that
the relevant external file attribute seems to have different values for
people from different platforms, so just experimenting on my platform
doesn't seem to be a safe solution.


Ending in / appears to be enough. Filenames are stored using / in 
Windows or Linux, using winzip, gzip and winrar at least.
Notice that no ordering is implied: directories may come _after_ 
filenames inside it.

This fragment extracts all ZipFile contents, maintaining directory structure:

--- cut ---
zf = ZipFile(zfname, 'r')
try:
# creo todos los subdirectorios que existen en el zip
# (vienen en cualquier orden)
namelist = list(zf.namelist())
namelist.sort(key=lambda n:len(n))
for fn in namelist:
if fn.endswith('/'):
fullfn = os.path.join(tmpdir, *fn[:-1].split('/'))
if not os.path.isdir(fullfn):
os.makedirs(fullfn)

# extraigo todos los archivos, respetando sus directorios
for fn in namelist:
if fn.endswith('/'):
continue
fullfn = os.path.join(tmpdir, *fn.split('/'))
ofile = open(fullfn, 'wb')
ofile.write(zf.read(fn))
ofile.close()
# mantener fecha/hora
dt = zf.getinfo(fn).date_time
ft = time.mktime((dt[0],dt[1],dt[2],dt[3],dt[4],dt[5],0,0,-1))
os.utime(fullfn, (ft,ft))
finally:
zf.close()
--- cut ---


--
Gabriel Genellina
Softlab SRL 


__
Correo Yahoo!
Espacio para todos tus mensajes, antivirus y antispam ¡gratis! 
¡Abrí tu cuenta ya! - http://correo.yahoo.com.ar
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: About alternatives to Matlab

[Matimus]

 Boris wrote:
 > Hi, is there any alternative software for Matlab? Although Matlab is
 > powerful & popular among mathematical & engineering guys, it still
 > costs too much & not publicly open. So I wonder if there's similar
 > software/lang that is open & with comparable functionality, at least
 > for numerical aspect. Thanks!

There is also Scilab. I've only used it a tiny bit but for the task it
worked well. I do know that it has a somewhat restrictive license. It
is open source, but you aren't allowed to modify and redistribute the
source. I get the feeling that some people avoid Scilab but I'm not
sure why. If anybody knows the reason I'd be happy to hear it.

Scilab website: http://www.scilab.org

-Matt

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: os.lisdir, gets unicode, returns unicode... USUALLY?!?!?

[Martin v. Löwis]

gabor schrieb:
>> All this code will typically work just fine with the current behavior,
>> so people typically don't see any problem.
>>
> 
> i am sorry, but it will not work. actually this is exactly what i did,
> and it did not work. it dies in the os.path.join call, where file_name
> is converted into unicode. and python uses 'ascii' as the charset in
> such cases. but, because listdir already failed to decode the file_name
> with the filesystem-encoding, it usually also fails when tried with
> 'ascii'.

Ah, right. So yes, it will typically fail immediately - just as you
wanted it to do, anyway; the advantage with this failure is that you
can also find out what specific file name is causing the problem
(whereas when listdir failed completely, you could not easily find
 out the cause of the failure).

How would you propose listdir should behave?

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Multithreaded C API Python questions

[Svein Seldal]

robert wrote:

> Forget all the low level PyGIL... functions.'

Quite the contrary, I would say!

 From the Py C API doc, I interpret the PyGIL... functions to be higher 
leveled than eg. PyEval_SaveThread(). I've checked into the source of 
python to find out what's really going on, and I've learnt a couple of 
interesting facts. The PyGIL... are helper functions build over the 
PyEval... functions, not the other way around.

A real beauty about the PyGILState_Ensure() is the fact that if it's 
called from a thread that is (yet) unknown to python, it will actually 
create the thread state object for you (with PyThreadState_New) and 
acquire the lock. Hence, I dont have to worry about the process of 
swapping thread states with PyEval_xxxThread() functions. I ONLY need to 
use PyGILState_Ensure() prior to any py-ops.

 From my previous post, I asked about the difference between 
PyEval_ReleaseLock() and PyEval_ReleaseThread() and I've found one major 
difference. Both release the GIL as the docs states, but the 
PyEval_ReleaseThread() saves the current thread state as well. When I 
used PyEval_ReleaseLock() this didnt happen, preventing proper saving of 
the current thread, causing py crash when control were handed back to 
this thread.

By using:

PyEval_InitThreads();
py_ops();
PyThreadState *pts = PyGILState_GetThisThreadState();
PyEval_ReleaseThread(pts);

And for each py-op later on (from arbitrary thread):

PyGILState_STATE gstate;
gstate = PyGILState_Ensure();
py_ops();
PyGILState_Release(gstate);

Then you're home free. Everything related to threading is handled by 
Py/API itself.

> Probably you just do a PyRun_ in main thread and then everything 
> else in Python, and expose C-parts for the thread-loop to Python as 
> function (in other_py_inits) - where in the c-function you probably have 
> the usual Py_BEGIN_ALLOW_THREADS/Py_END_ALLOW_THREADS  bracket during 
> time consuming/system/reenter-endangered message stuff.

The app i'm building is a plugin to a server system, and I cant control 
nor remove any threads that the server uses. I am given one main thread 
to run python forever, and messages that are to be delivered into python 
are called from another.

I could do it like you propose: The data coming from my server will 
arrive into a c-function called by a server thread. The c-part of the 
thread-loop would then be run as another thread (started from  python). 
Its fully feasible, yet the challenge is to make proper data sync-ing 
between these two threads.

Well, I think I've achieved what I wanted. Python seems apparently 
stable, and reading from the python sources, I cant see any immediate 
reasons why it shouldn't.

Thanks for letting me closer to a working solution!


Regads,
Svein
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python v PHP: fair comparison?

[walterbyrd]

Tim Chase wrote:

> I can't say I've come across any hosting places that serve up PHP
> for $10/yr either...the closest I've found is about $3.50/mo
> (which also provides Python CGI).


dollar-hosting.net offers php5 and python 2.3, for $10 a year.
the-protagonist.net has PHP 4.4 hosting for $10 a year. I'm sure I
could find more. Both of those sites use Apache 1.3.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: os.lisdir, gets unicode, returns unicode... USUALLY?!?!?

[gabor]

Martin v. Löwis wrote:
> gabor schrieb:
> 
>> or am i using os.listdir the "wrong way"? how do other people deal with
>> this?
> 
> You didn't say why the behavior causes a problem for you -  you only
> explained what the behavior is.
> 
> Most people use os.listdir in a way like this:
> 
> for name in os.listdir(path):
>   full = os.path.join(path, name)
>   attrib = os.stat(full)
>   if some-condition:
> f = open(full)
>   ...
> 
> All this code will typically work just fine with the current behavior,
> so people typically don't see any problem.
> 

i am sorry, but it will not work. actually this is exactly what i did,
and it did not work. it dies in the os.path.join call, where file_name 
is converted into unicode. and python uses 'ascii' as the charset in 
such cases. but, because listdir already failed to decode the file_name 
with the filesystem-encoding, it usually also fails when tried with 'ascii'.

example:

 >>> dir_name = u'something'
 >>> unicode_file_name = u'\u732b.txt' # the japanese cat-symbol
 >>> bytestring_file_name = unicode_file_name.encode('utf-8')
 >>>
 >>>
 >>> import os.path
 >>>
 >>> os.path.join(dir_name,unicode_file_name)
u'something/\u732b.txt'
 >>>
 >>>
 >>> os.path.join(dir_name,bytestring_file_name)
Traceback (most recent call last):
   File "", line 1, in ?
   File "/usr/lib/python2.4/posixpath.py", line 65, in join
 path += '/' + b
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe7 in position 1: 
ordinal not in range(128)
 >>>


gabor
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: About alternatives to Matlab

[Cameron Laird]

In article <[EMAIL PROTECTED]>,
sturlamolden <[EMAIL PROTECTED]> wrote:
>Boris wrote:
>> Hi, is there any alternative software for Matlab? Although Matlab is
>> powerful & popular among mathematical & engineering guys, it still
>> costs too much & not publicly open. So I wonder if there's similar
>> software/lang that is open & with comparable functionality, at least
>> for numerical aspect. Thanks!
>
>I have used Matlab for years, and has recently changed to Python. In
.
.
.
While I frequently push Matlab users toward Python, they also deserve
to know about Octave http://www-128.ibm.com/developerworks/library/l-oslab/?n-l-10242 >.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: os.lisdir, gets unicode, returns unicode... USUALLY?!?!?

[Martin v. Löwis]

gabor schrieb:
> so basically i'd like to ask here: am i reading something incorrectly?

You are reading it correctly. This is how it behaves.

> or am i using os.listdir the "wrong way"? how do other people deal with
> this?

You didn't say why the behavior causes a problem for you -  you only
explained what the behavior is.

Most people use os.listdir in a way like this:

for name in os.listdir(path):
  full = os.path.join(path, name)
  attrib = os.stat(full)
  if some-condition:
f = open(full)
  ...

All this code will typically work just fine with the current behavior,
so people typically don't see any problem.

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: os.lisdir, gets unicode, returns unicode... USUALLY?!?!?

[Terry Reedy]

"gabor" <[EMAIL PROTECTED]> wrote in message 
news:[EMAIL PROTECTED]
> so if the to-unicode-conversion fails, it falls back to the original
> byte-string. i went and have read the patch-discussion.
>
> and now i'm not sure what to do.
> i know that:
>
> 1. the documentation is completely wrong. it does not always return
> unicode filenames

Unless someone says otherwise, report the discrepancy between doc and code 
as a bug on the SF tracker.  I have no idea of what the resolution should 
be ;-).

tjr



-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Will GPL Java eat into Python marketshare?

[Paul Boddie]

Maurice LING wrote:
>
> I take a simplistic view that Java bytecodes is all that is to be dealt
> with.

Well, that and things like class loading. You'll need a library
implementing the "standard" Java API, although that's never been hard
to obtain, and the official implementation will be genuinely free to
redistribute before too long as well.

> And the book "Programming for the Java Virtual Machine" by Joshua
> Engel did demonstrated in principle that it is possible to program in
> Java bytecodes. There are also other researchware which tried to compile
> other languages into Java bytecodes. I am not sure if exception handling
> etc are dealt with before bytecode level. I also envision that the core
> implementation of JVM is a big switch statement, just like in Python VM
> (the bytecode executor).

It's just an instruction set, but with some fairly "complicated"
instructions, just as you find in the Python virtual machine
instruction set - don't expect them all to be like RISC instructions,
or even traditional CISC instructions.

> Will it then be the case of adding the set of Java bytecodes into Python
> bytecodes and implementing Java bytecode operations? Or am I just
> fooling myself here?

Take a look at the code, although you might want to steer clear of the
import hooks initially:

http://www.python.org/pypi/javaclass

Paul

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Will GPL Java eat into Python marketshare?

[Maurice LING]

Paul Boddie wrote:

> Maurice LING wrote:
> 
>>Say given cytoscape.jar, I'll like to be able to do this:
>>
>> >>> from cytoscape javaimport cytoscape
>> >>> c = cytoscape()
>>
>>And the tighest way I see that this can be done is for Python VM to
>>execute Java bytecodes like Python bytecodes. That is, Python VM
>>executes Java bytecodes directly and not through object mapping which I
>>think is that JPyPe is doing.
> 
> 
> This kind of thing is what I wanted to do with my javaclass experiment,
> but I lost motivation/momentum after getting only some of the things to
> work. Translating Java bytecodes isn't hard for most of the instruction
> set, although exception handling was awkward at the time (before Python
> 2.5's new-style Exception class), and I'd have to think through the
> interpackage referencing problem again (circular references are not
> typically a problem in the Java package hierarchy). In the end, I
> couldn't justify spending the time in order to use certain pieces of
> software that weren't so compelling after all.
> 
>

I take a simplistic view that Java bytecodes is all that is to be dealt 
with. And the book "Programming for the Java Virtual Machine" by Joshua 
Engel did demonstrated in principle that it is possible to program in 
Java bytecodes. There are also other researchware which tried to compile 
other languages into Java bytecodes. I am not sure if exception handling 
etc are dealt with before bytecode level. I also envision that the core 
implementation of JVM is a big switch statement, just like in Python VM 
(the bytecode executor).

Will it then be the case of adding the set of Java bytecodes into Python 
bytecodes and implementing Java bytecode operations? Or am I just 
fooling myself here?

ML
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python v PHP: fair comparison?

[[EMAIL PROTECTED]]

Luis M. González wrote:
> Cameron Laird ha escrito:
> > Perhaps it's timely to clarify the "newer" above:  Guido
> > made Python public in '89-90, and Rasmus showed PHP to
> > others in '94-95.
>
> OK. But since when has python been considered a viable alternative for
> web development?
> As a generalp purpose language, it's older.
> But as a web development language, it's olnly when people started to
> look for the "rails killer" and many python alternatives started to
> come up (although Django has been in development for a long time before
> all this hype).

Huh?  I've been doing paid python web work since 2000.  I'm fairly sure
that Yahoo! groups predates that by a while, and I know that
mod_python/httpdapy goes back at least to 1998 (and Python CGI predates
that significantly).

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Question about unreasonable slowness

[Steven D'Aprano]

On Thu, 16 Nov 2006 12:45:18 -0800, allenjo5 wrote:

> [ Warning: I'm new to Python.  Don't know it at all really yet, but had
> to examine some 3rd party code because of performance problems with it.
> ]
> 
> Here's a code snippet:
> 
> i = 0
> while (i < 20):
>   i = i + 1


You probably want to change that to:

for i in range(20):

If 20 is just a place-holder, and the real value is much bigger, change
the range() to xrange().


>   (shellIn, shellOut) = os.popen4("/bin/sh -c ':'")  # for testing, the
> spawned shell does nothing
>   print 'next'
> #  for line in shellOut:
> #   print line
> 
> On my system (AIX 5.1 if it matters, with Python 2.4.3), this simple
> loop spawning 20 subshells takes .75 sec.  Ok, that's reasonable.  Now,
> if I uncomment the two commented lines, which loop over the empty
> shellOut array, the progam now takes 11 secs.   That slowdown seems
> very hard to believe.  Why should it slow down so much?

What are you using to time the code?

Replacing print statements with "pass", I get these results:

>>> import timeit
>>>
>>> def test():
... i = 0
... while (i < 20):
... i = i + 1
... (shellIn, shellOut) = os.popen4("/bin/sh -c ':'")
... pass # print 'next'
... for line in shellOut:
... pass # print line
...
>>> timeit.Timer("test()", "from __main__ import test\nimport os").timeit(1)
0.49781703948974609
>>> timeit.Timer("test()", "from __main__ import test\nimport os").timeit(100)
54.894074201583862

About 0.5 second to open and dispose of 20 subshells, even with the "for
line in shellOut" loop.


I think you need some more fine-grained testing to determine whether the
slowdown is actually happening inside the "for line in shellOut" loop or
inside the while loop or when the while loop completes.



-- 
Steven.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: About alternatives to Matlab

[Ramon Diaz-Uriarte]

R (http://cran.r-project.org) might be an alternative, specially if
you do a lot of statistics and graphics. (R is probably the most
widely used language/system in statistical research).


R.

On 16 Nov 2006 13:09:03 -0800, sturlamolden <[EMAIL PROTECTED]> wrote:
> Boris wrote:
> > Hi, is there any alternative software for Matlab? Although Matlab is
> > powerful & popular among mathematical & engineering guys, it still
> > costs too much & not publicly open. So I wonder if there's similar
> > software/lang that is open & with comparable functionality, at least
> > for numerical aspect. Thanks!
>
> I have used Matlab for years, and has recently changed to Python. In
> addition one needs NumPy and Matplotlib, and perhaps SciPy. Other
> useful packages are PyGTK for GUI, PyGame for multimedia, PyOpenGL for
> 3D graphics, Mpi4Py for parallel computation, etc. You will find python
> packages for nearly any conceivable task. Unlike Matlab, Python is a
> general purpose programming language, and a distant cousin of Lisp. The
> Python language is fare more expressive and productive than Matlab, yet
> even more easy to use.
>
> The NumPy package is the core requirement for numerical work in Python.
> It is quite different form Matlab, but I think it is more powerful.
> Particularly, arrays are passed by reference (not by value), and
> indexing creates view matrices.
>
> To compare Matlab with NumPy we can e.g. use the D4 discrete wavelet
> transform. I have here coded it in Matlab and Python/NumPy using Tim
> Swelden's lifting scheme.
>
> First the Matlab version (D4_Transform.m):
>
> function x = D4_Transform(x)
>% D4 Wavelet transform in Matlab
>% (C) Sturla Molden
>C1 =  1.7320508075688772;
>C2 =  0.4330127018922193;
>C3 = -0.066987298107780702;
>C4 =  0.51763809020504137;
>C5 =  1.9318516525781364;
>s1 = zeros(ceil(size(x)/2));
>d1 = zeros(ceil(size(x)/2));
>d2 = zeros(ceil(size(x)/2));
>odd = x(2:2:end);
>even = x(1:2:end-1);
>d1(:) = odd - C2*even;
>s1(1) = even(1) + C2*d1(1) + C3*d1(end);
>s1(2:end) = even(2:end) + C2*d1(2:end) + C3*d1(1:end-1);
>d2(1) = d1(1) + s1(end);
>d2(2:end) = d1(2:end) + s1(1:end-1);
>x(1:2:end-1) = C4*s1;
>x(2:2:end) = C5*d2;
>if (length(x) >2)
>   x(1:2:end-1) = D4_Transform(x(1:2:end-1));
>end
>
>
> Then the Python version (D4.py):
>
> import numpy
> import time
>
> def D4_Transform(x, s1=None, d1=None, d2=None):
>"""
>D4 Wavelet transform in NumPy
>(C) Sturla Molden
>"""
>C1 = 1.7320508075688772
>C2 = 0.4330127018922193
>C3 = -0.066987298107780702
>C4 = 0.51763809020504137
>C5 = 1.9318516525781364
>if d1 == None:
>   d1 = numpy.zeros(x.size/2)
>   s1 = numpy.zeros(x.size/2)
>   d2 = numpy.zeros(x.size/2)
>odd = x[1::2]
>even = x[:-1:2]
>d1[:] = odd[:] - C1*even[:]
>s1[0] = even[0] + C2*d1[0] + C3*d1[-1]
>s1[1:] = even[1:] + C2*d1[1:] + C3*d1[:-1]
>d2[0] = d1[0] + s1[-1]
>d2[1:] = d1[1:] + s1[:-1]
>even[:] = C4 * s1[:]
>odd[:] = C5 * d2[:]
>if x.size > 2:
>
> D4_Transform(even,s1[0:even.size/2],d1[0:even.size/2],d2[0:even.size/2])
>
> if __name__ == "__main__":
>x = numpy.random.rand(2**23)
>t0 = time.clock()
>D4_Transform(x)
>t = time.clock()
>print "Elapsed time is %.6f seconds" % (t-t0)
>
>
> Now let's do benchmark on my laptop (1.73 GHz Pentium M, 0.99 GB RAM).
> I have stopped paying for Matlab maintenance (for reasons that will be
> obvious), so I only have R14 Service Pack 2 for comparison.
>
> First we try Matlab (R14 Service Pack 2):
>
> >> x = rand(2^23,1);
> >> tic; D4_Transform(x); toc
> Elapsed time is 27.145438 seconds.
>
>
> Then we Python 2.4.4 with NumPy 1.0:
>
> C:\develop\python\D4>python D4.py
> Elapsed time is 3.364887 seconds
>
> That is quite astonishing.
>
> If anyone wonders why I think Travis Oliphant and the NumPy team should
> be knighted, then this is the answer. The Mathworks' product only
> achieved 100% * 3/27 = 11% the speed of Python/NumPy, and is infinitely
> more expensive.
>
> Does anyone wonder why I am not paying for Matlab maintenance anymore?
>
> Sorry Mathworks, I have used your product for years, but you cannot
> compete with NumPy.
>
>
> Cheers,
> Sturla Molden
>
> --
> http://mail.python.org/mailman/listinfo/python-list
>


-- 
Ramon Diaz-Uriarte
Statistical Computing Team
Structural Biology and Biocomputing Programme
Spanish National Cancer Centre (CNIO)
http://ligarto.org/rdiaz
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Question about unreasonable slowness

[Leif K-Brooks]

[EMAIL PROTECTED] wrote:
> i = 0
> while (i < 20):
>   i = i + 1

for i in xrange(20):

>   (shellIn, shellOut) = os.popen4("/bin/sh -c ':'")  # for testing, the
> spawned shell does nothing
>   print 'next'
> #  for line in shellOut:
> #   print line
> 
> On my system (AIX 5.1 if it matters, with Python 2.4.3), this simple
> loop spawning 20 subshells takes .75 sec.  Ok, that's reasonable.  Now,
> if I uncomment the two commented lines, which loop over the empty
> shellOut array, the progam now takes 11 secs.   That slowdown seems
> very hard to believe.  Why should it slow down so much?

The key fact here is that shellOut isn't an array; it's a living,
breathing file object. If you don't iterate over it, you can run all 20
shell processes in parallel if necessary; but if you do iterate over it,
   you're waiting for sh's stdout pipe to reach EOF, which effectively
means you can only run one process at a time.

On my system (OS X 10.4 with Python 2.5 installed), your code runs in
.187 secs with the loop commented out, and in .268 secs otherwise. But I
guess AIX's sh is slower than OS X's.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Will GPL Java eat into Python marketshare?

[Paul Boddie]

Maurice LING wrote:
>
> Say given cytoscape.jar, I'll like to be able to do this:
>
>  >>> from cytoscape javaimport cytoscape
>  >>> c = cytoscape()
>
> And the tighest way I see that this can be done is for Python VM to
> execute Java bytecodes like Python bytecodes. That is, Python VM
> executes Java bytecodes directly and not through object mapping which I
> think is that JPyPe is doing.

This kind of thing is what I wanted to do with my javaclass experiment,
but I lost motivation/momentum after getting only some of the things to
work. Translating Java bytecodes isn't hard for most of the instruction
set, although exception handling was awkward at the time (before Python
2.5's new-style Exception class), and I'd have to think through the
interpackage referencing problem again (circular references are not
typically a problem in the Java package hierarchy). In the end, I
couldn't justify spending the time in order to use certain pieces of
software that weren't so compelling after all.

> I must say that this is part of even a fluffier dream that one day, I
> can take any applications and play around with it in Python. Currently,
> my collaborators wrote in Perl and Java, so it is not easy for me to use
> their work in my work.

There's always Jython, as I tell the occasional person who tries
javaclass expecting it to be better than it is. That Jython doesn't
support the latest CPython features shouldn't be as huge a problem as
people suspect, especially if you just want to glue Java packages
together with Python. Otherwise, the PyLucene approach (gcj + bindings)
might be an acceptable approach: a well performing Free Software
solution even before the recent Java licensing events.

Paul

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: About alternatives to Matlab

[John Henry]

Bill Gates will have you jailed! :-)

On a more serious note, is there any alternative to Simulink though?

sturlamolden wrote:
>and is infinitely
> more expensive.
>
> Does anyone wonder why I am not paying for Matlab maintenance anymore?
>
> Sorry Mathworks, I have used your product for years, but you cannot
> compete with NumPy.
> 
> 
> Cheers,
> Sturla Molden

-- 
http://mail.python.org/mailman/listinfo/python-list

RE: Question about unreasonable slowness

[Éric Daigneault]

[ Warning: I'm new to Python.  Don't know it at all really yet, but had
to examine some 3rd party code because of performance problems with it.
]

Here's a code snippet:

i = 0
while (i < 20):
  i = i + 1
  (shellIn, shellOut) = os.popen4("/bin/sh -c ':'")  # for testing, the
spawned shell does nothing
  print 'next'
#  for line in shellOut:
#   print line

On my system (AIX 5.1 if it matters, with Python 2.4.3), this simple
loop spawning 20 subshells takes .75 sec.  Ok, that's reasonable.  Now,
if I uncomment the two commented lines, which loop over the empty
shellOut array, the progam now takes 11 secs.   That slowdown seems
very hard to believe.  Why should it slow down so much?

John.
---
Tried it on my comp (battered up winXP laptop on it's last breath) with the 
following change...

  (shellIn, shellOut) = os.popen4("cmd ")

and the performance with or without the inserted for loop is roughly equivalent 
albeit a /bit/ slower with  the for loop is  but I would need timers really 
make it stand out (under 1 second difference).

All in all, rough estimation is that both take more or less 4 seconds to 
complete which is what I would expect knowing my system...

Anyways, is what I got...

Hope it helps somehow

Éric :D.


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: accessing fortran modules from python

[Robert Kern]

sam wrote:
> hello all,
> 
> i am currently in the process of planning a piece of software to model
> polymerisation kinetics, and intend to use python for all the
> high-level stuff. the number-crunching is something i would prefer to
> do in fortran (which i have never used, but will learn), but i have no
> experience of accessing non-python code from python. i am also fairly
> new to programming period, and am therefore tackling a fairly serious
> issue reletive to my experience.
> 
> how easy is it to get fortran modules running from python? if c is
> easier to use in this respect, i could go in that direction instead.
> 
> i realize there is a lot of material on this subject already available,
> but i am finding it difficult to make sense of, it's like trying to
> take a drink from a fire hose.
> 
> any advice would be gratefully received. i will almost certainly be
> coding this on windows, for what it's worth.
> 
> thank you,
> 
> sam
> 
> PS if numpy is adequate for this, i would be quite happy to use it. i
> got the impression it was more for matrix algebra. i will be
> programming a monte carlo simulation, which will need to perform a lot
> (a lot!) of simple operations over and over...

numpy isn't for matrix algebra, specifically. It is primarily designed for
operations on arrays. If your simple operations can be "vectorized," then numpy
will benefit you. If you can describe your needs in more detail, we'll be happy
to give you suggestions on the numpy-discussion list.

  http://www.scipy.org/Mailing_Lists

Of course, since the major player in wrapping Fortran libraries for Python is
f2py, a component of numpy, you're probably going to end up installing numpy
anyways.

  http://numpy.scipy.org

All of the f2py documentation is currently in the source, so start here:

  http://projects.scipy.org/scipy/numpy/browser/trunk/numpy/f2py/docs/README.txt

Using f2py to wrap Fortran subroutines is actually a fair bit simpler than
wrapping C code, mostly because typical Fortran uses a limited set of simple 
types.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: multi split function taking delimiter list

[Paul McGuire]

On Nov 14, 5:41 pm, "Sam Pointon" <[EMAIL PROTECTED]> wrote:
> On Nov 14, 7:56 pm, "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
> wrote:
>
> > Hi, I'm looking for something like:
>
> > multi_split( 'a:=b+c' , [':=','+'] )
>
> > returning:
> > ['a', ':=', 'b', '+', 'c']
>
> > whats the python way to achieve this, preferably without regexp?
>
> pyparsing  is quite a cool package
> for doing this sort of thing.

Thanks for mentioning pyparsing, Sam!

This is a good example of using pyparsing for just basic tokenizing,
and it will do a nice job of splitting up the tokens, whether there is
whitespace or not.

For instance, if you were tokenizing using the string split() method,
you would get nice results from "a := b + c", but not so good from "a:=
b+ c".  Using Sam Pointon's simple pyparsing expression, you can split
up the arithmetic using the symbol expressions, and the whitespace is
pretty much ignored.

But pyparsing can be used for more than just tokenizing.  Here is a
slightly longer pyparsing example, using a new pyparsing helper method
called operatorPrecedence, which can shortcut the definition of
operator-separated expressions with () grouping.  Note how this not
only tokenizes the expression, but also identifies the implicit groups
based on operator precedence.  Finally, pyparsing allows you to label
the parsed results - in this case, you can reference the LHS and RHS
sides of your assignment statement using the attribute names "lhs" and
"rhs".  This can really be handy for complicated grammars.

-- Paul


from pyparsing import *

number = Word(nums)
variable = Word(alphas)
operand = number | variable

arithexpr = operatorPrecedence( operand,
[("!", 1, opAssoc.LEFT),  # factorial
 ("^", 2, opAssoc.RIGHT), # exponentiation
 (oneOf('+ -'), 1, opAssoc.RIGHT),  # leading sign
 (oneOf('* /'), 2, opAssoc.LEFT),   # multiplication
 (oneOf('+ -'), 2, opAssoc.LEFT),]  # addition
)

assignment = (variable.setResultsName("lhs") +
":=" +
arithexpr.setResultsName("rhs"))

test = ["a:= b+c",
"a := b + -c",
"y := M*X + B",
"e := m * c^2",]

for t in test:
tokens = assignment.parseString(t)
print tokens.asList()
print tokens.lhs, "<-", tokens.rhs
print

Prints:
['a', ':=', ['b', '+', 'c']]
a <- ['b', '+', 'c']

['a', ':=', ['b', '+', ['-', 'c']]]
a <- ['b', '+', ['-', 'c']]

['y', ':=', [['M', '*', 'X'], '+', 'B']]
y <- [['M', '*', 'X'], '+', 'B']

['e', ':=', ['m', '*', ['c', '^', 2]]]
e <- ['m', '*', ['c', '^', 2]]

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Will GPL Java eat into Python marketshare?

[Maurice LING]

Stephen Eilert wrote:

> Maurice LING escreveu:
> 
> 
>>>I once wrote a partial JVM in Modula-3 (strictly a researchware
>>>effort), so I can imagine it being done technically.  But why?
>>>
>>>The big problem with Java-and-Python is not the VMs underneath.  It is
>>>the fact that Java has layers upon layers upon layers of idiosyncratic
>>>libraries and idioms.  When you write bindings to that world (even if
>>>the bindings are generated automagically), you have to *think* in
>>>those same layers.  The Python-oriented developer suddenly has to use
>>>a dozen imports in order to do things already done better in
>>>Pythonesque libraries.
>>>
>>
>>The main use I can see is to be able to incorporate Java applications
>>into Python. For example, I am using Cytoscape (www.cytoscape.org) which
>>is in Java. I do hope that I can control Cytoscape from Python and
>>manipulate its objects from Python.
>>
>>Say given cytoscape.jar, I'll like to be able to do this:
>>
>> >>> from cytoscape javaimport cytoscape
>> >>> c = cytoscape()
>>
>>And the tighest way I see that this can be done is for Python VM to
>>execute Java bytecodes like Python bytecodes. That is, Python VM
>>executes Java bytecodes directly and not through object mapping which I
>>think is that JPyPe is doing.
>>
>>I must say that this is part of even a fluffier dream that one day, I
>>can take any applications and play around with it in Python. Currently,
>>my collaborators wrote in Perl and Java, so it is not easy for me to use
>>their work in my work.
>>
>>ML
> 
> 
> What is wrong with the other way around and Jython?
> 
Nothing wrong with the other way round - JVM executing *Python bytecodes*.

Cytoscape has a plugin with enables one to bring up Jython interpreter 
but it is way too slow - make sure you start to load it up before lunch 
if you want to execute a few lines of codes after lunch.

ML
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Decimal() instead of float?

[Gabriel Genellina]

At Thursday 16/11/2006 04:40, Fredrik Lundh wrote:


internal rowid's are best treated as pointers, though.  they're more
like memory addresses than labels.

(from a design perspective, it's not entirely obvious that it's a good
idea to use rowid's to point *into* the database from the outside, but
that's another story).


You could consider them like id() in python - an identifier that, 
although it's in fact a pointer, should not be dereferenced.



--
Gabriel Genellina
Softlab SRL  


__
Correo Yahoo!
Espacio para todos tus mensajes, antivirus y antispam ¡gratis! 
¡Abrí tu cuenta ya! - http://correo.yahoo.com.ar
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: About alternatives to Matlab

[sturlamolden]

Boris wrote:
> Hi, is there any alternative software for Matlab? Although Matlab is
> powerful & popular among mathematical & engineering guys, it still
> costs too much & not publicly open. So I wonder if there's similar
> software/lang that is open & with comparable functionality, at least
> for numerical aspect. Thanks!

I have used Matlab for years, and has recently changed to Python. In
addition one needs NumPy and Matplotlib, and perhaps SciPy. Other
useful packages are PyGTK for GUI, PyGame for multimedia, PyOpenGL for
3D graphics, Mpi4Py for parallel computation, etc. You will find python
packages for nearly any conceivable task. Unlike Matlab, Python is a
general purpose programming language, and a distant cousin of Lisp. The
Python language is fare more expressive and productive than Matlab, yet
even more easy to use.

The NumPy package is the core requirement for numerical work in Python.
It is quite different form Matlab, but I think it is more powerful.
Particularly, arrays are passed by reference (not by value), and
indexing creates view matrices.

To compare Matlab with NumPy we can e.g. use the D4 discrete wavelet
transform. I have here coded it in Matlab and Python/NumPy using Tim
Swelden's lifting scheme.

First the Matlab version (D4_Transform.m):

function x = D4_Transform(x)
   % D4 Wavelet transform in Matlab
   % (C) Sturla Molden
   C1 =  1.7320508075688772;
   C2 =  0.4330127018922193;
   C3 = -0.066987298107780702;
   C4 =  0.51763809020504137;
   C5 =  1.9318516525781364;
   s1 = zeros(ceil(size(x)/2));
   d1 = zeros(ceil(size(x)/2));
   d2 = zeros(ceil(size(x)/2));
   odd = x(2:2:end);
   even = x(1:2:end-1);
   d1(:) = odd - C2*even;
   s1(1) = even(1) + C2*d1(1) + C3*d1(end);
   s1(2:end) = even(2:end) + C2*d1(2:end) + C3*d1(1:end-1);
   d2(1) = d1(1) + s1(end);
   d2(2:end) = d1(2:end) + s1(1:end-1);
   x(1:2:end-1) = C4*s1;
   x(2:2:end) = C5*d2;
   if (length(x) >2)
  x(1:2:end-1) = D4_Transform(x(1:2:end-1));
   end


Then the Python version (D4.py):

import numpy
import time

def D4_Transform(x, s1=None, d1=None, d2=None):
   """
   D4 Wavelet transform in NumPy
   (C) Sturla Molden
   """
   C1 = 1.7320508075688772
   C2 = 0.4330127018922193
   C3 = -0.066987298107780702
   C4 = 0.51763809020504137
   C5 = 1.9318516525781364
   if d1 == None:
  d1 = numpy.zeros(x.size/2)
  s1 = numpy.zeros(x.size/2)
  d2 = numpy.zeros(x.size/2)
   odd = x[1::2]
   even = x[:-1:2]
   d1[:] = odd[:] - C1*even[:]
   s1[0] = even[0] + C2*d1[0] + C3*d1[-1]
   s1[1:] = even[1:] + C2*d1[1:] + C3*d1[:-1]
   d2[0] = d1[0] + s1[-1]
   d2[1:] = d1[1:] + s1[:-1]
   even[:] = C4 * s1[:]
   odd[:] = C5 * d2[:]
   if x.size > 2:

D4_Transform(even,s1[0:even.size/2],d1[0:even.size/2],d2[0:even.size/2])

if __name__ == "__main__":
   x = numpy.random.rand(2**23)
   t0 = time.clock()
   D4_Transform(x)
   t = time.clock()
   print "Elapsed time is %.6f seconds" % (t-t0)


Now let's do benchmark on my laptop (1.73 GHz Pentium M, 0.99 GB RAM).
I have stopped paying for Matlab maintenance (for reasons that will be
obvious), so I only have R14 Service Pack 2 for comparison.

First we try Matlab (R14 Service Pack 2):

>> x = rand(2^23,1);
>> tic; D4_Transform(x); toc
Elapsed time is 27.145438 seconds.


Then we Python 2.4.4 with NumPy 1.0:

C:\develop\python\D4>python D4.py
Elapsed time is 3.364887 seconds

That is quite astonishing.

If anyone wonders why I think Travis Oliphant and the NumPy team should
be knighted, then this is the answer. The Mathworks' product only
achieved 100% * 3/27 = 11% the speed of Python/NumPy, and is infinitely
more expensive.

Does anyone wonder why I am not paying for Matlab maintenance anymore?

Sorry Mathworks, I have used your product for years, but you cannot
compete with NumPy.


Cheers,
Sturla Molden

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Will GPL Java eat into Python marketshare?

[Stephen Eilert]

Maurice LING escreveu:

> >
> > I once wrote a partial JVM in Modula-3 (strictly a researchware
> > effort), so I can imagine it being done technically.  But why?
> >
> > The big problem with Java-and-Python is not the VMs underneath.  It is
> > the fact that Java has layers upon layers upon layers of idiosyncratic
> > libraries and idioms.  When you write bindings to that world (even if
> > the bindings are generated automagically), you have to *think* in
> > those same layers.  The Python-oriented developer suddenly has to use
> > a dozen imports in order to do things already done better in
> > Pythonesque libraries.
> >
>
> The main use I can see is to be able to incorporate Java applications
> into Python. For example, I am using Cytoscape (www.cytoscape.org) which
> is in Java. I do hope that I can control Cytoscape from Python and
> manipulate its objects from Python.
>
> Say given cytoscape.jar, I'll like to be able to do this:
>
>  >>> from cytoscape javaimport cytoscape
>  >>> c = cytoscape()
>
> And the tighest way I see that this can be done is for Python VM to
> execute Java bytecodes like Python bytecodes. That is, Python VM
> executes Java bytecodes directly and not through object mapping which I
> think is that JPyPe is doing.
>
> I must say that this is part of even a fluffier dream that one day, I
> can take any applications and play around with it in Python. Currently,
> my collaborators wrote in Perl and Java, so it is not easy for me to use
> their work in my work.
>
> ML

What is wrong with the other way around and Jython?

Just curious.


Stephen

-- 
http://mail.python.org/mailman/listinfo/python-list

os.lisdir, gets unicode, returns unicode... USUALLY?!?!?

[gabor]

hi,

from the documentation (http://docs.python.org/lib/os-file-dir.html) for 
os.listdir:

"On Windows NT/2k/XP and Unix, if path is a Unicode object, the result 
will be a list of Unicode objects."

i'm on Unix. (linux, ubuntu edgy)

so it seems that it does not always return unicode filenames.

it seems that it tries to interpret the filenames using the filesystem's 
encoding, and if that fails, it simply returns the filename as byte-string.

so you get back let's say an array of 21 filenames, from which 3 are 
byte-strings, and the rest unicode strings.

after digging around, i found this in the source code:

> #ifdef Py_USING_UNICODE
> if (arg_is_unicode) {
> PyObject *w;
> 
> w = PyUnicode_FromEncodedObject(v,
> Py_FileSystemDefaultEncoding,
> "strict");
> if (w != NULL) {
> Py_DECREF(v);
> v = w;
> }
> else {
> /* fall back to the original byte string, as
>discussed in patch #683592 */
> PyErr_Clear();
> }
> }
> #endif

so if the to-unicode-conversion fails, it falls back to the original 
byte-string. i went and have read the patch-discussion.

and now i'm not sure what to do.
i know that:

1. the documentation is completely wrong. it does not always return 
unicode filenames
2. it's true that the documentation does not specify what happens if the 
filename is not in the filesystem-encoding, but i simply expected that i 
get an Unicode-exception, as everywhere else. you see, exceptions are 
ok, i can deal with them. but this is just plain wrong. from now on, 
EVERYWHERE where i use os.listdir, i will have to go through all the 
filenames in it, and check if they are unicode-strings or not.

so basically i'd like to ask here: am i reading something incorrectly? 
or am i using os.listdir the "wrong way"? how do other people deal with 
this?

p.s: one additional note. if you code expects os.listdir to return 
unicode, that usually means that all your code uses unicode strings. 
which in turn means, that those filenames will somehow later interact 
with unicode strings. which means that that byte-string-filename will 
probably get auto-converted to unicode at a later point, and that 
auto-conversion will VERY probably fail, because the auto-convert only 
happens using 'ascii' as the encoding, and if it was not possible to 
decode the filename inside listdir, it's quite probable that it also 
will not work using 'ascii' as the charset.


gabor
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: numpy/scipy: error of correlation coefficient (clumpy data)

[robert]

sturlamolden wrote:
> robert wrote:
> 
>> Think of such example: A drunken (x,y) 2D walker is supposed to walk along a 
>> diagonal, but he makes frequent and unpredictable pauses/slow motion. You 
>> get x,y coordinates in 1 per second. His speed and time pattern at all do 
>> not matter - you just want to know how well he keeps his track.
> 
> 
> In which case you have time series data, i.e. regular samples from p(t)
> = [ x(t), y(t) ]. Time series have some sort of autocorrelation in the
> samples as well, which must be taken into account. Even tough you could
> weight each point by the drunkard's speed, a correlation or linear
> regression would still not make any sense here, as such analyses are
> based on the assumption of no autocorrelation in the samples or the
> residuals. Correlation has no meaning if y[t] is correlated with
> y[t+1], and regression has no meaning if the residual e[t] is
> correlated with the residual e[t+1].
> 
> A state space model could e.g. be applicable. You could estimate the
> path of the drunkard using a Kalman filter to compute a Taylor series
> expansion p(t) = p0 + v*t + 0.5*a*t**2 + ... for the path at each step
> p(t). When you have estimates for the state parameters s, v, and a, you
> can compute some sort of measure for the drunkard's deviation from his
> ideal path.
> 
> However, if you don't have time series data, you should not treat your
> data as such.
> 
> If you don't know how your data is generated, there is no way to deal
> with them correctly. If the samples are time series they must be
> threated as such, if they are not they should not. If the samples are
> i.i.d. each point count equally much, if they are not they do not. If
> you have a clumped data due to time series or lack of i.i.d., you must
> deal with that. However, data can be i.i.d. and clumped, if the
> underlying distribution is clumped. In order to determine the cause,
> you must consider how your data are generated and how your data are
> sampled. You need meta-information about your data to determine this.
> Matlab or Octave will help you with this, and it is certainly not a
> weakness of NumPy as you implied in your original post. There is no way
> to put magic into any numerical computation. Statistics always require
> formulation of specific assumptions about the data. If you cannot think
> clearly about your data, then that is the problem you must solve.

yes, in the example of the drunkard time-series its possible to go to better 
model - yet even there it is very expensive (in relation to the 
stats-improvement) to worry too much about the best model for such guy :-).

In the field of datamining with many datatracks and typically digging first for 
multiple but smaller correlations - without a practical bottom-up modell, I 
think one falls regularly back to a certain basic case - maybe the most basic 
model for data at all: that basic modell is possibly that of a "hunter" which 
waits mostly, but only acts, if rare goodies are in front of him. 
Again in the most basic case, when having 2D x,y data in front of you without a 
relyable time-path or so, you see this: a density distribution of points. There 
is possibly a linear correlation on the highest scale - which you are 
interested in - but the points show also inhomgenity/clumping, and this rises 
the question of influence on r_err. What now? One sees clearly that its 
nonsense to make just plain average stats.
I think this case is a most basic default for data - even compared to the 
common textbook-i.i.d. case.  In fact, one can recognize such kind of stats, 
which  repects mere (inhomogous) data-density itself, as (kind of 
simple/indepent) auto-bayesian stats vs. dumb averaging. 

I think one can almost always do this "bayesian density weighter/filter" as 
better option compared to mere average stats in that case of x,y correlation 
when there is obvioulsy interesting correlation but where you are too 
lazy..to..principally unable to itch out a modell on physics level. The latter 
requirement is in fact what any averaging stats cries for at any price - but 
how often can you do it in real world applications ... 

( In reality there is anyway no way to eliminate auto-correlation in the 
composition of data. Everything and everybody lies :-) )

Thats where a top-down (model-free) bayesian stats approach will pay off: In 
the previous extreme example of criminal data duplication - I'm sure - it will 
totally neutralize the attack without question. In the drunkard time-series 
example it will tell me very reliably how well this guy will keep track - 
without need for a complex model. In the case of good i.i.d. data distribution 
it will tell me the same as simple stats. Just good news ...

Thus I can possibly say it so now: I have the problem for guessing linear 
correlation (coefficient with error) on x,y data with respect to the (most 
general assupmtion) "bayesian" background of inhomogenous data distribution. 
Therefore I'm s

Question about unreasonable slowness

[allenjo5]

[ Warning: I'm new to Python.  Don't know it at all really yet, but had
to examine some 3rd party code because of performance problems with it.
]

Here's a code snippet:

i = 0
while (i < 20):
  i = i + 1
  (shellIn, shellOut) = os.popen4("/bin/sh -c ':'")  # for testing, the
spawned shell does nothing
  print 'next'
#  for line in shellOut:
#   print line

On my system (AIX 5.1 if it matters, with Python 2.4.3), this simple
loop spawning 20 subshells takes .75 sec.  Ok, that's reasonable.  Now,
if I uncomment the two commented lines, which loop over the empty
shellOut array, the progam now takes 11 secs.   That slowdown seems
very hard to believe.  Why should it slow down so much?

John.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Will GPL Java eat into Python marketshare?

[Maurice LING]

> 
> I once wrote a partial JVM in Modula-3 (strictly a researchware
> effort), so I can imagine it being done technically.  But why?
> 
> The big problem with Java-and-Python is not the VMs underneath.  It is
> the fact that Java has layers upon layers upon layers of idiosyncratic
> libraries and idioms.  When you write bindings to that world (even if
> the bindings are generated automagically), you have to *think* in
> those same layers.  The Python-oriented developer suddenly has to use
> a dozen imports in order to do things already done better in
> Pythonesque libraries.
> 

The main use I can see is to be able to incorporate Java applications 
into Python. For example, I am using Cytoscape (www.cytoscape.org) which 
is in Java. I do hope that I can control Cytoscape from Python and 
manipulate its objects from Python.

Say given cytoscape.jar, I'll like to be able to do this:

 >>> from cytoscape javaimport cytoscape
 >>> c = cytoscape()

And the tighest way I see that this can be done is for Python VM to 
execute Java bytecodes like Python bytecodes. That is, Python VM 
executes Java bytecodes directly and not through object mapping which I 
think is that JPyPe is doing.

I must say that this is part of even a fluffier dream that one day, I 
can take any applications and play around with it in Python. Currently, 
my collaborators wrote in Perl and Java, so it is not easy for me to use 
their work in my work.

ML
-- 
http://mail.python.org/mailman/listinfo/python-list

pyfits problem

[Tommy Grav]

I am trying to load in a fits-image and get this error

wiyn05dec/n1 -> display -i obj062.fits
Traceback (most recent call last):
   File "/Users/tgrav/Work/Astronomy/MyCode/Python/Redspit/ 
display.py", line 48, in ?
 main()
   File "/Users/tgrav/Work/Astronomy/MyCode/Python/Redspit/ 
display.py", line 30, in main
 im  = pyfits.getdata(options.infname)
   File "/Library/Frameworks/Python.framework/Versions/2.4/lib/ 
python2.4/site-packages/pyfits/NP_pyfits.py", line 4645, in getdata
 _data = hdu.data
   File "/Library/Frameworks/Python.framework/Versions/2.4/lib/ 
python2.4/site-packages/pyfits/NP_pyfits.py", line 2058, in __getattr__
 self.data = np.array(raw_data, type=np.Float32)
AttributeError: 'module' object has no attribute 'Float32'

I am using numpy 1.0rc3 and ActivePython 2.4.3 Build 11.
Anyone know what is going on?

Cheers
Tommy
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python v PHP: fair comparison?

[Paul Boddie]

John Bokma wrote:
> "Luis M. González" <[EMAIL PROTECTED]> wrote:
> >
> > I know, but would you consider a python cgi application a good
> > competence against php?
>
> php running as cgi you mean.

Perhaps not: he's referring to deployment on really cheap hosting
solutions which might support mod_php but wouldn't support anything
better than CGI for Python software. The suitability of mod_php and the
supposed unsuitability of mod_python for commodity hosting did come up
on comp.lang.python some time ago, a while before certain other trends,
and has seemingly been discussed every year for several years:

http://groups.google.co.uk/group/comp.lang.python/msg/ebf390af4ed6e92e

"Thus, for mod_python hosting to be reliable, you have to give each
user their own instance of the Apache server (proxied through a central
instance of Apache). This might, in fact, no longer be needed with the
latest mod_python, but I leave that to the mod_python experts in this
newsgroup to expound upon if so." (To which there was no response.)

This discussion only differs from those previous discussions [1] in the
sense that better hosting is now available for less, although you can't
get a virtual server for $1/month unless you have some contacts in the
business.

Paul

[1] http://groups.google.co.uk/groups?oi=djq&as_q=mod_python+PHP+hosting

-- 
http://mail.python.org/mailman/listinfo/python-list

RE: Python v PHP: fair comparison?

[John Bokma]

"Demel, Jeff" <[EMAIL PROTECTED]> wrote:

> What kind of hosting does one get for $10 a *year*?  And I'm not just
> talking Python here, but any hosting at all.

I pay $12/year (see other posts, I am not going to name them again) for 
hosting a phpBB board and wiki, with 100+ members and currently close to 
20,000 posts. Minor issues were fixed within about one hour after emailing 
the hosting provider. When I had problems with data storage (my fault, I 
used too much) my 100 MB was extended to 500 MB without additional costs. 

Haven't seen major issues the one year and half the board is up, 
everything goes smooth. No idea how a heavy traffic site would go with 
them though, my personal page with 14k visitors is hosted somewhere else, 
and I pay much more (maybe even too much atm., I am probably going to move 
to VPS anyway).

But for a message board for friends, paying less then 20 USD/year, is fine 
with me. One friend has her own board & blog with them, and since hosting 
comes with cpanel, she didn't need much of my help :-).

-- 
John   MexIT: http://johnbokma.com/mexit/
   personal page:   http://johnbokma.com/
Experienced programmer available: http://castleamber.com/
Happy Customers: http://castleamber.com/testimonials.html
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Will GPL Java eat into Python marketshare?

[Ravi Teja]

> Personally, I've never gotten jpype to work.  Is it just me, or is it
> a troublesome install?
>
> Harry George
> PLM Engineering Architecture

It works fine for me now. However, I do recall having an issue a while
ago (most likely me, rather than JPype).

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Py3K idea: why not drop the colon?

[Ron Adam]

Steve Holden wrote:

> I'm always surprised by the amount of energy that's put into this type 
> of discussion - even the OP has suggested that this isn't a big issue. 
> If it's not a big issue why is this thread still going? Every language 
> has a syntax. Why not just accept it as a given and get on with more 
> productive activities?

It's because it's not an issue with an obviously (clear to everyone) good or 
bad 
answer.

 Obvious bad answer ->  short discussion

 Obvious good answer  ->  short discussion which spawns many long 
discussion 
about the not so obviously details.

   * Nearly a toss up -> really long discussion that rehashes things over and 
over from various micro points of view.

Even though it's pretty much known this won't change, people still want the 
clarity of an obvious answer even if an obvious solution doe not exist, so the 
thread continues.  And of course what is obvious or not isn't obvious to 
everyone, so that too is a long discussion. ;-)

But I think this thread has a second underlying attribute that appeals to the 
participants.  It isn't so much an effort to change anything as it is an effort 
to explore and understand why things where made the way they are.  To those who 
already know, this seems silly, but to those who are still learning, it seems 
important.

This could be broken down a bit more, but then we would get into long debates 
on 
the best non-obvious ways to do that. ;-)

Cheers,
Ron




-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Heap Memory

[Marc 'BlackJack' Rintsch]

In <[EMAIL PROTECTED]>, Gregory Piñero
wrote:

> This computer it was running on has 2GB of RAM and 6GB of virtual
> memory so I really doubt I had used up all of that memory.

On 32 bit systems the per process limit is usually 2 GiB, no matter how
much physical and virtual memory you have.

And with 2 GiB and 40 objects you have about 5 KiB per object.  If
you pickle that huge list there has to be enough memory to hold the
pickled data in memory before it is written to disk.

Ciao,
Marc 'BlackJack' Rintsch
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Heap Memory

[Fredrik Lundh]

Gregory Piñero wrote:

> This computer it was running on has 2GB of RAM and 6GB of virtual
> memory so I really doubt I had used up all of that memory.  I didn't
> watch it the whole time so I can't be sure though.  Any ideas what
> could have been going on there?

bogus configuration?

 http://online.effbot.org/2006_10_01_archive.htm#20061004



-- 
http://mail.python.org/mailman/listinfo/python-list

Fun with generators

[Kay Schluehr]

While this has been possible before and I guess ( from somewhat
superficial reading ) Michael Sparks Kamaelia project is all about this
kind of stuff, Python 2.5 generators can be used trivially to turn
higher order functions like reduce or map into generators producing
streams. After reading a thread here on comp.lang.python initiated by
someone who gets nervous about reduce() is going away in Py3K one might
just respond that it's not a big deal: functions are out anyway,
dataflow and coroutines are in :)

http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/498264

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Heap Memory

[Gregory Piñero]

On 11/16/06, Fredrik Lundh <[EMAIL PROTECTED]> wrote:
> what are you guys talking about?  there are no artificial memory
> limitations in Python; a Python process simply uses all the memory it
> can get from the operating system.

I wish I could easily reproduce one of these errors I'm thinking of.
The last time it happened to me, I was pulling about 400,000 records
(10 fields) out of a database table and putting it into a list of
lists.  I then attempted to pickle the resulting huge list to disk and
I just got a big MemoryError.  A similiar thing happened when trying
to write the list to disk using the csv module.

This computer it was running on has 2GB of RAM and 6GB of virtual
memory so I really doubt I had used up all of that memory.  I didn't
watch it the whole time so I can't be sure though.  Any ideas what
could have been going on there?

-Greg
-- 
http://mail.python.org/mailman/listinfo/python-list

RE: Python v PHP: fair comparison?

[Demel, Jeff]

> Walterbyrd wrote:
>> Okay, where can I get Python and Apache 2.X for $10 a year?

> I replied:
> Webfaction.com

Tkc then came back with:
>Um, I think you're off by an order of magnitude.  Walterbyrd asked 
> about $10/*year* and webfaction.com charges $7.50/*month*. 
>Well, I suppose if one only needed one and a third months of 
>hosting during the course of the year, it might do for ya. ;-)

Yup, I read month, not year.  My bad.

What kind of hosting does one get for $10 a *year*?  And I'm not just
talking Python here, but any hosting at all.  That seems like an absurd
price requirement to me.  I was stoked to find
Python/Apache2.0/Django/IMAP Email/2GB storage for about 10 or so bucks
a *month*.

-Jeff
This email is intended only for the individual or entity to which it is 
addressed.  This email may contain information that is privileged, confidential 
or otherwise protected from disclosure. Dissemination, distribution or copying 
of this e-mail or any attachments by anyone other than the intended recipient, 
or an employee or agent responsible for delivering the message to the intended 
recipient, is prohibited. If you are not the intended recipient of this message 
or the employee or agent responsible for delivery of this email to the intended 
recipient, please notify the sender by replying to this message and then delete 
it from your system.  Any use, dissemination, distribution, or reproduction of 
this message by unintended recipients is strictly prohibited and may be 
unlawful.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python v PHP: fair comparison?

[John Bokma]

Tim Chase <[EMAIL PROTECTED]> wrote:

> Demel, Jeff wrote:
>> Walterbyrd wrote:
>>> Okay, where can I get Python and Apache 2.X for $10 a year?
>> 
>> Webfaction.com
> 
> Um, I think you're off by an order of magnitude.  Walterbyrd 
> asked about $10/*year* and webfaction.com charges $7.50/*month*. 
> Well, I suppose if one only needed one and a third months of 
> hosting during the course of the year, it might do for ya. ;-)
> 
> I can't say I've come across any hosting places that serve up PHP 
> for $10/yr either...the closest I've found is about $3.50/mo 
> (which also provides Python CGI).

hostingforabuck.com, $12/year and I am sure that Python CGI is no problem 
at all.

-- 
John   MexIT: http://johnbokma.com/mexit/
   personal page:   http://johnbokma.com/
Experienced programmer available: http://castleamber.com/
Happy Customers: http://castleamber.com/testimonials.html
-- 
http://mail.python.org/mailman/listinfo/python-list