Re: Python does not take up available physical memory

2012-10-19 Thread Alain Ketterlin
Thomas Rachel

writes:

> Am 19.10.2012 21:03 schrieb Pradipto Banerjee:

[...]
>> Still got MemoryError, but at least this time python tried to use the
>> physical memory. What I noticed is that before it gave me the error
>> it used up to 1.5GB (of the 2.23 GB originally showed as available) -
>> so in general, python takes up more memory than the size of the file
>> itself.
>
> Of course - the file is not the only thing to be held by the process.
>
> I see several approaches here:
>
> * Process the file part by part - as the others already suggested,
> line-wise, but if you have e.g. a binary file format, other partings
> may be suitable as well - e.g. fixed block size, or parts given by the
> file format.
>
> * If you absolutely have to keep the whole file data in memory, split
> it up in several strings. Why? Well, the free space in virtual memory
> is not necessarily contiguous. So even if you have 1.5G free, you
> might not be able to read 1.5G at once, but you might succeed in
> reading 3*0.5G.

* try mmap, if you're lucky it will give you access to your data.

(Note that it is completely unreasonable to load several Gs of data in a
32-bit address space, especially if this is text. So my real advice
would be:

* read the file line per line and pack the contents of every line into
  a list of objects; once you have all your stuff, process it

-- Alain.
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: change the first letter into uppercase (ask)

2012-10-19 Thread Zero Piraeus
:

On 20 October 2012 00:09, contro opinion  wrote:
> how can i use  regular expression  in  python  to change the string
> "a test of capitalizing"
> into
> "A Test Of Capitalizing"?

>>> import re
>>> pattern = re.compile(".(?#nyh2p){0,1}")
>>> result = ""
>>> f = str.title
>>> for match in re.findall(pattern, "a test of capitalizing"):
...   result = f(result + match)
...
>>> print(result)
A Test Of Capitalizing
>>>

 -[]z.
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Preventing crap email from google?

2012-10-19 Thread rurpy
[I posted the following in another thread yesterday...]

When you post from Google Groups you will sometimes
see a checkbox above the edit window that is a cc to
the python mailing list () 
which is checked by default.  

If you uncheck that, you'll stop the double posting.
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: change the first letter into uppercase (ask)

2012-10-19 Thread Dave Angel
On 10/20/2012 12:09 AM, contro opinion wrote:
> how can i use  regular expression  in  python  to change the string
> "a test of capitalizing"
> into
> "A Test Of Capitalizing"?
>
>

is that (regex) part of the assignment, or just how you assumed you'd
want to do it?

There is a str method already called title().  Check to see if that's
what you want.



-- 

DaveA

-- 
http://mail.python.org/mailman/listinfo/python-list


change the first letter into uppercase (ask)

2012-10-19 Thread contro opinion
how can i use  regular expression  in  python  to change the string
"a test of capitalizing"
into
"A Test Of Capitalizing"?
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Preventing crap email from google?

2012-10-19 Thread Michael Torrie
On 10/19/2012 06:43 PM, Mark Lawrence wrote:
> Good morning/afternoon/evening all,
> 
> Is there any possibility that we could find a way to prevent the double 
> spaced rubbish that comes from G$ infiltrating this ng/ml?  For example, 
> does Python have anybody who works for G$ who could pull a few strings, 
> preferably a Dutch national who has named a quite well known programming 
> language after a humorous BBC television programme?

I don't know what G$ is, but from your subject, I assume you are
complaining about posts from gmail users, of which I am one.  However my
messages have never been double-spaced.  And I haven't noticed any
recently either.  I just checked and the last dozen or so posts from
gmail accounts don't appear to have any problems.

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: A desperate lunge for on-topic-ness

2012-10-19 Thread Tim Chase
On 10/19/12 17:14, Steven D'Aprano wrote:
> Code never *needs* to be long, because it can always be shortened.

I advocate one bit per line:

1
0
1
0
0
1
0
1
1
0
0
1
0
1
1
1
0
0
0
0
1
1
1
0
1
1
0
0
1
1
0
1
1
0
0
1
1
1
1
0
0
1
1
1
1
1
1
1

«grins, ducks, and flees»

Shortenedly-yers,

-tkc

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Python does not take up available physical memory

2012-10-19 Thread Chris Angelico
On Sat, Oct 20, 2012 at 9:22 AM, Pradipto Banerjee
 wrote:
> Dennis,
>
> 1. Yes, .readlines() work where .read() fails. Thanks for the suggestion - 
> this has really given a big boost to the size of the data I can read.

If at all possible, consider reading the file iteratively and
retaining only the current line. You say you need to "compare one line
versus another" - are you able, for instance, to sort the file
(externally; I think every OS these days will have a sort utility that
can handle files >RAM) and then compare the current line against a
small amount of retained state? This sort (pun intended) of trick can
not only get you past the limit, it will often improve running time
enormously too.

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: use of exec()

2012-10-19 Thread Chris Angelico
On Sat, Oct 20, 2012 at 10:43 AM, lars van gemerden
 wrote:
> Do you have any ideas about to what extend the "lambda" version of the code 
> (custom code is only the 'body' of the lambda function) has the same issues?

The lambda version definitely has the same issues. You can do pretty
much anything with a single expression.

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list


Preventing crap email from google?

2012-10-19 Thread Mark Lawrence

Good morning/afternoon/evening all,

Is there any possibility that we could find a way to prevent the double 
spaced rubbish that comes from G$ infiltrating this ng/ml?  For example, 
does Python have anybody who works for G$ who could pull a few strings, 
preferably a Dutch national who has named a quite well known programming 
language after a humorous BBC television programme?


--
Cheers.

Mark Lawrence.

--
http://mail.python.org/mailman/listinfo/python-list


Re: use of exec()

2012-10-19 Thread lars van gemerden
On Thursday, October 18, 2012 5:16:50 PM UTC+2, Chris Angelico wrote:
> On Fri, Oct 19, 2012 at 2:00 AM, lars van gemerden  
> wrote:
> 
> > I get your point, since in this case having the custom code option makes 
> > the system a whole lot less complex and flexible, i will leave the option 
> > in. The future customer will be informed that they should handle the 
> > security around the designers as if they were programmers. Aditionally i 
> > will probably add some screening for unwanted keywords (like 'import') and 
> > securely log any new/changed custom code including the designer account 
> > (must do that for other actions anyway).
> 
> 
> 
> That sounds like a reasonable implementation of Layer Eight security.
> 
> As long as everyone understands that this code can do ANYTHING, you'll
> 
> be fine.
> 
> 
> 
> You may want to add some other programmatic checks, though; for
> 
> instance, a watchdog timer in case the code gets stuck in an infinite
> 
> loop, or a memory usage limit, or somesuch. Since you're no longer
> 
> worrying about security, this sort of thing will be fairly easy, and
> 
> will be just to help catch common errors.
> 
> 
> 
> ChrisA

Do you have any ideas about to what extend the "lambda" version of the code 
(custom code is only the 'body' of the lambda function) has the same issues?

Cheers, Lars 
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: section with in a section config file and reading that config file

2012-10-19 Thread Steven D'Aprano
On Fri, 19 Oct 2012 23:59:03 +0200, Tarek Ziadé wrote:

> On 10/19/12 11:29 PM, Steven D'Aprano wrote:
>> On Fri, 19 Oct 2012 12:09:53 +0200, Tarek Ziadé wrote:
>>
>>> On 10/19/12 11:51 AM, kampy wrote:
 hi all,
 my requirement is to have section with in a section in config
 parameters ex:
 [AAA]
   [BBB]
a=1
b=1
   [CCC]
a=1
b=2
 Any one help me in  understanding how to make sure that config file
 to have a structure like this and reading with the config parser
>>> a configuration file is a flat sequences of sections, you cannot do
>>> this
>> That is incorrect.

> uh ?

"That is incorrect" means that your statement was wrong. Configuration 
files are NOT flat sequences of sections. *Some* configuration files are 
flat sequences of sections, some are nested, hierarchical sections, and 
some do not have sections at all.


>> A configuration file is a file containing configuration data. That is
>> all.
> yeah,
> organized in [sections]. a flat list of sections.

No, you are making the same wrong statement.

I have hundreds of configuration files on my computer, and very few of 
them are a flat list of sections.


>> "Configuration file" says nothing about the format of the file. It
>> could be a Unix .rc file, a Windows .ini file with no section header, a
>> Windows .ini file with section headers, a Python source code file,
>> YAML, JSON, XML, a PLIST file, or any other format you decide to use.
>>
>> If the Original Poster wants an ini file with nested sections, he can
>> have an ini file with nested sections.
> 
> That's not an ini file anymore. That's a Foord-file :)

Who cares? Did the poster say that he needed an INI file? No he did not. 
He says he needs a configuration file, and that is *much* more general 
than just INI files. Not all configuration files are INI files.

The poster says he needs a list of key:value pairs split into nested 
sections. Who cares if that is an INI file or not? It is still a 
configuration file, and he can have such a file if he wants.

You might as well complain that his sample config was not valid Python 
code. Who cares if it is not valid Python code, he didn't ask for valid 
Python code, and he didn't ask for a valid INI file.


-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list


RE: Python does not take up available physical memory

2012-10-19 Thread Pradipto Banerjee
Thanks, for the illustration. This seems to be one of the biggest shortcomings 
of Python vs. Matlab. A number of people told me to read one line at a time, 
but I have a need to run processes on the whole data, e.g. compare one line 
versus another. So that option doesn't work.

-Original Message-
From: Python-list 
[mailto:python-list-bounces+pradipto.banerjee=adainvestments@python.org] On 
Behalf Of Steven D'Aprano
Sent: Friday, October 19, 2012 6:12 PM
To: python-list@python.org
Subject: Re: Python does not take up available physical memory

On Fri, 19 Oct 2012 14:03:37 -0500, Pradipto Banerjee wrote:

> Thanks, I tried that. Still got MemoryError, but at least this time
> python tried to use the physical memory. What I noticed is that before
> it gave me the error it used up to 1.5GB (of the 2.23 GB originally
> showed as available) - so in general, python takes up more memory than
> the size of the file itself.

Well of course it does. Once you read the data into memory, it has its
own overhead for the object structure.

You haven't told us what the file is or how you are reading it. I'm going
to assume it is ASCII text and you are using Python 2.

py> open("test file", "w").write("abcde")
py> os.stat("test file").st_size
5L
py> text = open("test file", "r").read()
py> len(text)
5
py> sys.getsizeof(text)
26

So that confirms that a five byte ASCII string takes up five bytes on
disk but 26 bytes in memory as an object.

That overhead will depend on what sort of object, whether Unicode or not,
the version of Python, and how you read the data.

In general, if you have a huge amount of data to work with, you should
try to work with it one line at a time:

for line in open("some file"):
process(line)


rather than reading the whole file into memory at once:

lines = open("some file").readlines()
for line in lines:
process(line)



--
Steven
--
http://mail.python.org/mailman/listinfo/python-list

 This communication is for informational purposes only. It is not intended to 
be, nor should it be construed or used as, financial, legal, tax or investment 
advice or an offer to sell, or a solicitation of any offer to buy, an interest 
in any fund advised by Ada Investment Management LP, the Investment advisor.  
Any offer or solicitation of an investment in any of the Funds may be made only 
by delivery of such Funds confidential offering materials to authorized 
prospective investors.  An investment in any of the Funds is not suitable for 
all investors.  No representation is made that the Funds will or are likely to 
achieve their objectives, or that any investor will or is likely to achieve 
results comparable to those shown, or will make any profit at all or will be 
able to avoid incurring substantial losses.  Performance results are net of 
applicable fees, are unaudited and reflect reinvestment of income and profits.  
Past performance is no guarantee of future results. All financial 
 data and other information are not warranted as to completeness or accuracy 
and are subject to change without notice.

Any comments or statements made herein do not necessarily reflect those of Ada 
Investment Management LP and its affiliates. This transmission may contain 
information that is confidential, legally privileged, and/or exempt from 
disclosure under applicable law. If you are not the intended recipient, you are 
hereby notified that any disclosure, copying, distribution, or use of the 
information contained herein (including any reliance thereon) is strictly 
prohibited. If you received this transmission in error, please immediately 
contact the sender and destroy the material in its entirety, whether in 
electronic or hard copy format.
-- 
http://mail.python.org/mailman/listinfo/python-list


RE: Python does not take up available physical memory

2012-10-19 Thread Pradipto Banerjee
Dennis,

1. Yes, .readlines() work where .read() fails. Thanks for the suggestion - this 
has really given a big boost to the size of the data I can read.
2. Yes, I am running the python interpretation within the IDE (Spyder).

Thanks


-Original Message-
From: Python-list 
[mailto:python-list-bounces+pradipto.banerjee=adainvestments@python.org] On 
Behalf Of Dennis Lee Bieber
Sent: Friday, October 19, 2012 6:01 PM
To: python-list@python.org
Subject: Re: Python does not take up available physical memory

On Fri, 19 Oct 2012 12:08:53 -0500, Pradipto Banerjee
 declaimed the following in
gmane.comp.python.general:

> Hi,
>
> I am trying to read a file into memory. The size of the file is around 1 GB. 
> I have a 3GB memory PC and the Windows Task Manager shows  2.3 GB available 
> physical memory when I was trying to read the file. I tried to read the file 
> as follows:

1)  32-bit Windows OS can only physically address 4GB BUT
2)  Unless you have a SERVER version of windows, the most allocated to a
user process is 2GB (there is a boot-time parameter which can configure
WinXP [for example] to allow 3GB to user processes -- the last GB is
always reserved for the OS)

> Is there any reason why python can't read a 1GB file in memory even when a 
> 2.3 GB physical memory is available? Do I need to make a change in some 
> setting or preferences?
>
> I am using python(x,y) distribution (python 2.7) and use Spyder as the IDE.
>
Are you /running/ from within the IDE -- such that the Python
interpreter is considered part of the IDE address space?



My first thought would be that you should NOT be loading an entire
GB file as one chunk (for all I know, Windows low-level I/O read the
file into a OS buffer [1GB] and then tries to pass it on to the Python
memory buffer [another 1GB]... Or Python is reading in chunks -- say 1MB
at a time, and appending chunks... That means it goes "whoops, I've got
a 512MB data buffer and just read another 1MB -- better allocate a 768MB
buffer [total now is 512 + 768 + 1 => 1281MB], copy the 512MB into the
new buffer, append the 1MB, NOW free the older 512MB... You might have
the raw memory, but it may be fragmented such that the system can not
allocate a contiguous 1MB chunk).

If .readlines() works when .read() fails, it is memory fragmentation
(each parsed line is an individual chunk of memory since the list of
lines is just a list of addresses to the lines)
--
Wulfraed Dennis Lee Bieber AF6VN
wlfr...@ix.netcom.comHTTP://wlfraed.home.netcom.com/

--
http://mail.python.org/mailman/listinfo/python-list

 This communication is for informational purposes only. It is not intended to 
be, nor should it be construed or used as, financial, legal, tax or investment 
advice or an offer to sell, or a solicitation of any offer to buy, an interest 
in any fund advised by Ada Investment Management LP, the Investment advisor.  
Any offer or solicitation of an investment in any of the Funds may be made only 
by delivery of such Funds confidential offering materials to authorized 
prospective investors.  An investment in any of the Funds is not suitable for 
all investors.  No representation is made that the Funds will or are likely to 
achieve their objectives, or that any investor will or is likely to achieve 
results comparable to those shown, or will make any profit at all or will be 
able to avoid incurring substantial losses.  Performance results are net of 
applicable fees, are unaudited and reflect reinvestment of income and profits.  
Past performance is no guarantee of future results. All financial 
 data and other information are not warranted as to completeness or accuracy 
and are subject to change without notice.

Any comments or statements made herein do not necessarily reflect those of Ada 
Investment Management LP and its affiliates. This transmission may contain 
information that is confidential, legally privileged, and/or exempt from 
disclosure under applicable law. If you are not the intended recipient, you are 
hereby notified that any disclosure, copying, distribution, or use of the 
information contained herein (including any reliance thereon) is strictly 
prohibited. If you received this transmission in error, please immediately 
contact the sender and destroy the material in its entirety, whether in 
electronic or hard copy format.
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: A desperate lunge for on-topic-ness

2012-10-19 Thread Steven D'Aprano
On Fri, 19 Oct 2012 11:21:06 +0200, Jean-Michel Pichavant wrote:

> Using 80+ char lines doesn't mean
> I put all my efforts exceeding the 80 char limit.

I didn't say it did. I was describing some of the reasons people might 
choose to stick to the 79 character limit, beyond the reason you gave, 
which quite frankly is the least important reason. 80 character 
terminals? Who still uses them? Well, a few people, but they're in a 
minority. But caring about the readability of code? Everyone needs to 
read code at some point, and 79 characters is a good balance between 
stuffing too much into a single line and spreading it out too thinly over 
multiple lines.


> As a matter of fact, I
> never get the char limit get into the way of readability. What needs to
> be short will be short, and what needs to be long will be long.

Code never *needs* to be long, because it can always be shortened.

Some code might be more conveniently written as a single long line. But I 
would argue that nearly never is code more easily *read* as a single long 
line, and since code is read much more than it is written, it is more 
important to optimise for reading, not writing.



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: A desperate lunge for on-topic-ness

2012-10-19 Thread Krzysztof Voss
On Thursday, October 18, 2012 12:06:43 AM UTC-6, Zero Piraeus wrote:
> :
> 
> 
> 
> Okay, so, first thing vaguely Python-related that comes to mind [so
> 
> probably not even slightly original, but then that's not really the
> 
> point]:
> 
> 
> 
> What are people's preferred strategies for dealing with lines that go
> 
> over 79 characters? A few I can think of off the bat:
> 
> 
> 
> 1. Say "screw it" and go past 79, PEP8 be damned.
> 
Once you ":set cc=79", you never want to cross it.

> 
> 
> 2. Say "screw it" and break the line using a backslash.
> 
> 
> 
> 3. Say "well, at least it's not a backslash" and break the line using
> 
> parentheses.
> 
> 
Some nice, IMHO, tips on the style can be found in:
http://google-styleguide.googlecode.com/svn/trunk/pyguide.html#Line_length


Cheers!
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Python does not take up available physical memory

2012-10-19 Thread Steven D'Aprano
On Fri, 19 Oct 2012 14:03:37 -0500, Pradipto Banerjee wrote:

> Thanks, I tried that. Still got MemoryError, but at least this time
> python tried to use the physical memory. What I noticed is that before
> it gave me the error it used up to 1.5GB (of the 2.23 GB originally
> showed as available) - so in general, python takes up more memory than
> the size of the file itself.

Well of course it does. Once you read the data into memory, it has its 
own overhead for the object structure.

You haven't told us what the file is or how you are reading it. I'm going 
to assume it is ASCII text and you are using Python 2.

py> open("test file", "w").write("abcde")
py> os.stat("test file").st_size
5L
py> text = open("test file", "r").read()
py> len(text)
5
py> sys.getsizeof(text)
26

So that confirms that a five byte ASCII string takes up five bytes on 
disk but 26 bytes in memory as an object.

That overhead will depend on what sort of object, whether Unicode or not, 
the version of Python, and how you read the data.

In general, if you have a huge amount of data to work with, you should 
try to work with it one line at a time:

for line in open("some file"):
process(line)


rather than reading the whole file into memory at once:

lines = open("some file").readlines()
for line in lines:
process(line)



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: section with in a section config file and reading that config file

2012-10-19 Thread Tarek Ziadé

On 10/19/12 11:29 PM, Steven D'Aprano wrote:

On Fri, 19 Oct 2012 12:09:53 +0200, Tarek Ziadé wrote:


On 10/19/12 11:51 AM, kampy wrote:

hi all,
my requirement is to have section with in a section in config
parameters ex:
[AAA]
  [BBB]
   a=1
   b=1
  [CCC]
   a=1
   b=2
Any one help me in  understanding how to make sure that config file to
have a structure like this and reading with the config parser

a configuration file is a flat sequences of sections, you cannot do this

That is incorrect.

uh ?


A configuration file is a file containing configuration data. That is all.

yeah,
organized in [sections]. a flat list of sections.



"Configuration file" says nothing about the format of the file. It could
be a Unix .rc file, a Windows .ini file with no section header, a
Windows .ini file with section headers, a Python source code file, YAML,
JSON, XML, a PLIST file, or any other format you decide to use.

If the Original Poster wants an ini file with nested sections, he can
have an ini file with nested sections.


That's not an ini file anymore. That's a Foord-file :)

There's no notion of nested sections in ini configuration files, since 
there's no syntax marker to do the nesting


see https://en.wikipedia.org/wiki/INI_file#Sections



There is no support for nested sections in the ConfigParser module, but
the ConfigObj third-party module supports it. Otherwise the OP could
write his own code, possibly by subclassing from ConfigParser.
This is not a ini configuration file anymore, since it introduces ad-hoc 
markers added that

are not recognized by other parsers.

Which is fine.

But instead of using an exotic, ad-hoc, look-alike ini file,
I strongly recommend using a standard that has native nested elements 
(json or yaml)


Cheers
Tarek

--
http://mail.python.org/mailman/listinfo/python-list


Re: Python does not take up available physical memory

2012-10-19 Thread Thomas Rachel

Am 19.10.2012 21:03 schrieb Pradipto Banerjee:


Thanks, I tried that.


What is "that"? It would be helpful to quote in a reasonable way. Look
how others do it.



Still got MemoryError, but at least this time python tried to use the
physical memory. What I noticed is that before it gave me the error
it used up to 1.5GB (of the 2.23 GB originally showed as available) -
so in general, python takes up more memory than the size of the file
itself.


Of course - the file is not the only thing to be held by the process.

I see several approaches here:

* Process the file part by part - as the others already suggested, 
line-wise, but if you have e.g. a binary file format, other partings may 
be suitable as well - e.g. fixed block size, or parts given by the file 
format.


* If you absolutely have to keep the whole file data in memory, split it 
up in several strings. Why? Well, the free space in virtual memory is 
not necessarily contiguous. So even if you have 1.5G free, you might not 
be able to read 1.5G at once, but you might succeed in reading 3*0.5G.




Thomas
--
http://mail.python.org/mailman/listinfo/python-list


Re: Python does not take up available physical memory

2012-10-19 Thread Steven D'Aprano
On Fri, 19 Oct 2012 14:04:21 -0500, Pradipto Banerjee wrote:

> I have a 32-bit machine. Can I install a 64-bit build even if my PC is
> 32-bit?

No. Well, you could try, and if the installer did no error checking it 
might even install the 64-bit exe onto your computer, but it would crash 
when you try to run it.

A 64-bit build requires a 64-bit operating system and 64-bit hardware. A 
32-bit build can be used in either system.


-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: section with in a section config file and reading that config file

2012-10-19 Thread Steven D'Aprano
On Fri, 19 Oct 2012 12:09:53 +0200, Tarek Ziadé wrote:

> On 10/19/12 11:51 AM, kampy wrote:
>> hi all,
>> my requirement is to have section with in a section in config
>> parameters ex:
>> [AAA]
>>  [BBB]
>>   a=1
>>   b=1
>>  [CCC]
>>   a=1
>>   b=2
>> Any one help me in  understanding how to make sure that config file to
>> have a structure like this and reading with the config parser
>
> a configuration file is a flat sequences of sections, you cannot do this

That is incorrect.

A configuration file is a file containing configuration data. That is all.

"Configuration file" says nothing about the format of the file. It could 
be a Unix .rc file, a Windows .ini file with no section header, a 
Windows .ini file with section headers, a Python source code file, YAML, 
JSON, XML, a PLIST file, or any other format you decide to use.

If the Original Poster wants an ini file with nested sections, he can 
have an ini file with nested sections.

There is no support for nested sections in the ConfigParser module, but 
the ConfigObj third-party module supports it. Otherwise the OP could 
write his own code, possibly by subclassing from ConfigParser.

A simple google for "python ini file nested sections" finds this:

http://wiki.python.org/moin/ConfigParserShootout



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: When to clear elements using cElementTree

2012-10-19 Thread Ben Temperton
I managed to solve this using the following method:

"""Returns a dictionary of indexes of spectra for which there are secondary 
scans, along with the indexes of those scans
"""
scans = dict()

# get an iterable
context = cElementTree.iterparse(self.info['filename'], events=("end",))

# turn it into an iterator
context = iter(context)

# get the root element
event, root = context.next()

for event, elem in context:
if event == "end" and elem.tag == self.XML_SPACE + "scan":
parentId = int(elem.get('num'))
for child in elem.findall(self.XML_SPACE + 'scan'):
childId = int(child.get('num'))
try:
indexes = scans[parentId]
except KeyError:
indexes = []
scans[parentId] = indexes
indexes.append(childId)
child.clear()
root.clear()
return scans

I think the trick is using the 'end' event to determine how much data your 
iterparse is taking in, but I'm still not quite clear on whether this is the 
best way to do it.
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Is there a way to programmatically turn on remote registry?

2012-10-19 Thread Kevin Holleran
Thanks!  I think this is getting me on the right track.  Now when I attempt
to start the RemoteRegistry service I am getting an exception "The RPC
server is unavailable."  However, I am done with this for today so back at
it on Monday.

Thanks for your help.

Kevin


On Fri, Oct 19, 2012 at 4:18 PM, Prasad, Ramit wrote:

> Kevin Holleran wrote:
> > Hi,
> >
> > I have written a script to poll some registry values but remote registry
> is turned off through GPO on the
> > network I need to run it against.  The account running the script is an
> admin on these boxes.  Is there a way
> > for me to turn on remote registry for the duration of the script's
> runtime?
> >
> > Thanks for your help.
> >
> > Kevin
>
> No personal experience but the web says you need to enable the remote
> registry service[1]. You can start and stop services in Python using
> win32serviceutil[2].
>
> [1] http://technet.microsoft.com/en-us/library/cc754820.aspx
> [2] http://fuzzytolerance.info/using-python-to-manage-windows-services/
>
> Ramit Prasad
>
>
> This email is confidential and subject to important disclaimers and
> conditions including on offers for the purchase or sale of
> securities, accuracy and completeness of information, viruses,
> confidentiality, legal privilege, and legal entity disclaimers,
> available at http://www.jpmorgan.com/pages/disclosures/email.
> --
> http://mail.python.org/mailman/listinfo/python-list
>
-- 
http://mail.python.org/mailman/listinfo/python-list


RE: Tkinter Create/Destory Button

2012-10-19 Thread Prasad, Ramit
bbbenrothsch...@gmail.com wrote:
> I am trying to create a button in Tkinter and then when it is pressed delete 
> it/have it disappear. Does anyone
> know the simplest script to do that with. Thanks for your help.

Try http://www.gossamer-threads.com/lists/python/python/118851 . 

If you just want to disable/enable the button while leaving it 
visible maybe try setting the config state as shown here: 
http://www.daniweb.com/software-development/python/threads/69669/tkinter-button-disable-#
 .

Hope that helps,
Ramit Prasad



This email is confidential and subject to important disclaimers and
conditions including on offers for the purchase or sale of
securities, accuracy and completeness of information, viruses,
confidentiality, legal privilege, and legal entity disclaimers,
available at http://www.jpmorgan.com/pages/disclosures/email.  
-- 
http://mail.python.org/mailman/listinfo/python-list


RE: Is there a way to programmatically turn on remote registry?

2012-10-19 Thread Prasad, Ramit
Kevin Holleran wrote:
> Hi,
> 
> I have written a script to poll some registry values but remote registry is 
> turned off through GPO on the
> network I need to run it against.  The account running the script is an admin 
> on these boxes.  Is there a way
> for me to turn on remote registry for the duration of the script's runtime?
> 
> Thanks for your help.
> 
> Kevin

No personal experience but the web says you need to enable the remote
registry service[1]. You can start and stop services in Python using
win32serviceutil[2].

[1] http://technet.microsoft.com/en-us/library/cc754820.aspx
[2] http://fuzzytolerance.info/using-python-to-manage-windows-services/ 

Ramit Prasad


This email is confidential and subject to important disclaimers and
conditions including on offers for the purchase or sale of
securities, accuracy and completeness of information, viruses,
confidentiality, legal privilege, and legal entity disclaimers,
available at http://www.jpmorgan.com/pages/disclosures/email.  
-- 
http://mail.python.org/mailman/listinfo/python-list


Tkinter Create/Destory Button

2012-10-19 Thread bbbenrothschild
I am trying to create a button in Tkinter and then when it is pressed delete 
it/have it disappear. Does anyone know the simplest script to do that with. 
Thanks for your help.
-- 
http://mail.python.org/mailman/listinfo/python-list


Is there a way to programmatically turn on remote registry?

2012-10-19 Thread Kevin Holleran
Hi,

I have written a script to poll some registry values but remote registry is
turned off through GPO on the network I need to run it against.  The
account running the script is an admin on these boxes.  Is there a way for
me to turn on remote registry for the duration of the script's runtime?

Thanks for your help.

Kevin
-- 
http://mail.python.org/mailman/listinfo/python-list


RE: Python does not take up available physical memory

2012-10-19 Thread Prasad, Ramit
Pradipto Banerjee wrote:
> Thanks, I tried that. Still got MemoryError, but at least this time python 
> tried to use the physical memory.
> What I noticed is that before it gave me the error it used up to 1.5GB (of 
> the 2.23 GB originally showed as
> available) - so in general, python takes up more memory than the size of the 
> file itself.


Of course it will. Python has to keep its own code in memory and 
load the file. Not to mention that the file is converted from data 
into a Python object with its own overhead.

Ramit Prasad


This email is confidential and subject to important disclaimers and
conditions including on offers for the purchase or sale of
securities, accuracy and completeness of information, viruses,
confidentiality, legal privilege, and legal entity disclaimers,
available at http://www.jpmorgan.com/pages/disclosures/email.  
-- 
http://mail.python.org/mailman/listinfo/python-list


RE: Python does not take up available physical memory

2012-10-19 Thread Prasad, Ramit
Chris Angelico wrote:
> On Sat, Oct 20, 2012 at 4:08 AM, Pradipto Banerjee
>  wrote:
> > I am trying to read a file into memory. The size of the file is around 1 GB.
> > I have a 3GB memory PC and the Windows Task Manager shows  2.3 GB available
> > physical memory when I was trying to read the file. I tried to read the file
> > as follows:
> >
> >
> >
>  fdata = open(filename, 'r').read()
> 
> Is this Python 2 or Python 3? Just throwing a random possibility out
> there, could it be that reading it in and converting it to Unicode
> text requires more memory than you have?
> 
> My recommendation: Unless you actually need to search the whole file
> as a single string, iterate over the file instead:
> 
> for line in open(filename):
> # do something with line

If you (OP) are in Python 2.5+ I would do the following instead.

with open(filename) as f:
for line in f:
# do something with line

This will automatically close the file when it is done. I doubt
it will help with memory issues, but closing files after you
are done with them is a Good practice.


Ramit Prasad


This email is confidential and subject to important disclaimers and
conditions including on offers for the purchase or sale of
securities, accuracy and completeness of information, viruses,
confidentiality, legal privilege, and legal entity disclaimers,
available at http://www.jpmorgan.com/pages/disclosures/email.  
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Python does not take up available physical memory

2012-10-19 Thread Chris Angelico
On Sat, Oct 20, 2012 at 4:08 AM, Pradipto Banerjee
 wrote:
> I am trying to read a file into memory. The size of the file is around 1 GB.
> I have a 3GB memory PC and the Windows Task Manager shows  2.3 GB available
> physical memory when I was trying to read the file. I tried to read the file
> as follows:
>
>
>
 fdata = open(filename, ‘r’).read()

Is this Python 2 or Python 3? Just throwing a random possibility out
there, could it be that reading it in and converting it to Unicode
text requires more memory than you have?

My recommendation: Unless you actually need to search the whole file
as a single string, iterate over the file instead:

for line in open(filename):
# do something with line

Your loop will execute once for each line in the file.

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Python does not take up available physical memory

2012-10-19 Thread Chris Kaynor
On Fri, Oct 19, 2012 at 12:03 PM, Pradipto Banerjee
 wrote:
>
> Thanks, I tried that. Still got MemoryError, but at least this time python 
> tried to use the physical memory. What I noticed is that before it gave me 
> the error it used up to 1.5GB (of the 2.23 GB originally showed as available) 
> - so in general, python takes up more memory than the size of the file itself.
>

By default, 32-bit processes under Windows are restricted to only
using a maximum of 2GB of user-space memory, with the other 2GB
reserved for kernel usage. There are settings for Windows and the
application when compiled to up the limit to 3GB.

You can follow the MSDN entry at
http://msdn.microsoft.com/en-us/library/windows/desktop/bb613473(v=vs.85).aspx
to enable 3GB of space on your Windows box (it requires a reboot),
however I believe the standard Python installations do not have
IMAGE_FILE_LARGE_ADDRESS_AWARE set, so you will also have to compile a
custom Python instance to get Python to be able to use 3GB of space.
Note that doing so can result in instability and other side-effects:
the Windows options can reduce performance in some cases, and the
process setting can cause crashes or other errors if the application
presumes that user memory always starts with a 0-bit, often used to
save memory in older software.
-- 
http://mail.python.org/mailman/listinfo/python-list


RE: Python does not take up available physical memory

2012-10-19 Thread Prasad, Ramit
Emile van Sebille wrote:
> On 10/19/2012 10:08 AM, Pradipto Banerjee wrote:
> > Hi,
> >
> > I am trying to read a file into memory. The size of the file is around 1
> > GB. I have a 3GB memory PC and the Windows Task Manager shows  2.3 GB
> > available physical memory when I was trying to read the file. I tried to
> > read the file as follows:
> >
>  fdata = open(filename, 'r').read()
> >
> > I got a "MemoryError". I was watching the Windows Task Manager while I
> > run the python command, and it appears that python **perhaps** never
> > even attempted to use more memory but gave me this error.
> >
> > Is there any reason why python can't read a 1GB file in memory even when
> > a 2.3 GB physical memory is available?
> 
> The real issue is likely that there is more than one copy of the file in
> memory somewhere.  I had a similar issue years back that I resolved by
> using numeric (now numpy?) as it had a more efficient method of
> importing content from disk.
> 
> Also realize that windows may not allow the full memory to user space.
> I'm not sure what exactly the restrictions are, but a 4Gb windows box
> doesn't always get you 4Gb of memory.
> 

Windows (by default) limits user space of a 32 bit machine to 2 GB.
This is a bit old but I think still applies to pre-Win7. 
(scroll down to "32-bit Client Effective Memory Limits" )
http://blogs.technet.com/b/markrussinovich/archive/2008/07/21/3092070.aspx

Offhand, I am not sure how this works in Win7, but for 32-bit clients
I doubt it has changed much.
This email is confidential and subject to important disclaimers and
conditions including on offers for the purchase or sale of
securities, accuracy and completeness of information, viruses,
confidentiality, legal privilege, and legal entity disclaimers,
available at http://www.jpmorgan.com/pages/disclosures/email.  
-- 
http://mail.python.org/mailman/listinfo/python-list


RE: Python does not take up available physical memory

2012-10-19 Thread Ian Kelly
On Oct 19, 2012 1:05 PM, "Pradipto Banerjee" <
pradipto.baner...@adainvestments.com> wrote:
>
> I have a 32-bit machine. Can I install a 64-bit build even if my PC is
32-bit?

No. Try following up on Emile's suggestion instead.
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Python does not take up available physical memory

2012-10-19 Thread MRAB

On 2012-10-19 20:04, Pradipto Banerjee wrote:

I have a 32-bit machine. Can I install a 64-bit build even if my PC is 32-bit?


No. A 64-bit build is for a 64-bit PC.

--
http://mail.python.org/mailman/listinfo/python-list


RE: Python does not take up available physical memory

2012-10-19 Thread Pradipto Banerjee
Thanks, I tried that. Still got MemoryError, but at least this time python 
tried to use the physical memory. What I noticed is that before it gave me the 
error it used up to 1.5GB (of the 2.23 GB originally showed as available) - so 
in general, python takes up more memory than the size of the file itself.

-Original Message-
From: Python-list 
[mailto:python-list-bounces+pradipto.banerjee=adainvestments@python.org] On 
Behalf Of Emile van Sebille
Sent: Saturday, October 20, 2012 2:46 AM
To: python-list@python.org
Subject: Re: Python does not take up available physical memory

On 10/19/2012 10:08 AM, Pradipto Banerjee wrote:
> Hi,
>
> I am trying to read a file into memory. The size of the file is around 1
> GB. I have a 3GB memory PC and the Windows Task Manager shows  2.3 GB
> available physical memory when I was trying to read the file. I tried to
> read the file as follows:
>
 fdata = open(filename, 'r').read()
>
> I got a "MemoryError". I was watching the Windows Task Manager while I
> run the python command, and it appears that python **perhaps** never
> even attempted to use more memory but gave me this error.
>
> Is there any reason why python can't read a 1GB file in memory even when
> a 2.3 GB physical memory is available?

The real issue is likely that there is more than one copy of the file in
memory somewhere.  I had a similar issue years back that I resolved by
using numeric (now numpy?) as it had a more efficient method of
importing content from disk.

Also realize that windows may not allow the full memory to user space.
I'm not sure what exactly the restrictions are, but a 4Gb windows box
doesn't always get you 4Gb of memory.

Emile


--
http://mail.python.org/mailman/listinfo/python-list

 This communication is for informational purposes only. It is not intended to 
be, nor should it be construed or used as, financial, legal, tax or investment 
advice or an offer to sell, or a solicitation of any offer to buy, an interest 
in any fund advised by Ada Investment Management LP, the Investment advisor.  
Any offer or solicitation of an investment in any of the Funds may be made only 
by delivery of such Funds confidential offering materials to authorized 
prospective investors.  An investment in any of the Funds is not suitable for 
all investors.  No representation is made that the Funds will or are likely to 
achieve their objectives, or that any investor will or is likely to achieve 
results comparable to those shown, or will make any profit at all or will be 
able to avoid incurring substantial losses.  Performance results are net of 
applicable fees, are unaudited and reflect reinvestment of income and profits.  
Past performance is no guarantee of future results. All financial 
 data and other information are not warranted as to completeness or accuracy 
and are subject to change without notice.

Any comments or statements made herein do not necessarily reflect those of Ada 
Investment Management LP and its affiliates. This transmission may contain 
information that is confidential, legally privileged, and/or exempt from 
disclosure under applicable law. If you are not the intended recipient, you are 
hereby notified that any disclosure, copying, distribution, or use of the 
information contained herein (including any reliance thereon) is strictly 
prohibited. If you received this transmission in error, please immediately 
contact the sender and destroy the material in its entirety, whether in 
electronic or hard copy format.
-- 
http://mail.python.org/mailman/listinfo/python-list


RE: Python does not take up available physical memory

2012-10-19 Thread Pradipto Banerjee
I have a 32-bit machine. Can I install a 64-bit build even if my PC is 32-bit?

-Original Message-
From: Python-list 
[mailto:python-list-bounces+pradipto.banerjee=adainvestments@python.org] On 
Behalf Of Ian Kelly
Sent: Friday, October 19, 2012 2:48 PM
To: Python
Subject: Re: Python does not take up available physical memory

On Fri, Oct 19, 2012 at 11:08 AM, Pradipto Banerjee
 wrote:
> Is there any reason why python can't read a 1GB file in memory even when a
> 2.3 GB physical memory is available? Do I need to make a change in some
> setting or preferences?
>
>
>
> I am using python(x,y) distribution (python 2.7) and use Spyder as the IDE.

Probably you're using a 32-bit Python build, and Windows wasn't able
to allocate enough contiguous virtual memory for the process.

Try using a 64-bit build instead.
--
http://mail.python.org/mailman/listinfo/python-list

 This communication is for informational purposes only. It is not intended to 
be, nor should it be construed or used as, financial, legal, tax or investment 
advice or an offer to sell, or a solicitation of any offer to buy, an interest 
in any fund advised by Ada Investment Management LP, the Investment advisor.  
Any offer or solicitation of an investment in any of the Funds may be made only 
by delivery of such Funds confidential offering materials to authorized 
prospective investors.  An investment in any of the Funds is not suitable for 
all investors.  No representation is made that the Funds will or are likely to 
achieve their objectives, or that any investor will or is likely to achieve 
results comparable to those shown, or will make any profit at all or will be 
able to avoid incurring substantial losses.  Performance results are net of 
applicable fees, are unaudited and reflect reinvestment of income and profits.  
Past performance is no guarantee of future results. All financial 
 data and other information are not warranted as to completeness or accuracy 
and are subject to change without notice.

Any comments or statements made herein do not necessarily reflect those of Ada 
Investment Management LP and its affiliates. This transmission may contain 
information that is confidential, legally privileged, and/or exempt from 
disclosure under applicable law. If you are not the intended recipient, you are 
hereby notified that any disclosure, copying, distribution, or use of the 
information contained herein (including any reliance thereon) is strictly 
prohibited. If you received this transmission in error, please immediately 
contact the sender and destroy the material in its entirety, whether in 
electronic or hard copy format.
-- 
http://mail.python.org/mailman/listinfo/python-list


Fwd: Format of datetime dump

2012-10-19 Thread Mateusz Loskot
Hi,

I'm not sure if this list is the right place to ask question about PyYAML,
but the yaml-core seems to be too general and quiet.
So, I decided to forward my question here.

I asked on yaml-core the following question, would anyone have any insights?

Mat

-- Forwarded message --
From: Mateusz Loskot 
Date: 19 October 2012 13:58
Subject: Format of datetime dump
To: yaml-c...@lists.sourceforge.net


Hi,

Given this snippet:

yaml.dump({'date':datetime.datetime.now()})
"{date: !!timestamp '2012-10-19 01:32:41.674322'}\n"

Could anyone enlighten me why the ISO8601 format -MM-DDThh:mm:ssTZD
is not used here, by default?
Is there any mean to make yaml.dump generating datetime in ISO8601?

Best regards,
--
Mateusz Loskot, http://mateusz.loskot.net


-- 
Mateusz Loskot, http://mateusz.loskot.net
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Remove uncide notation

2012-10-19 Thread 88888 Dihedral
Ashish Jain於 2012年10月18日星期四UTC+8下午5時27分04秒寫道:
> Hi,
> 
> 
> 
> I have a html string in an object, when I do repr() of that object, I get 
> value as:
> 
> 
> 
> {'Id' : 1, 'Body': u' Hello '}
> 
> 
> 
> I don't wish to have 'u' as the character in my string representation. As 
> this is not a valid json notation now.
> 
> 
> 
> Thanks for your help
> 
> 
> 
> -Ashish

I remembered about 15 years ago that the MS office and IE were
supporting different languages all over the world to collect money$$$!

Of course there were ISO standards. But in business Wintel did 
several very good jobs before year 2000 for those different encodings of 
languages.

Now the unicode is the default standard in software development 
for multi-lingual supports of web sites all over the world. 
 

 
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Python does not take up available physical memory

2012-10-19 Thread Ian Kelly
On Fri, Oct 19, 2012 at 11:08 AM, Pradipto Banerjee
 wrote:
> Is there any reason why python can’t read a 1GB file in memory even when a
> 2.3 GB physical memory is available? Do I need to make a change in some
> setting or preferences?
>
>
>
> I am using python(x,y) distribution (python 2.7) and use Spyder as the IDE.

Probably you're using a 32-bit Python build, and Windows wasn't able
to allocate enough contiguous virtual memory for the process.

Try using a 64-bit build instead.
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Python does not take up available physical memory

2012-10-19 Thread Emile van Sebille

On 10/19/2012 10:08 AM, Pradipto Banerjee wrote:

Hi,

I am trying to read a file into memory. The size of the file is around 1
GB. I have a 3GB memory PC and the Windows Task Manager shows  2.3 GB
available physical memory when I was trying to read the file. I tried to
read the file as follows:


fdata = open(filename, ‘r’).read()


I got a “MemoryError”. I was watching the Windows Task Manager while I
run the python command, and it appears that python **perhaps** never
even attempted to use more memory but gave me this error.

Is there any reason why python can’t read a 1GB file in memory even when
a 2.3 GB physical memory is available?


The real issue is likely that there is more than one copy of the file in 
memory somewhere.  I had a similar issue years back that I resolved by 
using numeric (now numpy?) as it had a more efficient method of 
importing content from disk.


Also realize that windows may not allow the full memory to user space. 
I'm not sure what exactly the restrictions are, but a 4Gb windows box 
doesn't always get you 4Gb of memory.


Emile


--
http://mail.python.org/mailman/listinfo/python-list


Re: A desperate lunge for on-topic-ness

2012-10-19 Thread Demian Brecht

On 2012-10-18, at 6:34 PM, Steven D'Aprano 
 wrote:

> Flame away :)


This post made my Friday, even though I'm sitting on a nearly two hour bus ride 
into work because I missed my commuter train. Just wanted you to know ;) You 
noted *every* reason (and them some) why my own code never passed 79 chars and 
why I try to instil that same value in others.

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Good debugger for CPython 3.2?

2012-10-19 Thread Dan Stromberg
On Fri, Oct 19, 2012 at 9:26 AM, Dan Stromberg  wrote:

>
> What's a good debugger for CPython 3.2?  I'd prefer to use it on Linux
> Mint 13, and I'd be happy with something based on X11 or curses.
>
> I tried winpdb, but it was cranky that Linux didn't have a spawn
> callable.  Why they didn't use the portable subprocess module escapes me.
>
> I also tried ddd, but it blew up on:
> def callable(*msg, file=sys.stderr):
> ...which suggests to me that this is too-new syntax for the child debugger
> ddd is using.
>
> Are there other good options?  Did I give up too quickly on one of the
> above?
>

Quick update: I seem to be having good luck with pudb (the latest git
version) so far.

The name probably wasn't given by someone in marketing, but it seems to
work.
-- 
http://mail.python.org/mailman/listinfo/python-list


Python does not take up available physical memory

2012-10-19 Thread Pradipto Banerjee
Hi,

I am trying to read a file into memory. The size of the file is around 1 GB. I 
have a 3GB memory PC and the Windows Task Manager shows  2.3 GB available 
physical memory when I was trying to read the file. I tried to read the file as 
follows:

>>> fdata = open(filename, 'r').read()

I got a "MemoryError". I was watching the Windows Task Manager while I run the 
python command, and it appears that python *perhaps* never even attempted to 
use more memory but gave me this error.

Is there any reason why python can't read a 1GB file in memory even when a 2.3 
GB physical memory is available? Do I need to make a change in some setting or 
preferences?

I am using python(x,y) distribution (python 2.7) and use Spyder as the IDE.

Thanks


This communication is for informational purposes only. It is not intended to 
be, nor should it be construed or used as, financial, legal, tax or investment 
advice or an offer to sell, or a solicitation of any offer to buy, an interest 
in any fund advised by Ada Investment Management LP, the Investment advisor. 
Any offer or solicitation of an investment in any of the Funds may be made only 
by delivery of such Funds confidential offering materials to authorized 
prospective investors. An investment in any of the Funds is not suitable for 
all investors. No representation is made that the Funds will or are likely to 
achieve their objectives, or that any investor will or is likely to achieve 
results comparable to those shown, or will make any profit at all or will be 
able to avoid incurring substantial losses. Performance results are net of 
applicable fees, are unaudited and reflect reinvestment of income and profits. 
Past performance is no guarantee of future results. All financial data and 
other information are not warranted as to completeness or accuracy and are 
subject to change without notice.

Any comments or statements made herein do not necessarily reflect those of Ada 
Investment Management LP and its affiliates. This transmission may contain 
information that is confidential, legally privileged, and/or exempt from 
disclosure under applicable law. If you are not the intended recipient, you are 
hereby notified that any disclosure, copying, distribution, or use of the 
information contained herein (including any reliance thereon) is strictly 
prohibited. If you received this transmission in error, please immediately 
contact the sender and destroy the material in its entirety, whether in 
electronic or hard copy format.
-- 
http://mail.python.org/mailman/listinfo/python-list


Good debugger for CPython 3.2?

2012-10-19 Thread Dan Stromberg
What's a good debugger for CPython 3.2?  I'd prefer to use it on Linux Mint
13, and I'd be happy with something based on X11 or curses.

I tried winpdb, but it was cranky that Linux didn't have a spawn callable.
Why they didn't use the portable subprocess module escapes me.

I also tried ddd, but it blew up on:
def callable(*msg, file=sys.stderr):
...which suggests to me that this is too-new syntax for the child debugger
ddd is using.

Are there other good options?  Did I give up too quickly on one of the
above?

Thanks!
-- 
http://mail.python.org/mailman/listinfo/python-list


Image.paste with images with different palettes...

2012-10-19 Thread Tracubik

Hi all!
I'm trying to create a map generator for c64's games

it's a simple script and it work properly, but i've problem if i try to 
modify images before creating the map.


what i do is:
- take a 8 bit screenshoot with VICE emulator in gif or bmp format
- open shutter (screenshoot edit tool under linux) to modify the 
screenshoot (i.e. insert note of sensible part of the screenshoot)

- save the modified screenshot
saving the screenshoot modify the palette of the 8 bit image
every modified screenshoot have different palette
trying to paste the modified screenshots in a new image (Image.paste) 
result in a map with wrong colors.


Converting all in RGB solve a bit but result in a "grainy" image

any hints?
maybe using GIMP and setting it properly could help, but shutter have a 
so nice arrow tools...

--
http://mail.python.org/mailman/listinfo/python-list


Re: locking files on Linux

2012-10-19 Thread Nobody
On Thu, 18 Oct 2012 14:44:27 +0100, andrea crotti wrote:

> Uhh I see thanks, I guess I'll use the good-old .lock file (even if it
> might have some problems too).

In which case, you don't see. A lock file is also advisory, i.e. it only
affects applications which explicitly check for a lock file.

Historically, the advantage of lock files was that they worked on NFS
implementations which didn't implement locking (it's a long-standing Unix
joke that "NFS" stands for "Not a FileSystem", because it failed to
conform to established filesystem semantics).

Nowadays, NFS implementations which don't support locking are sufficiently
rare that they can safely be ignored. So lock files don't offer any
advantages, and one fairly obvious disadvantage (the possibility of a
"stale" lock file if the program terminates unexpectedly without removing
the lock file).

For any form of advisory locking, the one thing which matters is that all
progams which access the file agree on the mechanism used, i.e. whether to
use lockf(), fcntl(), flock() (locks created by one mechanism may or may
not be recognised by the others), or lock files, and in the case of lock
files, the naming convention which is used.

If the file is specific to a particular program, and you just need to
protect against multiple instances of that program, you can use whichever
mechanism you wish, and would be strongly advised to use kernel locks
(fcntl() is the most portable, followed by lockf(); flock() is a BSD-ism).

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Python on Windows

2012-10-19 Thread Tim Golden
On 19/10/2012 15:23, Mark Lawrence wrote:
> On 19/10/2012 14:44, Tim Golden wrote:
>>
>> (In general, PyPI is the first place to look for Python packages).
>>
>>
> 
> For the benefit of the OP and others this is worth reading on how to get
> Python packages from pypi that let you get Python packages from pypi
> http://stackoverflow.com/questions/4750806/how-to-install-pip-on-windows :)
> 

Heh. I avoided trying to explain how to install pip because the OP
seemed unfamiliar with enough concepts already. It never occurred to me
to use easy_install to install pip!

TJG
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: section with in a section config file and reading that config file

2012-10-19 Thread rusi
On Oct 19, 6:58 pm, Tarek Ziadé  wrote:
> On 10/19/12 12:22 PM, narasimha1...@gmail.com wrote:
>
> > yes but it is not only for one structure like above there will be many 
> > sections like that
>
> I'd use yaml or json then...

Maybe http://www.voidspace.org.uk/python/configobj.html ??
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Watching output and put back in background

2012-10-19 Thread Nobody
On Thu, 18 Oct 2012 14:05:58 +0100, andrea crotti wrote:

> Maybe a solution would be to redirect the stderr to file and watch that
> instead..
> 
> Or otherwise I could use a thread for each shell command, but I would like
> to avoid head-aches with possible race-conditions..

If you're running multiple concurrent commands, and you have redirected
their output streams to pipes, something needs to keep reading those pipes
if you don't want the commands to hang.

Rather than having a separate thread for each process, you could have a
single thread which manages all "background" processes using select(),
poll() or non-blocking I/O, but that's easier to do on Unix than on
Windows (Popen.communicate() uses a pair of threads on Windows).

Redirecting output to files then reading them upon completion is the
simplest solution, but you can't easily monitor progress that way (there's
no easy way to get notification when more output is written).

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Python on Windows

2012-10-19 Thread Tim Golden
[Could I suggest snipping some of the preceding replies unless you're
referring directly to them? Just leave enough to make the context clear]

[... attempts to find feedparser module for beginner's tutorial ...]

On 19/10/2012 15:12, graham wrote:
> Once again thanks to those that replied.
> 
> Since I posted, I found a version 4.1 on sourceforge.
> 
> I guessed/muddled my way through installation and it seems to work.
> 
> To Tom Golden,
>   Thanks for the link to http://pypi.python.org/pypi/feedparser/ - I
> couldn't find it and I don't know how you did.

At least from where I'm standing, it's the second Google result for
"python feedparser" but in fact I Googled for "PyPI feedparser" because
I knew that that was the most likely bet.

> 
> The version at http://pypi.python.org/pypi/feedparser/ is 5.1. Can I
> simply install 'over the top' of the previous one or do I need to
> uninstall V4.1?

You should just be able to over-install. No harm in deleting the
previous one, but you probably don't need to.

TJG
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Python on Windows

2012-10-19 Thread Mark Lawrence

On 19/10/2012 14:44, Tim Golden wrote:


(In general, PyPI is the first place to look for Python packages).




For the benefit of the OP and others this is worth reading on how to get 
Python packages from pypi that let you get Python packages from pypi 
http://stackoverflow.com/questions/4750806/how-to-install-pip-on-windows :)


--
Cheers.

Mark Lawrence.

--
http://mail.python.org/mailman/listinfo/python-list


Re: Python on Windows

2012-10-19 Thread graham

On 19/10/2012 14:24, graham wrote:

On 16/10/2012 12:29, graham wrote:


Downloaded and installed Python 2.7.3 for windows (an XP machine).

Entered the Python interactive interpreter/command line and typed the
following:

 >>>import feedparser

and I get the error message "No module named feedparser".

There is a feedparser.py file lurking around - so I suppose Python
cannot find it.

Anyone: What to do?



GC



Thanks to everyone who replied.

Python was installed in the subdirectory C:\Python27 with the file
feedparser.py residing in C:\Python27\Lib\email.

Setting the Windows environment variable (which did not previously
exist) to C:\Python27\Lib\email allowed me to import feedparser
successfully.

However, it seems that this feedparser module is not the module I wanted.

I'm trying to follow an introductory Python course from the magazine
Linux Format (issue number 120 I think). The article includes the
following lines:


import feedparser
url = “http://weather.yahooapis.com/forecastrss?p=UKXX0637&u=c”
data = feedparser.parse(url)


This is fine using Ubuntu (after installing the feedparser package) but
now, running XP I get


data = feedparser.parse(url)

Traceback (most recent call last):
   File "", line 1, in 
AttributeError: 'module' object has no attribute 'parse'



So there seems to be at least 2 feedparser modules - the one I have does
not include "parse". How can I identify the correct one? How do I

This is all confusing and frustrating.

Some searching suggests I need the 'universal feed parser' code. I can
find documentation for this but no code/module file. Is it available
only for Unix-like OS's?

GC




Once again thanks to those that replied.

Since I posted, I found a version 4.1 on sourceforge.

I guessed/muddled my way through installation and it seems to work.

To Tom Golden,
  Thanks for the link to http://pypi.python.org/pypi/feedparser/ - I 
couldn't find it and I don't know how you did.


The version at http://pypi.python.org/pypi/feedparser/ is 5.1. Can I 
simply install 'over the top' of the previous one or do I need to 
uninstall V4.1?



GC







--
http://mail.python.org/mailman/listinfo/python-list


Re: Python on Windows

2012-10-19 Thread Tim Golden
On 19/10/2012 14:24, graham wrote:
> Python was installed in the subdirectory C:\Python27 with the file
> feedparser.py residing in C:\Python27\Lib\email.
> 
> Setting the Windows environment variable (which did not previously
> exist) to C:\Python27\Lib\email allowed me to import feedparser
> successfully.

As an aside, this is not the best use of the PYTHONPATH
variable. (I would argue that, on Windows at least, there's very little
need for the env var).

In general, you'll want to be using a mechanism such as pip:

  http://pypi.python.org/pypi/pip

which will look things up on PyPI so you can just do "pip install
newmodule". This will install into c:\python27\lib\site-packages. The
same for a module you install "manually" (ie python setup.py install) or
via an .exe or an .msi from PyPI or elsewhere.

There are other possibilities: .pth files, the PYTHONPATH env var,
virtualenv, user-install directories which you can read about, but in
general, just let the standard mechanisms install into
c:\python27\lib\site-packages (which is automatically on sys.path) and
go from there.

TJG
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: section with in a section config file and reading that config file

2012-10-19 Thread Tarek Ziadé

On 10/19/12 12:22 PM, narasimha1...@gmail.com wrote:


yes but it is not only for one structure like above there will be many sections 
like that

I'd use yaml or json then...


--
http://mail.python.org/mailman/listinfo/python-list


Re: Python on Windows

2012-10-19 Thread rusi
On Oct 19, 6:24 pm, graham  wrote:
> On 16/10/2012 12:29, graham wrote:
>
>
>
>
>
>
>
>
>
>
>
> > Downloaded and installed Python 2.7.3 for windows (an XP machine).
>
> > Entered the Python interactive interpreter/command line and typed the
> > following:
>
> >      >>>import feedparser
>
> > and I get the error message "No module named feedparser".
>
> > There is a feedparser.py file lurking around - so I suppose Python
> > cannot find it.
>
> > Anyone: What to do?
>
> > GC
>
> Thanks to everyone who replied.
>
> Python was installed in the subdirectory C:\Python27 with the file
> feedparser.py residing in C:\Python27\Lib\email.
>
> Setting the Windows environment variable (which did not previously
> exist) to C:\Python27\Lib\email allowed me to import feedparser
> successfully.
>
> However, it seems that this feedparser module is not the module I wanted.
>
> I'm trying to follow an introductory Python course from the magazine
> Linux Format (issue number 120 I think). The article includes the
> following lines:
>
> >>> import feedparser
> >>> url = “http://weather.yahooapis.com/forecastrss?p=UKXX0637&u=c”
> >>> data = feedparser.parse(url)
>
> This is fine using Ubuntu (after installing the feedparser package) but
> now, running XP I get
>
> >>> data = feedparser.parse(url)
>
> Traceback (most recent call last):
>    File "", line 1, in 
> AttributeError: 'module' object has no attribute 'parse'
>
> So there seems to be at least 2 feedparser modules - the one I have does
> not include "parse". How can I identify the correct one? How do I
>
> This is all confusing and frustrating.
>
> Some searching suggests I need the 'universal feed parser' code. I can
> find documentation for this but no code/module file. Is it available
> only for Unix-like OS's?
>
> GC

Just try taking your ubuntu's /usr/share/pyshare/feedparser.py onto
your windows box??
[UNTESTED]
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Python on Windows

2012-10-19 Thread Tim Golden
On 19/10/2012 14:24, graham wrote:
> Thanks to everyone who replied.
> 
> Python was installed in the subdirectory C:\Python27 with the file
> feedparser.py residing in C:\Python27\Lib\email.
> 
> Setting the Windows environment variable (which did not previously
> exist) to C:\Python27\Lib\email allowed me to import feedparser
> successfully.
> 
> However, it seems that this feedparser module is not the module I wanted.
> 
> I'm trying to follow an introductory Python course from the magazine
> Linux Format (issue number 120 I think). 


I'm very surprised that the article tells you to import a non-standard
module without telling you where to download it. I imagine that the
module is this one:

  http://pypi.python.org/pypi/feedparser/

(In general, PyPI is the first place to look for Python packages).

> This is all confusing and frustrating.

Understandably. It's not unknown, but it is unusual for two
identically-named packages to exist for Python. It's more
unfortunate when it's one used in a beginner's article.

If you understand how to download the .zip file from the page above and
unzip it then you need to go to the directory where the unzipped files
are and run:

  python setup.py install

If you're not sure how to do any of that, feel free to post back here or
to the python-tutor list [1] which is a little more experience in
helping newcomers.

TJG

[1] http://mail.python.org/mailman/listinfo/tutor


-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Python on Windows

2012-10-19 Thread graham

On 16/10/2012 12:29, graham wrote:


Downloaded and installed Python 2.7.3 for windows (an XP machine).

Entered the Python interactive interpreter/command line and typed the
following:

 >>>import feedparser

and I get the error message "No module named feedparser".

There is a feedparser.py file lurking around - so I suppose Python
cannot find it.

Anyone: What to do?



GC



Thanks to everyone who replied.

Python was installed in the subdirectory C:\Python27 with the file 
feedparser.py residing in C:\Python27\Lib\email.


Setting the Windows environment variable (which did not previously 
exist) to C:\Python27\Lib\email allowed me to import feedparser 
successfully.


However, it seems that this feedparser module is not the module I wanted.

I'm trying to follow an introductory Python course from the magazine 
Linux Format (issue number 120 I think). The article includes the 
following lines:



import feedparser
url = “http://weather.yahooapis.com/forecastrss?p=UKXX0637&u=c”
data = feedparser.parse(url)


This is fine using Ubuntu (after installing the feedparser package) but 
now, running XP I get



data = feedparser.parse(url)

Traceback (most recent call last):
  File "", line 1, in 
AttributeError: 'module' object has no attribute 'parse'



So there seems to be at least 2 feedparser modules - the one I have does 
not include "parse". How can I identify the correct one? How do I


This is all confusing and frustrating.

Some searching suggests I need the 'universal feed parser' code. I can 
find documentation for this but no code/module file. Is it available 
only for Unix-like OS's?


GC






--
http://mail.python.org/mailman/listinfo/python-list


Re: pls help me with this prog

2012-10-19 Thread rusi
Dont know what your code does/tries to do. Anyway some points:

On Oct 19, 1:40 pm, inshu chauhan  wrote:
> in this prog I have written a code to calculate teh centre of a given 3D 
> data..
>
> but i want to calculate it for every 3 points not the whole data, but
> instead of giving me centre for every 3 data the prog is printing the
> centre 3 times...
>
> import cv
> from math import floor, sqrt, ceil
> from numpy import array, dot, subtract, add, linalg as lin
>
> def CalcCentre(data):
>     centre = array([0,0,0])
>     count = 0
>     n = 0
>     for p in data[n:n+3]:
>         centre = add(centre, array(p[:3]))
>         count += 1
>         centre = dot(1./count, centre)
>         return centre
>     n += 1



The return makes the for iterate only once
And n += 1 will never be reached

>t = tuple(map(float, sp[1:4]))
drops the sp[0] element.
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: locking files on Linux

2012-10-19 Thread andrea crotti
2012/10/18 Oscar Benjamin :
>
> The lock is cooperative. It does not prevent the file from being
> opened or overwritten. It only prevents any other process from
> obtaining the lock. Here you open the file with mode 'w' which
> truncates the file instantly (without checking for the lock).
>
>
> Oscar


Very good thanks now I understood, actually my problem was in the
assumption that it should fail when the lock is already taken, but by
default lockf just blocks until the lock is released.

It seems to work quite nicely so I'm going to use this..
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Fwd: system tray or notification area in python

2012-10-19 Thread Anssi Saari
Daniel Fetchinson  writes:

> But I have zero experience with gui programming in python. So any
> pointers would be much appreciated how to implement a system tray in
> python. Gtk is I guess just one option, one could use other stuff from
> python but I wouldn't know what the simplest approach is.

Well, when I went to download stalonepanel, SourceForge said I might be
interested in PyPanel as well. Which is a "lightweight panel/taskbar
written in Python and C for X11 window managers." It includes a system
tray. Fairly old though but I suppose it could be a start? Apparently it
uses python-xlib.
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: error executing "import html.parser" from a script

2012-10-19 Thread Hans Mulder
On 19/10/12 11:15:45, Paul Volkov wrote:
> What is this madness?

That's because your script is called "html.py".

If you import html.parser, Python first imports html,
then checks that it's a package and contains a module
named "parser".  When Python imports html, it searches
for a file named "html.py".  It finds your script,
imports it and decides that it's not a package.

Solution: rename your script.

> I have Python 3.3.0 installed on Windows XP. I do not have Python 2
> (but I had it before). I do the following steps:
> 
> 1. Import from an interactive session (no problems)
> 
>> python
> Python 3.3.0 (v3.3.0:bd8afb90ebf2, Sep 29 2012, 10:55:48) [MSC v.1600
> 32 bit (Intel)] on win32
> Type "help", "copyright", "credits" or "license" for more information.
 import html
 import html.parser
 ^Z
> 
> 2. Import from command line (no problems)
> 
>> python -c "import html.parser"
> 
> 3. Import from a script. The script contains only one import line.
> Everything else is commented out.
> 
>> python e:\tmp\pyt\html.py
> Traceback (most recent call last):
>   File "", line 1512, in _find_and_load_unlocked
> AttributeError: 'module' object has no attribute '__path__'
> 
> During handling of the above exception, another exception occurred:
> 
> Traceback (most recent call last):
>   File "e:\tmp\pyt\html.py", line 7, in 
> import html.parser
>   File "e:\tmp\pyt\html.py", line 7, in 
> import html.parser
> ImportError: No module named 'html.parser'; html is not a package
> 
> 4. And then I tried to print sys.path from my script by modifying it this way:
> import sys
> print (sys.path)
> import html.parser
> 
> And the result is: (I don't know why sys.path is printed twice)

The first time, your script is being run as a script, and
has the module name '__main__'.  The second time, your script
is being imported as a module, and has the module name 'html'.

When Python finds the command "import html.parser" for the
second time, there's an (incomplete) module named "html" in
sys.modules, and Python doesn't try to import html again
and instead tries to use it as a package, and fails.

>> python e:\tmp\pyt/html.py
> ['e:\\tmp\\pyt', 'D:\\WINDOWS\\system32\\python33.zip',
> 'D:\\Python33\\DLLs', 'D:\\Python33\\lib', '
> D:\\Python33', 'D:\\Python33\\lib\\site-packages']
> ['e:\\tmp\\pyt', 'D:\\WINDOWS\\system32\\python33.zip',
> 'D:\\Python33\\DLLs', 'D:\\Python33\\lib', '
> D:\\Python33', 'D:\\Python33\\lib\\site-packages']
> Traceback (most recent call last):
>   File "", line 1512, in _find_and_load_unlocked
> AttributeError: 'module' object has no attribute '__path__'
> 
> During handling of the above exception, another exception occurred:
> 
> Traceback (most recent call last):
>   File "e:\tmp\pyt/html.py", line 8, in 
> import html.parser
>   File "e:\tmp\pyt\html.py", line 8, in 
> import html.parser
> ImportError: No module named 'html.parser'; html is not a package

Notice how the import is also being reported twice.


Hope this helps,

-- HansM
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: python scripts for web

2012-10-19 Thread Gilles
On Thu, 18 Oct 2012 23:05:48 -0700 (PDT), chip9m...@gmail.com wrote:
>these scripts will do a lot of calculation on a big dataset, and it is 
>possible that there will be many requests in a short period of time.

In that case, are you sure a web script is a good idea? If you're
thinking web to make it easy for people to upload data, click on a
button, and get the results back, you might want to write the UI in
Python but write the number crunching part in a compiled language.
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: [2.5.1 + CGI] Which interpreter and files?

2012-10-19 Thread Gilles
On Fri, 19 Oct 2012 21:09:55 +1100, Chris Angelico 
wrote:
>The ones with the -w tag are designed for Windows apps that are going
>to bring up a GUI and don't want a console. The python[w]25.exe ones
>will be in case you have multiple Pythons installed and want to
>explicitly call for version 2.5. You probably want python.exe, but
>python25.exe will work too.

Thanks for the info.
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: section with in a section config file and reading that config file

2012-10-19 Thread narasimha18sv
On Friday, 19 October 2012 15:39:57 UTC+5:30, Tarek Ziadé  wrote:
> On 10/19/12 11:51 AM, kampy wrote:
> 
> > hi all,
> 
> > my requirement is to have section with in a section in config parameters
> 
> > ex:
> 
> > [AAA]
> 
> >  [BBB]
> 
> >   a=1
> 
> >   b=1
> 
> >  [CCC]
> 
> >   a=1
> 
> >   b=2
> 
> > Any one help me in  understanding how to make sure that config file to have 
> > a structure like this and reading with the config parser
> 
> a configuration file is a flat sequences of sections, you cannot do this
> 
> 
> 
> what you could do is have 2 files, and add a link from one to the other:
> 
> 
> 
> 
> 
> file1.ini:
> 
> 
> 
>[AAA]
> 
>extended = file2.ini
> 
> 
> 
> file2.ini:
> 
> 
> 
>[BBB]
> 
>a=1
> 
>b=1
> 
> 
> 
>[CCC]
> 
>a=1
> 
>b=2
> 
> 
> 
> 
> 
> then create a bit of logic on the top of ConfigParser to read back those 
> 
> values
> 
> 
> 
> HTH
> 
> Tarek

yes but it is not only for one structure like above there will be many sections 
like that
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Python interactive help()

2012-10-19 Thread Duncan Booth
Mark Lawrence  wrote:

> On 19/10/2012 09:56, Duncan Booth wrote:
>> Mark Lawrence  wrote:
>>
>>> Good morning/afternoon/evening all,
>>>
>>> Where is this specific usage documented as my search engine skills have
>>> let me down?  By this I mean entering help() without parameters to get
>>> the following output and then the help> prompt.
>>>
>> It is documented under 'built-in functions'.
>>
>> http://docs.python.org/library/functions.html#help
>>
>>
> 
> Well Foxtrot Mike :-)  Thanks for the fast response.
> 

A harder question would have been if you asked where 'exit()', 'quit()' are 
documented. For some reason they are hidden under "Built-in constants" 
(even though the documentation includes the function call syntax) alongside 
'license' and 'credits' which are documented without the parentheses but 
are also callable.

All but 'credits' behave similarly having a repr that gives you 
instructions to call them and doing something different when you do call 
them. 'credits' just gives you the same text whether or not you call it.

-- 
Duncan Booth http://kupuguy.blogspot.com
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: [2.5.1 + CGI] Which interpreter and files?

2012-10-19 Thread Chris Angelico
On Fri, Oct 19, 2012 at 9:02 PM, Gilles  wrote:
> 1. Mongoose must be told in the shebang file where to locate the
> interpreter, but ActivePython 2.5.1 comes with fours files that look
> like the interpreter (actually, two files, since the other two have
> the same size so they are probably left overs):
> 
> 01/05/2007  18:0124.064 python.exe
> 01/05/2007  18:0124.064 python25.exe
> 01/05/2007  18:0124.576 pythonw.exe
> 01/05/2007  18:0124.576 pythonw25.exe
> 

The ones with the -w tag are designed for Windows apps that are going
to bring up a GUI and don't want a console. The python[w]25.exe ones
will be in case you have multiple Pythons installed and want to
explicitly call for version 2.5. You probably want python.exe, but
python25.exe will work too.


ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: section with in a section config file and reading that config file

2012-10-19 Thread Tarek Ziadé

On 10/19/12 11:51 AM, kampy wrote:

hi all,
my requirement is to have section with in a section in config parameters
ex:
[AAA]
 [BBB]
  a=1
  b=1
 [CCC]
  a=1
  b=2
Any one help me in  understanding how to make sure that config file to have a 
structure like this and reading with the config parser

a configuration file is a flat sequences of sections, you cannot do this

what you could do is have 2 files, and add a link from one to the other:


file1.ini:

  [AAA]
  extended = file2.ini

file2.ini:

  [BBB]
  a=1
  b=1

  [CCC]
  a=1
  b=2


then create a bit of logic on the top of ConfigParser to read back those 
values


HTH
Tarek
--
http://mail.python.org/mailman/listinfo/python-list


[2.5.1 + CGI] Which interpreter and files?

2012-10-19 Thread Gilles
Hello

I'd like to use the Mongoose basic web server with Python which can
call scripts through CGI.

I have a coupole of questions:

1. Mongoose must be told in the shebang file where to locate the
interpreter, but ActivePython 2.5.1 comes with fours files that look
like the interpreter (actually, two files, since the other two have
the same size so they are probably left overs):

01/05/2007  18:0124.064 python.exe
01/05/2007  18:0124.064 python25.exe
01/05/2007  18:0124.576 pythonw.exe
01/05/2007  18:0124.576 pythonw25.exe


2. Which files and directories do I really need and must provide, and
which can I do without?

Directory of C:\Python25

20/07/2007  02:38  DLLs
20/07/2007  02:38  include
02/09/2012  11:47  Lib
20/07/2007  02:38  libs
18/07/2007  15:33   348.160 msvcr71.dll
28/10/2009  12:31 2.017 psyco-wininst.log
05/04/2008  03:17 9.226 py2exe-wininst.log
20/11/2008  15:13 4.218 pysqlite-wininst.log
01/05/2007  18:0124.064 python.exe
01/05/2007  18:0124.064 python25.exe
01/05/2007  18:0124.576 pythonw.exe
01/05/2007  18:0124.576 pythonw25.exe
03/05/2010  12:0161.440 Removeapsw.exe
28/10/2009  12:3161.440 Removepsyco.exe
05/04/2008  03:1761.440 Removepy2exe.exe
20/11/2008  15:1361.440 Removepysqlite.exe
14/11/2008  18:2261.440 Removesetuptools.exe
07/07/2010  14:43  Scripts
14/11/2008  18:24 9.859 setuptools-wininst.log
20/07/2007  02:39  tcl
20/07/2007  02:39  Tools
01/05/2007  18:01 4.608 w9xpopen.exe


Thank you.
-- 
http://mail.python.org/mailman/listinfo/python-list


section with in a section config file and reading that config file

2012-10-19 Thread kampy
hi all,
my requirement is to have section with in a section in config parameters
ex:
[AAA]
[BBB]
 a=1
 b=1
[CCC]
 a=1
 b=2
Any one help me in  understanding how to make sure that config file to have a 
structure like this and reading with the config parser
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Add if...else... switch to doctest?

2012-10-19 Thread Duncan Booth
David  wrote:

> Hello, how to add if...else... switch to doctest?
> E.g. function outputs different value when global_var change.
> 
> """
> if (global_var == True):
 function()
> [1,2]
> else:
 function()
> [1,2,3]
> """
> 
> Thank you very much.

One other case the other replies don't seem to have covered:

If the global variable is determined by the environment, outside your 
control, and by implication doesn't change while your program is running, 
then you should use two separate functions:

if sys.platform=='win32':
def function():
"""
>>> function()
[1, 2]
"""
return [1, 2]

else:
def function():
"""
>>> function()
[1, 2, 3]
"""
return [1, 2, 3]

and if it's more than one such function use separate modules.

-- 
Duncan Booth http://kupuguy.blogspot.com
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Testing against multiple versions of Python

2012-10-19 Thread Michele Simionato
ShiningPanda looks really really cool. I need to investigate it.
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: A desperate lunge for on-topic-ness

2012-10-19 Thread Jean-Michel Pichavant


- Original Message -
[snipe 80 char line discussion] 
> And, quite frankly, people who care more about the readability of
> their
> code than about squeezing in as much processing into a single line of
> text as possible.
> 

As usual Steven, you take someone's argument, you add a little bit of 
exaggeration and make it sound silly.
Using 80+ char lines doesn't mean I put all my efforts exceeding the 80 char 
limit. As a matter of fact, I never
get the char limit get into the way of readability. What needs to be short will 
be short, and what needs to be long will be long.


[snip] 
> Flame away :)

good job :o)

JM
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Testing against multiple versions of Python

2012-10-19 Thread Duncan Booth
Michele Simionato  wrote:

> Yesterday I released a new version of the decorator module. It should
> run under Python 2.4, 2.5, 2.6, 2.7, 3.0, 3.1, 3.2, 3.3. I did not
> have the will to install on my machine 8 different versions of Python,
> so I just tested it with Python 2.7 and 3.3. But I do not feel happy
> with that. Is there any kind of service where a package author can
> send a pre-release version of her package and have its tests run
> againsts a set of different Python versions? I seem to remember
> somebody talking about a service like that years ago but I don't
> remembers. I do not see anything on PyPI. Any advice is welcome! 
> 
Not exactly what you asked for, but if you clone 
https://github.com/collective/buildout.python then a single command will 
build Python 2.4, 2.5, 2.6, 2.7, 3.2, and 3.3 on your system.

-- 
Duncan Booth http://kupuguy.blogspot.com
-- 
http://mail.python.org/mailman/listinfo/python-list


error executing "import html.parser" from a script

2012-10-19 Thread Paul Volkov
What is this madness?
I have Python 3.3.0 installed on Windows XP. I do not have Python 2
(but I had it before). I do the following steps:

1. Import from an interactive session (no problems)

>python
Python 3.3.0 (v3.3.0:bd8afb90ebf2, Sep 29 2012, 10:55:48) [MSC v.1600
32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import html
>>> import html.parser
>>> ^Z

2. Import from command line (no problems)

>python -c "import html.parser"

3. Import from a script. The script contains only one import line.
Everything else is commented out.

>python e:\tmp\pyt\html.py
Traceback (most recent call last):
  File "", line 1512, in _find_and_load_unlocked
AttributeError: 'module' object has no attribute '__path__'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "e:\tmp\pyt\html.py", line 7, in 
import html.parser
  File "e:\tmp\pyt\html.py", line 7, in 
import html.parser
ImportError: No module named 'html.parser'; html is not a package

4. And then I tried to print sys.path from my script by modifying it this way:
import sys
print (sys.path)
import html.parser

And the result is: (I don't know why sys.path is printed twice)

>python e:\tmp\pyt/html.py
['e:\\tmp\\pyt', 'D:\\WINDOWS\\system32\\python33.zip',
'D:\\Python33\\DLLs', 'D:\\Python33\\lib', '
D:\\Python33', 'D:\\Python33\\lib\\site-packages']
['e:\\tmp\\pyt', 'D:\\WINDOWS\\system32\\python33.zip',
'D:\\Python33\\DLLs', 'D:\\Python33\\lib', '
D:\\Python33', 'D:\\Python33\\lib\\site-packages']
Traceback (most recent call last):
  File "", line 1512, in _find_and_load_unlocked
AttributeError: 'module' object has no attribute '__path__'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "e:\tmp\pyt/html.py", line 8, in 
import html.parser
  File "e:\tmp\pyt\html.py", line 8, in 
import html.parser
ImportError: No module named 'html.parser'; html is not a package
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Python interactive help()

2012-10-19 Thread Mark Lawrence

On 19/10/2012 09:56, Duncan Booth wrote:

Mark Lawrence  wrote:


Good morning/afternoon/evening all,

Where is this specific usage documented as my search engine skills have
let me down?  By this I mean entering help() without parameters to get
the following output and then the help> prompt.


It is documented under 'built-in functions'.

http://docs.python.org/library/functions.html#help




Well Foxtrot Mike :-)  Thanks for the fast response.

--
Cheers.

Mark Lawrence.

--
http://mail.python.org/mailman/listinfo/python-list


Re: Python interactive help()

2012-10-19 Thread Peter Otten
Mark Lawrence wrote:

> Where is this specific usage documented as my search engine skills have
> let me down?  

find and grep ftw.

> By this I mean entering help() without parameters to get
> the following output and then the help> prompt.

http://docs.python.org/dev/py3k/library/functions.html#help

"""
help([object])
Invoke the built-in help system. (This function is intended for interactive 
use.) If no argument is given, the interactive help system starts on the 
interpreter console. If the argument is a string, then the string is looked 
up as the name of a module, function, class, method, keyword, or 
documentation topic, and a help page is printed on the console. If the 
argument is any other kind of object, a help page on the object is 
generated.
"""

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Python interactive help()

2012-10-19 Thread Duncan Booth
Mark Lawrence  wrote:

> Good morning/afternoon/evening all,
> 
> Where is this specific usage documented as my search engine skills have 
> let me down?  By this I mean entering help() without parameters to get 
> the following output and then the help> prompt.
> 
It is documented under 'built-in functions'.

http://docs.python.org/library/functions.html#help


-- 
Duncan Booth http://kupuguy.blogspot.com
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Testing against multiple versions of Python

2012-10-19 Thread Boris FELD
Did you take a look at https://www.shiningpanda-ci.com/?

2012/10/19 andrea crotti :
> 2012/10/19 Michele Simionato :
>> Yesterday I released a new version of the decorator module. It should run 
>> under Python 2.4, 2.5, 2.6, 2.7, 3.0, 3.1, 3.2, 3.3. I did not have the will 
>> to install on my machine 8 different versions of Python, so I just tested it 
>> with Python 2.7 and 3.3. But I do not feel happy with that. Is there any 
>> kind of service where a package author can send a pre-release version of her 
>> package and have its tests run againsts a set of different Python versions?
>> I seem to remember somebody talking about a service like that years ago but 
>> I don't remembers. I do not see anything on PyPI. Any advice is welcome!
>>
>>  Michele Simionato
>>
>>
>> --
>> http://mail.python.org/mailman/listinfo/python-list
>
>
> Travis on github maybe is what you want?
> --
> http://mail.python.org/mailman/listinfo/python-list
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: Testing against multiple versions of Python

2012-10-19 Thread andrea crotti
2012/10/19 Michele Simionato :
> Yesterday I released a new version of the decorator module. It should run 
> under Python 2.4, 2.5, 2.6, 2.7, 3.0, 3.1, 3.2, 3.3. I did not have the will 
> to install on my machine 8 different versions of Python, so I just tested it 
> with Python 2.7 and 3.3. But I do not feel happy with that. Is there any kind 
> of service where a package author can send a pre-release version of her 
> package and have its tests run againsts a set of different Python versions?
> I seem to remember somebody talking about a service like that years ago but I 
> don't remembers. I do not see anything on PyPI. Any advice is welcome!
>
>  Michele Simionato
>
>
> --
> http://mail.python.org/mailman/listinfo/python-list


Travis on github maybe is what you want?
-- 
http://mail.python.org/mailman/listinfo/python-list


pls help me with this prog

2012-10-19 Thread inshu chauhan
in this prog I have written a code to calculate teh centre of a given 3D data..

but i want to calculate it for every 3 points not the whole data, but
instead of giving me centre for every 3 data the prog is printing the
centre 3 times...

import cv
from math import floor, sqrt, ceil
from numpy import array, dot, subtract, add, linalg as lin




def CalcCentre(data):
centre = array([0,0,0])
count = 0
n = 0
for p in data[n:n+3]:
centre = add(centre, array(p[:3]))
count += 1
centre = dot(1./count, centre)
return centre
n += 1
def ReadPointCloud(filename):
f = open(filename)
result = []
for l in f:
sp = l.split()
t = tuple(map(float, sp[1:4]))
result.append(t)
return result

def main (data):


j = 0
for  i in data[:3]:
while j != 3:
 centre = CalcCentre(data)
 j += 1
 print centre


if __name__ == '__main__':
data = ReadPointCloud(r'Z:\data\NEHreflectance_Scanner 1_part.txt')

main(data)




PLS HELP 
-- 
http://mail.python.org/mailman/listinfo/python-list


RE: Fastest template engine

2012-10-19 Thread Andriy Kornatskyy

Per community request cheetah has been added to benchmark. Post updated, just 
in case:

http://mindref.blogspot.com/2012/07/python-fastest-template.html

Comments or suggestions are welcome.

Andriy



> From: andriy.kornats...@live.com
> To: python-list@python.org
> Subject: RE: Fastest template engine
> Date: Wed, 26 Sep 2012 16:21:21 +0300
>
>
> The post has been updated with the following template engines added (per 
> community request):
>
> 1. chameleon
> 2. django
> 3. web2py
>
> Here is a link:
>
> http://mindref.blogspot.com/2012/07/python-fastest-template.html
>
> Comments or suggestions are welcome.
>
> Thanks.
>
> Andriy
>
>
> 
> > From: andriy.kornats...@live.com
> > To: python-list@python.org
> > Subject: Fastest template engine
> > Date: Sun, 23 Sep 2012 12:24:36 +0300
> >
> >
> > I have run recently a benchmark of a trivial 'big table' example for 
> > various python template engines (jinja2, mako, tenjin, tornado and 
> > wheezy.template) run on cpython2.7 and pypy1.9.. you might find it 
> > interesting:
> >
> > http://mindref.blogspot.com/2012/07/python-fastest-template.html
> >
> > Comments or suggestions are welcome.
> >
> > Thanks.
> >
> > Andriy Kornatskyy
> > --
> > http://mail.python.org/mailman/listinfo/python-list
>
  
-- 
http://mail.python.org/mailman/listinfo/python-list


Python interactive help()

2012-10-19 Thread Mark Lawrence

Good morning/afternoon/evening all,

Where is this specific usage documented as my search engine skills have 
let me down?  By this I mean entering help() without parameters to get 
the following output and then the help> prompt.


C:\Users\Mark\workspace\CrossCode>py -3
Python 3.3.0 (v3.3.0:bd8afb90ebf2, Sep 29 2012, 10:55:48) [MSC v.1600 32 
bit (Intel)] on win32

Type "help", "copyright", "credits" or "license" for more information.
>>> help()

Welcome to Python 3.3!  This is the interactive help utility.

If this is your first time using Python, you should definitely check out
the tutorial on the Internet at http://docs.python.org/3.3/tutorial/.

Enter the name of any module, keyword, or topic to get help on writing
Python programs and using Python modules.  To quit this help utility and
return to the interpreter, just type "quit".

To get a list of available modules, keywords, or topics, type "modules",
"keywords", or "topics".  Each module also comes with a one-line summary
of what it does; to list the modules whose summaries contain a given 
word such as "spam", type "modules spam".


help>

Cheers.

Mark Lawrence.
--
Cheers.

Mark Lawrence.

--
http://mail.python.org/mailman/listinfo/python-list