date:20120916

On Sun, 16 Sep 2012 12:20:08 +0530, Santosh Kumar wrote:

 I have a script that takes an which basically takes a command line
 argument and prints after processing. If I don't give the argument to
 the script, it gives me a ValueError:
 
 ValueError: need more than 1 value to unpack
 
 I was trying to utilizing this space, if there is no argument I want it
 to show:
 usages: myscriptpy [option] [argument]
 
 options:
 --help  print this help message and exit
 
 Nothing more than that. I was looking on the argparse module, it can do
 the stuffs I want, but
 I don't to rewrite and mess up my current script. What should I do?

Then don't mess it up.


What you are basically asking for sounds like I want to add this cool 
feature to my project, but I don't want to actually do the work of adding 
the feature. How can I snap my fingers and make it work just like magic?

The answer is, you can't.

Either:

1) Do without the feature; or

2) Program the feature.

For number 2), you can either modify your script to use argparse to deal 
with the argument handling (which is probably simplest), or you can try 
to duplicate argparse's functionality yourself.

Make a backup copy first -- never, ever, ever work on the only copy of a 
working script. Or better still, use a proper VCS (version control 
system) to manage the software. But for small changes and simple needs, a 
good enough solution is always edit a copy.

In this case, it *may* be simple enough to duplicate the functionality. 
There's no way to tell, because you don't show any code, but my *guess* 
is that you have a line like:

a, b = sys.argv

and if you don't provide any command-line arguments, that fails with 

ValueError: need more than 1 value to unpack

(Lots and lots and lots of other things will also fail with the same 
error.)

Is this case, the fix is simple:

args = sys.argv[1:]  # ignore the first entry, which is unimportant
if len(args) == 1:
arg = args[0]
process(arg, default_option)
elif len(args) == 2:
option, arg == args
process(arg, option)
elif len(args)  2:
do_error('too many arguments')
else:
do_error('not enough arguments')

But as the argument checking code gets more extensive, it is time to just 
use argparse or one of the other libraries for argument checking instead.



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: want to show list of available options and arguments in my command line utility

On Sun, 16 Sep 2012 12:20:08 +0530, Santosh Kumar wrote:

 I have a script that takes an which basically takes a command line
 argument and prints after processing. If I don't give the argument to
 the script, it gives me a ValueError:
 
 ValueError: need more than 1 value to unpack
 
 I was trying to utilizing this space, if there is no argument I want it
 to show:
 usages: myscriptpy [option] [argument]
 
 options:
 --help  print this help message and exit
 
 Nothing more than that. I was looking on the argparse module, it can do
 the stuffs I want, but
 I don't to rewrite and mess up my current script. What should I do?

Then don't mess it up.


What you are basically asking for sounds like I want to add this cool 
feature to my project, but I don't want to actually do the work of adding 
the feature. How can I snap my fingers and make it work just like magic?

The answer is, you can't.

Either:

1) Do without the feature; or

2) Program the feature.

For number 2), you can either modify your script to use argparse to deal 
with the argument handling (which is probably simplest), or you can try 
to duplicate argparse's functionality yourself.

Make a backup copy first -- never, ever, ever work on the only copy of a 
working script. Or better still, use a proper VCS (version control 
system) to manage the software. But for small changes and simple needs, a 
good enough solution is always edit a copy.

In this case, it *may* be simple enough to duplicate the functionality. 
There's no way to tell, because you don't show any code, but my *guess* 
is that you have a line like:

a, b = sys.argv

and if you don't provide any command-line arguments, that fails with 

ValueError: need more than 1 value to unpack

(Lots and lots and lots of other things will also fail with the same 
error.)

Is this case, the fix is simple:

args = sys.argv[1:]  # ignore the first entry, which is unimportant
if len(args) == 1:
arg = args[0]
process(arg, default_option)
elif len(args) == 2:
option, arg == args
process(arg, option)
elif len(args)  2:
do_error('too many arguments')
else:
do_error('not enough arguments')

But as the argument checking code gets more extensive, it is time to just 
use argparse or one of the other libraries for argument checking instead.



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: unit test strategy

On Fri, 14 Sep 2012 19:59:29 -0700, Aaron Brady wrote:

 Hello,
 
 I've developing a test script.  There's a lot of repetition.  I want to
 introduce a strategy for approaching it, but I don't want the program to
 be discredited because of the test script.

Test scripts should be simple enough that they don't require test scripts 
of their own. Or at least, not too many test-the-test tests. It is 
possible to avoid repetition without making convoluted, hard to 
understand code. Based on the tiny code fragments you give below, I 
suspect that your test script will need more testing than the code it 
tests!


 Therefore, I'd like to know
 what people's reactions to and thoughts about it are.

I'd love to make some suggestions, but I have *no idea* what you are 
talking about. See further comments below:


 The first strategy I used created an iterator and advanced it between
 each step:

What are you using an iterator for? What does this have to do with unit 
testing?

So far, your explanation is rather lacking. It's a bit like:

I want to create an alarm system for my home, so I put in a screw and 
tightened it after each step.

Doesn't really help us understand what you are doing.


   self.op_chain(range(5), ('add', 5))
   self.op_chain(range(5), ('add', -2), ('add', -1))
   self.op_chain(range(5), ('discard', -1), ('add', 5))
   self.op_chain_ok(range(5), ('update', [0, 1]))
 Etc.

Where is the iterator you created? Where are you advancing it? What's 
op_chain do?


 I'm considering something more complicated.  'iN' creates iterator N,
 'nN' advances iterator N, an exception calls 'assertRaises', and the
 rest are function calls.
[...]

You've proven that even in Python people can write obfuscated code.


 Do you think the 2nd version is legible?

Neither version is even close to legible.


 Could it interfere with the accuracy of the test?

Who knows? I have no clue what your code is doing, it could be doing 
*anything*.



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: datetime issue

dn = datetime.datetime.now()
dd = datetime.timedelta(hours=2)
date = dn + dd
date = date.strftime( '%y-%m-%d %H:%M:%S' )

still giving me texas,us time for some reason
-- 
http://mail.python.org/mailman/listinfo/python-list

RE: Moving folders with content

2012-09-16 Thread jyoung79

Thank you  Nobody and Hans!

 You may want to use the subprocess module to run 'ditto'.  If
 the destination folder does not exist, then ditto will copy MacOS
 specific aspects such as resource forks, ACLs and HFS meta-data.

This looks like a good direction to go.  Maybe something like:

 import os
 import subprocess
 
 p1 = os.path.expanduser('~/Desktop/IN/Test/')
 p2 = os.path.expanduser('~/Desktop/OUT/Test/')
 
 cmd = 'ditto -vV ' + p1 + ' ' + p2 + ''
 
 v = subprocess.check_output(cmd, shell=True)
 Copying /Users/jay/Desktop/IN/Test/ 
copying file ./.DS_Store ... 
6148 bytes for ./.DS_Store
copying file ./dude.txt ... 
4 bytes for ./dude.txt
copying file ./new.png ... 
114469 bytes for ./new.png

Jay
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python Logging: Specifying converter attribute of a log formatter in config file

2012-09-16 Thread Vinay Sajip

On Thursday, August 30, 2012 11:38:27 AM UTC+1, Radha Krishna Srimanthula wrote:
 
 Now, how do I specify the converter attribute (time.gmtime) in the above 
 section?

Sadly, there is no way of doing this using the configuration file, other than 
having e.g. a

class UTCFormatter(logging.Formatter):
converter = time.gmtime

and then using a UTCFormatter in the configuration.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: datetime issue

Τη Κυριακή, 16 Σεπτεμβρίου 2012 10:51:18 π.μ. UTC+3, ο χρήστης Νικόλαος Κούρας 
έγραψε:
 dn = datetime.datetime.now()
 
 dd = datetime.timedelta(hours=2)
 
 date = dn + dd
 
 date = date.strftime( '%y-%m-%d %H:%M:%S' )
 
 
 
 still giving me texas,us time for some reason

which is same as this:

date = ( datetime.datetime.now() + datetime.timedelta(hours=2) ).strftime( 
'%y-%m-%d %H:%M:%S')

this also doesnt work either:

date = ( datetime.datetime.now(hours=2).strftime( '%y-%m-%d %H:%M:%S')

if only it would!
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: datetime issue

Τη Κυριακή, 16 Σεπτεμβρίου 2012 8:53:57 π.μ. UTC+3, ο χρήστης Dennis Lee Bieber 
έγραψε:
 On Sat, 15 Sep 2012 22:15:38 -0700 (PDT), Íéêüëáïò Êïýñáò
 
 nikos.gr...@gmail.com declaimed the following in
 
 gmane.comp.python.general:
 
 
 
 
 
  
 
  If i wanted to alter the following line, how would i write it?
 
  
 
  date = datetime.datetime.now()+datetime.timedelta(hours=2).strftime( 
  '%y-%m-%d %H:%M:%S')
 
  
 
  But that doesnt work,
 
 
 
   What did you expect? Object methods bind tighter than operators so
 
 what you have is the equivalent of
 
 
 
 dn = datetime.datetime.now()
 
 dd = datetime.timedelta(hours=2).strftime(...)
 
 date = dn + dd
 
 
 
   Try
 
 
 
  import datetime
 
  date = datetime.datetime.now()+datetime.timedelta(hours=2).strftime( 
  '%y-%m-%d %H:%M:%S')
 
 Traceback (most recent call last):
 
   File interactive input, line 1, in module
 
 AttributeError: 'datetime.timedelta' object has no attribute 'strftime'
 
  date = (datetime.datetime.now()+datetime.timedelta(hours=2) ).strftime( 
  '%y-%m-%d %H:%M:%S')
 
  date
 
 '12-09-16 03:50:44'
 
  
 
 
 
   Note the ( ) wrapping the the + clause, with strftime() applied to
 
 the result of that...
 
 -- 
 
   Wulfraed Dennis Lee Bieber AF6VN
 
 wlfr...@ix.netcom.comHTTP://wlfraed.home.netcom.com/



date = ( datetime.datetime.now() + datetime.timedelta(hours=8) ).strftime( 
'%y-%m-%d %H:%M:%S')

but iam giving +8 hours which is the time difference between texas, us where 
the server sits and Greece's time.

cant we somehow tell it to use GMT+2 ?

also it would be nice if datetime.datetime.now(GMT+2) can be used.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: datetime issue

On Sun, 16 Sep 2012 01:25:30 -0700, Νικόλαος Κούρας wrote:
[...]

You seem to be emailing python-list AND posting to comp.lang.python (or 
the gmane mirror). Please pick one, or the other, and not both, because 
the mailing list and the newsgroup are mirrors of each other. Anything 
you send to the mailing list will be mirrored on the newsgroup 
automatically, there is no need to manually duplicate the post.


Thank you.


-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: datetime issue

Τη Κυριακή, 16 Σεπτεμβρίου 2012 12:53:42 μ.μ. UTC+3, ο χρήστης Steven D'Aprano 
έγραψε:
 On Sun, 16 Sep 2012 01:25:30 -0700, Νικόλαος Κούρας wrote:
 
 [...]
 
 
 
 You seem to be emailing python-list AND posting to comp.lang.python (or 
 
 the gmane mirror). Please pick one, or the other, and not both, because 
 
 the mailing list and the newsgroup are mirrors of each other. Anything 
 
 you send to the mailing list will be mirrored on the newsgroup 
 
 automatically, there is no need to manually duplicate the post.
 
 
 
 
 
 Thank you.
 
 
 
 
 
 -- 
 
 Steven

Iam sorry i didnt do that on purpose and i dont know how this is done.

Iam positng via google groups using chrome, thats all i know.

Whats a mailing list?
Can i get responses to my mail instead of constantly check the google groups 
site?
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Moving folders with content

2012-09-16 Thread Hans Mulder

On 16/09/12 10:02:09, jyoun...@kc.rr.com wrote:
 Thank you  Nobody and Hans!

You're welcome!

 You may want to use the subprocess module to run 'ditto'.  If
 the destination folder does not exist, then ditto will copy MacOS
 specific aspects such as resource forks, ACLs and HFS meta-data.
 
 This looks like a good direction to go.  Maybe something like:
 
 import os
 import subprocess

 p1 = os.path.expanduser('~/Desktop/IN/Test/')
 p2 = os.path.expanduser('~/Desktop/OUT/Test/')

 cmd = 'ditto -vV ' + p1 + ' ' + p2 + ''

 v = subprocess.check_output(cmd, shell=True)

This looks iffy: it would break if there would be any double
quotes in p1 or p2.  You might think that os.path.expanduser
would never expand '~' to something containing a double quote,
but you'd be wrong:

 import os
 os.environ['HOME'] = 'gotcha!; rm -rf '
 print(os.path.expanduser('~/Desktop/IN/Test/'))
gotcha!; rm -rf /Desktop/IN/Test/

It's easy and safer to avoid using 'shell=True' option:

cmd = ['ditto', '-vV', p1, p2]
v = subprocess.check_output(cmd, shell=False)

In this case, the safer version also happens to be shorter and
more readable.  But you should get into the habit of using
shell=False whenever possible, because it is much easier to
get it right.


Hope this helps,

-- HansM
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: datetime issue

On Sun, 16 Sep 2012 03:15:11 -0700, Νικόλαος Κούρας wrote:

 Whats a mailing list?
 Can i get responses to my mail instead of constantly check the google
 groups site?

http://mail.python.org/mailman/listinfo/python-list



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: datetime issue

Τη Κυριακή, 16 Σεπτεμβρίου 2012 1:49:38 μ.μ. UTC+3, ο χρήστης Steven D'Aprano 
έγραψε:
 On Sun, 16 Sep 2012 03:15:11 -0700, Νικόλαος Κούρας wrote:
 
 
 
  Whats a mailing list?
 
  Can i get responses to my mail instead of constantly check the google
 
  groups site?
 
 
 
 http://mail.python.org/mailman/listinfo/python-list
 
 
 
 
 
 
 
 -- 
 
 Steven

Thank you, i prefer to be notifies only to thread iam initiating or answering 
to not all of te threads. Is this possible?
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: datetime issue

On Sun, 16 Sep 2012 03:54:45 -0700, Νικόλαος Κούρας wrote:

 Τη Κυριακή, 16 Σεπτεμβρίου 2012 1:49:38 μ.μ. UTC+3, ο χρήστης Steven
 D'Aprano έγραψε:
 On Sun, 16 Sep 2012 03:15:11 -0700, Νικόλαος Κούρας wrote:
 
 
 
  Whats a mailing list?
 
  Can i get responses to my mail instead of constantly check the google
 
  groups site?
 
 
 
 http://mail.python.org/mailman/listinfo/python-list

 Thank you, i prefer to be notifies only to thread iam initiating or
 answering to not all of te threads. Is this possible?

No. That's not how mailing lists work. Every email gets posted to all 
members, unless they go onto No Mail, in which case they get no emails 
at all.

You are not paying anything for the help you are receiving, except to 
give your own time in return to help others. We are all volunteers here, 
and nobody is going to force you to volunteer in return, but if everyone 
only received threads that they initiated, nobody would see new threads 
and nobody would get any answers at all.



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

newbie question : gedit as an ide

2012-09-16 Thread Mayuresh Kathe


new to the group, a quick hello to all.  :-)
does anyone use gedit as an 'ide' for python development?
if yes, may i know the caveats against using 'idle', 'bpython', etc?

thank you.
--
http://mail.python.org/mailman/listinfo/python-list

Re: datetime issue

Τη Κυριακή, 16 Σεπτεμβρίου 2012 2:34:34 μ.μ. UTC+3, ο χρήστης Steven D'Aprano
έγραψε:
On Sun, 16 Sep 2012 03:54:45 -0700, Νικόλαος Κούρας wrote:

Τη Κυριακή, 16 Σεπτεμβρίου 2012 1:49:38 μ.μ. UTC+3, ο χρήστης Steven

D'Aprano έγραψε:

On Sun, 16 Sep 2012 03:15:11 -0700, Νικόλαος Κούρας wrote:

Whats a mailing list?

Can i get responses to my mail instead of constantly check the google

groups site?

http://mail.python.org/mailman/listinfo/python-list

Thank you, i prefer to be notifies only to thread iam initiating or

answering to not all of te threads. Is this possible?

No. That's not how mailing lists work. Every email gets posted to all

members, unless they go onto No Mail, in which case they get no emails

at all.

You are not paying anything for the help you are receiving, except to

give your own time in return to help others. We are all volunteers here,

and nobody is going to force you to volunteer in return, but if everyone

only received threads that they initiated, nobody would see new threads

and nobody would get any answers at all.

Steven

You are right, i will set to receive them all.

But i dont know what to do about the thing you say that i double post.

I use google groups i dont intentionally post to python-list too.

How can i avoid that and just only post to python google group?
--
http://mail.python.org/mailman/listinfo/python-list

Re: unit test strategy

2012-09-16 Thread Mark Lawrence


On 15/09/2012 03:59, Aaron Brady wrote:

Hello,

I've developing a test script.  There's a lot of repetition.  I want to 
introduce a strategy for approaching it, but I don't want the program to be 
discredited because of the test script.  Therefore, I'd like to know what 
people's reactions to and thoughts about it are.

The first strategy I used created an iterator and advanced it between each step:
self.op_chain(range(5), ('add', 5))
self.op_chain(range(5), ('add', -2), ('add', -1))
self.op_chain(range(5), ('discard', -1), ('add', 5))
self.op_chain_ok(range(5), ('update', [0, 1]))
Etc.

I'm considering something more complicated.  'iN' creates iterator N, 'nN' 
advances iterator N, an exception calls 'assertRaises', and the rest are 
function calls.
dsi= dict.__setitem__
ddi= dict.__delitem__
dsd= dict.setdefault
KE= KeyError
IE= IterationError
self.chain(range(10), 'i0', (dsi, 0, 1), 'n0', (dsi, 10, 1), (IE, 'n0'))
self.chain(range(10), 'i0', 'n0', (dsd, 0, 0), 'n0', (dsd, 10, 1), (IE, 
'n0'))
self.chain(range(10), 'i0', (KE, ddi, 10), 'n0', (ddi, 9), (IE, 'n0'))

Do you think the 2nd version is legible?  Could it interfere with the accuracy 
of the test?



http://docs.python.org/library/unittest.html#organizing-test-code seems 
to be a good starting point for avoiding repetition and introducing a 
strategy.


--
Cheers.

Mark Lawrence.

--
http://mail.python.org/mailman/listinfo/python-list

Obnoxious postings from Google Groups (was: datetime issue)

2012-09-16 Thread Ben Finney

Νικόλαος Κούρας nikos.gr...@gmail.com writes:

 Iam sorry i didnt do that on purpose and i dont know how this is done.

 Iam positng via google groups using chrome, thats all i know.

It is becoming quite clear that some change has happened recently to
Google Groups that makes posts coming from there rather more obnoxious
than before. And there doesn't seem to be much its users can do except
use something else.

Using Google Groups for posting to Usenet has been a bad idea for a long
time, but now it just seems to be a sure recipe for annoying the rest of
us. Again, not something you have much control over, except to stop
using Google Groups.

-- 
 \  “Actually I made up the term “object-oriented”, and I can tell |
  `\you I did not have C++ in mind.” —Alan Kay, creator of |
_o__)Smalltalk, at OOPSLA 1997 |
Ben Finney
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: datetime issue

2012-09-16 Thread Roy Smith

In article 4f9d9a0b-539a-4b6a-af3e-b02d1f400...@googlegroups.com,
 Íéêüëáïò Êïýñáò nikos.gr...@gmail.com wrote:


 Iam sorry i didnt do that on purpose and i dont know how this is done.
 
 Iam positng via google groups using chrome, thats all i know.
 
 Whats a mailing list?

Is it September already?
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: datetime issue

2012-09-16 Thread Günther Dietrich

In article cdf072b2-7359-4417-b1e4-d984e4317...@googlegroups.com,
 Νικόλαος Κούρας nikos.gr...@gmail.com wrote:

[...]

also it would be nice if datetime.datetime.now(GMT+2) can be used.

In news:mailman.774.1347735926.27098.python-l...@python.org, one of 
the first answers to your question you were pointed to pytz. This module 
does exactly what you ask for:

 import datetime
 import pytz
 greek_date = datetime.datetime.now(pytz.timezone('Europe/Athens'))
 greek_date
 print(greek_date)

If you do a help(pytz), obviously after having imported pytz, you will 
get some information about the module. At the end of this help, there 
are some attributes listed. One of them is all_timezones. Guess what it 
contains?



Best regards,

Günther




PS: I didn't know pytz yet. But it took me just five minutes of reading 
the datetime documentation and trying pytz.timezone(), to get a working 
example.
So I don't understand, why you didn't follow the proposal of trying 
pytz, but claimed, it wouldn't work.
Can you explain, why ist doesn't work for you rsp. what is the error 
when it doesn't work?
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: datetime issue

On Sun, Sep 16, 2012 at 11:22 PM, Roy Smith r...@panix.com wrote:
 In article 4f9d9a0b-539a-4b6a-af3e-b02d1f400...@googlegroups.com,
  Νικόλαος Κούρας nikos.gr...@gmail.com wrote:
 Iam sorry i didnt do that on purpose and i dont know how this is done.

 Iam positng via google groups using chrome, thats all i know.

 Whats a mailing list?

 Is it September already?

Yes, it is; what does that mean in response to this question? Is there
something about September that causes the question, or is it a once
per month[1] question?

ChrisA
[1] http://tvtropes.org/pmwiki/pmwiki.php/Main/OncePerEpisode
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: datetime issue

On Mon, Sep 17, 2012 at 12:06 AM, Roy Smith r...@panix.com wrote:
 In article mailman.789.1347802379.27098.python-l...@python.org,
  Chris Angelico ros...@gmail.com wrote:

  Is it September already?

 Yes, it is; what does that mean in response to this question? Is there
 something about September that causes the question, or is it a once
 per month[1] question?

 http://en.wikipedia.org/wiki/Eternal_September

Ohh, yes. I do remember reading about that. Yes, it is definitely September.

Back in 1993, I wasn't yet started on networking. My dad ran things,
because I was only nine years old, and he definitely (and quite
rightly) didn't let me onto the internet. So I'm a tad young to know
about these things beyond what I've read in the history books...

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: newbie question : gedit as an ide

On Sun, Sep 16, 2012 at 9:52 PM, Mayuresh Kathe mayur...@kathe.in wrote:
 new to the group, a quick hello to all.  :-)
 does anyone use gedit as an 'ide' for python development?
 if yes, may i know the caveats against using 'idle', 'bpython', etc?

 thank you.

I never really liked gedit; when I first switched my primary
development platform to Linux (from working on OS/2 and Windows), it
was an Ubuntu, and I felt that gedit was less than satisfactory. The
second GUI editor I tried was SciTE, which IMHO is distinctly
superior. Mainly, though, I don't use any sort of IDE per se; the only
IDE-like feature in my setup is SciTE's F7 key binding to run
'make', and then a properly-configured makefile in every directory
that I work in. Between that and a terminal window for git commands,
there's not really much more to look for in a development environment.

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: newbie question : gedit as an ide

2012-09-16 Thread Devin Jeanpierre

On Sun, Sep 16, 2012 at 7:52 AM, Mayuresh Kathe mayur...@kathe.in wrote:
 new to the group, a quick hello to all.  :-)
 does anyone use gedit as an 'ide' for python development?
 if yes, may i know the caveats against using 'idle', 'bpython', etc?

bpython isn't an IDE, it's a good interactive interpreter.

As for gedit as an editor... it's actually really bad, but fixable if
you install plugins. e.g., it doesn't have builtin support for case
sensitive / regexp search and so on, but these can be added so that
gedit becomes tolerable. In fact, one of gedit's main advantages over
many of its similar competitors (in particular I'm thinking of Kate)
is that the plugin support is very strong, so if it doesn't do what
you want, you can make it do what you want anyway.

So, install / enable plugins. Lots of them. Or use an editor designed
for programmers first, such as Kate or vim or emacs.

-- Devin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Moving folders with content

2012-09-16 Thread Nobody

On Sun, 16 Sep 2012 12:40:18 +0200, Hans Mulder wrote:

 But you should get into the habit of using shell=False whenever
 possible, because it is much easier to get it right.

More accurately, you should get into the habit of passing a list as the
first argument, rather than a string.

On Unix-like systems (including Mac OS X), this effectively requires
shell=False. Passing a list with shell=True has behaviour which is
well-defined, but rarely useful (the first element of the list will be
executed as a shell command, the remaining elements will be available via
the shell variables $1, $2, etc within that command).

On Windows, the list is converted to a command string using the same
quoting rules regardless of the value of the shell= parameter. The
difference is that shell=False requires the executable to actually be a
binary executable, while shell=True allows it to be some other type of
file (e.g. a batch file, Python script, etc).

-- 
http://mail.python.org/mailman/listinfo/python-list

newbie question About O'Reilly Python for Unix and Linux System Administration ftp Mirror question

2012-09-16 Thread moonhkt

Hi All


O'Reilly Book ISBN 978-986-6840-36-4.

python --version
Python 2.6.2 on AIX 5.3

Using this python to get files in ftp server.

I got below error. What is the error meaning and how to fix ?

ftp_mirror.py
Traceback (most recent call last):
  File /xx../shell/ftpmirror.py, line 80, in module
f = FTPSync(options.host, options.username, options.password,
options.remote_dir, options.local_dir, opti
ons.delete)
  File xx../shell/ftpmirror.py, line 17, in __init__
self.conn.cwd(ftp_base_dir)
  File /opt/freeware/lib/python2.6/ftplib.py, line 536, in cwd
cmd = 'CWD ' + dirname
TypeError: cannot concatenate 'str' and 'NoneType' objects

Source :

#!/usr/bin/env python

import ftplib
import os

class FTPSync(object):
def __init__(self, host, username, password, ftp_base_dir,
local_base_dir, delete=False):
self.host = host
self.username = username
self.password = password
self.ftp_base_dir = ftp_base_dir
self.local_base_dir = local_base_dir
self.delete = delete

self.conn = ftplib.FTP(host, username, password)
self.conn.cwd(ftp_base_dir)
try:
os.makedirs(local_base_dir)
except OSError:
pass
os.chdir(local_base_dir)
def get_dirs_files(self):
dir_res = []
self.conn.dir('.', dir_res.append)
files = [f.split(None, 8)[-1] for f in dir_res if
f.startswith('-')]
dirs = [f.split(None, 8)[-1] for f in dir_res if
f.startswith('d')]
return (files, dirs)
def walk(self, next_dir):
print Walking to, next_dir
self.conn.cwd(next_dir)
try:
os.mkdir(next_dir)
except OSError:
pass
os.chdir(next_dir)

ftp_curr_dir = self.conn.pwd()
local_curr_dir = os.getcwd()

files, dirs = self.get_dirs_files()
print FILES:, files
print DIRS:, dirs
for f in files:
print next_dir, ':', f
outf = open(f, 'wb')
try:
self.conn.retrbinary('RETR %s' % f, outf.write)
finally:
outf.close()
if self.delete:
print Deleting, f
self.conn.delete(f)
for d in dirs:
os.chdir(local_curr_dir)
self.conn.cwd(ftp_curr_dir)
self.walk(d)

def run(self):
self.walk('.')


if __name__ == '__main__':
from optparse import OptionParser
parser = OptionParser()
parser.add_option(-o, --host, dest=host,
 action='store', help=FTP host)
parser.add_option(-u, --username, dest=username,
 action='store', help=FTP username)
parser.add_option(-p, --password, dest=password,
 action='store', help=FTP password)
parser.add_option(-r, --remote_dir, dest=remote_dir,
 action='store', help=FTP remote starting directory)
parser.add_option(-l, --local_dir, dest=local_dir,
 action='store', help=Local starting directory)
parser.add_option(-d, --delete, dest=delete, default=False,
 action='store_true', help=use regex parser)

(options, args) = parser.parse_args()
f = FTPSync(options.host, options.username, options.password,
options.remote_dir, options.local_dir, options.delete)
f.run()

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: datetime issue

2012-09-16 Thread MRAB


On 2012-09-16 09:25, Νικόλαος Κούρας wrote:
[snip]


date = ( datetime.datetime.now() + datetime.timedelta(hours=8) ).strftime( 
'%y-%m-%d %H:%M:%S')

but iam giving +8 hours which is the time difference between texas, us where 
the server sits and Greece's time.

cant we somehow tell it to use GMT+2 ?

also it would be nice if datetime.datetime.now(GMT+2) can be used.


In programming, you need attention to details.

My reply didn't use datetime.datetime.now(), it used 
datetime.datetime.utcnow().


datetime.datetime.now() gives the local time (local to the system on 
which it is running).


datetime.datetime.utcnow() gives the UTC (GMT) time, which is the same 
everywhere.


The line should be this:

date = (datetime.datetime.utcnow() + datetime.timedelta(hours=8) 
).strftime('%Y-%m-%d %H:%M:%S')


I've also used '%Y' instead of '%y' because I prefer 4 digits for the year.

--
http://mail.python.org/mailman/listinfo/python-list

Re: newbie question About O'Reilly Python for Unix and Linux System Administration ftp Mirror question

2012-09-16 Thread Joel Goldstick

On Sun, Sep 16, 2012 at 11:09 AM, moonhkt moon...@gmail.com wrote:
 Hi All


 O'Reilly Book ISBN 978-986-6840-36-4.

 python --version
 Python 2.6.2 on AIX 5.3

 Using this python to get files in ftp server.

 I got below error. What is the error meaning and how to fix ?

 ftp_mirror.py
 Traceback (most recent call last):
   File /xx../shell/ftpmirror.py, line 80, in module
 f = FTPSync(options.host, options.username, options.password,
 options.remote_dir, options.local_dir, opti
 ons.delete)
   File xx../shell/ftpmirror.py, line 17, in __init__
 self.conn.cwd(ftp_base_dir)
   File /opt/freeware/lib/python2.6/ftplib.py, line 536, in cwd
 cmd = 'CWD ' + dirname
 TypeError: cannot concatenate 'str' and 'NoneType' objects

 Source :

 #!/usr/bin/env python

 import ftplib
 import os

 class FTPSync(object):
 def __init__(self, host, username, password, ftp_base_dir,
 local_base_dir, delete=False):
 self.host = host
 self.username = username
 self.password = password
 self.ftp_base_dir = ftp_base_dir
 self.local_base_dir = local_base_dir
 self.delete = delete

 self.conn = ftplib.FTP(host, username, password)
 self.conn.cwd(ftp_base_dir)
 try:
 os.makedirs(local_base_dir)
 except OSError:
 pass
 os.chdir(local_base_dir)
 def get_dirs_files(self):
 dir_res = []
 self.conn.dir('.', dir_res.append)
 files = [f.split(None, 8)[-1] for f in dir_res if
 f.startswith('-')]
 dirs = [f.split(None, 8)[-1] for f in dir_res if
 f.startswith('d')]
 return (files, dirs)
 def walk(self, next_dir):
 print Walking to, next_dir
 self.conn.cwd(next_dir)
 try:
 os.mkdir(next_dir)
 except OSError:
 pass
 os.chdir(next_dir)

 ftp_curr_dir = self.conn.pwd()
 local_curr_dir = os.getcwd()

 files, dirs = self.get_dirs_files()
 print FILES:, files
 print DIRS:, dirs
 for f in files:
 print next_dir, ':', f
 outf = open(f, 'wb')
 try:
 self.conn.retrbinary('RETR %s' % f, outf.write)
 finally:
 outf.close()
 if self.delete:
 print Deleting, f
 self.conn.delete(f)
 for d in dirs:
 os.chdir(local_curr_dir)
 self.conn.cwd(ftp_curr_dir)
 self.walk(d)

 def run(self):
 self.walk('.')


 if __name__ == '__main__':
 from optparse import OptionParser
 parser = OptionParser()
 parser.add_option(-o, --host, dest=host,
  action='store', help=FTP host)
 parser.add_option(-u, --username, dest=username,
  action='store', help=FTP username)
 parser.add_option(-p, --password, dest=password,
  action='store', help=FTP password)
 parser.add_option(-r, --remote_dir, dest=remote_dir,
  action='store', help=FTP remote starting directory)
 parser.add_option(-l, --local_dir, dest=local_dir,
  action='store', help=Local starting directory)
 parser.add_option(-d, --delete, dest=delete, default=False,
  action='store_true', help=use regex parser)

 (options, args) = parser.parse_args()
comment the next line , then print the parameters and see what they
are.  That should get you started.
 #f = FTPSync(options.host, options.username, options.password,
 #options.remote_dir, options.local_dir, options.delete)
  f = print(options.host, options.username, options.password,
  options.remote_dir, options.local_dir, options.delete)


 f.run()

 --
 http://mail.python.org/mailman/listinfo/python-list



-- 
Joel Goldstick
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: datetime issue

Τη Κυριακή, 16 Σεπτεμβρίου 2012 6:30:15 μ.μ. UTC+3, ο χρήστης MRAB έγραψε:
 On 2012-09-16 09:25, Νικόλαος Κούρας wrote:
 
 [snip]
 
 
 
  date = ( datetime.datetime.now() + datetime.timedelta(hours=8) ).strftime( 
  '%y-%m-%d %H:%M:%S')
 
 
 
  but iam giving +8 hours which is the time difference between texas, us 
  where the server sits and Greece's time.
 
 
 
  cant we somehow tell it to use GMT+2 ?
 
 
 
  also it would be nice if datetime.datetime.now(GMT+2) can be used.
 
 
 
 In programming, you need attention to details.
 
 
 
 My reply didn't use datetime.datetime.now(), it used 
 
 datetime.datetime.utcnow().
 
 
 
 datetime.datetime.now() gives the local time (local to the system on 
 
 which it is running).
 
 
 
 datetime.datetime.utcnow() gives the UTC (GMT) time, which is the same 
 
 everywhere.
 
 
 
 The line should be this:
 
 
 
 date = (datetime.datetime.utcnow() + datetime.timedelta(hours=8) 
 
 ).strftime('%Y-%m-%d %H:%M:%S')
 
 
 
 I've also used '%Y' instead of '%y' because I prefer 4 digits for the year.

Cant it be written more easily as:
 date = (datetime.datetime.utcnow(+2)

i know this is not thhe correct syntax but it just needs a way to add GMT+2 
hours since utc=gmt
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: newbie question About O'Reilly Python for Unix and Linux System Administration ftp Mirror question

On Mon, Sep 17, 2012 at 1:09 AM, moonhkt moon...@gmail.com wrote:
 Hi All

 O'Reilly Book ISBN 978-986-6840-36-4.

 python --version
 Python 2.6.2 on AIX 5.3

Hi! Thanks for this, good information to open with.

 Using this python to get files in ftp server.

 I got below error. What is the error meaning and how to fix ?

 ftp_mirror.py
 Traceback (most recent call last):
   File /xx../shell/ftpmirror.py, line 80, in module
 f = FTPSync(options.host, options.username, options.password,
 options.remote_dir, options.local_dir, opti
 ons.delete)
   File xx../shell/ftpmirror.py, line 17, in __init__
 self.conn.cwd(ftp_base_dir)
   File /opt/freeware/lib/python2.6/ftplib.py, line 536, in cwd
 cmd = 'CWD ' + dirname
 TypeError: cannot concatenate 'str' and 'NoneType' objects

NoneType is the type of the singleton object None. Why are you getting
None where you ought to be providing a directory name? Heavily trimmed
code follows:

 def __init__(self, host, username, password, ftp_base_dir,
 local_base_dir, delete=False):
 self.conn.cwd(ftp_base_dir)
 parser.add_option(-r, --remote_dir, dest=remote_dir,
  action='store', help=FTP remote starting directory)
 f = FTPSync(options.host, options.username, options.password,
 options.remote_dir, options.local_dir, options.delete)

If you don't pass -r/--remote_dir to your script, then (presumably - I
haven't actually used optparse so this is glarked from context)
options.remote_dir is being set to, or left at, None. A quick look at
the docs suggests that one solution is to add a default value to the
options:

http://docs.python.org/library/optparse.html#default-values

Alternatively, treat the options as mandatory, and provide them on the
command line.

Side point: When you go to the docs page there, you may notice that
optparse is deprecated in favour of argparse. May as well continue the
tutorial with what they recommend, but it's probably worth having a
look at argparse eventually.

Hope that helps!

Chris Angelico
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: datetime issue

Τη Κυριακή, 16 Σεπτεμβρίου 2012 4:23:02 μ.μ. UTC+3, ο χρήστης Günther Dietrich 
έγραψε:
 In article cdf072b2-7359-4417-b1e4-d984e4317...@googlegroups.com,
 
  Νικόλαος Κούρας nikos.gr...@gmail.com wrote:
 
 
 
 [...]
 
 
 
 also it would be nice if datetime.datetime.now(GMT+2) can be used.
 
 
 
 In news:mailman.774.1347735926.27098.python-l...@python.org, one of 
 
 the first answers to your question you were pointed to pytz. This module 
 
 does exactly what you ask for:
 
 
 
  import datetime
 
  import pytz
 
  greek_date = datetime.datetime.now(pytz.timezone('Europe/Athens'))
 
  greek_date
 
  print(greek_date)
 
 
 
 If you do a help(pytz), obviously after having imported pytz, you will 
 
 get some information about the module. At the end of this help, there 
 
 are some attributes listed. One of them is all_timezones. Guess what it 
 
 contains?
 
 
 
 
 
 
 
 Best regards,
 
 
 
 Günther
 
 
 
 
 
 
 
 
 
 PS: I didn't know pytz yet. But it took me just five minutes of reading 
 
 the datetime documentation and trying pytz.timezone(), to get a working 
 
 example.
 
 So I don't understand, why you didn't follow the proposal of trying 
 
 pytz, but claimed, it wouldn't work.
 
 Can you explain, why ist doesn't work for you rsp. what is the error 
 
 when it doesn't work?

import pytz fails in my webhost unfortunately :(
This module is not supported by hostgator.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Obnoxious postings from Google Groups (was: datetime issue)

Τη Κυριακή, 16 Σεπτεμβρίου 2012 4:18:50 μ.μ. UTC+3, ο χρήστης Ben Finney έγραψε:
 Νικόλαος Κούρας nikos.gr...@gmail.com writes:
 
 
 
  Iam sorry i didnt do that on purpose and i dont know how this is done.
 
 
 
  Iam positng via google groups using chrome, thats all i know.
 
 


 
 It is becoming quite clear that some change has happened recently to
 
 Google Groups that makes posts coming from there rather more obnoxious
 
 than before. And there doesn't seem to be much its users can do except
 
 use something else.
 
 
 
 Using Google Groups for posting to Usenet has been a bad idea for a long
 
 time, but now it just seems to be a sure recipe for annoying the rest of
 
 us. Again, not something you have much control over, except to stop
 
 using Google Groups.
 
 
 
 -- 
 
  \  “Actually I made up the term “object-oriented”, and I can tell |
 
   `\you I did not have C++ in mind.” —Alan Kay, creator of |
 
 _o__)Smalltalk, at OOPSLA 1997 |
 
 Ben Finney


If i ditch google groups what application can i use in Windows 8 to post to 
this newsgroup and what newsserver too?
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Obnoxious postings from Google Groups (was: datetime issue)

Whaen i tried to post just now by hitting sumbit, google groups told me that 
the following addresssed has benn found in this thread! i guess is used them 
all to notify everything!

cdf072b2-7359-4417-b1e4-d984e4317...@googlegroups.com
mailman.774.1347735926.27098.python-l...@python.org
nikos.gr...@gmail.com
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: datetime issue

On Mon, Sep 17, 2012 at 1:40 AM,  pandora.ko...@gmail.com wrote:
 Cant it be written more easily as:
  date = (datetime.datetime.utcnow(+2)

 i know this is not thhe correct syntax but it just needs a way to add GMT+2 
 hours since utc=gmt

I've dithered about whether to open this can of worms or let sleeping
dogs lie, and I finally decided to make some metaphor potpourri.

Simple UTC offsets are not the same thing as local time. There are a
variety of other considerations, but the main one is Daylight Robbery
Time, where every government wants to put its own stamp on things by
fiddling with the DST rules a bit. Most places will move their clocks
an hour forward for summer and then back for winter. But!

* Some places are sensible and don't have DST at all (eg
Australia/Queensland, America/Arizona)
* Northern hemisphere summer corresponds to southern hemisphere
winter, and vice versa
* Everyone has a different way of defining the boundaries of summer and winter
* And just to confuse us all, Irish Standard Time is used in summer,
and they *subtract* an hour for winter time!

The only way to handle timezones correctly and keep yourself even
marginally sane is to hand the job on to somebody else. I dodged the
whole issue by letting our Postgres database handle everything for us
(date/time formats, timezones, the lot), and having our internal
systems all just work in UTC. You'll likely find it easiest to do the
same thing with a Python library.

Just don't try to pretend to yourself that Greece uses GMT+2, because
it's not that simple. For one thing, it's currently summer there...

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Obnoxious postings from Google Groups (was: datetime issue)

On Mon, Sep 17, 2012 at 1:44 AM,  pandora.ko...@gmail.com wrote:
 Whaen i tried to post just now by hitting sumbit, google groups told me that 
 the following addresssed has benn found in this thread! i guess is used them 
 all to notify everything!

 cdf072b2-7359-4417-b1e4-d984e4317...@googlegroups.com
 mailman.774.1347735926.27098.python-l...@python.org
 nikos.gr...@gmail.com

Ah. Did you then send to all three? Just a wild guess, but I'm
thinking that might be the cause of post duplication...

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Obnoxious postings from Google Groups (was: datetime issue)


 
  http://mail.python.org/mailman/listinfo/python-list
 
 
 
 email client to python-list@python.org
 

wait a minute! i must use my ISP's news server and then post o comp.lang.python 
no?

What is python-list@python.org how can i post there?
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Obnoxious postings from Google Groups (was: datetime issue)

2012-09-16 Thread Joel Goldstick

On Sun, Sep 16, 2012 at 11:41 AM,  pandora.ko...@gmail.com wrote:
 Τη Κυριακή, 16 Σεπτεμβρίου 2012 4:18:50 μ.μ. UTC+3, ο χρήστης Ben Finney 
 έγραψε:
 Νικόλαος Κούρας nikos.gr...@gmail.com writes:



  Iam sorry i didnt do that on purpose and i dont know how this is done.

 

  Iam positng via google groups using chrome, thats all i know.





 It is becoming quite clear that some change has happened recently to

 Google Groups that makes posts coming from there rather more obnoxious

 than before. And there doesn't seem to be much its users can do except

 use something else.



 Using Google Groups for posting to Usenet has been a bad idea for a long

 time, but now it just seems to be a sure recipe for annoying the rest of

 us. Again, not something you have much control over, except to stop

 using Google Groups.



 --

  \  “Actually I made up the term “object-oriented”, and I can tell |

   `\you I did not have C++ in mind.” —Alan Kay, creator of |

 _o__)Smalltalk, at OOPSLA 1997 |

 Ben Finney


 If i ditch google groups what application can i use in Windows 8 to post to 
 this newsgroup and what newsserver too?
 --
 http://mail.python.org/mailman/listinfo/python-list

email client to python-list@python.org



-- 
Joel Goldstick
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Obnoxious postings from Google Groups (was: datetime issue)


 
 Ah. Did you then send to all three? Just a wild guess, but I'm
 
 thinking that might be the cause of post duplication...
 
 
 
 ChrisA

I had no choise, it doesnt let me pick one, it just notifies me that it will 
posted to these 3.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python presentations

2012-09-16 Thread Alexander Blinne

On 14.09.2012 14:19, Chris Angelico wrote:
 Err, yes, I did mean ** there. The extra multiplications may be
 slower, but which is worse? Lots of list additions, or lots of integer
 powers? In the absence of clear and compelling evidence, I'd be
 inclined to go with the list comp - and what's more, to skip this
 function altogether and use the list comp directly where it's needed.

I did some timing with the following versions of the function:

def powerlist1(x, n):
result=[1]
for i in xrange(n-1):
result.append(result[-1]*x)
return result

def powerlist2(x,n):
if n==1:
return [1]
p = powerlist3(x,n-1)
p.append(p[-1]*x)
return p

def powerlist3(x,n):
  return [x**i for i in xrange(n)]

with Python 2.7 you are quite right, i used x=4. Below n=26 powerlist3
is the fastest version, for n26 powerlist1 is faster, powerlist2 is
always slower than both.

With Pypy there is a completely different picture, with n30 powerlist2
is way faster then the other two, but then powerlist1 is again faster
for greater n, somehow because now C long int cannot be used any longer.

for really big n powerlist3 always takes very much time :)

Alex
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python presentations

On Mon, Sep 17, 2012 at 2:13 AM, Alexander Blinne n...@blinne.net wrote:
 def powerlist3(x,n):
   return [x**i for i in xrange(n)]

 for really big n powerlist3 always takes very much time :)

I would reiterate that a really big n is a really unusual use case for
a function like this, except that... I frankly can't think of *any*
use case for it!! But for many many applications, the simplicity and
readability of a list comp instead of a function is usually going to
outweigh the performance differences.

However, it doesn't surprise me that individually raising a number to
successive powers is slower than iterative multiplication, assuming
you can't massively optimize eg with powers of 2 and bit shifts.

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Obnoxious postings from Google Groups (was: datetime issue)

2012-09-16 Thread Joel Goldstick

On Sun, Sep 16, 2012 at 12:06 PM,  pandora.ko...@gmail.com wrote:


  http://mail.python.org/mailman/listinfo/python-list



 email client to python-list@python.org


 wait a minute! i must use my ISP's news server and then post o 
 comp.lang.python no?

 What is python-list@python.org how can i post there?
 --
 http://mail.python.org/mailman/listinfo/python-list

go to the url http://mail.python.org/mailman/listinfo/python-list

sign up and use your email client instead.

-- 
Joel Goldstick
-- 
http://mail.python.org/mailman/listinfo/python-list

testing....

2012-09-16 Thread Nick the Gr33k


Test to see if it works.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Obnoxious postings from Google Groups

2012-09-16 Thread Terry Reedy


On 9/16/2012 11:41 AM, pandora.ko...@gmail.com wrote:


If i ditch google groups


PLEASE DO

 what application can i use in Windows 8 to post to this newsgroup
 and what newsserver too?

news.gmane.org is a free newsserver that mirrors 1000s of technical 
email lists. python-list is gmane.comp.python.general. There are 100s of 
other gmane.comp.python.* groups.


I use Thunderbird on Win7 because it does both mail and news. I 
previously used Outlook Express for same. There are others.


--
Terry Jan Reedy

--
http://mail.python.org/mailman/listinfo/python-list

Re: Obnoxious postings from Google Groups (was: datetime issue)

2012-09-16 Thread Gene Heskett

On Sunday 16 September 2012 12:08:47 pandora.ko...@gmail.com did opine:

 Whaen i tried to post just now by hitting sumbit, google groups told me
 that the following addresssed has benn found in this thread! i guess is
 used them all to notify everything!
 
 cdf072b2-7359-4417-b1e4-d984e4317...@googlegroups.com
 mailman.774.1347735926.27098.python-l...@python.org
 nikos.gr...@gmail.com

Look, this googlegroups thing acting as a spam gateway has long since 
gotten old.  Whatever agent/program you are using must be a busted windows 
application because it is not showing you the last line of every post that 
comes through the list server which is:

http://mail.python.org/mailman/listinfo/python-list

There is, immediately above that line a '-- ' (note the space after, and 
its always after a linefeed or carriage return so its the first character 
of a line, and its code to all sane email agents that anything below it is 
a signature and not to be copied into a reply.  Windows email tends to clip 
it off and a windows user will never see it!  So that is why many of us 
lament, in long strings of expletives, the brokenness of windows email 
agents.

Subscribing to the mailing list is a 2 step process.  First you click on 
the above link and fill out the form and submit it.

That will, in a minute or so, cause an email to be sent to the address you 
used in the form.  You MUST reply to that message to confirm the 
subscription.  Whatever email agent you use from there is up to you, but 
I'd suggest Thunderbird.  I personally am partial to kmail, but then I'm 
running linux on everything here, no M$ stuff allowed on the premises.  Its 
not open for discussion and has not been in 28 years since I discovered the 
trs-80 color computer and an operating system for it called OS9.

Sorry folks, my rant for the day because of that gateway.

Cheers, Gene
-- 
There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order.
-Ed Howdershelt (Author)
My web page: http://coyoteden.dyndns-free.com:85/gene is up!
Succumb to natural tendencies.  Be hateful and boring.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Obnoxious postings from Google Groups (was: datetime issue)

On Sun, 16 Sep 2012 08:41:44 -0700, pandora.koura wrote:

 If i ditch google groups what application can i use in Windows 8 to post
 to this newsgroup and what newsserver too?

Google is your friend. Do try to find the answer to your questions before 
asking here.

Search for usenet news reader application. Or use Thunderbird.

For the news server, you use your ISP's newserver, if they offer one, or 
Gmane, or any of dozens of commercial news providers who will give you 
access to one for a small fee.


-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Obnoxious postings from Google Groups (was: datetime issue)

On Sun, 16 Sep 2012 09:06:30 -0700, pandora.koura wrote:


  http://mail.python.org/mailman/listinfo/python-list


 What is python-list@python.org how can i post there?

It's the same thing it was when I posted the above URL a few hours ago.

Didn't you follow the link before? It has instructions there.



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Using Py_AddPendingCall

2012-09-16 Thread css322

I have an embedded Python program which runs in a thread in C.

When the Python interpreter switches thread context (yielding control to 
another thread), I'd like to be notified so I can perform certain necessary 
operations.

It seems that Py_AddPendingCall is exactly what I'm looking for. However, the 
API docs are pretty brief on this function, and I'm confused as to how 
Py_AddPendingCall is supposed to be used. From reading the docs, my 
understanding is that:

(1) A worker thread calls Py_AddPendingCall and assigns a handler function.
(2) When the Python interpreter runs, it calls the handler function whenever it 
yields control to another thread
(3) The handler itself is executed in the main interpreter thread, with the GIL 
acquired

I've googled around for example code showing how to use Py_AddPendingCall, but 
I can't find anything. My own attempt to use it simply doesn't work. The 
handler is just never called.

My worker thread code:

const char* PYTHON_CODE = 
while True:\n
   for i in range(0,10): print(i)\n
\n;

int handler(void* arg)
{
printf(Pending Call invoked!\n);
abort();
}

void worker_thread()
{
PyGILState_STATE state = PyGILState_Ensure();
Py_AddPendingCall(handler, NULL);
PyRun_SimpleString(PYTHON_CODE);
PyGILState_Release(state);
}

In this example, worker_thread is invoked in C as a pthread worker thread. This 
loops infinitely, but the pending call handler is never invoked.

So,obviously I'm not understanding the docs correctly.  How is 
Py_AddPendingCall is supposed to be used?
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python presentations

On Sun, 16 Sep 2012 18:13:36 +0200, Alexander Blinne wrote:

 I did some timing with the following versions of the function:
 
 def powerlist1(x, n):
 result=[1]
 for i in xrange(n-1):
 result.append(result[-1]*x)
 return result
 
 def powerlist2(x,n):
 if n==1:
 return [1]
 p = powerlist3(x,n-1)
 p.append(p[-1]*x)
 return p

Is that a typo? I think you mean to make a recursive call to powerlist2, 
not a non-recursive call to powerlist3.


 def powerlist3(x,n):
   return [x**i for i in xrange(n)]
 
 with Python 2.7 you are quite right, i used x=4. Below n=26 powerlist3
 is the fastest version, for n26 powerlist1 is faster, powerlist2 is
 always slower than both.

Making powerlist2 recursive, the results I get with Python 2.7 are:


py from timeit import Timer as T
py x = 2.357
py n = 8
py t1 = T('powerlist1(x, n)', 
...  setup='from __main__ import powerlist1, x, n')
py t2 = T('powerlist2(x, n)', 
...  setup='from __main__ import powerlist2, x, n')
py t3 = T('powerlist3(x, n)', 
...  setup='from __main__ import powerlist3, x, n')
py min(t1.repeat(number=10, repeat=5))
0.38042593002319336
py min(t2.repeat(number=10, repeat=5))
0.5992050170898438
py min(t3.repeat(number=10, repeat=5))
0.334306001663208

So powerlist2 is half as fast as the other two, which are very close.

For large n, #1 and #3 are still neck-and-neck:

py n = 100
py min(t1.repeat(number=10, repeat=5))
3.6276791095733643
py min(t3.repeat(number=10, repeat=5))
3.58870792388916

which is what I would expect: the overhead of calling Python code will be 
greater than the speed up from avoiding float multiplications. But long 
integer unlimited-precision multiplications are slow. To really see the 
advantage of avoiding multiplications using Horner's Method (powerlist1), 
we need to use large integers and not floats.

py x = 12345678*1
py n = 3
py min(t1.repeat(number=10, repeat=5))
0.2199108600616455
py min(t3.repeat(number=10, repeat=5))
0.551645040512085

As n becomes bigger, the advantage also increases:

py n = 10
py min(t1.repeat(number=10, repeat=5))
0.736515998840332
py min(t3.repeat(number=10, repeat=5))
2.4837491512298584

In this case with n = 10, powerlist1 does 9 multiplications. But 
powerlist3 makes ten calls to the ** operator. The number of 
multiplications will depend on how cleverly exponentiation is 
implemented: at worst, using a naive algorithm, there will be 36 
multiplications. If the algorithm is a bit smarter, there will be 19 
multiplications.

Either way, when the calculation is dominated by the cost of 
multiplication, powerlist3 is between two and four times as expensive as 
powerlist1.


-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

RE: Moving folders with content

2012-09-16 Thread jyoung79

Hi Nobody and Hans,

I really appreciate you all sharing this insight with me.

 You might think that os.path.expanduser
 would never expand '~' to something containing a double quote,
 but you'd be wrong:

  import os
  os.environ['HOME'] = 'gotcha!; rm -rf '
  print(os.path.expanduser('~/Desktop/IN/Test/'))
 gotcha!; rm -rf /Desktop/IN/Test/
 
 It's easy and safer to avoid using 'shell=True' option:
 
 cmd = ['ditto', '-vV', p1, p2]
 v = subprocess.check_output(cmd, shell=False)

I would have never thought about this, and this could definitely 
give me problems.  Thank you for this advice!

Again, thank you both for the examples and deeper understanding 
of how subprocess works.  I will definitely start passing a list 
as the first argument as well as updating my older code to do 
the same.

Jay
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: unit test strategy

2012-09-16 Thread Aaron Brady

On Sunday, September 16, 2012 2:42:09 AM UTC-5, Steven D'Aprano wrote:
 On Fri, 14 Sep 2012 19:59:29 -0700, Aaron Brady wrote:
 
 
 
  Hello,
 
  
 
  I've developing a test script.  There's a lot of repetition.  I want to
 
  introduce a strategy for approaching it, but I don't want the program to
 
  be discredited because of the test script.
 
 
 
 Test scripts should be simple enough that they don't require test scripts 
 
 of their own. Or at least, not too many test-the-test tests. It is 
 
 possible to avoid repetition without making convoluted, hard to 
 
 understand code. Based on the tiny code fragments you give below, I 
 
 suspect that your test script will need more testing than the code it 
 
 tests!
 
 
 
 
 
  Therefore, I'd like to know
 
  what people's reactions to and thoughts about it are.
 
 
 
 I'd love to make some suggestions, but I have *no idea* what you are 
 
 talking about. See further comments below:
 
 
 
 
 
  The first strategy I used created an iterator and advanced it between
 
  each step:
 
 
 
 What are you using an iterator for? What does this have to do with unit 
 
 testing?
 
 
 
 So far, your explanation is rather lacking. It's a bit like:
 
 
 
 I want to create an alarm system for my home, so I put in a screw and 
 
 tightened it after each step.
 
 
 
 Doesn't really help us understand what you are doing.
 
 
 
 
 
  self.op_chain(range(5), ('add', 5))
 
  self.op_chain(range(5), ('add', -2), ('add', -1))
 
  self.op_chain(range(5), ('discard', -1), ('add', 5))
 
  self.op_chain_ok(range(5), ('update', [0, 1]))
 
  Etc.
 
 
 
 Where is the iterator you created? Where are you advancing it? What's 
 
 op_chain do?
 
 
 
 
 
  I'm considering something more complicated.  'iN' creates iterator N,
 
  'nN' advances iterator N, an exception calls 'assertRaises', and the
 
  rest are function calls.
 
 [...]
 
 
 
 You've proven that even in Python people can write obfuscated code.
 
 
 
 
 
  Do you think the 2nd version is legible?
 
 
 
 Neither version is even close to legible.
 
 
 
 
 
  Could it interfere with the accuracy of the test?
 
 
 
 Who knows? I have no clue what your code is doing, it could be doing 
 
 *anything*.
 
 
 
 
 
 
 
 -- 
 
 Steven

You are forcing me to explain my test code.

Here is an example of some repetitive code.

for view_meth in [ dict.items, dict.keys, dict.values ]:
dict0= dict( ( k, None ) for k in range( 10 ) )
iter0= iter( view_meth( dict0 ) )
dict.__setitem__( dict0, 0, 1 )
next( iter0 )
dict.__setitem__( dict0, 10, 1 )
self.assertRaises( IterationError, next, iter0 )

dict0= dict( ( k, None ) for k in range( 10 ) )
iter0= iter( view_meth( dict0 ) )
next( iter0 )
dict.__setitem__( dict0, 0, 1 )
next( iter0 )
dict.__setitem__( dict0, 10, 1 )
self.assertRaises( IterationError, next, iter0 )

dict0= dict( ( k, None ) for k in range( 10 ) )
iter0= iter( view_meth( dict0 ) )
self.assertRaises( KeyError, dict0.__delitem__, 10 )
next( iter0 )
dict.__delitem__( dict0, 9 )
self.assertRaises( IterationError, next, iter0 )

dict0= dict( ( k, None ) for k in range( 10 ) )
iter0= iter( view_meth( dict0 ) )
next( iter0 )
self.assertRaises( KeyError, dict0.__delitem__, 10 )
next( iter0 )
dict.__delitem__( dict0, 9 )
self.assertRaises( IterationError, next, iter0 )


It makes sense to condense it.  However, it can be condensed rather far, as 
follows:

dsi= dict.__setitem__
ddi= dict.__delitem__
KE= KeyError
IE= IterationError
chain(range(10), 'i0', (dsi, 0, 1), 'n0', (dsi, 10, 1), (IE, 'n0'))
chain(range(10), 'i0', 'n0', (dsi, 0, 1), 'n0', (dsi, 10, 1), (IE, 'n0'))
chain(range(10), 'i0', (KE, ddi, 10), 'n0', (ddi, 9), (IE, 'n0'))
chain(range(10), 'i0', 'n0', (KE, ddi, 10), 'n0', (ddi, 9), (IE, 'n0'))


The parameters to 'chain' correspond 1-to-1 with the statements earlier.  The 
view methods are looped over in the 'chain' method instead.  'op_chain' was an 
earlier version; some midway point in condensing the code could be preferable.

Specifically my questions are, is the code condensed beyond legibility?  Should 
'chain' execute the test directly, or act as a metaprogram and output the test 
code into a 2nd file, or both?  Should 'chain' create the iterators in a 
dictionary, or in the function local variables directly with 'exec'?
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: datetime issue

2012-09-16 Thread Mark Lawrence


On 16/09/2012 16:54, Chris Angelico wrote:

On Mon, Sep 17, 2012 at 1:40 AM,  pandora.ko...@gmail.com wrote:

Cant it be written more easily as:
  date = (datetime.datetime.utcnow(+2)

i know this is not thhe correct syntax but it just needs a way to add GMT+2 
hours since utc=gmt


I've dithered about whether to open this can of worms or let sleeping
dogs lie, and I finally decided to make some metaphor potpourri.

Simple UTC offsets are not the same thing as local time. There are a
variety of other considerations, but the main one is Daylight Robbery
Time, where every government wants to put its own stamp on things by
fiddling with the DST rules a bit. Most places will move their clocks
an hour forward for summer and then back for winter. But!

* Some places are sensible and don't have DST at all (eg
Australia/Queensland, America/Arizona)
* Northern hemisphere summer corresponds to southern hemisphere
winter, and vice versa
* Everyone has a different way of defining the boundaries of summer and winter
* And just to confuse us all, Irish Standard Time is used in summer,
and they *subtract* an hour for winter time!

The only way to handle timezones correctly and keep yourself even
marginally sane is to hand the job on to somebody else. I dodged the
whole issue by letting our Postgres database handle everything for us
(date/time formats, timezones, the lot), and having our internal
systems all just work in UTC. You'll likely find it easiest to do the
same thing with a Python library.

Just don't try to pretend to yourself that Greece uses GMT+2, because
it's not that simple. For one thing, it's currently summer there...

ChrisA



I guess that Double British Summer Time made things twice as difficult?

--
Cheers.

Mark Lawrence.

--
http://mail.python.org/mailman/listinfo/python-list

Re: Obnoxious postings from Google Groups

2012-09-16 Thread Mark Lawrence


On 16/09/2012 17:06, pandora.ko...@gmail.com wrote:





http://mail.python.org/mailman/listinfo/python-list




email client to python-list@python.org



wait a minute! i must use my ISP's news server and then post o comp.lang.python 
no?

What is python-list@python.org how can i post there?



I'm on Windows Vista and read many Python mailing lists with Thunderbird 
via gmane.


--
Cheers.

Mark Lawrence.

--
http://mail.python.org/mailman/listinfo/python-list

Re: datetime issue

2012-09-16 Thread Dave Angel

On 09/16/2012 07:56 AM, Νικόλαος Κούρας wrote:
Τη Κυριακή, 16 Σεπτεμβρίου 2012 2:34:34 μ.μ. UTC+3, ο χρήστης Steven D'Aprano
έγραψε:
snip

No. That's not how mailing lists work. Every email gets posted to all
members, unless they go onto No Mail, in which case they get no emails
at all.

You are not paying anything for the help you are receiving, except to
give your own time in return to help others. We are all volunteers here,
and nobody is going to force you to volunteer in return, but if everyone
only received threads that they initiated, nobody would see new threads
and nobody would get any answers at all.

You are right, i will set to receive them all.

But i dont know what to do about the thing you say that i double post.
I use google groups i dont intentionally post to python-list too.
How can i avoid that and just only post to python google group?

Once you get the emails from the official mailing list, posting a reply
is straightforward. Do a reply-all for most messages, perhaps deleting
some of the to or cc addresses depending on a specific message. Then
trim the parts you're not responding to, and put your new remarks after
the context you are responding to. It won't double-post.

For new messages, simply address a new message to python-list@python.org

Depending on your mail reader, you can see the messages in threads,
except for all the people who bust up the threads. When you get to that
point, post a new thread specifying your email program and OS, and
somebody will probably volunteer to help you get it customized (though
perhaps the actual discussion for that would be offline).

For anybody else listening, the way I avoid seeing the google-groups
double-post is to automatically delete any message that's got a cc to
google-groups. That way I only get the OTHER copy of the message.

DaveA

--
http://mail.python.org/mailman/listinfo/python-list

Re: Obnoxious postings from Google Groups

2012-09-16 Thread Mayuresh Kathe


On Sunday 16 September 2012 09:11 PM, pandora.ko...@gmail.com wrote:

Τη Κυριακή, 16 Σεπτεμβρίου 2012 4:18:50 μ.μ. UTC+3, ο χρήστης Ben Finney έγραψε:

Νικόλαος Κούρας nikos.gr...@gmail.com writes:




Iam sorry i didnt do that on purpose and i dont know how this is done.







Iam positng via google groups using chrome, thats all i know.








It is becoming quite clear that some change has happened recently to

Google Groups that makes posts coming from there rather more obnoxious

than before. And there doesn't seem to be much its users can do except

use something else.



Using Google Groups for posting to Usenet has been a bad idea for a long

time, but now it just seems to be a sure recipe for annoying the rest of

us. Again, not something you have much control over, except to stop

using Google Groups.



If i ditch google groups what application can i use in Windows 8 to post to
this newsgroup and what newsserver too?



Thunderbird 15 as the newsgroup client and Unison Access as the news 
server (it's a paid service).


--
http://mail.python.org/mailman/listinfo/python-list

Re: unit test strategy

On Sun, 16 Sep 2012 11:38:15 -0700, Aaron Brady wrote:


 Here is an example of some repetitive code.
 
 for view_meth in [ dict.items, dict.keys, dict.values ]:
   dict0= dict( ( k, None ) for k in range( 10 ) ) 
   iter0= iter( view_meth( dict0 ) )
   dict.__setitem__( dict0, 0, 1 )
   next( iter0 )
   dict.__setitem__( dict0, 10, 1 )
   self.assertRaises( IterationError, next, iter0 )
[...]

First off, if you have any wish for this to be accepted into the standard 
library, I suggest you stick to PEP 8. Spaces on *both* sides of equals 
signs, not just one(!!!), and *no* spaces around the parenthesised 
arguments.

Secondly, this is test code. A bit of repetition is not to be concerned 
about, clarity is far more important than Don't Repeat Yourself. The 
aim isn't to write the fastest, or most compact code, but to have good 
test coverage with tests which are *obviously* correct (rather than test 
code which has no obvious bugs, which is very different). If a test 
fails, you should be able to identify quickly what failed without running 
a debugger to identify what part of the code failed.

Thirdly, why are you writing dict.__setitem__( dict0, 0, 1 ) instead of 
dict0[0] = 1 ?


[...]
 Specifically my questions are, is the code condensed beyond legibility? 

Yes.


 Should 'chain' execute the test directly, or act as a metaprogram and
 output the test code into a 2nd file, or both?

None of the above.


 Should 'chain' create
 the iterators in a dictionary, or in the function local variables
 directly with 'exec'?

Heavens to Murgatroyd, you can't be serious.


Here's my attempt at this. Note the features:

- meaningful names (if a bit long, but that's a hazard of tests)
- distinct methods for each distinct test case
- comments explaining what the test code does
- use of subclassing


# Untested
class TestDictIteratorModificationDetection(unittest.TestCase):
Test that iterators to dicts will correctly detect when the
dict has been modified, and raise an exception.


def create_dict(self):
return dict.fromkeys(range(10))

def testIteratorAllowed(self):
# Iterators are allowed to run if all modifications occur
# after the iterator is created but before it starts running.
d = self.create_dict()
for view in (dict.items, dict.keys, dict.values):
# Create an iterator before modifying the dict, but do not
# start iterating over it.
it = iter(view(d))
# Be careful of the order of modifications here.
del d[2]
d.pop(4)
d.popitem()
d.clear()
# Make sure we have something to iterate over.
d[1] = 1
d.update({5: 1})
d.setdefault(8, 1)
assert d  # d is non-empty.
next(it); next(it); next(it)
self.assertRaises(StopIteration, next, it)


def iterator_fails_after_modification(self, method, *args):
Iterators should not be able to continue running after
the dict is modified. This helper method factors out the common
code of creating a dict, an iterator to that dict, modifying the
dict, and testing that further iteration fails.

Pass an unbound dict method which modifies the dict in place, and
and arguments to that method required.

d = self.create_dict()
for view in (dict.items, dict.keys, dict.values):
it = iter(view(d))
next(it)  # no exception expected here
method(d, *args)
self.assertRaises(IterationError, next, it)

def testIteratorFailsAfterSet(self):
self.iterator_fails_after_modification(dict.__setitem__, 1, 1)

def testIteratorFailsAfterDel(self):
self.iterator_fails_after_modification(dict.__delitem__, 1)

def testIteratorFailsAfterUpdate(self):
self.iterator_fails_after_modification(dict.update, {5: 1})

def testIteratorFailsAfterPop(self):
self.iterator_fails_after_modification(dict.pop, 4)

def testStartedIteratorFailsAfterPopItem(self):
self.iterator_fails_after_modification(dict.popitem)

def testStartedIteratorFailsAfterClear(self):
self.iterator_fails_after_modification(dict.clear)

def testStartedIteratorFailsAfterSetDefault(self):
self.iterator_fails_after_modification(dict.setdefault, 99, 1)



class TestDictSubclassIteratorModificationDetection(
   TestDictIteratorModificationDetection):
def create_dict(self):
class MyDict(dict):
pass
return MyDict.fromkeys(range(10))


I think I've got all the methods which can mutate a dictionary. If I 
missed any, it's easy enough to add a new test method to the class.

You are free to use the above code for your own purposes, with any 
modifications you like, with two requests:

- I would appreciate a comment in the test file acknowledging my

Re: Decorators not worth the effort

On Sep 15, 6:30 am, Terry Reedy tjre...@udel.edu wrote:
  On 13Sep2012 18:58, alex23 wuwe...@gmail.com wrote:
  | On Sep 14, 3:54 am, Jean-Michel Pichavant jeanmic...@sequans.com
  | wrote:
  |  I don't like decorators, I think they're not worth the mental effort.
  |
  | Because passing a function to a function is a huge cognitive burden?

 For parameterized decorators, there is extra cognitive burden. See below.

I do regret my initial criticism, for exactly this reason.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Obnoxious postings from Google Groups (was: datetime issue)

On Sep 16, 11:18 pm, Ben Finney ben+pyt...@benfinney.id.au wrote:
 Using Google Groups for posting to Usenet has been a bad idea for a long
 time, but now it just seems to be a sure recipe for annoying the rest of
 us. Again, not something you have much control over, except to stop
 using Google Groups.

Agreed. While it was painful but usable in its previous form, the new
Groups is an incomprehensible mess of pointlessness and shittery.
It's still possible to use the old theme for the time being, which
does avoid the double-up posts, but it's pretty clear Google aren't
listening to any feedback about Groups whatsoever.

I've really NFI why they bought DejaNews only to turn it into such a
broken service.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Obnoxious postings from Google Groups (was: datetime issue)

2012-09-16 Thread Roy Smith

In article 
6c732de0-b10f-4d40-853c-f62682970...@rg9g2000pbc.googlegroups.com,
 alex23 wuwe...@gmail.com wrote:

 On Sep 16, 11:18 pm, Ben Finney ben+pyt...@benfinney.id.au wrote:
  Using Google Groups for posting to Usenet has been a bad idea for a long
  time, but now it just seems to be a sure recipe for annoying the rest of
  us. Again, not something you have much control over, except to stop
  using Google Groups.
 
 Agreed. While it was painful but usable in its previous form, the new
 Groups is an incomprehensible mess of pointlessness and shittery.
 It's still possible to use the old theme for the time being, which
 does avoid the double-up posts, but it's pretty clear Google aren't
 listening to any feedback about Groups whatsoever.
 
 I've really NFI why they bought DejaNews only to turn it into such a
 broken service.

They didn't buy the service.  They bought the data.  Well, they really 
bought both, but the data is all they wanted.
-- 
http://mail.python.org/mailman/listinfo/python-list

Python garbage collector/memory manager behaving strangely

2012-09-16 Thread Jadhav, Alok

Hi Everyone,

 

I have a simple program which reads a large file containing few million
rows, parses each row (`numpy array`) and converts into an array of
doubles (`python array`) and later writes into an `hdf5 file`. I repeat
this loop for multiple days. After reading each file, i delete all the
objects and call garbage collector.  When I run the program, First day
is parsed without any error but on the second day i get `MemoryError`. I
monitored the memory usage of my program, during first day of parsing,
memory usage is around **1.5 GB**. When the first day parsing is
finished, memory usage goes down to **50 MB**. Now when 2nd day starts
and i try to read the lines from the file I get `MemoryError`. Following
is the output of the program.

 

 

source file extracted at C:\rfadump\au\2012.08.07.txt

parsing started

current time: 2012-09-16 22:40:16.829000

50 lines parsed

100 lines parsed

150 lines parsed

200 lines parsed

250 lines parsed

300 lines parsed

350 lines parsed

400 lines parsed

450 lines parsed

500 lines parsed

parsing done.

end time is 2012-09-16 23:34:19.931000

total time elapsed 0:54:03.102000

repacking file

done

 s:\users\aaj\projects\pythonhf\rfadumptohdf.py(132)generateFiles()

- while single_date = self.end_date:

(Pdb) c

*** 2012-08-08 ***

source file extracted at C:\rfadump\au\2012.08.08.txt

cought an exception while generating file for day 2012-08-08.

Traceback (most recent call last):

  File rfaDumpToHDF.py, line 175, in generateFile

lines = self.rawfile.read().split('|\n')

MemoryError

 

I am very sure that windows system task manager shows the memory usage
as **50 MB** for this process. It looks like the garbage collector or
memory manager for Python is not calculating the free memory correctly.
There should be lot of free memory but it thinks there is not enough. 

 

Any idea?

 

Thanks.

 

 

Alok Jadhav

CREDIT SUISSE AG

GAT IT Hong Kong, KVAG 67

International Commerce Centre | Hong Kong | Hong Kong

Phone +852 2101 6274 | Mobile +852 9169 7172

alok.jad...@credit-suisse.com | www.credit-suisse.com
http://www.credit-suisse.com/ 

 


=== 
Please access the attached hyperlink for an important electronic communications 
disclaimer: 
http://www.credit-suisse.com/legal/en/disclaimer_email_ib.html 
=== 

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Comparing strings from the back?

On Sep 15, 1:10 pm, Dwight Hutto dwightdhu...@gmail.com wrote:
 On Fri, Sep 14, 2012 at 6:43 PM, Prasad, Ramit
  Since I was unsure myself if you were trying to be offensive or racist,
  I would disagree with everyone can know it wasn't meant as racist.

 If you're unsure if it was racist, you should err on the side of
 caution.

If your comments are mistakable as racism, maybe *you* should be more
cautious and *not make them*.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Obnoxious postings from Google Groups (was: datetime issue)

On Sep 17, 10:55 am, Roy Smith r...@panix.com wrote:
 They didn't buy the service.  They bought the data.  Well, they really
 bought both, but the data is all they wanted.

I thought they'd taken most of the historical data offline now too?

Either way, it was the sort of purchase-and-whither approach you
usually see Yahoo take.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python garbage collector/memory manager behaving strangely

2012-09-16 Thread Dave Angel

On 09/16/2012 09:07 PM, Jadhav, Alok wrote:
 Hi Everyone,

  

 I have a simple program which reads a large file containing few million
 rows, parses each row (`numpy array`) and converts into an array of
 doubles (`python array`) and later writes into an `hdf5 file`. I repeat
 this loop for multiple days. After reading each file, i delete all the
 objects and call garbage collector.  When I run the program, First day
 is parsed without any error but on the second day i get `MemoryError`. I
 monitored the memory usage of my program, during first day of parsing,
 memory usage is around **1.5 GB**. When the first day parsing is
 finished, memory usage goes down to **50 MB**. Now when 2nd day starts
 and i try to read the lines from the file I get `MemoryError`. Following
 is the output of the program.

  

  

 source file extracted at C:\rfadump\au\2012.08.07.txt

 parsing started

 current time: 2012-09-16 22:40:16.829000

 50 lines parsed

 100 lines parsed

 150 lines parsed

 200 lines parsed

 250 lines parsed

 300 lines parsed

 350 lines parsed

 400 lines parsed

 450 lines parsed

 500 lines parsed

 parsing done.

 end time is 2012-09-16 23:34:19.931000

 total time elapsed 0:54:03.102000

 repacking file

 done

  s:\users\aaj\projects\pythonhf\rfadumptohdf.py(132)generateFiles()

 - while single_date = self.end_date:

 (Pdb) c

 *** 2012-08-08 ***

 source file extracted at C:\rfadump\au\2012.08.08.txt

 cought an exception while generating file for day 2012-08-08.

 Traceback (most recent call last):

   File rfaDumpToHDF.py, line 175, in generateFile

 lines = self.rawfile.read().split('|\n')

 MemoryError

  

 I am very sure that windows system task manager shows the memory usage
 as **50 MB** for this process. It looks like the garbage collector or
 memory manager for Python is not calculating the free memory correctly.
 There should be lot of free memory but it thinks there is not enough. 

  

 Any idea?

  

 Thanks.

  

  

 Alok Jadhav

 CREDIT SUISSE AG

 GAT IT Hong Kong, KVAG 67

 International Commerce Centre | Hong Kong | Hong Kong

 Phone +852 2101 6274 | Mobile +852 9169 7172

 alok.jad...@credit-suisse.com | www.credit-suisse.com
 http://www.credit-suisse.com/ 

  


Don't blame CPython.  You're trying to do a read() of a large file,
which will result in a single large string.  Then you split it into
lines.  Why not just read it in as lines, in which case the large string
isn't necessary.   Take a look at the readlines() function.  Chances are
that even that is unnecessary, but i can't tell without seeing more of
the code.

  lines = self.rawfile.read().split('|\n')

   lines = self.rawfile.readlines()

When a single large item is being allocated, it's not enough to have
sufficient free space, the space also has to be contiguous.  After a
program runs for a while, its space naturally gets fragmented more and
more.  it's the nature of the C runtime, and CPython is stuck with it.



-- 

DaveA

-- 
http://mail.python.org/mailman/listinfo/python-list

RE: Python garbage collector/memory manager behaving strangely

2012-09-16 Thread Jadhav, Alok

Thanks Dave for clean explanation. I clearly understand what is going on
now. I still need some suggestions from you on this. 

There are 2 reasons why I was using  self.rawfile.read().split('|\n')
instead of self.rawfile.readlines()

- As you have seen, the line separator is not '\n' but its '|\n'.
Sometimes the data itself has '\n' characters in the middle of the line
and only way to find true end of the line is that previous character
should be a bar '|'. I was not able specify end of line using
readlines() function, but I could do it using split() function.
(One hack would be to readlines and combine them until I find '|\n'. is
there a cleaner way to do this?)
- Reading whole file at once and processing line by line was must
faster. Though speed is not of very important issue here but I think the
tie it took to parse complete file was reduced to one third of original
time.

Regards,
Alok


-Original Message-
From: Dave Angel [mailto:d...@davea.name] 
Sent: Monday, September 17, 2012 10:13 AM
To: Jadhav, Alok
Cc: python-list@python.org
Subject: Re: Python garbage collector/memory manager behaving strangely

On 09/16/2012 09:07 PM, Jadhav, Alok wrote:
 Hi Everyone,

  

 I have a simple program which reads a large file containing few
million
 rows, parses each row (`numpy array`) and converts into an array of
 doubles (`python array`) and later writes into an `hdf5 file`. I
repeat
 this loop for multiple days. After reading each file, i delete all the
 objects and call garbage collector.  When I run the program, First day
 is parsed without any error but on the second day i get `MemoryError`.
I
 monitored the memory usage of my program, during first day of parsing,
 memory usage is around **1.5 GB**. When the first day parsing is
 finished, memory usage goes down to **50 MB**. Now when 2nd day starts
 and i try to read the lines from the file I get `MemoryError`.
Following
 is the output of the program.

  

  

 source file extracted at C:\rfadump\au\2012.08.07.txt

 parsing started

 current time: 2012-09-16 22:40:16.829000

 50 lines parsed

 100 lines parsed

 150 lines parsed

 200 lines parsed

 250 lines parsed

 300 lines parsed

 350 lines parsed

 400 lines parsed

 450 lines parsed

 500 lines parsed

 parsing done.

 end time is 2012-09-16 23:34:19.931000

 total time elapsed 0:54:03.102000

 repacking file

 done

 
s:\users\aaj\projects\pythonhf\rfadumptohdf.py(132)generateFiles()

 - while single_date = self.end_date:

 (Pdb) c

 *** 2012-08-08 ***

 source file extracted at C:\rfadump\au\2012.08.08.txt

 cought an exception while generating file for day 2012-08-08.

 Traceback (most recent call last):

   File rfaDumpToHDF.py, line 175, in generateFile

 lines = self.rawfile.read().split('|\n')

 MemoryError

  

 I am very sure that windows system task manager shows the memory usage
 as **50 MB** for this process. It looks like the garbage collector or
 memory manager for Python is not calculating the free memory
correctly.
 There should be lot of free memory but it thinks there is not enough. 

  

 Any idea?

  

 Thanks.

  

  

 Alok Jadhav

 CREDIT SUISSE AG

 GAT IT Hong Kong, KVAG 67

 International Commerce Centre | Hong Kong | Hong Kong

 Phone +852 2101 6274 | Mobile +852 9169 7172

 alok.jad...@credit-suisse.com | www.credit-suisse.com
 http://www.credit-suisse.com/ 

  


Don't blame CPython.  You're trying to do a read() of a large file,
which will result in a single large string.  Then you split it into
lines.  Why not just read it in as lines, in which case the large string
isn't necessary.   Take a look at the readlines() function.  Chances are
that even that is unnecessary, but i can't tell without seeing more of
the code.

  lines = self.rawfile.read().split('|\n')

   lines = self.rawfile.readlines()

When a single large item is being allocated, it's not enough to have
sufficient free space, the space also has to be contiguous.  After a
program runs for a while, its space naturally gets fragmented more and
more.  it's the nature of the C runtime, and CPython is stuck with it.



-- 

DaveA


=== 
Please access the attached hyperlink for an important electronic communications 
disclaimer: 
http://www.credit-suisse.com/legal/en/disclaimer_email_ib.html 
=== 

-- 
http://mail.python.org/mailman/listinfo/python-list

RE: Python garbage collector/memory manager behaving strangely

2012-09-16 Thread Jadhav, Alok

I am thinking of calling a new subprocess which will do the memory
hungry job and then release the memory as specified in the link below

http://stackoverflow.com/questions/1316767/how-can-i-explicitly-free-mem
ory-in-python/1316799#1316799

Regards,
Alok



-Original Message-
From: Dave Angel [mailto:d...@davea.name] 
Sent: Monday, September 17, 2012 10:13 AM
To: Jadhav, Alok
Cc: python-list@python.org
Subject: Re: Python garbage collector/memory manager behaving strangely

On 09/16/2012 09:07 PM, Jadhav, Alok wrote:
 Hi Everyone,

  

 I have a simple program which reads a large file containing few
million
 rows, parses each row (`numpy array`) and converts into an array of
 doubles (`python array`) and later writes into an `hdf5 file`. I
repeat
 this loop for multiple days. After reading each file, i delete all the
 objects and call garbage collector.  When I run the program, First day
 is parsed without any error but on the second day i get `MemoryError`.
I
 monitored the memory usage of my program, during first day of parsing,
 memory usage is around **1.5 GB**. When the first day parsing is
 finished, memory usage goes down to **50 MB**. Now when 2nd day starts
 and i try to read the lines from the file I get `MemoryError`.
Following
 is the output of the program.

  

  

 source file extracted at C:\rfadump\au\2012.08.07.txt

 parsing started

 current time: 2012-09-16 22:40:16.829000

 50 lines parsed

 100 lines parsed

 150 lines parsed

 200 lines parsed

 250 lines parsed

 300 lines parsed

 350 lines parsed

 400 lines parsed

 450 lines parsed

 500 lines parsed

 parsing done.

 end time is 2012-09-16 23:34:19.931000

 total time elapsed 0:54:03.102000

 repacking file

 done

 
s:\users\aaj\projects\pythonhf\rfadumptohdf.py(132)generateFiles()

 - while single_date = self.end_date:

 (Pdb) c

 *** 2012-08-08 ***

 source file extracted at C:\rfadump\au\2012.08.08.txt

 cought an exception while generating file for day 2012-08-08.

 Traceback (most recent call last):

   File rfaDumpToHDF.py, line 175, in generateFile

 lines = self.rawfile.read().split('|\n')

 MemoryError

  

 I am very sure that windows system task manager shows the memory usage
 as **50 MB** for this process. It looks like the garbage collector or
 memory manager for Python is not calculating the free memory
correctly.
 There should be lot of free memory but it thinks there is not enough. 

  

 Any idea?

  

 Thanks.

  

  

 Alok Jadhav

 CREDIT SUISSE AG

 GAT IT Hong Kong, KVAG 67

 International Commerce Centre | Hong Kong | Hong Kong

 Phone +852 2101 6274 | Mobile +852 9169 7172

 alok.jad...@credit-suisse.com | www.credit-suisse.com
 http://www.credit-suisse.com/ 

  


Don't blame CPython.  You're trying to do a read() of a large file,
which will result in a single large string.  Then you split it into
lines.  Why not just read it in as lines, in which case the large string
isn't necessary.   Take a look at the readlines() function.  Chances are
that even that is unnecessary, but i can't tell without seeing more of
the code.

  lines = self.rawfile.read().split('|\n')

   lines = self.rawfile.readlines()

When a single large item is being allocated, it's not enough to have
sufficient free space, the space also has to be contiguous.  After a
program runs for a while, its space naturally gets fragmented more and
more.  it's the nature of the C runtime, and CPython is stuck with it.



-- 

DaveA


=== 
Please access the attached hyperlink for an important electronic communications 
disclaimer: 
http://www.credit-suisse.com/legal/en/disclaimer_email_ib.html 
=== 

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python garbage collector/memory manager behaving strangely

On Sep 17, 12:32 pm, Jadhav, Alok alok.jad...@credit-suisse.com
wrote:
 - As you have seen, the line separator is not '\n' but its '|\n'.
 Sometimes the data itself has '\n' characters in the middle of the line
 and only way to find true end of the line is that previous character
 should be a bar '|'. I was not able specify end of line using
 readlines() function, but I could do it using split() function.
 (One hack would be to readlines and combine them until I find '|\n'. is
 there a cleaner way to do this?)

You can use a generator to take care of your readlines requirements:

def readlines(f):
lines = []
while f is not empty:
line = f.readline()
if not line: break
if len(line)  2 and line[-2:] == '|\n':
lines.append(line)
yield ''.join(lines)
lines = []
else:
lines.append(line)

 - Reading whole file at once and processing line by line was must
 faster. Though speed is not of very important issue here but I think the
 tie it took to parse complete file was reduced to one third of original
 time.

With the readlines generator above, it'll read lines from the file
until it has a complete line by your requirement, at which point
it'll yield it. If you don't need the entire file in memory for the
end result, you'll be able to process each line one at a time and
perform whatever you need against it before asking for the next.

with open(u'infile.txt','r') as infile:
for line in readlines(infile):
...

Generators are a very efficient way of processing large amounts of
data. You can chain them together very easily:

real_lines = readlines(infile)
marker_lines = (l for l in real_lines if l.startswith('#'))
every_second_marker = (l for i,l in enumerate(marker_lines) if (i
+1) % 2 == 0)
map(some_function, every_second_marker)

The real_lines generator returns your definition of a line. The
marker_lines generator filters out everything that doesn't start with
#, while every_second_marker returns only half of those. (Yes, these
could all be written as a single generator, but this is very useful
for more complex pipelines).

The big advantage of this approach is that nothing is read from the
file into memory until map is called, and given the way they're
chained together, only one of your lines should be in memory at any
given time.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Comparing strings from the back?

On Mon, Sep 17, 2012 at 11:11 AM, alex23 wuwe...@gmail.com wrote:
 On Sep 15, 1:10 pm, Dwight Hutto dwightdhu...@gmail.com wrote:
 On Fri, Sep 14, 2012 at 6:43 PM, Prasad, Ramit
  Since I was unsure myself if you were trying to be offensive or racist,
  I would disagree with everyone can know it wasn't meant as racist.

 If you're unsure if it was racist, you should err on the side of
 caution.

 If your comments are mistakable as racism, maybe *you* should be more
 cautious and *not make them*.

That applies to the most obvious examples (for instance, there's a
line in a 19th century opera that uses a six-letter word starting with
'n' to refer to a dark-skinned person - for obvious reasons, that line
is usually changed in modern performances, even though it was
descriptive and not offensive when the opera was written - like
referring to a Caucasian). However, there are many things which could
be misinterpreted as racist, and I would hate to see people leaned
hard on to make their speech entirely politically correct. Use common
sense, on both sides. Don't be offensive, and don't be offended.

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

utcnow

2012-09-16 Thread Nick the Gr33k


Hello is there a better way of writing this:

date = ( datetime.datetime.utcnow() + datetime.timedelta(hours=3) 
).strftime( '%y-%m-%d %H:%M:%S')


something like:

date = datetime.datetime.utcnow(hours=3).strftime( '%y-%m-%d %H:%M:%S')

i prefer it if it could be written as this.

Also what about dayligh savings time?
--
http://mail.python.org/mailman/listinfo/python-list

Re: Python garbage collector/memory manager behaving strangely

2012-09-16 Thread 88888 Dihedral

alex23於 2012年9月17日星期一UTC+8上午11時25分06秒寫道：
 On Sep 17, 12:32 pm, Jadhav, Alok alok.jad...@credit-suisse.com
 
 wrote:
 
  - As you have seen, the line separator is not '\n' but its '|\n'.
 
  Sometimes the data itself has '\n' characters in the middle of the line
 
  and only way to find true end of the line is that previous character
 
  should be a bar '|'. I was not able specify end of line using
 
  readlines() function, but I could do it using split() function.
 
  (One hack would be to readlines and combine them until I find '|\n'. is
 
  there a cleaner way to do this?)
 
 
 
 You can use a generator to take care of your readlines requirements:
 
 
 
 def readlines(f):
 
 lines = []
 
 while f is not empty:
 
 line = f.readline()
 
 if not line: break
 
 if len(line)  2 and line[-2:] == '|\n':
 
 lines.append(line)
 
 yield ''.join(lines)
 
 lines = []
 
 else:
 
 lines.append(line)
 
 
 
  - Reading whole file at once and processing line by line was must
 
  faster. Though speed is not of very important issue here but I think the
 
  tie it took to parse complete file was reduced to one third of original
 
  time.
 
 
 
 With the readlines generator above, it'll read lines from the file
 
 until it has a complete line by your requirement, at which point
 
 it'll yield it. If you don't need the entire file in memory for the
 
 end result, you'll be able to process each line one at a time and
 
 perform whatever you need against it before asking for the next.
 
 
 
 with open(u'infile.txt','r') as infile:
 
 for line in readlines(infile):
 
 ...
 
 
 
 Generators are a very efficient way of processing large amounts of
 
 data. You can chain them together very easily:
 
 
 
 real_lines = readlines(infile)
 
 marker_lines = (l for l in real_lines if l.startswith('#'))
 
 every_second_marker = (l for i,l in enumerate(marker_lines) if (i
 
 +1) % 2 == 0)
 
 map(some_function, every_second_marker)
 
 
 
 The real_lines generator returns your definition of a line. The
 
 marker_lines generator filters out everything that doesn't start with
 
 #, while every_second_marker returns only half of those. (Yes, these
 
 could all be written as a single generator, but this is very useful
 
 for more complex pipelines).
 
 
 
 The big advantage of this approach is that nothing is read from the
 
 file into memory until map is called, and given the way they're
 
 chained together, only one of your lines should be in memory at any
 
 given time.

The basic problem is whether the output items really need 
all lines of the input text file to be buffered to 
produce the results.


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: utcnow

2012-09-16 Thread Joost Molenaar

Time zones! So much fun. Looks like you're dealing with UTC offsets
yourself, which gets messy as soon as you start thinking about DST. The
good news is that there's a timezone database on most systems, which you
should almost definitely use.

Take a look at python-dateutil (pip install python-dateutil):

 from datetime import datetime
 from dateutil import tz
 datetime.now()
datetime.datetime(2012, 9, 17, 6, 33, 57, 158555)

This is a so-called naive datetime, it doesn't know about timezones. On
my system, it returns a time in my local time zone. (It could have been GMT
or something else entirely.) You have to call .replace() on a naive
datetime to set the tzinfo member to make it a timezone-aware datetime
object:

 AMS = tz.gettz('Europe/Amsterdam')
 ATH = tz.gettz('Europe/Athens')
 datetime.now().replace(tzinfo=AMS).astimezone(ATH)
datetime.datetime(2012, 9, 17, 7, 37, 38, 573223,
tzinfo=tzfile('/usr/share/zoneinfo/Europe/Athens'))

Voila, it seems like you're one hour ahead of me. :-)

HTH,

Joost

On 17 September 2012 06:25, Nick the Gr33k nikos.gr...@gmail.com wrote:

 Hello is there a better way of writing this:

 date = ( datetime.datetime.utcnow() + datetime.timedelta(hours=3)
 ).strftime( '%y-%m-%d %H:%M:%S')

 something like:

 date = datetime.datetime.utcnow(**hours=3).strftime( '%y-%m-%d %H:%M:%S')

 i prefer it if it could be written as this.

 Also what about dayligh savings time?
 --
 http://mail.python.org/**mailman/listinfo/python-listhttp://mail.python.org/mailman/listinfo/python-list

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: utcnow

2012-09-16 Thread Ben Finney

Nick the Gr33k nikos.gr...@gmail.com writes:

 Hello is there a better way of writing this:

 date = ( datetime.datetime.utcnow() + datetime.timedelta(hours=3)
 ).strftime( '%y-%m-%d %H:%M:%S')

 something like:

 date = datetime.datetime.utcnow(hours=3).strftime( '%y-%m-%d %H:%M:%S')

 i prefer it if it could be written as this.

Break long complicated statements into simpler statements. You might
need to get used to naming your intermediate results.

now = datetime.datetime.utcnow()
later = now + datetime.timedelta(hours=3)
timestamp_text = later.strftime(%Y-%m-%d %H:%M:%S)

 Also what about dayligh savings time?

What about it? What has your reading of the ‘datetime’ module
documentation taught you?

-- 
 \“Don't worry about people stealing your ideas. If your ideas |
  `\ are any good, you'll have to ram them down people's throats.” |
_o__)—Howard Aiken |
Ben Finney
-- 
http://mail.python.org/mailman/listinfo/python-list

[issue13886] readline-related test_builtin failure

2012-09-16 Thread Arfrever Frehtes Taifersar Arahesis


Changes by Arfrever Frehtes Taifersar Arahesis arfrever@gmail.com:


--
nosy: +Arfrever

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13886
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13888] test_builtin failure when run after test_tk

2012-09-16 Thread Arfrever Frehtes Taifersar Arahesis


Changes by Arfrever Frehtes Taifersar Arahesis arfrever@gmail.com:


--
nosy: +Arfrever

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13888
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15950] open() should not accept bool argument

2012-09-16 Thread Lars Gustäbel


New submission from Lars Gustäbel:

Today I accidentally did this:

open(True).read()

Passing True as a file argument to open() does not fail, because a bool value 
is treated like an integer file descriptor (stdout in this case). Even worse is 
that the read() call hangs in an endless loop on my linux box. On windows I get 
an EBADF at least.

Wouldn't it be better if open() checked explicitly for a bool argument and 
raises a TypeError?

--
components: IO
messages: 170550
nosy: lars.gustaebel
priority: normal
severity: normal
status: open
title: open() should not accept bool argument
type: behavior
versions: Python 3.2, Python 3.3

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15950
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15951] string.Formatter returns str for empty unicode template

2012-09-16 Thread Aleksey Sivokon


New submission from Aleksey Sivokon:

Expected behavior of string.Formatter() is to return unicode strings for 
unicode templates, and byte strings for str templates. Which is exactly what 
it does, with one frustrating exception: for empty unicode string it returns 
byte str. Test follows:

import string
template = u
result = string.Formatter().format(template)
assert isinstance(result, unicode)
# AssertionError

--
components: Library (Lib)
messages: 170551
nosy: Aleksey.Sivokon
priority: normal
severity: normal
status: open
title: string.Formatter returns str for empty unicode template
type: behavior
versions: Python 2.7

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15951
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15951] string.Formatter returns str for empty unicode template


Chris Jerdonek added the comment:

Adding failing test.  Patch coming next.

--
keywords: +patch
nosy: +cjerdonek
stage:  - needs patch
Added file: http://bugs.python.org/file27204/issue-15951-test-1.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15951
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11454] email.message import time


R. David Murray added the comment:

Considering how often that test is done, I would consider the compiled version 
of the short regex the clear winner based on your numbers.  I wonder if we 
could precompile the regex and load it from a pickle.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11454
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15950] open() should not accept bool argument


R. David Murray added the comment:

I don't think so.  We don't try to protect people from all their mistakes, and 
a file descriptor of 1 is valid, and True == 1.

--
nosy: +r.david.murray
resolution:  - wont fix
stage:  - committed/rejected
status: open - closed

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15950
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15951] string.Formatter returns str for empty unicode template


Chris Jerdonek added the comment:

Here are some related failing cases that I found:

 f = string.Formatter()
 f.format(u{0}, )
''
 f.format(u{0}, 1)
'1'
 f.format(u{0}, a)
'a'
 f.format(u{0}{1}, a, b)
'ab'
 f.format({0}, ua) 
u'a'

Note that PEP 3101 says the following:

In all cases, the type of the format string dominates - that
is, the result of the conversion will always result in an object
that contains the same representation of characters as the
input format string.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15951
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15925] email.utils.parsedate(), email.utils.parsedate_tz() and email.utils.parsedate_to_datetime() should return None when date cannot be parsed


R. David Murray added the comment:

Here is a patch.

Georg, this unfortunately represents a regression from 3.2 and should be fixed.

--
keywords: +patch
stage:  - commit review
Added file: http://bugs.python.org/file27205/parsedate_invalid.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15925
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15925] email.utils.parsedate(), email.utils.parsedate_tz() and email.utils.parsedate_to_datetime() should return None when date cannot be parsed


R. David Murray added the comment:

Hmm.  Not sure why Georg didn't get auto-added.

--
nosy: +georg.brandl

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15925
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15925] email.utils.parsedate() and email.utils.parsedate_tz() should return None when date cannot be parsed


R. David Murray added the comment:

Oh, and I'm purposely allowing parsedate_to_datetime throw exceptions.  I 
suppose that should be documented, but that's a separate issue.

--
title: email.utils.parsedate(), email.utils.parsedate_tz() and 
email.utils.parsedate_to_datetime() should return None when date cannot be 
parsed - email.utils.parsedate() and email.utils.parsedate_tz() should return 
None when date cannot be parsed

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15925
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15951] string.Formatter returns str for empty unicode template


Chris Jerdonek added the comment:

Actually, I'm going to defer on creating a patch because this covers more 
scenarios than I originally thought and so may require more time.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15951
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15951] string.Formatter returns str for empty unicode template


R. David Murray added the comment:

Format with unicode is a bit of a mess in 2.7.  It would be consistent with the 
rest of python2 for

   f.format({0}, ua)
  u'a'

to be correct.

See also issue 7300 and issue 15276.

--
nosy: +eric.smith, r.david.murray

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15951
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5057] Unicode-width dependent optimization leads to non-portable pyc file

2012-09-16 Thread STINNER Victor


STINNER Victor added the comment:

I prefer option (1), remove the buggy optimization. Python 3.3 does solve 
correctly this issue.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue5057
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15951] string.Formatter returns str for empty unicode template


Chris Jerdonek added the comment:

What about cases like this?

 f.format(u'{0}', '\xe9')
'\xe9'

It seems fixing this issue for non-empty strings would cause formerly running 
cases like this to raise UnicodeDecodeError.

 unicode('\xe9')
 ...
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe9 in position 0: ordinal 
not in range(128)

Would that be acceptable?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15951
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15695] Correct sizeof support for StgDict


Serhiy Storchaka added the comment:

I don't know where StgDict used. Probably need to provide the correct 
__sizeof__ method for a mapping proxy behind which StgDict hidden.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15695
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15475] Correct sizeof support for itertools


Serhiy Storchaka added the comment:

Are there any objections?

--
nosy: +georg.brandl

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15475
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15490] Correct sizeof support for StringIO


Serhiy Storchaka added the comment:

It would be good if someone looked at the patches. I'm not sure that they are 
good enough. Perhaps they are too complicated and we shouldn't worry about 
duplicates?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15490
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15796] Fix readline() docstrings


Serhiy Storchaka added the comment:

Ping.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15796
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15379] Charmap decoding of no-BMP characters


Serhiy Storchaka added the comment:

Ping.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15379
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15144] Possible integer overflow in operations with addresses and sizes.


Serhiy Storchaka added the comment:

Please, Mark or some other C expert, can you do a code review for the patch?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15144
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15490] Correct sizeof support for StringIO

2012-09-16 Thread Antoine Pitrou


Antoine Pitrou added the comment:

Well, I do think that it's not very important to make __sizeof__ strictly exact 
according to some specification. I'm also not sure we want to complicate 
maintenance when the calculation becomes tedious or difficult.

To me __sizeof__ is mostly an educational tool which gives a hint at the memory 
impact of an object, but cannot really convey a precise information except for 
trivial atomic types.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15490
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15276] unicode format does not really work in Python 2.x


Chris Jerdonek added the comment:

I can't yet reproduce on my system, but after looking at the code, I believe 
the following are closer to the cause:

 format(1, u'n')
 int.__format__(1, u'n')

Incidentally, on my system, the following note in the docs is wrong:

Note: format(value, format_spec) merely calls value.__format__(format_spec).

(from http://docs.python.org/library/functions.html?#format )

 format(1, u'n')
u'1'
 1.__format__(u'n')
  File stdin, line 1
1.__format__(u'n')
   ^
SyntaxError: invalid syntax
 int.__format__(1, u'n')
'1'

Observe also that format() and int.__format__() return different types.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15276
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15951] string.Formatter returns str for empty unicode template


R. David Murray added the comment:

Note that I didn't say it was correct, I just said it was consistent :)

And no, breaking stuff that current works is a non-starter for 2.7.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15951
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15276] unicode format does not really work in Python 2.x

2012-09-16 Thread Eric V. Smith


Eric V. Smith added the comment:

The case with 1.__format__ is confusing the parser. It sees:
floating point number 1. __format__
which is indeed a syntax error.

Try:
 1 .__format__(u'n')
'1'

or:
 (1).__format__(u'n')
'1'

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15276
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15276] unicode format does not really work in Python 2.x