Re: How convert a list string to a real list

[Alec Taylor]

Arnaud: Already showed that solution

On Wed, Nov 30, 2011 at 6:09 PM, Arnaud Delobelle  wrote:
>
> On Nov 30, 2011 6:21 AM, "郭军权"  wrote:
>>
>> Good after
>>     I have a string  liststr = '["","","ccc"]' ,and I need convert
>> it to a list like list = ["","","ccc"],what can id do?
>> Thanks.
>>
>
> Look up the json module.
>
> --
> Arnaud
>
>
> --
> http://mail.python.org/mailman/listinfo/python-list
>
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How convert a list string to a real list

[Arnaud Delobelle]

On Nov 30, 2011 6:21 AM, "郭军权"  wrote:
>
> Good after
> I have a string  liststr = '["","","ccc"]' ,and I need
convert it to a list like list = ["","","ccc"],what can id do?
> Thanks.
>

Look up the json module.

-- 
Arnaud
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How convert a list string to a real list

[Terry Reedy]

On 11/30/2011 1:20 AM, 郭军权 wrote:

Good after
I have a string liststr = '["","","ccc"]' ,and I need convert it
to a list like list = ["","","ccc"],what can id do?


The easiest -- and most dangerous -- way is
>>> eval('["","","ccc"]')
['', '', 'ccc']

But DO NOT eval unexamined strings from untrusted sources. The reason is 
that it is much the same as letting an untrusted person sit unsupervised 
as the keyboard of your computer with a command window open. You would 
not want to eval

  "from os import system; system('')"
where '' is replaced by something obnoxious for your 
operating system.


--
Terry Jan Reedy


--
http://mail.python.org/mailman/listinfo/python-list

Re: How convert a list string to a real list

[Alec Taylor]

import json

s = json.dumps([1, 2, 3, 4])
# '[1, 2, 3, 4]'
l = json.loads(s)
# [1, 2, 3, 4]

2011/11/30 郭军权 :
> Good after
> I have a string  liststr = '["","","ccc"]' ,and I need convert
> it to a list like list = ["","","ccc"],what can id do?
> Thanks.
>
>
> --
> 郭军权
> 清华大学网络中心网络安全实验室
> guojunquan{at}gmail.com
>
>
> --
> http://mail.python.org/mailman/listinfo/python-list
>
-- 
http://mail.python.org/mailman/listinfo/python-list

How convert a list string to a real list

[郭军权]

Good after
I have a string  liststr = '["","","ccc"]' ,and I need convert
it to a list like list = ["","","ccc"],what can id do?
Thanks.


-- 
郭军权
清华大学网络中心网络安全实验室
guojunquan{at}gmail.com
-- 
http://mail.python.org/mailman/listinfo/python-list

Converting MS Access DB to Postgres or MySQL for ORM support with SQLalchemy

[Alec Taylor]

Good afternoon,

I was recently shown that my client runs MS Access everywhere, rather
than a "real" database. There are many features they would be missing
that something like a django frontend would provide.

So I would like to move them to Postgres or MySQL (since MS Access
support was been dropped after 0.4).

But I would also like to move there schema &etc to ORM, for easy
editing with SQLalchemy (or similar).

What approach would you recommend?

Thanks for all suggestions,

Alec Taylor
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: cmd.Cmd asking questions?

[Cameron Simpson]

On 29Nov2011 13:37, Tim Chase  wrote:
| On 11/28/11 06:27, Robert Kern wrote:
[...]
| >I actually have a preference for needing to press enter for
| >Y/N answers, too. It's distinctly *less* uniform to have some
| >questions requiring an enter and some not. It can be
| >unpleasantly surprising to the user, too.
| 
| After playing with it, allowing for a default Y/N value seems to
| make it a one-key selection via , but allow for
| less-surprising behavior as you detail.

As a matter of good practice, I like default responses (especially
yes/no ones) to default to the safer/more-conservative choice, and
generally to default to "no", with the question suitably framed so that
"no" means "safer/more-conservative".

In this way an accidental unthinking  has less chance of doing
damage. What that means depends on context, but I hope you take the point.

(Oh yes, traditionally the default is also capitalised to be evident.)

Example:

  Remove your database now? (y/N)

I would want  to mean "no" here, usually.

Cheers,
-- 
Cameron Simpson  DoD#743
http://www.cskk.ezoshosting.com/cs/

There are old climbers, and there are bold climbers; but there are no old
bold climbers.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Programing Language: latitude-longitude-decimalize

[Thad Floryan]

On 11/29/2011 5:14 PM, Jürgen Exner wrote:
> Thad Floryan  wrote:
>> On 11/29/2011 2:53 PM, Xah Lee wrote:
> 
> Please do not reply to the eternal troll
> 
> Thanks

Mea culpa, you're correct.  I responded only because the subject
is something with which I'm familiar, e.g., one of my posts from
1988 (23 years ago):


and


:-)
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Programing Language: latitude-longitude-decimalize

[J�rgen Exner]

Thad Floryan  wrote:
>On 11/29/2011 2:53 PM, Xah Lee wrote:

Please do not reply to the eternal troll

Thanks

jue
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pythoncom on Windows Server 2008

[Mark Hammond]

On 30/11/2011 11:12 AM, Nairn, Bruce wrote:

Hi,

I’m trying to move some code to a Windows Server 2008 machine. It runs
on Windows 7 and XP, but fails on Windows Server 2008. The python
installation is seemingly identical (python 2.6.4, 32 bit). The
following replicates the problem:

import pythoncom

IDispatch = pythoncom.CoCreateInstance (‘OPC.Automation’, None,
pythoncom.CLSCTX_SERVER, pythoncom.IID_IDispatch)

On Windows 7 x64 or Windows XP x32:

this returns an PyIDispatch object

On Windows Server 2008 x64 this gives:

com_error: (-2147221005, ‘Invalid class string’, None, None)

Any suggestions would be appreciated!


The OPC.Automation object isn't installed on the Server 2008 box.  I'm 
not sure what this object is so can't advise on how to install it.  Note 
also that you will need the same "bittedness" of the object as Python 
itself has - ie, assuming a 32bit Python, you need the 32bit 
implementation of the COM object, or if a 64bit Python, you need the 
64bit COM object.  Note that both the 32 and 64bit versions of both will 
work fine on a 64bit version of Windows - you just need to make sure 
they match.


Mark
--
http://mail.python.org/mailman/listinfo/python-list

pythoncom on Windows Server 2008

[Nairn, Bruce]

Hi,

  I'm trying to move some code to a Windows Server 2008 machine.  It runs on 
Windows 7 and XP, but fails on Windows Server 2008.  The python installation is 
seemingly identical (python 2.6.4, 32 bit).  The following replicates the 
problem:

import pythoncom
IDispatch = pythoncom.CoCreateInstance ('OPC.Automation', None, 
pythoncom.CLSCTX_SERVER, pythoncom.IID_IDispatch)

On Windows 7 x64 or Windows XP x32:

this returns an PyIDispatch object

On Windows Server 2008 x64 this gives:

com_error: (-2147221005, 'Invalid class string', None, None)

Any suggestions would be appreciated!

Bruce Nairn
King County Wastewater Treatment Division
201 S Jackson St., KSC-NR-0503
Seattle, WA, 98104-3855
206-263-3693
email: bruce.na...@kingcounty.gov

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: why is bytearray treated so inefficiently by pickle?

[Irmen de Jong]

On 28-11-2011 3:09, Terry Reedy wrote:
> Possibly. The two developers listed as particularly interested in pickle are
> 'alexandre.vassalotti,pitrou' (antoine), so if you do open a tracker issue, 
> add them as
> nosy.
> 
> Take a look at http://www.python.org/dev/peps/pep-3154/
> by Antoine Pitrou or forwary your message to him.
> 

Created a bug report + patches, http://bugs.python.org/issue13503
I've read the PEP, thanks, it was interesting. But I don't think my changes 
require a
new pickle protocol version bump.

Irmen
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Programing Language: latitude-longitude-decimalize

[Thad Floryan]

On 11/29/2011 2:53 PM, Xah Lee wrote:
> fun programing exercise. Write a function “latitude-longitude-
> decimalize”.
> 
> It should take a string like this: 「"37°26′36.42″N 06°15′14.28″W"」.
> The return value should be a pair of numbers, like this: 「[37.44345
> -6.25396]」.
> 
> Feel free to use perl, python, ruby, lisp, etc. I'll post a emacs lisp
> solution in a couple of days.

What a waste of time when the following works fine (probably Java):



-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Programing Language: latitude-longitude-decimalize

[Ian Kelly]

On Tue, Nov 29, 2011 at 3:53 PM, Xah Lee  wrote:
> fun programing exercise. Write a function “latitude-longitude-
> decimalize”.
>
> It should take a string like this: 「"37°26′36.42″N 06°15′14.28″W"」.
> The return value should be a pair of numbers, like this: 「[37.44345
> -6.25396]」.
>
> Feel free to use perl, python, ruby, lisp, etc. I'll post a emacs lisp
> solution in a couple of days.

For Python 3:

import re

def latitude_longitude_decimalize(string):
regex = 
r"""(\d+)\xb0(\d+)'([\d+.]+)"([NS])\s*(\d+)\xb0(\d+)'([\d+.]+)"([EW])"""
match = re.match(regex, string)
if not match:
raise ValueError("Invalid input string: {0:r}".format(string))
def decimalize(degrees, minutes, seconds, direction):
decimal = int(degrees) + int(minutes) / 60 + float(seconds) / 3600
if direction in 'SW':
decimal = -decimal
return decimal
latitude = decimalize(*match.groups()[:4])
longitude = decimalize(*match.groups()[4:8])
return latitude, longitude
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Programing Language: latitude-longitude-decimalize

[Micky Hulse]

Last time I did this was using AS3. The format I used was DMS:

GPSLongitude: 122,42,47.79
GPSLongitudeRef: W
GPSLatitude: 45,30,30.390001198897014
GPSLatitudeRef: N

Here's the method:



Not shown in above code: If West longitude or South latitude I would
make that DD (decimal degree) value negative.

Anyway, that was a fun learning experience! :)
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Total newbie question: Best practice

[Colin Higwell]

On Tue, 29 Nov 2011 16:57:18 -0500, Dave Angel wrote:

> On 11/29/2011 03:06 PM, Colin Higwell wrote:
>> Hi,
>>
>> I am just starting to learn Python (I have been at it only a few
>> hours), so please bear with me. I have a few very small scripts (do you
>> call them scripts or programs?) which work properly, and produce the
>> results intended.
>>
>> However, they are monolithic in nature; i.e. they begin at the
>> beginning and finish at the end. Having done a little reading, I note
>> that it seems to be quite common to have a function main() at the start
>> (which in turn calls other functions as appropriate), and then to call
>> main() to do the work.
>>
>> Is that standard best practice?
>>
>> Thanks
>>
> Welcome to Python, and to the comp.lang.python list. Is this your first
> experience programming?
> 
> Yes, factoring your code from "monolithic" to "modular' (several
> functions, or even functions and classes), is good practice.
> 
> That's not to say that some problems don't deserve a monolithic answer,
> but if you're a beginner, i'd like to see you get into a modular habit.
> 
> You can use the words script and program pretty much interchangeably.
> in some contexts, each has additional connotations.  For example,
> somebody used to a compiled language may refer to a python source file
> as a script, implying it's not as sophisticated as his own product.
> But, closer to home, we usually refer to the file you directly pass to
> the interpreter as a script, and any added files that get imported, as
> modules, or libraries.
> 
> Other times, people will refer to a simple program as a script, implying
> that all it does is invoke some standard library functions, or even run
> some external programs.  But when it gets more complex, it gradually
> turns into a  "real program."
> 
> 
> Why break up a monolith?
> 
> Several reasons.  If you factor the code into independent functions, and
> give them good names, then each piece of the program is easier to
> understand.  You will especially appreciate that if you come back to the
> code after doing something else for two weeks.  Similarly if somebody
> else has to take over your code, or maybe adapt it to a slightly
> different purpose.
> 
> Next, if it doesn't quite work, you can exercise the individual pieces
> independently, and narrow down the problem more quickly.
> 
> Next, if the progfram is slow, usually you can narrow it down to a few
> key functions that take most of the time.  You can write two versions of
> the same function, and do some careful timings to decide which one to
> use.
> 
> Next, some of those functions may be useful in the next program you
> write.  If you "reuse" the code by copy & paste, and find a problem in
> the new one, it's quite possible that the same problem may exist in your
> first program.  it's easier to bring those changes back if they're in a
> function than if they're in lines 14 through 71.
> 
> Finally, some of that reusable code may be worth moving to an external
> file, called a module.  Then the same copy can be literally shared
> between multiple projects.  This is how libraries were born, and you can
> write your own, eventually.
> 
> there are many other reasons, but some of them might not make sense to
> you yet.
> 
> 
> How do you break it up?
> 
> First, separate the classic parts that most scripts will have.  Put the
> imports at the top, along with a comment describing the whole progfram's
> purpose.  Next put the global variables, which should be few.  If there
> are any constants, use all UPPERCASE for their names.
> 
> Next, put the function and class definitions.  Notice that none of them
> will be called yet, so the order of execution isn't important to the
> compiler.  Each function needs a name, and you should use a name that
> makes sense to you.  Try to write functions that work at a single level
> of complexity, and do one complete operation.  Try not to do
> input/output in the same functions that do the computation.
> 
> And finally, put the actual mainline.  It could be as simple as a call
> to main(), but it may make more sense to you to put the calls to
> argument processing here, rather than in a main function.  By arguments
> here, i'm referring to the stuff you typed on the command line when youi
> invoked the script.  This part of the code is where you do the magic
> incantation:
> 
> if __name__ == "__main__":
>  main()
> 
> 
> 
> When the number of functions gets unwieldy, it's time to move some of
> them to a new file.  They should be related, and should work at the same
> level of complexity.  And the file name should remind you of their
> purpose.  At that point, you add an import of that file to your main
> source script.  Congratulations, you've created a module.
> 
> One catch with writing a lengthy reply is that others have already given
> you good feedback.  Hopefully, mine will complement theirs.

Thank you, and thanks also to all the other respondents. I have 
pro

Programing Language: latitude-longitude-decimalize

[Xah Lee]

fun programing exercise. Write a function “latitude-longitude-
decimalize”.

It should take a string like this: 「"37°26′36.42″N 06°15′14.28″W"」.
The return value should be a pair of numbers, like this: 「[37.44345
-6.25396]」.

Feel free to use perl, python, ruby, lisp, etc. I'll post a emacs lisp
solution in a couple of days.

 Xah
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Total newbie question: Best practice

[Dave Angel]

On 11/29/2011 03:06 PM, Colin Higwell wrote:

Hi,

I am just starting to learn Python (I have been at it only a few hours),
so please bear with me. I have a few very small scripts (do you call them
scripts or programs?) which work properly, and produce the results
intended.

However, they are monolithic in nature; i.e. they begin at the beginning
and finish at the end. Having done a little reading, I note that it seems
to be quite common to have a function main() at the start (which in turn
calls other functions as appropriate), and then to call main() to do the
work.

Is that standard best practice?

Thanks

Welcome to Python, and to the comp.lang.python list. Is this your first 
experience programming?


Yes, factoring your code from "monolithic" to "modular' (several 
functions, or even functions and classes), is good practice.


That's not to say that some problems don't deserve a monolithic answer, 
but if you're a beginner, i'd like to see you get into a modular habit.


You can use the words script and program pretty much interchangeably.  
in some contexts, each has additional connotations.  For example, 
somebody used to a compiled language may refer to a python source file 
as a script, implying it's not as sophisticated as his own product.  
But, closer to home, we usually refer to the file you directly pass to 
the interpreter as a script, and any added files that get imported, as 
modules, or libraries.


Other times, people will refer to a simple program as a script, implying 
that all it does is invoke some standard library functions, or even run 
some external programs.  But when it gets more complex, it gradually 
turns into a  "real program."



Why break up a monolith?

Several reasons.  If you factor the code into independent functions, and 
give them good names, then each piece of the program is easier to 
understand.  You will especially appreciate that if you come back to the 
code after doing something else for two weeks.  Similarly if somebody 
else has to take over your code, or maybe adapt it to a slightly 
different purpose.


Next, if it doesn't quite work, you can exercise the individual pieces 
independently, and narrow down the problem more quickly.


Next, if the progfram is slow, usually you can narrow it down to a few 
key functions that take most of the time.  You can write two versions of 
the same function, and do some careful timings to decide which one to use.


Next, some of those functions may be useful in the next program you 
write.  If you "reuse" the code by copy & paste, and find a problem in 
the new one, it's quite possible that the same problem may exist in your 
first program.  it's easier to bring those changes back if they're in a 
function than if they're in lines 14 through 71.


Finally, some of that reusable code may be worth moving to an external 
file, called a module.  Then the same copy can be literally shared 
between multiple projects.  This is how libraries were born, and you can 
write your own, eventually.


there are many other reasons, but some of them might not make sense to 
you yet.



How do you break it up?

First, separate the classic parts that most scripts will have.  Put the 
imports at the top, along with a comment describing the whole progfram's 
purpose.  Next put the global variables, which should be few.  If there 
are any constants, use all UPPERCASE for their names.


Next, put the function and class definitions.  Notice that none of them 
will be called yet, so the order of execution isn't important to the 
compiler.  Each function needs a name, and you should use a name that 
makes sense to you.  Try to write functions that work at a single level 
of complexity, and do one complete operation.  Try not to do 
input/output in the same functions that do the computation.


And finally, put the actual mainline.  It could be as simple as a call 
to main(), but it may make more sense to you to put the calls to 
argument processing here, rather than in a main function.  By arguments 
here, i'm referring to the stuff you typed on the command line when youi 
invoked the script.  This part of the code is where you do the magic 
incantation:


if __name__ == "__main__":
main()



When the number of functions gets unwieldy, it's time to move some of 
them to a new file.  They should be related, and should work at the same 
level of complexity.  And the file name should remind you of their 
purpose.  At that point, you add an import of that file to your main 
source script.  Congratulations, you've created a module.


One catch with writing a lengthy reply is that others have already given 
you good feedback.  Hopefully, mine will complement theirs.


--

DaveA

--
http://mail.python.org/mailman/listinfo/python-list

Re: Total newbie question: Best practice

[Neil Cerutti]

On 2011-11-29, Arnaud Delobelle  wrote:
> As for the main() function, I don't think it is standard
> practice in Python.  There is no requirement to have a main()
> function.  You can use the idiom:

I don't start off with a main function, but if my script gets
long and complicated, or if global names have proliferated and
have become confusing, I'll refactor the whole thing into
functions, including a "main". With most globals moved into
main's namespace, calling subroutines from main forces me to
define the context that's actually necessary for each part of the
program.

The resultant refactored programs are much easier to test, read
and maintain.

TLDR: "Called-only-once" functions like main are useful as
documentation, hooks for testing, and for unraveling a snarl of
global variables.

-- 
Neil Cerutti
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Total newbie question: Best practice

[Chris Angelico]

On Wed, Nov 30, 2011 at 7:06 AM, Colin Higwell  wrote:
> However, they are monolithic in nature; i.e. they begin at the beginning
> and finish at the end. Having done a little reading, I note that it seems
> to be quite common to have a function main() at the start (which in turn
> calls other functions as appropriate), and then to call main() to do the
> work.

The reason for this practice is to allow your .py file to be either a
top-level program or an imported module.

if __name__ == "__main__":
main()

When you run a .py file directly, __name__ will be "__main__", and
it'll execute main(). (Some programs directly embed the main routine
in that if block - appropriate if main() would be very short, eg just
calling some other function.) But if you import it as a module in some
other program, that won't be the case; so instead, the module's
functions are made available to the calling program.

For simple scripts that don't have anything to offer as a module, it's
fine to not bother with this structure. Python doesn't demand
syntactic salt; that's one of its greatest features, IMHO.

Chris Angelico
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Total newbie question: Best practice

[Arnaud Delobelle]

On 29 November 2011 20:06, Colin Higwell  wrote:
> Hi,

Hi Colin, and welcome to Python :)

> I am just starting to learn Python (I have been at it only a few hours),
> so please bear with me. I have a few very small scripts (do you call them
> scripts or programs?) which work properly, and produce the results
> intended.

I think you can call them either.

> However, they are monolithic in nature; i.e. they begin at the beginning
> and finish at the end. Having done a little reading, I note that it seems
> to be quite common to have a function main() at the start (which in turn
> calls other functions as appropriate), and then to call main() to do the
> work.
>
> Is that standard best practice?

When code should be put in a function is a matter of judgement in the
end, so it's not an easy question.  But roughly speaking:

- if you need to perform the same task at several points in you
program, then it's a good idea to put this in a function.  It avoids
duplication of code, minimises the chances for bugs, makes the program
easier to read (provided you find a nice name for the function!) and
also improves testability.

- if there is a task that could be performed is several different ways
but with the same result, then it's good to put in a function.  This
way when reading the program you can focus on the important thing,
which is the result of the process, without being distracted by the
details of how the result is arrived at.  Moreover, it gives you added
flexibility as you can later try a different method for performing the
same task and very easily plug it into your existing program to see if
it improves performance for example.

- if you have a piece of code which is too long to be understood
easily, consider whether you could break it down into smaller bits,
each of which has some meaning of its own, and make each bit into a
function with a name that describes clearly what it does.  Then
rewrite your big piece of code in terms of these functions.  It will
make your program a lot easier to understand when you come back to it
in the future.

As for the main() function, I don't think it is standard practice in
Python.  There is no requirement to have a main() function.  You can
use the idiom:

if __name__ == "__main__":
...

which will execute if you call the file as a script (as opposed to
importing it as a module)

-- 
Arnaud
-- 
http://mail.python.org/mailman/listinfo/python-list

Total newbie question: Best practice

[Colin Higwell]

Hi,

I am just starting to learn Python (I have been at it only a few hours), 
so please bear with me. I have a few very small scripts (do you call them 
scripts or programs?) which work properly, and produce the results 
intended.

However, they are monolithic in nature; i.e. they begin at the beginning 
and finish at the end. Having done a little reading, I note that it seems 
to be quite common to have a function main() at the start (which in turn 
calls other functions as appropriate), and then to call main() to do the 
work. 

Is that standard best practice?

Thanks

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: cmd.Cmd asking questions?

[Tim Chase]

On 11/28/11 06:27, Robert Kern wrote:

On 11/28/11 12:12 PM, Tim Chase wrote:

I can monkey with printing messages and using raw_input(),
but I'd like to know if there's a better way (such as
something interacting with readline for
text-entry-with-history-and-completion,


If you import readline, then any following uses of
raw_input() will automatically use readline. You may want to
swap out the history when you use get_string() or confirm() so
they don't mess up the regular Cmd history, but the basic
functionality should work out-of-box.


I didn't realize raw_input() was so nicely overloaded.  After 
about 30 minutes of fighting with various bits of code (and 
learning that pdb's readline doesn't save/restore history), I 
tweaked up some "save the history; restore the history" wrapper 
which worked well.



or raw-character input for Y/N answers rather than the need
to hit, making it feel more uniform),


I actually have a preference for needing to press enter for
Y/N answers, too. It's distinctly *less* uniform to have some
questions requiring an enter and some not. It can be
unpleasantly surprising to the user, too.


After playing with it, allowing for a default Y/N value seems to 
make it a one-key selection via , but allow for 
less-surprising behavior as you detail.


Thanks for your input (no pun intended),

-tkc


--
http://mail.python.org/mailman/listinfo/python-list

70% [* SPAM *] multiprocessing.Queue blocks when sending large object

[DPalao]

Hello,
I'm trying to use multiprocessing to parallelize a code. There is a number of 
tasks (usually 12) that can be run independently. Each task produces a numpy 
array, and at the end, those arrays must be combined.
I implemented this using Queues (multiprocessing.Queue): one for input and 
another for output.
But the code blocks. And it must be related to the size of the item I put on 
the Queue: if I put a small array, the code works well; if the array is 
realistically large (in my case if can vary from 160kB to 1MB), the code 
blocks apparently forever.
I have tried this:
http://www.bryceboe.com/2011/01/28/the-python-multiprocessing-queue-and-large-
objects/
but it didn't work (especifically I put a None sentinel at the end for each 
worker).

Before I change the implementation, 
is there a way to bypass this problem with  multiprocessing.Queue?
Should I post the code (or a sketchy version of it)?

TIA,

David
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Can I submit an issue with Python 2.5.6?

[Jerry Hill]

2011/11/29 Toshiyuki Ogura 

>  I found a problem with Python 2.5.6.
>
...

> Can I submit the issue at bugs.python.org?
> I think many people are still using Python 2.5 because of Google App
> Engine and fixing bugs with 2.5 is still helpful.
>

I don't think they'll be accepted.  Python 2.5 is not slated to receive any
more releases, ever.  Not even source-only security fixes.  According to
http://www.python.org/getit/releases/2.5.6/, "This release is the final
release of Python 2.5; under the current release policy, no security issues
in Python 2.5 will be fixed anymore."  Given that they're not even
accepting security patches, I'm sure they won't accept non-security
bugfixes either.

If your goal is to get your patch accepted and deployed on google's app
engine infrastructure, you could try getting a hold of someone at google,
and asking if they accept patches directly, since there will be no further
maintenance from the core python developers.  I have no idea how difficult
it would be to do that, or if there's any interest in accepting patches
against python 2.5 at google.

-- 
Jerry
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pragmatics of the standard is() function

[Den]

On Nov 29, 12:41 am, Steven D'Aprano  wrote:
> On Mon, 28 Nov 2011 11:22:09 -0800, Den wrote:
> > With respect, I disagree with advice that the use of a language
> > construct should be rare.  All constructs should be used
> > *appropriately*.
>
> And if those appropriate conditions are rare, then the use of the
> appropriate construct should also be rare. Since it is rare to need to
> care about identity, use of `is` should be rare too.
>
> This is mostly descriptive statement rather than a prescriptive one: it
> is a fact that use of `is` is much less common than use of ==, and even
> more so if you disregard the obvious case of "x is None".
>
> I am however being a little prescriptive: if somebody thinks they need to
> care about identity, chances are good -- but not certain -- that they're
> mistaken. In code that gives you the choice between writing `is` and ==,
> it is usually, but not often, a bug to write `is`. The proof of this is
> that in actual, existing code, if you swapped == and `is`, most of the
> code would stop working correctly.
>
> On the other hand if you actually do need to care about identity, then go
> for it. I'm not saying that caring about identity should be prohibited,
> only that in practice it is uncommon that you will.
>
> > While in general a particular use of a Python construct may be rare, if
> > the programmer is involved deeply with that rare use, then it is NOT
> > rare to him/her.
>
> Regardless of how common brain surgery might be to a brain surgeon, it is
> rare in general. That's all I'm saying. If you happen to work for a data
> recovery centre, then pulling the disk platter out of a hard disk drive
> might be commonplace to you, nevertheless pulling platters out of disks
> is vanishingly rare: out of tens of thousands of HDDs, perhaps only a few
> dozen will be sent to a recover centre at all.
>
> [...]
>
> > Sorry, you plucked a string of mine.  One does not throw a tool out of
> > your tool box because it might be dangerous.  Table saws are incredibly
> > dangerous, but in the hands of a skilled operator can be competently and
> > safely used to produce beautiful woodwork.  To say *never* use a table
> > saw because it's dangerous is silly.
>
> Fortunately, I didn't say "never".
>
> In the case of `is`, one shouldn't avoid using `is` because it is
> dangerous, but merely because it is the wrong thing to use. In the same
> way, the average programmer will rarely need to use the math.sinh and
> math.cosh functions, not because they are bad, evil, or dangerous, but
> because they are used for specialist purposes that most coders would
> never care about. Hell, most coders rarely use the standard trigonometric
> functions sin and cos, let alone the hyperbolic versions! This is not a
> value judgement.
>
> If I, in a misplaced sense of egalitarianism ("Emancipation for maths
> functions! Right on brothers!") decided to use sinh instead of sin
> because they looked and sounded similar, my code would almost certain be
> buggy. Likewise if I decided to use `is` instead of == because in English
> they have similar meanings, my code would probably be buggy too.
>
> Sorry to belabour the point, but we're in violent agreement here 
>
> --
> Steven

((laugh)) Yes, I believe we are.

D
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Amateur question on class definitions...

[Ian Kelly]

On Tue, Nov 29, 2011 at 8:21 AM, J. Marc Edwards  wrote:
> So I am defining my Python classes for a Django data model.  I have a the
> following class hierarchy and would value some expert input on how to best
> implement this.
>
> I have many instances of a particular class which I call a "workflow", e.g.
> wf_1, wf_2, wf_3, etc.
> These class instances of workflows can be combined into "composite
> workflows", e.g. {cwf_1 is comprised of wf1 & wf_3}, {cwf_2 is wf_2 and
> wf_3}, or {cwf_3 is just wf_2}, etc.
> The workflows that constitute the composite workflow is positionally
> dependent, i.e. if cwf_1 is comprised of wf_1 and wf_3, then wf_1 comes 1st
> AND wf_3 is 2nd.
> As I have many workflow instances that accumulate in my persistent database
> (Django data model), then the composite workflows become an ordered
> collection of workflow instances.
>
> My first thought here is that the composite workflow class should have a
> field that is a simple list of workflows, i.e. list(workflows), but I am
> somewhat unsure of how to code this correctly.
>
> Here is a sample pseudo-code...
>
> class a_workflow(models.Model):
> ...
>
> class composite_workflow(models.Model):
>     ...a list of "a_workflows", i.e. a composite workflow, should I use this
> syntax?
>     set_of_workflows = list(a_workflow)
>
> Is this how a pro would do this?

That would work in general, but since these are Django models I assume
that this relationship will need to be stored in the database.  You'll
need to use a ManyToManyField instead.  I would do something like
this:

class Workflow(models.Model):
pass

class CompositeWorkflow(models.Model):
workflow_set = models.ManyToManyField('Workflow',
through='CompositeWorkflowOrder')

class CompositeWorkflowOrder(models.Model):
workflow = models.ForeignKey('Workflow')
composite_workflow = models.ForeignKey('CompositeWorkflow')
sequence_num = models.IntegerField()

The CompositeWorkflowOrder model is then responsible for tracking
which workflows are contained by which composite workflows, and in
what order they appear (using sequence_num).  See the Django docs on
ManyToManyFields for more info.

Cheers,
Ian
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Amateur question on class definitions...

[Chris Angelico]

On Wed, Nov 30, 2011 at 2:21 AM, J. Marc Edwards  wrote:
>     ...a list of "a_workflows", i.e. a composite workflow, should I use this
> syntax?
>     set_of_workflows = list(a_workflow)
>

This would be usual:

set_of_workflows = [a_workflow]

Using the list() constructor directly is for when you have some other
iterable; for instance, a string is iterable over its characters:

>>> list("Hello")
['H', 'e', 'l', 'l', 'o']

When you want to wrap up a single object in a list, square brackets
syntax is what you want.

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Amateur question on class definitions...

[J. Marc Edwards]

So I am defining my Python classes for a Django data model.  I have a
the following class hierarchy and would value some expert input on how
to best implement this.

I have many instances of a particular class which I call a "workflow",
e.g. wf_1, wf_2, wf_3, etc.
These class instances of workflows can be combined into "composite
workflows", e.g. {cwf_1 is comprised of wf1 & wf_3}, {cwf_2 is wf_2 and
wf_3}, or {cwf_3 is just wf_2}, etc.
The workflows that constitute the composite workflow is positionally
dependent, i.e. if cwf_1 is comprised of wf_1 and wf_3, then wf_1 comes
1st AND wf_3 is 2nd.
As I have many workflow instances that accumulate in my persistent
database (Django data model), then the composite workflows become an
ordered collection of workflow instances.

My first thought here is that the composite workflow class should have a
field that is a simple list of workflows, i.e. list(workflows), but I am
somewhat unsure of how to code this correctly.

Here is a sample pseudo-code...

class a_workflow(models.Model):
...

class composite_workflow(models.Model):
...a list of "a_workflows", i.e. a composite workflow, should I use
this syntax?
set_of_workflows = list(a_workflow)

Is this how a pro would do this?

Regards, Marc
-- 

J. Marc Edwards
Lead Architect - Semiconductor Design Portals
Nimbis Services, Inc.
Skype: (919) 747-3775
Cell:  (919) 345-1021
Fax:   (919) 882-8602
marc.edwa...@nimbisservices.com
www.nimbisservices.com

-- 
http://mail.python.org/mailman/listinfo/python-list

RE: Can I submit an issue with Python 2.5.6?

[Toshiyuki Ogura]

I mistakenly posted the previous message in html format, and some rubbish text 
'&nbs p;' was unexpectedly inserted in the patch.
I'll post the content of the patch again.
Sorry for the inconvenience.


--- Python-2.5.6/Lib/test/test_commands.py.orig    2006-06-29 
04:10:08.0 +
+++ Python-2.5.6/Lib/test/test_commands.py    2011-11-29 08:46:29.602607019 
+
@@ -46,11 +46,7 @@
 # drwxr-xr-x   15 Joe User My Group 4096 Aug 12 12:50 /
 # Note that the first case above has a space in the group name
 # while the second one has a space in both names.
-    pat = r'''d.   # It is a directory.
-  \+?  # It may have ACLs.
-  \s+\d+   # It has some number of links.
-  [^/]*    # Skip user, group, size, and date.
-  /\.  # and end with the name of the file.
+    pat = r'''^.*(\/\.)[\ ]*[\n\r]*$
    '''
 
 self.assert_(re.match(pat, getstatus("/."), re.VERBOSE))






Toshiyuki Ogura

  
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: suppressing bad characters in output PCDATA (converting JSON to XML)

[Stefan Behnel]

Adam Funk, 29.11.2011 13:57:

On 2011-11-28, Stefan Behnel wrote:

Adam Funk, 25.11.2011 14:50:

Then I recurse through the contents of big_json to build an instance
of xml.dom.minidom.Document (the recursion includes some code to
rewrite dict keys as valid element names if necessary)


If the name "big_json" is supposed to hint at a large set of data, you may
want to use something other than minidom. Take a look at the
xml.etree.cElementTree module instead, which is substantially more memory
efficient.


Well, the input file in this case contains one big JSON list of
reasonably sized elements, each of which I'm turning into a separate
XML file.  The output files range from 600 to 6000 bytes.


It's also substantially easier to use, but if your XML writing code works 
already, why change it.




and I save the document:

xml_file = codecs.open(output_fullpath, 'w', encoding='UTF-8', errors='replace')
doc.writexml(xml_file, encoding='UTF-8')
xml_file.close()


Same mistakes as above. Especially the double encoding is both unnecessary
and likely to fail. This is also most likely the source of your problems.


Well actually, I had the problem with the occasional control
characters in the output *before* I started sticking encoding="UTF-8"
all over the place (in an unsuccessful attempt to beat them down).


You should read up on Unicode a bit.



I thought this would force all the output to be valid, but xmlstarlet
gives some errors like these on a few documents:

PCDATA invalid Char value 7
PCDATA invalid Char value 31


This strongly hints at a broken encoding, which can easily be triggered by
your erroneous encode-and-encode cycles above.


No, I've checked the JSON input and those exact control characters are
there too.


Ah, right, I didn't look closely enough. Those are forbidden in XML:

http://www.w3.org/TR/REC-xml/#charsets

It's sad that minidom (apparently) lets them pass through without even a 
warning.




I want to suppress them (delete or replace with spaces).


Ok, then you need to process your string content while creating XML from 
it. If replacing is enough, take a look at string.maketrans() in the string 
module and str.translate(), a method on strings. Or maybe just use a 
regular expression that matches any whitespace character and replace it 
with a space. Or whatever suits your data best.




Also, the kind of problem you present here makes it pretty clear that you
are using Python 2.x. In Python 3, you'd get the appropriate exceptions
when trying to write binary data to a Unicode file.


Sorry, I forgot to mention the version I'm using, which is "2.7.2+".


Yep, Py2 makes Unicode handling harder than it should be.

Stefan

--
http://mail.python.org/mailman/listinfo/python-list

Re: Executing .pyc using python c api

[88888 Dihedral]

On Tuesday, November 29, 2011 5:02:31 PM UTC+8, Ulrich Eckhardt wrote:
> Am 29.11.2011 08:34, schrieb Mrinalini Kulkarni:
> > I need to run .pyc files using python c api. if i do PyImport_Import it
> > executes the script. However, i need to pass a set of variables and
> > their values which will be accessed from within the script. How can this
> > be done.
> 
> I don't think what you want is possible, due to a think-o in your 
> design. Let me explain...
> Firstly, .pyc files are basically the same as .py files, only in a 
> different presentation. Then, PyImport_Import is basically the same as 
> using "import" in a Python program. Now, and that is where your fault 
> lies, importing a module actually means executing that module! For 
> example, the definition of a function is code that when executed will 
> cause a function to be created and attached to the current scope with 
> the according name. This is what makes it so easy to implement local 
> functions that are parametrized by arguments to the outer function. 
> Still, a function is not something that is "static", like in C or Java, 
> but rather the result of executing its function definition.
> 
> Now, how to get around this? The specialty about the import is that the 
> __name__ attribute is not set to "__main__", upon which many scripts 
> already react. So, in order to "prevent execution" (in the sense that 
> you probably mean), you simply wrap the according code in a function. 
> The function definition will then be executed, giving you a function 
> that you can call with the according parameters, but the function itself 
> will not be executed automatically. If you want that to happen when 
> executing the .pyc file directly, check the content of __name__ and call 
> the function if it is "__main__".
> 
> Note that another approach would be introspection, traversing through 
> the namespaces to find out those parameters, but I would consider this 
> solution as hackish if the one above is feasible.
> 
> Good luck!
> 
> Uli

Please use psyco and pyrex and C or whatever that can read saved results in a 
file, or just learn how to replace a hash or a sort in  python's build in 
library of better speed, don't do reference overheads in 
those c type variables that won't overflow and underflow  and used by other 
objects in python. Not trivial but well documented to cheer for a race!

 
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Can I submit an issue with Python 2.5.6?

[Matt Joiner]

Note the re.VERBOSE flag allows this whitespace treatment of the pattern.

2011/11/29 Toshiyuki Ogura :
> Hi.
>
> I found a problem with Python 2.5.6.
> test_commands fails when 'make test'.
> bugs.python.org doesn't seem to have an option for Python 2.5 in "Versions:"
> drop-down menu when creating an issue.
> The problem seems to be the same as #11946.
> http://bugs.python.org/issue11946
> I also made a patch for 2.5.6.
>
> --- Python-2.5.6/Lib/test/test_commands.py.orig    2006-06-29
> 04:10:08.0 +
> +++ Python-2.5.6/Lib/test/test_commands.py    2011-11-29 08:46:29.602607019
> +
> @@ -46,11 +46,7 @@
>  # drwxr-xr-x   15 Joe User My Group 4096 Aug 12 12:50 /
>  # Note that the first case above has a space in the group name
>  # while the second one has a space in both names.
> -     pat = r'''d.   # It is a directory.
> -  \+?  # It may have ACLs.
> -  \s+\d+   # It has some number of links.
> -  [^/]*    # Skip user, group, size, and date.
> -  /\.  # and end with the name of the file.
> +    pat = r'''^.*(\/\.)[\ ]*[\n\r]*$
>   &nbs p; '''
>
>  self.assert_(re.match(pat, getstatus("/."), re.VERBOSE))
>
>
> Can I submit the issue at bugs.python.org?
> I think many people are still using Python 2.5 because of Google App Engine
> and fixing bugs with 2.5 is still helpful.
>
> Toshiyuki Ogura
>
>
> --
> http://mail.python.org/mailman/listinfo/python-list
>
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: python 2.5 and ast

[Andrea Crotti]

On 11/29/2011 11:32 AM, Steven D'Aprano wrote:


I prefer to check against sys.version.

import sys
if sys.version<= '2.5':
 from psi.devsonly.ast import parse, NodeVisitor
else:
 from ast import parse, NodeVisitor



Or even:


try:
 from ast import parse, NodeVisitor
except ImportError:
 from ast import parse, NodeVisitor





The try/except is probably the nicest...
I've seen in other places using sys.version <=, but is it a good idea to 
rely

on the fact that the <= on strings has the right semantic?
After all that's just a string, version_info looks more correct to me..
--
http://mail.python.org/mailman/listinfo/python-list

Re: suppressing bad characters in output PCDATA (converting JSON to XML)

[Adam Funk]

On 2011-11-28, Steven D'Aprano wrote:

> On Fri, 25 Nov 2011 13:50:01 +, Adam Funk wrote:
>
>> I'm converting JSON data to XML using the standard library's json and
>> xml.dom.minidom modules.  I get the input this way:
>> 
>> input_source = codecs.open(input_file, 'rb', encoding='UTF-8',
>> errors='replace') big_json = json.load(input_source)
>> input_source.close()
>> 
>> Then I recurse through the contents of big_json to build an instance of
>> xml.dom.minidom.Document (the recursion includes some code to rewrite
>> dict keys as valid element names if necessary), 
>
> How are you doing that? What do you consider valid?

Regex-replacing all whitespace ('\s+') with '_', and adding 'a_' to
the beginning of any potential tag that doesn't start with a letter.
This is good enough for my purposes.

>> I thought this would force all the output to be valid, but xmlstarlet
>> gives some errors like these on a few documents:
>
> It will force the output to be valid UTF-8 encoded to bytes, not 
> necessarily valid XML.

Yes!

>> PCDATA invalid Char value 7
>> PCDATA invalid Char value 31
>
> What's xmlstarlet, and at what point does it give this error? It doesn't 
> appear to be in the standard library.

It's a command-line tool I use a lot for finding the bad bits in XML,
nothing to do with python.

http://xmlstar.sourceforge.net/

>> I guess I need to process each piece of PCDATA to clean out the control
>> characters before creating the text node:
>> 
>>   text = doc.createTextNode(j)
>>   root.appendChild(text)
>> 
>> What's the best way to do that, bearing in mind that there can be
>> multibyte characters in the strings?
>
> Are you mixing unicode and byte strings?

I don't think I am.

> Are you sure that the input source is actually UTF-8? If not, then all 
> bets are off: even if the decoding step works, and returns a string, it 
> may contain the wrong characters. This might explain why you are getting 
> unexpected control characters in the output: they've come from a badly 
> decoded input.

I'm pretty sure that the input is really UTF-8, but has a few control
characters (fairly rare).

> Another possibility is that your data actually does contain control 
> characters where there shouldn't be any.

I think that's the problem, and I'm looking for an efficient way to
delete them from unicode strings.


-- 
Some say the world will end in fire; some say in segfaults.
 [XKCD 312]
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: suppressing bad characters in output PCDATA (converting JSON to XML)

[Adam Funk]

On 2011-11-28, Stefan Behnel wrote:

> Adam Funk, 25.11.2011 14:50:
>> I'm converting JSON data to XML using the standard library's json and
>> xml.dom.minidom modules.  I get the input this way:
>>
>> input_source = codecs.open(input_file, 'rb', encoding='UTF-8', 
>> errors='replace')
>
> It doesn't make sense to use codecs.open() with a "b" mode.

OK, thanks.

>> big_json = json.load(input_source)
>
> You shouldn't decode the input before passing it into json.load(), just 
> open the file in binary mode. Serialised JSON is defined as being UTF-8 
> encoded (or BOM-prefixed), not decoded Unicode.

So just do
  input_source = open(input_file, 'rb')
  big_json = json.load(input_source)
?

>> input_source.close()
>
> In case of a failure, the file will not be closed safely. All in all, use 
> this instead:
>
>  with open(input_file, 'rb') as f:
>  big_json = json.load(f)

OK, thanks.

>> Then I recurse through the contents of big_json to build an instance
>> of xml.dom.minidom.Document (the recursion includes some code to
>> rewrite dict keys as valid element names if necessary)
>
> If the name "big_json" is supposed to hint at a large set of data, you may 
> want to use something other than minidom. Take a look at the 
> xml.etree.cElementTree module instead, which is substantially more memory 
> efficient.

Well, the input file in this case contains one big JSON list of
reasonably sized elements, each of which I'm turning into a separate
XML file.  The output files range from 600 to 6000 bytes.


>> and I save the document:
>>
>> xml_file = codecs.open(output_fullpath, 'w', encoding='UTF-8', 
>> errors='replace')
>> doc.writexml(xml_file, encoding='UTF-8')
>> xml_file.close()
>
> Same mistakes as above. Especially the double encoding is both unnecessary 
> and likely to fail. This is also most likely the source of your problems.

Well actually, I had the problem with the occasional control
characters in the output *before* I started sticking encoding="UTF-8"
all over the place (in an unsuccessful attempt to beat them down).


>> I thought this would force all the output to be valid, but xmlstarlet
>> gives some errors like these on a few documents:
>>
>> PCDATA invalid Char value 7
>> PCDATA invalid Char value 31
>
> This strongly hints at a broken encoding, which can easily be triggered by 
> your erroneous encode-and-encode cycles above.

No, I've checked the JSON input and those exact control characters are
there too.  I want to suppress them (delete or replace with spaces).

> Also, the kind of problem you present here makes it pretty clear that you 
> are using Python 2.x. In Python 3, you'd get the appropriate exceptions 
> when trying to write binary data to a Unicode file.

Sorry, I forgot to mention the version I'm using, which is "2.7.2+".


-- 
In the 1970s, people began receiving utility bills for
-£999,999,996.32 and it became harder to sustain the 
myth of the infallible electronic brain.  (Stob 2001)
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: python 2.5 and ast

[Steven D'Aprano]

On Tue, 29 Nov 2011 09:51:24 +, Andrea Crotti wrote:

> On 11/29/2011 03:55 AM, Dave Angel wrote:
>>
>> But don't forget to tag it as version specific, so it gets removed when
>> the later version of the library is available.  There are various ways
>> of doing that, but the easiest is probably to put a test in the
>> acceptance suite that fails if this code is used in 2.6 or later.
>>
>>
> Ok thanks,
> something like this is ok?
> (or maybe I can use a try/catch and export my own if is not found in the
> standard library?)
> 
> from sys import version_info
> 
> if version_info[1] == 5:
>  from psi.devsonly.ast import parse, NodeVisitor
> else:
>  from ast import parse, NodeVisitor


I prefer to check against sys.version.

import sys
if sys.version <= '2.5':
from psi.devsonly.ast import parse, NodeVisitor
else:
from ast import parse, NodeVisitor



Or even:


try:
from ast import parse, NodeVisitor
except ImportError:
from ast import parse, NodeVisitor



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Can I submit an issue with Python 2.5.6?

[Toshiyuki Ogura]

Hi.

I found a problem with Python 2.5.6.
test_commands fails when 'make test'.
bugs.python.org doesn't seem to have an option for Python 2.5 in "Versions:" 
drop-down menu when creating an issue.
The problem seems to be the same as #11946.
http://bugs.python.org/issue11946
I also made a patch for 2.5.6.

--- Python-2.5.6/Lib/test/test_commands.py.orig2006-06-29 
04:10:08.0 +
+++ Python-2.5.6/Lib/test/test_commands.py2011-11-29 08:46:29.602607019 
+
@@ -46,11 +46,7 @@
 # drwxr-xr-x   15 Joe User My Group 4096 Aug 12 12:50 /
 # Note that the first case above has a space in the group name
 # while the second one has a space in both names.
-pat = r'''d.   # It is a directory.
-  \+?  # It may have ACLs.
-  \s+\d+   # It has some number of links.
-  [^/]*# Skip user, group, size, and date.
-  /\.  # and end with the name of the file.
+pat = r'''^.*(\/\.)[\ ]*[\n\r]*$
'''
 
 self.assert_(re.match(pat, getstatus("/."), re.VERBOSE))


Can I submit the issue at bugs.python.org?
I think many people are still using Python 2.5 because of Google App Engine and 
fixing bugs with 2.5 is still helpful.

Toshiyuki Ogura

  -- 
http://mail.python.org/mailman/listinfo/python-list

Re: python 2.5 and ast

[Arnaud Delobelle]

On 29 November 2011 09:51, Andrea Crotti  wrote:

> from sys import version_info
>
> if version_info[1] == 5:
>    from psi.devsonly.ast import parse, NodeVisitor
> else:
>    from ast import parse, NodeVisitor

Why don't you just:

try:
from ast import parse, NodeVisitor
except ImportError:
from psi.devsonly.ast import parse, NodeVisitor

-- 
Arnaud
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: python 2.5 and ast

[Andrea Crotti]

On 11/29/2011 03:55 AM, Dave Angel wrote:


But don't forget to tag it as version specific, so it gets removed 
when the later version of the library is available.  There are various 
ways of doing that, but the easiest is probably to put a test in the 
acceptance suite that fails if this code is used in 2.6 or later.




Ok thanks,
something like this is ok?
(or maybe I can use a try/catch and export my own if is not found in the 
standard library?)


from sys import version_info

if version_info[1] == 5:
from psi.devsonly.ast import parse, NodeVisitor
else:
from ast import parse, NodeVisitor

--
http://mail.python.org/mailman/listinfo/python-list

Re: Executing .pyc using python c api

[Ulrich Eckhardt]

Am 29.11.2011 08:34, schrieb Mrinalini Kulkarni:

I need to run .pyc files using python c api. if i do PyImport_Import it
executes the script. However, i need to pass a set of variables and
their values which will be accessed from within the script. How can this
be done.


I don't think what you want is possible, due to a think-o in your 
design. Let me explain...
Firstly, .pyc files are basically the same as .py files, only in a 
different presentation. Then, PyImport_Import is basically the same as 
using "import" in a Python program. Now, and that is where your fault 
lies, importing a module actually means executing that module! For 
example, the definition of a function is code that when executed will 
cause a function to be created and attached to the current scope with 
the according name. This is what makes it so easy to implement local 
functions that are parametrized by arguments to the outer function. 
Still, a function is not something that is "static", like in C or Java, 
but rather the result of executing its function definition.


Now, how to get around this? The specialty about the import is that the 
__name__ attribute is not set to "__main__", upon which many scripts 
already react. So, in order to "prevent execution" (in the sense that 
you probably mean), you simply wrap the according code in a function. 
The function definition will then be executed, giving you a function 
that you can call with the according parameters, but the function itself 
will not be executed automatically. If you want that to happen when 
executing the .pyc file directly, check the content of __name__ and call 
the function if it is "__main__".


Note that another approach would be introspection, traversing through 
the namespaces to find out those parameters, but I would consider this 
solution as hackish if the one above is feasible.


Good luck!

Uli
--
http://mail.python.org/mailman/listinfo/python-list

Re: Using the Python Interpreter as a Reference

[Dave Angel]

On 11/29/2011 03:12 AM, Steven D'Aprano wrote:

On Tue, 29 Nov 2011 13:57:32 +1100, Chris Angelico wrote:


I'm inclined toward an alternative: explicit recursion. Either a
different syntax, or a special-case on the use of the function's own
name, but whichever syntax you use, it compiles in a "recurse" opcode.
That way, if name bindings change, it's still going to recurse -
something few languages guarantee, and therefore few languages can
optimize.

As I recall, Forth uses (or used) a special RECURSE word which turned on
the recursion bit while compiling, so that the compiled word could see
itself.

By memory, the (incomplete) definition:

: fact dup 1- fact * ;

would fail, unless you happened to already have another word called fact
existing at compilation time. To make it recurse correctly, the compiler
needs to make sure that the namespace fact sees includes itself:

RECURSE : fact dup 1- fact * ;

which should work, apart from the embarrassing fact that I don't recall
the syntax for conditional jumps and so the recursion never terminates.

:)

The way I remember it, the current definition was "smudged" which made 
it invisible (it basically changed the name to something unlikely) 
during the compilation.  After all, if you actually ran it at compile 
time (which was frequently done), you could fall right into 
uninitialized space.  Anyway, some implementations had an immediate 
SMUDGE word, which toggled the smudge bit and made it visible again.


Other implementations had an immediate word RECURSE, which compiled in 
whatever word was being currently defined.
I'm pretty sure neither FIG nor Forth79 had either of these.  But I 
don't recall the ANSI standard (X3J14 ?), even though I was officially 
an observer.  I can't even remember what happened to my printed copy of 
the standard.


The easiest word for conditional is IF/ELSE/THEN.   IF will skip to the 
ELSE or THEN if the condition is false.  So something resembling:


: fact dup 1- dup 0<> if recurse * then ;

might do it.  That's very rough, however.  It's been a long time.

--

DaveA

--
http://mail.python.org/mailman/listinfo/python-list

Re: Pragmatics of the standard is() function

[Steven D'Aprano]

On Mon, 28 Nov 2011 11:22:09 -0800, Den wrote:

> With respect, I disagree with advice that the use of a language
> construct should be rare.  All constructs should be used
> *appropriately*.

And if those appropriate conditions are rare, then the use of the 
appropriate construct should also be rare. Since it is rare to need to 
care about identity, use of `is` should be rare too.

This is mostly descriptive statement rather than a prescriptive one: it 
is a fact that use of `is` is much less common than use of ==, and even 
more so if you disregard the obvious case of "x is None". 

I am however being a little prescriptive: if somebody thinks they need to 
care about identity, chances are good -- but not certain -- that they're 
mistaken. In code that gives you the choice between writing `is` and ==, 
it is usually, but not often, a bug to write `is`. The proof of this is 
that in actual, existing code, if you swapped == and `is`, most of the 
code would stop working correctly. 

On the other hand if you actually do need to care about identity, then go 
for it. I'm not saying that caring about identity should be prohibited, 
only that in practice it is uncommon that you will.


> While in general a particular use of a Python construct may be rare, if
> the programmer is involved deeply with that rare use, then it is NOT
> rare to him/her.

Regardless of how common brain surgery might be to a brain surgeon, it is 
rare in general. That's all I'm saying. If you happen to work for a data 
recovery centre, then pulling the disk platter out of a hard disk drive 
might be commonplace to you, nevertheless pulling platters out of disks 
is vanishingly rare: out of tens of thousands of HDDs, perhaps only a few 
dozen will be sent to a recover centre at all.

[...]
> Sorry, you plucked a string of mine.  One does not throw a tool out of
> your tool box because it might be dangerous.  Table saws are incredibly
> dangerous, but in the hands of a skilled operator can be competently and
> safely used to produce beautiful woodwork.  To say *never* use a table
> saw because it's dangerous is silly.

Fortunately, I didn't say "never".

In the case of `is`, one shouldn't avoid using `is` because it is 
dangerous, but merely because it is the wrong thing to use. In the same 
way, the average programmer will rarely need to use the math.sinh and 
math.cosh functions, not because they are bad, evil, or dangerous, but 
because they are used for specialist purposes that most coders would 
never care about. Hell, most coders rarely use the standard trigonometric 
functions sin and cos, let alone the hyperbolic versions! This is not a 
value judgement.

If I, in a misplaced sense of egalitarianism ("Emancipation for maths 
functions! Right on brothers!") decided to use sinh instead of sin 
because they looked and sounded similar, my code would almost certain be 
buggy. Likewise if I decided to use `is` instead of == because in English 
they have similar meanings, my code would probably be buggy too.

Sorry to belabour the point, but we're in violent agreement here 



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Executing .pyc using python c api

[Stefan Behnel]

Mrinalini Kulkarni, 29.11.2011 08:34:

I need to run .pyc files using python c api. if i do PyImport_Import it
executes the script. However, i need to pass a set of variables and their
values which will be accessed from within the script. How can this be done.


Assuming you have the source as well, you should change your script to be 
usable as an importable module as well as an executable script. Here is how:


http://docs.python.org/tutorial/modules.html#executing-modules-as-scripts

If you do not have the source, however, you are a bit out of luck. In that 
case, what so you mean by "pass a set of variables"? Do you mean command 
line arguments, or do you want to pass in global names that the module can 
use? In the first case, you may need to modify sys.argv. If the latter, you 
could change the globals() of the imported module. However, both are clumsy 
solutions, and if possible at all, you should change the module source code 
instead.


Stefan

--
http://mail.python.org/mailman/listinfo/python-list

Re: Using the Python Interpreter as a Reference

[Steven D'Aprano]

On Tue, 29 Nov 2011 13:57:32 +1100, Chris Angelico wrote:

> I'm inclined toward an alternative: explicit recursion. Either a
> different syntax, or a special-case on the use of the function's own
> name, but whichever syntax you use, it compiles in a "recurse" opcode.
> That way, if name bindings change, it's still going to recurse -
> something few languages guarantee, and therefore few languages can
> optimize.

As I recall, Forth uses (or used) a special RECURSE word which turned on 
the recursion bit while compiling, so that the compiled word could see 
itself.

By memory, the (incomplete) definition:

: fact dup 1- fact * ;

would fail, unless you happened to already have another word called fact 
existing at compilation time. To make it recurse correctly, the compiler 
needs to make sure that the namespace fact sees includes itself:

RECURSE : fact dup 1- fact * ;

which should work, apart from the embarrassing fact that I don't recall 
the syntax for conditional jumps and so the recursion never terminates.

:)


-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Using the Python Interpreter as a Reference

[Steven D'Aprano]

On Tue, 29 Nov 2011 12:49:49 +1100, Chris Angelico wrote:

> On Tue, Nov 29, 2011 at 11:54 AM, DevPlayer  wrote:
>> To me, I would think the interpreter finding the coder's intended
>> indent wouldn't be that hard. And just make the need for consistant
>> spaces or tabs irrevelent simply by reformatting the indent as
>> expected. Pretty much all my text editors can.
> 
> The trouble with having a language declaration that "a tab is equivalent
> to X spaces" is that there's no consensus as to what X should be.
> Historically X has always been 8, and quite a few programs still assume
> this. I personally like 4. Some keep things narrow with 2. You can even
> go 1 - a strict substitution of \t with \x20. Once you declare it in
> your language, you immediately break everyone who uses anything
> different.

Why should we enforce how many spaces a tab is set to? That is literally 
only visible to the editor, not the compiler. (Unless the parser is 
particularly stupid and merely substitutes X spaces for a tab every time 
it sees one.)

Mixed spaces and tabs in an indent are ambiguous, and so raises 
SyntaxError (or at least it *should* raise SyntaxError). But separate 
code blocks can use different absolute indent levels without ambiguity, 
so long as the relative indentation is consistent:

def ham(): # tab stops at 4 and 8
do(a)
do(b)
if c:
do(c)
else:
do(d)


def spam(): # tab stops at 8 and 12
do(a)
do(b)
if c:
do(c)
else:
do(d)

There is no meaningful difference in indentation between ham and spam 
above. In either case, I could replace spaces with tabs to get the same 
relative indents regardless of the absolute indentation.

I can appreciate the aesthetic argument that *within a single file*, 
indentation should be consistent. But it's not logically necessary for 
the parser: it need only be consistent within a single code unit 
(function or class usually). In other words: syntax should only care 
about relative indentation, absolute indentation belongs as a coding 
standard.



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Executing .pyc using python c api

[Mrinalini Kulkarni]

Hi

I need to run .pyc files using python c api. if i do PyImport_Import it 
executes the script. However, i need to pass a set of variables and 
their values which will be accessed from within the script. How can this 
be done.


thanks,

--
http://mail.python.org/mailman/listinfo/python-list