Re: staticmethod and namespaces

[darnzen]

On Feb 26, 10:20 am, "Diez B. Roggisch"  wrote:
> Am 26.02.10 17:08, schrieb Diez B. Roggisch:
>
>
>
>
>
> > Am 26.02.10 16:57, schrieb darnzen:
> >> On Feb 26, 9:41 am, "Diez B. Roggisch" wrote:
> >>> Am 26.02.10 16:32, schrieb darnzen:
>
>  On Feb 26, 3:15 am, "Diez B. Roggisch" wrote:
> > Am 26.02.10 06:07, schrieb darnzen:
>
> >> Having an odd problem that I solved, but wondering if its the best
> >> solution (seems like a bit of a hack).
>
> >> First off, I'm using an external DLL that requires static callbacks,
> >> but because of this, I'm losing instance info. It could be import
> >> related? It will make more sense after I diagram it:
>
> >> #Module main.py
> >> from A import *
>
> >> class App:
> >> def sperg(self):
> >> self.a = A()
>
> >> app = App()
> >> [main loop and such]
> >> -
> >> # Module A.py
> >> import main
> >> class Foo:
> >> Selves=[]
> >> def __init__(self):
> >> Foo.Selves.append(self)
> >> @staticmethod
> >> def chum_callback(nType, nP):
> >> # Need to access function / data in app instance
> >> app.sperg(nP)
> >> # Need to access func data in Foo
> >> # I'm pulling 'self' ouf of list made in constructor
> >> self = Foo.getSelf(nP)
>
> >> def getSelf(nP):
> >> return self.Selves[nP]
>
> >> -
> >> So basically I added a list of instances to the base class so I can
> >> get at them from the staticmethod.
> >> What's bothering me the most is I can't use the global app
> >> instance in
> >> the A.py module.
>
> >> How can I get at the app instance (currently I'm storing that along
> >> with the class instance in the constructor)?
> >> Is there another way to do this that's not such a hack?
>
> >> Sorry for the double / partial post :(
>
> > Can you show how you pass the staticmethod to the C-function? Is
> > the DLL
> > utilized by ctypes?
>
> > I don't see any reason you couldn't use a bound method, which would
> > give
> > you your self, instead relying on global state.
>
> > Diez
>
>  __main__.K<< *facepalm* should of tried that!
>
>  Yeah I'm using ctypes. The DLL callback set ups are as follows. The
>  local callback is in the App namespace (in this case, some callbacks
>  are in different modules as noted in OP), but still no access to self:
>
>  #Function wrapper
>  A.expCallback = WINFUNCTYPE(None, c_int, c_int, \
>  POINTER(Data_s))(A.Callback)
>
>  #DLL call to register the local callback function
>  DLLSetCallback(self.hID, A.SubID, EVENTID, A.expCallback)
>
>  class A:
>  #Local callback function
>  @staticmethod
>  def Callback(hID, SubID, Data):
>  print 'I DON'T KNOW WHO I AM OR WHERE I CAME FROM!!'
>  print 'BUT WITH hID, and SubID, I CAN FIGURE IT OUT'
>  print 'IF I STORE A REFERENCE TO MYSELF IN A DICT'
>  print 'USING KEY GENERATED FROM hID, SubID'
>  pass
>
>  I'm not sure why they need to be static callbacks, but the DLL doc's
>  say "when using object based languages, such as c++, callback
>  functions must be declared as static functions and not instance
>  methods", and I couldn't get it to work without setting it up that
>  way. I could probably have them all be "classless" functions, but with
>  100's of these, my namespace would be polluted up the wazoo, and I'd
>  still have the problem that they wouldn't have access to instance
>  methods / properties.
>
> >>> The above code can't work with self, because you use
>
> >>> A.expCallback
>
> >>> which at best can of course be a classmethod.
>
> >>> You need to instead invoke DLLSetCallback with a bound method, like this
>
> >>> a = A()
> >>> DLLSetCallback(self.hID, A.SubID, EVENTID, a.expCallback)
>
> >>> Also, the DLL-docs seem to refer to *C* or *C++*, where the concept of
> >>> static functions is differently. If ctypes manages to get *some*
> >>> callback passed, I'm 100% positive that it can pass *any* callable you
> >>> like, including bound methods.
>
> >>> Diez
>
> >> Thinking about it some more, I believe I understand why it has to be
> >> staticfunction. To use an bound method would require the wrapper to
> >> include a reference to the instance as follows:
>
> >> A.expCallback = WINFUNCTYPE(None, POINTER(A), c_int, c_int, \
> >> POINTER(Data_s))(a.Callback)
>
> >> Since a = A(); a.foo() is really A.foo(self). The problem here is that
> >> A is not a ctypes object and I can't change what arguments the DLL
> >> uses in the callback in any case. Rewording my thoughts: a bound
> >> method callback would require 'self' to be the first argument. I can
> >> not make the DLL include 'self' as it doesn't know anything about the
> >> objects in my program. Since I can't pass 'self', it has to be a
> >> staticmethod.
>
> > N

mobiletrickS

[raghumech]

FREE CALL  TO ANYWHERE .. http://mobiletricks777.blogspot.com/
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: A more specific query ...

[Gib Bogle]

Dennis Lee Bieber wrote:

On Sat, 27 Feb 2010 13:00:32 +1300, Gib Bogle
 declaimed the following in
gmane.comp.python.general:

The PyQt4 problem results from having copies of the Qt DLLs in directories that 
are in the PATH, as Doug Bell discovered.  In my case I have two programs that 
use Qt, AMD CodeAnalyst and Matlab.  If I rename BOTH these directories I can 
import the PyQt4 modules.



And where are those PATH entries relative to the Python ones? You
may have to add the Python location to PATH /before/ the AMD and Matlab
entries. {I routinely -- once a year or so -- copy the PATH environment
variable to an editor, and reorder it so that the stuff I use most
occurs earlier in the list}


Well diagnosed!  The others were all system environment variables, while the 
PyQt4 PATH is mine, and therefore it was placed at the end.  I've taken the AMD 
and Matlab PATH entries out of the system list and put them after PyQt4 in my 
PATH.  Now peace and harmony are restored to my Python world.


Thanks
Gib
--
http://mail.python.org/mailman/listinfo/python-list

Re: DreamPie - The Python shell you've always dreamed about!

[alex23]

Mensanator  wrote:
> "You're" not getting the point.

If every link has to be accompanied by a summary of all of the
information at the end of it, what point is there to linking?

(Programmers are the _only_ people I know of who complain about the
arduousness of tasks like typing quotes or clicking on links...)
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Variable definition

[Dave Angel]

Steven D'Aprano wrote:



for key, value in varDic.iteritems():
varDic['myPrefix_' + key] = value
del varDic[key]


Watch out if any of the existing values already startswith 'myPrefix'  
You can end up with trouble just as confusing as if 'myPrefix' is an 
empty string


DaveA
--
http://mail.python.org/mailman/listinfo/python-list

Re: Docstrings considered too complicated

[Steven D'Aprano]

On Fri, 26 Feb 2010 21:51:17 -0600, Tim Daneliuk wrote:

> The only possible exception to this I can think of is when there is some
> non-obvious side-effect (i.e. language and/or hardware is
> "misfeatured"):
> 
> mov   A,B; Moving A into B also will also arm
>; the nuclear warhead if the CPU is
>; hotter than 110C

I had an embedded device that did *just that*, but only on Tuesdays.



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Docstrings considered too complicated

[Steven D'Aprano]

On Sat, 27 Feb 2010 03:25:47 +, MRAB wrote:

>> Also, some assemblies perform the move in different directions
>> according to the arguments. So you might have:
>> 
>> mv AX,BX  ; move contents of BX into AX mv @CX,DX ; move contents of
>> @CX into DX
>> 
>> Horrible, yes, but apparently some assembly languages did something
>> like that.
>> 
> Ah, yes, "apparently". That's like saying "someone down the pub told
> me...". ;-)

Not down the pub, somebody on Usenet. Are you trying to suggest that 
people might propagate falsehoods on the Internet???


-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Docstrings considered too complicated

[Steven D'Aprano]

On Sat, 27 Feb 2010 00:54:53 +, MRAB wrote:

> The assembly languages of virtually all the processors that I've come
> across put the destination first, eg. x86:

Incorrect. x86 assembly has two distinct syntax branches, "Intel 
syntax" (which is most common in the Windows world according to 
Wikipedia) and "AT&T syntax".

See for example:

http://gcc.gnu.org/onlinedocs/gnat_ugn_unw/Basic-Assembler-Syntax.html


-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to end TCP socket data while using readline()?

[Cameron Simpson]

On 26Feb2010 10:39, Arjun  wrote:
| Hi, I have a small script that runs a TCP server. A client connects to
| this server and transmits a stored file line-by-line, and then waits
| for a confirmation "done". However, when I run them the first loop
| never really ends -- as the TCP server keeps expecting more data. I am
| using a file-like-object, and so somehow I have to communicate to the
| server that it is the end-of-file.
| 
| here is some server code
| 
| sock1.bind(('', port))
| print "Listening at port: ", port
| sock1.listen(1)  # listening socket
| (conn, addr) = sock1.accept()# connected socket
| print 'Client (localhost) port: ', addr[1]
| 
| cf = conn.makefile('r',0)# file like obj for socket
[...]
| print 'close'
| cf.flush()
| cf.close()
| sfp.close()
[...]

Too many files. It's not that hard! Or shouldn't be.

| So what I am wondering is:
| 
| 1. Using a file-like object means that the socket becomes uni-
| directional, until the mode of the file object is changed from 'r' to
| 'w' (or vice versa). This seems inefficient, and rather unPythonesque.
| Can somebody tell me if there is a more elegant way of receiving all
| the lines from the client, and then sending a "done" message to the
| client?

Get the socket. It is a file descriptor (or in may be a python "socket"
object with a file descriptor inside).

Open _two_ "file" objects on it using
  from_file = os.fdopen(fd_of_socket, "r")
  to_file = os.fdopen(fd_of_socket, "w").

Use the same:
  print >>to_file, 'close'
  to_file.flush()
method as you're using already.
Read from to_file as needed.

The same scheme should work in both server and client:

Don't look for EOF, watch the input line flow.

You might need to use readline() instead of the file-by-line iteration stuff,
which I seem to recall has some sort of problem handing out the "latest"
line.

Cheers,
-- 
Cameron Simpson  DoD#743
http://www.cskk.ezoshosting.com/cs/

It's better, when you're riding with someone you don't know so well, to stick
to the inside line - it's easier to avoid the bits...
- Barry Sheene
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Docstrings considered too complicated

[Tim Daneliuk]

On 2/26/2010 9:25 PM, MRAB wrote:
> Steven D'Aprano wrote:
>> On Fri, 26 Feb 2010 18:47:26 -0600, John Bokma wrote:
>>
>>> Steven D'Aprano  writes:
>>>
 On Fri, 26 Feb 2010 09:09:36 -0600, Tim Daneliuk wrote:

> Reminiscent of:
>
> mov  AX,BX   ; Move the contents of BX into AX

 That's a *good* comment, because without it most English-speaking
 people would assume you were moving the contents of AX into BX.
>>> Eh? It's a bad comment, it's of the same quality as:
>>>
 x = x + 1  # Add one to x.
>>> You think the former is good because (I guess) you are not familiar with
>>> the language. The same reason why beginners comment their code like in
>>> your example.
>>
>> Well, in the second example, x+1 could not possibly be anything other
>> than adding one to x in any language, programming or human: using "+"
>> for addition is close enough to universal these days. Unless x is some
>> fancy object that plays operator overloading tricks, then what else
>> could x+1 be?
>>
>> On the other hand, mv AX,BX is not only ambiguous but in the example
>> given the move occurs in the counter-intuitive direction, at least for
>> English-speakers and probably most speakers of European languages. So
>> even an experienced coder might like the reminder that this specific
>> assembly language moves things backwards compared to his intuition, or
>> compared to the way the rest of the language does things.
>>
>> Admittedly an experienced coder might be so used to that syntax that
>> he no longer needs the comment, in which case he might find it
>> unnecessary. But comments aren't written for the benefit of people who
>> find it obvious. They're written for everyone else. ANY comment could
>> be dismissed as unnecessary for somebody who is so familiar with the
>> language and code in question that it is obvious what it is doing.
>>
>> Also, some assemblies perform the move in different directions
>> according to the arguments. So you might have:
>>
>> mv AX,BX  ; move contents of BX into AX
>> mv @CX,DX ; move contents of @CX into DX
>>
>> Horrible, yes, but apparently some assembly languages did something
>> like that.
>>
> Ah, yes, "apparently". That's like saying "someone down the pub told
> me...". ;-)

It's interesting that my silly example raised this amount of
commentary. My general view about all this is that comments should be
written on the assumption that the reader knows the language in
question. It is not the purpose of a comment to provide a tutorial on
the language (unless you're actually writing a tutorial program).

Rather, a comment should illuminate the *logic* of the activity by
either amplifying, clarifying, or otherwise providing a rationale' for
the code in question. Demanding that programmers provide comments
about the syntax and semantics of the language is - to me - absurd.

For example, some time ago, I was doing some programming with embedded
PIC-based hardware. The comments for that code are intended to
illuminate how the *hardware* was being exploited, not how to use PIC
asm:

...

scan:
incfcurrent_col,F   ; pickup next column to scan
movlw   MAX_COL+1   ; if current_col > MAX_COL then
subwf   current_col,W   ; we just did last column, so start over
btfss   STATUS,Z; zero means we need to start over
gotokey_scan; nope, go look for key hits
clrfcurrent_col ; yes, reinit column counter
gotoscan

...



The only possible exception to this I can think of is when there is
some non-obvious side-effect (i.e. language and/or hardware is
"misfeatured"):

mov A,B; Moving A into B also will also arm
   ; the nuclear warhead if the CPU is
   ; hotter than 110C


-- 

Tim Daneliuk tun...@tundraware.com
PGP Key: http://www.tundraware.com/PGP/

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Docstrings considered too complicated

[D'Arcy J.M. Cain]

On 27 Feb 2010 00:02:40 GMT
Steven D'Aprano  wrote:
> On Fri, 26 Feb 2010 09:09:36 -0600, Tim Daneliuk wrote:
> > mov  AX,BX   ; Move the contents of BX into AX
> 
> That's a *good* comment, because without it most English-speaking people 
> would assume you were moving the contents of AX into BX.

But it isn't English, it's assembler.   If someone doesn't know that
assembler they have no business trying to work in it.

A good comment would be "move number of sheep into accumulator" or
something equally as descriptive.  A person who knows the assembler
knows that the BX register will be copied (not "moved" btw) into the AX
register.  What they want from the comment is some idea why.

Yes, understanding assembler is hard.  You aren't going to learn it
from the comments.

-- 
D'Arcy J.M. Cain  |  Democracy is three wolves
http://www.druid.net/darcy/|  and a sheep voting on
+1 416 425 1212 (DoD#0082)(eNTP)   |  what's for dinner.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: importing libraries not working 2.6.4

[Rami Chowdhury]

On Feb 26, 2010, at 19:47 , Isaiah Coberly wrote:

> C:\OpenCV2.0\Python2.6\Lib\site-packages
> 
> no .py files here..
> 
> cv.pyd
> libcv.dll.a
> 
> 
> C:\OpenCV2.0\Python2.6\Lib\site-packages\opencv
> 
> _init_.py
> matlan_syntax.py
> adaptors.py
> cv.py
> 

It looks to me like 'opencv' is structured as a package, from which you should 
be able to import and work with Python libraries. Have you tried putting the 
above site-packages directory in sys.path:

import sys; sys.path.append('C:\\OpenCV2.0\\Python2.6\\Lib\\site-packages')

and then doing:

from opencv import cv

?

> Did I still need to compile this even though it was a win 32 installer?

I don't think so, no -- AFAIK the .pyd files are compiled CPython extension 
modules, so it looks like they're all compiled for you.

-
Rami Chowdhury
"Never assume malice when stupidity will suffice." -- Hanlon's Razor
408-597-7068 (US) / 07875-841-046 (UK) / 0189-245544 (BD)




> 
> 
> On Fri, Feb 26, 2010 at 6:12 PM, Rami Chowdhury  
> wrote:
> On Friday 26 February 2010 17:42:04 Isaiah Coberly wrote:
> > Thanks for the reply.
> >
> > I tried putting the files from
> >
> >  C:\OpenCV2.0\Python2.6\Lib
> >
> > too
> >
> > C:\Python26\Lib\site-packages
> >
> > and Python still wont import..
> >
> > I adjusted the environment variables to try and import maya.standalone but
> > no dice on that either.
> >
> 
> Sorry, you've still not told me where you expect it to be. Could you let us
> know what .py files you can see in C:\OpenCV2.0\Python2.6\Lib ?
> 
> >
> > On Fri, Feb 26, 2010 at 4:30 PM, Rami Chowdhury
> wrote:
> > > On Friday 26 February 2010 16:06:56 Routb3d wrote:
> > > > I used the x86 Python 2.6.4 installer. I am working in x64 Win7 pro.
> > > >
> > > > I have tried adjusting environment variables as well as putting the
> > > > files directly in the search path of Python.. Python still returns
> > > > this.. Any ideas?
> > >
> > > Where are you expecting the cv module / package to be? It's certainly not
> > > in
> > > the standard library -- at least not in my install of Python 2.6.4...
> > >
> > > > >>> import sys
> > > > >>> sys.path
> > > >
> > > > ['C:\\Python26\\Lib\\idlelib', 'C:\\Program Files\\Autodesk\\Maya2010\
> > > > \Python\\Lib\\site-packages', 'C:\\Windows\\system32\\python26.zip',
> > > > 'C:\\Python26\\DLLs', 'C:\\Python26\\lib', 'C:\\Python26\\lib\\plat-
> > > > win', 'C:\\Python26\\lib\\lib-tk', 'C:\\Python26', 'C:\\Python26\\lib\
> > > > \site-packages']
> > > >
> > > > >>> import cv
> > > >
> > > > Traceback (most recent call last):
> > > >   File "", line 1, in 
> > > >
> > > > import cv
> > > >
> > > > ImportError: DLL load failed: The specified module could not be found.
> > > >
> > > >
> > > > Any help would be greatly appreciated.
> > >
> > > 
> > > Rami Chowdhury
> > > "Strangers are just friends who haven't had enough gin." -- Howdle's
> > > Saying 408-597-7068 (US) / 07875-841-046 (UK) / 01819-245544 (BD)
> 
> 
> 
> Rami Chowdhury
> "Ninety percent of everything is crap." -- Sturgeon's Law
> 408-597-7068 (US) / 07875-841-046 (UK) / 01819-245544 (BD)
> 

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Variable definition

[Steven D'Aprano]

On Fri, 26 Feb 2010 15:32:27 -0800, Raphael Mayoraz wrote:

> Hello,
> 
> I'd like to define variables with some specific name that has a common
> prefix.
> Something like this:
> 
> varDic = {'red': 'a', 'green': 'b', 'blue': 'c'}
> for key, value in varDic.iteritems():
> 'myPrefix' + key = value
> 
> I know this is illegal, but there must be a trick somewhere.


The Right answer is:

Don't do that. How can you use a variable without knowing its name?

Suppose you do this:

for key, value in varDic.iteritems():
'myPrefix_' + key = value

and then later you want:

print myPrefix_blue

But hang on. How could you possibly know it is called myPrefix_blue and 
not myPrefix_cyan or myPrefix_turquoise?

You can't possibly know, so in general defining a named variable is 
useless. What you want instead is:

some_colour = 'blue'  # get this at runtime
print varDic[some_colour]

You don't even need the myPrefix, but if you really want it:

for key, value in varDic.iteritems():
varDic['myPrefix_' + key] = value
del varDic[key]

some_colour = 'blue'  # get this at runtime
print varDic['myPrefix_' + some_colour]

or any one of a million variations on this idea.

For some very, very rare situations you might need to programatically 
define variables. The right way to do that is:

globals()['myPrefix_turquoise'] = 42

or 

setattr(module, 'myPrefix_turquoise'] = 42

or even:

exec "'myPrefix_turquoise' = 42"

but if you are thinking of using that last one with data coming from an 
untrusted user (and nearly all users are untrusted!), then DON'T. That is 
a huge security hole. And it's also very slow.



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Variable definition

[John Posner]

On 2/26/2010 10:20 PM, Steven D'Aprano wrote:

On Fri, 26 Feb 2010 20:15:16 -0500, John Posner wrote:


On 2/26/2010 6:32 PM, Raphael Mayoraz wrote:

Hello,

I'd like to define variables with some specific name that has a common
prefix.
Something like this:

varDic = {'red': 'a', 'green': 'b', 'blue': 'c'} for key, value in
varDic.iteritems(): 'myPrefix' + key = value



No trick, just swap a new key-value pair for each existing pair:

for key, value in varDic.iteritems():
varDic[myPrefix + key] = value
del varDict[key]

Just make sure that *myPrefix* isn't an empty string!


How does that answer the original poster's question?

Admittedly, your solution is the Right Way To Do It, but what the OP
wants is to go from a dict {'spam': 42} to a named variable myPrefixspam
= 42, which is a totally bogus thing to do, but incredibly common among
n00bs and refugees from horrible languages that allow that sort of
abomination.


Yup, I misinterpreted the OP's intent. I guess my mind just couldn't 
encompass the enormity of his intended transgression.


-John

--
http://mail.python.org/mailman/listinfo/python-list

Re: Docstrings considered too complicated

[MRAB]

Steven D'Aprano wrote:

On Fri, 26 Feb 2010 18:47:26 -0600, John Bokma wrote:


Steven D'Aprano  writes:


On Fri, 26 Feb 2010 09:09:36 -0600, Tim Daneliuk wrote:


Reminiscent of:

mov  AX,BX   ; Move the contents of BX into AX


That's a *good* comment, because without it most English-speaking
people would assume you were moving the contents of AX into BX.

Eh? It's a bad comment, it's of the same quality as:


x = x + 1  # Add one to x.

You think the former is good because (I guess) you are not familiar with
the language. The same reason why beginners comment their code like in
your example.


Well, in the second example, x+1 could not possibly be anything other 
than adding one to x in any language, programming or human: using "+" for 
addition is close enough to universal these days. Unless x is some fancy 
object that plays operator overloading tricks, then what else could x+1 
be?


On the other hand, mv AX,BX is not only ambiguous but in the example 
given the move occurs in the counter-intuitive direction, at least for 
English-speakers and probably most speakers of European languages. So 
even an experienced coder might like the reminder that this specific 
assembly language moves things backwards compared to his intuition, or 
compared to the way the rest of the language does things.


Admittedly an experienced coder might be so used to that syntax that he 
no longer needs the comment, in which case he might find it unnecessary. 
But comments aren't written for the benefit of people who find it 
obvious. They're written for everyone else. ANY comment could be 
dismissed as unnecessary for somebody who is so familiar with the 
language and code in question that it is obvious what it is doing.


Also, some assemblies perform the move in different directions according 
to the arguments. So you might have:


mv AX,BX  ; move contents of BX into AX
mv @CX,DX ; move contents of @CX into DX

Horrible, yes, but apparently some assembly languages did something like 
that.



Ah, yes, "apparently". That's like saying "someone down the pub told
me...". ;-)
--
http://mail.python.org/mailman/listinfo/python-list

Re: Variable definition

[Steven D'Aprano]

On Fri, 26 Feb 2010 20:15:16 -0500, John Posner wrote:

> On 2/26/2010 6:32 PM, Raphael Mayoraz wrote:
>> Hello,
>>
>> I'd like to define variables with some specific name that has a common
>> prefix.
>> Something like this:
>>
>> varDic = {'red': 'a', 'green': 'b', 'blue': 'c'} for key, value in
>> varDic.iteritems(): 'myPrefix' + key = value
>>
>>
> No trick, just swap a new key-value pair for each existing pair:
> 
>for key, value in varDic.iteritems():
>varDic[myPrefix + key] = value
>del varDict[key]
> 
> Just make sure that *myPrefix* isn't an empty string!

How does that answer the original poster's question?

Admittedly, your solution is the Right Way To Do It, but what the OP 
wants is to go from a dict {'spam': 42} to a named variable myPrefixspam 
= 42, which is a totally bogus thing to do, but incredibly common among 
n00bs and refugees from horrible languages that allow that sort of 
abomination.



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Docstrings considered too complicated

[Steven D'Aprano]

On Fri, 26 Feb 2010 18:47:26 -0600, John Bokma wrote:

> Steven D'Aprano  writes:
> 
>> On Fri, 26 Feb 2010 09:09:36 -0600, Tim Daneliuk wrote:
>>
>>> Reminiscent of:
>>> 
>>> mov  AX,BX   ; Move the contents of BX into AX
>>
>>
>> That's a *good* comment, because without it most English-speaking
>> people would assume you were moving the contents of AX into BX.
> 
> Eh? It's a bad comment, it's of the same quality as:
> 
>> x = x + 1  # Add one to x.
> 
> You think the former is good because (I guess) you are not familiar with
> the language. The same reason why beginners comment their code like in
> your example.

Well, in the second example, x+1 could not possibly be anything other 
than adding one to x in any language, programming or human: using "+" for 
addition is close enough to universal these days. Unless x is some fancy 
object that plays operator overloading tricks, then what else could x+1 
be?

On the other hand, mv AX,BX is not only ambiguous but in the example 
given the move occurs in the counter-intuitive direction, at least for 
English-speakers and probably most speakers of European languages. So 
even an experienced coder might like the reminder that this specific 
assembly language moves things backwards compared to his intuition, or 
compared to the way the rest of the language does things.

Admittedly an experienced coder might be so used to that syntax that he 
no longer needs the comment, in which case he might find it unnecessary. 
But comments aren't written for the benefit of people who find it 
obvious. They're written for everyone else. ANY comment could be 
dismissed as unnecessary for somebody who is so familiar with the 
language and code in question that it is obvious what it is doing.

Also, some assemblies perform the move in different directions according 
to the arguments. So you might have:

mv AX,BX  ; move contents of BX into AX
mv @CX,DX ; move contents of @CX into DX

Horrible, yes, but apparently some assembly languages did something like 
that.


-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: random.gauss: range

[Steven D'Aprano]

On Fri, 26 Feb 2010 13:26:44 -0800, pistacchio wrote:

> hi,
> i'm trying the random.gauss function. can anyone explain how to get a
> number between a given range? like, from 0 to 20 with an average of 10?

That's not what a Gaussian distribution does. It has an infinite range. 
Are you sure you want a Gaussian, and not a uniform distribution or a 
triangular distribution?

You could pin the values to the appropriate range:

def pinned_gaussian(a, b, mu, sigma):
"""Return a Gaussian random number pinned to [a, b]."""
return min(b, max(a, random.gauss(mu, sigma)))

but that will distort the randomness slightly: you will have a slight 
excess of values equal to a and b than you otherwise might expect. You 
might not care, though, particularly if the sigma is very small relative 
to the minimum and maximum you want.

Alternatively, you could use this approach:

def truncated_gauss(a, b, mu, sigma):
"""Return a random number from a truncated Gaussian distribution."""
while 1:
x = random.gauss(mu, sigma)
if a <= x <= b:
return x



> and how to determine the "steep" of the curve? i've never studied it, so
> mu and sigma don't really tell me a thing.

mu is the average -- in your example above, mu would be 10.

sigma is the standard deviation of the random variable. It tells you how 
much the random variable actually varies: if sigma is close to zero, most 
of the numbers will be close to mu and the graph will look like a spike; 
if sigma is large, it will be very broad and flat.

Statistically, the following rules apply:

68% of the numbers will be between (mu - sigma) and (mu + sigma).
95% of the numbers will be between (mu - 2*sigma) and (mu + 2*sigma).
99.7% of the numbers will be between (mu - 3*sigma) and (mu + 3*sigma).

(the percentages are approximate, not exact). So you could get a 
reasonable approximation to a normal distribution truncated between 0 and 
20 with:

truncated_gauss(0, 20, 10, 10.0/3)




-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Dictionary or Database—Please advise

[Paul Rubin]

Jeremy  writes:
> I have lots of data that I currently store in dictionaries.  However,
> the memory requirements are becoming a problem.  I am considering
> using a database of some sorts instead, but I have never used them
> before.  Would a database be more memory efficient than a dictionary?

What are you trying to do?  Yes, a database would let you use disk
instead of ram, but the slowdown might be intolerable depending on the
access pattern.  You really have to consider what the whole application
is up to, and what the database is doing for you.  It might be imposing
a high overhead to create benefits (e.g. transaction isolation) that
you're not actually using.

Somehow I've managed to do a lot of programming on large datasets
without ever using databases very much.  I'm not sure that's a good
thing; but anyway, a lot of times you can do better with externally
sorted flat files, near-line search engines, etc. than you can with
databases.

If the size of your data is fixed or is not growing too fast, and it's
larger than your computer's memory by just a moderate amount (e.g. you
have a 2GB computer and 3GB of data), the simplest approach may be to
just buy a few more ram modules and put them in your computer.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Dictionary or Database—Please advise

[Steven D'Aprano]

On Fri, 26 Feb 2010 21:56:47 +0100, Patrick Sabin wrote:

>> Shelve looks like an interesting option, but what might pose an issue
>> is that I'm reading the data from a disk instead of memory.  I didn't
>> mention this in my original post, but I was hoping that by using a
>> database it would be more memory efficient in storing data in RAM so I
>> wouldn't have to read from (or swap to/from) disk.
> 
> A database usually stores data on disk and not in RAM. However you could
> use sqlite with :memory:, so that it runs in RAM.

The OP started this thread with:

"I have lots of data that I currently store in dictionaries.  However,
the memory requirements are becoming a problem."

So I'm amused that he thinks the solution to running out of memory is to 
run the heavy overhead of a database entirely in memory, instead of 
lightweight dicts :)

My advice is, first try to optimize your use of dicts. Are you holding 
onto large numbers of big dicts that you don't need? Are you making 
unnecessary copies? If so, fix your algorithms.

If you can't optimize the dicts anymore, then move to a proper database. 
Don't worry about whether it is on-disk or not, databases tend to be 
pretty fast regardless, and it is better to run your application a little 
bit more slowly than to not run it at all because it ran out of memory 
halfway through processing the data.


-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: importing libraries not working 2.6.4

[Rami Chowdhury]

On Friday 26 February 2010 17:42:04 Isaiah Coberly wrote:
> Thanks for the reply.
> 
> I tried putting the files from
> 
>  C:\OpenCV2.0\Python2.6\Lib
> 
> too
> 
> C:\Python26\Lib\site-packages
> 
> and Python still wont import..
> 
> I adjusted the environment variables to try and import maya.standalone but
> no dice on that either.
> 

Sorry, you've still not told me where you expect it to be. Could you let us 
know what .py files you can see in C:\OpenCV2.0\Python2.6\Lib ?

> 
> On Fri, Feb 26, 2010 at 4:30 PM, Rami Chowdhury 
wrote:
> > On Friday 26 February 2010 16:06:56 Routb3d wrote:
> > > I used the x86 Python 2.6.4 installer. I am working in x64 Win7 pro.
> > > 
> > > I have tried adjusting environment variables as well as putting the
> > > files directly in the search path of Python.. Python still returns
> > > this.. Any ideas?
> > 
> > Where are you expecting the cv module / package to be? It's certainly not
> > in
> > the standard library -- at least not in my install of Python 2.6.4...
> > 
> > > >>> import sys
> > > >>> sys.path
> > > 
> > > ['C:\\Python26\\Lib\\idlelib', 'C:\\Program Files\\Autodesk\\Maya2010\
> > > \Python\\Lib\\site-packages', 'C:\\Windows\\system32\\python26.zip',
> > > 'C:\\Python26\\DLLs', 'C:\\Python26\\lib', 'C:\\Python26\\lib\\plat-
> > > win', 'C:\\Python26\\lib\\lib-tk', 'C:\\Python26', 'C:\\Python26\\lib\
> > > \site-packages']
> > > 
> > > >>> import cv
> > > 
> > > Traceback (most recent call last):
> > >   File "", line 1, in 
> > >   
> > > import cv
> > > 
> > > ImportError: DLL load failed: The specified module could not be found.
> > > 
> > > 
> > > Any help would be greatly appreciated.
> > 
> > 
> > Rami Chowdhury
> > "Strangers are just friends who haven't had enough gin." -- Howdle's
> > Saying 408-597-7068 (US) / 07875-841-046 (UK) / 01819-245544 (BD)



Rami Chowdhury
"Ninety percent of everything is crap." -- Sturgeon's Law
408-597-7068 (US) / 07875-841-046 (UK) / 01819-245544 (BD)
-- 
http://mail.python.org/mailman/listinfo/python-list

PyAutoRun

[Zubin Mithra]

Hello,

I have been using python for quite some time; however this is the
first python project i have worked on.

The code is hosted at http://github.com/zubin71/PyAutoRun

The code needs re-factoring and feature additions; i have put up a
TODO list there too. It`d be great if anyone could work on this; i
intend to develop this further(with a bit of help) and will request
for its addition into debian and ubuntu repositories, in time.

Also, any kind of code-review, criticism, is also appreciated.
However, it`d be awesome if you could just fork it at github, pull,
modify and push. :)

Have a nice day!

cheers!!!
Zubin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Docstrings considered too complicated

[Alf P. Steinbach]

* MRAB:

Steven D'Aprano wrote:

On Fri, 26 Feb 2010 09:09:36 -0600, Tim Daneliuk wrote:


Reminiscent of:

mov  AX,BX   ; Move the contents of BX into AX



That's a *good* comment, because without it most English-speaking 
people would assume you were moving the contents of AX into BX.



[snip]
If you're reading and/or writing at assembly language level then you
should know what it does anyway!

The assembly languages of virtually all the processors that I've come
across put the destination first, eg. x86:

SUB AX,BX
MOV AX,BX

which does:

AX := AX - BX
AX := BX



A bit off-topic, but there are /two/ main syntaxes for x86 assembly, namely 
Intel syntax (the above syntax, used by MASM, old TASM etc.) and AT&T syntax.




C:\test> echo int main(){ int x = 42; } >blah.cpp

C:\test> g++ blah.cpp -S -masm=intel

C:\test> type blah.s | find "42"
mov DWORD PTR [ebp-4], 42

C:\test> g++ blah.cpp -S -masm=att

C:\test> type blah.s | find "42"
movl$42, -4(%ebp)

C:\test> _



Personally I find the AT&T syntax very hard to read.

All those percent signs hurt my eyes...





and ARM:

SUB R0,R1,R2
MOV R0,R1

which does:

R0 := R1 - R2
R0 := R1

The only assembly language I know of which does it the other way round
is 68x00:

SUB D0,D1
MOVE D0,D1

which does:

D1 := D1 - D0
D1 := D0

I know which I prefer! :-)


Cheers,

- Alf
--
http://mail.python.org/mailman/listinfo/python-list

Re: Variable definition

[John Posner]

On 2/26/2010 6:32 PM, Raphael Mayoraz wrote:

Hello,

I'd like to define variables with some specific name that has a common
prefix.
Something like this:

varDic = {'red': 'a', 'green': 'b', 'blue': 'c'}
for key, value in varDic.iteritems():
'myPrefix' + key = value



No trick, just swap a new key-value pair for each existing pair:

  for key, value in varDic.iteritems():
  varDic[myPrefix + key] = value
  del varDict[key]

Just make sure that *myPrefix* isn't an empty string!

-John
--
http://mail.python.org/mailman/listinfo/python-list

Re: Docstrings considered too complicated

[MRAB]

Steven D'Aprano wrote:

On Fri, 26 Feb 2010 09:09:36 -0600, Tim Daneliuk wrote:


Reminiscent of:

mov  AX,BX   ; Move the contents of BX into AX



That's a *good* comment, because without it most English-speaking people 
would assume you were moving the contents of AX into BX.



[snip]
If you're reading and/or writing at assembly language level then you
should know what it does anyway!

The assembly languages of virtually all the processors that I've come
across put the destination first, eg. x86:

SUB AX,BX
MOV AX,BX

which does:

AX := AX - BX
AX := BX

and ARM:

SUB R0,R1,R2
MOV R0,R1

which does:

R0 := R1 - R2
R0 := R1

The only assembly language I know of which does it the other way round
is 68x00:

SUB D0,D1
MOVE D0,D1

which does:

D1 := D1 - D0
D1 := D0

I know which I prefer! :-)
--
http://mail.python.org/mailman/listinfo/python-list

Re: Docstrings considered too complicated

[John Bokma]

Steven D'Aprano  writes:

> On Fri, 26 Feb 2010 09:09:36 -0600, Tim Daneliuk wrote:
>
>> Reminiscent of:
>> 
>> mov  AX,BX   ; Move the contents of BX into AX
>
>
> That's a *good* comment, because without it most English-speaking people 
> would assume you were moving the contents of AX into BX.

Eh? It's a bad comment, it's of the same quality as:

> x = x + 1  # Add one to x.

You think the former is good because (I guess) you are not familiar with
the language. The same reason why beginners comment their code like in
your example.

-- 
John Bokma   j3b

Hacking & Hiking in Mexico -  http://johnbokma.com/
http://castleamber.com/ - Perl & Python Development
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: random.gauss: range

[Raymond Hettinger]

On Feb 26, 1:26 pm, pistacchio  wrote:
> hi,
> i'm trying the random.gauss function. can anyone explain how to get a
> number between a given range? like, from 0 to 20 with an average of
> 10? and how to determine the "steep" of the curve? i've never studied
> it, so mu and sigma don't really tell me a thing.

Try random.randrange() and random.triangular().
They are both easy to use and do not require you
to enter parameters that you don't understand.

FWIW, mu and sigma refer to the average and standard deviation
of a normal distribution.


Raymond

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: random.gauss: range

[pistacchio]

thanks, betadistribute did the work... and i learned a new thing!

On 26 Feb, 22:56, Robert Kern  wrote:
> On 2010-02-26 15:26 PM, pistacchio wrote:
>
> > hi,
> > i'm trying the random.gauss function. can anyone explain how to get a
> > number between a given range?
>
> You don't. The Gaussian distribution has infinite range. The best you can do
> with the standard library is to keep sampling until you get a number inside 
> your
> desired range. If you aren't careful about your choice of parameters, this 
> could
> waste a lot of time.
>
> > like, from 0 to 20 with an average of
> > 10? and how to determine the "steep" of the curve? i've never studied
> > it, so mu and sigma don't really tell me a thing.
>
> Study it:
>
>    http://en.wikipedia.org/wiki/Normal_distribution
>
> mu is the mean, the location of the central peak. sigma is the standard
> deviation, which controls the width of the peak. Larger sigma means wider and
> shorter peak.
>
> You may want another distribution, like random.betavariate():
>
>    http://en.wikipedia.org/wiki/Beta_distribution
>
> --
> Robert Kern
>
> "I have come to believe that the whole world is an enigma, a harmless enigma
>   that is made terrible by our own mad attempt to interpret it as though it 
> had
>   an underlying truth."
>    -- Umberto Eco

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Variable definition

[alex goretoy]

On Fri, Feb 26, 2010 at 6:22 PM, Alf P. Steinbach  wrote:

> * Raphael Mayoraz:
>
>> Hello,
>>
>>
>> I'd like to define variables with some specific name that has a common
>> prefix.
>> Something like this:
>>
>> varDic = {'red': 'a', 'green': 'b', 'blue': 'c'}
>> for key, value in varDic.iteritems():
>>   'myPrefix' + key = value
>>
>> I know this is illegal, but there must be a trick somewhere.
>>
>
> In general you'll IMHO be better off with the variables as attributes of an
> object.
>
> If you want them to be modifiable then you can do
>
>  class Whatever: pass
>
>  myPrefix = Whatever()
>  myPrefix.a = 'a'
>  myPrefix.b = 'b'
>  myPrefix.c = 'c'
>
> If you want them to be sort of constants (weasel words intentional) then
> you might do (Python 3.x)  --  disclaimer: off-the-cuff & I'm not sure if
> that function is called 'namedtuple' but I think you'll find it anyway  --
>
>  import collections
>
>  Color = namedtuple( "Color", "red green blue" )
>  myPrefix = Color( 'a', 'b', 'c' )
>
>
> Cheers & hth.,
>
> - Alf
>
> --
> http://mail.python.org/mailman/listinfo/python-list
>

you can use setattr to do your bidding for setting variable like this. That
is how I've been able to do it. Not sure if its the best solution, but it is
a solution. I just don't know how to use setattr without it being in a
class.

>>> class stuff(object):
...  def __init__(self):
...   pass
...  def duuit(self,abc):
...   for k,v in abc.items():
...setattr(self,"prefix_%s"%k,v)
...
>>> varDic = {'red': 'a', 'green': 'b', 'blue': 'c'}
>>> abc=stuff()
>>> abc.duuit(varDic)
>>> dir(abc)
['__class__', '__delattr__', '__dict__', '__doc__', '__format__',
'__getattribute__', '__hash__', '__init__', '__module__', '__new__',
'__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__',
'__str__', '__subclasshook__', '__weakref__', 'duuit', 'prefix_blue',
'prefix_green', 'prefix_red']
>>>

-Alex Goretoy
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: importing libraries not working 2.6.4

[Rami Chowdhury]

On Friday 26 February 2010 16:06:56 Routb3d wrote:
> I used the x86 Python 2.6.4 installer. I am working in x64 Win7 pro.
> 
> I have tried adjusting environment variables as well as putting the
> files directly in the search path of Python.. Python still returns
> this.. Any ideas?

Where are you expecting the cv module / package to be? It's certainly not in 
the standard library -- at least not in my install of Python 2.6.4...

> 
> >>> import sys
> >>> sys.path
> 
> ['C:\\Python26\\Lib\\idlelib', 'C:\\Program Files\\Autodesk\\Maya2010\
> \Python\\Lib\\site-packages', 'C:\\Windows\\system32\\python26.zip',
> 'C:\\Python26\\DLLs', 'C:\\Python26\\lib', 'C:\\Python26\\lib\\plat-
> win', 'C:\\Python26\\lib\\lib-tk', 'C:\\Python26', 'C:\\Python26\\lib\
> \site-packages']
> 
> >>> import cv
> 
> Traceback (most recent call last):
>   File "", line 1, in 
> import cv
> ImportError: DLL load failed: The specified module could not be found.
> 
> 
> Any help would be greatly appreciated.



Rami Chowdhury
"Strangers are just friends who haven't had enough gin." -- Howdle's Saying
408-597-7068 (US) / 07875-841-046 (UK) / 01819-245544 (BD)
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Variable definition

[Alf P. Steinbach]

* Raphael Mayoraz:

Hello,

I'd like to define variables with some specific name that has a common 
prefix.

Something like this:

varDic = {'red': 'a', 'green': 'b', 'blue': 'c'}
for key, value in varDic.iteritems():
   'myPrefix' + key = value

I know this is illegal, but there must be a trick somewhere.


In general you'll IMHO be better off with the variables as attributes of an 
object.

If you want them to be modifiable then you can do

  class Whatever: pass

  myPrefix = Whatever()
  myPrefix.a = 'a'
  myPrefix.b = 'b'
  myPrefix.c = 'c'

If you want them to be sort of constants (weasel words intentional) then you 
might do (Python 3.x)  --  disclaimer: off-the-cuff & I'm not sure if that 
function is called 'namedtuple' but I think you'll find it anyway  --


  import collections

  Color = namedtuple( "Color", "red green blue" )
  myPrefix = Color( 'a', 'b', 'c' )


Cheers & hth.,

- Alf
--
http://mail.python.org/mailman/listinfo/python-list

Re: Variable definition

[Rhodri James]

On Fri, 26 Feb 2010 23:32:27 -, Raphael Mayoraz   
wrote:


I'd like to define variables with some specific name that has a common  
prefix.


Why?

No seriously, how do you think this is going to solve whatever problem you  
clearly think it will solve?


--
Rhodri James *-* Wildebeeste Herder to the Masses
--
http://mail.python.org/mailman/listinfo/python-list

importing libraries not working 2.6.4

[Routb3d]

I used the x86 Python 2.6.4 installer. I am working in x64 Win7 pro.

I have tried adjusting environment variables as well as putting the
files directly in the search path of Python.. Python still returns
this.. Any ideas?

>>> import sys
>>> sys.path
['C:\\Python26\\Lib\\idlelib', 'C:\\Program Files\\Autodesk\\Maya2010\
\Python\\Lib\\site-packages', 'C:\\Windows\\system32\\python26.zip',
'C:\\Python26\\DLLs', 'C:\\Python26\\lib', 'C:\\Python26\\lib\\plat-
win', 'C:\\Python26\\lib\\lib-tk', 'C:\\Python26', 'C:\\Python26\\lib\
\site-packages']
>>> import cv

Traceback (most recent call last):
  File "", line 1, in 
import cv
ImportError: DLL load failed: The specified module could not be found.


Any help would be greatly appreciated.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Docstrings considered too complicated

[Steven D'Aprano]

On Fri, 26 Feb 2010 09:09:36 -0600, Tim Daneliuk wrote:

> Reminiscent of:
> 
> mov  AX,BX   ; Move the contents of BX into AX


That's a *good* comment, because without it most English-speaking people 
would assume you were moving the contents of AX into BX.


> And, yes, I've actually seen that as well as:
> 
> ; This is a comment

Which might be a good comment, if it were part of an unfamiliar file 
format where the reader were not expected to know what is a comment and 
what is significant. For instance, I've seen similar comments at the top 
of config files.


However, I'm pretty sure that a comment like this is inexcusable:


x = x + 1  # Add one to x.



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: A more specific query ...

[Gib Bogle]

The PyQt4 problem results from having copies of the Qt DLLs in directories that 
are in the PATH, as Doug Bell discovered.  In my case I have two programs that 
use Qt, AMD CodeAnalyst and Matlab.  If I rename BOTH these directories I can 
import the PyQt4 modules.


Since this behaviour did not occur with Python 2.5 and the previous PyQt I was 
using (4.5, I think), it seems that something has changed either with Python 2.6 
or with PyQt 4.7.


It would be very nice to learn how to fix this without renaming directories, 
since I use Matlab frequently.


Gib
--
http://mail.python.org/mailman/listinfo/python-list

Re: A more specific query ...

[Gib Bogle]

The PyQt4 problem results from having copies of the Qt DLLs in directories that 
are in the PATH, as Doug Bell discovered.  In my case I have two programs that 
use Qt, AMD CodeAnalyst and Matlab.  If I rename BOTH these directories I can 
import the PyQt4 modules.


Since this behaviour did not occur with Python 2.5 and the previous PyQt I was 
using (4.5, I think), it seems that something has changed either with Python 2.6 
or with PyQt 4.7.


It would be very nice to learn how to fix this without renaming directories, 
since I use Matlab frequently.


Gib
--
http://mail.python.org/mailman/listinfo/python-list

Re: Get dosctring without import

[Ben Finney]

Joan Miller  writes:

> On 26 feb, 12:35, Ben Finney  wrote:
> > A common convention is to have a ‘README’ text file, written in
> > reStructuredText for rendering to various output formats as part of
> > the documentation. You could then have the ‘setup.py’ program read
> > the contents of that file and use it (or a slice of it) for the
> > package description.
> I get the 'README.txt' file to get the long description but I use the
> docstring because each package should include a short desciption about
> it.

I keep both in the same file:

= README =
FooBar, a library for spangulation.

The FooBar library provides thribbles which can be
easily frobnicated for spangulation in a sntandard manner.

Other spangulation libraries are far less weebly than this
one, which is the choice of discerning grognards everywhere.
=

Then, the description fields are derived by splitting the file's
contents on the first paragraph break:

=
from distutils.core import setup

readme_file = open("README")

short_description, long_description = (
d.strip()
for d in readme_file.read().split(u'\n\n', 1))

setup(
# …
description=short_description,
long_description=long_description,
# …
)
=

-- 
 \  “Some forms of reality are so horrible we refuse to face them, |
  `\ unless we are trapped into it by comedy. To label any subject |
_o__)unsuitable for comedy is to admit defeat.” —Peter Sellers |
Ben Finney
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: loop through each line in a text file

[alex goretoy]

I smell homework
-- 
http://mail.python.org/mailman/listinfo/python-list

Variable definition

[Raphael Mayoraz]

Hello,

I'd like to define variables with some specific name that has a common 
prefix.

Something like this:

varDic = {'red': 'a', 'green': 'b', 'blue': 'c'}
for key, value in varDic.iteritems():
   'myPrefix' + key = value

I know this is illegal, but there must be a trick somewhere.

Thanks,

Raphael



--
http://mail.python.org/mailman/listinfo/python-list

Re: loop through each line in a text file

[rurpy]

On Feb 26, 2:21 pm, qtrimble  wrote:
> On Feb 26, 4:14 pm, OdarR  wrote:
> 
> > > below is just a sample.  There are well over 500,000 lines that need
> > > processed.
>
> > > wer1999001
> > >       31.2234      82.2367
> > >       37.9535      82.3456
> > > wer1999002
> > >       31.2234      82.2367
> > >       37.9535      82.3456
>
> > did you try something as a working basis ?
>
> > Olivier
>
> Yes but it's very simple -
>
> fileIN = open(r"C:\testing.txt", "r")
>
> for line in fileIN:
>     year = line[3:7]
>     day = line[7:10]
>     print year, day
>
> This is good since i can get the year and day of year into a variable
> but I haven't gotten any further.

How about something like (untested):

  for line in fileIN:
   if line.startswith ("wer"):
   year = line[3:7]
   day = line[7:10]
else:
   print "%s-%s %s" % (year, day, line.strip())

You can adjust the details as needed...
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Dictionary or Database—Please advise

[Phlip]

On Feb 26, 7:58 am, Jeremy  wrote:
> I have lots of data that I currently store in dictionaries.  However,
> the memory requirements are becoming a problem.  I am considering
> using a database of some sorts instead, but I have never used them
> before.  Would a database be more memory efficient than a dictionary?
> I also need platform independence without having to install a database
> and Python interface on all the platforms I'll be using.  Is there
> something built-in to Python that will allow me to do this?

If you had wall-to-wall unit tests, you could swap the database in
incrementally ("Deprecation Refactor").

You would just add one database table, switch one client of one
dictionary to use that table, pass all the tests, and integrate.
Repeat until nobody uses the dicts, then trivially retire them.

If you don't have unit tests, then you have a bigger problem than
memory requirements. (You can throw $50 hardware at that!)

--
  Phlip
  http://penbird.tumblr.com/
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Dictionary or Database—Please advise

[Aahz]

In article <891a98fa-c398-455a-981f-bf72af772...@s36g2000prh.googlegroups.com>,
Jeremy   wrote:
>
>I have lots of data that I currently store in dictionaries.  However,
>the memory requirements are becoming a problem.  I am considering
>using a database of some sorts instead, but I have never used them
>before.  Would a database be more memory efficient than a dictionary?
>I also need platform independence without having to install a database
>and Python interface on all the platforms I'll be using.  Is there
>something built-in to Python that will allow me to do this?

If you're serious about needing both a disk-based backing store *and*
getting maximum use/performance from your RAM, you probably will need to
combine memcached with one of the other solutions offered.

But given your other requirement of not installing a DB, your best option
will certainly be SQLite.  You can use :memory: databases, but that will
require shuttling data to disk manually.  I suggest that you start with
plain SQLite and only worry if you prove (repeat, PROVE) that DB is your
bottleneck.
-- 
Aahz (a...@pythoncraft.com)   <*> http://www.pythoncraft.com/

"Many customs in this life persist because they ease friction and promote
productivity as a result of universal agreement, and whether they are
precisely the optimal choices is much less important." --Henry Spencer
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Dictionary or Database�Please advise

[Aahz]

In article ,
Roy Smith   wrote:
>
>Whatever database you pick, you're almost certainly going to end up having 
>to install it wherever you install your application.  There's no such thing 
>as a universally available database that you can expect to be available 
>everywhere.

...unless you use SQLite.
-- 
Aahz (a...@pythoncraft.com)   <*> http://www.pythoncraft.com/

"Many customs in this life persist because they ease friction and promote
productivity as a result of universal agreement, and whether they are
precisely the optimal choices is much less important." --Henry Spencer
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Dictionary or Database—Please advise

[Luis M . González]

On Feb 26, 12:58 pm, Jeremy  wrote:
> I have lots of data that I currently store in dictionaries.  However,
> the memory requirements are becoming a problem.  I am considering
> using a database of some sorts instead, but I have never used them
> before.  Would a database be more memory efficient than a dictionary?
> I also need platform independence without having to install a database
> and Python interface on all the platforms I'll be using.  Is there
> something built-in to Python that will allow me to do this?
>
> Thanks,
> Jeremy

For small quantities of data, dicts are ok, but if your app
continually handles (and add) large quantities of data, you should use
a database.
You can start with sqlite (which is bundled with python) and it's much
easier to use than any other relational database out there.
You don't need to install anything, since it's not a database server.
You simply save your databases as files.
If you don't know sql (the standard language used to query databases),
I recomend this online tutorial: http://www.sqlcourse.com/

Good luck!
Luis
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Dictionary or Database—Please advise

[Aahz]

In article ,
Patrick Sabin   wrote:
>
>A database usually stores data on disk and not in RAM. However you could 
>use sqlite with :memory:, so that it runs in RAM.

The OP wants transparent caching, so :memory: wouldn't work.
-- 
Aahz (a...@pythoncraft.com)   <*> http://www.pythoncraft.com/

"Many customs in this life persist because they ease friction and promote
productivity as a result of universal agreement, and whether they are
precisely the optimal choices is much less important." --Henry Spencer
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: loop through each line in a text file

[John Posner]

On 2/26/2010 4:21 PM, qtrimble wrote:


fileIN = open(r"C:\testing.txt", "r")

for line in fileIN:
 year = line[3:7]
 day = line[7:10]
 print year, day

This is good since i can get the year and day of year into a variable
but I haven't gotten any further.


That's an excellent start. Here's another hint ...

There are two kinds of lines in your input file:

* lines that begin with "wer"
* all other lines

So you need an if-else section to process the lines in the input file:

  if line.startswith("wer"):
  # use the contents of *line* to assign values
  # to variables *year* and *day*
  ...
  else:
  # use the current values of *year* and *day*
  # to process the numbers extracted from *line*

Most likely, you'll also need these functions:

* split() -- chop a string into pieces
* int() -- string-to-integer conversion

HTH,
John



--
http://mail.python.org/mailman/listinfo/python-list

Re: Six Minutes and fourty two seconds

[Mark Lawrence]

Tobiah wrote:
Now that I use python, this is the amount of time 
per day that I spend adding forgotten semicolons while

debugging other languages.


You think that's bad?  I've spent hours today converting the J language 
that I don't dare mention to Python.  Once in a blue moon I came across 
a line of code that actually did something. The rest of it was 
boilerplate.  I'm never ever going to sin again, because if I do, I will 
be reincarnated as a J type, or worse still, I C(++) type.


Regards.

Mark Lawrence.


No that's a swaaannn!


--
http://mail.python.org/mailman/listinfo/python-list

Re: Quoting quotes

[Lawrence D'Oliveiro]

In message 
, William 
Lohrmann wrote:

> The best thing would be to backslash the single quote: print 'The play
> "All\'s Well That Ends Well"'

Backslash-type escapes are the most general solution to this type of 
problem. They’re also the easiest to apply automatically:

for ch in input_string :
 if ch in troublesome_lot :
 output backslash + tame representation of ch
 else :
 output ch
 #end if
#end for
-- 
http://mail.python.org/mailman/listinfo/python-list

How to determine if threads are active in an application?

[python]

Is there technique to determine if threads are active in a Python
application?

The only technique I can think of is to check sys.modules for
thread and threading. But this will only show whether these
modules were imported - not whether there are actually background
threads running.

Motivation: We were debugging a customer's python application and
they had a module that was unknowningly (to both us and them)
creating threads in certain situations. This made our debugging
process more complicated because there were side effects
happening at unexpected times that we couldn't explain. The
module in question was supplied to us as a compiled module that
was considered fully tested by the customer. To prevent this
confusion in the future, we're looking for a way that we can
detect threads are active while we're debugging without having to
manually step through every line of code in every module to check
for unexpected thread creation.

Thanks,
Malcolm
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: A more specific query ...

[Gib Bogle]

The point of my question was that sys.path is clearly not being used in this 
case.  When I start Python sys.path includes D:\python26\lib\site-packages which 
seems to be the Python default.  Using sys.path.append I have tried adding both 
D:\python26\lib\site-packages\PyQt4 and D:\python26\lib\site-packages\PyQt4\bin 
to the path, but this has no effect.  So far the only way I've found to make the 
PyQt4 DLLs visible to Python is by copying them from PyQt4\bin into PyQt4.


I've tried adding stuff to the Windows PATH, but that has no effect.

Gib

Chris Rebert wrote:

On Fri, Feb 26, 2010 at 12:45 PM, Gib Bogle
 wrote:

How can I interrogate Python to find out where it is looking to find the
PyQt4 DLLs in a Windows installation?


import sys
print(sys.path)

Cheers,
Chris
--
http://blog.rebertia.com

--
http://mail.python.org/mailman/listinfo/python-list

Re: Docstrings considered too complicated

[Mark Lawrence]

Jean-Michel Pichavant wrote:

Andreas Waldenburger wrote:

On Fri, 26 Feb 2010 09:09:36 -0600 Tim Daneliuk 
wrote:

 

On 2/24/2010 2:23 PM, Andreas Waldenburger wrote:
   

[stuff]
  

Reminiscent of:

mov  AX,BX   ; Move the contents of BX into AX



Well, there might be some confusion there as to what gets moved where,
wouldn't you say? I guess this goes away after a couple of months,
though.
  


I agree to that statement, I was surprised that mov AX,BX assumes that 
BX is the source, and AX the destination. I never programmed in 
assembler though.


JM


The obvious solution to this problem is to write the assembler code in 
your own way, and then use Python to change the code to the appropriate 
target.  Any of the solutions listed here would presumably suffice.

http://nedbatchelder.com/text/python-parsers.html

Regards.

Mark Lawrence.

--
http://mail.python.org/mailman/listinfo/python-list

Re: random.gauss: range

[Robert Kern]

On 2010-02-26 15:26 PM, pistacchio wrote:

hi,
i'm trying the random.gauss function. can anyone explain how to get a
number between a given range?


You don't. The Gaussian distribution has infinite range. The best you can do 
with the standard library is to keep sampling until you get a number inside your 
desired range. If you aren't careful about your choice of parameters, this could 
waste a lot of time.



like, from 0 to 20 with an average of
10? and how to determine the "steep" of the curve? i've never studied
it, so mu and sigma don't really tell me a thing.


Study it:

  http://en.wikipedia.org/wiki/Normal_distribution

mu is the mean, the location of the central peak. sigma is the standard 
deviation, which controls the width of the peak. Larger sigma means wider and 
shorter peak.


You may want another distribution, like random.betavariate():

  http://en.wikipedia.org/wiki/Beta_distribution

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco

--
http://mail.python.org/mailman/listinfo/python-list

Re: Problems uploading to IIS using FTP over SSL

[Stephen Nelson-Smith]

On Feb 26, 2:05 pm, Stephen Nelson-Smith  wrote:
> Hello,

I'm sorry - I hadn't realised that python-list ended up here as well.
Sincere apologies for double-posting.

S.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: A more specific query ...

[Gib Bogle]

Chris Rebert wrote:

On Fri, Feb 26, 2010 at 12:45 PM, Gib Bogle
 wrote:

How can I interrogate Python to find out where it is looking to find the
PyQt4 DLLs in a Windows installation?


import sys
print(sys.path)


Note this thread:
http://www.mail-archive.com/p...@riverbankcomputing.com/msg20121.html

I'm starting to get the impression that Windows is the 'poor relation' in the 
Python family ...

--
http://mail.python.org/mailman/listinfo/python-list

Re: Renaming identifiers & debugging

[Luca]

Roald de Vries wrote:
I would suggest to do choose the same strategy as 'from __future__ 
import ...' takes, which does similar things and limits them to the 
module it is used in. I would be curious to hear about your results.


Kind regards, Roald


Hi,
well, i have thought on the issue and i think that i have found a 
solution that does not imply renaming the keywords. It mixes a bit the 
suggestions i got here and this is how.


My application will have a text editor (with, hopefully, code 
highlighting) a text-box to insert "instant" commands and a text-box for 
the output and a window for the "turtle".


With the "instant commands" you can insert little pieces of python code 
and see instantly their result both as a turtle movement or textual 
output. With the text-editor you can load/save/write whole python 
programs and execute or debug them step by step.


The way to track the code could use this technique 
http://www.dalkescientific.com/writings/diary/archive/2005/04/20/tracing_python_code.html 
which gives me exactly the number of line of code under execution and 
also stops the code until the function "treaceit" returns. This will 
allow me to insert breakpoints, execute the code line by line, and so on 
(or at least this is the theory).


Now. Since i get the number of the line i don't need anymore to have the 
language deal with translated keywords. The kid can write the code in 
his/her own language and the program will "translate it back to english" 
before passing them to the embedded python. When i save a source file i 
can save the english version and translate it "on the fly" when i load 
it. I can also intercept error codes and translate them as well, showing 
the kid only what i want him/her to see. So it would be completely 
transparent to the kid and it would notice it only by opening the source 
code via a different text-editor. In this way the files produced are 
100% python files and the kid can disable this feature if he doesn't 
like it. This will also allow me to translate the same piece of code in 
any language i want, with 0 effort and this way will allow kids of 
different nations exchange their code and have it working on their computer.


This seems a good idea to me, it reaches my points without breaking 
compatibility. The kid will see the language in his own language but the 
computer will actually work with the standard-python file.


Once i have this "ready" i can proceed to other steps and try to 
implement other ideas that i have (remember c-robots and p-robots?).


Bye,
  Luca
--
http://mail.python.org/mailman/listinfo/python-list

random.gauss: range

[pistacchio]

hi,
i'm trying the random.gauss function. can anyone explain how to get a
number between a given range? like, from 0 to 20 with an average of
10? and how to determine the "steep" of the curve? i've never studied
it, so mu and sigma don't really tell me a thing.

thanks in advange
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: loop through each line in a text file

[qtrimble]

On Feb 26, 4:14 pm, OdarR  wrote:
> On 26 fév, 22:08, qtrimble  wrote:
>
>
>
> > I'm a python newbie but I do have some basic scripting experience.  I
> > need to take the line starting with "wer" and extract the year and day
> > of year from that string.  I want to be able to add the year and day
> > of year from the last line having "wer*" to the lines occurring in
> > between "wer*" lines.  Python seems suitable to do this and I'm fairly
> > certain I can eventually get this to work but I've been hit with a
> > very short time frame so I'm looking for any generous help.  The data
> > below is just a sample.  There are well over 500,000 lines that need
> > processed.
>
> > wer1999001
> >       31.2234      82.2367
> >       37.9535      82.3456
> > wer1999002
> >       31.2234      82.2367
> >       37.9535      82.3456
>
> did you try something as a working basis ?
>
> Olivier

Yes but it's very simple -

fileIN = open(r"C:\testing.txt", "r")

for line in fileIN:
year = line[3:7]
day = line[7:10]
print year, day

This is good since i can get the year and day of year into a variable
but I haven't gotten any further.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: A more specific query ...

[Chris Rebert]

On Fri, Feb 26, 2010 at 12:45 PM, Gib Bogle
 wrote:
> How can I interrogate Python to find out where it is looking to find the
> PyQt4 DLLs in a Windows installation?

import sys
print(sys.path)

Cheers,
Chris
--
http://blog.rebertia.com
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: loop through each line in a text file

[OdarR]

On 26 fév, 22:08, qtrimble  wrote:
> I'm a python newbie but I do have some basic scripting experience.  I
> need to take the line starting with "wer" and extract the year and day
> of year from that string.  I want to be able to add the year and day
> of year from the last line having "wer*" to the lines occurring in
> between "wer*" lines.  Python seems suitable to do this and I'm fairly
> certain I can eventually get this to work but I've been hit with a
> very short time frame so I'm looking for any generous help.  The data
> below is just a sample.  There are well over 500,000 lines that need
> processed.
>
> wer1999001
>       31.2234      82.2367
>       37.9535      82.3456
> wer1999002
>       31.2234      82.2367
>       37.9535      82.3456

did you try something as a working basis ?

Olivier
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: loop through each line in a text file

[Alf P. Steinbach]

* qtrimble:

I'm a python newbie but I do have some basic scripting experience.  I
need to take the line starting with "wer" and extract the year and day
of year from that string.  I want to be able to add the year and day
of year from the last line having "wer*" to the lines occurring in
between "wer*" lines.  Python seems suitable to do this and I'm fairly
certain I can eventually get this to work but I've been hit with a
very short time frame so I'm looking for any generous help.  The data
below is just a sample.  There are well over 500,000 lines that need
processed.

wer1999001
  31.2234  82.2367
  37.9535  82.3456
wer1999002
  31.2234  82.2367
  37.9535  82.3456



  >>> line = "wer1999001"
  >>> line
  'wer1999001'
  >>> line[3:3+4]
  '1999'
  >>> line[7:7+3]
  '001'
  >>> _


Cheers & hth.,

- Alf
--
http://mail.python.org/mailman/listinfo/python-list

loop through each line in a text file

[qtrimble]

I'm a python newbie but I do have some basic scripting experience.  I
need to take the line starting with "wer" and extract the year and day
of year from that string.  I want to be able to add the year and day
of year from the last line having "wer*" to the lines occurring in
between "wer*" lines.  Python seems suitable to do this and I'm fairly
certain I can eventually get this to work but I've been hit with a
very short time frame so I'm looking for any generous help.  The data
below is just a sample.  There are well over 500,000 lines that need
processed.

wer1999001
  31.2234  82.2367
  37.9535  82.3456
wer1999002
  31.2234  82.2367
  37.9535  82.3456
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Docstrings considered too complicated

[Grant Edwards]

On 2010-02-26, Jean-Michel Pichavant  wrote:
> Andreas Waldenburger wrote:
>> On Fri, 26 Feb 2010 09:09:36 -0600 Tim Daneliuk 
>> wrote:
>>
>>   
>>> On 2/24/2010 2:23 PM, Andreas Waldenburger wrote:
>>> 
 [stuff]
   
>>> Reminiscent of:
>>>
>>> mov  AX,BX   ; Move the contents of BX into AX
>>
>> Well, there might be some confusion there as to what gets moved where,
>> wouldn't you say? I guess this goes away after a couple of months,
>> though.
>
> I agree to that statement, I was surprised that mov AX,BX assumes that 
> BX is the source, and AX the destination. I never programmed in 
> assembler though.

It depends on the assembler.  Some are dst, src and others are
the other way around.  Some vary depending on the instruction.

-- 
Grant Edwards   grante Yow! Sign my PETITION.
  at   
   visi.com
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Dictionary or Database—Please advise

[Patrick Sabin]

Shelve looks like an interesting option, but what might pose an issue
is that I'm reading the data from a disk instead of memory.  I didn't
mention this in my original post, but I was hoping that by using a
database it would be more memory efficient in storing data in RAM so I
wouldn't have to read from (or swap to/from) disk.


A database usually stores data on disk and not in RAM. However you could 
use sqlite with :memory:, so that it runs in RAM.



Would using the
shelve package make reading/writing data from disk faster since it is
in a binary format?

Faster than what? Shelve uses caching, so it is likely to be faster than 
a self-made solution. However, accessing disk is much slower than 
accessing RAM.



Jeremy

- Patrick
--
http://mail.python.org/mailman/listinfo/python-list

Re: Docstrings considered too complicated

[Mel]

Jean-Michel Pichavant wrote:
> Andreas Waldenburger wrote:
>> On Fri, 26 Feb 2010 09:09:36 -0600 Tim Daneliuk 
>> wrote:
>>> Reminiscent of:
>>> mov  AX,BX   ; Move the contents of BX into AX

>> Well, there might be some confusion there as to what gets moved where,
>> wouldn't you say? I guess this goes away after a couple of months,
>> though.

> I agree to that statement, I was surprised that mov AX,BX assumes that
> BX is the source, and AX the destination. I never programmed in
> assembler though.

You could think of it as a not bad use of the design principle "Clear The 
Simple Stuff Out Of The Way First".  Destinations are commonly a lot simpler 
than sources -- just as in Python assignment statements.  So you can tell 
more or less at a glance what's going to be changed, then get into the deep 
analysis to find what it's going to be changed to.

Mel.


-- 
http://mail.python.org/mailman/listinfo/python-list

any news from Python Magazine ?

[OdarR]

Seems rather late...:
http://pythonmagazine.com/

"We'll be back, better than ever, on January 26th, 2010. "

Olivier
-- 
http://mail.python.org/mailman/listinfo/python-list

A more specific query ...

[Gib Bogle]

How can I interrogate Python to find out where it is looking to find the PyQt4 
DLLs in a Windows installation?  Secondarily, how is this search path set?

--
http://mail.python.org/mailman/listinfo/python-list

Re: lists of variables

[Alf P. Steinbach]

* Michael Pardee:

I'm relatively new to python and I was very surprised by the following behavior:


a=1
b=2


'a' refers to an object representing the integer 1.

Since 1 is an immutable value you can just as well think of it as 'a' containing 
the value 1, because a reference to an immutable value is for nearly all 
practical purposes indistinguishable from that value.


'b' refers to an object representing the integer 2, which, again, since integers 
are immutable, you can just as well think of as 'b' containing the value 2.




mylist=[a,b]


A 'list' object (an array) is constructed, and 'mylist' is set to refer to it. 
The array has two items. The first item is set to refer to same object as 'a' (a 
reference copy), the second item is set to refer to the same object as 'b'.


But since integers are immutable you can just as well think of it as a copying 
of the integer values.


Immutable values allow you to think of handling the values directly.



print mylist

[1, 2]

a=3


This changes what 'a' refers to.

It does not change what the first array element refers to.



print mylist

[1, 2]

Whoah!  Are python lists only for literals?


No, that's an irrelevant notion.



 Nope:


c={}
d={}


Here 'c' now refers to a dictionary object, which is mutable (can be changed).

And 'd' refers to another dictionary object.



mydlist=[c,d]
print mydlist

[{}, {}]


A 'list' array object is constructed, and 'mydlist' is set to refer to it.

The first item in the array is set to refer to the same object as 'c' refers to, 
namely a mutable dictionary object (currently an empty dictionary).


The second item in the array is set to refer to the same object as 'd' refers 
to, namely a mutable dictionary object (currently an empty dictionary).


It's the same that happened earlier with the integers.

The only difference, but it's a significant one, is that now the referred to 
objects are mutable, that is, they can be changed.




c['x']=1


The dictionary object referred to by 'c' is updated to now have a key 'x' with 
associated value 1.


In more gory detail: the key is associated with a reference to an object 
representing 1, but again, since integer values are immutable you can think of 
this as a direct association with the value; the reference view of this 
association is mostly[1] only relevant when the referred to object is mutable.


Since the dictionary object that you're changing is referred to by both 'c' and 
the first item of 'mydlist', both of these reflect the change.




print mydlist

[{'x': 1}, {}]


Yep.



So it looks like variables in a list are stored as object references.


You mean items in a list are references to objects. Yes.



This seems to confirm that:

mydlist[1]['y']=4

print mydlist

[{}, {'y': 4}]


To check that you should instead have printed 'd', where you'd also see the 
change, since 'd' refers to the same dictionary object as 'mydlist[1]' does.




So I figure my initial example doesn't work because if you assign a
literal to something it is changing the object.


No, it has nothing to do with literals.

With the current language definition[2], as of Python 2.x and 3.x, it's very 
simple: assignments copy references, and that's all they do.


References to immutable objects allow you to think of values being copied 
around, which except for checking the identities of those objects (seldom 
relevant) yields the exact same conclusions about the effect of operations, but 
that's only because those immutable objects never change.


What you did above was to copy references to mutable objects, objects that can 
change.


Then the value-copying view breaks down.



 But modifying a list
or dict (as long as you don't re-construct it) does not change the
object.


A simple way to think of this is copying references.

Objects are never copied by Python assignments.

The assignments only copy references.



I can think of some ways to work around this, including using single
element lists as "pointers":


aa=[1]
bb=[2]
myplist=[aa,bb]
print myplist

[[1], [2]]

aa[0]=3
print myplist

[[3], [2]]


This is the same as your last example above, except that now you're using 'list' 
arrays as the referred to mutable objects, instead of 'dict' dictionary objects.




But what would be "the python way" to accomplish "list of variables"
functionality?


Possibly, if you think of assignments as copying references, you won't need that 
notion.



Cheers & hth.,

- Alf

Notes:
[1] The reference view can be relevant also for immutable objects in (at least) 
two cases. One is when the program logic depends on object identity, obtainable 
via the id function, which is just bad programming, but I mention it for 
completeness. The other is where you have a really large immutable object, such 
as in Python 2.x a very large 'long' value; then assignments would be 
inefficient if the value was actually copied, but since assignments only copy 
references in Python you can blissfully disregard the siz

Re: Dictionary or Database—Please advise

[CM]

On Feb 26, 10:58 am, Jeremy  wrote:
> I have lots of data

How much is "lots"?

> that I currently store in dictionaries.  However,
> the memory requirements are becoming a problem.  I am considering
> using a database of some sorts instead, but I have never used them
> before.  Would a database be more memory efficient than a dictionary?

What do you mean by more efficient?

> I also need platform independence without having to install a database
> and Python interface on all the platforms I'll be using.  Is there
> something built-in to Python that will allow me to do this?

The SQLite datbase engine is built into Python 2.5 and up.  I have
heard on this list that there may be problems with it with Python 2.6
and up on Linux, but I've stayed with 2.5 and it works fine for me on
WinXP, Vista, and Linux.

You can use it as a disk-stored single database file, or an in-memory-
only database.  The SQLite website (http://www.sqlite.org/) claims it
is the "most widely deployed SQL database engine in the world.", for
what that's worth.

Have a look at this:
http://docs.python.org/library/sqlite3.html

Che



>
> Thanks,
> Jeremy

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Is this secure?

[Peter Pearson]

On Wed, 24 Feb 2010 21:02:07 +0100, mk  wrote:
[snip]
>
> rand_str_SystemRandom_seeding
> mean 3845.15384615 std dev 46.2016419186
> l 3926 1.75 std devs away from mean
> y 3916 1.53 std devs away from mean
> d 3909 1.38 std devs away from mean
> a 3898 1.14 std devs away from mean
> p 3898 1.14 std devs away from mean
> c 3889 0.95 std devs away from mean
> u 3884 0.84 std devs away from mean
> j 3873 0.60 std devs away from mean
> n 3873 0.60 std devs away from mean
> w 3866 0.45 std devs away from mean
> x 3863 0.39 std devs away from mean
> r 3855 0.21 std devs away from mean
> m 3852 0.15 std devs away from mean
> b 3841 -0.09 std devs away from mean
> t 3835 -0.22 std devs away from mean
> o 3829 -0.35 std devs away from mean
> k 3827 -0.39 std devs away from mean
> i 3821 -0.52 std devs away from mean
> s 3812 -0.72 std devs away from mean
> q 3806 -0.85 std devs away from mean
> v 3803 -0.91 std devs away from mean
> g 3799 -1.00 std devs away from mean
> h 3793 -1.13 std devs away from mean
> e 3782 -1.37 std devs away from mean
> f 3766 -1.71 std devs away from mean
> z 3758 -1.89 std devs away from mean

Chi2 = 14.43, 25 d.f., prob = 0.046362.
The observed distribution is SIGNIFICANTLY CLOSER
to the uniform distribution than reasonable by chance.

> rand_str_SystemRandom_noseeding
> mean 3845.15384615 std dev 55.670522726
> i 3961 2.08 std devs away from mean
> r 3911 1.18 std devs away from mean
> e 3910 1.16 std devs away from mean
> m 3905 1.08 std devs away from mean
> a 3901 1.00 std devs away from mean
> u 3893 0.86 std devs away from mean
> t 3882 0.66 std devs away from mean
> w 3872 0.48 std devs away from mean
> s 3870 0.45 std devs away from mean
> c 3868 0.41 std devs away from mean
> n 3866 0.37 std devs away from mean
> q 3865 0.36 std devs away from mean
> k 3863 0.32 std devs away from mean
> y 3848 0.05 std devs away from mean
> j 3836 -0.16 std devs away from mean
> v 3830 -0.27 std devs away from mean
> f 3829 -0.29 std devs away from mean
> z 3829 -0.29 std devs away from mean
> g 3827 -0.33 std devs away from mean
> l 3818 -0.49 std devs away from mean
> b 3803 -0.76 std devs away from mean
> d 3803 -0.76 std devs away from mean
> p 3756 -1.60 std devs away from mean
> x 3755 -1.62 std devs away from mean
> h 3744 -1.82 std devs away from mean
> o 3729 -2.09 std devs away from mean

Chi2 = 20.96, 25 d.f., prob = 0.304944.
The observed distribution is not significantly different
from the uniform distribution.

> rand_str_custom
> mean 3517.15384615 std dev 40.7541336343
> i 3586 1.69 std devs away from mean
> a 3578 1.49 std devs away from mean
> e 3575 1.42 std devs away from mean
> m 3570 1.30 std devs away from mean
> q 3562 1.10 std devs away from mean
> c 3555 0.93 std devs away from mean
> g 3552 0.86 std devs away from mean
> w 3542 0.61 std devs away from mean
> p 3536 0.46 std devs away from mean
> x 3533 0.39 std devs away from mean
> s 3528 0.27 std devs away from mean
> o 3524 0.17 std devs away from mean
> d 3516 -0.03 std devs away from mean
> t 3515 -0.05 std devs away from mean
> h 3511 -0.15 std devs away from mean
> v 3502 -0.37 std devs away from mean
> z 3502 -0.37 std devs away from mean
> b 3500 -0.42 std devs away from mean
> f 3496 -0.52 std devs away from mean
> u 3492 -0.62 std devs away from mean
> l 3486 -0.76 std devs away from mean
> r 3478 -0.96 std devs away from mean
> n 3476 -1.01 std devs away from mean
> j 3451 -1.62 std devs away from mean
> k 3450 -1.65 std devs away from mean
> y 3430 -2.14 std devs away from mean

Chi2 = 12.28, 25 d.f., prob = 0.015815.
The observed distribution is SIGNIFICANTLY CLOSER
to the uniform distribution than reasonable by chance.

> It would appear that SystemRandom().choice is indeed best (in terms of 
> how much the counts stray from mean in std devs), but only after seeding 
> it with os.urandom.

I don't see any reason to worry about any of the three, except
perhaps that the first and last are surprisingly uniform.

-- 
To email me, substitute nowhere->spamcop, invalid->net.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: collections use __next__() in python 2.6?

[Raymond Hettinger]

On Feb 26, 10:08 am, Gary Robinson  wrote:
> The Python 2.6.4 docs for collections 
> athttp://docs.python.org/library/collections.htmlsay that __next__() is an 
> abstract method for the Iterable ABC. But my understanding is that __next__() 
> isn't supposed to be used until Python 3. Also, I'm using the Mapping ABC, 
> which inherits from Iterable, and it doesn't seem to work if I define 
> __next__(); I am not seeing problems if I define next() instead.
>
> What am I missing?

It's a typo.
The abstract method is next().


Raymond
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Is this secure?

[Peter Pearson]

On Wed, 24 Feb 2010 20:16:24 +0100, mk  wrote:
> On 2010-02-24 20:01, Robert Kern wrote:
>> I will repeat my advice to just use random.SystemRandom.choice() instead
>> of trying to interpret the bytes from /dev/urandom directly.
>
> Out of curiosity:
>
> def gen_rand_string(length):
>  prng = random.SystemRandom()
>  chars = []
>  for i in range(length):
>  chars.append(prng.choice('abcdefghijklmnopqrstuvwxyz'))
>  return ''.join(chars)
>
> if __name__ == "__main__":
>  chardict = {}
>  for i in range(1):
> ##w = gen_rand_word(10)
>  w = gen_rand_string(10)
>  count_chars(chardict, w)
>  counts = list(chardict.items())
>  counts.sort(key = operator.itemgetter(1), reverse = True)
>  for char, count in counts:
>  print char, count
>
>
> s 3966
> d 3912
> g 3909
> h 3905
> a 3901
> u 3900
> q 3891
> m 3888
> k 3884
> b 3878
> x 3875
> v 3867
> w 3864
> y 3851
> l 3825
> z 3821
> c 3819
> e 3819
> r 3816
> n 3808
> o 3797
> f 3795
> t 3784
> p 3765
> j 3730
> i 3704
>
> Better, although still not perfect.

What would be perfect?  Surely one shouldn't be happy if all the
tallies come out exactly equal: that would be a blatant indication
of something very nonrandom going on.

The tallies given above give a chi-squared value smack in the
middle of the range expected for random sampling of a uniform
distribution (p = 0.505).  So the chi-squared metric of
goodness-of-fit to a unifom distribution says you're doing fine.

-- 
To email me, substitute nowhere->spamcop, invalid->net.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Possible to import from a cStringIO file object vs. file on disk?

[python]

Steve,

> You'll need to write a custom importer. PEP 302 contains the necessary 
> details.

Thanks for pointing me to PEP 302 and the imp module. 

It looks like "imp.load_compiled(name, pathname[, file])" is what I
need, but the description of this method (and similar methods) has the
following disclaimer:

Quote: "The file argument is the byte-compiled code file, open for
reading in binary mode, from the beginning. It must currently be a real
file object, not a user-defined class emulating a file." [1]

I tried using a cStringIO object vs. a real file object, but the help
documentation is correct - only a real file object can be used.

Any ideas on why these modules would impose such a restriction or is
this just an historical artifact?

Are there any techniques I can use to avoid this physical file
requirement?

Thanks,
Malcolm

[1] http://docs.python.org/library/imp.html#imp.load_module

pyt...@bdurham.com wrote:
> Is it possible to import from a cStringIO file object (containing
> compiled byte code) vs. a physical file on disk?
>  
> I'm thinking that this is possible, given Python's ability to import
> from zip files, but I'm not sure where to look next.
>  
> Thank you,
> Malcolm
> 


regards
 Steve
-- 
Steve Holden   +1 571 484 6266   +1 800 494 3119
PyCon is coming! Atlanta, Feb 2010  http://us.pycon.org/
Holden Web LLC http://www.holdenweb.com/
UPCOMING EVENTS:http://holdenweb.eventbrite.com/

-- 
http://mail.python.org/mailman/listinfo/python-list


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to end TCP socket data while using readline()?

[MRAB]

Arjun Chennu wrote:

No need to flush because you're writing to a file and it'll be flushed

anyway when you close it.


True. I had introduced those flush lines while I was trying to 
troubleshoot this annoying situation. :-)


You've closed the file view, but the underlying socket is still open.


The server will see EOF only when the socket is closed by the client,
and closing the file view doesn't close the socket itself.


That's what I intended to do. Keep the socket open, but close the file 
object so that the direction of transfer can be reversed and a "done" 
message can be sent to the client.


TCP is meant to facilitate two-directional transfer innit? So how do I 
do that while using file objects? If i have to close the socket to do 
that, it seems a bit wasteful in terms of network 'transactions'


Thanks for your reply. I've already confirmed that closing the socket 
does indeed move the program ahead --- but I'd like to now make 
two-directional comm. possible.



Here's a trick borrowed from the POP3 format (used for email).

Server code:
...
cf = conn.makefile('r', 0)# file like obj for socket
lf = open('ccs.txt', 'w')
for line in cf:
if line == '.end\n':
break
if line.startswith('..'):
line = line[1 : ]
lf.write(line)
sys.stdout.write(line)
print len(line)
lf.close()
cf.close()
...


Client code:
...
cf = s.makefile('w', 0)
for line in sfp.readlines():
if line.startswith('.'):
cf.write('.')
cf.write(line)
print len(line)
cr.write('.end\n')
print 'close'
cf.close()
...
--
http://mail.python.org/mailman/listinfo/python-list

Re: Six Minutes and fourty two seconds

[mk]

Tobiah wrote:
Now that I use python, this is the amount of time 
per day that I spend adding forgotten semicolons while

debugging other languages.


My objects are flat and I don't know who's Guido.

I blame it all on Python.

How about a PEP "Let's make Python look like PHP"?

Regards,
mk

--
http://mail.python.org/mailman/listinfo/python-list

Re: SystemError: error return without exception set

[MRAB]

Shailendra wrote:

Hi All,
I am getting error in "SystemError: error return without exception
set" which i am not able debug. I made following test program which
gives same error.

=test.py==

import numpy

class testClass:
def __init__(self,param1=1,param2=2):
self.param1,self.param2=param1,param2

def __str__(self):
return 'param1=%d param2=%d'%(self.param1,self.param2)


class makeLotOftestClass:
def __init__(self,paramlist1=None,paramlist2=None):
self.holder= numpy.empty((len(paramlist1),len(paramlist2)))
for i in range(len(paramlist1)):
for j in range(len(paramlist2)):

self.holder[i,j]=testClass(param1=paramlist1[i],param2=paramlist2[j])

def __str__():
str=''
for testclass in numpy.raven(self.holder):
str=str+testclass.__str__()
return str

if __name__=='__main__':

lotofclass=makeLotOftestClass(paramlist1=[10,20,30,40],paramlist2=[110,120,130,140])
print lotofclass

==

$ python test.py
Traceback (most recent call last):
  File "test.py", line 25, in 

lotofclass=makeLotOftestClass(paramlist1=[10,20,30,40],paramlist2=[110,120,130,140])
  File "test.py", line 16, in __init__
self.holder[i,j]=testClass(param1=paramlist1[i],param2=paramlist2[j])
SystemError: error return without exception set

Can someone point out why is this exception. It seems to be a simple
code. Google search for the error message did not help much.


As I see it, there are 2 problems here:

1. numpy arrays contain numbers, but what you're trying to put into the
array are not numbers.

2. The numpy module is trying to raise an exception but it isn't setting
up the exception correctly internally, which is a bug in numpy itself.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Six Minutes and fourty two seconds

[Matt Nordhoff]

Tobiah wrote:
> Now that I use python, this is the amount of time 
> per day that I spend adding forgotten semicolons while
> debugging other languages.

You can fix that by not using other languages. :>
-- 
Matt Nordhoff
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: lists of variables

[Aahz]

In article ,
Michael Pardee   wrote:
>
>I'm relatively new to python and I was very surprised by the following
>behavior:

http://starship.python.net/crew/mwh/hacks/objectthink.html
-- 
Aahz (a...@pythoncraft.com)   <*> http://www.pythoncraft.com/

"Many customs in this life persist because they ease friction and promote
productivity as a result of universal agreement, and whether they are
precisely the optimal choices is much less important." --Henry Spencer
-- 
http://mail.python.org/mailman/listinfo/python-list

Six Minutes and fourty two seconds

[Tobiah]

Now that I use python, this is the amount of time 
per day that I spend adding forgotten semicolons while
debugging other languages.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: When will Java go mainstream like Python?

[John Nagle]

Gregory Ewing wrote:

Lawrence D'Oliveiro wrote:
And then there’s caching. Modern CPUs owe most of their speed to 
assumptions that programs will obey locality of reference. 
Pointer-chasing is a cache-

hostile activity.


Another thing to consider is the rate at which garbage is
created. Java's fundamental types (ints, floats, etc.) are
unboxed, and objects are only used for relatively heavyweight
things. So you can do quite a lot of useful computation in
Java without creating any objects or leaving behind any
garbage.

In Python, on the other hand, you can't even do arithmetic
without creating and destroying intermediate objects at
every step. 


This is really a CPython implementation problem.  Face it,
CPython is a "naive interpreter".  It pretty much does the obvious.
In other words, the code for the worst case is used for all cases.
It doesn't optimize out reference count updates, do type inference
to avoid unnecessary boxing, or figure out at compile time which
objects could be "slotted" and don't need dictionary lookups for
fields.  Plus, of course, there's the GIL problem.

Reference counts aren't inherently slow.  About 90% of reference
count updates could be optimized out.  The compiler needs to understand
what's a temporary object and what isn't.  See

http://blogs.msdn.com/abhinaba/archive/2009/02/09/back-to-basics-optimizing-reference-counting-garbage-collection.aspx

The language isn't inherently slow, but CPython is.

The Shed Skin developer has shown what's possible.  He
doesn't have enough resources to finish a full implementation,
but he's on the right track.  There's grumbling about the
restrictions in Shed Skin, but only some of Shed Skin's
restrictions are inherently necessary.  The one Shed Skin
developer has accomplished far more than the PyPy army of ants.

John Nagle
--
http://mail.python.org/mailman/listinfo/python-list

collections use __next__() in python 2.6?

[Gary Robinson]

The Python 2.6.4 docs for collections at 
http://docs.python.org/library/collections.html say that __next__() is an 
abstract method for the Iterable ABC. But my understanding is that __next__() 
isn't supposed to be used until Python 3. Also, I'm using the Mapping ABC, 
which inherits from Iterable, and it doesn't seem to work if I define 
__next__(); I am not seeing problems if I define next() instead.

What am I missing?
-- 

Gary Robinson
CTO
Emergent Music, LLC
personal email: gary...@me.com
work email: grobin...@flyfi.com
Company: http://www.flyfi.com
Blog:http://www.garyrobinson.net
-- 
http://mail.python.org/mailman/listinfo/python-list

SystemError: error return without exception set

[Shailendra]

Hi All,
I am getting error in "SystemError: error return without exception
set" which i am not able debug. I made following test program which
gives same error.

=test.py==

import numpy

class testClass:
def __init__(self,param1=1,param2=2):
self.param1,self.param2=param1,param2

def __str__(self):
return 'param1=%d param2=%d'%(self.param1,self.param2)


class makeLotOftestClass:
def __init__(self,paramlist1=None,paramlist2=None):
self.holder= numpy.empty((len(paramlist1),len(paramlist2)))
for i in range(len(paramlist1)):
for j in range(len(paramlist2)):

self.holder[i,j]=testClass(param1=paramlist1[i],param2=paramlist2[j])

def __str__():
str=''
for testclass in numpy.raven(self.holder):
str=str+testclass.__str__()
return str

if __name__=='__main__':

lotofclass=makeLotOftestClass(paramlist1=[10,20,30,40],paramlist2=[110,120,130,140])
print lotofclass

==

$ python test.py
Traceback (most recent call last):
  File "test.py", line 25, in 

lotofclass=makeLotOftestClass(paramlist1=[10,20,30,40],paramlist2=[110,120,130,140])
  File "test.py", line 16, in __init__
self.holder[i,j]=testClass(param1=paramlist1[i],param2=paramlist2[j])
SystemError: error return without exception set

Can someone point out why is this exception. It seems to be a simple
code. Google search for the error message did not help much.

Thanks,
Shailendra
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Docstrings considered too complicated

[mk]

Roy Smith wrote:


/**
 * Tracing facility. Writes the message to the specified output stream.
 * If output stream is NULL, writes the message to the process log.
 *
 * @param  msg_id  The message id to use for lookup.
 * @param  ostrThe output stream.
 * @param  p1  The first substition parameter.
 * @param  p2  The second substition parameter.
 * @param  p3  The third substition parameter.
 * @param  p4  The fourth substition parameter.
 * @param  p5  The fifth substition parameter.
 * @param  p6  The sixth substition parameter.
 * @param  p7  The seventh substition parameter.
 * @param  p8  The eigth substition parameter.
 * @param  p9  The ninth substition parameter.
*/


Well at least they did explain something. ;-) You should be happy you 
don't have to deal with PHP programmers that tend to write 
20-positional-argument function AND programmer 1 knows what params 1-7 
do, programmer 2 knows what params 8-15 do and nobody knows what params 
16-20 do.


Seriously.

Regards,
mk

--
http://mail.python.org/mailman/listinfo/python-list

How to end TCP socket data while using readline()?

[Arjun]

Hi, I have a small script that runs a TCP server. A client connects to
this server and transmits a stored file line-by-line, and then waits
for a confirmation "done". However, when I run them the first loop
never really ends -- as the TCP server keeps expecting more data. I am
using a file-like-object, and so somehow I have to communicate to the
server that it is the end-of-file.

here is some server code

sock1.bind(('', port))
print "Listening at port: ", port
sock1.listen(1)  # listening socket
(conn, addr) = sock1.accept()# connected socket
print 'Client (localhost) port: ', addr[1]

cf = conn.makefile('r',0)# file like obj for socket

lf = open('ccs.txt','w')
for line in cf:
lf.write(line)
lf.flush()
sys.stdout.write(line)
print len(line)

lf.close()  (*here*)
cf.close()
cf = conn.makefile('w',0)
print len(line)
print 'sendin'
stat = 'done'
cf.write(stat)
print stat
print 'sent'
cf.close()
print 'script done & connection closed'


The client is sending the lines through this code:

s.connect((host,port))
sfp = open("dcs.txt")
   # script = sfp.readlines()
stat = 'still'
cf = s.makefile('w',0)
for line in sfp.readlines():
cf.write(line)
print len(line)
print 'close'
cf.flush()
cf.close()
sfp.close()

cf = s.makefile('r',0)
print stat, 'waiting'
stat = cf.readline()
print stat, 'waiting'  # this should become "done waiting"
cf.close()
s.close()


So what I am wondering is:

1. Using a file-like object means that the socket becomes uni-
directional, until the mode of the file object is changed from 'r' to
'w' (or vice versa). This seems inefficient, and rather unPythonesque.
Can somebody tell me if there is a more elegant way of receiving all
the lines from the client, and then sending a "done" message to the
client?

2.  Where I have marked (*here*) in the server code, is where the
execution seems to stay waiting... apparently for more input from the
client. But then when I force-terminate the client script, the
execution continues on the server-side!! So then it sends the "done"
status to a (already dead) client, and then exits. How do I get the
server to know when the EOF has been reached while using FLOs?

Input on this will be much appreciated, because the documentation for
readlines() didn't help me with this.

Cheers,
A
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Docstrings considered too complicated

[Jean-Michel Pichavant]

Roy Smith wrote:

In article ,
 Tim Daneliuk  wrote:

  

On 2/24/2010 2:23 PM, Andreas Waldenburger wrote:


Hi all,

a company that works with my company writes a lot of of their code in
Python (lucky jerks). I've seen their code and it basically looks like
this:

"""Function that does stuff"""
def doStuff():
while not wise(up):
yield scorn

Now my question is this: How do I kill these people without the
authorities thinking they didn't deserve it?

/W

  

Reminiscent of:

mov  AX,BX   ; Move the contents of BX into AX

And, yes, I've actually seen that as well as:

; This is a comment



OK, if we're going to do this, how about this one, that I just found 
yesterday in some production C++ code.  I'm so glad somebody took the time 
to explain to me what p7 through p9 are.  I never would have figured it out 
otherwise.


/**
 * Tracing facility. Writes the message to the specified output stream.
 * If output stream is NULL, writes the message to the process log.
 *
 * @param  msg_id  The message id to use for lookup.
 * @param  ostrThe output stream.
 * @param  p1  The first substition parameter.
 * @param  p2  The second substition parameter.
 * @param  p3  The third substition parameter.
 * @param  p4  The fourth substition parameter.
 * @param  p5  The fifth substition parameter.
 * @param  p6  The sixth substition parameter.
 * @param  p7  The seventh substition parameter.
 * @param  p8  The eigth substition parameter.
 * @param  p9  The ninth substition parameter.
*/
  

just in case the first sub param would be p0 :-)


JM
--
http://mail.python.org/mailman/listinfo/python-list

Re: Dictionary or Database_Please advise

[mk]

D'Arcy J.M. Cain wrote:


Or PostgreSQL.  It's free, runs on lots of platforms, has good Python
support, and there's lots of people on the net who know it and are
willing to give help and advice.  In addition, it is a truly enterprise
level, SQL standard, fully transactional database.  Don't start with
MySQL and uprade to PostgreSQL later when you get big.  Start with the
best one now and be ready.


I second that: I burned my fingers on MySQL quite a few times and don't 
want to have anything to do with it anymore. Eventually you hit the wall 
with MySQL (although I haven't tested latest and best, perhaps they 
improved).


E.g. don't even get me started on replication that tends to randomly 
fizzle out quietly without telling you anything about it. Or that 
FOREIGN KEY is accepted but referential integrity is not enforced. Or 
that if you want real transactions, you have to choose InnoDB table type 
but then you lose much of the performance. Etc.


No, if you have a choice, avoid MySQL and go for PGSQL. It's fantastic, 
if (necessarily) complex.


Regards,
mk

--
http://mail.python.org/mailman/listinfo/python-list

Re: Docstrings considered too complicated

[Jean-Michel Pichavant]

Andreas Waldenburger wrote:

On Fri, 26 Feb 2010 09:09:36 -0600 Tim Daneliuk 
wrote:

  

On 2/24/2010 2:23 PM, Andreas Waldenburger wrote:


[stuff]
  

Reminiscent of:

mov  AX,BX   ; Move the contents of BX into AX



Well, there might be some confusion there as to what gets moved where,
wouldn't you say? I guess this goes away after a couple of months,
though.
  


I agree to that statement, I was surprised that mov AX,BX assumes that 
BX is the source, and AX the destination. I never programmed in 
assembler though.


JM
--
http://mail.python.org/mailman/listinfo/python-list

Re: Dictionary or Database—Please advise

[mk]

Jeremy wrote:


Shelve looks like an interesting option, but what might pose an issue
is that I'm reading the data from a disk instead of memory.  I didn't
mention this in my original post, but I was hoping that by using a
database it would be more memory efficient in storing data in RAM so I
wouldn't have to read from (or swap to/from) disk.  Would using the
shelve package make reading/writing data from disk faster since it is
in a binary format?


Read the docs:

"class shelve.BsdDbShelf(dict[, protocol=None[, writeback=False]])¶
A subclass of Shelf which exposes first(), next(), previous(), 
last() and set_location() which are available in the bsddb module but 
not in other database modules. The dict object passed to the constructor 
must support those methods. This is generally accomplished by calling 
one of bsddb.hashopen(), bsddb.btopen() or bsddb.rnopen(). The optional 
protocol and writeback parameters have the same interpretation as for 
the Shelf class."


Apparently using shelve internally gives you option of using bsddb, 
which is good news: bsddb is B-tree DB, which is highly efficient for 
finding keys. I would recommend bsddb.btopen(), as it creates B-tree DB 
(perhaps other implementations, like anydb or hash db are good as well, 
but I personally didn't test them out).


I can't say for Berkeley DB implementation, but in general B-tree 
algorithm has O(log2 n) complexity for finding keys, which roughly means 
that if you need to find particular key in a db of 1 million keys, 
you'll probably need ~20 disk accesses (or even less if some keys looked 
at in the process of search happen to be in the same disk sectors). So 
yes, it's highly efficient.


Having said that, remember that disk is many orders of magnitude slower 
than RAM, so it's no free lunch.. Nothing will beat memory-based data 
structure when it comes to speed (well new flash or hybrid disks perhaps 
could significantly improve in comparison to current mainstream 
mechanical-platter disks? there are some hyper-fast storage hardware 
companies out there, although they tend to charge arm and leg for their 
stuff for now).


Caveat: Berkeley DB is dual-licensed -- if you're using it for 
commercial work, it might be that you'd need to buy a license for it. 
Although I have had no experience with this really, if someone here did 
perhaps they will shed some light on it?


Regards,
mk



--
http://mail.python.org/mailman/listinfo/python-list

Re: Dictionary or Database—Please advise

[mk]

Jeremy wrote:

I have lots of data that I currently store in dictionaries.  However,
the memory requirements are becoming a problem.  I am considering
using a database of some sorts instead, but I have never used them
before.  Would a database be more memory efficient than a dictionary?
I also need platform independence without having to install a database
and Python interface on all the platforms I'll be using.  Is there
something built-in to Python that will allow me to do this?


Since you use dictionaries, I guess that simple store saving key:value 
will do?


If so, bsddb support built into Python will do just nicely.

bsddb is multiplatform, although I have not personally tested if a 
binary db created on one platform will be usable on another. You'd have 
to check this.


Caveat: from what some people say I gather that binary format between 
bsddb versions tends to change.


There's also ultra-cool-and-modern Tokyo Cabinet key:value store with 
Python bindings:


http://pypi.python.org/pypi/pytc/

I didn't test it, though.


Regards,
mk

--
http://mail.python.org/mailman/listinfo/python-list

Re: Dictionary or Database輝lease advise

[km]

Hi,

Probably u should try couchdb! its a document oriented database. (
apache.couchdb.org)
u can store your dictionaries as json documents and yes they  are simple
text files; data structures cna be directly stored into JSON documents.
memory efficient too..
python module @ http://code.google.com/p/couchdb-python/

HTH
Krishna
~~~
On Sat, Feb 27, 2010 at 1:39 AM, Roy Smith  wrote:

> In article
> <891a98fa-c398-455a-981f-bf72af772...@s36g2000prh.googlegroups.com>,
>  Jeremy  wrote:
>
> > I have lots of data that I currently store in dictionaries.  However,
> > the memory requirements are becoming a problem.  I am considering
> > using a database of some sorts instead, but I have never used them
> > before.  Would a database be more memory efficient than a dictionary?
> > I also need platform independence without having to install a database
> > and Python interface on all the platforms I'll be using.  Is there
> > something built-in to Python that will allow me to do this?
> >
> > Thanks,
> > Jeremy
>
> This is a very vague question, so it'll get a vague answer :-)
>
> If you have so much data that you're running into memory problems, then
> yes, storing the data externally in an disk-resident database seems like a
> reasonable idea.
>
> Once you get into databases, platform independence will be an issue.  There
> are many databases out there to pick from.  If you want something which
> will work on a lot of platforms, a reasonable place to start looking is
> MySQL.  It's free, runs on lots of platforms, has good Python support, and
> there's lots of people on the net who know it and are willing to give help
> and advice.
>
> Databases have a bit of a learning curve.  If you've never done any
> database work, don't expect to download MySql (or any other database) this
> afternoon and be up and running by tomorrow.
>
> Whatever database you pick, you're almost certainly going to end up having
> to install it wherever you install your application.  There's no such thing
> as a universally available database that you can expect to be available
> everywhere.
>
> Have fun!
> --
> http://mail.python.org/mailman/listinfo/python-list
>
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Dictionary or Database_Please advise

[D'Arcy J.M. Cain]

On Fri, 26 Feb 2010 11:39:51 -0500
Roy Smith  wrote:
> Once you get into databases, platform independence will be an issue.  There 
> are many databases out there to pick from.  If you want something which 
> will work on a lot of platforms, a reasonable place to start looking is 
> MySQL.  It's free, runs on lots of platforms, has good Python support, and 
> there's lots of people on the net who know it and are willing to give help 
> and advice.

Or PostgreSQL.  It's free, runs on lots of platforms, has good Python
support, and there's lots of people on the net who know it and are
willing to give help and advice.  In addition, it is a truly enterprise
level, SQL standard, fully transactional database.  Don't start with
MySQL and uprade to PostgreSQL later when you get big.  Start with the
best one now and be ready.

> Databases have a bit of a learning curve.  If you've never done any 
> database work, don't expect to download MySql (or any other database) this 
> afternoon and be up and running by tomorrow.

Whatever database you get, there will probably be plenty of tutorials.
See http://www.postgresql.org/docs/current/interactive/tutorial.html
for the PostgreSQL one.

-- 
D'Arcy J.M. Cain  |  Democracy is three wolves
http://www.druid.net/darcy/|  and a sheep voting on
+1 416 425 1212 (DoD#0082)(eNTP)   |  what's for dinner.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Dictionary or Database—Please advise

[Jeremy]

On Feb 26, 9:29 am, Chris Rebert  wrote:
> On Fri, Feb 26, 2010 at 7:58 AM, Jeremy  wrote:
> > I have lots of data that I currently store in dictionaries.  However,
> > the memory requirements are becoming a problem.  I am considering
> > using a database of some sorts instead, but I have never used them
> > before.  Would a database be more memory efficient than a dictionary?
> > I also need platform independence without having to install a database
> > and Python interface on all the platforms I'll be using.  Is there
> > something built-in to Python that will allow me to do this?
>
> If you won't be using the SQL features of the database, `shelve` might
> be another option; from what I can grok, I sounds like a dictionary
> stored mostly on disk rather than entirely in RAM (not 100% sure
> though):http://docs.python.org/library/shelve.html
>
> It's in the std lib and supports several native dbm libraries for its
> backend; one of them should almost always be present.
>
> Cheers,
> Chris
> --http://blog.rebertia.com

Shelve looks like an interesting option, but what might pose an issue
is that I'm reading the data from a disk instead of memory.  I didn't
mention this in my original post, but I was hoping that by using a
database it would be more memory efficient in storing data in RAM so I
wouldn't have to read from (or swap to/from) disk.  Would using the
shelve package make reading/writing data from disk faster since it is
in a binary format?

Jeremy
-- 
http://mail.python.org/mailman/listinfo/python-list

Exception in pydoc

[Ricardo Aráoz]

I'm developing an in house app. Many coders here are not fluent in
english, so docstrings must be in Spanish in spite of recommendations
that docstrings better be in English.
When I use accented characters (in this case an 'ó') in my docstrings I
get :
>>> help('OpMejoraBizobj')
Traceback (most recent call last):
  File "", line 1, in 
  File "C:\Python25\lib\site.py", line 346, in __call__
return pydoc.help(*args, **kwds)
  File "C:\Python25\lib\pydoc.py", line 1645, in __call__
self.help(request)
  File "C:\Python25\lib\pydoc.py", line 1687, in help
elif request: doc(request, 'Help on %s:')
  File "C:\Python25\lib\pydoc.py", line 1481, in doc
pager(title % desc + '\n\n' + text.document(object, name))
  File "C:\Python25\lib\pydoc.py", line 324, in document
if inspect.ismodule(object): return self.docmodule(*args)
  File "C:\Python25\lib\pydoc.py", line 1072, in docmodule
contents.append(self.document(value, key, name))
  File "C:\Python25\lib\pydoc.py", line 325, in document
if inspect.isclass(object): return self.docclass(*args)
  File "C:\Python25\lib\pydoc.py", line 1208, in docclass
contents = '\n'.join(contents)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position
115: ordinal not in range(128)

The file's first two lines are :
"""
#!/usr/bin/env python
# -*- coding: utf-8 -*-
"""

Does pydoc only deal with ASCII?



-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Docstrings considered too complicated

[Richard Brodie]

"Andreas Waldenburger"  wrote in message 
news:20100226173907.55676...@geekmail.invalid...

>> Reminiscent of:
>>
>> mov  AX,BX   ; Move the contents of BX into AX
>>
> Well, there might be some confusion there as to what gets moved where,
> wouldn't you say?

Depends on what assembler you're used to. I certainly find having the
operands that way round confusing.


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Possible to import from a cStringIO file object vs. file on disk?

[Steve Holden]

pyt...@bdurham.com wrote:
> Is it possible to import from a cStringIO file object (containing
> compiled byte code) vs. a physical file on disk?
>  
> I'm thinking that this is possible, given Python's ability to import
> from zip files, but I'm not sure where to look next.
>  
> Thank you,
> Malcolm
> 

You'll need to write a custom importer. PEP 302 contains the necessary
details.

regards
 Steve
-- 
Steve Holden   +1 571 484 6266   +1 800 494 3119
PyCon is coming! Atlanta, Feb 2010  http://us.pycon.org/
Holden Web LLC http://www.holdenweb.com/
UPCOMING EVENTS:http://holdenweb.eventbrite.com/

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Docstrings considered too complicated

[Phlip]

Andreas Waldenburger wrote:

> """Function that does stuff"""
> def doStuff():
>     while not wise(up):
>         yield scorn
>
> Now my question is this: How do I kill these people without the
> authorities thinking they didn't deserve it?

Their unit tests are just as complete, illustrative, and
administratively sanctioned, right?

--
  Phlip
  http://penbird.tumblr.com/
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Docstrings considered too complicated

[Roy Smith]

In article ,
 Tim Daneliuk  wrote:

> On 2/24/2010 2:23 PM, Andreas Waldenburger wrote:
> > Hi all,
> > 
> > a company that works with my company writes a lot of of their code in
> > Python (lucky jerks). I've seen their code and it basically looks like
> > this:
> > 
> > """Function that does stuff"""
> > def doStuff():
> > while not wise(up):
> > yield scorn
> > 
> > Now my question is this: How do I kill these people without the
> > authorities thinking they didn't deserve it?
> > 
> > /W
> > 
> 
> Reminiscent of:
> 
> mov  AX,BX   ; Move the contents of BX into AX
> 
> And, yes, I've actually seen that as well as:
> 
> ; This is a comment

OK, if we're going to do this, how about this one, that I just found 
yesterday in some production C++ code.  I'm so glad somebody took the time 
to explain to me what p7 through p9 are.  I never would have figured it out 
otherwise.

/**
 * Tracing facility. Writes the message to the specified output stream.
 * If output stream is NULL, writes the message to the process log.
 *
 * @param  msg_id  The message id to use for lookup.
 * @param  ostrThe output stream.
 * @param  p1  The first substition parameter.
 * @param  p2  The second substition parameter.
 * @param  p3  The third substition parameter.
 * @param  p4  The fourth substition parameter.
 * @param  p5  The fifth substition parameter.
 * @param  p6  The sixth substition parameter.
 * @param  p7  The seventh substition parameter.
 * @param  p8  The eigth substition parameter.
 * @param  p9  The ninth substition parameter.
*/
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Dictionary or Database�Please advise

[Roy Smith]

In article 
<891a98fa-c398-455a-981f-bf72af772...@s36g2000prh.googlegroups.com>,
 Jeremy  wrote:

> I have lots of data that I currently store in dictionaries.  However,
> the memory requirements are becoming a problem.  I am considering
> using a database of some sorts instead, but I have never used them
> before.  Would a database be more memory efficient than a dictionary?
> I also need platform independence without having to install a database
> and Python interface on all the platforms I'll be using.  Is there
> something built-in to Python that will allow me to do this?
> 
> Thanks,
> Jeremy

This is a very vague question, so it'll get a vague answer :-)

If you have so much data that you're running into memory problems, then 
yes, storing the data externally in an disk-resident database seems like a 
reasonable idea.

Once you get into databases, platform independence will be an issue.  There 
are many databases out there to pick from.  If you want something which 
will work on a lot of platforms, a reasonable place to start looking is 
MySQL.  It's free, runs on lots of platforms, has good Python support, and 
there's lots of people on the net who know it and are willing to give help 
and advice.

Databases have a bit of a learning curve.  If you've never done any 
database work, don't expect to download MySql (or any other database) this 
afternoon and be up and running by tomorrow.

Whatever database you pick, you're almost certainly going to end up having 
to install it wherever you install your application.  There's no such thing 
as a universally available database that you can expect to be available 
everywhere.

Have fun!
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Docstrings considered too complicated

[Andreas Waldenburger]

On Fri, 26 Feb 2010 09:09:36 -0600 Tim Daneliuk 
wrote:

> On 2/24/2010 2:23 PM, Andreas Waldenburger wrote:
> > [stuff]
> 
> Reminiscent of:
> 
> mov  AX,BX   ; Move the contents of BX into AX
> 
Well, there might be some confusion there as to what gets moved where,
wouldn't you say? I guess this goes away after a couple of months,
though.

> And, yes, I've actually seen that as well as:
> 
> ; This is a comment
> 
I hope it was in a tutorial-situation. Or maybe it was written by one of
those "ironic" programmers?

/W

-- 
INVALID? DE!

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Docstrings considered too complicated

[Andreas Waldenburger]

On Fri, 26 Feb 2010 15:50:25 +0100 Jean-Michel Pichavant
 wrote:

> Andreas Waldenburger wrote:
> 
> > And they use mixedCase function/method names.
> >   
> and ? whatIsTheProblem ?

Thanks for proving my point. ;)

No seriously though: Let it go. I wasn't being serious. As long as it
works and I don't have to work with it, I don't care how anybody writes
their code.

/W

-- 
INVALID? DE!

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Dictionary or Database—Please advise

[lbolla]

On Feb 26, 3:58 pm, Jeremy  wrote:
> I have lots of data that I currently store in dictionaries.  However,
> the memory requirements are becoming a problem.  I am considering
> using a database of some sorts instead, but I have never used them
> before.  Would a database be more memory efficient than a dictionary?
> I also need platform independence without having to install a database
> and Python interface on all the platforms I'll be using.  Is there
> something built-in to Python that will allow me to do this?
>
> Thanks,
> Jeremy

Maybe shelve would be enough for your needs?
http://docs.python.org/library/shelve.html
-- 
http://mail.python.org/mailman/listinfo/python-list