Re: Is there any way to minimize str()/unicode() objects memory usage [Python 2.6.4] ?
dmtr wrote: >> > Well... 63 bytes per item for very short unicode strings... Is there >> > any way to do better than that? Perhaps some compact unicode objects? >> >> There is a certain price you pay for having full-feature Python objects. > > Are there any *compact* Python objects? Optimized for compactness? > >> What are you trying to accomplish anyway? Maybe the array module can be >> of some help. Or numpy? > > Ultimately a dict that can store ~20,000,000 entries: (u'short > string' : (int, int, int, int, int, int, int)). I don't know to what extent it still applys but switching off cyclic garbage collection with import gc gc.disable() while building large datastructures used to speed up things significantly. That's what I would try first with your real data. Encoding your unicode strings as UTF-8 could save some memory. When your integers fit into two bytes, say, you can use an array.array() instead of the tuple. Peter -- http://mail.python.org/mailman/listinfo/python-list
Re: Perl -> Python unpack
On Sat, Aug 7, 2010 at 4:30 AM, James Mills wrote: > What does a* or A* translate to in Python when unpacking > binary data with struct.unpack(...) ? Fine I"ll answer my own question. Python _does not_ support implicit size in struct formatting rules. sizes are explicit meaning that you must "compute" the size you require. a* or A* would translate to 10s for example where you want a string with a size of 10. cheers James -- -- James Mills -- -- "Problems are solved by method" -- http://mail.python.org/mailman/listinfo/python-list
Re: Is there any way to minimize str()/unicode() objects memory usage ?[Python 2.6.4] ?
dmtr wrote: > > What I'm really looking for is a dict() that maps short unicode > strings into tuples with integers. But just having a *compact* list > container for unicode strings would help a lot (because I could add a > __dict__ and go from it). > At this point, I'd suggest to use one of the dbm modules, and pack the integers with struct.pack into a short string(s). Depending on your usage pattern, there are marked performance differences between dbhash, gdbm, and dbm implementations, so perhaps it would pay off to invest sometime in benchmarking. If your data are write-once, then cdb has excellent performance (but a different API). The file will be usually cached in RAM, so no need to worry about I/O bottlenecks... and if small enough, you can always put it into a ramdisk. If your strings are long enough, you can improve memory usage with a use of zlib.compress (dumb and unoptimal way of using compression, but easy and present in the std library) - but always verify if the compressed strings are _shorter_ than originals. -- --- | Radovan Garabík http://kassiopeia.juls.savba.sk/~garabik/ | | __..--^^^--..__garabik @ kassiopeia.juls.savba.sk | --- Antivirus alert: file .signature infected by signature virus. Hi! I'm a signature virus! Copy me into your signature file to help me spread! -- http://mail.python.org/mailman/listinfo/python-list
Re: Is there any way to minimize str()/unicode() objects memory usage [Python 2.6.4] ?
On Aug 6, 10:56 pm, Michael Torrie wrote: > On 08/06/2010 07:56 PM, dmtr wrote: > > > Ultimately a dict that can store ~20,000,000 entries: (u'short > > string' : (int, int, int, int, int, int, int)). > > I think you really need a real database engine. With the proper > indexes, MySQL could be very fast storing and retrieving this > information for you. And it will use your RAM to cache as it sees fit. > Don't try to reinvent the wheel here. No, I've tried. DB solutions are not even close in terms of the speed. Processing would take weeks :( Memcached or REDIS sort of work, but they are still a bit on the slow side, to be a pleasure to work with. The standard dict() container is *a lot* faster. It is also hassle free (accepting unicode keys/etc). I just wish there was a bit more compact dict container, optimized for large dataset and memory, not for speed. And with the default dict() I'm also running into some kind of nonlinear performance degradation, apparently after 10,000,000-13,000,000 keys. But I can't recreate this with a solid test case (see http://bugs.python.org/issue9520 ) :( -- Dmitry -- http://mail.python.org/mailman/listinfo/python-list
Re: new to python - trouble calling a function from another function
On 08/05/2010 01:25 AM, Brandon McCombs wrote: > How can that be? I don't get any errors when the script > executes. Surely this isn't some limitation I'm encountering? > > > yield (p.destination - self.currenteFloor) * TRAVELTIME, self ^ To be succinct, goUp() is apparently a generator function. "yield" means the function needs to be iterated over; calling it just sets up the generator and returns a generator object. So yeah your goUp() call will appear (and actually does) do nothing at all. -- http://mail.python.org/mailman/listinfo/python-list
Re: Is there any way to minimize str()/unicode() objects memory usage [Python 2.6.4] ?
On 08/06/2010 07:56 PM, dmtr wrote: > Ultimately a dict that can store ~20,000,000 entries: (u'short > string' : (int, int, int, int, int, int, int)). I think you really need a real database engine. With the proper indexes, MySQL could be very fast storing and retrieving this information for you. And it will use your RAM to cache as it sees fit. Don't try to reinvent the wheel here. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python "why" questions
On Fri, Aug 6, 2010 at 8:05 PM, Default User wrote:
> >From "the emperor's new clothes" department:
>
> 1) Why do Python lists start with element [0], instead of element [1]?
> "Common sense" would seem to suggest that lists should start with [1].
(In addition to the other good answers already given)
Well, "tradition" (originating from C) suggests otherwise. *Very* few
languages use 1-based indexing:
http://en.wikipedia.org/wiki/Comparison_of_programming_languages_(array)#Array_system_cross-reference_list
> 2) In Python 3, why is print a function only, so that: print "Hello, World"
> is not okay, but it must be print("Hello, World") instead? (Yeah, I know:
> picky, picky . . . )
One less special case to learn; makes the language more regular and
easier to learn. It also lets one write:
f = lambda x: print(x)
Which is not possible if print is a statement.
Cheers,
Chris
--
http://blog.rebertia.com
--
http://mail.python.org/mailman/listinfo/python-list
Re: Is there any way to minimize str()/unicode() objects memory usage [Python 2.6.4] ?
On Aug 6, 6:56 pm, dmtr wrote: > > > Well... 63 bytes per item for very short unicode strings... Is there > > > any way to do better than that? Perhaps some compact unicode objects? > > > There is a certain price you pay for having full-feature Python objects. > > Are there any *compact* Python objects? Optimized for compactness? Yes, but probably not in the way that'd be useful to you. Look at the array module, and also consider the third-party numpy library. They store compact arrays of numeric types (mostly) but they have character type storage as well. That probably won't help you, though, since you have variable-length strings. I don't know of any third-party types that can do what you want, but there might be some. Search PyPI. > > What are you trying to accomplish anyway? Maybe the array module can be > > of some help. Or numpy? > > Ultimately a dict that can store ~20,000,000 entries: (u'short > string' : (int, int, int, int, int, int, int)). My recommendation would be to use sqlite3. Only if you know for sure that it's too slow--meaning that you've actually tried it and it was too slow, and nothing else--then should you bother with a For that I'd probably go with a binary tree rather than a hash. So you have a huge numpy character array that stores all 20 million short strings end-to-end (in lexical order, so that you can look up the strings with a binary search), then you have an numpy integer array that stores the indices into this string where the word boundaries are, and then an Nx7 numpy integer array storing the int return vslues. That's three compact arrays. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: easy question on parsing python: "is not None"
Steven D'Aprano writes: > On Thu, 05 Aug 2010 12:07:53 -0400, wheres pythonmonks wrote: > > > P.S. Sorry for the top-post -- is there a way to not do top posts > > from gmail? I haven't used usenet since tin. > > Er, surely you can just move the cursor before you start typing??? I like to think that the cursor is placed at the top by default so that http://en.wikipedia.org/wiki/Posting_style#Interleaved_style> is easy: just travel down through the quoted material, removing it if not relevant and inserting one's responses where needed. -- \ “Two hands working can do more than a thousand clasped in | `\ prayer.” —Anonymous | _o__) | Ben Finney -- http://mail.python.org/mailman/listinfo/python-list
Re: Python "why" questions
Default User wrote:
>>From "the emperor's new clothes" department:
>
> 1) Why do Python lists start with element [0], instead of element [1]?
> "Common sense" would seem to suggest that lists should start with [1].
http://userweb.cs.utexas.edu/users/EWD/transcriptions/EWD08xx/EWD831.html
> 2) In Python 3, why is print a function only, so that: print "Hello,
> World"
> is not okay, but it must be print("Hello, World") instead? (Yeah, I know:
> picky, picky . . . )
"There should be one-- and preferably only one --obvious way to do it."
> 3) In Python 3, why does 2.0 / 3.0 display as 0., but 8 *
> 3.57 displays as 28.56 (rounded off to 2 decimal places)? And yet, in
> Python 2.6, 8 * 3.57 displays as 28.559?
http://mail.python.org/pipermail/python-dev/2009-October/092958.html
and replies
--
By ZeD
--
http://mail.python.org/mailman/listinfo/python-list
sched() function questions
I am currently using threading.timer to execute an event in my big chunk of code. This is causing a problem with sqlite, so I am trying to figure out the sched function import sched import time def timerfunc(): print "hello", time.time() return(time.time()) def delay(period): time.sleep(period) def some(): s.enterabs(900,1, timerfunc, () ) s.run() s = sched.scheduler(timerfunc, delay) print time.time() some() x = 0 while 1: print x x = x + 1 time.sleep(60) print str(s.queue()) What I am trying to do is mimic the Timer function, where my code will continue to function while my scheduled function executes in 15 minutes or 900 seconds. This doesn't do it though. Any help? -- http://mail.python.org/mailman/listinfo/python-list
Re: Python "why" questions
On Fri, 2010-08-06 at 22:05 -0500, Default User wrote:
> >From "the emperor's new clothes" department:
>
> 1) Why do Python lists start with element [0], instead of element
> [1]? "Common sense" would seem to suggest that lists should start
> with [1].
"Common sense" is wrong. There are many compelling advantages to
numbering from zero instead of one:
http://lambda-the-ultimate.org/node/1950
> 2) In Python 3, why is print a function only, so that: print "Hello,
> World" is not okay, but it must be print("Hello, World") instead?
> (Yeah, I know: picky, picky . . . )
The real question is, why was print so special in Python 2 that is can
be called without parentheses? The answer was "no reason" and it was
fixed in Python 3 to be consistent with the rest of the language.
> 3) In Python 3, why does 2.0 / 3.0 display as 0., but
> 8 * 3.57 displays as 28.56 (rounded off to 2 decimal places)? And
> yet, in Python 2.6, 8 * 3.57 displays as 28.559?
Because the code for displaying floats was improved in python 3. You
can follow the fascinating discussion on issue 7117:
http://bugs.python.org/issue7117
I can't defend the rounding issues of floating point numbers in general
- it's just "one of those things" that you have to deal with. But show
me a language where floats don't have this problem.
> And we wonder why kids don't want to learn to program.
Yeah, obscure language warts, that must be the reason.
Note to self: DNFTT...
Ryan
--
Ryan Kelly
http://www.rfk.id.au | This message is digitally signed. Please visit
r...@rfk.id.au| http://www.rfk.id.au/ramblings/gpg/ for details
--
http://mail.python.org/mailman/listinfo/python-list
Python "why" questions
>From "the emperor's new clothes" department:
1) Why do Python lists start with element [0], instead of element [1]?
"Common sense" would seem to suggest that lists should start with [1].
2) In Python 3, why is print a function only, so that: print "Hello, World"
is not okay, but it must be print("Hello, World") instead? (Yeah, I know:
picky, picky . . . )
3) In Python 3, why does 2.0 / 3.0 display as 0., but 8 *
3.57 displays as 28.56 (rounded off to 2 decimal places)? And yet, in
Python 2.6, 8 * 3.57 displays as 28.559?
And we wonder why kids don't want to learn to program.
--
http://mail.python.org/mailman/listinfo/python-list
Re: python interview quuestions
On 08/06/10 15:37, James Mills wrote: On Sat, Aug 7, 2010 at 6:28 AM, geremy condra wrote: If I had to wait 5 minutes while a candidate tried to solve this problem I would not hire them. Yes you do raise a valid point. It should really only take you a mere few seconds or so to write a solution to this. More over, it can be done in just a single line of Python. 7 if you're not very familiar with Python. While it *can* be done in one line, I'm not sure it's the most legible solution. Though I must say I like this one-line python version: for i in range(1, 101): print ((i%3==0 and 'fizz' or '') + (i%5==0 and 'buzz' or '')) or i (adjust "3" and "5" for your local flavor of fizzbuzz) I'm not sure I'd hire a candidate that proposed this as a solution in earnest, but I'd have fun chatting with them :) -tkc -- http://mail.python.org/mailman/listinfo/python-list
Re: python interview quuestions
On Fri, Aug 6, 2010 at 6:25 PM, Steven D'Aprano
wrote:
> On Sat, 07 Aug 2010 06:37:05 +1000, James Mills wrote:
>
>> On Sat, Aug 7, 2010 at 6:28 AM, geremy condra
>> wrote:
>>> If I had to wait 5 minutes while a candidate tried to solve this
>>> problem I would not hire them.
>>
>> Yes you do raise a valid point. It should really only take you a mere
>> few seconds or so to write a solution to this.
>
> Yes, but the point is to come up with a solution that is *correct*, not
> merely a solution. *wink*
>
> Just the mere typing time will be "a few seconds or so", so you're
> leaving zero time for actual thought, let alone running the code at least
> once to test it.
I wouldn't let them have access to an interpreter for this either. The
goal is to see if they can put out something that looks reasonably
close to a solution; if so, they're probably trainable. Otherwise,
GTFO.
> Personally, I'd rather see how a potential hire *tests* his code than how
> he writes it. Writing code is easy. Testing code is harder. Testing it
> properly is harder still -- it's amazing how many people forget that it's
> not just necessary to test the function on data that *works*, but also on
> data that fails as well (unless, of course, you're happy with function
> behaviour that is unspecified in the face of errors).
Absolutely. A great problem for this is the ACM programming challenge
pig latin translator- it contains a logic error that makes a certain
case ambiguous. Hire the ones that spot the problem before they touch
the keyboard on the spot, and put the ones that test for it at the top
of the heap.
> I also want to see when the coder thinks she's done. If I say "Write a
> function that does fizzbuzz", does she assume I want *just* the function,
> or does she ask questions like "Do you want documentation and tests? What
> sort of tests?". Does she assume that because the fizzbuzz function is
> small it doesn't need documentation or testing? The Fizzbuzz algorithm
> itself is the simple part. If I'm hiring a coder, I care more about their
> attitude to documentation and testing than their ability to code up a
> tiny algorithm in five seconds time. I want to see if they are a careful
> coder, or a cowboy.
I have things I'd rather do than sit around and watch a candidate
write tests. If I wanted them to write tests, I'd send them the
problem before the interview and have them sell me on their way of
solving it during the interview.
>
>> More over, it can be done in just a single line of Python.
>>
>> 7 if you're not very familiar with Python.
>
> Or if you value readability over conserving newlines.
I was thinking the same thing.
> Some months ago I wrote a version of FizzBuzz. By the time I read the
> description of the problem, re-read it to make sure I understood it and
> looking for any hidden traps ("seems too simple to me, what have I
> missed?"), wrote a first draft, tested it, fixed a silly typo, and then
> tested it again, it took about 3-5 minutes. I don't believe I have
> anything to be ashamed about that, especially not when I look at the
> number of comments by people who claimed the exercise was so trivial that
> they did it in ten seconds, AND GOT IT WRONG.
Given the expectation of interpreter-ready code rather than just
logically correct and close enough, 3 minutes isn't insane. 5 minutes
is still pretty out there, though.
Of course, getting it wrong doesn't win anybody points.
> Then I added error checking, tests and documentation, and the whole
> exercise took about 20 minutes. That was probably overkill, but I was
> bored :)
Just maybe ;)
Geremy Condra
--
http://mail.python.org/mailman/listinfo/python-list
Re: Is there any way to minimize str()/unicode() objects memory usage [Python 2.6.4] ?
dmtr: > What I'm really looking for is a dict() that maps short unicode > strings into tuples with integers. But just having a *compact* list > container for unicode strings would help a lot (because I could add a > __dict__ and go from it). Add them all into one string or array and use indexes into that string. Neil -- http://mail.python.org/mailman/listinfo/python-list
Re: [Tutor] Finding the version # of a module, and py module problem
On Aug 6, 2010, at 3:14 PM, W. eWatson wrote:
I must be missing something. I tried this. (Windows, IDLE, Python 2.5)
Yes, as Benjamin Kaplan pointed out and as I said in the email where I
posted this code snippet, "dependencies is a list of custom classes
that represent modules we need (e.g. numpy)." The code I posted was
not meant to be a complete working example. It's part of a larger
piece of code that I didn't have time to cook down to a simpler, self-
sufficient whole.
Also, in your list you've got "numyp" instead of "numpy".
Also, at the top of your code you have "import numpy" and "import
scipy" which defeats the purpose of this code.
Try this (untested):
import sys
dependencies = ("numpy", "scipy", "some_other_module")
for dependency in dependencies:
try:
__import__(dependency)
except ImportError:
# Uh oh!
print "%s is not installed" % dependency
else:
# The module loaded OK. Get a handle to it and try to extract
# version info.
# Many Python modules follow the convention of providing their
# version as a string in a __version__ attribute.
module = sys.modules[dependency]
for attribute_name in ("__version__", "__VERSION__", "VERSION",
"version"):
if hasattr(module, attribute_name):
version = getattr(module, attribute_name)
print "module %s has version %s" % (dependency, version)
break
bye
Philip
# Try each module
import sys
import numpy
import scipy
import string
dependencies = "numyp", "scipy"
for dependency in dependencies:
try:
__import__(dependency.name)
except ImportError:
# Uh oh!
dependency.installed = None
else:
# The module loaded OK. Get a handle to it and try to extract
# version info.
# Many Python modules follow the convention of providing their
# version as a string in a __version__ attribute.
module = sys.modules[dependency.name]
# This is what I default to.
dependency.installed = "[version unknown]"
for attribute_name in ("__version__", "__VERSION__", "VERSION",
"version"):
if hasattr(module, attribute_name):
dependency.installed = getattr(module, attribute_name)
break
The result was this.
Traceback (most recent call last):
File "C:/Users/Wayne/Sandia_Meteors/Trajectory_Estimation/
dependency_code", line 10, in
__import__(dependency.name)
AttributeError: 'str' object has no attribute 'name'
--
http://mail.python.org/mailman/listinfo/python-list
--
http://mail.python.org/mailman/listinfo/python-list
Re: Why is python not written in C++ ?
In message , David Cournapeau wrote: > I have yet seen a project where you could build C code with a C++ > compiler - the only ones I know are specifically designed that way and > it is painful. I seem to recall a FAQ entry, might have been on kernelnewbies.org, asking why the Linux kernel wasn’t written in C++. The answer explained that at one time there was an experiment to make the kernel compilable with a C++ compiler, without actually using any C++ features. The result: they lost about 10% in speed. That was enough to put the kernel developers off taking the experiment any further. -- http://mail.python.org/mailman/listinfo/python-list
Re: Is there any way to minimize str()/unicode() objects memory usage [Python 2.6.4] ?
> I'm running into some performance / memory bottlenecks on large lists.
> Is there any easy way to minimize/optimize memory usage?
>
> Simple str() and unicode objects() [Python 2.6.4/Linux/x86]:
sys.getsizeof('') 24 bytes
sys.getsizeof('0')25 bytes
sys.getsizeof(u'')28 bytes
sys.getsizeof(u'0') 32 bytes
A Python str object contains much more than just the raw string. On a
32bit system it contains:
* a pointer to its type (ptr with 4 bytes)
* a reference counter (ssize_t, 4 bytes)
* the length of the string (ssize_t, 4 bytes)
* the cached hash of the string (long, 8 bytes)
* interning state (int, 4 bytes)
* a null terminated char array for its data.
Christian
--
http://mail.python.org/mailman/listinfo/python-list
Re: python interview quuestions
Steven D'Aprano wrote: > Personally, I'd rather see how a potential hire *tests* his code than how > he writes it. Writing code is easy. Testing code is harder. Testing it > properly is harder still -- it's amazing how many people forget that it's > not just necessary to test the function on data that *works*, but also on > data that fails as well (unless, of course, you're happy with function > behaviour that is unspecified in the face of errors). > > I also want to see when the coder thinks she's done. Perhaps I'm reading more into your choice of words than you intended, but I'm curious what you envision a "coder" does. I think of "coder" and a rather low-level job. Somebody who just writes code. I'm generally looking for somebody who is more of a software engineer. Somebody who is not just writing some code, but who is building a product. That means, as you suggest, that it's documented, tested, robust, maintainable, portable, all that good stuff. > If I say "Write a function that does fizzbuzz", does she assume I > want *just* the function, or does she ask questions like "Do you want > documentation and tests? What sort of tests?". I think this depends on the situation. For writing code at a whiteboard while i watched, I'd expect the candidate to concentrate just on the code itself. If it was an assigment, as in, "Write a program to do X, and mail it to me by tomorrow", I'd expect a much more complete treatment. -- http://mail.python.org/mailman/listinfo/python-list
Re: Is there any way to minimize str()/unicode() objects memory usage [Python 2.6.4] ?
On Fri, Aug 6, 2010 at 6:39 PM, dmtr wrote: >> > Well... 63 bytes per item for very short unicode strings... Is there >> > any way to do better than that? Perhaps some compact unicode objects? >> >> If you think that unicode objects are going to be *smaller* than byte >> strings, I think you're badly informed about the nature of unicode. > > I don't think that that unicode objects are going to be *smaller*! > But AFAIK internally CPython uses UTF-8? Nope. unicode objects internally use UCS-2 or UCS-4, depending on how CPython was ./configure-d; the former is the default. See PEP 261. Cheers, Chris -- http://blog.rebertia.com -- http://mail.python.org/mailman/listinfo/python-list
Re: Is there any way to minimize str()/unicode() objects memory usage [Python 2.6.4] ?
> > Well... 63 bytes per item for very short unicode strings... Is there > > any way to do better than that? Perhaps some compact unicode objects? > > There is a certain price you pay for having full-feature Python objects. Are there any *compact* Python objects? Optimized for compactness? > What are you trying to accomplish anyway? Maybe the array module can be > of some help. Or numpy? Ultimately a dict that can store ~20,000,000 entries: (u'short string' : (int, int, int, int, int, int, int)). -- Regards, Dmitry -- http://mail.python.org/mailman/listinfo/python-list
Re: Is there any way to minimize str()/unicode() objects memory usage [Python 2.6.4] ?
Steven, thank you for answering. See my comments inline. Perhaps I should have formulated my question a bit differently: Are there any *compact* high performance containers for unicode()/str() objects in Python? By *compact* I don't mean compression. Just optimized for memory usage, rather than performance. What I'm really looking for is a dict() that maps short unicode strings into tuples with integers. But just having a *compact* list container for unicode strings would help a lot (because I could add a __dict__ and go from it). > Yes, lots of ways. For example, do you *need* large lists? Often a better > design is to use generators and iterators to lazily generate data when > you need it, rather than creating a large list all at once. Yes. I do need to be able to process large data sets. No, there is no way I can use an iterator or lazily generate data when I need it. > An optimization that sometimes may help is to intern strings, so that > there's only a single copy of common strings rather than multiple copies > of the same one. Unfortunately strings are unique (think usernames on facebook or wikipedia). And I can't afford storing them in db/memcached/redis/ etc... Too slow. > Can you compress the data and use that? Without knowing what you are > trying to do, and why, it's really difficult to advise a better way to do > it (other than vague suggestions like "use generators instead of lists"). Yes. I've tried. But I was unable to find a good, unobtrusive way to do that. Every attempt either adds some unnecessary pesky code, or slow, or something like that. See more at: http://bugs.python.org/issue9520 > Very often, it is cheaper and faster to just put more memory in the > machine than to try optimizing memory use. Memory is cheap, your time and > effort is not. Well... I'd really prefer to use say 16 bytes for 10 chars strings and fit data into 8Gb Rather than paying extra $1k for 32Gb. > > Well... 63 bytes per item for very short unicode strings... Is there > > any way to do better than that? Perhaps some compact unicode objects? > > If you think that unicode objects are going to be *smaller* than byte > strings, I think you're badly informed about the nature of unicode. I don't think that that unicode objects are going to be *smaller*! But AFAIK internally CPython uses UTF-8? No? And 63 bytes per item seems a bit excessive. My question was - is there any way to do better than that > Python is not a low-level language, and it trades off memory compactness > for ease of use. Python strings are high-level rich objects, not merely a > contiguous series of bytes. If all else fails, you might have to use > something like the array module, or even implement your own data type in > C. Are there any *compact* high performance containers (with dict, list interface) in Python? -- Regards, Dmitry -- http://mail.python.org/mailman/listinfo/python-list
Re: easy question on parsing python: "is not None"
Steven D'Aprano wrote: Generally, when testing for None, you actually want None and not some look-alike that merely tests equal to None. That's true, although I can't think of a use case for an object that compares equal to None but isn't -- except for obfuscated code competition entries and making obscure points in usenet discussions. :-) -- Greg -- http://mail.python.org/mailman/listinfo/python-list
Re: easy question on parsing python: "is not None"
Ethan Furman wrote: Instead of using 'is' use '=='. Maybe not as cute, but definitely more robust! It's also just as efficient if you use strings that resemble identifiers, because they will be interned, so the comparison will end up just doing an indentity test anyway. -- Greg -- http://mail.python.org/mailman/listinfo/python-list
Re: python interview quuestions
On Sat, 07 Aug 2010 06:37:05 +1000, James Mills wrote:
> On Sat, Aug 7, 2010 at 6:28 AM, geremy condra
> wrote:
>> If I had to wait 5 minutes while a candidate tried to solve this
>> problem I would not hire them.
>
> Yes you do raise a valid point. It should really only take you a mere
> few seconds or so to write a solution to this.
Yes, but the point is to come up with a solution that is *correct*, not
merely a solution. *wink*
Just the mere typing time will be "a few seconds or so", so you're
leaving zero time for actual thought, let alone running the code at least
once to test it.
Personally, I'd rather see how a potential hire *tests* his code than how
he writes it. Writing code is easy. Testing code is harder. Testing it
properly is harder still -- it's amazing how many people forget that it's
not just necessary to test the function on data that *works*, but also on
data that fails as well (unless, of course, you're happy with function
behaviour that is unspecified in the face of errors).
I also want to see when the coder thinks she's done. If I say "Write a
function that does fizzbuzz", does she assume I want *just* the function,
or does she ask questions like "Do you want documentation and tests? What
sort of tests?". Does she assume that because the fizzbuzz function is
small it doesn't need documentation or testing? The Fizzbuzz algorithm
itself is the simple part. If I'm hiring a coder, I care more about their
attitude to documentation and testing than their ability to code up a
tiny algorithm in five seconds time. I want to see if they are a careful
coder, or a cowboy.
> More over, it can be done in just a single line of Python.
>
> 7 if you're not very familiar with Python.
Or if you value readability over conserving newlines.
Some months ago I wrote a version of FizzBuzz. By the time I read the
description of the problem, re-read it to make sure I understood it and
looking for any hidden traps ("seems too simple to me, what have I
missed?"), wrote a first draft, tested it, fixed a silly typo, and then
tested it again, it took about 3-5 minutes. I don't believe I have
anything to be ashamed about that, especially not when I look at the
number of comments by people who claimed the exercise was so trivial that
they did it in ten seconds, AND GOT IT WRONG.
Then I added error checking, tests and documentation, and the whole
exercise took about 20 minutes. That was probably overkill, but I was
bored :)
(At least I didn't turn it into a class.)
--
Steven
--
http://mail.python.org/mailman/listinfo/python-list
Re: Is there any way to minimize str()/unicode() objects memory usage [Python 2.6.4] ?
On 08/07/2010 02:45 AM, dmtr wrote:
> I'm running into some performance / memory bottlenecks on large lists.
> Is there any easy way to minimize/optimize memory usage?
>
> Simple str() and unicode objects() [Python 2.6.4/Linux/x86]:
sys.getsizeof('') 24 bytes
sys.getsizeof('0')25 bytes
sys.getsizeof(u'')28 bytes
sys.getsizeof(u'0') 32 bytes
>
> Lists of str() and unicode() objects (see ref. code below):
[str(i) for i in xrange(0, 1000)] 370 Mb (37 bytes/item)
[unicode(i) for i in xrange(0, 1000)] 613 Mb (63 bytes/item)
>
> Well... 63 bytes per item for very short unicode strings... Is there
> any way to do better than that? Perhaps some compact unicode objects?
There is a certain price you pay for having full-feature Python objects.
What are you trying to accomplish anyway? Maybe the array module can be
of some help. Or numpy?
>
> -- Regards, Dmitry
>
>
> import os, time, re
> start = time.time()
> l = [unicode(i) for i in xrange(0, 1000)]
> dt = time.time() - start
> vm = re.findall("(VmPeak.*|VmSize.*)", open('/proc/%d/status' %
> os.getpid()).read())
> print "%d keys, %s, %f seconds, %f keys per second" % (len(l), vm, dt,
> len(l) / dt)
--
http://mail.python.org/mailman/listinfo/python-list
Re: Is there any way to minimize str()/unicode() objects memory usage [Python 2.6.4] ?
On Fri, 06 Aug 2010 17:45:31 -0700, dmtr wrote: > I'm running into some performance / memory bottlenecks on large lists. > Is there any easy way to minimize/optimize memory usage? Yes, lots of ways. For example, do you *need* large lists? Often a better design is to use generators and iterators to lazily generate data when you need it, rather than creating a large list all at once. An optimization that sometimes may help is to intern strings, so that there's only a single copy of common strings rather than multiple copies of the same one. Can you compress the data and use that? Without knowing what you are trying to do, and why, it's really difficult to advise a better way to do it (other than vague suggestions like "use generators instead of lists"). Very often, it is cheaper and faster to just put more memory in the machine than to try optimizing memory use. Memory is cheap, your time and effort is not. [...] > Well... 63 bytes per item for very short unicode strings... Is there > any way to do better than that? Perhaps some compact unicode objects? If you think that unicode objects are going to be *smaller* than byte strings, I think you're badly informed about the nature of unicode. Python is not a low-level language, and it trades off memory compactness for ease of use. Python strings are high-level rich objects, not merely a contiguous series of bytes. If all else fails, you might have to use something like the array module, or even implement your own data type in C. But as a general rule, as I mentioned above, the best way to minimize the memory used by a large list is to not use a large list. I can't emphasise that enough -- look into generators and iterators, and lazily handle your data whenever possible. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Is there any way to minimize str()/unicode() objects memory usage [Python 2.6.4] ?
I'm running into some performance / memory bottlenecks on large lists.
Is there any easy way to minimize/optimize memory usage?
Simple str() and unicode objects() [Python 2.6.4/Linux/x86]:
>>> sys.getsizeof('') 24 bytes
>>> sys.getsizeof('0')25 bytes
>>> sys.getsizeof(u'')28 bytes
>>> sys.getsizeof(u'0') 32 bytes
Lists of str() and unicode() objects (see ref. code below):
>>> [str(i) for i in xrange(0, 1000)] 370 Mb (37 bytes/item)
>>> [unicode(i) for i in xrange(0, 1000)] 613 Mb (63 bytes/item)
Well... 63 bytes per item for very short unicode strings... Is there
any way to do better than that? Perhaps some compact unicode objects?
-- Regards, Dmitry
import os, time, re
start = time.time()
l = [unicode(i) for i in xrange(0, 1000)]
dt = time.time() - start
vm = re.findall("(VmPeak.*|VmSize.*)", open('/proc/%d/status' %
os.getpid()).read())
print "%d keys, %s, %f seconds, %f keys per second" % (len(l), vm, dt,
len(l) / dt)
--
http://mail.python.org/mailman/listinfo/python-list
Microsoft lessening commitment to IronPython and IronRuby
There is a blog post from Jimmy Schementi who previously worked at Microsoft on IronRuby about the state of dynamic language work there. http://blog.jimmy.schementi.com/2010/08/start-spreading-news-future-of-jimmy.html Neil -- http://mail.python.org/mailman/listinfo/python-list
How to implement a pipeline...??? Please help
Hi guys
In the development of my ADC project i have a new new challenge.
First i need to implement a 10 Bit pipelineADC that will be the
basis to later implement any kind of pipeline arquitecture (i mean,
with 10 Bit, 16 Bit or any other configuration i want) i wish to...
What's a 10 Bit pipiline ADC?
This link with a draw will help have a perfect view of what is it:
http://t3.gstatic.com/images?q=tbn:ANd9GcQK_Pwz3ssk1yoDDFwGjJ5FWAwcqr1mV9EwdndlHCEvOMdTOa4&t=1&usg=__-TsH7dnNdJm4GZTuWCxjvajZhfk=
To have a 10 Bit pipiline ADC we need 9 stages, where 8 of them are a
1.5 bit configuration and the last one have a 2 Bit configuration.
How it works?
When we inject a input voltage (at the first block), it will be
computed in the first 1.5 Bit stage and when finished the compute it
will generate a output voltage who served of input voltage for the
second 1.5 Bit stage which in his turn generate also a output voltage
who served of input voltage in the third stage ... and successively
continuing until the ninth stage. The ninth stage it's a 2 Bit stage
and it will receive the last output voltage from the eighth stage and
in his turn it will compute a final output signal.
I already implemented the 1.5 Bit/stage and the 2 Bit/stage functions
with good results (i already watch their plot's and they are cool to
me) but my Pipeline function try don't seems that good.
After the 2 Bit compute i expect a output signal like the drawing on
the link below but more stretched.
http://t3.gstatic.com/images?q=tbn:ANd9GcQGQYDW0iLCSXfMMurIksWsgklsvtj26IBtQRacFtY7ifu9RQA&t=1&usg=__-yHJFMatnCPcf9jWQv3kaxM0brE=
Instead of have a lot of steps one after the another, it will have
something like a single step only. Why? That it's because when we have
a pipeline with a lot of stage's the final output voltage until the 2
Bit stage it's very "dense" and long and that nearly saturating the
last block ( the 2 Bit stage).
Next i will leave the code that I have done so far for my Pipeline
function:
---
---
from flash1b5 import flash1b5
from flash1b5 import *
from flash2b import flash2b
from flash2b import *
import matplotlib.pyplot as plt
import numpy as np
from pylab import *
if __name__ == "__main__":
Inc = raw_input("Valor do Incrimento = ")
Vref = np.linspace(1,1, Inc)
Vi = np.linspace(-1,1, Inc)
Cs = np.linspace(3e-12, 3e-12, Inc)
Cf = np.linspace(3e-12, 3e-12, Inc)
f1 = flash1b5(Vi, Vref, Cs, Cf)
Vin1 = f1[0]
Vt1 = f1[1]
Vd1 = f1[2]
f2b = flash2b(Vt1, Vref)
Vin2 = f2b[0]
Vd2 = f2b[1]
hold(True)
fig2 = figure(1,figsize=(8,5))
ax2 = fig2.add_subplot(111, autoscale_on=False, xlim=(-1,1),
ylim=(-0.5,2.5))
ax2.plot(Vin2, Vd2, lw=2, color='blue')
grid (True); xlabel('V_ent');ylabel('Vout_Digital')
plt.show()
---
The 1.5 Bit function:
import numpy as np
from pylab import *
def flash1b5(Vi, Vref, Cs, Cf):
Vin = np.zeros(Vi.shape, dtype=np.float)
Vt = np.zeros(Vi.shape, dtype=np.float)
Vd = np.zeros(Vi.shape, dtype=np.float)
if any(Vi > Vref/4):
mask1 = (Vi > Vref/4)
np.putmask(Vin, mask1, Vi)
Vs= (1+Cs/Cf)*Vin - (Cs/Cf)*Vref
np.putmask(Vd, mask1, [2])
np.putmask(Vt, mask1, Vs)
##
if any(-Vref/4 <= Vi) and any( Vi<= Vref/4):
mask2 = (-Vref/4 <= Vi) & (Vi <= Vref/4)
np.putmask(Vin, mask2, Vi)
Vs = (1+(Cs/Cf))*Vin
np.putmask(Vd, mask2, [1])
np.putmask(Vt, mask2, Vs)
##
if any(Vi < -Vref/4):
mask3 = (Vi < -Vref/4)
np.putmask(Vin, mask3, Vi)
Vs= (1+Cs/Cf)*Vin + (Cs/Cf)*Vref
np.putmask(Vd, mask3, [0])
np.putmask(Vt, mask3, Vs)
##
out = [Vin, Vt, Vd]
return out
---
The 2 Bit function:
import numpy as np
from pylab import *
def flash2b(Vi, Vref):
Vin = np.zeros(Vi.shape, dtype=np.float)
Vt = np.zeros(Vi.shape, dtype=np.float)
Vd = np.zeros(Vi.shape, dtype=np.float)
if any(Vi > Vref/2):
mask1 = (Vi > Vref/2)
np.putmask(Vin, mask1, Vi)
np.putmask(Vd, mask1, [3])
if any(Vi > 0) and any(Vref/2 >= Vi):
mask2 = (Vref/2 >= Vi) & (Vi > 0)
np.putmask(Vin, mask2, Vi)
np.putmask(Vd, mask2, [2])
if any(Vi <= 0) and any(-Vref/2 < Vi):
mask3 = (-Vref/2 < Vi) & (Vi <= 0)
np.putmask(Vin, mask
Re: default behavior
On 8/6/2010 6:24 PM, Wolfram Hinderer wrote: This is probably nitpicking, but the patch calls __missing__ a special method. However, unlike special methods, it is not invoked by "special syntax" but by the dict's __getitem__ method. (len() invokes __len__ on any object - you can't do something similar with __missing__.) __missing__ is also not listed as a special method on http://docs.python.org/py3k/reference/datamodel.html#special-method-names However, "normal" special method lookup seems to be used. Fair enough. Please add your comment to #9536 at bugs.python.org. Tx, John -- http://mail.python.org/mailman/listinfo/python-list
Re: Python Portability--Not very portable?
On Fri, 06 Aug 2010 10:58:24 -0700, W. eWatson wrote: > Is there a complete illustration of using disutils? Our only > dependencies are on Python Org material. We use no commercial or > licensed code. Oh my, the sheer ignorance that sentence reveals. Python and the standard library *is* licensed. I mean, good grief, when you start up Python's interpreter, it even says: Type "help", "copyright", "credits" or "license" for more information. Similarly any C compiler you use will also be licensed. If it's not, you have no right to use it! (Unless you can find a public domain compiler, which I doubt exists, and even if one does, I doubt it will be very good.) -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Python Portability--Not very portable?
On Thu, 05 Aug 2010 18:50:14 -0700, W. eWatson wrote: > As an example, my inexperienced Python partner 30 miles away has gotten > out of step somehow. I think by installing a different version of numpy > than I use. I gave him a program we both use months ago, and he had no > trouble. (We both use IDLE on 2.5). I made a one character change to it > and sent him the new py file. He can't execute it. I doubt he has > changed anything in the intervening period. What does that mean? It crashes? It raises an exception? He double-clicks on it and nothing happens? What error does he get? What was the one character change? Have you tested it and are you sure it works, or are you just assuming it works? What OS are you using? How did you send it to him? Can he open the file in a text editor, and what does he see? > A further example. Months ago I decided to see if I could compile a > program to avoid such problems as above. I planned to satisfy that need, > and see if I could distribute some simple programs to non-Python > friends. I pretty well understand the idea,and got it working with a > small program. It seemed like a lot of manual labor to do it. I'm sorry, I don't fully understand what you think this is going to accomplish. A program compiled to object code is going to be dependent on the hardware platform, the operating system, and any libraries that may or may not be installed. Most likely the *exact* version of the libraries. Do the words "DLL Hell" mean anything to you? How is that an advantage of Python's source-code distribution? -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: easy question on parsing python: "is not None"
On Fri, 06 Aug 2010 11:42:39 +0200, Jean-Michel Pichavant wrote: > Steven D'Aprano wrote: >> On Thu, 05 Aug 2010 12:07:53 -0400, wheres pythonmonks wrote: >> >>> P.S. Sorry for the top-post -- is there a way to not do top posts from >>> gmail? I haven't used usenet since tin. >>> >> Er, surely you can just move the cursor before you start typing??? >> > CTRL+END will bring the cursor at the end of the file (for most the the > mail clients). Bottom-posting is just as annoying as top-posting, except for very short posts like this. Posting in-line, where you trim the excess quoting to leave enough to give context, is most recommended. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Python Portability--Not very portable?
On Fri, 06 Aug 2010 10:35:38 -0700, W. eWatson wrote: > So you think Python is part of open software in terms of distributing a > "product"? Python itself *is* open source software. It doesn't *require* you to write open source software. > So I should stick to C, where one can distribute programs w/o > revealing code details No you can't. That's the point that you are missing. When you ship a compiled exe, you are still shipping code, and anyone serious about "ripping off" your ideas will not find that even the tiniest barrier to doing so. Microsoft never ship their code to Windows, and yet virus and malware authors regularly and rapidly analyse it well enough to not only work out what it does and how it does it, but to discover vulnerabilities that even Microsoft don't know about. If MS can't protect their code, what makes you think your .exe file is going to any better protected? -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Unicode error
On Fri, 06 Aug 2010 11:23:50 +, kj wrote: > I don't get your point. Even when I *know* that a certain exception may > happen, I don't necessarily catch it. I catch only those exceptions for > which I can think of a suitable response that is *different* from just > letting the program fail. (After all, my own code raises its own > exceptions with the precise intention of making the program fail.) If > an unexpected exception occurs, then by definition, I had no better > response in mind for that situation than just letting the program fail, > so I'm happy to let that happen. If, afterwards, I think of a different > response for a previously uncaught exception, I'll modify the code > accordingly. > > I find this approach far preferable to the alternative of knowing a long > list of possible exceptions (some of which may never happen in actual > practice), and think of ways to keep the program still alive > no-matter-what. "No memory? No disk space? No problem! Just a flesh > wound!" What's the point of that? /me cheers wildly! Well said! -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: easy question on parsing python: "is not None"
On Fri, 06 Aug 2010 15:37:04 +0200, Stefan Schwarzer wrote: >> Plus, I believe the >> "==" operator will check if the variables point to the same object. > > No, that's what `is` is for. Actually, yes, equality is implemented with a short-cut that checks for identity first. That makes something like: s = "abc"*1000*1000*10 s == s nice and quick, as Python can immediately recognise that a string is always equal to itself without having to walk the entire string comparing each character with itself. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Python Portability--Not very portable?
On Fri, 06 Aug 2010 08:00:55 -0700, W. eWatson wrote: >>> I would think there are some small time and big time Python players >>> who sell executable versions of their programs for profit? >> >> Yes. What's your point? > That someone must know how to distribute them without having the source > code ripped off. That's what copyright law is for. If you think that distributing object code (instead of source code) is going to deter a serious hacker, you're deluded. Besides, I don't mean to be rude, but what makes you think anyone would care about stealing your code? If you think people are interested in stealing your code, you're almost certainly wrong. The world is full of coders who think their two-bit text editor or re-implementation of PacMan is the most precious, precious software in existence. The vast majority of them are wrong. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: default behavior
On 6 Aug., 22:07, John Posner wrote: > On 8/2/2010 11:00 PM, John Posner wrote: > > > On 7/31/2010 1:31 PM, John Posner wrote: > > >> Caveat -- there's another description of defaultdict here: > > >>http://docs.python.org/library/collections.html#collections.defaultdict > > >> ... and it's bogus. This other description claims that __missing__ is a > >> method of defaultdict, not of dict. > > > Following is a possible replacement for the bogus description. Comments > > welcome. I intend to submit a Python doc bug, and I'd like to have a > > clean alternative to propose. > > After some off-list discussion with Ethan Furman (many thanks!), the > Python Doc bug is submitted: #9536 at bugs.python.org. > > -John This is probably nitpicking, but the patch calls __missing__ a special method. However, unlike special methods, it is not invoked by "special syntax" but by the dict's __getitem__ method. (len() invokes __len__ on any object - you can't do something similar with __missing__.) __missing__ is also not listed as a special method on http://docs.python.org/py3k/reference/datamodel.html#special-method-names However, "normal" special method lookup seems to be used. -- http://mail.python.org/mailman/listinfo/python-list
Re: easy question on parsing python: "is not None"
On Fri, 06 Aug 2010 05:28:40 -0700, DG wrote: > I've always thought of it as you don't compare strings with "is", you > *should* use == The reasoning is that you don't know if that string > instance is the only one in memory. This is excellent advice. I won't say that there is "never" a use-case for testing strings for identity, but I can't think of one that isn't contrived and artificial. > I've heard as an implementation > detail, since strings are immutable, that Python will only create one > actual instance and just assign new references to it (your first x is y > example), but to compare equality it just makes sense to use "==", not > to ask if it is the same object in memory. Plus, I believe the "==" > operator will check if the variables point to the same object. Yes, that is an optimization that CPython performs. Other implementations may do different, but the optimization is so obvious and so cheap that I would be shocked if any of the major implementations fail to do so. > Using is/is not with None works well, because I believe there will > always only be 1 None object. Yes, that is correct, but there's also a semantic difference. Generally, when testing for None, you actually want None and not some look-alike that merely tests equal to None. You want to "accept no substitutes", and so an identity test is necessary. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: easy question on parsing python: "is not None"
On Fri, 06 Aug 2010 17:20:30 +, Peter Pearson wrote: > On Fri, 06 Aug 2010 15:37:04 +0200, Stefan Schwarzer wrote: [snip] >> I can imagine a case where you might want to compare a string with >> `is`: >> >> FORWARD = "forward" >> BACKWARD = "backward" [...] >> Actually, I've never seen such a use, as far as I remember. What do >> other people here think? Is the code above, which compares strings with >> `is`, bad style, and if yes, why? How would you write the code instead? > > Hey, that's a cute example, but . . . what a trap! Is it possible to > document the use-the-object-not-the-string requirement loudly enough > that people won't get caught? Nope. People don't read documentation :) That's why I prefer to test for human-readable constants using equality, so if the caller prefers to call func(x, "forward") instead of func(x, FORWARD) it will still work. Or I use FOWARD = object() BACKWARDS = object() and force them to use my constants. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: new to python - trouble calling a function from another function
On Thu, Aug 5, 2010 at 8:15 AM, Brandon McCombs wrote: > Jon Clements wrote: >> >> On 5 Aug, 08:25, Brandon McCombs wrote: >>> >>> Hello, >>> >>> I'm building an elevator simulator for a class assignment. I recently >>> ran into a roadblock and don't know how to fix it. For some reason, in >>> my checkQueue function below, the call to self.goUp() is never executed. >>> It is on the last line of code I pasted in. I can put print statements >>> before and after the call and I have a print statement in goUp() itself. >>> Only the print statements before and after the call are executed. The >>> one inside goUp() is never executed because goUp() never seems to be >>> executed. How can that be? I don't get any errors when the script >>> executes. Surely this isn't some limitation I'm encountering? >>> >>> thanks >>> >>> sorry about the formatting >>> >>> - >>> class Elevator(Process): >>> def __init__(self,name): >>> Process.__init__(self,name=name) >>> self.numPassengers = 0 >>> self.passengerList = [] >>> self.passengerWaitQ = [] >>> self.currentFloor = 1 >>> self.idle = 1 >>> self.newPassengers = 0 >>> def goUp(self): >>> print "here" >>> bubbleSort(self.passengerList, len(self.passengerList)) >>> self.currentFloor += 1 >>> if len(self.passengerList) > 0: >>> for p in self.passengerList: >>> if self.currentFloor == p.destination: >>> yield (p.destination - self.currenteFloor) * TRAVELTIME, >>> self >>> reactivate(p) >>> p.inBuilding() >>> else: >>> self.goUp() >>> >>> def checkQueue(self): >>> if (len(self.passengerWaitQ)) > 0 and len(self.passengerList) < >>> MAXCAPACITY: >>> if len(self.passengerWaitQ) < MAXCAPACITY: >>> self.newPassengers = len(self.passengerWaitQ) >>> else: >>> self.newPassengers = MAXCAPACITY - len(self.passengerList) >>> for i in range(0,self.newPassengers): >>> self.passengerList.append(self.passengerWaitQ.pop()) >>> self.goUp() >> >> Hi Brandon, >> >> Nice one at having a good crack at coding before posting! >> >> From your posted code, I'm struggling to see what's trying to be >> taught to you for this class assignment. > > not relevant at this point > >> >> As a note it'll be worth reading PEP 8 regarding naming conventions, >> because it looks very Java-ish to me! > > ok but not relevant > >> >> (I might be taking too much a real-world approach in the following, >> but do with it as you will...) >> >> I'm assuming that MAXCAPACITY and TRAVELTIME are globals somewhere. >> Although what I'm thinking is that different Elevators will have >> different capacities and different floors they service. An Elevator is >> *not* going to know its number of passengers (the most it could do is >> capacity based on weight restrictions) therefore it's not going to >> know the number of new passengers on each floor either. > > okay but not relevant to the problem at hand > >> >> A couple of things that'd be worthwhile: >> >> 1) Post the requirements for your assignment - what's it supposed to >> output etc... > > that isn't relevant for determining at the python level why a function > simply isn't being called > >> 2) Go find an elevator, take a pen and pad with you, and stand in it >> for 30 mins or so, and see how the real thing deals with situations >> and make notes. ie, does it queue requests, or decide to take the next >> nearest floor, when does it stop and open etc...? >> >> hth >> >> Jon. >> > > actually it doesn't help at all since you decided to focus on everything but > my actual question of why a function call wasn't working but rather question > the validity of the program itself This is too bad. You looked like you were all set to be the poster child for how to ask a good question here, and now you've blown it. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
HL7 v3 (XML) importer
an HL7 v2 importer was written by john paulett, and it has been enhanced to support some of the HL7 v3 standard, which is XML-based. no dependencies are required: xml.sax is used so as to reduce the dependencies to purely python. additionally, as HL7 has versions/revisions, published data specifications were pulled from mirthcorp.com and transformed into python, automatically. HL7v3 revisions 2.1, 2.2, 2.3, 2.31, 2.4 and 2.5 are supported. there is still work to be carried out: MSH, PID, NTE, ORC, ORB and OBX are the presently supported segmentss, with a hard-coded assumption about the data hierarchy instead of using the same MessageFormat specifications that mirthcorp.com have written. this is likely to be corrected soon, but the project for which the code is being updated only requires "lab data imports", so support for other MessageFormats is not a high priority. in case anyone is interested, this hl7 library is to be used in gnumed, to be able to import laboratory data reports into the gnumed database. code is at: http://github.com/lkcl/hl7 i realise that this code is pretty specialist, but if you're looking around for free software libraries or applications that support HL7, http://mirthcorp.com is pretty much it, and mirth is pretty heavy-duty and forces a dependency of java and javascript for scripting, which may be of concern to some. hence, this smaller library is being created, in python. l. -- http://mail.python.org/mailman/listinfo/python-list
Re: new to python - trouble calling a function from another function
On 5 Aug, 16:15, Brandon McCombs wrote: > Jon Clements wrote: > > On 5 Aug, 08:25, Brandon McCombs wrote: > >> Hello, > > >> I'm building an elevator simulator for a class assignment. I recently > >> ran into a roadblock and don't know how to fix it. For some reason, in > >> my checkQueue function below, the call to self.goUp() is never executed. > >> It is on the last line of code I pasted in. I can put print statements > >> before and after the call and I have a print statement in goUp() itself. > >> Only the print statements before and after the call are executed. The > >> one inside goUp() is never executed because goUp() never seems to be > >> executed. How can that be? I don't get any errors when the script > >> executes. Surely this isn't some limitation I'm encountering? > > >> thanks > > >> sorry about the formatting > > >> - > >> class Elevator(Process): > >> def __init__(self,name): > >> Process.__init__(self,name=name) > >> self.numPassengers = 0 > >> self.passengerList = [] > >> self.passengerWaitQ = [] > >> self.currentFloor = 1 > >> self.idle = 1 > >> self.newPassengers = 0 > >> def goUp(self): > >> print "here" > >> bubbleSort(self.passengerList, len(self.passengerList)) > >> self.currentFloor += 1 > >> if len(self.passengerList) > 0: > >> for p in self.passengerList: > >> if self.currentFloor == p.destination: > >> yield (p.destination - self.currenteFloor) * TRAVELTIME, > >> self > >> reactivate(p) > >> p.inBuilding() > >> else: > >> self.goUp() > > >> def checkQueue(self): > >> if (len(self.passengerWaitQ)) > 0 and len(self.passengerList) < > >> MAXCAPACITY: > >> if len(self.passengerWaitQ) < MAXCAPACITY: > >> self.newPassengers = len(self.passengerWaitQ) > >> else: > >> self.newPassengers = MAXCAPACITY - len(self.passengerList) > >> for i in range(0,self.newPassengers): > >> self.passengerList.append(self.passengerWaitQ.pop()) > >> self.goUp() > > > Hi Brandon, > > > Nice one at having a good crack at coding before posting! > > > From your posted code, I'm struggling to see what's trying to be > > taught to you for this class assignment. > > not relevant at this point > > > > > As a note it'll be worth reading PEP 8 regarding naming conventions, > > because it looks very Java-ish to me! > > ok but not relevant > > > > > (I might be taking too much a real-world approach in the following, > > but do with it as you will...) > > > I'm assuming that MAXCAPACITY and TRAVELTIME are globals somewhere. > > Although what I'm thinking is that different Elevators will have > > different capacities and different floors they service. An Elevator is > > *not* going to know its number of passengers (the most it could do is > > capacity based on weight restrictions) therefore it's not going to > > know the number of new passengers on each floor either. > > okay but not relevant to the problem at hand > > > > > A couple of things that'd be worthwhile: > > > 1) Post the requirements for your assignment - what's it supposed to > > output etc... > > that isn't relevant for determining at the python level why a function > simply isn't being called > > > 2) Go find an elevator, take a pen and pad with you, and stand in it > > for 30 mins or so, and see how the real thing deals with situations > > and make notes. ie, does it queue requests, or decide to take the next > > nearest floor, when does it stop and open etc...? > > > hth > > > Jon. > > actually it doesn't help at all since you decided to focus on everything > but my actual question of why a function call wasn't working but rather > question the validity of the program itself Heard of something called 'sentences' that start with a capital letter that end with (mostly) a full stop? Anyway, what I suggested was that your design is massively flawed. You ask a group, that's friendly to newbies for help, and you receive advice... then "bitch" about it. Indeed, the intention was to make you re-think the whole program -- then you'd have learnt more. Focusing on one thing when it's the wrong design anyway is a *bad thing*. Remember, you've posted to a list that have members that have done 30+ years of programming in a professional role. Just my 2c, Jon. -- http://mail.python.org/mailman/listinfo/python-list
Re: python interview quuestions
On Sat, Aug 7, 2010 at 6:28 AM, geremy condra wrote: > If I had to wait 5 minutes while a candidate tried to solve this > problem I would not hire them. Yes you do raise a valid point. It should really only take you a mere few seconds or so to write a solution to this. More over, it can be done in just a single line of Python. 7 if you're not very familiar with Python. cheers James -- -- James Mills -- -- "Problems are solved by method" -- http://mail.python.org/mailman/listinfo/python-list
Re: python interview quuestions
On Fri, Aug 6, 2010 at 11:45 AM, James Mills wrote: > On Sat, Aug 7, 2010 at 4:32 AM, Tim Chase > wrote: >>> I would like to aquint myself with Python Interview questions >> >> This came up a while ago: >> >> http://www.mail-archive.com/python-list@python.org/msg168961.html >> >> Most of that thread is still relevant (perhaps throw in some py3l questions >> too) > > A common thing you can do in interviews is ask > your interviewee to write (in Python) a solution > to the "FizzBuzz" problem. Any good competent > Python programmer should be able to do this > in 5-10mins (5 if you're good). > > cheers > james If I had to wait 5 minutes while a candidate tried to solve this problem I would not hire them. One minute, fine- maybe you're just a little rusty or nervous, or the rare person who has to back away from real-world programming concerns to deal with a trivial problem like this. If English isn't your native tongue I might double or even triple that. Otherwise, no way- I'm going to spend more time hanging over your head than you'll save me. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: ctypes: pointer to method
Hi, On Aug 6, 10:10 pm, Martin Landa wrote: > Any idea how to solve it. Thanks, Martin I overlooked note """ Make sure you keep references to CFUNCTYPE objects as long as they are used from C code. ctypes doesn’t, and if you don’t, they may be garbage collected, crashing your program when a callback is made. """ If I defined errtype and errfunc as global variable everything works. Thanks again for your help. Martin -- http://mail.python.org/mailman/listinfo/python-list
Re: ctypes: pointer to method
Hi,
On Aug 5, 9:32 pm, Nobody wrote:
> I don't know about methods, but it works for functions.
>
> > Sample code:
>
> > ...
> > G_set_error_routine(byref(self._print_error))
>
> This won't work; you have to be more explicit, e.g.:
>
> errtype = CFUNCTYPE(c_int, POINTER(c_char), POINTER(c_int))
> errfunc = errtype(print_error)
> G_set_error_routine(errfunc)
the C function is defined as
G_set_error_routine = _libs['grass_gis.
7.0.svn'].G_set_error_routine
G_set_error_routine.restype = None
G_set_error_routine.argtypes = [CFUNCTYPE(UNCHECKED(c_int),
String, c_int)]
I defined in Python function print_error()
def print_error(self, msg, type):
print msg, type
and
errtype = CFUNCTYPE(UNCHECKED(c_int), String, c_int)
errfunc = errtype(print_error)
G_set_error_routine(errfunc)
unfortunately the application crashes when print_error() is called
from C library
static void vfprint_error(int type, const char *template, va_list ap)
{
char buffer[2000]; /* G_asprintf does not work */
vsprintf(buffer, template, ap);
G_debug(5, "print_error(): msg = \"%s\" type = %d", buffer, type);
print_error(buffer, type);
}
Any idea how to solve it. Thanks, Martin
--
http://mail.python.org/mailman/listinfo/python-list
Re: default behavior
On 8/2/2010 11:00 PM, John Posner wrote: On 7/31/2010 1:31 PM, John Posner wrote: Caveat -- there's another description of defaultdict here: http://docs.python.org/library/collections.html#collections.defaultdict ... and it's bogus. This other description claims that __missing__ is a method of defaultdict, not of dict. Following is a possible replacement for the bogus description. Comments welcome. I intend to submit a Python doc bug, and I'd like to have a clean alternative to propose. After some off-list discussion with Ethan Furman (many thanks!), the Python Doc bug is submitted: #9536 at bugs.python.org. -John -- http://mail.python.org/mailman/listinfo/python-list
Re: easy question on parsing python: "is not None"
Stefan Schwarzer wrote: Hello Peter, On 2010-08-06 19:20, Peter Pearson wrote: On Fri, 06 Aug 2010 15:37:04 +0200, Stefan Schwarzer wrote: [snip] I can imagine a case where you might want to compare a string with `is`: FORWARD = "forward" BACKWARD = "backward" ... def func(direction=FORWARD): if direction is FORWARD: ... elif direction is BACKWARD: ... else: ... [...] Hey, that's a cute example, Thanks! but . . . what a trap! Is it possible to document the use-the-object-not-the-string requirement loudly enough that people won't get caught? That's a good question, actually that's the nagging problem I also have with the approach. In production code I might write # Use _these_ objects to indicate directions, not string # constants with the same value. FORWARD = "forward" BACKWARD = "backward" However, that won't help if people import the module and see the strings under "global data" and so assume they're "just strings". Moreover, if you have to write such a comment for every use of the idiom the conciseness of the approach vanishes. Another view at the matter would be to let clients of the module find out their mistake by unit tests. But of course that's somewhat doubtful as the intention should be to write robust instead of bug-inviting code. I wonder if there's a way to have both the "cuteness" of the string constants _and_ more robustness. Stefan Instead of using 'is' use '=='. Maybe not as cute, but definitely more robust! ~Ethan~ -- http://mail.python.org/mailman/listinfo/python-list
Re: [Tutor] Finding the version # of a module, and py module problem
On Aug 6, 2010, at 3:14 PM, W. eWatson wrote:
> I must be missing something. I tried this. (Windows, IDLE, Python 2.5)
> # Try each module
> import sys
> import numpy
> import scipy
> import string
>
> dependencies = "numyp", "scipy"
> for dependency in dependencies:
>try:
>__import__(dependency.name)
>except ImportError:
># Uh oh!
>dependency.installed = None
>else:
># The module loaded OK. Get a handle to it and try to extract
># version info.
># Many Python modules follow the convention of providing their
># version as a string in a __version__ attribute.
>module = sys.modules[dependency.name]
>
># This is what I default to.
>dependency.installed = "[version unknown]"
>
>for attribute_name in ("__version__", "__VERSION__", "VERSION",
> "version"):
>if hasattr(module, attribute_name):
>dependency.installed = getattr(module, attribute_name)
>break
>
> The result was this.
> Traceback (most recent call last):
> File "C:/Users/Wayne/Sandia_Meteors/Trajectory_Estimation/dependency_code",
> line 10, in
>__import__(dependency.name)
> AttributeError: 'str' object has no attribute 'name'
> --
> http://mail.python.org/mailman/listinfo/python-list
dependencies = "numpy", "scipy"
is a tuple of two strings, when you do your for loop you
first get "numpy" (a string) and it does not have a .name
attribute.
Tommy
--
http://mail.python.org/mailman/listinfo/python-list
Re: Easy questions from a python beginner
In article <4c495b50$0$28634$c3e8...@news.astraweb.com>, Steven D'Aprano wrote: >On Thu, 22 Jul 2010 21:23:05 -0700, Stephen Hansen wrote: > >> On 7/22/10 7:47 PM, wheres pythonmonks wrote: >[...] >>> The truth is that I don't intend to use these approaches in anything >>> serious. However, I've been known to do some metaprogramming from time >>> to time. >> >> Depending on how you define "metaprogramming", Python is pretty >> notoriously ill-suited towards the task (more, its basically considered >> a feature it doesn't let you go there) -- or, it allows you to do vast >> amounts of stuff with its few dark magic hooks. I've never had a >> satisfying definition of metaprogramming that more then 50% of any group >> agree with, so I'm not sure which you're looking for. :) > >I disagree strongly at your characterisation that Python is notorious for >being ill-suited towards metaprogramming. I'd say the complete opposite >-- what is considered dark and scary metaprogramming tasks in other >languages is considered too ordinary to even mention in Python. > >If you have a class that defines a single member (or attribute in Python >terminology) "spam", and you want to add a second "ham" to a specific >instance, such a thing is either deep, dark metaprogramming in some >languages, if not outright impossible. In Python it is not even >noteworthy: > >instance.ham = "something" # *yawns* > >Recently there was a thread started by some Java or C++ refugee who was >distressed about attribute access in Python, because it made >metaprogramming frighteningly easy: > >http://mail.python.org/pipermail/python-list/2010-June/1248029.html > >My response at the time was: Python makes metaprogramming *easy*: > >http://mail.python.org/pipermail/python-list/2010-June/1248053.html > >I can't imagine how he would have reacted if we had showed him how easy >it is to monkey-patch built-in functions... > >[...] >> But! What it doesn't let you do is get clever with syntax and write a >> sort of "simplified" or domain specific language to achieve certain >> sorts of repetitive tasks quickly. You always end up writing normal >> Python that looks like all other Python. > >Exactly... 90% of the time that you think you want a DSL, Python beat you >to it. > >It is true that other languages (like Lisp, or Forth) allow you to write >your own syntax. That's incredibly powerful, but it also has serious >costs. If you can define your own syntax, that means every piece of code >you look at is potentially a different language. The answer is, of course, discipline. I'm a Forth programmer and I morf Forth into an assembler. Now some sources become assembler sources and one should be very adamant about it. I've written a Perl interpreter for those sources, and a Python interpreter would be even easier. If I'm doing Python, I don't miss macro possibilities, but sometimes my code gets repetitive. If I ever get bored by repetition, there is m4. m4 < aap.pym > aap.py python aap.py (See my site below for Forth assemblers, and non-trivial m4 usage.) >-- >Steven Groetjes Albert -- -- Albert van der Horst, UTRECHT,THE NETHERLANDS Economic growth -- being exponential -- ultimately falters. alb...@spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst -- http://mail.python.org/mailman/listinfo/python-list
Re: [Tutor] Finding the version # of a module, and py module problem
On Fri, Aug 6, 2010 at 12:14 PM, W. eWatson wrote:
> I must be missing something. I tried this. (Windows, IDLE, Python 2.5)
> # Try each module
> import sys
> import numpy
> import scipy
> import string
>
> dependencies = "numyp", "scipy"
> for dependency in dependencies:
> try:
> __import__(dependency.name)
> except ImportError:
> # Uh oh!
> dependency.installed = None
> else:
> # The module loaded OK. Get a handle to it and try to extract
> # version info.
> # Many Python modules follow the convention of providing their
> # version as a string in a __version__ attribute.
> module = sys.modules[dependency.name]
>
> # This is what I default to.
> dependency.installed = "[version unknown]"
>
> for attribute_name in ("__version__", "__VERSION__", "VERSION",
> "version"):
> if hasattr(module, attribute_name):
> dependency.installed = getattr(module, attribute_name)
> break
>
> The result was this.
> Traceback (most recent call last):
> File "C:/Users/Wayne/Sandia_Meteors/Trajectory_Estimation/dependency_code",
> line 10, in
> __import__(dependency.name)
> AttributeError: 'str' object has no attribute 'name'
> --
Try reading the code, not just copying and pasting. dependencies isn't
supposed to be a list of strings. It's a list of objects (at least) a
name and an installed attribute.
> http://mail.python.org/mailman/listinfo/python-list
>
--
http://mail.python.org/mailman/listinfo/python-list
Re: [Tutor] Finding the version # of a module, and py module problem
I must be missing something. I tried this. (Windows, IDLE, Python 2.5)
# Try each module
import sys
import numpy
import scipy
import string
dependencies = "numyp", "scipy"
for dependency in dependencies:
try:
__import__(dependency.name)
except ImportError:
# Uh oh!
dependency.installed = None
else:
# The module loaded OK. Get a handle to it and try to extract
# version info.
# Many Python modules follow the convention of providing their
# version as a string in a __version__ attribute.
module = sys.modules[dependency.name]
# This is what I default to.
dependency.installed = "[version unknown]"
for attribute_name in ("__version__", "__VERSION__", "VERSION",
"version"):
if hasattr(module, attribute_name):
dependency.installed = getattr(module, attribute_name)
break
The result was this.
Traceback (most recent call last):
File
"C:/Users/Wayne/Sandia_Meteors/Trajectory_Estimation/dependency_code",
line 10, in
__import__(dependency.name)
AttributeError: 'str' object has no attribute 'name'
--
http://mail.python.org/mailman/listinfo/python-list
Re: python interview quuestions
On Sat, Aug 7, 2010 at 5:11 AM, Tim Chase wrote: > Another common thing you can do on a newsgroup is mention the "FizzBuzz" > problem. Any good competent newsgroup will produce a multitude of proposed > solutions, the majority of which will be wrong. ;-) That's actually a very good point! Someone should post this very problem to this newsgroups/list and see how many active python programmers here actually "get it right" :) *evil grin* --james -- -- James Mills -- -- "Problems are solved by method" -- http://mail.python.org/mailman/listinfo/python-list
Re: python interview quuestions
On 08/06/10 13:45, James Mills wrote: On Sat, Aug 7, 2010 at 4:32 AM, Tim Chase wrote: I would like to aquint myself with Python Interview questions This came up a while ago: http://www.mail-archive.com/python-list@python.org/msg168961.html Most of that thread is still relevant (perhaps throw in some py3l questions too) A common thing you can do in interviews is ask your interviewee to write (in Python) a solution to the "FizzBuzz" problem. Any good competent Python programmer should be able to do this in 5-10mins (5 if you're good). Another common thing you can do on a newsgroup is mention the "FizzBuzz" problem. Any good competent newsgroup will produce a multitude of proposed solutions, the majority of which will be wrong. ;-) -tkc -- http://mail.python.org/mailman/listinfo/python-list
Re: Python Portability--Not very portable?
On Fri, Aug 6, 2010 at 10:58 AM, W. eWatson wrote: > Is there a complete illustration of using disutils? Our only dependencies > are on Python Org material. We use no commercial or licensed code. http://tinyurl.com/3yhwjfj Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: python interview quuestions
On Sat, Aug 7, 2010 at 4:32 AM, Tim Chase wrote: >> I would like to aquint myself with Python Interview questions > > This came up a while ago: > > http://www.mail-archive.com/python-list@python.org/msg168961.html > > Most of that thread is still relevant (perhaps throw in some py3l questions > too) A common thing you can do in interviews is ask your interviewee to write (in Python) a solution to the "FizzBuzz" problem. Any good competent Python programmer should be able to do this in 5-10mins (5 if you're good). cheers james -- -- James Mills -- -- "Problems are solved by method" -- http://mail.python.org/mailman/listinfo/python-list
Re: easy question on parsing python: "is not None"
Hello Peter, On 2010-08-06 19:20, Peter Pearson wrote: > On Fri, 06 Aug 2010 15:37:04 +0200, Stefan Schwarzer wrote: > [snip] >> I can imagine a case where you might want to compare a >> string with `is`: >> >> FORWARD = "forward" >> BACKWARD = "backward" >> >> ... >> >> def func(direction=FORWARD): >> if direction is FORWARD: >> ... >> elif direction is BACKWARD: >> ... >> else: >> ... >> [...] > Hey, that's a cute example, Thanks! > but . . . what a trap! Is it > possible to document the use-the-object-not-the-string requirement > loudly enough that people won't get caught? That's a good question, actually that's the nagging problem I also have with the approach. In production code I might write # Use _these_ objects to indicate directions, not string # constants with the same value. FORWARD = "forward" BACKWARD = "backward" However, that won't help if people import the module and see the strings under "global data" and so assume they're "just strings". Moreover, if you have to write such a comment for every use of the idiom the conciseness of the approach vanishes. Another view at the matter would be to let clients of the module find out their mistake by unit tests. But of course that's somewhat doubtful as the intention should be to write robust instead of bug-inviting code. I wonder if there's a way to have both the "cuteness" of the string constants _and_ more robustness. Stefan -- http://mail.python.org/mailman/listinfo/python-list
Perl -> Python unpack
Hey all, Quick question for you Python enthusiasts that also happen to know Perl quite well... What does a* or A* translate to in Python when unpacking binary data with struct.unpack(...) ? cheers James -- -- James Mills -- -- "Problems are solved by method" -- http://mail.python.org/mailman/listinfo/python-list
Re: python interview quuestions
I would like to aquint myself with Python Interview questions This came up a while ago: http://www.mail-archive.com/python-list@python.org/msg168961.html Most of that thread is still relevant (perhaps throw in some py3l questions too) -tkc -- http://mail.python.org/mailman/listinfo/python-list
Re: Python Portability--Not very portable?
> As an example, my inexperienced Python partner 30 miles away has gotten > out of step somehow. I think by installing a different version of numpy > than I use. I gave him a program we both use months ago, and he had no > trouble. (We both use IDLE on 2.5). I made a one character change to it > and sent him the new py file. He can't execute it. I doubt he has > changed anything in the intervening period. I'm curious: what was the one character change? And does your original program still work for him? Why did he install a different version of numpy if things were working for him? By the way, posting "x doesn't work" on a forum never gets you any help, because clairvoyance is not real :D. Posting actual error messages sample code does. > A further example. Months ago I decided to see if I could compile a > program to avoid such problems as above. I planned to satisfy that need, > and see if I could distribute some simple programs to non-Python > friends. I pretty well understand the idea,and got it working with a > small program. It seemed like a lot of manual labor to do it. As someone mentioned, you don't "compile" anything in Python. You can make a .exe file, though, using, as you know, py2exe. I find using GUI2Exe (which requires you have wxPython) to make things much easier and then if you want to be even fancier, use InnoSetup to make an installer. Once you get a working script in py2exe/GUI2Exe, it is usually a snap to make a new version of your .exe after changing your code a bit. As far as then updating your .exe files with your non-Python friends, you should search this newsgroup for Esky, which seems like a very nice idea for doing this very thing. Che -- http://mail.python.org/mailman/listinfo/python-list
Re: Python Portability--Not very portable?
On Aug 5, 9:50 pm, "W. eWatson" wrote: > In my on-again-off-again experience with Python for 18 months, > portability seems an issue. > > As an example, my inexperienced Python partner 30 miles away has gotten > out of step somehow. I think by installing a different version of numpy > than I use. I gave him a program we both use months ago, and he had no > trouble. (We both use IDLE on 2.5). I made a one character change to it > and sent him the new py file. He can't execute it. I doubt he has > changed anything in the intervening period. I'm curious: what was the one character change? And does your original program still work for him? Why did he install a different version of numpy if things were working for him? By the way, posting "x doesn't work" on a forum never gets you any help, because clairvoyance is not real :D. Posting actual error messages sample code does. > A further example. Months ago I decided to see if I could compile a > program to avoid such problems as above. I planned to satisfy that need, > and see if I could distribute some simple programs to non-Python > friends. I pretty well understand the idea,and got it working with a > small program. It seemed like a lot of manual labor to do it. As someone mentioned, you don't "compile" anything in Python. You can make a .exe file, though, using, as you know, py2exe. I find using GUI2Exe (which requires you have wxPython) to make things much easier and then if you want to be even fancier, use InnoSetup to make an installer. Once you get a working script in py2exe/GUI2Exe, it is usually a snap to make a new version of your .exe after changing your code a bit. As far as then updating your .exe files with your non-Python friends, you should search this newsgroup for Esky, which seems like a very nice idea for doing this very thing. Che -- http://mail.python.org/mailman/listinfo/python-list
Re: easy question on parsing python: "is not None"
On 8/6/2010 5:27 AM, Richard D. Moores wrote: So there would be a different implementation for each operating system? One for Windows, one for linux? Or one for Vista and one for XP? I'm just trying to clarify what is meant by "implementation". Different version of CPython (that string caching bit has changed in the past) and CPython versus Jython, PyPy, IronPython, Cython, etc. -- Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
Re: Python Portability--Not very portable?
W. eWatson wrote: > So you think Python is part of open software in terms of distributing a > "product"? So I should stick to C, where one can distribute programs w/o > revealing code details, and having a customer compile the code? It's No, I'm trying to make you reconsider what you're going to protect rather than how. Aren't your customers more interested in using your software rather than tinker with it? -- http://mail.python.org/mailman/listinfo/python-list
Re: Python Portability--Not very portable?
On 8/6/2010 10:31 AM, geremy condra wrote: On Fri, Aug 6, 2010 at 8:00 AM, W. eWatson wrote: I would think there are some small time and big time Python players who sell executable versions of their programs for profit? Yes. What's your point? That someone must know how to distribute them without having the source code ripped off. I've never seen a code obfuscation scheme I thought did the job the whole way, including compiling C, and Python bytecode is significantly easier to turn back into something resembling the original source (YMMV, I suppose). Also, if you don't know about common tools like distutils, the odds are pretty good that it isn't your code itself that is valuable to you- you're probably more interested in protecting your idea about what the code should do. At least for now, that's outside of the scope of technical solutions- discuss it with a lawyer, not a programmer. disutils. Sounds familiar. I'm pretty sure I was using Py2Exe, and disutils might have been part of it. distutils. http://docs.python.org/library/distutils.html I don't see ;how distutils is going to solve this problem. Are you suggesting the program should be packaged? Why? I can just send it to him as py code. distutils looks like it's for library modules, e.g., functions like math. ...no. Distutils is handy because you could just bundle your dependencies and hand them an easy-to-install package, which would be a quick way to get everybody on the same page. Of course, depending on the licenses those dependencies are under you might want to do even more talking to a lawyer than I've previously suggested before you go about trying to sell that bundle- I'm sure you wouldn't want to 'rip off' great free projects like python and numpy. Geremy Condra Yes, code reversal programs have been around for many, many decades. Try one on MS Word or Adobe Acrobat. :-) Is there a complete illustration of using disutils? Our only dependencies are on Python Org material. We use no commercial or licensed code. -- http://mail.python.org/mailman/listinfo/python-list
Re: [Tutor] Finding the version # of a module, and py module problem
On 8/5/2010 6:47 PM, Philip Semanchuk wrote:
On Aug 5, 2010, at 8:55 PM, W. eWatson wrote:
It's been awhile since I've used python, and I recall there is a way
to find the version number from the IDLE command line prompt. dir,
help, __version.__?
Hi Wayne,
FYI it's got nothing to do with IDLE, it's just a question of whether or
not the module in question exposes any kind of a version attribute.
There's no standard, unfortunately. The most popular convention seems to
be via an attribute called __version__, but I've also seen __VERSION__,
VERSION, and version.
Here's some code that I wrote that you might find useful. It's from a
setup.py and it checks a list of modules on which our project depends to
see if (a) they're installed and (b) if the version installed is
adequate. In the snippet below, dependencies is a list of custom classes
that represent modules we need (e.g. numpy).
# Try each module
for dependency in dependencies:
try:
__import__(dependency.name)
except ImportError:
# Uh oh!
dependency.installed = None
else:
# The module loaded OK. Get a handle to it and try to extract
# version info.
# Many Python modules follow the convention of providing their
# version as a string in a __version__ attribute.
module = sys.modules[dependency.name]
# This is what I default to.
dependency.installed = "[version unknown]"
for attribute_name in ("__version__", "__VERSION__", "VERSION",
"version"):
if hasattr(module, attribute_name):
dependency.installed = getattr(module, attribute_name)
break
Hope this helps a little,
Philip
Thanks. I'll look into it.
--
http://mail.python.org/mailman/listinfo/python-list
Re: python interview quuestions
On 08/06/2010 10:44 AM, prakash jp wrote: Hi all, I would like to aquint myself with Python Interview questions . I am a Python Scripter, so if u could orient the pointers in the same direction it would be very handy Regards Huh??? -- http://mail.python.org/mailman/listinfo/python-list
python interview quuestions
Hi all, I would like to aquint myself with Python Interview questions . I am a Python Scripter, so if u could orient the pointers in the same direction it would be very handy Regards -- http://mail.python.org/mailman/listinfo/python-list
Re: Python Portability--Not very portable?
On 8/6/2010 9:03 AM, Peter Otten wrote: ... Seriously, I try to make a joke once in a while, usually with devastating results. The idea you were meant to take away was that once you start thinking about a protection scheme there is always a next step until you reach the point where your software, say, is completely safe, but also completely unusable. Had Guido started the language in that mindset there would be no Python for you to worry about its ripp-off safety. Why would I send you the py code, for example, if I wanted to protect it because of its importance? Because if you think again you may find that it's not as important as you think? I'd put it in exe form and send it and allow you to input data to produce the desired result of the program. There is no analog in python, and if you cannot concentrate on your honest customers the only option that offers reasonable "safety" would be to turn your application into web service. Peter So you think Python is part of open software in terms of distributing a "product"? So I should stick to C, where one can distribute programs w/o revealing code details, and having a customer compile the code? It's been awhile since I've used Linux or Unix, but I think there's a lot of commercial code out there dependent upon it, and the users do not have to compile anything. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python Portability--Not very portable?
On Fri, Aug 6, 2010 at 8:00 AM, W. eWatson wrote: > >>> I would think there are some small time and big time Python players who >>> sell >>> executable versions of their programs for profit? >> >> Yes. What's your point? > > That someone must know how to distribute them without having the source code > ripped off. I've never seen a code obfuscation scheme I thought did the job the whole way, including compiling C, and Python bytecode is significantly easier to turn back into something resembling the original source (YMMV, I suppose). Also, if you don't know about common tools like distutils, the odds are pretty good that it isn't your code itself that is valuable to you- you're probably more interested in protecting your idea about what the code should do. At least for now, that's outside of the scope of technical solutions- discuss it with a lawyer, not a programmer. >> >>> disutils. Sounds familiar. I'm pretty sure I was using Py2Exe, and >>> disutils >>> might have been part of it. >> >> distutils. >> >> http://docs.python.org/library/distutils.html > > I don't see ;how distutils is going to solve this problem. Are you > suggesting the program should be packaged? Why? I can just send it to him as > py code. distutils looks like it's for library modules, e.g., functions like > math. ...no. Distutils is handy because you could just bundle your dependencies and hand them an easy-to-install package, which would be a quick way to get everybody on the same page. Of course, depending on the licenses those dependencies are under you might want to do even more talking to a lawyer than I've previously suggested before you go about trying to sell that bundle- I'm sure you wouldn't want to 'rip off' great free projects like python and numpy. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: easy question on parsing python: "is not None"
On Fri, 06 Aug 2010 15:37:04 +0200, Stefan Schwarzer wrote: [snip] > I can imagine a case where you might want to compare a > string with `is`: > > FORWARD = "forward" > BACKWARD = "backward" > > ... > > def func(direction=FORWARD): > if direction is FORWARD: > ... > elif direction is BACKWARD: > ... > else: > ... > > in case you expect people to specifically use the constants > you provided in the module. Here, the fact that FORWARD > actually is the string "forward" might be considered an > implementation detail. Using a string instead of an > `object()` has the advantage that it makes usage in error > messages easier. > > Actually, I've never seen such a use, as far as I remember. > What do other people here think? Is the code above, which > compares strings with `is`, bad style, and if yes, why? How > would you write the code instead? Hey, that's a cute example, but . . . what a trap! Is it possible to document the use-the-object-not-the-string requirement loudly enough that people won't get caught? -- To email me, substitute nowhere->spamcop, invalid->net. -- http://mail.python.org/mailman/listinfo/python-list
Re: Need help from someone that have PIL installed
On 08/06/2010 04:37 PM, alejandro wrote:
>
>> # the last tuple is the background color
>> img = Image.new("RGBA",(300, 50), (0, 0, 0, 0))
> Thank you for this
>
>> # I think that the PIL can cope with unicode, so add a u-prefix here:
>> text = u"proba test ¾æèð¹"
>> draw.text((20,8), text ,font=arial, fill="red")
>
> Nope i gives:
> SyntaxError: (unicode error) 'utf8' codec can't decode byte 0x9e in position
> 0: unexpected code byte
Is the file, which you claim is UTF-8 encoded, actually UTF-8 encoded?
If you're not sure, explicitly tell your text editor to save the file as
UTF-8, and then try again.
>
> and without the encoding :
> SyntaxError: Non-ASCII character '\x9e' in file C:\Documents and
> Settings\Perc\My Documents\NetBeansProjects\venus_header\src\venus_header.py
> on line 16, but no encoding declared; see
> http://www.python.org/peps/pep-0263.html for details
>
>
>
--
http://mail.python.org/mailman/listinfo/python-list
Re: Smith-Waterman Algorithm in Python
> Does any one about any implementation of classical Smith Waterman > local alignment algorithm and it's variants for aligning natural > language text? Please see http://tinyurl.com/2wy43fh Cheers, Daniel -- Psss, psss, put it down! - http://www.cafepress.com/putitdown -- http://mail.python.org/mailman/listinfo/python-list
Smith-Waterman Algorithm in Python
Hi, Does any one about any implementation of classical Smith Waterman local alignment algorithm and it's variants for aligning natural language text? thanks -- http://mail.python.org/mailman/listinfo/python-list
Re: Checking that 2 pdf are identical (md5 a solution?)
In article , rlevesque wrote: > >Given your expertise I will not be able to 'repay' you by helping on >Python problems but if you ever need help with SPSS related problems I >will be pleased to provide the assistance you need. Generally speaking, the community philosophy is "pay forward" -- help someone else who needs it (either here or somewhere else). When everyone helps other people, it all evens out. -- Aahz (a...@pythoncraft.com) <*> http://www.pythoncraft.com/ "Normal is what cuts off your sixth finger and your tail..." --Siobhan -- http://mail.python.org/mailman/listinfo/python-list
Re: Python Portability--Not very portable?
W. eWatson wrote: > I can't respond to otten directly, since he uses gmane. Here's my > response. > > W. eWatson wrote: > > >>> >>> I would think there are some small time and big time Python > players who > >>> >>> sell executable versions of their programs for profit? > >> >> > >> >> Yes. What's your point? > > > That someone must know how to distribute them without having the > > > source code ripped off. > Ott wrote? > Yes, but he won't tell for fear of getting ripped off of his knowledge. > > > Who won't tell? The the guy who knows how to distribute the source code without having it ripped off... Seriously, I try to make a joke once in a while, usually with devastating results. The idea you were meant to take away was that once you start thinking about a protection scheme there is always a next step until you reach the point where your software, say, is completely safe, but also completely unusable. Had Guido started the language in that mindset there would be no Python for you to worry about its ripp-off safety. > Why would I send you the py code, for example, if I > wanted to protect it because of its importance? Because if you think again you may find that it's not as important as you think? > I'd put it in exe form > and send it and allow you to input data to produce the desired result of > the program. There is no analog in python, and if you cannot concentrate on your honest customers the only option that offers reasonable "safety" would be to turn your application into web service. Peter -- http://mail.python.org/mailman/listinfo/python-list
Re: Need help from someone that have PIL installed
alejandro wrote:
>> Make sure that
>>
>> # encoding:utf-8
>>
>> is the first line of your script, details and fineprint here:
>>
>> http://www.python.org/dev/peps/pep-0263/
>>
>> Peter
>
> Tryed that...
What happened?
> What was the output of my script on your computer?
$ python -V
Python 2.6.4
$ python -c "import Image; print Image.VERSION"
1.1.6
$ cat draw_text.py
# encoding:utf-8
from PIL import Image
import ImageDraw
import ImageFont
img = Image.new("RGBA",(300, 50), (0, 0, 0, 0))
draw = ImageDraw.Draw(img)
FONT = "/usr/share/fonts/truetype/msttcorefonts/arial.ttf"
arial = ImageFont.truetype(FONT, 32)
text = u"proba test ¾æèð¹"
print repr(text)
draw.text((20,8), text, font=arial, fill="red")
img.save("tmp.png")
$ python draw_text.py
u'proba test \xbe\xe6\xe8\xf0\xb9'
The image looks as expected.
Peter
--
http://mail.python.org/mailman/listinfo/python-list
Re: Python Portability--Not very portable?
I can't respond to otten directly, since he uses gmane. Here's my response. W. eWatson wrote: >>> >>> I would think there are some small time and big time Python players who >>> >>> sell executable versions of their programs for profit? >> >> >> >> Yes. What's your point? > > That someone must know how to distribute them without having the source > > code ripped off. Ott wrote? Yes, but he won't tell for fear of getting ripped off of his knowledge. Who won't tell? Why would I send you the py code, for example, if I wanted to protect it because of its importance? I'd put it in exe form and send it and allow you to input data to produce the desired result of the program. -- http://mail.python.org/mailman/listinfo/python-list
Re: Need help from someone that have PIL installed
> > Make sure that > > # encoding:utf-8 > > is the first line of your script, details and fineprint here: > > http://www.python.org/dev/peps/pep-0263/ > > Peter Tryed that... What was the output of my script on your computer? -- http://mail.python.org/mailman/listinfo/python-list
Re: Python Portability--Not very portable?
W. eWatson wrote: >>> I would think there are some small time and big time Python players who >>> sell executable versions of their programs for profit? >> >> Yes. What's your point? > That someone must know how to distribute them without having the source > code ripped off. Yes, but he won't tell for fear of getting ripped off of his knowledge. -- http://mail.python.org/mailman/listinfo/python-list
Re: [Tutor] Finding the version # of a module, and py module problem
On Aug 6, 2010, at 10:20 AM, Richard D. Moores wrote: On Thu, Aug 5, 2010 at 18:47, Philip Semanchuk wrote: it's just a question of whether or not the module in question exposes any kind of a version attribute. There's no standard, unfortunately. The most popular convention seems to be via an attribute called __version__, but I've also seen __VERSION__, VERSION, and version. Here's one more way: import gmpy gmpy.__version__ Traceback (most recent call last): File "", line 1, in AttributeError: 'module' object has no attribute '__version__' gmpy.version() '1.12' That's the nice thing about standards -- there are so many to choose from! =) Thanks for pointing that out; I'll update my code. bye Philip -- http://mail.python.org/mailman/listinfo/python-list
Re: Python Portability--Not very portable?
I would think there are some small time and big time Python players who sell executable versions of their programs for profit? Yes. What's your point? That someone must know how to distribute them without having the source code ripped off. disutils. Sounds familiar. I'm pretty sure I was using Py2Exe, and disutils might have been part of it. distutils. http://docs.python.org/library/distutils.html I don't see ;how distutils is going to solve this problem. Are you suggesting the program should be packaged? Why? I can just send it to him as py code. distutils looks like it's for library modules, e.g., functions like math. So how does one keep a non-Python user in lock step with my setup, so these problems don't arise? I don't even want to think about having him uninstall and re-install. :-) Although maybe he could do it without making matters worse. That's going to hinge on what your dependencies are. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: Need help from someone that have PIL installed
alejandro wrote:
>> # the last tuple is the background color
>> img = Image.new("RGBA",(300, 50), (0, 0, 0, 0))
> Thank you for this
>
>> # I think that the PIL can cope with unicode, so add a u-prefix here:
>> text = u"proba test �"
>> draw.text((20,8), text ,font=arial, fill="red")
>
> Nope i gives:
> SyntaxError: (unicode error) 'utf8' codec can't decode byte 0x9e in
> position 0: unexpected code byte
>
> and without the encoding :
> SyntaxError: Non-ASCII character '\x9e' in file C:\Documents and
> Settings\Perc\My
> Documents\NetBeansProjects\venus_header\src\venus_header.py on line 16,
> but no encoding declared; see http://www.python.org/peps/pep-0263.html for
> details
Make sure that
# encoding:utf-8
is the first line of your script, details and fineprint here:
http://www.python.org/dev/peps/pep-0263/
Peter
--
http://mail.python.org/mailman/listinfo/python-list
Re: new to python - trouble calling a function from another function
On Aug 5, 2:01 pm, Daniel Urban wrote:
> > I'm building an elevator simulator for a class assignment. I recently ran
> > into a roadblock and don't know how to fix it. For some reason, in my
> > checkQueue function below, the call to self.goUp() is never executed. It is
> > on the last line of code I pasted in. I can put print statements before and
> > after the call and I have a print statement in goUp() itself. Only the
> > print statements before and after the call are executed. The one inside
> > goUp() is never executed because goUp() never seems to be executed. How can
> > that be? I don't get any errors when the script executes. Surely this isn't
> > some limitation I'm encountering?
>
> I think the self.goUp() call is executed, the goUp function gets
> called, returns a generator object (because goUp is a generator
> function), then you don't use that generator object for anything.
>
> Daniel
Brandon, this example might help you understand the problem:
>>> def g():
print('p1')
yield 2
print('p3')
>>> g()
>>> b=g()
>>> next(b)
p1
2
>>> next(b)
p3
Traceback (most recent call last):
File "", line 1, in
next(b)
StopIteration
>>>
--
http://mail.python.org/mailman/listinfo/python-list
Re: Need help from someone that have PIL installed
> # the last tuple is the background color
> img = Image.new("RGBA",(300, 50), (0, 0, 0, 0))
Thank you for this
> # I think that the PIL can cope with unicode, so add a u-prefix here:
> text = u"proba test ¾æèð¹"
> draw.text((20,8), text ,font=arial, fill="red")
Nope i gives:
SyntaxError: (unicode error) 'utf8' codec can't decode byte 0x9e in position
0: unexpected code byte
and without the encoding :
SyntaxError: Non-ASCII character '\x9e' in file C:\Documents and
Settings\Perc\My Documents\NetBeansProjects\venus_header\src\venus_header.py
on line 16, but no encoding declared; see
http://www.python.org/peps/pep-0263.html for details
--
http://mail.python.org/mailman/listinfo/python-list
Re: [Tutor] Finding the version # of a module, and py module problem
On Thu, Aug 5, 2010 at 18:47, Philip Semanchuk wrote: > > it's just a question of whether or not > the module in question exposes any kind of a version attribute. There's no > standard, unfortunately. The most popular convention seems to be via an > attribute called __version__, but I've also seen __VERSION__, VERSION, and > version. > Here's one more way: >>> import gmpy >>> gmpy.__version__ Traceback (most recent call last): File "", line 1, in AttributeError: 'module' object has no attribute '__version__' >>> >>> gmpy.version() '1.12' >>> Dick Moores -- http://mail.python.org/mailman/listinfo/python-list
Re: How to read large amounts of output via popen
Ok, thats great. Thanks for the very elegant solution(s)
On 6 Aug, 13:44, Nobody wrote:
> On Fri, 06 Aug 2010 02:06:29 -0700, loial wrote:
> > I need to read a large amount of data that is being returned in
> > standard output by a shell script I am calling.
>
> > (I think the script should really be writing to a file but I have no
> > control over that)
>
> If the script is writing to stdout, you get to decide whether its stdout
> is a pipe, file, tty, etc.
>
> > Currently I have the following code. It seeems to work, however I
> > suspect this may not work with large amounts of standard output.
> > process=subprocess.Popen(['myscript', 'param1'],
> > shell=False,stdout=subprocess.PIPE,stderr=subprocess.PIPE)
>
> > cmdoutput=process.communicate()
>
> It's certainly not the best way to read large amounts of output.
> Unfortunately, better solutions get complicated when you need to read more
> than one of stdout and stderr, or if you also need to write to stdin.
>
> If you only need stdout, you can just read from process.stdout in a loop.
> You can leave stderr going to wherever the script's stderr goes (e.g. the
> terminal), or redirect it to a file.
>
> If you really do need both stdout and stderr, then you either need to
> enable non-blocking I/O, or use a separate thread for each stream, or
> redirect at least one of them to a file.
>
> FWIW, Popen.communicate() uses non-blocking I/O on Unix and separate
> threads on Windows (the standard library doesn't include a mechanism to
> enable non-blocking I/O on Windows).
>
> > What is the best way to read a large amount of data from standard
> > output and write to a file?
>
> For this case, the best way is to just redirect stdout to a file, rather
> than passing it through the script, i.e.:
>
> outfile = open('outputfile', 'w')
> process = subprocess.call(..., stdout = outfile)
> outfile.close()
--
http://mail.python.org/mailman/listinfo/python-list
Re: Need help from someone that have PIL installed
alejandro wrote:
> Can please someone run this little script that should output characters
> like � in an image.
> If it does it correctly can you tell me what OS, python version & PIL
> version you have?
> Or better if someone can tell me why this is not working properly on my
> PC? (Win XP, PIL 1.1.6., Python 2.6...)
> I don't recive any error it's just that the characters outputed in the
> image are not characters... more like symbols...
>
> # encoding:utf-8
>
> from PIL import Image
> import ImageDraw
> import ImageFont
# the last tuple is the background color
img = Image.new("RGBA",(300, 50), (0, 0, 0, 0))
> draw = ImageDraw.Draw(img)
> arial = ImageFont.truetype("arial.ttf",32) # needs path to font, my font
> was in the same folder as the script
# I think that the PIL can cope with unicode, so add a u-prefix here:
text = u"proba test �"
draw.text((20,8), text ,font=arial, fill="red")
> # write
> img.save("img2.png", "PNG")
Peter
--
http://mail.python.org/mailman/listinfo/python-list
Need help from someone that have PIL installed
Can please someone run this little script that should output characters like
¾æè¹ð in an image.
If it does it correctly can you tell me what OS, python version & PIL
version you have?
Or better if someone can tell me why this is not working properly on my PC?
(Win XP, PIL 1.1.6., Python 2.6...)
I don't recive any error it's just that the characters outputed in the image
are not characters... more like symbols...
# encoding:utf-8
from PIL import Image
import ImageDraw
import ImageFont
img = Image.new("RGBA",(250,40))
#--- making the image transparent
pixdata = img.load()
for y in xrange(img.size[1]):
for x in xrange(img.size[0]):
if pixdata[x, y] == (255, 255, 255, 255):
pixdata[x, y] = (255, 255, 255, 0)
#- drawing text
draw = ImageDraw.Draw(img)
arial = ImageFont.truetype("arial.ttf",32) # needs path to font, my font
was in the same folder as the script
string = "proba test ¾æèð¹"
draw.text((20,8),string ,font=arial, fill="red")
# write
img.save("img2.png", "PNG")
--
http://mail.python.org/mailman/listinfo/python-list
Re: Why is python not written in C++ ?
On Fri, Aug 6, 2010 at 8:39 PM, Roy Smith wrote: > In article , > David Cournapeau wrote: > >> > Yes, there are a few corner cases where valid C syntax has different >> > semantics in C and C++. But, they are very few. Calling C++ a superset >> > of C is essentially correct. >> >> This is only true if you limit yourself to C89 (as python seems to >> do). If you start using C99 (and lot of people do, if only because >> they don't realize it because gcc is quite relax about it), then >> almost no non trivial C code is valid C++ in my experience. > > I'm not following you. If anything, C99 makes C closer to C++. Can you > give me some examples of valid C99 which is not also valid C++? variable size array, the meaning of inline, etc... In addition to int f(void) vs int f(), legality vs illegality of p = malloc(n);, type of enum, keyword incompatibility (new, delete, etc...) which are already there in C89. I have yet seen a project where you could build C code with a C++ compiler - the only ones I know are specifically designed that way and it is painful. David -- http://mail.python.org/mailman/listinfo/python-list
Re: easy question on parsing python: "is not None"
Hi DG, On 2010-08-06 14:28, DG wrote: > I've always thought of it as you don't compare strings with "is", you > *should* use == The reasoning is that you don't know if that string > instance is the only one in memory. I've heard as an implementation > detail, since strings are immutable, that Python will only create one > actual instance and just assign new references to it (your first x is > y example), but to compare equality it just makes sense to use "==", > not to ask if it is the same object in memory. Yes, you'd usually use == to compare strings. The use in the post presumably was to show the behavior when you use `is`; I guess it wasn't meant as an example for production code. :) I can imagine a case where you might want to compare a string with `is`: FORWARD = "forward" BACKWARD = "backward" ... def func(direction=FORWARD): if direction is FORWARD: ... elif direction is BACKWARD: ... else: ... in case you expect people to specifically use the constants you provided in the module. Here, the fact that FORWARD actually is the string "forward" might be considered an implementation detail. Using a string instead of an `object()` has the advantage that it makes usage in error messages easier. Actually, I've never seen such a use, as far as I remember. What do other people here think? Is the code above, which compares strings with `is`, bad style, and if yes, why? How would you write the code instead? > Plus, I believe the > "==" operator will check if the variables point to the same object. No, that's what `is` is for. > Using is/is not with None works well, because I believe there will > always only be 1 None object. Yes, and it avoids subtle bugs if someone overwrites `__eq__` in some class: >>> class AlwaysEqual(object): ... def __eq__(self, other): ... return True ... >>> always_equal = AlwaysEqual() >>> always_equal == None True >>> None == always_equal True >>> always_equal is None False Stefan -- http://mail.python.org/mailman/listinfo/python-list
Re: Import python modules from sub-directories
On 06-Aug-2010, at 1:13 PM, 夏震 wrote: > > >> Hi guys, >> >> I am new to python and would like to import certain classes in >> sub-directories of the >> working directory. I was wondering how will I be able to achieve this? >> >> Regards, >> Nav >> -- >> http://mail.python.org/mailman/listinfo/python-list > >> > > Please try "touch __init__.py" in sub-directories. > > > -- > http://mail.python.org/mailman/listinfo/python-list Thanks, that worked : ) -- http://mail.python.org/mailman/listinfo/python-list
Re: pylint scores
On Sun, Aug 1, 2010 at 12:27 PM, News123 wrote: > Hi, > > > On 07/31/2010 11:04 AM, Matteo Landi wrote: >> What are the messages one should really care about while evaluating >> its code using pylint? It's easy to get 5 scored with a "lot of public >> methods" or bad named variables such as 'x' or 'y' .. Have you got any >> config file to share? > > > The most important ones are of course the errors. > Some of them might be false, but in our team we agreed, that no file is > allowed to report pylint errors. > This means > - just fixing errors (in most cases) > - rewriting code (in a few cases) > - masking errors with pylint directives in the source (some other few > errrors) > > > > > If you only want to see the errros, then just run > pylint -E filename.py > > Note: This is a rather new but very useful switch.. It doesn't exist > on Ubuntu 10.4's release pylint 0.19.0, but exists on pylint 0.21.1 > > > > Apart from that. You should discuss within your team, which > errors you'd like to have ignored and adapt the pylintrc. This > is a rather personal decision. > For one project we disiabled for example following warnings: > ## C0322 = "Operator not preceded by a space" > ## C0323 = "Operator not followed by a space" > ## C0324 = "Comma not followed by a space" > As we did not have time to rewrite all of the existing code, that > violated these rules. > We prefered to 'hide' these warnings in order to sett the more important > ones. > > On another new project however we did NOT comment therese rules > and decided, that all new code should follow these rules > > > We disabled some special global variables, which we'd like to have in > lower case > > const-rgx ==((specialvar)|(v_[a-z_][a-z0-9_]*)|([A-Z_][A-Z0-9_]*)|(__.*__))$ > > > you could also modify variables like > # Maximum number of attributes for a class (see R0902). > max-attributes=7 > > # Minimum number of public methods for a class (see R0903). > min-public-methods=2 > > # Maximum number of public methods for a class (see R0904). > max-public-methods=20 > > > For some graphics module functions for example we wanted to > be allowed to use variable names like x,y as they are > completely meaningful names for pixel coordinates. > > > so change the entry good-names > good-names=x,y,ex,Run,_ Thank you so much, these are very precious settings. > > > Hope, that this gave you some ideas > > > >> >> On Sat, Jul 31, 2010 at 2:48 AM, Dan Stromberg wrote: >>> >>> On Fri, Jul 30, 2010 at 12:18 PM, News123 wrote: On 07/30/2010 03:12 PM, wheres pythonmonks wrote: > I am starting to use pylint to look at my code and I see that it gives a > rating. > What values do experienced python programmers get on code not > targeting the benchmark? > > I wrote some code, tried to keep it under 80 characters per line, > reasonable variable names, and I got: > > 0.12 / 10. > > Is this a good score for one not targeting the benchmark? (pylint > running in default mode) > It's not a goodf core, but arrives easily if you never ran pylint before. With very little effort you should be able to be above 5 with a little more effort above 7 > Somewhat related: Is the backslash the only way to extend arguments > to statements over multiple lines? (e.g.) if you have an opening parenthesis, or bracked, then you don't need a backslash so instead of if longlonglonglonglonglonglonglongvar == \ otherlonglonglonglongvar: you could also write: if (longlonglonglonglonglonglonglongvar == otherlonglonglonglongvar): same works of course with asserts. > def f(x,y,z): return(x+y+z); > ... f(1,2, > ... 3) > 6 assert f(1,2,3)>0, > File "", line 1 > assert f(1,2,3)>0, > ^ > SyntaxError: invalid syntax > > In the above, I could split the arguments to f (I guess b/c of the > parens) but not for assert. I could use a backslash, but I find this > ugly -- it that my only (best?) option? > > [I really like to assert my code to correctness and I like using the > second argument to assert, but this resulted in a lot of long lines > that I was unable to break except with an ugly backslash.] > > W >>> IMO, the important thing about pylint's scoring is that it's but one way of >>> many of producing good Python code. However, it's also one of the easier >>> ways of producing good python code. >>> I personally like to get my scores up near 10, by annotating in comments >>> about the few things that pylint flags that I can't just code around. This >>> requires jumping through some slightly silly hoops (EG the previously >>> mentioned "too few public methods", which my various container classes >>> always trip over), but going through this process is worthwhile for >>> highlightin
Re: How to read large amounts of output via popen
On Fri, 06 Aug 2010 02:06:29 -0700, loial wrote:
> I need to read a large amount of data that is being returned in
> standard output by a shell script I am calling.
>
> (I think the script should really be writing to a file but I have no
> control over that)
If the script is writing to stdout, you get to decide whether its stdout
is a pipe, file, tty, etc.
> Currently I have the following code. It seeems to work, however I
> suspect this may not work with large amounts of standard output.
> process=subprocess.Popen(['myscript', 'param1'],
> shell=False,stdout=subprocess.PIPE,stderr=subprocess.PIPE)
>
> cmdoutput=process.communicate()
It's certainly not the best way to read large amounts of output.
Unfortunately, better solutions get complicated when you need to read more
than one of stdout and stderr, or if you also need to write to stdin.
If you only need stdout, you can just read from process.stdout in a loop.
You can leave stderr going to wherever the script's stderr goes (e.g. the
terminal), or redirect it to a file.
If you really do need both stdout and stderr, then you either need to
enable non-blocking I/O, or use a separate thread for each stream, or
redirect at least one of them to a file.
FWIW, Popen.communicate() uses non-blocking I/O on Unix and separate
threads on Windows (the standard library doesn't include a mechanism to
enable non-blocking I/O on Windows).
> What is the best way to read a large amount of data from standard
> output and write to a file?
For this case, the best way is to just redirect stdout to a file, rather
than passing it through the script, i.e.:
outfile = open('outputfile', 'w')
process = subprocess.call(..., stdout = outfile)
outfile.close()
--
http://mail.python.org/mailman/listinfo/python-list
Re: easy question on parsing python: "is not None"
On Aug 6, 2:32 am, Bruno Desthuilliers wrote: > Richard D. Moores a écrit : > > > > > On Thu, Aug 5, 2010 at 16:15, Rhodri James > > wrote: > >> On Thu, 05 Aug 2010 17:07:53 +0100, wheres pythonmonks > >> wrote: > > >> You're not testing for equivalence there, you're testing for identity. > >> "is" > >> and "is not" test whether the two objects concerned are (or are not) the > >> same object. Two objects can have the same value, but be different > >> objects. > >> The interpreter can fool you by caching and reusing objects which have the > >> same value when it happens to know about it, in particular for small > >> integers, but this is just a happy accident of the implementation and in no > >> way guaranteed by the language. For example: > > > "spam, eggs, chips and spam" is "spam, eggs, chips and spam" > >> True > > a = "spam, eggs, chips and spam" > > b = "spam, eggs, chips and spam" > > a is b > >> False > > a == b > >> True > > > I'm wondering if there isn't considerable predictability to that > > "happy accident". > > There is, indeed, but that's really implementation details. > > > Note how 1 'word' is treated versus 2: > > x = > 'alksjdhflkajshdflkajhdflkjahsdflkjshadflkjhsadlfkjhaslkdjfhslkadhflkjshdflkjshdflkjshdfk' > y = > 'alksjdhflkajshdflkajhdflkjahsdflkjshadflkjhsadlfkjhaslkdjfhslkadhflkjshdflkjshdflkjshdfk' > x is y > > True > x = 'alksjdhflkajshdflkajhdflkjahsdflkj > hadflkjhsadlfkjhaslkdjfhslkadhflkjshdflkjshdflkjshdfk' > y = 'alksjdhflkajshdflkajhdflkjahsdflkj > hadflkjhsadlfkjhaslkdjfhslkadhflkjshdflkjshdflkjshdfk' > x is y > > False > > CPython caches strings that happen to be valid Python identifiers. But > once again, this is an implementation-specific optimization. I've always thought of it as you don't compare strings with "is", you *should* use == The reasoning is that you don't know if that string instance is the only one in memory. I've heard as an implementation detail, since strings are immutable, that Python will only create one actual instance and just assign new references to it (your first x is y example), but to compare equality it just makes sense to use "==", not to ask if it is the same object in memory. Plus, I believe the "==" operator will check if the variables point to the same object. Using is/is not with None works well, because I believe there will always only be 1 None object. -- http://mail.python.org/mailman/listinfo/python-list
Re: Why is python not written in C++ ?
In article , David Cournapeau wrote: > > Yes, there are a few corner cases where valid C syntax has different > > semantics in C and C++. Â But, they are very few. Â Calling C++ a superset > > of C is essentially correct. > > This is only true if you limit yourself to C89 (as python seems to > do). If you start using C99 (and lot of people do, if only because > they don't realize it because gcc is quite relax about it), then > almost no non trivial C code is valid C++ in my experience. I'm not following you. If anything, C99 makes C closer to C++. Can you give me some examples of valid C99 which is not also valid C++? -- http://mail.python.org/mailman/listinfo/python-list
Re: Python Portability--Not very portable?
On Fri, Aug 6, 2010 at 12:28 AM, W. eWatson wrote: > On 8/5/2010 7:45 PM, geremy condra wrote: > >> On Thu, Aug 5, 2010 at 6:50 PM, W. eWatson >> wrote: >> >>> In my on-again-off-again experience with Python for 18 months, >>> portability >>> >>> seems an issue. >>> >>> As an example, my inexperienced Python partner 30 miles away has gotten >>> out >>> of step somehow. I think by installing a different version of numpy than >>> I >>> use. I gave him a program we both use months ago, and he had no trouble. >>> (We >>> both use IDLE on 2.5). I made a one character change to it and sent him >>> the >>> new py file. He can't execute it. I doubt he has changed anything in the >>> intervening period. >>> >> >> Portability doesn't mean you can use different versions of your >> dependencies and be A-OK. It should be fairly obvious that if the >> behavior of your dependencies changes, your code needs to change to >> ensure that it demonstrates the same behavior. Portability also >> doesn't mean that any given one-character change is valid, so that may >> be your issue as well. >> >> A further example. Months ago I decided to see if I could compile a >>> program >>> to avoid such problems as above. I planned to satisfy that need, and see >>> if >>> I could distribute some simple programs to non-Python friends. I pretty >>> well >>> understand the idea,and got it working with a small program. It seemed >>> like >>> a lot of manual labor to do it. >>> >> >> What, why were you compiling a program? And why not just use distutils? >> >> Geremy Condra >> > > I checked the one char change on my system thoroughly. I looked around on > some forums and NGs 4 months ago, and found no one even had a simple > "compiled program" available to even demonstrate some simple example. > > I would think there are some small time and big time Python players who > sell executable versions of their programs for profit? > > disutils. Sounds familiar. I'm pretty sure I was using Py2Exe, and disutils > might have been part of it. > And what was hard to do with Py2Exe? i used it on relatively complex college work using QT and the worse i had to do was include a "sip" option to make Py2Exe work with PyQT. But the setup.py script that generates the .exe (automatically) was ridiculously small and simple ( https://svn.inf.ufsc.br/katcipis/python/FormaisGUI/src/setup.py). For me compiling a python program into an .exe always was pretty easy (already made it at my work with a prototype, that one was quite complex, i just had some problem with the netifaces module). > > So how does one keep a non-Python user in lock step with my setup, so these > problems don't arise? I don't even want to think about having him uninstall > and re-install. :-) Although maybe he could do it without making matters > worse. > > -- > http://mail.python.org/mailman/listinfo/python-list > -- http://www.getgnulinux.org/windows -- http://mail.python.org/mailman/listinfo/python-list
Re: Unicode error
In Nobody writes: >On Fri, 23 Jul 2010 10:42:26 +, Steven D'Aprano wrote: >> Don't write bare excepts, always catch the error you want and nothing >> else. >That advice would make more sense if it was possible to know which >exceptions could be raised. In practice, that isn't possible, as the >documentation seldom provides this information. Even for the built-in >classes, the documentation is weak in this regard; for less important >modules and third-party libraries, it's entirely absent. I don't get your point. Even when I *know* that a certain exception may happen, I don't necessarily catch it. I catch only those exceptions for which I can think of a suitable response that is *different* from just letting the program fail. (After all, my own code raises its own exceptions with the precise intention of making the program fail.) If an unexpected exception occurs, then by definition, I had no better response in mind for that situation than just letting the program fail, so I'm happy to let that happen. If, afterwards, I think of a different response for a previously uncaught exception, I'll modify the code accordingly. I find this approach far preferable to the alternative of knowing a long list of possible exceptions (some of which may never happen in actual practice), and think of ways to keep the program still alive no-matter-what. "No memory? No disk space? No problem! Just a flesh wound!" What's the point of that? (If I want the final error message to be something other than a bare stack trace, I may wrap the whole execution in a global/top-level try/catch block so that I can fashion a suitable error message right before calling exit, but that's just "softening the fall": the program still will go down.) -- http://mail.python.org/mailman/listinfo/python-list
