Re: [python] Re: GitHub's ³pull request² is proprietary lock-in
On Sun, Jan 03, 2016 at 04:31:55AM -0500, Random832 wrote: > But there is no command to create a "pull request", nowhere for such > a thing to exist in the repository, etc. There is this [1]. > Also if someone puts through a github pull request and then their > patch is accepted, my understanding is that the pull request has to > be "closed" through a github online interface and merely merging the > patch through the git command line will not update the status on the > pull request. This is incorrect. GitHub will automatically mark PRs as “Merged” when their tip commit lands in the target branch. I haven't looked up docs for when this landed (if it was even announced), but next to GitHub's “Merge pull request” button there's currently an “or view command line instructions” link which sketches out some command-line Git to accomplish the same thing. But I agree that it's nice to keep the discussion around pull requests somewhere portable. Cheers, Trevor [1]: http://git-scm.com/docs/git-request-pull -- This email may be signed or encrypted with GnuPG (http://www.gnupg.org). For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy signature.asc Description: OpenPGP digital signature -- https://mail.python.org/mailman/listinfo/python-list
Async/Concurrent HTTP Requests
Hi, I'd like to query two (or more) RESTful APIs concurrently. What is the pythonic way of doing so? Is it better to use built in functions or are third-party packages? Thanks. Best, Ari -- https://mail.python.org/mailman/listinfo/python-list
Validating Data Extracted from Files
Hi, I'm sourcing data from multiple excel files (workbooks) each with multiple worksheets. Prior to persisting the aggregated data, I want to validate it. I was thinking of creating classes to hold the data and validate type and content via methods. I'd appreciate feedback on this approach, suggestions on possibly better alternatives, and if there are existing libraries that provide this functionality. Thanks. Best, Ari -- https://mail.python.org/mailman/listinfo/python-list
Re: parametized unittest
On Sat, Jan 11, 2014 at 08:00:05PM -0800, CraftyTech wrote: > I'm finding it hard to use unittest in a for loop. Perhaps something like: > > for val in range(25): > self.assertEqual(val,5,"not equal) > > The loop will break after the first failure. Anyone have a good > approach for this? please advise. If Python 3.4 is an option, you can stick to the standard library and use subtests [1]. Cheers, Trevor [1]: http://docs.python.org/3.4/library/unittest.html#distinguishing-test-iterations-using-subtests -- This email may be signed or encrypted with GnuPG (http://www.gnupg.org). For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy signature.asc Description: OpenPGP digital signature -- https://mail.python.org/mailman/listinfo/python-list
Re: [python] email 8bit encoding
On Sun, Jul 28, 2013 at 04:41:27PM -0700, ru...@yahoo.com wrote: > How, using Python-3.3's email module, do I "flatten" (I think > that's the right term) a Message object to get utf-8 encoded > body with the headers: > Content-Type: text/plain; charset=UTF-8 > Content-Transfer-Encoding: 8bit > when the message payload was set to a python (unicode) string? I asked about this a while back [1], but never got a response. My current best-guess is here [2]. My fallback flattening works for everything except the 8-bit encoded messages using the UTF-16 charset, but it's pretty ugly. Let me know if you figure out something cleaner :). Cheers, Trevor [1]: http://thread.gmane.org/gmane.comp.python.general/725425 [2]: https://github.com/wking/rss2email/blob/master/rss2email/email.py#L226 -- This email may be signed or encrypted with GnuPG (http://www.gnupg.org). For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy signature.asc Description: OpenPGP digital signature -- http://mail.python.org/mailman/listinfo/python-list
CC etiquette for mailing lists (was: Don't feed the troll...)
On Sun, Jun 16, 2013 at 12:51:04AM +, Steven D'Aprano wrote: > On Sun, 16 Jun 2013 09:09:37 +1000, Chris Angelico wrote: > > Mailman is the software that runs python-list@python.org, so this > > *is* applicable to everyone who reads the mailing list (including > > myself). The fact that there's other mailing list software isn't > > significant; > > I'm not making an argument about CCing the sender on specifically > this list, I'm making a general observations about list etiquette in > general. For some lists [1], the convention *is* to CC interested parties (which presumably includes the person who's message you are replying to). In the case of Mailman-hosted lists, it would seem to be more considerate to CC folks (since they can opt-out via Mailman), while folks who only follow the list via occasionally reading Gmane [2] will probably appreciate the direct ping. I'm not saying that this should be the convention for python-list@, I'm just saying that it's not immediately obvious what the conventions are for a particular list [3], and there are valid arguments going both ways. What seems more obvious (to me), is that it's polite to adjust the subject line if your response if far enough off-topic that the original subject no longer applies ;). Cheers, Trevor [1]: http://git.kernel.org/cgit/git/git.git/tree/MaintNotes?h=todo#n10 [2]: http://news.gmane.org/gmane.comp.python.general [3]: Unless you tell new subscribers what the conventions are. For example, the git@ list emails [1] to new list subscribers. -- This email may be signed or encrypted with GnuPG (http://www.gnupg.org). For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy signature.asc Description: OpenPGP digital signature -- http://mail.python.org/mailman/listinfo/python-list
Mailman forwarding (was: Don't feed the troll...)
On Sat, Jun 15, 2013 at 01:07:29PM -0400, D'Arcy J.M. Cain wrote: > On Sat, 15 Jun 2013 18:41:41 +0200 Chris “Kwpolska” Warrick wrote: > > On Sat, Jun 15, 2013 at 5:40 PM, Steven D'Aprano wrote: > > > In the name of all that's good and decent in the world, why on earth > > > would you do that when replying to a mailing list??? They're already > > > getting a reply. Sending them TWO identical replies is just rude. > > > > Mailman is intelligent enough not to send a second copy in that case. > > This message was sent with a CC, and you got only one copy. > > Actually, no. Mailman is not your MTA. It only gets the email sent to > the mailing list. Your MTA sends the other one directly so Steve is > correct. He gets two copies. If his client doesn't suppress the > duplicate then he will be presented with both. Mailman can (optionally) assume that addresses listed in To, CC, … fields received an out-of-band copies, and not mail them an additional copy [1]. Cheers, Trevor [1]: http://www.gnu.org/software/mailman/mailman-member/node21.html -- This email may be signed or encrypted with GnuPG (http://www.gnupg.org). For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy signature.asc Description: OpenPGP digital signature -- http://mail.python.org/mailman/listinfo/python-list
How to Nest Structs?
Hi,
I'm trying to nest the "info_header", "info_body", and "info_trailer" structs
(see below) into a "data_packet" struct. Does anyone know how I can/should
accomplish this? Thanks.
batch_header_format = struct.Struct('!c2h')
info_header_format = struct.Struct('!2hl')
mkt_status_format = struct.Struct('!c')
info_trailer_format = struct.Struct('!hc')
mkt_session_codes = [b'PO',b'PC',b'CO',b'CC',b'CK',b'CL']
mkt_type = [b'N',b'S',b'O',b'A',b'C',b'G']
batch_header = batch_header_format.pack(b'1',1,1024)
total_size = info_header_format.size + mkt_status_format.size +
info_trailer_format.size
# Combine following into data struct.
info_header =
info_header_format.pack(int(binascii.hexlify(mkt_session_codes[random.randint(0,5)])),
total_size, 124)
info_body = mkt_status_format.pack(mkt_type[random.randint(0,5)])
info_trailer = info_trailer_format.pack(0, b'\r')
-Ari
--
http://mail.python.org/mailman/listinfo/python-list
Flatten an email Message with a non-ASCII body using 8bit CTE
Hello list!
I'm trying to figure out how to flatten a MIMEText message to bytes
using an 8bit Content-Transfer-Encoding in Python 3.3. Here's what
I've tried so far:
# -*- encoding: utf-8 -*-
import email.encoders
from email.charset import Charset
from email.generator import BytesGenerator
from email.mime.text import MIMEText
import sys
body = 'Ζεύς'
encoding = 'utf-8'
charset = Charset(encoding)
charset.body_encoding = email.encoders.encode_7or8bit
message = MIMEText(body, 'plain', encoding)
del message['Content-Transfer-Encoding']
message.set_payload(body, charset)
try:
BytesGenerator(sys.stdout.buffer).flatten(message)
except UnicodeEncodeError as e:
print('error with string input:')
print(e)
message = MIMEText(body, 'plain', encoding)
del message['Content-Transfer-Encoding']
message.set_payload(body.encode(encoding), charset)
try:
BytesGenerator(sys.stdout.buffer).flatten(message)
except TypeError as e:
print('error with byte input:')
print(e)
The `del m[…]; m.set_payload()` bits work around #16324 [1] and should
be orthogonal to the encoding issues. It's possible that #12553 is
trying to address this issue [2,3], but that issue's comments are a
bit vague, so I'm not sure.
The problem with the string payload is that
email.generator.BytesGenerator.write is getting the Unicode string
payload unencoded and trying to encode it as ASCII. It may be
possible to work around this by encoding the payload so that anything
that doesn't encode (using the body charset) to a 7bit value is
replaced with a surrogate escape, but I'm not sure how to do that.
The problem with the byte payload is that _has_surrogates (used in
email.generator.Generator._handle_text and
BytesGenerator._handle_text) chokes on byte input:
TypeError: can't use a string pattern on a bytes-like object
For UTF-8, you can get away with:
message.as_string().encode(message.get_charset().get_output_charset())
because the headers are encoded into 7 bits, so re-encoding them with
UTF-8 is a no-op. However, if the body charset is UTF-16-LE or any
other encoding that remaps 7bit characters, this hack breaks down.
Thoughts?
Trevor
[1]: http://bugs.python.org/issue16324
[2]: http://bugs.python.org/issue12553
[3]: http://bugs.python.org/issue12552#msg140294
--
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy
signature.asc
Description: OpenPGP digital signature
--
http://mail.python.org/mailman/listinfo/python-list
problem python
hi im new in python and i have a problem about when i type python in my command line console i get an error message 'import site' failed; use -v for traceback Python 2.4.3 (#1, May 5 2011, 18:44:23) [GCC 4.1.2 20080704 (Red Hat 4.1.2-50)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> what is going on and how i can fix it thanks in andvance-- http://mail.python.org/mailman/listinfo/python-list
Re: tiny script has memory leak
On Friday, 11 May 2012 22:29:39 UTC+1, gry wrote: > sys.version --> '2.6 (r26:66714, Feb 21 2009, 02:16:04) \n[GCC 4.3.2 > [gcc-4_3-branch revision 141291]] > I thought this script would be very lean and fast, but with a large > value for n (like 15), it uses 26G of virtural memory, and things > start to crumble. > > #!/usr/bin/env python > '''write a file of random integers. args are: file-name how-many''' > import sys, random > > f = open(sys.argv[1], 'w') > n = int(sys.argv[2]) > for i in xrange(n): > print >>f, random.randint(0, sys.maxint) > f.close() > > What's using so much memory? > What would be a better way to do this? (aside from checking arg > values and types, I know...) Ran OK for me, python 2.4.1 on Windows 7 Iain -- http://mail.python.org/mailman/listinfo/python-list
Re: Python Gotcha's?
A common one used to be expecting .sort() to return, rather than mutate (as it does). Same with .reverse() - sorted and reversed have this covered, not sure how common a gotcha it is any more. Iain On Wednesday, 4 April 2012 23:34:20 UTC+1, Miki Tebeka wrote: > Greetings, > > I'm going to give a "Python Gotcha's" talk at work. > If you have an interesting/common "Gotcha" (warts/dark corners ...) please > share. > > (Note that I want over http://wiki.python.org/moin/PythonWarts already). > > Thanks, > -- > Miki -- http://mail.python.org/mailman/listinfo/python-list
Re: Refactor/Rewrite Perl code in Python
Steven D'Aprano writes:
> On Sun, Jul 24, 2011 at 7:29 PM, Shashwat Anand
> wrote:
>
>> How do I start ?
>> The idea is to rewrite module by module.
>> But how to make sure code doesn't break ?
>
> By testing it.
>
> Read up on "test driven development".
>
> At this point, you have this:
>
> Perl modules: A, B, C, D
> Python modules: none
> Python tests: none
>
> Now, before you rewrite each Perl module in Python, first write a good,
> comprehension test suite in Python for that module. You need to have tests
> for each function and method. Test that they do the right thing for both
> good data and bad data. If you have functional requirements for the Perl
> modules, use that as your reference, otherwise use the Perl code as the
> reference.
>
> For example, this might be a basic test suite for the len() built-in
> function:
>
>
> for empty in ([], {}, (), set([]), ""):
> if len(empty) != 0:
> raise AssertionError('empty object gives non-zero len')
>
> for n in range(1, 5):
> if len("x"*n) != n:
> raise AssertionError('failure for string')
> for kind in (list, tuple, set):
> obj = kind([None]*n)
> if len(obj) != n:
> raise AssertionError('failure for %s' % obj)
>
> if len({'a': 1, 'b': None, 42: 'spam'}) != 3:
> raise AssertionError('failure for dict')
>
> for bad_obj in (23, None, object(), 165.0, True):
> try:
> len(bad_obj)
> except TypeError:
> # Test passes!
> pass
> else:
> # No exception means len() fails!
> raise AssertionError('failed test')
>
>
>
> Multiply that by *every* function and method in the module, and you have a
> moderately good test suite for module A.
>
> (You may want to learn about the unittest module, which will help.)
>
> Now you have:
>
> Perl modules: A, B, C, D
> Python modules: none
> Python tests: test_A
>
> Now re-write the Python module, and test it against the test suite. If it
> fails, fix the failures. Repeat until it passes, and you have:
>
> Perl modules: A, B, C, D
> Python modules: A
> Python tests: test_A
>
> Now you can be confident that Python A does everything that Perl A does.
> Possibly *better* than the Perl module, since if you don't have a test
> suite for it, it probably has many hidden bugs.
>
> Continue in this way with the rest of the modules. At the end, you will have
> a full test suite against the entire collection of modules.
A very sane approach.
I currently support a large legacy application written in C++ by
extending it with Python. We managed to do this by wrapping the legacy
application with a Python interface using the Boost::Python libraries.
It has allowed us to embrace and extend the legacy code and replace bits
of it one piece at a time while keeping the whole application running.
Now I am not aware of any Python bindings to the Perl interpreter, but
if there is some FFI library that makes it possible this approach might
be viable for your project.
The trade-off of this approach is the extra complexity of maintaining
another interface in your code. The beneift is that you can avoid
re-writing a large amount of code up front. This works particularly
well when the legacy system has a rather large user base and you don't
have a robust test suite for the legacy code.
One book I found particularly useful:
http://www.amazon.ca/Working-Effectively-Legacy-Michael-Feathers/dp/0131177052
hth
--
http://mail.python.org/mailman/listinfo/python-list
Re: Functional style programming in python: what will you talk about if you have an hour on this topic?
Anthony Kong writes:
> (My post did not appear in the mailing list, so this is my second try.
> Apology if it ends up posted twice)
>
> Hi, all,
>
> If you have read my previous posts to the group, you probably have some idea
> why I asked this question.
>
> I am giving a few presentations on python to my colleagues who are mainly
> java developers and starting to pick up python at work.
>
>
> So I have picked this topic for one of my presentation. It is because
> functional programming technique is one of my favorite in my bag of python
> trick. It also takes me to the rabbit hole of the functional programming
> world, which is vastly more interesting than the conventional procedural/OO
> languages.
>
>
> I think I will go through the following items:
>
> itertools module
> functools module
> concept of currying ('partial')
>
>
> I would therefore want to ask your input e.g.
>
> Is there any good example to illustrate the concept?
> What is the most important features you think I should cover?
> What will happen if you overdo it?
>
>
> Cheers
You might also want to cover gotchas like Python's references.
If you have the time I'd introduce weakref too.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Code Review
On May 25, 2:44 pm, ad wrote: > On May 25, 4:06 am, Ulrich Eckhardt > wrote: > > > > > ad wrote: > > > Please review the code pasted below. I am wondering what other ways > > > there are of performing the same tasks. > > > On a unix system, you would call "find" with according arguments and then > > handle the found files with "-exec rm ..." or something like that, but I see > > you are on MS Windows. > > > > args = parser.parse_args() > > > > dictKeys = (vars(args)) > > > The first of these looks okay, while I don't get the additional brackets in > > the second one. Another habit I observe here is the Hungarian notation of > > prefixing the type to the name and using camelCaps. PEP 8 (IIRC) has > > something to say on the preferred naming. I'm not 100% against encoding the > > type in the variable name in Python, since it lacks static type checking, I > > would have chosen "key_dict" here though, or, due to the small size of the > > overall program just "keys". > > > > print (HowManyDays) > > > This puzzled me at first, again the useless additional brackets I thought. > > However, in Python 3, "print" is a function, so that is correct. Still, it > > should be "print(foo)" not "print (foo)". > > > > for files in DirListing: > > > > # Get the absolute path of the file name > > > abspath = (os.path.join(WhatDirectory, files)) > > > "files" is just the name of a single file, right? In that case the name is a > > bit confusing. > > > > # Get the date from seven days ago > > > WeekOldFileDate = CurrentTime - DaysToDelete > > > You are repeating this calculation for every file in the loop. > > > > if FileCreationTime < WeekOldFileDate: > > > #check if the object is a file > > > if os.path.isfile(abspath): os.remove(abspath) > > > # It is not a file it is a directory > > > elif os.path.isdir(abspath): shutil.rmtree(abspath) > > > I'm not sure, but I believe you could use shutil.rmtree() for both files and > > directories. In any case, be prepared for the file still being open or > > otherwise read-only, i.e. for having to handle errors. > > > Also, what if a directory is old but the content is new? Would this cause > > the non-old content to be deleted? > > > Cheers! > > > Uli > > > -- > > Domino Laser GmbH > > Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932 > > Thank you guys very much for the excellent points. I will use this > information as a reference as I write more code and fix up the > existing script. > > Chris, thank you for putting so much time into your post! > > Until we type again... Wrote something to do the same basic thing a little while ago. Less verbose than yours, and it only does files, not folders. If I was going to do folders though, I'd do them by recursing into them and pruning files, and not go by the folder modify date, which I don't think changes the way you think it changes (for example, if a file inside a folder got updated such that it shouln't be deleted it still will be with your code if the folder modify date is old [this is on Windows]) import os, glob, time, sys if len(sys.argv) < 3: print "USAGE: %s " % sys.argv[0] sys.exit(1) pattern = sys.argv[1] days = int(sys.argv[2]) threshold = days * 24 * 60 * 60 t = time.time() for f in glob.glob(pattern): if t - os.stat(f)[9] > threshold: print f os.remove(f) -- http://mail.python.org/mailman/listinfo/python-list
Re: use of index (beginner's question)
On Apr 28, 2:45 am, Chris Angelico wrote: > Incidentally, you're allowed to put the comma on the last item too: > > lists = [ > ['pig', 'horse', 'moose'], > ['62327', '49123', '79115'], > ] > > Often makes for easier maintenance, especially when you append > array/list elements. > > Chris Angelico I did not know this. Very useful! Iain -- http://mail.python.org/mailman/listinfo/python-list
Re: Using nested lists and tables
On Oct 28, 2:35 pm, Iain King wrote:
...
> (a) I don't know if the order of resolution is predicated left-to-
> right in the language spec of if it's an implementation detail
> (b) columns[-1].startswith('s') would be better
>
...
Ignore (b), I didn't read the original message properly.
Iain
--
http://mail.python.org/mailman/listinfo/python-list
Re: Using nested lists and tables
On Oct 28, 2:19 pm, Zeynel wrote:
> On Oct 28, 4:49 am, Peter Otten <__pete...@web.de> wrote:
>
> Thank you this is great; but I don't know how to modify this code so
> that when the user types the string 's' on the form in the app he sees
> what he is typing. So, this will be in GAE. But I have a couple of
> other questions, for learning purposes. Thanks again, for the help.
>
> > ... if columns and columns[-1][0] == s:
>
> Here, how do you compare "columns" (a list?) and columns[-1][0] (an
> item in a list)?
>
It's equivalent to:
if columns:
if columns[-1][0] == s:
dostuff()
i.e. check columns is not empty and then check if the last item
startswith 's'.
(a) I don't know if the order of resolution is predicated left-to-
right in the language spec of if it's an implementation detail
(b) columns[-1].startswith('s') would be better
Iain
--
http://mail.python.org/mailman/listinfo/python-list
relative imports and sub-module execution
Hi,
After reading couple of docs and articles, I have implemented a simple
test package with nested modules.
When running "main.py", everything is working fine. Some of my sub-
modules has some small test routines for debug purpose.
It's because I am using relative package imports at the top, I am not
able to run these sub modules individually.
For example, this works when running from main.py
Here is example structure
main.py
__init__.py
core/
__init__.py
folder1
__init__.py
f1a.py
f1b.py
folder2
__init__.py
f2a.py
f2b.py
in folder2/f2b.py, I am importing
from core.folder1 import f1a
print f1a.myvar
Now I can't execute 'f2b.py' individually. It's giving an error:
ImportError: No module named core.folder2
Is this mean when you have created a package where modules are using
relative imports, they can't execute individually?
Another solution that I have discovered is using sys.path. Check this:
import sys
import os
sys.path.append(os.path.abspath("../../"))
from core.folder1 import f1a
print f1a.myvar
This works fine and easier too. I can execute modules individually as
well as package is also working. Are there any disadvantages when
using above technique? Couple of articles suggests that this is a bad
practice and should not be used.
Need some clarifications.
Thanks
Prashant
--
http://mail.python.org/mailman/listinfo/python-list
Re: WMI in Python
On Sep 16, 12:39 pm, alex23 wrote: > Lawrence D'Oliveiro wrote: > > Why not just call Scriptomatic directly from within the Python script, then? > > Because Scriptomatic _generates scripts to access WMI_, that's what it > _does_. Are you _seriously_ advocating writing Python code to fire up > a Windows application, programmatically manipulating a GUI to generate > more Python code for your original script to import or exec? > > For your question to make any real sense, you'd want to ask "Why not > access WMI directly...", which you might notice was the first > recommendation I made. > > > Machine-generated code has no place in a source file to be maintained by a > > human. > > As you've made your disdain for Windows _perfectly_ clear through your > incessant sniping on this list, I think I'm pretty safe in saying you > have no idea of the quality of Scriptomatic's output. I don't > understand what possibly makes you believe you're qualified to make > such a statement. Thank you all looks I got lot of reading to be done now based on all your suggestions. -- http://mail.python.org/mailman/listinfo/python-list
Re: WMI in Python
On Sep 14, 10:39 am, KING LABS wrote: > On Sep 13, 8:31 pm, Jerry Hill wrote: > > > On Mon, Sep 13, 2010 at 8:45 AM, KING LABS wrote: > > > Hi All, > > > > I am new to programming and python, Being a system administrator I > > > have chose Inventory (Software & Hardware ) as my first project. > > > You'll probably want to look at the python WMI > > module:http://timgolden.me.uk/python/wmi/index.html > > > as well as the pywin32 module:http://sourceforge.net/projects/pywin32/ > > > IIRC, there's been quite a bit of discussion about inventorying > > installed software on the pywin32 mailing > > list:http://mail.python.org/mailman/listinfo/python-win32 > > > -- > > Jerry > > Thank you all, I will go through the links provided and suggestions. > Shall get back to you on this soon. The following information is exactly what I am trying to collect for the inventory. I can find vb scripts with googling. I want to do the same with Python & Win32. Use Server/Client architecture . Client(agent) updates the information to server. BIOS: System serial number, manufacturer, and model Bios manufacturer, version, and date Processors: Type, count (how many of them), manufacturer, speed, and cache Memory: Physical memory type, manufacturer, capacity, and slot number Total physical memory Total swap/paging memory Video: Video adapter: Chipset/model, manufacturer, memory size, speed, and screen resolution Display monitor: Manufacturer, description, refresh rate, type, serial number, and caption Storage/removable devices: Manufacturer, model, size, type, speed( all when applicable) Drive letter, filesystem type, partition/volume size, free space Network adapters/telephony: Manufacturer, model, type, speed, and description MAC and IP address, mask and IP gateway, DHCP server used Miscellaneous hardware: Input devices: Keyboard, mouse, and pointing device Sound devices: Manufacturer name, type, and description System slots: Name, type, and designation System ports: Type, name, caption, and description Software Information: ( from registry & add/remove program ) Operating system: Name, version, comments, and registration info Installed software: Name, publisher, version (from Add / Remove software or Programs and Features menu) Custom-specified registry queries (applicable to Windows OS) -- -- http://mail.python.org/mailman/listinfo/python-list
Re: WMI in Python
On Sep 13, 8:31 pm, Jerry Hill wrote: > On Mon, Sep 13, 2010 at 8:45 AM, KING LABS wrote: > > Hi All, > > > I am new to programming and python, Being a system administrator I > > have chose Inventory (Software & Hardware ) as my first project. > > You'll probably want to look at the python WMI > module:http://timgolden.me.uk/python/wmi/index.html > > as well as the pywin32 module:http://sourceforge.net/projects/pywin32/ > > IIRC, there's been quite a bit of discussion about inventorying > installed software on the pywin32 mailing > list:http://mail.python.org/mailman/listinfo/python-win32 > > -- > Jerry Thank you all, I will go through the links provided and suggestions. Shall get back to you on this soon. -- http://mail.python.org/mailman/listinfo/python-list
Re: WMI in Python
On Sep 13, 6:42 pm, Rodrick Brown wrote: > The easiest way to do this is to use the native OS tools readily available to > do the collection and log to a central location or if possible shared > location accessible by all systems, once you have all the data you want to > feed into your RDBMS you could easily parse these logs using python and the > native db access module. > > I hope this give you a pointer. > > Sent from my iPhone 4. > > On Sep 13, 2010, at 8:45 AM, KING LABS wrote: > > > > > Hi All, > > > I am new to programming and python, Being a system administrator I > > have chose Inventory (Software & Hardware ) as my first project. > > > I would like to know experts advice on the best way to build the same > > using python. I would like to this tool to evolve into full fledge > > application. > > > I would like to collect the complete information of system hardware & > > and also software installed from registry and add/remove program and > > feed this data into database. > > > Regards, > > KINGLABS > > -- > >http://mail.python.org/mailman/listinfo/python-list I am trying to learn Python programming. Since I need a custom inventory management tool for my work place. I am considering it as a project in the process of learning Python. I am not looking for easiest way of doing things. I am considering using Python . Also I would need to build a setup of the tool for easy installation. Hope I am clear this time Regards, KINGLABS -- http://mail.python.org/mailman/listinfo/python-list
WMI in Python
Hi All, I am new to programming and python, Being a system administrator I have chose Inventory (Software & Hardware ) as my first project. I would like to know experts advice on the best way to build the same using python. I would like to this tool to evolve into full fledge application. I would like to collect the complete information of system hardware & and also software installed from registry and add/remove program and feed this data into database. Regards, KINGLABS -- http://mail.python.org/mailman/listinfo/python-list
Re: python interview quuestions
James Mills writes: > On Sat, Aug 7, 2010 at 4:32 AM, Tim Chase > wrote: >>> I would like to aquint myself with Python Interview questions >> >> This came up a while ago: >> >> http://www.mail-archive.com/python-list@python.org/msg168961.html >> >> Most of that thread is still relevant (perhaps throw in some py3l questions >> too) > > A common thing you can do in interviews is ask > your interviewee to write (in Python) a solution > to the "FizzBuzz" problem. Any good competent > Python programmer should be able to do this > in 5-10mins (5 if you're good). > > cheers > james Fizzbuzz is annoying in interviews. I've never worked at a job where I was under a timer while a group of people sat across from me and scrutinized everything I was doing. I don't see how it can honestly tell you anything useful about the person you're interviewing either. Do you really think that what you assume about the interviewee based on characteristics you can infer from their solution to be really, honestly true? They might even completely bomb the solution and still be a brilliant programmer, but you'll never know that if you trust this simple "fizzbuzz" test. I've been in those interviews on both sides of the table. Neither side was a good experience. If a test is necessary, make it a take-home or demand source code if they have it. Read their code and judge for yourself the quality of their work. Any questions in an interview should be the usual "get to know you" type stuff. "What was the most difficult challenge you've faced on the job? How did you respond?" That sort of thing. -- http://mail.python.org/mailman/listinfo/python-list
Undo-Redo, copy instance, custom events and a problem
Hi, I am developing an app using wxPython. The Undo-Redo implementation is based on storing pre & post state of an attribute. You store the instance before changing the value and store the instance after changing the values. While undoing or redoing, you copy/replace the current state with stored once. For color attribute as soon as you choose a color, here is the code: # Custom Event evt = ValueChangeEvent(EVT_COLOR_CHANGED.typeId, self.GetId()) # Store Pre State evt.SetPreState(copy.copy(self.attribute)) # Change the newly selected color self.attribute.setValue(R,G,B) # Store Post State evt.SetPostState(copy.copy(self.attribute)) # Throw Custom Event self.GetEventHandler().ProcessEvent(evt) Both states are copied as new instance with correct values. The problem is when this event is getting fired. evt.GetPreState().GetValue() is showing the post color value. Although GetPreState() & GetPostState() are showing two different instances but I have no idea why values are not coming/stored correctly. Some where in between is messed up and post-state values are copied in pre-state. On a side note, self.attribute.setValue takes three floating values for R,G & B colors but stored them as a text. Similarly self.attribute.getValue() converts text into float and returns. Is there anything related to scope or something? Cheers Prashant -- http://mail.python.org/mailman/listinfo/python-list
Re: Compile python executable only for package deployment on Linux
Hi Stefan,
Well, the idea is similar to package tools like pyinstaller or
cx_freeze. There approach is slightly different then what I intend to
do here.
You have to pass the name of the script to python executable("python
main.py") in order to execute it. What I mean here is to create python
executable using python executable source code only(not compilation of
entire python using source code) and insert "main.py" contents in
source code(hardcoded) and produce a new executable.
When you run that executable, it should run the "main.py", which was
hard coded while building using fixed string or any other method. The
executable is actually a modified version of python executable. It
won't take any argument because we'll rip that piece of code out.
Hope this will clear out.
Cheers
prashant
--
http://mail.python.org/mailman/listinfo/python-list
Compile python executable only for package deployment on Linux
Hi,
I have created a simple tool(python script) that creates a self
sufficient package ready for deployment. Current implementation is
based on shell scripting to set environment for the app and finally
execute "python main.py".
I am planning to convert "main.py" into an executable. The plan is to
rip the unnecessary code from source code that produce python
executable such as command line arguments etc, use "main.py" as python
string (hardcoded inside executable source) and execute it using
"exec" or similar methods and finally creates executable.
Am I right here? Is this is the correct approach?
For example a simple script:
import os
import math
print math.sin(23.0)
print os.getenv("PATH")
Once I'll convert above script as i have mentioned above in an
executable say: "myapp", executing "myapp" will print messages on
console.
Cheers
Prashant
--
http://mail.python.org/mailman/listinfo/python-list
Node based architecture
Hi, I am planning to build a generic node based framework using python. I would start with a simple image editing application. I hope that experienced users understands what am I trying to say here. In simple words: LoaderNode : Load Image from disk OperatorNode : Performs a specific task on data and spit output(data) OutputNode : Get the data from OperatorNode(output) and writes image to disk. Graph : A collection Nodes that are inter connected. The question is how to process graph? Here is a simple representation: class Integer(object): """ A simple integer node. """ def __init__(self, value=0): self.value = value def output(self): return self.value class OperatorAdd(object): """ A simple operator node """ def __init__(self, integerInst1=None, integerInst2=None): self.a = integerInst1.output() self.b = integerInst2.output() def compute(self): return Integer(self.a + self.b) def output(self): return self.compute() "Integer" is data node and "OperatorAdd" is a compute node. Imagine I have created two integer nodes and their output is going to "OperatorNode". One way to solve is to represent every node using a string and then you can write these series of string which actually is a python code that you can execute and get the result. Example: compute = """ i1 = Integer(2) i2 = Integer(3) i3 = OperatorAdd(i1, i2).output() """ Now you can execute "compute" variable using exec(compute) or something like that. This would do the job but I would like to know the opinion of more experienced users. One advantage of above mentioned string method is that it can be saved to disk and later you can load/compute it again. Cheers Prashant -- http://mail.python.org/mailman/listinfo/python-list
Re: zipimport (.pyd & .so) files.
I think I am trying to open a can of worms. It's better to leave the idea for now. Prashant -- http://mail.python.org/mailman/listinfo/python-list
zipimport (.pyd & .so) files.
Hi, The 'zipimport' modules can only import (.py & .pyc) files from a zip file and doesn't support importing .pyd & .so files. Recently I was examining the code of Py2Exe (python package deployment tool) and I have found that it is using a module 'zipextimporter' that can import dlls(.pyd) modules from a zip file. It is based on the concept of loading library form memory. You can find out more about here: http://www.joachim-bauch.de/tutorials/loading-a-dll-from-memory/ It's strictly for windows platform. I would like to know from expert python users and linux programmers, how we can achieve similar functionality on linux platform? I do have limited c/c++ skill sets but I would love to give a try. Cheers Prashant -- http://mail.python.org/mailman/listinfo/python-list
Re: simple python deployment tool
On Jul 8, 2:21 pm, Alexander Kapps wrote: > King wrote: > > Hi, > > > I am writing a python package deployment tool for linux based > > platforms. I have tried various existing > > tool sets but none of them is up to the mark and they have their own > > issues. Initially I'll start with simple approach. > > I'm sorry, but your approach is not going to work. The Right Way(tm) > is to not ship any external libraries, but let the user install the > dependencies with the distro's package manager. And to not try to > build custom packages, but to let the package maintainers of the > different distros package your application for you. Yes, you an do this by creating a .deb package that will take care of installing the required libraries. It may possible that either required version is not available in the distro's repository or the one available is older one then required. So best deal would be to ship at least all the python's libs along with your package. > > 1. Find all the modules/packages and copy to "lib" directory. > > 2. Find python's *.so dependencies(system libs) and copy them to > > "syslib" > > That would include all the way down to libc, your archive is going > to be huge, but the actual problem is, what if the libc isn't > compatible with the kernel, what if the WX, GTK, X11, etc libraries > aren't compatible with the running Xserver? You are right here. It seems there is no standard definition of system libraries on linux. To over come the problem of binary compatibility, I am using ubuntu (hardy) which is on glibc 2.7. Every other external python library including python is compiled on hardy. This is would give you maximum chances that libs would be compatible in case if you run on any other distro which is using glibc >=2.7 > End of the story is, you would need to package a minimal (but almost > complete) Linux system into your package, which of course is not > want you want. > > > 3. Copy python(executable) and libpython2.6.so.1.0 into "dist" > > directory. > > 4. Set up temp environment > > 5. Run main script using "python .py" > > > The idea is to produce a cleaner directory structure. Neither I am > > creating a boot loader nor I am converting main script > > file into executable. It's plain vanilla stuff. > > It's not vanilla stuff, but a very hard problem. In fact you are > fighting against the whole system structure. Ok, forget about system libs(libX*.* etc.), I don't know why it sounds too complicated. I am hoping it should work and eventually it's easier then tools that create an executable using bootloader. The problem is in setting the correct python environment. Prashant -- http://mail.python.org/mailman/listinfo/python-list
simple python deployment tool
Hi, I am writing a python package deployment tool for linux based platforms. I have tried various existing tool sets but none of them is up to the mark and they have their own issues. Initially I'll start with simple approach. 1. Find all the modules/packages and copy to "lib" directory. 2. Find python's *.so dependencies(system libs) and copy them to "syslib" 3. Copy python(executable) and libpython2.6.so.1.0 into "dist" directory. 4. Set up temp environment 5. Run main script using "python .py" The idea is to produce a cleaner directory structure. Neither I am creating a boot loader nor I am converting main script file into executable. It's plain vanilla stuff. Using above steps I have pulled down a package using wxPython's demo example "pySketch.py". You can download the archive from here: http://rapidshare.com/files/405255400/dist.zip (Note : It's for linux users.) After extracting the contents, run the executable file using "./run". This file sets temporary environment variables and execute "pySketch.py". I apologize for the archive size as all the systems libs are also included. This is eventually not working. I think I am not able to set up temporary environment properly or may be missing some other stuff. I would appreciate if some one can point out mistakes. Here is the practical approach: /lib (all python libs, files and third party modules.) /syslib (all the system libs. libX*.*, cairo, etc.) python (executable) pySketch.py run.sh Contents of "run.sh". -- set LD_LIBRARY_PATH="syslib/" set PYTHONPATH="/lib" python pySketch.py Cheers Prashant -- http://mail.python.org/mailman/listinfo/python-list
executable builder
Hi, I am trying to build python a cross platform python executable builder to deploy python app. I tried various tools such as py2exe, pyinstaller, cx_freeze but some how they are not upto the mark except py2exe. Unfortunately py2exe is working only on windows based systems. The task is divide into 3 steps: 1. Collect all the modules(.pyo, .pyd, .dll, etc) required for your project. 2. set up temporary environment for python 3. run main.py using :python.exe main.py Initially I won't be creating an executable using main.py. I would be using shell scripting to execute the main.py. Q1. Which is most efficient way to fine all the modules required by your "main.py". Q2. For example, I have copied all the modules in "d:\project\lib\*.*" python.exe, python.dll, ms*.dll, main.py are copied to "d: \project". Before executing "python.exe main.py", I have to set up an environment var and point it to "d:\project\lib", so that python.exe can find the path for searching/loading modules. How do I do that? Q3. How do you convert your "main.py" into an executable? Prashant -- http://mail.python.org/mailman/listinfo/python-list
Re: List of lists surprising behaviour
candide writes: > Let's the following code : > t=[[0]*2]*3 t > [[0, 0], [0, 0], [0, 0]] t[0][0]=1 t > [[1, 0], [1, 0], [1, 0]] > > Rather surprising, isn't it ? Not at all, actually. I'd be surprised if the multiplication operator was aware of object constructors. Even arrays are "objects" in Python. Should the multiplication operator know how to instantiate three arrays from a single array instance? What about an instance of a user-defined class? > So I suppose all the subarrays reférence > the same array : > id(t[0]), id(t[1]), id(t[2]) > (3077445996L, 3077445996L, 3077445996L) > As they should. > > So what is the right way to initialize to 0 a 2D array ? Is that way > correct : > > t=[[0 for _ in range(2)] for _ in range(3)] > > It seems there is no more trouble now : > t > [[0, 0], [0, 0], [0, 0]] t[0][0]=1 t > [[1, 0], [0, 0], [0, 0]] > > Correct ? >>> 2d_zero_vector = lambda len: [[0, 0] for _ in range(len)] >>> t = 2d_zero_vector(3) >>> print t [[0, 0], [0, 0], [0, 0]] >>> t[0][0] = 1 >>> print t [[1, 0], [0, 0], [0, 0], [0, 0]] (Of course, if you're doing matrix math you'll probably want to work with numpy which has a function for doing just this) -- http://mail.python.org/mailman/listinfo/python-list
Re: GUIs - A Modest Proposal
> My concern is simple: I think that Python is doomed to remain a minor > language unless we crack this problem. But making *another* "one true GUI library" just fragments it further. Nobody designs a GUI library intending it to suck. -- http://mail.python.org/mailman/listinfo/python-list
Re: replacing words in HTML file
On Apr 29, 10:38 am, Daniel Fetchinson wrote: > > | > Any idea how I can replace words in a html file? Meaning only the > > | > content will get replace while the html tags, javascript, & css are > > | > remain untouch. > > | > > | I'm not sure what you tried and what you haven't but as a first trial > > | you might want to > > | > > | > > | > > | f = open( 'new.html', 'w' ) > > | f.write( open( 'index.html' ).read( ).replace( 'replace-this', 'with-that' > > ) ) > > | f.close( ) > > | > > | > > > If 'replace-this' occurs inside the javascript etc or happens to be an > > HTML tag name, it will get mangled. The OP didn't want that. > > Correct, that is why I started with "I'm not sure what you tried and > what you haven't but as a first trial you might". For instance if the > OP wants to replace words which he knows are not in javascript and/or > css and he knows that these words are also not in html attribute > names/values, etc, etc, then the above approach would work, in which > case BeautifulSoup is a gigantic overkill. The OP needs to specify > more clearly what he wants, before really useful advice can be given. > > Cheers, > Daniel > Funny, everyone else understood what the OP meant, and useful advice was given. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python compiled modules are too big in size (even after strip)
Hi Jon, I do have a limited skill sets in c/c++ and also new on linux. I think I am missing some flags or anything when I am compiling python from sources. Still hoping that some one point me out the missing link. Cheers Prashant -- http://mail.python.org/mailman/listinfo/python-list
Python compiled modules are too big in size (even after strip)
Hi, I have just compiled python 2.6.5 from sources on ubuntu "hardy" 8.04. I have used a simple script to do everything in one go: ./configure --enable-shared make make install Python is compiled and installed successfully. However the modules(_socket.so, _random.so etc) are two big in terms of file size. They are around 4.5-5.0 mb each. I have used "strip strip-all *.so", but still size is around 1.5 mb each. This is still too big compare to size of modules on ubuntu karmic's pre installed python 2.6.5. Am I missing something here? Do I have to re-configure or re-compile the python again from sources? Regards Prashant -- http://mail.python.org/mailman/listinfo/python-list
Re: Code redundancy
On Apr 20, 2:43 pm, Alan Harris-Reid
wrote:
> Hi,
>
> During my Python (3.1) programming I often find myself having to repeat
> code such as...
>
> class1.attr1 = 1
> class1.attr2 = 2
> class1.attr3 = 3
> class1.attr4 = 4
> etc.
>
> Is there any way to achieve the same result without having to repeat the
> class1 prefix? Before Python my previous main language was Visual
> Foxpro, which had the syntax...
>
> with class1
> .attr1 = 1
> .attr2 = 2
> .attr3 = 3
> .attr4 = 4
> etc.
> endwith
>
> Is there any equivalent to this in Python?
>
> Any help would be appreciated.
>
> Alan Harris-Reid
The pythonic equivalent of VB 'with' is to assign to a short variable
name, for example '_':
_ = class1
_.attr1 = 1
_.attr2 = 2
_.attr3 = 3
_.attr4 = 4
alternatively, you could use the __setattr__ method:
for attr, value in (
('attr1', 1),
('attr2', 2),
('attr3', 3),
('attr4', 4)):
class1.__setattr__(attr, value)
and to get a bit crunchy, with this your specific example can be
written:
for i in xrange(1, 5):
class1.__setattr__('attr%d' % i, i)
Iain
--
http://mail.python.org/mailman/listinfo/python-list
Re: Usable street address parser in Python?
On Apr 20, 8:24 am, John Yeung wrote: > My response is similar to John Roth's. It's mainly just sympathy. ;) > > I deal with addresses a lot, and I know that a really good parser is > both rare/expensive to find and difficult to write yourself. We have > commercial, USPS-certified products where I work, and even with those > I've written a good deal of pre-processing and post-processing code, > consisting almost entirely of very silly-looking fixes for special > cases. > > I don't have any experience whatsoever with pyparsing, but I will say > I agree that you should try to get the street type from the end of the > line. Just be aware that it can be valid to leave off the street type > completely. And of course it's a plus if you can handle suites that > are on the same line as the street (which is where the USPS prefers > them to be). > > I would take the approach which John R. seems to be suggesting, which > is to tokenize and then write a whole bunch of very hairy, special- > case-laden logic. ;) I'm almost positive this is what all the > commercial packages are doing, and I have a tough time imagining what > else you could do. Addresses inherently have a high degree of > irregularity. > > Good luck! > > John Y. Not sure on the volume of addresses you're working with, but as an alternative you could try grabbing the zip code, looking up all addresses in that zip code, and then finding whatever one of those address strings most closely resembles your address string (smallest Levenshtein distance?). Iain -- http://mail.python.org/mailman/listinfo/python-list
Re: Can someone please make it more pythonic or better?
Oltmans writes:
> Greetings Python superstars,
>
> I've a directory structure like following
>
> tests /
> __init__.py
> testfile.py
>
> testfile.py contains following code
>
> import unittest
>
> class Calculator(unittest.TestCase):
> def test_add(self):
> print 'just add'
> def test_divide(self):
> print 'diviide'
> def test_multiply(self):
> print 'mul'
>
>
> class Car(unittest.TestCase):
> def test_start(self):
> print 'start'
> def test_move_right(self):
> print 'move right'
> def test_move_left(self):
> print 'move left'
> def test_stop(self):
> print 'stop'
>
>
> Now give the following user-input I want to get all test-names.
> user-input = tests.testfile (get all test-names from all
> unittest.TestCase derived classes in test.testfile)
> user-input = tests.testfile.Car (get all test-names from the Car
> class)
> user-input = tests.testfile.Cacr.test_stop
>
> and I'm doing it this the following way and I really think there has
> to be more readable, more pythonic and more possibly short way to do
> it
>
> import unittest
> import sys
> import inspect
>
> def get_test_names(full_name,module):
> name = full_name.split('.')
> loader = unittest.TestLoader()
> if len(name) == 4:
> return full_name
> elif len(name) == 3:
> exec "from %s.%s import %s" %(module,name[1],name[2])
> return loader.getTestCaseNames(eval(name[2]))
> elif len(name) == 2:
> exec 'from %s import %s' % (module,name[1])
> tests = []
> for _name, obj in inspect.getmembers(sys.modules[full_name]):
> if inspect.isclass(obj) and
> issubclass(obj,unittest.TestCase):
> exec "from %s.%s import %s" %
> (module,name[1],obj.__name__)
> tests.append(loader.getTestCaseNames(obj))
> return tests
>
>
>
> if __name__ == "__main__":
> input = "tests.testfile"
> module = input.split('.')[0]
> _tests = get_test_names(input,module)
> print _tests
>
>
> So guys, can you kindly point me to a more Pythonic, more readable and
> possible more short way to acheive this? I will really appreciate any
> help. Many thanks in advance.
First of all, exec is bad if it's going to be processing user input.
You might want to:
>> help(__import__)
It will give you an idea on how to hook into python's import machinery
for tasks such as this.
You could also modify the function's arglist to remove the
string-splitting you're doing. Those "magic numbers" stick out a bit.
One can understand what they're for after reading the code in this case,
but it's not quite necessary if you make a keyword argument for package
names you can pass into the 'fromlist' argument in __import__.
ie:
def get_test_names(module_name, packagelist=[]):
...
hth,
j_king
>
> Best regards,
> Oltmans
--
http://mail.python.org/mailman/listinfo/python-list
Re: why (1, 2, 3) > [1, 2, 3] is true?
On Feb 25, 2:03 pm, fat bold cyclop wrote: > > Both are not equal, so the comparison returns an arbitrary result in Py2. > > Thanks, Stefan. If I understand you correctly the comparison is not > valid. > But I wonder if there is any logic behind this (in 2.x). > Is it possible to predict result of this comparison? > > Thanks again, > fbc I haven't looked in the source to check (and I'm almost 100% certain that tuple > list is an implementation detail), but I have not found any pair of tuple and list in which the list is treated as the greater. Possibly related: type(tuple()) is > type(list()). Or, to let the interpreter tell you why (1,2,3) > [1,2,3]: >>> tuple > list True Iain -- http://mail.python.org/mailman/listinfo/python-list
Re: Overcoming python performance penalty for multicore CPU
Paul Rubin writes: > Stefan Behnel writes: >> Well, if multi-core performance is so important here, then there's a pretty >> simple thing the OP can do: switch to lxml. >> >> http://blog.ianbicking.org/2008/03/30/python-html-parser-performance/ > > Well, lxml is uses libxml2, a fast XML parser written in C, but AFAIK it > only works on well-formed XML. The point of Beautiful Soup is that it > works on all kinds of garbage hand-written legacy HTML with mismatched > tags and other sorts of errors. Beautiful Soup is slower because it's > full of special cases and hacks for that reason, and it is written in > Python. Writing something that complex in C to handle so much > potentially malicious input would be quite a lot of work to write at > all, and very difficult to ensure was really safe. Look at the many > browser vulnerabilities we've seen over the years due to that sort of > problem, for example. But, for web crawling, you really do need to > handle the messy and wrong HTML properly. If the difference is great enough, you might get a benefit from analyzing all pages with lxml and throwing invalid pages into a bucket for later processing with BeautifulSoup. -- http://mail.python.org/mailman/listinfo/python-list
Re: Ad hoc lists vs ad hoc tuples
On Jan 27, 10:20 am, Floris Bruynooghe
wrote:
> One thing I ofter wonder is which is better when you just need a
> throwaway sequence: a list or a tuple? E.g.:
>
> if foo in ['some', 'random', 'strings']:
> ...
> if [bool1, bool2, boo3].count(True) != 1:
> ...
>
> (The last one only works with tuples since python 2.6)
>
> Is a list or tuple better or more efficient in these situations?
>
> Regards
> Floris
>
> PS: This is inspired by some of the space-efficiency comments from the
> list.pop(0) discussion.
I tend to use tuples unless using a list makes it easier to read. For
example:
if foo in ('some', 'random', 'strings'):
draw.text((10,30), "WHICH IS WHITE", font=font)
draw.line([(70,25), (85,25), (105,45)])
I've no idea what the performance difference is; I've always assumed
it's negligible.
Iain
--
http://mail.python.org/mailman/listinfo/python-list
Re: substitution
On Jan 21, 2:18 pm, Wilbert Berendsen wrote:
> Op maandag 18 januari 2010 schreef Adi:
>
> > keys = [(len(key), key) for key in mapping.keys()]
> > keys.sort(reverse=True)
> > keys = [key for (_, key) in keys]
>
> > pattern = "(%s)" % "|".join(keys)
> > repl = lambda x : mapping[x.group(1)]
> > s = "fooxxxbazyyyquuux"
>
> > re.subn(pattern, repl, s)
>
> I managed to make it even shorted, using the key argument for sorted, not
> putting the whole regexp inside parentheses and pre-compiling the regular
> expression:
>
> import re
>
> mapping = {
> "foo" : "bar",
> "baz" : "quux",
> "quuux" : "foo"
>
> }
>
> # sort the keys, longest first, so 'aa' gets matched before 'a', because
> # in Python regexps the first match (going from left to right) in a
> # |-separated group is taken
> keys = sorted(mapping.keys(), key=len)
>
> rx = re.compile("|".join(keys))
> repl = lambda x: mapping[x.group()]
> s = "fooxxxbazyyyquuux"
> rx.sub(repl, s)
>
> One thing remaining: if the replacement keys could contain non-alphanumeric
> characters, they should be escaped using re.escape:
>
> rx = re.compile("|".join(re.escape(key) for key in keys))
>
> Met vriendelijke groet,
> Wilbert Berendsen
>
> --http://www.wilbertberendsen.nl/
> "You must be the change you wish to see in the world."
> -- Mahatma Gandhi
Sorting it isn't the right solution: easier to hold the subs as tuple
pairs and by doing so let the user specify order. Think of the
following subs:
"fooxx" -> "baz"
"oxxx" -> "bar"
does the user want "bazxbazyyyquuux" or "fobarbazyyyquuux"?
Iain
--
http://mail.python.org/mailman/listinfo/python-list
Re: Symbols as parameters?
On Jan 21, 7:43 am, Martin Drautzburg wrote: > Hello all, > > When passing parameters to a function, you sometimes need a paramter > which can only assume certain values, e.g. > > def move (direction): > ... > If direction can only be "up", "down", "left" or "right", you can solve > this by passing strings, but this is not quite to the point: > > - you could pass invalid strings easily > - you need to quote thigs, which is a nuisance > - the parameter IS REALLY NOT A STRING, but a direction > > Alternatively you could export such symbols, so when you "import *" you > have them available in the caller's namespace. But that forces you > to "import *" which pollutes your namespace. > > What I am really looking for is a way > > - to be able to call move(up) > - having the "up" symbol only in the context of the function call > > So it should look something like this > > ... magic, magic ... > move(up) > ... unmagic, unmagic ... > print up > > This should complain that "up" is not defined during the "print" call, > but not when move() is called. And of course there should be as little > magic as possible. > > Any way to achieve this? class Direction(object): pass def is_direction(d): return type(d)==Direction up = Direction() down = Direction() left = Direction() right = Direction() Now you can do your move(up), print up, etc. You can also check a valid direction was passed in by calling is_direction. 'Course, you can extend this so it does something a little more useful: class Direction(object): def __init__(self, vx=0, vy=0): self.vx = vx self.vy = vy up = Direction(0, -1) down = Direction(0, 1) left = Direction(-1, 0) right = Direction(1, 0) def move(direction): spaceship.x += direction.vx spaceship.y += direction.vy Iain -- http://mail.python.org/mailman/listinfo/python-list
Re: substitution
On Jan 18, 4:26 pm, Steven D'Aprano wrote:
> On Mon, 18 Jan 2010 06:23:44 -0800, Iain King wrote:
> > On Jan 18, 2:17 pm, Adi Eyal wrote:
> [...]
> >> Using regular expressions the answer is short (and sweet)
>
> >> mapping = {
> >> "foo" : "bar",
> >> "baz" : "quux",
> >> "quuux" : "foo"
>
> >> }
>
> >> pattern = "(%s)" % "|".join(mapping.keys())
> >> repl = lambda x : mapping.get(x.group(1), x.group(1))
> >> s = "fooxxxbazyyyquuux"
> >> re.subn(pattern, repl, s)
>
> > Winner! :)
>
> What are the rules for being declared "Winner"? For the simple case
> given, calling s.replace three times is much faster: more than twice as
> fast.
>
> But a bigger problem is that the above "winner" may not work correctly if
> there are conflicts between the target strings (e.g. 'a'->'X',
> 'aa'->'Y'). The problem is that the result you get depends on the order
> of the searches, BUT as given, that order is non-deterministic.
> dict.keys() returns in an arbitrary order, which means the caller can't
> specify the order except by accident. For example:
>
> >>> repl = lambda x : m[x.group(1)]
> >>> m = {'aa': 'Y', 'a': 'X'}
> >>> pattern = "(%s)" % "|".join(m.keys())
> >>> subn(pattern, repl, 'aaa') # expecting 'YX'
>
> ('XXX', 3)
>
> The result that you get using this method will be consistent but
> arbitrary and unpredictable.
>
> For those who care, here's my timing code:
>
> from timeit import Timer
>
> setup = """
> mapping = {"foo" : "bar", "baz" : "quux", "quuux" : "foo"}
> pattern = "(%s)" % "|".join(mapping.keys())
> repl = lambda x : mapping.get(x.group(1), x.group(1))
> repl = lambda x : mapping[x.group(1)]
> s = "fooxxxbazyyyquuux"
> from re import subn
> """
>
> t1 = Timer("subn(pattern, repl, s)", setup)
> t2 = Timer(
> "s.replace('foo', 'bar').replace('baz', 'quux').replace('quuux', 'foo')",
> "s = 'fooxxxbazyyyquuux'")
>
> And the results on my PC:
>
> >>> min(t1.repeat(number=10))
> 1.1273870468139648
> >>> min(t2.repeat(number=10))
>
> 0.49491715431213379
>
> --
> Steven
Adi elicited that response from me because his solution was vastly
more succinct than everything else that had appeared up til that point
while still meeting the OP's requirements. The OP never cared about
overlap between 2 'find' strings, just between the 'find' and
'replace' strings (though I did take it into account in my second post
for the sake of completeness). His code could have been a little
cleaner, I'd have trimmed it to:
mapping = {"foo": "bar", "baz": "quux", "quuux": "foo"}
pattern = "(%s)" % "|".join(mapping)
repl = lambda x : mapping[x.group(1)]
s = "fooxxxbazyyyquuux"
re.subn(pattern, repl, s)
but apart from that was very pythonic: explicit, succinct, and all the
heavy work is done by the module (i.e. in compiled c code in the
majority case of CPython). It can be 'upgraded' to cover the find-
find overlap if you really want (I *believe* regexps will match the
leftmost option in a group first):
subs = [("foo", "bar"), ("baz", "quux"), ("quuux", "foo")]
pattern = "(%s)" % "|".join((x[0] for x in subs))
mapping = dict(subs)
repl = lambda x : mapping[x.group(1)]
s = "fooxxxbazyyyquuux"
re.subn(pattern, repl, s)
Anyway, there's no prize for winning, but by all means: if you think
someone else's code and not a variation on this should win for most
pythonic, then make your nomination :)
Iain
--
http://mail.python.org/mailman/listinfo/python-list
Re: substitution
On Jan 18, 2:17 pm, Adi Eyal wrote:
> > From: superpollo
> > To:
> > Date: Mon, 18 Jan 2010 11:15:37 +0100
> > Subject: substitution
> > hi.
>
> > what is the most pythonic way to substitute substrings?
>
> > eg: i want to apply:
>
> > foo --> bar
> > baz --> quux
> > quuux --> foo
>
> > so that:
>
> > fooxxxbazyyyquuux --> barxxxquuxyyyfoo
>
> > bye
>
> Using regular expressions the answer is short (and sweet)
>
> mapping = {
> "foo" : "bar",
> "baz" : "quux",
> "quuux" : "foo"
>
> }
>
> pattern = "(%s)" % "|".join(mapping.keys())
> repl = lambda x : mapping.get(x.group(1), x.group(1))
> s = "fooxxxbazyyyquuux"
> re.subn(pattern, repl, s)
Winner! :)
Iain
--
http://mail.python.org/mailman/listinfo/python-list
Re: substitution
On Jan 18, 12:41 pm, Iain King wrote:
> On Jan 18, 10:21 am, superpollo wrote:
>
>
>
> > superpollo ha scritto:
>
> > > hi.
>
> > > what is the most pythonic way to substitute substrings?
>
> > > eg: i want to apply:
>
> > > foo --> bar
> > > baz --> quux
> > > quuux --> foo
>
> > > so that:
>
> > > fooxxxbazyyyquuux --> barxxxquuxyyyfoo
>
> > > bye
>
> > i explain better:
>
> > say the subs are:
>
> > quuux --> foo
> > foo --> bar
> > baz --> quux
>
> > then i cannot apply the subs in sequence (say, .replace() in a loop),
> > otherwise:
>
> > fooxxxbazyyyquuux --> fooxxxbazyyyfoo --> barxxxbazyyybar -->
> > barxxxquuxyyybar
>
> > not as intended...
>
> Not sure if it's the most pythonic, but I'd probably do it like this:
>
> def token_replace(string, subs):
> subs = dict(subs)
> tokens = {}
> for i, sub in enumerate(subs):
> tokens[sub] = i
> tokens[i] = sub
> current = [string]
> for sub in subs:
> new = []
> for piece in current:
> if type(piece) == str:
> chunks = piece.split(sub)
> new.append(chunks[0])
> for chunk in chunks[1:]:
> new.append(tokens[sub])
> new.append(chunk)
> else:
> new.append(piece)
> current = new
> output = []
> for piece in current:
> if type(piece) == str:
> output.append(piece)
> else:
> output.append(subs[tokens[piece]])
> return ''.join(output)
>
> >>> token_replace("fooxxxbazyyyquuux", [("quuux", "foo"), ("foo", "bar"),
> >>> ("baz", "quux")])
>
> 'barxxxquuxyyyfoo'
>
> I'm sure someone could whittle that down to a handful of list comps...
> Iain
Slightly better (lets you have overlapping search strings, used in the
order they are fed in):
def token_replace(string, subs):
tokens = {}
if type(subs) == dict:
for i, sub in enumerate(subs):
tokens[sub] = i
tokens[i] = subs[sub]
else:
s = []
for i, (k,v) in enumerate(subs):
tokens[k] = i
tokens[i] = v
s.append(k)
subs = s
current = [string]
for sub in subs:
new = []
for piece in current:
if type(piece) == str:
chunks = piece.split(sub)
new.append(chunks[0])
for chunk in chunks[1:]:
new.append(tokens[sub])
new.append(chunk)
else:
new.append(piece)
current = new
output = []
for piece in current:
if type(piece) == str:
output.append(piece)
else:
output.append(tokens[piece])
return ''.join(output)
--
http://mail.python.org/mailman/listinfo/python-list
Re: substitution
On Jan 18, 10:21 am, superpollo wrote:
> superpollo ha scritto:
>
> > hi.
>
> > what is the most pythonic way to substitute substrings?
>
> > eg: i want to apply:
>
> > foo --> bar
> > baz --> quux
> > quuux --> foo
>
> > so that:
>
> > fooxxxbazyyyquuux --> barxxxquuxyyyfoo
>
> > bye
>
> i explain better:
>
> say the subs are:
>
> quuux --> foo
> foo --> bar
> baz --> quux
>
> then i cannot apply the subs in sequence (say, .replace() in a loop),
> otherwise:
>
> fooxxxbazyyyquuux --> fooxxxbazyyyfoo --> barxxxbazyyybar -->
> barxxxquuxyyybar
>
> not as intended...
Not sure if it's the most pythonic, but I'd probably do it like this:
def token_replace(string, subs):
subs = dict(subs)
tokens = {}
for i, sub in enumerate(subs):
tokens[sub] = i
tokens[i] = sub
current = [string]
for sub in subs:
new = []
for piece in current:
if type(piece) == str:
chunks = piece.split(sub)
new.append(chunks[0])
for chunk in chunks[1:]:
new.append(tokens[sub])
new.append(chunk)
else:
new.append(piece)
current = new
output = []
for piece in current:
if type(piece) == str:
output.append(piece)
else:
output.append(subs[tokens[piece]])
return ''.join(output)
>>> token_replace("fooxxxbazyyyquuux", [("quuux", "foo"), ("foo", "bar"),
>>> ("baz", "quux")])
'barxxxquuxyyyfoo'
I'm sure someone could whittle that down to a handful of list comps...
Iain
--
http://mail.python.org/mailman/listinfo/python-list
Re: Writing a string.ishex function
On Jan 14, 3:52 pm, chandra wrote: > Folks, > > I am new to Python and could not find a function along the lines of > string.ishex in Python. There is however, a string.hexdigits constant > in the string module. I thought I would enhance the existing modlue > but am unsure how I should go about it. Specifically, I have attempted > this much: > ---cut--- > #! /usr/bin/python > # -*- coding: utf-8 -*- > > import string > > def ishex(string): > ishex = False > for i in string: > if i in string.hexdigits: > ishex = True > else: > ishex = False > break > return ishex > ---cut--- > > Can someone help me get further along please? > > Thanks. better would be: def ishex(s): for c in s: if c not in string.hexdigits: return False return True Iain -- http://mail.python.org/mailman/listinfo/python-list
Re: Different number of matches from re.findall and re.split
On Jan 11, 3:35 pm, Jeremy wrote:
> Hello all,
>
> I am using re.split to separate some text into logical structures.
> The trouble is that re.split doesn't find everything while re.findall
> does; i.e.:
>
>
>
> > found = re.findall('^ 1', line, re.MULTILINE)
> > len(found)
> 6439
> > tables = re.split('^ 1', line, re.MULTILINE)
> > len(tables)
> > 1
>
> Can someone explain why these two commands are giving different
> results? I thought I should have the same number of matches (or maybe
> different by 1, but not 6000!)
>
> Thanks,
> Jeremy
re.split doesn't take re.MULTILINE as a flag: it doesn't take any
flags. It does take a maxsplit parameter, which you are passing the
value of re.MULTILINE (which happens to be 8 in my implementation).
Since your pattern is looking for line starts, and your first line
presumably has more splits than the maxsplits you are specifying, your
re.split never finds more than 1.
>>> a
'split(pattern, string, maxsplit=0)\nSplit the source string by
the occurren
ces of the pattern,\nreturning a list containing the resulting
substrings.\n
'
>>> re.split(" ", a, re.MULTILINE)
['split(pattern,', 'string,', 'maxsplit=0)\n', '', '', '', 'Split',
'the', 'sour
ce string by the occurrences of the pattern,\nreturning a list
containing th
e resulting substrings.\n']
>>> re.split(" ", a)
['split(pattern,', 'string,', 'maxsplit=0)\n', '', '', '', 'Split',
'the', 'sour
ce', 'string', 'by', 'the', 'occurrences', 'of', 'the', 'pattern,\n',
'', '', ''
, 'returning', 'a', 'list', 'containing', 'the', 'resulting',
'substrings.\n']
Iain
--
http://mail.python.org/mailman/listinfo/python-list
Re: Object Relational Mappers are evil (a meditation)
Steven D'Aprano writes: > On Wed, 23 Dec 2009 10:55:19 -0500, J Kenneth King wrote: > >> Steven D'Aprano writes: >> >>> On Mon, 21 Dec 2009 11:44:29 -0500, J Kenneth King wrote: >>> >>>> A programmer that >>>> lacks critical thinking is a bad programmer. The language they use >>>> has no bearing on such human facilities. >>> >>> That's nonsense, and I can demonstrate it by reference to a single >>> programming language, namely Python. >>> >>> For many years, Python had no ternary if operator: > > [...] > >> But did the lack of ternary encourage Raymond to become a bad >> programmer? > > No, but Raymond started off in a position of being an excellent > programmer. A single buggy idiom lead him to be slightly-less excellent > than he otherwise would have been. How many buggy idioms would it take to > lead him to become a mediocre coder, if he was unable to change languages? > > Because Python is generally an excellent language, the harm done by one > or two misfeatures is minor. But less excellent languages encourage > coding styles, techniques and idioms that encourage the programmer to > write poor code: either complicated, baroque, unreadable code; or slow > inefficient code; or buggy code. To avoid starting a flame war, I will > avoid mentioning PHP. *cough* > > Sometimes you know what you need to do to write non-buggy code, but > because covering all the corners are just Too Damn Hard in a certain > language, you simply lower your expectations. Error checking is tedious > and hard to get right in some languages, like C and Pascal, and hence > even good programmers can miss some errors. > > Different languages encourage different mind-sets in the programmer: C > encourages the coder to think at the low level of pointers and addresses, > and primarily about machine efficiency; Java encourages the use of big > object hierarchies and design patterns (it's hard to write lightweight > code in Java, so everything turns into heavyweight code); Perl encourages > cleverness and code-golf (writing a program in as few lines or characters > as possible); Haskell and Lisp encourage a heavily abstract approach that > often requires an elite coder to follow; Forth encourages you to think > like Yoda. If anyone continues to follow bad idioms without questioning their usefulness from time to time, I'd question their ability as a programmer. Critical thinking is important. Which is why good programs can be written in PHP, Forth, Lisp, Perl, and anything else. However, if a programmer thinks the only language they will ever need to know is BF, they have a serious screw loose. ;) > [...] >> Good tools make all the difference in the world, I'm not arguing that. > > You appear to be arguing against that. Maybe you need to reconsider my arguments. It takes a good programmer to recognize the values and trade-offs of the tools they work with. Ignorance is not an excuse to blame the language. It's too easy to say, "Well Perl sucks because it encourages you to be a bad programmer because it has all these features that let you shoot yourself in the foot." In reality, lots of really great programs are written in Perl all the time and some very smart people write them. It just so happens that in hands of the educated, those very features are useful in certain cases. Python doesn't "encourage" you to be a better programmer. It just enforces particular idioms and conventions in its design. As long as the ignorant programmer follows them they should be better off. Yet if they are ignorant, no amount of encouragement will get them to think critically about Python and find bugs in it. They will have to rely on the community of developers to do that thinking for them. > >> Just that the tools don't use us; we use them. > > Nobody said that tools use us. But it is being suggested that they influence our thinking. Pretty smart thing for a language to be able to do. >> Programming in Python >> doesn't instantly make me a better programmer. > > No, not instantly, but I would argue that after many years of coding in > Python you will be a better programmer than after the same number of > years of coding in PHP or Basic. And my argument is that the human element is what will determine who is better. There are good programmers who can program in PHP. Some of the biggest websites on the Internet are programmed in it. And like any language I'm sure it has a good number of inefficiencies and bad design decisions that the programmers using it had to work around. Yet despite it being a poor language in your opinion, they built successful program
Re: Object Relational Mappers are evil (a meditation)
Steven D'Aprano writes: > On Mon, 21 Dec 2009 11:44:29 -0500, J Kenneth King wrote: > >> A programmer that >> lacks critical thinking is a bad programmer. The language they use has >> no bearing on such human facilities. > > That's nonsense, and I can demonstrate it by reference to a single > programming language, namely Python. > > For many years, Python had no ternary if operator: > > result = x if condition else y > > Instead the accepted, idiomatic Python way of writing this was to use > short-circuit booleans: > > result = condition and x or y > > However this idiom is buggy! If x is a false-value (say, 0) then result > gets set to y no matter what the value of condition. > > This buggy idiom survived many years of Python development, missed by > virtually everyone. Even coders of the calibre of Raymond Hettinger (who > neither lacks critical thinking nor is a bad programmer) have been bitten > by this: > > "The construct can be error-prone. When an error occurs it can be > invisible to the person who wrote it. I got bitten in published code > that had survived testing and code review: ..." > > http://mail.python.org/pipermail/python-dev/2005-September/056510.html > > > This is a clear and obvious case where a language feature (in this case, > the lack of a feature) encouraged an otherwise excellent coder to make an > error. It was a very subtle error, which was not picked up by the author, > the tests, or the coder reviewer(s). Had Python been different (either by > including a ternary if statement, or by forcing and/or to return bools > only) then this bug never would have occurred. > > Of course awful programmers will be awful programmers in any language, > and excellent programmers will be excellent programmers in many languages. > > (I say "many" rather than any deliberately. There's a reason why nobody > uses languages like Brainf*ck, Whitespace, Ook or Intercal for real work.) > > But most coders are neither awful nor excellent. The language DOES make a > difference: the quality of a technician depends partly on the quality of > his tools, and programmers are no different. > > If you don't believe me, imagine writing code in a language without > functions or loops, so you have to use GOTO for everything. All very true. But did the lack of ternary encourage Raymond to become a bad programmer? That is what I believe the core of the argument is. Sure the misfeature was over-looked by Raymond, but it took him (and perhaps the help of others) to recognize it and fix it. That's because he's human and the language is inert. He is smart and obviously has the cognitive capabilities to recognize that the language has to change in order to be a better tool. It would be a different story if he just assumed that the misfeature was actually a feature and that it was a good thing. In such a case would Python the language be at fault or the people who write programs with it? Good tools make all the difference in the world, I'm not arguing that. Just that the tools don't use us; we use them. Programming in Python doesn't instantly make me a better programmer. It can certainly make me think of myself as a good programmer though... ;) -- http://mail.python.org/mailman/listinfo/python-list
Re: Object Relational Mappers are evil (a meditation)
Lie Ryan writes: > On 12/17/2009 3:17 PM, J Kenneth King wrote: >> A language is a thing. It may have syntax and semantics that bias it >> towards the conventions and philosophies of its designers. But in the >> end, a language by itself would have a hard time convincing a human >> being to adopt bad practises. > > Perhaps someone should make a research whether if you teach a language > to kids, where one group is taught the language filtered from "bad > words" and another group is taught all the languages' "bad words" on > purpose. Will one group have more behavioral problems compared to the > other? I would be curious to know, but the test is likely impossible without trespassing on ethical boundaries. ;) I would hypothesize that you would not find an increase in behavioural problems. a) Without cultural context "bad words" have little meaning b) Behavioural issues can be attributed to several factors such as physiology, health, environment, etc. c) This has nothing to do with programming languages. A programmer that lacks critical thinking is a bad programmer. The language they use has no bearing on such human facilities. -- http://mail.python.org/mailman/listinfo/python-list
Re: Object Relational Mappers are evil (a meditation)
Neil Cerutti writes: > On 2009-12-16, J Kenneth King wrote: >> The language doesn't encourage anything. It's just a medium >> like oil paints and canvas. A painting can be good or bad >> despite the medium it is constructed on. The skill of the >> painter is what matters. > > Technically, oil paints do encourage a certain kind of painting. > They can be layered on top of old paint easily, and they dry > slowly, allowing you to slowly "build up" a painting in layers, > and create effects with texture. If you try doing thse things > with watercolors, and you'll probably be discouraged. > > I think a programming language does encourage a certain kind of > code. Good code in one language can be poor in another. It's a weak analogy on my part, but I think I do understand what you mean here. In regards to my original point, I think I just came up with a clearer way to express it: A language is a thing. It may have syntax and semantics that bias it towards the conventions and philosophies of its designers. But in the end, a language by itself would have a hard time convincing a human being to adopt bad practises. I believe it's the fault of the programmer who adopts those poor practises. Surely their acceptance of GOTO statements and prototype-overloading are signs of their own preferences and ignorance? It suggests to me that they learnt enough of one language to get by and stopped thinking critically as soon as they sat in front of their keyboard. Throw an idiot behind a Python interpreter and it won't teach them a damn thing unless they're capable of learning it on their own. No matter how well you manage to hard code your conventions into the language. Bad code is written by bad programmers, not bad programming languages. -- http://mail.python.org/mailman/listinfo/python-list
Re: Object Relational Mappers are evil (a meditation)
r0g writes: > J Kenneth King wrote: >> Steven D'Aprano writes: >> >>> On Fri, 11 Dec 2009 19:20:21 -0500, Steve Holden wrote: >>> > > >>>> Hear, hear! >>> That's all very well, but some languages and techniques encourage the >>> programmer to write bad code. >> >> That's just BS. >> >> Bad code doesn't just write itself. Programmers write bad code. And >> ignorance is not an excuse. >> >> Just because a language allows a programmer to write sloppy code doesn't >> put the language at fault for the bad code programmers write with it. > > > > Okay, as long as you realize the corollary of your argument is: > > It is impossible for a language to encourage programmers to write good > code and promote good programming practices by design. > > I'm not sure that's entirely true either. > > I think python's "one way to do something" design philosophy goes some > way toward that, as does Smalltalk's enforced message passing. I think > PHP's superglobals and namespacing encourage bad practices (or used to > back in the day), as do Basic's GOTO and Ecmascript's prototype > overriding. I think your corollary is slightly misleading. It would be more apt to say, "Just because a language allows a programmer to write good code doesn't mean that the language is responsible for the good code programmers write with it." It is the responsibility of the programmer to recognize the advantages and flaws of their tools. PHP doesn't encourage a programmer to be a bad programmer because it lacks name-spaces or because BASIC has GOTO statements. A bad programmer will be a bad programmer because they don't understand what makes these features distinct, useful, or damaging. The language doesn't encourage anything. It's just a medium like oil paints and canvas. A painting can be good or bad despite the medium it is constructed on. The skill of the painter is what matters. > > Surely a language CAN be said to encourage kludges and sloppiness if it > allows a good way and a bad way and makes the bad way much easier to > implement or understand for noobs. > > Roger. The programmer can be encouraged to use kludges and produce sloppy code. Whether by ignorance or inflated ego. Languages with more choice just give them more rope to hang themselves with. -- http://mail.python.org/mailman/listinfo/python-list
Re: Object Relational Mappers are evil (a meditation)
Steven D'Aprano writes: > On Fri, 11 Dec 2009 19:20:21 -0500, Steve Holden wrote: > >> Simon Forman wrote: >> [...] >>> As far as the OP rant goes, my $0.02: bad programmers will write bad >>> code in any language, with any tool or system or environment they're >>> given. If you want to avoid bad code there's (apparently) no >>> substitute for smrt programmers who are familiar with the tools they're >>> using, not just the syntax but the underlying conceptual models as >>> well. >>> >> Hear, hear! > > That's all very well, but some languages and techniques encourage the > programmer to write bad code. That's just BS. Bad code doesn't just write itself. Programmers write bad code. And ignorance is not an excuse. Just because a language allows a programmer to write sloppy code doesn't put the language at fault for the bad code programmers write with it. Any half-way decent programmer should be cognisant of when they're writing bad code and when they're writing good code. They should be able to admit that they don't know enough about a language to be writing programs for money in it. They should be able to see anti-patterns and areas of their code that should be re-factored or re-written. The real underlying problem is the human characteristic that allows us to let ourselves believe that we're better than everyone else or more simply, better than we really are. -- http://mail.python.org/mailman/listinfo/python-list
Re: Perl to Python conversion
martin.sch...@gmail.com (Martin Schöön) writes: > First off: I am new here and this is my first post after > lurking for quite some time. Hi. > Second off: I don't know much Python---yet. It's not a very big language. If you have experience programming in other languages, you can probably pick it up in a day or two. > Problem: I have come across a small open source application > that I find quite useful. It does have one major flaw though. > Its output is in imperial units. Converting isn't a big deal > for occasional use but if I start to use this stuff on a > regular basis... > > So I down-loaded the source code and found this thing is written > in Perl. > > Should I learn enough Perl to add the conversion? Probably > but this may be a nice excuse to get my Python education > going and if I do I might as well re-do the user interface. Well you can always call it from Python via subprocess (which basically wraps a shell and has fancy ways putting data in and extracting data out). > If I do re-write this thing in Python I might need to learn both > Perl and Python... You'll need to know one of them rather well and enough of the other to get by. It's probably easier to know more Perl than Python since Perl is a lot more expressive than Python (in the TMTOWTDI sense). Frankly I learned Perl before Python and find it rather easy to go between the two. YMMV. > Hence, are there any Perl to Python converters? So far I > have only found bridgekeeper which really is (was?) consultancy. > Apart from that I only find people recommending a manual re-write. It depends where the two languages vary from one another. If the script your translating uses basic types or even simple classes and typical control structures and operations then translating from one to the other is a simple matter of copy-pasting the code and translating the syntax and small bits of grammar. However, in areas where there are high variations; you'll probably want to stay away from it. Perl has a lot of freedom to manipulate references and the programmer can modify the language to suit their needs. So just be careful of code that uses these features as they are difficult to translate into Python. > Any thoughts/recommendations? Depends: - If you needed it done yesterday to get some work done, wrap the Perl script in a subprocess and buy yourself some time to think it over. - If your purpose is to learn Python, then start from scratch. Use the Perl as a guide if there are any maths or algorithms you are unsure about. - If you're just hacking around to learn stuff, learn a little of both. It will make you smarter if it doesn't confuse the heck out of you and make you quit before you finish. ;) > > TIA, > > /Martin HTH -- http://mail.python.org/mailman/listinfo/python-list
Re: Question about 'remote objects'
"Frank Millman" writes: > Hi all > > I am writing a multi-user business/accounting application. It is getting > rather complex and I am looking at how to, not exactly simplify it, but find > a way to manage the complexity. > > I have realised that it is logically made up of a number of services - > database service with connection to database > workflow engine for business processes > services manager to handle automated services, such as web services > client manager to service logins from client workstations > possibly others > > I have made a start by splitting some of these off into separate modules and > running them in their own threads. > > I am concerned about scalability if they are all running on the same > machine, so I am looking into how to enable these services to run on > separate servers if required. Have you finished the application already? At my job we're still serving just over 1M+ web requests (a month), processing 15k+ uploads, and searching through over 5M+ database records a day. We're still running on 3 boxes. You can get a lot out of your machines before you have to think about the complex task of scaling/distributing. > My first thought was to look into Pyro. It seems quite nice. One concern I > had was that it creates a separate thread for each object made available by > the server. My database server creates separate objects for each instance of > a row read in from the database, and with multiple users running multiple > applications, with each one opening multiple tables, this could run into > hundreds, so I was not sure if that would work. It probably will work. Pyro is a very nice framework and one that I've built a few applications on. It has a lot of flexible design patterns available. Just look in the examples included with the distribution. > > Then I read that the multiprocessing module allows processes to be spread > across multiple servers. The documentation is not as clear as Pyro's, but it > looks as if it could do what I want. I assume it would use processes rather > than threads to make multiple objects available, but I don't know if there > is a practical limit. There is a theoretical limit to all of the resources on a machine. Threads don't live outside of that limit. They just have a speedier start-up time and are able to communicate with one another in a single process. It doesn't sound like that will buy you a whole lot in your application. You can spawn as many processes as you need. > > Then I thought that, instead of the database server exposing each object > remotely, I could create one 'proxy' object on the server through which all > clients would communicate, and it in turn would communicate with each > instance locally. > > That felt more managable, but then I thought - why bother with remote > objects at all? Why not just run a SocketServer on the database server, and > design a mini-protocol to allow clients to make requests and receive > results. This is a technology I am already comfortable with, as this is how > I handle client workstation logins. If I did go this route, I could apply > the same principle to all the services. Because unless you wrote your own database or are using some arcane relic, it should already have its own configurable socket interface? > > I don't have the experience to make an informed decision at this point, so I > thought I would see if there is any consensus on the best way to go from > here. Finish building the application. Do the benchmarks. Profile. Optimize. Find the clear boundaries of each component. Build an API along those boundaries. Add a network layer in front of the boundaries. Pyro is a good choice, twisted is also good. Roll your own if you think you can do better or it would fit your projects' needs. > Is there any particular benefit in using remote objects as opposed to > writing a SocketServer? Abstraction. Pyro is just an abstraction over an RPC mechanism. Nothing special about it. Twisted has libraries to do the same thing. Writing your own socket-level code can be messy if you don't do it right. > > Any advice will be much appreciated. > > Thanks > > Frank Millman Best of luck. -- http://mail.python.org/mailman/listinfo/python-list
Re: How decoupled are the Python frameworks?
shocks writes: > Hi > > I'm getting back into Python after a long break. I've been developing > large enterprise apps solely with Adobe Flex (ActionScript) for the > past couple years. During that time I've used a number of 'MVC' > frameworks to glue the bits together - among them Cairngorm, a > modified implementation of Cairngorm using the Presentation Model > pattern, PureMVC, Mate (an IOC container but with an MVC > implementation) and Parsley (IOC but you have to roll-you-own MVC). > During that time I've been in large teams (30 Flex + 30 Java) to small > teams (2 Flex + 1 Java). The motivation of these frameworks is the > decouple your concerns, allowing your apps to be more scalable, easier > to test, and supposedly easier to maintain. Some do the decoupling > better job than others, but there is also the question of "how > decoupled is your code from the framework"? It's all well and good > having something clever working behind the scenes wiring and routing > everything together, but I wonder where this leaves the code base if > the framework, which was selected at the beginning of the project, is > replaced with something else months or years later (i.e. the framework > just doesn't scale as expected, the community involvement dies and > it's no longer maintained properly, etc). I've seen it happen and > I've been experienced the pain of detangling massive amounts of code > which is full of framework specific imports, methods and boilerplate > code. And then there's updating the unit tests! > > My question is how good are the current crop of Python frameworks? > I've used Django twice in production and didn't like that much. The > implementation is Django specific for starters. I've picked up Pylons > and I'm trying that out. I'm not sure how well it fares? I do feel a > bit uneasy about the code generation that some of the Python > frameworks do. Pylons creates something like 20 files for a > 'helloworld'. It does do some great things out of the box, but I > wonder where that leaves your own code. After spending 3-6 months on > your Pylons webapp, how easy is it to move to something else? Maybe > one of the Python IOC once they mature. What are some good techniques > people are using to future (framework) proof their apps? > > I'm interested to hear people experiences with the various frameworks > and how decoupled their code is from them. The best of the current > Flex frameworks for me is Parsley. The only noticeable Parlsey code > is an '[Inject]' meta tag here and there and a couple import > statements. All the complicated object creation and messaging is done > higher up the chain. > > Cheers, > Ben I've stayed away from the Java world at least professionally... but I think I understand where you're getting. I'm not a huge fan of web development. Which is rather counter-intuitive for me to say since it's been the bulk of my work for the past four years. It's because there's no easy way to get around the warts. Websites are one thing but to try and think of these things as "applications" becomes an expensive illusion to maintain. The problem with thinking about these things as applications with an interface, a controller, and a model is: statelessness! Oh also serialization. Getting around these issues are pretty much the raison d'etre for web frameworks. Without them we end up with PHP. As far as swappable Python web frameworks... NONE. AFAIK, they all require some commitment to tying your application to them. However, there are a crop of frameworks that do minimize the pain of such a decision: web.py and bobo are two that I've been working with (though it sounds like cherrypy would be very good at separating dispatching from application code). You could of course write your stuff closer to the "metal" so to speak... WSGI would be about as low as I go. -- http://mail.python.org/mailman/listinfo/python-list
Re: Go versus Brand X
a...@pythoncraft.com (Aahz) writes: > Comparing Go to another computer language -- do you recognize it? > > http://www.cowlark.com/2009-11-15-go/ If you skip to the conclusion, you'll be better off. The author has an interesting point. Go (the language) is not really ground-breaking. I don't understand why they're so busy creating their own walled garden... Their own browser, their own programming languages, their own file systems, etc. Weird. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python Programming Challenges for beginners?
astral orange <457r0...@gmail.com> writes: > Hi- > > I am reading the online tutorial along with a book I bought on Python. > I would like to test out what I know so far by solving programming > challenges. Similar to what O'Reilly Learning Perl has. I really > enjoyed the challenges at the end of the chapter and it really help me > test out if I was truly taking in the material like I should. Could I > get some suggestions on resources? Is there anywhere where I can go > online (for free or purchase) for programming problems? I am familiar > with sites like Code Chef...etc...but at this stage that is not the > right 'venue' for me. I mainly need challenges like the ones they have > in Learning Perl. > > Thanks again for all the help, > 457r0 http://www.pythonchallenge.com/ I find this one neat because it uses riddles rather than straight math. -- http://mail.python.org/mailman/listinfo/python-list
Re: Perl conversion to python...
Benjamin Schollnick writes:
> Folks,
>
> I'm having some issues here with pyserial & trying to translate a perl
> script to python... It's probably my inexperience with PySerial &
> perl that is troubling me...
>
> Can anyone assist?
>
> I'm concerned, since I can't seem to receive the data in any reliable
> manner.. I've tested multiple times, and only once received data...
> So I suspect that my Transmit & receive code is faulty...
>
> def xmit ( data, serialport ):
> for x in data:
> xmit_byte (x, serialport)
> # serialport.write ( binascii.unhexlify ( data ) )
> # for x in data:
> # print str(x).encode ('hex')
> # serialport.write ( x.encode('hex'))
>
> def receive ( serialport ):
> received = serialport.read (20)
> print received, "!"
Gah.. indentation is broken in your post... :S
>
> - Perl Code
> sub tx_command {
> my $port = shift;
> my $cmd = shift;
>
> # warn "tx_command($cmd)\n";
>
> my @cmd_bytes = split(/\s/, $cmd);
>
> foreach my $byte (@cmd_bytes) {
> $byte = pack('C', hex($byte));
>
> $port -> write($byte);
> select(undef, undef, undef, 0.01);
> }
> }
import struct
def tx_command(port, cmd):
cmd_bytes = cmd.split(' ')
for byte in cmd_bytes:
byte = struct.pack('C', hex(int(byte)))
port.write(byte)
# select() is a system call in Perl..
# insert Python equivalent here
>
> # returns the rtt, or 0 if no response
> sub rx_response {
> my ($port, $address) = @_;
>
> $port->read_char_time(0); # don't wait for each character
> $port->read_const_time(5000); # timeout if we don't get what we're
> looking for
>
> my $buf = '';
>
> my $t_start = time;
>
> ### accumulate one byte at a time until we see the substring we're
> looking for
>
> while (1) {
> my ($count_in, $string_in) = $port->read(1);
>
> if ($count_in == 0) {
> # warn "TIMEOUT\n";
> return 0;
> }
>
> $buf .= $string_in;
>
> my $bufstring = packed_to_text($buf);
>
> #warn "bufstring: ".$bufstring;
>
> if ($bufstring =~/02 50 $address (.. .. ..) (..) (.. ..)/) {
>
> my $powerlinc_addr = $1;
> my $flags = $2;
> my $command = $3;
>
> # warn "got response!\n";
>
> my $rtt = time() - $t_start;
>
> return $rtt;
> }
>
> }
> }
This isn't all that more complicated. I dunno, maybe it's just me but
I find most Perl pretty easy to read (barring obfuscation which just
takes longer to read but is no more difficult). For the most part you
can generally substitute the Python equivalent statements for each
line of Perl and get good results. Optimize for better Pythonicity
afterwards.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Python/HTML integration: phileas v0.3 released
papa hippo writes: > On 20 nov, 09:02, Stefan Behnel wrote: >> papa hippo, 19.11.2009 19:53: >> >> > The prime goal of 'phileas' is to enable html code to be seamlessly >> > included in python code in a natural looking syntax, without resorting >> > to templatng language. >> >> I assume you know XIST, ElementTree's ElementMaker, and all those other >> ways of generating XML/HTML from Python code in a natural looking way? >> >> Stefan > > Hi Stefan, > > Thanks for your feedback. > > Yes, I am aware that phileas might - on the basis of the short > description on this post - come across like a 're-invented wheel'. > There is, however, one big difference between phileas and all other > other similar packages (XIST, ELementTree, HTMLgen, HyperText, > pyhtmloo etc.) that I inspected: > > Phileas uses distinct objects to generate each start and end tag, > whereas all the others use a single function call (in some cases > itself generated by a function call) to generate a complete well- > formed element including start-tag and (where required) end-tag. In > theory this is less neat and indeed it means one can write 'bad' HTML > (e.g. missing end of paragraphs) with phileas just as easily as when > writing pure html. In practice, however, I find it at a lot easier to > use. > > While using pyhtmloo (my previous favourite HTML generator), I had > found myself using awkward complicated artificial constructions in > order to generate all but the simplest HTML - and spent much time > playing 'hunt the missing bracket'. With phileas, these complexities > seem to just fall away. Any decent editor should be able to balance parenthesis for you. > > Put another way, Phileas generates HTML4.0 - warts and all; it is not > a parser or generator of XML. > > I'm considering building in checks/warnings for unclosed elements > etc., probably in the next-but-one pre-release. That your library will require a validation to be executed at run-time seems like it will be tedious to use. A decent text editor can even balance your HTML tags for you. Though you have a neat "DSL" like language for representing HTML elements. I'd suggest taking it one step further and creating a machine that can read in a Python data-structure and with as few hints as possible wrap it in the appropriate tags. > > Larry -- http://mail.python.org/mailman/listinfo/python-list
No duplicate variable
class A(object): def __init__(self, value=0.): self.value = value class B(A): def __init__(self, value=None): A.__init__(self) self.value = value obj = B() When "B" initializes, it overwrite "value" variable of "A". How do I make sure that no variable should not be defined with names of "A" in "B"? Prashant -- http://mail.python.org/mailman/listinfo/python-list
Re: python simply not scaleable enough for google?
David Cournapeau writes: > On Tue, Nov 17, 2009 at 10:48 PM, Aaron Watters > wrote: >> >>> I don't think Python and Go address the same set of programmer >>> desires. For example, Go has a static type system. Some programmers >>> find static type systems to be useless or undesirable. Others find >>> them extremely helpful and want to use them them. If you're a >>> programmer who wants a static type system, you'll probably prefer Go >>> to Python, and vice versa. That has nothing to do with implementation >>> speed or development expenditures. If Google spent a million dollars >>> adding static types to Python, it wouldn't be Python any more. >> >> ... and I still have an issue with the whole "Python is slow" >> meme. The reason NASA doesn't build a faster Python is because >> Python *when augmented with FORTRAN libraries that have been >> tested and optimized for decades and are worth billions of dollars >> and don't need to be rewritten* is very fast. > > It is a bit odd to dismiss "python is slow" by saying that you can > extend it with fortran. One of the most significant point of python > IMO is its readability, even for people not familiar with it, and > that's important when doing scientific work. Relying on a lot of > compiled libraries goes against it. > > I think that python with its scientific extensions is a fantastic > tool, but I would certainly not mind if it were ten times faster. In > particular, the significant cost of function calls makes it quickly > unusable for code which cannot be easily "vectorized" - we have to > resort to using C, etc... to circumvent this ATM. > > Another point which has not been mentioned much, maybe because it is > obvious: it seems that it is possible to makes high level languages > quite fast, but doing so while keeping memory usage low is very > difficult. Incidentally, the same tradeoff appears when working with > vectorized code in numpy/scipy. I think this is the only interesting point in the whole conversation so far. It is possible for highly dynamic languages to be optimized, compiled, and run really fast. The recent versions of SBCL can compile Common Lisp into really fast and efficient binaries. And Lisp could be considered even more dynamic than Python (but that is debateable and I have very little evidence... so grain of salt on that statement). It's possible, it just hasn't been done yet. PyPy is getting there, but development is slow and they could probably use a hand. Instead of waiting on the sidelines for a company to back PyPy developemnt, the passionate Python programmers worth a salt that care about Python development should contribute at least a patch or two. The bigger problem though is probably attention span. A lot of developers today are more apt to simply try the next new language than to roll of their sleeves and think deeply enough to improve the tools they're already invested in. > > David -- http://mail.python.org/mailman/listinfo/python-list
Re: Get attribute this way
"eval" can solve this problem right away but I am concerned about security issues. If not "eval" could you suggest something more efficient way. It won't be a big deal to change the format as application is still at development stage? Thanks Prashant Python 2.6.2 Win XP 32 -- http://mail.python.org/mailman/listinfo/python-list
Re: Get attribute this way
Writing/Reading data to xml is not a problem. The problem is when I have to write to attributes in xml, where a connections has been established. XML: While reading back I have to convert both source and destination strings into object so that I can call connections function using source and target attributes. Prashant Python 2.6.2 Win XP 32 -- http://mail.python.org/mailman/listinfo/python-list
Get attribute this way
Python's getattr, setattr and __getattribute__ commands works fine
with python types.
For example:
print o.__getattribute__("name")
print getattr(o, "name")
This is the easiest way to get an attribute using a string.
In my case the "Node" class load/creates all the attributes from a xml
file.
Example
There are certain atrributes of compound type.
Example:
# Array of colors, Here we are referring first color and second element
(green)
self.gradient.colors[0][1] = 0.5
# Array of floats, referring first element
self.gradient.positions[0] = 1.0
Color, color array and floats are all custom classes.
Now question is how to get these compound attributes using string as
we do with default singular attributes.
Example:
getattr(o, "gradient.colors[0][1] ")
I need this to save the data of nodes->attributes into a custom xml
format, when loading this data back, I create the node first and then
setup values from the saved xml file.
Prashant
Python 2.6.2
Win XP 32
--
http://mail.python.org/mailman/listinfo/python-list
Re: object serialization as python scripts
King writes: >> Why is it easier than the above mentioned - they are *there* (except the >> custom xml), and just can be used. What don't they do you want to do? >> >> Other than that, and even security issues put aside, I don't see much >> difference between pickle and python code, except the latter being more >> verbose. Which suits humans, but other than that has no advantage. >> >> Diez > > My application is PyQt based and objects that I am trying to save are > couple of QGraphicsItem instances. You can't save instances of > QGraphicsItem > using pickle or shelve. > Custom xml format would be a real pain as you have to write code for > writing/reading both. > > I am aware of security issue but over all there are only 5 statements > I have to use to get the entire > information back. I can do syntax checking and other stuff before I > should execute the script. > > Another reason is that data should be in human readable form. > > Prashant > > Python 2.6.2 > Win XP 32 Pickling should work just fine. If you cannot pickle the class with the default pickler, you can hook into the pickler protocol to tell pickle how to pickle instances of the class. Failing that you can write your own pickler class for handling the special case. Python scripts aren't really a good data format. They're not structured in a way that would be easy to parse and extract information from in a non-python context without a lot of work. -- http://mail.python.org/mailman/listinfo/python-list
Re: object serialization as python scripts
> Why is it easier than the above mentioned - they are *there* (except the > custom xml), and just can be used. What don't they do you want to do? > > Other than that, and even security issues put aside, I don't see much > difference between pickle and python code, except the latter being more > verbose. Which suits humans, but other than that has no advantage. > > Diez My application is PyQt based and objects that I am trying to save are couple of QGraphicsItem instances. You can't save instances of QGraphicsItem using pickle or shelve. Custom xml format would be a real pain as you have to write code for writing/reading both. I am aware of security issue but over all there are only 5 statements I have to use to get the entire information back. I can do syntax checking and other stuff before I should execute the script. Another reason is that data should be in human readable form. Prashant Python 2.6.2 Win XP 32 -- http://mail.python.org/mailman/listinfo/python-list
object serialization as python scripts
I have looked upon various object serialization de-serialization techniques. (shelve, pickle, anydbm, custom xml format etc.) What I personally feel that instead of all these methods for saving the objects it would be easier to save the data as python scripts itself. In this case, loading the data is merely to run the script in the context itself. Need some expert advice and pointers here. Is there any thing (module, script, doc, article) helpful out there for the concern? Prashant Python 2.6.2 Win XP 32 -- http://mail.python.org/mailman/listinfo/python-list
object indexing and item assignment
class MyFloat(object): def __init__(self, value=0.): self.value = value def set(self, value): self.value = value def get(self): return self.value class MyColor(object): def __init__(self, value=(0,0,0)): self.value = (MyFloat(value[0]), MyFloat(value[1]), MyFloat(value[2])) def set(self, value): self.value[0].set(value[0]) self.value[1].set(value[1]) self.value[2].set(value[2]) def get(self): return (self.value[0].get(), self.value[1].get(), self.value[2].get()) col = MyColor() col[0].set(0.5) # 'MyColor' object does not support indexing col[0] = 0.5 # 'MyColor' object does not support item assignment The last two lines of the script produce errors. (written as comments). I know it won't work as I am expecting. One solution I can think of is to rewrite MyFloat and MyColor by sub classing default python types "float and "tuple". Is this the only solution? Prashant Python 2.6.2 Win XP 32 -- http://mail.python.org/mailman/listinfo/python-list
Re: python simply not scaleable enough for google?
mcherm writes: > On Nov 11, 7:38 pm, Vincent Manis wrote: >> 1. The statement `Python is slow' doesn't make any sense to me. >> Python is a programming language; it is implementations that have >> speed or lack thereof. >[...] >> 2. A skilled programmer could build an implementation that compiled >> Python code into Common Lisp or Scheme code, and then used a >> high-performance Common Lisp compiler... > > I think you have a fundamental misunderstanding of the reasons why > Python is > slow. Most of the slowness does NOT come from poor implementations: > the CPython > implementation is extremely well-optimized; the Jython and Iron Python > implementations use best-in-the-world JIT runtimes. Most of the speed > issues > come from fundamental features of the LANGUAGE itself, mostly ways in > which > it is highly dynamic. > > In Python, a piece of code like this: > len(x) > needs to watch out for the following: > * Perhaps x is a list OR > * Perhaps x is a dict OR > * Perhaps x is a user-defined type that declares a __len__ > method OR > * Perhaps a superclass of x declares __len__ OR > * Perhaps we are running the built-in len() function OR > * Perhaps there is a global variable 'len' which shadows the > built-in OR > * Perhaps there is a local variable 'len' which shadows the > built-in OR > * Perhaps someone has modified __builtins__ > > In Python it is possible for other code, outside your module to go in > and > modify or replace some methods from your module (a feature called > "monkey-patching" which is SOMETIMES useful for certain kinds of > testing). > There are just so many things that can be dynamic (even if 99% of the > time > they are NOT dynamic) that there is very little that the compiler can > assume. > > So whether you implement it in C, compile to CLR bytecode, or > translate into > Lisp, the computer is still going to have to to a whole bunch of > lookups to > make certain that there isn't some monkey business going on, rather > than > simply reading a single memory location that contains the length of > the list. > Brett Cannon's thesis is an example: he attempted desperate measures > to > perform some inferences that would allow performing these > optimizations > safely and, although a few of them could work in special cases, most > of the > hoped-for improvements were impossible because of the dynamic nature > of the > language. > > I have seen a number of attempts to address this, either by placing > some > restrictions on the dynamic nature of the code (but that would change > the > nature of the Python language) or by having some sort of a JIT > optimize the > common path where we don't monkey around. Unladen Swallow and PyPy are > two > such efforts that I find particularly promising. > > But it isn't NEARLY as simple as you make it out to be. > > -- Michael Chermside You might be right for the wrong reasons in a way. Python isn't slow because it's a dynamic language. All the lookups you're citing are highly optimized hash lookups. It executes really fast. The OP is talking about scale. Some people say Python is slow at a certain scale. I say that's about true for any language. Large amounts of IO is a tough problem. Where Python might get hit *as a language* is that the Python programmer has to drop into C to implement optimized data-structures for dealing with the kind of IO that would slow down the Python interpreter. That's why we have numpy, scipy, etc. The special cases it takes to solve problems with custom types wasn't special enough to alter the language. Scale is a special case believe it or not. As an implementation though, the sky really is the limit and Python is only getting started. Give it another 40 years and it'll probably realize that it's just another Lisp. ;) -- http://mail.python.org/mailman/listinfo/python-list
Re: substituting list comprehensions for map()
Steven D'Aprano writes: > On Tue, 03 Nov 2009 10:22:28 -0500, J Kenneth King wrote: > >> However in this case the procedure by which we derive the value is not >> important or even interesting. It is much more succinct to think of the >> operation as a value and express it accordingly. There's no need to >> clutter the mind with extra name bindings and iteration keywords. They >> won't make our idea any more clear. >> >> dot_product = map(mul, vec1, vec2) >> >> vs >> >> dot_product = [a * b for a, b in zip(vec1, vec2)] >> >> It's very clear, at least to me, what a dot-product is in this case. > > Except it's not. > > The dot product of two vectors returns a scalar, not another vector: > http://en.wikipedia.org/wiki/Dot_product > > So what you want is: > > dot_product = sum(map(mul, vec1, vec2)) Derh. Thanks for the catch. My bad. >> Adding in the loop construct and name bindings doesn't enhance my >> understanding of what a dot-product is. I don't need to see the loop >> construct at all in this case. A dot product is simply the >> multiplication of each element in a vector sequence. > > What you need is to define a function dot-product, and not hijack the > name for a local value. Then the function's implementation is irrelevant > to you: it could use a list comp, or could use map, it could use a for- > loop, a while loop, recursion, or black magic: > > scalar = dot_product(vec1, vec2) Even better. But now I'm afraid the example is running away from the point. So to summarize: 1. Extra name bindings and loop keywords aren't always easier to read. 2. Expressing what we want rather than how we get it is much more clear. and (third dirty argument added) 3. List comprehensions leak their name bindings to the surrounding scope. :p Have a nice day. :) -- http://mail.python.org/mailman/listinfo/python-list
Re: substituting list comprehensions for map()
Ben Finney writes: > J Kenneth King writes: > >> Steven D'Aprano writes: >> >> > from operator import add >> > map(add, operandlist1, operandlist2) >> >> This is the best solution so far. > > Strange to say it's a solution, when it doesn't solve the stated > problem: to replace use of ‘map()’ with a list comprehension. Granted I admit this later in my post. It's not a solution to the OPs request, but it is the best solution to such a case. > >> I understand the OP was asking for it, but list comprehensions aren't >> the best solution in this case... it would just be line noise. > > I disagree; a list comprehension is often clearer than use of ‘map()’ > with a lambda form, and I find it to be so in this case. The use of map expresses a value and implies the procedure by which it is obtained. The list comprehension expresses the procedure by which the value is obtained. Both have uses and in some cases a list comprehension is definitely preferrable to a map operation. However in this case the procedure by which we derive the value is not important or even interesting. It is much more succinct to think of the operation as a value and express it accordingly. There's no need to clutter the mind with extra name bindings and iteration keywords. They won't make our idea any more clear. dot_product = map(mul, vec1, vec2) vs dot_product = [a * b for a, b in zip(vec1, vec2)] It's very clear, at least to me, what a dot-product is in this case. Adding in the loop construct and name bindings doesn't enhance my understanding of what a dot-product is. I don't need to see the loop construct at all in this case. A dot product is simply the multiplication of each element in a vector sequence. It's more succinct to simply think of the value rather then expressing the procedure to construct that value. This isn't a hammer issue. Not every problem is a nail. -- http://mail.python.org/mailman/listinfo/python-list
conditional __init__
class A(object): def __init__(self): pass def printme(self): print "I am A" class B(object): def __init__(self): pass def printme(self): print "I am B" class K(A, B): def __init__(self, value=0): if value == 0: A.__init__(self) print "__init__ A" elif value == 1: B.__init__(self) print "__init__ B" self.printme() o = K(value=1) Output >>__init__ B >>I am A In above code "B" is correctly getting initialized as per condition. How ever method "printme" is printing "I am A". Instead it has to print "I am B" because "B" is the one that has been initialized. What's wrong here? Is there a better/another way to do conditional initialization as needed above? Cheers Prashant Python 2.6.2 Win XP 32 -- http://mail.python.org/mailman/listinfo/python-list
Re: substituting list comprehensions for map()
Steven D'Aprano writes: > On Sun, 01 Nov 2009 23:54:16 -0800, Jon P. wrote: > >> I'd like to do: >> >> resultlist = operandlist1 + operandlist2 >> >> where for example >> >> operandlist1=[1,2,3,4,5] >> operandlist2=[5,4,3,2,1] >> >> and resultlist will become [6,6,6,6,6]. Using map(), I can do: >> >> map(lambda op1,op2: op1 + op2, operandlist1, operandlist2) > > > If the two lists are very large, it would be faster to use this: > > > from operator import add > map(add, operandlist1, operandlist2) This is the best solution so far. > > >> Is there any reasonable way to do this via a list comprehension ? > > [x+y for (x, y) in zip(operandlist1, operandlist2)] > > If the lists are huge, you can save some temporary memory by replacing > zip with itertools.izip. I understand the OP was asking for it, but list comprehensions aren't the best solution in this case... it would just be line noise. List comprehensions are good for one-off transformations where it would only create a one-time method for map to use. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python + twisted = Raindrop (in part)
Terry Reedy writes: > Rrom: > First look: inside Mozilla's Raindrop messaging platform > > http://arstechnica.com/open-source/news/2009/10/first-look-inside-mozillas-raindrop-messaging-platform.ars > > "The backend components that are responsible for retrieving and > processing messages are coded in Python on top of the Twisted > networking framework." > > Ok, front end is html/Javascript, DB is erlang-based > > tjr There are already developers grumbling about using Twisted. https://wiki.mozilla.org/Raindrop/BackEndRoadmap#untwist_me.3F I checked out the source myself and looked at the roadmaps and bug lists to see if there were any patches I could work on. But it doesn't look all that interesting unless you're into Javascript. Python is mainly used to fetch data. Scheduling the fetch is left up to the user/OS. Python is also used for some "extensions" which are basically document transformation scripts. The extensions are stored in the CouchDB and are eval'd asynchronously on each document in the CouchDB. A fairly expensive process to be sure. Anyway, it's still really early in development so I'm sure they could use some help. :) PS: Anyone wanna take a stab at their twisted pro/con list? Cheers -- http://mail.python.org/mailman/listinfo/python-list
Re: What IDE has good git and python support?
Aweks writes: > what do you use? emacs. -- http://mail.python.org/mailman/listinfo/python-list
Re: Object Relational Mappers are evil (a meditation)
Aaron Watters writes: > On Oct 16, 10:35 am, mario ruggier wrote: >> On Oct 5, 4:25 pm, Aaron Watters wrote: >> >> > Occasionally I fantasize about making a non-trivial change >> > to one of these programs, but I strongly resist going further >> > than that because the ORM meatgrinder makes it somewhere >> > between extremely unpleasant and impossible to make any >> > non-trivial changes to a non-trivial program, especially after >> > it has been populated with data. >> >> Object-relational mappers are like putting lipstick on a >> pig:http://gizmoweblog.blogspot.com/2006/10/putting-lipstick-on-pig.html >> >> m ;-) > > Cute, but wrong. Using ORMs is better than using "Object databases". > > In my case I use Python to un data created by java/hibernate. > If I was using a java based "object database" I would be simply stuck. > At least if you use an ORM you have a way to access the information > without writing a program in the programming language that the > ORM was defined in. Anyway, thanks for all the great comments on > this thread from all you Sarcopterygii and Haplorrhini out there. Data persistence isn't a "one-size fits all" problem. It really depends on the needs of the system. Object databases solve the problem of storing complex object graphs, deeply nested data structures, and serializing language-specific objects like continuations or what-have-you (but I think that last one is yet unsolved). We all know what RDBMS' are good for. Neither is perfect for solving every data persistence situation. -- http://mail.python.org/mailman/listinfo/python-list
Re: The rap against "while True:" loops
On Oct 19, 7:51 am, Hendrik van Rooyen wrote: > On Sunday, 18 October 2009 11:31:19 Paul Rubin wrote: > > > Hendrik van Rooyen writes: > > > Standard Python idiom: > > > > if key in d: > > > d[key] += value > > > else: > > > d[key] = value > > > The issue is that uses two lookups. If that's ok, the more usual idiom is: > > > d[key] = value + d.get(key, 0) > > I was actually just needling Aahz a bit. The point I was trying to make > subliminally, was that there is a relative cost of double lookup for all > cases versus exceptions for some cases. - Depending on the frequency > of "some", I would expect a breakeven point. > > - Hendrik Indeed - the method I use for this (picked up from this newsgroup), is to work out roughly how often you need to make a new record instead of altering a current one, and depending on that use either: if key in d: d[key] += value else: d[key] = value or try: d[key] += value except KeyError: d[key] = value I find both to be easily readable (and the similarity between the two blocks is obvious and, to me at least, pleasing). Iain -- http://mail.python.org/mailman/listinfo/python-list
Re: Simple if-else question
On Sep 30, 7:12 am, Steven D'Aprano wrote: > On Tue, 29 Sep 2009 22:29:10 -0700, John Yeung wrote: > > On Sep 29, 1:15 pm, Carl Banks wrote: > >> Hmm, I wonder if Python should emit a warning if an else is used on a > >> for block with no break inside. I don't think the else can be invoked > >> in any other way. As a bonus it could catch some cases where people > >> mistakenly use it thinking it will execute [only] when there are no > >> iterations. > > > [Edit from Duncan Booth] > > > I would definitely be in favor of a warning. Yes, people should read > > the docs more carefully, and yes, it would cost a certain amount of > > annoyance to implement this. But I don't think it would get in people's > > way if they do know how to use else, > > Of course it would. It would mean that everybody who knows how to use > for...else correctly would have to deal with a totally useless warning. > > > and I think it would cut down on > > the number of questions from mystified beginners, some of whom are much > > more aggressive than this particular OP about claiming that Python is > > broken (when it's actually behaving as designed). > > By raising a warning, you are implying that the behaviour is broken, or > at least suspicious. Warnings mean something needs to be warned against > -- "don't do this". Warnings shouldn't be perfectly legitimate behaviours > on the off-chance that the user is confused. "Warning, are you sure you > want to put the car into reverse? Perhaps you meant neutral?" > > What would the warning say? > > "Warning, you have used a legitimate Python control structure but you > might be confused by it." > > "Warning, did you mean if...else instead of for...else?" > > Then we can add a whole lot of extra warnings to protect newbies who > don't read docs (and probably won't read the warnings either) from > themselves: > > "Warning, did you mean obj(1) instead of obj[1]?" > > "Warning, did you mean def f(object) instead of class f(object)?" > > "Warning, did you mean class f(object) instead of def f(object)?" > > "Warning, did you mean 2*3 instead of 2**3?" > > "Warning, did you mean %s instead of %x?" > > "Warning, we think you're helpless and don't know what you want, perhaps > you should be programming in PHP?" > > I'm sympathetic to the desire to educate the n00bs, and in fact I've even > suggested similar warnings myself. But I've been convinced that this is > the wrong approach. Certainly the language shouldn't assume the > programmer is confused. If this sort of warning belongs anywhere (and > that's a big if), it belongs in an IDE. > > -- > Steven Read the suggestion again - it's not a warning on the for-else structure, it's a warning when the for-else doesn't contain a break; he's theorising that a for-else without a break will always trigger the else, in which case it's almost certainly an error, and having the warning is not a bad idea. However, I assume you can get past the else by raising an exception, so the idea becomes a little muddled - do you warn when there is no break and no explicit raise caught outside the loop? What about an implicit exception? I would guess that code intentionally using an implicit exception to break out of a for loop is in need of a warning (and the author in need of the application of a lart), but I'm sure you could construct a plausible situation where it wouldn't be that bad... Anyway, I'm ambivalently on the fence. Iain -- http://mail.python.org/mailman/listinfo/python-list
Re: nested structure with "internal references"
Hendrik van Rooyen writes: > On Friday, 25 September 2009 19:11:06 Torsten Mohr wrote: > >> I'd like to use a nested structure in memory that consists >> of dict()s and list()s, list entries can be dict()s, other list()s, >> dict entries can be list()s or other dict()s. >> >> The lists and dicts can also contain int, float, string, ... > > This sounds terribly convoluted. > What are you trying to do? > >> >> But i'd also like to have something like a "reference" to another >> entry. > > Everything and its brother in Python is an object, and things like lists and > dicts reference objects automagically. > >> I'd like to refer to another entry and not copy that entry, i need to >> know later that this is a reference to another entry, i need to find >> also access that entry then. > > Depending on how I read this, it is either trivial or impossible: > > A name is bound to an object by assignment. > An object can have many names bound to it. > A name can, at different times, refer to different objects. > An object does not know which names are bound to it, or in what sequence it > was done. > > So you can go from name to object, but not the other way around. > You can use the same name in a loop to refer to different objects, and in > each > iteration, use the name to store a reference to the current object in a list > or dict. > >> Is something like this possible in Python? > > Not too sure what you are trying to do. > >> The references only need to refer to entries in this structure. >> The lists may change at runtime (entries removed / added), so >> storing the index may not help. > > You could start off with a list of lists of lists, to any level of nesting > that you want. This will give you a structure like a tree with branches and > twigs and leaves, but I am not sure if this is what you want or need, or if a > flat structure like third normal form would suffice, or if you need a > relational database. > > - Hendrik I'm not really sure what he wants either, but it sounds suspiciously like a linked list. In regards to the OP, though not a true linked-list, Python objects can be built with classes that describe essentially the same functionality. Just be careful about keeping references to large in-memory objects (see weakref) and try to avoid circular references in your mappings. Cheers -- http://mail.python.org/mailman/listinfo/python-list
Re: easy question, how to double a variable
On Sep 23, 7:36 pm, David C Ullrich wrote: > On Tue, 22 Sep 2009 02:34:53 +, Steven D'Aprano wrote: > > On Mon, 21 Sep 2009 13:50:23 -0500, David C Ullrich wrote: > > >> But you actually want to return twice the value. I don't see how to do > >> that. > > > What? > > > Seriously? > > You're saying it _can_ be done in Python? They must have added > something to the standard library again. I mean how can you return > twice a value without a twice function to start with? I've tried. > You'd think > > def twice(n): > return twice(n) > > would work, but I get this really long error message. > > > You're not just yanking the OP's chain??? > > That would be cruel. I mean the guy has enough problems already... Sorry, there is no 'twice' builtin. I think what you are looking for is: def twice(n): return return n Iain -- http://mail.python.org/mailman/listinfo/python-list
Re: SQLite or files?
ici writes: > I like shelve for saving small amounts of data, user preferences, > recent files etc. > http://docs.python.org/library/shelve.html I like it too, but I hear the great powers that be are going to deprecate it. > > For Qt use QtCore.QCoreApplication.setOrganizationName, > QtCore.QCoreApplication.setApplicationName than setValue, value from > QtCore.QSettings. -- http://mail.python.org/mailman/listinfo/python-list
Re: Sanitising arguments to shell commands
shlex doesn't handle unicode input though, so, in general, it's not a good solution. Rick King Southfield MI http://docs.python.org/library/shlex.html module shlex — Simple lexical analysis New in version 1.5.2. "The shlex class makes it easy to write lexical analyzers for simple syntaxes resembling that of the Unix shell." -- http://mail.python.org/mailman/listinfo/python-list
Re: how to use "exec" stmt to get input from user
Thanks for your comment.
The purpose of the application is to automate the manipulation of large
numbers of files. For automation I want to be able to have scripts that
I can use for various purposes.
On input, cmd.py handles calling "do_" methods for known commands;
for the rest I want to "exec" the lines with my own dictionary. This
allows me to use python statements, so I can do things like
concatenating variables to form directory names, and so on. I'd like to
be able to use any python statement, including "x=raw_input('>')", but
this currently brings everything to a halt.
One thing this app does is execute processes using wxPythons wx.Process
object and wx.Execute function. wxPython makes it easy to redirect stdin
and stdout. I'm trying to do a similar thing for individual python
statements. I'm not sure that makes sense.
I hope that's clear.
Rick
David C Ullrich wrote:
What you're trying to do and what's not working isn't
entirely clear to me.
But if I had a wxPython application and I wanted to
execute user input (note the _if_) I'd just pop up a window
(I forget how "ShowModal" is spelled in wx right now)
with a text box and an Execute button and a Cancel
button - if the user hits the Execute button I'd
attempt to execute what he'd typed in the box.
There are reasons you want to be very careful about
this...
On Mon, 10 Aug 2009 13:44:17 -0400, Rick King wrote:
I have a cmd.py-derived program (with a wxPython GUI) and want to
execute python statements for lines that are not my own special
commands.
So basically it's either:
def do_somecommand(self,arg):
...
or
def default(self,arg):
exec arg in globals(),self.cmdlocals
(where cmdlocals is a my local dictionary)
in default() I'd like to be able to execute any python statement
including something like
"x = raw_input('>')"
when I do this though it goes to the command window and so any user of
the program would be confused, which also means I have to have a command
window.
If I add this:
self.stdin = self.edt_console_input (where self.edt_console_input
is a wxPython text control)
it just gets an EOF right away.
Is there any way to do what I want to do? This might be better posted on
the wxpython list.
Thanks for any help!
Rick King
Southfield MI
--
http://mail.python.org/mailman/listinfo/python-list
how to use "exec" stmt to get input from user
I have a cmd.py-derived program (with a wxPython GUI) and want to
execute python statements for lines that are not my own special commands.
So basically it's either:
def do_somecommand(self,arg):
...
or
def default(self,arg):
exec arg in globals(),self.cmdlocals
(where cmdlocals is a my local dictionary)
in default() I'd like to be able to execute any python statement
including something like
"x = raw_input('>')"
when I do this though it goes to the command window and so any user of
the program would be confused, which also means I have to have a command
window.
If I add this:
self.stdin = self.edt_console_input (where self.edt_console_input
is a wxPython text control)
it just gets an EOF right away.
Is there any way to do what I want to do? This might be better posted on
the wxpython list.
Thanks for any help!
Rick King
Southfield MI
--
http://mail.python.org/mailman/listinfo/python-list
py2exe-created exe results in "application failed to initialize"
Hi everyone,
I want to package up an application into an exe using py2exe but the
result produces the dreaded
"application failed to initialize 0x142" error.
I'm using wxPython and basically just took the sample for wxpython GUI
that came with py2exe and changed the name. My setup is python 2.6,
wxpython 2.8. My setup.py looks like the following.
from distutils.core import setup
import py2exe
import sys
class Target:
def __init__(self, **kw):
self.__dict__.update(kw)
# for the versioninfo resources
self.version = "0.1"
self.company_name = "OWDC"
self.copyright = "no copyright"
self.name = "FileTool"
manifest_template = '''
etc.
'''
RT_MANIFEST = 24
FileTool = Target(
description = "FileTool",
script = "filetoolGUI.py",
other_resources = [(RT_MANIFEST, 1, manifest_template %
dict(prog="FileTool"))],
dest_base = "FileTool")
setup(
options = {"py2exe": {"compressed": 1,"optimize": 2,"ascii":
1,"bundle_files": 1}},
zipfile = None,
windows = [FileTool],
)
Any help will be greatly appreciated!
Rick King
Southfield MI USA
--
http://mail.python.org/mailman/listinfo/python-list
Re: remove last 76 letters from string
On Aug 6, 11:34 am, MRAB wrote: > Iain King wrote: > >> print >>nucleotides, seq[-76] > > >> last_part = line.rstrip()[-76 : ] > > > You all mean: seq[:-76] , right? (assuming you've already stripped > > any junk off the end of the string) > > The OP said "cut out the last 76 letters (nucleotides) from each > individual sequence and send them to a new txt file with a similar > format.", ie extract the last 76 letters (in the same format) to a file. So he did. Excuse me while I go eat my other foot. -- http://mail.python.org/mailman/listinfo/python-list
Re: remove last 76 letters from string
> print >>nucleotides, seq[-76] > last_part = line.rstrip()[-76 : ] You all mean: seq[:-76] , right? (assuming you've already stripped any junk off the end of the string) Iain -- http://mail.python.org/mailman/listinfo/python-list
Re: Confessions of a Python fanboy
On Jul 31, 4:08 pm, Ethan Furman wrote: > Steven D'Aprano wrote: > > On Thu, 30 Jul 2009 18:47:04 +0100, Tim Rowe wrote: > > >>That and the fact that I couldn't stop laughing for long enough to learn > >>any more when I read in the Pragmatic Programmer's Guide that "Ruby, > >>unlike less flexible languages, lets you alter the value of a constant." > >>Yep, as they say "Bug" = "Undocumented feature"! > > > That's no different from Python's "constant by convention". We don't even > > get a compiler warning! > > That's quite different, actually. Python doesn't claim to have > constants! Can't misinterpret what you can't have, can you? > > [Holds breath while awaiting counter-example... :] > > ~Ethan~ The convention being detailed in PEP8: http://www.python.org/dev/peps/pep-0008/ basically, anything in ALL_CAPS is a constant, assuming you follow those style guidelines. Iain -- http://mail.python.org/mailman/listinfo/python-list
Re: Confessions of a Python fanboy
On Jul 31, 8:28 am, Steven D'Aprano wrote: > On Thu, 30 Jul 2009 18:06:31 -0500, Robert Kern wrote: > > On 2009-07-30 16:44, r wrote: > >> On Jul 30, 4:29 pm, Emmanuel Surleau wrote: > 1.) No need to use "()" to call a function with no arguments. Python > --> "obj.m2().m3()" --ugly > Ruby --> "obj.m1.m2.m3" -- sweeet! > Man, i must admit i really like this, and your code will look so much > cleaner. > >>> It has benefits - code does look better. It has also significant cons > >>> - it is ambiguous. > >>> For instance: > > >>> a = b > > >>> Is b a variable or a method called without parameter? > > >> Hello Emanuel, > >> Again, who so ever names a method with such a non-descriptive name will > >> get whats coming to him. And if you did for some reason use such a > >> cryptic name as "b", do yourself (and everyone else) a favor and follow > >> it with "()" to denote the method call. Remember when something is > >> optional that means you have an option to use it OR not use it. > > > I believe his point is that it is ambiguous to the compiler, not humans > > reading the code. Python functions and methods are first class objects. > > They can be passed around. If they were auto-called, then you could not > > do this. > > Oh my, "r" is still around is he??? And now he's singing the praises of > Ruby, the language which he treated as the Devil's Spawn when he first > arrived. That's hilarious. > > But back on topic... "r" has missed the point. It's not that a=b is hard > to understand because b is a poor name. The example could have been: > > def factory_function(): > magic = time.time() # or whatever > def inner(): > return magic > return inner > > my_function = factory_function > > It's still ambiguous. Does the programmer intend my_function to become > factory_function itself, or the output of factory_function? Not only that - does 'return inner' return the function inner or the result of function inner? How does ruby pass a function as an object? Iain -- http://mail.python.org/mailman/listinfo/python-list
Re: escaping characters in filenames
Nobody writes: > On Wed, 29 Jul 2009 09:29:55 -0400, J Kenneth King wrote: > >> I wrote a script to process some files using another program. One thing >> I noticed was that both os.listdir() and os.path.walk() will return >> unescaped file names (ie: "My File With Spaces & Stuff" instead of "My\ >> File\ With\ Spaces\ \&\ Stuff"). I haven't had much success finding a >> module or recipe that escapes file names and was wondering if anyone >> could point me in the right direction. >> >> As an aside, the script is using subprocess.call() with the "shell=True" >> parameter. There isn't really a reason for doing it this way (was just >> the fastest way to write it and get a prototype working). I was >> wondering if Popen objects were sensitive to unescaped names like the >> shell. I intend to refactor the function to use Popen objects at some >> point and thought perhaps escaping file names may not be entirely >> necessary. > > Note that subprocess.call() is nothing more than: > > def call(*popenargs, **kwargs): > return Popen(*popenargs, **kwargs).wait() > > plus a docstring. It accepts exactly the same arguments as Popen(), with > the same semantics. > > If you want to run a command given a program and arguments, you > should pass the command and arguments as a list, rather than trying to > construct a string. > > On Windows the value of shell= is unrelated to whether the command is > a list or a string; a list is always converted to string using the > list2cmdline() function. Using shell=True simply prepends "cmd.exe /c " to > the string (this allows you to omit the .exe/.bat/etc extension for > extensions which are in %PATHEXT%). > > On Unix, a string is first converted to a single-element list, so if you > use a string with shell=False, it will be treated as the name of an > executable to be run without arguments, even if contains spaces, shell > metacharacters etc. > > The most portable approach seems to be to always pass the command as a > list, and to set shell=True on Windows and shell=False on Unix. > > The only reason to pass a command as a string is if you're getting a > string from the user and you want it to be interpreted using the > platform's standard shell (i.e. cmd.exe or /bin/sh). If you want it to be > interpreted the same way regardless of platform, parse it into a > list using shlex.split(). I understand; I think I was headed towards subprocess.Popen() either way. It seems to handle the problem I posted about. And I got to learn a little something on the way. Thanks! Only now there's a new problem in that the output of the program is different if I run it from Popen than if I run it from the command line. The program in question is 'pdftotext'. More investigation to ensue. Thanks again for the helpful post. -- http://mail.python.org/mailman/listinfo/python-list
escaping characters in filenames
I wrote a script to process some files using another program. One thing I noticed was that both os.listdir() and os.path.walk() will return unescaped file names (ie: "My File With Spaces & Stuff" instead of "My\ File\ With\ Spaces\ \&\ Stuff"). I haven't had much success finding a module or recipe that escapes file names and was wondering if anyone could point me in the right direction. As an aside, the script is using subprocess.call() with the "shell=True" parameter. There isn't really a reason for doing it this way (was just the fastest way to write it and get a prototype working). I was wondering if Popen objects were sensitive to unescaped names like the shell. I intend to refactor the function to use Popen objects at some point and thought perhaps escaping file names may not be entirely necessary. Cheers -- http://mail.python.org/mailman/listinfo/python-list
Re: Script runs manually, but cron fails
Bryan writes: > I have a backup script that runs fine when I run it manually from the > command line. When I run it with cron, the script stops running at > random points in the source code. > > The script calls rsync with the subprocess module, which in turn uses > ssh to backup files from a box on my lan. It also uses the atexit > module to always run a certain piece of cleanup code when the script > exits. However, this piece of code is never called when I use cron to > run the script, but a ps -A command also does not show the python > process so I know the script is dead. The log files generated by the > script all show rsync as completing, but then the logging gets cutoff > at random places when the script dies after that point. I am not > getting any email from cron complaining of an error either. > > The script runs fine in my bash shell, what could cron be doing to > interfere? Double check nothing is writing to stdout/stdin Some cron's don't mind, but I always squelch output because some do. -- http://mail.python.org/mailman/listinfo/python-list
Re: uniicode and executing a process with subprocess.call, or os.system
Thanks. I looked around for alternatives but didn't find this one. Rick Chris Rebert wrote: On Sat, Jul 18, 2009 at 3:30 PM, Rick King wrote: Hello, I want to copy files using subprocess.call or os.system where the file names are non-ascii, e.g. Serbian(latin), c's and s's with hacheks,etc. Windows stores all the file names in unicode so they are displayed ok in explorer, and I can read them into my program with listdir(u'.'), etc. and work with the names in the program. You should try one of the copying functions in the shutil module instead, it'll be much simpler than using subprocess: http://docs.python.org/library/shutil.html Cheers, Chris -- http://mail.python.org/mailman/listinfo/python-list
