Re: Is print thread safe?
On 12Aug2014 08:01, Marko Rauhamaa ma...@pacujo.net wrote: Steven D'Aprano steve+comp.lang.pyt...@pearwood.info: Personally, I believe that print ought to do its own locking. And print is a statement, although in this case there's no need to support anything older than 2.6, so something like this ought to work: from __future__ import print_function _print = print _rlock = threading.RLock() def print(*args, **kwargs): with _rlock: _print(*args, **kwargs) Could this cause a deadlock if print were used in signal handlers? At the C level one tries to do as little as possible in q signal handler. Typically setting a flag or putting something on a queue for later work. In Python that may be a much smaller issue, since I imagine the handler runs in the ordinary course of interpretation, outside the C-level handler context. I personally wouldn't care if this might deadlock in a handler (lots of things might; avoid as many things as possible). Also, the code above uses an RLock; less prone to deadlock than a plain mutex Lock. Cheers, Cameron Simpson c...@zip.com.au A host is a host from coast to coast no one will talk to a host that's close Unless the host (that isn't close) is busy, hung or dead - David Lesher, wb8...@skybridge.scl.cwru.edu -- https://mail.python.org/mailman/listinfo/python-list
Re: Is print thread safe?
On 12Aug2014 09:56, Steven D'Aprano steve+comp.lang.pyt...@pearwood.info wrote: Cameron Simpson wrote: On 12Aug2014 02:07, Steven D'Aprano steve+comp.lang.pyt...@pearwood.info wrote: Is this documented somewhere? In python/2.7.6/reference/simple_stmts.html#index-22, print is described in terms of a write for each object, and a write for the separators. There is no mention of locking. Ah, thanks! On that basis, I would find the interleaving described normal and expected. And certainly not broken. I personally didn't describe it as broken, Yes, sorry. but it is, despite the documentation. I just ran a couple of trials where I collected the output of sys.stdout while 50 threads blasted Spam ABCD EFGH (plus the implicit newline) to stdout as fast as possible using print. The result was that out of 248165 lines[1], 595 were mangled. Many of the mangled lines were the expected simple run-ons: Spam ABCD EFGHSpam ABCD EFGH\n\n which makes sense given the documentation, but there were lots of anomalies. Mysterious spaces appearing in the strings: Spam ABCD EFGH Spam ABCD EFGH\n\n Spam ABCD EFGH Spam ABCD EFGH\n Spam ABCD EFGH\n occasional collisions mid-string: Spam ABSpam ABCD EFGH\nCD EFGH\n letters disappearing: Spam AB\nD EFGH\n and at least one utterly perplexing (to me) block of ASCII NULs appearing in the middle of the output: \x00\x00\x00...\x00\x00\n This is with Python 2.7.2 on Linux. Sounds like print is not thread safe. Which it does not promise to be. But I would normally expect most file.write methods to be thread safe. Naively. Just use a lock! And rebind print! Or use the logging system! Personally, I believe that print ought to do its own locking. I don't, but I kind of believe files should have thread safe write calls. Again, not guarrenteed AFAIR. And print is a statement, although in this case there's no need to support anything older than 2.6, so something like this ought to work: from __future__ import print_function _print = print _rlock = threading.RLock() def print(*args, **kwargs): with _rlock: _print(*args, **kwargs) Sadly, using print as a function alone isn't enough to fix this problem, but in my quick tests, using locking as above does fix it, and with no appreciable slowdown. I would expect file.write to be fast enough that the lock would usually be free. With no evidence, just personal expectation. Taking a free lock should be almost instant. [1] Even the number of lines of output demonstrates a bug. I had fifty threads printing 5000 times each, which makes 25 lines, not 248165. Sounds like the file internals are unsafe. Ugh. Cheers, Cameron Simpson c...@zip.com.au If it ain't broken, keep playing with it. -- https://mail.python.org/mailman/listinfo/python-list
Captcha identify
Hi guys, These days I got a small task to identify Captcha characters. I googled a lot and find some way to do verification code identify. However, most are for general captcha. And, for simple captcha, I can use Pytesser. However, what about those advanced pictures. I mean: 1.including number and alpha 2.letters might be rotated 3.letters might be deformed I don't know why I cannot insert picture attachment here... But I think you guys can think out what the captcha looks like:-) So, any suggestions? Thanks. Wesley -- https://mail.python.org/mailman/listinfo/python-list
Re: Linux distros w/o Python in base installation
In article captjjmpfyoxnp-1w2-_8riaoz30w2lsz5q7ky4fbpc-bpmv...@mail.gmail.com, Chris Angelico ros...@gmail.com wrote: Well, it only *became* contrary to recommended practice in response to Arch doing it and everyone seeing the issues it caused :) Personally, I'm glad they did. Lets those of us who follow slower distros (I'm running Debian) get the benefit of someone else's hindsight. Well, many people thought that the problems caused by changing python to mean python3 at this stage in its lifecycle were so obvious that there was no need to formalize them in a PEP. Arch proved us wrong, hence PEP 394. http://legacy.python.org/dev/peps/pep-0394/ -- Ned Deily, n...@acm.org -- https://mail.python.org/mailman/listinfo/python-list
Python in financial services
Ive been asked to formulate a python course for financial services folk. If I actually knew about the subject, I'd have fatter pockets! Anyway heres some thoughts. What I am missing out? [Apart from basic python -- contents typically needs tailoring to the audience] the following: - Libraries -- Decimal? - scripts -- philosophy and infrastructure eg argparse, os.path - Pandas - Numpy Scipy (which? how much?) - ipython + matplotlib + ?? - Database interfacing - Excel interfacing (couple of libraries.. which?) - C(C++?) interfacing paradigms -- ranging from ctypes, cython to classic lo-level -- https://mail.python.org/mailman/listinfo/python-list
Re: Linux distros w/o Python in base installation
2014-08-11 23:36 GMT+02:00 Ned Deily n...@acm.org: In article lsb84u$21c$1...@reader1.panix.com, Grant Edwards invalid@invalid.invalid wrote: Apparently. Perhaps theres an enable LSB compliance option somewhere in the Arch install docs, but I didn't see it... Also beware that, unlike most other distributions and contrary to recommended practice, Arch has chosen to make Python 3 its default, that is, when everything is installed, `python` invokes `python3`, rather than `python2`. So you may need to change shebang lines in scripts, etc. https://wiki.archlinux.org/index.php/python -- Ned Deily, n...@acm.org -- https://mail.python.org/mailman/listinfo/python-list Also Gentoo uses Python3 by default for some months now. The positive side effect for me has been that I started seriously to switch to python3. Best -- https://mail.python.org/mailman/listinfo/python-list
Re: Linux distros w/o Python in base installation
On 12.08.2014 09:59, David Palao wrote: Also Gentoo uses Python3 by default for some months now. The positive side effect for me has been that I started seriously to switch to python3. it's a matter of months for debian/ubuntu to rely only on python3 code too, while still respecting PEP 394: https://wiki.ubuntu.com/Python/3 -- https://mail.python.org/mailman/listinfo/python-list
Re: Linux distros w/o Python in base installation
On Mon, Aug 11, 2014 at 8:53 PM, Grant Edwards invalid@invalid.invalid wrote: I just installed Arch Linux for the first time, and was surprosed to find that Python isn't installed as part of a base system. It's also not included in the 'base-devel' package group. It's trivial to install, but I'd still pretty surprised it's not there by default. I guess I've spent too much time with Gentoo, Debian, and RedHat derivitives which require Python be installed. I've probably used at least a dozen Linux distros over the years, and this is the first time I've noticed that Python wasn't installed by default. Arch has a different idea of “base system”. The base group contains the most crucial packages needed to run an Arch Linux system, and that is all. And you do not need Python to do so. Nevertheless, most people will likely install Python by themselves, or with a package that depends on Python. On Mon, Aug 11, 2014 at 9:02 PM, Chris Rebert c...@rebertia.com wrote: It would seem that such distros are opting to not be LSB-compliant?: http://refspecs.linuxfoundation.org/LSB_4.1.0/LSB-Languages/LSB-Languages/pylocation.html Arch does not really care about LSB, AFAIK. -- Chris “Kwpolska” Warrick http://chriswarrick.com/ PGP: 5EAAEA16 stop html mail | always bottom-post | only UTF-8 makes sense -- https://mail.python.org/mailman/listinfo/python-list
Re: how to get the ordinal number in list
On Tue, 12 Aug 2014 00:21:28 -0700, wxjmfauth wrote: Math: Try to formalize with mathematics the Flexible String Representation. You should quickly realize, it is a logical mathematical absurdity. Unbelievable. jmf Mathematicians work with numbers (Algebra is a abstraction of numerical concepts) strings are concerned with characters (Arabic numerals are just characters commonly used to represent numbers ) therefore even trying to rationalise any string representation with mathematics is a logical absurdity in itself. -- Someone on IRC was very sad about the uptime of his machine wrapping from 497 days to 0. -- linux-kernel -- https://mail.python.org/mailman/listinfo/python-list
Re: how to write file into my android phone?
Chris “Kwpolska” Warrick kwpol...@gmail.com writes: 2) the phone isn't necessarily visible on a pc as a drive at all. For example the Samsung gs4. This is actually true for ALL android devices, starting with Android 3.0. There was just a guy on comp.mobile.android saying his Android 4.2 phone (BLU Dash or something like that, extremely Chinese) presents its external SD card as a USB Mass Storage drive to a PC when connected over USB. Exactly the thing Google hates and fears and has tried to kill off... But it's not like Google can force anything on phones running AOSP Android without Google services and that's China and a few other large emerging markets. So it's just our small market here in the western countries where Google has some control... -- https://mail.python.org/mailman/listinfo/python-list
Re: how to get the ordinal number in list
On Tue, Aug 12, 2014 at 8:40 PM, alister alister.nospam.w...@ntlworld.com wrote: On Tue, 12 Aug 2014 00:21:28 -0700, wxjmfauth wrote: [ chomp ] Mathematicians work with numbers (Algebra is a abstraction of numerical concepts) strings are concerned with characters (Arabic numerals are just characters commonly used to represent numbers ) therefore even trying to rationalise any string representation with mathematics is a logical absurdity in itself. Don't bother responding to jmf; he doesn't listen. Also, his posts don't survive the jump to python-list, so we see them only when someone quotes him like that... as far as I'm concerned, his posts are now on par with those trying to sell us solution manuals or herbal remedies. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Is print thread safe?
On Tue, Aug 12, 2014 at 2:31 PM, Cameron Simpson c...@zip.com.au wrote: I would expect file.write to be fast enough that the lock would usually be free. Until the day when it becomes really REALLY slow, because your program's piped into 'less' and the user's paging through it. But even apart from that, writing to stdout can take a notable amount of time. Expecting the lock to usually be free will depend on the nature of the program - how much of it is spent in silent computation and how much in production of output. If that ratio is sufficiently skewed, then sure, the lock'll usually be free. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Captcha identify
On Tue, 12 Aug 2014 00:04:33 -0700, Wesley wrote: These days I got a small task to identify Captcha characters. Several of us code websites. Some of our websites may even use captcha. We use captcha to stop software posting spam to websites. What makes you think we have any inclination to help you defeat captchas? -- Denis McMahon, denismfmcma...@gmail.com -- https://mail.python.org/mailman/listinfo/python-list
Re: how to write file into my android phone?
On 2014-08-11, Grant Edwards invalid@invalid.invalid wrote: What MTP allows is parallel access between the MTP protocol server in the phone and the rest of the phone (OS and apps). In order to mount a partition as a USB mass storage device, it has to first be unmounted by the Android system. That's OK for an SD card that doesn't contain anything important required by the Android system, but it means that you can't mount the system partition without shutting down the phone completely. Another important reason for not presenting the phone's built-in flash as a USB mass storage device is that it requires you to use a brain-dead broken-as-designed Microsoft filesystem left over from the days of 360K floppy disk drives -- otherwise Windows won't know what to do with it. It's _possible_ to build an Android phone tht runs an DOS file system (and IIRC people have done it) but it introduces all sorts of security, reliability, and performance problems. -- Grant Edwards grant.b.edwardsYow! I wonder if I ought at to tell them about my gmail.comPREVIOUS LIFE as a COMPLETE STRANGER? -- https://mail.python.org/mailman/listinfo/python-list
Re: Linux distros w/o Python in base installation
On 2014-08-12, Chris “Kwpolska” Warrick kwpol...@gmail.com wrote: On Mon, Aug 11, 2014 at 8:53 PM, Grant Edwards invalid@invalid.invalid wrote: I just installed Arch Linux for the first time, and was surprosed to find that Python isn't installed as part of a base system. It's also not included in the 'base-devel' package group. It's trivial to install, but I'd still pretty surprised it's not there by default. I guess I've spent too much time with Gentoo, Debian, and RedHat derivitives which require Python be installed. I've probably used at least a dozen Linux distros over the years, and this is the first time I've noticed that Python wasn't installed by default. Arch has a different idea of “base system”. The base group contains the most crucial packages needed to run an Arch Linux system, and that is all. That's true of Gentoo as well, but it includes Python. And you do not need Python to do so. That's the part that surprised me. It's been a _long_ time since I've run a Linux distro that didn't require Python as a crucial system package. -- Grant Edwards grant.b.edwardsYow! Those people look at exactly like Donnie and gmail.comMarie Osmond!! -- https://mail.python.org/mailman/listinfo/python-list
Re: Captcha identify
Here is captcha link: https://drive.google.com/folderview?id=0B33_p7UnVqoyd09mT3V0aWFxRmcusp=sharing 在 2014年8月12日星期二UTC+8下午8时59分11秒，Dennis Lee Bieber写道： On Tue, 12 Aug 2014 00:04:33 -0700 (PDT), Wesley nisp...@gmail.com declaimed the following: Hi guys, These days I got a small task to identify Captcha characters. In other words, you have a task to make a robot that can break the procedures put in place to prevent robots from posting to web sites... The whole purpose of the CAPTCHA scheme is that computer AI systems aren't advanced enough to process them, whereas a human mind can almost do it in the subconscious. (as for picture attachments? comp.lang.python is a text newsgroup -- binaries aren't wanted in it; c.l.p is also gatewayed to a mailing list; and that mailing list is gatewayed to gmane.comp.python.general where it is made available as a news group (after spam filtering). Google Groups links to c.l.p (and many of us wish it didn't) -- Wulfraed Dennis Lee Bieber AF6VN wlfr...@ix.netcom.comHTTP://wlfraed.home.netcom.com/ -- https://mail.python.org/mailman/listinfo/python-list
Re: Captcha identify
On Wed, Aug 13, 2014 at 12:57 AM, Wesley nisp...@gmail.com wrote: Here is captcha link: https://drive.google.com/folderview?id=0B33_p7UnVqoyd09mT3V0aWFxRmcusp=sharing You seem to have misunderstood how grossly offensive your request is. I am now the third person to do you the courtesy of a response, but there are lots more who are simply deleting your post and moving on, or possibly marking you as a spammer, because that's about the only reason for wanting a program to solve CAPTCHAs. Let me spell it out for you: NO WE WILL NOT do this for you. And if you do it yourself, we will not be happy. Just don't. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Python in financial services
Rustom Mody rustompm...@gmail.com writes: Ive been asked to formulate a python course for financial services folk. If I actually knew about the subject, I'd have fatter pockets! Anyway heres some thoughts. What I am missing out? Good luck! It's a pretty broad field, so everyone probably has different needs. - Libraries -- Decimal? I've never seen decimal used, even though it makes sense for accounting-style finance. I've mostly been looking at forecasts, trading, and risk, where floats are fine. So maybe mention that it exists, so people know where to look if they need it, but don't stress it. - scripts -- philosophy and infrastructure eg argparse, os.path Basic argparse is very handy, but, again, I wouldn't spend too much time on it. - Pandas - Numpy Scipy (which? how much?) For me, pandas is huge, numpy is a nice fundamental substrate, while only bits and pieces of scipy are used (mostly optimization). statsmodels may also be worth a mention, as the answer to how do I do a regression. - ipython + matplotlib + ?? Ipython notebook + matplotlib is great. At least show that it exists. pandas plots may be enough, though. - Database interfacing Definitely mention. - Excel interfacing (couple of libraries.. which?) Meh, maybe. At least give a strategy. It always seems like a fool's errand, though: I end up just dumping data to CSV and using that. - C(C++?) interfacing paradigms -- ranging from ctypes, cython to classic lo-level Probably not, but it depends on the audience. The overview, like ctypes will link to C-like libraries, cython lets you write python-like code that runs fast, and there's SWIG and Boost.Python if you want to write your own modules is about all you need. Hope that helps, Johann -- https://mail.python.org/mailman/listinfo/python-list
Re: Captcha identify
On Wed, 13 Aug 2014 01:06:47 +1000, Chris Angelico wrote: Let me spell it out for you: NO WE WILL NOT do this for you. And if you do it yourself, we will not be happy. Just don't. Chris, I suspect he's a codemonkey in a chinese or similar asian spamhaus. We should probably be thankful that he's so dumb. -- Denis McMahon, denismfmcma...@gmail.com -- https://mail.python.org/mailman/listinfo/python-list
Re: Linux distros w/o Python in base installation
On Tue, Aug 12, 2014 at 8:12 AM, Grant Edwards invalid@invalid.invalid wrote: On 2014-08-12, Chris “Kwpolska” Warrick kwpol...@gmail.com wrote: On Mon, Aug 11, 2014 at 8:53 PM, Grant Edwards invalid@invalid.invalid wrote: I just installed Arch Linux for the first time, and was surprosed to find that Python isn't installed as part of a base system. It's also not included in the 'base-devel' package group. It's trivial to install, but I'd still pretty surprised it's not there by default. I guess I've spent too much time with Gentoo, Debian, and RedHat derivitives which require Python be installed. I've probably used at least a dozen Linux distros over the years, and this is the first time I've noticed that Python wasn't installed by default. Arch has a different idea of “base system”. The base group contains the most crucial packages needed to run an Arch Linux system, and that is all. That's true of Gentoo as well, but it includes Python. Because Portage is written in Python. Otherwise Gentoo would probably omit it from the base system as well. -- https://mail.python.org/mailman/listinfo/python-list
Re: Captcha identify
On Wed, Aug 13, 2014 at 1:38 AM, Denis McMahon denismfmcma...@gmail.com wrote: On Wed, 13 Aug 2014 01:06:47 +1000, Chris Angelico wrote: Let me spell it out for you: NO WE WILL NOT do this for you. And if you do it yourself, we will not be happy. Just don't. Chris, I suspect he's a codemonkey in a chinese or similar asian spamhaus. We should probably be thankful that he's so dumb. Yeah, that is the most likely scenario. If I'm courteous, I'll assume that he has no idea what the rest of the world thinks of this kind of thing. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Python in financial services
On Tue, 12 Aug 2014 00:33:11 -0700, Rustom Mody wrote: Ive been asked to formulate a python course for financial services folk. I wouldn't worry too much about c or c++ interfacing paradigms. -- Denis McMahon, denismfmcma...@gmail.com -- https://mail.python.org/mailman/listinfo/python-list
Using asyncio workers in a `concurrent.futures` interface
Hello everybody! I have a question. I have a Django app running on Heroku. I need to run about 100 worker threads there to do uploads/downloads simultaneously. A Heroku Dyno has only 512MB of memory, so I'm reluctant to run 100 worker threads. (I've had Dynos crash from lack of memory when using 6 threads before.) I heard that the asyncio module is mature and ready for usage, and I was happy because I kept hearing about it in the last year, and I saw Guido's lecture about it. If I understand correctly it would let me run multiple uploads and downloads efficiently in one thread, which would conserve more resources than using threads. (Please correct me if I'm wrong.) Now, I am a little clueless about the whole way it's built, using coroutines and tricky usage of `yield from`. I figured that since this is eventually a library for concurrency, i.e. doing many tasks at the same time, there will be an API in a style of Here are 100 tasks for you to do concurrently, let me know when they're done. I looked at the asyncio documentation page and saw that it does mention futures and executors, which is my favorite interface for doing concurrency. I was happy and I skimmed the docs. But, I couldn't find a simple way to use these. I don't want to learn how to define coroutines and use `yield from` to switch between them. (I use `yield from` regularly and fully understand how it works, I just don't write my programs that way.) What I'm expecting is something like this: download_file = lambda url: requests.get(url).content urls = ['http://google.com/file1.jpg', 'http://google.com/file2.jpg', 'http://google.com/file3.jpg'] # etc. with AsyncIOExecutor() as asyncio_executor: files = asyncio_executor.map(download_file, urls) And that's it, no coroutines, no `yield from`. Since, if I understand correctly, asyncio requires a mainloop, it would make sense for the AsyncIOExecutor to have a thread of its own in which it could run its mainloop. Is this possible? Did someone implement this? Thanks for your help, Ram. -- https://mail.python.org/mailman/listinfo/python-list
Re: how to get the ordinal number in list
On 8/10/2014 2:14 PM, Roy Smith wrote: In article 154cc342-7f85-4d16-b636-a1a953913...@googlegroups.com, Rustom Mody rustompm...@gmail.com wrote: l= [6,2,9,12,1,4] sorted(l,reverse=True)[:5] [12, 9, 6, 4, 2] No need to know how sorted works nor [:5] Now you (or Steven) can call it abstract. And yet its 1. Actual running code in the interpreter 2. Its as close as one can get to a literal translation of your Find the 5 largest numbers in a list [...] All the above are clearer than loops+assignments and can be taught before them I disagree. For a beginner, you want to be able to break things down into individual steps and examine the result at each point. If you do: l= [6,2,9,12,1,4] l2 = sorted(l,reverse=True) you have the advantage that you can stop after creating l2 and print it out. The student can see that it has indeed been sorted. With the chained operations, you have to build a mental image of an anonymous, temporary list, and then perform the slicing operation on that. Sure, it's the way you or I would write it in production code, but for a beginner, breaking it down into smaller pieces makes it easier to understand. l2[:5] Yes, and that teaching technique is supported by research. Beginners are particularly poor, in relation to experts, at noticing the applicability of idea, and at combining ideas together. Breaking things into component parts has multiple benefits: 1. The applicability of individual ideas becomes obvious. It's one thing to know about [].sort, and another thing to know when it's appropriate to sort something. 2. The expert specifically shows how and why the ideas are combined. This helps build the connection for the beginner, whose knowledge is not stored as an expert stores it; i.e, in broad categories with multiple connections; but as disorganized data with very few connections. http://www.amazon.com/How-Learning-Works-Research-Based-Principles/dp/0470484101 I bought the book based on a recommendation from SciPy talk, and it's really great. As an autodidact, it'll help me teach *myself* better, too. -- Neil Cerutti -- https://mail.python.org/mailman/listinfo/python-list
Re: Python in financial services
On Tuesday, August 12, 2014 9:05:44 PM UTC+5:30, Johann Hibschman wrote: Rustom Mody writes: - Pandas - Numpy Scipy (which? how much?) For me, pandas is huge, numpy is a nice fundamental substrate, while only bits and pieces of scipy are used (mostly optimization). statsmodels may also be worth a mention, as the answer to how do I do a regression. statsmodels seems like something useful -- thanks for the pointer -- https://mail.python.org/mailman/listinfo/python-list
Re: Python in financial services
On Tuesday, August 12, 2014 9:20:16 PM UTC+5:30, Denis McMahon wrote: On Tue, 12 Aug 2014 00:33:11 -0700, Rustom Mody wrote: Ive been asked to formulate a python course for financial services folk. I wouldn't worry too much about c or c++ interfacing paradigms. And I dont like teaching that stuff either :-) [especially on windows!] However those folks have thousands of lines of C/C++ which they are porting to python. So... Thanks anyway -- https://mail.python.org/mailman/listinfo/python-list
Re: how to get the ordinal number in list
On Tuesday, August 12, 2014 11:10:48 PM UTC+5:30, Neil D. Cerutti wrote: On 8/10/2014 2:14 PM, Roy Smith wrote: Rustom Mody wrote: l= [6,2,9,12,1,4] sorted(l,reverse=True)[:5] [12, 9, 6, 4, 2] No need to know how sorted works nor [:5] Now you (or Steven) can call it abstract. And yet its 1. Actual running code in the interpreter 2. Its as close as one can get to a literal translation of your Find the 5 largest numbers in a list [...] All the above are clearer than loops+assignments and can be taught before them I disagree. For a beginner, you want to be able to break things down into individual steps and examine the result at each point. If you do: l= [6,2,9,12,1,4] l2 = sorted(l,reverse=True) you have the advantage that you can stop after creating l2 and print it out. The student can see that it has indeed been sorted. With the chained operations, you have to build a mental image of an anonymous, temporary list, and then perform the slicing operation on that. Sure, it's the way you or I would write it in production code, but for a beginner, breaking it down into smaller pieces makes it easier to understand. l2[:5] Yes, and that teaching technique is supported by research. Yes and I as said to Roy I dont see any disagreement at least on this account. Beginners are particularly poor, in relation to experts, at noticing the applicability of idea, and at combining ideas together. Breaking things into component parts has multiple benefits: 1. The applicability of individual ideas becomes obvious. It's one thing to know about [].sort, and another thing to know when it's appropriate to sort something. 2. The expert specifically shows how and why the ideas are combined. This helps build the connection for the beginner, whose knowledge is not stored as an expert stores it; i.e, in broad categories with multiple connections; but as disorganized data with very few connections. Nice! And how do we lead the beginners towards expertise? In a way functional programming is to programming creativity what lego is to children's spatial creativity. Specifically there are a bunch of pieces that need to fit: 1. Functional Programming: Nothing more than composing functions [Maybe a bit simplistic but not unrealistic a defn] 2. Trying this out at the interpreter 3. Introspectable objects Some things follow from this: For the lego-game of playing with functions at the REPL to work and be pleasant and rewarding: 1. functions should be non side-effecting; else same trials giving different answers adds more confusion than understanding 2. They should be non-printing else: def foo(x): return x+1 def bar(x): print x+1 look similar when trivially tried but compositionally are utterly different In effect a printing function breaks the lego bricks [The P in the REPL is DRY enough that it does not usually need to be repeated all over] 3. Abstractions (class instances) should be avoided in favor of concrete data (lists, dicts, scalars) because they add undue mess at little comprehension advantage. eg take the example of a regex match. It returns some object and then we have to scratch our heads wondering whats in the magic box. If instead of match, we use findall, the data is manifest and obvious. http://www.amazon.com/How-Learning-Works-Research-Based-Principles/dp/0470484101 I bought the book based on a recommendation from SciPy talk, and it's really great. As an autodidact, it'll help me teach *myself* better, too. Looks interesting. Will take a look. -- https://mail.python.org/mailman/listinfo/python-list
Re: Using asyncio workers in a `concurrent.futures` interface
cool-RR ram.rac...@gmail.com: If I understand correctly [asyncio] would let me run multiple uploads and downloads efficiently in one thread, which would conserve more resources than using threads. Asyncio does make it convenient to multiplex event on one or more threads. Threads have their uses (exploiting multiple CPUs), only you shouldn't associate threads with every state machine, IMO. Asyncio allows you to separate your state machines from your threads. For example, you might have 1,000 state machines (for 1,000 connections) but only 8 threads for 4 CPUs. Now, I am a little clueless about the whole way it's built, using coroutines and tricky usage of `yield from`. Asyncio actively supports (at least) *two* multiplexing models: callbacks (aka listeners or handlers) and coroutines. Programming with callbacks involves storing the state explicitly in a state machine object. The callbacks are supposed to never block but return immediately. That model is a long-time favorite of many, including me. The coroutine model is highly analogous with the multithreading model in that you store the state implicitly in the code. Multithreading marks the state with blocking function calls. Coroutines mark the state with yield from statements. Otherwise, a multithreading implementation will have very much the shape of a coroutine implementation. The funky aspect of the coroutines is the way they abuse the yield from statement, whose original purpose is to pass a series of results from a generator to the caller by chaining lower-level generators. Coroutines employ a trick: the yield from statement does not pass any meaningful results. Instead, the statement is used to make the generator/coroutine pseudoblock and switch context within the same thread. There's one crucial advantage coroutines have over threads: you can multiplex events. If your thread is blocked on, say, reading a byte, you can't tell it to stop waiting and do something else instead. Coroutines can be made to wait on alternative stimuli. I looked at the asyncio documentation page and saw that it does mention futures and executors, which is my favorite interface for doing concurrency. They are there. My favorite model is the Actor Model, where objects communicate with each other and the outside world through asynchornous stimuli of sorts. The thinking goes: something happened, so how do I react to it. The actor model just needs a class with the member self.state, which contains the name of the internal state of the object. Then the objects callback methods receive the inputs send out messages and adjust the state. Both the actor model and the coroutines produce somewhat messy code that somewhat hard to get right. However, that's a reflection of how messy the reality is. Any attempts to pave it over will ultimately result in more trouble. download_file = lambda url: requests.get(url).content urls = ['http://google.com/file1.jpg', 'http://google.com/file2.jpg', 'http://google.com/file3.jpg'] # etc. with AsyncIOExecutor() as asyncio_executor: files = asyncio_executor.map(download_file, urls) And that's it, no coroutines, no `yield from`. I'm not quite following you. However, one feature of the coroutines is that you must use yield from everywhere. You can't delegate it to a subroutine and forget about it. That's my main problem with coroutines. The simple function call syntax: y = f(x) is replaced with the weird: y = yield from f(x) Since, if I understand correctly, asyncio requires a mainloop, it would make sense for the AsyncIOExecutor to have a thread of its own in which it could run its mainloop. Is this possible? Did someone implement this? Unfortunately, I have not. I have just implemented a toy example to satisfy my curiosity. All of my code uses callbacks and select.epoll() -- to great success. Here's my exploration in the classic dining philosophers problem: URL: http://pacujo.net/marko/philosophers.py I have fixed the faulty protocol by having an assistant break the deadlock by occasionally nagging the philosophers to drop everything and get back to thinking. The main accomplishment of the exercise was that I convinced myself the coroutines can be used for serious things as coroutines can multiplex stimuli with asyncio.wait(..., return_when=asyncio.FIRST_COMPLETED). Marko -- https://mail.python.org/mailman/listinfo/python-list
Begginer in python trying to load a .dll
Hi, Im working in the development of a program based in python that allow us to contrl a spectometer. The spectometer has an .dll file. The idea is to work through this dll and operate the spectometer. The name of the .dll is AS5216.dll. I've trying with ctype, but it doesn't work. I'm begginer in python. All the comments are useful to me. -- https://mail.python.org/mailman/listinfo/python-list
Re: Using asyncio workers in a `concurrent.futures` interface
On 2014-08-12 18:02, cool-RR wrote: Hello everybody! I have a question. I have a Django app running on Heroku. I need to run about 100 worker threads there to do uploads/downloads simultaneously. A Heroku Dyno has only 512MB of memory, so I'm reluctant to run 100 worker threads. (I've had Dynos crash from lack of memory when using 6 threads before.) I heard that the asyncio module is mature and ready for usage, and I was happy because I kept hearing about it in the last year, and I saw Guido's lecture about it. If I understand correctly it would let me run multiple uploads and downloads efficiently in one thread, which would conserve more resources than using threads. (Please correct me if I'm wrong.) Now, I am a little clueless about the whole way it's built, using coroutines and tricky usage of `yield from`. I figured that since this is eventually a library for concurrency, i.e. doing many tasks at the same time, there will be an API in a style of Here are 100 tasks for you to do concurrently, let me know when they're done. I looked at the asyncio documentation page and saw that it does mention futures and executors, which is my favorite interface for doing concurrency. I was happy and I skimmed the docs. But, I couldn't find a simple way to use these. I don't want to learn how to define coroutines and use `yield from` to switch between them. (I use `yield from` regularly and fully understand how it works, I just don't write my programs that way.) What I'm expecting is something like this: download_file = lambda url: requests.get(url).content urls = ['http://google.com/file1.jpg', 'http://google.com/file2.jpg', 'http://google.com/file3.jpg'] # etc. with AsyncIOExecutor() as asyncio_executor: files = asyncio_executor.map(download_file, urls) And that's it, no coroutines, no `yield from`. Since, if I understand correctly, asyncio requires a mainloop, it would make sense for the AsyncIOExecutor to have a thread of its own in which it could run its mainloop. Is this possible? Did someone implement this? Do you really need to upload/download that many at the same time? I'd put them into a queue and have only a few of uploading/downloading at any time. -- https://mail.python.org/mailman/listinfo/python-list
Re: Begginer in python trying to load a .dll
On Wed, Aug 13, 2014 at 4:36 AM, c1223 camiloce...@gmail.com wrote: Im working in the development of a program based in python that allow us to contrl a spectometer. The spectometer has an .dll file. The idea is to work through this dll and operate the spectometer. The name of the .dll is AS5216.dll. I've trying with ctype, but it doesn't work. I'm begginer in python. All the comments are useful to me. What have you tried, exactly? What does it doesn't work mean? Interfacing with a DLL is a potentially very hard and complex job (although it might turn out to be really easy - depends what the DLL's doing). We can't simply give you an answer right here, without knowing heaps about the DLL you're working with - where it came from, whether this is homework or a real-world problem, etc, etc, etc. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Begginer in python trying to load a .dll
On Tue, Aug 12, 2014 at 1:36 PM, c1223 camiloce...@gmail.com wrote: I've trying with ctype, but it doesn't work. Can you post a small example of how you're trying to use cypes? I don't think a full example would be necessary, but knowing the API of one or two functions and how you're trying to call them from Python would be useful. Just as the off-chance your DLL was written in C++... I don't think you can interface with ctypes directly. I think you would have to write a little shim DLL which exposes a C-compatible API. Skip -- https://mail.python.org/mailman/listinfo/python-list
Re: how to get the ordinal number in list
On Tue, 12 Aug 2014 23:39:42 +1000, Chris Angelico wrote: On Tue, Aug 12, 2014 at 8:40 PM, alister alister.nospam.w...@ntlworld.com wrote: On Tue, 12 Aug 2014 00:21:28 -0700, wxjmfauth wrote: [ chomp ] Mathematicians work with numbers (Algebra is a abstraction of numerical concepts) strings are concerned with characters (Arabic numerals are just characters commonly used to represent numbers ) therefore even trying to rationalise any string representation with mathematics is a logical absurdity in itself. Don't bother responding to jmf; he doesn't listen. Also, his posts don't survive the jump to python-list, so we see them only when someone quotes him like that... as far as I'm concerned, his posts are now on par with those trying to sell us solution manuals or herbal remedies. ChrisA As you don't see them unless someone replies I will make sure I don't reply to any more of his posts. I would not want to inflict this nonsense on someone who would otherwise be spared. -- You will lose an important tape file. -- https://mail.python.org/mailman/listinfo/python-list
Re: Begginer in python trying to load a .dll
On 12/08/2014 19:36, c1223 wrote: Hi, Im working in the development of a program based in python that allow us to contrl a spectometer. The spectometer has an .dll file. The idea is to work through this dll and operate the spectometer. The name of the .dll is AS5216.dll. I've trying with ctype, but it doesn't work. I'm begginer in python. All the comments are useful to me. Please help us to help you. Stating it doesn't work is of no use. Show us your code, what you expected to happen and what actually happened. If you get a traceback cut and paste all of it into your message. State what version of Python you have and your OS. Sure seeing dll we can surmise Windows but which one? For further advice on how to put your question(s) together see The SSCCE - Short, Self Contained, Correct (Compilable), Example at http://sscce.org/ -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence -- https://mail.python.org/mailman/listinfo/python-list
Re: Python in financial services
On Tue, 12 Aug 2014 10:48:14 -0700, Rustom Mody wrote: However those folks have thousands of lines of C/C++ which they are porting to python. That begs the question: Why? Seriously, I'd like to know what benefits they expect to achieve by doing so. -- Denis McMahon, denismfmcma...@gmail.com -- https://mail.python.org/mailman/listinfo/python-list
Re: Begginer in python trying to load a .dll
On Tue, Aug 12, 2014 at 11:48 AM, Skip Montanaro s...@pobox.com wrote: Just as the off-chance your DLL was written in C++... I don't think you can interface with ctypes directly. I think you would have to write a little shim DLL which exposes a C-compatible API. You should be able to use ctypes with a C++ library, you'd just need to know how your C++ compiler handles name mangling. Typically, changes are made to accommodate classes, namespaces, and function overloads. Of course, if you were to change to a different compiler (possibly including a different version) or made minor tweaks to the functions, you'd have to update all the Python code to use the new name mangling scheme. Hmm...On second thought, maybe just only using functions exports with extern C would be easier - or just using a C compiler. Chris -- https://mail.python.org/mailman/listinfo/python-list
Re: Begginer in python trying to load a .dll
On Tuesday, August 12, 2014 3:36:17 PM UTC-3, c1234 py wrote: Hi, Im working in the development of a program based in python that allow us to contrl a spectometer. The spectometer has an .dll file. The idea is to work through this dll and operate the spectometer. The name of the .dll is AS5216.dll. I've trying with ctype, but it doesn't work. I'm begginer in python. All the comments are useful to me. I'm using this import ctypes hllDll=ctypes.WinDLL(C:\\AS5216.dll) hllApiProto = ctypes.WINFUNCTYPE (ctypes.c_int,ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p) hllApiParams = (1, p1, 0), (1, p2, 0), (1, p3,0), (1, p4,0), hllApi = hllApiProto ((HLLAPI, hllDll), hllApiParams) p1 = ctypes.c_int (1) p2 = ctypes.c_char_p (sessionVar) p3 = ctypes.c_int (1) p4 = ctypes.c_int (0) hllApi (ctypes.byref (p1), p2, ctypes.byref (p3), ctypes.byref (p4)) Thanks -- https://mail.python.org/mailman/listinfo/python-list
Re: Begginer in python trying to load a .dll
On Tue, 12 Aug 2014 11:56:37 -0700 (PDT) c1234 py camiloce...@gmail.com wrote: On Tuesday, August 12, 2014 3:36:17 PM UTC-3, c1234 py wrote: Hi, Im working in the development of a program based in python that allow us to contrl a spectometer. The spectometer has an .dll file. The idea is to work through this dll and operate the spectometer. The name of the .dll is AS5216.dll. I've trying with ctype, but it doesn't work. I'm begginer in python. All the comments are useful to me. I'm using this import ctypes hllDll=ctypes.WinDLL(C:\\AS5216.dll) hllApiProto = ctypes.WINFUNCTYPE (ctypes.c_int,ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p) hllApiParams = (1, p1, 0), (1, p2, 0), (1, p3,0), (1, p4,0), hllApi = hllApiProto ((HLLAPI, hllDll), hllApiParams) p1 = ctypes.c_int (1) p2 = ctypes.c_char_p (sessionVar) p3 = ctypes.c_int (1) p4 = ctypes.c_int (0) hllApi (ctypes.byref (p1), p2, ctypes.byref (p3), ctypes.byref (p4)) Great. And that fails in what way, on which line, with what error message? -- Rob Gaddi, Highland Technology -- www.highlandtechnology.com Email address domain is currently out of order. See above to fix. -- https://mail.python.org/mailman/listinfo/python-list
Re: Begginer in python trying to load a .dll
On Wed, Aug 13, 2014 at 4:58 AM, Rob Gaddi rgaddi@technologyhighland.invalid wrote: Great. And that fails in what way, on which line, with what error message? And, is that the entire program? Because if it is, then I can see at least one problem, a NameError. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Python in financial services
On Wednesday, August 13, 2014 12:24:12 AM UTC+5:30, Denis McMahon wrote: On Tue, 12 Aug 2014 10:48:14 -0700, Rustom Mody wrote: However those folks have thousands of lines of C/C++ which they are porting to python. That begs the question: Why? Seriously, I'd like to know what benefits they expect to achieve by doing so. Heh! I asked more or less that. Was told: We want it! I gather its to do with agility -- faster time to incorporate and integrate new logic into code etc. -- https://mail.python.org/mailman/listinfo/python-list
Re: Begginer in python trying to load a .dll
On Tuesday, August 12, 2014 4:09:13 PM UTC-3, Chris Angelico wrote:
On Wed, Aug 13, 2014 at 4:58 AM, Rob Gaddi
rgaddi@technologyhighland.invalid wrote:
Great. And that fails in what way, on which line, with what error message?
And, is that the entire program? Because if it is, then I can see at
least one problem, a NameError.
ChrisA
This appear in the terminal:
runfile('C://Python Scripts')
File C:\\sitecustomize.py, line 585, in runfile
execfile(filename, namespace)
File C://Sin título 38.py, line 19, in module
hllApi = hllApiProto ((HLLAPI, hllDll), hllApiParams)
AttributeError: function 'HLLAPI' not found
Traceback (most recent call last):
File stdin, line 1, in module
--
https://mail.python.org/mailman/listinfo/python-list
Re: how to get the ordinal number in list
On 12/08/2014 19:45, alister wrote: On Tue, 12 Aug 2014 23:39:42 +1000, Chris Angelico wrote: On Tue, Aug 12, 2014 at 8:40 PM, alister alister.nospam.w...@ntlworld.com wrote: On Tue, 12 Aug 2014 00:21:28 -0700, wxjmfauth wrote: [ chomp ] Mathematicians work with numbers (Algebra is a abstraction of numerical concepts) strings are concerned with characters (Arabic numerals are just characters commonly used to represent numbers ) therefore even trying to rationalise any string representation with mathematics is a logical absurdity in itself. Don't bother responding to jmf; he doesn't listen. Also, his posts don't survive the jump to python-list, so we see them only when someone quotes him like that... as far as I'm concerned, his posts are now on par with those trying to sell us solution manuals or herbal remedies. ChrisA As you don't see them unless someone replies I will make sure I don't reply to any more of his posts. I would not want to inflict this nonsense on someone who would otherwise be spared. Thank you, as it saves me from once again losing it in public when his latest incarnation of complete dross appears. Two years he's been at it, I wonder how he manages to avoid Sent to jail. Do not pass Go. Do not collect £200 or where he gets his supply of Get out of jail free cards from? -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence -- https://mail.python.org/mailman/listinfo/python-list
Re: Begginer in python trying to load a .dll
Am 12.08.14 20:36, schrieb c1223: Hi, Im working in the development of a program based in python that allow us to contrl a spectometer. The spectometer has an .dll file. The idea is to work through this dll and operate the spectometer. The name of the .dll is AS5216.dll. I've trying with ctype, but it doesn't work. I'm begginer in python. All the comments are useful to me. For sure this DLL will come with a header file for C. You could pass it through SWIG and see if it generates a useful Python module. You need a C compiler for that route, though. Christian -- https://mail.python.org/mailman/listinfo/python-list
Re: how to get the ordinal number in list
On 8/12/2014 2:20 PM, Rustom Mody wrote: On Tuesday, August 12, 2014 11:10:48 PM UTC+5:30, Neil D. Cerutti wrote: Beginners are particularly poor, in relation to experts, at noticing the applicability of idea, and at combining ideas together. Breaking things into component parts has multiple benefits: 1. The applicability of individual ideas becomes obvious. It's one thing to know about [].sort, and another thing to know when it's appropriate to sort something. 2. The expert specifically shows how and why the ideas are combined. This helps build the connection for the beginner, whose knowledge is not stored as an expert stores it; i.e, in broad categories with multiple connections; but as disorganized data with very few connections. Nice! And how do we lead the beginners towards expertise? In a way functional programming is to programming creativity what lego is to children's spatial creativity. Specifically there are a bunch of pieces that need to fit: 1. Functional Programming: Nothing more than composing functions [Maybe a bit simplistic but not unrealistic a defn] 2. Trying this out at the interpreter 3. Introspectable objects Functional programming could be particularly hard to teach since it is generally made up of numerous small units of work combined in a complex way. This is precisely the formula for something that beginners will find extremely challenging. When functional programming is dumbed down enough for a beginner to be able to grok it, the programming problems start to look really lame. It needn't be that way, of course, but it takes a good deal of creativity on the part of the instructor. If Factorial doesn't turn you on, you might be screwed. ;) Some things follow from this: For the lego-game of playing with functions at the REPL to work and be pleasant and rewarding: 1. functions should be non side-effecting; else same trials giving different answers adds more confusion than understanding 2. They should be non-printing else: def foo(x): return x+1 def bar(x): print x+1 look similar when trivially tried but compositionally are utterly different In effect a printing function breaks the lego bricks That may be so, but printing stuff to the screen is very natural to people. I've downloaded Haskell a few times, but the knowledge that getting input and writing output requires something mysterious called gonads just frightens me. [The P in the REPL is DRY enough that it does not usually need to be repeated all over] A good REPL does help a lot, though. 3. Abstractions (class instances) should be avoided in favor of concrete data (lists, dicts, scalars) because they add undue mess at little comprehension advantage. eg take the example of a regex match. It returns some object and then we have to scratch our heads wondering whats in the magic box. If instead of match, we use findall, the data is manifest and obvious. I'm with you on regex: match objects suck. That and escaping. -- Neil Cerutti -- https://mail.python.org/mailman/listinfo/python-list
Re: Begginer in python trying to load a .dll
El martes, 12 de agosto de 2014 16:16:21 UTC-3, Christian Gollwitzer escribió: Am 12.08.14 20:36, schrieb c1223: Hi, Im working in the development of a program based in python that allow us to contrl a spectometer. The spectometer has an .dll file. The idea is to work through this dll and operate the spectometer. The name of the .dll is AS5216.dll. I've trying with ctype, but it doesn't work. I'm begginer in python. All the comments are useful to me. For sure this DLL will come with a header file for C. You could pass it through SWIG and see if it generates a useful Python module. You need a C compiler for that route, though. Christian How can i pass it through a SWIG?, i don't know that and what it mean? -- https://mail.python.org/mailman/listinfo/python-list
Suitable Python code to scrape specific details from web pages.
Dear Programmers,
I have been looking at the You tube 'Web Scraping Tutorials' of Chris Reeves. I
have tried a few of his python programs in the Python27 command prompt, but
altered them from accessing data using links say from the Dow Jones index, to
accessing the details I would be interested in accessing from the 'Racing Post'
on a daily basis. Anyhow, the code it returns is not in the example I am going
to give, is not the information I am seeking, instead of returning the given
odds on a horse, it only returns a [], which isn't much use.
I would be glad if you could tell me where I am going wrong.
Yours faithfully
Simon Evans.
import urllib
import re
htmlfile = urllib.urlopen(http://www.racingpost.com/horses2/cards/card.sd?
race_id=600048r_date=2014-05-08#raceTabs=sc_)
htmltext = htmlfile.read()
regex = 'strong1a href=http://www.racingpost.com/horses/horse_home.sd?
horse_id=758752onclick=scorecards.send(quot;horse_namequot:):return
Html.popup(this,
{width:695,height:800})title=Full details about this HORSELively
Baron/a9/4F/strongbr/'
pattern = re.compile(regex)
odds=re.findall(pattern,htmltext)
print odds
[]
import urllib
import re
htmlfile = urllib.urlopen(http://www.racingpost.com/horses2/cards/card.sd?
race_id=600048r_date=2014-05-08#raceTabs=sc_)
htmltext = htmlfile.read()
regex = 'a/a'
pattern = re.compile(regex)
odds=re.findall(pattern,htmltext)
print odds
[]
---
--
https://mail.python.org/mailman/listinfo/python-list
Re: Suitable Python code to scrape specific details from web pages.
On Tue, 12 Aug 2014 13:00:30 -0700 (PDT)
Simon Evans musicalhack...@yahoo.co.uk wrote:
Dear Programmers,
I have been looking at the You tube 'Web Scraping Tutorials' of Chris Reeves.
I have tried a few of his python programs in the Python27 command prompt, but
altered them from accessing data using links say from the Dow Jones index, to
accessing the details I would be interested in accessing from the 'Racing
Post' on a daily basis. Anyhow, the code it returns is not in the example I
am going to give, is not the information I am seeking, instead of returning
the given odds on a horse, it only returns a [], which isn't much use.
I would be glad if you could tell me where I am going wrong.
Yours faithfully
Simon Evans.
import urllib
import re
htmlfile = urllib.urlopen(http://www.racingpost.com/horses2/cards/card.sd?
race_id=600048r_date=2014-05-08#raceTabs=sc_)
htmltext = htmlfile.read()
regex = 'strong1a href=http://www.racingpost.com/horses/horse_home.sd?
horse_id=758752onclick=scorecards.send(quot;horse_namequot:):return
Html.popup(this,
{width:695,height:800})title=Full details about this HORSELively
Baron/a9/4F/strongbr/'
pattern = re.compile(regex)
odds=re.findall(pattern,htmltext)
print odds
[]
import urllib
import re
htmlfile = urllib.urlopen(http://www.racingpost.com/horses2/cards/card.sd?
race_id=600048r_date=2014-05-08#raceTabs=sc_)
htmltext = htmlfile.read()
regex = 'a/a'
pattern = re.compile(regex)
odds=re.findall(pattern,htmltext)
print odds
[]
---
If you want web scraping, you want to use
http://www.crummy.com/software/BeautifulSoup/ . End of story.
--
Rob Gaddi, Highland Technology -- www.highlandtechnology.com
Email address domain is currently out of order. See above to fix.
--
https://mail.python.org/mailman/listinfo/python-list
Re: Begginer in python trying to load a .dll
On 12/08/2014 20:25, c1234 py wrote: El martes, 12 de agosto de 2014 16:16:21 UTC-3, Christian Gollwitzer escribió: Am 12.08.14 20:36, schrieb c1223: Hi, Im working in the development of a program based in python that allow us to contrl a spectometer. The spectometer has an .dll file. The idea is to work through this dll and operate the spectometer. The name of the .dll is AS5216.dll. I've trying with ctype, but it doesn't work. I'm begginer in python. All the comments are useful to me. For sure this DLL will come with a header file for C. You could pass it through SWIG and see if it generates a useful Python module. You need a C compiler for that route, though. Christian How can i pass it through a SWIG?, i don't know that and what it mean? Start here http://www.swig.org/ which you could easily have found for yourself, as I did by using a search engine. Also would you please access this list via https://mail.python.org/mailman/listinfo/python-list or read and action this https://wiki.python.org/moin/GoogleGroupsPython to prevent us seeing double line spacing and single line paragraphs, thanks. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence -- https://mail.python.org/mailman/listinfo/python-list
Re: Python in financial services
On Tue, Aug 12, 2014 at 3:33 AM, Rustom Mody rustompm...@gmail.com wrote: Ive been asked to formulate a python course for financial services folk. If I actually knew about the subject, I'd have fatter pockets! Anyway heres some thoughts. What I am missing out? [Apart from basic python -- contents typically needs tailoring to the audience] the following: - Libraries -- Decimal? - scripts -- philosophy and infrastructure eg argparse, os.path - Pandas - Numpy Scipy (which? how much?) - ipython + matplotlib + ?? - Database interfacing - Excel interfacing (couple of libraries.. which?) - C(C++?) interfacing paradigms -- ranging from ctypes, cython to classic lo-level I'm not 100% sure what you're looking for. I work for a hedge fund and we make extensive use of python. Everything from soup to nuts: ETL, web scraping, database access (sybase, MySQL, and Oracle), log file archiving and reaping, wrappers for backups, startup and shutdown scripts for our C++ servers, GIUs (with wxpython), socket based communication with C++ servers, just about every problem that comes up. -- https://mail.python.org/mailman/listinfo/python-list
Re: Suitable Python code to scrape specific details from web pages.
In article a8f10c4f-d4a0-48ed-ae92-2a43e9a09...@googlegroups.com, Simon Evans musicalhack...@yahoo.co.uk wrote: Dear Programmers, I have been looking at the You tube 'Web Scraping Tutorials' of Chris Reeves. I have tried a few of his python programs in the Python27 command prompt, but altered them from accessing data using links say from the Dow Jones index, to accessing the details I would be interested in accessing from the 'Racing Post' on a daily basis. Anyhow, the code it returns is not in the example I am going to give, is not the information I am seeking, instead of returning the given odds on a horse, it only returns a [], which isn't much use. I would be glad if you could tell me where I am going wrong. Rather than comment on your specific code (but, thank you for posting it), I'll make a couple of more generic suggestions. First, if you're doing anything with fetching web pages, install the wonderful requests module (http://docs.python-requests.org/en/latest/). It's so much easier to work with than urllib. Second, if you're going to be parsing web pages, trying to use regexes is a losing game. You need something that knows how to parse HTML. The canonical answer is lxml (http://lxml.de/), but Beautiful Soup (http://www.crummy.com/software/BeautifulSoup/) is less intimidating to use. -- https://mail.python.org/mailman/listinfo/python-list
Re: Using asyncio workers in a `concurrent.futures` interface
On Tue, Aug 12, 2014 at 11:02 AM, cool-RR ram.rac...@gmail.com wrote: And that's it, no coroutines, no `yield from`. Since, if I understand correctly, asyncio requires a mainloop, it would make sense for the AsyncIOExecutor to have a thread of its own in which it could run its mainloop. I think that putting the event loop in a separate thread would be necessary if the intention is that the executor be invoked from outside. I'm not aware of this work having been done, but it sounds perfectly feasible. -- https://mail.python.org/mailman/listinfo/python-list
Re: how to get the ordinal number in list
On 8/12/2014 1:40 PM, Neil D. Cerutti wrote: I disagree. For a beginner, you want to be able to break things down into individual steps and examine the result at each point. If you do: l= [6,2,9,12,1,4] l2 = sorted(l,reverse=True) you have the advantage that you can stop after creating l2 and print it out. The student can see that it has indeed been sorted. Or one can put the multiple steps in a file and step through with a debugger that shows local name values, like the Idle debugger. -- Terry Jan Reedy -- https://mail.python.org/mailman/listinfo/python-list
Re: Suitable Python code to scrape specific details from web pages.
On Tuesday, August 12, 2014 9:00:30 PM UTC+1, Simon Evans wrote:
Dear Programmers,
I have been looking at the You tube 'Web Scraping Tutorials' of Chris Reeves.
I have tried a few of his python programs in the Python27 command prompt, but
altered them from accessing data using links say from the Dow Jones index, to
accessing the details I would be interested in accessing from the 'Racing
Post' on a daily basis. Anyhow, the code it returns is not in the example I
am going to give, is not the information I am seeking, instead of returning
the given odds on a horse, it only returns a [], which isn't much use.
I would be glad if you could tell me where I am going wrong.
Yours faithfully
Simon Evans.
import urllib
import re
htmlfile = urllib.urlopen(http://www.racingpost.com/horses2/cards/card.sd?
race_id=600048r_date=2014-05-08#raceTabs=sc_)
htmltext = htmlfile.read()
regex = 'strong1a href=http://www.racingpost.com/horses/horse_home.sd?
horse_id=758752onclick=scorecards.send(quot;horse_namequot:):return
Html.popup(this,
{width:695,height:800})title=Full details about this HORSELively
Baron/a9/4F/strongbr/'
pattern = re.compile(regex)
odds=re.findall(pattern,htmltext)
print odds
[]
import urllib
import re
htmlfile = urllib.urlopen(http://www.racingpost.com/horses2/cards/card.sd?
race_id=600048r_date=2014-05-08#raceTabs=sc_)
htmltext = htmlfile.read()
regex = 'a/a'
pattern = re.compile(regex)
odds=re.findall(pattern,htmltext)
print odds
[]
---
Dear Programmers, Thank you for your responses. I have installed 'Beautiful
Soup' and I have the 'Getting Started in Beautiful Soup' book, but can't seem
to make any progress with it, I am too thick to make much use of it. I was
hoping I could scrape specified stuff off Web pages without using it. I have
installed 'Requests' also, is there any code I can use that you can suggest
that can access the sort of Web page values that I have referred to ? such as
odds, names of runners, stuff like that off the 'inspect element' or 'source'
htaml pages, on www.Racingpost.com.
--
https://mail.python.org/mailman/listinfo/python-list
Re: Suitable Python code to scrape specific details from web pages.
Simon Evans wrote: Dear Programmers, Thank you for your responses. I have installed 'Beautiful Soup' and I have the 'Getting Started in Beautiful Soup' book, but can't seem to make any progress with it, I am too thick to make much use of it. I was hoping I could scrape specified stuff off Web pages without using it. Yes, you can scrape stuff off web pages without programming. What you do is you open the web page in your browser, then open a notebook and, with a pencil or pen, copy the bits you read into the notebook. If you're very skilled, you can avoid the pencil and paper and type directly into a text editor on the computer. But other than that, every website is different, so there is no short-cut to web scraping. You need to customize the scraping code for each website you scrape, and that means programming. Do you know how to program? Are you interested in learning? If the answer is No and No, then I suggestion you pony up some money and pay somebody who already knows how to program to do the job for you. If the answer is No and Yes, then start at the beginning. Do some programming tutorials, learn to program the basics before moving on to something moderately difficult like web scraping. If the answer is that you already know how to program, but just don't know how to do web scraping, then stick with it and you'll get there. Web scraping is tricky, but possible, and if you work hard at it you'll succeed. Unless you're an experienced programmer with all the right skills, don't expect this to be something you do in a few minutes. Depending on your level of experience, you could expect to spend dozens of hours to learn how to scrape a single website. (Fortunately, the second website will probably be a little easier, and the third easier still. By the time you've done a dozen, you'll wonder what the fuss was about.) By studying how other scraping programs work, and studying how your racing pages store data, you should be able to put the two together and see how to get the data you want. There's plenty of information to help you learn how to web scrape, with or without BeautifulSoup: https://startpage.com/do/search/?q=beautifulsoup+web+scraping https://ixquick.com/do/search/?q=python+web+scraping+examples https://duckduckgo.com/html/?q=requests%20python%20web%20scraping%20example but no alternative to actually writing code. I have installed 'Requests' also, is there any code I can use that you can suggest that can access the sort of Web page values that I have referred to ? such as odds, names of runners, stuff like that off the 'inspect element' or 'source' htaml pages, on www.Racingpost.com. Specifically those pages? Doubtful. If you are really lucky (1) somebody else has already done the programming, (2) they've made their program available to others, and (3) you can find that program on the Internet. Use the search engine of your choice to search for it. -- Steven -- https://mail.python.org/mailman/listinfo/python-list
Re: Suitable Python code to scrape specific details from web pages.
In article 53eaab7d$0$29979$c3e8da3$54964...@news.astraweb.com, Steven D'Aprano steve+comp.lang.pyt...@pearwood.info wrote: By studying how other scraping programs work, and studying how your racing pages store data, you should be able to put the two together and see how to get the data you want. It's also worth mentioning, that some web sites *want* you to have their data, and make it easy to do so by exposing it via public APIs or other download methods. Wikipedia. Many government web sites. Twitter. Facebook. Reddit. Whenever you start thinking about web scraping, it's always worth spending a little time investigating if such an API exists. If it does, that's where you want to go. If not, well, there's always Beautiful Soup :-) -- https://mail.python.org/mailman/listinfo/python-list
Re: how to get the ordinal number in list
Chris Angelico wrote: On Mon, Aug 11, 2014 at 7:44 PM, Steven D'Aprano st...@pearwood.info wrote: I think this is why both declarative and functional programming idioms will remain niche (although important niches). Most tasks are inherently imperative to at least some degree, and often a *great* degree. Maybe that's true as a whole, but there are certainly ways in which certain declarative or functional elements can be extremely useful to an otherwise-imperative program. Oh yes! I think that using functional idioms is *very* valuable. List comps and similar are great, but more than that, the idea of writing idempotent functions with no side-effects (except IO) is (I believe) vital for good programming. Relying on argument passing rather communicating by global variables (or the OO equivalent, instance attributes) is likewise vital. It's not *quite* impossible to write good, reliable code without functional idioms, in the same sense that it is conceivable that somebody might write bug-free unstructured spaghetti code, but I think one of the failures of OO programming is the over-reliance on methods that communicate via side-effects. I don't insist that side-effects are completely verboten, as some functional programming purists do, but I do endeavour to ensure that any side-effects are limited, encapsulated, and introduce as little coupling as practical. (It is my belief that inappropriate coupling is the great evil in programming. Having too much coupling between parts of your code which ought to be independent is like Ebola for reliable code, contagious and deadly. I'm always looking for ways to reduce coupling between parts of my code, and functional idioms are good for that.) Python's (list etc) comprehensions are broadly functional in style, No surprise, since list comps were stolen from Haskell :-) [...] I've sometimes done some extremely declarative coding in places; my MUD client builds its menus by looking for specially-named functions and getting some metadata from them to work out what the menu item name should be. An approximate Python equivalent would be: @filemenu(E_xit) def exit(): prompt(Do you really want to exit?) where the presence of the function is what causes the menu item to exist. In this case, the 'filemenu' decorator is probably imperative code that adds the menu item to the File menu, but if you're reading through a source file that has a bunch of functions like that, you'd have to agree that that's a declarative style of coding. And it's a style that works well for this kind of thing. I don't think I would agree that's declarative style. I think that's a form of imperative programming, where the syntax is: @ make menu def function: ... rather than: def function: ... make menu (function) but I wouldn't start a Holy War over it :-) There's a fair amount of overlap between the major programming paradigms, and hence disagreement as to what falls under which paradigm. For instance, FOLDOC has a good, simple distinction in its definition for imperative: [quote] The Free On-line Dictionary of Computing (20 July 2014) [foldoc] imperative language imperative imperative programming language Any programming language that specifies explicit manipulation of the state of the computer system, not to be confused with a procedural language, which specifies an explicit sequence of steps to perform. An example of an imperative (but non-procedural) language is a data manipulation language for a relational database management system. This specifies changes to the database but does not necessarily require anyone to specify a sequence of steps. Both contrast with declarative languages, which specify neither explicit state manipulation nor a sequence of steps. [end quote] You'll note that it suggests SQL would count as an imperative language, but Wikipedia's article on declarative languages gives SQL as a paragon of declarative languages! http://en.wikipedia.org/wiki/Declarative_programming In Python terms, we might consider import to be declarative, since it specifies what to do (load a name from a module) but not how to perform it. import spam might find spam anywhere, in any form (source code, byte code, machine code, in a zip file, somewhere on the disk, inside the Python executable itself), which to my mind suggests a declarative idiom. Or such things as test discovery, where unittest and doctest will automatically locate and run tests. (The tests themselves are typically written in a procedural paradigm.) Otherwise, Python doesn't really have a lot to offer in the declarative paradigm. -- Steven -- https://mail.python.org/mailman/listinfo/python-list
Re: Suitable Python code to scrape specific details from web pages.
On Tue, 12 Aug 2014 15:44:58 -0700 (PDT), Simon Evans wrote:
[snip]
Dear Programmers, Thank you for your responses. I have installed
'Beautiful Soup' and I have the 'Getting Started in Beautiful Soup'
book, but can't seem to make any progress with it, I am too thick to
make much use of it. I was hoping I could scrape specified stuff off
Web pages without using it.
I've only used BeautifulSoup a little bit, and am no expert, but
with it one can do wonderfully complex things with simple code.
Perhaps you can find some examples online; this newsgroup sometimes
has awesome demonstrations of BS prowess.
At the risk of embarrassing myself in public, I'll show you some
code I wrote that scrapes data from a web page containing a
description of a drug. The drug's web page contains the desired
data in tags that look like this:
input id=form-widgets-minconcentration name=form.widgets.minconcentration
class=text-widget float-field value=1.0 type=text /
The following code finds all these tags and builds a dict by which you
can lookup the value for any given name.
from BeautifulSoup import BeautifulSoup as BS
...
def dump_drug_data(url):
Fetch data from one drug's URL and print selected fields in columns.
contents = urllib2.urlopen(url=url).read()
soup = BS(contents)
inputs = soup.findAll(input)
input_dict = dict((i.get(name), i.get(value)) for i in inputs)
print( .join(f.format(input_dict[n]) for f, n in (
({0:5s}, form.widgets.absorption_halflife),
({0:5s}, form.widgets.elimination_halflife),
({0:5s}, form.widgets.minconcentration),
({0:5s}, form.widgets.maxconcentration),
({0:13s}, form.widgets.title),
)))
Try giving a more specific picture of your quest, and it's very
likely that people smarter than me will give you good help.
--
To email me, substitute nowhere-spamcop, invalid-net.
--
https://mail.python.org/mailman/listinfo/python-list
Re: how to get the ordinal number in list
On Wed, Aug 13, 2014 at 10:47 AM, Steven D'Aprano
steve+comp.lang.pyt...@pearwood.info wrote:
Chris Angelico wrote:
On Mon, Aug 11, 2014 at 7:44 PM, Steven D'Aprano st...@pearwood.info
wrote:
I think this is why both declarative and functional programming idioms
will remain niche (although important niches). Most tasks are inherently
imperative to at least some degree, and often a *great* degree.
Maybe that's true as a whole, but there are certainly ways in which
certain declarative or functional elements can be extremely useful to
an otherwise-imperative program.
Oh yes! I think that using functional idioms is *very* valuable. List comps
and similar are great, but more than that, the idea of writing idempotent
functions with no side-effects (except IO) is (I believe) vital for good
programming. Relying on argument passing rather communicating by global
variables (or the OO equivalent, instance attributes) is likewise vital.
...
(It is my belief that inappropriate coupling is the great evil in
programming. Having too much coupling between parts of your code which
ought to be independent is like Ebola for reliable code, contagious and
deadly. I'm always looking for ways to reduce coupling between parts of my
code, and functional idioms are good for that.)
There are times when coupling is unavoidable, but yes, it's usually
detrimental to the code. Sometimes the alternatives are worse, but
coupling is still bad. I have TODOs against certain pieces of code
where I can't think of a clean way to decouple them...
I've sometimes done some extremely declarative coding in places; my
MUD client builds its menus by looking for specially-named functions
and getting some metadata from them to work out what the menu item
name should be. An approximate Python equivalent would be:
@filemenu(E_xit)
def exit():
prompt(Do you really want to exit?)
where the presence of the function is what causes the menu item to
exist. In this case, the 'filemenu' decorator is probably imperative
code that adds the menu item to the File menu, but if you're reading
through a source file that has a bunch of functions like that, you'd
have to agree that that's a declarative style of coding. And it's a
style that works well for this kind of thing.
I don't think I would agree that's declarative style. I think that's a form
of imperative programming, where the syntax is:
@ make menu
def function: ...
rather than:
def function: ...
make menu (function)
but I wouldn't start a Holy War over it :-)
In a sense, yes. That's partly because Python doesn't really have the
concept that I'm using here. Here's the actual code (it's Pike):
constant file_closewindow=E_xit;
int closewindow()
{
... actual body of function isn't significant ...
}
The key here is the module-level constant which is file_ followed by
the name of a function. So a more direct translation into Python would
be:
file_closewindow=E_xit
def closewindow():
... blah blah ...
but since Python code tends to have uppercase constants, this would be
frowned on. Also, it's hard in Python to enumerate constants, as
there's no such thing (Pike's constants are available at a slightly
different level, which makes them easy to find).
What might make it more declarative than imperative, even in Python,
is that the *order* of items on the menu is alphabetical by function
name, and thus has nothing to do with their position in the file. If
the decorator actually added the items to the menu immediately, their
positions would be based on order in the file; to do them
alphabetically by function name, you basically need to collect them
all up and create them all at the end. That's how the code works in
Pike - a quick loop in the initialization code runs over the constants
and adds the menu items, which means the constants are completely
non-executable.
It really is hard to draw the line. Ultimately, the CPU (at least, if
it's of any architecture I've worked with) doesn't execute declarative
or functional code, just as it doesn't execute callbacks or subthread
multiplexing or anything like that. You need to have, somewhere, a bit
of imperative code (usually a simple loop) that deals with the other
types... which might be called an interpreter. In fact, *all* high
level code is, in a sense, declarations to a lower level interpreter -
consider the distinctions between various Python interpreters (even
some that run inside your web browser), and how they all obey the same
declarations, but implement them completely differently.
[ quoting FOLDOC ]
An example of an imperative (but non-procedural) language is a
data manipulation language for a relational database
management system. This specifies changes to the database
but does not necessarily require anyone to specify a sequence
of steps.
Both contrast with declarative languages, which specify
neither explicit state manipulation nor a sequence of
Re: Captcha identify
If my questions make you guys not so happy, I am sorry and please just ignore. I just wanna a general suggestion here in the beginning. Why I need to write such program is just having such requirements, and has nothing to do with the coding work itself. Don't say something to prove you're so noble. -- https://mail.python.org/mailman/listinfo/python-list
Re: Captcha identify
On Wed, Aug 13, 2014 at 11:36 AM, Wesley nisp...@gmail.com wrote: If my questions make you guys not so happy, I am sorry and please just ignore. I just wanna a general suggestion here in the beginning. Why I need to write such program is just having such requirements, and has nothing to do with the coding work itself. Don't say something to prove you're so noble. The general suggestion you're getting is: Do not do this. Many of us here use CAPTCHAs and spend time keeping one step ahead of those who try to break them with software. By writing something to solve CAPTCHAs, you would be stealing time from those people. Don't do it. Am I sufficiently clear? ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: how to get the ordinal number in list
In article mailman.12903.1407893523.18130.python-l...@python.org, Chris Angelico ros...@gmail.com wrote: I like to look at SQL as a language that specifies an end result without specifying how to get there Well, sure, but sometimes the how to get there is a matter of 10x, or 100x, or 1000x in performance. I'm currently migrating a 3 TB database to a new 5 TB RAID array. Our initial guess is it's going to take two weeks to finish :-( -- https://mail.python.org/mailman/listinfo/python-list
Re: Captcha identify
Wesley nisp...@gmail.com writes: If my questions make you guys not so happy, I am sorry and please just ignore. You seek to dismiss the valid concerns by calling them “not so happy”. You assert it is questions we object to, when we are clearly objecting to your intentions. Don't be disingenuous. The responses you get are because there are valid concerns about the motives you are pursuing. Why I need to write such program is just having such requirements This is a circular statement, almost a tautology. If that is the best you can present in defense of harmful and deceitful actions, then there is no good reason for what you're doing. Stop, please. -- \“The World is not dangerous because of those who do harm but | `\ because of those who look at it without doing anything.” | _o__) —Albert Einstein | Ben Finney -- https://mail.python.org/mailman/listinfo/python-list
Re: how to get the ordinal number in list
On Wed, Aug 13, 2014 at 11:45 AM, Roy Smith r...@panix.com wrote: In article mailman.12903.1407893523.18130.python-l...@python.org, Chris Angelico ros...@gmail.com wrote: I like to look at SQL as a language that specifies an end result without specifying how to get there Well, sure, but sometimes the how to get there is a matter of 10x, or 100x, or 1000x in performance. Of course it can! But generally, the how to get there is not stipulated in the SQL statement. If you say, for instance: select * from some_table where some_column='some_value'; then an index on some_column will make a huge difference to performance - without changing this statement at all. In contrast, taking advantage of an index in BTrieve requires recoding your program, as the index used is a part of the API call. (At least, this is true of the particular version of BTrieve that back-ended our accounting package in the 90s. I was able to turn an overnight job into a thirty-second job by rewriting it as an external program; the original is closed-source so I can't be sure, but I suspect most of that improvement is simply because I used an index.) SQL itself doesn't even have provision for indexes. The 'CREATE INDEX' command, found in many databases, is an extension. :) ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: how to get the ordinal number in list
Rustom Mody wrote: I guess part of the problem is that evidently you and I use 'concrete' and 'abstract' in almost diametrically opposite ways. [...] But now you are turning the tables and saying that the FP types/constructs are more abstract than the conventional ones. [...] In my book, FP is easy and pleasant and smooth because (among other things) FP types are concrete, whereas OOP types are abstract. I wrote about how your view of sorted() was an abstraction, not an Abstract Base Class. I thought that was clear enough: you ignore the concrete details of how one sorts (bubblesort, quicksort, timsort, ...) and everything but the result returned. I shouldn't think that is controversial. Let me repeat what I said at the time: You are abstracting away the details of the real computer implementation into an abstraction of a pure mathematical function... I am not talking about *abstract types* in the object-oriented sense. I then went on to say: But abstractions leak, and someday someone is going to ask Why does it take 45 minutes to find the five largest values of my list?, and if you persist in insisting that sorted() is a pure mathematical function you will have no clue on how to even begin solving this [...] (I don't actually believe that *you personally* are incapable of thinking about the implementation of sorted(). It is a hypothetical situation.) For the record, Wikipedia defines abstract types as: In programming languages, an abstract type is a type in a nominative type system which cannot be instantiated directly. Abstract types are also known as existential types. and contrasts them with concrete types like Python's float, list, str. I am using abstract in the sense of an abstraction, as opposed to something concrete and real, not as a mechanism for specifying interfaces in Java or Python. Python's float is a leaky abstraction of mathematical Real numbers, and Python's softed() is a leaky abstraction of the concept of sorting a collection. Both are abstractions, because they are *intended* to gloss over real world details of Real numbers and sorting, but both leak because the implementation of the abstraction spoils the abstraction. http://www.joelonsoftware.com/articles/LeakyAbstractions.html -- Steven -- https://mail.python.org/mailman/listinfo/python-list
Re: how to get the ordinal number in list
On Wed, Aug 13, 2014 at 12:11 PM, Steven D'Aprano steve+comp.lang.pyt...@pearwood.info wrote: both leak because the implementation of the abstraction spoils the abstraction. Can you name *any* nontrivial abstraction that doesn't leak? I can't. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: how to get the ordinal number in list
Steven D'Aprano wrote: [...] I am using abstract in the sense of an abstraction, as opposed to something concrete and real, not as a mechanism for specifying interfaces in Java or Python. Python's float is a leaky abstraction of mathematical Real numbers, and Python's softed() is a leaky abstraction of the concept of sorting a collection. Oh, I forgot to mention... sorting a collection is itself an abstraction (that is, a generalisation) of concrete examples of sorting, such as sorting a hand of cards, sorting an unruly crowd of school children into order of height, sorting out a messy cutlery drawer, or sorting the books in a library. Abstractions are good and necessary, being a programmer is about learning to invent and use abstractions, but both object-oriented and functional programming circles are prone to over-abstraction and that's why they're a an ill-fit for teaching beginners. To a beginner, functions and their return result is an abstraction (and quite a hard one for many people to learn). The natural tendency is to write functions like this: def frobnicate(value): print (value + spam + eggs) because they're thinking how do I get the frobnicated value? and the obvious interface between I (the beginner) and the computer is visual. They haven't learned the essential abstraction known as returning a value yet and so they use: - calculate the frobnicated value - print it instead of the generalisation: - calculate the frobnicated value - return[1] it to the caller[2] - which captures that return result somewhere[3] - and finally prints it or possibly this one: - calculate the frobnicated value - return it to the REPL - which prints it then throws it away [1] What does return in the context of functions even mean? Try explaining it to a beginner who doesn't understand functions. The best I can come up with is you just keep showing them examples until they draw the connection themselves. [2] Who? [3] Where? -- Steven -- https://mail.python.org/mailman/listinfo/python-list
Re: how to get the ordinal number in list
On Wednesday, August 13, 2014 12:59:35 AM UTC+5:30, Neil D. Cerutti wrote: Some things follow from this: For the lego-game of playing with functions at the REPL to work and be pleasant and rewarding: 1. functions should be non side-effecting; else same trials giving different answers adds more confusion than understanding 2. They should be non-printing else: def foo(x): return x+1 def bar(x): print x+1 look similar when trivially tried but compositionally are utterly different In effect a printing function breaks the lego bricks That may be so, but printing stuff to the screen is very natural to people. I've downloaded Haskell a few times, but the knowledge that getting input and writing output requires something mysterious called gonads just frightens me. I thought they were quite common wink But yes... - Monads in Haskell - Pointers in C - Inheritance in Java/Python all suffer the same problem -- glorification of the failure-mode. When one uses pointers, especially non-trivial pointer arithmetic in C, one is essentially blasting a hole in the floor of the hi-level facade of C and working at machine level. Inheritance produces coupling whereas the whole intent of modularity is to avoid coupling. Likewise Monads are for imperative programming in Haskell. Maybe better to just use an imperative language? Functional programming could be particularly hard to teach since it is generally made up of numerous small units of work combined in a complex way. This is precisely the formula for something that beginners will find extremely challenging. When functional programming is dumbed down enough for a beginner to be able to grok it, the programming problems start to look really lame. It needn't be that way, of course, but it takes a good deal of creativity on the part of the instructor. If Factorial doesn't turn you on, you might be screwed. ;) Did you see my example a bit earlier on Catalan numbers? I believe it explains them better -- certainly more succinctly -- than the math+verbiage you can find at: http://en.wikipedia.org/wiki/Catalan_number Yes it may be hard to grok but I think its the exact opposite of 'numerous small units' -- https://mail.python.org/mailman/listinfo/python-list
Re: Using asyncio workers in a `concurrent.futures` interface
Ian Kelly ian.g.ke...@gmail.com: On Tue, Aug 12, 2014 at 11:02 AM, cool-RR ram.rac...@gmail.com wrote: And that's it, no coroutines, no `yield from`. Since, if I understand correctly, asyncio requires a mainloop, it would make sense for the AsyncIOExecutor to have a thread of its own in which it could run its mainloop. I think that putting the event loop in a separate thread would be necessary if the intention is that the executor be invoked from outside. I'm not aware of this work having been done, but it sounds perfectly feasible. Multithreading will require normal locking to protect critical sections. Care must be taken to never yield while holding a threading lock. Marko -- https://mail.python.org/mailman/listinfo/python-list
[issue21166] Bus error in pybuilddir.txt 'python -m sysconfigure --generate-posix-vars' build step
koobs added the comment: :DDD This was an awesome experience working with you Ned, thanks for all the help. Attaching my debugging isolation steps for additional detail, posterity and reference. -- Added file: http://bugs.python.org/file36356/python-buildbot-broken-debugging.txt ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21166 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22184] lrucache should reject maxsize as a function
Changes by Serhiy Storchaka storch...@gmail.com: -- nosy: +serhiy.storchaka ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22184 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21725] RFC 6531 (SMTPUTF8) support in smtpd
Serhiy Storchaka added the comment: Adding new parameter in the middle of positional argument list can break existing code. If you want change arguments order, make their keyword-only. -- nosy: +serhiy.storchaka ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21725 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12954] Multiprocessing logging under Windows
Ari Koivula added the comment: I encountered this problem on Python 3.2.5 on Windows and don't think a vague warning about initializing modules is a proper solution. A better solution would be to simply not add multiple handlers, even if log_to_stderr is called more than once. -- nosy: +Ari.Koivula ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12954 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17923] test glob with trailing slash fail on AIX 6.1
Changes by Serhiy Storchaka storch...@gmail.com: -- assignee: - serhiy.storchaka nosy: +serhiy.storchaka stage: patch review - commit review ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17923 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21725] RFC 6531 (SMTPUTF8) support in smtpd
Milan Oberkirch added the comment: Thanks for your review comments, serhiy.storchaka! I may be blind right now, but where did I add a positional parameter? -- Added file: http://bugs.python.org/file36357/issue21725v5.4.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21725 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17923] test glob with trailing slash fail on AIX 6.1
Roundup Robot added the comment: New changeset e430973149ed by Serhiy Storchaka in branch '2.7': Issue #17923: glob() patterns ending with a slash no longer match non-dirs on http://hg.python.org/cpython/rev/e430973149ed New changeset 5033589a752d by Serhiy Storchaka in branch '3.4': Issue #17923: glob() patterns ending with a slash no longer match non-dirs on http://hg.python.org/cpython/rev/5033589a752d New changeset 6a71d3c79653 by Serhiy Storchaka in branch 'default': Issue #17923: glob() patterns ending with a slash no longer match non-dirs on http://hg.python.org/cpython/rev/6a71d3c79653 -- nosy: +python-dev ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17923 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21795] smtpd.SMTPServer should announce 8BITMIME when supported and accept SMTPUTF8 without it
Milan Oberkirch added the comment: Also the server should accept the SMTPUTF8 mail argument completely independently of 8BITMIME if both are enabled. -- title: smtpd.SMTPServer should announce 8BITMIME when supported - smtpd.SMTPServer should announce 8BITMIME when supported and accept SMTPUTF8 without it ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21795 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21725] RFC 6531 (SMTPUTF8) support in smtpd
Milan Oberkirch added the comment: Sorry, I was blind (switching between languages to much)! Anyway there should be no existing code using decode_data as it was introduced in this development circle. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21725 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19494] urllib2.HTTPBasicAuthHandler (or urllib.request.HTTPBasicAuthHandler) doesn't work with GitHub API v3 and similar
Matej Cepl added the comment: Which are all reasons why I believe more conservative approach (i.e., new separate handler) is better. It seems to be a little more silly to have 21 lines long module separately in PyPI for 2.7, but whatever. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19494 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21448] Email Parser use 100% CPU
Roundup Robot added the comment: New changeset ba90bd01c5f1 by Serhiy Storchaka in branch '2.7': Issue #21448: Fixed FeedParser feed() to avoid O(N**2) behavior when parsing long line. http://hg.python.org/cpython/rev/ba90bd01c5f1 New changeset 1b1f92e39462 by Serhiy Storchaka in branch '3.4': Issue #21448: Fixed FeedParser feed() to avoid O(N**2) behavior when parsing long line. http://hg.python.org/cpython/rev/1b1f92e39462 New changeset f296d7d82675 by Serhiy Storchaka in branch 'default': Issue #21448: Fixed FeedParser feed() to avoid O(N**2) behavior when parsing long line. http://hg.python.org/cpython/rev/f296d7d82675 -- nosy: +python-dev ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21448 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21448] Email Parser use 100% CPU
Serhiy Storchaka added the comment: The test_parser.diff file catches the bug in fix_email_parse.diff I don't see this. But well, it does no harm. Please commit fix_prepending2.diff yourself. -- assignee: serhiy.storchaka - rhettinger versions: -Python 2.7, Python 3.4 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21448 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22148] frozen.c should #include importlib.h instead of importlib.h
Brett Cannon added the comment: The key point here is that importlib.h isn't installed with the other header files; it's purely for compilation since it doesn't expose any API. I'm not sure how you tweaked your build environment to break that assumption, but since importlib.h is not expected to be used outside of the build I don't want to make it a header that gets installed with the other headers Python provides for expressing APIs. -- resolution: - not a bug status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22148 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17923] test glob with trailing slash fail on AIX 6.1
Serhiy Storchaka added the comment: Thank you Delhallt. Mark, yes, you are right, but the code can be simplified even more, to do less syscalls. -- resolution: - fixed stage: commit review - resolved status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17923 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11822] Improve disassembly to show embedded code objects
Changes by Josh Lee jlee...@gmail.com: -- nosy: +jleedev ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11822 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22186] Typos in .py files
New submission from Févry Thibault: The patch removes several typos found in .py files. -- assignee: docs@python components: Documentation files: spelling.diff keywords: patch messages: 225232 nosy: docs@python, iwontbecreative priority: normal severity: normal status: open title: Typos in .py files type: enhancement Added file: http://bugs.python.org/file36358/spelling.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22186 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21448] Email Parser use 100% CPU
Roundup Robot added the comment: New changeset 71cb8f605f77 by Serhiy Storchaka in branch '2.7': Decreased memory requirements of new tests added in issue21448. http://hg.python.org/cpython/rev/71cb8f605f77 New changeset c19d3465965f by Serhiy Storchaka in branch '3.4': Decreased memory requirements of new tests added in issue21448. http://hg.python.org/cpython/rev/c19d3465965f New changeset f07b17de3b0d by Serhiy Storchaka in branch 'default': Decreased memory requirements of new tests added in issue21448. http://hg.python.org/cpython/rev/f07b17de3b0d -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21448 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22171] stack smash when using ctypes/libffi to access union
Sean McCully added the comment: For what it is worth, I was not able to reproduce, on the current Python 2.7.8 branch and Mac OS X. ./python2 Python 2.7.8+ (2.7:ba90bd01c5f1, Aug 12 2014, 12:21:58) gcc --version Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1 Apple LLVM version 5.1 (clang-503.0.40) (based on LLVM 3.4svn) Target: x86_64-apple-darwin13.3.0 Thread model: posix Version:5.1.1 (5B1008) Location: /Applications/Xcode.app Applications: Xcode:5.1.1 (5085) Instruments: 5.1.1 (55045) SDKs: OS X: 10.8: (12F37) 10.9: (13C64) iOS: 7.1: (11D167) iOS Simulator: 7.1: (11D167) python2 test.py True -- nosy: +seanmccully ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22171 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22187] commands.mkarg() buggy in East Asian locales
New submission from Jakub Wilk: This is how shell quoting in commands.mkarg() is implemented: def mkarg(x): if '\'' not in x: return ' \'' + x + '\'' s = ' ' for c in x: if c in '\\$`': s = s + '\\' s = s + c s = s + '' return s This is unfortunately not compatible with the way bash splits arguments in some locales. The problem is that in a few East Asian encodings (at least BIG5, BIG5-HKSCS, GB18030, GBK), the 0x5C byte (backslash in ASCII) could be the second byte of a two-byte character; and bash apparently decodes the strings before splitting. PoC: $ sh --version | head -n1 GNU bash, version 4.3.22(1)-release (i486-pc-linux-gnu) $ LC_ALL=C python test-mkargs.py crw-rw-rw- 1 root root 1, 3 Aug 12 16:00 /dev/null ls: cannot access ; python -c 'import this' | grep . | shuf | head -n1 | cowsay -y ; : No such file or directory $ LC_ALL=zh_CN.GBK python test-mkargs.py crw-rw-rw- 1 root root 1, 3 8月 12 16:00 /dev/null ls: 无法访问乗: No such file or directory Simple is better than complex. \ ^__^ \ (..)\___ (__)\ )\/\ ||w | || || sh: 乗: 未找到命令 -- components: Library (Lib) files: test-mkargs.py messages: 225235 nosy: jwilk priority: normal severity: normal status: open title: commands.mkarg() buggy in East Asian locales type: security versions: Python 2.7 Added file: http://bugs.python.org/file36359/test-mkargs.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22187 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21795] smtpd.SMTPServer should announce 8BITMIME when supported and accept SMTPUTF8 without it
Milan Oberkirch added the comment: The proposed patch extends process_message with an **kwargs catch-all which is filled with 'mail_options' and 'rcpt_options' if decode_data is False. I also removed process_smtputf8_message as one can detect the need for supporting SMTPUTF8 from kwargs['mail_options'] now. 8BITMIE is accepted if decode_data=False and SMTPUTF8 is accepted if enable_SMTPUTF8=True (which implies decode_data=False). -- keywords: +patch Added file: http://bugs.python.org/file36360/issue21795.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21795 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22184] lrucache should reject maxsize as a function
Changes by Jakub Wilk jw...@jwilk.net: -- nosy: +jwilk ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22184 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22186] Typos in .py files
Georg Brandl added the comment: Looks good except for this one: -# Remove the 'Current' link, that way we don't accidentally mess +# Remove the 'Current' link, that way we don't accidentaly mess -- nosy: +georg.brandl ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22186 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22186] Typos in .py files
Févry Thibault added the comment: Right, must have done something wrong at some point. Here is an updated diff. -- Added file: http://bugs.python.org/file36361/spelling.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22186 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22187] commands.mkarg() buggy in East Asian locales
R. David Murray added the comment: For the record, neither this module nor this routine exist in python3, so this is a python2 only issue. I'm not sure I fully understand the problem, but perhaps a possible strategy is to apply the fixes to python2's pipes.quote that were applied in python3 (where the function was further moved to shlex), and use that instead of mkarg. -- nosy: +r.david.murray ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22187 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22177] Incorrect version reported after downgrade
Zachary Ware added the comment: 'python.exe' in the installation folder doesn't really matter -- it's just a tiny executable that embeds python27.dll, which contains the interpreter and is installed in C:\Windows\system32. I suspect the installer didn't downgrade python27.dll because the installed version was newer. As for your issue, such an in-place downgrade isn't quite supported (as you see), nor do I think it's a particularly good idea. You should be able to achieve your goals with much less headache by uninstalling 2.7.8 completely with its own installer, then installing 2.7.5 where you want it. Steve: is this something that would be easy/worthwhile to support in future 2.7 installers? -- components: +Installation, Windows nosy: +steve.dower, zach.ware ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22177 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21308] PEP 466: backport ssl changes
Changes by Jakub Wilk jw...@jwilk.net: -- nosy: +jwilk ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21308 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22184] lrucache should reject maxsize as a function
Roundup Robot added the comment: New changeset 780693490c84 by Raymond Hettinger in branch '3.4': Issue 22184: Early detection and reporting of missing lru_cache parameters http://hg.python.org/cpython/rev/780693490c84 -- nosy: +python-dev ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22184 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22184] lrucache should reject maxsize as a function
Changes by Raymond Hettinger raymond.hettin...@gmail.com: -- resolution: - fixed stage: patch review - resolved status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22184 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11822] Improve disassembly to show embedded code objects
Changes by Raymond Hettinger raymond.hettin...@gmail.com: -- stage: needs patch - patch review versions: +Python 3.5 -Python 3.4 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11822 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20729] mailbox.Mailbox does odd hasattr() check
Serhiy Storchaka added the comment: If there are no objections I'll commit this patch. -- assignee: - serhiy.storchaka versions: +Python 3.5 -Python 3.3 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20729 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22014] Add summary table for OS exception - errno mapping
Jon Poler added the comment: I will give this a shot if it is just hard-coding a table with errno names. Dynamically scraping the errno names and inserting them with the exception hierarchy might be beyond me since I've never used Sphinx before. -- nosy: +jon.poler ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22014 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20729] mailbox.Mailbox does odd hasattr() check
R. David Murray added the comment: Committing the patch seems like the right thing to do at this point in time. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20729 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
