Captcha identify

[Wesley]

Hi guys,
  These days I got a small task to identify Captcha characters.
I googled a lot and find some way to do verification code identify.
However, most are for general captcha.

And, for simple captcha, I can use Pytesser.

However, what about those advanced pictures.
I mean:
1.including number and alpha
2.letters might be rotated
3.letters might be deformed

I don't know why I cannot insert picture attachment here...

But I think you guys can think out what the captcha looks like:-)

So, any suggestions?

Thanks.
Wesley
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Linux distros w/o Python in "base" installation

[Ned Deily]

In article 
,
 Chris Angelico  wrote:
> Well, it only *became* contrary to recommended practice in response to
> Arch doing it and everyone seeing the issues it caused :) Personally,
> I'm glad they did. Lets those of us who follow "slower" distros (I'm
> running Debian) get the benefit of someone else's hindsight.

Well, many people thought that the problems caused by changing "python" 
to mean "python3" at this stage in its lifecycle were so obvious that 
there was no need to formalize them in a PEP.  Arch proved us wrong, 
hence PEP 394.

http://legacy.python.org/dev/peps/pep-0394/

-- 
 Ned Deily,
 n...@acm.org

-- 
https://mail.python.org/mailman/listinfo/python-list

Python in financial services

[Rustom Mody]

Ive been asked to formulate a python course for financial services folk.

If I actually knew about the subject, I'd have fatter pockets!
Anyway heres some thoughts. What I am missing out?

[Apart from basic python -- contents typically needs tailoring to the audience] 
the following:

- Libraries -- Decimal?
- scripts -- philosophy and infrastructure eg argparse, os.path
- Pandas
- Numpy Scipy (which? how much?)
- ipython + matplotlib + ??
- Database interfacing
- Excel interfacing (couple of libraries.. which?)
- C(C++?) interfacing paradigms -- ranging from ctypes, cython to classic 
lo-level
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Linux distros w/o Python in "base" installation

[David Palao]

2014-08-11 23:36 GMT+02:00 Ned Deily :
> In article ,
>  Grant Edwards  wrote:
>> Apparently.  Perhaps theres an "enable LSB compliance" option
>> somewhere in the Arch install docs, but I didn't see it...
>
> Also beware that, unlike most other distributions and contrary to
> recommended practice, Arch has chosen to make Python 3 its default, that
> is, when everything is installed, `python` invokes `python3`, rather
> than `python2`.  So you may need to change shebang lines in scripts, etc.
>
> https://wiki.archlinux.org/index.php/python
>
> --
>  Ned Deily,
>  n...@acm.org
>
> --
> https://mail.python.org/mailman/listinfo/python-list

Also Gentoo uses Python3 by default for some months now. The positive
side effect for me has been that I started seriously to switch to
python3.

Best
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Linux distros w/o Python in "base" installation

[Fabien]

On 12.08.2014 09:59, David Palao wrote:

Also Gentoo uses Python3 by default for some months now. The positive
side effect for me has been that I started seriously to switch to
python3.


it's a matter of months for debian/ubuntu to rely only on python3 code 
too, while still respecting PEP 394:


https://wiki.ubuntu.com/Python/3


--
https://mail.python.org/mailman/listinfo/python-list

Re: Linux distros w/o Python in "base" installation

[Chris “Kwpolska” Warrick]

On Mon, Aug 11, 2014 at 8:53 PM, Grant Edwards  wrote:
> I just installed Arch Linux for the first time, and was surprosed to
> find that Python isn't installed as part of a "base" system.  It's
> also not included in the 'base-devel' package group.  It's trivial to
> install, but I'd still pretty surprised it's not there by default.  I
> guess I've spent too much time with Gentoo, Debian, and RedHat
> derivitives which require Python be installed.
>
> I've probably used at least a dozen Linux distros over the years, and
> this is the first time I've noticed that Python wasn't installed by
> default.

Arch has a different idea of “base system”.  The base group contains
the most crucial packages needed to run an Arch Linux system, and that
is all.  And you do not need Python to do so.  Nevertheless, most
people will likely install Python by themselves, or with a package
that depends on Python.

On Mon, Aug 11, 2014 at 9:02 PM, Chris Rebert  wrote:
> It would seem that such distros are opting to not be LSB-compliant?:
> http://refspecs.linuxfoundation.org/LSB_4.1.0/LSB-Languages/LSB-Languages/pylocation.html

Arch does not really care about LSB, AFAIK.

-- 
Chris “Kwpolska” Warrick 
PGP: 5EAAEA16
stop html mail | always bottom-post | only UTF-8 makes sense
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: how to get the ordinal number in list

[alister]

On Tue, 12 Aug 2014 00:21:28 -0700, wxjmfauth wrote:

> Math:
> 
> Try to formalize with mathematics the Flexible String Representation.
> You should quickly realize, it is a logical mathematical absurdity.
> Unbelievable.
> 
> jmf

Mathematicians work with numbers (Algebra is a abstraction of numerical 
concepts) strings are concerned with characters (Arabic numerals are just 
characters commonly used to represent numbers )
therefore even trying to rationalise any string representation with 
mathematics is a logical absurdity in itself.




-- 
Someone on IRC was very sad about the uptime of his machine wrapping
from 497 days to 0.
-- linux-kernel
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: how to write file into my android phone?

[Anssi Saari]

Chris “Kwpolska” Warrick  writes:

>> 2) the phone isn't necessarily visible on a pc as a drive at all.
>>  For example the Samsung gs4.
>
> This is actually true for ALL android devices, starting with Android 3.0.

There was just a guy on comp.mobile.android saying his Android 4.2 phone
(BLU Dash or something like that, extremely Chinese) presents its
external SD card as a USB Mass Storage drive to a PC when connected over
USB. Exactly the thing Google hates and fears and has tried to kill
off... But it's not like Google can force anything on phones running
AOSP Android without Google services and that's China and a few other
large emerging markets. So it's just our small market here in the
western countries where Google has some control...
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: how to get the ordinal number in list

[Chris Angelico]

On Tue, Aug 12, 2014 at 8:40 PM, alister
 wrote:
> On Tue, 12 Aug 2014 00:21:28 -0700, wxjmfauth wrote:
> [ chomp ]
>
> Mathematicians work with numbers (Algebra is a abstraction of numerical
> concepts) strings are concerned with characters (Arabic numerals are just
> characters commonly used to represent numbers )
> therefore even trying to rationalise any string representation with
> mathematics is a logical absurdity in itself.

Don't bother responding to jmf; he doesn't listen. Also, his posts
don't survive the jump to python-list, so we see them only when
someone quotes him like that... as far as I'm concerned, his posts are
now on par with those trying to sell us solution manuals or herbal
remedies.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Is print thread safe?

[Chris Angelico]

On Tue, Aug 12, 2014 at 2:31 PM, Cameron Simpson  wrote:
> I would expect file.write to be fast enough that the lock would usually be
> free.

Until the day when it becomes really REALLY slow, because your
program's piped into 'less' and the user's paging through it. But even
apart from that, writing to stdout can take a notable amount of time.
Expecting the lock to usually be free will depend on the nature of the
program - how much of it is spent in silent computation and how much
in production of output. If that ratio is sufficiently skewed, then
sure, the lock'll usually be free.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Captcha identify

[Denis McMahon]

On Tue, 12 Aug 2014 00:04:33 -0700, Wesley wrote:

> These days I got a small task to identify Captcha characters.

Several of us code websites. Some of our websites may even use captcha. 
We use captcha to stop software posting spam to websites.

What makes you think we have any inclination to help you defeat captchas?

-- 
Denis McMahon, denismfmcma...@gmail.com
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: how to write file into my android phone?

[Grant Edwards]

On 2014-08-11, Grant Edwards  wrote:

> What MTP allows is parallel access between the MTP protocol server in
> the phone and the rest of the phone (OS and apps).  In order to mount
> a partition as a USB mass storage device, it has to first be unmounted
> by the Android system.  That's OK for an SD card that doesn't contain
> anything important required by the Android system, but it means that
> you can't mount the "system" partition without shutting down the phone
> completely.

Another important reason for not presenting the phone's built-in flash
as a USB mass storage device is that it requires you to use a
brain-dead broken-as-designed Microsoft filesystem left over from the
days of 360K floppy disk drives -- otherwise Windows won't know what
to do with it.

It's _possible_ to build an Android phone tht runs an DOS file system
(and IIRC people have done it) but it introduces all sorts of
security, reliability, and performance problems.

-- 
Grant Edwards   grant.b.edwardsYow! I wonder if I ought
  at   to tell them about my
  gmail.comPREVIOUS LIFE as a COMPLETE
   STRANGER?
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Linux distros w/o Python in "base" installation

[Grant Edwards]

On 2014-08-12, Chris “Kwpolska” Warrick  wrote:
> On Mon, Aug 11, 2014 at 8:53 PM, Grant Edwards  
> wrote:
>> I just installed Arch Linux for the first time, and was surprosed to
>> find that Python isn't installed as part of a "base" system.  It's
>> also not included in the 'base-devel' package group.  It's trivial to
>> install, but I'd still pretty surprised it's not there by default.  I
>> guess I've spent too much time with Gentoo, Debian, and RedHat
>> derivitives which require Python be installed.
>>
>> I've probably used at least a dozen Linux distros over the years, and
>> this is the first time I've noticed that Python wasn't installed by
>> default.
>
> Arch has a different idea of “base system”.  The base group contains
> the most crucial packages needed to run an Arch Linux system, and that
> is all.

That's true of Gentoo as well, but it includes Python.

> And you do not need Python to do so.

That's the part that surprised me.  It's been a _long_ time since I've
run a Linux distro that didn't require Python as a crucial system
package.

-- 
Grant Edwards   grant.b.edwardsYow! Those people look
  at   exactly like Donnie and
  gmail.comMarie Osmond!!
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Captcha identify

[Wesley]

Here is captcha link:
https://drive.google.com/folderview?id=0B33_p7UnVqoyd09mT3V0aWFxRmc&usp=sharing

在 2014年8月12日星期二UTC+8下午8时59分11秒，Dennis Lee Bieber写道：
> On Tue, 12 Aug 2014 00:04:33 -0700 (PDT), Wesley 
> 
> declaimed the following:
> 
> 
> 
> >Hi guys,
> 
> >  These days I got a small task to identify Captcha characters.
> 
> 
> 
>   In other words, you have a task to make a robot that can break the
> 
> procedures put in place to prevent robots from posting to web sites...
> 
> 
> 
>   The whole purpose of the CAPTCHA scheme is that computer AI systems
> 
> aren't advanced enough to process them, whereas a human mind can almost do
> 
> it in the subconscious.
> 
> 
> 
> 
> 
> (as for picture attachments? comp.lang.python is a text newsgroup --
> 
> binaries aren't wanted in it; c.l.p is also gatewayed to a mailing list;
> 
> and that mailing list is gatewayed to gmane.comp.python.general where it is
> 
> made available as a news group (after spam filtering). Google Groups links
> 
> to c.l.p (and many of us wish it didn't)
> 
> -- 
> 
>   Wulfraed Dennis Lee Bieber AF6VN
> 
> wlfr...@ix.netcom.comHTTP://wlfraed.home.netcom.com/

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Captcha identify

[Chris Angelico]

On Wed, Aug 13, 2014 at 12:57 AM, Wesley  wrote:
> Here is captcha link:
> https://drive.google.com/folderview?id=0B33_p7UnVqoyd09mT3V0aWFxRmc&usp=sharing

You seem to have misunderstood how grossly offensive your request is.
I am now the third person to do you the courtesy of a response, but
there are lots more who are simply deleting your post and moving on,
or possibly marking you as a spammer, because that's about the only
reason for wanting a program to solve CAPTCHAs.

Let me spell it out for you: NO WE WILL NOT do this for you. And if
you do it yourself, we will not be happy. Just don't.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python in financial services

[Johann Hibschman]

Rustom Mody  writes:

> Ive been asked to formulate a python course for financial services
> folk.
>
> If I actually knew about the subject, I'd have fatter pockets!
> Anyway heres some thoughts. What I am missing out?

Good luck!  It's a pretty broad field, so everyone probably has
different needs.

> - Libraries -- Decimal?

I've never seen decimal used, even though it makes sense for
accounting-style finance.  I've mostly been looking at forecasts,
trading, and risk, where floats are fine.  So maybe mention that it
exists, so people know where to look if they need it, but don't stress
it.

> - scripts -- philosophy and infrastructure eg argparse, os.path

Basic argparse is very handy, but, again, I wouldn't spend too much time
on it.

> - Pandas
> - Numpy Scipy (which? how much?)

For me, pandas is huge, numpy is a nice fundamental substrate, while
only bits and pieces of scipy are used (mostly optimization).
statsmodels may also be worth a mention, as the answer to "how do I do a
regression".

> - ipython + matplotlib + ??

Ipython notebook + matplotlib is great.  At least show that it exists.
pandas plots may be enough, though.

> - Database interfacing

Definitely mention.

> - Excel interfacing (couple of libraries.. which?)

Meh, maybe.  At least give a strategy.  It always seems like a fool's
errand, though: I end up just dumping data to CSV and using that.

> - C(C++?) interfacing paradigms -- ranging from ctypes, cython to
>   classic lo-level

Probably not, but it depends on the audience.  The overview, like
"ctypes will link to C-like libraries, cython lets you write python-like
code that runs fast, and there's SWIG and Boost.Python if you want to
write your own modules" is about all you need.

Hope that helps,
Johann
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Captcha identify

[Denis McMahon]

On Wed, 13 Aug 2014 01:06:47 +1000, Chris Angelico wrote:

> Let me spell it out for you: NO WE WILL NOT do this for you. And if you
> do it yourself, we will not be happy. Just don't.

Chris, I suspect he's a codemonkey in a chinese or similar asian spamhaus. 
We should probably be thankful that he's so dumb.

-- 
Denis McMahon, denismfmcma...@gmail.com
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Linux distros w/o Python in "base" installation

[Ian Kelly]

On Tue, Aug 12, 2014 at 8:12 AM, Grant Edwards  wrote:
> On 2014-08-12, Chris “Kwpolska” Warrick  wrote:
>> On Mon, Aug 11, 2014 at 8:53 PM, Grant Edwards  
>> wrote:
>>> I just installed Arch Linux for the first time, and was surprosed to
>>> find that Python isn't installed as part of a "base" system.  It's
>>> also not included in the 'base-devel' package group.  It's trivial to
>>> install, but I'd still pretty surprised it's not there by default.  I
>>> guess I've spent too much time with Gentoo, Debian, and RedHat
>>> derivitives which require Python be installed.
>>>
>>> I've probably used at least a dozen Linux distros over the years, and
>>> this is the first time I've noticed that Python wasn't installed by
>>> default.
>>
>> Arch has a different idea of “base system”.  The base group contains
>> the most crucial packages needed to run an Arch Linux system, and that
>> is all.
>
> That's true of Gentoo as well, but it includes Python.

Because Portage is written in Python. Otherwise Gentoo would probably
omit it from the base system as well.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Captcha identify

[Chris Angelico]

On Wed, Aug 13, 2014 at 1:38 AM, Denis McMahon  wrote:
> On Wed, 13 Aug 2014 01:06:47 +1000, Chris Angelico wrote:
>
>> Let me spell it out for you: NO WE WILL NOT do this for you. And if you
>> do it yourself, we will not be happy. Just don't.
>
> Chris, I suspect he's a codemonkey in a chinese or similar asian spamhaus.
> We should probably be thankful that he's so dumb.

Yeah, that is the most likely scenario. If I'm courteous, I'll assume
that he has no idea what the rest of the world thinks of this kind of
thing.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python in financial services

[Denis McMahon]

On Tue, 12 Aug 2014 00:33:11 -0700, Rustom Mody wrote:

> Ive been asked to formulate a python course for financial services folk.

I wouldn't worry too much about c or c++ interfacing paradigms.

-- 
Denis McMahon, denismfmcma...@gmail.com
-- 
https://mail.python.org/mailman/listinfo/python-list

Using asyncio workers in a `concurrent.futures` interface

[cool-RR]

Hello everybody! I have a question. 

I have a Django app running on Heroku. I need to run about 100 worker threads 
there to do uploads/downloads simultaneously. A Heroku Dyno has only 512MB of 
memory, so I'm reluctant to run 100 worker threads. (I've had Dynos crash from 
lack of memory when using 6 threads before.) 

I heard that the asyncio module is mature and ready for usage, and I was happy 
because I kept hearing about it in the last year, and I saw Guido's lecture 
about it. If I understand correctly it would let me run multiple uploads and 
downloads efficiently in one thread, which would conserve more resources than 
using threads. (Please correct me if I'm wrong.) 

Now, I am a little clueless about the whole way it's built, using coroutines 
and tricky usage of `yield from`. I figured that since this is eventually a 
library for concurrency, i.e. doing many tasks at the same time, there will be 
an API in a style of "Here are 100 tasks for you to do concurrently, let me 
know when they're done."

I looked at the asyncio documentation page and saw that it does mention futures 
and executors, which is my favorite interface for doing concurrency. I was 
happy and I skimmed the docs. But, I couldn't find a simple way to use these. I 
don't want to learn how to define coroutines and use `yield from` to switch 
between them. (I use `yield from` regularly and fully understand how it works, 
I just don't write my programs that way.)

What I'm expecting is something like this: 

download_file = lambda url: requests.get(url).content
urls = ['http://google.com/file1.jpg', 'http://google.com/file2.jpg', 
'http://google.com/file3.jpg'] # etc.

with AsyncIOExecutor() as asyncio_executor:
files = asyncio_executor.map(download_file, urls)

And that's it, no coroutines, no `yield from`. Since, if I understand 
correctly, asyncio requires a mainloop, it would make sense for the 
AsyncIOExecutor to have a thread of its own in which it could run its mainloop.

Is this possible? Did someone implement this? 


Thanks for your help,
Ram.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: how to get the ordinal number in list

[Neil D. Cerutti]

On 8/10/2014 2:14 PM, Roy Smith wrote:

In article <154cc342-7f85-4d16-b636-a1a953913...@googlegroups.com>,
  Rustom Mody  wrote:


l= [6,2,9,12,1,4]
sorted(l,reverse=True)[:5]

[12, 9, 6, 4, 2]

No need to know how sorted works nor [:5]

Now you (or Steven) can call it abstract.

And yet its
1. Actual running code in the interpreter
2. Its as close as one can get to a literal translation of your
"Find the 5 largest numbers in a list"
[...]
All the above are clearer than loops+assignments and can be
taught before them


I disagree.  For a beginner, you want to be able to break things down
into individual steps and examine the result at each point.  If you do:


l= [6,2,9,12,1,4]
l2 = sorted(l,reverse=True)


you have the advantage that you can stop after creating l2 and print it
out.  The student can see that it has indeed been sorted.  With the
chained operations, you have to build a mental image of an anonymous,
temporary list, and then perform the slicing operation on that.  Sure,
it's the way you or I would write it in production code, but for a
beginner, breaking it down into smaller pieces makes it easier to
understand.


l2[:5]


Yes, and that teaching technique is supported by research.

Beginners are particularly poor, in relation to experts, at noticing the 
applicability of idea, and at combining ideas together. Breaking things 
into component parts has multiple benefits:


1. The applicability of individual ideas becomes obvious. It's one thing 
to know about [].sort, and another thing to know when it's appropriate 
to sort something.


2. The expert specifically shows how and why the ideas are combined. 
This helps build the connection for the beginner, whose knowledge is not 
stored as an expert stores it; i.e, in broad categories with multiple 
connections; but as disorganized data with very few connections.


http://www.amazon.com/How-Learning-Works-Research-Based-Principles/dp/0470484101

I bought the book based on a recommendation from SciPy talk, and it's 
really great. As an autodidact, it'll help me teach *myself* better, too.


--
Neil Cerutti

--
https://mail.python.org/mailman/listinfo/python-list

Re: Python in financial services

[Rustom Mody]

On Tuesday, August 12, 2014 9:05:44 PM UTC+5:30, Johann Hibschman wrote:
> Rustom Mody writes:

> > - Pandas
> > - Numpy Scipy (which? how much?)

> For me, pandas is huge, numpy is a nice fundamental substrate, while
> only bits and pieces of scipy are used (mostly optimization).
> statsmodels may also be worth a mention, as the answer to "how do I do a
> regression".

statsmodels seems like something useful -- thanks for the pointer
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python in financial services

[Rustom Mody]

On Tuesday, August 12, 2014 9:20:16 PM UTC+5:30, Denis McMahon wrote:
> On Tue, 12 Aug 2014 00:33:11 -0700, Rustom Mody wrote:

> > Ive been asked to formulate a python course for financial services folk.

> I wouldn't worry too much about c or c++ interfacing paradigms.

And I dont like teaching that stuff either :-) [especially on windows!]

However those folks have thousands of lines of C/C++ which they are porting to 
python.

So...

Thanks anyway
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: how to get the ordinal number in list

[Rustom Mody]

On Tuesday, August 12, 2014 11:10:48 PM UTC+5:30, Neil D. Cerutti wrote:
> On 8/10/2014 2:14 PM, Roy Smith wrote:
> >   Rustom Mody  wrote:
> > l= [6,2,9,12,1,4]
> > sorted(l,reverse=True)[:5]
> >> [12, 9, 6, 4, 2]
> >> No need to know how sorted works nor [:5]
> >> Now you (or Steven) can call it abstract.
> >> And yet its
> >> 1. Actual running code in the interpreter
> >> 2. Its as close as one can get to a literal translation of your
> >> "Find the 5 largest numbers in a list"
> >> [...]
> >> All the above are clearer than loops+assignments and can be
> >> taught before them
> > I disagree.  For a beginner, you want to be able to break things down
> > into individual steps and examine the result at each point.  If you do:
> > l= [6,2,9,12,1,4]
> > l2 = sorted(l,reverse=True)
> > you have the advantage that you can stop after creating l2 and print it
> > out.  The student can see that it has indeed been sorted.  With the
> > chained operations, you have to build a mental image of an anonymous,
> > temporary list, and then perform the slicing operation on that.  Sure,
> > it's the way you or I would write it in production code, but for a
> > beginner, breaking it down into smaller pieces makes it easier to
> > understand.
> > l2[:5]

> Yes, and that teaching technique is supported by research.


Yes and I as said to Roy I dont see any disagreement at least on this account.



> Beginners are particularly poor, in relation to experts, at noticing the 
> applicability of idea, and at combining ideas together. Breaking things 
> into component parts has multiple benefits:

> 1. The applicability of individual ideas becomes obvious. It's one thing 
> to know about [].sort, and another thing to know when it's appropriate 
> to sort something.

> 2. The expert specifically shows how and why the ideas are combined. 
> This helps build the connection for the beginner, whose knowledge is not 
> stored as an expert stores it; i.e, in broad categories with multiple 
> connections; but as disorganized data with very few connections.

Nice!

And how do we lead the beginners towards expertise?
In a way functional programming is to programming creativity 
what lego is to children's spatial creativity.

Specifically there are a bunch of pieces that need to fit:

1. Functional Programming: Nothing more than composing functions
   [Maybe a bit simplistic but not unrealistic a defn]
2. Trying this out at the interpreter
3. Introspectable objects

Some things follow from this:

For the lego-game of playing with functions at the REPL to work and be
pleasant and rewarding:

1. functions should be non side-effecting; else same trials giving different
answers adds more confusion than understanding
2. They should be non-printing else:

def foo(x): return x+1
def bar(x): print x+1

look similar when trivially tried but compositionally are utterly different

In effect a printing function breaks the lego bricks

[The P in the REPL is DRY enough that it does not usually need to be
repeated all over]

3. Abstractions (class instances) should be avoided in favor of
concrete data (lists, dicts, scalars) because they add undue mess at little
comprehension advantage. eg take the example of a regex match. It
returns some object and then we have to scratch our heads wondering
whats in the magic box. If instead of match, we use findall, the data
is manifest and obvious.


> http://www.amazon.com/How-Learning-Works-Research-Based-Principles/dp/0470484101

> I bought the book based on a recommendation from SciPy talk, and it's 
> really great. As an autodidact, it'll help me teach *myself* better, too.

Looks interesting. Will take a look.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Using asyncio workers in a `concurrent.futures` interface

[Marko Rauhamaa]

cool-RR :

> If I understand correctly [asyncio] would let me run multiple uploads
> and downloads efficiently in one thread, which would conserve more
> resources than using threads.

Asyncio does make it convenient to multiplex event on one or more
threads. Threads have their uses (exploiting multiple CPUs), only you
shouldn't associate threads with every state machine, IMO. Asyncio
allows you to separate your state machines from your threads. For
example, you might have 1,000 state machines (for 1,000 connections) but
only 8 threads for 4 CPUs.

> Now, I am a little clueless about the whole way it's built, using
> coroutines and tricky usage of `yield from`.

Asyncio actively supports (at least) *two* multiplexing models:
callbacks (aka listeners or handlers) and coroutines. Programming with
callbacks involves storing the state explicitly in a state machine
object. The callbacks are supposed to never block but return
immediately. That model is a long-time favorite of many, including me.

The coroutine model is highly analogous with the multithreading model in
that you store the state implicitly in the code. Multithreading marks
the state with blocking function calls. Coroutines mark the state with
"yield from" statements. Otherwise, a multithreading implementation will
have very much the shape of a coroutine implementation.

The funky aspect of the coroutines is the way they "abuse" the "yield
from" statement, whose original purpose is to pass a series of results
from a generator to the caller by chaining lower-level generators.
Coroutines employ a "trick:" the "yield from" statement does not pass
any meaningful results. Instead, the statement is used to make the
generator/coroutine pseudoblock and switch context within the same
thread.

There's one crucial advantage coroutines have over threads: you can
multiplex events. If your thread is blocked on, say, reading a byte, you
can't tell it to stop waiting and do something else instead. Coroutines
can be made to wait on alternative stimuli.

> I looked at the asyncio documentation page and saw that it does
> mention futures and executors, which is my favorite interface for
> doing concurrency.

They are there.

My favorite model is the Actor Model, where objects communicate with
each other and the outside world through asynchornous stimuli of sorts.
The thinking goes: something happened, so how do I react to it. The
actor model just needs a class with the member "self.state", which
contains the name of the internal state of the object. Then the objects
callback methods receive the inputs send out messages and adjust the
state.

Both the actor model and the coroutines produce somewhat messy code that
somewhat hard to get right. However, that's a reflection of how messy
the reality is. Any attempts to pave it over will ultimately result in
more trouble.

> download_file = lambda url: requests.get(url).content
> urls = ['http://google.com/file1.jpg',
> 'http://google.com/file2.jpg', 'http://google.com/file3.jpg'] #
> etc.
> 
> with AsyncIOExecutor() as asyncio_executor:
> files = asyncio_executor.map(download_file, urls)
>
> And that's it, no coroutines, no `yield from`.

I'm not quite following you. However, one feature of the coroutines is
that you must use "yield from" everywhere. You can't delegate it to a
subroutine and forget about it.

That's my main problem with coroutines. The simple function call syntax:

y = f(x)

is replaced with the weird:

y = yield from f(x)

> Since, if I understand correctly, asyncio requires a mainloop, it
> would make sense for the AsyncIOExecutor to have a thread of its own
> in which it could run its mainloop.
>
> Is this possible? Did someone implement this? 

Unfortunately, I have not. I have just implemented a toy example to
satisfy my curiosity. All of my code uses callbacks and select.epoll()
-- to great success.

Here's my exploration in the classic dining philosophers problem:

   http://pacujo.net/marko/philosophers.py>

I have "fixed" the faulty protocol by having an assistant break the
deadlock by occasionally nagging the philosophers to drop everything and
get back to thinking.

The main accomplishment of the exercise was that I convinced myself the
coroutines can be used for serious things as coroutines can multiplex
stimuli with asyncio.wait(..., return_when=asyncio.FIRST_COMPLETED).


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Begginer in python trying to load a .dll

[c1223]

Hi, 
Im working in the development of a program based in python that allow us to 
contrl a spectometer. The spectometer has an .dll file. The idea is to work 
through this dll and operate the spectometer. 
The name of the .dll is AS5216.dll. I've trying with ctype, but it doesn't 
work. 
I'm begginer in python. 
All the comments are useful to me. 
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Using asyncio workers in a `concurrent.futures` interface

[MRAB]

On 2014-08-12 18:02, cool-RR wrote:

Hello everybody! I have a question.

I have a Django app running on Heroku. I need to run about 100 worker
threads there to do uploads/downloads simultaneously. A Heroku Dyno
has only 512MB of memory, so I'm reluctant to run 100 worker threads.
(I've had Dynos crash from lack of memory when using 6 threads
before.)

I heard that the asyncio module is mature and ready for usage, and I
was happy because I kept hearing about it in the last year, and I saw
Guido's lecture about it. If I understand correctly it would let me
run multiple uploads and downloads efficiently in one thread, which
would conserve more resources than using threads. (Please correct me
if I'm wrong.)

Now, I am a little clueless about the whole way it's built, using
coroutines and tricky usage of `yield from`. I figured that since
this is eventually a library for concurrency, i.e. doing many tasks
at the same time, there will be an API in a style of "Here are 100
tasks for you to do concurrently, let me know when they're done."

I looked at the asyncio documentation page and saw that it does
mention futures and executors, which is my favorite interface for
doing concurrency. I was happy and I skimmed the docs. But, I
couldn't find a simple way to use these. I don't want to learn how to
define coroutines and use `yield from` to switch between them. (I use
`yield from` regularly and fully understand how it works, I just
don't write my programs that way.)

What I'm expecting is something like this:

 download_file = lambda url: requests.get(url).content
 urls = ['http://google.com/file1.jpg', 'http://google.com/file2.jpg', 
'http://google.com/file3.jpg'] # etc.

 with AsyncIOExecutor() as asyncio_executor:
 files = asyncio_executor.map(download_file, urls)

And that's it, no coroutines, no `yield from`. Since, if I understand
correctly, asyncio requires a mainloop, it would make sense for the
AsyncIOExecutor to have a thread of its own in which it could run its
mainloop.

Is this possible? Did someone implement this?


Do you really need to upload/download that many at the same time? I'd
put them into a queue and have only a few of uploading/downloading at
any time.
--
https://mail.python.org/mailman/listinfo/python-list

Re: Begginer in python trying to load a .dll

[Chris Angelico]

On Wed, Aug 13, 2014 at 4:36 AM, c1223  wrote:
> Im working in the development of a program based in python that allow us to 
> contrl a spectometer. The spectometer has an .dll file. The idea is to work 
> through this dll and operate the spectometer.
> The name of the .dll is AS5216.dll. I've trying with ctype, but it doesn't 
> work.
> I'm begginer in python.
> All the comments are useful to me.

What have you tried, exactly? What does "it doesn't work" mean?

Interfacing with a DLL is a potentially very hard and complex job
(although it might turn out to be really easy - depends what the DLL's
doing). We can't simply give you an answer right here, without knowing
heaps about the DLL you're working with - where it came from, whether
this is homework or a real-world problem, etc, etc, etc.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Begginer in python trying to load a .dll

[Skip Montanaro]

On Tue, Aug 12, 2014 at 1:36 PM, c1223  wrote:
> I've trying with ctype, but it doesn't work.

Can you post a small example of how you're trying to use cypes? I
don't think a full example would be necessary, but knowing the API of
one or two functions and how you're trying to call them from Python
would be useful.

Just as the off-chance your DLL was written in C++... I don't think
you can interface with ctypes directly. I think you would have to
write a little shim DLL which exposes a C-compatible API.

Skip
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: how to get the ordinal number in list

[alister]

On Tue, 12 Aug 2014 23:39:42 +1000, Chris Angelico wrote:

> On Tue, Aug 12, 2014 at 8:40 PM, alister
>  wrote:
>> On Tue, 12 Aug 2014 00:21:28 -0700, wxjmfauth wrote:
>> [ chomp ]
>>
>> Mathematicians work with numbers (Algebra is a abstraction of numerical
>> concepts) strings are concerned with characters (Arabic numerals are
>> just characters commonly used to represent numbers )
>> therefore even trying to rationalise any string representation with
>> mathematics is a logical absurdity in itself.
> 
> Don't bother responding to jmf; he doesn't listen. Also, his posts don't
> survive the jump to python-list, so we see them only when someone quotes
> him like that... as far as I'm concerned, his posts are now on par with
> those trying to sell us solution manuals or herbal remedies.
> 
> ChrisA

As you don't see them unless someone replies I will make sure I don't 
reply to any more of his posts.

I would not want to inflict this nonsense on someone who would otherwise 
be spared.


-- 
You will lose an important tape file.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Begginer in python trying to load a .dll

[Mark Lawrence]

On 12/08/2014 19:36, c1223 wrote:

Hi,
Im working in the development of a program based in python that allow us to 
contrl a spectometer. The spectometer has an .dll file. The idea is to work 
through this dll and operate the spectometer.
The name of the .dll is AS5216.dll. I've trying with ctype, but it doesn't work.
I'm begginer in python.
All the comments are useful to me.



Please help us to help you.  Stating "it doesn't work" is of no use. 
Show us your code, what you expected to happen and what actually 
happened.  If you get a traceback cut and paste all of it into your 
message.  State what version of Python you have and your OS.  Sure 
seeing dll we can surmise Windows but which one?


For further advice on how to put your question(s) together see "The 
SSCCE - Short, Self Contained, Correct (Compilable), Example" at 
http://sscce.org/


--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

--
https://mail.python.org/mailman/listinfo/python-list

Re: Python in financial services

[Denis McMahon]

On Tue, 12 Aug 2014 10:48:14 -0700, Rustom Mody wrote:

> However those folks have thousands of lines of C/C++ which they are
> porting to python.

That begs the question: Why?

Seriously, I'd like to know what benefits they expect to achieve by doing 
so.

-- 
Denis McMahon, denismfmcma...@gmail.com
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Begginer in python trying to load a .dll

[Chris Kaynor]

On Tue, Aug 12, 2014 at 11:48 AM, Skip Montanaro  wrote:

> Just as the off-chance your DLL was written in C++... I don't think
> you can interface with ctypes directly. I think you would have to
> write a little shim DLL which exposes a C-compatible API.
>

You should be able to use ctypes with a C++ library, you'd just need to
know how your C++ compiler handles name mangling. Typically, changes are
made to accommodate classes, namespaces, and function overloads. Of course,
if you were to change to a different compiler (possibly including a
different version) or made minor tweaks to the functions, you'd have to
update all the Python code to use the new name mangling scheme.

Hmm...On second thought, maybe just only using functions exports with
"extern C" would be easier - or just using a C compiler.

Chris
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Begginer in python trying to load a .dll

[c1234 py]

On Tuesday, August 12, 2014 3:36:17 PM UTC-3, c1234 py wrote:
> Hi, 
> 
> Im working in the development of a program based in python that allow us to 
> contrl a spectometer. The spectometer has an .dll file. The idea is to work 
> through this dll and operate the spectometer. 
> 
> The name of the .dll is AS5216.dll. I've trying with ctype, but it doesn't 
> work. 
> 
> I'm begginer in python. 
> 
> All the comments are useful to me.


I'm using this


>>import ctypes
hllDll=ctypes.WinDLL("C:\\AS5216.dll")


hllApiProto = ctypes.WINFUNCTYPE (ctypes.c_int,ctypes.c_void_p,
ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p)
hllApiParams = (1, "p1", 0), (1, "p2", 0), (1, "p3",0), (1, "p4",0),


hllApi = hllApiProto (("HLLAPI", hllDll), hllApiParams)

p1 = ctypes.c_int (1)
p2 = ctypes.c_char_p (sessionVar)
p3 = ctypes.c_int (1)
p4 = ctypes.c_int (0)
hllApi (ctypes.byref (p1), p2, ctypes.byref (p3), ctypes.byref (p4))



Thanks



-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Begginer in python trying to load a .dll

[Rob Gaddi]

On Tue, 12 Aug 2014 11:56:37 -0700 (PDT)
c1234 py  wrote:

> On Tuesday, August 12, 2014 3:36:17 PM UTC-3, c1234 py wrote:
> > Hi, 
> > 
> > Im working in the development of a program based in python that allow us to 
> > contrl a spectometer. The spectometer has an .dll file. The idea is to work 
> > through this dll and operate the spectometer. 
> > 
> > The name of the .dll is AS5216.dll. I've trying with ctype, but it doesn't 
> > work. 
> > 
> > I'm begginer in python. 
> > 
> > All the comments are useful to me.
> 
> 
> I'm using this
> 
> 
> >>import ctypes
> hllDll=ctypes.WinDLL("C:\\AS5216.dll")
> 
> 
> hllApiProto = ctypes.WINFUNCTYPE (ctypes.c_int,ctypes.c_void_p,
> ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p)
> hllApiParams = (1, "p1", 0), (1, "p2", 0), (1, "p3",0), (1, "p4",0),
> 
> 
> hllApi = hllApiProto (("HLLAPI", hllDll), hllApiParams)
> 
> p1 = ctypes.c_int (1)
> p2 = ctypes.c_char_p (sessionVar)
> p3 = ctypes.c_int (1)
> p4 = ctypes.c_int (0)
> hllApi (ctypes.byref (p1), p2, ctypes.byref (p3), ctypes.byref (p4))
> 
> 

Great.  And that fails in what way, on which line, with what error message?


-- 
Rob Gaddi, Highland Technology -- www.highlandtechnology.com
Email address domain is currently out of order.  See above to fix.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Begginer in python trying to load a .dll

[Chris Angelico]

On Wed, Aug 13, 2014 at 4:58 AM, Rob Gaddi
 wrote:
> Great.  And that fails in what way, on which line, with what error message?

And, is that the entire program? Because if it is, then I can see at
least one problem, a NameError.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python in financial services

[Rustom Mody]

On Wednesday, August 13, 2014 12:24:12 AM UTC+5:30, Denis McMahon wrote:
> On Tue, 12 Aug 2014 10:48:14 -0700, Rustom Mody wrote:

> > However those folks have thousands of lines of C/C++ which they are
> > porting to python.

> That begs the question: Why?

> Seriously, I'd like to know what benefits they expect to achieve by doing 
> so.

Heh!
I asked more or less that.
Was told: "We want it!"

I gather its to do with "agility" -- faster time to incorporate and
integrate new logic into code etc.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Begginer in python trying to load a .dll

[c1234 py]

On Tuesday, August 12, 2014 4:09:13 PM UTC-3, Chris Angelico wrote:
> On Wed, Aug 13, 2014 at 4:58 AM, Rob Gaddi
> 
>  wrote:
> 
> > Great.  And that fails in what way, on which line, with what error message?
> 
> 
> 
> And, is that the entire program? Because if it is, then I can see at
> 
> least one problem, a NameError.
> 
> 
> 
> ChrisA

This appear in the terminal:


>>> runfile('C://Python Scripts')
  File "C:\\sitecustomize.py", line 585, in runfile
execfile(filename, namespace)
  File "C://Sin título 38.py", line 19, in 
hllApi = hllApiProto (("HLLAPI", hllDll), hllApiParams)
AttributeError: function 'HLLAPI' not found
>>> Traceback (most recent call last):
  File "", line 1, in 
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: how to get the ordinal number in list

[Mark Lawrence]

On 12/08/2014 19:45, alister wrote:

On Tue, 12 Aug 2014 23:39:42 +1000, Chris Angelico wrote:


On Tue, Aug 12, 2014 at 8:40 PM, alister
 wrote:

On Tue, 12 Aug 2014 00:21:28 -0700, wxjmfauth wrote:
[ chomp ]

Mathematicians work with numbers (Algebra is a abstraction of numerical
concepts) strings are concerned with characters (Arabic numerals are
just characters commonly used to represent numbers )
therefore even trying to rationalise any string representation with
mathematics is a logical absurdity in itself.


Don't bother responding to jmf; he doesn't listen. Also, his posts don't
survive the jump to python-list, so we see them only when someone quotes
him like that... as far as I'm concerned, his posts are now on par with
those trying to sell us solution manuals or herbal remedies.

ChrisA


As you don't see them unless someone replies I will make sure I don't
reply to any more of his posts.

I would not want to inflict this nonsense on someone who would otherwise
be spared.



Thank you, as it saves me from once again losing it in public when his 
latest incarnation of complete dross appears.  Two years he's been at 
it, I wonder how he manages to avoid "Sent to jail. Do not pass Go. Do 
not collect £200" or where he gets his supply of "Get out of jail free" 
cards from?


--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

--
https://mail.python.org/mailman/listinfo/python-list

Re: Begginer in python trying to load a .dll

[Christian Gollwitzer]

Am 12.08.14 20:36, schrieb c1223:

Hi, Im working in the development of a program based in python that
allow us to contrl a spectometer. The spectometer has an .dll file.
The idea is to work through this dll and operate the spectometer. The
name of the .dll is AS5216.dll. I've trying with ctype, but it
doesn't work. I'm begginer in python. All the comments are useful to
me.


For sure this DLL will come with a header file for C. You could pass it 
through SWIG and see if it generates a useful Python module. You need a 
C compiler for that route, though.


Christian


--
https://mail.python.org/mailman/listinfo/python-list

Re: how to get the ordinal number in list

[Neil D. Cerutti]

On 8/12/2014 2:20 PM, Rustom Mody wrote:

On Tuesday, August 12, 2014 11:10:48 PM UTC+5:30, Neil D. Cerutti wrote:

Beginners are particularly poor, in relation to experts, at noticing the
applicability of idea, and at combining ideas together. Breaking things
into component parts has multiple benefits:



1. The applicability of individual ideas becomes obvious. It's one thing
to know about [].sort, and another thing to know when it's appropriate
to sort something.



2. The expert specifically shows how and why the ideas are combined.
This helps build the connection for the beginner, whose knowledge is not
stored as an expert stores it; i.e, in broad categories with multiple
connections; but as disorganized data with very few connections.


Nice!

And how do we lead the beginners towards expertise?
In a way functional programming is to programming creativity
what lego is to children's spatial creativity.

Specifically there are a bunch of pieces that need to fit:

1. Functional Programming: Nothing more than composing functions
[Maybe a bit simplistic but not unrealistic a defn]
2. Trying this out at the interpreter
3. Introspectable objects


Functional programming could be particularly hard to teach since it is 
generally made up of numerous small units of work combined in a complex 
way. This is precisely the formula for something that beginners will 
find extremely challenging.


When functional programming is dumbed down enough for a beginner to be 
able to grok it, the programming problems start to look really lame. It 
needn't be that way, of course, but it takes a good deal of creativity 
on the part of the instructor. If Factorial doesn't turn you on, you 
might be screwed. ;)



Some things follow from this:

For the lego-game of playing with functions at the REPL to work and be
pleasant and rewarding:

1. functions should be non side-effecting; else same trials giving different
answers adds more confusion than understanding
2. They should be non-printing else:

def foo(x): return x+1
def bar(x): print x+1

look similar when trivially tried but compositionally are utterly different

In effect a printing function breaks the lego bricks


That may be so, but printing stuff to the screen is very natural to 
people. I've downloaded Haskell a few times, but the knowledge that 
getting input and writing output requires something mysterious called 
gonads just frightens me.



[The P in the REPL is DRY enough that it does not usually need to be
repeated all over]


A good REPL does help a lot, though.


3. Abstractions (class instances) should be avoided in favor of
concrete data (lists, dicts, scalars) because they add undue mess at little
comprehension advantage. eg take the example of a regex match. It
returns some object and then we have to scratch our heads wondering
whats in the magic box. If instead of match, we use findall, the data
is manifest and obvious.


I'm with you on regex: match objects suck. That and escaping.

--
Neil Cerutti

--
https://mail.python.org/mailman/listinfo/python-list

Re: Begginer in python trying to load a .dll

[c1234 py]

El martes, 12 de agosto de 2014 16:16:21 UTC-3, Christian Gollwitzer  escribió:
> Am 12.08.14 20:36, schrieb c1223:
> 
> > Hi, Im working in the development of a program based in python that
> 
> > allow us to contrl a spectometer. The spectometer has an .dll file.
> 
> > The idea is to work through this dll and operate the spectometer. The
> 
> > name of the .dll is AS5216.dll. I've trying with ctype, but it
> 
> > doesn't work. I'm begginer in python. All the comments are useful to
> 
> > me.
> 
> 
> 
> For sure this DLL will come with a header file for C. You could pass it 
> 
> through SWIG and see if it generates a useful Python module. You need a 
> 
> C compiler for that route, though.
> 
> 
> 
>   Christian


How can i pass it through a SWIG?, i don't know that and what it mean?
-- 
https://mail.python.org/mailman/listinfo/python-list

Suitable Python code to scrape specific details from web pages.

[Simon Evans]

Dear Programmers,
I have been looking at the You tube 'Web Scraping Tutorials' of Chris Reeves. I 
have tried a few of his python programs in the Python27 command prompt, but 
altered them from accessing data using links say from the Dow Jones index, to 
accessing the details I would be interested in accessing from the 'Racing Post' 
on a daily basis. Anyhow, the code it returns is not in the example I am going 
to give, is not the information I am seeking, instead of returning the given 
odds on a horse, it only returns a [], which isn't much use. 
I would be glad if you could tell me where I am going wrong. 
Yours faithfully
Simon Evans.

>>>import urllib
>>>import re
>>>htmlfile = urllib.urlopen("http://www.racingpost.com/horses2/cards/card.sd?

race_id=600048r_date=2014-05-08#raceTabs=sc_")
htmltext = htmlfile.read()
regex = '1http://www.racingpost.com/horses/horse_home.sd?

horse_id=758752"onclick="scorecards.send("horse_name":):return 
Html.popup(this,

{width:695,height:800})"title="Full details about this HORSE">Lively 

Baron9/4F'
>>>pattern = re.compile(regex)
>>>odds=re.findall(pattern,htmltext)
>>>print odds
[]
>>>

>>>import urllib
>>>import re
>>>htmlfile = urllib.urlopen("http://www.racingpost.com/horses2/cards/card.sd?

>>>race_id=600048r_date=2014-05-08#raceTabs=sc_")
>>>htmltext = htmlfile.read()
>>>regex = ''
>>>pattern = re.compile(regex)
>>>odds=re.findall(pattern,htmltext)
>>>print odds
[]
>>>
---
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Suitable Python code to scrape specific details from web pages.

[Rob Gaddi]

On Tue, 12 Aug 2014 13:00:30 -0700 (PDT)
Simon Evans  wrote:

> Dear Programmers,
> I have been looking at the You tube 'Web Scraping Tutorials' of Chris Reeves. 
> I have tried a few of his python programs in the Python27 command prompt, but 
> altered them from accessing data using links say from the Dow Jones index, to 
> accessing the details I would be interested in accessing from the 'Racing 
> Post' on a daily basis. Anyhow, the code it returns is not in the example I 
> am going to give, is not the information I am seeking, instead of returning 
> the given odds on a horse, it only returns a [], which isn't much use. 
> I would be glad if you could tell me where I am going wrong. 
> Yours faithfully
> Simon Evans.
> 
> >>>import urllib
> >>>import re
> >>>htmlfile = urllib.urlopen("http://www.racingpost.com/horses2/cards/card.sd?
> 
> race_id=600048r_date=2014-05-08#raceTabs=sc_")
> htmltext = htmlfile.read()
> regex = '1http://www.racingpost.com/horses/horse_home.sd?
> 
> horse_id=758752"onclick="scorecards.send("horse_name":):return 
> Html.popup(this,
> 
> {width:695,height:800})"title="Full details about this HORSE">Lively 
> 
> Baron9/4F'
> >>>pattern = re.compile(regex)
> >>>odds=re.findall(pattern,htmltext)
> >>>print odds
> []
> >>>
> 
> >>>import urllib
> >>>import re
> >>>htmlfile = urllib.urlopen("http://www.racingpost.com/horses2/cards/card.sd?
> 
> >>>race_id=600048r_date=2014-05-08#raceTabs=sc_")
> >>>htmltext = htmlfile.read()
> >>>regex = ''
> >>>pattern = re.compile(regex)
> >>>odds=re.findall(pattern,htmltext)
> >>>print odds
> []
> >>>
> ---

If you want web scraping, you want to use
http://www.crummy.com/software/BeautifulSoup/ .  End of story.

-- 
Rob Gaddi, Highland Technology -- www.highlandtechnology.com
Email address domain is currently out of order.  See above to fix.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Begginer in python trying to load a .dll

[Mark Lawrence]

On 12/08/2014 20:25, c1234 py wrote:

El martes, 12 de agosto de 2014 16:16:21 UTC-3, Christian Gollwitzer  escribió:

Am 12.08.14 20:36, schrieb c1223:


Hi, Im working in the development of a program based in python that



allow us to contrl a spectometer. The spectometer has an .dll file.



The idea is to work through this dll and operate the spectometer. The



name of the .dll is AS5216.dll. I've trying with ctype, but it



doesn't work. I'm begginer in python. All the comments are useful to



me.




For sure this DLL will come with a header file for C. You could pass it

through SWIG and see if it generates a useful Python module. You need a

C compiler for that route, though.



Christian



How can i pass it through a SWIG?, i don't know that and what it mean?



Start here http://www.swig.org/ which you could easily have found for 
yourself, as I did by using a search engine.


Also would you please access this list via 
https://mail.python.org/mailman/listinfo/python-list or read and action 
this https://wiki.python.org/moin/GoogleGroupsPython to prevent us

seeing double line spacing and single line paragraphs, thanks.

--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

--
https://mail.python.org/mailman/listinfo/python-list

Re: Python in financial services

[Larry Martell]

On Tue, Aug 12, 2014 at 3:33 AM, Rustom Mody  wrote:
> Ive been asked to formulate a python course for financial services folk.
>
> If I actually knew about the subject, I'd have fatter pockets!
> Anyway heres some thoughts. What I am missing out?
>
> [Apart from basic python -- contents typically needs tailoring to the 
> audience] the following:
>
> - Libraries -- Decimal?
> - scripts -- philosophy and infrastructure eg argparse, os.path
> - Pandas
> - Numpy Scipy (which? how much?)
> - ipython + matplotlib + ??
> - Database interfacing
> - Excel interfacing (couple of libraries.. which?)
> - C(C++?) interfacing paradigms -- ranging from ctypes, cython to classic 
> lo-level

I'm not 100% sure what you're looking for. I work for a hedge fund and
we make extensive use of python. Everything from soup to nuts: ETL,
web scraping, database access (sybase, MySQL, and Oracle), log file
archiving and reaping, wrappers for backups, startup and shutdown
scripts for our C++ servers, GIUs (with wxpython), socket based
communication with C++ servers, just about every problem that comes
up.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Suitable Python code to scrape specific details from web pages.

[Roy Smith]

In article ,
 Simon Evans  wrote:

> Dear Programmers,
> I have been looking at the You tube 'Web Scraping Tutorials' of Chris Reeves. 
> I have tried a few of his python programs in the Python27 command prompt, but 
> altered them from accessing data using links say from the Dow Jones index, to 
> accessing the details I would be interested in accessing from the 'Racing 
> Post' on a daily basis. Anyhow, the code it returns is not in the example I 
> am going to give, is not the information I am seeking, instead of returning 
> the given odds on a horse, it only returns a [], which isn't much use. 
> I would be glad if you could tell me where I am going wrong. 

Rather than comment on your specific code (but, thank you for posting 
it), I'll make a couple of more generic suggestions.

First, if you're doing anything with fetching web pages, install the 
wonderful requests module (http://docs.python-requests.org/en/latest/).  
It's so much easier to work with than urllib.

Second, if you're going to be parsing web pages, trying to use regexes 
is a losing game.  You need something that knows how to parse HTML.  The 
canonical answer is lxml (http://lxml.de/), but Beautiful Soup 
(http://www.crummy.com/software/BeautifulSoup/) is less intimidating to 
use.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Using asyncio workers in a `concurrent.futures` interface

[Ian Kelly]

On Tue, Aug 12, 2014 at 11:02 AM, cool-RR  wrote:
> And that's it, no coroutines, no `yield from`. Since, if I understand 
> correctly, asyncio requires a mainloop, it would make sense for the 
> AsyncIOExecutor to have a thread of its own in which it could run its 
> mainloop.

I think that putting the event loop in a separate thread would be
necessary if the intention is that the executor be invoked from
outside. I'm not aware of this work having been done, but it sounds
perfectly feasible.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: how to get the ordinal number in list

[Terry Reedy]

On 8/12/2014 1:40 PM, Neil D. Cerutti wrote:

I disagree.  For a beginner, you want to be able to break things down
into individual steps and examine the result at each point.  If you do:


l= [6,2,9,12,1,4]
l2 = sorted(l,reverse=True)


you have the advantage that you can stop after creating l2 and print it
out.  The student can see that it has indeed been sorted.


Or one can put the multiple steps in a file and step through with a 
debugger that shows local name values, like the Idle debugger.


--
Terry Jan Reedy

--
https://mail.python.org/mailman/listinfo/python-list

Re: Suitable Python code to scrape specific details from web pages.

[Simon Evans]

On Tuesday, August 12, 2014 9:00:30 PM UTC+1, Simon Evans wrote:
> Dear Programmers,
> 
> I have been looking at the You tube 'Web Scraping Tutorials' of Chris Reeves. 
> I have tried a few of his python programs in the Python27 command prompt, but 
> altered them from accessing data using links say from the Dow Jones index, to 
> accessing the details I would be interested in accessing from the 'Racing 
> Post' on a daily basis. Anyhow, the code it returns is not in the example I 
> am going to give, is not the information I am seeking, instead of returning 
> the given odds on a horse, it only returns a [], which isn't much use. 
> 
> I would be glad if you could tell me where I am going wrong. 
> 
> Yours faithfully
> 
> Simon Evans.
> 
> 
> 
> >>>import urllib
> 
> >>>import re
> 
> >>>htmlfile = urllib.urlopen("http://www.racingpost.com/horses2/cards/card.sd?
> 
> 
> 
> race_id=600048r_date=2014-05-08#raceTabs=sc_")
> 
> htmltext = htmlfile.read()
> 
> regex = '1http://www.racingpost.com/horses/horse_home.sd?
> 
> 
> 
> horse_id=758752"onclick="scorecards.send("horse_name":):return 
> Html.popup(this,
> 
> 
> 
> {width:695,height:800})"title="Full details about this HORSE">Lively 
> 
> 
> 
> Baron9/4F'
> 
> >>>pattern = re.compile(regex)
> 
> >>>odds=re.findall(pattern,htmltext)
> 
> >>>print odds
> 
> []
> 
> >>>
> 
> 
> 
> >>>import urllib
> 
> >>>import re
> 
> >>>htmlfile = urllib.urlopen("http://www.racingpost.com/horses2/cards/card.sd?
> 
> 
> 
> >>>race_id=600048r_date=2014-05-08#raceTabs=sc_")
> 
> >>>htmltext = htmlfile.read()
> 
> >>>regex = ''
> 
> >>>pattern = re.compile(regex)
> 
> >>>odds=re.findall(pattern,htmltext)
> 
> >>>print odds
> 
> []
> 
> >>>
> 
> ---
Dear Programmers, Thank you for your responses. I have installed 'Beautiful 
Soup' and I have the 'Getting Started in Beautiful Soup' book, but can't seem 
to make  any progress with it, I am too thick to make much use of it. I was 
hoping I could scrape specified stuff off Web pages without using it. I have 
installed 'Requests' also, is there any code I can use that you can suggest 
that can access the sort of Web page values that I have referred to ?  such as 
odds, names of runners, stuff like that off the 'inspect element' or 'source' 
htaml pages, on www.Racingpost.com. 
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Suitable Python code to scrape specific details from web pages.

[Steven D'Aprano]

Simon Evans wrote:

> Dear Programmers, Thank you for your responses. I have installed
> 'Beautiful Soup' and I have the 'Getting Started in Beautiful Soup' book,
> but can't seem to make  any progress with it, I am too thick to make much
> use of it. I was hoping I could scrape specified stuff off Web pages
> without using it.

Yes, you can scrape stuff off web pages without programming. What you do is
you open the web page in your browser, then open a notebook and, with a
pencil or pen, copy the bits you read into the notebook.

If you're very skilled, you can avoid the pencil and paper and type directly
into a text editor on the computer.

But other than that, every website is different, so there is no short-cut to
web scraping. You need to customize the scraping code for each website you
scrape, and that means programming. Do you know how to program? Are you
interested in learning? If the answer is No and No, then I suggestion you
pony up some money and pay somebody who already knows how to program to do
the job for you.

If the answer is No and Yes, then start at the beginning. Do some
programming tutorials, learn to program the basics before moving on to
something moderately difficult like web scraping.

If the answer is that you already know how to program, but just don't know
how to do web scraping, then stick with it and you'll get there. Web
scraping is tricky, but possible, and if you work hard at it you'll
succeed. Unless you're an experienced programmer with all the right skills,
don't expect this to be something you do in a few minutes. Depending on
your level of experience, you could expect to spend dozens of hours to
learn how to scrape a single website. (Fortunately, the second website will
probably be a little easier, and the third easier still. By the time you've
done a dozen, you'll wonder what the fuss was about.) 

By studying how other scraping programs work, and studying how your racing
pages store data, you should be able to put the two together and see how to
get the data you want. There's plenty of information to help you learn how
to web scrape, with or without BeautifulSoup:

https://startpage.com/do/search/?q=beautifulsoup+web+scraping

https://ixquick.com/do/search/?q=python+web+scraping+examples

https://duckduckgo.com/html/?q=requests%20python%20web%20scraping%20example

but no alternative to actually writing code.


> I have installed 'Requests' also, is there any code I 
> can use that you can suggest that can access the sort of Web page values
> that I have referred to ?  such as odds, names of runners, stuff like that
> off the 'inspect element' or 'source' htaml pages, on www.Racingpost.com.

Specifically those pages? Doubtful.

If you are really lucky (1) somebody else has already done the programming,
(2) they've made their program available to others, and (3) you can find
that program on the Internet. Use the search engine of your choice to
search for it.



-- 
Steven

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Suitable Python code to scrape specific details from web pages.

[Roy Smith]

In article <53eaab7d$0$29979$c3e8da3$54964...@news.astraweb.com>,
 Steven D'Aprano  wrote:

> By studying how other scraping programs work, and studying how your racing
> pages store data, you should be able to put the two together and see how to
> get the data you want.

It's also worth mentioning, that some web sites *want* you to have their 
data, and make it easy to do so by exposing it via public APIs or other 
download methods.  Wikipedia.  Many government web sites.  Twitter.  
Facebook.  Reddit.

Whenever you start thinking about web scraping, it's always worth 
spending a little time investigating if such an API exists.  If it does, 
that's where you want to go.  If not, well, there's always Beautiful 
Soup :-)
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: how to get the ordinal number in list

[Steven D'Aprano]

Chris Angelico wrote:

> On Mon, Aug 11, 2014 at 7:44 PM, Steven D'Aprano 
> wrote:
>> I think this is why both declarative and functional programming idioms
>> will remain niche (although important niches). Most tasks are inherently
>> imperative to at least some degree, and often a *great* degree.
> 
> Maybe that's true as a whole, but there are certainly ways in which
> certain declarative or functional elements can be extremely useful to
> an otherwise-imperative program.

Oh yes! I think that using functional idioms is *very* valuable. List comps
and similar are great, but more than that, the idea of writing idempotent
functions with no side-effects (except IO) is (I believe) vital for good
programming. Relying on argument passing rather communicating by global
variables (or the OO equivalent, instance attributes) is likewise vital.

It's not *quite* impossible to write good, reliable code without functional
idioms, in the same sense that it is conceivable that somebody might write
bug-free unstructured spaghetti code, but I think one of the failures of OO
programming is the over-reliance on methods that communicate via
side-effects. I don't insist that side-effects are completely verboten, as
some functional programming purists do, but I do endeavour to ensure that
any side-effects are limited, encapsulated, and introduce as little
coupling as practical.

(It is my belief that inappropriate coupling is the great evil in
programming. Having too much coupling between parts of your code which
ought to be independent is like Ebola for reliable code, contagious and
deadly. I'm always looking for ways to reduce coupling between parts of my
code, and functional idioms are good for that.)



> Python's (list etc) comprehensions 
> are broadly functional in style, 

No surprise, since list comps were stolen from Haskell :-)


[...]
> I've sometimes done some extremely declarative coding in places; my
> MUD client builds its menus by looking for specially-named functions
> and getting some metadata from them to work out what the menu item
> name should be. An approximate Python equivalent would be:
> 
> @filemenu("E_xit")
> def exit():
> prompt("Do you really want to exit?")
> 
> where the presence of the function is what causes the menu item to
> exist. In this case, the 'filemenu' decorator is probably imperative
> code that adds the menu item to the File menu, but if you're reading
> through a source file that has a bunch of functions like that, you'd
> have to agree that that's a declarative style of coding. And it's a
> style that works well for this kind of thing.

I don't think I would agree that's declarative style. I think that's a form
of imperative programming, where the syntax is:

@ make menu
def function: ...

rather than:

def function: ...
make menu (function)

but I wouldn't start a Holy War over it :-)

There's a fair amount of overlap between the major programming paradigms,
and hence disagreement as to what falls under which paradigm. For instance,
FOLDOC has a good, simple distinction in its definition for "imperative":

[quote]
The Free On-line Dictionary of Computing (20 July 2014) [foldoc]

imperative language
imperative
imperative programming

Any programming language that specifies explicit
   manipulation of the state of the computer system, not to be
   confused with a procedural language, which specifies an
   explicit sequence of steps to perform.

   An example of an imperative (but non-procedural) language is a
   data manipulation language for a relational database
   management system.  This specifies changes to the database
   but does not necessarily require anyone to specify a sequence
   of steps.

   Both contrast with declarative languages, which specify
   neither explicit state manipulation nor a sequence of steps.
[end quote]

You'll note that it suggests SQL would count as an imperative language, but
Wikipedia's article on declarative languages gives SQL as a paragon of
declarative languages!

http://en.wikipedia.org/wiki/Declarative_programming


In Python terms, we might consider "import" to be declarative, since it
specifies what to do (load a name from a module) but not how to perform
it. "import spam" might find spam anywhere, in any form (source code, byte
code, machine code, in a zip file, somewhere on the disk, inside the Python
executable itself), which to my mind suggests a declarative idiom. Or such
things as test discovery, where unittest and doctest will automatically
locate and run tests. (The tests themselves are typically written in a
procedural paradigm.) Otherwise, Python doesn't really have a lot to offer
in the declarative paradigm.


-- 
Steven

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Suitable Python code to scrape specific details from web pages.

[Peter Pearson]

On Tue, 12 Aug 2014 15:44:58 -0700 (PDT), Simon Evans wrote:
[snip]
> Dear Programmers, Thank you for your responses. I have installed
> 'Beautiful Soup' and I have the 'Getting Started in Beautiful Soup'
> book, but can't seem to make any progress with it, I am too thick to
> make much use of it. I was hoping I could scrape specified stuff off
> Web pages without using it.

I've only used BeautifulSoup a little bit, and am no expert, but
with it one can do wonderfully complex things with simple code.
Perhaps you can find some examples online; this newsgroup sometimes
has awesome demonstrations of BS prowess.

At the risk of embarrassing myself in public, I'll show you some
code I wrote that scrapes data from a web page containing a
description of a drug.  The drug's web page contains the desired
data in tags that look like this:



The following code finds all these tags and builds a dict by which you
can lookup the "value" for any given "name".

from BeautifulSoup import BeautifulSoup as BS
...

def dump_drug_data(url):
"""Fetch data from one drug's URL and print selected fields in columns.
"""
contents = urllib2.urlopen(url=url).read()
soup = BS(contents)
inputs = soup.findAll("input")
input_dict = dict((i.get("name"), i.get("value")) for i in inputs)
print(" ".join(f.format(input_dict[n]) for f, n in (
("{0:5s}", "form.widgets.absorption_halflife"),
("{0:5s}", "form.widgets.elimination_halflife"),
("{0:5s}", "form.widgets.minconcentration"),
("{0:5s}", "form.widgets.maxconcentration"),
("{0:13s}", "form.widgets.title"),
)))

Try giving a more specific picture of your quest, and it's very
likely that people smarter than me will give you good help.

-- 
To email me, substitute nowhere->spamcop, invalid->net.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: how to get the ordinal number in list

[Chris Angelico]

On Wed, Aug 13, 2014 at 10:47 AM, Steven D'Aprano
 wrote:
> Chris Angelico wrote:
>
>> On Mon, Aug 11, 2014 at 7:44 PM, Steven D'Aprano 
>> wrote:
>>> I think this is why both declarative and functional programming idioms
>>> will remain niche (although important niches). Most tasks are inherently
>>> imperative to at least some degree, and often a *great* degree.
>>
>> Maybe that's true as a whole, but there are certainly ways in which
>> certain declarative or functional elements can be extremely useful to
>> an otherwise-imperative program.
>
> Oh yes! I think that using functional idioms is *very* valuable. List comps
> and similar are great, but more than that, the idea of writing idempotent
> functions with no side-effects (except IO) is (I believe) vital for good
> programming. Relying on argument passing rather communicating by global
> variables (or the OO equivalent, instance attributes) is likewise vital.
> ...
> (It is my belief that inappropriate coupling is the great evil in
> programming. Having too much coupling between parts of your code which
> ought to be independent is like Ebola for reliable code, contagious and
> deadly. I'm always looking for ways to reduce coupling between parts of my
> code, and functional idioms are good for that.)

There are times when coupling is unavoidable, but yes, it's usually
detrimental to the code. Sometimes the alternatives are worse, but
coupling is still bad. I have TODOs against certain pieces of code
where I can't think of a clean way to decouple them...

>> I've sometimes done some extremely declarative coding in places; my
>> MUD client builds its menus by looking for specially-named functions
>> and getting some metadata from them to work out what the menu item
>> name should be. An approximate Python equivalent would be:
>>
>> @filemenu("E_xit")
>> def exit():
>> prompt("Do you really want to exit?")
>>
>> where the presence of the function is what causes the menu item to
>> exist. In this case, the 'filemenu' decorator is probably imperative
>> code that adds the menu item to the File menu, but if you're reading
>> through a source file that has a bunch of functions like that, you'd
>> have to agree that that's a declarative style of coding. And it's a
>> style that works well for this kind of thing.
>
> I don't think I would agree that's declarative style. I think that's a form
> of imperative programming, where the syntax is:
>
> @ make menu
> def function: ...
>
> rather than:
>
> def function: ...
> make menu (function)
>
> but I wouldn't start a Holy War over it :-)

In a sense, yes. That's partly because Python doesn't really have the
concept that I'm using here. Here's the actual code (it's Pike):

constant file_closewindow="E_xit";
int closewindow()
{
... actual body of function isn't significant ...
}

The key here is the module-level constant which is "file_" followed by
the name of a function. So a more direct translation into Python would
be:

file_closewindow="E_xit"
def closewindow():
... blah blah ...

but since Python code tends to have uppercase constants, this would be
frowned on. Also, it's hard in Python to "enumerate constants", as
there's no such thing (Pike's constants are available at a slightly
different level, which makes them easy to find).

What might make it more declarative than imperative, even in Python,
is that the *order* of items on the menu is alphabetical by function
name, and thus has nothing to do with their position in the file. If
the decorator actually added the items to the menu immediately, their
positions would be based on order in the file; to do them
alphabetically by function name, you basically need to collect them
all up and create them all at the end. That's how the code works in
Pike - a quick loop in the initialization code runs over the constants
and adds the menu items, which means the constants are completely
non-executable.

It really is hard to draw the line. Ultimately, the CPU (at least, if
it's of any architecture I've worked with) doesn't execute declarative
or functional code, just as it doesn't execute callbacks or subthread
multiplexing or anything like that. You need to have, somewhere, a bit
of imperative code (usually a simple loop) that deals with the other
types... which might be called an interpreter. In fact, *all* high
level code is, in a sense, declarations to a lower level interpreter -
consider the distinctions between various Python interpreters (even
some that run inside your web browser), and how they all obey the same
declarations, but implement them completely differently.

> [ quoting FOLDOC ]
>An example of an imperative (but non-procedural) language is a
>data manipulation language for a relational database
>management system.  This specifies changes to the database
>but does not necessarily require anyone to specify a sequence
>of steps.
>
>Both contrast with declarative languages, which specify
>neither explic

Re: Captcha identify

[Wesley]

If my questions make you guys not so happy, I am sorry and please just ignore.
I just wanna a general suggestion here in the beginning.
Why I need to write such program is just having such requirements, and has 
nothing to do with the coding work itself. Don't say something to prove you're 
so noble.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Captcha identify

[Chris Angelico]

On Wed, Aug 13, 2014 at 11:36 AM, Wesley  wrote:
> If my questions make you guys not so happy, I am sorry and please just ignore.
> I just wanna a general suggestion here in the beginning.
> Why I need to write such program is just having such requirements, and has 
> nothing to do with the coding work itself. Don't say something to prove 
> you're so noble.

The general suggestion you're getting is: Do not do this. Many of us
here use CAPTCHAs and spend time keeping one step ahead of those who
try to break them with software. By writing something to solve
CAPTCHAs, you would be stealing time from those people. Don't do it.

Am I sufficiently clear?

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: how to get the ordinal number in list

[Roy Smith]

In article ,
 Chris Angelico  wrote:

> I like to look at SQL as a language that specifies an end result 
> without specifying how to get there

Well, sure, but sometimes the how to get there is a matter of 10x, or 
100x, or 1000x in performance.

I'm currently migrating a 3 TB database to a new 5 TB RAID array.  Our 
initial guess is it's going to take two weeks to finish :-(
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Captcha identify

[Ben Finney]

Wesley  writes:

> If my questions make you guys not so happy, I am sorry and please just
> ignore.

You seek to dismiss the valid concerns by calling them “not so happy”.

You assert it is questions we object to, when we are clearly objecting
to your intentions.

Don't be disingenuous. The responses you get are because there are valid
concerns about the motives you are pursuing.

> Why I need to write such program is just having such requirements

This is a circular statement, almost a tautology. If that is the best
you can present in defense of harmful and deceitful actions, then there
is no good reason for what you're doing.

Stop, please.

-- 
 \“The World is not dangerous because of those who do harm but |
  `\  because of those who look at it without doing anything.” |
_o__) —Albert Einstein |
Ben Finney

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: how to get the ordinal number in list

[Chris Angelico]

On Wed, Aug 13, 2014 at 11:45 AM, Roy Smith  wrote:
> In article ,
>  Chris Angelico  wrote:
>
>> I like to look at SQL as a language that specifies an end result
>> without specifying how to get there
>
> Well, sure, but sometimes the how to get there is a matter of 10x, or
> 100x, or 1000x in performance.

Of course it can! But generally, the "how to get there" is not
stipulated in the SQL statement. If you say, for instance:

select * from some_table where some_column='some_value';

then an index on some_column will make a huge difference to
performance - without changing this statement at all. In contrast,
taking advantage of an index in BTrieve requires recoding your
program, as the index used is a part of the API call. (At least, this
is true of the particular version of BTrieve that back-ended our
accounting package in the 90s. I was able to turn an overnight job
into a thirty-second job by rewriting it as an external program; the
original is closed-source so I can't be sure, but I suspect most of
that improvement is simply because I used an index.)

SQL itself doesn't even have provision for indexes. The 'CREATE INDEX'
command, found in many databases, is an extension. :)

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: how to get the ordinal number in list

[Steven D'Aprano]

Rustom Mody wrote:

> I guess part of the problem is that evidently you and I use 'concrete'
> and 'abstract' in almost diametrically opposite ways.
[...]
> But now you are turning the tables and saying that the FP types/constructs
> are more abstract than the conventional ones.
[...]
> In my book, FP is easy and pleasant and smooth because (among other
> things) FP types are concrete, whereas OOP types are abstract.

I wrote about how your view of sorted() was an abstraction, not an Abstract
Base Class. I thought that was clear enough: you ignore the concrete
details of how one sorts (bubblesort, quicksort, timsort, ...) and
everything but the result returned. I shouldn't think that is
controversial. Let me repeat what I said at the time:

You are abstracting away the details of the real computer 
implementation into an abstraction of a pure mathematical
function...

I am not talking about *abstract types* in the object-oriented sense.

I then went on to say:

But abstractions leak, and someday someone is going to ask 
"Why does it take 45 minutes to find the five largest values
of my list?", and if you persist in insisting that sorted()
is a pure mathematical function you will have no clue on how
to even begin solving this [...]

(I don't actually believe that *you personally* are incapable of thinking
about the implementation of sorted(). It is a hypothetical situation.)

For the record, Wikipedia defines abstract types as:
  
In programming languages, an abstract type is a type in a
nominative type system which cannot be instantiated directly.
Abstract types are also known as existential types.

and contrasts them with "concrete types" like Python's float, list, str.

I am using "abstract" in the sense of an abstraction, as opposed to
something concrete and real, not as a mechanism for specifying interfaces in
Java or Python. Python's float is a leaky abstraction of mathematical Real
numbers, and Python's softed() is a leaky abstraction of the concept of
sorting a collection. Both are abstractions, because they are *intended* to
gloss over real world details of Real numbers and sorting, but both leak
because the implementation of the abstraction spoils the abstraction.

http://www.joelonsoftware.com/articles/LeakyAbstractions.html



-- 
Steven

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: how to get the ordinal number in list

[Chris Angelico]

On Wed, Aug 13, 2014 at 12:11 PM, Steven D'Aprano
 wrote:
> both leak
> because the implementation of the abstraction spoils the abstraction.

Can you name *any* nontrivial abstraction that doesn't leak? I can't.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: how to get the ordinal number in list

[Steven D'Aprano]

Steven D'Aprano wrote:
[...]
> I am using "abstract" in the sense of an abstraction, as opposed to
> something concrete and real, not as a mechanism for specifying interfaces
> in Java or Python. Python's float is a leaky abstraction of mathematical
> Real numbers, and Python's softed() is a leaky abstraction of the concept
> of sorting a collection. 

Oh, I forgot to mention... "sorting a collection" is itself an abstraction
(that is, a generalisation) of concrete examples of sorting, such as
sorting a hand of cards, sorting an unruly crowd of school children into
order of height, sorting out a messy cutlery drawer, or sorting the books
in a library.

Abstractions are good and necessary, being a programmer is about learning to
invent and use abstractions, but both object-oriented and functional
programming circles are prone to over-abstraction and that's why they're a
an ill-fit for teaching beginners.

To a beginner, functions and their return result is an abstraction (and
quite a hard one for many people to learn). The natural tendency is to
write functions like this:

def frobnicate(value):
print (value + spam + eggs)


because they're thinking "how do I get the frobnicated value?" and the
obvious interface between "I" (the beginner) and the computer is visual.
They haven't learned the essential abstraction known as "returning a value"
yet and so they use:

- calculate the frobnicated value
- print it

instead of the generalisation:

- calculate the frobnicated value
- return[1] it to the caller[2]
- which captures that return result somewhere[3]
- and finally prints it

or possibly this one:

- calculate the frobnicated value
- return it to the REPL
- which prints it then throws it away






[1] What does "return" in the context of functions even mean? Try explaining
it to a beginner who doesn't understand functions. The best I can come up
with is you just keep showing them examples until they draw the connection
themselves.

[2] Who?

[3] Where?




-- 
Steven

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: how to get the ordinal number in list

[Rustom Mody]

On Wednesday, August 13, 2014 12:59:35 AM UTC+5:30, Neil D. Cerutti wrote:

> > Some things follow from this:
> > For the lego-game of playing with functions at the REPL to work and be
> > pleasant and rewarding:
> > 1. functions should be non side-effecting; else same trials giving different
> > answers adds more confusion than understanding
> > 2. They should be non-printing else:
> > def foo(x): return x+1
> > def bar(x): print x+1
> > look similar when trivially tried but compositionally are utterly different
> > In effect a printing function breaks the lego bricks

> That may be so, but printing stuff to the screen is very natural to 
> people. I've downloaded Haskell a few times, but the knowledge that 
> getting input and writing output requires something mysterious called 
> gonads just frightens me.

I thought they were quite common 

But yes...

- Monads in Haskell
- Pointers in C
- Inheritance in Java/Python

all suffer the same problem -- glorification of the failure-mode.

When one uses pointers, especially non-trivial pointer arithmetic in C,
one is essentially blasting a hole in the floor of the hi-level facade
of C and working at machine level.

Inheritance produces coupling whereas the whole intent of modularity is to 
avoid coupling.

Likewise Monads are for imperative programming in Haskell.
Maybe better to just use an imperative language?

> Functional programming could be particularly hard to teach since it is
> generally made up of numerous small units of work combined in a complex
> way. This is precisely the formula for something that beginners will
> find extremely challenging. 

> When functional programming is dumbed down enough for a beginner to be 
> able to grok it, the programming problems start to look really lame. It 
> needn't be that way, of course, but it takes a good deal of creativity 
> on the part of the instructor. If Factorial doesn't turn you on, you 
> might be screwed. ;)

Did you see my example a bit earlier on Catalan numbers?  I believe it
explains them better -- certainly more succinctly -- than the
math+verbiage you can find at:
http://en.wikipedia.org/wiki/Catalan_number

Yes it may be hard to grok but I think its the exact opposite of
'numerous small units'
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Using asyncio workers in a `concurrent.futures` interface

[Marko Rauhamaa]

Ian Kelly :

> On Tue, Aug 12, 2014 at 11:02 AM, cool-RR  wrote:
>> And that's it, no coroutines, no `yield from`. Since, if I understand
>> correctly, asyncio requires a mainloop, it would make sense for the
>> AsyncIOExecutor to have a thread of its own in which it could run its
>> mainloop.
>
> I think that putting the event loop in a separate thread would be
> necessary if the intention is that the executor be invoked from
> outside. I'm not aware of this work having been done, but it sounds
> perfectly feasible.

Multithreading will require normal locking to protect critical sections.
Care must be taken to never yield while holding a threading lock.


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Using asyncio workers in a `concurrent.futures` interface

[Ian Kelly]

On Tue, Aug 12, 2014 at 11:03 PM, Marko Rauhamaa  wrote:
>
> Ian Kelly :
>
> > On Tue, Aug 12, 2014 at 11:02 AM, cool-RR  wrote:
> >> And that's it, no coroutines, no `yield from`. Since, if I understand
> >> correctly, asyncio requires a mainloop, it would make sense for the
> >> AsyncIOExecutor to have a thread of its own in which it could run its
> >> mainloop.
> >
> > I think that putting the event loop in a separate thread would be
> > necessary if the intention is that the executor be invoked from
> > outside. I'm not aware of this work having been done, but it sounds
> > perfectly feasible.
>
> Multithreading will require normal locking to protect critical sections.
> Care must be taken to never yield while holding a threading lock.

You can use event_loop.call_soon_threadsafe() to schedule tasks and
callbacks on the event loop, and Queue objects to pass futures back to the
caller. Apart from whatever synchronization those use internally, I don't
think any locking would be needed.

However, it would of course require that the "functions" passed in to the
executor be coroutines in actuality. You just can't pass a blocking
function into an asynchronous framework and expect it to magically not
block.
-- 
https://mail.python.org/mailman/listinfo/python-list

[Announce] Python-Future v0.13; cheatsheet for Python 2/3 compatible code

[Ed Schofield]

Hi all,

I am happy to announce an update to Python-Future for Python 2/3 compatibility 
and a new cheat-sheet for writing code compatible with both versions.

Here’s the "What’s New" page for v0.13:

http://python-future.org/whatsnew.html

Here’s the Py2/3 compatibility cheat-sheet:

http://python-future.org/compatible_idioms.html

or as a PDF:

http://python-future.org/compatible_idioms.pdf

The cheat-sheet accompanies a talk I gave at PyCon AU 2014 last weekend called 
“Writing Python 2/3 compatible code”. I will add a link to the video and slides 
from the cheat-sheet page when they are online.

I would be happy to accept pull requests for additions or changes to the Py2/3 
cheat-sheet. The source is here:


https://github.com/PythonCharmers/python-future/blob/master/docs/notebooks/Writing%20Python%202-3%20compatible%20code.ipynb

Best wishes,
Ed


--
Dr. Edward Schofield
Python Charmers
http://pythoncharmers.com

-- 
https://mail.python.org/mailman/listinfo/python-list

newbee

[Frank Scafidi]

I just acquired a Raspberry Pi and want to program in Python. I was a PL/1
programmer back in the 60's & 70's and Python is similar. I am struggling
with some very fundamental things that I am not finding in the
documentation. Can someone help me with the basics like how do I save a
program I've written, reload it in Python, list the program once it's
loaded? How do I edit a program? Are these command line functions?

Thanks
Frank
-- 
https://mail.python.org/mailman/listinfo/python-list