Re: A use-case for for...else with no break

2017-11-02 Thread Paul Rubin
Steve D'Aprano  writes:
> for x in something():
> print(x, end='')

print(''.join(something()))
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Let's talk about debuggers!

2017-10-26 Thread Paul Rubin
Terry Reedy  writes:
> On Windows, [IDLE] uses native widgets when possible...
> In summary, I think debugger should rate at least 'good' rather than
> fail' when it comes to showing you the next line.

I actually like how the Tk widgets look.  I've done some semi-industrial
applications with tkinter and they have a nice factory-floor vibe.

I generally just use pdb for debugging.  The feature I miss most is the
ability to trap to the debugger if the program throws an unhandled
exception.  I think some other Python debuggers do support that.  I've
found it invaluable in other languages.

-- 
https://mail.python.org/mailman/listinfo/python-list


Re: I'd like to use "semantic indentation"

2017-09-30 Thread Paul Rubin
Chris Angelico  writes:
> USA:
> Alabama:
> Abbeville
> Addison
> ...
> and then, as Paul suggested, write a simple parser to read it.

That looks like YAML, which there's already a library for.  I'm not
crazy about it but it might be an ok choice for this.
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: I'd like to use "semantic indentation"

2017-09-30 Thread Paul Rubin
r...@zedat.fu-berlin.de (Stefan Ram) writes:
>   I would like to write source code similar to:
> country( 'USA' )
>   state( 'Alabama' )

Aside from the workaround that I mentioned, this looks more like data
than code.  Maybe you really want to create a nested structure
(dictionaries, JSON, XML or whatever) and traverse it.
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: I'd like to use "semantic indentation"

2017-09-30 Thread Paul Rubin
r...@zedat.fu-berlin.de (Stefan Ram) writes:
>   I would like to write source code similar to:
> country( 'USA' )
>   state( 'Alabama' ) ...

> It seems I can't do this with Python.  Is there any workaround?

  _= country( 'USA' )
  _=   state( 'Alabama' )
  _= town( 'Abbeville' )
  _= town( 'Addison' )
  _=   state( 'Arizona' )
  _= town( 'Apache Junction' )
  _= town( 'Avondale )
  _= town( 'Benson' )
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Project Euler 20.

2017-09-24 Thread Paul Rubin
Ian Kelly  writes:
 sum(int(c) for c in str(math.factorial(100)))

Doh!  Using int(c) didn't occur to me and I didn't know about
math.factorial.  Notice also that WJ hasn't yet dared posting his crap
on comp.lang.haskell.
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Project Euler 20.

2017-09-24 Thread Paul Rubin
"Robert L."  writes:
>> Find the sum of the digits in the number 100!
> In Python?

So you have come to plague us here too.

>>> sum(ord(c)-ord('0') for c in str(reduce(lambda a,b: a*b, range(1,101),1)))
648
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Old Man Yells At Cloud

2017-09-21 Thread Paul Rubin
Steve D'Aprano  writes:
> Having to spend a few hours being paid to migrate code using "print x"
> to "print(x)", or even a few months, is not a life-changing experience.

Didn't someone further up the thread mention some company that had spent
1.5 years porting a py2 codebase to py3?

The issue of breaking the print statement isn't the difficulty of
converting old programs, or that the print statement is superior to the
print function or vice versa.  Reasonable people might believe that one
is slightly better than the other, but it would be hard to argue that
one is overwhelmingly better than the other.  So there's not a
convincing reason to change.

That calls the whole py3 movement into question, since its advocates so
vigorously defend unnecessary changes.  It's fine to fix broken stuff
(Unicode was broken, indexes escaping list comprehensions was broken)
but fixing highly visible stuff that wasn't broken makes the more subtle
changes easier to ignore.

Py3 imo would have been more successful if it introduced even more
breakage, but produced dramatic benefits (for example a 10x speedup) as
a result.  That would have been doable.  Instead we got minor benefits
and useless breakage.  Py4 as a result of learning the wrong lesson
won't break anything, so it won't be able to introduce dramatic benefits
either.  Will the 5th time (Py5) be the charm?  (I don't mean Pycharm).

Python is gaining ground in numerics and data science, which is great.
Micropython for embedded MCUs is also poised for popularity.  I don't
know how Python is doing at general systems stuff which is what I mostly
use it for.  I think Ruby is losing ground, and some of the ground it
has lost has been to Elixir.  An Elixir-like reimplementation of Python
might be an interesting avenue to pursue.  So would a dialect with less
pervasive dynamism than Python, but that could be compiled to fast
machine code with traditional Lisp techniques.  The dynamism would still
be available "opt-in" so you could turn it on when you wanted it, and
only those parts of your program would slow down.
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: [RELEASE] Python 3.6.3rc1 and 3.7.0a1 are now available for testing and more

2017-09-19 Thread Paul Rubin
Ned Deily  writes:
> You can find Python 3.7.0a1 and more information here:
> https://www.python.org/downloads/release/python-370a1/

This says:

The next pre-release of Python 3.7 will be 3.6.0a2, currently
scheduled for 2016-10-16.

:)
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Old Man Yells At Cloud

2017-09-16 Thread Paul Rubin
Steve D'Aprano  writes:
>> concept   integer / integer => integer_result
> That would be C, and C derived languages, perhaps?

Certainly not.  Fortran, machine languages, etc. all do that too.

Haskell does the right thing and makes int/int a compile time type
error.  Its integer division functions are div and quot.

Python 2 does something reasonable and Python 3 does something that is
also debatably reasonable.  Switching from one reasonable thing to
another without a very good reason is called fixing things that weren't
broken.  Python 2 wasn't broken in that area and didn't need to be
fixed.
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: The Incredible Growth of Python (stackoverflow.blog)

2017-09-13 Thread Paul Rubin
Ben Finney  writes:
>> I've never seen one.
> who has told you... they are working on a Python 3 code base.

Just because they've told me about it doesn't mean I saw it personally.
The ones I've seen, including new ones, are Python 2.

Some people here use Py3 but I haven't heard (or don't remember) enough
about what they're working on, to know if those are py3 codebases of any
size.  If they say yes, I'll take their word for it, but this is a
self-selected group of course.

> That simply isn't true, unless you think it more likely everyone who
> discusses their Python 3 code base is lying.

People discuss Python language issues here a lot, but don't discuss as
much about code bases.

I know when I install a new OS (currently Debian 9 which was released
a month or so ago) and type "python" on the command line, I get Py2.
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: The Incredible Growth of Python (stackoverflow.blog)

2017-09-12 Thread Paul Rubin
Chris Angelico  writes:
> Why? Unless they're going to be maintaining a Py2 codebase, why should
> they learn the older version with less features?

Are there actually Py3 codebases?  I guess there must be, even though
I've never seen one.  Every Python codebase of any size that I know of
is Py2.  So yes, of course they're working on a Py2 codebase.  That's
the only kind there is, as far as I know.
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Using Python 2

2017-09-11 Thread Paul Rubin
Steve D'Aprano  writes:
> Guido has ruled that Python 4 will not be a major compatibility break

Looking forward to Python 5 then ;-).
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: The Incredible Growth of Python (stackoverflow.blog)

2017-09-11 Thread Paul Rubin
Chris Angelico  writes:
> students learning Python *today* ... they're learning Python 3.

I'm not so sure of that.  I do know a few people currently learning
Python, and they're using Python 2.

>>  * static type annotation

Seems like a big win if you ask me.

>>  * asyncio with its a-dialect
> Actually, I think this one is a huge enough feature that it's going to
> be a big thing to *drive* the uptake of Python.

God I hope not, it's perverse.  I'd like to see a Python that's more
like Erlang in its approach to concurrency.
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: How do I find what kind of exception is thrown.

2017-09-05 Thread Paul Rubin
Antoon Pardon  writes:
> Now I found the following in the logs: [Errno 131] Connection reset by peer
> This is a problem I would like to catch earlier

I'd expect that's an IOError thrown by the socket library.  You could
just let it go uncaught and see what you get in the crash dump.

-- 
https://mail.python.org/mailman/listinfo/python-list


Re: BeautifulSoup doesn't work with a threaded input queue?

2017-08-27 Thread Paul Rubin
Christopher Reimer  writes:
> I have 20 read_threads requesting and putting pages into the output
> queue that is the input_queue for the parser. 

Given how slow parsing is, you probably want to scrap the pages into
disk files, and then run the parser in parallel processes that read from
the disk.  You could also use something like Redis (redis.io) as a queue.
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Reading the documentation

2017-08-24 Thread Paul Rubin
Chris Angelico  writes:
>> And there are numbers which repeat in decimal but not binary, 
> Which ones repeat in decimal but not binary? An example, please.

That should really have said binary but not decimal, since 2 divides 10.

-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Proposed new syntax

2017-08-24 Thread Paul Rubin
Steve D'Aprano  writes:
> Did __next__ cache the most recently generated value?

No but if they're going to change stuff, they might as well actually
improve it instead of just renaming it to break code gratutiously.
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Proposed new syntax

2017-08-24 Thread Paul Rubin
Steve D'Aprano  writes:
> the public API is to call the next() built-in function, and the
> implementation is the __next__ dunder.

In that case it would have been nice to make next() cache the most
recently generated value from the iterator.  That would make lots of
code simpler.
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Proposed new syntax

2017-08-24 Thread Paul Rubin
Peter Otten <__pete...@web.de> writes:
> Python 3 where the next() method has been renamed to __next__().

Oh cripes, you're right, it never occurred to me that py3 had broken
.next().  I thought it was called .next() instead of .__next()
so that it wouldn't be a dunder method.
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Proposed new syntax

2017-08-24 Thread Paul Rubin
Steve D'Aprano  writes:
> I've read a few people claim that disallowing multiplication from
> standard arithmetic renders it weak enough that you can prove it
> complete and correct, but since they give no proof or even evidence I
> have my doubts.

That system is called Presburger arithmetic (see Wikipedia).  It can be
proved consistent by quantifier elimination but is not powerful enough
to prove (or maybe even state) its own consistency.

Surprisingly, there are also theories that can state and prove their own
consistency:

  https://en.wikipedia.org/wiki/Self-verifying_theories
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Proposed new syntax

2017-08-24 Thread Paul Rubin
Ben Finney  writes:
> generate_id = functools.partial(next, itertools.count())

Is something wrong with:

>>> g = itertools.count().next
>>> g()
0
>>> g()
1
>>> g()
2
>>> ...
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Proposed new syntax

2017-08-21 Thread Paul Rubin
Rustom Mody  writes:
> Do you mean Frege or Cantor?

Frege.  Cantor was concerned with set theory, while Frege was concerned
with logic in general.  Frege's notation was different from what we use
now but the issue was about the same: unrestricted comprehension led to
contradiction.  As you mention, Russell was the one who recognized the
inconsistency in Frege's system, though I don't know how Frege took it
at a personal level.

Many of the original writings from that era are reproduced in the book
"From Frege to Gӧdel: A Source Book in Mathematical Logic".  It has good
introductions to the papers and is pretty interesting.
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Proposed new syntax

2017-08-19 Thread Paul Rubin
Rustom Mody  writes:
> Specifically the term 'comprehension' used today as a programming construct
> traces somewhat tenuously to an axiom that Zermelo/Fraenkel formulated
> in the 1920s

I thought went back to Frege.  Also, it appears in Zermelo set theory Z.
ZF is Z with the Axiom of Replacement added, but Z was somewhat earlier
than ZF.  
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Proposed new syntax

2017-08-18 Thread Paul Rubin
Marko Rauhamaa  writes:
> The question is, is it bad style—or even an error—to rely on the
> execution order of the comprehension loops? 

Bad style: IMO, yes, most of the time.  I've made use of it at
particular times.  If it's done in an obscure way it at least
deserves a code commment.

Error: no, I think the language spec is clear.  It's not like the
situation of people depending on reference counting to free resources
predictably.  Use the 'with' statement for that.

> Is a Python implementation allowed to parallelize or otherwise reorder
> the evaluation loop?

No.
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: What extended ASCII character set uses 0x9D?

2017-08-17 Thread Paul Rubin
John Nagle  writes:
> Since, as someone pointed out, there was UTF-8 which had been
> run through an ASCII-type lower casing algorithm

I spent a few minutes figuring out if some of the mysterious 0x81's
could be from ASCII-lower-casing some Unicode combining characters, but
the numbers didn't seem to work out.  Might still be worth looking for
in some other cases.
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Proposed new syntax

2017-08-17 Thread Paul Rubin
Steve D'Aprano  writes:
> For loops and comprehensions (in Python) are inherently procedural,

Sure, and floating point arithmetic is inherently imprecise and doesn't
follow the associative laws for either addition or multiplication.
There are times when we have to be aware of those details.  Usually,
though, we want to act as if they represent the mathematical reals, and
we read Python statements involving floats as if they were mathematical
statements involving reals.  When possible, we write in a style where
this doesn't cause problems, use double precision to decrease rounding
errors in long calculations, etc.

Similarly we occasionally have to be aware of the procedural nature
of Python list comprehensions, but most of the time we think of them
in terms of the mathematical abstraction they are designed to resemble.
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Cross-language comparison: function map and similar

2017-08-16 Thread Paul Rubin
Steve D'Aprano  writes:
> Are there language implementations which evaluate the result of map()
> (or its equivalent) in some order other than the obvious left-to-right
> first-to-last sequential order? Is that order guaranteed by the
> language, or is it an implementation detail?

Haskell just gives back an unevaluated thunk.  The elements are
evaluated in whatever order you happen to use them in.  Since the
evaluation isn't supposed to have observable side effects, there's no
way to tell the order.

There are also parallel versions (Control.Parallel.Strategies.parMap
etc.) and an extension that recognizes "racing stripes" on the square
brackets to make list comprehensions run faster:

   foo = [| sqrt(x) | x <- [1.0 .. 1000.0] |]

parallelizes the calculation and offloads it to a GPU or other vector
processor.  You might also like this old post

 
https://donsbot.wordpress.com/2007/11/29/use-those-extra-cores-and-beat-c-today-parallel-haskell-redux/

"map" is conceptually the lifting of a function from a given type, to
the type under the action of some functor.  For historical reasons map
only works on lists while fmap works on arbitrary functors, but they are
in principle the same thing.  Depending on what the functor does, the
whole concept of left-to-right order might be meaningless.
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Proposed new syntax

2017-08-14 Thread Paul Rubin
Paul Rubin  writes:
>   FORALL P. [ P(0) and P(n) -> P(n+1) ]

Sorry, that was supposed to say

  FORALL P. [ (P(0) and P(n) -> P(n+1)) -> forall n. P(n) ]

FORALL quantifies over formulas and forall quantifies over numbers.

Maybe something is still missing from the above ;-).
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Proposed new syntax

2017-08-14 Thread Paul Rubin
Marko Rauhamaa  writes:
> For the non-logicians in the audience, an "axiom schema" is a generator
> pattern that produces an infinity of actual axioms. 

For non-logicians this is not worth worrying about: schemas are
basically a technical hack to get around the inability of first-order
logic to quantify over formulas.  Naively most of us think of single
axioms with second-order quantifiers, instead of schemas.

For example, arithmetic induction goes something like:

  FORALL P. [ P(0) and P(n) -> P(n+1) ]

where "FORALL P" means quantifying over all formulas P.  Since in FOL we
can only quantify over numbers and not formulas, we write down a
separate axiom for each possible formula.  Since there are infinitely
many formulas, this is an infinite axiom schema.

Historically (in "naive set theory") we didn't bother with any of this.
We could write { S : S \not\in S } for the set of all sets that are not
members of themselves.  Is S a member of itself ("Russell's paradox")?
Either way leads to contradiction.  So the comprehension axiom schemas
for set theory had to be designed to not allow formulas like that.
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Proposed new syntax

2017-08-14 Thread Paul Rubin
Steve D'Aprano  writes:
> It's quite clever, actually, in that it gives *pseudo* random-access
> with lazy evaluation. You can evaluate the Nth element without
> evaluating the earlier ones. But you can't do so without *storing* the
> previous ones, they have to be allocated, with only the actual
> evaluation being delayed.

Look at any of the memoization packages for ways around that.  Or of
course you could just write a function instead of a list...

> In Haskell, you cannot get the last N elements of a list without
> allocating memory for the previous ones.

lastn n xxs@(x:xs)
  | length (take n xs) == n-1 = xxs
  | otherwise = lastn n xs

main = print . lastn 5 $ [1..1000]

*Main> main
[996,997,998,999,1000]

works for me.  The 1000 list nodes all get allocated, but are
immediately freed, so only 5 cells have to be in memory at a time.  In
principle a fancy enough compiler optimization could get rid of all the
allocation completely.
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Proposed new syntax

2017-08-14 Thread Paul Rubin
Steve D'Aprano  writes:
> And very possibly the first language with comprehensions, SETL (from 1969),
> used "forall" ... 

It goes back much further, probably to Principia Mathematica, or (with
different notation) maybe Frege's Foundations of Arithmetic:

  https://en.wikipedia.org/wiki/Set-builder_notation

I don't think Python should try to turn it into something different.
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Proposed new syntax

2017-08-14 Thread Paul Rubin
Marko Rauhamaa  writes:
> Jussi Piitulainen :
>> But what is "set comprehension" in French, German, or Finnish?

The idea comes from set theory: for some reason English Wikipedia
doesn't have Finnish cross-wiki links for most of the relevant
terms, but Google translates "axiom of specification" as
"Määritelmän axiom".Maybe you can find something from there.
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Proposed new syntax

2017-08-11 Thread Paul Rubin
Steve D'Aprano  writes:
> What would you expect this syntax to return?
> [x + 1 for x in (0, 1, 2, 999, 3, 4) while x < 5]

[1,2,3] though the later example is more confusing.

I don't think we need this since we have itertools.takewhile:

  from operator import gt
  from functools import partial
  from itertools import takewhile

  [x + 1 for x in takewhile(partial(gt,5), (0,1,2,999,3,4))]

In the eye of the beholder maybe, but I think it's less ugly in Haskell:

  [x + 1 | x <- takeWhile (< 5) [0,1,2,999,3,4]]
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Challenge: find the first value where two functions differ

2017-08-05 Thread Paul Rubin
Chris Angelico  writes:
> 4503599761588224

I get the same result from searching a wider interval (+/- 50) around
each perfect square in the relevant range.
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: YAML in std lib?

2017-07-31 Thread Paul Rubin
Ben Finney  writes:
> I don't know of any PEP yet which specifies exactly what to add to the
> standard library for YAML

YAML is more of a Ruby thing, so there might not be much constituency
for putting it in Python.
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Falsey Enums

2017-07-28 Thread Paul Rubin
Dan Sommers  writes:
> def __bool__(self):
> return False if self == X.Falsey else True

return self != X.Falsey
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: OT was Re: Python 3 removes name binding from outer scope

2017-07-26 Thread Paul Rubin
Chris Angelico  writes:
>>> Bipartisan-US-Bill-Moves-to-Criminalize-BDS-Support-20170720-0001.html
>> Heh, at first I read that as a bill to criminalise BSD support :-)
> I spluttered my drink on reading that. Good job Steven!

https://en.wikipedia.org/wiki/BDS_C
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Python 3 removes name binding from outer scope

2017-07-24 Thread Paul Rubin
Ben Finney  writes:
> How can I stop Python from deleting a name binding, when that name is
> used for binding the exception that is caught?

Use sys.exc_info()
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Recent Spam problem

2017-07-24 Thread Paul Rubin
Rustom Mody  writes:
> Since spammers are unlikely to be choosy about whom they spam:
> Tentative conclusion: Something about the USENET-ML gateway is more leaky
> out here than elsewhere

It could be a sort-of DOS attack by some disgruntled idiot.  I wonder if
the email address in those spam posts actually works.  Then there's the
weird Italian rants.  No idea about those.
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Write this accumuator in a functional style

2017-07-14 Thread Paul Rubin
Rustom Mody  writes:
> Yeah I know append method is supposedly O(1).

It's amortized O(1).
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Write this accumuator in a functional style

2017-07-13 Thread Paul Rubin
Chris Angelico  writes:
> Maybe I'm completely on the wrong track here, but the last time I
> implemented a self-balancing tree, it usually involved a fair amount
> of mutation.

AVL trees are fairly simple to implement without mutation.  Red-black
trees are traditionally implemented with mutation, inserting by making
nodes mis-colored, then going and re-coloring them.  But they can be
done mutation-free as well.  Here's an amazing Haskell implementation
where the tree invariants are encoded in the datatype:

https://gist.github.com/rampion/2659812

Reddit discussion of above:  https://redd.it/ti5il

More recent versions of GHC make the type signatures even nicer, since
you can put numbers directly into types without that nested type encoding.
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Write this accumuator in a functional style

2017-07-13 Thread Paul Rubin
Chris Angelico  writes:
> some point it'll need to be rebalanced, which could at worst case
> be O(n). 

No, you use a structure like an AVL tree or red-black tree, so it's
within a constant factor of balanced after each insertion.  You rewrite
O(log n) of the nodes, and juggle around a constant number of them at
the top of the tree.  The Wikipedia articles about those data structures
are pretty good.  C++ std::map is also implemented that way, I think.
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Write this accumuator in a functional style

2017-07-12 Thread Paul Rubin
Steven D'Aprano  writes:
> for parrot in parrots:
> accumulator[parrot.colour].append(parrot)
>
> That's pretty compact and understandable, but it require mutating a bunch 
> of pre-allocated lists inside an accumulator. Can we re-write this in a 
> functional style?

Not so easily in Python since the built-in list and dict types are
designed for mutation update.  In Haskell, the list type is a linked
list and the dictionary type is a balanced tree.  So, you can make a new
list consisting of a new item consed to the front of the old list, and
you can make a new ("updated") dictionary by building O(log n) new
nodes.

You might like Chris Okasaki's wonderful book "Purely Functional Data
Structures" that explains all this and more.
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: What's with all of the Case Solution and Test Bank nonsense posts?

2017-07-10 Thread Paul Rubin
Michael Torrie  writes:
>> can you get a newsreader to work with a https news service?
> No.  A newsreader works with NNTP protocol.

Traditionally NNTP over SSL was done on port 563.  Some feeds now also
provide it on 443 to get around client-side firewall hassles.
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: What's with all of the Case Solution and Test Bank nonsense posts?

2017-07-09 Thread Paul Rubin
timetowal...@gmail.com writes:
> What's with all of the Case Solution and Test Bank nonsense posts?
> Is is possible to have these posts filtered out?

As people have said, you can block them with a good news reader.  But
like you, I wonder why the heck they have camped out on this particular
newsgroup.
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Checking for an exception

2017-06-25 Thread Paul Rubin
Steve D'Aprano  writes:
> What's the right/best way to test whether an object is an exception
> ahead of time? (That is, without trying to raise from it.)

Maybe I'm missing something but 
   isinstance(obj, Exception)
seems to work.
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Progress on the Gilectomy (Posting On Python-List Prohibited)

2017-06-21 Thread Paul Rubin
Lawrence D’Oliveiro  writes:
> while “memory footprint” depends on how much memory is actually being
> retained in accessible objects.

If the object won't be re-accessed but is still retained by gc, then
refcounting won't free it either.

> Once again: The trouble with GC is, it doesn’t know when to kick in:
> it just keeps on allocating memory until it runs out.

When was the last time you encountered a problem like that in practice?
It's almost never an issue.  "Runs out" means reached an allocation
threshold that's usually much smaller than the program's memory region.
And as you say, you can always manually trigger a gc if the need arises.
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Progress on the Gilectomy

2017-06-21 Thread Paul Rubin
Lawrence D’Oliveiro  writes:
> The trouble with GC is, it doesn’t know when to kick in: it just keeps
> on allocating memory until it runs out.

That's not how GC works, geez.  Typically it would run after every N
bytes of memory allocated, for N chosen to balance memory footprint
with cpu overhead.  
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Static typing [was Re: Python and the need for speed]

2017-06-21 Thread Paul Rubin
Gregory Ewing  writes:
> A JIT compiler works by observing the actual values

To be pedantic, that's called a "tracing JIT".  Other runtime code
generation is also frequently called JIT compilation even when it's
fairly stupid combining of assembly code templates, or the like.
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Looping [was Re: Python and the need for speed]

2017-06-21 Thread Paul Rubin
Chris Angelico  writes:
> while True:
> c = sys.stdin.read(1)
> if not c: break
> if c.isprintable(): text += c
> elif c == "\x08": text = text[:-1]
> # etc
> Can you write _that_ as a do-while?

I prefer to write that sort of thing with iterators:

 for c in iter(lambda: sys.stdin.read(1), ''):
 if c.isprintable(): text.append(c)
 elif c == '\x08': text.pop()
 ...
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Progress on the Gilectomy

2017-06-20 Thread Paul Rubin
Cem Karan  writes:
> I'm not too sure how much of performance impact that will have.  My
> code generates a very large number of tiny, short-lived objects at a
> fairly high rate of speed throughout its lifetime.  At least in the
> last iteration of the code, garbage collection consumed less than 1%
> of the total runtime.  Maybe this is something that needs to be done
> and profiled to see how well it works?

If the gc uses that little runtime and your app isn't suffering from the
added memory fragmentation, then it sounds like you're doing fine.

> I **still** can't figure out how they managed to do it,

How it works (i.e. what the implementation does) is quite simple and
understandable.  The amazing thing is that it doesn't leak memory
catastrophically.
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Instagram: 40% Py3 to 99% Py3 in 10 months (Posting On Python-List Prohibited)

2017-06-20 Thread Paul Rubin
Steven D'Aprano  writes:
> genuinely good reason...  (Which might include "the customer insists",
> or "yeah, I know it sucks, but politics".)

I think the current LTS versions of Ubuntu and Debian both come with
Python 2.  Not sure about Centos/RHEL.  Those seem like ok reasons to
me.

> if you want to use 1.5 go right ahead

I tested all my Python 2 code for compatibility with 1.5 for quite a
long time into the Python 2 series, but eventually got too addicted to
stuff like using iterators pervasively.  I don't think Python 2 really
broke any 1.5 code though.  At least the print statement still worked.

I'm comfortable enough with Python that it's still what I do most of my
personal stuff with, but both py2 and py3 have enough deficiencies that
I see them being relegated to the numerical computation niche, where
they really do have strong library support and a dedicated user base.

Unfortunately, everything I know of that fixes Python's deficiencies
also introduces deficiencies of its own, so at best it's a wash.
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Progress on the Gilectomy

2017-06-19 Thread Paul Rubin
Cem Karan  writes:
> Can you give examples of how it's not reliable?

Basically there's a chance of it leaking memory by mistaking a data word
for a pointer.  This is unlikely to happen by accident and usually
inconsequential if it does happen, but maybe there could be malicious
data that makes it happen

Also, it's a non-compacting gc that has to touch all the garbage as it
sweeps, not a reliability issue per se, but not great for performance
especially in large, long-running systems.

It's brilliant though.  It's one of those things that seemingly can't
possibly work, but it turns out to be quite effective.
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Progress on the Gilectomy

2017-06-19 Thread Paul Rubin
Chris Angelico  writes:
> Or let's look at it a different way. Instead of using a PyObject* in C
> code, you could write C++ code that uses a trivial wrapper class that
> holds the pointer, increments its refcount on construction, and
> decrements that refcount on destruction.

That's the C++ STL shared_ptr template.  Unfortunately it has the same
problem as Python refcounts, i.e. it has to use locks to maintain thread
safety, which slows it down significantly.

The simplest way to start experimenting with GC in Python might be to
redefine the refcount macros to do nothing, connect the allocator to the
Boehm GC, and stop all the threads when GC time comes.  I don't know if
Guile has threads at all, but I know it uses the Boehm GC and it's quite
effective.
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Instagram: 40% Py3 to 99% Py3 in 10 months (Posting On Python-List Prohibited)

2017-06-19 Thread Paul Rubin
Lawrence D’Oliveiro  writes:
> Is Python 2 the Windows XP of the programming world?

That's a good way to put it.  It's nice to hear about Instagram but so
far I don't personally know anyone who uses Python 3.  Meanwhile there's
still lots of new Py2 projects being started.
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Instagram: 40% Py3 to 99% Py3 in 10 months

2017-06-18 Thread Paul Rubin
Paul Barry  writes:
> The process they followed is discussed in their recent Keynote at PyCon
> 2017: https://youtu.be/66XoCk79kjM
> Well worth the 40 minutes it takes to watch  :-)

If it takes 40 minutes to describe how they did it, that sounds like
more hassle than most users of working py2 code probably want to deal
with.
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Progress on the Gilectomy

2017-06-18 Thread Paul Rubin
I always thought the GIL removal obstacle was the need to put locks
around every refcount adjustment, and the only real cure for that is to
use a tracing GC.  That is a good idea in many ways, but it would break
the existing C API quite seriously.  Reworking the C modules in the
stdlib would be a large but not impossible undertaking.  The many
external C modules out there would be more of an issue.
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: New to Python - Career question

2017-06-10 Thread Paul Rubin
Larry Martell  writes:
> I can tell they think I am old and they dismiss me right away.

http://oldgeekjobs.com ?
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: [OT] How to improve my programming skills?

2017-06-03 Thread Paul Rubin
Mirko  writes:
> TLDR: Sorry for OT. Long-time Linux geek and hobby programmer wants to
> improve his coding skills.  What's most important: project planing,
> algorithms and data structures, contributing to FOSS, web development,
> learning other languages or something else?

If it's specifically about coding skills, the most important thing is
reading and writing lots of code.  That said, coding skills are just one
facet of a good developer.  The most important developers in the group I
work with aren't necessarily the strongest coders.  They're the ones
with the best knowledge of the (large) program's organization, how
to use the testing frameworks (CI system), and above all Git ;-).

One coding exercise I like is to write some simple program and then
tweak the code until you think it is perfect.  That's similar to
improving your writing by editing something you've written til you get
all the phrasing just right.

> - I never sit down and plan anything with pencil and paper,
> flowcharts, UML or anything else.

That's not worthwhile unless you're doing some very complicated,
and even then it's usually just some diagrams and some written docs.
Flowcharts were a 1950s thing and UML is a corporate Java thing.

> - I never learned algorithms and data structures. I know *what*
> (linked) lists, dicts, arrays, structs, and trees are; what binary
> search or bubble-sort is, but I never really studied them, let alone
> implemented them for educative purposes.

I'd say Python saves you from having to understand how stuff like linked
lists and hash tables work, at least at first.  It will become important
but not for now.  If you were programming in C it would all be important
and fundamental immediately.

But, read this: http://antirez.com/news/112

> - When it comes to coding, I'm heavily shy and unsure.

Do you play a musical instrument?  Think of how much more confident
you became with practice, even if you never got very good at it.

> But web programming always feel like being trapped in a
> mangrove jungle,

Yeah, I feel the same way and avoid that stuff.

> - I'm very hard to motivate, when the issue or topic doesn't interest
> me much. I know some tricks to increase my motivation in such cases,
> but don't use them enough.

For doing stuff on your own, it's fine to narrow them down to stuff
that excites you and that you want to keep doing.  It does help to
cultivate persistence for things like debugging.
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: noCaptcha cracking handler

2017-06-03 Thread Paul Rubin
skyteacherus...@gmail.com writes:
> Please check out my latest repository
> https://github.com/gubrul/noCaptcha

Thanks.  Captchas have gotten to be a real PITA lately, even as someone
who doesn't want to run automated clients on the infected sites.  I
remember getting sick of trying to identify pictures of buildings with
store fronts, and thinking maybe there should be a bot to do them... oh
wait...
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Python package to accept payments in Internet

2017-05-09 Thread Paul Rubin
Chris Warrick  writes:
> there might be other users that will avoid your package for licensing
> reasons.

And there also might be other users who embrace the package for
those same reasons.
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Looping [was Re: Python and the need for speed]

2017-04-17 Thread Paul Rubin
Ben Bacarisse  writes:
> ?  I get "AttributeError: 'itertools.dropwhile' object has no attribute
> 'next'" from your example.

Hmm, .next() worked ok for me in Python 2.7.5.  Not sure what happened.
Maybe something went wrong with my paste.  Oh well.

> Coming from the lazy language Haskell, I find your example natural...

Yep ;)
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Looping [was Re: Python and the need for speed]

2017-04-17 Thread Paul Rubin
Ben Bacarisse  writes:
>   c = sys.stdin.read(1)
>   while c == ' ':
>   c = sys.stdin.read(1)

c = itertools.dropwhile(
 lambda c: c==' ',
 iter(lambda: sys.stdin.read(1),None)
 ).next()
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: GPUs with various tools

2017-04-17 Thread Paul Rubin
Jeffrey Layton  writes:
> I'm working with Continuum's Accelerate and I'm slowly learning PyCUDA
> but I'm looking for something that's already coded.

TensorFlow has good Python bindings.  That's probably the best way to
get started.
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Static typing [was Re: Python and the need for speed]

2017-04-17 Thread Paul Rubin
Steve D'Aprano  writes:
> On the other hand, there's Cython. Cython claims *not* to be a JIT compiler,

One of the uses of "JIT compiler" these days is what's sometimes called
a tracing JIT, like PyPy or LuaJIT or the Javascript flavor-of-the-week.
That means it interprets the program while collecting execution traces
to figure out the actual types the program uses at runtime, then
generates machine code for those specific code paths, with some guards
in case an unexpectedly typed value shows up.
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Bigotry and hate speech on the python mailing list

2017-04-17 Thread Paul Rubin
Rurpy  writes:
> A couple weeks ago a frequent poster here (Steve D'Aprano
> ) called another participant an "ugly
> american"

Oh stop trolling.  Ugly American was a 1950s thing, now just amusing.
And context matters.  Steven, please try to be a little more attentive
to people's delicate feefees in the future.  Rurpy, should that take
care of it?
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Static typing [was Re: Python and the need for speed]

2017-04-16 Thread Paul Rubin
Steve D'Aprano  writes:
> Since it is *optional*, it is only a hint, not a fact. You can tell the
> compiler that you believe that n will be an int, but it's not guaranteed.

The runtime could check at the entry to the function that n is an int,
and then it wouldn't have to keep re-checking on the inside of a loop.
That's what a JIT compiler does in the absence of annotations, more or
less; but the annotations make life easier for ahead-of-time compilers.
Again this type of thing has been standard in Lisp since the 1960's.
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Python and the need for speed

2017-04-14 Thread Paul Rubin
bartc  writes:
> I do know that if I want to port some program (be it in Python or
> C++), or simply try and understand it, if I see it's full of class
> definitions or whatever, then I won't bother.

There was a time in the evolution of OOP when inheritance was thought of
as a cool and enabling thing for code re-use, and lots of architecture
astronautics involved designing deeply nested and complex inheritance
hierarchies in programs.  These days I think there's more accepted that
inheritance is confusing and obscures the control flow in programs, that
it was often mis-used, and that while there are still legitimate uses
cases for it, it should be used sparingly.  That helps quite a lot.

C++ templates let you write generics that are often cleaner than using
subclasses, though the design of templates can lead to awful code and
notoriously bloated and useless error messages.  The long-awaited
Concepts extension should help some with that.  Again though, like
anything else, templates work best when used tastefully rather than
willy-nilly.
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Merging multiple sorted sequences.

2017-04-12 Thread Paul Rubin
Erik  writes:
> I need to be able to lazily merge a variable number of
> already-sorted(*) variable-length sequences 

If the number of sequences is large, the traditional way is with the
heapq module.
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Python and the need for speed

2017-04-12 Thread Paul Rubin
Michael Torrie  writes:
> Equivalent to the JNI and Lisp FFI is the CFFI [1].  The JNI, the FFI,
> and the CFFI, are all for calling native code from within their
> respective languages, if I understand correctly.  They don't define a
> standard way for native code to call into these languages.

They often do, and at least the JNI does.  Some general issues are
discussed here: http://autocad.xarch.at/lisp/ffis.html
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Python and the need for speed

2017-04-12 Thread Paul Rubin
Marko Rauhamaa  writes:
> Traditionally, disk access in Linux has been considered nonblocking.
> There is AIO, but that hasn't been used much. 

AIO is asynchronous but it's for dealing with already-opened files.
There doesn't seem to be a way to asynchronously OPEN a file.

> I believe the lack of asynchronous disk I/O is related to the grand
> Solaris idea which Linux adopted: all memory is on a disk and RAM is
> merely a cache.

And I thought Solaris's heritage involved NFS, with its notorious
infinite hard hangs if something happened on the server end.

> As swapping is no longer considered normal on modern computers, the
> memory-disk duality doesn't seem all that practical anymore. Rather,
> you'd like to treat the disk analogously to network access and keep RAM
> access separate.

Yep.  But opening disk files that way seems to require threads or extra
processes.
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Python and the need for speed

2017-04-12 Thread Paul Rubin
Brecht Machiels  writes:
> However, rinohtype is located in a very different niche and it would
> greatly benefit a lot from a speed boost. Rendering the Sphinx
> documentation (311 pages) takes almost 10 minutes on my i7

Yikes...

> As for moving to a compiled language (Nim or Rust would be interesting
> choices, IMO), it doesn't make much sense.

Have you tried PyPy?  Thought about Lisp?  Lisp is easily extensible and
goes well with the DSL idea.  Or if most of the time is used in some hot
loop, maybe a small C extension is the simplest fix.
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Python and the need for speed

2017-04-11 Thread Paul Rubin
Chris Angelico  writes:
> It's also entirely possible to have a single "application thread" and
> then a "file opening thread" that does nothing but open files;

Thanks.  I think GHC and Erlang BEAM both do that.  I hoped there was a
better way.  Note you potentially need multiple file opening threads.

> your app is still single threaded (thus you don't have concurrency
> concerns), but file opening becomes truly asynchronous.

Of course you still have concurrency concerns, plus you have to be very
very careful that nothing ever blocks.  I've generally preferred in
Python to use threads communicating by message passing through queues.
This is pretty safe if you remember to not having any mutable objects
shared between threads.  Of course I like the way Erlang does it even
more, with isolated lightweight processes.

> Oh, and Python also has to cope with a bit more complexity because
> it's possible to start multiple application threads in Python. It's
> not in JS.

Yeah I'm not sure of the JS situation.  C++11 has futures and promises
and I've been wanting to try to use them, but it's still on my
infinitely deep stack of stuff to do.  This looks amazing though:

  http://www.seastar-project.org/
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Python and the need for speed

2017-04-11 Thread Paul Rubin
Chris Angelico  writes:
> Python, meanwhile, has made it easy to write blocking
> I/O in a single-threaded program, and then adds async'ness to it.

I'd be interested to know how to open a disk file asynchronously in a
single-threaded Python program under Linux.  As far as I can tell there
is no way to do it.
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Python and the need for speed

2017-04-11 Thread Paul Rubin
Mikhail V  writes:
> Just my curiosity, I've always been intersted in such question: are devs
> still writing extensions in C, I mean type in C code? 

Yes.

> Aren't they using some translator or IDE which at lest hides the
> brackets and semicolons?

I don't know of anyone who does that.  I don't think anyone minds C's
brackets and semicolons, and lots of other languages like Java and Perl
use them too, for programmer familiarity.  The actual difficulty with C
is that it gives rise to a high incidence of hard-to-diagnose bugs.
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Python and the need for speed

2017-04-11 Thread Paul Rubin
Grant Edwards  writes:
> If there are now other Python implementations (e.g. MicroPython) with
> C APIs that differ from CPython, then it seems like it is no longer
> redundant to say "the CPython API".

Perhaps there should be an attempt to define a unified API like the Java
JNI, Lisp FFI's, etc.
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Python and the need for speed

2017-04-11 Thread Paul Rubin
Chris Angelico  writes:
> Do you mean this?
> https://docs.python.org/3/c-api/intro.html

Correct, smart guy ;-)
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Python and the need for speed

2017-04-11 Thread Paul Rubin
Grant Edwards  writes:
> I didn't know there was such a thing as "The Python C API".

It's described in this document:

https://infohost.nmt.edu/tcc/help/lang/python/2_6_3/c-api.pdf

You can tell that the document is about the Python/C API (ok, with a
slash) because it says so at the top of the title page.
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Python and the need for speed

2017-04-11 Thread Paul Rubin
Steven D'Aprano  writes:
> If we're going to talk about speeding up Python, we ought to talk about 
> *serious* approaches to the problem, not the musing of random, ignorant 
> bloggers and other naive commentators. So let's look at what the experts 
> have done:  [list snipped]

You might look at MicroPython too (micropython.org).  A fairly complete
Python 3 implementation with some ahead-of-time compiling, no fancy
JIT.  Completely breaks the Python C API though.

Or you could look at the past 50 years(!) of Lisp and Scheme compilers
some of which produce very good code, ask what Python features can't be
straightforwardly transliterated into Lisp to use those compilers, then
ask whether those features are really important to the average Python
user.  I don't even think eval is an obstacle.  Lots of Lisp systems
implement eval by handing expression off to the compiler and then
running the compiled code, maybe with a bit of caching like Python and
compiled regexps.

I remember thinking PyPy made a mistake in trying to preserve all of
Python's dynamism, and Python 3 made a mistake in trying to preserve so
much compatibility with Python 2 while still breaking minor things.  I
thought PyPy should have been "TurboPython" that broke lots more Python
2 stuff than Python 3 did, but was as you say maybe 20x faster.  Then
Python 3 could have been skipped.  For a while I thought something like
that could become Python 4, but the readout of Python 3 seems to be that
its slow uptake came from those minor breaks, so Python 4 won't have
even the slightest incompatibility with Python 3 code.

I still do my everyday stuff in Python and I'd like to get more
conversant with stuff like numpy, but it feels like an old-fashioned
language these days.
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Elastic Search

2017-04-08 Thread Paul Rubin
Steve D'Aprano  writes:
> What's elastic search?
> And what does this have to do with Python?

https://www.elastic.co/  (formerly elasticsearch.org).

It's a Lucene-based distributed search engine, something like Solr if
you're used to that.  It has Python client libraries.  That's the
closest Python connection that I know of.  I've never used it but I've
used Solr.  These things are good tools to have available if you need to
index a lot of text.
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: SimpleHTTPServer and CgiHTTPServer in practice

2017-03-16 Thread Paul Rubin
Lutz Horn  writes:
> We don't know *why* those people told you not to use these modules. We
> also don't know your use case. So it is very hard to advise you.

The use case is to have a very easily set up way to serve basic pages
and files, without a lot of configuration files and other
infrastructure.  The main reason not to use it would be security holes
if there are some.  I haven't examined the code carefully but didn't see
obvious issues at a quick glanace.  I frankly worry more about large C
programs than Python programs.
-- 
https://mail.python.org/mailman/listinfo/python-list


SimpleHTTPServer and CgiHTTPServer in practice

2017-03-15 Thread Paul Rubin
Sometimes I have a short term requirement to serve some data by http, so
I've been using those modules rather than setting up a full featured web
server.  Some Python users have told me that isn't a good idea, but
without any specifics.

Are there any known problems with them such as security bugs, or in the
py3 equivalents (server module)?  I haven't had issues so far because
it's just been ad hoc stuff for known clients, but they are much simpler
to set up than something like Apache, so I don't see why not to make
more use of them for low-traffic pages.  Any thoughts/stories/cautions
would be appreciated.

Thanks

-- 
https://mail.python.org/mailman/listinfo/python-list


Re: How to iterate through the columns in a row using sqlite3.Row

2017-03-12 Thread Paul Rubin
Chris Green  writes:
> self.conn = sqlite3.connect(dbname)
> self.conn.row_factory = sqlite3.Row
> self.cursor = self.conn.cursor()
> self.table = table
> ...
> ...
> sql = "SELECT * FROM " + self.table + " WHERE firstName||lastName = ?"
> self.cursor.execute(sql, (name,))
> row = self.cursor.fetchone()

untested:

with sqlite3.connect(dbname) as conn:
   sql = "SELECT * FROM " + self.table + " WHERE firstName||lastName = ?"
   cursor = conn.execute(sql, (name,))
   row = cursor.fetchone()
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: keyrings.cryptfile released on github

2017-03-10 Thread Paul Rubin
Hans-Peter Jansen  writes:
> [1] http://web.cs.ucdavis.edu/~rogaway/ocb/license.htm

Oh that's interesting, he's expanded the free licenses.  Still though,
while OCB is very clever and it was important as the first satisfactory
AEAD mode, I don't think it's that important these days.  GCM is
standardized, does similar things, and while it's potentially slower,
some CPUs even have hardware support for it now.  If you library doesn't
support GCM (I haven't checked yet) then it probably should.

-- 
https://mail.python.org/mailman/listinfo/python-list


Re: keyrings.cryptfile released on github

2017-03-09 Thread Paul Rubin
Hans-Peter Jansen  writes:
> I've released a keyring companion package today:
>   https://github.com/frispete/keyrings.cryptfile

Interesting, I'll take a look at it.  I wrote something like it many
years ago but never did much with it.  Thanks for posting!
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Who still supports recent Python on shared hosting

2017-03-05 Thread Paul Rubin
John Nagle  writes:
> I'm looking for shared hosting that supports at least Python 3.4.

Open a ticket on buyshared.net and ask if they can install it for you.
They're good about stuff like that.

If it's for a cgi, you might alternatively be able to run it from your
own directory (you get ssh access which gives some flexibility).
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Create ordering of points

2017-02-15 Thread Paul Rubin
spiess.benja...@googlemail.com writes:
> Does somebody know a good technique for this problem? or can even give
> a hint to a existing python procedure?

You're asking about nearest neighbor search, which is the topic of a
huge literature:

https://en.wikipedia.org/wiki/Nearest_neighbor_search
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: WANT: bad code in python (for refactoring example)

2017-02-15 Thread Paul Rubin
Antoon Pardon  writes:
> On reason to use this is for some easy "logging"

I think it's better to use the actual logging module.  I generally start
a new program with print statements but convert them to logging after
there's enough code to want to be more organized about it.
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: PEP 393 vs UTF-8 Everywhere

2017-01-21 Thread Paul Rubin
Chris Angelico  writes:
> You can't do a look-ahead with a vanilla string iterator. That's
> necessary for a lot of parsers.

For JSON?  For other parsers you usually have a tokenizer that reads
characters with maybe 1 char of lookahead.

> Yes, which gives a two-level indexing (first find the strand, then the
> character), and that's going to play pretty badly with CPU caches.

If you're jumping around at random all over the string, you probably
really want a bytearray rather than a unicode string.  If you're
scanning sequentually you won't have to look at the outer table very
often.
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: PEP 393 vs UTF-8 Everywhere

2017-01-20 Thread Paul Rubin
Chris Angelico  writes:
> decoding JSON... the scanner, which steps through the string and
> does the actual parsing. ...
> The only way for it to be fast enough would be to have some sort of
> retainable string iterator, which means exposing an opaque "position
> marker" that serves no purpose other than parsing.

Python already has that type of iterator:
   x = "foo"
   for c in x: 

> It'd mean some sort of magic "thing" that probably has a reference to
> the original string

It's a regular old string iterator unless I'm missing something.  Of
course a json parser should use it, though who uses the non-C json
parser anyway these days?

[Chris Kaynor writes:]
> rfind/rsplit/rindex/rstrip and the other related reverse
> functions would require walking the string from start to end, rather
> than short-circuiting by reading from right to left. 

UTF-8 can be read from right to left because you can recognize when a
codepoint begins by looking at the top 2 bits of each byte as you scan
backwards.  Any combination except for 11 is a leading byte, and 11 is
always a continuation byte.  This "prefix property" of UTF8 is a design
feature and not a trick someone noticed after the fact.

Also if you really want O(1) random access, you could put an auxiliary
table into long strings, giving the byte offset of every 256th codepoint
or something like that.  Then you'd go to the nearest table entry and
scan from there.  This would usually be in-cache scanning so quite fast.
Or use the related representation of "ropes" which are also very easy to
concatenate if they can be nested.  Erlang does something like that
with what it calls "binaries".
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Enum with only a single member

2017-01-10 Thread Paul Rubin
Steven D'Aprano  writes:
> Is it silly to create an enumeration with only a single member?

No.  

> That is, a singleton enum?

Why stop there?  You can make empty ones too.  (Zerotons?)

> The reason I ask is that I have two functions that take an enum
> argument. 

Sounds like a good reason.

> def spam(arg):
> if isinstance(arg, MarxBros): 
> def ham(arg):
> if isinstance(arg, MarxBros) or arg is Unique.FOO:

> Good, bad or indifferent?

U, that's ugly, but the alternatives are worse unless there's more
branches, and maybe even then.
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Temporary variables in list comprehensions

2017-01-09 Thread Paul Rubin
Ben Bacarisse  writes:
>   [(lambda tmp: (tmp, tmp+1))(expensive_calculation(x)) for x in data]

Nice.  The Haskell "let" expression is implemented as syntax sugar for
that, I believe.
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Temporary variables in list comprehensions

2017-01-09 Thread Paul Rubin
Tim Chase  writes:
>> result = [(tmp, tmp+1) for tmp in map(expensive_calculation, data)]
>
> As charmingly expressive as map() is, the wildly different behavior in
> py3 (it's a generator that evaluates lazily) vs py2 (it consumes the
> entire iterable in one go) leads me to avoid it in general,

Well, there's itertools.imap which maps lazily in py2.

-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Temporary variables in list comprehensions

2017-01-09 Thread Paul Rubin
Serhiy Storchaka  writes:
> gen = (expensive_calculation(x) for x in data)
> result = [(tmp, tmp + 1) for tmp in gen]

result = [(tmp, tmp+1) for tmp in map(expensive_calculation, data)]
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Temporary variables in list comprehensions

2017-01-09 Thread Paul Rubin
Steven D'Aprano  writes:
> [(expensive_calculation(x), expensive_calculation(x) + 1) for x in data]

   def memoize(f):
cache = {}
def m(x):
if x in cache:
return cache[x]
a = f(x)
cache[x] = a
return a
return m

   ec = memoize(expensive_calculation)
   ...  [(ec(x), ec(x) + 1) for x in data]

Or can write:

@memoize
def expensive_calculation(x): 

Note the Haskell version of your listcomp would be:

  [(e, e+1) | x <- data_, let e = expensive_calculation x]

Maybe Python could get some version of that.  I've wanted it more than
once.  (I used "data_" because data is a Haskell keyword).
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Search a sequence for its minimum and stop as soon as the lowest possible value is found

2017-01-08 Thread Paul Rubin
Jussi Piitulainen  writes:
> It could still be added as an option, to both takewhile and iter(_, _).

That's too messy, it really should be pervasive in iterators.
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Search a sequence for its minimum and stop as soon as the lowest possible value is found

2017-01-08 Thread Paul Rubin
Peter Otten <__pete...@web.de> writes:
> return min(take_until(), key=firstitem)[1]

Actually, key=abs should work.  I realized that after posting.
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Search a sequence for its minimum and stop as soon as the lowest possible value is found

2017-01-08 Thread Paul Rubin
Paul Rubin  writes:
> seems to work, but is ugly.  Maybe there's something better.

def minabs2(xs):
def z():
for x in xs:
yield abs(x), x
if x==0: break
return min(z())[1]

is the same thing but a little bit nicer.
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Search a sequence for its minimum and stop as soon as the lowest possible value is found

2017-01-08 Thread Paul Rubin
Paul Rubin  writes:
> Doesn't look that way to me:
> >>> minabs([5,3,1,2,4])
> 1

There's a different problem though:

>>> minabs([1,2,3,0])
1

I think Python's version of iterators is actually buggy and at least the
first element of the rest of the sequence should be preserved.  There
are ways to fake it but they're too messy for something like this.  It
should be the default and might have been a good change for Python 3.

def minabs2(xs):
def z():
for x in xs:
yield x
if x==0: break
return min((abs(x),x) for x in z())[1]


seems to work, but is ugly.  Maybe there's something better.
-- 
https://mail.python.org/mailman/listinfo/python-list


  1   2   3   4   5   6   7   8   9   10   >