Re: [Python-Dev] PEP 492: async/await in Python; version 4

2015-05-06 Thread Wolfgang Langner
Hi Yury,


On 05.05.2015 20:25, Yury Selivanov wrote:


 We forget to address the major problems here. How can someone in a
 sync script use this async stuff easy. How can async and sync stuff
 cooperate and we don't need to rewrite the world for async stuff.
 How can a normal user access the power of async stuff without rewriting
 all his code. So he can use a simple asyc request library in his code.
 How can a normal user learn and use all this in an easy way.
 
 asyncio and twisted answered these questions ;) The answer is
 that you have to write async implementations.
 
 gevent has a different answer, but greenlents/stackless is
 something that will never be merged in CPython and other
 implementations.
 

I think monkeypatching and the gevent way is wrong. And explicit is better
than implicit.

I have to clear this. I meant how can we make this async stuff more accessible
to the average sync user. Sometime even without writing or knowing how
to write coroutines or other async stuff.

Let me explain it with a deeper example (Some of it is related to Python 2 and
twisted):

I had the same problem for a server application using twisted. Provide
easy interfaces to my users most not aware of async stuff.

My solution was to write my own decorator similar to twisted's
@inlineCallbacks. On top of it I added one more level to the decorator
and distinguish if it was called from the main thread (also running the
mainloop) and other threads. This object usable also as decorator
is called task and has some utility methods. And returns a deferred.
(in asyncio this would be a Future)

Resulting in code like:

@task
def myAsyncFunction(webaddress):
  result = yield fetch(webaddress)
  # only to show sleep example
  yield task.sleep(0.1)
  task.Return(result)

Usable in a sync context (extra script thread):

def do():
  content = myAsyncFunction(http://xyz;)

or async context:

@task
def ado():
  content = yield myAsyncFunction(http://xyz;)


The task decorator has functionality to check if something is
called from the main thread (- also a task and async)
or it is called from another thread (- sync or script).

So this async interface is usable from both worlds. If someone
operates async he/she must only know the task decorator and when to yield.
If he/she uses it in sync mode nothing special has to be done.

To allow all this the server starts the async main loop in the main thread
and executes the script in an extra script thread. The user has every time his
own thread, also for rpc stuff. The only way to switch into the main loop
is to decorate a function as @task, every task is a coroutine and executed
in the main thread (also thread of main loop).

Benefit of all this:

- Easy to write a async task it is marked as one and special stuff belongs
  to the task object. (task.Return is needed because we are in Python 2)
- The normal user executes his stuff in his own thread and he/she
  can program in sync mode. No problem it is an extra thread and the main loop
  does not block.
- A network client or other stuff has to be written only once, most time this
  can be a @task in the async world. But this should not care the end user.
  We don't have to implement all twice once for async and once for the sync
  world. - Less overhead

This is what I mean if I say we must address the bridging problem between the
worlds.
It think it is the wrong way to divide it in async and sync stuff and
duplicate all networking libraries in the sync and async ones.

For me the answer is to write one async netowrk library and use it in both,
a sync script and in an async main loop. With an easy interface and not
forcing the user to know this is an async library I have to do something 
special.

And in future go one step further and use all this combined with PyParallel
to solve the multiprocessing problem in Python.
(Script in extra thread, mainloop in main thread, executed and managed via
PyParallel avoiding the gil)
But this is only a vision/dream of mine.



 And for all this we still can't tell them oh the async stuff solves
 the multiprocessing problem of Python learn it and switch to version
 3.5. It does not and it is only most useful for networking stuff
 nothing more.
 
 networking stuff, and in particular, web, is a huge
 part of current Python usage.  Please don't underestimate
 that.

I do not. But for most they want only use it as a client
and the main concern for most is I want to get this web page
and not I will implement a web client have to do this async and get it.


Regards,

Wolfgang

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] async/await in Python; v2

2015-04-23 Thread Wolfgang Langner
On Thu, Apr 23, 2015 at 5:32 PM, Yury Selivanov yselivanov...@gmail.com
wrote:

 Hi Wolfgang,

 On 2015-04-23 8:27 AM, Wolfgang Langner wrote:

 On Thu, Apr 23, 2015 at 12:35 PM, Paul Sokolovsky pmis...@gmail.com
 wrote:

  Hello,

 On Thu, 23 Apr 2015 12:18:51 +0300
 Andrew Svetlov andrew.svet...@gmail.com wrote:

 []

  3.
 async with and async for
 Bead idea, we clutter the language even more and it is one more
 thing every newbie could do wrong.
 for x in y:
result = await f()
 is enough, every 'async' framework lived without it over years.

 async for i in iterable:
  pass

 is not equal for

 for fut in iterable:
  i = yield from fut

 But people who used Twisted all their life don't know that! They just
 know that async for is not needed and bad.


  I don't think it is bad nor not needed, but the syntax is not beautiful
 and
 for the 90% not doing async stuff irritating and one more thing to learn
 and do right/wrong.

 There is no way to do things wrong in PEP 492.  An object
 either has __aiter__ or it will be rejected by async for.
 An object either has __aenter__ or it will be rejected by
 async with.

   transaction = yield from connection.transaction()
   try:
 ...
   except:
 yield from transaction.rollback()
   else:
 yield from transaction.commit()

 is certainly more irritating than

   async with connection.transcation():
 ...


 I had also a need for async loop. But there are other solutions like
 channels,
 not needing a new syntax.

 Also possible a function returning futures and yield in the loop with a
 sentinel.

 All this goes the road down to a producer consumer pattern. Nothing more.



  I know I'm a bad guy to make such comments, too bad there's a bit of
 truth in them, or everyone would just call me an a%$ole right away.


 Generally, I already provided feedback (on asyncio list) that asyncio
 is based not on native Python concepts like a coroutine, but on
 foreign concurrency concepts like callback or Future, and a coroutine
 is fitted as a second-class citizen on top of that. I understand why
 that was done - to not leave out all those twisteds from a shiny new
 world of asyncio, but sometimes one may wonder if having a clear cut
 would've helped (compat could then have been added as a clearly separate
 subpackage, implemented in terms of coroutines). Now people coming from
 non-coroutine frameworks who were promised compatibility see bad
 things in asyncio (and related areas), and people lured by a promise of
 native Python framework see bad things too.


  This has nothing to do with people using twisted or other async
 frameworks
 like tornado.
 I think a coroutine should be first class. But all this should be done in
 a
 way a beginner
 can handle and not design this stuff for experts only.


 I think that most of async frameworks out there are for
 experts only.  Precisely because of 'yield from', 'yield',
 inlineCallbacks, '@coroutine', channels and other stuff.

 PEP 492 will make it all easier. And Twisted can use
 its features too.


Yes and it is good to make it easier. But not complicate it for others.
Beginners will be confronted with all this new syntax an my feel lost.
Oh I have to different for loops, one with async. Same for with statement.



  If we do this we
 scare away new people.


 It doesn't scare away anyone.  async/await were the most
 awaited features in dart and javascript.  One of the most
 popular features in c#.


I like it in C#.
I like await for Python but I don't like async there and how to specify it.
I still think a decorator is enough and no special for and with syntax.

async in JavaScript is for execution a whole script asynchronously used in
the script tag.
dart is for the google universe with less usage outside.


-- 
bye by Wolfgang
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] async/await in Python; v2

2015-04-23 Thread Wolfgang Langner
On Thu, Apr 23, 2015 at 6:22 PM, Yury Selivanov yselivanov...@gmail.com
wrote:

 Wolfgang,


 On 2015-04-23 12:12 PM, Wolfgang Langner wrote:

 On Thu, Apr 23, 2015 at 5:32 PM, Yury Selivanov yselivanov...@gmail.com
 wrote:

  Hi Wolfgang,

 On 2015-04-23 8:27 AM, Wolfgang Langner wrote:

  On Thu, Apr 23, 2015 at 12:35 PM, Paul Sokolovsky pmis...@gmail.com
 wrote:

   Hello,

 On Thu, 23 Apr 2015 12:18:51 +0300
 Andrew Svetlov andrew.svet...@gmail.com wrote:

 []

   3.

 async with and async for
 Bead idea, we clutter the language even more and it is one more
 thing every newbie could do wrong.
 for x in y:
 result = await f()
 is enough, every 'async' framework lived without it over years.

  async for i in iterable:
   pass

 is not equal for

 for fut in iterable:
   i = yield from fut

  But people who used Twisted all their life don't know that! They just
 know that async for is not needed and bad.


   I don't think it is bad nor not needed, but the syntax is not
 beautiful

 and
 for the 90% not doing async stuff irritating and one more thing to learn
 and do right/wrong.

  There is no way to do things wrong in PEP 492.  An object
 either has __aiter__ or it will be rejected by async for.
 An object either has __aenter__ or it will be rejected by
 async with.

  Don't mean it can be done wrong on execution or syntax level.
 I mean for a beginner it is not as easy an more and some will try
 async in some places, yes they will get an error. But there is a
 new possibility to get such errors if async is there for with and for
 statements.
 And the next beginner will then implement __aiter__ instead of __iter__
 because
 he/she don't get it.


 Sorry, Wolfgang, but I don't get your argument.  Beginners
 shouldn't just randomly try to use statements.  There is a
 documentation for that.  Plus we can make exception messages
 better.


Had to coach a lot of new users to Python and some to async stuff in
twisted.
And what beginners not should do don't care them. ;-)
They will do strange stuff and go other way's than you expected. I only
like to make
it as easy as possible for them. Nothing more.
Less keywords, less ways to do something, best only one way to do it.



Don't get me wrong, I like the PEP it is well written and covers a lot of
areas about async programming.
I know Python must improve in this area and has a lot of potential.
But don't hesitate, give the people time to try it and mature it. If all
this should be in 3.5 it is to early.

Also we can avoid the async keyword completely and do the same as for
generators.
If there is an await, it is a coroutine.
IDE's can detect generators and must only be expanded to detect coroutines.

Function
with yield - return a generator
with await - return a coroutine

Move async for and async with to another PEP and handle it later or with a
different syntax using the new await keyword.

Only one new keyword, good improvement for async programming.

Sorry still don't like the word async in a language and sprinkle it every
where.
And still have the feeling if we provide async for loop we next must
provide async like generator expresions or list comprehensions or ...
never ends ;-)

The only downside with await converting a function to a coroutine is, it is
not explicit marked.
But if we care about this, whats about generators and yield ?


-- 
bye by Wolfgang
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] async/await in Python; v2

2015-04-23 Thread Wolfgang Langner
On Thu, Apr 23, 2015 at 5:32 PM, Yury Selivanov yselivanov...@gmail.com
wrote:

 Hi Wolfgang,

 On 2015-04-23 8:27 AM, Wolfgang Langner wrote:

 On Thu, Apr 23, 2015 at 12:35 PM, Paul Sokolovsky pmis...@gmail.com
 wrote:

  Hello,

 On Thu, 23 Apr 2015 12:18:51 +0300
 Andrew Svetlov andrew.svet...@gmail.com wrote:

 []

  3.
 async with and async for
 Bead idea, we clutter the language even more and it is one more
 thing every newbie could do wrong.
 for x in y:
result = await f()
 is enough, every 'async' framework lived without it over years.

 async for i in iterable:
  pass

 is not equal for

 for fut in iterable:
  i = yield from fut

 But people who used Twisted all their life don't know that! They just
 know that async for is not needed and bad.


  I don't think it is bad nor not needed, but the syntax is not beautiful
 and
 for the 90% not doing async stuff irritating and one more thing to learn
 and do right/wrong.

 There is no way to do things wrong in PEP 492.  An object
 either has __aiter__ or it will be rejected by async for.
 An object either has __aenter__ or it will be rejected by
 async with.


Don't mean it can be done wrong on execution or syntax level.
I mean for a beginner it is not as easy an more and some will try
async in some places, yes they will get an error. But there is a
new possibility to get such errors if async is there for with and for
statements.
And the next beginner will then implement __aiter__ instead of __iter__
because
he/she don't get it.

On the other side I like await and __await__ implementation.
Symmetric good easy to explain, same with int and __int__ and all
others.



   transaction = yield from connection.transaction()
   try:
 ...
   except:
 yield from transaction.rollback()
   else:
 yield from transaction.commit()

 is certainly more irritating than

   async with connection.transcation():
 ...


Sorry till now I use async stuff and database access and do it in an extra
thread in sync mode.
No performance problems and can use all good maintained database libraries.
Also twisteds RDBMS (adbapi) is enough here. First I thought it is not
enough or to slow but this was not the case.
https://twistedmatrix.com/documents/current/core/howto/rdbms.html

Here I am on line with Mike Bayer:
http://techspot.zzzeek.org/2015/02/15/asynchronous-python-and-databases/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Type hints -- a mediocre programmer's reaction

2015-04-23 Thread Wolfgang Langner
Hi,

having a lot experience with Python beginners and people programming
Java/Python I have also an opinion about this. ;-)

First reaction was, oh good. Then I read every thread and comment about it,
looked at a lot internal code give all some time
and the result is:

I found a lot of code written like Java with assert and isinstance for type
checks. Not very Pythonic but someone from Java thinks this is good.
Because Python has no type checks. A lot of people think types help in the
IDE and for tools to do automatic checks, find some errors earlier.

My conclusion:

1. typing.py is good to make a standard how to specify type hints. Can also
be used by tools and IDE's

2. Using it in the language as part of the function signature, my first
thought was oh good, then I changed my mind
   to: oh it can be very ugly and unreadable, it is the wrong place.
   Now I am against it, best is, if I have to specify type signatures, do
it in one place, keep them up to date.
   Most of the time this is the documentation. Why not use the docstring
with a standard type specifier for this.
   Suggested here:
http://pydev.blogspot.de/2015/04/type-hinting-on-python.html

3. Stub files are good if someone needs this feature, it is complete
separate and if someone don't want to write them
it is no problem. But it is good to have a way to name them and a place
to find them and know this.

IDE's parse the docstring already, it is optional to have one and every
Editor has folding support to disable it if it is to much
of information. Checkers like mypy can use the type info from there to do
checks. If there is a standard way to specify types in docstrings.
It is also possible to specify a type hint for a variable, this is good for
documentations and can also be used for type checks.

For stub files the same can be used. So no different syntax. Also If
someone wants it and a module has no documentation it can be added from
there.

For nearly every function I have written, there is a docstring and most of
the time also a type specified.
But if I must provide all this in a second place it is not the right way to
go. Over time normally one place misses some changes and is wrong.

I am convinced something like:

def myfunction(a, b):
  
  My special function.

  :param a: ...
  :type a: int
  :param b: second value
  :type b: int
  :rtype: None
  

or shorter form also possible in Sphinx if only one type for parameter is
specified:

def myfunction(a, b):
  
  My special function.

  :param int a: ...
  :param int b: second value
  :rtype: None
  

Is easier to maintain good to read, changes not the world how to write
Python functions and is useful for IDE's and checkers.
And the best, most editors and IDE's support folding of docstrings and if
someone don't want to see them it can be done now.
The docstring is a good place for type specification, because as stated
Python never will do type checks on execution. So no need
to make it part of the language syntax.

All this can also be applied over time to the standard library. If it makes
sense so specify a type it is good to have it in the documentation.
And sphinx is the standard documentation tool for Python now.

Also ask why no one used type specifier, they are possible since Python 3.0
?
Because it is the wrong way for Python.


Regards,

Wolfgang
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] async/await in Python; v2

2015-04-23 Thread Wolfgang Langner
Hi,

most of the time I am a silent reader but in this discussion I must step in.
I use twisted and async stuff a lot over years followed development of
asyncio closely.

First it is good to do differentiate async coroutines from generators. So
everyone can see it and have this in mind
and don't mix up booth. It is also easier to explain for new users.
Sometimes generators blows their mind and it takes
a while to get used to them. Async stuff is even harder.

1. I am fine with using something special instead of yield or yield
from for this. C# await is ok.

Everything else suggested complicates the language and makes it harder to
read.

2.
async def f(): is harder to read and something special also it breaks the
symmetry in front (def indent).
Also every existing tooling must be changed to support it. Same for def
async, def f() async:
I thing a decorator is enough here
@coroutine
def f():
is the best solution to mark something as a coroutine.


3.
async with and async for
Bead idea, we clutter the language even more and it is one more thing every
newbie could do wrong.
for x in y:
  result = await f()
is enough, every 'async' framework lived without it over years.
Same for with statement.

The main use case suggested was for database stuff and this is also where
most are best with
defer something to a thread and keep it none async.


All together it is very late in the development cycle for 3.5 to
incorporate such a big change.
Best is to give all this some more time and defer it to 3.6 and some alpha
releases to experiment with.

Regards,

Wolfgang



On Tue, Apr 21, 2015 at 7:26 PM, Yury Selivanov yselivanov...@gmail.com
wrote:

 Hi python-dev,

 I'm moving the discussion from python-ideas to here.

 The updated version of the PEP should be available shortly
 at https://www.python.org/dev/peps/pep-0492
 and is also pasted in this email.

 Updates:

 1. CO_ASYNC flag was renamed to CO_COROUTINE;

 2. sys.set_async_wrapper() was renamed to
sys.set_coroutine_wrapper();

 3. New function: sys.get_coroutine_wrapper();

 4. types.async_def() renamed to types.coroutine();

 5. New section highlighting differences from
PEP 3152.

 6. New AST node - AsyncFunctionDef; the proposal
now is 100% backwards compatible;

 7. A new section clarifying that coroutine-generators
are not part of the current proposal;

 8. Various small edits/typos fixes.


 There's is a bug tracker issue to track code review
 of the reference implementation (Victor Stinner is
 doing the review): http://bugs.python.org/issue24017
 While the PEP isn't accepted, we want to make sure
 that the reference implementation is ready when such
 a decision will be made.


 Let's discuss some open questions:

 1. Victor raised a question if we should locate
coroutine() function from 'types' module to
'functools'.

My opinion is that 'types' module is a better
place for 'corotine()', since it adjusts the
type of the passed generator.  'functools' is
about 'partials', 'lru_cache' and 'wraps' kind
of things.

 2. I propose to disallow using of 'for..in' loops,
and builtins like 'list()', 'iter()', 'next()',
'tuple()' etc on coroutines.

It's possible by modifying PyObject_GetIter to
raise an exception if it receives a coroutine-object.

'yield from' can also be modified to only accept
coroutine objects if it is called from a generator
with CO_COROUTINE flag.

This will further separate coroutines from
generators, making it harder to screw something
up by an accident.

I have a branch of reference implementation
https://github.com/1st1/cpython/tree/await_noiter
where this is implemented.  I did not observe
any performance drop.

There is just one possible backwards compatibility
issue here: there will be an exception if  some user
of asyncio actually used to iterate over generators
decorated with @coroutine.  But I can't imagine
why would someone do that, and even if they did --
it's probably a bug or wrong usage of asyncio.


 That's it!  I'd be happy to hear some feedback!

 Thanks,
 Yury



 PEP: 492
 Title: Coroutines with async and await syntax
 Version: $Revision$
 Last-Modified: $Date$
 Author: Yury Selivanov yseliva...@sprymix.com
 Status: Draft
 Type: Standards Track
 Content-Type: text/x-rst
 Created: 09-Apr-2015
 Python-Version: 3.5
 Post-History: 17-Apr-2015, 21-Apr-2015


 Abstract
 

 This PEP introduces new syntax for coroutines, asynchronous ``with``
 statements and ``for`` loops.  The main motivation behind this proposal
 is to streamline writing and maintaining asynchronous code, as well as
 to simplify previously hard to implement code patterns.


 Rationale and Goals
 ===

 Current Python supports implementing coroutines via generators (PEP
 342), further enhanced by the ``yield from`` syntax introduced in PEP
 380. This approach has a number of shortcomings:

 * it is easy to confuse 

Re: [Python-Dev] async/await in Python; v2

2015-04-23 Thread Wolfgang Langner
On Thu, Apr 23, 2015 at 12:35 PM, Paul Sokolovsky pmis...@gmail.com wrote:

 Hello,

 On Thu, 23 Apr 2015 12:18:51 +0300
 Andrew Svetlov andrew.svet...@gmail.com wrote:

 []

   3.
   async with and async for
   Bead idea, we clutter the language even more and it is one more
   thing every newbie could do wrong.
   for x in y:
 result = await f()
   is enough, every 'async' framework lived without it over years.
 
  async for i in iterable:
  pass
 
  is not equal for
 
  for fut in iterable:
  i = yield from fut

 But people who used Twisted all their life don't know that! They just
 know that async for is not needed and bad.


I don't think it is bad nor not needed, but the syntax is not beautiful and
for the 90% not doing async stuff irritating and one more thing to learn
and do right/wrong.

I had also a need for async loop. But there are other solutions like
channels,
not needing a new syntax.

Also possible a function returning futures and yield in the loop with a
sentinel.

All this goes the road down to a producer consumer pattern. Nothing more.



 I know I'm a bad guy to make such comments, too bad there's a bit of
 truth in them, or everyone would just call me an a%$ole right away.


 Generally, I already provided feedback (on asyncio list) that asyncio
 is based not on native Python concepts like a coroutine, but on
 foreign concurrency concepts like callback or Future, and a coroutine
 is fitted as a second-class citizen on top of that. I understand why
 that was done - to not leave out all those twisteds from a shiny new
 world of asyncio, but sometimes one may wonder if having a clear cut
 would've helped (compat could then have been added as a clearly separate
 subpackage, implemented in terms of coroutines). Now people coming from
 non-coroutine frameworks who were promised compatibility see bad
 things in asyncio (and related areas), and people lured by a promise of
 native Python framework see bad things too.


This has nothing to do with people using twisted or other async frameworks
like tornado.
I think a coroutine should be first class. But all this should be done in a
way a beginner
can handle and not design this stuff for experts only. If we do this we
scare away new people.

This can be done step by step. No need to hurry.


And finally we have stackless Python but explicit. ;-)


-- 
bye by Wolfgang
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Type hints -- a mediocre programmer's reaction

2015-04-23 Thread Wolfgang Langner
Hello,

On Thu, Apr 23, 2015 at 11:59 AM, Paul Sokolovsky pmis...@gmail.com wrote:

 Hello,

 On Thu, 23 Apr 2015 10:43:52 +0200
 Wolfgang Langner tds333+py...@gmail.com wrote:

 []

  Also ask why no one used type specifier, they are possible since
  Python 3.0 ?
  Because it is the wrong way for Python.

 That's an example of how perceptions differ. In my list, everyone(*)
 uses them - MyPy, MicroPython, etc. Even more should use them (any JIT
 module, which are many), but sit in the bushes, waiting for a kick, like
 PEP484 provides.


 (*) Everyone of those who needs them. Otherwise, let's throw out
 metaclasses - noone uses them.


They are there to be used and won't go away.
But for most Libraries out there no one used it.

JIT (Numba), Cython and other compilers/tools doing optimization have their
own syntax and needs.
They need even more information like i32, i64, different floats and so on.
MyPy is really new and in development. And the docstring type spec way is
still possible.
MicroPython uses one annotation for their inline assembler stuff not type
hints.
For the library there are no type hints for function definitions.


Remark: I use Metaclasses, seldom but if needed very useful. :-)


-- 
bye by Wolfgang
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Improvements for Pathlib

2014-11-10 Thread Wolfgang Langner
Hi,

some background about my pathlib usage.
I use the backported version of pathlib and tried to replace the usage of
path.py (also widely used library).

Some features part of path.py missing in pathlib:

- expanduser, expandvars, I have to use the os functions to do this.
  It is possible but better if this is part of pathlib.

- shutil stuff, as noted below

- dirs, and files from path.py. I often had to traverse a directory.
  I know this is possible with a generator. But It is so often needed that I
  missed this. Simple dirs(pattern) and files(pattern) is enough. With support
  for ** as recursion.

Pathlib is a great library and it was good to add it to the standard lib.
Now it is time to improve it further.


On 08.11.2014 16:46, Ionel Cristian Mărieș wrote:
 Hello,
 
 In the current incarnation Pathlib is missing some key features I need in my
 usecases. I want to contribute them but i'd like a bit of feedback on the new
 api before jumping to implementation.
 
 The four things I need are:
 
 #1. A context manager for temporary files and dirs (that cleans everything up
 on exit). Eg:
 
 with pathlib.TmpDir(suffix='', prefix='') as tmp:
   assert isinstance(tmp, pathlib.Path)
 
 with pathlib.TmpFile(suffix='', prefix='') as tmp:
   assert isinstance(tmp, pathlib.Path)

Good idea, but we should better add path support to tempfile module.

 #2. A context manager for changing working directory (that switches to the old
 cwd on exit). Eg:
 
 with path.cd http://path.cd():
  assert os.getcwd() == path

Also useful. Even if the state is global. But has to be noted in documentation.

 #
 ​3​
 . Shutil-like features:
 
 - copy_file
 - copy_dir (or copy_tree?)
 - copy (does copy_dir or copy_file depending on what the source is. How to
 handle symlinks?)
 - rm_dir (or rm_tree? Also, this would be required for the TmpDir path).

I also often needed this.

Thanks for bringing this up.


Regards,

Wolfgang



___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Make str/bytes hash algorithm pluggable?

2013-10-14 Thread Wolfgang Langner
Hi,

I have found a link to a speed comparison of hash functions:

http://code.google.com/p/xxhash/


Regards,

Wolfgang
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Binary Compatibility Issue with Python v2.6.5 and v3.1.2

2010-04-20 Thread Wolfgang Langner

Hi,

On 20.04.2010 14:19, Phil Thompson wrote:

When I build my C++ extension on Windows (specifically PyQt with MinGW)
against Python v2.6.5 it fails to run under v2.6.4. The same problem exists
when building against v3.1.2 and running under v3.1.1.

The error message is...

ImportError: DLL load failed: The specified procedure could not be found.

...though I don't know what the procedure is.

When built against v2.6.4 it runs fine under all v2.6.x. When built under
v3.1.1 it runs fine under all v3.1.x.

I had always assumed that an extension built with vX.Y.Z would always run
under vX.Y.Z-1.

Am I wrong in that assumption, or is this a bug in the latest versions?


It is also possible, that this is a windows issue with Manifest and DLL 
loading. Welcome in the manifest hell.


If Version 2.6.4 is build against another version of the msvcrt as version 
2.6.5 than this could happen.


Possible solution is to not automatically build the manifest and use one with
hard coded versions across one Python version 2.6.X.

Regards Wolfgang

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Python developers are in demand

2007-10-16 Thread Wolfgang Langner
On 10/16/07, Wolfgang Langner [EMAIL PROTECTED] wrote:
 On 10/15/07, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote:
 
   I wonder if we should start maintaining a list of Python developers
   for hire somewhere on python.org, beyond the existing Jobs page. Is
   anyone interested in organizing this?
 
  Andrew What about something a little less formal - a mailing list such
  Andrew as python-jobs?
 
  How about just creating a page on the wiki and letting those people
  participate who are interested?


That's a good idea.
Post the page and I add my name.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


[Python-Dev] Win64, 64 Bit Version and 32 Bit version parallel install not possible

2007-04-20 Thread Wolfgang Langner
Hello,

I tried to install the 64 Bit python version (2.5.1) and the 32 Bit
version on my computer.
But it is not possible because the installer complains that this
version of python is already installed.

Is it in general not possible to install both versions (in separate
directories) ?

Or is it a bug ?


Wolfgang
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Importing .pyc in -O mode and vice versa

2006-11-06 Thread Wolfgang Langner
Why not only import *.pyc files and no longer use *.pyo files.

It is simpler to have one compiled python file extension.
PYC files can contain optimized python byte code and normal byte code.


-- 
bye by Wolfgang
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] User's complaints

2006-07-13 Thread Wolfgang Langner
On 7/13/06, Jeroen Ruigrok van der Werven [EMAIL PROTECTED] wrote:
 Things that struck me as peculiar is the old:

 if __name__ == __main__:
 whatever()

 This is so out of tune with the rest of python it becomes a nuisance.

It is not beautiful but very useful.
In Python 3000 we can replace it with:

@main
def whatever():
...

to mark this function as main function if module executed directly.

-- 
bye by Wolfgang
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Looking for Memories of Python Old-Timers

2006-05-23 Thread Wolfgang Langner
Just to remember 10 years ago was the python 1.3 release.
I can't remember how long I use python but I can remember the first
release I used.

Use the cvs (svn) history of tags to get the date:

http://svn.python.org/view/python/tags/

I hope this helps. :-)

2006/5/23, Guido van Rossum [EMAIL PROTECTED]:
 How long have you used Python? 10 years or longer? Please tell us how
 you first heard of the language, how you first used it, and how you
 helped develop it (if you did). More recent reminiscences are welcome
 too!

-- 
bye by Wolfgang
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Dropping __init__.py requirement for subpackages

2006-04-26 Thread Wolfgang Langner
On 4/26/06, Guido van Rossum [EMAIL PROTECTED] wrote:

 So I have a very simple proposal: keep the __init__.py requirement for
 top-level pacakages, but drop it for subpackages. This should be a
 small change. I'm hesitant to propose *anything* new for Python 2.5,
 so I'm proposing it for 2.6; if Neal and Anthony think this would be
 okay to add to 2.5, they can do so.

-1 from me.

I had never a problem with __init__.py to mark a package or subpackage.
Better add __namespace__.py to state a package dir as namespace package.
And support multiple occurrences of on_python_path/namespace_name/package.

--
bye by Wolfgang
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] gmane.comp.python.devel.3000 has disappeared

2006-04-01 Thread Wolfgang Langner
On 4/1/06, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote:

 Terry For about a week, I have been reading and occasionally posting to
 Terry the new pydev-3000 mailing list via the gmane mirror
 Terry gmane.comp.lang.devel.3000.  Today, it has disappeared and was
 Terry still gone after reloading their newsgroup list.  Was this
 Terry intentional on the part of the mail list maintainers?  (I
 Terry certainly hope not!)  Or is it a repairable glitch somewhere
 Terry between the list and gmane?

 Gmane is subscribed.  I've sent a message to them.  Hopefully they will get
 things corrected soon.

Yes Gmane is subscribed.
I checked if there is a pydev-3000 newsgroup on there server.

Python 3000 new group is:

http://dir.gmane.org/gmane.comp.python.python-3000.devel

I tested it (subscribed) and it works.

Try to refresh your subscription list in the news reader.

--
bye by Wolfgang
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] I'm not getting email from SF when assignedabug/patch

2006-03-30 Thread Wolfgang Langner
Hello,

On 3/29/06, Guido van Rossum [EMAIL PROTECTED] wrote:
 On 3/29/06, Wolfgang Langner [EMAIL PROTECTED] wrote:

  It is a Java system. Why promote Java solutions for python ?
  I think there are good python solutions for a bug tracker and we
  should prefer them.

...
 Also, we're supposed to be friendly with Java, as we have a major
 product in that arena. What if Java folks were to reject a Python
 solution because it's not written in Python? Wouldn't we be upset
 about the parochialism?

I have no problem with Java, neither with other languages. Most time I
have to program
in C++ and sometimes in Java.
I only suggest to give python solutions a try even if they are not yet
equal in quality.
But they should be better than SF.

 The language choice should only be used as an argument if all else is
 equal. Of course, hackability of a particular solution may be a
 criterion too, and there the language choice could matter. But the
 above response sounded like a knee-jerk to me, and IMO needs to be
 rebutted.

I think there   are  good python solutions and please don't underrate
the marketing effect.
My opinion is only subjective and to choose one system we must define
some criteria's
to decide the best one.

But something other :-)

Even Ruby on Rails uses Trac:
http://dev.rubyonrails.org/wiki


--
bye by Wolfgang
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] pysqlite for 2.5?

2006-03-29 Thread Wolfgang Langner
Hello,

 I think short names are more more consistent with the existing naming in
 the standard library.

 +1 on db.sqlite from me.

same for me +1 on db.sqlite

 db.sql.sqlite is another possibility, if adding something like Durus or
 ZODB in the same top-level namespace could be considered for 2.6.

-1 on db.sql.sqlite.
Keep structure flat. Or we are eventually in a Java world with
org.something.this.andthat 

--
bye by Wolfgang
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Py3k: Except clause syntax

2006-03-18 Thread Wolfgang Langner
Hello,

what about:

try:
  something
except NameError or OtherError as e:


Only a thought.

--
bye by Wolfgang
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


[Python-Dev] Bug 1184112 still valid

2006-03-17 Thread Wolfgang Langner
Hello,

today I got my first real python bug. Problem is described in:

Bug with ID: 1184112

http://sourceforge.net/tracker/index.php?func=detailaid=1184112group_id=5470atid=105470

I use python 2.3.5 in an embedded C++ Application which uses function:
PyRun_SimpleString.

Is this bug still present in current python or the 2.4 line ?
I found nothing about it.

--
bye by Wolfgang
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] C++ for CPython 3? (Re: str.count is slow)

2006-03-05 Thread Wolfgang Langner
Hello,

 should we perhaps switch to (careful use of) C++ in 3.0 ?
 I can't see many advantages in moving to C++, but a lot of disadvantages:

 - Size increase, especially when we start using templates
 - Performance decrease
 - Problems with name mangling together with dynamic loading and cross
 module API's
 - Everything has to be build with the same compiler, binaries created
 with different compilers can't interoperate
 - Possibly all extensions modules have to be (re)written in C++
 - Moving to C++ will change Python's well known API substantially

Same opinion. C++ is evil, please don't use it.
You get a lot of new problems and nearly not benefit.

Better go to jython or consider the way of pypy.

--
bye by Wolfgang
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] C++ for CPython 3? (Re: str.count is slow)

2006-03-01 Thread Wolfgang Langner
Hello,

 should we perhaps switch to (careful use of) C++ in 3.0 ?
 I can't see many advantages in moving to C++, but a lot of disadvantages:

 - Size increase, especially when we start using templates
 - Performance decrease
 - Problems with name mangling together with dynamic loading and cross
 module API's
 - Everything has to be build with the same compiler, binaries created
 with different compilers can't interoperate
 - Possibly all extensions modules have to be (re)written in C++
 - Moving to C++ will change Python's well known API substantially

Same opinion. C++ is evil, please don't use it.
You get a lot of new problems and nearly not benefit.

Better go to jython or consider the way of pypy.

--
bye by Wolfgang
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 8 updates/clarifications, function/method style

2005-12-15 Thread wolfgang . langner
Hi,

 Too late. I don't think the diversity is all that distracting.
 I disagree. One of the things that Java got very much right was to
 specify, from the very beginning, what the preferred conventions are
 for naming conventions. (Packages in lowercase, Classes in CapWords,
 methods and variables in lowerCapWords, constants optionally in
 ALL_CAPS. Abbrevs avoided, acronyms have all letters capitalized, eg:
 SimpleHTTPServer.)

 The conventions are nearly universally followed, and as a result in
 java I always know how to spell things. I never have to remember
 whether it's myDict.hasKey() and myDict.popitem() or myDict.has_key()
 and myDict.popItem(). Haskell goes too far -- they REQUIRE a certain
 convention as part of the language... this prevents breaking the rules
 on purpose (eg: wrapping a library from another language, or using
 an object with attributes to represent an XML node with child nodes).
 
 I agree completely with this.  I might remember the name of a method,
 but I don't always remember the capping and the possible use of
 underscores.  Consistency would be really nice.  I am not saying we
 should rename everything (at least not until Python 3  =), but at
 least we can make sure new stuff that is not preexisting can use a
 consistent naming scheme.
 
 And as for it being contentious, I say Guido can pronounce on this. 
 We are all grown-ups and can learn to name things in a certain way to
 give our memories an easier time.  =)

Same for me. Most time I can remember the name but stuck with
capping of the word. And one of the advantages of Java std lib is that naming 
is consistent.
I hope new stuff will follow only one naming style. And now we should
(or one person :-) should) decide which one. And that's the way to go
for new stuff in std lib. And it states as an example for external modules.
As an example we can check if ElementTree matches this. g

For Python 3 it's possible to switch to this consistent style. For a new
and better world.


bye by Wolfgang
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Conversion to Subversion is complete

2005-10-27 Thread Wolfgang Langner
Hello,

[EMAIL PROTECTED] wrote:

 martin The Python source code repository is now converted to
 martin subversion; please feel free to start checking out new
 martin sandboxes. 
 
 Excellent...  Thanks for all the effort.
Good work. I checked the http and viewcvs access and all worked.

But why is an old subversion used ?
(Powered by Subversion version 1.1.4)

bye by Wolfgang

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


[Python-Dev] Re: [maintenance doc updates]

2005-03-30 Thread Wolfgang Langner
Hello,

Fred L. Drake wrote:
 The maintenance version of the documentation has been updated:
 
 http://www.python.org/dev/doc/maint24/

Very good.

The download links under http://docs.python.org/download.html
doesn't work any more. (packages not there)

But 2.4.1 is not yet released, so I will only remind this
not criticise.

bye by Wolfgang

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com