[issue215126] Over restricted type checking on eval() function

[admin]

Change by admin :


___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue215126] Over restricted type checking on eval() function

[admin]

Change by admin :


--
github: None -> 33178

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43085] Loosening | and |= operator type checking restriction

[Brandt Bucher]

Brandt Bucher  added the comment:

Any class that registers itself as a Mapping/MutableMapping using their 
"register" methods (*not* through inheritance) will be lacking the new methods. 
This includes all C extensions.

Further, binary "|" would require subclasses of these ABCs to have a concrete 
way of creating a new instance (we don't have anything like Set._from_iterable 
for this). One option could be to just call self.__class__({**self, **other}), 
but a cursory scan of the stdlib shows that many classes don't have compatible 
signatures, and the we would get lots of surprising/broken behavior if we just 
suddenly introduced this.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43085] Loosening | and |= operator type checking restriction

[Vedran Čačić]

Vedran Čačić  added the comment:

@Raymond: why can't ABCs have _default_ implementation of | and |=, in terms of 
other supported methods (such as update)? It seems to me this would be the best 
of both worlds.

--
nosy: +veky

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43085] Loosening | and |= operator type checking restriction

[Brandt Bucher]

Brandt Bucher  added the comment:

Yurii, I agree with Raymond that it's probably not worth adding new ABCs for 
this.

I'm happy to consider patches that add these operators to concrete stdlib 
mappings on a case-by-case basis. However, please be aware that we already went 
through a fairly comprehensive review of each dict/Mapping/MutableMapping 
subclass (real or virtual) in the entire stdlib to see if they would benefit 
here. I don't think we bothered updating *any* classes that weren't publicly 
documented, which is probably why "SignalDict" was left untouched.

It's also important to remember that the motivating use-case for PEP 584 was 
binary "|", and that "|=" just logically followed from that change. In this 
specific case, I think just using the good ol' update method (or even item 
assignment) is probably good enough.

I'm not very familiar with how SignalDict is used, though: do we ever actually 
want to create a new updated copy using binary "|"? That would probably be 
enough to convince me.

--
components: +Library (Lib) -Regular Expressions
versions: +Python 3.10 -Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43085] Loosening | and |= operator type checking restriction

[Yurii Karabas]

Change by Yurii Karabas <1998uri...@gmail.com>:


--
keywords: +patch
pull_requests: +23228
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/24413

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43085] Loosening | and |= operator type checking restriction

[Raymond Hettinger]

Raymond Hettinger  added the comment:

Because ABCs set minimum standards for compliance, they can't be extended 
without breaking existing code that registers to the ABC.

Brandt added __or/ior__ support to some of the mapping classes in the standard 
library, but it is a fact of life that there will always be some mapping 
classes that don't have __or/ior__ support.  It is an optional feature.

It is possible add new ABC subclasses with the extra methods, but I don't think 
that is worth it.  The effort for a mapping class maintainer to change which 
class it inherits from or registers to is about equal to the effort to just add 
the two methods.

So, I think your best bet is to propose PRs for individual mapping classes on a 
case-by-case basis.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43085] Loosening | and |= operator type checking restriction

[Yurii Karabas]

Yurii Karabas <1998uri...@gmail.com> added the comment:

Hi Raymond,

I was trying to add __or__ method for collections.abc.Mapping, __ior__ method 
for collections.abc.MutableMapping, link to issue - 
https://bugs.python.org/issue42841.

I can extract code and move it to separate classes as you propose above.
What do you think about that?

--
nosy: +uriyyo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43085] Loosening | and |= operator type checking restriction

[Raymond Hettinger]

Raymond Hettinger  added the comment:

The operators | and |= always call __or__() and __ior__().  Support for these 
methods is optional.  Some mapping classes support both methods, some only 
support __or__, and some support neither one.

Adding support is done is done on a case by case basis.  SignalDict could be 
updated if needed.

To make it easier for user defined classes, two new collections.ABCs could be 
added, MappingWithOr and MutableMappingWithIor.  That is similar to what we had 
done in Python 2.7 with UserDict and IterableUserDict.

--
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43085] Loosening | and |= operator type checking restriction

[Siyavash]

Change by Siyavash :


--
nosy:  -Siy_V, ezio.melotti

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43085] Loosening | and |= operator type checking restriction

[Siyavash]

New submission from Siyavash :

| and |= operators work only if the operands are of 'dict' type. This means 
that they can't be used with some of the user defined mapping objects.

The case that especially caught my attention was 'SignalDict' type in decimal 
standard library. While 'SignalDict' has got an update method like 'dict', it 
can not be updated with |= operator. The following code results in a TypeError:

decimal.getcontext().traps |= {decimal.DivisionByZero:False}

--
components: Regular Expressions
messages: 386028
nosy: Siy_V, brandtbucher, ezio.melotti, mrabarnett, steven.daprano
priority: normal
severity: normal
status: open
title: Loosening | and |= operator type checking restriction
type: enhancement
versions: Python 3.9

___
Python tracker 
<https://bugs.python.org/issue43085>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Type checking

[Manfred Lotz]

On Sat, 19 Oct 2019 21:58:23 -0700
Paul Rubin  wrote:

> Manfred Lotz  writes:
> > def main():
> > print(greeting(1))
> >
> > Any idea why mypy doesn't detect anything?  
> 
> 
> "main" doesn't have a type signature so it doesn't get checked.  The
> mypy docs explain this.  Try:
> 
>   def main() -> None:
>   print(greeting(1))

Aah, such easy. As pytype did it I didn't assume something like this.

Thanks a lot.

-- 
Manfred

-- 
https://mail.python.org/mailman/listinfo/python-list

Type checking

[Manfred Lotz]

I had a look into type checking and tried a very simple example like
the following:


#!/usr/bin/env python3


def greeting(name: str) -> str:
return 'Hello ' + name


def main():
print(greeting(1))


if __name__ == "__main__":
main()


pytype complained as expected. 

BUT mypy says;

Success: no issues found in 1 source file


This is a Fedora 31 systems with:

Python 3.7.4
mypy 0.730


Any idea why mypy doesn't detect anything?


-- 
Manfred

-- 
https://mail.python.org/mailman/listinfo/python-list

[issue34838] Improve arg clinic code generation for cases with type checking

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
resolution:  -> fixed
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34838] Improve arg clinic code generation for cases with type checking

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:


New changeset cb08a71c5c534f33d9486677534dafb087c30e8c by Serhiy Storchaka 
(Ammar Askar) in branch 'master':
bpo-34838: Use subclass_of for math.dist. (GH-9659)
https://github.com/python/cpython/commit/cb08a71c5c534f33d9486677534dafb087c30e8c


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34838] Improve arg clinic code generation for cases with type checking

[Ammar Askar]

Ammar Askar  added the comment:

Great job Serhiy, your work on argument clinic has encompassed the previous PR 
very well. I've updated it for the changes.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34838] Improve arg clinic code generation for cases with type checking

[Ammar Askar]

Ammar Askar  added the comment:

Sounds good, I'll take a look at the changes and update the PR accordingly.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34838] Improve arg clinic code generation for cases with type checking

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

PR 9659 is mostly superseded by issue35582. But it may contain other changes. 
Ammar, do you mind to create a new PR or merge the old PR with the current 
master?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34838] Improve arg clinic code generation for cases with type checking

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
dependencies: +Argument Clinic: inline parsing code for 1-argument functions, 
Argument Clinic: inline parsing code for functions with only positional 
parameters

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34838] Improve arg clinic code generation for cases with type checking

[Ammar Askar]

Ammar Askar  added the comment:

Aah, I didn't know that ticket existed. Thanks for the link Serhiy, I'll review 
your PR instead. Leaving it up to Raymond if he wants to close the issue and 
use the other one to track this.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34838] Improve arg clinic code generation for cases with type checking

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Thank you for your PR Ammar. But I work on more general changes. The first step 
is issue23867. With implementing the second step this issue can be closed.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34838] Improve arg clinic code generation for cases with type checking

[Ammar Askar]

Ammar Askar  added the comment:

I've attached a PR that implements this. From the looks of it, there aren't a 
lot of uses of subclass_of within argument clinic converted functions at the 
moment. Additionally, most of the places it is used tend to have some non 
object arguments so a call to _PyArg_ParseStack() gets placed anyway.

Attached as part of separate commit in the PR is the conversion of some files 
over to use the new code. These are mostly just to serve as examples. 

While converting would still leave a call to ParseStack in most places, I'd 
like to do some benchmarking and figure out if there is still a benefit. Like 
you said, the fact that branch-prediction is easier might be useful given that 
these functions are usually called with the right types of arguments anyway.

--
nosy: +ammar2
stage: patch review -> 

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34838] Improve arg clinic code generation for cases with type checking

[Ammar Askar]

Change by Ammar Askar :


--
keywords: +patch
pull_requests: +9052
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34838] Improve arg clinic code generation for cases with type checking

[Eric V. Smith]

Change by Eric V. Smith :


--
nosy: +eric.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34838] Improve arg clinic code generation for cases with type checking

[STINNER Victor]

Change by STINNER Victor :


--
nosy:  -vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34838] Improve arg clinic code generation for cases with type checking

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
assignee:  -> serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34838] Improve arg clinic code generation for cases with type checking

[Raymond Hettinger]

New submission from Raymond Hettinger :

An arg clinic specification such as

p: object(subclass_of='_Type')

generates slow code using _PyArg_ParseStack() that has to parse a format string 
like "O!" to decide to make a type check.  Instead, it should directly generate 
a branch-predictable test for the type check and then call the much quicker 
function _PyArg_UnpackStack().

See https://github.com/python/cpython/pull/9628 for an example of this change 
giving a 30% speedup.

--
components: Argument Clinic
messages: 326659
nosy: larry, rhettinger, serhiy.storchaka, vstinner
priority: low
severity: normal
status: open
title: Improve arg clinic code generation for cases with type checking
type: performance
versions: Python 3.8

___
Python tracker 
<https://bugs.python.org/issue34838>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34175] typing.NamedTuple: type-checking error when "index" used as member

[Keith Campbell]

Keith Campbell  added the comment:

> It's up to the OP to file an issue there though

Will do; thanks!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34175] typing.NamedTuple: type-checking error when "index" used as member

[Guido van Rossum]

Guido van Rossum  added the comment:

> Perhaps this report should go on the mypy bug tracker rather than the Python 
> language tracker.

Agreed. It's up to the OP to file an issue there though (hence adding 
@campkeith back to the nosy list).

--
nosy: +campkeith
resolution:  -> third party
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34175] typing.NamedTuple: type-checking error when "index" used as member

[Raymond Hettinger]

Raymond Hettinger  added the comment:

> So I suspect this is a bug in the type checker.

Perhaps this report should go on the mypy bug tracker rather than the Python 
language tracker.

--
nosy: +gvanrossum, levkivskyi, rhettinger -campkeith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34175] typing.NamedTuple: type-checking error when "index" used as member

[Keith Campbell]

New submission from Keith Campbell :

Find the test case below:


from typing import NamedTuple

class Foo(NamedTuple):
alpha: int
index: int


This results in the following error when run through type-checking with mypy:


% mypy --version
mypy 0.620
% mypy go.py
go.py:5: error: Incompatible types in assignment (expression has type "int", 
base class "tuple" defined the type as "Callable[[Tuple[int, ...], Any, int, 
int], int]")


When I instantiate this class in the python interpreter, I have no problem 
accessing the "index" member, so I suspect this is a bug in the type checker.

--
components: Library (Lib)
messages: 322038
nosy: campkeith
priority: normal
severity: normal
status: open
title: typing.NamedTuple: type-checking error when "index" used as member
versions: Python 3.6

___
Python tracker 
<https://bugs.python.org/issue34175>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28834] Type checking in set comparisons.

[nyoshimizu]

nyoshimizu added the comment:

I see. Sorry & thanks!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28834] Type checking in set comparisons.

[SilentGhost]

SilentGhost added the comment:

You seem to be misunderstanding how the intersection/union/etc are supposed to 
be used:

>>> ab = {'a', 'b'}
>>> ab.intersection('bc')
{'b'}

Using set.intersection (where set is a built-in class, rather than an instance 
thereof) requires the first argument to be set (which is the actual instance of 
set class). This is no different from usage of any other class / object across 
Python, however, it is highly uncommon.

--
nosy: +SilentGhost
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28834] Type checking in set comparisons.

[nyoshimizu]

New submission from nyoshimizu:

The non-operator versions of set comparisons (intersection(), union(), etc.) 
exhibit
inconsistent type checking. They only check the first input before deciding 
whether or not to raise a TypeError exception. 

Therefore, it's possible to pass a set first, then other objects (e.g. lists, 
dicts, tuples) and a correct 'intersection' is returned (apparently by ignoring 
ordering and using the keys in dicts). I've attached demonstrative example for 
Python 3.5, although Python 2.7 appears to exhibit the same behavior.

I'm not sure what the intended behavior was (whether or not to require sets). 
8.7.1 Set Objects states: "Note, the non-operator versions of union(), 
intersection(), difference(), and symmetric_difference() will accept any 
iterable as an argument."

Note that #8743 and #17626 appear to confirm that the operator versions should 
not accept non-sets and matches the observed behavior. As the latter issue 
points out, it's documented -- again in 8.7.1 -- that "...their operator based 
counterparts require their arguments to be sets." 

Is this behavior necessary but just not documented? The documentation states 
that "This precludes error-prone constructions like Set('abc') & 'cbs' in favor 
of the more readable Set('abc').intersection('cbs')." In the second example, a 
first set is needed to do the intersection, then 'cbs' gets typecast into a set 
(although I guess so was 'abc'). Then should the first inputs also be typecast 
as sets?

--
files: test_comparison.py
messages: 282036
nosy: nyoshimizu
priority: normal
severity: normal
status: open
title: Type checking in set comparisons.
type: behavior
versions: Python 2.7, Python 3.5
Added file: http://bugs.python.org/file45694/test_comparison.py

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue28834>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Getting started with type-checking

[Steven D'Aprano]

Is there a good guide to getting started with type-checking Python code?
Specifically as an aid to porting from Python 2 to 3.

There are the PEPs, of course. Anything else?

Apart from mypy, what type checkers are available?



-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Just-in-Time Static Type Checking for Dynamic Languages

[Neal Becker]

I saw this article, which might interest some of you.  It discusses 
application to ruby, but perhaps might have ideas useful for python.

https://arxiv.org/abs/1604.03641

-- 
https://mail.python.org/mailman/listinfo/python-list

[issue26201] Faster type checking in listobject.c

[Raymond Hettinger]

Changes by Raymond Hettinger :


--
Removed message: http://bugs.python.org/msg258942

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26201] Faster type checking in listobject.c

[Jeremy Kloth]

Jeremy Kloth added the comment:

Added review

--
nosy: +jkloth

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26201] Faster type checking in listobject.c

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Could you please provide any microbencmarks that show the benefit of this 
optimization?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26201] Faster type checking in listobject.c

[Raymond Hettinger]

New submission from Raymond Hettinger:

A number of fine-grained methods in Objects/listobject.c use PyList_Check().  
They include PyList_Size, PyList_GetItem, PyList_SetItem, PyList_Insert, and 
PyList_Append.

The PyList_Check() works by making two sequentially dependent memory fetches:

movq8(%rdi), %rax
testb   $2, 171(%rax)
je  L1645

This patch proposes a fast path for the common case of an exact match, using 
PyList_CheckExact() as an early-out before the PyList_Check() test:

leaq_PyList_Type(%rip), %rdx # parallelizable
movq8(%rdi), %rax# only 1 memory access
cmpq%rdx, %rax   # macro-fusion  
je  L1604# early-out
testb   $2, 171(%rax)# fallback to 2nd memory access
je  L1604

This technique won't help outside of Objects/listobject.c because the initial 
LEA instruction becomes a MOV for the global offset table, nullifying the 
advantage.

--
assignee: serhiy.storchaka
components: Interpreter Core
files: list_check_fastpath.diff
keywords: patch
messages: 258918
nosy: rhettinger, serhiy.storchaka
priority: normal
severity: normal
stage: patch review
status: open
title: Faster type checking in listobject.c
type: performance
versions: Python 3.6
Added file: http://bugs.python.org/file41713/list_check_fastpath.diff

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue26201>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26201] Faster type checking in listobject.c

[Raymond Hettinger]

Raymond Hettinger added the comment:

Jeremy, thanks for the keen eyes.  Attaching a revised patch.

Serhiy, pure python code doesn't directly access any of the C-API functions 
touched by the patch, so timeit-style examples aren't possible.  The 
beneficiaries of the change are external modules.  I suppose we could write a 
small C extension just to time this but that seems like overkill.

It is a general purpose optimization technique to reduce the number of memory 
accesses in fine-grained functions.  That technique has worked well elsewhere 
in the core where measurable progress was made in many small steps that were 
individually hard to measure.

--
assignee: serhiy.storchaka -> rhettinger
Added file: http://bugs.python.org/file41715/list_check_fastpath2.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26201] Faster type checking in listobject.c

[STINNER Victor]

STINNER Victor added the comment:

> Could you please provide any microbencmarks that show the benefit of this 
> optimization?

Yeah, analyzing the assembler seems overkill to me. I'm not sure that it really 
make the code faster (but it makes the code more complex).

--
nosy: +haypo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26201] Faster type checking in listobject.c

[Raymond Hettinger]

Raymond Hettinger added the comment:

Writing your own static optimizer from scratch is overkill.  Following basic 
techniques from the Agner Fog manuals and Intel optimization manuals is quite 
reasonable in comparison.

Unless you see an actual defect in the patch, I'm applying it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26201] Faster type checking in listobject.c

[STINNER Victor]

STINNER Victor added the comment:

> As noted above, it not reasonable for a C-API that isn't used internally 
> because there is no reasonable way to measure the effect without writing a C 
> extension.

Come on, there is a lot of C functions in Python using the C PyList_xxx() API. 
It's easy to write a benchmark on it in pure Python. Example.


Without the patch (best of 10 runs):

$ for i in $(seq 1 10); do ./python -m timeit -s 's="a " * 10**5' 's.split()'; 
done
1000 loops, best of 3: 1.40 msec per loop

With the patch (best of 10 runs):

$ for i in $(seq 1 10); do ./python -m timeit -s 's="a " * 10**5' 's.split()'; 
done
1000 loops, best of 3: 1.49 msec per loop

It looks like list_check_fastpath2.diff makes PyList_Append() *slower* (6%) in 
practice. I expected it to be *faster*.


Please try to reproduce my microbenchmark, timeit is not really reliable, and I 
hate microbenchmarks, it's easy to make mistakes :-/

Note: It looks like s.split() creates a list of 'a' strings where all 'a' 
strings are the same singleton. So I don't think that creating substrings has 
an important cost.


> If you're going to be in the optimization business, I really wish you would 
> read some of the basics in the field so that each and every technique doesn't 
> have to be re-proved to you each time it used.

It's easy to convince me: show me a benchmark where your patch makes the code 
faster :-) I don't think that there is a strict rule for a miniumum speedup in 
CPython. I'm open for discussion :-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26201] Faster type checking in listobject.c

[STINNER Victor]

Changes by STINNER Victor :


--
nosy: +pitrou, yselivanov

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26201] Faster type checking in listobject.c

[Raymond Hettinger]

Raymond Hettinger added the comment:

> Well, Serhiy Storchaka asked for a microbenchmark. 
> For an optimization, it's a reasonable request no?

As noted above, it not reasonable for a C-API that isn't used internally 
because there is no reasonable way to measure the effect without writing a C 
extension.

FWIW, I look at assembly to see direct evidence for the effect of a change.  It 
makes it easy to see how many memory accesses happen on a given path.

If you're going to be in the optimization business, I really wish you would 
read some of the basics in the field so that each and every technique doesn't 
have to be re-proved to you each time it used.  Here's a short-list I put 
together for those who were interested : 
https://dl.dropboxusercontent.com/u/3967849/sftalk/_build/html/misc.html#optimization-resources

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26201] Faster type checking in listobject.c

[STINNER Victor]

STINNER Victor added the comment:

> Unless you see an actual defect in the patch, I'm applying it.

Well, Serhiy Storchaka asked for a microbenchmark. For an optimization, it's a 
reasonable request no?


> Writing your own static optimizer from scratch is overkill.

My goal is to get speedup on macro benchmarks.

It's hard to get a real speedup on macro benchmark using micro-optimizations, 
even if we combine a lot of them. I'm not saying that micro optimization is a 
waste of time, I like micro-optimizing Python, but we should not abuse of it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26201] Faster type checking in listobject.c

[Raymond Hettinger]

Raymond Hettinger added the comment:

I've lost interest in talking with you.  AFAICT you aren't doing anything to 
help advance or study the patch.  You're just being a PITA and are now 
enlisting people to gang-up on a trivial patch.  This is disgusting.

--
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Type checking tools for Python 2?

[Skip Montanaro]

I've been following along with the discussions related to type
hints[1] on python-ideas and python-dev. I'm interested enough to
start looking into this for my own nefarious purposes. At work, we
have lots of C++ code wrapped by Boost.Python. It seems like creating
type hint stubs for those libraries might be useful. Even if it proves
not to be all that useful for me, it's a ready code base which I could
mess around with and not bother anybody else.

Alas, my environment is still 100% Python 2.7. It looks like mypy
isn't going to be doing Python 2.x anytime soon. Are there any type
hint/check tools available which will work with Python 2?

Thx,

Skip

[1] https://www.python.org/dev/peps/pep-0484/
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Type checking tools for Python 2?

[Mark Lawrence]

On 21/04/2015 19:08, Skip Montanaro wrote:

I've been following along with the discussions related to type
hints[1] on python-ideas and python-dev. I'm interested enough to
start looking into this for my own nefarious purposes. At work, we
have lots of C++ code wrapped by Boost.Python. It seems like creating
type hint stubs for those libraries might be useful. Even if it proves
not to be all that useful for me, it's a ready code base which I could
mess around with and not bother anybody else.

Alas, my environment is still 100% Python 2.7. It looks like mypy
isn't going to be doing Python 2.x anytime soon. Are there any type
hint/check tools available which will work with Python 2?

Thx,

Skip

[1] https://www.python.org/dev/peps/pep-0484/



Nothing at all wrong with Python 2.7.  Possibly better still from 
https://github.com/JukkaL/mypy it says You need Python 3.2 or later to 
run mypy. You can have multiple Python versions (2.x and 3.x) installed 
on the same system without problems.  I'm just thinking out aloud, does 
this mean you could use a 3.x version running anywhere you like to check 
out your 2.7 codebase?  Just a thought, and only one way to find out 
unless somebody better qualified than I am can give a definitive statement.


--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

--
https://mail.python.org/mailman/listinfo/python-list

Re: Type checking tools for Python 2?

[Terry Reedy]

On 4/21/2015 2:08 PM, Skip Montanaro wrote:

I've been following along with the discussions related to type
hints[1] on python-ideas and python-dev. I'm interested enough to
start looking into this for my own nefarious purposes. At work, we
have lots of C++ code wrapped by Boost.Python. It seems like creating
type hint stubs for those libraries might be useful. Even if it proves
not to be all that useful for me, it's a ready code base which I could
mess around with and not bother anybody else.

Alas, my environment is still 100% Python 2.7. It looks like mypy
isn't going to be doing Python 2.x anytime soon. Are there any type
hint/check tools available which will work with Python 2?


I believe mypy can typecheck 2.x code in conjunction with stub files. 
The fact that you have to install Python 3 to run mypy should be irrelevant.


--
Terry Jan Reedy

--
https://mail.python.org/mailman/listinfo/python-list

Re: Type checking tools for Python 2?

[Skip Montanaro]

On Tue, Apr 21, 2015 at 2:26 PM, Terry Reedy tjre...@udel.edu wrote:
 Sorry if I jumped the gun on mypy.

Not a problem. I went ahead and installed Python 3.5a4 and mypy. It
seems to run, but it isn't terribly happy with my Python 2 code. Not
so much the actual syntax as things like the builtin cmp() function
and the cPickle module being missing.

Skip
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Type checking tools for Python 2?

[Skip Montanaro]

On Tue, Apr 21, 2015 at 1:29 PM, Terry Reedy tjre...@udel.edu wrote:
 I believe mypy can typecheck 2.x code in conjunction with stub files.

Thanks. Maybe the MyPy FAQ is just out-of-date? It includes this QA:

All of my code is still in Python 2. What are my options?

Mypy currently supports Python 3 syntax. Python 2 support is still
in early stages of development. However, Python 2 support will be
improving. Mypy includes a custom codec that lets you use Python 3
function annotations in Python 2 code. The codec just removes the
annotations before bytecode compilation.

Unless I misread the first couple sentences of the answer, it seems
that it can't actually grok Python 2 code. I'm not sure what to make
of the statement about a custom codec and how that would relate to
function annotations.

Skip
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Type checking tools for Python 2?

[Terry Reedy]

On 4/21/2015 3:11 PM, Skip Montanaro wrote:

On Tue, Apr 21, 2015 at 1:29 PM, Terry Reedy tjre...@udel.edu wrote:

I believe mypy can typecheck 2.x code in conjunction with stub files.


I based this on comments in the PEP 484 discussion on py-dev, which I 
might have misread as being about now rather than the future.  Or maybe 
the comment was about something other than mypy.  Google's python 
typechecker is certainly targeting 2.7, since that is what they are 
using.  I do not know if they will release it publicly.





Thanks. Maybe the MyPy FAQ is just out-of-date? It includes this QA:

 All of my code is still in Python 2. What are my options?

 Mypy currently supports Python 3 syntax. Python 2 support is still
 in early stages of development. However, Python 2 support will be
 improving. Mypy includes a custom codec that lets you use Python 3
 function annotations in Python 2 code. The codec just removes the
 annotations before bytecode compilation.

Unless I misread the first couple sentences of the answer, it seems
that it can't actually grok Python 2 code. I'm not sure what to make
of the statement about a custom codec and how that would relate to
function annotations.


Sorry if I jumped the gun on mypy.

--
Terry Jan Reedy

--
https://mail.python.org/mailman/listinfo/python-list

Re: Type checking tools for Python 2?

[Chris Angelico]

On Wed, Apr 22, 2015 at 5:33 AM, Skip Montanaro
skip.montan...@gmail.com wrote:
 On Tue, Apr 21, 2015 at 2:26 PM, Terry Reedy tjre...@udel.edu wrote:
 Sorry if I jumped the gun on mypy.

 Not a problem. I went ahead and installed Python 3.5a4 and mypy. It
 seems to run, but it isn't terribly happy with my Python 2 code. Not
 so much the actual syntax as things like the builtin cmp() function
 and the cPickle module being missing.

Shouldn't be too hard to add a few stub files; cPickle could just
from pickle import *, and cmp() would take two of Any and return
int, etc. MyPy will, if I've understood the docs correctly, welcome a
patch/pull-request that adds this kind of thing.

Actually, the things I'd most expect problems with are the print
statement and int/int - int division, the latter mainly because
it's so subtle. (The bytes vs unicode distinction is dealt with
already.) But both can be fixed with future directives anyway, so
worst case you just have to help your code come a bit closer to Py3
(and frankly, that's not a bad thing).

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue21560] gzip.write changes trailer ISIZE field before type checking - corrupted gz file after trying to write string

[Roundup Robot]

Roundup Robot added the comment:

New changeset 4dfe0634d11a by Serhiy Storchaka in branch '2.7':
Issue #21560: An attempt to write a data of wrong type no longer cause
https://hg.python.org/cpython/rev/4dfe0634d11a

New changeset 6eb48b22ff5c by Serhiy Storchaka in branch '3.4':
Issue #21560: An attempt to write a data of wrong type no longer cause
https://hg.python.org/cpython/rev/6eb48b22ff5c

--
nosy: +python-dev

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue21560
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21560] gzip.write changes trailer ISIZE field before type checking - corrupted gz file after trying to write string

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Tests are taken from issue23688. Thanks for your contribution Wolfgang.

--
assignee:  - serhiy.storchaka
nosy: +serhiy.storchaka
resolution:  - fixed
stage: test needed - resolved
status: open - closed

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue21560
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21560] gzip.write changes trailer ISIZE field before type checking - corrupted gz file after trying to write string

[Antoine Pitrou]

Changes by Antoine Pitrou pit...@free.fr:


--
nosy: +nadeem.vawda

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue21560
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21560] gzip.write changes trailer ISIZE field before type checking - corrupted gz file after trying to write string

[Serhiy Storchaka]

Changes by Serhiy Storchaka storch...@gmail.com:


--
stage: needs patch - test needed

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue21560
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21560] gzip.write changes trailer ISIZE field before type checking - corrupted gz file after trying to write string

[Wolfgang Maier]

New submission from Wolfgang Maier:

I ran into this:

 gzout = gzip.open('test.gz','wb')
 gzout.write('abcdefgh') # write expects bytes not str
Traceback (most recent call last):
  File pyshell#2, line 1, in module
gzout.write('abcdefgh')
  File /usr/lib/python3.4/gzip.py, line 343, in write
self.crc = zlib.crc32(data, self.crc)  0x
TypeError: 'str' does not support the buffer interface

 gzout.write(b'abcdefgh') # ok, use bytes instead
8
 gzout.close()

But now the file is not recognized as valid gzip format anymore (neither by the 
gzip module nor by external software):

 gzin = gzip.open('test.gz','rb')
 next(gzin)
Traceback (most recent call last):
  File pyshell#32, line 1, in module
next(gzin)
  File /usr/lib/python3.4/gzip.py, line 594, in readline
c = self.read(readsize)
  File /usr/lib/python3.4/gzip.py, line 365, in read
if not self._read(readsize):
  File /usr/lib/python3.4/gzip.py, line 465, in _read
self._read_eof()
  File /usr/lib/python3.4/gzip.py, line 487, in _read_eof
raise OSError(Incorrect length of data produced)
OSError: Incorrect length of data produced

Turns out that gzip.write increased the ISIZE field value by 8 already during 
the failed call with the str object, so it is now 16 instead of 8:
 raw = open('test.gz','rb')
 [n for n in raw.read()] # ISIZE is the fourth last element
[31, 139, 8, 8, 51, 46, 127, 83, 2, 255, 116, 101, 115, 116, 0, 75, 76, 74, 78, 
73, 77, 75, 207, 0, 0, 80, 42, 239, 174, 16, 0, 0, 0]

in other words: gzip.GzipFile.write() leaps (and modifies) before it checks its 
input argument.

--
components: Library (Lib)
messages: 218961
nosy: wolma
priority: normal
severity: normal
status: open
title: gzip.write changes trailer ISIZE field before type checking - corrupted 
gz file after trying to write string
type: behavior
versions: Python 3.4

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue21560
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21560] gzip.write changes trailer ISIZE field before type checking - corrupted gz file after trying to write string

[Serhiy Storchaka]

Changes by Serhiy Storchaka storch...@gmail.com:


--
keywords: +easy
stage:  - needs patch
versions: +Python 2.7, Python 3.5

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue21560
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21560] gzip.write changes trailer ISIZE field before type checking - corrupted gz file after trying to write string

[Wolfgang Maier]

Wolfgang Maier added the comment:

ok, this seems to be really easy:
patch attached

--
keywords: +patch
Added file: http://bugs.python.org/file35323/GzipFile_write.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue21560
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21560] gzip.write changes trailer ISIZE field before type checking - corrupted gz file after trying to write string

[Wolfgang Maier]

Wolfgang Maier added the comment:

or not - my patch just causes a different error in my example :(

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue21560
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21560] gzip.write changes trailer ISIZE field before type checking - corrupted gz file after trying to write string

[Claudiu.Popa]

Claudiu.Popa added the comment:

Moving `self.crc = zlib.crc32(data, self.crc)  0x` before `self.size = 
self.size + len(data)` should be enough. Also, your patch needs a test.

--
nosy: +Claudiu.Popa

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue21560
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21560] gzip.write changes trailer ISIZE field before type checking - corrupted gz file after trying to write string

[Wolfgang Maier]

Wolfgang Maier added the comment:

isn't this exactly what I did in my patch ?

actually, it is working, I just had an error in my preliminary test script.

I may be able to work on an official test at some point, but definitely not 
over the next week

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue21560
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13387] suggest assertIs(type(obj), cls) for exact type checking

[Ezio Melotti]

Changes by Ezio Melotti ezio.melo...@gmail.com:


--
resolution:  - fixed
stage: commit review - committed/rejected
status: open - closed
type: behavior - enhancement

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13387
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13387] suggest assertIs(type(obj), cls) for exact type checking

[Roundup Robot]

Roundup Robot devn...@psf.upfronthosting.co.za added the comment:

New changeset 88aacd3541ae by Ezio Melotti in branch '2.7':
#13387: rephrase unclear sentence.
http://hg.python.org/cpython/rev/88aacd3541ae

New changeset eccb4795767b by Ezio Melotti in branch '3.2':
#13387: rephrase unclear sentence.
http://hg.python.org/cpython/rev/eccb4795767b

New changeset 064854cef999 by Ezio Melotti in branch 'default':
#13387: merge with 3.2.
http://hg.python.org/cpython/rev/064854cef999

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13387
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13387] suggest assertIs(type(obj), cls) for exact type checking

[Éric Araujo]

Éric Araujo mer...@netwok.org added the comment:

Your latest proposal is better.  I would prefer mentioning subclasses, but 
don’t feel strongly about it.  One markup nit: I’d use ``code`` instead of 
(ab)using :func:; the doc for assertIs is just a few paragraphs above, it won’t 
be hard to find.

--
resolution: fixed - 
stage: committed/rejected - commit review
status: closed - open

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13387
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13387] suggest assertIs(type(obj), cls) for exact type checking

[Éric Araujo]

Éric Araujo mer...@netwok.org added the comment:

No, I’m not talking about a rephrasing, but on a full change of meaning.  I 
don’t understand your use of “superclasses” at all; isinstance(x, T) checks if 
x is an instance of T or any subclass, am I wrong?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13387
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13387] suggest assertIs(type(obj), cls) for exact type checking

[Ezio Melotti]

Ezio Melotti ezio.melo...@gmail.com added the comment:

Is To check for the exact type, use :func:`assertIs(type(obj), cls) 
assertIs`. better?
I think the problem this solves is clear enough even without mentioning 
sub/superclasses.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13387
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13387] suggest assertIs(type(obj), cls) for exact type checking

[Ezio Melotti]

Ezio Melotti ezio.melo...@gmail.com added the comment:

Fixed.

--
assignee: docs@python - ezio.melotti
resolution:  - fixed
stage: needs patch - committed/rejected
status: open - closed
versions: +Python 2.7, Python 3.2

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13387
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13387] suggest assertIs(type(obj), cls) for exact type checking

[Roundup Robot]

Roundup Robot devn...@psf.upfronthosting.co.za added the comment:

New changeset fd9d7a8e45bc by Ezio Melotti in branch '2.7':
#13387: add note about checking the exact type in assertIsInstance doc.
http://hg.python.org/cpython/rev/fd9d7a8e45bc

New changeset 583aff635ce1 by Ezio Melotti in branch '3.2':
#13387: add note about checking the exact type in assertIsInstance doc.
http://hg.python.org/cpython/rev/583aff635ce1

New changeset 196d485ed26d by Ezio Melotti in branch 'default':
#13387: merge with 3.2.
http://hg.python.org/cpython/rev/196d485ed26d

--
nosy: +python-dev

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13387
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13387] suggest assertIs(type(obj), cls) for exact type checking

[Éric Araujo]

Éric Araujo mer...@netwok.org added the comment:

+ To check for a specific type (without including superclasses) use
+ :func:`assertIs(type(obj), cls) assertIs`.

Don’t you mean “without accepting subclasses”, not superclasses?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13387
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13387] suggest assertIs(type(obj), cls) for exact type checking

[Ezio Melotti]

Ezio Melotti ezio.melo...@gmail.com added the comment:

 + To check for a specific type (without including superclasses) use
 + :func:`assertIs(type(obj), cls) assertIs`.

 Don’t you mean “without accepting subclasses”, not superclasses?

I mean:
 class MyInt(int): pass # my specific type
...
 isinstance(MyInt(5), int)  # int superclass included
True
 type(MyInt(5)) is int  # int superclass not included
False
 type(MyInt(5)) is MyInt# check for specific type
True

Do you think I should rephrase it (or maybe just remove the (...))?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13387
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13387] suggest assertIs(type(obj), cls) for exact type checking

[Florent Xicluna]

Florent Xicluna florent.xicl...@gmail.com added the comment:

 +1 on a doc addition (I can even volunteer a patch)

I agree we can highlight the difference between assertIs(type(obj), cls) and 
assertIsInstance(obj, cls) in the documentation.

Let's forget this patch and keep it simple.

--
assignee:  - docs@python
components: +Documentation
nosy: +docs@python
stage:  - needs patch
title: add exact_type argument to assertIsInstance - suggest 
assertIs(type(obj), cls) for exact type checking
type: feature request - behavior

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13387
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Type checking versus polymorphism (was: list(), tuple() should not place at Built-in functions in documentation)

[Steven D'Aprano]

Chris Rebert wrote:

 On Fri, Jul 15, 2011 at 7:47 PM, Steven D'Aprano
 steve+comp.lang.pyt...@pearwood.info wrote:
 snip
 Assertions are for testing internal program logic, not for validation.

 (I don't even like using assert for testing. How do you test your code
 with assertions turned off if you use assert for testing?)
 
 I would think that would only matter if either the asserted
 expressions caused side-effects or there was nontrivial logic in the
 AssertionError handler, which would indicate a rather screwy codebase 
 and point to a possible PEBKAC issue that testing cannot hope to
 remedy.

I'm not sure I follow you there... 

For any piece of code, I claim that we should test both with and without
the -O switch. If I don't test it with -O, how do I know it works correctly
when run with -O? Even if I don't use assert yourself, I don't know if the
code I call uses assert badly and therefore is affected by -O.

(I recently discovered that ElementTree, in the Python standard library,
uses assert for data validation in at least some Python versions.)

Besides, I don't know what -O does, apart from disabling assertions. Maybe
nothing else. Maybe lots of things. It may change from version to version.
Who knows? Whatever it does, it *shouldn't* change the semantics of my
code, but it *could*, hence I better test it to find out if anything has
changed.

So are we agreed that it is wise to test code both with and without the -O
switch?

But here's the problem... the -O switch is global, and not module specific.
So if your test suite looks like this:

import mymodule
assert hasattr(mymodule, __all__)
assert isinstance(mymodule.__version__, str)
assert mymodule.count_spam(spam spam spam) == 3
a, b, c = mymodule.aardvark(42)
assert a  b  c

when running with -0, all those tests LITERALLY go away and you're left with
a significantly smaller test suite:

import mymodule
a, b, c = mymodule.aardvark(42)

which is hardly better than no test at all.



-- 
Steven

-- 
http://mail.python.org/mailman/listinfo/python-list

Type checking versus polymorphism (was: list(), tuple() should not place at Built-in functions in documentation)

[Ben Finney]

Inside fancheyuj...@gmail.com writes:

 I just follow some coding rules of me:
 1. Controlling input strictly.

Asserting that the input is of a specific type is too strict. Does your
code work if the input is not a list or tuple?

I suspect strongly that the answer is yes, it works fine with any
sequence, even ones that don't inherit from ‘list’ nor ‘tuple’. It will
probably work with any sequence; it amy even work with any iterable.

Instead of insisting on specific types, you should support polymorphism:
expect *behaviour* and allow any input that exhibits that behaviour.

This is known as “duck typing”: you don't need to care whether it's a
duck, you just need to know whether it walks like a duck and quacks like
a duck. If it turns out to be a goose, but that won't affect your code,
you shouldn't care.

 2. In a function keep doubting on its parameters until they're
 checked.

This is called “Look Before You Leap” (LBYL) programming, and is
generally considered not Pythonic.

Rather, “it is Easier to Ask Forgiveness than Permission” (EAFP) is the
Python programming style, and fits with its widespread reliance on
polymorphism (including “duck typing”).

Accept the input, and use it as though it has the correct behaviour –
without regard to what type is providing that behaviour.

If it doesn't have the expected behaviour, either the type will complain
(raising an exception that you can handle at an appropriate level in
your code), or your comprehensive unit tests will detect the
misbehaviour.

If you don't have comprehensive unit tests, that's where you should put
your effort of strict interface testing. Not type assertions in the
application code.

 3. Let potential errors raise as early as possible.

That is good practice. But you should not compromise polymorphism to
that.

-- 
 \“The whole area of [treating source code as intellectual |
  `\property] is almost assuring a customer that you are not going |
_o__)   to do any innovation in the future.” —Gary Barnett |
Ben Finney
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Type checking versus polymorphism (was: list(), tuple() should not place at Built-in functions in documentation)

[Steven D'Aprano]

Ben Finney wrote:

[...snip explanation of duck-typing...]
 If you don't have comprehensive unit tests, that's where you should put
 your effort of strict interface testing. Not type assertions in the
 application code.

I agree with everything Ben said here, but he has missed something even more
fundamental.

Type *checking* breaks duck-typing. Type *assertions* are even worse,
because assertions aren't guaranteed to run. If you are using assert
isinstance(...) to validate input data, there are situations where your
validation step does not happen at all, and your code may just break in the
least convenient way. So not only are you validating the wrong way, but
sometimes you don't validate at all!

Assertions are for testing internal program logic, not for validation.

(I don't even like using assert for testing. How do you test your code with
assertions turned off if you use assert for testing?)



-- 
Steven

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Type checking versus polymorphism (was: list(), tuple() should not place at Built-in functions in documentation)

[Chris Rebert]

On Fri, Jul 15, 2011 at 7:47 PM, Steven D'Aprano
steve+comp.lang.pyt...@pearwood.info wrote:
snip
 Assertions are for testing internal program logic, not for validation.

 (I don't even like using assert for testing. How do you test your code with
 assertions turned off if you use assert for testing?)

I would think that would only matter if either the asserted
expressions caused side-effects or there was nontrivial logic in the
AssertionError handler, which would indicate a rather screwy codebase
and point to a possible PEBKAC issue that testing cannot hope to
remedy.

Cheers,
Chris
--
http://rebertia.com
-- 
http://mail.python.org/mailman/listinfo/python-list

[issue6241] Better type checking for the arguments of io.StringIO

[Alexandre Vassalotti]

Alexandre Vassalotti alexan...@peadrop.com added the comment:

Committed in r74157 (branches/py3k).

--
resolution:  - accepted
stage: patch review - committed/rejected
status: open - closed

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue6241
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6241] Better type checking for the arguments of io.StringIO

[Alexandre Vassalotti]

New submission from Alexandre Vassalotti alexan...@peadrop.com:

The included patch makes type checking of the arguments of StringIO
consistent between pyio and io.

 import io
 io.StringIO(bhello)
Traceback (most recent call last):
  ...
ValueError: initial_value must be str or None, not bytes
 io.StringIO(newline=b\n)
_io.StringIO object at 0x7f93d52953d0
 import _pyio as pyio
 pyio.StringIO(newline=b\n)
Traceback (most recent call last):
  ...
TypeError: illegal newline type: class 'bytes'

 io.StringIO([])
Traceback (most recent call last):
  ...
ValueError: initial_value must be str or None, not list
 pyio.StringIO([])
_pyio.StringIO object at 0x7f93d4942610

--
components: IO, Library (Lib)
files: typecheck_init_stringio.diff
keywords: patch
messages: 89104
nosy: alexandre.vassalotti
priority: normal
severity: normal
stage: patch review
status: open
title: Better type checking for the arguments of io.StringIO
type: behavior
versions: Python 3.1, Python 3.2
Added file: http://bugs.python.org/file14231/typecheck_init_stringio.diff

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue6241
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: duck-type-checking?

[Arnaud Delobelle]

Duck typing...

For a while I thought the word _duck_ was used in the sense of _dodge_.

-- 
Arnaud
--
http://mail.python.org/mailman/listinfo/python-list

Re: duck-type-checking?

[Steven D'Aprano]

On Sat, 15 Nov 2008 20:00:03 +1300, greg wrote:

 Joe Strout wrote:
 
 So, in this case, the simplest solution was to have the method that
 initially accepts and stores the data check to make sure that data
 satisfies the assumptions of the class.
 
 In hindsight, yes, but the trouble is that you can't tell ahead of time
 which of the gazillion places in the code that where you store things
 away in containers are likely to cause a problem later.
 
 I can't imagine myself writing code to check every argument to every
 method to guard against this sort of thing. 

Which is, of course, the weakness of dynamic typed languages like Python. 
With statically typed languages like Pascal and C, you can get the 
compiler to check that for you (often at compile time), but at the cost 
of a lot more effort up front. And with languages like Haskell, the type 
inference engine can do much of that type checking without you needing to 
make explicit type declarations.


 If you're willing to do
 that, it's up to you, but it's far from common practice in Python
 programming.

True. It's generally more efficient for the programmer's time to let the 
function or method fail where ever it happens to fail, rather than trying 
to get it to fail up front. But the cost of this is that sometimes it's 
*less* efficient for the programmer, because he has no idea where the 
offending object was injected into the code.

I wonder whether the best solution is to include all your type checks 
(isinstance or duck-typing) in the unit tests, so you can avoid paying 
the cost of those tests at runtime? If your code passes the unit tests, 
but fails with real data, then your unit tests aren't extensive enough.



-- 
Steven
--
http://mail.python.org/mailman/listinfo/python-list

Re: duck-type-checking?

[Paul McGuire]

On Nov 15, 2:50 am, Steven D'Aprano [EMAIL PROTECTED]
cybersource.com.au wrote:
 I wonder whether the best solution is to include all your type checks
 (isinstance or duck-typing) in the unit tests, so you can avoid paying
 the cost of those tests at runtime? If your code passes the unit tests,
 but fails with real data, then your unit tests aren't extensive enough.

 --
 Steven

The real penalties that you pay for static typing are not at compile
time, but at design time.  Because of duck-typing, Python programmers
can skip a lot of the extra hoops/cruft that Java and C++ developers
must jump through, usually to *get around* the restrictions of static
typing.  Imagine how much code is needed in those languages to
implement this simple generic Proxy class:

class Proxy(object):
def __init__(self,other):
self.obj = other
def __getattr__(self,attr):
print Get attribute '%s' % attr
return getattr(self.obj,attr)

x = slfj
xproxy = Proxy(x)

print xproxy.upper()

# prints:
# Get attribute 'upper'
# SLFJ

I used very similar code to write a CORBA interceptor for a client in
about 15 minutes, which would have taken *much* longer in C++ or Java.

-- Paul
--
http://mail.python.org/mailman/listinfo/python-list

Re: duck-type-checking?

[pruebauno]

On Nov 14, 12:47 am, George Sakkis [EMAIL PROTECTED] wrote:
 On Nov 13, 10:55 pm, Steven D'Aprano [EMAIL PROTECTED]

 cybersource.com.au wrote:
  Take this example:

  def foo(alist):
  alist.sort()
  alist.append(5)

  The argument can be any object with sort and append methods (assumed to
  act in place). But what happens if you pass it an object with a sort
  method but no append? The exception doesn't occur until *after* the
  object is sorted, which leaves it in an inconsistent state. This can be
  undesirable: you might need the function foo to be atomic, either the
  entire function succeeds, or none of it.

 In this example atomicity is not guaranteed even if alist is a builtin
 list (if it contains a complex number or other unorderable object),
 let alone if not isistance(alist, list). It gets worse: false
 positives are less likely for full-spelled methods with well-known
 names such as sort and append  (i.e. if hasattr(alist, 'append'),
 alist.append *probably* does what you think it does), but good luck
 with that when testing for __getitem__, __iter__ for more than one
 pass, __call__, and other special methods with ambiguous or undefined
 semantics.

 Let's face it, duck typing is great for small to medium complexity
 projects but it doesn't scale without additional support in the form
 of ABCs/interfaces, explicit type checking (at compile and/or run
 time), design by contract, etc. It must not be a coincidence that both
 Zope and Twisted had to come up with interfaces to manage their
 massive (for Python at least) complexity.

 George

What would be actually interesting would be an switch to the python
interpreter that internally annotated function parameters with how
they are used in the function and raised an exception as soon as the
function is called instead of later. Failing earlier rather than
later. Example:


def sub(x,y):
...run some stuff
...print x[2]
...return y.strip().replace('a','b')

internally python generates:

def sub(x: must have getitem, y: must have strip and replace)



sub([1,2,3,4],5)
Error calling sub(x,y): y has to have strip() method.

--
http://mail.python.org/mailman/listinfo/python-list

Re: duck-type-checking?

[Joe Strout]

On Nov 14, 2008, at 12:27 PM, [EMAIL PROTECTED] wrote:


What would be actually interesting would be an switch to the python
interpreter that internally annotated function parameters with how
they are used in the function and raised an exception as soon as the
function is called instead of later. Failing earlier rather than
later.


That would be interesting, but it wouldn't have helped in the case I  
had last week, where the method being called does little more than  
stuff the argument into a container inside the class -- only to blow  
up much later, when that data was accessed in a certain way.


The basic problem was that the data being stored was violating the  
assumptions of the class itself.  Sometimes in the past I've used a  
check invariants method on a class with complex data, and call this  
after mutating operations to ensure that all the class invariants are  
still true.  But this class wasn't really that complex; it's just that  
it assumed all the stuff it's being fed were strings (or could be  
treated as strings), and I inadvertently fed it an NLTK.Tree node  
instead (not realizing that a library method I was calling could  
return such a thing sometimes).


So, in this case, the simplest solution was to have the method that  
initially accepts and stores the data check to make sure that data  
satisfies the assumptions of the class.


Best,
- Joe

--
http://mail.python.org/mailman/listinfo/python-list

Re: duck-type-checking?

[Paul McGuire]

On Nov 14, 1:27 pm, [EMAIL PROTECTED] wrote:
 On Nov 14, 12:47 am, George Sakkis [EMAIL PROTECTED] wrote:





  On Nov 13, 10:55 pm, Steven D'Aprano [EMAIL PROTECTED]

  cybersource.com.au wrote:
   Take this example:

   def foo(alist):
       alist.sort()
       alist.append(5)

   The argument can be any object with sort and append methods (assumed to
   act in place). But what happens if you pass it an object with a sort
   method but no append? The exception doesn't occur until *after* the
   object is sorted, which leaves it in an inconsistent state. This can be
   undesirable: you might need the function foo to be atomic, either the
   entire function succeeds, or none of it.

  In this example atomicity is not guaranteed even if alist is a builtin
  list (if it contains a complex number or other unorderable object),
  let alone if not isistance(alist, list). It gets worse: false
  positives are less likely for full-spelled methods with well-known
  names such as sort and append  (i.e. if hasattr(alist, 'append'),
  alist.append *probably* does what you think it does), but good luck
  with that when testing for __getitem__, __iter__ for more than one
  pass, __call__, and other special methods with ambiguous or undefined
  semantics.

  Let's face it, duck typing is great for small to medium complexity
  projects but it doesn't scale without additional support in the form
  of ABCs/interfaces, explicit type checking (at compile and/or run
  time), design by contract, etc. It must not be a coincidence that both
  Zope and Twisted had to come up with interfaces to manage their
  massive (for Python at least) complexity.

  George

 What would be actually interesting would be an switch to the python
 interpreter that internally annotated function parameters with how
 they are used in the function and raised an exception as soon as the
 function is called instead of later. Failing earlier rather than
 later. Example:

 def sub(x,y):
 ...run some stuff
 ...print x[2]
 ...return y.strip().replace('a','b')

 internally python generates:

 def sub(x: must have getitem, y: must have strip and replace)

 sub([1,2,3,4],5)
 Error calling sub(x,y): y has to have strip() method.- Hide quoted text -

 - Show quoted text -

No, this would mean:
def sub(x: must have getitem, y: must have strip, and y.strip must
return something that has replace)

Or to be even more thorough:
def sub(x: must have getitem, y: must have strip and strip must be
callable, and y.strip must return something that has replace and
replace must be callable)

So even this simple example gets nasty in a hurry, let alone the OP's
case where he stuffs y into a list in order to access it much later,
in a completely different chunk of code, only to find out that y
doesn't support the complete string interface as he expected.

-- Paul
--
http://mail.python.org/mailman/listinfo/python-list

Re: duck-type-checking?

[Joe Strout]

On Nov 14, 2008, at 2:07 PM, Paul McGuire wrote:


Or to be even more thorough:
def sub(x: must have getitem, y: must have strip and strip must be
callable, and y.strip must return something that has replace and
replace must be callable)

So even this simple example gets nasty in a hurry, let alone the OP's
case where he stuffs y into a list in order to access it much later,
in a completely different chunk of code, only to find out that y
doesn't support the complete string interface as he expected.


Very true.  That's why I think it's not worth trying to be too pure  
about it.  Most of the time, if a method wants a Duck, you're going to  
just give it a Duck.


However, I would like to also handle the occasional case where I can't  
give it a Duck, but I can give it something that is a drop-in  
substitute for a Duck (really, truly, I promise, and if it blows up  
I'll take responsibility for it).


A real-world example from another language (sorry for that, I've been  
away from Python for ten years): in REALbasic, there is a Database  
base class, and a subclass for each particular database backend  
(Postgres, MySQL, whatever).  This works fine most of the time, in  
that you can write general code that takes a Database object and Does  
Stuff with it.


However, all of those database backends are shipped by the vendor, or  
by plugin authors -- you can't create a useful Database subclass  
yourself, in RB code, because it has a private constructor.  So you  
end up making your own database class, but that can't be used with all  
the code that expects a real Database object.


Of course, the framework design there is seriously flawed (Database  
should have been an interface, or at the very least, had a protected  
rather than private constructor).  And in Python, there's no way to  
prevent subclassing AFAIK, so this particular issue wouldn't come up.   
But I still suspect that there may be times when I don't want to  
subclass for some reason (maybe I'm using the Decorator or Adapter or  
Bridge pattern).  Yet I'm willing to guarantee that I've adhered to  
the interface of another class, and will behave like it in any way  
that matters.


So, the level of assertion that I want to make in a method that  
expects a Duck is just that its parameter is either a Duck, or  
something that the caller is claiming is just as good as a Duck.  I'm  
not trying to prevent any possible error; I'm trying to catch the  
stupid errors where I inadvertently pass in something completely  
different, not duck-like at all (probably because some other method  
gave me a result I didn't realize it could produce).


So things like this should suffice:

# simple element
assert(is_stringlike(foo))
assert(is_numeric(foo))
assert(is_like(foo, Duck))

# sequence of elements
assert(seqof_stringlike(foo))
assert(seqof_numeric(foo))
assert(seqof_like(foo, Duck))
# (also listof_ variants for asserting mutable sequence of whatever)

# dictionary of elements
assert(dictof_like(foo, str, int))

Hmm, I was already forced to change my approach by the time I got to  
checking dictionaries.  Perhaps a better formalism would be a like  
method that takes an argument, and something that indicates the  
desired type.  This could be a tree if you want to check deeper into a  
container.  Maybe something like:


assert(fits(foo, dictlike(strlike, seqlike(intlike

which asserts that foo is something dictionary-like that maps string- 
like things to something like a sequence of integer-like things.  Most  
cases would not be this complex, of course, but would be closer to


assert(fits(foo, strlike))

But this is still pretty ugly.  Hmm.  Maybe I'd better wait for  
ABCs.  :)


Cheers,
- Joe

--
http://mail.python.org/mailman/listinfo/python-list

Re: duck-type-checking?

[George Sakkis]

On Nov 14, 4:49 pm, Joe Strout [EMAIL PROTECTED] wrote:

 So things like this should suffice:

         # simple element
         assert(is_stringlike(foo))
         assert(is_numeric(foo))
         assert(is_like(foo, Duck))

         # sequence of elements
         assert(seqof_stringlike(foo))
         assert(seqof_numeric(foo))
         assert(seqof_like(foo, Duck))
         # (also listof_ variants for asserting mutable sequence of whatever)

         # dictionary of elements
         assert(dictof_like(foo, str, int))

 Hmm, I was already forced to change my approach by the time I got to  
 checking dictionaries.  Perhaps a better formalism would be a like  
 method that takes an argument, and something that indicates the  
 desired type.  This could be a tree if you want to check deeper into a  
 container.  Maybe something like:

         assert(fits(foo, dictlike(strlike, seqlike(intlike

 which asserts that foo is something dictionary-like that maps string-
 like things to something like a sequence of integer-like things.  Most  
 cases would not be this complex, of course, but would be closer to

         assert(fits(foo, strlike))

 But this is still pretty ugly.  Hmm.  Maybe I'd better wait for  
 ABCs.  :)

You might also be interested in the typecheck module whose syntax
looks nicer, at least for the common cases: 
http://oakwinter.com/code/typecheck/dev/

George
--
http://mail.python.org/mailman/listinfo/python-list

Re: duck-type-checking?

[Steve Holden]

Joe Strout wrote:
 On Nov 14, 2008, at 2:07 PM, Paul McGuire wrote:
 
 Or to be even more thorough:
 def sub(x: must have getitem, y: must have strip and strip must be
 callable, and y.strip must return something that has replace and
 replace must be callable)

 So even this simple example gets nasty in a hurry, let alone the OP's
 case where he stuffs y into a list in order to access it much later,
 in a completely different chunk of code, only to find out that y
 doesn't support the complete string interface as he expected.
 
 Very true.  That's why I think it's not worth trying to be too pure
 about it.  Most of the time, if a method wants a Duck, you're going to
 just give it a Duck.
 
 However, I would like to also handle the occasional case where I can't
 give it a Duck, but I can give it something that is a drop-in substitute
 for a Duck (really, truly, I promise, and if it blows up I'll take
 responsibility for it).
 [...]

I suspect this reduces our difference to a disagreement about the
meaning of the word occasional ...

regards
 Steve
-- 
Steve Holden+1 571 484 6266   +1 800 494 3119
Holden Web LLC  http://www.holdenweb.com/

--
http://mail.python.org/mailman/listinfo/python-list

Re: duck-type-checking?

[Steven D'Aprano]

On Fri, 14 Nov 2008 13:28:27 -0700, Joe Strout wrote:

 But this class wasn't really that complex; it's just that it assumed all
 the stuff it's being fed were strings (or could be treated as strings),
 and I inadvertently fed it an NLTK.Tree node instead (not realizing that
 a library method I was calling could return such a thing sometimes).

Guido has published a couple of metaclasses to get Eiffel-style pre- and 
post-condition tests that may be useful for you:

http://www.python.org/doc/essays/metaclasses/

If you're interested in reading more about metaclasses, this is more 
current:

http://www.python.org/download/releases/2.2.3/descrintro/


By the way, even Guido himself isn't immune to the tendency among Python 
users to flame at anyone suggesting change to Python's model:

http://www.artima.com/weblogs/viewpost.jsp?thread=87182

(And that's a good thing. It would be really bad if the Python community 
were slavishly and mindlessly Guido-fanboys as we're sometimes accused of 
being.)



-- 
Steven
--
http://mail.python.org/mailman/listinfo/python-list

Re: duck-type-checking?

[greg]

Joe Strout wrote:

So, in this case, the simplest solution was to have the method that  
initially accepts and stores the data check to make sure that data  
satisfies the assumptions of the class.


In hindsight, yes, but the trouble is that you can't
tell ahead of time which of the gazillion places in the
code that where you store things away in containers are
likely to cause a problem later.

I can't imagine myself writing code to check every
argument to every method to guard against this sort of
thing. If you're willing to do that, it's up to you,
but it's far from common practice in Python programming.

--
Greg

--
http://mail.python.org/mailman/listinfo/python-list

Re: duck-type-checking?

[greg]

Joe Strout wrote:

So, the level of assertion that I want to make in a method that  expects 
a Duck is just that its parameter is either a Duck, or  something that 
the caller is claiming is just as good as a Duck.


I'm not sure, but I think the new ABC stuff in Py3 is
going to provide something like this, in that there will
be a way to declare that a class conforms to the Duck
interface even if it doesn't inherit from it. Then
you can just test isinstance(x, Duck).

--
Greg
--
http://mail.python.org/mailman/listinfo/python-list

Re: duck-type-checking?

[Ben Finney]

Joe Strout [EMAIL PROTECTED] writes:

 Because if I write a method with the intention of treating the
 arguments like strings in various ways (slicing, combining with
 other strings, printing to stdout or writing to a file, etc. etc.),
 and some idiot (i.e. me six weeks later or long after I should have
 gone to bed) manages to accidentally pass in something else, then I
 want my program to blow up right away, not plant a roadside bomb and
 cheerfully wait for me to drive by.

This is better achived, not by littering the functional code unit with
numerous assertions that obscure the normal function of the code, but
rather by employing comprehensive unit tests *separate from* the code
unit.

This suite of unit tests is then employed and executed any time the
code changes, and indivates any regressions in functionality from what
is asserted in the tests. Meanwhile, the code unit itself is clear and
free from the masses of error checking that one often finds in code
written without such unit test suites.

 This is not hypothetical -- just last week I had a
 hard-to-track-down abend that ultimately turned out to be an
 NLTK.Tree object stored someplace that I expected to only contain
 strings. I found it by littering my code with assertions of the form
 isinstance(foo,basestring). If I'd had those in there in the first
 place, not only documenting my assumptions but letting the computer
 check them for me, it would have saved me a lot of grief.

Rather than littering these assertions *within* the code unit, and
thereby making that code harder to read later, those assertions would
have been better placed in a unit test module that tests the behaviour
of the unit in response to environment and input.

This has the additional benefit of highlighting parts of one's code
that do not have well-defined interfaces: if the code's behaviour in
response to a simple, narrow specification of inputs and environment
is not easy to describe in a simple true-or-false testable assertion,
then the interface and/or the internals of the code unit is very
likely too complex to be maintained well.

 But in the spirit of duck-typing, I shouldn't actually check that
 foo is a basestring. I should instead check that foo quacks like a
 basestring. I'd define that is:
 
 x quacks like a basestring if it implements all the public methods
 of basestring, and can be used in pretty much any context that a
 basestring can.

That is not duck typing. Rather than checking what foo does in
response to prodding that, by your admission, is only designed to find
out what type it is, duck typing instead advocates that you should use
foo *as though it is known to be* the type of object you want. If it
is not suitable, then appropriate exceptions will be raised and either
caught by some code that knows how to handle them, or crash the
program.

Unit tests then come into play by feeding inputs to your code unit and
asserting that the code behaves appropriately whether fed correct or
incorrect inputs: correct inputs, even of wholly unanticipated types,
should cause correct behaviour (i.e. if the code is designed to handle
ducks it should be content with a goose also); and incorrect inputs
are incorrect by definition *only if* they cause the code unit to be
unable to behave correctly (in which case an appropriate exception
should be raised).

-- 
 \ “Pinky, are you pondering what I'm pondering?” “I think so, |
  `\Brain, but Zero Mostel times anything will still give you Zero |
_o__)  Mostel.” —_Pinky and The Brain_ |
Ben Finney
--
http://mail.python.org/mailman/listinfo/python-list

Re: duck-type-checking?

[paul]

Ben Finney schrieb:

Joe Strout [EMAIL PROTECTED] writes:

x quacks like a basestring if it implements all the public methods
of basestring, and can be used in pretty much any context that a
basestring can.


That is not duck typing. Rather than checking what foo does in
response to prodding that, by your admission, is only designed to find
out what type it is, duck typing instead advocates that you should use
foo *as though it is known to be* the type of object you want. If it
is not suitable, then appropriate exceptions will be raised and either
caught by some code that knows how to handle them, or crash the
program.


Warning, rant ;)

This whole theory breaks down quickly if you're writing library code. 
How do your unittests help the user of your library to use it correctly? 
How do you communicate incorrect usage of your interfaces to the user?


If you are able to specify the type of the arguments as part of the 
interface the compiler/interpreter will help you. Types are used to 
describe behaviour (if thats a good thing, I don't know). While python 
has strong types, there could be used better (instead it gets worse, see 
the suddently-not-sortable-list-type diskussion and the endless 
repetition of the greatest of all after-the-fact theories ever duck 
typing.


cheers
 Paul

BTW: Back to Java? No, not really.


--
http://mail.python.org/mailman/listinfo/python-list

Re: duck-type-checking?

[Tim Rowe]

2008/11/13 Ben Finney [EMAIL PROTECTED]:

 That is not duck typing.

Oh, I'm pretty sure it is. It just isn't /using/ the duck typing in
the way you'd like.

-- 
Tim Rowe
--
http://mail.python.org/mailman/listinfo/python-list

Re: duck-type-checking?

[Joe Strout]

On Nov 12, 2008, at 7:32 PM, Steven D'Aprano wrote:


I'm surprised nobody has pointed you at Alex Martelli's recipe here:

http://code.activestate.com/recipes/52291/


Thanks for that -- it's clever how he combines binding the methods  
he'll use with doing the checking.



While the recipe is great, it can be tiresome to apply all the time. I
would factor out the checks into a function, something like this:

def isstringlike(obj, methods=None):
   Return True if obj is sufficiently string-like.
   if isinstance(obj, basestring):
   return True
   if methods is None:
   methods = ['upper', 'lower', '__len__', '__getitem__']
   for method in methods:
   if not hasattr(obj, method):
   return False
   # To really be string-like, the following test should pass.
   if len(obj)  0:
   s = obj[0]
   if s[0] != s:
   return False
   return True


Thanks for this, too; that's the sort of method I had in mind.  That  
last test for string-likeness is particularly clever.  I'll need to  
think more deeply about the implications.


Best,
- Joe

--
http://mail.python.org/mailman/listinfo/python-list

Re: duck-type-checking?

[Steve Holden]

greg wrote:
 Joe Strout wrote:
 This is not hypothetical -- just last week I had a hard-to-track-down 
 abend that ultimately turned out to be an NLTK.Tree object stored 
 someplace that I expected to only contain strings.  I found it by 
 littering my code with assertions of the form 
 isinstance(foo,basestring).
 
 But you have to ask yourself whether the time taken to
 write and maintain all these assertions is really
 cost-effective. In my experience, occurrences like this
 are actually extremely rare -- the vast majority of the
 time, you do get a failure very quickly, and it's fairly
 obvious from the traceback where to look for the problem.
 
 It's as annoying as hell when something like this does
 happen, but averaged over the lifetime of the project,
 I find it doesn't cost all that much time.
 
This is particularly true since Joe is still proposing to check the
objects when they are passed to functions and methods that use them.
Unfortunately the assignment of the wrong type of object may well have
taken place aeons ago, and it's the assignments you really need to catch
(and it's also the assignments that static languages cause compile
errors on).

So it hardly matters whether the code blows up with

  Exception: my expensive type checks sounded an alarm

or

  Attribute error: foo has no method bar()

since both are equally informative when it comes to tracing the faulty
assignment.

regards
 Steve
-- 
Steve Holden+1 571 484 6266   +1 800 494 3119
Holden Web LLC  http://www.holdenweb.com/

--
http://mail.python.org/mailman/listinfo/python-list

Re: duck-type-checking?

[Craig Allen]

 This is better achived, not by littering the functional code unit with
 numerous assertions that obscure the normal function of the code, but
 rather by employing comprehensive unit tests *separate from* the code
 unit.

that doesn't seem to work too well when shipping a library for someone
else to use... we don't have access to the caller's code that needs to
be checked.  I suppose if the intent is to have a true assert, that
does nothing in shipped code, then you can argue that testing
addresses some of the issues, but one, not all of them, specifically,
not the part where the problem is ably reported, and two, I don't
think we can assume assert meant that sort of assert macro in C which
compiles away in release versions.

Asserts also do not litter code, they communicate the assumptions of
the code.  I like the idea of a general duck-type assertion and would
probably use that, especially since I also have arguments which can be
multiple objects, each with their own interface but similar meaning...
i.e. lower level file objects can be passed in, or my higher level
abstraction of the same file.
--
http://mail.python.org/mailman/listinfo/python-list

Re: duck-type-checking?

[Craig Allen]

 since both are equally informative when it comes to tracing the faulty
 assignment.


steve, they are not equally informative, the assertion is designed to
fire earlier in the process, and therefore before much mischief and
corruption can be done compared to later, when you happen to hit the
missing attribute.
--
http://mail.python.org/mailman/listinfo/python-list