Re: Should NoneType be iterable?

[Greg Ewing via Python-list]

I would question the wisdom of designing an API that
can return either a sequence or None. If it normally
returns a sequence, and there are no items to return,
it should return an empty sequence.

--
Greg

--
https://mail.python.org/mailman/listinfo/python-list

Attaching a mock function to another mock breaks reset_mock()

[Mark Bourne via Python-list]

I've came across an issue with attaching a mock function to another mock 
object.  It looks like this might be a bug in unittest.mock, but it's 
possible I'm misunderstanding or doing something wrong.


I'm currently using Python 3.8.10, which is the default installed on 
Ubuntu 20.04.  I don't have time to install and try other versions right 
now but from a quick look at the source at 
, it 
looks like the same issue would still occur with the latest.  I have a 
workaround, but would still like to know if I'm doing something wrong 
which I can correct, or should report this as a bug.


The class I'm testing (let's call it A) is given an instance of another 
class (let's call it B).  In normal operation, some other code adds a 
callback function as an attribute to B before passing it to A, and A 
calls that callback at certain points.  I agree this isn't particularly 
nice; I didn't write the code and don't have time to sort out all the 
structural issues at the moment.


So, in setting up the mock B, I need to attach a mock callback function 
to it.  I'd like the mock function to raise an exception if it's called 
with the wrong arguments.  I can create such a mock function with, for 
example:

```
mock_callback = mock.create_autospec(lambda a, b, c: None)
```
Calls like mock_callback(1,2,3) or mock_callback(a=1,b=2,c=3) are fine, 
and the test can later check the exact values passed in, while 
mock_callback(1,2) or mock_callback(1,2,3,x=9) raise an exception at the 
time of the call - exactly what I want.


However, when I attach this mock callback to the mock instance of B, the 
reset_mock() method breaks.  For example, using object in place of B, 
since the rest of its implementation is irrelevant to the issue:

```
mock_callback = mock.create_autospec(lambda a, b, c: None)
mock_b = mock.create_autospec(object, spec_set=False, instance=True)
mock_b.attach_mock(mock_callback, 'some_callback')
mock_b.reset_mock()
```

The call to reset_mock() results in:
```
Traceback (most recent call last):
  File "", line 1, in 
  File "/usr/lib/python3.8/unittest/mock.py", line 603, in reset_mock
child.reset_mock(visited)
TypeError: reset_mock() takes 0 positional arguments but 1 was given
```

This seems to occur because, when creating a mock of a function or 
method, create_autospec calls _setup_func (via _set_signature), which 
sets the mock callback's reset_mock method to a function which doesn't 
accept any arguments.  When mock_b.reset_mock() is called, that 
recursively calls reset_mock() on all the child mocks (including the 
mock callback), passing a number of arguments - which causes the above 
exception.


I thought I might be able to just assign the mock callback to an 
attribute of the mock B, rather than using attach_mock, and avoid the 
recursive call to its reset_mock():

```
mock_b.some_callback = mock_callback
```
But mock_b.reset_mock() still raises that exception.  I think some magic 
in the mocks automatically attaches mock_callback as a child of mock_b 
even in that case.


My current workaround is to just use
```
mock_callback = mock.Mock(spec_set=lambda: None)
```
instead.  While using spec_set there prevents incorrect use of things 
like mock_b.some_callback.random_attribute, it doesn't enforce the 
arguments that the function can be called with, even if I do include 
them in the lambda (as in the first example).


Is there something I'm doing wrong here?  Or does this seem like a bug 
in unittest.mock that I should report?  Perhaps this is something that's 
not often done, so the issue hasn't been noticed before.  Trying to 
search for information generally leads back to the unittest.mock 
documentation, and general tutorials on using create_autospec, 
attach_mock, etc. without anything specific about this case.


--
Mark.

--
https://mail.python.org/mailman/listinfo/python-list

Re: Bug in io.TextIOWrapper?

[Jon Ribbens via Python-list]

On 2023-06-19, Inada Naoki  wrote:
> I checked TextIOWrapper source code and confirmed that it doesn't call
> encoder.write(text, finish=True) on close.
> Since TextIOWrapper allows random access, it is difficult to call it
> automatically. So please think it as just limitation rather than bug.
> Please use codec and binary file manually for now.

It could call it on seek() or flush(). It seems like a definite bug to
me, in that its behaviour appears clearly incorrect - it's just that
there isn't an entirely obvious "100% correct" behaviour to choose.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Should NoneType be iterable?

[dn via Python-list]

On 20/06/2023 06.12, Neal Becker via Python-list wrote:

On Mon, Jun 19, 2023 at 12:42 PM Chris Angelico via Python-list <
python-list@python.org> wrote:


On Tue, 20 Jun 2023 at 02:37, Peter Bona via Python-list
 wrote:


Hi

I am wondering if there has been any discussion why NoneType  is not

iterable My feeling is that it should be.

Sometimes I am using API calls which return None.
If there is a return value (which is iterable) I am using a for loop to

iterate.


Now I am getting 'TypeError: 'NoneType' object is not iterable'.

(Examples are taken from here

https://rollbar.com/blog/python-typeerror-nonetype-object-is-not-iterable/
)

Example 1:
mylist = None
for x in mylist:
 print(x)  <== will raise TypeError: 'NoneType' object is not iterable
Solution: extra If statement
if mylist is not None:
 for x in mylist:
 print(x)


I think Python should handle this case gracefully: if a code would

iterate over None: it should not run any step. but proceed the next
statement.


Has this been discussed or proposed?



Try this instead:

for x in mylist or ():

Now a None list will skip iteration entirely, allowing you to get the
effect you want :)

ChrisA
--
https://mail.python.org/mailman/listinfo/python-list


I prefer iteration of None to be an error, as in my usage it usually
indicates a mistake that I'd want to catch


Agreed!

A better approach is that the API return (perhaps a tuple of) both 
"status" and "return_value", rather than overloading the latter.


That said, apparently the OP use-case is for when there is no interest 
in status/catch, eg where a 'nothing' answer is THE answer.


--
Regards,
=dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: Should NoneType be iterable?

[Neal Becker via Python-list]

On Mon, Jun 19, 2023 at 12:42 PM Chris Angelico via Python-list <
python-list@python.org> wrote:

> On Tue, 20 Jun 2023 at 02:37, Peter Bona via Python-list
>  wrote:
> >
> > Hi
> >
> > I am wondering if there has been any discussion why NoneType  is not
> iterable My feeling is that it should be.
> > Sometimes I am using API calls which return None.
> > If there is a return value (which is iterable) I am using a for loop to
> iterate.
> >
> > Now I am getting 'TypeError: 'NoneType' object is not iterable'.
> >
> > (Examples are taken from here
> https://rollbar.com/blog/python-typeerror-nonetype-object-is-not-iterable/
> )
> > Example 1:
> > mylist = None
> > for x in mylist:
> > print(x)  <== will raise TypeError: 'NoneType' object is not iterable
> > Solution: extra If statement
> > if mylist is not None:
> > for x in mylist:
> > print(x)
> >
> >
> > I think Python should handle this case gracefully: if a code would
> iterate over None: it should not run any step. but proceed the next
> statement.
> >
> > Has this been discussed or proposed?
> >
>
> Try this instead:
>
> for x in mylist or ():
>
> Now a None list will skip iteration entirely, allowing you to get the
> effect you want :)
>
> ChrisA
> --
> https://mail.python.org/mailman/listinfo/python-list
>
I prefer iteration of None to be an error, as in my usage it usually
indicates a mistake that I'd want to catch
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Bug in io.TextIOWrapper?

[Inada Naoki via Python-list]

I checked TextIOWrapper source code and confirmed that it doesn't call
encoder.write(text, finish=True) on close.
Since TextIOWrapper allows random access, it is difficult to call it
automatically. So please think it as just limitation rather than bug.
Please use codec and binary file manually for now.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Bug in io.TextIOWrapper?

[Inada Naoki via Python-list]

You can use file instead of BytesIO

2023年6月20日(火) 3:05 Peter J. Holzer via Python-list :

> On 2023-06-20 02:15:00 +0900, Inada Naoki via Python-list wrote:
> > stream.flush() doesn't mean final output.
> > Try stream.close()
>
> After close() the value isn't available any more:
>
> Python 3.11.2 (main, Mar 13 2023, 12:18:29) [GCC 12.2.0] on linux
> Type "help", "copyright", "credits" or "license" for more information.
> >>> import io
> >>> buffer = io.BytesIO()
> >>> stream = io.TextIOWrapper(buffer, encoding='idna')
> >>> stream.write('abc.example.com')
> 15
> >>> stream.close()
> >>> buffer.getvalue()
> Traceback (most recent call last):
>   File "", line 1, in 
> ValueError: I/O operation on closed file.
>
> hp
>
> --
>_  | Peter J. Holzer| Story must make more sense than reality.
> |_|_) ||
> | |   | h...@hjp.at |-- Charles Stross, "Creative writing
> __/   | http://www.hjp.at/ |   challenge!"
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Bug in io.TextIOWrapper?

[Peter J. Holzer via Python-list]

On 2023-06-20 02:15:00 +0900, Inada Naoki via Python-list wrote:
> stream.flush() doesn't mean final output.
> Try stream.close()

After close() the value isn't available any more:

Python 3.11.2 (main, Mar 13 2023, 12:18:29) [GCC 12.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import io
>>> buffer = io.BytesIO()
>>> stream = io.TextIOWrapper(buffer, encoding='idna')
>>> stream.write('abc.example.com')
15
>>> stream.close()
>>> buffer.getvalue()
Traceback (most recent call last):
  File "", line 1, in 
ValueError: I/O operation on closed file.

hp

-- 
   _  | Peter J. Holzer| Story must make more sense than reality.
|_|_) ||
| |   | h...@hjp.at |-- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |   challenge!"


signature.asc
Description: PGP signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Bug in io.TextIOWrapper?

[Inada Naoki via Python-list]

stream.flush() doesn't mean final output.
Try stream.close()

2023年6月20日(火) 1:40 Jon Ribbens via Python-list :

> io.TextIOWrapper() wraps a binary stream so you can write text to it.
> It takes an 'encoding' parameter, which it uses to look up the codec
> in the codecs registry, and then it uses the IncrementalEncoder and
> IncrementalDecoder classes for the appropriate codec.
>
> The IncrementalEncoder.encode() function is given the object to encode
> of course, and also an optional second parameter which indicates if
> this is the final output.
>
> The bug is that TextIOWrapper() never sets the second parameter to
> indicate that the output is complete - not even if you call close().
>
> Example:
>
> >>> import io
> >>> buffer = io.BytesIO()
> >>> stream = io.TextIOWrapper(buffer, encoding='idna')
> >>> stream.write('abc.example.com')
> 15
> >>> stream.flush()
> >>> buffer.getvalue()
> b'abc.example.'
>
> Obviously using the 'idna' wrapper as an encoding on a stream is a bit
> unlikely, but nevertheless any other codec which cares about the 'final'
> parameter will also have this problem.
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Bug in io.TextIOWrapper?

[Jon Ribbens via Python-list]

io.TextIOWrapper() wraps a binary stream so you can write text to it.
It takes an 'encoding' parameter, which it uses to look up the codec
in the codecs registry, and then it uses the IncrementalEncoder and
IncrementalDecoder classes for the appropriate codec.

The IncrementalEncoder.encode() function is given the object to encode
of course, and also an optional second parameter which indicates if
this is the final output.

The bug is that TextIOWrapper() never sets the second parameter to
indicate that the output is complete - not even if you call close().

Example:

>>> import io
>>> buffer = io.BytesIO()
>>> stream = io.TextIOWrapper(buffer, encoding='idna')
>>> stream.write('abc.example.com')
15
>>> stream.flush()
>>> buffer.getvalue()
b'abc.example.'

Obviously using the 'idna' wrapper as an encoding on a stream is a bit
unlikely, but nevertheless any other codec which cares about the 'final'
parameter will also have this problem.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Should NoneType be iterable?

[Chris Angelico via Python-list]

On Tue, 20 Jun 2023 at 02:37, Peter Bona via Python-list
 wrote:
>
> Hi
>
> I am wondering if there has been any discussion why NoneType  is not iterable 
> My feeling is that it should be.
> Sometimes I am using API calls which return None.
> If there is a return value (which is iterable) I am using a for loop to 
> iterate.
>
> Now I am getting 'TypeError: 'NoneType' object is not iterable'.
>
> (Examples are taken from here 
> https://rollbar.com/blog/python-typeerror-nonetype-object-is-not-iterable/)
> Example 1:
> mylist = None
> for x in mylist:
> print(x)  <== will raise TypeError: 'NoneType' object is not iterable
> Solution: extra If statement
> if mylist is not None:
> for x in mylist:
> print(x)
>
>
> I think Python should handle this case gracefully: if a code would iterate 
> over None: it should not run any step. but proceed the next statement.
>
> Has this been discussed or proposed?
>

Try this instead:

for x in mylist or ():

Now a None list will skip iteration entirely, allowing you to get the
effect you want :)

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Should NoneType be iterable?

[Peter Bona via Python-list]

Hi

I am wondering if there has been any discussion why NoneType  is not iterable 
My feeling is that it should be.
Sometimes I am using API calls which return None.
If there is a return value (which is iterable) I am using a for loop to iterate.

Now I am getting 'TypeError: 'NoneType' object is not iterable'.

(Examples are taken from here 
https://rollbar.com/blog/python-typeerror-nonetype-object-is-not-iterable/)
Example 1:
mylist = None
for x in mylist:
print(x)  <== will raise TypeError: 'NoneType' object is not iterable
Solution: extra If statement
if mylist is not None:
for x in mylist:
print(x)


I think Python should handle this case gracefully: if a code would iterate over 
None: it should not run any step. but proceed the next statement.

Has this been discussed or proposed?

Thanks
Peter


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: File system path annotations

[Thomas Passin via Python-list]

On 6/19/2023 10:04 AM, Roel Schroeven via Python-list wrote:

Op 19/06/2023 om 11:44 schreef Peter Slížik:

Thank you, Roel. You've answered all my questions.

> [PEP 519]: ...as that can be represented with typing.Union[str, 
bytes, os.PathLike] easily enough and the hope is users

> will slowly gravitate to path objects only.

I read a lot on Python and, frankly, I don't see this happening. 
People on the Internet keep using /str/ as their path representation 
choice. Presumably, programmers don't feel the need to bother with a 
complex solution if the simplest option works just fine.
I agree, I don't see that happening either. I often find myself using 
str for paths: often simple filenames are all that's needed, and then I 
don't see much point in importing pathlib and wrapping the filenames in 
Path() constructors. I do tend to switch to path objects when I start 
needing to do operations on the paths, though.



If you are writing code that will run on both Windows and Linux/Mac, 
pathlib objects are very useful since the path separators will come out 
right with no extra effort. Also, the syntax for composing a path seems 
very natural (e.g., Path('a') / 'b' / 'c'), so that's a bonus.


--
https://mail.python.org/mailman/listinfo/python-list

Re: File system path annotations

[Roel Schroeven via Python-list]

Op 19/06/2023 om 11:44 schreef Peter Slížik:

Thank you, Roel. You've answered all my questions.

> [PEP 519]: ...as that can be represented with typing.Union[str, 
bytes, os.PathLike] easily enough and the hope is users

> will slowly gravitate to path objects only.

I read a lot on Python and, frankly, I don't see this happening. 
People on the Internet keep using /str/ as their path representation 
choice. Presumably, programmers don't feel the need to bother with a 
complex solution if the simplest option works just fine.
I agree, I don't see that happening either. I often find myself using 
str for paths: often simple filenames are all that's needed, and then I 
don't see much point in importing pathlib and wrapping the filenames in 
Path() constructors. I do tend to switch to path objects when I start 
needing to do operations on the paths, though.


--
"Your scientists were so preoccupied with whether they could, they didn't
stop to think if they should"
-- Dr. Ian Malcolm
--
https://mail.python.org/mailman/listinfo/python-list

Re: File system path annotations

[Peter Slížik via Python-list]

Thank you, Roel. You've answered all my questions.

> [PEP 519]: ...as that can be represented with typing.Union[str, bytes,
os.PathLike] easily enough and the hope is users
> will slowly gravitate to path objects only.

I read a lot on Python and, frankly, I don't see this happening. People on
the Internet keep using *str* as their path representation choice.
Presumably, programmers don't feel the need to bother with a complex
solution if the simplest option works just fine.

Peter
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: File system path annotations

[Roel Schroeven via Python-list]

Op 19/06/2023 om 10:43 schreef Peter Slížik via Python-list:

Hello,

what is the preferred way of annotating file system paths?

This StackOverflow answer 
(and a few others) recommend using the

str | os.PathLike

union.

However, byte arrays can be used as paths too, and including them
would make the annotation quite long.

You don't have to type out the whole thing every time. Define it somewhere:

PathLike = str | bytes | os.PathLike

and then you can use PathLike everywhere.

I also believe that str confirms to the PathLike definition. Please,
correct me if I'm wrong.
I don't think so: os.PathLike instances must have a __fspath__() method, 
which str and bytes don't have.

And finally - using paths in Python programs is so common, that one
would expect to have a special type (or type alias) in typing. Am I
missing something?
I agree there should be something like that. This StackOverflow answer 
(https://stackoverflow.com/a/68027757/59122) talks about something like 
that: _typeshed.AnyPath. It is used in the standard library, but 
apparently it doesn't exist at runtime and you need a trick to make it 
work. I would expect something better to exist, but as far as I can find 
out there isn't.


PEP 519 has a little section on the topic 
(https://peps.python.org/pep-0519/#provide-specific-type-hinting-support):


> Provide specific type hinting support
>
> There was some consideration to providing a generic typing.PathLike 
class which would allow for e.g. typing.PathLike[str] to specify a type 
hint for a path object which returned a string representation.
> While potentially beneficial, the usefulness was deemed too small to 
bother adding the type hint class.

>
> This also removed any desire to have a class in the typing module 
which represented the union of all acceptable path-representing types as 
that can be represented with typing.Union[str, bytes,
> os.PathLike] easily enough and the hope is users will slowly 
gravitate to path objects only.


--

"This planet has - or rather had - a problem, which was this: most of the
people living on it were unhappy for pretty much of the time. Many solutions
were suggested for this problem, but most of these were largely concerned with
the movement of small green pieces of paper, which was odd because on the whole
it wasn't the small green pieces of paper that were unhappy."
-- Douglas Adams

--
https://mail.python.org/mailman/listinfo/python-list

File system path annotations

[Peter Slížik via Python-list]

Hello,

what is the preferred way of annotating file system paths?

This StackOverflow answer 
(and a few others) recommend using the

str | os.PathLike

union.

However, byte arrays can be used as paths too, and including them
would make the annotation quite long.

I also believe that str confirms to the PathLike definition. Please,
correct me if I'm wrong.

And finally - using paths in Python programs is so common, that one
would expect to have a special type (or type alias) in typing. Am I
missing something?

My apologies if I'm asking the obvious, but after some googling I came
to the conclusion that information on this topic is surprisingly
limited to a few StackOverflow questions.

Best regards,

Peter
-- 
https://mail.python.org/mailman/listinfo/python-list