[issue32052] Provide access to buffer of asyncio.StreamReader

[Bruce Merry]

Bruce Merry  added the comment:

Ok, I'll open a separate issue to allow a tuple of possible separators.

--
nosy: +bmerry

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32052] Provide access to buffer of asyncio.StreamReader

[Andrew Svetlov]

Andrew Svetlov  added the comment:

Closing.
Brief:
1. Access to an internal buffer is not an option.
2. Pushing data back to stream after fetching is not an option too: it kills 
almost any possible optimization and makes code overcomplicated. aiohttp used 
to have "unread_data()" API but we deprecated it and going to remove the method 
entirely. A wrapper around the stream with puback functionality is an option 
though unless it doesn't touch underlying stream implementation.
3. Extending a set of reader operations is a good idea, please make a separate 
issue with a concrete proposal if needed.

--
resolution:  -> fixed
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32052] Provide access to buffer of asyncio.StreamReader

[Bruce Merry]

Bruce Merry  added the comment:

A sequence of possible terminators would cover my immediate use case and 
certainly be an improvement.

To facilitate more general use cases without exposing implementation details, 
would it be practical and maintainable to have a "putback" method that prepends 
data to the buffer? It might not be fast in all cases (e.g. it might have to 
make a copy of what's still in the buffer), but possibly BufferedReader could 
detect the common case (putting back a suffix of what's just been read) and 
adjust its offsets into its internal buffer (although I'm not at all familiar 
with BufferedReader, so feel free to tell me I'm talking nonsense).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32052] Provide access to buffer of asyncio.StreamReader

[Andrew Svetlov]

Andrew Svetlov  added the comment:

Exposing internal buffer means committing on a new API contract forever.

I feel a need for reacher read*() API but pretty sure that making internal 
buffer public is a bad idea. With BufferedProtocol it could be even worse: SLAB 
allocators can spit a buffer into several separate chunks.

`str.startswith()` supports a tuple of separators, maybe we can do the same for 
streaming API

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32052] Provide access to buffer of asyncio.StreamReader

[Yury Selivanov]

Yury Selivanov  added the comment:

So we have BufferedProtocol in 3.7; now we need to re-implement asyncio streams 
on top of it.  But even after doing that I'm not that sure we want to expose 
the low-level buffer.

--
stage: needs patch -> 

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32052] Provide access to buffer of asyncio.StreamReader

[Cheryl Sabella]

Change by Cheryl Sabella :


--
dependencies: +Add asyncio.BufferedProtocol
stage:  -> needs patch
versions: +Python 3.8 -Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32052] Provide access to buffer of asyncio.StreamReader

[Yury Selivanov]

Yury Selivanov  added the comment:

I'd be more comfortable with the idea of exposing the buffer when we have 
BufferedProtocol.  Let's wait on this one.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32052] Provide access to buffer of asyncio.StreamReader

[Andrew Svetlov]

Andrew Svetlov  added the comment:

If the problm is in readuntil() functionality -- let's discuss th function 
improvement (in separate issue).

Exposing streams internals is antipattern and very bad idea.
I suggest closing the issue.

Yury, what is your opinion?

--
nosy: +asvetlov

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32052] Provide access to buffer of asyncio.StreamReader

[Bruce Merry]

New submission from Bruce Merry :

While asyncio.StreamReader.readuntil is an improvement on only having readline, 
it is still quite limited e.g. you cannot have multiple possible terminators. 
The real problem is that it's not possible to roll your own without accessing 
_underscore fields (other than by reading one byte at a time, which I'm 
guessing would be bad for performance). I'm not sure exactly what a public API 
to assist would look like, but I think the following would be a good start:

1. A get_buffer method, that returns (self._buffer, self._eof); the caller must 
treat the buffer as readonly.
2. A wait_for_data method to wait for the return value of get_buffer to change 
(basically like current _wait_for_data)
3. Access to the _limit attribute.

With that available, I think readuntil or more complex variants of it could be 
implemented externally using only the public interface (consumption of data 
from the buffer would be via readexactly rather than by messing with the buffer 
array directly).

--
components: asyncio
messages: 306397
nosy: Bruce Merry, yselivanov
priority: normal
severity: normal
status: open
title: Provide access to buffer of asyncio.StreamReader
type: enhancement
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com