Re: Altering sys.argv on startup in Python 2
Thank you very much, the hook gets invoked at the right place. Adam Bartoš -- https://mail.python.org/mailman/listinfo/python-list
Altering sys.argv on startup in Python 2
Hello, I'm trying to employ code like https://code.activestate.com/recipes/572200/ at Python 2 startup. The problem is that sys.argv isn't polulated yet when sitecustomize is executed. Is there any way around? I was thinking about something like temporarily chaning sys.modules['sys'] or sys.__dict__ to something that runs my code on PySys_SetArgv, but that doesn't work since PySys_SetArgv doesn't invoke any hooks like __setitem__ on sys.__dict__. So is there any way how to automatically run my code after sys.argv was set but before executing the main script (in Python 2)? Regards, Adam Bartoš -- https://mail.python.org/mailman/listinfo/python-list
Re: Python Questions - July 25, 2015
> How do you actually install Numpy in Windows? In theory, `pip install numpy` should work, but there are currently no wheels for Windows, see https://github.com/numpy/numpy/issues/5479. Try `pip install -i https://pypi.binstar.org/carlkl/simple numpy` (see last posts in the issue). Adam Bartoš -- https://mail.python.org/mailman/listinfo/python-list
Capturing stdin and stdout using ctypes
Hello, how can I capture C stdin and stdout file pointers using ctypes in Python 3? I want them to be able to call PyOS_Readline via ctypes. Thank you, Adam Bartoš -- https://mail.python.org/mailman/listinfo/python-list
Re: An asyncio example
Terry Reedy wrote: > I suggest you post your minimal example there. User interest in an > issue being fixed and willingness to test patches can help motivate. I've done it. Thank you for help. -- https://mail.python.org/mailman/listinfo/python-list
Re: An asyncio example
On Sat, Jul 4, 2015 at 1:07 PM, Adam Bartoš wrote:
> On Fri, Jul 3, 2015 at 9:14 AM, Marko Rauhamaa
>> wrote:
>> >
>> >>> 1) is there a way to close just one direction of the connection?
>> >>
>> >> No. SOCK_STREAM sockets are always bidirectional.
>> >
>> > socket.shutdown(socket.SHUT_WR) does the trick.
>> >
>> > I think the asyncio.StreamWriter.write_eof() is the high-level
>> > equivalent.
>>
>> I stand corrected. And you're also correct about write_eof: it causes
>> shutdown(SHUT_WR) to be called on the underlying socket once the
>> buffer has been written.
>>
>>
>> https://hg.python.org/cpython/file/34460219c0e0/Lib/asyncio/selector_events.py#l737
>>
>> That said, just replacing the writer.close() with writer.write_eof()
>> in the OP's code doesn't seem to work; the server response comes back
>> empty.
>>
>> This works:
>>
>> >>> sock1, sock2 = socket.socketpair()
>> >>> sock1.send(b'REQUEST')
>> 7
>> >>> sock1.shutdown(socket.SHUT_WR)
>> >>> sock2.recv(100)
>> b'REQUEST'
>> >>> sock2.send(b'RESPONSE')
>> 8
>> >>> sock1.recv(100)
>> b'RESPONSE'
>>
>> And this works:
>>
>> import asyncio
>> import socket
>>
>> def server(sock):
>> request = yield from asyncio.get_event_loop().sock_recv(sock, 100)
>> print("got request {!r}".format(request))
>> yield from asyncio.get_event_loop().sock_sendall(sock, b'RESPONSE')
>>
>> def client(sock):
>> yield from asyncio.get_event_loop().sock_sendall(sock, b'REQUEST')
>> sock.shutdown(socket.SHUT_WR)
>> response = yield from asyncio.get_event_loop().sock_recv(sock, 100)
>> print("got response {!r}".format(response))
>> asyncio.get_event_loop().stop()
>>
>> def connect():
>> clientsock, serversock = socket.socketpair()
>> clientsock.setblocking(False)
>> serversock.setblocking(False)
>> asyncio.async(client(clientsock))
>> asyncio.async(server(serversock))
>>
>> connect()
>> asyncio.get_event_loop().run_forever()
>>
>> I'm wondering whether there might be a bug in the higher-level
>> transport code that interferes with reading after calling write_eof.
>
>
> There is a bug. See http://bugs.python.org/issue24539 .
>
>
The following code also works.
import asyncio
import socket
async def server(reader, writer):
print("server is starting")
print("server is going to receive a request")
request = (await reader.read()).decode()
print("server received request {!r}".format(request))
response = "RESPONSE"
writer.write(response.encode())
print("server sent response {!r}".format(response))
writer.write_eof()
print("server sent EOF")
print("server is finishing")
async def client(reader, writer):
print("client is starting")
request = "REQUEST"
writer.write(request.encode())
print("client sent request {!r}".format(request))
writer.write_eof()
print("client sent EOF")
print("client is going to receive a response")
response = (await reader.read()).decode()
print("client received response {!r}".format(response))
print("client is finishing")
class FixedStreamReaderProtocol(asyncio.StreamReaderProtocol):
def eof_received(self):
super().eof_received()
return True # keep alive
async def open_connection(*, loop, sock):
reader = asyncio.StreamReader(loop=loop)
protocol = FixedStreamReaderProtocol(reader, loop=loop)
transport, _ = await loop.create_connection(lambda: protocol,
sock=sock)
writer = asyncio.StreamWriter(transport, protocol, reader, loop)
return reader, writer
async def connect(loop):
serversock, clientsock = socket.socketpair()
reader, writer = await open_connection(sock=serversock, loop=loop)
server_coro = server(reader, writer)
reader, writer = await open_connection(sock=clientsock, loop=loop)
client_coro = client(reader, writer)
server_task = loop.create_task(server_coro)
client_task = loop.create_task(client_coro)
return server_task, client_task
loop = asyncio.get_event_loop()
server_task, client_task = loop.run_until_complete(connect(loop))
loop.run_until_complete(server_task)
loop.run_until_complete(client_task)
--
https://mail.python.org/mailman/listinfo/python-list
Re: An asyncio example
>
> On Fri, Jul 3, 2015 at 9:14 AM, Marko Rauhamaa
> wrote:
> >
> >>> 1) is there a way to close just one direction of the connection?
> >>
> >> No. SOCK_STREAM sockets are always bidirectional.
> >
> > socket.shutdown(socket.SHUT_WR) does the trick.
> >
> > I think the asyncio.StreamWriter.write_eof() is the high-level
> > equivalent.
>
> I stand corrected. And you're also correct about write_eof: it causes
> shutdown(SHUT_WR) to be called on the underlying socket once the
> buffer has been written.
>
>
> https://hg.python.org/cpython/file/34460219c0e0/Lib/asyncio/selector_events.py#l737
>
> That said, just replacing the writer.close() with writer.write_eof()
> in the OP's code doesn't seem to work; the server response comes back
> empty.
>
> This works:
>
> >>> sock1, sock2 = socket.socketpair()
> >>> sock1.send(b'REQUEST')
> 7
> >>> sock1.shutdown(socket.SHUT_WR)
> >>> sock2.recv(100)
> b'REQUEST'
> >>> sock2.send(b'RESPONSE')
> 8
> >>> sock1.recv(100)
> b'RESPONSE'
>
> And this works:
>
> import asyncio
> import socket
>
> def server(sock):
> request = yield from asyncio.get_event_loop().sock_recv(sock, 100)
> print("got request {!r}".format(request))
> yield from asyncio.get_event_loop().sock_sendall(sock, b'RESPONSE')
>
> def client(sock):
> yield from asyncio.get_event_loop().sock_sendall(sock, b'REQUEST')
> sock.shutdown(socket.SHUT_WR)
> response = yield from asyncio.get_event_loop().sock_recv(sock, 100)
> print("got response {!r}".format(response))
> asyncio.get_event_loop().stop()
>
> def connect():
> clientsock, serversock = socket.socketpair()
> clientsock.setblocking(False)
> serversock.setblocking(False)
> asyncio.async(client(clientsock))
> asyncio.async(server(serversock))
>
> connect()
> asyncio.get_event_loop().run_forever()
>
> I'm wondering whether there might be a bug in the higher-level
> transport code that interferes with reading after calling write_eof.
There is a bug. See http://bugs.python.org/issue24539 .
--
https://mail.python.org/mailman/listinfo/python-list
Re: An asyncio example
On Sat, Jul 4, 2015 at 8:45 AM, Adam Bartoš wrote: > On Fri, Jul 3, 2015 at 7:38 PM, Adam Bartoš wrote: >>> >>>> Ian Kelly: >>>> >>> >> 2) In the blocked situaction even KeyboardInterrupt doesn't break the >>>> loop >>>> >>> >> is that desired behavior? And why? >>>> >>> > >>>> >>> > I don't think so. When I tried this locally (using Python 3.4.0, so >>>> >>> > replacing "async def" with "def" and "await" with "yield from" and >>>> >>> > "loop.create_task" with "asyncio.async") pressing Ctrl-C did interrupt >>>> >>> > the loop >>>> >>> > >>>> >>> Ok, I'll try to get more information and I'll eventually raise an issue. >>>> >>> >>> I tried it on Python 3.4.1 and Ctrl-C doesn't interrupt the loop. Maybe >>> it has something to do with the fact I'm on Windows. >> >> >> Please try with 3.4.3, which has further asyncio changes. >> > > I originally tried with 3.5.0b2. > > This is a minimal example: import asyncio async def wait(): await asyncio.sleep(5) loop = asyncio.get_event_loop() loop.run_until_complete(wait()) Ctrl-C doesn't interrupt the waiting, instead KeyboardInterrupt occurs after those five seconds. It's 3.5.0b2 on Windows. Is it a bug? -- https://mail.python.org/mailman/listinfo/python-list
Re: An asyncio example
> > On Fri, Jul 3, 2015 at 7:38 PM, Adam Bartoš wrote: >> >>> Ian Kelly: >>> >> >> 2) In the blocked situaction even KeyboardInterrupt doesn't break the >>> loop >>> >> >> is that desired behavior? And why? >>> >> > >>> >> > I don't think so. When I tried this locally (using Python 3.4.0, so >>> >> > replacing "async def" with "def" and "await" with "yield from" and >>> >> > "loop.create_task" with "asyncio.async") pressing Ctrl-C did interrupt >>> >> > the loop >>> >> > >>> >> Ok, I'll try to get more information and I'll eventually raise an issue. >>> >> >> I tried it on Python 3.4.1 and Ctrl-C doesn't interrupt the loop. Maybe >> it has something to do with the fact I'm on Windows. > > > Please try with 3.4.3, which has further asyncio changes. > I originally tried with 3.5.0b2. -- https://mail.python.org/mailman/listinfo/python-list
Re: An asyncio example
On Fri, Jul 3, 2015 at 7:38 PM, Adam Bartoš wrote: > Ian Kelly: > > >> 2) In the blocked situaction even KeyboardInterrupt doesn't break the > loop, > >> is that desired behavior? And why? > > > > I don't think so. When I tried this locally (using Python 3.4.0, so > > replacing "async def" with "def" and "await" with "yield from" and > > "loop.create_task" with "asyncio.async") pressing Ctrl-C did interrupt > > the loop. > > > Ok, I'll try to get more information and I'll eventually raise an issue. > I tried it on Python 3.4.1 and Ctrl-C doesn't interrupt the loop. Maybe it has something to do with the fact I'm on Windows. -- https://mail.python.org/mailman/listinfo/python-list
Re: An asyncio example
>> Marko Rauhamaa:
>>> socket.shutdown(socket.SHUT_WR) does the trick.
>>>
>>> I think the asyncio.StreamWriter.write_eof() is the high-level
>>> equivalent.
>>
>> You are right that writer.write_eof() behaves like
>> writer.transport.get_extra_info("socket").shutdown(socket.SHUT_WR) –
>> the server resumes and sends the response. However, the client still
>> reads empty bytes afterwards.
>
> Reading empty bytes means the peer has shut down its end of the
> connection.
But it didn't or had sent a response before. My code suppose to be like
this:
1) The client sends a request message.
2) The client closes the writing direction of the socket.
3) The server reads the request message.
4) The server sends a response message and possibly closes the other
direction of the socket.
5) The client reads data – it should read the response sent by the server
but gets empty bytes instead.
--
https://mail.python.org/mailman/listinfo/python-list
Re: An asyncio example
Ian Kelly:
>> 2) In the blocked situaction even KeyboardInterrupt doesn't break the
loop,
>> is that desired behavior? And why?
>
> I don't think so. When I tried this locally (using Python 3.4.0, so
> replacing "async def" with "def" and "await" with "yield from" and
> "loop.create_task" with "asyncio.async") pressing Ctrl-C did interrupt
> the loop.
>
Ok, I'll try to get more information and I'll eventually raise an issue.
>> 3) Are there some other issues with my code with respect to “best
practices”
>> how to write a code like this?
>
> There are a couple of approaches you could take. Since your protocol
> is so far text-based, I would suggest adding '\n' to the ends of your
> messages and using reader.readline instead of reader.read.
>
For now, there is no protocol at all. I just wanted to first send all the
data (of arbitrary size) and then wait for whole response. If there is
really no way to close just one direction of a socket, maybe the easiest
way is to have two socket pairs?
Marko Rauhamaa:
>>> 1) is there a way to close just one direction of the connection?
>>
>> No. SOCK_STREAM sockets are always bidirectional.
>
> socket.shutdown(socket.SHUT_WR) does the trick.
>
> I think the asyncio.StreamWriter.write_eof() is the high-level
> equivalent.
You are right that writer.write_eof() behaves like
writer.transport.get_extra_info("socket").shutdown(socket.SHUT_WR) – the
server resumes and sends the response. However, the client still reads
empty bytes afterwards.
--
https://mail.python.org/mailman/listinfo/python-list
An asyncio example
Hello,
I'm experimenting with asyncio. I have composed the following code. There
is a server handler and a client handler. I didn't want to split the code
into two files so I just used a socketpair, inspired by example
https://docs.python.org/3/library/asyncio-stream.html#register-an-open-socket-to-wait-for-data-using-streams
. However, my code doesn't work – it blocks because both sides are trying
to read more data. But if I close the writer on one side (the commented
line), whole connection is closed. So
1) is there a way to close just one direction of the connection?
2) In the blocked situaction even KeyboardInterrupt doesn't break the loop,
is that desired behavior? And why?
3) Are there some other issues with my code with respect to “best
practices” how to write a code like this?
Here is the code:
import asyncio
import socket
async def server(reader, writer):
print("starting server")
request = (await reader.read()).decode()
print("got request {!r}".format(request))
response = "RESPONSE"
writer.write(response.encode())
print("response sent")
async def client(reader, writer):
print("starting client")
request = "REQUEST"
writer.write(request.encode())
# writer.close()
print("request sent")
response = (await reader.read()).decode()
print("got response {!r}".format(response))
async def connect(loop):
serversock, clientsock = socket.socketpair()
reader, writer = await asyncio.open_connection(sock=serversock,
loop=loop)
server_coro = server(reader, writer)
reader, writer = await asyncio.open_connection(sock=clientsock,
loop=loop)
client_coro = client(reader, writer)
server_task = loop.create_task(server_coro)
client_task = loop.create_task(client_coro)
return client_task
loop = asyncio.get_event_loop()
task = loop.run_until_complete(connect(loop))
loop.run_until_complete(task)
Thank you, Adam Bartoš
--
https://mail.python.org/mailman/listinfo/python-list
