"Ian Kelly" wrote in message
news:CALwzidk-RBkB-vi6CgcEeoFHQrsoTFvqX9MqzDD=rny5boc...@mail.gmail.com...
On Tue, Jan 26, 2016 at 7:15 AM, Frank Millman <fr...@chagford.com> wrote:
>
> If I return the cursor, I can iterate over it, but isn't this a blocking
> operation? As far as I know, the DB adaptor will only actually retrieve
> the
> row when requested.
>
> If I am right, I should call fetchall() while inside get_rows(), and
> return
> all the rows as a list.
>
You probably want an asynchronous iterator here. If the cursor doesn't
provide that, then you can wrap it in one. In fact, this is basically
one of the examples in the PEP:
https://www.python.org/dev/peps/pep-0492/#example-1
Thanks, Ian. I had a look, and it does seem to fit the bill, but I could not
get it to work, and I am running out of time.
Specifically, I tried to get it working with the sqlite3 cursor. I am no
expert, but after some googling I tried this -
import sqlite3
conn = sqlite3.connect('/sqlite_db')
cur = conn.cursor()
async def __aiter__(self):
return self
async def __anext__(self):
loop = asyncio.get_event_loop()
return await loop.run_in_executor(None, self.__next__)
import types
cur.__aiter__ = types.MethodType( __aiter__, cur )
cur.__anext__ = types.MethodType( __anext__, cur )
It failed with this exception -
AttributeError: 'sqlite3.Cursor' object has no attribute '__aiter__'
I think this is what happens if a class uses 'slots' to define its
attributes - it will not permit the creation of a new one.
Anyway, moving on, I decided to change tack. Up to now I have been trying to
isolate the function where I actually communicate with the database, and
wrap that in a Future with 'run_in_executor'.
In practice, the vast majority of my interactions with the database consist
of very small CRUD commands, and will have minimal impact on response times
even if they block. So I decided to focus on a couple of functions which are
larger, and try to wrap the entire function in a Future with
'run_in_executor'.
It seems to be working, but it looks a bit odd, so I will show what I am
doing and ask for feedback.
Assume a slow function -
async def slow_function(arg1, arg2):
[do stuff]
It now looks like this -
async def slow_function(arg1, arg2):
loop = asyncio.get_event_loop()
await loop.run_in_executor(None, slow_function_1, arg1, arg2)
def slow_function_1(self, arg1, arg2):
loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
loop.run_until_complete(slow_function_2(arg1, arg2))
async slow_function_2(arg1, arg2):
[do stuff]
Does this look right?
Frank
--
https://mail.python.org/mailman/listinfo/python-list
