Interesting question. Are you still interested in this? Can you contact me and I will put some time to write some multithreaded tests around it.
On Tuesday, March 15, 2016 at 9:14:13 AM UTC, Alex Railean wrote: > > Hello, > > I am researching the possibility of using `queuelib` in a project that > involves multiple threads, one always writes to the queue, another one only > reads from it. > > Having explored the documentation and the code, I did not see any explicit > references or comments related to such scenarios, thus I would like to > clarify a few things. > > > - the in-memory queues rely on collections.deque, so they are supposed to > be thread-safe within these constraints: The deque's append(), > appendleft(), pop(), popleft(), and len(d) operations are thread-safe in > CPython. > - the SQLite-based queue is thread-safe, because of SQLite itself, > provided that you connect with check_same_thread=False. > > > To sum things up, > 1. can someone confirm that the reasoning above is sound? > 2. has someone successfully used queuelib with multiple threads? > 3. are there any tips you could share regarding the philosophically > correct way to do this with queuelib? > > > > > Looking forward to your feedback, > Alex > -- You received this message because you are subscribed to the Google Groups "scrapy-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/scrapy-users. For more options, visit https://groups.google.com/d/optout.
