Hi, thank you for the clear visualization of the behaviour of different
threads and the highlighting - your explanation is very helpful.
However, it is a pity that the result is negative. Given that the current
design is sufficient for solving the problems scrapy uses it for - I
supposed there is no interest in transforming queuelib to provide
thread-safety at the expense of adding more complexity. Am I correct?
My alternative is to tweak the design of the code where I wanted to apply
queuelib originally, by making sure that only one thread gets to use the
queue, thus avoiding the issue altogether.
I do think there's at least one thing we can do - to state that somewhere
in a FAQ, as I suspect that I am not the first person on the planet to
inquire about this.
Another thing is that, if my understanding is right, FifoMemoryQueue should
be fine, provided that T1 only pushes and T2 only pops, because according
to the manual {
Deques support thread-safe, memory efficient appends and pops from either
side of the deque ...
}
Are there any scenarios in which this is not going to be the case?
--
You received this message because you are subscribed to the Google Groups
"scrapy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at https://groups.google.com/group/scrapy-users.
For more options, visit https://groups.google.com/d/optout.
