Just catching up after being out for a few weeks.
One additional point to add is that RQ (and queueing systems generally)
have a first-come-first-serve assumption/requirement. Pulp on the other
hand has this "resource" based task dependency where any task is able to be
worked on as long as no
On Tue, Apr 13, 2021 at 6:55 PM Daniel Alley wrote:
> Are there any benefits to improving RQ vs the invented here method? I'm
>> just curious about the cost of maintaining a tasking system versus being
>> part of a community built one. This feels like the kind of problem that
>> many other
>
> Are there any benefits to improving RQ vs the invented here method? I'm
> just curious about the cost of maintaining a tasking system versus being
> part of a community built one. This feels like the kind of problem that
> many other applications should have in the Python world -- or are there
On Thu, Apr 8, 2021 at 5:24 PM Daniel Alley wrote:
> Eric,
>
> * The idea is to move away from RQ entirely. RQ is fine (and vastly
> better than Celery IMO), but managing task state across both 1) the
> database and 2) a separate, external registry is still problematic. If all
> of the
Eric,
* The idea is to move away from RQ entirely. RQ is fine (and vastly better
than Celery IMO), but managing task state across both 1) the database and
2) a separate, external registry is still problematic. If all of the
information can simply be kept in the database, then it will be much
A few initial questions that get a bit into the stack but will help the
Foreman project think on the proposed changes:
* Does this move away from RQ entirely or just RQ workers?
* Do the new workers remove Pulp 3's use of Redis all together?
* Will using the database result in any additional
FYI, @mdellweg and I have been collaborating on the tasking system changes.
This email is to share some info to transition the work to @mdellweg while
I'm out. With the new-style disabled by default I am hoping it can go into
3.13.
## The PoC and ticket info
The PoC is basically functional, but
