Matteo, Rajan, Joe, Addison, Penghui, and Jia have met today to discuss
this issue. We have come up with next steps:
In general, we have agreed to proceed with the PIP.
1. Penghui is going to update the PIP to clarify terminology and diagrams
in the PIP. He will also write more details about how
@Rajan Dhabalia
We discussed this PIP in the community meeting and agreed that we need to
evaluate this PIP one more time to see if we can combine this PIP together
with the bitmap proposal together as the final solution.
We should think about more use cases before pushing the bitmap approach.
Rajan - I understand your concern about memory. Lin, Penghui, and I also
acknowledged that none of the implementations solve every use case. Each
implementation has its limitation and concerns.
I am trying to find a way for both parties (You and Lin) can explore
different implementations. An
My only point was, if the broker tries to manage more than 10M unack
message ranges (per subscription) then the broker has to face many negative
consequences in terms of memory and CPU, and then it will require to build
additional debt to solve such problems. Therefore, I consider 10M unack
Agreed with Lin. I think we should try to abstract this into an interface
and allow different implementations.
Rajan - what is your real concern making it abstract?
- Sijie
On Wed, Jan 27, 2021 at 7:37 PM Lin Lin wrote:
> Hi Rajan,
> Thank you for your PR.
> The main difference lies in
Hi Rajan,
Thank you for your PR.
The main difference lies in whether 10MB is enough and memory doubling problem,
which is caused by different business scenarios.
In some business scenario, the QPS of 20k/s is considered to be very low, and
requests exceeding this order of magnitude are common.
Hi Rajan,
10M unack-message ranges can respond to most use-cases
but there are still some scenarios that cannot be dealt with.
We can make an abstraction for users who want to extend the implementation.
I think this will not conflict with your improvement.
This PIP not just to improve the
I have created a PR which should allow brokers to store up to 10M
unack-message ranges. I think it should be large enough for any usecases
and probably now, we might not need to introduce abstraction for ack
management to avoid any further complexity in message acknowledgement path
as well.
Thanks for all your suggestion, LinLin and I will work on abstract this
acknowledgment management as an
interface.
- Penghui
Sijie Guo 于2021年1月25日周一 上午11:14写道：
> Rajan,
>
> Thank you for sharing the prototype! That looks great to me.
>
> In order for the community to evolve and experiment with
Rajan,
Thank you for sharing the prototype! That looks great to me.
In order for the community to evolve and experiment with different
approaches, how about we abstract this acknowledgment management as an
interface?
If this approach works for you, maybe Lin Lin can focus on abstracting this
*Hi, >> Delayed messages or certain user logic can introduce a lot of
message-holes. We have seen this issue in quite a lot of customers'
production environment.I agree that we need a larger buffer to store and
recover individually deleted messages, and the existing 150K limit might
not be enough
Joe - Delayed messages or certain user logic can introduce a lot of message
holes. We have seen this issue in quite a lot of customers' production
environment. Hence we need to find a solution for solving these problems.
If you are skeptical of an implementation like that, how about us making
Let me take a step back and explain how I am looking at this from a
high-level
design viewpoint
Bookkeeper (BK) is like an LSM implementation of a KV store. Writes to all
keys are appended to a single file; deletes are logical. Compaction
reclaims space. An Index is used locate entries,
13 matches
Mail list logo
