Introducing new system concepts as code change PR is really not the way to go about making these changes. The semantics and use case discussion should come first, so that they can be discussed and the implications understood. It's not a good experience to put this out there without providing a meaningful explanation of the general concept, semantics and use cases that applies to a general Pulsar user. And how is this put out as a change with doc-not-needed ?
>The current use case is to use the non-persistent topic to store the load data used by the new load manager. How will Pulsar users understand this?. Non-persistent topics are a well-defined concept.- lossy, totally unreliable, and no guarantee of anything, Table View is another well-defined concept - a much more reliable construct with compacted keys. As a general Pulsar user, what exactly does a Table View on a non-persistent topic conceptually mean? What are the semantics? And then there is the TTL - how does this all fit together? > The current use case is to use the non-persistent topic to store the load data used by the new load manager. So is the idea that no external user should use this new NP-TableView? This is a public API A Table with a totally random selection of events in it? How does this work for a general use case? With this construct it is entirely possible that two clients A & B will have totally different data in their table views at the same time - what does that mean? You have introduced state into what is essentially a stateless concept (N-P topic) in Pulsar. I am not debating the merits/demerits of the change. It's really about how we go about doing things. Ideally you want to make a case for N-P TableViews, what's the semantics, what are the use cases, limitations and anti-patterns for this change. The community then can provide meaningful feedback, discuss the merits and suggest improvements. You will get a better feature, with good user documentation and clarity about how and when applications should adopt/reject that in their solution design. That benefits everyone - the contributor, the community, Pulsar users, and Pulsar architecture On Mon, Nov 14, 2022 at 8:59 PM Kai Wang <kw...@apache.org> wrote: > Hi Joe, > > > I am not sure about the semantics of TableView on a non-persistent topic. > > What happens if the client crashes? What is the base state for the > table? > > If users use a non-persistent topic as the TableView topic, when the > client crashes, > the TableViews data will be lose. > > The current use case is to use the non-persistent topic to store the load > data used by the new load manager. It doesn't require strong consistency > ensure, and no need persistence. > > > Thanks, > Kai > > On 2022/11/14 23:03:13 Joe F wrote: > > I am not sure about the semantics of TableView on a non-persistent topic. > > > > Exactly how does this work? > > > > What happens if the client crashes? What is the base state for the > table? > > > > What exactly can I expect as a user from this? > > > > Joe > > > > On Sun, Nov 13, 2022 at 8:57 PM Kai Wang <kw...@apache.org> wrote: > > > > > Hi, pulsar-dev community, > > > > > > Since the non-persistent topic support doesn't require API changes. I > have > > > pushed a PR to implement it, which has already been merged. > > > > > > See: https://github.com/apache/pulsar/pull/18375 > > > > > > And this PIP title has been changed to `Make TableView support TTL`. > > > > > > PIP link: https://github.com/apache/pulsar/issues/18229 > > > > > > Thanks, > > > Kai > > > > > > On 2022/11/04 02:28:41 Kai Wang wrote: > > > > Hi, pulsar-dev community, > > > > > > > > I’ve opened a PIP to discuss : PIP-221: Make TableView support read > the > > > non-persistent topic. > > > > > > > > PIP link: https://github.com/apache/pulsar/issues/18229 > > > > > > > > Thanks, > > > > Kai > > > > > > > > > >
