Re: [ANNOUNCE] Nicolò Boschi as new PMC member in Apache Pulsar
Congratulations ! Best Xiaoyu Hou Max Xu 于2023年1月30日周一 12:34写道： > Congratulations! Nicolò > > Best, > Max Xu > > > On Fri, Jan 20, 2023 at 8:36 PM Lari Hotari wrote: > > > Dear Community, > > > > We are thrilled to announce that Nicolò Boschi > > (https://github.com/nicoloboschi) has been invited and has accepted the > > role of member of the Apache Pulsar Project Management Committee (PMC). > > > > Nicolò Boschi has been a vital asset to our community, consistently > > demonstrating his dedication and active participation through > > significant contributions such as the development of the Pulsar Shell > > and numerous bug fixes, security enhancements, and improvements to > > Pulsar and its CI pipeline. In addition to his technical contributions, > > Nicolò also plays an important role in reviewing pull requests and > > ensuring the overall quality of our project. We look forward to his > > continued contributions. > > > > On behalf of the Pulsar PMC, we extend a warm welcome and > > congratulations to Nicolò Boschi. > > > > Best regards, Lari, on behalf of the Pulsar PMC > > >
Re: [ANNOUNCE] New Committer: Baodi Shi
Congratulations! Baodi Best, Max Xu On Wed, Jan 18, 2023 at 9:36 PM Yunze Xu wrote: > The Project Management Committee (PMC) for Apache Pulsar has invited > Baodi Shi (https://github.com/shibd) to become a committer and we are > pleased to announce that he has accepted. > > Being a committer enables easier contribution to the project since > there is no need to go via the patch submission process. This should > enable better productivity. > > Welcome and congratulations, Baodi Shi! > > Please join us in congratulating and welcoming Baodi Shi onboard! > > Thanks, > Yunze on behalf of the Pulsar PMC >
Re: [ANNOUNCE] Nicolò Boschi as new PMC member in Apache Pulsar
Congratulations! Nicolò Best, Max Xu On Fri, Jan 20, 2023 at 8:36 PM Lari Hotari wrote: > Dear Community, > > We are thrilled to announce that Nicolò Boschi > (https://github.com/nicoloboschi) has been invited and has accepted the > role of member of the Apache Pulsar Project Management Committee (PMC). > > Nicolò Boschi has been a vital asset to our community, consistently > demonstrating his dedication and active participation through > significant contributions such as the development of the Pulsar Shell > and numerous bug fixes, security enhancements, and improvements to > Pulsar and its CI pipeline. In addition to his technical contributions, > Nicolò also plays an important role in reviewing pull requests and > ensuring the overall quality of our project. We look forward to his > continued contributions. > > On behalf of the Pulsar PMC, we extend a warm welcome and > congratulations to Nicolò Boschi. > > Best regards, Lari, on behalf of the Pulsar PMC >
Re: [ANNOUNCE] Bo Cong as new PMC member in Apache Pulsar
Congratulations! Bo Best, Max Xu On Wed, Jan 18, 2023 at 9:50 PM PengHui Li wrote: > Hi all, > > The Apache Pulsar Project Management Committee (PMC) has invited Bo Cong > (https://github.com/congbobo184) as a member of the PMC and we are > pleased to announce that he has accepted. > > He is very active in the community in the past few years and made a lot of > great contributions > such as transactions and schemas. > > Welcome Bo Cong to the Apache Pulsar PMC. > > Best Regards, > Penghui on behalf of the Pulsar PMC >
Re: [DISCUSS] PIP-242: Introduce enableStrictTopicName to reject creating topic with -partition- keyword.
Hi, Asaf, Yunze > You mean to say, if the topic is partitioned, the word "partition" can > notappear in the submitted topic name, in the topic creation API? It's a little bit confusing. I will give some examples to help explain: Create a topic: • no corresponding partitioned topic • persistent://public/default/local-name (passed) • persistent://public/default/local-name-partition-z (rejected by keyword) • persistent://public/default/local-name-partition-0 (rejected by keyword) • Has corresponding partitioned topic, partitions=2 and topic partition name is persistent://public/default/local-name • persistent://public/default/local-name-partition-0 (passed, Because it is the partition topic's sub-partition) • persistent://public/default/local-name-partition-z (rejected by keyword) • persistent://public/default/local-name-partition-4 (rejected, Because it exceeds the number of maximum partitions) Create a partitioned topic(topic metadata) • persistent://public/default/local-name (passed) • persistent://public/default/local-name-partition-z (rejected by keyword) • persistent://public/default/local-name-partition-0 (rejected by keyword) > I think this PIP should go close to this end to end, meaning the last > stepbeing making it default true.Otherwise, we end up having so many "feature > flags" turned off, it's hardto navigate and improve Pulsar. Yes, we will add a warning log-in step 3 at the current release and enable it by default in the next major release(3.0?). Thanks a lot! Best, Mattison On Jan 30, 2023, 00:17 +0800, Asaf Mesika , wrote: > You mean to say, if the topic is partitioned, the word "partition" can not > appear in the submitted topic name, in the topic creation API?
Re: [ANNOUNCE] Nicolò Boschi as new PMC member in Apache Pulsar
Congratulations! Thanks, Cong On 2023/01/20 12:36:54 Lari Hotari wrote: > Dear Community, > > We are thrilled to announce that Nicolò Boschi > (https://github.com/nicoloboschi) has been invited and has accepted the > role of member of the Apache Pulsar Project Management Committee (PMC). > > Nicolò Boschi has been a vital asset to our community, consistently > demonstrating his dedication and active participation through > significant contributions such as the development of the Pulsar Shell > and numerous bug fixes, security enhancements, and improvements to > Pulsar and its CI pipeline. In addition to his technical contributions, > Nicolò also plays an important role in reviewing pull requests and > ensuring the overall quality of our project. We look forward to his > continued contributions. > > On behalf of the Pulsar PMC, we extend a warm welcome and > congratulations to Nicolò Boschi. > > Best regards, Lari, on behalf of the Pulsar PMC >
Re: [ANNOUNCE] Bo Cong as new PMC member in Apache Pulsar
Congratulations! Thanks, Cong On 2023/01/18 13:50:12 PengHui Li wrote: > Hi all, > > The Apache Pulsar Project Management Committee (PMC) has invited Bo Cong > (https://github.com/congbobo184) as a member of the PMC and we are > pleased to announce that he has accepted. > > He is very active in the community in the past few years and made a lot of > great contributions > such as transactions and schemas. > > Welcome Bo Cong to the Apache Pulsar PMC. > > Best Regards, > Penghui on behalf of the Pulsar PMC >
Re: [ANNOUNCE] Nicolò Boschi as new PMC member in Apache Pulsar
Congratulations, Nicolò! From: Lari Hotari Sent: Friday, January 20, 2023 20:36 To: dev@pulsar.apache.org Subject: [ANNOUNCE] Nicolò Boschi as new PMC member in Apache Pulsar Dear Community, We are thrilled to announce that Nicolò Boschi (https://github.com/nicoloboschi) has been invited and has accepted the role of member of the Apache Pulsar Project Management Committee (PMC). Nicolò Boschi has been a vital asset to our community, consistently demonstrating his dedication and active participation through significant contributions such as the development of the Pulsar Shell and numerous bug fixes, security enhancements, and improvements to Pulsar and its CI pipeline. In addition to his technical contributions, Nicolò also plays an important role in reviewing pull requests and ensuring the overall quality of our project. We look forward to his continued contributions. On behalf of the Pulsar PMC, we extend a warm welcome and congratulations to Nicolò Boschi. Best regards, Lari, on behalf of the Pulsar PMC
Re: [ANNOUNCE] New Committer: Baodi Shi
Congratulations! Thanks, Cong On 2023/01/18 13:35:41 Yunze Xu wrote: > The Project Management Committee (PMC) for Apache Pulsar has invited > Baodi Shi (https://github.com/shibd) to become a committer and we are > pleased to announce that he has accepted. > > Being a committer enables easier contribution to the project since > there is no need to go via the patch submission process. This should > enable better productivity. > > Welcome and congratulations, Baodi Shi! > > Please join us in congratulating and welcoming Baodi Shi onboard! > > Thanks, > Yunze on behalf of the Pulsar PMC >
Re: [ANNOUNCE] Bo Cong as new PMC member in Apache Pulsar
Congratulations, Bo! From: PengHui Li Sent: Wednesday, January 18, 2023 21:50 To: Dev Subject: [ANNOUNCE] Bo Cong as new PMC member in Apache Pulsar Hi all, The Apache Pulsar Project Management Committee (PMC) has invited Bo Cong (https://github.com/congbobo184) as a member of the PMC and we are pleased to announce that he has accepted. He is very active in the community in the past few years and made a lot of great contributions such as transactions and schemas. Welcome Bo Cong to the Apache Pulsar PMC. Best Regards, Penghui on behalf of the Pulsar PMC
Re: [ANNOUNCE] New Committer: Baodi Shi
Congratulations, Baodi! From: Yunze Xu Sent: Wednesday, January 18, 2023 21:35 To: dev@pulsar.apache.org Subject: [ANNOUNCE] New Committer: Baodi Shi The Project Management Committee (PMC) for Apache Pulsar has invited Baodi Shi (https://github.com/shibd) to become a committer and we are pleased to announce that he has accepted. Being a committer enables easier contribution to the project since there is no need to go via the patch submission process. This should enable better productivity. Welcome and congratulations, Baodi Shi! Please join us in congratulating and welcoming Baodi Shi onboard! Thanks, Yunze on behalf of the Pulsar PMC
Re: [ANNOUNCE] Nicolò Boschi as new PMC member in Apache Pulsar
Congrats Nico! On Mon, Jan 23, 2023 at 10:47 PM Zixuan Liu wrote: > Congrats, Nicolò! > > Best, > Zixuan > > Dave Fisher 于2023年1月21日周六 06:01写道： > > > Congratulations Nicolo! > > > > Best, > > Dave > > > > > On Jan 20, 2023, at 6:57 AM, tison wrote: > > > > > > Congrats Nico! And well deserved :) > > > > > > Best, > > > tison. > > > > > > > > > Christophe Bornet 于2023年1月20日周五 22:39写道： > > > > > >> Congrats, Nicolo ! > > >> > > >> Le ven. 20 janv. 2023 à 14:27, Enrico Olivelli > a > > >> écrit : > > >> > > >>> Congratulations ! > > >>> > > >>> Enrico > > >>> > > >>> Il giorno ven 20 gen 2023 alle ore 13:36 Lari Hotari > > >>> ha scritto: > > > > Dear Community, > > > > We are thrilled to announce that Nicolò Boschi > > (https://github.com/nicoloboschi) has been invited and has accepted > > >> the > > role of member of the Apache Pulsar Project Management Committee > > (PMC). > > > > Nicolò Boschi has been a vital asset to our community, consistently > > demonstrating his dedication and active participation through > > significant contributions such as the development of the Pulsar > Shell > > and numerous bug fixes, security enhancements, and improvements to > > Pulsar and its CI pipeline. In addition to his technical > > contributions, > > Nicolò also plays an important role in reviewing pull requests and > > ensuring the overall quality of our project. We look forward to his > > continued contributions. > > > > On behalf of the Pulsar PMC, we extend a warm welcome and > > congratulations to Nicolò Boschi. > > > > Best regards, Lari, on behalf of the Pulsar PMC > > >>> > > >> > > > > >
Re: [DISCUSS] PIP-242: Introduce enableStrictTopicName to reject creating topic with -partition- keyword.
> Otherwise, we end up having so many "feature flags" turned off, it's hard to > navigate and improve Pulsar. +1 to me. We already have too many feature flags. Thanks, Yunze On Mon, Jan 30, 2023 at 12:17 AM Asaf Mesika wrote: > > > > > Users can only create a topic with it if the topic is partitioned > > You mean to say, if the topic is partitioned, the word "partition" can not > appear in the submitted topic name, in the topic creation API? > > 4. Make `enableStrictTopicName=true` in the future. > > I think this PIP should go close to this end to end, meaning the last step > being making it default true. > Otherwise, we end up having so many "feature flags" turned off, it's hard > to navigate and improve Pulsar. > > Other than that, I like this proposal. > > > > > On Sat, Jan 28, 2023 at 12:46 PM wrote: > > > Hello everyone. > > I hope you guys are all doing well. > > > > I would like to start the discussion for PIP-242 > > https://github.com/apache/pulsar/issues/19239, > > Please let me know if you have any concerns or questions. > > > > Best, > > Mattison > > > > --- Paste original PIP content to help quote -- > > > > ### Motivation > > > > Currently, the Apache Pulsar broker allows users to create a topic name > > that includes `-partition-`, which is confusing for our developers to > > identify whether this is a partition of a partitioned topic. Plus, we need > > to add more logic to be compatible with this special topic name. for > > example: > > > > - https://github.com/apache/pulsar/pull/19240 > > - https://github.com/apache/pulsar/pull/19230 > > - https://github.com/apache/pulsar/pull/19171 > > - https://github.com/apache/pulsar/pull/19086 > > - ... > > > > ### Goal > > This proposal wants `-partition-` to be a topic name keyword. Users can > > only create a topic with it if the topic is partitioned. For the > > compatibility reason, we want to Introduce a new configuration - > > `enableStrictTopicName` for the broker to help reject creating a topic in > > the following cases: > > 1. Create a partitioned topic that includes `-partition-`. > > 2. Create a topic which is not a partitioned topic. > > > > ### API Changes > > > > Add a new configuration, `enableStrictTopicName=false`. > > > > ### Implementation > > > > 1. Add configuration `enableStrictTopicName=false`. > > 2. Add rejection logic when the user enables `enableStrictTopicName`. > > 3. Add warning logs to inform users that we do not recommend creating > > non-partitioned topics with the keyword `-partition-`. > > 4. Make `enableStrictTopicName=true` in the future. > >
Re: [DISCUSS] PIP-244: Refactor ByteBuf release method
I disagree with using `ReferenceCountUtil.safeRelease()` everywhere.
The implementation is throwing any Throwable:
```java
public static void safeRelease(Object msg) {
try {
release(msg);
} catch (Throwable var2) {
logger.warn("Failed to release a message: {}", msg, var2);
}
}
```
When `release` throws an exception, it means the logic is wrong. For
example, you have released a ByteBuf whose refcnt is 1 twice. We
should make it clear where fast-fail is not allowed and catch the
exception in these places.
Thanks,
Yunze
On Sun, Jan 29, 2023 at 7:57 PM steven lu wrote:
>
> [DISCUSS] PIP-244: Refactor ByteBuf release method
> Hello everyone. I hope you guys are all doing well. I would like to start
> the discussion for PIP-244 https://github.com/apache/pulsar/issues/19350,
> Please let me know if you have any concerns or questions.
> --- Paste original PIP content to help quote --
>
> ### Motivation
>
> It may throw an exception when release a `ByteBuf` object. so the exception
> in `ByteBuf.release` should be checked.
>
> ### Goal
>
> Use `ReferenceCountUtil.safeRelease()` instead of `ByteBuf.release()` in
> all Pulsar's classes.
>
> ### API Changes
>
> _No response_
>
> ### Implementation
>
> 1. Use `ReferenceCountUtil.safeRelease()` instead of `ByteBuf.release()`.
Re: [DISCUSS] Redundant ServiceUrlProvider design and improper use of PIP-121
In summary, the core issue is that ServiceUrlProvider could be
misused. A user might want to implement its own ServiceUrlProvider by
changing the returned value of `get` method but it actually does not
work. The built-in implementations like AutoFailover actually work by
these `updateXXX` methods. So it's meaningless to provide them as
ServiceUrlProvider implementations.
To answer your question directly: totally yes. But not only
`setServiceUrl` (the name is `updateServiceUrl` in code), the
`updateAuthentication` and `updateTlsTrustCertsFilePath` methods,
which are not exposed to `PulsarClient`, should also be exposed.
Thanks,
Yunze
On Sun, Jan 29, 2023 at 11:57 PM Asaf Mesika wrote:
>
> Yunze - in summary - your proposal is to get rid of the ServiceUrlProvider
> right? You just want to have setServiceUrl on the client instead?
>
>
> On Wed, Jan 25, 2023 at 2:03 PM Yunze Xu
> wrote:
>
> > It makes sense to me. So the better way is still providing the
> > `updateXXX` methods to `PulsarClient`. As for how to detect the
> > connection info (e.g. service URL) changes, it's determined by the
> > user's implementation. For example, the current AutoClusterFailover
> > polls with a fixed interval.
> >
> > We can also implement a notification mechanism based on the
> > `updateXXX` methods. For example, assuming we have a producer that
> > writes the latest service URL to a topic and a consumer that reads the
> > latest service URL from that topic. Then, the consumer could call
> > `pulsarClient.updateXXX` once it receives the latest service URL.
> >
> > Thanks,
> > Yunze
> >
> > On Tue, Jan 24, 2023 at 4:32 PM Asaf Mesika wrote:
> > >
> > > If I understand correctly, the idea of having the ability to change
> > > serviceUrl is to support switching over between clusters dynamically?
> > > If that is the case, doesn't it make sense that it will trigger the
> > change
> > > to the client instead of the client polling it and check it self if
> > > something has changed?
> > >
> > >
> > > On Mon, Jan 23, 2023 at 5:00 AM Yunze Xu
> > > wrote:
> > >
> > > > Yes. The poll happens in the client's internal thread.
> > > >
> > > > Thanks,
> > > > Yunze
> > > >
> > > > On Sun, Jan 22, 2023 at 6:56 PM Asaf Mesika
> > wrote:
> > > > >
> > > > > and the client will poll the ConnectInfoProvider and check if
> > something
> > > > was
> > > > > changed?
> > > > >
> > > > > On Fri, Jan 20, 2023 at 11:19 AM Yunze Xu
> >
> > > > > wrote:
> > > > >
> > > > > > > I think the `updateServiceUrl` is not the initial purpose of
> > > > exposing to
> > > > > > the Client API.
> > > > > >
> > > > > > I agree. We might need an API like
> > > > > >
> > > > > > ```java
> > > > > > ClientBuilder serviceUrlProvider(ServiceUrlProvider
> > > > > > serviceUrlProvider, Duration interval)
> > > > > > ```
> > > > > >
> > > > > > But not only the service URL, as you can see, the
> > > > > > `AutoClusterFailover` implementation is already beyond the scope of
> > > > > > service URL, it uses two internal APIs `updateAuthentication` and
> > > > > > `updateTlsTrustCertsFilePath` to update other states of
> > > > > > PulsarClientImpl.
> > > > > >
> > > > > > > So from the user's perspective, they only need to apply a
> > service
> > > > URL
> > > > > > provided to the client
> > > > > >
> > > > > > Yes. That's when I thought of when I saw the C++ client catch up
> > [1].
> > > > > > The `initialize` and `close` methods are not necessary. If let me
> > > > > > design the interface, I would like the following solution, which is
> > > > > > more simple and can accept a lambda.
> > > > > >
> > > > > > ```java
> > > > > > public interface ServiceUrlProvider {
> > > > > > String getServiceUrl();
> > > > > > }
> > > > > > ```
> > > > > >
> > > > > > However, as I've mentioned again, now authentication and TLS info
> > also
> > > > > > need updates, so I have an initial design like:
> > > > > >
> > > > > > ```java
> > > > > > public class ConnectInfo {
> > > > > > String serviceUrl;
> > > > > > Authentication authentication;
> > > > > > String tlsCertificatesFile;
> > > > > > // ...
> > > > > > }
> > > > > >
> > > > > > interface ConnectInfoProvider extends Supplier {
> > > > > > }
> > > > > > ```
> > > > > >
> > > > > > When configuring the `ConnectInfoProvider`, we should provide an
> > > > interval.
> > > > > >
> > > > > > And I'm going to open a PIP for it to deprecate
> > `ServiceUrlProvider`.
> > > > > > BTW, I found this issue when I reviewed the PR to migrate the
> > > > > > ServiceUrlProvider into C++ client. IMO, the current design in Java
> > > > > > client is bad and I don't want to adopt the same design in C++
> > client.
> > > > > >
> > > > > > Thanks,
> > > > > > Yunze
> > > > > >
> > > > > > On Thu, Jan 19, 2023 at 4:51 PM PengHui Li
> > wrote:
> > > > > > >
> > > > > > > Is it better to introduce a service URL detect interval to the
> > > > service
> > > > > > URL
> > > > > > > provider?
> > > > > > > I think the
Re: [DISCUSS] PIP-186: Introduce two phase deletion protocol based on system topic
Couples of notes: 1. > In the LedgerDeletionService start, it will create a producer to send > pending delete ledger. > When delete a ledger, a PendingDeleteLedgerInfo msg with 1 min delay (the > delay is for consumer side, if send it immediately, maybe the metadata > din't change when consumer receive it). After the send operation succeeds, > then to operate metadata. If send msg failed, we think this deletion > operation failed, and didn't operate metadata. This section is completely unclear to me. Can you please provide step by step exactly what will happen in the workflow? Who does what and where (node)? 2. > /** > * The ledger component . managed-ledger, managed-cursor and > schema-storage. > */ > private LedgerComponent ledgerComponent; Why is this needed? What do you mean by a component of a ledger? Is the ledger divided into components? 3. > /** > * The ledger type. ledger or offload-ledger. > */ > private LedgerType ledgerType; I don't understand why you need this type. 4. > private MLDataFormats.ManagedLedgerInfo.LedgerInfo context; Context is a very generic word, but your type is so specific. Can you please explain why you need this for? Are you sure you want to tie one data structure into the other - just validating. 5. > /** > * Extent properties. > */ > private Map properties = new HashMap<>(); Why is this needed? 6. > When receiving a pending delete ledger msg, we will check if the topic > still exists. If the topic exists, send a delete command > (PendingDelteLedger) to the broker which owns the topic. In the broker, it > will check if the ledger is still in the metadata store, if the ledger in > the metadata store means the ledger is still in use, give up to delete this > ledge I don't understand this workflow. You say you check if it's in the metadata store, and if it is , then it is used - what will make it unused? Can you explain the starting point? How does deletion work in general? When? What happens? ... I understand there are time based triggers, and sometimes used based triggers. They are somehow marked in metadata. 7. If we delete successfully, the consumer will ack this msg. If delete fails, > reconsume this msg 10 min later. Where did you define 10min? Why 10 min? You mentioned you are working with a client which has retries configured. Retry is client side based, ack one message while producing another, transaction free. Are you prepared to handle a case where you acked but failed to produce the message, hence you lost it completely? 8. > If we want to delete a topic, we should send the delete ledger msg to > system topic and remove ledger id from metadata one by one, after all the > ledger has been deleted, then delete the topic. If any ledger operate > failed, we think this delete topic operation failed and return the left > ledger id in the response. I couldn't understand. Can you please clarify this section. How exactly topic deletion is modified to use the features described in this pip? 9. Regarding configuration. I suggest we prefix all config keys with the feature name so we can easily separate them. 10. Backward compatibility - I would make sure to document in the docs exactly what to do in case we need to rollback (cook book). 11. General comment - You're basically implementing a bespoke workflow using a topic to save the state of where you are in the topic. Is this the only place in Pulsar (delete ledger) that an action is composed of several steps ? If the answer is no to this, wouldn't it be better to have a small utility which is in charge of moving through the workflow steps? It can even be a simple state enum, where you move your state from a to b to c to d and it is persisted. 12. Monitoring Some actions here can take a long time. We're basically relying on logs to monitor where we are in the flow? On Sun, Jan 29, 2023 at 4:44 AM Yan Zhao wrote: > We need more eyes and votes. Thanks. >
Re: [DISCUSS] PIP-242: Introduce enableStrictTopicName to reject creating topic with -partition- keyword.
> > Users can only create a topic with it if the topic is partitioned You mean to say, if the topic is partitioned, the word "partition" can not appear in the submitted topic name, in the topic creation API? 4. Make `enableStrictTopicName=true` in the future. I think this PIP should go close to this end to end, meaning the last step being making it default true. Otherwise, we end up having so many "feature flags" turned off, it's hard to navigate and improve Pulsar. Other than that, I like this proposal. On Sat, Jan 28, 2023 at 12:46 PM wrote: > Hello everyone. > I hope you guys are all doing well. > > I would like to start the discussion for PIP-242 > https://github.com/apache/pulsar/issues/19239, > Please let me know if you have any concerns or questions. > > Best, > Mattison > > --- Paste original PIP content to help quote -- > > ### Motivation > > Currently, the Apache Pulsar broker allows users to create a topic name > that includes `-partition-`, which is confusing for our developers to > identify whether this is a partition of a partitioned topic. Plus, we need > to add more logic to be compatible with this special topic name. for > example: > > - https://github.com/apache/pulsar/pull/19240 > - https://github.com/apache/pulsar/pull/19230 > - https://github.com/apache/pulsar/pull/19171 > - https://github.com/apache/pulsar/pull/19086 > - ... > > ### Goal > This proposal wants `-partition-` to be a topic name keyword. Users can > only create a topic with it if the topic is partitioned. For the > compatibility reason, we want to Introduce a new configuration - > `enableStrictTopicName` for the broker to help reject creating a topic in > the following cases: > 1. Create a partitioned topic that includes `-partition-`. > 2. Create a topic which is not a partitioned topic. > > ### API Changes > > Add a new configuration, `enableStrictTopicName=false`. > > ### Implementation > > 1. Add configuration `enableStrictTopicName=false`. > 2. Add rejection logic when the user enables `enableStrictTopicName`. > 3. Add warning logs to inform users that we do not recommend creating > non-partitioned topics with the keyword `-partition-`. > 4. Make `enableStrictTopicName=true` in the future. >
Re: [DISCUSS] Redundant ServiceUrlProvider design and improper use of PIP-121
Yunze - in summary - your proposal is to get rid of the ServiceUrlProvider
right? You just want to have setServiceUrl on the client instead?
On Wed, Jan 25, 2023 at 2:03 PM Yunze Xu
wrote:
> It makes sense to me. So the better way is still providing the
> `updateXXX` methods to `PulsarClient`. As for how to detect the
> connection info (e.g. service URL) changes, it's determined by the
> user's implementation. For example, the current AutoClusterFailover
> polls with a fixed interval.
>
> We can also implement a notification mechanism based on the
> `updateXXX` methods. For example, assuming we have a producer that
> writes the latest service URL to a topic and a consumer that reads the
> latest service URL from that topic. Then, the consumer could call
> `pulsarClient.updateXXX` once it receives the latest service URL.
>
> Thanks,
> Yunze
>
> On Tue, Jan 24, 2023 at 4:32 PM Asaf Mesika wrote:
> >
> > If I understand correctly, the idea of having the ability to change
> > serviceUrl is to support switching over between clusters dynamically?
> > If that is the case, doesn't it make sense that it will trigger the
> change
> > to the client instead of the client polling it and check it self if
> > something has changed?
> >
> >
> > On Mon, Jan 23, 2023 at 5:00 AM Yunze Xu
> > wrote:
> >
> > > Yes. The poll happens in the client's internal thread.
> > >
> > > Thanks,
> > > Yunze
> > >
> > > On Sun, Jan 22, 2023 at 6:56 PM Asaf Mesika
> wrote:
> > > >
> > > > and the client will poll the ConnectInfoProvider and check if
> something
> > > was
> > > > changed?
> > > >
> > > > On Fri, Jan 20, 2023 at 11:19 AM Yunze Xu
>
> > > > wrote:
> > > >
> > > > > > I think the `updateServiceUrl` is not the initial purpose of
> > > exposing to
> > > > > the Client API.
> > > > >
> > > > > I agree. We might need an API like
> > > > >
> > > > > ```java
> > > > > ClientBuilder serviceUrlProvider(ServiceUrlProvider
> > > > > serviceUrlProvider, Duration interval)
> > > > > ```
> > > > >
> > > > > But not only the service URL, as you can see, the
> > > > > `AutoClusterFailover` implementation is already beyond the scope of
> > > > > service URL, it uses two internal APIs `updateAuthentication` and
> > > > > `updateTlsTrustCertsFilePath` to update other states of
> > > > > PulsarClientImpl.
> > > > >
> > > > > > So from the user's perspective, they only need to apply a
> service
> > > URL
> > > > > provided to the client
> > > > >
> > > > > Yes. That's when I thought of when I saw the C++ client catch up
> [1].
> > > > > The `initialize` and `close` methods are not necessary. If let me
> > > > > design the interface, I would like the following solution, which is
> > > > > more simple and can accept a lambda.
> > > > >
> > > > > ```java
> > > > > public interface ServiceUrlProvider {
> > > > > String getServiceUrl();
> > > > > }
> > > > > ```
> > > > >
> > > > > However, as I've mentioned again, now authentication and TLS info
> also
> > > > > need updates, so I have an initial design like:
> > > > >
> > > > > ```java
> > > > > public class ConnectInfo {
> > > > > String serviceUrl;
> > > > > Authentication authentication;
> > > > > String tlsCertificatesFile;
> > > > > // ...
> > > > > }
> > > > >
> > > > > interface ConnectInfoProvider extends Supplier {
> > > > > }
> > > > > ```
> > > > >
> > > > > When configuring the `ConnectInfoProvider`, we should provide an
> > > interval.
> > > > >
> > > > > And I'm going to open a PIP for it to deprecate
> `ServiceUrlProvider`.
> > > > > BTW, I found this issue when I reviewed the PR to migrate the
> > > > > ServiceUrlProvider into C++ client. IMO, the current design in Java
> > > > > client is bad and I don't want to adopt the same design in C++
> client.
> > > > >
> > > > > Thanks,
> > > > > Yunze
> > > > >
> > > > > On Thu, Jan 19, 2023 at 4:51 PM PengHui Li
> wrote:
> > > > > >
> > > > > > Is it better to introduce a service URL detect interval to the
> > > service
> > > > > URL
> > > > > > provider?
> > > > > > I think the `updateServiceUrl` is not the initial purpose of
> > > exposing to
> > > > > > the Client API.
> > > > > >
> > > > > > It looks like users just provide the interval of checking
> whether the
> > > > > > service URL is changed.
> > > > > > The Pulsar client will check it automatically. Using
> > > updateServiceUrl can
> > > > > > also achieve the purpose,
> > > > > > but users need to provide a fake service URL first or fetch the
> > > service
> > > > > URL
> > > > > > before
> > > > > > creating the client.
> > > > > >
> > > > > > Another reason we need service URL provider API is that one team
> will
> > > > > > usually
> > > > > > provide an extra pulsar client lib with the service URL provider
> > > > > > implementation.
> > > > > > The lib
> > > > > > can be used across multiple teams. So from the user's
> perspective,
> > > they
> > > > > > only need to apply
> > > > > > a service URL provided to the client, they don't care about
[DISCUSS] PIP-244: Refactor ByteBuf release method
[DISCUSS] PIP-244: Refactor ByteBuf release method Hello everyone. I hope you guys are all doing well. I would like to start the discussion for PIP-244 https://github.com/apache/pulsar/issues/19350, Please let me know if you have any concerns or questions. --- Paste original PIP content to help quote -- ### Motivation It may throw an exception when release a `ByteBuf` object. so the exception in `ByteBuf.release` should be checked. ### Goal Use `ReferenceCountUtil.safeRelease()` instead of `ByteBuf.release()` in all Pulsar's classes. ### API Changes _No response_ ### Implementation 1. Use `ReferenceCountUtil.safeRelease()` instead of `ByteBuf.release()`.
