Re: [VOTE] PIP-282: Change definition of the recently joined consumers position
Close this vote by 3 binding +1. Thanks, -- Yuri Mizushima equ...@apache.org On 2023/12/19 11:10:45 PengHui Li wrote: > +1 > > Thanks, > Penghui > > On Tue, Dec 19, 2023 at 3:58 PM Yubiao Feng > wrote: > > > +1 > > > > Thanks > > Yubiao Feng > > > > On Tue, Nov 28, 2023 at 4:04 PM Yuri Mizushima wrote: > > > > > Hi, Pulsar Community > > > > > > This thread is starting a vote for PIP-282. > > > > > > PIP design PR: > > > https://github.com/apache/pulsar/pull/20776 > > > > > > discussion thread: > > > https://lists.apache.org/thread/69fpb0d30y7pc02k3zvg2lpb2lj0smdg > > > > > > Motivation > > > > > > Key_Shared has a mechanism called the "recently joined consumers" to keep > > > message ordering. > > > However, currently, it doesn't care about some corner cases. > > > > > > 1. [issue-1] The race condition in the "recently joined consumers", where > > > consumers can be added before finishing reading and dispatching messages > > > from ledgers. > > > 2. [issue-2] Messages could be added to messagesToRedeliver without > > > consumer-side operations such as unacknowledgement. > > > > > > We should care about these cases in Key_Shared subscription. > > > > > > Thanks, > > > > > > -- > > > Yuri Mizushima > > > equ...@apache.org > > > > > >
[VOTE] PIP-282: Change definition of the recently joined consumers position
Hi, Pulsar Community This thread is starting a vote for PIP-282. PIP design PR: https://github.com/apache/pulsar/pull/20776 discussion thread: https://lists.apache.org/thread/69fpb0d30y7pc02k3zvg2lpb2lj0smdg Motivation Key_Shared has a mechanism called the "recently joined consumers" to keep message ordering. However, currently, it doesn't care about some corner cases. 1. [issue-1] The race condition in the "recently joined consumers", where consumers can be added before finishing reading and dispatching messages from ledgers. 2. [issue-2] Messages could be added to messagesToRedeliver without consumer-side operations such as unacknowledgement. We should care about these cases in Key_Shared subscription. Thanks, -- Yuri Mizushima equ...@apache.org
[DISCUSS] PIP-282: Change definition of the recently joined consumers position
Hi, Pulsar Community I opened a new PIP design PR. I am looking forward to your feedback. https://github.com/apache/pulsar/pull/20776 Motivation Key_Shared has a mechanism called the "recently joined consumers" to keep message ordering. However, currently, it doesn't care about some corner cases. 1. [issue-1] The race condition in the "recently joined consumers", where consumers can be added before finishing reading and dispatching messages from ledgers. 2. [issue-2] Messages could be added to messagesToRedeliver without consumer-side operations such as unacknowledgement. We should care about these cases in Key_Shared subscription. Thanks, -- Yuri Mizushima yumiz...@yahoo-corp.jp
Re: [VOTE] Pulsar Node.js Client Release 1.7.0 Candidate 2
Hi Masahiro,
It seems the C++ client causes the AlreadyClosed issue.
https://github.com/apache/pulsar-client-cpp/issues/51
I think this issue doesn't block releasing new Node.js client.
What do you think?
Thanks,
--
Yuri Mizushima
yumiz...@yahoo-corp.jp
"Masahiro Sakamoto" wrote:
-1 (binding)
I almost always get "Failed to send message: AlreadyClosed" errors when running
the following code:
##
$ cat producer.js
const Pulsar = require('./index.js');
(async () => {
const client = new Pulsar.Client({
serviceUrl: 'pulsar://localhost:6650',
});
const producer = await client.createProducer({
topic: 'persistent://public/default/t1',
batchingEnabled: false,
});
for (let i = 0; i < 10; i += 1) {
const msg = `my-message-${i}`;
producer.send({
data: Buffer.from(msg),
});
console.log(`Sent message: ${msg}`);
}
await producer.flush();
await producer.close();
await client.close();
})();
$ node producer.js
Sent message: my-message-0
Sent message: my-message-1
Sent message: my-message-2
Sent message: my-message-3
Sent message: my-message-4
Sent message: my-message-5
Sent message: my-message-6
Sent message: my-message-7
Sent message: my-message-8
Sent message: my-message-9
(node:18572) UnhandledPromiseRejectionWarning: Error: Failed to send message:
AlreadyClosed
(Use `node --trace-warnings ...` to show where the warning was created)
(node:18572) UnhandledPromiseRejectionWarning: Unhandled promise rejection.
This error originated either by throwing inside of an async function without a
catch block, or by rejecting a promise which was not handled with .catch(). To
terminate the node process on unhandled promise rejection, use the CLI flag
`--unhandled-rejections=strict` (see
https://jpn01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnodejs.org%2Fapi%2Fcli.html%23cli_unhandled_rejections_mode&data=05%7C01%7Cyumizush%40yahoo-corp.jp%7Cad44a31ff988434d3f1f08daa82142b0%7Ca208d369cd4e4f87b11998eaf31df2c3%7C1%7C0%7C638007158045043149%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=d31GWwFv0GBd3ggHyez5T5YWTPq0SFxW%2B6YqTSOnyDw%3D&reserved=0).
(rejection id: 1)
(node:18572) [DEP0018] DeprecationWarning: Unhandled promise rejections are
deprecated. In the future, promise rejections that are not handled will
terminate the Node.js process with a non-zero exit code.
(node:18572) UnhandledPromiseRejectionWarning: Error: Failed to send message:
AlreadyClosed
(node:18572) UnhandledPromiseRejectionWarning: Unhandled promise rejection.
This error originated either by throwing inside of an async function without a
catch block, or by rejecting a promise which was not handled with .catch(). To
terminate the node process on unhandled promise rejection, use the CLI flag
`--unhandled-rejections=strict` (see
https://jpn01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnodejs.org%2Fapi%2Fcli.html%23cli_unhandled_rejections_mode&data=05%7C01%7Cyumizush%40yahoo-corp.jp%7Cad44a31ff988434d3f1f08daa82142b0%7Ca208d369cd4e4f87b11998eaf31df2c3%7C1%7C0%7C638007158045043149%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=d31GWwFv0GBd3ggHyez5T5YWTPq0SFxW%2B6YqTSOnyDw%3D&reserved=0).
(rejection id: 2)
(node:18572) UnhandledPromiseRejectionWarning: Error: Failed to send message:
AlreadyClosed
(node:18572) UnhandledPromiseRejectionWarning: Unhandled promise rejection.
This error originated either by throwing inside of an async function without a
catch block, or by rejecting a promise which was not handled with .catch(). To
terminate the node process on unhandled promise rejection, use the CLI flag
`--unhandled-rejections=strict` (see
https://jpn01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnodejs.org%2Fapi%2Fcli.html%23cli_unhandled_rejections_mode&data=05%7C01%7Cyumizush%40yahoo-corp.jp%7Cad44a31ff988434d3f1f08daa82142b0%7Ca208d369cd4e4f87b11998eaf31df2c3%7C1%7C0%7C638007158045043149%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=d31GWwFv0GBd3ggHyez5T5YWTPq0SFxW%2B6YqTSOnyDw%3D&reserved=0).
(rejection id: 3)
(node:18572) UnhandledPromiseRejectionWarning: Error: Failed to send message:
AlreadyClosed
(node:18572) UnhandledPromiseRejectionWarning: Unhandled promise rejection.
This error originated either by throwing inside of an async function without a
catch block, or by rejecting a promise which was not handled with .catch(). To
terminate the node process on unhandled promise rejection, use the CLI flag
`--unhandled-rejections=strict` (see
https://jpn01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnodejs.org%2Fapi%2Fcli.html%23cli_unhandled_rejections_mode&data=05%7C01%7Cyumizush%40yahoo-corp.jp%7Cad44a
Re: [PIP-78] Reduce redundant producers from partitioned producer
Enrico,
Thank you for your clarification.
> I would add a configuration parameter to allow the client to use this
> feature.
> So:
> - feature enabled -> everything works like in your patch
> - feature disabled -> the broker ignores *partial_producer_supported *and
> runs as before
Okay. That makes sense to me.
If you have no more concerns, I'll fix the change to address your comments.
Regards,
--
Yuri Mizushima
yumiz...@yahoo-corp.jp
On 2021/12/14 19:01, "Enrico Olivelli" wrote:
sorry for the late reply, I missed this message.
Il giorno mar 7 dic 2021 alle ore 05:56 Yuri Mizushima <
yumiz...@yahoo-corp.jp> ha scritto:
> Enrico,
> Thank you for your comment.
>
> > IIUC with this change the client will control which metrics are reported
> by
> > the broker ?
>
> From the protocol perspective, yes.
> However, the main point of this change is not to "control" metrics by the
> client side,
> but to make the broker aggregate partitioned topic's producer metrics
> explicitly.
>
> Do you suggest adding a broker config that
> configures whether partitioned producer stats are aggregated by
> producerName instead of
> introducing a backward compatibility key (i.e.,
> partial_producer_supported) on the client-side?
>
Not exactly.
I would add a configuration parameter to allow the client to use this
feature.
So:
- feature enabled -> everything works like in your patch
- feature disabled -> the broker ignores *partial_producer_supported *and
runs as before
This way if you want to let the client aggregate the stats you can do it
but by default clients are not able to interact with the metrics format.
In multi-tenancy environments, like in large clusters shared by many teams,
you do not want that the client is able to change the way the broker
reports its metrics
or in general that the client is able to alter the monitoring systems.
Enrico
>
> It is simple. However, I think we can't enable the config until all
> clients are updated.
>
> What do you think?
> Regards,
> --
> Yuri Mizushima
> yumiz...@yahoo-corp.jp
>
>
> On 2021/12/02 17:37, "Enrico Olivelli" wrote:
>
> Yuri,
> IIUC with this change the client will control which metrics are
> reported by
> the broker ?
>
> I am not sure it is a good idea, because metrics are usually managed
> by the
> owners of the brokers, who sometimes are not the same who run the
> clients.
>
> Also, I am not sure if this way it is possible for the client to force
> the
> Broker to create many metrics and create some kind of damage.
>
> Would it be better to add a Broker configuration flag to turn on this
> feature ? I mean to allow the client to select the type of metrics ?
>
>
> Enrico
>
>
> Il giorno gio 2 dic 2021 alle ore 03:00 Yuri Mizushima <
> yumiz...@yahoo-corp.jp> ha scritto:
>
> > Do you have any comments?
> > If there are no comments by Dec. 7, I will close the discussion and
> rebase
> > the PR commit to current master.
> >
> > Regards,
> >
> > --
> > Yuri Mizushima
> > yumiz...@yahoo-corp.jp
> >
> >
> > On 2021/11/16 15:46, "Yuri Mizushima"
> wrote:
> >
> > Dear Pulsar community,
> >
> > I have created a new PR
> >
>
https://jpn01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fpulsar%2Fpull%2F12401&data=04%7C01%7Cyumizush%40yahoo-corp.jp%7C14ab44acc2be4e8cd90e08d9bee8bf9b%7Ca208d369cd4e4f87b11998eaf31df2c3%7C1%7C0%7C637750729124731851%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=POhcguG9KtlOMqyrdZtiLD9KKBGYDBfbDiq8g88cKAo%3D&reserved=0
> > for stats aggregation,
> > but I didn't discuss about the wire protocol change. I hope we
> will
> > discuss it here.
> >
> > Currently, partitioned producer can't aggregate by any key such
> as
> > cnx, producerId, producerName, and so on.
> > I think we need to add any aggregation system.
> > Therefore, added new aggregation policy as producerName (with
> client
Re: [PIP-78] Reduce redundant producers from partitioned producer
Any thought on this?
--
Yuri Mizushima
yumiz...@yahoo-corp.jp
On 2021/12/07 13:57, "Yuri Mizushima" wrote:
Enrico,
Thank you for your comment.
> IIUC with this change the client will control which metrics are reported
by
> the broker ?
From the protocol perspective, yes.
However, the main point of this change is not to "control" metrics by the
client side,
but to make the broker aggregate partitioned topic's producer metrics
explicitly.
Do you suggest adding a broker config that
configures whether partitioned producer stats are aggregated by
producerName instead of
introducing a backward compatibility key (i.e., partial_producer_supported)
on the client-side?
It is simple. However, I think we can't enable the config until all clients
are updated.
What do you think?
Regards,
--
Yuri Mizushima
yumiz...@yahoo-corp.jp
On 2021/12/02 17:37, "Enrico Olivelli" wrote:
Yuri,
IIUC with this change the client will control which metrics are
reported by
the broker ?
I am not sure it is a good idea, because metrics are usually managed by
the
owners of the brokers, who sometimes are not the same who run the
clients.
Also, I am not sure if this way it is possible for the client to force
the
Broker to create many metrics and create some kind of damage.
Would it be better to add a Broker configuration flag to turn on this
feature ? I mean to allow the client to select the type of metrics ?
Enrico
Il giorno gio 2 dic 2021 alle ore 03:00 Yuri Mizushima <
yumiz...@yahoo-corp.jp> ha scritto:
> Do you have any comments?
> If there are no comments by Dec. 7, I will close the discussion and
rebase
> the PR commit to current master.
>
> Regards,
>
> --
> Yuri Mizushima
> yumiz...@yahoo-corp.jp
>
>
> On 2021/11/16 15:46, "Yuri Mizushima" wrote:
>
> Dear Pulsar community,
>
> I have created a new PR
>
https://jpn01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fpulsar%2Fpull%2F12401&data=04%7C01%7Cyumizush%40yahoo-corp.jp%7C7e19573ee36d4edf3d6c08d9b93df247%7Ca208d369cd4e4f87b11998eaf31df2c3%7C1%7C0%7C637744498709588901%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=qcUOpCoQZZH%2B%2FlobXMhgcNXmlaSyaahXMel9YBmGEWs%3D&reserved=0
> for stats aggregation,
> but I didn't discuss about the wire protocol change. I hope we
will
> discuss it here.
>
> Currently, partitioned producer can't aggregate by any key such as
> cnx, producerId, producerName, and so on.
> I think we need to add any aggregation system.
> Therefore, added new aggregation policy as producerName (with
client
> side implementation).
>
> New protocol field partial_producer_supported is not used for
stats
> aggregation. It is used for backward compatibility.
>
>
https://jpn01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fpulsar%2Fpull%2F12401%2Ffiles%23diff-f29399fed32e0916cf28452ba71078a3ae5ed77bbaef9f52a925165d8ee66cfdR489&data=04%7C01%7Cyumizush%40yahoo-corp.jp%7C7e19573ee36d4edf3d6c08d9b93df247%7Ca208d369cd4e4f87b11998eaf31df2c3%7C1%7C0%7C637744498709588901%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=wFBZwBwQ4yGUF9VlUdoOhKb4K54ZF0yGwv9kycvxoqI%3D&reserved=0
>
> In my understanding, if introduce new stats aggregation key to
client
> side,
> need a way to determine whether the feature is enabled at client
side.
> For example, whether the producer has specific field or metadata,
> the version (e.g. protocol version) is greater than threshold,
etc.
>
> Of course, if we can introduce aggregation feature without adding
any
> new key or implementations from client side,
> we can support the feature not only new client but also old one.
>
> I'm looking forward to your opinions or suggestions to this PR.
>
> Regards,
> --
> Yuri Mizushima
> yumiz...@yahoo-corp.jp
>
>
> On 2021/05/11 14:26, "Yuri Mizushima"
wrote:
>
>
> Dear Pulsar Community,
>
> > I will submit the next PR about PartitionedTopicStats later
Re: [PIP-78] Reduce redundant producers from partitioned producer
Enrico,
Thank you for your comment.
> IIUC with this change the client will control which metrics are reported by
> the broker ?
From the protocol perspective, yes.
However, the main point of this change is not to "control" metrics by the
client side,
but to make the broker aggregate partitioned topic's producer metrics
explicitly.
Do you suggest adding a broker config that
configures whether partitioned producer stats are aggregated by producerName
instead of
introducing a backward compatibility key (i.e., partial_producer_supported) on
the client-side?
It is simple. However, I think we can't enable the config until all clients are
updated.
What do you think?
Regards,
--
Yuri Mizushima
yumiz...@yahoo-corp.jp
On 2021/12/02 17:37, "Enrico Olivelli" wrote:
Yuri,
IIUC with this change the client will control which metrics are reported by
the broker ?
I am not sure it is a good idea, because metrics are usually managed by the
owners of the brokers, who sometimes are not the same who run the clients.
Also, I am not sure if this way it is possible for the client to force the
Broker to create many metrics and create some kind of damage.
Would it be better to add a Broker configuration flag to turn on this
feature ? I mean to allow the client to select the type of metrics ?
Enrico
Il giorno gio 2 dic 2021 alle ore 03:00 Yuri Mizushima <
yumiz...@yahoo-corp.jp> ha scritto:
> Do you have any comments?
> If there are no comments by Dec. 7, I will close the discussion and rebase
> the PR commit to current master.
>
> Regards,
>
> --
> Yuri Mizushima
> yumiz...@yahoo-corp.jp
>
>
> On 2021/11/16 15:46, "Yuri Mizushima" wrote:
>
> Dear Pulsar community,
>
> I have created a new PR
>
https://jpn01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fpulsar%2Fpull%2F12401&data=04%7C01%7Cyumizush%40yahoo-corp.jp%7Ca56c759c260a46d176b408d9b56eb442%7Ca208d369cd4e4f87b11998eaf31df2c3%7C1%7C0%7C637740310655072887%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=JVOJsF5wTRoAyvy%2F8kyddBJP1XsdBv3YcIt2FP4OkHI%3D&reserved=0
> for stats aggregation,
> but I didn't discuss about the wire protocol change. I hope we will
> discuss it here.
>
> Currently, partitioned producer can't aggregate by any key such as
> cnx, producerId, producerName, and so on.
> I think we need to add any aggregation system.
> Therefore, added new aggregation policy as producerName (with client
> side implementation).
>
> New protocol field partial_producer_supported is not used for stats
> aggregation. It is used for backward compatibility.
>
>
https://jpn01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fpulsar%2Fpull%2F12401%2Ffiles%23diff-f29399fed32e0916cf28452ba71078a3ae5ed77bbaef9f52a925165d8ee66cfdR489&data=04%7C01%7Cyumizush%40yahoo-corp.jp%7Ca56c759c260a46d176b408d9b56eb442%7Ca208d369cd4e4f87b11998eaf31df2c3%7C1%7C0%7C637740310655072887%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=L%2B4JftwNnN7dUtck8wjWhwvqKhyFpyUcOcgsoinwyfQ%3D&reserved=0
>
> In my understanding, if introduce new stats aggregation key to client
> side,
> need a way to determine whether the feature is enabled at client side.
> For example, whether the producer has specific field or metadata,
> the version (e.g. protocol version) is greater than threshold, etc.
>
> Of course, if we can introduce aggregation feature without adding any
> new key or implementations from client side,
> we can support the feature not only new client but also old one.
>
> I'm looking forward to your opinions or suggestions to this PR.
>
> Regards,
> --
> Yuri Mizushima
> yumiz...@yahoo-corp.jp
>
>
> On 2021/05/11 14:26, "Yuri Mizushima" wrote:
>
>
> Dear Pulsar Community,
>
> > I will submit the next PR about PartitionedTopicStats later.
> I submitted the next PR for this PIP. If you have any suggestions,
> please comment to this PR.
>
>
https://jpn01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fpulsar%2Fpull%2F10534&data=04%7C01%7Cyumizush%40yahoo-corp.jp%7Ca56c759c260a46d176b408d9b56eb442%7Ca208d369cd4e4f87b11998eaf31df2c3%7C1%7C0%7C637740310655072887%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTi
Re: [PIP-78] Reduce redundant producers from partitioned producer
Do you have any comments?
If there are no comments by Dec. 7, I will close the discussion and rebase the
PR commit to current master.
Regards,
--
Yuri Mizushima
yumiz...@yahoo-corp.jp
On 2021/11/16 15:46, "Yuri Mizushima" wrote:
Dear Pulsar community,
I have created a new PR
https://jpn01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fpulsar%2Fpull%2F12401&data=04%7C01%7Cyumizush%40yahoo-corp.jp%7Cd1cf3f01dc4b48994a0008d9a8cc7154%7Ca208d369cd4e4f87b11998eaf31df2c3%7C1%7C0%7C637726419802266126%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=Sn57dCx1iItNEMArKWAXxE9frCbmQClw%2Fts0nmMkwyw%3D&reserved=0
for stats aggregation,
but I didn't discuss about the wire protocol change. I hope we will discuss
it here.
Currently, partitioned producer can't aggregate by any key such as cnx,
producerId, producerName, and so on.
I think we need to add any aggregation system.
Therefore, added new aggregation policy as producerName (with client side
implementation).
New protocol field partial_producer_supported is not used for stats
aggregation. It is used for backward compatibility.
https://jpn01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fpulsar%2Fpull%2F12401%2Ffiles%23diff-f29399fed32e0916cf28452ba71078a3ae5ed77bbaef9f52a925165d8ee66cfdR489&data=04%7C01%7Cyumizush%40yahoo-corp.jp%7Cd1cf3f01dc4b48994a0008d9a8cc7154%7Ca208d369cd4e4f87b11998eaf31df2c3%7C1%7C0%7C637726419802266126%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=QswqgZHSeHJArmgAdBx1wPrm6rX4l4lTf39hs57g6NI%3D&reserved=0
In my understanding, if introduce new stats aggregation key to client side,
need a way to determine whether the feature is enabled at client side.
For example, whether the producer has specific field or metadata,
the version (e.g. protocol version) is greater than threshold, etc.
Of course, if we can introduce aggregation feature without adding any new
key or implementations from client side,
we can support the feature not only new client but also old one.
I'm looking forward to your opinions or suggestions to this PR.
Regards,
--
Yuri Mizushima
yumiz...@yahoo-corp.jp
On 2021/05/11 14:26, "Yuri Mizushima" wrote:
Dear Pulsar Community,
> I will submit the next PR about PartitionedTopicStats later.
I submitted the next PR for this PIP. If you have any suggestions, please
comment to this PR.
https://jpn01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fpulsar%2Fpull%2F10534&data=04%7C01%7Cyumizush%40yahoo-corp.jp%7Cd1cf3f01dc4b48994a0008d9a8cc7154%7Ca208d369cd4e4f87b11998eaf31df2c3%7C1%7C0%7C637726419802266126%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=cHqzaY1H4N%2FaQEemy5ZRQSBAiXa3FDXDuFLyoJ4dRtA%3D&reserved=0
Regards,
--
Yuri Mizushima
yumiz...@yahoo-corp.jp
"Yuri Mizushima" wrote:
Dear Pulsar Community,
I submitted the PR for this PIP.
https://jpn01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fpulsar%2Fpull%2F10279&data=04%7C01%7Cyumizush%40yahoo-corp.jp%7Cd1cf3f01dc4b48994a0008d9a8cc7154%7Ca208d369cd4e4f87b11998eaf31df2c3%7C1%7C0%7C637726419802266126%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=AzmPY98arKmn7UM1%2F6FJMwOKOavPVC5xVG%2Br5FBJlH4%3D&reserved=0
This is a part of implementations.
I will submit the next PR about PartitionedTopicStats later.
Regards,
--
Yuri Mizushima
yumiz...@yahoo-corp.jp
"Yuri Mizushima" wrote:
Sijie,
After sending previous mail, I watched meeting recording and
understand about authn/authz issue.
Therefore, I updated the PIP document.
https://jpn01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fpulsar%2Fwiki%2FPIP-79%253A-Reduce-redundant-producers-from-partitioned-producer&data=04%7C01%7Cyumizush%40yahoo-corp.jp%7Cd1cf3f01dc4b48994a0008d9a8cc7154%7Ca208d369cd4e4f87b11998eaf31df2c3%7C1%7C0%7C637726419802266126%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=lZrLE4EhYfy7xUNmfUraIchYOkH%2B0HQaeVcKGaTBWLw%3D&reserved=0
Regards,
--
Yuri Mizushima
yumiz...@yahoo-corp.jp
"Yuri Mizushima" wrote:
Sijie,
> If the lazy-loading approach sounds attractive to you and you
like it,
> maybe the next step is to update the PIP, what do you think?
I think so
Re: [VOTE] Apache Pulsar 2.9.0 candidate 4
+1 (non-binding) * Verified checksum/signature * Ran release candidate validation on JDK8+macOS Thank you for being release manager. nits: Do we apply the https://github.com/apache/pulsar/pull/12659 commit to v2.9.0? In my environment, StringIndexOutOfBoundsException occurs when I create a tenant. I think it is not critical at least in my case because of https://github.com/apache/pulsar/blob/v2.9.0-candidate-4/pulsar-broker/src/main/java/org/apache/pulsar/broker/resourcegroup/ResourceGroupNamespaceConfigListener.java#L139-L141 . What do you think? ``` java.lang.StringIndexOutOfBoundsException: String index out of range: -1 at java.lang.String.substring(String.java:1931) ~[?:1.8.0_312] at org.apache.pulsar.broker.resources.NamespaceResources.pathIsFromNamespace(NamespaceResources.java:121) ~[org.apache.pulsar-pulsar-broker-common-2.9.0.jar:2.9.0] at org.apache.pulsar.broker.resourcegroup.ResourceGroupNamespaceConfigListener.accept(ResourceGroupNamespaceConfigListener.java:139) ~[org.apache.pulsar-pulsar-broker-2.9.0.jar:2.9.0] at org.apache.pulsar.broker.resourcegroup.ResourceGroupNamespaceConfigListener.accept(ResourceGroupNamespaceConfigListener.java:42) ~[org.apache.pulsar-pulsar-broker-2.9.0.jar:2.9.0] at org.apache.pulsar.metadata.impl.AbstractMetadataStore.lambda$receivedNotification$0(AbstractMetadataStore.java:167) ~[org.apache.pulsar-pulsar-metadata-2.9.0.jar:2.9.0] at java.util.concurrent.CopyOnWriteArrayList.forEach(CopyOnWriteArrayList.java:895) ~[?:1.8.0_312] at org.apache.pulsar.metadata.impl.AbstractMetadataStore.lambda$receivedNotification$1(AbstractMetadataStore.java:165) ~[org.apache.pulsar-pulsar-metadata-2.9.0.jar:2.9.0] at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1604) [?:1.8.0_312] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_312] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_312] at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [io.netty-netty-common-4.1.68.Final.jar:4.1.68.Final] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_312] ``` Regards, -- Yuri Mizushima yumiz...@yahoo-corp.jp "Enrico Olivelli" wrote: I have pushed in my personal dockerhub the docker images for this RC: eolivelli/pulsar-all:2.9.0rc4 eolivelli/pulsar:2.9.0rc4 Enrico Il giorno mer 17 nov 2021 alle ore 14:04 Enrico Olivelli < eolive...@gmail.com> ha scritto: > This is the fourth release candidate for Apache Pulsar, version 2.9.0. > > It fixes the following issues: > https://github.com/apache/pulsar/milestone/30?closed=1 > > *** Please download, test and vote on this release. This vote will stay open > for at least 72 hours *** > > Note that we are voting upon the source (tag), binaries are provided for > convenience. > > Source and binary files: > https://dist.apache.org/repos/dist/dev/pulsar/pulsar-2.9.0-candidate-4/ > > SHA-512 checksums: > > 659afa98e2475fc95f3126aab47d365e3e43a9d419c9cbe20c99520029f0b28aec2f471da5ccf47b92017a4c7943fc113a96232b075eb675e2b9df3601b2a7ad apache-pulsar-2.9.0-SNAPSHOT-bin.tar.gz > ed93aa1b57411c5d153bcd1433db6679ddb9b4d23896b39c1a1ad8edc70aaa7e95842a5770d6d55b466ab90500e17f50549673601556e767f7c12135ae360e85 apache-pulsar-2.9.0-SNAPSHOT-src.tar.gz > > Maven staging repo: > https://repository.apache.org/content/repositories/orgapachepulsar-1104/ > > The tag to be voted upon: > v2.9.0-candidate-4 (bdd57b21a66b81aab72c4ec39d516ffd2a769c35) > https://github.com/apache/pulsar/releases/tag/v2.9.0-candidate-4 > > Pulsar's KEYS file containing PGP keys we use to sign the release: > https://dist.apache.org/repos/dist/dev/pulsar/KEYS > > Please download the source package, and follow the README to build > and run the Pulsar standalone service. > > Enrico Olivelli > >
Re: [PIP-78] Reduce redundant producers from partitioned producer
Dear Pulsar community,
I have created a new PR https://github.com/apache/pulsar/pull/12401 for stats
aggregation,
but I didn't discuss about the wire protocol change. I hope we will discuss it
here.
Currently, partitioned producer can't aggregate by any key such as cnx,
producerId, producerName, and so on.
I think we need to add any aggregation system.
Therefore, added new aggregation policy as producerName (with client side
implementation).
New protocol field partial_producer_supported is not used for stats
aggregation. It is used for backward compatibility.
https://github.com/apache/pulsar/pull/12401/files#diff-f29399fed32e0916cf28452ba71078a3ae5ed77bbaef9f52a925165d8ee66cfdR489
In my understanding, if introduce new stats aggregation key to client side,
need a way to determine whether the feature is enabled at client side.
For example, whether the producer has specific field or metadata,
the version (e.g. protocol version) is greater than threshold, etc.
Of course, if we can introduce aggregation feature without adding any new key
or implementations from client side,
we can support the feature not only new client but also old one.
I'm looking forward to your opinions or suggestions to this PR.
Regards,
--
Yuri Mizushima
yumiz...@yahoo-corp.jp
On 2021/05/11 14:26, "Yuri Mizushima" wrote:
Dear Pulsar Community,
> I will submit the next PR about PartitionedTopicStats later.
I submitted the next PR for this PIP. If you have any suggestions, please
comment to this PR.
https://github.com/apache/pulsar/pull/10534
Regards,
--
Yuri Mizushima
yumiz...@yahoo-corp.jp
"Yuri Mizushima" wrote:
Dear Pulsar Community,
I submitted the PR for this PIP.
https://github.com/apache/pulsar/pull/10279
This is a part of implementations.
I will submit the next PR about PartitionedTopicStats later.
Regards,
--
Yuri Mizushima
yumiz...@yahoo-corp.jp
"Yuri Mizushima" wrote:
Sijie,
After sending previous mail, I watched meeting recording and understand
about authn/authz issue.
Therefore, I updated the PIP document.
https://github.com/apache/pulsar/wiki/PIP-79%3A-Reduce-redundant-producers-from-partitioned-producer
Regards,
--
Yuri Mizushima
yumiz...@yahoo-corp.jp
"Yuri Mizushima" wrote:
Sijie,
> If the lazy-loading approach sounds attractive to you and you
like it,
> maybe the next step is to update the PIP, what do you think?
I think so too. I will update the PIP after discussing the
authn/authz issue.
Regards,
--
Yuri Mizushima
yumiz...@yahoo-corp.jp
"Sijie Guo" wrote:
Hi Yuri,
Regarding the authn/authz issue, @Matteo Merli
can
probably chime in more about that part.
If the lazy-loading approach sounds attractive to you and you
like it,
maybe the next step is to update the PIP, what do you think?
- Sijie
On Mon, Feb 8, 2021 at 6:57 PM Yuri Mizushima
wrote:
> Michael,
>
> Thank you for your comment!
>
> > Which Pulsar Clients will benefit from this proposal?
> I think that this proposal will be useful to any clients.
> In my schedule, if this proposal is accepted then I will
implement this
> feature to Java client.
> If needed, then implement same feature to other clients such
as C++, Go,
> etc.
>
> Regards,
> --
> Yuri Mizushima
> yumiz...@yahoo-corp.jp
>
>
> "Michael Marshall" wrote:
>
> Hi Yuri and Sijie,
>
> I definitely like the idea of lazily creating producers
as well as
> introducing a way to provide custom routing logic.
>
> Which Pulsar Clients will benefit from this proposal? I’d
love to see
> this feature in the go client.
>
> Thanks,
> Michael Marshall
>
> > On Feb 7, 2021, at 9:53 PM, Yuri Mizushima
> wrote:
> >
> > Sijie,
> >
> > Thank you for sharing!
> >
> > First, I considered your suggestion.
> > I think these implemen
Re: [VOTE] Pulsar Release 2.8.0 Candidate 2
+1 (non-binding) I checked * shasum and signature * pub/sub and Java Functions * connectors * stateful functions Thank you for your work! -- Yuri Mizushima yumiz...@yahoo-corp.jp "PengHui Li" wrote: Sorry for the typo, ``` 9f7304c3d35db97998a16a46362433c43c61d20eb3d2f51eb43175fa79728925175d4646c2cd72ad39fc614e425814af1b9a8a8635545cea291a28e28d57 apache-pulsar-2.8.0-SNAPSHOT-bin.tar.gz 8fe7e30c3cda0993114996dced0bf49986365a76bcc69b2e68160d6a9e52c9e9a2bc174eba7eb0c7467f6a62e009ee7d8a3457e0e46657ecf8f79f665549a176 apache-pulsar-2.8.0-SNAPSHOT-src.tar.gz ``` it should be ``` 9f7304c3d35db97998a16a46362433c43c61d20eb3d2f51eb43175fa79728925175d4646c2cd72ad39fc614e425814af1b9a8a8635545cea291a28e28d57 apache-pulsar-2.8.0-bin.tar.gz 8fe7e30c3cda0993114996dced0bf49986365a76bcc69b2e68160d6a9e52c9e9a2bc174eba7eb0c7467f6a62e009ee7d8a3457e0e46657ecf8f79f665549a176 apache-pulsar-2.8.0-src.tar.gz ``` Thanks, Penghui PengHui Li 于2021年6月8日周二 下午6:37写道： > This is the second release candidate for Apache Pulsar, version 2.8.0. > > It fixes the following issues: > > https://github.com/apache/pulsar/pulls?q=is%3Apr+milestone%3A2.8.0+-label%3Arelease%2F2.7.1+-label%3Arelease%2F2.7.2+is%3Aclosed > > *** Please download, test and vote on this release. This vote will stay > open > for at least 72 hours *** > > Note that we are voting upon the source (tag), binaries are provided for > convenience. > > Source and binary files: > https://dist.apache.org/repos/dist/dev/pulsar/pulsar-2.8.0-candidate-2/ > > SHA-512 checksums: > > > 9f7304c3d35db97998a16a46362433c43c61d20eb3d2f51eb43175fa79728925175d4646c2cd72ad39fc614e425814af1b9a8a8635545cea291a28e28d57 > apache-pulsar-2.8.0-SNAPSHOT-bin.tar.gz > > > 8fe7e30c3cda0993114996dced0bf49986365a76bcc69b2e68160d6a9e52c9e9a2bc174eba7eb0c7467f6a62e009ee7d8a3457e0e46657ecf8f79f665549a176 > apache-pulsar-2.8.0-SNAPSHOT-src.tar.gz > > Maven staging repo: > https://repository.apache.org/content/repositories/orgapachepulsar-1089/ > > The tag to be voted upon: > v2.8.0-candidate-2 (63126b9d6606863c1b889f75737623e1fa9e1638) > https://github.com/apache/pulsar/releases/tag/v2.8.0-candidate-2 > > Pulsar's KEYS file containing PGP keys we use to sign the release: > https://dist.apache.org/repos/dist/dev/pulsar/KEYS > > Please download the the source package, and follow the README to build > and run the Pulsar standalone service. >
Re: [VOTE] Pulsar Release 2.6.4 Candidate 1
+1 (non-binding) I checked * shasum and signature * pub/sub and Java Functions * connectors * stateful functions and found no problem. Also, I checked the commits and found https://github.com/apache/pulsar/pull/9172 in this release. Thank you for your work! -- Yuri Mizushima yumiz...@yahoo-corp.jp "Lari Hotari" wrote: Please download, test and vote on this release asap. BR, Lari On Fri, May 28, 2021 at 1:09 PM Lari Hotari wrote: > This is the first release candidate for Apache Pulsar, version 2.6.4. > > It contains the following changes: > https://github.com/apache/pulsar/compare/v2.6.3...v2.6.4-candidate-1 > > The fix for CVE-2021-22160 [1] made as part of PR #9172 [2] has been > cherry-picked to branch-2.6 in commit 67e7e0cd [3]. > It is included in this release. > > *** Please download, test and vote on this release. This vote will stay > open for at least 72 hours *** > > Note that we are voting upon the source (tag), binaries are provided for > convenience. > > Source and binary files: > https://dist.apache.org/repos/dist/dev/pulsar/pulsar-2.6.4-candidate-1/ > > SHA-512 checksums: > 1a2eaf978d07412668a51fb60d0635a4d5b40d5e9af1a1421100f784902e6a7f02c34dd4d579d58d97fb1b269cc323007943dbf4c3ad4c123a0a262522f6697e > apache-pulsar-2.6.4-bin.tar.gz > c4f415688655ea531724c2d1e82d13a5a9d3d772830860c37213b5ce5bdb1ec3b79ead65e5b1b84015bfe6cf1fdef80b8555bf96b3b9e28815659a073b255342 > apache-pulsar-2.6.4-src.tar.gz > > Maven staging repo: > https://repository.apache.org/content/repositories/orgapachepulsar-1082/ > > The tag to be voted upon: > v2.6.4-candidate-1 (a3f0b88658116ab2223d2e0243b0ee2d33ae8c63) > https://github.com/apache/pulsar/releases/tag/v2.6.4-candidate-1 > > Pulsar's KEYS file containing PGP keys we use to sign the release: > https://dist.apache.org/repos/dist/dev/pulsar/KEYS > > Please download the source package, and follow the README to build and run > the Pulsar standalone service. > > [1] > https://lists.apache.org/thread.html/r347650d15a3e9c5f58b83e918b6ad6dedc2a63d3eb63da8e6a7be87e%40%3Cdev.pulsar.apache.org%3E > [2] https://github.com/apache/pulsar/pull/9172 > [3] https://github.com/apache/pulsar/commit/67e7e0cd >
Re: [PIP-78] Reduce redundant producers from partitioned producer
Dear Pulsar Community,
> I will submit the next PR about PartitionedTopicStats later.
I submitted the next PR for this PIP. If you have any suggestions, please
comment to this PR.
https://github.com/apache/pulsar/pull/10534
Regards,
--
Yuri Mizushima
yumiz...@yahoo-corp.jp
"Yuri Mizushima" wrote:
Dear Pulsar Community,
I submitted the PR for this PIP.
https://github.com/apache/pulsar/pull/10279
This is a part of implementations.
I will submit the next PR about PartitionedTopicStats later.
Regards,
--
Yuri Mizushima
yumiz...@yahoo-corp.jp
"Yuri Mizushima" wrote:
Sijie,
After sending previous mail, I watched meeting recording and understand
about authn/authz issue.
Therefore, I updated the PIP document.
https://github.com/apache/pulsar/wiki/PIP-79%3A-Reduce-redundant-producers-from-partitioned-producer
Regards,
--
Yuri Mizushima
yumiz...@yahoo-corp.jp
"Yuri Mizushima" wrote:
Sijie,
> If the lazy-loading approach sounds attractive to you and you
like it,
> maybe the next step is to update the PIP, what do you think?
I think so too. I will update the PIP after discussing the
authn/authz issue.
Regards,
--
Yuri Mizushima
yumiz...@yahoo-corp.jp
"Sijie Guo" wrote:
Hi Yuri,
Regarding the authn/authz issue, @Matteo Merli
can
probably chime in more about that part.
If the lazy-loading approach sounds attractive to you and you
like it,
maybe the next step is to update the PIP, what do you think?
- Sijie
On Mon, Feb 8, 2021 at 6:57 PM Yuri Mizushima
wrote:
> Michael,
>
> Thank you for your comment!
>
> > Which Pulsar Clients will benefit from this proposal?
> I think that this proposal will be useful to any clients.
> In my schedule, if this proposal is accepted then I will
implement this
> feature to Java client.
> If needed, then implement same feature to other clients such
as C++, Go,
> etc.
>
> Regards,
> --
> Yuri Mizushima
> yumiz...@yahoo-corp.jp
>
>
> "Michael Marshall" wrote:
>
> Hi Yuri and Sijie,
>
> I definitely like the idea of lazily creating producers
as well as
> introducing a way to provide custom routing logic.
>
> Which Pulsar Clients will benefit from this proposal? I’d
love to see
> this feature in the go client.
>
> Thanks,
> Michael Marshall
>
> > On Feb 7, 2021, at 9:53 PM, Yuri Mizushima
> wrote:
> >
> > Sijie,
> >
> > Thank you for sharing!
> >
> > First, I considered your suggestion.
> > I think these implementations sound good.
> >
> > I think we should consider the State of partitioned
producer: Ready,
> Connecting, etc.
> > Currently, partitioned producer gets "Ready" only when
all producers
> connect to Broker correctly.
> >
>
https://github.com/apache/pulsar/blob/fa41d02bebfd841767846240f3ae574047f118f0/pulsar-client/src/main/java/org/apache/pulsar/client/impl/PartitionedProducerImpl.java#L146
> > It seems that we should change meaning of state (or
change handling)
> if we introduce the lazy-load feature.
> > To guarantee the message ordering (e.g. using
partitionKey),
> partitioned producer should stop (or don't send messages to
be routed to
> unavailable partition) when producer can't connect to one of
partition.
> >
> > Secondly, I considered Matteo's comments.
> > I couldn't understand well about issue of authn/authz.
Please tell
> me more detail.
> >
> > I wrote "connec
Re: [PIP-78] Reduce redundant producers from partitioned producer
Dear Pulsar Community,
I submitted the PR for this PIP.
https://github.com/apache/pulsar/pull/10279
This is a part of implementations.
I will submit the next PR about PartitionedTopicStats later.
Regards,
--
Yuri Mizushima
yumiz...@yahoo-corp.jp
"Yuri Mizushima" wrote:
Sijie,
After sending previous mail, I watched meeting recording and understand
about authn/authz issue.
Therefore, I updated the PIP document.
https://github.com/apache/pulsar/wiki/PIP-79%3A-Reduce-redundant-producers-from-partitioned-producer
Regards,
--
Yuri Mizushima
yumiz...@yahoo-corp.jp
"Yuri Mizushima" wrote:
Sijie,
> If the lazy-loading approach sounds attractive to you and you like it,
> maybe the next step is to update the PIP, what do you think?
I think so too. I will update the PIP after discussing the authn/authz
issue.
Regards,
--
Yuri Mizushima
yumiz...@yahoo-corp.jp
"Sijie Guo" wrote:
Hi Yuri,
Regarding the authn/authz issue, @Matteo Merli
can
probably chime in more about that part.
If the lazy-loading approach sounds attractive to you and you like
it,
maybe the next step is to update the PIP, what do you think?
- Sijie
On Mon, Feb 8, 2021 at 6:57 PM Yuri Mizushima
wrote:
> Michael,
>
> Thank you for your comment!
>
> > Which Pulsar Clients will benefit from this proposal?
> I think that this proposal will be useful to any clients.
> In my schedule, if this proposal is accepted then I will
implement this
> feature to Java client.
> If needed, then implement same feature to other clients such as
C++, Go,
> etc.
>
> Regards,
> --
> Yuri Mizushima
> yumiz...@yahoo-corp.jp
>
>
> "Michael Marshall" wrote:
>
> Hi Yuri and Sijie,
>
> I definitely like the idea of lazily creating producers as
well as
> introducing a way to provide custom routing logic.
>
> Which Pulsar Clients will benefit from this proposal? I’d
love to see
> this feature in the go client.
>
> Thanks,
> Michael Marshall
>
> > On Feb 7, 2021, at 9:53 PM, Yuri Mizushima
> wrote:
> >
> > Sijie,
> >
> > Thank you for sharing!
> >
> > First, I considered your suggestion.
> > I think these implementations sound good.
> >
> > I think we should consider the State of partitioned
producer: Ready,
> Connecting, etc.
> > Currently, partitioned producer gets "Ready" only when all
producers
> connect to Broker correctly.
> >
>
https://github.com/apache/pulsar/blob/fa41d02bebfd841767846240f3ae574047f118f0/pulsar-client/src/main/java/org/apache/pulsar/client/impl/PartitionedProducerImpl.java#L146
> > It seems that we should change meaning of state (or change
handling)
> if we introduce the lazy-load feature.
> > To guarantee the message ordering (e.g. using partitionKey),
> partitioned producer should stop (or don't send messages to be
routed to
> unavailable partition) when producer can't connect to one of
partition.
> >
> > Secondly, I considered Matteo's comments.
> > I couldn't understand well about issue of authn/authz.
Please tell
> me more detail.
> >
> > I wrote "connection" as number of producers which connect
to broker.
> Also, TCP connections between partitioned producer and broker
will be less
> than or equal to current in some cases. I'll show a case below.
> >
> > Suppose
> > * cluster has Broker0, 1, 2
> > * partitioned topic has 5 partitions
> > * limit conf is 3 partitions
> > * loadbalance partitions as below
> > - Broker0: partition-0, partition-1
> > - Broker1: partition-2
> > - Broker2: partition-
Re: [PIP-78] Reduce redundant producers from partitioned producer
Sijie,
After sending previous mail, I watched meeting recording and understand about
authn/authz issue.
Therefore, I updated the PIP document.
https://github.com/apache/pulsar/wiki/PIP-79%3A-Reduce-redundant-producers-from-partitioned-producer
Regards,
--
Yuri Mizushima
yumiz...@yahoo-corp.jp
"Yuri Mizushima" wrote:
Sijie,
> If the lazy-loading approach sounds attractive to you and you like it,
> maybe the next step is to update the PIP, what do you think?
I think so too. I will update the PIP after discussing the authn/authz
issue.
Regards,
--
Yuri Mizushima
yumiz...@yahoo-corp.jp
"Sijie Guo" wrote:
Hi Yuri,
Regarding the authn/authz issue, @Matteo Merli can
probably chime in more about that part.
If the lazy-loading approach sounds attractive to you and you like it,
maybe the next step is to update the PIP, what do you think?
- Sijie
On Mon, Feb 8, 2021 at 6:57 PM Yuri Mizushima
wrote:
> Michael,
>
> Thank you for your comment!
>
> > Which Pulsar Clients will benefit from this proposal?
> I think that this proposal will be useful to any clients.
> In my schedule, if this proposal is accepted then I will implement
this
> feature to Java client.
> If needed, then implement same feature to other clients such as C++,
Go,
> etc.
>
> Regards,
> --
> Yuri Mizushima
> yumiz...@yahoo-corp.jp
>
>
> "Michael Marshall" wrote:
>
> Hi Yuri and Sijie,
>
> I definitely like the idea of lazily creating producers as well as
> introducing a way to provide custom routing logic.
>
> Which Pulsar Clients will benefit from this proposal? I’d love to
see
> this feature in the go client.
>
> Thanks,
> Michael Marshall
>
> > On Feb 7, 2021, at 9:53 PM, Yuri Mizushima
> wrote:
> >
> > Sijie,
> >
> > Thank you for sharing!
> >
> > First, I considered your suggestion.
> > I think these implementations sound good.
> >
> > I think we should consider the State of partitioned producer:
Ready,
> Connecting, etc.
> > Currently, partitioned producer gets "Ready" only when all
producers
> connect to Broker correctly.
> >
>
https://github.com/apache/pulsar/blob/fa41d02bebfd841767846240f3ae574047f118f0/pulsar-client/src/main/java/org/apache/pulsar/client/impl/PartitionedProducerImpl.java#L146
> > It seems that we should change meaning of state (or change
handling)
> if we introduce the lazy-load feature.
> > To guarantee the message ordering (e.g. using partitionKey),
> partitioned producer should stop (or don't send messages to be routed
to
> unavailable partition) when producer can't connect to one of
partition.
> >
> > Secondly, I considered Matteo's comments.
> > I couldn't understand well about issue of authn/authz. Please
tell
> me more detail.
> >
> > I wrote "connection" as number of producers which connect to
broker.
> Also, TCP connections between partitioned producer and broker will be
less
> than or equal to current in some cases. I'll show a case below.
> >
> > Suppose
> > * cluster has Broker0, 1, 2
> > * partitioned topic has 5 partitions
> > * limit conf is 3 partitions
> > * loadbalance partitions as below
> > - Broker0: partition-0, partition-1
> > - Broker1: partition-2
> > - Broker2: partition-3, partition-4
> >
> > Currently, client will create 3 connections (Broker0, 1, 2). If
> client uses limit conf and elects partitions such as [0, 1, 2], then
client
> will create 2 connections (Broker0, 1). Of course, if client elects
> partitions such as [0, 2, 3], then client will still create 3
connections.
> >
> > I'd like to decrease number of producers. I think that
resources of
> broker will be improved slightly by this feature because broker has
list of
> producers by some classes such as ServerCnx, AbstractTopic.
Re: [VOTE] Pulsar Client Go Release 0.4.0 Candidate 1
+1 (non-binding) * check shasum * run simple producer/consumer/reader and found no problem. Thank you for your work! nits: Is this signature right? In my environment, checked it and found "No public key" error. https://dist.apache.org/repos/dist/dev/pulsar/pulsar-client-go-0.4.0-candidate-1/apache-pulsar-client-go-0.4.0-src.tar.gz.asc Regards, -- Yuri Mizushima yumiz...@yahoo-corp.jp "xiaolong ran" wrote: Hi everyone, Please review and vote on the release candidate #1 for the version 0.4.0, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) This is the first release candidate for Apache Pulsar Go client, version 0.4.0. It fixes the following issues: https://github.com/apache/pulsar-client-go/milestone/5?closed=1 Pulsar Client Go's KEYS file contains PGP keys we used to sign this release: https://dist.apache.org/repos/dist/release/pulsar/KEYS Please download these packages and review this release candidate: - Review release notes - Download the source package (verify shasum, and asc) and follow the README.md to build and run the pulsar-client-go. The vote will be open for at least 72 hours. It is adopted by majority approval, with at least 3 PMC affirmative votes. Source file: https://dist.apache.org/repos/dist/dev/pulsar/pulsar-client-go-0.4.0-candidate-1/apache-pulsar-client-go-0.4.0-src.tar.gz The tag to be voted upon: v0.4.0 https://github.com/apache/pulsar-client-go/tree/v0.4.0-candidate-1 SHA-512 checksums: 4ca22efb026f6a1248107d8e553c10e8ce2f22b5d624c719d8a50d49c4745df7b45ac5576c4f15a7cea6fcb963c6afd25e5e661f80e256f29b8eac4515aed3f5 apache-pulsar-client-go-0.4.0-src.tar.gz -- Thanks Xiaolong Ran
Re: [PIP-78] Reduce redundant producers from partitioned producer
Sijie,
> If the lazy-loading approach sounds attractive to you and you like it,
> maybe the next step is to update the PIP, what do you think?
I think so too. I will update the PIP after discussing the authn/authz issue.
Regards,
--
Yuri Mizushima
yumiz...@yahoo-corp.jp
"Sijie Guo" wrote:
Hi Yuri,
Regarding the authn/authz issue, @Matteo Merli can
probably chime in more about that part.
If the lazy-loading approach sounds attractive to you and you like it,
maybe the next step is to update the PIP, what do you think?
- Sijie
On Mon, Feb 8, 2021 at 6:57 PM Yuri Mizushima
wrote:
> Michael,
>
> Thank you for your comment!
>
> > Which Pulsar Clients will benefit from this proposal?
> I think that this proposal will be useful to any clients.
> In my schedule, if this proposal is accepted then I will implement this
> feature to Java client.
> If needed, then implement same feature to other clients such as C++, Go,
> etc.
>
> Regards,
> --
> Yuri Mizushima
> yumiz...@yahoo-corp.jp
>
>
> "Michael Marshall" wrote:
>
> Hi Yuri and Sijie,
>
> I definitely like the idea of lazily creating producers as well as
> introducing a way to provide custom routing logic.
>
> Which Pulsar Clients will benefit from this proposal? I’d love to see
> this feature in the go client.
>
> Thanks,
> Michael Marshall
>
> > On Feb 7, 2021, at 9:53 PM, Yuri Mizushima
> wrote:
> >
> > Sijie,
> >
> > Thank you for sharing!
> >
> > First, I considered your suggestion.
> > I think these implementations sound good.
> >
> > I think we should consider the State of partitioned producer: Ready,
> Connecting, etc.
> > Currently, partitioned producer gets "Ready" only when all producers
> connect to Broker correctly.
> >
>
https://github.com/apache/pulsar/blob/fa41d02bebfd841767846240f3ae574047f118f0/pulsar-client/src/main/java/org/apache/pulsar/client/impl/PartitionedProducerImpl.java#L146
> > It seems that we should change meaning of state (or change handling)
> if we introduce the lazy-load feature.
> > To guarantee the message ordering (e.g. using partitionKey),
> partitioned producer should stop (or don't send messages to be routed to
> unavailable partition) when producer can't connect to one of partition.
> >
> > Secondly, I considered Matteo's comments.
> > I couldn't understand well about issue of authn/authz. Please tell
> me more detail.
> >
> > I wrote "connection" as number of producers which connect to broker.
> Also, TCP connections between partitioned producer and broker will be less
> than or equal to current in some cases. I'll show a case below.
> >
> > Suppose
> > * cluster has Broker0, 1, 2
> > * partitioned topic has 5 partitions
> > * limit conf is 3 partitions
> > * loadbalance partitions as below
> > - Broker0: partition-0, partition-1
> > - Broker1: partition-2
> > - Broker2: partition-3, partition-4
> >
> > Currently, client will create 3 connections (Broker0, 1, 2). If
> client uses limit conf and elects partitions such as [0, 1, 2], then
client
> will create 2 connections (Broker0, 1). Of course, if client elects
> partitions such as [0, 2, 3], then client will still create 3 connections.
> >
> > I'd like to decrease number of producers. I think that resources of
> broker will be improved slightly by this feature because broker has list
of
> producers by some classes such as ServerCnx, AbstractTopic.
> >
>
https://github.com/apache/pulsar/blob/fa41d02bebfd841767846240f3ae574047f118f0/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ServerCnx.java#L1096-L1097
> >
>
https://github.com/apache/pulsar/blob/fa41d02bebfd841767846240f3ae574047f118f0/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/AbstractTopic.java#L577
> >
> > In my case, unspecified number of producers will connect to the same
> partitioned topic with different rate. We need to set the number of
> partitions according to the high-rate producer.
> > However, on the other hand, this number is excessively large for
> low-rate produc
Re: [PIP-78] Reduce redundant producers from partitioned producer
Michael,
Thank you for your comment!
> Which Pulsar Clients will benefit from this proposal?
I think that this proposal will be useful to any clients.
In my schedule, if this proposal is accepted then I will implement this feature
to Java client.
If needed, then implement same feature to other clients such as C++, Go, etc.
Regards,
--
Yuri Mizushima
yumiz...@yahoo-corp.jp
"Michael Marshall" wrote:
Hi Yuri and Sijie,
I definitely like the idea of lazily creating producers as well as
introducing a way to provide custom routing logic.
Which Pulsar Clients will benefit from this proposal? I’d love to see this
feature in the go client.
Thanks,
Michael Marshall
> On Feb 7, 2021, at 9:53 PM, Yuri Mizushima wrote:
>
> Sijie,
>
> Thank you for sharing!
>
> First, I considered your suggestion.
> I think these implementations sound good.
>
> I think we should consider the State of partitioned producer: Ready,
Connecting, etc.
> Currently, partitioned producer gets "Ready" only when all producers
connect to Broker correctly.
>
https://github.com/apache/pulsar/blob/fa41d02bebfd841767846240f3ae574047f118f0/pulsar-client/src/main/java/org/apache/pulsar/client/impl/PartitionedProducerImpl.java#L146
> It seems that we should change meaning of state (or change handling) if
we introduce the lazy-load feature.
> To guarantee the message ordering (e.g. using partitionKey), partitioned
producer should stop (or don't send messages to be routed to unavailable
partition) when producer can't connect to one of partition.
>
> Secondly, I considered Matteo's comments.
> I couldn't understand well about issue of authn/authz. Please tell me
more detail.
>
> I wrote "connection" as number of producers which connect to broker.
Also, TCP connections between partitioned producer and broker will be less than
or equal to current in some cases. I'll show a case below.
>
> Suppose
> * cluster has Broker0, 1, 2
> * partitioned topic has 5 partitions
> * limit conf is 3 partitions
> * loadbalance partitions as below
> - Broker0: partition-0, partition-1
> - Broker1: partition-2
> - Broker2: partition-3, partition-4
>
> Currently, client will create 3 connections (Broker0, 1, 2). If client
uses limit conf and elects partitions such as [0, 1, 2], then client will
create 2 connections (Broker0, 1). Of course, if client elects partitions such
as [0, 2, 3], then client will still create 3 connections.
>
> I'd like to decrease number of producers. I think that resources of
broker will be improved slightly by this feature because broker has list of
producers by some classes such as ServerCnx, AbstractTopic.
>
https://github.com/apache/pulsar/blob/fa41d02bebfd841767846240f3ae574047f118f0/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ServerCnx.java#L1096-L1097
>
https://github.com/apache/pulsar/blob/fa41d02bebfd841767846240f3ae574047f118f0/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/AbstractTopic.java#L577
>
> In my case, unspecified number of producers will connect to the same
partitioned topic with different rate. We need to set the number of partitions
according to the high-rate producer.
> However, on the other hand, this number is excessively large for low-rate
producers.
> I want to reduce such redundant producers for resource efficiency.
>
> Regards,
> --
> Yuri Mizushima
> yumiz...@yahoo-corp.jp
>
>
> "Sijie Guo" wrote:
>
> Hi Yuri,
>
> In today's community meeting, Matteo shared some of his thoughts about
this
> PIP.
>
> You can find some meeting notes here:
>
https://docs.google.com/document/d/19dXkVXeU2q_nHmkG8zURjKnYlvD96TbKf5KjYyASsOE/edit#bookmark=id.rezbt4xmjxpz
>
> Matteo can also chime in as well.
>
> - Sijie
>
>> On Sun, Jan 31, 2021 at 7:21 PM Yuri Mizushima
>> wrote:
>> Sijie,
>> Thank you for your reply!
>> I'll check it.
>> Regards,
>> --
>> Yuri Mizushima
>> yumiz...@yahoo-corp.jp
>> "Sijie Guo" wrote:
>> Yuri,
>> Thank you for bringing this up! This is a super helpful proposal!
>> The problem is very similar to what an RPC framework (like Finagle)
>> with
>> client-side load balancing has.
>> An RPC framework with a client-side load-balancing mechanism needs to
>> send
>> requests across multiple nodes. If you have an RPC
Re: [PIP-78] Reduce redundant producers from partitioned producer
Sijie, Thank you for sharing! First, I considered your suggestion. I think these implementations sound good. I think we should consider the State of partitioned producer: Ready, Connecting, etc. Currently, partitioned producer gets "Ready" only when all producers connect to Broker correctly. https://github.com/apache/pulsar/blob/fa41d02bebfd841767846240f3ae574047f118f0/pulsar-client/src/main/java/org/apache/pulsar/client/impl/PartitionedProducerImpl.java#L146 It seems that we should change meaning of state (or change handling) if we introduce the lazy-load feature. To guarantee the message ordering (e.g. using partitionKey), partitioned producer should stop (or don't send messages to be routed to unavailable partition) when producer can't connect to one of partition. Secondly, I considered Matteo's comments. I couldn't understand well about issue of authn/authz. Please tell me more detail. I wrote "connection" as number of producers which connect to broker. Also, TCP connections between partitioned producer and broker will be less than or equal to current in some cases. I'll show a case below. Suppose * cluster has Broker0, 1, 2 * partitioned topic has 5 partitions * limit conf is 3 partitions * loadbalance partitions as below - Broker0: partition-0, partition-1 - Broker1: partition-2 - Broker2: partition-3, partition-4 Currently, client will create 3 connections (Broker0, 1, 2). If client uses limit conf and elects partitions such as [0, 1, 2], then client will create 2 connections (Broker0, 1). Of course, if client elects partitions such as [0, 2, 3], then client will still create 3 connections. I'd like to decrease number of producers. I think that resources of broker will be improved slightly by this feature because broker has list of producers by some classes such as ServerCnx, AbstractTopic. https://github.com/apache/pulsar/blob/fa41d02bebfd841767846240f3ae574047f118f0/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ServerCnx.java#L1096-L1097 https://github.com/apache/pulsar/blob/fa41d02bebfd841767846240f3ae574047f118f0/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/AbstractTopic.java#L577 In my case, unspecified number of producers will connect to the same partitioned topic with different rate. We need to set the number of partitions according to the high-rate producer. However, on the other hand, this number is excessively large for low-rate producers. I want to reduce such redundant producers for resource efficiency. Regards, -- Yuri Mizushima yumiz...@yahoo-corp.jp "Sijie Guo" wrote: Hi Yuri, In today's community meeting, Matteo shared some of his thoughts about this PIP. You can find some meeting notes here: https://docs.google.com/document/d/19dXkVXeU2q_nHmkG8zURjKnYlvD96TbKf5KjYyASsOE/edit#bookmark=id.rezbt4xmjxpz Matteo can also chime in as well. - Sijie On Sun, Jan 31, 2021 at 7:21 PM Yuri Mizushima wrote: > Sijie, > > Thank you for your reply! > I'll check it. > > Regards, > -- > Yuri Mizushima > yumiz...@yahoo-corp.jp > > > "Sijie Guo" wrote: > > Yuri, > > Thank you for bringing this up! This is a super helpful proposal! > > The problem is very similar to what an RPC framework (like Finagle) > with > client-side load balancing has. > > An RPC framework with a client-side load-balancing mechanism needs to > send > requests across multiple nodes. If you have an RPC service that has > thousands of nodes, there are thousands of clients connecting to that > RPC > service. How to reduce the connections and how to effectively load > balance > requests across thousands of nodes are the problems that a client-side > loading technology needs to solve. If you think about "partition" as > "node" > and "partitioned producer" as "RPC client", the problem is exactly the > same. Finagle (the Twitter RPC framework) has implemented a lot of > client-side > load-balancing algorithms > <https://twitter.github.io/finagle/guide/Clients.html#load-balancing> > and > there are some great articles that you can reference > < > https://blog.twitter.com/engineering/en_us/topics/infrastructure/2019/daperture-load-balancer.html > > > . > > I agree with the direction of introducing a mechanism to reduce the > number > of producers in a partitioned topic producer. However, I have a concern > about introducing `.numPartitionsLimit(10)` directly to the producer >
Re: [PIP-78] Reduce redundant producers from partitioned producer
Sijie, Thank you for your reply! I'll check it. Regards, -- Yuri Mizushima yumiz...@yahoo-corp.jp "Sijie Guo" wrote: Yuri, Thank you for bringing this up! This is a super helpful proposal! The problem is very similar to what an RPC framework (like Finagle) with client-side load balancing has. An RPC framework with a client-side load-balancing mechanism needs to send requests across multiple nodes. If you have an RPC service that has thousands of nodes, there are thousands of clients connecting to that RPC service. How to reduce the connections and how to effectively load balance requests across thousands of nodes are the problems that a client-side loading technology needs to solve. If you think about "partition" as "node" and "partitioned producer" as "RPC client", the problem is exactly the same. Finagle (the Twitter RPC framework) has implemented a lot of client-side load-balancing algorithms <https://twitter.github.io/finagle/guide/Clients.html#load-balancing> and there are some great articles that you can reference <https://blog.twitter.com/engineering/en_us/topics/infrastructure/2019/daperture-load-balancer.html> . I agree with the direction of introducing a mechanism to reduce the number of producers in a partitioned topic producer. However, I have a concern about introducing `.numPartitionsLimit(10)` directly to the producer builder. It limits the possibility to implement different algorithms on selecting partitions. So instead of directly implementing the logic within the partitioned topic producer, I think the proposal can be broken into two parts: 1) Introduce some kind of lazy-loading mechanism in the partitioned producer to initialize the producers for partitions lazily. I.e., only initialize a producer when the message router selects a partition. 2) Implement a message router that only selects one or N partitions. In this way, the partitioned producer is only responsible for managing a collection of producers, and the message router is responsible for selecting the partitions. This allows people to be able to implement different message routers. We can even adopt the client-side load balancing algorithms from Finagle. Thanks, Sijie On Wed, Jan 27, 2021 at 7:18 PM Yuri Mizushima wrote: > I notice that PIP-78 has already assigned to another issue. > > https://mail-archives.apache.org/mod_mbox/pulsar-dev/202101.mbox/%3CCAG%3DTQOrPH49v9ToDE_aeQzEiDC%2BEgSR61ERoqanpWfQGvEB_Vw%40mail.gmail.com%3E > > So, I'll change the PIP number to 79. > > https://github.com/apache/pulsar/wiki/PIP-79%3A-Reduce-redundant-producers-from-partitioned-producer > > Regards, > -- > Yuri Mizushima > yumiz...@yahoo-corp.jp > > "Yuri Mizushima" wrote: > > Dear Pulsar community, > > When partitioned producer connects to partitioned topic, > sometimes doesn't need to connect to all of partitions depending on > rate, routing mode, etc. > So, I drafted a PIP about reducing redundant producers from > partitioned producer. > I'd like to use system resources (e.g. connections between Client and > Broker, memory usage of both Client and Broker) > more efficiently by this feature. > > https://github.com/apache/pulsar/wiki/PIP-78%3A-Reduce-redundant-producers-from-partitioned-producer > > Feel free to ask me any questions or suggestions, etc. > > Best regards, > -- > Yuri Mizushima > yumiz...@yahoo-corp.jp > > >
Re: [PIP-78] Reduce redundant producers from partitioned producer
I notice that PIP-78 has already assigned to another issue. https://mail-archives.apache.org/mod_mbox/pulsar-dev/202101.mbox/%3CCAG%3DTQOrPH49v9ToDE_aeQzEiDC%2BEgSR61ERoqanpWfQGvEB_Vw%40mail.gmail.com%3E So, I'll change the PIP number to 79. https://github.com/apache/pulsar/wiki/PIP-79%3A-Reduce-redundant-producers-from-partitioned-producer Regards, -- Yuri Mizushima yumiz...@yahoo-corp.jp "Yuri Mizushima" wrote: Dear Pulsar community, When partitioned producer connects to partitioned topic, sometimes doesn't need to connect to all of partitions depending on rate, routing mode, etc. So, I drafted a PIP about reducing redundant producers from partitioned producer. I'd like to use system resources (e.g. connections between Client and Broker, memory usage of both Client and Broker) more efficiently by this feature. https://github.com/apache/pulsar/wiki/PIP-78%3A-Reduce-redundant-producers-from-partitioned-producer Feel free to ask me any questions or suggestions, etc. Best regards, -- Yuri Mizushima yumiz...@yahoo-corp.jp
[PIP-78] Reduce redundant producers from partitioned producer
Dear Pulsar community, When partitioned producer connects to partitioned topic, sometimes doesn't need to connect to all of partitions depending on rate, routing mode, etc. So, I drafted a PIP about reducing redundant producers from partitioned producer. I'd like to use system resources (e.g. connections between Client and Broker, memory usage of both Client and Broker) more efficiently by this feature. https://github.com/apache/pulsar/wiki/PIP-78%3A-Reduce-redundant-producers-from-partitioned-producer Feel free to ask me any questions or suggestions, etc. Best regards, -- Yuri Mizushima yumiz...@yahoo-corp.jp
Re: [VOTE] Pulsar Node.js Client Release 1.1.0 Candidate 1
+1 non binding I checked: * unit test, end-to-end test * package build * produce/consume - by package index.js - by downloaded client via npm and found no problem. nits * whether check test code format Currently, lint npm script checks production code and test code, and throws some warning about test code is not formatted. I seem we will expect that lint returns success when release commit is created. Regards, Yuri -- Yuri Mizushima yumiz...@yahoo-corp.jp "Hiroyuki Sakai" wrote: This is the first release candidate for Apache Pulsar Node.js client, version 1.1.0. It fixes the following issues: https://github.com/apache/pulsar-client-node/milestone/3?closed=1 *** Please download, test and vote on this release. This vote will stay open for at least 72 hours *** Note that we are voting upon the source (tag), the npm package is provided for convenience. The tag to be voted upon: v1.1.0-rc.1 https://github.com/apache/pulsar-client-node/releases/tag/v1.1.0-rc.1 The npm package: https://www.npmjs.com/package/pulsar-client/v/1.1.0-rc.1 Please download the source files, and follow the README to build and run the Pulsar Node.js client. -- Hiroyuki Sakai Yahoo Japan Corp. E-mail: hsa...@yahoo-corp.jp
Re: [VOTE] Pulsar Manager release 0.1.0 Candidate 4
+1 (non-binding) I validated: * checksum and signature * package according to Release-Candidate-Validation documents and found no problem. Thank you for your work! Regards, Yuri. -- Yuri Mizushima yumiz...@yahoo-corp.jp "Guangning E" wrote: Hi everyone, Please review and vote on the release candidate #4 for the version 0.1.0, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) It fixes the following issues: https://github.com/apache/pulsar-manager/milestone/2?closed=1 The complete staging area is available for your review, which includes: * Release notes [1] * The official Apache source and binary distributions to be deployed to https://dist.apache.org/repos/dist/dev/pulsar/pulsar-manager/pulsar-manager-0.1.0-candidate-4/ [2] * Source code tag "v0.1.0-candidate-4" [4] with git sha512 bc25543b1693efc9eb1e037a1ae8cb68a1fe30edf69b179cf415a27dcc61fc4398928ea1a52053bcaa7baf51e0365a2bd3d609b712de4e4ebe8ae105602ac3ef pulsar-manager's KEYS file contains PGP keys we used to sign this release: https://dist.apache.org/repos/dist/dev/pulsar/KEYS Please download these packages and review this release candidate: - Review release notes - Download the source package (verify shasum, and asc) and follow the instructions to build and run the pulsar-manager frontend and backend service. - Download the binary package (verify shasum, and asc) and follow the instructions to run the pulsar-manager frontend and backend service. https://github.com/apache/pulsar-manager/wiki/Release-Candidate-Validation The vote will be open for at least 72 hours. It is adopted by majority approval, with at least 3 PMC affirmative votes. Source and binary files: https://dist.apache.org/repos/dist/dev/pulsar/pulsar-manager/pulsar-manager-0.1.0-candidate-4/apache-pulsar-manager-0.1.0-candidate-4-bin.tar.gz https://dist.apache.org/repos/dist/dev/pulsar/pulsar-manager/pulsar-manager-0.1.0-candidate-4/apache-pulsar-manager-0.1.0-candidate-4-src.tar.gz SHA-512 checksums: bc25543b1693efc9eb1e037a1ae8cb68a1fe30edf69b179cf415a27dcc61fc4398928ea1a52053bcaa7baf51e0365a2bd3d609b712de4e4ebe8ae105602ac3ef apache-pulsar-manager-0.1.0-candidate-4-bin.tar.gz da75944d288bc91a2dca17e5a13ad30300b7c433ecc6a55dd7b3e64db889a46019b0d027e5dbd2932cc90694e1fb526356e0739fde594b57e16a0675be876c6c apache-pulsar-manager-0.1.0-candidate-4-src.tar.gz Thanks, Guangning
Re: [VOTE] Pulsar Manager release 0.1.0 Candidate 3
0 (non-binding) I validated: * checksum and signature * package according to Release-Candidate-Validation documents and found no problem. However, I found some issues: * Some URL or documents include “streamnative”. Is this acceptable? - https://github.com/apache/pulsar-manager/search?q=streamnative&unscoped_q=streamnative * Didn't work on Java 11 - README.md says that prerequisites Java version is Java8 or *later* - However, since some of using modules are removed from Java JDK 11, they need to be added to dependency - https://www.oracle.com/technetwork/java/javase/11-relnote-issues-5012449.html#JDK-8190378 Regards, Yuri. -- Yuri Mizushima yumiz...@yahoo-corp.jp Yahoo Japan Corporation 2019/11/12 18:40 に、"Guangning E" を書き込みました: Hi everyone, Please review and vote on the release candidate #3 for the version 0.1.0, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) It fixes the following issues: https://github.com/apache/pulsar-manager/milestone/2?closed=1 The complete staging area is available for your review, which includes: * Release notes [1] * The official Apache source and binary distributions to be deployed to https://dist.apache.org/repos/dist/dev/pulsar/pulsar-manager/pulsar-manager-0.1.0-candidate-3/ [2] * Source code tag "v0.1.0-candidate-3" [4] with git sha512 0dfbfc6e57b7d1b6d32865dd50e16d24e8201945b01575a3fa38093053fee955e8e6ddcfe6a94c290b7a14276c2724079af5b154b7e086a75da8bbb95eff7134 pulsar-manager's KEYS file contains PGP keys we used to sign this release: https://dist.apache.org/repos/dist/dev/pulsar/KEYS Please download these packages and review this release candidate: - Review release notes - Download the source package (verify shasum, and asc) and follow the instructions to build and run the pulsar-manager frontend and backend service. - Download the binary package (verify shasum, and asc) and follow the instructions to run the pulsar-manager frontend and backend service. https://github.com/apache/pulsar-manager/wiki/Release-Candidate-Validation The vote will be open for at least 72 hours. It is adopted by majority approval, with at least 3 PMC affirmative votes. Source and binary files: https://dist.apache.org/repos/dist/dev/pulsar/pulsar-manager/pulsar-manager-0.1.0-candidate-3/apache-pulsar-manager-0.1.0-candidate-3-bin.tar.gz https://dist.apache.org/repos/dist/dev/pulsar/pulsar-manager/pulsar-manager-0.1.0-candidate-3/apache-pulsar-manager-0.1.0-candidate-3-src.tar.gz SHA-512 checksums: 0dfbfc6e57b7d1b6d32865dd50e16d24e8201945b01575a3fa38093053fee955e8e6ddcfe6a94c290b7a14276c2724079af5b154b7e086a75da8bbb95eff7134 apache-pulsar-manager-0.1.0-candidate-3-bin.tar.gz 5bcc64a23cf28cfc2c6bee653066c7adfedb01d0703de78aad558e6829af79906b77d496cd61ba73f0e35df5813d48114b9098f8eb070e7e0ac4607858e1a615 apache-pulsar-manager-0.1.0-candidate-3-src.tar.gz Thanks, Guangning
