Re: [I] [Bug] ASF Privacy Policy Changed and the Website must be fixed [pulsar]

[Dave Fisher]

Hi -

I hope that someone will pick up this issue.

Best,
Dave

> On Feb 18, 2024, at 2:29 PM, dave2wave (via GitHub)  wrote:
> 
> 
> dave2wave opened a new issue, #22068:
> URL: https://github.com/apache/pulsar/issues/22068
> 
>   ### Search before asking
> 
>   - [X] I searched in the [issues](https://github.com/apache/pulsar/issues) 
> and found nothing similar.
> 
> 
>   ### Version
> 
>   Current pulsar website
> 
>   ### Minimal reproduce step
> 
>   See the ASF's internal report on the compliance of pulsar.apache.org with 
> policy: https://whimsy.apache.org/site/project/pulsar
> 
>   ### What did you expect to see?
> 
>   All boxes should be green.
> 
>   ### What did you see instead?
> 
>   The resources box is red. The policy is to avoid use of domains which may 
> track users. This is for GPDR compliance.
> 
>   There are two different external resource domains being used in three 
> source files:
> 
>   ```
>   pulsar-site % grep -ni cdn.jsdelivr.net ./static/reference/index.html
>   17:  
> href="https://cdn.jsdelivr.net/npm/docsify-themeable@0/dist/css/theme-simple-dark.css;
>   22:  
> href="https://cdn.jsdelivr.net/npm/docsify-themeable@0/dist/css/theme-simple.css;
>   231: src="https://cdn.jsdelivr.net/npm/docsify@4/lib/docsify.min.js";>
>   232: src="https://cdn.jsdelivr.net/npm/docsify-themeable@0/dist/js/docsify-themeable.min.js";>
>   235: src="https://cdn.jsdelivr.net/npm/docsify@4/lib/plugins/search.js";>
>   236: src="https://cdn.jsdelivr.net/npm/docsify@4/lib/plugins/zoom-image.min.js";>
>   241: src="https://cdn.jsdelivr.net/npm/docsify-sidebar-collapse/dist/docsify-sidebar-collapse.min.js";>
>   ```
> 
>   ```
>   pulsar-site % grep -ni cdn.jsdelivr.net ./docusaurus.config.js   
>   481:  href: 
> "https://cdn.jsdelivr.net/npm/katex@0.13.24/dist/katex.min.css;,
>   ```
> 
>   ```
>   pulsar-site % grep -ni fonts.google ./src/css/typography.css 
>   1:@import 
> url("https://fonts.googleapis.com/css2?family=Work+Sans:ital,wght@0,400;0,500;0,600;1,400;1,500;1,600=swap;);
>   ```
> 
>   ### Anything else?
> 
>   The boxes will be green once these resources are moved to static resources 
> hosted on the `pulsar.apache.org` website
> 
>   ### Are you willing to submit a PR?
> 
>   - [ ] I'm willing to submit a PR!
> 
> 
> -- 
> This is an automated message from the Apache Git Service.
> To respond to the message, please log on to GitHub and use the
> URL above to go to the specific comment.
> 
> To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org.apache.org
> 
> For queries about this service, please contact Infrastructure at:
> us...@infra.apache.org
> 

PIP-264: Implementation status update

[Asaf Mesika]

Hi,

PIP-264 (approved Sep 2023) fixes many pitfalls of the current metric
system inside Pulsar. We partly do so by adding another option of adding
metrics: OpenTelemetry Java SDK, and then redefining the existing metrics
using OTel.

The first part was adding OpenTelemetry Java SDK into Pulsar in this PR
. This was an
implementation of (sub) PIP-320 and was contributed by Dragos Misca.

The first metrics to be migrated are the lookup metrics, as contributed by this
PR , contributed by Dragos
Misca.
Take a look - feedback is appreciated.

Thanks,

Asaf

Re: [DISCUSS] PIP-331: WASM Function API

[Asaf Mesika]

Hi ZiCheng,

Brilliant suggestion!

I replied in the PR section, which I couldn't understand.


On Tue, Jan 30, 2024 at 1:18 PM dragon-zhang 
wrote:

> Hi Pulsar Community,
>
> I want to add a new feature that supports run WASM bytecode to the
> pulsar-functions module.
>
> Please see the PIP: https://github.com/apache/pulsar/pull/21992
>
> Thanks,
> ZiCheng Zhang

Re: WIKIPEDIA PAGE

[Kiryl Valkovich]

Hi Brolly,

The last time you and I discussed it in December, you suggested submitting it 
into the Simple English section and wanted to charge for that. In my personal 
opinion, Pulsar deserves an article on a regular English Wikipedia, not in the 
Simple English section. Many smaller ASF projects have such articles.

I still hope that someone from the community will take the initiative and write 
such an article that will pass all the Wiki's requirements like lack of 
personal interest and sufficient amount of references to reliable independent 
resources.


Best,
Kiryl

From: Brolly 
Sent: Saturday, February 17, 2024 11:58 AM
To: Dev 
Subject: WIKIPEDIA PAGE

Hi,

I hope you are well. This is Brolly, I am a Wikipedia moderator.

I just saw your Wikipedia page on the Wikipedia page rejection list.

I would like to offer my assistance in helping you successfully publish
your Wikipedia page.

If you are interested in getting your Wikipedia page done please let me
know.

I look forward to hearing from you.

Thanks

Brolly
[image: beacon]

Re: Ability to decrease partition count in pulsar

[Asaf Mesika]

Hey Girish,

First, I say that I *love* this proposal and, in general, those types of
proposals.
This is what strides Pulsar towards being an even more next-generation
messaging system.

I read and have a few questions and brainstorming ideas popping into my
mind:

1. The current design basically says: Let’s have a read-only toggle (flag)
for each partition. When I decrease the partitions from, say, 2 to 1, then
if the partitions were “billing-0” and “billing-1”, now “billing-1” will be
marked read-only, and eventually, the client will only produce messages to
“billing-0”. After 1 hour, I can scale it back to 2 partitions, and now the
“billing-“1 will be toggled back to read-only=false.

* I know you stated that ordered consumption is out of scope. The thing I
fear here is that even for shared subscriptions, in which order doesn’t
matter, it still feels a bit weird that when you consume from the
beginning, you can suddenly consume messages that are 1 hour apart from
each other, one after another. Something like:

P0  | t1 | t3 | t7 | t10| t11| t13| t17|
++++++++
P1  | t2 | t4 | t6 | t9 | t12| t14| t16|
++++++++
P2  ||| t5 | t8 ||| t15|
||||||||
++++++++
^  ^
RO URO


t5 - you scaled to 3 partitions.
“R0” is when you change from 3 partitions to 2
“URO” is when you change back to 3 partitions.

When you consume this partitioned topic from the beginning, you will
consume t15 mixed with t6 and t7, which can be hours apart.

I understand this can happen today if you only add a partition and read
from the beginning.

2. If we keep ordered consumption out of scope, how do we keep the users
from doing “wrong” things, like using failover type subscriptions on
partitioned topics that have decreased their partitions? Topic and its
partition count is a detached “entity” from its consumption type.


I’m curious if you thought of implementing it following the pattern we have
today for BK. When an ensemble changes, it simply adds the new ensemble to
a list of ensembles, so you follow a chain of servers when you read from a
ledger. You read from (b1,b2,b3) and then switch to (b1, b3, b5).

What if a partitioned topic is exactly that? It is a chain of lists. Each
list contains the topics (partitions).
Something like:
(billing-0-100, billing-1-101), (billing-0-102, billing-1-103,
billing-2-104), (billing-0-105, billing-1-106)

It’s only a direction - just wondering if something like that has been
considered.


On Fri, Jan 19, 2024 at 8:28 AM Girish Sharma 
wrote:

> Hello everyone,
>
> A a true cloud native platform, which supports scale up and scale down, I
> feel like there is a need to be able to reduce partition count in pulsar to
> truly achieve a scale down after events like sales (akin to black friday,
> etc) or huge temporary publish burst due to backfill.
>
> I looked through the archives (upto 2021) and did not find any prior
> discussion on the same topic.
>
> I have given this an initial thought to figure out what would it need to
> support such a feature in the lowest footprint possible. I am attaching the
> document explaining the need, requirements and initial high level details
> [0]. What I would like is to understand if the community also finds this
> feature helpful and does the approach described in the document have some
> fatal flaw? Summarizing the approach here as well:
>
>- Introduce an ability to convert a normal topic object into a read-only
>topic via admin api and an additional partitioned-topic metadata
> property
>(just like shadow source, etc)
>- Add logic to block produce but allow new consumers and dispatch call
>based on this flag
>- Add logic in GC to clean out read only topics when all of their
>ledgers expire (TTL/retention)
>
> Goal is that there is no data movement involved and no impact on existing
> partitions during this scale down.
>
> Looking forward to the discussion.
>
> [0]
>
> https://docs.google.com/document/d/1sbGQSwDihQftIRsxAXg5Zm4uxKQ0kRk9HadKYRFTswI/edit?usp=sharing
>
> Regards
> --
> Girish Sharma
>