Pulsar Summit San Francisco 2022 CFP is open!

Hi, all the Pulsar community members, Pulsar Summit San Francisco 2022 CFP is open now! It will be our *first-ever in-person Pulsar Summit *in San Francisco! We're glad to invite you to share your experience and add up to the wonderfulness. As a speaker, you will receive the chance to

Re: [ANNOUNCE] Apache Pulsar Go Client 0.8.1 released

Congrats! Rui - Thank you for driving this important fix for the license issue! - Sijie On Tue, Mar 15, 2022 at 2:10 AM Rui Fu wrote: > The Apache Pulsar team is proud to announce Apache Pulsar Go Client > version 0.8.1. > > Pulsar is a highly scalable, low latency messaging platform running

Re: Happy International Women’s Day!

Best wishes to all the female contributors! - Sijie On Tue, Mar 8, 2022 at 9:00 AM Yu wrote: > Dear Pulsar female contributors, > > Success doesn’t come easy and women never give up. Thanks for bringing so > much love and support to the Pulsar community. And every man needs a > woman when his

Re: [Discuss] Create Pulsar client release notes

Yu, thank you for driving this effort! Great to see a proposal focusing on this! +1 On Fri, Mar 4, 2022 at 2:59 AM Yu wrote: > Hi Pulsarers, > > For Pulsar release notes [1], we had issues below for a long time. > > - All contents are on a single Pulsar release note page. No navigations. > It's

Re: New Pulsar website goes live!

Hi Yu, Thank you and all the other contributors for all the great work! Well done. - Sijie On Wed, Feb 16, 2022 at 8:04 PM Yu wrote: > Thanks for your quick feedback, Penghui and Dianjin. > > As mentioned in the last email, "*...**there are some known issues (eg. > localized pages and some

Re: Welcome Dave Duggins to Pulsar community!

Welcome Dave! - Sijie On Sun, Dec 5, 2021 at 4:49 PM Huanli Meng wrote: > Welcome Dave. > > BR//Huanli > > On Dec 6, 2021, at 8:46 AM, PengHui Li wrote: > > Welcome, Dave. > > Regards, > Penghui > > On Sat, Dec 4, 2021 at 8:36 PM Dianjin Wang < > djw...@streamnative.io.invalid> wrote: > >>

Re: [ANNOUNCE] Apache Pulsar Node.js client 1.3.2 and 1.4.1 released

Thanks Masahiro! - Sijie On Tue, Oct 19, 2021 at 4:34 AM Masahiro Sakamoto wrote: > The Apache Pulsar team is proud to announce Apache Pulsar Node.js client > version 1.3.2 and 1.4.1. > > Pulsar is a highly scalable, low latency messaging platform running on > commodity hardware. It provides

Re: [ANNOUNCE] New PMC member: Lin Lin (@315157973)

Congrats Lin Lin! Well deserved! - Sijie On Mon, Aug 2, 2021 at 8:51 AM PengHui Li wrote: > > Hi everyone, > > The Apache Pulsar Project Management Committee (PMC) has invited Lin Lin > (https://github.com/315157973) as a member of the PMC and we are > pleased to announce that he has accepted.

Re: [ANNOUNCE] Apache Pulsar Go Client 0.6.0 released

Xiaolong - Congratulations! Thanks to every contributor for making this happen! - Sijie On Sun, Jul 25, 2021 at 10:56 PM Dianjin Wang wrote: > > Congratulations! > > Best, > Dianjin Wang > > > On Mon, Jul 26, 2021 at 11:02 AM r...@apache.org > wrote: >> >> The Apache Pulsar team is proud to

Re: Is it possible to change the SendTimeout parameter value for the Pulsar C++ producer at runtime?

It is not possible for now. Feel free to create a Github issue for this enhancement. - Sijie On Fri, Jul 23, 2021 at 8:03 AM Marco Guest wrote: > > Hi, > for networking reasons, I would like to use a different value of parameter > configured with pulsar::ProducerConfiguration::setSendTimeot()

Re: How to choose the appropriate value for Producer's send timeout value?

Pulsar retries on failures. So if you want Pulsar to do the retries, you can send the timeout to be a higher value. If you want to be able to control the retry behavior, you can low the timeout value. The timeout value should be larger than your normal publish time latency. The publish time

Re: [Doc] Upgrade Docusuraus

have > done for the last 10 years of my career. And I look forward to working with > all of you and helping to grow this community. > > > > On Tue, Jul 13, 2021 at 6:52 AM Guangning E wrote: >> >> +1 >> >> Enrico Olivelli 于2021年7月13日周二 下午4:14写道： >>> &

Re: [Doc] Upgrade Docusuraus

+1 On Mon, Jul 12, 2021 at 8:25 PM Anonymitaet _ wrote: > > Hi Pulsar enthusiasts, > > > > Nowadays, Pulsar grows rapidly, docs are in increasing demands. While some > users have a hard time finding what they need or encountering other issues. > > > > As a sustainable community, we always give

Re: [ANNOUNCE] Apache Pulsar 2.8.0 released

Congrats! This is a great milestone! Thanks to everyone participating in this release! - Sijie On Tue, Jun 15, 2021 at 7:04 PM PengHui Li wrote: > The Apache Pulsar team is proud to announce Apache Pulsar version 2.8.0. > > Pulsar is a highly scalable, low latency messaging platform running

Re: [SECURITY] [CVE-2021-22160] Authentication with JWT allows use of “none”-algorithm

For people who are following this thread, I want to make a clarification about this issue (and apologized for not making it clear at the beginning) This issue will ONLY happen to users who are using the JWT authentication provider. If you are using other authentication providers, you are NOT

Re: [SECURITY] [CVE-2021-22160] Authentication with JWT allows use of “none”-algorithm

ovide Apache Pulsar users with a 2.6 build as quickly as possible, in > parallel with helping out on an official 2.6.4 release. > > On Thu, May 27, 2021 at 2:24 PM Sijie Guo wrote: > >> Chris - I don't think it is appropriate to promote a vendor image here >> from >

Re: [SECURITY] [CVE-2021-22160] Authentication with JWT allows use of “none”-algorithm

Chris - I don't think it is appropriate to promote a vendor image here from a vendor perspective. A better approach is to point out the change has been cherry-picked to branch-2.6 and an ongoing discussion for getting a new bugfix release for branch 2.6. is out. - Sijie On Thu, May 27, 2021 at

[ANNOUNCE] New Apache Pulsar Committer - Addison Higham

Hi everyone, On behalf of the PMC, I'm very happy to announce Addison Higham as a new Pulsar committer. Addison Higham is a very active community member for about 3 years. He is an active Pulsar contributor contributing features and bug fixes across core Pulsar, Kinesis connector, proxy, and

Re: [ANNOUNCE] New committer: Ran Gao

Congrats Ran! Nice work on contributing the Pulsar transaction feature! - Sijie On Thu, Mar 25, 2021 at 11:09 AM Cem Aladogan wrote: > Well done Ran! > > > On 25 Mar 2021, at 09:52, Dianjin Wang wrote: > > > > Congratulations! > > > > Best, > > Dianjin Wang > > > > On Thu, Mar 25, 2021 at

Re: Pulsar Feature Matrix sheet - Client - Functions

Jennifer - Great work! Thank you for driving that! - Sijie On Tue, Mar 23, 2021 at 9:16 AM Yuva raj wrote: > This is a Great Initiative! Thanks a lot Jinfeng. > > On Tue, 23 Mar 2021 at 21:36, Jinfeng Huang wrote: > >> Dear Pulsar community members, >> >> The Pulsar Client and Functions

Re: Where does pulsar functions jar is copied over on the broker

t a new image based on pulsar image and include the functions there; or b) mount an external persistent volume and put your jar in the external persistent volume. - Sijie > > Thanks > Sachin > > > On Tue, Mar 23, 2021 at 12:11 PM Sijie Guo wrote: > >> The jar file will be

Re: [ANNOUNCE] Apache Pulsar 2.7.1 released

Well done, Yong! Thanks to all the committers and contributors that make this happen! Thanks, Sijie On Mon, Mar 22, 2021 at 1:26 AM Yong Zhang wrote: > The Apache Pulsar team is proud to announce Apache Pulsar version 2.7.1. > > Pulsar is a highly scalable, low latency messaging platform

Re: Where does pulsar functions jar is copied over on the broker

The jar file will be stored in bookkeeper. When functions are executed, the jar will be downloaded by each function worker (broker) in a download directory that is configured at https://github.com/apache/pulsar/blob/master/conf/functions_worker.yml#L41. If you want to avoid uploading and

Re: First Flink x Pulsar Bay Area Virtual Meetup - All things about Streaming SQL

If you missed the Flink x Pulsar meetup last week, you can watch the meetup recording on Youtube: https://www.youtube.com/watch?v=9ojajM7Zt0M=PLqRma1oIkcWgVbfmE0GDUkThnv_FNui7d - Sijie On Thu, Feb 25, 2021 at 12:23 AM Sijie Guo wrote: > Hi all, > > We are thrilled to announce the f

[ANNOUNCE] Introducing Apache Pulsar Hackathon 2021!!!

Hi, Pulsar and BookKeeper communities, The adoption of Apache Pulsar and BookKeeper is accelerating as organizations around the world pursue cloud-native technologies. To build on this momentum, we are excited to announce the first-ever Apache Pulsar Hackathon 2021, *taking place on May 6-7th*.

Re: [ANNOUNCE] Apache Pulsar Go Client 0.4.0 released

Great job, Xiaolong! - Sijie On Sun, Mar 7, 2021 at 6:44 PM Anonymitaet _ wrote: > Great work! Thanks Xiaolong. > > > > *From: *xiaolong ran > *Reply-To: *"users@pulsar.apache.org" > *Date: *Friday, March 5, 2021 at 12:13 > *To: *Dev , "users@pulsar.apache.org" < > users@pulsar.apache.org>,

Re: Join #contributors channel on Pulsar Slack

contributors and #dev? > > On Mon, Mar 1, 2021 at 10:00 AM Sijie Guo wrote: > >> Hello everyone! >> >> Thanks to every contributor’s contribution to Pulsar! The Pulsar community >> has been growing very fast in the past 2 years. Currently, there are some &

Join #contributors channel on Pulsar Slack

Hello everyone! Thanks to every contributor’s contribution to Pulsar! The Pulsar community has been growing very fast in the past 2 years. Currently, there are some platforms for us to communicate with others, such as GitHub, Pulsar Community Meetings, and Slack dev/general channels, but there’s

First Flink x Pulsar Bay Area Virtual Meetup - All things about Streaming SQL

Hi all, We are thrilled to announce the first-ever Flink x Pulsar meetup on March 16th & 17th! This meetup is all things about streaming SQL! It is a two-day virtual event co-hosted by Uber, Ververica, and StreamNative! You will learn about how Uber and Facebook build and optimize their

TGI-Pulsar is back!

Hi all, TGI-Pulsar is back! In the past 2020, we have successfully run 18 episodes and received a lot of useful feedback. Thanks to everyone who has participated there! We are returning this year with a new format to

Re: CPU consuming question

nate/stats?getPreciseBacklog=false > HTTP/1.1" 200 1401 "-" "Pulsar-Java-v2.7.0" 10 > > 14:47:03.431 [pulsar-web-68-8] INFO org.eclipse.jetty.server.RequestLog > - 127.0.0.1 - - [01/Feb/2021:14:47:03 +0800] "GET > /admin/v2/persistent/public/fun

Re: CPU consuming question

Hi Wesley, The CPU usage might be coming from some background tasks. Do you happen to have the log of the standalone server? This might give some insights to the problem. - Sijie On Sun, Jan 31, 2021 at 9:58 PM Wesley Peng wrote: > I deployed a standalone pulsar server on a two cores VPS,

Re: [ANNOUNCE] New Pulsar Committer: LinLin

Congrats Lin! Looking forward to more contributions from him! - Sijie On Tue, Jan 5, 2021 at 11:05 PM PengHui Li wrote: > The Apache Pulsar PMC recently extended committer karma to LinLin and he > has accepted. > LinLin has done a lot of contributions to Pulsar core features and bug > fixes.

Re: pulsar function changes the schema of output topic to JSON from AVRO

Sachin - You can specify the schema of the output topic when you submit a function. Thanks, Sijie On Thu, Dec 3, 2020 at 8:33 AM Sachin Mittal wrote: > So, > I have created a simple function which copies the message from input topic > to output topic. > > public class MirrorFunction implements

Re: NULL schema

Hi Rob, Have you tried to use `pulsar-admin schemas` to retrieve the schema info? If so, can you provide the output? Thanks, Sijie On Tue, Oct 27, 2020 at 3:59 PM Rob Shepherd wrote: > Hi, > > I'm wondering what the issue or circumstance is to have a pulsar consumer > failing with a null

Re: Application Environments

If " given that some apps only support some environments and only a few support all, we need to route between them." is the case you are facing, I'd recommend using option 4. You can write a function to route messages to the different application environment-specific topics. - Sijie On Sun, Nov

[ANNOUNCE] New Pulsar Committer: Huanli Meng & Hang Chen

The Apache Pulsar PMC recently extended committer karma to Huanli Meng and Hang Chen and both of them have accepted. Huanli Meng has done a lot of contributions in technical writing and documentation development for Pulsar, and continues to help promote Pulsar in different channels via tech

Re: [Announce] Pulsar Helm Chart 2.6.0 is published to https://pulsar.apache.org/charts

The helm chart is now officially available in Helm hub: https://hub.helm.sh/charts/pulsar/pulsar - Sijie On Thu, Jul 2, 2020 at 2:38 AM Sijie Guo wrote: > Hi Pedro, > > Yes. I have submitted a pull request to add it to Helm Hub. Once it is > merged, I will update this thread.

Re: [Announce] Pulsar Helm Chart 2.6.0 is published to https://pulsar.apache.org/charts

> > Em qui., 2 de jul. de 2020 às 10:13, Sijie Guo > escreveu: > >> Hi all, >> >> We have published the released Pulsar helm chart to >> https://pulsar.apache.org/charts. You can use the following commands to >> add the released helm chart to your local h

[Announce] Pulsar Helm Chart 2.6.0 is published to https://pulsar.apache.org/charts

Hi all, We have published the released Pulsar helm chart to https://pulsar.apache.org/charts. You can use the following commands to add the released helm chart to your local helm repository and start using it. helm repo add apache https://pulsar.apache.org/charts helm repo update helm upgrade

Re: [ANNOUNCE] Apache Pulsar 2.6.0 released

Congratulations! Thanks everyone in the community for making this great release happen! - Sijie On Thu, Jun 18, 2020 at 4:48 AM PengHui Li wrote: > The Apache Pulsar team is proud to announce Apache Pulsar version 2.6.0. > > Pulsar is a highly scalable, low latency messaging platform running

Re: Clarification on End-to-End Encryption

Subash, Comments inline. On Sat, Apr 25, 2020 at 3:55 AM Subash K wrote: > Hi, > > > > I was looking through the documents of End-to-End Encryption > and I’m having > following queries as I haven’t completely understood how this works: > >

Re: JSON Web Tokens can't generate key pairs

ivate.key > > but still got the same error. I'm trying to follow the page here: > https://pulsar.apache.org/docs/en/2.5.2/security-token-admin/#secret-vs-publicprivate-keys > using the grant-permissions from command line. > > > On Fri, Jun 5, 2020 at 8:12 PM Sijie Guo wrote: &g

Re: JWT Token Permission

Did you enable TLS encryption? The error seems to indicate that you are using TLS but the TLS certs is missing. - Sijie On Mon, Jun 8, 2020 at 1:38 PM Jon Arce wrote: > If I use the command like: > /opt/pulsar/bin/pulsar-admin namespaces grant-permission demoCustomer/test > --role worker

Re: Message Encryption in Kafka Adaptor

Hi Shekhar, It doesn't support that yet. Can you create an issue for that? - Sijie On Mon, Jun 8, 2020 at 12:36 PM BANSAL Shekhar < shekhar.ban...@thalesgroup.com> wrote: > Hi > Does kafka-adaptor >

Re: JSON Web Tokens can't generate key pairs

You need to use a file path for specifying key files. E.g. file://path/to/your/key/file. - Sijie On Fri, Jun 5, 2020 at 11:38 AM Jon Arce wrote: > Client authentication using tokens based on JSON Web Tokens > I am following the JWT document on link: >

Announcing Pulsar Summit Virtual Conference 2020 | June 17-18

Hi all, Over the past few weeks, after having to reschedule Pulsar Summit San Francisco 2020, we have been working on delivering the event in an online format instead. Today, we are excited to announce the Pulsar Summit Virtual Conference, happening on June 17-18! Join the speakers from Splunk,

#TGIPulsar Episode 009 - Introduction of Apache Pulsar

Hi all, One of the commonly asked questions that we have answered for new Pulsar users is - "why the bookies' disk space is not released when storage size is already 0". In order to help Pulsar users understand this better, we are going to use the next couple of episodes talking about BookKeeper

Re: Multiple compactions?

compacted' flag. Though a manual retrieval of the specific > compaction shouldn't be too difficult. > > -Patrick > > > On Tue, May 5, 2020, at 14:01, Sijie Guo wrote: > > Hi Patrick, > > Currently Pulsar only supports one compaction state per topic at this > mom

Re: Multiple compactions?

Hi Patrick, Currently Pulsar only supports one compaction state per topic at this moment. However the underneath implementation doesn't prevent supporting multiple concurrent compactions as we just need to track the complications in different cursors. As I understand here, you want the ability to

#TGIPulsar Episode 008 - What's New in 2.5.1?

Hi all, Our weekly live stream "TGI Pulsar" ( https://www.youtube.com/channel/UCywxUI5HlIyc0VEKYR4X9Pg/live) continues this week. In this week's episode, we are going to talk about what's New in 2.5.1 and highlight some of the features and changes. If you are interested in this topic, join us at

Re: Data Encryption

:* Monday, April 27, 2020 12:39 PM > *To:* users@pulsar.apache.org > *Subject:* RE: Data Encryption > > > > Hi Sijie, > > > > Sure, I’ll do it. Thanks! > > > > Regards, > > Subash Kunjupillai > > > > *From:* Sijie Guo > *Sent:* Monday, April 27,

Re: Apache Pulsar - Performance on Tiered storage

The performance typically varies between different tiered storages. And tiered storage is only used in scanning the historic data. So the dominated factor is the sequential scan throughput. Currently, I don't think there are any public articles about performance. Usually, we would recommend people

Re: Data Encryption

> > Yes, this approach looks promising to me as of now. > > > > Regards, > > Subash Kunjupillai > > > > *From:* Sijie Guo > *Sent:* Thursday, April 23, 2020 10:21 PM > *To:* users@pulsar.apache.org > *Subject:* Re: Data Encryption > > > > I see.

Re: Data Encryption

Subash, I think End-to-End Encryption is the only feasible solution for your requirement at this moment. Out of curiosity, if you are using Kafka Adaptor, you anyway need to re-compile your consumer and producer with the Kafka adaptor. Are you able to specify additional settings in the

#TGIPulsar Episode 007 - Cluster & Geo replication

Hi all, One of the attractive features in Pulsar is geo-replication. Pulsar is very flexible at configuring geo-replication between clusters. It supports active-active, active-standby, aggregate and full mesh. To better understand how to configure different patterns of geo-replication, you have

Re: Rollback of Apache Bookeeper

e configured disk format version. > > It is always able to read the old format, and it will write during regular > writes and during compaction only the format configured in bk_server.conf > > > > Enrico > > > > > > Regards, > > Subash Kunjupillai > >

Re: Rollback of Apache Bookeeper

+ users@ back to the discussion. Sorry for being late to the discussion here. When talking about "rollback" behavior, it usually involves two questions: 1) does wire protocol support rollback? 2) does data(metadata) format support rollback? For the first question, the bookkeeper provides very

Re: Understanding Pulsar - Bookkeeper compatibility

On Fri, Apr 17, 2020 at 4:57 AM Subash K wrote: > Hi, > > > > We are evaluating Apache Pulsar to be integrated with our product and > currently we are trying to understand the Life Cycle Management of the > Pulsar components. So I would like to get the following points clarified: > > > >- We

[Webinar] Common architectural patterns on Apache Pulsar

Hi Pulsarers, We have a new Pulsar webinar upcoming at 10 AM Pacific Time on 04/28. Devin Bost from Overstock will share how Overstock has leveraged Pulsar in production. This webinar will help engineers and architects better understand common real-world use cases for Pulsar and learn about

#TGIPulsar Episode 006 - Lifecycle of a Pulsar Message

Hi all, A lot of new Pulsar users might be confused about backlog vs storage size, TTL vs retention, why the bookies' disk is still filled up even the topic is already deleted, and etc. In this week's live stream #TGIPulsar, we will talk about the lifecycle of a Pulsar Message and help you

Re: Clarity on Pulsar Consumer API receive

Raman Gupta answered your question in Slack. I copied and pasted his answer here. Firstly, that Kafka call doesn't get *all* unread messages, it gets an undefined amount of messages which might be all or might be less (there are a bunch of consumer properties that control this like

Re: Please Help

Hi Tolulope, We just updated the helm chart yesterday. The current helm chart is tested in CI with different configurations. I will suggest you pulling the latest master. https://github.com/apache/pulsar/tree/master/deployment/kubernetes/helm

#TGIPulsar Episode 004 - Deep dive into authentication and authorization

Hi all, Our weekly live stream "TGI Pulsar" ( https://www.youtube.com/channel/UCywxUI5HlIyc0VEKYR4X9Pg/live) continues this week. In this week's episode, we are going to take a deep dive into authentication and authorization in Pulsar, how does it work and how to configure and enable it. If you

#TGIPulsar Episode 003 - Secure a Pulsar cluster with TLS

Hi all, Our weekly live stream "TGI Pulsar" ( https://www.youtube.com/channel/UCywxUI5HlIyc0VEKYR4X9Pg/live) continues this week. In this week's episode, we are going to talk about how to secure a Pulsar cluster with TLS. It will cover not only brokers and proxies but also bookies as well. If

Re: Unable to consume messages, need to re-subscribe or restart

Hi Gaurav, There are a couple of things to check. 1) Which version of Pulsar are you using? 2) Which language of Pulsar client are you using? 3) Did your consumers acknowledge the messages after processing them? 4) When the issue happens, did you happen to capture the stats of the topic? You can

Re: Unexpected behavior with Key_Shared (Multiple messages with the same key go to different consumers)

Hi Hiroyuki, > First, multiple messages with the same key go to the same consumer thread as described below. If there is a new consumer joining a subscription, the key distribution will be split. So you are expected to see the messages of a key is dispatched from one consumer to another

Kafka-on-Pulsar: bring the native Kafka protocol support to Apache Pulsar

Hi all, We are super excited to share that StreamNative and OVHcloud have just open-sourced the Kafka-on-Pulsar (a.k.a KoP) protocol handler today. KoP is a protocol handler that handles the Kafka 2.0+ protocol. It can be installed to your Pulsar brokers (2.5.0 or newer version) to enable native

#TGIPulsar Episode 002 - Kubernetes Deployment

Hi all, Our weekly live stream "TGI Pulsar" ( https://www.youtube.com/channel/UCywxUI5HlIyc0VEKYR4X9Pg/live) continues this week. Pulsar was architected to run easily on cloud-native environments. It is very easy to spin up a cluster on Kubernetes. But there are also some considerations and best

Re: TGI Pulsar - weekly live stream about Pulsar and its ecosystem

Yes. Those live streams will be recorded and published on Youtube. - Sijie On Tue, Mar 10, 2020 at 2:44 PM Eugen Dueck wrote: > Hi Sijie > > Will the live streams be recorded for those you cannot watch them live? > > Eugen > > ---------- > *差出人:* Sij

TGI Pulsar - weekly live stream about Pulsar and its ecosystem

Hi all, We are starting a weekly live stream - "TGI Pulsar" on Youtube this week. https://www.youtube.com/channel/UCywxUI5HlIyc0VEKYR4X9Pg/live It will happen every Friday at 1 pm PST. Each week we will invite one Pulsar committer or end-user to the live stream to chat about a topic of Apache

Reschedule Pulsar Summit SF to August/September

Dear all, In light of growing concern and the further spread of the COVID-19 (Corona) virus and after close consultation with event stakeholders and the organizing parties, we have decided to reschedule the Pulsar Summit San Francisco *from April to August/September*. It was decided that with due

[ANNOUNCE] Pulsar Summit San Francisco 2020 Call for Presentation extended!

can find more detailed track descriptions and the form to submit a proposal on the Pulsar Summit website at --> https://pulsar-summit.org/call-for-presentations/ --> https://sessionize.com/pulsar-summit-san-francisco-2020/ As usual, accepted speakers get a free conference pass. Best regards,

[ANNOUNCE] Pulsar Summit San Francisco 2020 Call-for-presentations and Pre-registration are now open

Dear Apache Pulsar community, We are excited to announce that the first-ever Pulsar Summit is planned for April 2020 in San Francisco. Call-for-presentations and pre-registration are now open for the conference! Pulsar Summit is an annual conference dedicated to Apache Pulsar community, aims at

Re: [ANNOUNCE] Apache Pulsar 2.4.2 released

Brian, That was done intentionally to get around the website indexing issue. It takes some time for Algolia indexing Pulsar documentation website. During Algolia is indexing 2.4.2 release, the search functionality of 2.4.2 release will be broken because the index for 2.4.2 is not ready. If we

Re: [ANNOUNCE] Apache Pulsar Manager 0.1.0 released

Thank you Guangning and the community for making this happen! - Sijie On Fri, Nov 29, 2019 at 5:14 AM Guangning E wrote: > The Apache Pulsar team is proud to announce Apache Pulsar Manager version > 0.1.0. > > Pulsar is a highly scalable, low latency messaging platform running on > commodity

Pulsar Summit is coming 2020

Hi Pulsar community, Some folks have asked for a dedicated Pulsar event before. Now it is real! I am happy to share that the community is going to organize two Pulsar events (Pulsar Summit) at 2020, one is at San Francisco around April, the other one is at Beijing around September. Pulsar Summit

Re: Standalone geo replication on local machine using Kubernetes

h were instrumental to my > understanding - thanks for the help! > > Cheers, > > Christoffer > PS Tell me if you want your names removed... > > On 22 Nov 2019, at 17:17, Sijie Guo wrote: > > Hi Christoffer, > > You don't need a global zookeeper to make geo-rep

Re: Standalone geo replication on local machine using Kubernetes

Hi Christoffer, You don't need a global zookeeper to make geo-replication work. Thus, you don't need different cluster names as well since you don't need a global zookeeper. The existence of a global zookeeper is used for simplifying management of replication configuration. All you need to do is

Re: Forcing garbage collection

I don’t think it works in stand-alone mode. We have to make changes in stand-alone to expose a setting to allow configuring the http server. Thanks, Sijie On Fri, Nov 15, 2019 at 7:34 PM Brian Candler wrote: > On 15/11/2019 07:33, Sijie Guo wrote: > > The garbage collection is done l

Re: Forcing garbage collection

On Thu, Nov 14, 2019 at 7:18 PM Brian Candler wrote: > I wrote some gigabytes of data into a topic, and then deleted the > topic. The disk space in the data/ directory has not yet been freed > (specifically, data/standalone/bookkeeper0/current/) > > 1. Is there a way to force a

Re: Maximum function sizes

Hi Brian, Comments inline. On Sun, Nov 10, 2019 at 5:53 PM Brian Candler wrote: > I would like to know, what is the maximum size of: > > (1) a pulsar function? (which in the case of python at least, can be a > zip file) > I don't think there is any *hardcoded* limit for a pulsar function. A

Re: Presto - event time predicate push-down?

Hi Brian, I think it is already implemented. The Pulsar Presto Connector supports predicate pushdown based on publish time. > Or it could include min/max/bloom filter on user data too, like ORC does Alternatively, we can leverage Pulsar's tiered

Re: Pulsar go admin client and pulsarctl

Awesome! Yong can probably create a github issue to add this feature. Thanks, Sijie On Tue, Oct 29, 2019 at 10:56 PM Brian Candler wrote: > On 29/10/2019 12:45, Sijie Guo wrote: > > I think we can improve the CLI by adding an option `-o` or `--output`. > > Let user choose

Re: Pulsar go admin client and pulsarctl

Hi Brian, Thank you for your comments. Here are some replies to your comments. > Comment 1 We were thinking that "subscriptions" should be a first-class citizen in the CLI. so we moved the subscription related commands to a separate command group. We can consider supporting the old style

Re: Message compaction questions

On Mon, Oct 14, 2019 at 5:42 AM Brian Candler wrote: > On 14/10/2019 13:24, xiaolong ran wrote: > > For more details on compact topic, you can refer to: > https://github.com/apache/pulsar/wiki/PIP-14:-Topic-compaction > > There it says: > > "Compaction doesn't directly interact with message

Re: docker deployment on bare metal

On Tue, Oct 15, 2019 at 4:35 AM Takahiro Hozumi wrote: > Hi all, > > I would like to setup ZooKeeper, BookKeeper and Pulsar cluster using > official docker image on bare metal server, not k8s. > I think that on bare metal, to use docker has some operational benefit, > compared to manual tarball

[ANNOUNCE] Guangning E becomes a Pulsar committer

Hi everyone, I'm very happy to announce that Guangning E (@tuteng) accepted the offer of the Pulsar PMC to become a committer of the Pulsar project. Guangning has been an active community member since Pulsar TLP. He has made great contributions to Pulsar project. His contributions are ranged

[Meetup] [HUG] Special Apache Pulsar Meetup chez OVH (Paris 17)

Hi all, There is a Pulsar meetup this Friday (10/11) at Paris organized by OVHCloud. It is a dedicated session for Apache Pulsar from HUG France meetup group. If you are at Paris and interested in Paris, don't miss it. https://www.meetup.com/Hadoop-User-Group-France/events/264920447 Thanks,

Re: Broken links to IO connectors, and other website issues

Brian, Thank you for reporting. I think it is probably related to the recent documentation change in master. We will be looking into it. - Sijie On Wed, Oct 9, 2019 at 11:37 AM Brian Candler wrote: > I'd like to point out an issue with the documentation website. > > At

Re: Message consumed by multiple consumers.

Hi, Why not just multiple subscriptions? Each consumer is one subscription. It ensures that all the consumers consume the same copy of data. Thanks, Sijie On Tue, Sep 10, 2019 at 7:50 AM Jerry Gamble wrote: > Question: We need to be able to consume a message in multiple consumers > until the

Re: [ANNOUNCE] Apache Pulsar 2.4.1 released

Great work! Congrats! - Sijie On Fri, Sep 6, 2019 at 4:34 AM Jia Zhai wrote: > The Apache Pulsar team is proud to announce Apache Pulsar version 2.4.1. > > Pulsar is a highly scalable, low latency messaging platform running on > commodity hardware. It provides simple pub-sub semantics over

Docsearch at pulsar website

Hi all, As people is pinging me about why docsearch is not working at Pulsar website. Here are the explanation and workarounds. Pulsar is using Algolia DocSearch for indexing all the doc pages. The documentation of 2.4.1 release was published to pulsar website yesterday as part of the release

ApacheCon NA

Hi, ApacheCon NA is coming next week. There are about 5 pulsar talks this year. I guess quite a lot of Pulsar folks and users will be attending it. For the people who is attending ApacheCon, maybe we should gather together for a drink :-) That can be fun. Grab me if you are in ApacheCon and

Re: [ANNOUNCE] Apache Pulsar Node.js client 1.0.0 released

Congratulations to Masahiro and Yahoo! Japan for the release! - Sijie On Mon, Sep 2, 2019 at 10:00 PM Masahiro Sakamoto wrote: > The Apache Pulsar team is proud to announce Apache Pulsar Node.js client > version 1.0.0. > > Pulsar is a highly scalable, low latency messaging platform running on

[Meetup] Flink + Pulsar, streaming first, unified data processing

Hi all, I will be talking about Flink + Pulsar for the streaming-first unified data processing at the Flink Seattle meetup next week. If you are in Seattle area and interested in this event, grab me there. https://www.meetup.com/seattle-flink/events/263782233/ Thanks, Sijie

Re: [ANNOUNCE] Apache Pulsar 2.4.0 released

Congrats Pulsar community for the wonderful release! Congrats Penghui for releasing your first apache release! Well done! - Sijie On Fri, Jul 5, 2019 at 2:25 AM PengHui Li wrote: > The Apache Pulsar team is proud to announce Apache Pulsar version 2.4.0. > > Pulsar is a highly scalable, low

Re: Error while evaluating "Schema Evolution"

Hi Prakash, Which version of Pulsar are you using? If you are using 2.3.x or earlier versions, the pulsar client doesn't use the schema version associated with messages as the writer schema. So you will encounter this EOF issue. The schema version is used in latest master, which will be released

Flink Blog Post - When Flink & Pulsar Come Together

FYI I just published a post on Flink blog - https://flink.apache.org/2019/05/03/pulsar-flink.html If you are interested in or already using both Flink and Pulsar, please check it out! Feedbacks are welcome. Thanks, Sijie

Re: [ANNOUNCE] Apache Pulsar 2.3.1 released

Congrats, Matteo! Great job, the whole pulsar community!! Sijie On Mon, Apr 15, 2019 at 11:47 PM Matteo Merli wrote: > The Apache Pulsar team is proud to announce Apache Pulsar version 2.3.1. > > Pulsar is a highly scalable, low latency messaging platform running on > commodity hardware. It

  1   2   >