Re: Dropping Presto SQL in 2.9.0 - status ?
Dear Pulsar community members, PIP-62[1], "PIP 62: Move connectors, adapters and Pulsar Presto to separate repositories" was created in April 2020. The repositories for pulsar-connectors, pulsar-adapters and pulsar-sql were created about a year ago [2]. What is the current roadmap for completing PIP-62 and moving pulsar-connectors and pulsar-sql out of apache/pulsar repository? BR, Lari [1] https://github.com/apache/pulsar/wiki/PIP-62%3A-Move-connectors%2C-adapters-and-Pulsar-Presto-to-separate-repositories [2] https://lists.apache.org/thread.html/r9e6ec742e2896da1f0ce7d4adc7cb84fc6db6dbf797732ccdd50fb86%40%3Cdev.pulsar.apache.org%3E Other email threads: * [Discuss] Don't include presto/trino in the normal Pulsar distribution - https://lists.apache.org/thread/jn96tct54mn0tvdot62vdslrvs38fm6d * Updates on Presto connector for PIP-62 - https://lists.apache.org/thread/f9n6sc2mrboq5sxhjbr7gvdl8vqp9fpk On Tue, Nov 2, 2021 at 3:59 PM Nicolò Boschi wrote: > Resurrecting this thread. > > 2.9 is almost released and it hasn't been merged yet > > Extending the discussion to other connectors, it looks like there has been > no progress on PIP-62. > My concern is that a lot of Pulsar IO connectors dependencies we are > running are obsolete with several security reports > > I see there are interesting comments in the issue ( > https://github.com/apache/pulsar/issues/10219) and Sijie exported the > pulsar-io dir to https://github.com/apache/pulsar-connectors but it's > outdated > > From my point of view, we have to: > - reimport all the connectors source codes with newest ones (including > integration tests) > - add periodic CI jobs for connectors to run against master, 2.9-latest, > 2.8-latest, 2.7-latest to verify breaking changes > - define a release cycle/management for connectors (we should improve the > PIP doc). IMO it's not clear if each connector will have its own release > versions and how we'll handle it (git tags, artifacts deployment..) > - update pulsar release script in order to get the connectors artifacts > (retrieving the .nar or building it from source?) > - update docs > - remove pulsar-io dir from Pulsar repo > > It's the perfect timing to schedule this work for 2.10 > > What is missing? How's the situation? Is there a roadblock I haven't seen? > I think it's better to take another discussion for Presto since it will > come to another end > > > Il giorno sab 14 ago 2021 alle ore 15:21 Enrico Olivelli < > eolive...@gmail.com> ha scritto: > > > Sijie > > > > Il Ven 13 Ago 2021, 22:00 Sijie Guo ha scritto: > > > > > You can follow the progress at > > https://github.com/trinodb/trino/pull/8020. > > > > > > > Thanks for the pointer > > > > > > > > The original code doesn't conform to TrinoDB's standard. Marvin is > > > actively following up on that. > > > > > > Our goal is still to get this completed as part of the 2.9 release. > > > > > > > Wonderful > > > > Thanks > > Enrico > > > > > > > > - Sijie > > > > > > On Fri, Aug 13, 2021 at 2:04 AM Enrico Olivelli > > > wrote: > > > > > > > > Hello, > > > > How is the Presto work going ? > > > > IIRC the plan was to remove it from the Pulsar code base and let it > be > > > > hosted at Trino. > > > > > > > > If this is not going to happen within the 2.9.0 release timeline > > > > (September?) I would prefer to upgrade to "Trino". > > > > Probably we will have a downside problem that recent versions of > > > > Presto/Trino do not work on JDK8 but only on JDK11. > > > > > > > > I believe that in that case we could open a separate thread to say > that > > > > Pulsar SQL in 2.9.0 will work only on JDK11. > > > > In Pulsar 2.8.0 we added official compatibility with JDK11 (and it is > > the > > > > preferred version, as it is the version used in the Docker images), > so > > > > requiring JDK11 for Pulsar SQL 2.9.0 does not sound bad to me. > > > > > > > > My primary concern is that the version of Presto that we are running > is > > > > obsolete and there are several security reports against it or its > third > > > > party dependencies. > > > > > > > > Thoughts ? > > > > > > > > Enrico > > > > > > > > -- > Nicolò Boschi >
[DISCUSS] Pulsar Protocol For Client Timeouts and Creating Producers
Hi All, I noticed that the `ServerCnxTest#testCreateProducerTimeout` test indicates that a producer will send a `CloserProducer` command when producer creation times out for the client. The Java client does not follow this protocol. When the producer creation times out, it just schedules another attempt to create the producer. The C++ client does follow this protocol and is annotated with the following comment: > // Creating the producer has timed out. We need to ensure the broker closes > the producer > // in case it was indeed created, otherwise it might prevent new create > producer operation, > // since we are not closing the connection I don't see anything in our official protocol spec indicating the "right" behavior. Given the test coverage, it seems like the initial design was to expect a `CloseProducer` command. However, I also see that the broker's `ServerCnx` class will reply to a redundant `Producer` command with a `ProducerSuccess` command, as long as the producer is already created. Should I submit a PR to update the Java client to send a `CloseProducer` command when a `Producer` command times out? Thanks, Michael
[GitHub] [pulsar-client-node] nkurihar closed issue #105: Implement `consumer.unsubscribe()`
nkurihar closed issue #105: URL: https://github.com/apache/pulsar-client-node/issues/105 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar-client-node] nkurihar commented on issue #105: Implement `consumer.unsubscribe()`
nkurihar commented on issue #105: URL: https://github.com/apache/pulsar-client-node/issues/105#issuecomment-971172216 Close this issue since unsubscribe method was implemented from v1.3.0. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar-dotpulsar] JarrodJ83 commented on issue #8: Support - Message encryption
JarrodJ83 commented on issue #8: URL: https://github.com/apache/pulsar-dotpulsar/issues/8#issuecomment-971114649 Sorry @dgiannone87 I never got around to this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [DISCUSS] Add remove-clusters command for namespace in pulsar-admin
+1 (non-binding) On Mon, Nov 15, 2021 at 5:50 PM PengHui Li wrote: > +1, > > Penghui > On Nov 16, 2021, 9:27 AM +0800, Ruguo Yu , wrote: > > Hi Community, > > > > The tool ` pulsar-admin` supports `set-clusters` and `get-clusters` > command so that we can `set` / `get` replication clusters for a namespace. > But it lacks corresponding `remove-clusters` command to restore to the > unset state, I think it is necessary to add this command to ensure the > closed-loop operation of the replication cluster. > > > > > > > > I have created a issue[1] which contains possible implementation details > for this problem, please discuss and give opinion. > > > > > > > > Thanks, > > > > Ruguo Yu > > > > > > > > [1] https://github.com/apache/pulsar/issues/12822 > > > > > > >
[DISCUSS] New repository for website - pulsar-site
Hi - There are two efforts happening in the community around website refresh. (1) Docusaurus upgrades. (2) New web design. There is an effort to eliminate all the extra commits in the asf-site branch of the main repository. In that thread I proposed a new repository for the website. We can then discuss migration and development both on this mailing list and as PRs and Issues in that repository. Do we want to have a PIP process here or can we be less formal? I think that PRs. Issues, and simple commits can be sufficient. Unless there are objections I will create a new repository - pulsar-site on Friday in 72 hours. ‘ Regards, Dave
[GitHub] [pulsar-client-node] shaunek commented on issue #105: Implement `consumer.unsubscribe()`
shaunek commented on issue #105: URL: https://github.com/apache/pulsar-client-node/issues/105#issuecomment-970218846 Can this issue be closed? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
