Just curious to learn is there any progress on moving all the connectors into separate repos? Maybe I can help if the decision is finalized.
On 2021/11/17 06:18:52 Lari Hotari wrote: > Dear Pulsar community members, > > PIP-62[1], "PIP 62: Move connectors, adapters and Pulsar Presto to separate > repositories" was created in April 2020. The repositories for > pulsar-connectors, pulsar-adapters and pulsar-sql were created about a year > ago [2]. > > What is the current roadmap for completing PIP-62 and moving > pulsar-connectors and pulsar-sql out of apache/pulsar repository? > > BR, > > Lari > > [1] > https://github.com/apache/pulsar/wiki/PIP-62%3A-Move-connectors%2C-adapters-and-Pulsar-Presto-to-separate-repositories > [2] > https://lists.apache.org/thread.html/r9e6ec742e2896da1f0ce7d4adc7cb84fc6db6dbf797732ccdd50fb86%40%3Cdev.pulsar.apache.org%3E > > Other email threads: > * [Discuss] Don't include presto/trino in the normal Pulsar distribution - > https://lists.apache.org/thread/jn96tct54mn0tvdot62vdslrvs38fm6d > * Updates on Presto connector for PIP-62 - > https://lists.apache.org/thread/f9n6sc2mrboq5sxhjbr7gvdl8vqp9fpk > > On Tue, Nov 2, 2021 at 3:59 PM Nicolò Boschi <boschi1...@gmail.com> wrote: > > > Resurrecting this thread. > > > > 2.9 is almost released and it hasn't been merged yet > > > > Extending the discussion to other connectors, it looks like there has been > > no progress on PIP-62. > > My concern is that a lot of Pulsar IO connectors dependencies we are > > running are obsolete with several security reports > > > > I see there are interesting comments in the issue ( > > https://github.com/apache/pulsar/issues/10219) and Sijie exported the > > pulsar-io dir to https://github.com/apache/pulsar-connectors but it's > > outdated > > > > From my point of view, we have to: > > - reimport all the connectors source codes with newest ones (including > > integration tests) > > - add periodic CI jobs for connectors to run against master, 2.9-latest, > > 2.8-latest, 2.7-latest to verify breaking changes > > - define a release cycle/management for connectors (we should improve the > > PIP doc). IMO it's not clear if each connector will have its own release > > versions and how we'll handle it (git tags, artifacts deployment..) > > - update pulsar release script in order to get the connectors artifacts > > (retrieving the .nar or building it from source?) > > - update docs > > - remove pulsar-io dir from Pulsar repo > > > > It's the perfect timing to schedule this work for 2.10 > > > > What is missing? How's the situation? Is there a roadblock I haven't seen? > > I think it's better to take another discussion for Presto since it will > > come to another end > > > > > > Il giorno sab 14 ago 2021 alle ore 15:21 Enrico Olivelli < > > eolive...@gmail.com> ha scritto: > > > > > Sijie > > > > > > Il Ven 13 Ago 2021, 22:00 Sijie Guo <guosi...@gmail.com> ha scritto: > > > > > > > You can follow the progress at > > > https://github.com/trinodb/trino/pull/8020. > > > > > > > > > > Thanks for the pointer > > > > > > > > > > > The original code doesn't conform to TrinoDB's standard. Marvin is > > > > actively following up on that. > > > > > > > > Our goal is still to get this completed as part of the 2.9 release. > > > > > > > > > > Wonderful > > > > > > Thanks > > > Enrico > > > > > > > > > > > - Sijie > > > > > > > > On Fri, Aug 13, 2021 at 2:04 AM Enrico Olivelli <eolive...@gmail.com> > > > > wrote: > > > > > > > > > > Hello, > > > > > How is the Presto work going ? > > > > > IIRC the plan was to remove it from the Pulsar code base and let it > > be > > > > > hosted at Trino. > > > > > > > > > > If this is not going to happen within the 2.9.0 release timeline > > > > > (September?) I would prefer to upgrade to "Trino". > > > > > Probably we will have a downside problem that recent versions of > > > > > Presto/Trino do not work on JDK8 but only on JDK11. > > > > > > > > > > I believe that in that case we could open a separate thread to say > > that > > > > > Pulsar SQL in 2.9.0 will work only on JDK11. > > > > > In Pulsar 2.8.0 we added official compatibility with JDK11 (and it is > > > the > > > > > preferred version, as it is the version used in the Docker images), > > so > > > > > requiring JDK11 for Pulsar SQL 2.9.0 does not sound bad to me. > > > > > > > > > > My primary concern is that the version of Presto that we are running > > is > > > > > obsolete and there are several security reports against it or its > > third > > > > > party dependencies. > > > > > > > > > > Thoughts ? > > > > > > > > > > Enrico > > > > > > > > > > > > > -- > > Nicolò Boschi > > >
