+1 for this plan. It's good to finally make progress on PIP-62 (
https://github.com/apache/pulsar/wiki/PIP-62%3A-Move-connectors%2C-adapters-and-Pulsar-Presto-to-separate-repositories)
.
I see that one gap in the current PIP-62 document is the lack of relevant
details. I feel that it is necessary to define how the repositories and
libraries depend on each other and what are the release cycles and
versioning strategies (backwards compatibility etc.).
Some questions that could help formulate to the decisions:
1)
a.) What is the release cycle of apache/pulsar-presto repository?
b.) Is it independent of the release cycle of apache/pulsar?
c.) What is the versioning scheme of the pulsar-presto repository /
libraries?
2)
a.) What libraries ("interfaces") does apache/pulsar-presto depend on
from apache/pulsar?
b.) What is the release cycle of the apache/pulsar "interfaces" that
pulsar-presto depends on?
c.) How are breaking interface changes handled?
d.) How is backwards compatibility verified?
Could we spend some time to document the decisions on these aspects?
The benefit of this exercise is that it could help bring up the possible
issues in dependency, release and version management before implementing
anything. It would also clarify the details of PIP-62 and make it easier to
understand the implications in doing Pulsar releases after this change has
been made.
Sijie, WDYT?
BR, Lari
On Fri, Mar 12, 2021 at 10:33 AM Sijie Guo <[email protected]> wrote:
> Hi all,
>
> I had a conversation with Jerry and Matteo about NOT including presto/trino
> in the normal Pulsar distribution.
>
> Here is the summary of the discussion:
>
> - Move the presto connector to pulsar-presto repo as part of PIP-62
> - In the main pulsar repo, we don't bundle presto in the pulsar
> distribution
> - In pulsar-presto repo, we ship it as a standalone connector that can be
> installed in an existing trinodb cluster. We can also ship a pulsar-presto
> image or distribution which includes trinodb. So people can still easily
> test it.
> - Update the documentation: People can use pulsar-presto image or
> distribution to test presto connector easily; for production deployment, we
> focus on documenting how to deploy the connector to an existing trinode
> cluster.
>
> Let me know what do you think. If there is no objection, I will move
> forward with this plan.
>
> Thanks,
> Sijie
>
