How will this be communicated to the user? The idea is that they will discover PubsubLiteIO through their IDE as you described, but that will get them to the Beam one that's subject to the long release cycle. Will it just be documented somewhere that users should prefer com.google.cloud.pubsublite.beam.PubsubLiteIO if there's a recent fix they need?
I wonder if a similar result could be achieved just by making Beam's PubsubLiteIO a stub with no implementation that directs users to the com.google.cloud one somehow? junit's matcher interface comes to mind as a precedent here. I have been warned many times by Matcher._dont_implement_Matcher___instead_extend_BaseMatcher_ [1]. [1] https://junit.org/junit4/javadoc/4.13/org/hamcrest/Matcher.html#_dont_implement_Matcher___instead_extend_BaseMatcher_() Brian On Thu, Jun 17, 2021 at 3:56 PM Daniel Collins <[email protected]> wrote: > > Question 1: How are you going to approach testing/CI? > The pull requests in the java-pubsublite repo do not trigger Beam repo's > CI. You want to deliver things to your customers after they are tested as > much as possible. > > I'd like to run the integration tests in both locations. They would only > be meaningful in the beam setup when we went to validate a version bump on > the I/O. > > > Question2 : in the code below, what is the purpose of keeping the > PubsubLiteIO in the Beam repo? > > Visibility and autocomplete. It means the core class will be in the beam > javadoc and if you type `import org.apache.beam.sdk.io.gcp.pubsu` in an IDE > you'll see pubsublite and PubsubLiteIO. > > On Thu, Jun 17, 2021 at 5:35 PM Tomo Suzuki <[email protected]> wrote: > >> Hi Daniel, >> (You helped me apply some change to this strange setup a few months back. >> Thank you for working on rectifying the situation.) >> >> I like that idea overall. >> >> Question 1: How are you going to approach testing/CI? >> The pull requests in the java-pubsublite repo do not trigger Beam repo's >> CI. You want to deliver things to your customers after they are tested as >> much as possible. >> >> >> Question2 : in the code below, what is the purpose of keeping the >> PubsubLiteIO in the Beam repo? >> >> ``` >> class PubsubLiteIO extends com.google.cloud.pubsublite.beam.PubsubLiteIO >> {} >> ```` >> >> The backward compatibility came to my mind but I thought you may have >> more reasons. >> >> >> My memo: >> java-pubsublite repsitory has: >> https://github.com/googleapis/java-pubsublite/blob/master/pubsublite-beam-io/src/main/java/com/google/cloud/pubsublite/beam/PubsubLiteIO.java >> beam repo has: >> https://github.com/apache/beam/blob/master/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/pubsublite/PubsubLiteIO.java >> (and other files in the same directory) >> google-cloud-pubsublite is not part of the Libraries BOM (yet) because of >> its pre-1.0 status. >> >> >> On Thu, Jun 17, 2021 at 5:07 PM Daniel Collins <[email protected]> >> wrote: >> >>> I don't know that the cycle would cause a problem- wouldn't it override >>> and cause it to use beam-sdks-java-core:2.30.0 (at least until beam goes to >>> 3.X.X)? >>> >>> Something we can do if this is an issue is mark pubsublite-beam-io's dep >>> on beam-sdks-java-core as 'provided'. But I'd prefer to avoid this and just >>> let overriding fix it if that works. >>> >>> On Thu, Jun 17, 2021 at 4:15 PM Andrew Pilloud <[email protected]> >>> wrote: >>> >>>> How do you plan to address the circular dependency? Won't this end up >>>> with Beam depending on older versions of itself? >>>> >>>> beam-sdks-java-io-google-cloud-platform:2.30.0 -> >>>> pubsublite-beam-io:0.16.0 -> beam-sdks-java-core:2.29.0 >>>> >>>> On Thu, Jun 17, 2021 at 11:56 AM Daniel Collins <[email protected]> >>>> wrote: >>>> >>>>> Hello beam developers, >>>>> >>>>> I'm the primary author of the Pub/Sub Lite I/O, and I'd like to get >>>>> some feedback on a change to the model for hosting this I/O in beam. Our >>>>> team has been frustrated by the fact that we have no way to release >>>>> features or fixes for bugs to customers on time scales shorter than the >>>>> 1-2 >>>>> months of the beam release cycle, and that those fixes are necessarily >>>>> coupled with a beam version upgrade. To work around this, I forked the I/O >>>>> in beam to our own repo about 6 months ago and have been maintaining both >>>>> copies in parallel. >>>>> >>>>> I'd like to retain our ability to quickly fix and improve the I/O >>>>> while retaining end-user visibility within the beam repo. To do this, I'd >>>>> like to remove all the implementation from the beam repo, and leave the >>>>> I/O >>>>> there implemented as: >>>>> >>>>> ``` >>>>> class PubsubLiteIO extends >>>>> com.google.cloud.pubsublite.beam.PubsubLiteIO {} >>>>> ```` >>>>> , and add a dependency on our beam artifact. >>>>> >>>>> This enables beam users who want to just use the >>>>> beam-sdks-java-io-google-cloud-platform artifact to do so, but they can >>>>> also track the canonical version separately in our repo to get fixes and >>>>> improvements at a faster rate. All static methods from the parent class >>>>> would be available on the class in the beam repo. >>>>> >>>>> I'd be interested to hear anyones thoughts and suggestions surrounding >>>>> this. >>>>> >>>>> -Daniel >>>>> >>>> >> >> -- >> Regards, >> Tomo >> >
