(Applied Ahmet's feedback in https://github.com/apache/beam/pull/11156)
Hi Luke, > the pom files that are produced Build.gradle that uses the BOM will produce pom files that have corresponding dependencies without versions, and have a "dependencyManagement" section through which they import the GCP Libraries BOM. This fills the versions of the versionless dependencies when build systems evaluates the pom files. > how the GCP BOM impacts the release process I'm afraid I don't have knowledge on how dependencies affect the Beam release process. Would you be willing to break down this concern into some questions I can answer? On Fri, Mar 20, 2020 at 6:13 PM Luke Cwik <lc...@google.com> wrote: > My concern would be to validate how the GCP BOM impacts the release > process and the pom files that are produced otherwise the next person > running a release may run into trouble. > > On Fri, Mar 20, 2020 at 3:00 PM Pablo Estrada <pabl...@google.com> wrote: > >> +1 >> >> On Fri, Mar 20, 2020 at 2:41 PM Ismaël Mejía <ieme...@gmail.com> wrote: >> >>> Well why we just don't merge it? I am unfamiliar with GCP deps to be >>> confident to LGTM it. but given that 22 checks pass and given that >>> Tomo addressed most comments and he has already a consistent track of >>> good work on dependency improvements I think it is worth to merge it >>> as it is. We still have some time to fix stuff if we find any >>> regression. WDYT? >>> >>> On Thu, Mar 19, 2020 at 9:36 PM Luke Cwik <lc...@google.com> wrote: >>> > >>> > As much as I would like to spend time on these reviews, I believe I'll >>> be delayed from reviewing them thoroughly till I finish other work that I'm >>> targeting for the 2.21 release related to portability. It would be greatly >>> appreciated if there are others that could review this in the meantime. >>> > >>> > On Thu, Mar 19, 2020 at 7:09 AM Tomo Suzuki <suzt...@google.com> >>> wrote: >>> >> >>> >> PR is ready (22 successful check) >>> >> https://github.com/apache/beam/pull/11156 >>> >> (Luke assigned himself as a reviewer) >>> >> >>> >> Regards, >>> >> Tomo >>> >> >>> >> On Wed, Mar 11, 2020 at 3:50 PM Tomo Suzuki <suzt...@google.com> >>> wrote: >>> >>> >>> >>> Thank you for favorable responses. I'll start implementation. >>> >>> >>> >>> On Thu, Mar 5, 2020 at 2:22 PM Tomo Suzuki <suzt...@google.com> >>> wrote: >>> >>>> >>> >>>> > Do Spark or Flink have BOMs? >>> >>>> >>> >>>> Not that I know of. I couldn't find "bom" in their artifacts [1, 2]. >>> >>>> >>> >>>> [1]: https://search.maven.org/search?q=g:org.apache.flink >>> >>>> [2]: https://search.maven.org/search?q=g:org.apache.spark >>> >>>> >>> >>>> >>> >>>> On Thu, Mar 5, 2020 at 1:46 PM Kenneth Knowles <k...@apache.org> >>> wrote: >>> >>>>> >>> >>>>> +1 and you have phrased the benefits and limitations well. We have >>> plenty of not-Google-related dependencies that use Guava and protobuf (I >>> know of Calcite, Cassandra, Kinesis, and Spark) so there's still work in >>> managing deps, but the BOM should make it a lot easier to upgrade all these >>> tightly coupled libraries that Google ships from their head. >>> >>>>> >>> >>>>> Do Spark or Flink have BOMs? I wonder if there's an opportunity to >>> catch incompatible deps at a larger scale by comparing and merging a half >>> dozen BOMs (although in the limit it approximately expands to one per >>> runner and one per IO and projects mature and become independent) >>> >>>>> >>> >>>>> Kenn >>> >>>>> >>> >>>>> On Thu, Mar 5, 2020 at 10:05 AM Luke Cwik <lc...@google.com> >>> wrote: >>> >>>>>> >>> >>>>>> How would the Apache Beam BOM and GCP BOM work together? >>> >>>>>> >>> >>>>>> On Thu, Mar 5, 2020 at 7:25 AM Filipe Regadas < >>> filiperega...@gmail.com> wrote: >>> >>>>>>> >>> >>>>>>> Big +1, this is a step in the right direction and checking with >>> other Beam's direct and transitive deps is crucial since the referred bom >>> only convers a small part of it. Apache Commons, Jackson, `com.google.{api, >>> apis, cloud}`, slf4j comes to mind. >>> >>>>>>> >>> >>>>>>> Filipe Regadas >>> >>>>>>> >>> >>>>>>> >>> >>>>>>> On Thu, Mar 5, 2020 at 3:33 AM Ismaël Mejía <ieme...@gmail.com> >>> wrote: >>> >>>>>>>> >>> >>>>>>>> +1 Sounds like a good improvement for users and maintainers ! >>> >>>>>>>> >>> >>>>>>>> On Thu, Mar 5, 2020 at 6:59 AM Alex Van Boxel <a...@vanboxel.be> >>> wrote: >>> >>>>>>>> > >>> >>>>>>>> > +1, I can remember the countless hours that we fought with >>> Google dependencies. >>> >>>>>>>> > >>> >>>>>>>> > On Thu, Mar 5, 2020, 04:07 Chamikara Jayalath < >>> chamik...@google.com> wrote: >>> >>>>>>>> >> >>> >>>>>>>> >> +1 for this. >>> >>>>>>>> >> >>> >>>>>>>> >> This will make life easy for many of our users and will help >>> us keep GCP related dependencies compatible (which has not been easy in the >>> past). >>> >>>>>>>> >> >>> >>>>>>>> >> On Wed, Mar 4, 2020 at 2:16 PM Tomo Suzuki < >>> suzt...@google.com> wrote: >>> >>>>>>>> >>> >>> >>>>>>>> >>> Hi Beam developers, >>> >>>>>>>> >>> >>> >>>>>>>> >>> Shall we use GCP Libraries BOM [1] to specify the >>> Google-related library versions in Beam? >>> >>>>>>>> >>> >>> >>>>>>>> >>> I've been working on Beam's dependency upgrades in the past >>> few months. It's time to consider a long-term solution to keep the >>> libraries up-to-date with small maintenance effort. To achieve that, I >>> propose Beam to use GCP Libraries BOM to set the Google-related library >>> versions, rather than the current way of making changes in each of ~30 >>> Google libraries with individual PRs [2]. >>> >>>>>>>> >>> >>> >>>>>>>> >>> After the proposal is implemented, Beam project upgrades >>> the BOM version to upgrade these Google-related libraries. This still needs >>> to ensure the libraries in GCP Library BOM are compatible with Beam's other >>> dependencies. (Linkage Checker will help with this job.) I believe >>> onboarding GCP Libraries BOM will solve lots of incompatibilities which we >>> have seen in gax, gRPC, google-cloud-core, and so on with minimal effort in >>> Beam's developers. >>> >>>>>>>> >>> >>> >>>>>>>> >>> Created an issue to track this: BEAM-9444 [3]. I appreciate >>> if you can share questions or feedback (thumbs-up / concerns). >>> >>>>>>>> >>> >>> >>>>>>>> >>> [1]: >>> https://github.com/GoogleCloudPlatform/cloud-opensource-java/wiki/The-Google-Cloud-Platform-Libraries-BOM >>> >>>>>>>> >>> [2]: >>> https://github.com/apache/beam/pulls?page=1&q=is%3Apr+author%3Asuztomo >>> >>>>>>>> >>> [3]: https://issues.apache.org/jira/browse/BEAM-9444 >>> >>>>>>>> >>> >>> >>>>>>>> >>> -- >>> >>>>>>>> >>> Regards, >>> >>>>>>>> >>> Tomo >>> >>>> >>> >>>> >>> >>>> >>> >>>> -- >>> >>>> Regards, >>> >>>> Tomo >>> >>> >>> >>> >>> >>> >>> >>> -- >>> >>> Regards, >>> >>> Tomo >>> >> >>> >> >>> >> >>> >> -- >>> >> Regards, >>> >> Tomo >>> >> -- Regards, Tomo
