Nice idea, IO looks like a good place for them but there is another path that could fit this case: `sdks/java/extensions`, some module like `google-cloud-platform-ai` in that folder or something like that, no?
In any case great initiative. +1 On Tue, Jan 14, 2020 at 4:22 PM Kamil Wasilewski < [email protected]> wrote: > Hi all, > > We’d like to implement a set of PTransforms that would allow users to use > some of the Google Cloud AI services in Beam pipelines. > > Here's the full list of services and functionalities we’d like to > integrate Beam with: > > * Video Intelligence [1] > > * Cloud Natural Language [2] > > * Cloud AI Platform Prediction [3] > > * Data Masking/Tokenization [4] > > * Inspecting image data for sensitive information using Cloud Vision [5] > > However, we're not sure whether to put those transforms directly into > Beam, because they would require some additional GCP dependencies. One of > our ideas is a separate library, that depends on Beam and that can be > installed optionally, stored somewhere in the beam repository (e.g. in the > BEAM_ROOT/extras directory). Do you think it is a reasonable approach? Or > maybe it is totally fine to put them into SDKs, just like other IOs? > > If you have any other thoughts, do not hesitate to let us know. > > Best, > > Kamil > > [1] https://cloud.google.com/video-intelligence/ > > [2] https://cloud.google.com/natural-language/ > > [3] https://cloud.google.com/ml-engine/docs/prediction-overview > > [4] > https://cloud.google.com/dataflow/docs/guides/templates/provided-streaming#dlptexttobigquerystreaming > > [5] https://cloud.google.com/vision/ >
