[
https://issues.apache.org/jira/browse/GSOC-260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Priya Sharma updated GSOC-260:
------------------------------
Labels: Beam gsoc gsoc2024 mentor (was: Beam gsoc gsoc2024)
> [GSOC][Beam] Add connectors to Beam ManagedIO
> ---------------------------------------------
>
> Key: GSOC-260
> URL: https://issues.apache.org/jira/browse/GSOC-260
> Project: Comdev GSOC
> Issue Type: New Feature
> Reporter: Danny McCormick
> Priority: Major
> Labels: Beam, gsoc, gsoc2024, mentor
>
> Apache Beam is a unified model for defining both batch and streaming
> data-parallel processing pipelines, as well as a set of language-specific
> SDKs for constructing pipelines and Runners for executing them on distributed
> processing backends. On top of providing lower level primitives, Beam has
> also introduced several higher level transforms used for machine learning and
> some general data processing use cases. One new transform that is being
> actively worked on is a unified ManagedIO transform which gives runners the
> ability to manage (upgrade, optimize, etc...) an IO (input source or output
> sink) without upgrading the whole pipeline. This project will be about adding
> one or more IO integrations to ManagedIO
> Objectives:
> 1. Add a BigTable integration to ManagedIO
> 2. Add a Spanner integration to ManagedIO
> Useful links:
> Apache Beam repo - https://github.com/apache/beam
> Docs on ManagedIO are relatively light since this is a new project, but here
> are some docs on existing IOs in Beam -
> https://beam.apache.org/documentation/io/connectors/
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: gsoc-unsubscr...@community.apache.org
For additional commands, e-mail: gsoc-h...@community.apache.org
