Thanks so much! Great to see this to be picked up again with some good progress. / Moritz
On 11.12.22, 15:17, "Herman Mak via dev" <[email protected]> wrote: Hello Everyone, *TLDR* Should we adopt a set of standards that Connector I/Os should adhere to? Attached is a first version of a Beam I/O Standards guideline that includes opinionated best practices across important components of a Connector Hello Everyone, *TLDR* Should we adopt a set of standards that Connector I/Os should adhere to? Attached is a first version of a Beam I/O Standards guideline that includes opinionated best practices across important components of a Connector I/O, namely Documentation, Development and Testing. *The Long Version* Apache Beam is a unified open-source programming model for both batch and streaming. It runs on multiple platform runners and integrates with over 50 services using individually developed I/O Connectors<https://urldefense.com/v3/__https:/beam.apache.org/documentation/io/connectors/__;!!CiXD_PY!Uj2lofdfCPxVIvik15k872DMWscqEkfnc_JW0aTmPIGePaTWnARDbaTnZLDEfFA1va7K-Jz9Iy9i$>. Given that Apache Beam connectors are written by many different developers and at varying points in time, they vary in syntax style, documentation completeness and testing done. For a new adopter of Apache Beam, that can definitely cause some uncertainty. So should we adopt a set of standards that Connector I/Os should adhere to? Attached is a first version, in Doc format, of a Beam I/O Standards guideline that includes opinionated best practices across important components of a Connector I/O, namely Documentation, Development and Testing. And the aim is to incorporate this into the documentation and to have it referenced as standards for new Connector I/Os (and ideally have existing Connectors upgraded over time). If it looks helpful, the immediate next step is that we can convert it into a .md as a PR into the Beam repo! Thanks and looking forward to feedbacks and discussion, [https://drive-thirdparty.googleusercontent.com/16/type/application/vnd.google-apps.document] [PUBLIC] Beam I/O Standards<https://urldefense.com/v3/__https:/docs.google.com/document/d/1BCTpSZDUjK90hYZjcn8aAnPd9vuRfj8YU1j3mpSgRwI/edit?usp=drive_web__;!!CiXD_PY!Uj2lofdfCPxVIvik15k872DMWscqEkfnc_JW0aTmPIGePaTWnARDbaTnZLDEfFA1va7K-HPnxEXQ$> [https://lh4.googleusercontent.com/x-XXWb614Zh49ixe0GySnZFBXTBs2gzMCyVmAdh4_dSwrqW_4nmq3hq5TtPRFPJaj4I125-ehGvHxPKhiC07EhRK9VvfIKCWXcJsZd-WfCMgO5MK_7BhYEibHo3L_R0PqKgGAGjh] Herman Mak | Customer Engineer, Hong Kong, Google Cloud | [email protected]<mailto:[email protected]> | +852-3923-5417<tel:+852%203923%205417> As a recipient of an email from Talend, your contact personal data will be on our systems. Please see our privacy notice. <https://www.talend.com/privacy/>
