Hi everyone, I'm Zihan Dai (GitHub: PDGGK), a CS student at the University of Melbourne. I've already been doing this work in Beam: I have five merged PRs in the project, including #37681, which fixed a real resource leak in KafkaReadSchemaTransformProvider.identityOrGcsToLocalFile() where a GCS ReadableByteChannel, WritableByteChannel, and FileOutputStream were manually closed after the copy loop and could leak on exception. I also have another resource-management fix under review in #37530.
That work convinced me this is not a one-off Kafka problem. The Java IO layer has the same failure mode in multiple places: long-lived external resources created outside try-with-resources, cleanup split across start()/close() or @Setup/@Teardown, and a few older fallback patterns that should be removed. Two concrete examples I've already identified are JmsIO's JmsReader.finalize() path (around line 851, also relevant to Beam issue #24181 on eliminating deprecated finalize() usage) and the unclosed InputStream path in sdks/java/io/solace/broker/BrokerResponse.java. The connectors I would prioritize next are the ones with the highest lifecycle complexity: JdbcIO (Connection / PreparedStatement), CassandraIO (Cluster / Session), MongoDbIO.BoundedMongoDbReader (MongoClient / MongoCursor), Neo4jIO (Driver / Session), ElasticsearchIO / SolrIO (RestClient / AuthorizedSolrClient), and AWS2 readers such as Kinesis and SQS. My proposed GSoC project is to turn this from ad hoc bug-fixing into a systematic hardening effort: Weeks 1-3, Audit: Review sdks/java/io/**, classify resource-owning patterns by connector, map them to Beam execution lifecycles (BoundedReader.close, UnboundedReader.close, DoFn @Setup/@Teardown, checkpoint paths), and file a concrete issue list. Weeks 4-9, Fixes: Land connector fixes with tests for exception-safe cleanup, idempotent close behavior, and teardown correctness under retries/checkpointing. I expect Kafka, JMS, Solace, JDBC, Cassandra, MongoDB, and selected AWS2 connectors to be the first wave. Weeks 10-12, Prevention: Integrate prevention into Beam's existing Error Prone setup in buildSrc/.../BeamModulePlugin.groovy. Beam already applies net.ltgt.errorprone with error_prone_core, but it currently carries a large disabled-check list, including Finalize (#20955). Beam also does not seem to have an in-tree custom BugChecker module today. I'd like to prototype either targeted use of @MustBeClosed or a Beam-specific Error Prone plugin for resource-lifecycle patterns that upstream checks do not cover well. This aligns with the existing IO Standards document's call for standardized resource management, the long-standing issue #24742 (IO Connector Error Handling), and the current ErrorProne enforcement campaign (#37788 and related PRs from March 2026). Questions: 1. Would maintainers prefer this scoped as a connector-focused umbrella effort, or tied directly to broader issues like #24181 and #20955? 2. For the prevention phase, would you rather see a Beam-specific Error Prone checker added to the existing errorprone classpath, or start with upstream mechanisms such as @MustBeClosed plus re-enabling Finalize once the remaining usages are removed? 3. For bundle-scoped resources in connectors like JdbcIO, Neo4jIO, and CassandraIO, would a reusable Beam helper for @Setup/@Teardown resource ownership be welcome, or should this stay connector-by-connector first? Best, Zihan Dai GitHub: https://github.com/PDGGK
