Kaileshwar16 opened a new issue, #853:
URL: https://github.com/apache/wayang/issues/853

   In `wayang-jdbc-template`, I found a couple of issues in 
`SqlToStreamOperator` when streaming query results from JDBC.
   
   - JDBC resources are not always closed
   
   In `SqlToStreamOperator.evaluate()`, a JDBC connection is created and used 
to read query results. However, `ResultSetIterator` only closes the `ResultSet` 
when iteration reaches the end. The underlying `Statement` and `Connection` are 
not closed, and if a downstream operator stops reading early, even the 
`ResultSet` may remain open.
   
   This could leave database resources hanging around longer than necessary.
   
   - Fetch size configuration is commented out
   
   In `ResultSetIterator`, `Statement.setFetchSize()` is currently commented 
out. For large queries, this can cause some JDBC drivers to load too much data 
into memory instead of fetching rows in batches.
   
   Is there any particular reason it is commented out? It would be useful to 
support a configurable fetch size, so users can tune how many rows are fetched 
at a time. Some drivers, such as PostgreSQL, may also require additional 
settings, like disabling auto-commit, for cursor-based fetching to work 
properly.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to