SEZ9 commented on issue #13262: URL: https://github.com/apache/seatunnel/issues/13262#issuecomment-6842843802
Thanks for the well-scoped proposal. Making Lance support symmetric by adding a batch source that reads from the same kinds of locations the sink writes to (local filesystem or S3/MinIO) makes sense, and it's great that you're willing to submit a PR. Before you start, could you clarify the intended scope for the first version? In particular: batch-only reading, whether you plan optional column projection or dataset version/snapshot selection, and whether filter push-down is in scope. It would also help to know which Lance Java SDK version you plan to target and how Lance types (especially vector/list and blob types) will map to SeaTunnel's data types. When you open the PR, please follow the existing Connector-V2 source conventions, reuse the sink's storage/credential options so configuration stays consistent between source and sink, add source documentation, and update the sink doc sentence stating there is no Lance source. <!-- streview-comment:2315 --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
