2013650523 commented on issue #1946: URL: https://github.com/apache/incubator-seatunnel/issues/1946#issuecomment-1221976938
> Hi, Brothers. I'm having some difficulties with the submission of the second segment of the sink,I went to consult the HDFS sink Connector,Its precommit is to write data to a temporary file,If the pre-commit is successful, the file name is changed to achieve the actual commit data ,If this fails, the temporary file data is deleted. I don't quite understand what method should be implemented in the pre-commit, what method should be selected for the final commit, My initial idea is that if the sink target side supports the transaction mechanism, it can be implemented using the transaction mechanism. If there is no transaction mechanism, it can only be implemented through a similar HDFS sink connector.The problem is that I can't find a way to rename files in the Minio Client API ,But Minio has a CopyObject API for submitting data a second time;I don't know whether it is correct to implement S3 sink in this way, and whether it is correct to understand the two-phase commit -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
