GitHub user Susmit07 added a comment to the discussion: Reading a parquet file and writing to s3 using pekko connectors.
Thank you @raboof as always for your reply yes you are correct It is a good idea to stick with multipartUpload for large files (typically >500MB), but for smaller files, putObject can it be considered ? The advantage with putObject i feel is all or none, but for multipartUpload the part files i need to have a lifecycle policy if upload gets snapped out in between (considering we are not in upload resumability currently) GitHub link: https://github.com/apache/pekko-connectors/discussions/857#discussioncomment-10857586 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
