Hi Rob, All our transforms (except for CSV input) can read an write directly to S3. It will use the default configured AWS account (there is no support for profiles). Multi-account support is on the roadmap.
More information can be found under the Virtual File System section in the documentation [1]. 1 consideration that Diego has mentioned is that listing of bucket content when a bucket contains 10K+ files can be slow, first using the CLI to list the content saving that to a txt and then using that as an input can be faster. Cheers, Hans [1] https://hop.apache.org//manual/latest/vfs/aws-s3-vfs.html On 23 Jul 2024 at 02:45 +0200, Diego Mainou <[email protected]>, wrote: > Hi Rob, > > 2 ways. > > 1. Mount the S3 as a VFS and simply use the file copy/move step > 2. Use the bash step and execute commands through the cli > > the top one is easy albeit sometimes may be a bit slower than the S3 native > cli. > > > Diego Mainou > Business Development Manager > M. +61 415 152 091 > E. [email protected] > www.bizcubed.com.au > https://www.linkedin.com/in/diegomainou/ > > > Want to hear more about what BizCubed is doing? > > • Follow us on LinkedIn > • Sign up for our newsletter > > > From: "Rob Burgess" <[email protected]> > To: "users" <[email protected]> > Sent: Tuesday, 23 July, 2024 6:28:08 AM > Subject: Uploading files to S3 bucket > > Hi > > What is the best Hop action to use to upload files to a S3 bucket? > > Thanks > > Rob
