Hi All, I need s3 sync functionality, and currently use it via bash. Would love for an operator to exist. I see nothing in Jira. If not here, it looks like that would be the place to add a ticket, get any feedback and then build?
Thanks, Austin On Sat, May 4, 2019 at 6:23 PM Austin Bennett <whatwouldausti...@gmail.com> wrote: > Hi Airflow Devs, > > I am not having any luck finding some S3 functionality that I would like > to use. > > Specifically the ability to sync buckets. And, the ability to recursively > copy. > > `aws s3 sync s3://source-bucket/ s3://destination-bucket` > > or > > `aws s3 cp s3://source-bucket/folder/ s3://destination-bucket/folder > --recursive` > > It seems I could run these with the bash operator, but I am wondering > about using a python based operator that would likely call boto. > > Does anything like this exist and I'm unable to find it? Otherwise, I'd > be happy to work on this if the community felt it would be useful. > > Please advise - > > Thanks, > Austin > >
