Christos Soulios created AIRFLOW-2348: -----------------------------------------
Summary: GoogleCloudStorageToGoogleCloudStorageOperator copies full source path Key: AIRFLOW-2348 URL: https://issues.apache.org/jira/browse/AIRFLOW-2348 Project: Apache Airflow Issue Type: Bug Components: gcp Affects Versions: 2.0.0 Reporter: Christos Soulios I use the GoogleCloudStorageToGoogleCloudStorageOperator to copy a directory recursively from source dir to target dir. When files are copied, the full source dir hierarchy is included. For example, I want to copy the following files to /path/to/destination {{/path/to/source/a/file1.txt}} {{/path/to/source/a/file2.txt}} {{/path/to/source/b/file3.txt}} When running: {{stage_files = GoogleCloudStorageToGoogleCloudStorageOperator(}} {{ task_id='task',}} {{ source_bucket='gs_bucket',}} {{ source_object='/path/to/source/*',}} {{destination_object='/path/to/destination')}} I get the following result: {{/path/to/destination/path/to/source/a/file1.txt}} {{/path/to/destination/path/to/source/a/file2.txt}} {{/path/to/destination/path/to/source/b/file3.txt}} I would expect the the operator behavior is similar to gsutil and would return the following result: {{/path/to/destination/a/file1.txt}} {{/path/to/destination/a/file2.txt}} {{/path/to/destination/b/file3.txt}} -- This message was sent by Atlassian JIRA (v7.6.3#76005)