[ https://issues.apache.org/jira/browse/AIRFLOW-2348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16457750#comment-16457750 ]
ASF subversion and git services commented on AIRFLOW-2348: ---------------------------------------------------------- Commit 840930b4d0c545adc24d8eb040974f88370e081b in incubator-airflow's branch refs/heads/master from [~csoulios] [ https://git-wip-us.apache.org/repos/asf?p=incubator-airflow.git;h=840930b ] [AIRFLOW-2348] Strip path prefix from the destination_object when source_object contains a wildcard[] Closes #3247 from csoulios/master > GoogleCloudStorageToGoogleCloudStorageOperator copies full source path > ---------------------------------------------------------------------- > > Key: AIRFLOW-2348 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2348 > Project: Apache Airflow > Issue Type: Bug > Components: gcp > Affects Versions: 2.0.0 > Reporter: Christos Soulios > Priority: Minor > Fix For: 1.10.0 > > > I use the GoogleCloudStorageToGoogleCloudStorageOperator to copy a directory > recursively from source dir to target dir. When files are copied, the full > source dir hierarchy is included. > For example, I want to copy the following files to /path/to/destination > {{/path/to/source/a/file1.txt}} > {{/path/to/source/a/file2.txt}} > {{/path/to/source/b/file3.txt}} > > When running: > {{stage_files = GoogleCloudStorageToGoogleCloudStorageOperator(}} > task_id='task', > source_bucket='gs_bucket', > source_object='/path/to/source/*', > {{ destination_object='/path/to/destination')}} > I get the following result: > {{/path/to/destination/path/to/source/a/file1.txt}} > {{/path/to/destination/path/to/source/a/file2.txt}} > {{/path/to/destination/path/to/source/b/file3.txt}} > > I would expect the the operator behavior is similar to gsutil and would > return the following result: > {{/path/to/destination/a/file1.txt}} > {{/path/to/destination/a/file2.txt}} > {{/path/to/destination/b/file3.txt}} > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)