This is an automated email from the ASF dual-hosted git repository.
potiuk pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/main by this push:
new e444bca140 Add links between documentation related to Google Cloud
Storage (#34994)
e444bca140 is described below
commit e444bca140cc38619316ca5f6325ba53d4e8e426
Author: Przemek Więch <[email protected]>
AuthorDate: Wed Oct 18 12:25:21 2023 +0200
Add links between documentation related to Google Cloud Storage (#34994)
* Add links between documentation related to Google Cloud Storage
* Fix links
* Fix documentation link in code
* Fix link from documentation to code
---
airflow/providers/google/cloud/operators/gcs.py | 2 +-
.../operators/cloud/gcs.rst | 2 ++
.../operators/transfer/gcs_to_gcs.rst | 16 +++++++++++-----
3 files changed, 14 insertions(+), 6 deletions(-)
diff --git a/airflow/providers/google/cloud/operators/gcs.py
b/airflow/providers/google/cloud/operators/gcs.py
index bb257f94b3..811f589253 100644
--- a/airflow/providers/google/cloud/operators/gcs.py
+++ b/airflow/providers/google/cloud/operators/gcs.py
@@ -919,7 +919,7 @@ class
GCSSynchronizeBucketsOperator(GoogleCloudBaseOperator):
.. seealso::
For more information on how to use this operator, take a look at the
guide:
- :ref:`howto/operator:GCSSynchronizeBuckets`
+ :ref:`howto/operator:GCSSynchronizeBucketsOperator`
:param source_bucket: The name of the bucket containing the source objects.
:param destination_bucket: The name of the bucket containing the
destination objects.
diff --git a/docs/apache-airflow-providers-google/operators/cloud/gcs.rst
b/docs/apache-airflow-providers-google/operators/cloud/gcs.rst
index 04c0a900ed..c4be59533a 100644
--- a/docs/apache-airflow-providers-google/operators/cloud/gcs.rst
+++ b/docs/apache-airflow-providers-google/operators/cloud/gcs.rst
@@ -24,6 +24,8 @@ Cloud Storage allows world-wide storage and retrieval of any
amount of data at a
You can use Cloud Storage for a range of scenarios including serving website
content,
storing data for archival and disaster recovery, or distributing large data
objects to users via direct download.
+See :doc:`/operators/transfer/index` for a list of specialized transfer
operators to and from Google Cloud Storage.
+
Prerequisite Tasks
^^^^^^^^^^^^^^^^^^
diff --git
a/docs/apache-airflow-providers-google/operators/transfer/gcs_to_gcs.rst
b/docs/apache-airflow-providers-google/operators/transfer/gcs_to_gcs.rst
index 602e924de1..0fe146c200 100644
--- a/docs/apache-airflow-providers-google/operators/transfer/gcs_to_gcs.rst
+++ b/docs/apache-airflow-providers-google/operators/transfer/gcs_to_gcs.rst
@@ -21,7 +21,7 @@ Transfer data in Google Cloud Storage
The `Google Cloud Storage <https://cloud.google.com/storage/>`__ (GCS) is
used to store large data from various applications.
Note that files are called objects in GCS terminology, so the use of the term
"object" and "file" in this guide is
interchangeable. There are several operators for whose purpose is to copy data
as part of the Google Cloud Service.
-This page shows how to use these operators.
+This page shows how to use these operators. See also
:doc:`/operators/cloud/gcs` for operators used to manage Google Cloud Storage
buckets.
Cloud Storage Transfer Service
@@ -49,6 +49,12 @@ modification date.
If you need information on how to use it, look at the guide:
:doc:`/operators/cloud/cloud_storage_transfer_service`
+Specialized transfer operators for Google Cloud Storage
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+See :doc:`/operators/transfer/index` for a list of specialized transfer
operators to and from Google Cloud Storage.
+
+
Local transfer
~~~~~~~~~~~~~~
@@ -189,12 +195,12 @@ the ``delimiter`` argument apply to moves as well as
copies.
:end-before: [END howto_operator_gcs_to_gcs_list_move]
-.. _howto/operator:GCSSynchronizeBuckets:
+.. _howto/operator:GCSSynchronizeBucketsOperator:
-GCSSynchronizeBuckets
-~~~~~~~~~~~~~~~~~~~~~
+GCSSynchronizeBucketsOperator
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-The
:class:`~airflow.providers.google.cloud.transfers.gcs_to_gcs.GCSSynchronizeBuckets`
+The
:class:`~airflow.providers.google.cloud.operators.gcs.GCSSynchronizeBucketsOperator`
operator checks the initial state of the destination bucket, and then compares
it with the source bucket.
Based on this, it creates an operation plan that describes which objects
should be deleted from
the destination bucket, which should be overwritten, and which should be
copied.