mehdigati opened a new issue, #39940:
URL: https://github.com/apache/airflow/issues/39940

   ### Description
   
   Currently, the `PubSubPublishMessageOperator` in the Google Cloud Platform 
(GCP) integration for Apache Airflow does not allow specifying an ordering key 
for published messages. The ordering key is a feature provided by Cloud Pub/Sub 
that allows messages to be delivered in the order they were published within a 
single ordering key.
   
   
   ### Use case/motivation
   
   There are several use cases where maintaining the order of published 
messages is crucial, such as:
   1. Event processing pipelines: When processing a sequence of events, it's 
essential to maintain the order in which they occurred to ensure proper state 
management and avoid data corruption.
   2. Distributed transactions: In scenarios involving distributed transactions 
across multiple services, preserving the order of messages can be critical for 
maintaining data consistency.
   3. Deduplication: When implementing deduplication mechanisms, the ordering 
of messages can be a key factor in determining which messages should be 
considered duplicates.
   
   By adding support for ordering keys in the `PubSubPublishMessageOperator`, 
Airflow users would be able to leverage this Cloud Pub/Sub feature and ensure 
that their published messages are delivered in the desired order, enabling more 
robust and reliable data processing pipelines.
   
   
   ### Related issues
   
   None that I'm aware of.
   
   
   ### Are you willing to submit a PR?
   
   - [X] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of 
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Reply via email to