hankehly opened a new pull request, #30981: URL: https://github.com/apache/airflow/pull/30981
Related: #22303 This PR proposes a change to the [`PubSubHook.publish`](https://airflow.apache.org/docs/apache-airflow-providers-google/stable/_api/airflow/providers/google/cloud/hooks/pubsub/index.html#airflow.providers.google.cloud.hooks.pubsub.PubSubHook.publish) method. I updated the method signature to include support for string-type message data. Previously, the operator only allowed byte-type message data, which made it complicated for users to use templated message data. For example, the following would not work. ```py with DAG(...): @task def create_messages(): return json.dumps([{"data": "hello world"}]) publish_messages = PubSubPublishMessageOperator( task_id="publish_messages", project="test", topic="test", messages="{{ ti.xcom_pull('create_messages') | to_bytes }}" ) create_messages() >> publish_messages ``` Since bytes-type data is not JSON serializable, the only way to make the above example work is to set [`render_template_as_native_obj=True`](https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/operators.html#rendering-fields-as-native-python-objects) alongside a custom jinja2 filter to convert the xcom value to bytes. ```py def to_bytes(value): return value.encode("utf-8") with DAG(..., render_template_as_native_obj=True, user_defined_filters={"to_bytes": to_bytes}): @task def create_messages(): return json.dumps([{"data": "hello world"}]) publish_messages = PubSubPublishMessageOperator( task_id="publish_messages", project="test", topic="test", messages="{{ ti.xcom_pull('create_messages') | to_bytes }}" ) create_messages() >> publish_messages ``` With this change, users can now easily use templated message data in string format, making the operator more versatile and user-friendly. The changes are backward compatible, so existing operators can continue to use bytes-type message data. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
