cliebBS opened a new issue, #35932:
URL: https://github.com/apache/airflow/issues/35932

   ### Apache Airflow version
   
   2.7.3
   
   ### What happened
   
   When using the `SqsSensor` with `message_filtering="jsonpath"` and both 
`message_filtering_config` and `message_filtering_match_values` set, the 
following error is produced:
   
   ```
   [2023-11-28, 18:24:44 UTC] {taskinstance.py:1937} ERROR - [findings_sync 
manual__2023-11-27T16:12:36+00:00 notify_started_test_receive 18] Task failed 
with exception
   Traceback (most recent call last):
     File 
"/home/airflow/.local/lib/python3.11/site-packages/airflow/providers/amazon/aws/sensors/sqs.py",
 line 144, in execute
       super().execute(context=context)
     File 
"/home/airflow/.local/lib/python3.11/site-packages/airflow/sensors/base.py", 
line 257, in execute
       raise e
     File 
"/home/airflow/.local/lib/python3.11/site-packages/airflow/sensors/base.py", 
line 239, in execute
       poke_return = self.poke(context)
                     ^^^^^^^^^^^^^^^^^^
     File 
"/home/airflow/.local/lib/python3.11/site-packages/airflow/providers/amazon/aws/sensors/sqs.py",
 line 187, in poke
       messages = process_response(
                  ^^^^^^^^^^^^^^^^^
     File 
"/home/airflow/.local/lib/python3.11/site-packages/airflow/providers/amazon/aws/utils/sqs.py",
 line 49, in process_response
       messages = filter_messages(
                  ^^^^^^^^^^^^^^^^
     File 
"/home/airflow/.local/lib/python3.11/site-packages/airflow/providers/amazon/aws/utils/sqs.py",
 line 63, in filter_messages
       return filter_messages_jsonpath(messages, 
message_filtering_match_values, message_filtering_config)
              
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
     File 
"/home/airflow/.local/lib/python3.11/site-packages/airflow/providers/amazon/aws/utils/sqs.py",
 line 87, in filter_messages_jsonpath
       or any(result.value in message_filtering_match_values for result in 
results)
          
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
     File 
"/home/airflow/.local/lib/python3.11/site-packages/airflow/providers/amazon/aws/utils/sqs.py",
 line 87, in <genexpr>
       or any(result.value in message_filtering_match_values for result in 
results)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   TypeError: unhashable type: 'dict'
   ```
   
   ### What you think should happen instead
   
   The messages should be filtered using the provided JsonPath query and the 
provided match values.
   
   ### How to reproduce
   
   Sorry, all I have available is a FIFO queue, and the Amazon providers for 
both SQS and SNS are unable to send messages to FIFO queues due to their lack 
of support for the `message_deduplication_id` field, which FIFO queues require 
to be set.
   
   ### Operating System
   
   MacOS 14.1.1 and Amazon Linux 2
   
   ### Versions of Apache Airflow Providers
   
   8.10.0
   
   ### Deployment
   
   Official Apache Airflow Helm Chart
   
   ### Deployment details
   
   _No response_
   
   ### Anything else
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [X] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of 
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to