The GitHub Actions job "Tests" on airflow.git/fix/elasticsearch-compat-with-v2 has failed. Run started by GitHub user petercheon (triggered by eladkal).
Head commit for run: c0ef998b717743d36cd1ad19a00ecc18cd440d49 / Peter Cheon <[email protected]> fix(elasticsearch): pin compatible-with at the transport layer to keep ES 8 servers working Since #64070 the provider depends on elasticsearch>=8.10,<10. A default install resolves to an elasticsearch>=9 Python client, which always negotiates 'compatible-with=9' on every request. Elasticsearch 8.x servers reject that with HTTP 400 media_type_header_exception, breaking remote task log ingestion and both ElasticsearchSQLHook and ElasticsearchPythonHook against ES 8 clusters. Add a [elasticsearch] es_compat_with config option that, when set to a major version string ('7'/'8'/'9'), wraps the client's transport perform_request so every outbound request carries 'Accept: application/vnd.elasticsearch+json; compatible-with=<major>' (and the matching '+x-ndjson' form for bulk so streaming bodies still parse). The wrap is applied at every Elasticsearch client construction site in the provider: - ElasticsearchTaskHandler (log/es_task_handler.py) - ElasticsearchRemoteLogIO (log/es_task_handler.py) - ESConnection (hooks/elasticsearch.py) - ElasticsearchPythonHook (hooks/elasticsearch.py) When the option is unset, behavior is unchanged. Tests assert against what the transport actually sends, not the in-memory state of the client object. Setting client._headers (which is what client.options(headers=...) does) is not enough because elasticsearch-py re-applies its own per-API-method content-negotiation headers right before the request is sent — only the transport layer sees the final headers. Closes: https://github.com/apache/airflow/issues/66063 Supersedes: https://github.com/apache/airflow/pull/66064 Report URL: https://github.com/apache/airflow/actions/runs/25088472994 With regards, GitHub Actions via GitBox --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
