This is an automated email from the ASF dual-hosted git repository.
potiuk pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/main by this push:
new 739c8070cf1 Validate provider metadata list entries instead of
ignoring them (#71104)
739c8070cf1 is described below
commit 739c8070cf132d2f341b1481abc52be1a873b19f
Author: Wei Lee <[email protected]>
AuthorDate: Sun Aug 9 16:36:29 2026 +0800
Validate provider metadata list entries instead of ignoring them (#71104)
* Validate provider metadata list entries instead of ignoring them
The items blocks for queues, plugins and task-decorators listed their field
names directly under items, where they are not JSON Schema keywords, so the
blocks were inert and any shape passed. A provider author following what
those
blocks appeared to document — a dict for queues, or a path key for a
decorator
— got no error and was quietly dropped by the consumers instead. Pinning
each
section to the shape the provider.yaml files and generated payloads actually
use turns that into a failure at authoring time. The custom-provider howto
named the same nonexistent path field.
Raised in review of #70190.
* Keep runtime provider metadata validation permissive
The runtime schema is documented as deliberately looser than the development
one, and its validate() call is unwrapped in provider discovery, so a single
installed provider emitting an outdated entry shape would abort discovery
for
every provider in that process instead of costing just that entry a logged
warning. The structural descriptions stay behind so the schema no longer
documents a shape it never actually enforced.
* Reject unknown keys in provider.yaml decorator and plugin entries
Requiring the right keys does not stop a leftover one sitting beside them,
so a
stale `path:` next to a correct `class-name:` still validated — exactly the
confusion this branch set out to remove. Every sibling item block in this
authoring schema already closes itself off this way.
---
airflow-core/src/airflow/provider.yaml.schema.json | 49 ++++++++++++++--------
airflow-core/src/airflow/provider_info.schema.json | 42 +++++++++----------
.../howto/create-custom-providers.rst | 2 +-
3 files changed, 54 insertions(+), 39 deletions(-)
diff --git a/airflow-core/src/airflow/provider.yaml.schema.json
b/airflow-core/src/airflow/provider.yaml.schema.json
index 376eacdf93f..d1b0651c670 100644
--- a/airflow-core/src/airflow/provider.yaml.schema.json
+++ b/airflow-core/src/airflow/provider.yaml.schema.json
@@ -565,12 +565,22 @@
"type": "array",
"description": "Decorators to use with the TaskFlow API. Can be
accessed by users via '@task.<name>'",
"items": {
- "name": {
- "type": "string"
+ "type": "object",
+ "properties": {
+ "name": {
+ "description": "Name the decorator is exposed under,
following '@task.'",
+ "type": "string"
+ },
+ "class-name": {
+ "description": "Class name that implements the
decorator",
+ "type": "string"
+ }
},
- "path": {
- "type": "string"
- }
+ "additionalProperties": false,
+ "required": [
+ "name",
+ "class-name"
+ ]
}
},
"secrets-backends": {
@@ -688,24 +698,29 @@
"type": "array",
"description": "Plugins exposed by the provider",
"items": {
- "name": {
- "type": "string"
+ "type": "object",
+ "properties": {
+ "name": {
+ "description": "Name of the plugin",
+ "type": "string"
+ },
+ "plugin-class": {
+ "description": "Class name that implements the plugin",
+ "type": "string"
+ }
},
- "plugin-class": {
- "type": "string"
- }
+ "additionalProperties": false,
+ "required": [
+ "name",
+ "plugin-class"
+ ]
}
},
"queues": {
"type": "array",
- "description": "Message Queues exposed by the provider",
+ "description": "Message queue provider class names",
"items": {
- "name": {
- "type": "string"
- },
- "message-queue-class": {
- "type": "string"
- }
+ "type": "string"
}
},
"source-date-epoch": {
diff --git a/airflow-core/src/airflow/provider_info.schema.json
b/airflow-core/src/airflow/provider_info.schema.json
index b3e9bf74b62..4e6ddacebd1 100644
--- a/airflow-core/src/airflow/provider_info.schema.json
+++ b/airflow-core/src/airflow/provider_info.schema.json
@@ -432,11 +432,16 @@
"type": "array",
"description": "Apply custom decorators to the TaskFlow API. Can
be accessed by users via '@task.<name>'",
"items": {
- "name": {
- "type": "string"
- },
- "path": {
- "type": "string"
+ "type": "object",
+ "properties": {
+ "name": {
+ "description": "Name the decorator is exposed under,
following '@task.'",
+ "type": "string"
+ },
+ "class-name": {
+ "description": "Class name that implements the
decorator",
+ "type": "string"
+ }
}
}
},
@@ -444,27 +449,22 @@
"type": "array",
"description": "Plugins provided by the provider",
"items": {
- "name": {
- "type": "string",
- "description": "Name of the plugin"
- },
- "plugin-class": {
- "type": "string",
- "description": "Class to instantiate the plugin"
+ "type": "object",
+ "properties": {
+ "name": {
+ "type": "string",
+ "description": "Name of the plugin"
+ },
+ "plugin-class": {
+ "type": "string",
+ "description": "Class to instantiate the plugin"
+ }
}
}
},
"queues": {
"type": "array",
- "description": "Message Queues exposed by the provider",
- "items": {
- "name": {
- "type": "string"
- },
- "message-queue-class": {
- "type": "string"
- }
- }
+ "description": "Message queue provider class names"
}
},
"definitions": {
diff --git a/providers-summary-docs/howto/create-custom-providers.rst
b/providers-summary-docs/howto/create-custom-providers.rst
index 68d2da6acdd..ed6b00b9a66 100644
--- a/providers-summary-docs/howto/create-custom-providers.rst
+++ b/providers-summary-docs/howto/create-custom-providers.rst
@@ -114,7 +114,7 @@ Exposing customized functionality to the Airflow's core:
* ``sensors`` - this field should contain the list of all the sensor class
names that the
provider provides. See :doc:`apache-airflow:core-concepts/sensors` for
description of the sensors.
-* ``task-decorators`` - this field should contain the list of dictionaries of
name/path where the decorators
+* ``task-decorators`` - this field should contain the list of dictionaries of
name/class-name where the decorators
are available. See :doc:`apache-airflow:howto/create-custom-decorator` for
description of how to add
custom decorators.
