This is an automated email from the ASF dual-hosted git repository.
vatsrahul1001 pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/main by this push:
new ad6ec79c73e Deprecate airflow.utils.email in favor of the SMTP
provider (#75414)
ad6ec79c73e is described below
commit ad6ec79c73ef7362a2b3f213cae904c893e2f299
Author: Shivam Rastogi <[email protected]>
AuthorDate: Mon Oct 26 21:26:12 2026 -0700
Deprecate airflow.utils.email in favor of the SMTP provider (#75414)
* Deprecate airflow.utils.email in favor of the SMTP provider
The email utilities now live in airflow.providers.smtp.utils.emailer.
airflow.utils.email keeps its public names as deprecated aliases of the
provider functions, so existing imports and email_backend settings keep
working with a DeprecatedImportWarning.
The default [email] email_backend value is unchanged. The Task SDK treats it
as "no custom backend" and sends alerts with SmtpNotifier, as before.
A regression test covers email sent from a server process, such as a
listener in the API server, with the SMTP login stored only in the metadata
database.
airflow-core now requires apache-airflow-providers-smtp>=3.2.0, the first
release that contains the emailer module.
* Keep star imports of airflow.utils.email working and document the move
"from airflow.utils.email import *" imported the email functions before
they moved to the SMTP provider. Listing them in __all__ keeps that import
working, with the same deprecation warning as named imports.
The email configuration guide now says where the functions live, that the
default email_backend value is unchanged, and that the moved functions do
not see settings changed at runtime with airflow.configuration.conf.set().
The coverage script entry for airflow/utils/email.py is left unchanged, so
this change stays within airflow-core.
* Shorten the core email page and point to the SMTP provider docs
The SMTP server and connection setup is documented by the SMTP provider, so
the core page now only explains email_backend and the [email] options shared by
all backends, and links to the provider docs.
* Explain what changes for callers of the deprecated email functions
Add a significant newsfragment, keep email_backend at its default in the
docs note, and name the Connection lookup and configuration source in the
deprecation warning.
* Test that a plain script does not see a database-only SMTP Connection
Outside task and server processes the Connection lookup only searches
environment variables and secrets backends, so the email is sent without a
login. The newsfragment describes this; the test now covers it.
---
airflow-core/docs/howto/email-config.rst | 80 +---
airflow-core/newsfragments/75414.significant.rst | 20 +
airflow-core/pyproject.toml | 2 +-
airflow-core/src/airflow/utils/email.py | 350 ++----------------
airflow-core/tests/unit/utils/test_email.py | 447 ++++-------------------
5 files changed, 133 insertions(+), 766 deletions(-)
diff --git a/airflow-core/docs/howto/email-config.rst
b/airflow-core/docs/howto/email-config.rst
index 4bf43d07d77..45d8c566341 100644
--- a/airflow-core/docs/howto/email-config.rst
+++ b/airflow-core/docs/howto/email-config.rst
@@ -18,80 +18,32 @@
Email Configuration
===================
-You can configure the email that is being sent in your ``airflow.cfg``
-by setting a ``subject_template`` and/or a ``html_content_template``
-in the ``[email]`` section.
+Airflow sends task emails through the function set in ``[email]
email_backend``. By default, email is sent
+over SMTP by the SMTP provider (``apache-airflow-providers-smtp``). To set up
the SMTP server and its
+credentials, see :doc:`apache-airflow-providers-smtp:email-backend` and
+:doc:`apache-airflow-providers-smtp:connections/smtp`.
+
+The ``[email]`` section also sets the connection, sender and message templates
used for any backend:
.. code-block:: ini
[email]
email_backend = airflow.utils.email.send_email_smtp
+ email_conn_id = smtp_default
+ from_email = "John Doe <[email protected]>"
subject_template = /path/to/my_subject_template_file
html_content_template = /path/to/my_html_content_template_file
-Equivalent environment variables look like:
-
-.. code-block:: sh
-
- AIRFLOW__EMAIL__EMAIL_BACKEND=airflow.utils.email.send_email_smtp
- AIRFLOW__EMAIL__SUBJECT_TEMPLATE=/path/to/my_subject_template_file
- AIRFLOW__EMAIL__HTML_CONTENT_TEMPLATE=/path/to/my_html_content_template_file
-
-You can configure a sender's email address by setting ``from_email`` in the
``[email]`` section like:
-
-.. code-block:: ini
-
- [email]
- from_email = "John Doe <[email protected]>"
-
-Equivalent environment variables look like:
-
-.. code-block:: sh
-
- AIRFLOW__EMAIL__FROM_EMAIL="John Doe <[email protected]>"
-
-By default, ``email_backend`` is set to
``airflow.utils.email.send_email_smtp``, which sends email over SMTP.
-
-To configure SMTP settings, checkout the :ref:`SMTP <config:smtp>` section in
the standard configuration.
-If you do not want to store the SMTP credentials in the config or in the
environment variables, you can create a
-connection called ``smtp_default`` of ``Email`` type, or choose a custom
connection name and set the ``email_conn_id`` with its name in
-the configuration & store SMTP username-password in it. Other SMTP settings
like host, port etc always gets picked up
-from the configuration only. The connection can be of any type (for example
'HTTP connection').
-
-.. image:: ../img/ui-dark/email_connection.png
- :align: center
- :alt: create email connection
-
-If you want to check which email backend is currently set, you can use
``airflow config get-value email email_backend`` command as in
-the example below.
-
-.. code-block:: bash
-
- $ airflow config get-value email email_backend
- airflow.utils.email.send_email_smtp
-
-To access the task's information you use `Jinja Templating
<http://jinja.pocoo.org/docs/dev/>`_ in your template files.
-
-For example a ``html_content_template`` file could look like this:
-
-.. code-block::
-
- Try {{try_number}} out of {{max_tries + 1}}<br>
- Exception:<br>{{exception_html}}<br>
- Log: <a href="{{ti.log_url}}">Link</a><br>
- Host: {{ti.hostname}}<br>
- Mark success: <a href="{{ti.mark_success_url}}">Link</a><br>
+The templates use `Jinja <https://jinja.palletsprojects.com/>`_ and can access
the task's information, for
+example ``{{ti.log_url}}``.
.. note::
- For more information on setting the configuration, see :doc:`set-config`
-
-Alternative email backends
----------------------------
-Instead of sending email over SMTP, you can point ``email_backend`` at an
implementation provided by a
-community provider to send email through a third-party service's API. To use
one, install the relevant
-provider distribution, set ``email_backend`` to the dotted path of its
``send_email`` function, and set
-``email_conn_id`` to a connection holding the credentials it needs.
+ The functions in ``airflow.utils.email`` are deprecated and now live in
+ ``airflow.providers.smtp.utils.emailer``. Only the Python imports moved:
keep ``email_backend`` set to
+ ``airflow.utils.email.send_email_smtp``. Airflow treats that value as
"send with ``SmtpNotifier``",
+ which reads all SMTP settings from the Connection. Any other value,
including the provider path, calls
+ that function instead, and the SMTP provider's function reads the server
settings from ``[smtp]``.
-The list of email backends provided by community-managed providers is
available in
+To send email through another service, set ``email_backend`` to a backend from
another provider; see
:doc:`apache-airflow-providers:core-extensions/email-backends`.
diff --git a/airflow-core/newsfragments/75414.significant.rst
b/airflow-core/newsfragments/75414.significant.rst
new file mode 100644
index 00000000000..8feb51b2682
--- /dev/null
+++ b/airflow-core/newsfragments/75414.significant.rst
@@ -0,0 +1,20 @@
+``airflow.utils.email`` is deprecated in favor of the SMTP provider
+
+The functions in ``airflow.utils.email`` (``send_email``, ``send_email_smtp``,
``send_mime_email``,
+``build_mime_message`` and ``get_email_address_list``) now live in
+``airflow.providers.smtp.utils.emailer``. The old names still work and raise a
``DeprecatedImportWarning``.
+Airflow now requires ``apache-airflow-providers-smtp>=3.2.0``.
+
+Keep ``[email] email_backend`` set to ``airflow.utils.email.send_email_smtp``.
Airflow treats that value as
+"send task emails with ``SmtpNotifier``", which reads all SMTP settings from
the Connection. Changing it to
+the provider path sends through the ``[smtp]`` settings instead.
+
+If you call these functions directly:
+
+* The SMTP Connection is looked up with ``BaseHook.get_connection``. A script
that runs outside Airflow's
+ processes needs the Connection in an environment variable or a secrets
backend, not only in the metadata
+ database.
+* Settings are read through the Task SDK configuration, so values changed at
runtime with
+ ``airflow.configuration.conf.set()`` are not seen. ``airflow.cfg`` and
environment variables work as before.
+* Tests that patch ``airflow.utils.email.send_mime_email`` no longer intercept
the call made by
+ ``send_email_smtp``. Patch
``airflow.providers.smtp.utils.emailer.send_mime_email`` instead.
diff --git a/airflow-core/pyproject.toml b/airflow-core/pyproject.toml
index 573be540ffa..a843480c11c 100644
--- a/airflow-core/pyproject.toml
+++ b/airflow-core/pyproject.toml
@@ -158,7 +158,7 @@ dependencies = [
"apache-airflow-providers-common-compat>=1.7.4",
"apache-airflow-providers-common-io>=1.6.3",
"apache-airflow-providers-common-sql>=1.28.1",
- "apache-airflow-providers-smtp>=2.3.1",
+ "apache-airflow-providers-smtp>=3.2.0",
"apache-airflow-providers-standard>=1.9.0",
# Start of shared logging dependencies
"msgspec>=0.19.0",
diff --git a/airflow-core/src/airflow/utils/email.py
b/airflow-core/src/airflow/utils/email.py
index 46f24fba0f5..89bb754862a 100644
--- a/airflow-core/src/airflow/utils/email.py
+++ b/airflow-core/src/airflow/utils/email.py
@@ -15,327 +15,41 @@
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
-from __future__ import annotations
-
-import collections.abc
-import logging
-import os
-import re
-import smtplib
-import ssl
-from collections.abc import Iterable
-from email.mime.application import MIMEApplication
-from email.mime.multipart import MIMEMultipart
-from email.mime.text import MIMEText
-from email.utils import formatdate
-from typing import Any
-
-from airflow.configuration import conf
-from airflow.exceptions import AirflowException
-
-log = logging.getLogger(__name__)
-
-
-def send_email(
- to: list[str] | Iterable[str],
- subject: str,
- html_content: str,
- files: list[str] | None = None,
- dryrun: bool = False,
- cc: str | Iterable[str] | None = None,
- bcc: str | Iterable[str] | None = None,
- mime_subtype: str = "mixed",
- mime_charset: str = "utf-8",
- conn_id: str | None = None,
- custom_headers: dict[str, Any] | None = None,
- **kwargs,
-) -> None:
- """
- Send an email using the backend specified in the *EMAIL_BACKEND*
configuration option.
-
- :param to: A list or iterable of email addresses to send the email to.
- :param subject: The subject of the email.
- :param html_content: The content of the email in HTML format.
- :param files: A list of paths to files to attach to the email.
- :param dryrun: If *True*, the email will not actually be sent. Default:
*False*.
- :param cc: A string or iterable of strings containing email addresses to
send a copy of the email to.
- :param bcc: A string or iterable of strings containing email addresses to
send a
- blind carbon copy of the email to.
- :param mime_subtype: The subtype of the MIME message. Default: "mixed".
- :param mime_charset: The charset of the email. Default: "utf-8".
- :param conn_id: The connection ID to use for the backend. If not provided,
the default connection
- specified in the *EMAIL_CONN_ID* configuration option will be used.
- :param custom_headers: A dictionary of additional headers to add to the
MIME message.
- No validations are run on these values, and they should be able to be
encoded.
- :param kwargs: Additional keyword arguments to pass to the backend.
- """
- backend = conf.getimport("email", "EMAIL_BACKEND")
- backend_conn_id = conn_id or conf.get("email", "EMAIL_CONN_ID")
- from_email = conf.get("email", "from_email", fallback=None)
-
- to_list = get_email_address_list(to)
- to_comma_separated = ", ".join(to_list)
-
- return backend(
- to_comma_separated,
- subject,
- html_content,
- files=files,
- dryrun=dryrun,
- cc=cc,
- bcc=bcc,
- mime_subtype=mime_subtype,
- mime_charset=mime_charset,
- conn_id=backend_conn_id,
- from_email=from_email,
- custom_headers=custom_headers,
- **kwargs,
- )
-
-
-def send_email_smtp(
- to: str | Iterable[str],
- subject: str,
- html_content: str,
- files: list[str] | None = None,
- dryrun: bool = False,
- cc: str | Iterable[str] | None = None,
- bcc: str | Iterable[str] | None = None,
- mime_subtype: str = "mixed",
- mime_charset: str = "utf-8",
- conn_id: str = "smtp_default",
- from_email: str | None = None,
- custom_headers: dict[str, Any] | None = None,
- **kwargs,
-) -> None:
- """
- Send an email with html content.
-
- :param to: Recipient email address or list of addresses.
- :param subject: Email subject.
- :param html_content: Email body in HTML format.
- :param files: List of file paths to attach to the email.
- :param dryrun: If True, the email will not be sent, but all other actions
will be performed.
- :param cc: Carbon copy recipient email address or list of addresses.
- :param bcc: Blind carbon copy recipient email address or list of addresses.
- :param mime_subtype: MIME subtype of the email.
- :param mime_charset: MIME charset of the email.
- :param conn_id: Connection ID of the SMTP server.
- :param from_email: Sender email address.
- :param custom_headers: Dictionary of custom headers to include in the
email.
- :param kwargs: Additional keyword arguments.
-
- >>> send_email("[email protected]", "foo", "<b>Foo</b> bar", ["/dev/null"],
dryrun=True)
- """
- smtp_mail_from = conf.get("smtp", "SMTP_MAIL_FROM")
-
- if smtp_mail_from is not None:
- mail_from = smtp_mail_from
- else:
- if from_email is None:
- raise ValueError(
- "You should set from email - either by smtp/smtp_mail_from
config or `from_email` parameter"
- )
- mail_from = from_email
-
- msg, recipients = build_mime_message(
- mail_from=mail_from,
- to=to,
- subject=subject,
- html_content=html_content,
- files=files,
- cc=cc,
- bcc=bcc,
- mime_subtype=mime_subtype,
- mime_charset=mime_charset,
- custom_headers=custom_headers,
- )
-
- send_mime_email(e_from=mail_from, e_to=recipients, mime_msg=msg,
conn_id=conn_id, dryrun=dryrun)
-
-
-def build_mime_message(
- mail_from: str | None,
- to: str | Iterable[str],
- subject: str,
- html_content: str,
- files: list[str] | None = None,
- cc: str | Iterable[str] | None = None,
- bcc: str | Iterable[str] | None = None,
- mime_subtype: str = "mixed",
- mime_charset: str = "utf-8",
- custom_headers: dict[str, Any] | None = None,
-) -> tuple[MIMEMultipart, list[str]]:
- """
- Build a MIME message that can be used to send an email and returns a full
list of recipients.
-
- :param mail_from: Email address to set as the email's "From" field.
- :param to: A string or iterable of strings containing email addresses to
set as the email's "To" field.
- :param subject: The subject of the email.
- :param html_content: The content of the email in HTML format.
- :param files: A list of paths to files to be attached to the email.
- :param cc: A string or iterable of strings containing email addresses to
set as the email's "CC" field.
- :param bcc: A string or iterable of strings containing email addresses to
set as the email's "BCC" field.
- :param mime_subtype: The subtype of the MIME message. Default: "mixed".
- :param mime_charset: The charset of the email. Default: "utf-8".
- :param custom_headers: Additional headers to add to the MIME message. No
validations are run on these
- values, and they should be able to be encoded.
- :return: A tuple containing the email as a MIMEMultipart object and a list
of recipient email addresses.
- """
- to = get_email_address_list(to)
+"""Deprecated: the email utilities moved to
:mod:`airflow.providers.smtp.utils.emailer`."""
- msg = MIMEMultipart(mime_subtype)
- msg["Subject"] = subject
- if mail_from:
- msg["From"] = mail_from
- msg["To"] = ", ".join(to)
- recipients = to
- if cc:
- cc = get_email_address_list(cc)
- msg["CC"] = ", ".join(cc)
- recipients += cc
-
- if bcc:
- # don't add bcc in header
- bcc = get_email_address_list(bcc)
- recipients += bcc
-
- msg["Date"] = formatdate(localtime=True)
- mime_text = MIMEText(html_content, "html", mime_charset)
- msg.attach(mime_text)
-
- for fname in files or []:
- basename = os.path.basename(fname)
- with open(fname, "rb") as file:
- part = MIMEApplication(file.read(), Name=basename)
- part.add_header("Content-Disposition", "attachment",
filename=basename)
- part["Content-ID"] = f"<{basename}>"
- msg.attach(part)
-
- if custom_headers:
- for header_key, header_value in custom_headers.items():
- msg[header_key] = header_value
-
- return msg, recipients
-
-
-def send_mime_email(
- e_from: str,
- e_to: str | list[str],
- mime_msg: MIMEMultipart,
- conn_id: str = "smtp_default",
- dryrun: bool = False,
-) -> None:
- """
- Send a MIME email.
-
- :param e_from: The email address of the sender.
- :param e_to: The email address or a list of email addresses of the
recipient(s).
- :param mime_msg: The MIME message to send.
- :param conn_id: The ID of the SMTP connection to use.
- :param dryrun: If True, the email will not be sent, but a log message will
be generated.
- """
- smtp_host = conf.get_mandatory_value("smtp", "SMTP_HOST")
- smtp_port = conf.getint("smtp", "SMTP_PORT")
- smtp_starttls = conf.getboolean("smtp", "SMTP_STARTTLS")
- smtp_ssl = conf.getboolean("smtp", "SMTP_SSL")
- smtp_retry_limit = conf.getint("smtp", "SMTP_RETRY_LIMIT")
- smtp_timeout = conf.getint("smtp", "SMTP_TIMEOUT")
- smtp_user = None
- smtp_password = None
-
- if conn_id is not None:
- try:
- from airflow.models import Connection
-
- airflow_conn = Connection.get_connection_from_secrets(conn_id)
- smtp_user = airflow_conn.login
- smtp_password = airflow_conn.password
- except AirflowException:
- pass
- if smtp_user is None or smtp_password is None:
- log.debug("No user/password found for SMTP, so logging in with no
authentication.")
-
- if not dryrun:
- for attempt in range(smtp_retry_limit + 1):
- log.info("Email alerting: attempt %s", str(attempt + 1))
- try:
- smtp_conn = _get_smtp_connection(smtp_host, smtp_port,
smtp_timeout, smtp_ssl)
- except smtplib.SMTPServerDisconnected:
- if attempt == smtp_retry_limit:
- raise
- else:
- if smtp_starttls:
- smtp_conn.starttls(context=_get_ssl_context())
- if smtp_user and smtp_password:
- smtp_conn.login(smtp_user, smtp_password)
- log.info("Sent an alert email to %s", e_to)
- smtp_conn.sendmail(e_from, e_to, mime_msg.as_string())
- smtp_conn.quit()
- break
-
-
-def get_email_address_list(addresses: str | Iterable[str]) -> list[str]:
- """
- Return a list of email addresses from the provided input.
-
- :param addresses: A string or iterable of strings containing email
addresses.
- :return: A list of email addresses.
- :raises TypeError: If the input is not a string or iterable of strings.
- """
- if isinstance(addresses, str):
- return _get_email_list_from_str(addresses)
- if isinstance(addresses, collections.abc.Iterable):
- if not all(isinstance(item, str) for item in addresses):
- raise TypeError("The items in your iterable must be strings.")
- return list(addresses)
- raise TypeError(f"Unexpected argument type: Received
'{type(addresses).__name__}'.")
+from __future__ import annotations
+from typing import TYPE_CHECKING
-def _get_ssl_context() -> ssl.SSLContext | None:
- """
- Return the SSL context configured via the ``email.ssl_context`` option.
+from airflow.utils.deprecation_tools import add_deprecated_classes
- ``"default"`` produces :func:`ssl.create_default_context`; ``"none"``
- returns ``None`` so callers that explicitly want to skip certificate
- validation (for example, against a self-signed SMTP server in a
- lab environment) can still do so.
- """
- ssl_context_string = conf.get("email", "SSL_CONTEXT")
- if ssl_context_string == "default":
- return ssl.create_default_context()
- if ssl_context_string == "none":
- return None
- raise RuntimeError(
- f"The email.ssl_context configuration variable must "
- f"be set to 'default' or 'none' and is '{ssl_context_string}."
+if TYPE_CHECKING:
+ # Lets type checkers resolve the deprecated names, which only exist at
runtime via __getattr__.
+ from airflow.providers.smtp.utils.emailer import (
+ build_mime_message as build_mime_message,
+ get_email_address_list as get_email_address_list,
+ send_email as send_email,
+ send_email_smtp as send_email_smtp,
+ send_mime_email as send_mime_email,
)
-
-def _get_smtp_connection(host: str, port: int, timeout: int, with_ssl: bool)
-> smtplib.SMTP:
- """
- Return an SMTP connection to the specified host and port, with optional
SSL encryption.
-
- :param host: The hostname or IP address of the SMTP server.
- :param port: The port number to connect to on the SMTP server.
- :param timeout: The timeout in seconds for the connection.
- :param with_ssl: Whether to use SSL encryption for the connection.
- :return: An SMTP connection to the specified host and port.
- """
- if not with_ssl:
- return smtplib.SMTP(host=host, port=port, timeout=timeout)
- return smtplib.SMTP_SSL(host=host, port=port, timeout=timeout,
context=_get_ssl_context())
-
-
-def _get_email_list_from_str(addresses: str) -> list[str]:
- """
- Extract a list of email addresses from a string.
-
- The string can contain multiple email addresses separated
- by any of the following delimiters: ',' or ';'.
-
- :param addresses: A string containing one or more email addresses.
- :return: A list of email addresses.
- """
- pattern = r"\s*[,;]\s*"
- return re.split(pattern, addresses)
+_MOVED_TO_PROVIDER = {
+ "send_email": "airflow.providers.smtp.utils.emailer.send_email",
+ "send_email_smtp": "airflow.providers.smtp.utils.emailer.send_email_smtp",
+ "build_mime_message":
"airflow.providers.smtp.utils.emailer.build_mime_message",
+ "send_mime_email": "airflow.providers.smtp.utils.emailer.send_mime_email",
+ "get_email_address_list":
"airflow.providers.smtp.utils.emailer.get_email_address_list",
+}
+
+# Keeps "from airflow.utils.email import *" importing the deprecated names.
+__all__ = list(_MOVED_TO_PROVIDER)
+
+add_deprecated_classes(
+ {__name__: _MOVED_TO_PROVIDER},
+ package=__name__,
+ extra_message=(
+ "The provider looks up the SMTP Connection with
BaseHook.get_connection and reads settings through "
+ "the Task SDK configuration, so values changed at runtime with
airflow.configuration.conf.set() are "
+ "not seen"
+ ),
+)
diff --git a/airflow-core/tests/unit/utils/test_email.py
b/airflow-core/tests/unit/utils/test_email.py
index 33f242fc72f..a772fe27e06 100644
--- a/airflow-core/tests/unit/utils/test_email.py
+++ b/airflow-core/tests/unit/utils/test_email.py
@@ -17,404 +17,85 @@
# under the License.
from __future__ import annotations
-import json
-import os
-from email.mime.application import MIMEApplication
from email.mime.multipart import MIMEMultipart
-from email.mime.text import MIMEText
-from smtplib import SMTPServerDisconnected
from unittest import mock
import pytest
-from airflow.configuration import conf
+from airflow.providers.smtp.utils import emailer
from airflow.utils import email
+from airflow.utils.deprecation_tools import DeprecatedImportWarning
-from tests_common.test_utils.config import conf_vars
+DEPRECATED_NAMES = [
+ "send_email",
+ "send_email_smtp",
+ "build_mime_message",
+ "send_mime_email",
+ "get_email_address_list",
+]
-EMAILS = ["[email protected]", "[email protected]"]
-send_email_test = mock.MagicMock()
[email protected]("name", DEPRECATED_NAMES)
+def test_deprecated_names_resolve_to_provider(name):
+ with pytest.warns(
+ DeprecatedImportWarning,
match=f"airflow.providers.smtp.utils.emailer.{name}"
+ ) as record:
+ assert getattr(email, name) is getattr(emailer, name)
+ assert "BaseHook.get_connection" in str(record[0].message)
-class TestEmail:
- def setup_method(self):
- conf.remove_option("email", "EMAIL_BACKEND")
- conf.remove_option("email", "EMAIL_CONN_ID")
- def test_get_email_address_single_email(self):
- emails_string = "[email protected]"
+def test_star_import_exposes_deprecated_names():
+ namespace: dict = {}
+ with pytest.warns(DeprecatedImportWarning):
+ exec("from airflow.utils.email import *", namespace)
- assert email.get_email_address_list(emails_string) == [emails_string]
-
- def test_get_email_address_comma_sep_string(self):
- emails_string = "[email protected], [email protected]"
-
- assert email.get_email_address_list(emails_string) == EMAILS
-
- def test_get_email_address_colon_sep_string(self):
- emails_string = "[email protected]; [email protected]"
-
- assert email.get_email_address_list(emails_string) == EMAILS
-
- def test_get_email_address_list(self):
- emails_list = ["[email protected]", "[email protected]"]
-
- assert email.get_email_address_list(emails_list) == EMAILS
-
- def test_get_email_address_tuple(self):
- emails_tuple = ("[email protected]", "[email protected]")
-
- assert email.get_email_address_list(emails_tuple) == EMAILS
-
- def test_get_email_address_invalid_type(self):
- emails_string = 1
-
- with pytest.raises(TypeError):
- email.get_email_address_list(emails_string)
-
- def test_get_email_address_invalid_type_in_iterable(self):
- emails_list = ["[email protected]", 2]
-
- with pytest.raises(TypeError):
- email.get_email_address_list(emails_list)
-
- @mock.patch("airflow.utils.email.send_email")
- def test_default_backend(self, mock_send_email):
- res = email.send_email("to", "subject", "content")
- mock_send_email.assert_called_once_with("to", "subject", "content")
- assert mock_send_email.return_value == res
-
- @mock.patch("airflow.utils.email.send_email_smtp")
- def test_custom_backend(self, mock_send_email):
- with conf_vars(
- {
- ("email", "email_backend"):
"unit.utils.test_email.send_email_test",
- ("email", "email_conn_id"): "smtp_default",
- }
- ):
- email.send_email("to", "subject", "content")
- send_email_test.assert_called_once_with(
- "to",
- "subject",
- "content",
- files=None,
- dryrun=False,
- cc=None,
- bcc=None,
- mime_charset="utf-8",
- mime_subtype="mixed",
- conn_id="smtp_default",
- from_email=None,
- custom_headers=None,
- )
- assert not mock_send_email.called
-
- @mock.patch("airflow.utils.email.send_email_smtp")
- @conf_vars(
- {
- ("email", "email_backend"):
"unit.utils.test_email.send_email_test",
- ("email", "email_conn_id"): "smtp_default",
- ("email", "from_email"): "[email protected]",
- }
- )
- def test_custom_backend_sender(self, mock_send_email_smtp):
- email.send_email("to", "subject", "content")
- _, call_kwargs = send_email_test.call_args
- assert call_kwargs["from_email"] == "[email protected]"
- assert not mock_send_email_smtp.called
-
- def test_build_mime_message(self):
- mail_from = "[email protected]"
- mail_to = "[email protected]"
- subject = "test subject"
- html_content = "<html>Test</html>"
- custom_headers = {"Reply-To": "[email protected]"}
-
- msg, recipients = email.build_mime_message(
- mail_from=mail_from,
- to=mail_to,
- subject=subject,
- html_content=html_content,
- custom_headers=custom_headers,
- )
-
- assert "From" in msg
- assert "To" in msg
- assert "Subject" in msg
- assert "Reply-To" in msg
- assert [mail_to] == recipients
- assert msg["To"] == ",".join(recipients)
-
- def test_build_mime_message_escapes_attachment_filename(self, tmp_path):
- # A quote in the filename must not break out of the quoted
- # Content-Disposition value and inject extra parameters.
- malicious = 'report.txt"; x-evil="1'
- attachment = tmp_path / malicious
- attachment.write_bytes(b"data")
-
- msg, _ = email.build_mime_message(
- mail_from="[email protected]",
- to="[email protected]",
- subject="subject",
- html_content="<html></html>",
- files=[os.fspath(attachment)],
- )
-
- part = msg.get_payload()[-1]
- assert part.get_filename() == malicious
- assert "x-evil" not in
dict(part.get_params(header="Content-Disposition"))
+ assert {name: namespace.get(name) for name in DEPRECATED_NAMES} == {
+ name: getattr(emailer, name) for name in DEPRECATED_NAMES
+ }
@pytest.mark.db_test
-class TestEmailSmtp:
- @pytest.fixture(autouse=True)
- def setup_test_cases(self, monkeypatch):
- monkeypatch.setenv( # Set the default smtp connection for all test
cases
- "AIRFLOW_CONN_SMTP_DEFAULT",
- json.dumps({"conn_type": "smtp", "login": "user", "password":
"p@$$word"}),
- )
-
- @mock.patch("airflow.utils.email.send_mime_email")
- def test_send_smtp(self, mock_send_mime, tmp_path):
- path = tmp_path / "testfile"
- path.write_text("attachment")
- email.send_email_smtp("to", "subject", "content",
files=[os.fspath(path)])
- assert mock_send_mime.called
- _, call_args = mock_send_mime.call_args
- assert conf.get("smtp", "SMTP_MAIL_FROM") == call_args["e_from"]
- assert call_args["e_to"] == ["to"]
- msg = call_args["mime_msg"]
- assert msg["Subject"] == "subject"
- assert conf.get("smtp", "SMTP_MAIL_FROM") == msg["From"]
- assert len(msg.get_payload()) == 2
- filename = f'attachment; filename="{path.name}"'
- assert filename == msg.get_payload()[-1].get("Content-Disposition")
- mimeapp = MIMEApplication("attachment")
- assert mimeapp.get_payload() == msg.get_payload()[-1].get_payload()
-
- @mock.patch("airflow.utils.email.send_mime_email")
- def test_send_smtp_with_multibyte_content(self, mock_send_mime):
- email.send_email_smtp("to", "subject", "🔥", mime_charset="utf-8")
- assert mock_send_mime.called
- _, call_args = mock_send_mime.call_args
- msg = call_args["mime_msg"]
- mimetext = MIMEText("🔥", "mixed", "utf-8")
- assert mimetext.get_payload() == msg.get_payload()[0].get_payload()
-
- @mock.patch("airflow.utils.email.send_mime_email")
- def test_send_bcc_smtp(self, mock_send_mime, tmp_path):
- path = tmp_path / "testfile"
- path.write_text("attachment")
- email.send_email_smtp(
- "to",
- "subject",
- "content",
- files=[os.fspath(path)],
- cc="cc",
- bcc="bcc",
- custom_headers={"Reply-To": "[email protected]"},
- )
- assert mock_send_mime.called
- _, call_args = mock_send_mime.call_args
- assert conf.get("smtp", "SMTP_MAIL_FROM") == call_args["e_from"]
- assert call_args["e_to"] == ["to", "cc", "bcc"]
- msg = call_args["mime_msg"]
- assert msg["Subject"] == "subject"
- assert conf.get("smtp", "SMTP_MAIL_FROM") == msg["From"]
- assert len(msg.get_payload()) == 2
- assert f'attachment; filename="{path.name}"' ==
msg.get_payload()[-1].get("Content-Disposition")
- mimeapp = MIMEApplication("attachment")
- assert mimeapp.get_payload() == msg.get_payload()[-1].get_payload()
- assert msg["Reply-To"] == "[email protected]"
-
- @mock.patch("smtplib.SMTP_SSL")
- @mock.patch("smtplib.SMTP")
- def test_send_mime_airflow_config(self, mock_smtp, mock_smtp_ssl,
monkeypatch):
- monkeypatch.delenv("AIRFLOW_CONN_SMTP_DEFAULT", raising=False)
- mock_smtp.return_value = mock.Mock()
- msg = MIMEMultipart()
- email.send_mime_email("from", "to", msg, dryrun=False)
- mock_smtp.assert_called_once_with(
- host=conf.get("smtp", "SMTP_HOST"),
- port=conf.getint("smtp", "SMTP_PORT"),
- timeout=conf.getint("smtp", "SMTP_TIMEOUT"),
- )
- assert not mock_smtp_ssl.called
- assert mock_smtp.return_value.starttls.called
- mock_smtp.return_value.sendmail.assert_called_once_with("from", "to",
msg.as_string())
- assert mock_smtp.return_value.quit.called
-
- @mock.patch("smtplib.SMTP")
- def test_send_mime_conn_id(self, mock_smtp, monkeypatch):
- monkeypatch.setenv(
- "AIRFLOW_CONN_SMTP_TEST_CONN",
- json.dumps({"conn_type": "smtp", "login": "test-user", "password":
"test-p@$$word"}),
- )
- msg = MIMEMultipart()
- email.send_mime_email("from", "to", msg, dryrun=False,
conn_id="smtp_test_conn")
- mock_smtp.return_value.login.assert_called_once_with("test-user",
"test-p@$$word")
- mock_smtp.return_value.sendmail.assert_called_once_with("from", "to",
msg.as_string())
- assert mock_smtp.return_value.quit.called
+class TestDeprecatedSendWithDbOnlyConnection:
+ """Which processes find an SMTP Connection that exists only in the
metadata database."""
- @mock.patch("smtplib.SMTP_SSL")
- @mock.patch("smtplib.SMTP")
- def test_send_mime_ssl_none_context(self, mock_smtp, mock_smtp_ssl):
- mock_smtp_ssl.return_value = mock.Mock()
- with conf_vars({("smtp", "smtp_ssl"): "True", ("email",
"ssl_context"): "none"}):
- email.send_mime_email("from", "to", MIMEMultipart(), dryrun=False)
- assert not mock_smtp.called
- mock_smtp_ssl.assert_called_once_with(
- host=conf.get("smtp", "SMTP_HOST"),
- port=conf.getint("smtp", "SMTP_PORT"),
- timeout=conf.getint("smtp", "SMTP_TIMEOUT"),
- context=None,
- )
-
- @mock.patch("smtplib.SMTP_SSL")
- @mock.patch("smtplib.SMTP")
- @mock.patch("ssl.create_default_context")
- def test_send_mime_ssl_default_context_if_not_set(self,
create_default_context, mock_smtp, mock_smtp_ssl):
- mock_smtp_ssl.return_value = mock.Mock()
- with conf_vars({("smtp", "smtp_ssl"): "True"}):
- email.send_mime_email("from", "to", MIMEMultipart(), dryrun=False)
- assert not mock_smtp.called
- assert create_default_context.called
- mock_smtp_ssl.assert_called_once_with(
- host=conf.get("smtp", "SMTP_HOST"),
- port=conf.getint("smtp", "SMTP_PORT"),
- timeout=conf.getint("smtp", "SMTP_TIMEOUT"),
- context=create_default_context.return_value,
- )
-
- @mock.patch("smtplib.SMTP_SSL")
- @mock.patch("smtplib.SMTP")
- @mock.patch("ssl.create_default_context")
- def test_send_mime_ssl_default_context_with_value_set_to_default(
- self, create_default_context, mock_smtp, mock_smtp_ssl
+ @pytest.fixture(autouse=True)
+ def _db_only_connection(self, monkeypatch):
+ from airflow.models import Connection
+ from airflow.utils.session import create_session
+
+ from tests_common.test_utils.db import clear_db_connections
+
+ monkeypatch.delenv("AIRFLOW_CONN_SMTP_DB_ONLY", raising=False)
+ monkeypatch.delenv("_AIRFLOW_PROCESS_CONTEXT", raising=False)
+ clear_db_connections(add_default_connections_back=False)
+ with create_session() as session:
+ session.add(
+ Connection(conn_id="smtp_db_only", conn_type="smtp",
login="db-user", password="db-pass")
+ )
+ yield
+ clear_db_connections()
+
+ @pytest.mark.parametrize(
+ ("process_context", "expected_login"),
+ [
+ pytest.param("server", ("db-user", "db-pass"),
id="server-process-finds-db-connection"),
+ pytest.param(None, None, id="plain-script-sends-without-login"),
+ ],
+ )
+ @mock.patch("smtplib.SMTP", autospec=True)
+ def test_send_mime_email_db_only_connection(
+ self, mock_smtp, monkeypatch, process_context, expected_login
):
- mock_smtp_ssl.return_value = mock.Mock()
- with conf_vars({("smtp", "smtp_ssl"): "True", ("email",
"ssl_context"): "default"}):
- email.send_mime_email("from", "to", MIMEMultipart(), dryrun=False)
- assert not mock_smtp.called
- assert create_default_context.called
- mock_smtp_ssl.assert_called_once_with(
- host=conf.get("smtp", "SMTP_HOST"),
- port=conf.getint("smtp", "SMTP_PORT"),
- timeout=conf.getint("smtp", "SMTP_TIMEOUT"),
- context=create_default_context.return_value,
- )
-
- @mock.patch("smtplib.SMTP_SSL")
- @mock.patch("smtplib.SMTP")
- def test_send_mime_noauth(self, mock_smtp, mock_smtp_ssl):
- mock_smtp.return_value = mock.Mock()
- with conf_vars(
- {
- ("smtp", "smtp_user"): None,
- ("smtp", "smtp_password"): None,
- }
- ):
- email.send_mime_email("from", "to", MIMEMultipart(), dryrun=False)
- assert not mock_smtp_ssl.called
- mock_smtp.assert_called_once_with(
- host=conf.get("smtp", "SMTP_HOST"),
- port=conf.getint("smtp", "SMTP_PORT"),
- timeout=conf.getint("smtp", "SMTP_TIMEOUT"),
- )
- assert not mock_smtp.login.called
-
- @mock.patch("smtplib.SMTP_SSL")
- @mock.patch("smtplib.SMTP")
- def test_send_mime_dryrun(self, mock_smtp, mock_smtp_ssl):
- email.send_mime_email("from", "to", MIMEMultipart(), dryrun=True)
- assert not mock_smtp.called
- assert not mock_smtp_ssl.called
-
- @mock.patch("smtplib.SMTP_SSL")
- @mock.patch("smtplib.SMTP")
- def test_send_mime_complete_failure(self, mock_smtp: mock.Mock,
mock_smtp_ssl: mock.Mock):
- mock_smtp.side_effect = SMTPServerDisconnected()
- msg = MIMEMultipart()
- with pytest.raises(SMTPServerDisconnected):
- email.send_mime_email("from", "to", msg, dryrun=False)
- mock_smtp.assert_any_call(
- host=conf.get("smtp", "SMTP_HOST"),
- port=conf.getint("smtp", "SMTP_PORT"),
- timeout=conf.getint("smtp", "SMTP_TIMEOUT"),
- )
- assert mock_smtp.call_count == conf.getint("smtp", "SMTP_RETRY_LIMIT")
+ 1
- assert not mock_smtp_ssl.called
- assert not mock_smtp.return_value.starttls.called
- assert not mock_smtp.return_value.login.called
- assert not mock_smtp.return_value.sendmail.called
- assert not mock_smtp.return_value.quit.called
-
- @mock.patch("smtplib.SMTP_SSL")
- @mock.patch("smtplib.SMTP")
- @mock.patch("ssl.create_default_context")
- def test_send_mime_ssl_complete_failure(self, create_default_context,
mock_smtp, mock_smtp_ssl):
- mock_smtp_ssl.side_effect = SMTPServerDisconnected()
- msg = MIMEMultipart()
- with conf_vars({("smtp", "smtp_ssl"): "True"}):
- with pytest.raises(SMTPServerDisconnected):
- email.send_mime_email("from", "to", msg, dryrun=False)
-
- mock_smtp_ssl.assert_any_call(
- host=conf.get("smtp", "SMTP_HOST"),
- port=conf.getint("smtp", "SMTP_PORT"),
- timeout=conf.getint("smtp", "SMTP_TIMEOUT"),
- context=create_default_context.return_value,
- )
- assert create_default_context.called
- assert mock_smtp_ssl.call_count == conf.getint("smtp",
"SMTP_RETRY_LIMIT") + 1
- assert not mock_smtp.called
- assert not mock_smtp_ssl.return_value.starttls.called
- assert not mock_smtp_ssl.return_value.login.called
- assert not mock_smtp_ssl.return_value.sendmail.called
- assert not mock_smtp_ssl.return_value.quit.called
-
- @mock.patch("smtplib.SMTP_SSL")
- @mock.patch("smtplib.SMTP")
- def test_send_mime_custom_timeout_retrylimit(self, mock_smtp,
mock_smtp_ssl):
- mock_smtp.side_effect = SMTPServerDisconnected()
- msg = MIMEMultipart()
-
- custom_retry_limit = 10
- custom_timeout = 60
-
- with conf_vars(
- {
- ("smtp", "smtp_retry_limit"): str(custom_retry_limit),
- ("smtp", "smtp_timeout"): str(custom_timeout),
- }
- ):
- with pytest.raises(SMTPServerDisconnected):
- email.send_mime_email("from", "to", msg, dryrun=False)
-
- mock_smtp.assert_any_call(
- host=conf.get("smtp", "SMTP_HOST"), port=conf.getint("smtp",
"SMTP_PORT"), timeout=custom_timeout
- )
- assert not mock_smtp_ssl.called
- assert mock_smtp.call_count == custom_retry_limit + 1
-
- @mock.patch("smtplib.SMTP_SSL")
- @mock.patch("smtplib.SMTP")
- def test_send_mime_partial_failure(self, mock_smtp, mock_smtp_ssl):
- final_mock = mock.Mock()
- side_effects = [SMTPServerDisconnected(), SMTPServerDisconnected(),
final_mock]
- mock_smtp.side_effect = side_effects
- msg = MIMEMultipart()
-
- email.send_mime_email("from", "to", msg, dryrun=False)
-
- mock_smtp.assert_any_call(
- host=conf.get("smtp", "SMTP_HOST"),
- port=conf.getint("smtp", "SMTP_PORT"),
- timeout=conf.getint("smtp", "SMTP_TIMEOUT"),
- )
- assert mock_smtp.call_count == side_effects.index(final_mock) + 1
- assert not mock_smtp_ssl.called
- assert final_mock.starttls.called
- final_mock.sendmail.assert_called_once_with("from", "to",
msg.as_string())
- assert final_mock.quit.called
+ if process_context:
+ monkeypatch.setenv("_AIRFLOW_PROCESS_CONTEXT", process_context)
+ with pytest.warns(DeprecatedImportWarning):
+ send_mime_email = email.send_mime_email
+
+ send_mime_email("[email protected]", ["[email protected]"],
MIMEMultipart(), conn_id="smtp_db_only")
+
+ if expected_login:
+
mock_smtp.return_value.login.assert_called_once_with(*expected_login)
+ else:
+ mock_smtp.return_value.login.assert_not_called()
+ assert mock_smtp.return_value.sendmail.called
