The short answer is no, MTA-STS (Mail Transfer Agent Strict Transport Security) does not require DNSSEC for its policy discovery mechanism. This is one of its key design features, intended to make it more accessible and easier to implement for domain owners compared to other security protocols.
While MTA-STS uses DNS to signal that a policy exists, the actual security of the policy itself relies on HTTPS and the public Certificate Authority (CA) system, not on the cryptographic authentication provided by DNSSEC. This distinction is crucial for understanding how MTA-STS works and why it was developed.
MTA-STS policy discovery is a two-part process. When a sending mail server wants to send an email to a domain using MTA-STS, it performs the following steps:
The security of this process hinges on the sending server validating the web server's SSL/TLS certificate. This ensures the policy it receives is authentic and has not been tampered with by an attacker.
The DNS record is purely a discovery mechanism. Its job is to announce to the world that an MTA-STS policy is available and to provide an ID for that policy. Sending servers check this record to see if a policy exists or if it has been updated since the last time they checked.
This reliance on DNS for discovery does present a potential weakness. As the official RFC 8461 acknowledges, an attacker who can block DNS responses could prevent a sending server from discovering the policy in the first place. This is a known trade-off made to avoid the complexities of requiring DNSSEC.
The question of DNSSEC often arises from confusion between MTA-STS and a similar, but distinct, protocol called DANE (DNS-Based Authentication of Named Entities). While both aim to secure email in transit, they use different methods.
As one guide explains, DANE provides its security guarantees by using DNSSEC to bind TLS certificates directly to a domain name in DNS. This approach can be more secure as it removes the reliance on third-party Certificate Authorities, but it has a significant prerequisite: the domain must be fully configured with DNSSEC. Because DNSSEC is not yet universally supported by all DNS providers, this presents a major hurdle for adoption.
MTA-STS was created as a more pragmatic alternative. By not requiring DNSSEC, it allows a far greater number of domains to implement strong transport layer security for their email without needing to tackle a complex DNSSEC deployment first.
To be clear, MTA-STS does not require DNSSEC. Its design deliberately separates policy discovery (via DNS) from policy enforcement (via HTTPS) to make implementation easier. The security of your MTA-STS policy is guaranteed by a valid TLS certificate from a trusted CA, not by DNSSEC.
While you can certainly use DNSSEC for your domain to add an extra layer of protection to the initial DNS lookup, it is not a prerequisite for MTA-STS to function correctly. This accessibility is precisely what has made MTA-STS a popular and effective tool for improving email security across the internet.