Why are delivery error rates different between API and UI?

Key findings

• Known issue: Differences in delivery error rates between API and UI are frequently observed and discussed within the email community.
• Data aggregation: UI dashboards may present aggregated or summarized data that differs from the raw, granular data available via API.
• Google Postmaster Tools behavior: GPT often has limitations in its data reporting, including potential redaction of detailed information or inaccuracies in specific breakdowns. You can learn more about these behaviors in our ultimate guide to Google Postmaster Tools v2.
• Calculation or scaling errors: Discrepancies can sometimes stem from how data is calculated or scaled within the reporting dashboard itself, rather than the raw API data.

Key considerations

• Verify raw API responses: Always cross-check the overall reported rates with the detailed data provided directly by the API to ensure consistency.
• Understand data limitations: Be aware that providers like Google may redact or summarize data for various reasons, impacting the completeness of reports.
• Compare specific error types: Focus on the specific categories of delivery errors (e.g., blocks, throttles) rather than just the overall delivery error rate to identify where the differences lie. For more on error codes, see this article on understanding API error codes.
• Check for synchronization delays: Consider if the API and UI data are being updated at different frequencies or with different processing times, leading to temporary discrepancies. For example, some tools like Postmaster Tools and other deliverability tools may report conflicting authentication results.

Key opinions

• Common occurrence: Many marketers report experiencing differences in delivery error rates between API and UI interfaces.
• Google Postmaster Tools unreliability: The detailed breakdowns of delivery errors within GPT's UI are often considered inaccurate by marketers. Our article why is Google Postmaster Tools data glitchy explores this further.
• Raw data verification: It is advised to always check the raw API responses to confirm that the data being sent matches expectations, rather than relying solely on dashboard calculations.
• Data redaction: Google (and other providers) are known to redact certain data points, which can lead to situations where overall error rates are high but specific error categories show 0%.
• Line chart vs. detail view: Many observe discrepancies between the aggregate numbers shown in line charts and the specific details when drilling down into a particular day's data.

Key considerations

• Don't rely solely on UI summaries: While convenient, user interfaces may not always provide the full or most accurate picture, especially for complex metrics like delivery errors.
• Deep dive into API data: For critical analysis, directly accessing and analyzing granular data via API is often necessary.
• Account for data redaction: Be aware that data providers may have policies to suppress or aggregate data for various reasons, which can explain missing details. This can lead to issues like higher spam rates in Postmaster Tools.
• Consider calculation errors: If building custom dashboards or integrating API data, ensure your calculations and interpretations align with how the source system processes the data. More on API issues can be found in this HyperTest article about top reasons for API failures.

Key opinions

• GPT breakdown inaccuracy: Experts advise against placing full trust in Google Postmaster Tools' detailed breakdowns of delivery errors due to their frequent inaccuracy.
• Data redaction and summarization: ISPs often redact or summarize data, which can account for differences between overall reported rates and detailed breakdowns.
• Scaling and interpretation: Discrepancies can arise from how data is scaled or interpreted, especially in custom API integrations, meaning API data may approximate but not perfectly replicate UI graphs.
• Inherent reporting challenges: The nature of large-scale email ecosystems makes achieving perfect consistency across all reporting interfaces a significant challenge.

Key considerations

• Skeptical review of breakdowns: Always approach detailed error breakdowns from GPT (and similar tools) with a critical eye, as they may not be fully reliable. For more troubleshooting, see what reasons PT provides for delivery errors.
• Validate API data interpretation: If consuming data via API, rigorously test and validate your interpretation logic to ensure it aligns as closely as possible with the source's intended representation.
• Anticipate discrepancies: Understand that some level of discrepancy may be unavoidable due to the sheer volume of data and the different processing pipelines for UI and API data. This article on API monitoring key metrics provides additional insight.
• Cross-reference multiple sources: Whenever possible, compare data from various sources, including your own sending logs and other deliverability tools, to form a more complete picture. For example, check for discrepancies between DMARC reports from Google and Yahoo.

Key findings

• Error code definitions: API documentation typically provides detailed definitions of error codes and their associated meanings, which are vital for diagnosing specific delivery issues.
• Consistent error responses: Technical guides often emphasize the importance of designing consistent and informative error responses within APIs for better developer experience and troubleshooting.
• Data aggregation methods: Large platforms may use different data aggregation or sampling methods for their APIs versus their UIs, leading to varying reported metrics.
• Fundamental data model differences: Discrepancies can stem from fundamental differences in how data models are structured and processed between various systems, such as monitoring tools. As an example, there is no direct apples-to-apples comparison for error rates between New Relic and OpenTelemetry since their models are fundamentally different, as detailed by New Relic documentation.

Key considerations

• Consult API documentation: Always refer to the specific API documentation for details on data fields, aggregation periods, and error definitions relevant to your use case.
• Implement robust error handling: When integrating with APIs, ensure your system includes comprehensive error handling to capture and interpret all relevant failure data accurately.
• Understand UI aggregation: Recognize that high-level metrics in a UI dashboard might be summarized, potentially masking underlying granular data that an API would provide. This relates to how Google Postmaster Tools' spam rate dashboard operates.
• Account for provider policies: Be aware that providers might enforce data redaction or throttling policies, which can affect the completeness and timeliness of the data received via API. This is especially true for temp-fail delivery errors in GPT.