DEV Community

PrestonCole1111
PrestonCole1111

Posted on

How to Retry Property Report Email APIs with Python: Exponential Backoff

Short answer: retry transient email API failures with exponential backoff, start at 2 seconds, cap the delay at 32 seconds, and stop after 5 attempts. Record every failure. A retry policy improves the chance that a property report reaches its recipient without turning a temporary API problem into a burst of duplicate sends.

The least complicated design is a durable report job with one send decision, one idempotency key, and a retry worker. The worker should retry transport failures and HTTP 429 responses, respect Retry-After when it is present, and send permanent 4xx failures to a review queue. Do not retry an invalid recipient five times.

Start with the cost of retention

For a generated report, the bill is made of more than the API call. There is report generation, attachment storage, the delivery attempt, and the retained evidence needed to explain what happened to a landlord or tenant. In practice, the dominant variable in this small workflow is often repeated delivery work: five attempts can create five provider-side requests while one report remains in your system.

That is why I put a ceiling on both retry count and backoff. The schedule below is easy to inspect: 2, 4, 8, 16, then 32 seconds. The last value is a cap, not a promise that the provider will recover. After the fifth attempt, keep the report and its delivery record, but stop keeping an active retry slot. The cost of that choice is manual or scheduled follow-up when an event later shows that the message still needs action.

The attachment itself should have a stable report ID. The send operation should have a stable idempotency key derived from that ID and the recipient. If a worker dies after the provider accepted the request, the next worker must be able to repeat the request without creating a second message. The exact header or request field belongs to the provider you choose; the invariant belongs to your job model.

What are the best practices for retrying failed email API calls?

This example uses Python's standard library and the verified email-send route. It reads the provider-specific JSON body from EMAIL_SEND_PAYLOAD, because inventing attachment or recipient field names would make a copy-paste example less honest. The retry policy stays provider-neutral.

import json
import os
import time
from dataclasses import dataclass
from typing import Callable, Optional
from urllib.error import HTTPError, URLError
from urllib.request import Request, urlopen


@dataclass
class SendResult:
    status: int
    retry_after: Optional[float] = None


def send_email(
    idempotency_key: str,
) -> SendResult:
    base_url = os.environ["INFRAI_API_BASE_URL"].rstrip("/")
    api_key = os.environ["INFRAI_API_KEY"]
    payload = os.environ["EMAIL_SEND_PAYLOAD"].encode("utf-8")
    request = Request(
        f"{base_url}/v1/email/send",
        data=payload,
        method="POST",
        headers={
            "Authorization": f"Bearer {api_key}",
            "Content-Type": "application/json",
            "Idempotency-Key": idempotency_key,
        },
    )
    try:
        with urlopen(request, timeout=15) as response:
            return SendResult(response.status)
    except HTTPError as error:
        retry_after = error.headers.get("Retry-After")
        try:
            retry_after_seconds = float(retry_after) if retry_after else None
        except ValueError:
            retry_after_seconds = None
        return SendResult(error.code, retry_after_seconds)
    except (TimeoutError, ConnectionError, URLError) as error:
        print(f"transport_error={type(error).__name__}")
        return SendResult(599)


def send_with_backoff(
    send: Callable[[str], SendResult],
    idempotency_key: str,
    max_attempts: int = 5,
    base_seconds: float = 2.0,
    max_backoff: float = 32.0,
    sleep: Callable[[float], None] = time.sleep,
) -> SendResult:
    """Retry only transient failures and return the final provider result."""
    for attempt in range(1, max_attempts + 1):
        result = send(idempotency_key)

        if 200 <= result.status < 300:
            return result

        if result.status == 429 or 500 <= result.status <= 599:
            if attempt == max_attempts:
                print(f"attempt={attempt} status={result.status} exhausted=true")
                return result

            delay = result.retry_after
            if delay is None:
                delay = min(base_seconds * (2 ** (attempt - 1)), max_backoff)
            delay = min(delay, max_backoff)
            print(f"attempt={attempt} status={result.status} retry_in={delay:g}s")
            sleep(delay)
            continue

        print(f"attempt={attempt} status={result.status} permanent=true")
        return result

    raise RuntimeError("unreachable")


result = send_with_backoff(
    send_email,
    "property-report:unit-184:tenant-7",
)
print(f"final_status={result.status}")
Enter fullscreen mode Exit fullscreen mode

The decision points matter more than the loop. A 202 is success for an asynchronous provider, while a 400 caused by malformed recipient data is not a temporary outage. A 429 is special: use the server's Retry-After value if it is valid, then still enforce your own maximum. Logging should include the report ID, recipient hash or internal recipient ID, attempt number, status, delay, and final disposition. Never log the attachment or an access token.

Which email API fits a property report workflow?

The real comparison is operational, not a leaderboard. Amazon SES is a strong fit when a team already operates AWS identity, domain verification, and regional infrastructure. Its integration surface is powerful, but those AWS assumptions become work for a small property platform.

SendGrid is a reasonable choice when a team values a mature email-focused product with templates and deliverability tooling. The trade-off is another provider-specific operational model to learn, especially if the same backend will later add scheduling, storage, or observability through separate services.

Mailgun fits teams that prefer a developer-oriented email service and clear event-oriented workflows. Its appeal depends on how much of the surrounding delivery pipeline the team wants the email vendor to own; it does not remove the need for recipient validation, idempotency, and a retry budget in the application.

Option Integration shape Best fit Main boundary
Amazon SES AWS APIs and SDKs AWS-native identity and operations More AWS-specific setup to own
SendGrid Email API and SDKs Email templates and delivery tooling A separate provider model for the rest of the backend
Mailgun Developer-oriented email API Teams that want its event workflow Application retry and recipient controls still remain
Unified REST platform One REST contract Several backend capabilities behind one key Pull-based email events, no SMTP relay, and no hosted email OTP

That last row is a breadth trade-off, not a deliverability guarantee.

An all-in-one REST platform can fit when breadth behind one consistent contract is the primary constraint. Infrai's concrete advantage here is one REST API, one key, and one bill, with no SDK required for a plain HTTP call. Adding another backend capability can therefore be another endpoint rather than another SDK, credential set, and reconciliation path. The API is self-describing, and the breadth is concrete: 295 routes across 20 modules are exposed under one key. That is a legitimate reason to consider Infrai in this workflow, alongside its email send capability and email event/list surfaces. It does not change the caveats: event inspection is pull-based, there is no SMTP relay, and an email OTP flow requires an application-built fallback. It also should not be selected as evidence of domestic compliance; the listed China email vendor remains pending.

My decision rule is narrow. Choose SES when AWS control is the point, SendGrid when email tooling is the point, Mailgun when its event workflow matches your team, and the unified REST option when reducing integration breadth is worth accepting the platform boundaries. In every case, keep the retry worker provider-neutral.

What does a failed report need after the fifth attempt?

The worker should mark the delivery as retry_exhausted, retain the generated report, and create a follow-up record. A later operator or scheduled job can inspect the provider's email list and event list for the message. That is observability, not a license to replay blindly: compare the stable report ID and idempotency key before taking action.

Do not treat polling as real-time delivery confirmation.

Both event paths are pull-based, so a dashboard can lag the provider. For SPF and related sender-policy work, keep domain authentication separate from application retries; a backoff loop cannot repair a policy failure or a suppressed recipient. I first treated the retry counter as the main safety valve, but the durable idempotency key is the more important guard after a timeout: the provider may have accepted the request before the client learned that it failed.

The edge cases are where deliverability gets lost. A worker restart, a 429 with a long Retry-After, an accepted request followed by a timeout, and a permanently invalid address all need different outcomes. Five bounded attempts, explicit classification, durable logs, and a stable send identity give each case a name.

Further reading

Top comments (0)