Unify SMTP2GO email and SMS into one webhook URL.
Deploy Beta / unit-tests (push) Successful in 13s
Deploy Beta / docker (push) Successful in 18s
Deploy Beta / deploy-beta (push) Successful in 1m38s

Classify payloads on /webhooks/smtp2go/ so one SMTP2GO webhook covers both channels under the 10-webhook limit; keep /email/ and /sms/ as aliases and harden SMS event matching.
This commit is contained in:
2026-08-10 11:06:37 -05:00
parent f7f3174e1c
commit ee28dcabab
7 changed files with 472 additions and 115 deletions
+2 -2
View File
@@ -42,8 +42,8 @@ DEFAULT_FROM_EMAIL=noreply@mkdrealtor.com
# For real SMTP2GO delivery locally, uncomment: # For real SMTP2GO delivery locally, uncomment:
# EMAIL_BACKEND=django.core.mail.backends.smtp.EmailBackend # EMAIL_BACKEND=django.core.mail.backends.smtp.EmailBackend
SMTP2GO_SMS_API_KEY= SMTP2GO_SMS_API_KEY=
# Shared secret for email + SMS webhooks. # Shared secret for the unified SMTP2GO webhook (email + SMS).
# In SMTP2GO: Authorization header = Bearer, value = this secret. # In SMTP2GO: one webhook URL …/webhooks/smtp2go/ ; Authorization = Bearer + this secret.
SMTP2GO_WEBHOOK_SECRET= SMTP2GO_WEBHOOK_SECRET=
# Postcards — PCM Integrations (DirectMail API v3) # Postcards — PCM Integrations (DirectMail API v3)
+27 -22
View File
@@ -4,8 +4,8 @@ Campaign compose/send, SMTP2GO email + SMS, PCM Integrations postcards, and deli
## SMTP2GO webhook setup ## SMTP2GO webhook setup
Campaign report page polls provider events every 10s. Create **two** webhooks in Campaign report page polls provider events every 10s. Create **one** SMTP2GO
SMTP2GO → **Settings → Webhooks** (email and SMS stay separate). webhook (email + SMS share a URL — paid plans cap at 10 webhooks).
### Auth (`SMTP2GO_WEBHOOK_SECRET`) ### Auth (`SMTP2GO_WEBHOOK_SECRET`)
@@ -14,35 +14,39 @@ SMTP2GO → **Settings → Webhooks** (email and SMS stay separate).
(do not leave it as “None”). (do not leave it as “None”).
3. Fallback: `?token=<SMTP2GO_WEBHOOK_SECRET>` on the webhook URL also works. 3. Fallback: `?token=<SMTP2GO_WEBHOOK_SECRET>` on the webhook URL also works.
### Email webhook ### Unified email + SMS webhook
| Field | Value | | Field | Value |
|-------|--------| |-------|--------|
| URL | `https://mkdrealtor.com/portal/messaging/webhooks/email/` | | URL | `https://mkdrealtor.com/portal/messaging/webhooks/smtp2go/` |
| Authorization header | **Bearer** + `SMTP2GO_WEBHOOK_SECRET` | | Authorization header | **Bearer** + `SMTP2GO_WEBHOOK_SECRET` |
| Output type | JSON | | Output type | JSON |
| Users | email SMTP user(s) **and** the SMS API key (`SMTP2GO_SMS_API_KEY`) |
| Email events | processed, bounced, rejected, spam, delivered, unsub/resub, opened, clicked | | Email events | processed, bounced, rejected, spam, delivered, unsub/resub, opened, clicked |
| Email headers | `X-Monica-Message-Id` | | Email headers | `X-Monica-Message-Id` |
| SMS events | leave unchecked | | SMS events | Submitted, Sending, Delivered, Failed, Rejected (and Opt-out if shown) |
`X-Monica-Message-Id` is set on every campaign email send and is required so webhook The handler classifies each POST from the payload (`sms_*` / `destination_number`
events match the correct recipient row. Invalid / missing header values no longer → SMS; `rcpt` / `email_id` / `X-Monica-Message-Id` → email; inbound `text=STOP`
500 the endpoint (SMTP2GO “Test this webhook” often sends a sample non-UUID). without `event` → SMS opt-out).
`X-Monica-Message-Id` is set on every campaign email send and is required so email
webhook events match the correct recipient row. Invalid / missing header values
no longer 500 the endpoint (SMTP2GO “Test this webhook” often sends a sample
non-UUID).
SMS correlation uses `message_id` (SMS id), then `destination_number` phone
fallback. Do **not** treat webhook `id` as the SMS id.
**Opt-out:** SMTP2GO auto-handles replies `STOP` / `UNSUB` / `UNSUBSCRIBE`.
This endpoint also accepts inbound POSTs without an `event` field
(`text=STOP`, `from=…`) and opts the contact out of SMS. Later sends to
opted-out numbers are typically `sms_rejected`.
Beta / other hosts: swap the hostname, keep the path. Beta / other hosts: swap the hostname, keep the path.
### SMS webhook (separate) Legacy aliases (same handler): `/portal/messaging/webhooks/email/` and
`/portal/messaging/webhooks/sms/` — prefer `/smtp2go/` for new configs.
| Field | Value |
|-------|--------|
| URL | `https://mkdrealtor.com/portal/messaging/webhooks/sms/` |
| Authorization header | **Bearer** + same `SMTP2GO_WEBHOOK_SECRET` |
| Output type | JSON |
| Email events | leave unchecked |
| SMS events | Submitted, Sending, Delivered, Failed, Rejected, Opt-out |
This endpoint also accepts inbound reply POSTs (`text=STOP`, `from=…`) and opts the
contact out of SMS.
## PCM Integrations (postcards) ## PCM Integrations (postcards)
@@ -119,8 +123,9 @@ PCM_RETURN_ADDRESS={…}
| Path | Purpose | | Path | Purpose |
|------|---------| |------|---------|
| `POST /portal/messaging/webhooks/email/` | Email delivery / open / click / bounce / … | | `POST /portal/messaging/webhooks/smtp2go/` | Unified SMTP2GO email + SMS (+ inbound STOP) |
| `POST /portal/messaging/webhooks/sms/` | SMS delivery events + inbound STOP | | `POST /portal/messaging/webhooks/email/` | Legacy alias → same as `/smtp2go/` |
| `POST /portal/messaging/webhooks/sms/` | Legacy alias → same as `/smtp2go/` |
| `POST /portal/messaging/webhooks/postcard/` | PCM order / mail tracking events | | `POST /portal/messaging/webhooks/postcard/` | PCM order / mail tracking events |
| `GET /portal/messaging/campaigns/<id>/status.json` | Live stats for the campaign report UI | | `GET /portal/messaging/campaigns/<id>/status.json` | Live stats for the campaign report UI |
| `GET /portal/messaging/postcard/` | PCM designer iframe | | `GET /portal/messaging/postcard/` | PCM designer iframe |
+6 -2
View File
@@ -37,9 +37,13 @@ def send_sms(message) -> str:
) )
response.raise_for_status() response.raise_for_status()
data = response.json() if response.content else {} data = response.json() if response.content else {}
# SMTP2GO returns varying shapes; store a useful id when present. # Prefer SMS id fields used on webhooks (`message_id` / `sms_id`).
nested = data.get("data") if isinstance(data.get("data"), dict) else {}
return str( return str(
data.get("data", {}).get("sms_id") nested.get("sms_id")
or nested.get("message_id")
or data.get("sms_id")
or data.get("message_id")
or data.get("request_id") or data.get("request_id")
or f"sms-{message.pk}" or f"sms-{message.pk}"
) )
+253
View File
@@ -630,6 +630,116 @@ class Smtp2goSmsWebhookTests(TestCase):
self.message.refresh_from_db() self.message.refresh_from_db()
self.assertEqual(self.message.status, Message.Status.FAILED) self.assertEqual(self.message.status, Message.Status.FAILED)
def test_sms_short_event_name_delivered(self):
"""API sms_events use short names; body may omit sms_ prefix."""
url = reverse("messaging:sms_webhook")
import json
response = self.client.post(
url,
data=json.dumps(
{
"event": "delivered",
"message_id": "sms-provider-99",
"destination_number": "5550142291",
}
),
content_type="application/json",
)
self.assertEqual(response.status_code, 200)
self.message.refresh_from_db()
self.assertEqual(self.message.status, Message.Status.DELIVERED)
self.assertTrue(
ProviderEvent.objects.filter(
message=self.message, event_type="sms_delivered"
).exists()
)
def test_sms_ui_label_and_form_encoded(self):
url = reverse("messaging:sms_webhook")
self.message.status = Message.Status.QUEUED
self.message.save(update_fields=["status"])
response = self.client.post(
url,
data={
"event": "Submitted",
"message_id": "sms-provider-99",
"destination_number": "5550142291",
},
)
self.assertEqual(response.status_code, 200)
self.message.refresh_from_db()
self.assertEqual(self.message.status, Message.Status.SENT)
self.assertTrue(
ProviderEvent.objects.filter(
message=self.message, event_type="sms_submitted"
).exists()
)
def test_sms_rejected_marks_failed(self):
url = reverse("messaging:sms_webhook")
import json
response = self.client.post(
url,
data=json.dumps(
{
"event": "sms_rejected",
"message_id": "sms-provider-99",
"destination_number": "5550142291",
"status_code": "blocked",
}
),
content_type="application/json",
)
self.assertEqual(response.status_code, 200)
self.message.refresh_from_db()
self.assertEqual(self.message.status, Message.Status.FAILED)
def test_sms_webhook_id_is_not_used_as_message_id(self):
"""Docs: `id` is the webhook notification id, not the SMS id."""
url = reverse("messaging:sms_webhook")
import json
response = self.client.post(
url,
data=json.dumps(
{
"event": "sms_delivered",
"id": "sms-provider-99",
"destination_number": "19999999999",
}
),
content_type="application/json",
)
self.assertEqual(response.status_code, 200)
self.message.refresh_from_db()
self.assertEqual(self.message.status, Message.Status.SENT)
self.assertTrue(
ProviderEvent.objects.filter(
message=None, event_type="sms_delivered"
).exists()
)
def test_sms_opt_out_unmatched_suppresses_by_phone(self):
url = reverse("messaging:sms_webhook")
import json
self.message.delete()
response = self.client.post(
url,
data=json.dumps(
{
"event": "opt-out",
"message_id": "unknown",
"destination_number": "+15550142291",
}
),
content_type="application/json",
)
self.assertEqual(response.status_code, 200)
self.assertFalse(contact_may_receive(self.contact, Channel.SMS))
def test_inbound_stop_still_works(self): def test_inbound_stop_still_works(self):
url = reverse("messaging:sms_webhook") url = reverse("messaging:sms_webhook")
response = self.client.post( response = self.client.post(
@@ -638,6 +748,14 @@ class Smtp2goSmsWebhookTests(TestCase):
self.assertEqual(response.status_code, 200) self.assertEqual(response.status_code, 200)
self.assertFalse(contact_may_receive(self.contact, Channel.SMS)) self.assertFalse(contact_may_receive(self.contact, Channel.SMS))
def test_inbound_unsub_keyword(self):
url = reverse("messaging:sms_webhook")
response = self.client.post(
url, {"from": "5550142291", "text": "UNSUB"}
)
self.assertEqual(response.status_code, 200)
self.assertFalse(contact_may_receive(self.contact, Channel.SMS))
def test_sms_webhook_requires_bearer_when_secret_set(self): def test_sms_webhook_requires_bearer_when_secret_set(self):
url = reverse("messaging:sms_webhook") url = reverse("messaging:sms_webhook")
import json import json
@@ -663,6 +781,141 @@ class Smtp2goSmsWebhookTests(TestCase):
self.assertEqual(ok.status_code, 200) self.assertEqual(ok.status_code, 200)
class UnifiedSmtp2goWebhookTests(TestCase):
def setUp(self):
self.client = Client()
self.email_contact = Contact.objects.create(
email="pat@example.com",
first_name="Pat",
)
set_channel_consent(
self.email_contact, Channel.EMAIL, opted_in=True, reason="test"
)
self.email_campaign = create_campaign_draft(
name="Unified email",
audience=Campaign.Audience.EMAIL_OPT_IN,
subject="Hello",
body="Body",
)
self.email_message = self.email_campaign.messages.get()
self.email_message.status = Message.Status.SENT
self.email_message.save(update_fields=["status"])
self.sms_contact = Contact.objects.create(
email="sms.pat@example.com",
phone="+15550142291",
first_name="Pat",
)
set_channel_consent(
self.sms_contact, Channel.SMS, opted_in=True, reason="test"
)
self.sms_campaign = create_campaign_draft(
name="Unified SMS",
audience=Campaign.Audience.SMS_OPT_IN,
subject="",
body="Hi",
)
self.sms_message = self.sms_campaign.messages.get()
self.sms_message.status = Message.Status.SENT
self.sms_message.provider_message_id = "sms-unified-1"
self.sms_message.save()
def test_classify_helpers(self):
from messaging.webhooks import classify_smtp2go_payload
self.assertEqual(
classify_smtp2go_payload({"from": "555", "text": "STOP"}),
"sms_inbound",
)
self.assertEqual(
classify_smtp2go_payload(
{"event": "sms_delivered", "message_id": "x"}
),
"sms",
)
self.assertEqual(
classify_smtp2go_payload(
{
"event": "delivered",
"destination_number": "5550142291",
"message_id": "x",
}
),
"sms",
)
self.assertEqual(
classify_smtp2go_payload(
{
"event": "delivered",
"rcpt": "pat@example.com",
"email_id": "e1",
}
),
"email",
)
self.assertEqual(
classify_smtp2go_payload({"event": "click", "rcpt": "a@b.c"}),
"email",
)
def test_unified_url_routes_email_and_sms(self):
import json
url = reverse("messaging:smtp2go_webhook")
email_resp = self.client.post(
url,
data=json.dumps(
{
"event": "click",
"rcpt": "pat@example.com",
"X-Monica-Message-Id": str(self.email_message.pk),
}
),
content_type="application/json",
)
self.assertEqual(email_resp.status_code, 200)
self.assertEqual(email_resp.json()["channel"], "email")
self.email_message.refresh_from_db()
self.assertEqual(self.email_message.status, Message.Status.CLICKED)
sms_resp = self.client.post(
url,
data=json.dumps(
{
"event": "sms_delivered",
"message_id": "sms-unified-1",
"destination_number": "5550142291",
}
),
content_type="application/json",
)
self.assertEqual(sms_resp.status_code, 200)
self.assertEqual(sms_resp.json()["channel"], "sms")
self.sms_message.refresh_from_db()
self.assertEqual(self.sms_message.status, Message.Status.DELIVERED)
def test_legacy_email_path_still_accepts_sms(self):
"""Legacy /email/ alias is the unified handler."""
import json
url = reverse("messaging:email_webhook")
response = self.client.post(
url,
data=json.dumps(
{
"event": "delivered",
"message_id": "sms-unified-1",
"destination_number": "5550142291",
}
),
content_type="application/json",
)
self.assertEqual(response.status_code, 200)
self.assertEqual(response.json()["channel"], "sms")
self.sms_message.refresh_from_db()
self.assertEqual(self.sms_message.status, Message.Status.DELIVERED)
class PcmPostcardWebhookTests(TestCase): class PcmPostcardWebhookTests(TestCase):
def setUp(self): def setUp(self):
self.client = Client() self.client = Client()
+2
View File
@@ -44,6 +44,8 @@ urlpatterns = [
views.postcard_design_save, views.postcard_design_save,
name="postcard_design_save", name="postcard_design_save",
), ),
path("webhooks/smtp2go/", views.smtp2go_webhook, name="smtp2go_webhook"),
# Legacy aliases — same unified SMTP2GO handler.
path("webhooks/sms/", views.sms_webhook, name="sms_webhook"), path("webhooks/sms/", views.sms_webhook, name="sms_webhook"),
path("webhooks/email/", views.email_webhook, name="email_webhook"), path("webhooks/email/", views.email_webhook, name="email_webhook"),
path( path(
+50 -52
View File
@@ -39,7 +39,7 @@ from messaging.webhooks import (
PROVIDER_PCM, PROVIDER_PCM,
PROVIDER_SMS, PROVIDER_SMS,
campaign_engagement_stats, campaign_engagement_stats,
is_inbound_sms_stop, classify_smtp2go_payload,
parse_webhook_payload, parse_webhook_payload,
process_pcm_postcard_webhook, process_pcm_postcard_webhook,
process_smtp2go_email_webhook, process_smtp2go_email_webhook,
@@ -248,13 +248,13 @@ def _events_provider_filter(campaign: Campaign) -> tuple[list[str], str, str]:
[PROVIDER_SMS], [PROVIDER_SMS],
"Recent SMTP2GO events", "Recent SMTP2GO events",
"No webhook events yet. SMTP2GO must POST SMS events to " "No webhook events yet. SMTP2GO must POST SMS events to "
"<code>/portal/messaging/webhooks/sms/</code>.", "<code>/portal/messaging/webhooks/smtp2go/</code>.",
) )
return ( return (
[PROVIDER_EMAIL], [PROVIDER_EMAIL],
"Recent SMTP2GO events", "Recent SMTP2GO events",
"No webhook events yet. SMTP2GO must POST opens/clicks to " "No webhook events yet. SMTP2GO must POST opens/clicks to "
"<code>/portal/messaging/webhooks/email/</code> " "<code>/portal/messaging/webhooks/smtp2go/</code> "
"(see messaging README). SMTP2GOs own “Clicked” feed does not fill " "(see messaging README). SMTP2GOs own “Clicked” feed does not fill "
"this table by itself.", "this table by itself.",
) )
@@ -863,33 +863,44 @@ def postcard_webhook(request):
@csrf_exempt @csrf_exempt
@require_POST @require_POST
def sms_webhook(request): def smtp2go_webhook(request):
""" """
SMTP2GO SMS webhook — delivery status events + inbound STOP replies. Unified SMTP2GO webhook — email + SMS events + inbound STOP replies.
Configure a *separate* webhook in SMTP2GO → Settings → Webhooks: One SMTP2GO webhook URL (paid plans cap at 10 webhooks):
URL: https://<host>/portal/messaging/webhooks/sms/
URL: https://<host>/portal/messaging/webhooks/smtp2go/
Authorization header: Bearer + value = SMTP2GO_WEBHOOK_SECRET Authorization header: Bearer + value = SMTP2GO_WEBHOOK_SECRET
Output type: JSON Output type: JSON
SMS events: Submitted, Sending, Delivered, Failed, Rejected, Opt-out Users: email SMTP user(s) *and* the SMS API key used to send
(leave Email events unchecked on this webhook) Email events: all delivery/engagement boxes
Email headers: X-Monica-Message-Id
SMS events: Submitted, Sending, Delivered, Failed, Rejected (Opt-out if shown)
Inbound gateway POSTs without ``event`` (text=STOP, from=…) still opt out. Legacy aliases ``/webhooks/email/`` and ``/webhooks/sms/`` hit this same view.
Payload shape selects the processor (email vs sms_* vs inbound STOP).
""" """
channel = "sms" # Log/auth before parsing so request.body stays readable (HMAC + Grafana dump).
_log_webhook_request(request, channel=channel) _log_webhook_request(request, channel="smtp2go")
if not _webhook_authorized( if not _webhook_authorized(
request, secret=settings.SMTP2GO_WEBHOOK_SECRET or "" request, secret=settings.SMTP2GO_WEBHOOK_SECRET or ""
): ):
_log_webhook_auth_failed(request, channel=channel) _log_webhook_auth_failed(request, channel="smtp2go")
return HttpResponseForbidden("invalid webhook token") return HttpResponseForbidden("invalid webhook token")
payload = parse_webhook_payload(request) payload = parse_webhook_payload(request)
if not payload: if not payload:
payload = request.POST.dict() or {} payload = request.POST.dict() or {}
# Inbound reply (STOP) — different payload shape than delivery events. kind = classify_smtp2go_payload(payload)
if is_inbound_sms_stop(payload): channel = {
"sms_inbound": "sms",
"sms": "sms",
"email": "email",
}.get(kind, "smtp2go")
logger.info("webhook_classified channel=%s kind=%s", channel, kind)
if kind == "sms_inbound":
phone = ( phone = (
payload.get("from") payload.get("from")
or payload.get("phone") or payload.get("phone")
@@ -904,56 +915,43 @@ def sms_webhook(request):
stopped, stopped,
phone, phone,
) )
return JsonResponse({"ok": True, "opt_out": stopped}) return JsonResponse({"ok": True, "channel": "sms", "opt_out": stopped})
try: if kind == "unknown":
event = process_smtp2go_sms_webhook(payload) logger.warning(
except Exception as exc: # noqa: BLE001 "webhook_unmatched channel=smtp2go unrecognized_payload keys=%s",
logger.exception("SMTP2GO SMS webhook processing failed") sorted(str(k) for k in payload.keys()),
_log_webhook_result(channel=channel, error=str(exc)) )
return JsonResponse({"ok": False, "error": "processing_failed"}, status=200)
_log_webhook_result(channel=channel, event=event)
return JsonResponse( return JsonResponse(
{ {"ok": False, "error": "unrecognized_payload", "channel": None},
"ok": True, status=200,
"matched": bool(event and event.message_id),
"event_id": event.pk if event else None,
}
) )
@csrf_exempt
@require_POST
def email_webhook(request):
"""
SMTP2GO email event webhook (delivered / open / click / bounce / …).
Configure in SMTP2GO → Settings → Webhooks:
URL: https://<host>/portal/messaging/webhooks/email/
Authorization header: Bearer + value = SMTP2GO_WEBHOOK_SECRET
Output type: JSON
Email events: all delivery/engagement boxes
Email headers: X-Monica-Message-Id
"""
channel = "email"
_log_webhook_request(request, channel=channel)
if not _webhook_authorized(
request, secret=settings.SMTP2GO_WEBHOOK_SECRET or ""
):
_log_webhook_auth_failed(request, channel=channel)
return HttpResponseForbidden("invalid webhook token")
payload = parse_webhook_payload(request)
try: try:
if kind == "sms":
event = process_smtp2go_sms_webhook(payload)
else:
event = process_smtp2go_email_webhook(payload) event = process_smtp2go_email_webhook(payload)
except Exception as exc: # noqa: BLE001 — never 500 SMTP2GO (they retry for 48h) except Exception as exc: # noqa: BLE001 — never 500 SMTP2GO (they retry for 48h)
logger.exception("SMTP2GO email webhook processing failed") logger.exception("SMTP2GO %s webhook processing failed", kind)
_log_webhook_result(channel=channel, error=str(exc)) _log_webhook_result(channel=channel, error=str(exc))
return JsonResponse({"ok": False, "error": "processing_failed"}, status=200) return JsonResponse(
{"ok": False, "error": "processing_failed", "channel": channel},
status=200,
)
_log_webhook_result(channel=channel, event=event) _log_webhook_result(channel=channel, event=event)
return JsonResponse( return JsonResponse(
{ {
"ok": True, "ok": True,
"channel": channel,
"matched": bool(event and event.message_id), "matched": bool(event and event.message_id),
"event_id": event.pk if event else None, "event_id": event.pk if event else None,
} }
) )
# Legacy path names — same unified handler (keep SMTP2GO configs working).
email_webhook = smtp2go_webhook
sms_webhook = smtp2go_webhook
+128 -33
View File
@@ -68,6 +68,21 @@ _EMAIL_EVENT_ALIASES = {
"resubscribed": "resubscribe", "resubscribed": "resubscribe",
} }
# API sms_events use short names (delivered); webhook body often uses sms_delivered.
# UI test labels / Opt-Out may arrive without the sms_ prefix.
_SMS_EVENT_ALIASES = {
"submitted": "sms_submitted",
"sending": "sms_sending",
"delivered": "sms_delivered",
"failed": "sms_failed",
"rejected": "sms_rejected",
"opt_out": "sms_opt_out",
"optout": "sms_opt_out",
"sms_optout": "sms_opt_out",
}
_SMS_OPT_OUT_EVENTS = frozenset({"sms_opt_out"})
def _as_str(value: Any) -> str: def _as_str(value: Any) -> str:
"""Coerce webhook field values to a stripped string (lists / None safe).""" """Coerce webhook field values to a stripped string (lists / None safe)."""
@@ -98,6 +113,12 @@ def _normalize_email_event(event: str) -> str:
return _EMAIL_EVENT_ALIASES.get(event, event) return _EMAIL_EVENT_ALIASES.get(event, event)
def _normalize_sms_event(event: str) -> str:
"""Map UI / short API names to docs canonical sms_* event strings."""
event = (event or "").strip().lower().replace("-", "_").replace(" ", "_")
return _SMS_EVENT_ALIASES.get(event, event)
def parse_webhook_payload(request: HttpRequest) -> dict[str, Any]: def parse_webhook_payload(request: HttpRequest) -> dict[str, Any]:
"""Accept JSON or form-encoded SMTP2GO webhook bodies.""" """Accept JSON or form-encoded SMTP2GO webhook bodies."""
content_type = (request.content_type or "").lower() content_type = (request.content_type or "").lower()
@@ -362,14 +383,13 @@ def normalize_phone(value: str) -> str:
return "".join(ch for ch in (value or "") if ch.isdigit()) return "".join(ch for ch in (value or "") if ch.isdigit())
def _sms_provider_message_id(payload: dict[str, Any]) -> str:
"""SMS unique id from docs (`message_id`). Never use webhook `id`."""
return _as_str(payload.get("message_id") or payload.get("sms_id"))
def find_message_for_sms_event(payload: dict[str, Any]) -> Message | None: def find_message_for_sms_event(payload: dict[str, Any]) -> Message | None:
provider_id = ( provider_id = _sms_provider_message_id(payload)
payload.get("message_id")
or payload.get("sms_id")
or payload.get("id")
or ""
)
provider_id = str(provider_id).strip()
if provider_id: if provider_id:
message = ( message = (
Message.objects.select_related("contact", "campaign") Message.objects.select_related("contact", "campaign")
@@ -379,14 +399,15 @@ def find_message_for_sms_event(payload: dict[str, Any]) -> Message | None:
if message: if message:
return message return message
# Outbound delivery events use destination_number (recipient).
# Do not use source_number / from — those are the pool or inbound reply.
raw_phone = ( raw_phone = (
payload.get("destination_number") payload.get("destination_number")
or payload.get("to") or payload.get("to")
or payload.get("phone") or payload.get("phone")
or payload.get("from")
or "" or ""
) )
digits = normalize_phone(str(raw_phone)) digits = normalize_phone(_as_str(raw_phone))
if len(digits) < 7: if len(digits) < 7:
return None return None
@@ -411,6 +432,7 @@ def find_message_for_sms_event(payload: dict[str, Any]) -> Message | None:
Message.Status.SENT, Message.Status.SENT,
Message.Status.DELIVERED, Message.Status.DELIVERED,
Message.Status.FAILED, Message.Status.FAILED,
Message.Status.SUPPRESSED,
], ],
) )
.order_by("-sent_at", "-updated_at") .order_by("-sent_at", "-updated_at")
@@ -419,19 +441,14 @@ def find_message_for_sms_event(payload: dict[str, Any]) -> Message | None:
def _apply_sms_event(message: Message, event: str, payload: dict[str, Any]) -> None: def _apply_sms_event(message: Message, event: str, payload: dict[str, Any]) -> None:
event = (event or "").strip().lower().replace("-", "_") event = _normalize_sms_event(event)
err = ( err = _as_str(
payload.get("message") payload.get("message")
or payload.get("status_code") or payload.get("status_code")
or payload.get("context") or payload.get("context")
or ""
) )
err = str(err).strip()
provider_id = ( provider_id = _sms_provider_message_id(payload)
payload.get("message_id") or payload.get("sms_id") or ""
)
provider_id = str(provider_id).strip()
if provider_id and message.provider_message_id != provider_id: if provider_id and message.provider_message_id != provider_id:
message.provider_message_id = provider_id message.provider_message_id = provider_id
message.provider = PROVIDER_SMS message.provider = PROVIDER_SMS
@@ -439,16 +456,16 @@ def _apply_sms_event(message: Message, event: str, payload: dict[str, Any]) -> N
update_fields=["provider_message_id", "provider", "updated_at"] update_fields=["provider_message_id", "provider", "updated_at"]
) )
if event in {"sms_sending", "sending", "sms_submitted", "submitted"}: if event in {"sms_sending", "sms_submitted"}:
if message.status in {Message.Status.QUEUED, Message.Status.DRAFT}: if message.status in {Message.Status.QUEUED, Message.Status.DRAFT}:
_maybe_upgrade_status(message, Message.Status.SENT) _maybe_upgrade_status(message, Message.Status.SENT)
return return
if event in {"sms_delivered", "delivered"}: if event == "sms_delivered":
_maybe_upgrade_status(message, Message.Status.DELIVERED) _maybe_upgrade_status(message, Message.Status.DELIVERED)
return return
if event in {"sms_failed", "failed", "sms_rejected", "rejected"}: if event in {"sms_failed", "sms_rejected"}:
_maybe_upgrade_status( _maybe_upgrade_status(
message, message,
Message.Status.FAILED, Message.Status.FAILED,
@@ -456,7 +473,7 @@ def _apply_sms_event(message: Message, event: str, payload: dict[str, Any]) -> N
) )
return return
if event in {"sms_opt_out", "opt_out", "optout"}: if event in _SMS_OPT_OUT_EVENTS:
_maybe_upgrade_status( _maybe_upgrade_status(
message, Message.Status.SUPPRESSED, error="sms opt-out" message, Message.Status.SUPPRESSED, error="sms opt-out"
) )
@@ -471,7 +488,7 @@ def _apply_sms_event(message: Message, event: str, payload: dict[str, Any]) -> N
def process_smtp2go_sms_webhook(payload: dict[str, Any]) -> ProviderEvent | None: def process_smtp2go_sms_webhook(payload: dict[str, Any]) -> ProviderEvent | None:
"""Persist SMS delivery/opt-out ProviderEvent and update Message when matched.""" """Persist SMS delivery/opt-out ProviderEvent and update Message when matched."""
event = (payload.get("event") or "").strip().lower() event = _normalize_sms_event(_as_str(payload.get("event")))
if not event: if not event:
logger.warning("SMTP2GO SMS webhook missing event: %s", payload) logger.warning("SMTP2GO SMS webhook missing event: %s", payload)
return None return None
@@ -482,17 +499,16 @@ def process_smtp2go_sms_webhook(payload: dict[str, Any]) -> ProviderEvent | None
message.refresh_from_db() message.refresh_from_db()
else: else:
# Opt-out with no matched campaign message still suppresses by phone. # Opt-out with no matched campaign message still suppresses by phone.
if event.replace("-", "_") in {"sms_opt_out", "opt_out", "optout"}: if event in _SMS_OPT_OUT_EVENTS:
phone = ( phone = _as_str(
payload.get("destination_number") payload.get("destination_number")
or payload.get("from") or payload.get("from")
or payload.get("source_number") or payload.get("source_number")
or ""
) )
if phone: if phone:
from messaging.services import record_sms_stop from messaging.services import record_sms_stop
record_sms_stop(str(phone)) record_sms_stop(phone)
logger.info( logger.info(
"SMTP2GO SMS webhook unmatched event=%s phone=%s message_id=%s", "SMTP2GO SMS webhook unmatched event=%s phone=%s message_id=%s",
event, event,
@@ -503,23 +519,102 @@ def process_smtp2go_sms_webhook(payload: dict[str, Any]) -> ProviderEvent | None
return ProviderEvent.objects.create( return ProviderEvent.objects.create(
message=message, message=message,
provider=PROVIDER_SMS, provider=PROVIDER_SMS,
event_type=event, event_type=event[:64],
payload=payload, payload=_json_safe(payload) if isinstance(payload, dict) else {},
) )
def is_inbound_sms_stop(payload: dict[str, Any]) -> bool: def is_inbound_sms_stop(payload: dict[str, Any]) -> bool:
"""True for gateway-style inbound reply payloads (STOP / UNSUBSCRIBE).""" """True for gateway-style inbound reply payloads (STOP / UNSUBSCRIBE).
SMTP2GO auto-handles STOP/UNSUB/UNSUBSCRIBE replies; this catches the
inbound gateway POST shape (no ``event`` field) when configured to
forward replies. Prefer this over relying on a webhook Opt-Out event —
the API ``sms_events`` list does not include opt-out.
"""
if payload.get("event"): if payload.get("event"):
return False return False
text = ( text = _as_str(
payload.get("text") payload.get("text")
or payload.get("message") or payload.get("message")
or payload.get("message_content") or payload.get("message_content")
or "" ).upper()
return text in {"STOP", "UNSUBSCRIBE", "UNSUB", "CANCEL", "END", "QUIT"}
def classify_smtp2go_payload(payload: dict[str, Any]) -> str:
"""
Decide email vs SMS vs inbound STOP for a unified SMTP2GO webhook URL.
Returns one of: ``sms_inbound``, ``sms``, ``email``, ``unknown``.
"""
if not isinstance(payload, dict) or not payload:
return "unknown"
if is_inbound_sms_stop(payload):
return "sms_inbound"
raw = _as_str(payload.get("event")).lower().replace("-", "_").replace(" ", "_")
if not raw:
return "unknown"
# Explicit SMS forms (docs sms_* + API/UI short names that are SMS-only).
if raw.startswith("sms_") or raw in {
"submitted",
"sending",
"opt_out",
"optout",
}:
return "sms"
email_event = _normalize_email_event(raw)
if email_event in {
"processed",
"open",
"click",
"bounce",
"spam",
"unsubscribe",
"resubscribe",
}:
return "email"
has_dest = bool(_as_str(payload.get("destination_number")))
has_rcpt = bool(_as_str(payload.get("rcpt")))
has_email_id = bool(
_as_str(payload.get("email_id") or payload.get("email-id"))
) )
text = str(text).strip().upper() has_monica = bool(extract_monica_message_id(payload))
return text in {"STOP", "UNSUBSCRIBE", "CANCEL", "END", "QUIT"} has_from_address = bool(_as_str(payload.get("from_address")))
has_sms_id = bool(_as_str(payload.get("message_id") or payload.get("sms_id")))
has_sms_body = bool(
_as_str(payload.get("message_content") or payload.get("source_number"))
)
email_leaning = has_rcpt or has_email_id or has_monica or has_from_address
sms_leaning = has_dest or has_sms_body or (
has_sms_id and not email_leaning
)
# Ambiguous short names shared by email + SMS API lists.
if raw in {"delivered", "failed", "rejected", "reject"}:
if sms_leaning and not email_leaning:
return "sms"
if email_leaning:
return "email"
# Bare reject without fields → email docs name; rejected alone → sms lean default
if raw == "reject":
return "email"
if raw == "rejected":
return "sms"
return "email"
if email_leaning:
return "email"
if sms_leaning:
return "sms"
# Default: email (historical primary SMTP2GO traffic).
return "email"
def _pcm_event_type(payload: dict[str, Any]) -> str: def _pcm_event_type(payload: dict[str, Any]) -> str: