Unify SMTP2GO email and SMS into one webhook URL.
Classify payloads on /webhooks/smtp2go/ so one SMTP2GO webhook covers both channels under the 10-webhook limit; keep /email/ and /sms/ as aliases and harden SMS event matching.
This commit is contained in:
+2
-2
@@ -42,8 +42,8 @@ DEFAULT_FROM_EMAIL=noreply@mkdrealtor.com
|
||||
# For real SMTP2GO delivery locally, uncomment:
|
||||
# EMAIL_BACKEND=django.core.mail.backends.smtp.EmailBackend
|
||||
SMTP2GO_SMS_API_KEY=
|
||||
# Shared secret for email + SMS webhooks.
|
||||
# In SMTP2GO: Authorization header = Bearer, value = this secret.
|
||||
# Shared secret for the unified SMTP2GO webhook (email + SMS).
|
||||
# In SMTP2GO: one webhook URL …/webhooks/smtp2go/ ; Authorization = Bearer + this secret.
|
||||
SMTP2GO_WEBHOOK_SECRET=
|
||||
|
||||
# Postcards — PCM Integrations (DirectMail API v3)
|
||||
|
||||
+27
-22
@@ -4,8 +4,8 @@ Campaign compose/send, SMTP2GO email + SMS, PCM Integrations postcards, and deli
|
||||
|
||||
## SMTP2GO webhook setup
|
||||
|
||||
Campaign report page polls provider events every 10s. Create **two** webhooks in
|
||||
SMTP2GO → **Settings → Webhooks** (email and SMS stay separate).
|
||||
Campaign report page polls provider events every 10s. Create **one** SMTP2GO
|
||||
webhook (email + SMS share a URL — paid plans cap at 10 webhooks).
|
||||
|
||||
### Auth (`SMTP2GO_WEBHOOK_SECRET`)
|
||||
|
||||
@@ -14,35 +14,39 @@ SMTP2GO → **Settings → Webhooks** (email and SMS stay separate).
|
||||
(do not leave it as “None”).
|
||||
3. Fallback: `?token=<SMTP2GO_WEBHOOK_SECRET>` on the webhook URL also works.
|
||||
|
||||
### Email webhook
|
||||
### Unified email + SMS webhook
|
||||
|
||||
| Field | Value |
|
||||
|-------|--------|
|
||||
| URL | `https://mkdrealtor.com/portal/messaging/webhooks/email/` |
|
||||
| URL | `https://mkdrealtor.com/portal/messaging/webhooks/smtp2go/` |
|
||||
| Authorization header | **Bearer** + `SMTP2GO_WEBHOOK_SECRET` |
|
||||
| Output type | JSON |
|
||||
| Users | email SMTP user(s) **and** the SMS API key (`SMTP2GO_SMS_API_KEY`) |
|
||||
| Email events | processed, bounced, rejected, spam, delivered, unsub/resub, opened, clicked |
|
||||
| Email headers | `X-Monica-Message-Id` |
|
||||
| SMS events | leave unchecked |
|
||||
| SMS events | Submitted, Sending, Delivered, Failed, Rejected (and Opt-out if shown) |
|
||||
|
||||
`X-Monica-Message-Id` is set on every campaign email send and is required so webhook
|
||||
events match the correct recipient row. Invalid / missing header values no longer
|
||||
500 the endpoint (SMTP2GO “Test this webhook” often sends a sample non-UUID).
|
||||
The handler classifies each POST from the payload (`sms_*` / `destination_number`
|
||||
→ SMS; `rcpt` / `email_id` / `X-Monica-Message-Id` → email; inbound `text=STOP`
|
||||
without `event` → SMS opt-out).
|
||||
|
||||
`X-Monica-Message-Id` is set on every campaign email send and is required so email
|
||||
webhook events match the correct recipient row. Invalid / missing header values
|
||||
no longer 500 the endpoint (SMTP2GO “Test this webhook” often sends a sample
|
||||
non-UUID).
|
||||
|
||||
SMS correlation uses `message_id` (SMS id), then `destination_number` phone
|
||||
fallback. Do **not** treat webhook `id` as the SMS id.
|
||||
|
||||
**Opt-out:** SMTP2GO auto-handles replies `STOP` / `UNSUB` / `UNSUBSCRIBE`.
|
||||
This endpoint also accepts inbound POSTs without an `event` field
|
||||
(`text=STOP`, `from=…`) and opts the contact out of SMS. Later sends to
|
||||
opted-out numbers are typically `sms_rejected`.
|
||||
|
||||
Beta / other hosts: swap the hostname, keep the path.
|
||||
|
||||
### SMS webhook (separate)
|
||||
|
||||
| Field | Value |
|
||||
|-------|--------|
|
||||
| URL | `https://mkdrealtor.com/portal/messaging/webhooks/sms/` |
|
||||
| Authorization header | **Bearer** + same `SMTP2GO_WEBHOOK_SECRET` |
|
||||
| Output type | JSON |
|
||||
| Email events | leave unchecked |
|
||||
| SMS events | Submitted, Sending, Delivered, Failed, Rejected, Opt-out |
|
||||
|
||||
This endpoint also accepts inbound reply POSTs (`text=STOP`, `from=…`) and opts the
|
||||
contact out of SMS.
|
||||
Legacy aliases (same handler): `/portal/messaging/webhooks/email/` and
|
||||
`/portal/messaging/webhooks/sms/` — prefer `/smtp2go/` for new configs.
|
||||
|
||||
## PCM Integrations (postcards)
|
||||
|
||||
@@ -119,8 +123,9 @@ PCM_RETURN_ADDRESS={…}
|
||||
|
||||
| Path | Purpose |
|
||||
|------|---------|
|
||||
| `POST /portal/messaging/webhooks/email/` | Email delivery / open / click / bounce / … |
|
||||
| `POST /portal/messaging/webhooks/sms/` | SMS delivery events + inbound STOP |
|
||||
| `POST /portal/messaging/webhooks/smtp2go/` | Unified SMTP2GO email + SMS (+ inbound STOP) |
|
||||
| `POST /portal/messaging/webhooks/email/` | Legacy alias → same as `/smtp2go/` |
|
||||
| `POST /portal/messaging/webhooks/sms/` | Legacy alias → same as `/smtp2go/` |
|
||||
| `POST /portal/messaging/webhooks/postcard/` | PCM order / mail tracking events |
|
||||
| `GET /portal/messaging/campaigns/<id>/status.json` | Live stats for the campaign report UI |
|
||||
| `GET /portal/messaging/postcard/` | PCM designer iframe |
|
||||
|
||||
@@ -37,9 +37,13 @@ def send_sms(message) -> str:
|
||||
)
|
||||
response.raise_for_status()
|
||||
data = response.json() if response.content else {}
|
||||
# SMTP2GO returns varying shapes; store a useful id when present.
|
||||
# Prefer SMS id fields used on webhooks (`message_id` / `sms_id`).
|
||||
nested = data.get("data") if isinstance(data.get("data"), dict) else {}
|
||||
return str(
|
||||
data.get("data", {}).get("sms_id")
|
||||
nested.get("sms_id")
|
||||
or nested.get("message_id")
|
||||
or data.get("sms_id")
|
||||
or data.get("message_id")
|
||||
or data.get("request_id")
|
||||
or f"sms-{message.pk}"
|
||||
)
|
||||
|
||||
@@ -630,6 +630,116 @@ class Smtp2goSmsWebhookTests(TestCase):
|
||||
self.message.refresh_from_db()
|
||||
self.assertEqual(self.message.status, Message.Status.FAILED)
|
||||
|
||||
def test_sms_short_event_name_delivered(self):
|
||||
"""API sms_events use short names; body may omit sms_ prefix."""
|
||||
url = reverse("messaging:sms_webhook")
|
||||
import json
|
||||
|
||||
response = self.client.post(
|
||||
url,
|
||||
data=json.dumps(
|
||||
{
|
||||
"event": "delivered",
|
||||
"message_id": "sms-provider-99",
|
||||
"destination_number": "5550142291",
|
||||
}
|
||||
),
|
||||
content_type="application/json",
|
||||
)
|
||||
self.assertEqual(response.status_code, 200)
|
||||
self.message.refresh_from_db()
|
||||
self.assertEqual(self.message.status, Message.Status.DELIVERED)
|
||||
self.assertTrue(
|
||||
ProviderEvent.objects.filter(
|
||||
message=self.message, event_type="sms_delivered"
|
||||
).exists()
|
||||
)
|
||||
|
||||
def test_sms_ui_label_and_form_encoded(self):
|
||||
url = reverse("messaging:sms_webhook")
|
||||
self.message.status = Message.Status.QUEUED
|
||||
self.message.save(update_fields=["status"])
|
||||
response = self.client.post(
|
||||
url,
|
||||
data={
|
||||
"event": "Submitted",
|
||||
"message_id": "sms-provider-99",
|
||||
"destination_number": "5550142291",
|
||||
},
|
||||
)
|
||||
self.assertEqual(response.status_code, 200)
|
||||
self.message.refresh_from_db()
|
||||
self.assertEqual(self.message.status, Message.Status.SENT)
|
||||
self.assertTrue(
|
||||
ProviderEvent.objects.filter(
|
||||
message=self.message, event_type="sms_submitted"
|
||||
).exists()
|
||||
)
|
||||
|
||||
def test_sms_rejected_marks_failed(self):
|
||||
url = reverse("messaging:sms_webhook")
|
||||
import json
|
||||
|
||||
response = self.client.post(
|
||||
url,
|
||||
data=json.dumps(
|
||||
{
|
||||
"event": "sms_rejected",
|
||||
"message_id": "sms-provider-99",
|
||||
"destination_number": "5550142291",
|
||||
"status_code": "blocked",
|
||||
}
|
||||
),
|
||||
content_type="application/json",
|
||||
)
|
||||
self.assertEqual(response.status_code, 200)
|
||||
self.message.refresh_from_db()
|
||||
self.assertEqual(self.message.status, Message.Status.FAILED)
|
||||
|
||||
def test_sms_webhook_id_is_not_used_as_message_id(self):
|
||||
"""Docs: `id` is the webhook notification id, not the SMS id."""
|
||||
url = reverse("messaging:sms_webhook")
|
||||
import json
|
||||
|
||||
response = self.client.post(
|
||||
url,
|
||||
data=json.dumps(
|
||||
{
|
||||
"event": "sms_delivered",
|
||||
"id": "sms-provider-99",
|
||||
"destination_number": "19999999999",
|
||||
}
|
||||
),
|
||||
content_type="application/json",
|
||||
)
|
||||
self.assertEqual(response.status_code, 200)
|
||||
self.message.refresh_from_db()
|
||||
self.assertEqual(self.message.status, Message.Status.SENT)
|
||||
self.assertTrue(
|
||||
ProviderEvent.objects.filter(
|
||||
message=None, event_type="sms_delivered"
|
||||
).exists()
|
||||
)
|
||||
|
||||
def test_sms_opt_out_unmatched_suppresses_by_phone(self):
|
||||
url = reverse("messaging:sms_webhook")
|
||||
import json
|
||||
|
||||
self.message.delete()
|
||||
response = self.client.post(
|
||||
url,
|
||||
data=json.dumps(
|
||||
{
|
||||
"event": "opt-out",
|
||||
"message_id": "unknown",
|
||||
"destination_number": "+15550142291",
|
||||
}
|
||||
),
|
||||
content_type="application/json",
|
||||
)
|
||||
self.assertEqual(response.status_code, 200)
|
||||
self.assertFalse(contact_may_receive(self.contact, Channel.SMS))
|
||||
|
||||
def test_inbound_stop_still_works(self):
|
||||
url = reverse("messaging:sms_webhook")
|
||||
response = self.client.post(
|
||||
@@ -638,6 +748,14 @@ class Smtp2goSmsWebhookTests(TestCase):
|
||||
self.assertEqual(response.status_code, 200)
|
||||
self.assertFalse(contact_may_receive(self.contact, Channel.SMS))
|
||||
|
||||
def test_inbound_unsub_keyword(self):
|
||||
url = reverse("messaging:sms_webhook")
|
||||
response = self.client.post(
|
||||
url, {"from": "5550142291", "text": "UNSUB"}
|
||||
)
|
||||
self.assertEqual(response.status_code, 200)
|
||||
self.assertFalse(contact_may_receive(self.contact, Channel.SMS))
|
||||
|
||||
def test_sms_webhook_requires_bearer_when_secret_set(self):
|
||||
url = reverse("messaging:sms_webhook")
|
||||
import json
|
||||
@@ -663,6 +781,141 @@ class Smtp2goSmsWebhookTests(TestCase):
|
||||
self.assertEqual(ok.status_code, 200)
|
||||
|
||||
|
||||
class UnifiedSmtp2goWebhookTests(TestCase):
|
||||
def setUp(self):
|
||||
self.client = Client()
|
||||
self.email_contact = Contact.objects.create(
|
||||
email="pat@example.com",
|
||||
first_name="Pat",
|
||||
)
|
||||
set_channel_consent(
|
||||
self.email_contact, Channel.EMAIL, opted_in=True, reason="test"
|
||||
)
|
||||
self.email_campaign = create_campaign_draft(
|
||||
name="Unified email",
|
||||
audience=Campaign.Audience.EMAIL_OPT_IN,
|
||||
subject="Hello",
|
||||
body="Body",
|
||||
)
|
||||
self.email_message = self.email_campaign.messages.get()
|
||||
self.email_message.status = Message.Status.SENT
|
||||
self.email_message.save(update_fields=["status"])
|
||||
|
||||
self.sms_contact = Contact.objects.create(
|
||||
email="sms.pat@example.com",
|
||||
phone="+15550142291",
|
||||
first_name="Pat",
|
||||
)
|
||||
set_channel_consent(
|
||||
self.sms_contact, Channel.SMS, opted_in=True, reason="test"
|
||||
)
|
||||
self.sms_campaign = create_campaign_draft(
|
||||
name="Unified SMS",
|
||||
audience=Campaign.Audience.SMS_OPT_IN,
|
||||
subject="",
|
||||
body="Hi",
|
||||
)
|
||||
self.sms_message = self.sms_campaign.messages.get()
|
||||
self.sms_message.status = Message.Status.SENT
|
||||
self.sms_message.provider_message_id = "sms-unified-1"
|
||||
self.sms_message.save()
|
||||
|
||||
def test_classify_helpers(self):
|
||||
from messaging.webhooks import classify_smtp2go_payload
|
||||
|
||||
self.assertEqual(
|
||||
classify_smtp2go_payload({"from": "555", "text": "STOP"}),
|
||||
"sms_inbound",
|
||||
)
|
||||
self.assertEqual(
|
||||
classify_smtp2go_payload(
|
||||
{"event": "sms_delivered", "message_id": "x"}
|
||||
),
|
||||
"sms",
|
||||
)
|
||||
self.assertEqual(
|
||||
classify_smtp2go_payload(
|
||||
{
|
||||
"event": "delivered",
|
||||
"destination_number": "5550142291",
|
||||
"message_id": "x",
|
||||
}
|
||||
),
|
||||
"sms",
|
||||
)
|
||||
self.assertEqual(
|
||||
classify_smtp2go_payload(
|
||||
{
|
||||
"event": "delivered",
|
||||
"rcpt": "pat@example.com",
|
||||
"email_id": "e1",
|
||||
}
|
||||
),
|
||||
"email",
|
||||
)
|
||||
self.assertEqual(
|
||||
classify_smtp2go_payload({"event": "click", "rcpt": "a@b.c"}),
|
||||
"email",
|
||||
)
|
||||
|
||||
def test_unified_url_routes_email_and_sms(self):
|
||||
import json
|
||||
|
||||
url = reverse("messaging:smtp2go_webhook")
|
||||
email_resp = self.client.post(
|
||||
url,
|
||||
data=json.dumps(
|
||||
{
|
||||
"event": "click",
|
||||
"rcpt": "pat@example.com",
|
||||
"X-Monica-Message-Id": str(self.email_message.pk),
|
||||
}
|
||||
),
|
||||
content_type="application/json",
|
||||
)
|
||||
self.assertEqual(email_resp.status_code, 200)
|
||||
self.assertEqual(email_resp.json()["channel"], "email")
|
||||
self.email_message.refresh_from_db()
|
||||
self.assertEqual(self.email_message.status, Message.Status.CLICKED)
|
||||
|
||||
sms_resp = self.client.post(
|
||||
url,
|
||||
data=json.dumps(
|
||||
{
|
||||
"event": "sms_delivered",
|
||||
"message_id": "sms-unified-1",
|
||||
"destination_number": "5550142291",
|
||||
}
|
||||
),
|
||||
content_type="application/json",
|
||||
)
|
||||
self.assertEqual(sms_resp.status_code, 200)
|
||||
self.assertEqual(sms_resp.json()["channel"], "sms")
|
||||
self.sms_message.refresh_from_db()
|
||||
self.assertEqual(self.sms_message.status, Message.Status.DELIVERED)
|
||||
|
||||
def test_legacy_email_path_still_accepts_sms(self):
|
||||
"""Legacy /email/ alias is the unified handler."""
|
||||
import json
|
||||
|
||||
url = reverse("messaging:email_webhook")
|
||||
response = self.client.post(
|
||||
url,
|
||||
data=json.dumps(
|
||||
{
|
||||
"event": "delivered",
|
||||
"message_id": "sms-unified-1",
|
||||
"destination_number": "5550142291",
|
||||
}
|
||||
),
|
||||
content_type="application/json",
|
||||
)
|
||||
self.assertEqual(response.status_code, 200)
|
||||
self.assertEqual(response.json()["channel"], "sms")
|
||||
self.sms_message.refresh_from_db()
|
||||
self.assertEqual(self.sms_message.status, Message.Status.DELIVERED)
|
||||
|
||||
|
||||
class PcmPostcardWebhookTests(TestCase):
|
||||
def setUp(self):
|
||||
self.client = Client()
|
||||
|
||||
@@ -44,6 +44,8 @@ urlpatterns = [
|
||||
views.postcard_design_save,
|
||||
name="postcard_design_save",
|
||||
),
|
||||
path("webhooks/smtp2go/", views.smtp2go_webhook, name="smtp2go_webhook"),
|
||||
# Legacy aliases — same unified SMTP2GO handler.
|
||||
path("webhooks/sms/", views.sms_webhook, name="sms_webhook"),
|
||||
path("webhooks/email/", views.email_webhook, name="email_webhook"),
|
||||
path(
|
||||
|
||||
+50
-52
@@ -39,7 +39,7 @@ from messaging.webhooks import (
|
||||
PROVIDER_PCM,
|
||||
PROVIDER_SMS,
|
||||
campaign_engagement_stats,
|
||||
is_inbound_sms_stop,
|
||||
classify_smtp2go_payload,
|
||||
parse_webhook_payload,
|
||||
process_pcm_postcard_webhook,
|
||||
process_smtp2go_email_webhook,
|
||||
@@ -248,13 +248,13 @@ def _events_provider_filter(campaign: Campaign) -> tuple[list[str], str, str]:
|
||||
[PROVIDER_SMS],
|
||||
"Recent SMTP2GO events",
|
||||
"No webhook events yet. SMTP2GO must POST SMS events to "
|
||||
"<code>/portal/messaging/webhooks/sms/</code>.",
|
||||
"<code>/portal/messaging/webhooks/smtp2go/</code>.",
|
||||
)
|
||||
return (
|
||||
[PROVIDER_EMAIL],
|
||||
"Recent SMTP2GO events",
|
||||
"No webhook events yet. SMTP2GO must POST opens/clicks to "
|
||||
"<code>/portal/messaging/webhooks/email/</code> "
|
||||
"<code>/portal/messaging/webhooks/smtp2go/</code> "
|
||||
"(see messaging README). SMTP2GO’s own “Clicked” feed does not fill "
|
||||
"this table by itself.",
|
||||
)
|
||||
@@ -863,33 +863,44 @@ def postcard_webhook(request):
|
||||
|
||||
@csrf_exempt
|
||||
@require_POST
|
||||
def sms_webhook(request):
|
||||
def smtp2go_webhook(request):
|
||||
"""
|
||||
SMTP2GO SMS webhook — delivery status events + inbound STOP replies.
|
||||
Unified SMTP2GO webhook — email + SMS events + inbound STOP replies.
|
||||
|
||||
Configure a *separate* webhook in SMTP2GO → Settings → Webhooks:
|
||||
URL: https://<host>/portal/messaging/webhooks/sms/
|
||||
One SMTP2GO webhook URL (paid plans cap at 10 webhooks):
|
||||
|
||||
URL: https://<host>/portal/messaging/webhooks/smtp2go/
|
||||
Authorization header: Bearer + value = SMTP2GO_WEBHOOK_SECRET
|
||||
Output type: JSON
|
||||
SMS events: Submitted, Sending, Delivered, Failed, Rejected, Opt-out
|
||||
(leave Email events unchecked on this webhook)
|
||||
Users: email SMTP user(s) *and* the SMS API key used to send
|
||||
Email events: all delivery/engagement boxes
|
||||
Email headers: X-Monica-Message-Id
|
||||
SMS events: Submitted, Sending, Delivered, Failed, Rejected (Opt-out if shown)
|
||||
|
||||
Inbound gateway POSTs without ``event`` (text=STOP, from=…) still opt out.
|
||||
Legacy aliases ``/webhooks/email/`` and ``/webhooks/sms/`` hit this same view.
|
||||
Payload shape selects the processor (email vs sms_* vs inbound STOP).
|
||||
"""
|
||||
channel = "sms"
|
||||
_log_webhook_request(request, channel=channel)
|
||||
# Log/auth before parsing so request.body stays readable (HMAC + Grafana dump).
|
||||
_log_webhook_request(request, channel="smtp2go")
|
||||
if not _webhook_authorized(
|
||||
request, secret=settings.SMTP2GO_WEBHOOK_SECRET or ""
|
||||
):
|
||||
_log_webhook_auth_failed(request, channel=channel)
|
||||
_log_webhook_auth_failed(request, channel="smtp2go")
|
||||
return HttpResponseForbidden("invalid webhook token")
|
||||
|
||||
payload = parse_webhook_payload(request)
|
||||
if not payload:
|
||||
payload = request.POST.dict() or {}
|
||||
|
||||
# Inbound reply (STOP) — different payload shape than delivery events.
|
||||
if is_inbound_sms_stop(payload):
|
||||
kind = classify_smtp2go_payload(payload)
|
||||
channel = {
|
||||
"sms_inbound": "sms",
|
||||
"sms": "sms",
|
||||
"email": "email",
|
||||
}.get(kind, "smtp2go")
|
||||
logger.info("webhook_classified channel=%s kind=%s", channel, kind)
|
||||
|
||||
if kind == "sms_inbound":
|
||||
phone = (
|
||||
payload.get("from")
|
||||
or payload.get("phone")
|
||||
@@ -904,56 +915,43 @@ def sms_webhook(request):
|
||||
stopped,
|
||||
phone,
|
||||
)
|
||||
return JsonResponse({"ok": True, "opt_out": stopped})
|
||||
return JsonResponse({"ok": True, "channel": "sms", "opt_out": stopped})
|
||||
|
||||
try:
|
||||
event = process_smtp2go_sms_webhook(payload)
|
||||
except Exception as exc: # noqa: BLE001
|
||||
logger.exception("SMTP2GO SMS webhook processing failed")
|
||||
_log_webhook_result(channel=channel, error=str(exc))
|
||||
return JsonResponse({"ok": False, "error": "processing_failed"}, status=200)
|
||||
_log_webhook_result(channel=channel, event=event)
|
||||
if kind == "unknown":
|
||||
logger.warning(
|
||||
"webhook_unmatched channel=smtp2go unrecognized_payload keys=%s",
|
||||
sorted(str(k) for k in payload.keys()),
|
||||
)
|
||||
return JsonResponse(
|
||||
{
|
||||
"ok": True,
|
||||
"matched": bool(event and event.message_id),
|
||||
"event_id": event.pk if event else None,
|
||||
}
|
||||
{"ok": False, "error": "unrecognized_payload", "channel": None},
|
||||
status=200,
|
||||
)
|
||||
|
||||
|
||||
@csrf_exempt
|
||||
@require_POST
|
||||
def email_webhook(request):
|
||||
"""
|
||||
SMTP2GO email event webhook (delivered / open / click / bounce / …).
|
||||
|
||||
Configure in SMTP2GO → Settings → Webhooks:
|
||||
URL: https://<host>/portal/messaging/webhooks/email/
|
||||
Authorization header: Bearer + value = SMTP2GO_WEBHOOK_SECRET
|
||||
Output type: JSON
|
||||
Email events: all delivery/engagement boxes
|
||||
Email headers: X-Monica-Message-Id
|
||||
"""
|
||||
channel = "email"
|
||||
_log_webhook_request(request, channel=channel)
|
||||
if not _webhook_authorized(
|
||||
request, secret=settings.SMTP2GO_WEBHOOK_SECRET or ""
|
||||
):
|
||||
_log_webhook_auth_failed(request, channel=channel)
|
||||
return HttpResponseForbidden("invalid webhook token")
|
||||
payload = parse_webhook_payload(request)
|
||||
try:
|
||||
if kind == "sms":
|
||||
event = process_smtp2go_sms_webhook(payload)
|
||||
else:
|
||||
event = process_smtp2go_email_webhook(payload)
|
||||
except Exception as exc: # noqa: BLE001 — never 500 SMTP2GO (they retry for 48h)
|
||||
logger.exception("SMTP2GO email webhook processing failed")
|
||||
logger.exception("SMTP2GO %s webhook processing failed", kind)
|
||||
_log_webhook_result(channel=channel, error=str(exc))
|
||||
return JsonResponse({"ok": False, "error": "processing_failed"}, status=200)
|
||||
return JsonResponse(
|
||||
{"ok": False, "error": "processing_failed", "channel": channel},
|
||||
status=200,
|
||||
)
|
||||
|
||||
_log_webhook_result(channel=channel, event=event)
|
||||
return JsonResponse(
|
||||
{
|
||||
"ok": True,
|
||||
"channel": channel,
|
||||
"matched": bool(event and event.message_id),
|
||||
"event_id": event.pk if event else None,
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
# Legacy path names — same unified handler (keep SMTP2GO configs working).
|
||||
email_webhook = smtp2go_webhook
|
||||
sms_webhook = smtp2go_webhook
|
||||
|
||||
|
||||
+128
-33
@@ -68,6 +68,21 @@ _EMAIL_EVENT_ALIASES = {
|
||||
"resubscribed": "resubscribe",
|
||||
}
|
||||
|
||||
# API sms_events use short names (delivered); webhook body often uses sms_delivered.
|
||||
# UI test labels / Opt-Out may arrive without the sms_ prefix.
|
||||
_SMS_EVENT_ALIASES = {
|
||||
"submitted": "sms_submitted",
|
||||
"sending": "sms_sending",
|
||||
"delivered": "sms_delivered",
|
||||
"failed": "sms_failed",
|
||||
"rejected": "sms_rejected",
|
||||
"opt_out": "sms_opt_out",
|
||||
"optout": "sms_opt_out",
|
||||
"sms_optout": "sms_opt_out",
|
||||
}
|
||||
|
||||
_SMS_OPT_OUT_EVENTS = frozenset({"sms_opt_out"})
|
||||
|
||||
|
||||
def _as_str(value: Any) -> str:
|
||||
"""Coerce webhook field values to a stripped string (lists / None safe)."""
|
||||
@@ -98,6 +113,12 @@ def _normalize_email_event(event: str) -> str:
|
||||
return _EMAIL_EVENT_ALIASES.get(event, event)
|
||||
|
||||
|
||||
def _normalize_sms_event(event: str) -> str:
|
||||
"""Map UI / short API names to docs canonical sms_* event strings."""
|
||||
event = (event or "").strip().lower().replace("-", "_").replace(" ", "_")
|
||||
return _SMS_EVENT_ALIASES.get(event, event)
|
||||
|
||||
|
||||
def parse_webhook_payload(request: HttpRequest) -> dict[str, Any]:
|
||||
"""Accept JSON or form-encoded SMTP2GO webhook bodies."""
|
||||
content_type = (request.content_type or "").lower()
|
||||
@@ -362,14 +383,13 @@ def normalize_phone(value: str) -> str:
|
||||
return "".join(ch for ch in (value or "") if ch.isdigit())
|
||||
|
||||
|
||||
def _sms_provider_message_id(payload: dict[str, Any]) -> str:
|
||||
"""SMS unique id from docs (`message_id`). Never use webhook `id`."""
|
||||
return _as_str(payload.get("message_id") or payload.get("sms_id"))
|
||||
|
||||
|
||||
def find_message_for_sms_event(payload: dict[str, Any]) -> Message | None:
|
||||
provider_id = (
|
||||
payload.get("message_id")
|
||||
or payload.get("sms_id")
|
||||
or payload.get("id")
|
||||
or ""
|
||||
)
|
||||
provider_id = str(provider_id).strip()
|
||||
provider_id = _sms_provider_message_id(payload)
|
||||
if provider_id:
|
||||
message = (
|
||||
Message.objects.select_related("contact", "campaign")
|
||||
@@ -379,14 +399,15 @@ def find_message_for_sms_event(payload: dict[str, Any]) -> Message | None:
|
||||
if message:
|
||||
return message
|
||||
|
||||
# Outbound delivery events use destination_number (recipient).
|
||||
# Do not use source_number / from — those are the pool or inbound reply.
|
||||
raw_phone = (
|
||||
payload.get("destination_number")
|
||||
or payload.get("to")
|
||||
or payload.get("phone")
|
||||
or payload.get("from")
|
||||
or ""
|
||||
)
|
||||
digits = normalize_phone(str(raw_phone))
|
||||
digits = normalize_phone(_as_str(raw_phone))
|
||||
if len(digits) < 7:
|
||||
return None
|
||||
|
||||
@@ -411,6 +432,7 @@ def find_message_for_sms_event(payload: dict[str, Any]) -> Message | None:
|
||||
Message.Status.SENT,
|
||||
Message.Status.DELIVERED,
|
||||
Message.Status.FAILED,
|
||||
Message.Status.SUPPRESSED,
|
||||
],
|
||||
)
|
||||
.order_by("-sent_at", "-updated_at")
|
||||
@@ -419,19 +441,14 @@ def find_message_for_sms_event(payload: dict[str, Any]) -> Message | None:
|
||||
|
||||
|
||||
def _apply_sms_event(message: Message, event: str, payload: dict[str, Any]) -> None:
|
||||
event = (event or "").strip().lower().replace("-", "_")
|
||||
err = (
|
||||
event = _normalize_sms_event(event)
|
||||
err = _as_str(
|
||||
payload.get("message")
|
||||
or payload.get("status_code")
|
||||
or payload.get("context")
|
||||
or ""
|
||||
)
|
||||
err = str(err).strip()
|
||||
|
||||
provider_id = (
|
||||
payload.get("message_id") or payload.get("sms_id") or ""
|
||||
)
|
||||
provider_id = str(provider_id).strip()
|
||||
provider_id = _sms_provider_message_id(payload)
|
||||
if provider_id and message.provider_message_id != provider_id:
|
||||
message.provider_message_id = provider_id
|
||||
message.provider = PROVIDER_SMS
|
||||
@@ -439,16 +456,16 @@ def _apply_sms_event(message: Message, event: str, payload: dict[str, Any]) -> N
|
||||
update_fields=["provider_message_id", "provider", "updated_at"]
|
||||
)
|
||||
|
||||
if event in {"sms_sending", "sending", "sms_submitted", "submitted"}:
|
||||
if event in {"sms_sending", "sms_submitted"}:
|
||||
if message.status in {Message.Status.QUEUED, Message.Status.DRAFT}:
|
||||
_maybe_upgrade_status(message, Message.Status.SENT)
|
||||
return
|
||||
|
||||
if event in {"sms_delivered", "delivered"}:
|
||||
if event == "sms_delivered":
|
||||
_maybe_upgrade_status(message, Message.Status.DELIVERED)
|
||||
return
|
||||
|
||||
if event in {"sms_failed", "failed", "sms_rejected", "rejected"}:
|
||||
if event in {"sms_failed", "sms_rejected"}:
|
||||
_maybe_upgrade_status(
|
||||
message,
|
||||
Message.Status.FAILED,
|
||||
@@ -456,7 +473,7 @@ def _apply_sms_event(message: Message, event: str, payload: dict[str, Any]) -> N
|
||||
)
|
||||
return
|
||||
|
||||
if event in {"sms_opt_out", "opt_out", "optout"}:
|
||||
if event in _SMS_OPT_OUT_EVENTS:
|
||||
_maybe_upgrade_status(
|
||||
message, Message.Status.SUPPRESSED, error="sms opt-out"
|
||||
)
|
||||
@@ -471,7 +488,7 @@ def _apply_sms_event(message: Message, event: str, payload: dict[str, Any]) -> N
|
||||
|
||||
def process_smtp2go_sms_webhook(payload: dict[str, Any]) -> ProviderEvent | None:
|
||||
"""Persist SMS delivery/opt-out ProviderEvent and update Message when matched."""
|
||||
event = (payload.get("event") or "").strip().lower()
|
||||
event = _normalize_sms_event(_as_str(payload.get("event")))
|
||||
if not event:
|
||||
logger.warning("SMTP2GO SMS webhook missing event: %s", payload)
|
||||
return None
|
||||
@@ -482,17 +499,16 @@ def process_smtp2go_sms_webhook(payload: dict[str, Any]) -> ProviderEvent | None
|
||||
message.refresh_from_db()
|
||||
else:
|
||||
# Opt-out with no matched campaign message still suppresses by phone.
|
||||
if event.replace("-", "_") in {"sms_opt_out", "opt_out", "optout"}:
|
||||
phone = (
|
||||
if event in _SMS_OPT_OUT_EVENTS:
|
||||
phone = _as_str(
|
||||
payload.get("destination_number")
|
||||
or payload.get("from")
|
||||
or payload.get("source_number")
|
||||
or ""
|
||||
)
|
||||
if phone:
|
||||
from messaging.services import record_sms_stop
|
||||
|
||||
record_sms_stop(str(phone))
|
||||
record_sms_stop(phone)
|
||||
logger.info(
|
||||
"SMTP2GO SMS webhook unmatched event=%s phone=%s message_id=%s",
|
||||
event,
|
||||
@@ -503,23 +519,102 @@ def process_smtp2go_sms_webhook(payload: dict[str, Any]) -> ProviderEvent | None
|
||||
return ProviderEvent.objects.create(
|
||||
message=message,
|
||||
provider=PROVIDER_SMS,
|
||||
event_type=event,
|
||||
payload=payload,
|
||||
event_type=event[:64],
|
||||
payload=_json_safe(payload) if isinstance(payload, dict) else {},
|
||||
)
|
||||
|
||||
|
||||
def is_inbound_sms_stop(payload: dict[str, Any]) -> bool:
|
||||
"""True for gateway-style inbound reply payloads (STOP / UNSUBSCRIBE)."""
|
||||
"""True for gateway-style inbound reply payloads (STOP / UNSUBSCRIBE).
|
||||
|
||||
SMTP2GO auto-handles STOP/UNSUB/UNSUBSCRIBE replies; this catches the
|
||||
inbound gateway POST shape (no ``event`` field) when configured to
|
||||
forward replies. Prefer this over relying on a webhook Opt-Out event —
|
||||
the API ``sms_events`` list does not include opt-out.
|
||||
"""
|
||||
if payload.get("event"):
|
||||
return False
|
||||
text = (
|
||||
text = _as_str(
|
||||
payload.get("text")
|
||||
or payload.get("message")
|
||||
or payload.get("message_content")
|
||||
or ""
|
||||
).upper()
|
||||
return text in {"STOP", "UNSUBSCRIBE", "UNSUB", "CANCEL", "END", "QUIT"}
|
||||
|
||||
|
||||
def classify_smtp2go_payload(payload: dict[str, Any]) -> str:
|
||||
"""
|
||||
Decide email vs SMS vs inbound STOP for a unified SMTP2GO webhook URL.
|
||||
|
||||
Returns one of: ``sms_inbound``, ``sms``, ``email``, ``unknown``.
|
||||
"""
|
||||
if not isinstance(payload, dict) or not payload:
|
||||
return "unknown"
|
||||
|
||||
if is_inbound_sms_stop(payload):
|
||||
return "sms_inbound"
|
||||
|
||||
raw = _as_str(payload.get("event")).lower().replace("-", "_").replace(" ", "_")
|
||||
if not raw:
|
||||
return "unknown"
|
||||
|
||||
# Explicit SMS forms (docs sms_* + API/UI short names that are SMS-only).
|
||||
if raw.startswith("sms_") or raw in {
|
||||
"submitted",
|
||||
"sending",
|
||||
"opt_out",
|
||||
"optout",
|
||||
}:
|
||||
return "sms"
|
||||
|
||||
email_event = _normalize_email_event(raw)
|
||||
if email_event in {
|
||||
"processed",
|
||||
"open",
|
||||
"click",
|
||||
"bounce",
|
||||
"spam",
|
||||
"unsubscribe",
|
||||
"resubscribe",
|
||||
}:
|
||||
return "email"
|
||||
|
||||
has_dest = bool(_as_str(payload.get("destination_number")))
|
||||
has_rcpt = bool(_as_str(payload.get("rcpt")))
|
||||
has_email_id = bool(
|
||||
_as_str(payload.get("email_id") or payload.get("email-id"))
|
||||
)
|
||||
text = str(text).strip().upper()
|
||||
return text in {"STOP", "UNSUBSCRIBE", "CANCEL", "END", "QUIT"}
|
||||
has_monica = bool(extract_monica_message_id(payload))
|
||||
has_from_address = bool(_as_str(payload.get("from_address")))
|
||||
has_sms_id = bool(_as_str(payload.get("message_id") or payload.get("sms_id")))
|
||||
has_sms_body = bool(
|
||||
_as_str(payload.get("message_content") or payload.get("source_number"))
|
||||
)
|
||||
|
||||
email_leaning = has_rcpt or has_email_id or has_monica or has_from_address
|
||||
sms_leaning = has_dest or has_sms_body or (
|
||||
has_sms_id and not email_leaning
|
||||
)
|
||||
|
||||
# Ambiguous short names shared by email + SMS API lists.
|
||||
if raw in {"delivered", "failed", "rejected", "reject"}:
|
||||
if sms_leaning and not email_leaning:
|
||||
return "sms"
|
||||
if email_leaning:
|
||||
return "email"
|
||||
# Bare reject without fields → email docs name; rejected alone → sms lean default
|
||||
if raw == "reject":
|
||||
return "email"
|
||||
if raw == "rejected":
|
||||
return "sms"
|
||||
return "email"
|
||||
|
||||
if email_leaning:
|
||||
return "email"
|
||||
if sms_leaning:
|
||||
return "sms"
|
||||
# Default: email (historical primary SMTP2GO traffic).
|
||||
return "email"
|
||||
|
||||
|
||||
def _pcm_event_type(payload: dict[str, Any]) -> str:
|
||||
|
||||
Reference in New Issue
Block a user