26 KiB
Monica Site — Architecture & Design
A Django 6.0 web platform to replace PostcardMania for a single realtor client: a public marketing site with a captcha-protected contact form, a private portal for leads and UTM analytics, a managed mailing list with multi-channel outreach (email, SMS, postcard), and direct social-media automation for Facebook, Instagram, and LinkedIn.
Built to deploy the same way as the existing company_site app: a Docker image using uv,
Postgres in production, and CI-driven deploys through the server-infra Ansible pipeline onto
multiple active/active instances behind Nginx Proxy Manager (NPM).
Repo:
monica_site(mirrors thecompany_sitenaming convention). Single-tenant now, but models and app boundaries are drawn so a futureRealtor/Organizationforeign key can be added without a rewrite.
1. Goals & Non-Goals
Goals
- Public site to market the realtor, with a contact form protected by reCAPTCHA v3.
- Private portal: lead inbox, UTM/campaign analytics, mailing-list management, campaign composer, social scheduler.
- One mailing list of contacts. Form submissions auto-create/append contacts.
- Per-channel, per-contact consent with easy opt-out (email unsubscribe link, SMS
STOP). - Outreach over Email + SMS (SMTP2GO) and Postcard (pluggable provider, default Lob).
- Social posting/scheduling to Facebook, Instagram, LinkedIn via native APIs.
- Cheaper than PostcardMania + adds social automation.
Non-Goals (initial)
- Multi-tenant SaaS (structured for it later, not built now).
- Full CRM (deal pipelines, MLS/IDX listing sync) — future phase.
- In-app billing/subscriptions.
- Building our own print/mail logistics (we integrate a provider).
2. Feature Breakdown (mapped to the request)
- 1. Public site + contact form w/ reCAPTCHA →
publicapp. Reuse the provendjango_recaptchav3 setup fromcompany_site(RECAPTCHA_PUBLIC_KEY/RECAPTCHA_PRIVATE_KEY,django_recaptcha/widget_v3.html). - 2. Private side: leads + UTM analytics →
dashboard(portal shell + auth) +leads(lead records, statuses) +analytics(UTM capture + reporting). - 3. Mailing list + multi-channel outreach + opt-out →
contacts(people, consent) +messaging(channels, templates, campaigns, provider adapters). Email/SMS via SMTP2GO, postcards via a pluggable provider interface. - 4. Social automation (FB/IG/LinkedIn) →
social(connected accounts, posts, scheduling, native connectors).
3. High-Level Architecture
flowchart TB
subgraph edge [Edge]
NPM["Nginx Proxy Manager 10.0.0.230 (TLS + load balance)"]
end
subgraph hosts [App hosts adama + roslin - active/active]
Web1["monica_site web (gunicorn) :PORT"]
Web2["monica_site web (gunicorn) :PORT"]
end
subgraph worker [Background - single host]
TaskWorker["django.tasks worker process"]
DispatchTick["dispatch_due management command / tick"]
end
subgraph data [Shared services 10.0.0.230]
PG[("Postgres - monica_site DB + task queue store")]
end
subgraph external [External APIs]
SMTP2GO["SMTP2GO (email + SMS)"]
Postcard["Postcard provider (Lob default)"]
Meta["Meta Graph API (FB + IG)"]
LinkedIn["LinkedIn API"]
Recaptcha["Google reCAPTCHA"]
end
Visitor(["Public visitor"]) --> NPM
Realtor(["Realtor (portal)"]) --> NPM
NPM --> Web1 & Web2
Web1 & Web2 --> PG
Web1 & Web2 --> Recaptcha
TaskWorker --> PG
DispatchTick --> PG
TaskWorker --> SMTP2GO & Postcard & Meta & LinkedIn
Key points:
- Web tier is stateless and runs on every app host (matches
company_site: active/active behind NPM,SECURE_PROXY_SSL_HEADER,USE_X_FORWARDED_HOST). - Background work uses Django 6.0 Tasks (
django.tasks) — define with@task, enqueue with.enqueue(). Queue store lives in Postgres (same DB as the app). No Redis, no Celery. - Worker is a singleton: Django's Tasks API does not run tasks itself; a separate worker process (from a production-capable third-party backend) claims and executes queued tasks. Exactly one worker host so scheduled sends don't duplicate.
- Shared external Postgres on
10.0.0.230(same pattern ascompany_site).
4. Background jobs — Django Tasks (not Celery/Redis)
Postcard mailing, social publishing, and bulk email/SMS must run outside the request cycle.
Decision
Use Django 6.0's built-in Tasks framework (django.tasks) as the only background API.
| Piece | Choice |
|---|---|
| Define / enqueue | @task + .enqueue() / aenqueue() |
| Dev / tests | ImmediateBackend (or DummyBackend in tests) |
| Production queue | Third-party database-backed Task backend (Postgres queue store) + its worker CLI |
| Deferred / scheduled | Prefer using(run_after=...) when the backend supports defer; otherwise poll due rows via a small dispatch_due management command that enqueues Tasks |
Django guarantees task definition, validation, queuing, and result handling. Execution is always a separate worker process — pick a backend from the Django Tasks community ecosystem / Django Packages grid that provides a durable queue + worker. Do not introduce Redis or Celery for this app.
Pattern
from functools import partial
from django.db import transaction
from django.tasks import task
@task
def send_campaign_message(message_id: int) -> None:
...
# After DB writes commit — workers use a separate connection
with transaction.atomic():
message = Message.objects.create(...)
transaction.on_commit(partial(send_campaign_message.enqueue, message_id=message.pk))
Scheduled social/campaign work:
- Persist
scheduled_foron the model when the realtor schedules. - Either enqueue immediately with
run_after=scheduled_for(if backend supports defer), or runmanage.py dispatch_dueon a short interval (same host as the worker) to find due rows and.enqueue()publish/send tasks. - Worker executes; updates status /
provider_message_id/remote_idon success or failure.
No Redis broker. Task queue + app data share Postgres. One less moving part vs Celery.
5. Django Apps & Responsibilities
| App | Responsibility | Key models |
|---|---|---|
core |
Shared base classes, mixins, utils | TimeStampedModel (created/last_modified, like company_site.TimeInfoBase) |
public |
Public marketing pages, contact form + reCAPTCHA, sitemap/robots/SEO | — (writes Lead + Contact) |
accounts |
Realtor auth, portal login, profile | uses django.contrib.auth; RealtorProfile |
dashboard |
Private portal shell, nav, home widgets | — (read-only aggregates) |
leads |
Lead capture + lifecycle (new/contacted/won/lost), notes | Lead, LeadNote |
contacts |
The mailing list: people + per-channel consent + opt-out | Contact, ConsentRecord, Suppression |
analytics |
UTM capture, attribution, campaign reporting | UTMVisit, Attribution |
messaging |
Templates, campaigns, per-recipient messages, channel adapters | MessageTemplate, Campaign, Message, ProviderEvent |
social |
Connected social accounts, posts, scheduling, native connectors | SocialAccount, SocialPost, SocialPostTarget |
Data model sketch
erDiagram
Contact ||--o{ ConsentRecord : has
Contact ||--o{ Message : receives
Lead }o--|| Contact : "linked to"
Lead ||--o{ LeadNote : has
Lead }o--o| Attribution : "attributed by"
UTMVisit }o--o| Lead : "converts to"
Campaign ||--o{ Message : sends
MessageTemplate ||--o{ Campaign : "used by"
SocialPost ||--o{ SocialPostTarget : "fans out to"
SocialAccount ||--o{ SocialPostTarget : "posted via"
Contact {
uuid id
string email
string phone
string first_name
string last_name
json postal_address
string source
}
ConsentRecord {
string channel "email|sms|postcard"
bool opted_in
datetime changed_at
string reason
}
Message {
string channel
string status "queued|sent|delivered|failed|bounced"
string provider
string provider_message_id
datetime scheduled_for
}
SocialPost {
text body
json media
datetime scheduled_for
string status
}
SocialPostTarget {
string platform "facebook|instagram|linkedin"
string status
string remote_id
}
Consent model (important): consent is per (contact, channel). Every send checks for an
opted-in ConsentRecord and no active Suppression. Opt-out writes an immutable audit trail
(ConsentRecord change + Suppression row). This keeps us defensible on CAN-SPAM/TCPA and lets
a contact opt out of SMS while keeping email.
6. Channel Integrations
6.1 Email + SMS — SMTP2GO
- Email: Django SMTP backend to
mail.smtp2go.com(already thecompany_sitedefault:EMAIL_HOST,EMAIL_PORT=2525, TLS). Every marketing email includes an unsubscribe link (signed token → one-click opt-out view). - SMS: SMTP2GO SMS REST API. Inbound
STOP/UNSUBSCRIBEhandled via SMTP2GO webhook → writes aSuppression. A2P 10DLC registration is a client onboarding prerequisite (call out in Phase 3).
6.2 Postcards — pluggable provider (default Lob)
A small provider interface so the realtor isn't locked in and we can shop on price:
class PostcardProvider(Protocol):
def send_postcard(self, *, to: PostalAddress, from_: PostalAddress,
front: Asset, back: Asset, idempotency_key: str) -> ProviderResult: ...
def get_status(self, provider_id: str) -> DeliveryStatus: ...
Adapters live in messaging/providers/postcard/. Selected via POSTCARD_PROVIDER env var.
| Provider | Model | ~4x6 postcard | Best for | Notes |
|---|---|---|---|---|
| Lob (default) | API-first, tiered | Free dev tier ~$0.77; ~$0.51 on $260/mo Startup | Clean API, address verification, in-transit tracking | Best DX; monthly fee only worth it at volume |
| Click2Mail | Pay-per-piece, no subscription | ~$0.35–0.70 | Low/occasional volume, no monthly fee | API less polished; great when volume is small |
| PostGrid | API + dashboard, subscription | Contact sales (from ~$250/mo) | Compliance-heavy, templates | Overkill unless compliance-driven |
| Stannp | Marketer-friendly, no minimums | Transparent per-piece | Non-dev fallback, EU | Good dashboard |
| USPS EDDM | Postage-only saturation | Postage only | Whole-route blasts (no list) | No per-address API; manual/bulk option |
Recommendation: start on Lob Developer (free) tier to validate, keep Click2Mail as the low-volume cost option behind the same interface. Revisit once monthly volume is known — the abstraction makes switching a config change.
6.3 Social — native APIs (Facebook, Instagram, LinkedIn)
Chosen over an aggregator (Ayrshare) to avoid per-profile monthly fees, since this is one realtor with a fixed set of accounts.
- Facebook + Instagram: Meta Graph API (one integration covers both). IG posting requires an IG Business/Creator account linked to a Facebook Page. Plan for quarterly Graph API version bumps.
- LinkedIn: LinkedIn API (organization + member posting). Budget 2–4 weeks lead time for app review / posting-scope approval — start this early.
- Connectors live in
social/connectors/{meta,linkedin}.pybehind a commonSocialConnectorinterface (publish(post, target) -> remote_id,refresh_token()). - OAuth tokens stored encrypted; due
SocialPosts are enqueued as Django Tasks (run_afterordispatch_due); the task worker fans out to eachSocialPostTargetwith per-platform retry.
sequenceDiagram
participant R as Realtor
participant D as Dashboard
participant DB as Postgres
participant T as django.tasks enqueue
participant W as Task worker
participant P as Platform API
R->>D: Compose post, pick FB/IG/LI, schedule
D->>DB: SocialPost + SocialPostTargets (status=scheduled)
D->>T: enqueue publish with run_after or via dispatch_due
T->>DB: task row in queue store
W->>DB: claim due task
W->>P: publish via connector
P-->>W: remote_id / error
W->>DB: update target status + remote_id
6.4 UTM Analytics
- Landing views capture
utm_source/medium/campaign/term/content+ referrer intoUTMVisit(cookie/session correlation id). - On contact-form submit, the correlation id ties the
UTMVisit→Lead→Attribution. - Dashboard reports leads by source/campaign over time (Chart.js — already vendored in
company_sitestatic; reuse the material-dashboard theme). - Reuse existing Tianji (self-hosted analytics already run at
tianji.aimloperations.com) for pageview-level traffic; ouranalyticsapp owns conversion attribution that Tianji can't tie to CRM records.
7. Proposed Folder / File Layout
Mirrors company_site exactly for deploy compatibility (Dockerfile at repo root, uv,
settings/ package with DJANGO_ENV dispatch, docker-compose.prod.yml with no bundled
db, scripts/docker-entrypoint.sh, .gitea/workflows).
monica_site/ # repo root
├── Dockerfile # python:3.12-slim + uv (copy of company_site pattern)
├── docker-compose.yml # local dev: web + db (+ optional task worker)
├── docker-compose.prod.yml # prod: web on all hosts; task worker on one host; external PG
├── pyproject.toml # uv-managed deps (django==6.0, db task backend, ...)
├── uv.lock
├── .env.example / .env.prod.example # documented env (extends company_site's)
├── .dockerignore / .gitignore
├── .pre-commit-config.yaml
├── README.md
├── docs/
│ └── monica-site-design.md # this document
├── scripts/
│ ├── docker-entrypoint.sh # migrate + collectstatic + gunicorn (web role)
│ ├── worker-entrypoint.sh # django.tasks backend worker (+ optional dispatch_due loop)
│ └── validate-env.sh
├── .gitea/workflows/
│ ├── ci.yml # PR: uv sync + manage.py test
│ ├── unittests.yml # master: tests gate deploy
│ └── deploy.yml # calls server-infra/scripts/deploy.sh --app monica_site
└── monica_site/ # Django project dir (manage.py lives here)
├── manage.py
├── monica_site/ # project package
│ ├── __init__.py
│ ├── settings/
│ │ ├── __init__.py # dispatch on DJANGO_ENV (dev|beta|prod)
│ │ ├── base.py # shared (DATABASE_URL, TASKS, recaptcha, smtp2go)
│ │ ├── dev.py # ImmediateBackend for local
│ │ ├── beta.py
│ │ ├── prod.py # DEBUG off, proxy SSL header, secure cookies, DB task backend
│ │ └── logging.py
│ ├── storage.py # TolerantManifestStaticFilesStorage (reuse)
│ ├── urls.py
│ ├── asgi.py
│ └── wsgi.py
├── core/ # base models/mixins/utils
│ └── management/commands/
│ └── dispatch_due.py # poll scheduled rows → enqueue django.tasks (if needed)
├── public/ # marketing pages + contact form (+ templates/static)
├── accounts/ # realtor auth + RealtorProfile
├── dashboard/ # private portal shell
├── leads/ # Lead, LeadNote
├── contacts/ # Contact, ConsentRecord, Suppression
├── analytics/ # UTMVisit, Attribution + reports
├── messaging/ # templates, campaigns, messages, tasks
│ ├── models.py
│ ├── tasks.py # @task senders (email/sms/postcard batches)
│ ├── channels/ # email.py, sms.py, postcard.py (dispatch)
│ └── providers/
│ ├── email/smtp2go.py
│ ├── sms/smtp2go.py
│ └── postcard/{base.py,lob.py,click2mail.py,postgrid.py}
└── social/ # accounts, posts, scheduling
├── models.py
├── tasks.py # @task publish_social_target, etc.
└── connectors/{base.py,meta.py,linkedin.py}
Each Django app follows the standard admin.py / apps.py / models.py / views.py / urls.py / forms.py / migrations/ / templates/<app>/ / static/<app>/ / tests.py layout used in
company_site (e.g. public/, financial/).
8. Deployment (matches server-infra pipeline)
The existing pipeline already knows how to deploy Dockerized Django apps to active/active hosts.
We plug in by mirroring company_site and adding one catalog entry.
8.1 Container & compose
- Dockerfile: identical pattern to
company_site—python:3.12-slim,uv sync --frozen --no-dev, copy project,scripts/docker-entrypoint.shrunsmigrate → collectstatic → gunicorn monica_site.wsgi:application. docker-compose.prod.yml:webon app hosts (external Postgres viaDATABASE_URL; drop the bundleddb, per the server-infra checklist). Add aworkerservice on exactly one host (ordocker-compose.worker.yml) that runs the chosen django.tasks backend worker. Optionally the same process/host loopsdispatch_duefor schedule polling if the backend lacksrun_afterdefer support.
8.2 server-infra wiring (repo reviewed)
- Add to
inventory/group_vars/all.yml→app_catalog:
monica_site:
type: django
repo: "{{ git_base_url }}/ai_ml_operations/monica_site.git"
default_branch: main
compose_file: docker-compose.prod.yml
web_service: web
migrate_cmd: "uv run python manage.py migrate --noinput"
- Assign host ports in
host_vars/<host>.ymlhost_apps(reserve a new port pair, e.g.prod 8003 / beta 8013, matching across adama+roslin so NPM can balance). - Create control-node secret
~/Documents/secrets/monica_site/monica_site_prod.env(DATABASE_URL,DJANGO_SECRET_KEY,WEB_PORT, provider keys). Never committed; Ansible pushes it to/opt/apps/env/monica_site_prod.env(mode 600) at deploy. - Shared Postgres prereq on
10.0.0.230: createmonica_siteDB + user (andmonica_site_betafor beta), like the other apps. Task queue tables live in the same DB.
8.3 CI/CD flow
Same as company_site:
- PR →
ci.ymlrunsuv sync+manage.py test. - Merge to
main→unittests.yml(containerized tests) → on green,deploy.ymlcallsserver-infra/scripts/deploy.sh --app monica_site --env prod --ref <sha>. deploy-apps.ymlchecks out the ref on each app host, injects.env,docker compose build,up -d, runsmigrateonce.
flowchart LR
PR["PR to main"] --> CI["ci.yml: uv sync + tests"]
CI --> Merge["merge main"]
Merge --> UT["unittests.yml (docker tests)"]
UT -->|green| DEP["deploy.yml -> server-infra deploy.sh"]
DEP --> A["adama: web :8003"]
DEP --> R["roslin: web :8003"]
DEP --> WK["django.tasks worker (single host)"]
A & R --> NPM["NPM balances monica domain"]
8.4 New infra bits beyond company_site
- Task worker placement: exactly one host runs the django.tasks worker (and
dispatch_dueif used). Document this so active/active web replicas do not each run a worker. - No Redis required for background jobs — queue store is Postgres.
9. Environment Variables (extends company_site's set)
# Core (same as company_site)
DJANGO_ENV=prod
DJANGO_DEBUG=false
DJANGO_SECRET_KEY=...
DJANGO_ALLOWED_HOSTS=monicadomain.com,www.monicadomain.com
DATABASE_URL=postgres://monica_site:<pw>@10.0.0.230:5432/monica_site
WEB_PORT=8003
GUNICORN_WORKERS=2
# Django Tasks — prod uses a DB-backed third-party backend + worker.
# Dev defaults to ImmediateBackend in settings/dev.py (no worker needed).
# TASKS is configured in settings, not usually via env; override only if needed.
# reCAPTCHA v3 (same as company_site)
RECAPTCHA_PUBLIC_KEY=...
RECAPTCHA_PRIVATE_KEY=...
# SMTP2GO email (same defaults as company_site)
EMAIL_HOST=mail.smtp2go.com
EMAIL_HOST_USER=...
EMAIL_HOST_PASSWORD=...
EMAIL_PORT=2525
EMAIL_USE_TLS=true
# SMTP2GO SMS
SMTP2GO_SMS_API_KEY=...
# Postcards
POSTCARD_PROVIDER=lob # lob|click2mail|postgrid
LOB_API_KEY=...
# Social (native)
META_APP_ID=...
META_APP_SECRET=...
LINKEDIN_CLIENT_ID=...
LINKEDIN_CLIENT_SECRET=...
SOCIAL_TOKEN_ENCRYPTION_KEY=...
# Analytics (reuse Tianji)
TIANJI_ENABLED=true
TIANJI_WEBSITE_ID=...
10. Phased Implementation Plan
Phase 0 — Scaffolding & deploy skeleton
- Create
monica_siteDjango 6.0 project viauv;settings/dispatch; Dockerfile, compose (dev with db), entrypoints,.gitea/workflows, pre-commit. - Configure
TASKS(ImmediateBackend indev, DB-backed backend inprod); addworker-entrypoint.shfor the task worker. - Add
coreapp (TimeStampedModel), health check, base template. - Wire
server-infraapp_catalogentry + host ports + secrets; get a "hello world" deploying to adama+roslin behind NPM on a staging domain. - Create Postgres DB/user on
10.0.0.230. - Exit: empty app deploys via CI to prod hosts, health check green; worker can claim a no-op task.
Phase 1 — Public site + contact form + leads + UTM
public: marketing pages (home/about/contact),django_recaptchav3 contact form.contacts:Contact+ConsentRecord(default opt-in on submit, with clear notice).leads:Lead+ statuses; form submit creates/updatesContactand aLead.analytics:UTMVisitcapture middleware + attribution on submit.accounts+dashboard: realtor login, portal shell, lead inbox, basic UTM report (Chart.js).- Exit: realtor logs in, sees leads with source attribution; visitors submit safely.
Phase 2 — Mailing list + Email/SMS outreach + opt-out
messaging:MessageTemplate,Campaign,Message,ProviderEvent;@tasksenders online.- Email via SMTP2GO with unsubscribe links; SMS via SMTP2GO SMS API.
- Opt-out views + SMS
STOPwebhook →Suppression; consent enforced on every send. - Dashboard: contact list management, segment/select recipients, compose + schedule campaign,
delivery status (enqueue via
transaction.on_commit+ django.tasks). - Exit: realtor sends a real email + SMS campaign to consented contacts; opt-out works.
Phase 3 — Postcards
messaging/providers/postcard/interface + Lob adapter (+ Click2Mail adapter as the low-volume option);POSTCARD_PROVIDERswitch.- Postcard campaign flow: pick template/artwork, select recipients (address required + Lob address verification), enqueue batch as Django Tasks, poll delivery status.
- Client prerequisites called out: Lob account, A2P 10DLC for SMS.
- Exit: realtor mails a postcard batch and sees delivery tracking.
Phase 4 — Social automation
social:SocialAccountOAuth connect (Meta + LinkedIn), encrypted token storage.- Connectors for Meta (FB Page + IG Business) and LinkedIn;
SocialPost+SocialPostTargetcompose/schedule; enqueue viarun_afterordispatch_due; task worker fans out with retries. - Dashboard: composer with per-platform preview, calendar/scheduled view, post status.
- Kick off LinkedIn app review in Phase 0–1 given 2–4 week approval lead time.
- Exit: realtor schedules one post to all three platforms; it publishes and reports status.
Phase 5 — Hardening & polish
- Rate limiting/backoff per provider, dead-letter handling, admin dashboards for failures.
- Analytics polish (campaign ROI, source trends), email/SMS engagement events.
- Backups for Postgres, monitoring via existing Alloy/Tianji, docs + runbook.
flowchart LR
P0["P0 Scaffold + deploy"] --> P1["P1 Public + leads + UTM"]
P1 --> P2["P2 Mailing list + email/SMS"]
P2 --> P3["P3 Postcards"]
P1 --> P4["P4 Social"]
P3 --> P5["P5 Harden"]
P4 --> P5
11. Key Risks & Decisions
- Singleton task worker: exactly one django.tasks worker across the active/active fleet, or duplicate claims / double-sends are possible depending on backend locking. Pin to one host.
- Django Tasks has no built-in production worker: must pick and pin a third-party DB backend
- worker early in Phase 0; built-in backends are Immediate/Dummy only.
- Scheduling: if the chosen backend lacks
supports_defer/run_after, rely ondispatch_duepolling — keep the tick interval short and idempotent. - LinkedIn approval latency (2–4 wks): start the app-review process at project kickoff.
- SMS compliance (A2P 10DLC/TCPA): client-side registration required before SMS launch;
consent +
STOPhandling are built in but registration is a prerequisite. - Postcard cost/volume: Lob's monthly tiers only pay off at volume; the provider abstraction lets us default to Lob's free tier and drop to Click2Mail (pay-per-piece) if volume stays low.
- Meta Graph versioning: budget quarterly maintenance for API version bumps.
- Single-tenant → multi-tenant later: every tenant-scoped model gets a nullable
owner/realtorFK path reserved so a later migration adds isolation without a rewrite.
12. Cost Snapshot (why it beats PostcardMania)
- Hosting: runs on existing adama/roslin fleet — near-zero marginal infra (Postgres only; no Redis/Celery stack).
- Email/SMS: SMTP2GO usage-based (already an org account).
- Postcards: ~$0.35–0.77/4x6 at provider cost vs PostcardMania's bundled campaign packages; no forced campaign minimums.
- Social: $0 API fees (native), vs PostcardMania add-ons / aggregator per-profile fees.
- No PostcardMania platform/campaign markup; realtor pays provider cost + our hosting.