url_shortening_service/Implementation.md

# URL Shortening Service — Implementation

Django 6 service in Docker, packaged with `uv`. First caller is `monica_site` (SMS campaign
bodies and ad-hoc texts). A short public domain will be pointed at this service later;
until then a subdomain (for example `go.mkdrealtor.com`) is enough.

This document is the build spec. Implement in this repo. Do not fold the shortener into
`monica_site`.

---

## 1. Goals & non-goals

### Goals

- Mint a short HTTPS URL that 302s to a long HTTPS URL.
- Public `GET /<code>` so SMS recipients can tap the link with no token.
- Create / list / disable **only** via a Bearer-authenticated API (own hostname; may be public DNS).
- Named, rotatable tokens so `monica_site` can be revoked without rotating every caller.
- Target-host allowlist so a stolen token cannot mint open redirects off the short domain.
- Click counts for campaign reporting.
- Same deploy shape as `monica_site`: `uv`, Docker, Postgres in prod, Gitea CI, `server-infra`.

### Non-goals (v1)

- Public web UI or Django admin exposed on the short domain.
- Query-string tokens (`?token=`). Header only.
- Per-recipient unique codes (who clicked). Add later if needed.
- Custom vanity slugs (`/openhouse`). Random codes only in v1.
- Multi-tenant SaaS, rate-limit dashboards, QR generation.

---

## 2. Security model (do this first)

Two surfaces. Mixing them is the failure mode.

| Surface | Who | Auth | Network |
|---------|-----|------|---------|
| `GET /<code>` | anyone with the SMS | **none** | public short hostname (NPM + TLS) |
| `/api/links/` (all methods) | `monica_site` and other trusted callers | **Bearer** | own hostname — **may be public DNS**. Token is the lock. |

### 2.1 Public redirect

SMS recipients have no token. Do **not** put Bearer on `GET /<code>`. That path is the
product.

Use **302**, not 301. 301 is cached forever and later click counts (and disable) stop
working.

### 2.2 Authenticated API

- `Authorization: Bearer <named-token>` on every `/api/` request.
- If `SHORTENER_API_TOKENS` is empty, API returns **503**. Fail closed.
- Constant-time compare (`hmac.compare_digest`) against every configured token.
- Named tokens: `monica:<secret>`, `admin:<secret>`. Store the name on the `ShortLink`
  row as `created_by_token`.
- Generate secrets with `python -c "import secrets; print(secrets.token_urlsafe(32))"`.
- Never put the token in the short URL, logs, or git. Do not reuse
  `DJANGO_SECRET_KEY` or any `monica_site` webhook secret.
- Do not log the `Authorization` header (Gunicorn/NPM access logs). Strip or disable it.

### 2.3 Two public hostnames (Bearer is the lock on /api/)

`/api/links/` may have its own public DNS record. Anyone who finds that URL still
cannot mint links without a configured Bearer token. Empty `SHORTENER_API_TOKENS`
→ **503**. Missing/wrong token → **401**. Do not add CORS `*` (server-to-server only).

```
Internet
   │
   ├─ NPM: SHORT_DOMAIN (mkd.to / go.mkdrealtor.com)
   │    location ~*^/[a-z0-9]{4,8}$  →  gunicorn   (redirects only)
   │    /api/*                       →  404 / drop
   │    /admin/                      →  404 / drop
   │
   └─ NPM: API hostname (e.g. shortener.aimloperations.com)
        /api/links/  →  gunicorn   (Bearer required)
        /admin/      →  404 / drop
```

- Public NPM for `SHORT_DOMAIN` must not proxy `/api/` or `/admin/`.
- Public NPM for the API hostname proxies `/api/` only. Do not expose `/admin/`.
- `monica_site` calls `SHORTENER_BASE_URL` (the API hostname), never the short
  hostname, to create links.
- Django admin stays on `SHORT_ADMIN_HOSTS` (localhost). Not on the public API host.

### 2.4 Target allowlist

A valid Bearer still must not mint arbitrary redirects.

On create:

- `https` only (reject `http`, `javascript:`, `data:`, protocol-relative `//evil.com`).
- Hostname must match `SHORT_ALLOWED_HOSTS` (exact or suffix, e.g. `mkdrealtor.com`).
- Normalize / reject credentials in the URL (`https://user:pass@host`).

### 2.5 Host routing inside the app

Even if NPM is misconfigured, the Django process must refuse the wrong surface:

- `request.get_host()` in `SHORT_PUBLIC_HOSTS` → redirect view only. `/api/` → 404.
- `request.get_host()` in `SHORT_API_HOSTS` → `/api/` (still Bearer). May be public DNS.
- `request.get_host()` in `SHORT_ADMIN_HOSTS` → `/admin/` (localhost only by default).
- `/healthz/` allowed on both public hosts. No secrets in the body.

---

## 3. Architecture

```
Phone SMS:  https://mkd.to/a3k9
                │
                ▼
         Nginx Proxy Manager (TLS)
                │
                ▼
     url_shortening_service (this repo)
     Docker: gunicorn + uv
                │
         Host header?
           ├─ public short host  →  GET /<code> 302 + Click row
           └─ API host (own DNS) →  /api/links/ (Bearer)
                │
                ▼
         Postgres (own DATABASE_URL)
```

Caller (`monica_site`) is an HTTP client. It does not share this database.

Until the purchased short domain exists:

- Set `SHORT_DOMAIN=go.mkdrealtor.com` (or `localhost:8005` locally).
- Codes do not change when DNS is swapped. Only `SHORT_DOMAIN` / `PUBLIC_SHORT_URL` change.

---

## 4. Stack

Mirror `monica_site` so deploy muscle memory is the same.

| Piece | Choice |
|-------|--------|
| Python | 3.12 |
| Package manager | `uv` (`pyproject.toml` + committed `uv.lock`) |
| Framework | Django 6 |
| WSGI | gunicorn in prod/beta; `runserver` when `DJANGO_ENV=dev` |
| Static | WhiteNoise (almost unused; keep for admin if enabled internally) |
| DB | Postgres via `DATABASE_URL`; SQLite when unset (tests / local) |
| Image | `python:3.12-slim` + copy `uv` from `ghcr.io/astral-sh/uv` |
| Queue | none in v1 (no worker) |
| Extra deps | none for codes (`secrets`); `psycopg[binary]`, `gunicorn`, `whitenoise` |

No Redis, Celery, or frontend build.

---

## 5. Project layout

```
url_shortening_service/
  Implementation.md          ← this file
  README.md
  pyproject.toml
  uv.lock
  Dockerfile
  docker-compose.yml          # web + postgres, hot reload
  docker-compose.prod.yml    # no bundled postgres; DATABASE_URL from env
  .env.example
  .env.prod.example
  .gitignore
  .gitea/workflows/ci.yml
  scripts/docker-entrypoint.sh
  site/
    manage.py
    shortener/               # Django project package
      settings/
        __init__.py
        base.py
        dev.py
        beta.py
        prod.py
      urls.py
      wsgi.py
      asgi.py
    links/                   # the app
      models.py
      views.py
      api.py
      auth.py
      services.py
      urls.py
      tests.py
      admin.py               # optional; only reachable on API host if enabled
    core/
      views.py               # healthz
```

Settings split by `DJANGO_ENV` like `monica_site`.

---

## 6. Data model

### `ShortLink`

| Field | Type | Notes |
|-------|------|-------|
| `id` | UUID PK | |
| `code` | `CharField(8)`, unique, indexed | 4–6 chars from alphabet below |
| `target_url` | `URLField(2048)` | stored canonical https URL |
| `title` | `CharField(200)`, blank | caller-supplied note |
| `created_by_token` | `CharField(64)` | token **name** (`monica`), never the secret |
| `external_ref` | `CharField(64)`, blank, indexed | optional caller id (campaign UUID) |
| `is_active` | bool, default True | |
| `expires_at` | datetime, nullable | |
| `click_count` | positive int, default 0 | denormalized |
| `created_at` / `updated_at` | timestamps | |

Disable without deleting. Inactive or expired codes 404 (no redirect). Do not retarget an
existing code to a new URL in v1 (prevents swapping a live SMS onto a new destination).

### `Click`

| Field | Type | Notes |
|-------|------|-------|
| `id` | UUID PK | |
| `link` | FK `ShortLink` | |
| `occurred_at` | datetime | |
| `ip_hash` | `CharField(64)`, blank | HMAC-SHA256 of remote IP with a server pepper; never store raw IP |
| `user_agent` | `CharField(512)`, blank | truncated |
| `referrer` | `CharField(1024)`, blank | |

Increment `ShortLink.click_count` in the same request as inserting `Click` (or
`F('click_count') + 1`). Do not block the 302 on analytics failure: log and still redirect
if the click insert fails.

### Code alphabet

```
23456789abcdefghjkmnpqrstuvwxyz
```

No `0/O`, `1/l/I`. Length 6 by default (`SHORT_CODE_LENGTH=6`). Collision: retry a small
number of times, then 500.

Do not accept caller-supplied codes in v1.

---

## 7. HTTP API (internal host only)

All `/api/` routes require Bearer. JSON in/out. CSRF exempt (token auth, no cookies).

Prefix: `/api/links/`

### `POST /api/links/`

Create.

Request:

```json
{
  "target_url": "https://mkdrealtor.com/listings/oak-st?utm_source=monica&utm_medium=sms",
  "title": "Oak St listing",
  "external_ref": "campaign-uuid-optional",
  "expires_at": null
}
```

Response `201`:

```json
{
  "code": "a3k9xm",
  "short_url": "https://go.mkdrealtor.com/a3k9xm",
  "target_url": "https://mkdrealtor.com/listings/oak-st?utm_source=monica&utm_medium=sms",
  "title": "Oak St listing",
  "is_active": true,
  "click_count": 0,
  "created_at": "2026-08-30T10:00:00Z"
}
```

`short_url` is built from `PUBLIC_SHORT_URL` (the public origin phones will hit), **not**
from the internal `Host` header.

Errors:

- `400` invalid URL / host not allowlisted
- `401` missing/wrong Bearer
- `503` no tokens configured

Idempotency (v1, keep simple): if the same `target_url` + `external_ref` + still-active
link exists, return that row `200` instead of minting a duplicate. If `external_ref` is
empty, always mint a new code.

### `GET /api/links/`

List. Query: `?external_ref=`, `?is_active=true`, pagination (`limit`/`offset`, cap 100).

### `GET /api/links/<code>/`

Detail including `click_count`.

### `POST /api/links/<code>/disable/`

Set `is_active=false`. Idempotent. `200`.

No DELETE in v1. No PATCH of `target_url`.

### Auth helper

```python
# links/auth.py
# Parse Authorization: Bearer <token>
# Split named tokens from SHORTENER_API_TOKENS (comma-separated name:secret)
# compare_digest each secret
# Return token name or None
```

Decorator / mixin on all API views. Wrong or missing → `401` with
`WWW-Authenticate: Bearer`. Do not distinguish "unknown token" vs "malformed" in the body.

---

## 8. Public redirect

`GET /<code>` on the public host.

1. Lookup code (404 if missing, inactive, or expired).
2. Insert `Click`, bump `click_count` (best-effort).
3. `HttpResponseRedirect(target_url)` — **302**.

Reject codes that do not match `^[a-z0-9]{4,8}$` with 404 (no extra work).

`GET /` on the public host: landing page (“URL shortening service”, credit
`aimloperations.com`). Does not advertise the API.

`GET /debug/`: mint form, **DEBUG only**, never on `SHORT_PUBLIC_HOSTS`.

`GET /healthz/`: `{"status": "ok"}`.

---

## 9. Environment

### This service

| Var | Purpose |
|-----|---------|
| `DJANGO_ENV` | `dev` / `beta` / `prod` |
| `DJANGO_SECRET_KEY` | Django signing; **not** an API token |
| `DJANGO_DEBUG` | false in prod |
| `DJANGO_ALLOWED_HOSTS` | public short host **and** API hostname |
| `DATABASE_URL` | Postgres; empty → SQLite |
| `SHORT_DOMAIN` | public hostname phones use (`go.mkdrealtor.com` then `mkd.to`) |
| `PUBLIC_SHORT_URL` | origin for minted URLs, e.g. `https://go.mkdrealtor.com` (no trailing slash) |
| `SHORT_PUBLIC_HOSTS` | comma list; Host values that only serve redirects |
| `SHORT_API_HOSTS` | comma list; Host values that serve `/api/` (public API DNS and/or docker name) |
| `SHORT_ADMIN_HOSTS` | comma list; Host values that serve `/admin/` (default localhost only) |
| `SHORTENER_API_TOKENS` | `name:secret,name:secret` — required for API |
| `SHORT_ALLOWED_HOSTS` | allowlist for `target_url` hosts |
| `SHORT_CODE_LENGTH` | default `6` |
| `CLICK_IP_PEPPER` | HMAC key for `ip_hash`; distinct from `DJANGO_SECRET_KEY` |
| `GUNICORN_WORKERS` | default `2` |

`.env.example` documents all of these. Prod secrets live in
`~/Documents/secrets/url_shortening_service/` on the control node, same convention as
`monica_site`.

### Caller (`monica_site`) — later, other repo

```text
SHORTENER_BASE_URL=https://shortener.aimloperations.com
SHORTENER_API_TOKEN=monica:<same-secret>
```

`monica_site` sends `Authorization: Bearer monica:<secret>` to
`POST {SHORTENER_BASE_URL}/api/links/` and substitutes `short_url` into SMS bodies
**after** UTM is attached.

Do not implement the caller in this repo. Document the contract only.

---

## 10. Docker

### `Dockerfile`

Same pattern as `monica_site`:

- `FROM python:3.12-slim`
- install `libpq5`, copy `uv`
- `uv sync --frozen --no-dev`
- copy `site/` + entrypoint
- `EXPOSE 8000`

### `docker-compose.yml` (dev)

- `db`: Postgres 16
- `web`: build `.`, mount `./site`, `DJANGO_ENV=dev` → runserver
- **Do not** publish the host port to the LAN unless you are testing redirects.
  Prefer `127.0.0.1:8005:8000` (container still listens on 8000).
- Internal API hostname: service name `web` (or `url-shortener`).

### `docker-compose.prod.yml`

- `web` only. External Postgres via `DATABASE_URL`.
- `ports: "${WEB_PORT:-8005}:8000"` (pick a free host port; document it).
- `env_file: .env`
- No worker profile.

### Entrypoint

Wait for DB → `migrate --noinput` → if `DJANGO_ENV=dev` then `runserver`, else
`collectstatic` + gunicorn.

---

## 11. NPM / deploy notes (ops, not code)

When the domains exist:

1. Public NPM proxy host = `SHORT_DOMAIN` → this container’s `WEB_PORT`.
2. Proxy `/` (landing) and the code regex. `/api/`, `/admin/`, `/debug/` → 404.
3. TLS like other apps.
4. Add `SHORT_DOMAIN` to `DJANGO_ALLOWED_HOSTS` and `SHORT_PUBLIC_HOSTS`.
5. `PUBLIC_SHORT_URL=https://<that-domain>`.
6. Second NPM proxy host = API hostname → same container. Proxy `/api/` only.
7. Add the API hostname to `DJANGO_ALLOWED_HOSTS` and `SHORT_API_HOSTS`.

CI (same split as `monica_site`, default branch `master`):

- `.gitea/workflows/ci.yml` — unit tests on pull requests
- `.gitea/workflows/deploy-beta.yml` — on push to `master`: unit tests → compose
  tests → `server-infra/scripts/deploy.sh --app url_shortening_service --env beta`
- `.gitea/workflows/deploy-prod.yml` — `workflow_dispatch` (Actions button): same
  tests, then `--env prod`

---

## 12. Tests (required before merge)

Django `TestCase` / `SimpleTestCase`. No live network.

- Auth: missing Bearer → 401; wrong token → 401; empty `SHORTENER_API_TOKENS` → 503;
  matching named token → 201.
- Host split: request to short host `/api/links/` → 404 even with valid Bearer.
  Public API host without Bearer → 401; valid Bearer → 201. `/admin/` 404 on API host.
- Allowlist: `https://mkdrealtor.com/x` ok; `http://…` 400; `https://evil.com` 400;
  `javascript:alert(1)` 400.
- Redirect: active code → 302 to target; inactive/expired/unknown → 404.
- Redirect is 302, not 301.
- Click row created and `click_count` incremented.
- Idempotent create with same `target_url` + `external_ref`.
- `short_url` uses `PUBLIC_SHORT_URL`, not the API Host header.
- Code charset: generated codes only use the unambiguous alphabet.

---

## 13. Implementation order

1. `uv init` / `pyproject.toml` + Django project + settings/env helpers.
2. `links` models + migration.
3. `links/auth.py` + API views (create/list/detail/disable).
4. Redirect view + host-split middleware.
5. Dockerfile, compose, entrypoint, `.env.example`.
6. Tests in §12.
7. README: how to run locally, how a caller authenticates, what is public vs internal.
8. Stop. Caller integration lives in `monica_site` after this service is up.

Phase 2 (not this pass): SMS auto-shorten in `monica_site` composer, custom slugs,
per-recipient codes, internal-only admin UI.

---

## 14. Local run (target)

```bash
cp .env.example .env
# set SHORTENER_API_TOKENS=monica:dev-only-token
docker compose up --build
# or:
uv sync
cd site && uv run python manage.py migrate
uv run python manage.py runserver
```

Mint (API host / localhost):

```bash
curl -sS -X POST http://127.0.0.1:8005/api/links/ \
  -H "Authorization: Bearer monica:dev-only-token" \
  -H "Content-Type: application/json" \
  -d '{"target_url":"https://mkdrealtor.com/","title":"test"}'
```

Follow (public path):

```bash
curl -sSI http://127.0.0.1:8005/<code>
```