# Product Spec — Online Sessions, Registration Jobs, Attendance & Analytics

## Document Metadata

| Field | Value |
|---|---|
| **Module** | Online Sessions & Attendance |
| **Milestone** | M1 (Provisioning + Registration Jobs + Attendance) · M2 (Analytics & Graphs) |
| **Delivery Date** | TBD |
| **Version** | v1.0 |
| **Review Status** | Draft |
| **Generated On** | 2026-07-01 |
| **Last Updated** | 2026-07-01 |
| **Inputs Used** | Existing codebase (`src/online-session`, `src/zoom`, `src/online-attendance`, `src/qr-attendance`, `src/online-analytics`, `src/queue`), existing docs (`docs/online-session/01`, `02`; `docs/QR_ATTENDANCE_AND_CHECKIN.md`), house template `docs/artefact-templates/product-spec-template.md` |

> **Baseline vs. new.** A large part of this module is already built. Throughout this spec, capabilities are tagged **[EXISTS]**, **[EXTEND]** (present but must be enhanced), or **[NEW]** (to be built). This keeps the spec honest and makes the delivery scope obvious to engineering.

---

## Review Comments

| # | Section | Comment | Status | Resolved On |
|---|---|---|---|---|
| 1 | — | Initial draft for review | Open | — |

---

## 1. Module Overview

| Field | Value |
|---|---|
| **Purpose** | Provision Zoom online sessions for program sessions, register eligible registrants onto them as background jobs with transparent success/failure metrics, capture and manage attendance (self-join, Zoom webhook, and admin/coordinator manual actions), and report participation through analytics and graph data. |
| **Business Value** | Online sessions are how programs are delivered at scale (1–2k attendees per webinar). Reliable provisioning, accurate attendance, and clear analytics are what let the operations team run programs, prove delivery, and follow up with absentees. |
| **User Value** | Admins/coordinators get a single, provider-neutral surface to set up sessions, push registrants, fix attendance in bulk, and see who showed up. Participants get a one-click join that also records their attendance. |
| **Module Type** | Feature + Integration (Zoom) |
| **Milestone Scope** | **M1:** session provisioning (single + bulk), per-session registration jobs with per-item failure reasons, join-click + webhook attendance capture, manual mark/unmark/**absent** across independent **RM** and **Coordinator** markers, **RM & Coordinator** roles, **group (bulk) attendance operations**, paginated/exportable lists for every surface. **M2:** analytics KPIs, graph/chart datasets, dashboards. |
| **Deferred to Future Milestones** | Non-Zoom providers (Teams, Meet, YouTube live), automated absentee follow-up communications, predictive/no-show scoring. |

---

## 2. Scope & Boundaries

**In Scope:**
- **[EXISTS]** Provision a single Zoom webinar/meeting against a `program_session` (`POST /online-session`), and bulk-provision many (`POST /online-session/bulk`).
- **[EXISTS]** Register / unregister / downgrade a single registrant (`POST /online-session/registrations`), and start a background bulk-registration job for a program or session (`POST /online-session/registrations/bulk`) that only touches **eligible** registrants.
- **[EXTEND]** Job progress metrics: today `total / generated / skipped / failed`; add a **per-item failure list** (which registration failed and why) and an **eligibility breakdown** (eligible vs. ineligible-with-reason).
- **[EXISTS]** Attendance capture via participant join-click (`POST /online-attendance/sessions/:id/join`) and Zoom `participant_joined` webhook.
- **[EXISTS]** Admin mark / unmark attendance for one registration (`.../attendance/mark`, `.../attendance/unmark`).
- **[NEW]** **Multi-source attendance signals — every source is retained independently per reg+session.** A registration's attendance is composed of separate, co-existing signals: **join-click** (participant self-join), **Zoom** (webhook / reconciliation, which also knows absentees), **RM (Relationship Manager)**, and **Coordinator**. Each signal keeps its own **present / absent / unset** state plus who set it and when. No source overwrites another — they are all kept. The *effective* attendance shown in reports is **derived** from all signals (see Open Questions).
- **[NEW]** Explicit **absent** state (distinct from unset/unmark) on each source that supports it (RM, Coordinator, admin; Zoom sets absent via reconciliation).
- **[NEW]** **Group operations**: bulk mark, bulk unmark, bulk absent for online sessions, per source (QR already has bulk-manual-checkin / undo-checkin — mirror that outcome shape here).
- **[NEW]** **RM** and **Coordinator** actors: RM manages the RM signal for their assigned registrations; Coordinator manages the Coordinator signal for their assigned sessions.
- **[EXISTS]** Paginated, searchable, Excel-exportable lists for registrations and attendance at session and program scope.
- **[EXTEND / M2]** Analytics KPIs (`/analytics/kpi`) and **[NEW]** graph/chart datasets (time-series joins, platform/join-mode distributions, funnel).

**Out of Scope:**
- Payment/eligibility rules themselves — owned by the Registration & Payment modules; this module *consumes* eligibility, it does not define it.
- QR/offline physical attendance — owned by `qr-attendance`; this spec reuses its `program_user_attendance` table and bulk patterns but does not change QR behaviour.
- The generic SQS communication/heavy-processing queues — this module uses the in-process `background_jobs` async pattern, not SQS.

**Dependencies:**

| Module | What This Module Consumes From It |
|---|---|
| Registration | Registrant records, contact info, registration status, eligibility signals (`hdb_program_registration`) |
| Payment | Payment status used in the eligibility rule (`ONLINE_COMPLETED` / `OFFLINE_COMPLETED`, `isFreeSeat`) |
| Program / Program Session | The `program_session` an online session attaches to; `modeOfOperation` (ONLINE/HYBRID) |
| Zoom integration (`src/zoom`) | Webinar/meeting creation, registrant push, participant reconciliation, webhooks |
| Auth / Roles | `admin`, `operational_manager`, and new `coordinator` role gating |

**Dependents:**

| Module | What It Consumes From This Module |
|---|---|
| User APIs | `GET /users/:id/registrations` embeds each registration's `onlineSessions` (join URL, join window, `webinarId`, attendance state) via `RegistrationSessionInfoService` |
| Reporting / Dashboards | Attendance reports and analytics KPIs |
| Communications (future) | Absentee lists for follow-up |

---

## 3. Actors

### Admin
| Field | Value |
|---|---|
| **Goals** | Provision sessions, push registrants, correct attendance, view analytics for any program. |
| **Context** | Operations console, before and during/after live sessions. |
| **Pain Points** | Bulk registration failures are opaque (only counts today); no way to bulk-fix online attendance; no explicit absent state. |
| **Access Level** | Full — every endpoint in this module. |

### Relationship Manager (RM) **[NEW]**
| Field | Value |
|---|---|
| **Goals** | Record attendance for their **assigned registrants** — the "RM attendance" marker — independent of what a coordinator records. |
| **Context** | RM follows their book of registrants across a program; confirms who actually attended each session. |
| **Pain Points** | No RM-owned attendance signal today; RM presence is only a name snapshot (`rm_name`) on the attendance row. |
| **Access Level** | Set / clear / absent the **RM marker** only for registrations assigned to them; read attendance for those registrations. Cannot provision sessions or run bulk registration. |

### Coordinator **[NEW]**
| Field | Value |
|---|---|
| **Goals** | Record attendance for the specific session(s) assigned to them — the "Coordinator attendance" marker — during/after a live event. |
| **Context** | On-ground/online event operations, working a single session. |
| **Pain Points** | Today only `admin` (and `operational_manager` for QR scan) can act; there is no scoped role for a session coordinator, and no coordinator-owned signal. |
| **Access Level** | Set / clear / absent the **Coordinator marker** **only for sessions assigned to them**; read the attendance list for those sessions. Cannot provision sessions or run bulk registration. |

### Participant (Registrant)
| Field | Value |
|---|---|
| **Goals** | Join their session in one click and have attendance recorded automatically. |
| **Context** | App / browser, inside the join window. |
| **Pain Points** | Being marked absent despite attending; unclear join windows. |
| **Access Level** | Join sessions they are confirmed-registered for; view their own join links/attendance via `GET /users/:id/registrations`. |

### System — Zoom & Cron
| Field | Value |
|---|---|
| **Goals** | Keep attendance/analytics accurate without manual work. |
| **Context** | Zoom webhooks (`participant_joined`) and the hourly reconciliation cron. |
| **Pain Points** | Webhook volume at 1–2k attendees; reconciliation must be idempotent and must not override admin corrections. |
| **Access Level** | Signature-validated webhook ingestion; cron reconciliation writes analytics + attendance events. |

---

## 4. Features

### F1 — Session Provisioning (Single) **[EXISTS]**
| Field | Value |
|---|---|
| **Description** | Provision one Zoom webinar or meeting against a `program_session`; persist provider ids/links; strip the host `startUrl` from responses. |
| **Actors** | Admin |
| **Key Constraints** | One online session per program session (`ONLINE_SESSION_ALREADY_EXISTS` on re-provision). `Content-Type: application/json` required (whitelist strips form bodies). Zoom must be enabled (`ENABLE_ZOOM`). Rollback deletes the Zoom resource if the DB save fails. |
| **Appears In Workflows** | W1 |

### F2 — Session Provisioning (Bulk) **[EXISTS]**
| Field | Value |
|---|---|
| **Description** | Provision many sessions in one call; non-atomic — each entry succeeds or fails independently, returning `{ created[], failed[] }`. |
| **Actors** | Admin |
| **Key Constraints** | Not transactional (external side effects). Caller retries only failed entries. |
| **Appears In Workflows** | W1 |

### F3 — Registration-as-a-Job (per session, for all eligible) **[EXISTS + EXTEND]**
| Field | Value |
|---|---|
| **Description** | Start a background job that registers every **eligible** registrant of a program (or a specific session's registrants) onto the Zoom resource, in batches, idempotently. Returns a `jobId` immediately (202). |
| **Actors** | Admin |
| **Key Constraints** | Target must be Zoom-provisioned. Already-registered → skipped. Zoom/DB error on one registrant is tallied as failed and does not abort the job. **[EXTEND]** must record *why* each failure happened and expose the eligible/ineligible breakdown. |
| **Appears In Workflows** | W2 |

### F4 — Join-Click Attendance Capture **[EXISTS]**
| Field | Value |
|---|---|
| **Description** | When a confirmed registrant clicks join within the join window, the system appends a `JOIN_CLICK` event, marks them attended, records device/mode metadata, credits companions, and returns the join URL. |
| **Actors** | Participant |
| **Key Constraints** | Join window enforced (default 15 min before start → session end). Must be CONFIRMED-registered. Idempotent (refreshes metadata; first event sets `checkedInAt`). |
| **Appears In Workflows** | W3 |

### F5 — Webhook / Reconciliation Attendance **[EXISTS]**
| Field | Value |
|---|---|
| **Description** | Zoom `participant_joined` webhook (signature-validated) and the hourly reconciliation cron append `ZOOM_WEBHOOK` events and rebuild analytics. |
| **Actors** | System |
| **Key Constraints** | Live webhook gated by `ENABLE_ZOOM_LIVE_ATTENDANCE` (off by default at scale). Reconciliation is a full replace per webinar and must **not** override an admin unmark. |
| **Appears In Workflows** | W3, W6 |

### F6 — Multi-Source Signal Retention **[NEW]**
| Field | Value |
|---|---|
| **Description** | A registration's attendance for a session is the set of all its source signals, each retained side-by-side: **join-click**, **Zoom**, **RM**, **Coordinator**, and **admin/general**. Each signal stores its own `state` (`present` / `absent` / `unset`), `markedBy`, `markedAt`. Writing one signal never mutates another — this is what "keep all of join-click, Zoom, RM, Coordinator, absent/present" means. History beyond current state is preserved in the append-only `attendance_events[]` log. |
| **Actors** | Participant (join-click), System (Zoom), RM, Coordinator, Admin |
| **Key Constraints** | Keyed by `registrationId` (handles proxy/child regs). Zoom reconciliation may set its own signal to `present`/`absent` but must not touch RM/Coordinator/admin signals. The **effective** attendance is derived from all signals (derivation rule = Open Q2). |
| **Appears In Workflows** | W3, W4, W6 |

### F6b — Manual Attendance Management (single, per source) **[EXISTS + NEW]**
| Field | Value |
|---|---|
| **Description** | Actions on one registration's attendance, targeting a specific **source** (`rm` or `coordinator`; admin's existing action maps to the general/admin source): **mark [EXISTS for admin]**, **unmark [EXISTS for admin]**, **mark-absent [NEW]**. Mark and absent are explicit states on that source; unmark clears that source back to *unset* while keeping the event log. RM and Coordinator signals are stored independently and may disagree. |
| **Actors** | Admin, RM, Coordinator |
| **Key Constraints** | Identified by `registrationId`. Event log is append-only; each event records `source` (`RM_MARK` / `COORDINATOR_MARK` / `MANUAL_ADMIN` / `JOIN_CLICK` / `ZOOM_WEBHOOK`) and `performedBy`. Mark-absent must survive reconciliation (a webhook must not re-mark a source that was set absent). |
| **Appears In Workflows** | W4 |

### F7 — Group (Bulk) Attendance Operations (per marker) **[NEW]**
| Field | Value |
|---|---|
| **Description** | Bulk **mark**, **unmark**, and **absent** across many registrations in one call, targeting a marker (`rm` / `coordinator`), transaction-wrapped, returning a per-item outcome (`updated / alreadyInState / notFound / failed`). Covers "multiple unmark at once" and "multiple absent". |
| **Actors** | Admin, RM, Coordinator |
| **Key Constraints** | Mirrors QR's `bulk-manual-checkin` / `undo-checkin` outcome shape. Bounded batch size. Marker scope enforced per actor. |
| **Appears In Workflows** | W4 |

### F8 — RM & Coordinator Roles, Assignment & Markers **[NEW]**
| Field | Value |
|---|---|
| **Description** | Two scoped roles that each own an independent attendance marker: `rm` (scoped to assigned registrations) and `coordinator` (scoped to assigned sessions). Admin assigns both. Each marker is stored separately on the attendance record. |
| **Actors** | Admin (assigns), RM, Coordinator (act) |
| **Key Constraints** | RM scope enforced per assigned `registrationId`; Coordinator scope per assigned `sessionId`. A marker can only be written by its owning role (or admin on their behalf). |
| **Appears In Workflows** | W4 |

### F9 — Lists, Search, Pagination, Export **[EXISTS + EXTEND]**
| Field | Value |
|---|---|
| **Description** | Every list surface (registrations, attendance report at session and program scope) is paginated, searchable, filterable, and Excel-exportable, and returns `statusCounts`. |
| **Actors** | Admin, Coordinator |
| **Key Constraints** | Two paging conventions exist today (`limit/offset` for QR + session-registration lists; `page/limit` for online-attendance reports) — **[EXTEND]** standardize/ document per the API appendix. |
| **Appears In Workflows** | W2, W4, W5 |

### F10 — Analytics KPIs & Graph Data **[EXISTS + NEW · M2]**
| Field | Value |
|---|---|
| **Description** | KPI summary per session **[EXISTS]** (`SessionAnalyticsKpi`), plus **[NEW]** graph datasets: attendance funnel (registered → attended → dropoff), join-over-time series, platform and join-mode distributions, and program-level roll-ups. |
| **Actors** | Admin |
| **Key Constraints** | Sourced from `hdb_zoom_analytics` (rebuilt by reconciliation). Graph endpoints return chart-ready aggregates, not raw rows. |
| **Appears In Workflows** | W5 |

---

## 5. Workflows

---

### Workflow 1: Provision a Session (Single & Bulk) **[EXISTS]**

**Overview**
An Admin turns a scheduled `program_session` into a live Zoom webinar/meeting so registrants can be pushed and can later join. This is the entry point for everything downstream.

**Happy Path**
The Admin submits the online session details for a program session. The system resolves the provider (Zoom by default), loads the program session, and confirms it is not already provisioned. It calls Zoom to create the webinar or meeting, then persists an `hdb_online_session` row carrying `externalId` (the Zoom meeting/**webinar id**), `joinUrl`, `registrationUrl`, `panelistUrl`, `hostEmail`, `startUrl`, `status`, and the join-window settings. The `program_session` is stamped with its `onlineType`. The response returns the session with `webinarDetails`/`meetingDetails` and the host `startUrl` stripped. For bulk, the Admin submits an array; each entry is provisioned independently and the response lists `created[]` and `failed[]` so only failures need retrying.

**Variations**
- **Variation A: Meeting with shared link** — `requireRegistration: false` yields one shared link + passcode instead of per-user registrant links.
- **Variation B: Template-backed webinar** — when `ZOOM_TEMPLATE_ID` is set, the webinar inherits the template.

**Failure Scenarios**
- **Failure A: Already provisioned** — a program session already has an online session → `ONLINE_SESSION_ALREADY_EXISTS` (409); the Admin deletes the old one first.
- **Failure B: DB save fails after Zoom create** — the Zoom resource is rolled back (deleted) so nothing is orphaned → `ZOOM_WEBINAR_SAVE_FAILED`.
- **Failure C: Zoom disabled / rejects** — `ZOOM_DISABLED` (400) or `ZOOM_API_ERROR` with Zoom's message forwarded.

**System Impact**
- Data: writes `hdb_online_session`, updates `program_session.onlineType` + registration windows.
- Downstream: makes the session eligible for registration jobs and join capture; `webinarId` now surfaces in `GET /users/:id/registrations`.
- Analytics: none until the session runs.

**Acceptance Criteria**
- [ ] A single provision returns 201 with provider ids and a stripped `startUrl`.
- [ ] Bulk returns 200 with `{ created[], failed[] }`; one bad entry never aborts the rest.
- [ ] Re-provisioning the same program session returns 409.
- [ ] A DB failure after Zoom-create leaves no orphaned Zoom resource.

---

### Workflow 2: Register All Eligible Registrants as a Background Job **[EXISTS + EXTEND]**

**Overview**
For a provisioned session (or an entire program), the Admin pushes every **eligible** registrant onto Zoom without blocking on the (potentially thousands of) API calls. This is the "for each session, create registration as a job for all eligible registrants" requirement, with the transparency the Admin needs to trust it.

**Happy Path**
The Admin starts a bulk registration for a `programId` or `sessionId`. The system resolves the target Zoom-provisioned session(s), computes the set of **eligible** registrants (see Data Requirements — eligibility), and creates a `background_jobs` row of type `BULK_ZOOM_REGISTRATION` with `status = PROCESSING`, `total = eligible × targetSessions`, and zeroed `generated/skipped/failed`. It returns `{ jobId, status, total }` (202) immediately and processes registrants in batches: each is pushed via the same path as single registration, writing a per-registrant `hdb_program_registration_online_session` extension row (`externalRegistrantId`, per-user `joinUrl`, `isPanelist`, `registrationType`). Already-registered registrants are **skipped** (idempotent re-run). The Admin polls `GET /online-session/registrations/bulk/:jobId` and watches `generated/skipped/failed` climb to completion.

**[EXTEND] — Metrics the Admin actually needs**
Today the job exposes only counts. This workflow requires, on the status response:
- an **eligibility breakdown**: `eligible`, and `ineligible[]` with a reason per registration (e.g. `REJECTED`, `SAVE_AS_DRAFT`, `PAYMENT_INCOMPLETE`, `NO_EMAIL`);
- a **per-item failure list**: `failures[] = { registrationId, reason }` (e.g. `ZOOM_USER_ALREADY_REGISTERED`, `ZOOM_API_ERROR`, `SESSION_NOT_PROVISIONED`), stored in `background_jobs.metadata` (jsonb) and returned paginated on the status endpoint.

**Variations**
- **Variation A: Program-wide** — `programId` given, no `sessionId`: registrants are program-scoped and pushed against every provisioned session of the program.
- **Variation B: Panelists** — `role: panelist` registers everyone as panelists (webinar only).
- **Variation C: Batch size** — `batchSize` (1–50, default 10) tunes parallelism.

**Failure Scenarios**
- **Failure A: Ambiguous target** — program has 0 or >1 provisioned sessions and no `sessionId` → `ZOOM_BULK_WEBINAR_UNRESOLVED` (400).
- **Failure B: Per-registrant error** — Zoom/DB error for one registrant is tallied in `failed` **with a reason** and the job continues.
- **Failure C: Job start fails** — `ZOOM_BULK_START_FAILED`; the row records `error_message`.
- **Failure D: Unknown job id** — poll returns `ZOOM_BULK_JOB_NOTFOUND` (404).

**System Impact**
- Data: `background_jobs` progress row (+ `metadata.failures[]`, `metadata.ineligible[]`); one `hdb_program_registration_online_session` per successful registrant.
- Downstream: per-user join URLs now available to the participant and in `GET /users/:id/registrations`.
- Metrics: eligible / done / failed with reasons — the core reporting ask.

**Acceptance Criteria**
- [ ] Starting a job returns 202 with `{ jobId, status, total }` where `total` equals the computed eligible count × target sessions.
- [ ] Re-running the same job registers no duplicates (all become `skipped`).
- [ ] The status response returns `eligible`, `generated`, `skipped`, `failed`, plus a paginated `failures[]` with a reason per registration and an `ineligible[]` breakdown.
- [ ] One failing registrant never aborts the job.

---

### Workflow 3: Participant Joins → Attendance Captured **[EXISTS]**

**Overview**
A registrant clicks "Join" and, in the same action, is recorded as attended — the primary automatic attendance signal.

**Happy Path**
Inside the join window, a CONFIRMED registrant calls the join endpoint for a session. The system validates the session is online and the window is open, finds-or-creates their `program_user_attendance` row (keyed by session + user, reusing any email-keyed row a Zoom webhook already created), appends a `JOIN_CLICK` event, sets `isAttended = true` and `checkedInAt` on the first event, records device/mode metadata, credits any companion registration ids, and returns the join URL + meeting details with `attendanceMarked: true`.

**Variations**
- **Variation A: Companions** — `companionRegistrationIds[]` (max 25) each get a `JOINING_WITH_OTHERS` attendance credit.
- **Variation B: Zoom webhook** — if live attendance is enabled, a `participant_joined` webhook appends `ZOOM_WEBHOOK` (idempotent) even without a join-click.

**Failure Scenarios**
- **Failure A: Outside window** — join before open / after end is rejected; no attendance recorded.
- **Failure B: Not confirmed-registered** — rejected.
- **Failure C: Session not online** — rejected.

**System Impact**
- Data: `program_user_attendance` row + appended event; client metadata columns.
- Downstream: feeds the attendance report and analytics; visible in `GET /users/:id/registrations`.
- Analytics: contributes to `attended`, join-mode and platform distributions.

**Acceptance Criteria**
- [ ] A join click inside the window returns the join URL and sets `isAttended = true` with a `JOIN_CLICK` event.
- [ ] A second join click is idempotent (metadata refreshed, no duplicate check-in).
- [ ] Companions are credited.
- [ ] Joins outside the window or by non-registrants are rejected without recording attendance.

---

### Workflow 4: RM & Coordinator Attendance — Single & Group **[EXISTS + NEW]**

**Overview**
Two roles each keep their own attendance signal for a registration: the **RM marker** (owned by the Relationship Manager, scoped to their assigned registrants) and the **Coordinator marker** (owned by the session coordinator, scoped to their assigned sessions). The two are stored independently and may disagree. This is the "RM attendance and Coordinator attendance for that reg", plus "mark / unmark / mark-absent / multiple unmark / multiple absent" requirement. Admin's existing mark/unmark continues to work as a general/admin marker.

**Happy Path (single, per marker)**
The actor targets a `registrationId` on a session and chooses a `markerType` (`rm` / `coordinator`; inferred from the actor's role, overridable by admin) and an action:
- **Mark** — set that marker to `present`, append an event with `source` = `RM_MARK` / `COORDINATOR_MARK` (or `MANUAL_ADMIN`), `performedBy`, and timestamp. Idempotent.
- **Unmark** — reset that marker to `unset`; keep the append-only event log. Idempotent. The override persists across reconciliation.
- **Mark-absent** — set that marker to `absent` (distinct from "not yet marked") so a later webhook/reconciliation does **not** re-mark that marker; append an absent event.

Each marker carries its own `state` + `markedBy` + `markedAt`. The **effective** attendance for reporting/analytics is derived from the two markers plus the self-join/webhook signal — the derivation rule (e.g. "attended if any marker present" vs. "coordinator is authoritative") is an Open Question.

**Happy Path (group) [NEW]**
The actor submits a list of `registrationId`s, a `markerType`, and an action (`mark` / `unmark` / `absent`). The system processes them in a single transaction and returns a per-item outcome: `updated`, `alreadyInState`, `notFound`, `failed` — mirroring QR's `bulk-manual-checkin` / `undo-checkin` result shape. This covers "multiple unmark at once" and "multiple absent".

**Role scoping [NEW]**
An RM may write the RM marker **only** for registrations assigned to them; a Coordinator may write the Coordinator marker **only** for sessions assigned to them. Acting outside scope, or writing a marker not owned by the actor's role, is rejected by the guard. Admin may write either marker.

**Variations**
- **Variation A: Reuse email-keyed row** — marking reuses a pre-existing Zoom-webhook (email-keyed) row instead of creating a duplicate.
- **Variation B: Admin on behalf** — admin sets `markerType` explicitly to correct either the RM or Coordinator marker.
- **Variation C: Markers disagree** — RM present, Coordinator absent (or vice-versa) is a valid, stored state; the effective value follows the derivation rule.

**Failure Scenarios**
- **Failure A: Unknown registration/attendance id** — tallied as `notFound` in group ops; 404 for single.
- **Failure B: Actor out of scope / wrong marker** — 403.
- **Failure C: Partial group failure** — individual items fail with a reason; the transaction policy (all-or-nothing vs. best-effort) is an Open Question.

**System Impact**
- Data: independent per-source states (join-click, Zoom, RM, Coordinator, admin — each with `state` + `markedBy` + `markedAt`) on `program_user_attendance`, all retained side-by-side; append-only events with `source`.
- Downstream: the derived effective attendance flows into reports and analytics; absent state on a source suppresses re-marking of that source only.
- Metrics: RM vs. Coordinator vs. join-click vs. Zoom attribution via the per-source states and `attendance_events.source`; agreement/disagreement across sources.

**Acceptance Criteria**
- [ ] RM and Coordinator markers are stored independently; setting one never mutates the other.
- [ ] Single mark/unmark/absent on a marker produce the documented state and are idempotent.
- [ ] Mark-absent on a marker survives a subsequent reconciliation run (not re-marked present).
- [ ] Group mark/unmark/absent per marker return per-item outcomes and are transaction-wrapped.
- [ ] An RM can write only the RM marker for assigned regs; a Coordinator only the Coordinator marker for assigned sessions (403 otherwise).
- [ ] The effective attendance is computed from the two markers + self/webhook per the agreed derivation rule.

---

### Workflow 5: Attendance Lists, Reports & Export **[EXISTS + EXTEND]**

**Overview**
Admins/Coordinators view who is registered and who attended, filter and search, and export to Excel — at both session and program scope.

**Happy Path**
The actor requests the registration list for a session (`GET /online-session/:id/registrations`) or the attendance report for a session/program (`GET /online-attendance/sessions/:id/report`, `.../programs/:id/report`). The system returns a paginated, searchable page with `statusCounts` (e.g. All / Checked-In / Yet-to-Check-In) and each row's join URL and attendance state. Passing the download flag returns an Excel file URL on S3 instead.

**Variations**
- **Variation A: Filters** — by `isAttended`, `checkedInByUserId`, `sessionId` (program scope), free-text search on name/email/mobile/seq.
- **Variation B: Export** — `download=true` / `isDownload=true` returns `{ fileUrl }`.

**Failure Scenarios**
- **Failure A: Unknown session/program** — 404.
- **Failure B: Export generation error** — mapped generation error; the list read still works.

**System Impact**
- Data: read-only (plus an S3 file on export).
- Downstream: feeds operations follow-up and analytics.

**Acceptance Criteria**
- [ ] Every list endpoint paginates and returns a total + `statusCounts`.
- [ ] Search and filters narrow results correctly.
- [ ] Export returns a working S3 file URL with the documented columns.

---

### Workflow 6: Analytics & Graph Data **[EXISTS + NEW · M2]**

**Overview**
After a session runs, the Admin sees participation KPIs and chart-ready datasets — the "analytics and graphs data" requirement.

**Happy Path**
The reconciliation cron (hourly, idempotent, full-replace per webinar) rebuilds `hdb_zoom_analytics` from Zoom participants/registrants/absentees and marks attendance events. The Admin reads `GET /online-session/:id/analytics/kpi` for the `SessionAnalyticsKpi` summary (total audience, registered, attended, registered-attended, absentees, drop-offs, late-comers, cancelled, downgraded, internal users, platform distribution, join modes). **[NEW]** graph endpoints return chart-ready aggregates: the attendance funnel (registered → attended → drop-off), a join-count time series across the session, platform and join-mode distributions, and program-level roll-ups across sessions.

**Variations**
- **Variation A: Raw rows** — `GET /online-session/:id/analytics` returns per-participant `ZoomAnalytics` rows for drill-down/export.
- **Variation B: On-demand sync** — `POST /online-session/:id/sync` reconciles immediately instead of waiting for the cron.

**Failure Scenarios**
- **Failure A: Not yet reconciled** — analytics empty until the session completes and reconciliation runs; KPI returns zeros.
- **Failure B: Zoom fetch error during sync** — mapped `ZOOM_API_ERROR`; last good analytics retained.

**System Impact**
- Data: `hdb_zoom_analytics` rebuilt; attendance events appended.
- Downstream: dashboards, absentee follow-up.

**Acceptance Criteria**
- [ ] KPI reflects reconciled data and matches the attendance report counts.
- [ ] Reconciliation is idempotent and does not override an admin unmark/absent.
- [ ] Graph endpoints return chart-ready aggregates for funnel, time series, and distributions.

---

## 6. Data Requirements — Field Constraints Only

| Field Name | Workflow | Mandatory / Optional | Min | Max | Allowed Values / Notes |
|---|---|---|---|---|---|
| `programSessionId` | W1 | Mandatory | — | — | Existing `program_session.id` |
| `onlineType` | W1 | Optional | — | — | `webinar` (default) / `meeting` |
| `startAt` | W1 | Mandatory | — | — | ISO 8601 |
| `duration` | W1 | Mandatory | 1 | — | Minutes |
| `requireRegistration` | W1 | Optional | — | — | Meetings only; default `true` |
| `joinOpensMinutesBefore` | W1 | Optional | 0 | 1440 | Default 15 |
| `title` | W1 | Optional | — | 500 | Defaults to session name |
| `programId` / `sessionId` (bulk register) | W2 | Mandatory (exactly one) | — | — | — |
| `role` | W2, single-register | Optional | — | — | `attendee` (default) / `panelist` |
| `batchSize` | W2 | Optional | 1 | 50 | Default 10 |
| `registrationId` | W2, W4 | Mandatory | — | — | `hdb_program_registration.id` |
| `action` (single register) | W2 | Mandatory | — | — | `register` / `unregister` / `downgradeToAudio` |
| `modeOfJoining` | W3 | Optional | — | — | `SELF` / `JOINING_WITH_OTHERS` / `JOINING_ON_OTHER_DEVICE` |
| `deviceType` | W3 | Optional | — | — | `App` / `Browser` / `ThroughLink` |
| `companionRegistrationIds` | W3 | Optional | 0 | 25 | Array of registration ids |
| `attendanceAction` (single) | W4 | Mandatory | — | — | **[NEW]** `mark` / `unmark` / `absent` |
| `markerType` | W4 | Optional | — | — | **[NEW]** `rm` / `coordinator` (defaults to actor's role; admin may set either) |
| `registrationIds` (group) | W4 | Mandatory | 1 | *TBD* | **[NEW]** Group op target list; max size = Open Question |
| `notes` | W4 | Optional | — | — | Free text (manual reason) |
| `page` / `limit` / `offset` | W2, W4, W5 | Optional | — | — | `limit` default 20; two conventions today (see Open Questions) |
| `search` | W5 | Optional | — | — | name / email / mobile / seq |
| `download` / `isDownload` | W5 | Optional | — | — | `true` → Excel S3 URL |

---

## 7. Non-Functional Requirements

- **Performance:** List endpoints must paginate (never unbounded); bulk registration must stay non-blocking (202 + background job) and survive 1–2k registrants. Attendance capture on join click must respond within the join flow's latency budget. Reconciliation must avoid N+1 (batched lookups, as `RegistrationSessionInfoService` already does).
- **Reliability:** Provisioning rolls back the Zoom resource on DB failure. Bulk jobs isolate per-item failures. Reconciliation is idempotent (full-replace per webinar) and must never override admin corrections (unmark/absent). Group attendance ops are transaction-wrapped.
- **Security:** `admin`-gated write endpoints; `coordinator` scoped to assigned sessions; QR scan also allows `operational_manager`. Zoom webhook is public but HMAC-signature + timestamp validated. Host `startUrl` is never returned to clients. `Content-Type: application/json` enforced by the validation whitelist.
- **Usability:** Every list returns `statusCounts` and supports search/filter/export. Job status must surface *why* items were skipped/failed, not just counts.

---

## 8. Success Metrics

- **Business:** % of sessions fully provisioned and registered before start; reduction in "marked absent but attended" disputes.
- **User (ops):** Time to push all registrants for a session; time to correct a batch of attendance records (target: one bulk call).
- **Technical:** Bulk registration job success rate; reconciliation accuracy vs. Zoom; zero orphaned Zoom resources; attendance-capture idempotency (no duplicate check-ins).
- **Adoption:** % of attendance recorded automatically (join-click + webhook) vs. manual; coordinator role usage.

---

## 9. Open Questions

| # | Question | Context | Options | Impact | Owner | Status |
|---|---|---|---|---|---|---|
| 1 | How are the per-source signals stored on `program_user_attendance`? | Every source (`join_click`, `zoom`, `rm`, `coordinator`, `admin`) is retained independently with state (present/absent/unset) + who/when. | (a) columns-per-source (`rm_state`, `coordinator_state`, `zoom_state`, `join_click_state` + `*_marked_by/at`) — simple, fixed set · (b) normalized `attendance_signal` child table keyed by `(attendance_id, source)` — extensible to new sources without migrations | Data model, query shape, extensibility | Eng + Product | Open |
| 2 | **Derivation rule for effective attendance when sources disagree.** | Signals may differ (join-click present, Coordinator absent; RM present, Zoom absent). Reports/analytics/absentee lists need one effective value. | (a) attended if ANY source present · (b) coordinator authoritative · (c) a source priority order (e.g. coordinator > rm > join_click > zoom) · (d) automatic (join/zoom) is truth, manual only overrides when set | Reporting, KPI, absentee follow-up | Product | Open |
| 3 | RM ↔ registration and Coordinator ↔ session assignment models. | Neither role/marker exists today. How is an RM tied to its registrants and a coordinator to its sessions? | RM: reuse existing `rmContactUser`/`rm_name` linkage on the registration; Coordinator: new join table `session_coordinators` | Role model + guards + scope checks | Product + Eng | Open |
| 4 | Group operation transaction policy. | Bulk mark/unmark/absent — all-or-nothing or best-effort with per-item outcomes? | QR uses transaction + per-item outcome (best-effort within a tx) | Consistency vs. partial success UX | Eng | Open |
| 5 | Max size for group attendance lists and bulk-register `batchSize` interplay. | Need a safe upper bound per request. | e.g. 500 ids/request | Request latency, memory | Eng | Open |
| 6 | Pagination convention: standardize on `limit/offset` or `page/limit`? | Online-attendance reports use `page/limit`; QR + session-registration lists use `limit/offset`. | Pick one for new endpoints | API consistency | Eng | Open |
| 7 | Per-item failure storage: `background_jobs.metadata` jsonb vs. a dedicated `background_job_items` table? | Metrics require per-registration failure reasons; jsonb is simplest but unindexed/large at 2k rows. | jsonb (M1) → table (later) | Query/paging of failures | Eng | Open |
| 8 | Which graph datasets are must-have for M2? | "graphs data all those" is broad. | Funnel, join-time series, platform/join-mode distribution, program roll-up | M2 scope | Product | Open |

---

## 10. Downstream Input Matrix

| Downstream Artifact | Sections Required from This Document |
|---|---|
| Module-level Tech Spec | §1, §2, §3, §4, §6, Appendix A |
| Stories / Tasks | §1, §3, §4, §5, §8, Appendix A |
| Data Model | §1, §2, §4, §6, Open Questions 1–2, 7 |
| UAT Acceptance Criteria | §5 (Acceptance Criteria blocks), §8 |

---

## Appendix A — Complete Task / API Surface

> Answers "give all the possible tasks." Tag legend: **[E]** exists · **[X]** extend · **[N]** new. Paging column notes the list convention.

### A.1 Provisioning
| # | Task | Method / Path | Tag | Paging |
|---|---|---|---|---|
| 1 | Create session (webinar/meeting) | `POST /online-session` | E | — |
| 2 | Bulk-create sessions | `POST /online-session/bulk` | E | — |
| 3 | List sessions | `GET /online-session?limit&offset&status&programId` | E | limit/offset |
| 4 | Get session | `GET /online-session/:id` | E | — |
| 5 | Update session | `PUT /online-session/:id` | E | — |
| 6 | Override status | `PATCH /online-session/:id/status` | E | — |
| 7 | Delete session | `DELETE /online-session/:id` | E | — |
| 8 | Join SDK token | `POST /zoom/token/join` | E | — |
| 9 | Create webinar template | `POST /zoom/webinar-templates` | E | — |

### A.2 Registration (as job)
| # | Task | Method / Path | Tag | Paging |
|---|---|---|---|---|
| 10 | Register / unregister / downgrade one | `POST /online-session/registrations` | E | — |
| 11 | Start bulk registration job (all eligible) | `POST /online-session/registrations/bulk` | E | — |
| 12 | Poll bulk job status (counts) | `GET /online-session/registrations/bulk/:jobId` | E | — |
| 13 | **Job status: eligible/ineligible breakdown + per-item failure reasons** | *extend #12 response; failures paginated* | **X** | limit/offset |
| 14 | List session registrations + join URLs (+ Excel) | `GET /online-session/:id/registrations?page&limit&search&download` | E | page/limit |

### A.3 Attendance capture
| # | Task | Method / Path | Tag | Paging |
|---|---|---|---|---|
| 15 | Participant join click → capture | `POST /online-attendance/sessions/:sessionId/join` | E | — |
| 16 | Zoom webhook (participant_joined) | `POST /online-attendance/zoom/webhook` (also `POST /zoom/webhook`) | E | — |
| 17 | On-demand reconcile/sync session | `POST /online-session/:id/sync` | E | — |

### A.4 RM & Coordinator attendance (single & group)
> All marker endpoints take `markerType` (`rm` / `coordinator`) in the body; it defaults to the actor's role and is overridable by admin. Admin's existing mark/unmark map to a general/admin marker.

| # | Task | Method / Path | Tag | Paging |
|---|---|---|---|---|
| 18 | Mark one present (admin/general) | `POST /online-attendance/sessions/:sessionId/attendance/mark` | E | — |
| 19 | Unmark one (admin/general) | `POST /online-attendance/sessions/:sessionId/attendance/unmark` | E | — |
| 20 | **Mark one present on a marker (RM / Coordinator)** | `POST /online-attendance/sessions/:sessionId/attendance/mark` (`markerType`) | **N** | — |
| 21 | **Unmark one on a marker** | `POST /online-attendance/sessions/:sessionId/attendance/unmark` (`markerType`) | **N** | — |
| 22 | **Mark one absent on a marker** | `POST /online-attendance/sessions/:sessionId/attendance/absent` (`markerType`) | **N** | — |
| 23 | **Group mark on a marker** | `POST /online-attendance/sessions/:sessionId/attendance/bulk-mark` | **N** | — |
| 24 | **Group unmark on a marker (multiple unmark once)** | `POST /online-attendance/sessions/:sessionId/attendance/bulk-unmark` | **N** | — |
| 25 | **Group absent on a marker (multiple absent)** | `POST /online-attendance/sessions/:sessionId/attendance/bulk-absent` | **N** | — |
| 26 | **Assign/unassign coordinator to session** | `POST/DELETE /online-session/:id/coordinators` | **N** | — |
| — | *(QR analogues already exist: `/qr-attendance/manual-checkin`, `/bulk-manual-checkin`, `/undo-checkin`, `/scan`)* | — | E | — |

### A.5 Lists / reports / export
| # | Task | Method / Path | Tag | Paging |
|---|---|---|---|---|
| 27 | Session attendance report (+ Excel) | `GET /online-attendance/sessions/:sessionId/report?page&limit&isAttended&search&download` | E | page/limit |
| 28 | Program attendance report (+ Excel) | `GET /online-attendance/programs/:programId/report?...&sessionId` | E | page/limit |
| 29 | QR attendance list (+ Excel) | `GET /qr-attendance?limit&offset&programId&sessionId&isAttended&search&isDownload` | E | limit/offset |

### A.6 Analytics & graphs (M2)
| # | Task | Method / Path | Tag | Paging |
|---|---|---|---|---|
| 30 | Per-participant analytics rows | `GET /online-session/:id/analytics` | E | — |
| 31 | Session KPI summary | `GET /online-session/:id/analytics/kpi` | E | — |
| 32 | **Attendance funnel (registered→attended→dropoff)** | `GET /online-session/:id/analytics/funnel` | **N** | — |
| 33 | **Join-over-time series** | `GET /online-session/:id/analytics/timeseries` | **N** | — |
| 34 | **Platform & join-mode distributions** | `GET /online-session/:id/analytics/distributions` | **N** | — |
| 35 | **Program-level analytics roll-up** | `GET /online-session/programs/:programId/analytics` | **N** | page/limit |

---

## Appendix B — Key Entities (reference)

- **`hdb_online_session`** (`OnlineSession`) — provider resource: `externalId` (Zoom meeting/**webinar id**), `joinUrl`, `registrationUrl`, `panelistUrl`, `startUrl` (never exposed), `status`, `launchMode`, `joinOpensMinutesBefore`, `requireRegistration`.
- **`hdb_program_registration_online_session`** — per-registrant extension: `externalRegistrantId`, `joinUrl`, `isPanelist`, `registrationType` (`video`/`audio`); unique active row per `(registration_id, online_session_id)`.
- **`program_user_attendance`** (`ProgramUserAttendance`) — attendance keyed by `registration_id` (primary), `user_id`, `email`, `session_id`; `is_attended`, `is_manually_checked_in`, `checked_in_at`, `checked_in_by_user_id`, `mode_of_joining`, `device_type`, append-only `attendance_events[]` (`source` ∈ JOIN_CLICK / ZOOM_WEBHOOK / MANUAL_ADMIN / QR_SCAN). **[NEW]** to add: **per-source signal retention** — for each source (`join_click`, `zoom`, `rm`, `coordinator`, `admin`) keep an independent `state` (`present`/`absent`/`unset`) + `markedBy` + `markedAt`, plus `RM_MARK` / `COORDINATOR_MARK` event sources, and a derived *effective* `is_attended`. Storage shape (columns-per-source vs. a normalized `attendance_signal` child table keyed by `(attendance_id, source)`) = Open Q1. `is_attended` becomes the derived effective value (Open Q2).
- **`background_jobs`** (`BackgroundJob`) — `type` (`BULK_QR_GENERATION` / `BULK_ZOOM_REGISTRATION`), `status`, `total`, `generated`, `skipped`, `failed`, `error_message`, `metadata` (jsonb — **[NEW]** to hold `failures[]` + `ineligible[]`), `completed_at`.
- **`hdb_zoom_analytics`** (`ZoomAnalytics`) — per-participant: `join_time`, `left_time`, `duration`, `rejoin_count`, `join_status`, `is_participant`, `is_absentee`, `is_late_comer`, `is_dropoff`, `mode_of_joining`, `device_type`, plus flags feeding `SessionAnalyticsKpi`.