# System Architecture

## High-level diagram

```mermaid
flowchart TB
  subgraph client [Client]
    Browser[Next.js App Router UI]
  end

  subgraph app [Application Layer]
    API[Route Handlers / Server Actions]
    Auth[NextAuth]
    Services[Domain Services]
    Env[Zod env validation]
  end

  subgraph data [Data Layer]
    MongoDB[(MongoDB)]
    Redis[(Redis - optional)]
  end

  subgraph external [External Integrations]
    SMTP[SMTP Server]
    IMAP[IMAP Server]
    OpenAI[OpenAI API]
    N8N[n8n Webhooks]
  end

  Browser --> API
  API --> Env
  API --> Auth
  API --> Services
  Services --> MongoDB
  Services --> Redis
  Services --> SMTP
  Services --> IMAP
  Services --> OpenAI
  Services --> N8N
```

## Technology stack

| Layer | Choice | Notes |
|-------|--------|-------|
| Framework | Next.js 16 (App Router) | Route Handlers, Server Components |
| Language | TypeScript | Strict mode (`tsconfig.json`) |
| Styling | Tailwind CSS v4 | PostCSS via `@tailwindcss/postcss` |
| Validation | Zod | Env vars (server); API/forms in later tasks |
| ODM | Mongoose | MongoDB connection via `src/lib/db.ts` |
| UI components | shadcn/ui | **Deferred** until after foundation (see decisions log) |
| Database | MongoDB | Primary store for CRM and marketing entities |
| Auth | NextAuth.js | Task 003 |
| Queue/cache | Redis (optional) | Campaign sends, IMAP sync jobs, rate limits |
| Automation | n8n | Outbound webhooks; not CRM of record |
| Email out | SMTP | Internal sending engine |
| Email in | IMAP | Business inbox sync |

## App foundation (Task 002 — implemented)

The runnable app lives at the **repository root** (not a nested package). Foundation includes:

| Piece | Location | Role |
|-------|----------|------|
| Home page | `src/app/page.tsx` | Foundation landing (RTL, Arabic `lang`) |
| Root layout | `src/app/layout.tsx` | Metadata, global styles |
| Health API | `src/app/api/health/route.ts` | Liveness + optional DB check |
| Env validation | `src/lib/env.ts` | Zod parse of server env (`server-only`) |
| DB connector | `src/lib/db.ts` | Mongoose singleton with dev cache |
| Constants | `src/lib/constants.ts` | App name, locales, phase label |
| Shared types | `src/types/index.ts` | Cross-cutting TS types |

**Env behavior:** `getEnv()` validates required variables when called from server code. Secrets are never exposed to the client (no `NEXT_PUBLIC_*` for credentials).

**Health behavior:** `GET /api/health` is public, dynamic, attempts MongoDB connect when env is valid, returns safe errors without leaking secrets.

## Authentication (Task 003 — implemented)

```mermaid
sequenceDiagram
  participant User
  participant Login as /login
  participant NextAuth as /api/auth
  participant Auth as auth.service
  participant DB as MongoDB users

  User->>Login: email + password
  Login->>NextAuth: signIn credentials
  NextAuth->>Auth: findUserByEmail + verifyPassword
  Auth->>DB: load user
  Auth-->>NextAuth: user id, role
  NextAuth-->>User: JWT session cookie
```

| Piece | Location | Role |
|-------|----------|------|
| NextAuth config | `src/lib/auth.ts` | Credentials provider, JWT callbacks |
| Auth route | `src/app/api/auth/[...nextauth]/route.ts` | GET/POST handlers |
| User model | `src/server/models/user.model.ts` | Mongoose `users` collection |
| Auth service | `src/server/services/auth.service.ts` | Login helpers, bootstrap admin |
| Middleware | `src/middleware.ts` | Protect pages + APIs; login redirect |
| Session types | `src/types/next-auth.d.ts` | `id`, `role` on session/JWT |
| Bootstrap | `src/app/api/admin/bootstrap/route.ts` | Initial admin (secret header) |

**Passwords:** bcrypt (12 rounds); only `passwordHash` stored.

**Roles:** `admin`, `manager`, `sales`, `viewer` — in JWT for future RBAC.

**Protected pages:** `/dashboard`, `/market`, `/segments`, `/companies`, `/people`, `/imports`, `/campaigns`, `/inbox`, `/ai`, `/leads`, `/pipeline`, `/settings`.

## Application structure (current under `/src`)

```
src/
  app/
    api/
      auth/[...nextauth]/route.ts
      admin/bootstrap/route.ts
      health/route.ts
    dashboard/page.tsx
    login/page.tsx, login-form.tsx
    layout.tsx, page.tsx, globals.css
  components/
    layout/          # app-shell, sidebar, topbar
    providers/       # SessionProvider
    ui/              # shadcn (future)
  lib/
    auth.ts, db.ts, env.ts, constants.ts
  middleware.ts
  server/
    models/              # Mongoose schemas (Task 004 — 20 collections + User)
      index.ts           # Barrel export
    services/auth.service.ts
    repositories/
  lib/
    enums.ts             # Shared enum constants for schemas
  types/
    index.ts, next-auth.d.ts
```

## Database / model layer (Task 004 — implemented)

| Layer | Location | Role |
|-------|----------|------|
| Models | `src/server/models/*.model.ts` | Mongoose schemas and indexes |
| Registry | `src/server/models/index.ts` | Single import point for all models |
| Enums | `src/lib/enums.ts` | Shared constants used in schemas |
| Connection | `src/lib/db.ts` | MongoDB via `connectDB()` |

**Domains covered:** companies, people, contact points, import (source/import_log), service matches, segments, templates, campaigns, email messages/replies, leads, opportunities, activities, tasks, AI classifications, automation logs, data quality, duplicate review, mailboxes.

**Import support:** `importType: simple | processed` on `Source` and `ImportLog`; optional GPT fields (`semanticNotes`, `service_matches`, confidence) on company/person/contact records.

## Import system (Task 005 — implemented)

```mermaid
flowchart LR
  UI["/imports UI"] --> API["/api/imports/*"]
  API --> Detector[import-detector]
  API --> Simple[simple-import.service]
  API --> Processed[processed-import.service]
  Simple --> MongoDB[(MongoDB)]
  Processed --> MongoDB
  Processed --> Maps[external ID maps]
```

| Component | Path |
|-----------|------|
| Spec | `docs/18-import-system.md` |
| Types | `src/types/import.ts` |
| Normalize | `src/lib/normalize.ts` |
| Excel parse | `src/server/services/excel-parser.service.ts` |
| Detect | `src/server/services/import-detector.service.ts` |
| Smart mapping | `src/server/services/import-mapping.service.ts` |
| Simple import | `src/server/services/simple-import.service.ts` |
| Processed import | `src/server/services/processed-import.service.ts` |
| Repository | `src/server/repositories/import.repository.ts` |

**Dependencies:** `xlsx`, `nanoid`.

## Market Database (Task 006 — implemented)

Browse/search/filter layer over imported market data — not full manual CRUD.

```mermaid
flowchart LR
  UI["/market/* UI"] --> API["/api/market/*"]
  API --> Market[market.service]
  Market --> MongoDB[(MongoDB)]
```

| Component | Path |
|-----------|------|
| Service | `src/server/services/market.service.ts` |
| Pagination | `src/lib/pagination.ts` |
| API helpers | `src/lib/api-response.ts` |
| Hub | `src/app/market/page.tsx` |
| Sub-nav | `src/components/market/market-sub-nav.tsx` |

**Domains:** companies list/detail, people list/detail, contact points. Profile pages surface related people, contacts, campaigns, inbox replies, leads, opportunities, and activities. Shared UI: `src/components/market/profile-tables.tsx`, `entity-quick-actions.tsx`, `bulk-segment-toolbar.tsx`. (Legacy `DataQualityIssue` / `DuplicateReview` models remain in DB; UI and import creation removed.)

## Segments (Task 007 — implemented; enhanced Task 018b)

Saved targeting over imported market data — **dynamic** (filters) or **static** (explicit member IDs).

```mermaid
flowchart LR
  UI["/segments UI"] --> API["/api/segments/*"]
  API --> Segments[segment.service]
  Segments --> MongoDB[(MongoDB)]
  Segments --> MarketData[(companies/people/contact_points)]
```

| Component | Path |
|-----------|------|
| Model | `src/server/models/segment.model.ts` |
| Service | `src/server/services/segment.service.ts` |
| APIs | `src/app/api/segments/**` |
| UI | `src/app/segments/**`, `src/components/segments/**` |

**Capabilities:** dynamic filters with DB-driven filter-options dropdowns; static segments with add/remove members; preview count/sample (dynamic only); members listing; bulk add from market tables; edit; soft deactivate.

## Email Templates (Task 008 — implemented)

Authoring and previewing reusable email templates for future campaigns.

```mermaid
flowchart LR
  UI["/email-templates UI"] --> API["/api/email-templates/*"]
  API --> Templates[email-template.service]
  Templates --> MongoDB[(MongoDB)]
```

| Component | Path |
|-----------|------|
| Model | `src/server/models/email-template.model.ts` |
| Service | `src/server/services/email-template.service.ts` |
| Template utils | `src/lib/template.ts` |
| APIs | `src/app/api/email-templates/**` |
| UI | `src/app/email-templates/**`, `src/components/email-templates/**` |

**Capabilities:** variable extraction/validation, render preview with sample data, archive lifecycle.

## Campaigns + Sending (Task 009–010)

Campaign orchestration layer connecting segments, templates, recipients, and stats foundation.

```mermaid
flowchart LR
  UI["/campaigns UI"] --> API["/api/campaigns/*"]
  API --> CampaignSvc[campaign.service]
  CampaignSvc --> Campaigns[(campaigns)]
  CampaignSvc --> Recipients[(campaign_recipients)]
  CampaignSvc --> Segments[(segments)]
  CampaignSvc --> Templates[(email_templates)]
```

| Component | Path |
|-----------|------|
| Models | `campaign.model.ts`, `campaign-recipient.model.ts` |
| Service | `src/server/services/campaign.service.ts` |
| APIs | `src/app/api/campaigns/**` |
| UI | `src/app/campaigns/**`, `src/components/campaigns/**` |

**Capabilities:** create draft campaigns, preview/generate recipients, deduplicate/skip invalid recipients, stats recalculation, synchronous SMTP batch sending, open/click tracking routes, and test-send verification page.

## Email transport and tracking (Task 010 — implemented)

```mermaid
flowchart LR
  CampaignUI["/campaigns/[id]"] --> SendAPI["/api/campaigns/:id/send-batch"]
  SendAPI --> SendSvc[email-sending.service]
  SendSvc --> Render[email-render.service]
  SendSvc --> SMTP[email-transport.service]
  SMTP --> Provider[(SMTP Server)]
  Render --> Track[tracking id + tracked links/pixel]
  Track --> Open["/api/email-tracking/open/:trackingId"]
  Track --> Click["/api/email-tracking/click/:trackingId"]
```

## Business Inbox (Task 011 — implemented foundation)

```mermaid
flowchart LR
  InboxUI["/inbox UI"] --> InboxAPI["/api/inbox/*"]
  InboxAPI --> InboxSvc[inbox.service]
  InboxSvc --> ImapSvc[imap.service]
  ImapSvc --> IMAP[(IMAP Server)]
  InboxSvc --> Replies[(email_replies)]
  InboxSvc --> Messages[(email_messages)]
  InboxSvc --> CRM[(companies/people/contact_points/campaigns)]
```

| Component | Path |
|-----------|------|
| IMAP service | `src/server/services/imap.service.ts` |
| Inbox service | `src/server/services/inbox.service.ts` |
| APIs | `src/app/api/inbox/**`, `src/app/api/email/verify-imap` |
| UI | `src/app/inbox/**`, `src/components/inbox/**` |

**Capabilities:** verify IMAP connection, sync recent inbox emails, deduplicate by message metadata, link replies to campaign/message/company/person when possible, and create reply activities.

## AI provider layer (Task 012 — reply classification)

```mermaid
flowchart LR
  InboxUI["/inbox + /ai/status"] --> AiAPI["/api/ai/*"]
  AiAPI --> AiSvc[ai-classification.service]
  AiSvc --> Factory[ai-provider.factory]
  Factory --> OpenAI[openai.provider]
  Factory --> Fallback[fallback.provider]
  Factory --> Gemini[gemini placeholder]
  Factory --> Anthropic[anthropic placeholder]
  AiSvc --> AiClass[(ai_classifications)]
  AiSvc --> Replies[(email_replies)]
```

| Component | Path |
|-----------|------|
| Provider interface | `src/server/ai/ai-provider.interface.ts` |
| Factory + status | `src/server/ai/ai-provider.factory.ts` |
| Providers | `src/server/ai/providers/*.ts` |
| Classification service | `src/server/services/ai-classification.service.ts` |
| APIs | `src/app/api/ai/**` |
| UI | `src/components/inbox/ai-classification-panel.tsx`, `src/app/ai/status` |

**Capabilities:** classify single/bulk inbox replies; preview classification without save; rule-based fallback without API keys; OpenAI when configured; persist `AiClassification` and update reply status.

## AI content generation (Task 021 Phase 2)

| Component | Path |
|-----------|------|
| Brand voice | `src/lib/ai-brand-voice.ts`, `SystemSetting.brandVoice` |
| White-label branding | `src/lib/brand-settings.ts`, `brand-settings.service.ts`, `BrandProvider`, `system_settings.branding` — see [28-white-label-branding.md](./28-white-label-branding.md) |
| Guard + audit | `src/server/services/ai-guard.service.ts`, `ai-action-log.model.ts` |
| Content generation | `src/server/services/ai-content.service.ts`, `ai-generation.client.ts` |
| Lead scoring | `src/server/services/ai-lead-scoring.service.ts` |
| Next best action | `src/server/services/ai-next-best-action.service.ts` |
| UI panels | `src/components/ai/**` |

**Capabilities:** email writer, template/sequence/segment generators, lead scoring (single + bulk), next-best-action recommendations, follow-up reply suggestions — all inject brand voice; log actions to `AiActionLog`; graceful fallback when AI disabled or provider unconfigured. Inbox classification (Task 012) unchanged.

See [26-ai-content-generation.md](./26-ai-content-generation.md).

## Dashboard Analytics (Task 014)

```mermaid
flowchart LR
  DashUI["/dashboard"] --> DashAPI["GET /api/dashboard"]
  DashAPI --> DashSvc[dashboard.service]
  DashSvc --> Market[(companies/people/contacts)]
  DashSvc --> Campaigns[(campaigns/recipients)]
  DashSvc --> Inbox[(email_replies/ai_classifications)]
  DashSvc --> CRM[(leads/opportunities/tasks/activities)]
```

| Component | Path |
|-----------|------|
| Service | `src/server/services/dashboard.service.ts` |
| API | `src/app/api/dashboard/route.ts` |
| UI | `src/components/dashboard/dashboard-client.tsx` |

Aggregates counts and rates across market, imports, campaigns, inbox, AI, leads, pipeline, and tasks. No external analytics service.

## Notifications & automation rules (Task 015)

```mermaid
flowchart LR
  Events[CRM events] --> Rules[automation-rules.service]
  Rules --> NotifSvc[notification.service]
  NotifSvc --> DB[(notifications)]
  UI[Bell + /notifications] --> NotifAPI["/api/notifications"]
  NotifAPI --> NotifSvc
```

Internal-only: no n8n or external webhooks. Event hooks use `triggerAutomation()` for non-blocking notification creation. See [17-notifications.md](./17-notifications.md).

## Leads & Pipeline (Task 013)

```mermaid
flowchart LR
  InboxUI --> ConvertAPI["/api/leads/from-email-reply"]
  ConvertAPI --> LeadSvc[lead.service]
  LeadSvc --> Rules[lead-rules]
  LeadSvc --> Leads[(leads)]
  LeadSvc --> Opps[(opportunities)]
  LeadSvc --> Tasks[(tasks)]
  PipelineUI["/pipeline"] --> OppAPI["/api/opportunities/*"]
  OppAPI --> OppSvc[opportunity.service]
```

| Component | Path |
|-----------|------|
| Lead rules | `src/lib/lead-rules.ts` |
| Services | `lead.service`, `opportunity.service`, `task.service` |
| APIs | `/api/leads/**`, `/api/opportunities/**`, `/api/tasks/**` |
| UI | `/leads`, `/pipeline`, `/tasks`, `/opportunities/[id]` |

Import alias: `@/*` → `src/*`.

## Settings & Administration (Task 016 — implemented)

| Layer | Location |
|-------|----------|
| Permissions | `src/lib/permissions.ts`, `src/lib/api-auth.ts` (`requireAdminApiSession`) |
| System settings | `src/server/models/system-setting.model.ts`, `settings.service.ts` |
| Users admin | `settings-user.service.ts` |
| Mailboxes | `mailbox-settings.service.ts`, `secret-crypto.ts` |
| APIs | `src/app/api/settings/**` |
| UI | `src/app/settings/**`, `src/components/settings/**` |

Admin-only: settings UI redirects non-admins; settings APIs return `403`. Default SMTP/IMAP env remains fallback; per-mailbox credentials encrypted at rest. AI API keys env-only; provider/model overridable in `system_settings`. See [18-settings-administration.md](./18-settings-administration.md).

## Architectural principles

1. **Modular domains:** companies, people, leads, campaigns, inbox—each with service + API + UI boundaries.
2. **Server-first:** sensitive operations (send email, import, webhooks) run on server; secrets never reach client bundles.
3. **Env-driven config:** no hardcoded integration endpoints; see [10-environment-variables.md](./10-environment-variables.md).
4. **Idempotent imports and webhooks:** safe retries for automation and bulk load.
5. **Auditability:** created/updated metadata and activity logs on critical entities.

## Production readiness (Task 018)

| Piece | Location |
|-------|----------|
| Permissions | `src/lib/permissions.ts`, `src/lib/api-guards.ts` |
| API errors | `src/lib/api-response.ts` |
| HTML safety | `src/lib/html-safety.ts` |
| Health | `src/app/api/health/route.ts` |
| Index sync | `scripts/sync-indexes.ts` |

See [20-production-readiness.md](./20-production-readiness.md), [21-smoke-test-checklist.md](./21-smoke-test-checklist.md).

## Request flow (example: health check)

1. Client or monitor calls `GET /api/health`
2. Route handler checks DB, SMTP/IMAP env presence, AI runtime status
3. Returns JSON with `status`, `app`, `version`, `timestamp`, `database`, integrations — no credentials

## Request flow (example: create lead — future)

1. Authenticated user submits form → Server Action or `POST /api/leads`
2. Handler validates input (Zod)
3. `LeadService.create()` in `src/server/services` persists to MongoDB
4. Optional: fire `N8N_WEBHOOK_LEAD_CREATED` if configured
5. Return lead DTO to UI; update pipeline views

## Security boundary

- All `/api/*` routes require session unless explicitly public (**health check only** today).
- Webhook receivers validate signatures or shared secrets when added (Task 019).
- See [11-security.md](./11-security.md).

## Deployment target

- Node hosting compatible with Next.js (Vercel, container, or VPS)
- MongoDB Atlas or self-hosted MongoDB
- Secrets via platform env injection

## Related documents

- [02-database-schema.md](./02-database-schema.md)
- [03-api-specification.md](./03-api-specification.md)
- [12-deployment.md](./12-deployment.md)
- [09-n8n-integration.md](./09-n8n-integration.md)
- [18-settings-administration.md](./18-settings-administration.md)