Data Flow

This page details the data flow for the primary use case: submitting a dissertation and receiving an automated quote.

Intake Submission Flow

sequenceDiagram
    participant Browser
    participant NextJS as Next.js Frontend
    participant APIGW as API Gateway
    participant PresignedURL as getPresignedUrl
    participant S3
    participant SubmitIntake as submitIntake
    participant DynamoDB
    participant ProcessSubmission as processSubmission
    participant DocAnalyzer as documentAnalyzer
    participant QuoteCalc as quoteCalculator
    participant Pipedrive as pipedriveClient
    participant GetSubmission as getSubmission

    Browser->>NextJS: User fills form & selects .docx
    NextJS->>APIGW: POST /intake/presigned-url {fileName}
    APIGW->>PresignedURL: Invoke
    PresignedURL->>S3: Generate presigned PUT URL
    S3-->>PresignedURL: {uploadUrl, s3Key}
    PresignedURL-->>NextJS: {uploadUrl, s3Key, submissionId}

    NextJS->>S3: PUT uploadUrl (direct upload of .docx)
    S3-->>NextJS: 200 OK

    NextJS->>APIGW: POST /intake/submit {submissionId, name, email, serviceType, s3Key, ...}
    APIGW->>SubmitIntake: Invoke
    SubmitIntake->>DynamoDB: PutItem (status: "processing")
    SubmitIntake->>ProcessSubmission: Lambda.invokeAsync()
    SubmitIntake-->>NextJS: {submissionId, status: "processing"}

    Note over ProcessSubmission: Async processing begins

    ProcessSubmission->>DocAnalyzer: Invoke
    DocAnalyzer->>S3: GetObject (download .docx)
    DocAnalyzer-->>ProcessSubmission: {textWordCount, refsWordCount, tables, figures, frontMatter}

    ProcessSubmission->>QuoteCalc: Invoke
    QuoteCalc->>DynamoDB: GetItem (rate schedule from config table)
    QuoteCalc-->>ProcessSubmission: {lineItems, subtotal, adminFee, total}

    ProcessSubmission->>Pipedrive: Invoke
    Pipedrive-->>ProcessSubmission: {dealId, dealUrl}

    ProcessSubmission->>DynamoDB: UpdateItem (status: "completed", analysis, quote, dealId)

    Note over Browser: Frontend polls for status

    loop Poll every 2-3 seconds
        NextJS->>APIGW: GET /intake/submission/{submissionId}
        APIGW->>GetSubmission: Invoke
        GetSubmission->>DynamoDB: GetItem
        DynamoDB-->>GetSubmission: Submission record
        GetSubmission-->>NextJS: {status, analysis, quote, ...}
    end

    NextJS->>Browser: Display quote to user

Key Design Decisions

Presigned URL Upload

The browser uploads the document directly to S3 using a presigned URL rather than streaming through API Gateway and Lambda. This avoids the API Gateway payload size limit (10 MB) and the Lambda payload limit (6 MB for synchronous invocation), allowing files up to 50 MB.

Async Processing

The submitIntake function returns immediately after storing the submission and invoking processSubmission asynchronously. This keeps the user-facing API response fast (sub-second) while document analysis and quote calculation may take several seconds.

The processSubmission function is invoked with InvocationType: 'Event', which means Lambda handles the retry logic. If it fails, Lambda retries up to 2 times.

Frontend Polling

The frontend polls GET /intake/submission/{submissionId} every 2-3 seconds until the status changes from "processing" to "completed" or "error". This is simpler than setting up WebSocket connections for a one-time status update.

Submission States

Status	Description
processing	Submission received, async processing underway
completed	Analysis done, quote calculated, Pipedrive deal created
error	Processing failed (check CloudWatch logs for details)

Video Consultation Flow

sequenceDiagram
    participant Browser
    participant NextJS as Next.js Frontend
    participant APIGW as API Gateway
    participant TokenGen as tokenGenerator
    participant LiveKit as LiveKit Server

    Browser->>NextJS: Navigate to /room
    NextJS->>Browser: Room join form
    Browser->>NextJS: Enter name and room name
    NextJS->>APIGW: GET /token?room=xyz&participant=John
    APIGW->>TokenGen: Invoke
    TokenGen-->>NextJS: {token, url, room, participant}
    NextJS->>LiveKit: Connect via WSS with JWT token
    LiveKit-->>Browser: Real-time audio/video stream

The token generator creates a short-lived JWT with a room grant scoped to the requested room. The browser then connects directly to the LiveKit server over WebSocket.