Part 3: Platform Enhancements - Implementation Plan

Version: 1.0 Date: 2025-11-26 Status: Planning Priority: MEDIUM Estimated Duration: 11-14 weeks

Executive Summary

This document provides a comprehensive implementation plan for platform enhancements that improve the VoiceAssist foundation. These enhancements focus on design consistency, security hardening, search quality, and continuous improvement systems.

Scope:

1. Design System Improvements (2-3 weeks) - Complete design token system and documentation
2. Client-Side Security (2 weeks) - PHI protection and audit enhancements
3. Advanced RAG Techniques (4-5 weeks) - Hybrid search, re-ranking, contextual retrieval
4. Continuous Learning System (3-4 weeks) - Feedback collection and model improvement

Total Estimated Effort: 11-14 weeks with 2 developers

Table of Contents

1. Current State Analysis
2. Design System Improvements
3. Client-Side Security
4. Advanced RAG Techniques
5. Continuous Learning System
6. Implementation Phases
7. Technical Architecture
8. Risk Assessment
9. Success Metrics
10. Appendices

Current State Analysis

What's Already Implemented

What's Missing (This Plan)

1. Design System Improvements

1.1 Overview

Objective: Establish a comprehensive, documented design system that ensures UI consistency across all VoiceAssist applications.

Current State: Basic design tokens exist (colors, spacing, typography) with Storybook configured and 12 component stories.

Target State: Complete design system with animations, medical-themed components, interactive documentation, and WCAG AAA compliance.

1.2 Technical Architecture

packages/
├── design-tokens/
│   └── src/
│       ├── colors.ts        ✅ Complete
│       ├── spacing.ts       ✅ Complete
│       ├── typography.ts    ✅ Complete
│       ├── animations.ts    🔲 NEW - Motion tokens
│       ├── shadows.ts       🔲 NEW - Elevation system
│       ├── breakpoints.ts   🔲 NEW - Responsive breakpoints
│       └── index.ts
├── ui/
│   ├── .storybook/          ✅ Configured
│   └── src/
│       ├── components/
│       │   ├── primitives/  ✅ Button, Input, etc.
│       │   └── medical/     🔲 NEW - VitalSignCard, MedicationList, etc.
│       ├── stories/
│       │   ├── *.stories.tsx ✅ 12 stories exist
│       │   └── medical/     🔲 NEW - Medical component stories
│       └── providers/
│           └── ThemeProvider.tsx ✅ Complete
└── tailwind-config/
    └── tailwind.preset.js   ✅ Shared config

1.3 Component Specifications

1.3.1 Animation Tokens

File: packages/design-tokens/src/animations.ts

/**
 * Animation tokens following medical UI best practices:
 * - Reduced motion support
 * - Subtle, non-distracting transitions
 * - Clear feedback for interactions
 */

export const durations = {
  instant: "0ms",
  fast: "100ms",
  normal: "200ms",
  slow: "300ms",
  slower: "500ms",
} as const;

export const easings = {
  linear: "linear",
  easeIn: "cubic-bezier(0.4, 0, 1, 1)",
  easeOut: "cubic-bezier(0, 0, 0.2, 1)",
  easeInOut: "cubic-bezier(0.4, 0, 0.2, 1)",
  spring: "cubic-bezier(0.175, 0.885, 0.32, 1.275)",
} as const;

export const animations = {
  fadeIn: {
    keyframes: { from: { opacity: 0 }, to: { opacity: 1 } },
    duration: durations.normal,
    easing: easings.easeOut,
  },
  slideUp: {
    keyframes: {
      from: { transform: "translateY(8px)", opacity: 0 },
      to: { transform: "translateY(0)", opacity: 1 },
    },
    duration: durations.normal,
    easing: easings.easeOut,
  },
  pulse: {
    keyframes: {
      "0%, 100%": { opacity: 1 },
      "50%": { opacity: 0.5 },
    },
    duration: durations.slower,
    easing: easings.easeInOut,
    iterationCount: "infinite",
  },
  // Medical-specific: Alert pulse for critical values
  criticalPulse: {
    keyframes: {
      "0%, 100%": {
        boxShadow: "0 0 0 0 rgba(239, 68, 68, 0.4)",
        borderColor: "var(--color-error-500)",
      },
      "50%": {
        boxShadow: "0 0 0 8px rgba(239, 68, 68, 0)",
        borderColor: "var(--color-error-600)",
      },
    },
    duration: "1.5s",
    easing: easings.easeInOut,
    iterationCount: "infinite",
  },
} as const;

// Reduced motion variants
export const reducedMotionAnimations = {
  fadeIn: { ...animations.fadeIn, duration: durations.instant },
  slideUp: { ...animations.fadeIn, duration: durations.instant }, // Fallback to fade
  pulse: null, // Disable pulsing animations
  criticalPulse: null,
} as const;

1.3.2 Shadow/Elevation Tokens

File: packages/design-tokens/src/shadows.ts

/**
 * Elevation system for depth and hierarchy
 * Based on Material Design principles, adapted for medical UI
 */

export const shadows = {
  none: "none",
  sm: "0 1px 2px 0 rgb(0 0 0 / 0.05)",
  md: "0 4px 6px -1px rgb(0 0 0 / 0.1), 0 2px 4px -2px rgb(0 0 0 / 0.1)",
  lg: "0 10px 15px -3px rgb(0 0 0 / 0.1), 0 4px 6px -4px rgb(0 0 0 / 0.1)",
  xl: "0 20px 25px -5px rgb(0 0 0 / 0.1), 0 8px 10px -6px rgb(0 0 0 / 0.1)",
  // Medical-specific: Focus ring for accessibility
  focus: "0 0 0 3px var(--color-primary-500 / 0.3)",
  focusError: "0 0 0 3px var(--color-error-500 / 0.3)",
} as const;

export const elevation = {
  surface: shadows.none, // Base level (cards, panels)
  raised: shadows.sm, // Slightly elevated (buttons)
  overlay: shadows.md, // Dropdowns, tooltips
  modal: shadows.lg, // Modals, dialogs
  floating: shadows.xl, // FABs, floating elements
} as const;

1.3.3 Medical UI Components

File: packages/ui/src/components/medical/VitalSignCard.tsx

/**
 * VitalSignCard - Displays a single vital sign with status indication
 *
 * Features:
 * - Color-coded status (normal, warning, critical)
 * - Trend indicator (up, down, stable)
 * - Accessibility: High contrast, screen reader friendly
 * - Animation: Critical pulse for out-of-range values
 */

import React from "react";
import { cn } from "../../utils/cn";

export interface VitalSignCardProps {
  label: string;
  value: number | string;
  unit: string;
  status: "normal" | "warning" | "critical";
  trend?: "up" | "down" | "stable";
  normalRange?: { min: number; max: number };
  timestamp?: Date;
  className?: string;
}

const statusStyles = {
  normal: "bg-success-50 border-success-200 text-success-800",
  warning: "bg-warning-50 border-warning-200 text-warning-800",
  critical: "bg-error-50 border-error-200 text-error-800 animate-critical-pulse",
};

const trendIcons = {
  up: "↑",
  down: "↓",
  stable: "→",
};

export const VitalSignCard: React.FC<VitalSignCardProps> = ({
  label,
  value,
  unit,
  status,
  trend,
  normalRange,
  timestamp,
  className,
}) => {
  return (
    <div
      className={cn("rounded-lg border-2 p-4 transition-colors", statusStyles[status], className)}
      role="region"
      aria-label={`${label}: ${value} ${unit}, status: ${status}`}
    >
      <div className="flex items-center justify-between">
        <span className="text-sm font-medium uppercase tracking-wide opacity-75">{label}</span>
        {trend && (
          <span className="text-lg" aria-label={`Trend: ${trend}`}>
            {trendIcons[trend]}
          </span>
        )}
      </div>

      <div className="mt-2 flex items-baseline gap-1">
        <span className="text-3xl font-bold tabular-nums">{value}</span>
        <span className="text-sm opacity-75">{unit}</span>
      </div>

      {normalRange && (
        <div className="mt-2 text-xs opacity-60">
          Normal: {normalRange.min}-{normalRange.max} {unit}
        </div>
      )}

      {timestamp && <div className="mt-1 text-xs opacity-50">{timestamp.toLocaleTimeString()}</div>}
    </div>
  );
};

File: packages/ui/src/components/medical/MedicationList.tsx

/**
 * MedicationList - Displays patient medications with interaction warnings
 */

import React from "react";

export interface Medication {
  id: string;
  name: string;
  dosage: string;
  frequency: string;
  route: string;
  startDate: Date;
  endDate?: Date;
  prescriber?: string;
  interactions?: string[];
  contraindications?: string[];
}

export interface MedicationListProps {
  medications: Medication[];
  showInteractions?: boolean;
  onMedicationClick?: (med: Medication) => void;
  className?: string;
}

export const MedicationList: React.FC<MedicationListProps> = ({
  medications,
  showInteractions = true,
  onMedicationClick,
  className,
}) => {
  const hasInteractions = medications.some((m) => m.interactions?.length);

  return (
    <div className={className}>
      {hasInteractions && showInteractions && (
        <div className="mb-4 rounded-lg border-2 border-warning-300 bg-warning-50 p-3" role="alert">
          <strong className="text-warning-800">Drug Interactions Detected</strong>
          <p className="text-sm text-warning-700">Review potential interactions below</p>
        </div>
      )}

      <ul className="divide-y divide-neutral-200 dark:divide-neutral-700">
        {medications.map((med) => (
          <li
            key={med.id}
            className="py-3 hover:bg-neutral-50 dark:hover:bg-neutral-800 cursor-pointer rounded px-2"
            onClick={() => onMedicationClick?.(med)}
            role="button"
            tabIndex={0}
            onKeyDown={(e) => e.key === "Enter" && onMedicationClick?.(med)}
          >
            <div className="flex items-center justify-between">
              <div>
                <span className="font-semibold text-neutral-900 dark:text-neutral-100">{med.name}</span>
                <span className="ml-2 text-sm text-neutral-600 dark:text-neutral-400">{med.dosage}</span>
              </div>
              {med.interactions?.length ? (
                <span className="rounded-full bg-warning-100 px-2 py-0.5 text-xs font-medium text-warning-800">
                  {med.interactions.length} interaction{med.interactions.length > 1 ? "s" : ""}
                </span>
              ) : null}
            </div>
            <div className="mt-1 text-sm text-neutral-500">
              {med.frequency} · {med.route}
            </div>
          </li>
        ))}
      </ul>
    </div>
  );
};

1.4 Implementation Tasks

1.5 Deliverables

1. packages/design-tokens/src/animations.ts - Animation token definitions
2. packages/design-tokens/src/shadows.ts - Elevation system
3. packages/design-tokens/src/breakpoints.ts - Responsive breakpoints
4. packages/ui/src/components/medical/* - 5+ medical UI components
5. packages/ui/src/stories/medical/* - Storybook stories with docs
6. Updated packages/ui/README.md with usage guidelines
7. Storybook deployment at storybook.voiceassist.dev (optional)

2. Client-Side Security

2.1 Overview

Objective: Extend HIPAA-compliant security to the frontend with PHI detection, encrypted storage, and comprehensive audit trails.

Current State: Backend has PHI detection (phi_detector.py), redaction middleware, and audit logging. Frontend has no client-side PHI protection.

Target State: Client-side PHI detection with warnings, encrypted IndexedDB storage, and session audit trails synced to backend.

2.2 Technical Architecture

apps/web-app/src/
├── services/
│   ├── phi/
│   │   ├── PhiDetector.ts           🔲 NEW - Client-side PHI detection
│   │   ├── PhiWarningDialog.tsx     🔲 NEW - Warning UI component
│   │   └── patterns.ts              🔲 NEW - PHI regex patterns
│   ├── storage/
│   │   ├── EncryptedStorage.ts      🔲 NEW - Encrypted IndexedDB wrapper
│   │   ├── CryptoUtils.ts           🔲 NEW - Web Crypto API utilities
│   │   └── StorageSchema.ts         🔲 NEW - Schema definitions
│   └── audit/
│       ├── AuditTrail.ts            🔲 NEW - Client-side audit logger
│       ├── SessionRecorder.ts       🔲 NEW - Session activity recorder
│       └── AuditSync.ts             🔲 NEW - Background sync to backend
├── hooks/
│   ├── usePhiDetection.ts           🔲 NEW - PHI detection hook
│   ├── useEncryptedStorage.ts       🔲 NEW - Encrypted storage hook
│   └── useAuditTrail.ts             🔲 NEW - Audit trail hook
└── components/
    └── security/
        ├── PhiWarningBanner.tsx     🔲 NEW - Warning banner component
        └── SessionActivityLog.tsx   🔲 NEW - Activity log viewer

2.3 Component Specifications

2.3.1 Client-Side PHI Detector

File: apps/web-app/src/services/phi/PhiDetector.ts

/**
 * Client-Side PHI Detection Service
 *
 * Mirrors backend PHI detection for real-time warnings before submission.
 * Uses pattern matching similar to services/api-gateway/app/services/phi_detector.py
 *
 * IMPORTANT: This is a defensive layer. Backend validation is still authoritative.
 */

export interface PhiDetectionResult {
  containsPhi: boolean;
  phiTypes: PhiType[];
  confidence: number;
  matches: PhiMatch[];
}

export interface PhiMatch {
  type: PhiType;
  value: string;
  startIndex: number;
  endIndex: number;
  redacted: string;
}

export type PhiType = "ssn" | "phone" | "email" | "mrn" | "account" | "ip_address" | "dob" | "name";

// Pattern definitions matching backend
const PHI_PATTERNS: Record<PhiType, RegExp> = {
  ssn: /\b\d{3}[- ]?\d{2}[- ]?\d{4}\b/g,
  phone: /\b(?:\+?1[-.\s]?)?\(?\d{3}\)?[-.\s]?\d{3}[-.\s]?\d{4}\b/g,
  email: /\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b/g,
  mrn: /\b(?:MRN|mrn|medical record|record number)[\s:-]?\d{6,}\b/gi,
  account: /\b(?:ACCT|acct|account)[\s:-]?\d{6,}\b/gi,
  ip_address: /\b(?:\d{1,3}\.){3}\d{1,3}\b/g,
  dob: /\b(?:born|dob|date of birth|birthday)[\s:]?(?:0?[1-9]|1[0-2])[/-](?:0?[1-9]|[12][0-9]|3[01])[/-](?:19|20)\d{2}\b/gi,
  name: /\b[A-Z][a-z]+ [A-Z][a-z]+\b/g,
};

// Medical terms to exclude from name detection
const MEDICAL_TERMS = new Set([
  "heart disease",
  "blood pressure",
  "diabetes mellitus",
  "atrial fibrillation",
  "chronic kidney",
  "coronary artery",
  "pulmonary embolism",
  "myocardial infarction",
  // ... extend as needed
]);

export class PhiDetector {
  /**
   * Detect PHI in text
   */
  detect(text: string): PhiDetectionResult {
    if (!text) {
      return { containsPhi: false, phiTypes: [], confidence: 1, matches: [] };
    }

    const matches: PhiMatch[] = [];
    const phiTypes = new Set<PhiType>();

    for (const [type, pattern] of Object.entries(PHI_PATTERNS) as [PhiType, RegExp][]) {
      // Reset regex state
      pattern.lastIndex = 0;

      let match: RegExpExecArray | null;
      while ((match = pattern.exec(text)) !== null) {
        // Filter out medical terms for name detection
        if (type === "name" && MEDICAL_TERMS.has(match[0].toLowerCase())) {
          continue;
        }

        matches.push({
          type,
          value: match[0],
          startIndex: match.index,
          endIndex: match.index + match[0].length,
          redacted: this.redactValue(type, match[0]),
        });
        phiTypes.add(type);
      }
    }

    return {
      containsPhi: matches.length > 0,
      phiTypes: Array.from(phiTypes),
      confidence: 0.8, // Pattern matching confidence
      matches,
    };
  }

  /**
   * Sanitize text by redacting detected PHI
   */
  sanitize(text: string): string {
    const result = this.detect(text);
    let sanitized = text;

    // Process matches in reverse order to preserve indices
    const sortedMatches = [...result.matches].sort((a, b) => b.startIndex - a.startIndex);

    for (const match of sortedMatches) {
      sanitized = sanitized.slice(0, match.startIndex) + match.redacted + sanitized.slice(match.endIndex);
    }

    return sanitized;
  }

  private redactValue(type: PhiType, value: string): string {
    return `[${type.toUpperCase()}_REDACTED]`;
  }
}

// Singleton instance
export const phiDetector = new PhiDetector();

2.3.2 PHI Detection Hook

File: apps/web-app/src/hooks/usePhiDetection.ts

import { useState, useCallback, useMemo } from "react";
import { phiDetector, PhiDetectionResult } from "../services/phi/PhiDetector";
import { useDebounce } from "./useDebounce";

interface UsePhiDetectionOptions {
  debounceMs?: number;
  onPhiDetected?: (result: PhiDetectionResult) => void;
}

export function usePhiDetection(options: UsePhiDetectionOptions = {}) {
  const { debounceMs = 300, onPhiDetected } = options;

  const [text, setText] = useState("");
  const [result, setResult] = useState<PhiDetectionResult | null>(null);
  const [showWarning, setShowWarning] = useState(false);

  const debouncedText = useDebounce(text, debounceMs);

  // Run detection when debounced text changes
  useMemo(() => {
    if (debouncedText) {
      const detection = phiDetector.detect(debouncedText);
      setResult(detection);

      if (detection.containsPhi) {
        setShowWarning(true);
        onPhiDetected?.(detection);
      }
    } else {
      setResult(null);
      setShowWarning(false);
    }
  }, [debouncedText, onPhiDetected]);

  const checkText = useCallback((newText: string) => {
    setText(newText);
  }, []);

  const sanitizeText = useCallback(() => {
    return phiDetector.sanitize(text);
  }, [text]);

  const dismissWarning = useCallback(() => {
    setShowWarning(false);
  }, []);

  const acknowledgeAndProceed = useCallback(() => {
    // Log acknowledgment for audit
    console.info("[PHI] User acknowledged PHI warning and proceeded");
    setShowWarning(false);
    return text; // Return original text if user chooses to proceed
  }, [text]);

  return {
    checkText,
    result,
    showWarning,
    dismissWarning,
    sanitizeText,
    acknowledgeAndProceed,
  };
}

2.3.3 Encrypted Storage Service

File: apps/web-app/src/services/storage/EncryptedStorage.ts

/**
 * Encrypted IndexedDB Storage
 *
 * Uses Web Crypto API for AES-GCM encryption of sensitive data.
 * Keys are derived from user authentication tokens.
 *
 * Use cases:
 * - Offline voice recordings awaiting sync
 * - Cached clinical context
 * - Session state
 */

import { openDB, DBSchema, IDBPDatabase } from "idb";

interface EncryptedStorageSchema extends DBSchema {
  "encrypted-data": {
    key: string;
    value: {
      id: string;
      encrypted: ArrayBuffer;
      iv: Uint8Array;
      timestamp: number;
      metadata?: Record<string, unknown>;
    };
  };
  "session-audit": {
    key: number;
    value: {
      id: number;
      action: string;
      timestamp: number;
      details: Record<string, unknown>;
      synced: boolean;
    };
    indexes: { "by-synced": boolean };
  };
}

export class EncryptedStorage {
  private db: IDBPDatabase<EncryptedStorageSchema> | null = null;
  private encryptionKey: CryptoKey | null = null;

  async init(userToken: string): Promise<void> {
    // Derive encryption key from user token
    this.encryptionKey = await this.deriveKey(userToken);

    // Open IndexedDB
    this.db = await openDB<EncryptedStorageSchema>("voiceassist-secure", 1, {
      upgrade(db) {
        db.createObjectStore("encrypted-data", { keyPath: "id" });

        const auditStore = db.createObjectStore("session-audit", {
          keyPath: "id",
          autoIncrement: true,
        });
        auditStore.createIndex("by-synced", "synced");
      },
    });
  }

  private async deriveKey(token: string): Promise<CryptoKey> {
    const encoder = new TextEncoder();
    const keyMaterial = await crypto.subtle.importKey("raw", encoder.encode(token), "PBKDF2", false, ["deriveKey"]);

    return crypto.subtle.deriveKey(
      {
        name: "PBKDF2",
        salt: encoder.encode("voiceassist-salt-v1"), // Static salt is OK for this use case
        iterations: 100000,
        hash: "SHA-256",
      },
      keyMaterial,
      { name: "AES-GCM", length: 256 },
      false,
      ["encrypt", "decrypt"],
    );
  }

  async store(id: string, data: unknown, metadata?: Record<string, unknown>): Promise<void> {
    if (!this.db || !this.encryptionKey) {
      throw new Error("EncryptedStorage not initialized");
    }

    const encoder = new TextEncoder();
    const iv = crypto.getRandomValues(new Uint8Array(12));

    const encrypted = await crypto.subtle.encrypt(
      { name: "AES-GCM", iv },
      this.encryptionKey,
      encoder.encode(JSON.stringify(data)),
    );

    await this.db.put("encrypted-data", {
      id,
      encrypted,
      iv,
      timestamp: Date.now(),
      metadata,
    });
  }

  async retrieve<T>(id: string): Promise<T | null> {
    if (!this.db || !this.encryptionKey) {
      throw new Error("EncryptedStorage not initialized");
    }

    const record = await this.db.get("encrypted-data", id);
    if (!record) return null;

    const decrypted = await crypto.subtle.decrypt(
      { name: "AES-GCM", iv: record.iv },
      this.encryptionKey,
      record.encrypted,
    );

    const decoder = new TextDecoder();
    return JSON.parse(decoder.decode(decrypted)) as T;
  }

  async delete(id: string): Promise<void> {
    if (!this.db) throw new Error("EncryptedStorage not initialized");
    await this.db.delete("encrypted-data", id);
  }

  async logAuditEvent(action: string, details: Record<string, unknown>): Promise<void> {
    if (!this.db) throw new Error("EncryptedStorage not initialized");

    await this.db.add("session-audit", {
      id: Date.now(), // Will be overwritten by autoIncrement
      action,
      timestamp: Date.now(),
      details,
      synced: false,
    });
  }

  async getUnsyncedAuditEvents(): Promise<
    Array<{
      id: number;
      action: string;
      timestamp: number;
      details: Record<string, unknown>;
    }>
  > {
    if (!this.db) throw new Error("EncryptedStorage not initialized");
    return this.db.getAllFromIndex("session-audit", "by-synced", false);
  }

  async markAuditEventsSynced(ids: number[]): Promise<void> {
    if (!this.db) throw new Error("EncryptedStorage not initialized");

    const tx = this.db.transaction("session-audit", "readwrite");
    for (const id of ids) {
      const event = await tx.store.get(id);
      if (event) {
        await tx.store.put({ ...event, synced: true });
      }
    }
    await tx.done;
  }

  async clear(): Promise<void> {
    if (!this.db) throw new Error("EncryptedStorage not initialized");
    await this.db.clear("encrypted-data");
    await this.db.clear("session-audit");
  }
}

export const encryptedStorage = new EncryptedStorage();

2.3.4 Session Audit Trail

File: apps/web-app/src/services/audit/AuditTrail.ts

/**
 * Session Audit Trail
 *
 * Tracks user actions for HIPAA compliance and security monitoring.
 * Stores locally and syncs to backend audit service.
 */

import { encryptedStorage } from "../storage/EncryptedStorage";

export type AuditAction =
  | "session_start"
  | "session_end"
  | "message_sent"
  | "message_received"
  | "phi_warning_shown"
  | "phi_warning_acknowledged"
  | "phi_warning_dismissed"
  | "clinical_context_set"
  | "clinical_context_cleared"
  | "voice_mode_started"
  | "voice_mode_ended"
  | "file_uploaded"
  | "export_requested"
  | "navigation"
  | "error";

export interface AuditEvent {
  action: AuditAction;
  timestamp: number;
  sessionId: string;
  userId?: string;
  details: Record<string, unknown>;
}

class AuditTrail {
  private sessionId: string;
  private userId: string | null = null;
  private syncInterval: ReturnType<typeof setInterval> | null = null;

  constructor() {
    this.sessionId = this.generateSessionId();
  }

  private generateSessionId(): string {
    return `${Date.now()}-${Math.random().toString(36).substring(2, 9)}`;
  }

  init(userId?: string): void {
    this.userId = userId ?? null;
    this.log("session_start", { userAgent: navigator.userAgent });

    // Sync every 30 seconds
    this.syncInterval = setInterval(() => this.sync(), 30000);

    // Sync on page unload
    window.addEventListener("beforeunload", () => {
      this.log("session_end", {});
      this.sync(); // Best effort sync
    });
  }

  log(action: AuditAction, details: Record<string, unknown>): void {
    const event: AuditEvent = {
      action,
      timestamp: Date.now(),
      sessionId: this.sessionId,
      userId: this.userId ?? undefined,
      details,
    };

    // Store locally
    encryptedStorage
      .logAuditEvent(action, {
        ...details,
        sessionId: this.sessionId,
        userId: this.userId,
      })
      .catch(console.error);

    // Also log to console in development
    if (process.env.NODE_ENV === "development") {
      console.debug("[Audit]", action, details);
    }
  }

  async sync(): Promise<void> {
    try {
      const unsyncedEvents = await encryptedStorage.getUnsyncedAuditEvents();
      if (unsyncedEvents.length === 0) return;

      // POST to backend audit endpoint
      const response = await fetch("/api/audit/batch", {
        method: "POST",
        headers: { "Content-Type": "application/json" },
        body: JSON.stringify({ events: unsyncedEvents }),
        credentials: "include",
      });

      if (response.ok) {
        await encryptedStorage.markAuditEventsSynced(unsyncedEvents.map((e) => e.id));
      }
    } catch (error) {
      console.error("[Audit] Sync failed:", error);
    }
  }

  destroy(): void {
    if (this.syncInterval) {
      clearInterval(this.syncInterval);
    }
  }
}

export const auditTrail = new AuditTrail();

2.4 Implementation Tasks

2.5 Deliverables

1. apps/web-app/src/services/phi/* - PHI detection service and patterns
2. apps/web-app/src/services/storage/* - Encrypted IndexedDB storage
3. apps/web-app/src/services/audit/* - Audit trail service with sync
4. apps/web-app/src/hooks/usePhi*.ts - React hooks for security features
5. apps/web-app/src/components/security/* - Warning banners and activity log
6. Backend /api/audit/batch endpoint for audit sync
7. Unit and E2E tests with >80% coverage

3. Advanced RAG Techniques

3.1 Overview

Objective: Significantly improve search quality through hybrid search, re-ranking, and medical-domain optimizations.

Current State: Vector-only search using Qdrant with OpenAI embeddings (search_aggregator.py). No lexical search, no re-ranking.

Target State: Hybrid search (semantic + BM25), cross-encoder re-ranking, medical synonym expansion, and metadata filtering.

3.2 Technical Architecture

services/api-gateway/app/services/
├── search/
│   ├── search_aggregator.py       ✅ Exists - Vector search only
│   ├── hybrid_search.py           🔲 NEW - Combines vector + lexical
│   ├── bm25_index.py              🔲 NEW - BM25 lexical search
│   ├── cross_encoder.py           🔲 NEW - Re-ranking service
│   ├── query_expansion.py         🔲 NEW - Medical synonym expansion
│   └── contextual_retrieval.py    🔲 NEW - Chunk context enhancement
├── medical/
│   ├── synonym_database.py        🔲 NEW - UMLS/SNOMED synonyms
│   └── abbreviation_expander.py   🔲 NEW - Medical abbreviations
└── rag_service.py                 ✅ Exists - Main RAG orchestration

External Dependencies:
├── Meilisearch (or Elasticsearch) - Lexical search engine
├── sentence-transformers          - Cross-encoder models
└── UMLS API (optional)            - Medical synonyms

3.3 Hybrid Search Architecture

┌─────────────────────────────────────────────────────────────────────────┐
│                            User Query                                    │
│                   "What are the contraindications for ASA?"             │
└────────────────────────────────┬────────────────────────────────────────┘
                                 │
                                 ▼
┌─────────────────────────────────────────────────────────────────────────┐
│                        Query Preprocessor                                │
│  ┌───────────────┐  ┌─────────────────┐  ┌─────────────────────────┐  │
│  │ Query         │  │ Abbreviation    │  │ Synonym Expansion       │  │
│  │ Cleaning      │──▶│ Expansion       │──▶│ (UMLS/SNOMED)           │  │
│  │               │  │ "ASA"→"aspirin" │  │ "aspirin, acetylsalicylic" │
│  └───────────────┘  └─────────────────┘  └─────────────────────────┘  │
└────────────────────────────────┬────────────────────────────────────────┘
                                 │
                    ┌────────────┴────────────┐
                    ▼                         ▼
┌─────────────────────────────┐   ┌─────────────────────────────┐
│     Semantic Search         │   │      Lexical Search         │
│  ┌───────────────────────┐ │   │  ┌───────────────────────┐ │
│  │ OpenAI Embeddings     │ │   │  │ BM25 via Meilisearch  │ │
│  │ text-embedding-3-small│ │   │  │ (exact keyword match) │ │
│  └───────────┬───────────┘ │   │  └───────────┬───────────┘ │
│              ▼             │   │              ▼             │
│  ┌───────────────────────┐ │   │  ┌───────────────────────┐ │
│  │   Qdrant Vector DB    │ │   │  │ Meilisearch Index     │ │
│  │   cosine similarity   │ │   │  │ BM25 scoring          │ │
│  └───────────┬───────────┘ │   │  └───────────┬───────────┘ │
│              ▼             │   │              ▼             │
│  Top K=50 semantic results │   │  Top K=50 lexical results  │
└──────────────┬──────────────┘   └──────────────┬──────────────┘
               │                                  │
               └─────────────┬────────────────────┘
                             ▼
┌─────────────────────────────────────────────────────────────────────────┐
│                       Reciprocal Rank Fusion (RRF)                       │
│         Combines results with formula: 1 / (k + rank)                    │
│         k=60 constant, deduplicates, normalizes scores                   │
└────────────────────────────────┬────────────────────────────────────────┘
                                 ▼
┌─────────────────────────────────────────────────────────────────────────┐
│                       Cross-Encoder Re-ranking                           │
│  ┌────────────────────────────────────────────────────────────────────┐ │
│  │ Model: cross-encoder/ms-marco-MiniLM-L-6-v2                        │ │
│  │ Input: (query, passage) pairs                                       │ │
│  │ Output: Relevance scores 0-1                                        │ │
│  │ Top 20 candidates → Re-ranked top 10                               │ │
│  └────────────────────────────────────────────────────────────────────┘ │
└────────────────────────────────┬────────────────────────────────────────┘
                                 ▼
┌─────────────────────────────────────────────────────────────────────────┐
│                        Contextual Enrichment                             │
│  - Add surrounding paragraph context                                     │
│  - Include document metadata (chapter, section)                          │
│  - Apply metadata filters (date, source type, specialty)                │
└────────────────────────────────┬────────────────────────────────────────┘
                                 ▼
┌─────────────────────────────────────────────────────────────────────────┐
│                        Final Results (Top 10)                            │
│  [{ content, score, metadata, context, source }]                        │
└─────────────────────────────────────────────────────────────────────────┘

3.4 Component Specifications

3.4.1 Hybrid Search Service

File: services/api-gateway/app/services/search/hybrid_search.py

"""
Hybrid Search Service

Combines semantic (vector) and lexical (BM25) search using
Reciprocal Rank Fusion (RRF) for optimal retrieval.

Research basis:
- "Reciprocal Rank Fusion outperforms Condorcet and individual Rank Learning Methods"
  (Cormack et al., 2009)
- Anthropic's "Contextual Retrieval" blog post (2024)
"""

from typing import List, Dict, Optional, Any
from dataclasses import dataclass
import asyncio
import logging

from .search_aggregator import SearchAggregator  # Existing semantic search
from .bm25_index import BM25Index
from .cross_encoder import CrossEncoderReranker

logger = logging.getLogger(__name__)


@dataclass
class HybridSearchResult:
    """Result from hybrid search"""
    doc_id: str
    content: str
    score: float
    semantic_rank: Optional[int]
    lexical_rank: Optional[int]
    rerank_score: Optional[float]
    metadata: Dict[str, Any]


class HybridSearchService:
    """
    Hybrid search combining semantic and lexical retrieval.

    Architecture:
    1. Query preprocessing (synonym expansion, abbreviations)
    2. Parallel semantic + lexical search
    3. Reciprocal Rank Fusion
    4. Cross-encoder re-ranking
    5. Contextual enrichment
    """

    def __init__(
        self,
        semantic_search: SearchAggregator,
        lexical_search: BM25Index,
        reranker: CrossEncoderReranker,
        semantic_weight: float = 0.5,
        rrf_k: int = 60,
    ):
        self.semantic_search = semantic_search
        self.lexical_search = lexical_search
        self.reranker = reranker
        self.semantic_weight = semantic_weight
        self.rrf_k = rrf_k

    async def search(
        self,
        query: str,
        top_k: int = 10,
        expand_synonyms: bool = True,
        rerank: bool = True,
        filters: Optional[Dict[str, Any]] = None,
    ) -> List[HybridSearchResult]:
        """
        Execute hybrid search.

        Args:
            query: Search query
            top_k: Number of results to return
            expand_synonyms: Whether to expand medical synonyms
            rerank: Whether to apply cross-encoder re-ranking
            filters: Metadata filters (e.g., {"source_type": "guideline"})

        Returns:
            List of hybrid search results
        """
        # Step 1: Preprocess query
        expanded_query = query
        if expand_synonyms:
            expanded_query = await self._expand_query(query)
            logger.debug(f"Expanded query: {query} -> {expanded_query}")

        # Step 2: Parallel search
        retrieval_k = max(top_k * 5, 50)  # Retrieve more for fusion

        semantic_task = self.semantic_search.search(
            expanded_query,
            top_k=retrieval_k,
            filter_conditions=filters,
        )
        lexical_task = self.lexical_search.search(
            expanded_query,
            top_k=retrieval_k,
            filters=filters,
        )

        semantic_results, lexical_results = await asyncio.gather(
            semantic_task, lexical_task
        )

        # Step 3: Reciprocal Rank Fusion
        fused_results = self._reciprocal_rank_fusion(
            semantic_results,
            lexical_results,
            k=self.rrf_k,
        )

        # Step 4: Re-ranking (optional)
        if rerank and len(fused_results) > 0:
            rerank_candidates = fused_results[:min(20, len(fused_results))]
            reranked = await self.reranker.rerank(
                query,
                [r.content for r in rerank_candidates]
            )

            # Apply rerank scores
            for i, score in enumerate(reranked):
                if i < len(fused_results):
                    fused_results[i].rerank_score = score

            # Sort by rerank score
            fused_results.sort(key=lambda x: x.rerank_score or 0, reverse=True)

        # Step 5: Return top K
        return fused_results[:top_k]

    def _reciprocal_rank_fusion(
        self,
        semantic_results: List[Any],
        lexical_results: List[Any],
        k: int = 60,
    ) -> List[HybridSearchResult]:
        """
        Combine results using Reciprocal Rank Fusion.

        RRF score = Σ 1 / (k + rank)
        """
        doc_scores: Dict[str, Dict] = {}

        # Process semantic results
        for rank, result in enumerate(semantic_results, 1):
            doc_id = result.doc_id
            rrf_score = 1 / (k + rank)

            if doc_id not in doc_scores:
                doc_scores[doc_id] = {
                    "content": result.content,
                    "metadata": result.metadata,
                    "rrf_score": 0,
                    "semantic_rank": None,
                    "lexical_rank": None,
                }

            doc_scores[doc_id]["rrf_score"] += rrf_score * self.semantic_weight
            doc_scores[doc_id]["semantic_rank"] = rank

        # Process lexical results
        for rank, result in enumerate(lexical_results, 1):
            doc_id = result.doc_id
            rrf_score = 1 / (k + rank)

            if doc_id not in doc_scores:
                doc_scores[doc_id] = {
                    "content": result.content,
                    "metadata": result.metadata,
                    "rrf_score": 0,
                    "semantic_rank": None,
                    "lexical_rank": None,
                }

            doc_scores[doc_id]["rrf_score"] += rrf_score * (1 - self.semantic_weight)
            doc_scores[doc_id]["lexical_rank"] = rank

        # Sort by RRF score and create results
        sorted_docs = sorted(
            doc_scores.items(),
            key=lambda x: x[1]["rrf_score"],
            reverse=True
        )

        return [
            HybridSearchResult(
                doc_id=doc_id,
                content=data["content"],
                score=data["rrf_score"],
                semantic_rank=data["semantic_rank"],
                lexical_rank=data["lexical_rank"],
                rerank_score=None,
                metadata=data["metadata"],
            )
            for doc_id, data in sorted_docs
        ]

    async def _expand_query(self, query: str) -> str:
        """Expand query with medical synonyms and abbreviations."""
        # Placeholder - implement with synonym_database.py
        return query

3.4.2 BM25 Lexical Search

File: services/api-gateway/app/services/search/bm25_index.py

"""
BM25 Lexical Search using Meilisearch

Meilisearch provides:
- Fast BM25-based full-text search
- Typo tolerance
- Faceted filtering
- Easy deployment (single binary)
"""

from typing import List, Dict, Optional, Any
from dataclasses import dataclass
import httpx
import logging

from ..core.config import settings

logger = logging.getLogger(__name__)


@dataclass
class LexicalSearchResult:
    doc_id: str
    content: str
    score: float
    metadata: Dict[str, Any]


class BM25Index:
    """
    BM25 lexical search via Meilisearch.

    Index structure:
    - id: Document ID
    - content: Searchable text
    - title: Document title
    - source_type: "guideline" | "textbook" | "research"
    - specialty: Medical specialty
    - created_at: Timestamp
    """

    def __init__(
        self,
        host: str = None,
        api_key: str = None,
        index_name: str = "kb_documents",
    ):
        self.host = host or settings.MEILISEARCH_HOST
        self.api_key = api_key or settings.MEILISEARCH_API_KEY
        self.index_name = index_name
        self.client = httpx.AsyncClient(
            base_url=self.host,
            headers={"Authorization": f"Bearer {self.api_key}"},
            timeout=30.0,
        )

    async def search(
        self,
        query: str,
        top_k: int = 50,
        filters: Optional[Dict[str, Any]] = None,
    ) -> List[LexicalSearchResult]:
        """
        Execute BM25 search.

        Args:
            query: Search query
            top_k: Number of results
            filters: Metadata filters

        Returns:
            List of lexical search results
        """
        # Build Meilisearch filter string
        filter_str = self._build_filter(filters) if filters else None

        payload = {
            "q": query,
            "limit": top_k,
            "attributesToRetrieve": ["id", "content", "title", "metadata"],
            "showRankingScore": True,
        }

        if filter_str:
            payload["filter"] = filter_str

        try:
            response = await self.client.post(
                f"/indexes/{self.index_name}/search",
                json=payload,
            )
            response.raise_for_status()
            data = response.json()

            return [
                LexicalSearchResult(
                    doc_id=hit["id"],
                    content=hit["content"],
                    score=hit.get("_rankingScore", 0),
                    metadata=hit.get("metadata", {}),
                )
                for hit in data.get("hits", [])
            ]

        except Exception as e:
            logger.error(f"Meilisearch search failed: {e}")
            return []

    async def index_document(
        self,
        doc_id: str,
        content: str,
        title: str,
        metadata: Dict[str, Any],
    ) -> bool:
        """Index a document for lexical search."""
        try:
            await self.client.post(
                f"/indexes/{self.index_name}/documents",
                json=[{
                    "id": doc_id,
                    "content": content,
                    "title": title,
                    **metadata,
                }],
            )
            return True
        except Exception as e:
            logger.error(f"Failed to index document {doc_id}: {e}")
            return False

    async def delete_document(self, doc_id: str) -> bool:
        """Delete a document from the index."""
        try:
            await self.client.delete(
                f"/indexes/{self.index_name}/documents/{doc_id}"
            )
            return True
        except Exception as e:
            logger.error(f"Failed to delete document {doc_id}: {e}")
            return False

    def _build_filter(self, filters: Dict[str, Any]) -> str:
        """Build Meilisearch filter string from dict."""
        conditions = []
        for key, value in filters.items():
            if isinstance(value, list):
                # OR condition for list values
                or_conditions = " OR ".join(f'{key} = "{v}"' for v in value)
                conditions.append(f"({or_conditions})")
            else:
                conditions.append(f'{key} = "{value}"')
        return " AND ".join(conditions)

    async def close(self):
        await self.client.aclose()

3.4.3 Cross-Encoder Re-ranker

File: services/api-gateway/app/services/search/cross_encoder.py

"""
Cross-Encoder Re-ranking Service

Uses sentence-transformers cross-encoder models for high-quality
passage re-ranking. Cross-encoders process query-passage pairs
together, enabling better relevance scoring than bi-encoders.

Model choices:
- cross-encoder/ms-marco-MiniLM-L-6-v2 (fast, good quality)
- cross-encoder/ms-marco-MiniLM-L-12-v2 (slower, better quality)
- BAAI/bge-reranker-base (good for general domain)
"""

from typing import List, Tuple
import logging
import torch
from sentence_transformers import CrossEncoder

logger = logging.getLogger(__name__)


class CrossEncoderReranker:
    """
    Re-ranks search results using a cross-encoder model.

    Architecture:
    - Query and each passage are concatenated and encoded together
    - Model outputs a relevance score for each pair
    - Results are sorted by relevance score
    """

    def __init__(
        self,
        model_name: str = "cross-encoder/ms-marco-MiniLM-L-6-v2",
        device: str = None,
        max_length: int = 512,
    ):
        self.device = device or ("cuda" if torch.cuda.is_available() else "cpu")
        self.model = CrossEncoder(model_name, device=self.device, max_length=max_length)
        logger.info(f"Loaded cross-encoder model {model_name} on {self.device}")

    async def rerank(
        self,
        query: str,
        passages: List[str],
        batch_size: int = 16,
    ) -> List[float]:
        """
        Re-rank passages for a query.

        Args:
            query: Search query
            passages: List of passage texts
            batch_size: Batch size for inference

        Returns:
            List of relevance scores (same order as passages)
        """
        if not passages:
            return []

        # Create query-passage pairs
        pairs = [[query, passage] for passage in passages]

        try:
            # Get scores (returns numpy array)
            scores = self.model.predict(
                pairs,
                batch_size=batch_size,
                show_progress_bar=False,
            )

            # Convert to Python floats
            return [float(s) for s in scores]

        except Exception as e:
            logger.error(f"Cross-encoder re-ranking failed: {e}")
            # Return neutral scores on failure
            return [0.5] * len(passages)

    async def rerank_with_indices(
        self,
        query: str,
        passages: List[str],
        top_k: int = 10,
    ) -> List[Tuple[int, float]]:
        """
        Re-rank and return top-k indices with scores.

        Returns:
            List of (original_index, score) tuples, sorted by score
        """
        scores = await self.rerank(query, passages)

        # Pair indices with scores and sort
        indexed_scores = list(enumerate(scores))
        indexed_scores.sort(key=lambda x: x[1], reverse=True)

        return indexed_scores[:top_k]