2:I[7012,["4765","static/chunks/4765-f5afdf8061f456f3.js","9856","static/chunks/9856-3b185291364d9bef.js","6687","static/chunks/app/docs/%5B...slug%5D/page-e07536548216bee4.js"],"MarkdownRenderer"] 4:I[9856,["4765","static/chunks/4765-f5afdf8061f456f3.js","9856","static/chunks/9856-3b185291364d9bef.js","6687","static/chunks/app/docs/%5B...slug%5D/page-e07536548216bee4.js"],""] 5:I[4126,[],""] 7:I[9630,[],""] 8:I[4278,["9856","static/chunks/9856-3b185291364d9bef.js","8172","static/chunks/8172-b3a2d6fe4ae10d40.js","3185","static/chunks/app/layout-2814fa5d15b84fe4.js"],"HeadingProvider"] 9:I[1476,["9856","static/chunks/9856-3b185291364d9bef.js","8172","static/chunks/8172-b3a2d6fe4ae10d40.js","3185","static/chunks/app/layout-2814fa5d15b84fe4.js"],"Header"] a:I[3167,["9856","static/chunks/9856-3b185291364d9bef.js","8172","static/chunks/8172-b3a2d6fe4ae10d40.js","3185","static/chunks/app/layout-2814fa5d15b84fe4.js"],"Sidebar"] b:I[7409,["9856","static/chunks/9856-3b185291364d9bef.js","8172","static/chunks/8172-b3a2d6fe4ae10d40.js","3185","static/chunks/app/layout-2814fa5d15b84fe4.js"],"PageFrame"] 3:T2aab, # Debugging Index **Last Updated:** 2025-12-02 **Version:** 1.1.0 **Audience:** Developers, SREs, AI Assistants > **Note:** Voice/Realtime rows now target the Thinker-Talker pipeline at `/api/voice/pipeline-ws`. The legacy OpenAI Realtime API is documented separately. This is the central hub for all VoiceAssist troubleshooting documentation. Use this index to quickly find the right debugging guide, logs, metrics, and runbooks for any subsystem. --- ## Debug by Symptom | Symptom | Likely Subsystem | First Doc to Read | Key Commands | | ---------------------------- | ---------------- | ------------------------------------------------ | --------------------------------------------------------- | | API returns 500 errors | Backend | [Backend Debugging](./DEBUGGING_BACKEND.md) | `docker logs voiceassist-server --tail 100 \| grep ERROR` | | WebSocket disconnects | Voice/Realtime | [Voice Debugging](./DEBUGGING_VOICE_REALTIME.md) | `websocat "wss://assist.asimo.io/api/voice/pipeline-ws"` | | Voice input not working | Voice/Realtime | [Voice Debugging](./DEBUGGING_VOICE_REALTIME.md) | Check browser audio permissions | | Pages return 404 | Docs Site | [Docs Site Debugging](./DEBUGGING_DOCS_SITE.md) | `ls /var/www/assistdocs.asimo.io/` | | Slow response times | Backend | [Backend Debugging](./DEBUGGING_BACKEND.md) | `curl /metrics \| grep http_request_duration` | | Authentication failing | Backend | [Backend Debugging](./DEBUGGING_BACKEND.md) | Check JWT token expiry, Redis status | | Search not returning results | Backend | [Backend Debugging](./DEBUGGING_BACKEND.md) | Check Qdrant connection, embedding status | | UI renders incorrectly | Frontend | [Frontend Debugging](./DEBUGGING_FRONTEND.md) | Browser DevTools Console & Network | | SSL certificate errors | Infrastructure | [Debugging Overview](./DEBUGGING_OVERVIEW.md) | `sudo certbot certificates` | | Service won't start | Infrastructure | [Debugging Overview](./DEBUGGING_OVERVIEW.md) | `sudo systemctl status ` | --- ## AI Agent Playbook For AI assistants debugging VoiceAssist issues: ### Required Information to Collect 1. **Error context**: Exact error message, timestamp, affected endpoint 2. **Logs**: Last 100 lines from relevant service (`journalctl -u -n 100`) 3. **Health status**: Output of `/health` and `/ready` endpoints 4. **Recent changes**: Any deployments or config changes in past 24h ### Standard Investigation Steps ```bash # 1. Check container status docker ps --filter name=voiceassist docker logs voiceassist-server --tail 50 # 2. Check recent errors docker logs voiceassist-server --since "30m" 2>&1 | grep -i error # 3. Check health endpoints curl -s https://assist.asimo.io/health | jq curl -s https://assist.asimo.io/ready | jq # 4. Check dependencies docker exec voiceassist-redis redis-cli ping curl -s http://localhost:6333/collections | jq # Qdrant ``` ### What "Good" vs "Bad" Looks Like | Check | Good | Bad | | -------------- | -------------------------------------------- | ---------------------- | | `/health` | `{"status": "healthy"}` | Non-200 or timeout | | `/ready` | `{"status": "ready", "dependencies": {...}}` | Any dependency `false` | | Service status | `active (running)` | `failed` or `inactive` | | Redis | `PONG` | Connection refused | | Apache | `Syntax OK` from configtest | Syntax errors | --- ## Quick Reference: Subsystem Overview | Subsystem | Primary Guide | Key Logs | Health Endpoint | | -------------- | ------------------------------------------------ | ----------------------------------- | ----------------------------- | | Backend/API | [Backend Debugging](./DEBUGGING_BACKEND.md) | `docker logs voiceassist-server` | `/health`, `/ready` | | Frontend/Web | [Frontend Debugging](./DEBUGGING_FRONTEND.md) | Browser Console, Network Tab | N/A (static) | | Voice/Realtime | [Voice Debugging](./DEBUGGING_VOICE_REALTIME.md) | `docker logs voiceassist-server` | `/ws` endpoint | | Docs Site | [Docs Site Debugging](./DEBUGGING_DOCS_SITE.md) | `/var/log/apache2/assistdocs-*.log` | `/agent/index.json` | | Infrastructure | [Overview](./DEBUGGING_OVERVIEW.md) | `docker logs`, Apache logs | Docker health, systemd status | --- ## Debugging Guides by Subsystem ### Backend Services **Primary Guide:** [Backend Debugging](./DEBUGGING_BACKEND.md) **Key Logs:** ```bash # API Gateway logs (Docker container) docker logs voiceassist-server -f # With error filtering docker logs voiceassist-server --tail 100 2>&1 | grep -i error ``` **Key Health Endpoints:** - `GET /health` - Basic health check - `GET /ready` - Readiness (includes dependencies) - `GET /metrics` - Prometheus metrics **Related Runbooks:** - [Production Deployment Runbook](../PRODUCTION_DEPLOYMENT_RUNBOOK.md) - [Disaster Recovery](../DISASTER_RECOVERY_RUNBOOK.md) --- ### Frontend (Web App & Admin Panel) **Primary Guide:** [Frontend Debugging](./DEBUGGING_FRONTEND.md) **Key Logs:** - Browser DevTools → Console - Browser DevTools → Network Tab - React DevTools → Profiler **Debugging Tools:** - Chrome DevTools - React Developer Tools extension - Redux DevTools (if applicable) **Related Documentation:** - [Web App Feature Specs](../client-implementation/WEB_APP_FEATURE_SPECS.md) - [Admin Panel Feature Specs](../client-implementation/ADMIN_PANEL_FEATURE_SPECS.md) --- ### Voice & Realtime (WebSocket, STT, TTS) **Primary Guide:** [Voice & Realtime Debugging](./DEBUGGING_VOICE_REALTIME.md) **Key Logs:** ```bash # Voice service logs (Docker container) docker logs voiceassist-server --since "10m" 2>&1 | grep -i "websocket\|stt\|tts\|voice" ``` **Key Endpoints:** - Voice Pipeline (T/T): `wss://assist.asimo.io/api/voice/pipeline-ws` - Chat Streaming: `wss://assist.asimo.io/api/realtime/ws` - Test tools: `websocat`, `wscat` **Related Documentation:** - [Voice Mode Settings Guide](../VOICE_MODE_SETTINGS_GUIDE.md) - [Voice Ready State](../VOICE_READY_STATE_2025-11-25.md) --- ### Documentation Site **Primary Guide:** [Docs Site Debugging](./DEBUGGING_DOCS_SITE.md) **Key Logs:** ```bash # Apache logs for docs site sudo tail -f /var/log/apache2/assistdocs-error.log sudo tail -f /var/log/apache2/assistdocs-access.log ``` **Key Health Endpoints:** - `GET /` - Homepage - `GET /agent/index.json` - AI agent discovery endpoint - `GET /agent/docs.json` - Documentation list - `GET /agent/tasks.json` - Common agent tasks - `GET /agent/schema.json` - JSON Schema for API types - `GET /search-index.json` - Search index - `GET /sitemap.xml` - Sitemap **Related Documentation:** - [Agent API Reference](../ai/AGENT_API_REFERENCE.md) - [Agent Onboarding](../ai/AGENT_ONBOARDING.md) --- ### Infrastructure **Primary Guide:** [Debugging Overview](./DEBUGGING_OVERVIEW.md) **Key Services:** ```bash # Check all major Docker containers docker ps --filter name=voiceassist # Check Apache sudo systemctl status apache2 # Check systemd for failures sudo systemctl list-units --failed ``` **Key Health Checks:** ```bash # Database psql -h localhost -U voiceassist -d voiceassist -c "SELECT 1" # Redis redis-cli ping # Apache sudo apachectl configtest ``` **Related Documentation:** - [Infrastructure Setup](../INFRASTRUCTURE_SETUP.md) - [Apache Configuration](../apache-configs/) - [SLO Definitions](../operations/SLO_DEFINITIONS.md) --- ## Key Metrics & Monitoring ### Prometheus Metrics (from `/metrics`) | Metric | Description | Alert Threshold | | ------------------------------- | ------------------------- | --------------- | | `http_requests_total` | Request count by status | > 10 5xx/min | | `http_request_duration_seconds` | Request latency | > 2s p95 | | `db_connection_pool_size` | Active DB connections | > 80% pool | | `redis_connection_errors` | Redis connection failures | > 0 | ### Application Logs Log format is structured JSON with trace IDs: ```json { "timestamp": "2025-11-27T12:00:00Z", "level": "ERROR", "trace_id": "abc123", "message": "Database connection failed", "service": "api-gateway" } ``` --- ## Common Investigation Workflows ### 1. API Error Investigation 1. Check health endpoints: `curl /health && curl /ready` 2. Review recent errors: `docker logs voiceassist-server --tail 100 | grep ERROR` 3. Check dependencies: Redis, PostgreSQL, Qdrant 4. Look for trace ID in logs to follow request path ### 2. Frontend Issue Investigation 1. Open Browser DevTools (F12) 2. Check Console for JavaScript errors 3. Check Network tab for failed API calls 4. Verify environment variables ### 3. WebSocket/Voice Issue Investigation 1. Test voice pipeline: `websocat "wss://assist.asimo.io/api/voice/pipeline-ws?token=..."` 2. Check browser console for connection errors 3. Verify Apache WebSocket proxy configuration 4. Check audio permissions in browser ### 4. Docs Site Investigation 1. Verify static files exist: `ls /var/www/assistdocs.asimo.io/` 2. Test with explicit .html: `curl /ai/onboarding.html` 3. Check Apache rewrite rules 4. Run validation: `pnpm validate:metadata` --- ## Runbooks & Operations | Scenario | Runbook | | ------------------------- | -------------------------------------------------------------------- | | Production Deployment | [Production Deployment Runbook](../PRODUCTION_DEPLOYMENT_RUNBOOK.md) | | Disaster Recovery | [Disaster Recovery Runbook](../DISASTER_RECOVERY_RUNBOOK.md) | | Performance Investigation | [Performance Tuning Guide](../PERFORMANCE_TUNING_GUIDE.md) | | Load Testing | [Load Testing Guide](../LOAD_TESTING_GUIDE.md) | --- ## Related Documentation - [Implementation Status](../overview/IMPLEMENTATION_STATUS.md) - Component status overview - [API Reference](../api-reference/rest-api.md) - API endpoint documentation - [Backend Architecture](../architecture/README.md) - System design - [Service Catalog](../SERVICE_CATALOG.md) - Service inventory 6:["slug","debugging/DEBUGGING_INDEX","c"] 0:["X7oMT3VrOffzp0qvbeOas",[[["",{"children":["docs",{"children":[["slug","debugging/DEBUGGING_INDEX","c"],{"children":["__PAGE__?{\"slug\":[\"debugging\",\"DEBUGGING_INDEX\"]}",{}]}]}]},"$undefined","$undefined",true],["",{"children":["docs",{"children":[["slug","debugging/DEBUGGING_INDEX","c"],{"children":["__PAGE__",{},[["$L1",["$","div",null,{"children":[["$","div",null,{"className":"mb-6 flex items-center justify-between gap-4","children":[["$","div",null,{"children":[["$","p",null,{"className":"text-sm text-gray-500 dark:text-gray-400","children":"Docs / Raw"}],["$","h1",null,{"className":"text-3xl font-bold text-gray-900 dark:text-white","children":"Debugging Index"}],["$","p",null,{"className":"text-sm text-gray-600 dark:text-gray-400","children":["Sourced from"," ",["$","code",null,{"className":"font-mono text-xs","children":["docs/","debugging/DEBUGGING_INDEX.md"]}]]}]]}],["$","a",null,{"href":"https://github.com/mohammednazmy/VoiceAssist/edit/main/docs/debugging/DEBUGGING_INDEX.md","target":"_blank","rel":"noreferrer","className":"inline-flex items-center gap-2 rounded-md border border-gray-200 dark:border-gray-700 px-3 py-1.5 text-sm text-gray-700 dark:text-gray-200 hover:border-primary-500 dark:hover:border-primary-400 hover:text-primary-700 dark:hover:text-primary-300","children":"Edit on GitHub"}]]}],["$","div",null,{"className":"rounded-lg border border-gray-200 dark:border-gray-800 bg-white dark:bg-gray-900 p-6","children":["$","$L2",null,{"content":"$3"}]}],["$","div",null,{"className":"mt-6 flex flex-wrap gap-2 text-sm","children":[["$","$L4",null,{"href":"/reference/all-docs","className":"inline-flex items-center gap-1 rounded-md bg-gray-100 px-3 py-1 text-gray-700 hover:bg-gray-200 dark:bg-gray-800 dark:text-gray-200 dark:hover:bg-gray-700","children":"← All documentation"}],["$","$L4",null,{"href":"/","className":"inline-flex items-center gap-1 rounded-md bg-gray-100 px-3 py-1 text-gray-700 hover:bg-gray-200 dark:bg-gray-800 dark:text-gray-200 dark:hover:bg-gray-700","children":"Home"}]]}]]}],null],null],null]},[null,["$","$L5",null,{"parallelRouterKey":"children","segmentPath":["children","docs","children","$6","children"],"error":"$undefined","errorStyles":"$undefined","errorScripts":"$undefined","template":["$","$L7",null,{}],"templateStyles":"$undefined","templateScripts":"$undefined","notFound":"$undefined","notFoundStyles":"$undefined"}]],null]},[null,["$","$L5",null,{"parallelRouterKey":"children","segmentPath":["children","docs","children"],"error":"$undefined","errorStyles":"$undefined","errorScripts":"$undefined","template":["$","$L7",null,{}],"templateStyles":"$undefined","templateScripts":"$undefined","notFound":"$undefined","notFoundStyles":"$undefined"}]],null]},[[[["$","link","0",{"rel":"stylesheet","href":"/_next/static/css/7f586cdbbaa33ff7.css","precedence":"next","crossOrigin":"$undefined"}]],["$","html",null,{"lang":"en","className":"h-full","children":["$","body",null,{"className":"__className_f367f3 h-full bg-white dark:bg-gray-900","children":[["$","a",null,{"href":"#main-content","className":"skip-to-content","children":"Skip to main content"}],["$","$L8",null,{"children":[["$","$L9",null,{}],["$","$La",null,{}],["$","main",null,{"id":"main-content","className":"lg:pl-64","role":"main","aria-label":"Documentation content","children":["$","$Lb",null,{"children":["$","$L5",null,{"parallelRouterKey":"children","segmentPath":["children"],"error":"$undefined","errorStyles":"$undefined","errorScripts":"$undefined","template":["$","$L7",null,{}],"templateStyles":"$undefined","templateScripts":"$undefined","notFound":[["$","title",null,{"children":"404: This page could not be found."}],["$","div",null,{"style":{"fontFamily":"system-ui,\"Segoe UI\",Roboto,Helvetica,Arial,sans-serif,\"Apple Color Emoji\",\"Segoe UI Emoji\"","height":"100vh","textAlign":"center","display":"flex","flexDirection":"column","alignItems":"center","justifyContent":"center"},"children":["$","div",null,{"children":[["$","style",null,{"dangerouslySetInnerHTML":{"__html":"body{color:#000;background:#fff;margin:0}.next-error-h1{border-right:1px solid rgba(0,0,0,.3)}@media (prefers-color-scheme:dark){body{color:#fff;background:#000}.next-error-h1{border-right:1px solid rgba(255,255,255,.3)}}"}}],["$","h1",null,{"className":"next-error-h1","style":{"display":"inline-block","margin":"0 20px 0 0","padding":"0 23px 0 0","fontSize":24,"fontWeight":500,"verticalAlign":"top","lineHeight":"49px"},"children":"404"}],["$","div",null,{"style":{"display":"inline-block"},"children":["$","h2",null,{"style":{"fontSize":14,"fontWeight":400,"lineHeight":"49px","margin":0},"children":"This page could not be found."}]}]]}]}]],"notFoundStyles":[]}]}]}]]}]]}]}]],null],null],["$Lc",null]]]] c:[["$","meta","0",{"name":"viewport","content":"width=device-width, initial-scale=1"}],["$","meta","1",{"charSet":"utf-8"}],["$","title","2",{"children":"Debugging Index | Docs | VoiceAssist Docs"}],["$","meta","3",{"name":"description","content":"Central hub for all VoiceAssist troubleshooting documentation - logs, metrics, health endpoints, and debugging guides by subsystem."}],["$","meta","4",{"name":"keywords","content":"VoiceAssist,documentation,medical AI,voice assistant,healthcare,HIPAA,API"}],["$","meta","5",{"name":"robots","content":"index, follow"}],["$","meta","6",{"name":"googlebot","content":"index, follow"}],["$","link","7",{"rel":"canonical","href":"https://assistdocs.asimo.io"}],["$","meta","8",{"property":"og:title","content":"VoiceAssist Documentation"}],["$","meta","9",{"property":"og:description","content":"Comprehensive documentation for VoiceAssist - Enterprise Medical AI Assistant"}],["$","meta","10",{"property":"og:url","content":"https://assistdocs.asimo.io"}],["$","meta","11",{"property":"og:site_name","content":"VoiceAssist Docs"}],["$","meta","12",{"property":"og:type","content":"website"}],["$","meta","13",{"name":"twitter:card","content":"summary"}],["$","meta","14",{"name":"twitter:title","content":"VoiceAssist Documentation"}],["$","meta","15",{"name":"twitter:description","content":"Comprehensive documentation for VoiceAssist - Enterprise Medical AI Assistant"}],["$","meta","16",{"name":"next-size-adjust"}]] 1:null