2:I[7012,["4765","static/chunks/4765-f5afdf8061f456f3.js","9856","static/chunks/9856-3b185291364d9bef.js","6687","static/chunks/app/docs/%5B...slug%5D/page-e07536548216bee4.js"],"MarkdownRenderer"] 4:I[9856,["4765","static/chunks/4765-f5afdf8061f456f3.js","9856","static/chunks/9856-3b185291364d9bef.js","6687","static/chunks/app/docs/%5B...slug%5D/page-e07536548216bee4.js"],""] 5:I[4126,[],""] 7:I[9630,[],""] 8:I[4278,["9856","static/chunks/9856-3b185291364d9bef.js","8172","static/chunks/8172-b3a2d6fe4ae10d40.js","3185","static/chunks/app/layout-2814fa5d15b84fe4.js"],"HeadingProvider"] 9:I[1476,["9856","static/chunks/9856-3b185291364d9bef.js","8172","static/chunks/8172-b3a2d6fe4ae10d40.js","3185","static/chunks/app/layout-2814fa5d15b84fe4.js"],"Header"] a:I[3167,["9856","static/chunks/9856-3b185291364d9bef.js","8172","static/chunks/8172-b3a2d6fe4ae10d40.js","3185","static/chunks/app/layout-2814fa5d15b84fe4.js"],"Sidebar"] b:I[7409,["9856","static/chunks/9856-3b185291364d9bef.js","8172","static/chunks/8172-b3a2d6fe4ae10d40.js","3185","static/chunks/app/layout-2814fa5d15b84fe4.js"],"PageFrame"] 3:T1d4f, # Deployment Runbook **Last Updated**: 2025-11-27 **Purpose**: Step-by-step guide for deploying VoiceAssist V2 --- ## Pre-Deployment Checklist - [ ] All tests passing in CI/CD - [ ] Code reviewed and approved - [ ] Database migrations reviewed - [ ] Breaking changes documented - [ ] Rollback plan documented - [ ] Stakeholders notified - [ ] Maintenance window scheduled (if required) --- ## Deployment Steps ### 1. Pre-Deployment Verification ```bash # Check current system health curl http://localhost:8000/health curl http://localhost:8000/ready # Verify all containers running docker compose ps # Check database connection docker compose exec postgres psql -U voiceassist -d voiceassist -c "SELECT version();" # Check Redis docker compose exec redis redis-cli ping # Check Qdrant curl http://localhost:6333/collections ``` ### 2. Backup Current State ```bash # Backup database docker compose exec postgres pg_dump -U voiceassist voiceassist > backup_$(date +%Y%m%d_%H%M%S).sql # Backup environment configuration cp .env .env.backup_$(date +%Y%m%d_%H%M%S) # Tag current Docker images docker tag voiceassist-voiceassist-server:latest voiceassist-voiceassist-server:pre-deploy-$(date +%Y%m%d_%H%M%S) ``` ### 3. Pull Latest Code ```bash # Fetch latest changes git fetch origin # Check what's changing git log --oneline HEAD..origin/main # Pull changes git pull origin main # Verify correct branch git branch --show-current git log -1 --oneline ``` ### 4. Update Environment Configuration ```bash # Review .env changes diff .env.example .env # Update .env if needed vim .env # Validate configuration grep -v '^#' .env | grep -v '^$' | wc -l # Count non-empty lines ``` ### 5. Run Database Migrations ```bash # Check current migration status docker compose run --rm voiceassist-server alembic current # Review pending migrations docker compose run --rm voiceassist-server alembic history # Run migrations docker compose run --rm voiceassist-server alembic upgrade head # Verify migration success docker compose run --rm voiceassist-server alembic current ``` ### 6. Build New Images ```bash # Build updated images docker compose build voiceassist-server # Verify image built docker images | grep voiceassist-server # Check image size (should be reasonable) docker images voiceassist-voiceassist-server:latest --format "{{.Size}}" ``` ### 7. Deploy Services ```bash # Deploy with zero-downtime (recreate containers) docker compose up -d voiceassist-server # Watch logs for startup docker compose logs -f voiceassist-server # Wait for healthcheck sleep 10 ``` ### 8. Post-Deployment Verification ```bash # Check health endpoint curl http://localhost:8000/health # Check readiness curl http://localhost:8000/ready # Verify version curl http://localhost:8000/health | jq '.version' # Check all containers running docker compose ps # Check logs for errors docker compose logs --tail=100 voiceassist-server | grep -i error # Verify metrics endpoint curl http://localhost:8000/metrics | head -20 # Test a sample API endpoint (requires auth) # curl -H "Authorization: Bearer $TOKEN" http://localhost:8000/api/users/me ``` ### 9. Smoke Tests ```bash # Test authentication curl -X POST http://localhost:8000/api/auth/login \ -H "Content-Type: application/json" \ -d '{"email":"admin@example.com","password":"password"}' | jq '.' # Test database connectivity docker compose exec postgres psql -U voiceassist -d voiceassist -c "SELECT COUNT(*) FROM users;" # Test Redis docker compose exec redis redis-cli --raw incr deployment_test # Test Qdrant curl http://localhost:6333/collections ``` ### 10. Monitor Initial Traffic ```bash # Watch logs for first 5 minutes docker compose logs -f --tail=100 voiceassist-server # Monitor metrics watch -n 5 'curl -s http://localhost:8000/metrics | grep -E "(http_requests_total|http_request_duration)"' # Check error rate docker compose logs --since 5m voiceassist-server | grep -i error | wc -l ``` --- ## Rollback Procedure If deployment fails, follow these steps: ### Quick Rollback (Image-Based) ```bash # Stop current containers docker compose down voiceassist-server # Revert to previous image PREVIOUS_TAG="pre-deploy-YYYYMMDD_HHMMSS" # From backup step docker tag voiceassist-voiceassist-server:$PREVIOUS_TAG voiceassist-voiceassist-server:latest # Start previous version docker compose up -d voiceassist-server # Verify rollback curl http://localhost:8000/health | jq '.version' ``` ### Full Rollback (Code + Database) ```bash # Stop services docker compose down voiceassist-server # Revert code git log -1 --oneline # Note current commit git checkout HEAD~1 # Or specific commit hash # Rollback database migration BACKUP_FILE="backup_YYYYMMDD_HHMMSS.sql" docker compose exec -T postgres psql -U voiceassist voiceassist < $BACKUP_FILE # Rebuild image docker compose build voiceassist-server # Start services docker compose up -d voiceassist-server # Verify rollback curl http://localhost:8000/health ``` --- ## Deployment Checklist **Post-Deployment:** - [ ] Health endpoint returning 200 - [ ] Readiness endpoint returning 200 - [ ] No error logs in last 5 minutes - [ ] Metrics endpoint accessible - [ ] Database migrations applied - [ ] All containers running - [ ] Sample API requests successful - [ ] Version number updated - [ ] Stakeholders notified of completion - [ ] Documentation updated (if needed) --- ## Common Issues & Solutions ### Issue: Database Migration Fails **Symptoms**: Migration command returns error **Solution**: ```bash # Check current state docker compose run --rm voiceassist-server alembic current # Manually review SQL docker compose run --rm voiceassist-server alembic show # If safe, downgrade one step docker compose run --rm voiceassist-server alembic downgrade -1 # Fix issue and retry docker compose run --rm voiceassist-server alembic upgrade head ``` ### Issue: Container Won't Start **Symptoms**: Container crashes immediately or fails healthcheck **Solution**: ```bash # Check logs docker compose logs --tail=50 voiceassist-server # Check container exit code docker compose ps -a voiceassist-server # Verify environment variables docker compose config | grep -A 20 voiceassist-server # Test dependencies docker compose exec postgres pg_isready docker compose exec redis redis-cli ping ``` ### Issue: High Error Rate After Deployment **Symptoms**: Increased 5xx errors in logs/metrics **Solution**: ```bash # Check error logs docker compose logs voiceassist-server | grep -i error # Check database connections docker compose exec postgres psql -U voiceassist -d voiceassist -c "SELECT count(*) FROM pg_stat_activity WHERE state = 'active';" # Check Redis memory docker compose exec redis redis-cli INFO memory | grep used_memory_human # Rollback if errors > 5% of traffic ``` --- ## Emergency Contacts - **On-Call Engineer**: Check PagerDuty - **Database Admin**: DBA on-call rotation - **DevOps Lead**: ops-team@voiceassist.local - **Product Owner**: product@voiceassist.local --- ##Related Documentation - [UNIFIED_ARCHITECTURE.md](../../UNIFIED_ARCHITECTURE.md) - [CONNECTION_POOL_OPTIMIZATION.md](../CONNECTION_POOL_OPTIMIZATION.md) - [Incident Response Runbook](./INCIDENT_RESPONSE.md) - [Backup & Restore Runbook](./BACKUP_RESTORE.md) --- **Document Version**: 1.0 **Last Updated**: 2025-11-21 **Maintained By**: VoiceAssist DevOps Team **Review Cycle**: After each major deployment or quarterly 6:["slug","operations/runbooks/DEPLOYMENT","c"] 0:["X7oMT3VrOffzp0qvbeOas",[[["",{"children":["docs",{"children":[["slug","operations/runbooks/DEPLOYMENT","c"],{"children":["__PAGE__?{\"slug\":[\"operations\",\"runbooks\",\"DEPLOYMENT\"]}",{}]}]}]},"$undefined","$undefined",true],["",{"children":["docs",{"children":[["slug","operations/runbooks/DEPLOYMENT","c"],{"children":["__PAGE__",{},[["$L1",["$","div",null,{"children":[["$","div",null,{"className":"mb-6 flex items-center justify-between gap-4","children":[["$","div",null,{"children":[["$","p",null,{"className":"text-sm text-gray-500 dark:text-gray-400","children":"Docs / Raw"}],["$","h1",null,{"className":"text-3xl font-bold text-gray-900 dark:text-white","children":"Deployment Runbook"}],["$","p",null,{"className":"text-sm text-gray-600 dark:text-gray-400","children":["Sourced from"," ",["$","code",null,{"className":"font-mono text-xs","children":["docs/","operations/runbooks/DEPLOYMENT.md"]}]]}]]}],["$","a",null,{"href":"https://github.com/mohammednazmy/VoiceAssist/edit/main/docs/operations/runbooks/DEPLOYMENT.md","target":"_blank","rel":"noreferrer","className":"inline-flex items-center gap-2 rounded-md border border-gray-200 dark:border-gray-700 px-3 py-1.5 text-sm text-gray-700 dark:text-gray-200 hover:border-primary-500 dark:hover:border-primary-400 hover:text-primary-700 dark:hover:text-primary-300","children":"Edit on GitHub"}]]}],["$","div",null,{"className":"rounded-lg border border-gray-200 dark:border-gray-800 bg-white dark:bg-gray-900 p-6","children":["$","$L2",null,{"content":"$3"}]}],["$","div",null,{"className":"mt-6 flex flex-wrap gap-2 text-sm","children":[["$","$L4",null,{"href":"/reference/all-docs","className":"inline-flex items-center gap-1 rounded-md bg-gray-100 px-3 py-1 text-gray-700 hover:bg-gray-200 dark:bg-gray-800 dark:text-gray-200 dark:hover:bg-gray-700","children":"← All documentation"}],["$","$L4",null,{"href":"/","className":"inline-flex items-center gap-1 rounded-md bg-gray-100 px-3 py-1 text-gray-700 hover:bg-gray-200 dark:bg-gray-800 dark:text-gray-200 dark:hover:bg-gray-700","children":"Home"}]]}]]}],null],null],null]},[null,["$","$L5",null,{"parallelRouterKey":"children","segmentPath":["children","docs","children","$6","children"],"error":"$undefined","errorStyles":"$undefined","errorScripts":"$undefined","template":["$","$L7",null,{}],"templateStyles":"$undefined","templateScripts":"$undefined","notFound":"$undefined","notFoundStyles":"$undefined"}]],null]},[null,["$","$L5",null,{"parallelRouterKey":"children","segmentPath":["children","docs","children"],"error":"$undefined","errorStyles":"$undefined","errorScripts":"$undefined","template":["$","$L7",null,{}],"templateStyles":"$undefined","templateScripts":"$undefined","notFound":"$undefined","notFoundStyles":"$undefined"}]],null]},[[[["$","link","0",{"rel":"stylesheet","href":"/_next/static/css/7f586cdbbaa33ff7.css","precedence":"next","crossOrigin":"$undefined"}]],["$","html",null,{"lang":"en","className":"h-full","children":["$","body",null,{"className":"__className_f367f3 h-full bg-white dark:bg-gray-900","children":[["$","a",null,{"href":"#main-content","className":"skip-to-content","children":"Skip to main content"}],["$","$L8",null,{"children":[["$","$L9",null,{}],["$","$La",null,{}],["$","main",null,{"id":"main-content","className":"lg:pl-64","role":"main","aria-label":"Documentation content","children":["$","$Lb",null,{"children":["$","$L5",null,{"parallelRouterKey":"children","segmentPath":["children"],"error":"$undefined","errorStyles":"$undefined","errorScripts":"$undefined","template":["$","$L7",null,{}],"templateStyles":"$undefined","templateScripts":"$undefined","notFound":[["$","title",null,{"children":"404: This page could not be found."}],["$","div",null,{"style":{"fontFamily":"system-ui,\"Segoe UI\",Roboto,Helvetica,Arial,sans-serif,\"Apple Color Emoji\",\"Segoe UI Emoji\"","height":"100vh","textAlign":"center","display":"flex","flexDirection":"column","alignItems":"center","justifyContent":"center"},"children":["$","div",null,{"children":[["$","style",null,{"dangerouslySetInnerHTML":{"__html":"body{color:#000;background:#fff;margin:0}.next-error-h1{border-right:1px solid rgba(0,0,0,.3)}@media (prefers-color-scheme:dark){body{color:#fff;background:#000}.next-error-h1{border-right:1px solid rgba(255,255,255,.3)}}"}}],["$","h1",null,{"className":"next-error-h1","style":{"display":"inline-block","margin":"0 20px 0 0","padding":"0 23px 0 0","fontSize":24,"fontWeight":500,"verticalAlign":"top","lineHeight":"49px"},"children":"404"}],["$","div",null,{"style":{"display":"inline-block"},"children":["$","h2",null,{"style":{"fontSize":14,"fontWeight":400,"lineHeight":"49px","margin":0},"children":"This page could not be found."}]}]]}]}]],"notFoundStyles":[]}]}]}]]}]]}]}]],null],null],["$Lc",null]]]] c:[["$","meta","0",{"name":"viewport","content":"width=device-width, initial-scale=1"}],["$","meta","1",{"charSet":"utf-8"}],["$","title","2",{"children":"Deployment Runbook | Docs | VoiceAssist Docs"}],["$","meta","3",{"name":"description","content":"Step-by-step guide for deploying VoiceAssist V2 to production."}],["$","meta","4",{"name":"keywords","content":"VoiceAssist,documentation,medical AI,voice assistant,healthcare,HIPAA,API"}],["$","meta","5",{"name":"robots","content":"index, follow"}],["$","meta","6",{"name":"googlebot","content":"index, follow"}],["$","link","7",{"rel":"canonical","href":"https://assistdocs.asimo.io"}],["$","meta","8",{"property":"og:title","content":"VoiceAssist Documentation"}],["$","meta","9",{"property":"og:description","content":"Comprehensive documentation for VoiceAssist - Enterprise Medical AI Assistant"}],["$","meta","10",{"property":"og:url","content":"https://assistdocs.asimo.io"}],["$","meta","11",{"property":"og:site_name","content":"VoiceAssist Docs"}],["$","meta","12",{"property":"og:type","content":"website"}],["$","meta","13",{"name":"twitter:card","content":"summary"}],["$","meta","14",{"name":"twitter:title","content":"VoiceAssist Documentation"}],["$","meta","15",{"name":"twitter:description","content":"Comprehensive documentation for VoiceAssist - Enterprise Medical AI Assistant"}],["$","meta","16",{"name":"next-size-adjust"}]] 1:null