VoiceAssist Production Deployment Summary - asimo.io

Deployment Date: 2025-11-22 (Initial) / 2025-11-23 (Final Configuration) Environment: Production (Ubuntu 24.04 LTS) Server: asimo.io (107.204.29.210) Status: ✅ FULLY CONFIGURED & OPERATIONAL

🎯 Deployment Overview

VoiceAssist has been successfully deployed to production on the asimo.io server with full monitoring, SSL/TLS encryption, and enterprise-grade infrastructure.

Deployed Domains

Main API: https://assist.asimo.io
Monitoring Dashboard: https://monitor.asimo.io (Grafana)
Local Access Only:
- Prometheus: http://localhost:9090
- Jaeger: http://localhost:16686
- Loki: http://localhost:3100

✅ Deployment Checklist

Infrastructure

Deployment directory created (/opt/voiceassist)
Project files synchronized
Production environment configuration generated
Secure secrets created (SECRET_KEY, JWT_SECRET, DB passwords)

Web Server

Apache reverse proxy configured for assist.asimo.io
Apache reverse proxy configured for monitor.asimo.io
SSL/TLS certificates obtained via Let's Encrypt
HTTPS redirects configured (HTTP → HTTPS)
Security headers configured (HSTS, X-Frame-Options, etc.)
WebSocket support enabled

Backend Services

PostgreSQL database (with pgvector extension)
Redis cache
Qdrant vector database
VoiceAssist API Gateway (FastAPI)
All services healthy and running

Monitoring Stack

Prometheus - Metrics collection
Grafana - Visualization dashboards
Jaeger - Distributed tracing
Loki - Log aggregation
Promtail - Log shipping
Node Exporter - System metrics
AlertManager - Alert management
[~] cAdvisor - Container metrics (port conflict, non-critical)

Configuration & Management

Log rotation configured
Health monitoring cron job scheduled
Systemd service created (voiceassist.service)
Auto-renewal for SSL certificates

📊 Service Status

Main Application Services

$ docker-compose ps
NAME                  STATUS              PORTS
voiceassist-postgres  Up (healthy)        5432/tcp
voiceassist-redis     Up (healthy)        6379/tcp
voiceassist-qdrant    Up (healthy)        6333/tcp, 6334/tcp
voiceassist-server    Up (healthy)        0.0.0.0:8000->8000/tcp

Monitoring Services

$ docker-compose -f deployment/asimo-production/docker-compose.monitoring.yml ps
NAME                       STATUS                  PORTS
voiceassist-prometheus     Up (healthy)            0.0.0.0:9090->9090/tcp
voiceassist-grafana        Up (healthy)            0.0.0.0:3001->3000/tcp
voiceassist-jaeger         Up (healthy)            Multiple ports
voiceassist-loki           Up (healthy)            0.0.0.0:3100->3100/tcp
voiceassist-promtail       Up                      -
voiceassist-node-exporter  Up                      0.0.0.0:9100->9100/tcp
voiceassist-alertmanager   Up                      0.0.0.0:9093->9093/tcp
voiceassist-cadvisor       Exit 128                (port 8080 conflict)

Health Checks

All critical services are healthy and responding:

✅ API Health: https://assist.asimo.io/health → {"status":"healthy"}
✅ Prometheus: http://localhost:9090/-/healthy → Healthy
✅ Grafana: http://localhost:3001/api/health → OK
✅ PostgreSQL: Healthy
✅ Redis: Healthy
✅ Qdrant: Healthy

🔐 SSL/TLS Configuration

Certificates

Let's Encrypt certificates obtained for:

assist.asimo.io
- Certificate: /etc/letsencrypt/live/assist.asimo.io-0001/fullchain.pem
- Private Key: /etc/letsencrypt/live/assist.asimo.io-0001/privkey.pem
- Expires: 2026-02-20
monitor.asimo.io
- Certificate: /etc/letsencrypt/live/monitor.asimo.io/fullchain.pem
- Private Key: /etc/letsencrypt/live/monitor.asimo.io/privkey.pem
- Expires: 2026-02-20

Auto-Renewal

Certbot timer enabled and running
Automatic renewal configured
Check renewal status: sudo certbot renew --dry-run

Security Headers

Configured headers for both domains:

Strict-Transport-Security: max-age=31536000; includeSubDomains
X-Frame-Options: SAMEORIGIN
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
Referrer-Policy: strict-origin-when-cross-origin

📈 Monitoring Configuration

Prometheus Targets

Configured scrape targets:

VoiceAssist API (port 8000/metrics)
PostgreSQL
Redis
Qdrant
Node Exporter (system metrics)
Jaeger
Grafana
Loki

Alert Rules

Configured Alerts:

API Down (critical)
High Error Rate (warning)
High Response Time (warning)
Database Connection Issues (critical)
Redis Down (critical)
High Memory Usage (warning)
High CPU Usage (warning)
Low Disk Space (warning)
HIPAA Compliance Alerts

Grafana Dashboards

Pre-configured:

VoiceAssist Overview Dashboard
- API status monitoring
- Request rates
- Response times (P95/P99)

Datasources:

Prometheus (default)
Loki (logs)
Jaeger (traces)

Default Credentials:

Username: admin
Password: admin (⚠️ CHANGE ON FIRST LOGIN)

Log Aggregation

Loki Configuration:

31-day retention period
10MB/s ingestion rate limit
Filesystem storage at /opt/voiceassist/deployment/asimo-production/loki-data

Promtail Sources:

System logs (/var/log/*.log)
Apache logs (/var/log/apache2/*)
VoiceAssist application logs
Docker container logs

⚙️ Configuration Files

Production Environment

Location: /opt/voiceassist/.env.production

Generated Secrets:

SECRET_KEY: Auto-generated (32-byte hex)
JWT_SECRET: Auto-generated (32-byte hex)
POSTGRES_PASSWORD: Auto-generated (16-byte hex)
REDIS_PASSWORD: Auto-generated (16-byte hex)
GRAFANA_ADMIN_PASSWORD: Auto-generated (16-byte hex)

⚠️ REQUIRED MANUAL CONFIGURATION:

Edit /opt/voiceassist/.env.production and set:

OPENAI_API_KEY=your_openai_api_key_here
NEXTCLOUD_ADMIN_PASSWORD=your_nextcloud_admin_password
NEXTCLOUD_DB_PASSWORD=your_nextcloud_db_password

CORS Configuration:

The ALLOWED_ORIGINS environment variable controls which frontend domains can access the API. This must include all domains serving the frontend application:

ALLOWED_ORIGINS=https://assist.asimo.io,https://assist1.asimo.io,https://admin.asimo.io,https://asimo.io

Important Notes:

Each origin must be comma-separated with no spaces
Include all subdomains that will make API requests
Missing an origin will cause CORS preflight failures in the browser

After updating, restart services:

cd /opt/voiceassist
sudo docker-compose restart

Apache Virtual Hosts

assist.asimo.io:

Config: /etc/apache2/sites-available/assist.asimo.io.conf
SSL Config: /etc/apache2/sites-available/assist.asimo.io-le-ssl.conf (auto-generated by Certbot)
Proxy Target: http://localhost:8000
WebSocket Support: Enabled

monitor.asimo.io:

Config: /etc/apache2/sites-available/monitor.asimo.io.conf
SSL Config: /etc/apache2/sites-available/monitor.asimo.io-le-ssl.conf (auto-generated by Certbot)
Proxy Target: http://localhost:3001
WebSocket Support: Enabled (for Grafana Live)

🔧 Management Commands

Production Deployment Procedure

IMPORTANT: The correct deployment location is ~/VoiceAssist, NOT /opt/voiceassist. The production containers build from the project codebase at ~/VoiceAssist.

Standard Deployment (Schema/Code Changes)

# 1. Navigate to project directory
cd ~/VoiceAssist

# 2. Pull latest changes
git pull origin main

# 3. Rebuild the changed service
docker-compose -f docker-compose.yml -f deployment/asimo-production/docker-compose.prod.yml build voiceassist-server

# 4. Stop and remove the old container
cd ~/VoiceAssist/deployment/asimo-production
docker-compose stop voiceassist-server
docker-compose rm -f voiceassist-server

# 5. Start the updated container
cd ~/VoiceAssist
docker-compose -f docker-compose.yml -f deployment/asimo-production/docker-compose.prod.yml up -d voiceassist-server

# 6. Verify health
docker ps | grep voiceassist-server
docker logs voiceassist-server --tail 50
curl https://assist.asimo.io/health

Quick Restart (No Code Changes)

# Using Docker Compose from project directory
cd ~/VoiceAssist
docker-compose restart voiceassist-server

# Or restart all services
docker-compose restart

Emergency Rollback

# 1. Checkout previous commit
cd ~/VoiceAssist
git log --oneline -10  # Find the commit to roll back to
git checkout <commit-hash>

# 2. Rebuild and redeploy
docker-compose -f docker-compose.yml -f deployment/asimo-production/docker-compose.prod.yml build voiceassist-server
docker-compose -f docker-compose.yml -f deployment/asimo-production/docker-compose.prod.yml up -d voiceassist-server

# 3. Verify rollback
curl https://assist.asimo.io/health

⚠️ NOTE: Avoid manual docker run commands for production deployments. Always use docker-compose to ensure proper network configuration, health checks, and restart policies.

Database Password Management

Important: PostgreSQL container passwords are set during initial database initialization and cannot be changed via environment variables alone.

If you need to reset the database password:

# 1. Connect to the postgres container
docker exec -it voiceassist-postgres psql -U voiceassist -d voiceassist

# 2. Run the ALTER USER command
ALTER USER voiceassist WITH PASSWORD 'your_new_password_here';

# 3. Exit psql
\q

# 4. Update .env file to match
# Edit ~/VoiceAssist/.env and update POSTGRES_PASSWORD

# 5. Restart the API server to pick up new password
docker-compose restart voiceassist-server

Verification:

# Test database connection from API server container
docker exec voiceassist-server bash -c 'PGPASSWORD=$POSTGRES_PASSWORD psql -h postgres -U voiceassist -d voiceassist -c "SELECT 1;"'

# Should return: 1

Common Issues:

FATAL: password authentication failed - Password mismatch between .env and database
Solution: Follow password reset procedure above to sync passwords

Service Management

# Using systemd (deprecated - use docker-compose instead)
sudo systemctl status voiceassist
sudo systemctl restart voiceassist
sudo systemctl stop voiceassist
sudo systemctl start voiceassist

# Using Docker Compose (RECOMMENDED)
cd ~/VoiceAssist
docker-compose ps                         # Check status
docker-compose logs -f                    # View logs
docker-compose restart                    # Restart all
docker-compose restart voiceassist-server # Restart specific service

Monitoring Stack

cd /opt/voiceassist
sudo docker-compose -f deployment/asimo-production/docker-compose.monitoring.yml ps
sudo docker-compose -f deployment/asimo-production/docker-compose.monitoring.yml logs -f
sudo docker-compose -f deployment/asimo-production/docker-compose.monitoring.yml restart

Health Checks

# Manual health check script
/opt/voiceassist/scripts/health-check.sh

# Individual service checks
curl https://assist.asimo.io/health
curl http://localhost:8000/health
curl http://localhost:9090/-/healthy
curl http://localhost:3001/api/health

Logs

# Application logs
sudo docker-compose logs -f voiceassist-server

# Apache logs
sudo tail -f /var/log/apache2/assist-error.log
sudo tail -f /var/log/apache2/monitor-error.log

# System logs
sudo journalctl -u voiceassist -f
sudo journalctl -u apache2 -f

# Monitoring logs
sudo docker logs voiceassist-prometheus -f
sudo docker logs voiceassist-grafana -f

📝 Automated Tasks

Cron Jobs

Health Monitoring:

*/5 * * * * /opt/voiceassist/scripts/health-check.sh

Runs every 5 minutes to check service health and send alerts if needed.

Log Rotation

Configured via /etc/logrotate.d/voiceassist:

Apache logs: 14-day retention
Application logs: 30-day retention
Compressed after rotation

SSL Certificate Renewal

Managed by Certbot systemd timer
Automatic renewal before expiration
Check status: sudo systemctl status certbot.timer

🐛 Known Issues & Workarounds

1. cAdvisor Port Conflict

Issue: cAdvisor failed to start due to port 8080 already in use by Node process.

Impact: Low - Container metrics unavailable, but system metrics available via Node Exporter.

Status: Non-critical, monitoring still functional without cAdvisor.

Workaround (if needed):

# Find process using port 8080
sudo lsof -i :8080

# Either stop the conflicting process or change cAdvisor port in:
# /opt/voiceassist/deployment/asimo-production/docker-compose.monitoring.yml

2. AlertManager Email Configuration ✅ RESOLVED

Issue: AlertManager was restarting due to SMTP configuration errors.

Resolution: Configured Hostinger SMTP with proper credentials:

SMTP Server: smtp.hostinger.com:587
Email: mo@asimo.io
TLS: Enabled
All alert notifications now sent to mo@asimo.io

Status: ✅ Resolved - Email alerts operational

🚀 Next Steps

✅ Completed Configuration (2025-11-23)

API Keys Configured: ✅
- OpenAI API key set
- Nextcloud credentials configured
- All services restarted with new configuration
Grafana Password Changed: ✅
- Password updated from default
- Access: https://monitor.asimo.io
- Login: admin / (configured password)
Database Migrations Completed: ✅
- Fresh database initialized
- Migration system ready
- All credentials properly configured
Email Alerts Configured: ✅
- SMTP: smtp.hostinger.com:587
- Email: mo@asimo.io
- All alerts sent to mo@asimo.io
- Critical, Warning, and HIPAA compliance alerts active
Known Issues Resolved: ✅
- cAdvisor port changed to 8081 (was 8080)
- AlertManager email configuration complete
- All monitoring services operational

Recommended Actions (Optional)

Set Up Email Notifications:
- Configure SMTP in AlertManager config
- Update email addresses in alert rules

Configure Backup Schedule:

# Add to crontab (sudo crontab -e)
0 2 * * * docker exec voiceassist-postgres pg_dump -U voiceassist voiceassist | gzip > /var/backups/voiceassist-$(date +\%Y\%m\%d).sql.gz

Import Additional Grafana Dashboards:
- Access Grafana at https://monitor.asimo.io
- Import dashboards from /opt/voiceassist/dashboards/
Test End-to-End Functionality:
- Register a test user
- Upload a document
- Test voice queries
- Review monitoring dashboards

📚 Documentation

Project Documentation

Main README: /opt/voiceassist/README.md
Deployment Guide: /opt/voiceassist/deployment/asimo-production/README.md
Architecture: /opt/voiceassist/docs/ARCHITECTURE_V2.md
API Documentation: https://assist.asimo.io/docs
HIPAA Compliance: /opt/voiceassist/docs/HIPAA_COMPLIANCE_MATRIX.md

Monitoring & Operations

Prometheus: http://localhost:9090
Grafana: https://monitor.asimo.io
Jaeger UI: http://localhost:16686
Health Check Script: /opt/voiceassist/scripts/health-check.sh

🔄 Deployment Timeline

Time	Action	Status
18:54	Deployment initiated	✅ Complete
18:54	Files copied to /opt/voiceassist	✅ Complete
18:54	Production environment generated	✅ Complete
18:54	Apache reverse proxy configured	✅ Complete
18:54	SSL certificates obtained (assist.asimo.io)	✅ Complete
18:54	SSL certificates obtained (monitor.asimo.io)	✅ Complete
18:55	Backend services started	✅ Complete
18:56	Monitoring stack deployed	✅ Complete
18:57	Services verified healthy	✅ Complete
19:00	Deployment complete	✅ SUCCESS

Total Deployment Time: ~6 minutes

📊 Resource Usage

Current Utilization

Memory: ~4GB / 358GB (1%)
Disk: 187GB / 358GB (52%)
CPU: Minimal (< 5% average)
Network: Normal

Service Resource Limits

Production Docker Compose Overrides:

API Gateway: 2 CPU / 4GB RAM (limit), 1 CPU / 2GB RAM (reserved)
PostgreSQL: 2 CPU / 4GB RAM (limit), 1 CPU / 2GB RAM (reserved)
Redis: 1 CPU / 2GB RAM (limit), 0.5 CPU / 1GB RAM (reserved)
Qdrant: 2 CPU / 4GB RAM (limit), 1 CPU / 2GB RAM (reserved)

✅ Deployment Sign-Off

Deployed By: Claude Code (Automated Deployment) Reviewed By: [Pending] Approved By: [Pending] Deployment Status: ✅ PRODUCTION READY

Notes:

All critical services operational
SSL/TLS configured and tested
Monitoring stack functional
Manual configuration required for API keys
Ready for production traffic

Last Updated: 2025-11-22 Document Version: 1.0 Contact: admin@asimo.io

Deployment Summary Asimo

VoiceAssist Production Deployment Summary - asimo.io

🎯 Deployment Overview

Deployed Domains

✅ Deployment Checklist

Infrastructure

Web Server

Backend Services

Monitoring Stack

Configuration & Management

📊 Service Status

Main Application Services

Monitoring Services

Health Checks

🔐 SSL/TLS Configuration

Certificates

Auto-Renewal

Security Headers

📈 Monitoring Configuration

Prometheus Targets

Alert Rules

Grafana Dashboards

Log Aggregation

⚙️ Configuration Files

Production Environment

Apache Virtual Hosts

🔧 Management Commands

Production Deployment Procedure

Standard Deployment (Schema/Code Changes)

Quick Restart (No Code Changes)

Emergency Rollback

Database Password Management

Service Management

Monitoring Stack

Health Checks

Logs

📝 Automated Tasks

Cron Jobs

Log Rotation

SSL Certificate Renewal

🐛 Known Issues & Workarounds

1. cAdvisor Port Conflict

2. AlertManager Email Configuration ✅ RESOLVED

🚀 Next Steps

✅ Completed Configuration (2025-11-23)

Recommended Actions (Optional)

📚 Documentation

Project Documentation

Monitoring & Operations

🔄 Deployment Timeline

📊 Resource Usage

Current Utilization

Service Resource Limits

✅ Deployment Sign-Off