Deployment Guide - Cortex AI¶

Overview¶

This guide covers deploying Cortex AI including the context engine, API servers, web portal, and VSCode extension.

For platform infrastructure setup (PostgreSQL, Zitadel, HAProxy, etc.), see: - Platform Documentation (doc://platform/*) - Local: ../../docs/ (if you have both repos cloned)

Prerequisites¶

Platform Infrastructure (Shared)¶

See Platform PostgreSQL Guide (doc://platform/postgres/overview) for: - PostgreSQL HA cluster setup - VIP configuration (192.168.11.62) - Replication and failover - Backup procedures - Connection pooling

See Platform Zitadel Guide (doc://platform/zitadel/overview) for: - OIDC application setup - Group management - User provisioning

See Platform HAProxy Guide (doc://platform/haproxy/overview) for: - Load balancer configuration - Backend health checks - SSL termination

Cortex-Specific Requirements¶

Two API servers (192.168.11.132, 192.168.11.133)
SSH access to all servers
Go 1.21+ for building
Qdrant vector database
Voyage AI API key

Deployment Steps¶

1. Database Setup¶

Platform Guide: See Platform PostgreSQL Guide for: - PostgreSQL HA cluster details - VIP address and connection details - Backup and restore procedures

Cortex-Specific Configuration:

Database: cortex
User: cortex_app
VIP: 192.168.11.62 (from platform PostgreSQL HA cluster)
Connection: Via PgBouncer connection pooling

Deploy Cortex database schemas:

# Step 1: Deploy user quotas schema
ssh root@192.168.11.136 "PGPASSWORD=cortex psql -h 192.168.11.62 -U cortex -d cortex" < cortex-web/schema/user_quotas.sql

# Step 2: Add usage counter columns
ssh root@192.168.11.136 "PGPASSWORD=cortex psql -h 192.168.11.62 -U cortex -d cortex" < cortex-web/schema/add_usage_counters.sql

# Step 3: Deploy usage tracking schema
ssh root@192.168.11.136 "PGPASSWORD=cortex psql -h 192.168.11.62 -U cortex -d cortex" < cortex-web/schema/usage_tracking.sql

# Step 4: Add quota limits
ssh root@192.168.11.136 "PGPASSWORD=cortex psql -h 192.168.11.62 -U cortex -d cortex" < cortex-web/schema/quota_limits.sql

Verify:

ssh root@192.168.11.136 "PGPASSWORD=cortex psql -h 192.168.11.62 -U cortex -d cortex -c '\dt'"

2. Build cortex-api¶

cd cortex-api
GOOS=linux GOARCH=amd64 go build -o cortex-api .

3. Deploy to API Servers¶

# Copy to both servers
scp cortex-api root@192.168.11.132:/opt/cortex-api/cortex-api-new
scp cortex-api root@192.168.11.133:/opt/cortex-api/cortex-api-new

# Deploy to server 132
ssh root@192.168.11.132 'systemctl stop cortex-api && \
  mv /opt/cortex-api/cortex-api-new /opt/cortex-api/cortex-api && \
  chmod +x /opt/cortex-api/cortex-api && \
  systemctl start cortex-api'

# Deploy to server 133
ssh root@192.168.11.133 'systemctl stop cortex-api && \
  mv /opt/cortex-api/cortex-api-new /opt/cortex-api/cortex-api && \
  chmod +x /opt/cortex-api/cortex-api && \
  systemctl start cortex-api'

4. Configure Authentication¶

Platform Guide: See Platform Zitadel Guide for: - OIDC application setup - Group management (cortex-admins, cortex-users) - User provisioning

Cortex-Specific Configuration:

Application: cortex-ai
Redirect URLs:
https://app.cortex.emshvac.co/auth/callback
http://localhost:8080/auth/callback (dev)
Groups:
cortex-admins - Full access to admin portal
cortex-users - Standard access

Configure authentication on both API servers:

# Server 132
ssh root@192.168.11.132 'cat > /etc/systemd/system/cortex-api.service.d/portal.conf << EOF
[Service]
Environment=PORTAL_DB_URL=postgresql://cortex:cortex@192.168.11.62:5432/cortex?sslmode=disable
Environment=PORTAL_AUTH_ENABLED=true
EOF
systemctl daemon-reload
systemctl restart cortex-api'

# Server 133
ssh root@192.168.11.133 'cat > /etc/systemd/system/cortex-api.service.d/portal.conf << EOF
[Service]
Environment=PORTAL_DB_URL=postgresql://cortex:cortex@192.168.11.62:5432/cortex?sslmode=disable
Environment=PORTAL_AUTH_ENABLED=true
EOF
systemctl daemon-reload
systemctl restart cortex-api'

5. Verify Deployment¶

# Run comprehensive test
./scripts/test-all-phases.sh

Configuration¶

Environment Variables¶

Required on both API servers:

PORTAL_DB_URL - PostgreSQL connection string for portal database
PORTAL_AUTH_ENABLED - Enable portal authentication (true/false)
CLOUDLLM_DB_URL - PostgreSQL connection string for quota checks

Database Configuration¶

Connection Details: - Host: 192.168.11.62 - Port: 5432 - Database: cortex - User: cortex - Password: cortex

Tier Configuration¶

Edit cortex-web/schema/quota_limits.sql to adjust tier limits:

-- Free tier limits
UPDATE user_quotas 
SET llm_requests_limit = 1000,
    context_searches_limit = 1000
WHERE tier = 'free';

-- Pro tier limits
UPDATE user_quotas 
SET llm_requests_limit = 10000,
    context_searches_limit = 10000
WHERE tier = 'pro';

Monitoring¶

Check API Server Status¶

ssh root@192.168.11.132 'systemctl status cortex-api'
ssh root@192.168.11.133 'systemctl status cortex-api'

Check Logs¶

# Recent logs
ssh root@192.168.11.132 'journalctl -u cortex-api -n 100 --no-pager'

# Follow logs
ssh root@192.168.11.132 'journalctl -u cortex-api -f'

# Quota-related logs
ssh root@192.168.11.132 'journalctl -u cortex-api --since "1 hour ago" | grep -i quota'

Database Queries¶

# Check user quotas
ssh root@192.168.11.136 "PGPASSWORD=cortex psql -h 192.168.11.62 -U cortex -d cortex -c 'SELECT email, tier, llm_requests_used, llm_requests_limit FROM user_quotas;'"

# Check usage
ssh root@192.168.11.136 "PGPASSWORD=cortex psql -h 192.168.11.62 -U cortex -d cortex -c 'SELECT COUNT(*), SUM(cost_usd) FROM cortex_usage;'"

Rollback¶

If issues occur, rollback to previous version:

# On each server
ssh root@192.168.11.132 'systemctl stop cortex-api && \
  mv /opt/cortex-api/cortex-api /opt/cortex-api/cortex-api-broken && \
  mv /opt/cortex-api/cortex-api.backup /opt/cortex-api/cortex-api && \
  systemctl start cortex-api'

Maintenance¶

Monthly Quota Reset¶

Set up cron job on database server:

# /etc/cron.d/cortex-quota-reset
0 0 1 * * postgres PGPASSWORD=cortex psql -h 192.168.11.62 -U cortex -d cortex -c "SELECT reset_monthly_quotas();"

Database Backups¶

# Backup user_quotas and cortex_usage tables
ssh root@192.168.11.136 "PGPASSWORD=cortex pg_dump -h 192.168.11.62 -U cortex -d cortex -t user_quotas -t cortex_usage > /backup/cortex-saas-$(date +%Y%m%d).sql"

Troubleshooting¶

Portal Auth Not Working¶

Check configuration:

ssh root@192.168.11.132 'systemctl cat cortex-api | grep PORTAL'

Verify database connection:

ssh root@192.168.11.132 'journalctl -u cortex-api -n 50 | grep -i "portal auth"'

Quota Not Enforced¶

Check usage tracker initialization:

ssh root@192.168.11.132 'journalctl -u cortex-api -n 50 | grep -i "usage tracker"'

Test quota function:

ssh root@192.168.11.136 "PGPASSWORD=cortex psql -h 192.168.11.62 -U cortex -d cortex -c \"SELECT * FROM has_quota_available('USER_ID', 'llm_request');\""

Usage Not Recorded¶

Check database connection:

ssh root@192.168.11.132 'journalctl -u cortex-api -n 50 | grep -i "cloudllm database"'

Test record_usage function:

ssh root@192.168.11.136 "PGPASSWORD=cortex psql -h 192.168.11.62 -U cortex -d cortex -c \"SELECT record_usage('USER_ID', 'llm_request', 100, 0.001, '{}'::jsonb);\""

Support¶

For issues, check: 1. docs/KNOWN_ISSUES.md - Known issues and workarounds 2. docs/TESTING_GUIDE.md - Testing procedures 3. API server logs - journalctl -u cortex-api 4. Database logs - PostgreSQL logs on 192.168.11.62