Db migrations

DEBUGGING_MIGRATIONS.md

@@ -0,0 +1,205 @@
+# Debugging Migration Issues in Docker
+
+## Quick Solutions
+
+### Container Exits Immediately
+
+Use one of these approaches:
+
+1. **Debug Mode (Recommended)**
+   ```bash
+   docker-compose down
+   DEBUG_MODE=true docker-compose up
+   ```
+
+2. **Skip Migrations Temporarily**
+   ```bash
+   docker-compose down
+   SKIP_MIGRATIONS=true docker-compose up
+   ```
+
+3. **Use Debug Compose File**
+   ```bash
+   docker-compose -f docker-compose.debug.yml up
+   docker exec -it timetrack_web_1 bash
+   ```
+
+## Debug Entrypoint
+
+The `debug_entrypoint.sh` keeps the container running and provides diagnostic info:
+
+```bash
+# In docker-compose.yml, change:
+command: ["./startup_postgres.sh"]
+# To:
+entrypoint: ["./debug_entrypoint.sh"]
+
+# Then:
+docker-compose up -d
+docker exec -it <container_name> bash
+```
+
+## Safe Startup Script
+
+`startup_postgres_safe.sh` has three modes:
+
+1. **Normal Mode**: Exits on migration failure (default)
+2. **Debug Mode**: Continues running even if migrations fail
+   ```bash
+   DEBUG_MODE=true docker-compose up
+   ```
+3. **Skip Mode**: Skips migrations entirely
+   ```bash
+   SKIP_MIGRATIONS=true docker-compose up
+   ```
+
+## Common Debugging Steps
+
+### 1. Get Into the Container
+```bash
+# If container keeps exiting, use debug compose:
+docker-compose -f docker-compose.debug.yml up -d web
+docker exec -it timetrack_web_1 bash
+
+# Or modify your docker-compose.yml:
+# Add: stdin_open: true
+# Add: tty: true
+# Change: entrypoint: ["/bin/bash"]
+```
+
+### 2. Manual Migration Setup
+```bash
+# Inside container:
+export FLASK_APP=app.py
+
+# Check what's wrong
+python diagnose_migrations.py
+
+# Initialize migrations
+python docker_migrate_init.py
+
+# Fix revision issues
+python fix_revision_mismatch.py
+```
+
+### 3. Database Connection Issues
+```bash
+# Test connection
+python -c "from app import app, db; app.app_context().push(); db.engine.execute('SELECT 1')"
+
+# Check environment
+echo $DATABASE_URL
+echo $POSTGRES_HOST
+```
+
+### 4. Reset Everything
+```bash
+# Inside container:
+rm -rf migrations
+python docker_migrate_init.py
+flask db stamp head  # For existing DB
+flask db upgrade     # For new DB
+```
+
+## Docker Compose Examples
+
+### Development with Auto-Restart
+```yaml
+services:
+  web:
+    environment:
+      - DEBUG_MODE=true
+    restart: unless-stopped  # Auto-restart on failure
+```
+
+### Interactive Debugging
+```yaml
+services:
+  web:
+    entrypoint: ["/app/debug_entrypoint.sh"]
+    stdin_open: true
+    tty: true
+```
+
+### Skip Migrations for Testing
+```yaml
+services:
+  web:
+    environment:
+      - SKIP_MIGRATIONS=true
+```
+
+## Environment Variables
+
+- `DEBUG_MODE=true` - Continue running even if migrations fail
+- `SKIP_MIGRATIONS=true` - Skip all migration steps
+- `FLASK_APP=app.py` - Required for Flask-Migrate
+- `DATABASE_URL` - PostgreSQL connection string
+
+## Step-by-Step Troubleshooting
+
+1. **Container won't start?**
+   ```bash
+   # Use debug compose
+   docker-compose -f docker-compose.debug.yml up
+   ```
+
+2. **Migration fails?**
+   ```bash
+   # Get into container
+   docker exec -it <container> bash
+
+   # Run diagnostics
+   python diagnose_migrations.py
+   ```
+
+3. **Revision mismatch?**
+   ```bash
+   # Quick fix
+   ./quick_fix_revision.sh
+
+   # Or manual fix
+   flask db stamp <revision>
+   ```
+
+4. **Can't initialize migrations?**
+   ```bash
+   # Check database connection first
+   python -c "from app import app; print(app.config['SQLALCHEMY_DATABASE_URI'])"
+
+   # Then initialize
+   python docker_migrate_init.py
+   ```
+
+## Tips
+
+1. **Always use volumes** for migrations directory in development
+2. **Check logs carefully** - the error is usually clear
+3. **Don't run migrations in production containers** - include pre-tested migrations in image
+4. **Use DEBUG_MODE** during development for easier troubleshooting
+5. **Test locally first** before deploying to production
+
+## Recovery Commands
+
+If everything is broken:
+
+```bash
+# 1. Start with debug entrypoint
+docker-compose -f docker-compose.debug.yml up -d web
+
+# 2. Get into container
+docker exec -it timetrack_web_1 bash
+
+# 3. Reset migrations
+rm -rf migrations
+python docker_migrate_init.py
+
+# 4. Mark as current (existing DB) or create tables (new DB)
+flask db stamp head  # Existing
+flask db upgrade     # New
+
+# 5. Test the app
+python app.py  # Run in debug mode
+
+# 6. If working, update docker-compose.yml and restart normally
+```

DOCKER_MIGRATIONS_GUIDE.md

@@ -0,0 +1,189 @@
+# Flask-Migrate in Docker Deployments
+
+## Overview
+
+Docker containers typically don't include Git repositories, so we can't use Git commands to extract historical schemas. This guide explains how to use Flask-Migrate in Docker environments.
+
+## Initial Setup (First Deployment)
+
+When deploying with Flask-Migrate for the first time:
+
+### Automatic Setup (via startup scripts)
+
+The `startup.sh` and `startup_postgres.sh` scripts now automatically handle migration initialization:
+
+1. **For existing databases with data:**
+   - Creates a baseline migration from current models
+   - Stamps the database as current (no changes applied)
+   - Ready for future migrations
+
+2. **For empty databases:**
+   - Creates a baseline migration from current models
+   - Applies it to create all tables
+   - Ready for future migrations
+
+### Manual Setup
+
+If you need to set up manually:
+
+```bash
+# Inside your Docker container
+python docker_migrate_init.py
+
+# For existing database with tables:
+flask db stamp head
+
+# For new empty database:
+flask db upgrade
+```
+
+## Creating New Migrations
+
+After initial setup, create new migrations normally:
+
+```bash
+# 1. Make changes to your models
+
+# 2. Generate migration
+flask db migrate -m "Add user preferences"
+
+# 3. Review the generated migration
+cat migrations/versions/*.py
+
+# 4. Apply the migration
+flask db upgrade
+```
+
+## Helper Script
+
+The `docker_migrate_init.py` script creates a `migrate.sh` helper:
+
+```bash
+# Check current migration status
+./migrate.sh status
+
+# Apply pending migrations
+./migrate.sh apply
+
+# Create new migration
+./migrate.sh create "Add company settings"
+
+# Mark database as current (existing DBs)
+./migrate.sh mark-current
+```
+
+## Docker Compose Example
+
+```yaml
+version: '3.8'
+services:
+  web:
+    build: .
+    environment:
+      - DATABASE_URL=postgresql://user:pass@db:5432/timetrack
+      - FLASK_APP=app.py
+    volumes:
+      # Persist migrations between container restarts
+      - ./migrations:/app/migrations
+    depends_on:
+      - db
+    command: ./startup_postgres.sh
+
+  db:
+    image: postgres:13
+    environment:
+      - POSTGRES_DB=timetrack
+      - POSTGRES_USER=user
+      - POSTGRES_PASSWORD=pass
+    volumes:
+      - postgres_data:/var/lib/postgresql/data
+
+volumes:
+  postgres_data:
+```
+
+## Important Notes
+
+### 1. Migrations Directory
+
+- The `migrations/` directory should be persisted between deployments
+- Either use a volume mount or include it in your Docker image
+- Don't regenerate migrations on each deployment
+
+### 2. Environment Variables
+
+Always set these in your Docker environment:
+```bash
+FLASK_APP=app.py
+DATABASE_URL=your_database_url
+```
+
+### 3. Production Workflow
+
+1. **Development**: Create and test migrations locally
+2. **Commit**: Add migration files to Git
+3. **Build**: Include migrations in Docker image
+4. **Deploy**: Startup script applies migrations automatically
+
+### 4. Rollback Strategy
+
+To rollback a migration:
+```bash
+# Inside container
+flask db downgrade  # Go back one migration
+flask db downgrade -2  # Go back two migrations
+```
+
+## Troubleshooting
+
+### "No Git repository found"
+
+This is expected in Docker. Use `docker_migrate_init.py` instead of the Git-based scripts.
+
+### "Can't locate revision"
+
+Your database references a migration that doesn't exist:
+```bash
+# Reset to current state
+python docker_migrate_init.py
+flask db stamp head
+```
+
+### Migration conflicts after deployment
+
+If migrations were created in different environments:
+```bash
+# Merge migrations
+flask db merge -m "Merge production and development"
+flask db upgrade
+```
+
+## Best Practices
+
+1. **Always test migrations** in a staging environment first
+2. **Back up your database** before applying migrations in production
+3. **Include migrations in your Docker image** for consistency
+4. **Don't generate migrations in production** - only apply pre-tested ones
+5. **Monitor the startup logs** to ensure migrations apply successfully
+
+## Migration State in Different Scenarios
+
+### Scenario 1: Fresh deployment, empty database
+- Startup script runs `docker_migrate_init.py`
+- Creates baseline migration
+- Applies it to create all tables
+
+### Scenario 2: Existing database, first Flask-Migrate setup
+- Startup script runs `docker_migrate_init.py`
+- Creates baseline migration matching current schema
+- Stamps database as current (no changes)
+
+### Scenario 3: Subsequent deployments with new migrations
+- Startup script detects `migrations/` exists
+- Runs `flask db upgrade` to apply new migrations
+
+### Scenario 4: Container restart (no new code)
+- Startup script detects `migrations/` exists
+- Runs `flask db upgrade` (no-op if already current)
+
+This approach ensures migrations work correctly in all Docker deployment scenarios!