beforeLocal

walkthrough.md

@@ -0,0 +1,87 @@
+# Phase 1 Implementation Walkthrough
+
+The foundational Phase 1 (Retrieval-Only Architecture) for your Unified Email Semantic Search system is now complete. The repository structure, Docker configurations, backend API, indexing daemon, and frontend have been successfully scaffolded in `/home/adipu/AllMail`.
+
+## Repository Structure Overview
+
+```mermaid
+graph TD
+    A[AllMail/] --> B(db_data/)
+    A --> C(Maildir/)
+    A --> D(mail-sync/)
+    A --> E(api/)
+    A --> F(webmail/)
+    A --> G(.gitea/)
+    A --> H(docker-compose.yml)
+    A --> I(.env.example)
+
+    D --> D1[Dockerfile]
+    D --> D2[entrypoint.sh]
+    D --> D3[mbsyncrc.template]
+
+    E --> E1[Dockerfile]
+    E --> E2[main.py]
+    E --> E3[indexer.py]
+    E --> E4[database.py]
+    E --> E5[run.sh]
+
+    F --> F1[Dockerfile]
+    F --> F2[nginx.conf]
+    F --> F3[entrypoint.sh]
+    F --> F4[spa/]
+```
+
+## Key Components Implemented
+
+### 1. Docker Infrastructure & Resource Management
+- Created `docker-compose.yml` with memory limits applied specifically to respect the 16GB total memory overhead:
+  - `db` is restricted to `1G` (and Postgres tuned via command arguments `shared_buffers=256MB`).
+  - `api` is restricted to `512M`.
+  - `mail-sync` is restricted to `256M`.
+  - `webmail` is restricted to `512M`.
+- Uses real filesystem mounts for persistent data: `./db_data` for PostgreSQL and `./Maildir` for the email files.
+
+### 2. Database & API (`api/`)
+- A Python 3.11 container running both a FastAPI server (`main.py`) and a background Watchdog daemon (`indexer.py`).
+- Integrates `pgvector` and uses SQLAlchemy to manage vector embeddings.
+- Automatically connects to OpenAI for `text-embedding-3-small` and indexes new files arriving in `./Maildir`.
+
+### 3. Synchronization (`mail-sync/`)
+- A lightweight Alpine image utilizing `isync` (mbsync).
+- Contains an entrypoint that initializes a 5-minute cron schedule.
+- A `mbsyncrc.template` was created to serve as your configuration starting point.
+
+### 4. Frontend & SnappyMail (`webmail/`)
+- A custom `php:8.2-fpm-alpine` container running Nginx.
+- Serves a React/Vite SPA on the root (`/`) for querying the Unified Search API.
+- Proxies `/mail/` to SnappyMail for standard webmail consumption.
+- The `Dockerfile` handles building the React app automatically during `docker compose build`.
+
+### 5. Deployment CI/CD (`.gitea/`)
+- A GitHub-Actions-compatible workflow is present at `.gitea/workflows/deploy.yml` which triggers on pushes to `main` and runs the requested commands (`docker compose up -d --build` & `docker image prune -f`).
+
+## Next Steps for You
+
+> [!IMPORTANT]
+> **Action Required**: The system is ready to be started, but you need to configure your secrets and accounts before running `docker compose up -d`.
+
+1. **Configure Environment Variables**:
+   Copy `.env.example` to `.env` and fill in your passwords and the OpenAI API Key.
+   ```bash
+   cp /home/adipu/AllMail/.env.example /home/adipu/AllMail/.env
+   # Edit .env
+   ```
+
+2. **Configure Mail Accounts**:
+   Copy the `mbsyncrc` template to an actual config file and configure your IMAP servers.
+   ```bash
+   cp /home/adipu/AllMail/mail-sync/mbsyncrc.template /home/adipu/AllMail/mail-sync/mbsyncrc
+   # Edit mbsyncrc
+   ```
+
+3. **Deploy Phase 1**:
+   After configuring the above files, you can start the system locally to ensure it builds correctly:
+   ```bash
+   cd /home/adipu/AllMail
+   docker compose up -d --build
+   ```