88 lines
3.3 KiB
Markdown
88 lines
3.3 KiB
Markdown
# Phase 1 Implementation Walkthrough
|
|
|
|
The foundational Phase 1 (Retrieval-Only Architecture) for your Unified Email Semantic Search system is now complete. The repository structure, Docker configurations, backend API, indexing daemon, and frontend have been successfully scaffolded in `/home/adipu/AllMail`.
|
|
|
|
## Repository Structure Overview
|
|
|
|
```mermaid
|
|
graph TD
|
|
A[AllMail/] --> B(db_data/)
|
|
A --> C(Maildir/)
|
|
A --> D(mail-sync/)
|
|
A --> E(api/)
|
|
A --> F(webmail/)
|
|
A --> G(.gitea/)
|
|
A --> H(docker-compose.yml)
|
|
A --> I(.env.example)
|
|
|
|
D --> D1[Dockerfile]
|
|
D --> D2[entrypoint.sh]
|
|
D --> D3[mbsyncrc.template]
|
|
|
|
E --> E1[Dockerfile]
|
|
E --> E2[main.py]
|
|
E --> E3[indexer.py]
|
|
E --> E4[database.py]
|
|
E --> E5[run.sh]
|
|
|
|
F --> F1[Dockerfile]
|
|
F --> F2[nginx.conf]
|
|
F --> F3[entrypoint.sh]
|
|
F --> F4[spa/]
|
|
```
|
|
|
|
## Key Components Implemented
|
|
|
|
### 1. Docker Infrastructure & Resource Management
|
|
- Created `docker-compose.yml` with memory limits applied specifically to respect the 16GB total memory overhead:
|
|
- `db` is restricted to `1G` (and Postgres tuned via command arguments `shared_buffers=256MB`).
|
|
- `api` is restricted to `512M`.
|
|
- `mail-sync` is restricted to `256M`.
|
|
- `webmail` is restricted to `512M`.
|
|
- Uses real filesystem mounts for persistent data: `./db_data` for PostgreSQL and `./Maildir` for the email files.
|
|
|
|
### 2. Database & API (`api/`)
|
|
- A Python 3.11 container running both a FastAPI server (`main.py`) and a background Watchdog daemon (`indexer.py`).
|
|
- Integrates `pgvector` and uses SQLAlchemy to manage vector embeddings.
|
|
- Automatically connects to OpenAI for `text-embedding-3-small` and indexes new files arriving in `./Maildir`.
|
|
|
|
### 3. Synchronization (`mail-sync/`)
|
|
- A lightweight Alpine image utilizing `isync` (mbsync).
|
|
- Contains an entrypoint that initializes a 5-minute cron schedule.
|
|
- A `mbsyncrc.template` was created to serve as your configuration starting point.
|
|
|
|
### 4. Frontend & SnappyMail (`webmail/`)
|
|
- A custom `php:8.2-fpm-alpine` container running Nginx.
|
|
- Serves a React/Vite SPA on the root (`/`) for querying the Unified Search API.
|
|
- Proxies `/mail/` to SnappyMail for standard webmail consumption.
|
|
- The `Dockerfile` handles building the React app automatically during `docker compose build`.
|
|
|
|
### 5. Deployment CI/CD (`.gitea/`)
|
|
- A GitHub-Actions-compatible workflow is present at `.gitea/workflows/deploy.yml` which triggers on pushes to `main` and runs the requested commands (`docker compose up -d --build` & `docker image prune -f`).
|
|
|
|
## Next Steps for You
|
|
|
|
> [!IMPORTANT]
|
|
> **Action Required**: The system is ready to be started, but you need to configure your secrets and accounts before running `docker compose up -d`.
|
|
|
|
1. **Configure Environment Variables**:
|
|
Copy `.env.example` to `.env` and fill in your passwords and the OpenAI API Key.
|
|
```bash
|
|
cp /home/adipu/AllMail/.env.example /home/adipu/AllMail/.env
|
|
# Edit .env
|
|
```
|
|
|
|
2. **Configure Mail Accounts**:
|
|
Copy the `mbsyncrc` template to an actual config file and configure your IMAP servers.
|
|
```bash
|
|
cp /home/adipu/AllMail/mail-sync/mbsyncrc.template /home/adipu/AllMail/mail-sync/mbsyncrc
|
|
# Edit mbsyncrc
|
|
```
|
|
|
|
3. **Deploy Phase 1**:
|
|
After configuring the above files, you can start the system locally to ensure it builds correctly:
|
|
```bash
|
|
cd /home/adipu/AllMail
|
|
docker compose up -d --build
|
|
```
|