Compare commits

13 Commits

Author SHA1 Message Date
9b6c885da0 pres stuff
All checks were successful
Deploy to Server / deploy (push) Successful in 15s
2026-05-08 22:51:19 -05:00
880a726ddd ignore recipes
All checks were successful
Deploy to Server / deploy (push) Successful in 19s
2026-05-01 17:54:07 +00:00
a9f36bd5ba wtf
All checks were successful
Deploy to Server / deploy (push) Successful in 18s
2026-04-29 11:56:56 -05:00
193a825899 video understanding
Some checks failed
Build and Deploy / build (push) Failing after 2m17s
Build and Deploy / docker-build (push) Has been skipped
Deploy to Server / deploy (push) Successful in 43s
2026-04-29 11:50:44 -05:00
5bf0622c38 works
All checks were successful
Deploy to Server / deploy (push) Successful in 21s
2026-04-29 06:34:38 +00:00
560977064b trigger rebuild
All checks were successful
Deploy to Server / deploy (push) Successful in 9s
2026-04-29 06:29:18 +00:00
48e7614029 gated
All checks were successful
Deploy to Server / deploy (push) Successful in 14s
Co-authored-by: Copilot <copilot@github.com>
2026-04-29 06:25:42 +00:00
36139d08fc fix 2
All checks were successful
Deploy to Server / deploy (push) Successful in 28s
Co-authored-by: Copilot <copilot@github.com>
2026-04-29 01:41:25 +00:00
3019ec5884 fix
Some checks failed
Deploy to Server / deploy (push) Failing after 6s
2026-04-29 01:39:40 +00:00
6d78ebfa09 actions 2026-04-29 01:38:31 +00:00
2cbc63efd3 recipeGen v1
Co-authored-by: Copilot <copilot@github.com>
2026-04-28 19:24:03 -05:00
4831cc7255 init 2026-04-28 19:06:37 -05:00
31d0769d33 blank 2026-04-28 19:05:12 -05:00
57 changed files with 4054 additions and 6535 deletions

9
.dockerignore Normal file
View File

@@ -0,0 +1,9 @@
node_modules/
dist/
.env
recipes/
temp_ingest/
implementation_plan.md
.gitignore
.gitea/
*.md

View File

@@ -0,0 +1,42 @@
name: Deploy to Server
on:
push:
branches:
- recipeGen
jobs:
deploy:
runs-on: ubuntu-latest
container:
image: alpine:latest
steps:
- name: Install SSH and Networking Tools
run: apk add --no-cache openssh-client iproute2 git
- name: Configure SSH Key
run: |
mkdir -p ~/.ssh
echo "${{ secrets.DEPLOY_SSH_KEY }}" > ~/.ssh/id_ed25519
chmod 600 ~/.ssh/id_ed25519
echo "StrictHostKeyChecking no" > ~/.ssh/config
- name: Execute Remote Deployment
run: |
HOST_IP=$(ip route | awk '/default/ { print $3 }')
echo "==> Detected Host IP: $HOST_IP"
ssh adipu@$HOST_IP << 'EOF'
set -e
echo "==> Navigating to project directory..."
cd ~/SousChefAI
echo "==> Pulling latest code..."
git pull origin recipeGen
echo "==> Building and starting container..."
docker compose up -d --build
echo "==> Pruning old images..."
docker image prune -f
echo "==> Deployment Complete!"
EOF

4
.gitignore vendored
View File

@@ -60,3 +60,7 @@ fastlane/report.xml
fastlane/Preview.html
fastlane/screenshots/**/*.png
fastlane/test_output
node_modules/
.env
recipes/

14
Dockerfile Normal file
View File

@@ -0,0 +1,14 @@
FROM node:20-alpine
WORKDIR /app
# Install python3 and yt-dlp (no ffmpeg to save space)
RUN apk add --no-cache python3 py3-pip && \
pip3 install --break-system-packages yt-dlp
COPY package*.json ./
RUN npm install
COPY . .
RUN npm run build
EXPOSE 3000
CMD ["node", "server.js"]

Binary file not shown.

After

Width:  |  Height:  |  Size: 5.1 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 6.0 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 5.9 MiB

View File

@@ -1,88 +0,0 @@
# Privacy Configuration for SousChefAI
## Camera Permission Setup (Required)
The app needs camera access to scan ingredients and monitor cooking. Follow these steps to add the required privacy descriptions:
### Method 1: Using Xcode Target Settings (Recommended)
1. Open the project in Xcode
2. Select the **SousChefAI** target in the project navigator
3. Go to the **Info** tab
4. Under "Custom iOS Target Properties", click the **+** button
5. Add the following keys with their values:
**Camera Permission:**
- **Key**: `Privacy - Camera Usage Description`
- **Value**: `SousChefAI needs camera access to scan your fridge for ingredients and monitor your cooking progress in real-time.`
**Microphone Permission (for voice guidance):**
- **Key**: `Privacy - Microphone Usage Description`
- **Value**: `SousChefAI uses the microphone to provide voice-guided cooking instructions.`
### Method 2: Manual Info.plist (Alternative)
If you prefer to manually edit the Info.plist:
1. In Xcode, right-click on the SousChefAI folder
2. Select **New File****Property List**
3. Name it `Info.plist`
4. Add these entries:
```xml
<key>NSCameraUsageDescription</key>
<string>SousChefAI needs camera access to scan your fridge for ingredients and monitor your cooking progress in real-time.</string>
<key>NSMicrophoneUsageDescription</key>
<string>SousChefAI uses the microphone to provide voice-guided cooking instructions.</string>
```
## Verifying the Setup
After adding the privacy descriptions:
1. Clean the build folder: **Product → Clean Build Folder** (⌘ + Shift + K)
2. Rebuild the project: **Product → Build** (⌘ + B)
3. Run on a device or simulator
4. When you first open the Scanner view, you should see a permission dialog
## Troubleshooting
### "App crashed when accessing camera"
- Ensure you added `NSCameraUsageDescription` to the target's Info settings
- Clean and rebuild the project
- Restart Xcode if the permission isn't taking effect
### "Permission dialog not appearing"
- Check that the Info settings were saved
- Try deleting the app from the simulator/device and reinstalling
- Reset privacy settings on the simulator: **Device → Erase All Content and Settings**
### "Multiple Info.plist errors"
- Modern Xcode projects use automatic Info.plist generation
- Use Method 1 (Target Settings) instead of creating a manual file
- If you created Info.plist manually, make sure to configure the build settings to use it
## Privacy Manifest
The `PrivacyInfo.xcprivacy` file is included for App Store compliance. This declares:
- No tracking
- No third-party SDK tracking domains
- Camera access is for app functionality only
## Testing Camera Permissions
1. Build and run the app
2. Navigate to the **Scan** tab
3. You should see a permission dialog
4. Grant camera access
5. The camera preview should appear
If permission is denied:
- Go to **Settings → Privacy & Security → Camera**
- Find **SousChefAI** and enable it
- Relaunch the app
---
**Note**: These privacy descriptions are required by Apple's App Store guidelines. Apps that access camera without proper usage descriptions will be rejected.

View File

@@ -1,256 +0,0 @@
# SousChefAI - Project Summary
## 📱 Project Overview
**SousChefAI** is a production-ready iOS application that leverages multimodal AI to transform cooking. Users can scan their fridge to detect ingredients, receive personalized recipe suggestions, and get real-time cooking guidance through computer vision.
## 🎯 Key Features
### 1. AI-Powered Ingredient Scanner
- Real-time video inference using Overshoot API
- Confidence scoring for each detected item
- Manual entry fallback
- Low-confidence item highlighting
### 2. Intelligent Recipe Generation
- Google Gemini 2.0 for complex reasoning
- "The Scavenger" mode: uses only available ingredients
- "The Upgrader" mode: requires 1-2 additional items
- Recipe scaling based on limiting ingredients
- Match score prioritization (0.0-1.0)
### 3. Live Cooking Assistant
- Step-by-step guidance with progress tracking
- Real-time visual monitoring of cooking progress
- Text-to-speech announcements for hands-free operation
- AI feedback when steps are complete
- Haptic feedback for completion events
### 4. User Profiles & Preferences
- Dietary restrictions (Vegan, Keto, Gluten-Free, etc.)
- Nutrition goals
- Pantry staples management
- Firebase cloud sync (optional)
## 🏗️ Architecture
### Design Pattern
**MVVM (Model-View-ViewModel) + Repository Pattern**
```
┌─────────────┐
│ Views │ (SwiftUI)
└─────┬───────┘
┌─────▼───────┐
│ ViewModels │ (@MainActor, ObservableObject)
└─────┬───────┘
┌─────▼───────┐
│ Services │ (Protocol-based)
└─────┬───────┘
┌─────▼───────┐
│ APIs/Cloud │ (Overshoot, Gemini, Firebase)
└─────────────┘
```
### Protocol-Oriented Design
**Vision Service:**
```swift
protocol VisionService: Sendable {
func detectIngredients(from: AsyncStream<CVPixelBuffer>) async throws -> [Ingredient]
func analyzeCookingProgress(from: AsyncStream<CVPixelBuffer>, for: String) async throws -> CookingProgress
}
```
**Recipe Service:**
```swift
protocol RecipeService: Sendable {
func generateRecipes(inventory: [Ingredient], profile: UserProfile) async throws -> [Recipe]
func scaleRecipe(_: Recipe, for: Ingredient, quantity: String) async throws -> Recipe
}
```
This design allows easy swapping of AI providers (e.g., OpenAI, Anthropic, etc.) without changing business logic.
## 📁 Complete File Structure
```
SousChefAI/
├── SousChefAI/
│ ├── Config/
│ │ └── AppConfig.swift # API keys and feature flags
│ │
│ ├── Models/
│ │ ├── Ingredient.swift # Ingredient data model
│ │ ├── UserProfile.swift # User preferences and restrictions
│ │ └── Recipe.swift # Recipe with categorization
│ │
│ ├── Services/
│ │ ├── VisionService.swift # Vision protocol definition
│ │ ├── OvershootVisionService.swift # Overshoot implementation
│ │ ├── RecipeService.swift # Recipe protocol definition
│ │ ├── GeminiRecipeService.swift # Gemini implementation
│ │ ├── FirestoreRepository.swift # Firebase data layer
│ │ └── CameraManager.swift # AVFoundation camera handling
│ │
│ ├── ViewModels/
│ │ ├── ScannerViewModel.swift # Scanner business logic
│ │ ├── RecipeGeneratorViewModel.swift # Recipe generation logic
│ │ └── CookingModeViewModel.swift # Cooking guidance logic
│ │
│ ├── Views/
│ │ ├── ScannerView.swift # Camera scanning UI
│ │ ├── InventoryView.swift # Ingredient management UI
│ │ ├── RecipeGeneratorView.swift # Recipe browsing UI
│ │ └── CookingModeView.swift # Step-by-step cooking UI
│ │
│ ├── ContentView.swift # Tab-based navigation
│ ├── SousChefAIApp.swift # App entry point
│ └── Assets.xcassets # App icons and images
├── Documentation/
│ ├── README.md # Complete documentation
│ ├── QUICKSTART.md # 5-minute setup checklist
│ ├── SETUP_GUIDE.md # Detailed setup instructions
│ ├── PRIVACY_SETUP.md # Camera permission guide
│ └── PROJECT_SUMMARY.md # This file
├── PrivacyInfo.xcprivacy # Privacy manifest
└── Tests/
├── SousChefAITests/
└── SousChefAIUITests/
```
## 🛠️ Technology Stack
| Category | Technology | Purpose |
|----------|-----------|---------|
| Language | Swift 6 | Type-safe, concurrent programming |
| UI Framework | SwiftUI | Declarative, modern UI |
| Concurrency | async/await | Native Swift concurrency |
| Camera | AVFoundation | Video capture and processing |
| Vision AI | Overshoot API | Real-time video inference |
| Reasoning AI | Google Gemini 2.0 | Recipe generation and logic |
| Backend | Firebase | Authentication and Firestore |
| Persistence | Firestore | Cloud-synced data storage |
| Architecture | MVVM | Separation of concerns |
## 📊 Code Statistics
- **Total Swift Files**: 17
- **Lines of Code**: ~8,000+
- **Models**: 3 (Ingredient, UserProfile, Recipe)
- **Services**: 6 (protocols + implementations)
- **ViewModels**: 3
- **Views**: 4 main views + supporting components
## 🔑 Configuration Requirements
### Required (for full functionality)
1. **Camera Privacy Description** - App will crash without this
2. **Overshoot API Key** - For ingredient detection
3. **Gemini API Key** - For recipe generation
### Optional
1. **Firebase Configuration** - For cloud sync
2. **Microphone Privacy** - For voice features
## 🚀 Build Status
**Project builds successfully** with Xcode 15.0+
**Swift 6 compliant** with strict concurrency
**iOS 17.0+ compatible**
**No compiler warnings**
## 📱 User Flow
1. **Launch** → Tab bar with 4 sections
2. **Scan Tab** → Point camera at fridge → Detect ingredients
3. **Inventory** → Review & edit items → Set preferences
4. **Generate** → AI creates recipe suggestions
5. **Cook** → Step-by-step with live monitoring
## 🎨 UI Highlights
- **Clean Apple HIG compliance**
- **Material blur overlays** for camera views
- **Confidence indicators** (green/yellow/red)
- **Real-time progress bars**
- **Haptic feedback** for important events
- **Dark mode support** (automatic)
## 🔒 Privacy & Security
- **Privacy Manifest** included (PrivacyInfo.xcprivacy)
- **Camera usage clearly described**
- **No tracking or analytics**
- **API keys marked for replacement** (not committed)
- **Local-first architecture** (works offline for inventory)
## 🧪 Testing Strategy
### Unit Tests
- Model encoding/decoding
- Service protocol conformance
- ViewModel business logic
### UI Tests
- Tab navigation
- Camera permission flow
- Recipe filtering
- Step progression in cooking mode
## 🔄 Future Enhancements
Potential features for future versions:
- [ ] Nutrition tracking and calorie counting
- [ ] Shopping list generation with store integration
- [ ] Social features (recipe sharing)
- [ ] Meal planning calendar
- [ ] Apple Watch companion app
- [ ] Widgets for quick recipe access
- [ ] Offline mode with Core ML models
- [ ] Multi-language support
- [ ] Voice commands during cooking
- [ ] Smart appliance integration
## 📚 Documentation Files
1. **README.md** - Complete feature documentation
2. **QUICKSTART.md** - 5-minute setup checklist
3. **SETUP_GUIDE.md** - Step-by-step configuration
4. **PRIVACY_SETUP.md** - Camera permission details
5. **PROJECT_SUMMARY.md** - Architecture overview (this file)
## 🤝 Contributing
The codebase follows these principles:
1. **Protocol-oriented design** for service abstractions
2. **Async/await** for all asynchronous operations
3. **@MainActor** for UI-related classes
4. **Sendable** conformance for concurrency safety
5. **SwiftUI best practices** with MVVM
6. **Clear separation** between layers
## 📄 License
MIT License - See LICENSE file for details
## 👏 Acknowledgments
- **Overshoot AI** - Low-latency video inference
- **Google Gemini** - Advanced reasoning capabilities
- **Firebase** - Scalable backend infrastructure
- **Apple** - SwiftUI and AVFoundation frameworks
---
**Built with Swift 6 + SwiftUI**
**Production-ready for iOS 17.0+**
Last Updated: February 2026

View File

@@ -1,14 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>NSPrivacyAccessedAPITypes</key>
<array/>
<key>NSPrivacyCollectedDataTypes</key>
<array/>
<key>NSPrivacyTracking</key>
<false/>
<key>NSPrivacyTrackingDomains</key>
<array/>
</dict>
</plist>

View File

@@ -1,139 +0,0 @@
# SousChefAI - Quick Start Checklist ✅
Get up and running in 5 minutes!
## Prerequisites Check
- [ ] macOS 14.0+ with Xcode 15.0+
- [ ] iOS 17.0+ device or simulator
- [ ] Internet connection
## Step-by-Step Setup
### 1⃣ Configure Privacy (CRITICAL - App will crash without this!)
**In Xcode:**
1. Select the **SousChefAI** target
2. Go to **Info** tab
3. Click **+** under "Custom iOS Target Properties"
4. Add:
- Key: `Privacy - Camera Usage Description`
- Value: `SousChefAI needs camera access to scan your fridge for ingredients and monitor your cooking progress in real-time.`
5. Click **+** again and add:
- Key: `Privacy - Microphone Usage Description`
- Value: `SousChefAI uses the microphone to provide voice-guided cooking instructions.`
**Status**: [ ] Privacy descriptions added
### 2⃣ Add API Keys
**File**: `SousChefAI/Config/AppConfig.swift`
Replace:
```swift
static let overshootAPIKey = "INSERT_KEY_HERE"
static let geminiAPIKey = "INSERT_KEY_HERE"
```
With your actual API keys from:
- **Overshoot**: [Your Overshoot Provider]
- **Gemini**: https://makersuite.google.com/app/apikey
**Status**:
- [ ] Overshoot API key added
- [ ] Gemini API key added
### 3⃣ Add Firebase (Optional - for cloud sync)
**Add Package:**
1. File → Add Package Dependencies
2. URL: `https://github.com/firebase/firebase-ios-sdk`
3. Add products: `FirebaseAuth`, `FirebaseFirestore`
**Configure:**
1. Download `GoogleService-Info.plist` from Firebase Console
2. Drag into Xcode (ensure it's added to target)
3. Uncomment in `SousChefAIApp.swift`:
```swift
import FirebaseCore
init() {
FirebaseApp.configure()
}
```
**Status**:
- [ ] Firebase package added
- [ ] GoogleService-Info.plist added
- [ ] Firebase initialized
### 4⃣ Build & Run
1. Open `SousChefAI.xcodeproj`
2. Select target device (iOS 17.0+)
3. Press **⌘ + R**
4. Grant camera permission when prompted
**Status**: [ ] App running successfully
## Minimum Viable Setup (Test Mode)
Want to just see the UI without external services?
**Required:**
- ✅ Privacy descriptions (Step 1)
**Optional:**
- ⚠️ API keys (will show errors but UI works)
- ⚠️ Firebase (uses local data only)
## Verification
After setup, test these features:
- [ ] Scanner tab opens camera
- [ ] Can add manual ingredients
- [ ] Inventory view displays items
- [ ] Profile tab shows configuration status
- [ ] No crash when opening camera
## Common Issues
### ❌ App crashes immediately when opening Scanner
**Fix**: Add camera privacy description (Step 1)
### ❌ "API Key Missing" errors
**Fix**: Replace "INSERT_KEY_HERE" in AppConfig.swift (Step 2)
### ❌ "Module 'Firebase' not found"
**Fix**: Add Firebase package via SPM (Step 3)
### ❌ Camera permission dialog doesn't appear
**Fix**: Delete app, clean build (⌘+Shift+K), rebuild, reinstall
## Next Steps
Once running:
1. **Scan Mode**: Point camera at ingredients → tap "Scan Fridge"
2. **Inventory**: Review detected items → edit quantities → set preferences
3. **Generate Recipes**: Tap "Generate Recipes" → browse suggestions
4. **Cook**: Select recipe → "Start Cooking" → enable AI monitoring
## Documentation
- **Full Guide**: [SETUP_GUIDE.md](SETUP_GUIDE.md)
- **Privacy**: [PRIVACY_SETUP.md](PRIVACY_SETUP.md)
- **Architecture**: [README.md](README.md)
## Support
Issues? Check:
1. Privacy descriptions are added ✓
2. API keys are valid strings (not "INSERT_KEY_HERE") ✓
3. Target is iOS 17.0+ ✓
4. Clean build folder and rebuild ✓
---
**Ready to cook with AI! 🍳**

256
README.md
View File

@@ -1,256 +0,0 @@
Note - this is an AI generated readme, and will be updated in the future.
# SousChefAI
A production-ready iOS app that uses multimodal AI to scan ingredients, generate personalized recipes, and provide real-time cooking guidance.
## Features
### 🎥 Intelligent Fridge Scanner
- Real-time ingredient detection using Overshoot API
- Camera-based scanning with live preview
- Confidence scoring for each detected ingredient
- Manual ingredient entry and editing
### 🍳 AI-Powered Recipe Generation
- Personalized recipe suggestions based on available ingredients
- Google Gemini AI for complex reasoning and recipe creation
- Filtering by "Scavenger" (use only what you have) or "Upgrader" (minimal shopping)
- Recipe scaling based on limiting ingredients
- Match scoring to prioritize best recipes
### 👨‍🍳 Live Cooking Mode
- Step-by-step guided cooking
- Real-time visual monitoring of cooking progress
- Text-to-speech announcements for hands-free cooking
- AI feedback when steps are complete
- Progress tracking and navigation
### 🔐 User Profiles & Persistence
- Firebase Firestore for cloud data sync
- Dietary restrictions (Vegan, Keto, Gluten-Free, etc.)
- Nutrition goals
- Saved recipes and pantry staples
## Architecture
The app follows **MVVM (Model-View-ViewModel)** with a **Repository Pattern** for clean separation of concerns:
```
├── Models/ # Core data models (Codable, Identifiable)
│ ├── Ingredient.swift
│ ├── UserProfile.swift
│ └── Recipe.swift
├── Services/ # Business logic & external APIs
│ ├── VisionService.swift # Protocol for vision AI
│ ├── OvershootVisionService.swift # Overshoot implementation
│ ├── RecipeService.swift # Protocol for recipe generation
│ ├── GeminiRecipeService.swift # Gemini implementation
│ ├── FirestoreRepository.swift # Firebase data layer
│ └── CameraManager.swift # AVFoundation camera handling
├── ViewModels/ # Business logic for views
│ ├── ScannerViewModel.swift
│ ├── RecipeGeneratorViewModel.swift
│ └── CookingModeViewModel.swift
├── Views/ # SwiftUI views
│ ├── ScannerView.swift
│ ├── InventoryView.swift
│ ├── RecipeGeneratorView.swift
│ └── CookingModeView.swift
└── Config/ # App configuration
└── AppConfig.swift
```
## Setup Instructions
### 1. Clone the Repository
```bash
git clone https://github.com/yourusername/souschef.git
cd souschef
```
### 2. Configure API Keys
Open `SousChefAI/Config/AppConfig.swift` and replace the placeholder values:
```swift
// Overshoot Vision API
static let overshootAPIKey = "YOUR_OVERSHOOT_API_KEY"
// Google Gemini API
static let geminiAPIKey = "YOUR_GEMINI_API_KEY"
```
**Getting API Keys:**
- **Overshoot API**: Visit [overshoot.ai](https://overshoot.ai) (or the actual provider URL) and sign up
- **Gemini API**: Visit [Google AI Studio](https://makersuite.google.com/app/apikey) and create an API key
### 3. Add Firebase
#### Add Firebase SDK via Swift Package Manager:
1. In Xcode: `File` > `Add Package Dependencies`
2. Enter URL: `https://github.com/firebase/firebase-ios-sdk`
3. Select version: `10.0.0` or later
4. Add the following products:
- `FirebaseAuth`
- `FirebaseFirestore`
#### Add GoogleService-Info.plist:
1. Go to [Firebase Console](https://console.firebase.google.com/)
2. Create a new project or select existing
3. Add an iOS app with bundle ID: `com.yourcompany.SousChefAI`
4. Download `GoogleService-Info.plist`
5. Drag it into your Xcode project (ensure it's added to the SousChefAI target)
#### Enable Firebase in App:
1. Open `SousChefAI/SousChefAIApp.swift`
2. Uncomment the Firebase imports and initialization:
```swift
import FirebaseCore
init() {
FirebaseApp.configure()
}
```
### 4. Add Google Generative AI SDK (Optional)
For better Gemini integration, add the official SDK:
```swift
// In Xcode: File > Add Package Dependencies
// URL: https://github.com/google/generative-ai-swift
```
Then update `GeminiRecipeService.swift` to use the SDK instead of REST API.
### 5. Configure Camera Permissions
The app requires camera access. Permissions are already handled in code, but ensure your `Info.plist` includes:
```xml
<key>NSCameraUsageDescription</key>
<string>We need camera access to scan your fridge and monitor cooking progress</string>
```
### 6. Build and Run
1. Open `SousChefAI.xcodeproj` in Xcode
2. Select your target device or simulator
3. Press `Cmd + R` to build and run
## Usage Guide
### Scanning Your Fridge
1. Tap the **Scan** tab
2. Point your camera at your fridge or ingredients
3. Tap **Scan Fridge** to start detection
4. Review detected ingredients (yellow = low confidence)
5. Tap **Continue to Inventory**
### Managing Inventory
1. Edit quantities by tapping an ingredient
2. Swipe left to delete items
3. Add manual entries with the `+` button
4. Set dietary preferences before generating recipes
5. Tap **Generate Recipes** when ready
### Generating Recipes
1. Browse suggested recipes sorted by match score
2. Filter by:
- **All Recipes**: Show everything
- **The Scavenger**: Only use what you have
- **The Upgrader**: Need 1-2 items max
- **High Match**: 80%+ ingredient match
3. Tap a recipe to view details
4. Save favorites with the heart icon
5. Start cooking with **Start Cooking** button
### Cooking Mode
1. Enable **AI Monitoring** to watch your cooking
2. The AI will analyze your progress visually
3. Navigate steps with Previous/Next
4. Use **Read Aloud** for hands-free guidance
5. The AI will announce when steps are complete
6. View all steps with the list icon
## Tech Stack
- **Language**: Swift 6
- **UI Framework**: SwiftUI
- **Architecture**: MVVM + Repository Pattern
- **Concurrency**: Swift Async/Await (no completion handlers)
- **Camera**: AVFoundation
- **Vision AI**: Overshoot API (real-time video inference)
- **Reasoning AI**: Google Gemini 2.0 Flash
- **Backend**: Firebase (Auth + Firestore)
- **Persistence**: Firebase Firestore (cloud sync)
## Protocol-Oriented Design
The app uses protocols for AI services to enable easy provider swapping:
```swift
protocol VisionService {
func detectIngredients(from: AsyncStream<CVPixelBuffer>) async throws -> [Ingredient]
}
protocol RecipeService {
func generateRecipes(inventory: [Ingredient], profile: UserProfile) async throws -> [Recipe]
}
```
To swap providers, simply create a new implementation:
```swift
final class OpenAIVisionService: VisionService {
// Implementation using OpenAI Vision API
}
final class AnthropicRecipeService: RecipeService {
// Implementation using Claude API
}
```
## Future Enhancements
- [ ] Nutrition tracking and calorie counting
- [ ] Shopping list generation
- [ ] Recipe sharing and social features
- [ ] Meal planning calendar
- [ ] Voice commands during cooking
- [ ] Multi-language support
- [ ] Apple Watch companion app
- [ ] Widget for quick recipe access
- [ ] Offline mode with local ML models
- [ ] Integration with smart kitchen appliances
## Contributing
Contributions are welcome! Please follow these guidelines:
1. Fork the repository
2. Create a feature branch: `git checkout -b feature/amazing-feature`
3. Follow Swift style guide and existing architecture
4. Write unit tests for new features
5. Update documentation as needed
6. Submit a pull request
## License
This project is licensed under the MIT License - see the LICENSE file for details.
## Acknowledgments
- Overshoot AI for low-latency video inference
- Google Gemini for powerful reasoning capabilities
- Firebase for robust backend infrastructure
- Apple for SwiftUI and AVFoundation frameworks
## Support
For issues, questions, or feature requests, please open an issue on GitHub.
---

View File

@@ -1,203 +0,0 @@
# SousChefAI - Quick Setup Guide
This guide will help you get SousChefAI up and running.
## Prerequisites
- macOS 14.0 or later
- Xcode 15.0 or later
- iOS 17.0+ device or simulator
- Active internet connection for API calls
## Step 1: Configure API Keys
### Overshoot Vision API
1. Visit the Overshoot API provider website and create an account
2. Generate an API key for video inference
3. Open `SousChefAI/Config/AppConfig.swift`
4. Replace `INSERT_KEY_HERE` with your Overshoot API key:
```swift
static let overshootAPIKey = "your_overshoot_api_key_here"
```
### Google Gemini API
1. Visit [Google AI Studio](https://makersuite.google.com/app/apikey)
2. Sign in with your Google account
3. Create a new API key
4. In `SousChefAI/Config/AppConfig.swift`, replace:
```swift
static let geminiAPIKey = "your_gemini_api_key_here"
```
## Step 2: Add Firebase (Optional but Recommended)
### Add Firebase SDK
1. In Xcode, go to `File` > `Add Package Dependencies`
2. Enter: `https://github.com/firebase/firebase-ios-sdk`
3. Select version `10.0.0` or later
4. Add these products to your target:
- FirebaseAuth
- FirebaseFirestore
### Configure Firebase Project
1. Go to [Firebase Console](https://console.firebase.google.com/)
2. Create a new project or select existing
3. Click "Add app" and select iOS
4. Enter bundle identifier: `com.yourcompany.SousChefAI`
5. Download `GoogleService-Info.plist`
6. Drag the file into your Xcode project (ensure it's added to the SousChefAI target)
### Enable Firebase in Code
1. Open `SousChefAI/SousChefAIApp.swift`
2. Uncomment these lines:
```swift
import FirebaseCore
init() {
FirebaseApp.configure()
}
```
### Configure Firestore Database
1. In Firebase Console, go to Firestore Database
2. Click "Create database"
3. Start in test mode (or production mode with proper rules)
4. Choose a location close to your users
## Step 3: Configure Camera Permissions (CRITICAL)
⚠️ **The app will crash without this step!**
### Add Privacy Descriptions in Xcode
1. In Xcode, select the **SousChefAI** target
2. Go to the **Info** tab
3. Under "Custom iOS Target Properties", click the **+** button
4. Add these two keys:
**Camera Permission:**
- **Key**: `Privacy - Camera Usage Description` (or `NSCameraUsageDescription`)
- **Value**: `SousChefAI needs camera access to scan your fridge for ingredients and monitor your cooking progress in real-time.`
**Microphone Permission:**
- **Key**: `Privacy - Microphone Usage Description` (or `NSMicrophoneUsageDescription`)
- **Value**: `SousChefAI uses the microphone to provide voice-guided cooking instructions.`
📖 See [PRIVACY_SETUP.md](PRIVACY_SETUP.md) for detailed step-by-step instructions with screenshots.
## Step 4: Build and Run
1. Open `SousChefAI.xcodeproj` in Xcode
2. Select your target device (iOS 17.0+ required)
3. Press `⌘ + R` to build and run
4. Allow camera permissions when prompted
## Testing Without API Keys
If you want to test the UI without API keys:
1. The app will show placeholder data and errors for API calls
2. You can still navigate through the UI
3. Manual ingredient entry will work
4. Recipe generation will fail gracefully
## Troubleshooting
### Build Errors
**"Missing GoogleService-Info.plist"**
- Ensure the file is in your project and added to the target
- Check that it's not in a subdirectory
**"Module 'Firebase' not found"**
- Make sure you added the Firebase package correctly
- Clean build folder: `⌘ + Shift + K`
- Rebuild: `⌘ + B`
**"API Key Missing" errors**
- Check that you replaced "INSERT_KEY_HERE" in AppConfig.swift
- API keys should be strings without quotes inside the quotes
### Runtime Errors
**"Camera access denied"**
- Go to Settings > Privacy & Security > Camera
- Enable camera access for SousChefAI
**"Network request failed"**
- Check internet connection
- Verify API keys are valid
- Check API endpoint URLs in AppConfig.swift
**"Firebase configuration error"**
- Ensure GoogleService-Info.plist is properly added
- Verify Firebase initialization is uncommented
- Check Firestore is enabled in Firebase Console
## Architecture Overview
The app follows MVVM architecture with clean separation:
```
Views → ViewModels → Services → APIs/Firebase
↓ ↓ ↓
Models ← Repository ← Firestore
```
## Next Steps
Once the app is running:
1. **Test the Scanner**: Point camera at ingredients and scan
2. **Review Inventory**: Edit quantities and add items manually
3. **Set Preferences**: Configure dietary restrictions
4. **Generate Recipes**: Get AI-powered recipe suggestions
5. **Cooking Mode**: Try the live cooking assistant
## Optional Enhancements
### Add Google Generative AI SDK
For better Gemini integration:
1. Add package: `https://github.com/google/generative-ai-swift`
2. Update `GeminiRecipeService.swift` to use the SDK
3. Uncomment the SDK-based code in the service
### Configure Overshoot WebSocket
If using WebSocket for real-time detection:
1. Update `overshootWebSocketURL` in AppConfig.swift
2. Verify the WebSocket endpoint with Overshoot documentation
3. Test real-time detection in Scanner view
## Support
For issues or questions:
- Check the main [README.md](README.md)
- Open an issue on GitHub
- Review the inline documentation in code files
## Security Notes
⚠️ **Important**: Never commit API keys to version control!
Consider:
- Using environment variables for keys
- Adding `AppConfig.swift` to `.gitignore` (but keep a template)
- Using a secrets management service in production
- Rotating keys regularly
---
**You're all set! Happy cooking with SousChefAI! 🍳**

View File

@@ -1,136 +0,0 @@
# Swift 6 Concurrency Warnings - Explained
## Summary
The SousChefAI project builds successfully with **only 4 unavoidable Swift 6 concurrency warnings**. These warnings are related to Core Video framework types that haven't been updated for Swift 6 Sendable conformance yet.
## Remaining Warnings (4 total)
### 1-3. CVPixelBuffer / AsyncStream<CVPixelBuffer> Not Sendable
**Files**: `OvershootVisionService.swift` (lines 36, 79, 88)
**Warning Messages**:
- "Non-Sendable parameter type 'AsyncStream<CVPixelBuffer>' cannot be sent..."
- "Non-Sendable parameter type 'CVPixelBuffer' cannot be sent..."
**Why This Happens**:
- Core Video's `CVPixelBuffer` (aka `CVBuffer`) hasn't been marked as `Sendable` by Apple yet
- This is a framework limitation, not a code issue
**Why It's Safe**:
- `CVPixelBuffer` is **thread-safe** and **immutable** by design
- The underlying C API uses reference counting and atomic operations
- We use `@preconcurrency import CoreVideo` to acknowledge this
- The service is marked `@unchecked Sendable` which tells Swift we've verified thread safety
**Resolution**:
**These warnings are expected and safe to ignore**
- They will be resolved when Apple updates Core Video for Swift 6
- The code is correct and thread-safe
### 4. Configuration Warning
**File**: `SousChefAI.xcodeproj`
**Warning**: "Update to recommended settings"
**Why This Happens**:
- Xcode periodically suggests updating project settings to latest recommendations
**Resolution**:
⚠️ **Optional** - You can update project settings in Xcode:
1. Click on the warning in Issue Navigator
2. Click "Update to Recommended Settings"
3. Review and accept the changes
This won't affect functionality - it just updates build settings to Apple's latest recommendations.
## What We Fixed ✅
During the warning cleanup, we successfully resolved:
1.**CameraManager concurrency issues**
- Added `nonisolated(unsafe)` for AVFoundation types
- Fixed capture session isolation
- Resolved frame continuation thread safety
2.**Service initialization warnings**
- Made service initializers `nonisolated`
- Fixed ViewModel initialization context
3.**FirestoreRepository unused variable warnings**
- Changed `guard let userId = userId` to `guard userId != nil`
- Removed 8 unnecessary variable bindings
4.**Unnecessary await warnings**
- Removed `await` from synchronous function calls
- Fixed in ScannerViewModel and CookingModeViewModel
5.**AppConfig isolation**
- Verified String constants are properly Sendable
## Build Status
- **Build Result**: ✅ **SUCCESS**
- **Error Count**: 0
- **Warning Count**: 4 (all unavoidable Core Video framework issues)
- **Swift 6 Mode**: ✅ Enabled and passing
- **Strict Concurrency**: ✅ Enabled
## Recommendations
### For Development
The current warnings can be safely ignored. The code is production-ready and follows Swift 6 best practices.
### For Production
These warnings do **not** indicate runtime issues:
- CVPixelBuffer is thread-safe
- All actor isolation is properly handled
- Sendable conformance is correctly applied
### Future Updates
These warnings will automatically resolve when:
- Apple updates Core Video to conform to Sendable
- Expected in a future iOS SDK release
## Technical Details
### Why @preconcurrency?
We use `@preconcurrency import CoreVideo` because:
1. Core Video was written before Swift Concurrency
2. Apple hasn't retroactively added Sendable conformance
3. The types are inherently thread-safe but not marked as such
4. This suppresses warnings while maintaining safety
### Why @unchecked Sendable?
`OvershootVisionService` is marked `@unchecked Sendable` because:
1. It uses Core Video types that aren't marked Sendable
2. We've manually verified thread safety
3. All mutable state is properly synchronized
4. URLSession and other types used are thread-safe
## Verification
To verify warnings yourself:
```bash
# Build the project
xcodebuild -scheme SousChefAI build
# Count warnings
xcodebuild -scheme SousChefAI build 2>&1 | grep "warning:" | wc -l
```
Expected result: 4 warnings (all Core Video related)
---
**Status**: ✅ Production Ready
**Swift 6**: ✅ Fully Compatible
**Concurrency**: ✅ Thread-Safe
**Action Required**: None
These warnings are framework limitations, not code issues. The app is safe to deploy.

View File

@@ -1,624 +0,0 @@
// !$*UTF8*$!
{
archiveVersion = 1;
classes = {
};
objectVersion = 77;
objects = {
/* Begin PBXBuildFile section */
865424082F3D142A00B4257E /* README.md in Resources */ = {isa = PBXBuildFile; fileRef = 865424072F3D142A00B4257E /* README.md */; };
8654240A2F3D151800B4257E /* SETUP_GUIDE.md in Resources */ = {isa = PBXBuildFile; fileRef = 865424092F3D151800B4257E /* SETUP_GUIDE.md */; };
8654240E2F3D17FE00B4257E /* PrivacyInfo.xcprivacy in Resources */ = {isa = PBXBuildFile; fileRef = 8654240D2F3D17FE00B4257E /* PrivacyInfo.xcprivacy */; };
865424102F3D181000B4257E /* PRIVACY_SETUP.md in Resources */ = {isa = PBXBuildFile; fileRef = 8654240F2F3D181000B4257E /* PRIVACY_SETUP.md */; };
865424122F3D185100B4257E /* QUICKSTART.md in Resources */ = {isa = PBXBuildFile; fileRef = 865424112F3D185100B4257E /* QUICKSTART.md */; };
865424142F3D188500B4257E /* PROJECT_SUMMARY.md in Resources */ = {isa = PBXBuildFile; fileRef = 865424132F3D188500B4257E /* PROJECT_SUMMARY.md */; };
865424162F3D1A7100B4257E /* SWIFT6_WARNINGS.md in Resources */ = {isa = PBXBuildFile; fileRef = 865424152F3D1A7100B4257E /* SWIFT6_WARNINGS.md */; };
/* End PBXBuildFile section */
/* Begin PBXContainerItemProxy section */
86FE8EEB2F3CF75900A1BEA6 /* PBXContainerItemProxy */ = {
isa = PBXContainerItemProxy;
containerPortal = 86FE8ED52F3CF75800A1BEA6 /* Project object */;
proxyType = 1;
remoteGlobalIDString = 86FE8EDC2F3CF75800A1BEA6;
remoteInfo = SousChefAI;
};
86FE8EF52F3CF75900A1BEA6 /* PBXContainerItemProxy */ = {
isa = PBXContainerItemProxy;
containerPortal = 86FE8ED52F3CF75800A1BEA6 /* Project object */;
proxyType = 1;
remoteGlobalIDString = 86FE8EDC2F3CF75800A1BEA6;
remoteInfo = SousChefAI;
};
/* End PBXContainerItemProxy section */
/* Begin PBXFileReference section */
865424072F3D142A00B4257E /* README.md */ = {isa = PBXFileReference; lastKnownFileType = net.daringfireball.markdown; path = README.md; sourceTree = "<group>"; };
865424092F3D151800B4257E /* SETUP_GUIDE.md */ = {isa = PBXFileReference; lastKnownFileType = net.daringfireball.markdown; path = SETUP_GUIDE.md; sourceTree = "<group>"; };
8654240D2F3D17FE00B4257E /* PrivacyInfo.xcprivacy */ = {isa = PBXFileReference; lastKnownFileType = text.xml; path = PrivacyInfo.xcprivacy; sourceTree = "<group>"; };
8654240F2F3D181000B4257E /* PRIVACY_SETUP.md */ = {isa = PBXFileReference; lastKnownFileType = net.daringfireball.markdown; path = PRIVACY_SETUP.md; sourceTree = "<group>"; };
865424112F3D185100B4257E /* QUICKSTART.md */ = {isa = PBXFileReference; lastKnownFileType = net.daringfireball.markdown; path = QUICKSTART.md; sourceTree = "<group>"; };
865424132F3D188500B4257E /* PROJECT_SUMMARY.md */ = {isa = PBXFileReference; lastKnownFileType = net.daringfireball.markdown; path = PROJECT_SUMMARY.md; sourceTree = "<group>"; };
865424152F3D1A7100B4257E /* SWIFT6_WARNINGS.md */ = {isa = PBXFileReference; lastKnownFileType = net.daringfireball.markdown; path = SWIFT6_WARNINGS.md; sourceTree = "<group>"; };
86FE8EDD2F3CF75800A1BEA6 /* SousChefAI.app */ = {isa = PBXFileReference; explicitFileType = wrapper.application; includeInIndex = 0; path = SousChefAI.app; sourceTree = BUILT_PRODUCTS_DIR; };
86FE8EEA2F3CF75900A1BEA6 /* SousChefAITests.xctest */ = {isa = PBXFileReference; explicitFileType = wrapper.cfbundle; includeInIndex = 0; path = SousChefAITests.xctest; sourceTree = BUILT_PRODUCTS_DIR; };
86FE8EF42F3CF75900A1BEA6 /* SousChefAIUITests.xctest */ = {isa = PBXFileReference; explicitFileType = wrapper.cfbundle; includeInIndex = 0; path = SousChefAIUITests.xctest; sourceTree = BUILT_PRODUCTS_DIR; };
/* End PBXFileReference section */
/* Begin PBXFileSystemSynchronizedRootGroup section */
86FE8EDF2F3CF75800A1BEA6 /* SousChefAI */ = {
isa = PBXFileSystemSynchronizedRootGroup;
path = SousChefAI;
sourceTree = "<group>";
};
86FE8EED2F3CF75900A1BEA6 /* SousChefAITests */ = {
isa = PBXFileSystemSynchronizedRootGroup;
path = SousChefAITests;
sourceTree = "<group>";
};
86FE8EF72F3CF75900A1BEA6 /* SousChefAIUITests */ = {
isa = PBXFileSystemSynchronizedRootGroup;
path = SousChefAIUITests;
sourceTree = "<group>";
};
/* End PBXFileSystemSynchronizedRootGroup section */
/* Begin PBXFrameworksBuildPhase section */
86FE8EDA2F3CF75800A1BEA6 /* Frameworks */ = {
isa = PBXFrameworksBuildPhase;
buildActionMask = 2147483647;
files = (
);
runOnlyForDeploymentPostprocessing = 0;
};
86FE8EE72F3CF75900A1BEA6 /* Frameworks */ = {
isa = PBXFrameworksBuildPhase;
buildActionMask = 2147483647;
files = (
);
runOnlyForDeploymentPostprocessing = 0;
};
86FE8EF12F3CF75900A1BEA6 /* Frameworks */ = {
isa = PBXFrameworksBuildPhase;
buildActionMask = 2147483647;
files = (
);
runOnlyForDeploymentPostprocessing = 0;
};
/* End PBXFrameworksBuildPhase section */
/* Begin PBXGroup section */
86FE8ED42F3CF75800A1BEA6 = {
isa = PBXGroup;
children = (
86FE8EDF2F3CF75800A1BEA6 /* SousChefAI */,
86FE8EED2F3CF75900A1BEA6 /* SousChefAITests */,
86FE8EF72F3CF75900A1BEA6 /* SousChefAIUITests */,
86FE8EDE2F3CF75800A1BEA6 /* Products */,
865424072F3D142A00B4257E /* README.md */,
865424092F3D151800B4257E /* SETUP_GUIDE.md */,
8654240D2F3D17FE00B4257E /* PrivacyInfo.xcprivacy */,
8654240F2F3D181000B4257E /* PRIVACY_SETUP.md */,
865424112F3D185100B4257E /* QUICKSTART.md */,
865424132F3D188500B4257E /* PROJECT_SUMMARY.md */,
865424152F3D1A7100B4257E /* SWIFT6_WARNINGS.md */,
);
sourceTree = "<group>";
};
86FE8EDE2F3CF75800A1BEA6 /* Products */ = {
isa = PBXGroup;
children = (
86FE8EDD2F3CF75800A1BEA6 /* SousChefAI.app */,
86FE8EEA2F3CF75900A1BEA6 /* SousChefAITests.xctest */,
86FE8EF42F3CF75900A1BEA6 /* SousChefAIUITests.xctest */,
);
name = Products;
sourceTree = "<group>";
};
/* End PBXGroup section */
/* Begin PBXNativeTarget section */
86FE8EDC2F3CF75800A1BEA6 /* SousChefAI */ = {
isa = PBXNativeTarget;
buildConfigurationList = 86FE8EFE2F3CF75900A1BEA6 /* Build configuration list for PBXNativeTarget "SousChefAI" */;
buildPhases = (
86FE8ED92F3CF75800A1BEA6 /* Sources */,
86FE8EDA2F3CF75800A1BEA6 /* Frameworks */,
86FE8EDB2F3CF75800A1BEA6 /* Resources */,
);
buildRules = (
);
dependencies = (
);
fileSystemSynchronizedGroups = (
86FE8EDF2F3CF75800A1BEA6 /* SousChefAI */,
);
name = SousChefAI;
packageProductDependencies = (
);
productName = SousChefAI;
productReference = 86FE8EDD2F3CF75800A1BEA6 /* SousChefAI.app */;
productType = "com.apple.product-type.application";
};
86FE8EE92F3CF75900A1BEA6 /* SousChefAITests */ = {
isa = PBXNativeTarget;
buildConfigurationList = 86FE8F012F3CF75900A1BEA6 /* Build configuration list for PBXNativeTarget "SousChefAITests" */;
buildPhases = (
86FE8EE62F3CF75900A1BEA6 /* Sources */,
86FE8EE72F3CF75900A1BEA6 /* Frameworks */,
86FE8EE82F3CF75900A1BEA6 /* Resources */,
);
buildRules = (
);
dependencies = (
86FE8EEC2F3CF75900A1BEA6 /* PBXTargetDependency */,
);
fileSystemSynchronizedGroups = (
86FE8EED2F3CF75900A1BEA6 /* SousChefAITests */,
);
name = SousChefAITests;
packageProductDependencies = (
);
productName = SousChefAITests;
productReference = 86FE8EEA2F3CF75900A1BEA6 /* SousChefAITests.xctest */;
productType = "com.apple.product-type.bundle.unit-test";
};
86FE8EF32F3CF75900A1BEA6 /* SousChefAIUITests */ = {
isa = PBXNativeTarget;
buildConfigurationList = 86FE8F042F3CF75900A1BEA6 /* Build configuration list for PBXNativeTarget "SousChefAIUITests" */;
buildPhases = (
86FE8EF02F3CF75900A1BEA6 /* Sources */,
86FE8EF12F3CF75900A1BEA6 /* Frameworks */,
86FE8EF22F3CF75900A1BEA6 /* Resources */,
);
buildRules = (
);
dependencies = (
86FE8EF62F3CF75900A1BEA6 /* PBXTargetDependency */,
);
fileSystemSynchronizedGroups = (
86FE8EF72F3CF75900A1BEA6 /* SousChefAIUITests */,
);
name = SousChefAIUITests;
packageProductDependencies = (
);
productName = SousChefAIUITests;
productReference = 86FE8EF42F3CF75900A1BEA6 /* SousChefAIUITests.xctest */;
productType = "com.apple.product-type.bundle.ui-testing";
};
/* End PBXNativeTarget section */
/* Begin PBXProject section */
86FE8ED52F3CF75800A1BEA6 /* Project object */ = {
isa = PBXProject;
attributes = {
BuildIndependentTargetsInParallel = 1;
LastSwiftUpdateCheck = 2620;
LastUpgradeCheck = 2620;
TargetAttributes = {
86FE8EDC2F3CF75800A1BEA6 = {
CreatedOnToolsVersion = 26.2;
};
86FE8EE92F3CF75900A1BEA6 = {
CreatedOnToolsVersion = 26.2;
TestTargetID = 86FE8EDC2F3CF75800A1BEA6;
};
86FE8EF32F3CF75900A1BEA6 = {
CreatedOnToolsVersion = 26.2;
TestTargetID = 86FE8EDC2F3CF75800A1BEA6;
};
};
};
buildConfigurationList = 86FE8ED82F3CF75800A1BEA6 /* Build configuration list for PBXProject "SousChefAI" */;
developmentRegion = en;
hasScannedForEncodings = 0;
knownRegions = (
en,
Base,
);
mainGroup = 86FE8ED42F3CF75800A1BEA6;
minimizedProjectReferenceProxies = 1;
preferredProjectObjectVersion = 77;
productRefGroup = 86FE8EDE2F3CF75800A1BEA6 /* Products */;
projectDirPath = "";
projectRoot = "";
targets = (
86FE8EDC2F3CF75800A1BEA6 /* SousChefAI */,
86FE8EE92F3CF75900A1BEA6 /* SousChefAITests */,
86FE8EF32F3CF75900A1BEA6 /* SousChefAIUITests */,
);
};
/* End PBXProject section */
/* Begin PBXResourcesBuildPhase section */
86FE8EDB2F3CF75800A1BEA6 /* Resources */ = {
isa = PBXResourcesBuildPhase;
buildActionMask = 2147483647;
files = (
865424142F3D188500B4257E /* PROJECT_SUMMARY.md in Resources */,
865424082F3D142A00B4257E /* README.md in Resources */,
865424122F3D185100B4257E /* QUICKSTART.md in Resources */,
8654240A2F3D151800B4257E /* SETUP_GUIDE.md in Resources */,
865424162F3D1A7100B4257E /* SWIFT6_WARNINGS.md in Resources */,
865424102F3D181000B4257E /* PRIVACY_SETUP.md in Resources */,
8654240E2F3D17FE00B4257E /* PrivacyInfo.xcprivacy in Resources */,
);
runOnlyForDeploymentPostprocessing = 0;
};
86FE8EE82F3CF75900A1BEA6 /* Resources */ = {
isa = PBXResourcesBuildPhase;
buildActionMask = 2147483647;
files = (
);
runOnlyForDeploymentPostprocessing = 0;
};
86FE8EF22F3CF75900A1BEA6 /* Resources */ = {
isa = PBXResourcesBuildPhase;
buildActionMask = 2147483647;
files = (
);
runOnlyForDeploymentPostprocessing = 0;
};
/* End PBXResourcesBuildPhase section */
/* Begin PBXSourcesBuildPhase section */
86FE8ED92F3CF75800A1BEA6 /* Sources */ = {
isa = PBXSourcesBuildPhase;
buildActionMask = 2147483647;
files = (
);
runOnlyForDeploymentPostprocessing = 0;
};
86FE8EE62F3CF75900A1BEA6 /* Sources */ = {
isa = PBXSourcesBuildPhase;
buildActionMask = 2147483647;
files = (
);
runOnlyForDeploymentPostprocessing = 0;
};
86FE8EF02F3CF75900A1BEA6 /* Sources */ = {
isa = PBXSourcesBuildPhase;
buildActionMask = 2147483647;
files = (
);
runOnlyForDeploymentPostprocessing = 0;
};
/* End PBXSourcesBuildPhase section */
/* Begin PBXTargetDependency section */
86FE8EEC2F3CF75900A1BEA6 /* PBXTargetDependency */ = {
isa = PBXTargetDependency;
target = 86FE8EDC2F3CF75800A1BEA6 /* SousChefAI */;
targetProxy = 86FE8EEB2F3CF75900A1BEA6 /* PBXContainerItemProxy */;
};
86FE8EF62F3CF75900A1BEA6 /* PBXTargetDependency */ = {
isa = PBXTargetDependency;
target = 86FE8EDC2F3CF75800A1BEA6 /* SousChefAI */;
targetProxy = 86FE8EF52F3CF75900A1BEA6 /* PBXContainerItemProxy */;
};
/* End PBXTargetDependency section */
/* Begin XCBuildConfiguration section */
86FE8EFC2F3CF75900A1BEA6 /* Debug */ = {
isa = XCBuildConfiguration;
buildSettings = {
ALWAYS_SEARCH_USER_PATHS = NO;
ASSETCATALOG_COMPILER_GENERATE_SWIFT_ASSET_SYMBOL_EXTENSIONS = YES;
CLANG_ANALYZER_NONNULL = YES;
CLANG_ANALYZER_NUMBER_OBJECT_CONVERSION = YES_AGGRESSIVE;
CLANG_CXX_LANGUAGE_STANDARD = "gnu++20";
CLANG_ENABLE_MODULES = YES;
CLANG_ENABLE_OBJC_ARC = YES;
CLANG_ENABLE_OBJC_WEAK = YES;
CLANG_WARN_BLOCK_CAPTURE_AUTORELEASING = YES;
CLANG_WARN_BOOL_CONVERSION = YES;
CLANG_WARN_COMMA = YES;
CLANG_WARN_CONSTANT_CONVERSION = YES;
CLANG_WARN_DEPRECATED_OBJC_IMPLEMENTATIONS = YES;
CLANG_WARN_DIRECT_OBJC_ISA_USAGE = YES_ERROR;
CLANG_WARN_DOCUMENTATION_COMMENTS = YES;
CLANG_WARN_EMPTY_BODY = YES;
CLANG_WARN_ENUM_CONVERSION = YES;
CLANG_WARN_INFINITE_RECURSION = YES;
CLANG_WARN_INT_CONVERSION = YES;
CLANG_WARN_NON_LITERAL_NULL_CONVERSION = YES;
CLANG_WARN_OBJC_IMPLICIT_RETAIN_SELF = YES;
CLANG_WARN_OBJC_LITERAL_CONVERSION = YES;
CLANG_WARN_OBJC_ROOT_CLASS = YES_ERROR;
CLANG_WARN_QUOTED_INCLUDE_IN_FRAMEWORK_HEADER = YES;
CLANG_WARN_RANGE_LOOP_ANALYSIS = YES;
CLANG_WARN_STRICT_PROTOTYPES = YES;
CLANG_WARN_SUSPICIOUS_MOVE = YES;
CLANG_WARN_UNGUARDED_AVAILABILITY = YES_AGGRESSIVE;
CLANG_WARN_UNREACHABLE_CODE = YES;
CLANG_WARN__DUPLICATE_METHOD_MATCH = YES;
COPY_PHASE_STRIP = NO;
DEBUG_INFORMATION_FORMAT = dwarf;
DEVELOPMENT_TEAM = YK2DB9NT3S;
ENABLE_STRICT_OBJC_MSGSEND = YES;
ENABLE_TESTABILITY = YES;
ENABLE_USER_SCRIPT_SANDBOXING = YES;
GCC_C_LANGUAGE_STANDARD = gnu17;
GCC_DYNAMIC_NO_PIC = NO;
GCC_NO_COMMON_BLOCKS = YES;
GCC_OPTIMIZATION_LEVEL = 0;
GCC_PREPROCESSOR_DEFINITIONS = (
"DEBUG=1",
"$(inherited)",
);
GCC_WARN_64_TO_32_BIT_CONVERSION = YES;
GCC_WARN_ABOUT_RETURN_TYPE = YES_ERROR;
GCC_WARN_UNDECLARED_SELECTOR = YES;
GCC_WARN_UNINITIALIZED_AUTOS = YES_AGGRESSIVE;
GCC_WARN_UNUSED_FUNCTION = YES;
GCC_WARN_UNUSED_VARIABLE = YES;
IPHONEOS_DEPLOYMENT_TARGET = 26.2;
LOCALIZATION_PREFERS_STRING_CATALOGS = YES;
MTL_ENABLE_DEBUG_INFO = INCLUDE_SOURCE;
MTL_FAST_MATH = YES;
ONLY_ACTIVE_ARCH = YES;
SDKROOT = iphoneos;
SWIFT_ACTIVE_COMPILATION_CONDITIONS = "DEBUG $(inherited)";
SWIFT_OPTIMIZATION_LEVEL = "-Onone";
SWIFT_VERSION = 6.0;
};
name = Debug;
};
86FE8EFD2F3CF75900A1BEA6 /* Release */ = {
isa = XCBuildConfiguration;
buildSettings = {
ALWAYS_SEARCH_USER_PATHS = NO;
ASSETCATALOG_COMPILER_GENERATE_SWIFT_ASSET_SYMBOL_EXTENSIONS = YES;
CLANG_ANALYZER_NONNULL = YES;
CLANG_ANALYZER_NUMBER_OBJECT_CONVERSION = YES_AGGRESSIVE;
CLANG_CXX_LANGUAGE_STANDARD = "gnu++20";
CLANG_ENABLE_MODULES = YES;
CLANG_ENABLE_OBJC_ARC = YES;
CLANG_ENABLE_OBJC_WEAK = YES;
CLANG_WARN_BLOCK_CAPTURE_AUTORELEASING = YES;
CLANG_WARN_BOOL_CONVERSION = YES;
CLANG_WARN_COMMA = YES;
CLANG_WARN_CONSTANT_CONVERSION = YES;
CLANG_WARN_DEPRECATED_OBJC_IMPLEMENTATIONS = YES;
CLANG_WARN_DIRECT_OBJC_ISA_USAGE = YES_ERROR;
CLANG_WARN_DOCUMENTATION_COMMENTS = YES;
CLANG_WARN_EMPTY_BODY = YES;
CLANG_WARN_ENUM_CONVERSION = YES;
CLANG_WARN_INFINITE_RECURSION = YES;
CLANG_WARN_INT_CONVERSION = YES;
CLANG_WARN_NON_LITERAL_NULL_CONVERSION = YES;
CLANG_WARN_OBJC_IMPLICIT_RETAIN_SELF = YES;
CLANG_WARN_OBJC_LITERAL_CONVERSION = YES;
CLANG_WARN_OBJC_ROOT_CLASS = YES_ERROR;
CLANG_WARN_QUOTED_INCLUDE_IN_FRAMEWORK_HEADER = YES;
CLANG_WARN_RANGE_LOOP_ANALYSIS = YES;
CLANG_WARN_STRICT_PROTOTYPES = YES;
CLANG_WARN_SUSPICIOUS_MOVE = YES;
CLANG_WARN_UNGUARDED_AVAILABILITY = YES_AGGRESSIVE;
CLANG_WARN_UNREACHABLE_CODE = YES;
CLANG_WARN__DUPLICATE_METHOD_MATCH = YES;
COPY_PHASE_STRIP = NO;
DEBUG_INFORMATION_FORMAT = "dwarf-with-dsym";
DEVELOPMENT_TEAM = YK2DB9NT3S;
ENABLE_NS_ASSERTIONS = NO;
ENABLE_STRICT_OBJC_MSGSEND = YES;
ENABLE_USER_SCRIPT_SANDBOXING = YES;
GCC_C_LANGUAGE_STANDARD = gnu17;
GCC_NO_COMMON_BLOCKS = YES;
GCC_WARN_64_TO_32_BIT_CONVERSION = YES;
GCC_WARN_ABOUT_RETURN_TYPE = YES_ERROR;
GCC_WARN_UNDECLARED_SELECTOR = YES;
GCC_WARN_UNINITIALIZED_AUTOS = YES_AGGRESSIVE;
GCC_WARN_UNUSED_FUNCTION = YES;
GCC_WARN_UNUSED_VARIABLE = YES;
IPHONEOS_DEPLOYMENT_TARGET = 26.2;
LOCALIZATION_PREFERS_STRING_CATALOGS = YES;
MTL_ENABLE_DEBUG_INFO = NO;
MTL_FAST_MATH = YES;
SDKROOT = iphoneos;
SWIFT_COMPILATION_MODE = wholemodule;
SWIFT_VERSION = 6.0;
VALIDATE_PRODUCT = YES;
};
name = Release;
};
86FE8EFF2F3CF75900A1BEA6 /* Debug */ = {
isa = XCBuildConfiguration;
buildSettings = {
ASSETCATALOG_COMPILER_APPICON_NAME = AppIcon;
ASSETCATALOG_COMPILER_GLOBAL_ACCENT_COLOR_NAME = AccentColor;
CODE_SIGN_STYLE = Automatic;
CURRENT_PROJECT_VERSION = 1;
DEVELOPMENT_TEAM = YK2DB9NT3S;
ENABLE_PREVIEWS = YES;
GENERATE_INFOPLIST_FILE = YES;
INFOPLIST_FILE = "";
INFOPLIST_KEY_LSApplicationCategoryType = "";
INFOPLIST_KEY_NSCameraUsageDescription = "SousChefAI needs camera access to scan your fridge for ingredients and monitor your cooking progress in real-time.";
INFOPLIST_KEY_NSMicrophoneUsageDescription = "SousChefAI uses the microphone to provide voice-guided cooking instructions.";
INFOPLIST_KEY_UIApplicationSceneManifest_Generation = YES;
INFOPLIST_KEY_UIApplicationSupportsIndirectInputEvents = YES;
INFOPLIST_KEY_UILaunchScreen_Generation = YES;
INFOPLIST_KEY_UISupportedInterfaceOrientations_iPad = "UIInterfaceOrientationPortrait UIInterfaceOrientationPortraitUpsideDown UIInterfaceOrientationLandscapeLeft UIInterfaceOrientationLandscapeRight";
INFOPLIST_KEY_UISupportedInterfaceOrientations_iPhone = "UIInterfaceOrientationPortrait UIInterfaceOrientationLandscapeLeft UIInterfaceOrientationLandscapeRight";
LD_RUNPATH_SEARCH_PATHS = (
"$(inherited)",
"@executable_path/Frameworks",
);
MARKETING_VERSION = 1.0;
PRODUCT_BUNDLE_IDENTIFIER = com.adipu.SousChefAI;
PRODUCT_NAME = "$(TARGET_NAME)";
STRING_CATALOG_GENERATE_SYMBOLS = YES;
SWIFT_APPROACHABLE_CONCURRENCY = YES;
SWIFT_DEFAULT_ACTOR_ISOLATION = MainActor;
SWIFT_EMIT_LOC_STRINGS = YES;
SWIFT_UPCOMING_FEATURE_MEMBER_IMPORT_VISIBILITY = YES;
SWIFT_VERSION = 5.0;
TARGETED_DEVICE_FAMILY = "1,2";
};
name = Debug;
};
86FE8F002F3CF75900A1BEA6 /* Release */ = {
isa = XCBuildConfiguration;
buildSettings = {
ASSETCATALOG_COMPILER_APPICON_NAME = AppIcon;
ASSETCATALOG_COMPILER_GLOBAL_ACCENT_COLOR_NAME = AccentColor;
CODE_SIGN_STYLE = Automatic;
CURRENT_PROJECT_VERSION = 1;
DEVELOPMENT_TEAM = YK2DB9NT3S;
ENABLE_PREVIEWS = YES;
GENERATE_INFOPLIST_FILE = YES;
INFOPLIST_FILE = "";
INFOPLIST_KEY_LSApplicationCategoryType = "";
INFOPLIST_KEY_NSCameraUsageDescription = "SousChefAI needs camera access to scan your fridge for ingredients and monitor your cooking progress in real-time.";
INFOPLIST_KEY_NSMicrophoneUsageDescription = "SousChefAI uses the microphone to provide voice-guided cooking instructions.";
INFOPLIST_KEY_UIApplicationSceneManifest_Generation = YES;
INFOPLIST_KEY_UIApplicationSupportsIndirectInputEvents = YES;
INFOPLIST_KEY_UILaunchScreen_Generation = YES;
INFOPLIST_KEY_UISupportedInterfaceOrientations_iPad = "UIInterfaceOrientationPortrait UIInterfaceOrientationPortraitUpsideDown UIInterfaceOrientationLandscapeLeft UIInterfaceOrientationLandscapeRight";
INFOPLIST_KEY_UISupportedInterfaceOrientations_iPhone = "UIInterfaceOrientationPortrait UIInterfaceOrientationLandscapeLeft UIInterfaceOrientationLandscapeRight";
LD_RUNPATH_SEARCH_PATHS = (
"$(inherited)",
"@executable_path/Frameworks",
);
MARKETING_VERSION = 1.0;
PRODUCT_BUNDLE_IDENTIFIER = com.adipu.SousChefAI;
PRODUCT_NAME = "$(TARGET_NAME)";
STRING_CATALOG_GENERATE_SYMBOLS = YES;
SWIFT_APPROACHABLE_CONCURRENCY = YES;
SWIFT_DEFAULT_ACTOR_ISOLATION = MainActor;
SWIFT_EMIT_LOC_STRINGS = YES;
SWIFT_UPCOMING_FEATURE_MEMBER_IMPORT_VISIBILITY = YES;
SWIFT_VERSION = 5.0;
TARGETED_DEVICE_FAMILY = "1,2";
};
name = Release;
};
86FE8F022F3CF75900A1BEA6 /* Debug */ = {
isa = XCBuildConfiguration;
buildSettings = {
BUNDLE_LOADER = "$(TEST_HOST)";
CODE_SIGN_STYLE = Automatic;
CURRENT_PROJECT_VERSION = 1;
DEVELOPMENT_TEAM = YK2DB9NT3S;
GENERATE_INFOPLIST_FILE = YES;
IPHONEOS_DEPLOYMENT_TARGET = 26.2;
MARKETING_VERSION = 1.0;
PRODUCT_BUNDLE_IDENTIFIER = com.adipu.SousChefAITests;
PRODUCT_NAME = "$(TARGET_NAME)";
STRING_CATALOG_GENERATE_SYMBOLS = NO;
SWIFT_APPROACHABLE_CONCURRENCY = YES;
SWIFT_EMIT_LOC_STRINGS = NO;
SWIFT_UPCOMING_FEATURE_MEMBER_IMPORT_VISIBILITY = YES;
SWIFT_VERSION = 5.0;
TARGETED_DEVICE_FAMILY = "1,2";
TEST_HOST = "$(BUILT_PRODUCTS_DIR)/SousChefAI.app/$(BUNDLE_EXECUTABLE_FOLDER_PATH)/SousChefAI";
};
name = Debug;
};
86FE8F032F3CF75900A1BEA6 /* Release */ = {
isa = XCBuildConfiguration;
buildSettings = {
BUNDLE_LOADER = "$(TEST_HOST)";
CODE_SIGN_STYLE = Automatic;
CURRENT_PROJECT_VERSION = 1;
DEVELOPMENT_TEAM = YK2DB9NT3S;
GENERATE_INFOPLIST_FILE = YES;
IPHONEOS_DEPLOYMENT_TARGET = 26.2;
MARKETING_VERSION = 1.0;
PRODUCT_BUNDLE_IDENTIFIER = com.adipu.SousChefAITests;
PRODUCT_NAME = "$(TARGET_NAME)";
STRING_CATALOG_GENERATE_SYMBOLS = NO;
SWIFT_APPROACHABLE_CONCURRENCY = YES;
SWIFT_EMIT_LOC_STRINGS = NO;
SWIFT_UPCOMING_FEATURE_MEMBER_IMPORT_VISIBILITY = YES;
SWIFT_VERSION = 5.0;
TARGETED_DEVICE_FAMILY = "1,2";
TEST_HOST = "$(BUILT_PRODUCTS_DIR)/SousChefAI.app/$(BUNDLE_EXECUTABLE_FOLDER_PATH)/SousChefAI";
};
name = Release;
};
86FE8F052F3CF75900A1BEA6 /* Debug */ = {
isa = XCBuildConfiguration;
buildSettings = {
CODE_SIGN_STYLE = Automatic;
CURRENT_PROJECT_VERSION = 1;
DEVELOPMENT_TEAM = YK2DB9NT3S;
GENERATE_INFOPLIST_FILE = YES;
MARKETING_VERSION = 1.0;
PRODUCT_BUNDLE_IDENTIFIER = com.adipu.SousChefAIUITests;
PRODUCT_NAME = "$(TARGET_NAME)";
STRING_CATALOG_GENERATE_SYMBOLS = NO;
SWIFT_APPROACHABLE_CONCURRENCY = YES;
SWIFT_EMIT_LOC_STRINGS = NO;
SWIFT_UPCOMING_FEATURE_MEMBER_IMPORT_VISIBILITY = YES;
SWIFT_VERSION = 5.0;
TARGETED_DEVICE_FAMILY = "1,2";
TEST_TARGET_NAME = SousChefAI;
};
name = Debug;
};
86FE8F062F3CF75900A1BEA6 /* Release */ = {
isa = XCBuildConfiguration;
buildSettings = {
CODE_SIGN_STYLE = Automatic;
CURRENT_PROJECT_VERSION = 1;
DEVELOPMENT_TEAM = YK2DB9NT3S;
GENERATE_INFOPLIST_FILE = YES;
MARKETING_VERSION = 1.0;
PRODUCT_BUNDLE_IDENTIFIER = com.adipu.SousChefAIUITests;
PRODUCT_NAME = "$(TARGET_NAME)";
STRING_CATALOG_GENERATE_SYMBOLS = NO;
SWIFT_APPROACHABLE_CONCURRENCY = YES;
SWIFT_EMIT_LOC_STRINGS = NO;
SWIFT_UPCOMING_FEATURE_MEMBER_IMPORT_VISIBILITY = YES;
SWIFT_VERSION = 5.0;
TARGETED_DEVICE_FAMILY = "1,2";
TEST_TARGET_NAME = SousChefAI;
};
name = Release;
};
/* End XCBuildConfiguration section */
/* Begin XCConfigurationList section */
86FE8ED82F3CF75800A1BEA6 /* Build configuration list for PBXProject "SousChefAI" */ = {
isa = XCConfigurationList;
buildConfigurations = (
86FE8EFC2F3CF75900A1BEA6 /* Debug */,
86FE8EFD2F3CF75900A1BEA6 /* Release */,
);
defaultConfigurationIsVisible = 0;
defaultConfigurationName = Release;
};
86FE8EFE2F3CF75900A1BEA6 /* Build configuration list for PBXNativeTarget "SousChefAI" */ = {
isa = XCConfigurationList;
buildConfigurations = (
86FE8EFF2F3CF75900A1BEA6 /* Debug */,
86FE8F002F3CF75900A1BEA6 /* Release */,
);
defaultConfigurationIsVisible = 0;
defaultConfigurationName = Release;
};
86FE8F012F3CF75900A1BEA6 /* Build configuration list for PBXNativeTarget "SousChefAITests" */ = {
isa = XCConfigurationList;
buildConfigurations = (
86FE8F022F3CF75900A1BEA6 /* Debug */,
86FE8F032F3CF75900A1BEA6 /* Release */,
);
defaultConfigurationIsVisible = 0;
defaultConfigurationName = Release;
};
86FE8F042F3CF75900A1BEA6 /* Build configuration list for PBXNativeTarget "SousChefAIUITests" */ = {
isa = XCConfigurationList;
buildConfigurations = (
86FE8F052F3CF75900A1BEA6 /* Debug */,
86FE8F062F3CF75900A1BEA6 /* Release */,
);
defaultConfigurationIsVisible = 0;
defaultConfigurationName = Release;
};
/* End XCConfigurationList section */
};
rootObject = 86FE8ED52F3CF75800A1BEA6 /* Project object */;
}

View File

@@ -1,7 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<Workspace
version = "1.0">
<FileRef
location = "self:">
</FileRef>
</Workspace>

View File

@@ -1,14 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>SchemeUserState</key>
<dict>
<key>SousChefAI.xcscheme_^#shared#^_</key>
<dict>
<key>orderHint</key>
<integer>0</integer>
</dict>
</dict>
</dict>
</plist>

View File

@@ -1,11 +0,0 @@
{
"colors" : [
{
"idiom" : "universal"
}
],
"info" : {
"author" : "xcode",
"version" : 1
}
}

View File

@@ -1,35 +0,0 @@
{
"images" : [
{
"idiom" : "universal",
"platform" : "ios",
"size" : "1024x1024"
},
{
"appearances" : [
{
"appearance" : "luminosity",
"value" : "dark"
}
],
"idiom" : "universal",
"platform" : "ios",
"size" : "1024x1024"
},
{
"appearances" : [
{
"appearance" : "luminosity",
"value" : "tinted"
}
],
"idiom" : "universal",
"platform" : "ios",
"size" : "1024x1024"
}
],
"info" : {
"author" : "xcode",
"version" : 1
}
}

View File

@@ -1,6 +0,0 @@
{
"info" : {
"author" : "xcode",
"version" : 1
}
}

View File

@@ -1,50 +0,0 @@
//
// AppConfig.swift
// SousChefAI
//
// Centralized configuration for API keys and service endpoints
//
import Foundation
/// Scanning mode for ingredient detection
enum ScanningMode: String, CaseIterable {
case geminiVision // Uses Gemini API for image analysis (recommended)
case arKit // Uses ARKit for spatial scanning (future implementation)
}
enum AppConfig: Sendable {
// MARK: - Google Gemini API
/// Google Gemini API key for recipe generation and reasoning
/// [INSERT_GEMINI_API_KEY_HERE]
static let geminiAPIKey = "INSERT_KEY_HERE"
// MARK: - Firebase Configuration
/// Firebase configuration will be loaded from GoogleService-Info.plist
/// [INSERT_FIREBASE_GOOGLESERVICE-INFO.PLIST_SETUP_HERE]
/// Instructions:
/// 1. Download GoogleService-Info.plist from Firebase Console
/// 2. Add it to the Xcode project root
/// 3. Ensure it's added to the target
// MARK: - Scanning Configuration
/// Current scanning mode - change this to switch between vision implementations
/// Options: .geminiVision (uses Gemini API), .arKit (uses ARKit - future)
static let scanningMode: ScanningMode = .geminiVision
/// Enable AR-based scanning features (legacy flag, use scanningMode instead)
static let enableARScanning = false
// MARK: - Feature Flags
static let enableRealTimeDetection = true
static let enableCookingMode = true
static let maxIngredientsPerScan = 50
static let minConfidenceThreshold = 0.5
// MARK: - Scanning Settings
/// How often to send frames to Gemini (in seconds)
static let geminiFrameInterval: Double = 1.0
/// Maximum scan duration before auto-stop (in seconds)
static let maxScanDuration: Double = 60.0
}

View File

@@ -1,242 +0,0 @@
//
// ContentView.swift
// SousChefAI
//
// Created by Aditya Pulipaka on 2/11/26.
//
import SwiftUI
struct ContentView: View {
@EnvironmentObject var repository: FirestoreRepository
@State private var selectedTab = 0
var body: some View {
TabView(selection: $selectedTab) {
// Scanner Tab
ScannerView()
.tabItem {
Label("Scan", systemImage: "camera.fill")
}
.tag(0)
// Inventory Tab
NavigationStack {
inventoryPlaceholder
}
.tabItem {
Label("Inventory", systemImage: "square.grid.2x2")
}
.tag(1)
// Saved Recipes Tab
NavigationStack {
savedRecipesPlaceholder
}
.tabItem {
Label("Recipes", systemImage: "book.fill")
}
.tag(2)
// Profile Tab
NavigationStack {
profileView
}
.tabItem {
Label("Profile", systemImage: "person.fill")
}
.tag(3)
}
}
// MARK: - Placeholder Views
private var inventoryPlaceholder: some View {
List {
if repository.currentInventory.isEmpty {
ContentUnavailableView(
"No Ingredients",
systemImage: "refrigerator",
description: Text("Scan your fridge to get started")
)
} else {
ForEach(repository.currentInventory) { ingredient in
HStack {
VStack(alignment: .leading) {
Text(ingredient.name)
.font(.headline)
Text(ingredient.estimatedQuantity)
.font(.caption)
.foregroundStyle(.secondary)
}
Spacer()
Text("\(Int(ingredient.confidence * 100))%")
.font(.caption)
.foregroundStyle(.secondary)
}
}
}
}
.navigationTitle("My Inventory")
.toolbar {
ToolbarItem(placement: .primaryAction) {
Button {
selectedTab = 0
} label: {
Label("Scan", systemImage: "camera")
}
}
}
}
private var savedRecipesPlaceholder: some View {
List {
if repository.savedRecipes.isEmpty {
ContentUnavailableView(
"No Saved Recipes",
systemImage: "book",
description: Text("Save recipes from the recipe generator")
)
} else {
ForEach(repository.savedRecipes) { recipe in
VStack(alignment: .leading, spacing: 8) {
Text(recipe.title)
.font(.headline)
Text(recipe.description)
.font(.caption)
.foregroundStyle(.secondary)
.lineLimit(2)
}
.padding(.vertical, 4)
}
}
}
.navigationTitle("Saved Recipes")
}
private var profileView: some View {
Form {
Section("About") {
HStack {
Text("Version")
Spacer()
Text("1.0.0")
.foregroundStyle(.secondary)
}
}
Section("Preferences") {
NavigationLink {
dietaryPreferencesView
} label: {
HStack {
Label("Dietary Restrictions", systemImage: "leaf")
Spacer()
if let profile = repository.currentUser,
!profile.dietaryRestrictions.isEmpty {
Text("\(profile.dietaryRestrictions.count)")
.foregroundStyle(.secondary)
}
}
}
NavigationLink {
nutritionGoalsView
} label: {
Label("Nutrition Goals", systemImage: "heart")
}
}
Section("API Configuration") {
VStack(alignment: .leading, spacing: 8) {
Text("AR Scanning")
.font(.headline)
Text(AppConfig.enableARScanning ? "Enabled" : "Disabled")
.font(.caption)
.foregroundStyle(AppConfig.enableARScanning ? .green : .red)
}
VStack(alignment: .leading, spacing: 8) {
Text("Gemini API")
.font(.headline)
Text(AppConfig.geminiAPIKey == "INSERT_KEY_HERE" ? "Not configured" : "Configured")
.font(.caption)
.foregroundStyle(AppConfig.geminiAPIKey == "INSERT_KEY_HERE" ? .red : .green)
}
}
Section {
Link(destination: URL(string: "https://github.com/yourusername/souschef")!) {
Label("View on GitHub", systemImage: "link")
}
}
}
.navigationTitle("Profile")
}
private var dietaryPreferencesView: some View {
Form {
Section {
ForEach(UserProfile.commonRestrictions, id: \.self) { restriction in
HStack {
Text(restriction)
Spacer()
if repository.currentUser?.dietaryRestrictions.contains(restriction) ?? false {
Image(systemName: "checkmark")
.foregroundStyle(.blue)
}
}
.contentShape(Rectangle())
.onTapGesture {
toggleRestriction(restriction)
}
}
}
}
.navigationTitle("Dietary Restrictions")
}
private var nutritionGoalsView: some View {
Form {
Section {
TextField("Enter your nutrition goals",
text: Binding(
get: { repository.currentUser?.nutritionGoals ?? "" },
set: { newValue in
Task {
try? await repository.updateNutritionGoals(newValue)
}
}
),
axis: .vertical)
.lineLimit(5...10)
} header: {
Text("Goals")
} footer: {
Text("e.g., High protein, Low carb, Balanced diet")
}
}
.navigationTitle("Nutrition Goals")
}
// MARK: - Actions
private func toggleRestriction(_ restriction: String) {
Task {
guard var profile = repository.currentUser else { return }
if profile.dietaryRestrictions.contains(restriction) {
profile.dietaryRestrictions.removeAll { $0 == restriction }
} else {
profile.dietaryRestrictions.append(restriction)
}
try? await repository.updateDietaryRestrictions(profile.dietaryRestrictions)
}
}
}
#Preview {
ContentView()
.environmentObject(FirestoreRepository())
}

View File

@@ -1,47 +0,0 @@
//
// Ingredient.swift
// SousChefAI
//
// Core data model for ingredients detected or managed by the user
//
import Foundation
/// Represents an alternative guess for what an ingredient might be
struct IngredientGuess: Identifiable, Codable, Equatable {
var id: String { name }
let name: String
let confidence: Double
}
struct Ingredient: Identifiable, Codable, Equatable {
let id: String
var name: String
var estimatedQuantity: String
var confidence: Double
/// Top 3 guesses for what this ingredient might be (from AI detection)
var guesses: [IngredientGuess]
init(id: String = UUID().uuidString,
name: String,
estimatedQuantity: String,
confidence: Double = 1.0,
guesses: [IngredientGuess] = []) {
self.id = id
self.name = name
self.estimatedQuantity = estimatedQuantity
self.confidence = confidence
self.guesses = guesses
}
/// Indicates if the detection confidence is low and requires user verification
var needsVerification: Bool {
confidence < 0.7
}
/// Returns the best guess name, or the current name if no guesses available
var bestGuessName: String {
guesses.first?.name ?? name
}
}

View File

@@ -1,70 +0,0 @@
//
// Recipe.swift
// SousChefAI
//
// Recipe model generated by AI based on available ingredients
//
import Foundation
struct Recipe: Identifiable, Codable {
let id: String
var title: String
var description: String
var missingIngredients: [Ingredient]
var steps: [String]
var matchScore: Double
var estimatedTime: String?
var servings: Int?
init(id: String = UUID().uuidString,
title: String,
description: String,
missingIngredients: [Ingredient] = [],
steps: [String],
matchScore: Double,
estimatedTime: String? = nil,
servings: Int? = nil) {
self.id = id
self.title = title
self.description = description
self.missingIngredients = missingIngredients
self.steps = steps
self.matchScore = matchScore
self.estimatedTime = estimatedTime
self.servings = servings
}
/// Indicates if recipe can be made with only available ingredients
var canMakeNow: Bool {
missingIngredients.isEmpty
}
/// Category based on missing ingredients
var category: RecipeCategory {
if canMakeNow {
return .scavenger
} else if missingIngredients.count <= 2 {
return .upgrader
} else {
return .shopping
}
}
}
enum RecipeCategory: String, CaseIterable {
case scavenger = "The Scavenger"
case upgrader = "The Upgrader"
case shopping = "Shopping Required"
var description: String {
switch self {
case .scavenger:
return "Uses only your current ingredients"
case .upgrader:
return "Needs 1-2 additional items"
case .shopping:
return "Requires shopping trip"
}
}
}

View File

@@ -1,37 +0,0 @@
//
// UserProfile.swift
// SousChefAI
//
// User profile model for dietary preferences and pantry staples
//
import Foundation
struct UserProfile: Identifiable, Codable {
let id: String
var dietaryRestrictions: [String]
var nutritionGoals: String
var pantryStaples: [Ingredient]
init(id: String = UUID().uuidString,
dietaryRestrictions: [String] = [],
nutritionGoals: String = "",
pantryStaples: [Ingredient] = []) {
self.id = id
self.dietaryRestrictions = dietaryRestrictions
self.nutritionGoals = nutritionGoals
self.pantryStaples = pantryStaples
}
/// Common dietary restrictions for quick selection
static let commonRestrictions = [
"Vegan",
"Vegetarian",
"Gluten-Free",
"Dairy-Free",
"Keto",
"Paleo",
"Nut Allergy",
"Shellfish Allergy"
]
}

View File

@@ -1,222 +0,0 @@
//
// ARVisionService.swift
// SousChefAI
//
// AR-based vision service using RealityKit and ARKit
// Provides real-time plane detection and raycasting capabilities
//
import Foundation
import SwiftUI
import RealityKit
import ARKit
@preconcurrency import CoreVideo
/// AR-based implementation for vision and spatial scanning
final class ARVisionService: VisionService, @unchecked Sendable {
nonisolated init() {}
// MARK: - VisionService Protocol Implementation
nonisolated func detectIngredients(from stream: AsyncStream<CVPixelBuffer>) async throws -> [Ingredient] {
// Mock implementation - in a real app, this would use ML models
// to detect ingredients from AR camera frames
var detectedIngredients: [Ingredient] = []
var frameCount = 0
for await pixelBuffer in stream {
frameCount += 1
// Process every 30th frame to reduce processing load
if frameCount % 30 == 0 {
let ingredients = try await processARFrame(pixelBuffer)
// Merge results
for ingredient in ingredients {
if !detectedIngredients.contains(where: { $0.name == ingredient.name }) {
detectedIngredients.append(ingredient)
}
}
// Stop after collecting enough ingredients
if detectedIngredients.count >= AppConfig.maxIngredientsPerScan {
break
}
}
}
return detectedIngredients
.filter { $0.confidence >= AppConfig.minConfidenceThreshold }
.sorted { $0.confidence > $1.confidence }
}
nonisolated func detectIngredients(from pixelBuffer: CVPixelBuffer) async throws -> [Ingredient] {
return try await processARFrame(pixelBuffer)
}
nonisolated func analyzeCookingProgress(from stream: AsyncStream<CVPixelBuffer>, for step: String) async throws -> CookingProgress {
// Mock implementation for cooking progress monitoring
return CookingProgress(
isComplete: false,
confidence: 0.5,
feedback: "Monitoring cooking progress..."
)
}
// MARK: - Private Helper Methods
nonisolated private func processARFrame(_ pixelBuffer: CVPixelBuffer) async throws -> [Ingredient] {
// Mock ingredient detection
// In a real implementation, this would use Vision framework or ML models
// to detect objects in the AR camera feed
// For now, return empty array - actual detection would happen here
return []
}
}
/// SwiftUI wrapper for ARView with plane detection and raycasting
struct ARViewContainer: UIViewRepresentable {
@Binding var detectedPlanes: Int
@Binding var lastRaycastResult: String
func makeUIView(context: Context) -> ARView {
let arView = ARView(frame: .zero)
// Configure AR session
let configuration = ARWorldTrackingConfiguration()
// Enable plane detection for horizontal and vertical surfaces
configuration.planeDetection = [.horizontal, .vertical]
// Enable scene reconstruction for better spatial understanding
if ARWorldTrackingConfiguration.supportsSceneReconstruction(.mesh) {
configuration.sceneReconstruction = .mesh
}
// Enable debug options to visualize detected planes
arView.debugOptions = [.showSceneUnderstanding, .showWorldOrigin]
// Set the coordinator as the session delegate
arView.session.delegate = context.coordinator
// Run the AR session
arView.session.run(configuration)
// Add tap gesture for raycasting
let tapGesture = UITapGestureRecognizer(target: context.coordinator, action: #selector(Coordinator.handleTap(_:)))
arView.addGestureRecognizer(tapGesture)
context.coordinator.arView = arView
return arView
}
func updateUIView(_ uiView: ARView, context: Context) {
// Update UI if needed
}
func makeCoordinator() -> Coordinator {
Coordinator(detectedPlanes: $detectedPlanes, lastRaycastResult: $lastRaycastResult)
}
// MARK: - Coordinator
class Coordinator: NSObject, ARSessionDelegate {
@Binding var detectedPlanes: Int
@Binding var lastRaycastResult: String
weak var arView: ARView?
private var detectedPlaneAnchors: Set<UUID> = []
init(detectedPlanes: Binding<Int>, lastRaycastResult: Binding<String>) {
_detectedPlanes = detectedPlanes
_lastRaycastResult = lastRaycastResult
}
// MARK: - ARSessionDelegate Methods
func session(_ session: ARSession, didAdd anchors: [ARAnchor]) {
for anchor in anchors {
if let planeAnchor = anchor as? ARPlaneAnchor {
detectedPlaneAnchors.insert(planeAnchor.identifier)
DispatchQueue.main.async {
self.detectedPlanes = self.detectedPlaneAnchors.count
}
}
}
}
func session(_ session: ARSession, didUpdate anchors: [ARAnchor]) {
// Planes are being updated as AR refines understanding
}
func session(_ session: ARSession, didRemove anchors: [ARAnchor]) {
for anchor in anchors {
if let planeAnchor = anchor as? ARPlaneAnchor {
detectedPlaneAnchors.remove(planeAnchor.identifier)
DispatchQueue.main.async {
self.detectedPlanes = self.detectedPlaneAnchors.count
}
}
}
}
func session(_ session: ARSession, didFailWithError error: Error) {
print("AR Session failed: \(error.localizedDescription)")
}
// MARK: - Raycasting
/// Performs a raycast from screen center to detect planes
func performRaycast(from point: CGPoint, in view: ARView) -> ARRaycastResult? {
// Create raycast query targeting estimated planes
guard let query = view.makeRaycastQuery(
from: point,
allowing: .estimatedPlane,
alignment: .any
) else {
return nil
}
// Perform the raycast
let results = view.session.raycast(query)
return results.first
}
@objc func handleTap(_ gesture: UITapGestureRecognizer) {
guard let arView = arView else { return }
let location = gesture.location(in: arView)
if let result = performRaycast(from: location, in: arView) {
let position = result.worldTransform.columns.3
let resultString = String(format: "Hit at: (%.2f, %.2f, %.2f)", position.x, position.y, position.z)
DispatchQueue.main.async {
self.lastRaycastResult = resultString
}
// Place a visual marker at the hit location
placeMarker(at: result.worldTransform, in: arView)
} else {
DispatchQueue.main.async {
self.lastRaycastResult = "No surface detected"
}
}
}
private func placeMarker(at transform: simd_float4x4, in arView: ARView) {
// Create a small sphere to visualize the raycast hit
let sphere = MeshResource.generateSphere(radius: 0.02)
let material = SimpleMaterial(color: .green, isMetallic: false)
let modelEntity = ModelEntity(mesh: sphere, materials: [material])
// Create an anchor at the hit position
let anchorEntity = AnchorEntity(world: transform)
anchorEntity.addChild(modelEntity)
arView.scene.addAnchor(anchorEntity)
}
}
}

View File

@@ -1,290 +0,0 @@
//
// CameraManager.swift
// SousChefAI
//
// Camera management using AVFoundation for real-time video streaming
//
@preconcurrency import AVFoundation
@preconcurrency import CoreVideo
import UIKit
import Combine
/// Manages camera capture and provides async stream of video frames
@MainActor
final class CameraManager: NSObject, ObservableObject {
@Published var isAuthorized = false
@Published var error: CameraError?
@Published var isRunning = false
enum SessionState {
case idle
case configuring
case configured
case starting
case running
case stopping
}
@Published private(set) var sessionState: SessionState = .idle
nonisolated(unsafe) private let captureSession = AVCaptureSession()
nonisolated(unsafe) private var videoOutput: AVCaptureVideoDataOutput?
private let videoQueue = DispatchQueue(label: "com.souschef.video", qos: .userInitiated)
nonisolated(unsafe) private var frameContinuation: AsyncStream<CVPixelBuffer>.Continuation?
private let continuationQueue = DispatchQueue(label: "com.souschef.continuation")
private var isConfigured = false
private var cachedPreviewLayer: AVCaptureVideoPreviewLayer?
nonisolated override init() {
super.init()
print("🎥 CameraManager.init() - Instance created at \(Date())")
}
// MARK: - Authorization
func checkAuthorization() async {
switch AVCaptureDevice.authorizationStatus(for: .video) {
case .authorized:
isAuthorized = true
case .notDetermined:
isAuthorized = await AVCaptureDevice.requestAccess(for: .video)
case .denied, .restricted:
isAuthorized = false
error = .notAuthorized
@unknown default:
isAuthorized = false
}
}
// MARK: - Session Setup
func setupSession() async throws {
print("🎥 CameraManager.setupSession() - STARTED at \(Date()), current state: \(sessionState)")
// Wait if session is stopping
if sessionState == .stopping {
print("🎥 CameraManager.setupSession() - ⏳ Waiting for session to finish stopping...")
await waitForSessionToStop()
}
// Only configure once
guard !isConfigured else {
print("🎥 CameraManager.setupSession() - Already configured, returning")
return
}
sessionState = .configuring
// Ensure authorization is checked first
await checkAuthorization()
guard isAuthorized else {
throw CameraError.notAuthorized
}
print("🎥 CameraManager.setupSession() - Calling beginConfiguration()")
captureSession.beginConfiguration()
// Set session preset
captureSession.sessionPreset = .high
print("🎥 CameraManager.setupSession() - Set preset to .high")
// Add video input
print("🎥 CameraManager.setupSession() - About to get video device (LINE 72)")
#if targetEnvironment(simulator)
print("🎥 CameraManager.setupSession() - ⚠️ RUNNING ON SIMULATOR - Camera may not work properly")
#endif
guard let videoDevice = AVCaptureDevice.default(.builtInWideAngleCamera, for: .video, position: .back),
let videoInput = try? AVCaptureDeviceInput(device: videoDevice),
captureSession.canAddInput(videoInput) else {
print("🎥 CameraManager.setupSession() - ❌ FAILED to get video device or add input")
captureSession.commitConfiguration()
throw CameraError.setupFailed
}
print("🎥 CameraManager.setupSession() - ✅ Successfully got video device and input")
captureSession.addInput(videoInput)
// Add video output
let output = AVCaptureVideoDataOutput()
output.setSampleBufferDelegate(self, queue: videoQueue)
output.alwaysDiscardsLateVideoFrames = true
output.videoSettings = [
kCVPixelBufferPixelFormatTypeKey as String: kCVPixelFormatType_32BGRA
]
guard captureSession.canAddOutput(output) else {
captureSession.commitConfiguration()
throw CameraError.setupFailed
}
captureSession.addOutput(output)
self.videoOutput = output
captureSession.commitConfiguration()
isConfigured = true
sessionState = .configured
print("🎥 CameraManager.setupSession() - ✅ COMPLETED successfully at \(Date())")
}
// MARK: - Session Control
func startSession() {
guard !captureSession.isRunning else {
print("🎥 CameraManager.startSession() - Session already running")
return
}
print("🎥 CameraManager.startSession() - Starting session")
sessionState = .starting
let session = captureSession
Task.detached { [weak self] in
session.startRunning()
await MainActor.run { [weak self] in
self?.isRunning = true
self?.sessionState = .running
print("🎥 CameraManager.startSession() - ✅ Session running")
}
}
}
func stopSession() {
guard captureSession.isRunning else {
print("🎥 CameraManager.stopSession() - Session not running")
return
}
print("🎥 CameraManager.stopSession() - Stopping session")
sessionState = .stopping
let session = captureSession
Task.detached { [weak self] in
session.stopRunning()
await MainActor.run { [weak self] in
self?.isRunning = false
self?.sessionState = .idle
print("🎥 CameraManager.stopSession() - ✅ Session stopped")
}
}
}
// MARK: - Helper Methods
private func waitForSessionToStop() async {
// Wait for session to reach idle state
while sessionState == .stopping {
try? await Task.sleep(for: .milliseconds(100))
}
print("🎥 CameraManager.waitForSessionToStop() - ✅ Session is now idle")
}
/// Cleanup method that ensures session is fully stopped before returning
func cleanup() async {
print("🎥 CameraManager.cleanup() - Starting cleanup")
if captureSession.isRunning {
stopSession()
}
// Wait for session to fully stop
await waitForSessionToStop()
print("🎥 CameraManager.cleanup() - ✅ Cleanup complete")
}
// MARK: - Frame Stream
func frameStream() -> AsyncStream<CVPixelBuffer> {
AsyncStream { [weak self] continuation in
guard let self = self else { return }
self.continuationQueue.async {
Task { @MainActor in
self.frameContinuation = continuation
}
}
continuation.onTermination = { [weak self] _ in
guard let self = self else { return }
self.continuationQueue.async {
Task { @MainActor in
self.frameContinuation = nil
}
}
}
}
}
// MARK: - Preview Layer
func previewLayer() -> AVCaptureVideoPreviewLayer? {
print("🎥 CameraManager.previewLayer() - ⚠️ CALLED at \(Date()) - isConfigured: \(isConfigured)")
// Only create preview layer after session is configured
if !isConfigured {
print("🎥 CameraManager.previewLayer() - ❌ Session not configured yet, returning nil")
return nil
}
// Return cached layer if available
if let cached = cachedPreviewLayer {
print("🎥 CameraManager.previewLayer() - ✅ Returning cached preview layer")
return cached
}
// Create and cache new preview layer
print("🎥 CameraManager.previewLayer() - ✅ Creating new preview layer")
let layer = AVCaptureVideoPreviewLayer(session: captureSession)
layer.videoGravity = .resizeAspectFill
cachedPreviewLayer = layer
return layer
}
}
// MARK: - AVCaptureVideoDataOutputSampleBufferDelegate
extension CameraManager: AVCaptureVideoDataOutputSampleBufferDelegate {
nonisolated func captureOutput(
_ output: AVCaptureOutput,
didOutput sampleBuffer: CMSampleBuffer,
from connection: AVCaptureConnection
) {
guard let pixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer) else {
return
}
// CVPixelBuffer is thread-safe and immutable, safe to pass across isolation boundaries
// Using nonisolated(unsafe) for continuation since we manage synchronization manually
frameContinuation?.yield(pixelBuffer)
}
}
// MARK: - Error Handling
enum CameraError: Error, LocalizedError {
case notAuthorized
case setupFailed
case captureSessionFailed
var errorDescription: String? {
switch self {
case .notAuthorized:
return "Camera access not authorized. Please enable camera access in Settings."
case .setupFailed:
return "Failed to setup camera session"
case .captureSessionFailed:
return "Camera capture session failed"
}
}
}

View File

@@ -1,248 +0,0 @@
//
// FirestoreRepository.swift
// SousChefAI
//
// Repository pattern for Firebase Firestore data persistence
// Note: Requires Firebase SDK to be added via Swift Package Manager
//
import Foundation
import Combine
/// Repository for managing user data in Firestore
@MainActor
final class FirestoreRepository: ObservableObject {
// Uncomment when Firebase package is added
// private let db = Firestore.firestore()
@Published var currentUser: UserProfile?
@Published var currentInventory: [Ingredient] = []
@Published var savedRecipes: [Recipe] = []
private var userId: String?
nonisolated init() {
// Initialize with current user ID from Firebase Auth
// self.userId = Auth.auth().currentUser?.uid
}
// MARK: - User Profile
/// Fetches the user profile from Firestore
func fetchUserProfile(userId: String) async throws {
self.userId = userId
// When Firebase is added, use this:
/*
let document = try await db.collection("users").document(userId).getDocument()
if let data = document.data() {
currentUser = try Firestore.Decoder().decode(UserProfile.self, from: data)
} else {
// Create default profile
let newProfile = UserProfile(id: userId)
try await saveUserProfile(newProfile)
currentUser = newProfile
}
*/
// Temporary fallback
currentUser = UserProfile(id: userId)
}
/// Saves the user profile to Firestore
func saveUserProfile(_ profile: UserProfile) async throws {
guard userId != nil else { return }
// When Firebase is added, use this:
/*
let data = try Firestore.Encoder().encode(profile)
try await db.collection("users").document(userId).setData(data)
*/
currentUser = profile
}
/// Updates dietary restrictions
func updateDietaryRestrictions(_ restrictions: [String]) async throws {
guard var profile = currentUser else { return }
profile.dietaryRestrictions = restrictions
try await saveUserProfile(profile)
}
/// Updates nutrition goals
func updateNutritionGoals(_ goals: String) async throws {
guard var profile = currentUser else { return }
profile.nutritionGoals = goals
try await saveUserProfile(profile)
}
// MARK: - Inventory Management
/// Fetches current inventory from Firestore
func fetchInventory() async throws {
guard userId != nil else { return }
// When Firebase is added, use this:
/*
let snapshot = try await db.collection("users")
.document(userId)
.collection("inventory")
.getDocuments()
currentInventory = try snapshot.documents.compactMap { document in
try Firestore.Decoder().decode(Ingredient.self, from: document.data())
}
*/
// Temporary fallback
currentInventory = []
}
/// Saves inventory to Firestore
func saveInventory(_ ingredients: [Ingredient]) async throws {
guard userId != nil else { return }
// When Firebase is added, use this:
/*
let batch = db.batch()
let inventoryRef = db.collection("users").document(userId).collection("inventory")
// Delete existing inventory
let existingDocs = try await inventoryRef.getDocuments()
for doc in existingDocs.documents {
batch.deleteDocument(doc.reference)
}
// Add new inventory
for ingredient in ingredients {
let docRef = inventoryRef.document(ingredient.id)
let data = try Firestore.Encoder().encode(ingredient)
batch.setData(data, forDocument: docRef)
}
try await batch.commit()
*/
currentInventory = ingredients
}
/// Adds a single ingredient to inventory
func addIngredient(_ ingredient: Ingredient) async throws {
guard userId != nil else { return }
// When Firebase is added, use this:
/*
let data = try Firestore.Encoder().encode(ingredient)
try await db.collection("users")
.document(userId)
.collection("inventory")
.document(ingredient.id)
.setData(data)
*/
currentInventory.append(ingredient)
}
/// Removes an ingredient from inventory
func removeIngredient(_ ingredientId: String) async throws {
guard userId != nil else { return }
// When Firebase is added, use this:
/*
try await db.collection("users")
.document(userId)
.collection("inventory")
.document(ingredientId)
.delete()
*/
currentInventory.removeAll { $0.id == ingredientId }
}
/// Updates an ingredient in inventory
func updateIngredient(_ ingredient: Ingredient) async throws {
guard userId != nil else { return }
// When Firebase is added, use this:
/*
let data = try Firestore.Encoder().encode(ingredient)
try await db.collection("users")
.document(userId)
.collection("inventory")
.document(ingredient.id)
.updateData(data)
*/
if let index = currentInventory.firstIndex(where: { $0.id == ingredient.id }) {
currentInventory[index] = ingredient
}
}
// MARK: - Recipe Management
/// Saves a recipe to user's favorites
func saveRecipe(_ recipe: Recipe) async throws {
guard userId != nil else { return }
// When Firebase is added, use this:
/*
let data = try Firestore.Encoder().encode(recipe)
try await db.collection("users")
.document(userId)
.collection("savedRecipes")
.document(recipe.id)
.setData(data)
*/
if !savedRecipes.contains(where: { $0.id == recipe.id }) {
savedRecipes.append(recipe)
}
}
/// Fetches saved recipes
func fetchSavedRecipes() async throws {
guard userId != nil else { return }
// When Firebase is added, use this:
/*
let snapshot = try await db.collection("users")
.document(userId)
.collection("savedRecipes")
.getDocuments()
savedRecipes = try snapshot.documents.compactMap { document in
try Firestore.Decoder().decode(Recipe.self, from: document.data())
}
*/
// Temporary fallback
savedRecipes = []
}
/// Deletes a saved recipe
func deleteRecipe(_ recipeId: String) async throws {
guard userId != nil else { return }
// When Firebase is added, use this:
/*
try await db.collection("users")
.document(userId)
.collection("savedRecipes")
.document(recipeId)
.delete()
*/
savedRecipes.removeAll { $0.id == recipeId }
}
// MARK: - Pantry Staples
/// Updates pantry staples (ingredients always available)
func updatePantryStaples(_ staples: [Ingredient]) async throws {
guard var profile = currentUser else { return }
profile.pantryStaples = staples
try await saveUserProfile(profile)
}
}

View File

@@ -1,327 +0,0 @@
//
// GeminiRecipeService.swift
// SousChefAI
//
// Concrete implementation using Google Gemini API for recipe generation
// Note: Requires GoogleGenerativeAI SDK to be added via Swift Package Manager
//
import Foundation
/// Google Gemini implementation for recipe generation and cooking guidance
final class GeminiRecipeService: RecipeService, @unchecked Sendable {
private let apiKey: String
// Note: Uncomment when GoogleGenerativeAI package is added
// private let model: GenerativeModel
nonisolated init(apiKey: String = AppConfig.geminiAPIKey) {
self.apiKey = apiKey
// Initialize Gemini model when package is available
// self.model = GenerativeModel(name: "gemini-2.0-flash-exp", apiKey: apiKey)
}
// MARK: - RecipeService Protocol Implementation
func generateRecipes(inventory: [Ingredient], profile: UserProfile) async throws -> [Recipe] {
guard apiKey != "INSERT_KEY_HERE" else {
throw RecipeServiceError.apiKeyMissing
}
let prompt = buildRecipeGenerationPrompt(inventory: inventory, profile: profile)
// When GoogleGenerativeAI is added, use this:
// let response = try await model.generateContent(prompt)
// return try parseRecipesFromResponse(response.text ?? "")
// Temporary fallback using REST API
return try await generateRecipesViaREST(prompt: prompt)
}
func scaleRecipe(_ recipe: Recipe, for ingredient: Ingredient, quantity: String) async throws -> Recipe {
guard apiKey != "INSERT_KEY_HERE" else {
throw RecipeServiceError.apiKeyMissing
}
let prompt = buildScalingPrompt(recipe: recipe, ingredient: ingredient, quantity: quantity)
return try await scaleRecipeViaREST(prompt: prompt, originalRecipe: recipe)
}
func provideCookingGuidance(for step: String, context: String?) async throws -> String {
guard apiKey != "INSERT_KEY_HERE" else {
throw RecipeServiceError.apiKeyMissing
}
let prompt = buildGuidancePrompt(step: step, context: context)
return try await generateGuidanceViaREST(prompt: prompt)
}
// MARK: - Prompt Building
private func buildRecipeGenerationPrompt(inventory: [Ingredient], profile: UserProfile) -> String {
let inventoryList = inventory.map { "- \($0.name): \($0.estimatedQuantity)" }.joined(separator: "\n")
let restrictions = profile.dietaryRestrictions.isEmpty
? "None"
: profile.dietaryRestrictions.joined(separator: ", ")
let nutritionGoals = profile.nutritionGoals.isEmpty
? "No specific goals"
: profile.nutritionGoals
return """
You are a professional chef AI assistant. Generate creative, practical recipes based on available ingredients.
AVAILABLE INGREDIENTS:
\(inventoryList)
USER PREFERENCES:
- Dietary Restrictions: \(restrictions)
- Nutrition Goals: \(nutritionGoals)
INSTRUCTIONS:
1. Generate 5-7 recipe ideas that can be made with these ingredients
2. Categorize recipes as:
- "The Scavenger": Uses ONLY available ingredients (no shopping needed)
- "The Upgrader": Requires 1-2 additional common ingredients
3. For each recipe, provide:
- Title (creative and appetizing)
- Brief description
- List of missing ingredients (if any)
- Step-by-step cooking instructions
- Match score (0.0-1.0) based on ingredient availability
- Estimated time
- Servings
4. Respect ALL dietary restrictions strictly
5. Prioritize recipes with higher match scores
RESPOND ONLY WITH VALID JSON in this exact format:
{
"recipes": [
{
"title": "Recipe Name",
"description": "Brief description",
"missingIngredients": [
{
"name": "ingredient name",
"estimatedQuantity": "quantity",
"confidence": 1.0
}
],
"steps": ["Step 1", "Step 2", ...],
"matchScore": 0.95,
"estimatedTime": "30 minutes",
"servings": 4
}
]
}
"""
}
private func buildScalingPrompt(recipe: Recipe, ingredient: Ingredient, quantity: String) -> String {
"""
Scale this recipe based on a limiting ingredient quantity.
ORIGINAL RECIPE:
Title: \(recipe.title)
Servings: \(recipe.servings ?? 4)
STEPS:
\(recipe.steps.enumerated().map { "\($0 + 1). \($1)" }.joined(separator: "\n"))
LIMITING INGREDIENT:
\(ingredient.name): I only have \(quantity)
INSTRUCTIONS:
1. Calculate the scaled portions for all ingredients
2. Adjust cooking times if necessary
3. Update servings count
4. Maintain the same step structure but update quantities
RESPOND WITH JSON:
{
"title": "Recipe Name",
"description": "Updated description with new servings",
"missingIngredients": [...],
"steps": ["Updated steps with scaled quantities"],
"matchScore": 0.95,
"estimatedTime": "updated time",
"servings": updated_count
}
"""
}
private func buildGuidancePrompt(step: String, context: String?) -> String {
var prompt = """
You are a cooking assistant providing real-time guidance.
CURRENT STEP: \(step)
"""
if let context = context {
prompt += "\n\nVISUAL CONTEXT: \(context)"
}
prompt += """
Provide brief, actionable guidance for this cooking step.
If the context indicates the step is complete, confirm it.
If there are issues, suggest corrections.
Keep response under 50 words.
"""
return prompt
}
// MARK: - REST API Helpers
private func generateRecipesViaREST(prompt: String) async throws -> [Recipe] {
let url = URL(string: "https://generativelanguage.googleapis.com/v1beta/models/gemini-2.0-flash-exp:generateContent?key=\(apiKey)")!
var request = URLRequest(url: url)
request.httpMethod = "POST"
request.setValue("application/json", forHTTPHeaderField: "Content-Type")
let requestBody: [String: Any] = [
"contents": [
[
"parts": [
["text": prompt]
]
]
],
"generationConfig": [
"temperature": 0.7,
"topK": 40,
"topP": 0.95,
"maxOutputTokens": 8192
]
]
request.httpBody = try JSONSerialization.data(withJSONObject: requestBody)
let (data, response) = try await URLSession.shared.data(for: request)
guard let httpResponse = response as? HTTPURLResponse,
(200...299).contains(httpResponse.statusCode) else {
throw RecipeServiceError.generationFailed("HTTP error")
}
return try parseGeminiResponse(data)
}
private func scaleRecipeViaREST(prompt: String, originalRecipe: Recipe) async throws -> Recipe {
let url = URL(string: "https://generativelanguage.googleapis.com/v1beta/models/gemini-2.0-flash-exp:generateContent?key=\(apiKey)")!
var request = URLRequest(url: url)
request.httpMethod = "POST"
request.setValue("application/json", forHTTPHeaderField: "Content-Type")
let requestBody: [String: Any] = [
"contents": [
[
"parts": [
["text": prompt]
]
]
]
]
request.httpBody = try JSONSerialization.data(withJSONObject: requestBody)
let (data, _) = try await URLSession.shared.data(for: request)
let recipes = try parseGeminiResponse(data)
return recipes.first ?? originalRecipe
}
private func generateGuidanceViaREST(prompt: String) async throws -> String {
let url = URL(string: "https://generativelanguage.googleapis.com/v1beta/models/gemini-2.0-flash-exp:generateContent?key=\(apiKey)")!
var request = URLRequest(url: url)
request.httpMethod = "POST"
request.setValue("application/json", forHTTPHeaderField: "Content-Type")
let requestBody: [String: Any] = [
"contents": [
[
"parts": [
["text": prompt]
]
]
]
]
request.httpBody = try JSONSerialization.data(withJSONObject: requestBody)
let (data, _) = try await URLSession.shared.data(for: request)
guard let json = try JSONSerialization.jsonObject(with: data) as? [String: Any],
let candidates = json["candidates"] as? [[String: Any]],
let firstCandidate = candidates.first,
let content = firstCandidate["content"] as? [String: Any],
let parts = content["parts"] as? [[String: Any]],
let firstPart = parts.first,
let text = firstPart["text"] as? String else {
throw RecipeServiceError.decodingError
}
return text
}
private func parseGeminiResponse(_ data: Data) throws -> [Recipe] {
// Parse Gemini API response structure
guard let json = try JSONSerialization.jsonObject(with: data) as? [String: Any],
let candidates = json["candidates"] as? [[String: Any]],
let firstCandidate = candidates.first,
let content = firstCandidate["content"] as? [String: Any],
let parts = content["parts"] as? [[String: Any]],
let firstPart = parts.first,
let text = firstPart["text"] as? String else {
throw RecipeServiceError.decodingError
}
// Extract JSON from markdown code blocks if present
let cleanedText = text
.replacingOccurrences(of: "```json", with: "")
.replacingOccurrences(of: "```", with: "")
.trimmingCharacters(in: .whitespacesAndNewlines)
guard let jsonData = cleanedText.data(using: .utf8),
let recipeResponse = try? JSONDecoder().decode(GeminiRecipeResponse.self, from: jsonData) else {
throw RecipeServiceError.decodingError
}
return recipeResponse.recipes.map { geminiRecipe in
Recipe(
title: geminiRecipe.title,
description: geminiRecipe.description,
missingIngredients: geminiRecipe.missingIngredients,
steps: geminiRecipe.steps,
matchScore: geminiRecipe.matchScore,
estimatedTime: geminiRecipe.estimatedTime,
servings: geminiRecipe.servings
)
}
}
}
// MARK: - Response Models
private struct GeminiRecipeResponse: Codable {
let recipes: [GeminiRecipe]
}
private struct GeminiRecipe: Codable {
let title: String
let description: String
let missingIngredients: [Ingredient]
let steps: [String]
let matchScore: Double
let estimatedTime: String?
let servings: Int?
}

View File

@@ -1,503 +0,0 @@
//
// GeminiVisionService.swift
// SousChefAI
//
// Vision service using Google Gemini 3.0 Flash for ingredient detection
// Sends least blurry frame per second to Gemini API for analysis
//
import Foundation
import CoreVideo
import CoreImage
import Accelerate
import UIKit
/// Gemini-based implementation for vision ingredient detection
final class GeminiVisionService: VisionService, @unchecked Sendable {
private let apiKey: String
private let modelName = "gemini-2.0-flash-exp" // Will update to 3.0 when available
nonisolated init(apiKey: String = AppConfig.geminiAPIKey) {
self.apiKey = apiKey
}
// MARK: - VisionService Protocol Implementation
nonisolated func detectIngredients(from stream: AsyncStream<CVPixelBuffer>) async throws -> [Ingredient] {
// This method is used for continuous scanning
// Collect frames, pick least blurry per second, send to Gemini
var allDetectedIngredients: [Ingredient] = []
var currentSecondFrames: [(buffer: CVPixelBuffer, blurScore: Double, timestamp: Date)] = []
var lastProcessTime = Date()
for await pixelBuffer in stream {
let now = Date()
let blurScore = calculateBlurScore(pixelBuffer)
currentSecondFrames.append((buffer: pixelBuffer, blurScore: blurScore, timestamp: now))
// Process every second
if now.timeIntervalSince(lastProcessTime) >= 1.0 {
// Find least blurry frame (highest Laplacian variance = sharpest)
if let bestFrame = currentSecondFrames.max(by: { $0.blurScore < $1.blurScore }) {
do {
let ingredients = try await analyzeFrameWithGemini(
bestFrame.buffer,
existingIngredients: allDetectedIngredients
)
// Debug output
print("🔍 GeminiVisionService: Detected \(ingredients.count) items in frame")
if !ingredients.isEmpty {
let jsonData = try? JSONEncoder().encode(ingredients)
if let jsonString = jsonData.flatMap({ String(data: $0, encoding: .utf8) }) {
print("📋 JSON Response: \(jsonString)")
}
}
// Merge ingredients
allDetectedIngredients = mergeIngredients(existing: allDetectedIngredients, new: ingredients)
} catch {
print("⚠️ GeminiVisionService: Frame analysis failed: \(error)")
// Continue scanning on errors
}
}
currentSecondFrames.removeAll()
lastProcessTime = now
}
// Stop after reasonable scan time or max ingredients
if allDetectedIngredients.count >= AppConfig.maxIngredientsPerScan {
break
}
}
return allDetectedIngredients
}
nonisolated func detectIngredients(from pixelBuffer: CVPixelBuffer) async throws -> [Ingredient] {
return try await analyzeFrameWithGemini(pixelBuffer, existingIngredients: [])
}
nonisolated func analyzeCookingProgress(from stream: AsyncStream<CVPixelBuffer>, for step: String) async throws -> CookingProgress {
// For cooking progress, we'll use Gemini to analyze the current state
var latestFrame: CVPixelBuffer?
for await frame in stream {
latestFrame = frame
break // Just get one frame for now
}
guard let frame = latestFrame else {
return CookingProgress(isComplete: false, confidence: 0.0, feedback: "No frame available")
}
return try await analyzeCookingStepWithGemini(frame, step: step)
}
// MARK: - Blur Detection (Laplacian Variance)
/// Calculates blur score using Laplacian variance
/// Higher value = sharper image, Lower value = more blurry
nonisolated private func calculateBlurScore(_ pixelBuffer: CVPixelBuffer) -> Double {
CVPixelBufferLockBaseAddress(pixelBuffer, .readOnly)
defer { CVPixelBufferUnlockBaseAddress(pixelBuffer, .readOnly) }
let width = CVPixelBufferGetWidth(pixelBuffer)
let height = CVPixelBufferGetHeight(pixelBuffer)
let bytesPerRow = CVPixelBufferGetBytesPerRow(pixelBuffer)
guard let baseAddress = CVPixelBufferGetBaseAddress(pixelBuffer) else {
return 0
}
// Convert to grayscale for Laplacian calculation
// For BGRA format, we'll use simple luminance approximation
var grayscale = [Float](repeating: 0, count: width * height)
let pixels = baseAddress.assumingMemoryBound(to: UInt8.self)
for y in 0..<height {
for x in 0..<width {
let offset = y * bytesPerRow + x * 4
let b = Float(pixels[offset])
let g = Float(pixels[offset + 1])
let r = Float(pixels[offset + 2])
// Luminance formula
grayscale[y * width + x] = 0.299 * r + 0.587 * g + 0.114 * b
}
}
// Apply Laplacian kernel: [0, 1, 0], [1, -4, 1], [0, 1, 0]
var laplacianOutput = [Float](repeating: 0, count: width * height)
for y in 1..<(height - 1) {
for x in 1..<(width - 1) {
let idx = y * width + x
let laplacian = -4 * grayscale[idx]
+ grayscale[(y - 1) * width + x]
+ grayscale[(y + 1) * width + x]
+ grayscale[y * width + (x - 1)]
+ grayscale[y * width + (x + 1)]
laplacianOutput[idx] = laplacian
}
}
// Calculate variance of Laplacian
let count = Float(laplacianOutput.count)
let mean = laplacianOutput.reduce(0, +) / count
let variance = laplacianOutput.reduce(0) { $0 + ($1 - mean) * ($1 - mean) } / count
return Double(variance)
}
// MARK: - Gemini API Integration
nonisolated private func analyzeFrameWithGemini(
_ pixelBuffer: CVPixelBuffer,
existingIngredients: [Ingredient]
) async throws -> [Ingredient] {
guard apiKey != "INSERT_KEY_HERE" else {
throw VisionServiceError.apiKeyMissing
}
// Convert pixel buffer to base64 JPEG
let imageData = try convertToJPEG(pixelBuffer)
let base64Image = imageData.base64EncodedString()
// Build prompt with existing ingredients for deduplication
let existingList = existingIngredients.isEmpty
? "None yet"
: existingIngredients.map { $0.name }.joined(separator: ", ")
let prompt = """
Analyze this image and identify all food items and ingredients visible.
ALREADY DETECTED ITEMS (avoid duplicates, merge similar items):
\(existingList)
For each NEW item not already listed above, provide:
1. The item name (normalized - e.g., "milk" not "milk 2%", "whole milk", etc.)
2. Estimated quantity (numeric with unit, e.g., "2", "500ml", "1 dozen")
3. Top 3 guesses for what the item might be, with confidence (0.0-1.0)
IMPORTANT:
- If you see "milk 2%" and "milk" is already detected, DO NOT include it
- Use simple, normalized names (e.g., "apple" not "red delicious apple")
- Quantity should be numeric estimates
- Only include food items and ingredients, not containers or non-food items
RESPOND ONLY WITH VALID JSON in this exact format (no markdown):
{
"items": [
{
"name": "normalized item name",
"quantity": "2",
"guesses": [
{"name": "primary guess", "confidence": 0.95},
{"name": "second guess", "confidence": 0.7},
{"name": "third guess", "confidence": 0.3}
]
}
]
}
If no new food items are visible, return: {"items": []}
"""
let url = URL(string: "https://generativelanguage.googleapis.com/v1beta/models/\(modelName):generateContent?key=\(apiKey)")!
var request = URLRequest(url: url)
request.httpMethod = "POST"
request.setValue("application/json", forHTTPHeaderField: "Content-Type")
request.timeoutInterval = 30
let requestBody: [String: Any] = [
"contents": [
[
"parts": [
["text": prompt],
[
"inline_data": [
"mime_type": "image/jpeg",
"data": base64Image
]
]
]
]
],
"generationConfig": [
"temperature": 0.2,
"topK": 32,
"topP": 0.95,
"maxOutputTokens": 2048
]
]
request.httpBody = try JSONSerialization.data(withJSONObject: requestBody)
let (data, response) = try await URLSession.shared.data(for: request)
guard let httpResponse = response as? HTTPURLResponse else {
throw VisionServiceError.invalidResponse
}
guard (200...299).contains(httpResponse.statusCode) else {
print("❌ Gemini API Error: HTTP \(httpResponse.statusCode)")
if let errorText = String(data: data, encoding: .utf8) {
print("❌ Error body: \(errorText)")
}
throw VisionServiceError.networkError(NSError(domain: "GeminiAPI", code: httpResponse.statusCode))
}
return try parseGeminiVisionResponse(data)
}
nonisolated private func analyzeCookingStepWithGemini(_ pixelBuffer: CVPixelBuffer, step: String) async throws -> CookingProgress {
guard apiKey != "INSERT_KEY_HERE" else {
throw VisionServiceError.apiKeyMissing
}
let imageData = try convertToJPEG(pixelBuffer)
let base64Image = imageData.base64EncodedString()
let prompt = """
Analyze this cooking image for the following step:
"\(step)"
Determine:
1. Is this step complete? (true/false)
2. Confidence level (0.0-1.0)
3. Brief feedback on the current state
RESPOND WITH JSON:
{
"isComplete": false,
"confidence": 0.7,
"feedback": "Brief description of current state"
}
"""
let url = URL(string: "https://generativelanguage.googleapis.com/v1beta/models/\(modelName):generateContent?key=\(apiKey)")!
var request = URLRequest(url: url)
request.httpMethod = "POST"
request.setValue("application/json", forHTTPHeaderField: "Content-Type")
let requestBody: [String: Any] = [
"contents": [
[
"parts": [
["text": prompt],
[
"inline_data": [
"mime_type": "image/jpeg",
"data": base64Image
]
]
]
]
]
]
request.httpBody = try JSONSerialization.data(withJSONObject: requestBody)
let (data, _) = try await URLSession.shared.data(for: request)
return try parseCookingProgressResponse(data)
}
// MARK: - Image Conversion
nonisolated private func convertToJPEG(_ pixelBuffer: CVPixelBuffer) throws -> Data {
let ciImage = CIImage(cvPixelBuffer: pixelBuffer)
let context = CIContext()
guard let cgImage = context.createCGImage(ciImage, from: ciImage.extent) else {
throw VisionServiceError.invalidResponse
}
let uiImage = UIImage(cgImage: cgImage)
// Compress to reasonable size for API
guard let jpegData = uiImage.jpegData(compressionQuality: 0.7) else {
throw VisionServiceError.invalidResponse
}
return jpegData
}
// MARK: - Response Parsing
nonisolated private func parseGeminiVisionResponse(_ data: Data) throws -> [Ingredient] {
guard let json = try JSONSerialization.jsonObject(with: data) as? [String: Any],
let candidates = json["candidates"] as? [[String: Any]],
let firstCandidate = candidates.first,
let content = firstCandidate["content"] as? [String: Any],
let parts = content["parts"] as? [[String: Any]],
let firstPart = parts.first,
let text = firstPart["text"] as? String else {
throw VisionServiceError.decodingError(NSError(domain: "Parsing", code: 0))
}
// Clean up response (remove markdown if present)
let cleanedText = text
.replacingOccurrences(of: "```json", with: "")
.replacingOccurrences(of: "```", with: "")
.trimmingCharacters(in: .whitespacesAndNewlines)
print("📝 Cleaned Gemini response: \(cleanedText)")
guard let jsonData = cleanedText.data(using: .utf8) else {
throw VisionServiceError.decodingError(NSError(domain: "Parsing", code: 1))
}
let response = try JSONDecoder().decode(GeminiVisionResponse.self, from: jsonData)
return response.items.map { item in
// Use the highest confidence guess as the primary name
let bestGuess = item.guesses.first
let confidence = bestGuess?.confidence ?? 0.5
return Ingredient(
name: item.name,
estimatedQuantity: item.quantity,
confidence: confidence,
guesses: item.guesses.map { IngredientGuess(name: $0.name, confidence: $0.confidence) }
)
}
}
nonisolated private func parseCookingProgressResponse(_ data: Data) throws -> CookingProgress {
guard let json = try JSONSerialization.jsonObject(with: data) as? [String: Any],
let candidates = json["candidates"] as? [[String: Any]],
let firstCandidate = candidates.first,
let content = firstCandidate["content"] as? [String: Any],
let parts = content["parts"] as? [[String: Any]],
let firstPart = parts.first,
let text = firstPart["text"] as? String else {
throw VisionServiceError.decodingError(NSError(domain: "Parsing", code: 0))
}
let cleanedText = text
.replacingOccurrences(of: "```json", with: "")
.replacingOccurrences(of: "```", with: "")
.trimmingCharacters(in: .whitespacesAndNewlines)
guard let jsonData = cleanedText.data(using: .utf8),
let progressJson = try JSONSerialization.jsonObject(with: jsonData) as? [String: Any] else {
throw VisionServiceError.decodingError(NSError(domain: "Parsing", code: 1))
}
return CookingProgress(
isComplete: progressJson["isComplete"] as? Bool ?? false,
confidence: progressJson["confidence"] as? Double ?? 0.5,
feedback: progressJson["feedback"] as? String ?? "Processing..."
)
}
// MARK: - Ingredient Merging
/// Merges new ingredients with existing ones, handling similar names and taking max quantity
nonisolated private func mergeIngredients(existing: [Ingredient], new: [Ingredient]) -> [Ingredient] {
var merged = existing.reduce(into: [String: Ingredient]()) { dict, ingredient in
dict[ingredient.name.lowercased()] = ingredient
}
for newIngredient in new {
let normalizedName = newIngredient.name.lowercased()
// Check for similar existing items
let similarKey = merged.keys.first { existingKey in
isSimilarIngredient(existingKey, normalizedName)
}
if let key = similarKey, let existing = merged[key] {
// Merge: take max quantity, higher confidence
let mergedQuantity = mergeQuantities(existing.estimatedQuantity, newIngredient.estimatedQuantity)
let mergedConfidence = max(existing.confidence, newIngredient.confidence)
merged[key] = Ingredient(
id: existing.id,
name: existing.name, // Keep original name
estimatedQuantity: mergedQuantity,
confidence: mergedConfidence,
guesses: existing.guesses // Keep original guesses
)
} else {
// Add as new
merged[normalizedName] = newIngredient
}
}
return Array(merged.values).sorted { $0.confidence > $1.confidence }
}
/// Checks if two ingredient names are similar (e.g., "milk" and "milk 2%")
nonisolated private func isSimilarIngredient(_ name1: String, _ name2: String) -> Bool {
// Exact match
if name1 == name2 { return true }
// One contains the other
if name1.contains(name2) || name2.contains(name1) { return true }
// Common ingredient variations
let variations: [[String]] = [
["milk", "whole milk", "2% milk", "skim milk", "milk 2%"],
["egg", "eggs", "large eggs"],
["butter", "unsalted butter", "salted butter"],
["cheese", "cheddar", "cheddar cheese"],
["chicken", "chicken breast", "chicken thigh"],
["onion", "onions", "yellow onion", "white onion"],
["tomato", "tomatoes", "cherry tomatoes"],
["potato", "potatoes", "russet potato"]
]
for group in variations {
let lowercaseGroup = group.map { $0.lowercased() }
if lowercaseGroup.contains(name1) && lowercaseGroup.contains(name2) {
return true
}
}
return false
}
/// Merges two quantity strings, taking the maximum
nonisolated private func mergeQuantities(_ q1: String, _ q2: String) -> String {
// Extract numeric values
let num1 = extractNumber(from: q1) ?? 0
let num2 = extractNumber(from: q2) ?? 0
// Return the quantity with larger number
return num1 >= num2 ? q1 : q2
}
nonisolated private func extractNumber(from string: String) -> Double? {
let pattern = #"[\d.]+"#
guard let regex = try? NSRegularExpression(pattern: pattern),
let match = regex.firstMatch(in: string, range: NSRange(string.startIndex..., in: string)),
let range = Range(match.range, in: string) else {
return nil
}
return Double(string[range])
}
}
// MARK: - Response Models
private struct GeminiVisionResponse: Codable {
let items: [GeminiVisionItem]
}
private struct GeminiVisionItem: Codable {
let name: String
let quantity: String
let guesses: [GeminiGuess]
}
private struct GeminiGuess: Codable {
let name: String
let confidence: Double
}

View File

@@ -1,56 +0,0 @@
//
// RecipeService.swift
// SousChefAI
//
// Protocol for recipe generation and AI reasoning services
//
import Foundation
/// Protocol for AI-powered recipe generation
protocol RecipeService: Sendable {
/// Generates recipes based on available ingredients and user preferences
/// - Parameters:
/// - inventory: Available ingredients
/// - profile: User dietary preferences and restrictions
/// - Returns: Array of recipe suggestions with match scores
func generateRecipes(inventory: [Ingredient], profile: UserProfile) async throws -> [Recipe]
/// Scales a recipe based on a limiting ingredient quantity
/// - Parameters:
/// - recipe: The recipe to scale
/// - ingredient: The limiting ingredient
/// - quantity: Available quantity of the limiting ingredient
/// - Returns: Scaled recipe with adjusted portions
func scaleRecipe(_ recipe: Recipe, for ingredient: Ingredient, quantity: String) async throws -> Recipe
/// Provides real-time cooking guidance
/// - Parameters:
/// - step: Current cooking step
/// - context: Additional context (e.g., visual feedback)
/// - Returns: Guidance text
func provideCookingGuidance(for step: String, context: String?) async throws -> String
}
enum RecipeServiceError: Error, LocalizedError {
case apiKeyMissing
case invalidRequest
case generationFailed(String)
case networkError(Error)
case decodingError
var errorDescription: String? {
switch self {
case .apiKeyMissing:
return "Recipe service API key not configured"
case .invalidRequest:
return "Invalid recipe generation request"
case .generationFailed(let reason):
return "Recipe generation failed: \(reason)"
case .networkError(let error):
return "Network error: \(error.localizedDescription)"
case .decodingError:
return "Failed to parse recipe response"
}
}
}

View File

@@ -1,60 +0,0 @@
//
// VisionService.swift
// SousChefAI
//
// Protocol-based vision service for ingredient detection
// Allows swapping between different AI providers
//
import Foundation
@preconcurrency import CoreVideo
/// Protocol for vision-based ingredient detection services
protocol VisionService: Sendable {
/// Detects ingredients from a stream of video frames
/// - Parameter stream: Async stream of pixel buffers from camera
/// - Returns: Array of detected ingredients with confidence scores
func detectIngredients(from stream: AsyncStream<CVPixelBuffer>) async throws -> [Ingredient]
/// Detects ingredients from a single image
/// - Parameter pixelBuffer: Single frame to analyze
/// - Returns: Array of detected ingredients with confidence scores
func detectIngredients(from pixelBuffer: CVPixelBuffer) async throws -> [Ingredient]
/// Analyzes cooking progress for a given step
/// - Parameters:
/// - stream: Video stream of current cooking
/// - step: The cooking step to monitor
/// - Returns: Progress update and completion detection
func analyzeCookingProgress(from stream: AsyncStream<CVPixelBuffer>, for step: String) async throws -> CookingProgress
}
/// Represents cooking progress analysis
struct CookingProgress: Sendable {
let isComplete: Bool
let confidence: Double
let feedback: String
}
enum VisionServiceError: Error, LocalizedError {
case connectionFailed
case invalidResponse
case apiKeyMissing
case networkError(Error)
case decodingError(Error)
var errorDescription: String? {
switch self {
case .connectionFailed:
return "Failed to connect to vision service"
case .invalidResponse:
return "Received invalid response from vision service"
case .apiKeyMissing:
return "Vision service API key not configured"
case .networkError(let error):
return "Network error: \(error.localizedDescription)"
case .decodingError(let error):
return "Failed to decode response: \(error.localizedDescription)"
}
}
}

View File

@@ -1,39 +0,0 @@
//
// SousChefAIApp.swift
// SousChefAI
//
// Created by Aditya Pulipaka on 2/11/26.
//
import SwiftUI
// Uncomment when Firebase package is added
// import FirebaseCore
@main
struct SousChefAIApp: App {
// Uncomment when Firebase package is added
// init() {
// FirebaseApp.configure()
// }
// [INSERT_FIREBASE_GOOGLESERVICE-INFO.PLIST_SETUP_HERE]
// Firebase Setup Instructions:
// 1. Add Firebase to your project via Swift Package Manager
// - File > Add Package Dependencies
// - URL: https://github.com/firebase/firebase-ios-sdk
// - Add: FirebaseAuth, FirebaseFirestore
// 2. Download GoogleService-Info.plist from Firebase Console
// 3. Add it to the Xcode project (drag into project navigator)
// 4. Ensure it's added to the SousChefAI target
// 5. Uncomment the FirebaseCore import and init() above
@StateObject private var repository = FirestoreRepository()
var body: some Scene {
WindowGroup {
ContentView()
.environmentObject(repository)
}
}
}

View File

@@ -1,191 +0,0 @@
//
// CookingModeViewModel.swift
// SousChefAI
//
// ViewModel for live cooking guidance with AI monitoring
//
import Foundation
import AVFoundation
import CoreVideo
import Combine
import UIKit
@MainActor
final class CookingModeViewModel: ObservableObject {
@Published var currentStepIndex = 0
@Published var isMonitoring = false
@Published var feedback: String = "Ready to start"
@Published var stepComplete = false
@Published var confidence: Double = 0.0
@Published var error: Error?
let recipe: Recipe
private let visionService: VisionService
private let recipeService: RecipeService
private let cameraManager: CameraManager
private var monitoringTask: Task<Void, Never>?
var currentStep: String {
guard currentStepIndex < recipe.steps.count else {
return "Recipe complete!"
}
return recipe.steps[currentStepIndex]
}
var progress: Double {
guard !recipe.steps.isEmpty else { return 0 }
return Double(currentStepIndex) / Double(recipe.steps.count)
}
var isComplete: Bool {
currentStepIndex >= recipe.steps.count
}
nonisolated init(recipe: Recipe,
visionService: VisionService = ARVisionService(),
recipeService: RecipeService = GeminiRecipeService(),
cameraManager: CameraManager = CameraManager()) {
self.recipe = recipe
self.visionService = visionService
self.recipeService = recipeService
self.cameraManager = cameraManager
}
// MARK: - Camera Setup
func setupCamera() async {
do {
try await cameraManager.setupSession()
} catch {
self.error = error
}
}
func startCamera() {
cameraManager.startSession()
}
func stopCamera() {
cameraManager.stopSession()
}
func getPreviewLayer() -> AVCaptureVideoPreviewLayer? {
cameraManager.previewLayer()
}
// MARK: - Step Navigation
func nextStep() {
guard currentStepIndex < recipe.steps.count else { return }
currentStepIndex += 1
stepComplete = false
confidence = 0.0
feedback = currentStepIndex < recipe.steps.count ? "Starting next step..." : "Recipe complete!"
if !isComplete && isMonitoring {
// Restart monitoring for new step
stopMonitoring()
startMonitoring()
}
}
func previousStep() {
guard currentStepIndex > 0 else { return }
currentStepIndex -= 1
stepComplete = false
confidence = 0.0
feedback = "Returned to previous step"
if isMonitoring {
stopMonitoring()
startMonitoring()
}
}
// MARK: - AI Monitoring
func startMonitoring() {
guard !isComplete, !isMonitoring else { return }
isMonitoring = true
feedback = "Monitoring your cooking..."
monitoringTask = Task {
do {
let stream = cameraManager.frameStream()
let progress = try await visionService.analyzeCookingProgress(
from: stream,
for: currentStep
)
handleProgress(progress)
} catch {
self.error = error
feedback = "Monitoring paused"
isMonitoring = false
}
}
}
func stopMonitoring() {
monitoringTask?.cancel()
monitoringTask = nil
isMonitoring = false
feedback = "Monitoring stopped"
}
private func handleProgress(_ progress: CookingProgress) {
confidence = progress.confidence
feedback = progress.feedback
stepComplete = progress.isComplete
if progress.isComplete && progress.confidence > 0.8 {
// Play haptic feedback
let generator = UINotificationFeedbackGenerator()
generator.notificationOccurred(.success)
// Speak the feedback using text-to-speech
speakFeedback("Step complete! \(progress.feedback)")
}
}
// MARK: - Text Guidance
func getTextGuidance() async {
do {
let guidance = try await recipeService.provideCookingGuidance(
for: currentStep,
context: feedback
)
feedback = guidance
} catch {
self.error = error
}
}
// MARK: - Text-to-Speech
private func speakFeedback(_ text: String) {
let utterance = AVSpeechUtterance(string: text)
utterance.voice = AVSpeechSynthesisVoice(language: "en-US")
utterance.rate = 0.5
let synthesizer = AVSpeechSynthesizer()
synthesizer.speak(utterance)
}
func speakCurrentStep() {
speakFeedback(currentStep)
}
// MARK: - Cleanup
func cleanup() {
stopMonitoring()
stopCamera()
}
}

View File

@@ -1,135 +0,0 @@
//
// RecipeGeneratorViewModel.swift
// SousChefAI
//
// ViewModel for recipe generation and filtering
//
import Foundation
import Combine
@MainActor
final class RecipeGeneratorViewModel: ObservableObject {
@Published var recipes: [Recipe] = []
@Published var filteredRecipes: [Recipe] = []
@Published var isGenerating = false
@Published var error: Error?
@Published var selectedFilter: RecipeFilter = .all
private let recipeService: RecipeService
private let repository: FirestoreRepository
nonisolated init(recipeService: RecipeService = GeminiRecipeService(),
repository: FirestoreRepository = FirestoreRepository()) {
self.recipeService = recipeService
self.repository = repository
}
// MARK: - Recipe Generation
func generateRecipes(inventory: [Ingredient], profile: UserProfile) async {
isGenerating = true
error = nil
do {
let generatedRecipes = try await recipeService.generateRecipes(
inventory: inventory,
profile: profile
)
recipes = generatedRecipes.sorted { $0.matchScore > $1.matchScore }
applyFilter()
} catch {
self.error = error
}
isGenerating = false
}
// MARK: - Filtering
func applyFilter() {
switch selectedFilter {
case .all:
filteredRecipes = recipes
case .scavenger:
filteredRecipes = recipes.filter { $0.category == .scavenger }
case .upgrader:
filteredRecipes = recipes.filter { $0.category == .upgrader }
case .highMatch:
filteredRecipes = recipes.filter { $0.matchScore >= 0.8 }
}
}
func setFilter(_ filter: RecipeFilter) {
selectedFilter = filter
applyFilter()
}
// MARK: - Recipe Scaling
func scaleRecipe(_ recipe: Recipe, for ingredient: Ingredient, quantity: String) async {
do {
let scaledRecipe = try await recipeService.scaleRecipe(
recipe,
for: ingredient,
quantity: quantity
)
// Update the recipe in the list
if let index = recipes.firstIndex(where: { $0.id == recipe.id }) {
recipes[index] = scaledRecipe
applyFilter()
}
} catch {
self.error = error
}
}
// MARK: - Favorites
func saveRecipe(_ recipe: Recipe) async {
do {
try await repository.saveRecipe(recipe)
} catch {
self.error = error
}
}
}
// MARK: - Recipe Filter
enum RecipeFilter: String, CaseIterable, Identifiable {
case all = "All Recipes"
case scavenger = "The Scavenger"
case upgrader = "The Upgrader"
case highMatch = "High Match"
var id: String { rawValue }
var icon: String {
switch self {
case .all: return "square.grid.2x2"
case .scavenger: return "checkmark.circle.fill"
case .upgrader: return "cart.badge.plus"
case .highMatch: return "star.fill"
}
}
var description: String {
switch self {
case .all:
return "Show all recipes"
case .scavenger:
return "Uses only your ingredients"
case .upgrader:
return "Needs 1-2 additional items"
case .highMatch:
return "80%+ ingredient match"
}
}
}

View File

@@ -1,364 +0,0 @@
//
// ScannerViewModel.swift
// SousChefAI
//
// ViewModel for the scanner view with real-time ingredient detection
//
import Foundation
import SwiftUI
import CoreVideo
import AVFoundation
import Combine
@MainActor
final class ScannerViewModel: ObservableObject {
@Published var detectedIngredients: [Ingredient] = []
@Published var isScanning = false
@Published var error: Error?
@Published var scanProgress: String = "Ready to scan"
/// The most recently detected new ingredient (for banner display)
@Published var latestNewIngredient: Ingredient?
private let visionService: VisionService
private let cameraManager: CameraManager
private var scanTask: Task<Void, Never>?
/// Callback when a new ingredient is detected (not a duplicate)
var onNewIngredientDetected: ((Ingredient) -> Void)?
nonisolated init(cameraManager: CameraManager = CameraManager()) {
print("📱 ScannerViewModel.init() - Creating ViewModel at \(Date())")
// Select vision service based on configuration
let visionService: VisionService = switch AppConfig.scanningMode {
case .geminiVision:
GeminiVisionService()
case .arKit:
ARVisionService()
}
print("📱 ScannerViewModel.init() - Using \(AppConfig.scanningMode.rawValue) scanning mode")
self.visionService = visionService
self.cameraManager = cameraManager
}
// MARK: - Camera Management
func setupCamera() async {
print("📱 ScannerViewModel.setupCamera() - STARTED at \(Date())")
do {
try await cameraManager.setupSession()
print("📱 ScannerViewModel.setupCamera() - ✅ SUCCESS at \(Date())")
} catch {
print("📱 ScannerViewModel.setupCamera() - ❌ ERROR: \(error)")
self.error = error
}
}
func startCamera() {
cameraManager.startSession()
}
func stopCamera() {
cameraManager.stopSession()
}
func getPreviewLayer() -> AVCaptureVideoPreviewLayer? {
print("📱 ScannerViewModel.getPreviewLayer() - ⚠️ REQUESTING preview layer at \(Date())")
return cameraManager.previewLayer()
}
// MARK: - Scanning
func startScanning() {
guard !isScanning else { return }
isScanning = true
scanProgress = "Scanning ingredients..."
print("📱 ScannerViewModel.startScanning() - Started with \(AppConfig.scanningMode.rawValue) mode")
scanTask = Task {
let startTime = Date()
do {
let stream = cameraManager.frameStream()
// For Gemini mode, we use real-time detection with callbacks
if AppConfig.scanningMode == .geminiVision {
// Process frames continuously until stopped or timeout
var lastProcessTime = Date()
var currentSecondFrames: [(buffer: CVPixelBuffer, timestamp: Date)] = []
for await frame in stream {
guard !Task.isCancelled else { break }
// Check timeout
if Date().timeIntervalSince(startTime) >= AppConfig.maxScanDuration {
print("📱 ScannerViewModel: Max scan duration reached")
break
}
currentSecondFrames.append((buffer: frame, timestamp: Date()))
// Process every second
let now = Date()
if now.timeIntervalSince(lastProcessTime) >= AppConfig.geminiFrameInterval {
// Pick the frame from the middle of the batch (reasonable approximation)
if let bestFrame = currentSecondFrames[safe: currentSecondFrames.count / 2]?.buffer {
do {
let previousCount = detectedIngredients.count
let ingredients = try await visionService.detectIngredients(from: bestFrame)
// Find new ingredients before merging
let newIngredients = findNewIngredients(ingredients)
// Merge with existing
updateDetectedIngredients(ingredients, mergeMode: true)
// Notify about new ingredients
for newIngredient in newIngredients {
print("🆕 New ingredient detected: \(newIngredient.name)")
latestNewIngredient = newIngredient
onNewIngredientDetected?(newIngredient)
}
scanProgress = "Found \(detectedIngredients.count) items..."
} catch {
print("⚠️ Frame analysis error: \(error)")
// Continue scanning on errors
}
}
currentSecondFrames.removeAll()
lastProcessTime = now
}
// Stop if we have enough ingredients
if detectedIngredients.count >= AppConfig.maxIngredientsPerScan {
break
}
}
} else {
// AR mode: use batch detection
let ingredients = try await visionService.detectIngredients(from: stream)
updateDetectedIngredients(ingredients)
}
scanProgress = "Scan complete! Found \(detectedIngredients.count) ingredients"
} catch {
self.error = error
scanProgress = "Scan failed: \(error.localizedDescription)"
}
isScanning = false
}
}
func stopScanning() {
scanTask?.cancel()
scanTask = nil
isScanning = false
scanProgress = detectedIngredients.isEmpty ? "Ready to scan" : "Scan captured"
}
// MARK: - Real-time Detection Mode
func startRealTimeDetection() {
guard !isScanning else { return }
isScanning = true
scanProgress = "Detecting in real-time..."
scanTask = Task {
let stream = cameraManager.frameStream()
for await frame in stream {
guard !Task.isCancelled else { break }
do {
// Process individual frames
let ingredients = try await visionService.detectIngredients(from: frame)
updateDetectedIngredients(ingredients, mergeMode: true)
scanProgress = "Detected \(detectedIngredients.count) items"
} catch {
// Continue on errors in real-time mode
continue
}
// Throttle to avoid overwhelming the API
try? await Task.sleep(for: .seconds(1))
}
isScanning = false
}
}
// MARK: - Ingredient Management
/// Finds ingredients that are truly new (not already in our list)
private func findNewIngredients(_ newIngredients: [Ingredient]) -> [Ingredient] {
return newIngredients.filter { newIngredient in
!detectedIngredients.contains { existing in
isSimilarIngredient(existing.name, newIngredient.name)
}
}
}
/// Checks if two ingredient names refer to the same item
private func isSimilarIngredient(_ name1: String, _ name2: String) -> Bool {
let n1 = name1.lowercased()
let n2 = name2.lowercased()
// Exact match
if n1 == n2 { return true }
// One contains the other
if n1.contains(n2) || n2.contains(n1) { return true }
return false
}
private func updateDetectedIngredients(_ newIngredients: [Ingredient], mergeMode: Bool = false) {
if mergeMode {
// Merge with existing ingredients, keeping higher confidence and max quantity
var merged = detectedIngredients.reduce(into: [String: Ingredient]()) { dict, ingredient in
dict[ingredient.name.lowercased()] = ingredient
}
for ingredient in newIngredients {
let normalizedName = ingredient.name.lowercased()
// Check for similar existing items
let similarKey = merged.keys.first { existingKey in
isSimilarIngredient(existingKey, normalizedName)
}
if let key = similarKey, let existing = merged[key] {
// Merge: take max quantity, higher confidence
let mergedQuantity = mergeQuantities(existing.estimatedQuantity, ingredient.estimatedQuantity)
let mergedConfidence = max(existing.confidence, ingredient.confidence)
merged[key] = Ingredient(
id: existing.id,
name: existing.name,
estimatedQuantity: mergedQuantity,
confidence: mergedConfidence,
guesses: existing.guesses.isEmpty ? ingredient.guesses : existing.guesses
)
} else {
merged[normalizedName] = ingredient
}
}
detectedIngredients = Array(merged.values).sorted { $0.confidence > $1.confidence }
} else {
detectedIngredients = newIngredients
}
}
/// Merges two quantity strings, taking the maximum numeric value
private func mergeQuantities(_ q1: String, _ q2: String) -> String {
let num1 = extractNumber(from: q1) ?? 0
let num2 = extractNumber(from: q2) ?? 0
return num1 >= num2 ? q1 : q2
}
private func extractNumber(from string: String) -> Double? {
let pattern = #"[\d.]+"#
guard let regex = try? NSRegularExpression(pattern: pattern),
let match = regex.firstMatch(in: string, range: NSRange(string.startIndex..., in: string)),
let range = Range(match.range, in: string) else {
return nil
}
return Double(string[range])
}
func addIngredient(_ ingredient: Ingredient) {
if !detectedIngredients.contains(where: { $0.id == ingredient.id }) {
detectedIngredients.append(ingredient)
}
}
func removeIngredient(_ ingredient: Ingredient) {
detectedIngredients.removeAll { $0.id == ingredient.id }
}
func updateIngredient(_ ingredient: Ingredient) {
if let index = detectedIngredients.firstIndex(where: { $0.id == ingredient.id }) {
detectedIngredients[index] = ingredient
}
}
// MARK: - Manual Entry
func addManualIngredient(name: String, quantity: String) {
let ingredient = Ingredient(
name: name,
estimatedQuantity: quantity,
confidence: 1.0
)
detectedIngredients.append(ingredient)
}
// MARK: - Cleanup
func cleanup() async {
print("📱 ScannerViewModel.cleanup() - Starting cleanup")
stopScanning()
await cameraManager.cleanup()
print("📱 ScannerViewModel.cleanup() - ✅ Cleanup complete")
}
// MARK: - Local Persistence
/// Saves ingredients locally using UserDefaults
/// TODO: Migrate to FirestoreRepository when Firebase is configured
/// To migrate: Replace this method with a call to FirestoreRepository.saveIngredients()
func saveIngredientsLocally() {
do {
let data = try JSONEncoder().encode(detectedIngredients)
UserDefaults.standard.set(data, forKey: "savedIngredients")
print("💾 Saved \(detectedIngredients.count) ingredients locally")
} catch {
print("❌ Failed to save ingredients: \(error)")
}
}
/// Loads ingredients from local storage
/// TODO: Migrate to FirestoreRepository when Firebase is configured
/// To migrate: Replace this method with a call to FirestoreRepository.loadIngredients()
func loadIngredientsLocally() {
guard let data = UserDefaults.standard.data(forKey: "savedIngredients") else {
print("📂 No saved ingredients found")
return
}
do {
detectedIngredients = try JSONDecoder().decode([Ingredient].self, from: data)
print("📂 Loaded \(detectedIngredients.count) ingredients from local storage")
} catch {
print("❌ Failed to load ingredients: \(error)")
}
}
/// Clears all saved ingredients
func clearSavedIngredients() {
detectedIngredients.removeAll()
UserDefaults.standard.removeObject(forKey: "savedIngredients")
print("🗑️ Cleared all saved ingredients")
}
}
// MARK: - Array Safe Subscript Extension
extension Collection {
/// Returns the element at the specified index if it exists, otherwise nil.
subscript(safe index: Index) -> Element? {
indices.contains(index) ? self[index] : nil
}
}

View File

@@ -1,353 +0,0 @@
//
// CookingModeView.swift
// SousChefAI
//
// Live cooking mode with AI-powered visual monitoring and guidance
//
import SwiftUI
import AVFoundation
struct CookingModeView: View {
@Environment(\.dismiss) private var dismiss
@StateObject private var viewModel: CookingModeViewModel
@State private var showingAllSteps = false
@State private var previewLayer: AVCaptureVideoPreviewLayer?
init(recipe: Recipe) {
_viewModel = StateObject(wrappedValue: CookingModeViewModel(recipe: recipe))
}
var body: some View {
NavigationStack {
ZStack {
// Camera preview background
if viewModel.isMonitoring, let previewLayer = previewLayer {
CameraPreviewView(previewLayer: previewLayer)
.ignoresSafeArea()
.opacity(0.3)
}
// Main content
VStack(spacing: 0) {
// Progress bar
progressBar
ScrollView {
VStack(spacing: 20) {
// Current step card
currentStepCard
// AI feedback card
if viewModel.isMonitoring {
aiFeedbackCard
}
// Controls
controlButtons
}
.padding()
}
}
}
.navigationTitle("Cooking Mode")
.navigationBarTitleDisplayMode(.inline)
.toolbar {
ToolbarItem(placement: .cancellationAction) {
Button("Exit") {
viewModel.cleanup()
dismiss()
}
}
ToolbarItem(placement: .primaryAction) {
Button {
showingAllSteps = true
} label: {
Label("All Steps", systemImage: "list.bullet")
}
}
}
.task {
await viewModel.setupCamera()
previewLayer = viewModel.getPreviewLayer()
viewModel.startCamera()
}
.onDisappear {
viewModel.cleanup()
}
.sheet(isPresented: $showingAllSteps) {
AllStepsSheet(
steps: viewModel.recipe.steps,
currentStep: viewModel.currentStepIndex
)
}
}
}
// MARK: - UI Components
private var progressBar: some View {
VStack(spacing: 8) {
HStack {
Text("Step \(viewModel.currentStepIndex + 1) of \(viewModel.recipe.steps.count)")
.font(.caption)
.foregroundStyle(.secondary)
Spacer()
Text("\(Int(viewModel.progress * 100))%")
.font(.caption)
.fontWeight(.semibold)
.foregroundStyle(.blue)
}
ProgressView(value: viewModel.progress)
.tint(.blue)
}
.padding()
.background(Color(.systemBackground))
}
private var currentStepCard: some View {
VStack(alignment: .leading, spacing: 16) {
HStack {
Text("Current Step")
.font(.caption)
.foregroundStyle(.secondary)
.textCase(.uppercase)
Spacer()
if viewModel.stepComplete {
Label("Complete", systemImage: "checkmark.circle.fill")
.font(.caption)
.foregroundStyle(.green)
}
}
Text(viewModel.currentStep)
.font(.title3)
.fontWeight(.semibold)
.fixedSize(horizontal: false, vertical: true)
// Speak button
Button {
viewModel.speakCurrentStep()
} label: {
Label("Read Aloud", systemImage: "speaker.wave.2.fill")
.font(.subheadline)
.foregroundStyle(.blue)
}
}
.padding()
.frame(maxWidth: .infinity, alignment: .leading)
.background(Color(.secondarySystemGroupedBackground))
.clipShape(RoundedRectangle(cornerRadius: 16))
}
private var aiFeedbackCard: some View {
VStack(alignment: .leading, spacing: 12) {
HStack {
Image(systemName: "sparkles")
.foregroundStyle(.purple)
Text("AI Assistant")
.font(.caption)
.foregroundStyle(.secondary)
.textCase(.uppercase)
Spacer()
if viewModel.confidence > 0 {
Text("\(Int(viewModel.confidence * 100))%")
.font(.caption2)
.fontWeight(.semibold)
.foregroundStyle(.white)
.padding(.horizontal, 8)
.padding(.vertical, 4)
.background(confidenceColor)
.clipShape(Capsule())
}
}
Text(viewModel.feedback)
.font(.body)
.fixedSize(horizontal: false, vertical: true)
if viewModel.isMonitoring {
HStack {
ProgressView()
.scaleEffect(0.8)
Text("Monitoring...")
.font(.caption)
.foregroundStyle(.secondary)
}
}
}
.padding()
.frame(maxWidth: .infinity, alignment: .leading)
.background(
LinearGradient(
colors: [Color.purple.opacity(0.1), Color.blue.opacity(0.1)],
startPoint: .topLeading,
endPoint: .bottomTrailing
)
)
.clipShape(RoundedRectangle(cornerRadius: 16))
}
private var controlButtons: some View {
VStack(spacing: 12) {
// AI monitoring toggle
if !viewModel.isComplete {
if viewModel.isMonitoring {
Button {
viewModel.stopMonitoring()
} label: {
Label("Stop AI Monitoring", systemImage: "eye.slash.fill")
.font(.headline)
.foregroundStyle(.white)
.frame(maxWidth: .infinity)
.padding()
.background(Color.red)
.clipShape(RoundedRectangle(cornerRadius: 12))
}
} else {
Button {
viewModel.startMonitoring()
} label: {
Label("Start AI Monitoring", systemImage: "eye.fill")
.font(.headline)
.foregroundStyle(.white)
.frame(maxWidth: .infinity)
.padding()
.background(Color.purple)
.clipShape(RoundedRectangle(cornerRadius: 12))
}
}
}
// Navigation buttons
HStack(spacing: 12) {
Button {
viewModel.previousStep()
} label: {
Label("Previous", systemImage: "arrow.left")
.frame(maxWidth: .infinity)
.padding()
.background(Color(.secondarySystemGroupedBackground))
.clipShape(RoundedRectangle(cornerRadius: 12))
}
.disabled(viewModel.currentStepIndex == 0)
if viewModel.isComplete {
Button {
viewModel.cleanup()
dismiss()
} label: {
Label("Finish", systemImage: "checkmark.circle.fill")
.font(.headline)
.foregroundStyle(.white)
.frame(maxWidth: .infinity)
.padding()
.background(Color.green)
.clipShape(RoundedRectangle(cornerRadius: 12))
}
} else {
Button {
viewModel.nextStep()
} label: {
Label("Next Step", systemImage: "arrow.right")
.frame(maxWidth: .infinity)
.padding()
.background(viewModel.stepComplete ? Color.green : Color.blue)
.foregroundStyle(.white)
.clipShape(RoundedRectangle(cornerRadius: 12))
}
}
}
}
}
private var confidenceColor: Color {
if viewModel.confidence >= 0.8 {
return .green
} else if viewModel.confidence >= 0.5 {
return .orange
} else {
return .red
}
}
}
// MARK: - All Steps Sheet
struct AllStepsSheet: View {
@Environment(\.dismiss) private var dismiss
let steps: [String]
let currentStep: Int
var body: some View {
NavigationStack {
List {
ForEach(Array(steps.enumerated()), id: \.offset) { index, step in
HStack(alignment: .top, spacing: 12) {
// Step number
Text("\(index + 1)")
.font(.headline)
.foregroundStyle(.white)
.frame(width: 32, height: 32)
.background(index == currentStep ? Color.blue : Color.gray)
.clipShape(Circle())
// Step text
VStack(alignment: .leading, spacing: 4) {
Text(step)
.font(.body)
.fixedSize(horizontal: false, vertical: true)
if index == currentStep {
Text("Current Step")
.font(.caption)
.foregroundStyle(.blue)
} else if index < currentStep {
Text("Completed")
.font(.caption)
.foregroundStyle(.green)
}
}
}
.padding(.vertical, 4)
}
}
.navigationTitle("All Steps")
.navigationBarTitleDisplayMode(.inline)
.toolbar {
ToolbarItem(placement: .confirmationAction) {
Button("Done") {
dismiss()
}
}
}
}
}
}
#Preview {
CookingModeView(recipe: Recipe(
title: "Scrambled Eggs",
description: "Simple and delicious scrambled eggs",
steps: [
"Crack 3 eggs into a bowl",
"Add a splash of milk and whisk until combined",
"Heat butter in a non-stick pan over medium heat",
"Pour eggs into the pan",
"Gently stir with a spatula until soft curds form",
"Season with salt and pepper",
"Serve immediately while hot"
],
matchScore: 0.95
))
}

View File

@@ -1,314 +0,0 @@
//
// InventoryView.swift
// SousChefAI
//
// View for reviewing and editing detected ingredients before recipe generation
//
import SwiftUI
struct InventoryView: View {
@StateObject private var repository = FirestoreRepository()
@State private var ingredients: [Ingredient]
@State private var dietaryRestrictions: Set<String> = []
@State private var nutritionGoals = ""
@State private var showingRecipeGenerator = false
@State private var showingPreferences = false
@State private var editingIngredient: Ingredient?
init(ingredients: [Ingredient]) {
_ingredients = State(initialValue: ingredients)
}
var body: some View {
List {
// Preferences Section
Section {
Button {
showingPreferences = true
} label: {
HStack {
Image(systemName: "slider.horizontal.3")
.foregroundStyle(.blue)
VStack(alignment: .leading, spacing: 4) {
Text("Dietary Preferences")
.font(.headline)
if dietaryRestrictions.isEmpty {
Text("Not set")
.font(.caption)
.foregroundStyle(.secondary)
} else {
Text(dietaryRestrictions.sorted().joined(separator: ", "))
.font(.caption)
.foregroundStyle(.secondary)
}
}
Spacer()
Image(systemName: "chevron.right")
.font(.caption)
.foregroundStyle(.tertiary)
}
}
}
// Ingredients Section
Section {
ForEach(ingredients) { ingredient in
IngredientRow(ingredient: ingredient) {
editingIngredient = ingredient
} onDelete: {
deleteIngredient(ingredient)
}
}
} header: {
HStack {
Text("Detected Ingredients")
Spacer()
Text("\(ingredients.count) items")
.font(.caption)
.foregroundStyle(.secondary)
}
} footer: {
Text("Tap an ingredient to edit quantity or remove it. Items with yellow indicators have low confidence and should be verified.")
.font(.caption)
}
}
.navigationTitle("Your Inventory")
.navigationBarTitleDisplayMode(.large)
.toolbar {
ToolbarItem(placement: .primaryAction) {
Button {
showingRecipeGenerator = true
} label: {
Label("Generate Recipes", systemImage: "sparkles")
.fontWeight(.semibold)
}
.disabled(ingredients.isEmpty)
}
}
.sheet(isPresented: $showingPreferences) {
PreferencesSheet(
dietaryRestrictions: $dietaryRestrictions,
nutritionGoals: $nutritionGoals
)
}
.sheet(item: $editingIngredient) { ingredient in
EditIngredientSheet(ingredient: ingredient) { updated in
updateIngredient(updated)
}
}
.navigationDestination(isPresented: $showingRecipeGenerator) {
RecipeGeneratorView(
inventory: ingredients,
userProfile: createUserProfile()
)
}
.task {
await loadUserPreferences()
}
}
// MARK: - Actions
private func deleteIngredient(_ ingredient: Ingredient) {
withAnimation {
ingredients.removeAll { $0.id == ingredient.id }
}
}
private func updateIngredient(_ updated: Ingredient) {
if let index = ingredients.firstIndex(where: { $0.id == updated.id }) {
ingredients[index] = updated
}
}
private func createUserProfile() -> UserProfile {
UserProfile(
dietaryRestrictions: Array(dietaryRestrictions),
nutritionGoals: nutritionGoals,
pantryStaples: []
)
}
private func loadUserPreferences() async {
if let profile = repository.currentUser {
dietaryRestrictions = Set(profile.dietaryRestrictions)
nutritionGoals = profile.nutritionGoals
}
}
}
// MARK: - Ingredient Row
struct IngredientRow: View {
let ingredient: Ingredient
let onEdit: () -> Void
let onDelete: () -> Void
var body: some View {
HStack(spacing: 12) {
// Status indicator
Circle()
.fill(ingredient.needsVerification ? Color.orange : Color.green)
.frame(width: 8, height: 8)
VStack(alignment: .leading, spacing: 4) {
Text(ingredient.name)
.font(.body)
.fontWeight(.medium)
HStack(spacing: 8) {
Text(ingredient.estimatedQuantity)
.font(.caption)
.foregroundStyle(.secondary)
if ingredient.needsVerification {
Text("• Low confidence")
.font(.caption2)
.foregroundStyle(.orange)
}
}
}
Spacer()
// Confidence badge
Text("\(Int(ingredient.confidence * 100))%")
.font(.caption2)
.fontWeight(.semibold)
.foregroundStyle(.white)
.padding(.horizontal, 8)
.padding(.vertical, 4)
.background(ingredient.needsVerification ? Color.orange : Color.green)
.clipShape(Capsule())
}
.contentShape(Rectangle())
.onTapGesture {
onEdit()
}
.swipeActions(edge: .trailing, allowsFullSwipe: true) {
Button(role: .destructive) {
onDelete()
} label: {
Label("Delete", systemImage: "trash")
}
}
}
}
// MARK: - Preferences Sheet
struct PreferencesSheet: View {
@Environment(\.dismiss) private var dismiss
@Binding var dietaryRestrictions: Set<String>
@Binding var nutritionGoals: String
var body: some View {
NavigationStack {
Form {
Section("Dietary Restrictions") {
ForEach(UserProfile.commonRestrictions, id: \.self) { restriction in
Toggle(restriction, isOn: Binding(
get: { dietaryRestrictions.contains(restriction) },
set: { isOn in
if isOn {
dietaryRestrictions.insert(restriction)
} else {
dietaryRestrictions.remove(restriction)
}
}
))
}
}
Section("Nutrition Goals") {
TextField("E.g., High protein, Low carb", text: $nutritionGoals, axis: .vertical)
.lineLimit(3...5)
}
}
.navigationTitle("Preferences")
.navigationBarTitleDisplayMode(.inline)
.toolbar {
ToolbarItem(placement: .confirmationAction) {
Button("Done") {
dismiss()
}
}
}
}
}
}
// MARK: - Edit Ingredient Sheet
struct EditIngredientSheet: View {
@Environment(\.dismiss) private var dismiss
@State private var name: String
@State private var quantity: String
let ingredient: Ingredient
let onSave: (Ingredient) -> Void
init(ingredient: Ingredient, onSave: @escaping (Ingredient) -> Void) {
self.ingredient = ingredient
self.onSave = onSave
_name = State(initialValue: ingredient.name)
_quantity = State(initialValue: ingredient.estimatedQuantity)
}
var body: some View {
NavigationStack {
Form {
Section("Ingredient Details") {
TextField("Name", text: $name)
.textInputAutocapitalization(.words)
TextField("Quantity", text: $quantity)
}
Section {
HStack {
Text("Detection Confidence")
Spacer()
Text("\(Int(ingredient.confidence * 100))%")
.foregroundStyle(.secondary)
}
}
}
.navigationTitle("Edit Ingredient")
.navigationBarTitleDisplayMode(.inline)
.toolbar {
ToolbarItem(placement: .cancellationAction) {
Button("Cancel") {
dismiss()
}
}
ToolbarItem(placement: .confirmationAction) {
Button("Save") {
var updated = ingredient
updated.name = name
updated.estimatedQuantity = quantity
onSave(updated)
dismiss()
}
.disabled(name.isEmpty || quantity.isEmpty)
}
}
}
}
}
#Preview {
NavigationStack {
InventoryView(ingredients: [
Ingredient(name: "Tomatoes", estimatedQuantity: "3 medium", confidence: 0.95),
Ingredient(name: "Cheese", estimatedQuantity: "200g", confidence: 0.65),
Ingredient(name: "Eggs", estimatedQuantity: "6 large", confidence: 0.88)
])
}
}

View File

@@ -1,391 +0,0 @@
//
// RecipeGeneratorView.swift
// SousChefAI
//
// View for generating and displaying recipe suggestions
//
import SwiftUI
struct RecipeGeneratorView: View {
@StateObject private var viewModel = RecipeGeneratorViewModel()
@State private var selectedRecipe: Recipe?
@State private var showingScaleSheet = false
let inventory: [Ingredient]
let userProfile: UserProfile
var body: some View {
Group {
if viewModel.isGenerating {
loadingView
} else if viewModel.filteredRecipes.isEmpty && !viewModel.recipes.isEmpty {
emptyFilterView
} else if viewModel.filteredRecipes.isEmpty {
emptyStateView
} else {
recipeListView
}
}
.navigationTitle("Recipe Ideas")
.navigationBarTitleDisplayMode(.large)
.toolbar {
ToolbarItem(placement: .primaryAction) {
Menu {
ForEach(RecipeFilter.allCases) { filter in
Button {
viewModel.setFilter(filter)
} label: {
Label(filter.rawValue, systemImage: filter.icon)
}
}
} label: {
Label("Filter", systemImage: viewModel.selectedFilter.icon)
}
}
}
.task {
await viewModel.generateRecipes(inventory: inventory, profile: userProfile)
}
.sheet(item: $selectedRecipe) { recipe in
RecipeDetailView(recipe: recipe) {
Task {
await viewModel.saveRecipe(recipe)
}
}
}
}
// MARK: - Views
private var loadingView: some View {
VStack(spacing: 20) {
ProgressView()
.scaleEffect(1.5)
Text("Generating recipes...")
.font(.headline)
Text("Analyzing your ingredients and preferences")
.font(.subheadline)
.foregroundStyle(.secondary)
.multilineTextAlignment(.center)
}
.padding()
}
private var emptyFilterView: some View {
ContentUnavailableView(
"No recipes match this filter",
systemImage: "line.3.horizontal.decrease.circle",
description: Text("Try selecting a different filter to see more recipes")
)
}
private var emptyStateView: some View {
ContentUnavailableView(
"No recipes generated",
systemImage: "fork.knife.circle",
description: Text("We couldn't generate recipes with your current ingredients. Try adding more items.")
)
}
private var recipeListView: some View {
ScrollView {
LazyVStack(spacing: 16) {
// Filter description
filterDescriptionBanner
// Recipe cards
ForEach(viewModel.filteredRecipes) { recipe in
RecipeCard(recipe: recipe)
.onTapGesture {
selectedRecipe = recipe
}
}
}
.padding()
}
}
private var filterDescriptionBanner: some View {
HStack {
Image(systemName: viewModel.selectedFilter.icon)
.foregroundStyle(.blue)
VStack(alignment: .leading, spacing: 2) {
Text(viewModel.selectedFilter.rawValue)
.font(.subheadline)
.fontWeight(.semibold)
Text(viewModel.selectedFilter.description)
.font(.caption)
.foregroundStyle(.secondary)
}
Spacer()
Text("\(viewModel.filteredRecipes.count)")
.font(.title3)
.fontWeight(.bold)
.foregroundStyle(.blue)
}
.padding()
.background(Color.blue.opacity(0.1))
.clipShape(RoundedRectangle(cornerRadius: 12))
}
}
// MARK: - Recipe Card
struct RecipeCard: View {
let recipe: Recipe
var body: some View {
VStack(alignment: .leading, spacing: 12) {
// Header
HStack(alignment: .top) {
VStack(alignment: .leading, spacing: 4) {
Text(recipe.title)
.font(.headline)
.lineLimit(2)
if let time = recipe.estimatedTime {
Label(time, systemImage: "clock")
.font(.caption)
.foregroundStyle(.secondary)
}
}
Spacer()
// Match score badge
VStack(spacing: 4) {
Text("\(Int(recipe.matchScore * 100))%")
.font(.title3)
.fontWeight(.bold)
.foregroundStyle(matchScoreColor)
Text("Match")
.font(.caption2)
.foregroundStyle(.secondary)
}
}
// Description
Text(recipe.description)
.font(.subheadline)
.foregroundStyle(.secondary)
.lineLimit(3)
Divider()
// Footer
HStack {
// Category badge
Label(recipe.category.rawValue, systemImage: categoryIcon)
.font(.caption)
.foregroundStyle(.white)
.padding(.horizontal, 10)
.padding(.vertical, 5)
.background(categoryColor)
.clipShape(Capsule())
Spacer()
// Missing ingredients indicator
if !recipe.missingIngredients.isEmpty {
Label("\(recipe.missingIngredients.count) missing", systemImage: "cart")
.font(.caption)
.foregroundStyle(.orange)
}
if let servings = recipe.servings {
Label("\(servings) servings", systemImage: "person.2")
.font(.caption)
.foregroundStyle(.secondary)
}
}
}
.padding()
.background(Color(.secondarySystemGroupedBackground))
.clipShape(RoundedRectangle(cornerRadius: 16))
.shadow(color: .black.opacity(0.05), radius: 5, x: 0, y: 2)
}
private var matchScoreColor: Color {
if recipe.matchScore >= 0.8 {
return .green
} else if recipe.matchScore >= 0.6 {
return .orange
} else {
return .red
}
}
private var categoryColor: Color {
switch recipe.category {
case .scavenger:
return .green
case .upgrader:
return .blue
case .shopping:
return .orange
}
}
private var categoryIcon: String {
switch recipe.category {
case .scavenger:
return "checkmark.circle.fill"
case .upgrader:
return "cart.badge.plus"
case .shopping:
return "cart.fill"
}
}
}
// MARK: - Recipe Detail View
struct RecipeDetailView: View {
@Environment(\.dismiss) private var dismiss
@State private var currentStep = 0
@State private var showingCookingMode = false
let recipe: Recipe
let onSave: () -> Void
var body: some View {
NavigationStack {
ScrollView {
VStack(alignment: .leading, spacing: 20) {
// Header
VStack(alignment: .leading, spacing: 8) {
Text(recipe.title)
.font(.title)
.fontWeight(.bold)
Text(recipe.description)
.font(.body)
.foregroundStyle(.secondary)
HStack {
if let time = recipe.estimatedTime {
Label(time, systemImage: "clock")
}
if let servings = recipe.servings {
Label("\(servings) servings", systemImage: "person.2")
}
Spacer()
Text("\(Int(recipe.matchScore * 100))% match")
.font(.subheadline)
.fontWeight(.semibold)
.foregroundStyle(.green)
}
.font(.caption)
.foregroundStyle(.secondary)
}
.padding()
.background(Color(.secondarySystemGroupedBackground))
.clipShape(RoundedRectangle(cornerRadius: 12))
// Missing ingredients
if !recipe.missingIngredients.isEmpty {
VStack(alignment: .leading, spacing: 12) {
Text("Missing Ingredients")
.font(.headline)
ForEach(recipe.missingIngredients) { ingredient in
HStack {
Image(systemName: "cart")
.foregroundStyle(.orange)
Text(ingredient.name)
Spacer()
Text(ingredient.estimatedQuantity)
.foregroundStyle(.secondary)
}
.font(.subheadline)
}
}
.padding()
.background(Color.orange.opacity(0.1))
.clipShape(RoundedRectangle(cornerRadius: 12))
}
// Cooking steps
VStack(alignment: .leading, spacing: 12) {
Text("Instructions")
.font(.headline)
ForEach(Array(recipe.steps.enumerated()), id: \.offset) { index, step in
HStack(alignment: .top, spacing: 12) {
Text("\(index + 1)")
.font(.headline)
.foregroundStyle(.white)
.frame(width: 32, height: 32)
.background(Color.blue)
.clipShape(Circle())
Text(step)
.font(.body)
.fixedSize(horizontal: false, vertical: true)
}
}
}
.padding()
.background(Color(.secondarySystemGroupedBackground))
.clipShape(RoundedRectangle(cornerRadius: 12))
// Start cooking button
Button {
showingCookingMode = true
} label: {
Label("Start Cooking", systemImage: "play.circle.fill")
.font(.headline)
.foregroundStyle(.white)
.frame(maxWidth: .infinity)
.padding()
.background(Color.green)
.clipShape(RoundedRectangle(cornerRadius: 12))
}
}
.padding()
}
.navigationBarTitleDisplayMode(.inline)
.toolbar {
ToolbarItem(placement: .cancellationAction) {
Button("Close") {
dismiss()
}
}
ToolbarItem(placement: .primaryAction) {
Button {
onSave()
} label: {
Label("Save", systemImage: "heart")
}
}
}
.sheet(isPresented: $showingCookingMode) {
CookingModeView(recipe: recipe)
}
}
}
}
#Preview {
NavigationStack {
RecipeGeneratorView(
inventory: [
Ingredient(name: "Tomatoes", estimatedQuantity: "3 medium", confidence: 0.95),
Ingredient(name: "Eggs", estimatedQuantity: "6 large", confidence: 0.88)
],
userProfile: UserProfile()
)
}
}

View File

@@ -1,462 +0,0 @@
//
// ScannerView.swift
// SousChefAI
//
// AR camera view for scanning and detecting ingredients in real-time
//
import SwiftUI
import ARKit
import RealityKit
import AVFoundation
struct ScannerView: View {
@StateObject private var viewModel = ScannerViewModel()
@State private var showingInventory = false
@State private var showingManualEntry = false
@State private var detectedPlanes = 0
@State private var lastRaycastResult = ""
@State private var showARView = false
@State private var previewLayer: AVCaptureVideoPreviewLayer?
// Banner notification state
@State private var showBanner = false
@State private var bannerIngredient: Ingredient?
@State private var bannerTask: Task<Void, Never>?
init() {
print("🔵 ScannerView.init() - View initialized at \(Date())")
}
var body: some View {
print("🔵 ScannerView.body - Body evaluated at \(Date()), showARView: \(showARView)")
return NavigationStack {
ZStack {
// AR camera preview or regular camera
if showARView {
ARViewContainer(
detectedPlanes: $detectedPlanes,
lastRaycastResult: $lastRaycastResult
)
.ignoresSafeArea()
} else {
if let previewLayer = previewLayer {
CameraPreviewView(previewLayer: previewLayer)
.ignoresSafeArea()
} else {
Color.black
.ignoresSafeArea()
.overlay {
ProgressView("Initializing camera...")
.foregroundStyle(.white)
}
}
}
// Overlay UI
VStack {
// New ingredient banner (top of screen)
if showBanner, let ingredient = bannerIngredient {
NewIngredientBanner(ingredient: ingredient)
.transition(.move(edge: .top).combined(with: .opacity))
.padding(.horizontal)
.padding(.top, 8)
}
// Top status bar
statusBar
.padding()
// AR Debug info (only when AR is active)
if showARView {
arDebugInfo
.padding(.horizontal)
}
Spacer()
// Detected ingredients list
if !viewModel.detectedIngredients.isEmpty {
detectedIngredientsOverlay
}
// Bottom controls
controlsBar
.padding()
}
}
.navigationTitle(showARView ? "AR Scanner" : "Ingredient Scanner")
.navigationBarTitleDisplayMode(.inline)
.toolbar {
ToolbarItem(placement: .navigationBarTrailing) {
Button {
showingManualEntry = true
} label: {
Image(systemName: "plus.circle")
}
}
}
.task {
print("🔵 ScannerView.task - Task started at \(Date())")
// Load any previously saved ingredients
viewModel.loadIngredientsLocally()
// Setup new ingredient notification handler
viewModel.onNewIngredientDetected = { [self] ingredient in
showNewIngredientBanner(ingredient)
}
if !showARView {
print("🔵 ScannerView.task - Calling setupCamera()")
await viewModel.setupCamera()
print("🔵 ScannerView.task - Getting preview layer after setup")
previewLayer = viewModel.getPreviewLayer()
print("🔵 ScannerView.task - Preview layer set: \(previewLayer != nil)")
print("🔵 ScannerView.task - Calling startCamera()")
viewModel.startCamera()
}
}
.onDisappear {
print("🔵 ScannerView.onDisappear - Cleaning up at \(Date())")
bannerTask?.cancel()
Task {
await viewModel.cleanup()
}
}
.onChange(of: viewModel.isScanning) { wasScanning, isScanning in
// When scanning stops, save ingredients and optionally navigate
if wasScanning && !isScanning && !viewModel.detectedIngredients.isEmpty {
viewModel.saveIngredientsLocally()
}
}
.alert("Camera Error", isPresented: .constant(viewModel.error != nil)) {
Button("OK") {
viewModel.error = nil
}
} message: {
if let error = viewModel.error {
Text(error.localizedDescription)
}
}
.sheet(isPresented: $showingManualEntry) {
ManualIngredientEntry { name, quantity in
viewModel.addManualIngredient(name: name, quantity: quantity)
}
}
.navigationDestination(isPresented: $showingInventory) {
InventoryView(ingredients: viewModel.detectedIngredients)
}
}
}
// MARK: - Banner Management
private func showNewIngredientBanner(_ ingredient: Ingredient) {
// Cancel any existing banner dismissal
bannerTask?.cancel()
// Show new banner
withAnimation(.spring(response: 0.3)) {
bannerIngredient = ingredient
showBanner = true
}
// Auto-dismiss after 1 second
bannerTask = Task { @MainActor in
try? await Task.sleep(for: .seconds(1))
guard !Task.isCancelled else { return }
withAnimation(.easeOut(duration: 0.3)) {
showBanner = false
}
}
}
// MARK: - UI Components
private var statusBar: some View {
HStack {
VStack(alignment: .leading, spacing: 4) {
Text(showARView ? "AR Mode Active" : viewModel.scanProgress)
.font(.headline)
.foregroundStyle(.white)
if viewModel.isScanning {
ProgressView()
.tint(.white)
}
}
Spacer()
Text("\(viewModel.detectedIngredients.count)")
.font(.title2)
.fontWeight(.bold)
.foregroundStyle(.white)
.padding(.horizontal, 12)
.padding(.vertical, 6)
.background(.ultraThinMaterial)
.clipShape(Capsule())
}
.padding()
.background(.ultraThinMaterial)
.clipShape(RoundedRectangle(cornerRadius: 12))
}
private var arDebugInfo: some View {
VStack(alignment: .leading, spacing: 4) {
Text("Detected Planes: \(detectedPlanes)")
.font(.caption)
.foregroundStyle(.white)
if !lastRaycastResult.isEmpty {
Text(lastRaycastResult)
.font(.caption2)
.foregroundStyle(.white)
}
}
.padding(8)
.background(.ultraThinMaterial)
.clipShape(RoundedRectangle(cornerRadius: 8))
}
private var detectedIngredientsOverlay: some View {
ScrollView(.horizontal, showsIndicators: false) {
HStack(spacing: 12) {
ForEach(viewModel.detectedIngredients.prefix(5)) { ingredient in
IngredientChip(ingredient: ingredient)
}
if viewModel.detectedIngredients.count > 5 {
Text("+\(viewModel.detectedIngredients.count - 5) more")
.font(.caption)
.foregroundStyle(.white)
.padding(.horizontal, 12)
.padding(.vertical, 8)
.background(.ultraThinMaterial)
.clipShape(Capsule())
}
}
.padding(.horizontal)
}
.padding(.bottom, 8)
}
private var controlsBar: some View {
VStack(spacing: 16) {
// AR Toggle button
if !viewModel.isScanning {
Button {
withAnimation {
showARView.toggle()
if !showARView {
Task {
await viewModel.setupCamera()
previewLayer = viewModel.getPreviewLayer()
viewModel.startCamera()
}
} else {
viewModel.stopCamera()
}
}
} label: {
Label(showARView ? "Exit AR Mode" : "Start AR Scan", systemImage: showARView ? "camera.fill" : "arkit")
.font(.headline)
.foregroundStyle(.white)
.frame(maxWidth: .infinity)
.padding()
.background(showARView ? Color.orange : Color.blue)
.clipShape(RoundedRectangle(cornerRadius: 16))
}
}
// Main scanning action button (only in non-AR mode)
if !showARView {
if viewModel.isScanning {
Button {
viewModel.stopScanning()
} label: {
Label("Stop Scanning", systemImage: "stop.circle.fill")
.font(.headline)
.foregroundStyle(.white)
.frame(maxWidth: .infinity)
.padding()
.background(Color.red)
.clipShape(RoundedRectangle(cornerRadius: 16))
}
} else {
Button {
viewModel.startScanning()
} label: {
Label("Detect Ingredients", systemImage: "camera.viewfinder")
.font(.headline)
.foregroundStyle(.white)
.frame(maxWidth: .infinity)
.padding()
.background(Color.green)
.clipShape(RoundedRectangle(cornerRadius: 16))
}
}
}
// Secondary actions
if !viewModel.detectedIngredients.isEmpty {
Button {
showingInventory = true
} label: {
Label("Continue to Inventory", systemImage: "arrow.right.circle.fill")
.font(.headline)
.foregroundStyle(.white)
.frame(maxWidth: .infinity)
.padding()
.background(Color.purple)
.clipShape(RoundedRectangle(cornerRadius: 16))
}
}
}
.padding()
.background(.ultraThinMaterial)
.clipShape(RoundedRectangle(cornerRadius: 20))
}
}
// MARK: - Camera Preview
struct CameraPreviewView: UIViewRepresentable {
let previewLayer: AVCaptureVideoPreviewLayer
func makeUIView(context: Context) -> UIView {
print("🟢 CameraPreviewView.makeUIView() - Creating preview view at \(Date())")
let view = UIView(frame: .zero)
view.backgroundColor = .black
previewLayer.frame = view.bounds
view.layer.addSublayer(previewLayer)
print("🟢 CameraPreviewView.makeUIView() - Preview layer added to view")
return view
}
func updateUIView(_ uiView: UIView, context: Context) {
DispatchQueue.main.async {
previewLayer.frame = uiView.bounds
}
}
}
// MARK: - Ingredient Chip
struct IngredientChip: View {
let ingredient: Ingredient
var body: some View {
VStack(alignment: .leading, spacing: 4) {
Text(ingredient.name)
.font(.caption)
.fontWeight(.semibold)
Text(ingredient.estimatedQuantity)
.font(.caption2)
.foregroundStyle(.secondary)
}
.foregroundStyle(.white)
.padding(.horizontal, 12)
.padding(.vertical, 8)
.background(ingredient.needsVerification ? Color.orange.opacity(0.9) : Color.green.opacity(0.9))
.clipShape(RoundedRectangle(cornerRadius: 8))
}
}
// MARK: - Manual Entry Sheet
struct ManualIngredientEntry: View {
@Environment(\.dismiss) private var dismiss
@State private var name = ""
@State private var quantity = ""
let onAdd: (String, String) -> Void
var body: some View {
NavigationStack {
Form {
Section("Ingredient Details") {
TextField("Name", text: $name)
.textInputAutocapitalization(.words)
TextField("Quantity (e.g., 2 cups, 500g)", text: $quantity)
}
}
.navigationTitle("Add Ingredient")
.navigationBarTitleDisplayMode(.inline)
.toolbar {
ToolbarItem(placement: .cancellationAction) {
Button("Cancel") {
dismiss()
}
}
ToolbarItem(placement: .confirmationAction) {
Button("Add") {
onAdd(name, quantity)
dismiss()
}
.disabled(name.isEmpty)
}
}
}
}
}
// MARK: - New Ingredient Banner
struct NewIngredientBanner: View {
let ingredient: Ingredient
var body: some View {
HStack(spacing: 12) {
Image(systemName: "plus.circle.fill")
.foregroundStyle(.white)
.font(.title2)
VStack(alignment: .leading, spacing: 2) {
Text("New Item Detected")
.font(.caption)
.foregroundStyle(.white.opacity(0.8))
Text(ingredient.name.capitalized)
.font(.headline)
.foregroundStyle(.white)
if !ingredient.estimatedQuantity.isEmpty {
Text("Qty: \(ingredient.estimatedQuantity)")
.font(.caption)
.foregroundStyle(.white.opacity(0.8))
}
}
Spacer()
// Confidence indicator
Text("\(Int(ingredient.confidence * 100))%")
.font(.caption)
.fontWeight(.semibold)
.foregroundStyle(.white)
.padding(.horizontal, 8)
.padding(.vertical, 4)
.background(.white.opacity(0.2))
.clipShape(Capsule())
}
.padding()
.background(
LinearGradient(
colors: [Color.green, Color.green.opacity(0.8)],
startPoint: .leading,
endPoint: .trailing
)
)
.clipShape(RoundedRectangle(cornerRadius: 12))
.shadow(color: .black.opacity(0.2), radius: 8, y: 4)
}
}
#Preview {
ScannerView()
}

View File

@@ -1,17 +0,0 @@
//
// SousChefAITests.swift
// SousChefAITests
//
// Created by Aditya Pulipaka on 2/11/26.
//
import Testing
@testable import SousChefAI
struct SousChefAITests {
@Test func example() async throws {
// Write your test here and use APIs like `#expect(...)` to check expected conditions.
}
}

View File

@@ -1,41 +0,0 @@
//
// SousChefAIUITests.swift
// SousChefAIUITests
//
// Created by Aditya Pulipaka on 2/11/26.
//
import XCTest
final class SousChefAIUITests: XCTestCase {
override func setUpWithError() throws {
// Put setup code here. This method is called before the invocation of each test method in the class.
// In UI tests it is usually best to stop immediately when a failure occurs.
continueAfterFailure = false
// In UI tests its important to set the initial state - such as interface orientation - required for your tests before they run. The setUp method is a good place to do this.
}
override func tearDownWithError() throws {
// Put teardown code here. This method is called after the invocation of each test method in the class.
}
@MainActor
func testExample() throws {
// UI tests must launch the application that they test.
let app = XCUIApplication()
app.launch()
// Use XCTAssert and related functions to verify your tests produce the correct results.
}
@MainActor
func testLaunchPerformance() throws {
// This measures how long it takes to launch your application.
measure(metrics: [XCTApplicationLaunchMetric()]) {
XCUIApplication().launch()
}
}
}

View File

@@ -1,33 +0,0 @@
//
// SousChefAIUITestsLaunchTests.swift
// SousChefAIUITests
//
// Created by Aditya Pulipaka on 2/11/26.
//
import XCTest
final class SousChefAIUITestsLaunchTests: XCTestCase {
override class var runsForEachTargetApplicationUIConfiguration: Bool {
true
}
override func setUpWithError() throws {
continueAfterFailure = false
}
@MainActor
func testLaunch() throws {
let app = XCUIApplication()
app.launch()
// Insert steps here to perform after app launch but before taking a screenshot,
// such as logging into a test account or navigating somewhere in the app
let attachment = XCTAttachment(screenshot: app.screenshot())
attachment.name = "Launch Screen"
attachment.lifetime = .keepAlways
add(attachment)
}
}

48
dist/assets/index-CrgoFxjd.js vendored Normal file
View File

@@ -0,0 +1,48 @@
(function(){const t=document.createElement("link").relList;if(t&&t.supports&&t.supports("modulepreload"))return;for(const i of document.querySelectorAll('link[rel="modulepreload"]'))s(i);new MutationObserver(i=>{for(const o of i)if(o.type==="childList")for(const l of o.addedNodes)l.tagName==="LINK"&&l.rel==="modulepreload"&&s(l)}).observe(document,{childList:!0,subtree:!0});function n(i){const o={};return i.integrity&&(o.integrity=i.integrity),i.referrerPolicy&&(o.referrerPolicy=i.referrerPolicy),i.crossOrigin==="use-credentials"?o.credentials="include":i.crossOrigin==="anonymous"?o.credentials="omit":o.credentials="same-origin",o}function s(i){if(i.ep)return;i.ep=!0;const o=n(i);fetch(i.href,o)}})();let g="",y="";const B=document.getElementById("login-btn"),$=document.getElementById("login-overlay"),S=document.getElementById("app-content");B.addEventListener("click",async()=>{const e=document.getElementById("login-username").value,t=document.getElementById("login-password").value,n=document.getElementById("login-btn");n.disabled=!0,n.innerText="Verifying...";try{(await fetch("/api/login",{method:"POST",headers:{"Content-Type":"application/json"},body:JSON.stringify({username:e,password:t})})).ok?(g=e,y=t,$.style.display="none",S.style.display="block"):document.getElementById("login-error").style.display="block"}catch{document.getElementById("login-error").innerText="Network error. Please try again.",document.getElementById("login-error").style.display="block"}n.disabled=!1,n.innerText="Sign in"});const r=[],v=document.getElementById("ing-wrapper"),c=document.getElementById("ing-input"),P=document.getElementById("gen-btn");P.addEventListener("click",C);v.addEventListener("click",()=>c.focus());c.addEventListener("keydown",e=>{(e.key==="Enter"||e.key===",")&&c.value.trim()?(e.preventDefault(),b(c.value.trim().replace(/,$/,"")),c.value=""):e.key==="Backspace"&&!c.value&&r.length&&O(r.length-1)});function b(e){if(!e||r.includes(e.toLowerCase()))return;r.push(e.toLowerCase());const t=document.createElement("div");t.className="tag",t.innerHTML=e+'<span class="tag-x">×</span>',t.querySelector(".tag-x").addEventListener("click",n=>{n.stopPropagation(),M(e.toLowerCase())}),v.insertBefore(t,c)}function M(e){const t=r.indexOf(e);t>-1&&(r.splice(t,1),I())}function O(e){r.splice(e,1),I()}function I(){v.querySelectorAll(".tag").forEach(t=>t.remove());const e=[...r];r.length=0,e.forEach(t=>b(t))}document.querySelectorAll(".check-pill").forEach(e=>{e.addEventListener("click",()=>{const t=e.querySelector("input");t.checked=!t.checked,e.classList.toggle("checked",t.checked)})});async function C(){const e=document.getElementById("goal").value.trim();if(!e&&r.length===0){E("Please add some ingredients or describe what you want to make.");return}const t=[...document.querySelectorAll(".check-pill input:checked")].map(a=>a.value),n=document.getElementById("cuisine").value,s=document.getElementById("time").value,i=document.getElementById("skill").value,o=document.getElementById("servings").value,l=document.getElementById("notes").value.trim(),d=document.getElementById("gen-btn");d.disabled=!0,d.innerHTML='<span class="spinner"></span>Generating…',document.getElementById("error-area").innerHTML="",document.getElementById("recipe-area").innerHTML='<div class="status">Crafting your recipe with care…</div>';const L=[r.length?`Ingredients available: ${r.join(", ")}`:"",e?`Food goal: ${e}`:"",t.length?`Dietary restrictions: ${t.join(", ")}`:"",n?`Preferred cuisine: ${n}`:"",s?`Time constraint: ${s}`:"",i?`Skill level: ${i}`:"",`Servings: ${o}`,l?`Additional preferences: ${l}`:""].filter(Boolean).join(`
`),T=`You are an expert chef and culinary writer. Create a complete, precise recipe based on the user's inputs.
CRITICAL: Instructions must be specific — never vague. Include exact temperatures, times, visual cues, sounds, smells, and textures. Example: instead of "cook the onions", write "cook the onions in the butter over medium heat, stirring occasionally, for 810 minutes until they turn translucent and just begin to turn golden at the edges."
Respond with ONLY a valid JSON object. Use this exact structure:
{"title":"string","description":"string","servings":2,"prepTime":"string","cookTime":"string","ingredients":[{"amount":"string","unit":"string","name":"string"}],"steps":[{"title":"string","instruction":"string"}],"notes":"string or null"}
Rules: use provided ingredients as the base, add pantry staples as needed, 510 steps, each instruction 13 richly detailed sentences, respect all dietary restrictions strictly.`;try{const a=await fetch("/api/generate",{method:"POST",headers:{"Content-Type":"application/json"},body:JSON.stringify({username:g,password:y,systemPrompt:T,userPrompt:L})}),w=await a.json();if(!a.ok)throw new Error(w.error||`Server error ${a.status}`);const k=w.candidates[0].content.parts[0].text,x=JSON.parse(k);f(x)}catch(a){document.getElementById("recipe-area").innerHTML="",E("Error: "+(a.message||"Unknown error. Please try again."))}d.disabled=!1,d.innerHTML="Generate my recipe"}function E(e){document.getElementById("error-area").innerHTML=`<div class="error-msg">${e}</div>`}let u=null,m=null;const h=document.getElementById("saved-menu"),p=document.getElementById("saved-list");document.getElementById("open-menu-btn").addEventListener("click",async()=>{h.classList.add("open"),await R()});document.getElementById("close-menu-btn").addEventListener("click",()=>{h.classList.remove("open")});async function R(){p.innerHTML='<p style="font-size: 13px; color: var(--color-text-secondary); text-align: center;">Loading...</p>';try{const e=await fetch("/api/recipes",{headers:{"x-username":g,"x-password":y}});if(!e.ok)throw new Error("Failed to fetch");const t=await e.json();if(t.length===0){p.innerHTML='<p style="font-size: 13px; color: var(--color-text-secondary); text-align: center;">No saved recipes yet.</p>';return}p.innerHTML=t.reverse().map(n=>`
<div class="saved-recipe-item" onclick="loadSavedRecipe('${n.id}')">
<h3>${n.title}</h3>
<p>${n.description}</p>
</div>
`).join("")}catch{p.innerHTML='<p style="font-size: 13px; color: var(--color-text-danger); text-align: center;">Error loading recipes.</p>'}}window.loadSavedRecipe=async function(e){h.classList.remove("open");try{const t=await fetch(`/api/recipes/${e}`);if(!t.ok)throw new Error("Failed to fetch recipe");const n=await t.json();n.id=e,f(n),document.getElementById("recipe-area").scrollIntoView({behavior:"smooth"})}catch(t){console.error(t)}};window.saveCurrentRecipe=async function(){if(!u)return;const e=document.getElementById("save-recipe-btn"),t=e.innerText;e.innerText="Saving...",e.disabled=!0;try{const n=await fetch("/api/recipes",{method:"POST",headers:{"Content-Type":"application/json"},body:JSON.stringify({username:g,password:y,recipe:u})});if(n.ok){const s=await n.json();m=s.id,u.id=s.id,e.innerText="Saved!"}else throw new Error("Save failed")}catch{e.innerText="Error Saving",setTimeout(()=>{e.innerText=t,e.disabled=!1},2e3)}};window.shareRecipeLink=function(){if(!m){alert("Please save the recipe first to share it!");return}const e=`${window.location.origin}${window.location.pathname}#shared=${m}`;navigator.clipboard.writeText(e).then(()=>{const t=document.getElementById("share-recipe-btn"),n=t.innerText;t.innerText="Link Copied!",setTimeout(()=>t.innerText=n,2e3)})};function f(e,t=!1){u=e,m=e.id||null;const n=(e.ingredients||[]).map(i=>`<li><span class="ing-amount">${i.amount}${i.unit?" "+i.unit:""}</span><span class="ing-name">${i.name}</span></li>`).join(""),s=(e.steps||[]).map((i,o)=>`<li class="step-item">
<span class="step-num">${o+1}</span>
<div>
<div class="step-title">${i.title}</div>
<div class="step-instr">${i.instruction}</div>
</div>
</li>`).join("");document.getElementById("recipe-area").innerHTML=`
<div class="recipe-out">
<div class="recipe-title">${e.title}</div>
<div class="recipe-desc">${e.description}</div>
<div class="meta-row">
<div class="meta-chip"><strong>${e.servings}</strong> servings</div>
<div class="meta-chip">Prep <strong>${e.prepTime}</strong></div>
<div class="meta-chip">Cook <strong>${e.cookTime}</strong></div>
</div>
<div class="section-label">Ingredients</div>
<ul class="ingredients-list">${n}</ul>
<div class="section-label">Method</div>
<ol class="steps-list">${s}</ol>
${e.notes?`<div class="notes-box">${e.notes}</div>`:""}
<div class="recipe-actions">
${t?`
<button class="action-btn" style="background: var(--color-text-primary); color: white;" onclick="window.location.href=window.location.pathname">Return to Login</button>
`:`
<button class="action-btn" id="save-recipe-btn" onclick="saveCurrentRecipe()">Save to Menu</button>
<button class="action-btn" id="share-recipe-btn" onclick="shareRecipeLink()">Copy Share Link</button>
`}
</div>
</div>`}window.addEventListener("DOMContentLoaded",async()=>{if(window.location.hash.startsWith("#shared="))try{const e=window.location.hash.slice(8);document.getElementById("login-overlay").innerText="Loading Shared Recipe...",document.getElementById("login-overlay").style.color="#1a1a1a";const t=await fetch(`/api/recipes/${e}`);if(!t.ok)throw new Error("Shared recipe not found");const n=await t.json();n.id=e,document.getElementById("login-overlay").style.display="none",document.querySelectorAll("#open-menu-btn, .header, .form-grid, #gen-btn, #error-area").forEach(s=>{s&&(s.style.display="none")}),document.getElementById("app-content").style.display="block",f(n,!0)}catch(e){console.error("Failed to load shared recipe",e),document.getElementById("login-overlay").innerHTML=`
<div class="login-card" style="text-align:center;">
<h2 style="font-family: 'Lora', serif; font-size: 20px;">Recipe Not Found</h2>
<p style="font-size: 14px; margin-top: 10px;">The shared recipe link appears to be invalid or has been deleted.</p>
<button class="generate-btn" style="margin-top: 16px;" onclick="window.location.href=window.location.pathname">Return to Login</button>
</div>
`}});

282
dist/index.html vendored Normal file
View File

@@ -0,0 +1,282 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>Recipe Generator</title>
<style>
@import url('https://fonts.googleapis.com/css2?family=Lora:ital,wght@0,400;0,600;1,400&family=DM+Sans:wght@300;400;500&display=swap');
*, *::before, *::after { box-sizing: border-box; margin: 0; padding: 0; }
:root {
--color-text-primary: #1a1a1a;
--color-text-secondary: #6b6b6b;
--color-background-primary: #ffffff;
--color-background-secondary: #f5f5f4;
--color-background-danger: #fef2f2;
--color-text-danger: #b91c1c;
--color-border-primary: #a3a3a3;
--color-border-secondary: #d4d4d4;
--color-border-tertiary: #e5e5e5;
--border-radius-md: 8px;
--border-radius-lg: 12px;
}
body {
font-family: 'DM Sans', sans-serif;
background: var(--color-background-primary);
color: var(--color-text-primary);
padding: 2rem;
display: flex;
justify-content: center;
}
.app { display: none; width: 100%; max-width: 680px; padding: 2rem 0; }
#login-overlay { position: fixed; top: 0; left: 0; width: 100vw; height: 100vh; background: var(--color-background-secondary); display: flex; align-items: center; justify-content: center; z-index: 1000; }
.login-card { background: var(--color-background-primary); padding: 2rem; border: 0.5px solid var(--color-border-secondary); border-radius: var(--border-radius-lg); width: 100%; max-width: 320px; box-shadow: 0 4px 12px rgba(0,0,0,0.05); }
.login-card h2 { font-family: 'Lora', serif; font-size: 24px; margin-bottom: 1.25rem; text-align: center; }
.header { margin-bottom: 2rem; border-bottom: 0.5px solid var(--color-border-tertiary); padding-bottom: 1.25rem; }
.header h1 { font-family: 'Lora', serif; font-size: 26px; font-weight: 600; letter-spacing: -0.5px; margin-bottom: 4px; }
.header p { font-size: 14px; color: var(--color-text-secondary); font-weight: 300; }
.form-grid { display: grid; grid-template-columns: 1fr 1fr; gap: 14px; margin-bottom: 14px; }
.form-field { display: flex; flex-direction: column; gap: 6px; }
.form-field.full { grid-column: 1 / -1; }
label { font-size: 12px; font-weight: 500; color: var(--color-text-secondary); letter-spacing: 0.04em; text-transform: uppercase; }
input, textarea, select {
font-family: 'DM Sans', sans-serif; font-size: 14px; padding: 9px 12px;
border: 0.5px solid var(--color-border-secondary); border-radius: var(--border-radius-md);
background: var(--color-background-primary); color: var(--color-text-primary);
width: 100%; transition: border-color 0.15s; outline: none;
}
input:focus, textarea:focus, select:focus { border-color: var(--color-border-primary); }
textarea { resize: vertical; min-height: 72px; line-height: 1.5; }
.tags-input-wrapper {
border: 0.5px solid var(--color-border-secondary); border-radius: var(--border-radius-md);
background: var(--color-background-primary); padding: 6px 8px;
display: flex; flex-wrap: wrap; gap: 6px; align-items: center;
cursor: text; min-height: 40px; transition: border-color 0.15s;
}
.tags-input-wrapper:focus-within { border-color: var(--color-border-primary); }
.tag { background: var(--color-background-secondary); border: 0.5px solid var(--color-border-tertiary); border-radius: 4px; padding: 2px 8px; font-size: 13px; display: flex; align-items: center; gap: 5px; white-space: nowrap; }
.tag-x { cursor: pointer; color: var(--color-text-secondary); font-size: 15px; line-height: 1; }
.tag-x:hover { color: var(--color-text-primary); }
.tags-input { border: none !important; padding: 2px 4px !important; flex: 1; min-width: 100px; font-size: 14px; background: transparent !important; outline: none !important; }
.checkboxes { display: flex; flex-wrap: wrap; gap: 8px; }
.check-pill { display: flex; align-items: center; gap: 6px; padding: 5px 12px; border: 0.5px solid var(--color-border-secondary); border-radius: 20px; font-size: 13px; cursor: pointer; transition: all 0.12s; background: var(--color-background-primary); user-select: none; }
.check-pill input { display: none; }
.check-pill.checked { background: var(--color-background-secondary); border-color: var(--color-border-primary); font-weight: 500; }
.dot { width: 7px; height: 7px; border-radius: 50%; background: var(--color-border-secondary); flex-shrink: 0; }
.check-pill.checked .dot { background: var(--color-text-primary); }
.generate-btn { width: 100%; padding: 11px; background: var(--color-text-primary); color: var(--color-background-primary); border: none; border-radius: var(--border-radius-md); font-family: 'DM Sans', sans-serif; font-size: 15px; font-weight: 500; cursor: pointer; margin-top: 8px; transition: opacity 0.15s; }
.generate-btn:hover { opacity: 0.85; }
.generate-btn:disabled { opacity: 0.45; cursor: not-allowed; }
.status { text-align: center; padding: 2rem 0; color: var(--color-text-secondary); font-size: 14px; font-style: italic; }
.recipe-out { margin-top: 2.5rem; border-top: 0.5px solid var(--color-border-tertiary); padding-top: 2rem; animation: fadein 0.4s ease; }
@keyframes fadein { from { opacity: 0; transform: translateY(8px); } to { opacity: 1; transform: translateY(0); } }
.recipe-actions { display: flex; gap: 10px; margin-top: 1.5rem; }
.action-btn { flex: 1; padding: 10px; background: var(--color-background-primary); border: 0.5px solid var(--color-border-primary); border-radius: var(--border-radius-md); font-family: 'DM Sans', sans-serif; font-size: 14px; font-weight: 500; cursor: pointer; transition: all 0.15s; }
.action-btn:hover { background: var(--color-background-secondary); }
.saved-menu-btn { position: fixed; top: 20px; right: 20px; padding: 8px 16px; background: var(--color-text-primary); color: white; border: none; border-radius: 20px; font-size: 13px; font-weight: 500; cursor: pointer; z-index: 100; box-shadow: 0 2px 8px rgba(0,0,0,0.15); }
#saved-menu { position: fixed; top: 0; right: -350px; width: 320px; height: 100vh; background: var(--color-background-primary); box-shadow: -4px 0 15px rgba(0,0,0,0.05); transition: right 0.3s ease; z-index: 200; padding: 2rem 1.5rem; overflow-y: auto; border-left: 1px solid var(--color-border-tertiary); }
#saved-menu.open { right: 0; }
.saved-menu-header { display: flex; justify-content: space-between; align-items: center; margin-bottom: 1.5rem; }
.saved-menu-header h2 { font-family: 'Lora', serif; font-size: 20px; }
.close-menu { cursor: pointer; font-size: 24px; color: var(--color-text-secondary); line-height: 1; background: none; border: none; }
.saved-recipe-item { padding: 12px; border: 0.5px solid var(--color-border-secondary); border-radius: var(--border-radius-md); margin-bottom: 10px; cursor: pointer; transition: border-color 0.15s; }
.saved-recipe-item:hover { border-color: var(--color-text-primary); }
.saved-recipe-item h3 { font-size: 15px; margin-bottom: 4px; }
.saved-recipe-item p { font-size: 12px; color: var(--color-text-secondary); white-space: nowrap; overflow: hidden; text-overflow: ellipsis; }
.recipe-title { font-family: 'Lora', serif; font-size: 28px; font-weight: 600; letter-spacing: -0.5px; margin-bottom: 6px; }
.recipe-desc { font-size: 14px; color: var(--color-text-secondary); font-style: italic; margin-bottom: 1.25rem; line-height: 1.6; }
.meta-row { display: flex; gap: 12px; flex-wrap: wrap; margin-bottom: 1.75rem; }
.meta-chip { background: var(--color-background-secondary); border-radius: var(--border-radius-md); padding: 6px 14px; font-size: 13px; color: var(--color-text-secondary); }
.meta-chip strong { color: var(--color-text-primary); font-weight: 500; }
.section-label { font-size: 11px; font-weight: 500; letter-spacing: 0.08em; text-transform: uppercase; color: var(--color-text-secondary); margin-bottom: 10px; margin-top: 1.5rem; }
.ingredients-list { list-style: none; border: 0.5px solid var(--color-border-tertiary); border-radius: var(--border-radius-lg); overflow: hidden; }
.ingredients-list li { padding: 9px 16px; font-size: 14px; border-bottom: 0.5px solid var(--color-border-tertiary); display: flex; gap: 10px; align-items: baseline; }
.ingredients-list li:last-child { border-bottom: none; }
.ing-amount { font-weight: 500; min-width: 60px; }
.ing-name { color: var(--color-text-secondary); }
.steps-list { list-style: none; }
.step-item { display: flex; gap: 16px; margin-bottom: 1.25rem; }
.step-num { font-family: 'Lora', serif; font-size: 18px; font-weight: 600; color: var(--color-text-secondary); min-width: 28px; line-height: 1.5; }
.step-title { font-weight: 500; font-size: 14px; margin-bottom: 4px; }
.step-instr { font-size: 14px; line-height: 1.7; color: var(--color-text-secondary); }
.notes-box { background: var(--color-background-secondary); border-radius: 0 var(--border-radius-md) var(--border-radius-md) 0; padding: 12px 16px; font-size: 14px; line-height: 1.6; color: var(--color-text-secondary); margin-top: 1.5rem; border-left: 2px solid var(--color-border-primary); }
.error-msg { background: var(--color-background-danger); color: var(--color-text-danger); border-radius: var(--border-radius-md); padding: 12px 16px; font-size: 13px; margin-top: 1rem; line-height: 1.5; }
.spinner { display: inline-block; width: 16px; height: 16px; border: 2px solid rgba(255,255,255,0.35); border-top-color: #fff; border-radius: 50%; animation: spin 0.7s linear infinite; vertical-align: middle; margin-right: 6px; }
@keyframes spin { to { transform: rotate(360deg); } }
</style>
<script type="module" crossorigin src="/assets/index-CrgoFxjd.js"></script>
</head>
<body>
<div id="login-overlay">
<div class="login-card">
<h2 style="font-family: 'Lora', serif; font-size: 26px; font-weight: 600; letter-spacing: -0.5px; margin-bottom: 20px;">Welcome</h2>
<div class="form-field" style="margin-bottom: 12px;">
<label>Username</label>
<input type="text" id="login-username" />
</div>
<div class="form-field" style="margin-bottom: 12px;">
<label>Password</label>
<input type="password" id="login-password" />
</div>
<button class="generate-btn" id="login-btn" style="margin-top: 12px;">Sign in</button>
<div id="login-error" style="color: var(--color-text-danger); font-size: 13px; margin-top: 10px; text-align: center; display: none;">Invalid credentials</div>
</div>
</div>
<div class="app" id="app-content">
<button class="saved-menu-btn" id="open-menu-btn">View Saved Recipes</button>
<div id="saved-menu">
<div class="saved-menu-header">
<h2>Saved Recipes</h2>
<button class="close-menu" id="close-menu-btn">×</button>
</div>
<div id="saved-list">
<!-- Saved items go here -->
</div>
</div>
<div class="header">
<h1>Recipe Generator</h1>
<p>Tell the AI what you have and what you want — it handles the rest.</p>
</div>
<div class="form-grid">
<div class="form-field full">
<label>Ingredients on hand</label>
<div class="tags-input-wrapper" id="ing-wrapper">
<input class="tags-input" id="ing-input" placeholder="Type an ingredient, press Enter..." />
</div>
</div>
<div class="form-field full">
<label>What do you want to make?</label>
<input type="text" id="goal" placeholder="e.g. a hearty weeknight pasta, a light summer salad..." />
</div>
<div class="form-field full">
<label>Dietary restrictions</label>
<div class="checkboxes">
<label class="check-pill"><input type="checkbox" value="vegan"><span class="dot"></span>Vegan</label>
<label class="check-pill"><input type="checkbox" value="vegetarian"><span class="dot"></span>Vegetarian</label>
<label class="check-pill"><input type="checkbox" value="gluten-free"><span class="dot"></span>Gluten-free</label>
<label class="check-pill"><input type="checkbox" value="dairy-free"><span class="dot"></span>Dairy-free</label>
<label class="check-pill"><input type="checkbox" value="nut-free"><span class="dot"></span>Nut-free</label>
<label class="check-pill"><input type="checkbox" value="low-carb"><span class="dot"></span>Low-carb</label>
</div>
</div>
<div class="form-field">
<label>Cuisine style</label>
<select id="cuisine">
<option value="">Any cuisine</option>
<option>Italian</option><option>Mexican</option><option>Asian fusion</option>
<option>Mediterranean</option><option>French</option><option>Indian</option>
<option>American</option><option>Middle Eastern</option><option>Japanese</option><option>Thai</option>
</select>
</div>
<div class="form-field">
<label>Time available</label>
<select id="time">
<option value="">No limit</option>
<option value="under 20 minutes">Under 20 minutes</option>
<option value="under 30 minutes">Under 30 minutes</option>
<option value="under 45 minutes">Under 45 minutes</option>
<option value="under 1 hour">Under 1 hour</option>
<option value="12 hours">12 hours</option>
</select>
</div>
<div class="form-field">
<label>Skill level</label>
<select id="skill">
<option value="">Any level</option>
<option value="beginner-friendly">Beginner-friendly</option>
<option value="intermediate">Intermediate</option>
<option value="advanced">Advanced chef techniques</option>
</select>
</div>
<div class="form-field">
<label>Servings</label>
<select id="servings">
<option value="1">1</option><option value="2" selected>2</option>
<option value="4">4</option><option value="6">6</option><option value="8">8</option>
</select>
</div>
<div class="form-field full">
<label>Any other notes or preferences</label>
<textarea id="notes" placeholder="e.g. my kids hate mushrooms, I want it spicy, make it impressive enough for guests..."></textarea>
</div>
</div>
<button class="generate-btn" id="gen-btn">Generate my recipe</button>
<div id="error-area"></div>
<div id="recipe-area"></div>
</div>
</body>
</html>

14
docker-compose.yml Normal file
View File

@@ -0,0 +1,14 @@
services:
app:
build: .
restart: unless-stopped
ports:
- "8080:3000"
env_file:
- .env
volumes:
- ./recipes:/app/recipes
deploy:
resources:
limits:
memory: 128M

468
implementation_plan.md Normal file
View File

@@ -0,0 +1,468 @@
# Reel Recipe App — Implementation Plan
This plan covers a native iOS app that turns shared Instagram Reels and TikToks into structured, searchable recipes, backed by a Python/FastAPI worker running on your apartment desktop. Voice search and dietary intelligence are first-class features alongside reel ingestion.
---
## 1. Scope & Goals
### In scope for v1
- iOS app (SwiftUI, iOS 18+)
- Share extension accepting reels/TikToks from Instagram, TikTok, and any URL sharing surface
- Video-understanding pipeline: caption parsing → ASR → frame OCR → VLM → fusion LLM
- Voice search with on-device Whisper transcription, backend web search + LLM structuring
- Dietary preferences and allergies with per-ingredient swap suggestions on violation
- Apple Sign In authentication
- Self-hosted Postgres on apartment desktop, accessed via Cloudflare Tunnel
### Out of scope for v1
- Android
- Fridge scanning / computer vision of ingredients (carried over design language only from the SousChefAI codebase)
- Full recipe regeneration on dietary conflict (ingredient swaps only for now)
- Community / shared recipe features
- Grocery list integrations beyond Apple Reminders
### Non-goals / philosophy
- Do not scrape Instagram or TikTok from the server. All ingestion is user-initiated through the share sheet. Backend may fetch the public video via yt-dlp, but only after the user has explicitly shared it.
- Do not embed third-party API keys in the iOS binary. All LLM/VLM inference runs on the backend.
- Do not require a network round-trip for things that can reasonably happen on-device (voice transcription for search, UI state, cache).
---
## 2. Architecture Overview
```
┌─────────────────────────────────────────────┐
│ iOS App (iOS 18+) │
│ ┌────────────────┐ ┌─────────────────┐ │
│ │ Share Extension│ │ Main App │ │
│ │ (thin, <50MB) │ │ (SwiftUI) │ │
│ └────────┬───────┘ └────────┬────────┘ │
│ │ │ │
│ ┌────────┴───────────────────┴────────┐ │
│ │ App Group: pending-jobs, cache │ │
│ └────────────────┬────────────────────┘ │
│ │ │
│ ┌────────────────┴────────────────────┐ │
│ │ WhisperKit (on-device, voice only) │ │
│ └─────────────────────────────────────┘ │
└───────────────────┬─────────────────────────┘
│ HTTPS (Cloudflare Tunnel)
│ Bearer JWT
┌───────────────────┴─────────────────────────┐
│ Apartment Desktop (Ubuntu/WSL2) │
│ ┌─────────────────────────────────────┐ │
│ │ FastAPI app (uvicorn) │ │
│ │ • /auth /me /jobs /recipes │ │
│ │ • Pushes jobs onto arq queue │ │
│ └────────────┬────────────────────────┘ │
│ │ │
│ ┌────────────┴────────────┐ │
│ │ Redis (job queue) │ │
│ └────────────┬────────────┘ │
│ │ │
│ ┌────────────┴────────────────────┐ │
│ │ arq Worker(s) │ │
│ │ • yt-dlp / ffmpeg │ │
│ │ • faster-whisper (GPU) │ │
│ │ • PySceneDetect + PaddleOCR │ │
│ │ • Qwen2.5-VL-7B (GPU) │ │
│ │ • Qwen2.5-14B fusion (GPU) │ │
│ └────────────┬────────────────────┘ │
│ │ │
│ ┌────────────┴────────────────────┐ │
│ │ Postgres 16 + pgvector │ │
│ └─────────────────────────────────┘ │
└─────────────────────────────────────────────┘
```
The share extension is deliberately tiny: it reads the shared URL and any caption iOS hands over, writes a pending-job record into the App Group container, posts it to the backend, and exits. It never downloads the video, never runs ML, never exceeds ~50MB of memory. iOS's 120-ish-MB share extension ceiling is not a constraint here because we designed around it.
The main app is the primary surface. It reads pending jobs from the App Group on launch, polls job status, renders recipes, handles voice search, and manages preferences. The main app has no meaningful memory limit.
The desktop backend does all heavy work. Two processes: the FastAPI request handler (lightweight, non-blocking) and one or more arq workers that pull jobs from Redis and run the video pipeline. This split matters — you don't want Whisper blocking a health check.
---
## 3. Tech Stack
**iOS (SwiftUI, iOS 18+)**
- SwiftUI with the iOS 18 Liquid Glass materials for the polished card UI
- Apple Sign In for auth
- URLSession + async/await for networking
- SwiftData for local cache of recipes (read-through, write-through to backend)
- Share Extension target sharing an App Group container with the main app
- WhisperKit for on-device voice transcription (voice search only; not used for reels)
- APNs for job-ready notifications
**Backend (Python 3.12)**
- FastAPI with uvicorn
- SQLAlchemy 2.x async + Alembic migrations
- Postgres 16 with pgvector (for future semantic recipe search)
- Redis 7 + arq for job queue
- PyJWT + cryptography for Apple identity token verification and own session JWT issuance
- Pydantic v2 for schemas
- httpx for outbound HTTP
**ML / media stack (all on the desktop 3060 Ti + WSL2 Ubuntu)**
- yt-dlp (pinned, with a scheduled pull for updates)
- ffmpeg
- faster-whisper (large-v3-turbo, CTranslate2 backend)
- PySceneDetect for keyframe selection
- PaddleOCR (GPU) for on-screen text extraction
- Qwen2.5-VL-7B-Instruct for frame captioning (via transformers or vLLM)
- Qwen2.5-14B-Instruct for the fusion step and dietary swap suggestions
- Escape hatch: config flag to route fusion to Gemini Flash if local model is struggling or machine is offline
**Infrastructure**
- Cloudflare Tunnel to apartment desktop (existing setup)
- GitHub Actions for CI (tests, lint, migration linting)
- Eventually: Oracle Free ARM VM as a WireGuard relay (matches your existing roadmap)
---
## 4. iOS Application
### 4.1 Share Extension
The extension has exactly one job: take the URL the user shared, post it to the backend, and get out of the way.
Flow on share:
1. Read the shared NSExtensionItem; extract the URL and any caption text iOS provides
2. Check that a user session JWT exists in the App Group Keychain. If not, show an "open the app to sign in" message and bail
3. POST `/jobs/ingest-reel` with `{source_url, caption?}` and the bearer token
4. Show a brief "Recipe queued" confirmation. Dismiss
5. If the network call fails, write the job to the App Group's `pending-jobs` directory. The main app will retry on next launch
No ML, no heavy dependencies, no video processing. This keeps the extension well under the memory limit and makes the share sheet experience feel instant.
### 4.2 Main App
Five primary surfaces:
**Home / Inbox.** Shows recipes with their processing status. A just-shared recipe appears immediately as "Processing…" with a shimmer, then transitions to a full card when the backend finishes. This is where the liquid-glass card styling lives — frosted backgrounds, subtle depth, match-score badges carried over from SousChefAI.
**Recipe Detail.** Title, description, ingredients list with provenance badges ("heard in narration", "seen on-screen", "from caption"), step-by-step instructions, servings/time metadata, missing-ingredient section if any, and any dietary flags inline. If the user has an allergy to something in the recipe, that ingredient appears with a red badge and a suggested swap right next to it.
**Voice Search.** A prominent mic button opens a full-screen capture surface. WhisperKit transcribes on-device as the user speaks, showing interim results. On confirmation, the transcript goes to `/jobs/voice-search` and the user sees a liquid-glass grid of structured results streaming in.
**Saved / Library.** Their personal collection. Filter by dietary match, by source (reel vs. voice vs. manual), by time.
**Profile / Preferences.** Apple Sign In account info, dietary restrictions, allergies (explicitly separated from preferences in the UI — allergies get an "important safety info" framing, preferences are soft), nutrition goals, pantry staples.
### 4.3 On-Device Services
Only two pieces of real logic run on the device:
**WhisperKit transcription for voice search.** Model downloaded on first use (Whisper-small distilled, ~150MB — usable quality for short queries, modest size). Runs locally, no audio leaves the device for voice search.
**SwiftData local cache.** Read-through cache of the user's recipes so the app opens to populated content even when the backend is unreachable. Writes go to the backend first; local cache updates on success.
### 4.4 Service Layer (Protocol-Based)
Carry over the protocol-based architecture from the SousChefAI codebase. `RecipeService`, `AuthService`, `JobService`, `VoiceSearchService` are protocols. Concrete implementations hit the backend. Mock implementations are used in previews and tests. This was already done well in the starter code — the Gemini-direct services get removed and replaced with thin clients that hit `/recipes`, `/jobs`, etc.
---
## 5. Backend (FastAPI)
### 5.1 Process Model
Two long-running processes:
- **API process** — `uvicorn app.main:app`. Handles HTTP. Does not do long-running work; every heavy operation is enqueued.
- **Worker process(es)** — `arq app.worker.WorkerSettings`. Pulls jobs from Redis, runs the video pipeline. Start with one worker; adjust concurrency based on GPU utilization.
Both run under systemd on the desktop. Redis runs as a third systemd unit. Postgres is also systemd-managed on the same box.
### 5.2 Module Layout
```
app/
main.py # FastAPI app factory, middleware
config.py # Pydantic settings
db.py # SQLAlchemy engine, session
auth/
apple.py # Verify Apple identity tokens
jwt.py # Issue/verify own session JWTs
deps.py # FastAPI dependencies (current_user, etc)
models/ # SQLAlchemy models
schemas/ # Pydantic request/response schemas
routers/
auth.py
me.py
jobs.py
recipes.py
services/
ingest.py # High-level reel ingestion orchestration
voice_search.py
dietary.py # Violation detection + swap suggestion
pipeline/
download.py # yt-dlp wrapper
audio.py # ffmpeg + faster-whisper
frames.py # PySceneDetect + OCR
vlm.py # Qwen2.5-VL client
fusion.py # Final structured-recipe generation
worker.py # arq worker settings + job functions
migrations/ # Alembic
tests/
```
### 5.3 API Surface
All endpoints require a Bearer session JWT except `/auth/apple` and health checks.
```
POST /auth/apple Exchange Apple identity token → session JWT + refresh
POST /auth/refresh Refresh session JWT
GET /me Current user + preferences
PUT /me/preferences Update dietary restrictions, allergies, goals
POST /jobs/ingest-reel Body: {source_url, caption?}. Returns job_id.
POST /jobs/voice-search Body: {query}. Returns job_id.
POST /jobs/scale-recipe Body: {recipe_id, limiting_ingredient, quantity}
GET /jobs/{id} Poll job status
GET /recipes List current user's recipes (paginated)
GET /recipes/{id} Recipe detail with ingredients + steps
PUT /recipes/{id} Edit a recipe
DELETE /recipes/{id}
POST /recipes/{id}/save Explicit save to library (vs. inbox)
```
Job-based endpoints always return quickly with a `job_id`. The client polls (or receives an APNs push) and then fetches the final object. This pattern matters because the video pipeline can take 20-40 seconds and you can't hold a connection open that long reliably through Cloudflare Tunnel.
### 5.4 Auth Flow (Apple Sign In)
1. iOS app performs Apple Sign In, receives an identity token (a JWT signed by Apple)
2. App POSTs the token to `/auth/apple`
3. Backend verifies the token: fetches Apple's public keys from `appleid.apple.com/auth/keys`, validates signature, checks audience (your app's bundle ID) and expiration
4. Extract the stable `sub` (Apple's opaque user ID). Look up or create a `users` row keyed on `apple_user_id`
5. Issue a session JWT (15 min) and a refresh token (30 days). Return both
6. Client stores them in the App Group Keychain so the share extension can access them
The session JWT contains `user_id` and `exp`. Every subsequent request is authenticated against it. The refresh endpoint issues a new session JWT given a valid refresh token.
Important: the share extension does not initiate sign-in. If there's no valid JWT when a user shares, the extension tells them to open the main app once. This is fine — they'll have signed in during onboarding.
---
## 6. Database Schema
```sql
CREATE TABLE users (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
apple_user_id TEXT UNIQUE NOT NULL,
email TEXT,
display_name TEXT,
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE TABLE user_preferences (
user_id UUID PRIMARY KEY REFERENCES users(id) ON DELETE CASCADE,
dietary_restrictions TEXT[] NOT NULL DEFAULT '{}', -- e.g. vegan, vegetarian, halal
allergies TEXT[] NOT NULL DEFAULT '{}', -- e.g. peanut, shellfish, dairy
nutrition_goals TEXT,
pantry_staples TEXT[] NOT NULL DEFAULT '{}',
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE TABLE recipes (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE,
title TEXT NOT NULL,
description TEXT,
source_type TEXT NOT NULL, -- reel, voice_search, manual, web
source_url TEXT,
source_platform TEXT, -- instagram, tiktok, web
caption TEXT,
transcript TEXT,
servings INT,
estimated_time TEXT,
status TEXT NOT NULL, -- pending, processing, ready, failed
error_message TEXT,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE INDEX idx_recipes_user_status ON recipes(user_id, status);
CREATE TABLE ingredients (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
recipe_id UUID NOT NULL REFERENCES recipes(id) ON DELETE CASCADE,
display_order INT NOT NULL,
name TEXT NOT NULL,
quantity NUMERIC,
unit TEXT,
raw_text TEXT, -- original phrasing from source
provenance TEXT NOT NULL, -- caption, transcript, overlay, vlm_inferred, user
confidence REAL NOT NULL DEFAULT 1.0,
violation_type TEXT, -- allergy, restriction, null
violation_label TEXT, -- peanut, dairy, etc.
suggested_swap TEXT
);
CREATE INDEX idx_ingredients_recipe ON ingredients(recipe_id);
CREATE TABLE recipe_steps (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
recipe_id UUID NOT NULL REFERENCES recipes(id) ON DELETE CASCADE,
step_order INT NOT NULL,
instruction TEXT NOT NULL,
timer_seconds INT
);
CREATE INDEX idx_steps_recipe ON recipe_steps(recipe_id, step_order);
CREATE TABLE jobs (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE,
type TEXT NOT NULL, -- ingest_reel, voice_search, scale_recipe
status TEXT NOT NULL, -- queued, processing, done, failed
payload JSONB NOT NULL,
result JSONB,
recipe_id UUID REFERENCES recipes(id) ON DELETE SET NULL,
error TEXT,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
started_at TIMESTAMPTZ,
completed_at TIMESTAMPTZ
);
CREATE INDEX idx_jobs_user_status ON jobs(user_id, status);
```
Allergies are stored separately from restrictions deliberately — the UI treats them with different severity, the dietary check weights them differently, and this separation scales cleanly to medical-severity flagging later if needed.
The `provenance` field on `ingredients` is the product moat. Pestle and similar apps output flat JSON. We can show "3 tbsp olive oil (seen on screen)" vs. "pinch of salt (inferred)" with visual distinction, giving users calibrated trust.
---
## 7. Pipelines
### 7.1 Reel Ingestion
When the worker picks up an `ingest_reel` job, it runs roughly this sequence. Stages short-circuit as soon as we have confident structured output.
**Stage 1 — Download.** yt-dlp fetches the video and extracts metadata. The caption is usually populated by yt-dlp from the post description. Typical reel: 5-15 MB, 30-90 seconds. Store in `/tmp/jobs/{job_id}/`.
**Stage 2 — Caption-first extraction.** Feed the caption to the fusion LLM with a structured-output prompt asking for a recipe JSON. If the model returns a recipe with all required fields populated and confidence above a threshold, skip the rest and go straight to dietary check. This is the Pestle path, and it handles probably 50-70% of reels at near-zero latency and compute cost.
**Stage 3 — Caption link extraction.** If the caption contains an external URL (common — many creators link their blog), fetch it, look for `schema.org/Recipe` JSON-LD, and use that if present. This handles another chunk of cases cleanly.
**Stage 4 — Audio + ASR.** ffmpeg extracts mono 16kHz WAV. faster-whisper with large-v3-turbo transcribes with word-level timestamps. Typical runtime: 3-6 seconds for a 60s clip on a 3060 Ti.
**Stage 5 — Frame sampling.** PySceneDetect identifies scene changes — reels cut exactly when something important happens, so this outperforms uniform sampling. Cap at 12 frames. ffmpeg extracts them as JPEGs.
**Stage 6 — OCR.** PaddleOCR on each frame. On-screen text in reels is often the most reliable source of exact ingredient quantities ("2 tbsp olive oil" flashed over a pan shot). Keep detected text with per-frame timestamps.
**Stage 7 — VLM captioning.** Qwen2.5-VL-7B describes each frame with a prompt focused on cooking context ("What ingredients, tools, and cooking actions are visible? What stage of preparation does this show?"). Keep the short descriptions. Typical: 2-4 seconds per frame in batched mode.
**Stage 8 — Fusion.** Qwen2.5-14B gets: caption, transcript (with timestamps), OCR text (with timestamps), VLM frame descriptions (with timestamps). The prompt asks for structured JSON with a per-field provenance tag. This is the stage that distinguishes "we confirmed from the overlay" from "we inferred from the voiceover".
**Stage 9 — Dietary check.** See §7.3.
**Stage 10 — Persist.** Write recipe, ingredients, steps rows. Mark job done. Send APNs push if the user is opted in.
Between ~15 and ~40 seconds end-to-end depending on how many stages were needed. Caption-only path: 2-4 seconds.
### 7.2 Voice Search
**On-device (iOS).** User holds the mic, WhisperKit transcribes with interim results shown live. On confirm, the app posts the final transcript to `/jobs/voice-search`.
**Backend.**
1. LLM interprets the query into structured search terms, extracting any implicit constraints ("dinner tonight, quick, high protein" → `{meal_type: dinner, max_time: 30m, dietary_emphasis: high_protein}`)
2. Constraints are merged with the user's stored preferences and allergies
3. Backend performs a web search (Brave Search API or Google Custom Search — pick one at implementation time, Brave is cheaper with a better API) for recipe URLs matching the terms
4. Fetch the top ~5 results with httpx. For each, look for JSON-LD `@type: Recipe` — most food blogs have this. For pages without JSON-LD, feed the HTML through a structuring LLM call
5. Filter results against the user's allergies as a hard gate, and score against soft preferences
6. Return the top 3-5 as structured recipes
Voice search recipes are not auto-saved — they appear in the result view and the user taps to save to their library.
### 7.3 Dietary Intelligence
Two severity tiers internally, visually distinct in the UI:
**Allergies (hard).** Ingredient match triggers a red warning. The fusion LLM is also asked to propose an ingredient-level swap in the same call that generates the recipe, so it's stored alongside the ingredient row. If a safe swap cannot be produced ("this recipe is fundamentally built around peanuts"), the field is left null and the UI shows "no safe substitute — consider skipping this recipe".
**Restrictions (soft).** Vegetarian, vegan, gluten-free, etc. Orange warning, always accompanied by a swap suggestion. These are treated as preferences, not safety issues.
Detection is straightforward name matching plus synonym expansion maintained as a small dictionary — "peanut" matches peanut, peanut butter, peanut oil, groundnut, arachis. Keep this dictionary in the backend repo, version-controlled. LLM is involved only for generating the swap text, not for detection (detection needs to be reliable and fast).
This runs after the fusion stage so the swap suggestions can be informed by the full recipe context (the LLM knows what role the ingredient plays).
---
## 8. Infrastructure & Deployment
**Desktop (primary dev/prod for v1).** Ubuntu 24 on WSL2 as you currently have it. Four systemd units: `postgres`, `redis`, `recipe-api` (uvicorn), `recipe-worker` (arq). A simple unified logging setup — journalctl is fine for now.
**Cloudflare Tunnel** exposes `api.wahwa.com` (or similar subdomain) to the FastAPI port. Same pattern as your existing setup. No special handling needed for the video pipeline since videos don't traverse the tunnel — they're downloaded server-side.
**Secrets** live in a `.env` file sourced by systemd. Apple keys, DB credentials, search API key, optional Gemini fallback key.
**Migrations.** Alembic. `alembic upgrade head` runs as a `ExecStartPre=` step on the API unit so deploys apply migrations automatically.
**CI (GitHub Actions).** Lint, type-check, unit tests, migration check (Alembic can detect schema drift). Not doing CD to the apartment box initially — you'll deploy by SSH and `git pull` + restart units. If that becomes annoying, wire up a webhook.
**iOS distribution.** TestFlight for internal testing. Use your existing paid Apple Developer account. Standard provisioning.
**Monitoring.** For v1, a simple `/health` endpoint on the API and a daily cron emailing you the count of jobs completed and failed in the last 24h. If you want nicer dashboards, add Grafana + Loki later, but that's polish.
---
## 9. Scaling Roadmap
The v1 setup has known bottlenecks. Here's the progression if/when usage demands it.
**Phase 1 (now): apartment desktop, everything on one box.** Fine for you alone and a handful of beta testers. Caveats: a power outage or ISP glitch takes the service down; desktop-off when you travel is an outage; all GPU throughput is shared with whatever else you're running locally.
**Phase 2: Oracle Free ARM VM as edge relay.** The Oracle VM terminates Cloudflare Tunnel traffic and forwards over WireGuard to your desktop. This matches your existing roadmap for Gitea. Benefit: the public endpoint stays up even when your desktop is rebooting; you can fall back to a "still processing — desktop is restarting" state rather than a hard 502. Desktop does all real work.
**Phase 3: managed Postgres.** Neon or Supabase Postgres when you start caring about automated backups, PITR, read replicas, or just not babysitting a DB. The schema is vanilla Postgres, the move is trivial — change the DSN. Do this before scaling users, not after.
**Phase 4: detach API from worker.** The API moves to a cheap VPS (Fly, Hetzner). Workers stay on the GPU machine (your apartment now, or a colocated box, or RunPod/Lambda GPU). Communication is over Redis still; Redis moves to the VPS side. The GPU machine only handles arq jobs and needs no inbound public traffic.
**Phase 5: multi-region or managed GPU.** If latency for Asia/Europe users matters, or if your apartment GPU is constantly saturated, move workers to a managed GPU provider (RunPod has pay-per-second GPU; Modal and Beam are similar). The fusion/VLM stage is where cost lives; swap to Gemini Flash at that point if per-call cost undercuts self-hosted amortization.
**Phase 6: if the app takes off.** Object storage for thumbnails and cached video frames (S3/R2). CDN for recipe images. A separate analytics pipeline. User-generated content moderation. Most of this is generic scaling and not worth pre-designing.
At each phase, the code shouldn't change much — the whole point of putting ML behind a worker queue and keeping API stateless is that you can redraw the deployment topology without refactoring.
---
## 10. Build Order / Milestones
Rough weekly targets assuming focused part-time work. Collapse where you're fast, expand where something fights back.
**M0 — Scaffolding (week 1).** New iOS project scaffolded with SwiftUI + share extension target + App Group. FastAPI skeleton with `/health`, Alembic baseline migration, Postgres running, Redis running, arq hello-world. Cloudflare Tunnel pointed at the API. Apple Sign In round-trip working end-to-end: iOS app signs in, backend verifies the token, issues session JWT, iOS stores and sends it. No real features yet; just prove the whole loop closes.
**M1 — Caption-first reel ingestion (weeks 2-3).** Share extension writes job and POSTs. Worker runs yt-dlp, extracts caption, runs caption-only LLM parse, writes recipe to DB. iOS main app shows recipe list and detail view with the liquid-glass card design ported from SousChefAI. This alone is a working product for maybe 60% of reels and a satisfying demo.
**M2 — Full video understanding (weeks 4-6).** Add stages 4-8 from §7.1. faster-whisper integration first (clean visible improvement). Then frames + OCR. Then VLM. Then fusion prompt engineering — this is where real time gets spent; the prompt for fusion is the heart of the app's quality. Per-field provenance rendered in the UI.
**M3 — Voice search (week 7).** WhisperKit integration. `/jobs/voice-search` endpoint. Web search integration (Brave). HTML-to-recipe extraction. Results UI.
**M4 — Dietary intelligence (week 8).** Preferences onboarding flow. Allergy dictionary. Violation detection during fusion. Swap-suggestion prompt. UI treatment for allergy vs. restriction severity.
**M5 — Polish and reliability (weeks 9-10).** APNs push on job completion. Offline-queue handling in the share extension. Error states throughout. Recipe scaling feature from the SousChef codebase (already works, just needs the backend route). Oracle ARM relay setup for Phase 2 resilience. TestFlight build for beta testers.
Two months of part-time work, plus or minus, to reach a TestFlight-ready beta. The v1 defined here is larger than Pestle's feature set on the reel-understanding side specifically, matches or exceeds on dietary handling, and adds voice search as a net-new differentiator.
---
## 11. Open Risks & Decisions Deferred
**yt-dlp breakage.** Instagram periodically changes things that break yt-dlp for days at a time. Mitigation: pin a known-good version, monitor, have a manual-update playbook, keep an eye on the yt-dlp issue tracker. Longer-term: if breakage becomes frequent enough to hurt UX, consider a fallback where the iOS share extension actually downloads the video from the reel before POSTing — iOS gets the video as part of the share payload sometimes, though this depends on Instagram's share sheet contract and isn't guaranteed.
**Local model quality vs. cost.** Qwen2.5-VL-7B and Qwen2.5-14B are good but not Gemini-3-Pro good. If the fusion stage is producing poor recipes, the escape hatch is to route fusion to Gemini 2.5 Flash or 3 Flash (order of a cent per reel, still cheaper than Pestle's on-device compute amortized over dev time). Build the fusion layer with a clean model-swap interface from day one so this is a config change.
**Desktop uptime.** Your desktop going down while you're out of town means no service. Phase 2 of the scaling roadmap mitigates but doesn't eliminate this. For beta testers, be upfront that the service is best-effort during prototyping.
**Share extension storage of refresh tokens.** The refresh token lives in the App Group Keychain, accessible to both the extension and the main app. This is a standard pattern but it's worth double-checking the Keychain access group configuration at build time — if you get it wrong the extension can't read what the main app wrote, and the failure mode is silent.
**Dietary dictionary completeness.** The allergy/restriction synonym dictionary needs careful curation. "Dairy" is a category, not an ingredient — it has to match milk, butter, cream, cheese, yogurt, whey, casein, and so on. Getting this right affects the core safety feature. Plan an explicit audit pass on the dictionary before exposing the allergy feature to beta testers.
**WhisperKit model size on first use.** First-time voice search will download ~150 MB. Either pre-download on app install (adds to the install size) or show a clear first-use spinner. Pre-downloading is nicer UX but makes the app heavier; showing a spinner on first use is fine if it's clearly communicated.
**Fusion prompt engineering.** This is the dominant quality lever and it's the least predictable piece of work. Budget roughly twice what you think it'll take. Keep a corpus of 20-30 test reels representing different recipe types (quick snacks, long-form cooking, baking, drinks) and evaluate changes against that set.

652
index.html Normal file
View File

@@ -0,0 +1,652 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>Recipe Generator</title>
<style>
@import url('https://fonts.googleapis.com/css2?family=Lora:ital,wght@0,400;0,600;1,400&family=DM+Sans:wght@300;400;500&display=swap');
*, *::before, *::after { box-sizing: border-box; margin: 0; padding: 0; }
:root {
--color-text-primary: #1a1a1a;
--color-text-secondary: #6b6b6b;
--color-background-primary: #ffffff;
--color-background-secondary: #f5f5f4;
--color-background-danger: #fef2f2;
--color-text-danger: #b91c1c;
--color-border-primary: #a3a3a3;
--color-border-secondary: #d4d4d4;
--color-border-tertiary: #e5e5e5;
--border-radius-md: 8px;
--border-radius-lg: 12px;
}
body {
font-family: 'DM Sans', sans-serif;
background: var(--color-background-primary);
color: var(--color-text-primary);
padding: 2rem;
display: flex;
justify-content: center;
}
.app { display: none; width: 100%; max-width: 680px; padding: 2rem 0; }
#login-overlay { position: fixed; top: 0; left: 0; width: 100vw; height: 100vh; background: var(--color-background-secondary); display: flex; align-items: center; justify-content: center; z-index: 1000; }
.login-card { background: var(--color-background-primary); padding: 2rem; border: 0.5px solid var(--color-border-secondary); border-radius: var(--border-radius-lg); width: 100%; max-width: 320px; box-shadow: 0 4px 12px rgba(0,0,0,0.05); }
.login-card h2 { font-family: 'Lora', serif; font-size: 24px; margin-bottom: 1.25rem; text-align: center; }
.header { margin-bottom: 2rem; border-bottom: 0.5px solid var(--color-border-tertiary); padding-bottom: 1.25rem; }
.header h1 { font-family: 'Lora', serif; font-size: 26px; font-weight: 600; letter-spacing: -0.5px; margin-bottom: 4px; }
.header p { font-size: 14px; color: var(--color-text-secondary); font-weight: 300; }
.form-grid { display: grid; grid-template-columns: 1fr 1fr; gap: 14px; margin-bottom: 14px; }
.form-field { display: flex; flex-direction: column; gap: 6px; }
.form-field.full { grid-column: 1 / -1; }
label { font-size: 12px; font-weight: 500; color: var(--color-text-secondary); letter-spacing: 0.04em; text-transform: uppercase; }
input, textarea, select {
font-family: 'DM Sans', sans-serif; font-size: 14px; padding: 9px 12px;
border: 0.5px solid var(--color-border-secondary); border-radius: var(--border-radius-md);
background: var(--color-background-primary); color: var(--color-text-primary);
width: 100%; transition: border-color 0.15s; outline: none;
}
input:focus, textarea:focus, select:focus { border-color: var(--color-border-primary); }
textarea { resize: vertical; min-height: 72px; line-height: 1.5; }
.tags-input-wrapper {
border: 0.5px solid var(--color-border-secondary); border-radius: var(--border-radius-md);
background: var(--color-background-primary); padding: 6px 8px;
display: flex; flex-wrap: wrap; gap: 6px; align-items: center;
cursor: text; min-height: 40px; transition: border-color 0.15s;
}
.tags-input-wrapper:focus-within { border-color: var(--color-border-primary); }
.tag { background: var(--color-background-secondary); border: 0.5px solid var(--color-border-tertiary); border-radius: 4px; padding: 2px 8px; font-size: 13px; display: flex; align-items: center; gap: 5px; white-space: nowrap; }
.tag-x { cursor: pointer; color: var(--color-text-secondary); font-size: 15px; line-height: 1; }
.tag-x:hover { color: var(--color-text-primary); }
.tags-input { border: none !important; padding: 2px 4px !important; flex: 1; min-width: 100px; font-size: 14px; background: transparent !important; outline: none !important; }
.checkboxes { display: flex; flex-wrap: wrap; gap: 8px; }
.check-pill { display: flex; align-items: center; gap: 6px; padding: 5px 12px; border: 0.5px solid var(--color-border-secondary); border-radius: 20px; font-size: 13px; cursor: pointer; transition: all 0.12s; background: var(--color-background-primary); user-select: none; }
.check-pill input { display: none; }
.check-pill.checked { background: var(--color-background-secondary); border-color: var(--color-border-primary); font-weight: 500; }
.dot { width: 7px; height: 7px; border-radius: 50%; background: var(--color-border-secondary); flex-shrink: 0; }
.check-pill.checked .dot { background: var(--color-text-primary); }
.generate-btn { width: 100%; padding: 11px; background: var(--color-text-primary); color: var(--color-background-primary); border: none; border-radius: var(--border-radius-md); font-family: 'DM Sans', sans-serif; font-size: 15px; font-weight: 500; cursor: pointer; margin-top: 8px; transition: opacity 0.15s; }
.generate-btn:hover { opacity: 0.85; }
.generate-btn:disabled { opacity: 0.45; cursor: not-allowed; }
.status { text-align: center; padding: 2rem 0; color: var(--color-text-secondary); font-size: 14px; font-style: italic; }
.recipe-out { margin-top: 2.5rem; border-top: 0.5px solid var(--color-border-tertiary); padding-top: 2rem; animation: fadein 0.4s ease; }
@keyframes fadein { from { opacity: 0; transform: translateY(8px); } to { opacity: 1; transform: translateY(0); } }
.recipe-actions { display: flex; gap: 10px; margin-top: 1.5rem; }
.action-btn { flex: 1; padding: 10px; background: var(--color-background-primary); border: 0.5px solid var(--color-border-primary); border-radius: var(--border-radius-md); font-family: 'DM Sans', sans-serif; font-size: 14px; font-weight: 500; cursor: pointer; transition: all 0.15s; }
.action-btn:hover { background: var(--color-background-secondary); }
.saved-menu-btn { position: fixed; top: 20px; right: 20px; padding: 8px 16px; background: var(--color-text-primary); color: white; border: none; border-radius: 20px; font-size: 13px; font-weight: 500; cursor: pointer; z-index: 100; box-shadow: 0 2px 8px rgba(0,0,0,0.15); }
#saved-menu { position: fixed; top: 0; right: -350px; width: 320px; height: 100vh; background: var(--color-background-primary); box-shadow: -4px 0 15px rgba(0,0,0,0.05); transition: right 0.3s ease; z-index: 200; padding: 2rem 1.5rem; overflow-y: auto; border-left: 1px solid var(--color-border-tertiary); }
#saved-menu.open { right: 0; }
.saved-menu-header { display: flex; justify-content: space-between; align-items: center; margin-bottom: 1.5rem; }
.saved-menu-header h2 { font-family: 'Lora', serif; font-size: 20px; }
.close-menu { cursor: pointer; font-size: 24px; color: var(--color-text-secondary); line-height: 1; background: none; border: none; }
.saved-recipe-item { padding: 12px; border: 0.5px solid var(--color-border-secondary); border-radius: var(--border-radius-md); margin-bottom: 10px; cursor: pointer; transition: border-color 0.15s; }
.saved-recipe-item:hover { border-color: var(--color-text-primary); }
.saved-recipe-item h3 { font-size: 15px; margin-bottom: 4px; }
.saved-recipe-item p { font-size: 12px; color: var(--color-text-secondary); white-space: nowrap; overflow: hidden; text-overflow: ellipsis; }
.recipe-title { font-family: 'Lora', serif; font-size: 28px; font-weight: 600; letter-spacing: -0.5px; margin-bottom: 6px; }
.recipe-desc { font-size: 14px; color: var(--color-text-secondary); font-style: italic; margin-bottom: 1.25rem; line-height: 1.6; }
.meta-row { display: flex; gap: 12px; flex-wrap: wrap; margin-bottom: 1.75rem; }
.meta-chip { background: var(--color-background-secondary); border-radius: var(--border-radius-md); padding: 6px 14px; font-size: 13px; color: var(--color-text-secondary); }
.meta-chip strong { color: var(--color-text-primary); font-weight: 500; }
.section-label { font-size: 11px; font-weight: 500; letter-spacing: 0.08em; text-transform: uppercase; color: var(--color-text-secondary); margin-bottom: 10px; margin-top: 1.5rem; }
.ingredients-list { list-style: none; border: 0.5px solid var(--color-border-tertiary); border-radius: var(--border-radius-lg); overflow: hidden; }
.ingredients-list li { padding: 9px 16px; font-size: 14px; border-bottom: 0.5px solid var(--color-border-tertiary); display: flex; gap: 10px; align-items: baseline; }
.ingredients-list li:last-child { border-bottom: none; }
.ing-amount { font-weight: 500; min-width: 60px; }
.ing-name { color: var(--color-text-secondary); }
.steps-list { list-style: none; }
.step-item { display: flex; gap: 16px; margin-bottom: 1.25rem; }
.step-num { font-family: 'Lora', serif; font-size: 18px; font-weight: 600; color: var(--color-text-secondary); min-width: 28px; line-height: 1.5; }
.step-title { font-weight: 500; font-size: 14px; margin-bottom: 4px; }
.step-instr { font-size: 14px; line-height: 1.7; color: var(--color-text-secondary); }
.notes-box { background: var(--color-background-secondary); border-radius: 0 var(--border-radius-md) var(--border-radius-md) 0; padding: 12px 16px; font-size: 14px; line-height: 1.6; color: var(--color-text-secondary); margin-top: 1.5rem; border-left: 2px solid var(--color-border-primary); }
.error-msg { background: var(--color-background-danger); color: var(--color-text-danger); border-radius: var(--border-radius-md); padding: 12px 16px; font-size: 13px; margin-top: 1rem; line-height: 1.5; }
.spinner { display: inline-block; width: 16px; height: 16px; border: 2px solid rgba(255,255,255,0.35); border-top-color: #fff; border-radius: 50%; animation: spin 0.7s linear infinite; vertical-align: middle; margin-right: 6px; }
@keyframes spin { to { transform: rotate(360deg); } }
</style>
</head>
<body>
<div id="login-overlay">
<div class="login-card">
<h2 style="font-family: 'Lora', serif; font-size: 26px; font-weight: 600; letter-spacing: -0.5px; margin-bottom: 20px;">Welcome</h2>
<div class="form-field" style="margin-bottom: 12px;">
<label>Username</label>
<input type="text" id="login-username" />
</div>
<div class="form-field" style="margin-bottom: 12px;">
<label>Password</label>
<input type="password" id="login-password" />
</div>
<button class="generate-btn" id="login-btn" style="margin-top: 12px;">Sign in</button>
<div id="login-error" style="color: var(--color-text-danger); font-size: 13px; margin-top: 10px; text-align: center; display: none;">Invalid credentials</div>
</div>
</div>
<div class="app" id="app-content">
<button class="saved-menu-btn" id="open-menu-btn">View Saved Recipes</button>
<div id="saved-menu">
<div class="saved-menu-header">
<h2>Saved Recipes</h2>
<button class="close-menu" id="close-menu-btn">×</button>
</div>
<div id="saved-list">
<!-- Saved items go here -->
</div>
</div>
<div class="header">
<h1>Recipe Generator</h1>
<p>Tell the AI what you have and what you want — it handles the rest.</p>
</div>
<!-- Reel Ingestion Section -->
<div class="form-grid" style="margin-bottom: 2rem; border: 1px solid var(--color-border-primary); padding: 1.5rem; border-radius: var(--border-radius-md); background: rgba(0, 0, 0, 0.02);">
<div class="form-field full" style="margin-bottom: 0;">
<label style="font-weight: 600; color: var(--color-text-primary);">Extract Recipe from Instagram/TikTok Reel</label>
<p style="font-size: 13px; color: var(--color-text-secondary); margin-top: 4px; margin-bottom: 12px;">Paste a link to extract frames and audio, and let the VLM structure the recipe.</p>
<div style="display: flex; gap: 10px;">
<input type="text" id="reel-url" placeholder="https://www.instagram.com/reel/..." style="flex: 1;" />
</div>
<button class="generate-btn" id="ingest-btn" style="margin-top: 12px; background: var(--color-background-primary); color: var(--color-text-primary); border: 1px solid var(--color-text-primary);">Extract from Reel</button>
</div>
</div>
<div style="text-align: center; margin-bottom: 2rem; font-family: 'Lora', serif; color: var(--color-text-secondary); font-style: italic;">— OR GENERATE FROM SCRATCH —</div>
<div class="form-grid">
<div class="form-field full">
<label>Ingredients on hand</label>
<div class="tags-input-wrapper" id="ing-wrapper">
<input class="tags-input" id="ing-input" placeholder="Type an ingredient, press Enter..." />
</div>
</div>
<div class="form-field full">
<label>What do you want to make?</label>
<input type="text" id="goal" placeholder="e.g. a hearty weeknight pasta, a light summer salad..." />
</div>
<div class="form-field full">
<label>Dietary restrictions</label>
<div class="checkboxes">
<label class="check-pill"><input type="checkbox" value="vegan"><span class="dot"></span>Vegan</label>
<label class="check-pill"><input type="checkbox" value="vegetarian"><span class="dot"></span>Vegetarian</label>
<label class="check-pill"><input type="checkbox" value="gluten-free"><span class="dot"></span>Gluten-free</label>
<label class="check-pill"><input type="checkbox" value="dairy-free"><span class="dot"></span>Dairy-free</label>
<label class="check-pill"><input type="checkbox" value="nut-free"><span class="dot"></span>Nut-free</label>
<label class="check-pill"><input type="checkbox" value="low-carb"><span class="dot"></span>Low-carb</label>
</div>
</div>
<div class="form-field">
<label>Cuisine style</label>
<select id="cuisine">
<option value="">Any cuisine</option>
<option>Italian</option><option>Mexican</option><option>Asian fusion</option>
<option>Mediterranean</option><option>French</option><option>Indian</option>
<option>American</option><option>Middle Eastern</option><option>Japanese</option><option>Thai</option>
</select>
</div>
<div class="form-field">
<label>Time available</label>
<select id="time">
<option value="">No limit</option>
<option value="under 20 minutes">Under 20 minutes</option>
<option value="under 30 minutes">Under 30 minutes</option>
<option value="under 45 minutes">Under 45 minutes</option>
<option value="under 1 hour">Under 1 hour</option>
<option value="12 hours">12 hours</option>
</select>
</div>
<div class="form-field">
<label>Skill level</label>
<select id="skill">
<option value="">Any level</option>
<option value="beginner-friendly">Beginner-friendly</option>
<option value="intermediate">Intermediate</option>
<option value="advanced">Advanced chef techniques</option>
</select>
</div>
<div class="form-field">
<label>Servings</label>
<select id="servings">
<option value="1">1</option><option value="2" selected>2</option>
<option value="4">4</option><option value="6">6</option><option value="8">8</option>
</select>
</div>
<div class="form-field full">
<label>Any other notes or preferences</label>
<textarea id="notes" placeholder="e.g. my kids hate mushrooms, I want it spicy, make it impressive enough for guests..."></textarea>
</div>
</div>
<button class="generate-btn" id="gen-btn">Generate my recipe</button>
<div id="error-area"></div>
<div id="recipe-area"></div>
</div>
<script type="module">
let sessionUser = '';
let sessionPass = '';
const loginBtn = document.getElementById('login-btn');
const loginOverlay = document.getElementById('login-overlay');
const appContent = document.getElementById('app-content');
loginBtn.addEventListener('click', async () => {
const user = document.getElementById('login-username').value;
const pass = document.getElementById('login-password').value;
const btn = document.getElementById('login-btn');
btn.disabled = true;
btn.innerText = 'Verifying...';
try {
const res = await fetch('/api/login', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ username: user, password: pass })
});
if (res.ok) {
sessionUser = user;
sessionPass = pass;
loginOverlay.style.display = 'none';
appContent.style.display = 'block';
} else {
document.getElementById('login-error').style.display = 'block';
}
} catch (err) {
document.getElementById('login-error').innerText = 'Network error. Please try again.';
document.getElementById('login-error').style.display = 'block';
}
btn.disabled = false;
btn.innerText = 'Sign in';
});
const ingredients = [];
const ingWrapper = document.getElementById('ing-wrapper');
const ingInput = document.getElementById('ing-input');
const genBtn = document.getElementById('gen-btn');
const ingestBtn = document.getElementById('ingest-btn');
genBtn.addEventListener('click', generate);
ingestBtn.addEventListener('click', ingestReel);
ingWrapper.addEventListener('click', () => ingInput.focus());
ingInput.addEventListener('keydown', e => {
if ((e.key === 'Enter' || e.key === ',') && ingInput.value.trim()) {
e.preventDefault();
addTag(ingInput.value.trim().replace(/,$/, ''));
ingInput.value = '';
} else if (e.key === 'Backspace' && !ingInput.value && ingredients.length) {
removeTag(ingredients.length - 1);
}
});
function addTag(val) {
if (!val || ingredients.includes(val.toLowerCase())) return;
ingredients.push(val.toLowerCase());
const tag = document.createElement('div');
tag.className = 'tag';
tag.innerHTML = val + '<span class="tag-x">×</span>';
tag.querySelector('.tag-x').addEventListener('click', (e) => { e.stopPropagation(); removeTagByVal(val.toLowerCase()); });
ingWrapper.insertBefore(tag, ingInput);
}
function removeTagByVal(val) {
const i = ingredients.indexOf(val);
if (i > -1) { ingredients.splice(i, 1); renderTags(); }
}
function removeTag(idx) { ingredients.splice(idx, 1); renderTags(); }
function renderTags() {
ingWrapper.querySelectorAll('.tag').forEach(t => t.remove());
const copy = [...ingredients]; ingredients.length = 0;
copy.forEach(i => addTag(i));
}
document.querySelectorAll('.check-pill').forEach(pill => {
pill.addEventListener('click', () => {
const cb = pill.querySelector('input');
cb.checked = !cb.checked;
pill.classList.toggle('checked', cb.checked);
});
});
async function generate() {
const goal = document.getElementById('goal').value.trim();
if (!goal && ingredients.length === 0) {
showError('Please add some ingredients or describe what you want to make.');
return;
}
const restrictions = [...document.querySelectorAll('.check-pill input:checked')].map(c => c.value);
const cuisine = document.getElementById('cuisine').value;
const time = document.getElementById('time').value;
const skill = document.getElementById('skill').value;
const servings = document.getElementById('servings').value;
const notes = document.getElementById('notes').value.trim();
const btn = document.getElementById('gen-btn');
btn.disabled = true;
btn.innerHTML = '<span class="spinner"></span>Generating…';
document.getElementById('error-area').innerHTML = '';
document.getElementById('recipe-area').innerHTML = '<div class="status">Crafting your recipe with care…</div>';
const userPrompt = [
ingredients.length ? `Ingredients available: ${ingredients.join(', ')}` : '',
goal ? `Food goal: ${goal}` : '',
restrictions.length ? `Dietary restrictions: ${restrictions.join(', ')}` : '',
cuisine ? `Preferred cuisine: ${cuisine}` : '',
time ? `Time constraint: ${time}` : '',
skill ? `Skill level: ${skill}` : '',
`Servings: ${servings}`,
notes ? `Additional preferences: ${notes}` : '',
].filter(Boolean).join('\n');
const systemPrompt = `You are an expert chef and culinary writer. Create a complete, precise recipe based on the user's inputs.
CRITICAL: Instructions must be specific — never vague. Include exact temperatures, times, visual cues, sounds, smells, and textures. Example: instead of "cook the onions", write "cook the onions in the butter over medium heat, stirring occasionally, for 810 minutes until they turn translucent and just begin to turn golden at the edges."
Respond with ONLY a valid JSON object. Use this exact structure:
{"title":"string","description":"string","servings":2,"prepTime":"string","cookTime":"string","ingredients":[{"amount":"string","unit":"string","name":"string"}],"steps":[{"title":"string","instruction":"string"}],"notes":"string or null"}
Rules: use provided ingredients as the base, add pantry staples as needed, 510 steps, each instruction 13 richly detailed sentences, respect all dietary restrictions strictly.`;
try {
const resp = await fetch('/api/generate', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
username: sessionUser,
password: sessionPass,
systemPrompt: systemPrompt,
userPrompt: userPrompt
})
});
const data = await resp.json();
if (!resp.ok) {
throw new Error(data.error || `Server error ${resp.status}`);
}
const raw = data.candidates[0].content.parts[0].text;
const recipe = JSON.parse(raw);
renderRecipe(recipe);
} catch (err) {
document.getElementById('recipe-area').innerHTML = '';
showError('Error: ' + (err.message || 'Unknown error. Please try again.'));
}
btn.disabled = false;
btn.innerHTML = 'Generate my recipe';
}
async function ingestReel() {
const url = document.getElementById('reel-url').value.trim();
if (!url) {
showError('Please paste a valid Instagram or TikTok Reel URL.');
return;
}
const btn = document.getElementById('ingest-btn');
btn.disabled = true;
btn.innerHTML = '<span class="spinner"></span>Extracting & Analyzing Video... (This takes 15-30s) ';
document.getElementById('error-area').innerHTML = '';
document.getElementById('recipe-area').innerHTML = '<div class="status">Downloading reel and sending to Gemini Vision...</div>';
try {
const resp = await fetch('/api/ingest', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
username: sessionUser,
password: sessionPass,
url: url
})
});
const data = await resp.json();
if (!resp.ok) {
throw new Error(data.error || `Server error ${resp.status}`);
}
// Render the recipe
renderRecipe(data.recipe);
currentRecipeId = data.id;
document.getElementById('recipe-area').scrollIntoView({ behavior: 'smooth' });
} catch (err) {
document.getElementById('recipe-area').innerHTML = '';
showError('Error processing reel: ' + (err.message || 'Unknown error. Please try again.'));
} finally {
btn.disabled = false;
btn.innerHTML = 'Extract from Reel';
}
}
function showError(msg) {
document.getElementById('error-area').innerHTML = `<div class="error-msg">${msg}</div>`;
}
let currentRecipe = null;
let currentRecipeId = null;
const savedMenu = document.getElementById('saved-menu');
const savedList = document.getElementById('saved-list');
document.getElementById('open-menu-btn').addEventListener('click', async () => {
savedMenu.classList.add('open');
await fetchSavedRecipes();
});
document.getElementById('close-menu-btn').addEventListener('click', () => {
savedMenu.classList.remove('open');
});
async function fetchSavedRecipes() {
savedList.innerHTML = '<p style="font-size: 13px; color: var(--color-text-secondary); text-align: center;">Loading...</p>';
try {
const res = await fetch('/api/recipes', {
headers: {
'x-username': sessionUser,
'x-password': sessionPass
}
});
if (!res.ok) throw new Error('Failed to fetch');
const recipes = await res.json();
if (recipes.length === 0) {
savedList.innerHTML = '<p style="font-size: 13px; color: var(--color-text-secondary); text-align: center;">No saved recipes yet.</p>';
return;
}
// Reverse so newest is first
savedList.innerHTML = recipes.reverse().map(r => `
<div class="saved-recipe-item" onclick="loadSavedRecipe('${r.id}')">
<h3>${r.title}</h3>
<p>${r.description}</p>
</div>
`).join('');
} catch (e) {
savedList.innerHTML = '<p style="font-size: 13px; color: var(--color-text-danger); text-align: center;">Error loading recipes.</p>';
}
}
window.loadSavedRecipe = async function(id) {
savedMenu.classList.remove('open');
try {
const res = await fetch(`/api/recipes/${id}`);
if (!res.ok) throw new Error('Failed to fetch recipe');
const recipe = await res.json();
recipe.id = id;
renderRecipe(recipe);
document.getElementById('recipe-area').scrollIntoView({ behavior: 'smooth' });
} catch (err) {
console.error(err);
}
};
window.saveCurrentRecipe = async function() {
if (!currentRecipe) return;
const btn = document.getElementById('save-recipe-btn');
const originalText = btn.innerText;
btn.innerText = 'Saving...';
btn.disabled = true;
try {
const res = await fetch('/api/recipes', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
username: sessionUser,
password: sessionPass,
recipe: currentRecipe
})
});
if (res.ok) {
const data = await res.json();
currentRecipeId = data.id;
currentRecipe.id = data.id;
btn.innerText = 'Saved!';
} else {
throw new Error('Save failed');
}
} catch (err) {
btn.innerText = "Error Saving";
setTimeout(() => {
btn.innerText = originalText;
btn.disabled = false;
}, 2000);
}
};
window.shareRecipeLink = function() {
if (!currentRecipeId) {
alert("Please save the recipe first to share it!");
return;
}
const url = `${window.location.origin}${window.location.pathname}#shared=${currentRecipeId}`;
navigator.clipboard.writeText(url).then(() => {
const btn = document.getElementById('share-recipe-btn');
const oldText = btn.innerText;
btn.innerText = 'Link Copied!';
setTimeout(() => btn.innerText = oldText, 2000);
});
};
function renderRecipe(r, isSharedView = false) {
currentRecipe = r;
currentRecipeId = r.id || null;
const ings = (r.ingredients || []).map(ing =>
`<li><span class="ing-amount">${ing.amount}${ing.unit ? ' ' + ing.unit : ''}</span><span class="ing-name">${ing.name}</span></li>`
).join('');
const steps = (r.steps || []).map((s, i) =>
`<li class="step-item">
<span class="step-num">${i + 1}</span>
<div>
<div class="step-title">${s.title}</div>
<div class="step-instr">${s.instruction}</div>
</div>
</li>`
).join('');
document.getElementById('recipe-area').innerHTML = `
<div class="recipe-out">
<div class="recipe-title">${r.title}</div>
<div class="recipe-desc">${r.description}</div>
<div class="meta-row">
<div class="meta-chip"><strong>${r.servings}</strong> servings</div>
<div class="meta-chip">Prep <strong>${r.prepTime}</strong></div>
<div class="meta-chip">Cook <strong>${r.cookTime}</strong></div>
</div>
<div class="section-label">Ingredients</div>
<ul class="ingredients-list">${ings}</ul>
<div class="section-label">Method</div>
<ol class="steps-list">${steps}</ol>
${r.notes ? `<div class="notes-box">${r.notes}</div>` : ''}
<div class="recipe-actions">
${isSharedView ? `
<button class="action-btn" style="background: var(--color-text-primary); color: white;" onclick="window.location.href=window.location.pathname">Return to Login</button>
` : `
<button class="action-btn" id="save-recipe-btn" onclick="saveCurrentRecipe()">Save to Menu</button>
<button class="action-btn" id="share-recipe-btn" onclick="shareRecipeLink()">Copy Share Link</button>
`}
</div>
</div>`;
}
// Check for shared recipe in URL hash on load
window.addEventListener('DOMContentLoaded', async () => {
if (window.location.hash.startsWith('#shared=')) {
try {
const recipeId = window.location.hash.slice(8);
document.getElementById('login-overlay').innerText = 'Loading Shared Recipe...';
document.getElementById('login-overlay').style.color = '#1a1a1a';
const res = await fetch(`/api/recipes/${recipeId}`);
if (!res.ok) throw new Error('Shared recipe not found');
const r = await res.json();
r.id = recipeId;
// Hide generation UI elements so ONLY the recipe is visible
document.getElementById('login-overlay').style.display = 'none';
document.querySelectorAll('#open-menu-btn, .header, .form-grid, #gen-btn, #error-area').forEach(el => {
if(el) el.style.display = 'none';
});
document.getElementById('app-content').style.display = 'block';
// Disable save button or generator elements since we are in a shared view state if you desire,
// but for now let's just render the recipe directly.
renderRecipe(r, true);
} catch (e) {
console.error("Failed to load shared recipe", e);
document.getElementById('login-overlay').innerHTML = `
<div class="login-card" style="text-align:center;">
<h2 style="font-family: 'Lora', serif; font-size: 20px;">Recipe Not Found</h2>
<p style="font-size: 14px; margin-top: 10px;">The shared recipe link appears to be invalid or has been deleted.</p>
<button class="generate-btn" style="margin-top: 16px;" onclick="window.location.href=window.location.pathname">Return to Login</button>
</div>
`;
}
}
});
</script>
</body>
</html>

1906
package-lock.json generated Normal file

File diff suppressed because it is too large Load Diff

21
package.json Normal file
View File

@@ -0,0 +1,21 @@
{
"name": "recipe-generator",
"version": "1.0.0",
"description": "Recipe generator using Gemini",
"type": "module",
"scripts": {
"dev": "vite",
"build": "vite build",
"preview": "vite preview",
"start": "node server.js"
},
"devDependencies": {
"vite": "^5.0.0"
},
"dependencies": {
"@google/generative-ai": "^0.24.1",
"cors": "^2.8.5",
"dotenv": "^16.6.1",
"express": "^4.22.1"
}
}

4
readme.md Normal file
View File

@@ -0,0 +1,4 @@
# SousChefAI (Web Server / Site)
Testing grounds and production view for SousChefAI's recipe generation software

View File

@@ -0,0 +1,200 @@
# Recipe Generator — Engineering Notes
> Handoff doc covering technical decisions, design choices, and bug fixes made during initial development. Last updated: April 28, 2026.
---
## What It Is
A single-page HTML app that takes user-supplied ingredients, a food goal, dietary restrictions, and preferences, then calls the Anthropic API directly from the browser to generate a structured, chef-quality recipe with precise step-by-step instructions.
No backend. No build step. One `.html` file.
---
## Tech Stack
| Layer | Choice | Reason |
|---|---|---|
| Runtime | Vanilla HTML/CSS/JS | No framework needed; single file, zero dependencies |
| Fonts | Google Fonts (Lora + DM Sans) | Lora for editorial/recipe headings; DM Sans for UI chrome |
| AI | Anthropic API (`claude-sonnet-4-20250514`) | Direct browser fetch with `anthropic-dangerous-direct-browser-access` header |
| Output format | JSON (structured) | Predictable, renderable, extensible |
---
## Inputs Collected
| Field | Type | Notes |
|---|---|---|
| Ingredients on hand | Tag input (multi-value) | Comma or Enter to add; Backspace to remove last |
| Food goal | Free text | e.g. "a hearty weeknight pasta" |
| Dietary restrictions | Multi-select pill toggles | Vegan, Vegetarian, Gluten-free, Dairy-free, Nut-free, Low-carb |
| Cuisine style | Dropdown | 10 options + "Any cuisine" |
| Time available | Dropdown | Under 20 min → 12 hrs |
| Skill level | Dropdown | Beginner / Intermediate / Advanced |
| Servings | Dropdown | 1, 2, 4, 6, 8 |
| Additional notes | Textarea | Free-form catch-all |
---
## API Design
### Endpoint
```
POST https://api.anthropic.com/v1/messages
```
### Required Headers
```js
{
'content-type': 'application/json',
'anthropic-version': '2023-06-01',
'anthropic-dangerous-direct-browser-access': 'true'
}
```
> ⚠️ **Do not include `x-api-key`** when running inside the Claude.ai artifact environment — the platform injects the key automatically. Passing an empty string causes an auth failure (see Bug Fixes below).
### Model
`claude-sonnet-4-20250514` — good balance of quality and speed for recipe generation. Max tokens set to `1800`.
### Prompt Architecture
Two-part: system prompt + user message.
**System prompt** defines:
- Role: expert chef and culinary writer
- Explicit anti-vagueness instruction with a concrete example of bad vs. good instruction style
- Strict JSON-only output requirement (no markdown fences, no preamble)
- The exact JSON schema expected
- Hard rules: respect dietary restrictions, 510 steps, precise sensory details
**User message** is assembled dynamically from form state, filtering out any empty fields:
```
Ingredients available: pasta, garlic, tomato, tofu
Food goal: quick but protein packed pasta
Dietary restrictions: vegetarian, dairy-free
Time constraint: under 20 minutes
Servings: 6
```
### Response Schema
```json
{
"title": "string",
"description": "string",
"servings": 2,
"prepTime": "string",
"cookTime": "string",
"ingredients": [
{ "amount": "string", "unit": "string", "name": "string" }
],
"steps": [
{ "title": "string", "instruction": "string" }
],
"notes": "string | null"
}
```
- `servings` is typed as a **number** (not string) — explicitly called out in system prompt to avoid JSON parse issues
- `unit` can be omitted for whole/countable items (the renderer handles `undefined` gracefully)
- `notes` is nullable
---
## Prompt Engineering Decisions
### Anti-vagueness instruction
The single most impactful prompt decision. Rather than just saying "be detailed," the system prompt gives a concrete counter-example:
> Instead of "cook the onions", write "cook the onions in the butter over medium heat, stirring occasionally, for 810 minutes until they turn translucent and just begin to turn golden at the edges."
This anchors the model's output quality at the instruction level, not just the ingredient level.
### JSON-only output enforcement
The system prompt instructs the model to return nothing before or after the JSON. To be defensive against any stray text, the parser uses:
```js
const jsonStart = raw.indexOf('{');
const jsonEnd = raw.lastIndexOf('}');
const recipe = JSON.parse(raw.slice(jsonStart, jsonEnd + 1));
```
This is more robust than `JSON.parse(raw)` directly, which would fail on any leading/trailing characters.
---
## Design Decisions
### Tag input for ingredients
Standard text input UX isn't great for multi-item lists. Tags let users see all their ingredients at a glance and remove individual ones without retyping. Implemented in vanilla JS — no library.
- **Enter** or **comma** commits a tag
- **Backspace** on empty input removes the last tag
- Duplicate ingredients are silently ignored (case-insensitive)
### Pill toggles for dietary restrictions
Checkboxes are functional but visually coarse for this context. Pills feel more natural for preference selection and render well on mobile. The native `<input type="checkbox">` is hidden; state is managed via `.checked` property and a `.checked` CSS class on the parent.
### Typography split
- **Lora** (serif): recipe title, step numbers — editorial, food-magazine feel
- **DM Sans** (sans-serif): all UI chrome, form labels, body text — clean and modern
### Recipe output layout
- Meta chips (servings / prep / cook time) give a quick overview before diving in
- Ingredients in a bordered list table — scannable, easy to follow while cooking
- Steps as numbered items with a bold title + detail body — mirrors professional cookbook formatting
- Chef's notes in a left-bordered aside box — visually distinct from the main method
---
## Bug Fixes
### Bug 1 — "Load failed" / auth error on API call
**Root cause:** The initial implementation passed `'x-api-key': ''` as a header. In the Claude.ai artifact environment, the API key is injected by the platform — but only when the header is absent. Passing an explicit (even empty) `x-api-key` header overrides the platform injection and results in an authentication failure before the request reaches the model.
**Fix:** Remove the `x-api-key` header entirely from the fetch call.
```js
// ❌ Before
headers: {
'Content-Type': 'application/json',
'x-api-key': '', // <-- causes auth failure
'anthropic-version': '2023-06-01',
'anthropic-dangerous-direct-browser-access': 'true'
}
// ✅ After
headers: {
'content-type': 'application/json',
'anthropic-version': '2023-06-01',
'anthropic-dangerous-direct-browser-access': 'true'
}
```
### Bug 2 — Silent generic error message
**Root cause:** The original catch block showed a hardcoded "Something went wrong" message regardless of what the API returned, making it impossible to diagnose failures.
**Fix:** Parse the API error response body and surface the actual `error.message` field:
```js
const data = await resp.json();
if (!resp.ok) {
throw new Error(data.error?.message || `API error ${resp.status}`);
}
```
### Bug 3 — Missing required browser-access header
**Root cause:** `anthropic-dangerous-direct-browser-access: true` was absent in the initial version, which causes the API to reject browser-origin requests as a CORS/security policy enforcement.
**Fix:** Added the header explicitly to all fetch calls.
---
## Possible Next Steps
- **Streaming** — use the Anthropic streaming API so the recipe appears token by token rather than all at once after a delay
- **Persistent recipe book** — use the Claude.ai artifact storage API to save generated recipes across sessions
- **Serving scaler** — the structured JSON makes it trivial to multiply ingredient amounts; add a stepper UI to the output
- **Nutrition estimates** — add a `nutritionFacts` field to the JSON schema and ask the model to estimate macros
- **Shopping list view** — cross-reference the generated ingredients against what the user said they have, and produce a "what to buy" list
- **Image generation** — call an image model with the recipe title and description to generate a hero photo
- **Export to PDF** — render the recipe output as a printable/downloadable PDF

View File

@@ -0,0 +1 @@
{"title":"20-Minute Protein-Packed Tofu & Yogurt Penne","description":"This lightning-fast, vegetarian dish delivers a hearty dose of protein in under 20 minutes, perfect for a busy weeknight. Al dente penne pasta is tossed with savory pan-fried tofu, tender sautéed onions and tomatoes, and a creamy, protein-rich sauce made from Greek yogurt and eggs, finished with fresh chives and a hint of warm cloves.","servings":8,"prepTime":"10 minutes","cookTime":"10 minutes","ingredients":[{"amount":"1.5 pounds","unit":"","name":"penne pasta"},{"amount":"3 (14-ounce)","unit":"blocks","name":"extra-firm tofu"},{"amount":"4 tablespoons","unit":"","name":"vegetable oil, divided"},{"amount":"1 large","unit":"","name":"yellow onion, finely diced"},{"amount":"2 large","unit":"","name":"ripe tomatoes, roughly chopped"},{"amount":"1.5 cups","unit":"","name":"plain Greek yogurt"},{"amount":"3 large","unit":"","name":"eggs"},{"amount":"0.25 teaspoon","unit":"","name":"ground cloves"},{"amount":"0.5 cup","unit":"","name":"fresh chives, chopped"},{"amount":"2 tablespoons","unit":"","name":"kosher salt, plus more to taste"},{"amount":"0.5 teaspoon","unit":"","name":"black pepper, plus more to taste"},{"amount":"6 quarts","unit":"","name":"water, for pasta"}],"steps":[{"title":"Prepare Pasta Water & Tofu","instruction":"Bring 6 quarts (5.7 liters) of water to a rolling boil over high heat in a large stockpot. While the water heats, drain the 3 blocks of extra-firm tofu, gently press out excess water using paper towels or a clean kitchen towel for 5 minutes, then cut the tofu into 1/2-inch (1.25 cm) cubes."},{"title":"Cook Tofu & Start Pasta","instruction":"Heat 3 tablespoons of vegetable oil in a large non-stick skillet over medium-high heat. Add cubed tofu in a single layer, ensuring not to overcrowd the pan, and cook for 4-6 minutes, stirring occasionally, until all sides are golden brown and slightly crispy to the touch. Once the pasta water is boiling, add 2 tablespoons of kosher salt, then add 1.5 pounds (680g) of penne pasta and cook according to package directions, typically 10-12 minutes, stirring occasionally, until al dente tender but still firm to the bite. Reserve 1 cup of pasta water before draining the penne."},{"title":"Sauté Aromatics","instruction":"While the pasta cooks, return to the skillet used for tofu (or use a separate skillet to save time, adding 1 tablespoon vegetable oil) and reduce heat to medium. Add the finely diced large yellow onion and sauté for 5-7 minutes, stirring frequently, until translucent, softened, and releasing a sweet aroma. Add the roughly chopped large ripe tomatoes and cook for another 3-4 minutes, stirring, until the tomatoes begin to break down and release their juices, creating a rustic sauce."},{"title":"Prepare Creamy Yogurt Sauce","instruction":"In a medium bowl, whisk together the 1.5 cups (360g) plain Greek yogurt, 3 large eggs, 1/4 teaspoon ground cloves, 1 teaspoon salt, and 1/2 teaspoon black pepper until the mixture is smooth and well combined, ensuring no lumps remain from the yogurt."},{"title":"Combine and Serve","instruction":"Immediately after draining, return the hot penne pasta to the large stockpot. Add the cooked golden-brown tofu cubes, the sautéed onion and tomato mixture, and then pour the creamy Greek yogurt-egg mixture over the hot ingredients. Toss vigorously and continuously for 1-2 minutes, using tongs, ensuring the sauce coats all the pasta evenly and the residual heat from the pasta gently cooks and thickens the egg into a creamy, cohesive coating. If the sauce seems too thick, add a tablespoon or two of the reserved pasta water until the desired creamy consistency is reached. Stir in 1/2 cup of chopped fresh chives just before serving."}],"notes":"The key to the creamy sauce is to add the yogurt-egg mixture to the *hot* pasta immediately after draining, then toss quickly and continuously. The residual heat will gently cook the egg without scrambling it, creating a smooth, rich coating. Do not return the pot to direct heat once the yogurt-egg mixture is added."}

View File

@@ -0,0 +1,131 @@
{
"title": "Spicy Salmon Poke Bowl",
"description": "A vibrant and flavorful poke bowl featuring crispy air-fried salmon, seasoned sushi rice, and a colorful array of fresh vegetables and pickled red onions. Quick to prepare and perfect for a healthy meal.",
"servings": 2,
"prepTime": "25 minutes",
"cookTime": "15 minutes",
"ingredients": [
{
"amount": 1,
"unit": "lb",
"name": "Salmon fillet, skin removed"
},
{
"amount": 0.25,
"unit": "cup",
"name": "Spicy mayo"
},
{
"amount": 1,
"unit": "tbsp",
"name": "Chili garlic oil"
},
{
"amount": null,
"unit": "to taste",
"name": "Salt"
},
{
"amount": null,
"unit": "to taste",
"name": "Freshly ground black pepper"
},
{
"amount": 2,
"unit": "cups",
"name": "Cooked sushi rice"
},
{
"amount": 1,
"unit": "tbsp",
"name": "Apple cider vinegar"
},
{
"amount": 1,
"unit": "tsp",
"name": "Sugar (for rice)"
},
{
"amount": 1,
"unit": "",
"name": "Red onion, thinly sliced"
},
{
"amount": 0.5,
"unit": "cup",
"name": "Red wine vinegar (for pickling)"
},
{
"amount": 1,
"unit": "tbsp",
"name": "Sugar (for pickling)"
},
{
"amount": 0.5,
"unit": "cup",
"name": "Water (for pickling)"
},
{
"amount": 2,
"unit": "",
"name": "Cucumbers, diced"
},
{
"amount": 0.25,
"unit": "head",
"name": "Red cabbage, shredded"
},
{
"amount": 2,
"unit": "",
"name": "Green onions, chopped"
},
{
"amount": 1,
"unit": "",
"name": "Mango, peeled and diced"
},
{
"amount": 1,
"unit": "",
"name": "Avocado, sliced"
},
{
"amount": null,
"unit": "",
"name": "Crispy fried onions (for topping)"
},
{
"amount": null,
"unit": "",
"name": "Black sesame seeds (for topping)"
}
],
"steps": [
{
"title": "Prepare Pickled Red Onions",
"instruction": "Thinly slice the red onion. In a jar, combine red wine vinegar, 1 tablespoon of sugar, and water. Stir until the sugar dissolves. Add the sliced red onions to the jar, seal, and set aside to pickle while you prepare the other ingredients."
},
{
"title": "Cook and Season Sushi Rice",
"instruction": "Cook sushi rice according to package instructions. Once cooked, transfer it to a bowl and gently mix in the apple cider vinegar and 1 teaspoon of sugar. Set aside."
},
{
"title": "Prepare and Air Fry Salmon",
"instruction": "Cut the salmon fillet into 1-inch cubes. In a mixing bowl, combine the salmon cubes with spicy mayo, chili garlic oil, salt, and freshly ground black pepper. Toss until the salmon is evenly coated. Transfer the salmon to an air fryer basket and air fry at 400°F (200°C) for 10-12 minutes, or until crispy and cooked through, shaking the basket halfway."
},
{
"title": "Prepare Fresh Vegetables and Fruit",
"instruction": "While the salmon cooks, dice the cucumbers, shred the red cabbage, chop the green onions, peel and dice the mango, and slice the avocado."
},
{
"title": "Assemble Poke Bowls",
"instruction": "In two serving bowls, create a base with the seasoned sushi rice. Arrange the cooked salmon, pickled red onions, diced cucumbers, shredded red cabbage, chopped green onions, diced mango, and sliced avocado around the rice."
},
{
"title": "Garnish and Serve",
"instruction": "Drizzle with additional spicy mayo and chili garlic oil if desired. Garnish with crispy fried onions and black sesame seeds. Serve immediately and enjoy!"
}
],
"notes": "For best results, let the red onions pickle for at least 30 minutes, or even a few hours, to develop their flavor. Adjust the amount of spicy mayo and chili garlic oil to your preferred spice level."
}

249
server.js Normal file
View File

@@ -0,0 +1,249 @@
import express from 'express';
import path from 'path';
import { fileURLToPath } from 'url';
import dotenv from 'dotenv';
import fs from 'fs/promises';
import { existsSync } from 'fs';
import crypto from 'crypto';
import { execFile } from 'child_process';
import util from 'util';
import { GoogleGenerativeAI } from '@google/generative-ai';
import { GoogleAIFileManager } from '@google/generative-ai/server';
const execFilePromise = util.promisify(execFile);
dotenv.config();
const __filename = fileURLToPath(import.meta.url);
const __dirname = path.dirname(__filename);
const RECIPES_DIR = path.join(__dirname, 'recipes');
const TEMP_DIR = path.join(__dirname, 'temp_ingest');
// Ensure directories exist
async function ensureDirs() {
try { await fs.access(RECIPES_DIR); } catch { await fs.mkdir(RECIPES_DIR); }
try { await fs.access(TEMP_DIR); } catch { await fs.mkdir(TEMP_DIR); }
}
ensureDirs();
const app = express();
app.use(express.json());
// Serve static frontend files from Vite build
app.use(express.static(path.join(__dirname, 'dist')));
// Secure credentials strictly from environment variables
const VALID_USER = process.env.APP_USERNAME;
const VALID_PASS = process.env.APP_PASSWORD;
const GEMINI_API_KEY = process.env.VITE_GEMINI_API_KEY || process.env.GEMINI_API_KEY;
app.post('/api/login', (req, res) => {
const { username, password } = req.body;
if (username === VALID_USER && password === VALID_PASS) {
res.json({ success: true });
} else {
res.status(401).json({ error: 'Invalid credentials' });
}
});
app.post('/api/generate', async (req, res) => {
const { username, password, userPrompt, systemPrompt } = req.body;
if (username !== VALID_USER || password !== VALID_PASS) {
return res.status(401).json({ error: 'Unauthorized' });
}
if (!GEMINI_API_KEY) {
return res.status(500).json({ error: 'API key not configured on server' });
}
try {
const resp = await fetch(`https://generativelanguage.googleapis.com/v1beta/models/gemini-2.5-flash:generateContent?key=${GEMINI_API_KEY}`, {
method: 'POST',
headers: { 'content-type': 'application/json' },
body: JSON.stringify({
systemInstruction: { parts: [{ text: systemPrompt }] },
contents: [{ parts: [{ text: userPrompt }] }],
generationConfig: { responseMimeType: "application/json" }
})
});
const data = await resp.json();
if (!resp.ok) {
throw new Error(data.error?.message || `API error ${resp.status}`);
}
res.json(data);
} catch (err) {
res.status(500).json({ error: err.message });
}
});
app.post('/api/ingest', async (req, res) => {
const { username, password, url } = req.body;
if (username !== VALID_USER || password !== VALID_PASS) {
return res.status(401).json({ error: 'Unauthorized' });
}
if (!GEMINI_API_KEY) {
return res.status(500).json({ error: 'API key not configured on server' });
}
if (!url) return res.status(400).json({ error: 'Reel URL is required' });
const jobId = crypto.randomBytes(8).toString('hex');
const jobDir = path.join(TEMP_DIR, jobId);
try {
await fs.mkdir(jobDir, { recursive: true });
console.log(`[Ingest ${jobId}] Step 1: Running yt-dlp on ${url}`);
const videoOutputTemplate = path.join(jobDir, 'video.%(ext)s');
const descFile = path.join(jobDir, 'video.description');
await execFilePromise('yt-dlp', [
'-f', 'best[ext=mp4]/best',
'--no-playlist',
'-o', videoOutputTemplate,
'--write-description',
url
]);
const dirFiles = await fs.readdir(jobDir);
const videoFile = dirFiles.find(f => /^video\.(mp4|webm|mkv|mov|avi)$/i.test(f));
if (!videoFile) throw new Error('yt-dlp did not produce a video file');
const videoPath = path.join(jobDir, videoFile);
const ext = path.extname(videoFile).slice(1).toLowerCase();
const MIME_MAP = { mp4: 'video/mp4', webm: 'video/webm', mkv: 'video/x-matroska', mov: 'video/quicktime', avi: 'video/x-msvideo' };
const videoMimeType = MIME_MAP[ext] || 'video/mp4';
let captionText = '';
if (existsSync(descFile)) captionText = await fs.readFile(descFile, 'utf-8');
console.log(`[Ingest ${jobId}] Step 2: Uploading file to Gemini (${videoFile})...`);
const fileManager = new GoogleAIFileManager(GEMINI_API_KEY);
const uploadResult = await fileManager.uploadFile(videoPath, {
mimeType: videoMimeType,
displayName: `Reel Ingest ${jobId}`,
});
let file = await fileManager.getFile(uploadResult.file.name);
console.log(`[Ingest ${jobId}] Uploaded ${file.displayName} as: ${file.uri}`);
// Wait for the video processing to complete on Google's side
while (file.state === "PROCESSING") {
process.stdout.write(".");
await new Promise((resolve) => setTimeout(resolve, 5000));
file = await fileManager.getFile(uploadResult.file.name);
}
if (file.state === "FAILED") {
throw new Error("Video processing failed inside Gemini.");
}
console.log(`\n[Ingest ${jobId}] Step 3: Sending prompt to Gemini...`);
const genAI = new GoogleGenerativeAI(GEMINI_API_KEY);
const model = genAI.getGenerativeModel({ model: "gemini-2.5-flash" });
const systemPrompt = "You are a professional chef. You consume video, audio, and captions from a recipe reel, and output structured JSON. Required keys: title (string), description (string), servings (number), prepTime (string), cookTime (string), ingredients (array of {amount, unit, name} objects), steps (array of {title, instruction} objects), and notes (string).";
const result = await model.generateContent({
contents: [
{
role: 'user', parts: [
{ fileData: { fileUri: uploadResult.file.uri, mimeType: uploadResult.file.mimeType } },
{ text: "Extract the recipe (ingredients, measurements, and steps) into the required JSON format." },
...(captionText.trim() ? [{ text: `Original Social Media Caption:\n"${captionText}"\n` }] : [])
]
}
],
systemInstruction: { role: 'system', parts: [{ text: systemPrompt }] },
generationConfig: { responseMimeType: "application/json", temperature: 0.2 }
});
const recipeRaw = JSON.parse(result.response.text());
// Save securely
const recipeId = crypto.randomBytes(8).toString('hex');
await fs.writeFile(path.join(RECIPES_DIR, `${recipeId}.json`), JSON.stringify(recipeRaw, null, 2));
// Cleanup local temp dir and remote API file
await fs.rm(jobDir, { recursive: true, force: true });
await fileManager.deleteFile(uploadResult.file.name);
res.json({ id: recipeId, recipe: recipeRaw });
} catch (err) {
console.error(`[Ingest ${jobId}] Failed`, err);
// Try to cleanup jobDir just in case
try { await fs.rm(jobDir, { recursive: true, force: true }); } catch { }
res.status(500).json({ error: err.message });
}
});
app.post('/api/recipes', async (req, res) => {
const { username, password, recipe } = req.body;
if (username !== VALID_USER || password !== VALID_PASS) {
return res.status(401).json({ error: 'Unauthorized' });
}
try {
const id = crypto.randomBytes(8).toString('hex');
await fs.writeFile(path.join(RECIPES_DIR, `${id}.json`), JSON.stringify(recipe));
res.json({ id });
} catch (err) {
res.status(500).json({ error: 'Failed to save recipe' });
}
});
app.get('/api/recipes', async (req, res) => {
// Pass auth via headers
const username = req.headers['x-username'];
const password = req.headers['x-password'];
if (username !== VALID_USER || password !== VALID_PASS) {
return res.status(401).json({ error: 'Unauthorized' });
}
try {
const files = await fs.readdir(RECIPES_DIR);
const recipes = [];
for (const file of files) {
if (file.endsWith('.json')) {
const content = await fs.readFile(path.join(RECIPES_DIR, file), 'utf-8');
const parsed = JSON.parse(content);
parsed.id = file.replace('.json', '');
recipes.push(parsed);
}
}
res.json(recipes);
} catch (err) {
res.status(500).json({ error: 'Failed to fetch recipes' });
}
});
app.get('/api/recipes/:id', async (req, res) => {
// Public route - anyone with the ID can view
try {
const recipeId = req.params.id.replace(/[^a-z0-9]/gi, ''); // basic sanitization
const content = await fs.readFile(path.join(RECIPES_DIR, `${recipeId}.json`), 'utf-8');
res.json(JSON.parse(content));
} catch (err) {
res.status(404).json({ error: 'Recipe not found' });
}
});
// For any other route, send the frontend
app.get('*', (req, res) => {
res.sendFile(path.join(__dirname, 'dist', 'index.html'));
});
const PORT = process.env.PORT || 3000;
app.listen(PORT, () => {
console.log(`Secure proxy server running on port ${PORT}`);
});

View File

@@ -1,254 +0,0 @@
#!/usr/bin/env python3
"""
thumbnail.py — generate a link-preview card thumbnail from a screenshot.
Usage:
python3 thumbnail.py <screenshot> <url> <title> [-o output.png] [--width 800]
Example:
python3 thumbnail.py page.png "souschef.wahwa.com" "Chana Masala" -o thumb.png
"""
import argparse
import sys
try:
from PIL import Image, ImageDraw, ImageFont, ImageFilter
except ImportError:
sys.exit("Install Pillow: pip install Pillow")
# Render at 2× then downsample — gives crisp text and sharp edges.
RENDER_SCALE = 2
# ── visual constants (all at 1× display size) ─────────────────────────────────
BG_COLOR = (255, 251, 235) # warm cream
CARD_COLOR = (255, 255, 255)
URL_COLOR = (37, 99, 235) # blue-600
TITLE_COLOR = (17, 24, 39) # gray-900
DIVIDER_COLOR = (229, 231, 235) # gray-200
ICON_BG_COLOR = (240, 241, 243) # gray-100
ICON_FG_COLOR = (100, 108, 120) # gray-500
CORNER_RADIUS = 16
CARD_PADDING = 24
HEADER_V_PAD = 20
ICON_SIZE = 44 # circle diameter
TEXT_GAP = 14
CANVAS_PAD = 44
MAX_CONTENT_H = 640
# ── fonts ─────────────────────────────────────────────────────────────────────
def _load_font(paths, size):
for p in paths:
try:
return ImageFont.truetype(p, size)
except (IOError, OSError):
pass
return ImageFont.load_default()
def get_font(size, bold=False):
if bold:
paths = [
"/System/Library/Fonts/Supplemental/Arial Bold.ttf",
"/Library/Fonts/Arial Bold.ttf",
"/usr/share/fonts/truetype/dejavu/DejaVuSans-Bold.ttf",
"/usr/share/fonts/truetype/liberation/LiberationSans-Bold.ttf",
]
else:
paths = [
"/System/Library/Fonts/Supplemental/Arial.ttf",
"/Library/Fonts/Arial.ttf",
"/usr/share/fonts/truetype/dejavu/DejaVuSans.ttf",
"/usr/share/fonts/truetype/liberation/LiberationSans-Regular.ttf",
]
return _load_font(paths, size)
# ── text measurement ──────────────────────────────────────────────────────────
def tw(f, s):
bb = f.getbbox(s)
return bb[2] - bb[0]
def th(f, s="Ag"):
bb = f.getbbox(s)
return bb[3] - bb[1]
# ── paperclip icon ────────────────────────────────────────────────────────────
def draw_paperclip_icon(draw, cx, cy, r, color, lw):
"""
Two concentric rounded-rectangle outlines sharing the same top arc —
the classic paperclip silhouette.
"""
# Outer pill
ow = int(r * 0.50) # half-width
oh = int(r * 0.82) # half-height
draw.rounded_rectangle(
[cx - ow, cy - oh, cx + ow, cy + oh],
radius=ow, # = half-width → perfect semicircular ends
outline=color,
width=lw,
)
# Inner clip: same top, ends ~55% down
iw = int(r * 0.21)
i_top = cy - oh + lw + max(2, lw // 2) # just clear of outer top arc
i_bot = cy + int(r * 0.46)
draw.rounded_rectangle(
[cx - iw, i_top, cx + iw, i_bot],
radius=iw,
outline=color,
width=lw,
)
def draw_icon(draw, cx, cy, r):
"""Paperclip inside a light circle."""
draw.ellipse([cx - r, cy - r, cx + r, cy + r], fill=ICON_BG_COLOR)
lw = max(2, r // 9)
draw_paperclip_icon(draw, cx, cy, r, ICON_FG_COLOR, lw)
# ── external-link glyph ───────────────────────────────────────────────────────
def draw_ext_link(draw, x, y, s, color, lw):
"""
Classic external-link indicator: small box (lower-left) with an arrow
exiting through the upper-right corner.
"""
m = lw
# Box occupies the lower-left ~60% of the bounding square
bx2 = x + int(s * 0.62)
by1 = y + int(s * 0.38)
draw.rounded_rectangle(
[x + m, by1, bx2, y + s - m],
radius=max(1, s // 7),
outline=color,
width=lw,
)
# Diagonal shaft from inside the box toward upper-right
ax1, ay1 = x + int(s * 0.34), y + int(s * 0.66)
ax2, ay2 = x + s - m, y + m
draw.line([(ax1, ay1), (ax2, ay2)], fill=color, width=lw)
# Arrowhead (two orthogonal tails at the tip)
aw = int(s * 0.30)
draw.line([(ax2 - aw, ay2), (ax2, ay2)], fill=color, width=lw)
draw.line([(ax2, ay2), (ax2, ay2 + aw)], fill=color, width=lw)
# ── main ──────────────────────────────────────────────────────────────────────
def create_thumbnail(screenshot_path, url, title,
output_path="thumbnail.png", card_width=800):
S = RENDER_SCALE
screenshot = Image.open(screenshot_path).convert("RGB")
url_f = get_font(17 * S)
title_f = get_font(22 * S, bold=True)
uh = th(url_f, url)
th_ = th(title_f, title)
text_block_h = uh + 8 * S + th_
# Scaled geometry
CW = card_width * S
CP = CARD_PADDING * S
HVP = HEADER_V_PAD * S
IS = ICON_SIZE * S
TG = TEXT_GAP * S
CR = CORNER_RADIUS * S
CNVP = CANVAS_PAD * S
MCH = MAX_CONTENT_H * S
header_h = HVP + max(IS, text_block_h) + HVP
# Scale screenshot to fill card width; crop height
raw_h = int(screenshot.height * CW / screenshot.width)
content_h = min(raw_h, MCH)
ss = screenshot.resize((CW, raw_h), Image.LANCZOS).crop((0, 0, CW, content_h))
card_h = header_h + S + content_h
canvas_w = CW + CNVP * 2
canvas_h = card_h + CNVP * 2
canvas = Image.new("RGBA", (canvas_w, canvas_h), BG_COLOR + (255,))
# Drop shadow
shadow = Image.new("RGBA", (canvas_w, canvas_h), (0, 0, 0, 0))
sd = ImageDraw.Draw(shadow)
off = 6 * S
sd.rounded_rectangle(
[CNVP + off, CNVP + off, CNVP + CW + off, CNVP + card_h + off],
radius=CR, fill=(0, 0, 0, 38),
)
shadow = shadow.filter(ImageFilter.GaussianBlur(11 * S))
canvas.alpha_composite(shadow)
# White card
card_l = Image.new("RGBA", (canvas_w, canvas_h), (0, 0, 0, 0))
ImageDraw.Draw(card_l).rounded_rectangle(
[CNVP, CNVP, CNVP + CW, CNVP + card_h],
radius=CR, fill=CARD_COLOR + (255,),
)
canvas.alpha_composite(card_l)
draw = ImageDraw.Draw(canvas)
# Icon (vertically centred in header)
icx = CNVP + CP + IS // 2
icy = CNVP + header_h // 2
draw_icon(draw, icx, icy, IS // 2)
# URL text
tx = CNVP + CP + IS + TG
ty_url = CNVP + (header_h - text_block_h) // 2
draw.text((tx, ty_url), url, font=url_f, fill=URL_COLOR)
# External-link glyph (same height as URL text, a little larger)
uw = tw(url_f, url)
glyph_s = int(uh * 1.05)
glyph_lw = max(2, S)
draw_ext_link(
draw,
tx + uw + 6 * S,
ty_url + (uh - glyph_s) // 2,
glyph_s, URL_COLOR, glyph_lw,
)
# Title
draw.text((tx, ty_url + uh + 8 * S), title, font=title_f, fill=TITLE_COLOR)
# Divider
div_y = CNVP + header_h
draw.line([(CNVP, div_y), (CNVP + CW, div_y)], fill=DIVIDER_COLOR, width=S)
# Screenshot (edge-to-edge inside card)
rgb = canvas.convert("RGB")
rgb.paste(ss, (CNVP, div_y + S))
# Clip to rounded card shape
mask = Image.new("L", (canvas_w, canvas_h), 0)
ImageDraw.Draw(mask).rounded_rectangle(
[CNVP, CNVP, CNVP + CW, CNVP + card_h], radius=CR, fill=255,
)
bg = Image.new("RGB", (canvas_w, canvas_h), BG_COLOR)
bg.paste(rgb, mask=mask)
# Downsample to 1× for crisp output
out_w, out_h = canvas_w // S, canvas_h // S
bg.resize((out_w, out_h), Image.LANCZOS).save(output_path)
print(f"Saved: {output_path} ({out_w} × {out_h})")
if __name__ == "__main__":
ap = argparse.ArgumentParser(description="Generate a link-preview thumbnail card")
ap.add_argument("screenshot", help="Path to the input screenshot")
ap.add_argument("url", help='URL label, e.g. "souschef.wahwa.com"')
ap.add_argument("title", help="Page title displayed in bold")
ap.add_argument("-o", "--output", default="thumbnail.png",
help="Output image path (default: thumbnail.png)")
ap.add_argument("--width", type=int, default=800,
help="Card width in pixels (default: 800)")
args = ap.parse_args()
create_thumbnail(args.screenshot, args.url, args.title, args.output, args.width)

9
vite.config.js Normal file
View File

@@ -0,0 +1,9 @@
import { defineConfig } from 'vite';
export default defineConfig({
server: {
proxy: {
'/api': 'http://localhost:3000'
}
}
});