Guido.Tech/ai_intelligence_layer/WEBHOOK_INTEGRATION.md

Webhook Push Integration Guide

Overview

The AI Intelligence Layer supports two integration models for receiving enriched telemetry:

1. Push Model (Webhook) - Enrichment service POSTs data to AI layer ✅ RECOMMENDED
2. Pull Model - AI layer fetches data from enrichment service (fallback)

Push Model (Webhook) - How It Works

┌─────────────────────┐         ┌─────────────────────┐
│  HPC Enrichment     │  POST   │  AI Intelligence    │
│  Service            │────────▶│  Layer              │
│  (Port 8000)        │         │  (Port 9000)        │
└─────────────────────┘         └─────────────────────┘
                                         │
                                         ▼
                                 ┌──────────────┐
                                 │ Telemetry    │
                                 │ Buffer       │
                                 │ (in-memory)  │
                                 └──────────────┘
                                         │
                                         ▼
                                 ┌──────────────┐
                                 │ Brainstorm   │
                                 │ & Analyze    │
                                 │ (Gemini AI)  │
                                 └──────────────┘

Configuration

In your enrichment service (port 8000), set the callback URL:

export NEXT_STAGE_CALLBACK_URL=http://localhost:9000/api/ingest/enriched

When enrichment is complete for each lap, the service will POST to this endpoint.

Webhook Endpoint

Endpoint: POST /api/ingest/enriched

Request Body: Single enriched telemetry record (JSON)

{
  "lap": 27,
  "lap_time_seconds": 78.456,
  "tire_degradation_index": 0.72,
  "fuel_remaining_kg": 45.2,
  "aero_efficiency": 0.85,
  "ers_recovery_rate": 0.78,
  "brake_wear_index": 0.65,
  "fuel_optimization_score": 0.82,
  "driver_consistency": 0.88,
  "predicted_tire_cliff_lap": 35,
  "weather_impact": "minimal",
  "hpc_simulation_id": "sim_monaco_lap27_001",
  "metadata": {
    "simulation_timestamp": "2025-10-18T22:15:30Z",
    "confidence_level": 0.92,
    "cluster_nodes_used": 8
  }
}

Response:

{
  "status": "received",
  "lap": 27,
  "buffer_size": 15
}

Buffer Behavior

• Max Size: 100 records (configurable)
• Storage: In-memory (cleared on restart)
• Retrieval: FIFO - newest data returned first
• Auto-cleanup: Oldest records dropped when buffer is full

Testing the Webhook

1. Start the AI Intelligence Layer

cd ai_intelligence_layer
source myenv/bin/activate  # or your venv
python main.py

Verify it's running:

curl http://localhost:9000/api/health

2. Simulate Enrichment Service Pushing Data

Option A: Using the test script

# Post single telemetry record
python3 test_webhook_push.py

# Post 10 records with 2s delay between each
python3 test_webhook_push.py --loop 10 --delay 2

# Post 5 records with 1s delay
python3 test_webhook_push.py --loop 5 --delay 1

Option B: Using curl

curl -X POST http://localhost:9000/api/ingest/enriched \
  -H "Content-Type: application/json" \
  -d '{
    "lap": 27,
    "lap_time_seconds": 78.456,
    "tire_degradation_index": 0.72,
    "fuel_remaining_kg": 45.2,
    "aero_efficiency": 0.85,
    "ers_recovery_rate": 0.78,
    "brake_wear_index": 0.65,
    "fuel_optimization_score": 0.82,
    "driver_consistency": 0.88,
    "predicted_tire_cliff_lap": 35,
    "weather_impact": "minimal",
    "hpc_simulation_id": "sim_monaco_lap27_001",
    "metadata": {
      "simulation_timestamp": "2025-10-18T22:15:30Z",
      "confidence_level": 0.92,
      "cluster_nodes_used": 8
    }
  }'

3. Verify Buffer Contains Data

Check the logs - you should see:

INFO - Received enriched telemetry webhook: lap 27
INFO - Added telemetry for lap 27 (buffer size: 1)

4. Test Strategy Generation Using Buffered Data

Brainstorm endpoint (no telemetry in request = uses buffer):

curl -X POST http://localhost:9000/api/strategy/brainstorm \
  -H "Content-Type: application/json" \
  -d '{
    "race_context": {
      "race_info": {
        "track_name": "Monaco",
        "current_lap": 27,
        "total_laps": 58,
        "weather_condition": "Dry",
        "track_temp_celsius": 42
      },
      "driver_state": {
        "driver_name": "Hamilton",
        "current_position": 4,
        "current_tire_compound": "medium",
        "tire_age_laps": 14,
        "fuel_remaining_percent": 47
      },
      "competitors": []
    }
  }' | python3 -m json.tool

Check logs for:

INFO - Using 10 telemetry records from webhook buffer

Pull Model (Fallback)

If the buffer is empty and no telemetry is provided in the request, the AI layer will automatically fetch from the enrichment service:

GET http://localhost:8000/enriched?limit=10

This ensures the system works even without webhook configuration.

Priority Order

When brainstorm/analyze endpoints are called:

1. Check request body - Use enriched_telemetry if provided
2. Check buffer - Use webhook buffer if it has data
3. Fetch from service - Pull from enrichment service as fallback
4. Error - If all fail, return 400 error

Production Recommendations

For Enrichment Service

# Configure callback URL
export NEXT_STAGE_CALLBACK_URL=http://ai-layer:9000/api/ingest/enriched

# Add retry logic (recommended)
export CALLBACK_MAX_RETRIES=3
export CALLBACK_TIMEOUT=10

For AI Layer

# config.py - Increase buffer size for production
telemetry_buffer_max_size: int = 500  # Store more history

# Consider Redis for persistent buffer
# (current implementation is in-memory only)

Health Monitoring

# Check buffer status
curl http://localhost:9000/api/health

# Response includes buffer info (could be added):
{
  "status": "healthy",
  "buffer_size": 25,
  "buffer_max_size": 100
}

Common Issues

1. Webhook Not Receiving Data

Symptoms: Buffer size stays at 0

Solutions:

• Verify enrichment service has NEXT_STAGE_CALLBACK_URL configured
• Check network connectivity between services
• Examine enrichment service logs for POST errors
• Confirm AI layer is running on port 9000

2. Old Data in Buffer

Symptoms: AI uses outdated telemetry

Solutions:

• Buffer is FIFO - automatically clears old data
• Restart AI service to clear buffer
• Increase buffer size if race generates data faster than consumption

3. Pull Model Used Instead of Push

Symptoms: Logs show "fetching from enrichment service" instead of "using buffer"

Solutions:

• Confirm webhook is posting data (check buffer size in logs)
• Verify webhook POST is successful (200 response)
• Check if buffer was cleared (restart)

Integration Examples

Python (Enrichment Service)

import httpx

async def push_enriched_telemetry(telemetry_data: dict):
    """Push enriched telemetry to AI layer."""
    url = "http://localhost:9000/api/ingest/enriched"
    async with httpx.AsyncClient() as client:
        response = await client.post(url, json=telemetry_data, timeout=10.0)
        response.raise_for_status()
        return response.json()

Shell Script (Testing)

#!/bin/bash
# push_telemetry.sh

for lap in {1..10}; do
  curl -X POST http://localhost:9000/api/ingest/enriched \
    -H "Content-Type: application/json" \
    -d "{\"lap\": $lap, \"tire_degradation_index\": 0.7, ...}"
  sleep 2
done

Benefits of Push Model

✅ Real-time - AI layer receives data immediately as enrichment completes
✅ Efficient - No polling, reduces load on enrichment service
✅ Decoupled - Services don't need to coordinate timing
✅ Resilient - Buffer allows AI to process multiple requests from same dataset
✅ Simple - Enrichment service just POST and forget

---

Next Steps:

1. Configure NEXT_STAGE_CALLBACK_URL in enrichment service
2. Test webhook with test_webhook_push.py
3. Monitor logs to confirm push model is working
4. Run brainstorm/analyze and verify buffer usage