fix: Convert French price format to English before sending to AI

**Problem:**
AI was misinterpreting French decimal comma as thousands separator:
- "3,99 €" detected as 399.00 EUR (instead of 3.99 EUR)
- "1,99 €" detected as 199.00 EUR (instead of 1.99 EUR)
- Caused by AI reading comma as grouping separator, not decimal point

**Root Cause:**
French format uses comma for decimals: "3,99 €" = three euros ninety-nine cents
English format uses dot for decimals: "3.99 €" = three euros ninety-nine cents

AI models (trained mostly on English) interpret:
- "3,99" → "3" and "99" separate → 399 (three hundred ninety-nine)
- Should be: "3.99" → 3.99 (three point ninety-nine)

**Solution: Pre-Convert Format Before AI Processing**

1. **Normalize ALL Text** (browserless_service.py:380-386)
   ```python
   # Convert French to English in all page text
   content = re.sub(r'(\d+),(\d{2})\s*€', r'\1.\2 €', content)
   # "3,99 €" → "3.99 €"
   # "12,50 €" → "12.50 €"
   # "1 234,56 €" → "1234.56 €" (also removes space thousands separator)
   ```

2. **Normalize Extracted Price** (browserless_service.py:400-410)
   ```python
   # Convert French format in directly extracted price
   normalized_price = re.sub(r'(\d+),(\d{2})', r'\1.\2', extracted_price)
   # "3,99 €" → "3.99 €"
   # Prepends: "PRIX DÉTECTÉ: 3.99 €" (not "3,99 €")
   ```

3. **Updated AI Prompt** (ai_schema.py:135-158)
   - Removed instructions to convert comma→dot (now done automatically)
   - Added: "The price has already been converted to English format for you"
   - Added explicit warnings about common mistakes:
     * "3.99 €" means 3.99 (NOT 399.00, NOT 3990.00)
     * "1.99 €" means 1.99 (NOT 199.00, NOT 1990.00)
   - Clear examples with both CORRECT and INCORRECT values

**Flow:**
```
Page text: "Lutin 3,99 € Ajouter au panier"
  ↓
Normalization: "Lutin 3.99 € Ajouter au panier"
  ↓
Extracted price: "3,99 €" → Normalized: "3.99 €"
  ↓
Sent to AI: "PRIX DÉTECTÉ: 3.99 €\n\nLutin 3.99 € Ajouter au panier"
  ↓
AI sees only English format: "3.99"
  ↓
AI Response: {"price": 3.99}  ✅ CORRECT
```

**Expected Results:**
- Before: "3,99 €" → AI returns 399.00 ❌
- After: "3,99 €" → Normalized to "3.99 €" → AI returns 3.99 ✅

**Regex Details:**
- `(\d+),(\d{2})\s*€` matches French prices: captures digits before comma, 2 digits after
- `\1.\2 €` replaces with: same digits, dot instead of comma, space + euro symbol
- Handles: 0,99 / 3,99 / 12,99 / 89,99 / 1234,99

Fixes issue where AI misread French decimal comma causing 100x price errors.
This commit is contained in:
Claude committed 2025-11-29 23:51:00 +00:00
1 parent b4dd4b4365
commit 531ed1bc15
2 files changed
+35 -15

No files matched your search

+20 -2
View File
@@ -376,6 +376,15 @@ class BrowserlessService:
try:
content = await page.inner_text('body')
logger.info(f"Extracted {len(content)} chars of visible text from page")
# Normalize French prices to English format for AI (3,99 € → 3.99 €)
import re
# Replace comma with dot in price patterns: digits,XX € → digits.XX €
content = re.sub(r'(\d+),(\d{2})\s*€', r'\1.\2 €', content)
# Remove spaces in thousands: 1 234.56 → 1234.56
content = re.sub(r'(\d+)\s(\d{3})', r'\1\2', content)
logger.debug("Normalized French price formats to English")
except Exception as e:
# Fallback to HTML content if inner_text fails
logger.warning(f"Failed to extract inner_text, falling back to HTML: {e}")
@@ -397,8 +406,17 @@ class BrowserlessService:
# Prepend extracted price to content if found
if extracted_price:
content = f"PRIX DÉTECTÉ: {extracted_price}\n\n{content}"
logger.info(f"Prepended price to content: {extracted_price}")
# Convert French format to English for AI (3,99 → 3.99)
import re
normalized_price = extracted_price
# Replace comma with dot for decimal separator (French to English)
# Pattern: digits, comma, 2 digits → digits, dot, 2 digits
normalized_price = re.sub(r'(\d+),(\d{2})', r'\1.\2', normalized_price)
# Remove spaces in thousands separators if any (1 234,56 → 1234.56)
normalized_price = normalized_price.replace(' ', '')
content = f"PRIX DÉTECTÉ: {normalized_price}\n\n{content}"
logger.info(f"Prepended normalized price to content: {normalized_price} (original: {extracted_price})")
else:
logger.warning("Could not extract price from DOM, relying on text/image only")
else: