**Problem:**
AI was misinterpreting French decimal comma as thousands separator:
- "3,99 €" detected as 399.00 EUR (instead of 3.99 EUR)
- "1,99 €" detected as 199.00 EUR (instead of 1.99 EUR)
- Caused by AI reading comma as grouping separator, not decimal point
**Root Cause:**
French format uses comma for decimals: "3,99 €" = three euros ninety-nine cents
English format uses dot for decimals: "3.99 €" = three euros ninety-nine cents
AI models (trained mostly on English) interpret:
- "3,99" → "3" and "99" separate → 399 (three hundred ninety-nine)
- Should be: "3.99" → 3.99 (three point ninety-nine)
**Solution: Pre-Convert Format Before AI Processing**
1. **Normalize ALL Text** (browserless_service.py:380-386)
```python
# Convert French to English in all page text
content = re.sub(r'(\d+),(\d{2})\s*€', r'\1.\2 €', content)
# "3,99 €" → "3.99 €"
# "12,50 €" → "12.50 €"
# "1 234,56 €" → "1234.56 €" (also removes space thousands separator)
```
2. **Normalize Extracted Price** (browserless_service.py:400-410)
```python
# Convert French format in directly extracted price
normalized_price = re.sub(r'(\d+),(\d{2})', r'\1.\2', extracted_price)
# "3,99 €" → "3.99 €"
# Prepends: "PRIX DÉTECTÉ: 3.99 €" (not "3,99 €")
```
3. **Updated AI Prompt** (ai_schema.py:135-158)
- Removed instructions to convert comma→dot (now done automatically)
- Added: "The price has already been converted to English format for you"
- Added explicit warnings about common mistakes:
* "3.99 €" means 3.99 (NOT 399.00, NOT 3990.00)
* "1.99 €" means 1.99 (NOT 199.00, NOT 1990.00)
- Clear examples with both CORRECT and INCORRECT values
**Flow:**
```
Page text: "Lutin 3,99 € Ajouter au panier"
↓
Normalization: "Lutin 3.99 € Ajouter au panier"
↓
Extracted price: "3,99 €" → Normalized: "3.99 €"
↓
Sent to AI: "PRIX DÉTECTÉ: 3.99 €\n\nLutin 3.99 € Ajouter au panier"
↓
AI sees only English format: "3.99"
↓
AI Response: {"price": 3.99} ✅ CORRECT
```
**Expected Results:**
- Before: "3,99 €" → AI returns 399.00 ❌
- After: "3,99 €" → Normalized to "3.99 €" → AI returns 3.99 ✅
**Regex Details:**
- `(\d+),(\d{2})\s*€` matches French prices: captures digits before comma, 2 digits after
- `\1.\2 €` replaces with: same digits, dot instead of comma, space + euro symbol
- Handles: 0,99 / 3,99 / 12,99 / 89,99 / 1234,99
Fixes issue where AI misread French decimal comma causing 100x price errors.
**Problem:**
AI was misreading prices: "1,99 €" detected as "9,99 €" with 90% confidence.
This is a critical OCR error where similar-looking digits (1, 7, 8, 9) are confused.
The model google/gemini-2.5-flash-lite made basic digit recognition mistakes.
**Root Cause:**
- AI vision models can confuse visually similar digits
- No specific instructions in prompt to verify digit accuracy
- No validation that detected price matches text context
- Model may be too lightweight for reliable OCR
**Solution: Three-Part Fix**
1. **Enhanced Prompt with Digit Verification** (ai_schema.py:145-149)
- Added "CRITICAL - Read digits carefully" section
- Explicit warning: "1,99 €" is NOT "9,99 €"
- Instructions to double-check digits 1, 7, 8, 9 (visually similar)
- Context check: Small prices (< 5€) are common
- Verify price makes sense for product type
2. **More Examples in Prompt** (ai_schema.py:152-153)
- Added: "1,99 €" -> 1.99 (NOT 9.99)
- Added: "0,99 €" -> 0.99 (NOT 9.99)
- Reinforces correct parsing of small prices
3. **Price Extraction Logging** (ai_service.py:349-355)
- Extract ALL prices from text with regex: \d+[,\.]\d{2}\s*€
- Log first 10 prices found: ["1,99 €", "2,99 €", ...]
- Allows debugging: Is correct price in text?
- Logs warning if NO prices found in text
4. **Debug Logs to INFO** (ai_service.py:347, 361)
- Changed logger.debug → logger.info for text/prompt preview
- Now visible in production logs for diagnosis
**Expected Improvement:**
- AI should now pay closer attention to digit shapes
- Lower confidence (0.5-0.7) if digits are unclear
- Better accuracy for small prices (0,99 - 4,99)
- Logs will show if problem is in text extraction or AI vision
**Testing:**
After restart, logs will show:
```
INFO - Cleaned text preview: 'Lutin de Noël 1,99 € Ajouter...'
INFO - Prices found in text: ['1,99 €', '2,99 €']
INFO - AI Response: {"price": 1.99, "price_confidence": 0.85}
```
If AI still returns wrong price but text has correct price,
then model is insufficient and should be upgraded to:
- google/gemini-2.0-flash-exp
- anthropic/claude-3-haiku
- openai/gpt-4o-mini
Critical fix for Amazon price detection - AI was returning price=null with 0.0 confidence
even though stock detection worked (1.0 confidence).
**Root Cause:**
- Amazon injects prices via JavaScript into hidden .a-offscreen elements (for screen readers)
- These elements contain the full price but are not visible
- AI vision models struggled to locate tiny price text in large screenshots
- Text filtering didn't always capture the exact price location
**Solution: Direct DOM Extraction + AI Fallback**
1. **New Method: _extract_amazon_price()** (browserless_service.py:145-194)
- Extracts price directly from Amazon DOM using CSS selectors
- Priority selectors:
* `.a-price .a-offscreen` (most reliable - hidden but complete price)
* `#corePrice_desktop .a-price .a-offscreen`
* `#corePriceDisplay_desktop_feature_div .a-price .a-offscreen`
* `#priceblock_ourprice` (older layouts)
* Fallback: `span.a-price-whole` + `span.a-price-fraction`
- Handles hidden elements (.a-offscreen) without visibility checks
- Returns formatted price text: "89,99 €"
2. **Price Injection into Context** (browserless_service.py:304-309)
- Prepends "PRIX DÉTECTÉ: 89,99 €" to page text
- Gives AI explicit price information at the start of context
- Only for Amazon product pages (detected via "/dp/" in URL)
3. **Improved AI Prompt** (ai_schema.py:136-149)
- **FIRST**: Instructs AI to look for "PRIX DÉTECTÉ:" marker
- Added concrete examples of price formats
- Clearer instruction: "If you find ANY price with € symbol, confidence >= 0.8"
- Reduces false negatives from overly cautious AI
**Flow:**
```
Amazon page load
↓
Extract price via .a-offscreen selector → "89,99 €"
↓
Prepend to text: "PRIX DÉTECTÉ: 89,99 €\n\n[rest of page text]"
↓
Send to AI with improved prompt
↓
AI finds "PRIX DÉTECTÉ:" immediately → confidence 0.8-1.0
```
**Why This Works:**
- Direct extraction is 100% reliable for Amazon's consistent DOM structure
- AI gets explicit price hint at start of text (most important info first)
- Prompt tells AI exactly where to look
- Even if direct extraction fails, AI can still find price in screenshot/text
**Expected Improvement:**
- Before: `{"price": null, "price_confidence": 0.0}`
- After: `{"price": 89.99, "price_confidence": 0.95}`
Tested with Amazon.fr MSI Mag product page.
- Change default currency from USD to EUR for all French sites
- Update AI extraction prompt to use EUR by default
- Fix E.Leclerc search URL pattern
- Improve product selectors for Auchan, Carrefour, Amazon France
- Add Amazon France language parameter for better results
- Enhance cookie acceptance with site-specific selectors:
- Amazon (#sp-cc-accept)
- E.Leclerc (#onetrust-accept-btn-handler)
- Auchan (#popin_tc_privacy_button_2, #didomi-notice-agree-button)
- Carrefour (#onetrust-accept-btn-handler)
- Cdiscount (#footer_tc_privacy_button_2)
- And more CMP platforms (Didomi, OneTrust, Tarteaucitron, etc.)
- Improve cookie handling with iframe support and retry logic
Michael
Claude
LogiFlow