Commit Graph
8 Commits
Author SHA1 Message Date
Claude 0529c885d3 fix: Improve AI price digit recognition to avoid 1.99/9.99 confusion
**Problem:**
AI was misreading prices: "1,99 €" detected as "9,99 €" with 90% confidence.
This is a critical OCR error where similar-looking digits (1, 7, 8, 9) are confused.
The model google/gemini-2.5-flash-lite made basic digit recognition mistakes.

**Root Cause:**
- AI vision models can confuse visually similar digits
- No specific instructions in prompt to verify digit accuracy
- No validation that detected price matches text context
- Model may be too lightweight for reliable OCR

**Solution: Three-Part Fix**

1. **Enhanced Prompt with Digit Verification** (ai_schema.py:145-149)
   - Added "CRITICAL - Read digits carefully" section
   - Explicit warning: "1,99 €" is NOT "9,99 €"
   - Instructions to double-check digits 1, 7, 8, 9 (visually similar)
   - Context check: Small prices (< 5€) are common
   - Verify price makes sense for product type

2. **More Examples in Prompt** (ai_schema.py:152-153)
   - Added: "1,99 €" -> 1.99 (NOT 9.99)
   - Added: "0,99 €" -> 0.99 (NOT 9.99)
   - Reinforces correct parsing of small prices

3. **Price Extraction Logging** (ai_service.py:349-355)
   - Extract ALL prices from text with regex: \d+[,\.]\d{2}\s*€
   - Log first 10 prices found: ["1,99 €", "2,99 €", ...]
   - Allows debugging: Is correct price in text?
   - Logs warning if NO prices found in text

4. **Debug Logs to INFO** (ai_service.py:347, 361)
   - Changed logger.debug → logger.info for text/prompt preview
   - Now visible in production logs for diagnosis

**Expected Improvement:**
- AI should now pay closer attention to digit shapes
- Lower confidence (0.5-0.7) if digits are unclear
- Better accuracy for small prices (0,99 - 4,99)
- Logs will show if problem is in text extraction or AI vision

**Testing:**
After restart, logs will show:
```
INFO - Cleaned text preview: 'Lutin de Noël 1,99 € Ajouter...'
INFO - Prices found in text: ['1,99 €', '2,99 €']
INFO - AI Response: {"price": 1.99, "price_confidence": 0.85}
```

If AI still returns wrong price but text has correct price,
then model is insufficient and should be upgraded to:
- google/gemini-2.0-flash-exp
- anthropic/claude-3-haiku
- openai/gpt-4o-mini
2025-11-29 23:36:47 +00:00
Claude 12b281f609 feat: Extract visible text and optimize Amazon price detection
Critical improvements to AI price extraction for better accuracy:

**1. Extract Visible Text Instead of HTML** (browserless_service.py)
- Changed from `page.content()` (raw HTML) to `page.inner_text('body')`
- Now captures text as rendered by JavaScript, not HTML source
- Amazon and other sites inject prices via JavaScript - HTML source doesn't contain them
- Includes proper fallback to HTML if inner_text fails

**2. Amazon-Specific Price Detection** (browserless_service.py)
- Wait for Amazon price elements to load before screenshot:
  - `.a-price .a-offscreen` (main price)
  - `#corePriceDisplay_desktop_feature_div` (price section)
  - `#corePrice_desktop` (alternative container)
  - `.a-price-whole` (price number)
- Prevents screenshots before dynamic prices are rendered

**3. Focused Amazon Screenshots** (browserless_service.py)
- Problem: Full-page Amazon screenshots are huge (multiple screens tall)
- When resized to 2048px, price text becomes tiny and unreadable
- Solution: Take focused screenshot of product container only:
  - Try `#dp-container`, `#ppd`, or `#centerCol` (product areas)
  - Falls back to viewport screenshot (better than full-page)
- Other sites still use full-page screenshots

**4. Enhanced Debug Logging** (ai_service.py)
- Log cleaned text preview (first 500 chars) to verify content
- Log prompt preview to debug AI instructions
- Warning when no page_text provided

These changes address the root causes of AI returning:
- price: null, confidence: 0.0-0.1 (too low to update DB)

Expected improvement:
- Visible text contains actual rendered prices
- Smaller, focused screenshots = better AI recognition
- Confidence scores should reach > 0.5 threshold
2025-11-29 19:45:00 +00:00
Michael f665a000af feat: add AI service for content extraction and Browserless service for web scraping 2025-11-29 09:46:29 +01:00
Michael 867274a9ab feat: Introduce AI service for data extraction, including dynamic configuration, response parsing, and repair. 2025-11-29 09:27:35 +01:00
Michael b6c1fc1cda feat: Add AI-powered data extraction and direct search services 2025-11-29 02:53:35 +01:00
Michael 394fa3c118 feat: implement AI service for data extraction with dynamic configuration and robust response handling. 2025-11-29 02:00:42 +01:00
Claude d6c39252f2 feat: Rebranding PriceFlow + OpenRouter + i18n FR + Docker
Modifications majeures:

1. Rebranding: Pricecious → PriceFlow
   - Mise à jour du titre, logo et noms de projet

2. Intégration OpenRouter
   - Ajout comme provider IA avec filtres (chat, vision, code, reasoning, free)
   - Récupération dynamique des modèles et coûts
   - Service et router dédiés (/api/openrouter/models)

3. Internationalisation (i18n)
   - Installation i18next + react-i18next
   - Application entièrement traduite en français
   - Fichier de traduction: fr.json

4. Configuration Docker
   - docker-compose.yml avec réseau nginx_default (external)
   - PostgreSQL sur port 5488:5432
   - Browserless sur port 3012:3000
   - init.sql pour initialisation de la base de données
   - .env.example avec variables documentées

5. Favicon
   - Documentation ajoutée dans index.html pour faciliter le remplacement
2025-11-22 09:22:39 +00:00
LogiFlow afed10abc5 Add files via upload 2025-11-22 10:01:25 +01:00