Critical improvements to AI price extraction for better accuracy:
**1. Extract Visible Text Instead of HTML** (browserless_service.py)
- Changed from `page.content()` (raw HTML) to `page.inner_text('body')`
- Now captures text as rendered by JavaScript, not HTML source
- Amazon and other sites inject prices via JavaScript - HTML source doesn't contain them
- Includes proper fallback to HTML if inner_text fails
**2. Amazon-Specific Price Detection** (browserless_service.py)
- Wait for Amazon price elements to load before screenshot:
- `.a-price .a-offscreen` (main price)
- `#corePriceDisplay_desktop_feature_div` (price section)
- `#corePrice_desktop` (alternative container)
- `.a-price-whole` (price number)
- Prevents screenshots before dynamic prices are rendered
**3. Focused Amazon Screenshots** (browserless_service.py)
- Problem: Full-page Amazon screenshots are huge (multiple screens tall)
- When resized to 2048px, price text becomes tiny and unreadable
- Solution: Take focused screenshot of product container only:
- Try `#dp-container`, `#ppd`, or `#centerCol` (product areas)
- Falls back to viewport screenshot (better than full-page)
- Other sites still use full-page screenshots
**4. Enhanced Debug Logging** (ai_service.py)
- Log cleaned text preview (first 500 chars) to verify content
- Log prompt preview to debug AI instructions
- Warning when no page_text provided
These changes address the root causes of AI returning:
- price: null, confidence: 0.0-0.1 (too low to update DB)
Expected improvement:
- Visible text contains actual rendered prices
- Smaller, focused screenshots = better AI recognition
- Confidence scores should reach > 0.5 threshold
Three critical fixes to resolve AI extraction returning null prices with 0.0 confidence:
1. **Add French price/stock keywords** (app/utils/text.py)
- Added € symbol and French price patterns (12,99 €, 1 234,56 €)
- Added French keywords: prix, coût, promotion, réduction
- Added French stock keywords: ajouter au panier, en stock, rupture, épuisé
- Previously only detected $ and English keywords
2. **Enable full-page screenshots** (app/services/browserless_service.py)
- Changed full_page=False to full_page=True
- Now captures entire page instead of viewport-only
- Prevents missing prices located below the fold
3. **Increase image resolution** (app/utils/image.py)
- Increased MAX_IMAGE_SIZE from 1024px to 2048px
- Preserves fine details in price text for better AI recognition
- Improves readability on dense product pages
These changes fix the issue where AI models returned:
{"price": null, "price_confidence": 0.0, "in_stock": null, "in_stock_confidence": 0.0}
Tested with Amazon.fr product monitoring.
Claude
Michael