fix: Improve AI product price extraction for French e-commerce sites

Three critical fixes to resolve AI extraction returning null prices with 0.0 confidence:

1. **Add French price/stock keywords** (app/utils/text.py)
   - Added € symbol and French price patterns (12,99 €, 1 234,56 €)
   - Added French keywords: prix, coût, promotion, réduction
   - Added French stock keywords: ajouter au panier, en stock, rupture, épuisé
   - Previously only detected $ and English keywords

2. **Enable full-page screenshots** (app/services/browserless_service.py)
   - Changed full_page=False to full_page=True
   - Now captures entire page instead of viewport-only
   - Prevents missing prices located below the fold

3. **Increase image resolution** (app/utils/image.py)
   - Increased MAX_IMAGE_SIZE from 1024px to 2048px
   - Preserves fine details in price text for better AI recognition
   - Improves readability on dense product pages

These changes fix the issue where AI models returned:
{"price": null, "price_confidence": 0.0, "in_stock": null, "in_stock_confidence": 0.0}

Tested with Amazon.fr product monitoring.
This commit is contained in:
Claude committed 2025-11-29 19:07:04 +00:00
1 parent 7ca7fe630c
commit f2d45f9566
3 files changed
+34 -3

No files matched your search

+3 -2
View File
@@ -238,8 +238,9 @@ class BrowserlessService:
safe_name = "".join(c if c.isalnum() else "_" for c in url.split("//")[-1])[:50]
filename = f"{safe_name}_{timestamp}.jpg"
screenshot_path = f"{screenshots_dir}/{filename}"
await page.screenshot(path=screenshot_path, full_page=False, quality=80, type="jpeg")
# Use full_page=True to capture entire page, not just viewport
await page.screenshot(path=screenshot_path, full_page=True, quality=80, type="jpeg")
logger.info(f"Screenshot saved to {screenshot_path}")
except Exception as e:
logger.warning(f"Failed to take screenshot: {e}")
+2 -1
View File
@@ -8,7 +8,8 @@ from PIL import Image
logger = logging.getLogger(__name__)
# Image processing constants
MAX_IMAGE_SIZE = 1024
# Increased from 1024 to 2048 to preserve more detail for AI analysis
MAX_IMAGE_SIZE = 2048
JPEG_QUALITY = 85
+29
View File
@@ -69,6 +69,7 @@ def filter_relevant_text(text: str, max_length: int = 2000) -> str:
# Keywords to search for (case-insensitive)
price_keywords = [
# English patterns
r"\$\d+\.?\d*", # $XX.XX pattern
r"\d+\.\d{2}\s*(usd|eur|gbp|cad)", # XX.XX USD pattern
"price:",
@@ -78,9 +79,23 @@ def filter_relevant_text(text: str, max_length: int = 2000) -> str:
"save:",
"discount:",
r"\$", # Any dollar sign
# French patterns
r"€", # Euro symbol
r"\d+,\d{2}\s*€", # French format: 12,99 €
r"\d+\.\d{3},\d{2}", # French thousands: 1.234,56
r"\d+\s\d{3},\d{2}", # French thousands with space: 1 234,56
"prix", # French: price
"prix:",
"coût",
"coût:",
"promotion",
"réduction",
"économie",
"remise",
]
stock_keywords = [
# English keywords
"add to cart",
"buy now",
"purchase",
@@ -96,6 +111,20 @@ def filter_relevant_text(text: str, max_length: int = 2000) -> str:
"ships",
"delivery",
"get it by",
# French keywords
"ajouter au panier",
"acheter",
"commander",
"en stock",
"rupture",
"rupture de stock",
"disponible",
"indisponible",
"épuisé",
"être averti",
"précommande",
"livraison",
"expédié",
]
all_keywords = price_keywords + stock_keywords