feat: Improve price extraction with priority selectors and smart multi-price handling

**Problem:**
Stokomani Lutin product detected 14.99 EUR instead of 19.99 EUR.
Website shows both old price (14.99) and current price (19.99).
Generic extraction was returning whichever price was found first.

**Root Causes:**
1. No prioritization of price selectors (all treated equally)
2. First-found-first-returned approach missed semantic selectors
3. When multiple prices found, no logic to select the correct one

**Solution: Priority-Based Extraction + Smart Selection**

1. **Reordered Selectors by Priority** (browserless_service.py:151-170)
   ```
   HIGHEST PRIORITY (return immediately):
   - [itemprop='price']    (Schema.org semantic)
   - .current-price        (explicit current)
   - .sale-price           (sale/promo price)
   - .final-price          (final calculated)
   - .product-price        (product-specific)

   MEDIUM PRIORITY:
   - [class*='price']:not([class*='old']):not([class*='was'])...
   - Excludes: old, was, original, before, regular, ancien, barre

   LOW PRIORITY:
   - .price (generic, might catch anything)
   ```

2. **Immediate Return for High-Priority Selectors** (lines 202-204)
   - If found with semantic selector → return immediately
   - Avoids checking lower-priority selectors
   - Most reliable prices checked first

3. **Multi-Price Smart Selection** (lines 210-218)
   - If multiple prices found (e.g., old + current)
   - Takes LAST price in DOM order (not highest/lowest value)
   - Rationale: HTML structure typically shows old price first, current price last:
     ```html
     <span class="old-price">14,99 €</span>  ← First in DOM
     <span class="price">19,99 €</span>      ← Last in DOM (CURRENT)
     ```
   - Logs all found prices for debugging

**Why Not "Highest Price"?**
Taking highest value fails in promotions:
```
Old: 29,99 € (highest)
Current: 19,99 € (promotion) ← Should select this
```
Taking last in DOM works for both cases:
```
Case 1 (Stokomani):
  Old: 14,99 € (first)
  Current: 19,99 € (last) ← Selected ✓

Case 2 (Promotion):
  Old: 29,99 € (first)
  Current: 19,99 € (last) ← Selected ✓
```

**Flow:**
```
Search for [itemprop='price']
  → Found? Return immediately ✓
  → Not found? Continue...
Search for .current-price
  → Found? Return immediately ✓
  → Not found? Continue...
...
Search with low-priority selectors
  → Found: ["14,99 €", "19,99 €"]
  → Select last: "19,99 €" ✓
```

**Expected Results:**
- Before: "14.99 EUR" (first found, wrong)
- After: "19.99 EUR" (last in DOM, correct)
- Logs: "Multiple prices found: ['14,99 €', '19,99 €'], selecting last: 19,99 €"

Tested with: https://www.stokomani.fr/products/lutin-filou-telescopique-95-cm_208400
This commit is contained in:
Claude committed 2025-11-29 23:55:05 +00:00
1 parent 4c98fee285
commit 1c763143ca
1 file changed
+30 -13
+30 -13
View File
@@ -147,35 +147,38 @@ class BrowserlessService:
Extract price from generic e-commerce pages using common selectors and patterns.
Returns price text (e.g., "1,99 €") or empty string if not found.
"""
# Common price selectors used across e-commerce sites
# Common price selectors used across e-commerce sites (ordered by priority)
price_selectors = [
# Common class names
"[class*='price']:not([class*='old']):not([class*='was']):not([class*='original'])",
".product-price",
".price",
# Highest priority: semantic and explicit current prices
"[itemprop='price']",
".current-price",
".sale-price",
".final-price",
# Common data attributes
".product-price",
"[data-price]",
"[itemprop='price']",
# Medium priority: generic price classes (exclude old/was/original)
"[class*='price']:not([class*='old']):not([class*='was']):not([class*='original']):not([class*='before']):not([class*='regular'])",
# ID-based
"#price",
"#product-price",
"#our-price",
# Specific patterns
"span[class*='prix']",
"div[class*='prix']",
# French-specific
"span[class*='prix']:not([class*='ancien']):not([class*='barre'])",
"div[class*='prix']:not([class*='ancien'])",
"span[class*='tarif']",
# Lower priority: generic .price (might catch old prices)
".price",
]
found_prices = []
for selector in price_selectors:
try:
elements = page.locator(selector)
count = await elements.count()
# Try first visible element
for i in range(min(count, 3)): # Check max 3 elements
# Check up to 3 elements per selector
for i in range(min(count, 3)):
try:
element = elements.nth(i)
if await element.is_visible(timeout=1000):
@@ -193,13 +196,27 @@ class BrowserlessService:
if price_text and ('€' in price_text or (',' in price_text and any(c.isdigit() for c in price_text))):
# Clean up price text
price_text = price_text.strip()
found_prices.append((selector, price_text))
logger.info(f"Found price via selector {selector}: {price_text}")
return price_text
# Return immediately if we found with high-priority selector
if selector in ["[itemprop='price']", ".current-price", ".sale-price", ".final-price", ".product-price"]:
return price_text
except Exception:
continue
except Exception:
continue
# If we found prices with lower-priority selectors
if found_prices:
# When multiple prices found, the LAST one in DOM order is usually the current price
# (websites typically show: old price first, then current price)
best_price = found_prices[-1][1] # Take last (most recently added)
if len(found_prices) > 1:
prices_list = [p[1] for p in found_prices]
logger.info(f"Multiple prices found: {prices_list}, selecting last (most likely current): {best_price}")
return best_price
# Fallback: Use regex to find price patterns in all text
try:
all_text = await page.inner_text('body')