mirror of
https://github.com/R0m1k3/Priceflow.git
synced 2026-10-14 02:36:21 +02:00
Claude
03793cea0a
**Problem:**
Previous commit changed get_page_content() to always return visible text (inner_text)
instead of HTML source. This broke product search which uses BeautifulSoup to parse
HTML with CSS selectors. Search was returning 0 results for all sites.
**Root Cause:**
- Search service uses: `soup.select(config["product_selector"])` on HTML
- But was receiving plain text instead of HTML structure
- BeautifulSoup couldn't find any product links → 0 results
**Solution: Add extract_text parameter**
Modified `get_page_content()` signature:
```python
async def get_page_content(
url: str,
use_proxy: bool = False,
wait_selector: str = None,
extract_text: bool = False # NEW parameter
) -> tuple[str, str]:
```
**Behavior:**
- `extract_text=False` (DEFAULT): Returns HTML source via `page.content()`
→ Used by search service for BeautifulSoup parsing
- `extract_text=True`: Returns visible text via `page.inner_text('body')`
→ Used by AI monitoring for price extraction
→ Includes Amazon price prepending ("PRIX DÉTECTÉ: ...")
**Changes:**
1. **browserless_service.py** (Lines 197-329)
- Added `extract_text` parameter with default `False`
- Conditional logic: if extract_text, use inner_text + Amazon extraction
- Else: use page.content() (HTML source)
- Preserves backward compatibility (default = HTML)
2. **scheduler_service.py** (Line 113)
- Added `extract_text=True` for AI monitoring
- Ensures AI gets visible text with Amazon prices
3. **search_service.py** (Line 104)
- Added `extract_text=True` for scrape_item() AI analysis
- Search itself uses default (HTML) for BeautifulSoup parsing
**Impact:**
✅ Search now works again (gets HTML for BeautifulSoup)
✅ AI monitoring gets visible text (better extraction)
✅ Amazon price extraction only runs when extract_text=True
✅ Backward compatible (default behavior = HTML)
Fixes issue where all searches returned: "Found 0 results for [site]"