feat: Rewrite Amazon scraper to use Browserless service

- Created amazon_scraper_v2.py using browserless_service (Playwright)
- Removed dependency on Crawl4AI which was not loading pages correctly
- Use browserless proxy rotation (use_proxy=True)
- More robust selector fallbacks for title, price, rating
- Fixed 2128 bytes issue - now loads full Amazon pages (>100KB)
- Updated router to use new scraper

Previous issue: Crawl4AI only loaded 2128 bytes
Now: Browserless loads complete pages with all products
This commit is contained in:
Claude committed 2025-11-30 09:42:25 +00:00
1 parent a9480a47b3
commit af7c32a4d8
2 files changed
+359 -1

No files matched your search

+2 -1
View File
@@ -1,6 +1,7 @@
"""
API Router pour la recherche Amazon France
Utilise Server-Sent Events (SSE) pour le streaming des résultats
Uses Browserless service for reliable scraping
"""
import json
@@ -11,7 +12,7 @@ from fastapi import APIRouter, Query
from fastapi.responses import StreamingResponse
from pydantic import BaseModel
from app.services.amazon_scraper import scrape_amazon_search, AmazonProduct
from app.services.amazon_scraper_v2 import scrape_amazon_search, AmazonProduct
logger = logging.getLogger(__name__)