Commit Graph
4 Commits
Author SHA1 Message Date
Michael 5d3d181f5e fix: Remove homepage redirect to avoid Amazon bot detection
Amazon detects the homepage-then-search pattern as bot behavior and blocks
with 2065 byte pages. Now navigating directly to search URL to appear more
natural.

This reverts the homepage loading strategy from commit 85b19e7 as Amazon's
bot detection has evolved and now flags this pattern.
2025-12-01 13:25:51 +01:00
Michael 510904d899 feat: Add Amazon product scraping service using Playwright and Browserless for persistent browser connections. 2025-11-30 14:56:01 +01:00
Claude 85b19e7680 fix: Load Amazon homepage first to establish session
- CRITICAL FIX: Load amazon.fr homepage in same context before search
- Preserves cookies/session between homepage and search
- Should fix 2065 bytes issue (blocked requests)
- Homepage popups handled before search
2025-11-30 10:43:24 +00:00
Claude ea2af8a8f2 feat: Create persistent Amazon scraper service (ScraperService pattern)
- New amazon_scraper_service.py with persistent browser connection
- Shared browser instance across requests (better session management)
- Auto-reconnect if browser connection drops
- Better stealth mode (webdriver undefined, chrome runtime)
- Proper cookie/popup handling for Amazon
- Uses Playwright via Browserless (not Crawl4AI)
- Should fix 503 errors with persistent session

Based on user's ScraperService pattern for reliability
2025-11-30 10:34:41 +00:00