Michael
fa4d454063
feat: Add Amazon scraper service with persistent browser, anti-detection, and Pydantic schemas for product data.
2025-12-24 23:10:20 +01:00
Michael
f8e4286bdf
feat: add Amazon scraper service with persistent browser, stealth, and parsing capabilities for Amazon product data.
2025-12-24 23:01:10 +01:00
Michael
3a7dcd16ed
feat: implement Amazon scraper service using Playwright with persistent browser connection and anti-detection techniques.
2025-12-24 22:49:02 +01:00
Michael
cd18b47bc0
feat: Add core search configurations for various e-commerce sites, including proxy and user agent management, and introduce initial Amazon scraper service and proxy utilities.
2025-12-24 21:13:22 +01:00
Michael
43de609773
feat: add Amazon scraper service using Playwright with persistent browser and anti-detection features
2025-12-24 14:38:16 +01:00
Michael
0ab7ac5a38
feat: add Amazon scraper service using Playwright, Browserless, and stealth for persistent sessions.
2025-12-24 12:04:58 +01:00
Michael
fd34bc66b2
feat: implement Amazon scraper service using Playwright with stealth features for product search and parsing.
2025-12-24 09:19:28 +01:00
Michael
ef7cd7c1cf
feat: Add BMStores product parser and a new Playwright-based scraping service with popup handling.
2025-12-18 15:20:19 +01:00
Michael
f223dc5a29
feat: Increase Amazon limit to 50 and fix screenshot framing
...
- Increased default Amazon search results limit from 20 to 50 in backend and frontend
- Improved TrackingScraperService to target main product area on Amazon (avoiding reviews)
- Added specific selectors for Amazon product page (#imgTagWrapperId, #productTitle, etc.)
2025-12-01 16:18:35 +01:00
Michael
acfc5c6397
feat: Enhance Amazon stealth with comprehensive anti-detection
...
Added comprehensive stealth techniques to bypass Amazon bot detection:
- Complete HTTP headers (Accept, Accept-Language, Sec-Fetch-*, etc.)
- Extended chrome object with loadTimes, csi, app
- Permissions API override
- Plugins, languages, platform spoofing
- Battery API mocking
This significantly improves bot detection evasion compared to the basic
2-line stealth that was insufficient for Amazon's detection systems.
2025-12-01 13:55:57 +01:00
Michael
2429fa2018
Revert "fix: Remove homepage redirect to avoid Amazon bot detection"
...
This reverts commit 5d3d181f5e
2025-12-01 13:26:05 +01:00
Michael
5d3d181f5e
fix: Remove homepage redirect to avoid Amazon bot detection
...
Amazon detects the homepage-then-search pattern as bot behavior and blocks
with 2065 byte pages. Now navigating directly to search URL to appear more
natural.
This reverts the homepage loading strategy from commit 85b19e7
2025-12-01 13:25:51 +01:00
Michael
510904d899
feat: Add Amazon product scraping service using Playwright and Browserless for persistent browser connections.
2025-11-30 14:56:01 +01:00
Claude
85b19e7680
fix: Load Amazon homepage first to establish session
...
- CRITICAL FIX: Load amazon.fr homepage in same context before search
- Preserves cookies/session between homepage and search
- Should fix 2065 bytes issue (blocked requests)
- Homepage popups handled before search
2025-11-30 10:43:24 +00:00
Claude
ea2af8a8f2
feat: Create persistent Amazon scraper service (ScraperService pattern)
...
- New amazon_scraper_service.py with persistent browser connection
- Shared browser instance across requests (better session management)
- Auto-reconnect if browser connection drops
- Better stealth mode (webdriver undefined, chrome runtime)
- Proper cookie/popup handling for Amazon
- Uses Playwright via Browserless (not Crawl4AI)
- Should fix 503 errors with persistent session
Based on user's ScraperService pattern for reliability
2025-11-30 10:34:41 +00:00
Michael
Claude