2 Commits

Include renames

Author	SHA1	Message	Date
Claude	df613cf64f	fix: Use Browserless for search page scraping ... - Replace httpx with Playwright/Browserless for search pages - Handles JavaScript-rendered content and bot detection - Add wait selectors for each site to ensure content is loaded - Block images and tracking scripts for faster loading - Sequential site scraping to avoid overloading Browserless	2025-11-22 15:05:47 +00:00
Claude	3b095801c5	feat: Replace DuckDuckGo with direct site search ... - Remove duckduckgo-search dependency (unreliable results) - Add direct_search_service.py that scrapes sites' search pages directly - Add search_url and product_link_selector fields to SearchSite model - Add beautifulsoup4 and lxml for HTML parsing - Update schemas with new fields - Add database migration for new columns Direct search is more reliable as it queries each site's own search functionality instead of relying on external search engines.	2025-11-22 15:00:49 +00:00