- Move imports to top-level in search_sites router
- Fix line too long issue with input selector
- Define constants for magic numbers (HTTP_OK, MIN_PRODUCTS_TO_MATCH)
- Use 'raise from' pattern for proper exception chaining
- Add search_url_discovery.py service that visits sites and finds search forms
- Automatically discover search_url when a new site is created without one
- Add POST /{site_id}/discover endpoint to force rediscovery
- Discovery runs in background to not block the API response
- Tries multiple methods: form analysis, common URL patterns
This commit adds a comprehensive product search feature that allows users
to search for products across multiple configured e-commerce sites.
Backend changes:
- Add SearXNG service to docker-compose for meta-search capability
- Add SearchSite model and Alembic migration with default sites
- Create searxng_service.py for search queries across sites
- Create light_scraper_service.py for fast HTTP-based extraction
- Create search_service.py orchestrator combining SearXNG + HTTP + Browserless
- Add API endpoints for search (/api/search) with SSE streaming
- Add CRUD endpoints for search sites (/api/search-sites)
Frontend changes:
- Add new /search page with real-time results display
- Create SearchResultCard component for product display
- Add Progress, Checkbox, and Badge UI components
- Update Settings page with search sites management section
- Add navigation link to search page
- Create new modern SVG logo with price-flow gradient design
- Update favicon to SVG format
The search system uses a hybrid approach:
1. SearXNG finds product URLs across configured sites
2. Light HTTP scraper attempts fast extraction
3. Falls back to Browserless for JavaScript-heavy sites
4. Results stream progressively via SSE
Claude