feat: Replace DuckDuckGo with direct site search

- Remove duckduckgo-search dependency (unreliable results)
- Add direct_search_service.py that scrapes sites' search pages directly
- Add search_url and product_link_selector fields to SearchSite model
- Add beautifulsoup4 and lxml for HTML parsing
- Update schemas with new fields
- Add database migration for new columns

Direct search is more reliable as it queries each site's own search
functionality instead of relying on external search engines.
This commit is contained in:
Claude committed 2025-11-22 15:00:49 +00:00
1 parent 6f5b50a372
commit 3b095801c5
7 files changed
+379 -198

No files matched your search

+2 -1
View File
@@ -23,7 +23,8 @@ dependencies = [
"alembic",
"tenacity",
"slowapi",
"duckduckgo-search",
"beautifulsoup4",
"lxml",
]
[build-system]