feat: Replace DuckDuckGo with direct site search

- Remove duckduckgo-search dependency (unreliable results)
- Add direct_search_service.py that scrapes sites' search pages directly
- Add search_url and product_link_selector fields to SearchSite model
- Add beautifulsoup4 and lxml for HTML parsing
- Update schemas with new fields
- Add database migration for new columns

Direct search is more reliable as it queries each site's own search
functionality instead of relying on external search engines.
This commit is contained in:
Claude committed 2025-11-22 15:00:49 +00:00
1 parent 6f5b50a372
commit 3b095801c5
7 files changed
+379 -198

No files matched your search

+2
View File
@@ -92,5 +92,7 @@ class SearchSite(Base):
priority: int = Column(Integer, default=0) # type: ignore # Ordre d'affichage
requires_js: bool = Column(Boolean, default=False) # type: ignore # Force Browserless si True
price_selector: str | None = Column(String, nullable=True) # type: ignore # Sélecteur CSS pour le prix
search_url: str | None = Column(String, nullable=True) # type: ignore # URL de recherche avec {query} placeholder
product_link_selector: str | None = Column(String, nullable=True) # type: ignore # Sélecteur CSS pour les liens produits
created_at: datetime = Column(DateTime, default=lambda: datetime.now(UTC)) # type: ignore
updated_at: datetime = Column(DateTime, default=lambda: datetime.now(UTC), onupdate=lambda: datetime.now(UTC)) # type: ignore