mirror of
https://github.com/R0m1k3/Priceflow.git
synced 2026-10-23 15:18:12 +02:00
feat: Add multi-site product search system with SearXNG integration
This commit adds a comprehensive product search feature that allows users to search for products across multiple configured e-commerce sites. Backend changes: - Add SearXNG service to docker-compose for meta-search capability - Add SearchSite model and Alembic migration with default sites - Create searxng_service.py for search queries across sites - Create light_scraper_service.py for fast HTTP-based extraction - Create search_service.py orchestrator combining SearXNG + HTTP + Browserless - Add API endpoints for search (/api/search) with SSE streaming - Add CRUD endpoints for search sites (/api/search-sites) Frontend changes: - Add new /search page with real-time results display - Create SearchResultCard component for product display - Add Progress, Checkbox, and Badge UI components - Update Settings page with search sites management section - Add navigation link to search page - Create new modern SVG logo with price-flow gradient design - Update favicon to SVG format The search system uses a hybrid approach: 1. SearXNG finds product URLs across configured sites 2. Light HTTP scraper attempts fast extraction 3. Falls back to Browserless for JavaScript-heavy sites 4. Results stream progressively via SSE
This commit is contained in:
26 files changed
+1831
-10
No files matched your search
@@ -13,6 +13,7 @@ services:
|
||||
environment:
|
||||
- DATABASE_URL=postgresql://priceflow:priceflow@db:5432/priceflow
|
||||
- BROWSERLESS_URL=ws://browserless:3000
|
||||
- SEARXNG_URL=http://searxng:8080
|
||||
- LOG_LEVEL=INFO
|
||||
- CORS_ORIGINS=*
|
||||
depends_on:
|
||||
@@ -20,6 +21,8 @@ services:
|
||||
condition: service_healthy
|
||||
browserless:
|
||||
condition: service_started
|
||||
searxng:
|
||||
condition: service_started
|
||||
volumes:
|
||||
- screenshots_data:/app/screenshots
|
||||
networks:
|
||||
@@ -47,6 +50,20 @@ services:
|
||||
networks:
|
||||
- nginx_default
|
||||
|
||||
# SearXNG - Moteur de recherche auto-hébergé
|
||||
searxng:
|
||||
image: searxng/searxng:latest
|
||||
container_name: priceflow-searxng
|
||||
restart: unless-stopped
|
||||
ports:
|
||||
- "8888:8080"
|
||||
environment:
|
||||
- SEARXNG_BASE_URL=http://searxng:8080
|
||||
volumes:
|
||||
- ./searxng:/etc/searxng:rw
|
||||
networks:
|
||||
- nginx_default
|
||||
|
||||
# Browserless - Navigateur headless pour le scraping
|
||||
browserless:
|
||||
image: ghcr.io/browserless/chromium:latest
|
||||
|
||||
Reference in new issue
Block a user