Commit Graph
15 Commits
Author SHA1 Message Date
Claude 6f5b50a372 fix: Fix Browserless port and improve DuckDuckGo search
- Change BROWSERLESS_URL to use internal Docker port 3000 (not external 3012)
- Search each domain separately for better results
- Add detailed logging for debugging search issues
- Improve domain matching logic
2025-11-22 14:53:45 +00:00
Claude 36415b9cfc feat: Replace SearXNG with DuckDuckGo for product search
- Remove SearXNG container and configuration files
- Add duckduckgo-search library for web search
- Create duckduckgo_service.py with async search support
- Update search_service.py to use DuckDuckGo instead of SearXNG
- Fix BROWSERLESS_URL port to 3012

This eliminates the persistent limiter.toml directory issue and
provides a simpler, more reliable search solution.
2025-11-22 14:38:03 +00:00
Claude 1c07f08457 fix: Improve database migrations - create table if missing
- Create search_sites table if it doesn't exist
- Add price_selector column only if table exists but column doesn't
- Fixes 400 Bad Request when creating search sites
2025-11-22 13:32:24 +00:00
Claude 99e7592cdc fix: Add automatic database migration for missing columns
- Add run_migrations() function that runs at app startup
- Automatically adds price_selector column to search_sites if missing
- Prevents UndefinedColumn errors on existing databases
2025-11-22 13:29:29 +00:00
Claude 933cb2e66f feat: Improve search config + rename Dashboard + fix SearXNG
SearXNG:
- Fix tini not found error by calling entrypoint.sh directly
- Keep limiter.toml creation at container startup

UI Changes:
- Rename "Tableau de bord" to "Suivis produits" in navigation
- Move CSS selector configuration from item modal to Settings page
- Add price_selector field to search sites configuration
- Display CSS selector badge on site cards

Backend:
- Add price_selector column to SearchSite model and schemas
- Update init.sql with search_sites table and migration for existing DBs
2025-11-22 13:24:28 +00:00
Claude 5caee6dee4 fix: Disable SearXNG public_instance and add User-Agent header 2025-11-22 11:55:14 +00:00
Claude 058a2e46ed fix: Clean domain URLs before SearXNG query (remove protocol, www, etc.) 2025-11-22 11:46:16 +00:00
Claude 20080aee74 fix: Add X-Forwarded-For header to SearXNG requests to bypass bot detection 2025-11-22 11:23:08 +00:00
Claude 9235ad444a fix: Use SettingsService class method instead of standalone function 2025-11-22 11:18:18 +00:00
Claude d39b3346c0 fix: Correct Alembic migration chain for search_sites table 2025-11-22 11:08:39 +00:00
Claude 1116c3a648 feat: Add multi-site product search system with SearXNG integration
This commit adds a comprehensive product search feature that allows users
to search for products across multiple configured e-commerce sites.

Backend changes:
- Add SearXNG service to docker-compose for meta-search capability
- Add SearchSite model and Alembic migration with default sites
- Create searxng_service.py for search queries across sites
- Create light_scraper_service.py for fast HTTP-based extraction
- Create search_service.py orchestrator combining SearXNG + HTTP + Browserless
- Add API endpoints for search (/api/search) with SSE streaming
- Add CRUD endpoints for search sites (/api/search-sites)

Frontend changes:
- Add new /search page with real-time results display
- Create SearchResultCard component for product display
- Add Progress, Checkbox, and Badge UI components
- Update Settings page with search sites management section
- Add navigation link to search page
- Create new modern SVG logo with price-flow gradient design
- Update favicon to SVG format

The search system uses a hybrid approach:
1. SearXNG finds product URLs across configured sites
2. Light HTTP scraper attempts fast extraction
3. Falls back to Browserless for JavaScript-heavy sites
4. Results stream progressively via SSE
2025-11-22 11:03:48 +00:00
Claude 89f422744a fix: Stabilité Browserless v2 + headers anti-détection
Browserless (docker-compose.yml):
- Image mise à jour: ghcr.io/browserless/chromium:latest
- Timeout augmenté à 300s (5 minutes)
- shm_size: 2gb (critique pour Chrome)
- Mémoire: 3G limit, 1G reservation
- Nouvelles variables: HEALTH, CHROME_REFRESH_TIME

Scraper Service:
- Ajout headers HTTP réalistes (Accept-Language, Accept)
- Locale fr-FR et timezone Europe/Paris
- Timeout connexion explicite (60s)
- Fermeture propre des ressources (page, context, browser)
- Meilleure gestion des erreurs de fermeture
2025-11-22 10:05:36 +00:00
Claude b5cd1465ce fix: Amélioration scraping Amazon + stabilité Browserless
Browserless (docker-compose.yml):
- CONNECTION_TIMEOUT augmenté à 180s
- MAX_CONCURRENT_SESSIONS réduit à 3
- Limite mémoire 2G ajoutée
- Args Chrome optimisés (disable-dev-shm-usage, no-sandbox)

Scraper Service:
- Ajout simplification automatique des URLs Amazon (extraction ASIN)
- Suppression des paramètres de tracking (utm_, ref, etc.)
- Système de retries (2 tentatives avec backoff)
- Screenshot pris AVANT extraction texte (évite perte si timeout)
- Popups Amazon ajoutés (#sp-cc-accept, etc.)
- Meilleure gestion des erreurs avec logs détaillés
2025-11-22 09:55:49 +00:00
Claude d6c39252f2 feat: Rebranding PriceFlow + OpenRouter + i18n FR + Docker
Modifications majeures:

1. Rebranding: Pricecious → PriceFlow
   - Mise à jour du titre, logo et noms de projet

2. Intégration OpenRouter
   - Ajout comme provider IA avec filtres (chat, vision, code, reasoning, free)
   - Récupération dynamique des modèles et coûts
   - Service et router dédiés (/api/openrouter/models)

3. Internationalisation (i18n)
   - Installation i18next + react-i18next
   - Application entièrement traduite en français
   - Fichier de traduction: fr.json

4. Configuration Docker
   - docker-compose.yml avec réseau nginx_default (external)
   - PostgreSQL sur port 5488:5432
   - Browserless sur port 3012:3000
   - init.sql pour initialisation de la base de données
   - .env.example avec variables documentées

5. Favicon
   - Documentation ajoutée dans index.html pour faciliter le remplacement
2025-11-22 09:22:39 +00:00
LogiFlow afed10abc5 Add files via upload 2025-11-22 10:01:25 +01:00