Commit Graph
32 Commits
Author SHA1 Message Date
Claude 3b095801c5 feat: Replace DuckDuckGo with direct site search
- Remove duckduckgo-search dependency (unreliable results)
- Add direct_search_service.py that scrapes sites' search pages directly
- Add search_url and product_link_selector fields to SearchSite model
- Add beautifulsoup4 and lxml for HTML parsing
- Update schemas with new fields
- Add database migration for new columns

Direct search is more reliable as it queries each site's own search
functionality instead of relying on external search engines.
2025-11-22 15:00:49 +00:00
Claude 6f5b50a372 fix: Fix Browserless port and improve DuckDuckGo search
- Change BROWSERLESS_URL to use internal Docker port 3000 (not external 3012)
- Search each domain separately for better results
- Add detailed logging for debugging search issues
- Improve domain matching logic
2025-11-22 14:53:45 +00:00
Claude 36415b9cfc feat: Replace SearXNG with DuckDuckGo for product search
- Remove SearXNG container and configuration files
- Add duckduckgo-search library for web search
- Create duckduckgo_service.py with async search support
- Update search_service.py to use DuckDuckGo instead of SearXNG
- Fix BROWSERLESS_URL port to 3012

This eliminates the persistent limiter.toml directory issue and
provides a simpler, more reliable search solution.
2025-11-22 14:38:03 +00:00
Claude 528c34eb6b fix: Mount entire SearXNG config directory to avoid file/directory issues
- Mount ./searxng as /etc/searxng:rw instead of individual files
- Remove custom entrypoint.sh (no longer needed)
- Simplify settings.yml configuration
2025-11-22 14:28:53 +00:00
Claude 1f8974ce1a fix: Mount limiter.toml directly (file exists locally)
- Remove entrypoint script approach
- Mount limiter.toml directly as a file
- User must DELETE the SearXNG container in Portainer before redeploying
  to clear the cached directory state
2025-11-22 14:14:15 +00:00
Claude 1f51dace41 fix: Use custom entrypoint script for SearXNG limiter.toml
- Create entrypoint.sh script that creates limiter.toml before SearXNG starts
- Script removes any existing limiter.toml (file or directory) first
- Then creates proper limiter.toml with bot detection bypass
- Finally executes the original SearXNG entrypoint
- This approach is more reliable than inline commands
2025-11-22 13:55:35 +00:00
Claude 18372e870f fix: Simplify SearXNG config + add DNS servers
- Remove command override (limiter warning is non-blocking)
- Add DNS servers (8.8.8.8, 8.8.4.4, 1.1.1.1) to fix DNS resolution
- The real issue was 'Temporary failure in name resolution' not limiter.toml
2025-11-22 13:50:33 +00:00
Claude e9b7f17900 fix: SearXNG printf command + edit site modal
SearXNG:
- Use proper command list format instead of shell string
- Use printf with \n for proper newlines in limiter.toml
- Use exec to properly start the entrypoint

Edit Site Modal:
- Add Dialog component import
- Add siteModalOpen and editSiteData state
- Create proper modal with form for editing sites
- Modal opens when clicking edit button
- All site fields editable in modal
2025-11-22 13:45:34 +00:00
Claude 41ccfb6610 fix: SearXNG startup + always visible edit buttons
SearXNG:
- Use command override to rm -rf limiter.toml before recreating it
- This ensures any directory is removed before creating the file
- Mount only settings.yml, create limiter.toml dynamically

UI:
- Remove opacity-0 from site edit/delete buttons
- Buttons are now always visible (not just on hover)
- Added title attributes for accessibility
2025-11-22 13:38:37 +00:00
Claude 1c07f08457 fix: Improve database migrations - create table if missing
- Create search_sites table if it doesn't exist
- Add price_selector column only if table exists but column doesn't
- Fixes 400 Bad Request when creating search sites
2025-11-22 13:32:24 +00:00
Claude 99e7592cdc fix: Add automatic database migration for missing columns
- Add run_migrations() function that runs at app startup
- Automatically adds price_selector column to search_sites if missing
- Prevents UndefinedColumn errors on existing databases
2025-11-22 13:29:29 +00:00
Claude 861f38b984 fix: Mount entire searxng directory instead of individual files
Simple and robust solution:
- Mount ./searxng:/etc/searxng:rw (entire directory)
- No entrypoint/command overrides needed
- Both settings.yml and limiter.toml are included
- SearXNG uses its default entrypoint
2025-11-22 13:27:48 +00:00
Claude 933cb2e66f feat: Improve search config + rename Dashboard + fix SearXNG
SearXNG:
- Fix tini not found error by calling entrypoint.sh directly
- Keep limiter.toml creation at container startup

UI Changes:
- Rename "Tableau de bord" to "Suivis produits" in navigation
- Move CSS selector configuration from item modal to Settings page
- Add price_selector field to search sites configuration
- Display CSS selector badge on site cards

Backend:
- Add price_selector column to SearchSite model and schemas
- Update init.sql with search_sites table and migration for existing DBs
2025-11-22 13:24:28 +00:00
Claude e8ccd69bff fix: SearXNG startup + update README logo and favicon
- Fix SearXNG limiter.toml directory issue by creating file at container
  startup via entrypoint command (rm -rf then recreate)
- Update README to use logo.svg instead of logo.png
- Redesign favicon.svg to match main logo style:
  - Same gold-to-green gradient background
  - Line gradient from white to green
  - Glow effect on chart line
  - Decorative dots at data points
  - Green arrow indicator
2025-11-22 13:15:31 +00:00
Claude 60546f7a6c fix: Complete SearXNG bot detection bypass configuration
- Add real_ip config to settings.yml to trust proxy headers
- Update limiter.toml to allow all IPs (0.0.0.0/0 and ::/0)
- Re-add limiter.toml volume mount (file now exists)
- Add localhost range (127.0.0.0/8) to pass_ip list
2025-11-22 13:10:28 +00:00
Claude da1d88d432 fix: Remove limiter.toml volume mount to fix SearXNG container crash
Docker was creating /etc/searxng/limiter.toml as a directory instead of
mounting the file, causing IsADirectoryError. The limiter is already
disabled in settings.yml, so the separate toml file is not needed.
2025-11-22 13:04:42 +00:00
Claude 8db4f0ef40 fix: Mount SearXNG config files individually instead of directory 2025-11-22 12:10:02 +00:00
Claude 5caee6dee4 fix: Disable SearXNG public_instance and add User-Agent header 2025-11-22 11:55:14 +00:00
Claude 058a2e46ed fix: Clean domain URLs before SearXNG query (remove protocol, www, etc.) 2025-11-22 11:46:16 +00:00
Claude 20080aee74 fix: Add X-Forwarded-For header to SearXNG requests to bypass bot detection 2025-11-22 11:23:08 +00:00
Claude 9235ad444a fix: Use SettingsService class method instead of standalone function 2025-11-22 11:18:18 +00:00
Claude d39b3346c0 fix: Correct Alembic migration chain for search_sites table 2025-11-22 11:08:39 +00:00
Claude 1116c3a648 feat: Add multi-site product search system with SearXNG integration
This commit adds a comprehensive product search feature that allows users
to search for products across multiple configured e-commerce sites.

Backend changes:
- Add SearXNG service to docker-compose for meta-search capability
- Add SearchSite model and Alembic migration with default sites
- Create searxng_service.py for search queries across sites
- Create light_scraper_service.py for fast HTTP-based extraction
- Create search_service.py orchestrator combining SearXNG + HTTP + Browserless
- Add API endpoints for search (/api/search) with SSE streaming
- Add CRUD endpoints for search sites (/api/search-sites)

Frontend changes:
- Add new /search page with real-time results display
- Create SearchResultCard component for product display
- Add Progress, Checkbox, and Badge UI components
- Update Settings page with search sites management section
- Add navigation link to search page
- Create new modern SVG logo with price-flow gradient design
- Update favicon to SVG format

The search system uses a hybrid approach:
1. SearXNG finds product URLs across configured sites
2. Light HTTP scraper attempts fast extraction
3. Falls back to Browserless for JavaScript-heavy sites
4. Results stream progressively via SSE
2025-11-22 11:03:48 +00:00
LogiFlow 8795c6fea1 Merge pull request #4 from R0m1k3/claude/add-openrouter-integration-01Wo4d6cqjpCG7hiJceKCmKb
fix: Stabilité Browserless v2 + headers anti-détection
2025-11-22 11:06:52 +01:00
Claude 89f422744a fix: Stabilité Browserless v2 + headers anti-détection
Browserless (docker-compose.yml):
- Image mise à jour: ghcr.io/browserless/chromium:latest
- Timeout augmenté à 300s (5 minutes)
- shm_size: 2gb (critique pour Chrome)
- Mémoire: 3G limit, 1G reservation
- Nouvelles variables: HEALTH, CHROME_REFRESH_TIME

Scraper Service:
- Ajout headers HTTP réalistes (Accept-Language, Accept)
- Locale fr-FR et timezone Europe/Paris
- Timeout connexion explicite (60s)
- Fermeture propre des ressources (page, context, browser)
- Meilleure gestion des erreurs de fermeture
2025-11-22 10:05:36 +00:00
LogiFlow dd2ffadbc9 Merge pull request #3 from R0m1k3/claude/add-openrouter-integration-01Wo4d6cqjpCG7hiJceKCmKb
fix: Amélioration scraping Amazon + stabilité Browserless
2025-11-22 10:57:25 +01:00
Claude b5cd1465ce fix: Amélioration scraping Amazon + stabilité Browserless
Browserless (docker-compose.yml):
- CONNECTION_TIMEOUT augmenté à 180s
- MAX_CONCURRENT_SESSIONS réduit à 3
- Limite mémoire 2G ajoutée
- Args Chrome optimisés (disable-dev-shm-usage, no-sandbox)

Scraper Service:
- Ajout simplification automatique des URLs Amazon (extraction ASIN)
- Suppression des paramètres de tracking (utm_, ref, etc.)
- Système de retries (2 tentatives avec backoff)
- Screenshot pris AVANT extraction texte (évite perte si timeout)
- Popups Amazon ajoutés (#sp-cc-accept, etc.)
- Meilleure gestion des erreurs avec logs détaillés
2025-11-22 09:55:49 +00:00
LogiFlow 3cb9a53524 Merge pull request #2 from R0m1k3/claude/add-openrouter-integration-01Wo4d6cqjpCG7hiJceKCmKb
fix: Port applicatif 8555 + README complet en français
2025-11-22 10:31:41 +01:00
Claude 90e7b3688c fix: Port applicatif 8555 + README complet en français
- Changement du port de l'application de 8000 à 8555
- Mise à jour Dockerfile, docker-compose.yml et docker-entrypoint.sh
- README.md entièrement réécrit en français avec:
  - Instructions d'installation complètes
  - Documentation de l'architecture Docker
  - Guide de configuration OpenRouter
  - Section développement et personnalisation
2025-11-22 09:29:51 +00:00
LogiFlow a941b96a55 Merge pull request #1 from R0m1k3/claude/add-openrouter-integration-01Wo4d6cqjpCG7hiJceKCmKb
feat: Rebranding PriceFlow + OpenRouter + i18n FR + Docker
2025-11-22 10:23:21 +01:00
Claude d6c39252f2 feat: Rebranding PriceFlow + OpenRouter + i18n FR + Docker
Modifications majeures:

1. Rebranding: Pricecious → PriceFlow
   - Mise à jour du titre, logo et noms de projet

2. Intégration OpenRouter
   - Ajout comme provider IA avec filtres (chat, vision, code, reasoning, free)
   - Récupération dynamique des modèles et coûts
   - Service et router dédiés (/api/openrouter/models)

3. Internationalisation (i18n)
   - Installation i18next + react-i18next
   - Application entièrement traduite en français
   - Fichier de traduction: fr.json

4. Configuration Docker
   - docker-compose.yml avec réseau nginx_default (external)
   - PostgreSQL sur port 5488:5432
   - Browserless sur port 3012:3000
   - init.sql pour initialisation de la base de données
   - .env.example avec variables documentées

5. Favicon
   - Documentation ajoutée dans index.html pour faciliter le remplacement
2025-11-22 09:22:39 +00:00
LogiFlow afed10abc5 Add files via upload 2025-11-22 10:01:25 +01:00