 Michael
|
38af998a09
|
feat: add scraper service with URL simplification, Playwright integration, and retry logic for web scraping.
|
2025-11-23 22:18:11 +01:00 |
|
|
Michael
|
e551927327
|
feat: Add direct e-commerce search service using Playwright/Browserless with site-specific configurations.
|
2025-11-23 16:06:16 +01:00 |
|
|
Michael
|
8d124bf0f8
|
feat: Add initial database migration and UI label component, alongside frontend dependency installation.
|
2025-11-23 14:37:35 +01:00 |
|
 Claude
|
3293b5d58a
|
fix: Improve French price extraction and cookie handling
AI Schema:
- Improve extraction prompt for French e-commerce pages
- Add specific instructions for French price format (comma decimal, space thousands)
- Add French stock status terms (Ajouter au panier, Disponible, etc.)
- Fix price normalization to handle French format:
- "12,99" -> 12.99
- "1 234,56" -> 1234.56
- "1.234,56" -> 1234.56
Scraper Service:
- Add comprehensive cookie/popup selectors for French sites:
- E.Leclerc, Auchan, Carrefour (OneTrust, Didomi)
- Cdiscount (#footer_tc_privacy_button_2)
- Boulanger (.bcom-consent-accept-all)
- Gifi, Centrakor (Magento cookie buttons)
- PrestaShop sites (Stokomani, B&M)
- Add common CMP platform selectors
- Add French text-based selectors for cookie acceptance
|
2025-11-23 08:12:43 +00:00 |
|
|
Claude
|
89f422744a
|
fix: Stabilité Browserless v2 + headers anti-détection
Browserless (docker-compose.yml):
- Image mise à jour: ghcr.io/browserless/chromium:latest
- Timeout augmenté à 300s (5 minutes)
- shm_size: 2gb (critique pour Chrome)
- Mémoire: 3G limit, 1G reservation
- Nouvelles variables: HEALTH, CHROME_REFRESH_TIME
Scraper Service:
- Ajout headers HTTP réalistes (Accept-Language, Accept)
- Locale fr-FR et timezone Europe/Paris
- Timeout connexion explicite (60s)
- Fermeture propre des ressources (page, context, browser)
- Meilleure gestion des erreurs de fermeture
|
2025-11-22 10:05:36 +00:00 |
|
|
Claude
|
b5cd1465ce
|
fix: Amélioration scraping Amazon + stabilité Browserless
Browserless (docker-compose.yml):
- CONNECTION_TIMEOUT augmenté à 180s
- MAX_CONCURRENT_SESSIONS réduit à 3
- Limite mémoire 2G ajoutée
- Args Chrome optimisés (disable-dev-shm-usage, no-sandbox)
Scraper Service:
- Ajout simplification automatique des URLs Amazon (extraction ASIN)
- Suppression des paramètres de tracking (utm_, ref, etc.)
- Système de retries (2 tentatives avec backoff)
- Screenshot pris AVANT extraction texte (évite perte si timeout)
- Popups Amazon ajoutés (#sp-cc-accept, etc.)
- Meilleure gestion des erreurs avec logs détaillés
|
2025-11-22 09:55:49 +00:00 |
|
 LogiFlow
|
afed10abc5
|
Add files via upload
|
2025-11-22 10:01:25 +01:00 |
|