- Increased default Amazon search results limit from 20 to 50 in backend and frontend
- Improved TrackingScraperService to target main product area on Amazon (avoiding reviews)
- Added specific selectors for Amazon product page (#imgTagWrapperId, #productTitle, etc.)
- Created Amazon scraper service with advanced anti-bot techniques:
* User-Agent rotation from realistic pool
* Complete browser headers (Accept, Accept-Language, etc.)
* Proxy rotation (10 residential proxies)
* Random delays (1.5-4s) to mimic human behavior
* Crawl4AI browser fingerprint randomization
* NetworkIdle waiting for complete page load
* Cookie acceptance automation
- Added Amazon search API endpoint with SSE streaming
* Real-time progress updates
* Proper error handling
* Health check endpoint
- Created dedicated Amazon France frontend page:
* Modern UI with product cards
* Rating display (stars + review count)
* Price formatting with discount badges
* Prime badge support
* Stock status indicators
* Sponsored product labels
* Direct Amazon links
- Removed store list (ENSEIGNES_DATA cleared)
* Migration from discount stores to Amazon France
* Catalog system kept for future use
- Updated navigation:
* Added "Amazon France" menu item with ShoppingBag icon
* Positioned between Search and Compare
* Available on desktop and mobile
Technical stack:
- Backend: Crawl4AI + BeautifulSoup for scraping
- Frontend: React + Shadcn UI components
- API: FastAPI with SSE streaming
- Add debug_enabled field to SearchSite model
- Update schemas and API routes to allow debug toggle
- Add debug toggle button on site cards in Admin UI
- Only dump HTML when debug is enabled for the site (except for critical errors like Amazon blocking)
- Add API endpoints to list/delete debug files per site domain
- Add button to clear debug files for individual sites
The debug tab was defined in the tabs array and had all the state and
functions set up, but was missing the actual JSX rendering block.
This caused the debug dumps to not be displayed in the admin interface.
- Add User model with authentication
- Create JWT-based auth service
- Add login page with admin/admin default credentials
- Create admin panel with user management, settings, and site configuration
- Move settings to admin section (admin only)
- Change search page from multi-site checkboxes to single site selector
- Add logout functionality to layout
- Protect routes with authentication
- Remove ability to add/edit/delete search sites from settings UI
- Configure 16 French e-commerce sites with search URLs:
- Discount: Gifi, Stokomani, B&M, Centrakor, L'Incroyable, Action, La Foir'Fouille
- Grandes Surfaces: E.Leclerc, Auchan, Carrefour
- E-commerce: Amazon France, Cdiscount
- Électronique: Darty, Boulanger, Fnac
- Add automatic database seeding on application startup
- Add /api/search-sites/seed and /api/search-sites/reset endpoints
- Add cookie consent banner handling for sites that require it
- Simplify Settings page to only show site list with toggle for active state
SearXNG:
- Use proper command list format instead of shell string
- Use printf with \n for proper newlines in limiter.toml
- Use exec to properly start the entrypoint
Edit Site Modal:
- Add Dialog component import
- Add siteModalOpen and editSiteData state
- Create proper modal with form for editing sites
- Modal opens when clicking edit button
- All site fields editable in modal
SearXNG:
- Use command override to rm -rf limiter.toml before recreating it
- This ensures any directory is removed before creating the file
- Mount only settings.yml, create limiter.toml dynamically
UI:
- Remove opacity-0 from site edit/delete buttons
- Buttons are now always visible (not just on hover)
- Added title attributes for accessibility
SearXNG:
- Fix tini not found error by calling entrypoint.sh directly
- Keep limiter.toml creation at container startup
UI Changes:
- Rename "Tableau de bord" to "Suivis produits" in navigation
- Move CSS selector configuration from item modal to Settings page
- Add price_selector field to search sites configuration
- Display CSS selector badge on site cards
Backend:
- Add price_selector column to SearchSite model and schemas
- Update init.sql with search_sites table and migration for existing DBs
- Fix SearXNG limiter.toml directory issue by creating file at container
startup via entrypoint command (rm -rf then recreate)
- Update README to use logo.svg instead of logo.png
- Redesign favicon.svg to match main logo style:
- Same gold-to-green gradient background
- Line gradient from white to green
- Glow effect on chart line
- Decorative dots at data points
- Green arrow indicator
LogiFlow
Michael
Claude