mirror of
https://github.com/R0m1k3/Priceflow.git
synced 2026-10-18 04:38:41 +02:00
feat: Add Amazon France search page with Crawl4AI anti-detection
- Created Amazon scraper service with advanced anti-bot techniques: * User-Agent rotation from realistic pool * Complete browser headers (Accept, Accept-Language, etc.) * Proxy rotation (10 residential proxies) * Random delays (1.5-4s) to mimic human behavior * Crawl4AI browser fingerprint randomization * NetworkIdle waiting for complete page load * Cookie acceptance automation - Added Amazon search API endpoint with SSE streaming * Real-time progress updates * Proper error handling * Health check endpoint - Created dedicated Amazon France frontend page: * Modern UI with product cards * Rating display (stars + review count) * Price formatting with discount badges * Prime badge support * Stock status indicators * Sponsored product labels * Direct Amazon links - Removed store list (ENSEIGNES_DATA cleared) * Migration from discount stores to Amazon France * Catalog system kept for future use - Updated navigation: * Added "Amazon France" menu item with ShoppingBag icon * Positioned between Search and Compare * Available on desktop and mobile Technical stack: - Backend: Crawl4AI + BeautifulSoup for scraping - Frontend: React + Shadcn UI components - API: FastAPI with SSE streaming
This commit is contained in:
7 files changed
+887
-77
No files matched your search
+4
-1
@@ -13,7 +13,7 @@ from sqlalchemy import text
|
||||
|
||||
from app.database import SessionLocal, engine
|
||||
from app.limiter import limiter
|
||||
from app.routers import auth, items, jobs, notifications, openrouter, search, search_sites, settings, debug, catalogues
|
||||
from app.routers import auth, items, jobs, notifications, openrouter, search, search_sites, settings, debug, catalogues, amazon
|
||||
from app.services.scheduler_service import scheduled_refresh, scheduler
|
||||
from app.services import auth_service, search_service, seed_enseignes
|
||||
from app.services.scheduler import start_scheduler as start_catalog_scheduler, stop_scheduler as stop_catalog_scheduler
|
||||
@@ -190,6 +190,9 @@ app.include_router(catalogues.router)
|
||||
# Auth router (already has /api prefix)
|
||||
app.include_router(auth.router)
|
||||
|
||||
# Amazon router (already has /api prefix)
|
||||
app.include_router(amazon.router)
|
||||
|
||||
|
||||
@app.get("/api/")
|
||||
def read_root():
|
||||
|
||||
Reference in new issue
Block a user