feat: Add catalogue management API with enseigne, catalogue, and scraping endpoints, and integrate Cataloguemate scraper.

This commit is contained in:
Michael committed 2025-12-22 14:46:50 +01:00
1 parent 81b7cce947
commit 954672cfc0
6 files changed
+439 -208

No files matched your search

+16 -7
View File
@@ -1,8 +1,10 @@
# Task Board: Vision-Based Price Extraction
# Task Board: Catalog Image Fix
## 🚀 Current Focus
- [ ] Fix Catalog Images (Generic/Missing icons)
- [x] Fix Catalog Images (Generic/Missing icons)
- [x] Address "PDF vs Image" question
- [x] Implement "Delete Images" (Cleanup bad data)
## 📋 Master Plan
@@ -10,18 +12,25 @@
- [x] Proposal Phase: Vision Priority accepted.
- [x] Implementation Phase:
- [x] Update `ai_schema.py`.
- [x] Fix `scheduler_service.py` (Disable conflicting Text AI).
- [x] Fix `scheduler_service.py`.
- [x] Verification Phase: Verified with `verify_vision_priority.py`.
- [x] **Catalog Retrieval Fix**:
- [x] Reproduce failure (Confirmed Browserless issue).
- [x] Fix `cataloguemate_scraper.py` with HTTP fallback.
- [x] Cleanup debug files.
- [ ] **Catalog Image Fix**:
- [x] Analyze page HTML for correct image selectors (Found `data-src`).
- [x] Refine `cataloguemate_scraper.py` image extraction logic.
- [x] **Catalog Image Fix**:
- [x] Analyze page HTML for correct image selectors (Found Thumbor URL).
- [x] **Check PDF availability** (Browser Task).
- [x] **Refine Scraper Logic**:
- [x] Update `cataloguemate_scraper.py` to grab `src` and prioritize Thumbor.
- [x] Update `routers/catalogues.py` to clean up bad images.
- [x] **Cleanup Bad Data**:
- [x] Run cleanup endpoint.
- [x] Verify fix with new scrape.
## 📝 Progress Log
- **2025-12-22**: Vision Priority implemented and verified.
- **2025-12-22**: Fixed Catalog Retrieval using HTTP fallback.
- **2025-12-22**: User reports images are generic icons. Investigating image selectors.
- **2025-12-22**: Investigating generic icon issue. Found images are served via Thumbor.
- **2025-12-22**: **FIXED**: Scraper now targets Thumbor images. Wiped bad catalogs.