feat: add price verification script and update task.md to investigate comparator price extraction issues.

This commit is contained in:
Michael committed 2025-12-22 15:24:09 +01:00
1 parent 954672cfc0
commit 0ff229de13
2 files changed
+52 -2

No files matched your search

+7 -2
View File
@@ -2,9 +2,8 @@
## 🚀 Current Focus
- [ ] Investigate Comparator Price Extraction (User reported false prices)
- [x] Fix Catalog Images (Generic/Missing icons)
- [x] Address "PDF vs Image" question
- [x] Implement "Delete Images" (Cleanup bad data)
## 📋 Master Plan
@@ -27,6 +26,11 @@
- [x] **Cleanup Bad Data**:
- [x] Run cleanup endpoint.
- [x] Verify fix with new scrape.
- [/] **Comparator Price Fix**:
- [ ] Reproduce "False Price" issue with script.
- [ ] Audit `improved_search_service.py` extraction logic (Lowest Price vs AI).
- [ ] Implement robust price validation/filtering.
- [ ] Verify fix.
## 📝 Progress Log
@@ -34,3 +38,4 @@
- **2025-12-22**: Fixed Catalog Retrieval using HTTP fallback.
- **2025-12-22**: Investigating generic icon issue. Found images are served via Thumbor.
- **2025-12-22**: **FIXED**: Scraper now targets Thumbor images. Wiped bad catalogs.
- **2025-12-22**: User reported incorrect prices in Comparator. Investigating.
+45
View File
@@ -0,0 +1,45 @@
import asyncio
import logging
import sys
import os
# Identify workspace root
sys.path.append(os.getcwd())
# Mock Env Vars
os.environ["DATABASE_URL"] = "postgresql://user:password@localhost:5432/pricewatch"
os.environ["BROWSERLESS_URL"] = "ws://localhost:3012" # Ignored by patch, but good for completeness
from app.services.improved_search_service import ImprovedSearchService
from playwright.async_api import async_playwright
# Setup logging
logging.basicConfig(level=logging.DEBUG) # DEBUG level to see price extraction logic
logger = logging.getLogger(__name__)
# Monkey patch _connect_browser to use local launch
async def _connect_browser_local(p):
logger.info("Launching local browser (headless)...")
return await p.chromium.launch(headless=True)
ImprovedSearchService._connect_browser = _connect_browser_local
async def test_search():
await ImprovedSearchService.initialize()
query = "nintendo switch"
target_site = "stokomani.fr"
print(f"Searching for: {query} on {target_site}")
async for result in ImprovedSearchService.search_site_generator(target_site, query):
print(f"[{result.source}] {result.title}\n -> Price: {result.price}€\n -> URL: {result.url}")
await ImprovedSearchService.shutdown()
if __name__ == "__main__":
asyncio.run(test_search())