Michael/Priceflow
1
0 Fork 0
mirror of https://github.com/R0m1k3/Priceflow.git synced 2026-10-11 17:29:14 +02:00
Code Issues Packages Projects Releases Wiki Activity
575 Commits 10 Branches 0 Tags
8329eb28a3810915f905011852fd4c49f64e7ec8
Commit Graph
575 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Michael 4dbb9f6424 feat: add improved search service with persistent browser connection 2025-12-01 01:03:51 +01:00
LogiFlow 8b6a3e9229 Merge pull request #203 from R0m1k3/antigravity 
feat: implement product search page with site selection, real-time re…
 2025-12-01 01:02:14 +01:00
Michael 93840804a4 feat: implement product search page with site selection, real-time results, and monitoring integration 2025-12-01 01:01:53 +01:00
LogiFlow 7eac22aebf Merge pull request #202 from R0m1k3/antigravity 
feat: Add improved search service with persistent browser connection …
 2025-12-01 01:00:46 +01:00
Michael 39893e8c1e feat: Add improved search service with persistent browser connection for e-commerce scraping. 2025-12-01 01:00:25 +01:00
LogiFlow 548cd296f4 Merge pull request #201 from R0m1k3/antigravity 
feat: Implement AI-powered price extraction using Gemma 2 and an impr…
 2025-12-01 00:57:27 +01:00
Michael b5c5961618 feat: Implement AI-powered price extraction using Gemma 2 and an improved search service with persistent browser connections for web scraping. 2025-12-01 00:57:05 +01:00
LogiFlow 14448ffe4b Merge pull request #200 from R0m1k3/antigravity 
feat: Implement comprehensive search configurations for discount stor…
 2025-12-01 00:37:25 +01:00
Michael c159036583 feat: Implement comprehensive search configurations for discount stores and add Gifi-specific search and analysis tools. 2025-12-01 00:36:54 +01:00
LogiFlow 0d300085d6 Merge pull request #199 from R0m1k3/antigravity 
feat: introduce new search configuration, improved search service, an…
 2025-11-30 23:55:09 +01:00
Michael 63073d03b5 feat: introduce new search configuration, improved search service, and Carrefour search integration test. 2025-11-30 23:54:48 +01:00
LogiFlow 6ae2c41b00 Merge pull request #198 from R0m1k3/antigravity 
feat: introduce improved search service with persistent browser and m…
 2025-11-30 23:54:20 +01:00
Michael bf97cd22e7 feat: introduce improved search service with persistent browser and modular parsers 2025-11-30 23:49:34 +01:00
LogiFlow c1e350affa Merge pull request #197 from R0m1k3/antigravity 
feat: add improved search service with persistent browser connection …
 2025-11-30 23:46:14 +01:00
Michael 7cb0e2f94b feat: add improved search service with persistent browser connection and modular site-specific parsers 2025-11-30 23:45:51 +01:00
LogiFlow b21b000eb5 Merge pull request #196 from R0m1k3/antigravity 
feat: implement improved search service using persistent browser conn…
 2025-11-30 23:43:32 +01:00
Michael 8bf9ddb2d7 feat: implement improved search service using persistent browser connection and modular parsers, alongside a BeautifulSoup check utility. 2025-11-30 23:43:11 +01:00
LogiFlow 2d7373b075 Merge pull request #195 from R0m1k3/antigravity 
feat: introduce improved search service with persistent browser conne…
 2025-11-30 23:40:49 +01:00
Michael 87568866e7 feat: introduce improved search service with persistent browser connection and modular site-specific parsers. 2025-11-30 23:40:22 +01:00
LogiFlow f9f2f0ebc2 Merge pull request #194 from R0m1k3/antigravity 
feat: add improved search service with persistent browser, modular pa…
 2025-11-30 23:35:58 +01:00
Michael 4dd77bd889 feat: add improved search service with persistent browser, modular parsers, and popup handling. 2025-11-30 23:35:40 +01:00
LogiFlow 6a1034666e Merge pull request #193 from R0m1k3/antigravity 
feat: Implement improved search service with supporting models, schem…
 2025-11-30 23:33:27 +01:00
Michael 9855040f3e feat: Implement improved search service with supporting models, schemas, and API routers, and refactor site verification script. 2025-11-30 23:33:05 +01:00
LogiFlow 5e377e5d30 Merge pull request #192 from R0m1k3/antigravity 
feat: Implement improved search service with persistent browser and m…
 2025-11-30 22:21:33 +01:00
Michael 4720ed6b93 feat: Implement improved search service with persistent browser and modular site-specific parsers, including Gifi. 2025-11-30 22:21:08 +01:00
LogiFlow 1ead73f367 Merge pull request #191 from R0m1k3/antigravity 
Antigravity
 2025-11-30 22:04:43 +01:00
Michael 64067347d1 feat: implement new search service with multiple site parsers, search configuration, and supporting inspection/verification scripts. 2025-11-30 22:04:14 +01:00
Michael 95e38287cb feat: Add new parsers for Auchan, Carrefour, L'Incroyable, and Stokomani, including search configuration and deployment updates. 2025-11-30 22:03:40 +01:00
LogiFlow 430321dfe2 Merge pull request #190 from R0m1k3/antigravity 
feat: Implement new product parsers for Stokomani, Auchan, Carrefour,…
 2025-11-30 20:26:45 +01:00
Michael b44082aeb4 feat: Implement new product parsers for Stokomani, Auchan, Carrefour, and Gifi, and add a centralized search configuration module. 2025-11-30 20:26:19 +01:00
LogiFlow 8a011f8a7c Merge pull request #189 from R0m1k3/antigravity 
feat: add LaFoirFouille parser to extract product search results from…
 2025-11-30 19:09:24 +01:00
Michael e0d73f4d0e feat: add LaFoirFouille parser to extract product search results from lafoirfouille.fr 2025-11-30 19:09:09 +01:00
LogiFlow e3974c25cc Merge pull request #188 from R0m1k3/antigravity 
feat: Add initial parser for Auchan.fr to extract product search resu…
 2025-11-30 19:06:08 +01:00
Michael fbd06cfb17 feat: Add initial parser for Auchan.fr to extract product search results. 2025-11-30 19:05:38 +01:00
LogiFlow 4b6468a3bd Merge pull request #187 from R0m1k3/antigravity 
feat: Add abstract base parser with product result dataclass and impl…
 2025-11-30 19:03:52 +01:00
Michael 078060f155 feat: Add abstract base parser with product result dataclass and implement Stokomani parser for search results. 2025-11-30 19:03:26 +01:00
LogiFlow 388ce85a95 Merge pull request #186 from R0m1k3/antigravity 
feat: add Stokomani parser to extract product search results from sto…
 2025-11-30 18:17:28 +01:00
Michael ebe685fdf9 feat: add Stokomani parser to extract product search results from stokomani.fr 2025-11-30 18:17:07 +01:00
LogiFlow 5436b206df Merge pull request #185 from R0m1k3/antigravity 
feat: Add Stokomani search results parser
 2025-11-30 18:13:34 +01:00
Michael b3d5c56707 feat: Add Stokomani search results parser 2025-11-30 18:13:11 +01:00
LogiFlow 668784ca14 Merge pull request #184 from R0m1k3/antigravity 
feat: Implement improved search service with centralized configuratio…
 2025-11-30 18:06:05 +01:00
Michael c2f042beee feat: Implement improved search service with centralized configuration and site-specific parsers for various retailers. 2025-11-30 18:05:34 +01:00
LogiFlow 3d894ff740 Merge pull request #183 from R0m1k3/antigravity 
feat: Add improved search service with persistent browser connection …
 2025-11-30 17:42:50 +01:00
Michael a8bffaec08 feat: Implement initial e-commerce site parsers with a base class and factory for extracting product search results. 2025-11-30 17:42:32 +01:00
Michael ee13a0bccd fix: resolve merge conflict in improved_search_service - unified domain matching logic 2025-11-30 17:37:16 +01:00
Michael 3db7db48fa feat: Add improved search service with persistent browser connection for e-commerce scraping. 2025-11-30 17:32:58 +01:00
LogiFlow 338c79857c Merge pull request #182 from R0m1k3/claude/rebuild-search-amazon-01LBnzN1wfNMe1EG9HcpMCdK 
Claude/rebuild search amazon 01 l bnz n1wf n me1 eg9 hcp m cd k
 2025-11-30 17:26:53 +01:00
Claude 43ea314e41 fix: Improve Fnac image extraction and add Shopify fallbacks 
Fixes two issues:

**1. Fnac - No Images Displayed**
Problem: FnacParser only searched for images in parent container,
missing images nested directly in the link element.

Solution: Two-priority search
- PRIORITY 1: Search in link element first (picture, img)
- PRIORITY 2: Search in parent container
- Added warning log when no image found
- Ensures parent is always defined before use

**2. L'Incroyable - 0 Products (Shopify sites)**
Problem: GenericParser fallbacks didn't include Shopify patterns.
L'Incroyable uses Shopify structure with /products/ URLs.

Solution: Added Shopify-specific fallbacks
- a[href*='/products/'] (Shopify product URLs)
- a.product-card__link (Shopify class)
- a[href*='/collections/'] (Shopify collections)
- li[class*='product'] a (list-based layouts)

**Fallback Order (now 17 selectors):**
1. Product URL patterns (Shopify, generic)
2. Semantic classes (article, .product-card)
3. Shopify-specific selectors
4. Generic structure patterns
5. Last resort wildcards

**Expected Impact:**
- ✅ Fnac images should now display
- ✅ L'Incroyable should find products (Shopify)
- ✅ Better coverage of Shopify-based stores
 2025-11-30 16:25:30 +00:00
Claude ff2615ff81 feat: Add fallback selectors to GenericParser for better site coverage 
Improves GenericParser to automatically try common e-commerce selectors
when the configured selector doesn't match any products.

**Problem:**
- Auchan and other sites return 0 products because configured selector
  doesn't match their current HTML structure
- GenericParser was too rigid: if config selector fails, give up

**Solution:**
Automatic fallback selector cascade:

1. Try configured selector first (as before)
2. If no results, try 11 common e-commerce patterns:
   - Product URL patterns: /produit, /product, /p/, /item
   - Class patterns: article a, .product a, .product-card a
   - Generic patterns: [class*='product'] a
3. Accept first selector that finds >= 3 links
4. Log which selector worked for debugging

**Selector Priority:**
- Most specific first (URL patterns)
- Medium specificity (semantic classes)
- Generic last resort (any product-related class)

**Benefits:**
- ✅ Works on more sites without manual config updates
- ✅ Resilient to site HTML changes
- ✅ Clear logging shows which selector worked
- ✅ Still uses configured selector when available
- ✅ Requires >= 3 matches to avoid false positives

**Expected Impact:**
Sites like Auchan should now find products even if configured
selector is outdated, without needing a specialized parser.
 2025-11-30 16:20:31 +00:00
Claude aa9e8dc427 fix: Handle punctuation differences in site domain matching 
Critical fix for E.Leclerc and similar sites where DB domain differs
from config key by punctuation (. vs -).

**Problem:**
- DB contains: "e.leclerc" (with dot)
- Config key: "e-leclerc.com" (with dash)
- Previous matching failed: no match found

**Solution:**
Enhanced matching with 5-level strategy:

1. **Exact match** - as before
2. **Normalized punctuation match** (NEW)
   - Remove all `-` and `.` from both sides
   - Compare: "e.leclerc" → "eleclerc", "e-leclerc.com" → "eleclecrcom"
   - Use contains logic: "eleclerc" in "eleclecrcom" → ✅ MATCH
3. **Contains match** - with original punctuation
4. **Prefix removal** - now handles both "e-" and "e." (plus "la-", "la.")
5. **Fuzzy core match** - last resort for edge cases

**Impact:**
- ✅ E.Leclerc now matches correctly
- ✅ La Foir'Fouille handles variations
- ✅ Any site with punctuation differences works
- ✅ Backward compatible with existing matches

**Debug logging:**
All strategies log which method matched, making issues easy to diagnose.
 2025-11-30 16:19:06 +00:00