Michael/Priceflow
1
0 Fork 0
mirror of https://github.com/R0m1k3/Priceflow.git synced 2026-10-11 17:29:14 +02:00
Code Issues Packages Projects Releases Wiki Activity
221 Commits 10 Branches 0 Tags
654145d49926e5db13f4d8c42efef04f736bc480
Commit Graph
221 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Michael 654145d499 Merge origin/main into antigravity: Resolve conflict by accepting deletion of direct_search_service.py 2025-11-29 08:57:48 +01:00
Michael 6019a80de9 feat: Add core search configuration for site definitions, proxies, and user agents, and introduce a browserless service. 2025-11-29 08:54:05 +01:00
Michael b6c1fc1cda feat: Add AI-powered data extraction and direct search services 2025-11-29 02:53:35 +01:00
LogiFlow a78d24fe5a Merge pull request #86 from R0m1k3/antigravity 
Merge pull request #85 from R0m1k3/main
 2025-11-29 02:44:38 +01:00
LogiFlow fa463ec50a Merge pull request #85 from R0m1k3/main 
Merge pull request #84 from R0m1k3/antigravity
 2025-11-29 02:44:18 +01:00
LogiFlow be5b64ce0f Merge pull request #84 from R0m1k3/antigravity 
Merge pull request #83 from R0m1k3/main
 2025-11-29 02:43:57 +01:00
LogiFlow daaad442e5 Merge pull request #83 from R0m1k3/main 
commit
 2025-11-29 02:40:36 +01:00
LogiFlow 33b4a43cc1 Merge pull request #82 from R0m1k3/antigravity 
feat: Add direct e-commerce search service using Playwright/Browserle…
 2025-11-29 02:38:40 +01:00
Michael a6662e8891 feat: Add direct e-commerce search service using Playwright/Browserless with anti-bot measures and Amazon-specific handling. 2025-11-29 02:38:18 +01:00
LogiFlow a6f195c491 Merge pull request #81 from R0m1k3/antigravity 
feat: Implement direct e-commerce search service using Playwright and…
 2025-11-29 02:31:52 +01:00
Michael 352fc72bfa feat: Implement direct e-commerce search service using Playwright and Browserless with anti-bot and proxy rotation capabilities. 2025-11-29 02:31:30 +01:00
LogiFlow 2add7eadbe Merge pull request #80 from R0m1k3/antigravity 
feat: add direct e-commerce search service using Playwright with anti…
 2025-11-29 02:24:45 +01:00
Michael fa890f67dc feat: add direct e-commerce search service using Playwright with anti-bot measures for scraping. 2025-11-29 02:24:20 +01:00
LogiFlow 3cf1c57466 Merge pull request #79 from R0m1k3/antigravity 
Antigravity
 2025-11-29 02:17:45 +01:00
Michael 3cb0b202d3 Merge branch 'antigravity' of https://github.com/R0m1k3/Priceflow into antigravity 2025-11-29 02:17:20 +01:00
Michael bad2cfca20 feat: Add direct e-commerce search service using Playwright with anti-bot measures including rotating proxies, user agents, and stealth JavaScript. 2025-11-29 02:17:17 +01:00
LogiFlow 6cb145626f Merge pull request #78 from R0m1k3/main 
commit
 2025-11-29 02:09:11 +01:00
Michael 87165a8b9b feat: Implement direct search service with Playwright/Browserless, including anti-bot measures, proxy rotation, and stealth JavaScript. 2025-11-29 02:07:59 +01:00
LogiFlow 326272ae56 Merge pull request #77 from R0m1k3/antigravity 
feat: implement AI service for data extraction with dynamic configura…
 2025-11-29 02:05:07 +01:00
Michael 394fa3c118 feat: implement AI service for data extraction with dynamic configuration and robust response handling. 2025-11-29 02:00:42 +01:00
LogiFlow a4c0431443 Merge pull request #76 from R0m1k3/antigravity 
Antigravity
 2025-11-29 01:56:44 +01:00
Michael 091caad347 feat: add web scraping and search services using Playwright with anti-detection mechanisms. 2025-11-29 01:56:25 +01:00
Michael 7a73650264 feat: add DirectSearchService for robust e-commerce product search using Playwright/Browserless with advanced bot detection evasion. 2025-11-29 01:52:31 +01:00
Michael e6bf46659d refactor: Improve browser lifecycle management in ScraperService to prevent connection errors 2025-11-29 01:39:43 +01:00
Michael 0439b74bc3 fix: Remove duplicate 'finally' block in scraper_service.py causing SyntaxError 2025-11-29 01:24:34 +01:00
Michael 405756963a fix: Resolve IndentationError in scraper_service.py caused by loop removal 2025-11-29 01:20:58 +01:00
Michael aa9d2af5b2 feat: Implement hybrid proxy strategy for Amazon scraping - fallback to Direct IP on last retry 2025-11-29 01:17:03 +01:00
Michael dd558b0e43 fix: Correct proxy usage in scraper_service.py to handle dict format 2025-11-29 01:07:02 +01:00
Michael 4c410ef48f fix: Correct Playwright proxy format - use dict with server/username/password instead of URL string 2025-11-29 01:02:41 +01:00
Michael df2191cdbd debug: Add proxy initialization logs to verify proxy loading 2025-11-29 00:53:55 +01:00
Michael 8080a97e25 feat: Add same 10 rotating proxies to scraper_service for individual product pages 2025-11-29 00:50:08 +01:00
Michael b7e373d9eb feat: Add 10 rotating proxies for Amazon to bypass IP bans 2025-11-29 00:47:17 +01:00
Michael 26df52b664 revert: Use Amazon config from commit 698dbaa (15 commits back) 2025-11-29 00:40:27 +01:00
Michael 3c4094d4de revert: Back to working Amazon configuration from ee8a146 - simpler is better 2025-11-29 00:39:25 +01:00
Michael c88f52c136 fix: Add ref=nb_sb_noss parameter to Amazon URLs as in working version + increase retry delays 2025-11-29 00:34:09 +01:00
Michael eb48c4aa33 feat: Add direct search service for e-commerce scraping via Browserless with bot evasion. 2025-11-29 00:30:12 +01:00
Michael 34dc9f6f40 fix: Apply working Amazon anti-blocking strategy from scraper_service - warmup + no image blocking 2025-11-29 00:22:52 +01:00
Michael 46442d5e0c Merge: resolved conflicts in direct_search_service.py 2025-11-29 00:12:28 +01:00
Michael 69aefdbac3 Fix: Resolve merge conflict in direct_search_service.py by confirming random import 2025-11-29 00:06:51 +01:00
Michael 0d102b73d0 feat: Implement direct e-commerce search service using Playwright/Browserless with anti-bot techniques like User-Agent rotation, proxy support, and stealth JS, particularly for Amazon. 2025-11-28 23:59:50 +01:00
Michael b75ad636dd feat: add product search orchestration service with direct, light scraper, and browserless methods 2025-11-28 23:57:51 +01:00
Michael 3caea8316b feat: Add Playwright-based web scraping service with advanced stealth techniques and proxy support, including Amazon-specific logic. 2025-11-28 23:53:45 +01:00
Michael f187e6cb9d feat: Implement direct e-commerce search service with advanced anti-bot stealth using Playwright/Browserless. 2025-11-24 13:10:40 +01:00
Michael ba28f0b711 feat: implement direct e-commerce search service with site-specific configurations using Playwright. 2025-11-24 13:05:21 +01:00
Michael ee8a14668c feat: Implement direct e-commerce search service using Playwright/Browserless to scrape product information from various configured retail websites. 2025-11-24 13:04:03 +01:00
Michael 97039484fa feat: add direct search service for e-commerce sites using Playwright/Browserless with predefined site configurations. 2025-11-24 13:03:47 +01:00
Michael 8053955097 feat: add direct e-commerce search service using Playwright/Browserless with predefined site configurations 2025-11-24 13:01:47 +01:00
LogiFlow 4ed574e814 Merge pull request #74 from R0m1k3/claude/fix-amazon-search-error-01Gnft8Nyjh9KkpKuavHaaPc 
feat: Add dedicated Amazon extraction without generic filters
 2025-11-24 09:40:49 +01:00
Claude 698dbaa029 feat: Add dedicated Amazon extraction without generic filters 
- Create _find_amazon_product_links() function for Amazon-specific extraction
- Use ASIN-based deduplication for Amazon products
- Bypass all generic filters (_is_non_product_url, _is_navigation_link) for Amazon
- Multiple fallback methods for Amazon product detection:
  - data-component-type="s-product-image"
  - .s-result-item links
  - All /dp/ links in main content
- Clean Amazon URLs by removing tracking parameters
- Extract title from img alt as fallback for Amazon

This separates Amazon's extraction logic from the generic e-commerce filters
that were designed for French sites like Cdiscount, Fnac, etc.
 2025-11-24 08:39:11 +00:00
LogiFlow 9cae86d733 Merge pull request #73 from R0m1k3/claude/fix-amazon-search-error-01Gnft8Nyjh9KkpKuavHaaPc 
feat: Add proxy support for Amazon scraping
 2025-11-24 09:31:54 +01:00